W0611 06:08:18.948000 1077320 site-packages/torch/distributed/run.py:766] 
W0611 06:08:18.948000 1077320 site-packages/torch/distributed/run.py:766] *****************************************
W0611 06:08:18.948000 1077320 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
W0611 06:08:18.948000 1077320 site-packages/torch/distributed/run.py:766] *****************************************
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
Trainer._get_train_sampler replaced with custom implementation.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48
StartRecentKVCache: 8, 48

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.13it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.13it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.17it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.17it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.16it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.16it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.17it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.17it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.09it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.09it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.01it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.01it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.16it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.16it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B

Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.15it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:01<00:00,  1.15it/s]
Some weights of Qwen2_5_VLForConditionalGenerationForJanusVLN were not initialized from the model checkpoint at /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/Qwen2.5-VL-3B-Instruct and are newly initialized: ['language_feature_fusion.fusion_layers.0.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.0.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.1.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.1.0.geo_mlp.2.weight', 'language_feature_fusion.fusion_layers.2.0.geo_ln.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.0.weight', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.bias', 'language_feature_fusion.fusion_layers.2.0.geo_mlp.2.weight', 'merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight', 'vggt.aggregator.camera_token', 'vggt.aggregator.frame_blocks.0.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.proj.bias', 'vggt.aggregator.frame_blocks.0.attn.proj.weight', 'vggt.aggregator.frame_blocks.0.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.0.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.0.attn.qkv.bias', 'vggt.aggregator.frame_blocks.0.attn.qkv.weight', 'vggt.aggregator.frame_blocks.0.ls1.gamma', 'vggt.aggregator.frame_blocks.0.ls2.gamma', 'vggt.aggregator.frame_blocks.0.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.0.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.0.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.0.norm1.bias', 'vggt.aggregator.frame_blocks.0.norm1.weight', 'vggt.aggregator.frame_blocks.0.norm2.bias', 'vggt.aggregator.frame_blocks.0.norm2.weight', 'vggt.aggregator.frame_blocks.1.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.proj.bias', 'vggt.aggregator.frame_blocks.1.attn.proj.weight', 'vggt.aggregator.frame_blocks.1.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.1.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.1.attn.qkv.bias', 'vggt.aggregator.frame_blocks.1.attn.qkv.weight', 'vggt.aggregator.frame_blocks.1.ls1.gamma', 'vggt.aggregator.frame_blocks.1.ls2.gamma', 'vggt.aggregator.frame_blocks.1.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.1.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.1.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.1.norm1.bias', 'vggt.aggregator.frame_blocks.1.norm1.weight', 'vggt.aggregator.frame_blocks.1.norm2.bias', 'vggt.aggregator.frame_blocks.1.norm2.weight', 'vggt.aggregator.frame_blocks.10.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.proj.bias', 'vggt.aggregator.frame_blocks.10.attn.proj.weight', 'vggt.aggregator.frame_blocks.10.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.10.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.10.attn.qkv.bias', 'vggt.aggregator.frame_blocks.10.attn.qkv.weight', 'vggt.aggregator.frame_blocks.10.ls1.gamma', 'vggt.aggregator.frame_blocks.10.ls2.gamma', 'vggt.aggregator.frame_blocks.10.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.10.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.10.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.10.norm1.bias', 'vggt.aggregator.frame_blocks.10.norm1.weight', 'vggt.aggregator.frame_blocks.10.norm2.bias', 'vggt.aggregator.frame_blocks.10.norm2.weight', 'vggt.aggregator.frame_blocks.11.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.proj.bias', 'vggt.aggregator.frame_blocks.11.attn.proj.weight', 'vggt.aggregator.frame_blocks.11.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.11.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.11.attn.qkv.bias', 'vggt.aggregator.frame_blocks.11.attn.qkv.weight', 'vggt.aggregator.frame_blocks.11.ls1.gamma', 'vggt.aggregator.frame_blocks.11.ls2.gamma', 'vggt.aggregator.frame_blocks.11.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.11.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.11.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.11.norm1.bias', 'vggt.aggregator.frame_blocks.11.norm1.weight', 'vggt.aggregator.frame_blocks.11.norm2.bias', 'vggt.aggregator.frame_blocks.11.norm2.weight', 'vggt.aggregator.frame_blocks.12.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.proj.bias', 'vggt.aggregator.frame_blocks.12.attn.proj.weight', 'vggt.aggregator.frame_blocks.12.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.12.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.12.attn.qkv.bias', 'vggt.aggregator.frame_blocks.12.attn.qkv.weight', 'vggt.aggregator.frame_blocks.12.ls1.gamma', 'vggt.aggregator.frame_blocks.12.ls2.gamma', 'vggt.aggregator.frame_blocks.12.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.12.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.12.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.12.norm1.bias', 'vggt.aggregator.frame_blocks.12.norm1.weight', 'vggt.aggregator.frame_blocks.12.norm2.bias', 'vggt.aggregator.frame_blocks.12.norm2.weight', 'vggt.aggregator.frame_blocks.13.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.proj.bias', 'vggt.aggregator.frame_blocks.13.attn.proj.weight', 'vggt.aggregator.frame_blocks.13.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.13.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.13.attn.qkv.bias', 'vggt.aggregator.frame_blocks.13.attn.qkv.weight', 'vggt.aggregator.frame_blocks.13.ls1.gamma', 'vggt.aggregator.frame_blocks.13.ls2.gamma', 'vggt.aggregator.frame_blocks.13.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.13.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.13.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.13.norm1.bias', 'vggt.aggregator.frame_blocks.13.norm1.weight', 'vggt.aggregator.frame_blocks.13.norm2.bias', 'vggt.aggregator.frame_blocks.13.norm2.weight', 'vggt.aggregator.frame_blocks.14.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.proj.bias', 'vggt.aggregator.frame_blocks.14.attn.proj.weight', 'vggt.aggregator.frame_blocks.14.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.14.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.14.attn.qkv.bias', 'vggt.aggregator.frame_blocks.14.attn.qkv.weight', 'vggt.aggregator.frame_blocks.14.ls1.gamma', 'vggt.aggregator.frame_blocks.14.ls2.gamma', 'vggt.aggregator.frame_blocks.14.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.14.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.14.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.14.norm1.bias', 'vggt.aggregator.frame_blocks.14.norm1.weight', 'vggt.aggregator.frame_blocks.14.norm2.bias', 'vggt.aggregator.frame_blocks.14.norm2.weight', 'vggt.aggregator.frame_blocks.15.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.proj.bias', 'vggt.aggregator.frame_blocks.15.attn.proj.weight', 'vggt.aggregator.frame_blocks.15.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.15.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.15.attn.qkv.bias', 'vggt.aggregator.frame_blocks.15.attn.qkv.weight', 'vggt.aggregator.frame_blocks.15.ls1.gamma', 'vggt.aggregator.frame_blocks.15.ls2.gamma', 'vggt.aggregator.frame_blocks.15.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.15.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.15.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.15.norm1.bias', 'vggt.aggregator.frame_blocks.15.norm1.weight', 'vggt.aggregator.frame_blocks.15.norm2.bias', 'vggt.aggregator.frame_blocks.15.norm2.weight', 'vggt.aggregator.frame_blocks.16.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.proj.bias', 'vggt.aggregator.frame_blocks.16.attn.proj.weight', 'vggt.aggregator.frame_blocks.16.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.16.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.16.attn.qkv.bias', 'vggt.aggregator.frame_blocks.16.attn.qkv.weight', 'vggt.aggregator.frame_blocks.16.ls1.gamma', 'vggt.aggregator.frame_blocks.16.ls2.gamma', 'vggt.aggregator.frame_blocks.16.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.16.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.16.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.16.norm1.bias', 'vggt.aggregator.frame_blocks.16.norm1.weight', 'vggt.aggregator.frame_blocks.16.norm2.bias', 'vggt.aggregator.frame_blocks.16.norm2.weight', 'vggt.aggregator.frame_blocks.17.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.proj.bias', 'vggt.aggregator.frame_blocks.17.attn.proj.weight', 'vggt.aggregator.frame_blocks.17.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.17.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.17.attn.qkv.bias', 'vggt.aggregator.frame_blocks.17.attn.qkv.weight', 'vggt.aggregator.frame_blocks.17.ls1.gamma', 'vggt.aggregator.frame_blocks.17.ls2.gamma', 'vggt.aggregator.frame_blocks.17.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.17.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.17.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.17.norm1.bias', 'vggt.aggregator.frame_blocks.17.norm1.weight', 'vggt.aggregator.frame_blocks.17.norm2.bias', 'vggt.aggregator.frame_blocks.17.norm2.weight', 'vggt.aggregator.frame_blocks.18.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.proj.bias', 'vggt.aggregator.frame_blocks.18.attn.proj.weight', 'vggt.aggregator.frame_blocks.18.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.18.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.18.attn.qkv.bias', 'vggt.aggregator.frame_blocks.18.attn.qkv.weight', 'vggt.aggregator.frame_blocks.18.ls1.gamma', 'vggt.aggregator.frame_blocks.18.ls2.gamma', 'vggt.aggregator.frame_blocks.18.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.18.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.18.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.18.norm1.bias', 'vggt.aggregator.frame_blocks.18.norm1.weight', 'vggt.aggregator.frame_blocks.18.norm2.bias', 'vggt.aggregator.frame_blocks.18.norm2.weight', 'vggt.aggregator.frame_blocks.19.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.proj.bias', 'vggt.aggregator.frame_blocks.19.attn.proj.weight', 'vggt.aggregator.frame_blocks.19.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.19.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.19.attn.qkv.bias', 'vggt.aggregator.frame_blocks.19.attn.qkv.weight', 'vggt.aggregator.frame_blocks.19.ls1.gamma', 'vggt.aggregator.frame_blocks.19.ls2.gamma', 'vggt.aggregator.frame_blocks.19.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.19.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.19.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.19.norm1.bias', 'vggt.aggregator.frame_blocks.19.norm1.weight', 'vggt.aggregator.frame_blocks.19.norm2.bias', 'vggt.aggregator.frame_blocks.19.norm2.weight', 'vggt.aggregator.frame_blocks.2.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.proj.bias', 'vggt.aggregator.frame_blocks.2.attn.proj.weight', 'vggt.aggregator.frame_blocks.2.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.2.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.2.attn.qkv.bias', 'vggt.aggregator.frame_blocks.2.attn.qkv.weight', 'vggt.aggregator.frame_blocks.2.ls1.gamma', 'vggt.aggregator.frame_blocks.2.ls2.gamma', 'vggt.aggregator.frame_blocks.2.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.2.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.2.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.2.norm1.bias', 'vggt.aggregator.frame_blocks.2.norm1.weight', 'vggt.aggregator.frame_blocks.2.norm2.bias', 'vggt.aggregator.frame_blocks.2.norm2.weight', 'vggt.aggregator.frame_blocks.20.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.proj.bias', 'vggt.aggregator.frame_blocks.20.attn.proj.weight', 'vggt.aggregator.frame_blocks.20.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.20.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.20.attn.qkv.bias', 'vggt.aggregator.frame_blocks.20.attn.qkv.weight', 'vggt.aggregator.frame_blocks.20.ls1.gamma', 'vggt.aggregator.frame_blocks.20.ls2.gamma', 'vggt.aggregator.frame_blocks.20.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.20.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.20.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.20.norm1.bias', 'vggt.aggregator.frame_blocks.20.norm1.weight', 'vggt.aggregator.frame_blocks.20.norm2.bias', 'vggt.aggregator.frame_blocks.20.norm2.weight', 'vggt.aggregator.frame_blocks.21.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.proj.bias', 'vggt.aggregator.frame_blocks.21.attn.proj.weight', 'vggt.aggregator.frame_blocks.21.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.21.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.21.attn.qkv.bias', 'vggt.aggregator.frame_blocks.21.attn.qkv.weight', 'vggt.aggregator.frame_blocks.21.ls1.gamma', 'vggt.aggregator.frame_blocks.21.ls2.gamma', 'vggt.aggregator.frame_blocks.21.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.21.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.21.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.21.norm1.bias', 'vggt.aggregator.frame_blocks.21.norm1.weight', 'vggt.aggregator.frame_blocks.21.norm2.bias', 'vggt.aggregator.frame_blocks.21.norm2.weight', 'vggt.aggregator.frame_blocks.22.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.proj.bias', 'vggt.aggregator.frame_blocks.22.attn.proj.weight', 'vggt.aggregator.frame_blocks.22.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.22.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.22.attn.qkv.bias', 'vggt.aggregator.frame_blocks.22.attn.qkv.weight', 'vggt.aggregator.frame_blocks.22.ls1.gamma', 'vggt.aggregator.frame_blocks.22.ls2.gamma', 'vggt.aggregator.frame_blocks.22.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.22.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.22.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.22.norm1.bias', 'vggt.aggregator.frame_blocks.22.norm1.weight', 'vggt.aggregator.frame_blocks.22.norm2.bias', 'vggt.aggregator.frame_blocks.22.norm2.weight', 'vggt.aggregator.frame_blocks.23.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.proj.bias', 'vggt.aggregator.frame_blocks.23.attn.proj.weight', 'vggt.aggregator.frame_blocks.23.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.23.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.23.attn.qkv.bias', 'vggt.aggregator.frame_blocks.23.attn.qkv.weight', 'vggt.aggregator.frame_blocks.23.ls1.gamma', 'vggt.aggregator.frame_blocks.23.ls2.gamma', 'vggt.aggregator.frame_blocks.23.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.23.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.23.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.23.norm1.bias', 'vggt.aggregator.frame_blocks.23.norm1.weight', 'vggt.aggregator.frame_blocks.23.norm2.bias', 'vggt.aggregator.frame_blocks.23.norm2.weight', 'vggt.aggregator.frame_blocks.3.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.proj.bias', 'vggt.aggregator.frame_blocks.3.attn.proj.weight', 'vggt.aggregator.frame_blocks.3.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.3.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.3.attn.qkv.bias', 'vggt.aggregator.frame_blocks.3.attn.qkv.weight', 'vggt.aggregator.frame_blocks.3.ls1.gamma', 'vggt.aggregator.frame_blocks.3.ls2.gamma', 'vggt.aggregator.frame_blocks.3.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.3.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.3.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.3.norm1.bias', 'vggt.aggregator.frame_blocks.3.norm1.weight', 'vggt.aggregator.frame_blocks.3.norm2.bias', 'vggt.aggregator.frame_blocks.3.norm2.weight', 'vggt.aggregator.frame_blocks.4.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.proj.bias', 'vggt.aggregator.frame_blocks.4.attn.proj.weight', 'vggt.aggregator.frame_blocks.4.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.4.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.4.attn.qkv.bias', 'vggt.aggregator.frame_blocks.4.attn.qkv.weight', 'vggt.aggregator.frame_blocks.4.ls1.gamma', 'vggt.aggregator.frame_blocks.4.ls2.gamma', 'vggt.aggregator.frame_blocks.4.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.4.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.4.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.4.norm1.bias', 'vggt.aggregator.frame_blocks.4.norm1.weight', 'vggt.aggregator.frame_blocks.4.norm2.bias', 'vggt.aggregator.frame_blocks.4.norm2.weight', 'vggt.aggregator.frame_blocks.5.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.proj.bias', 'vggt.aggregator.frame_blocks.5.attn.proj.weight', 'vggt.aggregator.frame_blocks.5.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.5.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.5.attn.qkv.bias', 'vggt.aggregator.frame_blocks.5.attn.qkv.weight', 'vggt.aggregator.frame_blocks.5.ls1.gamma', 'vggt.aggregator.frame_blocks.5.ls2.gamma', 'vggt.aggregator.frame_blocks.5.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.5.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.5.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.5.norm1.bias', 'vggt.aggregator.frame_blocks.5.norm1.weight', 'vggt.aggregator.frame_blocks.5.norm2.bias', 'vggt.aggregator.frame_blocks.5.norm2.weight', 'vggt.aggregator.frame_blocks.6.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.proj.bias', 'vggt.aggregator.frame_blocks.6.attn.proj.weight', 'vggt.aggregator.frame_blocks.6.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.6.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.6.attn.qkv.bias', 'vggt.aggregator.frame_blocks.6.attn.qkv.weight', 'vggt.aggregator.frame_blocks.6.ls1.gamma', 'vggt.aggregator.frame_blocks.6.ls2.gamma', 'vggt.aggregator.frame_blocks.6.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.6.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.6.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.6.norm1.bias', 'vggt.aggregator.frame_blocks.6.norm1.weight', 'vggt.aggregator.frame_blocks.6.norm2.bias', 'vggt.aggregator.frame_blocks.6.norm2.weight', 'vggt.aggregator.frame_blocks.7.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.proj.bias', 'vggt.aggregator.frame_blocks.7.attn.proj.weight', 'vggt.aggregator.frame_blocks.7.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.7.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.7.attn.qkv.bias', 'vggt.aggregator.frame_blocks.7.attn.qkv.weight', 'vggt.aggregator.frame_blocks.7.ls1.gamma', 'vggt.aggregator.frame_blocks.7.ls2.gamma', 'vggt.aggregator.frame_blocks.7.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.7.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.7.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.7.norm1.bias', 'vggt.aggregator.frame_blocks.7.norm1.weight', 'vggt.aggregator.frame_blocks.7.norm2.bias', 'vggt.aggregator.frame_blocks.7.norm2.weight', 'vggt.aggregator.frame_blocks.8.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.proj.bias', 'vggt.aggregator.frame_blocks.8.attn.proj.weight', 'vggt.aggregator.frame_blocks.8.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.8.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.8.attn.qkv.bias', 'vggt.aggregator.frame_blocks.8.attn.qkv.weight', 'vggt.aggregator.frame_blocks.8.ls1.gamma', 'vggt.aggregator.frame_blocks.8.ls2.gamma', 'vggt.aggregator.frame_blocks.8.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.8.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.8.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.8.norm1.bias', 'vggt.aggregator.frame_blocks.8.norm1.weight', 'vggt.aggregator.frame_blocks.8.norm2.bias', 'vggt.aggregator.frame_blocks.8.norm2.weight', 'vggt.aggregator.frame_blocks.9.attn.k_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.k_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.proj.bias', 'vggt.aggregator.frame_blocks.9.attn.proj.weight', 'vggt.aggregator.frame_blocks.9.attn.q_norm.bias', 'vggt.aggregator.frame_blocks.9.attn.q_norm.weight', 'vggt.aggregator.frame_blocks.9.attn.qkv.bias', 'vggt.aggregator.frame_blocks.9.attn.qkv.weight', 'vggt.aggregator.frame_blocks.9.ls1.gamma', 'vggt.aggregator.frame_blocks.9.ls2.gamma', 'vggt.aggregator.frame_blocks.9.mlp.fc1.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc1.weight', 'vggt.aggregator.frame_blocks.9.mlp.fc2.bias', 'vggt.aggregator.frame_blocks.9.mlp.fc2.weight', 'vggt.aggregator.frame_blocks.9.norm1.bias', 'vggt.aggregator.frame_blocks.9.norm1.weight', 'vggt.aggregator.frame_blocks.9.norm2.bias', 'vggt.aggregator.frame_blocks.9.norm2.weight', 'vggt.aggregator.global_blocks.0.attn.k_norm.bias', 'vggt.aggregator.global_blocks.0.attn.k_norm.weight', 'vggt.aggregator.global_blocks.0.attn.proj.bias', 'vggt.aggregator.global_blocks.0.attn.proj.weight', 'vggt.aggregator.global_blocks.0.attn.q_norm.bias', 'vggt.aggregator.global_blocks.0.attn.q_norm.weight', 'vggt.aggregator.global_blocks.0.attn.qkv.bias', 'vggt.aggregator.global_blocks.0.attn.qkv.weight', 'vggt.aggregator.global_blocks.0.ls1.gamma', 'vggt.aggregator.global_blocks.0.ls2.gamma', 'vggt.aggregator.global_blocks.0.mlp.fc1.bias', 'vggt.aggregator.global_blocks.0.mlp.fc1.weight', 'vggt.aggregator.global_blocks.0.mlp.fc2.bias', 'vggt.aggregator.global_blocks.0.mlp.fc2.weight', 'vggt.aggregator.global_blocks.0.norm1.bias', 'vggt.aggregator.global_blocks.0.norm1.weight', 'vggt.aggregator.global_blocks.0.norm2.bias', 'vggt.aggregator.global_blocks.0.norm2.weight', 'vggt.aggregator.global_blocks.1.attn.k_norm.bias', 'vggt.aggregator.global_blocks.1.attn.k_norm.weight', 'vggt.aggregator.global_blocks.1.attn.proj.bias', 'vggt.aggregator.global_blocks.1.attn.proj.weight', 'vggt.aggregator.global_blocks.1.attn.q_norm.bias', 'vggt.aggregator.global_blocks.1.attn.q_norm.weight', 'vggt.aggregator.global_blocks.1.attn.qkv.bias', 'vggt.aggregator.global_blocks.1.attn.qkv.weight', 'vggt.aggregator.global_blocks.1.ls1.gamma', 'vggt.aggregator.global_blocks.1.ls2.gamma', 'vggt.aggregator.global_blocks.1.mlp.fc1.bias', 'vggt.aggregator.global_blocks.1.mlp.fc1.weight', 'vggt.aggregator.global_blocks.1.mlp.fc2.bias', 'vggt.aggregator.global_blocks.1.mlp.fc2.weight', 'vggt.aggregator.global_blocks.1.norm1.bias', 'vggt.aggregator.global_blocks.1.norm1.weight', 'vggt.aggregator.global_blocks.1.norm2.bias', 'vggt.aggregator.global_blocks.1.norm2.weight', 'vggt.aggregator.global_blocks.10.attn.k_norm.bias', 'vggt.aggregator.global_blocks.10.attn.k_norm.weight', 'vggt.aggregator.global_blocks.10.attn.proj.bias', 'vggt.aggregator.global_blocks.10.attn.proj.weight', 'vggt.aggregator.global_blocks.10.attn.q_norm.bias', 'vggt.aggregator.global_blocks.10.attn.q_norm.weight', 'vggt.aggregator.global_blocks.10.attn.qkv.bias', 'vggt.aggregator.global_blocks.10.attn.qkv.weight', 'vggt.aggregator.global_blocks.10.ls1.gamma', 'vggt.aggregator.global_blocks.10.ls2.gamma', 'vggt.aggregator.global_blocks.10.mlp.fc1.bias', 'vggt.aggregator.global_blocks.10.mlp.fc1.weight', 'vggt.aggregator.global_blocks.10.mlp.fc2.bias', 'vggt.aggregator.global_blocks.10.mlp.fc2.weight', 'vggt.aggregator.global_blocks.10.norm1.bias', 'vggt.aggregator.global_blocks.10.norm1.weight', 'vggt.aggregator.global_blocks.10.norm2.bias', 'vggt.aggregator.global_blocks.10.norm2.weight', 'vggt.aggregator.global_blocks.11.attn.k_norm.bias', 'vggt.aggregator.global_blocks.11.attn.k_norm.weight', 'vggt.aggregator.global_blocks.11.attn.proj.bias', 'vggt.aggregator.global_blocks.11.attn.proj.weight', 'vggt.aggregator.global_blocks.11.attn.q_norm.bias', 'vggt.aggregator.global_blocks.11.attn.q_norm.weight', 'vggt.aggregator.global_blocks.11.attn.qkv.bias', 'vggt.aggregator.global_blocks.11.attn.qkv.weight', 'vggt.aggregator.global_blocks.11.ls1.gamma', 'vggt.aggregator.global_blocks.11.ls2.gamma', 'vggt.aggregator.global_blocks.11.mlp.fc1.bias', 'vggt.aggregator.global_blocks.11.mlp.fc1.weight', 'vggt.aggregator.global_blocks.11.mlp.fc2.bias', 'vggt.aggregator.global_blocks.11.mlp.fc2.weight', 'vggt.aggregator.global_blocks.11.norm1.bias', 'vggt.aggregator.global_blocks.11.norm1.weight', 'vggt.aggregator.global_blocks.11.norm2.bias', 'vggt.aggregator.global_blocks.11.norm2.weight', 'vggt.aggregator.global_blocks.12.attn.k_norm.bias', 'vggt.aggregator.global_blocks.12.attn.k_norm.weight', 'vggt.aggregator.global_blocks.12.attn.proj.bias', 'vggt.aggregator.global_blocks.12.attn.proj.weight', 'vggt.aggregator.global_blocks.12.attn.q_norm.bias', 'vggt.aggregator.global_blocks.12.attn.q_norm.weight', 'vggt.aggregator.global_blocks.12.attn.qkv.bias', 'vggt.aggregator.global_blocks.12.attn.qkv.weight', 'vggt.aggregator.global_blocks.12.ls1.gamma', 'vggt.aggregator.global_blocks.12.ls2.gamma', 'vggt.aggregator.global_blocks.12.mlp.fc1.bias', 'vggt.aggregator.global_blocks.12.mlp.fc1.weight', 'vggt.aggregator.global_blocks.12.mlp.fc2.bias', 'vggt.aggregator.global_blocks.12.mlp.fc2.weight', 'vggt.aggregator.global_blocks.12.norm1.bias', 'vggt.aggregator.global_blocks.12.norm1.weight', 'vggt.aggregator.global_blocks.12.norm2.bias', 'vggt.aggregator.global_blocks.12.norm2.weight', 'vggt.aggregator.global_blocks.13.attn.k_norm.bias', 'vggt.aggregator.global_blocks.13.attn.k_norm.weight', 'vggt.aggregator.global_blocks.13.attn.proj.bias', 'vggt.aggregator.global_blocks.13.attn.proj.weight', 'vggt.aggregator.global_blocks.13.attn.q_norm.bias', 'vggt.aggregator.global_blocks.13.attn.q_norm.weight', 'vggt.aggregator.global_blocks.13.attn.qkv.bias', 'vggt.aggregator.global_blocks.13.attn.qkv.weight', 'vggt.aggregator.global_blocks.13.ls1.gamma', 'vggt.aggregator.global_blocks.13.ls2.gamma', 'vggt.aggregator.global_blocks.13.mlp.fc1.bias', 'vggt.aggregator.global_blocks.13.mlp.fc1.weight', 'vggt.aggregator.global_blocks.13.mlp.fc2.bias', 'vggt.aggregator.global_blocks.13.mlp.fc2.weight', 'vggt.aggregator.global_blocks.13.norm1.bias', 'vggt.aggregator.global_blocks.13.norm1.weight', 'vggt.aggregator.global_blocks.13.norm2.bias', 'vggt.aggregator.global_blocks.13.norm2.weight', 'vggt.aggregator.global_blocks.14.attn.k_norm.bias', 'vggt.aggregator.global_blocks.14.attn.k_norm.weight', 'vggt.aggregator.global_blocks.14.attn.proj.bias', 'vggt.aggregator.global_blocks.14.attn.proj.weight', 'vggt.aggregator.global_blocks.14.attn.q_norm.bias', 'vggt.aggregator.global_blocks.14.attn.q_norm.weight', 'vggt.aggregator.global_blocks.14.attn.qkv.bias', 'vggt.aggregator.global_blocks.14.attn.qkv.weight', 'vggt.aggregator.global_blocks.14.ls1.gamma', 'vggt.aggregator.global_blocks.14.ls2.gamma', 'vggt.aggregator.global_blocks.14.mlp.fc1.bias', 'vggt.aggregator.global_blocks.14.mlp.fc1.weight', 'vggt.aggregator.global_blocks.14.mlp.fc2.bias', 'vggt.aggregator.global_blocks.14.mlp.fc2.weight', 'vggt.aggregator.global_blocks.14.norm1.bias', 'vggt.aggregator.global_blocks.14.norm1.weight', 'vggt.aggregator.global_blocks.14.norm2.bias', 'vggt.aggregator.global_blocks.14.norm2.weight', 'vggt.aggregator.global_blocks.15.attn.k_norm.bias', 'vggt.aggregator.global_blocks.15.attn.k_norm.weight', 'vggt.aggregator.global_blocks.15.attn.proj.bias', 'vggt.aggregator.global_blocks.15.attn.proj.weight', 'vggt.aggregator.global_blocks.15.attn.q_norm.bias', 'vggt.aggregator.global_blocks.15.attn.q_norm.weight', 'vggt.aggregator.global_blocks.15.attn.qkv.bias', 'vggt.aggregator.global_blocks.15.attn.qkv.weight', 'vggt.aggregator.global_blocks.15.ls1.gamma', 'vggt.aggregator.global_blocks.15.ls2.gamma', 'vggt.aggregator.global_blocks.15.mlp.fc1.bias', 'vggt.aggregator.global_blocks.15.mlp.fc1.weight', 'vggt.aggregator.global_blocks.15.mlp.fc2.bias', 'vggt.aggregator.global_blocks.15.mlp.fc2.weight', 'vggt.aggregator.global_blocks.15.norm1.bias', 'vggt.aggregator.global_blocks.15.norm1.weight', 'vggt.aggregator.global_blocks.15.norm2.bias', 'vggt.aggregator.global_blocks.15.norm2.weight', 'vggt.aggregator.global_blocks.16.attn.k_norm.bias', 'vggt.aggregator.global_blocks.16.attn.k_norm.weight', 'vggt.aggregator.global_blocks.16.attn.proj.bias', 'vggt.aggregator.global_blocks.16.attn.proj.weight', 'vggt.aggregator.global_blocks.16.attn.q_norm.bias', 'vggt.aggregator.global_blocks.16.attn.q_norm.weight', 'vggt.aggregator.global_blocks.16.attn.qkv.bias', 'vggt.aggregator.global_blocks.16.attn.qkv.weight', 'vggt.aggregator.global_blocks.16.ls1.gamma', 'vggt.aggregator.global_blocks.16.ls2.gamma', 'vggt.aggregator.global_blocks.16.mlp.fc1.bias', 'vggt.aggregator.global_blocks.16.mlp.fc1.weight', 'vggt.aggregator.global_blocks.16.mlp.fc2.bias', 'vggt.aggregator.global_blocks.16.mlp.fc2.weight', 'vggt.aggregator.global_blocks.16.norm1.bias', 'vggt.aggregator.global_blocks.16.norm1.weight', 'vggt.aggregator.global_blocks.16.norm2.bias', 'vggt.aggregator.global_blocks.16.norm2.weight', 'vggt.aggregator.global_blocks.17.attn.k_norm.bias', 'vggt.aggregator.global_blocks.17.attn.k_norm.weight', 'vggt.aggregator.global_blocks.17.attn.proj.bias', 'vggt.aggregator.global_blocks.17.attn.proj.weight', 'vggt.aggregator.global_blocks.17.attn.q_norm.bias', 'vggt.aggregator.global_blocks.17.attn.q_norm.weight', 'vggt.aggregator.global_blocks.17.attn.qkv.bias', 'vggt.aggregator.global_blocks.17.attn.qkv.weight', 'vggt.aggregator.global_blocks.17.ls1.gamma', 'vggt.aggregator.global_blocks.17.ls2.gamma', 'vggt.aggregator.global_blocks.17.mlp.fc1.bias', 'vggt.aggregator.global_blocks.17.mlp.fc1.weight', 'vggt.aggregator.global_blocks.17.mlp.fc2.bias', 'vggt.aggregator.global_blocks.17.mlp.fc2.weight', 'vggt.aggregator.global_blocks.17.norm1.bias', 'vggt.aggregator.global_blocks.17.norm1.weight', 'vggt.aggregator.global_blocks.17.norm2.bias', 'vggt.aggregator.global_blocks.17.norm2.weight', 'vggt.aggregator.global_blocks.18.attn.k_norm.bias', 'vggt.aggregator.global_blocks.18.attn.k_norm.weight', 'vggt.aggregator.global_blocks.18.attn.proj.bias', 'vggt.aggregator.global_blocks.18.attn.proj.weight', 'vggt.aggregator.global_blocks.18.attn.q_norm.bias', 'vggt.aggregator.global_blocks.18.attn.q_norm.weight', 'vggt.aggregator.global_blocks.18.attn.qkv.bias', 'vggt.aggregator.global_blocks.18.attn.qkv.weight', 'vggt.aggregator.global_blocks.18.ls1.gamma', 'vggt.aggregator.global_blocks.18.ls2.gamma', 'vggt.aggregator.global_blocks.18.mlp.fc1.bias', 'vggt.aggregator.global_blocks.18.mlp.fc1.weight', 'vggt.aggregator.global_blocks.18.mlp.fc2.bias', 'vggt.aggregator.global_blocks.18.mlp.fc2.weight', 'vggt.aggregator.global_blocks.18.norm1.bias', 'vggt.aggregator.global_blocks.18.norm1.weight', 'vggt.aggregator.global_blocks.18.norm2.bias', 'vggt.aggregator.global_blocks.18.norm2.weight', 'vggt.aggregator.global_blocks.19.attn.k_norm.bias', 'vggt.aggregator.global_blocks.19.attn.k_norm.weight', 'vggt.aggregator.global_blocks.19.attn.proj.bias', 'vggt.aggregator.global_blocks.19.attn.proj.weight', 'vggt.aggregator.global_blocks.19.attn.q_norm.bias', 'vggt.aggregator.global_blocks.19.attn.q_norm.weight', 'vggt.aggregator.global_blocks.19.attn.qkv.bias', 'vggt.aggregator.global_blocks.19.attn.qkv.weight', 'vggt.aggregator.global_blocks.19.ls1.gamma', 'vggt.aggregator.global_blocks.19.ls2.gamma', 'vggt.aggregator.global_blocks.19.mlp.fc1.bias', 'vggt.aggregator.global_blocks.19.mlp.fc1.weight', 'vggt.aggregator.global_blocks.19.mlp.fc2.bias', 'vggt.aggregator.global_blocks.19.mlp.fc2.weight', 'vggt.aggregator.global_blocks.19.norm1.bias', 'vggt.aggregator.global_blocks.19.norm1.weight', 'vggt.aggregator.global_blocks.19.norm2.bias', 'vggt.aggregator.global_blocks.19.norm2.weight', 'vggt.aggregator.global_blocks.2.attn.k_norm.bias', 'vggt.aggregator.global_blocks.2.attn.k_norm.weight', 'vggt.aggregator.global_blocks.2.attn.proj.bias', 'vggt.aggregator.global_blocks.2.attn.proj.weight', 'vggt.aggregator.global_blocks.2.attn.q_norm.bias', 'vggt.aggregator.global_blocks.2.attn.q_norm.weight', 'vggt.aggregator.global_blocks.2.attn.qkv.bias', 'vggt.aggregator.global_blocks.2.attn.qkv.weight', 'vggt.aggregator.global_blocks.2.ls1.gamma', 'vggt.aggregator.global_blocks.2.ls2.gamma', 'vggt.aggregator.global_blocks.2.mlp.fc1.bias', 'vggt.aggregator.global_blocks.2.mlp.fc1.weight', 'vggt.aggregator.global_blocks.2.mlp.fc2.bias', 'vggt.aggregator.global_blocks.2.mlp.fc2.weight', 'vggt.aggregator.global_blocks.2.norm1.bias', 'vggt.aggregator.global_blocks.2.norm1.weight', 'vggt.aggregator.global_blocks.2.norm2.bias', 'vggt.aggregator.global_blocks.2.norm2.weight', 'vggt.aggregator.global_blocks.20.attn.k_norm.bias', 'vggt.aggregator.global_blocks.20.attn.k_norm.weight', 'vggt.aggregator.global_blocks.20.attn.proj.bias', 'vggt.aggregator.global_blocks.20.attn.proj.weight', 'vggt.aggregator.global_blocks.20.attn.q_norm.bias', 'vggt.aggregator.global_blocks.20.attn.q_norm.weight', 'vggt.aggregator.global_blocks.20.attn.qkv.bias', 'vggt.aggregator.global_blocks.20.attn.qkv.weight', 'vggt.aggregator.global_blocks.20.ls1.gamma', 'vggt.aggregator.global_blocks.20.ls2.gamma', 'vggt.aggregator.global_blocks.20.mlp.fc1.bias', 'vggt.aggregator.global_blocks.20.mlp.fc1.weight', 'vggt.aggregator.global_blocks.20.mlp.fc2.bias', 'vggt.aggregator.global_blocks.20.mlp.fc2.weight', 'vggt.aggregator.global_blocks.20.norm1.bias', 'vggt.aggregator.global_blocks.20.norm1.weight', 'vggt.aggregator.global_blocks.20.norm2.bias', 'vggt.aggregator.global_blocks.20.norm2.weight', 'vggt.aggregator.global_blocks.21.attn.k_norm.bias', 'vggt.aggregator.global_blocks.21.attn.k_norm.weight', 'vggt.aggregator.global_blocks.21.attn.proj.bias', 'vggt.aggregator.global_blocks.21.attn.proj.weight', 'vggt.aggregator.global_blocks.21.attn.q_norm.bias', 'vggt.aggregator.global_blocks.21.attn.q_norm.weight', 'vggt.aggregator.global_blocks.21.attn.qkv.bias', 'vggt.aggregator.global_blocks.21.attn.qkv.weight', 'vggt.aggregator.global_blocks.21.ls1.gamma', 'vggt.aggregator.global_blocks.21.ls2.gamma', 'vggt.aggregator.global_blocks.21.mlp.fc1.bias', 'vggt.aggregator.global_blocks.21.mlp.fc1.weight', 'vggt.aggregator.global_blocks.21.mlp.fc2.bias', 'vggt.aggregator.global_blocks.21.mlp.fc2.weight', 'vggt.aggregator.global_blocks.21.norm1.bias', 'vggt.aggregator.global_blocks.21.norm1.weight', 'vggt.aggregator.global_blocks.21.norm2.bias', 'vggt.aggregator.global_blocks.21.norm2.weight', 'vggt.aggregator.global_blocks.22.attn.k_norm.bias', 'vggt.aggregator.global_blocks.22.attn.k_norm.weight', 'vggt.aggregator.global_blocks.22.attn.proj.bias', 'vggt.aggregator.global_blocks.22.attn.proj.weight', 'vggt.aggregator.global_blocks.22.attn.q_norm.bias', 'vggt.aggregator.global_blocks.22.attn.q_norm.weight', 'vggt.aggregator.global_blocks.22.attn.qkv.bias', 'vggt.aggregator.global_blocks.22.attn.qkv.weight', 'vggt.aggregator.global_blocks.22.ls1.gamma', 'vggt.aggregator.global_blocks.22.ls2.gamma', 'vggt.aggregator.global_blocks.22.mlp.fc1.bias', 'vggt.aggregator.global_blocks.22.mlp.fc1.weight', 'vggt.aggregator.global_blocks.22.mlp.fc2.bias', 'vggt.aggregator.global_blocks.22.mlp.fc2.weight', 'vggt.aggregator.global_blocks.22.norm1.bias', 'vggt.aggregator.global_blocks.22.norm1.weight', 'vggt.aggregator.global_blocks.22.norm2.bias', 'vggt.aggregator.global_blocks.22.norm2.weight', 'vggt.aggregator.global_blocks.23.attn.k_norm.bias', 'vggt.aggregator.global_blocks.23.attn.k_norm.weight', 'vggt.aggregator.global_blocks.23.attn.proj.bias', 'vggt.aggregator.global_blocks.23.attn.proj.weight', 'vggt.aggregator.global_blocks.23.attn.q_norm.bias', 'vggt.aggregator.global_blocks.23.attn.q_norm.weight', 'vggt.aggregator.global_blocks.23.attn.qkv.bias', 'vggt.aggregator.global_blocks.23.attn.qkv.weight', 'vggt.aggregator.global_blocks.23.ls1.gamma', 'vggt.aggregator.global_blocks.23.ls2.gamma', 'vggt.aggregator.global_blocks.23.mlp.fc1.bias', 'vggt.aggregator.global_blocks.23.mlp.fc1.weight', 'vggt.aggregator.global_blocks.23.mlp.fc2.bias', 'vggt.aggregator.global_blocks.23.mlp.fc2.weight', 'vggt.aggregator.global_blocks.23.norm1.bias', 'vggt.aggregator.global_blocks.23.norm1.weight', 'vggt.aggregator.global_blocks.23.norm2.bias', 'vggt.aggregator.global_blocks.23.norm2.weight', 'vggt.aggregator.global_blocks.3.attn.k_norm.bias', 'vggt.aggregator.global_blocks.3.attn.k_norm.weight', 'vggt.aggregator.global_blocks.3.attn.proj.bias', 'vggt.aggregator.global_blocks.3.attn.proj.weight', 'vggt.aggregator.global_blocks.3.attn.q_norm.bias', 'vggt.aggregator.global_blocks.3.attn.q_norm.weight', 'vggt.aggregator.global_blocks.3.attn.qkv.bias', 'vggt.aggregator.global_blocks.3.attn.qkv.weight', 'vggt.aggregator.global_blocks.3.ls1.gamma', 'vggt.aggregator.global_blocks.3.ls2.gamma', 'vggt.aggregator.global_blocks.3.mlp.fc1.bias', 'vggt.aggregator.global_blocks.3.mlp.fc1.weight', 'vggt.aggregator.global_blocks.3.mlp.fc2.bias', 'vggt.aggregator.global_blocks.3.mlp.fc2.weight', 'vggt.aggregator.global_blocks.3.norm1.bias', 'vggt.aggregator.global_blocks.3.norm1.weight', 'vggt.aggregator.global_blocks.3.norm2.bias', 'vggt.aggregator.global_blocks.3.norm2.weight', 'vggt.aggregator.global_blocks.4.attn.k_norm.bias', 'vggt.aggregator.global_blocks.4.attn.k_norm.weight', 'vggt.aggregator.global_blocks.4.attn.proj.bias', 'vggt.aggregator.global_blocks.4.attn.proj.weight', 'vggt.aggregator.global_blocks.4.attn.q_norm.bias', 'vggt.aggregator.global_blocks.4.attn.q_norm.weight', 'vggt.aggregator.global_blocks.4.attn.qkv.bias', 'vggt.aggregator.global_blocks.4.attn.qkv.weight', 'vggt.aggregator.global_blocks.4.ls1.gamma', 'vggt.aggregator.global_blocks.4.ls2.gamma', 'vggt.aggregator.global_blocks.4.mlp.fc1.bias', 'vggt.aggregator.global_blocks.4.mlp.fc1.weight', 'vggt.aggregator.global_blocks.4.mlp.fc2.bias', 'vggt.aggregator.global_blocks.4.mlp.fc2.weight', 'vggt.aggregator.global_blocks.4.norm1.bias', 'vggt.aggregator.global_blocks.4.norm1.weight', 'vggt.aggregator.global_blocks.4.norm2.bias', 'vggt.aggregator.global_blocks.4.norm2.weight', 'vggt.aggregator.global_blocks.5.attn.k_norm.bias', 'vggt.aggregator.global_blocks.5.attn.k_norm.weight', 'vggt.aggregator.global_blocks.5.attn.proj.bias', 'vggt.aggregator.global_blocks.5.attn.proj.weight', 'vggt.aggregator.global_blocks.5.attn.q_norm.bias', 'vggt.aggregator.global_blocks.5.attn.q_norm.weight', 'vggt.aggregator.global_blocks.5.attn.qkv.bias', 'vggt.aggregator.global_blocks.5.attn.qkv.weight', 'vggt.aggregator.global_blocks.5.ls1.gamma', 'vggt.aggregator.global_blocks.5.ls2.gamma', 'vggt.aggregator.global_blocks.5.mlp.fc1.bias', 'vggt.aggregator.global_blocks.5.mlp.fc1.weight', 'vggt.aggregator.global_blocks.5.mlp.fc2.bias', 'vggt.aggregator.global_blocks.5.mlp.fc2.weight', 'vggt.aggregator.global_blocks.5.norm1.bias', 'vggt.aggregator.global_blocks.5.norm1.weight', 'vggt.aggregator.global_blocks.5.norm2.bias', 'vggt.aggregator.global_blocks.5.norm2.weight', 'vggt.aggregator.global_blocks.6.attn.k_norm.bias', 'vggt.aggregator.global_blocks.6.attn.k_norm.weight', 'vggt.aggregator.global_blocks.6.attn.proj.bias', 'vggt.aggregator.global_blocks.6.attn.proj.weight', 'vggt.aggregator.global_blocks.6.attn.q_norm.bias', 'vggt.aggregator.global_blocks.6.attn.q_norm.weight', 'vggt.aggregator.global_blocks.6.attn.qkv.bias', 'vggt.aggregator.global_blocks.6.attn.qkv.weight', 'vggt.aggregator.global_blocks.6.ls1.gamma', 'vggt.aggregator.global_blocks.6.ls2.gamma', 'vggt.aggregator.global_blocks.6.mlp.fc1.bias', 'vggt.aggregator.global_blocks.6.mlp.fc1.weight', 'vggt.aggregator.global_blocks.6.mlp.fc2.bias', 'vggt.aggregator.global_blocks.6.mlp.fc2.weight', 'vggt.aggregator.global_blocks.6.norm1.bias', 'vggt.aggregator.global_blocks.6.norm1.weight', 'vggt.aggregator.global_blocks.6.norm2.bias', 'vggt.aggregator.global_blocks.6.norm2.weight', 'vggt.aggregator.global_blocks.7.attn.k_norm.bias', 'vggt.aggregator.global_blocks.7.attn.k_norm.weight', 'vggt.aggregator.global_blocks.7.attn.proj.bias', 'vggt.aggregator.global_blocks.7.attn.proj.weight', 'vggt.aggregator.global_blocks.7.attn.q_norm.bias', 'vggt.aggregator.global_blocks.7.attn.q_norm.weight', 'vggt.aggregator.global_blocks.7.attn.qkv.bias', 'vggt.aggregator.global_blocks.7.attn.qkv.weight', 'vggt.aggregator.global_blocks.7.ls1.gamma', 'vggt.aggregator.global_blocks.7.ls2.gamma', 'vggt.aggregator.global_blocks.7.mlp.fc1.bias', 'vggt.aggregator.global_blocks.7.mlp.fc1.weight', 'vggt.aggregator.global_blocks.7.mlp.fc2.bias', 'vggt.aggregator.global_blocks.7.mlp.fc2.weight', 'vggt.aggregator.global_blocks.7.norm1.bias', 'vggt.aggregator.global_blocks.7.norm1.weight', 'vggt.aggregator.global_blocks.7.norm2.bias', 'vggt.aggregator.global_blocks.7.norm2.weight', 'vggt.aggregator.global_blocks.8.attn.k_norm.bias', 'vggt.aggregator.global_blocks.8.attn.k_norm.weight', 'vggt.aggregator.global_blocks.8.attn.proj.bias', 'vggt.aggregator.global_blocks.8.attn.proj.weight', 'vggt.aggregator.global_blocks.8.attn.q_norm.bias', 'vggt.aggregator.global_blocks.8.attn.q_norm.weight', 'vggt.aggregator.global_blocks.8.attn.qkv.bias', 'vggt.aggregator.global_blocks.8.attn.qkv.weight', 'vggt.aggregator.global_blocks.8.ls1.gamma', 'vggt.aggregator.global_blocks.8.ls2.gamma', 'vggt.aggregator.global_blocks.8.mlp.fc1.bias', 'vggt.aggregator.global_blocks.8.mlp.fc1.weight', 'vggt.aggregator.global_blocks.8.mlp.fc2.bias', 'vggt.aggregator.global_blocks.8.mlp.fc2.weight', 'vggt.aggregator.global_blocks.8.norm1.bias', 'vggt.aggregator.global_blocks.8.norm1.weight', 'vggt.aggregator.global_blocks.8.norm2.bias', 'vggt.aggregator.global_blocks.8.norm2.weight', 'vggt.aggregator.global_blocks.9.attn.k_norm.bias', 'vggt.aggregator.global_blocks.9.attn.k_norm.weight', 'vggt.aggregator.global_blocks.9.attn.proj.bias', 'vggt.aggregator.global_blocks.9.attn.proj.weight', 'vggt.aggregator.global_blocks.9.attn.q_norm.bias', 'vggt.aggregator.global_blocks.9.attn.q_norm.weight', 'vggt.aggregator.global_blocks.9.attn.qkv.bias', 'vggt.aggregator.global_blocks.9.attn.qkv.weight', 'vggt.aggregator.global_blocks.9.ls1.gamma', 'vggt.aggregator.global_blocks.9.ls2.gamma', 'vggt.aggregator.global_blocks.9.mlp.fc1.bias', 'vggt.aggregator.global_blocks.9.mlp.fc1.weight', 'vggt.aggregator.global_blocks.9.mlp.fc2.bias', 'vggt.aggregator.global_blocks.9.mlp.fc2.weight', 'vggt.aggregator.global_blocks.9.norm1.bias', 'vggt.aggregator.global_blocks.9.norm1.weight', 'vggt.aggregator.global_blocks.9.norm2.bias', 'vggt.aggregator.global_blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.0.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.0.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.0.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.0.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.0.norm1.bias', 'vggt.aggregator.patch_embed.blocks.0.norm1.weight', 'vggt.aggregator.patch_embed.blocks.0.norm2.bias', 'vggt.aggregator.patch_embed.blocks.0.norm2.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.1.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.1.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.1.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.1.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.1.norm1.bias', 'vggt.aggregator.patch_embed.blocks.1.norm1.weight', 'vggt.aggregator.patch_embed.blocks.1.norm2.bias', 'vggt.aggregator.patch_embed.blocks.1.norm2.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.10.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.10.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.10.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.10.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.10.norm1.bias', 'vggt.aggregator.patch_embed.blocks.10.norm1.weight', 'vggt.aggregator.patch_embed.blocks.10.norm2.bias', 'vggt.aggregator.patch_embed.blocks.10.norm2.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.11.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.11.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.11.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.11.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.11.norm1.bias', 'vggt.aggregator.patch_embed.blocks.11.norm1.weight', 'vggt.aggregator.patch_embed.blocks.11.norm2.bias', 'vggt.aggregator.patch_embed.blocks.11.norm2.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.12.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.12.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.12.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.12.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.12.norm1.bias', 'vggt.aggregator.patch_embed.blocks.12.norm1.weight', 'vggt.aggregator.patch_embed.blocks.12.norm2.bias', 'vggt.aggregator.patch_embed.blocks.12.norm2.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.13.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.13.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.13.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.13.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.13.norm1.bias', 'vggt.aggregator.patch_embed.blocks.13.norm1.weight', 'vggt.aggregator.patch_embed.blocks.13.norm2.bias', 'vggt.aggregator.patch_embed.blocks.13.norm2.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.14.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.14.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.14.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.14.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.14.norm1.bias', 'vggt.aggregator.patch_embed.blocks.14.norm1.weight', 'vggt.aggregator.patch_embed.blocks.14.norm2.bias', 'vggt.aggregator.patch_embed.blocks.14.norm2.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.15.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.15.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.15.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.15.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.15.norm1.bias', 'vggt.aggregator.patch_embed.blocks.15.norm1.weight', 'vggt.aggregator.patch_embed.blocks.15.norm2.bias', 'vggt.aggregator.patch_embed.blocks.15.norm2.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.16.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.16.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.16.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.16.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.16.norm1.bias', 'vggt.aggregator.patch_embed.blocks.16.norm1.weight', 'vggt.aggregator.patch_embed.blocks.16.norm2.bias', 'vggt.aggregator.patch_embed.blocks.16.norm2.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.17.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.17.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.17.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.17.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.17.norm1.bias', 'vggt.aggregator.patch_embed.blocks.17.norm1.weight', 'vggt.aggregator.patch_embed.blocks.17.norm2.bias', 'vggt.aggregator.patch_embed.blocks.17.norm2.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.18.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.18.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.18.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.18.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.18.norm1.bias', 'vggt.aggregator.patch_embed.blocks.18.norm1.weight', 'vggt.aggregator.patch_embed.blocks.18.norm2.bias', 'vggt.aggregator.patch_embed.blocks.18.norm2.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.19.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.19.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.19.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.19.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.19.norm1.bias', 'vggt.aggregator.patch_embed.blocks.19.norm1.weight', 'vggt.aggregator.patch_embed.blocks.19.norm2.bias', 'vggt.aggregator.patch_embed.blocks.19.norm2.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.2.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.2.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.2.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.2.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.2.norm1.bias', 'vggt.aggregator.patch_embed.blocks.2.norm1.weight', 'vggt.aggregator.patch_embed.blocks.2.norm2.bias', 'vggt.aggregator.patch_embed.blocks.2.norm2.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.20.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.20.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.20.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.20.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.20.norm1.bias', 'vggt.aggregator.patch_embed.blocks.20.norm1.weight', 'vggt.aggregator.patch_embed.blocks.20.norm2.bias', 'vggt.aggregator.patch_embed.blocks.20.norm2.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.21.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.21.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.21.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.21.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.21.norm1.bias', 'vggt.aggregator.patch_embed.blocks.21.norm1.weight', 'vggt.aggregator.patch_embed.blocks.21.norm2.bias', 'vggt.aggregator.patch_embed.blocks.21.norm2.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.22.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.22.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.22.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.22.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.22.norm1.bias', 'vggt.aggregator.patch_embed.blocks.22.norm1.weight', 'vggt.aggregator.patch_embed.blocks.22.norm2.bias', 'vggt.aggregator.patch_embed.blocks.22.norm2.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.23.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.23.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.23.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.23.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.23.norm1.bias', 'vggt.aggregator.patch_embed.blocks.23.norm1.weight', 'vggt.aggregator.patch_embed.blocks.23.norm2.bias', 'vggt.aggregator.patch_embed.blocks.23.norm2.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.3.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.3.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.3.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.3.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.3.norm1.bias', 'vggt.aggregator.patch_embed.blocks.3.norm1.weight', 'vggt.aggregator.patch_embed.blocks.3.norm2.bias', 'vggt.aggregator.patch_embed.blocks.3.norm2.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.4.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.4.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.4.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.4.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.4.norm1.bias', 'vggt.aggregator.patch_embed.blocks.4.norm1.weight', 'vggt.aggregator.patch_embed.blocks.4.norm2.bias', 'vggt.aggregator.patch_embed.blocks.4.norm2.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.5.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.5.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.5.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.5.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.5.norm1.bias', 'vggt.aggregator.patch_embed.blocks.5.norm1.weight', 'vggt.aggregator.patch_embed.blocks.5.norm2.bias', 'vggt.aggregator.patch_embed.blocks.5.norm2.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.6.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.6.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.6.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.6.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.6.norm1.bias', 'vggt.aggregator.patch_embed.blocks.6.norm1.weight', 'vggt.aggregator.patch_embed.blocks.6.norm2.bias', 'vggt.aggregator.patch_embed.blocks.6.norm2.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.7.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.7.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.7.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.7.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.7.norm1.bias', 'vggt.aggregator.patch_embed.blocks.7.norm1.weight', 'vggt.aggregator.patch_embed.blocks.7.norm2.bias', 'vggt.aggregator.patch_embed.blocks.7.norm2.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.8.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.8.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.8.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.8.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.8.norm1.bias', 'vggt.aggregator.patch_embed.blocks.8.norm1.weight', 'vggt.aggregator.patch_embed.blocks.8.norm2.bias', 'vggt.aggregator.patch_embed.blocks.8.norm2.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.proj.weight', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.bias', 'vggt.aggregator.patch_embed.blocks.9.attn.qkv.weight', 'vggt.aggregator.patch_embed.blocks.9.ls1.gamma', 'vggt.aggregator.patch_embed.blocks.9.ls2.gamma', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc1.weight', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.bias', 'vggt.aggregator.patch_embed.blocks.9.mlp.fc2.weight', 'vggt.aggregator.patch_embed.blocks.9.norm1.bias', 'vggt.aggregator.patch_embed.blocks.9.norm1.weight', 'vggt.aggregator.patch_embed.blocks.9.norm2.bias', 'vggt.aggregator.patch_embed.blocks.9.norm2.weight', 'vggt.aggregator.patch_embed.cls_token', 'vggt.aggregator.patch_embed.mask_token', 'vggt.aggregator.patch_embed.norm.bias', 'vggt.aggregator.patch_embed.norm.weight', 'vggt.aggregator.patch_embed.patch_embed.proj.bias', 'vggt.aggregator.patch_embed.patch_embed.proj.weight', 'vggt.aggregator.patch_embed.pos_embed', 'vggt.aggregator.patch_embed.register_tokens', 'vggt.aggregator.register_token', 'vggt.depth_head.norm.bias', 'vggt.depth_head.norm.weight', 'vggt.depth_head.projects.0.bias', 'vggt.depth_head.projects.0.weight', 'vggt.depth_head.projects.1.bias', 'vggt.depth_head.projects.1.weight', 'vggt.depth_head.projects.2.bias', 'vggt.depth_head.projects.2.weight', 'vggt.depth_head.projects.3.bias', 'vggt.depth_head.projects.3.weight', 'vggt.depth_head.resize_layers.0.bias', 'vggt.depth_head.resize_layers.0.weight', 'vggt.depth_head.resize_layers.1.bias', 'vggt.depth_head.resize_layers.1.weight', 'vggt.depth_head.resize_layers.3.bias', 'vggt.depth_head.resize_layers.3.weight', 'vggt.depth_head.scratch.layer1_rn.weight', 'vggt.depth_head.scratch.layer2_rn.weight', 'vggt.depth_head.scratch.layer3_rn.weight', 'vggt.depth_head.scratch.layer4_rn.weight', 'vggt.depth_head.scratch.output_conv1.bias', 'vggt.depth_head.scratch.output_conv1.weight', 'vggt.depth_head.scratch.output_conv2.0.bias', 'vggt.depth_head.scratch.output_conv2.0.weight', 'vggt.depth_head.scratch.output_conv2.2.bias', 'vggt.depth_head.scratch.output_conv2.2.weight', 'vggt.depth_head.scratch.refinenet1.out_conv.bias', 'vggt.depth_head.scratch.refinenet1.out_conv.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet2.out_conv.bias', 'vggt.depth_head.scratch.refinenet2.out_conv.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet3.out_conv.bias', 'vggt.depth_head.scratch.refinenet3.out_conv.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.depth_head.scratch.refinenet4.out_conv.bias', 'vggt.depth_head.scratch.refinenet4.out_conv.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.depth_head.scratch.refinenet4.resConfUnit2.conv2.weight', 'vggt.point_head.norm.bias', 'vggt.point_head.norm.weight', 'vggt.point_head.projects.0.bias', 'vggt.point_head.projects.0.weight', 'vggt.point_head.projects.1.bias', 'vggt.point_head.projects.1.weight', 'vggt.point_head.projects.2.bias', 'vggt.point_head.projects.2.weight', 'vggt.point_head.projects.3.bias', 'vggt.point_head.projects.3.weight', 'vggt.point_head.resize_layers.0.bias', 'vggt.point_head.resize_layers.0.weight', 'vggt.point_head.resize_layers.1.bias', 'vggt.point_head.resize_layers.1.weight', 'vggt.point_head.resize_layers.3.bias', 'vggt.point_head.resize_layers.3.weight', 'vggt.point_head.scratch.layer1_rn.weight', 'vggt.point_head.scratch.layer2_rn.weight', 'vggt.point_head.scratch.layer3_rn.weight', 'vggt.point_head.scratch.layer4_rn.weight', 'vggt.point_head.scratch.output_conv1.bias', 'vggt.point_head.scratch.output_conv1.weight', 'vggt.point_head.scratch.output_conv2.0.bias', 'vggt.point_head.scratch.output_conv2.0.weight', 'vggt.point_head.scratch.output_conv2.2.bias', 'vggt.point_head.scratch.output_conv2.2.weight', 'vggt.point_head.scratch.refinenet1.out_conv.bias', 'vggt.point_head.scratch.refinenet1.out_conv.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet1.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet2.out_conv.bias', 'vggt.point_head.scratch.refinenet2.out_conv.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet2.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet3.out_conv.bias', 'vggt.point_head.scratch.refinenet3.out_conv.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit1.conv2.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet3.resConfUnit2.conv2.weight', 'vggt.point_head.scratch.refinenet4.out_conv.bias', 'vggt.point_head.scratch.refinenet4.out_conv.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv1.weight', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.bias', 'vggt.point_head.scratch.refinenet4.resConfUnit2.conv2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading VGGT from /mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Loading weights from local directory
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Vision Module - Attention Blocks:
Trainable Block Indices: None
Non-Trainable Block Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
Merger Module Trainable: True
LLM Module - Embed Tokens Trainable: True
LLM Module - Trainable Layer Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]
LLM Module - Non-Trainable Layer Indices: None
[rank0] language_feature_fusion trainable params: 125,853,696
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': 'train_r2r_rxr', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Total training samples: 2429620
Total training samples: 2429620
Total training samples: 2429620
Total training samples: 2429620
Formatting inputs...Skip in lazy mode
Total training samples: 2429620
Formatting inputs...Skip in lazy mode
Formatting inputs...Skip in lazy mode
Formatting inputs...Skip in lazy mode
Total training samples: 2429620
Formatting inputs...Skip in lazy mode
Formatting inputs...Skip in lazy mode
Total training samples: 2429620
Formatting inputs...Skip in lazy mode
Total training samples: 2429620
Formatting inputs...Skip in lazy mode
[RANK 0] Gradient accumulation steps mismatch: GradientAccumulationPlugin has 1, DeepSpeed config has 8. Using DeepSpeed's value.

  0%|          | 0/37962 [00:00<?, ?it/s]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

  0%|          | 1/37962 [00:33<349:27:15, 33.14s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

  0%|          | 2/37962 [00:42<199:36:15, 18.93s/it]
  0%|          | 3/37962 [00:51<153:23:15, 14.55s/it]
  0%|          | 4/37962 [01:00<131:55:29, 12.51s/it]
  0%|          | 5/37962 [01:09<119:07:29, 11.30s/it]
  0%|          | 6/37962 [01:18<110:51:26, 10.51s/it]
  0%|          | 7/37962 [01:27<105:35:03, 10.01s/it]
  0%|          | 8/37962 [01:37<102:28:53,  9.72s/it]
  0%|          | 9/37962 [01:45<99:51:49,  9.47s/it] 
  0%|          | 10/37962 [01:54<97:46:41,  9.27s/it]
                                                     
{'loss': 3.1922, 'grad_norm': 283.7031555175781, 'learning_rate': 1.7559262510974542e-07, 'epoch': 0.0}

  0%|          | 10/37962 [01:55<97:46:41,  9.27s/it]
  0%|          | 11/37962 [02:04<98:48:03,  9.37s/it]
  0%|          | 12/37962 [02:13<97:25:26,  9.24s/it]
  0%|          | 13/37962 [02:22<96:13:39,  9.13s/it]
  0%|          | 14/37962 [02:31<95:29:21,  9.06s/it]
  0%|          | 15/37962 [02:40<95:15:42,  9.04s/it]
  0%|          | 16/37962 [02:48<94:30:17,  8.97s/it]
  0%|          | 17/37962 [02:57<93:29:40,  8.87s/it]
  0%|          | 18/37962 [03:06<93:51:04,  8.90s/it]
  0%|          | 19/37962 [03:15<93:37:15,  8.88s/it]
  0%|          | 20/37962 [03:23<92:22:45,  8.77s/it]
                                                     
{'loss': 2.8665, 'grad_norm': 176.34739685058594, 'learning_rate': 3.5118525021949084e-07, 'epoch': 0.0}

  0%|          | 20/37962 [03:23<92:22:45,  8.77s/it]
  0%|          | 21/37962 [03:32<93:19:50,  8.86s/it]
  0%|          | 22/37962 [03:41<93:55:14,  8.91s/it]
  0%|          | 23/37962 [03:51<95:01:54,  9.02s/it]
  0%|          | 24/37962 [04:00<94:59:17,  9.01s/it]
  0%|          | 25/37962 [04:09<94:48:07,  9.00s/it]
  0%|          | 26/37962 [04:17<94:09:45,  8.94s/it]
  0%|          | 27/37962 [04:27<94:28:09,  8.97s/it]
  0%|          | 28/37962 [04:35<94:24:15,  8.96s/it]
  0%|          | 29/37962 [04:44<93:22:28,  8.86s/it]
  0%|          | 30/37962 [04:53<93:04:31,  8.83s/it]
                                                     
{'loss': 2.3801, 'grad_norm': 77.87225341796875, 'learning_rate': 5.267778753292362e-07, 'epoch': 0.0}

  0%|          | 30/37962 [04:53<93:04:31,  8.83s/it]
  0%|          | 31/37962 [05:02<93:04:12,  8.83s/it]
  0%|          | 32/37962 [05:10<92:11:25,  8.75s/it]
  0%|          | 33/37962 [05:19<92:47:33,  8.81s/it]
  0%|          | 34/37962 [05:28<92:23:06,  8.77s/it]
  0%|          | 35/37962 [05:37<93:26:58,  8.87s/it]
  0%|          | 36/37962 [05:46<93:43:11,  8.90s/it]
  0%|          | 37/37962 [05:55<93:21:27,  8.86s/it]
  0%|          | 38/37962 [06:03<91:53:00,  8.72s/it]
  0%|          | 39/37962 [06:12<90:54:38,  8.63s/it]
  0%|          | 40/37962 [06:20<91:39:02,  8.70s/it]
                                                     
{'loss': 1.9282, 'grad_norm': 28.646303176879883, 'learning_rate': 7.023705004389817e-07, 'epoch': 0.0}

  0%|          | 40/37962 [06:20<91:39:02,  8.70s/it]
  0%|          | 41/37962 [06:29<92:23:37,  8.77s/it]
  0%|          | 42/37962 [06:39<94:03:14,  8.93s/it]
  0%|          | 43/37962 [06:48<94:41:52,  8.99s/it]
  0%|          | 44/37962 [06:56<93:49:19,  8.91s/it]
  0%|          | 45/37962 [07:05<93:01:37,  8.83s/it]
  0%|          | 46/37962 [07:14<92:57:20,  8.83s/it]
  0%|          | 47/37962 [07:23<92:34:21,  8.79s/it]
  0%|          | 48/37962 [07:31<92:32:09,  8.79s/it]
  0%|          | 49/37962 [07:40<92:53:33,  8.82s/it]
  0%|          | 50/37962 [07:49<93:09:37,  8.85s/it]
                                                     
{'loss': 2.0488, 'grad_norm': 102.01678466796875, 'learning_rate': 8.77963125548727e-07, 'epoch': 0.0}

  0%|          | 50/37962 [07:49<93:09:37,  8.85s/it]
  0%|          | 51/37962 [07:58<93:16:46,  8.86s/it]
  0%|          | 52/37962 [08:07<92:46:54,  8.81s/it]
  0%|          | 53/37962 [08:15<91:40:45,  8.71s/it]
  0%|          | 54/37962 [08:24<92:13:43,  8.76s/it]
  0%|          | 55/37962 [08:33<91:56:42,  8.73s/it]
  0%|          | 56/37962 [08:41<91:19:27,  8.67s/it]
  0%|          | 57/37962 [08:50<91:16:42,  8.67s/it]
  0%|          | 58/37962 [08:59<90:56:32,  8.64s/it]
  0%|          | 59/37962 [09:07<91:29:55,  8.69s/it]
  0%|          | 60/37962 [09:16<91:40:57,  8.71s/it]
                                                     
{'loss': 1.9304, 'grad_norm': 83.87590789794922, 'learning_rate': 1.0535557506584724e-06, 'epoch': 0.0}

  0%|          | 60/37962 [09:16<91:40:57,  8.71s/it]
  0%|          | 61/37962 [09:25<91:41:15,  8.71s/it]
  0%|          | 62/37962 [09:33<90:37:04,  8.61s/it]
  0%|          | 63/37962 [09:42<91:11:58,  8.66s/it]
  0%|          | 64/37962 [09:51<91:27:34,  8.69s/it]
  0%|          | 65/37962 [10:00<92:03:57,  8.75s/it]
  0%|          | 66/37962 [10:08<90:59:32,  8.64s/it]
  0%|          | 67/37962 [10:17<92:06:18,  8.75s/it]
  0%|          | 68/37962 [10:26<92:09:13,  8.75s/it]
  0%|          | 69/37962 [10:34<91:24:47,  8.68s/it]
  0%|          | 70/37962 [10:43<91:36:34,  8.70s/it]
                                                     
{'loss': 1.8107, 'grad_norm': 46.90418243408203, 'learning_rate': 1.2291483757682179e-06, 'epoch': 0.0}

  0%|          | 70/37962 [10:43<91:36:34,  8.70s/it]
  0%|          | 71/37962 [10:52<91:49:46,  8.72s/it]
  0%|          | 72/37962 [11:01<92:02:41,  8.75s/it]
  0%|          | 73/37962 [11:10<92:49:03,  8.82s/it]
  0%|          | 74/37962 [11:18<92:21:14,  8.78s/it]
  0%|          | 75/37962 [11:27<92:05:25,  8.75s/it]
  0%|          | 76/37962 [11:36<93:08:08,  8.85s/it]
  0%|          | 77/37962 [11:45<93:48:52,  8.91s/it]
  0%|          | 78/37962 [11:54<93:13:08,  8.86s/it]
  0%|          | 79/37962 [12:03<93:14:25,  8.86s/it]
  0%|          | 80/37962 [12:11<92:48:11,  8.82s/it]
                                                     
{'loss': 1.9055, 'grad_norm': 23.468515396118164, 'learning_rate': 1.4047410008779634e-06, 'epoch': 0.0}

  0%|          | 80/37962 [12:11<92:48:11,  8.82s/it]
  0%|          | 81/37962 [12:20<92:59:23,  8.84s/it]
  0%|          | 82/37962 [12:29<93:21:44,  8.87s/it]
  0%|          | 83/37962 [12:38<92:55:20,  8.83s/it]
  0%|          | 84/37962 [12:47<92:33:50,  8.80s/it]
  0%|          | 85/37962 [12:56<93:08:11,  8.85s/it]
  0%|          | 86/37962 [13:05<93:08:28,  8.85s/it]
  0%|          | 87/37962 [13:13<92:36:29,  8.80s/it]
  0%|          | 88/37962 [13:22<91:16:17,  8.68s/it]
  0%|          | 89/37962 [13:30<91:31:32,  8.70s/it]
  0%|          | 90/37962 [13:39<90:53:12,  8.64s/it]
                                                     
{'loss': 1.79, 'grad_norm': 83.90035247802734, 'learning_rate': 1.5803336259877086e-06, 'epoch': 0.0}

  0%|          | 90/37962 [13:39<90:53:12,  8.64s/it]
  0%|          | 91/37962 [13:48<91:19:08,  8.68s/it]
  0%|          | 92/37962 [13:56<90:17:32,  8.58s/it]
  0%|          | 93/37962 [14:05<90:04:36,  8.56s/it]
  0%|          | 94/37962 [14:13<90:14:25,  8.58s/it]
  0%|          | 95/37962 [14:22<90:01:55,  8.56s/it]
  0%|          | 96/37962 [14:31<90:54:19,  8.64s/it]
  0%|          | 97/37962 [14:39<91:40:38,  8.72s/it]
  0%|          | 98/37962 [14:48<92:04:31,  8.75s/it]
  0%|          | 99/37962 [14:57<92:05:04,  8.76s/it]
  0%|          | 100/37962 [15:06<92:05:53,  8.76s/it]
                                                      
{'loss': 1.7498, 'grad_norm': 59.483978271484375, 'learning_rate': 1.755926251097454e-06, 'epoch': 0.0}

  0%|          | 100/37962 [15:06<92:05:53,  8.76s/it]
  0%|          | 101/37962 [15:14<91:52:27,  8.74s/it]
  0%|          | 102/37962 [15:23<91:53:10,  8.74s/it]
  0%|          | 103/37962 [15:32<91:44:37,  8.72s/it]
  0%|          | 104/37962 [15:41<91:43:39,  8.72s/it]
  0%|          | 105/37962 [15:49<91:44:53,  8.72s/it]
  0%|          | 106/37962 [15:58<92:46:20,  8.82s/it]
  0%|          | 107/37962 [16:07<93:33:22,  8.90s/it]
  0%|          | 108/37962 [16:16<93:21:53,  8.88s/it]
  0%|          | 109/37962 [16:25<93:31:28,  8.89s/it]
  0%|          | 110/37962 [16:34<93:35:53,  8.90s/it]
                                                      
{'loss': 1.81, 'grad_norm': 39.354087829589844, 'learning_rate': 1.9315188762071994e-06, 'epoch': 0.0}

  0%|          | 110/37962 [16:34<93:35:53,  8.90s/it]
  0%|          | 111/37962 [16:43<93:27:43,  8.89s/it]
  0%|          | 112/37962 [16:52<93:27:54,  8.89s/it]
  0%|          | 113/37962 [17:01<93:00:14,  8.85s/it]
  0%|          | 114/37962 [17:10<93:11:02,  8.86s/it]
  0%|          | 115/37962 [17:18<93:11:53,  8.87s/it]
  0%|          | 116/37962 [17:27<93:04:11,  8.85s/it]
  0%|          | 117/37962 [17:36<93:39:36,  8.91s/it]
  0%|          | 118/37962 [17:45<92:46:35,  8.83s/it]
  0%|          | 119/37962 [17:54<92:49:14,  8.83s/it]
  0%|          | 120/37962 [18:03<92:29:15,  8.80s/it]
                                                      
{'loss': 1.8988, 'grad_norm': 56.23602294921875, 'learning_rate': 2.1071115013169447e-06, 'epoch': 0.0}

  0%|          | 120/37962 [18:03<92:29:15,  8.80s/it]
  0%|          | 121/37962 [18:11<91:03:25,  8.66s/it]
  0%|          | 122/37962 [18:20<91:03:19,  8.66s/it]
  0%|          | 123/37962 [18:28<91:12:09,  8.68s/it]
  0%|          | 124/37962 [18:37<90:35:51,  8.62s/it]
  0%|          | 125/37962 [18:45<89:32:58,  8.52s/it]
  0%|          | 126/37962 [18:54<90:05:26,  8.57s/it]
  0%|          | 127/37962 [19:03<91:20:03,  8.69s/it]
  0%|          | 128/37962 [19:11<90:56:46,  8.65s/it]
  0%|          | 129/37962 [19:20<91:10:21,  8.68s/it]
  0%|          | 130/37962 [19:29<91:33:56,  8.71s/it]
                                                      
{'loss': 1.7794, 'grad_norm': 24.504413604736328, 'learning_rate': 2.2827041264266904e-06, 'epoch': 0.0}

  0%|          | 130/37962 [19:29<91:33:56,  8.71s/it]
  0%|          | 131/37962 [19:38<91:59:52,  8.75s/it]
  0%|          | 132/37962 [19:46<92:00:02,  8.76s/it]
  0%|          | 133/37962 [19:55<90:27:12,  8.61s/it]
  0%|          | 134/37962 [20:04<91:33:04,  8.71s/it]
  0%|          | 135/37962 [20:12<91:19:09,  8.69s/it]
  0%|          | 136/37962 [20:21<91:38:27,  8.72s/it]
  0%|          | 137/37962 [20:30<91:44:27,  8.73s/it]
  0%|          | 138/37962 [20:39<92:07:36,  8.77s/it]
  0%|          | 139/37962 [20:47<91:49:25,  8.74s/it]
  0%|          | 140/37962 [20:56<91:20:57,  8.69s/it]
                                                      
{'loss': 1.7848, 'grad_norm': 81.9885482788086, 'learning_rate': 2.4582967515364357e-06, 'epoch': 0.0}

  0%|          | 140/37962 [20:56<91:20:57,  8.69s/it]
  0%|          | 141/37962 [21:05<91:27:46,  8.71s/it]
  0%|          | 142/37962 [21:13<91:07:49,  8.67s/it]
  0%|          | 143/37962 [21:22<90:50:03,  8.65s/it]
  0%|          | 144/37962 [21:31<91:32:40,  8.71s/it]
  0%|          | 145/37962 [21:40<91:59:19,  8.76s/it]
  0%|          | 146/37962 [21:48<92:01:05,  8.76s/it]
  0%|          | 147/37962 [21:57<92:26:58,  8.80s/it]
  0%|          | 148/37962 [22:06<92:46:39,  8.83s/it]
  0%|          | 149/37962 [22:15<92:49:08,  8.84s/it]
  0%|          | 150/37962 [22:24<92:13:11,  8.78s/it]
                                                      
{'loss': 1.7798, 'grad_norm': 17.06349754333496, 'learning_rate': 2.633889376646181e-06, 'epoch': 0.0}

  0%|          | 150/37962 [22:24<92:13:11,  8.78s/it]
  0%|          | 151/37962 [22:32<91:55:58,  8.75s/it]
  0%|          | 152/37962 [22:41<92:32:13,  8.81s/it]
  0%|          | 153/37962 [22:50<92:41:09,  8.83s/it]
  0%|          | 154/37962 [22:59<92:33:52,  8.81s/it]
  0%|          | 155/37962 [23:08<92:53:06,  8.84s/it]
  0%|          | 156/37962 [23:16<92:22:37,  8.80s/it]
  0%|          | 157/37962 [23:25<92:06:39,  8.77s/it]
  0%|          | 158/37962 [23:34<91:58:16,  8.76s/it]
  0%|          | 159/37962 [23:43<91:29:48,  8.71s/it]
  0%|          | 160/37962 [23:51<91:48:03,  8.74s/it]
                                                      
{'loss': 1.8855, 'grad_norm': 83.96887969970703, 'learning_rate': 2.8094820017559267e-06, 'epoch': 0.0}

  0%|          | 160/37962 [23:51<91:48:03,  8.74s/it]
  0%|          | 161/37962 [24:00<91:01:18,  8.67s/it]
  0%|          | 162/37962 [24:09<91:08:28,  8.68s/it]
  0%|          | 163/37962 [24:17<90:20:23,  8.60s/it]
  0%|          | 164/37962 [24:26<91:40:26,  8.73s/it]
  0%|          | 165/37962 [24:35<92:24:58,  8.80s/it]
  0%|          | 166/37962 [24:44<92:57:25,  8.85s/it]
  0%|          | 167/37962 [24:52<91:54:49,  8.75s/it]
  0%|          | 168/37962 [25:01<91:53:45,  8.75s/it]
  0%|          | 169/37962 [25:10<91:01:04,  8.67s/it]
  0%|          | 170/37962 [25:18<91:17:18,  8.70s/it]
                                                      
{'loss': 1.8967, 'grad_norm': 67.86595916748047, 'learning_rate': 2.9850746268656716e-06, 'epoch': 0.0}

  0%|          | 170/37962 [25:18<91:17:18,  8.70s/it]
  0%|          | 171/37962 [25:27<90:20:13,  8.61s/it]
  0%|          | 172/37962 [25:36<91:18:42,  8.70s/it]
  0%|          | 173/37962 [25:44<91:19:22,  8.70s/it]
  0%|          | 174/37962 [25:53<91:34:12,  8.72s/it]
  0%|          | 175/37962 [26:02<92:07:04,  8.78s/it]
  0%|          | 176/37962 [26:11<92:10:20,  8.78s/it]
  0%|          | 177/37962 [26:19<91:24:36,  8.71s/it]
  0%|          | 178/37962 [26:28<91:53:19,  8.76s/it]
  0%|          | 179/37962 [26:37<92:02:28,  8.77s/it]
  0%|          | 180/37962 [26:46<92:00:55,  8.77s/it]
                                                      
{'loss': 1.802, 'grad_norm': 46.12517547607422, 'learning_rate': 3.1606672519754173e-06, 'epoch': 0.0}

  0%|          | 180/37962 [26:46<92:00:55,  8.77s/it]
  0%|          | 181/37962 [26:55<92:27:44,  8.81s/it]
  0%|          | 182/37962 [27:04<92:35:43,  8.82s/it]
  0%|          | 183/37962 [27:12<92:35:21,  8.82s/it]
  0%|          | 184/37962 [27:21<92:10:40,  8.78s/it]
  0%|          | 185/37962 [27:30<92:33:51,  8.82s/it]
  0%|          | 186/37962 [27:39<92:44:36,  8.84s/it]
  0%|          | 187/37962 [27:48<92:53:33,  8.85s/it]
  0%|          | 188/37962 [27:57<92:22:10,  8.80s/it]
  0%|          | 189/37962 [28:05<92:24:41,  8.81s/it]
  1%|          | 190/37962 [28:14<92:37:20,  8.83s/it]
                                                      
{'loss': 1.7118, 'grad_norm': 16.377017974853516, 'learning_rate': 3.336259877085163e-06, 'epoch': 0.01}

  1%|          | 190/37962 [28:14<92:37:20,  8.83s/it]
  1%|          | 191/37962 [28:23<92:36:17,  8.83s/it]
  1%|          | 192/37962 [28:32<92:24:45,  8.81s/it]
  1%|          | 193/37962 [28:40<91:29:22,  8.72s/it]
  1%|          | 194/37962 [28:49<91:11:34,  8.69s/it]
  1%|          | 195/37962 [28:58<92:22:46,  8.81s/it]
  1%|          | 196/37962 [29:07<92:32:14,  8.82s/it]
  1%|          | 197/37962 [29:16<92:04:14,  8.78s/it]
  1%|          | 198/37962 [29:24<92:19:42,  8.80s/it]
  1%|          | 199/37962 [29:33<92:46:09,  8.84s/it]
  1%|          | 200/37962 [29:42<91:29:12,  8.72s/it]
                                                      
{'loss': 1.8184, 'grad_norm': 70.087890625, 'learning_rate': 3.511852502194908e-06, 'epoch': 0.01}

  1%|          | 200/37962 [29:42<91:29:12,  8.72s/it]
  1%|          | 201/37962 [29:51<91:31:58,  8.73s/it]
  1%|          | 202/37962 [29:59<91:38:54,  8.74s/it]
  1%|          | 203/37962 [30:08<92:09:42,  8.79s/it]
  1%|          | 204/37962 [30:17<92:30:16,  8.82s/it]
  1%|          | 205/37962 [30:26<92:31:12,  8.82s/it]
  1%|          | 206/37962 [30:35<92:28:38,  8.82s/it]
  1%|          | 207/37962 [30:44<92:35:00,  8.83s/it]
  1%|          | 208/37962 [30:52<92:16:37,  8.80s/it]
  1%|          | 209/37962 [31:01<92:33:07,  8.83s/it]
  1%|          | 210/37962 [31:10<91:58:26,  8.77s/it]
                                                      
{'loss': 1.7608, 'grad_norm': 68.34032440185547, 'learning_rate': 3.6874451273046536e-06, 'epoch': 0.01}

  1%|          | 210/37962 [31:10<91:58:26,  8.77s/it]
  1%|          | 211/37962 [31:19<92:17:44,  8.80s/it]
  1%|          | 212/37962 [31:28<92:25:00,  8.81s/it]
  1%|          | 213/37962 [31:36<92:36:44,  8.83s/it]
  1%|          | 214/37962 [31:45<92:29:19,  8.82s/it]
  1%|          | 215/37962 [31:54<93:14:58,  8.89s/it]
  1%|          | 216/37962 [32:03<93:27:17,  8.91s/it]
  1%|          | 217/37962 [32:12<93:14:34,  8.89s/it]
  1%|          | 218/37962 [32:21<92:45:40,  8.85s/it]
  1%|          | 219/37962 [32:30<92:18:40,  8.80s/it]
  1%|          | 220/37962 [32:38<92:11:53,  8.79s/it]
                                                      
{'loss': 1.6859, 'grad_norm': 158.8023223876953, 'learning_rate': 3.863037752414399e-06, 'epoch': 0.01}

  1%|          | 220/37962 [32:38<92:11:53,  8.79s/it]
  1%|          | 221/37962 [32:47<90:59:39,  8.68s/it]
  1%|          | 222/37962 [32:56<91:33:37,  8.73s/it]
  1%|          | 223/37962 [33:04<91:59:39,  8.78s/it]
  1%|          | 224/37962 [33:13<92:13:51,  8.80s/it]
  1%|          | 225/37962 [33:22<92:35:11,  8.83s/it]
  1%|          | 226/37962 [33:31<91:15:25,  8.71s/it]
  1%|          | 227/37962 [33:40<91:46:09,  8.75s/it]
  1%|          | 228/37962 [33:49<92:35:01,  8.83s/it]
  1%|          | 229/37962 [33:57<91:44:27,  8.75s/it]
  1%|          | 230/37962 [34:06<91:42:41,  8.75s/it]
                                                      
{'loss': 1.7302, 'grad_norm': 79.01270294189453, 'learning_rate': 4.038630377524144e-06, 'epoch': 0.01}

  1%|          | 230/37962 [34:06<91:42:41,  8.75s/it]
  1%|          | 231/37962 [34:15<92:06:05,  8.79s/it]
  1%|          | 232/37962 [34:23<91:56:38,  8.77s/it]
  1%|          | 233/37962 [34:32<92:37:50,  8.84s/it]
  1%|          | 234/37962 [34:41<92:45:23,  8.85s/it]
  1%|          | 235/37962 [34:50<92:43:27,  8.85s/it]
  1%|          | 236/37962 [34:59<92:15:07,  8.80s/it]
  1%|          | 237/37962 [35:08<91:50:32,  8.76s/it]
  1%|          | 238/37962 [35:16<91:42:41,  8.75s/it]
  1%|          | 239/37962 [35:25<91:42:20,  8.75s/it]
  1%|          | 240/37962 [35:33<90:14:21,  8.61s/it]
                                                      
{'loss': 1.99, 'grad_norm': 110.54615020751953, 'learning_rate': 4.2142230026338894e-06, 'epoch': 0.01}

  1%|          | 240/37962 [35:33<90:14:21,  8.61s/it]
  1%|          | 241/37962 [35:42<89:15:09,  8.52s/it]
  1%|          | 242/37962 [35:50<89:48:25,  8.57s/it]
  1%|          | 243/37962 [35:59<90:41:06,  8.66s/it]
  1%|          | 244/37962 [36:08<91:53:32,  8.77s/it]
  1%|          | 245/37962 [36:17<91:58:44,  8.78s/it]
  1%|          | 246/37962 [36:26<92:05:06,  8.79s/it]
  1%|          | 247/37962 [36:34<91:29:51,  8.73s/it]
  1%|          | 248/37962 [36:43<92:01:18,  8.78s/it]
  1%|          | 249/37962 [36:52<91:01:26,  8.69s/it]
  1%|          | 250/37962 [37:01<91:11:58,  8.71s/it]
                                                      
{'loss': 1.5635, 'grad_norm': 22.725868225097656, 'learning_rate': 4.389815627743635e-06, 'epoch': 0.01}

  1%|          | 250/37962 [37:01<91:11:58,  8.71s/it]
  1%|          | 251/37962 [37:09<90:50:55,  8.67s/it]
  1%|          | 252/37962 [37:18<91:03:32,  8.69s/it]
  1%|          | 253/37962 [37:27<92:04:00,  8.79s/it]
  1%|          | 254/37962 [37:36<92:44:17,  8.85s/it]
  1%|          | 255/37962 [37:45<92:09:11,  8.80s/it]
  1%|          | 256/37962 [37:53<91:05:04,  8.70s/it]
  1%|          | 257/37962 [38:02<91:05:48,  8.70s/it]
  1%|          | 258/37962 [38:10<90:32:58,  8.65s/it]
  1%|          | 259/37962 [38:19<90:14:45,  8.62s/it]
  1%|          | 260/37962 [38:28<91:08:25,  8.70s/it]
                                                      
{'loss': 1.849, 'grad_norm': 28.27669906616211, 'learning_rate': 4.565408252853381e-06, 'epoch': 0.01}

  1%|          | 260/37962 [38:28<91:08:25,  8.70s/it]
  1%|          | 261/37962 [38:36<91:25:13,  8.73s/it]
  1%|          | 262/37962 [38:46<93:01:37,  8.88s/it]
  1%|          | 263/37962 [38:55<95:39:36,  9.13s/it]
  1%|          | 264/37962 [39:05<97:32:37,  9.32s/it]
  1%|          | 265/37962 [39:14<97:09:35,  9.28s/it]
  1%|          | 266/37962 [39:23<95:09:26,  9.09s/it]
  1%|          | 267/37962 [39:32<93:26:00,  8.92s/it]
  1%|          | 268/37962 [39:41<93:44:19,  8.95s/it]
  1%|          | 269/37962 [39:49<93:25:33,  8.92s/it]
  1%|          | 270/37962 [39:59<94:05:54,  8.99s/it]
                                                      
{'loss': 1.5261, 'grad_norm': 30.341520309448242, 'learning_rate': 4.741000877963126e-06, 'epoch': 0.01}

  1%|          | 270/37962 [39:59<94:05:54,  8.99s/it]
  1%|          | 271/37962 [40:08<95:58:48,  9.17s/it]
  1%|          | 272/37962 [40:19<101:58:58,  9.74s/it]
  1%|          | 273/37962 [40:29<103:18:03,  9.87s/it]
  1%|          | 274/37962 [40:38<100:25:33,  9.59s/it]
  1%|          | 275/37962 [40:47<98:00:53,  9.36s/it] 
  1%|          | 276/37962 [40:56<96:01:05,  9.17s/it]
  1%|          | 277/37962 [41:05<95:11:06,  9.09s/it]
  1%|          | 278/37962 [41:14<96:15:15,  9.20s/it]
  1%|          | 279/37962 [41:25<101:54:23,  9.74s/it]
  1%|          | 280/37962 [41:35<102:36:52,  9.80s/it]
                                                       
{'loss': 1.7342, 'grad_norm': 41.23030090332031, 'learning_rate': 4.916593503072871e-06, 'epoch': 0.01}

  1%|          | 280/37962 [41:35<102:36:52,  9.80s/it]
  1%|          | 281/37962 [41:44<100:56:59,  9.64s/it]
  1%|          | 282/37962 [41:53<97:09:37,  9.28s/it] 
  1%|          | 283/37962 [42:01<94:28:40,  9.03s/it]
  1%|          | 284/37962 [42:10<92:39:07,  8.85s/it]
  1%|          | 285/37962 [42:18<91:20:05,  8.73s/it]
  1%|          | 286/37962 [42:27<91:55:18,  8.78s/it]
  1%|          | 287/37962 [42:36<92:17:53,  8.82s/it]
  1%|          | 288/37962 [42:45<93:51:43,  8.97s/it]
  1%|          | 289/37962 [42:56<98:14:56,  9.39s/it]
  1%|          | 290/37962 [43:06<99:46:04,  9.53s/it]
                                                      
{'loss': 1.6142, 'grad_norm': 33.45962142944336, 'learning_rate': 5.092186128182617e-06, 'epoch': 0.01}

  1%|          | 290/37962 [43:06<99:46:04,  9.53s/it]
  1%|          | 291/37962 [43:15<99:19:43,  9.49s/it]
  1%|          | 292/37962 [43:23<96:00:10,  9.17s/it]
  1%|          | 293/37962 [43:32<93:28:29,  8.93s/it]
  1%|          | 294/37962 [43:40<91:04:03,  8.70s/it]
  1%|          | 295/37962 [43:48<89:43:54,  8.58s/it]
  1%|          | 296/37962 [43:56<88:22:40,  8.45s/it]
  1%|          | 297/37962 [44:05<87:59:34,  8.41s/it]
  1%|          | 298/37962 [44:13<87:27:51,  8.36s/it]
  1%|          | 299/37962 [44:21<86:48:20,  8.30s/it]
  1%|          | 300/37962 [44:29<86:13:40,  8.24s/it]
                                                      
{'loss': 1.6266, 'grad_norm': 54.00699234008789, 'learning_rate': 5.267778753292362e-06, 'epoch': 0.01}

  1%|          | 300/37962 [44:29<86:13:40,  8.24s/it]
  1%|          | 301/37962 [44:38<86:53:30,  8.31s/it]
  1%|          | 302/37962 [44:46<86:54:24,  8.31s/it]
  1%|          | 303/37962 [44:54<87:12:00,  8.34s/it]
  1%|          | 304/37962 [45:03<87:09:08,  8.33s/it]
  1%|          | 305/37962 [45:11<87:02:32,  8.32s/it]
  1%|          | 306/37962 [45:19<87:11:39,  8.34s/it]
  1%|          | 307/37962 [45:28<87:12:53,  8.34s/it]
  1%|          | 308/37962 [45:36<87:25:43,  8.36s/it]
  1%|          | 309/37962 [45:44<86:36:57,  8.28s/it]
  1%|          | 310/37962 [45:52<85:57:39,  8.22s/it]
                                                      
{'loss': 1.5956, 'grad_norm': 43.90302276611328, 'learning_rate': 5.443371378402108e-06, 'epoch': 0.01}

  1%|          | 310/37962 [45:52<85:57:39,  8.22s/it]
  1%|          | 311/37962 [46:00<85:07:22,  8.14s/it]
  1%|          | 312/37962 [46:08<84:45:27,  8.10s/it]
  1%|          | 313/37962 [46:16<85:09:59,  8.14s/it]
  1%|          | 314/37962 [46:25<87:10:16,  8.34s/it]
  1%|          | 315/37962 [46:34<88:42:28,  8.48s/it]
  1%|          | 316/37962 [46:44<91:38:48,  8.76s/it]
  1%|          | 317/37962 [46:53<94:05:08,  9.00s/it]
  1%|          | 318/37962 [47:04<98:45:30,  9.44s/it]
  1%|          | 319/37962 [47:14<100:30:57,  9.61s/it]
  1%|          | 320/37962 [47:23<98:55:14,  9.46s/it] 
                                                      
{'loss': 1.7029, 'grad_norm': 91.89335632324219, 'learning_rate': 5.618964003511853e-06, 'epoch': 0.01}

  1%|          | 320/37962 [47:23<98:55:14,  9.46s/it]
  1%|          | 321/37962 [47:31<96:06:01,  9.19s/it]
  1%|          | 322/37962 [47:39<93:04:44,  8.90s/it]
  1%|          | 323/37962 [47:48<91:43:07,  8.77s/it]
  1%|          | 324/37962 [47:56<89:54:19,  8.60s/it]
  1%|          | 325/37962 [48:04<88:50:24,  8.50s/it]
  1%|          | 326/37962 [48:13<88:06:27,  8.43s/it]
  1%|          | 327/37962 [48:21<87:01:05,  8.32s/it]
  1%|          | 328/37962 [48:29<87:29:53,  8.37s/it]
  1%|          | 329/37962 [48:37<87:13:16,  8.34s/it]
  1%|          | 330/37962 [48:46<87:39:38,  8.39s/it]
                                                      
{'loss': 1.7131, 'grad_norm': 13.909472465515137, 'learning_rate': 5.794556628621598e-06, 'epoch': 0.01}

  1%|          | 330/37962 [48:46<87:39:38,  8.39s/it]
  1%|          | 331/37962 [48:54<87:40:57,  8.39s/it]
  1%|          | 332/37962 [49:03<87:39:28,  8.39s/it]
  1%|          | 333/37962 [49:11<87:04:29,  8.33s/it]
  1%|          | 334/37962 [49:19<86:47:10,  8.30s/it]
  1%|          | 335/37962 [49:27<86:46:26,  8.30s/it]
  1%|          | 336/37962 [49:36<86:45:17,  8.30s/it]
  1%|          | 337/37962 [49:44<86:29:22,  8.28s/it]
  1%|          | 338/37962 [49:52<86:09:46,  8.24s/it]
  1%|          | 339/37962 [50:01<86:44:00,  8.30s/it]
  1%|          | 340/37962 [50:09<86:00:29,  8.23s/it]
                                                      
{'loss': 1.6425, 'grad_norm': 44.409873962402344, 'learning_rate': 5.970149253731343e-06, 'epoch': 0.01}

  1%|          | 340/37962 [50:09<86:00:29,  8.23s/it]
  1%|          | 341/37962 [50:17<85:09:19,  8.15s/it]
  1%|          | 342/37962 [50:25<85:18:25,  8.16s/it]
  1%|          | 343/37962 [50:33<86:00:30,  8.23s/it]
  1%|          | 344/37962 [50:42<86:25:18,  8.27s/it]
  1%|          | 345/37962 [50:50<85:58:49,  8.23s/it]
  1%|          | 346/37962 [50:58<86:15:20,  8.26s/it]
  1%|          | 347/37962 [51:06<86:15:24,  8.26s/it]
  1%|          | 348/37962 [51:15<86:27:03,  8.27s/it]
  1%|          | 349/37962 [51:23<86:46:40,  8.31s/it]
  1%|          | 350/37962 [51:31<87:03:15,  8.33s/it]
                                                      
{'loss': 1.697, 'grad_norm': 17.333356857299805, 'learning_rate': 6.145741878841089e-06, 'epoch': 0.01}

  1%|          | 350/37962 [51:31<87:03:15,  8.33s/it]
  1%|          | 351/37962 [51:39<85:41:34,  8.20s/it]
  1%|          | 352/37962 [51:48<86:15:16,  8.26s/it]
  1%|          | 353/37962 [51:56<86:23:10,  8.27s/it]
  1%|          | 354/37962 [52:04<86:55:34,  8.32s/it]
  1%|          | 355/37962 [52:13<86:23:02,  8.27s/it]
  1%|          | 356/37962 [52:21<86:11:26,  8.25s/it]
  1%|          | 357/37962 [52:29<86:05:45,  8.24s/it]
  1%|          | 358/37962 [52:37<86:15:48,  8.26s/it]
  1%|          | 359/37962 [52:45<85:15:50,  8.16s/it]
  1%|          | 360/37962 [52:54<85:45:03,  8.21s/it]
                                                      
{'loss': 1.6329, 'grad_norm': 7.805108070373535, 'learning_rate': 6.3213345039508346e-06, 'epoch': 0.01}

  1%|          | 360/37962 [52:54<85:45:03,  8.21s/it]
  1%|          | 361/37962 [53:02<85:50:10,  8.22s/it]
  1%|          | 362/37962 [53:10<86:26:29,  8.28s/it]
  1%|          | 363/37962 [53:18<86:24:22,  8.27s/it]
  1%|          | 364/37962 [53:27<86:32:53,  8.29s/it]
  1%|          | 365/37962 [53:35<86:49:54,  8.31s/it]
  1%|          | 366/37962 [53:43<86:31:58,  8.29s/it]
  1%|          | 367/37962 [53:52<86:21:38,  8.27s/it]
  1%|          | 368/37962 [54:00<85:58:07,  8.23s/it]
  1%|          | 369/37962 [54:08<86:04:17,  8.24s/it]
  1%|          | 370/37962 [54:16<86:37:17,  8.30s/it]
                                                      
{'loss': 1.7064, 'grad_norm': 18.26932716369629, 'learning_rate': 6.49692712906058e-06, 'epoch': 0.01}

  1%|          | 370/37962 [54:16<86:37:17,  8.30s/it]
  1%|          | 371/37962 [54:25<86:49:17,  8.31s/it]
  1%|          | 372/37962 [54:33<87:33:21,  8.39s/it]
  1%|          | 373/37962 [54:42<87:07:23,  8.34s/it]
  1%|          | 374/37962 [54:49<85:28:57,  8.19s/it]
  1%|          | 375/37962 [54:58<85:50:49,  8.22s/it]
  1%|          | 376/37962 [55:06<86:18:10,  8.27s/it]
  1%|          | 377/37962 [55:14<86:44:58,  8.31s/it]
  1%|          | 378/37962 [55:23<86:40:09,  8.30s/it]
  1%|          | 379/37962 [55:31<87:16:13,  8.36s/it]
  1%|          | 380/37962 [55:40<87:40:30,  8.40s/it]
                                                      
{'loss': 1.7632, 'grad_norm': 76.36417388916016, 'learning_rate': 6.672519754170326e-06, 'epoch': 0.01}

  1%|          | 380/37962 [55:40<87:40:30,  8.40s/it]
  1%|          | 381/37962 [55:48<87:25:24,  8.37s/it]
  1%|          | 382/37962 [55:56<87:16:35,  8.36s/it]
  1%|          | 383/37962 [56:05<87:13:27,  8.36s/it]
  1%|          | 384/37962 [56:13<86:41:37,  8.31s/it]
  1%|          | 385/37962 [56:21<86:31:41,  8.29s/it]
  1%|          | 386/37962 [56:30<86:49:44,  8.32s/it]
  1%|          | 387/37962 [56:38<86:56:07,  8.33s/it]
  1%|          | 388/37962 [56:46<85:16:28,  8.17s/it]
  1%|          | 389/37962 [56:54<85:02:50,  8.15s/it]
  1%|          | 390/37962 [57:02<85:40:52,  8.21s/it]
                                                      
{'loss': 1.6917, 'grad_norm': 94.7707748413086, 'learning_rate': 6.84811237928007e-06, 'epoch': 0.01}

  1%|          | 390/37962 [57:02<85:40:52,  8.21s/it]
  1%|          | 391/37962 [57:10<85:53:29,  8.23s/it]
  1%|          | 392/37962 [57:19<85:53:35,  8.23s/it]
  1%|          | 393/37962 [57:27<86:26:19,  8.28s/it]
  1%|          | 394/37962 [57:35<84:35:06,  8.11s/it]
  1%|          | 395/37962 [57:43<84:52:50,  8.13s/it]
  1%|          | 396/37962 [57:51<84:30:56,  8.10s/it]
  1%|          | 397/37962 [57:59<85:04:21,  8.15s/it]
  1%|          | 398/37962 [58:08<85:34:44,  8.20s/it]
  1%|          | 399/37962 [58:16<85:06:09,  8.16s/it]
  1%|          | 400/37962 [58:24<85:37:09,  8.21s/it]
                                                      
{'loss': 1.6339, 'grad_norm': 17.537052154541016, 'learning_rate': 7.023705004389816e-06, 'epoch': 0.01}

  1%|          | 400/37962 [58:24<85:37:09,  8.21s/it]
  1%|          | 401/37962 [58:32<85:57:51,  8.24s/it]
  1%|          | 402/37962 [58:40<85:22:38,  8.18s/it]
  1%|          | 403/37962 [58:48<84:24:21,  8.09s/it]
  1%|          | 404/37962 [58:56<84:51:19,  8.13s/it]
  1%|          | 405/37962 [59:04<84:15:20,  8.08s/it]
  1%|          | 406/37962 [59:13<85:20:41,  8.18s/it]
  1%|          | 407/37962 [59:21<85:19:59,  8.18s/it]
  1%|          | 408/37962 [59:29<83:56:22,  8.05s/it]
  1%|          | 409/37962 [59:37<84:31:04,  8.10s/it]
  1%|          | 410/37962 [59:45<84:42:30,  8.12s/it]
                                                      
{'loss': 1.5146, 'grad_norm': 18.1473445892334, 'learning_rate': 7.199297629499562e-06, 'epoch': 0.01}

  1%|          | 410/37962 [59:45<84:42:30,  8.12s/it]
  1%|          | 411/37962 [59:53<85:18:57,  8.18s/it]
  1%|          | 412/37962 [1:00:02<85:22:45,  8.19s/it]
  1%|          | 413/37962 [1:00:10<85:43:40,  8.22s/it]
  1%|          | 414/37962 [1:00:18<85:51:27,  8.23s/it]
  1%|          | 415/37962 [1:00:26<85:59:03,  8.24s/it]
  1%|          | 416/37962 [1:00:35<85:52:19,  8.23s/it]
  1%|          | 417/37962 [1:00:43<85:23:33,  8.19s/it]
  1%|          | 418/37962 [1:00:51<85:01:59,  8.15s/it]
  1%|          | 419/37962 [1:00:59<85:35:35,  8.21s/it]
  1%|          | 420/37962 [1:01:08<86:04:45,  8.25s/it]
                                                        
{'loss': 1.5836, 'grad_norm': 28.40329933166504, 'learning_rate': 7.374890254609307e-06, 'epoch': 0.01}

  1%|          | 420/37962 [1:01:08<86:04:45,  8.25s/it]
  1%|          | 421/37962 [1:01:16<85:39:11,  8.21s/it]
  1%|          | 422/37962 [1:01:23<84:18:18,  8.08s/it]
  1%|          | 423/37962 [1:01:32<85:38:12,  8.21s/it]
  1%|          | 424/37962 [1:01:40<85:54:45,  8.24s/it]
  1%|          | 425/37962 [1:01:49<86:02:56,  8.25s/it]
  1%|          | 426/37962 [1:01:57<85:18:05,  8.18s/it]
  1%|          | 427/37962 [1:02:05<85:03:08,  8.16s/it]
  1%|          | 428/37962 [1:02:13<85:37:50,  8.21s/it]
  1%|          | 429/37962 [1:02:21<85:25:44,  8.19s/it]
  1%|          | 430/37962 [1:02:29<85:27:21,  8.20s/it]
                                                        
{'loss': 1.5443, 'grad_norm': 45.898616790771484, 'learning_rate': 7.550482879719052e-06, 'epoch': 0.01}

  1%|          | 430/37962 [1:02:29<85:27:21,  8.20s/it]
  1%|          | 431/37962 [1:02:38<85:57:27,  8.25s/it]
  1%|          | 432/37962 [1:02:46<86:10:44,  8.27s/it]
  1%|          | 433/37962 [1:02:54<85:56:55,  8.24s/it]
  1%|          | 434/37962 [1:03:03<88:32:12,  8.49s/it]
  1%|          | 435/37962 [1:03:13<92:01:29,  8.83s/it]
  1%|          | 436/37962 [1:03:22<92:24:28,  8.87s/it]
  1%|          | 437/37962 [1:03:31<93:44:17,  8.99s/it]
  1%|          | 438/37962 [1:03:41<95:10:50,  9.13s/it]
  1%|          | 439/37962 [1:03:50<96:05:18,  9.22s/it]
  1%|          | 440/37962 [1:04:00<97:36:43,  9.37s/it]
                                                        
{'loss': 1.6585, 'grad_norm': 16.250577926635742, 'learning_rate': 7.726075504828798e-06, 'epoch': 0.01}

  1%|          | 440/37962 [1:04:00<97:36:43,  9.37s/it]
  1%|          | 441/37962 [1:04:09<98:38:58,  9.47s/it]
  1%|          | 442/37962 [1:04:18<96:17:37,  9.24s/it]
  1%|          | 443/37962 [1:04:26<93:18:05,  8.95s/it]
  1%|          | 444/37962 [1:04:35<91:42:50,  8.80s/it]
  1%|          | 445/37962 [1:04:43<89:54:20,  8.63s/it]
  1%|          | 446/37962 [1:04:51<88:31:49,  8.50s/it]
  1%|          | 447/37962 [1:04:59<86:23:40,  8.29s/it]
  1%|          | 448/37962 [1:05:07<86:09:21,  8.27s/it]
  1%|          | 449/37962 [1:05:16<86:13:30,  8.27s/it]
  1%|          | 450/37962 [1:05:24<86:19:08,  8.28s/it]
                                                        
{'loss': 1.6632, 'grad_norm': 58.57795715332031, 'learning_rate': 7.901668129938544e-06, 'epoch': 0.01}

  1%|          | 450/37962 [1:05:24<86:19:08,  8.28s/it]
  1%|          | 451/37962 [1:05:32<86:07:51,  8.27s/it]
  1%|          | 452/37962 [1:05:40<86:14:30,  8.28s/it]
  1%|          | 453/37962 [1:05:48<85:30:11,  8.21s/it]
  1%|          | 454/37962 [1:05:57<85:33:46,  8.21s/it]
  1%|          | 455/37962 [1:06:05<85:54:26,  8.25s/it]
  1%|          | 456/37962 [1:06:13<85:52:59,  8.24s/it]
  1%|          | 457/37962 [1:06:21<85:33:56,  8.21s/it]
  1%|          | 458/37962 [1:06:30<86:11:41,  8.27s/it]
  1%|          | 459/37962 [1:06:38<84:52:13,  8.15s/it]
  1%|          | 460/37962 [1:06:46<85:01:39,  8.16s/it]
                                                        
{'loss': 1.6166, 'grad_norm': 50.49228286743164, 'learning_rate': 8.077260755048288e-06, 'epoch': 0.01}

  1%|          | 460/37962 [1:06:46<85:01:39,  8.16s/it]
  1%|          | 461/37962 [1:06:54<85:37:05,  8.22s/it]
  1%|          | 462/37962 [1:07:02<85:46:40,  8.23s/it]
  1%|          | 463/37962 [1:07:11<85:50:39,  8.24s/it]
  1%|          | 464/37962 [1:07:19<85:56:17,  8.25s/it]
  1%|          | 465/37962 [1:07:27<85:55:31,  8.25s/it]
  1%|          | 466/37962 [1:07:35<85:22:08,  8.20s/it]
  1%|          | 467/37962 [1:07:43<84:24:02,  8.10s/it]
  1%|          | 468/37962 [1:07:52<85:13:12,  8.18s/it]
  1%|          | 469/37962 [1:08:00<85:11:56,  8.18s/it]
  1%|          | 470/37962 [1:08:08<85:54:05,  8.25s/it]
                                                        
{'loss': 1.5089, 'grad_norm': 18.522960662841797, 'learning_rate': 8.252853380158033e-06, 'epoch': 0.01}

  1%|          | 470/37962 [1:08:08<85:54:05,  8.25s/it]
  1%|          | 471/37962 [1:08:16<85:52:51,  8.25s/it]
  1%|          | 472/37962 [1:08:24<84:45:07,  8.14s/it]
  1%|          | 473/37962 [1:08:33<85:24:55,  8.20s/it]
  1%|          | 474/37962 [1:08:41<85:27:58,  8.21s/it]
  1%|▏         | 475/37962 [1:08:49<85:45:48,  8.24s/it]
  1%|▏         | 476/37962 [1:08:57<85:59:43,  8.26s/it]
  1%|▏         | 477/37962 [1:09:06<86:19:42,  8.29s/it]
  1%|▏         | 478/37962 [1:09:14<85:41:12,  8.23s/it]
  1%|▏         | 479/37962 [1:09:22<86:44:36,  8.33s/it]
  1%|▏         | 480/37962 [1:09:31<86:19:14,  8.29s/it]
                                                        
{'loss': 1.7003, 'grad_norm': 17.64044952392578, 'learning_rate': 8.428446005267779e-06, 'epoch': 0.01}

  1%|▏         | 480/37962 [1:09:31<86:19:14,  8.29s/it]
  1%|▏         | 481/37962 [1:09:39<86:03:57,  8.27s/it]
  1%|▏         | 482/37962 [1:09:47<86:18:50,  8.29s/it]
  1%|▏         | 483/37962 [1:09:55<85:56:09,  8.25s/it]
  1%|▏         | 484/37962 [1:10:04<86:22:30,  8.30s/it]
  1%|▏         | 485/37962 [1:10:12<86:07:27,  8.27s/it]
  1%|▏         | 486/37962 [1:10:20<85:17:18,  8.19s/it]
  1%|▏         | 487/37962 [1:10:28<84:23:53,  8.11s/it]
  1%|▏         | 488/37962 [1:10:36<85:27:44,  8.21s/it]
  1%|▏         | 489/37962 [1:10:45<85:43:09,  8.23s/it]
  1%|▏         | 490/37962 [1:10:53<85:38:29,  8.23s/it]
                                                        
{'loss': 1.6881, 'grad_norm': 25.662025451660156, 'learning_rate': 8.604038630377525e-06, 'epoch': 0.01}

  1%|▏         | 490/37962 [1:10:53<85:38:29,  8.23s/it]
  1%|▏         | 491/37962 [1:11:01<84:51:09,  8.15s/it]
  1%|▏         | 492/37962 [1:11:09<84:27:11,  8.11s/it]
  1%|▏         | 493/37962 [1:11:17<84:40:58,  8.14s/it]
  1%|▏         | 494/37962 [1:11:25<84:52:20,  8.15s/it]
  1%|▏         | 495/37962 [1:11:34<85:12:56,  8.19s/it]
  1%|▏         | 496/37962 [1:11:42<84:47:07,  8.15s/it]
  1%|▏         | 497/37962 [1:11:50<84:38:20,  8.13s/it]
  1%|▏         | 498/37962 [1:11:58<85:12:38,  8.19s/it]
  1%|▏         | 499/37962 [1:12:07<86:09:02,  8.28s/it]
  1%|▏         | 500/37962 [1:12:15<85:58:17,  8.26s/it]
                                                        
{'loss': 1.8645, 'grad_norm': 58.899742126464844, 'learning_rate': 8.77963125548727e-06, 'epoch': 0.01}

  1%|▏         | 500/37962 [1:12:15<85:58:17,  8.26s/it]
  1%|▏         | 501/37962 [1:12:23<85:47:34,  8.24s/it]
  1%|▏         | 502/37962 [1:12:31<85:35:20,  8.23s/it]
  1%|▏         | 503/37962 [1:12:39<85:39:08,  8.23s/it]
  1%|▏         | 504/37962 [1:12:47<85:14:09,  8.19s/it]
  1%|▏         | 505/37962 [1:12:56<85:21:59,  8.20s/it]
  1%|▏         | 506/37962 [1:13:04<85:48:57,  8.25s/it]
  1%|▏         | 507/37962 [1:13:12<85:41:21,  8.24s/it]
  1%|▏         | 508/37962 [1:13:21<85:56:52,  8.26s/it]
  1%|▏         | 509/37962 [1:13:29<86:04:03,  8.27s/it]
  1%|▏         | 510/37962 [1:13:37<86:07:02,  8.28s/it]
                                                        
{'loss': 1.6712, 'grad_norm': 72.4648666381836, 'learning_rate': 8.955223880597016e-06, 'epoch': 0.01}

  1%|▏         | 510/37962 [1:13:37<86:07:02,  8.28s/it]
  1%|▏         | 511/37962 [1:13:46<86:22:58,  8.30s/it]
  1%|▏         | 512/37962 [1:13:54<86:30:32,  8.32s/it]
  1%|▏         | 513/37962 [1:14:02<86:39:41,  8.33s/it]
  1%|▏         | 514/37962 [1:14:11<86:48:26,  8.35s/it]
  1%|▏         | 515/37962 [1:14:19<86:37:38,  8.33s/it]
  1%|▏         | 516/37962 [1:14:27<86:38:22,  8.33s/it]
  1%|▏         | 517/37962 [1:14:35<86:25:46,  8.31s/it]
  1%|▏         | 518/37962 [1:14:44<86:38:16,  8.33s/it]
  1%|▏         | 519/37962 [1:14:52<86:42:30,  8.34s/it]
  1%|▏         | 520/37962 [1:15:01<89:13:03,  8.58s/it]
                                                        
{'loss': 1.6911, 'grad_norm': 63.722835540771484, 'learning_rate': 9.130816505706762e-06, 'epoch': 0.01}

  1%|▏         | 520/37962 [1:15:01<89:13:03,  8.58s/it]
  1%|▏         | 521/37962 [1:15:11<91:53:19,  8.84s/it]
  1%|▏         | 522/37962 [1:15:21<97:23:41,  9.36s/it]
  1%|▏         | 523/37962 [1:15:30<96:18:25,  9.26s/it]
  1%|▏         | 524/37962 [1:15:40<97:22:06,  9.36s/it]
  1%|▏         | 525/37962 [1:15:49<96:54:06,  9.32s/it]
  1%|▏         | 526/37962 [1:15:59<97:07:11,  9.34s/it]
  1%|▏         | 527/37962 [1:16:09<99:28:49,  9.57s/it]
  1%|▏         | 528/37962 [1:16:19<101:13:35,  9.73s/it]
  1%|▏         | 529/37962 [1:16:28<99:10:11,  9.54s/it] 
  1%|▏         | 530/37962 [1:16:36<95:13:44,  9.16s/it]
                                                        
{'loss': 1.6364, 'grad_norm': 56.987552642822266, 'learning_rate': 9.306409130816506e-06, 'epoch': 0.01}

  1%|▏         | 530/37962 [1:16:36<95:13:44,  9.16s/it]
  1%|▏         | 531/37962 [1:16:44<92:20:57,  8.88s/it]
  1%|▏         | 532/37962 [1:16:52<89:34:44,  8.62s/it]
  1%|▏         | 533/37962 [1:17:01<88:19:54,  8.50s/it]
  1%|▏         | 534/37962 [1:17:09<88:21:19,  8.50s/it]
  1%|▏         | 535/37962 [1:17:17<87:21:35,  8.40s/it]
  1%|▏         | 536/37962 [1:17:26<86:49:05,  8.35s/it]
  1%|▏         | 537/37962 [1:17:34<86:37:05,  8.33s/it]
  1%|▏         | 538/37962 [1:17:42<87:01:36,  8.37s/it]
  1%|▏         | 539/37962 [1:17:50<85:07:13,  8.19s/it]
  1%|▏         | 540/37962 [1:17:58<85:36:38,  8.24s/it]
                                                        
{'loss': 1.668, 'grad_norm': 63.16893768310547, 'learning_rate': 9.482001755926252e-06, 'epoch': 0.01}

  1%|▏         | 540/37962 [1:17:58<85:36:38,  8.24s/it]
  1%|▏         | 541/37962 [1:18:07<85:43:58,  8.25s/it]
  1%|▏         | 542/37962 [1:18:15<85:47:09,  8.25s/it]
  1%|▏         | 543/37962 [1:18:23<86:00:02,  8.27s/it]
  1%|▏         | 544/37962 [1:18:32<86:04:55,  8.28s/it]
  1%|▏         | 545/37962 [1:18:40<86:28:35,  8.32s/it]
  1%|▏         | 546/37962 [1:18:48<86:25:29,  8.32s/it]
  1%|▏         | 547/37962 [1:18:57<86:32:02,  8.33s/it]
  1%|▏         | 548/37962 [1:19:05<86:20:21,  8.31s/it]
  1%|▏         | 549/37962 [1:19:13<85:55:08,  8.27s/it]
  1%|▏         | 550/37962 [1:19:21<85:41:03,  8.25s/it]
                                                        
{'loss': 1.5511, 'grad_norm': 14.675129890441895, 'learning_rate': 9.657594381035997e-06, 'epoch': 0.01}

  1%|▏         | 550/37962 [1:19:21<85:41:03,  8.25s/it]
  1%|▏         | 551/37962 [1:19:30<85:37:30,  8.24s/it]
  1%|▏         | 552/37962 [1:19:38<85:48:03,  8.26s/it]
  1%|▏         | 553/37962 [1:19:46<85:53:05,  8.26s/it]
  1%|▏         | 554/37962 [1:19:54<85:39:03,  8.24s/it]
  1%|▏         | 555/37962 [1:20:03<86:42:39,  8.34s/it]
  1%|▏         | 556/37962 [1:20:11<86:34:55,  8.33s/it]
  1%|▏         | 557/37962 [1:20:19<86:14:45,  8.30s/it]
  1%|▏         | 558/37962 [1:20:28<86:18:47,  8.31s/it]
  1%|▏         | 559/37962 [1:20:36<86:17:13,  8.31s/it]
  1%|▏         | 560/37962 [1:20:45<87:52:46,  8.46s/it]
                                                        
{'loss': 1.5133, 'grad_norm': 16.1970157623291, 'learning_rate': 9.833187006145743e-06, 'epoch': 0.01}

  1%|▏         | 560/37962 [1:20:45<87:52:46,  8.46s/it]
  1%|▏         | 561/37962 [1:20:54<91:12:40,  8.78s/it]
  1%|▏         | 562/37962 [1:21:04<92:58:58,  8.95s/it]
  1%|▏         | 563/37962 [1:21:13<94:55:08,  9.14s/it]
  1%|▏         | 564/37962 [1:21:22<94:24:07,  9.09s/it]
  1%|▏         | 565/37962 [1:21:32<95:33:57,  9.20s/it]
  1%|▏         | 566/37962 [1:21:41<96:13:23,  9.26s/it]
  1%|▏         | 567/37962 [1:21:51<97:19:50,  9.37s/it]
  1%|▏         | 568/37962 [1:22:00<98:09:26,  9.45s/it]
  1%|▏         | 569/37962 [1:22:09<96:40:00,  9.31s/it]
  2%|▏         | 570/37962 [1:22:19<97:09:33,  9.35s/it]
                                                        
{'loss': 1.6573, 'grad_norm': 39.011173248291016, 'learning_rate': 1.0008779631255487e-05, 'epoch': 0.02}

  2%|▏         | 570/37962 [1:22:19<97:09:33,  9.35s/it]
  2%|▏         | 571/37962 [1:22:28<97:10:21,  9.36s/it]
  2%|▏         | 572/37962 [1:22:38<97:40:32,  9.40s/it]
  2%|▏         | 573/37962 [1:22:47<98:20:34,  9.47s/it]
  2%|▏         | 574/37962 [1:22:56<96:58:36,  9.34s/it]
  2%|▏         | 575/37962 [1:23:04<93:07:41,  8.97s/it]
  2%|▏         | 576/37962 [1:23:13<90:47:30,  8.74s/it]
  2%|▏         | 577/37962 [1:23:21<89:28:22,  8.62s/it]
  2%|▏         | 578/37962 [1:23:29<88:25:37,  8.52s/it]
  2%|▏         | 579/37962 [1:23:38<87:51:02,  8.46s/it]
  2%|▏         | 580/37962 [1:23:46<86:52:09,  8.37s/it]
                                                        
{'loss': 1.6567, 'grad_norm': 55.41944885253906, 'learning_rate': 1.0184372256365233e-05, 'epoch': 0.02}

  2%|▏         | 580/37962 [1:23:46<86:52:09,  8.37s/it]
  2%|▏         | 581/37962 [1:23:54<86:12:50,  8.30s/it]
  2%|▏         | 582/37962 [1:24:02<87:00:08,  8.38s/it]
  2%|▏         | 583/37962 [1:24:12<89:53:45,  8.66s/it]
  2%|▏         | 584/37962 [1:24:21<92:14:26,  8.88s/it]
  2%|▏         | 585/37962 [1:24:31<94:31:10,  9.10s/it]
  2%|▏         | 586/37962 [1:24:40<95:07:57,  9.16s/it]
  2%|▏         | 587/37962 [1:24:50<98:39:56,  9.50s/it]
  2%|▏         | 588/37962 [1:25:00<100:02:50,  9.64s/it]
  2%|▏         | 589/37962 [1:25:10<99:11:12,  9.55s/it] 
  2%|▏         | 590/37962 [1:25:19<97:53:35,  9.43s/it]
                                                        
{'loss': 1.5952, 'grad_norm': 29.878705978393555, 'learning_rate': 1.035996488147498e-05, 'epoch': 0.02}

  2%|▏         | 590/37962 [1:25:19<97:53:35,  9.43s/it]
  2%|▏         | 591/37962 [1:25:28<98:29:05,  9.49s/it]
  2%|▏         | 592/37962 [1:25:38<98:17:35,  9.47s/it]
  2%|▏         | 593/37962 [1:25:47<98:09:12,  9.46s/it]
  2%|▏         | 594/37962 [1:25:57<97:36:15,  9.40s/it]
  2%|▏         | 595/37962 [1:26:07<101:57:27,  9.82s/it]
  2%|▏         | 596/37962 [1:26:16<99:04:33,  9.55s/it] 
  2%|▏         | 597/37962 [1:26:25<95:07:08,  9.16s/it]
  2%|▏         | 598/37962 [1:26:33<92:49:55,  8.94s/it]
  2%|▏         | 599/37962 [1:26:41<91:06:09,  8.78s/it]
  2%|▏         | 600/37962 [1:26:49<88:48:31,  8.56s/it]
                                                        
{'loss': 1.5811, 'grad_norm': 40.10651397705078, 'learning_rate': 1.0535557506584724e-05, 'epoch': 0.02}

  2%|▏         | 600/37962 [1:26:49<88:48:31,  8.56s/it]
  2%|▏         | 601/37962 [1:26:58<87:41:58,  8.45s/it]
  2%|▏         | 602/37962 [1:27:06<86:29:19,  8.33s/it]
  2%|▏         | 603/37962 [1:27:14<86:08:13,  8.30s/it]
  2%|▏         | 604/37962 [1:27:22<86:04:59,  8.30s/it]
  2%|▏         | 605/37962 [1:27:30<85:58:35,  8.29s/it]
  2%|▏         | 606/37962 [1:27:39<85:57:02,  8.28s/it]
  2%|▏         | 607/37962 [1:27:47<86:24:56,  8.33s/it]
  2%|▏         | 608/37962 [1:27:55<86:01:39,  8.29s/it]
  2%|▏         | 609/37962 [1:28:04<86:08:44,  8.30s/it]
  2%|▏         | 610/37962 [1:28:13<88:44:03,  8.55s/it]
                                                        
{'loss': 1.6497, 'grad_norm': 31.494121551513672, 'learning_rate': 1.071115013169447e-05, 'epoch': 0.02}

  2%|▏         | 610/37962 [1:28:13<88:44:03,  8.55s/it]
  2%|▏         | 611/37962 [1:28:22<89:49:26,  8.66s/it]
  2%|▏         | 612/37962 [1:28:32<94:45:16,  9.13s/it]
  2%|▏         | 613/37962 [1:28:41<94:56:24,  9.15s/it]
  2%|▏         | 614/37962 [1:28:50<95:19:48,  9.19s/it]
  2%|▏         | 615/37962 [1:29:00<95:46:29,  9.23s/it]
  2%|▏         | 616/37962 [1:29:09<96:09:42,  9.27s/it]
  2%|▏         | 617/37962 [1:29:19<97:46:14,  9.42s/it]
  2%|▏         | 618/37962 [1:29:28<96:03:45,  9.26s/it]
  2%|▏         | 619/37962 [1:29:36<92:58:20,  8.96s/it]
  2%|▏         | 620/37962 [1:29:44<90:59:10,  8.77s/it]
                                                        
{'loss': 1.6076, 'grad_norm': 26.42911720275879, 'learning_rate': 1.0886742756804216e-05, 'epoch': 0.02}

  2%|▏         | 620/37962 [1:29:44<90:59:10,  8.77s/it]
  2%|▏         | 621/37962 [1:29:53<89:32:03,  8.63s/it]
  2%|▏         | 622/37962 [1:30:01<88:52:19,  8.57s/it]
  2%|▏         | 623/37962 [1:30:09<87:49:56,  8.47s/it]
  2%|▏         | 624/37962 [1:30:18<87:19:25,  8.42s/it]
  2%|▏         | 625/37962 [1:30:26<86:53:34,  8.38s/it]
  2%|▏         | 626/37962 [1:30:34<86:44:38,  8.36s/it]
  2%|▏         | 627/37962 [1:30:43<86:27:21,  8.34s/it]
  2%|▏         | 628/37962 [1:30:51<86:36:12,  8.35s/it]
  2%|▏         | 629/37962 [1:30:59<86:36:39,  8.35s/it]
  2%|▏         | 630/37962 [1:31:08<87:11:01,  8.41s/it]
                                                        
{'loss': 1.7983, 'grad_norm': 66.62787628173828, 'learning_rate': 1.106233538191396e-05, 'epoch': 0.02}

  2%|▏         | 630/37962 [1:31:08<87:11:01,  8.41s/it]
  2%|▏         | 631/37962 [1:31:16<86:00:32,  8.29s/it]
  2%|▏         | 632/37962 [1:31:24<85:58:14,  8.29s/it]
  2%|▏         | 633/37962 [1:31:32<85:31:36,  8.25s/it]
  2%|▏         | 634/37962 [1:31:41<85:34:02,  8.25s/it]
  2%|▏         | 635/37962 [1:31:49<85:30:46,  8.25s/it]
  2%|▏         | 636/37962 [1:31:57<85:33:41,  8.25s/it]
  2%|▏         | 637/37962 [1:32:06<86:35:59,  8.35s/it]
  2%|▏         | 638/37962 [1:32:14<86:05:00,  8.30s/it]
  2%|▏         | 639/37962 [1:32:22<86:20:14,  8.33s/it]
  2%|▏         | 640/37962 [1:32:30<86:08:42,  8.31s/it]
                                                        
{'loss': 1.5466, 'grad_norm': 64.07002258300781, 'learning_rate': 1.1237928007023707e-05, 'epoch': 0.02}

  2%|▏         | 640/37962 [1:32:30<86:08:42,  8.31s/it]
  2%|▏         | 641/37962 [1:32:39<86:03:10,  8.30s/it]
  2%|▏         | 642/37962 [1:32:47<85:54:26,  8.29s/it]
  2%|▏         | 643/37962 [1:32:55<85:36:32,  8.26s/it]
  2%|▏         | 644/37962 [1:33:04<85:53:47,  8.29s/it]
  2%|▏         | 645/37962 [1:33:12<86:54:52,  8.38s/it]
  2%|▏         | 646/37962 [1:33:21<86:56:07,  8.39s/it]
  2%|▏         | 647/37962 [1:33:29<86:28:42,  8.34s/it]
  2%|▏         | 648/37962 [1:33:37<86:13:07,  8.32s/it]
  2%|▏         | 649/37962 [1:33:45<85:49:07,  8.28s/it]
  2%|▏         | 650/37962 [1:33:54<86:01:15,  8.30s/it]
                                                        
{'loss': 1.5491, 'grad_norm': 16.801776885986328, 'learning_rate': 1.1413520632133453e-05, 'epoch': 0.02}

  2%|▏         | 650/37962 [1:33:54<86:01:15,  8.30s/it]
  2%|▏         | 651/37962 [1:34:02<86:01:42,  8.30s/it]
  2%|▏         | 652/37962 [1:34:10<85:51:06,  8.28s/it]
  2%|▏         | 653/37962 [1:34:18<85:07:37,  8.21s/it]
  2%|▏         | 654/37962 [1:34:27<85:26:07,  8.24s/it]
  2%|▏         | 655/37962 [1:34:35<85:49:13,  8.28s/it]
  2%|▏         | 656/37962 [1:34:43<85:26:47,  8.25s/it]
  2%|▏         | 657/37962 [1:34:51<85:05:27,  8.21s/it]
  2%|▏         | 658/37962 [1:35:00<85:47:23,  8.28s/it]
  2%|▏         | 659/37962 [1:35:08<86:19:10,  8.33s/it]
  2%|▏         | 660/37962 [1:35:16<86:02:07,  8.30s/it]
                                                        
{'loss': 1.6565, 'grad_norm': 13.05986499786377, 'learning_rate': 1.1589113257243196e-05, 'epoch': 0.02}

  2%|▏         | 660/37962 [1:35:16<86:02:07,  8.30s/it]
  2%|▏         | 661/37962 [1:35:24<85:08:41,  8.22s/it]
  2%|▏         | 662/37962 [1:35:33<85:22:31,  8.24s/it]
  2%|▏         | 663/37962 [1:35:41<85:59:43,  8.30s/it]
  2%|▏         | 664/37962 [1:35:49<86:10:35,  8.32s/it]
  2%|▏         | 665/37962 [1:35:58<86:41:40,  8.37s/it]
  2%|▏         | 666/37962 [1:36:06<87:20:44,  8.43s/it]
  2%|▏         | 667/37962 [1:36:15<87:06:13,  8.41s/it]
  2%|▏         | 668/37962 [1:36:23<86:00:36,  8.30s/it]
  2%|▏         | 669/37962 [1:36:31<85:57:59,  8.30s/it]
  2%|▏         | 670/37962 [1:36:39<85:43:42,  8.28s/it]
                                                        
{'loss': 1.654, 'grad_norm': 36.30992889404297, 'learning_rate': 1.1764705882352942e-05, 'epoch': 0.02}

  2%|▏         | 670/37962 [1:36:39<85:43:42,  8.28s/it]
  2%|▏         | 671/37962 [1:36:48<85:18:19,  8.24s/it]
  2%|▏         | 672/37962 [1:36:56<85:38:25,  8.27s/it]
  2%|▏         | 673/37962 [1:37:04<86:20:46,  8.34s/it]
  2%|▏         | 674/37962 [1:37:13<87:05:55,  8.41s/it]
  2%|▏         | 675/37962 [1:37:21<86:34:51,  8.36s/it]
  2%|▏         | 676/37962 [1:37:30<86:45:38,  8.38s/it]
  2%|▏         | 677/37962 [1:37:38<87:14:01,  8.42s/it]
  2%|▏         | 678/37962 [1:37:46<86:33:43,  8.36s/it]
  2%|▏         | 679/37962 [1:37:55<86:27:39,  8.35s/it]
  2%|▏         | 680/37962 [1:38:03<86:26:29,  8.35s/it]
                                                        
{'loss': 1.627, 'grad_norm': 6.151001930236816, 'learning_rate': 1.1940298507462686e-05, 'epoch': 0.02}

  2%|▏         | 680/37962 [1:38:03<86:26:29,  8.35s/it]
  2%|▏         | 681/37962 [1:38:11<86:08:06,  8.32s/it]
  2%|▏         | 682/37962 [1:38:20<85:54:21,  8.30s/it]
  2%|▏         | 683/37962 [1:38:28<85:11:25,  8.23s/it]
  2%|▏         | 684/37962 [1:38:36<85:26:42,  8.25s/it]
  2%|▏         | 685/37962 [1:38:44<85:34:07,  8.26s/it]
  2%|▏         | 686/37962 [1:38:52<85:27:06,  8.25s/it]
  2%|▏         | 687/37962 [1:39:01<85:30:58,  8.26s/it]
  2%|▏         | 688/37962 [1:39:09<86:00:34,  8.31s/it]
  2%|▏         | 689/37962 [1:39:18<86:22:34,  8.34s/it]
  2%|▏         | 690/37962 [1:39:26<86:06:45,  8.32s/it]
                                                        
{'loss': 1.7311, 'grad_norm': 59.942779541015625, 'learning_rate': 1.2115891132572432e-05, 'epoch': 0.02}

  2%|▏         | 690/37962 [1:39:26<86:06:45,  8.32s/it]
  2%|▏         | 691/37962 [1:39:34<86:24:10,  8.35s/it]
  2%|▏         | 692/37962 [1:39:43<86:38:36,  8.37s/it]
  2%|▏         | 693/37962 [1:39:51<86:18:36,  8.34s/it]
  2%|▏         | 694/37962 [1:39:59<85:49:55,  8.29s/it]
  2%|▏         | 695/37962 [1:40:07<85:52:05,  8.29s/it]
  2%|▏         | 696/37962 [1:40:16<85:54:04,  8.30s/it]
  2%|▏         | 697/37962 [1:40:24<85:10:22,  8.23s/it]
  2%|▏         | 698/37962 [1:40:32<85:06:19,  8.22s/it]
  2%|▏         | 699/37962 [1:40:40<85:28:25,  8.26s/it]
  2%|▏         | 700/37962 [1:40:49<85:44:21,  8.28s/it]
                                                        
{'loss': 1.7379, 'grad_norm': 59.08943176269531, 'learning_rate': 1.2291483757682179e-05, 'epoch': 0.02}

  2%|▏         | 700/37962 [1:40:49<85:44:21,  8.28s/it]
  2%|▏         | 701/37962 [1:40:57<85:57:17,  8.30s/it]
  2%|▏         | 702/37962 [1:41:05<86:17:38,  8.34s/it]
  2%|▏         | 703/37962 [1:41:14<86:37:44,  8.37s/it]
  2%|▏         | 704/37962 [1:41:22<86:27:11,  8.35s/it]
  2%|▏         | 705/37962 [1:41:31<87:14:08,  8.43s/it]
  2%|▏         | 706/37962 [1:41:39<86:30:04,  8.36s/it]
  2%|▏         | 707/37962 [1:41:47<86:07:40,  8.32s/it]
  2%|▏         | 708/37962 [1:41:55<85:59:33,  8.31s/it]
  2%|▏         | 709/37962 [1:42:04<86:03:40,  8.32s/it]
  2%|▏         | 710/37962 [1:42:12<86:01:36,  8.31s/it]
                                                        
{'loss': 1.5843, 'grad_norm': 54.9960823059082, 'learning_rate': 1.2467076382791923e-05, 'epoch': 0.02}

  2%|▏         | 710/37962 [1:42:12<86:01:36,  8.31s/it]
  2%|▏         | 711/37962 [1:42:20<85:53:49,  8.30s/it]
  2%|▏         | 712/37962 [1:42:28<83:54:49,  8.11s/it]
  2%|▏         | 713/37962 [1:42:37<85:32:52,  8.27s/it]
  2%|▏         | 714/37962 [1:42:45<85:32:12,  8.27s/it]
  2%|▏         | 715/37962 [1:42:53<86:12:49,  8.33s/it]
  2%|▏         | 716/37962 [1:43:02<86:07:29,  8.32s/it]
  2%|▏         | 717/37962 [1:43:10<85:52:33,  8.30s/it]
  2%|▏         | 718/37962 [1:43:18<86:10:22,  8.33s/it]
  2%|▏         | 719/37962 [1:43:27<86:08:49,  8.33s/it]
  2%|▏         | 720/37962 [1:43:35<86:01:08,  8.32s/it]
                                                        
{'loss': 1.6225, 'grad_norm': 13.231147766113281, 'learning_rate': 1.2642669007901669e-05, 'epoch': 0.02}

  2%|▏         | 720/37962 [1:43:35<86:01:08,  8.32s/it]
  2%|▏         | 721/37962 [1:43:43<85:28:15,  8.26s/it]
  2%|▏         | 722/37962 [1:43:52<85:45:34,  8.29s/it]
  2%|▏         | 723/37962 [1:44:00<85:59:02,  8.31s/it]
  2%|▏         | 724/37962 [1:44:08<86:49:07,  8.39s/it]
  2%|▏         | 725/37962 [1:44:17<86:53:46,  8.40s/it]
  2%|▏         | 726/37962 [1:44:25<86:10:44,  8.33s/it]
  2%|▏         | 727/37962 [1:44:33<86:02:02,  8.32s/it]
  2%|▏         | 728/37962 [1:44:42<85:44:13,  8.29s/it]
  2%|▏         | 729/37962 [1:44:50<85:55:24,  8.31s/it]
  2%|▏         | 730/37962 [1:44:58<86:18:18,  8.34s/it]
                                                        
{'loss': 1.6166, 'grad_norm': 46.019798278808594, 'learning_rate': 1.2818261633011415e-05, 'epoch': 0.02}

  2%|▏         | 730/37962 [1:44:58<86:18:18,  8.34s/it]
  2%|▏         | 731/37962 [1:45:07<86:38:54,  8.38s/it]
  2%|▏         | 732/37962 [1:45:15<86:21:29,  8.35s/it]
  2%|▏         | 733/37962 [1:45:23<86:21:00,  8.35s/it]
  2%|▏         | 734/37962 [1:45:32<86:00:21,  8.32s/it]
  2%|▏         | 735/37962 [1:45:40<86:08:24,  8.33s/it]
  2%|▏         | 736/37962 [1:45:48<85:43:29,  8.29s/it]
  2%|▏         | 737/37962 [1:45:57<85:47:56,  8.30s/it]
  2%|▏         | 738/37962 [1:46:05<85:55:02,  8.31s/it]
  2%|▏         | 739/37962 [1:46:13<86:01:31,  8.32s/it]
  2%|▏         | 740/37962 [1:46:22<86:43:23,  8.39s/it]
                                                        
{'loss': 1.6617, 'grad_norm': 43.63382339477539, 'learning_rate': 1.299385425812116e-05, 'epoch': 0.02}

  2%|▏         | 740/37962 [1:46:22<86:43:23,  8.39s/it]
  2%|▏         | 741/37962 [1:46:30<86:36:39,  8.38s/it]
  2%|▏         | 742/37962 [1:46:38<86:14:53,  8.34s/it]
  2%|▏         | 743/37962 [1:46:47<86:25:35,  8.36s/it]
  2%|▏         | 744/37962 [1:46:55<86:17:13,  8.35s/it]
  2%|▏         | 745/37962 [1:47:03<86:23:21,  8.36s/it]
  2%|▏         | 746/37962 [1:47:12<85:57:16,  8.31s/it]
  2%|▏         | 747/37962 [1:47:20<85:50:39,  8.30s/it]
  2%|▏         | 748/37962 [1:47:28<85:55:43,  8.31s/it]
  2%|▏         | 749/37962 [1:47:36<85:13:08,  8.24s/it]
  2%|▏         | 750/37962 [1:47:44<84:48:06,  8.20s/it]
                                                        
{'loss': 1.7987, 'grad_norm': 57.042537689208984, 'learning_rate': 1.3169446883230906e-05, 'epoch': 0.02}

  2%|▏         | 750/37962 [1:47:44<84:48:06,  8.20s/it]
  2%|▏         | 751/37962 [1:47:53<84:54:36,  8.21s/it]
  2%|▏         | 752/37962 [1:48:01<85:24:12,  8.26s/it]
  2%|▏         | 753/37962 [1:48:10<85:50:05,  8.30s/it]
  2%|▏         | 754/37962 [1:48:18<86:28:34,  8.37s/it]
  2%|▏         | 755/37962 [1:48:26<86:02:26,  8.32s/it]
  2%|▏         | 756/37962 [1:48:34<85:45:14,  8.30s/it]
  2%|▏         | 757/37962 [1:48:43<86:01:17,  8.32s/it]
  2%|▏         | 758/37962 [1:48:51<85:46:19,  8.30s/it]
  2%|▏         | 759/37962 [1:48:59<85:45:00,  8.30s/it]
  2%|▏         | 760/37962 [1:49:08<85:51:09,  8.31s/it]
                                                        
{'loss': 1.6918, 'grad_norm': 51.18360137939453, 'learning_rate': 1.3345039508340652e-05, 'epoch': 0.02}

  2%|▏         | 760/37962 [1:49:08<85:51:09,  8.31s/it]
  2%|▏         | 761/37962 [1:49:16<85:35:14,  8.28s/it]
  2%|▏         | 762/37962 [1:49:24<84:45:36,  8.20s/it]
  2%|▏         | 763/37962 [1:49:33<85:53:57,  8.31s/it]
  2%|▏         | 764/37962 [1:49:41<85:39:00,  8.29s/it]
  2%|▏         | 765/37962 [1:49:49<85:43:22,  8.30s/it]
  2%|▏         | 766/37962 [1:49:57<85:51:05,  8.31s/it]
  2%|▏         | 767/37962 [1:50:06<86:05:47,  8.33s/it]
  2%|▏         | 768/37962 [1:50:14<86:31:09,  8.37s/it]
  2%|▏         | 769/37962 [1:50:23<86:35:08,  8.38s/it]
  2%|▏         | 770/37962 [1:50:31<86:43:31,  8.39s/it]
                                                        
{'loss': 1.6811, 'grad_norm': 48.76926803588867, 'learning_rate': 1.3520632133450395e-05, 'epoch': 0.02}

  2%|▏         | 770/37962 [1:50:31<86:43:31,  8.39s/it]
  2%|▏         | 771/37962 [1:50:39<86:26:27,  8.37s/it]
  2%|▏         | 772/37962 [1:50:48<86:03:16,  8.33s/it]
  2%|▏         | 773/37962 [1:50:56<86:20:32,  8.36s/it]
  2%|▏         | 774/37962 [1:51:04<85:44:05,  8.30s/it]
  2%|▏         | 775/37962 [1:51:13<85:58:57,  8.32s/it]
  2%|▏         | 776/37962 [1:51:21<85:45:11,  8.30s/it]
  2%|▏         | 777/37962 [1:51:29<85:12:33,  8.25s/it]
  2%|▏         | 778/37962 [1:51:37<85:24:12,  8.27s/it]
  2%|▏         | 779/37962 [1:51:46<86:00:02,  8.33s/it]
  2%|▏         | 780/37962 [1:51:54<85:51:49,  8.31s/it]
                                                        
{'loss': 1.6549, 'grad_norm': 43.75407409667969, 'learning_rate': 1.369622475856014e-05, 'epoch': 0.02}

  2%|▏         | 780/37962 [1:51:54<85:51:49,  8.31s/it]
  2%|▏         | 781/37962 [1:52:02<86:15:01,  8.35s/it]
  2%|▏         | 782/37962 [1:52:11<85:49:57,  8.31s/it]
  2%|▏         | 783/37962 [1:52:19<85:49:37,  8.31s/it]
  2%|▏         | 784/37962 [1:52:27<85:33:32,  8.28s/it]
  2%|▏         | 785/37962 [1:52:36<85:41:49,  8.30s/it]
  2%|▏         | 786/37962 [1:52:44<86:05:31,  8.34s/it]
  2%|▏         | 787/37962 [1:52:52<85:54:19,  8.32s/it]
  2%|▏         | 788/37962 [1:53:00<84:58:30,  8.23s/it]
  2%|▏         | 789/37962 [1:53:09<85:29:33,  8.28s/it]
  2%|▏         | 790/37962 [1:53:17<85:42:40,  8.30s/it]
                                                        
{'loss': 1.6094, 'grad_norm': 55.638633728027344, 'learning_rate': 1.3871817383669887e-05, 'epoch': 0.02}

  2%|▏         | 790/37962 [1:53:17<85:42:40,  8.30s/it]
  2%|▏         | 791/37962 [1:53:25<85:13:56,  8.25s/it]
  2%|▏         | 792/37962 [1:53:34<86:05:43,  8.34s/it]
  2%|▏         | 793/37962 [1:53:42<85:35:35,  8.29s/it]
  2%|▏         | 794/37962 [1:53:50<85:34:07,  8.29s/it]
  2%|▏         | 795/37962 [1:53:58<85:10:04,  8.25s/it]
  2%|▏         | 796/37962 [1:54:07<85:29:20,  8.28s/it]
  2%|▏         | 797/37962 [1:54:15<84:33:22,  8.19s/it]
  2%|▏         | 798/37962 [1:54:23<84:58:54,  8.23s/it]
  2%|▏         | 799/37962 [1:54:31<85:38:01,  8.30s/it]
  2%|▏         | 800/37962 [1:54:40<86:38:06,  8.39s/it]
                                                        
{'loss': 1.9367, 'grad_norm': 64.54676055908203, 'learning_rate': 1.4047410008779631e-05, 'epoch': 0.02}

  2%|▏         | 800/37962 [1:54:40<86:38:06,  8.39s/it]
  2%|▏         | 801/37962 [1:54:48<86:20:18,  8.36s/it]
  2%|▏         | 802/37962 [1:54:57<86:22:30,  8.37s/it]
  2%|▏         | 803/37962 [1:55:05<86:04:06,  8.34s/it]
  2%|▏         | 804/37962 [1:55:13<85:33:54,  8.29s/it]
  2%|▏         | 805/37962 [1:55:21<84:54:13,  8.23s/it]
  2%|▏         | 806/37962 [1:55:30<85:28:32,  8.28s/it]
  2%|▏         | 807/37962 [1:55:38<86:06:12,  8.34s/it]
  2%|▏         | 808/37962 [1:55:46<85:43:56,  8.31s/it]
  2%|▏         | 809/37962 [1:55:55<85:39:26,  8.30s/it]
  2%|▏         | 810/37962 [1:56:03<85:40:48,  8.30s/it]
                                                        
{'loss': 1.6365, 'grad_norm': 61.4254150390625, 'learning_rate': 1.4223002633889378e-05, 'epoch': 0.02}

  2%|▏         | 810/37962 [1:56:03<85:40:48,  8.30s/it]
  2%|▏         | 811/37962 [1:56:11<85:40:21,  8.30s/it]
  2%|▏         | 812/37962 [1:56:19<85:15:32,  8.26s/it]
  2%|▏         | 813/37962 [1:56:28<85:15:57,  8.26s/it]
  2%|▏         | 814/37962 [1:56:36<84:55:03,  8.23s/it]
  2%|▏         | 815/37962 [1:56:44<85:06:28,  8.25s/it]
  2%|▏         | 816/37962 [1:56:52<84:50:35,  8.22s/it]
  2%|▏         | 817/37962 [1:57:00<84:08:20,  8.15s/it]
  2%|▏         | 818/37962 [1:57:08<82:42:32,  8.02s/it]
  2%|▏         | 819/37962 [1:57:16<83:25:03,  8.09s/it]
  2%|▏         | 820/37962 [1:57:25<84:10:33,  8.16s/it]
                                                        
{'loss': 1.5813, 'grad_norm': 19.859241485595703, 'learning_rate': 1.4398595258999124e-05, 'epoch': 0.02}

  2%|▏         | 820/37962 [1:57:25<84:10:33,  8.16s/it]
  2%|▏         | 821/37962 [1:57:33<84:05:58,  8.15s/it]
  2%|▏         | 822/37962 [1:57:41<84:42:05,  8.21s/it]
  2%|▏         | 823/37962 [1:57:49<84:50:22,  8.22s/it]
  2%|▏         | 824/37962 [1:57:57<84:09:03,  8.16s/it]
  2%|▏         | 825/37962 [1:58:05<83:39:37,  8.11s/it]
  2%|▏         | 826/37962 [1:58:14<84:24:05,  8.18s/it]
  2%|▏         | 827/37962 [1:58:22<84:48:08,  8.22s/it]
  2%|▏         | 828/37962 [1:58:30<83:21:03,  8.08s/it]
  2%|▏         | 829/37962 [1:58:38<84:04:44,  8.15s/it]
  2%|▏         | 830/37962 [1:58:46<84:54:23,  8.23s/it]
                                                        
{'loss': 1.759, 'grad_norm': 83.693603515625, 'learning_rate': 1.4574187884108868e-05, 'epoch': 0.02}

  2%|▏         | 830/37962 [1:58:46<84:54:23,  8.23s/it]
  2%|▏         | 831/37962 [1:58:55<84:39:37,  8.21s/it]
  2%|▏         | 832/37962 [1:59:03<85:35:08,  8.30s/it]
  2%|▏         | 833/37962 [1:59:11<85:39:45,  8.31s/it]
  2%|▏         | 834/37962 [1:59:20<85:39:52,  8.31s/it]
  2%|▏         | 835/37962 [1:59:28<86:13:00,  8.36s/it]
  2%|▏         | 836/37962 [1:59:36<85:40:04,  8.31s/it]
  2%|▏         | 837/37962 [1:59:45<86:09:49,  8.36s/it]
  2%|▏         | 838/37962 [1:59:53<86:06:49,  8.35s/it]
  2%|▏         | 839/37962 [2:00:02<86:03:34,  8.35s/it]
  2%|▏         | 840/37962 [2:00:10<85:53:02,  8.33s/it]
                                                        
{'loss': 1.6277, 'grad_norm': 35.15703582763672, 'learning_rate': 1.4749780509218614e-05, 'epoch': 0.02}

  2%|▏         | 840/37962 [2:00:10<85:53:02,  8.33s/it]
  2%|▏         | 841/37962 [2:00:18<85:51:05,  8.33s/it]
  2%|▏         | 842/37962 [2:00:26<85:46:50,  8.32s/it]
  2%|▏         | 843/37962 [2:00:35<85:55:25,  8.33s/it]
  2%|▏         | 844/37962 [2:00:43<86:08:36,  8.35s/it]
  2%|▏         | 845/37962 [2:00:51<85:32:47,  8.30s/it]
  2%|▏         | 846/37962 [2:01:00<85:59:26,  8.34s/it]
  2%|▏         | 847/37962 [2:01:08<85:14:06,  8.27s/it]
  2%|▏         | 848/37962 [2:01:16<84:22:15,  8.18s/it]
  2%|▏         | 849/37962 [2:01:24<83:08:48,  8.07s/it]
  2%|▏         | 850/37962 [2:01:32<84:18:10,  8.18s/it]
                                                        
{'loss': 1.5686, 'grad_norm': 52.248008728027344, 'learning_rate': 1.492537313432836e-05, 'epoch': 0.02}

  2%|▏         | 850/37962 [2:01:32<84:18:10,  8.18s/it]
  2%|▏         | 851/37962 [2:01:40<84:37:13,  8.21s/it]
  2%|▏         | 852/37962 [2:01:49<85:12:33,  8.27s/it]
  2%|▏         | 853/37962 [2:01:57<84:19:30,  8.18s/it]
  2%|▏         | 854/37962 [2:02:05<84:30:00,  8.20s/it]
  2%|▏         | 855/37962 [2:02:13<84:51:48,  8.23s/it]
  2%|▏         | 856/37962 [2:02:22<85:06:25,  8.26s/it]
  2%|▏         | 857/37962 [2:02:30<85:15:56,  8.27s/it]
  2%|▏         | 858/37962 [2:02:38<85:17:26,  8.28s/it]
  2%|▏         | 859/37962 [2:02:47<85:25:37,  8.29s/it]
  2%|▏         | 860/37962 [2:02:55<84:28:40,  8.20s/it]
                                                        
{'loss': 1.5431, 'grad_norm': 21.968286514282227, 'learning_rate': 1.5100965759438105e-05, 'epoch': 0.02}

  2%|▏         | 860/37962 [2:02:55<84:28:40,  8.20s/it]
  2%|▏         | 861/37962 [2:03:03<84:47:21,  8.23s/it]
  2%|▏         | 862/37962 [2:03:11<85:33:14,  8.30s/it]
  2%|▏         | 863/37962 [2:03:20<85:03:57,  8.25s/it]
  2%|▏         | 864/37962 [2:03:28<85:34:21,  8.30s/it]
  2%|▏         | 865/37962 [2:03:36<85:48:20,  8.33s/it]
  2%|▏         | 866/37962 [2:03:45<85:24:35,  8.29s/it]
  2%|▏         | 867/37962 [2:03:53<85:18:45,  8.28s/it]
  2%|▏         | 868/37962 [2:04:01<85:20:39,  8.28s/it]
  2%|▏         | 869/37962 [2:04:09<84:35:52,  8.21s/it]
  2%|▏         | 870/37962 [2:04:17<84:39:08,  8.22s/it]
                                                        
{'loss': 1.6102, 'grad_norm': 76.19892120361328, 'learning_rate': 1.527655838454785e-05, 'epoch': 0.02}

  2%|▏         | 870/37962 [2:04:17<84:39:08,  8.22s/it]
  2%|▏         | 871/37962 [2:04:26<84:57:10,  8.25s/it]
  2%|▏         | 872/37962 [2:04:34<85:26:49,  8.29s/it]
  2%|▏         | 873/37962 [2:04:42<85:25:02,  8.29s/it]
  2%|▏         | 874/37962 [2:04:51<85:22:19,  8.29s/it]
  2%|▏         | 875/37962 [2:04:59<85:12:37,  8.27s/it]
  2%|▏         | 876/37962 [2:05:07<85:40:14,  8.32s/it]
  2%|▏         | 877/37962 [2:05:15<84:24:34,  8.19s/it]
  2%|▏         | 878/37962 [2:05:23<84:41:24,  8.22s/it]
  2%|▏         | 879/37962 [2:05:32<84:25:58,  8.20s/it]
  2%|▏         | 880/37962 [2:05:40<84:48:29,  8.23s/it]
                                                        
{'loss': 1.6927, 'grad_norm': 22.568992614746094, 'learning_rate': 1.5452151009657595e-05, 'epoch': 0.02}

  2%|▏         | 880/37962 [2:05:40<84:48:29,  8.23s/it]
  2%|▏         | 881/37962 [2:05:48<85:08:35,  8.27s/it]
  2%|▏         | 882/37962 [2:05:57<85:23:29,  8.29s/it]
  2%|▏         | 883/37962 [2:06:05<85:20:50,  8.29s/it]
  2%|▏         | 884/37962 [2:06:13<85:28:42,  8.30s/it]
  2%|▏         | 885/37962 [2:06:22<85:38:05,  8.31s/it]
  2%|▏         | 886/37962 [2:06:30<85:18:58,  8.28s/it]
  2%|▏         | 887/37962 [2:06:38<86:05:57,  8.36s/it]
  2%|▏         | 888/37962 [2:06:47<86:56:43,  8.44s/it]
  2%|▏         | 889/37962 [2:06:55<85:55:53,  8.34s/it]
  2%|▏         | 890/37962 [2:07:03<85:53:59,  8.34s/it]
                                                        
{'loss': 1.6507, 'grad_norm': 62.15504837036133, 'learning_rate': 1.562774363476734e-05, 'epoch': 0.02}

  2%|▏         | 890/37962 [2:07:03<85:53:59,  8.34s/it]
  2%|▏         | 891/37962 [2:07:12<86:02:41,  8.36s/it]
  2%|▏         | 892/37962 [2:07:20<84:18:55,  8.19s/it]
  2%|▏         | 893/37962 [2:07:28<84:38:48,  8.22s/it]
  2%|▏         | 894/37962 [2:07:36<84:50:55,  8.24s/it]
  2%|▏         | 895/37962 [2:07:44<84:23:10,  8.20s/it]
  2%|▏         | 896/37962 [2:07:53<84:35:15,  8.22s/it]
  2%|▏         | 897/37962 [2:08:00<83:44:32,  8.13s/it]
  2%|▏         | 898/37962 [2:08:09<84:06:26,  8.17s/it]
  2%|▏         | 899/37962 [2:08:17<84:40:16,  8.22s/it]
  2%|▏         | 900/37962 [2:08:26<85:32:48,  8.31s/it]
                                                        
{'loss': 1.5483, 'grad_norm': 25.25591278076172, 'learning_rate': 1.5803336259877088e-05, 'epoch': 0.02}

  2%|▏         | 900/37962 [2:08:26<85:32:48,  8.31s/it]
  2%|▏         | 901/37962 [2:08:34<84:46:06,  8.23s/it]
  2%|▏         | 902/37962 [2:08:42<84:52:01,  8.24s/it]
  2%|▏         | 903/37962 [2:08:50<84:09:15,  8.17s/it]
  2%|▏         | 904/37962 [2:08:58<84:34:29,  8.22s/it]
  2%|▏         | 905/37962 [2:09:06<83:39:13,  8.13s/it]
  2%|▏         | 906/37962 [2:09:14<84:15:32,  8.19s/it]
  2%|▏         | 907/37962 [2:09:22<83:40:47,  8.13s/it]
  2%|▏         | 908/37962 [2:09:31<84:37:13,  8.22s/it]
  2%|▏         | 909/37962 [2:09:39<83:38:44,  8.13s/it]
  2%|▏         | 910/37962 [2:09:47<84:06:40,  8.17s/it]
                                                        
{'loss': 1.6073, 'grad_norm': 10.233015060424805, 'learning_rate': 1.5978928884986834e-05, 'epoch': 0.02}

  2%|▏         | 910/37962 [2:09:47<84:06:40,  8.17s/it]
  2%|▏         | 911/37962 [2:09:55<83:39:31,  8.13s/it]
  2%|▏         | 912/37962 [2:10:03<82:52:54,  8.05s/it]
  2%|▏         | 913/37962 [2:10:11<84:01:42,  8.16s/it]
  2%|▏         | 914/37962 [2:10:20<84:51:04,  8.25s/it]
  2%|▏         | 915/37962 [2:10:28<84:34:37,  8.22s/it]
  2%|▏         | 916/37962 [2:10:36<85:13:00,  8.28s/it]
  2%|▏         | 917/37962 [2:10:44<84:15:57,  8.19s/it]
  2%|▏         | 918/37962 [2:10:53<85:00:53,  8.26s/it]
  2%|▏         | 919/37962 [2:11:01<85:02:43,  8.27s/it]
  2%|▏         | 920/37962 [2:11:09<85:07:20,  8.27s/it]
                                                        
{'loss': 1.7568, 'grad_norm': 38.76236343383789, 'learning_rate': 1.6154521510096577e-05, 'epoch': 0.02}

  2%|▏         | 920/37962 [2:11:09<85:07:20,  8.27s/it]
  2%|▏         | 921/37962 [2:11:18<85:10:20,  8.28s/it]
  2%|▏         | 922/37962 [2:11:26<84:44:55,  8.24s/it]
  2%|▏         | 923/37962 [2:11:34<85:12:01,  8.28s/it]
  2%|▏         | 924/37962 [2:11:42<84:52:27,  8.25s/it]
  2%|▏         | 925/37962 [2:11:51<84:41:25,  8.23s/it]
  2%|▏         | 926/37962 [2:11:59<83:56:42,  8.16s/it]
  2%|▏         | 927/37962 [2:12:07<84:21:38,  8.20s/it]
  2%|▏         | 928/37962 [2:12:15<83:42:21,  8.14s/it]
  2%|▏         | 929/37962 [2:12:23<83:57:17,  8.16s/it]
  2%|▏         | 930/37962 [2:12:31<84:23:40,  8.20s/it]
                                                        
{'loss': 1.6173, 'grad_norm': 30.897741317749023, 'learning_rate': 1.6330114135206323e-05, 'epoch': 0.02}

  2%|▏         | 930/37962 [2:12:31<84:23:40,  8.20s/it]
  2%|▏         | 931/37962 [2:12:39<83:58:59,  8.16s/it]
  2%|▏         | 932/37962 [2:12:48<83:40:31,  8.13s/it]
  2%|▏         | 933/37962 [2:12:56<84:16:57,  8.19s/it]
  2%|▏         | 934/37962 [2:13:04<84:33:03,  8.22s/it]
  2%|▏         | 935/37962 [2:13:12<84:34:05,  8.22s/it]
  2%|▏         | 936/37962 [2:13:21<84:46:58,  8.24s/it]
  2%|▏         | 937/37962 [2:13:29<84:51:51,  8.25s/it]
  2%|▏         | 938/37962 [2:13:37<85:07:05,  8.28s/it]
  2%|▏         | 939/37962 [2:13:46<85:33:11,  8.32s/it]
  2%|▏         | 940/37962 [2:13:54<85:24:26,  8.30s/it]
                                                        
{'loss': 1.5767, 'grad_norm': 37.78947830200195, 'learning_rate': 1.6505706760316065e-05, 'epoch': 0.02}

  2%|▏         | 940/37962 [2:13:54<85:24:26,  8.30s/it]
  2%|▏         | 941/37962 [2:14:02<85:21:59,  8.30s/it]
  2%|▏         | 942/37962 [2:14:11<85:15:17,  8.29s/it]
  2%|▏         | 943/37962 [2:14:19<85:06:35,  8.28s/it]
  2%|▏         | 944/37962 [2:14:27<85:10:29,  8.28s/it]
  2%|▏         | 945/37962 [2:14:35<85:14:09,  8.29s/it]
  2%|▏         | 946/37962 [2:14:44<85:13:59,  8.29s/it]
  2%|▏         | 947/37962 [2:14:52<84:43:58,  8.24s/it]
  2%|▏         | 948/37962 [2:15:00<84:46:02,  8.24s/it]
  2%|▏         | 949/37962 [2:15:08<85:02:05,  8.27s/it]
  3%|▎         | 950/37962 [2:15:16<84:22:14,  8.21s/it]
                                                        
{'loss': 1.4724, 'grad_norm': 30.57618522644043, 'learning_rate': 1.668129938542581e-05, 'epoch': 0.03}

  3%|▎         | 950/37962 [2:15:16<84:22:14,  8.21s/it]
  3%|▎         | 951/37962 [2:15:25<84:26:38,  8.21s/it]
  3%|▎         | 952/37962 [2:15:33<84:35:33,  8.23s/it]
  3%|▎         | 953/37962 [2:15:41<84:21:11,  8.21s/it]
  3%|▎         | 954/37962 [2:15:49<84:29:55,  8.22s/it]
  3%|▎         | 955/37962 [2:15:57<83:57:44,  8.17s/it]
  3%|▎         | 956/37962 [2:16:06<84:21:49,  8.21s/it]
  3%|▎         | 957/37962 [2:16:14<84:20:12,  8.20s/it]
  3%|▎         | 958/37962 [2:16:22<84:40:36,  8.24s/it]
  3%|▎         | 959/37962 [2:16:31<85:02:41,  8.27s/it]
  3%|▎         | 960/37962 [2:16:39<85:29:54,  8.32s/it]
                                                        
{'loss': 1.612, 'grad_norm': 27.889238357543945, 'learning_rate': 1.6856892010535558e-05, 'epoch': 0.03}

  3%|▎         | 960/37962 [2:16:39<85:29:54,  8.32s/it]
  3%|▎         | 961/37962 [2:16:47<85:15:51,  8.30s/it]
  3%|▎         | 962/37962 [2:16:55<84:17:56,  8.20s/it]
  3%|▎         | 963/37962 [2:17:03<84:05:55,  8.18s/it]
  3%|▎         | 964/37962 [2:17:12<84:26:40,  8.22s/it]
  3%|▎         | 965/37962 [2:17:20<84:20:42,  8.21s/it]
  3%|▎         | 966/37962 [2:17:28<84:30:03,  8.22s/it]
  3%|▎         | 967/37962 [2:17:36<84:48:54,  8.25s/it]
  3%|▎         | 968/37962 [2:17:45<84:55:22,  8.26s/it]
  3%|▎         | 969/37962 [2:17:53<84:49:41,  8.26s/it]
  3%|▎         | 970/37962 [2:18:01<85:25:28,  8.31s/it]
                                                        
{'loss': 1.483, 'grad_norm': 9.004977226257324, 'learning_rate': 1.7032484635645304e-05, 'epoch': 0.03}

  3%|▎         | 970/37962 [2:18:01<85:25:28,  8.31s/it]
  3%|▎         | 971/37962 [2:18:10<85:05:56,  8.28s/it]
  3%|▎         | 972/37962 [2:18:18<85:29:34,  8.32s/it]
  3%|▎         | 973/37962 [2:18:26<85:08:12,  8.29s/it]
  3%|▎         | 974/37962 [2:18:34<85:04:26,  8.28s/it]
  3%|▎         | 975/37962 [2:18:43<85:24:06,  8.31s/it]
  3%|▎         | 976/37962 [2:18:51<84:56:41,  8.27s/it]
  3%|▎         | 977/37962 [2:18:59<84:54:26,  8.26s/it]
  3%|▎         | 978/37962 [2:19:08<85:18:11,  8.30s/it]
  3%|▎         | 979/37962 [2:19:16<85:06:45,  8.29s/it]
  3%|▎         | 980/37962 [2:19:24<85:16:21,  8.30s/it]
                                                        
{'loss': 1.7255, 'grad_norm': 28.223093032836914, 'learning_rate': 1.720807726075505e-05, 'epoch': 0.03}

  3%|▎         | 980/37962 [2:19:24<85:16:21,  8.30s/it]
  3%|▎         | 981/37962 [2:19:33<85:08:28,  8.29s/it]
  3%|▎         | 982/37962 [2:19:41<85:02:35,  8.28s/it]
  3%|▎         | 983/37962 [2:19:49<85:09:08,  8.29s/it]
  3%|▎         | 984/37962 [2:19:57<84:58:22,  8.27s/it]
  3%|▎         | 985/37962 [2:20:06<85:09:40,  8.29s/it]
  3%|▎         | 986/37962 [2:20:14<85:32:55,  8.33s/it]
  3%|▎         | 987/37962 [2:20:22<85:38:39,  8.34s/it]
  3%|▎         | 988/37962 [2:20:31<85:27:14,  8.32s/it]
  3%|▎         | 989/37962 [2:20:38<83:43:11,  8.15s/it]
  3%|▎         | 990/37962 [2:20:47<84:16:47,  8.21s/it]
                                                        
{'loss': 1.562, 'grad_norm': 20.885543823242188, 'learning_rate': 1.7383669885864796e-05, 'epoch': 0.03}

  3%|▎         | 990/37962 [2:20:47<84:16:47,  8.21s/it]
  3%|▎         | 991/37962 [2:20:55<84:29:09,  8.23s/it]
  3%|▎         | 992/37962 [2:21:04<85:07:22,  8.29s/it]
  3%|▎         | 993/37962 [2:21:12<85:46:35,  8.35s/it]
  3%|▎         | 994/37962 [2:21:20<85:16:42,  8.30s/it]
  3%|▎         | 995/37962 [2:21:28<85:00:28,  8.28s/it]
  3%|▎         | 996/37962 [2:21:37<85:26:30,  8.32s/it]
  3%|▎         | 997/37962 [2:21:45<85:10:53,  8.30s/it]
  3%|▎         | 998/37962 [2:21:53<84:58:12,  8.28s/it]
  3%|▎         | 999/37962 [2:22:02<85:08:05,  8.29s/it]
  3%|▎         | 1000/37962 [2:22:10<85:22:19,  8.32s/it]
                                                         
{'loss': 1.6316, 'grad_norm': 26.906631469726562, 'learning_rate': 1.755926251097454e-05, 'epoch': 0.03}

  3%|▎         | 1000/37962 [2:22:10<85:22:19,  8.32s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

  3%|▎         | 1001/37962 [2:22:59<209:37:43, 20.42s/it]
  3%|▎         | 1002/37962 [2:23:07<172:20:28, 16.79s/it]
  3%|▎         | 1003/37962 [2:23:15<146:02:49, 14.23s/it]
  3%|▎         | 1004/37962 [2:23:24<128:29:24, 12.52s/it]
  3%|▎         | 1005/37962 [2:23:32<115:36:56, 11.26s/it]
  3%|▎         | 1006/37962 [2:23:40<106:20:59, 10.36s/it]
  3%|▎         | 1007/37962 [2:23:48<99:10:39,  9.66s/it] 
  3%|▎         | 1008/37962 [2:23:57<94:43:07,  9.23s/it]
  3%|▎         | 1009/37962 [2:24:05<91:47:24,  8.94s/it]
  3%|▎         | 1010/37962 [2:24:13<88:59:24,  8.67s/it]
                                                         
{'loss': 1.5701, 'grad_norm': 7.186174392700195, 'learning_rate': 1.7734855136084285e-05, 'epoch': 0.03}

  3%|▎         | 1010/37962 [2:24:13<88:59:24,  8.67s/it]
  3%|▎         | 1011/37962 [2:24:21<87:32:52,  8.53s/it]
  3%|▎         | 1012/37962 [2:24:29<86:34:12,  8.43s/it]
  3%|▎         | 1013/37962 [2:24:38<86:40:21,  8.44s/it]
  3%|▎         | 1014/37962 [2:24:47<87:45:08,  8.55s/it]
  3%|▎         | 1015/37962 [2:24:55<87:55:13,  8.57s/it]
  3%|▎         | 1016/37962 [2:25:04<88:46:07,  8.65s/it]
  3%|▎         | 1017/37962 [2:25:12<87:55:23,  8.57s/it]
  3%|▎         | 1018/37962 [2:25:21<87:23:39,  8.52s/it]
  3%|▎         | 1019/37962 [2:25:29<86:34:55,  8.44s/it]
  3%|▎         | 1020/37962 [2:25:37<86:03:37,  8.39s/it]
                                                         
{'loss': 1.7596, 'grad_norm': 17.23870086669922, 'learning_rate': 1.791044776119403e-05, 'epoch': 0.03}

  3%|▎         | 1020/37962 [2:25:37<86:03:37,  8.39s/it]
  3%|▎         | 1021/37962 [2:25:46<85:54:04,  8.37s/it]
  3%|▎         | 1022/37962 [2:25:54<85:50:51,  8.37s/it]
  3%|▎         | 1023/37962 [2:26:02<85:24:13,  8.32s/it]
  3%|▎         | 1024/37962 [2:26:10<84:12:33,  8.21s/it]
  3%|▎         | 1025/37962 [2:26:18<84:15:42,  8.21s/it]
  3%|▎         | 1026/37962 [2:26:27<84:44:39,  8.26s/it]
  3%|▎         | 1027/37962 [2:26:35<85:08:02,  8.30s/it]
  3%|▎         | 1028/37962 [2:26:43<83:51:41,  8.17s/it]
  3%|▎         | 1029/37962 [2:26:51<84:27:08,  8.23s/it]
  3%|▎         | 1030/37962 [2:27:00<85:00:06,  8.29s/it]
                                                         
{'loss': 1.6703, 'grad_norm': 17.744979858398438, 'learning_rate': 1.8086040386303777e-05, 'epoch': 0.03}

  3%|▎         | 1030/37962 [2:27:00<85:00:06,  8.29s/it]
  3%|▎         | 1031/37962 [2:27:08<84:53:31,  8.28s/it]
  3%|▎         | 1032/37962 [2:27:16<85:20:56,  8.32s/it]
  3%|▎         | 1033/37962 [2:27:25<84:52:20,  8.27s/it]
  3%|▎         | 1034/37962 [2:27:33<84:57:33,  8.28s/it]
  3%|▎         | 1035/37962 [2:27:41<85:37:35,  8.35s/it]
  3%|▎         | 1036/37962 [2:27:50<85:03:58,  8.29s/it]
  3%|▎         | 1037/37962 [2:27:58<85:17:35,  8.32s/it]
  3%|▎         | 1038/37962 [2:28:06<84:06:07,  8.20s/it]
  3%|▎         | 1039/37962 [2:28:14<84:28:32,  8.24s/it]
  3%|▎         | 1040/37962 [2:28:22<84:21:57,  8.23s/it]
                                                         
{'loss': 1.7382, 'grad_norm': 7.944180488586426, 'learning_rate': 1.8261633011413523e-05, 'epoch': 0.03}

  3%|▎         | 1040/37962 [2:28:22<84:21:57,  8.23s/it]
  3%|▎         | 1041/37962 [2:28:31<84:28:40,  8.24s/it]
  3%|▎         | 1042/37962 [2:28:39<84:37:02,  8.25s/it]
  3%|▎         | 1043/37962 [2:28:48<85:40:04,  8.35s/it]
  3%|▎         | 1044/37962 [2:28:56<85:21:23,  8.32s/it]
  3%|▎         | 1045/37962 [2:29:04<84:38:17,  8.25s/it]
  3%|▎         | 1046/37962 [2:29:12<83:45:44,  8.17s/it]
  3%|▎         | 1047/37962 [2:29:20<83:51:28,  8.18s/it]
  3%|▎         | 1048/37962 [2:29:28<84:13:49,  8.21s/it]
  3%|▎         | 1049/37962 [2:29:37<84:24:26,  8.23s/it]
  3%|▎         | 1050/37962 [2:29:45<84:37:57,  8.25s/it]
                                                         
{'loss': 1.6142, 'grad_norm': 17.63950538635254, 'learning_rate': 1.843722563652327e-05, 'epoch': 0.03}

  3%|▎         | 1050/37962 [2:29:45<84:37:57,  8.25s/it]
  3%|▎         | 1051/37962 [2:29:53<84:57:37,  8.29s/it]
  3%|▎         | 1052/37962 [2:30:02<85:40:07,  8.36s/it]
  3%|▎         | 1053/37962 [2:30:10<86:26:10,  8.43s/it]
  3%|▎         | 1054/37962 [2:30:19<86:16:43,  8.42s/it]
  3%|▎         | 1055/37962 [2:30:27<85:44:07,  8.36s/it]
  3%|▎         | 1056/37962 [2:30:35<85:34:05,  8.35s/it]
  3%|▎         | 1057/37962 [2:30:44<85:01:49,  8.29s/it]
  3%|▎         | 1058/37962 [2:30:52<85:01:37,  8.29s/it]
  3%|▎         | 1059/37962 [2:31:00<85:03:56,  8.30s/it]
  3%|▎         | 1060/37962 [2:31:08<84:48:05,  8.27s/it]
                                                         
{'loss': 1.6398, 'grad_norm': 48.01411056518555, 'learning_rate': 1.8612818261633012e-05, 'epoch': 0.03}

  3%|▎         | 1060/37962 [2:31:08<84:48:05,  8.27s/it]
  3%|▎         | 1061/37962 [2:31:17<84:41:19,  8.26s/it]
  3%|▎         | 1062/37962 [2:31:25<84:07:40,  8.21s/it]
  3%|▎         | 1063/37962 [2:31:33<84:40:23,  8.26s/it]
  3%|▎         | 1064/37962 [2:31:41<84:35:00,  8.25s/it]
  3%|▎         | 1065/37962 [2:31:49<84:01:53,  8.20s/it]
  3%|▎         | 1066/37962 [2:31:58<84:27:10,  8.24s/it]
  3%|▎         | 1067/37962 [2:32:06<83:53:38,  8.19s/it]
  3%|▎         | 1068/37962 [2:32:14<83:32:50,  8.15s/it]
  3%|▎         | 1069/37962 [2:32:22<83:06:36,  8.11s/it]
  3%|▎         | 1070/37962 [2:32:30<83:24:39,  8.14s/it]
                                                         
{'loss': 1.5782, 'grad_norm': 31.48131561279297, 'learning_rate': 1.878841088674276e-05, 'epoch': 0.03}

  3%|▎         | 1070/37962 [2:32:30<83:24:39,  8.14s/it]
  3%|▎         | 1071/37962 [2:32:38<83:52:04,  8.18s/it]
  3%|▎         | 1072/37962 [2:32:47<84:11:23,  8.22s/it]
  3%|▎         | 1073/37962 [2:32:55<83:38:20,  8.16s/it]
  3%|▎         | 1074/37962 [2:33:03<84:22:44,  8.23s/it]
  3%|▎         | 1075/37962 [2:33:12<84:58:31,  8.29s/it]
  3%|▎         | 1076/37962 [2:33:20<84:47:47,  8.28s/it]
  3%|▎         | 1077/37962 [2:33:28<85:18:13,  8.33s/it]
  3%|▎         | 1078/37962 [2:33:37<85:28:41,  8.34s/it]
  3%|▎         | 1079/37962 [2:33:45<84:42:56,  8.27s/it]
  3%|▎         | 1080/37962 [2:33:53<84:44:59,  8.27s/it]
                                                         
{'loss': 1.7118, 'grad_norm': 10.33256721496582, 'learning_rate': 1.8964003511852505e-05, 'epoch': 0.03}

  3%|▎         | 1080/37962 [2:33:53<84:44:59,  8.27s/it]
  3%|▎         | 1081/37962 [2:34:01<83:23:30,  8.14s/it]
  3%|▎         | 1082/37962 [2:34:09<83:32:32,  8.15s/it]
  3%|▎         | 1083/37962 [2:34:17<83:43:54,  8.17s/it]
  3%|▎         | 1084/37962 [2:34:25<83:32:46,  8.16s/it]
  3%|▎         | 1085/37962 [2:34:33<82:52:12,  8.09s/it]
  3%|▎         | 1086/37962 [2:34:41<83:15:46,  8.13s/it]
  3%|▎         | 1087/37962 [2:34:50<83:27:28,  8.15s/it]
  3%|▎         | 1088/37962 [2:34:58<83:38:12,  8.17s/it]
  3%|▎         | 1089/37962 [2:35:06<83:50:26,  8.19s/it]
  3%|▎         | 1090/37962 [2:35:14<82:11:07,  8.02s/it]
                                                         
{'loss': 1.6845, 'grad_norm': 11.275463104248047, 'learning_rate': 1.9139596136962247e-05, 'epoch': 0.03}

  3%|▎         | 1090/37962 [2:35:14<82:11:07,  8.02s/it]
  3%|▎         | 1091/37962 [2:35:22<82:07:23,  8.02s/it]
  3%|▎         | 1092/37962 [2:35:30<82:59:39,  8.10s/it]
  3%|▎         | 1093/37962 [2:35:38<83:42:49,  8.17s/it]
  3%|▎         | 1094/37962 [2:35:46<83:03:44,  8.11s/it]
  3%|▎         | 1095/37962 [2:35:54<82:37:50,  8.07s/it]
  3%|▎         | 1096/37962 [2:36:03<83:23:22,  8.14s/it]
  3%|▎         | 1097/37962 [2:36:11<83:43:13,  8.18s/it]
  3%|▎         | 1098/37962 [2:36:19<83:24:37,  8.15s/it]
  3%|▎         | 1099/37962 [2:36:27<84:01:34,  8.21s/it]
  3%|▎         | 1100/37962 [2:36:36<84:14:05,  8.23s/it]
                                                         
{'loss': 1.6839, 'grad_norm': 61.95518112182617, 'learning_rate': 1.9315188762071993e-05, 'epoch': 0.03}

  3%|▎         | 1100/37962 [2:36:36<84:14:05,  8.23s/it]
  3%|▎         | 1101/37962 [2:36:44<84:27:43,  8.25s/it]
  3%|▎         | 1102/37962 [2:36:52<84:57:29,  8.30s/it]
  3%|▎         | 1103/37962 [2:37:01<84:41:52,  8.27s/it]
  3%|▎         | 1104/37962 [2:37:09<84:11:13,  8.22s/it]
  3%|▎         | 1105/37962 [2:37:17<84:06:41,  8.22s/it]
  3%|▎         | 1106/37962 [2:37:25<84:29:04,  8.25s/it]
  3%|▎         | 1107/37962 [2:37:33<84:27:25,  8.25s/it]
  3%|▎         | 1108/37962 [2:37:42<84:16:55,  8.23s/it]
  3%|▎         | 1109/37962 [2:37:50<83:27:02,  8.15s/it]
  3%|▎         | 1110/37962 [2:37:58<83:33:06,  8.16s/it]
                                                         
{'loss': 1.6975, 'grad_norm': 69.8826904296875, 'learning_rate': 1.949078138718174e-05, 'epoch': 0.03}

  3%|▎         | 1110/37962 [2:37:58<83:33:06,  8.16s/it]
  3%|▎         | 1111/37962 [2:38:06<84:15:14,  8.23s/it]
  3%|▎         | 1112/37962 [2:38:15<84:59:00,  8.30s/it]
  3%|▎         | 1113/37962 [2:38:23<85:04:03,  8.31s/it]
  3%|▎         | 1114/37962 [2:38:31<85:50:22,  8.39s/it]
  3%|▎         | 1115/37962 [2:38:40<85:28:36,  8.35s/it]
  3%|▎         | 1116/37962 [2:38:48<85:14:08,  8.33s/it]
  3%|▎         | 1117/37962 [2:38:56<84:45:44,  8.28s/it]
  3%|▎         | 1118/37962 [2:39:04<84:26:46,  8.25s/it]
  3%|▎         | 1119/37962 [2:39:13<84:46:02,  8.28s/it]
  3%|▎         | 1120/37962 [2:39:21<84:28:29,  8.25s/it]
                                                         
{'loss': 1.5871, 'grad_norm': 12.102856636047363, 'learning_rate': 1.9666374012291486e-05, 'epoch': 0.03}

  3%|▎         | 1120/37962 [2:39:21<84:28:29,  8.25s/it]
  3%|▎         | 1121/37962 [2:39:29<84:07:12,  8.22s/it]
  3%|▎         | 1122/37962 [2:39:37<84:10:11,  8.23s/it]
  3%|▎         | 1123/37962 [2:39:45<84:02:03,  8.21s/it]
  3%|▎         | 1124/37962 [2:39:54<84:08:06,  8.22s/it]
  3%|▎         | 1125/37962 [2:40:02<84:15:22,  8.23s/it]
  3%|▎         | 1126/37962 [2:40:10<83:00:01,  8.11s/it]
  3%|▎         | 1127/37962 [2:40:18<82:46:10,  8.09s/it]
  3%|▎         | 1128/37962 [2:40:26<83:21:18,  8.15s/it]
  3%|▎         | 1129/37962 [2:40:34<83:13:29,  8.13s/it]
  3%|▎         | 1130/37962 [2:40:42<83:21:05,  8.15s/it]
                                                         
{'loss': 1.8457, 'grad_norm': 69.20232391357422, 'learning_rate': 1.9841966637401232e-05, 'epoch': 0.03}

  3%|▎         | 1130/37962 [2:40:42<83:21:05,  8.15s/it]
  3%|▎         | 1131/37962 [2:40:51<83:08:14,  8.13s/it]
  3%|▎         | 1132/37962 [2:40:58<82:37:58,  8.08s/it]
  3%|▎         | 1133/37962 [2:41:07<83:13:16,  8.13s/it]
  3%|▎         | 1134/37962 [2:41:15<83:58:30,  8.21s/it]
  3%|▎         | 1135/37962 [2:41:23<84:15:38,  8.24s/it]
  3%|▎         | 1136/37962 [2:41:32<84:29:56,  8.26s/it]
  3%|▎         | 1137/37962 [2:41:40<84:22:42,  8.25s/it]
  3%|▎         | 1138/37962 [2:41:48<84:24:53,  8.25s/it]
  3%|▎         | 1139/37962 [2:41:56<84:25:39,  8.25s/it]
  3%|▎         | 1140/37962 [2:42:05<84:27:34,  8.26s/it]
                                                         
{'loss': 1.7612, 'grad_norm': 36.88510513305664, 'learning_rate': 1.9999999963605866e-05, 'epoch': 0.03}

  3%|▎         | 1140/37962 [2:42:05<84:27:34,  8.26s/it]
  3%|▎         | 1141/37962 [2:42:13<85:06:24,  8.32s/it]
  3%|▎         | 1142/37962 [2:42:21<84:35:00,  8.27s/it]
  3%|▎         | 1143/37962 [2:42:30<85:16:45,  8.34s/it]
  3%|▎         | 1144/37962 [2:42:38<86:02:44,  8.41s/it]
  3%|▎         | 1145/37962 [2:42:47<85:05:54,  8.32s/it]
  3%|▎         | 1146/37962 [2:42:55<84:48:54,  8.29s/it]
  3%|▎         | 1147/37962 [2:43:03<84:59:43,  8.31s/it]
  3%|▎         | 1148/37962 [2:43:12<85:40:55,  8.38s/it]
  3%|▎         | 1149/37962 [2:43:20<85:00:36,  8.31s/it]
  3%|▎         | 1150/37962 [2:43:28<84:55:01,  8.30s/it]
                                                         
{'loss': 1.6623, 'grad_norm': 63.974308013916016, 'learning_rate': 1.9999995596309868e-05, 'epoch': 0.03}

  3%|▎         | 1150/37962 [2:43:28<84:55:01,  8.30s/it]
  3%|▎         | 1151/37962 [2:43:36<85:04:03,  8.32s/it]
  3%|▎         | 1152/37962 [2:43:44<83:32:48,  8.17s/it]
  3%|▎         | 1153/37962 [2:43:52<83:28:06,  8.16s/it]
  3%|▎         | 1154/37962 [2:44:00<81:59:24,  8.02s/it]
  3%|▎         | 1155/37962 [2:44:08<82:56:35,  8.11s/it]
  3%|▎         | 1156/37962 [2:44:16<82:39:50,  8.09s/it]
  3%|▎         | 1157/37962 [2:44:25<82:48:26,  8.10s/it]
  3%|▎         | 1158/37962 [2:44:33<83:28:13,  8.16s/it]
  3%|▎         | 1159/37962 [2:44:41<82:18:45,  8.05s/it]
  3%|▎         | 1160/37962 [2:44:49<83:01:54,  8.12s/it]
                                                         
{'loss': 1.5893, 'grad_norm': 39.72188186645508, 'learning_rate': 1.999998395019031e-05, 'epoch': 0.03}

  3%|▎         | 1160/37962 [2:44:49<83:01:54,  8.12s/it]
  3%|▎         | 1161/37962 [2:44:57<83:44:01,  8.19s/it]
  3%|▎         | 1162/37962 [2:45:05<83:11:15,  8.14s/it]
  3%|▎         | 1163/37962 [2:45:14<83:14:29,  8.14s/it]
  3%|▎         | 1164/37962 [2:45:22<83:27:19,  8.16s/it]
  3%|▎         | 1165/37962 [2:45:30<83:56:03,  8.21s/it]
  3%|▎         | 1166/37962 [2:45:38<83:48:03,  8.20s/it]
  3%|▎         | 1167/37962 [2:45:46<83:40:27,  8.19s/it]
  3%|▎         | 1168/37962 [2:45:55<83:41:21,  8.19s/it]
  3%|▎         | 1169/37962 [2:46:03<83:42:09,  8.19s/it]
  3%|▎         | 1170/37962 [2:46:11<84:23:53,  8.26s/it]
                                                         
{'loss': 1.6189, 'grad_norm': 38.60426712036133, 'learning_rate': 1.9999965025255676e-05, 'epoch': 0.03}

  3%|▎         | 1170/37962 [2:46:11<84:23:53,  8.26s/it]
  3%|▎         | 1171/37962 [2:46:20<84:41:16,  8.29s/it]
  3%|▎         | 1172/37962 [2:46:28<84:28:54,  8.27s/it]
  3%|▎         | 1173/37962 [2:46:36<84:44:17,  8.29s/it]
  3%|▎         | 1174/37962 [2:46:45<85:17:59,  8.35s/it]
  3%|▎         | 1175/37962 [2:46:53<85:13:31,  8.34s/it]
  3%|▎         | 1176/37962 [2:47:01<85:57:08,  8.41s/it]
  3%|▎         | 1177/37962 [2:47:10<86:34:37,  8.47s/it]
  3%|▎         | 1178/37962 [2:47:19<86:34:01,  8.47s/it]
  3%|▎         | 1179/37962 [2:47:27<87:27:55,  8.56s/it]
  3%|▎         | 1180/37962 [2:47:36<87:11:24,  8.53s/it]
                                                         
{'loss': 1.5236, 'grad_norm': 25.619131088256836, 'learning_rate': 1.9999938821519738e-05, 'epoch': 0.03}

  3%|▎         | 1180/37962 [2:47:36<87:11:24,  8.53s/it]
  3%|▎         | 1181/37962 [2:47:44<86:41:46,  8.49s/it]
  3%|▎         | 1182/37962 [2:47:52<86:00:03,  8.42s/it]
  3%|▎         | 1183/37962 [2:48:00<84:40:11,  8.29s/it]
  3%|▎         | 1184/37962 [2:48:08<83:45:26,  8.20s/it]
  3%|▎         | 1185/37962 [2:48:17<84:06:44,  8.23s/it]
  3%|▎         | 1186/37962 [2:48:25<83:41:43,  8.19s/it]
  3%|▎         | 1187/37962 [2:48:33<82:47:57,  8.11s/it]
  3%|▎         | 1188/37962 [2:48:41<83:37:39,  8.19s/it]
  3%|▎         | 1189/37962 [2:48:49<83:35:41,  8.18s/it]
  3%|▎         | 1190/37962 [2:48:58<84:41:04,  8.29s/it]
                                                         
{'loss': 1.5725, 'grad_norm': 20.88021469116211, 'learning_rate': 1.9999905339001568e-05, 'epoch': 0.03}

  3%|▎         | 1190/37962 [2:48:58<84:41:04,  8.29s/it]
  3%|▎         | 1191/37962 [2:49:06<83:55:03,  8.22s/it]
  3%|▎         | 1192/37962 [2:49:14<84:21:55,  8.26s/it]
  3%|▎         | 1193/37962 [2:49:22<84:04:23,  8.23s/it]
  3%|▎         | 1194/37962 [2:49:31<85:03:29,  8.33s/it]
  3%|▎         | 1195/37962 [2:49:39<84:33:06,  8.28s/it]
  3%|▎         | 1196/37962 [2:49:47<83:46:17,  8.20s/it]
  3%|▎         | 1197/37962 [2:49:55<84:00:30,  8.23s/it]
  3%|▎         | 1198/37962 [2:50:04<84:39:22,  8.29s/it]
  3%|▎         | 1199/37962 [2:50:12<84:39:17,  8.29s/it]
  3%|▎         | 1200/37962 [2:50:20<84:18:07,  8.26s/it]
                                                         
{'loss': 1.4729, 'grad_norm': 51.05508804321289, 'learning_rate': 1.9999864577725537e-05, 'epoch': 0.03}

  3%|▎         | 1200/37962 [2:50:20<84:18:07,  8.26s/it]
  3%|▎         | 1201/37962 [2:50:29<84:33:21,  8.28s/it]
  3%|▎         | 1202/37962 [2:50:37<84:48:19,  8.31s/it]
  3%|▎         | 1203/37962 [2:50:45<84:11:46,  8.25s/it]
  3%|▎         | 1204/37962 [2:50:53<84:24:18,  8.27s/it]
  3%|▎         | 1205/37962 [2:51:01<83:37:53,  8.19s/it]
  3%|▎         | 1206/37962 [2:51:10<83:50:06,  8.21s/it]
  3%|▎         | 1207/37962 [2:51:18<83:43:03,  8.20s/it]
  3%|▎         | 1208/37962 [2:51:26<84:09:28,  8.24s/it]
  3%|▎         | 1209/37962 [2:51:34<83:57:26,  8.22s/it]
  3%|▎         | 1210/37962 [2:51:43<84:01:02,  8.23s/it]
                                                         
{'loss': 1.5339, 'grad_norm': 16.338815689086914, 'learning_rate': 1.9999816537721318e-05, 'epoch': 0.03}

  3%|▎         | 1210/37962 [2:51:43<84:01:02,  8.23s/it]
  3%|▎         | 1211/37962 [2:51:51<84:35:07,  8.29s/it]
  3%|▎         | 1212/37962 [2:51:59<84:11:20,  8.25s/it]
  3%|▎         | 1213/37962 [2:52:07<84:06:59,  8.24s/it]
  3%|▎         | 1214/37962 [2:52:16<83:56:20,  8.22s/it]
  3%|▎         | 1215/37962 [2:52:24<84:10:53,  8.25s/it]
  3%|▎         | 1216/37962 [2:52:32<83:50:52,  8.21s/it]
  3%|▎         | 1217/37962 [2:52:40<83:58:21,  8.23s/it]
  3%|▎         | 1218/37962 [2:52:49<83:56:59,  8.23s/it]
  3%|▎         | 1219/37962 [2:52:57<84:23:13,  8.27s/it]
  3%|▎         | 1220/37962 [2:53:05<83:40:54,  8.20s/it]
                                                         
{'loss': 1.629, 'grad_norm': 67.26805114746094, 'learning_rate': 1.9999761219023873e-05, 'epoch': 0.03}

  3%|▎         | 1220/37962 [2:53:05<83:40:54,  8.20s/it]
  3%|▎         | 1221/37962 [2:53:13<84:04:30,  8.24s/it]
  3%|▎         | 1222/37962 [2:53:22<84:04:54,  8.24s/it]
  3%|▎         | 1223/37962 [2:53:30<84:36:58,  8.29s/it]
  3%|▎         | 1224/37962 [2:53:38<84:59:25,  8.33s/it]
  3%|▎         | 1225/37962 [2:53:47<84:57:00,  8.32s/it]
  3%|▎         | 1226/37962 [2:53:55<84:45:03,  8.31s/it]
  3%|▎         | 1227/37962 [2:54:03<84:36:52,  8.29s/it]
  3%|▎         | 1228/37962 [2:54:12<84:51:33,  8.32s/it]
  3%|▎         | 1229/37962 [2:54:20<84:27:08,  8.28s/it]
  3%|▎         | 1230/37962 [2:54:28<84:10:11,  8.25s/it]
                                                         
{'loss': 1.533, 'grad_norm': 11.371353149414062, 'learning_rate': 1.9999698621673474e-05, 'epoch': 0.03}

  3%|▎         | 1230/37962 [2:54:28<84:10:11,  8.25s/it]
  3%|▎         | 1231/37962 [2:54:36<84:20:24,  8.27s/it]
  3%|▎         | 1232/37962 [2:54:45<84:31:20,  8.28s/it]
  3%|▎         | 1233/37962 [2:54:52<83:08:48,  8.15s/it]
  3%|▎         | 1234/37962 [2:55:01<82:56:48,  8.13s/it]
  3%|▎         | 1235/37962 [2:55:09<82:49:46,  8.12s/it]
  3%|▎         | 1236/37962 [2:55:17<83:06:58,  8.15s/it]
  3%|▎         | 1237/37962 [2:55:25<83:34:45,  8.19s/it]
  3%|▎         | 1238/37962 [2:55:34<84:12:30,  8.25s/it]
  3%|▎         | 1239/37962 [2:55:42<84:04:06,  8.24s/it]
  3%|▎         | 1240/37962 [2:55:50<84:10:09,  8.25s/it]
                                                         
{'loss': 1.5784, 'grad_norm': 48.50962448120117, 'learning_rate': 1.9999628745715678e-05, 'epoch': 0.03}

  3%|▎         | 1240/37962 [2:55:50<84:10:09,  8.25s/it]
  3%|▎         | 1241/37962 [2:55:58<84:00:43,  8.24s/it]
  3%|▎         | 1242/37962 [2:56:06<83:42:56,  8.21s/it]
  3%|▎         | 1243/37962 [2:56:15<83:44:44,  8.21s/it]
  3%|▎         | 1244/37962 [2:56:23<83:28:43,  8.18s/it]
  3%|▎         | 1245/37962 [2:56:31<83:50:39,  8.22s/it]
  3%|▎         | 1246/37962 [2:56:39<84:13:17,  8.26s/it]
  3%|▎         | 1247/37962 [2:56:48<84:12:57,  8.26s/it]
  3%|▎         | 1248/37962 [2:56:56<84:14:46,  8.26s/it]
  3%|▎         | 1249/37962 [2:57:04<84:47:29,  8.31s/it]
  3%|▎         | 1250/37962 [2:57:12<83:36:03,  8.20s/it]
                                                         
{'loss': 1.5816, 'grad_norm': 21.110523223876953, 'learning_rate': 1.999955159120135e-05, 'epoch': 0.03}

  3%|▎         | 1250/37962 [2:57:12<83:36:03,  8.20s/it]
  3%|▎         | 1251/37962 [2:57:20<83:30:13,  8.19s/it]
  3%|▎         | 1252/37962 [2:57:28<82:17:33,  8.07s/it]
  3%|▎         | 1253/37962 [2:57:36<82:04:30,  8.05s/it]
  3%|▎         | 1254/37962 [2:57:44<82:36:53,  8.10s/it]
  3%|▎         | 1255/37962 [2:57:53<83:36:22,  8.20s/it]
  3%|▎         | 1256/37962 [2:58:01<84:21:51,  8.27s/it]
  3%|▎         | 1257/37962 [2:58:10<84:53:10,  8.33s/it]
  3%|▎         | 1258/37962 [2:58:18<83:47:40,  8.22s/it]
  3%|▎         | 1259/37962 [2:58:26<83:08:09,  8.15s/it]
  3%|▎         | 1260/37962 [2:58:34<83:29:00,  8.19s/it]
                                                         
{'loss': 1.6949, 'grad_norm': 15.816736221313477, 'learning_rate': 1.9999467158186652e-05, 'epoch': 0.03}

  3%|▎         | 1260/37962 [2:58:34<83:29:00,  8.19s/it]
  3%|▎         | 1261/37962 [2:58:42<83:40:17,  8.21s/it]
  3%|▎         | 1262/37962 [2:58:50<83:37:48,  8.20s/it]
  3%|▎         | 1263/37962 [2:58:59<83:45:51,  8.22s/it]
  3%|▎         | 1264/37962 [2:59:07<84:34:44,  8.30s/it]
  3%|▎         | 1265/37962 [2:59:15<84:41:59,  8.31s/it]
  3%|▎         | 1266/37962 [2:59:24<84:21:18,  8.28s/it]
  3%|▎         | 1267/37962 [2:59:32<83:18:10,  8.17s/it]
  3%|▎         | 1268/37962 [2:59:40<83:30:39,  8.19s/it]
  3%|▎         | 1269/37962 [2:59:48<82:15:04,  8.07s/it]
  3%|▎         | 1270/37962 [2:59:56<82:46:21,  8.12s/it]
                                                         
{'loss': 1.6069, 'grad_norm': 29.19423484802246, 'learning_rate': 1.9999375446733035e-05, 'epoch': 0.03}

  3%|▎         | 1270/37962 [2:59:56<82:46:21,  8.12s/it]
  3%|▎         | 1271/37962 [3:00:04<84:17:21,  8.27s/it]
  3%|▎         | 1272/37962 [3:00:13<84:11:11,  8.26s/it]
  3%|▎         | 1273/37962 [3:00:21<84:06:50,  8.25s/it]
  3%|▎         | 1274/37962 [3:00:29<83:34:40,  8.20s/it]
  3%|▎         | 1275/37962 [3:00:37<84:00:15,  8.24s/it]
  3%|▎         | 1276/37962 [3:00:46<84:14:42,  8.27s/it]
  3%|▎         | 1277/37962 [3:00:54<84:33:42,  8.30s/it]
  3%|▎         | 1278/37962 [3:01:02<84:33:45,  8.30s/it]
  3%|▎         | 1279/37962 [3:01:10<83:31:33,  8.20s/it]
  3%|▎         | 1280/37962 [3:01:19<83:31:17,  8.20s/it]
                                                         
{'loss': 1.5314, 'grad_norm': 12.36455249786377, 'learning_rate': 1.9999276456907262e-05, 'epoch': 0.03}

  3%|▎         | 1280/37962 [3:01:19<83:31:17,  8.20s/it]
  3%|▎         | 1281/37962 [3:01:27<83:32:36,  8.20s/it]
  3%|▎         | 1282/37962 [3:01:35<83:55:23,  8.24s/it]
  3%|▎         | 1283/37962 [3:01:43<84:28:09,  8.29s/it]
  3%|▎         | 1284/37962 [3:01:52<84:47:20,  8.32s/it]
  3%|▎         | 1285/37962 [3:02:00<83:43:46,  8.22s/it]
  3%|▎         | 1286/37962 [3:02:08<83:43:43,  8.22s/it]
  3%|▎         | 1287/37962 [3:02:16<83:44:27,  8.22s/it]
  3%|▎         | 1288/37962 [3:02:24<83:08:51,  8.16s/it]
  3%|▎         | 1289/37962 [3:02:33<83:26:15,  8.19s/it]
  3%|▎         | 1290/37962 [3:02:41<83:33:21,  8.20s/it]
                                                         
{'loss': 1.7558, 'grad_norm': 33.648460388183594, 'learning_rate': 1.999917018878138e-05, 'epoch': 0.03}

  3%|▎         | 1290/37962 [3:02:41<83:33:21,  8.20s/it]
  3%|▎         | 1291/37962 [3:02:49<83:39:55,  8.21s/it]
  3%|▎         | 1292/37962 [3:02:57<83:51:10,  8.23s/it]
  3%|▎         | 1293/37962 [3:03:05<83:32:55,  8.20s/it]
  3%|▎         | 1294/37962 [3:03:13<82:55:40,  8.14s/it]
  3%|▎         | 1295/37962 [3:03:22<84:04:21,  8.25s/it]
  3%|▎         | 1296/37962 [3:03:30<82:43:30,  8.12s/it]
  3%|▎         | 1297/37962 [3:03:38<83:03:20,  8.15s/it]
  3%|▎         | 1298/37962 [3:03:46<82:24:47,  8.09s/it]
  3%|▎         | 1299/37962 [3:03:54<83:08:00,  8.16s/it]
  3%|▎         | 1300/37962 [3:04:03<83:38:01,  8.21s/it]
                                                         
{'loss': 1.5849, 'grad_norm': 25.541271209716797, 'learning_rate': 1.9999056642432738e-05, 'epoch': 0.03}

  3%|▎         | 1300/37962 [3:04:03<83:38:01,  8.21s/it]
  3%|▎         | 1301/37962 [3:04:11<84:27:07,  8.29s/it]
  3%|▎         | 1302/37962 [3:04:20<84:58:37,  8.34s/it]
  3%|▎         | 1303/37962 [3:04:28<84:52:34,  8.34s/it]
  3%|▎         | 1304/37962 [3:04:36<84:24:46,  8.29s/it]
  3%|▎         | 1305/37962 [3:04:44<84:29:08,  8.30s/it]
  3%|▎         | 1306/37962 [3:04:53<83:59:44,  8.25s/it]
  3%|▎         | 1307/37962 [3:05:01<84:07:55,  8.26s/it]
  3%|▎         | 1308/37962 [3:05:09<84:26:58,  8.29s/it]
  3%|▎         | 1309/37962 [3:05:18<85:32:04,  8.40s/it]
  3%|▎         | 1310/37962 [3:05:25<83:04:43,  8.16s/it]
                                                         
{'loss': 1.5663, 'grad_norm': 31.491891860961914, 'learning_rate': 1.999893581794399e-05, 'epoch': 0.03}

  3%|▎         | 1310/37962 [3:05:25<83:04:43,  8.16s/it]
  3%|▎         | 1311/37962 [3:05:34<83:25:13,  8.19s/it]
  3%|▎         | 1312/37962 [3:05:42<83:42:04,  8.22s/it]
  3%|▎         | 1313/37962 [3:05:50<83:28:16,  8.20s/it]
  3%|▎         | 1314/37962 [3:05:58<83:28:22,  8.20s/it]
  3%|▎         | 1315/37962 [3:06:07<84:05:56,  8.26s/it]
  3%|▎         | 1316/37962 [3:06:15<84:42:30,  8.32s/it]
  3%|▎         | 1317/37962 [3:06:23<84:31:16,  8.30s/it]
  3%|▎         | 1318/37962 [3:06:32<84:42:20,  8.32s/it]
  3%|▎         | 1319/37962 [3:06:40<84:34:18,  8.31s/it]
  3%|▎         | 1320/37962 [3:06:48<83:05:24,  8.16s/it]
                                                         
{'loss': 1.5944, 'grad_norm': 27.946325302124023, 'learning_rate': 1.9998807715403083e-05, 'epoch': 0.03}

  3%|▎         | 1320/37962 [3:06:48<83:05:24,  8.16s/it]
  3%|▎         | 1321/37962 [3:06:56<82:12:09,  8.08s/it]
  3%|▎         | 1322/37962 [3:07:04<82:53:35,  8.14s/it]
  3%|▎         | 1323/37962 [3:07:12<82:21:36,  8.09s/it]
  3%|▎         | 1324/37962 [3:07:20<83:12:54,  8.18s/it]
  3%|▎         | 1325/37962 [3:07:29<83:35:16,  8.21s/it]
  3%|▎         | 1326/37962 [3:07:37<83:44:51,  8.23s/it]
  3%|▎         | 1327/37962 [3:07:45<83:37:25,  8.22s/it]
  3%|▎         | 1328/37962 [3:07:53<83:18:55,  8.19s/it]
  4%|▎         | 1329/37962 [3:08:01<82:21:15,  8.09s/it]
  4%|▎         | 1330/37962 [3:08:10<83:10:29,  8.17s/it]
                                                         
{'loss': 1.6291, 'grad_norm': 47.63523483276367, 'learning_rate': 1.9998672334903253e-05, 'epoch': 0.04}

  4%|▎         | 1330/37962 [3:08:10<83:10:29,  8.17s/it]
  4%|▎         | 1331/37962 [3:08:18<83:48:33,  8.24s/it]
  4%|▎         | 1332/37962 [3:08:26<83:47:19,  8.23s/it]
  4%|▎         | 1333/37962 [3:08:34<83:45:35,  8.23s/it]
  4%|▎         | 1334/37962 [3:08:43<83:48:24,  8.24s/it]
  4%|▎         | 1335/37962 [3:08:51<83:55:35,  8.25s/it]
  4%|▎         | 1336/37962 [3:08:59<84:05:17,  8.27s/it]
  4%|▎         | 1337/37962 [3:09:08<84:13:55,  8.28s/it]
  4%|▎         | 1338/37962 [3:09:16<84:31:00,  8.31s/it]
  4%|▎         | 1339/37962 [3:09:24<84:07:24,  8.27s/it]
  4%|▎         | 1340/37962 [3:09:33<84:41:46,  8.33s/it]
                                                         
{'loss': 1.6477, 'grad_norm': 32.599159240722656, 'learning_rate': 1.9998529676543048e-05, 'epoch': 0.04}

  4%|▎         | 1340/37962 [3:09:33<84:41:46,  8.33s/it]
  4%|▎         | 1341/37962 [3:09:41<83:52:19,  8.24s/it]
  4%|▎         | 1342/37962 [3:09:49<83:54:06,  8.25s/it]
  4%|▎         | 1343/37962 [3:09:57<84:26:20,  8.30s/it]
  4%|▎         | 1344/37962 [3:10:06<84:14:35,  8.28s/it]
  4%|▎         | 1345/37962 [3:10:14<84:05:04,  8.27s/it]
  4%|▎         | 1346/37962 [3:10:22<84:04:33,  8.27s/it]
  4%|▎         | 1347/37962 [3:10:30<84:18:06,  8.29s/it]
  4%|▎         | 1348/37962 [3:10:39<84:26:16,  8.30s/it]
  4%|▎         | 1349/37962 [3:10:47<83:58:42,  8.26s/it]
  4%|▎         | 1350/37962 [3:10:55<83:20:15,  8.19s/it]
                                                         
{'loss': 1.5897, 'grad_norm': 39.296539306640625, 'learning_rate': 1.9998379740426303e-05, 'epoch': 0.04}

  4%|▎         | 1350/37962 [3:10:55<83:20:15,  8.19s/it]
  4%|▎         | 1351/37962 [3:11:04<84:59:53,  8.36s/it]
  4%|▎         | 1352/37962 [3:11:12<84:38:14,  8.32s/it]
  4%|▎         | 1353/37962 [3:11:20<84:29:04,  8.31s/it]
  4%|▎         | 1354/37962 [3:11:28<84:17:56,  8.29s/it]
  4%|▎         | 1355/37962 [3:11:36<83:36:46,  8.22s/it]
  4%|▎         | 1356/37962 [3:11:45<83:26:18,  8.21s/it]
  4%|▎         | 1357/37962 [3:11:53<83:23:04,  8.20s/it]
  4%|▎         | 1358/37962 [3:12:01<83:54:26,  8.25s/it]
  4%|▎         | 1359/37962 [3:12:10<84:20:54,  8.30s/it]
  4%|▎         | 1360/37962 [3:12:18<84:13:46,  8.28s/it]
                                                         
{'loss': 1.6937, 'grad_norm': 17.99929428100586, 'learning_rate': 1.9998222526662157e-05, 'epoch': 0.04}

  4%|▎         | 1360/37962 [3:12:18<84:13:46,  8.28s/it]
  4%|▎         | 1361/37962 [3:12:26<84:04:40,  8.27s/it]
  4%|▎         | 1362/37962 [3:12:34<84:22:03,  8.30s/it]
  4%|▎         | 1363/37962 [3:12:43<83:45:41,  8.24s/it]
  4%|▎         | 1364/37962 [3:12:51<83:39:24,  8.23s/it]
  4%|▎         | 1365/37962 [3:12:59<83:32:51,  8.22s/it]
  4%|▎         | 1366/37962 [3:13:07<83:52:51,  8.25s/it]
  4%|▎         | 1367/37962 [3:13:15<83:40:51,  8.23s/it]
  4%|▎         | 1368/37962 [3:13:24<83:27:06,  8.21s/it]
  4%|▎         | 1369/37962 [3:13:32<83:09:53,  8.18s/it]
  4%|▎         | 1370/37962 [3:13:40<83:14:44,  8.19s/it]
                                                         
{'loss': 1.5566, 'grad_norm': 13.584074020385742, 'learning_rate': 1.9998058035365038e-05, 'epoch': 0.04}

  4%|▎         | 1370/37962 [3:13:40<83:14:44,  8.19s/it]
  4%|▎         | 1371/37962 [3:13:48<82:57:08,  8.16s/it]
  4%|▎         | 1372/37962 [3:13:56<83:31:44,  8.22s/it]
  4%|▎         | 1373/37962 [3:14:05<83:39:35,  8.23s/it]
  4%|▎         | 1374/37962 [3:14:13<83:40:43,  8.23s/it]
  4%|▎         | 1375/37962 [3:14:21<83:43:47,  8.24s/it]
  4%|▎         | 1376/37962 [3:14:29<83:42:54,  8.24s/it]
  4%|▎         | 1377/37962 [3:14:38<84:02:23,  8.27s/it]
  4%|▎         | 1378/37962 [3:14:46<83:46:48,  8.24s/it]
  4%|▎         | 1379/37962 [3:14:54<83:40:58,  8.23s/it]
  4%|▎         | 1380/37962 [3:15:03<84:12:17,  8.29s/it]
                                                         
{'loss': 1.5561, 'grad_norm': 22.093358993530273, 'learning_rate': 1.999788626665468e-05, 'epoch': 0.04}

  4%|▎         | 1380/37962 [3:15:03<84:12:17,  8.29s/it]
  4%|▎         | 1381/37962 [3:15:11<84:02:03,  8.27s/it]
  4%|▎         | 1382/37962 [3:15:19<83:48:29,  8.25s/it]
  4%|▎         | 1383/37962 [3:15:27<83:32:36,  8.22s/it]
  4%|▎         | 1384/37962 [3:15:36<84:09:57,  8.28s/it]
  4%|▎         | 1385/37962 [3:15:43<82:27:21,  8.12s/it]
  4%|▎         | 1386/37962 [3:15:52<82:49:55,  8.15s/it]
  4%|▎         | 1387/37962 [3:16:00<83:23:40,  8.21s/it]
  4%|▎         | 1388/37962 [3:16:08<83:41:01,  8.24s/it]
  4%|▎         | 1389/37962 [3:16:16<83:55:41,  8.26s/it]
  4%|▎         | 1390/37962 [3:16:25<83:52:56,  8.26s/it]
                                                         
{'loss': 1.6367, 'grad_norm': 11.384148597717285, 'learning_rate': 1.9997707220656114e-05, 'epoch': 0.04}

  4%|▎         | 1390/37962 [3:16:25<83:52:56,  8.26s/it]
  4%|▎         | 1391/37962 [3:16:33<83:19:17,  8.20s/it]
  4%|▎         | 1392/37962 [3:16:41<84:11:17,  8.29s/it]
  4%|▎         | 1393/37962 [3:16:49<83:37:46,  8.23s/it]
  4%|▎         | 1394/37962 [3:16:58<83:27:30,  8.22s/it]
  4%|▎         | 1395/37962 [3:17:06<84:01:52,  8.27s/it]
  4%|▎         | 1396/37962 [3:17:14<84:08:37,  8.28s/it]
  4%|▎         | 1397/37962 [3:17:22<83:35:42,  8.23s/it]
  4%|▎         | 1398/37962 [3:17:30<82:34:39,  8.13s/it]
  4%|▎         | 1399/37962 [3:17:38<82:19:34,  8.11s/it]
  4%|▎         | 1400/37962 [3:17:47<82:59:48,  8.17s/it]
                                                         
{'loss': 1.5147, 'grad_norm': 15.962431907653809, 'learning_rate': 1.9997520897499652e-05, 'epoch': 0.04}

  4%|▎         | 1400/37962 [3:17:47<82:59:48,  8.17s/it]
  4%|▎         | 1401/37962 [3:17:55<83:36:35,  8.23s/it]
  4%|▎         | 1402/37962 [3:18:04<84:51:36,  8.36s/it]
  4%|▎         | 1403/37962 [3:18:12<84:59:53,  8.37s/it]
  4%|▎         | 1404/37962 [3:18:20<84:52:27,  8.36s/it]
  4%|▎         | 1405/37962 [3:18:29<84:37:38,  8.33s/it]
  4%|▎         | 1406/37962 [3:18:37<83:06:32,  8.18s/it]
  4%|▎         | 1407/37962 [3:18:44<82:22:43,  8.11s/it]
  4%|▎         | 1408/37962 [3:18:53<83:01:20,  8.18s/it]
  4%|▎         | 1409/37962 [3:19:01<82:55:01,  8.17s/it]
  4%|▎         | 1410/37962 [3:19:09<83:09:04,  8.19s/it]
                                                         
{'loss': 1.5826, 'grad_norm': 38.767581939697266, 'learning_rate': 1.999732729732093e-05, 'epoch': 0.04}

  4%|▎         | 1410/37962 [3:19:09<83:09:04,  8.19s/it]
  4%|▎         | 1411/37962 [3:19:17<82:53:38,  8.16s/it]
  4%|▎         | 1412/37962 [3:19:26<83:36:15,  8.23s/it]
  4%|▎         | 1413/37962 [3:19:34<83:29:43,  8.22s/it]
  4%|▎         | 1414/37962 [3:19:42<82:58:13,  8.17s/it]
  4%|▎         | 1415/37962 [3:19:50<82:48:58,  8.16s/it]
  4%|▎         | 1416/37962 [3:19:59<83:45:26,  8.25s/it]
  4%|▎         | 1417/37962 [3:20:07<83:51:58,  8.26s/it]
  4%|▎         | 1418/37962 [3:20:15<83:58:27,  8.27s/it]
  4%|▎         | 1419/37962 [3:20:23<83:57:28,  8.27s/it]
  4%|▎         | 1420/37962 [3:20:32<83:59:38,  8.27s/it]
                                                         
{'loss': 1.6265, 'grad_norm': 6.569390296936035, 'learning_rate': 1.999712642026086e-05, 'epoch': 0.04}

  4%|▎         | 1420/37962 [3:20:32<83:59:38,  8.27s/it]
  4%|▎         | 1421/37962 [3:20:40<84:25:21,  8.32s/it]
  4%|▎         | 1422/37962 [3:20:48<84:38:09,  8.34s/it]
  4%|▎         | 1423/37962 [3:20:57<84:12:09,  8.30s/it]
  4%|▍         | 1424/37962 [3:21:05<84:08:50,  8.29s/it]
  4%|▍         | 1425/37962 [3:21:13<83:52:20,  8.26s/it]
  4%|▍         | 1426/37962 [3:21:22<84:17:41,  8.31s/it]
  4%|▍         | 1427/37962 [3:21:30<84:25:47,  8.32s/it]
  4%|▍         | 1428/37962 [3:21:38<83:41:26,  8.25s/it]
  4%|▍         | 1429/37962 [3:21:46<83:46:01,  8.25s/it]
  4%|▍         | 1430/37962 [3:21:55<84:08:03,  8.29s/it]
                                                         
{'loss': 1.5759, 'grad_norm': 40.68317413330078, 'learning_rate': 1.999691826646566e-05, 'epoch': 0.04}

  4%|▍         | 1430/37962 [3:21:55<84:08:03,  8.29s/it]
  4%|▍         | 1431/37962 [3:22:03<84:10:18,  8.29s/it]
  4%|▍         | 1432/37962 [3:22:11<84:24:53,  8.32s/it]
  4%|▍         | 1433/37962 [3:22:19<83:52:06,  8.27s/it]
  4%|▍         | 1434/37962 [3:22:28<83:47:50,  8.26s/it]
  4%|▍         | 1435/37962 [3:22:36<83:53:19,  8.27s/it]
  4%|▍         | 1436/37962 [3:22:45<84:44:20,  8.35s/it]
  4%|▍         | 1437/37962 [3:22:53<84:37:15,  8.34s/it]
  4%|▍         | 1438/37962 [3:23:01<84:27:40,  8.32s/it]
  4%|▍         | 1439/37962 [3:23:09<83:56:55,  8.27s/it]
  4%|▍         | 1440/37962 [3:23:17<83:02:28,  8.19s/it]
                                                         
{'loss': 1.6018, 'grad_norm': 27.016881942749023, 'learning_rate': 1.9996702836086835e-05, 'epoch': 0.04}

  4%|▍         | 1440/37962 [3:23:17<83:02:28,  8.19s/it]
  4%|▍         | 1441/37962 [3:23:25<82:47:41,  8.16s/it]
  4%|▍         | 1442/37962 [3:23:34<83:50:08,  8.26s/it]
  4%|▍         | 1443/37962 [3:23:42<83:20:31,  8.22s/it]
  4%|▍         | 1444/37962 [3:23:50<82:56:36,  8.18s/it]
  4%|▍         | 1445/37962 [3:23:58<82:05:46,  8.09s/it]
  4%|▍         | 1446/37962 [3:24:06<83:17:09,  8.21s/it]
  4%|▍         | 1447/37962 [3:24:15<83:30:34,  8.23s/it]
  4%|▍         | 1448/37962 [3:24:23<83:32:58,  8.24s/it]
  4%|▍         | 1449/37962 [3:24:31<83:37:45,  8.25s/it]
  4%|▍         | 1450/37962 [3:24:39<83:19:36,  8.22s/it]
                                                         
{'loss': 1.7494, 'grad_norm': 19.092866897583008, 'learning_rate': 1.9996480129281195e-05, 'epoch': 0.04}

  4%|▍         | 1450/37962 [3:24:39<83:19:36,  8.22s/it]
  4%|▍         | 1451/37962 [3:24:48<83:35:05,  8.24s/it]
  4%|▍         | 1452/37962 [3:24:56<83:39:41,  8.25s/it]
  4%|▍         | 1453/37962 [3:25:04<82:43:20,  8.16s/it]
  4%|▍         | 1454/37962 [3:25:12<83:19:54,  8.22s/it]
  4%|▍         | 1455/37962 [3:25:20<83:08:52,  8.20s/it]
  4%|▍         | 1456/37962 [3:25:28<82:28:56,  8.13s/it]
  4%|▍         | 1457/37962 [3:25:37<82:50:18,  8.17s/it]
  4%|▍         | 1458/37962 [3:25:45<83:02:43,  8.19s/it]
  4%|▍         | 1459/37962 [3:25:53<83:00:45,  8.19s/it]
  4%|▍         | 1460/37962 [3:26:01<83:37:16,  8.25s/it]
                                                         
{'loss': 1.5535, 'grad_norm': 19.89524269104004, 'learning_rate': 1.999625014621085e-05, 'epoch': 0.04}

  4%|▍         | 1460/37962 [3:26:01<83:37:16,  8.25s/it]
  4%|▍         | 1461/37962 [3:26:10<83:57:53,  8.28s/it]
  4%|▍         | 1462/37962 [3:26:18<83:24:47,  8.23s/it]
  4%|▍         | 1463/37962 [3:26:26<83:31:23,  8.24s/it]
  4%|▍         | 1464/37962 [3:26:35<83:52:19,  8.27s/it]
  4%|▍         | 1465/37962 [3:26:43<83:56:09,  8.28s/it]
  4%|▍         | 1466/37962 [3:26:51<83:36:35,  8.25s/it]
  4%|▍         | 1467/37962 [3:26:59<83:36:02,  8.25s/it]
  4%|▍         | 1468/37962 [3:27:08<84:33:15,  8.34s/it]
  4%|▍         | 1469/37962 [3:27:16<84:25:11,  8.33s/it]
  4%|▍         | 1470/37962 [3:27:24<84:00:55,  8.29s/it]
                                                         
{'loss': 1.4804, 'grad_norm': 37.28947830200195, 'learning_rate': 1.999601288704319e-05, 'epoch': 0.04}

  4%|▍         | 1470/37962 [3:27:24<84:00:55,  8.29s/it]
  4%|▍         | 1471/37962 [3:27:32<83:15:27,  8.21s/it]
  4%|▍         | 1472/37962 [3:27:40<82:45:05,  8.16s/it]
  4%|▍         | 1473/37962 [3:27:49<82:47:00,  8.17s/it]
  4%|▍         | 1474/37962 [3:27:57<83:00:13,  8.19s/it]
  4%|▍         | 1475/37962 [3:28:05<84:12:51,  8.31s/it]
  4%|▍         | 1476/37962 [3:28:14<84:24:34,  8.33s/it]
  4%|▍         | 1477/37962 [3:28:22<84:09:08,  8.30s/it]
  4%|▍         | 1478/37962 [3:28:30<84:17:49,  8.32s/it]
  4%|▍         | 1479/37962 [3:28:39<84:29:17,  8.34s/it]
  4%|▍         | 1480/37962 [3:28:47<84:17:56,  8.32s/it]
                                                         
{'loss': 1.6079, 'grad_norm': 11.237868309020996, 'learning_rate': 1.9995768351950926e-05, 'epoch': 0.04}

  4%|▍         | 1480/37962 [3:28:47<84:17:56,  8.32s/it]
  4%|▍         | 1481/37962 [3:28:55<84:09:30,  8.30s/it]
  4%|▍         | 1482/37962 [3:29:04<84:18:12,  8.32s/it]
  4%|▍         | 1483/37962 [3:29:12<85:08:01,  8.40s/it]
  4%|▍         | 1484/37962 [3:29:21<85:10:10,  8.41s/it]
  4%|▍         | 1485/37962 [3:29:29<85:31:26,  8.44s/it]
  4%|▍         | 1486/37962 [3:29:37<85:00:51,  8.39s/it]
  4%|▍         | 1487/37962 [3:29:46<84:35:59,  8.35s/it]
  4%|▍         | 1488/37962 [3:29:54<84:34:38,  8.35s/it]
  4%|▍         | 1489/37962 [3:30:02<84:24:10,  8.33s/it]
  4%|▍         | 1490/37962 [3:30:11<84:10:22,  8.31s/it]
                                                         
{'loss': 1.5832, 'grad_norm': 25.3029727935791, 'learning_rate': 1.999551654111204e-05, 'epoch': 0.04}

  4%|▍         | 1490/37962 [3:30:11<84:10:22,  8.31s/it]
  4%|▍         | 1491/37962 [3:30:19<83:25:34,  8.23s/it]
  4%|▍         | 1492/37962 [3:30:27<83:15:35,  8.22s/it]
  4%|▍         | 1493/37962 [3:30:35<83:18:34,  8.22s/it]
  4%|▍         | 1494/37962 [3:30:43<82:13:30,  8.12s/it]
  4%|▍         | 1495/37962 [3:30:51<82:51:48,  8.18s/it]
  4%|▍         | 1496/37962 [3:31:00<83:18:59,  8.23s/it]
  4%|▍         | 1497/37962 [3:31:08<83:07:15,  8.21s/it]
  4%|▍         | 1498/37962 [3:31:16<83:53:49,  8.28s/it]
  4%|▍         | 1499/37962 [3:31:25<83:56:17,  8.29s/it]
  4%|▍         | 1500/37962 [3:31:33<84:11:54,  8.31s/it]
                                                         
{'loss': 1.6602, 'grad_norm': 47.03425216674805, 'learning_rate': 1.9995257454709824e-05, 'epoch': 0.04}

  4%|▍         | 1500/37962 [3:31:33<84:11:54,  8.31s/it]
  4%|▍         | 1501/37962 [3:31:41<84:21:51,  8.33s/it]
  4%|▍         | 1502/37962 [3:31:49<83:59:19,  8.29s/it]
  4%|▍         | 1503/37962 [3:31:58<83:16:58,  8.22s/it]
  4%|▍         | 1504/37962 [3:32:06<83:34:57,  8.25s/it]
  4%|▍         | 1505/37962 [3:32:14<83:43:58,  8.27s/it]
  4%|▍         | 1506/37962 [3:32:22<83:00:12,  8.20s/it]
  4%|▍         | 1507/37962 [3:32:30<83:03:06,  8.20s/it]
  4%|▍         | 1508/37962 [3:32:39<82:54:27,  8.19s/it]
  4%|▍         | 1509/37962 [3:32:46<82:09:21,  8.11s/it]
  4%|▍         | 1510/37962 [3:32:55<82:39:48,  8.16s/it]
                                                         
{'loss': 1.415, 'grad_norm': 17.505346298217773, 'learning_rate': 1.9994991092932863e-05, 'epoch': 0.04}

  4%|▍         | 1510/37962 [3:32:55<82:39:48,  8.16s/it]
  4%|▍         | 1511/37962 [3:33:03<82:05:19,  8.11s/it]
  4%|▍         | 1512/37962 [3:33:11<82:35:29,  8.16s/it]
  4%|▍         | 1513/37962 [3:33:19<82:43:19,  8.17s/it]
  4%|▍         | 1514/37962 [3:33:28<83:51:03,  8.28s/it]
  4%|▍         | 1515/37962 [3:33:36<83:04:59,  8.21s/it]
  4%|▍         | 1516/37962 [3:33:44<82:59:00,  8.20s/it]
  4%|▍         | 1517/37962 [3:33:52<83:34:01,  8.25s/it]
  4%|▍         | 1518/37962 [3:34:01<83:50:27,  8.28s/it]
  4%|▍         | 1519/37962 [3:34:09<84:14:24,  8.32s/it]
  4%|▍         | 1520/37962 [3:34:17<84:22:26,  8.34s/it]
                                                         
{'loss': 1.4591, 'grad_norm': 20.101842880249023, 'learning_rate': 1.999471745597504e-05, 'epoch': 0.04}

  4%|▍         | 1520/37962 [3:34:17<84:22:26,  8.34s/it]
  4%|▍         | 1521/37962 [3:34:26<84:38:18,  8.36s/it]
  4%|▍         | 1522/37962 [3:34:34<83:56:32,  8.29s/it]
  4%|▍         | 1523/37962 [3:34:42<83:12:10,  8.22s/it]
  4%|▍         | 1524/37962 [3:34:50<83:20:06,  8.23s/it]
  4%|▍         | 1525/37962 [3:34:59<83:31:56,  8.25s/it]
  4%|▍         | 1526/37962 [3:35:07<84:02:55,  8.30s/it]
  4%|▍         | 1527/37962 [3:35:15<83:58:17,  8.30s/it]
  4%|▍         | 1528/37962 [3:35:23<83:23:50,  8.24s/it]
  4%|▍         | 1529/37962 [3:35:32<83:06:03,  8.21s/it]
  4%|▍         | 1530/37962 [3:35:40<82:48:47,  8.18s/it]
                                                         
{'loss': 1.4966, 'grad_norm': 25.03113555908203, 'learning_rate': 1.999443654403552e-05, 'epoch': 0.04}

  4%|▍         | 1530/37962 [3:35:40<82:48:47,  8.18s/it]
  4%|▍         | 1531/37962 [3:35:48<82:58:18,  8.20s/it]
  4%|▍         | 1532/37962 [3:35:56<83:13:39,  8.22s/it]
  4%|▍         | 1533/37962 [3:36:05<83:48:01,  8.28s/it]
  4%|▍         | 1534/37962 [3:36:13<83:36:31,  8.26s/it]
  4%|▍         | 1535/37962 [3:36:21<83:43:50,  8.27s/it]
  4%|▍         | 1536/37962 [3:36:29<83:39:10,  8.27s/it]
  4%|▍         | 1537/37962 [3:36:38<83:59:51,  8.30s/it]
  4%|▍         | 1538/37962 [3:36:46<84:08:59,  8.32s/it]
  4%|▍         | 1539/37962 [3:36:54<83:35:49,  8.26s/it]
  4%|▍         | 1540/37962 [3:37:03<84:13:00,  8.32s/it]
                                                         
{'loss': 1.6699, 'grad_norm': 66.55901336669922, 'learning_rate': 1.999414835731879e-05, 'epoch': 0.04}

  4%|▍         | 1540/37962 [3:37:03<84:13:00,  8.32s/it]
  4%|▍         | 1541/37962 [3:37:11<84:32:29,  8.36s/it]
  4%|▍         | 1542/37962 [3:37:19<83:27:36,  8.25s/it]
  4%|▍         | 1543/37962 [3:37:27<82:42:25,  8.18s/it]
  4%|▍         | 1544/37962 [3:37:35<82:48:34,  8.19s/it]
  4%|▍         | 1545/37962 [3:37:43<81:38:57,  8.07s/it]
  4%|▍         | 1546/37962 [3:37:51<81:59:50,  8.11s/it]
  4%|▍         | 1547/37962 [3:38:00<82:18:40,  8.14s/it]
  4%|▍         | 1548/37962 [3:38:08<82:29:14,  8.15s/it]
  4%|▍         | 1549/37962 [3:38:16<82:42:51,  8.18s/it]
  4%|▍         | 1550/37962 [3:38:24<82:50:14,  8.19s/it]
                                                         
{'loss': 1.571, 'grad_norm': 9.774673461914062, 'learning_rate': 1.9993852896034603e-05, 'epoch': 0.04}

  4%|▍         | 1550/37962 [3:38:24<82:50:14,  8.19s/it]
  4%|▍         | 1551/37962 [3:38:33<83:19:27,  8.24s/it]
  4%|▍         | 1552/37962 [3:38:41<83:51:34,  8.29s/it]
  4%|▍         | 1553/37962 [3:38:49<83:41:48,  8.28s/it]
  4%|▍         | 1554/37962 [3:38:58<84:46:04,  8.38s/it]
  4%|▍         | 1555/37962 [3:39:06<84:13:24,  8.33s/it]
  4%|▍         | 1556/37962 [3:39:14<83:45:50,  8.28s/it]
  4%|▍         | 1557/37962 [3:39:22<83:00:02,  8.21s/it]
  4%|▍         | 1558/37962 [3:39:30<82:30:10,  8.16s/it]
  4%|▍         | 1559/37962 [3:39:39<82:35:31,  8.17s/it]
  4%|▍         | 1560/37962 [3:39:47<82:36:39,  8.17s/it]
                                                         
{'loss': 1.4993, 'grad_norm': 11.24197769165039, 'learning_rate': 1.9993550160398026e-05, 'epoch': 0.04}

  4%|▍         | 1560/37962 [3:39:47<82:36:39,  8.17s/it]
  4%|▍         | 1561/37962 [3:39:55<82:52:37,  8.20s/it]
  4%|▍         | 1562/37962 [3:40:03<83:35:37,  8.27s/it]
  4%|▍         | 1563/37962 [3:40:12<83:50:38,  8.29s/it]
  4%|▍         | 1564/37962 [3:40:20<84:32:53,  8.36s/it]
  4%|▍         | 1565/37962 [3:40:29<84:27:31,  8.35s/it]
  4%|▍         | 1566/37962 [3:40:37<84:22:11,  8.35s/it]
  4%|▍         | 1567/37962 [3:40:45<84:05:29,  8.32s/it]
  4%|▍         | 1568/37962 [3:40:53<83:14:47,  8.23s/it]
  4%|▍         | 1569/37962 [3:41:02<83:21:24,  8.25s/it]
  4%|▍         | 1570/37962 [3:41:09<82:36:09,  8.17s/it]
                                                         
{'loss': 1.5106, 'grad_norm': 41.2164306640625, 'learning_rate': 1.999324015062941e-05, 'epoch': 0.04}

  4%|▍         | 1570/37962 [3:41:10<82:36:09,  8.17s/it]
  4%|▍         | 1571/37962 [3:41:18<83:31:03,  8.26s/it]
  4%|▍         | 1572/37962 [3:41:26<83:16:04,  8.24s/it]
  4%|▍         | 1573/37962 [3:41:34<83:12:17,  8.23s/it]
  4%|▍         | 1574/37962 [3:41:43<83:05:46,  8.22s/it]
  4%|▍         | 1575/37962 [3:41:51<82:43:17,  8.18s/it]
  4%|▍         | 1576/37962 [3:41:59<82:52:55,  8.20s/it]
  4%|▍         | 1577/37962 [3:42:07<83:22:57,  8.25s/it]
  4%|▍         | 1578/37962 [3:42:16<83:41:48,  8.28s/it]
  4%|▍         | 1579/37962 [3:42:24<83:35:21,  8.27s/it]
  4%|▍         | 1580/37962 [3:42:32<83:30:33,  8.26s/it]
                                                         
{'loss': 1.5473, 'grad_norm': 10.746949195861816, 'learning_rate': 1.9992922866954413e-05, 'epoch': 0.04}

  4%|▍         | 1580/37962 [3:42:32<83:30:33,  8.26s/it]
  4%|▍         | 1581/37962 [3:42:40<82:38:45,  8.18s/it]
  4%|▍         | 1582/37962 [3:42:48<81:44:11,  8.09s/it]
  4%|▍         | 1583/37962 [3:42:56<81:19:09,  8.05s/it]
  4%|▍         | 1584/37962 [3:43:04<82:37:41,  8.18s/it]
  4%|▍         | 1585/37962 [3:43:12<81:24:24,  8.06s/it]
  4%|▍         | 1586/37962 [3:43:20<81:29:45,  8.07s/it]
  4%|▍         | 1587/37962 [3:43:28<81:59:46,  8.12s/it]
  4%|▍         | 1588/37962 [3:43:37<83:08:37,  8.23s/it]
  4%|▍         | 1589/37962 [3:43:45<83:27:17,  8.26s/it]
  4%|▍         | 1590/37962 [3:43:53<83:08:54,  8.23s/it]
                                                         
{'loss': 1.5379, 'grad_norm': 19.454713821411133, 'learning_rate': 1.9992598309603976e-05, 'epoch': 0.04}

  4%|▍         | 1590/37962 [3:43:53<83:08:54,  8.23s/it]
  4%|▍         | 1591/37962 [3:44:02<83:42:02,  8.28s/it]
  4%|▍         | 1592/37962 [3:44:10<83:33:52,  8.27s/it]
  4%|▍         | 1593/37962 [3:44:18<82:51:09,  8.20s/it]
  4%|▍         | 1594/37962 [3:44:26<83:00:04,  8.22s/it]
  4%|▍         | 1595/37962 [3:44:35<83:17:24,  8.24s/it]
  4%|▍         | 1596/37962 [3:44:43<83:23:19,  8.25s/it]
  4%|▍         | 1597/37962 [3:44:51<83:26:02,  8.26s/it]
  4%|▍         | 1598/37962 [3:45:00<83:29:04,  8.26s/it]
  4%|▍         | 1599/37962 [3:45:08<83:15:25,  8.24s/it]
  4%|▍         | 1600/37962 [3:45:16<81:49:15,  8.10s/it]
                                                         
{'loss': 1.5885, 'grad_norm': 33.08735656738281, 'learning_rate': 1.999226647881434e-05, 'epoch': 0.04}

  4%|▍         | 1600/37962 [3:45:16<81:49:15,  8.10s/it]
  4%|▍         | 1601/37962 [3:45:24<81:57:07,  8.11s/it]
  4%|▍         | 1602/37962 [3:45:32<82:32:09,  8.17s/it]
  4%|▍         | 1603/37962 [3:45:40<82:42:43,  8.19s/it]
  4%|▍         | 1604/37962 [3:45:49<83:25:43,  8.26s/it]
  4%|▍         | 1605/37962 [3:45:57<83:25:05,  8.26s/it]
  4%|▍         | 1606/37962 [3:46:05<83:23:19,  8.26s/it]
  4%|▍         | 1607/37962 [3:46:13<83:14:30,  8.24s/it]
  4%|▍         | 1608/37962 [3:46:22<83:39:23,  8.28s/it]
  4%|▍         | 1609/37962 [3:46:30<84:02:49,  8.32s/it]
  4%|▍         | 1610/37962 [3:46:39<84:47:56,  8.40s/it]
                                                         
{'loss': 1.4829, 'grad_norm': 14.927067756652832, 'learning_rate': 1.999192737482704e-05, 'epoch': 0.04}

  4%|▍         | 1610/37962 [3:46:39<84:47:56,  8.40s/it]
  4%|▍         | 1611/37962 [3:46:47<84:31:47,  8.37s/it]
  4%|▍         | 1612/37962 [3:46:55<84:32:16,  8.37s/it]
  4%|▍         | 1613/37962 [3:47:04<84:29:12,  8.37s/it]
  4%|▍         | 1614/37962 [3:47:12<84:05:51,  8.33s/it]
  4%|▍         | 1615/37962 [3:47:20<84:07:13,  8.33s/it]
  4%|▍         | 1616/37962 [3:47:29<83:51:31,  8.31s/it]
  4%|▍         | 1617/37962 [3:47:37<83:18:14,  8.25s/it]
  4%|▍         | 1618/37962 [3:47:45<82:54:00,  8.21s/it]
  4%|▍         | 1619/37962 [3:47:53<83:36:08,  8.28s/it]
  4%|▍         | 1620/37962 [3:48:02<84:41:38,  8.39s/it]
                                                         
{'loss': 1.5836, 'grad_norm': 8.97916030883789, 'learning_rate': 1.9991580997888897e-05, 'epoch': 0.04}

  4%|▍         | 1620/37962 [3:48:02<84:41:38,  8.39s/it]
  4%|▍         | 1621/37962 [3:48:10<84:13:28,  8.34s/it]
  4%|▍         | 1622/37962 [3:48:18<83:51:54,  8.31s/it]
  4%|▍         | 1623/37962 [3:48:27<83:26:12,  8.27s/it]
  4%|▍         | 1624/37962 [3:48:35<83:38:15,  8.29s/it]
  4%|▍         | 1625/37962 [3:48:43<83:26:15,  8.27s/it]
  4%|▍         | 1626/37962 [3:48:51<83:10:20,  8.24s/it]
  4%|▍         | 1627/37962 [3:48:59<83:03:51,  8.23s/it]
  4%|▍         | 1628/37962 [3:49:07<82:17:09,  8.15s/it]
  4%|▍         | 1629/37962 [3:49:16<82:22:57,  8.16s/it]
  4%|▍         | 1630/37962 [3:49:24<81:57:18,  8.12s/it]
                                                         
{'loss': 1.5624, 'grad_norm': 253.7731170654297, 'learning_rate': 1.999122734825204e-05, 'epoch': 0.04}

  4%|▍         | 1630/37962 [3:49:24<81:57:18,  8.12s/it]
  4%|▍         | 1631/37962 [3:49:32<82:24:39,  8.17s/it]
  4%|▍         | 1632/37962 [3:49:40<82:49:39,  8.21s/it]
  4%|▍         | 1633/37962 [3:49:48<82:54:23,  8.22s/it]
  4%|▍         | 1634/37962 [3:49:57<82:55:32,  8.22s/it]
  4%|▍         | 1635/37962 [3:50:05<82:21:07,  8.16s/it]
  4%|▍         | 1636/37962 [3:50:13<82:52:01,  8.21s/it]
  4%|▍         | 1637/37962 [3:50:21<83:16:02,  8.25s/it]
  4%|▍         | 1638/37962 [3:50:29<82:17:07,  8.16s/it]
  4%|▍         | 1639/37962 [3:50:38<83:25:43,  8.27s/it]
  4%|▍         | 1640/37962 [3:50:46<83:43:07,  8.30s/it]
                                                         
{'loss': 22.6966, 'grad_norm': 5653.267578125, 'learning_rate': 1.9990866426173886e-05, 'epoch': 0.04}

  4%|▍         | 1640/37962 [3:50:46<83:43:07,  8.30s/it]
  4%|▍         | 1641/37962 [3:50:54<83:34:47,  8.28s/it]
  4%|▍         | 1642/37962 [3:51:03<83:26:07,  8.27s/it]
  4%|▍         | 1643/37962 [3:51:10<81:47:40,  8.11s/it]
  4%|▍         | 1644/37962 [3:51:19<82:28:36,  8.18s/it]
  4%|▍         | 1645/37962 [3:51:34<103:06:59, 10.22s/it]
  4%|▍         | 1646/37962 [3:51:42<97:11:31,  9.63s/it] 
  4%|▍         | 1647/37962 [3:51:50<92:48:50,  9.20s/it]
  4%|▍         | 1648/37962 [3:51:58<88:23:32,  8.76s/it]
  4%|▍         | 1649/37962 [3:52:06<86:50:08,  8.61s/it]
  4%|▍         | 1650/37962 [3:52:15<85:56:37,  8.52s/it]
                                                         
{'loss': 4.0821, 'grad_norm': 1016.4690551757812, 'learning_rate': 1.9990498231917135e-05, 'epoch': 0.04}

  4%|▍         | 1650/37962 [3:52:15<85:56:37,  8.52s/it]
  4%|▍         | 1651/37962 [3:52:23<85:13:14,  8.45s/it]
  4%|▍         | 1652/37962 [3:52:31<83:44:33,  8.30s/it]
  4%|▍         | 1653/37962 [3:52:39<83:31:57,  8.28s/it]
  4%|▍         | 1654/37962 [3:52:47<83:16:15,  8.26s/it]
  4%|▍         | 1655/37962 [3:52:56<83:46:49,  8.31s/it]
  4%|▍         | 1656/37962 [3:53:04<83:50:40,  8.31s/it]
  4%|▍         | 1657/37962 [3:53:12<83:45:41,  8.31s/it]
  4%|▍         | 1658/37962 [3:53:21<84:33:54,  8.39s/it]
  4%|▍         | 1659/37962 [3:53:29<84:19:46,  8.36s/it]
  4%|▍         | 1660/37962 [3:53:37<84:08:04,  8.34s/it]
                                                         
{'loss': 1.9765, 'grad_norm': 35.97153091430664, 'learning_rate': 1.9990122765749796e-05, 'epoch': 0.04}

  4%|▍         | 1660/37962 [3:53:37<84:08:04,  8.34s/it]
  4%|▍         | 1661/37962 [3:53:46<84:02:34,  8.33s/it]
  4%|▍         | 1662/37962 [3:53:54<83:24:41,  8.27s/it]
  4%|▍         | 1663/37962 [3:54:02<83:37:22,  8.29s/it]
  4%|▍         | 1664/37962 [3:54:10<82:43:37,  8.20s/it]
  4%|▍         | 1665/37962 [3:54:18<82:53:39,  8.22s/it]
  4%|▍         | 1666/37962 [3:54:27<83:09:35,  8.25s/it]
  4%|▍         | 1667/37962 [3:54:35<83:40:39,  8.30s/it]
  4%|▍         | 1668/37962 [3:54:43<82:50:45,  8.22s/it]
  4%|▍         | 1669/37962 [3:54:51<82:37:21,  8.20s/it]
  4%|▍         | 1670/37962 [3:55:00<83:02:55,  8.24s/it]
                                                         
{'loss': 1.8108, 'grad_norm': 198.94776916503906, 'learning_rate': 1.9989740027945164e-05, 'epoch': 0.04}

  4%|▍         | 1670/37962 [3:55:00<83:02:55,  8.24s/it]
  4%|▍         | 1671/37962 [3:55:08<83:56:14,  8.33s/it]
  4%|▍         | 1672/37962 [3:55:17<84:01:25,  8.34s/it]
  4%|▍         | 1673/37962 [3:55:25<83:23:18,  8.27s/it]
  4%|▍         | 1674/37962 [3:55:33<83:41:48,  8.30s/it]
  4%|▍         | 1675/37962 [3:55:41<83:34:34,  8.29s/it]
  4%|▍         | 1676/37962 [3:55:50<83:26:35,  8.28s/it]
  4%|▍         | 1677/37962 [3:55:58<83:00:20,  8.24s/it]
  4%|▍         | 1678/37962 [3:56:06<83:21:06,  8.27s/it]
  4%|▍         | 1679/37962 [3:56:14<83:13:47,  8.26s/it]
  4%|▍         | 1680/37962 [3:56:23<84:00:41,  8.34s/it]
                                                         
{'loss': 1.5637, 'grad_norm': 40.04741668701172, 'learning_rate': 1.9989350018781826e-05, 'epoch': 0.04}

  4%|▍         | 1680/37962 [3:56:23<84:00:41,  8.34s/it]
  4%|▍         | 1681/37962 [3:56:31<83:57:06,  8.33s/it]
  4%|▍         | 1682/37962 [3:56:40<84:04:54,  8.34s/it]
  4%|▍         | 1683/37962 [3:56:48<83:51:58,  8.32s/it]
  4%|▍         | 1684/37962 [3:56:56<83:10:06,  8.25s/it]
  4%|▍         | 1685/37962 [3:57:04<83:23:43,  8.28s/it]
  4%|▍         | 1686/37962 [3:57:12<83:19:38,  8.27s/it]
  4%|▍         | 1687/37962 [3:57:21<83:45:27,  8.31s/it]
  4%|▍         | 1688/37962 [3:57:29<83:32:06,  8.29s/it]
  4%|▍         | 1689/37962 [3:57:37<83:40:21,  8.30s/it]
  4%|▍         | 1690/37962 [3:57:46<83:26:19,  8.28s/it]
                                                         
{'loss': 1.4969, 'grad_norm': 18.696552276611328, 'learning_rate': 1.998895273854366e-05, 'epoch': 0.04}

  4%|▍         | 1690/37962 [3:57:46<83:26:19,  8.28s/it]
  4%|▍         | 1691/37962 [3:57:54<83:03:08,  8.24s/it]
  4%|▍         | 1692/37962 [3:58:02<83:55:37,  8.33s/it]
  4%|▍         | 1693/37962 [3:58:10<82:52:09,  8.23s/it]
  4%|▍         | 1694/37962 [3:58:18<82:30:01,  8.19s/it]
  4%|▍         | 1695/37962 [3:58:27<82:22:54,  8.18s/it]
  4%|▍         | 1696/37962 [3:58:35<83:22:16,  8.28s/it]
  4%|▍         | 1697/37962 [3:58:43<83:16:55,  8.27s/it]
  4%|▍         | 1698/37962 [3:58:51<82:19:13,  8.17s/it]
  4%|▍         | 1699/37962 [3:59:07<103:49:20, 10.31s/it]
  4%|▍         | 1700/37962 [3:59:15<96:51:20,  9.62s/it] 
                                                         
{'loss': 1.6165, 'grad_norm': 15.722201347351074, 'learning_rate': 1.998854818751985e-05, 'epoch': 0.04}

  4%|▍         | 1700/37962 [3:59:15<96:51:20,  9.62s/it]
  4%|▍         | 1701/37962 [3:59:23<92:54:54,  9.22s/it]
  4%|▍         | 1702/37962 [3:59:31<90:16:29,  8.96s/it]
  4%|▍         | 1703/37962 [3:59:40<88:15:49,  8.76s/it]
  4%|▍         | 1704/37962 [3:59:48<86:34:14,  8.60s/it]
  4%|▍         | 1705/37962 [3:59:56<85:03:29,  8.45s/it]
  4%|▍         | 1706/37962 [4:00:04<85:23:24,  8.48s/it]
  4%|▍         | 1707/37962 [4:00:13<85:09:59,  8.46s/it]
  4%|▍         | 1708/37962 [4:00:21<84:24:29,  8.38s/it]
  5%|▍         | 1709/37962 [4:00:29<84:31:47,  8.39s/it]
  5%|▍         | 1710/37962 [4:00:46<109:43:34, 10.90s/it]
                                                          
{'loss': 1.5769, 'grad_norm': 14.195305824279785, 'learning_rate': 1.9988136366004845e-05, 'epoch': 0.05}

  5%|▍         | 1710/37962 [4:00:46<109:43:34, 10.90s/it]
  5%|▍         | 1711/37962 [4:00:54<101:35:48, 10.09s/it]
  5%|▍         | 1712/37962 [4:01:03<96:05:56,  9.54s/it] 
  5%|▍         | 1713/37962 [4:01:11<92:28:35,  9.18s/it]
  5%|▍         | 1714/37962 [4:01:19<89:40:40,  8.91s/it]
  5%|▍         | 1715/37962 [4:01:27<87:22:48,  8.68s/it]
  5%|▍         | 1716/37962 [4:01:36<86:08:54,  8.56s/it]
  5%|▍         | 1717/37962 [4:01:44<84:11:16,  8.36s/it]
  5%|▍         | 1718/37962 [4:01:52<84:04:58,  8.35s/it]
  5%|▍         | 1719/37962 [4:02:00<83:46:48,  8.32s/it]
  5%|▍         | 1720/37962 [4:02:09<83:54:40,  8.34s/it]
                                                         
{'loss': 1.6393, 'grad_norm': 24.94065284729004, 'learning_rate': 1.9987717274298415e-05, 'epoch': 0.05}

  5%|▍         | 1720/37962 [4:02:09<83:54:40,  8.34s/it]
  5%|▍         | 1721/37962 [4:02:16<82:36:39,  8.21s/it]
  5%|▍         | 1722/37962 [4:02:25<83:13:29,  8.27s/it]
  5%|▍         | 1723/37962 [4:02:33<83:17:38,  8.27s/it]
  5%|▍         | 1724/37962 [4:02:50<109:23:26, 10.87s/it]
  5%|▍         | 1725/37962 [4:02:59<102:01:01, 10.13s/it]
  5%|▍         | 1726/37962 [4:03:06<94:41:01,  9.41s/it] 
  5%|▍         | 1727/37962 [4:03:14<91:09:49,  9.06s/it]
  5%|▍         | 1728/37962 [4:03:23<88:47:59,  8.82s/it]
  5%|▍         | 1729/37962 [4:03:31<87:22:54,  8.68s/it]
  5%|▍         | 1730/37962 [4:03:39<86:23:56,  8.58s/it]
                                                         
{'loss': 1.68, 'grad_norm': 27.443214416503906, 'learning_rate': 1.9987290912705607e-05, 'epoch': 0.05}

  5%|▍         | 1730/37962 [4:03:39<86:23:56,  8.58s/it]
  5%|▍         | 1731/37962 [4:03:48<85:11:22,  8.46s/it]
  5%|▍         | 1732/37962 [4:03:56<84:33:22,  8.40s/it]
  5%|▍         | 1733/37962 [4:04:04<84:13:05,  8.37s/it]
  5%|▍         | 1734/37962 [4:04:13<84:36:32,  8.41s/it]
  5%|▍         | 1735/37962 [4:04:21<84:35:04,  8.41s/it]
  5%|▍         | 1736/37962 [4:04:29<84:07:28,  8.36s/it]
  5%|▍         | 1737/37962 [4:04:45<104:58:04, 10.43s/it]
  5%|▍         | 1738/37962 [4:04:53<97:32:30,  9.69s/it] 
  5%|▍         | 1739/37962 [4:05:01<93:17:38,  9.27s/it]
  5%|▍         | 1740/37962 [4:05:09<90:04:26,  8.95s/it]
                                                         
{'loss': 1.5109, 'grad_norm': 21.934724807739258, 'learning_rate': 1.9986857281536763e-05, 'epoch': 0.05}

  5%|▍         | 1740/37962 [4:05:09<90:04:26,  8.95s/it]
  5%|▍         | 1741/37962 [4:05:17<88:02:13,  8.75s/it]
  5%|▍         | 1742/37962 [4:05:26<86:57:48,  8.64s/it]
  5%|▍         | 1743/37962 [4:05:34<86:37:27,  8.61s/it]
  5%|▍         | 1744/37962 [4:05:43<86:01:31,  8.55s/it]
  5%|▍         | 1745/37962 [4:05:51<85:32:14,  8.50s/it]
  5%|▍         | 1746/37962 [4:05:59<85:00:57,  8.45s/it]
  5%|▍         | 1747/37962 [4:06:08<84:25:39,  8.39s/it]
  5%|▍         | 1748/37962 [4:06:16<83:53:52,  8.34s/it]
  5%|▍         | 1749/37962 [4:06:24<84:04:15,  8.36s/it]
  5%|▍         | 1750/37962 [4:06:32<83:34:40,  8.31s/it]
                                                         
{'loss': 1.6754, 'grad_norm': 11.89421272277832, 'learning_rate': 1.9986416381107512e-05, 'epoch': 0.05}

  5%|▍         | 1750/37962 [4:06:32<83:34:40,  8.31s/it]
  5%|▍         | 1751/37962 [4:06:41<83:27:01,  8.30s/it]
  5%|▍         | 1752/37962 [4:06:56<103:31:29, 10.29s/it]
  5%|▍         | 1753/37962 [4:07:04<97:00:30,  9.64s/it] 
  5%|▍         | 1754/37962 [4:07:12<93:49:18,  9.33s/it]
  5%|▍         | 1755/37962 [4:07:21<90:19:19,  8.98s/it]
  5%|▍         | 1756/37962 [4:07:29<87:18:27,  8.68s/it]
  5%|▍         | 1757/37962 [4:07:37<86:26:03,  8.59s/it]
  5%|▍         | 1758/37962 [4:07:45<85:20:19,  8.49s/it]
  5%|▍         | 1759/37962 [4:07:53<84:32:02,  8.41s/it]
  5%|▍         | 1760/37962 [4:08:02<84:27:53,  8.40s/it]
                                                         
{'loss': 1.6331, 'grad_norm': 27.342628479003906, 'learning_rate': 1.9985968211738783e-05, 'epoch': 0.05}

  5%|▍         | 1760/37962 [4:08:02<84:27:53,  8.40s/it]
  5%|▍         | 1761/37962 [4:08:10<83:23:04,  8.29s/it]
  5%|▍         | 1762/37962 [4:08:18<83:27:24,  8.30s/it]
  5%|▍         | 1763/37962 [4:08:26<83:36:57,  8.32s/it]
  5%|▍         | 1764/37962 [4:08:35<83:27:11,  8.30s/it]
  5%|▍         | 1765/37962 [4:08:43<83:21:25,  8.29s/it]
  5%|▍         | 1766/37962 [4:08:51<82:31:11,  8.21s/it]
  5%|▍         | 1767/37962 [4:09:08<108:38:31, 10.81s/it]
  5%|▍         | 1768/37962 [4:09:16<101:43:05, 10.12s/it]
  5%|▍         | 1769/37962 [4:09:25<96:15:57,  9.58s/it] 
  5%|▍         | 1770/37962 [4:09:33<91:47:57,  9.13s/it]
                                                         
{'loss': 1.6099, 'grad_norm': 3.336338520050049, 'learning_rate': 1.9985512773756784e-05, 'epoch': 0.05}

  5%|▍         | 1770/37962 [4:09:33<91:47:57,  9.13s/it]
  5%|▍         | 1771/37962 [4:09:41<87:54:35,  8.74s/it]
  5%|▍         | 1772/37962 [4:09:49<85:44:34,  8.53s/it]
  5%|▍         | 1773/37962 [4:09:57<85:03:31,  8.46s/it]
  5%|▍         | 1774/37962 [4:10:05<84:54:18,  8.45s/it]
  5%|▍         | 1775/37962 [4:10:14<84:08:18,  8.37s/it]
  5%|▍         | 1776/37962 [4:10:22<83:28:44,  8.30s/it]
  5%|▍         | 1777/37962 [4:10:30<83:07:40,  8.27s/it]
  5%|▍         | 1778/37962 [4:10:38<83:23:24,  8.30s/it]
  5%|▍         | 1779/37962 [4:10:46<82:59:51,  8.26s/it]
  5%|▍         | 1780/37962 [4:10:55<83:12:34,  8.28s/it]
                                                         
{'loss': 1.5391, 'grad_norm': 33.684730529785156, 'learning_rate': 1.9985050067493028e-05, 'epoch': 0.05}

  5%|▍         | 1780/37962 [4:10:55<83:12:34,  8.28s/it]
  5%|▍         | 1781/37962 [4:11:03<83:22:21,  8.30s/it]
  5%|▍         | 1782/37962 [4:11:12<84:17:10,  8.39s/it]
  5%|▍         | 1783/37962 [4:11:29<110:25:26, 10.99s/it]
  5%|▍         | 1784/37962 [4:11:37<102:18:57, 10.18s/it]
  5%|▍         | 1785/37962 [4:11:45<96:29:25,  9.60s/it] 
  5%|▍         | 1786/37962 [4:11:53<91:06:49,  9.07s/it]
  5%|▍         | 1787/37962 [4:12:01<87:50:45,  8.74s/it]
  5%|▍         | 1788/37962 [4:12:09<84:40:15,  8.43s/it]
  5%|▍         | 1789/37962 [4:12:17<84:29:32,  8.41s/it]
  5%|▍         | 1790/37962 [4:12:25<83:15:13,  8.29s/it]
                                                         
{'loss': 1.4882, 'grad_norm': 16.756601333618164, 'learning_rate': 1.9984580093284305e-05, 'epoch': 0.05}

  5%|▍         | 1790/37962 [4:12:25<83:15:13,  8.29s/it]
  5%|▍         | 1791/37962 [4:12:34<83:59:12,  8.36s/it]
  5%|▍         | 1792/37962 [4:12:42<83:30:21,  8.31s/it]
  5%|▍         | 1793/37962 [4:12:50<82:56:14,  8.25s/it]
  5%|▍         | 1794/37962 [4:12:59<83:52:32,  8.35s/it]
  5%|▍         | 1795/37962 [4:13:07<84:02:27,  8.37s/it]
  5%|▍         | 1796/37962 [4:13:15<83:42:32,  8.33s/it]
  5%|▍         | 1797/37962 [4:13:23<83:20:58,  8.30s/it]
  5%|▍         | 1798/37962 [4:13:32<83:28:39,  8.31s/it]
  5%|▍         | 1799/37962 [4:13:40<83:53:28,  8.35s/it]
  5%|▍         | 1800/37962 [4:13:49<83:54:54,  8.35s/it]
                                                         
{'loss': 1.6272, 'grad_norm': 12.322381019592285, 'learning_rate': 1.99841028514727e-05, 'epoch': 0.05}

  5%|▍         | 1800/37962 [4:13:49<83:54:54,  8.35s/it]
  5%|▍         | 1801/37962 [4:13:57<83:32:39,  8.32s/it]
  5%|▍         | 1802/37962 [4:14:05<83:49:40,  8.35s/it]
  5%|▍         | 1803/37962 [4:14:14<83:33:02,  8.32s/it]
  5%|▍         | 1804/37962 [4:14:22<83:19:20,  8.30s/it]
  5%|▍         | 1805/37962 [4:14:30<83:19:50,  8.30s/it]
  5%|▍         | 1806/37962 [4:14:38<83:36:30,  8.32s/it]
  5%|▍         | 1807/37962 [4:14:47<83:19:56,  8.30s/it]
  5%|▍         | 1808/37962 [4:14:55<83:18:00,  8.29s/it]
  5%|▍         | 1809/37962 [4:15:03<83:53:24,  8.35s/it]
  5%|▍         | 1810/37962 [4:15:12<83:49:03,  8.35s/it]
                                                         
{'loss': 1.6318, 'grad_norm': 33.99985885620117, 'learning_rate': 1.9983618342405593e-05, 'epoch': 0.05}

  5%|▍         | 1810/37962 [4:15:12<83:49:03,  8.35s/it]
  5%|▍         | 1811/37962 [4:15:20<83:00:38,  8.27s/it]
  5%|▍         | 1812/37962 [4:15:28<82:37:55,  8.23s/it]
  5%|▍         | 1813/37962 [4:15:36<82:30:57,  8.22s/it]
  5%|▍         | 1814/37962 [4:15:44<82:41:06,  8.23s/it]
  5%|▍         | 1815/37962 [4:15:53<82:42:19,  8.24s/it]
  5%|▍         | 1816/37962 [4:16:01<83:37:06,  8.33s/it]
  5%|▍         | 1817/37962 [4:16:09<83:06:09,  8.28s/it]
  5%|▍         | 1818/37962 [4:16:18<82:44:39,  8.24s/it]
  5%|▍         | 1819/37962 [4:16:26<82:36:32,  8.23s/it]
  5%|▍         | 1820/37962 [4:16:34<82:49:15,  8.25s/it]
                                                         
{'loss': 1.6623, 'grad_norm': 535.0435180664062, 'learning_rate': 1.9983126566435655e-05, 'epoch': 0.05}

  5%|▍         | 1820/37962 [4:16:34<82:49:15,  8.25s/it]
  5%|▍         | 1821/37962 [4:16:42<82:45:00,  8.24s/it]
  5%|▍         | 1822/37962 [4:16:50<81:47:06,  8.15s/it]
  5%|▍         | 1823/37962 [4:16:58<81:50:12,  8.15s/it]
  5%|▍         | 1824/37962 [4:17:07<82:21:53,  8.21s/it]
  5%|▍         | 1825/37962 [4:17:15<83:28:36,  8.32s/it]
  5%|▍         | 1826/37962 [4:17:24<83:35:50,  8.33s/it]
  5%|▍         | 1827/37962 [4:17:32<83:53:09,  8.36s/it]
  5%|▍         | 1828/37962 [4:17:40<83:34:40,  8.33s/it]
  5%|▍         | 1829/37962 [4:17:48<82:35:37,  8.23s/it]
  5%|▍         | 1830/37962 [4:17:57<82:35:25,  8.23s/it]
                                                         
{'loss': 1.5772, 'grad_norm': 22.206634521484375, 'learning_rate': 1.998262752392083e-05, 'epoch': 0.05}

  5%|▍         | 1830/37962 [4:17:57<82:35:25,  8.23s/it]
  5%|▍         | 1831/37962 [4:18:05<82:50:48,  8.25s/it]
  5%|▍         | 1832/37962 [4:18:13<82:53:05,  8.26s/it]
  5%|▍         | 1833/37962 [4:18:22<83:52:08,  8.36s/it]
  5%|▍         | 1834/37962 [4:18:30<84:24:05,  8.41s/it]
  5%|▍         | 1835/37962 [4:18:39<83:51:17,  8.36s/it]
  5%|▍         | 1836/37962 [4:18:47<83:19:51,  8.30s/it]
  5%|▍         | 1837/37962 [4:18:55<83:00:04,  8.27s/it]
  5%|▍         | 1838/37962 [4:19:03<82:59:18,  8.27s/it]
  5%|▍         | 1839/37962 [4:19:11<82:13:56,  8.20s/it]
  5%|▍         | 1840/37962 [4:19:19<80:55:10,  8.06s/it]
                                                         
{'loss': 1.5535, 'grad_norm': 16.05791473388672, 'learning_rate': 1.9982121215224367e-05, 'epoch': 0.05}

  5%|▍         | 1840/37962 [4:19:19<80:55:10,  8.06s/it]
  5%|▍         | 1841/37962 [4:19:27<81:44:06,  8.15s/it]
  5%|▍         | 1842/37962 [4:19:36<83:15:20,  8.30s/it]
  5%|▍         | 1843/37962 [4:19:44<82:09:17,  8.19s/it]
  5%|▍         | 1844/37962 [4:19:52<82:44:47,  8.25s/it]
  5%|▍         | 1845/37962 [4:20:01<82:52:07,  8.26s/it]
  5%|▍         | 1846/37962 [4:20:09<83:21:20,  8.31s/it]
  5%|▍         | 1847/37962 [4:20:17<83:01:46,  8.28s/it]
  5%|▍         | 1848/37962 [4:20:25<82:53:46,  8.26s/it]
  5%|▍         | 1849/37962 [4:20:34<82:47:06,  8.25s/it]
  5%|▍         | 1850/37962 [4:20:42<82:44:39,  8.25s/it]
                                                         
{'loss': 1.4902, 'grad_norm': 9.760551452636719, 'learning_rate': 1.9981607640714804e-05, 'epoch': 0.05}

  5%|▍         | 1850/37962 [4:20:42<82:44:39,  8.25s/it]
  5%|▍         | 1851/37962 [4:20:50<83:07:01,  8.29s/it]
  5%|▍         | 1852/37962 [4:20:58<83:02:08,  8.28s/it]
  5%|▍         | 1853/37962 [4:21:07<83:05:43,  8.28s/it]
  5%|▍         | 1854/37962 [4:21:15<82:21:08,  8.21s/it]
  5%|▍         | 1855/37962 [4:21:23<83:01:06,  8.28s/it]
  5%|▍         | 1856/37962 [4:21:32<82:57:15,  8.27s/it]
  5%|▍         | 1857/37962 [4:21:40<82:54:49,  8.27s/it]
  5%|▍         | 1858/37962 [4:21:48<83:21:41,  8.31s/it]
  5%|▍         | 1859/37962 [4:21:57<83:32:00,  8.33s/it]
  5%|▍         | 1860/37962 [4:22:05<83:45:46,  8.35s/it]
                                                         
{'loss': 1.4701, 'grad_norm': 17.269702911376953, 'learning_rate': 1.9981086800765957e-05, 'epoch': 0.05}

  5%|▍         | 1860/37962 [4:22:05<83:45:46,  8.35s/it]
  5%|▍         | 1861/37962 [4:22:13<83:55:32,  8.37s/it]
  5%|▍         | 1862/37962 [4:22:21<83:01:23,  8.28s/it]
  5%|▍         | 1863/37962 [4:22:30<82:42:21,  8.25s/it]
  5%|▍         | 1864/37962 [4:22:38<83:15:08,  8.30s/it]
  5%|▍         | 1865/37962 [4:22:46<83:17:17,  8.31s/it]
  5%|▍         | 1866/37962 [4:22:54<82:41:25,  8.25s/it]
  5%|▍         | 1867/37962 [4:23:03<82:08:04,  8.19s/it]
  5%|▍         | 1868/37962 [4:23:11<81:43:58,  8.15s/it]
  5%|▍         | 1869/37962 [4:23:19<82:19:43,  8.21s/it]
  5%|▍         | 1870/37962 [4:23:27<82:04:32,  8.19s/it]
                                                         
{'loss': 1.6399, 'grad_norm': 44.855865478515625, 'learning_rate': 1.9980558695756937e-05, 'epoch': 0.05}

  5%|▍         | 1870/37962 [4:23:27<82:04:32,  8.19s/it]
  5%|▍         | 1871/37962 [4:23:35<82:39:37,  8.25s/it]
  5%|▍         | 1872/37962 [4:23:44<82:25:29,  8.22s/it]
  5%|▍         | 1873/37962 [4:23:52<82:27:54,  8.23s/it]
  5%|▍         | 1874/37962 [4:24:00<83:01:48,  8.28s/it]
  5%|▍         | 1875/37962 [4:24:09<83:20:03,  8.31s/it]
  5%|▍         | 1876/37962 [4:24:17<83:34:21,  8.34s/it]
  5%|▍         | 1877/37962 [4:24:25<83:31:38,  8.33s/it]
  5%|▍         | 1878/37962 [4:24:34<83:11:59,  8.30s/it]
  5%|▍         | 1879/37962 [4:24:42<83:00:04,  8.28s/it]
  5%|▍         | 1880/37962 [4:24:50<83:24:08,  8.32s/it]
                                                         
{'loss': 1.5269, 'grad_norm': 29.226150512695312, 'learning_rate': 1.9980023326072145e-05, 'epoch': 0.05}

  5%|▍         | 1880/37962 [4:24:50<83:24:08,  8.32s/it]
  5%|▍         | 1881/37962 [4:24:58<82:39:06,  8.25s/it]
  5%|▍         | 1882/37962 [4:25:07<83:15:18,  8.31s/it]
  5%|▍         | 1883/37962 [4:25:15<83:16:02,  8.31s/it]
  5%|▍         | 1884/37962 [4:25:23<83:02:22,  8.29s/it]
  5%|▍         | 1885/37962 [4:25:32<83:15:48,  8.31s/it]
  5%|▍         | 1886/37962 [4:25:40<84:01:22,  8.38s/it]
  5%|▍         | 1887/37962 [4:25:49<83:42:14,  8.35s/it]
  5%|▍         | 1888/37962 [4:25:57<83:39:54,  8.35s/it]
  5%|▍         | 1889/37962 [4:26:05<83:23:36,  8.32s/it]
  5%|▍         | 1890/37962 [4:26:13<83:09:06,  8.30s/it]
                                                         
{'loss': 1.5901, 'grad_norm': 22.832210540771484, 'learning_rate': 1.9979480692101263e-05, 'epoch': 0.05}

  5%|▍         | 1890/37962 [4:26:13<83:09:06,  8.30s/it]
  5%|▍         | 1891/37962 [4:26:22<82:51:57,  8.27s/it]
  5%|▍         | 1892/37962 [4:26:30<81:59:31,  8.18s/it]
  5%|▍         | 1893/37962 [4:26:38<82:25:33,  8.23s/it]
  5%|▍         | 1894/37962 [4:26:46<82:55:57,  8.28s/it]
  5%|▍         | 1895/37962 [4:26:55<82:48:21,  8.27s/it]
  5%|▍         | 1896/37962 [4:27:03<83:22:40,  8.32s/it]
  5%|▍         | 1897/37962 [4:27:11<83:05:46,  8.29s/it]
  5%|▍         | 1898/37962 [4:27:19<82:56:44,  8.28s/it]
  5%|▌         | 1899/37962 [4:27:28<82:49:40,  8.27s/it]
  5%|▌         | 1900/37962 [4:27:36<82:49:21,  8.27s/it]
                                                         
{'loss': 1.5673, 'grad_norm': 48.36526107788086, 'learning_rate': 1.9978930794239272e-05, 'epoch': 0.05}

  5%|▌         | 1900/37962 [4:27:36<82:49:21,  8.27s/it]
  5%|▌         | 1901/37962 [4:27:44<83:20:49,  8.32s/it]
  5%|▌         | 1902/37962 [4:27:53<82:53:28,  8.28s/it]
  5%|▌         | 1903/37962 [4:28:01<83:02:05,  8.29s/it]
  5%|▌         | 1904/37962 [4:28:09<83:03:22,  8.29s/it]
  5%|▌         | 1905/37962 [4:28:17<82:56:53,  8.28s/it]
  5%|▌         | 1906/37962 [4:28:26<83:33:17,  8.34s/it]
  5%|▌         | 1907/37962 [4:28:34<82:41:20,  8.26s/it]
  5%|▌         | 1908/37962 [4:28:42<82:09:14,  8.20s/it]
  5%|▌         | 1909/37962 [4:28:50<80:57:00,  8.08s/it]
  5%|▌         | 1910/37962 [4:28:58<81:03:07,  8.09s/it]
                                                         
{'loss': 1.5328, 'grad_norm': 21.323749542236328, 'learning_rate': 1.9978373632886424e-05, 'epoch': 0.05}

  5%|▌         | 1910/37962 [4:28:58<81:03:07,  8.09s/it]
  5%|▌         | 1911/37962 [4:29:06<81:33:53,  8.14s/it]
  5%|▌         | 1912/37962 [4:29:14<81:42:34,  8.16s/it]
  5%|▌         | 1913/37962 [4:29:23<82:40:23,  8.26s/it]
  5%|▌         | 1914/37962 [4:29:31<81:53:15,  8.18s/it]
  5%|▌         | 1915/37962 [4:29:39<82:26:42,  8.23s/it]
  5%|▌         | 1916/37962 [4:29:48<82:23:34,  8.23s/it]
  5%|▌         | 1917/37962 [4:29:56<82:25:29,  8.23s/it]
  5%|▌         | 1918/37962 [4:30:04<82:47:51,  8.27s/it]
  5%|▌         | 1919/37962 [4:30:12<82:41:37,  8.26s/it]
  5%|▌         | 1920/37962 [4:30:20<82:17:16,  8.22s/it]
                                                         
{'loss': 1.3632, 'grad_norm': 45.57045364379883, 'learning_rate': 1.9977809208448278e-05, 'epoch': 0.05}

  5%|▌         | 1920/37962 [4:30:20<82:17:16,  8.22s/it]
  5%|▌         | 1921/37962 [4:30:28<80:49:39,  8.07s/it]
  5%|▌         | 1922/37962 [4:30:36<81:17:11,  8.12s/it]
  5%|▌         | 1923/37962 [4:30:45<81:53:47,  8.18s/it]
  5%|▌         | 1924/37962 [4:30:53<81:14:43,  8.12s/it]
  5%|▌         | 1925/37962 [4:31:01<82:16:24,  8.22s/it]
  5%|▌         | 1926/37962 [4:31:10<82:47:26,  8.27s/it]
  5%|▌         | 1927/37962 [4:31:18<82:31:51,  8.25s/it]
  5%|▌         | 1928/37962 [4:31:26<81:57:13,  8.19s/it]
  5%|▌         | 1929/37962 [4:31:34<81:18:03,  8.12s/it]
  5%|▌         | 1930/37962 [4:31:42<82:02:23,  8.20s/it]
                                                         
{'loss': 1.6033, 'grad_norm': 17.159040451049805, 'learning_rate': 1.997723752133566e-05, 'epoch': 0.05}

  5%|▌         | 1930/37962 [4:31:42<82:02:23,  8.20s/it]
  5%|▌         | 1931/37962 [4:31:50<82:17:12,  8.22s/it]
  5%|▌         | 1932/37962 [4:31:59<82:26:11,  8.24s/it]
  5%|▌         | 1933/37962 [4:32:07<82:48:34,  8.27s/it]
  5%|▌         | 1934/37962 [4:32:15<82:54:57,  8.29s/it]
  5%|▌         | 1935/37962 [4:32:24<82:39:21,  8.26s/it]
  5%|▌         | 1936/37962 [4:32:32<83:05:55,  8.30s/it]
  5%|▌         | 1937/37962 [4:32:40<82:36:38,  8.26s/it]
  5%|▌         | 1938/37962 [4:32:48<82:38:48,  8.26s/it]
  5%|▌         | 1939/37962 [4:32:57<82:31:00,  8.25s/it]
  5%|▌         | 1940/37962 [4:33:05<82:38:05,  8.26s/it]
                                                         
{'loss': 1.6035, 'grad_norm': 26.004825592041016, 'learning_rate': 1.9976658571964693e-05, 'epoch': 0.05}

  5%|▌         | 1940/37962 [4:33:05<82:38:05,  8.26s/it]
  5%|▌         | 1941/37962 [4:33:13<82:55:29,  8.29s/it]
  5%|▌         | 1942/37962 [4:33:21<82:05:03,  8.20s/it]
  5%|▌         | 1943/37962 [4:33:30<82:42:10,  8.27s/it]
  5%|▌         | 1944/37962 [4:33:38<82:51:03,  8.28s/it]
  5%|▌         | 1945/37962 [4:33:46<82:23:13,  8.23s/it]
  5%|▌         | 1946/37962 [4:33:54<82:22:27,  8.23s/it]
  5%|▌         | 1947/37962 [4:34:03<83:04:43,  8.30s/it]
  5%|▌         | 1948/37962 [4:34:11<82:35:09,  8.26s/it]
  5%|▌         | 1949/37962 [4:34:19<82:30:29,  8.25s/it]
  5%|▌         | 1950/37962 [4:34:27<82:38:34,  8.26s/it]
                                                         
{'loss': 1.511, 'grad_norm': 36.11913299560547, 'learning_rate': 1.9976072360756787e-05, 'epoch': 0.05}

  5%|▌         | 1950/37962 [4:34:27<82:38:34,  8.26s/it]
  5%|▌         | 1951/37962 [4:34:36<82:54:55,  8.29s/it]
  5%|▌         | 1952/37962 [4:34:44<82:35:51,  8.26s/it]
  5%|▌         | 1953/37962 [4:34:52<82:00:53,  8.20s/it]
  5%|▌         | 1954/37962 [4:35:00<82:27:42,  8.24s/it]
  5%|▌         | 1955/37962 [4:35:09<82:49:26,  8.28s/it]
  5%|▌         | 1956/37962 [4:35:17<82:56:13,  8.29s/it]
  5%|▌         | 1957/37962 [4:35:25<82:37:37,  8.26s/it]
  5%|▌         | 1958/37962 [4:35:34<83:02:55,  8.30s/it]
  5%|▌         | 1959/37962 [4:35:42<82:49:41,  8.28s/it]
  5%|▌         | 1960/37962 [4:35:50<82:21:20,  8.24s/it]
                                                         
{'loss': 1.6204, 'grad_norm': 20.601947784423828, 'learning_rate': 1.997547888813863e-05, 'epoch': 0.05}

  5%|▌         | 1960/37962 [4:35:50<82:21:20,  8.24s/it]
  5%|▌         | 1961/37962 [4:35:58<82:48:37,  8.28s/it]
  5%|▌         | 1962/37962 [4:36:07<82:38:13,  8.26s/it]
  5%|▌         | 1963/37962 [4:36:15<82:36:32,  8.26s/it]
  5%|▌         | 1964/37962 [4:36:23<83:00:18,  8.30s/it]
  5%|▌         | 1965/37962 [4:36:32<82:49:03,  8.28s/it]
  5%|▌         | 1966/37962 [4:36:40<83:14:44,  8.33s/it]
  5%|▌         | 1967/37962 [4:36:48<83:32:14,  8.35s/it]
  5%|▌         | 1968/37962 [4:36:57<83:11:02,  8.32s/it]
  5%|▌         | 1969/37962 [4:37:05<83:13:45,  8.32s/it]
  5%|▌         | 1970/37962 [4:37:13<82:36:23,  8.26s/it]
                                                         
{'loss': 1.5366, 'grad_norm': 75.61493682861328, 'learning_rate': 1.997487815454221e-05, 'epoch': 0.05}

  5%|▌         | 1970/37962 [4:37:13<82:36:23,  8.26s/it]
  5%|▌         | 1971/37962 [4:37:22<83:17:49,  8.33s/it]
  5%|▌         | 1972/37962 [4:37:30<82:18:51,  8.23s/it]
  5%|▌         | 1973/37962 [4:37:38<82:16:11,  8.23s/it]
  5%|▌         | 1974/37962 [4:37:46<82:26:20,  8.25s/it]
  5%|▌         | 1975/37962 [4:37:55<83:13:22,  8.33s/it]
  5%|▌         | 1976/37962 [4:38:03<83:21:53,  8.34s/it]
  5%|▌         | 1977/37962 [4:38:11<82:12:34,  8.22s/it]
  5%|▌         | 1978/37962 [4:38:19<82:22:33,  8.24s/it]
  5%|▌         | 1979/37962 [4:38:27<81:32:47,  8.16s/it]
  5%|▌         | 1980/37962 [4:38:35<81:29:33,  8.15s/it]
                                                         
{'loss': 1.5607, 'grad_norm': 18.20416831970215, 'learning_rate': 1.997427016040478e-05, 'epoch': 0.05}

  5%|▌         | 1980/37962 [4:38:35<81:29:33,  8.15s/it]
  5%|▌         | 1981/37962 [4:38:44<81:55:00,  8.20s/it]
  5%|▌         | 1982/37962 [4:38:52<81:41:48,  8.17s/it]
  5%|▌         | 1983/37962 [4:39:00<82:40:50,  8.27s/it]
  5%|▌         | 1984/37962 [4:39:09<82:48:57,  8.29s/it]
  5%|▌         | 1985/37962 [4:39:17<83:09:24,  8.32s/it]
  5%|▌         | 1986/37962 [4:39:25<83:22:57,  8.34s/it]
  5%|▌         | 1987/37962 [4:39:34<83:01:53,  8.31s/it]
  5%|▌         | 1988/37962 [4:39:42<82:05:55,  8.22s/it]
  5%|▌         | 1989/37962 [4:39:50<82:39:03,  8.27s/it]
  5%|▌         | 1990/37962 [4:39:58<82:57:10,  8.30s/it]
                                                         
{'loss': 1.4479, 'grad_norm': 20.338754653930664, 'learning_rate': 1.9973654906168894e-05, 'epoch': 0.05}

  5%|▌         | 1990/37962 [4:39:58<82:57:10,  8.30s/it]
  5%|▌         | 1991/37962 [4:40:06<81:02:35,  8.11s/it]
  5%|▌         | 1992/37962 [4:40:14<81:23:37,  8.15s/it]
  5%|▌         | 1993/37962 [4:40:23<81:51:49,  8.19s/it]
  5%|▌         | 1994/37962 [4:40:31<82:18:38,  8.24s/it]
  5%|▌         | 1995/37962 [4:40:39<82:01:58,  8.21s/it]
  5%|▌         | 1996/37962 [4:40:47<82:00:48,  8.21s/it]
  5%|▌         | 1997/37962 [4:40:55<81:38:28,  8.17s/it]
  5%|▌         | 1998/37962 [4:41:04<82:23:14,  8.25s/it]
  5%|▌         | 1999/37962 [4:41:12<82:13:55,  8.23s/it]
  5%|▌         | 2000/37962 [4:41:20<81:31:49,  8.16s/it]
                                                         
{'loss': 1.5047, 'grad_norm': 48.998695373535156, 'learning_rate': 1.9973032392282382e-05, 'epoch': 0.05}

  5%|▌         | 2000/37962 [4:41:20<81:31:49,  8.16s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

  5%|▌         | 2001/37962 [4:42:11<210:27:05, 21.07s/it]
  5%|▌         | 2002/37962 [4:42:19<172:08:06, 17.23s/it]
  5%|▌         | 2003/37962 [4:42:27<144:02:05, 14.42s/it]
  5%|▌         | 2004/37962 [4:42:35<125:09:02, 12.53s/it]
  5%|▌         | 2005/37962 [4:42:44<112:24:29, 11.25s/it]
  5%|▌         | 2006/37962 [4:42:52<103:52:21, 10.40s/it]
  5%|▌         | 2007/37962 [4:43:01<97:58:08,  9.81s/it] 
  5%|▌         | 2008/37962 [4:43:08<92:25:38,  9.25s/it]
  5%|▌         | 2009/37962 [4:43:17<89:41:50,  8.98s/it]
  5%|▌         | 2010/37962 [4:43:25<87:50:30,  8.80s/it]
                                                         
{'loss': 1.6015, 'grad_norm': 60.957462310791016, 'learning_rate': 1.9972402619198365e-05, 'epoch': 0.05}

  5%|▌         | 2010/37962 [4:43:25<87:50:30,  8.80s/it]
  5%|▌         | 2011/37962 [4:43:33<86:09:37,  8.63s/it]
  5%|▌         | 2012/37962 [4:43:42<85:07:56,  8.53s/it]
  5%|▌         | 2013/37962 [4:43:50<84:11:48,  8.43s/it]
  5%|▌         | 2014/37962 [4:43:58<83:46:44,  8.39s/it]
  5%|▌         | 2015/37962 [4:44:07<83:53:09,  8.40s/it]
  5%|▌         | 2016/37962 [4:44:15<83:39:16,  8.38s/it]
  5%|▌         | 2017/37962 [4:44:23<82:01:06,  8.21s/it]
  5%|▌         | 2018/37962 [4:44:31<82:54:48,  8.30s/it]
  5%|▌         | 2019/37962 [4:44:40<82:44:23,  8.29s/it]
  5%|▌         | 2020/37962 [4:44:48<82:17:33,  8.24s/it]
                                                         
{'loss': 1.7273, 'grad_norm': 50.907962799072266, 'learning_rate': 1.9971765587375236e-05, 'epoch': 0.05}

  5%|▌         | 2020/37962 [4:44:48<82:17:33,  8.24s/it]
  5%|▌         | 2021/37962 [4:44:56<81:24:43,  8.15s/it]
  5%|▌         | 2022/37962 [4:45:04<81:23:02,  8.15s/it]
  5%|▌         | 2023/37962 [4:45:12<80:41:06,  8.08s/it]
  5%|▌         | 2024/37962 [4:45:20<81:55:09,  8.21s/it]
  5%|▌         | 2025/37962 [4:45:28<81:59:03,  8.21s/it]
  5%|▌         | 2026/37962 [4:45:37<81:54:08,  8.20s/it]
  5%|▌         | 2027/37962 [4:45:45<82:01:35,  8.22s/it]
  5%|▌         | 2028/37962 [4:45:53<82:12:13,  8.24s/it]
  5%|▌         | 2029/37962 [4:46:01<82:17:32,  8.24s/it]
  5%|▌         | 2030/37962 [4:46:10<82:30:36,  8.27s/it]
                                                         
{'loss': 1.5959, 'grad_norm': 29.735641479492188, 'learning_rate': 1.997112129727669e-05, 'epoch': 0.05}

  5%|▌         | 2030/37962 [4:46:10<82:30:36,  8.27s/it]
  5%|▌         | 2031/37962 [4:46:18<82:24:33,  8.26s/it]
  5%|▌         | 2032/37962 [4:46:26<81:42:26,  8.19s/it]
  5%|▌         | 2033/37962 [4:46:34<82:13:52,  8.24s/it]
  5%|▌         | 2034/37962 [4:46:43<82:28:06,  8.26s/it]
  5%|▌         | 2035/37962 [4:46:51<82:34:45,  8.27s/it]
  5%|▌         | 2036/37962 [4:46:59<82:36:53,  8.28s/it]
  5%|▌         | 2037/37962 [4:47:08<83:14:25,  8.34s/it]
  5%|▌         | 2038/37962 [4:47:16<83:07:16,  8.33s/it]
  5%|▌         | 2039/37962 [4:47:24<82:13:38,  8.24s/it]
  5%|▌         | 2040/37962 [4:47:32<81:53:17,  8.21s/it]
                                                         
{'loss': 1.5412, 'grad_norm': 27.526500701904297, 'learning_rate': 1.9970469749371688e-05, 'epoch': 0.05}

  5%|▌         | 2040/37962 [4:47:32<81:53:17,  8.21s/it]
  5%|▌         | 2041/37962 [4:47:41<82:29:00,  8.27s/it]
  5%|▌         | 2042/37962 [4:47:49<82:12:50,  8.24s/it]
  5%|▌         | 2043/37962 [4:47:57<82:23:07,  8.26s/it]
  5%|▌         | 2044/37962 [4:48:05<82:00:02,  8.22s/it]
  5%|▌         | 2045/37962 [4:48:13<82:09:32,  8.23s/it]
  5%|▌         | 2046/37962 [4:48:22<82:15:43,  8.25s/it]
  5%|▌         | 2047/37962 [4:48:30<82:08:52,  8.23s/it]
  5%|▌         | 2048/37962 [4:48:38<82:44:58,  8.29s/it]
  5%|▌         | 2049/37962 [4:48:47<82:52:13,  8.31s/it]
  5%|▌         | 2050/37962 [4:48:55<82:38:20,  8.28s/it]
                                                         
{'loss': 1.5584, 'grad_norm': 10.339805603027344, 'learning_rate': 1.9969810944134484e-05, 'epoch': 0.05}

  5%|▌         | 2050/37962 [4:48:55<82:38:20,  8.28s/it]
  5%|▌         | 2051/37962 [4:49:03<82:53:53,  8.31s/it]
  5%|▌         | 2052/37962 [4:49:11<82:16:09,  8.25s/it]
  5%|▌         | 2053/37962 [4:49:20<82:19:45,  8.25s/it]
  5%|▌         | 2054/37962 [4:49:28<83:01:09,  8.32s/it]
  5%|▌         | 2055/37962 [4:49:36<82:27:05,  8.27s/it]
  5%|▌         | 2056/37962 [4:49:44<81:40:21,  8.19s/it]
  5%|▌         | 2057/37962 [4:49:52<80:43:02,  8.09s/it]
  5%|▌         | 2058/37962 [4:50:00<81:19:09,  8.15s/it]
  5%|▌         | 2059/37962 [4:50:09<82:02:57,  8.23s/it]
  5%|▌         | 2060/37962 [4:50:17<82:15:53,  8.25s/it]
                                                         
{'loss': 1.5929, 'grad_norm': 36.826358795166016, 'learning_rate': 1.9969144882044606e-05, 'epoch': 0.05}

  5%|▌         | 2060/37962 [4:50:17<82:15:53,  8.25s/it]
  5%|▌         | 2061/37962 [4:50:25<81:40:37,  8.19s/it]
  5%|▌         | 2062/37962 [4:50:34<82:00:15,  8.22s/it]
  5%|▌         | 2063/37962 [4:50:42<82:47:25,  8.30s/it]
  5%|▌         | 2064/37962 [4:50:50<81:58:47,  8.22s/it]
  5%|▌         | 2065/37962 [4:50:58<82:34:41,  8.28s/it]
  5%|▌         | 2066/37962 [4:51:07<82:39:03,  8.29s/it]
  5%|▌         | 2067/37962 [4:51:15<82:27:22,  8.27s/it]
  5%|▌         | 2068/37962 [4:51:23<82:34:33,  8.28s/it]
  5%|▌         | 2069/37962 [4:51:32<82:47:23,  8.30s/it]
  5%|▌         | 2070/37962 [4:51:40<83:12:35,  8.35s/it]
                                                         
{'loss': 1.6307, 'grad_norm': 47.29088592529297, 'learning_rate': 1.9968471563586874e-05, 'epoch': 0.05}

  5%|▌         | 2070/37962 [4:51:40<83:12:35,  8.35s/it]
  5%|▌         | 2071/37962 [4:51:48<82:20:20,  8.26s/it]
  5%|▌         | 2072/37962 [4:51:56<82:06:36,  8.24s/it]
  5%|▌         | 2073/37962 [4:52:05<82:23:13,  8.26s/it]
  5%|▌         | 2074/37962 [4:52:13<82:51:25,  8.31s/it]
  5%|▌         | 2075/37962 [4:52:21<82:45:56,  8.30s/it]
  5%|▌         | 2076/37962 [4:52:30<83:16:58,  8.35s/it]
  5%|▌         | 2077/37962 [4:52:38<82:44:01,  8.30s/it]
  5%|▌         | 2078/37962 [4:52:46<82:24:30,  8.27s/it]
  5%|▌         | 2079/37962 [4:52:54<81:36:22,  8.19s/it]
  5%|▌         | 2080/37962 [4:53:03<82:00:04,  8.23s/it]
                                                         
{'loss': 1.5343, 'grad_norm': 17.101884841918945, 'learning_rate': 1.996779098925138e-05, 'epoch': 0.05}

  5%|▌         | 2080/37962 [4:53:03<82:00:04,  8.23s/it]
  5%|▌         | 2081/37962 [4:53:11<81:52:24,  8.21s/it]
  5%|▌         | 2082/37962 [4:53:19<81:41:53,  8.20s/it]
  5%|▌         | 2083/37962 [4:53:27<80:52:54,  8.12s/it]
  5%|▌         | 2084/37962 [4:53:35<81:25:08,  8.17s/it]
  5%|▌         | 2085/37962 [4:53:43<81:44:56,  8.20s/it]
  5%|▌         | 2086/37962 [4:53:52<81:46:58,  8.21s/it]
  5%|▌         | 2087/37962 [4:54:00<82:14:59,  8.25s/it]
  6%|▌         | 2088/37962 [4:54:08<81:58:00,  8.23s/it]
  6%|▌         | 2089/37962 [4:54:17<82:39:16,  8.29s/it]
  6%|▌         | 2090/37962 [4:54:25<82:00:59,  8.23s/it]
                                                         
{'loss': 1.4896, 'grad_norm': 18.82046127319336, 'learning_rate': 1.9967103159533508e-05, 'epoch': 0.06}

  6%|▌         | 2090/37962 [4:54:25<82:00:59,  8.23s/it]
  6%|▌         | 2091/37962 [4:54:33<82:51:16,  8.32s/it]
  6%|▌         | 2092/37962 [4:54:41<82:17:17,  8.26s/it]
  6%|▌         | 2093/37962 [4:54:50<82:30:49,  8.28s/it]
  6%|▌         | 2094/37962 [4:54:58<81:47:42,  8.21s/it]
  6%|▌         | 2095/37962 [4:55:06<81:53:01,  8.22s/it]
  6%|▌         | 2096/37962 [4:55:14<81:54:05,  8.22s/it]
  6%|▌         | 2097/37962 [4:55:23<82:42:51,  8.30s/it]
  6%|▌         | 2098/37962 [4:55:31<82:17:53,  8.26s/it]
  6%|▌         | 2099/37962 [4:55:39<82:26:47,  8.28s/it]
  6%|▌         | 2100/37962 [4:55:47<82:06:51,  8.24s/it]
                                                         
{'loss': 1.569, 'grad_norm': 6.147754192352295, 'learning_rate': 1.9966408074933913e-05, 'epoch': 0.06}

  6%|▌         | 2100/37962 [4:55:47<82:06:51,  8.24s/it]
  6%|▌         | 2101/37962 [4:55:56<82:17:09,  8.26s/it]
  6%|▌         | 2102/37962 [4:56:04<81:56:18,  8.23s/it]
  6%|▌         | 2103/37962 [4:56:12<82:44:03,  8.31s/it]
  6%|▌         | 2104/37962 [4:56:20<80:46:59,  8.11s/it]
  6%|▌         | 2105/37962 [4:56:28<80:27:12,  8.08s/it]
  6%|▌         | 2106/37962 [4:56:36<81:11:45,  8.15s/it]
  6%|▌         | 2107/37962 [4:56:45<81:47:01,  8.21s/it]
  6%|▌         | 2108/37962 [4:56:53<81:34:57,  8.19s/it]
  6%|▌         | 2109/37962 [4:57:01<81:54:58,  8.23s/it]
  6%|▌         | 2110/37962 [4:57:09<81:29:54,  8.18s/it]
                                                         
{'loss': 1.4459, 'grad_norm': 6.583403587341309, 'learning_rate': 1.9965705735958537e-05, 'epoch': 0.06}

  6%|▌         | 2110/37962 [4:57:09<81:29:54,  8.18s/it]
  6%|▌         | 2111/37962 [4:57:17<81:13:18,  8.16s/it]
  6%|▌         | 2112/37962 [4:57:25<81:26:53,  8.18s/it]
  6%|▌         | 2113/37962 [4:57:34<81:36:55,  8.20s/it]
  6%|▌         | 2114/37962 [4:57:42<82:04:45,  8.24s/it]
  6%|▌         | 2115/37962 [4:57:50<82:25:52,  8.28s/it]
  6%|▌         | 2116/37962 [4:57:59<82:04:35,  8.24s/it]
  6%|▌         | 2117/37962 [4:58:07<81:43:14,  8.21s/it]
  6%|▌         | 2118/37962 [4:58:14<80:10:06,  8.05s/it]
  6%|▌         | 2119/37962 [4:58:23<80:37:01,  8.10s/it]
  6%|▌         | 2120/37962 [4:58:31<81:12:29,  8.16s/it]
                                                         
{'loss': 1.5438, 'grad_norm': 15.984086990356445, 'learning_rate': 1.99649961431186e-05, 'epoch': 0.06}

  6%|▌         | 2120/37962 [4:58:31<81:12:29,  8.16s/it]
  6%|▌         | 2121/37962 [4:58:39<81:44:16,  8.21s/it]
  6%|▌         | 2122/37962 [4:58:47<81:48:15,  8.22s/it]
  6%|▌         | 2123/37962 [4:58:56<82:06:40,  8.25s/it]
  6%|▌         | 2124/37962 [4:59:04<83:26:17,  8.38s/it]
  6%|▌         | 2125/37962 [4:59:12<81:50:25,  8.22s/it]
  6%|▌         | 2126/37962 [4:59:20<81:49:14,  8.22s/it]
  6%|▌         | 2127/37962 [4:59:29<81:20:36,  8.17s/it]
  6%|▌         | 2128/37962 [4:59:37<81:51:17,  8.22s/it]
  6%|▌         | 2129/37962 [4:59:45<81:47:20,  8.22s/it]
  6%|▌         | 2130/37962 [4:59:53<82:02:20,  8.24s/it]
                                                         
{'loss': 1.5472, 'grad_norm': 33.396324157714844, 'learning_rate': 1.9964279296930602e-05, 'epoch': 0.06}

  6%|▌         | 2130/37962 [4:59:53<82:02:20,  8.24s/it]
  6%|▌         | 2131/37962 [5:00:02<82:28:42,  8.29s/it]
  6%|▌         | 2132/37962 [5:00:10<82:30:45,  8.29s/it]
  6%|▌         | 2133/37962 [5:00:18<82:34:33,  8.30s/it]
  6%|▌         | 2134/37962 [5:00:27<82:37:41,  8.30s/it]
  6%|▌         | 2135/37962 [5:00:35<83:09:16,  8.36s/it]
  6%|▌         | 2136/37962 [5:00:43<82:38:47,  8.30s/it]
  6%|▌         | 2137/37962 [5:00:52<83:18:17,  8.37s/it]
  6%|▌         | 2138/37962 [5:01:00<83:43:27,  8.41s/it]
  6%|▌         | 2139/37962 [5:01:09<83:34:07,  8.40s/it]
  6%|▌         | 2140/37962 [5:01:17<83:08:45,  8.36s/it]
                                                         
{'loss': 1.5113, 'grad_norm': 28.127849578857422, 'learning_rate': 1.9963555197916326e-05, 'epoch': 0.06}

  6%|▌         | 2140/37962 [5:01:17<83:08:45,  8.36s/it]
  6%|▌         | 2141/37962 [5:01:25<82:54:47,  8.33s/it]
  6%|▌         | 2142/37962 [5:01:34<83:05:51,  8.35s/it]
  6%|▌         | 2143/37962 [5:01:42<83:05:08,  8.35s/it]
  6%|▌         | 2144/37962 [5:01:50<82:54:36,  8.33s/it]
  6%|▌         | 2145/37962 [5:01:59<83:28:41,  8.39s/it]
  6%|▌         | 2146/37962 [5:02:07<82:55:52,  8.34s/it]
  6%|▌         | 2147/37962 [5:02:15<82:27:13,  8.29s/it]
  6%|▌         | 2148/37962 [5:02:24<82:33:20,  8.30s/it]
  6%|▌         | 2149/37962 [5:02:31<81:23:09,  8.18s/it]
  6%|▌         | 2150/37962 [5:02:40<81:14:55,  8.17s/it]
                                                         
{'loss': 1.45, 'grad_norm': 26.367124557495117, 'learning_rate': 1.9962823846602825e-05, 'epoch': 0.06}

  6%|▌         | 2150/37962 [5:02:40<81:14:55,  8.17s/it]
  6%|▌         | 2151/37962 [5:02:48<81:06:00,  8.15s/it]
  6%|▌         | 2152/37962 [5:02:56<81:33:32,  8.20s/it]
  6%|▌         | 2153/37962 [5:03:04<81:40:39,  8.21s/it]
  6%|▌         | 2154/37962 [5:03:12<81:02:18,  8.15s/it]
  6%|▌         | 2155/37962 [5:03:21<81:46:59,  8.22s/it]
  6%|▌         | 2156/37962 [5:03:29<81:58:25,  8.24s/it]
  6%|▌         | 2157/37962 [5:03:37<82:05:03,  8.25s/it]
  6%|▌         | 2158/37962 [5:03:45<81:19:38,  8.18s/it]
  6%|▌         | 2159/37962 [5:03:54<81:32:34,  8.20s/it]
  6%|▌         | 2160/37962 [5:04:02<81:41:02,  8.21s/it]
                                                         
{'loss': 1.4468, 'grad_norm': 15.139095306396484, 'learning_rate': 1.996208524352244e-05, 'epoch': 0.06}

  6%|▌         | 2160/37962 [5:04:02<81:41:02,  8.21s/it]
  6%|▌         | 2161/37962 [5:04:10<81:46:32,  8.22s/it]
  6%|▌         | 2162/37962 [5:04:18<81:46:50,  8.22s/it]
  6%|▌         | 2163/37962 [5:04:26<81:22:05,  8.18s/it]
  6%|▌         | 2164/37962 [5:04:35<81:28:41,  8.19s/it]
  6%|▌         | 2165/37962 [5:04:43<82:04:20,  8.25s/it]
  6%|▌         | 2166/37962 [5:04:51<82:00:24,  8.25s/it]
  6%|▌         | 2167/37962 [5:04:59<81:53:03,  8.24s/it]
  6%|▌         | 2168/37962 [5:05:08<81:56:39,  8.24s/it]
  6%|▌         | 2169/37962 [5:05:16<81:26:41,  8.19s/it]
  6%|▌         | 2170/37962 [5:05:24<80:39:50,  8.11s/it]
                                                         
{'loss': 1.411, 'grad_norm': 16.23615837097168, 'learning_rate': 1.9961339389212786e-05, 'epoch': 0.06}

  6%|▌         | 2170/37962 [5:05:24<80:39:50,  8.11s/it]
  6%|▌         | 2171/37962 [5:05:32<81:59:10,  8.25s/it]
  6%|▌         | 2172/37962 [5:05:40<81:47:04,  8.23s/it]
  6%|▌         | 2173/37962 [5:05:49<82:09:16,  8.26s/it]
  6%|▌         | 2174/37962 [5:05:57<81:45:17,  8.22s/it]
  6%|▌         | 2175/37962 [5:06:05<82:18:04,  8.28s/it]
  6%|▌         | 2176/37962 [5:06:14<82:42:17,  8.32s/it]
  6%|▌         | 2177/37962 [5:06:22<82:37:27,  8.31s/it]
  6%|▌         | 2178/37962 [5:06:30<82:17:12,  8.28s/it]
  6%|▌         | 2179/37962 [5:06:39<82:36:28,  8.31s/it]
  6%|▌         | 2180/37962 [5:06:47<82:58:44,  8.35s/it]
                                                         
{'loss': 1.6963, 'grad_norm': 7.391385078430176, 'learning_rate': 1.996058628421676e-05, 'epoch': 0.06}

  6%|▌         | 2180/37962 [5:06:47<82:58:44,  8.35s/it]
  6%|▌         | 2181/37962 [5:06:55<82:40:41,  8.32s/it]
  6%|▌         | 2182/37962 [5:07:04<82:36:43,  8.31s/it]
  6%|▌         | 2183/37962 [5:07:12<82:46:13,  8.33s/it]
  6%|▌         | 2184/37962 [5:07:20<82:51:24,  8.34s/it]
  6%|▌         | 2185/37962 [5:07:29<83:02:54,  8.36s/it]
  6%|▌         | 2186/37962 [5:07:37<82:48:44,  8.33s/it]
  6%|▌         | 2187/37962 [5:07:45<81:11:34,  8.17s/it]
  6%|▌         | 2188/37962 [5:07:53<81:27:35,  8.20s/it]
  6%|▌         | 2189/37962 [5:08:01<82:05:46,  8.26s/it]
  6%|▌         | 2190/37962 [5:08:10<82:21:01,  8.29s/it]
                                                         
{'loss': 1.6069, 'grad_norm': 29.048534393310547, 'learning_rate': 1.9959825929082535e-05, 'epoch': 0.06}

  6%|▌         | 2190/37962 [5:08:10<82:21:01,  8.29s/it]
  6%|▌         | 2191/37962 [5:08:18<82:37:38,  8.32s/it]
  6%|▌         | 2192/37962 [5:08:26<82:29:27,  8.30s/it]
  6%|▌         | 2193/37962 [5:08:34<81:37:53,  8.22s/it]
  6%|▌         | 2194/37962 [5:08:43<81:37:04,  8.21s/it]
  6%|▌         | 2195/37962 [5:08:51<81:27:52,  8.20s/it]
  6%|▌         | 2196/37962 [5:08:59<82:29:23,  8.30s/it]
  6%|▌         | 2197/37962 [5:09:08<82:14:17,  8.28s/it]
  6%|▌         | 2198/37962 [5:09:15<80:49:42,  8.14s/it]
  6%|▌         | 2199/37962 [5:09:24<81:14:02,  8.18s/it]
  6%|▌         | 2200/37962 [5:09:32<80:42:25,  8.12s/it]
                                                         
{'loss': 1.4333, 'grad_norm': 8.025056838989258, 'learning_rate': 1.9959058324363558e-05, 'epoch': 0.06}

  6%|▌         | 2200/37962 [5:09:32<80:42:25,  8.12s/it]
  6%|▌         | 2201/37962 [5:09:40<80:31:49,  8.11s/it]
  6%|▌         | 2202/37962 [5:09:48<81:17:21,  8.18s/it]
  6%|▌         | 2203/37962 [5:09:56<81:56:01,  8.25s/it]
  6%|▌         | 2204/37962 [5:10:05<82:21:46,  8.29s/it]
  6%|▌         | 2205/37962 [5:10:13<82:25:40,  8.30s/it]
  6%|▌         | 2206/37962 [5:10:21<82:09:38,  8.27s/it]
  6%|▌         | 2207/37962 [5:10:30<82:14:01,  8.28s/it]
  6%|▌         | 2208/37962 [5:10:38<82:42:49,  8.33s/it]
  6%|▌         | 2209/37962 [5:10:46<82:45:02,  8.33s/it]
  6%|▌         | 2210/37962 [5:10:55<82:38:34,  8.32s/it]
                                                         
{'loss': 1.5731, 'grad_norm': 10.638434410095215, 'learning_rate': 1.9958283470618555e-05, 'epoch': 0.06}

  6%|▌         | 2210/37962 [5:10:55<82:38:34,  8.32s/it]
  6%|▌         | 2211/37962 [5:11:03<82:21:59,  8.29s/it]
  6%|▌         | 2212/37962 [5:11:11<82:12:16,  8.28s/it]
  6%|▌         | 2213/37962 [5:11:19<81:49:37,  8.24s/it]
  6%|▌         | 2214/37962 [5:11:28<81:59:13,  8.26s/it]
  6%|▌         | 2215/37962 [5:11:36<82:10:24,  8.28s/it]
  6%|▌         | 2216/37962 [5:11:44<81:20:50,  8.19s/it]
  6%|▌         | 2217/37962 [5:11:52<81:09:25,  8.17s/it]
  6%|▌         | 2218/37962 [5:12:00<81:21:24,  8.19s/it]
  6%|▌         | 2219/37962 [5:12:09<81:32:53,  8.21s/it]
  6%|▌         | 2220/37962 [5:12:17<81:10:06,  8.18s/it]
                                                         
{'loss': 1.5306, 'grad_norm': 10.70621395111084, 'learning_rate': 1.9957501368411528e-05, 'epoch': 0.06}

  6%|▌         | 2220/37962 [5:12:17<81:10:06,  8.18s/it]
  6%|▌         | 2221/37962 [5:12:25<81:43:11,  8.23s/it]
  6%|▌         | 2222/37962 [5:12:33<81:53:00,  8.25s/it]
  6%|▌         | 2223/37962 [5:12:42<82:19:24,  8.29s/it]
  6%|▌         | 2224/37962 [5:12:50<82:35:36,  8.32s/it]
  6%|▌         | 2225/37962 [5:12:58<81:48:28,  8.24s/it]
  6%|▌         | 2226/37962 [5:13:06<80:44:32,  8.13s/it]
  6%|▌         | 2227/37962 [5:13:14<80:57:21,  8.16s/it]
  6%|▌         | 2228/37962 [5:13:23<81:39:37,  8.23s/it]
  6%|▌         | 2229/37962 [5:13:31<81:34:01,  8.22s/it]
  6%|▌         | 2230/37962 [5:13:39<81:40:12,  8.23s/it]
                                                         
{'loss': 1.5552, 'grad_norm': 22.240854263305664, 'learning_rate': 1.9956712018311756e-05, 'epoch': 0.06}

  6%|▌         | 2230/37962 [5:13:39<81:40:12,  8.23s/it]
  6%|▌         | 2231/37962 [5:13:48<82:22:13,  8.30s/it]
  6%|▌         | 2232/37962 [5:13:56<82:15:01,  8.29s/it]
  6%|▌         | 2233/37962 [5:14:04<82:01:13,  8.26s/it]
  6%|▌         | 2234/37962 [5:14:12<81:17:22,  8.19s/it]
  6%|▌         | 2235/37962 [5:14:20<81:17:50,  8.19s/it]
  6%|▌         | 2236/37962 [5:14:29<81:45:32,  8.24s/it]
  6%|▌         | 2237/37962 [5:14:37<82:15:41,  8.29s/it]
  6%|▌         | 2238/37962 [5:14:45<81:01:11,  8.16s/it]
  6%|▌         | 2239/37962 [5:14:53<81:22:45,  8.20s/it]
  6%|▌         | 2240/37962 [5:15:01<81:25:14,  8.21s/it]
                                                         
{'loss': 1.5237, 'grad_norm': 31.46197509765625, 'learning_rate': 1.99559154208938e-05, 'epoch': 0.06}

  6%|▌         | 2240/37962 [5:15:01<81:25:14,  8.21s/it]
  6%|▌         | 2241/37962 [5:15:09<80:48:32,  8.14s/it]
  6%|▌         | 2242/37962 [5:15:18<81:07:11,  8.18s/it]
  6%|▌         | 2243/37962 [5:15:26<81:25:51,  8.21s/it]
  6%|▌         | 2244/37962 [5:15:34<82:23:13,  8.30s/it]
  6%|▌         | 2245/37962 [5:15:43<81:53:45,  8.25s/it]
  6%|▌         | 2246/37962 [5:15:51<82:27:00,  8.31s/it]
  6%|▌         | 2247/37962 [5:15:59<82:48:58,  8.35s/it]
  6%|▌         | 2248/37962 [5:16:08<82:23:11,  8.30s/it]
  6%|▌         | 2249/37962 [5:16:16<82:37:41,  8.33s/it]
  6%|▌         | 2250/37962 [5:16:24<81:48:16,  8.25s/it]
                                                         
{'loss': 1.5934, 'grad_norm': 25.140926361083984, 'learning_rate': 1.9955111576737475e-05, 'epoch': 0.06}

  6%|▌         | 2250/37962 [5:16:24<81:48:16,  8.25s/it]
  6%|▌         | 2251/37962 [5:16:32<82:10:18,  8.28s/it]
  6%|▌         | 2252/37962 [5:16:40<81:06:12,  8.18s/it]
  6%|▌         | 2253/37962 [5:16:49<81:05:06,  8.17s/it]
  6%|▌         | 2254/37962 [5:16:57<81:54:46,  8.26s/it]
  6%|▌         | 2255/37962 [5:17:05<81:42:16,  8.24s/it]
  6%|▌         | 2256/37962 [5:17:14<82:11:32,  8.29s/it]
  6%|▌         | 2257/37962 [5:17:22<81:56:05,  8.26s/it]
  6%|▌         | 2258/37962 [5:17:30<81:18:26,  8.20s/it]
  6%|▌         | 2259/37962 [5:17:38<81:04:22,  8.17s/it]
  6%|▌         | 2260/37962 [5:17:46<81:01:59,  8.17s/it]
                                                         
{'loss': 1.4935, 'grad_norm': 25.01129913330078, 'learning_rate': 1.99543004864279e-05, 'epoch': 0.06}

  6%|▌         | 2260/37962 [5:17:46<81:01:59,  8.17s/it]
  6%|▌         | 2261/37962 [5:17:54<81:25:53,  8.21s/it]
  6%|▌         | 2262/37962 [5:18:03<81:30:52,  8.22s/it]
  6%|▌         | 2263/37962 [5:18:11<81:05:31,  8.18s/it]
  6%|▌         | 2264/37962 [5:18:19<81:38:38,  8.23s/it]
  6%|▌         | 2265/37962 [5:18:27<81:39:32,  8.24s/it]
  6%|▌         | 2266/37962 [5:18:36<81:51:59,  8.26s/it]
  6%|▌         | 2267/37962 [5:18:44<81:50:26,  8.25s/it]
  6%|▌         | 2268/37962 [5:18:52<82:27:54,  8.32s/it]
  6%|▌         | 2269/37962 [5:19:01<83:11:05,  8.39s/it]
  6%|▌         | 2270/37962 [5:19:09<82:44:04,  8.34s/it]
                                                         
{'loss': 1.4984, 'grad_norm': 31.89696502685547, 'learning_rate': 1.9953482150555446e-05, 'epoch': 0.06}

  6%|▌         | 2270/37962 [5:19:09<82:44:04,  8.34s/it]
  6%|▌         | 2271/37962 [5:19:18<82:42:11,  8.34s/it]
  6%|▌         | 2272/37962 [5:19:26<82:35:59,  8.33s/it]
  6%|▌         | 2273/37962 [5:19:34<82:15:58,  8.30s/it]
  6%|▌         | 2274/37962 [5:19:42<81:52:45,  8.26s/it]
  6%|▌         | 2275/37962 [5:19:51<82:09:04,  8.29s/it]
  6%|▌         | 2276/37962 [5:19:59<81:39:33,  8.24s/it]
  6%|▌         | 2277/37962 [5:20:07<81:58:46,  8.27s/it]
  6%|▌         | 2278/37962 [5:20:15<81:48:34,  8.25s/it]
  6%|▌         | 2279/37962 [5:20:23<81:08:58,  8.19s/it]
  6%|▌         | 2280/37962 [5:20:32<81:18:14,  8.20s/it]
                                                         
{'loss': 1.5848, 'grad_norm': 11.756664276123047, 'learning_rate': 1.9952656569715766e-05, 'epoch': 0.06}

  6%|▌         | 2280/37962 [5:20:32<81:18:14,  8.20s/it]
  6%|▌         | 2281/37962 [5:20:40<82:03:07,  8.28s/it]
  6%|▌         | 2282/37962 [5:20:48<81:18:26,  8.20s/it]
  6%|▌         | 2283/37962 [5:20:56<81:06:00,  8.18s/it]
  6%|▌         | 2284/37962 [5:21:05<81:31:47,  8.23s/it]
  6%|▌         | 2285/37962 [5:21:12<80:11:04,  8.09s/it]
  6%|▌         | 2286/37962 [5:21:20<80:26:21,  8.12s/it]
  6%|▌         | 2287/37962 [5:21:28<79:55:15,  8.06s/it]
  6%|▌         | 2288/37962 [5:21:37<80:45:29,  8.15s/it]
  6%|▌         | 2289/37962 [5:21:45<81:13:01,  8.20s/it]
  6%|▌         | 2290/37962 [5:21:53<81:15:12,  8.20s/it]
                                                         
{'loss': 1.4692, 'grad_norm': 25.914997100830078, 'learning_rate': 1.9951823744509787e-05, 'epoch': 0.06}

  6%|▌         | 2290/37962 [5:21:53<81:15:12,  8.20s/it]
  6%|▌         | 2291/37962 [5:22:02<81:22:46,  8.21s/it]
  6%|▌         | 2292/37962 [5:22:10<81:54:11,  8.27s/it]
  6%|▌         | 2293/37962 [5:22:18<81:53:47,  8.27s/it]
  6%|▌         | 2294/37962 [5:22:26<80:43:57,  8.15s/it]
  6%|▌         | 2295/37962 [5:22:34<81:40:40,  8.24s/it]
  6%|▌         | 2296/37962 [5:22:43<81:50:59,  8.26s/it]
  6%|▌         | 2297/37962 [5:22:51<81:44:09,  8.25s/it]
  6%|▌         | 2298/37962 [5:22:59<82:10:11,  8.29s/it]
  6%|▌         | 2299/37962 [5:23:08<82:11:59,  8.30s/it]
  6%|▌         | 2300/37962 [5:23:16<81:58:38,  8.28s/it]
                                                         
{'loss': 1.4829, 'grad_norm': 28.28131866455078, 'learning_rate': 1.995098367554371e-05, 'epoch': 0.06}

  6%|▌         | 2300/37962 [5:23:16<81:58:38,  8.28s/it]
  6%|▌         | 2301/37962 [5:23:24<80:59:44,  8.18s/it]
  6%|▌         | 2302/37962 [5:23:32<81:39:49,  8.24s/it]
  6%|▌         | 2303/37962 [5:23:41<82:16:46,  8.31s/it]
  6%|▌         | 2304/37962 [5:23:49<82:03:11,  8.28s/it]
  6%|▌         | 2305/37962 [5:23:57<81:41:57,  8.25s/it]
  6%|▌         | 2306/37962 [5:24:05<81:45:49,  8.26s/it]
  6%|▌         | 2307/37962 [5:24:14<81:24:03,  8.22s/it]
  6%|▌         | 2308/37962 [5:24:22<82:00:04,  8.28s/it]
  6%|▌         | 2309/37962 [5:24:30<82:25:12,  8.32s/it]
  6%|▌         | 2310/37962 [5:24:39<82:54:43,  8.37s/it]
                                                         
{'loss': 1.4676, 'grad_norm': 10.771398544311523, 'learning_rate': 1.9950136363429002e-05, 'epoch': 0.06}

  6%|▌         | 2310/37962 [5:24:39<82:54:43,  8.37s/it]
  6%|▌         | 2311/37962 [5:24:47<82:44:51,  8.36s/it]
  6%|▌         | 2312/37962 [5:24:56<82:40:57,  8.35s/it]
  6%|▌         | 2313/37962 [5:25:04<82:10:27,  8.30s/it]
  6%|▌         | 2314/37962 [5:25:12<82:01:21,  8.28s/it]
  6%|▌         | 2315/37962 [5:25:20<82:07:06,  8.29s/it]
  6%|▌         | 2316/37962 [5:25:28<81:51:34,  8.27s/it]
  6%|▌         | 2317/37962 [5:25:36<80:38:32,  8.14s/it]
  6%|▌         | 2318/37962 [5:25:45<80:50:58,  8.17s/it]
  6%|▌         | 2319/37962 [5:25:53<81:00:47,  8.18s/it]
  6%|▌         | 2320/37962 [5:26:01<81:08:50,  8.20s/it]
                                                         
{'loss': 1.629, 'grad_norm': 7.392683029174805, 'learning_rate': 1.994928180878241e-05, 'epoch': 0.06}

  6%|▌         | 2320/37962 [5:26:01<81:08:50,  8.20s/it]
  6%|▌         | 2321/37962 [5:26:09<81:50:08,  8.27s/it]
  6%|▌         | 2322/37962 [5:26:18<81:19:28,  8.21s/it]
  6%|▌         | 2323/37962 [5:26:26<81:49:57,  8.27s/it]
  6%|▌         | 2324/37962 [5:26:34<81:57:25,  8.28s/it]
  6%|▌         | 2325/37962 [5:26:42<81:51:42,  8.27s/it]
  6%|▌         | 2326/37962 [5:26:51<81:44:20,  8.26s/it]
  6%|▌         | 2327/37962 [5:26:59<82:08:37,  8.30s/it]
  6%|▌         | 2328/37962 [5:27:07<82:14:51,  8.31s/it]
  6%|▌         | 2329/37962 [5:27:15<81:18:14,  8.21s/it]
  6%|▌         | 2330/37962 [5:27:24<81:25:57,  8.23s/it]
                                                         
{'loss': 1.4655, 'grad_norm': 15.329215049743652, 'learning_rate': 1.994842001222595e-05, 'epoch': 0.06}

  6%|▌         | 2330/37962 [5:27:24<81:25:57,  8.23s/it]
  6%|▌         | 2331/37962 [5:27:32<80:39:18,  8.15s/it]
  6%|▌         | 2332/37962 [5:27:40<81:03:54,  8.19s/it]
  6%|▌         | 2333/37962 [5:27:48<81:09:02,  8.20s/it]
  6%|▌         | 2334/37962 [5:27:56<81:18:17,  8.22s/it]
  6%|▌         | 2335/37962 [5:28:05<81:26:28,  8.23s/it]
  6%|▌         | 2336/37962 [5:28:13<81:30:10,  8.24s/it]
  6%|▌         | 2337/37962 [5:28:21<81:46:10,  8.26s/it]
  6%|▌         | 2338/37962 [5:28:29<81:35:01,  8.24s/it]
  6%|▌         | 2339/37962 [5:28:38<81:51:20,  8.27s/it]
  6%|▌         | 2340/37962 [5:28:46<81:09:43,  8.20s/it]
                                                         
{'loss': 1.5131, 'grad_norm': 28.639211654663086, 'learning_rate': 1.9947550974386907e-05, 'epoch': 0.06}

  6%|▌         | 2340/37962 [5:28:46<81:09:43,  8.20s/it]
  6%|▌         | 2341/37962 [5:28:54<81:52:54,  8.28s/it]
  6%|▌         | 2342/37962 [5:29:02<81:42:13,  8.26s/it]
  6%|▌         | 2343/37962 [5:29:11<81:26:51,  8.23s/it]
  6%|▌         | 2344/37962 [5:29:19<81:27:58,  8.23s/it]
  6%|▌         | 2345/37962 [5:29:27<81:21:59,  8.22s/it]
  6%|▌         | 2346/37962 [5:29:35<81:06:58,  8.20s/it]
  6%|▌         | 2347/37962 [5:29:43<80:03:26,  8.09s/it]
  6%|▌         | 2348/37962 [5:29:51<80:13:51,  8.11s/it]
  6%|▌         | 2349/37962 [5:30:00<81:16:40,  8.22s/it]
  6%|▌         | 2350/37962 [5:30:08<81:55:51,  8.28s/it]
                                                         
{'loss': 1.5858, 'grad_norm': 46.474666595458984, 'learning_rate': 1.994667469589784e-05, 'epoch': 0.06}

  6%|▌         | 2350/37962 [5:30:08<81:55:51,  8.28s/it]
  6%|▌         | 2351/37962 [5:30:16<81:55:43,  8.28s/it]
  6%|▌         | 2352/37962 [5:30:25<81:46:15,  8.27s/it]
  6%|▌         | 2353/37962 [5:30:33<80:51:15,  8.17s/it]
  6%|▌         | 2354/37962 [5:30:41<80:26:13,  8.13s/it]
  6%|▌         | 2355/37962 [5:30:49<80:56:22,  8.18s/it]
  6%|▌         | 2356/37962 [5:30:57<81:15:48,  8.22s/it]
  6%|▌         | 2357/37962 [5:31:06<82:06:35,  8.30s/it]
  6%|▌         | 2358/37962 [5:31:14<81:47:16,  8.27s/it]
  6%|▌         | 2359/37962 [5:31:22<81:05:03,  8.20s/it]
  6%|▌         | 2360/37962 [5:31:30<81:21:06,  8.23s/it]
                                                         
{'loss': 1.5628, 'grad_norm': 13.97075366973877, 'learning_rate': 1.9945791177396573e-05, 'epoch': 0.06}

  6%|▌         | 2360/37962 [5:31:30<81:21:06,  8.23s/it]
  6%|▌         | 2361/37962 [5:31:39<82:11:00,  8.31s/it]
  6%|▌         | 2362/37962 [5:31:47<82:04:58,  8.30s/it]
  6%|▌         | 2363/37962 [5:31:55<82:16:46,  8.32s/it]
  6%|▌         | 2364/37962 [5:32:04<82:10:51,  8.31s/it]
  6%|▌         | 2365/37962 [5:32:12<81:09:15,  8.21s/it]
  6%|▌         | 2366/37962 [5:32:20<81:30:15,  8.24s/it]
  6%|▌         | 2367/37962 [5:32:28<81:23:43,  8.23s/it]
  6%|▌         | 2368/37962 [5:32:36<81:24:51,  8.23s/it]
  6%|▌         | 2369/37962 [5:32:45<81:05:29,  8.20s/it]
  6%|▌         | 2370/37962 [5:32:53<80:39:32,  8.16s/it]
                                                         
{'loss': 1.5245, 'grad_norm': 46.04969787597656, 'learning_rate': 1.994490041952621e-05, 'epoch': 0.06}

  6%|▌         | 2370/37962 [5:32:53<80:39:32,  8.16s/it]
  6%|▌         | 2371/37962 [5:33:01<81:29:32,  8.24s/it]
  6%|▌         | 2372/37962 [5:33:09<81:21:33,  8.23s/it]
  6%|▋         | 2373/37962 [5:33:18<81:26:06,  8.24s/it]
  6%|▋         | 2374/37962 [5:33:26<81:21:37,  8.23s/it]
  6%|▋         | 2375/37962 [5:33:34<81:34:14,  8.25s/it]
  6%|▋         | 2376/37962 [5:33:42<81:17:37,  8.22s/it]
  6%|▋         | 2377/37962 [5:33:50<81:30:49,  8.25s/it]
  6%|▋         | 2378/37962 [5:33:59<81:33:04,  8.25s/it]
  6%|▋         | 2379/37962 [5:34:07<82:13:53,  8.32s/it]
  6%|▋         | 2380/37962 [5:34:15<81:55:14,  8.29s/it]
                                                         
{'loss': 1.4347, 'grad_norm': 17.371849060058594, 'learning_rate': 1.9944002422935114e-05, 'epoch': 0.06}

  6%|▋         | 2380/37962 [5:34:15<81:55:14,  8.29s/it]
  6%|▋         | 2381/37962 [5:34:23<80:42:53,  8.17s/it]
  6%|▋         | 2382/37962 [5:34:32<81:13:43,  8.22s/it]
  6%|▋         | 2383/37962 [5:34:40<80:48:09,  8.18s/it]
  6%|▋         | 2384/37962 [5:34:48<80:05:55,  8.10s/it]
  6%|▋         | 2385/37962 [5:34:56<80:24:26,  8.14s/it]
  6%|▋         | 2386/37962 [5:35:04<80:33:15,  8.15s/it]
  6%|▋         | 2387/37962 [5:35:12<80:28:35,  8.14s/it]
  6%|▋         | 2388/37962 [5:35:20<80:16:11,  8.12s/it]
  6%|▋         | 2389/37962 [5:35:29<80:43:15,  8.17s/it]
  6%|▋         | 2390/37962 [5:35:37<81:21:46,  8.23s/it]
                                                         
{'loss': 1.5758, 'grad_norm': 30.52956199645996, 'learning_rate': 1.9943097188276926e-05, 'epoch': 0.06}

  6%|▋         | 2390/37962 [5:35:37<81:21:46,  8.23s/it]
  6%|▋         | 2391/37962 [5:35:45<81:44:23,  8.27s/it]
  6%|▋         | 2392/37962 [5:35:54<82:04:55,  8.31s/it]
  6%|▋         | 2393/37962 [5:36:02<81:46:24,  8.28s/it]
  6%|▋         | 2394/37962 [5:36:10<81:54:05,  8.29s/it]
  6%|▋         | 2395/37962 [5:36:18<81:50:01,  8.28s/it]
  6%|▋         | 2396/37962 [5:36:27<82:06:39,  8.31s/it]
  6%|▋         | 2397/37962 [5:36:35<82:32:35,  8.36s/it]
  6%|▋         | 2398/37962 [5:36:44<82:03:05,  8.31s/it]
  6%|▋         | 2399/37962 [5:36:52<82:00:22,  8.30s/it]
  6%|▋         | 2400/37962 [5:37:00<82:40:58,  8.37s/it]
                                                         
{'loss': 1.4984, 'grad_norm': 23.86384391784668, 'learning_rate': 1.994218471621054e-05, 'epoch': 0.06}

  6%|▋         | 2400/37962 [5:37:00<82:40:58,  8.37s/it]
  6%|▋         | 2401/37962 [5:37:09<82:15:06,  8.33s/it]
  6%|▋         | 2402/37962 [5:37:17<82:02:44,  8.31s/it]
  6%|▋         | 2403/37962 [5:37:25<82:37:28,  8.36s/it]
  6%|▋         | 2404/37962 [5:37:34<82:42:17,  8.37s/it]
  6%|▋         | 2405/37962 [5:37:42<82:11:38,  8.32s/it]
  6%|▋         | 2406/37962 [5:37:50<81:21:05,  8.24s/it]
  6%|▋         | 2407/37962 [5:37:58<81:27:40,  8.25s/it]
  6%|▋         | 2408/37962 [5:38:07<82:09:32,  8.32s/it]
  6%|▋         | 2409/37962 [5:38:15<81:34:17,  8.26s/it]
  6%|▋         | 2410/37962 [5:38:23<81:14:57,  8.23s/it]
                                                         
{'loss': 1.5404, 'grad_norm': 14.62930965423584, 'learning_rate': 1.994126500740014e-05, 'epoch': 0.06}

  6%|▋         | 2410/37962 [5:38:23<81:14:57,  8.23s/it]
  6%|▋         | 2411/37962 [5:38:31<81:16:31,  8.23s/it]
  6%|▋         | 2412/37962 [5:38:39<80:57:27,  8.20s/it]
  6%|▋         | 2413/37962 [5:38:48<81:31:11,  8.26s/it]
  6%|▋         | 2414/37962 [5:38:56<81:18:47,  8.23s/it]
  6%|▋         | 2415/37962 [5:39:04<81:33:55,  8.26s/it]
  6%|▋         | 2416/37962 [5:39:13<81:43:40,  8.28s/it]
  6%|▋         | 2417/37962 [5:39:20<80:34:12,  8.16s/it]
  6%|▋         | 2418/37962 [5:39:29<80:42:52,  8.18s/it]
  6%|▋         | 2419/37962 [5:39:37<81:21:36,  8.24s/it]
  6%|▋         | 2420/37962 [5:39:45<81:24:57,  8.25s/it]
                                                         
{'loss': 1.5485, 'grad_norm': 15.240072250366211, 'learning_rate': 1.9940338062515162e-05, 'epoch': 0.06}

  6%|▋         | 2420/37962 [5:39:45<81:24:57,  8.25s/it]
  6%|▋         | 2421/37962 [5:39:53<80:52:14,  8.19s/it]
  6%|▋         | 2422/37962 [5:40:02<81:21:41,  8.24s/it]
  6%|▋         | 2423/37962 [5:40:10<80:19:12,  8.14s/it]
  6%|▋         | 2424/37962 [5:40:18<80:27:33,  8.15s/it]
  6%|▋         | 2425/37962 [5:40:26<80:35:14,  8.16s/it]
  6%|▋         | 2426/37962 [5:40:35<82:01:19,  8.31s/it]
  6%|▋         | 2427/37962 [5:40:43<81:48:41,  8.29s/it]
  6%|▋         | 2428/37962 [5:40:51<81:27:52,  8.25s/it]
  6%|▋         | 2429/37962 [5:40:59<81:47:14,  8.29s/it]
  6%|▋         | 2430/37962 [5:41:07<80:40:50,  8.17s/it]
                                                         
{'loss': 1.4778, 'grad_norm': 13.681964874267578, 'learning_rate': 1.9939403882230313e-05, 'epoch': 0.06}

  6%|▋         | 2430/37962 [5:41:07<80:40:50,  8.17s/it]
  6%|▋         | 2431/37962 [5:41:16<81:08:54,  8.22s/it]
  6%|▋         | 2432/37962 [5:41:24<81:35:51,  8.27s/it]
  6%|▋         | 2433/37962 [5:41:32<81:45:30,  8.28s/it]
  6%|▋         | 2434/37962 [5:41:41<81:47:33,  8.29s/it]
  6%|▋         | 2435/37962 [5:41:49<80:55:43,  8.20s/it]
  6%|▋         | 2436/37962 [5:41:57<80:52:36,  8.20s/it]
  6%|▋         | 2437/37962 [5:42:05<81:08:15,  8.22s/it]
  6%|▋         | 2438/37962 [5:42:13<81:19:53,  8.24s/it]
  6%|▋         | 2439/37962 [5:42:22<81:52:30,  8.30s/it]
  6%|▋         | 2440/37962 [5:42:30<81:40:30,  8.28s/it]
                                                         
{'loss': 1.5996, 'grad_norm': 20.497394561767578, 'learning_rate': 1.9938462467225564e-05, 'epoch': 0.06}

  6%|▋         | 2440/37962 [5:42:30<81:40:30,  8.28s/it]
  6%|▋         | 2441/37962 [5:42:38<81:59:24,  8.31s/it]
  6%|▋         | 2442/37962 [5:42:47<81:21:04,  8.25s/it]
  6%|▋         | 2443/37962 [5:42:55<81:26:13,  8.25s/it]
  6%|▋         | 2444/37962 [5:43:03<81:47:48,  8.29s/it]
  6%|▋         | 2445/37962 [5:43:12<81:52:59,  8.30s/it]
  6%|▋         | 2446/37962 [5:43:20<81:01:49,  8.21s/it]
  6%|▋         | 2447/37962 [5:43:28<81:01:14,  8.21s/it]
  6%|▋         | 2448/37962 [5:43:36<81:26:17,  8.26s/it]
  6%|▋         | 2449/37962 [5:43:45<81:52:33,  8.30s/it]
  6%|▋         | 2450/37962 [5:43:53<81:32:55,  8.27s/it]
                                                         
{'loss': 1.3901, 'grad_norm': 28.412490844726562, 'learning_rate': 1.9937513818186162e-05, 'epoch': 0.06}

  6%|▋         | 2450/37962 [5:43:53<81:32:55,  8.27s/it]
  6%|▋         | 2451/37962 [5:44:01<81:36:14,  8.27s/it]
  6%|▋         | 2452/37962 [5:44:09<82:14:42,  8.34s/it]
  6%|▋         | 2453/37962 [5:44:18<81:35:06,  8.27s/it]
  6%|▋         | 2454/37962 [5:44:26<81:36:27,  8.27s/it]
  6%|▋         | 2455/37962 [5:44:34<81:18:20,  8.24s/it]
  6%|▋         | 2456/37962 [5:44:42<81:27:15,  8.26s/it]
  6%|▋         | 2457/37962 [5:44:50<80:54:50,  8.20s/it]
  6%|▋         | 2458/37962 [5:44:59<81:37:19,  8.28s/it]
  6%|▋         | 2459/37962 [5:45:07<80:46:13,  8.19s/it]
  6%|▋         | 2460/37962 [5:45:15<79:48:25,  8.09s/it]
                                                         
{'loss': 1.4919, 'grad_norm': 48.34121322631836, 'learning_rate': 1.9936557935802604e-05, 'epoch': 0.06}

  6%|▋         | 2460/37962 [5:45:15<79:48:25,  8.09s/it]
  6%|▋         | 2461/37962 [5:45:23<80:12:25,  8.13s/it]
  6%|▋         | 2462/37962 [5:45:31<81:04:57,  8.22s/it]
  6%|▋         | 2463/37962 [5:45:39<80:26:01,  8.16s/it]
  6%|▋         | 2464/37962 [5:45:48<80:45:14,  8.19s/it]
  6%|▋         | 2465/37962 [5:45:56<81:32:24,  8.27s/it]
  6%|▋         | 2466/37962 [5:46:04<81:38:47,  8.28s/it]
  6%|▋         | 2467/37962 [5:46:13<81:20:37,  8.25s/it]
  7%|▋         | 2468/37962 [5:46:21<81:00:40,  8.22s/it]
  7%|▋         | 2469/37962 [5:46:29<81:17:51,  8.25s/it]
  7%|▋         | 2470/37962 [5:46:38<82:02:21,  8.32s/it]
                                                         
{'loss': 1.5662, 'grad_norm': 39.16920471191406, 'learning_rate': 1.9935594820770663e-05, 'epoch': 0.07}

  7%|▋         | 2470/37962 [5:46:38<82:02:21,  8.32s/it]
  7%|▋         | 2471/37962 [5:46:46<81:12:59,  8.24s/it]
  7%|▋         | 2472/37962 [5:46:54<81:08:19,  8.23s/it]
  7%|▋         | 2473/37962 [5:47:02<81:09:20,  8.23s/it]
  7%|▋         | 2474/37962 [5:47:10<81:35:11,  8.28s/it]
  7%|▋         | 2475/37962 [5:47:19<81:45:39,  8.29s/it]
  7%|▋         | 2476/37962 [5:47:27<81:27:21,  8.26s/it]
  7%|▋         | 2477/37962 [5:47:35<81:03:38,  8.22s/it]
  7%|▋         | 2478/37962 [5:47:43<81:19:22,  8.25s/it]
  7%|▋         | 2479/37962 [5:47:52<81:09:46,  8.23s/it]
  7%|▋         | 2480/37962 [5:48:00<81:04:19,  8.23s/it]
                                                         
{'loss': 1.6143, 'grad_norm': 8.62296199798584, 'learning_rate': 1.9934624473791375e-05, 'epoch': 0.07}

  7%|▋         | 2480/37962 [5:48:00<81:04:19,  8.23s/it]
  7%|▋         | 2481/37962 [5:48:08<81:10:54,  8.24s/it]
  7%|▋         | 2482/37962 [5:48:17<82:05:49,  8.33s/it]
  7%|▋         | 2483/37962 [5:48:25<81:55:45,  8.31s/it]
  7%|▋         | 2484/37962 [5:48:33<82:37:09,  8.38s/it]
  7%|▋         | 2485/37962 [5:48:42<82:01:16,  8.32s/it]
  7%|▋         | 2486/37962 [5:48:50<82:04:22,  8.33s/it]
  7%|▋         | 2487/37962 [5:48:58<82:05:30,  8.33s/it]
  7%|▋         | 2488/37962 [5:49:07<81:50:31,  8.31s/it]
  7%|▋         | 2489/37962 [5:49:15<81:55:55,  8.31s/it]
  7%|▋         | 2490/37962 [5:49:23<82:24:43,  8.36s/it]
                                                         
{'loss': 1.6641, 'grad_norm': 37.58180236816406, 'learning_rate': 1.9933646895571036e-05, 'epoch': 0.07}

  7%|▋         | 2490/37962 [5:49:23<82:24:43,  8.36s/it]
  7%|▋         | 2491/37962 [5:49:31<81:50:11,  8.31s/it]
  7%|▋         | 2492/37962 [5:49:40<81:39:02,  8.29s/it]
  7%|▋         | 2493/37962 [5:49:48<80:22:25,  8.16s/it]
  7%|▋         | 2494/37962 [5:49:56<80:33:34,  8.18s/it]
  7%|▋         | 2495/37962 [5:50:04<81:13:19,  8.24s/it]
  7%|▋         | 2496/37962 [5:50:12<80:41:05,  8.19s/it]
  7%|▋         | 2497/37962 [5:50:20<80:44:18,  8.20s/it]
  7%|▋         | 2498/37962 [5:50:29<81:18:36,  8.25s/it]
  7%|▋         | 2499/37962 [5:50:37<80:32:16,  8.18s/it]
  7%|▋         | 2500/37962 [5:50:45<80:47:06,  8.20s/it]
                                                         
{'loss': 1.4943, 'grad_norm': 16.27813720703125, 'learning_rate': 1.9932662086821215e-05, 'epoch': 0.07}

  7%|▋         | 2500/37962 [5:50:45<80:47:06,  8.20s/it]
  7%|▋         | 2501/37962 [5:50:53<80:27:47,  8.17s/it]
  7%|▋         | 2502/37962 [5:51:01<80:24:56,  8.16s/it]
  7%|▋         | 2503/37962 [5:51:10<80:53:11,  8.21s/it]
  7%|▋         | 2504/37962 [5:51:18<80:46:42,  8.20s/it]
  7%|▋         | 2505/37962 [5:51:26<80:11:06,  8.14s/it]
  7%|▋         | 2506/37962 [5:51:34<80:49:49,  8.21s/it]
  7%|▋         | 2507/37962 [5:51:43<81:58:28,  8.32s/it]
  7%|▋         | 2508/37962 [5:51:51<81:42:30,  8.30s/it]
  7%|▋         | 2509/37962 [5:52:00<82:18:06,  8.36s/it]
  7%|▋         | 2510/37962 [5:52:08<82:02:32,  8.33s/it]
                                                         
{'loss': 1.5566, 'grad_norm': 28.223989486694336, 'learning_rate': 1.993167004825873e-05, 'epoch': 0.07}

  7%|▋         | 2510/37962 [5:52:08<82:02:32,  8.33s/it]
  7%|▋         | 2511/37962 [5:52:16<81:39:26,  8.29s/it]
  7%|▋         | 2512/37962 [5:52:24<81:32:12,  8.28s/it]
  7%|▋         | 2513/37962 [5:52:33<81:36:17,  8.29s/it]
  7%|▋         | 2514/37962 [5:52:41<82:05:34,  8.34s/it]
  7%|▋         | 2515/37962 [5:52:49<81:38:17,  8.29s/it]
  7%|▋         | 2516/37962 [5:52:58<82:31:16,  8.38s/it]
  7%|▋         | 2517/37962 [5:53:06<81:34:06,  8.28s/it]
  7%|▋         | 2518/37962 [5:53:14<81:16:00,  8.25s/it]
  7%|▋         | 2519/37962 [5:53:23<82:00:22,  8.33s/it]
  7%|▋         | 2520/37962 [5:53:31<82:09:44,  8.35s/it]
                                                         
{'loss': 1.527, 'grad_norm': 12.884455680847168, 'learning_rate': 1.9930670780605673e-05, 'epoch': 0.07}

  7%|▋         | 2520/37962 [5:53:31<82:09:44,  8.35s/it]
  7%|▋         | 2521/37962 [5:53:39<82:38:51,  8.40s/it]
  7%|▋         | 2522/37962 [5:53:48<82:13:22,  8.35s/it]
  7%|▋         | 2523/37962 [5:53:56<81:56:36,  8.32s/it]
  7%|▋         | 2524/37962 [5:54:04<82:07:59,  8.34s/it]
  7%|▋         | 2525/37962 [5:54:13<82:19:54,  8.36s/it]
  7%|▋         | 2526/37962 [5:54:21<81:33:00,  8.28s/it]
  7%|▋         | 2527/37962 [5:54:29<81:24:06,  8.27s/it]
  7%|▋         | 2528/37962 [5:54:37<81:37:15,  8.29s/it]
  7%|▋         | 2529/37962 [5:54:46<82:18:49,  8.36s/it]
  7%|▋         | 2530/37962 [5:54:54<82:33:52,  8.39s/it]
                                                         
{'loss': 1.6302, 'grad_norm': 28.79819679260254, 'learning_rate': 1.992966428458939e-05, 'epoch': 0.07}

  7%|▋         | 2530/37962 [5:54:54<82:33:52,  8.39s/it]
  7%|▋         | 2531/37962 [5:55:03<82:08:45,  8.35s/it]
  7%|▋         | 2532/37962 [5:55:11<80:45:47,  8.21s/it]
  7%|▋         | 2533/37962 [5:55:19<81:19:33,  8.26s/it]
  7%|▋         | 2534/37962 [5:55:27<81:06:51,  8.24s/it]
  7%|▋         | 2535/37962 [5:55:35<81:11:19,  8.25s/it]
  7%|▋         | 2536/37962 [5:55:44<81:11:57,  8.25s/it]
  7%|▋         | 2537/37962 [5:55:52<81:34:33,  8.29s/it]
  7%|▋         | 2538/37962 [5:56:00<81:31:09,  8.28s/it]
  7%|▋         | 2539/37962 [5:56:09<81:53:40,  8.32s/it]
  7%|▋         | 2540/37962 [5:56:17<81:25:00,  8.27s/it]
                                                         
{'loss': 1.5155, 'grad_norm': 24.192474365234375, 'learning_rate': 1.992865056094249e-05, 'epoch': 0.07}

  7%|▋         | 2540/37962 [5:56:17<81:25:00,  8.27s/it]
  7%|▋         | 2541/37962 [5:56:25<80:34:11,  8.19s/it]
  7%|▋         | 2542/37962 [5:56:33<81:07:11,  8.24s/it]
  7%|▋         | 2543/37962 [5:56:41<81:03:20,  8.24s/it]
  7%|▋         | 2544/37962 [5:56:50<81:17:01,  8.26s/it]
  7%|▋         | 2545/37962 [5:56:58<81:06:05,  8.24s/it]
  7%|▋         | 2546/37962 [5:57:06<81:25:20,  8.28s/it]
  7%|▋         | 2547/37962 [5:57:15<81:36:21,  8.30s/it]
  7%|▋         | 2548/37962 [5:57:23<81:25:33,  8.28s/it]
  7%|▋         | 2549/37962 [5:57:31<81:38:12,  8.30s/it]
  7%|▋         | 2550/37962 [5:57:40<81:28:49,  8.28s/it]
                                                         
{'loss': 1.383, 'grad_norm': 7.118890762329102, 'learning_rate': 1.9927629610402857e-05, 'epoch': 0.07}

  7%|▋         | 2550/37962 [5:57:40<81:28:49,  8.28s/it]
  7%|▋         | 2551/37962 [5:57:48<81:34:07,  8.29s/it]
  7%|▋         | 2552/37962 [5:57:56<82:03:32,  8.34s/it]
  7%|▋         | 2553/37962 [5:58:04<80:40:20,  8.20s/it]
  7%|▋         | 2554/37962 [5:58:12<80:03:52,  8.14s/it]
  7%|▋         | 2555/37962 [5:58:21<81:03:26,  8.24s/it]
  7%|▋         | 2556/37962 [5:58:29<81:50:07,  8.32s/it]
  7%|▋         | 2557/37962 [5:58:37<81:52:08,  8.32s/it]
  7%|▋         | 2558/37962 [5:58:46<82:01:26,  8.34s/it]
  7%|▋         | 2559/37962 [5:58:54<82:37:43,  8.40s/it]
  7%|▋         | 2560/37962 [5:59:03<82:37:34,  8.40s/it]
                                                         
{'loss': 1.6273, 'grad_norm': 18.268516540527344, 'learning_rate': 1.992660143371361e-05, 'epoch': 0.07}

  7%|▋         | 2560/37962 [5:59:03<82:37:34,  8.40s/it]
  7%|▋         | 2561/37962 [5:59:11<82:27:54,  8.39s/it]
  7%|▋         | 2562/37962 [5:59:20<82:18:53,  8.37s/it]
  7%|▋         | 2563/37962 [5:59:28<82:09:27,  8.36s/it]
  7%|▋         | 2564/37962 [5:59:36<82:28:47,  8.39s/it]
  7%|▋         | 2565/37962 [5:59:45<82:16:31,  8.37s/it]
  7%|▋         | 2566/37962 [5:59:52<80:48:08,  8.22s/it]
  7%|▋         | 2567/37962 [6:00:01<81:05:42,  8.25s/it]
  7%|▋         | 2568/37962 [6:00:09<81:41:27,  8.31s/it]
  7%|▋         | 2569/37962 [6:00:17<81:22:20,  8.28s/it]
  7%|▋         | 2570/37962 [6:00:26<81:09:41,  8.26s/it]
                                                         
{'loss': 1.5442, 'grad_norm': 31.832805633544922, 'learning_rate': 1.9925566031623146e-05, 'epoch': 0.07}

  7%|▋         | 2570/37962 [6:00:26<81:09:41,  8.26s/it]
  7%|▋         | 2571/37962 [6:00:34<80:59:31,  8.24s/it]
  7%|▋         | 2572/37962 [6:00:42<80:16:28,  8.17s/it]
  7%|▋         | 2573/37962 [6:00:50<80:36:08,  8.20s/it]
  7%|▋         | 2574/37962 [6:00:58<80:29:54,  8.19s/it]
  7%|▋         | 2575/37962 [6:01:07<80:53:58,  8.23s/it]
  7%|▋         | 2576/37962 [6:01:15<80:32:03,  8.19s/it]
  7%|▋         | 2577/37962 [6:01:23<80:44:26,  8.21s/it]
  7%|▋         | 2578/37962 [6:01:31<81:16:17,  8.27s/it]
  7%|▋         | 2579/37962 [6:01:40<81:15:57,  8.27s/it]
  7%|▋         | 2580/37962 [6:01:47<79:47:50,  8.12s/it]
                                                         
{'loss': 1.6271, 'grad_norm': 25.95115852355957, 'learning_rate': 1.9924523404885118e-05, 'epoch': 0.07}

  7%|▋         | 2580/37962 [6:01:47<79:47:50,  8.12s/it]
  7%|▋         | 2581/37962 [6:01:55<79:20:34,  8.07s/it]
  7%|▋         | 2582/37962 [6:02:04<79:42:30,  8.11s/it]
  7%|▋         | 2583/37962 [6:02:12<80:12:28,  8.16s/it]
  7%|▋         | 2584/37962 [6:02:20<81:00:28,  8.24s/it]
  7%|▋         | 2585/37962 [6:02:29<81:41:09,  8.31s/it]
  7%|▋         | 2586/37962 [6:02:37<81:30:32,  8.29s/it]
  7%|▋         | 2587/37962 [6:02:45<81:16:05,  8.27s/it]
  7%|▋         | 2588/37962 [6:02:54<81:17:38,  8.27s/it]
  7%|▋         | 2589/37962 [6:03:02<80:48:14,  8.22s/it]
  7%|▋         | 2590/37962 [6:03:10<80:23:56,  8.18s/it]
                                                         
{'loss': 1.5992, 'grad_norm': 27.0264949798584, 'learning_rate': 1.9923473554258434e-05, 'epoch': 0.07}

  7%|▋         | 2590/37962 [6:03:10<80:23:56,  8.18s/it]
  7%|▋         | 2591/37962 [6:03:18<79:46:24,  8.12s/it]
  7%|▋         | 2592/37962 [6:03:26<79:07:03,  8.05s/it]
  7%|▋         | 2593/37962 [6:03:34<78:57:17,  8.04s/it]
  7%|▋         | 2594/37962 [6:03:42<79:23:38,  8.08s/it]
  7%|▋         | 2595/37962 [6:03:50<80:07:45,  8.16s/it]
  7%|▋         | 2596/37962 [6:03:58<79:58:17,  8.14s/it]
  7%|▋         | 2597/37962 [6:04:07<80:39:02,  8.21s/it]
  7%|▋         | 2598/37962 [6:04:15<81:07:38,  8.26s/it]
  7%|▋         | 2599/37962 [6:04:23<81:41:48,  8.32s/it]
  7%|▋         | 2600/37962 [6:04:31<80:06:19,  8.16s/it]
                                                         
{'loss': 4.5666, 'grad_norm': 47.3375358581543, 'learning_rate': 1.992241648050726e-05, 'epoch': 0.07}

  7%|▋         | 2600/37962 [6:04:31<80:06:19,  8.16s/it]
  7%|▋         | 2601/37962 [6:04:39<79:56:14,  8.14s/it]
  7%|▋         | 2602/37962 [6:04:47<80:10:16,  8.16s/it]
  7%|▋         | 2603/37962 [6:04:56<80:36:16,  8.21s/it]
  7%|▋         | 2604/37962 [6:05:04<80:01:24,  8.15s/it]
  7%|▋         | 2605/37962 [6:05:12<80:03:02,  8.15s/it]
  7%|▋         | 2606/37962 [6:05:20<80:44:09,  8.22s/it]
  7%|▋         | 2607/37962 [6:05:28<79:33:18,  8.10s/it]
  7%|▋         | 2608/37962 [6:05:36<79:53:46,  8.14s/it]
  7%|▋         | 2609/37962 [6:05:45<80:11:12,  8.17s/it]
  7%|▋         | 2610/37962 [6:05:53<80:29:25,  8.20s/it]
                                                         
{'loss': 1.5641, 'grad_norm': 96.86378479003906, 'learning_rate': 1.992135218440103e-05, 'epoch': 0.07}

  7%|▋         | 2610/37962 [6:05:53<80:29:25,  8.20s/it]
  7%|▋         | 2611/37962 [6:06:01<79:43:33,  8.12s/it]
  7%|▋         | 2612/37962 [6:06:09<80:40:51,  8.22s/it]
  7%|▋         | 2613/37962 [6:06:17<79:54:41,  8.14s/it]
  7%|▋         | 2614/37962 [6:06:25<78:48:16,  8.03s/it]
  7%|▋         | 2615/37962 [6:06:33<77:28:34,  7.89s/it]
  7%|▋         | 2616/37962 [6:06:41<78:29:00,  7.99s/it]
  7%|▋         | 2617/37962 [6:06:49<78:09:45,  7.96s/it]
  7%|▋         | 2618/37962 [6:06:57<78:50:44,  8.03s/it]
  7%|▋         | 2619/37962 [6:07:05<79:39:59,  8.11s/it]
  7%|▋         | 2620/37962 [6:07:14<80:58:06,  8.25s/it]
                                                         
{'loss': 1.5759, 'grad_norm': 12.757888793945312, 'learning_rate': 1.9920280666714413e-05, 'epoch': 0.07}

  7%|▋         | 2620/37962 [6:07:14<80:58:06,  8.25s/it]
  7%|▋         | 2621/37962 [6:07:22<80:46:44,  8.23s/it]
  7%|▋         | 2622/37962 [6:07:30<79:46:00,  8.13s/it]
  7%|▋         | 2623/37962 [6:07:38<80:13:50,  8.17s/it]
  7%|▋         | 2624/37962 [6:07:46<80:44:55,  8.23s/it]
  7%|▋         | 2625/37962 [6:07:55<80:47:38,  8.23s/it]
  7%|▋         | 2626/37962 [6:08:03<81:07:46,  8.27s/it]
  7%|▋         | 2627/37962 [6:08:11<81:30:37,  8.30s/it]
  7%|▋         | 2628/37962 [6:08:20<80:53:30,  8.24s/it]
  7%|▋         | 2629/37962 [6:08:28<80:50:12,  8.24s/it]
  7%|▋         | 2630/37962 [6:08:36<81:18:51,  8.29s/it]
                                                         
{'loss': 1.4545, 'grad_norm': 18.262788772583008, 'learning_rate': 1.991920192822736e-05, 'epoch': 0.07}

  7%|▋         | 2630/37962 [6:08:36<81:18:51,  8.29s/it]
  7%|▋         | 2631/37962 [6:08:44<81:01:11,  8.26s/it]
  7%|▋         | 2632/37962 [6:08:52<79:59:14,  8.15s/it]
  7%|▋         | 2633/37962 [6:09:00<79:20:02,  8.08s/it]
  7%|▋         | 2634/37962 [6:09:08<79:34:01,  8.11s/it]
  7%|▋         | 2635/37962 [6:09:17<80:06:00,  8.16s/it]
  7%|▋         | 2636/37962 [6:09:25<81:00:10,  8.25s/it]
  7%|▋         | 2637/37962 [6:09:34<81:46:07,  8.33s/it]
  7%|▋         | 2638/37962 [6:09:42<81:31:22,  8.31s/it]
  7%|▋         | 2639/37962 [6:09:50<80:39:20,  8.22s/it]
  7%|▋         | 2640/37962 [6:09:58<80:32:22,  8.21s/it]
                                                         
{'loss': 1.5043, 'grad_norm': 7.8612380027771, 'learning_rate': 1.991811596972506e-05, 'epoch': 0.07}

  7%|▋         | 2640/37962 [6:09:58<80:32:22,  8.21s/it]
  7%|▋         | 2641/37962 [6:10:06<81:03:20,  8.26s/it]
  7%|▋         | 2642/37962 [6:10:14<79:40:01,  8.12s/it]
  7%|▋         | 2643/37962 [6:10:23<80:10:48,  8.17s/it]
  7%|▋         | 2644/37962 [6:10:31<80:04:46,  8.16s/it]
  7%|▋         | 2645/37962 [6:10:38<79:02:27,  8.06s/it]
  7%|▋         | 2646/37962 [6:10:47<79:42:06,  8.12s/it]
  7%|▋         | 2647/37962 [6:10:55<79:29:44,  8.10s/it]
  7%|▋         | 2648/37962 [6:11:03<79:50:41,  8.14s/it]
  7%|▋         | 2649/37962 [6:11:11<80:35:49,  8.22s/it]
  7%|▋         | 2650/37962 [6:11:20<80:15:09,  8.18s/it]
                                                         
{'loss': 1.4525, 'grad_norm': 9.754074096679688, 'learning_rate': 1.991702279199797e-05, 'epoch': 0.07}

  7%|▋         | 2650/37962 [6:11:20<80:15:09,  8.18s/it]
  7%|▋         | 2651/37962 [6:11:28<80:17:24,  8.19s/it]
  7%|▋         | 2652/37962 [6:11:36<81:05:38,  8.27s/it]
  7%|▋         | 2653/37962 [6:11:44<81:01:14,  8.26s/it]
  7%|▋         | 2654/37962 [6:11:53<81:13:47,  8.28s/it]
  7%|▋         | 2655/37962 [6:12:01<81:18:54,  8.29s/it]
  7%|▋         | 2656/37962 [6:12:09<80:58:34,  8.26s/it]
  7%|▋         | 2657/37962 [6:12:18<81:00:51,  8.26s/it]
  7%|▋         | 2658/37962 [6:12:26<81:12:59,  8.28s/it]
  7%|▋         | 2659/37962 [6:12:34<81:15:36,  8.29s/it]
  7%|▋         | 2660/37962 [6:12:43<81:35:57,  8.32s/it]
                                                         
{'loss': 1.5317, 'grad_norm': 34.565391540527344, 'learning_rate': 1.9915922395841783e-05, 'epoch': 0.07}

  7%|▋         | 2660/37962 [6:12:43<81:35:57,  8.32s/it]
  7%|▋         | 2661/37962 [6:12:51<81:20:42,  8.30s/it]
  7%|▋         | 2662/37962 [6:12:59<81:19:59,  8.29s/it]
  7%|▋         | 2663/37962 [6:13:07<81:30:46,  8.31s/it]
  7%|▋         | 2664/37962 [6:13:16<80:47:46,  8.24s/it]
  7%|▋         | 2665/37962 [6:13:24<80:50:24,  8.25s/it]
  7%|▋         | 2666/37962 [6:13:32<80:55:09,  8.25s/it]
  7%|▋         | 2667/37962 [6:13:40<80:41:09,  8.23s/it]
  7%|▋         | 2668/37962 [6:13:49<81:22:28,  8.30s/it]
  7%|▋         | 2669/37962 [6:13:57<81:05:27,  8.27s/it]
  7%|▋         | 2670/37962 [6:14:05<81:12:06,  8.28s/it]
                                                         
{'loss': 1.4435, 'grad_norm': 3.683598279953003, 'learning_rate': 1.991481478205747e-05, 'epoch': 0.07}

  7%|▋         | 2670/37962 [6:14:05<81:12:06,  8.28s/it]
  7%|▋         | 2671/37962 [6:14:13<81:10:12,  8.28s/it]
  7%|▋         | 2672/37962 [6:14:22<81:31:51,  8.32s/it]
  7%|▋         | 2673/37962 [6:14:30<80:35:07,  8.22s/it]
  7%|▋         | 2674/37962 [6:14:38<80:51:58,  8.25s/it]
  7%|▋         | 2675/37962 [6:14:46<80:29:30,  8.21s/it]
  7%|▋         | 2676/37962 [6:14:55<80:41:26,  8.23s/it]
  7%|▋         | 2677/37962 [6:15:03<80:11:21,  8.18s/it]
  7%|▋         | 2678/37962 [6:15:11<80:57:56,  8.26s/it]
  7%|▋         | 2679/37962 [6:15:19<80:57:27,  8.26s/it]
  7%|▋         | 2680/37962 [6:15:28<80:50:55,  8.25s/it]
                                                         
{'loss': 1.4105, 'grad_norm': 9.371899604797363, 'learning_rate': 1.9913699951451238e-05, 'epoch': 0.07}

  7%|▋         | 2680/37962 [6:15:28<80:50:55,  8.25s/it]
  7%|▋         | 2681/37962 [6:15:36<81:37:57,  8.33s/it]
  7%|▋         | 2682/37962 [6:15:44<81:12:01,  8.29s/it]
  7%|▋         | 2683/37962 [6:15:53<81:00:23,  8.27s/it]
  7%|▋         | 2684/37962 [6:16:01<80:55:36,  8.26s/it]
  7%|▋         | 2685/37962 [6:16:09<81:10:31,  8.28s/it]
  7%|▋         | 2686/37962 [6:16:17<80:50:24,  8.25s/it]
  7%|▋         | 2687/37962 [6:16:26<80:53:34,  8.26s/it]
  7%|▋         | 2688/37962 [6:16:34<81:31:21,  8.32s/it]
  7%|▋         | 2689/37962 [6:16:42<81:41:50,  8.34s/it]
  7%|▋         | 2690/37962 [6:16:50<80:47:47,  8.25s/it]
                                                         
{'loss': 1.5331, 'grad_norm': 7.9052324295043945, 'learning_rate': 1.9912577904834557e-05, 'epoch': 0.07}

  7%|▋         | 2690/37962 [6:16:50<80:47:47,  8.25s/it]
  7%|▋         | 2691/37962 [6:16:58<78:40:14,  8.03s/it]
  7%|▋         | 2692/37962 [6:17:06<77:56:23,  7.96s/it]
  7%|▋         | 2693/37962 [6:17:14<78:40:24,  8.03s/it]
  7%|▋         | 2694/37962 [6:17:22<79:07:53,  8.08s/it]
  7%|▋         | 2695/37962 [6:17:30<79:42:34,  8.14s/it]
  7%|▋         | 2696/37962 [6:17:39<79:49:23,  8.15s/it]
  7%|▋         | 2697/37962 [6:17:47<80:04:15,  8.17s/it]
  7%|▋         | 2698/37962 [6:17:55<79:55:06,  8.16s/it]
  7%|▋         | 2699/37962 [6:18:03<79:57:26,  8.16s/it]
  7%|▋         | 2700/37962 [6:18:11<80:39:20,  8.23s/it]
                                                         
{'loss': 1.5714, 'grad_norm': 23.81945037841797, 'learning_rate': 1.9911448643024142e-05, 'epoch': 0.07}

  7%|▋         | 2700/37962 [6:18:11<80:39:20,  8.23s/it]
  7%|▋         | 2701/37962 [6:18:19<79:34:06,  8.12s/it]
  7%|▋         | 2702/37962 [6:18:28<80:01:20,  8.17s/it]
  7%|▋         | 2703/37962 [6:18:36<79:42:10,  8.14s/it]
  7%|▋         | 2704/37962 [6:18:44<79:56:23,  8.16s/it]
  7%|▋         | 2705/37962 [6:18:52<80:06:20,  8.18s/it]
  7%|▋         | 2706/37962 [6:19:00<80:24:20,  8.21s/it]
  7%|▋         | 2707/37962 [6:19:09<81:21:41,  8.31s/it]
  7%|▋         | 2708/37962 [6:19:17<81:26:02,  8.32s/it]
  7%|▋         | 2709/37962 [6:19:25<81:06:32,  8.28s/it]
  7%|▋         | 2710/37962 [6:19:34<81:30:33,  8.32s/it]
                                                         
{'loss': 1.4574, 'grad_norm': 11.587237358093262, 'learning_rate': 1.991031216684196e-05, 'epoch': 0.07}

  7%|▋         | 2710/37962 [6:19:34<81:30:33,  8.32s/it]
  7%|▋         | 2711/37962 [6:19:42<82:03:53,  8.38s/it]
  7%|▋         | 2712/37962 [6:19:51<81:33:09,  8.33s/it]
  7%|▋         | 2713/37962 [6:19:59<81:10:55,  8.29s/it]
  7%|▋         | 2714/37962 [6:20:07<80:41:08,  8.24s/it]
  7%|▋         | 2715/37962 [6:20:15<80:57:46,  8.27s/it]
  7%|▋         | 2716/37962 [6:20:23<80:26:34,  8.22s/it]
  7%|▋         | 2717/37962 [6:20:32<80:55:35,  8.27s/it]
  7%|▋         | 2718/37962 [6:20:40<80:26:18,  8.22s/it]
  7%|▋         | 2719/37962 [6:20:48<79:51:09,  8.16s/it]
  7%|▋         | 2720/37962 [6:20:56<78:50:57,  8.05s/it]
                                                         
{'loss': 1.6168, 'grad_norm': 17.335559844970703, 'learning_rate': 1.990916847711524e-05, 'epoch': 0.07}

  7%|▋         | 2720/37962 [6:20:56<78:50:57,  8.05s/it]
  7%|▋         | 2721/37962 [6:21:04<79:43:03,  8.14s/it]
  7%|▋         | 2722/37962 [6:21:12<80:12:46,  8.19s/it]
  7%|▋         | 2723/37962 [6:21:21<80:40:23,  8.24s/it]
  7%|▋         | 2724/37962 [6:21:29<80:50:38,  8.26s/it]
  7%|▋         | 2725/37962 [6:21:37<80:31:33,  8.23s/it]
  7%|▋         | 2726/37962 [6:21:45<79:25:31,  8.11s/it]
  7%|▋         | 2727/37962 [6:21:53<79:36:57,  8.13s/it]
  7%|▋         | 2728/37962 [6:22:02<80:31:36,  8.23s/it]
  7%|▋         | 2729/37962 [6:22:10<80:16:31,  8.20s/it]
  7%|▋         | 2730/37962 [6:22:18<80:44:30,  8.25s/it]
                                                         
{'loss': 1.6199, 'grad_norm': 13.279732704162598, 'learning_rate': 1.990801757467645e-05, 'epoch': 0.07}

  7%|▋         | 2730/37962 [6:22:18<80:44:30,  8.25s/it]
  7%|▋         | 2731/37962 [6:22:27<81:00:25,  8.28s/it]
  7%|▋         | 2732/37962 [6:22:35<81:18:20,  8.31s/it]
  7%|▋         | 2733/37962 [6:22:43<81:23:49,  8.32s/it]
  7%|▋         | 2734/37962 [6:22:51<81:02:24,  8.28s/it]
  7%|▋         | 2735/37962 [6:23:00<80:59:36,  8.28s/it]
  7%|▋         | 2736/37962 [6:23:08<81:24:09,  8.32s/it]
  7%|▋         | 2737/37962 [6:23:16<81:28:58,  8.33s/it]
  7%|▋         | 2738/37962 [6:23:25<81:07:23,  8.29s/it]
  7%|▋         | 2739/37962 [6:23:33<81:22:26,  8.32s/it]
  7%|▋         | 2740/37962 [6:23:41<81:23:49,  8.32s/it]
                                                         
{'loss': 1.5208, 'grad_norm': 22.874755859375, 'learning_rate': 1.9906859460363307e-05, 'epoch': 0.07}

  7%|▋         | 2740/37962 [6:23:41<81:23:49,  8.32s/it]
  7%|▋         | 2741/37962 [6:23:49<80:14:09,  8.20s/it]
  7%|▋         | 2742/37962 [6:23:58<80:25:05,  8.22s/it]
  7%|▋         | 2743/37962 [6:24:06<80:48:09,  8.26s/it]
  7%|▋         | 2744/37962 [6:24:14<80:46:46,  8.26s/it]
  7%|▋         | 2745/37962 [6:24:22<80:37:26,  8.24s/it]
  7%|▋         | 2746/37962 [6:24:31<81:16:23,  8.31s/it]
  7%|▋         | 2747/37962 [6:24:39<81:09:09,  8.30s/it]
  7%|▋         | 2748/37962 [6:24:48<81:45:41,  8.36s/it]
  7%|▋         | 2749/37962 [6:24:56<81:14:29,  8.31s/it]
  7%|▋         | 2750/37962 [6:25:04<81:29:56,  8.33s/it]
                                                         
{'loss': 1.4941, 'grad_norm': 6.159449100494385, 'learning_rate': 1.9905694135018795e-05, 'epoch': 0.07}

  7%|▋         | 2750/37962 [6:25:04<81:29:56,  8.33s/it]
  7%|▋         | 2751/37962 [6:25:13<81:37:46,  8.35s/it]
  7%|▋         | 2752/37962 [6:25:21<81:32:38,  8.34s/it]
  7%|▋         | 2753/37962 [6:25:29<81:25:35,  8.33s/it]
  7%|▋         | 2754/37962 [6:25:37<81:27:25,  8.33s/it]
  7%|▋         | 2755/37962 [6:25:46<80:39:50,  8.25s/it]
  7%|▋         | 2756/37962 [6:25:54<80:12:33,  8.20s/it]
  7%|▋         | 2757/37962 [6:26:02<80:37:03,  8.24s/it]
  7%|▋         | 2758/37962 [6:26:10<80:11:02,  8.20s/it]
  7%|▋         | 2759/37962 [6:26:18<80:36:40,  8.24s/it]
  7%|▋         | 2760/37962 [6:26:27<80:25:35,  8.22s/it]
                                                         
{'loss': 1.5377, 'grad_norm': 14.45523738861084, 'learning_rate': 1.9904521599491122e-05, 'epoch': 0.07}

  7%|▋         | 2760/37962 [6:26:27<80:25:35,  8.22s/it]
  7%|▋         | 2761/37962 [6:26:35<80:24:18,  8.22s/it]
  7%|▋         | 2762/37962 [6:26:43<80:36:51,  8.24s/it]
  7%|▋         | 2763/37962 [6:26:51<80:22:31,  8.22s/it]
  7%|▋         | 2764/37962 [6:27:00<80:23:50,  8.22s/it]
  7%|▋         | 2765/37962 [6:27:08<81:00:07,  8.28s/it]
  7%|▋         | 2766/37962 [6:27:16<81:34:38,  8.34s/it]
  7%|▋         | 2767/37962 [6:27:25<81:59:40,  8.39s/it]
  7%|▋         | 2768/37962 [6:27:33<81:52:21,  8.37s/it]
  7%|▋         | 2769/37962 [6:27:41<81:10:23,  8.30s/it]
  7%|▋         | 2770/37962 [6:27:50<80:50:03,  8.27s/it]
                                                         
{'loss': 1.5445, 'grad_norm': 19.553503036499023, 'learning_rate': 1.9903341854633762e-05, 'epoch': 0.07}

  7%|▋         | 2770/37962 [6:27:50<80:50:03,  8.27s/it]
  7%|▋         | 2771/37962 [6:27:58<80:49:57,  8.27s/it]
  7%|▋         | 2772/37962 [6:28:06<79:49:49,  8.17s/it]
  7%|▋         | 2773/37962 [6:28:14<80:04:24,  8.19s/it]
  7%|▋         | 2774/37962 [6:28:22<80:13:18,  8.21s/it]
  7%|▋         | 2775/37962 [6:28:30<79:51:15,  8.17s/it]
  7%|▋         | 2776/37962 [6:28:39<80:50:27,  8.27s/it]
  7%|▋         | 2777/37962 [6:28:47<80:14:02,  8.21s/it]
  7%|▋         | 2778/37962 [6:28:55<80:10:10,  8.20s/it]
  7%|▋         | 2779/37962 [6:29:03<79:19:15,  8.12s/it]
  7%|▋         | 2780/37962 [6:29:11<79:44:05,  8.16s/it]
                                                         
{'loss': 1.4254, 'grad_norm': 19.57867431640625, 'learning_rate': 1.990215490130543e-05, 'epoch': 0.07}

  7%|▋         | 2780/37962 [6:29:11<79:44:05,  8.16s/it]
  7%|▋         | 2781/37962 [6:29:20<80:37:48,  8.25s/it]
  7%|▋         | 2782/37962 [6:29:28<80:28:20,  8.23s/it]
  7%|▋         | 2783/37962 [6:29:36<80:03:40,  8.19s/it]
  7%|▋         | 2784/37962 [6:29:44<80:25:22,  8.23s/it]
  7%|▋         | 2785/37962 [6:29:53<80:14:09,  8.21s/it]
  7%|▋         | 2786/37962 [6:30:01<80:15:10,  8.21s/it]
  7%|▋         | 2787/37962 [6:30:09<80:22:29,  8.23s/it]
  7%|▋         | 2788/37962 [6:30:17<80:21:01,  8.22s/it]
  7%|▋         | 2789/37962 [6:30:25<79:25:41,  8.13s/it]
  7%|▋         | 2790/37962 [6:30:33<79:49:48,  8.17s/it]
                                                         
{'loss': 1.4432, 'grad_norm': 13.853572845458984, 'learning_rate': 1.9900960740370085e-05, 'epoch': 0.07}

  7%|▋         | 2790/37962 [6:30:33<79:49:48,  8.17s/it]
  7%|▋         | 2791/37962 [6:30:42<79:55:21,  8.18s/it]
  7%|▋         | 2792/37962 [6:30:50<80:00:16,  8.19s/it]
  7%|▋         | 2793/37962 [6:30:58<80:41:34,  8.26s/it]
  7%|▋         | 2794/37962 [6:31:07<80:57:16,  8.29s/it]
  7%|▋         | 2795/37962 [6:31:15<80:57:19,  8.29s/it]
  7%|▋         | 2796/37962 [6:31:23<80:48:36,  8.27s/it]
  7%|▋         | 2797/37962 [6:31:32<81:37:21,  8.36s/it]
  7%|▋         | 2798/37962 [6:31:40<80:22:28,  8.23s/it]
  7%|▋         | 2799/37962 [6:31:48<80:22:49,  8.23s/it]
  7%|▋         | 2800/37962 [6:31:56<80:26:09,  8.24s/it]
                                                         
{'loss': 1.5541, 'grad_norm': 46.768489837646484, 'learning_rate': 1.9899759372696943e-05, 'epoch': 0.07}

  7%|▋         | 2800/37962 [6:31:56<80:26:09,  8.24s/it]
  7%|▋         | 2801/37962 [6:32:04<79:51:16,  8.18s/it]
  7%|▋         | 2802/37962 [6:32:12<79:11:08,  8.11s/it]
  7%|▋         | 2803/37962 [6:32:20<79:55:20,  8.18s/it]
  7%|▋         | 2804/37962 [6:32:28<79:18:40,  8.12s/it]
  7%|▋         | 2805/37962 [6:32:36<79:05:28,  8.10s/it]
  7%|▋         | 2806/37962 [6:32:45<80:33:54,  8.25s/it]
  7%|▋         | 2807/37962 [6:32:53<80:07:33,  8.21s/it]
  7%|▋         | 2808/37962 [6:33:01<79:35:43,  8.15s/it]
  7%|▋         | 2809/37962 [6:33:09<79:51:15,  8.18s/it]
  7%|▋         | 2810/37962 [6:33:18<80:34:15,  8.25s/it]
                                                         
{'loss': 1.5375, 'grad_norm': 12.32218074798584, 'learning_rate': 1.9898550799160454e-05, 'epoch': 0.07}

  7%|▋         | 2810/37962 [6:33:18<80:34:15,  8.25s/it]
  7%|▋         | 2811/37962 [6:33:26<79:59:35,  8.19s/it]
  7%|▋         | 2812/37962 [6:33:34<80:33:20,  8.25s/it]
  7%|▋         | 2813/37962 [6:33:42<80:07:55,  8.21s/it]
  7%|▋         | 2814/37962 [6:33:51<80:07:24,  8.21s/it]
  7%|▋         | 2815/37962 [6:33:59<80:33:45,  8.25s/it]
  7%|▋         | 2816/37962 [6:34:07<80:11:26,  8.21s/it]
  7%|▋         | 2817/37962 [6:34:15<80:07:32,  8.21s/it]
  7%|▋         | 2818/37962 [6:34:24<80:26:03,  8.24s/it]
  7%|▋         | 2819/37962 [6:34:32<80:41:02,  8.27s/it]
  7%|▋         | 2820/37962 [6:34:40<80:13:33,  8.22s/it]
                                                         
{'loss': 1.5565, 'grad_norm': 36.678226470947266, 'learning_rate': 1.9897335020640322e-05, 'epoch': 0.07}

  7%|▋         | 2820/37962 [6:34:40<80:13:33,  8.22s/it]
  7%|▋         | 2821/37962 [6:34:48<80:07:39,  8.21s/it]
  7%|▋         | 2822/37962 [6:34:57<81:13:55,  8.32s/it]
  7%|▋         | 2823/37962 [6:35:05<81:25:44,  8.34s/it]
  7%|▋         | 2824/37962 [6:35:13<81:12:52,  8.32s/it]
  7%|▋         | 2825/37962 [6:35:22<81:00:18,  8.30s/it]
  7%|▋         | 2826/37962 [6:35:30<80:45:56,  8.28s/it]
  7%|▋         | 2827/37962 [6:35:38<80:05:47,  8.21s/it]
  7%|▋         | 2828/37962 [6:35:46<80:29:05,  8.25s/it]
  7%|▋         | 2829/37962 [6:35:55<80:51:54,  8.29s/it]
  7%|▋         | 2830/37962 [6:36:03<80:54:30,  8.29s/it]
                                                         
{'loss': 1.613, 'grad_norm': 10.698866844177246, 'learning_rate': 1.9896112038021483e-05, 'epoch': 0.07}

  7%|▋         | 2830/37962 [6:36:03<80:54:30,  8.29s/it]
  7%|▋         | 2831/37962 [6:36:11<81:00:46,  8.30s/it]
  7%|▋         | 2832/37962 [6:36:20<81:03:51,  8.31s/it]
  7%|▋         | 2833/37962 [6:36:28<81:02:49,  8.31s/it]
  7%|▋         | 2834/37962 [6:36:36<80:24:49,  8.24s/it]
  7%|▋         | 2835/37962 [6:36:44<80:23:39,  8.24s/it]
  7%|▋         | 2836/37962 [6:36:52<80:04:42,  8.21s/it]
  7%|▋         | 2837/37962 [6:37:00<79:12:36,  8.12s/it]
  7%|▋         | 2838/37962 [6:37:09<79:36:27,  8.16s/it]
  7%|▋         | 2839/37962 [6:37:17<80:01:58,  8.20s/it]
  7%|▋         | 2840/37962 [6:37:25<79:16:33,  8.13s/it]
                                                         
{'loss': 1.4683, 'grad_norm': 17.686052322387695, 'learning_rate': 1.9894881852194135e-05, 'epoch': 0.07}

  7%|▋         | 2840/37962 [6:37:25<79:16:33,  8.13s/it]
  7%|▋         | 2841/37962 [6:37:33<79:46:26,  8.18s/it]
  7%|▋         | 2842/37962 [6:37:41<79:15:40,  8.12s/it]
  7%|▋         | 2843/37962 [6:37:49<78:28:13,  8.04s/it]
  7%|▋         | 2844/37962 [6:37:57<79:10:11,  8.12s/it]
  7%|▋         | 2845/37962 [6:38:06<79:58:32,  8.20s/it]
  7%|▋         | 2846/37962 [6:38:14<79:26:28,  8.14s/it]
  7%|▋         | 2847/37962 [6:38:22<79:58:03,  8.20s/it]
  8%|▊         | 2848/37962 [6:38:30<80:00:51,  8.20s/it]
  8%|▊         | 2849/37962 [6:38:38<80:05:44,  8.21s/it]
  8%|▊         | 2850/37962 [6:38:46<79:27:03,  8.15s/it]
                                                         
{'loss': 1.5298, 'grad_norm': 12.476663589477539, 'learning_rate': 1.9893644464053698e-05, 'epoch': 0.08}

  8%|▊         | 2850/37962 [6:38:46<79:27:03,  8.15s/it]
  8%|▊         | 2851/37962 [6:38:55<79:32:50,  8.16s/it]
  8%|▊         | 2852/37962 [6:39:03<80:13:13,  8.23s/it]
  8%|▊         | 2853/37962 [6:39:11<79:45:19,  8.18s/it]
  8%|▊         | 2854/37962 [6:39:19<79:42:58,  8.17s/it]
  8%|▊         | 2855/37962 [6:39:27<79:50:30,  8.19s/it]
  8%|▊         | 2856/37962 [6:39:36<80:11:00,  8.22s/it]
  8%|▊         | 2857/37962 [6:39:44<80:19:17,  8.24s/it]
  8%|▊         | 2858/37962 [6:39:52<80:16:06,  8.23s/it]
  8%|▊         | 2859/37962 [6:40:01<80:26:56,  8.25s/it]
  8%|▊         | 2860/37962 [6:40:09<80:46:58,  8.28s/it]
                                                         
{'loss': 1.6027, 'grad_norm': 39.018436431884766, 'learning_rate': 1.9892399874500857e-05, 'epoch': 0.08}

  8%|▊         | 2860/37962 [6:40:09<80:46:58,  8.28s/it]
  8%|▊         | 2861/37962 [6:40:17<81:14:51,  8.33s/it]
  8%|▊         | 2862/37962 [6:40:26<80:52:04,  8.29s/it]
  8%|▊         | 2863/37962 [6:40:34<80:44:01,  8.28s/it]
  8%|▊         | 2864/37962 [6:40:42<79:54:22,  8.20s/it]
  8%|▊         | 2865/37962 [6:40:50<79:56:39,  8.20s/it]
  8%|▊         | 2866/37962 [6:40:58<80:18:27,  8.24s/it]
  8%|▊         | 2867/37962 [6:41:07<80:43:17,  8.28s/it]
  8%|▊         | 2868/37962 [6:41:15<81:10:40,  8.33s/it]
  8%|▊         | 2869/37962 [6:41:23<81:07:08,  8.32s/it]
  8%|▊         | 2870/37962 [6:41:32<80:27:43,  8.25s/it]
                                                         
{'loss': 1.4975, 'grad_norm': 15.008227348327637, 'learning_rate': 1.989114808444152e-05, 'epoch': 0.08}

  8%|▊         | 2870/37962 [6:41:32<80:27:43,  8.25s/it]
  8%|▊         | 2871/37962 [6:41:40<80:04:50,  8.22s/it]
  8%|▊         | 2872/37962 [6:41:48<80:27:51,  8.26s/it]
  8%|▊         | 2873/37962 [6:41:56<80:23:02,  8.25s/it]
  8%|▊         | 2874/37962 [6:42:05<80:41:48,  8.28s/it]
  8%|▊         | 2875/37962 [6:42:13<80:04:51,  8.22s/it]
  8%|▊         | 2876/37962 [6:42:21<80:40:06,  8.28s/it]
  8%|▊         | 2877/37962 [6:42:29<80:47:31,  8.29s/it]
  8%|▊         | 2878/37962 [6:42:38<80:38:34,  8.27s/it]
  8%|▊         | 2879/37962 [6:42:46<80:19:58,  8.24s/it]
  8%|▊         | 2880/37962 [6:42:54<80:24:18,  8.25s/it]
                                                         
{'loss': 1.6336, 'grad_norm': 13.855270385742188, 'learning_rate': 1.988988909478684e-05, 'epoch': 0.08}

  8%|▊         | 2880/37962 [6:42:54<80:24:18,  8.25s/it]
  8%|▊         | 2881/37962 [6:43:02<80:28:14,  8.26s/it]
  8%|▊         | 2882/37962 [6:43:11<80:19:33,  8.24s/it]
  8%|▊         | 2883/37962 [6:43:19<80:03:00,  8.22s/it]
  8%|▊         | 2884/37962 [6:43:27<79:37:28,  8.17s/it]
  8%|▊         | 2885/37962 [6:43:35<79:44:25,  8.18s/it]
  8%|▊         | 2886/37962 [6:43:43<80:12:34,  8.23s/it]
  8%|▊         | 2887/37962 [6:43:52<80:16:40,  8.24s/it]
  8%|▊         | 2888/37962 [6:44:00<80:39:11,  8.28s/it]
  8%|▊         | 2889/37962 [6:44:08<80:50:41,  8.30s/it]
  8%|▊         | 2890/37962 [6:44:17<81:07:40,  8.33s/it]
                                                         
{'loss': 1.4263, 'grad_norm': 40.74266815185547, 'learning_rate': 1.9888622906453225e-05, 'epoch': 0.08}

  8%|▊         | 2890/37962 [6:44:17<81:07:40,  8.33s/it]
  8%|▊         | 2891/37962 [6:44:25<81:13:42,  8.34s/it]
  8%|▊         | 2892/37962 [6:44:33<81:22:36,  8.35s/it]
  8%|▊         | 2893/37962 [6:44:42<81:32:35,  8.37s/it]
  8%|▊         | 2894/37962 [6:44:50<80:32:07,  8.27s/it]
  8%|▊         | 2895/37962 [6:44:58<79:53:13,  8.20s/it]
  8%|▊         | 2896/37962 [6:45:06<80:07:21,  8.23s/it]
  8%|▊         | 2897/37962 [6:45:15<80:20:18,  8.25s/it]
  8%|▊         | 2898/37962 [6:45:23<80:06:55,  8.23s/it]
  8%|▊         | 2899/37962 [6:45:31<80:17:17,  8.24s/it]
  8%|▊         | 2900/37962 [6:45:39<79:56:13,  8.21s/it]
                                                         
{'loss': 1.2614, 'grad_norm': 21.152265548706055, 'learning_rate': 1.98873495203623e-05, 'epoch': 0.08}

  8%|▊         | 2900/37962 [6:45:39<79:56:13,  8.21s/it]
  8%|▊         | 2901/37962 [6:45:47<80:18:32,  8.25s/it]
  8%|▊         | 2902/37962 [6:45:56<80:22:49,  8.25s/it]
  8%|▊         | 2903/37962 [6:46:04<80:18:24,  8.25s/it]
  8%|▊         | 2904/37962 [6:46:12<80:48:48,  8.30s/it]
  8%|▊         | 2905/37962 [6:46:21<81:26:58,  8.36s/it]
  8%|▊         | 2906/37962 [6:46:29<80:10:39,  8.23s/it]
  8%|▊         | 2907/37962 [6:46:37<80:29:34,  8.27s/it]
  8%|▊         | 2908/37962 [6:46:46<80:56:06,  8.31s/it]
  8%|▊         | 2909/37962 [6:46:54<80:17:51,  8.25s/it]
  8%|▊         | 2910/37962 [6:47:02<80:59:21,  8.32s/it]
                                                         
{'loss': 1.5998, 'grad_norm': 5.441744327545166, 'learning_rate': 1.988606893744095e-05, 'epoch': 0.08}

  8%|▊         | 2910/37962 [6:47:02<80:59:21,  8.32s/it]
  8%|▊         | 2911/37962 [6:47:11<81:04:50,  8.33s/it]
  8%|▊         | 2912/37962 [6:47:19<80:36:46,  8.28s/it]
  8%|▊         | 2913/37962 [6:47:27<80:22:36,  8.26s/it]
  8%|▊         | 2914/37962 [6:47:36<82:00:53,  8.42s/it]
  8%|▊         | 2915/37962 [6:47:44<81:21:29,  8.36s/it]
  8%|▊         | 2916/37962 [6:47:52<80:58:45,  8.32s/it]
  8%|▊         | 2917/37962 [6:48:00<80:00:05,  8.22s/it]
  8%|▊         | 2918/37962 [6:48:08<79:52:05,  8.20s/it]
  8%|▊         | 2919/37962 [6:48:16<79:39:02,  8.18s/it]
  8%|▊         | 2920/37962 [6:48:25<79:23:25,  8.16s/it]
                                                         
{'loss': 1.4894, 'grad_norm': 30.66222381591797, 'learning_rate': 1.9884781158621283e-05, 'epoch': 0.08}

  8%|▊         | 2920/37962 [6:48:25<79:23:25,  8.16s/it]
  8%|▊         | 2921/37962 [6:48:33<79:46:47,  8.20s/it]
  8%|▊         | 2922/37962 [6:48:41<80:04:29,  8.23s/it]
  8%|▊         | 2923/37962 [6:48:49<80:17:07,  8.25s/it]
  8%|▊         | 2924/37962 [6:48:58<80:05:34,  8.23s/it]
  8%|▊         | 2925/37962 [6:49:06<80:06:40,  8.23s/it]
  8%|▊         | 2926/37962 [6:49:14<80:17:14,  8.25s/it]
  8%|▊         | 2927/37962 [6:49:23<81:04:29,  8.33s/it]
  8%|▊         | 2928/37962 [6:49:31<81:15:07,  8.35s/it]
  8%|▊         | 2929/37962 [6:49:39<81:02:30,  8.33s/it]
  8%|▊         | 2930/37962 [6:49:47<79:26:45,  8.16s/it]
                                                         
{'loss': 1.3842, 'grad_norm': 31.655927658081055, 'learning_rate': 1.9883486184840653e-05, 'epoch': 0.08}

  8%|▊         | 2930/37962 [6:49:47<79:26:45,  8.16s/it]
  8%|▊         | 2931/37962 [6:49:55<79:53:07,  8.21s/it]
  8%|▊         | 2932/37962 [6:50:03<78:31:42,  8.07s/it]
  8%|▊         | 2933/37962 [6:50:11<79:04:41,  8.13s/it]
  8%|▊         | 2934/37962 [6:50:20<79:33:03,  8.18s/it]
  8%|▊         | 2935/37962 [6:50:28<80:03:53,  8.23s/it]
  8%|▊         | 2936/37962 [6:50:37<80:51:11,  8.31s/it]
  8%|▊         | 2937/37962 [6:50:45<79:51:10,  8.21s/it]
  8%|▊         | 2938/37962 [6:50:53<79:55:05,  8.21s/it]
  8%|▊         | 2939/37962 [6:51:01<80:05:50,  8.23s/it]
  8%|▊         | 2940/37962 [6:51:09<80:30:44,  8.28s/it]
                                                         
{'loss': 1.631, 'grad_norm': 27.149215698242188, 'learning_rate': 1.9882184017041646e-05, 'epoch': 0.08}

  8%|▊         | 2940/37962 [6:51:09<80:30:44,  8.28s/it]
  8%|▊         | 2941/37962 [6:51:18<80:57:27,  8.32s/it]
  8%|▊         | 2942/37962 [6:51:26<80:34:49,  8.28s/it]
  8%|▊         | 2943/37962 [6:51:35<81:59:55,  8.43s/it]
  8%|▊         | 2944/37962 [6:51:43<81:08:49,  8.34s/it]
  8%|▊         | 2945/37962 [6:51:51<80:56:54,  8.32s/it]
  8%|▊         | 2946/37962 [6:51:59<79:52:28,  8.21s/it]
  8%|▊         | 2947/37962 [6:52:07<80:09:49,  8.24s/it]
  8%|▊         | 2948/37962 [6:52:15<78:35:35,  8.08s/it]
  8%|▊         | 2949/37962 [6:52:23<79:16:17,  8.15s/it]
  8%|▊         | 2950/37962 [6:52:32<79:52:18,  8.21s/it]
                                                         
{'loss': 1.4859, 'grad_norm': 4.17231559753418, 'learning_rate': 1.9880874656172094e-05, 'epoch': 0.08}

  8%|▊         | 2950/37962 [6:52:32<79:52:18,  8.21s/it]
  8%|▊         | 2951/37962 [6:52:40<79:55:29,  8.22s/it]
  8%|▊         | 2952/37962 [6:52:48<79:38:41,  8.19s/it]
  8%|▊         | 2953/37962 [6:52:56<79:53:31,  8.22s/it]
  8%|▊         | 2954/37962 [6:53:05<80:04:09,  8.23s/it]
  8%|▊         | 2955/37962 [6:53:13<80:08:06,  8.24s/it]
  8%|▊         | 2956/37962 [6:53:21<80:13:14,  8.25s/it]
  8%|▊         | 2957/37962 [6:53:30<81:31:35,  8.38s/it]
  8%|▊         | 2958/37962 [6:53:38<81:43:00,  8.40s/it]
  8%|▊         | 2959/37962 [6:53:47<81:35:52,  8.39s/it]
  8%|▊         | 2960/37962 [6:53:55<80:34:56,  8.29s/it]
                                                         
{'loss': 1.5555, 'grad_norm': 15.104559898376465, 'learning_rate': 1.9879558103185053e-05, 'epoch': 0.08}

  8%|▊         | 2960/37962 [6:53:55<80:34:56,  8.29s/it]
  8%|▊         | 2961/37962 [6:54:03<80:40:44,  8.30s/it]
  8%|▊         | 2962/37962 [6:54:11<80:02:03,  8.23s/it]
  8%|▊         | 2963/37962 [6:54:20<80:07:46,  8.24s/it]
  8%|▊         | 2964/37962 [6:54:28<79:57:25,  8.22s/it]
  8%|▊         | 2965/37962 [6:54:36<80:02:17,  8.23s/it]
  8%|▊         | 2966/37962 [6:54:44<79:33:37,  8.18s/it]
  8%|▊         | 2967/37962 [6:54:52<79:44:13,  8.20s/it]
  8%|▊         | 2968/37962 [6:55:01<80:32:07,  8.29s/it]
  8%|▊         | 2969/37962 [6:55:09<80:14:03,  8.25s/it]
  8%|▊         | 2970/37962 [6:55:17<80:18:21,  8.26s/it]
                                                         
{'loss': 1.5478, 'grad_norm': 8.142437934875488, 'learning_rate': 1.987823435903882e-05, 'epoch': 0.08}

  8%|▊         | 2970/37962 [6:55:17<80:18:21,  8.26s/it]
  8%|▊         | 2971/37962 [6:55:25<79:48:57,  8.21s/it]
  8%|▊         | 2972/37962 [6:55:33<79:05:20,  8.14s/it]
  8%|▊         | 2973/37962 [6:55:42<79:33:47,  8.19s/it]
  8%|▊         | 2974/37962 [6:55:50<79:30:32,  8.18s/it]
  8%|▊         | 2975/37962 [6:55:58<79:36:49,  8.19s/it]
  8%|▊         | 2976/37962 [6:56:06<79:55:24,  8.22s/it]
  8%|▊         | 2977/37962 [6:56:14<79:57:14,  8.23s/it]
  8%|▊         | 2978/37962 [6:56:23<79:58:44,  8.23s/it]
  8%|▊         | 2979/37962 [6:56:31<80:28:51,  8.28s/it]
  8%|▊         | 2980/37962 [6:56:39<80:33:53,  8.29s/it]
                                                         
{'loss': 1.5522, 'grad_norm': 13.499261856079102, 'learning_rate': 1.9876903424696923e-05, 'epoch': 0.08}

  8%|▊         | 2980/37962 [6:56:39<80:33:53,  8.29s/it]
  8%|▊         | 2981/37962 [6:56:48<80:11:26,  8.25s/it]
  8%|▊         | 2982/37962 [6:56:56<80:03:06,  8.24s/it]
  8%|▊         | 2983/37962 [6:57:04<80:23:28,  8.27s/it]
  8%|▊         | 2984/37962 [6:57:12<79:08:59,  8.15s/it]
  8%|▊         | 2985/37962 [6:57:20<80:03:20,  8.24s/it]
  8%|▊         | 2986/37962 [6:57:29<80:10:11,  8.25s/it]
  8%|▊         | 2987/37962 [6:57:37<79:47:42,  8.21s/it]
  8%|▊         | 2988/37962 [6:57:45<80:13:30,  8.26s/it]
  8%|▊         | 2989/37962 [6:57:54<80:28:23,  8.28s/it]
  8%|▊         | 2990/37962 [6:58:02<80:41:48,  8.31s/it]
                                                         
{'loss': 1.4805, 'grad_norm': 41.704341888427734, 'learning_rate': 1.9875565301128135e-05, 'epoch': 0.08}

  8%|▊         | 2990/37962 [6:58:02<80:41:48,  8.31s/it]
  8%|▊         | 2991/37962 [6:58:10<79:30:29,  8.18s/it]
  8%|▊         | 2992/37962 [6:58:18<79:48:22,  8.22s/it]
  8%|▊         | 2993/37962 [6:58:26<80:01:38,  8.24s/it]
  8%|▊         | 2994/37962 [6:58:34<79:34:23,  8.19s/it]
  8%|▊         | 2995/37962 [6:58:42<78:52:44,  8.12s/it]
  8%|▊         | 2996/37962 [6:58:51<78:54:46,  8.12s/it]
  8%|▊         | 2997/37962 [6:58:58<77:51:08,  8.02s/it]
  8%|▊         | 2998/37962 [6:59:07<78:57:34,  8.13s/it]
  8%|▊         | 2999/37962 [6:59:15<79:25:12,  8.18s/it]
  8%|▊         | 3000/37962 [6:59:23<80:10:07,  8.25s/it]
                                                         
{'loss': 1.4063, 'grad_norm': 15.227733612060547, 'learning_rate': 1.987421998930644e-05, 'epoch': 0.08}

  8%|▊         | 3000/37962 [6:59:23<80:10:07,  8.25s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

  8%|▊         | 3001/37962 [7:00:17<210:47:39, 21.71s/it]
  8%|▊         | 3002/37962 [7:00:25<172:15:00, 17.74s/it]
  8%|▊         | 3003/37962 [7:00:33<144:21:53, 14.87s/it]
  8%|▊         | 3004/37962 [7:00:42<125:16:14, 12.90s/it]
  8%|▊         | 3005/37962 [7:00:50<111:56:38, 11.53s/it]
  8%|▊         | 3006/37962 [7:00:58<102:38:36, 10.57s/it]
  8%|▊         | 3007/37962 [7:01:06<94:46:30,  9.76s/it] 
  8%|▊         | 3008/37962 [7:01:15<91:01:47,  9.38s/it]
  8%|▊         | 3009/37962 [7:01:23<87:33:24,  9.02s/it]
  8%|▊         | 3010/37962 [7:01:31<85:19:01,  8.79s/it]
                                                         
{'loss': 1.6202, 'grad_norm': 4.233680248260498, 'learning_rate': 1.987286749021108e-05, 'epoch': 0.08}

  8%|▊         | 3010/37962 [7:01:31<85:19:01,  8.79s/it]
  8%|▊         | 3011/37962 [7:01:39<84:06:07,  8.66s/it]
  8%|▊         | 3012/37962 [7:01:48<83:07:53,  8.56s/it]
  8%|▊         | 3013/37962 [7:01:56<82:10:36,  8.46s/it]
  8%|▊         | 3014/37962 [7:02:04<82:05:24,  8.46s/it]
  8%|▊         | 3015/37962 [7:02:13<81:49:45,  8.43s/it]
  8%|▊         | 3016/37962 [7:02:21<81:31:27,  8.40s/it]
  8%|▊         | 3017/37962 [7:02:29<80:47:36,  8.32s/it]
  8%|▊         | 3018/37962 [7:02:38<81:27:13,  8.39s/it]
  8%|▊         | 3019/37962 [7:02:46<80:47:40,  8.32s/it]
  8%|▊         | 3020/37962 [7:02:54<80:26:39,  8.29s/it]
                                                         
{'loss': 1.4142, 'grad_norm': 13.817293167114258, 'learning_rate': 1.9871507804826505e-05, 'epoch': 0.08}

  8%|▊         | 3020/37962 [7:02:54<80:26:39,  8.29s/it]
  8%|▊         | 3021/37962 [7:03:02<80:33:55,  8.30s/it]
  8%|▊         | 3022/37962 [7:03:10<79:43:06,  8.21s/it]
  8%|▊         | 3023/37962 [7:03:19<80:02:41,  8.25s/it]
  8%|▊         | 3024/37962 [7:03:27<80:38:11,  8.31s/it]
  8%|▊         | 3025/37962 [7:03:36<80:53:24,  8.34s/it]
  8%|▊         | 3026/37962 [7:03:44<81:05:42,  8.36s/it]
  8%|▊         | 3027/37962 [7:03:52<81:09:15,  8.36s/it]
  8%|▊         | 3028/37962 [7:04:00<80:23:35,  8.28s/it]
  8%|▊         | 3029/37962 [7:04:09<79:52:29,  8.23s/it]
  8%|▊         | 3030/37962 [7:04:17<79:48:24,  8.22s/it]
                                                         
{'loss': 1.3596, 'grad_norm': 27.799190521240234, 'learning_rate': 1.9870140934142413e-05, 'epoch': 0.08}

  8%|▊         | 3030/37962 [7:04:17<79:48:24,  8.22s/it]
  8%|▊         | 3031/37962 [7:04:25<79:51:31,  8.23s/it]
  8%|▊         | 3032/37962 [7:04:33<80:23:04,  8.28s/it]
  8%|▊         | 3033/37962 [7:04:41<79:38:16,  8.21s/it]
  8%|▊         | 3034/37962 [7:04:49<78:48:17,  8.12s/it]
  8%|▊         | 3035/37962 [7:04:58<79:05:53,  8.15s/it]
  8%|▊         | 3036/37962 [7:05:06<79:39:03,  8.21s/it]
  8%|▊         | 3037/37962 [7:05:14<79:37:49,  8.21s/it]
  8%|▊         | 3038/37962 [7:05:23<80:15:05,  8.27s/it]
  8%|▊         | 3039/37962 [7:05:31<80:26:25,  8.29s/it]
  8%|▊         | 3040/37962 [7:05:39<80:28:58,  8.30s/it]
                                                         
{'loss': 1.4682, 'grad_norm': 36.75443649291992, 'learning_rate': 1.9868766879153726e-05, 'epoch': 0.08}

  8%|▊         | 3040/37962 [7:05:39<80:28:58,  8.30s/it]
  8%|▊         | 3041/37962 [7:05:48<80:32:30,  8.30s/it]
  8%|▊         | 3042/37962 [7:05:56<81:21:32,  8.39s/it]
  8%|▊         | 3043/37962 [7:06:04<80:55:25,  8.34s/it]
  8%|▊         | 3044/37962 [7:06:13<80:49:59,  8.33s/it]
  8%|▊         | 3045/37962 [7:06:21<80:28:37,  8.30s/it]
  8%|▊         | 3046/37962 [7:06:29<80:03:39,  8.25s/it]
  8%|▊         | 3047/37962 [7:06:37<79:23:47,  8.19s/it]
  8%|▊         | 3048/37962 [7:06:45<79:18:33,  8.18s/it]
  8%|▊         | 3049/37962 [7:06:53<79:04:26,  8.15s/it]
  8%|▊         | 3050/37962 [7:07:02<79:40:27,  8.22s/it]
                                                         
{'loss': 1.5231, 'grad_norm': 22.045642852783203, 'learning_rate': 1.986738564086059e-05, 'epoch': 0.08}

  8%|▊         | 3050/37962 [7:07:02<79:40:27,  8.22s/it]
  8%|▊         | 3051/37962 [7:07:10<80:23:59,  8.29s/it]
  8%|▊         | 3052/37962 [7:07:18<79:07:57,  8.16s/it]
  8%|▊         | 3053/37962 [7:07:26<79:16:58,  8.18s/it]
  8%|▊         | 3054/37962 [7:07:34<79:09:24,  8.16s/it]
  8%|▊         | 3055/37962 [7:07:43<79:26:16,  8.19s/it]
  8%|▊         | 3056/37962 [7:07:51<79:39:20,  8.22s/it]
  8%|▊         | 3057/37962 [7:07:59<79:27:49,  8.20s/it]
  8%|▊         | 3058/37962 [7:08:07<79:39:22,  8.22s/it]
  8%|▊         | 3059/37962 [7:08:15<79:09:52,  8.17s/it]
  8%|▊         | 3060/37962 [7:08:24<79:15:43,  8.18s/it]
                                                         
{'loss': 1.5785, 'grad_norm': 17.84931755065918, 'learning_rate': 1.9865997220268385e-05, 'epoch': 0.08}

  8%|▊         | 3060/37962 [7:08:24<79:15:43,  8.18s/it]
  8%|▊         | 3061/37962 [7:08:32<79:31:14,  8.20s/it]
  8%|▊         | 3062/37962 [7:08:40<79:11:37,  8.17s/it]
  8%|▊         | 3063/37962 [7:08:48<78:50:49,  8.13s/it]
  8%|▊         | 3064/37962 [7:08:56<79:18:39,  8.18s/it]
  8%|▊         | 3065/37962 [7:09:05<79:30:10,  8.20s/it]
  8%|▊         | 3066/37962 [7:09:13<80:23:58,  8.29s/it]
  8%|▊         | 3067/37962 [7:09:21<79:38:24,  8.22s/it]
  8%|▊         | 3068/37962 [7:09:29<79:29:26,  8.20s/it]
  8%|▊         | 3069/37962 [7:09:38<79:51:25,  8.24s/it]
  8%|▊         | 3070/37962 [7:09:46<80:15:44,  8.28s/it]
                                                         
{'loss': 1.4401, 'grad_norm': 27.923755645751953, 'learning_rate': 1.986460161838772e-05, 'epoch': 0.08}

  8%|▊         | 3070/37962 [7:09:46<80:15:44,  8.28s/it]
  8%|▊         | 3071/37962 [7:09:54<80:41:29,  8.33s/it]
  8%|▊         | 3072/37962 [7:10:03<80:48:19,  8.34s/it]
  8%|▊         | 3073/37962 [7:10:11<80:59:41,  8.36s/it]
  8%|▊         | 3074/37962 [7:10:20<81:26:16,  8.40s/it]
  8%|▊         | 3075/37962 [7:10:28<81:35:20,  8.42s/it]
  8%|▊         | 3076/37962 [7:10:36<81:25:08,  8.40s/it]
  8%|▊         | 3077/37962 [7:10:45<80:44:05,  8.33s/it]
  8%|▊         | 3078/37962 [7:10:52<79:17:42,  8.18s/it]
  8%|▊         | 3079/37962 [7:11:01<79:15:20,  8.18s/it]
  8%|▊         | 3080/37962 [7:11:09<79:11:00,  8.17s/it]
                                                         
{'loss': 1.6572, 'grad_norm': 23.419239044189453, 'learning_rate': 1.9863198836234434e-05, 'epoch': 0.08}

  8%|▊         | 3080/37962 [7:11:09<79:11:00,  8.17s/it]
  8%|▊         | 3081/37962 [7:11:17<79:09:14,  8.17s/it]
  8%|▊         | 3082/37962 [7:11:25<79:50:20,  8.24s/it]
  8%|▊         | 3083/37962 [7:11:34<80:25:45,  8.30s/it]
  8%|▊         | 3084/37962 [7:11:42<80:45:52,  8.34s/it]
  8%|▊         | 3085/37962 [7:11:51<80:58:09,  8.36s/it]
  8%|▊         | 3086/37962 [7:11:59<80:41:04,  8.33s/it]
  8%|▊         | 3087/37962 [7:12:07<80:51:54,  8.35s/it]
  8%|▊         | 3088/37962 [7:12:16<80:32:01,  8.31s/it]
  8%|▊         | 3089/37962 [7:12:24<80:01:33,  8.26s/it]
  8%|▊         | 3090/37962 [7:12:32<78:55:53,  8.15s/it]
                                                         
{'loss': 1.4697, 'grad_norm': 1.9719781875610352, 'learning_rate': 1.9861788874829578e-05, 'epoch': 0.08}

  8%|▊         | 3090/37962 [7:12:32<78:55:53,  8.15s/it]
  8%|▊         | 3091/37962 [7:12:40<79:03:33,  8.16s/it]
  8%|▊         | 3092/37962 [7:12:48<79:53:58,  8.25s/it]
  8%|▊         | 3093/37962 [7:12:56<79:48:01,  8.24s/it]
  8%|▊         | 3094/37962 [7:13:05<80:06:24,  8.27s/it]
  8%|▊         | 3095/37962 [7:13:13<80:26:13,  8.31s/it]
  8%|▊         | 3096/37962 [7:13:21<80:17:58,  8.29s/it]
  8%|▊         | 3097/37962 [7:13:29<79:38:19,  8.22s/it]
  8%|▊         | 3098/37962 [7:13:38<79:51:28,  8.25s/it]
  8%|▊         | 3099/37962 [7:13:46<79:58:30,  8.26s/it]
  8%|▊         | 3100/37962 [7:13:54<79:31:55,  8.21s/it]
                                                         
{'loss': 1.7168, 'grad_norm': 9.623125076293945, 'learning_rate': 1.9860371735199446e-05, 'epoch': 0.08}

  8%|▊         | 3100/37962 [7:13:54<79:31:55,  8.21s/it]
  8%|▊         | 3101/37962 [7:14:02<79:52:13,  8.25s/it]
  8%|▊         | 3102/37962 [7:14:11<79:33:58,  8.22s/it]
  8%|▊         | 3103/37962 [7:14:19<79:04:26,  8.17s/it]
  8%|▊         | 3104/37962 [7:14:27<79:58:42,  8.26s/it]
  8%|▊         | 3105/37962 [7:14:35<79:34:53,  8.22s/it]
  8%|▊         | 3106/37962 [7:14:43<79:25:12,  8.20s/it]
  8%|▊         | 3107/37962 [7:14:52<79:35:30,  8.22s/it]
  8%|▊         | 3108/37962 [7:15:00<80:30:47,  8.32s/it]
  8%|▊         | 3109/37962 [7:15:09<80:33:12,  8.32s/it]
  8%|▊         | 3110/37962 [7:15:17<80:16:35,  8.29s/it]
                                                         
{'loss': 1.4969, 'grad_norm': 11.467750549316406, 'learning_rate': 1.985894741837555e-05, 'epoch': 0.08}

  8%|▊         | 3110/37962 [7:15:17<80:16:35,  8.29s/it]
  8%|▊         | 3111/37962 [7:15:25<79:47:21,  8.24s/it]
  8%|▊         | 3112/37962 [7:15:33<78:57:26,  8.16s/it]
  8%|▊         | 3113/37962 [7:15:41<79:15:12,  8.19s/it]
  8%|▊         | 3114/37962 [7:15:50<79:45:04,  8.24s/it]
  8%|▊         | 3115/37962 [7:15:58<79:49:00,  8.25s/it]
  8%|▊         | 3116/37962 [7:16:06<79:40:26,  8.23s/it]
  8%|▊         | 3117/37962 [7:16:14<80:02:07,  8.27s/it]
  8%|▊         | 3118/37962 [7:16:23<80:09:04,  8.28s/it]
  8%|▊         | 3119/37962 [7:16:31<80:46:47,  8.35s/it]
  8%|▊         | 3120/37962 [7:16:40<81:07:12,  8.38s/it]
                                                         
{'loss': 1.4742, 'grad_norm': 16.401763916015625, 'learning_rate': 1.9857515925394617e-05, 'epoch': 0.08}

  8%|▊         | 3120/37962 [7:16:40<81:07:12,  8.38s/it]
  8%|▊         | 3121/37962 [7:16:48<81:09:05,  8.39s/it]
  8%|▊         | 3122/37962 [7:16:56<80:39:00,  8.33s/it]
  8%|▊         | 3123/37962 [7:17:05<80:54:59,  8.36s/it]
  8%|▊         | 3124/37962 [7:17:13<81:18:59,  8.40s/it]
  8%|▊         | 3125/37962 [7:17:22<82:12:12,  8.49s/it]
  8%|▊         | 3126/37962 [7:17:30<81:38:05,  8.44s/it]
  8%|▊         | 3127/37962 [7:17:39<81:31:32,  8.43s/it]
  8%|▊         | 3128/37962 [7:17:47<81:07:19,  8.38s/it]
  8%|▊         | 3129/37962 [7:17:55<80:07:27,  8.28s/it]
  8%|▊         | 3130/37962 [7:18:03<80:12:00,  8.29s/it]
                                                         
{'loss': 1.5422, 'grad_norm': 28.620059967041016, 'learning_rate': 1.9856077257298616e-05, 'epoch': 0.08}

  8%|▊         | 3130/37962 [7:18:03<80:12:00,  8.29s/it]
  8%|▊         | 3131/37962 [7:18:11<79:05:14,  8.17s/it]
  8%|▊         | 3132/37962 [7:18:19<79:32:27,  8.22s/it]
  8%|▊         | 3133/37962 [7:18:28<80:23:18,  8.31s/it]
  8%|▊         | 3134/37962 [7:18:36<79:43:27,  8.24s/it]
  8%|▊         | 3135/37962 [7:18:44<79:58:38,  8.27s/it]
  8%|▊         | 3136/37962 [7:18:53<79:54:30,  8.26s/it]
  8%|▊         | 3137/37962 [7:19:01<79:43:28,  8.24s/it]
  8%|▊         | 3138/37962 [7:19:09<79:45:58,  8.25s/it]
  8%|▊         | 3139/37962 [7:19:17<79:51:09,  8.26s/it]
  8%|▊         | 3140/37962 [7:19:26<80:23:22,  8.31s/it]
                                                         
{'loss': 1.4457, 'grad_norm': 11.462540626525879, 'learning_rate': 1.9854631415134723e-05, 'epoch': 0.08}

  8%|▊         | 3140/37962 [7:19:26<80:23:22,  8.31s/it]
  8%|▊         | 3141/37962 [7:19:34<80:25:21,  8.31s/it]
  8%|▊         | 3142/37962 [7:19:42<80:28:36,  8.32s/it]
  8%|▊         | 3143/37962 [7:19:51<80:23:18,  8.31s/it]
  8%|▊         | 3144/37962 [7:19:59<80:19:55,  8.31s/it]
  8%|▊         | 3145/37962 [7:20:07<79:48:01,  8.25s/it]
  8%|▊         | 3146/37962 [7:20:15<79:59:29,  8.27s/it]
  8%|▊         | 3147/37962 [7:20:24<80:19:36,  8.31s/it]
  8%|▊         | 3148/37962 [7:20:32<80:08:30,  8.29s/it]
  8%|▊         | 3149/37962 [7:20:40<80:05:16,  8.28s/it]
  8%|▊         | 3150/37962 [7:20:49<80:00:15,  8.27s/it]
                                                         
{'loss': 1.5232, 'grad_norm': 18.240942001342773, 'learning_rate': 1.9853178399955342e-05, 'epoch': 0.08}

  8%|▊         | 3150/37962 [7:20:49<80:00:15,  8.27s/it]
  8%|▊         | 3151/37962 [7:20:57<79:54:11,  8.26s/it]
  8%|▊         | 3152/37962 [7:21:05<80:52:03,  8.36s/it]
  8%|▊         | 3153/37962 [7:21:14<80:48:12,  8.36s/it]
  8%|▊         | 3154/37962 [7:21:22<80:42:43,  8.35s/it]
  8%|▊         | 3155/37962 [7:21:30<80:52:56,  8.37s/it]
  8%|▊         | 3156/37962 [7:21:39<80:13:52,  8.30s/it]
  8%|▊         | 3157/37962 [7:21:47<79:31:12,  8.23s/it]
  8%|▊         | 3158/37962 [7:21:55<80:03:50,  8.28s/it]
  8%|▊         | 3159/37962 [7:22:03<80:09:33,  8.29s/it]
  8%|▊         | 3160/37962 [7:22:12<80:00:07,  8.28s/it]
                                                         
{'loss': 1.4572, 'grad_norm': 28.66497230529785, 'learning_rate': 1.98517182128181e-05, 'epoch': 0.08}

  8%|▊         | 3160/37962 [7:22:12<80:00:07,  8.28s/it]
  8%|▊         | 3161/37962 [7:22:20<79:33:01,  8.23s/it]
  8%|▊         | 3162/37962 [7:22:28<79:42:07,  8.25s/it]
  8%|▊         | 3163/37962 [7:22:37<80:41:16,  8.35s/it]
  8%|▊         | 3164/37962 [7:22:45<80:24:30,  8.32s/it]
  8%|▊         | 3165/37962 [7:22:53<80:15:06,  8.30s/it]
  8%|▊         | 3166/37962 [7:23:02<80:37:00,  8.34s/it]
  8%|▊         | 3167/37962 [7:23:10<80:17:53,  8.31s/it]
  8%|▊         | 3168/37962 [7:23:18<78:50:54,  8.16s/it]
  8%|▊         | 3169/37962 [7:23:26<79:26:02,  8.22s/it]
  8%|▊         | 3170/37962 [7:23:34<79:07:37,  8.19s/it]
                                                         
{'loss': 1.356, 'grad_norm': 4.384847640991211, 'learning_rate': 1.985025085478584e-05, 'epoch': 0.08}

  8%|▊         | 3170/37962 [7:23:34<79:07:37,  8.19s/it]
  8%|▊         | 3171/37962 [7:23:42<78:45:54,  8.15s/it]
  8%|▊         | 3172/37962 [7:23:51<79:20:59,  8.21s/it]
  8%|▊         | 3173/37962 [7:23:59<79:18:44,  8.21s/it]
  8%|▊         | 3174/37962 [7:24:07<79:37:56,  8.24s/it]
  8%|▊         | 3175/37962 [7:24:15<79:54:15,  8.27s/it]
  8%|▊         | 3176/37962 [7:24:24<80:10:38,  8.30s/it]
  8%|▊         | 3177/37962 [7:24:32<80:16:42,  8.31s/it]
  8%|▊         | 3178/37962 [7:24:41<80:53:49,  8.37s/it]
  8%|▊         | 3179/37962 [7:24:49<80:51:30,  8.37s/it]
  8%|▊         | 3180/37962 [7:24:57<80:25:40,  8.32s/it]
                                                         
{'loss': 1.3667, 'grad_norm': 20.130245208740234, 'learning_rate': 1.9848776326926623e-05, 'epoch': 0.08}

  8%|▊         | 3180/37962 [7:24:57<80:25:40,  8.32s/it]
  8%|▊         | 3181/37962 [7:25:05<80:05:40,  8.29s/it]
  8%|▊         | 3182/37962 [7:25:14<80:09:03,  8.30s/it]
  8%|▊         | 3183/37962 [7:25:22<80:01:28,  8.28s/it]
  8%|▊         | 3184/37962 [7:25:30<80:30:16,  8.33s/it]
  8%|▊         | 3185/37962 [7:25:39<81:03:59,  8.39s/it]
  8%|▊         | 3186/37962 [7:25:47<81:04:33,  8.39s/it]
  8%|▊         | 3187/37962 [7:25:55<80:06:55,  8.29s/it]
  8%|▊         | 3188/37962 [7:26:04<80:22:16,  8.32s/it]
  8%|▊         | 3189/37962 [7:26:12<80:18:21,  8.31s/it]
  8%|▊         | 3190/37962 [7:26:20<79:57:25,  8.28s/it]
                                                         
{'loss': 1.4746, 'grad_norm': 15.157307624816895, 'learning_rate': 1.984729463031374e-05, 'epoch': 0.08}

  8%|▊         | 3190/37962 [7:26:20<79:57:25,  8.28s/it]
  8%|▊         | 3191/37962 [7:26:29<80:05:21,  8.29s/it]
  8%|▊         | 3192/37962 [7:26:37<80:33:16,  8.34s/it]
  8%|▊         | 3193/37962 [7:26:45<79:34:41,  8.24s/it]
  8%|▊         | 3194/37962 [7:26:53<79:19:36,  8.21s/it]
  8%|▊         | 3195/37962 [7:27:01<79:22:06,  8.22s/it]
  8%|▊         | 3196/37962 [7:27:10<79:09:50,  8.20s/it]
  8%|▊         | 3197/37962 [7:27:18<79:16:23,  8.21s/it]
  8%|▊         | 3198/37962 [7:27:26<79:36:53,  8.24s/it]
  8%|▊         | 3199/37962 [7:27:35<80:05:44,  8.29s/it]
  8%|▊         | 3200/37962 [7:27:43<80:16:44,  8.31s/it]
                                                         
{'loss': 1.5988, 'grad_norm': 17.457744598388672, 'learning_rate': 1.9845805766025687e-05, 'epoch': 0.08}

  8%|▊         | 3200/37962 [7:27:43<80:16:44,  8.31s/it]
  8%|▊         | 3201/37962 [7:27:51<79:01:58,  8.18s/it]
  8%|▊         | 3202/37962 [7:27:59<78:54:44,  8.17s/it]
  8%|▊         | 3203/37962 [7:28:07<79:25:35,  8.23s/it]
  8%|▊         | 3204/37962 [7:28:15<79:00:18,  8.18s/it]
  8%|▊         | 3205/37962 [7:28:24<79:37:43,  8.25s/it]
  8%|▊         | 3206/37962 [7:28:32<79:24:09,  8.22s/it]
  8%|▊         | 3207/37962 [7:28:40<79:26:26,  8.23s/it]
  8%|▊         | 3208/37962 [7:28:48<78:17:32,  8.11s/it]
  8%|▊         | 3209/37962 [7:28:57<79:50:47,  8.27s/it]
  8%|▊         | 3210/37962 [7:29:05<79:53:02,  8.28s/it]
                                                         
{'loss': 1.4593, 'grad_norm': 28.916650772094727, 'learning_rate': 1.9844309735146184e-05, 'epoch': 0.08}

  8%|▊         | 3210/37962 [7:29:05<79:53:02,  8.28s/it]
  8%|▊         | 3211/37962 [7:29:13<80:22:52,  8.33s/it]
  8%|▊         | 3212/37962 [7:29:22<80:53:58,  8.38s/it]
  8%|▊         | 3213/37962 [7:29:30<81:10:02,  8.41s/it]
  8%|▊         | 3214/37962 [7:29:39<81:03:03,  8.40s/it]
  8%|▊         | 3215/37962 [7:29:47<80:43:09,  8.36s/it]
  8%|▊         | 3216/37962 [7:29:55<80:18:17,  8.32s/it]
  8%|▊         | 3217/37962 [7:30:04<80:32:20,  8.34s/it]
  8%|▊         | 3218/37962 [7:30:12<80:37:53,  8.35s/it]
  8%|▊         | 3219/37962 [7:30:20<80:36:33,  8.35s/it]
  8%|▊         | 3220/37962 [7:30:29<80:34:41,  8.35s/it]
                                                         
{'loss': 1.445, 'grad_norm': 6.115513801574707, 'learning_rate': 1.9842806538764165e-05, 'epoch': 0.08}

  8%|▊         | 3220/37962 [7:30:29<80:34:41,  8.35s/it]
  8%|▊         | 3221/37962 [7:30:37<80:22:50,  8.33s/it]
  8%|▊         | 3222/37962 [7:30:45<80:04:26,  8.30s/it]
  8%|▊         | 3223/37962 [7:30:53<79:55:04,  8.28s/it]
  8%|▊         | 3224/37962 [7:31:02<79:43:51,  8.26s/it]
  8%|▊         | 3225/37962 [7:31:10<80:06:29,  8.30s/it]
  8%|▊         | 3226/37962 [7:31:18<80:02:21,  8.30s/it]
  9%|▊         | 3227/37962 [7:31:26<79:22:10,  8.23s/it]
  9%|▊         | 3228/37962 [7:31:35<80:22:18,  8.33s/it]
  9%|▊         | 3229/37962 [7:31:43<80:27:26,  8.34s/it]
  9%|▊         | 3230/37962 [7:31:52<80:24:40,  8.33s/it]
                                                         
{'loss': 1.5044, 'grad_norm': 20.28491973876953, 'learning_rate': 1.984129617797378e-05, 'epoch': 0.09}

  9%|▊         | 3230/37962 [7:31:52<80:24:40,  8.33s/it]
  9%|▊         | 3231/37962 [7:32:00<79:48:10,  8.27s/it]
  9%|▊         | 3232/37962 [7:32:08<79:31:25,  8.24s/it]
  9%|▊         | 3233/37962 [7:32:16<80:18:12,  8.32s/it]
  9%|▊         | 3234/37962 [7:32:25<80:08:18,  8.31s/it]
  9%|▊         | 3235/37962 [7:32:33<81:03:12,  8.40s/it]
  9%|▊         | 3236/37962 [7:32:41<80:08:17,  8.31s/it]
  9%|▊         | 3237/37962 [7:32:49<78:52:37,  8.18s/it]
  9%|▊         | 3238/37962 [7:32:58<78:52:37,  8.18s/it]
  9%|▊         | 3239/37962 [7:33:06<80:22:52,  8.33s/it]
  9%|▊         | 3240/37962 [7:33:14<80:05:52,  8.30s/it]
                                                         
{'loss': 1.4102, 'grad_norm': 12.839987754821777, 'learning_rate': 1.98397786538744e-05, 'epoch': 0.09}

  9%|▊         | 3240/37962 [7:33:14<80:05:52,  8.30s/it]
  9%|▊         | 3241/37962 [7:33:22<79:04:43,  8.20s/it]
  9%|▊         | 3242/37962 [7:33:31<79:51:32,  8.28s/it]
  9%|▊         | 3243/37962 [7:33:39<79:51:35,  8.28s/it]
  9%|▊         | 3244/37962 [7:33:47<79:36:59,  8.26s/it]
  9%|▊         | 3245/37962 [7:33:56<79:31:40,  8.25s/it]
  9%|▊         | 3246/37962 [7:34:04<79:38:05,  8.26s/it]
  9%|▊         | 3247/37962 [7:34:12<79:46:00,  8.27s/it]
  9%|▊         | 3248/37962 [7:34:20<79:27:01,  8.24s/it]
  9%|▊         | 3249/37962 [7:34:29<79:21:15,  8.23s/it]
  9%|▊         | 3250/37962 [7:34:37<79:32:17,  8.25s/it]
                                                         
{'loss': 1.6508, 'grad_norm': 32.2998161315918, 'learning_rate': 1.98382539675706e-05, 'epoch': 0.09}

  9%|▊         | 3250/37962 [7:34:37<79:32:17,  8.25s/it]
  9%|▊         | 3251/37962 [7:34:45<80:02:43,  8.30s/it]
  9%|▊         | 3252/37962 [7:34:54<79:54:08,  8.29s/it]
  9%|▊         | 3253/37962 [7:35:02<79:49:48,  8.28s/it]
  9%|▊         | 3254/37962 [7:35:10<79:46:35,  8.27s/it]
  9%|▊         | 3255/37962 [7:35:18<79:07:50,  8.21s/it]
  9%|▊         | 3256/37962 [7:35:26<79:24:23,  8.24s/it]
  9%|▊         | 3257/37962 [7:35:35<79:32:29,  8.25s/it]
  9%|▊         | 3258/37962 [7:35:43<79:48:21,  8.28s/it]
  9%|▊         | 3259/37962 [7:35:51<80:09:10,  8.31s/it]
  9%|▊         | 3260/37962 [7:36:00<79:41:15,  8.27s/it]
                                                         
{'loss': 1.4087, 'grad_norm': 21.531455993652344, 'learning_rate': 1.9836722120172172e-05, 'epoch': 0.09}

  9%|▊         | 3260/37962 [7:36:00<79:41:15,  8.27s/it]
  9%|▊         | 3261/37962 [7:36:08<79:39:27,  8.26s/it]
  9%|▊         | 3262/37962 [7:36:16<79:58:07,  8.30s/it]
  9%|▊         | 3263/37962 [7:36:24<79:56:12,  8.29s/it]
  9%|▊         | 3264/37962 [7:36:33<80:15:29,  8.33s/it]
  9%|▊         | 3265/37962 [7:36:41<80:07:55,  8.31s/it]
  9%|▊         | 3266/37962 [7:36:49<79:50:25,  8.28s/it]
  9%|▊         | 3267/37962 [7:36:58<79:43:31,  8.27s/it]
  9%|▊         | 3268/37962 [7:37:06<79:43:14,  8.27s/it]
  9%|▊         | 3269/37962 [7:37:14<79:33:52,  8.26s/it]
  9%|▊         | 3270/37962 [7:37:22<79:20:16,  8.23s/it]
                                                         
{'loss': 1.5002, 'grad_norm': 8.552406311035156, 'learning_rate': 1.983518311279412e-05, 'epoch': 0.09}

  9%|▊         | 3270/37962 [7:37:22<79:20:16,  8.23s/it]
  9%|▊         | 3271/37962 [7:37:30<78:42:52,  8.17s/it]
  9%|▊         | 3272/37962 [7:37:38<78:36:01,  8.16s/it]
  9%|▊         | 3273/37962 [7:37:47<78:54:09,  8.19s/it]
  9%|▊         | 3274/37962 [7:37:55<78:00:35,  8.10s/it]
  9%|▊         | 3275/37962 [7:38:03<78:12:29,  8.12s/it]
  9%|▊         | 3276/37962 [7:38:11<79:01:24,  8.20s/it]
  9%|▊         | 3277/37962 [7:38:19<78:37:35,  8.16s/it]
  9%|▊         | 3278/37962 [7:38:27<78:52:01,  8.19s/it]
  9%|▊         | 3279/37962 [7:38:36<79:12:34,  8.22s/it]
  9%|▊         | 3280/37962 [7:38:44<79:21:22,  8.24s/it]
                                                         
{'loss': 1.5313, 'grad_norm': 13.527255058288574, 'learning_rate': 1.9833636946556663e-05, 'epoch': 0.09}

  9%|▊         | 3280/37962 [7:38:44<79:21:22,  8.24s/it]
  9%|▊         | 3281/37962 [7:38:52<78:21:09,  8.13s/it]
  9%|▊         | 3282/37962 [7:39:00<79:09:26,  8.22s/it]
  9%|▊         | 3283/37962 [7:39:08<78:40:39,  8.17s/it]
  9%|▊         | 3284/37962 [7:39:17<78:58:14,  8.20s/it]
  9%|▊         | 3285/37962 [7:39:25<79:26:35,  8.25s/it]
  9%|▊         | 3286/37962 [7:39:33<79:55:41,  8.30s/it]
  9%|▊         | 3287/37962 [7:39:42<79:43:01,  8.28s/it]
  9%|▊         | 3288/37962 [7:39:50<79:53:11,  8.29s/it]
  9%|▊         | 3289/37962 [7:39:58<79:41:42,  8.27s/it]
  9%|▊         | 3290/37962 [7:40:07<79:48:40,  8.29s/it]
                                                         
{'loss': 1.524, 'grad_norm': 34.39051818847656, 'learning_rate': 1.983208362258523e-05, 'epoch': 0.09}

  9%|▊         | 3290/37962 [7:40:07<79:48:40,  8.29s/it]
  9%|▊         | 3291/37962 [7:40:15<79:20:09,  8.24s/it]
  9%|▊         | 3292/37962 [7:40:23<78:37:57,  8.16s/it]
  9%|▊         | 3293/37962 [7:40:31<78:34:35,  8.16s/it]
  9%|▊         | 3294/37962 [7:40:39<78:55:37,  8.20s/it]
  9%|▊         | 3295/37962 [7:40:47<79:17:25,  8.23s/it]
  9%|▊         | 3296/37962 [7:40:56<79:42:43,  8.28s/it]
  9%|▊         | 3297/37962 [7:41:04<80:40:06,  8.38s/it]
  9%|▊         | 3298/37962 [7:41:12<79:05:13,  8.21s/it]
  9%|▊         | 3299/37962 [7:41:20<78:18:07,  8.13s/it]
  9%|▊         | 3300/37962 [7:41:28<78:03:20,  8.11s/it]
                                                         
{'loss': 1.6223, 'grad_norm': 16.269207000732422, 'learning_rate': 1.983052314201046e-05, 'epoch': 0.09}

  9%|▊         | 3300/37962 [7:41:28<78:03:20,  8.11s/it]
  9%|▊         | 3301/37962 [7:41:37<79:11:57,  8.23s/it]
  9%|▊         | 3302/37962 [7:41:45<79:32:31,  8.26s/it]
  9%|▊         | 3303/37962 [7:41:53<78:11:35,  8.12s/it]
  9%|▊         | 3304/37962 [7:42:01<77:58:02,  8.10s/it]
  9%|▊         | 3305/37962 [7:42:09<78:12:00,  8.12s/it]
  9%|▊         | 3306/37962 [7:42:17<78:48:40,  8.19s/it]
  9%|▊         | 3307/37962 [7:42:26<79:04:25,  8.21s/it]
  9%|▊         | 3308/37962 [7:42:34<79:51:42,  8.30s/it]
  9%|▊         | 3309/37962 [7:42:43<79:58:11,  8.31s/it]
  9%|▊         | 3310/37962 [7:42:51<79:08:09,  8.22s/it]
                                                         
{'loss': 1.4482, 'grad_norm': 31.936786651611328, 'learning_rate': 1.9828955505968195e-05, 'epoch': 0.09}

  9%|▊         | 3310/37962 [7:42:51<79:08:09,  8.22s/it]
  9%|▊         | 3311/37962 [7:42:59<78:28:42,  8.15s/it]
  9%|▊         | 3312/37962 [7:43:07<78:50:34,  8.19s/it]
  9%|▊         | 3313/37962 [7:43:15<78:13:46,  8.13s/it]
  9%|▊         | 3314/37962 [7:43:23<78:51:47,  8.19s/it]
  9%|▊         | 3315/37962 [7:43:32<79:28:53,  8.26s/it]
  9%|▊         | 3316/37962 [7:43:40<79:20:21,  8.24s/it]
  9%|▊         | 3317/37962 [7:43:48<78:52:39,  8.20s/it]
  9%|▊         | 3318/37962 [7:43:56<79:12:05,  8.23s/it]
  9%|▊         | 3319/37962 [7:44:04<79:22:53,  8.25s/it]
  9%|▊         | 3320/37962 [7:44:13<79:19:22,  8.24s/it]
                                                         
{'loss': 1.435, 'grad_norm': 61.02764129638672, 'learning_rate': 1.982738071559949e-05, 'epoch': 0.09}

  9%|▊         | 3320/37962 [7:44:13<79:19:22,  8.24s/it]
  9%|▊         | 3321/37962 [7:44:21<79:10:31,  8.23s/it]
  9%|▉         | 3322/37962 [7:44:29<79:07:46,  8.22s/it]
  9%|▉         | 3323/37962 [7:44:37<79:19:20,  8.24s/it]
  9%|▉         | 3324/37962 [7:44:46<79:02:36,  8.22s/it]
  9%|▉         | 3325/37962 [7:44:54<79:16:42,  8.24s/it]
  9%|▉         | 3326/37962 [7:45:02<79:19:37,  8.25s/it]
  9%|▉         | 3327/37962 [7:45:10<79:18:36,  8.24s/it]
  9%|▉         | 3328/37962 [7:45:19<79:03:46,  8.22s/it]
  9%|▉         | 3329/37962 [7:45:27<79:19:26,  8.25s/it]
  9%|▉         | 3330/37962 [7:45:35<79:46:23,  8.29s/it]
                                                         
{'loss': 1.5219, 'grad_norm': 16.22569465637207, 'learning_rate': 1.9825798772050616e-05, 'epoch': 0.09}

  9%|▉         | 3330/37962 [7:45:35<79:46:23,  8.29s/it]
  9%|▉         | 3331/37962 [7:45:44<80:02:25,  8.32s/it]
  9%|▉         | 3332/37962 [7:45:52<79:47:31,  8.29s/it]
  9%|▉         | 3333/37962 [7:46:00<79:52:39,  8.30s/it]
  9%|▉         | 3334/37962 [7:46:08<79:36:28,  8.28s/it]
  9%|▉         | 3335/37962 [7:46:17<79:33:22,  8.27s/it]
  9%|▉         | 3336/37962 [7:46:25<79:19:47,  8.25s/it]
  9%|▉         | 3337/37962 [7:46:33<79:58:12,  8.31s/it]
  9%|▉         | 3338/37962 [7:46:41<79:30:21,  8.27s/it]
  9%|▉         | 3339/37962 [7:46:50<79:08:12,  8.23s/it]
  9%|▉         | 3340/37962 [7:46:57<78:01:44,  8.11s/it]
                                                         
{'loss': 1.527, 'grad_norm': 12.43005084991455, 'learning_rate': 1.982420967647303e-05, 'epoch': 0.09}

  9%|▉         | 3340/37962 [7:46:57<78:01:44,  8.11s/it]
  9%|▉         | 3341/37962 [7:47:06<78:41:20,  8.18s/it]
  9%|▉         | 3342/37962 [7:47:14<78:49:24,  8.20s/it]
  9%|▉         | 3343/37962 [7:47:22<77:47:36,  8.09s/it]
  9%|▉         | 3344/37962 [7:47:30<77:01:13,  8.01s/it]
  9%|▉         | 3345/37962 [7:47:45<97:57:51, 10.19s/it]
  9%|▉         | 3346/37962 [7:47:53<92:23:23,  9.61s/it]
  9%|▉         | 3347/37962 [7:48:02<88:48:15,  9.24s/it]
  9%|▉         | 3348/37962 [7:48:09<84:26:44,  8.78s/it]
  9%|▉         | 3349/37962 [7:48:18<83:17:48,  8.66s/it]
  9%|▉         | 3350/37962 [7:48:26<81:49:02,  8.51s/it]
                                                         
{'loss': 1.5947, 'grad_norm': 4.167381763458252, 'learning_rate': 1.9822613430023416e-05, 'epoch': 0.09}

  9%|▉         | 3350/37962 [7:48:26<81:49:02,  8.51s/it]
  9%|▉         | 3351/37962 [7:48:34<81:42:42,  8.50s/it]
  9%|▉         | 3352/37962 [7:48:43<80:54:10,  8.42s/it]
  9%|▉         | 3353/37962 [7:48:51<80:14:43,  8.35s/it]
  9%|▉         | 3354/37962 [7:48:59<79:43:23,  8.29s/it]
  9%|▉         | 3355/37962 [7:49:07<79:34:37,  8.28s/it]
  9%|▉         | 3356/37962 [7:49:16<80:03:50,  8.33s/it]
  9%|▉         | 3357/37962 [7:49:24<80:38:19,  8.39s/it]
  9%|▉         | 3358/37962 [7:49:33<80:49:08,  8.41s/it]
  9%|▉         | 3359/37962 [7:49:41<80:46:37,  8.40s/it]
  9%|▉         | 3360/37962 [7:49:49<79:18:35,  8.25s/it]
                                                         
{'loss': 1.61, 'grad_norm': 8.551668167114258, 'learning_rate': 1.9821010033863648e-05, 'epoch': 0.09}

  9%|▉         | 3360/37962 [7:49:49<79:18:35,  8.25s/it]
  9%|▉         | 3361/37962 [7:49:57<79:28:10,  8.27s/it]
  9%|▉         | 3362/37962 [7:50:06<80:16:31,  8.35s/it]
  9%|▉         | 3363/37962 [7:50:14<80:12:54,  8.35s/it]
  9%|▉         | 3364/37962 [7:50:22<80:10:33,  8.34s/it]
  9%|▉         | 3365/37962 [7:50:31<79:50:29,  8.31s/it]
  9%|▉         | 3366/37962 [7:50:39<80:01:37,  8.33s/it]
  9%|▉         | 3367/37962 [7:50:47<79:12:07,  8.24s/it]
  9%|▉         | 3368/37962 [7:50:55<79:10:26,  8.24s/it]
  9%|▉         | 3369/37962 [7:51:04<79:37:37,  8.29s/it]
  9%|▉         | 3370/37962 [7:51:12<78:32:20,  8.17s/it]
                                                         
{'loss': 1.5044, 'grad_norm': 28.23368263244629, 'learning_rate': 1.9819399489160818e-05, 'epoch': 0.09}

  9%|▉         | 3370/37962 [7:51:12<78:32:20,  8.17s/it]
  9%|▉         | 3371/37962 [7:51:20<78:59:40,  8.22s/it]
  9%|▉         | 3372/37962 [7:51:28<79:04:54,  8.23s/it]
  9%|▉         | 3373/37962 [7:51:36<78:47:53,  8.20s/it]
  9%|▉         | 3374/37962 [7:51:45<78:54:29,  8.21s/it]
  9%|▉         | 3375/37962 [7:51:53<79:02:32,  8.23s/it]
  9%|▉         | 3376/37962 [7:52:01<79:15:10,  8.25s/it]
  9%|▉         | 3377/37962 [7:52:09<79:34:31,  8.28s/it]
  9%|▉         | 3378/37962 [7:52:18<80:14:26,  8.35s/it]
  9%|▉         | 3379/37962 [7:52:26<79:44:31,  8.30s/it]
  9%|▉         | 3380/37962 [7:52:34<79:30:00,  8.28s/it]
                                                         
{'loss': 1.3536, 'grad_norm': 33.19887161254883, 'learning_rate': 1.981778179708721e-05, 'epoch': 0.09}

  9%|▉         | 3380/37962 [7:52:34<79:30:00,  8.28s/it]
  9%|▉         | 3381/37962 [7:52:42<78:51:56,  8.21s/it]
  9%|▉         | 3382/37962 [7:52:50<78:20:19,  8.16s/it]
  9%|▉         | 3383/37962 [7:52:58<78:03:56,  8.13s/it]
  9%|▉         | 3384/37962 [7:53:07<78:50:03,  8.21s/it]
  9%|▉         | 3385/37962 [7:53:15<78:29:19,  8.17s/it]
  9%|▉         | 3386/37962 [7:53:23<78:03:04,  8.13s/it]
  9%|▉         | 3387/37962 [7:53:31<78:27:53,  8.17s/it]
  9%|▉         | 3388/37962 [7:53:40<79:32:02,  8.28s/it]
  9%|▉         | 3389/37962 [7:53:48<79:41:00,  8.30s/it]
  9%|▉         | 3390/37962 [7:53:56<79:32:17,  8.28s/it]
                                                         
{'loss': 1.4093, 'grad_norm': 8.066132545471191, 'learning_rate': 1.9816156958820308e-05, 'epoch': 0.09}

  9%|▉         | 3390/37962 [7:53:56<79:32:17,  8.28s/it]
  9%|▉         | 3391/37962 [7:54:05<79:19:08,  8.26s/it]
  9%|▉         | 3392/37962 [7:54:13<78:35:38,  8.18s/it]
  9%|▉         | 3393/37962 [7:54:20<77:44:12,  8.10s/it]
  9%|▉         | 3394/37962 [7:54:29<78:10:37,  8.14s/it]
  9%|▉         | 3395/37962 [7:54:37<78:26:00,  8.17s/it]
  9%|▉         | 3396/37962 [7:54:45<78:10:11,  8.14s/it]
  9%|▉         | 3397/37962 [7:54:53<78:13:06,  8.15s/it]
  9%|▉         | 3398/37962 [7:55:02<78:58:46,  8.23s/it]
  9%|▉         | 3399/37962 [7:55:10<78:55:36,  8.22s/it]
  9%|▉         | 3400/37962 [7:55:18<79:17:08,  8.26s/it]
                                                         
{'loss': 1.4888, 'grad_norm': 11.486530303955078, 'learning_rate': 1.981452497554281e-05, 'epoch': 0.09}

  9%|▉         | 3400/37962 [7:55:18<79:17:08,  8.26s/it]
  9%|▉         | 3401/37962 [7:55:26<79:11:47,  8.25s/it]
  9%|▉         | 3402/37962 [7:55:35<79:45:33,  8.31s/it]
  9%|▉         | 3403/37962 [7:55:43<79:44:51,  8.31s/it]
  9%|▉         | 3404/37962 [7:55:52<80:05:42,  8.34s/it]
  9%|▉         | 3405/37962 [7:56:00<79:54:13,  8.32s/it]
  9%|▉         | 3406/37962 [7:56:08<79:57:03,  8.33s/it]
  9%|▉         | 3407/37962 [7:56:16<79:38:07,  8.30s/it]
  9%|▉         | 3408/37962 [7:56:25<79:38:05,  8.30s/it]
  9%|▉         | 3409/37962 [7:56:33<79:56:49,  8.33s/it]
  9%|▉         | 3410/37962 [7:56:41<78:18:09,  8.16s/it]
                                                         
{'loss': 1.4612, 'grad_norm': 26.213485717773438, 'learning_rate': 1.981288584844261e-05, 'epoch': 0.09}

  9%|▉         | 3410/37962 [7:56:41<78:18:09,  8.16s/it]
  9%|▉         | 3411/37962 [7:56:49<78:38:53,  8.19s/it]
  9%|▉         | 3412/37962 [7:56:57<78:33:16,  8.19s/it]
  9%|▉         | 3413/37962 [7:57:06<79:28:19,  8.28s/it]
  9%|▉         | 3414/37962 [7:57:14<79:50:22,  8.32s/it]
  9%|▉         | 3415/37962 [7:57:22<78:48:17,  8.21s/it]
  9%|▉         | 3416/37962 [7:57:31<79:08:05,  8.25s/it]
  9%|▉         | 3417/37962 [7:57:39<79:34:13,  8.29s/it]
  9%|▉         | 3418/37962 [7:57:47<79:12:34,  8.25s/it]
  9%|▉         | 3419/37962 [7:57:55<79:16:10,  8.26s/it]
  9%|▉         | 3420/37962 [7:58:04<79:50:13,  8.32s/it]
                                                         
{'loss': 1.4407, 'grad_norm': 7.543020248413086, 'learning_rate': 1.98112395787128e-05, 'epoch': 0.09}

  9%|▉         | 3420/37962 [7:58:04<79:50:13,  8.32s/it]
  9%|▉         | 3421/37962 [7:58:12<78:45:00,  8.21s/it]
  9%|▉         | 3422/37962 [7:58:20<78:31:29,  8.18s/it]
  9%|▉         | 3423/37962 [7:58:28<78:49:14,  8.22s/it]
  9%|▉         | 3424/37962 [7:58:37<79:05:23,  8.24s/it]
  9%|▉         | 3425/37962 [7:58:44<77:47:21,  8.11s/it]
  9%|▉         | 3426/37962 [7:58:52<77:45:01,  8.10s/it]
  9%|▉         | 3427/37962 [7:59:01<78:16:40,  8.16s/it]
  9%|▉         | 3428/37962 [7:59:09<78:55:00,  8.23s/it]
  9%|▉         | 3429/37962 [7:59:17<79:10:04,  8.25s/it]
  9%|▉         | 3430/37962 [7:59:26<79:23:38,  8.28s/it]
                                                         
{'loss': 1.5133, 'grad_norm': 15.23840045928955, 'learning_rate': 1.9809586167551664e-05, 'epoch': 0.09}

  9%|▉         | 3430/37962 [7:59:26<79:23:38,  8.28s/it]
  9%|▉         | 3431/37962 [7:59:34<79:30:57,  8.29s/it]
  9%|▉         | 3432/37962 [7:59:42<78:07:42,  8.15s/it]
  9%|▉         | 3433/37962 [7:59:50<78:20:24,  8.17s/it]
  9%|▉         | 3434/37962 [7:59:58<78:22:32,  8.17s/it]
  9%|▉         | 3435/37962 [8:00:06<78:32:13,  8.19s/it]
  9%|▉         | 3436/37962 [8:00:15<79:24:35,  8.28s/it]
  9%|▉         | 3437/37962 [8:00:29<96:41:09, 10.08s/it]
  9%|▉         | 3438/37962 [8:00:38<91:48:55,  9.57s/it]
  9%|▉         | 3439/37962 [8:00:46<88:03:40,  9.18s/it]
  9%|▉         | 3440/37962 [8:00:54<85:06:29,  8.88s/it]
                                                         
{'loss': 1.55, 'grad_norm': 8.588821411132812, 'learning_rate': 1.98079256161627e-05, 'epoch': 0.09}

  9%|▉         | 3440/37962 [8:00:54<85:06:29,  8.88s/it]
  9%|▉         | 3441/37962 [8:01:02<82:46:43,  8.63s/it]
  9%|▉         | 3442/37962 [8:01:18<102:10:58, 10.66s/it]
  9%|▉         | 3443/37962 [8:01:26<94:43:23,  9.88s/it] 
  9%|▉         | 3444/37962 [8:01:34<89:59:57,  9.39s/it]
  9%|▉         | 3445/37962 [8:01:42<86:20:21,  9.00s/it]
  9%|▉         | 3446/37962 [8:01:50<84:39:41,  8.83s/it]
  9%|▉         | 3447/37962 [8:01:59<83:01:56,  8.66s/it]
  9%|▉         | 3448/37962 [8:02:07<82:00:39,  8.55s/it]
  9%|▉         | 3449/37962 [8:02:15<80:57:19,  8.44s/it]
  9%|▉         | 3450/37962 [8:02:23<80:14:24,  8.37s/it]
                                                         
{'loss': 1.483, 'grad_norm': 27.411909103393555, 'learning_rate': 1.9806257925754586e-05, 'epoch': 0.09}

  9%|▉         | 3450/37962 [8:02:23<80:14:24,  8.37s/it]
  9%|▉         | 3451/37962 [8:02:32<79:50:50,  8.33s/it]
  9%|▉         | 3452/37962 [8:02:40<80:27:11,  8.39s/it]
  9%|▉         | 3453/37962 [8:02:48<80:07:18,  8.36s/it]
  9%|▉         | 3454/37962 [8:02:56<79:08:35,  8.26s/it]
  9%|▉         | 3455/37962 [8:03:05<79:04:35,  8.25s/it]
  9%|▉         | 3456/37962 [8:03:13<79:46:32,  8.32s/it]
  9%|▉         | 3457/37962 [8:03:21<78:50:15,  8.23s/it]
  9%|▉         | 3458/37962 [8:03:29<78:53:22,  8.23s/it]
  9%|▉         | 3459/37962 [8:03:37<78:16:15,  8.17s/it]
  9%|▉         | 3460/37962 [8:03:45<78:03:15,  8.14s/it]
                                                         
{'loss': 1.4193, 'grad_norm': 10.379570007324219, 'learning_rate': 1.980458309754121e-05, 'epoch': 0.09}

  9%|▉         | 3460/37962 [8:03:45<78:03:15,  8.14s/it]
  9%|▉         | 3461/37962 [8:03:54<78:05:53,  8.15s/it]
  9%|▉         | 3462/37962 [8:04:02<78:40:00,  8.21s/it]
  9%|▉         | 3463/37962 [8:04:10<77:46:02,  8.12s/it]
  9%|▉         | 3464/37962 [8:04:18<78:48:04,  8.22s/it]
  9%|▉         | 3465/37962 [8:04:26<77:50:34,  8.12s/it]
  9%|▉         | 3466/37962 [8:04:35<78:40:56,  8.21s/it]
  9%|▉         | 3467/37962 [8:04:43<78:37:07,  8.20s/it]
  9%|▉         | 3468/37962 [8:04:51<78:37:53,  8.21s/it]
  9%|▉         | 3469/37962 [8:04:59<78:59:22,  8.24s/it]
  9%|▉         | 3470/37962 [8:05:08<79:51:52,  8.34s/it]
                                                         
{'loss': 1.415, 'grad_norm': 16.625946044921875, 'learning_rate': 1.9802901132741653e-05, 'epoch': 0.09}

  9%|▉         | 3470/37962 [8:05:08<79:51:52,  8.34s/it]
  9%|▉         | 3471/37962 [8:05:16<79:15:33,  8.27s/it]
  9%|▉         | 3472/37962 [8:05:24<79:05:51,  8.26s/it]
  9%|▉         | 3473/37962 [8:05:33<79:09:19,  8.26s/it]
  9%|▉         | 3474/37962 [8:05:41<79:19:14,  8.28s/it]
  9%|▉         | 3475/37962 [8:05:49<79:30:00,  8.30s/it]
  9%|▉         | 3476/37962 [8:05:57<79:18:39,  8.28s/it]
  9%|▉         | 3477/37962 [8:06:06<79:35:03,  8.31s/it]
  9%|▉         | 3478/37962 [8:06:14<79:23:47,  8.29s/it]
  9%|▉         | 3479/37962 [8:06:22<79:37:42,  8.31s/it]
  9%|▉         | 3480/37962 [8:06:31<80:13:28,  8.38s/it]
                                                         
{'loss': 1.5911, 'grad_norm': 16.713918685913086, 'learning_rate': 1.9801212032580186e-05, 'epoch': 0.09}

  9%|▉         | 3480/37962 [8:06:31<80:13:28,  8.38s/it]
  9%|▉         | 3481/37962 [8:06:39<80:15:21,  8.38s/it]
  9%|▉         | 3482/37962 [8:06:48<80:14:16,  8.38s/it]
  9%|▉         | 3483/37962 [8:06:56<79:34:05,  8.31s/it]
  9%|▉         | 3484/37962 [8:07:04<80:03:00,  8.36s/it]
  9%|▉         | 3485/37962 [8:07:13<80:00:12,  8.35s/it]
  9%|▉         | 3486/37962 [8:07:28<99:23:52, 10.38s/it]
  9%|▉         | 3487/37962 [8:07:36<93:17:49,  9.74s/it]
  9%|▉         | 3488/37962 [8:07:44<88:47:22,  9.27s/it]
  9%|▉         | 3489/37962 [8:08:00<107:01:57, 11.18s/it]
  9%|▉         | 3490/37962 [8:08:08<98:14:57, 10.26s/it] 
                                                         
{'loss': 1.3815, 'grad_norm': 1.4548166990280151, 'learning_rate': 1.9799515798286272e-05, 'epoch': 0.09}

  9%|▉         | 3490/37962 [8:08:08<98:14:57, 10.26s/it]
  9%|▉         | 3491/37962 [8:08:16<92:04:06,  9.62s/it]
  9%|▉         | 3492/37962 [8:08:25<89:06:07,  9.31s/it]
  9%|▉         | 3493/37962 [8:08:33<86:26:58,  9.03s/it]
  9%|▉         | 3494/37962 [8:08:41<83:23:25,  8.71s/it]
  9%|▉         | 3495/37962 [8:08:49<82:42:26,  8.64s/it]
  9%|▉         | 3496/37962 [8:08:58<82:43:13,  8.64s/it]
  9%|▉         | 3497/37962 [8:09:07<82:12:37,  8.59s/it]
  9%|▉         | 3498/37962 [8:09:15<81:18:43,  8.49s/it]
  9%|▉         | 3499/37962 [8:09:23<79:56:44,  8.35s/it]
  9%|▉         | 3500/37962 [8:09:31<79:48:35,  8.34s/it]
                                                         
{'loss': 1.5424, 'grad_norm': 5.410892486572266, 'learning_rate': 1.9797812431094574e-05, 'epoch': 0.09}

  9%|▉         | 3500/37962 [8:09:31<79:48:35,  8.34s/it]
  9%|▉         | 3501/37962 [8:09:39<79:41:36,  8.33s/it]
  9%|▉         | 3502/37962 [8:09:48<79:27:32,  8.30s/it]
  9%|▉         | 3503/37962 [8:09:56<79:14:44,  8.28s/it]
  9%|▉         | 3504/37962 [8:10:04<79:18:32,  8.29s/it]
  9%|▉         | 3505/37962 [8:10:13<79:19:04,  8.29s/it]
  9%|▉         | 3506/37962 [8:10:21<79:15:04,  8.28s/it]
  9%|▉         | 3507/37962 [8:10:29<79:12:19,  8.28s/it]
  9%|▉         | 3508/37962 [8:10:37<79:03:32,  8.26s/it]
  9%|▉         | 3509/37962 [8:10:46<79:46:17,  8.34s/it]
  9%|▉         | 3510/37962 [8:10:54<79:39:27,  8.32s/it]
                                                         
{'loss': 1.43, 'grad_norm': 13.775588035583496, 'learning_rate': 1.9796101932244944e-05, 'epoch': 0.09}

  9%|▉         | 3510/37962 [8:10:54<79:39:27,  8.32s/it]
  9%|▉         | 3511/37962 [8:11:03<79:59:27,  8.36s/it]
  9%|▉         | 3512/37962 [8:11:11<80:03:42,  8.37s/it]
  9%|▉         | 3513/37962 [8:11:19<80:05:33,  8.37s/it]
  9%|▉         | 3514/37962 [8:11:28<79:39:45,  8.33s/it]
  9%|▉         | 3515/37962 [8:11:36<79:44:24,  8.33s/it]
  9%|▉         | 3516/37962 [8:11:44<79:22:24,  8.30s/it]
  9%|▉         | 3517/37962 [8:11:52<78:30:06,  8.20s/it]
  9%|▉         | 3518/37962 [8:12:00<78:03:32,  8.16s/it]
  9%|▉         | 3519/37962 [8:12:08<78:12:28,  8.17s/it]
  9%|▉         | 3520/37962 [8:12:17<78:43:35,  8.23s/it]
                                                         
{'loss': 1.4723, 'grad_norm': 6.4887614250183105, 'learning_rate': 1.9794384302982422e-05, 'epoch': 0.09}

  9%|▉         | 3520/37962 [8:12:17<78:43:35,  8.23s/it]
  9%|▉         | 3521/37962 [8:12:25<79:20:41,  8.29s/it]
  9%|▉         | 3522/37962 [8:12:33<79:18:00,  8.29s/it]
  9%|▉         | 3523/37962 [8:12:42<79:11:10,  8.28s/it]
  9%|▉         | 3524/37962 [8:12:50<79:07:22,  8.27s/it]
  9%|▉         | 3525/37962 [8:12:58<78:58:02,  8.26s/it]
  9%|▉         | 3526/37962 [8:13:06<78:48:02,  8.24s/it]
  9%|▉         | 3527/37962 [8:13:15<79:04:19,  8.27s/it]
  9%|▉         | 3528/37962 [8:13:23<79:26:58,  8.31s/it]
  9%|▉         | 3529/37962 [8:13:31<79:05:18,  8.27s/it]
  9%|▉         | 3530/37962 [8:13:46<98:04:28, 10.25s/it]
                                                         
{'loss': 1.4584, 'grad_norm': 3.845939874649048, 'learning_rate': 1.9792659544557244e-05, 'epoch': 0.09}

  9%|▉         | 3530/37962 [8:13:46<98:04:28, 10.25s/it]
  9%|▉         | 3531/37962 [8:13:54<92:06:45,  9.63s/it]
  9%|▉         | 3532/37962 [8:14:02<87:27:02,  9.14s/it]
  9%|▉         | 3533/37962 [8:14:11<85:18:38,  8.92s/it]
  9%|▉         | 3534/37962 [8:14:19<83:39:39,  8.75s/it]
  9%|▉         | 3535/37962 [8:14:27<82:11:11,  8.59s/it]
  9%|▉         | 3536/37962 [8:14:35<81:03:19,  8.48s/it]
  9%|▉         | 3537/37962 [8:14:44<80:24:27,  8.41s/it]
  9%|▉         | 3538/37962 [8:14:52<80:18:06,  8.40s/it]
  9%|▉         | 3539/37962 [8:15:00<80:04:39,  8.37s/it]
  9%|▉         | 3540/37962 [8:15:09<79:39:30,  8.33s/it]
                                                         
{'loss': 1.5837, 'grad_norm': 15.996095657348633, 'learning_rate': 1.9790927658224825e-05, 'epoch': 0.09}

  9%|▉         | 3540/37962 [8:15:09<79:39:30,  8.33s/it]
  9%|▉         | 3541/37962 [8:15:17<79:18:34,  8.29s/it]
  9%|▉         | 3542/37962 [8:15:25<79:40:10,  8.33s/it]
  9%|▉         | 3543/37962 [8:15:34<79:32:06,  8.32s/it]
  9%|▉         | 3544/37962 [8:15:41<78:16:28,  8.19s/it]
  9%|▉         | 3545/37962 [8:15:50<78:18:34,  8.19s/it]
  9%|▉         | 3546/37962 [8:15:58<78:38:11,  8.23s/it]
  9%|▉         | 3547/37962 [8:16:06<78:30:25,  8.21s/it]
  9%|▉         | 3548/37962 [8:16:14<78:28:14,  8.21s/it]
  9%|▉         | 3549/37962 [8:16:23<78:45:32,  8.24s/it]
  9%|▉         | 3550/37962 [8:16:31<79:40:24,  8.34s/it]
                                                         
{'loss': 1.4761, 'grad_norm': 12.655293464660645, 'learning_rate': 1.9789188645245786e-05, 'epoch': 0.09}

  9%|▉         | 3550/37962 [8:16:31<79:40:24,  8.34s/it]
  9%|▉         | 3551/37962 [8:16:39<79:14:50,  8.29s/it]
  9%|▉         | 3552/37962 [8:16:48<79:36:25,  8.33s/it]
  9%|▉         | 3553/37962 [8:16:56<80:00:57,  8.37s/it]
  9%|▉         | 3554/37962 [8:17:05<80:06:29,  8.38s/it]
  9%|▉         | 3555/37962 [8:17:13<79:59:09,  8.37s/it]
  9%|▉         | 3556/37962 [8:17:21<79:40:39,  8.34s/it]
  9%|▉         | 3557/37962 [8:17:30<79:24:05,  8.31s/it]
  9%|▉         | 3558/37962 [8:17:45<99:20:38, 10.40s/it]
  9%|▉         | 3559/37962 [8:17:53<93:34:33,  9.79s/it]
  9%|▉         | 3560/37962 [8:18:02<89:23:33,  9.35s/it]
                                                         
{'loss': 1.6177, 'grad_norm': 10.486674308776855, 'learning_rate': 1.9787442506885913e-05, 'epoch': 0.09}

  9%|▉         | 3560/37962 [8:18:02<89:23:33,  9.35s/it]
  9%|▉         | 3561/37962 [8:18:10<86:19:32,  9.03s/it]
  9%|▉         | 3562/37962 [8:18:18<84:10:02,  8.81s/it]
  9%|▉         | 3563/37962 [8:18:26<82:35:23,  8.64s/it]
  9%|▉         | 3564/37962 [8:18:35<81:28:09,  8.53s/it]
  9%|▉         | 3565/37962 [8:18:43<80:32:52,  8.43s/it]
  9%|▉         | 3566/37962 [8:18:51<80:19:31,  8.41s/it]
  9%|▉         | 3567/37962 [8:19:00<80:06:55,  8.39s/it]
  9%|▉         | 3568/37962 [8:19:08<79:35:44,  8.33s/it]
  9%|▉         | 3569/37962 [8:19:16<79:18:50,  8.30s/it]
  9%|▉         | 3570/37962 [8:19:24<78:55:40,  8.26s/it]
                                                         
{'loss': 1.5198, 'grad_norm': 10.727705001831055, 'learning_rate': 1.9785689244416198e-05, 'epoch': 0.09}

  9%|▉         | 3570/37962 [8:19:24<78:55:40,  8.26s/it]
  9%|▉         | 3571/37962 [8:19:32<78:38:35,  8.23s/it]
  9%|▉         | 3572/37962 [8:19:41<79:09:40,  8.29s/it]
  9%|▉         | 3573/37962 [8:19:49<79:47:42,  8.35s/it]
  9%|▉         | 3574/37962 [8:19:58<80:01:15,  8.38s/it]
  9%|▉         | 3575/37962 [8:20:06<79:44:05,  8.35s/it]
  9%|▉         | 3576/37962 [8:20:14<79:26:25,  8.32s/it]
  9%|▉         | 3577/37962 [8:20:22<79:05:46,  8.28s/it]
  9%|▉         | 3578/37962 [8:20:31<79:01:41,  8.27s/it]
  9%|▉         | 3579/37962 [8:20:39<79:28:41,  8.32s/it]
  9%|▉         | 3580/37962 [8:20:47<79:07:04,  8.28s/it]
                                                         
{'loss': 1.4462, 'grad_norm': 26.463388442993164, 'learning_rate': 1.9783928859112808e-05, 'epoch': 0.09}

  9%|▉         | 3580/37962 [8:20:47<79:07:04,  8.28s/it]
  9%|▉         | 3581/37962 [8:20:55<78:58:44,  8.27s/it]
  9%|▉         | 3582/37962 [8:21:04<79:17:57,  8.30s/it]
  9%|▉         | 3583/37962 [8:21:12<79:37:51,  8.34s/it]
  9%|▉         | 3584/37962 [8:21:20<78:47:21,  8.25s/it]
  9%|▉         | 3585/37962 [8:21:29<78:56:57,  8.27s/it]
  9%|▉         | 3586/37962 [8:21:37<79:13:05,  8.30s/it]
  9%|▉         | 3587/37962 [8:21:45<79:10:22,  8.29s/it]
  9%|▉         | 3588/37962 [8:22:01<100:51:42, 10.56s/it]
  9%|▉         | 3589/37962 [8:22:10<94:43:58,  9.92s/it] 
  9%|▉         | 3590/37962 [8:22:18<89:52:17,  9.41s/it]
                                                         
{'loss': 1.4796, 'grad_norm': 15.121853828430176, 'learning_rate': 1.978216135225709e-05, 'epoch': 0.09}

  9%|▉         | 3590/37962 [8:22:18<89:52:17,  9.41s/it]
  9%|▉         | 3591/37962 [8:22:26<86:30:25,  9.06s/it]
  9%|▉         | 3592/37962 [8:22:34<84:21:20,  8.84s/it]
  9%|▉         | 3593/37962 [8:22:42<81:58:23,  8.59s/it]
  9%|▉         | 3594/37962 [8:22:51<81:06:28,  8.50s/it]
  9%|▉         | 3595/37962 [8:22:59<80:19:52,  8.41s/it]
  9%|▉         | 3596/37962 [8:23:07<79:37:19,  8.34s/it]
  9%|▉         | 3597/37962 [8:23:15<79:26:43,  8.32s/it]
  9%|▉         | 3598/37962 [8:23:24<79:06:56,  8.29s/it]
  9%|▉         | 3599/37962 [8:23:32<79:52:37,  8.37s/it]
  9%|▉         | 3600/37962 [8:23:40<79:36:16,  8.34s/it]
                                                         
{'loss': 1.5065, 'grad_norm': 18.63764762878418, 'learning_rate': 1.9780386725135595e-05, 'epoch': 0.09}

  9%|▉         | 3600/37962 [8:23:40<79:36:16,  8.34s/it]
  9%|▉         | 3601/37962 [8:23:48<78:44:56,  8.25s/it]
  9%|▉         | 3602/37962 [8:23:57<79:21:03,  8.31s/it]
  9%|▉         | 3603/37962 [8:24:05<79:27:19,  8.33s/it]
  9%|▉         | 3604/37962 [8:24:13<79:11:25,  8.30s/it]
  9%|▉         | 3605/37962 [8:24:22<79:23:08,  8.32s/it]
  9%|▉         | 3606/37962 [8:24:30<79:09:31,  8.29s/it]
 10%|▉         | 3607/37962 [8:24:38<79:02:53,  8.28s/it]
 10%|▉         | 3608/37962 [8:24:46<78:31:13,  8.23s/it]
 10%|▉         | 3609/37962 [8:24:55<79:11:36,  8.30s/it]
 10%|▉         | 3610/37962 [8:25:03<79:01:11,  8.28s/it]
                                                         
{'loss': 1.7058, 'grad_norm': 21.91177749633789, 'learning_rate': 1.977860497904004e-05, 'epoch': 0.1}

 10%|▉         | 3610/37962 [8:25:03<79:01:11,  8.28s/it]
 10%|▉         | 3611/37962 [8:25:11<78:40:06,  8.24s/it]
 10%|▉         | 3612/37962 [8:25:20<79:03:52,  8.29s/it]
 10%|▉         | 3613/37962 [8:25:28<79:01:17,  8.28s/it]
 10%|▉         | 3614/37962 [8:25:36<79:33:45,  8.34s/it]
 10%|▉         | 3615/37962 [8:25:45<79:13:07,  8.30s/it]
 10%|▉         | 3616/37962 [8:25:53<79:26:16,  8.33s/it]
 10%|▉         | 3617/37962 [8:26:01<79:32:46,  8.34s/it]
 10%|▉         | 3618/37962 [8:26:10<79:12:21,  8.30s/it]
 10%|▉         | 3619/37962 [8:26:18<78:56:27,  8.27s/it]
 10%|▉         | 3620/37962 [8:26:26<79:20:38,  8.32s/it]
                                                         
{'loss': 1.577, 'grad_norm': 38.426536560058594, 'learning_rate': 1.9776816115267314e-05, 'epoch': 0.1}

 10%|▉         | 3620/37962 [8:26:26<79:20:38,  8.32s/it]
 10%|▉         | 3621/37962 [8:26:34<78:12:10,  8.20s/it]
 10%|▉         | 3622/37962 [8:26:42<78:01:18,  8.18s/it]
 10%|▉         | 3623/37962 [8:26:50<77:18:59,  8.11s/it]
 10%|▉         | 3624/37962 [8:26:58<77:54:31,  8.17s/it]
 10%|▉         | 3625/37962 [8:27:07<78:28:30,  8.23s/it]
 10%|▉         | 3626/37962 [8:27:15<78:50:14,  8.27s/it]
 10%|▉         | 3627/37962 [8:27:23<77:59:40,  8.18s/it]
 10%|▉         | 3628/37962 [8:27:31<78:06:13,  8.19s/it]
 10%|▉         | 3629/37962 [8:27:40<78:29:36,  8.23s/it]
 10%|▉         | 3630/37962 [8:27:48<78:09:00,  8.19s/it]
                                                         
{'loss': 1.6971, 'grad_norm': 19.14558982849121, 'learning_rate': 1.9775020135119513e-05, 'epoch': 0.1}

 10%|▉         | 3630/37962 [8:27:48<78:09:00,  8.19s/it]
 10%|▉         | 3631/37962 [8:27:56<77:59:09,  8.18s/it]
 10%|▉         | 3632/37962 [8:28:04<77:24:19,  8.12s/it]
 10%|▉         | 3633/37962 [8:28:12<77:45:28,  8.15s/it]
 10%|▉         | 3634/37962 [8:28:20<77:36:18,  8.14s/it]
 10%|▉         | 3635/37962 [8:28:28<77:31:25,  8.13s/it]
 10%|▉         | 3636/37962 [8:28:37<77:33:31,  8.13s/it]
 10%|▉         | 3637/37962 [8:28:45<77:28:45,  8.13s/it]
 10%|▉         | 3638/37962 [8:28:53<77:01:28,  8.08s/it]
 10%|▉         | 3639/37962 [8:29:01<77:08:34,  8.09s/it]
 10%|▉         | 3640/37962 [8:29:09<76:16:27,  8.00s/it]
                                                         
{'loss': 1.8455, 'grad_norm': 39.78761672973633, 'learning_rate': 1.9773217039903897e-05, 'epoch': 0.1}

 10%|▉         | 3640/37962 [8:29:09<76:16:27,  8.00s/it]
 10%|▉         | 3641/37962 [8:29:17<76:21:31,  8.01s/it]
 10%|▉         | 3642/37962 [8:29:25<76:56:13,  8.07s/it]
 10%|▉         | 3643/37962 [8:29:33<78:05:55,  8.19s/it]
 10%|▉         | 3644/37962 [8:29:42<78:32:50,  8.24s/it]
 10%|▉         | 3645/37962 [8:29:50<78:22:59,  8.22s/it]
 10%|▉         | 3646/37962 [8:29:58<78:26:14,  8.23s/it]
 10%|▉         | 3647/37962 [8:30:06<78:47:25,  8.27s/it]
 10%|▉         | 3648/37962 [8:30:15<78:24:43,  8.23s/it]
 10%|▉         | 3649/37962 [8:30:23<78:40:18,  8.25s/it]
 10%|▉         | 3650/37962 [8:30:31<78:55:02,  8.28s/it]
                                                         
{'loss': 1.6466, 'grad_norm': 10.848459243774414, 'learning_rate': 1.9771406830932906e-05, 'epoch': 0.1}

 10%|▉         | 3650/37962 [8:30:31<78:55:02,  8.28s/it]
 10%|▉         | 3651/37962 [8:30:40<79:30:34,  8.34s/it]
 10%|▉         | 3652/37962 [8:30:48<78:32:01,  8.24s/it]
 10%|▉         | 3653/37962 [8:30:56<77:53:08,  8.17s/it]
 10%|▉         | 3654/37962 [8:31:04<78:28:38,  8.23s/it]
 10%|▉         | 3655/37962 [8:31:12<78:25:43,  8.23s/it]
 10%|▉         | 3656/37962 [8:31:20<77:21:08,  8.12s/it]
 10%|▉         | 3657/37962 [8:31:28<76:56:06,  8.07s/it]
 10%|▉         | 3658/37962 [8:31:37<78:19:45,  8.22s/it]
 10%|▉         | 3659/37962 [8:31:45<77:59:59,  8.19s/it]
 10%|▉         | 3660/37962 [8:31:53<77:58:32,  8.18s/it]
                                                         
{'loss': 1.4754, 'grad_norm': 22.35468864440918, 'learning_rate': 1.9769589509524163e-05, 'epoch': 0.1}

 10%|▉         | 3660/37962 [8:31:53<77:58:32,  8.18s/it]
 10%|▉         | 3661/37962 [8:32:01<78:22:40,  8.23s/it]
 10%|▉         | 3662/37962 [8:32:10<78:50:07,  8.27s/it]
 10%|▉         | 3663/37962 [8:32:18<77:48:40,  8.17s/it]
 10%|▉         | 3664/37962 [8:32:26<77:55:10,  8.18s/it]
 10%|▉         | 3665/37962 [8:32:34<78:07:11,  8.20s/it]
 10%|▉         | 3666/37962 [8:32:42<78:11:13,  8.21s/it]
 10%|▉         | 3667/37962 [8:32:51<78:41:16,  8.26s/it]
 10%|▉         | 3668/37962 [8:32:59<78:22:07,  8.23s/it]
 10%|▉         | 3669/37962 [8:33:07<78:42:48,  8.26s/it]
 10%|▉         | 3670/37962 [8:33:15<78:35:06,  8.25s/it]
                                                         
{'loss': 1.6114, 'grad_norm': 56.802330017089844, 'learning_rate': 1.976776507700046e-05, 'epoch': 0.1}

 10%|▉         | 3670/37962 [8:33:15<78:35:06,  8.25s/it]
 10%|▉         | 3671/37962 [8:33:24<78:31:37,  8.24s/it]
 10%|▉         | 3672/37962 [8:33:32<78:32:08,  8.25s/it]
 10%|▉         | 3673/37962 [8:33:40<78:20:07,  8.22s/it]
 10%|▉         | 3674/37962 [8:33:48<78:49:23,  8.28s/it]
 10%|▉         | 3675/37962 [8:33:57<79:07:46,  8.31s/it]
 10%|▉         | 3676/37962 [8:34:05<78:26:22,  8.24s/it]
 10%|▉         | 3677/37962 [8:34:13<78:28:29,  8.24s/it]
 10%|▉         | 3678/37962 [8:34:21<78:47:48,  8.27s/it]
 10%|▉         | 3679/37962 [8:34:30<79:17:09,  8.33s/it]
 10%|▉         | 3680/37962 [8:34:38<78:58:45,  8.29s/it]
                                                         
{'loss': 1.5733, 'grad_norm': 23.9422607421875, 'learning_rate': 1.9765933534689775e-05, 'epoch': 0.1}

 10%|▉         | 3680/37962 [8:34:38<78:58:45,  8.29s/it]
 10%|▉         | 3681/37962 [8:34:46<78:51:48,  8.28s/it]
 10%|▉         | 3682/37962 [8:34:55<78:38:05,  8.26s/it]
 10%|▉         | 3683/37962 [8:35:03<79:05:29,  8.31s/it]
 10%|▉         | 3684/37962 [8:35:11<78:57:40,  8.29s/it]
 10%|▉         | 3685/37962 [8:35:19<78:40:20,  8.26s/it]
 10%|▉         | 3686/37962 [8:35:28<78:24:59,  8.24s/it]
 10%|▉         | 3687/37962 [8:35:36<78:55:32,  8.29s/it]
 10%|▉         | 3688/37962 [8:35:44<78:49:49,  8.28s/it]
 10%|▉         | 3689/37962 [8:35:53<78:53:05,  8.29s/it]
 10%|▉         | 3690/37962 [8:36:01<79:05:00,  8.31s/it]
                                                         
{'loss': 1.5558, 'grad_norm': 11.309412956237793, 'learning_rate': 1.976409488392525e-05, 'epoch': 0.1}

 10%|▉         | 3690/37962 [8:36:01<79:05:00,  8.31s/it]
 10%|▉         | 3691/37962 [8:36:09<79:10:31,  8.32s/it]
 10%|▉         | 3692/37962 [8:36:18<79:00:58,  8.30s/it]
 10%|▉         | 3693/37962 [8:36:26<78:53:17,  8.29s/it]
 10%|▉         | 3694/37962 [8:36:34<79:01:19,  8.30s/it]
 10%|▉         | 3695/37962 [8:36:42<78:34:53,  8.26s/it]
 10%|▉         | 3696/37962 [8:36:51<78:53:22,  8.29s/it]
 10%|▉         | 3697/37962 [8:36:59<78:40:55,  8.27s/it]
 10%|▉         | 3698/37962 [8:37:07<78:12:57,  8.22s/it]
 10%|▉         | 3699/37962 [8:37:15<78:48:11,  8.28s/it]
 10%|▉         | 3700/37962 [8:37:23<78:08:52,  8.21s/it]
                                                         
{'loss': 1.4379, 'grad_norm': 9.368575096130371, 'learning_rate': 1.9762249126045214e-05, 'epoch': 0.1}

 10%|▉         | 3700/37962 [8:37:23<78:08:52,  8.21s/it]
 10%|▉         | 3701/37962 [8:37:32<78:18:08,  8.23s/it]
 10%|▉         | 3702/37962 [8:37:40<77:37:40,  8.16s/it]
 10%|▉         | 3703/37962 [8:37:48<77:12:13,  8.11s/it]
 10%|▉         | 3704/37962 [8:37:56<77:54:03,  8.19s/it]
 10%|▉         | 3705/37962 [8:38:04<77:15:48,  8.12s/it]
 10%|▉         | 3706/37962 [8:38:12<78:01:07,  8.20s/it]
 10%|▉         | 3707/37962 [8:38:21<78:38:27,  8.26s/it]
 10%|▉         | 3708/37962 [8:38:29<78:46:43,  8.28s/it]
 10%|▉         | 3709/37962 [8:38:37<78:13:06,  8.22s/it]
 10%|▉         | 3710/37962 [8:38:45<78:01:12,  8.20s/it]
                                                         
{'loss': 1.6052, 'grad_norm': 7.18232536315918, 'learning_rate': 1.9760396262393155e-05, 'epoch': 0.1}

 10%|▉         | 3710/37962 [8:38:45<78:01:12,  8.20s/it]
 10%|▉         | 3711/37962 [8:38:54<78:14:48,  8.22s/it]
 10%|▉         | 3712/37962 [8:39:02<78:55:45,  8.30s/it]
 10%|▉         | 3713/37962 [8:39:11<79:43:34,  8.38s/it]
 10%|▉         | 3714/37962 [8:39:19<79:38:17,  8.37s/it]
 10%|▉         | 3715/37962 [8:39:27<79:10:06,  8.32s/it]
 10%|▉         | 3716/37962 [8:39:35<78:30:00,  8.25s/it]
 10%|▉         | 3717/37962 [8:39:44<78:28:36,  8.25s/it]
 10%|▉         | 3718/37962 [8:39:52<77:46:00,  8.18s/it]
 10%|▉         | 3719/37962 [8:40:00<77:58:42,  8.20s/it]
 10%|▉         | 3720/37962 [8:40:08<77:56:28,  8.19s/it]
                                                         
{'loss': 1.4501, 'grad_norm': 7.639756202697754, 'learning_rate': 1.9758536294317748e-05, 'epoch': 0.1}

 10%|▉         | 3720/37962 [8:40:08<77:56:28,  8.19s/it]
 10%|▉         | 3721/37962 [8:40:16<78:02:26,  8.20s/it]
 10%|▉         | 3722/37962 [8:40:25<78:05:42,  8.21s/it]
 10%|▉         | 3723/37962 [8:40:33<77:42:58,  8.17s/it]
 10%|▉         | 3724/37962 [8:40:41<78:18:23,  8.23s/it]
 10%|▉         | 3725/37962 [8:40:49<78:28:28,  8.25s/it]
 10%|▉         | 3726/37962 [8:40:58<78:49:11,  8.29s/it]
 10%|▉         | 3727/37962 [8:41:06<78:05:25,  8.21s/it]
 10%|▉         | 3728/37962 [8:41:14<78:22:19,  8.24s/it]
 10%|▉         | 3729/37962 [8:41:22<78:27:18,  8.25s/it]
 10%|▉         | 3730/37962 [8:41:31<78:28:18,  8.25s/it]
                                                         
{'loss': 1.4066, 'grad_norm': 24.124452590942383, 'learning_rate': 1.975666922317282e-05, 'epoch': 0.1}

 10%|▉         | 3730/37962 [8:41:31<78:28:18,  8.25s/it]
 10%|▉         | 3731/37962 [8:41:39<78:33:59,  8.26s/it]
 10%|▉         | 3732/37962 [8:41:47<78:56:53,  8.30s/it]
 10%|▉         | 3733/37962 [8:41:55<78:48:54,  8.29s/it]
 10%|▉         | 3734/37962 [8:42:04<79:03:22,  8.31s/it]
 10%|▉         | 3735/37962 [8:42:12<79:17:25,  8.34s/it]
 10%|▉         | 3736/37962 [8:42:20<78:47:25,  8.29s/it]
 10%|▉         | 3737/37962 [8:42:29<78:41:42,  8.28s/it]
 10%|▉         | 3738/37962 [8:42:37<78:19:21,  8.24s/it]
 10%|▉         | 3739/37962 [8:42:45<78:00:23,  8.21s/it]
 10%|▉         | 3740/37962 [8:42:53<78:25:27,  8.25s/it]
                                                         
{'loss': 1.4057, 'grad_norm': 15.980554580688477, 'learning_rate': 1.9754795050317398e-05, 'epoch': 0.1}

 10%|▉         | 3740/37962 [8:42:53<78:25:27,  8.25s/it]
 10%|▉         | 3741/37962 [8:43:01<78:11:06,  8.22s/it]
 10%|▉         | 3742/37962 [8:43:10<78:04:04,  8.21s/it]
 10%|▉         | 3743/37962 [8:43:18<78:09:01,  8.22s/it]
 10%|▉         | 3744/37962 [8:43:26<78:26:32,  8.25s/it]
 10%|▉         | 3745/37962 [8:43:34<78:17:34,  8.24s/it]
 10%|▉         | 3746/37962 [8:43:42<77:50:21,  8.19s/it]
 10%|▉         | 3747/37962 [8:43:51<78:33:23,  8.27s/it]
 10%|▉         | 3748/37962 [8:43:59<77:55:37,  8.20s/it]
 10%|▉         | 3749/37962 [8:44:07<78:44:39,  8.29s/it]
 10%|▉         | 3750/37962 [8:44:16<78:58:48,  8.31s/it]
                                                         
{'loss': 1.4632, 'grad_norm': 24.554969787597656, 'learning_rate': 1.975291377711564e-05, 'epoch': 0.1}

 10%|▉         | 3750/37962 [8:44:16<78:58:48,  8.31s/it]
 10%|▉         | 3751/37962 [8:44:24<79:02:30,  8.32s/it]
 10%|▉         | 3752/37962 [8:44:32<78:11:24,  8.23s/it]
 10%|▉         | 3753/37962 [8:44:40<78:26:58,  8.26s/it]
 10%|▉         | 3754/37962 [8:44:49<78:06:10,  8.22s/it]
 10%|▉         | 3755/37962 [8:44:57<78:02:56,  8.21s/it]
 10%|▉         | 3756/37962 [8:45:05<78:26:58,  8.26s/it]
 10%|▉         | 3757/37962 [8:45:13<78:07:12,  8.22s/it]
 10%|▉         | 3758/37962 [8:45:22<78:20:12,  8.25s/it]
 10%|▉         | 3759/37962 [8:45:30<78:09:34,  8.23s/it]
 10%|▉         | 3760/37962 [8:45:38<79:02:08,  8.32s/it]
                                                         
{'loss': 1.3832, 'grad_norm': 4.167383193969727, 'learning_rate': 1.9751025404936907e-05, 'epoch': 0.1}

 10%|▉         | 3760/37962 [8:45:38<79:02:08,  8.32s/it]
 10%|▉         | 3761/37962 [8:45:47<78:40:50,  8.28s/it]
 10%|▉         | 3762/37962 [8:45:55<78:56:27,  8.31s/it]
 10%|▉         | 3763/37962 [8:46:03<78:52:13,  8.30s/it]
 10%|▉         | 3764/37962 [8:46:11<78:04:24,  8.22s/it]
 10%|▉         | 3765/37962 [8:46:19<77:10:58,  8.13s/it]
 10%|▉         | 3766/37962 [8:46:27<77:09:29,  8.12s/it]
 10%|▉         | 3767/37962 [8:46:35<76:58:38,  8.10s/it]
 10%|▉         | 3768/37962 [8:46:43<77:13:18,  8.13s/it]
 10%|▉         | 3769/37962 [8:46:52<77:45:04,  8.19s/it]
 10%|▉         | 3770/37962 [8:47:00<77:32:08,  8.16s/it]
                                                         
{'loss': 1.4631, 'grad_norm': 12.544156074523926, 'learning_rate': 1.9749129935155704e-05, 'epoch': 0.1}

 10%|▉         | 3770/37962 [8:47:00<77:32:08,  8.16s/it]
 10%|▉         | 3771/37962 [8:47:08<77:37:09,  8.17s/it]
 10%|▉         | 3772/37962 [8:47:16<77:44:58,  8.19s/it]
 10%|▉         | 3773/37962 [8:47:25<78:12:29,  8.24s/it]
 10%|▉         | 3774/37962 [8:47:33<78:16:48,  8.24s/it]
 10%|▉         | 3775/37962 [8:47:41<77:46:14,  8.19s/it]
 10%|▉         | 3776/37962 [8:47:49<78:00:47,  8.22s/it]
 10%|▉         | 3777/37962 [8:47:58<78:28:21,  8.26s/it]
 10%|▉         | 3778/37962 [8:48:06<78:24:52,  8.26s/it]
 10%|▉         | 3779/37962 [8:48:14<78:08:33,  8.23s/it]
 10%|▉         | 3780/37962 [8:48:22<78:08:26,  8.23s/it]
                                                         
{'loss': 1.4958, 'grad_norm': 17.877382278442383, 'learning_rate': 1.974722736915171e-05, 'epoch': 0.1}

 10%|▉         | 3780/37962 [8:48:22<78:08:26,  8.23s/it]
 10%|▉         | 3781/37962 [8:48:31<78:16:35,  8.24s/it]
 10%|▉         | 3782/37962 [8:48:39<78:07:26,  8.23s/it]
 10%|▉         | 3783/37962 [8:48:47<78:02:31,  8.22s/it]
 10%|▉         | 3784/37962 [8:48:56<78:59:30,  8.32s/it]
 10%|▉         | 3785/37962 [8:49:04<79:07:15,  8.33s/it]
 10%|▉         | 3786/37962 [8:49:12<79:20:17,  8.36s/it]
 10%|▉         | 3787/37962 [8:49:21<79:21:03,  8.36s/it]
 10%|▉         | 3788/37962 [8:49:29<78:23:27,  8.26s/it]
 10%|▉         | 3789/37962 [8:49:37<78:38:49,  8.29s/it]
 10%|▉         | 3790/37962 [8:49:45<78:35:39,  8.28s/it]
                                                         
{'loss': 1.5524, 'grad_norm': 17.42013168334961, 'learning_rate': 1.974531770830978e-05, 'epoch': 0.1}

 10%|▉         | 3790/37962 [8:49:45<78:35:39,  8.28s/it]
 10%|▉         | 3791/37962 [8:49:54<79:01:56,  8.33s/it]
 10%|▉         | 3792/37962 [8:50:02<78:52:06,  8.31s/it]
 10%|▉         | 3793/37962 [8:50:10<78:35:36,  8.28s/it]
 10%|▉         | 3794/37962 [8:50:18<78:35:00,  8.28s/it]
 10%|▉         | 3795/37962 [8:50:27<78:24:25,  8.26s/it]
 10%|▉         | 3796/37962 [8:50:35<78:08:05,  8.23s/it]
 10%|█         | 3797/37962 [8:50:43<77:48:22,  8.20s/it]
 10%|█         | 3798/37962 [8:50:51<77:28:54,  8.16s/it]
 10%|█         | 3799/37962 [8:50:59<78:06:09,  8.23s/it]
 10%|█         | 3800/37962 [8:51:08<78:14:04,  8.24s/it]
                                                         
{'loss': 1.5187, 'grad_norm': 26.338783264160156, 'learning_rate': 1.974340095401991e-05, 'epoch': 0.1}

 10%|█         | 3800/37962 [8:51:08<78:14:04,  8.24s/it]
 10%|█         | 3801/37962 [8:51:16<77:43:38,  8.19s/it]
 10%|█         | 3802/37962 [8:51:24<77:40:04,  8.19s/it]
 10%|█         | 3803/37962 [8:51:32<77:59:13,  8.22s/it]
 10%|█         | 3804/37962 [8:51:40<77:53:34,  8.21s/it]
 10%|█         | 3805/37962 [8:51:48<77:15:30,  8.14s/it]
 10%|█         | 3806/37962 [8:51:57<77:25:10,  8.16s/it]
 10%|█         | 3807/37962 [8:52:05<77:52:23,  8.21s/it]
 10%|█         | 3808/37962 [8:52:13<77:42:00,  8.19s/it]
 10%|█         | 3809/37962 [8:52:21<77:55:59,  8.21s/it]
 10%|█         | 3810/37962 [8:52:30<77:56:37,  8.22s/it]
                                                         
{'loss': 1.4967, 'grad_norm': 29.520784378051758, 'learning_rate': 1.9741477107677282e-05, 'epoch': 0.1}

 10%|█         | 3810/37962 [8:52:30<77:56:37,  8.22s/it]
 10%|█         | 3811/37962 [8:52:38<77:04:34,  8.12s/it]
 10%|█         | 3812/37962 [8:52:46<77:40:52,  8.19s/it]
 10%|█         | 3813/37962 [8:52:54<77:55:19,  8.21s/it]
 10%|█         | 3814/37962 [8:53:02<77:58:18,  8.22s/it]
 10%|█         | 3815/37962 [8:53:11<78:17:15,  8.25s/it]
 10%|█         | 3816/37962 [8:53:19<78:08:44,  8.24s/it]
 10%|█         | 3817/37962 [8:53:27<76:58:04,  8.11s/it]
 10%|█         | 3818/37962 [8:53:35<77:42:56,  8.19s/it]
 10%|█         | 3819/37962 [8:53:43<76:32:51,  8.07s/it]
 10%|█         | 3820/37962 [8:53:51<77:31:06,  8.17s/it]
                                                         
{'loss': 1.4196, 'grad_norm': 13.96440601348877, 'learning_rate': 1.973954617068223e-05, 'epoch': 0.1}

 10%|█         | 3820/37962 [8:53:51<77:31:06,  8.17s/it]
 10%|█         | 3821/37962 [8:54:00<78:08:16,  8.24s/it]
 10%|█         | 3822/37962 [8:54:08<78:06:18,  8.24s/it]
 10%|█         | 3823/37962 [8:54:16<77:58:01,  8.22s/it]
 10%|█         | 3824/37962 [8:54:24<77:16:36,  8.15s/it]
 10%|█         | 3825/37962 [8:54:33<78:39:25,  8.29s/it]
 10%|█         | 3826/37962 [8:54:41<77:41:35,  8.19s/it]
 10%|█         | 3827/37962 [8:54:49<77:41:29,  8.19s/it]
 10%|█         | 3828/37962 [8:54:57<77:44:15,  8.20s/it]
 10%|█         | 3829/37962 [8:55:05<77:58:40,  8.22s/it]
 10%|█         | 3830/37962 [8:55:14<77:52:37,  8.21s/it]
                                                         
{'loss': 1.5988, 'grad_norm': 12.393430709838867, 'learning_rate': 1.973760814444024e-05, 'epoch': 0.1}

 10%|█         | 3830/37962 [8:55:14<77:52:37,  8.21s/it]
 10%|█         | 3831/37962 [8:55:21<76:45:09,  8.10s/it]
 10%|█         | 3832/37962 [8:55:30<77:12:12,  8.14s/it]
 10%|█         | 3833/37962 [8:55:38<77:45:15,  8.20s/it]
 10%|█         | 3834/37962 [8:55:46<77:56:06,  8.22s/it]
 10%|█         | 3835/37962 [8:55:55<78:07:36,  8.24s/it]
 10%|█         | 3836/37962 [8:56:03<77:34:16,  8.18s/it]
 10%|█         | 3837/37962 [8:56:11<78:03:07,  8.23s/it]
 10%|█         | 3838/37962 [8:56:19<78:26:35,  8.28s/it]
 10%|█         | 3839/37962 [8:56:27<78:10:47,  8.25s/it]
 10%|█         | 3840/37962 [8:56:36<78:13:45,  8.25s/it]
                                                         
{'loss': 1.4195, 'grad_norm': 18.336427688598633, 'learning_rate': 1.9735663030361974e-05, 'epoch': 0.1}

 10%|█         | 3840/37962 [8:56:36<78:13:45,  8.25s/it]
 10%|█         | 3841/37962 [8:56:44<78:03:44,  8.24s/it]
 10%|█         | 3842/37962 [8:56:52<78:12:29,  8.25s/it]
 10%|█         | 3843/37962 [8:57:01<78:14:35,  8.26s/it]
 10%|█         | 3844/37962 [8:57:09<78:15:31,  8.26s/it]
 10%|█         | 3845/37962 [8:57:17<78:40:45,  8.30s/it]
 10%|█         | 3846/37962 [8:57:25<78:42:20,  8.31s/it]
 10%|█         | 3847/37962 [8:57:33<77:32:27,  8.18s/it]
 10%|█         | 3848/37962 [8:57:42<77:45:47,  8.21s/it]
 10%|█         | 3849/37962 [8:57:50<77:26:42,  8.17s/it]
 10%|█         | 3850/37962 [8:57:58<77:40:13,  8.20s/it]
                                                         
{'loss': 1.4876, 'grad_norm': 10.676070213317871, 'learning_rate': 1.9733710829863252e-05, 'epoch': 0.1}

 10%|█         | 3850/37962 [8:57:58<77:40:13,  8.20s/it]
 10%|█         | 3851/37962 [8:58:06<77:45:47,  8.21s/it]
 10%|█         | 3852/37962 [8:58:14<77:43:35,  8.20s/it]
 10%|█         | 3853/37962 [8:58:23<77:47:38,  8.21s/it]
 10%|█         | 3854/37962 [8:58:31<78:28:01,  8.28s/it]
 10%|█         | 3855/37962 [8:58:39<78:38:29,  8.30s/it]
 10%|█         | 3856/37962 [8:58:48<78:27:37,  8.28s/it]
 10%|█         | 3857/37962 [8:58:56<78:12:55,  8.26s/it]
 10%|█         | 3858/37962 [8:59:04<77:57:41,  8.23s/it]
 10%|█         | 3859/37962 [8:59:12<77:50:48,  8.22s/it]
 10%|█         | 3860/37962 [8:59:20<77:48:05,  8.21s/it]
                                                         
{'loss': 1.5484, 'grad_norm': 20.098827362060547, 'learning_rate': 1.973175154436504e-05, 'epoch': 0.1}

 10%|█         | 3860/37962 [8:59:20<77:48:05,  8.21s/it]
 10%|█         | 3861/37962 [8:59:29<77:39:42,  8.20s/it]
 10%|█         | 3862/37962 [8:59:37<77:31:54,  8.19s/it]
 10%|█         | 3863/37962 [8:59:45<77:35:56,  8.19s/it]
 10%|█         | 3864/37962 [8:59:53<76:56:50,  8.12s/it]
 10%|█         | 3865/37962 [9:00:01<77:09:51,  8.15s/it]
 10%|█         | 3866/37962 [9:00:09<76:44:36,  8.10s/it]
 10%|█         | 3867/37962 [9:00:17<77:28:42,  8.18s/it]
 10%|█         | 3868/37962 [9:00:26<77:15:33,  8.16s/it]
 10%|█         | 3869/37962 [9:00:34<77:13:00,  8.15s/it]
 10%|█         | 3870/37962 [9:00:42<77:53:22,  8.22s/it]
                                                         
{'loss': 1.5342, 'grad_norm': 7.938894748687744, 'learning_rate': 1.972978517529347e-05, 'epoch': 0.1}

 10%|█         | 3870/37962 [9:00:42<77:53:22,  8.22s/it]
 10%|█         | 3871/37962 [9:00:50<77:55:22,  8.23s/it]
 10%|█         | 3872/37962 [9:00:58<77:18:35,  8.16s/it]
 10%|█         | 3873/37962 [9:01:07<77:28:07,  8.18s/it]
 10%|█         | 3874/37962 [9:01:15<77:45:46,  8.21s/it]
 10%|█         | 3875/37962 [9:01:23<78:28:09,  8.29s/it]
 10%|█         | 3876/37962 [9:01:32<78:20:49,  8.27s/it]
 10%|█         | 3877/37962 [9:01:40<78:03:39,  8.24s/it]
 10%|█         | 3878/37962 [9:01:48<78:46:34,  8.32s/it]
 10%|█         | 3879/37962 [9:01:57<79:08:46,  8.36s/it]
 10%|█         | 3880/37962 [9:02:05<78:53:42,  8.33s/it]
                                                         
{'loss': 1.5417, 'grad_norm': 14.114721298217773, 'learning_rate': 1.9727811724079828e-05, 'epoch': 0.1}

 10%|█         | 3880/37962 [9:02:05<78:53:42,  8.33s/it]
 10%|█         | 3881/37962 [9:02:13<78:44:36,  8.32s/it]
 10%|█         | 3882/37962 [9:02:21<78:10:52,  8.26s/it]
 10%|█         | 3883/37962 [9:02:29<77:42:49,  8.21s/it]
 10%|█         | 3884/37962 [9:02:38<78:21:25,  8.28s/it]
 10%|█         | 3885/37962 [9:02:46<78:36:48,  8.30s/it]
 10%|█         | 3886/37962 [9:02:54<78:03:26,  8.25s/it]
 10%|█         | 3887/37962 [9:03:03<77:55:51,  8.23s/it]
 10%|█         | 3888/37962 [9:03:11<78:06:19,  8.25s/it]
 10%|█         | 3889/37962 [9:03:19<78:18:40,  8.27s/it]
 10%|█         | 3890/37962 [9:03:28<78:23:44,  8.28s/it]
                                                         
{'loss': 1.4928, 'grad_norm': 13.582354545593262, 'learning_rate': 1.9725831192160556e-05, 'epoch': 0.1}

 10%|█         | 3890/37962 [9:03:28<78:23:44,  8.28s/it]
 10%|█         | 3891/37962 [9:03:36<78:44:00,  8.32s/it]
 10%|█         | 3892/37962 [9:03:44<78:23:28,  8.28s/it]
 10%|█         | 3893/37962 [9:03:52<78:34:05,  8.30s/it]
 10%|█         | 3894/37962 [9:04:01<78:09:25,  8.26s/it]
 10%|█         | 3895/37962 [9:04:09<79:03:00,  8.35s/it]
 10%|█         | 3896/37962 [9:04:17<78:39:39,  8.31s/it]
 10%|█         | 3897/37962 [9:04:26<78:28:08,  8.29s/it]
 10%|█         | 3898/37962 [9:04:34<78:35:53,  8.31s/it]
 10%|█         | 3899/37962 [9:04:42<78:24:57,  8.29s/it]
 10%|█         | 3900/37962 [9:04:51<78:28:59,  8.29s/it]
                                                         
{'loss': 1.4466, 'grad_norm': 19.619625091552734, 'learning_rate': 1.9723843580977246e-05, 'epoch': 0.1}

 10%|█         | 3900/37962 [9:04:51<78:28:59,  8.29s/it]
 10%|█         | 3901/37962 [9:04:59<78:54:13,  8.34s/it]
 10%|█         | 3902/37962 [9:05:07<77:46:46,  8.22s/it]
 10%|█         | 3903/37962 [9:05:15<77:51:15,  8.23s/it]
 10%|█         | 3904/37962 [9:05:24<78:10:00,  8.26s/it]
 10%|█         | 3905/37962 [9:05:32<78:52:29,  8.34s/it]
 10%|█         | 3906/37962 [9:05:40<78:37:04,  8.31s/it]
 10%|█         | 3907/37962 [9:05:49<78:33:27,  8.30s/it]
 10%|█         | 3908/37962 [9:05:57<78:19:05,  8.28s/it]
 10%|█         | 3909/37962 [9:06:05<79:08:05,  8.37s/it]
 10%|█         | 3910/37962 [9:06:14<78:48:07,  8.33s/it]
                                                         
{'loss': 1.5625, 'grad_norm': 6.047610759735107, 'learning_rate': 1.9721848891976644e-05, 'epoch': 0.1}

 10%|█         | 3910/37962 [9:06:14<78:48:07,  8.33s/it]
 10%|█         | 3911/37962 [9:06:22<78:50:33,  8.34s/it]
 10%|█         | 3912/37962 [9:06:30<78:19:08,  8.28s/it]
 10%|█         | 3913/37962 [9:06:38<77:50:22,  8.23s/it]
 10%|█         | 3914/37962 [9:06:46<77:50:39,  8.23s/it]
 10%|█         | 3915/37962 [9:06:55<77:49:24,  8.23s/it]
 10%|█         | 3916/37962 [9:07:03<77:59:37,  8.25s/it]
 10%|█         | 3917/37962 [9:07:11<77:29:25,  8.19s/it]
 10%|█         | 3918/37962 [9:07:19<77:36:56,  8.21s/it]
 10%|█         | 3919/37962 [9:07:28<77:52:10,  8.23s/it]
 10%|█         | 3920/37962 [9:07:36<78:10:31,  8.27s/it]
                                                         
{'loss': 1.5215, 'grad_norm': 5.205148696899414, 'learning_rate': 1.971984712661066e-05, 'epoch': 0.1}

 10%|█         | 3920/37962 [9:07:36<78:10:31,  8.27s/it]
 10%|█         | 3921/37962 [9:07:44<78:41:53,  8.32s/it]
 10%|█         | 3922/37962 [9:07:52<77:47:46,  8.23s/it]
 10%|█         | 3923/37962 [9:08:01<77:45:38,  8.22s/it]
 10%|█         | 3924/37962 [9:08:09<77:22:57,  8.18s/it]
 10%|█         | 3925/37962 [9:08:17<77:24:01,  8.19s/it]
 10%|█         | 3926/37962 [9:08:25<77:26:38,  8.19s/it]
 10%|█         | 3927/37962 [9:08:33<77:57:43,  8.25s/it]
 10%|█         | 3928/37962 [9:08:42<78:34:57,  8.31s/it]
 10%|█         | 3929/37962 [9:08:50<77:04:16,  8.15s/it]
 10%|█         | 3930/37962 [9:08:58<77:51:29,  8.24s/it]
                                                         
{'loss': 1.4224, 'grad_norm': 16.013450622558594, 'learning_rate': 1.9717838286336334e-05, 'epoch': 0.1}

 10%|█         | 3930/37962 [9:08:58<77:51:29,  8.24s/it]
 10%|█         | 3931/37962 [9:09:06<77:21:30,  8.18s/it]
 10%|█         | 3932/37962 [9:09:14<77:26:10,  8.19s/it]
 10%|█         | 3933/37962 [9:09:23<77:46:14,  8.23s/it]
 10%|█         | 3934/37962 [9:09:31<78:14:20,  8.28s/it]
 10%|█         | 3935/37962 [9:09:39<78:09:27,  8.27s/it]
 10%|█         | 3936/37962 [9:09:48<77:53:40,  8.24s/it]
 10%|█         | 3937/37962 [9:09:56<77:48:46,  8.23s/it]
 10%|█         | 3938/37962 [9:10:04<77:41:45,  8.22s/it]
 10%|█         | 3939/37962 [9:10:12<76:58:16,  8.14s/it]
 10%|█         | 3940/37962 [9:10:20<75:51:40,  8.03s/it]
                                                         
{'loss': 1.4691, 'grad_norm': 11.868693351745605, 'learning_rate': 1.9715822372615867e-05, 'epoch': 0.1}

 10%|█         | 3940/37962 [9:10:20<75:51:40,  8.03s/it]
 10%|█         | 3941/37962 [9:10:28<77:10:32,  8.17s/it]
 10%|█         | 3942/37962 [9:10:36<77:16:37,  8.18s/it]
 10%|█         | 3943/37962 [9:10:45<77:31:53,  8.20s/it]
 10%|█         | 3944/37962 [9:10:53<77:46:35,  8.23s/it]
 10%|█         | 3945/37962 [9:11:01<77:01:54,  8.15s/it]
 10%|█         | 3946/37962 [9:11:09<77:33:43,  8.21s/it]
 10%|█         | 3947/37962 [9:11:18<78:04:26,  8.26s/it]
 10%|█         | 3948/37962 [9:11:26<77:56:22,  8.25s/it]
 10%|█         | 3949/37962 [9:11:34<77:25:23,  8.19s/it]
 10%|█         | 3950/37962 [9:11:42<77:46:46,  8.23s/it]
                                                         
{'loss': 1.6906, 'grad_norm': 10.655168533325195, 'learning_rate': 1.9713799386916612e-05, 'epoch': 0.1}

 10%|█         | 3950/37962 [9:11:42<77:46:46,  8.23s/it]
 10%|█         | 3951/37962 [9:11:50<77:46:41,  8.23s/it]
 10%|█         | 3952/37962 [9:11:59<78:27:40,  8.31s/it]
 10%|█         | 3953/37962 [9:12:07<77:41:28,  8.22s/it]
 10%|█         | 3954/37962 [9:12:15<77:45:48,  8.23s/it]
 10%|█         | 3955/37962 [9:12:23<77:10:47,  8.17s/it]
 10%|█         | 3956/37962 [9:12:32<77:41:10,  8.22s/it]
 10%|█         | 3957/37962 [9:12:40<77:40:49,  8.22s/it]
 10%|█         | 3958/37962 [9:12:48<77:39:54,  8.22s/it]
 10%|█         | 3959/37962 [9:12:56<77:45:39,  8.23s/it]
 10%|█         | 3960/37962 [9:13:04<77:38:34,  8.22s/it]
                                                         
{'loss': 1.6595, 'grad_norm': 15.245192527770996, 'learning_rate': 1.9711769330711067e-05, 'epoch': 0.1}

 10%|█         | 3960/37962 [9:13:05<77:38:34,  8.22s/it]
 10%|█         | 3961/37962 [9:13:13<77:49:39,  8.24s/it]
 10%|█         | 3962/37962 [9:13:21<78:05:29,  8.27s/it]
 10%|█         | 3963/37962 [9:13:29<77:32:20,  8.21s/it]
 10%|█         | 3964/37962 [9:13:37<77:44:17,  8.23s/it]
 10%|█         | 3965/37962 [9:13:46<77:40:51,  8.23s/it]
 10%|█         | 3966/37962 [9:13:54<77:44:47,  8.23s/it]
 10%|█         | 3967/37962 [9:14:02<77:51:07,  8.24s/it]
 10%|█         | 3968/37962 [9:14:11<78:51:15,  8.35s/it]
 10%|█         | 3969/37962 [9:14:19<78:33:01,  8.32s/it]
 10%|█         | 3970/37962 [9:14:27<78:22:06,  8.30s/it]
                                                         
{'loss': 1.5711, 'grad_norm': 40.055274963378906, 'learning_rate': 1.9709732205476867e-05, 'epoch': 0.1}

 10%|█         | 3970/37962 [9:14:27<78:22:06,  8.30s/it]
 10%|█         | 3971/37962 [9:14:36<78:45:41,  8.34s/it]
 10%|█         | 3972/37962 [9:14:44<78:25:10,  8.31s/it]
 10%|█         | 3973/37962 [9:14:52<78:48:32,  8.35s/it]
 10%|█         | 3974/37962 [9:15:01<78:42:30,  8.34s/it]
 10%|█         | 3975/37962 [9:15:09<78:12:18,  8.28s/it]
 10%|█         | 3976/37962 [9:15:17<78:37:46,  8.33s/it]
 10%|█         | 3977/37962 [9:15:25<78:06:37,  8.27s/it]
 10%|█         | 3978/37962 [9:15:34<78:10:01,  8.28s/it]
 10%|█         | 3979/37962 [9:15:42<79:08:39,  8.38s/it]
 10%|█         | 3980/37962 [9:15:50<77:53:17,  8.25s/it]
                                                         
{'loss': 1.5253, 'grad_norm': 37.79780960083008, 'learning_rate': 1.9707688012696805e-05, 'epoch': 0.1}

 10%|█         | 3980/37962 [9:15:50<77:53:17,  8.25s/it]
 10%|█         | 3981/37962 [9:15:59<78:19:53,  8.30s/it]
 10%|█         | 3982/37962 [9:16:07<78:46:04,  8.35s/it]
 10%|█         | 3983/37962 [9:16:16<78:50:30,  8.35s/it]
 10%|█         | 3984/37962 [9:16:24<78:12:59,  8.29s/it]
 10%|█         | 3985/37962 [9:16:32<78:16:15,  8.29s/it]
 10%|█         | 3986/37962 [9:16:40<78:47:49,  8.35s/it]
 11%|█         | 3987/37962 [9:16:48<77:47:01,  8.24s/it]
 11%|█         | 3988/37962 [9:16:57<78:12:25,  8.29s/it]
 11%|█         | 3989/37962 [9:17:05<77:20:47,  8.20s/it]
 11%|█         | 3990/37962 [9:17:13<77:29:25,  8.21s/it]
                                                         
{'loss': 1.5286, 'grad_norm': 29.19424057006836, 'learning_rate': 1.9705636753858813e-05, 'epoch': 0.11}

 11%|█         | 3990/37962 [9:17:13<77:29:25,  8.21s/it]
 11%|█         | 3991/37962 [9:17:21<77:16:29,  8.19s/it]
 11%|█         | 3992/37962 [9:17:29<77:26:28,  8.21s/it]
 11%|█         | 3993/37962 [9:17:38<77:26:39,  8.21s/it]
 11%|█         | 3994/37962 [9:17:46<77:41:18,  8.23s/it]
 11%|█         | 3995/37962 [9:17:54<76:58:18,  8.16s/it]
 11%|█         | 3996/37962 [9:18:02<77:23:19,  8.20s/it]
 11%|█         | 3997/37962 [9:18:11<78:07:28,  8.28s/it]
 11%|█         | 3998/37962 [9:18:19<77:08:52,  8.18s/it]
 11%|█         | 3999/37962 [9:18:27<77:31:30,  8.22s/it]
 11%|█         | 4000/37962 [9:18:35<77:48:44,  8.25s/it]
                                                         
{'loss': 1.4641, 'grad_norm': 29.660247802734375, 'learning_rate': 1.9703578430455966e-05, 'epoch': 0.11}

 11%|█         | 4000/37962 [9:18:35<77:48:44,  8.25s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 11%|█         | 4001/37962 [9:19:24<192:44:16, 20.43s/it]
 11%|█         | 4002/37962 [9:19:33<159:05:16, 16.86s/it]
 11%|█         | 4003/37962 [9:19:41<134:52:42, 14.30s/it]
 11%|█         | 4004/37962 [9:19:50<118:52:50, 12.60s/it]
 11%|█         | 4005/37962 [9:19:58<106:33:48, 11.30s/it]
 11%|█         | 4006/37962 [9:20:06<98:04:38, 10.40s/it] 
 11%|█         | 4007/37962 [9:20:14<91:49:08,  9.73s/it]
 11%|█         | 4008/37962 [9:20:23<88:26:04,  9.38s/it]
 11%|█         | 4009/37962 [9:20:31<85:16:21,  9.04s/it]
 11%|█         | 4010/37962 [9:20:40<83:23:22,  8.84s/it]
                                                         
{'loss': 1.5903, 'grad_norm': 9.799918174743652, 'learning_rate': 1.9701513043986482e-05, 'epoch': 0.11}

 11%|█         | 4010/37962 [9:20:40<83:23:22,  8.84s/it]
 11%|█         | 4011/37962 [9:20:48<81:08:17,  8.60s/it]
 11%|█         | 4012/37962 [9:20:56<80:00:42,  8.48s/it]
 11%|█         | 4013/37962 [9:21:04<80:02:49,  8.49s/it]
 11%|█         | 4014/37962 [9:21:12<78:44:27,  8.35s/it]
 11%|█         | 4015/37962 [9:21:21<78:25:28,  8.32s/it]
 11%|█         | 4016/37962 [9:21:29<78:04:20,  8.28s/it]
 11%|█         | 4017/37962 [9:21:37<78:09:58,  8.29s/it]
 11%|█         | 4018/37962 [9:21:45<77:45:56,  8.25s/it]
 11%|█         | 4019/37962 [9:21:53<77:34:03,  8.23s/it]
 11%|█         | 4020/37962 [9:22:02<77:44:44,  8.25s/it]
                                                         
{'loss': 1.4987, 'grad_norm': 11.159089088439941, 'learning_rate': 1.9699440595953722e-05, 'epoch': 0.11}

 11%|█         | 4020/37962 [9:22:02<77:44:44,  8.25s/it]
 11%|█         | 4021/37962 [9:22:10<77:57:58,  8.27s/it]
 11%|█         | 4022/37962 [9:22:18<78:16:31,  8.30s/it]
 11%|█         | 4023/37962 [9:22:27<77:54:03,  8.26s/it]
 11%|█         | 4024/37962 [9:22:35<78:04:40,  8.28s/it]
 11%|█         | 4025/37962 [9:22:43<77:58:46,  8.27s/it]
 11%|█         | 4026/37962 [9:22:51<77:52:17,  8.26s/it]
 11%|█         | 4027/37962 [9:23:00<77:42:07,  8.24s/it]
 11%|█         | 4028/37962 [9:23:08<77:34:55,  8.23s/it]
 11%|█         | 4029/37962 [9:23:16<76:40:50,  8.14s/it]
 11%|█         | 4030/37962 [9:23:24<76:24:51,  8.11s/it]
                                                         
{'loss': 1.5176, 'grad_norm': 11.761488914489746, 'learning_rate': 1.9697361087866182e-05, 'epoch': 0.11}

 11%|█         | 4030/37962 [9:23:24<76:24:51,  8.11s/it]
 11%|█         | 4031/37962 [9:23:32<77:20:41,  8.21s/it]
 11%|█         | 4032/37962 [9:23:41<78:24:02,  8.32s/it]
 11%|█         | 4033/37962 [9:23:49<78:37:07,  8.34s/it]
 11%|█         | 4034/37962 [9:23:57<78:13:45,  8.30s/it]
 11%|█         | 4035/37962 [9:24:06<78:06:32,  8.29s/it]
 11%|█         | 4036/37962 [9:24:14<78:19:24,  8.31s/it]
 11%|█         | 4037/37962 [9:24:22<77:45:23,  8.25s/it]
 11%|█         | 4038/37962 [9:24:31<78:10:52,  8.30s/it]
 11%|█         | 4039/37962 [9:24:39<77:54:22,  8.27s/it]
 11%|█         | 4040/37962 [9:24:47<78:19:52,  8.31s/it]
                                                         
{'loss': 1.5053, 'grad_norm': 3.4428839683532715, 'learning_rate': 1.9695274521237502e-05, 'epoch': 0.11}

 11%|█         | 4040/37962 [9:24:47<78:19:52,  8.31s/it]
 11%|█         | 4041/37962 [9:24:55<78:02:14,  8.28s/it]
 11%|█         | 4042/37962 [9:25:04<78:33:40,  8.34s/it]
 11%|█         | 4043/37962 [9:25:12<78:17:04,  8.31s/it]
 11%|█         | 4044/37962 [9:25:20<77:50:10,  8.26s/it]
 11%|█         | 4045/37962 [9:25:28<77:29:39,  8.23s/it]
 11%|█         | 4046/37962 [9:25:37<78:31:48,  8.34s/it]
 11%|█         | 4047/37962 [9:25:45<78:49:22,  8.37s/it]
 11%|█         | 4048/37962 [9:25:53<78:06:45,  8.29s/it]
 11%|█         | 4049/37962 [9:26:02<78:08:27,  8.29s/it]
 11%|█         | 4050/37962 [9:26:10<78:35:38,  8.34s/it]
                                                         
{'loss': 1.3169, 'grad_norm': 26.12874984741211, 'learning_rate': 1.9693180897586457e-05, 'epoch': 0.11}

 11%|█         | 4050/37962 [9:26:10<78:35:38,  8.34s/it]
 11%|█         | 4051/37962 [9:26:19<78:35:44,  8.34s/it]
 11%|█         | 4052/37962 [9:26:27<78:33:36,  8.34s/it]
 11%|█         | 4053/37962 [9:26:35<78:27:53,  8.33s/it]
 11%|█         | 4054/37962 [9:26:44<78:38:16,  8.35s/it]
 11%|█         | 4055/37962 [9:26:52<77:22:43,  8.22s/it]
 11%|█         | 4056/37962 [9:27:00<77:25:37,  8.22s/it]
 11%|█         | 4057/37962 [9:27:08<78:10:36,  8.30s/it]
 11%|█         | 4058/37962 [9:27:16<77:46:53,  8.26s/it]
 11%|█         | 4059/37962 [9:27:25<78:07:20,  8.30s/it]
 11%|█         | 4060/37962 [9:27:33<78:32:05,  8.34s/it]
                                                         
{'loss': 1.5412, 'grad_norm': 30.460630416870117, 'learning_rate': 1.969108021843696e-05, 'epoch': 0.11}

 11%|█         | 4060/37962 [9:27:33<78:32:05,  8.34s/it]
 11%|█         | 4061/37962 [9:27:41<77:20:48,  8.21s/it]
 11%|█         | 4062/37962 [9:27:50<77:57:03,  8.28s/it]
 11%|█         | 4063/37962 [9:27:58<77:55:54,  8.28s/it]
 11%|█         | 4064/37962 [9:28:06<78:31:44,  8.34s/it]
 11%|█         | 4065/37962 [9:28:15<78:13:00,  8.31s/it]
 11%|█         | 4066/37962 [9:28:22<76:40:00,  8.14s/it]
 11%|█         | 4067/37962 [9:28:31<77:05:40,  8.19s/it]
 11%|█         | 4068/37962 [9:28:39<77:26:54,  8.23s/it]
 11%|█         | 4069/37962 [9:28:47<77:41:58,  8.25s/it]
 11%|█         | 4070/37962 [9:28:56<78:00:57,  8.29s/it]
                                                         
{'loss': 1.5537, 'grad_norm': 14.724608421325684, 'learning_rate': 1.9688972485318058e-05, 'epoch': 0.11}

 11%|█         | 4070/37962 [9:28:56<78:00:57,  8.29s/it]
 11%|█         | 4071/37962 [9:29:04<78:02:34,  8.29s/it]
 11%|█         | 4072/37962 [9:29:12<77:54:22,  8.28s/it]
 11%|█         | 4073/37962 [9:29:20<77:55:28,  8.28s/it]
 11%|█         | 4074/37962 [9:29:29<77:34:21,  8.24s/it]
 11%|█         | 4075/37962 [9:29:37<78:08:09,  8.30s/it]
 11%|█         | 4076/37962 [9:29:45<77:48:19,  8.27s/it]
 11%|█         | 4077/37962 [9:29:53<77:19:13,  8.21s/it]
 11%|█         | 4078/37962 [9:30:02<77:55:34,  8.28s/it]
 11%|█         | 4079/37962 [9:30:10<77:36:41,  8.25s/it]
 11%|█         | 4080/37962 [9:30:18<77:12:41,  8.20s/it]
                                                         
{'loss': 1.4414, 'grad_norm': 14.210907936096191, 'learning_rate': 1.9686857699763932e-05, 'epoch': 0.11}

 11%|█         | 4080/37962 [9:30:18<77:12:41,  8.20s/it]
 11%|█         | 4081/37962 [9:30:26<76:28:41,  8.13s/it]
 11%|█         | 4082/37962 [9:30:34<77:06:30,  8.19s/it]
 11%|█         | 4083/37962 [9:30:43<77:06:33,  8.19s/it]
 11%|█         | 4084/37962 [9:30:51<77:22:22,  8.22s/it]
 11%|█         | 4085/37962 [9:30:59<77:17:30,  8.21s/it]
 11%|█         | 4086/37962 [9:31:08<78:12:26,  8.31s/it]
 11%|█         | 4087/37962 [9:31:16<78:04:52,  8.30s/it]
 11%|█         | 4088/37962 [9:31:24<78:13:26,  8.31s/it]
 11%|█         | 4089/37962 [9:31:32<78:14:32,  8.32s/it]
 11%|█         | 4090/37962 [9:31:41<78:46:17,  8.37s/it]
                                                         
{'loss': 1.4733, 'grad_norm': 11.68695068359375, 'learning_rate': 1.9684735863313904e-05, 'epoch': 0.11}

 11%|█         | 4090/37962 [9:31:41<78:46:17,  8.37s/it]
 11%|█         | 4091/37962 [9:31:49<78:33:56,  8.35s/it]
 11%|█         | 4092/37962 [9:31:58<78:48:08,  8.38s/it]
 11%|█         | 4093/37962 [9:32:06<78:14:05,  8.32s/it]
 11%|█         | 4094/37962 [9:32:14<77:53:19,  8.28s/it]
 11%|█         | 4095/37962 [9:32:22<77:42:21,  8.26s/it]
 11%|█         | 4096/37962 [9:32:30<77:07:46,  8.20s/it]
 11%|█         | 4097/37962 [9:32:39<77:53:18,  8.28s/it]
 11%|█         | 4098/37962 [9:32:47<77:57:53,  8.29s/it]
 11%|█         | 4099/37962 [9:32:55<77:25:17,  8.23s/it]
 11%|█         | 4100/37962 [9:33:04<77:33:09,  8.24s/it]
                                                         
{'loss': 1.6705, 'grad_norm': 12.959821701049805, 'learning_rate': 1.9682606977512416e-05, 'epoch': 0.11}

 11%|█         | 4100/37962 [9:33:04<77:33:09,  8.24s/it]
 11%|█         | 4101/37962 [9:33:12<77:51:56,  8.28s/it]
 11%|█         | 4102/37962 [9:33:20<76:48:49,  8.17s/it]
 11%|█         | 4103/37962 [9:33:28<76:58:36,  8.18s/it]
 11%|█         | 4104/37962 [9:33:36<77:11:20,  8.21s/it]
 11%|█         | 4105/37962 [9:33:44<76:33:17,  8.14s/it]
 11%|█         | 4106/37962 [9:33:53<77:20:53,  8.22s/it]
 11%|█         | 4107/37962 [9:34:01<77:57:08,  8.29s/it]
 11%|█         | 4108/37962 [9:34:09<77:45:17,  8.27s/it]
 11%|█         | 4109/37962 [9:34:18<78:04:33,  8.30s/it]
 11%|█         | 4110/37962 [9:34:26<78:14:33,  8.32s/it]
                                                         
{'loss': 1.3417, 'grad_norm': 17.853389739990234, 'learning_rate': 1.968047104390905e-05, 'epoch': 0.11}

 11%|█         | 4110/37962 [9:34:26<78:14:33,  8.32s/it]
 11%|█         | 4111/37962 [9:34:35<78:51:44,  8.39s/it]
 11%|█         | 4112/37962 [9:34:43<78:26:59,  8.34s/it]
 11%|█         | 4113/37962 [9:34:51<78:17:29,  8.33s/it]
 11%|█         | 4114/37962 [9:34:59<78:10:15,  8.31s/it]
 11%|█         | 4115/37962 [9:35:08<78:26:12,  8.34s/it]
 11%|█         | 4116/37962 [9:35:16<77:51:34,  8.28s/it]
 11%|█         | 4117/37962 [9:35:24<77:45:28,  8.27s/it]
 11%|█         | 4118/37962 [9:35:33<77:52:40,  8.28s/it]
 11%|█         | 4119/37962 [9:35:41<78:02:31,  8.30s/it]
 11%|█         | 4120/37962 [9:35:49<77:54:23,  8.29s/it]
                                                         
{'loss': 1.5744, 'grad_norm': 13.674656867980957, 'learning_rate': 1.967832806405851e-05, 'epoch': 0.11}

 11%|█         | 4120/37962 [9:35:49<77:54:23,  8.29s/it]
 11%|█         | 4121/37962 [9:35:57<77:11:38,  8.21s/it]
 11%|█         | 4122/37962 [9:36:05<77:29:16,  8.24s/it]
 11%|█         | 4123/37962 [9:36:14<77:39:12,  8.26s/it]
 11%|█         | 4124/37962 [9:36:22<77:51:48,  8.28s/it]
 11%|█         | 4125/37962 [9:36:30<77:47:19,  8.28s/it]
 11%|█         | 4126/37962 [9:36:39<78:03:39,  8.31s/it]
 11%|█         | 4127/37962 [9:36:47<77:18:42,  8.23s/it]
 11%|█         | 4128/37962 [9:36:55<77:18:06,  8.23s/it]
 11%|█         | 4129/37962 [9:37:03<77:25:33,  8.24s/it]
 11%|█         | 4130/37962 [9:37:11<77:19:50,  8.23s/it]
                                                         
{'loss': 1.5232, 'grad_norm': 36.04280090332031, 'learning_rate': 1.967617803952064e-05, 'epoch': 0.11}

 11%|█         | 4130/37962 [9:37:11<77:19:50,  8.23s/it]
 11%|█         | 4131/37962 [9:37:20<77:20:12,  8.23s/it]
 11%|█         | 4132/37962 [9:37:28<77:01:46,  8.20s/it]
 11%|█         | 4133/37962 [9:37:36<77:14:20,  8.22s/it]
 11%|█         | 4134/37962 [9:37:44<77:20:14,  8.23s/it]
 11%|█         | 4135/37962 [9:37:53<77:33:46,  8.25s/it]
 11%|█         | 4136/37962 [9:38:01<77:35:10,  8.26s/it]
 11%|█         | 4137/37962 [9:38:09<77:35:48,  8.26s/it]
 11%|█         | 4138/37962 [9:38:17<77:39:14,  8.26s/it]
 11%|█         | 4139/37962 [9:38:25<76:36:25,  8.15s/it]
 11%|█         | 4140/37962 [9:38:33<76:18:40,  8.12s/it]
                                                         
{'loss': 1.5108, 'grad_norm': 15.145776748657227, 'learning_rate': 1.9674020971860402e-05, 'epoch': 0.11}

 11%|█         | 4140/37962 [9:38:33<76:18:40,  8.12s/it]
 11%|█         | 4141/37962 [9:38:42<76:56:35,  8.19s/it]
 11%|█         | 4142/37962 [9:38:50<76:51:32,  8.18s/it]
 11%|█         | 4143/37962 [9:38:58<77:18:38,  8.23s/it]
 11%|█         | 4144/37962 [9:39:07<77:51:22,  8.29s/it]
 11%|█         | 4145/37962 [9:39:15<77:43:44,  8.27s/it]
 11%|█         | 4146/37962 [9:39:23<77:46:56,  8.28s/it]
 11%|█         | 4147/37962 [9:39:31<77:40:55,  8.27s/it]
 11%|█         | 4148/37962 [9:39:40<77:48:16,  8.28s/it]
 11%|█         | 4149/37962 [9:39:48<76:46:15,  8.17s/it]
 11%|█         | 4150/37962 [9:39:56<76:50:20,  8.18s/it]
                                                         
{'loss': 1.4892, 'grad_norm': 12.25979995727539, 'learning_rate': 1.967185686264789e-05, 'epoch': 0.11}

 11%|█         | 4150/37962 [9:39:56<76:50:20,  8.18s/it]
 11%|█         | 4151/37962 [9:40:04<77:14:04,  8.22s/it]
 11%|█         | 4152/37962 [9:40:13<77:31:41,  8.25s/it]
 11%|█         | 4153/37962 [9:40:21<77:37:51,  8.27s/it]
 11%|█         | 4154/37962 [9:40:29<77:28:50,  8.25s/it]
 11%|█         | 4155/37962 [9:40:37<77:25:34,  8.24s/it]
 11%|█         | 4156/37962 [9:40:46<77:38:33,  8.27s/it]
 11%|█         | 4157/37962 [9:40:54<78:05:06,  8.32s/it]
 11%|█         | 4158/37962 [9:41:02<77:48:12,  8.29s/it]
 11%|█         | 4159/37962 [9:41:10<77:29:53,  8.25s/it]
 11%|█         | 4160/37962 [9:41:19<77:47:29,  8.28s/it]
                                                         
{'loss': 1.4389, 'grad_norm': 44.1259651184082, 'learning_rate': 1.9669685713458323e-05, 'epoch': 0.11}

 11%|█         | 4160/37962 [9:41:19<77:47:29,  8.28s/it]
 11%|█         | 4161/37962 [9:41:27<78:00:43,  8.31s/it]
 11%|█         | 4162/37962 [9:41:36<78:29:32,  8.36s/it]
 11%|█         | 4163/37962 [9:41:44<77:07:17,  8.21s/it]
 11%|█         | 4164/37962 [9:41:52<77:10:15,  8.22s/it]
 11%|█         | 4165/37962 [9:42:00<77:19:34,  8.24s/it]
 11%|█         | 4166/37962 [9:42:08<77:07:43,  8.22s/it]
 11%|█         | 4167/37962 [9:42:16<77:15:43,  8.23s/it]
 11%|█         | 4168/37962 [9:42:25<77:39:00,  8.27s/it]
 11%|█         | 4169/37962 [9:42:33<77:49:31,  8.29s/it]
 11%|█         | 4170/37962 [9:42:42<78:12:47,  8.33s/it]
                                                         
{'loss': 1.5078, 'grad_norm': 38.08648681640625, 'learning_rate': 1.966750752587204e-05, 'epoch': 0.11}

 11%|█         | 4170/37962 [9:42:42<78:12:47,  8.33s/it]
 11%|█         | 4171/37962 [9:42:50<77:39:37,  8.27s/it]
 11%|█         | 4172/37962 [9:42:58<77:47:09,  8.29s/it]
 11%|█         | 4173/37962 [9:43:06<78:05:44,  8.32s/it]
 11%|█         | 4174/37962 [9:43:15<77:52:41,  8.30s/it]
 11%|█         | 4175/37962 [9:43:23<78:01:10,  8.31s/it]
 11%|█         | 4176/37962 [9:43:31<78:18:15,  8.34s/it]
 11%|█         | 4177/37962 [9:43:40<77:32:46,  8.26s/it]
 11%|█         | 4178/37962 [9:43:48<77:36:19,  8.27s/it]
 11%|█         | 4179/37962 [9:43:56<77:29:48,  8.26s/it]
 11%|█         | 4180/37962 [9:44:05<78:02:36,  8.32s/it]
                                                         
{'loss': 1.4254, 'grad_norm': 9.913771629333496, 'learning_rate': 1.9665322301474506e-05, 'epoch': 0.11}

 11%|█         | 4180/37962 [9:44:05<78:02:36,  8.32s/it]
 11%|█         | 4181/37962 [9:44:13<77:47:46,  8.29s/it]
 11%|█         | 4182/37962 [9:44:21<77:46:15,  8.29s/it]
 11%|█         | 4183/37962 [9:44:29<77:55:47,  8.31s/it]
 11%|█         | 4184/37962 [9:44:38<77:48:48,  8.29s/it]
 11%|█         | 4185/37962 [9:44:46<78:01:07,  8.32s/it]
 11%|█         | 4186/37962 [9:44:54<77:39:34,  8.28s/it]
 11%|█         | 4187/37962 [9:45:03<77:53:59,  8.30s/it]
 11%|█         | 4188/37962 [9:45:11<77:54:46,  8.30s/it]
 11%|█         | 4189/37962 [9:45:19<78:01:58,  8.32s/it]
 11%|█         | 4190/37962 [9:45:28<78:22:58,  8.36s/it]
                                                         
{'loss': 1.5044, 'grad_norm': 8.879205703735352, 'learning_rate': 1.966313004185631e-05, 'epoch': 0.11}

 11%|█         | 4190/37962 [9:45:28<78:22:58,  8.36s/it]
 11%|█         | 4191/37962 [9:45:36<78:23:59,  8.36s/it]
 11%|█         | 4192/37962 [9:45:44<78:03:19,  8.32s/it]
 11%|█         | 4193/37962 [9:45:53<78:02:10,  8.32s/it]
 11%|█         | 4194/37962 [9:46:01<77:50:57,  8.30s/it]
 11%|█         | 4195/37962 [9:46:09<77:50:17,  8.30s/it]
 11%|█         | 4196/37962 [9:46:17<77:01:27,  8.21s/it]
 11%|█         | 4197/37962 [9:46:25<77:05:49,  8.22s/it]
 11%|█         | 4198/37962 [9:46:34<77:48:55,  8.30s/it]
 11%|█         | 4199/37962 [9:46:42<77:43:18,  8.29s/it]
 11%|█         | 4200/37962 [9:46:50<76:46:26,  8.19s/it]
                                                         
{'loss': 1.5496, 'grad_norm': 5.761706352233887, 'learning_rate': 1.966093074861316e-05, 'epoch': 0.11}

 11%|█         | 4200/37962 [9:46:50<76:46:26,  8.19s/it]
 11%|█         | 4201/37962 [9:46:58<76:55:19,  8.20s/it]
 11%|█         | 4202/37962 [9:47:07<77:05:04,  8.22s/it]
 11%|█         | 4203/37962 [9:47:15<77:09:40,  8.23s/it]
 11%|█         | 4204/37962 [9:47:23<77:33:15,  8.27s/it]
 11%|█         | 4205/37962 [9:47:31<77:27:47,  8.26s/it]
 11%|█         | 4206/37962 [9:47:40<77:35:45,  8.28s/it]
 11%|█         | 4207/37962 [9:47:48<77:30:03,  8.27s/it]
 11%|█         | 4208/37962 [9:47:56<77:43:53,  8.29s/it]
 11%|█         | 4209/37962 [9:48:05<77:56:51,  8.31s/it]
 11%|█         | 4210/37962 [9:48:13<78:36:54,  8.39s/it]
                                                         
{'loss': 1.4568, 'grad_norm': 14.587227821350098, 'learning_rate': 1.965872442334588e-05, 'epoch': 0.11}

 11%|█         | 4210/37962 [9:48:13<78:36:54,  8.39s/it]
 11%|█         | 4211/37962 [9:48:21<77:10:43,  8.23s/it]
 11%|█         | 4212/37962 [9:48:29<77:16:47,  8.24s/it]
 11%|█         | 4213/37962 [9:48:38<77:54:03,  8.31s/it]
 11%|█         | 4214/37962 [9:48:46<77:48:01,  8.30s/it]
 11%|█         | 4215/37962 [9:48:54<76:43:40,  8.19s/it]
 11%|█         | 4216/37962 [9:49:02<76:52:29,  8.20s/it]
 11%|█         | 4217/37962 [9:49:11<77:30:51,  8.27s/it]
 11%|█         | 4218/37962 [9:49:19<76:32:46,  8.17s/it]
 11%|█         | 4219/37962 [9:49:27<76:30:17,  8.16s/it]
 11%|█         | 4220/37962 [9:49:35<77:33:17,  8.27s/it]
                                                         
{'loss': 1.3505, 'grad_norm': 5.551246166229248, 'learning_rate': 1.9656511067660417e-05, 'epoch': 0.11}

 11%|█         | 4220/37962 [9:49:35<77:33:17,  8.27s/it]
 11%|█         | 4221/37962 [9:49:43<77:11:52,  8.24s/it]
 11%|█         | 4222/37962 [9:49:52<77:23:58,  8.26s/it]
 11%|█         | 4223/37962 [9:50:00<76:49:43,  8.20s/it]
 11%|█         | 4224/37962 [9:50:08<77:06:53,  8.23s/it]
 11%|█         | 4225/37962 [9:50:16<76:23:11,  8.15s/it]
 11%|█         | 4226/37962 [9:50:24<76:50:18,  8.20s/it]
 11%|█         | 4227/37962 [9:50:33<77:02:39,  8.22s/it]
 11%|█         | 4228/37962 [9:50:41<76:59:22,  8.22s/it]
 11%|█         | 4229/37962 [9:50:49<76:54:12,  8.21s/it]
 11%|█         | 4230/37962 [9:50:57<76:56:58,  8.21s/it]
                                                         
{'loss': 1.5121, 'grad_norm': 3.064429998397827, 'learning_rate': 1.965429068316784e-05, 'epoch': 0.11}

 11%|█         | 4230/37962 [9:50:57<76:56:58,  8.21s/it]
 11%|█         | 4231/37962 [9:51:06<76:58:16,  8.21s/it]
 11%|█         | 4232/37962 [9:51:14<76:23:52,  8.15s/it]
 11%|█         | 4233/37962 [9:51:22<76:03:17,  8.12s/it]
 11%|█         | 4234/37962 [9:51:30<76:26:06,  8.16s/it]
 11%|█         | 4235/37962 [9:51:38<77:29:01,  8.27s/it]
 11%|█         | 4236/37962 [9:51:47<77:39:54,  8.29s/it]
 11%|█         | 4237/37962 [9:51:55<77:56:48,  8.32s/it]
 11%|█         | 4238/37962 [9:52:03<78:06:23,  8.34s/it]
 11%|█         | 4239/37962 [9:52:12<77:48:57,  8.31s/it]
 11%|█         | 4240/37962 [9:52:20<77:00:25,  8.22s/it]
                                                         
{'loss': 1.4931, 'grad_norm': 27.130346298217773, 'learning_rate': 1.965206327148432e-05, 'epoch': 0.11}

 11%|█         | 4240/37962 [9:52:20<77:00:25,  8.22s/it]
 11%|█         | 4241/37962 [9:52:28<77:08:33,  8.24s/it]
 11%|█         | 4242/37962 [9:52:36<77:12:46,  8.24s/it]
 11%|█         | 4243/37962 [9:52:44<76:40:57,  8.19s/it]
 11%|█         | 4244/37962 [9:52:53<77:26:26,  8.27s/it]
 11%|█         | 4245/37962 [9:53:01<77:50:25,  8.31s/it]
 11%|█         | 4246/37962 [9:53:09<77:02:40,  8.23s/it]
 11%|█         | 4247/37962 [9:53:17<77:00:44,  8.22s/it]
 11%|█         | 4248/37962 [9:53:26<77:21:10,  8.26s/it]
 11%|█         | 4249/37962 [9:53:34<77:25:39,  8.27s/it]
 11%|█         | 4250/37962 [9:53:42<77:16:47,  8.25s/it]
                                                         
{'loss': 1.4558, 'grad_norm': 19.038681030273438, 'learning_rate': 1.964982883423116e-05, 'epoch': 0.11}

 11%|█         | 4250/37962 [9:53:42<77:16:47,  8.25s/it]
 11%|█         | 4251/37962 [9:53:50<77:05:19,  8.23s/it]
 11%|█         | 4252/37962 [9:53:59<76:35:08,  8.18s/it]
 11%|█         | 4253/37962 [9:54:07<76:58:57,  8.22s/it]
 11%|█         | 4254/37962 [9:54:15<76:54:04,  8.21s/it]
 11%|█         | 4255/37962 [9:54:23<76:54:33,  8.21s/it]
 11%|█         | 4256/37962 [9:54:31<76:39:12,  8.19s/it]
 11%|█         | 4257/37962 [9:54:40<77:27:20,  8.27s/it]
 11%|█         | 4258/37962 [9:54:48<78:05:08,  8.34s/it]
 11%|█         | 4259/37962 [9:54:57<78:02:41,  8.34s/it]
 11%|█         | 4260/37962 [9:55:05<77:31:57,  8.28s/it]
                                                         
{'loss': 1.4834, 'grad_norm': 10.911199569702148, 'learning_rate': 1.9647587373034756e-05, 'epoch': 0.11}

 11%|█         | 4260/37962 [9:55:05<77:31:57,  8.28s/it]
 11%|█         | 4261/37962 [9:55:13<76:59:20,  8.22s/it]
 11%|█         | 4262/37962 [9:55:21<77:44:28,  8.30s/it]
 11%|█         | 4263/37962 [9:55:30<77:33:25,  8.29s/it]
 11%|█         | 4264/37962 [9:55:38<76:37:08,  8.19s/it]
 11%|█         | 4265/37962 [9:55:46<77:13:28,  8.25s/it]
 11%|█         | 4266/37962 [9:55:54<76:59:37,  8.23s/it]
 11%|█         | 4267/37962 [9:56:02<76:44:29,  8.20s/it]
 11%|█         | 4268/37962 [9:56:10<76:23:10,  8.16s/it]
 11%|█         | 4269/37962 [9:56:19<76:27:15,  8.17s/it]
 11%|█         | 4270/37962 [9:56:27<76:57:45,  8.22s/it]
                                                         
{'loss': 1.5514, 'grad_norm': 13.177024841308594, 'learning_rate': 1.964533888952664e-05, 'epoch': 0.11}

 11%|█         | 4270/37962 [9:56:27<76:57:45,  8.22s/it]
 11%|█▏        | 4271/37962 [9:56:35<76:05:34,  8.13s/it]
 11%|█▏        | 4272/37962 [9:56:43<75:57:26,  8.12s/it]
 11%|█▏        | 4273/37962 [9:56:51<75:32:47,  8.07s/it]
 11%|█▏        | 4274/37962 [9:56:59<76:27:43,  8.17s/it]
 11%|█▏        | 4275/37962 [9:57:08<76:50:17,  8.21s/it]
 11%|█▏        | 4276/37962 [9:57:16<76:32:11,  8.18s/it]
 11%|█▏        | 4277/37962 [9:57:24<76:44:26,  8.20s/it]
 11%|█▏        | 4278/37962 [9:57:32<76:37:55,  8.19s/it]
 11%|█▏        | 4279/37962 [9:57:40<76:32:39,  8.18s/it]
 11%|█▏        | 4280/37962 [9:57:48<76:08:20,  8.14s/it]
                                                         
{'loss': 1.5303, 'grad_norm': 20.846553802490234, 'learning_rate': 1.964308338534344e-05, 'epoch': 0.11}

 11%|█▏        | 4280/37962 [9:57:48<76:08:20,  8.14s/it]
 11%|█▏        | 4281/37962 [9:57:56<75:50:26,  8.11s/it]
 11%|█▏        | 4282/37962 [9:58:05<76:11:24,  8.14s/it]
 11%|█▏        | 4283/37962 [9:58:13<76:31:32,  8.18s/it]
 11%|█▏        | 4284/37962 [9:58:21<76:32:46,  8.18s/it]
 11%|█▏        | 4285/37962 [9:58:29<76:32:35,  8.18s/it]
 11%|█▏        | 4286/37962 [9:58:37<76:53:14,  8.22s/it]
 11%|█▏        | 4287/37962 [9:58:46<77:02:46,  8.24s/it]
 11%|█▏        | 4288/37962 [9:58:54<76:54:29,  8.22s/it]
 11%|█▏        | 4289/37962 [9:59:02<77:29:56,  8.29s/it]
 11%|█▏        | 4290/37962 [9:59:11<77:14:15,  8.26s/it]
                                                         
{'loss': 1.5342, 'grad_norm': 27.264062881469727, 'learning_rate': 1.96408208621269e-05, 'epoch': 0.11}

 11%|█▏        | 4290/37962 [9:59:11<77:14:15,  8.26s/it]
 11%|█▏        | 4291/37962 [9:59:19<76:57:46,  8.23s/it]
 11%|█▏        | 4292/37962 [9:59:27<76:56:38,  8.23s/it]
 11%|█▏        | 4293/37962 [9:59:35<76:45:42,  8.21s/it]
 11%|█▏        | 4294/37962 [9:59:43<77:04:43,  8.24s/it]
 11%|█▏        | 4295/37962 [9:59:52<77:13:51,  8.26s/it]
 11%|█▏        | 4296/37962 [10:00:00<77:26:26,  8.28s/it]
 11%|█▏        | 4297/37962 [10:00:08<77:34:42,  8.30s/it]
 11%|█▏        | 4298/37962 [10:00:17<77:21:39,  8.27s/it]
 11%|█▏        | 4299/37962 [10:00:25<77:30:16,  8.29s/it]
 11%|█▏        | 4300/37962 [10:00:33<77:34:32,  8.30s/it]
                                                          
{'loss': 1.3951, 'grad_norm': 29.97342300415039, 'learning_rate': 1.963855132152387e-05, 'epoch': 0.11}

 11%|█▏        | 4300/37962 [10:00:33<77:34:32,  8.30s/it]
 11%|█▏        | 4301/37962 [10:00:41<77:12:34,  8.26s/it]
 11%|█▏        | 4302/37962 [10:00:49<76:11:14,  8.15s/it]
 11%|█▏        | 4303/37962 [10:00:58<77:03:49,  8.24s/it]
 11%|█▏        | 4304/37962 [10:01:06<78:05:29,  8.35s/it]
 11%|█▏        | 4305/37962 [10:01:15<77:42:38,  8.31s/it]
 11%|█▏        | 4306/37962 [10:01:22<76:20:08,  8.17s/it]
 11%|█▏        | 4307/37962 [10:01:31<76:31:56,  8.19s/it]
 11%|█▏        | 4308/37962 [10:01:39<76:40:47,  8.20s/it]
 11%|█▏        | 4309/37962 [10:01:47<76:21:33,  8.17s/it]
 11%|█▏        | 4310/37962 [10:01:55<76:01:39,  8.13s/it]
                                                          
{'loss': 1.5764, 'grad_norm': 7.059381008148193, 'learning_rate': 1.9636274765186308e-05, 'epoch': 0.11}

 11%|█▏        | 4310/37962 [10:01:55<76:01:39,  8.13s/it]
 11%|█▏        | 4311/37962 [10:02:03<76:40:15,  8.20s/it]
 11%|█▏        | 4312/37962 [10:02:12<76:50:05,  8.22s/it]
 11%|█▏        | 4313/37962 [10:02:20<76:54:59,  8.23s/it]
 11%|█▏        | 4314/37962 [10:02:28<76:25:25,  8.18s/it]
 11%|█▏        | 4315/37962 [10:02:36<76:56:07,  8.23s/it]
 11%|█▏        | 4316/37962 [10:02:45<77:06:11,  8.25s/it]
 11%|█▏        | 4317/37962 [10:02:53<77:03:26,  8.25s/it]
 11%|█▏        | 4318/37962 [10:03:01<77:06:55,  8.25s/it]
 11%|█▏        | 4319/37962 [10:03:09<76:01:40,  8.14s/it]
 11%|█▏        | 4320/37962 [10:03:17<75:39:47,  8.10s/it]
                                                          
{'loss': 1.5224, 'grad_norm': 25.216533660888672, 'learning_rate': 1.9633991194771283e-05, 'epoch': 0.11}

 11%|█▏        | 4320/37962 [10:03:17<75:39:47,  8.10s/it]
 11%|█▏        | 4321/37962 [10:03:25<75:56:27,  8.13s/it]
 11%|█▏        | 4322/37962 [10:03:33<76:14:04,  8.16s/it]
 11%|█▏        | 4323/37962 [10:03:42<76:46:50,  8.22s/it]
 11%|█▏        | 4324/37962 [10:03:50<76:39:14,  8.20s/it]
 11%|█▏        | 4325/37962 [10:03:58<76:44:52,  8.21s/it]
 11%|█▏        | 4326/37962 [10:04:07<77:07:29,  8.25s/it]
 11%|█▏        | 4327/37962 [10:04:14<76:10:20,  8.15s/it]
 11%|█▏        | 4328/37962 [10:04:23<77:20:17,  8.28s/it]
 11%|█▏        | 4329/37962 [10:04:31<77:46:21,  8.32s/it]
 11%|█▏        | 4330/37962 [10:04:40<77:34:44,  8.30s/it]
                                                          
{'loss': 1.4854, 'grad_norm': 5.253264427185059, 'learning_rate': 1.9631700611940966e-05, 'epoch': 0.11}

 11%|█▏        | 4330/37962 [10:04:40<77:34:44,  8.30s/it]
 11%|█▏        | 4331/37962 [10:04:48<77:32:52,  8.30s/it]
 11%|█▏        | 4332/37962 [10:04:56<77:22:34,  8.28s/it]
 11%|█▏        | 4333/37962 [10:05:05<77:18:09,  8.28s/it]
 11%|█▏        | 4334/37962 [10:05:13<77:11:20,  8.26s/it]
 11%|█▏        | 4335/37962 [10:05:21<76:48:23,  8.22s/it]
 11%|█▏        | 4336/37962 [10:05:29<76:49:36,  8.23s/it]
 11%|█▏        | 4337/37962 [10:05:38<77:33:13,  8.30s/it]
 11%|█▏        | 4338/37962 [10:05:46<77:25:15,  8.29s/it]
 11%|█▏        | 4339/37962 [10:05:54<77:24:58,  8.29s/it]
 11%|█▏        | 4340/37962 [10:06:03<77:49:55,  8.33s/it]
                                                          
{'loss': 1.538, 'grad_norm': 16.352340698242188, 'learning_rate': 1.9629403018362628e-05, 'epoch': 0.11}

 11%|█▏        | 4340/37962 [10:06:03<77:49:55,  8.33s/it]
 11%|█▏        | 4341/37962 [10:06:11<78:15:55,  8.38s/it]
 11%|█▏        | 4342/37962 [10:06:19<78:00:51,  8.35s/it]
 11%|█▏        | 4343/37962 [10:06:28<77:33:36,  8.31s/it]
 11%|█▏        | 4344/37962 [10:06:35<76:20:31,  8.18s/it]
 11%|█▏        | 4345/37962 [10:06:44<76:53:39,  8.23s/it]
 11%|█▏        | 4346/37962 [10:06:52<76:32:50,  8.20s/it]
 11%|█▏        | 4347/37962 [10:07:00<76:29:44,  8.19s/it]
 11%|█▏        | 4348/37962 [10:07:08<76:17:48,  8.17s/it]
 11%|█▏        | 4349/37962 [10:07:16<75:43:18,  8.11s/it]
 11%|█▏        | 4350/37962 [10:07:25<76:21:46,  8.18s/it]
                                                          
{'loss': 1.5862, 'grad_norm': 26.118818283081055, 'learning_rate': 1.962709841570865e-05, 'epoch': 0.11}

 11%|█▏        | 4350/37962 [10:07:25<76:21:46,  8.18s/it]
 11%|█▏        | 4351/37962 [10:07:33<76:27:47,  8.19s/it]
 11%|█▏        | 4352/37962 [10:07:41<75:26:44,  8.08s/it]
 11%|█▏        | 4353/37962 [10:07:49<75:46:35,  8.12s/it]
 11%|█▏        | 4354/37962 [10:07:57<76:41:01,  8.21s/it]
 11%|█▏        | 4355/37962 [10:08:06<77:00:43,  8.25s/it]
 11%|█▏        | 4356/37962 [10:08:14<76:16:00,  8.17s/it]
 11%|█▏        | 4357/37962 [10:08:21<75:42:17,  8.11s/it]
 11%|█▏        | 4358/37962 [10:08:30<76:21:19,  8.18s/it]
 11%|█▏        | 4359/37962 [10:08:38<76:37:42,  8.21s/it]
 11%|█▏        | 4360/37962 [10:08:46<76:50:13,  8.23s/it]
                                                          
{'loss': 1.5165, 'grad_norm': 10.031881332397461, 'learning_rate': 1.9624786805656517e-05, 'epoch': 0.11}

 11%|█▏        | 4360/37962 [10:08:46<76:50:13,  8.23s/it]
 11%|█▏        | 4361/37962 [10:08:55<76:33:09,  8.20s/it]
 11%|█▏        | 4362/37962 [10:09:03<76:58:43,  8.25s/it]
 11%|█▏        | 4363/37962 [10:09:11<77:12:10,  8.27s/it]
 11%|█▏        | 4364/37962 [10:09:19<77:03:35,  8.26s/it]
 11%|█▏        | 4365/37962 [10:09:28<76:57:09,  8.25s/it]
 12%|█▏        | 4366/37962 [10:09:36<77:04:28,  8.26s/it]
 12%|█▏        | 4367/37962 [10:09:44<76:56:51,  8.25s/it]
 12%|█▏        | 4368/37962 [10:09:52<76:55:52,  8.24s/it]
 12%|█▏        | 4369/37962 [10:10:01<77:17:28,  8.28s/it]
 12%|█▏        | 4370/37962 [10:10:09<77:15:32,  8.28s/it]
                                                          
{'loss': 1.5422, 'grad_norm': 4.70387077331543, 'learning_rate': 1.9622468189888805e-05, 'epoch': 0.12}

 12%|█▏        | 4370/37962 [10:10:09<77:15:32,  8.28s/it]
 12%|█▏        | 4371/37962 [10:10:17<77:14:43,  8.28s/it]
 12%|█▏        | 4372/37962 [10:10:26<77:04:07,  8.26s/it]
 12%|█▏        | 4373/37962 [10:10:34<77:04:28,  8.26s/it]
 12%|█▏        | 4374/37962 [10:10:42<77:21:25,  8.29s/it]
 12%|█▏        | 4375/37962 [10:10:50<77:14:39,  8.28s/it]
 12%|█▏        | 4376/37962 [10:10:59<77:14:42,  8.28s/it]
 12%|█▏        | 4377/37962 [10:11:07<77:05:23,  8.26s/it]
 12%|█▏        | 4378/37962 [10:11:15<76:51:29,  8.24s/it]
 12%|█▏        | 4379/37962 [10:11:24<77:22:25,  8.29s/it]
 12%|█▏        | 4380/37962 [10:11:32<77:38:08,  8.32s/it]
                                                          
{'loss': 1.5215, 'grad_norm': 6.059713363647461, 'learning_rate': 1.9620142570093194e-05, 'epoch': 0.12}

 12%|█▏        | 4380/37962 [10:11:32<77:38:08,  8.32s/it]
 12%|█▏        | 4381/37962 [10:11:40<77:43:12,  8.33s/it]
 12%|█▏        | 4382/37962 [10:11:49<77:33:14,  8.31s/it]
 12%|█▏        | 4383/37962 [10:11:57<77:38:13,  8.32s/it]
 12%|█▏        | 4384/37962 [10:12:05<76:23:21,  8.19s/it]
 12%|█▏        | 4385/37962 [10:12:13<76:47:32,  8.23s/it]
 12%|█▏        | 4386/37962 [10:12:21<76:45:49,  8.23s/it]
 12%|█▏        | 4387/37962 [10:12:30<77:15:38,  8.28s/it]
 12%|█▏        | 4388/37962 [10:12:38<77:27:39,  8.31s/it]
 12%|█▏        | 4389/37962 [10:12:46<77:12:05,  8.28s/it]
 12%|█▏        | 4390/37962 [10:12:54<76:56:08,  8.25s/it]
                                                          
{'loss': 1.3604, 'grad_norm': 8.691885948181152, 'learning_rate': 1.9617809947962464e-05, 'epoch': 0.12}

 12%|█▏        | 4390/37962 [10:12:54<76:56:08,  8.25s/it]
 12%|█▏        | 4391/37962 [10:13:03<76:30:56,  8.21s/it]
 12%|█▏        | 4392/37962 [10:13:11<76:46:20,  8.23s/it]
 12%|█▏        | 4393/37962 [10:13:19<75:53:15,  8.14s/it]
 12%|█▏        | 4394/37962 [10:13:27<76:21:23,  8.19s/it]
 12%|█▏        | 4395/37962 [10:13:36<77:13:26,  8.28s/it]
 12%|█▏        | 4396/37962 [10:13:44<76:38:53,  8.22s/it]
 12%|█▏        | 4397/37962 [10:13:52<76:46:50,  8.24s/it]
 12%|█▏        | 4398/37962 [10:14:00<76:39:34,  8.22s/it]
 12%|█▏        | 4399/37962 [10:14:08<76:26:40,  8.20s/it]
 12%|█▏        | 4400/37962 [10:14:16<76:24:06,  8.20s/it]
                                                          
{'loss': 1.4971, 'grad_norm': 23.859235763549805, 'learning_rate': 1.9615470325194495e-05, 'epoch': 0.12}

 12%|█▏        | 4400/37962 [10:14:16<76:24:06,  8.20s/it]
 12%|█▏        | 4401/37962 [10:14:25<76:47:40,  8.24s/it]
 12%|█▏        | 4402/37962 [10:14:33<77:08:49,  8.28s/it]
 12%|█▏        | 4403/37962 [10:14:41<76:39:29,  8.22s/it]
 12%|█▏        | 4404/37962 [10:14:50<76:43:15,  8.23s/it]
 12%|█▏        | 4405/37962 [10:14:58<76:41:58,  8.23s/it]
 12%|█▏        | 4406/37962 [10:15:06<76:42:40,  8.23s/it]
 12%|█▏        | 4407/37962 [10:15:14<75:59:55,  8.15s/it]
 12%|█▏        | 4408/37962 [10:15:22<75:51:56,  8.14s/it]
 12%|█▏        | 4409/37962 [10:15:30<76:09:52,  8.17s/it]
 12%|█▏        | 4410/37962 [10:15:39<76:24:35,  8.20s/it]
                                                          
{'loss': 1.4667, 'grad_norm': 29.058679580688477, 'learning_rate': 1.9613123703492247e-05, 'epoch': 0.12}

 12%|█▏        | 4410/37962 [10:15:39<76:24:35,  8.20s/it]
 12%|█▏        | 4411/37962 [10:15:47<75:55:07,  8.15s/it]
 12%|█▏        | 4412/37962 [10:15:55<76:04:00,  8.16s/it]
 12%|█▏        | 4413/37962 [10:16:03<76:36:55,  8.22s/it]
 12%|█▏        | 4414/37962 [10:16:11<76:33:16,  8.21s/it]
 12%|█▏        | 4415/37962 [10:16:19<76:18:38,  8.19s/it]
 12%|█▏        | 4416/37962 [10:16:28<76:16:45,  8.19s/it]
 12%|█▏        | 4417/37962 [10:16:36<76:51:52,  8.25s/it]
 12%|█▏        | 4418/37962 [10:16:44<76:02:10,  8.16s/it]
 12%|█▏        | 4419/37962 [10:16:52<76:30:10,  8.21s/it]
 12%|█▏        | 4420/37962 [10:17:01<76:35:39,  8.22s/it]
                                                          
{'loss': 1.5282, 'grad_norm': 28.644695281982422, 'learning_rate': 1.9610770084563792e-05, 'epoch': 0.12}

 12%|█▏        | 4420/37962 [10:17:01<76:35:39,  8.22s/it]
 12%|█▏        | 4421/37962 [10:17:09<76:50:54,  8.25s/it]
 12%|█▏        | 4422/37962 [10:17:17<77:07:54,  8.28s/it]
 12%|█▏        | 4423/37962 [10:17:26<78:30:40,  8.43s/it]
 12%|█▏        | 4424/37962 [10:17:34<77:31:28,  8.32s/it]
 12%|█▏        | 4425/37962 [10:17:42<77:09:34,  8.28s/it]
 12%|█▏        | 4426/37962 [10:17:51<77:35:14,  8.33s/it]
 12%|█▏        | 4427/37962 [10:17:59<76:41:38,  8.23s/it]
 12%|█▏        | 4428/37962 [10:18:07<77:16:29,  8.30s/it]
 12%|█▏        | 4429/37962 [10:18:15<77:12:42,  8.29s/it]
 12%|█▏        | 4430/37962 [10:18:24<77:24:45,  8.31s/it]
                                                          
{'loss': 1.3776, 'grad_norm': 13.615556716918945, 'learning_rate': 1.9608409470122288e-05, 'epoch': 0.12}

 12%|█▏        | 4430/37962 [10:18:24<77:24:45,  8.31s/it]
 12%|█▏        | 4431/37962 [10:18:32<77:34:20,  8.33s/it]
 12%|█▏        | 4432/37962 [10:18:40<77:22:15,  8.31s/it]
 12%|█▏        | 4433/37962 [10:18:49<77:37:28,  8.33s/it]
 12%|█▏        | 4434/37962 [10:18:57<77:58:55,  8.37s/it]
 12%|█▏        | 4435/37962 [10:19:06<77:31:01,  8.32s/it]
 12%|█▏        | 4436/37962 [10:19:14<77:12:18,  8.29s/it]
 12%|█▏        | 4437/37962 [10:19:22<77:05:31,  8.28s/it]
 12%|█▏        | 4438/37962 [10:19:30<77:02:46,  8.27s/it]
 12%|█▏        | 4439/37962 [10:19:39<77:11:40,  8.29s/it]
 12%|█▏        | 4440/37962 [10:19:47<76:32:12,  8.22s/it]
                                                          
{'loss': 1.4184, 'grad_norm': 19.92193031311035, 'learning_rate': 1.960604186188599e-05, 'epoch': 0.12}

 12%|█▏        | 4440/37962 [10:19:47<76:32:12,  8.22s/it]
 12%|█▏        | 4441/37962 [10:19:55<76:45:35,  8.24s/it]
 12%|█▏        | 4442/37962 [10:20:03<76:55:10,  8.26s/it]
 12%|█▏        | 4443/37962 [10:20:11<76:54:59,  8.26s/it]
 12%|█▏        | 4444/37962 [10:20:20<76:23:00,  8.20s/it]
 12%|█▏        | 4445/37962 [10:20:28<76:53:13,  8.26s/it]
 12%|█▏        | 4446/37962 [10:20:36<77:01:13,  8.27s/it]
 12%|█▏        | 4447/37962 [10:20:44<76:53:49,  8.26s/it]
 12%|█▏        | 4448/37962 [10:20:53<77:01:30,  8.27s/it]
 12%|█▏        | 4449/37962 [10:21:01<77:06:15,  8.28s/it]
 12%|█▏        | 4450/37962 [10:21:09<76:59:19,  8.27s/it]
                                                          
{'loss': 1.5302, 'grad_norm': 9.075550079345703, 'learning_rate': 1.960366726157823e-05, 'epoch': 0.12}

 12%|█▏        | 4450/37962 [10:21:09<76:59:19,  8.27s/it]
 12%|█▏        | 4451/37962 [10:21:17<76:23:10,  8.21s/it]
 12%|█▏        | 4452/37962 [10:21:25<75:45:41,  8.14s/it]
 12%|█▏        | 4453/37962 [10:21:33<75:38:33,  8.13s/it]
 12%|█▏        | 4454/37962 [10:21:42<75:56:00,  8.16s/it]
 12%|█▏        | 4455/37962 [10:21:50<75:50:47,  8.15s/it]
 12%|█▏        | 4456/37962 [10:21:58<76:05:53,  8.18s/it]
 12%|█▏        | 4457/37962 [10:22:06<76:24:42,  8.21s/it]
 12%|█▏        | 4458/37962 [10:22:15<76:22:53,  8.21s/it]
 12%|█▏        | 4459/37962 [10:22:23<76:15:39,  8.19s/it]
 12%|█▏        | 4460/37962 [10:22:31<76:48:44,  8.25s/it]
                                                          
{'loss': 1.5333, 'grad_norm': 6.196775913238525, 'learning_rate': 1.9601285670927442e-05, 'epoch': 0.12}

 12%|█▏        | 4460/37962 [10:22:31<76:48:44,  8.25s/it]
 12%|█▏        | 4461/37962 [10:22:39<77:00:48,  8.28s/it]
 12%|█▏        | 4462/37962 [10:22:48<76:46:27,  8.25s/it]
 12%|█▏        | 4463/37962 [10:22:56<76:27:45,  8.22s/it]
 12%|█▏        | 4464/37962 [10:23:04<76:43:53,  8.25s/it]
 12%|█▏        | 4465/37962 [10:23:13<77:18:13,  8.31s/it]
 12%|█▏        | 4466/37962 [10:23:21<77:24:37,  8.32s/it]
 12%|█▏        | 4467/37962 [10:23:29<77:18:45,  8.31s/it]
 12%|█▏        | 4468/37962 [10:23:37<77:15:05,  8.30s/it]
 12%|█▏        | 4469/37962 [10:23:45<75:51:33,  8.15s/it]
 12%|█▏        | 4470/37962 [10:23:53<76:04:27,  8.18s/it]
                                                          
{'loss': 1.4136, 'grad_norm': 14.592398643493652, 'learning_rate': 1.9598897091667144e-05, 'epoch': 0.12}

 12%|█▏        | 4470/37962 [10:23:53<76:04:27,  8.18s/it]
 12%|█▏        | 4471/37962 [10:24:02<75:41:06,  8.14s/it]
 12%|█▏        | 4472/37962 [10:24:10<76:28:56,  8.22s/it]
 12%|█▏        | 4473/37962 [10:24:18<76:27:45,  8.22s/it]
 12%|█▏        | 4474/37962 [10:24:27<77:00:10,  8.28s/it]
 12%|█▏        | 4475/37962 [10:24:35<77:33:22,  8.34s/it]
 12%|█▏        | 4476/37962 [10:24:43<77:08:09,  8.29s/it]
 12%|█▏        | 4477/37962 [10:24:52<77:05:48,  8.29s/it]
 12%|█▏        | 4478/37962 [10:25:00<76:54:28,  8.27s/it]
 12%|█▏        | 4479/37962 [10:25:08<76:53:40,  8.27s/it]
 12%|█▏        | 4480/37962 [10:25:16<75:51:32,  8.16s/it]
                                                          
{'loss': 1.413, 'grad_norm': 15.705561637878418, 'learning_rate': 1.959650152553594e-05, 'epoch': 0.12}

 12%|█▏        | 4480/37962 [10:25:16<75:51:32,  8.16s/it]
 12%|█▏        | 4481/37962 [10:25:24<76:20:42,  8.21s/it]
 12%|█▏        | 4482/37962 [10:25:33<76:43:01,  8.25s/it]
 12%|█▏        | 4483/37962 [10:25:41<76:55:53,  8.27s/it]
 12%|█▏        | 4484/37962 [10:25:49<77:06:01,  8.29s/it]
 12%|█▏        | 4485/37962 [10:25:58<77:05:26,  8.29s/it]
 12%|█▏        | 4486/37962 [10:26:06<76:35:58,  8.24s/it]
 12%|█▏        | 4487/37962 [10:26:14<76:51:24,  8.27s/it]
 12%|█▏        | 4488/37962 [10:26:22<75:54:43,  8.16s/it]
 12%|█▏        | 4489/37962 [10:26:30<76:27:37,  8.22s/it]
 12%|█▏        | 4490/37962 [10:26:38<75:35:55,  8.13s/it]
                                                          
{'loss': 1.5398, 'grad_norm': 15.03614330291748, 'learning_rate': 1.959409897427753e-05, 'epoch': 0.12}

 12%|█▏        | 4490/37962 [10:26:38<75:35:55,  8.13s/it]
 12%|█▏        | 4491/37962 [10:26:46<75:40:12,  8.14s/it]
 12%|█▏        | 4492/37962 [10:26:54<75:35:51,  8.13s/it]
 12%|█▏        | 4493/37962 [10:27:03<76:12:27,  8.20s/it]
 12%|█▏        | 4494/37962 [10:27:11<76:27:01,  8.22s/it]
 12%|█▏        | 4495/37962 [10:27:19<76:21:14,  8.21s/it]
 12%|█▏        | 4496/37962 [10:27:28<76:37:03,  8.24s/it]
 12%|█▏        | 4497/37962 [10:27:36<76:57:16,  8.28s/it]
 12%|█▏        | 4498/37962 [10:27:44<75:19:37,  8.10s/it]
 12%|█▏        | 4499/37962 [10:27:52<75:49:56,  8.16s/it]
 12%|█▏        | 4500/37962 [10:28:00<76:19:06,  8.21s/it]
                                                          
{'loss': 1.4794, 'grad_norm': 4.167313575744629, 'learning_rate': 1.9591689439640678e-05, 'epoch': 0.12}

 12%|█▏        | 4500/37962 [10:28:00<76:19:06,  8.21s/it]
 12%|█▏        | 4501/37962 [10:28:08<76:20:19,  8.21s/it]
 12%|█▏        | 4502/37962 [10:28:17<76:41:56,  8.25s/it]
 12%|█▏        | 4503/37962 [10:28:25<77:02:38,  8.29s/it]
 12%|█▏        | 4504/37962 [10:28:33<76:56:59,  8.28s/it]
 12%|█▏        | 4505/37962 [10:28:41<75:41:11,  8.14s/it]
 12%|█▏        | 4506/37962 [10:28:50<76:23:39,  8.22s/it]
 12%|█▏        | 4507/37962 [10:28:58<76:31:21,  8.23s/it]
 12%|█▏        | 4508/37962 [10:29:06<76:31:47,  8.24s/it]
 12%|█▏        | 4509/37962 [10:29:14<76:27:47,  8.23s/it]
 12%|█▏        | 4510/37962 [10:29:23<76:36:24,  8.24s/it]
                                                          
{'loss': 1.3969, 'grad_norm': 19.23577308654785, 'learning_rate': 1.958927292337925e-05, 'epoch': 0.12}

 12%|█▏        | 4510/37962 [10:29:23<76:36:24,  8.24s/it]
 12%|█▏        | 4511/37962 [10:29:31<76:52:12,  8.27s/it]
 12%|█▏        | 4512/37962 [10:29:39<77:14:57,  8.31s/it]
 12%|█▏        | 4513/37962 [10:29:48<76:48:30,  8.27s/it]
 12%|█▏        | 4514/37962 [10:29:56<76:39:28,  8.25s/it]
 12%|█▏        | 4515/37962 [10:30:04<77:16:37,  8.32s/it]
 12%|█▏        | 4516/37962 [10:30:13<77:16:09,  8.32s/it]
 12%|█▏        | 4517/37962 [10:30:21<77:06:34,  8.30s/it]
 12%|█▏        | 4518/37962 [10:30:29<76:41:35,  8.26s/it]
 12%|█▏        | 4519/37962 [10:30:37<76:52:38,  8.28s/it]
 12%|█▏        | 4520/37962 [10:30:46<76:53:17,  8.28s/it]
                                                          
{'loss': 1.5522, 'grad_norm': 15.120834350585938, 'learning_rate': 1.9586849427252188e-05, 'epoch': 0.12}

 12%|█▏        | 4520/37962 [10:30:46<76:53:17,  8.28s/it]
 12%|█▏        | 4521/37962 [10:30:54<76:54:18,  8.28s/it]
 12%|█▏        | 4522/37962 [10:31:02<77:21:59,  8.33s/it]
 12%|█▏        | 4523/37962 [10:31:11<77:23:31,  8.33s/it]
 12%|█▏        | 4524/37962 [10:31:19<77:00:24,  8.29s/it]
 12%|█▏        | 4525/37962 [10:31:27<76:44:39,  8.26s/it]
 12%|█▏        | 4526/37962 [10:31:35<75:57:01,  8.18s/it]
 12%|█▏        | 4527/37962 [10:31:43<76:11:49,  8.20s/it]
 12%|█▏        | 4528/37962 [10:31:51<75:54:09,  8.17s/it]
 12%|█▏        | 4529/37962 [10:32:00<76:14:55,  8.21s/it]
 12%|█▏        | 4530/37962 [10:32:08<76:52:00,  8.28s/it]
                                                          
{'loss': 1.5639, 'grad_norm': 9.899072647094727, 'learning_rate': 1.9584418953023502e-05, 'epoch': 0.12}

 12%|█▏        | 4530/37962 [10:32:08<76:52:00,  8.28s/it]
 12%|█▏        | 4531/37962 [10:32:17<77:27:29,  8.34s/it]
 12%|█▏        | 4532/37962 [10:32:25<77:54:45,  8.39s/it]
 12%|█▏        | 4533/37962 [10:32:34<77:53:14,  8.39s/it]
 12%|█▏        | 4534/37962 [10:32:42<77:36:09,  8.36s/it]
 12%|█▏        | 4535/37962 [10:32:50<77:55:36,  8.39s/it]
 12%|█▏        | 4536/37962 [10:32:59<77:35:18,  8.36s/it]
 12%|█▏        | 4537/37962 [10:33:07<77:18:21,  8.33s/it]
 12%|█▏        | 4538/37962 [10:33:15<76:56:57,  8.29s/it]
 12%|█▏        | 4539/37962 [10:33:23<76:45:00,  8.27s/it]
 12%|█▏        | 4540/37962 [10:33:31<76:44:33,  8.27s/it]
                                                          
{'loss': 1.5244, 'grad_norm': 5.548903465270996, 'learning_rate': 1.95819815024623e-05, 'epoch': 0.12}

 12%|█▏        | 4540/37962 [10:33:31<76:44:33,  8.27s/it]
 12%|█▏        | 4541/37962 [10:33:40<77:03:35,  8.30s/it]
 12%|█▏        | 4542/37962 [10:33:48<76:25:32,  8.23s/it]
 12%|█▏        | 4543/37962 [10:33:56<75:21:29,  8.12s/it]
 12%|█▏        | 4544/37962 [10:34:04<76:16:13,  8.22s/it]
 12%|█▏        | 4545/37962 [10:34:12<76:20:57,  8.23s/it]
 12%|█▏        | 4546/37962 [10:34:21<75:57:16,  8.18s/it]
 12%|█▏        | 4547/37962 [10:34:29<76:43:57,  8.27s/it]
 12%|█▏        | 4548/37962 [10:34:37<77:01:27,  8.30s/it]
 12%|█▏        | 4549/37962 [10:34:46<76:57:35,  8.29s/it]
 12%|█▏        | 4550/37962 [10:34:54<76:47:27,  8.27s/it]
                                                          
{'loss': 1.3268, 'grad_norm': 14.200756072998047, 'learning_rate': 1.9579537077342764e-05, 'epoch': 0.12}

 12%|█▏        | 4550/37962 [10:34:54<76:47:27,  8.27s/it]
 12%|█▏        | 4551/37962 [10:35:02<77:16:32,  8.33s/it]
 12%|█▏        | 4552/37962 [10:35:11<77:24:39,  8.34s/it]
 12%|█▏        | 4553/37962 [10:35:19<77:17:34,  8.33s/it]
 12%|█▏        | 4554/37962 [10:35:27<77:09:44,  8.31s/it]
 12%|█▏        | 4555/37962 [10:35:35<76:42:34,  8.27s/it]
 12%|█▏        | 4556/37962 [10:35:44<77:00:54,  8.30s/it]
 12%|█▏        | 4557/37962 [10:35:52<76:58:41,  8.30s/it]
 12%|█▏        | 4558/37962 [10:36:00<76:44:17,  8.27s/it]
 12%|█▏        | 4559/37962 [10:36:09<76:56:37,  8.29s/it]
 12%|█▏        | 4560/37962 [10:36:17<76:51:45,  8.28s/it]
                                                          
{'loss': 1.5127, 'grad_norm': 11.563554763793945, 'learning_rate': 1.957708567944414e-05, 'epoch': 0.12}

 12%|█▏        | 4560/37962 [10:36:17<76:51:45,  8.28s/it]
 12%|█▏        | 4561/37962 [10:36:25<76:32:49,  8.25s/it]
 12%|█▏        | 4562/37962 [10:36:33<76:49:18,  8.28s/it]
 12%|█▏        | 4563/37962 [10:36:42<76:30:46,  8.25s/it]
 12%|█▏        | 4564/37962 [10:36:50<76:26:08,  8.24s/it]
 12%|█▏        | 4565/37962 [10:36:58<75:44:50,  8.17s/it]
 12%|█▏        | 4566/37962 [10:37:06<76:11:31,  8.21s/it]
 12%|█▏        | 4567/37962 [10:37:14<76:14:46,  8.22s/it]
 12%|█▏        | 4568/37962 [10:37:23<76:15:27,  8.22s/it]
 12%|█▏        | 4569/37962 [10:37:31<77:13:19,  8.33s/it]
 12%|█▏        | 4570/37962 [10:37:40<77:21:04,  8.34s/it]
                                                          
{'loss': 1.4877, 'grad_norm': 4.283222198486328, 'learning_rate': 1.9574627310550757e-05, 'epoch': 0.12}

 12%|█▏        | 4570/37962 [10:37:40<77:21:04,  8.34s/it]
 12%|█▏        | 4571/37962 [10:37:48<77:01:57,  8.31s/it]
 12%|█▏        | 4572/37962 [10:37:56<76:57:22,  8.30s/it]
 12%|█▏        | 4573/37962 [10:38:04<75:28:34,  8.14s/it]
 12%|█▏        | 4574/37962 [10:38:12<75:39:51,  8.16s/it]
 12%|█▏        | 4575/37962 [10:38:20<75:42:27,  8.16s/it]
 12%|█▏        | 4576/37962 [10:38:29<76:02:11,  8.20s/it]
 12%|█▏        | 4577/37962 [10:38:37<76:13:00,  8.22s/it]
 12%|█▏        | 4578/37962 [10:38:45<75:42:15,  8.16s/it]
 12%|█▏        | 4579/37962 [10:38:53<75:48:24,  8.17s/it]
 12%|█▏        | 4580/37962 [10:39:01<76:02:13,  8.20s/it]
                                                          
{'loss': 1.4381, 'grad_norm': 6.657790184020996, 'learning_rate': 1.957216197245203e-05, 'epoch': 0.12}

 12%|█▏        | 4580/37962 [10:39:01<76:02:13,  8.20s/it]
 12%|█▏        | 4581/37962 [10:39:10<76:25:45,  8.24s/it]
 12%|█▏        | 4582/37962 [10:39:18<76:22:33,  8.24s/it]
 12%|█▏        | 4583/37962 [10:39:26<76:08:50,  8.21s/it]
 12%|█▏        | 4584/37962 [10:39:34<76:44:59,  8.28s/it]
 12%|█▏        | 4585/37962 [10:39:43<76:33:23,  8.26s/it]
 12%|█▏        | 4586/37962 [10:39:51<76:34:32,  8.26s/it]
 12%|█▏        | 4587/37962 [10:39:59<76:30:57,  8.25s/it]
 12%|█▏        | 4588/37962 [10:40:07<76:26:31,  8.25s/it]
 12%|█▏        | 4589/37962 [10:40:15<75:36:55,  8.16s/it]
 12%|█▏        | 4590/37962 [10:40:24<75:50:31,  8.18s/it]
                                                          
{'loss': 1.4244, 'grad_norm': 17.211997985839844, 'learning_rate': 1.956968966694243e-05, 'epoch': 0.12}

 12%|█▏        | 4590/37962 [10:40:24<75:50:31,  8.18s/it]
 12%|█▏        | 4591/37962 [10:40:32<76:01:27,  8.20s/it]
 12%|█▏        | 4592/37962 [10:40:40<75:24:56,  8.14s/it]
 12%|█▏        | 4593/37962 [10:40:48<75:29:50,  8.15s/it]
 12%|█▏        | 4594/37962 [10:40:56<75:02:04,  8.10s/it]
 12%|█▏        | 4595/37962 [10:41:04<75:41:22,  8.17s/it]
 12%|█▏        | 4596/37962 [10:41:13<76:10:08,  8.22s/it]
 12%|█▏        | 4597/37962 [10:41:21<75:47:27,  8.18s/it]
 12%|█▏        | 4598/37962 [10:41:29<75:47:28,  8.18s/it]
 12%|█▏        | 4599/37962 [10:41:37<76:10:15,  8.22s/it]
 12%|█▏        | 4600/37962 [10:41:45<76:11:42,  8.22s/it]
                                                          
{'loss': 1.5535, 'grad_norm': 20.857580184936523, 'learning_rate': 1.95672103958215e-05, 'epoch': 0.12}

 12%|█▏        | 4600/37962 [10:41:45<76:11:42,  8.22s/it]
 12%|█▏        | 4601/37962 [10:41:54<77:18:10,  8.34s/it]
 12%|█▏        | 4602/37962 [10:42:02<76:57:31,  8.30s/it]
 12%|█▏        | 4603/37962 [10:42:11<77:08:19,  8.32s/it]
 12%|█▏        | 4604/37962 [10:42:19<76:46:37,  8.29s/it]
 12%|█▏        | 4605/37962 [10:42:27<76:36:40,  8.27s/it]
 12%|█▏        | 4606/37962 [10:42:35<76:45:32,  8.28s/it]
 12%|█▏        | 4607/37962 [10:42:44<76:31:41,  8.26s/it]
 12%|█▏        | 4608/37962 [10:42:52<76:43:12,  8.28s/it]
 12%|█▏        | 4609/37962 [10:43:00<75:55:55,  8.20s/it]
 12%|█▏        | 4610/37962 [10:43:08<76:02:22,  8.21s/it]
                                                          
{'loss': 1.5252, 'grad_norm': 23.938709259033203, 'learning_rate': 1.9564724160893866e-05, 'epoch': 0.12}

 12%|█▏        | 4610/37962 [10:43:08<76:02:22,  8.21s/it]
 12%|█▏        | 4611/37962 [10:43:16<76:02:36,  8.21s/it]
 12%|█▏        | 4612/37962 [10:43:25<76:02:23,  8.21s/it]
 12%|█▏        | 4613/37962 [10:43:33<76:16:13,  8.23s/it]
 12%|█▏        | 4614/37962 [10:43:41<76:11:47,  8.23s/it]
 12%|█▏        | 4615/37962 [10:43:49<76:26:57,  8.25s/it]
 12%|█▏        | 4616/37962 [10:43:57<74:59:15,  8.10s/it]
 12%|█▏        | 4617/37962 [10:44:06<75:55:12,  8.20s/it]
 12%|█▏        | 4618/37962 [10:44:14<76:02:33,  8.21s/it]
 12%|█▏        | 4619/37962 [10:44:22<75:39:45,  8.17s/it]
 12%|█▏        | 4620/37962 [10:44:30<76:43:39,  8.28s/it]
                                                          
{'loss': 1.4612, 'grad_norm': 20.435848236083984, 'learning_rate': 1.9562230963969213e-05, 'epoch': 0.12}

 12%|█▏        | 4620/37962 [10:44:30<76:43:39,  8.28s/it]
 12%|█▏        | 4621/37962 [10:44:39<76:58:42,  8.31s/it]
 12%|█▏        | 4622/37962 [10:44:47<77:13:50,  8.34s/it]
 12%|█▏        | 4623/37962 [10:44:55<76:59:32,  8.31s/it]
 12%|█▏        | 4624/37962 [10:45:04<77:01:45,  8.32s/it]
 12%|█▏        | 4625/37962 [10:45:12<77:33:27,  8.38s/it]
 12%|█▏        | 4626/37962 [10:45:20<76:37:59,  8.28s/it]
 12%|█▏        | 4627/37962 [10:45:29<76:44:33,  8.29s/it]
 12%|█▏        | 4628/37962 [10:45:37<76:44:14,  8.29s/it]
 12%|█▏        | 4629/37962 [10:45:45<75:53:33,  8.20s/it]
 12%|█▏        | 4630/37962 [10:45:53<75:51:55,  8.19s/it]
                                                          
{'loss': 1.4839, 'grad_norm': 25.573760986328125, 'learning_rate': 1.9559730806862295e-05, 'epoch': 0.12}

 12%|█▏        | 4630/37962 [10:45:53<75:51:55,  8.19s/it]
 12%|█▏        | 4631/37962 [10:46:01<76:02:26,  8.21s/it]
 12%|█▏        | 4632/37962 [10:46:10<76:13:45,  8.23s/it]
 12%|█▏        | 4633/37962 [10:46:18<76:52:00,  8.30s/it]
 12%|█▏        | 4634/37962 [10:46:26<76:46:28,  8.29s/it]
 12%|█▏        | 4635/37962 [10:46:35<76:44:14,  8.29s/it]
 12%|█▏        | 4636/37962 [10:46:43<76:30:45,  8.27s/it]
 12%|█▏        | 4637/37962 [10:46:51<76:51:56,  8.30s/it]
 12%|█▏        | 4638/37962 [10:47:00<76:50:37,  8.30s/it]
 12%|█▏        | 4639/37962 [10:47:08<76:50:03,  8.30s/it]
 12%|█▏        | 4640/37962 [10:47:16<76:16:33,  8.24s/it]
                                                          
{'loss': 1.428, 'grad_norm': 12.994621276855469, 'learning_rate': 1.9557223691392933e-05, 'epoch': 0.12}

 12%|█▏        | 4640/37962 [10:47:16<76:16:33,  8.24s/it]
 12%|█▏        | 4641/37962 [10:47:24<75:52:28,  8.20s/it]
 12%|█▏        | 4642/37962 [10:47:32<75:42:43,  8.18s/it]
 12%|█▏        | 4643/37962 [10:47:40<75:53:35,  8.20s/it]
 12%|█▏        | 4644/37962 [10:47:49<75:50:09,  8.19s/it]
 12%|█▏        | 4645/37962 [10:47:57<76:16:00,  8.24s/it]
 12%|█▏        | 4646/37962 [10:48:05<76:30:26,  8.27s/it]
 12%|█▏        | 4647/37962 [10:48:14<76:24:52,  8.26s/it]
 12%|█▏        | 4648/37962 [10:48:22<76:35:03,  8.28s/it]
 12%|█▏        | 4649/37962 [10:48:30<76:32:10,  8.27s/it]
 12%|█▏        | 4650/37962 [10:48:39<77:05:48,  8.33s/it]
                                                          
{'loss': 1.4504, 'grad_norm': 23.9560604095459, 'learning_rate': 1.9554709619386015e-05, 'epoch': 0.12}

 12%|█▏        | 4650/37962 [10:48:39<77:05:48,  8.33s/it]
 12%|█▏        | 4651/37962 [10:48:47<77:05:54,  8.33s/it]
 12%|█▏        | 4652/37962 [10:48:55<76:31:36,  8.27s/it]
 12%|█▏        | 4653/37962 [10:49:03<76:35:59,  8.28s/it]
 12%|█▏        | 4654/37962 [10:49:11<75:38:39,  8.18s/it]
 12%|█▏        | 4655/37962 [10:49:20<76:01:58,  8.22s/it]
 12%|█▏        | 4656/37962 [10:49:28<76:00:05,  8.21s/it]
 12%|█▏        | 4657/37962 [10:49:36<76:45:21,  8.30s/it]
 12%|█▏        | 4658/37962 [10:49:45<76:53:38,  8.31s/it]
 12%|█▏        | 4659/37962 [10:49:53<76:58:55,  8.32s/it]
 12%|█▏        | 4660/37962 [10:50:01<77:09:03,  8.34s/it]
                                                          
{'loss': 1.5798, 'grad_norm': 6.256830215454102, 'learning_rate': 1.9552188592671487e-05, 'epoch': 0.12}

 12%|█▏        | 4660/37962 [10:50:01<77:09:03,  8.34s/it]
 12%|█▏        | 4661/37962 [10:50:10<77:59:25,  8.43s/it]
 12%|█▏        | 4662/37962 [10:50:18<78:07:02,  8.45s/it]
 12%|█▏        | 4663/37962 [10:50:27<77:48:02,  8.41s/it]
 12%|█▏        | 4664/37962 [10:50:35<78:07:01,  8.45s/it]
 12%|█▏        | 4665/37962 [10:50:44<77:49:09,  8.41s/it]
 12%|█▏        | 4666/37962 [10:50:52<76:41:22,  8.29s/it]
 12%|█▏        | 4667/37962 [10:51:00<76:15:45,  8.25s/it]
 12%|█▏        | 4668/37962 [10:51:08<76:01:38,  8.22s/it]
 12%|█▏        | 4669/37962 [10:51:16<76:16:44,  8.25s/it]
 12%|█▏        | 4670/37962 [10:51:24<76:01:13,  8.22s/it]
                                                          
{'loss': 1.5442, 'grad_norm': 8.829842567443848, 'learning_rate': 1.9549660613084366e-05, 'epoch': 0.12}

 12%|█▏        | 4670/37962 [10:51:24<76:01:13,  8.22s/it]
 12%|█▏        | 4671/37962 [10:51:33<76:48:25,  8.31s/it]
 12%|█▏        | 4672/37962 [10:51:41<77:06:55,  8.34s/it]
 12%|█▏        | 4673/37962 [10:51:50<76:59:13,  8.33s/it]
 12%|█▏        | 4674/37962 [10:51:58<76:54:51,  8.32s/it]
 12%|█▏        | 4675/37962 [10:52:06<77:26:41,  8.38s/it]
 12%|█▏        | 4676/37962 [10:52:14<76:17:37,  8.25s/it]
 12%|█▏        | 4677/37962 [10:52:23<76:29:10,  8.27s/it]
 12%|█▏        | 4678/37962 [10:52:31<76:35:50,  8.28s/it]
 12%|█▏        | 4679/37962 [10:52:39<76:33:13,  8.28s/it]
 12%|█▏        | 4680/37962 [10:52:47<76:12:38,  8.24s/it]
                                                          
{'loss': 1.4395, 'grad_norm': 38.952789306640625, 'learning_rate': 1.954712568246472e-05, 'epoch': 0.12}

 12%|█▏        | 4680/37962 [10:52:47<76:12:38,  8.24s/it]
 12%|█▏        | 4681/37962 [10:52:56<76:01:42,  8.22s/it]
 12%|█▏        | 4682/37962 [10:53:04<75:54:09,  8.21s/it]
 12%|█▏        | 4683/37962 [10:53:12<75:53:46,  8.21s/it]
 12%|█▏        | 4684/37962 [10:53:20<76:21:11,  8.26s/it]
 12%|█▏        | 4685/37962 [10:53:29<76:10:38,  8.24s/it]
 12%|█▏        | 4686/37962 [10:53:37<75:15:20,  8.14s/it]
 12%|█▏        | 4687/37962 [10:53:45<74:52:38,  8.10s/it]
 12%|█▏        | 4688/37962 [10:53:53<75:16:36,  8.14s/it]
 12%|█▏        | 4689/37962 [10:54:01<75:35:10,  8.18s/it]
 12%|█▏        | 4690/37962 [10:54:09<76:00:09,  8.22s/it]
                                                          
{'loss': 1.3847, 'grad_norm': 16.142030715942383, 'learning_rate': 1.9544583802657683e-05, 'epoch': 0.12}

 12%|█▏        | 4690/37962 [10:54:09<76:00:09,  8.22s/it]
 12%|█▏        | 4691/37962 [10:54:18<76:11:44,  8.24s/it]
 12%|█▏        | 4692/37962 [10:54:26<76:41:43,  8.30s/it]
 12%|█▏        | 4693/37962 [10:54:34<76:27:43,  8.27s/it]
 12%|█▏        | 4694/37962 [10:54:43<76:25:42,  8.27s/it]
 12%|█▏        | 4695/37962 [10:54:51<76:29:25,  8.28s/it]
 12%|█▏        | 4696/37962 [10:54:59<75:28:23,  8.17s/it]
 12%|█▏        | 4697/37962 [10:55:07<75:40:41,  8.19s/it]
 12%|█▏        | 4698/37962 [10:55:15<74:57:16,  8.11s/it]
 12%|█▏        | 4699/37962 [10:55:23<74:49:36,  8.10s/it]
 12%|█▏        | 4700/37962 [10:55:31<75:36:51,  8.18s/it]
                                                          
{'loss': 1.3783, 'grad_norm': 8.3980712890625, 'learning_rate': 1.9542034975513442e-05, 'epoch': 0.12}

 12%|█▏        | 4700/37962 [10:55:31<75:36:51,  8.18s/it]
 12%|█▏        | 4701/37962 [10:55:40<76:33:30,  8.29s/it]
 12%|█▏        | 4702/37962 [10:55:48<76:24:04,  8.27s/it]
 12%|█▏        | 4703/37962 [10:55:56<75:22:50,  8.16s/it]
 12%|█▏        | 4704/37962 [10:56:04<75:17:52,  8.15s/it]
 12%|█▏        | 4705/37962 [10:56:13<75:50:44,  8.21s/it]
 12%|█▏        | 4706/37962 [10:56:20<74:56:02,  8.11s/it]
 12%|█▏        | 4707/37962 [10:56:29<75:24:56,  8.16s/it]
 12%|█▏        | 4708/37962 [10:56:37<75:41:35,  8.19s/it]
 12%|█▏        | 4709/37962 [10:56:45<75:56:54,  8.22s/it]
 12%|█▏        | 4710/37962 [10:56:54<76:07:02,  8.24s/it]
                                                          
{'loss': 1.5554, 'grad_norm': 30.560993194580078, 'learning_rate': 1.9539479202887248e-05, 'epoch': 0.12}

 12%|█▏        | 4710/37962 [10:56:54<76:07:02,  8.24s/it]
 12%|█▏        | 4711/37962 [10:57:02<76:19:38,  8.26s/it]
 12%|█▏        | 4712/37962 [10:57:10<76:44:18,  8.31s/it]
 12%|█▏        | 4713/37962 [10:57:19<76:34:13,  8.29s/it]
 12%|█▏        | 4714/37962 [10:57:26<75:15:41,  8.15s/it]
 12%|█▏        | 4715/37962 [10:57:35<75:38:27,  8.19s/it]
 12%|█▏        | 4716/37962 [10:57:43<74:52:57,  8.11s/it]
 12%|█▏        | 4717/37962 [10:57:51<75:32:12,  8.18s/it]
 12%|█▏        | 4718/37962 [10:57:59<74:44:16,  8.09s/it]
 12%|█▏        | 4719/37962 [10:58:07<75:08:43,  8.14s/it]
 12%|█▏        | 4720/37962 [10:58:15<75:25:49,  8.17s/it]
                                                          
{'loss': 1.5529, 'grad_norm': 23.749189376831055, 'learning_rate': 1.9536916486639405e-05, 'epoch': 0.12}

 12%|█▏        | 4720/37962 [10:58:15<75:25:49,  8.17s/it]
 12%|█▏        | 4721/37962 [10:58:23<75:30:50,  8.18s/it]
 12%|█▏        | 4722/37962 [10:58:32<76:22:00,  8.27s/it]
 12%|█▏        | 4723/37962 [10:58:40<76:29:03,  8.28s/it]
 12%|█▏        | 4724/37962 [10:58:48<76:15:23,  8.26s/it]
 12%|█▏        | 4725/37962 [10:58:57<76:01:45,  8.23s/it]
 12%|█▏        | 4726/37962 [10:59:05<76:16:14,  8.26s/it]
 12%|█▏        | 4727/37962 [10:59:13<76:19:10,  8.27s/it]
 12%|█▏        | 4728/37962 [10:59:21<76:14:01,  8.26s/it]
 12%|█▏        | 4729/37962 [10:59:29<75:30:43,  8.18s/it]
 12%|█▏        | 4730/37962 [10:59:38<76:37:38,  8.30s/it]
                                                          
{'loss': 1.5334, 'grad_norm': 24.92456817626953, 'learning_rate': 1.9534346828635266e-05, 'epoch': 0.12}

 12%|█▏        | 4730/37962 [10:59:38<76:37:38,  8.30s/it]
 12%|█▏        | 4731/37962 [10:59:46<75:55:42,  8.23s/it]
 12%|█▏        | 4732/37962 [10:59:54<75:32:08,  8.18s/it]
 12%|█▏        | 4733/37962 [11:00:02<75:42:03,  8.20s/it]
 12%|█▏        | 4734/37962 [11:00:11<75:49:48,  8.22s/it]
 12%|█▏        | 4735/37962 [11:00:19<75:57:57,  8.23s/it]
 12%|█▏        | 4736/37962 [11:00:27<75:51:09,  8.22s/it]
 12%|█▏        | 4737/37962 [11:00:35<76:09:51,  8.25s/it]
 12%|█▏        | 4738/37962 [11:00:44<75:49:20,  8.22s/it]
 12%|█▏        | 4739/37962 [11:00:52<75:43:57,  8.21s/it]
 12%|█▏        | 4740/37962 [11:01:00<76:04:29,  8.24s/it]
                                                          
{'loss': 1.5862, 'grad_norm': 12.458970069885254, 'learning_rate': 1.953177023074524e-05, 'epoch': 0.12}

 12%|█▏        | 4740/37962 [11:01:00<76:04:29,  8.24s/it]
 12%|█▏        | 4741/37962 [11:01:08<76:16:33,  8.27s/it]
 12%|█▏        | 4742/37962 [11:01:16<75:19:06,  8.16s/it]
 12%|█▏        | 4743/37962 [11:01:25<76:03:22,  8.24s/it]
 12%|█▏        | 4744/37962 [11:01:33<75:48:35,  8.22s/it]
 12%|█▏        | 4745/37962 [11:01:41<75:48:38,  8.22s/it]
 13%|█▎        | 4746/37962 [11:01:50<76:11:41,  8.26s/it]
 13%|█▎        | 4747/37962 [11:01:58<75:31:55,  8.19s/it]
 13%|█▎        | 4748/37962 [11:02:06<76:01:29,  8.24s/it]
 13%|█▎        | 4749/37962 [11:02:14<75:16:30,  8.16s/it]
 13%|█▎        | 4750/37962 [11:02:22<75:19:48,  8.17s/it]
                                                          
{'loss': 1.5699, 'grad_norm': 16.200027465820312, 'learning_rate': 1.9529186694844787e-05, 'epoch': 0.13}

 13%|█▎        | 4750/37962 [11:02:22<75:19:48,  8.17s/it]
 13%|█▎        | 4751/37962 [11:02:30<75:29:04,  8.18s/it]
 13%|█▎        | 4752/37962 [11:02:39<76:51:14,  8.33s/it]
 13%|█▎        | 4753/37962 [11:02:47<76:44:52,  8.32s/it]
 13%|█▎        | 4754/37962 [11:02:55<75:26:43,  8.18s/it]
 13%|█▎        | 4755/37962 [11:03:03<75:37:55,  8.20s/it]
 13%|█▎        | 4756/37962 [11:03:11<74:41:45,  8.10s/it]
 13%|█▎        | 4757/37962 [11:03:19<74:45:08,  8.10s/it]
 13%|█▎        | 4758/37962 [11:03:28<75:13:31,  8.16s/it]
 13%|█▎        | 4759/37962 [11:03:36<75:11:02,  8.15s/it]
 13%|█▎        | 4760/37962 [11:03:44<75:52:40,  8.23s/it]
                                                          
{'loss': 1.3821, 'grad_norm': 20.594797134399414, 'learning_rate': 1.9526596222814425e-05, 'epoch': 0.13}

 13%|█▎        | 4760/37962 [11:03:44<75:52:40,  8.23s/it]
 13%|█▎        | 4761/37962 [11:03:52<75:42:45,  8.21s/it]
 13%|█▎        | 4762/37962 [11:04:01<75:43:21,  8.21s/it]
 13%|█▎        | 4763/37962 [11:04:09<75:41:59,  8.21s/it]
 13%|█▎        | 4764/37962 [11:04:17<75:57:46,  8.24s/it]
 13%|█▎        | 4765/37962 [11:04:25<75:55:52,  8.23s/it]
 13%|█▎        | 4766/37962 [11:04:34<76:05:18,  8.25s/it]
 13%|█▎        | 4767/37962 [11:04:42<75:57:13,  8.24s/it]
 13%|█▎        | 4768/37962 [11:04:50<75:55:49,  8.23s/it]
 13%|█▎        | 4769/37962 [11:04:58<76:28:52,  8.29s/it]
 13%|█▎        | 4770/37962 [11:05:07<76:01:20,  8.25s/it]
                                                          
{'loss': 1.5564, 'grad_norm': 8.395151138305664, 'learning_rate': 1.9523998816539706e-05, 'epoch': 0.13}

 13%|█▎        | 4770/37962 [11:05:07<76:01:20,  8.25s/it]
 13%|█▎        | 4771/37962 [11:05:15<76:17:02,  8.27s/it]
 13%|█▎        | 4772/37962 [11:05:23<76:32:18,  8.30s/it]
 13%|█▎        | 4773/37962 [11:05:32<76:59:31,  8.35s/it]
 13%|█▎        | 4774/37962 [11:05:40<76:55:05,  8.34s/it]
 13%|█▎        | 4775/37962 [11:05:48<76:30:26,  8.30s/it]
 13%|█▎        | 4776/37962 [11:05:56<74:36:15,  8.09s/it]
 13%|█▎        | 4777/37962 [11:06:05<76:07:10,  8.26s/it]
 13%|█▎        | 4778/37962 [11:06:13<76:20:54,  8.28s/it]
 13%|█▎        | 4779/37962 [11:06:21<76:12:58,  8.27s/it]
 13%|█▎        | 4780/37962 [11:06:29<75:56:21,  8.24s/it]
                                                          
{'loss': 1.5579, 'grad_norm': 10.896271705627441, 'learning_rate': 1.9521394477911245e-05, 'epoch': 0.13}

 13%|█▎        | 4780/37962 [11:06:29<75:56:21,  8.24s/it]
 13%|█▎        | 4781/37962 [11:06:38<76:17:58,  8.28s/it]
 13%|█▎        | 4782/37962 [11:06:46<75:20:39,  8.17s/it]
 13%|█▎        | 4783/37962 [11:06:54<75:52:05,  8.23s/it]
 13%|█▎        | 4784/37962 [11:07:02<75:31:09,  8.19s/it]
 13%|█▎        | 4785/37962 [11:07:10<75:49:59,  8.23s/it]
 13%|█▎        | 4786/37962 [11:07:19<75:39:07,  8.21s/it]
 13%|█▎        | 4787/37962 [11:07:27<75:37:24,  8.21s/it]
 13%|█▎        | 4788/37962 [11:07:35<75:34:09,  8.20s/it]
 13%|█▎        | 4789/37962 [11:07:43<75:29:49,  8.19s/it]
 13%|█▎        | 4790/37962 [11:07:51<75:45:12,  8.22s/it]
                                                          
{'loss': 1.5197, 'grad_norm': 6.0535993576049805, 'learning_rate': 1.9518783208824688e-05, 'epoch': 0.13}

 13%|█▎        | 4790/37962 [11:07:51<75:45:12,  8.22s/it]
 13%|█▎        | 4791/37962 [11:08:00<76:03:53,  8.26s/it]
 13%|█▎        | 4792/37962 [11:08:08<76:18:40,  8.28s/it]
 13%|█▎        | 4793/37962 [11:08:16<76:24:06,  8.29s/it]
 13%|█▎        | 4794/37962 [11:08:25<77:05:37,  8.37s/it]
 13%|█▎        | 4795/37962 [11:08:33<77:05:09,  8.37s/it]
 13%|█▎        | 4796/37962 [11:08:41<76:29:56,  8.30s/it]
 13%|█▎        | 4797/37962 [11:08:50<76:20:25,  8.29s/it]
 13%|█▎        | 4798/37962 [11:08:58<76:25:21,  8.30s/it]
 13%|█▎        | 4799/37962 [11:09:06<76:47:37,  8.34s/it]
 13%|█▎        | 4800/37962 [11:09:15<76:38:16,  8.32s/it]
                                                          
{'loss': 1.4492, 'grad_norm': 7.617906093597412, 'learning_rate': 1.9516165011180735e-05, 'epoch': 0.13}

 13%|█▎        | 4800/37962 [11:09:15<76:38:16,  8.32s/it]
 13%|█▎        | 4801/37962 [11:09:23<75:58:22,  8.25s/it]
 13%|█▎        | 4802/37962 [11:09:31<76:09:41,  8.27s/it]
 13%|█▎        | 4803/37962 [11:09:39<76:08:25,  8.27s/it]
 13%|█▎        | 4804/37962 [11:09:48<76:32:20,  8.31s/it]
 13%|█▎        | 4805/37962 [11:09:56<76:30:42,  8.31s/it]
 13%|█▎        | 4806/37962 [11:10:04<76:27:31,  8.30s/it]
 13%|█▎        | 4807/37962 [11:10:13<76:45:21,  8.33s/it]
 13%|█▎        | 4808/37962 [11:10:21<76:09:12,  8.27s/it]
 13%|█▎        | 4809/37962 [11:10:29<75:45:18,  8.23s/it]
 13%|█▎        | 4810/37962 [11:10:37<75:03:43,  8.15s/it]
                                                          
{'loss': 1.4218, 'grad_norm': 4.157278060913086, 'learning_rate': 1.9513539886885128e-05, 'epoch': 0.13}

 13%|█▎        | 4810/37962 [11:10:37<75:03:43,  8.15s/it]
 13%|█▎        | 4811/37962 [11:10:45<75:27:45,  8.19s/it]
 13%|█▎        | 4812/37962 [11:10:53<75:27:21,  8.19s/it]
 13%|█▎        | 4813/37962 [11:11:02<75:49:38,  8.23s/it]
 13%|█▎        | 4814/37962 [11:11:10<75:57:30,  8.25s/it]
 13%|█▎        | 4815/37962 [11:11:18<75:58:02,  8.25s/it]
 13%|█▎        | 4816/37962 [11:11:27<75:50:43,  8.24s/it]
 13%|█▎        | 4817/37962 [11:11:35<76:04:17,  8.26s/it]
 13%|█▎        | 4818/37962 [11:11:43<76:35:22,  8.32s/it]
 13%|█▎        | 4819/37962 [11:11:52<76:48:00,  8.34s/it]
 13%|█▎        | 4820/37962 [11:12:00<76:55:56,  8.36s/it]
                                                          
{'loss': 1.5949, 'grad_norm': 17.745332717895508, 'learning_rate': 1.9510907837848646e-05, 'epoch': 0.13}

 13%|█▎        | 4820/37962 [11:12:00<76:55:56,  8.36s/it]
 13%|█▎        | 4821/37962 [11:12:08<76:34:17,  8.32s/it]
 13%|█▎        | 4822/37962 [11:12:17<76:53:20,  8.35s/it]
 13%|█▎        | 4823/37962 [11:12:25<76:43:36,  8.34s/it]
 13%|█▎        | 4824/37962 [11:12:33<76:37:41,  8.32s/it]
 13%|█▎        | 4825/37962 [11:12:41<75:13:23,  8.17s/it]
 13%|█▎        | 4826/37962 [11:12:49<74:59:08,  8.15s/it]
 13%|█▎        | 4827/37962 [11:12:57<75:06:07,  8.16s/it]
 13%|█▎        | 4828/37962 [11:13:06<75:35:40,  8.21s/it]
 13%|█▎        | 4829/37962 [11:13:14<75:08:46,  8.16s/it]
 13%|█▎        | 4830/37962 [11:13:22<75:37:41,  8.22s/it]
                                                          
{'loss': 1.5182, 'grad_norm': 35.33827209472656, 'learning_rate': 1.9508268865987112e-05, 'epoch': 0.13}

 13%|█▎        | 4830/37962 [11:13:22<75:37:41,  8.22s/it]
 13%|█▎        | 4831/37962 [11:13:30<75:53:42,  8.25s/it]
 13%|█▎        | 4832/37962 [11:13:39<76:21:00,  8.30s/it]
 13%|█▎        | 4833/37962 [11:13:47<76:33:23,  8.32s/it]
 13%|█▎        | 4834/37962 [11:13:55<75:23:36,  8.19s/it]
 13%|█▎        | 4835/37962 [11:14:03<74:45:32,  8.12s/it]
 13%|█▎        | 4836/37962 [11:14:11<75:18:56,  8.19s/it]
 13%|█▎        | 4837/37962 [11:14:20<75:15:29,  8.18s/it]
 13%|█▎        | 4838/37962 [11:14:28<75:56:13,  8.25s/it]
 13%|█▎        | 4839/37962 [11:14:36<74:51:10,  8.14s/it]
 13%|█▎        | 4840/37962 [11:14:44<75:14:53,  8.18s/it]
                                                          
{'loss': 1.5437, 'grad_norm': 32.935089111328125, 'learning_rate': 1.9505622973221394e-05, 'epoch': 0.13}

 13%|█▎        | 4840/37962 [11:14:44<75:14:53,  8.18s/it]
 13%|█▎        | 4841/37962 [11:14:52<75:19:30,  8.19s/it]
 13%|█▎        | 4842/37962 [11:15:01<75:51:49,  8.25s/it]
 13%|█▎        | 4843/37962 [11:15:09<76:18:03,  8.29s/it]
 13%|█▎        | 4844/37962 [11:15:17<75:10:47,  8.17s/it]
 13%|█▎        | 4845/37962 [11:15:25<75:28:40,  8.20s/it]
 13%|█▎        | 4846/37962 [11:15:33<75:07:44,  8.17s/it]
 13%|█▎        | 4847/37962 [11:15:42<75:07:43,  8.17s/it]
 13%|█▎        | 4848/37962 [11:15:50<76:11:00,  8.28s/it]
 13%|█▎        | 4849/37962 [11:15:58<76:05:54,  8.27s/it]
 13%|█▎        | 4850/37962 [11:16:07<75:50:29,  8.25s/it]
                                                          
{'loss': 1.5995, 'grad_norm': 17.70271873474121, 'learning_rate': 1.950297016147739e-05, 'epoch': 0.13}

 13%|█▎        | 4850/37962 [11:16:07<75:50:29,  8.25s/it]
 13%|█▎        | 4851/37962 [11:16:15<75:45:35,  8.24s/it]
 13%|█▎        | 4852/37962 [11:16:23<75:40:46,  8.23s/it]
 13%|█▎        | 4853/37962 [11:16:31<75:46:22,  8.24s/it]
 13%|█▎        | 4854/37962 [11:16:40<75:51:59,  8.25s/it]
 13%|█▎        | 4855/37962 [11:16:48<75:52:50,  8.25s/it]
 13%|█▎        | 4856/37962 [11:16:56<76:01:04,  8.27s/it]
 13%|█▎        | 4857/37962 [11:17:04<76:09:24,  8.28s/it]
 13%|█▎        | 4858/37962 [11:17:13<76:13:11,  8.29s/it]
 13%|█▎        | 4859/37962 [11:17:21<75:29:07,  8.21s/it]
 13%|█▎        | 4860/37962 [11:17:29<75:05:35,  8.17s/it]
                                                          
{'loss': 1.4685, 'grad_norm': 16.751222610473633, 'learning_rate': 1.9500310432686026e-05, 'epoch': 0.13}

 13%|█▎        | 4860/37962 [11:17:29<75:05:35,  8.17s/it]
 13%|█▎        | 4861/37962 [11:17:37<75:57:22,  8.26s/it]
 13%|█▎        | 4862/37962 [11:17:46<76:17:46,  8.30s/it]
 13%|█▎        | 4863/37962 [11:17:54<75:10:52,  8.18s/it]
 13%|█▎        | 4864/37962 [11:18:02<75:29:28,  8.21s/it]
 13%|█▎        | 4865/37962 [11:18:10<75:32:32,  8.22s/it]
 13%|█▎        | 4866/37962 [11:18:18<75:49:20,  8.25s/it]
 13%|█▎        | 4867/37962 [11:18:27<75:58:34,  8.26s/it]
 13%|█▎        | 4868/37962 [11:18:35<75:41:14,  8.23s/it]
 13%|█▎        | 4869/37962 [11:18:43<76:14:35,  8.29s/it]
 13%|█▎        | 4870/37962 [11:18:51<75:52:41,  8.25s/it]
                                                          
{'loss': 1.5412, 'grad_norm': 11.15281867980957, 'learning_rate': 1.9497643788783283e-05, 'epoch': 0.13}

 13%|█▎        | 4870/37962 [11:18:51<75:52:41,  8.25s/it]
 13%|█▎        | 4871/37962 [11:19:00<75:54:05,  8.26s/it]
 13%|█▎        | 4872/37962 [11:19:08<75:21:15,  8.20s/it]
 13%|█▎        | 4873/37962 [11:19:16<75:29:20,  8.21s/it]
 13%|█▎        | 4874/37962 [11:19:24<75:30:16,  8.21s/it]
 13%|█▎        | 4875/37962 [11:19:32<74:34:49,  8.11s/it]
 13%|█▎        | 4876/37962 [11:19:41<75:11:16,  8.18s/it]
 13%|█▎        | 4877/37962 [11:19:49<75:34:19,  8.22s/it]
 13%|█▎        | 4878/37962 [11:19:57<75:46:01,  8.24s/it]
 13%|█▎        | 4879/37962 [11:20:05<75:40:16,  8.23s/it]
 13%|█▎        | 4880/37962 [11:20:14<75:30:47,  8.22s/it]
                                                          
{'loss': 1.4748, 'grad_norm': 10.11408805847168, 'learning_rate': 1.949497023171016e-05, 'epoch': 0.13}

 13%|█▎        | 4880/37962 [11:20:14<75:30:47,  8.22s/it]
 13%|█▎        | 4881/37962 [11:20:22<75:41:48,  8.24s/it]
 13%|█▎        | 4882/37962 [11:20:30<75:52:59,  8.26s/it]
 13%|█▎        | 4883/37962 [11:20:38<75:55:13,  8.26s/it]
 13%|█▎        | 4884/37962 [11:20:47<76:03:19,  8.28s/it]
 13%|█▎        | 4885/37962 [11:20:55<76:10:58,  8.29s/it]
 13%|█▎        | 4886/37962 [11:21:03<75:24:24,  8.21s/it]
 13%|█▎        | 4887/37962 [11:21:11<75:29:05,  8.22s/it]
 13%|█▎        | 4888/37962 [11:21:19<74:04:20,  8.06s/it]
 13%|█▎        | 4889/37962 [11:21:27<73:52:12,  8.04s/it]
 13%|█▎        | 4890/37962 [11:21:35<74:36:30,  8.12s/it]
                                                          
{'loss': 1.5481, 'grad_norm': 5.804219722747803, 'learning_rate': 1.94922897634127e-05, 'epoch': 0.13}

 13%|█▎        | 4890/37962 [11:21:35<74:36:30,  8.12s/it]
 13%|█▎        | 4891/37962 [11:21:44<75:04:49,  8.17s/it]
 13%|█▎        | 4892/37962 [11:21:52<75:34:14,  8.23s/it]
 13%|█▎        | 4893/37962 [11:22:00<75:34:29,  8.23s/it]
 13%|█▎        | 4894/37962 [11:22:08<75:48:22,  8.25s/it]
 13%|█▎        | 4895/37962 [11:22:17<75:51:38,  8.26s/it]
 13%|█▎        | 4896/37962 [11:22:25<75:35:13,  8.23s/it]
 13%|█▎        | 4897/37962 [11:22:33<75:43:34,  8.24s/it]
 13%|█▎        | 4898/37962 [11:22:41<74:43:25,  8.14s/it]
 13%|█▎        | 4899/37962 [11:22:49<74:44:21,  8.14s/it]
 13%|█▎        | 4900/37962 [11:22:57<75:06:41,  8.18s/it]
                                                          
{'loss': 1.4965, 'grad_norm': 28.850074768066406, 'learning_rate': 1.948960238584196e-05, 'epoch': 0.13}

 13%|█▎        | 4900/37962 [11:22:57<75:06:41,  8.18s/it]
 13%|█▎        | 4901/37962 [11:23:05<74:43:11,  8.14s/it]
 13%|█▎        | 4902/37962 [11:23:14<75:14:00,  8.19s/it]
 13%|█▎        | 4903/37962 [11:23:22<75:20:39,  8.20s/it]
 13%|█▎        | 4904/37962 [11:23:30<75:53:49,  8.27s/it]
 13%|█▎        | 4905/37962 [11:23:39<76:13:19,  8.30s/it]
 13%|█▎        | 4906/37962 [11:23:47<76:25:26,  8.32s/it]
 13%|█▎        | 4907/37962 [11:23:55<75:34:31,  8.23s/it]
 13%|█▎        | 4908/37962 [11:24:04<75:48:27,  8.26s/it]
 13%|█▎        | 4909/37962 [11:24:12<75:46:30,  8.25s/it]
 13%|█▎        | 4910/37962 [11:24:20<75:04:31,  8.18s/it]
                                                          
{'loss': 1.4768, 'grad_norm': 35.23063278198242, 'learning_rate': 1.9486908100954035e-05, 'epoch': 0.13}

 13%|█▎        | 4910/37962 [11:24:20<75:04:31,  8.18s/it]
 13%|█▎        | 4911/37962 [11:24:28<75:00:47,  8.17s/it]
 13%|█▎        | 4912/37962 [11:24:36<75:37:37,  8.24s/it]
 13%|█▎        | 4913/37962 [11:24:45<75:40:10,  8.24s/it]
 13%|█▎        | 4914/37962 [11:24:53<75:46:36,  8.25s/it]
 13%|█▎        | 4915/37962 [11:25:01<76:26:19,  8.33s/it]
 13%|█▎        | 4916/37962 [11:25:10<76:16:39,  8.31s/it]
 13%|█▎        | 4917/37962 [11:25:18<76:07:50,  8.29s/it]
 13%|█▎        | 4918/37962 [11:25:26<76:24:23,  8.32s/it]
 13%|█▎        | 4919/37962 [11:25:35<76:47:45,  8.37s/it]
 13%|█▎        | 4920/37962 [11:25:43<76:29:03,  8.33s/it]
                                                          
{'loss': 1.592, 'grad_norm': 15.551379203796387, 'learning_rate': 1.948420691071006e-05, 'epoch': 0.13}

 13%|█▎        | 4920/37962 [11:25:43<76:29:03,  8.33s/it]
 13%|█▎        | 4921/37962 [11:25:51<75:45:33,  8.25s/it]
 13%|█▎        | 4922/37962 [11:25:59<75:39:46,  8.24s/it]
 13%|█▎        | 4923/37962 [11:26:08<75:57:31,  8.28s/it]
 13%|█▎        | 4924/37962 [11:26:16<75:38:20,  8.24s/it]
 13%|█▎        | 4925/37962 [11:26:24<75:53:03,  8.27s/it]
 13%|█▎        | 4926/37962 [11:26:32<75:55:47,  8.27s/it]
 13%|█▎        | 4927/37962 [11:26:41<75:49:47,  8.26s/it]
 13%|█▎        | 4928/37962 [11:26:49<76:10:45,  8.30s/it]
 13%|█▎        | 4929/37962 [11:26:58<76:40:01,  8.36s/it]
 13%|█▎        | 4930/37962 [11:27:06<75:42:16,  8.25s/it]
                                                          
{'loss': 1.5485, 'grad_norm': 12.843875885009766, 'learning_rate': 1.948149881707617e-05, 'epoch': 0.13}

 13%|█▎        | 4930/37962 [11:27:06<75:42:16,  8.25s/it]
 13%|█▎        | 4931/37962 [11:27:14<75:35:26,  8.24s/it]
 13%|█▎        | 4932/37962 [11:27:22<75:28:28,  8.23s/it]
 13%|█▎        | 4933/37962 [11:27:30<75:39:36,  8.25s/it]
 13%|█▎        | 4934/37962 [11:27:38<74:33:27,  8.13s/it]
 13%|█▎        | 4935/37962 [11:27:46<75:03:34,  8.18s/it]
 13%|█▎        | 4936/37962 [11:27:55<75:42:34,  8.25s/it]
 13%|█▎        | 4937/37962 [11:28:03<75:54:56,  8.28s/it]
 13%|█▎        | 4938/37962 [11:28:11<75:46:41,  8.26s/it]
 13%|█▎        | 4939/37962 [11:28:20<75:57:40,  8.28s/it]
 13%|█▎        | 4940/37962 [11:28:28<75:55:07,  8.28s/it]
                                                          
{'loss': 1.4603, 'grad_norm': 12.876972198486328, 'learning_rate': 1.9478783822023555e-05, 'epoch': 0.13}

 13%|█▎        | 4940/37962 [11:28:28<75:55:07,  8.28s/it]
 13%|█▎        | 4941/37962 [11:28:36<76:08:06,  8.30s/it]
 13%|█▎        | 4942/37962 [11:28:44<75:36:30,  8.24s/it]
 13%|█▎        | 4943/37962 [11:28:53<75:51:09,  8.27s/it]
 13%|█▎        | 4944/37962 [11:29:01<75:59:55,  8.29s/it]
 13%|█▎        | 4945/37962 [11:29:09<75:48:13,  8.27s/it]
 13%|█▎        | 4946/37962 [11:29:18<75:48:25,  8.27s/it]
 13%|█▎        | 4947/37962 [11:29:26<75:36:04,  8.24s/it]
 13%|█▎        | 4948/37962 [11:29:34<75:46:36,  8.26s/it]
 13%|█▎        | 4949/37962 [11:29:42<75:55:20,  8.28s/it]
 13%|█▎        | 4950/37962 [11:29:51<76:12:58,  8.31s/it]
                                                          
{'loss': 1.4661, 'grad_norm': 12.796365737915039, 'learning_rate': 1.94760619275284e-05, 'epoch': 0.13}

 13%|█▎        | 4950/37962 [11:29:51<76:12:58,  8.31s/it]
 13%|█▎        | 4951/37962 [11:29:59<76:19:39,  8.32s/it]
 13%|█▎        | 4952/37962 [11:30:08<76:57:42,  8.39s/it]
 13%|█▎        | 4953/37962 [11:30:16<76:07:10,  8.30s/it]
 13%|█▎        | 4954/37962 [11:30:24<76:07:02,  8.30s/it]
 13%|█▎        | 4955/37962 [11:30:32<75:27:33,  8.23s/it]
 13%|█▎        | 4956/37962 [11:30:40<75:21:32,  8.22s/it]
 13%|█▎        | 4957/37962 [11:30:49<75:12:09,  8.20s/it]
 13%|█▎        | 4958/37962 [11:30:57<75:13:57,  8.21s/it]
 13%|█▎        | 4959/37962 [11:31:05<75:57:50,  8.29s/it]
 13%|█▎        | 4960/37962 [11:31:13<75:54:22,  8.28s/it]
                                                          
{'loss': 1.559, 'grad_norm': 5.561975479125977, 'learning_rate': 1.947333313557193e-05, 'epoch': 0.13}

 13%|█▎        | 4960/37962 [11:31:13<75:54:22,  8.28s/it]
 13%|█▎        | 4961/37962 [11:31:22<75:40:37,  8.26s/it]
 13%|█▎        | 4962/37962 [11:31:30<76:06:43,  8.30s/it]
 13%|█▎        | 4963/37962 [11:31:38<75:51:08,  8.28s/it]
 13%|█▎        | 4964/37962 [11:31:46<74:44:34,  8.15s/it]
 13%|█▎        | 4965/37962 [11:31:54<74:52:42,  8.17s/it]
 13%|█▎        | 4966/37962 [11:32:03<75:28:29,  8.23s/it]
 13%|█▎        | 4967/37962 [11:32:11<74:42:32,  8.15s/it]
 13%|█▎        | 4968/37962 [11:32:19<74:47:18,  8.16s/it]
 13%|█▎        | 4969/37962 [11:32:27<74:54:35,  8.17s/it]
 13%|█▎        | 4970/37962 [11:32:35<75:11:26,  8.20s/it]
                                                          
{'loss': 1.501, 'grad_norm': 20.76727867126465, 'learning_rate': 1.9470597448140383e-05, 'epoch': 0.13}

 13%|█▎        | 4970/37962 [11:32:35<75:11:26,  8.20s/it]
 13%|█▎        | 4971/37962 [11:32:44<75:07:17,  8.20s/it]
 13%|█▎        | 4972/37962 [11:32:52<75:37:15,  8.25s/it]
 13%|█▎        | 4973/37962 [11:33:00<75:21:02,  8.22s/it]
 13%|█▎        | 4974/37962 [11:33:08<75:32:18,  8.24s/it]
 13%|█▎        | 4975/37962 [11:33:16<74:59:14,  8.18s/it]
 13%|█▎        | 4976/37962 [11:33:25<75:29:02,  8.24s/it]
 13%|█▎        | 4977/37962 [11:33:33<75:36:10,  8.25s/it]
 13%|█▎        | 4978/37962 [11:33:41<74:56:58,  8.18s/it]
 13%|█▎        | 4979/37962 [11:33:49<75:16:00,  8.22s/it]
 13%|█▎        | 4980/37962 [11:33:58<75:07:12,  8.20s/it]
                                                          
{'loss': 1.5444, 'grad_norm': 8.94873332977295, 'learning_rate': 1.946785486722502e-05, 'epoch': 0.13}

 13%|█▎        | 4980/37962 [11:33:58<75:07:12,  8.20s/it]
 13%|█▎        | 4981/37962 [11:34:06<75:12:19,  8.21s/it]
 13%|█▎        | 4982/37962 [11:34:14<75:24:45,  8.23s/it]
 13%|█▎        | 4983/37962 [11:34:22<75:20:59,  8.23s/it]
 13%|█▎        | 4984/37962 [11:34:31<75:45:16,  8.27s/it]
 13%|█▎        | 4985/37962 [11:34:39<76:17:01,  8.33s/it]
 13%|█▎        | 4986/37962 [11:34:47<75:27:59,  8.24s/it]
 13%|█▎        | 4987/37962 [11:34:55<74:06:16,  8.09s/it]
 13%|█▎        | 4988/37962 [11:35:03<74:45:20,  8.16s/it]
 13%|█▎        | 4989/37962 [11:35:11<74:52:32,  8.17s/it]
 13%|█▎        | 4990/37962 [11:35:20<74:55:41,  8.18s/it]
                                                          
{'loss': 1.4132, 'grad_norm': 13.087124824523926, 'learning_rate': 1.9465105394822123e-05, 'epoch': 0.13}

 13%|█▎        | 4990/37962 [11:35:20<74:55:41,  8.18s/it]
 13%|█▎        | 4991/37962 [11:35:28<75:05:16,  8.20s/it]
 13%|█▎        | 4992/37962 [11:35:36<75:57:58,  8.29s/it]
 13%|█▎        | 4993/37962 [11:35:45<76:24:05,  8.34s/it]
 13%|█▎        | 4994/37962 [11:35:53<74:56:52,  8.18s/it]
 13%|█▎        | 4995/37962 [11:36:01<75:10:00,  8.21s/it]
 13%|█▎        | 4996/37962 [11:36:09<74:36:15,  8.15s/it]
 13%|█▎        | 4997/37962 [11:36:17<74:55:54,  8.18s/it]
 13%|█▎        | 4998/37962 [11:36:25<74:49:10,  8.17s/it]
 13%|█▎        | 4999/37962 [11:36:34<75:06:02,  8.20s/it]
 13%|█▎        | 5000/37962 [11:36:42<75:29:24,  8.24s/it]
                                                          
{'loss': 1.4704, 'grad_norm': 7.142757892608643, 'learning_rate': 1.9462349032932982e-05, 'epoch': 0.13}

 13%|█▎        | 5000/37962 [11:36:42<75:29:24,  8.24s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 13%|█▎        | 5001/37962 [11:37:34<195:27:20, 21.35s/it]
 13%|█▎        | 5002/37962 [11:37:42<160:06:16, 17.49s/it]
 13%|█▎        | 5003/37962 [11:37:51<134:35:33, 14.70s/it]
 13%|█▎        | 5004/37962 [11:37:59<116:54:52, 12.77s/it]
 13%|█▎        | 5005/37962 [11:38:07<103:27:15, 11.30s/it]
 13%|█▎        | 5006/37962 [11:38:15<94:24:39, 10.31s/it] 
 13%|█▎        | 5007/37962 [11:38:23<88:49:17,  9.70s/it]
 13%|█▎        | 5008/37962 [11:38:31<85:20:14,  9.32s/it]
 13%|█▎        | 5009/37962 [11:38:40<82:48:43,  9.05s/it]
 13%|█▎        | 5010/37962 [11:38:48<80:39:34,  8.81s/it]
                                                          
{'loss': 1.4631, 'grad_norm': 9.076221466064453, 'learning_rate': 1.94595857835639e-05, 'epoch': 0.13}

 13%|█▎        | 5010/37962 [11:38:48<80:39:34,  8.81s/it]
 13%|█▎        | 5011/37962 [11:38:56<78:51:05,  8.61s/it]
 13%|█▎        | 5012/37962 [11:39:05<78:09:03,  8.54s/it]
 13%|█▎        | 5013/37962 [11:39:13<77:49:42,  8.50s/it]
 13%|█▎        | 5014/37962 [11:39:21<77:31:54,  8.47s/it]
 13%|█▎        | 5015/37962 [11:39:29<76:14:17,  8.33s/it]
 13%|█▎        | 5016/37962 [11:39:38<76:15:53,  8.33s/it]
 13%|█▎        | 5017/37962 [11:39:46<76:02:16,  8.31s/it]
 13%|█▎        | 5018/37962 [11:39:54<74:54:57,  8.19s/it]
 13%|█▎        | 5019/37962 [11:40:02<75:06:42,  8.21s/it]
 13%|█▎        | 5020/37962 [11:40:10<75:21:44,  8.24s/it]
                                                          
{'loss': 1.435, 'grad_norm': 6.705338954925537, 'learning_rate': 1.94568156487262e-05, 'epoch': 0.13}

 13%|█▎        | 5020/37962 [11:40:10<75:21:44,  8.24s/it]
 13%|█▎        | 5021/37962 [11:40:19<75:18:27,  8.23s/it]
 13%|█▎        | 5022/37962 [11:40:27<75:20:50,  8.23s/it]
 13%|█▎        | 5023/37962 [11:40:35<75:04:04,  8.20s/it]
 13%|█▎        | 5024/37962 [11:40:43<75:07:43,  8.21s/it]
 13%|█▎        | 5025/37962 [11:40:51<74:58:44,  8.20s/it]
 13%|█▎        | 5026/37962 [11:41:00<75:02:59,  8.20s/it]
 13%|█▎        | 5027/37962 [11:41:08<75:49:24,  8.29s/it]
 13%|█▎        | 5028/37962 [11:41:16<75:26:49,  8.25s/it]
 13%|█▎        | 5029/37962 [11:41:25<76:38:52,  8.38s/it]
 13%|█▎        | 5030/37962 [11:41:33<76:30:48,  8.36s/it]
                                                          
{'loss': 1.4933, 'grad_norm': 11.892135620117188, 'learning_rate': 1.9454038630436222e-05, 'epoch': 0.13}

 13%|█▎        | 5030/37962 [11:41:33<76:30:48,  8.36s/it]
 13%|█▎        | 5031/37962 [11:41:42<76:29:49,  8.36s/it]
 13%|█▎        | 5032/37962 [11:41:50<76:22:33,  8.35s/it]
 13%|█▎        | 5033/37962 [11:41:58<76:13:53,  8.33s/it]
 13%|█▎        | 5034/37962 [11:42:07<76:27:24,  8.36s/it]
 13%|█▎        | 5035/37962 [11:42:15<76:32:05,  8.37s/it]
 13%|█▎        | 5036/37962 [11:42:23<76:01:49,  8.31s/it]
 13%|█▎        | 5037/37962 [11:42:32<76:01:55,  8.31s/it]
 13%|█▎        | 5038/37962 [11:42:40<75:43:24,  8.28s/it]
 13%|█▎        | 5039/37962 [11:42:48<74:54:47,  8.19s/it]
 13%|█▎        | 5040/37962 [11:42:56<73:51:28,  8.08s/it]
                                                          
{'loss': 1.5221, 'grad_norm': 7.921156406402588, 'learning_rate': 1.9451254730715302e-05, 'epoch': 0.13}

 13%|█▎        | 5040/37962 [11:42:56<73:51:28,  8.08s/it]
 13%|█▎        | 5041/37962 [11:43:04<75:14:28,  8.23s/it]
 13%|█▎        | 5042/37962 [11:43:12<74:13:31,  8.12s/it]
 13%|█▎        | 5043/37962 [11:43:20<74:46:40,  8.18s/it]
 13%|█▎        | 5044/37962 [11:43:29<74:56:12,  8.20s/it]
 13%|█▎        | 5045/37962 [11:43:37<74:37:38,  8.16s/it]
 13%|█▎        | 5046/37962 [11:43:45<74:27:59,  8.14s/it]
 13%|█▎        | 5047/37962 [11:44:00<92:41:19, 10.14s/it]
 13%|█▎        | 5048/37962 [11:44:08<87:37:11,  9.58s/it]
 13%|█▎        | 5049/37962 [11:44:16<84:07:07,  9.20s/it]
 13%|█▎        | 5050/37962 [11:44:24<81:28:44,  8.91s/it]
                                                          
{'loss': 1.5345, 'grad_norm': 5.280564308166504, 'learning_rate': 1.9448463951589797e-05, 'epoch': 0.13}

 13%|█▎        | 5050/37962 [11:44:24<81:28:44,  8.91s/it]
 13%|█▎        | 5051/37962 [11:44:33<80:17:54,  8.78s/it]
 13%|█▎        | 5052/37962 [11:44:41<79:11:59,  8.66s/it]
 13%|█▎        | 5053/37962 [11:44:49<77:23:18,  8.47s/it]
 13%|█▎        | 5054/37962 [11:44:57<76:26:49,  8.36s/it]
 13%|█▎        | 5055/37962 [11:45:05<75:45:02,  8.29s/it]
 13%|█▎        | 5056/37962 [11:45:13<74:40:33,  8.17s/it]
 13%|█▎        | 5057/37962 [11:45:22<74:44:40,  8.18s/it]
 13%|█▎        | 5058/37962 [11:45:30<75:08:34,  8.22s/it]
 13%|█▎        | 5059/37962 [11:45:38<75:11:17,  8.23s/it]
 13%|█▎        | 5060/37962 [11:45:46<75:03:51,  8.21s/it]
                                                          
{'loss': 1.4772, 'grad_norm': 8.546884536743164, 'learning_rate': 1.9445666295091066e-05, 'epoch': 0.13}

 13%|█▎        | 5060/37962 [11:45:46<75:03:51,  8.21s/it]
 13%|█▎        | 5061/37962 [11:45:55<75:24:33,  8.25s/it]
 13%|█▎        | 5062/37962 [11:46:03<75:33:00,  8.27s/it]
 13%|█▎        | 5063/37962 [11:46:11<75:55:21,  8.31s/it]
 13%|█▎        | 5064/37962 [11:46:20<76:01:26,  8.32s/it]
 13%|█▎        | 5065/37962 [11:46:28<75:44:55,  8.29s/it]
 13%|█▎        | 5066/37962 [11:46:36<75:30:45,  8.26s/it]
 13%|█▎        | 5067/37962 [11:46:44<75:37:16,  8.28s/it]
 13%|█▎        | 5068/37962 [11:46:53<75:28:49,  8.26s/it]
 13%|█▎        | 5069/37962 [11:47:01<75:41:18,  8.28s/it]
 13%|█▎        | 5070/37962 [11:47:09<75:56:09,  8.31s/it]
                                                          
{'loss': 1.4185, 'grad_norm': 10.15125560760498, 'learning_rate': 1.9442861763255473e-05, 'epoch': 0.13}

 13%|█▎        | 5070/37962 [11:47:09<75:56:09,  8.31s/it]
 13%|█▎        | 5071/37962 [11:47:18<75:38:30,  8.28s/it]
 13%|█▎        | 5072/37962 [11:47:26<75:48:39,  8.30s/it]
 13%|█▎        | 5073/37962 [11:47:34<75:47:48,  8.30s/it]
 13%|█▎        | 5074/37962 [11:47:43<76:20:19,  8.36s/it]
 13%|█▎        | 5075/37962 [11:47:51<75:53:13,  8.31s/it]
 13%|█▎        | 5076/37962 [11:47:59<75:30:23,  8.27s/it]
 13%|█▎        | 5077/37962 [11:48:07<75:39:16,  8.28s/it]
 13%|█▎        | 5078/37962 [11:48:16<76:15:37,  8.35s/it]
 13%|█▎        | 5079/37962 [11:48:24<76:01:44,  8.32s/it]
 13%|█▎        | 5080/37962 [11:48:32<75:51:49,  8.31s/it]
                                                          
{'loss': 1.5685, 'grad_norm': 25.083036422729492, 'learning_rate': 1.944005035812439e-05, 'epoch': 0.13}

 13%|█▎        | 5080/37962 [11:48:32<75:51:49,  8.31s/it]
 13%|█▎        | 5081/37962 [11:48:40<74:48:51,  8.19s/it]
 13%|█▎        | 5082/37962 [11:48:49<75:03:58,  8.22s/it]
 13%|█▎        | 5083/37962 [11:48:57<75:07:03,  8.22s/it]
 13%|█▎        | 5084/37962 [11:49:05<74:54:24,  8.20s/it]
 13%|█▎        | 5085/37962 [11:49:13<74:30:13,  8.16s/it]
 13%|█▎        | 5086/37962 [11:49:21<75:03:57,  8.22s/it]
 13%|█▎        | 5087/37962 [11:49:30<75:10:46,  8.23s/it]
 13%|█▎        | 5088/37962 [11:49:38<75:10:59,  8.23s/it]
 13%|█▎        | 5089/37962 [11:49:46<75:19:15,  8.25s/it]
 13%|█▎        | 5090/37962 [11:49:54<75:13:23,  8.24s/it]
                                                          
{'loss': 1.3894, 'grad_norm': 16.347349166870117, 'learning_rate': 1.9437232081744183e-05, 'epoch': 0.13}

 13%|█▎        | 5090/37962 [11:49:54<75:13:23,  8.24s/it]
 13%|█▎        | 5091/37962 [11:50:03<75:06:43,  8.23s/it]
 13%|█▎        | 5092/37962 [11:50:11<75:28:17,  8.27s/it]
 13%|█▎        | 5093/37962 [11:50:19<75:09:51,  8.23s/it]
 13%|█▎        | 5094/37962 [11:50:28<75:41:00,  8.29s/it]
 13%|█▎        | 5095/37962 [11:50:36<75:43:24,  8.29s/it]
 13%|█▎        | 5096/37962 [11:50:44<75:36:32,  8.28s/it]
 13%|█▎        | 5097/37962 [11:50:52<75:21:31,  8.25s/it]
 13%|█▎        | 5098/37962 [11:51:01<75:17:11,  8.25s/it]
 13%|█▎        | 5099/37962 [11:51:09<76:09:13,  8.34s/it]
 13%|█▎        | 5100/37962 [11:51:17<75:21:31,  8.26s/it]
                                                          
{'loss': 1.4386, 'grad_norm': 25.382244110107422, 'learning_rate': 1.9434406936166236e-05, 'epoch': 0.13}

 13%|█▎        | 5100/37962 [11:51:17<75:21:31,  8.26s/it]
 13%|█▎        | 5101/37962 [11:51:25<75:12:11,  8.24s/it]
 13%|█▎        | 5102/37962 [11:51:34<75:40:52,  8.29s/it]
 13%|█▎        | 5103/37962 [11:51:42<75:54:51,  8.32s/it]
 13%|█▎        | 5104/37962 [11:51:50<75:42:12,  8.29s/it]
 13%|█▎        | 5105/37962 [11:51:59<75:36:57,  8.28s/it]
 13%|█▎        | 5106/37962 [11:52:07<75:54:26,  8.32s/it]
 13%|█▎        | 5107/37962 [11:52:15<74:51:19,  8.20s/it]
 13%|█▎        | 5108/37962 [11:52:23<75:22:08,  8.26s/it]
 13%|█▎        | 5109/37962 [11:52:32<75:03:17,  8.22s/it]
 13%|█▎        | 5110/37962 [11:52:40<74:45:33,  8.19s/it]
                                                          
{'loss': 1.4935, 'grad_norm': 13.641060829162598, 'learning_rate': 1.9431574923446924e-05, 'epoch': 0.13}

 13%|█▎        | 5110/37962 [11:52:40<74:45:33,  8.19s/it]
 13%|█▎        | 5111/37962 [11:52:48<74:46:21,  8.19s/it]
 13%|█▎        | 5112/37962 [11:52:56<74:41:50,  8.19s/it]
 13%|█▎        | 5113/37962 [11:53:04<74:35:15,  8.17s/it]
 13%|█▎        | 5114/37962 [11:53:12<74:56:21,  8.21s/it]
 13%|█▎        | 5115/37962 [11:53:21<74:57:49,  8.22s/it]
 13%|█▎        | 5116/37962 [11:53:29<75:04:27,  8.23s/it]
 13%|█▎        | 5117/37962 [11:53:37<75:18:17,  8.25s/it]
 13%|█▎        | 5118/37962 [11:53:45<74:36:54,  8.18s/it]
 13%|█▎        | 5119/37962 [11:53:54<75:01:55,  8.22s/it]
 13%|█▎        | 5120/37962 [11:54:01<73:45:49,  8.09s/it]
                                                          
{'loss': 1.4224, 'grad_norm': 30.79372787475586, 'learning_rate': 1.9428736045647613e-05, 'epoch': 0.13}

 13%|█▎        | 5120/37962 [11:54:01<73:45:49,  8.09s/it]
 13%|█▎        | 5121/37962 [11:54:10<74:15:45,  8.14s/it]
 13%|█▎        | 5122/37962 [11:54:18<74:20:13,  8.15s/it]
 13%|█▎        | 5123/37962 [11:54:26<74:47:54,  8.20s/it]
 13%|█▎        | 5124/37962 [11:54:34<74:53:12,  8.21s/it]
 14%|█▎        | 5125/37962 [11:54:43<75:28:56,  8.28s/it]
 14%|█▎        | 5126/37962 [11:54:51<74:52:21,  8.21s/it]
 14%|█▎        | 5127/37962 [11:54:59<75:04:42,  8.23s/it]
 14%|█▎        | 5128/37962 [11:55:07<75:08:28,  8.24s/it]
 14%|█▎        | 5129/37962 [11:55:16<75:32:48,  8.28s/it]
 14%|█▎        | 5130/37962 [11:55:24<74:56:30,  8.22s/it]
                                                          
{'loss': 1.3762, 'grad_norm': 9.646574020385742, 'learning_rate': 1.9425890304834675e-05, 'epoch': 0.14}

 14%|█▎        | 5130/37962 [11:55:24<74:56:30,  8.22s/it]
 14%|█▎        | 5131/37962 [11:55:32<75:13:03,  8.25s/it]
 14%|█▎        | 5132/37962 [11:55:40<75:09:16,  8.24s/it]
 14%|█▎        | 5133/37962 [11:55:49<75:21:09,  8.26s/it]
 14%|█▎        | 5134/37962 [11:55:57<75:59:29,  8.33s/it]
 14%|█▎        | 5135/37962 [11:56:05<75:39:30,  8.30s/it]
 14%|█▎        | 5136/37962 [11:56:14<75:33:53,  8.29s/it]
 14%|█▎        | 5137/37962 [11:56:22<75:36:36,  8.29s/it]
 14%|█▎        | 5138/37962 [11:56:30<76:07:41,  8.35s/it]
 14%|█▎        | 5139/37962 [11:56:38<75:08:11,  8.24s/it]
 14%|█▎        | 5140/37962 [11:56:47<75:06:03,  8.24s/it]
                                                          
{'loss': 1.4583, 'grad_norm': 10.041457176208496, 'learning_rate': 1.9423037703079476e-05, 'epoch': 0.14}

 14%|█▎        | 5140/37962 [11:56:47<75:06:03,  8.24s/it]
 14%|█▎        | 5141/37962 [11:56:55<75:04:47,  8.24s/it]
 14%|█▎        | 5142/37962 [11:57:03<75:31:46,  8.28s/it]
 14%|█▎        | 5143/37962 [11:57:11<75:18:56,  8.26s/it]
 14%|█▎        | 5144/37962 [11:57:20<75:17:42,  8.26s/it]
 14%|█▎        | 5145/37962 [11:57:28<75:16:52,  8.26s/it]
 14%|█▎        | 5146/37962 [11:57:36<75:30:23,  8.28s/it]
 14%|█▎        | 5147/37962 [11:57:44<74:57:21,  8.22s/it]
 14%|█▎        | 5148/37962 [11:57:53<74:50:33,  8.21s/it]
 14%|█▎        | 5149/37962 [11:58:01<75:21:14,  8.27s/it]
 14%|█▎        | 5150/37962 [11:58:09<75:46:37,  8.31s/it]
                                                          
{'loss': 1.4995, 'grad_norm': 10.575789451599121, 'learning_rate': 1.9420178242458377e-05, 'epoch': 0.14}

 14%|█▎        | 5150/37962 [11:58:09<75:46:37,  8.31s/it]
 14%|█▎        | 5151/37962 [11:58:17<75:06:21,  8.24s/it]
 14%|█▎        | 5152/37962 [11:58:26<75:18:54,  8.26s/it]
 14%|█▎        | 5153/37962 [11:58:34<75:24:02,  8.27s/it]
 14%|█▎        | 5154/37962 [11:58:42<75:25:43,  8.28s/it]
 14%|█▎        | 5155/37962 [11:58:51<75:18:51,  8.26s/it]
 14%|█▎        | 5156/37962 [11:58:59<75:10:55,  8.25s/it]
 14%|█▎        | 5157/37962 [11:59:07<75:16:30,  8.26s/it]
 14%|█▎        | 5158/37962 [11:59:15<75:14:24,  8.26s/it]
 14%|█▎        | 5159/37962 [11:59:24<75:11:45,  8.25s/it]
 14%|█▎        | 5160/37962 [11:59:32<75:34:08,  8.29s/it]
                                                          
{'loss': 1.4365, 'grad_norm': 6.112440586090088, 'learning_rate': 1.941731192505273e-05, 'epoch': 0.14}

 14%|█▎        | 5160/37962 [11:59:32<75:34:08,  8.29s/it]
 14%|█▎        | 5161/37962 [11:59:40<75:40:26,  8.31s/it]
 14%|█▎        | 5162/37962 [11:59:49<75:26:34,  8.28s/it]
 14%|█▎        | 5163/37962 [11:59:57<75:22:27,  8.27s/it]
 14%|█▎        | 5164/37962 [12:00:05<75:07:29,  8.25s/it]
 14%|█▎        | 5165/37962 [12:00:13<75:05:31,  8.24s/it]
 14%|█▎        | 5166/37962 [12:00:21<74:49:30,  8.21s/it]
 14%|█▎        | 5167/37962 [12:00:30<74:56:09,  8.23s/it]
 14%|█▎        | 5168/37962 [12:00:38<75:15:57,  8.26s/it]
 14%|█▎        | 5169/37962 [12:00:46<74:35:49,  8.19s/it]
 14%|█▎        | 5170/37962 [12:00:54<74:56:42,  8.23s/it]
                                                          
{'loss': 1.6049, 'grad_norm': 13.74649715423584, 'learning_rate': 1.9414438752948873e-05, 'epoch': 0.14}

 14%|█▎        | 5170/37962 [12:00:54<74:56:42,  8.23s/it]
 14%|█▎        | 5171/37962 [12:01:03<74:59:56,  8.23s/it]
 14%|█▎        | 5172/37962 [12:01:11<74:56:52,  8.23s/it]
 14%|█▎        | 5173/37962 [12:01:19<74:55:05,  8.23s/it]
 14%|█▎        | 5174/37962 [12:01:27<75:06:23,  8.25s/it]
 14%|█▎        | 5175/37962 [12:01:36<75:43:47,  8.32s/it]
 14%|█▎        | 5176/37962 [12:01:44<75:35:44,  8.30s/it]
 14%|█▎        | 5177/37962 [12:01:59<94:09:58, 10.34s/it]
 14%|█▎        | 5178/37962 [12:02:08<88:51:47,  9.76s/it]
 14%|█▎        | 5179/37962 [12:02:16<84:45:08,  9.31s/it]
 14%|█▎        | 5180/37962 [12:02:24<81:45:41,  8.98s/it]
                                                          
{'loss': 1.4401, 'grad_norm': 6.823697090148926, 'learning_rate': 1.9411558728238146e-05, 'epoch': 0.14}

 14%|█▎        | 5180/37962 [12:02:24<81:45:41,  8.98s/it]
 14%|█▎        | 5181/37962 [12:02:32<80:06:14,  8.80s/it]
 14%|█▎        | 5182/37962 [12:02:41<78:28:06,  8.62s/it]
 14%|█▎        | 5183/37962 [12:02:49<77:35:17,  8.52s/it]
 14%|█▎        | 5184/37962 [12:02:57<77:00:51,  8.46s/it]
 14%|█▎        | 5185/37962 [12:03:13<96:39:33, 10.62s/it]
 14%|█▎        | 5186/37962 [12:03:21<90:09:49,  9.90s/it]
 14%|█▎        | 5187/37962 [12:03:29<85:39:20,  9.41s/it]
 14%|█▎        | 5188/37962 [12:03:38<82:26:25,  9.06s/it]
 14%|█▎        | 5189/37962 [12:03:46<79:59:42,  8.79s/it]
 14%|█▎        | 5190/37962 [12:03:54<78:49:19,  8.66s/it]
                                                          
{'loss': 1.5129, 'grad_norm': 4.455138206481934, 'learning_rate': 1.9408671853016866e-05, 'epoch': 0.14}

 14%|█▎        | 5190/37962 [12:03:54<78:49:19,  8.66s/it]
 14%|█▎        | 5191/37962 [12:04:02<77:31:10,  8.52s/it]
 14%|█▎        | 5192/37962 [12:04:10<76:43:19,  8.43s/it]
 14%|█▎        | 5193/37962 [12:04:19<76:05:10,  8.36s/it]
 14%|█▎        | 5194/37962 [12:04:27<75:34:10,  8.30s/it]
 14%|█▎        | 5195/37962 [12:04:35<75:50:32,  8.33s/it]
 14%|█▎        | 5196/37962 [12:04:43<75:18:50,  8.27s/it]
 14%|█▎        | 5197/37962 [12:04:51<74:35:13,  8.20s/it]
 14%|█▎        | 5198/37962 [12:05:00<74:52:36,  8.23s/it]
 14%|█▎        | 5199/37962 [12:05:08<75:50:51,  8.33s/it]
 14%|█▎        | 5200/37962 [12:05:16<75:27:19,  8.29s/it]
                                                          
{'loss': 1.485, 'grad_norm': 5.544444561004639, 'learning_rate': 1.9405778129386332e-05, 'epoch': 0.14}

 14%|█▎        | 5200/37962 [12:05:16<75:27:19,  8.29s/it]
 14%|█▎        | 5201/37962 [12:05:25<75:07:22,  8.26s/it]
 14%|█▎        | 5202/37962 [12:05:33<75:35:18,  8.31s/it]
 14%|█▎        | 5203/37962 [12:05:41<74:41:54,  8.21s/it]
 14%|█▎        | 5204/37962 [12:05:49<74:56:34,  8.24s/it]
 14%|█▎        | 5205/37962 [12:05:58<74:51:25,  8.23s/it]
 14%|█▎        | 5206/37962 [12:06:06<75:11:04,  8.26s/it]
 14%|█▎        | 5207/37962 [12:06:14<74:19:00,  8.17s/it]
 14%|█▎        | 5208/37962 [12:06:22<74:26:07,  8.18s/it]
 14%|█▎        | 5209/37962 [12:06:30<74:01:58,  8.14s/it]
 14%|█▎        | 5210/37962 [12:06:38<74:23:58,  8.18s/it]
                                                          
{'loss': 1.5195, 'grad_norm': 42.43342971801758, 'learning_rate': 1.9402877559452845e-05, 'epoch': 0.14}

 14%|█▎        | 5210/37962 [12:06:38<74:23:58,  8.18s/it]
 14%|█▎        | 5211/37962 [12:06:47<74:48:42,  8.22s/it]
 14%|█▎        | 5212/37962 [12:06:55<75:44:36,  8.33s/it]
 14%|█▎        | 5213/37962 [12:07:04<76:34:09,  8.42s/it]
 14%|█▎        | 5214/37962 [12:07:12<75:10:53,  8.26s/it]
 14%|█▎        | 5215/37962 [12:07:20<75:15:18,  8.27s/it]
 14%|█▎        | 5216/37962 [12:07:28<75:04:22,  8.25s/it]
 14%|█▎        | 5217/37962 [12:07:36<74:50:39,  8.23s/it]
 14%|█▎        | 5218/37962 [12:07:45<74:38:37,  8.21s/it]
 14%|█▎        | 5219/37962 [12:07:53<74:21:15,  8.18s/it]
 14%|█▍        | 5220/37962 [12:08:01<73:43:49,  8.11s/it]
                                                          
{'loss': 1.5061, 'grad_norm': 6.364267826080322, 'learning_rate': 1.9399970145327674e-05, 'epoch': 0.14}

 14%|█▍        | 5220/37962 [12:08:01<73:43:49,  8.11s/it]
 14%|█▍        | 5221/37962 [12:08:09<74:19:52,  8.17s/it]
 14%|█▍        | 5222/37962 [12:08:17<74:59:45,  8.25s/it]
 14%|█▍        | 5223/37962 [12:08:25<74:20:36,  8.17s/it]
 14%|█▍        | 5224/37962 [12:08:34<74:18:17,  8.17s/it]
 14%|█▍        | 5225/37962 [12:08:42<74:24:49,  8.18s/it]
 14%|█▍        | 5226/37962 [12:08:50<74:45:08,  8.22s/it]
 14%|█▍        | 5227/37962 [12:08:58<74:52:08,  8.23s/it]
 14%|█▍        | 5228/37962 [12:09:07<74:59:36,  8.25s/it]
 14%|█▍        | 5229/37962 [12:09:15<75:38:13,  8.32s/it]
 14%|█▍        | 5230/37962 [12:09:23<75:40:32,  8.32s/it]
                                                          
{'loss': 1.5812, 'grad_norm': 6.046536445617676, 'learning_rate': 1.939705588912708e-05, 'epoch': 0.14}

 14%|█▍        | 5230/37962 [12:09:23<75:40:32,  8.32s/it]
 14%|█▍        | 5231/37962 [12:09:32<75:47:39,  8.34s/it]
 14%|█▍        | 5232/37962 [12:09:40<75:33:41,  8.31s/it]
 14%|█▍        | 5233/37962 [12:09:48<75:34:43,  8.31s/it]
 14%|█▍        | 5234/37962 [12:09:57<75:18:09,  8.28s/it]
 14%|█▍        | 5235/37962 [12:10:05<75:14:40,  8.28s/it]
 14%|█▍        | 5236/37962 [12:10:13<74:08:03,  8.16s/it]
 14%|█▍        | 5237/37962 [12:10:21<74:24:56,  8.19s/it]
 14%|█▍        | 5238/37962 [12:10:29<74:35:05,  8.21s/it]
 14%|█▍        | 5239/37962 [12:10:38<75:22:22,  8.29s/it]
 14%|█▍        | 5240/37962 [12:10:46<75:03:13,  8.26s/it]
                                                          
{'loss': 1.4204, 'grad_norm': 12.919092178344727, 'learning_rate': 1.93941347929723e-05, 'epoch': 0.14}

 14%|█▍        | 5240/37962 [12:10:46<75:03:13,  8.26s/it]
 14%|█▍        | 5241/37962 [12:10:54<75:14:51,  8.28s/it]
 14%|█▍        | 5242/37962 [12:11:03<75:34:10,  8.31s/it]
 14%|█▍        | 5243/37962 [12:11:11<75:10:13,  8.27s/it]
 14%|█▍        | 5244/37962 [12:11:26<94:26:51, 10.39s/it]
 14%|█▍        | 5245/37962 [12:11:35<88:46:15,  9.77s/it]
 14%|█▍        | 5246/37962 [12:11:43<84:12:27,  9.27s/it]
 14%|█▍        | 5247/37962 [12:11:51<81:19:22,  8.95s/it]
 14%|█▍        | 5248/37962 [12:11:59<79:08:33,  8.71s/it]
 14%|█▍        | 5249/37962 [12:12:07<77:59:45,  8.58s/it]
 14%|█▍        | 5250/37962 [12:12:16<77:08:41,  8.49s/it]
                                                          
{'loss': 1.4664, 'grad_norm': 14.953060150146484, 'learning_rate': 1.939120685898954e-05, 'epoch': 0.14}

 14%|█▍        | 5250/37962 [12:12:16<77:08:41,  8.49s/it]
 14%|█▍        | 5251/37962 [12:12:24<77:12:29,  8.50s/it]
 14%|█▍        | 5252/37962 [12:12:32<76:33:34,  8.43s/it]
 14%|█▍        | 5253/37962 [12:12:41<76:41:05,  8.44s/it]
 14%|█▍        | 5254/37962 [12:12:49<76:31:59,  8.42s/it]
 14%|█▍        | 5255/37962 [12:12:58<76:21:10,  8.40s/it]
 14%|█▍        | 5256/37962 [12:13:06<75:58:27,  8.36s/it]
 14%|█▍        | 5257/37962 [12:13:14<75:11:52,  8.28s/it]
 14%|█▍        | 5258/37962 [12:13:22<75:02:02,  8.26s/it]
 14%|█▍        | 5259/37962 [12:13:30<75:13:20,  8.28s/it]
 14%|█▍        | 5260/37962 [12:13:46<95:33:52, 10.52s/it]
                                                          
{'loss': 1.5816, 'grad_norm': 7.66500997543335, 'learning_rate': 1.9388272089310004e-05, 'epoch': 0.14}

 14%|█▍        | 5260/37962 [12:13:46<95:33:52, 10.52s/it]
 14%|█▍        | 5261/37962 [12:13:55<90:10:24,  9.93s/it]
 14%|█▍        | 5262/37962 [12:14:03<85:22:25,  9.40s/it]
 14%|█▍        | 5263/37962 [12:14:11<82:08:19,  9.04s/it]
 14%|█▍        | 5264/37962 [12:14:19<79:52:40,  8.79s/it]
 14%|█▍        | 5265/37962 [12:14:27<77:55:45,  8.58s/it]
 14%|█▍        | 5266/37962 [12:14:36<77:08:15,  8.49s/it]
 14%|█▍        | 5267/37962 [12:14:44<76:46:37,  8.45s/it]
 14%|█▍        | 5268/37962 [12:14:52<75:55:41,  8.36s/it]
 14%|█▍        | 5269/37962 [12:15:01<76:12:33,  8.39s/it]
 14%|█▍        | 5270/37962 [12:15:09<75:10:09,  8.28s/it]
                                                          
{'loss': 1.4704, 'grad_norm': 12.277414321899414, 'learning_rate': 1.9385330486069854e-05, 'epoch': 0.14}

 14%|█▍        | 5270/37962 [12:15:09<75:10:09,  8.28s/it]
 14%|█▍        | 5271/37962 [12:15:17<75:16:43,  8.29s/it]
 14%|█▍        | 5272/37962 [12:15:25<75:07:38,  8.27s/it]
 14%|█▍        | 5273/37962 [12:15:34<75:14:23,  8.29s/it]
 14%|█▍        | 5274/37962 [12:15:42<75:12:44,  8.28s/it]
 14%|█▍        | 5275/37962 [12:15:50<75:06:49,  8.27s/it]
 14%|█▍        | 5276/37962 [12:15:58<75:08:25,  8.28s/it]
 14%|█▍        | 5277/37962 [12:16:07<75:29:37,  8.32s/it]
 14%|█▍        | 5278/37962 [12:16:15<74:05:34,  8.16s/it]
 14%|█▍        | 5279/37962 [12:16:23<74:15:44,  8.18s/it]
 14%|█▍        | 5280/37962 [12:16:31<74:35:04,  8.22s/it]
                                                          
{'loss': 1.4266, 'grad_norm': 8.376077651977539, 'learning_rate': 1.938238205141023e-05, 'epoch': 0.14}

 14%|█▍        | 5280/37962 [12:16:31<74:35:04,  8.22s/it]
 14%|█▍        | 5281/37962 [12:16:39<74:29:47,  8.21s/it]
 14%|█▍        | 5282/37962 [12:16:48<74:58:51,  8.26s/it]
 14%|█▍        | 5283/37962 [12:16:56<74:52:07,  8.25s/it]
 14%|█▍        | 5284/37962 [12:17:04<74:52:21,  8.25s/it]
 14%|█▍        | 5285/37962 [12:17:12<74:47:46,  8.24s/it]
 14%|█▍        | 5286/37962 [12:17:21<75:00:20,  8.26s/it]
 14%|█▍        | 5287/37962 [12:17:29<74:45:29,  8.24s/it]
 14%|█▍        | 5288/37962 [12:17:37<74:38:27,  8.22s/it]
 14%|█▍        | 5289/37962 [12:17:45<74:23:38,  8.20s/it]
 14%|█▍        | 5290/37962 [12:17:54<74:53:35,  8.25s/it]
                                                          
{'loss': 1.4479, 'grad_norm': 15.680191993713379, 'learning_rate': 1.9379426787477255e-05, 'epoch': 0.14}

 14%|█▍        | 5290/37962 [12:17:54<74:53:35,  8.25s/it]
 14%|█▍        | 5291/37962 [12:18:02<74:51:44,  8.25s/it]
 14%|█▍        | 5292/37962 [12:18:10<75:11:18,  8.29s/it]
 14%|█▍        | 5293/37962 [12:18:18<75:09:01,  8.28s/it]
 14%|█▍        | 5294/37962 [12:18:27<75:18:19,  8.30s/it]
 14%|█▍        | 5295/37962 [12:18:35<75:47:27,  8.35s/it]
 14%|█▍        | 5296/37962 [12:18:43<75:25:18,  8.31s/it]
 14%|█▍        | 5297/37962 [12:18:52<75:12:23,  8.29s/it]
 14%|█▍        | 5298/37962 [12:19:00<75:08:45,  8.28s/it]
 14%|█▍        | 5299/37962 [12:19:08<75:47:17,  8.35s/it]
 14%|█▍        | 5300/37962 [12:19:17<75:21:29,  8.31s/it]
                                                          
{'loss': 1.4492, 'grad_norm': 22.53719711303711, 'learning_rate': 1.9376464696422003e-05, 'epoch': 0.14}

 14%|█▍        | 5300/37962 [12:19:17<75:21:29,  8.31s/it]
 14%|█▍        | 5301/37962 [12:19:25<74:32:08,  8.22s/it]
 14%|█▍        | 5302/37962 [12:19:33<74:51:12,  8.25s/it]
 14%|█▍        | 5303/37962 [12:19:41<75:25:18,  8.31s/it]
 14%|█▍        | 5304/37962 [12:19:50<75:01:30,  8.27s/it]
 14%|█▍        | 5305/37962 [12:19:58<74:49:19,  8.25s/it]
 14%|█▍        | 5306/37962 [12:20:06<74:59:56,  8.27s/it]
 14%|█▍        | 5307/37962 [12:20:14<74:54:27,  8.26s/it]
 14%|█▍        | 5308/37962 [12:20:23<74:41:26,  8.23s/it]
 14%|█▍        | 5309/37962 [12:20:31<74:42:37,  8.24s/it]
 14%|█▍        | 5310/37962 [12:20:39<74:28:56,  8.21s/it]
                                                          
{'loss': 1.4858, 'grad_norm': 9.848931312561035, 'learning_rate': 1.9373495780400534e-05, 'epoch': 0.14}

 14%|█▍        | 5310/37962 [12:20:39<74:28:56,  8.21s/it]
 14%|█▍        | 5311/37962 [12:20:47<74:48:31,  8.25s/it]
 14%|█▍        | 5312/37962 [12:20:56<74:58:46,  8.27s/it]
 14%|█▍        | 5313/37962 [12:21:04<75:20:31,  8.31s/it]
 14%|█▍        | 5314/37962 [12:21:12<74:46:16,  8.24s/it]
 14%|█▍        | 5315/37962 [12:21:20<74:48:54,  8.25s/it]
 14%|█▍        | 5316/37962 [12:21:29<75:13:52,  8.30s/it]
 14%|█▍        | 5317/37962 [12:21:37<75:34:29,  8.33s/it]
 14%|█▍        | 5318/37962 [12:21:45<74:54:19,  8.26s/it]
 14%|█▍        | 5319/37962 [12:21:54<75:02:36,  8.28s/it]
 14%|█▍        | 5320/37962 [12:22:02<75:01:27,  8.27s/it]
                                                          
{'loss': 1.4869, 'grad_norm': 14.082700729370117, 'learning_rate': 1.9370520041573876e-05, 'epoch': 0.14}

 14%|█▍        | 5320/37962 [12:22:02<75:01:27,  8.27s/it]
 14%|█▍        | 5321/37962 [12:22:10<74:46:35,  8.25s/it]
 14%|█▍        | 5322/37962 [12:22:18<74:36:39,  8.23s/it]
 14%|█▍        | 5323/37962 [12:22:34<93:57:23, 10.36s/it]
 14%|█▍        | 5324/37962 [12:22:42<88:20:20,  9.74s/it]
 14%|█▍        | 5325/37962 [12:22:50<82:41:26,  9.12s/it]
 14%|█▍        | 5326/37962 [12:22:58<80:20:26,  8.86s/it]
 14%|█▍        | 5327/37962 [12:23:06<78:44:44,  8.69s/it]
 14%|█▍        | 5328/37962 [12:23:14<77:31:17,  8.55s/it]
 14%|█▍        | 5329/37962 [12:23:22<76:20:56,  8.42s/it]
 14%|█▍        | 5330/37962 [12:23:31<75:34:08,  8.34s/it]
                                                          
{'loss': 1.5754, 'grad_norm': 11.256121635437012, 'learning_rate': 1.936753748210801e-05, 'epoch': 0.14}

 14%|█▍        | 5330/37962 [12:23:31<75:34:08,  8.34s/it]
 14%|█▍        | 5331/37962 [12:23:39<75:24:43,  8.32s/it]
 14%|█▍        | 5332/37962 [12:23:47<75:03:04,  8.28s/it]
 14%|█▍        | 5333/37962 [12:23:55<74:23:13,  8.21s/it]
 14%|█▍        | 5334/37962 [12:24:03<74:20:02,  8.20s/it]
 14%|█▍        | 5335/37962 [12:24:11<74:18:57,  8.20s/it]
 14%|█▍        | 5336/37962 [12:24:20<74:27:17,  8.22s/it]
 14%|█▍        | 5337/37962 [12:24:28<74:23:51,  8.21s/it]
 14%|█▍        | 5338/37962 [12:24:36<75:11:36,  8.30s/it]
 14%|█▍        | 5339/37962 [12:24:45<74:43:39,  8.25s/it]
 14%|█▍        | 5340/37962 [12:24:53<74:58:55,  8.27s/it]
                                                          
{'loss': 1.4487, 'grad_norm': 14.223258972167969, 'learning_rate': 1.9364548104173894e-05, 'epoch': 0.14}

 14%|█▍        | 5340/37962 [12:24:53<74:58:55,  8.27s/it]
 14%|█▍        | 5341/37962 [12:25:01<74:36:04,  8.23s/it]
 14%|█▍        | 5342/37962 [12:25:09<75:01:51,  8.28s/it]
 14%|█▍        | 5343/37962 [12:25:18<74:38:37,  8.24s/it]
 14%|█▍        | 5344/37962 [12:25:26<74:55:19,  8.27s/it]
 14%|█▍        | 5345/37962 [12:25:34<75:09:07,  8.29s/it]
 14%|█▍        | 5346/37962 [12:25:42<74:58:05,  8.27s/it]
 14%|█▍        | 5347/37962 [12:25:51<74:42:56,  8.25s/it]
 14%|█▍        | 5348/37962 [12:25:59<75:40:14,  8.35s/it]
 14%|█▍        | 5349/37962 [12:26:07<75:25:55,  8.33s/it]
 14%|█▍        | 5350/37962 [12:26:15<74:09:43,  8.19s/it]
                                                          
{'loss': 1.4111, 'grad_norm': 5.047085285186768, 'learning_rate': 1.936155190994744e-05, 'epoch': 0.14}

 14%|█▍        | 5350/37962 [12:26:15<74:09:43,  8.19s/it]
 14%|█▍        | 5351/37962 [12:26:24<75:03:54,  8.29s/it]
 14%|█▍        | 5352/37962 [12:26:39<94:16:43, 10.41s/it]
 14%|█▍        | 5353/37962 [12:26:48<88:58:28,  9.82s/it]
 14%|█▍        | 5354/37962 [12:26:56<83:56:21,  9.27s/it]
 14%|█▍        | 5355/37962 [12:27:04<80:24:51,  8.88s/it]
 14%|█▍        | 5356/37962 [12:27:12<77:43:16,  8.58s/it]
 14%|█▍        | 5357/37962 [12:27:19<75:54:42,  8.38s/it]
 14%|█▍        | 5358/37962 [12:27:28<75:34:37,  8.34s/it]
 14%|█▍        | 5359/37962 [12:27:36<76:08:33,  8.41s/it]
 14%|█▍        | 5360/37962 [12:27:45<75:56:52,  8.39s/it]
                                                          
{'loss': 1.5046, 'grad_norm': 11.50040340423584, 'learning_rate': 1.9358548901609532e-05, 'epoch': 0.14}

 14%|█▍        | 5360/37962 [12:27:45<75:56:52,  8.39s/it]
 14%|█▍        | 5361/37962 [12:27:53<75:48:43,  8.37s/it]
 14%|█▍        | 5362/37962 [12:28:01<75:50:04,  8.37s/it]
 14%|█▍        | 5363/37962 [12:28:10<75:27:41,  8.33s/it]
 14%|█▍        | 5364/37962 [12:28:18<76:31:50,  8.45s/it]
 14%|█▍        | 5365/37962 [12:28:26<75:55:14,  8.38s/it]
 14%|█▍        | 5366/37962 [12:28:35<74:59:07,  8.28s/it]
 14%|█▍        | 5367/37962 [12:28:43<75:19:53,  8.32s/it]
 14%|█▍        | 5368/37962 [12:28:51<75:46:27,  8.37s/it]
 14%|█▍        | 5369/37962 [12:29:00<75:43:09,  8.36s/it]
 14%|█▍        | 5370/37962 [12:29:08<75:15:30,  8.31s/it]
                                                          
{'loss': 1.4808, 'grad_norm': 11.894936561584473, 'learning_rate': 1.9355539081346003e-05, 'epoch': 0.14}

 14%|█▍        | 5370/37962 [12:29:08<75:15:30,  8.31s/it]
 14%|█▍        | 5371/37962 [12:29:16<75:10:47,  8.30s/it]
 14%|█▍        | 5372/37962 [12:29:24<74:47:47,  8.26s/it]
 14%|█▍        | 5373/37962 [12:29:33<74:49:04,  8.26s/it]
 14%|█▍        | 5374/37962 [12:29:41<73:48:51,  8.15s/it]
 14%|█▍        | 5375/37962 [12:29:49<73:57:16,  8.17s/it]
 14%|█▍        | 5376/37962 [12:29:56<72:02:35,  7.96s/it]
 14%|█▍        | 5377/37962 [12:30:05<73:35:21,  8.13s/it]
 14%|█▍        | 5378/37962 [12:30:13<73:19:30,  8.10s/it]
 14%|█▍        | 5379/37962 [12:30:21<73:37:06,  8.13s/it]
 14%|█▍        | 5380/37962 [12:30:29<74:10:11,  8.20s/it]
                                                          
{'loss': 1.4464, 'grad_norm': 10.629592895507812, 'learning_rate': 1.9352522451347647e-05, 'epoch': 0.14}

 14%|█▍        | 5380/37962 [12:30:29<74:10:11,  8.20s/it]
 14%|█▍        | 5381/37962 [12:30:37<73:32:48,  8.13s/it]
 14%|█▍        | 5382/37962 [12:30:46<73:55:30,  8.17s/it]
 14%|█▍        | 5383/37962 [12:30:54<74:07:17,  8.19s/it]
 14%|█▍        | 5384/37962 [12:31:02<74:21:13,  8.22s/it]
 14%|█▍        | 5385/37962 [12:31:10<74:45:34,  8.26s/it]
 14%|█▍        | 5386/37962 [12:31:19<75:28:37,  8.34s/it]
 14%|█▍        | 5387/37962 [12:31:27<75:06:49,  8.30s/it]
 14%|█▍        | 5388/37962 [12:31:36<75:06:00,  8.30s/it]
 14%|█▍        | 5389/37962 [12:31:44<74:49:37,  8.27s/it]
 14%|█▍        | 5390/37962 [12:31:52<75:48:58,  8.38s/it]
                                                          
{'loss': 1.5933, 'grad_norm': 18.75467872619629, 'learning_rate': 1.9349499013810224e-05, 'epoch': 0.14}

 14%|█▍        | 5390/37962 [12:31:52<75:48:58,  8.38s/it]
 14%|█▍        | 5391/37962 [12:32:01<75:33:48,  8.35s/it]
 14%|█▍        | 5392/37962 [12:32:09<75:28:31,  8.34s/it]
 14%|█▍        | 5393/37962 [12:32:17<75:38:24,  8.36s/it]
 14%|█▍        | 5394/37962 [12:32:26<75:29:09,  8.34s/it]
 14%|█▍        | 5395/37962 [12:32:34<75:34:20,  8.35s/it]
 14%|█▍        | 5396/37962 [12:32:43<76:19:32,  8.44s/it]
 14%|█▍        | 5397/37962 [12:32:51<76:01:02,  8.40s/it]
 14%|█▍        | 5398/37962 [12:32:59<75:30:39,  8.35s/it]
 14%|█▍        | 5399/37962 [12:33:15<94:38:57, 10.46s/it]
 14%|█▍        | 5400/37962 [12:33:23<87:52:28,  9.72s/it]
                                                          
{'loss': 1.5137, 'grad_norm': 40.747154235839844, 'learning_rate': 1.9346468770934438e-05, 'epoch': 0.14}

 14%|█▍        | 5400/37962 [12:33:23<87:52:28,  9.72s/it]
 14%|█▍        | 5401/37962 [12:33:31<83:23:26,  9.22s/it]
 14%|█▍        | 5402/37962 [12:33:39<80:03:42,  8.85s/it]
 14%|█▍        | 5403/37962 [12:33:47<77:38:30,  8.58s/it]
 14%|█▍        | 5404/37962 [12:33:55<76:31:39,  8.46s/it]
 14%|█▍        | 5405/37962 [12:34:03<76:16:44,  8.43s/it]
 14%|█▍        | 5406/37962 [12:34:11<76:00:34,  8.41s/it]
 14%|█▍        | 5407/37962 [12:34:20<75:01:47,  8.30s/it]
 14%|█▍        | 5408/37962 [12:34:28<74:46:20,  8.27s/it]
 14%|█▍        | 5409/37962 [12:34:36<74:55:43,  8.29s/it]
 14%|█▍        | 5410/37962 [12:34:44<74:52:42,  8.28s/it]
                                                          
{'loss': 1.5434, 'grad_norm': 42.30437088012695, 'learning_rate': 1.9343431724925948e-05, 'epoch': 0.14}

 14%|█▍        | 5410/37962 [12:34:44<74:52:42,  8.28s/it]
 14%|█▍        | 5411/37962 [12:34:53<75:00:59,  8.30s/it]
 14%|█▍        | 5412/37962 [12:35:01<74:58:24,  8.29s/it]
 14%|█▍        | 5413/37962 [12:35:09<74:54:40,  8.29s/it]
 14%|█▍        | 5414/37962 [12:35:18<75:36:40,  8.36s/it]
 14%|█▍        | 5415/37962 [12:35:26<75:57:20,  8.40s/it]
 14%|█▍        | 5416/37962 [12:35:35<75:35:39,  8.36s/it]
 14%|█▍        | 5417/37962 [12:35:43<75:35:23,  8.36s/it]
 14%|█▍        | 5418/37962 [12:35:51<75:11:19,  8.32s/it]
 14%|█▍        | 5419/37962 [12:35:59<74:55:03,  8.29s/it]
 14%|█▍        | 5420/37962 [12:36:07<74:27:50,  8.24s/it]
                                                          
{'loss': 1.6061, 'grad_norm': 18.229022979736328, 'learning_rate': 1.934038787799537e-05, 'epoch': 0.14}

 14%|█▍        | 5420/37962 [12:36:07<74:27:50,  8.24s/it]
 14%|█▍        | 5421/37962 [12:36:16<74:02:05,  8.19s/it]
 14%|█▍        | 5422/37962 [12:36:24<74:05:29,  8.20s/it]
 14%|█▍        | 5423/37962 [12:36:32<75:01:04,  8.30s/it]
 14%|█▍        | 5424/37962 [12:36:40<74:28:23,  8.24s/it]
 14%|█▍        | 5425/37962 [12:36:48<73:44:54,  8.16s/it]
 14%|█▍        | 5426/37962 [12:36:57<74:03:17,  8.19s/it]
 14%|█▍        | 5427/37962 [12:37:05<74:21:17,  8.23s/it]
 14%|█▍        | 5428/37962 [12:37:13<74:25:03,  8.23s/it]
 14%|█▍        | 5429/37962 [12:37:21<74:15:07,  8.22s/it]
 14%|█▍        | 5430/37962 [12:37:30<74:11:31,  8.21s/it]
                                                          
{'loss': 1.5176, 'grad_norm': 18.547706604003906, 'learning_rate': 1.9337337232358268e-05, 'epoch': 0.14}

 14%|█▍        | 5430/37962 [12:37:30<74:11:31,  8.21s/it]
 14%|█▍        | 5431/37962 [12:37:38<74:38:05,  8.26s/it]
 14%|█▍        | 5432/37962 [12:37:46<73:21:58,  8.12s/it]
 14%|█▍        | 5433/37962 [12:37:54<73:35:13,  8.14s/it]
 14%|█▍        | 5434/37962 [12:38:02<74:14:16,  8.22s/it]
 14%|█▍        | 5435/37962 [12:38:11<74:16:25,  8.22s/it]
 14%|█▍        | 5436/37962 [12:38:19<74:22:15,  8.23s/it]
 14%|█▍        | 5437/37962 [12:38:27<74:47:45,  8.28s/it]
 14%|█▍        | 5438/37962 [12:38:36<75:22:47,  8.34s/it]
 14%|█▍        | 5439/37962 [12:38:44<74:39:57,  8.26s/it]
 14%|█▍        | 5440/37962 [12:38:52<74:40:45,  8.27s/it]
                                                          
{'loss': 1.4847, 'grad_norm': 14.541882514953613, 'learning_rate': 1.9334279790235148e-05, 'epoch': 0.14}

 14%|█▍        | 5440/37962 [12:38:52<74:40:45,  8.27s/it]
 14%|█▍        | 5441/37962 [12:39:00<74:48:17,  8.28s/it]
 14%|█▍        | 5442/37962 [12:39:08<74:19:45,  8.23s/it]
 14%|█▍        | 5443/37962 [12:39:17<74:24:42,  8.24s/it]
 14%|█▍        | 5444/37962 [12:39:25<74:05:58,  8.20s/it]
 14%|█▍        | 5445/37962 [12:39:33<74:24:49,  8.24s/it]
 14%|█▍        | 5446/37962 [12:39:41<74:34:47,  8.26s/it]
 14%|█▍        | 5447/37962 [12:39:49<73:54:01,  8.18s/it]
 14%|█▍        | 5448/37962 [12:39:57<73:04:11,  8.09s/it]
 14%|█▍        | 5449/37962 [12:40:06<73:34:00,  8.15s/it]
 14%|█▍        | 5450/37962 [12:40:13<72:16:02,  8.00s/it]
                                                          
{'loss': 1.4701, 'grad_norm': 4.330589771270752, 'learning_rate': 1.9331215553851476e-05, 'epoch': 0.14}

 14%|█▍        | 5450/37962 [12:40:13<72:16:02,  8.00s/it]
 14%|█▍        | 5451/37962 [12:40:22<73:18:59,  8.12s/it]
 14%|█▍        | 5452/37962 [12:40:30<73:07:18,  8.10s/it]
 14%|█▍        | 5453/37962 [12:40:38<73:29:08,  8.14s/it]
 14%|█▍        | 5454/37962 [12:40:46<73:43:01,  8.16s/it]
 14%|█▍        | 5455/37962 [12:40:54<73:47:18,  8.17s/it]
 14%|█▍        | 5456/37962 [12:41:03<73:46:44,  8.17s/it]
 14%|█▍        | 5457/37962 [12:41:11<74:54:01,  8.30s/it]
 14%|█▍        | 5458/37962 [12:41:19<74:54:35,  8.30s/it]
 14%|█▍        | 5459/37962 [12:41:28<74:39:30,  8.27s/it]
 14%|█▍        | 5460/37962 [12:41:36<73:52:39,  8.18s/it]
                                                          
{'loss': 1.5454, 'grad_norm': 17.344158172607422, 'learning_rate': 1.9328144525437657e-05, 'epoch': 0.14}

 14%|█▍        | 5460/37962 [12:41:36<73:52:39,  8.18s/it]
 14%|█▍        | 5461/37962 [12:41:44<74:19:51,  8.23s/it]
 14%|█▍        | 5462/37962 [12:41:52<74:36:35,  8.26s/it]
 14%|█▍        | 5463/37962 [12:42:01<74:34:49,  8.26s/it]
 14%|█▍        | 5464/37962 [12:42:09<75:00:11,  8.31s/it]
 14%|█▍        | 5465/37962 [12:42:17<74:29:53,  8.25s/it]
 14%|█▍        | 5466/37962 [12:42:26<74:58:04,  8.31s/it]
 14%|█▍        | 5467/37962 [12:42:34<75:05:56,  8.32s/it]
 14%|█▍        | 5468/37962 [12:42:42<74:48:39,  8.29s/it]
 14%|█▍        | 5469/37962 [12:42:50<74:21:11,  8.24s/it]
 14%|█▍        | 5470/37962 [12:42:58<74:29:15,  8.25s/it]
                                                          
{'loss': 1.5498, 'grad_norm': 16.421100616455078, 'learning_rate': 1.9325066707229037e-05, 'epoch': 0.14}

 14%|█▍        | 5470/37962 [12:42:58<74:29:15,  8.25s/it]
 14%|█▍        | 5471/37962 [12:43:07<74:25:40,  8.25s/it]
 14%|█▍        | 5472/37962 [12:43:15<73:43:20,  8.17s/it]
 14%|█▍        | 5473/37962 [12:43:23<74:37:24,  8.27s/it]
 14%|█▍        | 5474/37962 [12:43:32<75:05:53,  8.32s/it]
 14%|█▍        | 5475/37962 [12:43:40<74:49:02,  8.29s/it]
 14%|█▍        | 5476/37962 [12:43:48<73:35:00,  8.15s/it]
 14%|█▍        | 5477/37962 [12:43:56<74:05:20,  8.21s/it]
 14%|█▍        | 5478/37962 [12:44:05<74:58:58,  8.31s/it]
 14%|█▍        | 5479/37962 [12:44:13<74:52:07,  8.30s/it]
 14%|█▍        | 5480/37962 [12:44:21<75:04:54,  8.32s/it]
                                                          
{'loss': 1.4948, 'grad_norm': 9.995844841003418, 'learning_rate': 1.9321982101465907e-05, 'epoch': 0.14}

 14%|█▍        | 5480/37962 [12:44:21<75:04:54,  8.32s/it]
 14%|█▍        | 5481/37962 [12:44:29<74:47:28,  8.29s/it]
 14%|█▍        | 5482/37962 [12:44:38<74:50:54,  8.30s/it]
 14%|█▍        | 5483/37962 [12:44:46<74:55:19,  8.30s/it]
 14%|█▍        | 5484/37962 [12:44:54<73:18:02,  8.12s/it]
 14%|█▍        | 5485/37962 [12:45:02<72:33:04,  8.04s/it]
 14%|█▍        | 5486/37962 [12:45:10<73:13:36,  8.12s/it]
 14%|█▍        | 5487/37962 [12:45:18<73:17:07,  8.12s/it]
 14%|█▍        | 5488/37962 [12:45:26<73:31:21,  8.15s/it]
 14%|█▍        | 5489/37962 [12:45:35<74:14:04,  8.23s/it]
 14%|█▍        | 5490/37962 [12:45:43<74:22:26,  8.25s/it]
                                                          
{'loss': 1.4809, 'grad_norm': 11.987297058105469, 'learning_rate': 1.9318890710393497e-05, 'epoch': 0.14}

 14%|█▍        | 5490/37962 [12:45:43<74:22:26,  8.25s/it]
 14%|█▍        | 5491/37962 [12:45:51<74:21:43,  8.24s/it]
 14%|█▍        | 5492/37962 [12:45:59<74:16:04,  8.23s/it]
 14%|█▍        | 5493/37962 [12:46:08<74:28:56,  8.26s/it]
 14%|█▍        | 5494/37962 [12:46:16<74:23:32,  8.25s/it]
 14%|█▍        | 5495/37962 [12:46:24<74:24:08,  8.25s/it]
 14%|█▍        | 5496/37962 [12:46:32<73:14:22,  8.12s/it]
 14%|█▍        | 5497/37962 [12:46:40<72:56:25,  8.09s/it]
 14%|█▍        | 5498/37962 [12:46:48<73:11:26,  8.12s/it]
 14%|█▍        | 5499/37962 [12:46:57<73:47:01,  8.18s/it]
 14%|█▍        | 5500/37962 [12:47:05<73:48:05,  8.18s/it]
                                                          
{'loss': 1.5004, 'grad_norm': 4.856269836425781, 'learning_rate': 1.9315792536261977e-05, 'epoch': 0.14}

 14%|█▍        | 5500/37962 [12:47:05<73:48:05,  8.18s/it]
 14%|█▍        | 5501/37962 [12:47:13<73:47:57,  8.18s/it]
 14%|█▍        | 5502/37962 [12:47:21<72:31:36,  8.04s/it]
 14%|█▍        | 5503/37962 [12:47:29<72:56:02,  8.09s/it]
 14%|█▍        | 5504/37962 [12:47:37<72:38:08,  8.06s/it]
 15%|█▍        | 5505/37962 [12:47:45<72:47:01,  8.07s/it]
 15%|█▍        | 5506/37962 [12:47:53<73:14:48,  8.12s/it]
 15%|█▍        | 5507/37962 [12:48:01<72:55:34,  8.09s/it]
 15%|█▍        | 5508/37962 [12:48:10<73:59:40,  8.21s/it]
 15%|█▍        | 5509/37962 [12:48:18<74:08:28,  8.22s/it]
 15%|█▍        | 5510/37962 [12:48:26<74:07:12,  8.22s/it]
                                                          
{'loss': 1.6047, 'grad_norm': 16.11746597290039, 'learning_rate': 1.9312687581326454e-05, 'epoch': 0.15}

 15%|█▍        | 5510/37962 [12:48:26<74:07:12,  8.22s/it]
 15%|█▍        | 5511/37962 [12:48:34<74:11:47,  8.23s/it]
 15%|█▍        | 5512/37962 [12:48:43<74:55:35,  8.31s/it]
 15%|█▍        | 5513/37962 [12:48:51<74:48:44,  8.30s/it]
 15%|█▍        | 5514/37962 [12:49:00<74:51:04,  8.30s/it]
 15%|█▍        | 5515/37962 [12:49:08<74:41:24,  8.29s/it]
 15%|█▍        | 5516/37962 [12:49:16<74:58:53,  8.32s/it]
 15%|█▍        | 5517/37962 [12:49:24<74:54:02,  8.31s/it]
 15%|█▍        | 5518/37962 [12:49:32<74:05:49,  8.22s/it]
 15%|█▍        | 5519/37962 [12:49:41<74:14:05,  8.24s/it]
 15%|█▍        | 5520/37962 [12:49:49<74:23:13,  8.25s/it]
                                                          
{'loss': 1.5072, 'grad_norm': 10.342576026916504, 'learning_rate': 1.930957584784697e-05, 'epoch': 0.15}

 15%|█▍        | 5520/37962 [12:49:49<74:23:13,  8.25s/it]
 15%|█▍        | 5521/37962 [12:49:57<74:21:34,  8.25s/it]
 15%|█▍        | 5522/37962 [12:50:05<73:57:34,  8.21s/it]
 15%|█▍        | 5523/37962 [12:50:14<74:17:16,  8.24s/it]
 15%|█▍        | 5524/37962 [12:50:22<74:36:34,  8.28s/it]
 15%|█▍        | 5525/37962 [12:50:31<75:08:07,  8.34s/it]
 15%|█▍        | 5526/37962 [12:50:39<75:36:16,  8.39s/it]
 15%|█▍        | 5527/37962 [12:50:47<75:18:43,  8.36s/it]
 15%|█▍        | 5528/37962 [12:50:56<75:39:24,  8.40s/it]
 15%|█▍        | 5529/37962 [12:51:04<75:18:58,  8.36s/it]
 15%|█▍        | 5530/37962 [12:51:12<75:01:40,  8.33s/it]
                                                          
{'loss': 1.5299, 'grad_norm': 209.44180297851562, 'learning_rate': 1.930645733808851e-05, 'epoch': 0.15}

 15%|█▍        | 5530/37962 [12:51:12<75:01:40,  8.33s/it]
 15%|█▍        | 5531/37962 [12:51:21<74:57:02,  8.32s/it]
 15%|█▍        | 5532/37962 [12:51:28<73:33:57,  8.17s/it]
 15%|█▍        | 5533/37962 [12:51:37<74:07:00,  8.23s/it]
 15%|█▍        | 5534/37962 [12:51:45<74:24:39,  8.26s/it]
 15%|█▍        | 5535/37962 [12:51:53<73:12:56,  8.13s/it]
 15%|█▍        | 5536/37962 [12:52:01<73:39:14,  8.18s/it]
 15%|█▍        | 5537/37962 [12:52:09<73:36:28,  8.17s/it]
 15%|█▍        | 5538/37962 [12:52:18<73:49:32,  8.20s/it]
 15%|█▍        | 5539/37962 [12:52:26<74:00:21,  8.22s/it]
 15%|█▍        | 5540/37962 [12:52:34<74:04:58,  8.23s/it]
                                                          
{'loss': 1.5775, 'grad_norm': 77.56652069091797, 'learning_rate': 1.930333205432097e-05, 'epoch': 0.15}

 15%|█▍        | 5540/37962 [12:52:34<74:04:58,  8.23s/it]
 15%|█▍        | 5541/37962 [12:52:42<74:09:00,  8.23s/it]
 15%|█▍        | 5542/37962 [12:52:51<74:23:36,  8.26s/it]
 15%|█▍        | 5543/37962 [12:52:59<74:21:49,  8.26s/it]
 15%|█▍        | 5544/37962 [12:53:07<74:22:49,  8.26s/it]
 15%|█▍        | 5545/37962 [12:53:16<75:14:08,  8.36s/it]
 15%|█▍        | 5546/37962 [12:53:24<74:44:30,  8.30s/it]
 15%|█▍        | 5547/37962 [12:53:32<74:38:30,  8.29s/it]
 15%|█▍        | 5548/37962 [12:53:41<74:53:00,  8.32s/it]
 15%|█▍        | 5549/37962 [12:53:49<74:52:14,  8.32s/it]
 15%|█▍        | 5550/37962 [12:53:57<74:43:26,  8.30s/it]
                                                          
{'loss': 1.4323, 'grad_norm': 50.099063873291016, 'learning_rate': 1.93001999988192e-05, 'epoch': 0.15}

 15%|█▍        | 5550/37962 [12:53:57<74:43:26,  8.30s/it]
 15%|█▍        | 5551/37962 [12:54:05<74:29:12,  8.27s/it]
 15%|█▍        | 5552/37962 [12:54:14<74:19:17,  8.26s/it]
 15%|█▍        | 5553/37962 [12:54:22<74:23:21,  8.26s/it]
 15%|█▍        | 5554/37962 [12:54:30<74:44:02,  8.30s/it]
 15%|█▍        | 5555/37962 [12:54:39<74:24:53,  8.27s/it]
 15%|█▍        | 5556/37962 [12:54:47<73:45:02,  8.19s/it]
 15%|█▍        | 5557/37962 [12:54:55<74:06:43,  8.23s/it]
 15%|█▍        | 5558/37962 [12:55:03<74:12:27,  8.24s/it]
 15%|█▍        | 5559/37962 [12:55:11<73:36:43,  8.18s/it]
 15%|█▍        | 5560/37962 [12:55:20<74:11:25,  8.24s/it]
                                                          
{'loss': 1.3863, 'grad_norm': 43.60694885253906, 'learning_rate': 1.9297061173862967e-05, 'epoch': 0.15}

 15%|█▍        | 5560/37962 [12:55:20<74:11:25,  8.24s/it]
 15%|█▍        | 5561/37962 [12:55:28<74:15:12,  8.25s/it]
 15%|█▍        | 5562/37962 [12:55:36<74:32:44,  8.28s/it]
 15%|█▍        | 5563/37962 [12:55:44<74:30:28,  8.28s/it]
 15%|█▍        | 5564/37962 [12:55:53<74:52:45,  8.32s/it]
 15%|█▍        | 5565/37962 [12:56:01<74:46:35,  8.31s/it]
 15%|█▍        | 5566/37962 [12:56:09<74:36:06,  8.29s/it]
 15%|█▍        | 5567/37962 [12:56:18<74:15:35,  8.25s/it]
 15%|█▍        | 5568/37962 [12:56:26<73:36:06,  8.18s/it]
 15%|█▍        | 5569/37962 [12:56:34<73:58:18,  8.22s/it]
 15%|█▍        | 5570/37962 [12:56:42<72:58:31,  8.11s/it]
                                                          
{'loss': 1.4396, 'grad_norm': 107.82313537597656, 'learning_rate': 1.9293915581736966e-05, 'epoch': 0.15}

 15%|█▍        | 5570/37962 [12:56:42<72:58:31,  8.11s/it]
 15%|█▍        | 5571/37962 [12:56:50<73:21:02,  8.15s/it]
 15%|█▍        | 5572/37962 [12:56:58<73:39:59,  8.19s/it]
 15%|█▍        | 5573/37962 [12:57:07<73:59:45,  8.22s/it]
 15%|█▍        | 5574/37962 [12:57:15<74:11:19,  8.25s/it]
 15%|█▍        | 5575/37962 [12:57:23<74:36:09,  8.29s/it]
 15%|█▍        | 5576/37962 [12:57:31<74:18:04,  8.26s/it]
 15%|█▍        | 5577/37962 [12:57:40<73:49:48,  8.21s/it]
 15%|█▍        | 5578/37962 [12:57:47<72:56:28,  8.11s/it]
 15%|█▍        | 5579/37962 [12:57:56<73:20:52,  8.15s/it]
 15%|█▍        | 5580/37962 [12:58:04<73:46:09,  8.20s/it]
                                                          
{'loss': 1.5672, 'grad_norm': 309.54583740234375, 'learning_rate': 1.9290763224730818e-05, 'epoch': 0.15}

 15%|█▍        | 5580/37962 [12:58:04<73:46:09,  8.20s/it]
 15%|█▍        | 5581/37962 [12:58:12<73:52:48,  8.21s/it]
 15%|█▍        | 5582/37962 [12:58:21<74:15:31,  8.26s/it]
 15%|█▍        | 5583/37962 [12:58:29<74:07:01,  8.24s/it]
 15%|█▍        | 5584/37962 [12:58:37<74:54:58,  8.33s/it]
 15%|█▍        | 5585/37962 [12:58:46<74:36:35,  8.30s/it]
 15%|█▍        | 5586/37962 [12:58:54<74:16:08,  8.26s/it]
 15%|█▍        | 5587/37962 [12:59:02<74:45:58,  8.31s/it]
 15%|█▍        | 5588/37962 [12:59:11<75:02:01,  8.34s/it]
 15%|█▍        | 5589/37962 [12:59:19<75:00:20,  8.34s/it]
 15%|█▍        | 5590/37962 [12:59:27<74:46:52,  8.32s/it]
                                                          
{'loss': 1.4473, 'grad_norm': 36.68277359008789, 'learning_rate': 1.928760410513907e-05, 'epoch': 0.15}

 15%|█▍        | 5590/37962 [12:59:27<74:46:52,  8.32s/it]
 15%|█▍        | 5591/37962 [12:59:35<74:03:20,  8.24s/it]
 15%|█▍        | 5592/37962 [12:59:43<73:59:06,  8.23s/it]
 15%|█▍        | 5593/37962 [12:59:51<73:13:45,  8.14s/it]
 15%|█▍        | 5594/37962 [13:00:00<73:22:10,  8.16s/it]
 15%|█▍        | 5595/37962 [13:00:08<73:36:27,  8.19s/it]
 15%|█▍        | 5596/37962 [13:00:16<73:02:46,  8.12s/it]
 15%|█▍        | 5597/37962 [13:00:24<73:08:57,  8.14s/it]
 15%|█▍        | 5598/37962 [13:00:32<73:51:22,  8.22s/it]
 15%|█▍        | 5599/37962 [13:00:41<73:42:44,  8.20s/it]
 15%|█▍        | 5600/37962 [13:00:49<73:49:03,  8.21s/it]
                                                          
{'loss': 1.473, 'grad_norm': 16.421710968017578, 'learning_rate': 1.928443822526119e-05, 'epoch': 0.15}

 15%|█▍        | 5600/37962 [13:00:49<73:49:03,  8.21s/it]
 15%|█▍        | 5601/37962 [13:00:57<73:59:18,  8.23s/it]
 15%|█▍        | 5602/37962 [13:01:05<74:11:01,  8.25s/it]
 15%|█▍        | 5603/37962 [13:01:14<74:01:54,  8.24s/it]
 15%|█▍        | 5604/37962 [13:01:22<73:58:38,  8.23s/it]
 15%|█▍        | 5605/37962 [13:01:30<74:32:18,  8.29s/it]
 15%|█▍        | 5606/37962 [13:01:38<73:10:13,  8.14s/it]
 15%|█▍        | 5607/37962 [13:01:46<73:58:48,  8.23s/it]
 15%|█▍        | 5608/37962 [13:01:55<73:48:59,  8.21s/it]
 15%|█▍        | 5609/37962 [13:02:03<74:44:37,  8.32s/it]
 15%|█▍        | 5610/37962 [13:02:11<74:37:46,  8.30s/it]
                                                          
{'loss': 1.5026, 'grad_norm': 46.022254943847656, 'learning_rate': 1.928126558740157e-05, 'epoch': 0.15}

 15%|█▍        | 5610/37962 [13:02:11<74:37:46,  8.30s/it]
 15%|█▍        | 5611/37962 [13:02:20<74:43:23,  8.32s/it]
 15%|█▍        | 5612/37962 [13:02:28<74:30:46,  8.29s/it]
 15%|█▍        | 5613/37962 [13:02:36<74:38:10,  8.31s/it]
 15%|█▍        | 5614/37962 [13:02:45<74:22:10,  8.28s/it]
 15%|█▍        | 5615/37962 [13:02:53<74:20:26,  8.27s/it]
 15%|█▍        | 5616/37962 [13:03:01<74:32:06,  8.30s/it]
 15%|█▍        | 5617/37962 [13:03:09<74:26:33,  8.29s/it]
 15%|█▍        | 5618/37962 [13:03:18<74:25:36,  8.28s/it]
 15%|█▍        | 5619/37962 [13:03:26<74:17:05,  8.27s/it]
 15%|█▍        | 5620/37962 [13:03:34<74:32:46,  8.30s/it]
                                                          
{'loss': 1.3835, 'grad_norm': 82.41382598876953, 'learning_rate': 1.9278086193869514e-05, 'epoch': 0.15}

 15%|█▍        | 5620/37962 [13:03:34<74:32:46,  8.30s/it]
 15%|█▍        | 5621/37962 [13:03:43<74:34:31,  8.30s/it]
 15%|█▍        | 5622/37962 [13:03:51<73:57:06,  8.23s/it]
 15%|█▍        | 5623/37962 [13:03:59<73:47:42,  8.21s/it]
 15%|█▍        | 5624/37962 [13:04:07<74:05:48,  8.25s/it]
 15%|█▍        | 5625/37962 [13:04:16<74:13:37,  8.26s/it]
 15%|█▍        | 5626/37962 [13:04:24<74:26:04,  8.29s/it]
 15%|█▍        | 5627/37962 [13:04:32<74:44:28,  8.32s/it]
 15%|█▍        | 5628/37962 [13:04:41<75:06:26,  8.36s/it]
 15%|█▍        | 5629/37962 [13:04:49<75:00:27,  8.35s/it]
 15%|█▍        | 5630/37962 [13:04:57<75:03:36,  8.36s/it]
                                                          
{'loss': 1.4417, 'grad_norm': 24.610567092895508, 'learning_rate': 1.927490004697925e-05, 'epoch': 0.15}

 15%|█▍        | 5630/37962 [13:04:57<75:03:36,  8.36s/it]
 15%|█▍        | 5631/37962 [13:05:06<75:04:10,  8.36s/it]
 15%|█▍        | 5632/37962 [13:05:14<73:28:33,  8.18s/it]
 15%|█▍        | 5633/37962 [13:05:22<74:00:15,  8.24s/it]
 15%|█▍        | 5634/37962 [13:05:30<74:31:00,  8.30s/it]
 15%|█▍        | 5635/37962 [13:05:39<74:51:56,  8.34s/it]
 15%|█▍        | 5636/37962 [13:05:47<74:29:08,  8.30s/it]
 15%|█▍        | 5637/37962 [13:05:55<74:06:20,  8.25s/it]
 15%|█▍        | 5638/37962 [13:06:04<74:41:37,  8.32s/it]
 15%|█▍        | 5639/37962 [13:06:12<74:28:07,  8.29s/it]
 15%|█▍        | 5640/37962 [13:06:20<73:25:06,  8.18s/it]
                                                          
{'loss': 1.4505, 'grad_norm': 20.438344955444336, 'learning_rate': 1.9271707149049917e-05, 'epoch': 0.15}

 15%|█▍        | 5640/37962 [13:06:20<73:25:06,  8.18s/it]
 15%|█▍        | 5641/37962 [13:06:28<73:58:35,  8.24s/it]
 15%|█▍        | 5642/37962 [13:06:37<74:24:25,  8.29s/it]
 15%|█▍        | 5643/37962 [13:06:45<73:51:17,  8.23s/it]
 15%|█▍        | 5644/37962 [13:06:53<74:01:43,  8.25s/it]
 15%|█▍        | 5645/37962 [13:07:01<74:08:33,  8.26s/it]
 15%|█▍        | 5646/37962 [13:07:09<74:10:09,  8.26s/it]
 15%|█▍        | 5647/37962 [13:07:18<74:06:55,  8.26s/it]
 15%|█▍        | 5648/37962 [13:07:26<74:44:02,  8.33s/it]
 15%|█▍        | 5649/37962 [13:07:34<74:20:35,  8.28s/it]
 15%|█▍        | 5650/37962 [13:07:43<74:28:25,  8.30s/it]
                                                          
{'loss': 1.4176, 'grad_norm': 43.40306854248047, 'learning_rate': 1.9268507502405574e-05, 'epoch': 0.15}

 15%|█▍        | 5650/37962 [13:07:43<74:28:25,  8.30s/it]
 15%|█▍        | 5651/37962 [13:07:51<73:30:31,  8.19s/it]
 15%|█▍        | 5652/37962 [13:07:59<73:35:47,  8.20s/it]
 15%|█▍        | 5653/37962 [13:08:07<73:36:37,  8.20s/it]
 15%|█▍        | 5654/37962 [13:08:15<73:43:48,  8.22s/it]
 15%|█▍        | 5655/37962 [13:08:24<73:45:57,  8.22s/it]
 15%|█▍        | 5656/37962 [13:08:32<73:42:28,  8.21s/it]
 15%|█▍        | 5657/37962 [13:08:40<73:50:19,  8.23s/it]
 15%|█▍        | 5658/37962 [13:08:48<74:04:48,  8.26s/it]
 15%|█▍        | 5659/37962 [13:08:57<73:49:50,  8.23s/it]
 15%|█▍        | 5660/37962 [13:09:05<74:25:12,  8.29s/it]
                                                          
{'loss': 1.6208, 'grad_norm': 24.8510684967041, 'learning_rate': 1.9265301109375185e-05, 'epoch': 0.15}

 15%|█▍        | 5660/37962 [13:09:05<74:25:12,  8.29s/it]
 15%|█▍        | 5661/37962 [13:09:13<73:49:20,  8.23s/it]
 15%|█▍        | 5662/37962 [13:09:21<74:09:01,  8.26s/it]
 15%|█▍        | 5663/37962 [13:09:30<73:55:51,  8.24s/it]
 15%|█▍        | 5664/37962 [13:09:38<74:13:24,  8.27s/it]
 15%|█▍        | 5665/37962 [13:09:46<74:15:18,  8.28s/it]
 15%|█▍        | 5666/37962 [13:09:55<74:19:06,  8.28s/it]
 15%|█▍        | 5667/37962 [13:10:03<74:39:21,  8.32s/it]
 15%|█▍        | 5668/37962 [13:10:11<74:41:08,  8.33s/it]
 15%|█▍        | 5669/37962 [13:10:19<74:20:31,  8.29s/it]
 15%|█▍        | 5670/37962 [13:10:28<74:25:35,  8.30s/it]
                                                          
{'loss': 1.5051, 'grad_norm': 30.880521774291992, 'learning_rate': 1.9262087972292628e-05, 'epoch': 0.15}

 15%|█▍        | 5670/37962 [13:10:28<74:25:35,  8.30s/it]
 15%|█▍        | 5671/37962 [13:10:36<73:53:54,  8.24s/it]
 15%|█▍        | 5672/37962 [13:10:44<73:58:40,  8.25s/it]
 15%|█▍        | 5673/37962 [13:10:53<74:18:59,  8.29s/it]
 15%|█▍        | 5674/37962 [13:11:01<74:32:14,  8.31s/it]
 15%|█▍        | 5675/37962 [13:11:09<74:46:44,  8.34s/it]
 15%|█▍        | 5676/37962 [13:11:18<75:42:17,  8.44s/it]
 15%|█▍        | 5677/37962 [13:11:26<75:32:21,  8.42s/it]
 15%|█▍        | 5678/37962 [13:11:34<74:18:20,  8.29s/it]
 15%|█▍        | 5679/37962 [13:11:42<73:58:29,  8.25s/it]
 15%|█▍        | 5680/37962 [13:11:51<74:36:01,  8.32s/it]
                                                          
{'loss': 1.4677, 'grad_norm': 5.721530914306641, 'learning_rate': 1.925886809349669e-05, 'epoch': 0.15}

 15%|█▍        | 5680/37962 [13:11:51<74:36:01,  8.32s/it]
 15%|█▍        | 5681/37962 [13:11:59<73:12:09,  8.16s/it]
 15%|█▍        | 5682/37962 [13:12:07<73:52:06,  8.24s/it]
 15%|█▍        | 5683/37962 [13:12:16<74:19:28,  8.29s/it]
 15%|█▍        | 5684/37962 [13:12:24<74:13:36,  8.28s/it]
 15%|█▍        | 5685/37962 [13:12:32<74:19:00,  8.29s/it]
 15%|█▍        | 5686/37962 [13:12:40<73:22:10,  8.18s/it]
 15%|█▍        | 5687/37962 [13:12:48<73:30:15,  8.20s/it]
 15%|█▍        | 5688/37962 [13:12:57<73:29:38,  8.20s/it]
 15%|█▍        | 5689/37962 [13:13:05<73:21:50,  8.18s/it]
 15%|█▍        | 5690/37962 [13:13:13<73:38:17,  8.21s/it]
                                                          
{'loss': 1.482, 'grad_norm': 5.317755699157715, 'learning_rate': 1.9255641475331066e-05, 'epoch': 0.15}

 15%|█▍        | 5690/37962 [13:13:13<73:38:17,  8.21s/it]
 15%|█▍        | 5691/37962 [13:13:21<73:28:58,  8.20s/it]
 15%|█▍        | 5692/37962 [13:13:30<74:25:43,  8.30s/it]
 15%|█▍        | 5693/37962 [13:13:38<73:54:01,  8.24s/it]
 15%|█▍        | 5694/37962 [13:13:46<73:42:34,  8.22s/it]
 15%|█▌        | 5695/37962 [13:13:54<73:34:18,  8.21s/it]
 15%|█▌        | 5696/37962 [13:14:02<73:52:51,  8.24s/it]
 15%|█▌        | 5697/37962 [13:14:10<72:49:44,  8.13s/it]
 15%|█▌        | 5698/37962 [13:14:19<73:04:43,  8.15s/it]
 15%|█▌        | 5699/37962 [13:14:27<73:17:20,  8.18s/it]
 15%|█▌        | 5700/37962 [13:14:35<73:15:17,  8.17s/it]
                                                          
{'loss': 1.2761, 'grad_norm': 4.381497383117676, 'learning_rate': 1.9252408120144357e-05, 'epoch': 0.15}

 15%|█▌        | 5700/37962 [13:14:35<73:15:17,  8.17s/it]
 15%|█▌        | 5701/37962 [13:14:43<73:49:52,  8.24s/it]
 15%|█▌        | 5702/37962 [13:14:52<74:03:45,  8.26s/it]
 15%|█▌        | 5703/37962 [13:15:00<73:55:53,  8.25s/it]
 15%|█▌        | 5704/37962 [13:15:08<73:56:38,  8.25s/it]
 15%|█▌        | 5705/37962 [13:15:16<73:56:14,  8.25s/it]
 15%|█▌        | 5706/37962 [13:15:25<74:34:25,  8.32s/it]
 15%|█▌        | 5707/37962 [13:15:33<74:49:11,  8.35s/it]
 15%|█▌        | 5708/37962 [13:15:41<74:02:28,  8.26s/it]
 15%|█▌        | 5709/37962 [13:15:50<74:04:20,  8.27s/it]
 15%|█▌        | 5710/37962 [13:15:58<73:59:06,  8.26s/it]
                                                          
{'loss': 1.4058, 'grad_norm': 26.98045539855957, 'learning_rate': 1.9249168030290062e-05, 'epoch': 0.15}

 15%|█▌        | 5710/37962 [13:15:58<73:59:06,  8.26s/it]
 15%|█▌        | 5711/37962 [13:16:06<74:58:14,  8.37s/it]
 15%|█▌        | 5712/37962 [13:16:15<75:13:27,  8.40s/it]
 15%|█▌        | 5713/37962 [13:16:23<74:29:28,  8.32s/it]
 15%|█▌        | 5714/37962 [13:16:31<74:46:38,  8.35s/it]
 15%|█▌        | 5715/37962 [13:16:40<74:42:15,  8.34s/it]
 15%|█▌        | 5716/37962 [13:16:48<74:04:47,  8.27s/it]
 15%|█▌        | 5717/37962 [13:16:56<74:18:45,  8.30s/it]
 15%|█▌        | 5718/37962 [13:17:05<74:23:03,  8.30s/it]
 15%|█▌        | 5719/37962 [13:17:12<73:17:59,  8.18s/it]
 15%|█▌        | 5720/37962 [13:17:21<73:26:50,  8.20s/it]
                                                          
{'loss': 1.4197, 'grad_norm': 31.20049285888672, 'learning_rate': 1.9245921208126592e-05, 'epoch': 0.15}

 15%|█▌        | 5720/37962 [13:17:21<73:26:50,  8.20s/it]
 15%|█▌        | 5721/37962 [13:17:29<73:18:00,  8.18s/it]
 15%|█▌        | 5722/37962 [13:17:37<73:59:45,  8.26s/it]
 15%|█▌        | 5723/37962 [13:17:46<73:54:48,  8.25s/it]
 15%|█▌        | 5724/37962 [13:17:54<73:56:37,  8.26s/it]
 15%|█▌        | 5725/37962 [13:18:02<73:27:35,  8.20s/it]
 15%|█▌        | 5726/37962 [13:18:10<73:37:27,  8.22s/it]
 15%|█▌        | 5727/37962 [13:18:18<72:28:46,  8.09s/it]
 15%|█▌        | 5728/37962 [13:18:26<72:39:23,  8.11s/it]
 15%|█▌        | 5729/37962 [13:18:34<73:14:28,  8.18s/it]
 15%|█▌        | 5730/37962 [13:18:43<73:40:28,  8.23s/it]
                                                          
{'loss': 1.5427, 'grad_norm': 56.033199310302734, 'learning_rate': 1.9242667656017247e-05, 'epoch': 0.15}

 15%|█▌        | 5730/37962 [13:18:43<73:40:28,  8.23s/it]
 15%|█▌        | 5731/37962 [13:18:51<74:00:23,  8.27s/it]
 15%|█▌        | 5732/37962 [13:18:59<73:52:02,  8.25s/it]
 15%|█▌        | 5733/37962 [13:19:08<74:02:51,  8.27s/it]
 15%|█▌        | 5734/37962 [13:19:16<73:45:53,  8.24s/it]
 15%|█▌        | 5735/37962 [13:19:24<74:01:42,  8.27s/it]
 15%|█▌        | 5736/37962 [13:19:33<74:25:17,  8.31s/it]
 15%|█▌        | 5737/37962 [13:19:41<74:09:07,  8.28s/it]
 15%|█▌        | 5738/37962 [13:19:49<73:11:03,  8.18s/it]
 15%|█▌        | 5739/37962 [13:19:57<73:38:15,  8.23s/it]
 15%|█▌        | 5740/37962 [13:20:05<73:57:53,  8.26s/it]
                                                          
{'loss': 1.5271, 'grad_norm': 17.526260375976562, 'learning_rate': 1.9239407376330233e-05, 'epoch': 0.15}

 15%|█▌        | 5740/37962 [13:20:05<73:57:53,  8.26s/it]
 15%|█▌        | 5741/37962 [13:20:14<74:41:00,  8.34s/it]
 15%|█▌        | 5742/37962 [13:20:22<74:19:44,  8.30s/it]
 15%|█▌        | 5743/37962 [13:20:31<74:29:37,  8.32s/it]
 15%|█▌        | 5744/37962 [13:20:39<74:50:00,  8.36s/it]
 15%|█▌        | 5745/37962 [13:20:47<74:28:41,  8.32s/it]
 15%|█▌        | 5746/37962 [13:20:56<75:11:49,  8.40s/it]
 15%|█▌        | 5747/37962 [13:21:04<75:11:38,  8.40s/it]
 15%|█▌        | 5748/37962 [13:21:13<74:59:26,  8.38s/it]
 15%|█▌        | 5749/37962 [13:21:21<74:26:32,  8.32s/it]
 15%|█▌        | 5750/37962 [13:21:29<73:36:42,  8.23s/it]
                                                          
{'loss': 1.3915, 'grad_norm': 12.591251373291016, 'learning_rate': 1.923614037143865e-05, 'epoch': 0.15}

 15%|█▌        | 5750/37962 [13:21:29<73:36:42,  8.23s/it]
 15%|█▌        | 5751/37962 [13:21:37<73:37:19,  8.23s/it]
 15%|█▌        | 5752/37962 [13:21:45<73:51:14,  8.25s/it]
 15%|█▌        | 5753/37962 [13:21:53<72:47:52,  8.14s/it]
 15%|█▌        | 5754/37962 [13:22:01<73:20:39,  8.20s/it]
 15%|█▌        | 5755/37962 [13:22:10<72:56:56,  8.15s/it]
 15%|█▌        | 5756/37962 [13:22:18<73:38:55,  8.23s/it]
 15%|█▌        | 5757/37962 [13:22:26<73:22:07,  8.20s/it]
 15%|█▌        | 5758/37962 [13:22:34<73:33:47,  8.22s/it]
 15%|█▌        | 5759/37962 [13:22:43<73:43:20,  8.24s/it]
 15%|█▌        | 5760/37962 [13:22:51<74:06:32,  8.28s/it]
                                                          
{'loss': 1.4193, 'grad_norm': 6.463188171386719, 'learning_rate': 1.9232866643720497e-05, 'epoch': 0.15}

 15%|█▌        | 5760/37962 [13:22:51<74:06:32,  8.28s/it]
 15%|█▌        | 5761/37962 [13:22:59<73:59:32,  8.27s/it]
 15%|█▌        | 5762/37962 [13:23:07<73:19:47,  8.20s/it]
 15%|█▌        | 5763/37962 [13:23:15<73:17:01,  8.19s/it]
 15%|█▌        | 5764/37962 [13:23:24<74:15:13,  8.30s/it]
 15%|█▌        | 5765/37962 [13:23:32<74:30:21,  8.33s/it]
 15%|█▌        | 5766/37962 [13:23:40<73:40:49,  8.24s/it]
 15%|█▌        | 5767/37962 [13:23:49<73:42:51,  8.24s/it]
 15%|█▌        | 5768/37962 [13:23:57<73:34:23,  8.23s/it]
 15%|█▌        | 5769/37962 [13:24:05<74:04:25,  8.28s/it]
 15%|█▌        | 5770/37962 [13:24:13<73:48:19,  8.25s/it]
                                                          
{'loss': 1.4187, 'grad_norm': 7.712193012237549, 'learning_rate': 1.9229586195558665e-05, 'epoch': 0.15}

 15%|█▌        | 5770/37962 [13:24:13<73:48:19,  8.25s/it]
 15%|█▌        | 5771/37962 [13:24:21<72:51:20,  8.15s/it]
 15%|█▌        | 5772/37962 [13:24:30<72:58:36,  8.16s/it]
 15%|█▌        | 5773/37962 [13:24:38<73:22:33,  8.21s/it]
 15%|█▌        | 5774/37962 [13:24:46<73:20:47,  8.20s/it]
 15%|█▌        | 5775/37962 [13:24:54<72:37:27,  8.12s/it]
 15%|█▌        | 5776/37962 [13:25:03<73:38:26,  8.24s/it]
 15%|█▌        | 5777/37962 [13:25:11<74:44:24,  8.36s/it]
 15%|█▌        | 5778/37962 [13:25:19<74:21:23,  8.32s/it]
 15%|█▌        | 5779/37962 [13:25:28<74:08:23,  8.29s/it]
 15%|█▌        | 5780/37962 [13:25:36<74:22:10,  8.32s/it]
                                                          
{'loss': 1.4172, 'grad_norm': 12.221109390258789, 'learning_rate': 1.922629902934093e-05, 'epoch': 0.15}

 15%|█▌        | 5780/37962 [13:25:36<74:22:10,  8.32s/it]
 15%|█▌        | 5781/37962 [13:25:44<73:44:00,  8.25s/it]
 15%|█▌        | 5782/37962 [13:25:52<73:14:00,  8.19s/it]
 15%|█▌        | 5783/37962 [13:26:00<73:21:26,  8.21s/it]
 15%|█▌        | 5784/37962 [13:26:09<73:42:34,  8.25s/it]
 15%|█▌        | 5785/37962 [13:26:17<73:48:39,  8.26s/it]
 15%|█▌        | 5786/37962 [13:26:25<73:50:23,  8.26s/it]
 15%|█▌        | 5787/37962 [13:26:34<74:50:07,  8.37s/it]
 15%|█▌        | 5788/37962 [13:26:42<74:44:38,  8.36s/it]
 15%|█▌        | 5789/37962 [13:26:51<74:47:43,  8.37s/it]
 15%|█▌        | 5790/37962 [13:26:59<74:38:46,  8.35s/it]
                                                          
{'loss': 1.3494, 'grad_norm': 13.509597778320312, 'learning_rate': 1.9223005147459968e-05, 'epoch': 0.15}

 15%|█▌        | 5790/37962 [13:26:59<74:38:46,  8.35s/it]
 15%|█▌        | 5791/37962 [13:27:07<74:20:27,  8.32s/it]
 15%|█▌        | 5792/37962 [13:27:15<74:07:11,  8.29s/it]
 15%|█▌        | 5793/37962 [13:27:24<73:48:38,  8.26s/it]
 15%|█▌        | 5794/37962 [13:27:32<74:13:52,  8.31s/it]
 15%|█▌        | 5795/37962 [13:27:40<74:22:47,  8.32s/it]
 15%|█▌        | 5796/37962 [13:27:49<74:28:41,  8.34s/it]
 15%|█▌        | 5797/37962 [13:27:57<74:57:42,  8.39s/it]
 15%|█▌        | 5798/37962 [13:28:06<75:18:54,  8.43s/it]
 15%|█▌        | 5799/37962 [13:28:14<74:44:07,  8.37s/it]
 15%|█▌        | 5800/37962 [13:28:22<74:26:40,  8.33s/it]
                                                          
{'loss': 1.5367, 'grad_norm': 42.09486770629883, 'learning_rate': 1.9219704552313335e-05, 'epoch': 0.15}

 15%|█▌        | 5800/37962 [13:28:22<74:26:40,  8.33s/it]
 15%|█▌        | 5801/37962 [13:28:31<74:37:13,  8.35s/it]
 15%|█▌        | 5802/37962 [13:28:39<74:40:36,  8.36s/it]
 15%|█▌        | 5803/37962 [13:28:47<74:27:07,  8.33s/it]
 15%|█▌        | 5804/37962 [13:28:55<73:25:56,  8.22s/it]
 15%|█▌        | 5805/37962 [13:29:04<73:42:57,  8.25s/it]
 15%|█▌        | 5806/37962 [13:29:12<73:32:06,  8.23s/it]
 15%|█▌        | 5807/37962 [13:29:20<73:52:58,  8.27s/it]
 15%|█▌        | 5808/37962 [13:29:28<73:47:05,  8.26s/it]
 15%|█▌        | 5809/37962 [13:29:37<73:32:34,  8.23s/it]
 15%|█▌        | 5810/37962 [13:29:45<73:30:39,  8.23s/it]
                                                          
{'loss': 1.4669, 'grad_norm': 12.472993850708008, 'learning_rate': 1.9216397246303482e-05, 'epoch': 0.15}

 15%|█▌        | 5810/37962 [13:29:45<73:30:39,  8.23s/it]
 15%|█▌        | 5811/37962 [13:29:53<73:34:08,  8.24s/it]
 15%|█▌        | 5812/37962 [13:30:01<73:18:29,  8.21s/it]
 15%|█▌        | 5813/37962 [13:30:10<73:39:51,  8.25s/it]
 15%|█▌        | 5814/37962 [13:30:18<73:44:40,  8.26s/it]
 15%|█▌        | 5815/37962 [13:30:26<73:49:20,  8.27s/it]
 15%|█▌        | 5816/37962 [13:30:34<73:45:30,  8.26s/it]
 15%|█▌        | 5817/37962 [13:30:43<73:38:02,  8.25s/it]
 15%|█▌        | 5818/37962 [13:30:51<73:35:42,  8.24s/it]
 15%|█▌        | 5819/37962 [13:30:59<73:45:55,  8.26s/it]
 15%|█▌        | 5820/37962 [13:31:07<73:39:53,  8.25s/it]
                                                          
{'loss': 1.4118, 'grad_norm': 6.267317771911621, 'learning_rate': 1.9213083231837733e-05, 'epoch': 0.15}

 15%|█▌        | 5820/37962 [13:31:07<73:39:53,  8.25s/it]
 15%|█▌        | 5821/37962 [13:31:16<73:35:30,  8.24s/it]
 15%|█▌        | 5822/37962 [13:31:24<73:24:24,  8.22s/it]
 15%|█▌        | 5823/37962 [13:31:32<73:42:55,  8.26s/it]
 15%|█▌        | 5824/37962 [13:31:40<72:42:09,  8.14s/it]
 15%|█▌        | 5825/37962 [13:31:48<73:03:36,  8.18s/it]
 15%|█▌        | 5826/37962 [13:31:56<73:10:08,  8.20s/it]
 15%|█▌        | 5827/37962 [13:32:05<73:30:08,  8.23s/it]
 15%|█▌        | 5828/37962 [13:32:13<73:43:10,  8.26s/it]
 15%|█▌        | 5829/37962 [13:32:21<73:41:54,  8.26s/it]
 15%|█▌        | 5830/37962 [13:32:29<73:19:54,  8.22s/it]
                                                          
{'loss': 1.4817, 'grad_norm': 25.158239364624023, 'learning_rate': 1.9209762511328308e-05, 'epoch': 0.15}

 15%|█▌        | 5830/37962 [13:32:29<73:19:54,  8.22s/it]
 15%|█▌        | 5831/37962 [13:32:38<73:34:49,  8.24s/it]
 15%|█▌        | 5832/37962 [13:32:46<73:53:40,  8.28s/it]
 15%|█▌        | 5833/37962 [13:32:54<73:17:36,  8.21s/it]
 15%|█▌        | 5834/37962 [13:33:03<74:07:19,  8.31s/it]
 15%|█▌        | 5835/37962 [13:33:11<73:51:50,  8.28s/it]
 15%|█▌        | 5836/37962 [13:33:19<73:16:04,  8.21s/it]
 15%|█▌        | 5837/37962 [13:33:27<73:52:30,  8.28s/it]
 15%|█▌        | 5838/37962 [13:33:36<73:59:02,  8.29s/it]
 15%|█▌        | 5839/37962 [13:33:44<74:08:38,  8.31s/it]
 15%|█▌        | 5840/37962 [13:33:52<74:15:57,  8.32s/it]
                                                          
{'loss': 1.3603, 'grad_norm': 3.0872983932495117, 'learning_rate': 1.9206435087192296e-05, 'epoch': 0.15}

 15%|█▌        | 5840/37962 [13:33:52<74:15:57,  8.32s/it]
 15%|█▌        | 5841/37962 [13:34:01<74:10:28,  8.31s/it]
 15%|█▌        | 5842/37962 [13:34:09<74:06:33,  8.31s/it]
 15%|█▌        | 5843/37962 [13:34:17<74:04:41,  8.30s/it]
 15%|█▌        | 5844/37962 [13:34:26<73:54:51,  8.28s/it]
 15%|█▌        | 5845/37962 [13:34:34<73:28:55,  8.24s/it]
 15%|█▌        | 5846/37962 [13:34:42<73:12:52,  8.21s/it]
 15%|█▌        | 5847/37962 [13:34:50<72:18:24,  8.11s/it]
 15%|█▌        | 5848/37962 [13:34:58<73:15:10,  8.21s/it]
 15%|█▌        | 5849/37962 [13:35:06<73:26:55,  8.23s/it]
 15%|█▌        | 5850/37962 [13:35:15<73:19:09,  8.22s/it]
                                                          
{'loss': 1.3983, 'grad_norm': 14.03447151184082, 'learning_rate': 1.9203100961851678e-05, 'epoch': 0.15}

 15%|█▌        | 5850/37962 [13:35:15<73:19:09,  8.22s/it]
 15%|█▌        | 5851/37962 [13:35:23<72:42:31,  8.15s/it]
 15%|█▌        | 5852/37962 [13:35:31<72:55:57,  8.18s/it]
 15%|█▌        | 5853/37962 [13:35:39<73:18:00,  8.22s/it]
 15%|█▌        | 5854/37962 [13:35:47<72:34:24,  8.14s/it]
 15%|█▌        | 5855/37962 [13:35:55<72:54:55,  8.18s/it]
 15%|█▌        | 5856/37962 [13:36:04<73:10:57,  8.21s/it]
 15%|█▌        | 5857/37962 [13:36:12<72:46:10,  8.16s/it]
 15%|█▌        | 5858/37962 [13:36:20<72:40:25,  8.15s/it]
 15%|█▌        | 5859/37962 [13:36:28<73:06:32,  8.20s/it]
 15%|█▌        | 5860/37962 [13:36:37<73:50:57,  8.28s/it]
                                                          
{'loss': 1.4053, 'grad_norm': 28.994007110595703, 'learning_rate': 1.9199760137733303e-05, 'epoch': 0.15}

 15%|█▌        | 5860/37962 [13:36:37<73:50:57,  8.28s/it]
 15%|█▌        | 5861/37962 [13:36:45<73:40:08,  8.26s/it]
 15%|█▌        | 5862/37962 [13:36:53<74:02:13,  8.30s/it]
 15%|█▌        | 5863/37962 [13:37:02<74:39:28,  8.37s/it]
 15%|█▌        | 5864/37962 [13:37:10<74:05:40,  8.31s/it]
 15%|█▌        | 5865/37962 [13:37:18<74:14:53,  8.33s/it]
 15%|█▌        | 5866/37962 [13:37:26<73:34:16,  8.25s/it]
 15%|█▌        | 5867/37962 [13:37:35<73:54:44,  8.29s/it]
 15%|█▌        | 5868/37962 [13:37:43<74:08:27,  8.32s/it]
 15%|█▌        | 5869/37962 [13:37:51<73:52:23,  8.29s/it]
 15%|█▌        | 5870/37962 [13:38:00<73:41:27,  8.27s/it]
                                                          
{'loss': 1.4518, 'grad_norm': 13.246676445007324, 'learning_rate': 1.9196412617268895e-05, 'epoch': 0.15}

 15%|█▌        | 5870/37962 [13:38:00<73:41:27,  8.27s/it]
 15%|█▌        | 5871/37962 [13:38:08<73:24:26,  8.23s/it]
 15%|█▌        | 5872/37962 [13:38:16<72:36:22,  8.15s/it]
 15%|█▌        | 5873/37962 [13:38:24<72:16:18,  8.11s/it]
 15%|█▌        | 5874/37962 [13:38:32<72:11:04,  8.10s/it]
 15%|█▌        | 5875/37962 [13:38:40<71:53:05,  8.07s/it]
 15%|█▌        | 5876/37962 [13:38:48<72:21:11,  8.12s/it]
 15%|█▌        | 5877/37962 [13:38:56<72:41:19,  8.16s/it]
 15%|█▌        | 5878/37962 [13:39:04<72:19:29,  8.12s/it]
 15%|█▌        | 5879/37962 [13:39:13<73:24:31,  8.24s/it]
 15%|█▌        | 5880/37962 [13:39:21<73:12:58,  8.22s/it]
                                                          
{'loss': 1.3854, 'grad_norm': 18.557640075683594, 'learning_rate': 1.9193058402895063e-05, 'epoch': 0.15}

 15%|█▌        | 5880/37962 [13:39:21<73:12:58,  8.22s/it]
 15%|█▌        | 5881/37962 [13:39:29<73:23:27,  8.24s/it]
 15%|█▌        | 5882/37962 [13:39:38<73:33:39,  8.25s/it]
 15%|█▌        | 5883/37962 [13:39:46<74:17:50,  8.34s/it]
 15%|█▌        | 5884/37962 [13:39:54<73:21:57,  8.23s/it]
 16%|█▌        | 5885/37962 [13:40:02<73:42:04,  8.27s/it]
 16%|█▌        | 5886/37962 [13:40:11<73:46:58,  8.28s/it]
 16%|█▌        | 5887/37962 [13:40:19<73:55:32,  8.30s/it]
 16%|█▌        | 5888/37962 [13:40:27<72:42:34,  8.16s/it]
 16%|█▌        | 5889/37962 [13:40:35<72:38:41,  8.15s/it]
 16%|█▌        | 5890/37962 [13:40:43<72:54:10,  8.18s/it]
                                                          
{'loss': 1.3978, 'grad_norm': 19.95437240600586, 'learning_rate': 1.9189697497053277e-05, 'epoch': 0.16}

 16%|█▌        | 5890/37962 [13:40:43<72:54:10,  8.18s/it]
 16%|█▌        | 5891/37962 [13:40:52<73:21:08,  8.23s/it]
 16%|█▌        | 5892/37962 [13:41:00<72:40:06,  8.16s/it]
 16%|█▌        | 5893/37962 [13:41:08<73:09:07,  8.21s/it]
 16%|█▌        | 5894/37962 [13:41:16<73:24:51,  8.24s/it]
 16%|█▌        | 5895/37962 [13:41:24<72:57:25,  8.19s/it]
 16%|█▌        | 5896/37962 [13:41:33<73:21:35,  8.24s/it]
 16%|█▌        | 5897/37962 [13:41:41<73:38:28,  8.27s/it]
 16%|█▌        | 5898/37962 [13:41:49<73:04:59,  8.21s/it]
 16%|█▌        | 5899/37962 [13:41:57<72:16:17,  8.11s/it]
 16%|█▌        | 5900/37962 [13:42:06<73:24:36,  8.24s/it]
                                                          
{'loss': 1.4218, 'grad_norm': 16.190902709960938, 'learning_rate': 1.9186329902189885e-05, 'epoch': 0.16}

 16%|█▌        | 5900/37962 [13:42:06<73:24:36,  8.24s/it]
 16%|█▌        | 5901/37962 [13:42:14<73:35:33,  8.26s/it]
 16%|█▌        | 5902/37962 [13:42:22<73:30:20,  8.25s/it]
 16%|█▌        | 5903/37962 [13:42:30<73:45:46,  8.28s/it]
 16%|█▌        | 5904/37962 [13:42:39<73:42:24,  8.28s/it]
 16%|█▌        | 5905/37962 [13:42:47<73:38:21,  8.27s/it]
 16%|█▌        | 5906/37962 [13:42:56<74:36:39,  8.38s/it]
 16%|█▌        | 5907/37962 [13:43:04<74:06:58,  8.32s/it]
 16%|█▌        | 5908/37962 [13:43:12<73:41:54,  8.28s/it]
 16%|█▌        | 5909/37962 [13:43:20<73:10:19,  8.22s/it]
 16%|█▌        | 5910/37962 [13:43:28<73:44:16,  8.28s/it]
                                                          
{'loss': 1.4526, 'grad_norm': 12.3582763671875, 'learning_rate': 1.9182955620756098e-05, 'epoch': 0.16}

 16%|█▌        | 5910/37962 [13:43:28<73:44:16,  8.28s/it]
 16%|█▌        | 5911/37962 [13:43:37<73:37:33,  8.27s/it]
 16%|█▌        | 5912/37962 [13:43:45<73:43:01,  8.28s/it]
 16%|█▌        | 5913/37962 [13:43:53<73:14:07,  8.23s/it]
 16%|█▌        | 5914/37962 [13:44:01<73:34:59,  8.27s/it]
 16%|█▌        | 5915/37962 [13:44:10<73:31:10,  8.26s/it]
 16%|█▌        | 5916/37962 [13:44:18<73:23:33,  8.24s/it]
 16%|█▌        | 5917/37962 [13:44:26<73:48:46,  8.29s/it]
 16%|█▌        | 5918/37962 [13:44:35<74:11:25,  8.33s/it]
 16%|█▌        | 5919/37962 [13:44:43<73:58:37,  8.31s/it]
 16%|█▌        | 5920/37962 [13:44:51<73:35:50,  8.27s/it]
                                                          
{'loss': 1.4434, 'grad_norm': 19.59125518798828, 'learning_rate': 1.9179574655208e-05, 'epoch': 0.16}

 16%|█▌        | 5920/37962 [13:44:51<73:35:50,  8.27s/it]
 16%|█▌        | 5921/37962 [13:44:59<73:30:09,  8.26s/it]
 16%|█▌        | 5922/37962 [13:45:08<73:19:02,  8.24s/it]
 16%|█▌        | 5923/37962 [13:45:16<72:50:26,  8.18s/it]
 16%|█▌        | 5924/37962 [13:45:24<72:47:47,  8.18s/it]
 16%|█▌        | 5925/37962 [13:45:32<71:42:53,  8.06s/it]
 16%|█▌        | 5926/37962 [13:45:39<71:10:30,  8.00s/it]
 16%|█▌        | 5927/37962 [13:45:48<71:49:40,  8.07s/it]
 16%|█▌        | 5928/37962 [13:45:56<71:58:57,  8.09s/it]
 16%|█▌        | 5929/37962 [13:46:04<71:48:57,  8.07s/it]
 16%|█▌        | 5930/37962 [13:46:12<71:40:08,  8.05s/it]
                                                          
{'loss': 1.5077, 'grad_norm': 7.594522476196289, 'learning_rate': 1.9176187008006534e-05, 'epoch': 0.16}

 16%|█▌        | 5930/37962 [13:46:12<71:40:08,  8.05s/it]
 16%|█▌        | 5931/37962 [13:46:20<71:58:46,  8.09s/it]
 16%|█▌        | 5932/37962 [13:46:28<72:26:36,  8.14s/it]
 16%|█▌        | 5933/37962 [13:46:37<72:51:08,  8.19s/it]
 16%|█▌        | 5934/37962 [13:46:45<73:34:01,  8.27s/it]
 16%|█▌        | 5935/37962 [13:46:53<72:48:59,  8.18s/it]
 16%|█▌        | 5936/37962 [13:47:01<72:55:19,  8.20s/it]
 16%|█▌        | 5937/37962 [13:47:10<73:04:50,  8.22s/it]
 16%|█▌        | 5938/37962 [13:47:18<73:24:46,  8.25s/it]
 16%|█▌        | 5939/37962 [13:47:26<73:41:16,  8.28s/it]
 16%|█▌        | 5940/37962 [13:47:34<73:16:56,  8.24s/it]
                                                          
{'loss': 1.5387, 'grad_norm': 13.605952262878418, 'learning_rate': 1.9172792681617508e-05, 'epoch': 0.16}

 16%|█▌        | 5940/37962 [13:47:34<73:16:56,  8.24s/it]
 16%|█▌        | 5941/37962 [13:47:42<72:46:00,  8.18s/it]
 16%|█▌        | 5942/37962 [13:47:51<73:47:48,  8.30s/it]
 16%|█▌        | 5943/37962 [13:47:59<73:22:09,  8.25s/it]
 16%|█▌        | 5944/37962 [13:48:07<73:35:24,  8.27s/it]
 16%|█▌        | 5945/37962 [13:48:16<73:42:59,  8.29s/it]
 16%|█▌        | 5946/37962 [13:48:24<73:30:58,  8.27s/it]
 16%|█▌        | 5947/37962 [13:48:32<73:48:00,  8.30s/it]
 16%|█▌        | 5948/37962 [13:48:41<73:37:00,  8.28s/it]
 16%|█▌        | 5949/37962 [13:48:49<74:09:07,  8.34s/it]
 16%|█▌        | 5950/37962 [13:48:57<73:44:59,  8.29s/it]
                                                          
{'loss': 1.5417, 'grad_norm': 10.045517921447754, 'learning_rate': 1.91693916785116e-05, 'epoch': 0.16}

 16%|█▌        | 5950/37962 [13:48:57<73:44:59,  8.29s/it]
 16%|█▌        | 5951/37962 [13:49:06<73:53:36,  8.31s/it]
 16%|█▌        | 5952/37962 [13:49:14<74:04:12,  8.33s/it]
 16%|█▌        | 5953/37962 [13:49:22<73:12:15,  8.23s/it]
 16%|█▌        | 5954/37962 [13:49:30<73:05:00,  8.22s/it]
 16%|█▌        | 5955/37962 [13:49:38<73:02:05,  8.21s/it]
 16%|█▌        | 5956/37962 [13:49:47<73:10:24,  8.23s/it]
 16%|█▌        | 5957/37962 [13:49:55<73:39:56,  8.29s/it]
 16%|█▌        | 5958/37962 [13:50:03<73:33:40,  8.27s/it]
 16%|█▌        | 5959/37962 [13:50:12<73:35:57,  8.28s/it]
 16%|█▌        | 5960/37962 [13:50:20<73:31:27,  8.27s/it]
                                                          
{'loss': 1.4175, 'grad_norm': 14.648824691772461, 'learning_rate': 1.916598400116434e-05, 'epoch': 0.16}

 16%|█▌        | 5960/37962 [13:50:20<73:31:27,  8.27s/it]
 16%|█▌        | 5961/37962 [13:50:28<73:30:18,  8.27s/it]
 16%|█▌        | 5962/37962 [13:50:36<73:32:17,  8.27s/it]
 16%|█▌        | 5963/37962 [13:50:45<73:28:19,  8.27s/it]
 16%|█▌        | 5964/37962 [13:50:53<73:33:07,  8.28s/it]
 16%|█▌        | 5965/37962 [13:51:01<74:09:53,  8.34s/it]
 16%|█▌        | 5966/37962 [13:51:10<74:14:23,  8.35s/it]
 16%|█▌        | 5967/37962 [13:51:18<73:18:47,  8.25s/it]
 16%|█▌        | 5968/37962 [13:51:26<73:46:12,  8.30s/it]
 16%|█▌        | 5969/37962 [13:51:35<73:58:26,  8.32s/it]
 16%|█▌        | 5970/37962 [13:51:43<73:54:36,  8.32s/it]
                                                          
{'loss': 1.4761, 'grad_norm': 11.95168685913086, 'learning_rate': 1.9162569652056114e-05, 'epoch': 0.16}

 16%|█▌        | 5970/37962 [13:51:43<73:54:36,  8.32s/it]
 16%|█▌        | 5971/37962 [13:51:51<73:26:22,  8.26s/it]
 16%|█▌        | 5972/37962 [13:51:59<73:43:37,  8.30s/it]
 16%|█▌        | 5973/37962 [13:52:08<74:25:50,  8.38s/it]
 16%|█▌        | 5974/37962 [13:52:16<73:01:37,  8.22s/it]
 16%|█▌        | 5975/37962 [13:52:24<73:03:04,  8.22s/it]
 16%|█▌        | 5976/37962 [13:52:33<73:38:21,  8.29s/it]
 16%|█▌        | 5977/37962 [13:52:41<73:08:00,  8.23s/it]
 16%|█▌        | 5978/37962 [13:52:49<72:34:04,  8.17s/it]
 16%|█▌        | 5979/37962 [13:52:57<72:56:16,  8.21s/it]
 16%|█▌        | 5980/37962 [13:53:05<73:02:46,  8.22s/it]
                                                          
{'loss': 1.4737, 'grad_norm': 6.80690860748291, 'learning_rate': 1.915914863367217e-05, 'epoch': 0.16}

 16%|█▌        | 5980/37962 [13:53:05<73:02:46,  8.22s/it]
 16%|█▌        | 5981/37962 [13:53:14<73:17:16,  8.25s/it]
 16%|█▌        | 5982/37962 [13:53:21<72:11:51,  8.13s/it]
 16%|█▌        | 5983/37962 [13:53:30<72:39:48,  8.18s/it]
 16%|█▌        | 5984/37962 [13:53:38<72:51:26,  8.20s/it]
 16%|█▌        | 5985/37962 [13:53:46<73:25:28,  8.27s/it]
 16%|█▌        | 5986/37962 [13:53:55<73:45:27,  8.30s/it]
 16%|█▌        | 5987/37962 [13:54:03<73:50:43,  8.31s/it]
 16%|█▌        | 5988/37962 [13:54:11<73:41:33,  8.30s/it]
 16%|█▌        | 5989/37962 [13:54:20<73:35:00,  8.29s/it]
 16%|█▌        | 5990/37962 [13:54:28<74:06:59,  8.35s/it]
                                                          
{'loss': 1.5266, 'grad_norm': 19.92470359802246, 'learning_rate': 1.9155720948502607e-05, 'epoch': 0.16}

 16%|█▌        | 5990/37962 [13:54:28<74:06:59,  8.35s/it]
 16%|█▌        | 5991/37962 [13:54:36<73:46:07,  8.31s/it]
 16%|█▌        | 5992/37962 [13:54:45<73:53:02,  8.32s/it]
 16%|█▌        | 5993/37962 [13:54:53<73:06:54,  8.23s/it]
 16%|█▌        | 5994/37962 [13:55:01<72:59:01,  8.22s/it]
 16%|█▌        | 5995/37962 [13:55:09<72:52:50,  8.21s/it]
 16%|█▌        | 5996/37962 [13:55:17<72:43:18,  8.19s/it]
 16%|█▌        | 5997/37962 [13:55:25<72:35:31,  8.18s/it]
 16%|█▌        | 5998/37962 [13:55:34<73:16:25,  8.25s/it]
 16%|█▌        | 5999/37962 [13:55:42<73:30:21,  8.28s/it]
 16%|█▌        | 6000/37962 [13:55:50<73:19:06,  8.26s/it]
                                                          
{'loss': 1.4909, 'grad_norm': 7.003292560577393, 'learning_rate': 1.9152286599042376e-05, 'epoch': 0.16}

 16%|█▌        | 6000/37962 [13:55:50<73:19:06,  8.26s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 16%|█▌        | 6001/37962 [13:56:42<188:38:48, 21.25s/it]
 16%|█▌        | 6002/37962 [13:56:50<155:03:08, 17.47s/it]
 16%|█▌        | 6003/37962 [13:56:59<130:50:09, 14.74s/it]
 16%|█▌        | 6004/37962 [13:57:07<113:09:29, 12.75s/it]
 16%|█▌        | 6005/37962 [13:57:15<101:25:36, 11.43s/it]
 16%|█▌        | 6006/37962 [13:57:24<92:58:44, 10.47s/it] 
 16%|█▌        | 6007/37962 [13:57:32<87:36:49,  9.87s/it]
 16%|█▌        | 6008/37962 [13:57:40<83:18:00,  9.38s/it]
 16%|█▌        | 6009/37962 [13:57:48<79:55:23,  9.00s/it]
 16%|█▌        | 6010/37962 [13:57:57<77:55:27,  8.78s/it]
                                                          
{'loss': 1.5475, 'grad_norm': 29.643287658691406, 'learning_rate': 1.9148845587791282e-05, 'epoch': 0.16}

 16%|█▌        | 6010/37962 [13:57:57<77:55:27,  8.78s/it]
 16%|█▌        | 6011/37962 [13:58:05<76:43:06,  8.64s/it]
 16%|█▌        | 6012/37962 [13:58:13<76:10:06,  8.58s/it]
 16%|█▌        | 6013/37962 [13:58:22<75:32:22,  8.51s/it]
 16%|█▌        | 6014/37962 [13:58:30<75:18:41,  8.49s/it]
 16%|█▌        | 6015/37962 [13:58:38<74:38:38,  8.41s/it]
 16%|█▌        | 6016/37962 [13:58:47<74:12:15,  8.36s/it]
 16%|█▌        | 6017/37962 [13:58:55<73:50:07,  8.32s/it]
 16%|█▌        | 6018/37962 [13:59:03<73:17:57,  8.26s/it]
 16%|█▌        | 6019/37962 [13:59:11<73:15:31,  8.26s/it]
 16%|█▌        | 6020/37962 [13:59:20<73:15:37,  8.26s/it]
                                                          
{'loss': 1.5377, 'grad_norm': 7.8319993019104, 'learning_rate': 1.9145397917253977e-05, 'epoch': 0.16}

 16%|█▌        | 6020/37962 [13:59:20<73:15:37,  8.26s/it]
 16%|█▌        | 6021/37962 [13:59:28<73:15:20,  8.26s/it]
 16%|█▌        | 6022/37962 [13:59:36<73:11:11,  8.25s/it]
 16%|█▌        | 6023/37962 [13:59:44<72:51:31,  8.21s/it]
 16%|█▌        | 6024/37962 [13:59:53<73:20:45,  8.27s/it]
 16%|█▌        | 6025/37962 [14:00:01<73:08:12,  8.24s/it]
 16%|█▌        | 6026/37962 [14:00:09<73:01:03,  8.23s/it]
 16%|█▌        | 6027/37962 [14:00:17<72:37:54,  8.19s/it]
 16%|█▌        | 6028/37962 [14:00:25<72:13:36,  8.14s/it]
 16%|█▌        | 6029/37962 [14:00:34<73:10:18,  8.25s/it]
 16%|█▌        | 6030/37962 [14:00:42<72:54:29,  8.22s/it]
                                                          
{'loss': 1.4419, 'grad_norm': 7.99707555770874, 'learning_rate': 1.9141943589939964e-05, 'epoch': 0.16}

 16%|█▌        | 6030/37962 [14:00:42<72:54:29,  8.22s/it]
 16%|█▌        | 6031/37962 [14:00:50<73:04:52,  8.24s/it]
 16%|█▌        | 6032/37962 [14:00:58<72:41:00,  8.19s/it]
 16%|█▌        | 6033/37962 [14:01:06<73:14:08,  8.26s/it]
 16%|█▌        | 6034/37962 [14:01:14<72:03:51,  8.13s/it]
 16%|█▌        | 6035/37962 [14:01:23<72:19:04,  8.15s/it]
 16%|█▌        | 6036/37962 [14:01:31<72:56:48,  8.23s/it]
 16%|█▌        | 6037/37962 [14:01:39<72:34:52,  8.18s/it]
 16%|█▌        | 6038/37962 [14:01:47<72:46:52,  8.21s/it]
 16%|█▌        | 6039/37962 [14:01:55<72:41:56,  8.20s/it]
 16%|█▌        | 6040/37962 [14:02:04<72:46:36,  8.21s/it]
                                                          
{'loss': 1.4792, 'grad_norm': 16.28982925415039, 'learning_rate': 1.9138482608363583e-05, 'epoch': 0.16}

 16%|█▌        | 6040/37962 [14:02:04<72:46:36,  8.21s/it]
 16%|█▌        | 6041/37962 [14:02:12<72:21:08,  8.16s/it]
 16%|█▌        | 6042/37962 [14:02:20<73:06:02,  8.24s/it]
 16%|█▌        | 6043/37962 [14:02:28<73:10:30,  8.25s/it]
 16%|█▌        | 6044/37962 [14:02:37<73:04:46,  8.24s/it]
 16%|█▌        | 6045/37962 [14:02:45<73:10:21,  8.25s/it]
 16%|█▌        | 6046/37962 [14:02:53<72:49:57,  8.22s/it]
 16%|█▌        | 6047/37962 [14:03:01<72:37:25,  8.19s/it]
 16%|█▌        | 6048/37962 [14:03:09<72:52:16,  8.22s/it]
 16%|█▌        | 6049/37962 [14:03:18<72:50:42,  8.22s/it]
 16%|█▌        | 6050/37962 [14:03:26<73:13:15,  8.26s/it]
                                                          
{'loss': 1.4198, 'grad_norm': 7.003361225128174, 'learning_rate': 1.9135014975044024e-05, 'epoch': 0.16}

 16%|█▌        | 6050/37962 [14:03:26<73:13:15,  8.26s/it]
 16%|█▌        | 6051/37962 [14:03:34<72:19:37,  8.16s/it]
 16%|█▌        | 6052/37962 [14:03:42<72:20:56,  8.16s/it]
 16%|█▌        | 6053/37962 [14:03:51<72:55:12,  8.23s/it]
 16%|█▌        | 6054/37962 [14:03:59<72:55:23,  8.23s/it]
 16%|█▌        | 6055/37962 [14:04:07<73:03:40,  8.24s/it]
 16%|█▌        | 6056/37962 [14:04:15<73:04:34,  8.25s/it]
 16%|█▌        | 6057/37962 [14:04:23<72:21:40,  8.16s/it]
 16%|█▌        | 6058/37962 [14:04:32<72:58:39,  8.23s/it]
 16%|█▌        | 6059/37962 [14:04:40<72:23:56,  8.17s/it]
 16%|█▌        | 6060/37962 [14:04:48<72:27:59,  8.18s/it]
                                                          
{'loss': 1.457, 'grad_norm': 11.890801429748535, 'learning_rate': 1.9131540692505316e-05, 'epoch': 0.16}

 16%|█▌        | 6060/37962 [14:04:48<72:27:59,  8.18s/it]
 16%|█▌        | 6061/37962 [14:04:56<71:35:26,  8.08s/it]
 16%|█▌        | 6062/37962 [14:05:04<72:02:28,  8.13s/it]
 16%|█▌        | 6063/37962 [14:05:12<71:46:03,  8.10s/it]
 16%|█▌        | 6064/37962 [14:05:20<71:41:33,  8.09s/it]
 16%|█▌        | 6065/37962 [14:05:28<72:01:29,  8.13s/it]
 16%|█▌        | 6066/37962 [14:05:36<71:50:38,  8.11s/it]
 16%|█▌        | 6067/37962 [14:05:45<72:08:46,  8.14s/it]
 16%|█▌        | 6068/37962 [14:05:53<72:08:56,  8.14s/it]
 16%|█▌        | 6069/37962 [14:06:01<72:40:35,  8.20s/it]
 16%|█▌        | 6070/37962 [14:06:09<73:07:28,  8.25s/it]
                                                          
{'loss': 1.4492, 'grad_norm': 9.749465942382812, 'learning_rate': 1.9128059763276336e-05, 'epoch': 0.16}

 16%|█▌        | 6070/37962 [14:06:09<73:07:28,  8.25s/it]
 16%|█▌        | 6071/37962 [14:06:18<72:49:32,  8.22s/it]
 16%|█▌        | 6072/37962 [14:06:26<72:50:19,  8.22s/it]
 16%|█▌        | 6073/37962 [14:06:34<73:18:38,  8.28s/it]
 16%|█▌        | 6074/37962 [14:06:43<73:26:03,  8.29s/it]
 16%|█▌        | 6075/37962 [14:06:51<73:14:49,  8.27s/it]
 16%|█▌        | 6076/37962 [14:06:59<73:04:03,  8.25s/it]
 16%|█▌        | 6077/37962 [14:07:07<73:03:19,  8.25s/it]
 16%|█▌        | 6078/37962 [14:07:15<72:50:42,  8.22s/it]
 16%|█▌        | 6079/37962 [14:07:24<73:10:45,  8.26s/it]
 16%|█▌        | 6080/37962 [14:07:32<73:17:56,  8.28s/it]
                                                          
{'loss': 1.4812, 'grad_norm': 24.678009033203125, 'learning_rate': 1.9124572189890787e-05, 'epoch': 0.16}

 16%|█▌        | 6080/37962 [14:07:32<73:17:56,  8.28s/it]
 16%|█▌        | 6081/37962 [14:07:40<72:42:28,  8.21s/it]
 16%|█▌        | 6082/37962 [14:07:48<72:35:59,  8.20s/it]
 16%|█▌        | 6083/37962 [14:07:56<72:34:45,  8.20s/it]
 16%|█▌        | 6084/37962 [14:08:05<72:58:41,  8.24s/it]
 16%|█▌        | 6085/37962 [14:08:13<72:26:06,  8.18s/it]
 16%|█▌        | 6086/37962 [14:08:21<72:22:48,  8.17s/it]
 16%|█▌        | 6087/37962 [14:08:29<72:30:22,  8.19s/it]
 16%|█▌        | 6088/37962 [14:08:37<72:44:34,  8.22s/it]
 16%|█▌        | 6089/37962 [14:08:46<73:24:26,  8.29s/it]
 16%|█▌        | 6090/37962 [14:08:54<72:54:42,  8.24s/it]
                                                          
{'loss': 1.4545, 'grad_norm': 24.09703826904297, 'learning_rate': 1.9121077974887213e-05, 'epoch': 0.16}

 16%|█▌        | 6090/37962 [14:08:54<72:54:42,  8.24s/it]
 16%|█▌        | 6091/37962 [14:09:02<72:32:55,  8.19s/it]
 16%|█▌        | 6092/37962 [14:09:11<73:00:59,  8.25s/it]
 16%|█▌        | 6093/37962 [14:09:18<71:59:25,  8.13s/it]
 16%|█▌        | 6094/37962 [14:09:27<71:57:14,  8.13s/it]
 16%|█▌        | 6095/37962 [14:09:35<72:22:54,  8.18s/it]
 16%|█▌        | 6096/37962 [14:09:43<73:17:00,  8.28s/it]
 16%|█▌        | 6097/37962 [14:09:52<73:14:22,  8.27s/it]
 16%|█▌        | 6098/37962 [14:10:00<72:57:08,  8.24s/it]
 16%|█▌        | 6099/37962 [14:10:08<72:41:34,  8.21s/it]
 16%|█▌        | 6100/37962 [14:10:16<72:32:12,  8.20s/it]
                                                          
{'loss': 1.5028, 'grad_norm': 5.335607051849365, 'learning_rate': 1.911757712080899e-05, 'epoch': 0.16}

 16%|█▌        | 6100/37962 [14:10:16<72:32:12,  8.20s/it]
 16%|█▌        | 6101/37962 [14:10:24<72:41:53,  8.21s/it]
 16%|█▌        | 6102/37962 [14:10:33<73:08:39,  8.26s/it]
 16%|█▌        | 6103/37962 [14:10:41<72:56:36,  8.24s/it]
 16%|█▌        | 6104/37962 [14:10:49<73:14:13,  8.28s/it]
 16%|█▌        | 6105/37962 [14:10:58<73:18:59,  8.29s/it]
 16%|█▌        | 6106/37962 [14:11:06<73:42:04,  8.33s/it]
 16%|█▌        | 6107/37962 [14:11:14<73:46:44,  8.34s/it]
 16%|█▌        | 6108/37962 [14:11:23<73:25:42,  8.30s/it]
 16%|█▌        | 6109/37962 [14:11:31<73:55:47,  8.36s/it]
 16%|█▌        | 6110/37962 [14:11:39<74:08:23,  8.38s/it]
                                                          
{'loss': 1.5123, 'grad_norm': 7.727433681488037, 'learning_rate': 1.9114069630204333e-05, 'epoch': 0.16}

 16%|█▌        | 6110/37962 [14:11:39<74:08:23,  8.38s/it]
 16%|█▌        | 6111/37962 [14:11:48<73:21:09,  8.29s/it]
 16%|█▌        | 6112/37962 [14:11:56<73:11:22,  8.27s/it]
 16%|█▌        | 6113/37962 [14:12:04<73:33:23,  8.31s/it]
 16%|█▌        | 6114/37962 [14:12:12<73:33:36,  8.32s/it]
 16%|█▌        | 6115/37962 [14:12:21<73:17:41,  8.29s/it]
 16%|█▌        | 6116/37962 [14:12:29<73:08:37,  8.27s/it]
 16%|█▌        | 6117/37962 [14:12:37<72:33:56,  8.20s/it]
 16%|█▌        | 6118/37962 [14:12:45<73:03:02,  8.26s/it]
 16%|█▌        | 6119/37962 [14:12:54<73:42:26,  8.33s/it]
 16%|█▌        | 6120/37962 [14:13:02<73:24:05,  8.30s/it]
                                                          
{'loss': 1.5112, 'grad_norm': 16.5827693939209, 'learning_rate': 1.9110555505626282e-05, 'epoch': 0.16}

 16%|█▌        | 6120/37962 [14:13:02<73:24:05,  8.30s/it]
 16%|█▌        | 6121/37962 [14:13:10<73:10:22,  8.27s/it]
 16%|█▌        | 6122/37962 [14:13:18<72:38:19,  8.21s/it]
 16%|█▌        | 6123/37962 [14:13:26<72:07:50,  8.16s/it]
 16%|█▌        | 6124/37962 [14:13:35<72:19:41,  8.18s/it]
 16%|█▌        | 6125/37962 [14:13:43<72:19:26,  8.18s/it]
 16%|█▌        | 6126/37962 [14:13:51<72:49:27,  8.23s/it]
 16%|█▌        | 6127/37962 [14:13:59<72:05:41,  8.15s/it]
 16%|█▌        | 6128/37962 [14:14:08<72:45:44,  8.23s/it]
 16%|█▌        | 6129/37962 [14:14:16<73:04:56,  8.26s/it]
 16%|█▌        | 6130/37962 [14:14:24<72:27:55,  8.20s/it]
                                                          
{'loss': 1.4472, 'grad_norm': 25.386280059814453, 'learning_rate': 1.9107034749632708e-05, 'epoch': 0.16}

 16%|█▌        | 6130/37962 [14:14:24<72:27:55,  8.20s/it]
 16%|█▌        | 6131/37962 [14:14:32<72:29:40,  8.20s/it]
 16%|█▌        | 6132/37962 [14:14:40<71:39:56,  8.11s/it]
 16%|█▌        | 6133/37962 [14:14:48<71:54:26,  8.13s/it]
 16%|█▌        | 6134/37962 [14:14:56<71:37:12,  8.10s/it]
 16%|█▌        | 6135/37962 [14:15:05<72:22:16,  8.19s/it]
 16%|█▌        | 6136/37962 [14:15:13<72:29:16,  8.20s/it]
 16%|█▌        | 6137/37962 [14:15:21<72:42:18,  8.22s/it]
 16%|█▌        | 6138/37962 [14:15:29<72:00:06,  8.14s/it]
 16%|█▌        | 6139/37962 [14:15:37<71:24:03,  8.08s/it]
 16%|█▌        | 6140/37962 [14:15:45<70:55:32,  8.02s/it]
                                                          
{'loss': 1.5119, 'grad_norm': 7.038825511932373, 'learning_rate': 1.9103507364786313e-05, 'epoch': 0.16}

 16%|█▌        | 6140/37962 [14:15:45<70:55:32,  8.02s/it]
 16%|█▌        | 6141/37962 [14:15:53<70:34:18,  7.98s/it]
 16%|█▌        | 6142/37962 [14:16:01<70:44:18,  8.00s/it]
 16%|█▌        | 6143/37962 [14:16:09<72:06:47,  8.16s/it]
 16%|█▌        | 6144/37962 [14:16:18<72:33:41,  8.21s/it]
 16%|█▌        | 6145/37962 [14:16:26<72:49:16,  8.24s/it]
 16%|█▌        | 6146/37962 [14:16:34<72:01:11,  8.15s/it]
 16%|█▌        | 6147/37962 [14:16:42<72:08:05,  8.16s/it]
 16%|█▌        | 6148/37962 [14:16:50<72:12:06,  8.17s/it]
 16%|█▌        | 6149/37962 [14:16:58<71:37:19,  8.10s/it]
 16%|█▌        | 6150/37962 [14:17:07<72:35:29,  8.21s/it]
                                                          
{'loss': 1.3824, 'grad_norm': 21.636554718017578, 'learning_rate': 1.9099973353654612e-05, 'epoch': 0.16}

 16%|█▌        | 6150/37962 [14:17:07<72:35:29,  8.21s/it]
 16%|█▌        | 6151/37962 [14:17:15<72:30:22,  8.21s/it]
 16%|█▌        | 6152/37962 [14:17:23<71:37:05,  8.11s/it]
 16%|█▌        | 6153/37962 [14:17:31<72:01:51,  8.15s/it]
 16%|█▌        | 6154/37962 [14:17:39<71:59:58,  8.15s/it]
 16%|█▌        | 6155/37962 [14:17:48<72:19:53,  8.19s/it]
 16%|█▌        | 6156/37962 [14:17:56<72:36:46,  8.22s/it]
 16%|█▌        | 6157/37962 [14:18:04<73:21:53,  8.30s/it]
 16%|█▌        | 6158/37962 [14:18:12<73:04:47,  8.27s/it]
 16%|█▌        | 6159/37962 [14:18:21<73:00:51,  8.26s/it]
 16%|█▌        | 6160/37962 [14:18:29<72:53:44,  8.25s/it]
                                                          
{'loss': 1.4172, 'grad_norm': 16.58153533935547, 'learning_rate': 1.909643271880995e-05, 'epoch': 0.16}

 16%|█▌        | 6160/37962 [14:18:29<72:53:44,  8.25s/it]
 16%|█▌        | 6161/37962 [14:18:37<72:04:58,  8.16s/it]
 16%|█▌        | 6162/37962 [14:18:45<71:05:46,  8.05s/it]
 16%|█▌        | 6163/37962 [14:18:53<72:00:33,  8.15s/it]
 16%|█▌        | 6164/37962 [14:19:02<72:51:52,  8.25s/it]
 16%|█▌        | 6165/37962 [14:19:10<73:00:13,  8.27s/it]
 16%|█▌        | 6166/37962 [14:19:18<72:09:00,  8.17s/it]
 16%|█▌        | 6167/37962 [14:19:26<72:58:05,  8.26s/it]
 16%|█▌        | 6168/37962 [14:19:34<72:08:32,  8.17s/it]
 16%|█▋        | 6169/37962 [14:19:43<72:23:34,  8.20s/it]
 16%|█▋        | 6170/37962 [14:19:51<72:29:51,  8.21s/it]
                                                          
{'loss': 1.4303, 'grad_norm': 12.029683113098145, 'learning_rate': 1.90928854628295e-05, 'epoch': 0.16}

 16%|█▋        | 6170/37962 [14:19:51<72:29:51,  8.21s/it]
 16%|█▋        | 6171/37962 [14:19:59<72:26:38,  8.20s/it]
 16%|█▋        | 6172/37962 [14:20:07<72:41:04,  8.23s/it]
 16%|█▋        | 6173/37962 [14:20:15<72:25:05,  8.20s/it]
 16%|█▋        | 6174/37962 [14:20:24<72:43:21,  8.24s/it]
 16%|█▋        | 6175/37962 [14:20:32<71:44:14,  8.12s/it]
 16%|█▋        | 6176/37962 [14:20:40<71:59:26,  8.15s/it]
 16%|█▋        | 6177/37962 [14:20:48<72:27:16,  8.21s/it]
 16%|█▋        | 6178/37962 [14:20:56<72:17:15,  8.19s/it]
 16%|█▋        | 6179/37962 [14:21:05<72:46:13,  8.24s/it]
 16%|█▋        | 6180/37962 [14:21:13<72:42:50,  8.24s/it]
                                                          
{'loss': 1.519, 'grad_norm': 15.57595443725586, 'learning_rate': 1.9089331588295243e-05, 'epoch': 0.16}

 16%|█▋        | 6180/37962 [14:21:13<72:42:50,  8.24s/it]
 16%|█▋        | 6181/37962 [14:21:21<73:08:16,  8.28s/it]
 16%|█▋        | 6182/37962 [14:21:30<73:18:11,  8.30s/it]
 16%|█▋        | 6183/37962 [14:21:38<72:23:45,  8.20s/it]
 16%|█▋        | 6184/37962 [14:21:46<72:46:22,  8.24s/it]
 16%|█▋        | 6185/37962 [14:21:54<72:09:10,  8.17s/it]
 16%|█▋        | 6186/37962 [14:22:02<72:38:31,  8.23s/it]
 16%|█▋        | 6187/37962 [14:22:10<72:40:05,  8.23s/it]
 16%|█▋        | 6188/37962 [14:22:19<72:48:21,  8.25s/it]
 16%|█▋        | 6189/37962 [14:22:27<72:51:39,  8.26s/it]
 16%|█▋        | 6190/37962 [14:22:36<73:41:14,  8.35s/it]
                                                          
{'loss': 1.4641, 'grad_norm': 12.059329986572266, 'learning_rate': 1.9085771097793986e-05, 'epoch': 0.16}

 16%|█▋        | 6190/37962 [14:22:36<73:41:14,  8.35s/it]
 16%|█▋        | 6191/37962 [14:22:44<73:21:49,  8.31s/it]
 16%|█▋        | 6192/37962 [14:22:52<73:21:05,  8.31s/it]
 16%|█▋        | 6193/37962 [14:23:00<73:12:28,  8.30s/it]
 16%|█▋        | 6194/37962 [14:23:09<73:38:04,  8.34s/it]
 16%|█▋        | 6195/37962 [14:23:17<73:18:57,  8.31s/it]
 16%|█▋        | 6196/37962 [14:23:25<72:49:32,  8.25s/it]
 16%|█▋        | 6197/37962 [14:23:34<72:54:20,  8.26s/it]
 16%|█▋        | 6198/37962 [14:23:42<73:19:38,  8.31s/it]
 16%|█▋        | 6199/37962 [14:23:50<73:03:29,  8.28s/it]
 16%|█▋        | 6200/37962 [14:23:58<72:52:21,  8.26s/it]
                                                          
{'loss': 1.5327, 'grad_norm': 8.388490676879883, 'learning_rate': 1.9082203993917344e-05, 'epoch': 0.16}

 16%|█▋        | 6200/37962 [14:23:58<72:52:21,  8.26s/it]
 16%|█▋        | 6201/37962 [14:24:07<72:50:33,  8.26s/it]
 16%|█▋        | 6202/37962 [14:24:15<73:39:35,  8.35s/it]
 16%|█▋        | 6203/37962 [14:24:23<73:21:29,  8.32s/it]
 16%|█▋        | 6204/37962 [14:24:31<72:41:46,  8.24s/it]
 16%|█▋        | 6205/37962 [14:24:40<73:03:53,  8.28s/it]
 16%|█▋        | 6206/37962 [14:24:48<73:07:17,  8.29s/it]
 16%|█▋        | 6207/37962 [14:24:56<71:23:34,  8.09s/it]
 16%|█▋        | 6208/37962 [14:25:04<72:20:28,  8.20s/it]
 16%|█▋        | 6209/37962 [14:25:12<72:08:51,  8.18s/it]
 16%|█▋        | 6210/37962 [14:25:21<72:42:02,  8.24s/it]
                                                          
{'loss': 1.4506, 'grad_norm': 6.851354122161865, 'learning_rate': 1.907863027926176e-05, 'epoch': 0.16}

 16%|█▋        | 6210/37962 [14:25:21<72:42:02,  8.24s/it]
 16%|█▋        | 6211/37962 [14:25:29<73:00:38,  8.28s/it]
 16%|█▋        | 6212/37962 [14:25:37<72:57:51,  8.27s/it]
 16%|█▋        | 6213/37962 [14:25:46<72:50:48,  8.26s/it]
 16%|█▋        | 6214/37962 [14:25:54<72:56:57,  8.27s/it]
 16%|█▋        | 6215/37962 [14:26:02<73:14:09,  8.30s/it]
 16%|█▋        | 6216/37962 [14:26:11<72:59:08,  8.28s/it]
 16%|█▋        | 6217/37962 [14:26:19<72:40:07,  8.24s/it]
 16%|█▋        | 6218/37962 [14:26:27<72:42:13,  8.25s/it]
 16%|█▋        | 6219/37962 [14:26:35<72:57:13,  8.27s/it]
 16%|█▋        | 6220/37962 [14:26:43<72:40:59,  8.24s/it]
                                                          
{'loss': 1.5331, 'grad_norm': 8.893203735351562, 'learning_rate': 1.907504995642847e-05, 'epoch': 0.16}

 16%|█▋        | 6220/37962 [14:26:43<72:40:59,  8.24s/it]
 16%|█▋        | 6221/37962 [14:26:52<72:28:06,  8.22s/it]
 16%|█▋        | 6222/37962 [14:27:00<72:19:54,  8.20s/it]
 16%|█▋        | 6223/37962 [14:27:08<72:17:42,  8.20s/it]
 16%|█▋        | 6224/37962 [14:27:16<72:09:10,  8.18s/it]
 16%|█▋        | 6225/37962 [14:27:24<72:15:03,  8.20s/it]
 16%|█▋        | 6226/37962 [14:27:33<73:02:53,  8.29s/it]
 16%|█▋        | 6227/37962 [14:27:41<72:59:27,  8.28s/it]
 16%|█▋        | 6228/37962 [14:27:49<72:56:30,  8.27s/it]
 16%|█▋        | 6229/37962 [14:27:58<72:44:04,  8.25s/it]
 16%|█▋        | 6230/37962 [14:28:06<72:51:48,  8.27s/it]
                                                          
{'loss': 1.4895, 'grad_norm': 5.783923149108887, 'learning_rate': 1.9071463028023527e-05, 'epoch': 0.16}

 16%|█▋        | 6230/37962 [14:28:06<72:51:48,  8.27s/it]
 16%|█▋        | 6231/37962 [14:28:14<73:11:02,  8.30s/it]
 16%|█▋        | 6232/37962 [14:28:23<73:39:14,  8.36s/it]
 16%|█▋        | 6233/37962 [14:28:31<73:50:35,  8.38s/it]
 16%|█▋        | 6234/37962 [14:28:39<73:30:48,  8.34s/it]
 16%|█▋        | 6235/37962 [14:28:48<73:19:59,  8.32s/it]
 16%|█▋        | 6236/37962 [14:28:56<73:07:18,  8.30s/it]
 16%|█▋        | 6237/37962 [14:29:04<72:49:17,  8.26s/it]
 16%|█▋        | 6238/37962 [14:29:12<71:31:23,  8.12s/it]
 16%|█▋        | 6239/37962 [14:29:20<72:07:50,  8.19s/it]
 16%|█▋        | 6240/37962 [14:29:28<71:36:17,  8.13s/it]
                                                          
{'loss': 1.4212, 'grad_norm': 7.178958415985107, 'learning_rate': 1.9067869496657802e-05, 'epoch': 0.16}

 16%|█▋        | 6240/37962 [14:29:28<71:36:17,  8.13s/it]
 16%|█▋        | 6241/37962 [14:29:37<72:06:19,  8.18s/it]
 16%|█▋        | 6242/37962 [14:29:45<72:19:59,  8.21s/it]
 16%|█▋        | 6243/37962 [14:29:53<72:18:14,  8.21s/it]
 16%|█▋        | 6244/37962 [14:30:01<72:40:40,  8.25s/it]
 16%|█▋        | 6245/37962 [14:30:10<72:50:06,  8.27s/it]
 16%|█▋        | 6246/37962 [14:30:18<72:23:40,  8.22s/it]
 16%|█▋        | 6247/37962 [14:30:26<72:18:06,  8.21s/it]
 16%|█▋        | 6248/37962 [14:30:34<73:04:30,  8.30s/it]
 16%|█▋        | 6249/37962 [14:30:42<72:00:35,  8.17s/it]
 16%|█▋        | 6250/37962 [14:30:51<72:19:19,  8.21s/it]
                                                          
{'loss': 1.503, 'grad_norm': 21.878192901611328, 'learning_rate': 1.906426936494696e-05, 'epoch': 0.16}

 16%|█▋        | 6250/37962 [14:30:51<72:19:19,  8.21s/it]
 16%|█▋        | 6251/37962 [14:30:59<72:04:41,  8.18s/it]
 16%|█▋        | 6252/37962 [14:31:07<72:10:27,  8.19s/it]
 16%|█▋        | 6253/37962 [14:31:15<72:05:40,  8.19s/it]
 16%|█▋        | 6254/37962 [14:31:23<72:03:02,  8.18s/it]
 16%|█▋        | 6255/37962 [14:31:32<72:33:15,  8.24s/it]
 16%|█▋        | 6256/37962 [14:31:40<72:37:17,  8.25s/it]
 16%|█▋        | 6257/37962 [14:31:48<72:40:24,  8.25s/it]
 16%|█▋        | 6258/37962 [14:31:56<72:18:00,  8.21s/it]
 16%|█▋        | 6259/37962 [14:32:04<72:02:46,  8.18s/it]
 16%|█▋        | 6260/37962 [14:32:13<72:29:32,  8.23s/it]
                                                          
{'loss': 1.4358, 'grad_norm': 7.039337158203125, 'learning_rate': 1.9060662635511474e-05, 'epoch': 0.16}

 16%|█▋        | 6260/37962 [14:32:13<72:29:32,  8.23s/it]
 16%|█▋        | 6261/37962 [14:32:21<72:25:04,  8.22s/it]
 16%|█▋        | 6262/37962 [14:32:29<71:41:02,  8.14s/it]
 16%|█▋        | 6263/37962 [14:32:37<71:43:13,  8.15s/it]
 17%|█▋        | 6264/37962 [14:32:45<71:51:01,  8.16s/it]
 17%|█▋        | 6265/37962 [14:32:54<72:01:22,  8.18s/it]
 17%|█▋        | 6266/37962 [14:33:02<72:11:42,  8.20s/it]
 17%|█▋        | 6267/37962 [14:33:10<72:30:50,  8.24s/it]
 17%|█▋        | 6268/37962 [14:33:18<72:41:12,  8.26s/it]
 17%|█▋        | 6269/37962 [14:33:26<71:43:10,  8.15s/it]
 17%|█▋        | 6270/37962 [14:33:35<72:09:06,  8.20s/it]
                                                          
{'loss': 1.5697, 'grad_norm': 15.890427589416504, 'learning_rate': 1.9057049310976618e-05, 'epoch': 0.17}

 17%|█▋        | 6270/37962 [14:33:35<72:09:06,  8.20s/it]
 17%|█▋        | 6271/37962 [14:33:43<72:10:13,  8.20s/it]
 17%|█▋        | 6272/37962 [14:33:51<72:43:07,  8.26s/it]
 17%|█▋        | 6273/37962 [14:34:00<72:57:38,  8.29s/it]
 17%|█▋        | 6274/37962 [14:34:08<72:53:33,  8.28s/it]
 17%|█▋        | 6275/37962 [14:34:16<72:40:32,  8.26s/it]
 17%|█▋        | 6276/37962 [14:34:24<72:27:21,  8.23s/it]
 17%|█▋        | 6277/37962 [14:34:33<73:03:52,  8.30s/it]
 17%|█▋        | 6278/37962 [14:34:41<73:38:16,  8.37s/it]
 17%|█▋        | 6279/37962 [14:34:50<73:43:35,  8.38s/it]
 17%|█▋        | 6280/37962 [14:34:58<73:24:10,  8.34s/it]
                                                          
{'loss': 1.5479, 'grad_norm': 7.094673156738281, 'learning_rate': 1.9053429393972478e-05, 'epoch': 0.17}

 17%|█▋        | 6280/37962 [14:34:58<73:24:10,  8.34s/it]
 17%|█▋        | 6281/37962 [14:35:06<73:01:40,  8.30s/it]
 17%|█▋        | 6282/37962 [14:35:14<72:11:26,  8.20s/it]
 17%|█▋        | 6283/37962 [14:35:22<72:32:25,  8.24s/it]
 17%|█▋        | 6284/37962 [14:35:31<72:30:38,  8.24s/it]
 17%|█▋        | 6285/37962 [14:35:39<72:48:15,  8.27s/it]
 17%|█▋        | 6286/37962 [14:35:47<72:08:41,  8.20s/it]
 17%|█▋        | 6287/37962 [14:35:54<70:14:31,  7.98s/it]
 17%|█▋        | 6288/37962 [14:36:03<70:54:19,  8.06s/it]
 17%|█▋        | 6289/37962 [14:36:11<70:28:04,  8.01s/it]
 17%|█▋        | 6290/37962 [14:36:19<71:01:49,  8.07s/it]
                                                          
{'loss': 1.4789, 'grad_norm': 25.221776962280273, 'learning_rate': 1.9049802887133917e-05, 'epoch': 0.17}

 17%|█▋        | 6290/37962 [14:36:19<71:01:49,  8.07s/it]
 17%|█▋        | 6291/37962 [14:36:27<71:26:28,  8.12s/it]
 17%|█▋        | 6292/37962 [14:36:35<72:21:36,  8.23s/it]
 17%|█▋        | 6293/37962 [14:36:44<72:11:24,  8.21s/it]
 17%|█▋        | 6294/37962 [14:36:52<72:16:11,  8.22s/it]
 17%|█▋        | 6295/37962 [14:37:00<72:39:00,  8.26s/it]
 17%|█▋        | 6296/37962 [14:37:09<73:06:13,  8.31s/it]
 17%|█▋        | 6297/37962 [14:37:17<71:50:55,  8.17s/it]
 17%|█▋        | 6298/37962 [14:37:25<72:06:29,  8.20s/it]
 17%|█▋        | 6299/37962 [14:37:33<72:31:40,  8.25s/it]
 17%|█▋        | 6300/37962 [14:37:41<72:45:18,  8.27s/it]
                                                          
{'loss': 1.4038, 'grad_norm': 21.335376739501953, 'learning_rate': 1.9046169793100614e-05, 'epoch': 0.17}

 17%|█▋        | 6300/37962 [14:37:41<72:45:18,  8.27s/it]
 17%|█▋        | 6301/37962 [14:37:50<73:04:22,  8.31s/it]
 17%|█▋        | 6302/37962 [14:37:58<72:52:10,  8.29s/it]
 17%|█▋        | 6303/37962 [14:38:06<71:59:44,  8.19s/it]
 17%|█▋        | 6304/37962 [14:38:14<72:09:04,  8.20s/it]
 17%|█▋        | 6305/37962 [14:38:23<72:12:04,  8.21s/it]
 17%|█▋        | 6306/37962 [14:38:31<72:33:39,  8.25s/it]
 17%|█▋        | 6307/37962 [14:38:39<72:53:39,  8.29s/it]
 17%|█▋        | 6308/37962 [14:38:47<72:40:37,  8.27s/it]
 17%|█▋        | 6309/37962 [14:38:56<72:44:29,  8.27s/it]
 17%|█▋        | 6310/37962 [14:39:04<72:31:22,  8.25s/it]
                                                          
{'loss': 1.4767, 'grad_norm': 23.99860191345215, 'learning_rate': 1.904253011451703e-05, 'epoch': 0.17}

 17%|█▋        | 6310/37962 [14:39:04<72:31:22,  8.25s/it]
 17%|█▋        | 6311/37962 [14:39:12<72:38:53,  8.26s/it]
 17%|█▋        | 6312/37962 [14:39:21<72:55:02,  8.29s/it]
 17%|█▋        | 6313/37962 [14:39:29<72:57:46,  8.30s/it]
 17%|█▋        | 6314/37962 [14:39:37<73:05:06,  8.31s/it]
 17%|█▋        | 6315/37962 [14:39:45<72:47:15,  8.28s/it]
 17%|█▋        | 6316/37962 [14:39:54<72:54:37,  8.29s/it]
 17%|█▋        | 6317/37962 [14:40:02<73:26:39,  8.36s/it]
 17%|█▋        | 6318/37962 [14:40:11<73:08:41,  8.32s/it]
 17%|█▋        | 6319/37962 [14:40:19<73:00:58,  8.31s/it]
 17%|█▋        | 6320/37962 [14:40:27<72:49:55,  8.29s/it]
                                                          
{'loss': 1.399, 'grad_norm': 15.537790298461914, 'learning_rate': 1.903888385403243e-05, 'epoch': 0.17}

 17%|█▋        | 6320/37962 [14:40:27<72:49:55,  8.29s/it]
 17%|█▋        | 6321/37962 [14:40:35<72:49:09,  8.29s/it]
 17%|█▋        | 6322/37962 [14:40:44<72:56:42,  8.30s/it]
 17%|█▋        | 6323/37962 [14:40:52<72:40:06,  8.27s/it]
 17%|█▋        | 6324/37962 [14:41:00<72:34:50,  8.26s/it]
 17%|█▋        | 6325/37962 [14:41:08<72:45:35,  8.28s/it]
 17%|█▋        | 6326/37962 [14:41:17<72:55:18,  8.30s/it]
 17%|█▋        | 6327/37962 [14:41:25<73:06:27,  8.32s/it]
 17%|█▋        | 6328/37962 [14:41:33<72:58:28,  8.30s/it]
 17%|█▋        | 6329/37962 [14:41:41<71:37:34,  8.15s/it]
 17%|█▋        | 6330/37962 [14:41:50<72:16:06,  8.22s/it]
                                                          
{'loss': 1.4695, 'grad_norm': 12.00252914428711, 'learning_rate': 1.9035231014300855e-05, 'epoch': 0.17}

 17%|█▋        | 6330/37962 [14:41:50<72:16:06,  8.22s/it]
 17%|█▋        | 6331/37962 [14:41:58<72:01:28,  8.20s/it]
 17%|█▋        | 6332/37962 [14:42:06<71:32:08,  8.14s/it]
 17%|█▋        | 6333/37962 [14:42:14<71:23:31,  8.13s/it]
 17%|█▋        | 6334/37962 [14:42:22<72:03:35,  8.20s/it]
 17%|█▋        | 6335/37962 [14:42:31<72:31:44,  8.26s/it]
 17%|█▋        | 6336/37962 [14:42:39<72:36:05,  8.26s/it]
 17%|█▋        | 6337/37962 [14:42:47<72:00:26,  8.20s/it]
 17%|█▋        | 6338/37962 [14:42:55<72:01:59,  8.20s/it]
 17%|█▋        | 6339/37962 [14:43:03<72:17:24,  8.23s/it]
 17%|█▋        | 6340/37962 [14:43:12<72:19:10,  8.23s/it]
                                                          
{'loss': 1.3675, 'grad_norm': 16.461694717407227, 'learning_rate': 1.9031571597981154e-05, 'epoch': 0.17}

 17%|█▋        | 6340/37962 [14:43:12<72:19:10,  8.23s/it]
 17%|█▋        | 6341/37962 [14:43:20<72:13:19,  8.22s/it]
 17%|█▋        | 6342/37962 [14:43:28<71:35:38,  8.15s/it]
 17%|█▋        | 6343/37962 [14:43:36<72:24:26,  8.24s/it]
 17%|█▋        | 6344/37962 [14:43:45<72:36:34,  8.27s/it]
 17%|█▋        | 6345/37962 [14:43:53<72:31:31,  8.26s/it]
 17%|█▋        | 6346/37962 [14:44:01<71:22:07,  8.13s/it]
 17%|█▋        | 6347/37962 [14:44:09<72:00:01,  8.20s/it]
 17%|█▋        | 6348/37962 [14:44:17<72:22:13,  8.24s/it]
 17%|█▋        | 6349/37962 [14:44:26<72:40:39,  8.28s/it]
 17%|█▋        | 6350/37962 [14:44:34<72:32:52,  8.26s/it]
                                                          
{'loss': 1.4101, 'grad_norm': 4.490846633911133, 'learning_rate': 1.902790560773695e-05, 'epoch': 0.17}

 17%|█▋        | 6350/37962 [14:44:34<72:32:52,  8.26s/it]
 17%|█▋        | 6351/37962 [14:44:42<72:33:22,  8.26s/it]
 17%|█▋        | 6352/37962 [14:44:50<71:34:46,  8.15s/it]
 17%|█▋        | 6353/37962 [14:44:58<70:57:59,  8.08s/it]
 17%|█▋        | 6354/37962 [14:45:06<71:18:02,  8.12s/it]
 17%|█▋        | 6355/37962 [14:45:15<71:47:14,  8.18s/it]
 17%|█▋        | 6356/37962 [14:45:23<72:00:01,  8.20s/it]
 17%|█▋        | 6357/37962 [14:45:31<71:10:00,  8.11s/it]
 17%|█▋        | 6358/37962 [14:45:39<71:54:07,  8.19s/it]
 17%|█▋        | 6359/37962 [14:45:47<72:11:31,  8.22s/it]
 17%|█▋        | 6360/37962 [14:45:56<72:49:20,  8.30s/it]
                                                          
{'loss': 1.4461, 'grad_norm': 12.76328182220459, 'learning_rate': 1.902423304623665e-05, 'epoch': 0.17}

 17%|█▋        | 6360/37962 [14:45:56<72:49:20,  8.30s/it]
 17%|█▋        | 6361/37962 [14:46:04<72:02:46,  8.21s/it]
 17%|█▋        | 6362/37962 [14:46:12<72:10:34,  8.22s/it]
 17%|█▋        | 6363/37962 [14:46:20<72:06:05,  8.21s/it]
 17%|█▋        | 6364/37962 [14:46:29<72:19:07,  8.24s/it]
 17%|█▋        | 6365/37962 [14:46:37<72:04:14,  8.21s/it]
 17%|█▋        | 6366/37962 [14:46:45<72:14:26,  8.23s/it]
 17%|█▋        | 6367/37962 [14:46:53<72:31:20,  8.26s/it]
 17%|█▋        | 6368/37962 [14:47:02<72:51:57,  8.30s/it]
 17%|█▋        | 6369/37962 [14:47:10<72:44:34,  8.29s/it]
 17%|█▋        | 6370/37962 [14:47:18<72:36:51,  8.27s/it]
                                                          
{'loss': 1.4715, 'grad_norm': 10.525606155395508, 'learning_rate': 1.9020553916153448e-05, 'epoch': 0.17}

 17%|█▋        | 6370/37962 [14:47:18<72:36:51,  8.27s/it]
 17%|█▋        | 6371/37962 [14:47:26<72:24:49,  8.25s/it]
 17%|█▋        | 6372/37962 [14:47:35<73:03:37,  8.33s/it]
 17%|█▋        | 6373/37962 [14:47:43<72:48:10,  8.30s/it]
 17%|█▋        | 6374/37962 [14:47:52<73:05:23,  8.33s/it]
 17%|█▋        | 6375/37962 [14:48:00<72:59:24,  8.32s/it]
 17%|█▋        | 6376/37962 [14:48:08<72:44:37,  8.29s/it]
 17%|█▋        | 6377/37962 [14:48:17<73:09:49,  8.34s/it]
 17%|█▋        | 6378/37962 [14:48:25<72:46:21,  8.29s/it]
 17%|█▋        | 6379/37962 [14:48:33<72:11:40,  8.23s/it]
 17%|█▋        | 6380/37962 [14:48:41<72:10:58,  8.23s/it]
                                                          
{'loss': 1.3916, 'grad_norm': 19.897045135498047, 'learning_rate': 1.901686822016532e-05, 'epoch': 0.17}

 17%|█▋        | 6380/37962 [14:48:41<72:10:58,  8.23s/it]
 17%|█▋        | 6381/37962 [14:48:49<72:18:13,  8.24s/it]
 17%|█▋        | 6382/37962 [14:48:57<71:31:39,  8.15s/it]
 17%|█▋        | 6383/37962 [14:49:06<71:58:16,  8.20s/it]
 17%|█▋        | 6384/37962 [14:49:14<71:56:06,  8.20s/it]
 17%|█▋        | 6385/37962 [14:49:22<71:59:53,  8.21s/it]
 17%|█▋        | 6386/37962 [14:49:30<71:51:40,  8.19s/it]
 17%|█▋        | 6387/37962 [14:49:38<71:13:25,  8.12s/it]
 17%|█▋        | 6388/37962 [14:49:47<71:53:57,  8.20s/it]
 17%|█▋        | 6389/37962 [14:49:55<72:02:44,  8.21s/it]
 17%|█▋        | 6390/37962 [14:50:03<71:53:18,  8.20s/it]
                                                          
{'loss': 1.2664, 'grad_norm': 22.964447021484375, 'learning_rate': 1.9013175960955027e-05, 'epoch': 0.17}

 17%|█▋        | 6390/37962 [14:50:03<71:53:18,  8.20s/it]
 17%|█▋        | 6391/37962 [14:50:11<72:28:22,  8.26s/it]
 17%|█▋        | 6392/37962 [14:50:19<71:11:20,  8.12s/it]
 17%|█▋        | 6393/37962 [14:50:28<71:52:59,  8.20s/it]
 17%|█▋        | 6394/37962 [14:50:36<72:25:55,  8.26s/it]
 17%|█▋        | 6395/37962 [14:50:44<72:10:21,  8.23s/it]
 17%|█▋        | 6396/37962 [14:50:52<71:57:22,  8.21s/it]
 17%|█▋        | 6397/37962 [14:51:00<71:43:38,  8.18s/it]
 17%|█▋        | 6398/37962 [14:51:09<71:49:49,  8.19s/it]
 17%|█▋        | 6399/37962 [14:51:17<71:21:57,  8.14s/it]
 17%|█▋        | 6400/37962 [14:51:25<71:33:57,  8.16s/it]
                                                          
{'loss': 1.4455, 'grad_norm': 13.278956413269043, 'learning_rate': 1.9009477141210088e-05, 'epoch': 0.17}

 17%|█▋        | 6400/37962 [14:51:25<71:33:57,  8.16s/it]
 17%|█▋        | 6401/37962 [14:51:33<71:34:10,  8.16s/it]
 17%|█▋        | 6402/37962 [14:51:41<70:47:31,  8.08s/it]
 17%|█▋        | 6403/37962 [14:51:49<70:42:09,  8.07s/it]
 17%|█▋        | 6404/37962 [14:51:57<71:03:44,  8.11s/it]
 17%|█▋        | 6405/37962 [14:52:05<71:31:30,  8.16s/it]
 17%|█▋        | 6406/37962 [14:52:14<71:45:46,  8.19s/it]
 17%|█▋        | 6407/37962 [14:52:22<71:47:19,  8.19s/it]
 17%|█▋        | 6408/37962 [14:52:30<72:13:47,  8.24s/it]
 17%|█▋        | 6409/37962 [14:52:39<72:56:33,  8.32s/it]
 17%|█▋        | 6410/37962 [14:52:47<72:41:48,  8.29s/it]
                                                          
{'loss': 1.311, 'grad_norm': 10.78231143951416, 'learning_rate': 1.900577176362282e-05, 'epoch': 0.17}

 17%|█▋        | 6410/37962 [14:52:47<72:41:48,  8.29s/it]
 17%|█▋        | 6411/37962 [14:52:55<72:17:52,  8.25s/it]
 17%|█▋        | 6412/37962 [14:53:03<72:33:46,  8.28s/it]
 17%|█▋        | 6413/37962 [14:53:12<72:37:09,  8.29s/it]
 17%|█▋        | 6414/37962 [14:53:20<71:58:52,  8.21s/it]
 17%|█▋        | 6415/37962 [14:53:28<72:14:49,  8.24s/it]
 17%|█▋        | 6416/37962 [14:53:37<73:02:46,  8.34s/it]
 17%|█▋        | 6417/37962 [14:53:45<72:51:30,  8.31s/it]
 17%|█▋        | 6418/37962 [14:53:53<72:31:06,  8.28s/it]
 17%|█▋        | 6419/37962 [14:54:01<72:52:50,  8.32s/it]
 17%|█▋        | 6420/37962 [14:54:10<72:19:04,  8.25s/it]
                                                          
{'loss': 1.3817, 'grad_norm': 11.10671615600586, 'learning_rate': 1.9002059830890302e-05, 'epoch': 0.17}

 17%|█▋        | 6420/37962 [14:54:10<72:19:04,  8.25s/it]
 17%|█▋        | 6421/37962 [14:54:18<72:10:22,  8.24s/it]
 17%|█▋        | 6422/37962 [14:54:26<71:40:18,  8.18s/it]
 17%|█▋        | 6423/37962 [14:54:34<71:54:33,  8.21s/it]
 17%|█▋        | 6424/37962 [14:54:42<71:48:54,  8.20s/it]
 17%|█▋        | 6425/37962 [14:54:50<71:49:26,  8.20s/it]
 17%|█▋        | 6426/37962 [14:54:58<71:07:00,  8.12s/it]
 17%|█▋        | 6427/37962 [14:55:07<71:39:44,  8.18s/it]
 17%|█▋        | 6428/37962 [14:55:15<72:04:23,  8.23s/it]
 17%|█▋        | 6429/37962 [14:55:23<72:24:00,  8.27s/it]
 17%|█▋        | 6430/37962 [14:55:32<72:37:48,  8.29s/it]
                                                          
{'loss': 1.5502, 'grad_norm': 13.355144500732422, 'learning_rate': 1.8998341345714376e-05, 'epoch': 0.17}

 17%|█▋        | 6430/37962 [14:55:32<72:37:48,  8.29s/it]
 17%|█▋        | 6431/37962 [14:55:40<72:46:38,  8.31s/it]
 17%|█▋        | 6432/37962 [14:55:48<72:38:08,  8.29s/it]
 17%|█▋        | 6433/37962 [14:55:57<72:49:38,  8.32s/it]
 17%|█▋        | 6434/37962 [14:56:05<72:18:21,  8.26s/it]
 17%|█▋        | 6435/37962 [14:56:13<72:17:39,  8.26s/it]
 17%|█▋        | 6436/37962 [14:56:21<72:19:02,  8.26s/it]
 17%|█▋        | 6437/37962 [14:56:29<71:42:16,  8.19s/it]
 17%|█▋        | 6438/37962 [14:56:38<72:05:45,  8.23s/it]
 17%|█▋        | 6439/37962 [14:56:46<71:43:34,  8.19s/it]
 17%|█▋        | 6440/37962 [14:56:54<72:09:46,  8.24s/it]
                                                          
{'loss': 1.4006, 'grad_norm': 15.899882316589355, 'learning_rate': 1.8994616310801674e-05, 'epoch': 0.17}

 17%|█▋        | 6440/37962 [14:56:54<72:09:46,  8.24s/it]
 17%|█▋        | 6441/37962 [14:57:02<72:15:25,  8.25s/it]
 17%|█▋        | 6442/37962 [14:57:11<71:45:25,  8.20s/it]
 17%|█▋        | 6443/37962 [14:57:18<70:51:44,  8.09s/it]
 17%|█▋        | 6444/37962 [14:57:27<71:29:57,  8.17s/it]
 17%|█▋        | 6445/37962 [14:57:35<71:44:56,  8.20s/it]
 17%|█▋        | 6446/37962 [14:57:43<72:06:01,  8.24s/it]
 17%|█▋        | 6447/37962 [14:57:52<72:05:26,  8.24s/it]
 17%|█▋        | 6448/37962 [14:58:00<72:11:26,  8.25s/it]
 17%|█▋        | 6449/37962 [14:58:08<72:01:39,  8.23s/it]
 17%|█▋        | 6450/37962 [14:58:16<71:53:18,  8.21s/it]
                                                          
{'loss': 1.4882, 'grad_norm': 3.653938055038452, 'learning_rate': 1.899088472886358e-05, 'epoch': 0.17}

 17%|█▋        | 6450/37962 [14:58:16<71:53:18,  8.21s/it]
 17%|█▋        | 6451/37962 [14:58:25<72:15:25,  8.26s/it]
 17%|█▋        | 6452/37962 [14:58:33<72:15:31,  8.26s/it]
 17%|█▋        | 6453/37962 [14:58:41<72:18:35,  8.26s/it]
 17%|█▋        | 6454/37962 [14:58:49<72:32:12,  8.29s/it]
 17%|█▋        | 6455/37962 [14:58:58<72:30:07,  8.28s/it]
 17%|█▋        | 6456/37962 [14:59:06<72:36:31,  8.30s/it]
 17%|█▋        | 6457/37962 [14:59:14<72:25:46,  8.28s/it]
 17%|█▋        | 6458/37962 [14:59:22<70:57:32,  8.11s/it]
 17%|█▋        | 6459/37962 [14:59:30<70:43:00,  8.08s/it]
 17%|█▋        | 6460/37962 [14:59:38<70:35:06,  8.07s/it]
                                                          
{'loss': 1.5547, 'grad_norm': 9.790179252624512, 'learning_rate': 1.898714660261625e-05, 'epoch': 0.17}

 17%|█▋        | 6460/37962 [14:59:38<70:35:06,  8.07s/it]
 17%|█▋        | 6461/37962 [14:59:46<71:18:30,  8.15s/it]
 17%|█▋        | 6462/37962 [14:59:54<71:13:54,  8.14s/it]
 17%|█▋        | 6463/37962 [15:00:03<71:12:27,  8.14s/it]
 17%|█▋        | 6464/37962 [15:00:11<71:21:19,  8.16s/it]
 17%|█▋        | 6465/37962 [15:00:19<71:47:31,  8.21s/it]
 17%|█▋        | 6466/37962 [15:00:27<71:56:04,  8.22s/it]
 17%|█▋        | 6467/37962 [15:00:36<72:02:04,  8.23s/it]
 17%|█▋        | 6468/37962 [15:00:44<72:07:11,  8.24s/it]
 17%|█▋        | 6469/37962 [15:00:52<72:01:52,  8.23s/it]
 17%|█▋        | 6470/37962 [15:01:00<72:04:00,  8.24s/it]
                                                          
{'loss': 1.517, 'grad_norm': 11.300407409667969, 'learning_rate': 1.8983401934780595e-05, 'epoch': 0.17}

 17%|█▋        | 6470/37962 [15:01:00<72:04:00,  8.24s/it]
 17%|█▋        | 6471/37962 [15:01:09<72:24:34,  8.28s/it]
 17%|█▋        | 6472/37962 [15:01:17<72:08:37,  8.25s/it]
 17%|█▋        | 6473/37962 [15:01:25<71:57:26,  8.23s/it]
 17%|█▋        | 6474/37962 [15:01:33<71:52:48,  8.22s/it]
 17%|█▋        | 6475/37962 [15:01:42<71:50:09,  8.21s/it]
 17%|█▋        | 6476/37962 [15:01:50<71:41:56,  8.20s/it]
 17%|█▋        | 6477/37962 [15:01:58<71:44:17,  8.20s/it]
 17%|█▋        | 6478/37962 [15:02:06<70:57:17,  8.11s/it]
 17%|█▋        | 6479/37962 [15:02:14<71:16:49,  8.15s/it]
 17%|█▋        | 6480/37962 [15:02:22<70:40:32,  8.08s/it]
                                                          
{'loss': 1.4514, 'grad_norm': 28.179386138916016, 'learning_rate': 1.8979650728082305e-05, 'epoch': 0.17}

 17%|█▋        | 6480/37962 [15:02:22<70:40:32,  8.08s/it]
 17%|█▋        | 6481/37962 [15:02:30<71:43:59,  8.20s/it]
 17%|█▋        | 6482/37962 [15:02:39<72:10:36,  8.25s/it]
 17%|█▋        | 6483/37962 [15:02:47<72:12:15,  8.26s/it]
 17%|█▋        | 6484/37962 [15:02:55<72:13:08,  8.26s/it]
 17%|█▋        | 6485/37962 [15:03:03<71:14:09,  8.15s/it]
 17%|█▋        | 6486/37962 [15:03:11<71:17:34,  8.15s/it]
 17%|█▋        | 6487/37962 [15:03:19<70:54:23,  8.11s/it]
 17%|█▋        | 6488/37962 [15:03:27<70:30:54,  8.07s/it]
 17%|█▋        | 6489/37962 [15:03:35<70:25:35,  8.06s/it]
 17%|█▋        | 6490/37962 [15:03:44<70:45:09,  8.09s/it]
                                                          
{'loss': 1.427, 'grad_norm': 21.107830047607422, 'learning_rate': 1.8975892985251808e-05, 'epoch': 0.17}

 17%|█▋        | 6490/37962 [15:03:44<70:45:09,  8.09s/it]
 17%|█▋        | 6491/37962 [15:03:52<71:26:33,  8.17s/it]
 17%|█▋        | 6492/37962 [15:04:00<71:45:11,  8.21s/it]
 17%|█▋        | 6493/37962 [15:04:08<70:52:18,  8.11s/it]
 17%|█▋        | 6494/37962 [15:04:16<70:46:25,  8.10s/it]
 17%|█▋        | 6495/37962 [15:04:24<70:57:23,  8.12s/it]
 17%|█▋        | 6496/37962 [15:04:33<71:25:10,  8.17s/it]
 17%|█▋        | 6497/37962 [15:04:41<71:14:23,  8.15s/it]
 17%|█▋        | 6498/37962 [15:04:49<70:58:53,  8.12s/it]
 17%|█▋        | 6499/37962 [15:04:57<70:46:46,  8.10s/it]
 17%|█▋        | 6500/37962 [15:05:05<71:29:30,  8.18s/it]
                                                          
{'loss': 1.4659, 'grad_norm': 2.137543201446533, 'learning_rate': 1.8972128709024305e-05, 'epoch': 0.17}

 17%|█▋        | 6500/37962 [15:05:05<71:29:30,  8.18s/it]
 17%|█▋        | 6501/37962 [15:05:13<71:28:43,  8.18s/it]
 17%|█▋        | 6502/37962 [15:05:22<71:40:03,  8.20s/it]
 17%|█▋        | 6503/37962 [15:05:30<71:35:56,  8.19s/it]
 17%|█▋        | 6504/37962 [15:05:38<71:54:55,  8.23s/it]
 17%|█▋        | 6505/37962 [15:05:46<72:07:43,  8.25s/it]
 17%|█▋        | 6506/37962 [15:05:54<70:59:41,  8.13s/it]
 17%|█▋        | 6507/37962 [15:06:02<70:38:09,  8.08s/it]
 17%|█▋        | 6508/37962 [15:06:11<71:15:26,  8.16s/it]
 17%|█▋        | 6509/37962 [15:06:19<71:21:08,  8.17s/it]
 17%|█▋        | 6510/37962 [15:06:27<71:09:59,  8.15s/it]
                                                          
{'loss': 1.4279, 'grad_norm': 31.691843032836914, 'learning_rate': 1.8968357902139744e-05, 'epoch': 0.17}

 17%|█▋        | 6510/37962 [15:06:27<71:09:59,  8.15s/it]
 17%|█▋        | 6511/37962 [15:06:35<71:55:09,  8.23s/it]
 17%|█▋        | 6512/37962 [15:06:44<71:53:20,  8.23s/it]
 17%|█▋        | 6513/37962 [15:06:52<71:53:46,  8.23s/it]
 17%|█▋        | 6514/37962 [15:07:00<71:59:36,  8.24s/it]
 17%|█▋        | 6515/37962 [15:07:08<70:59:30,  8.13s/it]
 17%|█▋        | 6516/37962 [15:07:16<71:17:27,  8.16s/it]
 17%|█▋        | 6517/37962 [15:07:24<71:47:35,  8.22s/it]
 17%|█▋        | 6518/37962 [15:07:33<72:11:20,  8.26s/it]
 17%|█▋        | 6519/37962 [15:07:41<71:06:27,  8.14s/it]
 17%|█▋        | 6520/37962 [15:07:49<71:42:13,  8.21s/it]
                                                          
{'loss': 1.4986, 'grad_norm': 15.218578338623047, 'learning_rate': 1.896458056734283e-05, 'epoch': 0.17}

 17%|█▋        | 6520/37962 [15:07:49<71:42:13,  8.21s/it]
 17%|█▋        | 6521/37962 [15:07:58<72:16:30,  8.28s/it]
 17%|█▋        | 6522/37962 [15:08:06<72:07:44,  8.26s/it]
 17%|█▋        | 6523/37962 [15:08:14<72:06:35,  8.26s/it]
 17%|█▋        | 6524/37962 [15:08:22<72:05:05,  8.25s/it]
 17%|█▋        | 6525/37962 [15:08:31<72:27:03,  8.30s/it]
 17%|█▋        | 6526/37962 [15:08:39<71:33:13,  8.19s/it]
 17%|█▋        | 6527/37962 [15:08:47<71:46:20,  8.22s/it]
 17%|█▋        | 6528/37962 [15:08:55<71:00:11,  8.13s/it]
 17%|█▋        | 6529/37962 [15:09:03<71:41:44,  8.21s/it]
 17%|█▋        | 6530/37962 [15:09:11<71:29:37,  8.19s/it]
                                                          
{'loss': 1.5328, 'grad_norm': 17.46087074279785, 'learning_rate': 1.8960796707383027e-05, 'epoch': 0.17}

 17%|█▋        | 6530/37962 [15:09:11<71:29:37,  8.19s/it]
 17%|█▋        | 6531/37962 [15:09:20<71:31:45,  8.19s/it]
 17%|█▋        | 6532/37962 [15:09:28<71:33:21,  8.20s/it]
 17%|█▋        | 6533/37962 [15:09:36<71:12:31,  8.16s/it]
 17%|█▋        | 6534/37962 [15:09:44<71:29:12,  8.19s/it]
 17%|█▋        | 6535/37962 [15:09:52<71:40:27,  8.21s/it]
 17%|█▋        | 6536/37962 [15:10:00<71:06:51,  8.15s/it]
 17%|█▋        | 6537/37962 [15:10:08<71:00:36,  8.13s/it]
 17%|█▋        | 6538/37962 [15:10:17<71:34:47,  8.20s/it]
 17%|█▋        | 6539/37962 [15:10:25<71:13:06,  8.16s/it]
 17%|█▋        | 6540/37962 [15:10:33<71:28:09,  8.19s/it]
                                                          
{'loss': 1.4926, 'grad_norm': 10.051220893859863, 'learning_rate': 1.8957006325014536e-05, 'epoch': 0.17}

 17%|█▋        | 6540/37962 [15:10:33<71:28:09,  8.19s/it]
 17%|█▋        | 6541/37962 [15:10:41<71:17:29,  8.17s/it]
 17%|█▋        | 6542/37962 [15:10:50<71:44:46,  8.22s/it]
 17%|█▋        | 6543/37962 [15:10:57<70:48:18,  8.11s/it]
 17%|█▋        | 6544/37962 [15:11:05<70:08:11,  8.04s/it]
 17%|█▋        | 6545/37962 [15:11:13<70:34:36,  8.09s/it]
 17%|█▋        | 6546/37962 [15:11:22<70:56:07,  8.13s/it]
 17%|█▋        | 6547/37962 [15:11:30<71:49:15,  8.23s/it]
 17%|█▋        | 6548/37962 [15:11:38<71:26:41,  8.19s/it]
 17%|█▋        | 6549/37962 [15:11:46<70:52:09,  8.12s/it]
 17%|█▋        | 6550/37962 [15:11:54<71:03:04,  8.14s/it]
                                                          
{'loss': 1.3932, 'grad_norm': 7.401064395904541, 'learning_rate': 1.895320942299631e-05, 'epoch': 0.17}

 17%|█▋        | 6550/37962 [15:11:54<71:03:04,  8.14s/it]
 17%|█▋        | 6551/37962 [15:12:03<71:32:58,  8.20s/it]
 17%|█▋        | 6552/37962 [15:12:11<71:01:36,  8.14s/it]
 17%|█▋        | 6553/37962 [15:12:19<71:26:41,  8.19s/it]
 17%|█▋        | 6554/37962 [15:12:27<71:15:21,  8.17s/it]
 17%|█▋        | 6555/37962 [15:12:35<71:14:25,  8.17s/it]
 17%|█▋        | 6556/37962 [15:12:44<71:17:42,  8.17s/it]
 17%|█▋        | 6557/37962 [15:12:52<71:12:45,  8.16s/it]
 17%|█▋        | 6558/37962 [15:13:00<71:01:33,  8.14s/it]
 17%|█▋        | 6559/37962 [15:13:08<71:47:45,  8.23s/it]
 17%|█▋        | 6560/37962 [15:13:16<71:57:08,  8.25s/it]
                                                          
{'loss': 1.6015, 'grad_norm': 8.164896965026855, 'learning_rate': 1.8949406004092045e-05, 'epoch': 0.17}

 17%|█▋        | 6560/37962 [15:13:16<71:57:08,  8.25s/it]
 17%|█▋        | 6561/37962 [15:13:25<71:39:56,  8.22s/it]
 17%|█▋        | 6562/37962 [15:13:33<71:52:25,  8.24s/it]
 17%|█▋        | 6563/37962 [15:13:41<71:55:55,  8.25s/it]
 17%|█▋        | 6564/37962 [15:13:50<72:06:38,  8.27s/it]
 17%|█▋        | 6565/37962 [15:13:58<71:59:05,  8.25s/it]
 17%|█▋        | 6566/37962 [15:14:06<72:14:14,  8.28s/it]
 17%|█▋        | 6567/37962 [15:14:14<72:32:53,  8.32s/it]
 17%|█▋        | 6568/37962 [15:14:23<72:29:21,  8.31s/it]
 17%|█▋        | 6569/37962 [15:14:31<72:53:12,  8.36s/it]
 17%|█▋        | 6570/37962 [15:14:39<72:23:59,  8.30s/it]
                                                          
{'loss': 1.417, 'grad_norm': 11.649212837219238, 'learning_rate': 1.894559607107019e-05, 'epoch': 0.17}

 17%|█▋        | 6570/37962 [15:14:39<72:23:59,  8.30s/it]
 17%|█▋        | 6571/37962 [15:14:48<72:23:39,  8.30s/it]
 17%|█▋        | 6572/37962 [15:14:56<73:04:39,  8.38s/it]
 17%|█▋        | 6573/37962 [15:15:05<73:04:46,  8.38s/it]
 17%|█▋        | 6574/37962 [15:15:13<72:49:04,  8.35s/it]
 17%|█▋        | 6575/37962 [15:15:21<71:44:33,  8.23s/it]
 17%|█▋        | 6576/37962 [15:15:29<72:16:29,  8.29s/it]
 17%|█▋        | 6577/37962 [15:15:38<72:20:51,  8.30s/it]
 17%|█▋        | 6578/37962 [15:15:46<72:13:46,  8.29s/it]
 17%|█▋        | 6579/37962 [15:15:54<72:33:20,  8.32s/it]
 17%|█▋        | 6580/37962 [15:16:03<72:39:31,  8.34s/it]
                                                          
{'loss': 1.4785, 'grad_norm': 7.982486724853516, 'learning_rate': 1.8941779626703928e-05, 'epoch': 0.17}

 17%|█▋        | 6580/37962 [15:16:03<72:39:31,  8.34s/it]
 17%|█▋        | 6581/37962 [15:16:11<72:40:05,  8.34s/it]
 17%|█▋        | 6582/37962 [15:16:19<72:18:21,  8.30s/it]
 17%|█▋        | 6583/37962 [15:16:27<71:36:20,  8.22s/it]
 17%|█▋        | 6584/37962 [15:16:36<71:45:47,  8.23s/it]
 17%|█▋        | 6585/37962 [15:16:44<71:58:10,  8.26s/it]
 17%|█▋        | 6586/37962 [15:16:52<72:27:42,  8.31s/it]
 17%|█▋        | 6587/37962 [15:17:00<71:46:09,  8.23s/it]
 17%|█▋        | 6588/37962 [15:17:09<71:38:08,  8.22s/it]
 17%|█▋        | 6589/37962 [15:17:17<71:59:15,  8.26s/it]
 17%|█▋        | 6590/37962 [15:17:25<72:09:52,  8.28s/it]
                                                          
{'loss': 1.4899, 'grad_norm': 16.339500427246094, 'learning_rate': 1.893795667377118e-05, 'epoch': 0.17}

 17%|█▋        | 6590/37962 [15:17:25<72:09:52,  8.28s/it]
 17%|█▋        | 6591/37962 [15:17:34<72:15:25,  8.29s/it]
 17%|█▋        | 6592/37962 [15:17:42<72:04:06,  8.27s/it]
 17%|█▋        | 6593/37962 [15:17:50<72:27:37,  8.32s/it]
 17%|█▋        | 6594/37962 [15:17:58<71:52:29,  8.25s/it]
 17%|█▋        | 6595/37962 [15:18:07<72:07:05,  8.28s/it]
 17%|█▋        | 6596/37962 [15:18:15<71:57:28,  8.26s/it]
 17%|█▋        | 6597/37962 [15:18:23<73:01:17,  8.38s/it]
 17%|█▋        | 6598/37962 [15:18:32<72:46:44,  8.35s/it]
 17%|█▋        | 6599/37962 [15:18:40<72:27:14,  8.32s/it]
 17%|█▋        | 6600/37962 [15:18:48<72:17:53,  8.30s/it]
                                                          
{'loss': 1.5503, 'grad_norm': 2.8750782012939453, 'learning_rate': 1.8934127215054604e-05, 'epoch': 0.17}

 17%|█▋        | 6600/37962 [15:18:48<72:17:53,  8.30s/it]
 17%|█▋        | 6601/37962 [15:18:57<72:16:59,  8.30s/it]
 17%|█▋        | 6602/37962 [15:19:05<71:46:41,  8.24s/it]
 17%|█▋        | 6603/37962 [15:19:13<71:51:25,  8.25s/it]
 17%|█▋        | 6604/37962 [15:19:21<71:26:16,  8.20s/it]
 17%|█▋        | 6605/37962 [15:19:29<71:14:53,  8.18s/it]
 17%|█▋        | 6606/37962 [15:19:38<71:55:51,  8.26s/it]
 17%|█▋        | 6607/37962 [15:19:45<70:35:03,  8.10s/it]
 17%|█▋        | 6608/37962 [15:19:54<71:01:45,  8.16s/it]
 17%|█▋        | 6609/37962 [15:20:02<70:48:29,  8.13s/it]
 17%|█▋        | 6610/37962 [15:20:10<71:57:08,  8.26s/it]
                                                          
{'loss': 1.4165, 'grad_norm': 25.59984588623047, 'learning_rate': 1.8930291253341608e-05, 'epoch': 0.17}

 17%|█▋        | 6610/37962 [15:20:10<71:57:08,  8.26s/it]
 17%|█▋        | 6611/37962 [15:20:18<71:44:26,  8.24s/it]
 17%|█▋        | 6612/37962 [15:20:27<71:53:28,  8.26s/it]
 17%|█▋        | 6613/37962 [15:20:35<72:33:37,  8.33s/it]
 17%|█▋        | 6614/37962 [15:20:44<73:04:28,  8.39s/it]
 17%|█▋        | 6615/37962 [15:20:52<72:30:52,  8.33s/it]
 17%|█▋        | 6616/37962 [15:21:00<72:46:42,  8.36s/it]
 17%|█▋        | 6617/37962 [15:21:09<72:46:11,  8.36s/it]
 17%|█▋        | 6618/37962 [15:21:17<72:38:03,  8.34s/it]
 17%|█▋        | 6619/37962 [15:21:25<72:26:52,  8.32s/it]
 17%|█▋        | 6620/37962 [15:21:34<72:13:24,  8.30s/it]
                                                          
{'loss': 1.4003, 'grad_norm': 9.73617935180664, 'learning_rate': 1.8926448791424313e-05, 'epoch': 0.17}

 17%|█▋        | 6620/37962 [15:21:34<72:13:24,  8.30s/it]
 17%|█▋        | 6621/37962 [15:21:42<72:03:37,  8.28s/it]
 17%|█▋        | 6622/37962 [15:21:50<71:25:02,  8.20s/it]
 17%|█▋        | 6623/37962 [15:21:58<71:21:11,  8.20s/it]
 17%|█▋        | 6624/37962 [15:22:07<72:12:35,  8.30s/it]
 17%|█▋        | 6625/37962 [15:22:15<71:48:57,  8.25s/it]
 17%|█▋        | 6626/37962 [15:22:23<71:44:04,  8.24s/it]
 17%|█▋        | 6627/37962 [15:22:32<72:50:58,  8.37s/it]
 17%|█▋        | 6628/37962 [15:22:40<73:05:14,  8.40s/it]
 17%|█▋        | 6629/37962 [15:22:48<72:25:36,  8.32s/it]
 17%|█▋        | 6630/37962 [15:22:56<72:07:45,  8.29s/it]
                                                          
{'loss': 1.4631, 'grad_norm': 6.157930374145508, 'learning_rate': 1.8922599832099585e-05, 'epoch': 0.17}

 17%|█▋        | 6630/37962 [15:22:56<72:07:45,  8.29s/it]
 17%|█▋        | 6631/37962 [15:23:05<72:53:12,  8.37s/it]
 17%|█▋        | 6632/37962 [15:23:13<72:38:55,  8.35s/it]
 17%|█▋        | 6633/37962 [15:23:22<73:04:51,  8.40s/it]
 17%|█▋        | 6634/37962 [15:23:30<72:44:16,  8.36s/it]
 17%|█▋        | 6635/37962 [15:23:39<73:11:15,  8.41s/it]
 17%|█▋        | 6636/37962 [15:23:47<72:47:21,  8.36s/it]
 17%|█▋        | 6637/37962 [15:23:55<72:23:30,  8.32s/it]
 17%|█▋        | 6638/37962 [15:24:03<72:05:18,  8.28s/it]
 17%|█▋        | 6639/37962 [15:24:12<72:11:07,  8.30s/it]
 17%|█▋        | 6640/37962 [15:24:20<71:49:41,  8.26s/it]
                                                          
{'loss': 1.548, 'grad_norm': 28.484355926513672, 'learning_rate': 1.8918744378169012e-05, 'epoch': 0.17}

 17%|█▋        | 6640/37962 [15:24:20<71:49:41,  8.26s/it]
 17%|█▋        | 6641/37962 [15:24:28<72:22:49,  8.32s/it]
 17%|█▋        | 6642/37962 [15:24:36<71:16:26,  8.19s/it]
 17%|█▋        | 6643/37962 [15:24:44<71:21:48,  8.20s/it]
 18%|█▊        | 6644/37962 [15:24:52<70:38:24,  8.12s/it]
 18%|█▊        | 6645/37962 [15:25:00<69:53:36,  8.03s/it]
 18%|█▊        | 6646/37962 [15:25:08<70:22:09,  8.09s/it]
 18%|█▊        | 6647/37962 [15:25:17<71:06:47,  8.18s/it]
 18%|█▊        | 6648/37962 [15:25:25<71:16:48,  8.19s/it]
 18%|█▊        | 6649/37962 [15:25:33<71:32:37,  8.23s/it]
 18%|█▊        | 6650/37962 [15:25:41<71:42:35,  8.24s/it]
                                                          
{'loss': 1.3877, 'grad_norm': 18.83116340637207, 'learning_rate': 1.891488243243891e-05, 'epoch': 0.18}

 18%|█▊        | 6650/37962 [15:25:41<71:42:35,  8.24s/it]
 18%|█▊        | 6651/37962 [15:25:50<71:58:05,  8.27s/it]
 18%|█▊        | 6652/37962 [15:25:58<71:19:22,  8.20s/it]
 18%|█▊        | 6653/37962 [15:26:06<71:11:16,  8.19s/it]
 18%|█▊        | 6654/37962 [15:26:14<71:05:27,  8.17s/it]
 18%|█▊        | 6655/37962 [15:26:23<71:45:35,  8.25s/it]
 18%|█▊        | 6656/37962 [15:26:31<72:08:35,  8.30s/it]
 18%|█▊        | 6657/37962 [15:26:39<72:04:04,  8.29s/it]
 18%|█▊        | 6658/37962 [15:26:47<71:49:42,  8.26s/it]
 18%|█▊        | 6659/37962 [15:26:56<71:26:09,  8.22s/it]
 18%|█▊        | 6660/37962 [15:27:04<71:33:04,  8.23s/it]
                                                          
{'loss': 1.4675, 'grad_norm': 14.265541076660156, 'learning_rate': 1.8911013997720327e-05, 'epoch': 0.18}

 18%|█▊        | 6660/37962 [15:27:04<71:33:04,  8.23s/it]
 18%|█▊        | 6661/37962 [15:27:12<71:26:10,  8.22s/it]
 18%|█▊        | 6662/37962 [15:27:20<71:22:03,  8.21s/it]
 18%|█▊        | 6663/37962 [15:27:28<71:35:28,  8.23s/it]
 18%|█▊        | 6664/37962 [15:27:37<71:36:24,  8.24s/it]
 18%|█▊        | 6665/37962 [15:27:45<71:51:49,  8.27s/it]
 18%|█▊        | 6666/37962 [15:27:53<71:36:40,  8.24s/it]
 18%|█▊        | 6667/37962 [15:28:02<71:43:30,  8.25s/it]
 18%|█▊        | 6668/37962 [15:28:10<73:00:42,  8.40s/it]
 18%|█▊        | 6669/37962 [15:28:18<71:38:36,  8.24s/it]
 18%|█▊        | 6670/37962 [15:28:26<71:34:18,  8.23s/it]
                                                          
{'loss': 1.4116, 'grad_norm': 23.068174362182617, 'learning_rate': 1.8907139076829026e-05, 'epoch': 0.18}

 18%|█▊        | 6670/37962 [15:28:26<71:34:18,  8.23s/it]
 18%|█▊        | 6671/37962 [15:28:35<71:49:32,  8.26s/it]
 18%|█▊        | 6672/37962 [15:28:43<71:31:59,  8.23s/it]
 18%|█▊        | 6673/37962 [15:28:51<71:41:11,  8.25s/it]
 18%|█▊        | 6674/37962 [15:28:59<71:46:39,  8.26s/it]
 18%|█▊        | 6675/37962 [15:29:08<71:53:39,  8.27s/it]
 18%|█▊        | 6676/37962 [15:29:16<72:00:36,  8.29s/it]
 18%|█▊        | 6677/37962 [15:29:25<72:50:19,  8.38s/it]
 18%|█▊        | 6678/37962 [15:29:33<72:51:34,  8.38s/it]
 18%|█▊        | 6679/37962 [15:29:41<72:27:13,  8.34s/it]
 18%|█▊        | 6680/37962 [15:29:49<72:07:23,  8.30s/it]
                                                          
{'loss': 1.4321, 'grad_norm': 4.396834850311279, 'learning_rate': 1.89032576725855e-05, 'epoch': 0.18}

 18%|█▊        | 6680/37962 [15:29:49<72:07:23,  8.30s/it]
 18%|█▊        | 6681/37962 [15:29:58<71:46:54,  8.26s/it]
 18%|█▊        | 6682/37962 [15:30:06<71:54:44,  8.28s/it]
 18%|█▊        | 6683/37962 [15:30:14<72:13:11,  8.31s/it]
 18%|█▊        | 6684/37962 [15:30:22<71:33:36,  8.24s/it]
 18%|█▊        | 6685/37962 [15:30:30<70:28:23,  8.11s/it]
 18%|█▊        | 6686/37962 [15:30:38<70:31:43,  8.12s/it]
 18%|█▊        | 6687/37962 [15:30:47<70:41:02,  8.14s/it]
 18%|█▊        | 6688/37962 [15:30:54<69:54:03,  8.05s/it]
 18%|█▊        | 6689/37962 [15:31:03<70:09:49,  8.08s/it]
 18%|█▊        | 6690/37962 [15:31:11<70:37:23,  8.13s/it]
                                                          
{'loss': 1.5086, 'grad_norm': 14.342424392700195, 'learning_rate': 1.8899369787814952e-05, 'epoch': 0.18}

 18%|█▊        | 6690/37962 [15:31:11<70:37:23,  8.13s/it]
 18%|█▊        | 6691/37962 [15:31:19<70:54:03,  8.16s/it]
 18%|█▊        | 6692/37962 [15:31:27<70:30:39,  8.12s/it]
 18%|█▊        | 6693/37962 [15:31:36<71:43:14,  8.26s/it]
 18%|█▊        | 6694/37962 [15:31:44<71:45:27,  8.26s/it]
 18%|█▊        | 6695/37962 [15:31:52<71:11:12,  8.20s/it]
 18%|█▊        | 6696/37962 [15:32:00<71:39:52,  8.25s/it]
 18%|█▊        | 6697/37962 [15:32:08<71:29:04,  8.23s/it]
 18%|█▊        | 6698/37962 [15:32:17<71:25:31,  8.22s/it]
 18%|█▊        | 6699/37962 [15:32:25<71:31:17,  8.24s/it]
 18%|█▊        | 6700/37962 [15:32:33<71:39:13,  8.25s/it]
                                                          
{'loss': 1.4711, 'grad_norm': 18.629901885986328, 'learning_rate': 1.8895475425347305e-05, 'epoch': 0.18}

 18%|█▊        | 6700/37962 [15:32:33<71:39:13,  8.25s/it]
 18%|█▊        | 6701/37962 [15:32:41<71:31:56,  8.24s/it]
 18%|█▊        | 6702/37962 [15:32:50<71:15:39,  8.21s/it]
 18%|█▊        | 6703/37962 [15:32:58<71:18:05,  8.21s/it]
 18%|█▊        | 6704/37962 [15:33:06<71:43:13,  8.26s/it]
 18%|█▊        | 6705/37962 [15:33:14<71:53:21,  8.28s/it]
 18%|█▊        | 6706/37962 [15:33:23<71:34:57,  8.24s/it]
 18%|█▊        | 6707/37962 [15:33:31<71:38:11,  8.25s/it]
 18%|█▊        | 6708/37962 [15:33:39<72:06:30,  8.31s/it]
 18%|█▊        | 6709/37962 [15:33:48<72:10:48,  8.31s/it]
 18%|█▊        | 6710/37962 [15:33:56<72:06:14,  8.31s/it]
                                                          
{'loss': 1.5166, 'grad_norm': 21.915367126464844, 'learning_rate': 1.88915745880172e-05, 'epoch': 0.18}

 18%|█▊        | 6710/37962 [15:33:56<72:06:14,  8.31s/it]
 18%|█▊        | 6711/37962 [15:34:04<71:45:16,  8.27s/it]
 18%|█▊        | 6712/37962 [15:34:12<71:54:10,  8.28s/it]
 18%|█▊        | 6713/37962 [15:34:21<71:36:15,  8.25s/it]
 18%|█▊        | 6714/37962 [15:34:29<71:30:16,  8.24s/it]
 18%|█▊        | 6715/37962 [15:34:37<71:58:22,  8.29s/it]
 18%|█▊        | 6716/37962 [15:34:46<71:49:29,  8.28s/it]
 18%|█▊        | 6717/37962 [15:34:53<70:49:50,  8.16s/it]
 18%|█▊        | 6718/37962 [15:35:02<71:11:43,  8.20s/it]
 18%|█▊        | 6719/37962 [15:35:10<71:30:12,  8.24s/it]
 18%|█▊        | 6720/37962 [15:35:18<71:51:00,  8.28s/it]
                                                          
{'loss': 1.3911, 'grad_norm': 23.514169692993164, 'learning_rate': 1.888766727866399e-05, 'epoch': 0.18}

 18%|█▊        | 6720/37962 [15:35:18<71:51:00,  8.28s/it]
 18%|█▊        | 6721/37962 [15:35:27<71:31:47,  8.24s/it]
 18%|█▊        | 6722/37962 [15:35:35<71:27:32,  8.23s/it]
 18%|█▊        | 6723/37962 [15:35:43<71:36:23,  8.25s/it]
 18%|█▊        | 6724/37962 [15:35:51<71:41:50,  8.26s/it]
 18%|█▊        | 6725/37962 [15:36:00<71:38:26,  8.26s/it]
 18%|█▊        | 6726/37962 [15:36:08<72:25:23,  8.35s/it]
 18%|█▊        | 6727/37962 [15:36:16<71:11:32,  8.21s/it]
 18%|█▊        | 6728/37962 [15:36:24<70:16:13,  8.10s/it]
 18%|█▊        | 6729/37962 [15:36:32<70:51:16,  8.17s/it]
 18%|█▊        | 6730/37962 [15:36:40<70:33:46,  8.13s/it]
                                                          
{'loss': 1.5038, 'grad_norm': 21.24363899230957, 'learning_rate': 1.888375350013173e-05, 'epoch': 0.18}

 18%|█▊        | 6730/37962 [15:36:40<70:33:46,  8.13s/it]
 18%|█▊        | 6731/37962 [15:36:48<70:44:39,  8.15s/it]
 18%|█▊        | 6732/37962 [15:36:57<71:07:09,  8.20s/it]
 18%|█▊        | 6733/37962 [15:37:05<71:08:02,  8.20s/it]
 18%|█▊        | 6734/37962 [15:37:13<70:38:36,  8.14s/it]
 18%|█▊        | 6735/37962 [15:37:21<69:32:58,  8.02s/it]
 18%|█▊        | 6736/37962 [15:37:29<70:31:41,  8.13s/it]
 18%|█▊        | 6737/37962 [15:37:38<71:21:27,  8.23s/it]
 18%|█▊        | 6738/37962 [15:37:46<71:25:05,  8.23s/it]
 18%|█▊        | 6739/37962 [15:37:54<71:07:03,  8.20s/it]
 18%|█▊        | 6740/37962 [15:38:02<70:25:03,  8.12s/it]
                                                          
{'loss': 1.4195, 'grad_norm': 12.076475143432617, 'learning_rate': 1.8879833255269203e-05, 'epoch': 0.18}

 18%|█▊        | 6740/37962 [15:38:02<70:25:03,  8.12s/it]
 18%|█▊        | 6741/37962 [15:38:10<70:53:38,  8.17s/it]
 18%|█▊        | 6742/37962 [15:38:18<71:15:49,  8.22s/it]
 18%|█▊        | 6743/37962 [15:38:26<70:42:56,  8.15s/it]
 18%|█▊        | 6744/37962 [15:38:35<70:54:05,  8.18s/it]
 18%|█▊        | 6745/37962 [15:38:43<71:27:22,  8.24s/it]
 18%|█▊        | 6746/37962 [15:38:51<71:18:05,  8.22s/it]
 18%|█▊        | 6747/37962 [15:39:00<71:20:19,  8.23s/it]
 18%|█▊        | 6748/37962 [15:39:08<71:46:13,  8.28s/it]
 18%|█▊        | 6749/37962 [15:39:16<72:06:51,  8.32s/it]
 18%|█▊        | 6750/37962 [15:39:25<71:50:19,  8.29s/it]
                                                          
{'loss': 1.5543, 'grad_norm': 22.349382400512695, 'learning_rate': 1.8875906546929882e-05, 'epoch': 0.18}

 18%|█▊        | 6750/37962 [15:39:25<71:50:19,  8.29s/it]
 18%|█▊        | 6751/37962 [15:39:33<71:52:18,  8.29s/it]
 18%|█▊        | 6752/37962 [15:39:48<89:49:16, 10.36s/it]
 18%|█▊        | 6753/37962 [15:39:56<83:53:34,  9.68s/it]
 18%|█▊        | 6754/37962 [15:40:04<80:16:27,  9.26s/it]
 18%|█▊        | 6755/37962 [15:40:13<77:33:23,  8.95s/it]
 18%|█▊        | 6756/37962 [15:40:21<75:35:29,  8.72s/it]
 18%|█▊        | 6757/37962 [15:40:29<74:40:08,  8.61s/it]
 18%|█▊        | 6758/37962 [15:40:38<73:56:19,  8.53s/it]
 18%|█▊        | 6759/37962 [15:40:46<73:05:36,  8.43s/it]
 18%|█▊        | 6760/37962 [15:40:54<73:13:49,  8.45s/it]
                                                          
{'loss': 1.4779, 'grad_norm': 27.642847061157227, 'learning_rate': 1.8871973377971947e-05, 'epoch': 0.18}

 18%|█▊        | 6760/37962 [15:40:54<73:13:49,  8.45s/it]
 18%|█▊        | 6761/37962 [15:41:03<73:24:59,  8.47s/it]
 18%|█▊        | 6762/37962 [15:41:11<72:17:36,  8.34s/it]
 18%|█▊        | 6763/37962 [15:41:19<71:26:30,  8.24s/it]
 18%|█▊        | 6764/37962 [15:41:27<71:12:45,  8.22s/it]
 18%|█▊        | 6765/37962 [15:41:35<72:06:01,  8.32s/it]
 18%|█▊        | 6766/37962 [15:41:44<71:44:15,  8.28s/it]
 18%|█▊        | 6767/37962 [15:41:52<71:28:36,  8.25s/it]
 18%|█▊        | 6768/37962 [15:42:00<71:55:55,  8.30s/it]
 18%|█▊        | 6769/37962 [15:42:09<71:57:32,  8.30s/it]
 18%|█▊        | 6770/37962 [15:42:17<72:15:14,  8.34s/it]
                                                          
{'loss': 1.4492, 'grad_norm': 17.915102005004883, 'learning_rate': 1.8868033751258288e-05, 'epoch': 0.18}

 18%|█▊        | 6770/37962 [15:42:17<72:15:14,  8.34s/it]
 18%|█▊        | 6771/37962 [15:42:25<72:03:16,  8.32s/it]
 18%|█▊        | 6772/37962 [15:42:34<71:58:27,  8.31s/it]
 18%|█▊        | 6773/37962 [15:42:42<71:43:02,  8.28s/it]
 18%|█▊        | 6774/37962 [15:42:50<72:08:35,  8.33s/it]
 18%|█▊        | 6775/37962 [15:42:58<70:50:11,  8.18s/it]
 18%|█▊        | 6776/37962 [15:43:06<70:58:59,  8.19s/it]
 18%|█▊        | 6777/37962 [15:43:14<70:08:45,  8.10s/it]
 18%|█▊        | 6778/37962 [15:43:22<70:36:39,  8.15s/it]
 18%|█▊        | 6779/37962 [15:43:31<70:30:03,  8.14s/it]
 18%|█▊        | 6780/37962 [15:43:39<70:28:29,  8.14s/it]
                                                          
{'loss': 1.4933, 'grad_norm': 19.781291961669922, 'learning_rate': 1.886408766965649e-05, 'epoch': 0.18}

 18%|█▊        | 6780/37962 [15:43:39<70:28:29,  8.14s/it]
 18%|█▊        | 6781/37962 [15:43:47<71:06:23,  8.21s/it]
 18%|█▊        | 6782/37962 [15:43:55<71:26:20,  8.25s/it]
 18%|█▊        | 6783/37962 [15:44:04<71:57:34,  8.31s/it]
 18%|█▊        | 6784/37962 [15:44:12<71:53:42,  8.30s/it]
 18%|█▊        | 6785/37962 [15:44:20<71:32:30,  8.26s/it]
 18%|█▊        | 6786/37962 [15:44:29<71:49:49,  8.29s/it]
 18%|█▊        | 6787/37962 [15:44:37<71:43:20,  8.28s/it]
 18%|█▊        | 6788/37962 [15:44:45<71:37:59,  8.27s/it]
 18%|█▊        | 6789/37962 [15:44:54<71:52:33,  8.30s/it]
 18%|█▊        | 6790/37962 [15:45:02<71:21:53,  8.24s/it]
                                                          
{'loss': 1.4874, 'grad_norm': 17.94605827331543, 'learning_rate': 1.8860135136038836e-05, 'epoch': 0.18}

 18%|█▊        | 6790/37962 [15:45:02<71:21:53,  8.24s/it]
 18%|█▊        | 6791/37962 [15:45:10<71:19:17,  8.24s/it]
 18%|█▊        | 6792/37962 [15:45:18<71:08:46,  8.22s/it]
 18%|█▊        | 6793/37962 [15:45:26<70:01:02,  8.09s/it]
 18%|█▊        | 6794/37962 [15:45:34<70:11:16,  8.11s/it]
 18%|█▊        | 6795/37962 [15:45:42<70:48:13,  8.18s/it]
 18%|█▊        | 6796/37962 [15:45:51<71:14:12,  8.23s/it]
 18%|█▊        | 6797/37962 [15:45:59<71:11:33,  8.22s/it]
 18%|█▊        | 6798/37962 [15:46:07<70:29:46,  8.14s/it]
 18%|█▊        | 6799/37962 [15:46:15<71:13:50,  8.23s/it]
 18%|█▊        | 6800/37962 [15:46:23<71:15:20,  8.23s/it]
                                                          
{'loss': 1.4479, 'grad_norm': 10.867156982421875, 'learning_rate': 1.885617615328231e-05, 'epoch': 0.18}

 18%|█▊        | 6800/37962 [15:46:23<71:15:20,  8.23s/it]
 18%|█▊        | 6801/37962 [15:46:32<71:23:15,  8.25s/it]
 18%|█▊        | 6802/37962 [15:46:40<72:08:06,  8.33s/it]
 18%|█▊        | 6803/37962 [15:46:49<72:44:07,  8.40s/it]
 18%|█▊        | 6804/37962 [15:46:57<72:06:47,  8.33s/it]
 18%|█▊        | 6805/37962 [15:47:05<71:17:46,  8.24s/it]
 18%|█▊        | 6806/37962 [15:47:13<71:32:37,  8.27s/it]
 18%|█▊        | 6807/37962 [15:47:21<71:06:09,  8.22s/it]
 18%|█▊        | 6808/37962 [15:47:30<71:21:08,  8.25s/it]
 18%|█▊        | 6809/37962 [15:47:38<71:03:13,  8.21s/it]
 18%|█▊        | 6810/37962 [15:47:46<71:05:54,  8.22s/it]
                                                          
{'loss': 1.4143, 'grad_norm': 4.8006367683410645, 'learning_rate': 1.8852210724268586e-05, 'epoch': 0.18}

 18%|█▊        | 6810/37962 [15:47:46<71:05:54,  8.22s/it]
 18%|█▊        | 6811/37962 [15:47:54<71:13:56,  8.23s/it]
 18%|█▊        | 6812/37962 [15:48:03<71:41:02,  8.28s/it]
 18%|█▊        | 6813/37962 [15:48:11<70:35:24,  8.16s/it]
 18%|█▊        | 6814/37962 [15:48:19<70:44:56,  8.18s/it]
 18%|█▊        | 6815/37962 [15:48:27<70:05:07,  8.10s/it]
 18%|█▊        | 6816/37962 [15:48:35<70:57:10,  8.20s/it]
 18%|█▊        | 6817/37962 [15:48:43<70:35:59,  8.16s/it]
 18%|█▊        | 6818/37962 [15:48:52<70:50:40,  8.19s/it]
 18%|█▊        | 6819/37962 [15:49:00<71:06:24,  8.22s/it]
 18%|█▊        | 6820/37962 [15:49:08<71:46:23,  8.30s/it]
                                                          
{'loss': 1.4277, 'grad_norm': 19.57815933227539, 'learning_rate': 1.8848238851884028e-05, 'epoch': 0.18}

 18%|█▊        | 6820/37962 [15:49:08<71:46:23,  8.30s/it]
 18%|█▊        | 6821/37962 [15:49:16<71:00:51,  8.21s/it]
 18%|█▊        | 6822/37962 [15:49:25<71:15:31,  8.24s/it]
 18%|█▊        | 6823/37962 [15:49:33<70:43:34,  8.18s/it]
 18%|█▊        | 6824/37962 [15:49:41<71:16:10,  8.24s/it]
 18%|█▊        | 6825/37962 [15:49:49<71:22:34,  8.25s/it]
 18%|█▊        | 6826/37962 [15:49:58<71:52:12,  8.31s/it]
 18%|█▊        | 6827/37962 [15:50:06<72:07:54,  8.34s/it]
 18%|█▊        | 6828/37962 [15:50:14<70:54:06,  8.20s/it]
 18%|█▊        | 6829/37962 [15:50:22<71:02:55,  8.22s/it]
 18%|█▊        | 6830/37962 [15:50:31<71:16:01,  8.24s/it]
                                                          
{'loss': 1.3884, 'grad_norm': 11.318155288696289, 'learning_rate': 1.8844260539019695e-05, 'epoch': 0.18}

 18%|█▊        | 6830/37962 [15:50:31<71:16:01,  8.24s/it]
 18%|█▊        | 6831/37962 [15:50:39<71:00:42,  8.21s/it]
 18%|█▊        | 6832/37962 [15:50:47<71:07:10,  8.22s/it]
 18%|█▊        | 6833/37962 [15:50:55<71:19:05,  8.25s/it]
 18%|█▊        | 6834/37962 [15:51:04<71:41:55,  8.29s/it]
 18%|█▊        | 6835/37962 [15:51:12<71:42:34,  8.29s/it]
 18%|█▊        | 6836/37962 [15:51:20<71:43:39,  8.30s/it]
 18%|█▊        | 6837/37962 [15:51:29<71:35:33,  8.28s/it]
 18%|█▊        | 6838/37962 [15:51:37<71:44:32,  8.30s/it]
 18%|█▊        | 6839/37962 [15:51:45<71:25:59,  8.26s/it]
 18%|█▊        | 6840/37962 [15:51:53<71:22:38,  8.26s/it]
                                                          
{'loss': 1.432, 'grad_norm': 18.798397064208984, 'learning_rate': 1.884027578857133e-05, 'epoch': 0.18}

 18%|█▊        | 6840/37962 [15:51:53<71:22:38,  8.26s/it]
 18%|█▊        | 6841/37962 [15:52:01<70:48:45,  8.19s/it]
 18%|█▊        | 6842/37962 [15:52:10<71:15:32,  8.24s/it]
 18%|█▊        | 6843/37962 [15:52:18<71:21:08,  8.25s/it]
 18%|█▊        | 6844/37962 [15:52:26<70:29:22,  8.15s/it]
 18%|█▊        | 6845/37962 [15:52:34<69:22:49,  8.03s/it]
 18%|█▊        | 6846/37962 [15:52:42<69:45:30,  8.07s/it]
 18%|█▊        | 6847/37962 [15:52:50<69:15:10,  8.01s/it]
 18%|█▊        | 6848/37962 [15:52:58<69:41:49,  8.06s/it]
 18%|█▊        | 6849/37962 [15:53:06<70:23:08,  8.14s/it]
 18%|█▊        | 6850/37962 [15:53:14<70:19:31,  8.14s/it]
                                                          
{'loss': 1.5442, 'grad_norm': 22.318639755249023, 'learning_rate': 1.8836284603439368e-05, 'epoch': 0.18}

 18%|█▊        | 6850/37962 [15:53:14<70:19:31,  8.14s/it]
 18%|█▊        | 6851/37962 [15:53:23<70:37:26,  8.17s/it]
 18%|█▊        | 6852/37962 [15:53:31<70:52:04,  8.20s/it]
 18%|█▊        | 6853/37962 [15:53:39<70:13:14,  8.13s/it]
 18%|█▊        | 6854/37962 [15:53:47<70:34:09,  8.17s/it]
 18%|█▊        | 6855/37962 [15:53:55<70:45:27,  8.19s/it]
 18%|█▊        | 6856/37962 [15:54:04<70:58:43,  8.21s/it]
 18%|█▊        | 6857/37962 [15:54:12<70:58:34,  8.21s/it]
 18%|█▊        | 6858/37962 [15:54:20<70:59:01,  8.22s/it]
 18%|█▊        | 6859/37962 [15:54:28<70:56:06,  8.21s/it]
 18%|█▊        | 6860/37962 [15:54:36<69:47:45,  8.08s/it]
                                                          
{'loss': 1.5079, 'grad_norm': 20.5391902923584, 'learning_rate': 1.883228698652892e-05, 'epoch': 0.18}

 18%|█▊        | 6860/37962 [15:54:36<69:47:45,  8.08s/it]
 18%|█▊        | 6861/37962 [15:54:44<69:32:47,  8.05s/it]
 18%|█▊        | 6862/37962 [15:54:52<69:52:11,  8.09s/it]
 18%|█▊        | 6863/37962 [15:55:00<69:27:23,  8.04s/it]
 18%|█▊        | 6864/37962 [15:55:08<69:04:52,  8.00s/it]
 18%|█▊        | 6865/37962 [15:55:16<69:44:25,  8.07s/it]
 18%|█▊        | 6866/37962 [15:55:25<70:15:47,  8.13s/it]
 18%|█▊        | 6867/37962 [15:55:33<71:04:20,  8.23s/it]
 18%|█▊        | 6868/37962 [15:55:41<70:51:03,  8.20s/it]
 18%|█▊        | 6869/37962 [15:55:50<71:17:25,  8.25s/it]
 18%|█▊        | 6870/37962 [15:55:58<71:05:15,  8.23s/it]
                                                          
{'loss': 1.4573, 'grad_norm': 7.627932071685791, 'learning_rate': 1.8828282940749785e-05, 'epoch': 0.18}

 18%|█▊        | 6870/37962 [15:55:58<71:05:15,  8.23s/it]
 18%|█▊        | 6871/37962 [15:56:06<71:22:22,  8.26s/it]
 18%|█▊        | 6872/37962 [15:56:14<71:15:11,  8.25s/it]
 18%|█▊        | 6873/37962 [15:56:22<70:18:45,  8.14s/it]
 18%|█▊        | 6874/37962 [15:56:31<70:58:11,  8.22s/it]
 18%|█▊        | 6875/37962 [15:56:39<71:33:59,  8.29s/it]
 18%|█▊        | 6876/37962 [15:56:47<71:43:46,  8.31s/it]
 18%|█▊        | 6877/37962 [15:56:56<71:48:29,  8.32s/it]
 18%|█▊        | 6878/37962 [15:57:04<70:41:34,  8.19s/it]
 18%|█▊        | 6879/37962 [15:57:12<71:16:30,  8.26s/it]
 18%|█▊        | 6880/37962 [15:57:20<71:45:14,  8.31s/it]
                                                          
{'loss': 1.3682, 'grad_norm': 2.4700329303741455, 'learning_rate': 1.8824272469016436e-05, 'epoch': 0.18}

 18%|█▊        | 6880/37962 [15:57:20<71:45:14,  8.31s/it]
 18%|█▊        | 6881/37962 [15:57:29<71:34:41,  8.29s/it]
 18%|█▊        | 6882/37962 [15:57:37<71:48:21,  8.32s/it]
 18%|█▊        | 6883/37962 [15:57:45<71:31:23,  8.28s/it]
 18%|█▊        | 6884/37962 [15:57:54<71:50:50,  8.32s/it]
 18%|█▊        | 6885/37962 [15:58:02<71:51:46,  8.32s/it]
 18%|█▊        | 6886/37962 [15:58:10<71:33:16,  8.29s/it]
 18%|█▊        | 6887/37962 [15:58:18<71:19:41,  8.26s/it]
 18%|█▊        | 6888/37962 [15:58:27<71:53:50,  8.33s/it]
 18%|█▊        | 6889/37962 [15:58:35<71:06:47,  8.24s/it]
 18%|█▊        | 6890/37962 [15:58:43<70:25:39,  8.16s/it]
                                                          
{'loss': 1.4452, 'grad_norm': 13.579471588134766, 'learning_rate': 1.8820255574248027e-05, 'epoch': 0.18}

 18%|█▊        | 6890/37962 [15:58:43<70:25:39,  8.16s/it]
 18%|█▊        | 6891/37962 [15:58:51<70:39:15,  8.19s/it]
 18%|█▊        | 6892/37962 [15:58:59<71:06:29,  8.24s/it]
 18%|█▊        | 6893/37962 [15:59:08<71:20:02,  8.27s/it]
 18%|█▊        | 6894/37962 [15:59:16<70:46:48,  8.20s/it]
 18%|█▊        | 6895/37962 [15:59:24<71:02:33,  8.23s/it]
 18%|█▊        | 6896/37962 [15:59:33<71:53:11,  8.33s/it]
 18%|█▊        | 6897/37962 [15:59:41<71:59:53,  8.34s/it]
 18%|█▊        | 6898/37962 [15:59:49<71:46:58,  8.32s/it]
 18%|█▊        | 6899/37962 [15:59:57<71:05:57,  8.24s/it]
 18%|█▊        | 6900/37962 [16:00:06<70:42:40,  8.20s/it]
                                                          
{'loss': 1.5167, 'grad_norm': 17.530471801757812, 'learning_rate': 1.8816232259368385e-05, 'epoch': 0.18}

 18%|█▊        | 6900/37962 [16:00:06<70:42:40,  8.20s/it]
 18%|█▊        | 6901/37962 [16:00:14<71:15:09,  8.26s/it]
 18%|█▊        | 6902/37962 [16:00:22<70:06:55,  8.13s/it]
 18%|█▊        | 6903/37962 [16:00:30<70:33:20,  8.18s/it]
 18%|█▊        | 6904/37962 [16:00:38<70:29:06,  8.17s/it]
 18%|█▊        | 6905/37962 [16:00:47<71:07:16,  8.24s/it]
 18%|█▊        | 6906/37962 [16:00:55<71:20:17,  8.27s/it]
 18%|█▊        | 6907/37962 [16:01:03<71:17:57,  8.27s/it]
 18%|█▊        | 6908/37962 [16:01:11<71:01:29,  8.23s/it]
 18%|█▊        | 6909/37962 [16:01:20<71:02:15,  8.24s/it]
 18%|█▊        | 6910/37962 [16:01:28<71:05:22,  8.24s/it]
                                                          
{'loss': 1.482, 'grad_norm': 7.288733959197998, 'learning_rate': 1.8812202527306014e-05, 'epoch': 0.18}

 18%|█▊        | 6910/37962 [16:01:28<71:05:22,  8.24s/it]
 18%|█▊        | 6911/37962 [16:01:36<70:33:10,  8.18s/it]
 18%|█▊        | 6912/37962 [16:01:44<70:43:20,  8.20s/it]
 18%|█▊        | 6913/37962 [16:01:59<88:33:22, 10.27s/it]
 18%|█▊        | 6914/37962 [16:02:15<102:12:42, 11.85s/it]
 18%|█▊        | 6915/37962 [16:02:23<93:02:42, 10.79s/it] 
 18%|█▊        | 6916/37962 [16:02:31<86:38:06, 10.05s/it]
 18%|█▊        | 6917/37962 [16:02:40<82:13:03,  9.53s/it]
 18%|█▊        | 6918/37962 [16:02:48<79:19:33,  9.20s/it]
 18%|█▊        | 6919/37962 [16:02:56<76:51:23,  8.91s/it]
 18%|█▊        | 6920/37962 [16:03:04<74:43:33,  8.67s/it]
                                                          
{'loss': 1.4378, 'grad_norm': 12.44015121459961, 'learning_rate': 1.8808166380994082e-05, 'epoch': 0.18}

 18%|█▊        | 6920/37962 [16:03:04<74:43:33,  8.67s/it]
 18%|█▊        | 6921/37962 [16:03:13<73:46:20,  8.56s/it]
 18%|█▊        | 6922/37962 [16:03:21<71:56:38,  8.34s/it]
 18%|█▊        | 6923/37962 [16:03:29<71:16:17,  8.27s/it]
 18%|█▊        | 6924/37962 [16:03:37<71:03:47,  8.24s/it]
 18%|█▊        | 6925/37962 [16:03:45<70:51:09,  8.22s/it]
 18%|█▊        | 6926/37962 [16:03:54<71:27:37,  8.29s/it]
 18%|█▊        | 6927/37962 [16:04:02<71:17:24,  8.27s/it]
 18%|█▊        | 6928/37962 [16:04:10<71:26:22,  8.29s/it]
 18%|█▊        | 6929/37962 [16:04:18<71:31:41,  8.30s/it]
 18%|█▊        | 6930/37962 [16:04:27<71:21:32,  8.28s/it]
                                                          
{'loss': 1.461, 'grad_norm': 9.751638412475586, 'learning_rate': 1.8804123823370436e-05, 'epoch': 0.18}

 18%|█▊        | 6930/37962 [16:04:27<71:21:32,  8.28s/it]
 18%|█▊        | 6931/37962 [16:04:35<72:12:02,  8.38s/it]
 18%|█▊        | 6932/37962 [16:04:44<72:08:02,  8.37s/it]
 18%|█▊        | 6933/37962 [16:04:52<71:56:57,  8.35s/it]
 18%|█▊        | 6934/37962 [16:05:00<71:26:53,  8.29s/it]
 18%|█▊        | 6935/37962 [16:05:08<71:19:01,  8.27s/it]
 18%|█▊        | 6936/37962 [16:05:17<71:23:10,  8.28s/it]
 18%|█▊        | 6937/37962 [16:05:25<71:28:24,  8.29s/it]
 18%|█▊        | 6938/37962 [16:05:33<71:34:43,  8.31s/it]
 18%|█▊        | 6939/37962 [16:05:41<71:18:19,  8.27s/it]
 18%|█▊        | 6940/37962 [16:05:50<71:01:41,  8.24s/it]
                                                          
{'loss': 1.4777, 'grad_norm': 16.764854431152344, 'learning_rate': 1.8800074857377576e-05, 'epoch': 0.18}

 18%|█▊        | 6940/37962 [16:05:50<71:01:41,  8.24s/it]
 18%|█▊        | 6941/37962 [16:05:58<71:03:56,  8.25s/it]
 18%|█▊        | 6942/37962 [16:06:06<71:25:47,  8.29s/it]
 18%|█▊        | 6943/37962 [16:06:14<70:48:21,  8.22s/it]
 18%|█▊        | 6944/37962 [16:06:22<70:43:29,  8.21s/it]
 18%|█▊        | 6945/37962 [16:06:31<70:41:50,  8.21s/it]
 18%|█▊        | 6946/37962 [16:06:39<71:12:10,  8.26s/it]
 18%|█▊        | 6947/37962 [16:06:47<70:15:01,  8.15s/it]
 18%|█▊        | 6948/37962 [16:06:55<70:19:25,  8.16s/it]
 18%|█▊        | 6949/37962 [16:07:03<70:37:43,  8.20s/it]
 18%|█▊        | 6950/37962 [16:07:12<70:31:21,  8.19s/it]
                                                          
{'loss': 1.4068, 'grad_norm': 12.488768577575684, 'learning_rate': 1.879601948596268e-05, 'epoch': 0.18}

 18%|█▊        | 6950/37962 [16:07:12<70:31:21,  8.19s/it]
 18%|█▊        | 6951/37962 [16:07:20<70:30:45,  8.19s/it]
 18%|█▊        | 6952/37962 [16:07:28<70:06:36,  8.14s/it]
 18%|█▊        | 6953/37962 [16:07:36<70:46:40,  8.22s/it]
 18%|█▊        | 6954/37962 [16:07:44<70:55:21,  8.23s/it]
 18%|█▊        | 6955/37962 [16:07:53<70:48:13,  8.22s/it]
 18%|█▊        | 6956/37962 [16:08:01<71:10:18,  8.26s/it]
 18%|█▊        | 6957/37962 [16:08:09<71:13:46,  8.27s/it]
 18%|█▊        | 6958/37962 [16:08:18<71:22:13,  8.29s/it]
 18%|█▊        | 6959/37962 [16:08:26<71:33:01,  8.31s/it]
 18%|█▊        | 6960/37962 [16:08:34<70:41:46,  8.21s/it]
                                                          
{'loss': 1.4182, 'grad_norm': 6.873770236968994, 'learning_rate': 1.879195771207758e-05, 'epoch': 0.18}

 18%|█▊        | 6960/37962 [16:08:34<70:41:46,  8.21s/it]
 18%|█▊        | 6961/37962 [16:08:42<70:56:41,  8.24s/it]
 18%|█▊        | 6962/37962 [16:08:50<70:06:15,  8.14s/it]
 18%|█▊        | 6963/37962 [16:08:58<70:09:03,  8.15s/it]
 18%|█▊        | 6964/37962 [16:09:06<69:48:44,  8.11s/it]
 18%|█▊        | 6965/37962 [16:09:15<70:18:16,  8.17s/it]
 18%|█▊        | 6966/37962 [16:09:23<70:18:19,  8.17s/it]
 18%|█▊        | 6967/37962 [16:09:31<71:22:42,  8.29s/it]
 18%|█▊        | 6968/37962 [16:09:40<71:45:09,  8.33s/it]
 18%|█▊        | 6969/37962 [16:09:48<71:26:23,  8.30s/it]
 18%|█▊        | 6970/37962 [16:09:56<71:20:31,  8.29s/it]
                                                          
{'loss': 1.4373, 'grad_norm': 19.818965911865234, 'learning_rate': 1.8787889538678775e-05, 'epoch': 0.18}

 18%|█▊        | 6970/37962 [16:09:56<71:20:31,  8.29s/it]
 18%|█▊        | 6971/37962 [16:10:04<70:33:00,  8.20s/it]
 18%|█▊        | 6972/37962 [16:10:13<70:42:15,  8.21s/it]
 18%|█▊        | 6973/37962 [16:10:21<70:55:28,  8.24s/it]
 18%|█▊        | 6974/37962 [16:10:29<70:01:46,  8.14s/it]
 18%|█▊        | 6975/37962 [16:10:37<70:44:23,  8.22s/it]
 18%|█▊        | 6976/37962 [16:10:46<71:12:06,  8.27s/it]
 18%|█▊        | 6977/37962 [16:10:54<71:28:08,  8.30s/it]
 18%|█▊        | 6978/37962 [16:11:02<71:31:14,  8.31s/it]
 18%|█▊        | 6979/37962 [16:11:11<71:21:26,  8.29s/it]
 18%|█▊        | 6980/37962 [16:11:19<71:11:04,  8.27s/it]
                                                          
{'loss': 1.5206, 'grad_norm': 14.178266525268555, 'learning_rate': 1.878381496872741e-05, 'epoch': 0.18}

 18%|█▊        | 6980/37962 [16:11:19<71:11:04,  8.27s/it]
 18%|█▊        | 6981/37962 [16:11:27<70:55:16,  8.24s/it]
 18%|█▊        | 6982/37962 [16:11:36<72:01:48,  8.37s/it]
 18%|█▊        | 6983/37962 [16:11:44<71:33:03,  8.31s/it]
 18%|█▊        | 6984/37962 [16:11:52<71:19:25,  8.29s/it]
 18%|█▊        | 6985/37962 [16:12:00<71:04:36,  8.26s/it]
 18%|█▊        | 6986/37962 [16:12:09<71:12:07,  8.28s/it]
 18%|█▊        | 6987/37962 [16:12:16<69:59:07,  8.13s/it]
 18%|█▊        | 6988/37962 [16:12:25<70:13:12,  8.16s/it]
 18%|█▊        | 6989/37962 [16:12:33<70:39:29,  8.21s/it]
 18%|█▊        | 6990/37962 [16:12:41<70:03:56,  8.14s/it]
                                                          
{'loss': 1.4891, 'grad_norm': 19.88665771484375, 'learning_rate': 1.8779734005189302e-05, 'epoch': 0.18}

 18%|█▊        | 6990/37962 [16:12:41<70:03:56,  8.14s/it]
 18%|█▊        | 6991/37962 [16:12:49<70:39:54,  8.21s/it]
 18%|█▊        | 6992/37962 [16:12:57<70:42:34,  8.22s/it]
 18%|█▊        | 6993/37962 [16:13:06<70:56:51,  8.25s/it]
 18%|█▊        | 6994/37962 [16:13:14<70:49:11,  8.23s/it]
 18%|█▊        | 6995/37962 [16:13:22<70:45:03,  8.22s/it]
 18%|█▊        | 6996/37962 [16:13:31<71:05:08,  8.26s/it]
 18%|█▊        | 6997/37962 [16:13:39<70:56:13,  8.25s/it]
 18%|█▊        | 6998/37962 [16:13:47<70:49:34,  8.23s/it]
 18%|█▊        | 6999/37962 [16:13:55<70:41:24,  8.22s/it]
 18%|█▊        | 7000/37962 [16:14:03<70:14:14,  8.17s/it]
                                                          
{'loss': 1.463, 'grad_norm': 4.3031206130981445, 'learning_rate': 1.8775646651034906e-05, 'epoch': 0.18}

 18%|█▊        | 7000/37962 [16:14:03<70:14:14,  8.17s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 18%|█▊        | 7001/37962 [16:14:53<178:52:57, 20.80s/it]
 18%|█▊        | 7002/37962 [16:15:02<146:28:23, 17.03s/it]
 18%|█▊        | 7003/37962 [16:15:10<123:43:26, 14.39s/it]
 18%|█▊        | 7004/37962 [16:15:18<107:09:22, 12.46s/it]
 18%|█▊        | 7005/37962 [16:15:26<96:06:25, 11.18s/it] 
 18%|█▊        | 7006/37962 [16:15:34<88:40:41, 10.31s/it]
 18%|█▊        | 7007/37962 [16:15:42<82:31:44,  9.60s/it]
 18%|█▊        | 7008/37962 [16:15:50<78:37:15,  9.14s/it]
 18%|█▊        | 7009/37962 [16:16:06<95:03:48, 11.06s/it]
 18%|█▊        | 7010/37962 [16:16:14<88:11:43, 10.26s/it]
                                                          
{'loss': 1.4278, 'grad_norm': 24.27794647216797, 'learning_rate': 1.8771552909239342e-05, 'epoch': 0.18}

 18%|█▊        | 7010/37962 [16:16:14<88:11:43, 10.26s/it]
 18%|█▊        | 7011/37962 [16:16:23<83:22:59,  9.70s/it]
 18%|█▊        | 7012/37962 [16:16:31<79:38:16,  9.26s/it]
 18%|█▊        | 7013/37962 [16:16:39<77:12:10,  8.98s/it]
 18%|█▊        | 7014/37962 [16:16:55<93:27:09, 10.87s/it]
 18%|█▊        | 7015/37962 [16:17:03<86:46:57, 10.10s/it]
 18%|█▊        | 7016/37962 [16:17:11<81:57:16,  9.53s/it]
 18%|█▊        | 7017/37962 [16:17:19<78:42:10,  9.16s/it]
 18%|█▊        | 7018/37962 [16:17:28<76:15:29,  8.87s/it]
 18%|█▊        | 7019/37962 [16:17:36<74:48:19,  8.70s/it]
 18%|█▊        | 7020/37962 [16:17:44<73:31:44,  8.55s/it]
                                                          
{'loss': 1.4421, 'grad_norm': 17.979900360107422, 'learning_rate': 1.8767452782782372e-05, 'epoch': 0.18}

 18%|█▊        | 7020/37962 [16:17:44<73:31:44,  8.55s/it]
 18%|█▊        | 7021/37962 [16:17:52<72:36:10,  8.45s/it]
 18%|█▊        | 7022/37962 [16:18:01<72:18:42,  8.41s/it]
 19%|█▊        | 7023/37962 [16:18:09<71:48:30,  8.36s/it]
 19%|█▊        | 7024/37962 [16:18:17<71:32:07,  8.32s/it]
 19%|█▊        | 7025/37962 [16:18:25<71:40:22,  8.34s/it]
 19%|█▊        | 7026/37962 [16:18:34<71:42:24,  8.34s/it]
 19%|█▊        | 7027/37962 [16:18:42<71:13:40,  8.29s/it]
 19%|█▊        | 7028/37962 [16:18:50<71:11:52,  8.29s/it]
 19%|█▊        | 7029/37962 [16:18:58<70:47:43,  8.24s/it]
 19%|█▊        | 7030/37962 [16:19:07<70:58:05,  8.26s/it]
                                                          
{'loss': 1.5131, 'grad_norm': 10.763005256652832, 'learning_rate': 1.876334627464841e-05, 'epoch': 0.19}

 19%|█▊        | 7030/37962 [16:19:07<70:58:05,  8.26s/it]
 19%|█▊        | 7031/37962 [16:19:15<71:01:21,  8.27s/it]
 19%|█▊        | 7032/37962 [16:19:23<69:48:27,  8.13s/it]
 19%|█▊        | 7033/37962 [16:19:31<70:32:58,  8.21s/it]
 19%|█▊        | 7034/37962 [16:19:39<70:43:46,  8.23s/it]
 19%|█▊        | 7035/37962 [16:19:48<71:06:44,  8.28s/it]
 19%|█▊        | 7036/37962 [16:19:56<71:23:37,  8.31s/it]
 19%|█▊        | 7037/37962 [16:20:05<71:38:14,  8.34s/it]
 19%|█▊        | 7038/37962 [16:20:13<71:50:38,  8.36s/it]
 19%|█▊        | 7039/37962 [16:20:21<71:53:37,  8.37s/it]
 19%|█▊        | 7040/37962 [16:20:30<71:32:25,  8.33s/it]
                                                          
{'loss': 1.4459, 'grad_norm': 4.550689220428467, 'learning_rate': 1.875923338782651e-05, 'epoch': 0.19}

 19%|█▊        | 7040/37962 [16:20:30<71:32:25,  8.33s/it]
 19%|█▊        | 7041/37962 [16:20:38<71:43:18,  8.35s/it]
 19%|█▊        | 7042/37962 [16:20:46<71:18:25,  8.30s/it]
 19%|█▊        | 7043/37962 [16:20:55<71:47:29,  8.36s/it]
 19%|█▊        | 7044/37962 [16:21:03<71:15:16,  8.30s/it]
 19%|█▊        | 7045/37962 [16:21:11<71:29:22,  8.32s/it]
 19%|█▊        | 7046/37962 [16:21:19<70:54:58,  8.26s/it]
 19%|█▊        | 7047/37962 [16:21:27<69:56:27,  8.14s/it]
 19%|█▊        | 7048/37962 [16:21:36<70:14:52,  8.18s/it]
 19%|█▊        | 7049/37962 [16:21:44<70:10:01,  8.17s/it]
 19%|█▊        | 7050/37962 [16:21:52<70:29:33,  8.21s/it]
                                                          
{'loss': 1.3491, 'grad_norm': 11.719257354736328, 'learning_rate': 1.8755114125310364e-05, 'epoch': 0.19}

 19%|█▊        | 7050/37962 [16:21:52<70:29:33,  8.21s/it]
 19%|█▊        | 7051/37962 [16:22:00<69:36:17,  8.11s/it]
 19%|█▊        | 7052/37962 [16:22:08<69:52:56,  8.14s/it]
 19%|█▊        | 7053/37962 [16:22:16<70:04:24,  8.16s/it]
 19%|█▊        | 7054/37962 [16:22:25<70:32:07,  8.22s/it]
 19%|█▊        | 7055/37962 [16:22:33<70:15:46,  8.18s/it]
 19%|█▊        | 7056/37962 [16:22:41<71:03:31,  8.28s/it]
 19%|█▊        | 7057/37962 [16:22:49<70:46:41,  8.24s/it]
 19%|█▊        | 7058/37962 [16:22:58<70:29:30,  8.21s/it]
 19%|█▊        | 7059/37962 [16:23:06<71:01:19,  8.27s/it]
 19%|█▊        | 7060/37962 [16:23:14<70:28:33,  8.21s/it]
                                                          
{'loss': 1.5198, 'grad_norm': 8.02199649810791, 'learning_rate': 1.8750988490098323e-05, 'epoch': 0.19}

 19%|█▊        | 7060/37962 [16:23:14<70:28:33,  8.21s/it]
 19%|█▊        | 7061/37962 [16:23:22<70:31:44,  8.22s/it]
 19%|█▊        | 7062/37962 [16:23:30<69:19:14,  8.08s/it]
 19%|█▊        | 7063/37962 [16:23:38<69:36:50,  8.11s/it]
 19%|█▊        | 7064/37962 [16:23:46<69:56:18,  8.15s/it]
 19%|█▊        | 7065/37962 [16:23:55<70:05:05,  8.17s/it]
 19%|█▊        | 7066/37962 [16:24:03<70:21:36,  8.20s/it]
 19%|█▊        | 7067/37962 [16:24:11<70:24:31,  8.20s/it]
 19%|█▊        | 7068/37962 [16:24:19<70:44:53,  8.24s/it]
 19%|█▊        | 7069/37962 [16:24:28<71:10:30,  8.29s/it]
 19%|█▊        | 7070/37962 [16:24:36<70:57:15,  8.27s/it]
                                                          
{'loss': 1.4509, 'grad_norm': 21.874345779418945, 'learning_rate': 1.8746856485193355e-05, 'epoch': 0.19}

 19%|█▊        | 7070/37962 [16:24:36<70:57:15,  8.27s/it]
 19%|█▊        | 7071/37962 [16:24:45<71:26:14,  8.33s/it]
 19%|█▊        | 7072/37962 [16:24:53<71:05:09,  8.28s/it]
 19%|█▊        | 7073/37962 [16:25:01<71:03:43,  8.28s/it]
 19%|█▊        | 7074/37962 [16:25:09<70:07:26,  8.17s/it]
 19%|█▊        | 7075/37962 [16:25:17<70:23:57,  8.21s/it]
 19%|█▊        | 7076/37962 [16:25:26<71:00:50,  8.28s/it]
 19%|█▊        | 7077/37962 [16:25:34<70:22:34,  8.20s/it]
 19%|█▊        | 7078/37962 [16:25:42<69:45:44,  8.13s/it]
 19%|█▊        | 7079/37962 [16:25:50<70:01:17,  8.16s/it]
 19%|█▊        | 7080/37962 [16:25:58<70:07:07,  8.17s/it]
                                                          
{'loss': 1.3743, 'grad_norm': 10.484390258789062, 'learning_rate': 1.8742718113603085e-05, 'epoch': 0.19}

 19%|█▊        | 7080/37962 [16:25:58<70:07:07,  8.17s/it]
 19%|█▊        | 7081/37962 [16:26:06<70:34:39,  8.23s/it]
 19%|█▊        | 7082/37962 [16:26:15<71:03:20,  8.28s/it]
 19%|█▊        | 7083/37962 [16:26:23<71:00:59,  8.28s/it]
 19%|█▊        | 7084/37962 [16:26:31<71:00:15,  8.28s/it]
 19%|█▊        | 7085/37962 [16:26:40<71:34:04,  8.34s/it]
 19%|█▊        | 7086/37962 [16:26:48<71:11:31,  8.30s/it]
 19%|█▊        | 7087/37962 [16:26:56<71:31:23,  8.34s/it]
 19%|█▊        | 7088/37962 [16:27:05<72:06:08,  8.41s/it]
 19%|█▊        | 7089/37962 [16:27:13<72:01:44,  8.40s/it]
 19%|█▊        | 7090/37962 [16:27:21<70:42:48,  8.25s/it]
                                                          
{'loss': 1.5748, 'grad_norm': 16.07930564880371, 'learning_rate': 1.8738573378339758e-05, 'epoch': 0.19}

 19%|█▊        | 7090/37962 [16:27:21<70:42:48,  8.25s/it]
 19%|█▊        | 7091/37962 [16:27:37<88:50:04, 10.36s/it]
 19%|█▊        | 7092/37962 [16:27:45<83:43:21,  9.76s/it]
 19%|█▊        | 7093/37962 [16:27:53<80:22:39,  9.37s/it]
 19%|█▊        | 7094/37962 [16:28:01<76:31:12,  8.92s/it]
 19%|█▊        | 7095/37962 [16:28:10<74:44:26,  8.72s/it]
 19%|█▊        | 7096/37962 [16:28:17<72:27:22,  8.45s/it]
 19%|█▊        | 7097/37962 [16:28:26<71:58:13,  8.39s/it]
 19%|█▊        | 7098/37962 [16:28:34<72:00:34,  8.40s/it]
 19%|█▊        | 7099/37962 [16:28:42<71:07:09,  8.30s/it]
 19%|█▊        | 7100/37962 [16:28:50<71:17:41,  8.32s/it]
                                                          
{'loss': 1.447, 'grad_norm': 20.511184692382812, 'learning_rate': 1.8734422282420254e-05, 'epoch': 0.19}

 19%|█▊        | 7100/37962 [16:28:50<71:17:41,  8.32s/it]
 19%|█▊        | 7101/37962 [16:28:59<71:14:21,  8.31s/it]
 19%|█▊        | 7102/37962 [16:29:07<70:47:22,  8.26s/it]
 19%|█▊        | 7103/37962 [16:29:15<70:46:50,  8.26s/it]
 19%|█▊        | 7104/37962 [16:29:23<70:32:36,  8.23s/it]
 19%|█▊        | 7105/37962 [16:29:32<70:33:15,  8.23s/it]
 19%|█▊        | 7106/37962 [16:29:40<70:39:37,  8.24s/it]
 19%|█▊        | 7107/37962 [16:29:48<71:01:51,  8.29s/it]
 19%|█▊        | 7108/37962 [16:29:57<71:09:26,  8.30s/it]
 19%|█▊        | 7109/37962 [16:30:05<70:36:20,  8.24s/it]
 19%|█▊        | 7110/37962 [16:30:13<70:48:36,  8.26s/it]
                                                          
{'loss': 1.4811, 'grad_norm': 25.905662536621094, 'learning_rate': 1.8730264828866084e-05, 'epoch': 0.19}

 19%|█▊        | 7110/37962 [16:30:13<70:48:36,  8.26s/it]
 19%|█▊        | 7111/37962 [16:30:21<70:38:48,  8.24s/it]
 19%|█▊        | 7112/37962 [16:30:29<70:20:44,  8.21s/it]
 19%|█▊        | 7113/37962 [16:30:38<70:46:55,  8.26s/it]
 19%|█▊        | 7114/37962 [16:30:46<70:50:40,  8.27s/it]
 19%|█▊        | 7115/37962 [16:30:54<70:18:36,  8.21s/it]
 19%|█▊        | 7116/37962 [16:31:02<69:48:42,  8.15s/it]
 19%|█▊        | 7117/37962 [16:31:10<69:48:01,  8.15s/it]
 19%|█▉        | 7118/37962 [16:31:18<69:11:54,  8.08s/it]
 19%|█▉        | 7119/37962 [16:31:26<69:35:43,  8.12s/it]
 19%|█▉        | 7120/37962 [16:31:34<69:29:12,  8.11s/it]
                                                          
{'loss': 1.7118, 'grad_norm': 14.59699535369873, 'learning_rate': 1.8726101020703386e-05, 'epoch': 0.19}

 19%|█▉        | 7120/37962 [16:31:34<69:29:12,  8.11s/it]
 19%|█▉        | 7121/37962 [16:31:43<69:25:24,  8.10s/it]
 19%|█▉        | 7122/37962 [16:31:51<69:36:07,  8.12s/it]
 19%|█▉        | 7123/37962 [16:31:59<69:56:30,  8.16s/it]
 19%|█▉        | 7124/37962 [16:32:07<70:36:00,  8.24s/it]
 19%|█▉        | 7125/37962 [16:32:16<70:23:53,  8.22s/it]
 19%|█▉        | 7126/37962 [16:32:24<70:16:49,  8.21s/it]
 19%|█▉        | 7127/37962 [16:32:32<70:36:23,  8.24s/it]
 19%|█▉        | 7128/37962 [16:32:40<70:34:30,  8.24s/it]
 19%|█▉        | 7129/37962 [16:32:49<70:44:36,  8.26s/it]
 19%|█▉        | 7130/37962 [16:32:57<71:05:30,  8.30s/it]
                                                          
{'loss': 1.4987, 'grad_norm': 18.654422760009766, 'learning_rate': 1.8721930860962926e-05, 'epoch': 0.19}

 19%|█▉        | 7130/37962 [16:32:57<71:05:30,  8.30s/it]
 19%|█▉        | 7131/37962 [16:33:06<72:07:33,  8.42s/it]
 19%|█▉        | 7132/37962 [16:33:14<71:26:05,  8.34s/it]
 19%|█▉        | 7133/37962 [16:33:22<70:11:25,  8.20s/it]
 19%|█▉        | 7134/37962 [16:33:30<70:47:12,  8.27s/it]
 19%|█▉        | 7135/37962 [16:33:39<71:08:57,  8.31s/it]
 19%|█▉        | 7136/37962 [16:33:47<70:53:18,  8.28s/it]
 19%|█▉        | 7137/37962 [16:33:55<70:49:21,  8.27s/it]
 19%|█▉        | 7138/37962 [16:34:03<70:38:46,  8.25s/it]
 19%|█▉        | 7139/37962 [16:34:11<69:50:11,  8.16s/it]
 19%|█▉        | 7140/37962 [16:34:19<70:11:08,  8.20s/it]
                                                          
{'loss': 1.6078, 'grad_norm': 18.301010131835938, 'learning_rate': 1.8717754352680087e-05, 'epoch': 0.19}

 19%|█▉        | 7140/37962 [16:34:19<70:11:08,  8.20s/it]
 19%|█▉        | 7141/37962 [16:34:28<70:28:22,  8.23s/it]
 19%|█▉        | 7142/37962 [16:34:36<71:13:06,  8.32s/it]
 19%|█▉        | 7143/37962 [16:34:44<71:00:26,  8.29s/it]
 19%|█▉        | 7144/37962 [16:35:00<89:46:32, 10.49s/it]
 19%|█▉        | 7145/37962 [16:35:09<84:44:30,  9.90s/it]
 19%|█▉        | 7146/37962 [16:35:17<79:35:10,  9.30s/it]
 19%|█▉        | 7147/37962 [16:35:25<77:02:34,  9.00s/it]
 19%|█▉        | 7148/37962 [16:35:33<75:07:22,  8.78s/it]
 19%|█▉        | 7149/37962 [16:35:41<72:30:43,  8.47s/it]
 19%|█▉        | 7150/37962 [16:35:49<71:40:36,  8.37s/it]
                                                          
{'loss': 1.3748, 'grad_norm': 2.747265100479126, 'learning_rate': 1.8713571498894878e-05, 'epoch': 0.19}

 19%|█▉        | 7150/37962 [16:35:49<71:40:36,  8.37s/it]
 19%|█▉        | 7151/37962 [16:35:57<70:44:58,  8.27s/it]
 19%|█▉        | 7152/37962 [16:36:05<70:27:30,  8.23s/it]
 19%|█▉        | 7153/37962 [16:36:14<70:51:18,  8.28s/it]
 19%|█▉        | 7154/37962 [16:36:22<70:43:02,  8.26s/it]
 19%|█▉        | 7155/37962 [16:36:30<71:04:58,  8.31s/it]
 19%|█▉        | 7156/37962 [16:36:38<70:49:14,  8.28s/it]
 19%|█▉        | 7157/37962 [16:36:47<70:52:30,  8.28s/it]
 19%|█▉        | 7158/37962 [16:36:55<70:42:35,  8.26s/it]
 19%|█▉        | 7159/37962 [16:37:03<70:42:26,  8.26s/it]
 19%|█▉        | 7160/37962 [16:37:12<71:06:39,  8.31s/it]
                                                          
{'loss': 1.5463, 'grad_norm': 6.859374046325684, 'learning_rate': 1.870938230265193e-05, 'epoch': 0.19}

 19%|█▉        | 7160/37962 [16:37:12<71:06:39,  8.31s/it]
 19%|█▉        | 7161/37962 [16:37:20<70:53:40,  8.29s/it]
 19%|█▉        | 7162/37962 [16:37:28<70:54:05,  8.29s/it]
 19%|█▉        | 7163/37962 [16:37:36<70:52:01,  8.28s/it]
 19%|█▉        | 7164/37962 [16:37:45<70:41:55,  8.26s/it]
 19%|█▉        | 7165/37962 [16:37:53<69:49:15,  8.16s/it]
 19%|█▉        | 7166/37962 [16:38:01<70:11:07,  8.20s/it]
 19%|█▉        | 7167/37962 [16:38:09<69:51:51,  8.17s/it]
 19%|█▉        | 7168/37962 [16:38:17<69:55:25,  8.17s/it]
 19%|█▉        | 7169/37962 [16:38:25<70:21:30,  8.23s/it]
 19%|█▉        | 7170/37962 [16:38:34<70:22:44,  8.23s/it]
                                                          
{'loss': 1.4062, 'grad_norm': 23.336410522460938, 'learning_rate': 1.8705186767000484e-05, 'epoch': 0.19}

 19%|█▉        | 7170/37962 [16:38:34<70:22:44,  8.23s/it]
 19%|█▉        | 7171/37962 [16:38:42<70:07:29,  8.20s/it]
 19%|█▉        | 7172/37962 [16:38:50<70:12:59,  8.21s/it]
 19%|█▉        | 7173/37962 [16:38:58<69:49:09,  8.16s/it]
 19%|█▉        | 7174/37962 [16:39:06<70:19:54,  8.22s/it]
 19%|█▉        | 7175/37962 [16:39:15<70:39:14,  8.26s/it]
 19%|█▉        | 7176/37962 [16:39:23<71:11:13,  8.32s/it]
 19%|█▉        | 7177/37962 [16:39:32<71:16:59,  8.34s/it]
 19%|█▉        | 7178/37962 [16:39:40<70:13:11,  8.21s/it]
 19%|█▉        | 7179/37962 [16:39:48<70:28:39,  8.24s/it]
 19%|█▉        | 7180/37962 [16:39:56<70:19:27,  8.22s/it]
                                                          
{'loss': 1.4505, 'grad_norm': 7.404491901397705, 'learning_rate': 1.8700984894994398e-05, 'epoch': 0.19}

 19%|█▉        | 7180/37962 [16:39:56<70:19:27,  8.22s/it]
 19%|█▉        | 7181/37962 [16:40:04<70:33:36,  8.25s/it]
 19%|█▉        | 7182/37962 [16:40:13<70:45:20,  8.28s/it]
 19%|█▉        | 7183/37962 [16:40:21<70:45:05,  8.28s/it]
 19%|█▉        | 7184/37962 [16:40:29<70:52:21,  8.29s/it]
 19%|█▉        | 7185/37962 [16:40:38<71:11:43,  8.33s/it]
 19%|█▉        | 7186/37962 [16:40:46<70:10:53,  8.21s/it]
 19%|█▉        | 7187/37962 [16:40:54<69:49:23,  8.17s/it]
 19%|█▉        | 7188/37962 [16:41:02<70:12:48,  8.21s/it]
 19%|█▉        | 7189/37962 [16:41:10<70:23:49,  8.24s/it]
 19%|█▉        | 7190/37962 [16:41:19<70:46:32,  8.28s/it]
                                                          
{'loss': 1.5122, 'grad_norm': 10.213303565979004, 'learning_rate': 1.8696776689692137e-05, 'epoch': 0.19}

 19%|█▉        | 7190/37962 [16:41:19<70:46:32,  8.28s/it]
 19%|█▉        | 7191/37962 [16:41:27<69:46:35,  8.16s/it]
 19%|█▉        | 7192/37962 [16:41:35<70:11:31,  8.21s/it]
 19%|█▉        | 7193/37962 [16:41:43<69:32:12,  8.14s/it]
 19%|█▉        | 7194/37962 [16:41:51<69:28:23,  8.13s/it]
 19%|█▉        | 7195/37962 [16:41:59<69:35:47,  8.14s/it]
 19%|█▉        | 7196/37962 [16:42:07<69:55:42,  8.18s/it]
 19%|█▉        | 7197/37962 [16:42:16<69:57:07,  8.19s/it]
 19%|█▉        | 7198/37962 [16:42:24<70:06:32,  8.20s/it]
 19%|█▉        | 7199/37962 [16:42:32<70:00:46,  8.19s/it]
 19%|█▉        | 7200/37962 [16:42:40<70:11:58,  8.22s/it]
                                                          
{'loss': 1.3118, 'grad_norm': 9.46784782409668, 'learning_rate': 1.869256215415679e-05, 'epoch': 0.19}

 19%|█▉        | 7200/37962 [16:42:40<70:11:58,  8.22s/it]
 19%|█▉        | 7201/37962 [16:42:49<70:26:22,  8.24s/it]
 19%|█▉        | 7202/37962 [16:42:57<70:53:38,  8.30s/it]
 19%|█▉        | 7203/37962 [16:43:05<71:12:58,  8.34s/it]
 19%|█▉        | 7204/37962 [16:43:14<71:06:14,  8.32s/it]
 19%|█▉        | 7205/37962 [16:43:22<70:54:59,  8.30s/it]
 19%|█▉        | 7206/37962 [16:43:30<70:43:18,  8.28s/it]
 19%|█▉        | 7207/37962 [16:43:38<70:04:58,  8.20s/it]
 19%|█▉        | 7208/37962 [16:43:47<70:13:26,  8.22s/it]
 19%|█▉        | 7209/37962 [16:43:55<70:20:54,  8.24s/it]
 19%|█▉        | 7210/37962 [16:44:10<88:57:36, 10.41s/it]
                                                          
{'loss': 1.4458, 'grad_norm': 14.267656326293945, 'learning_rate': 1.8688341291456035e-05, 'epoch': 0.19}

 19%|█▉        | 7210/37962 [16:44:10<88:57:36, 10.41s/it]
 19%|█▉        | 7211/37962 [16:44:19<83:39:17,  9.79s/it]
 19%|█▉        | 7212/37962 [16:44:27<79:50:14,  9.35s/it]
 19%|█▉        | 7213/37962 [16:44:35<76:44:39,  8.98s/it]
 19%|█▉        | 7214/37962 [16:44:43<74:53:03,  8.77s/it]
 19%|█▉        | 7215/37962 [16:44:52<73:32:48,  8.61s/it]
 19%|█▉        | 7216/37962 [16:44:59<71:35:59,  8.38s/it]
 19%|█▉        | 7217/37962 [16:45:08<71:05:54,  8.33s/it]
 19%|█▉        | 7218/37962 [16:45:16<70:32:02,  8.26s/it]
 19%|█▉        | 7219/37962 [16:45:24<70:33:18,  8.26s/it]
 19%|█▉        | 7220/37962 [16:45:32<71:01:59,  8.32s/it]
                                                          
{'loss': 1.4858, 'grad_norm': 4.393117904663086, 'learning_rate': 1.868411410466217e-05, 'epoch': 0.19}

 19%|█▉        | 7220/37962 [16:45:32<71:01:59,  8.32s/it]
 19%|█▉        | 7221/37962 [16:45:41<70:48:10,  8.29s/it]
 19%|█▉        | 7222/37962 [16:45:49<69:52:43,  8.18s/it]
 19%|█▉        | 7223/37962 [16:45:57<70:21:52,  8.24s/it]
 19%|█▉        | 7224/37962 [16:46:06<71:30:11,  8.37s/it]
 19%|█▉        | 7225/37962 [16:46:14<70:59:18,  8.31s/it]
 19%|█▉        | 7226/37962 [16:46:22<70:11:51,  8.22s/it]
 19%|█▉        | 7227/37962 [16:46:30<70:30:09,  8.26s/it]
 19%|█▉        | 7228/37962 [16:46:38<70:32:26,  8.26s/it]
 19%|█▉        | 7229/37962 [16:46:47<70:33:53,  8.27s/it]
 19%|█▉        | 7230/37962 [16:46:55<70:16:40,  8.23s/it]
                                                          
{'loss': 1.4746, 'grad_norm': 18.1352481842041, 'learning_rate': 1.8679880596852092e-05, 'epoch': 0.19}

 19%|█▉        | 7230/37962 [16:46:55<70:16:40,  8.23s/it]
 19%|█▉        | 7231/37962 [16:47:03<70:56:45,  8.31s/it]
 19%|█▉        | 7232/37962 [16:47:12<70:42:17,  8.28s/it]
 19%|█▉        | 7233/37962 [16:47:20<70:32:30,  8.26s/it]
 19%|█▉        | 7234/37962 [16:47:28<70:19:42,  8.24s/it]
 19%|█▉        | 7235/37962 [16:47:36<70:13:04,  8.23s/it]
 19%|█▉        | 7236/37962 [16:47:44<70:10:00,  8.22s/it]
 19%|█▉        | 7237/37962 [16:47:52<69:38:51,  8.16s/it]
 19%|█▉        | 7238/37962 [16:48:01<70:12:32,  8.23s/it]
 19%|█▉        | 7239/37962 [16:48:09<69:10:52,  8.11s/it]
 19%|█▉        | 7240/37962 [16:48:17<69:24:39,  8.13s/it]
                                                          
{'loss': 1.4183, 'grad_norm': 31.056867599487305, 'learning_rate': 1.8675640771107293e-05, 'epoch': 0.19}

 19%|█▉        | 7240/37962 [16:48:17<69:24:39,  8.13s/it]
 19%|█▉        | 7241/37962 [16:48:25<69:08:22,  8.10s/it]
 19%|█▉        | 7242/37962 [16:48:33<70:13:41,  8.23s/it]
 19%|█▉        | 7243/37962 [16:48:42<70:27:04,  8.26s/it]
 19%|█▉        | 7244/37962 [16:48:50<70:41:44,  8.29s/it]
 19%|█▉        | 7245/37962 [16:48:58<70:45:05,  8.29s/it]
 19%|█▉        | 7246/37962 [16:49:07<70:43:25,  8.29s/it]
 19%|█▉        | 7247/37962 [16:49:15<70:41:58,  8.29s/it]
 19%|█▉        | 7248/37962 [16:49:23<71:05:26,  8.33s/it]
 19%|█▉        | 7249/37962 [16:49:32<70:52:56,  8.31s/it]
 19%|█▉        | 7250/37962 [16:49:40<69:55:59,  8.20s/it]
                                                          
{'loss': 1.4688, 'grad_norm': 7.42111873626709, 'learning_rate': 1.8671394630513876e-05, 'epoch': 0.19}

 19%|█▉        | 7250/37962 [16:49:40<69:55:59,  8.20s/it]
 19%|█▉        | 7251/37962 [16:49:48<69:54:37,  8.20s/it]
 19%|█▉        | 7252/37962 [16:49:56<69:54:56,  8.20s/it]
 19%|█▉        | 7253/37962 [16:50:04<70:40:45,  8.29s/it]
 19%|█▉        | 7254/37962 [16:50:13<70:31:23,  8.27s/it]
 19%|█▉        | 7255/37962 [16:50:21<70:23:14,  8.25s/it]
 19%|█▉        | 7256/37962 [16:50:29<70:18:42,  8.24s/it]
 19%|█▉        | 7257/37962 [16:50:38<70:50:12,  8.31s/it]
 19%|█▉        | 7258/37962 [16:50:46<70:37:05,  8.28s/it]
 19%|█▉        | 7259/37962 [16:50:54<70:17:36,  8.24s/it]
 19%|█▉        | 7260/37962 [16:51:02<70:17:24,  8.24s/it]
                                                          
{'loss': 1.4876, 'grad_norm': 21.90095329284668, 'learning_rate': 1.8667142178162526e-05, 'epoch': 0.19}

 19%|█▉        | 7260/37962 [16:51:02<70:17:24,  8.24s/it]
 19%|█▉        | 7261/37962 [16:51:11<70:36:36,  8.28s/it]
 19%|█▉        | 7262/37962 [16:51:19<70:36:46,  8.28s/it]
 19%|█▉        | 7263/37962 [16:51:27<70:02:34,  8.21s/it]
 19%|█▉        | 7264/37962 [16:51:35<70:26:21,  8.26s/it]
 19%|█▉        | 7265/37962 [16:51:44<70:25:38,  8.26s/it]
 19%|█▉        | 7266/37962 [16:51:51<68:59:14,  8.09s/it]
 19%|█▉        | 7267/37962 [16:51:59<69:15:51,  8.12s/it]
 19%|█▉        | 7268/37962 [16:52:08<69:50:14,  8.19s/it]
 19%|█▉        | 7269/37962 [16:52:16<70:14:16,  8.24s/it]
 19%|█▉        | 7270/37962 [16:52:24<70:07:12,  8.22s/it]
                                                          
{'loss': 1.4538, 'grad_norm': 9.613465309143066, 'learning_rate': 1.8662883417148535e-05, 'epoch': 0.19}

 19%|█▉        | 7270/37962 [16:52:24<70:07:12,  8.22s/it]
 19%|█▉        | 7271/37962 [16:52:33<70:20:33,  8.25s/it]
 19%|█▉        | 7272/37962 [16:52:41<70:10:24,  8.23s/it]
 19%|█▉        | 7273/37962 [16:52:49<69:26:12,  8.15s/it]
 19%|█▉        | 7274/37962 [16:52:57<68:58:56,  8.09s/it]
 19%|█▉        | 7275/37962 [16:53:05<69:20:15,  8.13s/it]
 19%|█▉        | 7276/37962 [16:53:13<69:40:51,  8.17s/it]
 19%|█▉        | 7277/37962 [16:53:21<68:41:05,  8.06s/it]
 19%|█▉        | 7278/37962 [16:53:29<69:17:51,  8.13s/it]
 19%|█▉        | 7279/37962 [16:53:38<69:54:15,  8.20s/it]
 19%|█▉        | 7280/37962 [16:53:46<69:54:37,  8.20s/it]
                                                          
{'loss': 1.4316, 'grad_norm': 3.7513937950134277, 'learning_rate': 1.8658618350571775e-05, 'epoch': 0.19}

 19%|█▉        | 7280/37962 [16:53:46<69:54:37,  8.20s/it]
 19%|█▉        | 7281/37962 [16:53:54<69:22:38,  8.14s/it]
 19%|█▉        | 7282/37962 [16:54:02<69:48:36,  8.19s/it]
 19%|█▉        | 7283/37962 [16:54:10<69:25:57,  8.15s/it]
 19%|█▉        | 7284/37962 [16:54:18<69:04:58,  8.11s/it]
 19%|█▉        | 7285/37962 [16:54:27<69:27:44,  8.15s/it]
 19%|█▉        | 7286/37962 [16:54:35<69:09:03,  8.12s/it]
 19%|█▉        | 7287/37962 [16:54:43<69:08:11,  8.11s/it]
 19%|█▉        | 7288/37962 [16:54:51<70:02:29,  8.22s/it]
 19%|█▉        | 7289/37962 [16:54:59<69:59:14,  8.21s/it]
 19%|█▉        | 7290/37962 [16:55:07<69:46:49,  8.19s/it]
                                                          
{'loss': 1.5166, 'grad_norm': 7.581501007080078, 'learning_rate': 1.8654346981536722e-05, 'epoch': 0.19}

 19%|█▉        | 7290/37962 [16:55:07<69:46:49,  8.19s/it]
 19%|█▉        | 7291/37962 [16:55:16<69:59:19,  8.21s/it]
 19%|█▉        | 7292/37962 [16:55:24<69:38:14,  8.17s/it]
 19%|█▉        | 7293/37962 [16:55:32<69:46:23,  8.19s/it]
 19%|█▉        | 7294/37962 [16:55:40<70:09:09,  8.23s/it]
 19%|█▉        | 7295/37962 [16:55:49<70:25:36,  8.27s/it]
 19%|█▉        | 7296/37962 [16:55:57<71:13:01,  8.36s/it]
 19%|█▉        | 7297/37962 [16:56:05<70:43:38,  8.30s/it]
 19%|█▉        | 7298/37962 [16:56:14<70:04:30,  8.23s/it]
 19%|█▉        | 7299/37962 [16:56:22<70:13:54,  8.25s/it]
 19%|█▉        | 7300/37962 [16:56:30<70:18:05,  8.25s/it]
                                                          
{'loss': 1.4835, 'grad_norm': 9.13746452331543, 'learning_rate': 1.8650069313152426e-05, 'epoch': 0.19}

 19%|█▉        | 7300/37962 [16:56:30<70:18:05,  8.25s/it]
 19%|█▉        | 7301/37962 [16:56:38<70:22:54,  8.26s/it]
 19%|█▉        | 7302/37962 [16:56:47<70:18:33,  8.26s/it]
 19%|█▉        | 7303/37962 [16:56:55<70:42:36,  8.30s/it]
 19%|█▉        | 7304/37962 [16:57:03<70:40:38,  8.30s/it]
 19%|█▉        | 7305/37962 [16:57:11<70:04:46,  8.23s/it]
 19%|█▉        | 7306/37962 [16:57:20<70:05:27,  8.23s/it]
 19%|█▉        | 7307/37962 [16:57:28<70:09:12,  8.24s/it]
 19%|█▉        | 7308/37962 [16:57:36<69:21:47,  8.15s/it]
 19%|█▉        | 7309/37962 [16:57:44<69:28:47,  8.16s/it]
 19%|█▉        | 7310/37962 [16:57:52<69:34:27,  8.17s/it]
                                                          
{'loss': 1.3715, 'grad_norm': 9.900339126586914, 'learning_rate': 1.864578534853253e-05, 'epoch': 0.19}

 19%|█▉        | 7310/37962 [16:57:52<69:34:27,  8.17s/it]
 19%|█▉        | 7311/37962 [16:58:00<69:40:30,  8.18s/it]
 19%|█▉        | 7312/37962 [16:58:09<70:01:20,  8.22s/it]
 19%|█▉        | 7313/37962 [16:58:17<69:57:10,  8.22s/it]
 19%|█▉        | 7314/37962 [16:58:25<68:55:53,  8.10s/it]
 19%|█▉        | 7315/37962 [16:58:33<69:35:04,  8.17s/it]
 19%|█▉        | 7316/37962 [16:58:41<69:44:06,  8.19s/it]
 19%|█▉        | 7317/37962 [16:58:50<69:49:34,  8.20s/it]
 19%|█▉        | 7318/37962 [16:58:58<69:48:28,  8.20s/it]
 19%|█▉        | 7319/37962 [16:59:06<70:01:48,  8.23s/it]
 19%|█▉        | 7320/37962 [16:59:14<70:04:56,  8.23s/it]
                                                          
{'loss': 1.2981, 'grad_norm': 17.260103225708008, 'learning_rate': 1.8641495090795258e-05, 'epoch': 0.19}

 19%|█▉        | 7320/37962 [16:59:14<70:04:56,  8.23s/it]
 19%|█▉        | 7321/37962 [16:59:23<70:19:27,  8.26s/it]
 19%|█▉        | 7322/37962 [16:59:31<70:35:45,  8.29s/it]
 19%|█▉        | 7323/37962 [16:59:39<70:45:47,  8.31s/it]
 19%|█▉        | 7324/37962 [16:59:48<70:47:54,  8.32s/it]
 19%|█▉        | 7325/37962 [16:59:56<70:35:00,  8.29s/it]
 19%|█▉        | 7326/37962 [17:00:04<70:53:41,  8.33s/it]
 19%|█▉        | 7327/37962 [17:00:13<70:44:23,  8.31s/it]
 19%|█▉        | 7328/37962 [17:00:21<70:53:59,  8.33s/it]
 19%|█▉        | 7329/37962 [17:00:29<70:40:41,  8.31s/it]
 19%|█▉        | 7330/37962 [17:00:37<70:35:11,  8.30s/it]
                                                          
{'loss': 1.5262, 'grad_norm': 3.2433547973632812, 'learning_rate': 1.8637198543063407e-05, 'epoch': 0.19}

 19%|█▉        | 7330/37962 [17:00:37<70:35:11,  8.30s/it]
 19%|█▉        | 7331/37962 [17:00:46<70:23:44,  8.27s/it]
 19%|█▉        | 7332/37962 [17:00:54<70:07:38,  8.24s/it]
 19%|█▉        | 7333/37962 [17:01:02<70:21:12,  8.27s/it]
 19%|█▉        | 7334/37962 [17:01:11<70:37:52,  8.30s/it]
 19%|█▉        | 7335/37962 [17:01:19<70:39:15,  8.30s/it]
 19%|█▉        | 7336/37962 [17:01:27<70:53:37,  8.33s/it]
 19%|█▉        | 7337/37962 [17:01:36<70:45:21,  8.32s/it]
 19%|█▉        | 7338/37962 [17:01:44<70:35:15,  8.30s/it]
 19%|█▉        | 7339/37962 [17:01:52<70:25:17,  8.28s/it]
 19%|█▉        | 7340/37962 [17:02:00<70:14:13,  8.26s/it]
                                                          
{'loss': 1.5059, 'grad_norm': 14.375350952148438, 'learning_rate': 1.863289570846437e-05, 'epoch': 0.19}

 19%|█▉        | 7340/37962 [17:02:00<70:14:13,  8.26s/it]
 19%|█▉        | 7341/37962 [17:02:08<70:08:13,  8.25s/it]
 19%|█▉        | 7342/37962 [17:02:17<70:04:26,  8.24s/it]
 19%|█▉        | 7343/37962 [17:02:25<70:06:30,  8.24s/it]
 19%|█▉        | 7344/37962 [17:02:33<70:17:25,  8.26s/it]
 19%|█▉        | 7345/37962 [17:02:42<70:46:52,  8.32s/it]
 19%|█▉        | 7346/37962 [17:02:50<70:46:29,  8.32s/it]
 19%|█▉        | 7347/37962 [17:02:58<70:54:08,  8.34s/it]
 19%|█▉        | 7348/37962 [17:03:07<70:44:40,  8.32s/it]
 19%|█▉        | 7349/37962 [17:03:15<70:26:59,  8.28s/it]
 19%|█▉        | 7350/37962 [17:03:23<69:35:37,  8.18s/it]
                                                          
{'loss': 1.4571, 'grad_norm': 12.896259307861328, 'learning_rate': 1.8628586590130102e-05, 'epoch': 0.19}

 19%|█▉        | 7350/37962 [17:03:23<69:35:37,  8.18s/it]
 19%|█▉        | 7351/37962 [17:03:31<70:26:25,  8.28s/it]
 19%|█▉        | 7352/37962 [17:03:40<70:05:18,  8.24s/it]
 19%|█▉        | 7353/37962 [17:03:48<69:54:23,  8.22s/it]
 19%|█▉        | 7354/37962 [17:03:56<70:25:38,  8.28s/it]
 19%|█▉        | 7355/37962 [17:04:04<70:16:11,  8.27s/it]
 19%|█▉        | 7356/37962 [17:04:13<70:17:03,  8.27s/it]
 19%|█▉        | 7357/37962 [17:04:20<69:14:53,  8.15s/it]
 19%|█▉        | 7358/37962 [17:04:29<70:16:10,  8.27s/it]
 19%|█▉        | 7359/37962 [17:04:38<71:07:17,  8.37s/it]
 19%|█▉        | 7360/37962 [17:04:46<70:10:56,  8.26s/it]
                                                          
{'loss': 1.2972, 'grad_norm': 23.574466705322266, 'learning_rate': 1.8624271191197133e-05, 'epoch': 0.19}

 19%|█▉        | 7360/37962 [17:04:46<70:10:56,  8.26s/it]
 19%|█▉        | 7361/37962 [17:04:54<70:15:57,  8.27s/it]
 19%|█▉        | 7362/37962 [17:05:02<70:45:07,  8.32s/it]
 19%|█▉        | 7363/37962 [17:05:11<70:19:23,  8.27s/it]
 19%|█▉        | 7364/37962 [17:05:19<70:19:52,  8.27s/it]
 19%|█▉        | 7365/37962 [17:05:27<70:06:15,  8.25s/it]
 19%|█▉        | 7366/37962 [17:05:35<70:07:28,  8.25s/it]
 19%|█▉        | 7367/37962 [17:05:44<70:51:05,  8.34s/it]
 19%|█▉        | 7368/37962 [17:05:52<70:34:49,  8.31s/it]
 19%|█▉        | 7369/37962 [17:06:00<70:42:33,  8.32s/it]
 19%|█▉        | 7370/37962 [17:06:09<70:40:54,  8.32s/it]
                                                          
{'loss': 1.4591, 'grad_norm': 12.558173179626465, 'learning_rate': 1.8619949514806573e-05, 'epoch': 0.19}

 19%|█▉        | 7370/37962 [17:06:09<70:40:54,  8.32s/it]
 19%|█▉        | 7371/37962 [17:06:17<70:40:23,  8.32s/it]
 19%|█▉        | 7372/37962 [17:06:25<70:35:02,  8.31s/it]
 19%|█▉        | 7373/37962 [17:06:33<69:56:42,  8.23s/it]
 19%|█▉        | 7374/37962 [17:06:42<70:09:25,  8.26s/it]
 19%|█▉        | 7375/37962 [17:06:50<69:54:09,  8.23s/it]
 19%|█▉        | 7376/37962 [17:06:58<70:05:22,  8.25s/it]
 19%|█▉        | 7377/37962 [17:07:06<70:05:27,  8.25s/it]
 19%|█▉        | 7378/37962 [17:07:14<69:37:15,  8.19s/it]
 19%|█▉        | 7379/37962 [17:07:23<69:52:17,  8.22s/it]
 19%|█▉        | 7380/37962 [17:07:31<70:02:44,  8.25s/it]
                                                          
{'loss': 1.4461, 'grad_norm': 12.582833290100098, 'learning_rate': 1.8615621564104084e-05, 'epoch': 0.19}

 19%|█▉        | 7380/37962 [17:07:31<70:02:44,  8.25s/it]
 19%|█▉        | 7381/37962 [17:07:39<69:30:42,  8.18s/it]
 19%|█▉        | 7382/37962 [17:07:47<68:45:04,  8.09s/it]
 19%|█▉        | 7383/37962 [17:07:55<68:22:25,  8.05s/it]
 19%|█▉        | 7384/37962 [17:08:03<69:17:36,  8.16s/it]
 19%|█▉        | 7385/37962 [17:08:12<69:37:47,  8.20s/it]
 19%|█▉        | 7386/37962 [17:08:20<70:05:00,  8.25s/it]
 19%|█▉        | 7387/37962 [17:08:28<70:01:06,  8.24s/it]
 19%|█▉        | 7388/37962 [17:08:37<70:19:27,  8.28s/it]
 19%|█▉        | 7389/37962 [17:08:45<70:22:40,  8.29s/it]
 19%|█▉        | 7390/37962 [17:08:53<70:25:03,  8.29s/it]
                                                          
{'loss': 1.5034, 'grad_norm': 11.92378044128418, 'learning_rate': 1.8611287342239922e-05, 'epoch': 0.19}

 19%|█▉        | 7390/37962 [17:08:53<70:25:03,  8.29s/it]
 19%|█▉        | 7391/37962 [17:09:01<70:03:43,  8.25s/it]
 19%|█▉        | 7392/37962 [17:09:10<70:35:43,  8.31s/it]
 19%|█▉        | 7393/37962 [17:09:18<69:42:55,  8.21s/it]
 19%|█▉        | 7394/37962 [17:09:26<69:55:52,  8.24s/it]
 19%|█▉        | 7395/37962 [17:09:34<70:03:59,  8.25s/it]
 19%|█▉        | 7396/37962 [17:09:43<70:02:09,  8.25s/it]
 19%|█▉        | 7397/37962 [17:09:51<69:38:48,  8.20s/it]
 19%|█▉        | 7398/37962 [17:09:59<69:46:06,  8.22s/it]
 19%|█▉        | 7399/37962 [17:10:07<69:09:04,  8.15s/it]
 19%|█▉        | 7400/37962 [17:10:15<69:15:18,  8.16s/it]
                                                          
{'loss': 1.3946, 'grad_norm': 5.182107925415039, 'learning_rate': 1.8606946852368878e-05, 'epoch': 0.19}

 19%|█▉        | 7400/37962 [17:10:15<69:15:18,  8.16s/it]
 19%|█▉        | 7401/37962 [17:10:23<68:38:57,  8.09s/it]
 19%|█▉        | 7402/37962 [17:10:31<68:05:30,  8.02s/it]
 20%|█▉        | 7403/37962 [17:10:39<68:32:50,  8.08s/it]
 20%|█▉        | 7404/37962 [17:10:47<68:57:51,  8.12s/it]
 20%|█▉        | 7405/37962 [17:10:56<69:30:38,  8.19s/it]
 20%|█▉        | 7406/37962 [17:11:04<69:36:13,  8.20s/it]
 20%|█▉        | 7407/37962 [17:11:12<69:20:16,  8.17s/it]
 20%|█▉        | 7408/37962 [17:11:20<69:13:36,  8.16s/it]
 20%|█▉        | 7409/37962 [17:11:28<68:31:20,  8.07s/it]
 20%|█▉        | 7410/37962 [17:11:36<68:55:39,  8.12s/it]
                                                          
{'loss': 1.4579, 'grad_norm': 6.762869358062744, 'learning_rate': 1.860260009765033e-05, 'epoch': 0.2}

 20%|█▉        | 7410/37962 [17:11:36<68:55:39,  8.12s/it]
 20%|█▉        | 7411/37962 [17:11:44<69:15:45,  8.16s/it]
 20%|█▉        | 7412/37962 [17:11:53<69:35:30,  8.20s/it]
 20%|█▉        | 7413/37962 [17:12:01<69:37:48,  8.21s/it]
 20%|█▉        | 7414/37962 [17:12:09<68:51:35,  8.11s/it]
 20%|█▉        | 7415/37962 [17:12:17<68:58:25,  8.13s/it]
 20%|█▉        | 7416/37962 [17:12:25<69:13:25,  8.16s/it]
 20%|█▉        | 7417/37962 [17:12:33<68:11:48,  8.04s/it]
 20%|█▉        | 7418/37962 [17:12:41<67:55:11,  8.01s/it]
 20%|█▉        | 7419/37962 [17:12:49<68:24:05,  8.06s/it]
 20%|█▉        | 7420/37962 [17:12:57<67:41:12,  7.98s/it]
                                                          
{'loss': 1.3434, 'grad_norm': 16.47007942199707, 'learning_rate': 1.85982470812482e-05, 'epoch': 0.2}

 20%|█▉        | 7420/37962 [17:12:57<67:41:12,  7.98s/it]
 20%|█▉        | 7421/37962 [17:13:05<68:27:47,  8.07s/it]
 20%|█▉        | 7422/37962 [17:13:14<69:28:52,  8.19s/it]
 20%|█▉        | 7423/37962 [17:13:22<69:44:13,  8.22s/it]
 20%|█▉        | 7424/37962 [17:13:30<69:52:30,  8.24s/it]
 20%|█▉        | 7425/37962 [17:13:39<70:13:19,  8.28s/it]
 20%|█▉        | 7426/37962 [17:13:47<69:25:17,  8.18s/it]
 20%|█▉        | 7427/37962 [17:13:55<69:48:55,  8.23s/it]
 20%|█▉        | 7428/37962 [17:14:03<70:16:18,  8.29s/it]
 20%|█▉        | 7429/37962 [17:14:12<70:44:08,  8.34s/it]
 20%|█▉        | 7430/37962 [17:14:20<69:21:03,  8.18s/it]
                                                          
{'loss': 1.5624, 'grad_norm': 3.7040112018585205, 'learning_rate': 1.859388780633097e-05, 'epoch': 0.2}

 20%|█▉        | 7430/37962 [17:14:20<69:21:03,  8.18s/it]
 20%|█▉        | 7431/37962 [17:14:28<69:41:26,  8.22s/it]
 20%|█▉        | 7432/37962 [17:14:36<69:48:27,  8.23s/it]
 20%|█▉        | 7433/37962 [17:14:44<69:10:54,  8.16s/it]
 20%|█▉        | 7434/37962 [17:14:52<69:17:22,  8.17s/it]
 20%|█▉        | 7435/37962 [17:15:01<69:58:02,  8.25s/it]
 20%|█▉        | 7436/37962 [17:15:09<70:01:33,  8.26s/it]
 20%|█▉        | 7437/37962 [17:15:17<70:01:28,  8.26s/it]
 20%|█▉        | 7438/37962 [17:15:26<69:47:30,  8.23s/it]
 20%|█▉        | 7439/37962 [17:15:34<69:53:27,  8.24s/it]
 20%|█▉        | 7440/37962 [17:15:42<70:30:03,  8.32s/it]
                                                          
{'loss': 1.587, 'grad_norm': 10.654128074645996, 'learning_rate': 1.858952227607169e-05, 'epoch': 0.2}

 20%|█▉        | 7440/37962 [17:15:42<70:30:03,  8.32s/it]
 20%|█▉        | 7441/37962 [17:15:50<70:14:10,  8.28s/it]
 20%|█▉        | 7442/37962 [17:15:59<70:14:15,  8.28s/it]
 20%|█▉        | 7443/37962 [17:16:07<69:47:57,  8.23s/it]
 20%|█▉        | 7444/37962 [17:16:15<69:54:50,  8.25s/it]
 20%|█▉        | 7445/37962 [17:16:23<69:46:11,  8.23s/it]
 20%|█▉        | 7446/37962 [17:16:31<69:29:19,  8.20s/it]
 20%|█▉        | 7447/37962 [17:16:40<69:32:23,  8.20s/it]
 20%|█▉        | 7448/37962 [17:16:48<70:50:51,  8.36s/it]
 20%|█▉        | 7449/37962 [17:16:57<70:44:31,  8.35s/it]
 20%|█▉        | 7450/37962 [17:17:05<70:52:53,  8.36s/it]
                                                          
{'loss': 1.4183, 'grad_norm': 4.258488655090332, 'learning_rate': 1.8585150493647945e-05, 'epoch': 0.2}

 20%|█▉        | 7450/37962 [17:17:05<70:52:53,  8.36s/it]
 20%|█▉        | 7451/37962 [17:17:13<70:49:40,  8.36s/it]
 20%|█▉        | 7452/37962 [17:17:22<70:27:21,  8.31s/it]
 20%|█▉        | 7453/37962 [17:17:30<70:26:41,  8.31s/it]
 20%|█▉        | 7454/37962 [17:17:38<69:53:53,  8.25s/it]
 20%|█▉        | 7455/37962 [17:17:46<69:38:34,  8.22s/it]
 20%|█▉        | 7456/37962 [17:17:54<69:21:12,  8.18s/it]
 20%|█▉        | 7457/37962 [17:18:03<69:21:03,  8.18s/it]
 20%|█▉        | 7458/37962 [17:18:11<69:56:22,  8.25s/it]
 20%|█▉        | 7459/37962 [17:18:19<70:09:19,  8.28s/it]
 20%|█▉        | 7460/37962 [17:18:28<70:10:50,  8.28s/it]
                                                          
{'loss': 1.4875, 'grad_norm': 5.110781669616699, 'learning_rate': 1.8580772462241885e-05, 'epoch': 0.2}

 20%|█▉        | 7460/37962 [17:18:28<70:10:50,  8.28s/it]
 20%|█▉        | 7461/37962 [17:18:36<70:27:12,  8.32s/it]
 20%|█▉        | 7462/37962 [17:18:44<70:09:13,  8.28s/it]
 20%|█▉        | 7463/37962 [17:18:52<69:55:06,  8.25s/it]
 20%|█▉        | 7464/37962 [17:19:00<69:08:46,  8.16s/it]
 20%|█▉        | 7465/37962 [17:19:09<69:20:05,  8.18s/it]
 20%|█▉        | 7466/37962 [17:19:17<68:52:46,  8.13s/it]
 20%|█▉        | 7467/37962 [17:19:25<69:04:50,  8.16s/it]
 20%|█▉        | 7468/37962 [17:19:33<69:20:04,  8.19s/it]
 20%|█▉        | 7469/37962 [17:19:41<69:53:47,  8.25s/it]
 20%|█▉        | 7470/37962 [17:19:50<70:07:34,  8.28s/it]
                                                          
{'loss': 1.5466, 'grad_norm': 20.08193588256836, 'learning_rate': 1.85763881850402e-05, 'epoch': 0.2}

 20%|█▉        | 7470/37962 [17:19:50<70:07:34,  8.28s/it]
 20%|█▉        | 7471/37962 [17:19:58<69:36:22,  8.22s/it]
 20%|█▉        | 7472/37962 [17:20:06<69:38:27,  8.22s/it]
 20%|█▉        | 7473/37962 [17:20:14<69:43:06,  8.23s/it]
 20%|█▉        | 7474/37962 [17:20:23<69:45:06,  8.24s/it]
 20%|█▉        | 7475/37962 [17:20:31<69:44:40,  8.24s/it]
 20%|█▉        | 7476/37962 [17:20:39<69:34:51,  8.22s/it]
 20%|█▉        | 7477/37962 [17:20:47<69:44:13,  8.24s/it]
 20%|█▉        | 7478/37962 [17:20:56<69:54:03,  8.25s/it]
 20%|█▉        | 7479/37962 [17:21:04<69:08:46,  8.17s/it]
 20%|█▉        | 7480/37962 [17:21:12<69:32:35,  8.21s/it]
                                                          
{'loss': 1.4126, 'grad_norm': 11.538524627685547, 'learning_rate': 1.857199766523413e-05, 'epoch': 0.2}

 20%|█▉        | 7480/37962 [17:21:12<69:32:35,  8.21s/it]
 20%|█▉        | 7481/37962 [17:21:20<70:28:33,  8.32s/it]
 20%|█▉        | 7482/37962 [17:21:29<70:20:27,  8.31s/it]
 20%|█▉        | 7483/37962 [17:21:37<70:04:15,  8.28s/it]
 20%|█▉        | 7484/37962 [17:21:45<69:56:08,  8.26s/it]
 20%|█▉        | 7485/37962 [17:21:53<68:56:35,  8.14s/it]
 20%|█▉        | 7486/37962 [17:22:01<69:27:51,  8.21s/it]
 20%|█▉        | 7487/37962 [17:22:09<68:44:59,  8.12s/it]
 20%|█▉        | 7488/37962 [17:22:18<69:17:25,  8.19s/it]
 20%|█▉        | 7489/37962 [17:22:26<68:42:36,  8.12s/it]
 20%|█▉        | 7490/37962 [17:22:34<69:44:06,  8.24s/it]
                                                          
{'loss': 1.3248, 'grad_norm': 3.3468339443206787, 'learning_rate': 1.856760090601946e-05, 'epoch': 0.2}

 20%|█▉        | 7490/37962 [17:22:34<69:44:06,  8.24s/it]
 20%|█▉        | 7491/37962 [17:22:42<68:47:19,  8.13s/it]
 20%|█▉        | 7492/37962 [17:22:50<68:52:04,  8.14s/it]
 20%|█▉        | 7493/37962 [17:22:58<68:49:29,  8.13s/it]
 20%|█▉        | 7494/37962 [17:23:06<68:42:50,  8.12s/it]
 20%|█▉        | 7495/37962 [17:23:15<69:10:38,  8.17s/it]
 20%|█▉        | 7496/37962 [17:23:23<69:36:21,  8.22s/it]
 20%|█▉        | 7497/37962 [17:23:31<69:47:03,  8.25s/it]
 20%|█▉        | 7498/37962 [17:23:40<70:05:44,  8.28s/it]
 20%|█▉        | 7499/37962 [17:23:48<69:56:33,  8.27s/it]
 20%|█▉        | 7500/37962 [17:23:56<69:59:55,  8.27s/it]
                                                          
{'loss': 1.5416, 'grad_norm': 7.765769004821777, 'learning_rate': 1.8563197910596516e-05, 'epoch': 0.2}

 20%|█▉        | 7500/37962 [17:23:56<69:59:55,  8.27s/it]
 20%|█▉        | 7501/37962 [17:24:04<70:09:58,  8.29s/it]
 20%|█▉        | 7502/37962 [17:24:13<70:21:53,  8.32s/it]
 20%|█▉        | 7503/37962 [17:24:21<69:50:06,  8.25s/it]
 20%|█▉        | 7504/37962 [17:24:29<68:50:23,  8.14s/it]
 20%|█▉        | 7505/37962 [17:24:37<69:22:19,  8.20s/it]
 20%|█▉        | 7506/37962 [17:24:45<69:34:35,  8.22s/it]
 20%|█▉        | 7507/37962 [17:24:54<69:44:13,  8.24s/it]
 20%|█▉        | 7508/37962 [17:25:02<69:52:57,  8.26s/it]
 20%|█▉        | 7509/37962 [17:25:10<70:02:04,  8.28s/it]
 20%|█▉        | 7510/37962 [17:25:19<70:15:36,  8.31s/it]
                                                          
{'loss': 1.463, 'grad_norm': 11.68748950958252, 'learning_rate': 1.8558788682170157e-05, 'epoch': 0.2}

 20%|█▉        | 7510/37962 [17:25:19<70:15:36,  8.31s/it]
 20%|█▉        | 7511/37962 [17:25:27<70:34:50,  8.34s/it]
 20%|█▉        | 7512/37962 [17:25:35<70:16:27,  8.31s/it]
 20%|█▉        | 7513/37962 [17:25:44<69:44:13,  8.25s/it]
 20%|█▉        | 7514/37962 [17:25:52<69:42:39,  8.24s/it]
 20%|█▉        | 7515/37962 [17:26:00<69:37:13,  8.23s/it]
 20%|█▉        | 7516/37962 [17:26:08<69:45:07,  8.25s/it]
 20%|█▉        | 7517/37962 [17:26:16<69:32:51,  8.22s/it]
 20%|█▉        | 7518/37962 [17:26:25<69:47:34,  8.25s/it]
 20%|█▉        | 7519/37962 [17:26:33<70:11:15,  8.30s/it]
 20%|█▉        | 7520/37962 [17:26:41<70:07:25,  8.29s/it]
                                                          
{'loss': 1.3357, 'grad_norm': 6.1479010581970215, 'learning_rate': 1.855437322394979e-05, 'epoch': 0.2}

 20%|█▉        | 7520/37962 [17:26:41<70:07:25,  8.29s/it]
 20%|█▉        | 7521/37962 [17:26:49<69:11:14,  8.18s/it]
 20%|█▉        | 7522/37962 [17:26:58<69:22:00,  8.20s/it]
 20%|█▉        | 7523/37962 [17:27:06<70:05:44,  8.29s/it]
 20%|█▉        | 7524/37962 [17:27:14<70:04:37,  8.29s/it]
 20%|█▉        | 7525/37962 [17:27:23<69:51:14,  8.26s/it]
 20%|█▉        | 7526/37962 [17:27:31<69:03:40,  8.17s/it]
 20%|█▉        | 7527/37962 [17:27:39<69:30:00,  8.22s/it]
 20%|█▉        | 7528/37962 [17:27:47<68:12:55,  8.07s/it]
 20%|█▉        | 7529/37962 [17:27:55<68:53:44,  8.15s/it]
 20%|█▉        | 7530/37962 [17:28:03<69:03:41,  8.17s/it]
                                                          
{'loss': 1.4246, 'grad_norm': 4.37684440612793, 'learning_rate': 1.854995153914935e-05, 'epoch': 0.2}

 20%|█▉        | 7530/37962 [17:28:03<69:03:41,  8.17s/it]
 20%|█▉        | 7531/37962 [17:28:11<69:04:19,  8.17s/it]
 20%|█▉        | 7532/37962 [17:28:20<69:48:21,  8.26s/it]
 20%|█▉        | 7533/37962 [17:28:28<69:08:58,  8.18s/it]
 20%|█▉        | 7534/37962 [17:28:36<69:25:57,  8.21s/it]
 20%|█▉        | 7535/37962 [17:28:44<69:24:12,  8.21s/it]
 20%|█▉        | 7536/37962 [17:28:53<69:36:08,  8.24s/it]
 20%|█▉        | 7537/37962 [17:29:01<70:08:26,  8.30s/it]
 20%|█▉        | 7538/37962 [17:29:09<70:11:10,  8.30s/it]
 20%|█▉        | 7539/37962 [17:29:17<69:13:18,  8.19s/it]
 20%|█▉        | 7540/37962 [17:29:25<68:28:09,  8.10s/it]
                                                          
{'loss': 1.4105, 'grad_norm': 8.319536209106445, 'learning_rate': 1.8545523630987297e-05, 'epoch': 0.2}

 20%|█▉        | 7540/37962 [17:29:25<68:28:09,  8.10s/it]
 20%|█▉        | 7541/37962 [17:29:33<68:58:46,  8.16s/it]
 20%|█▉        | 7542/37962 [17:29:42<69:15:25,  8.20s/it]
 20%|█▉        | 7543/37962 [17:29:50<69:16:08,  8.20s/it]
 20%|█▉        | 7544/37962 [17:29:58<69:37:19,  8.24s/it]
 20%|█▉        | 7545/37962 [17:30:07<69:57:41,  8.28s/it]
 20%|█▉        | 7546/37962 [17:30:15<70:01:37,  8.29s/it]
 20%|█▉        | 7547/37962 [17:30:23<70:13:04,  8.31s/it]
 20%|█▉        | 7548/37962 [17:30:32<70:27:16,  8.34s/it]
 20%|█▉        | 7549/37962 [17:30:40<70:09:51,  8.31s/it]
 20%|█▉        | 7550/37962 [17:30:48<69:32:51,  8.23s/it]
                                                          
{'loss': 1.5232, 'grad_norm': 12.579939842224121, 'learning_rate': 1.8541089502686635e-05, 'epoch': 0.2}

 20%|█▉        | 7550/37962 [17:30:48<69:32:51,  8.23s/it]
 20%|█▉        | 7551/37962 [17:30:56<69:50:51,  8.27s/it]
 20%|█▉        | 7552/37962 [17:31:05<70:08:01,  8.30s/it]
 20%|█▉        | 7553/37962 [17:31:13<69:59:57,  8.29s/it]
 20%|█▉        | 7554/37962 [17:31:21<70:04:49,  8.30s/it]
 20%|█▉        | 7555/37962 [17:31:29<69:49:51,  8.27s/it]
 20%|█▉        | 7556/37962 [17:31:38<70:23:29,  8.33s/it]
 20%|█▉        | 7557/37962 [17:31:46<70:02:15,  8.29s/it]
 20%|█▉        | 7558/37962 [17:31:54<68:47:58,  8.15s/it]
 20%|█▉        | 7559/37962 [17:32:02<68:58:08,  8.17s/it]
 20%|█▉        | 7560/37962 [17:32:10<68:58:53,  8.17s/it]
                                                          
{'loss': 1.502, 'grad_norm': 6.631847858428955, 'learning_rate': 1.853664915747489e-05, 'epoch': 0.2}

 20%|█▉        | 7560/37962 [17:32:10<68:58:53,  8.17s/it]
 20%|█▉        | 7561/37962 [17:32:19<69:08:32,  8.19s/it]
 20%|█▉        | 7562/37962 [17:32:27<69:34:11,  8.24s/it]
 20%|█▉        | 7563/37962 [17:32:35<69:51:53,  8.27s/it]
 20%|█▉        | 7564/37962 [17:32:43<69:05:47,  8.18s/it]
 20%|█▉        | 7565/37962 [17:32:51<68:22:59,  8.10s/it]
 20%|█▉        | 7566/37962 [17:32:59<68:13:48,  8.08s/it]
 20%|█▉        | 7567/37962 [17:33:08<68:56:31,  8.17s/it]
 20%|█▉        | 7568/37962 [17:33:16<69:48:01,  8.27s/it]
 20%|█▉        | 7569/37962 [17:33:24<69:47:31,  8.27s/it]
 20%|█▉        | 7570/37962 [17:33:32<69:04:39,  8.18s/it]
                                                          
{'loss': 1.5116, 'grad_norm': 4.5043182373046875, 'learning_rate': 1.8532202598584107e-05, 'epoch': 0.2}

 20%|█▉        | 7570/37962 [17:33:32<69:04:39,  8.18s/it]
 20%|█▉        | 7571/37962 [17:33:41<69:44:15,  8.26s/it]
 20%|█▉        | 7572/37962 [17:33:49<69:36:11,  8.25s/it]
 20%|█▉        | 7573/37962 [17:33:57<70:04:12,  8.30s/it]
 20%|█▉        | 7574/37962 [17:34:05<69:27:44,  8.23s/it]
 20%|█▉        | 7575/37962 [17:34:14<70:13:40,  8.32s/it]
 20%|█▉        | 7576/37962 [17:34:22<69:58:19,  8.29s/it]
 20%|█▉        | 7577/37962 [17:34:31<70:23:24,  8.34s/it]
 20%|█▉        | 7578/37962 [17:34:39<70:30:39,  8.35s/it]
 20%|█▉        | 7579/37962 [17:34:47<70:14:16,  8.32s/it]
 20%|█▉        | 7580/37962 [17:34:56<70:11:10,  8.32s/it]
                                                          
{'loss': 1.4139, 'grad_norm': 6.453707695007324, 'learning_rate': 1.852774982925087e-05, 'epoch': 0.2}

 20%|█▉        | 7580/37962 [17:34:56<70:11:10,  8.32s/it]
 20%|█▉        | 7581/37962 [17:35:04<69:30:57,  8.24s/it]
 20%|█▉        | 7582/37962 [17:35:12<69:55:41,  8.29s/it]
 20%|█▉        | 7583/37962 [17:35:20<70:05:08,  8.31s/it]
 20%|█▉        | 7584/37962 [17:35:29<70:00:48,  8.30s/it]
 20%|█▉        | 7585/37962 [17:35:37<69:58:42,  8.29s/it]
 20%|█▉        | 7586/37962 [17:35:45<69:31:48,  8.24s/it]
 20%|█▉        | 7587/37962 [17:35:53<69:24:09,  8.23s/it]
 20%|█▉        | 7588/37962 [17:36:02<69:47:41,  8.27s/it]
 20%|█▉        | 7589/37962 [17:36:10<69:43:08,  8.26s/it]
 20%|█▉        | 7590/37962 [17:36:18<68:58:55,  8.18s/it]
                                                          
{'loss': 1.414, 'grad_norm': 7.739419460296631, 'learning_rate': 1.852329085271626e-05, 'epoch': 0.2}

 20%|█▉        | 7590/37962 [17:36:18<68:58:55,  8.18s/it]
 20%|█▉        | 7591/37962 [17:36:26<69:04:36,  8.19s/it]
 20%|█▉        | 7592/37962 [17:36:34<69:30:46,  8.24s/it]
 20%|██        | 7593/37962 [17:36:43<69:27:07,  8.23s/it]
 20%|██        | 7594/37962 [17:36:51<69:21:40,  8.22s/it]
 20%|██        | 7595/37962 [17:36:59<69:29:43,  8.24s/it]
 20%|██        | 7596/37962 [17:37:08<69:52:24,  8.28s/it]
 20%|██        | 7597/37962 [17:37:16<69:49:52,  8.28s/it]
 20%|██        | 7598/37962 [17:37:24<69:52:28,  8.28s/it]
 20%|██        | 7599/37962 [17:37:33<70:23:08,  8.35s/it]
 20%|██        | 7600/37962 [17:37:41<70:18:18,  8.34s/it]
                                                          
{'loss': 1.3792, 'grad_norm': 3.2093746662139893, 'learning_rate': 1.8518825672225894e-05, 'epoch': 0.2}

 20%|██        | 7600/37962 [17:37:41<70:18:18,  8.34s/it]
 20%|██        | 7601/37962 [17:37:49<70:20:59,  8.34s/it]
 20%|██        | 7602/37962 [17:37:58<70:04:00,  8.31s/it]
 20%|██        | 7603/37962 [17:38:06<70:19:51,  8.34s/it]
 20%|██        | 7604/37962 [17:38:14<69:37:48,  8.26s/it]
 20%|██        | 7605/37962 [17:38:22<69:45:50,  8.27s/it]
 20%|██        | 7606/37962 [17:38:31<69:40:51,  8.26s/it]
 20%|██        | 7607/37962 [17:38:39<69:44:49,  8.27s/it]
 20%|██        | 7608/37962 [17:38:47<69:47:00,  8.28s/it]
 20%|██        | 7609/37962 [17:38:55<69:52:51,  8.29s/it]
 20%|██        | 7610/37962 [17:39:04<69:56:23,  8.30s/it]
                                                          
{'loss': 1.5945, 'grad_norm': 4.608486652374268, 'learning_rate': 1.85143542910299e-05, 'epoch': 0.2}

 20%|██        | 7610/37962 [17:39:04<69:56:23,  8.30s/it]
 20%|██        | 7611/37962 [17:39:12<69:53:38,  8.29s/it]
 20%|██        | 7612/37962 [17:39:20<68:47:43,  8.16s/it]
 20%|██        | 7613/37962 [17:39:28<69:06:30,  8.20s/it]
 20%|██        | 7614/37962 [17:39:37<69:46:06,  8.28s/it]
 20%|██        | 7615/37962 [17:39:45<69:44:15,  8.27s/it]
 20%|██        | 7616/37962 [17:39:53<69:30:02,  8.25s/it]
 20%|██        | 7617/37962 [17:40:01<69:43:12,  8.27s/it]
 20%|██        | 7618/37962 [17:40:10<69:27:02,  8.24s/it]
 20%|██        | 7619/37962 [17:40:18<69:11:28,  8.21s/it]
 20%|██        | 7620/37962 [17:40:26<69:18:33,  8.22s/it]
                                                          
{'loss': 1.4966, 'grad_norm': 20.764062881469727, 'learning_rate': 1.8509876712382923e-05, 'epoch': 0.2}

 20%|██        | 7620/37962 [17:40:26<69:18:33,  8.22s/it]
 20%|██        | 7621/37962 [17:40:34<69:54:14,  8.29s/it]
 20%|██        | 7622/37962 [17:40:43<69:43:54,  8.27s/it]
 20%|██        | 7623/37962 [17:40:51<69:28:50,  8.24s/it]
 20%|██        | 7624/37962 [17:40:59<69:19:33,  8.23s/it]
 20%|██        | 7625/37962 [17:41:07<69:25:24,  8.24s/it]
 20%|██        | 7626/37962 [17:41:16<69:34:38,  8.26s/it]
 20%|██        | 7627/37962 [17:41:24<69:50:32,  8.29s/it]
 20%|██        | 7628/37962 [17:41:32<69:12:59,  8.21s/it]
 20%|██        | 7629/37962 [17:41:40<69:12:25,  8.21s/it]
 20%|██        | 7630/37962 [17:41:49<69:29:23,  8.25s/it]
                                                          
{'loss': 1.3065, 'grad_norm': 28.894126892089844, 'learning_rate': 1.8505392939544106e-05, 'epoch': 0.2}

 20%|██        | 7630/37962 [17:41:49<69:29:23,  8.25s/it]
 20%|██        | 7631/37962 [17:41:57<69:18:41,  8.23s/it]
 20%|██        | 7632/37962 [17:42:05<69:56:31,  8.30s/it]
 20%|██        | 7633/37962 [17:42:14<70:04:26,  8.32s/it]
 20%|██        | 7634/37962 [17:42:22<71:01:32,  8.43s/it]
 20%|██        | 7635/37962 [17:42:30<70:29:24,  8.37s/it]
 20%|██        | 7636/37962 [17:42:39<70:39:19,  8.39s/it]
 20%|██        | 7637/37962 [17:42:47<70:16:58,  8.34s/it]
 20%|██        | 7638/37962 [17:42:55<70:04:53,  8.32s/it]
 20%|██        | 7639/37962 [17:43:04<71:08:24,  8.45s/it]
 20%|██        | 7640/37962 [17:43:12<70:27:26,  8.37s/it]
                                                          
{'loss': 1.3922, 'grad_norm': 8.782061576843262, 'learning_rate': 1.8500902975777113e-05, 'epoch': 0.2}

 20%|██        | 7640/37962 [17:43:12<70:27:26,  8.37s/it]
 20%|██        | 7641/37962 [17:43:21<70:11:35,  8.33s/it]
 20%|██        | 7642/37962 [17:43:29<70:18:00,  8.35s/it]
 20%|██        | 7643/37962 [17:43:37<70:11:12,  8.33s/it]
 20%|██        | 7644/37962 [17:43:46<70:08:02,  8.33s/it]
 20%|██        | 7645/37962 [17:43:54<69:48:48,  8.29s/it]
 20%|██        | 7646/37962 [17:44:02<69:39:03,  8.27s/it]
 20%|██        | 7647/37962 [17:44:10<70:00:00,  8.31s/it]
 20%|██        | 7648/37962 [17:44:19<70:02:39,  8.32s/it]
 20%|██        | 7649/37962 [17:44:27<69:16:31,  8.23s/it]
 20%|██        | 7650/37962 [17:44:35<69:36:05,  8.27s/it]
                                                          
{'loss': 1.3658, 'grad_norm': 10.291102409362793, 'learning_rate': 1.8496406824350118e-05, 'epoch': 0.2}

 20%|██        | 7650/37962 [17:44:35<69:36:05,  8.27s/it]
 20%|██        | 7651/37962 [17:44:43<69:41:56,  8.28s/it]
 20%|██        | 7652/37962 [17:44:52<69:47:52,  8.29s/it]
 20%|██        | 7653/37962 [17:45:00<68:52:56,  8.18s/it]
 20%|██        | 7654/37962 [17:45:08<69:08:04,  8.21s/it]
 20%|██        | 7655/37962 [17:45:16<69:14:58,  8.23s/it]
 20%|██        | 7656/37962 [17:45:24<68:32:55,  8.14s/it]
 20%|██        | 7657/37962 [17:45:32<68:59:12,  8.20s/it]
 20%|██        | 7658/37962 [17:45:41<69:23:12,  8.24s/it]
 20%|██        | 7659/37962 [17:45:49<69:22:21,  8.24s/it]
 20%|██        | 7660/37962 [17:45:57<69:02:57,  8.20s/it]
                                                          
{'loss': 1.5259, 'grad_norm': 9.185393333435059, 'learning_rate': 1.8491904488535782e-05, 'epoch': 0.2}

 20%|██        | 7660/37962 [17:45:57<69:02:57,  8.20s/it]
 20%|██        | 7661/37962 [17:46:05<68:32:22,  8.14s/it]
 20%|██        | 7662/37962 [17:46:13<68:41:33,  8.16s/it]
 20%|██        | 7663/37962 [17:46:22<68:40:56,  8.16s/it]
 20%|██        | 7664/37962 [17:46:30<68:59:14,  8.20s/it]
 20%|██        | 7665/37962 [17:46:38<69:32:23,  8.26s/it]
 20%|██        | 7666/37962 [17:46:47<69:44:46,  8.29s/it]
 20%|██        | 7667/37962 [17:46:55<69:59:30,  8.32s/it]
 20%|██        | 7668/37962 [17:47:03<69:51:14,  8.30s/it]
 20%|██        | 7669/37962 [17:47:12<69:53:21,  8.31s/it]
 20%|██        | 7670/37962 [17:47:20<69:30:04,  8.26s/it]
                                                          
{'loss': 1.3788, 'grad_norm': 6.680044174194336, 'learning_rate': 1.848739597161128e-05, 'epoch': 0.2}

 20%|██        | 7670/37962 [17:47:20<69:30:04,  8.26s/it]
 20%|██        | 7671/37962 [17:47:28<69:24:49,  8.25s/it]
 20%|██        | 7672/37962 [17:47:36<70:02:08,  8.32s/it]
 20%|██        | 7673/37962 [17:47:45<69:39:17,  8.28s/it]
 20%|██        | 7674/37962 [17:47:53<68:52:11,  8.19s/it]
 20%|██        | 7675/37962 [17:48:00<68:11:00,  8.10s/it]
 20%|██        | 7676/37962 [17:48:09<69:19:25,  8.24s/it]
 20%|██        | 7677/37962 [17:48:17<68:37:42,  8.16s/it]
 20%|██        | 7678/37962 [17:48:25<68:47:37,  8.18s/it]
 20%|██        | 7679/37962 [17:48:34<69:54:44,  8.31s/it]
 20%|██        | 7680/37962 [17:48:42<69:35:33,  8.27s/it]
                                                          
{'loss': 1.3793, 'grad_norm': 6.952576637268066, 'learning_rate': 1.8482881276858286e-05, 'epoch': 0.2}

 20%|██        | 7680/37962 [17:48:42<69:35:33,  8.27s/it]
 20%|██        | 7681/37962 [17:48:50<69:27:50,  8.26s/it]
 20%|██        | 7682/37962 [17:48:59<69:27:51,  8.26s/it]
 20%|██        | 7683/37962 [17:49:07<69:15:37,  8.23s/it]
 20%|██        | 7684/37962 [17:49:15<69:13:12,  8.23s/it]
 20%|██        | 7685/37962 [17:49:23<69:14:28,  8.23s/it]
 20%|██        | 7686/37962 [17:49:31<68:49:59,  8.18s/it]
 20%|██        | 7687/37962 [17:49:39<68:58:38,  8.20s/it]
 20%|██        | 7688/37962 [17:49:48<68:58:37,  8.20s/it]
 20%|██        | 7689/37962 [17:49:56<69:10:57,  8.23s/it]
 20%|██        | 7690/37962 [17:50:04<69:08:39,  8.22s/it]
                                                          
{'loss': 1.4661, 'grad_norm': 11.261947631835938, 'learning_rate': 1.8478360407562964e-05, 'epoch': 0.2}

 20%|██        | 7690/37962 [17:50:04<69:08:39,  8.22s/it]
 20%|██        | 7691/37962 [17:50:12<68:59:59,  8.21s/it]
 20%|██        | 7692/37962 [17:50:21<69:19:11,  8.24s/it]
 20%|██        | 7693/37962 [17:50:29<69:50:37,  8.31s/it]
 20%|██        | 7694/37962 [17:50:37<69:33:50,  8.27s/it]
 20%|██        | 7695/37962 [17:50:46<69:28:14,  8.26s/it]
 20%|██        | 7696/37962 [17:50:54<70:26:37,  8.38s/it]
 20%|██        | 7697/37962 [17:51:02<70:09:42,  8.35s/it]
 20%|██        | 7698/37962 [17:51:11<69:48:08,  8.30s/it]
 20%|██        | 7699/37962 [17:51:19<69:39:29,  8.29s/it]
 20%|██        | 7700/37962 [17:51:27<69:30:01,  8.27s/it]
                                                          
{'loss': 1.4252, 'grad_norm': 21.51414680480957, 'learning_rate': 1.847383336701598e-05, 'epoch': 0.2}

 20%|██        | 7700/37962 [17:51:27<69:30:01,  8.27s/it]
 20%|██        | 7701/37962 [17:51:35<69:23:27,  8.26s/it]
 20%|██        | 7702/37962 [17:51:44<69:37:14,  8.28s/it]
 20%|██        | 7703/37962 [17:51:52<69:25:54,  8.26s/it]
 20%|██        | 7704/37962 [17:52:00<69:27:12,  8.26s/it]
 20%|██        | 7705/37962 [17:52:08<69:01:10,  8.21s/it]
 20%|██        | 7706/37962 [17:52:17<69:43:26,  8.30s/it]
 20%|██        | 7707/37962 [17:52:25<69:33:33,  8.28s/it]
 20%|██        | 7708/37962 [17:52:33<69:38:25,  8.29s/it]
 20%|██        | 7709/37962 [17:52:42<69:39:12,  8.29s/it]
 20%|██        | 7710/37962 [17:52:50<69:29:29,  8.27s/it]
                                                          
{'loss': 1.4936, 'grad_norm': 23.868408203125, 'learning_rate': 1.8469300158512488e-05, 'epoch': 0.2}

 20%|██        | 7710/37962 [17:52:50<69:29:29,  8.27s/it]
 20%|██        | 7711/37962 [17:52:58<68:36:41,  8.17s/it]
 20%|██        | 7712/37962 [17:53:06<69:39:30,  8.29s/it]
 20%|██        | 7713/37962 [17:53:15<69:49:37,  8.31s/it]
 20%|██        | 7714/37962 [17:53:23<70:07:53,  8.35s/it]
 20%|██        | 7715/37962 [17:53:31<70:03:05,  8.34s/it]
 20%|██        | 7716/37962 [17:53:40<69:56:09,  8.32s/it]
 20%|██        | 7717/37962 [17:53:48<69:04:26,  8.22s/it]
 20%|██        | 7718/37962 [17:53:56<69:04:17,  8.22s/it]
 20%|██        | 7719/37962 [17:54:04<69:11:08,  8.24s/it]
 20%|██        | 7720/37962 [17:54:13<69:19:26,  8.25s/it]
                                                          
{'loss': 1.443, 'grad_norm': 26.173173904418945, 'learning_rate': 1.846476078535213e-05, 'epoch': 0.2}

 20%|██        | 7720/37962 [17:54:13<69:19:26,  8.25s/it]
 20%|██        | 7721/37962 [17:54:21<68:44:22,  8.18s/it]
 20%|██        | 7722/37962 [17:54:29<68:50:37,  8.20s/it]
 20%|██        | 7723/37962 [17:54:37<69:07:41,  8.23s/it]
 20%|██        | 7724/37962 [17:54:45<68:27:59,  8.15s/it]
 20%|██        | 7725/37962 [17:54:54<69:26:15,  8.27s/it]
 20%|██        | 7726/37962 [17:55:02<69:02:25,  8.22s/it]
 20%|██        | 7727/37962 [17:55:10<69:13:49,  8.24s/it]
 20%|██        | 7728/37962 [17:55:18<69:19:14,  8.25s/it]
 20%|██        | 7729/37962 [17:55:27<69:45:37,  8.31s/it]
 20%|██        | 7730/37962 [17:55:35<69:41:38,  8.30s/it]
                                                          
{'loss': 1.5022, 'grad_norm': 4.012290954589844, 'learning_rate': 1.8460215250839044e-05, 'epoch': 0.2}

 20%|██        | 7730/37962 [17:55:35<69:41:38,  8.30s/it]
 20%|██        | 7731/37962 [17:55:43<69:34:14,  8.28s/it]
 20%|██        | 7732/37962 [17:55:52<70:36:26,  8.41s/it]
 20%|██        | 7733/37962 [17:56:00<70:18:02,  8.37s/it]
 20%|██        | 7734/37962 [17:56:08<70:02:52,  8.34s/it]
 20%|██        | 7735/37962 [17:56:17<69:42:33,  8.30s/it]
 20%|██        | 7736/37962 [17:56:25<69:26:27,  8.27s/it]
 20%|██        | 7737/37962 [17:56:33<69:33:46,  8.29s/it]
 20%|██        | 7738/37962 [17:56:41<69:30:39,  8.28s/it]
 20%|██        | 7739/37962 [17:56:50<69:13:15,  8.25s/it]
 20%|██        | 7740/37962 [17:56:58<69:32:28,  8.28s/it]
                                                          
{'loss': 1.5015, 'grad_norm': 7.1301774978637695, 'learning_rate': 1.8455663558281834e-05, 'epoch': 0.2}

 20%|██        | 7740/37962 [17:56:58<69:32:28,  8.28s/it]
 20%|██        | 7741/37962 [17:57:06<69:02:55,  8.23s/it]
 20%|██        | 7742/37962 [17:57:14<68:49:32,  8.20s/it]
 20%|██        | 7743/37962 [17:57:22<68:37:35,  8.18s/it]
 20%|██        | 7744/37962 [17:57:30<68:18:45,  8.14s/it]
 20%|██        | 7745/37962 [17:57:38<68:04:04,  8.11s/it]
 20%|██        | 7746/37962 [17:57:47<68:24:41,  8.15s/it]
 20%|██        | 7747/37962 [17:57:55<68:20:40,  8.14s/it]
 20%|██        | 7748/37962 [17:58:03<69:03:57,  8.23s/it]
 20%|██        | 7749/37962 [17:58:11<68:58:45,  8.22s/it]
 20%|██        | 7750/37962 [17:58:20<69:12:01,  8.25s/it]
                                                          
{'loss': 1.4628, 'grad_norm': 23.813858032226562, 'learning_rate': 1.8451105710993605e-05, 'epoch': 0.2}

 20%|██        | 7750/37962 [17:58:20<69:12:01,  8.25s/it]
 20%|██        | 7751/37962 [17:58:28<69:11:19,  8.24s/it]
 20%|██        | 7752/37962 [17:58:36<68:37:35,  8.18s/it]
 20%|██        | 7753/37962 [17:58:44<68:39:09,  8.18s/it]
 20%|██        | 7754/37962 [17:58:53<68:56:57,  8.22s/it]
 20%|██        | 7755/37962 [17:59:01<68:49:21,  8.20s/it]
 20%|██        | 7756/37962 [17:59:09<68:42:07,  8.19s/it]
 20%|██        | 7757/37962 [17:59:17<68:44:45,  8.19s/it]
 20%|██        | 7758/37962 [17:59:25<68:49:21,  8.20s/it]
 20%|██        | 7759/37962 [17:59:34<69:46:28,  8.32s/it]
 20%|██        | 7760/37962 [17:59:42<69:28:48,  8.28s/it]
                                                          
{'loss': 1.4534, 'grad_norm': 10.864272117614746, 'learning_rate': 1.8446541712291936e-05, 'epoch': 0.2}

 20%|██        | 7760/37962 [17:59:42<69:28:48,  8.28s/it]
 20%|██        | 7761/37962 [17:59:50<69:35:37,  8.30s/it]
 20%|██        | 7762/37962 [17:59:59<69:33:18,  8.29s/it]
 20%|██        | 7763/37962 [18:00:07<70:03:45,  8.35s/it]
 20%|██        | 7764/37962 [18:00:15<69:15:00,  8.26s/it]
 20%|██        | 7765/37962 [18:00:24<69:38:57,  8.30s/it]
 20%|██        | 7766/37962 [18:00:32<69:23:00,  8.27s/it]
 20%|██        | 7767/37962 [18:00:40<69:11:11,  8.25s/it]
 20%|██        | 7768/37962 [18:00:48<68:15:13,  8.14s/it]
 20%|██        | 7769/37962 [18:00:56<68:23:24,  8.15s/it]
 20%|██        | 7770/37962 [18:01:04<68:29:00,  8.17s/it]
                                                          
{'loss': 1.4285, 'grad_norm': 8.994521141052246, 'learning_rate': 1.844197156549888e-05, 'epoch': 0.2}

 20%|██        | 7770/37962 [18:01:04<68:29:00,  8.17s/it]
 20%|██        | 7771/37962 [18:01:13<68:46:49,  8.20s/it]
 20%|██        | 7772/37962 [18:01:20<68:07:50,  8.12s/it]
 20%|██        | 7773/37962 [18:01:29<68:23:34,  8.16s/it]
 20%|██        | 7774/37962 [18:01:37<68:46:44,  8.20s/it]
 20%|██        | 7775/37962 [18:01:45<68:51:29,  8.21s/it]
 20%|██        | 7776/37962 [18:01:53<68:49:08,  8.21s/it]
 20%|██        | 7777/37962 [18:02:02<68:58:22,  8.23s/it]
 20%|██        | 7778/37962 [18:02:10<68:53:23,  8.22s/it]
 20%|██        | 7779/37962 [18:02:18<68:54:21,  8.22s/it]
 20%|██        | 7780/37962 [18:02:27<69:26:55,  8.28s/it]
                                                          
{'loss': 1.4768, 'grad_norm': 7.040415287017822, 'learning_rate': 1.843739527394097e-05, 'epoch': 0.2}

 20%|██        | 7780/37962 [18:02:27<69:26:55,  8.28s/it]
 20%|██        | 7781/37962 [18:02:35<69:11:00,  8.25s/it]
 20%|██        | 7782/37962 [18:02:43<69:19:53,  8.27s/it]
 21%|██        | 7783/37962 [18:02:51<69:13:08,  8.26s/it]
 21%|██        | 7784/37962 [18:02:59<68:54:28,  8.22s/it]
 21%|██        | 7785/37962 [18:03:07<68:23:28,  8.16s/it]
 21%|██        | 7786/37962 [18:03:16<68:49:40,  8.21s/it]
 21%|██        | 7787/37962 [18:03:24<68:42:29,  8.20s/it]
 21%|██        | 7788/37962 [18:03:32<67:51:10,  8.10s/it]
 21%|██        | 7789/37962 [18:03:40<67:43:53,  8.08s/it]
 21%|██        | 7790/37962 [18:03:48<67:44:00,  8.08s/it]
                                                          
{'loss': 1.4001, 'grad_norm': 11.234932899475098, 'learning_rate': 1.8432812840949205e-05, 'epoch': 0.21}

 21%|██        | 7790/37962 [18:03:48<67:44:00,  8.08s/it]
 21%|██        | 7791/37962 [18:03:56<68:19:32,  8.15s/it]
 21%|██        | 7792/37962 [18:04:04<68:32:14,  8.18s/it]
 21%|██        | 7793/37962 [18:04:13<68:36:35,  8.19s/it]
 21%|██        | 7794/37962 [18:04:21<68:48:07,  8.21s/it]
 21%|██        | 7795/37962 [18:04:29<68:52:46,  8.22s/it]
 21%|██        | 7796/37962 [18:04:37<68:21:38,  8.16s/it]
 21%|██        | 7797/37962 [18:04:45<68:34:31,  8.18s/it]
 21%|██        | 7798/37962 [18:04:54<68:37:02,  8.19s/it]
 21%|██        | 7799/37962 [18:05:02<68:52:36,  8.22s/it]
 21%|██        | 7800/37962 [18:05:10<68:50:58,  8.22s/it]
                                                          
{'loss': 1.4512, 'grad_norm': 22.936113357543945, 'learning_rate': 1.8428224269859062e-05, 'epoch': 0.21}

 21%|██        | 7800/37962 [18:05:10<68:50:58,  8.22s/it]
 21%|██        | 7801/37962 [18:05:18<68:32:04,  8.18s/it]
 21%|██        | 7802/37962 [18:05:27<68:44:25,  8.21s/it]
 21%|██        | 7803/37962 [18:05:35<69:08:17,  8.25s/it]
 21%|██        | 7804/37962 [18:05:43<69:04:09,  8.24s/it]
 21%|██        | 7805/37962 [18:05:51<69:05:09,  8.25s/it]
 21%|██        | 7806/37962 [18:05:59<67:56:20,  8.11s/it]
 21%|██        | 7807/37962 [18:06:07<68:21:24,  8.16s/it]
 21%|██        | 7808/37962 [18:06:16<68:25:33,  8.17s/it]
 21%|██        | 7809/37962 [18:06:24<67:43:48,  8.09s/it]
 21%|██        | 7810/37962 [18:06:32<68:00:01,  8.12s/it]
                                                          
{'loss': 1.4184, 'grad_norm': 5.005373477935791, 'learning_rate': 1.8423629564010485e-05, 'epoch': 0.21}

 21%|██        | 7810/37962 [18:06:32<68:00:01,  8.12s/it]
 21%|██        | 7811/37962 [18:06:40<68:28:01,  8.17s/it]
 21%|██        | 7812/37962 [18:06:48<69:02:27,  8.24s/it]
 21%|██        | 7813/37962 [18:06:57<69:15:21,  8.27s/it]
 21%|██        | 7814/37962 [18:07:05<69:12:18,  8.26s/it]
 21%|██        | 7815/37962 [18:07:13<68:15:22,  8.15s/it]
 21%|██        | 7816/37962 [18:07:21<68:40:43,  8.20s/it]
 21%|██        | 7817/37962 [18:07:29<67:54:33,  8.11s/it]
 21%|██        | 7818/37962 [18:07:37<68:19:31,  8.16s/it]
 21%|██        | 7819/37962 [18:07:46<68:37:26,  8.20s/it]
 21%|██        | 7820/37962 [18:07:54<69:02:17,  8.25s/it]
                                                          
{'loss': 1.3978, 'grad_norm': 23.4188232421875, 'learning_rate': 1.8419028726747873e-05, 'epoch': 0.21}

 21%|██        | 7820/37962 [18:07:54<69:02:17,  8.25s/it]
 21%|██        | 7821/37962 [18:08:02<69:00:50,  8.24s/it]
 21%|██        | 7822/37962 [18:08:11<69:35:49,  8.31s/it]
 21%|██        | 7823/37962 [18:08:19<69:25:16,  8.29s/it]
 21%|██        | 7824/37962 [18:08:27<69:20:47,  8.28s/it]
 21%|██        | 7825/37962 [18:08:36<69:19:20,  8.28s/it]
 21%|██        | 7826/37962 [18:08:44<69:02:10,  8.25s/it]
 21%|██        | 7827/37962 [18:08:52<68:20:41,  8.16s/it]
 21%|██        | 7828/37962 [18:09:00<67:51:51,  8.11s/it]
 21%|██        | 7829/37962 [18:09:08<68:57:59,  8.24s/it]
 21%|██        | 7830/37962 [18:09:16<69:04:34,  8.25s/it]
                                                          
{'loss': 1.5123, 'grad_norm': 14.410547256469727, 'learning_rate': 1.8414421761420104e-05, 'epoch': 0.21}

 21%|██        | 7830/37962 [18:09:16<69:04:34,  8.25s/it]
 21%|██        | 7831/37962 [18:09:24<68:17:12,  8.16s/it]
 21%|██        | 7832/37962 [18:09:33<68:51:42,  8.23s/it]
 21%|██        | 7833/37962 [18:09:41<68:57:35,  8.24s/it]
 21%|██        | 7834/37962 [18:09:49<68:58:26,  8.24s/it]
 21%|██        | 7835/37962 [18:09:57<68:50:07,  8.23s/it]
 21%|██        | 7836/37962 [18:10:06<69:06:48,  8.26s/it]
 21%|██        | 7837/37962 [18:10:14<69:09:26,  8.26s/it]
 21%|██        | 7838/37962 [18:10:22<69:12:16,  8.27s/it]
 21%|██        | 7839/37962 [18:10:31<69:15:55,  8.28s/it]
 21%|██        | 7840/37962 [18:10:39<68:58:18,  8.24s/it]
                                                          
{'loss': 1.3974, 'grad_norm': 25.799779891967773, 'learning_rate': 1.8409808671380506e-05, 'epoch': 0.21}

 21%|██        | 7840/37962 [18:10:39<68:58:18,  8.24s/it]
 21%|██        | 7841/37962 [18:10:47<69:16:24,  8.28s/it]
 21%|██        | 7842/37962 [18:10:56<69:25:53,  8.30s/it]
 21%|██        | 7843/37962 [18:11:04<69:43:24,  8.33s/it]
 21%|██        | 7844/37962 [18:11:12<69:43:22,  8.33s/it]
 21%|██        | 7845/37962 [18:11:21<69:55:20,  8.36s/it]
 21%|██        | 7846/37962 [18:11:29<69:45:37,  8.34s/it]
 21%|██        | 7847/37962 [18:11:37<69:56:02,  8.36s/it]
 21%|██        | 7848/37962 [18:11:46<69:53:18,  8.35s/it]
 21%|██        | 7849/37962 [18:11:54<68:46:34,  8.22s/it]
 21%|██        | 7850/37962 [18:12:02<68:37:36,  8.20s/it]
                                                          
{'loss': 1.4017, 'grad_norm': 7.348156929016113, 'learning_rate': 1.8405189459986864e-05, 'epoch': 0.21}

 21%|██        | 7850/37962 [18:12:02<68:37:36,  8.20s/it]
 21%|██        | 7851/37962 [18:12:10<68:54:24,  8.24s/it]
 21%|██        | 7852/37962 [18:12:18<68:51:08,  8.23s/it]
 21%|██        | 7853/37962 [18:12:27<68:53:25,  8.24s/it]
 21%|██        | 7854/37962 [18:12:35<69:38:51,  8.33s/it]
 21%|██        | 7855/37962 [18:12:43<68:59:30,  8.25s/it]
 21%|██        | 7856/37962 [18:12:51<69:01:36,  8.25s/it]
 21%|██        | 7857/37962 [18:13:00<69:03:08,  8.26s/it]
 21%|██        | 7858/37962 [18:13:08<69:38:45,  8.33s/it]
 21%|██        | 7859/37962 [18:13:17<69:34:59,  8.32s/it]
 21%|██        | 7860/37962 [18:13:25<69:39:22,  8.33s/it]
                                                          
{'loss': 1.5068, 'grad_norm': 11.87939167022705, 'learning_rate': 1.8400564130601424e-05, 'epoch': 0.21}

 21%|██        | 7860/37962 [18:13:25<69:39:22,  8.33s/it]
 21%|██        | 7861/37962 [18:13:33<69:25:15,  8.30s/it]
 21%|██        | 7862/37962 [18:13:41<69:11:10,  8.27s/it]
 21%|██        | 7863/37962 [18:13:50<69:37:43,  8.33s/it]
 21%|██        | 7864/37962 [18:13:58<69:28:19,  8.31s/it]
 21%|██        | 7865/37962 [18:14:07<70:00:12,  8.37s/it]
 21%|██        | 7866/37962 [18:14:14<68:45:58,  8.23s/it]
 21%|██        | 7867/37962 [18:14:23<68:34:04,  8.20s/it]
 21%|██        | 7868/37962 [18:14:31<69:01:59,  8.26s/it]
 21%|██        | 7869/37962 [18:14:39<69:10:43,  8.28s/it]
 21%|██        | 7870/37962 [18:14:47<68:30:19,  8.20s/it]
                                                          
{'loss': 1.3974, 'grad_norm': 8.974173545837402, 'learning_rate': 1.8395932686590882e-05, 'epoch': 0.21}

 21%|██        | 7870/37962 [18:14:47<68:30:19,  8.20s/it]
 21%|██        | 7871/37962 [18:14:56<69:01:41,  8.26s/it]
 21%|██        | 7872/37962 [18:15:04<69:13:19,  8.28s/it]
 21%|██        | 7873/37962 [18:15:12<69:19:22,  8.29s/it]
 21%|██        | 7874/37962 [18:15:21<69:15:07,  8.29s/it]
 21%|██        | 7875/37962 [18:15:29<69:05:01,  8.27s/it]
 21%|██        | 7876/37962 [18:15:37<68:58:38,  8.25s/it]
 21%|██        | 7877/37962 [18:15:45<68:14:42,  8.17s/it]
 21%|██        | 7878/37962 [18:15:53<68:22:22,  8.18s/it]
 21%|██        | 7879/37962 [18:16:02<68:54:07,  8.25s/it]
 21%|██        | 7880/37962 [18:16:10<69:13:12,  8.28s/it]
                                                          
{'loss': 1.4999, 'grad_norm': 12.853569984436035, 'learning_rate': 1.839129513132639e-05, 'epoch': 0.21}

 21%|██        | 7880/37962 [18:16:10<69:13:12,  8.28s/it]
 21%|██        | 7881/37962 [18:16:18<68:02:30,  8.14s/it]
 21%|██        | 7882/37962 [18:16:26<67:54:23,  8.13s/it]
 21%|██        | 7883/37962 [18:16:35<68:55:45,  8.25s/it]
 21%|██        | 7884/37962 [18:16:43<68:16:49,  8.17s/it]
 21%|██        | 7885/37962 [18:16:51<68:20:45,  8.18s/it]
 21%|██        | 7886/37962 [18:16:59<68:22:57,  8.19s/it]
 21%|██        | 7887/37962 [18:17:07<67:55:42,  8.13s/it]
 21%|██        | 7888/37962 [18:17:15<68:01:55,  8.14s/it]
 21%|██        | 7889/37962 [18:17:23<66:55:49,  8.01s/it]
 21%|██        | 7890/37962 [18:17:31<67:37:51,  8.10s/it]
                                                          
{'loss': 1.4251, 'grad_norm': 13.428462982177734, 'learning_rate': 1.8386651468183537e-05, 'epoch': 0.21}

 21%|██        | 7890/37962 [18:17:31<67:37:51,  8.10s/it]
 21%|██        | 7891/37962 [18:17:39<68:13:02,  8.17s/it]
 21%|██        | 7892/37962 [18:17:47<67:35:23,  8.09s/it]
 21%|██        | 7893/37962 [18:17:55<66:38:46,  7.98s/it]
 21%|██        | 7894/37962 [18:18:03<67:27:56,  8.08s/it]
 21%|██        | 7895/37962 [18:18:12<68:07:48,  8.16s/it]
 21%|██        | 7896/37962 [18:18:20<68:40:43,  8.22s/it]
 21%|██        | 7897/37962 [18:18:28<68:54:56,  8.25s/it]
 21%|██        | 7898/37962 [18:18:36<68:17:48,  8.18s/it]
 21%|██        | 7899/37962 [18:18:45<68:24:31,  8.19s/it]
 21%|██        | 7900/37962 [18:18:53<68:51:23,  8.25s/it]
                                                          
{'loss': 1.3643, 'grad_norm': 21.609275817871094, 'learning_rate': 1.838200170054237e-05, 'epoch': 0.21}

 21%|██        | 7900/37962 [18:18:53<68:51:23,  8.25s/it]
 21%|██        | 7901/37962 [18:19:01<68:59:07,  8.26s/it]
 21%|██        | 7902/37962 [18:19:10<69:18:05,  8.30s/it]
 21%|██        | 7903/37962 [18:19:18<69:31:21,  8.33s/it]
 21%|██        | 7904/37962 [18:19:26<69:15:30,  8.29s/it]
 21%|██        | 7905/37962 [18:19:35<69:21:44,  8.31s/it]
 21%|██        | 7906/37962 [18:19:43<68:59:26,  8.26s/it]
 21%|██        | 7907/37962 [18:19:51<68:11:56,  8.17s/it]
 21%|██        | 7908/37962 [18:19:59<68:37:09,  8.22s/it]
 21%|██        | 7909/37962 [18:20:07<69:01:58,  8.27s/it]
 21%|██        | 7910/37962 [18:20:15<68:09:41,  8.17s/it]
                                                          
{'loss': 1.4328, 'grad_norm': 6.954063415527344, 'learning_rate': 1.8377345831787375e-05, 'epoch': 0.21}

 21%|██        | 7910/37962 [18:20:15<68:09:41,  8.17s/it]
 21%|██        | 7911/37962 [18:20:24<68:10:59,  8.17s/it]
 21%|██        | 7912/37962 [18:20:32<68:38:18,  8.22s/it]
 21%|██        | 7913/37962 [18:20:40<68:40:21,  8.23s/it]
 21%|██        | 7914/37962 [18:20:48<68:18:36,  8.18s/it]
 21%|██        | 7915/37962 [18:20:56<68:19:15,  8.19s/it]
 21%|██        | 7916/37962 [18:21:05<68:56:12,  8.26s/it]
 21%|██        | 7917/37962 [18:21:13<69:14:37,  8.30s/it]
 21%|██        | 7918/37962 [18:21:21<67:58:48,  8.15s/it]
 21%|██        | 7919/37962 [18:21:29<68:06:50,  8.16s/it]
 21%|██        | 7920/37962 [18:21:37<68:17:24,  8.18s/it]
                                                          
{'loss': 1.4021, 'grad_norm': 12.26108455657959, 'learning_rate': 1.8372683865307478e-05, 'epoch': 0.21}

 21%|██        | 7920/37962 [18:21:37<68:17:24,  8.18s/it]
 21%|██        | 7921/37962 [18:21:46<68:45:19,  8.24s/it]
 21%|██        | 7922/37962 [18:21:54<68:28:17,  8.21s/it]
 21%|██        | 7923/37962 [18:22:02<68:00:18,  8.15s/it]
 21%|██        | 7924/37962 [18:22:10<68:15:29,  8.18s/it]
 21%|██        | 7925/37962 [18:22:18<68:24:10,  8.20s/it]
 21%|██        | 7926/37962 [18:22:27<68:53:35,  8.26s/it]
 21%|██        | 7927/37962 [18:22:35<69:12:53,  8.30s/it]
 21%|██        | 7928/37962 [18:22:43<68:23:21,  8.20s/it]
 21%|██        | 7929/37962 [18:22:51<68:32:43,  8.22s/it]
 21%|██        | 7930/37962 [18:23:00<68:20:42,  8.19s/it]
                                                          
{'loss': 1.4142, 'grad_norm': 22.034774780273438, 'learning_rate': 1.8368015804496042e-05, 'epoch': 0.21}

 21%|██        | 7930/37962 [18:23:00<68:20:42,  8.19s/it]
 21%|██        | 7931/37962 [18:23:08<68:21:32,  8.19s/it]
 21%|██        | 7932/37962 [18:23:16<68:16:57,  8.19s/it]
 21%|██        | 7933/37962 [18:23:24<67:50:07,  8.13s/it]
 21%|██        | 7934/37962 [18:23:32<68:29:44,  8.21s/it]
 21%|██        | 7935/37962 [18:23:40<67:35:03,  8.10s/it]
 21%|██        | 7936/37962 [18:23:49<68:02:36,  8.16s/it]
 21%|██        | 7937/37962 [18:23:57<68:25:39,  8.20s/it]
 21%|██        | 7938/37962 [18:24:05<68:09:06,  8.17s/it]
 21%|██        | 7939/37962 [18:24:13<67:34:41,  8.10s/it]
 21%|██        | 7940/37962 [18:24:21<68:11:37,  8.18s/it]
                                                          
{'loss': 1.4216, 'grad_norm': 9.830514907836914, 'learning_rate': 1.8363341652750867e-05, 'epoch': 0.21}

 21%|██        | 7940/37962 [18:24:21<68:11:37,  8.18s/it]
 21%|██        | 7941/37962 [18:24:30<68:38:54,  8.23s/it]
 21%|██        | 7942/37962 [18:24:38<68:13:22,  8.18s/it]
 21%|██        | 7943/37962 [18:24:46<68:31:48,  8.22s/it]
 21%|██        | 7944/37962 [18:24:54<68:20:09,  8.20s/it]
 21%|██        | 7945/37962 [18:25:02<68:19:12,  8.19s/it]
 21%|██        | 7946/37962 [18:25:10<68:20:45,  8.20s/it]
 21%|██        | 7947/37962 [18:25:19<68:33:36,  8.22s/it]
 21%|██        | 7948/37962 [18:25:27<68:28:38,  8.21s/it]
 21%|██        | 7949/37962 [18:25:35<68:43:03,  8.24s/it]
 21%|██        | 7950/37962 [18:25:43<68:40:27,  8.24s/it]
                                                          
{'loss': 1.4748, 'grad_norm': 17.104509353637695, 'learning_rate': 1.8358661413474187e-05, 'epoch': 0.21}

 21%|██        | 7950/37962 [18:25:43<68:40:27,  8.24s/it]
 21%|██        | 7951/37962 [18:25:52<69:28:41,  8.33s/it]
 21%|██        | 7952/37962 [18:26:00<69:30:58,  8.34s/it]
 21%|██        | 7953/37962 [18:26:09<69:32:56,  8.34s/it]
 21%|██        | 7954/37962 [18:26:17<68:38:59,  8.24s/it]
 21%|██        | 7955/37962 [18:26:25<68:40:54,  8.24s/it]
 21%|██        | 7956/37962 [18:26:33<69:05:05,  8.29s/it]
 21%|██        | 7957/37962 [18:26:41<67:55:15,  8.15s/it]
 21%|██        | 7958/37962 [18:26:49<67:51:08,  8.14s/it]
 21%|██        | 7959/37962 [18:26:57<67:49:56,  8.14s/it]
 21%|██        | 7960/37962 [18:27:06<68:15:34,  8.19s/it]
                                                          
{'loss': 1.4474, 'grad_norm': 4.474307537078857, 'learning_rate': 1.835397509007267e-05, 'epoch': 0.21}

 21%|██        | 7960/37962 [18:27:06<68:15:34,  8.19s/it]
 21%|██        | 7961/37962 [18:27:14<68:13:23,  8.19s/it]
 21%|██        | 7962/37962 [18:27:22<68:17:16,  8.19s/it]
 21%|██        | 7963/37962 [18:27:30<67:09:56,  8.06s/it]
 21%|██        | 7964/37962 [18:27:38<67:50:39,  8.14s/it]
 21%|██        | 7965/37962 [18:27:46<67:35:25,  8.11s/it]
 21%|██        | 7966/37962 [18:27:55<68:30:25,  8.22s/it]
 21%|██        | 7967/37962 [18:28:03<69:01:15,  8.28s/it]
 21%|██        | 7968/37962 [18:28:12<69:41:01,  8.36s/it]
 21%|██        | 7969/37962 [18:28:20<68:34:00,  8.23s/it]
 21%|██        | 7970/37962 [18:28:28<68:34:53,  8.23s/it]
                                                          
{'loss': 1.3737, 'grad_norm': 20.870840072631836, 'learning_rate': 1.8349282685957406e-05, 'epoch': 0.21}

 21%|██        | 7970/37962 [18:28:28<68:34:53,  8.23s/it]
 21%|██        | 7971/37962 [18:28:36<67:37:53,  8.12s/it]
 21%|██        | 7972/37962 [18:28:44<67:45:44,  8.13s/it]
 21%|██        | 7973/37962 [18:28:52<67:53:53,  8.15s/it]
 21%|██        | 7974/37962 [18:29:00<68:15:53,  8.20s/it]
 21%|██        | 7975/37962 [18:29:09<68:43:28,  8.25s/it]
 21%|██        | 7976/37962 [18:29:17<68:34:14,  8.23s/it]
 21%|██        | 7977/37962 [18:29:25<68:56:15,  8.28s/it]
 21%|██        | 7978/37962 [18:29:34<68:57:07,  8.28s/it]
 21%|██        | 7979/37962 [18:29:42<68:07:22,  8.18s/it]
 21%|██        | 7980/37962 [18:29:50<67:27:02,  8.10s/it]
                                                          
{'loss': 1.4003, 'grad_norm': 3.135221242904663, 'learning_rate': 1.8344584204543915e-05, 'epoch': 0.21}

 21%|██        | 7980/37962 [18:29:50<67:27:02,  8.10s/it]
 21%|██        | 7981/37962 [18:29:58<68:02:30,  8.17s/it]
 21%|██        | 7982/37962 [18:30:06<68:32:06,  8.23s/it]
 21%|██        | 7983/37962 [18:30:15<68:55:00,  8.28s/it]
 21%|██        | 7984/37962 [18:30:23<68:33:35,  8.23s/it]
 21%|██        | 7985/37962 [18:30:31<69:00:51,  8.29s/it]
 21%|██        | 7986/37962 [18:30:39<68:54:47,  8.28s/it]
 21%|██        | 7987/37962 [18:30:48<68:49:06,  8.27s/it]
 21%|██        | 7988/37962 [18:30:56<68:04:48,  8.18s/it]
 21%|██        | 7989/37962 [18:31:04<68:32:40,  8.23s/it]
 21%|██        | 7990/37962 [18:31:12<68:41:40,  8.25s/it]
                                                          
{'loss': 1.4519, 'grad_norm': 19.170940399169922, 'learning_rate': 1.8339879649252143e-05, 'epoch': 0.21}

 21%|██        | 7990/37962 [18:31:12<68:41:40,  8.25s/it]
 21%|██        | 7991/37962 [18:31:20<68:33:28,  8.23s/it]
 21%|██        | 7992/37962 [18:31:28<67:33:51,  8.12s/it]
 21%|██        | 7993/37962 [18:31:37<67:47:32,  8.14s/it]
 21%|██        | 7994/37962 [18:31:45<68:20:44,  8.21s/it]
 21%|██        | 7995/37962 [18:31:53<68:25:05,  8.22s/it]
 21%|██        | 7996/37962 [18:32:01<67:12:00,  8.07s/it]
 21%|██        | 7997/37962 [18:32:09<66:40:52,  8.01s/it]
 21%|██        | 7998/37962 [18:32:17<67:17:31,  8.08s/it]
 21%|██        | 7999/37962 [18:32:25<67:37:22,  8.12s/it]
 21%|██        | 8000/37962 [18:32:33<67:45:55,  8.14s/it]
                                                          
{'loss': 1.5245, 'grad_norm': 12.897626876831055, 'learning_rate': 1.8335169023506454e-05, 'epoch': 0.21}

 21%|██        | 8000/37962 [18:32:33<67:45:55,  8.14s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 21%|██        | 8001/37962 [18:33:22<169:41:14, 20.39s/it]
 21%|██        | 8002/37962 [18:33:30<138:39:24, 16.66s/it]
 21%|██        | 8003/37962 [18:33:38<117:05:20, 14.07s/it]
 21%|██        | 8004/37962 [18:33:46<101:45:07, 12.23s/it]
 21%|██        | 8005/37962 [18:33:54<91:45:54, 11.03s/it] 
 21%|██        | 8006/37962 [18:34:03<85:18:49, 10.25s/it]
 21%|██        | 8007/37962 [18:34:11<80:26:37,  9.67s/it]
 21%|██        | 8008/37962 [18:34:19<76:22:04,  9.18s/it]
 21%|██        | 8009/37962 [18:34:28<74:14:04,  8.92s/it]
 21%|██        | 8010/37962 [18:34:36<72:09:06,  8.67s/it]
                                                          
{'loss': 1.4311, 'grad_norm': 4.79339075088501, 'learning_rate': 1.8330452330735628e-05, 'epoch': 0.21}

 21%|██        | 8010/37962 [18:34:36<72:09:06,  8.67s/it]
 21%|██        | 8011/37962 [18:34:44<71:13:44,  8.56s/it]
 21%|██        | 8012/37962 [18:34:52<69:52:01,  8.40s/it]
 21%|██        | 8013/37962 [18:35:00<69:36:36,  8.37s/it]
 21%|██        | 8014/37962 [18:35:08<69:00:07,  8.29s/it]
 21%|██        | 8015/37962 [18:35:17<68:58:16,  8.29s/it]
 21%|██        | 8016/37962 [18:35:25<69:00:44,  8.30s/it]
 21%|██        | 8017/37962 [18:35:34<69:47:53,  8.39s/it]
 21%|██        | 8018/37962 [18:35:42<69:02:31,  8.30s/it]
 21%|██        | 8019/37962 [18:35:50<68:20:08,  8.22s/it]
 21%|██        | 8020/37962 [18:35:58<68:27:36,  8.23s/it]
                                                          
{'loss': 1.5134, 'grad_norm': 18.399019241333008, 'learning_rate': 1.8325729574372868e-05, 'epoch': 0.21}

 21%|██        | 8020/37962 [18:35:58<68:27:36,  8.23s/it]
 21%|██        | 8021/37962 [18:36:06<68:39:40,  8.26s/it]
 21%|██        | 8022/37962 [18:36:15<68:45:19,  8.27s/it]
 21%|██        | 8023/37962 [18:36:23<68:49:03,  8.27s/it]
 21%|██        | 8024/37962 [18:36:31<69:17:14,  8.33s/it]
 21%|██        | 8025/37962 [18:36:40<68:59:22,  8.30s/it]
 21%|██        | 8026/37962 [18:36:48<68:57:10,  8.29s/it]
 21%|██        | 8027/37962 [18:36:56<68:40:00,  8.26s/it]
 21%|██        | 8028/37962 [18:37:04<68:56:04,  8.29s/it]
 21%|██        | 8029/37962 [18:37:12<68:07:36,  8.19s/it]
 21%|██        | 8030/37962 [18:37:21<68:12:14,  8.20s/it]
                                                          
{'loss': 1.5236, 'grad_norm': 14.639631271362305, 'learning_rate': 1.832100075785578e-05, 'epoch': 0.21}

 21%|██        | 8030/37962 [18:37:21<68:12:14,  8.20s/it]
 21%|██        | 8031/37962 [18:37:29<68:38:29,  8.26s/it]
 21%|██        | 8032/37962 [18:37:37<68:59:53,  8.30s/it]
 21%|██        | 8033/37962 [18:37:45<68:33:04,  8.25s/it]
 21%|██        | 8034/37962 [18:37:54<68:26:27,  8.23s/it]
 21%|██        | 8035/37962 [18:38:02<68:41:56,  8.26s/it]
 21%|██        | 8036/37962 [18:38:10<68:51:34,  8.28s/it]
 21%|██        | 8037/37962 [18:38:19<68:54:40,  8.29s/it]
 21%|██        | 8038/37962 [18:38:27<69:09:34,  8.32s/it]
 21%|██        | 8039/37962 [18:38:35<68:09:19,  8.20s/it]
 21%|██        | 8040/37962 [18:38:43<68:14:08,  8.21s/it]
                                                          
{'loss': 1.4059, 'grad_norm': 9.048181533813477, 'learning_rate': 1.8316265884626398e-05, 'epoch': 0.21}

 21%|██        | 8040/37962 [18:38:43<68:14:08,  8.21s/it]
 21%|██        | 8041/37962 [18:38:51<68:25:53,  8.23s/it]
 21%|██        | 8042/37962 [18:39:00<68:52:46,  8.29s/it]
 21%|██        | 8043/37962 [18:39:08<68:23:40,  8.23s/it]
 21%|██        | 8044/37962 [18:39:16<68:15:22,  8.21s/it]
 21%|██        | 8045/37962 [18:39:24<68:10:31,  8.20s/it]
 21%|██        | 8046/37962 [18:39:33<68:18:08,  8.22s/it]
 21%|██        | 8047/37962 [18:39:41<67:47:43,  8.16s/it]
 21%|██        | 8048/37962 [18:39:49<68:06:47,  8.20s/it]
 21%|██        | 8049/37962 [18:39:57<68:59:15,  8.30s/it]
 21%|██        | 8050/37962 [18:40:06<69:32:04,  8.37s/it]
                                                          
{'loss': 1.4439, 'grad_norm': 10.649868965148926, 'learning_rate': 1.8311524958131142e-05, 'epoch': 0.21}

 21%|██        | 8050/37962 [18:40:06<69:32:04,  8.37s/it]
 21%|██        | 8051/37962 [18:40:14<69:16:16,  8.34s/it]
 21%|██        | 8052/37962 [18:40:22<68:17:35,  8.22s/it]
 21%|██        | 8053/37962 [18:40:31<68:47:58,  8.28s/it]
 21%|██        | 8054/37962 [18:40:39<67:49:56,  8.16s/it]
 21%|██        | 8055/37962 [18:40:47<67:31:31,  8.13s/it]
 21%|██        | 8056/37962 [18:40:55<67:25:35,  8.12s/it]
 21%|██        | 8057/37962 [18:41:03<67:47:01,  8.16s/it]
 21%|██        | 8058/37962 [18:41:11<68:19:26,  8.23s/it]
 21%|██        | 8059/37962 [18:41:20<68:20:13,  8.23s/it]
 21%|██        | 8060/37962 [18:41:28<67:52:05,  8.17s/it]
                                                          
{'loss': 1.3566, 'grad_norm': 7.749479293823242, 'learning_rate': 1.830677798182086e-05, 'epoch': 0.21}

 21%|██        | 8060/37962 [18:41:28<67:52:05,  8.17s/it]
 21%|██        | 8061/37962 [18:41:36<68:15:38,  8.22s/it]
 21%|██        | 8062/37962 [18:41:44<68:33:58,  8.26s/it]
 21%|██        | 8063/37962 [18:41:52<67:33:05,  8.13s/it]
 21%|██        | 8064/37962 [18:42:00<67:55:46,  8.18s/it]
 21%|██        | 8065/37962 [18:42:09<68:11:43,  8.21s/it]
 21%|██        | 8066/37962 [18:42:17<68:54:14,  8.30s/it]
 21%|██▏       | 8067/37962 [18:42:25<68:37:42,  8.26s/it]
 21%|██▏       | 8068/37962 [18:42:34<68:37:06,  8.26s/it]
 21%|██▏       | 8069/37962 [18:42:42<67:49:33,  8.17s/it]
 21%|██▏       | 8070/37962 [18:42:49<66:54:11,  8.06s/it]
                                                          
{'loss': 1.3192, 'grad_norm': 19.676347732543945, 'learning_rate': 1.830202495915079e-05, 'epoch': 0.21}

 21%|██▏       | 8070/37962 [18:42:49<66:54:11,  8.06s/it]
 21%|██▏       | 8071/37962 [18:42:57<67:01:29,  8.07s/it]
 21%|██▏       | 8072/37962 [18:43:06<67:42:50,  8.16s/it]
 21%|██▏       | 8073/37962 [18:43:14<67:47:25,  8.17s/it]
 21%|██▏       | 8074/37962 [18:43:22<67:57:22,  8.19s/it]
 21%|██▏       | 8075/37962 [18:43:31<68:48:20,  8.29s/it]
 21%|██▏       | 8076/37962 [18:43:39<68:48:14,  8.29s/it]
 21%|██▏       | 8077/37962 [18:43:47<68:33:29,  8.26s/it]
 21%|██▏       | 8078/37962 [18:43:55<67:42:08,  8.16s/it]
 21%|██▏       | 8079/37962 [18:44:03<68:06:40,  8.21s/it]
 21%|██▏       | 8080/37962 [18:44:12<67:43:20,  8.16s/it]
                                                          
{'loss': 1.5363, 'grad_norm': 20.5679931640625, 'learning_rate': 1.829726589358057e-05, 'epoch': 0.21}

 21%|██▏       | 8080/37962 [18:44:12<67:43:20,  8.16s/it]
 21%|██▏       | 8081/37962 [18:44:20<67:47:36,  8.17s/it]
 21%|██▏       | 8082/37962 [18:44:28<67:52:49,  8.18s/it]
 21%|██▏       | 8083/37962 [18:44:36<67:54:22,  8.18s/it]
 21%|██▏       | 8084/37962 [18:44:44<68:09:25,  8.21s/it]
 21%|██▏       | 8085/37962 [18:44:53<68:09:55,  8.21s/it]
 21%|██▏       | 8086/37962 [18:45:01<67:39:03,  8.15s/it]
 21%|██▏       | 8087/37962 [18:45:09<67:49:31,  8.17s/it]
 21%|██▏       | 8088/37962 [18:45:17<68:29:06,  8.25s/it]
 21%|██▏       | 8089/37962 [18:45:26<68:30:35,  8.26s/it]
 21%|██▏       | 8090/37962 [18:45:34<68:32:13,  8.26s/it]
                                                          
{'loss': 1.4819, 'grad_norm': 14.283836364746094, 'learning_rate': 1.829250078857425e-05, 'epoch': 0.21}

 21%|██▏       | 8090/37962 [18:45:34<68:32:13,  8.26s/it]
 21%|██▏       | 8091/37962 [18:45:42<68:43:27,  8.28s/it]
 21%|██▏       | 8092/37962 [18:45:50<68:36:22,  8.27s/it]
 21%|██▏       | 8093/37962 [18:45:59<68:37:28,  8.27s/it]
 21%|██▏       | 8094/37962 [18:46:07<68:37:43,  8.27s/it]
 21%|██▏       | 8095/37962 [18:46:15<67:34:03,  8.14s/it]
 21%|██▏       | 8096/37962 [18:46:23<67:56:32,  8.19s/it]
 21%|██▏       | 8097/37962 [18:46:31<68:01:49,  8.20s/it]
 21%|██▏       | 8098/37962 [18:46:39<67:56:21,  8.19s/it]
 21%|██▏       | 8099/37962 [18:46:48<67:58:14,  8.19s/it]
 21%|██▏       | 8100/37962 [18:46:56<67:26:01,  8.13s/it]
                                                          
{'loss': 1.5937, 'grad_norm': 14.665865898132324, 'learning_rate': 1.8287729647600267e-05, 'epoch': 0.21}

 21%|██▏       | 8100/37962 [18:46:56<67:26:01,  8.13s/it]
 21%|██▏       | 8101/37962 [18:47:04<67:57:03,  8.19s/it]
 21%|██▏       | 8102/37962 [18:47:12<67:15:22,  8.11s/it]
 21%|██▏       | 8103/37962 [18:47:20<66:50:54,  8.06s/it]
 21%|██▏       | 8104/37962 [18:47:28<67:17:08,  8.11s/it]
 21%|██▏       | 8105/37962 [18:47:36<67:43:43,  8.17s/it]
 21%|██▏       | 8106/37962 [18:47:45<68:01:52,  8.20s/it]
 21%|██▏       | 8107/37962 [18:47:53<68:04:01,  8.21s/it]
 21%|██▏       | 8108/37962 [18:48:01<68:09:19,  8.22s/it]
 21%|██▏       | 8109/37962 [18:48:10<68:50:57,  8.30s/it]
 21%|██▏       | 8110/37962 [18:48:18<68:13:33,  8.23s/it]
                                                          
{'loss': 1.4631, 'grad_norm': 19.260404586791992, 'learning_rate': 1.8282952474131442e-05, 'epoch': 0.21}

 21%|██▏       | 8110/37962 [18:48:18<68:13:33,  8.23s/it]
 21%|██▏       | 8111/37962 [18:48:26<68:13:49,  8.23s/it]
 21%|██▏       | 8112/37962 [18:48:34<68:47:09,  8.30s/it]
 21%|██▏       | 8113/37962 [18:48:43<68:48:22,  8.30s/it]
 21%|██▏       | 8114/37962 [18:48:51<68:45:58,  8.29s/it]
 21%|██▏       | 8115/37962 [18:48:59<68:43:38,  8.29s/it]
 21%|██▏       | 8116/37962 [18:49:07<68:39:01,  8.28s/it]
 21%|██▏       | 8117/37962 [18:49:15<67:49:21,  8.18s/it]
 21%|██▏       | 8118/37962 [18:49:24<67:35:57,  8.15s/it]
 21%|██▏       | 8119/37962 [18:49:32<67:36:42,  8.16s/it]
 21%|██▏       | 8120/37962 [18:49:40<67:19:00,  8.12s/it]
                                                          
{'loss': 1.3554, 'grad_norm': 13.636507034301758, 'learning_rate': 1.8278169271645007e-05, 'epoch': 0.21}

 21%|██▏       | 8120/37962 [18:49:40<67:19:00,  8.12s/it]
 21%|██▏       | 8121/37962 [18:49:48<67:42:35,  8.17s/it]
 21%|██▏       | 8122/37962 [18:49:56<67:31:44,  8.15s/it]
 21%|██▏       | 8123/37962 [18:50:04<66:34:48,  8.03s/it]
 21%|██▏       | 8124/37962 [18:50:12<67:04:23,  8.09s/it]
 21%|██▏       | 8125/37962 [18:50:20<66:47:48,  8.06s/it]
 21%|██▏       | 8126/37962 [18:50:29<67:48:48,  8.18s/it]
 21%|██▏       | 8127/37962 [18:50:37<67:55:00,  8.20s/it]
 21%|██▏       | 8128/37962 [18:50:45<68:06:39,  8.22s/it]
 21%|██▏       | 8129/37962 [18:50:53<68:26:09,  8.26s/it]
 21%|██▏       | 8130/37962 [18:51:02<68:36:18,  8.28s/it]
                                                          
{'loss': 1.4486, 'grad_norm': 11.58022403717041, 'learning_rate': 1.8273380043622572e-05, 'epoch': 0.21}

 21%|██▏       | 8130/37962 [18:51:02<68:36:18,  8.28s/it]
 21%|██▏       | 8131/37962 [18:51:10<68:48:27,  8.30s/it]
 21%|██▏       | 8132/37962 [18:51:18<68:49:09,  8.31s/it]
 21%|██▏       | 8133/37962 [18:51:27<68:45:40,  8.30s/it]
 21%|██▏       | 8134/37962 [18:51:35<67:47:17,  8.18s/it]
 21%|██▏       | 8135/37962 [18:51:43<67:09:33,  8.11s/it]
 21%|██▏       | 8136/37962 [18:51:51<67:26:26,  8.14s/it]
 21%|██▏       | 8137/37962 [18:51:59<67:34:22,  8.16s/it]
 21%|██▏       | 8138/37962 [18:52:07<67:19:05,  8.13s/it]
 21%|██▏       | 8139/37962 [18:52:15<67:33:12,  8.15s/it]
 21%|██▏       | 8140/37962 [18:52:23<67:34:19,  8.16s/it]
                                                          
{'loss': 1.5198, 'grad_norm': 16.682479858398438, 'learning_rate': 1.8268584793550124e-05, 'epoch': 0.21}

 21%|██▏       | 8140/37962 [18:52:23<67:34:19,  8.16s/it]
 21%|██▏       | 8141/37962 [18:52:32<67:35:20,  8.16s/it]
 21%|██▏       | 8142/37962 [18:52:40<68:10:29,  8.23s/it]
 21%|██▏       | 8143/37962 [18:52:48<68:17:14,  8.24s/it]
 21%|██▏       | 8144/37962 [18:52:57<68:42:41,  8.30s/it]
 21%|██▏       | 8145/37962 [18:53:05<68:10:02,  8.23s/it]
 21%|██▏       | 8146/37962 [18:53:13<68:23:18,  8.26s/it]
 21%|██▏       | 8147/37962 [18:53:21<68:34:05,  8.28s/it]
 21%|██▏       | 8148/37962 [18:53:30<68:35:57,  8.28s/it]
 21%|██▏       | 8149/37962 [18:53:38<68:24:24,  8.26s/it]
 21%|██▏       | 8150/37962 [18:53:46<68:43:50,  8.30s/it]
                                                          
{'loss': 1.4521, 'grad_norm': 18.060522079467773, 'learning_rate': 1.8263783524918047e-05, 'epoch': 0.21}

 21%|██▏       | 8150/37962 [18:53:46<68:43:50,  8.30s/it]
 21%|██▏       | 8151/37962 [18:53:55<69:06:55,  8.35s/it]
 21%|██▏       | 8152/37962 [18:54:03<69:42:47,  8.42s/it]
 21%|██▏       | 8153/37962 [18:54:11<68:30:40,  8.27s/it]
 21%|██▏       | 8154/37962 [18:54:19<67:19:12,  8.13s/it]
 21%|██▏       | 8155/37962 [18:54:27<66:46:34,  8.07s/it]
 21%|██▏       | 8156/37962 [18:54:35<67:20:09,  8.13s/it]
 21%|██▏       | 8157/37962 [18:54:44<67:53:25,  8.20s/it]
 21%|██▏       | 8158/37962 [18:54:52<68:06:28,  8.23s/it]
 21%|██▏       | 8159/37962 [18:55:00<68:24:32,  8.26s/it]
 21%|██▏       | 8160/37962 [18:55:09<68:30:37,  8.28s/it]
                                                          
{'loss': 1.4031, 'grad_norm': 1.8772904872894287, 'learning_rate': 1.825897624122111e-05, 'epoch': 0.21}

 21%|██▏       | 8160/37962 [18:55:09<68:30:37,  8.28s/it]
 21%|██▏       | 8161/37962 [18:55:16<66:52:02,  8.08s/it]
 22%|██▏       | 8162/37962 [18:55:24<67:30:09,  8.15s/it]
 22%|██▏       | 8163/37962 [18:55:33<67:47:50,  8.19s/it]
 22%|██▏       | 8164/37962 [18:55:41<68:44:03,  8.30s/it]
 22%|██▏       | 8165/37962 [18:55:50<68:54:45,  8.33s/it]
 22%|██▏       | 8166/37962 [18:55:58<68:37:11,  8.29s/it]
 22%|██▏       | 8167/37962 [18:56:06<68:36:39,  8.29s/it]
 22%|██▏       | 8168/37962 [18:56:15<69:09:00,  8.36s/it]
 22%|██▏       | 8169/37962 [18:56:23<68:43:07,  8.30s/it]
 22%|██▏       | 8170/37962 [18:56:31<69:04:40,  8.35s/it]
                                                          
{'loss': 1.4261, 'grad_norm': 10.103849411010742, 'learning_rate': 1.825416294595844e-05, 'epoch': 0.22}

 22%|██▏       | 8170/37962 [18:56:31<69:04:40,  8.35s/it]
 22%|██▏       | 8171/37962 [18:56:39<68:36:42,  8.29s/it]
 22%|██▏       | 8172/37962 [18:56:48<68:20:42,  8.26s/it]
 22%|██▏       | 8173/37962 [18:56:56<68:25:55,  8.27s/it]
 22%|██▏       | 8174/37962 [18:57:04<68:25:00,  8.27s/it]
 22%|██▏       | 8175/37962 [18:57:12<67:42:01,  8.18s/it]
 22%|██▏       | 8176/37962 [18:57:21<68:02:05,  8.22s/it]
 22%|██▏       | 8177/37962 [18:57:29<67:36:00,  8.17s/it]
 22%|██▏       | 8178/37962 [18:57:37<67:18:17,  8.14s/it]
 22%|██▏       | 8179/37962 [18:57:45<67:29:51,  8.16s/it]
 22%|██▏       | 8180/37962 [18:57:53<68:02:35,  8.22s/it]
                                                          
{'loss': 1.3749, 'grad_norm': 10.959770202636719, 'learning_rate': 1.8249343642633556e-05, 'epoch': 0.22}

 22%|██▏       | 8180/37962 [18:57:53<68:02:35,  8.22s/it]
 22%|██▏       | 8181/37962 [18:58:02<68:35:50,  8.29s/it]
 22%|██▏       | 8182/37962 [18:58:10<68:40:49,  8.30s/it]
 22%|██▏       | 8183/37962 [18:58:18<68:30:06,  8.28s/it]
 22%|██▏       | 8184/37962 [18:58:26<68:14:20,  8.25s/it]
 22%|██▏       | 8185/37962 [18:58:35<68:25:49,  8.27s/it]
 22%|██▏       | 8186/37962 [18:58:43<67:28:24,  8.16s/it]
 22%|██▏       | 8187/37962 [18:58:51<67:48:31,  8.20s/it]
 22%|██▏       | 8188/37962 [18:58:59<68:09:37,  8.24s/it]
 22%|██▏       | 8189/37962 [18:59:08<68:26:47,  8.28s/it]
 22%|██▏       | 8190/37962 [18:59:16<68:18:52,  8.26s/it]
                                                          
{'loss': 1.4826, 'grad_norm': 21.110109329223633, 'learning_rate': 1.824451833475434e-05, 'epoch': 0.22}

 22%|██▏       | 8190/37962 [18:59:16<68:18:52,  8.26s/it]
 22%|██▏       | 8191/37962 [18:59:24<67:47:56,  8.20s/it]
 22%|██▏       | 8192/37962 [18:59:32<67:02:24,  8.11s/it]
 22%|██▏       | 8193/37962 [18:59:40<67:31:54,  8.17s/it]
 22%|██▏       | 8194/37962 [18:59:48<68:06:07,  8.24s/it]
 22%|██▏       | 8195/37962 [18:59:57<67:59:55,  8.22s/it]
 22%|██▏       | 8196/37962 [19:00:05<68:02:05,  8.23s/it]
 22%|██▏       | 8197/37962 [19:00:13<67:32:35,  8.17s/it]
 22%|██▏       | 8198/37962 [19:00:21<67:41:20,  8.19s/it]
 22%|██▏       | 8199/37962 [19:00:29<67:40:23,  8.19s/it]
 22%|██▏       | 8200/37962 [19:00:38<67:51:22,  8.21s/it]
                                                          
{'loss': 1.5701, 'grad_norm': 19.7383975982666, 'learning_rate': 1.823968702583306e-05, 'epoch': 0.22}

 22%|██▏       | 8200/37962 [19:00:38<67:51:22,  8.21s/it]
 22%|██▏       | 8201/37962 [19:00:46<68:01:53,  8.23s/it]
 22%|██▏       | 8202/37962 [19:00:54<67:35:15,  8.18s/it]
 22%|██▏       | 8203/37962 [19:01:02<68:00:46,  8.23s/it]
 22%|██▏       | 8204/37962 [19:01:10<67:28:04,  8.16s/it]
 22%|██▏       | 8205/37962 [19:01:19<67:39:59,  8.19s/it]
 22%|██▏       | 8206/37962 [19:01:27<68:07:05,  8.24s/it]
 22%|██▏       | 8207/37962 [19:01:35<68:08:05,  8.24s/it]
 22%|██▏       | 8208/37962 [19:01:43<67:52:30,  8.21s/it]
 22%|██▏       | 8209/37962 [19:01:52<67:53:47,  8.22s/it]
 22%|██▏       | 8210/37962 [19:02:00<68:25:13,  8.28s/it]
                                                          
{'loss': 1.4694, 'grad_norm': 11.008354187011719, 'learning_rate': 1.8234849719386333e-05, 'epoch': 0.22}

 22%|██▏       | 8210/37962 [19:02:00<68:25:13,  8.28s/it]
 22%|██▏       | 8211/37962 [19:02:08<68:40:45,  8.31s/it]
 22%|██▏       | 8212/37962 [19:02:17<68:36:55,  8.30s/it]
 22%|██▏       | 8213/37962 [19:02:25<68:39:44,  8.31s/it]
 22%|██▏       | 8214/37962 [19:02:33<68:42:56,  8.32s/it]
 22%|██▏       | 8215/37962 [19:02:41<68:12:05,  8.25s/it]
 22%|██▏       | 8216/37962 [19:02:50<68:15:26,  8.26s/it]
 22%|██▏       | 8217/37962 [19:02:58<68:12:08,  8.25s/it]
 22%|██▏       | 8218/37962 [19:03:06<69:01:39,  8.35s/it]
 22%|██▏       | 8219/37962 [19:03:15<69:17:46,  8.39s/it]
 22%|██▏       | 8220/37962 [19:03:23<69:20:50,  8.39s/it]
                                                          
{'loss': 1.4046, 'grad_norm': 6.011826038360596, 'learning_rate': 1.8230006418935157e-05, 'epoch': 0.22}

 22%|██▏       | 8220/37962 [19:03:23<69:20:50,  8.39s/it]
 22%|██▏       | 8221/37962 [19:03:32<68:59:48,  8.35s/it]
 22%|██▏       | 8222/37962 [19:03:40<69:04:59,  8.36s/it]
 22%|██▏       | 8223/37962 [19:03:48<69:20:19,  8.39s/it]
 22%|██▏       | 8224/37962 [19:03:56<68:04:22,  8.24s/it]
 22%|██▏       | 8225/37962 [19:04:05<68:01:56,  8.24s/it]
 22%|██▏       | 8226/37962 [19:04:13<68:12:02,  8.26s/it]
 22%|██▏       | 8227/37962 [19:04:21<68:00:44,  8.23s/it]
 22%|██▏       | 8228/37962 [19:04:29<67:57:11,  8.23s/it]
 22%|██▏       | 8229/37962 [19:04:38<68:28:50,  8.29s/it]
 22%|██▏       | 8230/37962 [19:04:46<68:13:48,  8.26s/it]
                                                          
{'loss': 1.4031, 'grad_norm': 7.7159647941589355, 'learning_rate': 1.822515712800488e-05, 'epoch': 0.22}

 22%|██▏       | 8230/37962 [19:04:46<68:13:48,  8.26s/it]
 22%|██▏       | 8231/37962 [19:04:54<68:31:02,  8.30s/it]
 22%|██▏       | 8232/37962 [19:05:03<68:58:28,  8.35s/it]
 22%|██▏       | 8233/37962 [19:05:11<69:10:17,  8.38s/it]
 22%|██▏       | 8234/37962 [19:05:19<68:55:51,  8.35s/it]
 22%|██▏       | 8235/37962 [19:05:28<69:06:16,  8.37s/it]
 22%|██▏       | 8236/37962 [19:05:36<69:08:48,  8.37s/it]
 22%|██▏       | 8237/37962 [19:05:45<68:58:43,  8.35s/it]
 22%|██▏       | 8238/37962 [19:05:53<68:54:01,  8.34s/it]
 22%|██▏       | 8239/37962 [19:06:01<68:49:06,  8.34s/it]
 22%|██▏       | 8240/37962 [19:06:10<69:05:53,  8.37s/it]
                                                          
{'loss': 1.3999, 'grad_norm': 3.4496095180511475, 'learning_rate': 1.8220301850125226e-05, 'epoch': 0.22}

 22%|██▏       | 8240/37962 [19:06:10<69:05:53,  8.37s/it]
 22%|██▏       | 8241/37962 [19:06:18<68:38:46,  8.31s/it]
 22%|██▏       | 8242/37962 [19:06:26<68:22:21,  8.28s/it]
 22%|██▏       | 8243/37962 [19:06:34<68:33:27,  8.30s/it]
 22%|██▏       | 8244/37962 [19:06:43<68:27:20,  8.29s/it]
 22%|██▏       | 8245/37962 [19:06:51<68:14:59,  8.27s/it]
 22%|██▏       | 8246/37962 [19:06:59<68:08:25,  8.25s/it]
 22%|██▏       | 8247/37962 [19:07:07<67:49:27,  8.22s/it]
 22%|██▏       | 8248/37962 [19:07:16<68:20:40,  8.28s/it]
 22%|██▏       | 8249/37962 [19:07:24<68:12:14,  8.26s/it]
 22%|██▏       | 8250/37962 [19:07:32<67:07:04,  8.13s/it]
                                                          
{'loss': 1.3857, 'grad_norm': 12.347916603088379, 'learning_rate': 1.821544058883026e-05, 'epoch': 0.22}

 22%|██▏       | 8250/37962 [19:07:32<67:07:04,  8.13s/it]
 22%|██▏       | 8251/37962 [19:07:40<66:52:18,  8.10s/it]
 22%|██▏       | 8252/37962 [19:07:48<67:05:19,  8.13s/it]
 22%|██▏       | 8253/37962 [19:07:56<66:59:23,  8.12s/it]
 22%|██▏       | 8254/37962 [19:08:04<67:15:55,  8.15s/it]
 22%|██▏       | 8255/37962 [19:08:12<67:12:42,  8.14s/it]
 22%|██▏       | 8256/37962 [19:08:21<67:47:43,  8.22s/it]
 22%|██▏       | 8257/37962 [19:08:29<68:13:26,  8.27s/it]
 22%|██▏       | 8258/37962 [19:08:37<68:05:20,  8.25s/it]
 22%|██▏       | 8259/37962 [19:08:46<68:22:56,  8.29s/it]
 22%|██▏       | 8260/37962 [19:08:54<67:46:41,  8.21s/it]
                                                          
{'loss': 1.5068, 'grad_norm': 6.3514838218688965, 'learning_rate': 1.8210573347658413e-05, 'epoch': 0.22}

 22%|██▏       | 8260/37962 [19:08:54<67:46:41,  8.21s/it]
 22%|██▏       | 8261/37962 [19:09:02<68:12:29,  8.27s/it]
 22%|██▏       | 8262/37962 [19:09:10<68:08:24,  8.26s/it]
 22%|██▏       | 8263/37962 [19:09:18<67:20:53,  8.16s/it]
 22%|██▏       | 8264/37962 [19:09:27<67:36:36,  8.20s/it]
 22%|██▏       | 8265/37962 [19:09:35<67:42:37,  8.21s/it]
 22%|██▏       | 8266/37962 [19:09:43<67:43:47,  8.21s/it]
 22%|██▏       | 8267/37962 [19:09:51<68:03:16,  8.25s/it]
 22%|██▏       | 8268/37962 [19:10:00<68:13:01,  8.27s/it]
 22%|██▏       | 8269/37962 [19:10:08<67:31:12,  8.19s/it]
 22%|██▏       | 8270/37962 [19:10:16<66:44:07,  8.09s/it]
                                                          
{'loss': 1.3398, 'grad_norm': 2.572840690612793, 'learning_rate': 1.8205700130152465e-05, 'epoch': 0.22}

 22%|██▏       | 8270/37962 [19:10:16<66:44:07,  8.09s/it]
 22%|██▏       | 8271/37962 [19:10:24<66:18:29,  8.04s/it]
 22%|██▏       | 8272/37962 [19:10:32<67:20:25,  8.17s/it]
 22%|██▏       | 8273/37962 [19:10:41<68:16:01,  8.28s/it]
 22%|██▏       | 8274/37962 [19:10:49<68:20:55,  8.29s/it]
 22%|██▏       | 8275/37962 [19:10:57<68:19:06,  8.28s/it]
 22%|██▏       | 8276/37962 [19:11:05<68:08:23,  8.26s/it]
 22%|██▏       | 8277/37962 [19:11:14<68:35:37,  8.32s/it]
 22%|██▏       | 8278/37962 [19:11:22<68:43:26,  8.33s/it]
 22%|██▏       | 8279/37962 [19:11:30<68:19:23,  8.29s/it]
 22%|██▏       | 8280/37962 [19:11:39<68:21:19,  8.29s/it]
                                                          
{'loss': 1.4208, 'grad_norm': 5.786165714263916, 'learning_rate': 1.8200820939859544e-05, 'epoch': 0.22}

 22%|██▏       | 8280/37962 [19:11:39<68:21:19,  8.29s/it]
 22%|██▏       | 8281/37962 [19:11:47<67:46:24,  8.22s/it]
 22%|██▏       | 8282/37962 [19:11:55<68:21:26,  8.29s/it]
 22%|██▏       | 8283/37962 [19:12:03<67:56:49,  8.24s/it]
 22%|██▏       | 8284/37962 [19:12:12<68:06:40,  8.26s/it]
 22%|██▏       | 8285/37962 [19:12:20<68:00:37,  8.25s/it]
 22%|██▏       | 8286/37962 [19:12:28<68:12:11,  8.27s/it]
 22%|██▏       | 8287/37962 [19:12:36<68:21:34,  8.29s/it]
 22%|██▏       | 8288/37962 [19:12:45<68:33:59,  8.32s/it]
 22%|██▏       | 8289/37962 [19:12:53<68:09:36,  8.27s/it]
 22%|██▏       | 8290/37962 [19:13:01<68:00:07,  8.25s/it]
                                                          
{'loss': 1.4727, 'grad_norm': 7.4862189292907715, 'learning_rate': 1.8195935780331128e-05, 'epoch': 0.22}

 22%|██▏       | 8290/37962 [19:13:01<68:00:07,  8.25s/it]
 22%|██▏       | 8291/37962 [19:13:09<68:02:53,  8.26s/it]
 22%|██▏       | 8292/37962 [19:13:17<67:00:05,  8.13s/it]
 22%|██▏       | 8293/37962 [19:13:26<67:33:07,  8.20s/it]
 22%|██▏       | 8294/37962 [19:13:34<67:53:03,  8.24s/it]
 22%|██▏       | 8295/37962 [19:13:42<67:19:21,  8.17s/it]
 22%|██▏       | 8296/37962 [19:13:50<67:22:09,  8.18s/it]
 22%|██▏       | 8297/37962 [19:13:59<67:49:33,  8.23s/it]
 22%|██▏       | 8298/37962 [19:14:07<67:58:12,  8.25s/it]
 22%|██▏       | 8299/37962 [19:14:15<68:18:06,  8.29s/it]
 22%|██▏       | 8300/37962 [19:14:23<68:06:29,  8.27s/it]
                                                          
{'loss': 1.3684, 'grad_norm': 6.145331859588623, 'learning_rate': 1.819104465512305e-05, 'epoch': 0.22}

 22%|██▏       | 8300/37962 [19:14:23<68:06:29,  8.27s/it]
 22%|██▏       | 8301/37962 [19:14:32<68:18:25,  8.29s/it]
 22%|██▏       | 8302/37962 [19:14:40<68:37:52,  8.33s/it]
 22%|██▏       | 8303/37962 [19:14:49<68:53:17,  8.36s/it]
 22%|██▏       | 8304/37962 [19:14:57<68:33:30,  8.32s/it]
 22%|██▏       | 8305/37962 [19:15:05<68:29:17,  8.31s/it]
 22%|██▏       | 8306/37962 [19:15:14<68:44:22,  8.34s/it]
 22%|██▏       | 8307/37962 [19:15:22<68:34:43,  8.33s/it]
 22%|██▏       | 8308/37962 [19:15:30<68:18:38,  8.29s/it]
 22%|██▏       | 8309/37962 [19:15:38<68:10:21,  8.28s/it]
 22%|██▏       | 8310/37962 [19:15:46<67:48:22,  8.23s/it]
                                                          
{'loss': 1.3679, 'grad_norm': 7.7216057777404785, 'learning_rate': 1.8186147567795462e-05, 'epoch': 0.22}

 22%|██▏       | 8310/37962 [19:15:46<67:48:22,  8.23s/it]
 22%|██▏       | 8311/37962 [19:15:55<67:52:45,  8.24s/it]
 22%|██▏       | 8312/37962 [19:16:03<68:20:37,  8.30s/it]
 22%|██▏       | 8313/37962 [19:16:11<68:03:01,  8.26s/it]
 22%|██▏       | 8314/37962 [19:16:20<67:58:33,  8.25s/it]
 22%|██▏       | 8315/37962 [19:16:28<67:14:26,  8.16s/it]
 22%|██▏       | 8316/37962 [19:16:36<67:32:21,  8.20s/it]
 22%|██▏       | 8317/37962 [19:16:44<67:57:52,  8.25s/it]
 22%|██▏       | 8318/37962 [19:16:52<67:36:47,  8.21s/it]
 22%|██▏       | 8319/37962 [19:17:01<68:10:51,  8.28s/it]
 22%|██▏       | 8320/37962 [19:17:09<68:22:11,  8.30s/it]
                                                          
{'loss': 1.4197, 'grad_norm': 7.676534175872803, 'learning_rate': 1.8181244521912874e-05, 'epoch': 0.22}

 22%|██▏       | 8320/37962 [19:17:09<68:22:11,  8.30s/it]
 22%|██▏       | 8321/37962 [19:17:17<68:02:09,  8.26s/it]
 22%|██▏       | 8322/37962 [19:17:26<68:16:17,  8.29s/it]
 22%|██▏       | 8323/37962 [19:17:34<68:12:17,  8.28s/it]
 22%|██▏       | 8324/37962 [19:17:42<68:25:20,  8.31s/it]
 22%|██▏       | 8325/37962 [19:17:50<67:53:19,  8.25s/it]
 22%|██▏       | 8326/37962 [19:17:59<67:56:12,  8.25s/it]
 22%|██▏       | 8327/37962 [19:18:07<66:59:37,  8.14s/it]
 22%|██▏       | 8328/37962 [19:18:15<67:23:36,  8.19s/it]
 22%|██▏       | 8329/37962 [19:18:23<67:45:24,  8.23s/it]
 22%|██▏       | 8330/37962 [19:18:31<66:33:18,  8.09s/it]
                                                          
{'loss': 1.3918, 'grad_norm': 5.494992256164551, 'learning_rate': 1.817633552104413e-05, 'epoch': 0.22}

 22%|██▏       | 8330/37962 [19:18:31<66:33:18,  8.09s/it]
 22%|██▏       | 8331/37962 [19:18:39<67:30:11,  8.20s/it]
 22%|██▏       | 8332/37962 [19:18:47<66:42:00,  8.10s/it]
 22%|██▏       | 8333/37962 [19:18:56<67:27:16,  8.20s/it]
 22%|██▏       | 8334/37962 [19:19:04<68:07:03,  8.28s/it]
 22%|██▏       | 8335/37962 [19:19:13<68:49:43,  8.36s/it]
 22%|██▏       | 8336/37962 [19:19:21<68:38:00,  8.34s/it]
 22%|██▏       | 8337/37962 [19:19:29<68:29:10,  8.32s/it]
 22%|██▏       | 8338/37962 [19:19:38<68:24:10,  8.31s/it]
 22%|██▏       | 8339/37962 [19:19:46<68:08:28,  8.28s/it]
 22%|██▏       | 8340/37962 [19:19:54<67:53:17,  8.25s/it]
                                                          
{'loss': 1.5457, 'grad_norm': 17.316965103149414, 'learning_rate': 1.817142056876241e-05, 'epoch': 0.22}

 22%|██▏       | 8340/37962 [19:19:54<67:53:17,  8.25s/it]
 22%|██▏       | 8341/37962 [19:20:02<67:59:23,  8.26s/it]
 22%|██▏       | 8342/37962 [19:20:10<67:45:56,  8.24s/it]
 22%|██▏       | 8343/37962 [19:20:19<67:48:11,  8.24s/it]
 22%|██▏       | 8344/37962 [19:20:27<66:54:35,  8.13s/it]
 22%|██▏       | 8345/37962 [19:20:35<67:39:19,  8.22s/it]
 22%|██▏       | 8346/37962 [19:20:43<67:00:04,  8.14s/it]
 22%|██▏       | 8347/37962 [19:20:51<67:05:26,  8.16s/it]
 22%|██▏       | 8348/37962 [19:20:59<66:41:21,  8.11s/it]
 22%|██▏       | 8349/37962 [19:21:07<66:55:18,  8.14s/it]
 22%|██▏       | 8350/37962 [19:21:16<67:27:36,  8.20s/it]
                                                          
{'loss': 1.55, 'grad_norm': 5.5368266105651855, 'learning_rate': 1.8166499668645213e-05, 'epoch': 0.22}

 22%|██▏       | 8350/37962 [19:21:16<67:27:36,  8.20s/it]
 22%|██▏       | 8351/37962 [19:21:24<67:24:02,  8.19s/it]
 22%|██▏       | 8352/37962 [19:21:32<67:32:56,  8.21s/it]
 22%|██▏       | 8353/37962 [19:21:40<67:20:13,  8.19s/it]
 22%|██▏       | 8354/37962 [19:21:48<67:22:15,  8.19s/it]
 22%|██▏       | 8355/37962 [19:21:57<67:23:25,  8.19s/it]
 22%|██▏       | 8356/37962 [19:22:05<67:18:34,  8.18s/it]
 22%|██▏       | 8357/37962 [19:22:13<67:03:17,  8.15s/it]
 22%|██▏       | 8358/37962 [19:22:21<67:52:49,  8.25s/it]
 22%|██▏       | 8359/37962 [19:22:30<68:02:02,  8.27s/it]
 22%|██▏       | 8360/37962 [19:22:38<67:33:34,  8.22s/it]
                                                          
{'loss': 1.4262, 'grad_norm': 12.39454460144043, 'learning_rate': 1.8161572824274384e-05, 'epoch': 0.22}

 22%|██▏       | 8360/37962 [19:22:38<67:33:34,  8.22s/it]
 22%|██▏       | 8361/37962 [19:22:46<68:12:42,  8.30s/it]
 22%|██▏       | 8362/37962 [19:22:54<68:00:38,  8.27s/it]
 22%|██▏       | 8363/37962 [19:23:03<67:56:20,  8.26s/it]
 22%|██▏       | 8364/37962 [19:23:11<67:53:50,  8.26s/it]
 22%|██▏       | 8365/37962 [19:23:19<68:01:10,  8.27s/it]
 22%|██▏       | 8366/37962 [19:23:27<66:45:58,  8.12s/it]
 22%|██▏       | 8367/37962 [19:23:35<67:01:54,  8.15s/it]
 22%|██▏       | 8368/37962 [19:23:43<67:17:01,  8.18s/it]
 22%|██▏       | 8369/37962 [19:23:52<66:51:38,  8.13s/it]
 22%|██▏       | 8370/37962 [19:24:00<66:54:33,  8.14s/it]
                                                          
{'loss': 1.3617, 'grad_norm': 3.8247854709625244, 'learning_rate': 1.8156640039236085e-05, 'epoch': 0.22}

 22%|██▏       | 8370/37962 [19:24:00<66:54:33,  8.14s/it]
 22%|██▏       | 8371/37962 [19:24:08<67:35:13,  8.22s/it]
 22%|██▏       | 8372/37962 [19:24:16<67:30:54,  8.21s/it]
 22%|██▏       | 8373/37962 [19:24:24<67:20:47,  8.19s/it]
 22%|██▏       | 8374/37962 [19:24:33<67:55:46,  8.27s/it]
 22%|██▏       | 8375/37962 [19:24:41<67:31:18,  8.22s/it]
 22%|██▏       | 8376/37962 [19:24:49<67:21:35,  8.20s/it]
 22%|██▏       | 8377/37962 [19:24:58<68:03:36,  8.28s/it]
 22%|██▏       | 8378/37962 [19:25:06<67:31:44,  8.22s/it]
 22%|██▏       | 8379/37962 [19:25:14<67:46:10,  8.25s/it]
 22%|██▏       | 8380/37962 [19:25:22<66:46:50,  8.13s/it]
                                                          
{'loss': 1.4845, 'grad_norm': 10.627710342407227, 'learning_rate': 1.8151701317120808e-05, 'epoch': 0.22}

 22%|██▏       | 8380/37962 [19:25:22<66:46:50,  8.13s/it]
 22%|██▏       | 8381/37962 [19:25:30<66:56:49,  8.15s/it]
 22%|██▏       | 8382/37962 [19:25:38<67:07:21,  8.17s/it]
 22%|██▏       | 8383/37962 [19:25:46<67:12:28,  8.18s/it]
 22%|██▏       | 8384/37962 [19:25:55<67:39:37,  8.24s/it]
 22%|██▏       | 8385/37962 [19:26:03<67:49:15,  8.25s/it]
 22%|██▏       | 8386/37962 [19:26:12<68:30:44,  8.34s/it]
 22%|██▏       | 8387/37962 [19:26:20<68:09:50,  8.30s/it]
 22%|██▏       | 8388/37962 [19:26:28<68:21:22,  8.32s/it]
 22%|██▏       | 8389/37962 [19:26:36<68:09:16,  8.30s/it]
 22%|██▏       | 8390/37962 [19:26:45<67:55:23,  8.27s/it]
                                                          
{'loss': 1.4815, 'grad_norm': 4.776463508605957, 'learning_rate': 1.814675666152336e-05, 'epoch': 0.22}

 22%|██▏       | 8390/37962 [19:26:45<67:55:23,  8.27s/it]
 22%|██▏       | 8391/37962 [19:26:53<67:59:47,  8.28s/it]
 22%|██▏       | 8392/37962 [19:27:01<67:54:43,  8.27s/it]
 22%|██▏       | 8393/37962 [19:27:09<67:32:30,  8.22s/it]
 22%|██▏       | 8394/37962 [19:27:18<68:05:39,  8.29s/it]
 22%|██▏       | 8395/37962 [19:27:26<67:57:53,  8.28s/it]
 22%|██▏       | 8396/37962 [19:27:35<68:39:15,  8.36s/it]
 22%|██▏       | 8397/37962 [19:27:43<68:00:17,  8.28s/it]
 22%|██▏       | 8398/37962 [19:27:51<67:29:53,  8.22s/it]
 22%|██▏       | 8399/37962 [19:27:59<67:37:59,  8.24s/it]
 22%|██▏       | 8400/37962 [19:28:07<68:10:54,  8.30s/it]
                                                          
{'loss': 1.3421, 'grad_norm': 9.285021781921387, 'learning_rate': 1.814180607604287e-05, 'epoch': 0.22}

 22%|██▏       | 8400/37962 [19:28:07<68:10:54,  8.30s/it]
 22%|██▏       | 8401/37962 [19:28:16<68:34:41,  8.35s/it]
 22%|██▏       | 8402/37962 [19:28:24<68:18:21,  8.32s/it]
 22%|██▏       | 8403/37962 [19:28:32<68:13:35,  8.31s/it]
 22%|██▏       | 8404/37962 [19:28:41<67:56:31,  8.27s/it]
 22%|██▏       | 8405/37962 [19:28:49<67:47:14,  8.26s/it]
 22%|██▏       | 8406/37962 [19:28:57<67:38:45,  8.24s/it]
 22%|██▏       | 8407/37962 [19:29:05<66:58:53,  8.16s/it]
 22%|██▏       | 8408/37962 [19:29:13<66:59:49,  8.16s/it]
 22%|██▏       | 8409/37962 [19:29:22<67:43:29,  8.25s/it]
 22%|██▏       | 8410/37962 [19:29:30<68:20:10,  8.32s/it]
                                                          
{'loss': 1.5045, 'grad_norm': 9.285212516784668, 'learning_rate': 1.8136849564282785e-05, 'epoch': 0.22}

 22%|██▏       | 8410/37962 [19:29:30<68:20:10,  8.32s/it]
 22%|██▏       | 8411/37962 [19:29:38<68:20:40,  8.33s/it]
 22%|██▏       | 8412/37962 [19:29:47<68:21:19,  8.33s/it]
 22%|██▏       | 8413/37962 [19:29:55<68:47:30,  8.38s/it]
 22%|██▏       | 8414/37962 [19:30:04<68:56:10,  8.40s/it]
 22%|██▏       | 8415/37962 [19:30:12<68:31:06,  8.35s/it]
 22%|██▏       | 8416/37962 [19:30:20<68:11:03,  8.31s/it]
 22%|██▏       | 8417/37962 [19:30:29<68:57:17,  8.40s/it]
 22%|██▏       | 8418/37962 [19:30:37<67:58:25,  8.28s/it]
 22%|██▏       | 8419/37962 [19:30:45<68:31:29,  8.35s/it]
 22%|██▏       | 8420/37962 [19:30:54<68:18:43,  8.32s/it]
                                                          
{'loss': 1.345, 'grad_norm': 5.17016077041626, 'learning_rate': 1.8131887129850862e-05, 'epoch': 0.22}

 22%|██▏       | 8420/37962 [19:30:54<68:18:43,  8.32s/it]
 22%|██▏       | 8421/37962 [19:31:02<68:33:03,  8.35s/it]
 22%|██▏       | 8422/37962 [19:31:10<68:12:41,  8.31s/it]
 22%|██▏       | 8423/37962 [19:31:18<68:02:49,  8.29s/it]
 22%|██▏       | 8424/37962 [19:31:27<67:57:03,  8.28s/it]
 22%|██▏       | 8425/37962 [19:31:35<67:58:20,  8.28s/it]
 22%|██▏       | 8426/37962 [19:31:43<67:59:54,  8.29s/it]
 22%|██▏       | 8427/37962 [19:31:52<67:52:14,  8.27s/it]
 22%|██▏       | 8428/37962 [19:32:00<67:17:08,  8.20s/it]
 22%|██▏       | 8429/37962 [19:32:08<67:44:25,  8.26s/it]
 22%|██▏       | 8430/37962 [19:32:16<67:53:43,  8.28s/it]
                                                          
{'loss': 1.4535, 'grad_norm': 3.511632204055786, 'learning_rate': 1.812691877635917e-05, 'epoch': 0.22}

 22%|██▏       | 8430/37962 [19:32:16<67:53:43,  8.28s/it]
 22%|██▏       | 8431/37962 [19:32:25<68:03:00,  8.30s/it]
 22%|██▏       | 8432/37962 [19:32:33<68:01:15,  8.29s/it]
 22%|██▏       | 8433/37962 [19:32:41<67:23:31,  8.22s/it]
 22%|██▏       | 8434/37962 [19:32:49<67:30:14,  8.23s/it]
 22%|██▏       | 8435/37962 [19:32:57<67:29:33,  8.23s/it]
 22%|██▏       | 8436/37962 [19:33:05<66:56:51,  8.16s/it]
 22%|██▏       | 8437/37962 [19:33:14<66:58:31,  8.17s/it]
 22%|██▏       | 8438/37962 [19:33:22<67:48:51,  8.27s/it]
 22%|██▏       | 8439/37962 [19:33:30<67:18:05,  8.21s/it]
 22%|██▏       | 8440/37962 [19:33:39<67:56:06,  8.28s/it]
                                                          
{'loss': 1.4652, 'grad_norm': 13.215763092041016, 'learning_rate': 1.8121944507424093e-05, 'epoch': 0.22}

 22%|██▏       | 8440/37962 [19:33:39<67:56:06,  8.28s/it]
 22%|██▏       | 8441/37962 [19:33:47<67:33:20,  8.24s/it]
 22%|██▏       | 8442/37962 [19:33:55<67:47:53,  8.27s/it]
 22%|██▏       | 8443/37962 [19:34:04<68:40:08,  8.37s/it]
 22%|██▏       | 8444/37962 [19:34:12<68:04:09,  8.30s/it]
 22%|██▏       | 8445/37962 [19:34:20<67:32:29,  8.24s/it]
 22%|██▏       | 8446/37962 [19:34:28<67:22:06,  8.22s/it]
 22%|██▏       | 8447/37962 [19:34:43<84:37:00, 10.32s/it]
 22%|██▏       | 8448/37962 [19:34:52<79:19:27,  9.68s/it]
 22%|██▏       | 8449/37962 [19:35:00<75:58:24,  9.27s/it]
 22%|██▏       | 8450/37962 [19:35:08<74:06:46,  9.04s/it]
                                                          
{'loss': 1.4009, 'grad_norm': 22.14901351928711, 'learning_rate': 1.8116964326666314e-05, 'epoch': 0.22}

 22%|██▏       | 8450/37962 [19:35:08<74:06:46,  9.04s/it]
 22%|██▏       | 8451/37962 [19:35:17<72:26:11,  8.84s/it]
 22%|██▏       | 8452/37962 [19:35:25<70:33:08,  8.61s/it]
 22%|██▏       | 8453/37962 [19:35:33<69:49:36,  8.52s/it]
 22%|██▏       | 8454/37962 [19:35:41<68:29:14,  8.36s/it]
 22%|██▏       | 8455/37962 [19:35:49<68:02:42,  8.30s/it]
 22%|██▏       | 8456/37962 [19:35:57<67:48:18,  8.27s/it]
 22%|██▏       | 8457/37962 [19:36:06<67:58:42,  8.29s/it]
 22%|██▏       | 8458/37962 [19:36:14<67:56:07,  8.29s/it]
 22%|██▏       | 8459/37962 [19:36:22<67:50:31,  8.28s/it]
 22%|██▏       | 8460/37962 [19:36:31<68:01:25,  8.30s/it]
                                                          
{'loss': 1.4193, 'grad_norm': 6.4849653244018555, 'learning_rate': 1.8111978237710817e-05, 'epoch': 0.22}

 22%|██▏       | 8460/37962 [19:36:31<68:01:25,  8.30s/it]
 22%|██▏       | 8461/37962 [19:36:39<67:52:49,  8.28s/it]
 22%|██▏       | 8462/37962 [19:36:47<68:10:23,  8.32s/it]
 22%|██▏       | 8463/37962 [19:36:56<68:15:24,  8.33s/it]
 22%|██▏       | 8464/37962 [19:37:04<67:32:36,  8.24s/it]
 22%|██▏       | 8465/37962 [19:37:12<67:32:48,  8.24s/it]
 22%|██▏       | 8466/37962 [19:37:20<67:44:01,  8.27s/it]
 22%|██▏       | 8467/37962 [19:37:29<68:11:45,  8.32s/it]
 22%|██▏       | 8468/37962 [19:37:37<67:48:51,  8.28s/it]
 22%|██▏       | 8469/37962 [19:37:45<67:46:51,  8.27s/it]
 22%|██▏       | 8470/37962 [19:37:54<67:54:09,  8.29s/it]
                                                          
{'loss': 1.3861, 'grad_norm': 4.807112216949463, 'learning_rate': 1.810698624418689e-05, 'epoch': 0.22}

 22%|██▏       | 8470/37962 [19:37:54<67:54:09,  8.29s/it]
 22%|██▏       | 8471/37962 [19:38:02<68:02:09,  8.31s/it]
 22%|██▏       | 8472/37962 [19:38:10<67:53:00,  8.29s/it]
 22%|██▏       | 8473/37962 [19:38:18<67:52:52,  8.29s/it]
 22%|██▏       | 8474/37962 [19:38:26<66:42:15,  8.14s/it]
 22%|██▏       | 8475/37962 [19:38:35<67:28:33,  8.24s/it]
 22%|██▏       | 8476/37962 [19:38:43<67:03:46,  8.19s/it]
 22%|██▏       | 8477/37962 [19:38:51<66:47:36,  8.16s/it]
 22%|██▏       | 8478/37962 [19:38:59<66:56:47,  8.17s/it]
 22%|██▏       | 8479/37962 [19:39:07<67:18:41,  8.22s/it]
 22%|██▏       | 8480/37962 [19:39:16<67:46:49,  8.28s/it]
                                                          
{'loss': 1.4903, 'grad_norm': 12.223381996154785, 'learning_rate': 1.810198834972812e-05, 'epoch': 0.22}

 22%|██▏       | 8480/37962 [19:39:16<67:46:49,  8.28s/it]
 22%|██▏       | 8481/37962 [19:39:24<67:14:38,  8.21s/it]
 22%|██▏       | 8482/37962 [19:39:32<67:37:57,  8.26s/it]
 22%|██▏       | 8483/37962 [19:39:40<67:17:15,  8.22s/it]
 22%|██▏       | 8484/37962 [19:39:49<67:52:30,  8.29s/it]
 22%|██▏       | 8485/37962 [19:39:57<66:52:34,  8.17s/it]
 22%|██▏       | 8486/37962 [19:40:05<66:20:02,  8.10s/it]
 22%|██▏       | 8487/37962 [19:40:13<66:47:09,  8.16s/it]
 22%|██▏       | 8488/37962 [19:40:21<67:08:42,  8.20s/it]
 22%|██▏       | 8489/37962 [19:40:30<67:29:34,  8.24s/it]
 22%|██▏       | 8490/37962 [19:40:38<67:17:46,  8.22s/it]
                                                          
{'loss': 1.3941, 'grad_norm': 6.607932090759277, 'learning_rate': 1.809698455797239e-05, 'epoch': 0.22}

 22%|██▏       | 8490/37962 [19:40:38<67:17:46,  8.22s/it]
 22%|██▏       | 8491/37962 [19:40:46<67:09:11,  8.20s/it]
 22%|██▏       | 8492/37962 [19:40:54<67:55:40,  8.30s/it]
 22%|██▏       | 8493/37962 [19:41:03<67:52:09,  8.29s/it]
 22%|██▏       | 8494/37962 [19:41:11<67:49:53,  8.29s/it]
 22%|██▏       | 8495/37962 [19:41:19<67:49:19,  8.29s/it]
 22%|██▏       | 8496/37962 [19:41:27<67:24:19,  8.24s/it]
 22%|██▏       | 8497/37962 [19:41:35<67:00:39,  8.19s/it]
 22%|██▏       | 8498/37962 [19:41:44<67:05:00,  8.20s/it]
 22%|██▏       | 8499/37962 [19:41:51<65:47:13,  8.04s/it]
 22%|██▏       | 8500/37962 [19:42:00<66:22:52,  8.11s/it]
                                                          
{'loss': 1.4079, 'grad_norm': 11.637933731079102, 'learning_rate': 1.8091974872561868e-05, 'epoch': 0.22}

 22%|██▏       | 8500/37962 [19:42:00<66:22:52,  8.11s/it]
 22%|██▏       | 8501/37962 [19:42:08<66:18:01,  8.10s/it]
 22%|██▏       | 8502/37962 [19:42:16<66:47:49,  8.16s/it]
 22%|██▏       | 8503/37962 [19:42:24<66:59:19,  8.19s/it]
 22%|██▏       | 8504/37962 [19:42:32<67:02:31,  8.19s/it]
 22%|██▏       | 8505/37962 [19:42:41<67:20:26,  8.23s/it]
 22%|██▏       | 8506/37962 [19:42:49<67:05:40,  8.20s/it]
 22%|██▏       | 8507/37962 [19:42:57<66:50:32,  8.17s/it]
 22%|██▏       | 8508/37962 [19:43:05<67:01:48,  8.19s/it]
 22%|██▏       | 8509/37962 [19:43:14<67:35:29,  8.26s/it]
 22%|██▏       | 8510/37962 [19:43:22<67:30:57,  8.25s/it]
                                                          
{'loss': 1.4067, 'grad_norm': 11.12977409362793, 'learning_rate': 1.808695929714302e-05, 'epoch': 0.22}

 22%|██▏       | 8510/37962 [19:43:22<67:30:57,  8.25s/it]
 22%|██▏       | 8511/37962 [19:43:30<67:36:02,  8.26s/it]
 22%|██▏       | 8512/37962 [19:43:39<67:57:07,  8.31s/it]
 22%|██▏       | 8513/37962 [19:43:47<67:56:40,  8.31s/it]
 22%|██▏       | 8514/37962 [19:43:55<67:24:07,  8.24s/it]
 22%|██▏       | 8515/37962 [19:44:03<67:29:35,  8.25s/it]
 22%|██▏       | 8516/37962 [19:44:11<67:13:17,  8.22s/it]
 22%|██▏       | 8517/37962 [19:44:20<67:36:20,  8.27s/it]
 22%|██▏       | 8518/37962 [19:44:28<67:29:21,  8.25s/it]
 22%|██▏       | 8519/37962 [19:44:36<67:30:51,  8.25s/it]
 22%|██▏       | 8520/37962 [19:44:44<66:44:55,  8.16s/it]
                                                          
{'loss': 1.4737, 'grad_norm': 8.856128692626953, 'learning_rate': 1.8081937835366598e-05, 'epoch': 0.22}

 22%|██▏       | 8520/37962 [19:44:44<66:44:55,  8.16s/it]
 22%|██▏       | 8521/37962 [19:44:52<65:46:59,  8.04s/it]
 22%|██▏       | 8522/37962 [19:45:00<66:04:24,  8.08s/it]
 22%|██▏       | 8523/37962 [19:45:09<67:01:25,  8.20s/it]
 22%|██▏       | 8524/37962 [19:45:17<66:54:22,  8.18s/it]
 22%|██▏       | 8525/37962 [19:45:25<66:59:07,  8.19s/it]
 22%|██▏       | 8526/37962 [19:45:33<67:16:52,  8.23s/it]
 22%|██▏       | 8527/37962 [19:45:41<66:38:20,  8.15s/it]
 22%|██▏       | 8528/37962 [19:45:49<66:49:11,  8.17s/it]
 22%|██▏       | 8529/37962 [19:45:57<66:09:19,  8.09s/it]
 22%|██▏       | 8530/37962 [19:46:06<67:03:32,  8.20s/it]
                                                          
{'loss': 1.4251, 'grad_norm': 6.893471717834473, 'learning_rate': 1.8076910490887635e-05, 'epoch': 0.22}

 22%|██▏       | 8530/37962 [19:46:06<67:03:32,  8.20s/it]
 22%|██▏       | 8531/37962 [19:46:14<67:29:48,  8.26s/it]
 22%|██▏       | 8532/37962 [19:46:23<67:37:41,  8.27s/it]
 22%|██▏       | 8533/37962 [19:46:31<67:48:18,  8.29s/it]
 22%|██▏       | 8534/37962 [19:46:39<67:18:51,  8.23s/it]
 22%|██▏       | 8535/37962 [19:46:47<67:33:09,  8.26s/it]
 22%|██▏       | 8536/37962 [19:46:55<67:21:31,  8.24s/it]
 22%|██▏       | 8537/37962 [19:47:04<66:58:06,  8.19s/it]
 22%|██▏       | 8538/37962 [19:47:12<66:56:57,  8.19s/it]
 22%|██▏       | 8539/37962 [19:47:20<65:59:23,  8.07s/it]
 22%|██▏       | 8540/37962 [19:47:28<66:40:41,  8.16s/it]
                                                          
{'loss': 1.346, 'grad_norm': 6.610567092895508, 'learning_rate': 1.8071877267365448e-05, 'epoch': 0.22}

 22%|██▏       | 8540/37962 [19:47:28<66:40:41,  8.16s/it]
 22%|██▏       | 8541/37962 [19:47:36<67:04:31,  8.21s/it]
 23%|██▎       | 8542/37962 [19:47:44<66:37:10,  8.15s/it]
 23%|██▎       | 8543/37962 [19:47:52<66:16:58,  8.11s/it]
 23%|██▎       | 8544/37962 [19:48:01<66:43:02,  8.16s/it]
 23%|██▎       | 8545/37962 [19:48:09<67:03:08,  8.21s/it]
 23%|██▎       | 8546/37962 [19:48:17<67:00:46,  8.20s/it]
 23%|██▎       | 8547/37962 [19:48:25<66:56:09,  8.19s/it]
 23%|██▎       | 8548/37962 [19:48:33<66:38:16,  8.16s/it]
 23%|██▎       | 8549/37962 [19:48:42<67:16:07,  8.23s/it]
 23%|██▎       | 8550/37962 [19:48:50<67:40:49,  8.28s/it]
                                                          
{'loss': 1.426, 'grad_norm': 5.463042736053467, 'learning_rate': 1.8066838168463637e-05, 'epoch': 0.23}

 23%|██▎       | 8550/37962 [19:48:50<67:40:49,  8.28s/it]
 23%|██▎       | 8551/37962 [19:48:58<67:23:44,  8.25s/it]
 23%|██▎       | 8552/37962 [19:49:07<67:34:16,  8.27s/it]
 23%|██▎       | 8553/37962 [19:49:15<67:18:49,  8.24s/it]
 23%|██▎       | 8554/37962 [19:49:23<67:13:26,  8.23s/it]
 23%|██▎       | 8555/37962 [19:49:31<67:22:24,  8.25s/it]
 23%|██▎       | 8556/37962 [19:49:39<67:20:17,  8.24s/it]
 23%|██▎       | 8557/37962 [19:49:48<67:38:28,  8.28s/it]
 23%|██▎       | 8558/37962 [19:49:56<68:06:00,  8.34s/it]
 23%|██▎       | 8559/37962 [19:50:05<67:58:10,  8.32s/it]
 23%|██▎       | 8560/37962 [19:50:13<67:04:01,  8.21s/it]
                                                          
{'loss': 1.4808, 'grad_norm': 8.718052864074707, 'learning_rate': 1.806179319785007e-05, 'epoch': 0.23}

 23%|██▎       | 8560/37962 [19:50:13<67:04:01,  8.21s/it]
 23%|██▎       | 8561/37962 [19:50:21<67:12:31,  8.23s/it]
 23%|██▎       | 8562/37962 [19:50:29<67:24:16,  8.25s/it]
 23%|██▎       | 8563/37962 [19:50:37<67:13:56,  8.23s/it]
 23%|██▎       | 8564/37962 [19:50:46<67:15:40,  8.24s/it]
 23%|██▎       | 8565/37962 [19:50:54<67:38:42,  8.28s/it]
 23%|██▎       | 8566/37962 [19:51:03<68:19:33,  8.37s/it]
 23%|██▎       | 8567/37962 [19:51:11<68:13:02,  8.35s/it]
 23%|██▎       | 8568/37962 [19:51:19<68:22:32,  8.37s/it]
 23%|██▎       | 8569/37962 [19:51:28<68:41:30,  8.41s/it]
 23%|██▎       | 8570/37962 [19:51:36<68:48:11,  8.43s/it]
                                                          
{'loss': 1.3365, 'grad_norm': 2.978691577911377, 'learning_rate': 1.805674235919689e-05, 'epoch': 0.23}

 23%|██▎       | 8570/37962 [19:51:36<68:48:11,  8.43s/it]
 23%|██▎       | 8571/37962 [19:51:44<68:11:08,  8.35s/it]
 23%|██▎       | 8572/37962 [19:51:53<67:43:53,  8.30s/it]
 23%|██▎       | 8573/37962 [19:52:01<67:49:57,  8.31s/it]
 23%|██▎       | 8574/37962 [19:52:09<67:34:17,  8.28s/it]
 23%|██▎       | 8575/37962 [19:52:17<67:37:37,  8.28s/it]
 23%|██▎       | 8576/37962 [19:52:26<67:24:53,  8.26s/it]
 23%|██▎       | 8577/37962 [19:52:34<67:34:02,  8.28s/it]
 23%|██▎       | 8578/37962 [19:52:42<67:35:13,  8.28s/it]
 23%|██▎       | 8579/37962 [19:52:50<66:28:31,  8.14s/it]
 23%|██▎       | 8580/37962 [19:52:58<67:08:09,  8.23s/it]
                                                          
{'loss': 1.4732, 'grad_norm': 3.123349666595459, 'learning_rate': 1.8051685656180528e-05, 'epoch': 0.23}

 23%|██▎       | 8580/37962 [19:52:58<67:08:09,  8.23s/it]
 23%|██▎       | 8581/37962 [19:53:07<67:54:57,  8.32s/it]
 23%|██▎       | 8582/37962 [19:53:15<67:34:09,  8.28s/it]
 23%|██▎       | 8583/37962 [19:53:23<67:29:08,  8.27s/it]
 23%|██▎       | 8584/37962 [19:53:32<67:10:35,  8.23s/it]
 23%|██▎       | 8585/37962 [19:53:40<67:08:24,  8.23s/it]
 23%|██▎       | 8586/37962 [19:53:48<67:25:50,  8.26s/it]
 23%|██▎       | 8587/37962 [19:53:57<67:45:18,  8.30s/it]
 23%|██▎       | 8588/37962 [19:54:05<67:30:42,  8.27s/it]
 23%|██▎       | 8589/37962 [19:54:13<67:21:14,  8.25s/it]
 23%|██▎       | 8590/37962 [19:54:21<67:29:02,  8.27s/it]
                                                          
{'loss': 1.4053, 'grad_norm': 14.050543785095215, 'learning_rate': 1.804662309248166e-05, 'epoch': 0.23}

 23%|██▎       | 8590/37962 [19:54:21<67:29:02,  8.27s/it]
 23%|██▎       | 8591/37962 [19:54:30<67:36:55,  8.29s/it]
 23%|██▎       | 8592/37962 [19:54:38<67:35:06,  8.28s/it]
 23%|██▎       | 8593/37962 [19:54:46<67:31:54,  8.28s/it]
 23%|██▎       | 8594/37962 [19:54:55<67:46:36,  8.31s/it]
 23%|██▎       | 8595/37962 [19:55:03<68:05:08,  8.35s/it]
 23%|██▎       | 8596/37962 [19:55:11<67:48:46,  8.31s/it]
 23%|██▎       | 8597/37962 [19:55:19<66:57:03,  8.21s/it]
 23%|██▎       | 8598/37962 [19:55:28<67:27:38,  8.27s/it]
 23%|██▎       | 8599/37962 [19:55:36<67:47:49,  8.31s/it]
 23%|██▎       | 8600/37962 [19:55:44<68:07:15,  8.35s/it]
                                                          
{'loss': 1.5644, 'grad_norm': 15.999217987060547, 'learning_rate': 1.8041554671785237e-05, 'epoch': 0.23}

 23%|██▎       | 8600/37962 [19:55:44<68:07:15,  8.35s/it]
 23%|██▎       | 8601/37962 [19:55:53<67:44:18,  8.31s/it]
 23%|██▎       | 8602/37962 [19:56:01<67:36:51,  8.29s/it]
 23%|██▎       | 8603/37962 [19:56:09<67:05:18,  8.23s/it]
 23%|██▎       | 8604/37962 [19:56:17<66:53:09,  8.20s/it]
 23%|██▎       | 8605/37962 [19:56:25<67:14:46,  8.25s/it]
 23%|██▎       | 8606/37962 [19:56:34<67:32:34,  8.28s/it]
 23%|██▎       | 8607/37962 [19:56:42<67:43:18,  8.31s/it]
 23%|██▎       | 8608/37962 [19:56:51<67:50:49,  8.32s/it]
 23%|██▎       | 8609/37962 [19:56:59<67:24:05,  8.27s/it]
 23%|██▎       | 8610/37962 [19:57:07<67:31:38,  8.28s/it]
                                                          
{'loss': 1.3877, 'grad_norm': 3.995640516281128, 'learning_rate': 1.8036480397780483e-05, 'epoch': 0.23}

 23%|██▎       | 8610/37962 [19:57:07<67:31:38,  8.28s/it]
 23%|██▎       | 8611/37962 [19:57:15<67:46:34,  8.31s/it]
 23%|██▎       | 8612/37962 [19:57:24<68:08:33,  8.36s/it]
 23%|██▎       | 8613/37962 [19:57:32<67:54:06,  8.33s/it]
 23%|██▎       | 8614/37962 [19:57:40<67:25:12,  8.27s/it]
 23%|██▎       | 8615/37962 [19:57:48<66:52:36,  8.20s/it]
 23%|██▎       | 8616/37962 [19:57:57<67:15:37,  8.25s/it]
 23%|██▎       | 8617/37962 [19:58:05<67:32:47,  8.29s/it]
 23%|██▎       | 8618/37962 [19:58:13<67:35:37,  8.29s/it]
 23%|██▎       | 8619/37962 [19:58:21<67:01:10,  8.22s/it]
 23%|██▎       | 8620/37962 [19:58:30<67:06:48,  8.23s/it]
                                                          
{'loss': 1.356, 'grad_norm': 10.210978507995605, 'learning_rate': 1.803140027416087e-05, 'epoch': 0.23}

 23%|██▎       | 8620/37962 [19:58:30<67:06:48,  8.23s/it]
 23%|██▎       | 8621/37962 [19:58:38<67:09:59,  8.24s/it]
 23%|██▎       | 8622/37962 [19:58:46<67:15:23,  8.25s/it]
 23%|██▎       | 8623/37962 [19:58:54<66:10:48,  8.12s/it]
 23%|██▎       | 8624/37962 [19:59:02<66:56:30,  8.21s/it]
 23%|██▎       | 8625/37962 [19:59:11<67:17:42,  8.26s/it]
 23%|██▎       | 8626/37962 [19:59:19<66:43:49,  8.19s/it]
 23%|██▎       | 8627/37962 [19:59:27<65:54:24,  8.09s/it]
 23%|██▎       | 8628/37962 [19:59:35<65:28:41,  8.04s/it]
 23%|██▎       | 8629/37962 [19:59:43<66:14:11,  8.13s/it]
 23%|██▎       | 8630/37962 [19:59:51<66:27:53,  8.16s/it]
                                                          
{'loss': 1.36, 'grad_norm': 22.377599716186523, 'learning_rate': 1.8026314304624133e-05, 'epoch': 0.23}

 23%|██▎       | 8630/37962 [19:59:51<66:27:53,  8.16s/it]
 23%|██▎       | 8631/37962 [19:59:59<66:46:35,  8.20s/it]
 23%|██▎       | 8632/37962 [20:00:07<65:53:24,  8.09s/it]
 23%|██▎       | 8633/37962 [20:00:15<65:50:28,  8.08s/it]
 23%|██▎       | 8634/37962 [20:00:24<66:02:23,  8.11s/it]
 23%|██▎       | 8635/37962 [20:00:32<65:57:38,  8.10s/it]
 23%|██▎       | 8636/37962 [20:00:40<66:16:49,  8.14s/it]
 23%|██▎       | 8637/37962 [20:00:48<66:29:14,  8.16s/it]
 23%|██▎       | 8638/37962 [20:00:56<67:05:53,  8.24s/it]
 23%|██▎       | 8639/37962 [20:01:05<67:42:47,  8.31s/it]
 23%|██▎       | 8640/37962 [20:01:13<67:26:24,  8.28s/it]
                                                          
{'loss': 1.4718, 'grad_norm': 7.86072301864624, 'learning_rate': 1.802122249287226e-05, 'epoch': 0.23}

 23%|██▎       | 8640/37962 [20:01:13<67:26:24,  8.28s/it]
 23%|██▎       | 8641/37962 [20:01:22<67:39:19,  8.31s/it]
 23%|██▎       | 8642/37962 [20:01:30<67:27:49,  8.28s/it]
 23%|██▎       | 8643/37962 [20:01:38<67:29:30,  8.29s/it]
 23%|██▎       | 8644/37962 [20:01:46<67:37:37,  8.30s/it]
 23%|██▎       | 8645/37962 [20:01:55<67:43:30,  8.32s/it]
 23%|██▎       | 8646/37962 [20:02:10<84:15:03, 10.35s/it]
 23%|██▎       | 8647/37962 [20:02:18<78:18:47,  9.62s/it]
 23%|██▎       | 8648/37962 [20:02:26<74:16:21,  9.12s/it]
 23%|██▎       | 8649/37962 [20:02:34<72:33:52,  8.91s/it]
 23%|██▎       | 8650/37962 [20:02:42<69:33:51,  8.54s/it]
                                                          
{'loss': 1.5526, 'grad_norm': 9.0934476852417, 'learning_rate': 1.801612484261149e-05, 'epoch': 0.23}

 23%|██▎       | 8650/37962 [20:02:42<69:33:51,  8.54s/it]
 23%|██▎       | 8651/37962 [20:02:50<68:39:32,  8.43s/it]
 23%|██▎       | 8652/37962 [20:02:58<67:44:43,  8.32s/it]
 23%|██▎       | 8653/37962 [20:03:06<67:58:39,  8.35s/it]
 23%|██▎       | 8654/37962 [20:03:15<68:17:49,  8.39s/it]
 23%|██▎       | 8655/37962 [20:03:23<67:52:46,  8.34s/it]
 23%|██▎       | 8656/37962 [20:03:31<67:43:00,  8.32s/it]
 23%|██▎       | 8657/37962 [20:03:40<67:51:59,  8.34s/it]
 23%|██▎       | 8658/37962 [20:03:48<67:46:26,  8.33s/it]
 23%|██▎       | 8659/37962 [20:03:56<67:11:48,  8.26s/it]
 23%|██▎       | 8660/37962 [20:04:04<67:12:34,  8.26s/it]
                                                          
{'loss': 1.4796, 'grad_norm': 8.159210205078125, 'learning_rate': 1.8011021357552317e-05, 'epoch': 0.23}

 23%|██▎       | 8660/37962 [20:04:04<67:12:34,  8.26s/it]
 23%|██▎       | 8661/37962 [20:04:13<67:27:34,  8.29s/it]
 23%|██▎       | 8662/37962 [20:04:21<67:14:24,  8.26s/it]
 23%|██▎       | 8663/37962 [20:04:36<83:58:09, 10.32s/it]
 23%|██▎       | 8664/37962 [20:04:44<78:41:40,  9.67s/it]
 23%|██▎       | 8665/37962 [20:04:53<75:24:38,  9.27s/it]
 23%|██▎       | 8666/37962 [20:05:00<71:56:34,  8.84s/it]
 23%|██▎       | 8667/37962 [20:05:09<70:51:13,  8.71s/it]
 23%|██▎       | 8668/37962 [20:05:17<69:48:26,  8.58s/it]
 23%|██▎       | 8669/37962 [20:05:25<68:59:49,  8.48s/it]
 23%|██▎       | 8670/37962 [20:05:33<67:07:47,  8.25s/it]
                                                          
{'loss': 1.4665, 'grad_norm': 2.4113881587982178, 'learning_rate': 1.800591204140948e-05, 'epoch': 0.23}

 23%|██▎       | 8670/37962 [20:05:33<67:07:47,  8.25s/it]
 23%|██▎       | 8671/37962 [20:05:41<67:27:03,  8.29s/it]
 23%|██▎       | 8672/37962 [20:05:50<67:33:19,  8.30s/it]
 23%|██▎       | 8673/37962 [20:05:58<68:06:42,  8.37s/it]
 23%|██▎       | 8674/37962 [20:06:07<68:26:10,  8.41s/it]
 23%|██▎       | 8675/37962 [20:06:15<68:23:26,  8.41s/it]
 23%|██▎       | 8676/37962 [20:06:24<68:19:29,  8.40s/it]
 23%|██▎       | 8677/37962 [20:06:32<67:40:24,  8.32s/it]
 23%|██▎       | 8678/37962 [20:06:40<67:44:49,  8.33s/it]
 23%|██▎       | 8679/37962 [20:06:48<67:46:56,  8.33s/it]
 23%|██▎       | 8680/37962 [20:06:57<67:47:44,  8.33s/it]
                                                          
{'loss': 1.4429, 'grad_norm': 13.823946952819824, 'learning_rate': 1.800079689790196e-05, 'epoch': 0.23}

 23%|██▎       | 8680/37962 [20:06:57<67:47:44,  8.33s/it]
 23%|██▎       | 8681/37962 [20:07:05<67:55:36,  8.35s/it]
 23%|██▎       | 8682/37962 [20:07:14<68:09:40,  8.38s/it]
 23%|██▎       | 8683/37962 [20:07:22<67:26:44,  8.29s/it]
 23%|██▎       | 8684/37962 [20:07:30<67:12:25,  8.26s/it]
 23%|██▎       | 8685/37962 [20:07:38<67:26:36,  8.29s/it]
 23%|██▎       | 8686/37962 [20:07:47<67:26:07,  8.29s/it]
 23%|██▎       | 8687/37962 [20:07:55<68:11:23,  8.39s/it]
 23%|██▎       | 8688/37962 [20:08:03<67:49:15,  8.34s/it]
 23%|██▎       | 8689/37962 [20:08:12<67:31:00,  8.30s/it]
 23%|██▎       | 8690/37962 [20:08:20<67:55:49,  8.35s/it]
                                                          
{'loss': 1.4694, 'grad_norm': 8.707887649536133, 'learning_rate': 1.7995675930752985e-05, 'epoch': 0.23}

 23%|██▎       | 8690/37962 [20:08:20<67:55:49,  8.35s/it]
 23%|██▎       | 8691/37962 [20:08:28<67:33:04,  8.31s/it]
 23%|██▎       | 8692/37962 [20:08:36<66:44:11,  8.21s/it]
 23%|██▎       | 8693/37962 [20:08:45<66:48:38,  8.22s/it]
 23%|██▎       | 8694/37962 [20:08:53<67:04:16,  8.25s/it]
 23%|██▎       | 8695/37962 [20:09:01<66:34:57,  8.19s/it]
 23%|██▎       | 8696/37962 [20:09:09<66:32:07,  8.18s/it]
 23%|██▎       | 8697/37962 [20:09:17<66:40:29,  8.20s/it]
 23%|██▎       | 8698/37962 [20:09:25<66:03:36,  8.13s/it]
 23%|██▎       | 8699/37962 [20:09:33<66:22:00,  8.16s/it]
 23%|██▎       | 8700/37962 [20:09:42<66:22:46,  8.17s/it]
                                                          
{'loss': 1.4056, 'grad_norm': 5.312180042266846, 'learning_rate': 1.7990549143690013e-05, 'epoch': 0.23}

 23%|██▎       | 8700/37962 [20:09:42<66:22:46,  8.17s/it]
 23%|██▎       | 8701/37962 [20:09:50<66:16:22,  8.15s/it]
 23%|██▎       | 8702/37962 [20:09:58<66:27:34,  8.18s/it]
 23%|██▎       | 8703/37962 [20:10:06<66:55:12,  8.23s/it]
 23%|██▎       | 8704/37962 [20:10:15<66:55:24,  8.23s/it]
 23%|██▎       | 8705/37962 [20:10:23<67:08:21,  8.26s/it]
 23%|██▎       | 8706/37962 [20:10:31<67:07:13,  8.26s/it]
 23%|██▎       | 8707/37962 [20:10:40<67:39:27,  8.33s/it]
 23%|██▎       | 8708/37962 [20:10:48<67:14:25,  8.27s/it]
 23%|██▎       | 8709/37962 [20:10:56<67:08:28,  8.26s/it]
 23%|██▎       | 8710/37962 [20:11:04<67:31:23,  8.31s/it]
                                                          
{'loss': 1.4202, 'grad_norm': 8.792194366455078, 'learning_rate': 1.7985416540444747e-05, 'epoch': 0.23}

 23%|██▎       | 8710/37962 [20:11:05<67:31:23,  8.31s/it]
 23%|██▎       | 8711/37962 [20:11:13<67:39:51,  8.33s/it]
 23%|██▎       | 8712/37962 [20:11:21<67:38:41,  8.33s/it]
 23%|██▎       | 8713/37962 [20:11:29<67:11:02,  8.27s/it]
 23%|██▎       | 8714/37962 [20:11:37<66:19:42,  8.16s/it]
 23%|██▎       | 8715/37962 [20:11:46<66:56:34,  8.24s/it]
 23%|██▎       | 8716/37962 [20:11:54<66:56:10,  8.24s/it]
 23%|██▎       | 8717/37962 [20:12:02<66:56:45,  8.24s/it]
 23%|██▎       | 8718/37962 [20:12:10<67:10:31,  8.27s/it]
 23%|██▎       | 8719/37962 [20:12:19<66:38:08,  8.20s/it]
 23%|██▎       | 8720/37962 [20:12:27<67:08:12,  8.27s/it]
                                                          
{'loss': 1.359, 'grad_norm': 8.998208999633789, 'learning_rate': 1.7980278124753118e-05, 'epoch': 0.23}

 23%|██▎       | 8720/37962 [20:12:27<67:08:12,  8.27s/it]
 23%|██▎       | 8721/37962 [20:12:35<67:11:52,  8.27s/it]
 23%|██▎       | 8722/37962 [20:12:43<66:11:58,  8.15s/it]
 23%|██▎       | 8723/37962 [20:12:51<66:15:12,  8.16s/it]
 23%|██▎       | 8724/37962 [20:13:00<66:36:08,  8.20s/it]
 23%|██▎       | 8725/37962 [20:13:08<67:13:06,  8.28s/it]
 23%|██▎       | 8726/37962 [20:13:16<67:08:19,  8.27s/it]
 23%|██▎       | 8727/37962 [20:13:24<66:57:51,  8.25s/it]
 23%|██▎       | 8728/37962 [20:13:33<67:25:17,  8.30s/it]
 23%|██▎       | 8729/37962 [20:13:41<67:27:36,  8.31s/it]
 23%|██▎       | 8730/37962 [20:13:49<67:01:46,  8.25s/it]
                                                          
{'loss': 1.4365, 'grad_norm': 9.951342582702637, 'learning_rate': 1.797513390035529e-05, 'epoch': 0.23}

 23%|██▎       | 8730/37962 [20:13:49<67:01:46,  8.25s/it]
 23%|██▎       | 8731/37962 [20:13:58<67:02:21,  8.26s/it]
 23%|██▎       | 8732/37962 [20:14:06<67:02:39,  8.26s/it]
 23%|██▎       | 8733/37962 [20:14:14<66:34:54,  8.20s/it]
 23%|██▎       | 8734/37962 [20:14:22<65:50:19,  8.11s/it]
 23%|██▎       | 8735/37962 [20:14:30<65:40:44,  8.09s/it]
 23%|██▎       | 8736/37962 [20:14:38<66:36:20,  8.20s/it]
 23%|██▎       | 8737/37962 [20:14:47<66:40:30,  8.21s/it]
 23%|██▎       | 8738/37962 [20:14:55<66:49:55,  8.23s/it]
 23%|██▎       | 8739/37962 [20:15:03<67:04:46,  8.26s/it]
 23%|██▎       | 8740/37962 [20:15:11<67:00:36,  8.26s/it]
                                                          
{'loss': 1.3924, 'grad_norm': 4.317872524261475, 'learning_rate': 1.796998387099566e-05, 'epoch': 0.23}

 23%|██▎       | 8740/37962 [20:15:11<67:00:36,  8.26s/it]
 23%|██▎       | 8741/37962 [20:15:20<66:44:29,  8.22s/it]
 23%|██▎       | 8742/37962 [20:15:28<66:44:22,  8.22s/it]
 23%|██▎       | 8743/37962 [20:15:36<67:05:06,  8.27s/it]
 23%|██▎       | 8744/37962 [20:15:45<67:20:02,  8.30s/it]
 23%|██▎       | 8745/37962 [20:15:53<67:30:51,  8.32s/it]
 23%|██▎       | 8746/37962 [20:16:01<67:48:15,  8.35s/it]
 23%|██▎       | 8747/37962 [20:16:10<67:40:55,  8.34s/it]
 23%|██▎       | 8748/37962 [20:16:18<67:07:18,  8.27s/it]
 23%|██▎       | 8749/37962 [20:16:26<67:00:48,  8.26s/it]
 23%|██▎       | 8750/37962 [20:16:34<67:04:44,  8.27s/it]
                                                          
{'loss': 1.39, 'grad_norm': 12.699377059936523, 'learning_rate': 1.7964828040422843e-05, 'epoch': 0.23}

 23%|██▎       | 8750/37962 [20:16:34<67:04:44,  8.27s/it]
 23%|██▎       | 8751/37962 [20:16:42<66:37:35,  8.21s/it]
 23%|██▎       | 8752/37962 [20:16:51<67:03:49,  8.27s/it]
 23%|██▎       | 8753/37962 [20:16:59<67:29:59,  8.32s/it]
 23%|██▎       | 8754/37962 [20:17:07<67:20:55,  8.30s/it]
 23%|██▎       | 8755/37962 [20:17:16<67:13:29,  8.29s/it]
 23%|██▎       | 8756/37962 [20:17:24<67:10:56,  8.28s/it]
 23%|██▎       | 8757/37962 [20:17:32<67:26:04,  8.31s/it]
 23%|██▎       | 8758/37962 [20:17:41<67:20:22,  8.30s/it]
 23%|██▎       | 8759/37962 [20:17:49<67:06:31,  8.27s/it]
 23%|██▎       | 8760/37962 [20:17:57<67:00:47,  8.26s/it]
                                                          
{'loss': 1.4199, 'grad_norm': 3.693254232406616, 'learning_rate': 1.795966641238967e-05, 'epoch': 0.23}

 23%|██▎       | 8760/37962 [20:17:57<67:00:47,  8.26s/it]
 23%|██▎       | 8761/37962 [20:18:05<66:38:33,  8.22s/it]
 23%|██▎       | 8762/37962 [20:18:13<66:42:43,  8.22s/it]
 23%|██▎       | 8763/37962 [20:18:22<66:24:01,  8.19s/it]
 23%|██▎       | 8764/37962 [20:18:30<66:20:43,  8.18s/it]
 23%|██▎       | 8765/37962 [20:18:38<65:59:25,  8.14s/it]
 23%|██▎       | 8766/37962 [20:18:46<66:35:52,  8.21s/it]
 23%|██▎       | 8767/37962 [20:18:54<66:50:00,  8.24s/it]
 23%|██▎       | 8768/37962 [20:19:03<66:29:54,  8.20s/it]
 23%|██▎       | 8769/37962 [20:19:10<65:45:33,  8.11s/it]
 23%|██▎       | 8770/37962 [20:19:19<66:09:26,  8.16s/it]
                                                          
{'loss': 1.3843, 'grad_norm': 4.422720909118652, 'learning_rate': 1.7954498990653208e-05, 'epoch': 0.23}

 23%|██▎       | 8770/37962 [20:19:19<66:09:26,  8.16s/it]
 23%|██▎       | 8771/37962 [20:19:27<66:25:58,  8.19s/it]
 23%|██▎       | 8772/37962 [20:19:36<67:31:56,  8.33s/it]
 23%|██▎       | 8773/37962 [20:19:44<66:59:41,  8.26s/it]
 23%|██▎       | 8774/37962 [20:19:59<84:25:07, 10.41s/it]
 23%|██▎       | 8775/37962 [20:20:07<79:16:03,  9.78s/it]
 23%|██▎       | 8776/37962 [20:20:16<75:25:34,  9.30s/it]
 23%|██▎       | 8777/37962 [20:20:24<73:01:57,  9.01s/it]
 23%|██▎       | 8778/37962 [20:20:33<72:00:15,  8.88s/it]
 23%|██▎       | 8779/37962 [20:20:48<88:48:22, 10.96s/it]
 23%|██▎       | 8780/37962 [20:20:57<82:28:04, 10.17s/it]
                                                          
{'loss': 1.3406, 'grad_norm': 5.710796356201172, 'learning_rate': 1.7949325778974737e-05, 'epoch': 0.23}

 23%|██▎       | 8780/37962 [20:20:57<82:28:04, 10.17s/it]
 23%|██▎       | 8781/37962 [20:21:05<78:28:27,  9.68s/it]
 23%|██▎       | 8782/37962 [20:21:13<75:00:47,  9.25s/it]
 23%|██▎       | 8783/37962 [20:21:22<72:03:31,  8.89s/it]
 23%|██▎       | 8784/37962 [20:21:30<71:26:46,  8.82s/it]
 23%|██▎       | 8785/37962 [20:21:39<70:25:44,  8.69s/it]
 23%|██▎       | 8786/37962 [20:21:47<68:44:47,  8.48s/it]
 23%|██▎       | 8787/37962 [20:21:54<67:10:41,  8.29s/it]
 23%|██▎       | 8788/37962 [20:22:03<66:51:22,  8.25s/it]
 23%|██▎       | 8789/37962 [20:22:11<66:25:16,  8.20s/it]
 23%|██▎       | 8790/37962 [20:22:19<65:57:31,  8.14s/it]
                                                          
{'loss': 1.5234, 'grad_norm': 15.8676118850708, 'learning_rate': 1.794414678111974e-05, 'epoch': 0.23}

 23%|██▎       | 8790/37962 [20:22:19<65:57:31,  8.14s/it]
 23%|██▎       | 8791/37962 [20:22:27<66:05:26,  8.16s/it]
 23%|██▎       | 8792/37962 [20:22:35<66:23:20,  8.19s/it]
 23%|██▎       | 8793/37962 [20:22:43<66:34:52,  8.22s/it]
 23%|██▎       | 8794/37962 [20:22:52<66:40:14,  8.23s/it]
 23%|██▎       | 8795/37962 [20:23:00<66:30:50,  8.21s/it]
 23%|██▎       | 8796/37962 [20:23:08<66:48:18,  8.25s/it]
 23%|██▎       | 8797/37962 [20:23:16<66:43:37,  8.24s/it]
 23%|██▎       | 8798/37962 [20:23:25<66:30:59,  8.21s/it]
 23%|██▎       | 8799/37962 [20:23:33<66:48:23,  8.25s/it]
 23%|██▎       | 8800/37962 [20:23:41<66:37:42,  8.23s/it]
                                                          
{'loss': 1.3995, 'grad_norm': 8.596609115600586, 'learning_rate': 1.7938962000857924e-05, 'epoch': 0.23}

 23%|██▎       | 8800/37962 [20:23:41<66:37:42,  8.23s/it]
 23%|██▎       | 8801/37962 [20:23:49<67:10:15,  8.29s/it]
 23%|██▎       | 8802/37962 [20:23:58<67:12:23,  8.30s/it]
 23%|██▎       | 8803/37962 [20:24:06<67:04:07,  8.28s/it]
 23%|██▎       | 8804/37962 [20:24:14<66:51:38,  8.25s/it]
 23%|██▎       | 8805/37962 [20:24:23<66:57:25,  8.27s/it]
 23%|██▎       | 8806/37962 [20:24:31<66:31:09,  8.21s/it]
 23%|██▎       | 8807/37962 [20:24:39<66:43:38,  8.24s/it]
 23%|██▎       | 8808/37962 [20:24:47<66:54:02,  8.26s/it]
 23%|██▎       | 8809/37962 [20:24:56<67:06:34,  8.29s/it]
 23%|██▎       | 8810/37962 [20:25:04<66:56:05,  8.27s/it]
                                                          
{'loss': 1.679, 'grad_norm': 13.734110832214355, 'learning_rate': 1.7933771441963203e-05, 'epoch': 0.23}

 23%|██▎       | 8810/37962 [20:25:04<66:56:05,  8.27s/it]
 23%|██▎       | 8811/37962 [20:25:12<67:20:21,  8.32s/it]
 23%|██▎       | 8812/37962 [20:25:20<67:14:03,  8.30s/it]
 23%|██▎       | 8813/37962 [20:25:29<66:54:05,  8.26s/it]
 23%|██▎       | 8814/37962 [20:25:37<67:03:12,  8.28s/it]
 23%|██▎       | 8815/37962 [20:25:45<67:22:43,  8.32s/it]
 23%|██▎       | 8816/37962 [20:25:54<67:30:36,  8.34s/it]
 23%|██▎       | 8817/37962 [20:26:02<66:57:34,  8.27s/it]
 23%|██▎       | 8818/37962 [20:26:10<67:17:17,  8.31s/it]
 23%|██▎       | 8819/37962 [20:26:18<66:38:54,  8.23s/it]
 23%|██▎       | 8820/37962 [20:26:26<66:19:01,  8.19s/it]
                                                          
{'loss': 1.4104, 'grad_norm': 5.3207173347473145, 'learning_rate': 1.792857510821369e-05, 'epoch': 0.23}

 23%|██▎       | 8820/37962 [20:26:26<66:19:01,  8.19s/it]
 23%|██▎       | 8821/37962 [20:26:35<66:21:37,  8.20s/it]
 23%|██▎       | 8822/37962 [20:26:43<66:16:03,  8.19s/it]
 23%|██▎       | 8823/37962 [20:26:51<66:39:26,  8.24s/it]
 23%|██▎       | 8824/37962 [20:26:59<66:45:23,  8.25s/it]
 23%|██▎       | 8825/37962 [20:27:08<66:41:48,  8.24s/it]
 23%|██▎       | 8826/37962 [20:27:16<66:45:26,  8.25s/it]
 23%|██▎       | 8827/37962 [20:27:24<66:09:37,  8.17s/it]
 23%|██▎       | 8828/37962 [20:27:32<66:22:57,  8.20s/it]
 23%|██▎       | 8829/37962 [20:27:40<65:22:48,  8.08s/it]
 23%|██▎       | 8830/37962 [20:27:48<65:41:58,  8.12s/it]
                                                          
{'loss': 1.3838, 'grad_norm': 5.227005958557129, 'learning_rate': 1.7923373003391714e-05, 'epoch': 0.23}

 23%|██▎       | 8830/37962 [20:27:48<65:41:58,  8.12s/it]
 23%|██▎       | 8831/37962 [20:27:56<65:32:10,  8.10s/it]
 23%|██▎       | 8832/37962 [20:28:05<66:03:04,  8.16s/it]
 23%|██▎       | 8833/37962 [20:28:13<65:56:06,  8.15s/it]
 23%|██▎       | 8834/37962 [20:28:21<65:50:15,  8.14s/it]
 23%|██▎       | 8835/37962 [20:28:29<66:12:59,  8.18s/it]
 23%|██▎       | 8836/37962 [20:28:37<66:05:07,  8.17s/it]
 23%|██▎       | 8837/37962 [20:28:45<66:13:45,  8.19s/it]
 23%|██▎       | 8838/37962 [20:28:54<66:15:34,  8.19s/it]
 23%|██▎       | 8839/37962 [20:29:02<66:32:49,  8.23s/it]
 23%|██▎       | 8840/37962 [20:29:10<65:59:53,  8.16s/it]
                                                          
{'loss': 1.3895, 'grad_norm': 29.93716812133789, 'learning_rate': 1.7918165131283786e-05, 'epoch': 0.23}

 23%|██▎       | 8840/37962 [20:29:10<65:59:53,  8.16s/it]
 23%|██▎       | 8841/37962 [20:29:18<66:19:09,  8.20s/it]
 23%|██▎       | 8842/37962 [20:29:26<65:38:28,  8.11s/it]
 23%|██▎       | 8843/37962 [20:29:34<65:59:43,  8.16s/it]
 23%|██▎       | 8844/37962 [20:29:43<66:29:39,  8.22s/it]
 23%|██▎       | 8845/37962 [20:29:51<66:50:58,  8.27s/it]
 23%|██▎       | 8846/37962 [20:30:00<67:13:56,  8.31s/it]
 23%|██▎       | 8847/37962 [20:30:08<67:13:45,  8.31s/it]
 23%|██▎       | 8848/37962 [20:30:16<66:57:21,  8.28s/it]
 23%|██▎       | 8849/37962 [20:30:24<67:03:58,  8.29s/it]
 23%|██▎       | 8850/37962 [20:30:33<67:01:00,  8.29s/it]
                                                          
{'loss': 1.5087, 'grad_norm': 10.15592098236084, 'learning_rate': 1.7912951495680634e-05, 'epoch': 0.23}

 23%|██▎       | 8850/37962 [20:30:33<67:01:00,  8.29s/it]
 23%|██▎       | 8851/37962 [20:30:41<67:07:59,  8.30s/it]
 23%|██▎       | 8852/37962 [20:30:49<67:15:40,  8.32s/it]
 23%|██▎       | 8853/37962 [20:30:58<67:18:39,  8.32s/it]
 23%|██▎       | 8854/37962 [20:31:06<67:56:26,  8.40s/it]
 23%|██▎       | 8855/37962 [20:31:14<67:15:23,  8.32s/it]
 23%|██▎       | 8856/37962 [20:31:22<66:32:18,  8.23s/it]
 23%|██▎       | 8857/37962 [20:31:31<66:40:57,  8.25s/it]
 23%|██▎       | 8858/37962 [20:31:39<66:34:28,  8.23s/it]
 23%|██▎       | 8859/37962 [20:31:47<66:23:14,  8.21s/it]
 23%|██▎       | 8860/37962 [20:31:55<66:14:18,  8.19s/it]
                                                          
{'loss': 1.4461, 'grad_norm': 27.096113204956055, 'learning_rate': 1.790773210037717e-05, 'epoch': 0.23}

 23%|██▎       | 8860/37962 [20:31:55<66:14:18,  8.19s/it]
 23%|██▎       | 8861/37962 [20:32:04<66:34:45,  8.24s/it]
 23%|██▎       | 8862/37962 [20:32:12<66:36:53,  8.24s/it]
 23%|██▎       | 8863/37962 [20:32:20<66:51:04,  8.27s/it]
 23%|██▎       | 8864/37962 [20:32:28<66:35:04,  8.24s/it]
 23%|██▎       | 8865/37962 [20:32:37<67:17:17,  8.33s/it]
 23%|██▎       | 8866/37962 [20:32:45<67:19:12,  8.33s/it]
 23%|██▎       | 8867/37962 [20:32:53<67:10:47,  8.31s/it]
 23%|██▎       | 8868/37962 [20:33:02<67:04:21,  8.30s/it]
 23%|██▎       | 8869/37962 [20:33:10<67:08:08,  8.31s/it]
 23%|██▎       | 8870/37962 [20:33:18<67:12:12,  8.32s/it]
                                                          
{'loss': 1.4495, 'grad_norm': 14.67995548248291, 'learning_rate': 1.7902506949172503e-05, 'epoch': 0.23}

 23%|██▎       | 8870/37962 [20:33:18<67:12:12,  8.32s/it]
 23%|██▎       | 8871/37962 [20:33:27<67:13:02,  8.32s/it]
 23%|██▎       | 8872/37962 [20:33:35<66:51:28,  8.27s/it]
 23%|██▎       | 8873/37962 [20:33:43<66:57:11,  8.29s/it]
 23%|██▎       | 8874/37962 [20:33:52<67:20:46,  8.33s/it]
 23%|██▎       | 8875/37962 [20:34:00<67:48:45,  8.39s/it]
 23%|██▎       | 8876/37962 [20:34:09<67:48:08,  8.39s/it]
 23%|██▎       | 8877/37962 [20:34:17<67:00:02,  8.29s/it]
 23%|██▎       | 8878/37962 [20:34:25<66:59:01,  8.29s/it]
 23%|██▎       | 8879/37962 [20:34:33<67:05:07,  8.30s/it]
 23%|██▎       | 8880/37962 [20:34:41<66:22:37,  8.22s/it]
                                                          
{'loss': 1.4429, 'grad_norm': 24.803268432617188, 'learning_rate': 1.789727604586993e-05, 'epoch': 0.23}

 23%|██▎       | 8880/37962 [20:34:41<66:22:37,  8.22s/it]
 23%|██▎       | 8881/37962 [20:34:49<66:20:39,  8.21s/it]
 23%|██▎       | 8882/37962 [20:35:05<83:47:17, 10.37s/it]
 23%|██▎       | 8883/37962 [20:35:13<78:49:49,  9.76s/it]
 23%|██▎       | 8884/37962 [20:35:21<75:02:41,  9.29s/it]
 23%|██▎       | 8885/37962 [20:35:30<72:48:59,  9.02s/it]
 23%|██▎       | 8886/37962 [20:35:38<70:10:06,  8.69s/it]
 23%|██▎       | 8887/37962 [20:35:46<69:05:42,  8.56s/it]
 23%|██▎       | 8888/37962 [20:35:54<68:16:28,  8.45s/it]
 23%|██▎       | 8889/37962 [20:36:02<67:55:18,  8.41s/it]
 23%|██▎       | 8890/37962 [20:36:11<67:42:17,  8.38s/it]
                                                          
{'loss': 1.3629, 'grad_norm': 32.97622299194336, 'learning_rate': 1.7892039394276934e-05, 'epoch': 0.23}

 23%|██▎       | 8890/37962 [20:36:11<67:42:17,  8.38s/it]
 23%|██▎       | 8891/37962 [20:36:19<67:23:43,  8.35s/it]
 23%|██▎       | 8892/37962 [20:36:27<67:13:15,  8.32s/it]
 23%|██▎       | 8893/37962 [20:36:35<66:03:33,  8.18s/it]
 23%|██▎       | 8894/37962 [20:36:43<65:50:21,  8.15s/it]
 23%|██▎       | 8895/37962 [20:36:52<66:50:55,  8.28s/it]
 23%|██▎       | 8896/37962 [20:37:00<66:51:14,  8.28s/it]
 23%|██▎       | 8897/37962 [20:37:08<66:59:20,  8.30s/it]
 23%|██▎       | 8898/37962 [20:37:17<66:28:10,  8.23s/it]
 23%|██▎       | 8899/37962 [20:37:25<66:32:39,  8.24s/it]
 23%|██▎       | 8900/37962 [20:37:33<66:58:22,  8.30s/it]
                                                          
{'loss': 1.5197, 'grad_norm': 11.019721984863281, 'learning_rate': 1.7886796998205183e-05, 'epoch': 0.23}

 23%|██▎       | 8900/37962 [20:37:33<66:58:22,  8.30s/it]
 23%|██▎       | 8901/37962 [20:37:41<66:41:47,  8.26s/it]
 23%|██▎       | 8902/37962 [20:37:50<66:47:56,  8.28s/it]
 23%|██▎       | 8903/37962 [20:37:58<66:44:41,  8.27s/it]
 23%|██▎       | 8904/37962 [20:38:06<66:51:26,  8.28s/it]
 23%|██▎       | 8905/37962 [20:38:15<67:05:51,  8.31s/it]
 23%|██▎       | 8906/37962 [20:38:23<67:12:07,  8.33s/it]
 23%|██▎       | 8907/37962 [20:38:32<67:54:29,  8.41s/it]
 23%|██▎       | 8908/37962 [20:38:40<68:02:25,  8.43s/it]
 23%|██▎       | 8909/37962 [20:38:49<68:05:35,  8.44s/it]
 23%|██▎       | 8910/37962 [20:38:57<67:36:39,  8.38s/it]
                                                          
{'loss': 1.4597, 'grad_norm': 15.964759826660156, 'learning_rate': 1.7881548861470525e-05, 'epoch': 0.23}

 23%|██▎       | 8910/37962 [20:38:57<67:36:39,  8.38s/it]
 23%|██▎       | 8911/37962 [20:39:05<67:17:23,  8.34s/it]
 23%|██▎       | 8912/37962 [20:39:13<67:20:15,  8.34s/it]
 23%|██▎       | 8913/37962 [20:39:22<67:36:06,  8.38s/it]
 23%|██▎       | 8914/37962 [20:39:30<67:12:12,  8.33s/it]
 23%|██▎       | 8915/37962 [20:39:38<67:10:49,  8.33s/it]
 23%|██▎       | 8916/37962 [20:39:47<67:32:26,  8.37s/it]
 23%|██▎       | 8917/37962 [20:39:55<66:27:48,  8.24s/it]
 23%|██▎       | 8918/37962 [20:40:03<67:01:22,  8.31s/it]
 23%|██▎       | 8919/37962 [20:40:12<66:59:34,  8.30s/it]
 23%|██▎       | 8920/37962 [20:40:20<66:55:37,  8.30s/it]
                                                          
{'loss': 1.4655, 'grad_norm': 5.054630279541016, 'learning_rate': 1.7876294987892992e-05, 'epoch': 0.23}

 23%|██▎       | 8920/37962 [20:40:20<66:55:37,  8.30s/it]
 23%|██▎       | 8921/37962 [20:40:28<66:46:47,  8.28s/it]
 24%|██▎       | 8922/37962 [20:40:36<66:37:33,  8.26s/it]
 24%|██▎       | 8923/37962 [20:40:44<66:25:17,  8.23s/it]
 24%|██▎       | 8924/37962 [20:40:53<66:12:52,  8.21s/it]
 24%|██▎       | 8925/37962 [20:41:01<65:33:01,  8.13s/it]
 24%|██▎       | 8926/37962 [20:41:09<66:00:25,  8.18s/it]
 24%|██▎       | 8927/37962 [20:41:17<65:47:22,  8.16s/it]
 24%|██▎       | 8928/37962 [20:41:25<66:26:50,  8.24s/it]
 24%|██▎       | 8929/37962 [20:41:34<66:48:15,  8.28s/it]
 24%|██▎       | 8930/37962 [20:41:42<67:17:20,  8.34s/it]
                                                          
{'loss': 1.4636, 'grad_norm': 9.918245315551758, 'learning_rate': 1.7871035381296782e-05, 'epoch': 0.24}

 24%|██▎       | 8930/37962 [20:41:42<67:17:20,  8.34s/it]
 24%|██▎       | 8931/37962 [20:41:51<67:11:40,  8.33s/it]
 24%|██▎       | 8932/37962 [20:41:59<67:42:17,  8.40s/it]
 24%|██▎       | 8933/37962 [20:42:08<68:07:44,  8.45s/it]
 24%|██▎       | 8934/37962 [20:42:16<67:46:31,  8.41s/it]
 24%|██▎       | 8935/37962 [20:42:24<67:15:10,  8.34s/it]
 24%|██▎       | 8936/37962 [20:42:32<66:45:15,  8.28s/it]
 24%|██▎       | 8937/37962 [20:42:41<66:41:55,  8.27s/it]
 24%|██▎       | 8938/37962 [20:42:49<66:13:46,  8.21s/it]
 24%|██▎       | 8939/37962 [20:42:57<66:07:40,  8.20s/it]
 24%|██▎       | 8940/37962 [20:43:05<66:13:29,  8.21s/it]
                                                          
{'loss': 1.3182, 'grad_norm': 23.965805053710938, 'learning_rate': 1.7865770045510276e-05, 'epoch': 0.24}

 24%|██▎       | 8940/37962 [20:43:05<66:13:29,  8.21s/it]
 24%|██▎       | 8941/37962 [20:43:14<66:52:57,  8.30s/it]
 24%|██▎       | 8942/37962 [20:43:22<66:48:44,  8.29s/it]
 24%|██▎       | 8943/37962 [20:43:30<66:56:34,  8.30s/it]
 24%|██▎       | 8944/37962 [20:43:38<66:29:47,  8.25s/it]
 24%|██▎       | 8945/37962 [20:43:47<67:03:01,  8.32s/it]
 24%|██▎       | 8946/37962 [20:43:55<66:13:23,  8.22s/it]
 24%|██▎       | 8947/37962 [20:44:03<66:45:44,  8.28s/it]
 24%|██▎       | 8948/37962 [20:44:12<67:15:56,  8.35s/it]
 24%|██▎       | 8949/37962 [20:44:20<67:24:30,  8.36s/it]
 24%|██▎       | 8950/37962 [20:44:28<66:00:18,  8.19s/it]
                                                          
{'loss': 1.5125, 'grad_norm': 7.497025012969971, 'learning_rate': 1.7860498984366018e-05, 'epoch': 0.24}

 24%|██▎       | 8950/37962 [20:44:28<66:00:18,  8.19s/it]
 24%|██▎       | 8951/37962 [20:44:36<66:32:16,  8.26s/it]
 24%|██▎       | 8952/37962 [20:44:44<65:26:55,  8.12s/it]
 24%|██▎       | 8953/37962 [20:44:52<65:48:10,  8.17s/it]
 24%|██▎       | 8954/37962 [20:45:00<65:35:33,  8.14s/it]
 24%|██▎       | 8955/37962 [20:45:16<83:23:34, 10.35s/it]
 24%|██▎       | 8956/37962 [20:45:24<78:28:32,  9.74s/it]
 24%|██▎       | 8957/37962 [20:45:33<75:16:24,  9.34s/it]
 24%|██▎       | 8958/37962 [20:45:41<73:22:43,  9.11s/it]
 24%|██▎       | 8959/37962 [20:45:50<71:24:55,  8.86s/it]
 24%|██▎       | 8960/37962 [20:45:58<70:04:13,  8.70s/it]
                                                          
{'loss': 1.3319, 'grad_norm': 6.097912788391113, 'learning_rate': 1.7855222201700725e-05, 'epoch': 0.24}

 24%|██▎       | 8960/37962 [20:45:58<70:04:13,  8.70s/it]
 24%|██▎       | 8961/37962 [20:46:06<68:50:25,  8.55s/it]
 24%|██▎       | 8962/37962 [20:46:14<68:23:00,  8.49s/it]
 24%|██▎       | 8963/37962 [20:46:22<67:20:09,  8.36s/it]
 24%|██▎       | 8964/37962 [20:46:30<66:26:42,  8.25s/it]
 24%|██▎       | 8965/37962 [20:46:39<66:36:02,  8.27s/it]
 24%|██▎       | 8966/37962 [20:46:47<66:50:55,  8.30s/it]
 24%|██▎       | 8967/37962 [20:46:55<66:38:14,  8.27s/it]
 24%|██▎       | 8968/37962 [20:47:03<65:26:52,  8.13s/it]
 24%|██▎       | 8969/37962 [20:47:12<66:10:11,  8.22s/it]
 24%|██▎       | 8970/37962 [20:47:20<66:18:35,  8.23s/it]
                                                          
{'loss': 1.4697, 'grad_norm': 10.548101425170898, 'learning_rate': 1.784993970135527e-05, 'epoch': 0.24}

 24%|██▎       | 8970/37962 [20:47:20<66:18:35,  8.23s/it]
 24%|██▎       | 8971/37962 [20:47:28<65:53:30,  8.18s/it]
 24%|██▎       | 8972/37962 [20:47:36<66:13:21,  8.22s/it]
 24%|██▎       | 8973/37962 [20:47:45<66:37:57,  8.27s/it]
 24%|██▎       | 8974/37962 [20:47:53<66:32:05,  8.26s/it]
 24%|██▎       | 8975/37962 [20:48:01<66:34:40,  8.27s/it]
 24%|██▎       | 8976/37962 [20:48:09<66:29:35,  8.26s/it]
 24%|██▎       | 8977/37962 [20:48:17<65:53:11,  8.18s/it]
 24%|██▎       | 8978/37962 [20:48:25<65:38:41,  8.15s/it]
 24%|██▎       | 8979/37962 [20:48:34<65:57:14,  8.19s/it]
 24%|██▎       | 8980/37962 [20:48:42<65:58:28,  8.20s/it]
                                                          
{'loss': 1.5635, 'grad_norm': 24.230907440185547, 'learning_rate': 1.78446514871747e-05, 'epoch': 0.24}

 24%|██▎       | 8980/37962 [20:48:42<65:58:28,  8.20s/it]
 24%|██▎       | 8981/37962 [20:48:50<65:57:37,  8.19s/it]
 24%|██▎       | 8982/37962 [20:48:58<66:04:58,  8.21s/it]
 24%|██▎       | 8983/37962 [20:49:07<66:28:26,  8.26s/it]
 24%|██▎       | 8984/37962 [20:49:15<66:20:47,  8.24s/it]
 24%|██▎       | 8985/37962 [20:49:23<66:03:43,  8.21s/it]
 24%|██▎       | 8986/37962 [20:49:31<66:30:52,  8.26s/it]
 24%|██▎       | 8987/37962 [20:49:39<65:22:32,  8.12s/it]
 24%|██▎       | 8988/37962 [20:49:48<65:36:24,  8.15s/it]
 24%|██▎       | 8989/37962 [20:49:56<65:59:44,  8.20s/it]
 24%|██▎       | 8990/37962 [20:50:04<66:03:33,  8.21s/it]
                                                          
{'loss': 1.4374, 'grad_norm': 20.92695426940918, 'learning_rate': 1.7839357563008214e-05, 'epoch': 0.24}

 24%|██▎       | 8990/37962 [20:50:04<66:03:33,  8.21s/it]
 24%|██▎       | 8991/37962 [20:50:12<65:35:53,  8.15s/it]
 24%|██▎       | 8992/37962 [20:50:20<65:56:57,  8.20s/it]
 24%|██▎       | 8993/37962 [20:50:28<65:26:49,  8.13s/it]
 24%|██▎       | 8994/37962 [20:50:36<65:20:47,  8.12s/it]
 24%|██▎       | 8995/37962 [20:50:45<66:04:28,  8.21s/it]
 24%|██▎       | 8996/37962 [20:50:53<65:45:18,  8.17s/it]
 24%|██▎       | 8997/37962 [20:51:01<66:14:49,  8.23s/it]
 24%|██▎       | 8998/37962 [20:51:09<65:45:50,  8.17s/it]
 24%|██▎       | 8999/37962 [20:51:18<65:55:35,  8.19s/it]
 24%|██▎       | 9000/37962 [20:51:26<66:03:55,  8.21s/it]
                                                          
{'loss': 1.3859, 'grad_norm': 16.315082550048828, 'learning_rate': 1.7834057932709163e-05, 'epoch': 0.24}

 24%|██▎       | 9000/37962 [20:51:26<66:03:55,  8.21s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 24%|██▎       | 9001/37962 [20:52:17<169:20:35, 21.05s/it]
 24%|██▎       | 9002/37962 [20:52:25<139:19:14, 17.32s/it]
 24%|██▎       | 9003/37962 [20:52:34<117:56:43, 14.66s/it]
 24%|██▎       | 9004/37962 [20:52:42<102:38:15, 12.76s/it]
 24%|██▎       | 9005/37962 [20:52:50<91:36:08, 11.39s/it] 
 24%|██▎       | 9006/37962 [20:52:59<84:13:02, 10.47s/it]
 24%|██▎       | 9007/37962 [20:53:07<79:26:28,  9.88s/it]
 24%|██▎       | 9008/37962 [20:53:15<74:37:57,  9.28s/it]
 24%|██▎       | 9009/37962 [20:53:23<72:15:42,  8.98s/it]
 24%|██▎       | 9010/37962 [20:53:32<70:20:23,  8.75s/it]
                                                          
{'loss': 1.4249, 'grad_norm': 6.226747989654541, 'learning_rate': 1.7828752600135054e-05, 'epoch': 0.24}

 24%|██▎       | 9010/37962 [20:53:32<70:20:23,  8.75s/it]
 24%|██▎       | 9011/37962 [20:53:40<69:39:58,  8.66s/it]
 24%|██▎       | 9012/37962 [20:53:48<68:10:22,  8.48s/it]
 24%|██▎       | 9013/37962 [20:53:56<67:36:43,  8.41s/it]
 24%|██▎       | 9014/37962 [20:54:04<66:52:20,  8.32s/it]
 24%|██▎       | 9015/37962 [20:54:13<66:47:06,  8.31s/it]
 24%|██▍       | 9016/37962 [20:54:21<66:43:29,  8.30s/it]
 24%|██▍       | 9017/37962 [20:54:29<66:33:45,  8.28s/it]
 24%|██▍       | 9018/37962 [20:54:38<66:45:42,  8.30s/it]
 24%|██▍       | 9019/37962 [20:54:46<66:08:00,  8.23s/it]
 24%|██▍       | 9020/37962 [20:54:54<67:02:15,  8.34s/it]
                                                          
{'loss': 1.4275, 'grad_norm': 10.492033958435059, 'learning_rate': 1.7823441569147558e-05, 'epoch': 0.24}

 24%|██▍       | 9020/37962 [20:54:54<67:02:15,  8.34s/it]
 24%|██▍       | 9021/37962 [20:55:11<86:01:23, 10.70s/it]
 24%|██▍       | 9022/37962 [20:55:19<79:42:31,  9.92s/it]
 24%|██▍       | 9023/37962 [20:55:27<75:55:26,  9.44s/it]
 24%|██▍       | 9024/37962 [20:55:35<73:12:14,  9.11s/it]
 24%|██▍       | 9025/37962 [20:55:44<72:04:12,  8.97s/it]
 24%|██▍       | 9026/37962 [20:55:52<70:07:44,  8.72s/it]
 24%|██▍       | 9027/37962 [20:56:00<69:13:45,  8.61s/it]
 24%|██▍       | 9028/37962 [20:56:09<68:45:35,  8.56s/it]
 24%|██▍       | 9029/37962 [20:56:17<68:23:53,  8.51s/it]
 24%|██▍       | 9030/37962 [20:56:25<67:34:50,  8.41s/it]
                                                          
{'loss': 1.5047, 'grad_norm': 20.60093879699707, 'learning_rate': 1.7818124843612473e-05, 'epoch': 0.24}

 24%|██▍       | 9030/37962 [20:56:25<67:34:50,  8.41s/it]
 24%|██▍       | 9031/37962 [20:56:34<67:47:59,  8.44s/it]
 24%|██▍       | 9032/37962 [20:56:42<68:02:51,  8.47s/it]
 24%|██▍       | 9033/37962 [20:56:51<67:42:38,  8.43s/it]
 24%|██▍       | 9034/37962 [20:56:59<67:36:19,  8.41s/it]
 24%|██▍       | 9035/37962 [20:57:07<67:13:13,  8.37s/it]
 24%|██▍       | 9036/37962 [20:57:15<66:21:34,  8.26s/it]
 24%|██▍       | 9037/37962 [20:57:23<65:44:55,  8.18s/it]
 24%|██▍       | 9038/37962 [20:57:32<65:59:17,  8.21s/it]
 24%|██▍       | 9039/37962 [20:57:40<65:52:45,  8.20s/it]
 24%|██▍       | 9040/37962 [20:57:48<66:31:26,  8.28s/it]
                                                          
{'loss': 1.2648, 'grad_norm': 8.934575080871582, 'learning_rate': 1.7812802427399753e-05, 'epoch': 0.24}

 24%|██▍       | 9040/37962 [20:57:48<66:31:26,  8.28s/it]
 24%|██▍       | 9041/37962 [20:57:56<65:36:38,  8.17s/it]
 24%|██▍       | 9042/37962 [20:58:04<65:17:32,  8.13s/it]
 24%|██▍       | 9043/37962 [20:58:13<65:31:05,  8.16s/it]
 24%|██▍       | 9044/37962 [20:58:21<65:41:30,  8.18s/it]
 24%|██▍       | 9045/37962 [20:58:29<65:34:39,  8.16s/it]
 24%|██▍       | 9046/37962 [20:58:37<65:51:39,  8.20s/it]
 24%|██▍       | 9047/37962 [20:58:46<66:22:32,  8.26s/it]
 24%|██▍       | 9048/37962 [20:58:54<66:23:36,  8.27s/it]
 24%|██▍       | 9049/37962 [20:59:02<65:29:02,  8.15s/it]
 24%|██▍       | 9050/37962 [20:59:10<65:59:02,  8.22s/it]
                                                          
{'loss': 1.4232, 'grad_norm': 3.702556848526001, 'learning_rate': 1.7807474324383496e-05, 'epoch': 0.24}

 24%|██▍       | 9050/37962 [20:59:10<65:59:02,  8.22s/it]
 24%|██▍       | 9051/37962 [20:59:18<65:59:41,  8.22s/it]
 24%|██▍       | 9052/37962 [20:59:26<65:40:52,  8.18s/it]
 24%|██▍       | 9053/37962 [20:59:34<65:20:23,  8.14s/it]
 24%|██▍       | 9054/37962 [20:59:43<65:33:33,  8.16s/it]
 24%|██▍       | 9055/37962 [20:59:51<66:16:44,  8.25s/it]
 24%|██▍       | 9056/37962 [20:59:59<66:19:24,  8.26s/it]
 24%|██▍       | 9057/37962 [21:00:08<66:16:51,  8.26s/it]
 24%|██▍       | 9058/37962 [21:00:16<66:20:43,  8.26s/it]
 24%|██▍       | 9059/37962 [21:00:24<65:35:21,  8.17s/it]
 24%|██▍       | 9060/37962 [21:00:32<66:19:01,  8.26s/it]
                                                          
{'loss': 1.4355, 'grad_norm': 1.5563379526138306, 'learning_rate': 1.780214053844193e-05, 'epoch': 0.24}

 24%|██▍       | 9060/37962 [21:00:32<66:19:01,  8.26s/it]
 24%|██▍       | 9061/37962 [21:00:40<65:50:59,  8.20s/it]
 24%|██▍       | 9062/37962 [21:00:49<66:12:15,  8.25s/it]
 24%|██▍       | 9063/37962 [21:00:57<65:40:17,  8.18s/it]
 24%|██▍       | 9064/37962 [21:01:05<65:47:46,  8.20s/it]
 24%|██▍       | 9065/37962 [21:01:13<65:58:49,  8.22s/it]
 24%|██▍       | 9066/37962 [21:01:22<65:59:57,  8.22s/it]
 24%|██▍       | 9067/37962 [21:01:30<66:09:37,  8.24s/it]
 24%|██▍       | 9068/37962 [21:01:38<66:10:31,  8.25s/it]
 24%|██▍       | 9069/37962 [21:01:46<65:34:12,  8.17s/it]
 24%|██▍       | 9070/37962 [21:01:54<65:45:15,  8.19s/it]
                                                          
{'loss': 1.4249, 'grad_norm': 5.938529968261719, 'learning_rate': 1.7796801073457436e-05, 'epoch': 0.24}

 24%|██▍       | 9070/37962 [21:01:54<65:45:15,  8.19s/it]
 24%|██▍       | 9071/37962 [21:02:03<65:49:19,  8.20s/it]
 24%|██▍       | 9072/37962 [21:02:11<66:07:46,  8.24s/it]
 24%|██▍       | 9073/37962 [21:02:19<66:07:17,  8.24s/it]
 24%|██▍       | 9074/37962 [21:02:27<65:26:19,  8.15s/it]
 24%|██▍       | 9075/37962 [21:02:35<65:35:06,  8.17s/it]
 24%|██▍       | 9076/37962 [21:02:43<65:33:55,  8.17s/it]
 24%|██▍       | 9077/37962 [21:02:51<64:51:30,  8.08s/it]
 24%|██▍       | 9078/37962 [21:03:00<65:12:08,  8.13s/it]
 24%|██▍       | 9079/37962 [21:03:08<65:39:13,  8.18s/it]
 24%|██▍       | 9080/37962 [21:03:16<66:05:16,  8.24s/it]
                                                          
{'loss': 1.4843, 'grad_norm': 23.56593894958496, 'learning_rate': 1.7791455933316505e-05, 'epoch': 0.24}

 24%|██▍       | 9080/37962 [21:03:16<66:05:16,  8.24s/it]
 24%|██▍       | 9081/37962 [21:03:24<65:30:03,  8.16s/it]
 24%|██▍       | 9082/37962 [21:03:33<66:22:23,  8.27s/it]
 24%|██▍       | 9083/37962 [21:03:41<65:56:33,  8.22s/it]
 24%|██▍       | 9084/37962 [21:03:49<66:06:39,  8.24s/it]
 24%|██▍       | 9085/37962 [21:03:57<66:00:56,  8.23s/it]
 24%|██▍       | 9086/37962 [21:04:06<66:07:16,  8.24s/it]
 24%|██▍       | 9087/37962 [21:04:14<66:35:17,  8.30s/it]
 24%|██▍       | 9088/37962 [21:04:22<66:23:50,  8.28s/it]
 24%|██▍       | 9089/37962 [21:04:31<66:35:47,  8.30s/it]
 24%|██▍       | 9090/37962 [21:04:39<65:55:26,  8.22s/it]
                                                          
{'loss': 1.4621, 'grad_norm': 7.194381237030029, 'learning_rate': 1.778610512190978e-05, 'epoch': 0.24}

 24%|██▍       | 9090/37962 [21:04:39<65:55:26,  8.22s/it]
 24%|██▍       | 9091/37962 [21:04:47<65:21:33,  8.15s/it]
 24%|██▍       | 9092/37962 [21:04:55<65:58:46,  8.23s/it]
 24%|██▍       | 9093/37962 [21:05:03<66:12:46,  8.26s/it]
 24%|██▍       | 9094/37962 [21:05:12<66:12:16,  8.26s/it]
 24%|██▍       | 9095/37962 [21:05:20<66:14:15,  8.26s/it]
 24%|██▍       | 9096/37962 [21:05:28<65:06:11,  8.12s/it]
 24%|██▍       | 9097/37962 [21:05:36<65:30:43,  8.17s/it]
 24%|██▍       | 9098/37962 [21:05:44<65:46:23,  8.20s/it]
 24%|██▍       | 9099/37962 [21:05:53<67:05:41,  8.37s/it]
 24%|██▍       | 9100/37962 [21:06:01<66:51:38,  8.34s/it]
                                                          
{'loss': 1.4509, 'grad_norm': 8.68651294708252, 'learning_rate': 1.7780748643132017e-05, 'epoch': 0.24}

 24%|██▍       | 9100/37962 [21:06:01<66:51:38,  8.34s/it]
 24%|██▍       | 9101/37962 [21:06:09<66:17:40,  8.27s/it]
 24%|██▍       | 9102/37962 [21:06:18<66:06:32,  8.25s/it]
 24%|██▍       | 9103/37962 [21:06:26<66:21:22,  8.28s/it]
 24%|██▍       | 9104/37962 [21:06:34<66:26:38,  8.29s/it]
 24%|██▍       | 9105/37962 [21:06:42<65:49:36,  8.21s/it]
 24%|██▍       | 9106/37962 [21:06:50<65:44:37,  8.20s/it]
 24%|██▍       | 9107/37962 [21:06:59<66:20:25,  8.28s/it]
 24%|██▍       | 9108/37962 [21:07:07<66:58:10,  8.36s/it]
 24%|██▍       | 9109/37962 [21:07:15<65:57:20,  8.23s/it]
 24%|██▍       | 9110/37962 [21:07:24<65:51:58,  8.22s/it]
                                                          
{'loss': 1.4456, 'grad_norm': 6.123380661010742, 'learning_rate': 1.7775386500882113e-05, 'epoch': 0.24}

 24%|██▍       | 9110/37962 [21:07:24<65:51:58,  8.22s/it]
 24%|██▍       | 9111/37962 [21:07:32<65:49:31,  8.21s/it]
 24%|██▍       | 9112/37962 [21:07:40<66:26:54,  8.29s/it]
 24%|██▍       | 9113/37962 [21:07:49<66:31:44,  8.30s/it]
 24%|██▍       | 9114/37962 [21:07:57<66:25:43,  8.29s/it]
 24%|██▍       | 9115/37962 [21:08:05<66:38:15,  8.32s/it]
 24%|██▍       | 9116/37962 [21:08:14<66:36:36,  8.31s/it]
 24%|██▍       | 9117/37962 [21:08:22<66:37:15,  8.31s/it]
 24%|██▍       | 9118/37962 [21:08:30<65:50:39,  8.22s/it]
 24%|██▍       | 9119/37962 [21:08:38<65:39:45,  8.20s/it]
 24%|██▍       | 9120/37962 [21:08:46<66:05:27,  8.25s/it]
                                                          
{'loss': 1.2593, 'grad_norm': 6.692594051361084, 'learning_rate': 1.7770018699063075e-05, 'epoch': 0.24}

 24%|██▍       | 9120/37962 [21:08:46<66:05:27,  8.25s/it]
 24%|██▍       | 9121/37962 [21:08:55<65:59:51,  8.24s/it]
 24%|██▍       | 9122/37962 [21:09:03<65:54:16,  8.23s/it]
 24%|██▍       | 9123/37962 [21:09:11<65:59:43,  8.24s/it]
 24%|██▍       | 9124/37962 [21:09:19<65:46:13,  8.21s/it]
 24%|██▍       | 9125/37962 [21:09:27<64:51:26,  8.10s/it]
 24%|██▍       | 9126/37962 [21:09:35<65:16:14,  8.15s/it]
 24%|██▍       | 9127/37962 [21:09:43<65:15:20,  8.15s/it]
 24%|██▍       | 9128/37962 [21:09:52<65:36:03,  8.19s/it]
 24%|██▍       | 9129/37962 [21:10:00<66:19:01,  8.28s/it]
 24%|██▍       | 9130/37962 [21:10:08<66:16:16,  8.27s/it]
                                                          
{'loss': 1.4939, 'grad_norm': 12.641352653503418, 'learning_rate': 1.776464524158203e-05, 'epoch': 0.24}

 24%|██▍       | 9130/37962 [21:10:08<66:16:16,  8.27s/it]
 24%|██▍       | 9131/37962 [21:10:17<66:15:56,  8.27s/it]
 24%|██▍       | 9132/37962 [21:10:25<66:24:23,  8.29s/it]
 24%|██▍       | 9133/37962 [21:10:34<66:47:37,  8.34s/it]
 24%|██▍       | 9134/37962 [21:10:42<66:23:22,  8.29s/it]
 24%|██▍       | 9135/37962 [21:10:50<65:42:34,  8.21s/it]
 24%|██▍       | 9136/37962 [21:10:58<65:53:11,  8.23s/it]
 24%|██▍       | 9137/37962 [21:11:06<65:57:33,  8.24s/it]
 24%|██▍       | 9138/37962 [21:11:15<66:22:29,  8.29s/it]
 24%|██▍       | 9139/37962 [21:11:23<66:24:30,  8.29s/it]
 24%|██▍       | 9140/37962 [21:11:31<66:14:08,  8.27s/it]
                                                          
{'loss': 1.4759, 'grad_norm': 8.251809120178223, 'learning_rate': 1.775926613235023e-05, 'epoch': 0.24}

 24%|██▍       | 9140/37962 [21:11:31<66:14:08,  8.27s/it]
 24%|██▍       | 9141/37962 [21:11:40<66:29:39,  8.31s/it]
 24%|██▍       | 9142/37962 [21:11:48<66:17:05,  8.28s/it]
 24%|██▍       | 9143/37962 [21:11:56<65:51:27,  8.23s/it]
 24%|██▍       | 9144/37962 [21:12:04<66:27:23,  8.30s/it]
 24%|██▍       | 9145/37962 [21:12:13<66:04:32,  8.25s/it]
 24%|██▍       | 9146/37962 [21:12:21<65:29:48,  8.18s/it]
 24%|██▍       | 9147/37962 [21:12:29<66:09:35,  8.27s/it]
 24%|██▍       | 9148/37962 [21:12:37<65:40:14,  8.20s/it]
 24%|██▍       | 9149/37962 [21:12:45<66:10:56,  8.27s/it]
 24%|██▍       | 9150/37962 [21:12:54<66:33:49,  8.32s/it]
                                                          
{'loss': 1.41, 'grad_norm': 3.0155279636383057, 'learning_rate': 1.775388137528303e-05, 'epoch': 0.24}

 24%|██▍       | 9150/37962 [21:12:54<66:33:49,  8.32s/it]
 24%|██▍       | 9151/37962 [21:13:02<66:21:14,  8.29s/it]
 24%|██▍       | 9152/37962 [21:13:10<66:12:05,  8.27s/it]
 24%|██▍       | 9153/37962 [21:13:19<66:28:06,  8.31s/it]
 24%|██▍       | 9154/37962 [21:13:27<66:40:02,  8.33s/it]
 24%|██▍       | 9155/37962 [21:13:35<65:40:03,  8.21s/it]
 24%|██▍       | 9156/37962 [21:13:43<65:50:28,  8.23s/it]
 24%|██▍       | 9157/37962 [21:13:52<65:52:28,  8.23s/it]
 24%|██▍       | 9158/37962 [21:14:00<65:42:29,  8.21s/it]
 24%|██▍       | 9159/37962 [21:14:08<65:44:54,  8.22s/it]
 24%|██▍       | 9160/37962 [21:14:16<65:51:56,  8.23s/it]
                                                          
{'loss': 1.3403, 'grad_norm': 13.33884334564209, 'learning_rate': 1.7748490974299906e-05, 'epoch': 0.24}

 24%|██▍       | 9160/37962 [21:14:16<65:51:56,  8.23s/it]
 24%|██▍       | 9161/37962 [21:14:25<66:43:16,  8.34s/it]
 24%|██▍       | 9162/37962 [21:14:33<66:44:15,  8.34s/it]
 24%|██▍       | 9163/37962 [21:14:41<66:02:43,  8.26s/it]
 24%|██▍       | 9164/37962 [21:14:49<65:39:30,  8.21s/it]
 24%|██▍       | 9165/37962 [21:14:58<65:44:01,  8.22s/it]
 24%|██▍       | 9166/37962 [21:15:06<65:17:23,  8.16s/it]
 24%|██▍       | 9167/37962 [21:15:14<65:20:21,  8.17s/it]
 24%|██▍       | 9168/37962 [21:15:22<65:14:51,  8.16s/it]
 24%|██▍       | 9169/37962 [21:15:30<65:50:35,  8.23s/it]
 24%|██▍       | 9170/37962 [21:15:39<65:51:04,  8.23s/it]
                                                          
{'loss': 1.4259, 'grad_norm': 16.353378295898438, 'learning_rate': 1.7743094933324432e-05, 'epoch': 0.24}

 24%|██▍       | 9170/37962 [21:15:39<65:51:04,  8.23s/it]
 24%|██▍       | 9171/37962 [21:15:47<66:06:26,  8.27s/it]
 24%|██▍       | 9172/37962 [21:15:55<66:17:28,  8.29s/it]
 24%|██▍       | 9173/37962 [21:16:03<65:56:56,  8.25s/it]
 24%|██▍       | 9174/37962 [21:16:12<65:46:33,  8.23s/it]
 24%|██▍       | 9175/37962 [21:16:20<65:48:57,  8.23s/it]
 24%|██▍       | 9176/37962 [21:16:28<65:29:25,  8.19s/it]
 24%|██▍       | 9177/37962 [21:16:36<66:14:03,  8.28s/it]
 24%|██▍       | 9178/37962 [21:16:45<66:22:36,  8.30s/it]
 24%|██▍       | 9179/37962 [21:16:53<66:23:42,  8.30s/it]
 24%|██▍       | 9180/37962 [21:17:01<66:32:44,  8.32s/it]
                                                          
{'loss': 1.4421, 'grad_norm': 19.454835891723633, 'learning_rate': 1.7737693256284298e-05, 'epoch': 0.24}

 24%|██▍       | 9180/37962 [21:17:01<66:32:44,  8.32s/it]
 24%|██▍       | 9181/37962 [21:17:10<66:24:09,  8.31s/it]
 24%|██▍       | 9182/37962 [21:17:18<66:11:42,  8.28s/it]
 24%|██▍       | 9183/37962 [21:17:26<66:45:29,  8.35s/it]
 24%|██▍       | 9184/37962 [21:17:35<66:38:29,  8.34s/it]
 24%|██▍       | 9185/37962 [21:17:43<66:51:31,  8.36s/it]
 24%|██▍       | 9186/37962 [21:17:52<66:54:53,  8.37s/it]
 24%|██▍       | 9187/37962 [21:18:00<66:26:08,  8.31s/it]
 24%|██▍       | 9188/37962 [21:18:08<66:05:10,  8.27s/it]
 24%|██▍       | 9189/37962 [21:18:16<66:04:23,  8.27s/it]
 24%|██▍       | 9190/37962 [21:18:24<66:07:21,  8.27s/it]
                                                          
{'loss': 1.3206, 'grad_norm': 5.868311405181885, 'learning_rate': 1.773228594711129e-05, 'epoch': 0.24}

 24%|██▍       | 9190/37962 [21:18:24<66:07:21,  8.27s/it]
 24%|██▍       | 9191/37962 [21:18:33<65:47:44,  8.23s/it]
 24%|██▍       | 9192/37962 [21:18:41<66:00:59,  8.26s/it]
 24%|██▍       | 9193/37962 [21:18:49<65:52:22,  8.24s/it]
 24%|██▍       | 9194/37962 [21:18:57<66:03:09,  8.27s/it]
 24%|██▍       | 9195/37962 [21:19:06<66:12:14,  8.29s/it]
 24%|██▍       | 9196/37962 [21:19:14<65:52:19,  8.24s/it]
 24%|██▍       | 9197/37962 [21:19:22<65:50:29,  8.24s/it]
 24%|██▍       | 9198/37962 [21:19:30<65:31:56,  8.20s/it]
 24%|██▍       | 9199/37962 [21:19:39<65:40:28,  8.22s/it]
 24%|██▍       | 9200/37962 [21:19:47<65:51:50,  8.24s/it]
                                                          
{'loss': 1.3324, 'grad_norm': 13.859254837036133, 'learning_rate': 1.7726873009741293e-05, 'epoch': 0.24}

 24%|██▍       | 9200/37962 [21:19:47<65:51:50,  8.24s/it]
 24%|██▍       | 9201/37962 [21:19:55<65:22:18,  8.18s/it]
 24%|██▍       | 9202/37962 [21:20:03<65:31:42,  8.20s/it]
 24%|██▍       | 9203/37962 [21:20:11<65:06:56,  8.15s/it]
 24%|██▍       | 9204/37962 [21:20:19<65:33:42,  8.21s/it]
 24%|██▍       | 9205/37962 [21:20:28<65:51:11,  8.24s/it]
 24%|██▍       | 9206/37962 [21:20:36<66:17:57,  8.30s/it]
 24%|██▍       | 9207/37962 [21:20:44<66:04:15,  8.27s/it]
 24%|██▍       | 9208/37962 [21:20:53<65:58:15,  8.26s/it]
 24%|██▍       | 9209/37962 [21:21:01<66:21:58,  8.31s/it]
 24%|██▍       | 9210/37962 [21:21:09<66:04:37,  8.27s/it]
                                                          
{'loss': 1.5416, 'grad_norm': 3.8151609897613525, 'learning_rate': 1.7721454448114287e-05, 'epoch': 0.24}

 24%|██▍       | 9210/37962 [21:21:09<66:04:37,  8.27s/it]
 24%|██▍       | 9211/37962 [21:21:18<66:01:23,  8.27s/it]
 24%|██▍       | 9212/37962 [21:21:26<65:19:13,  8.18s/it]
 24%|██▍       | 9213/37962 [21:21:34<65:48:00,  8.24s/it]
 24%|██▍       | 9214/37962 [21:21:42<65:39:33,  8.22s/it]
 24%|██▍       | 9215/37962 [21:21:50<65:45:03,  8.23s/it]
 24%|██▍       | 9216/37962 [21:21:59<65:36:14,  8.22s/it]
 24%|██▍       | 9217/37962 [21:22:07<65:48:21,  8.24s/it]
 24%|██▍       | 9218/37962 [21:22:15<65:00:27,  8.14s/it]
 24%|██▍       | 9219/37962 [21:22:23<65:08:15,  8.16s/it]
 24%|██▍       | 9220/37962 [21:22:31<64:51:54,  8.12s/it]
                                                          
{'loss': 1.3844, 'grad_norm': 8.186395645141602, 'learning_rate': 1.7716030266174357e-05, 'epoch': 0.24}

 24%|██▍       | 9220/37962 [21:22:31<64:51:54,  8.12s/it]
 24%|██▍       | 9221/37962 [21:22:39<65:02:55,  8.15s/it]
 24%|██▍       | 9222/37962 [21:22:47<65:22:47,  8.19s/it]
 24%|██▍       | 9223/37962 [21:22:56<65:54:18,  8.26s/it]
 24%|██▍       | 9224/37962 [21:23:04<66:42:24,  8.36s/it]
 24%|██▍       | 9225/37962 [21:23:13<66:17:54,  8.31s/it]
 24%|██▍       | 9226/37962 [21:23:21<66:16:04,  8.30s/it]
 24%|██▍       | 9227/37962 [21:23:29<66:24:06,  8.32s/it]
 24%|██▍       | 9228/37962 [21:23:37<66:03:33,  8.28s/it]
 24%|██▍       | 9229/37962 [21:23:46<66:13:23,  8.30s/it]
 24%|██▍       | 9230/37962 [21:23:54<66:34:13,  8.34s/it]
                                                          
{'loss': 1.5087, 'grad_norm': 16.205904006958008, 'learning_rate': 1.7710600467869666e-05, 'epoch': 0.24}

 24%|██▍       | 9230/37962 [21:23:54<66:34:13,  8.34s/it]
 24%|██▍       | 9231/37962 [21:24:02<65:59:30,  8.27s/it]
 24%|██▍       | 9232/37962 [21:24:11<66:11:25,  8.29s/it]
 24%|██▍       | 9233/37962 [21:24:19<66:01:18,  8.27s/it]
 24%|██▍       | 9234/37962 [21:24:27<65:09:32,  8.17s/it]
 24%|██▍       | 9235/37962 [21:24:35<65:23:54,  8.20s/it]
 24%|██▍       | 9236/37962 [21:24:43<65:35:12,  8.22s/it]
 24%|██▍       | 9237/37962 [21:24:52<65:32:56,  8.22s/it]
 24%|██▍       | 9238/37962 [21:25:00<65:38:53,  8.23s/it]
 24%|██▍       | 9239/37962 [21:25:08<65:31:03,  8.21s/it]
 24%|██▍       | 9240/37962 [21:25:16<64:51:57,  8.13s/it]
                                                          
{'loss': 1.46, 'grad_norm': 12.89169692993164, 'learning_rate': 1.7705165057152476e-05, 'epoch': 0.24}

 24%|██▍       | 9240/37962 [21:25:16<64:51:57,  8.13s/it]
 24%|██▍       | 9241/37962 [21:25:24<65:26:32,  8.20s/it]
 24%|██▍       | 9242/37962 [21:25:33<65:27:40,  8.21s/it]
 24%|██▍       | 9243/37962 [21:25:41<65:31:46,  8.21s/it]
 24%|██▍       | 9244/37962 [21:25:49<65:36:41,  8.22s/it]
 24%|██▍       | 9245/37962 [21:25:57<66:00:06,  8.27s/it]
 24%|██▍       | 9246/37962 [21:26:06<65:58:29,  8.27s/it]
 24%|██▍       | 9247/37962 [21:26:14<66:25:05,  8.33s/it]
 24%|██▍       | 9248/37962 [21:26:22<66:14:21,  8.30s/it]
 24%|██▍       | 9249/37962 [21:26:31<66:11:22,  8.30s/it]
 24%|██▍       | 9250/37962 [21:26:39<65:39:40,  8.23s/it]
                                                          
{'loss': 1.4239, 'grad_norm': 6.3555402755737305, 'learning_rate': 1.769972403797912e-05, 'epoch': 0.24}

 24%|██▍       | 9250/37962 [21:26:39<65:39:40,  8.23s/it]
 24%|██▍       | 9251/37962 [21:26:47<65:51:02,  8.26s/it]
 24%|██▍       | 9252/37962 [21:26:55<65:44:01,  8.24s/it]
 24%|██▍       | 9253/37962 [21:27:04<65:42:58,  8.24s/it]
 24%|██▍       | 9254/37962 [21:27:12<65:49:08,  8.25s/it]
 24%|██▍       | 9255/37962 [21:27:20<64:58:32,  8.15s/it]
 24%|██▍       | 9256/37962 [21:27:28<65:25:31,  8.20s/it]
 24%|██▍       | 9257/37962 [21:27:36<65:10:27,  8.17s/it]
 24%|██▍       | 9258/37962 [21:27:44<64:57:05,  8.15s/it]
 24%|██▍       | 9259/37962 [21:27:52<65:06:53,  8.17s/it]
 24%|██▍       | 9260/37962 [21:28:01<65:35:46,  8.23s/it]
                                                          
{'loss': 1.4282, 'grad_norm': 8.18881893157959, 'learning_rate': 1.7694277414310027e-05, 'epoch': 0.24}

 24%|██▍       | 9260/37962 [21:28:01<65:35:46,  8.23s/it]
 24%|██▍       | 9261/37962 [21:28:09<64:42:01,  8.12s/it]
 24%|██▍       | 9262/37962 [21:28:17<64:50:21,  8.13s/it]
 24%|██▍       | 9263/37962 [21:28:25<65:26:24,  8.21s/it]
 24%|██▍       | 9264/37962 [21:28:33<65:23:29,  8.20s/it]
 24%|██▍       | 9265/37962 [21:28:42<65:24:26,  8.21s/it]
 24%|██▍       | 9266/37962 [21:28:50<65:23:14,  8.20s/it]
 24%|██▍       | 9267/37962 [21:28:58<64:57:09,  8.15s/it]
 24%|██▍       | 9268/37962 [21:29:06<65:24:48,  8.21s/it]
 24%|██▍       | 9269/37962 [21:29:14<65:26:49,  8.21s/it]
 24%|██▍       | 9270/37962 [21:29:22<64:40:51,  8.12s/it]
                                                          
{'loss': 1.3653, 'grad_norm': 2.687380790710449, 'learning_rate': 1.7688825190109695e-05, 'epoch': 0.24}

 24%|██▍       | 9270/37962 [21:29:22<64:40:51,  8.12s/it]
 24%|██▍       | 9271/37962 [21:29:31<65:20:20,  8.20s/it]
 24%|██▍       | 9272/37962 [21:29:39<65:50:38,  8.26s/it]
 24%|██▍       | 9273/37962 [21:29:47<66:00:14,  8.28s/it]
 24%|██▍       | 9274/37962 [21:29:56<66:03:02,  8.29s/it]
 24%|██▍       | 9275/37962 [21:30:04<66:03:53,  8.29s/it]
 24%|██▍       | 9276/37962 [21:30:12<64:41:35,  8.12s/it]
 24%|██▍       | 9277/37962 [21:30:20<65:29:52,  8.22s/it]
 24%|██▍       | 9278/37962 [21:30:29<65:47:14,  8.26s/it]
 24%|██▍       | 9279/37962 [21:30:37<65:41:08,  8.24s/it]
 24%|██▍       | 9280/37962 [21:30:45<65:09:20,  8.18s/it]
                                                          
{'loss': 1.3129, 'grad_norm': 11.020258903503418, 'learning_rate': 1.7683367369346708e-05, 'epoch': 0.24}

 24%|██▍       | 9280/37962 [21:30:45<65:09:20,  8.18s/it]
 24%|██▍       | 9281/37962 [21:30:53<64:54:53,  8.15s/it]
 24%|██▍       | 9282/37962 [21:31:01<65:08:49,  8.18s/it]
 24%|██▍       | 9283/37962 [21:31:10<65:52:17,  8.27s/it]
 24%|██▍       | 9284/37962 [21:31:18<65:06:18,  8.17s/it]
 24%|██▍       | 9285/37962 [21:31:26<65:42:17,  8.25s/it]
 24%|██▍       | 9286/37962 [21:31:34<65:48:29,  8.26s/it]
 24%|██▍       | 9287/37962 [21:31:42<65:32:53,  8.23s/it]
 24%|██▍       | 9288/37962 [21:31:51<65:12:32,  8.19s/it]
 24%|██▍       | 9289/37962 [21:31:58<64:41:01,  8.12s/it]
 24%|██▍       | 9290/37962 [21:32:07<65:37:42,  8.24s/it]
                                                          
{'loss': 1.4425, 'grad_norm': 6.068477630615234, 'learning_rate': 1.767790395599372e-05, 'epoch': 0.24}

 24%|██▍       | 9290/37962 [21:32:07<65:37:42,  8.24s/it]
 24%|██▍       | 9291/37962 [21:32:16<66:17:31,  8.32s/it]
 24%|██▍       | 9292/37962 [21:32:24<65:35:39,  8.24s/it]
 24%|██▍       | 9293/37962 [21:32:32<65:50:26,  8.27s/it]
 24%|██▍       | 9294/37962 [21:32:40<64:41:47,  8.12s/it]
 24%|██▍       | 9295/37962 [21:32:48<64:55:22,  8.15s/it]
 24%|██▍       | 9296/37962 [21:32:56<64:02:53,  8.04s/it]
 24%|██▍       | 9297/37962 [21:33:04<64:33:05,  8.11s/it]
 24%|██▍       | 9298/37962 [21:33:12<65:08:27,  8.18s/it]
 24%|██▍       | 9299/37962 [21:33:21<65:46:11,  8.26s/it]
 24%|██▍       | 9300/37962 [21:33:29<64:38:21,  8.12s/it]
                                                          
{'loss': 1.3462, 'grad_norm': 17.503808975219727, 'learning_rate': 1.7672434954027452e-05, 'epoch': 0.24}

 24%|██▍       | 9300/37962 [21:33:29<64:38:21,  8.12s/it]
 25%|██▍       | 9301/37962 [21:33:37<64:41:49,  8.13s/it]
 25%|██▍       | 9302/37962 [21:33:45<65:08:31,  8.18s/it]
 25%|██▍       | 9303/37962 [21:33:54<65:59:24,  8.29s/it]
 25%|██▍       | 9304/37962 [21:34:02<66:09:58,  8.31s/it]
 25%|██▍       | 9305/37962 [21:34:10<65:55:32,  8.28s/it]
 25%|██▍       | 9306/37962 [21:34:18<65:52:05,  8.27s/it]
 25%|██▍       | 9307/37962 [21:34:27<66:10:58,  8.31s/it]
 25%|██▍       | 9308/37962 [21:34:35<66:09:36,  8.31s/it]
 25%|██▍       | 9309/37962 [21:34:44<66:25:44,  8.35s/it]
 25%|██▍       | 9310/37962 [21:34:52<66:30:33,  8.36s/it]
                                                          
{'loss': 1.3758, 'grad_norm': 6.320077896118164, 'learning_rate': 1.76669603674287e-05, 'epoch': 0.25}

 25%|██▍       | 9310/37962 [21:34:52<66:30:33,  8.36s/it]
 25%|██▍       | 9311/37962 [21:35:00<66:11:51,  8.32s/it]
 25%|██▍       | 9312/37962 [21:35:08<65:24:38,  8.22s/it]
 25%|██▍       | 9313/37962 [21:35:16<65:38:20,  8.25s/it]
 25%|██▍       | 9314/37962 [21:35:25<65:34:37,  8.24s/it]
 25%|██▍       | 9315/37962 [21:35:33<66:52:20,  8.40s/it]
 25%|██▍       | 9316/37962 [21:35:42<66:34:46,  8.37s/it]
 25%|██▍       | 9317/37962 [21:35:50<66:22:29,  8.34s/it]
 25%|██▍       | 9318/37962 [21:35:58<65:59:23,  8.29s/it]
 25%|██▍       | 9319/37962 [21:36:06<66:01:22,  8.30s/it]
 25%|██▍       | 9320/37962 [21:36:15<65:31:51,  8.24s/it]
                                                          
{'loss': 1.3541, 'grad_norm': 4.739846706390381, 'learning_rate': 1.7661480200182314e-05, 'epoch': 0.25}

 25%|██▍       | 9320/37962 [21:36:15<65:31:51,  8.24s/it]
 25%|██▍       | 9321/37962 [21:36:23<65:41:00,  8.26s/it]
 25%|██▍       | 9322/37962 [21:36:31<65:40:48,  8.26s/it]
 25%|██▍       | 9323/37962 [21:36:40<66:03:31,  8.30s/it]
 25%|██▍       | 9324/37962 [21:36:48<65:58:12,  8.29s/it]
 25%|██▍       | 9325/37962 [21:36:56<66:11:10,  8.32s/it]
 25%|██▍       | 9326/37962 [21:37:04<66:06:43,  8.31s/it]
 25%|██▍       | 9327/37962 [21:37:13<66:27:43,  8.36s/it]
 25%|██▍       | 9328/37962 [21:37:21<66:04:41,  8.31s/it]
 25%|██▍       | 9329/37962 [21:37:29<65:48:39,  8.27s/it]
 25%|██▍       | 9330/37962 [21:37:38<66:17:14,  8.33s/it]
                                                          
{'loss': 1.3929, 'grad_norm': 5.5355095863342285, 'learning_rate': 1.7655994456277213e-05, 'epoch': 0.25}

 25%|██▍       | 9330/37962 [21:37:38<66:17:14,  8.33s/it]
 25%|██▍       | 9331/37962 [21:37:46<66:09:44,  8.32s/it]
 25%|██▍       | 9332/37962 [21:37:54<65:56:47,  8.29s/it]
 25%|██▍       | 9333/37962 [21:38:03<66:21:00,  8.34s/it]
 25%|██▍       | 9334/37962 [21:38:11<66:27:41,  8.36s/it]
 25%|██▍       | 9335/37962 [21:38:19<66:15:21,  8.33s/it]
 25%|██▍       | 9336/37962 [21:38:28<65:53:02,  8.29s/it]
 25%|██▍       | 9337/37962 [21:38:36<66:02:24,  8.31s/it]
 25%|██▍       | 9338/37962 [21:38:44<65:49:36,  8.28s/it]
 25%|██▍       | 9339/37962 [21:38:52<65:50:19,  8.28s/it]
 25%|██▍       | 9340/37962 [21:39:01<65:53:26,  8.29s/it]
                                                          
{'loss': 1.3842, 'grad_norm': 8.930903434753418, 'learning_rate': 1.7650503139706387e-05, 'epoch': 0.25}

 25%|██▍       | 9340/37962 [21:39:01<65:53:26,  8.29s/it]
 25%|██▍       | 9341/37962 [21:39:09<66:00:39,  8.30s/it]
 25%|██▍       | 9342/37962 [21:39:17<65:46:40,  8.27s/it]
 25%|██▍       | 9343/37962 [21:39:26<65:41:54,  8.26s/it]
 25%|██▍       | 9344/37962 [21:39:34<66:15:22,  8.33s/it]
 25%|██▍       | 9345/37962 [21:39:42<66:14:45,  8.33s/it]
 25%|██▍       | 9346/37962 [21:39:51<65:54:31,  8.29s/it]
 25%|██▍       | 9347/37962 [21:39:59<65:12:26,  8.20s/it]
 25%|██▍       | 9348/37962 [21:40:07<65:30:00,  8.24s/it]
 25%|██▍       | 9349/37962 [21:40:15<65:13:19,  8.21s/it]
 25%|██▍       | 9350/37962 [21:40:23<65:21:16,  8.22s/it]
                                                          
{'loss': 1.4781, 'grad_norm': 38.31852722167969, 'learning_rate': 1.7645006254466852e-05, 'epoch': 0.25}

 25%|██▍       | 9350/37962 [21:40:23<65:21:16,  8.22s/it]
 25%|██▍       | 9351/37962 [21:40:32<65:28:38,  8.24s/it]
 25%|██▍       | 9352/37962 [21:40:40<65:26:46,  8.24s/it]
 25%|██▍       | 9353/37962 [21:40:48<65:19:34,  8.22s/it]
 25%|██▍       | 9354/37962 [21:40:56<65:18:10,  8.22s/it]
 25%|██▍       | 9355/37962 [21:41:05<66:00:17,  8.31s/it]
 25%|██▍       | 9356/37962 [21:41:13<66:15:05,  8.34s/it]
 25%|██▍       | 9357/37962 [21:41:22<66:34:25,  8.38s/it]
 25%|██▍       | 9358/37962 [21:41:30<66:03:26,  8.31s/it]
 25%|██▍       | 9359/37962 [21:41:38<66:14:46,  8.34s/it]
 25%|██▍       | 9360/37962 [21:41:46<65:50:21,  8.29s/it]
                                                          
{'loss': 1.3797, 'grad_norm': 14.614728927612305, 'learning_rate': 1.763950380455971e-05, 'epoch': 0.25}

 25%|██▍       | 9360/37962 [21:41:46<65:50:21,  8.29s/it]
 25%|██▍       | 9361/37962 [21:41:55<66:37:16,  8.39s/it]
 25%|██▍       | 9362/37962 [21:42:03<65:58:17,  8.30s/it]
 25%|██▍       | 9363/37962 [21:42:11<65:36:29,  8.26s/it]
 25%|██▍       | 9364/37962 [21:42:20<65:51:38,  8.29s/it]
 25%|██▍       | 9365/37962 [21:42:28<65:59:08,  8.31s/it]
 25%|██▍       | 9366/37962 [21:42:36<65:58:32,  8.31s/it]
 25%|██▍       | 9367/37962 [21:42:44<65:53:10,  8.29s/it]
 25%|██▍       | 9368/37962 [21:42:53<65:17:54,  8.22s/it]
 25%|██▍       | 9369/37962 [21:43:01<65:27:55,  8.24s/it]
 25%|██▍       | 9370/37962 [21:43:09<64:31:09,  8.12s/it]
                                                          
{'loss': 1.5341, 'grad_norm': 9.36707878112793, 'learning_rate': 1.7633995793990095e-05, 'epoch': 0.25}

 25%|██▍       | 9370/37962 [21:43:09<64:31:09,  8.12s/it]
 25%|██▍       | 9371/37962 [21:43:17<64:39:41,  8.14s/it]
 25%|██▍       | 9372/37962 [21:43:25<64:01:42,  8.06s/it]
 25%|██▍       | 9373/37962 [21:43:33<64:05:43,  8.07s/it]
 25%|██▍       | 9374/37962 [21:43:41<64:07:46,  8.08s/it]
 25%|██▍       | 9375/37962 [21:43:49<64:28:42,  8.12s/it]
 25%|██▍       | 9376/37962 [21:43:57<64:11:06,  8.08s/it]
 25%|██▍       | 9377/37962 [21:44:05<64:38:38,  8.14s/it]
 25%|██▍       | 9378/37962 [21:44:14<64:53:38,  8.17s/it]
 25%|██▍       | 9379/37962 [21:44:22<65:03:49,  8.19s/it]
 25%|██▍       | 9380/37962 [21:44:30<65:16:54,  8.22s/it]
                                                          
{'loss': 1.345, 'grad_norm': 11.94587230682373, 'learning_rate': 1.762848222676719e-05, 'epoch': 0.25}

 25%|██▍       | 9380/37962 [21:44:30<65:16:54,  8.22s/it]
 25%|██▍       | 9381/37962 [21:44:39<65:41:44,  8.27s/it]
 25%|██▍       | 9382/37962 [21:44:47<65:48:25,  8.29s/it]
 25%|██▍       | 9383/37962 [21:44:55<65:51:56,  8.30s/it]
 25%|██▍       | 9384/37962 [21:45:04<65:56:17,  8.31s/it]
 25%|██▍       | 9385/37962 [21:45:12<65:30:37,  8.25s/it]
 25%|██▍       | 9386/37962 [21:45:20<65:32:52,  8.26s/it]
 25%|██▍       | 9387/37962 [21:45:28<65:43:34,  8.28s/it]
 25%|██▍       | 9388/37962 [21:45:37<65:52:20,  8.30s/it]
 25%|██▍       | 9389/37962 [21:45:45<65:05:46,  8.20s/it]
 25%|██▍       | 9390/37962 [21:45:53<65:36:48,  8.27s/it]
                                                          
{'loss': 1.3378, 'grad_norm': 5.756828784942627, 'learning_rate': 1.7622963106904224e-05, 'epoch': 0.25}

 25%|██▍       | 9390/37962 [21:45:53<65:36:48,  8.27s/it]
 25%|██▍       | 9391/37962 [21:46:01<65:28:03,  8.25s/it]
 25%|██▍       | 9392/37962 [21:46:09<65:23:41,  8.24s/it]
 25%|██▍       | 9393/37962 [21:46:18<65:23:54,  8.24s/it]
 25%|██▍       | 9394/37962 [21:46:26<65:29:08,  8.25s/it]
 25%|██▍       | 9395/37962 [21:46:34<66:02:12,  8.32s/it]
 25%|██▍       | 9396/37962 [21:46:43<65:48:21,  8.29s/it]
 25%|██▍       | 9397/37962 [21:46:51<65:44:59,  8.29s/it]
 25%|██▍       | 9398/37962 [21:46:59<66:05:06,  8.33s/it]
 25%|██▍       | 9399/37962 [21:47:08<66:05:06,  8.33s/it]
 25%|██▍       | 9400/37962 [21:47:16<65:53:50,  8.31s/it]
                                                          
{'loss': 1.5249, 'grad_norm': 14.116488456726074, 'learning_rate': 1.7617438438418476e-05, 'epoch': 0.25}

 25%|██▍       | 9400/37962 [21:47:16<65:53:50,  8.31s/it]
 25%|██▍       | 9401/37962 [21:47:24<65:28:01,  8.25s/it]
 25%|██▍       | 9402/37962 [21:47:32<65:22:54,  8.24s/it]
 25%|██▍       | 9403/37962 [21:47:41<65:55:17,  8.31s/it]
 25%|██▍       | 9404/37962 [21:47:49<65:33:08,  8.26s/it]
 25%|██▍       | 9405/37962 [21:47:57<65:05:07,  8.20s/it]
 25%|██▍       | 9406/37962 [21:48:05<64:37:28,  8.15s/it]
 25%|██▍       | 9407/37962 [21:48:13<64:48:21,  8.17s/it]
 25%|██▍       | 9408/37962 [21:48:21<64:27:02,  8.13s/it]
 25%|██▍       | 9409/37962 [21:48:29<64:20:24,  8.11s/it]
 25%|██▍       | 9410/37962 [21:48:38<65:16:47,  8.23s/it]
                                                          
{'loss': 1.3007, 'grad_norm': 4.373140335083008, 'learning_rate': 1.761190822533125e-05, 'epoch': 0.25}

 25%|██▍       | 9410/37962 [21:48:38<65:16:47,  8.23s/it]
 25%|██▍       | 9411/37962 [21:48:46<65:26:14,  8.25s/it]
 25%|██▍       | 9412/37962 [21:48:55<65:57:28,  8.32s/it]
 25%|██▍       | 9413/37962 [21:49:03<65:49:00,  8.30s/it]
 25%|██▍       | 9414/37962 [21:49:11<65:44:48,  8.29s/it]
 25%|██▍       | 9415/37962 [21:49:19<65:43:35,  8.29s/it]
 25%|██▍       | 9416/37962 [21:49:28<65:59:10,  8.32s/it]
 25%|██▍       | 9417/37962 [21:49:36<66:31:11,  8.39s/it]
 25%|██▍       | 9418/37962 [21:49:45<66:23:07,  8.37s/it]
 25%|██▍       | 9419/37962 [21:49:53<66:17:46,  8.36s/it]
 25%|██▍       | 9420/37962 [21:50:02<66:42:13,  8.41s/it]
                                                          
{'loss': 1.45, 'grad_norm': 11.82955265045166, 'learning_rate': 1.7606372471667893e-05, 'epoch': 0.25}

 25%|██▍       | 9420/37962 [21:50:02<66:42:13,  8.41s/it]
 25%|██▍       | 9421/37962 [21:50:10<66:45:45,  8.42s/it]
 25%|██▍       | 9422/37962 [21:50:18<66:13:34,  8.35s/it]
 25%|██▍       | 9423/37962 [21:50:26<65:45:03,  8.29s/it]
 25%|██▍       | 9424/37962 [21:50:35<66:44:02,  8.42s/it]
 25%|██▍       | 9425/37962 [21:50:43<66:05:17,  8.34s/it]
 25%|██▍       | 9426/37962 [21:50:51<65:33:21,  8.27s/it]
 25%|██▍       | 9427/37962 [21:51:00<65:21:31,  8.25s/it]
 25%|██▍       | 9428/37962 [21:51:08<65:53:34,  8.31s/it]
 25%|██▍       | 9429/37962 [21:51:16<65:39:09,  8.28s/it]
 25%|██▍       | 9430/37962 [21:51:24<65:28:21,  8.26s/it]
                                                          
{'loss': 1.4779, 'grad_norm': 14.190711975097656, 'learning_rate': 1.7600831181457785e-05, 'epoch': 0.25}

 25%|██▍       | 9430/37962 [21:51:24<65:28:21,  8.26s/it]
 25%|██▍       | 9431/37962 [21:51:32<64:50:25,  8.18s/it]
 25%|██▍       | 9432/37962 [21:51:41<65:13:51,  8.23s/it]
 25%|██▍       | 9433/37962 [21:51:49<64:49:41,  8.18s/it]
 25%|██▍       | 9434/37962 [21:51:57<64:53:43,  8.19s/it]
 25%|██▍       | 9435/37962 [21:52:05<64:44:29,  8.17s/it]
 25%|██▍       | 9436/37962 [21:52:13<65:07:03,  8.22s/it]
 25%|██▍       | 9437/37962 [21:52:22<65:21:04,  8.25s/it]
 25%|██▍       | 9438/37962 [21:52:30<65:05:04,  8.21s/it]
 25%|██▍       | 9439/37962 [21:52:38<65:26:25,  8.26s/it]
 25%|██▍       | 9440/37962 [21:52:46<64:59:16,  8.20s/it]
                                                          
{'loss': 1.3642, 'grad_norm': 8.529472351074219, 'learning_rate': 1.759528435873434e-05, 'epoch': 0.25}

 25%|██▍       | 9440/37962 [21:52:46<64:59:16,  8.20s/it]
 25%|██▍       | 9441/37962 [21:52:55<65:20:24,  8.25s/it]
 25%|██▍       | 9442/37962 [21:53:03<65:09:01,  8.22s/it]
 25%|██▍       | 9443/37962 [21:53:11<65:03:46,  8.21s/it]
 25%|██▍       | 9444/37962 [21:53:19<65:10:42,  8.23s/it]
 25%|██▍       | 9445/37962 [21:53:28<65:45:50,  8.30s/it]
 25%|██▍       | 9446/37962 [21:53:36<65:37:04,  8.28s/it]
 25%|██▍       | 9447/37962 [21:53:44<65:32:03,  8.27s/it]
 25%|██▍       | 9448/37962 [21:53:53<66:08:55,  8.35s/it]
 25%|██▍       | 9449/37962 [21:54:01<65:55:40,  8.32s/it]
 25%|██▍       | 9450/37962 [21:54:09<65:59:21,  8.33s/it]
                                                          
{'loss': 1.461, 'grad_norm': 24.774723052978516, 'learning_rate': 1.7589732007534984e-05, 'epoch': 0.25}

 25%|██▍       | 9450/37962 [21:54:09<65:59:21,  8.33s/it]
 25%|██▍       | 9451/37962 [21:54:18<66:11:09,  8.36s/it]
 25%|██▍       | 9452/37962 [21:54:26<66:21:09,  8.38s/it]
 25%|██▍       | 9453/37962 [21:54:34<65:43:31,  8.30s/it]
 25%|██▍       | 9454/37962 [21:54:42<64:57:39,  8.20s/it]
 25%|██▍       | 9455/37962 [21:54:50<64:39:11,  8.16s/it]
 25%|██▍       | 9456/37962 [21:54:59<64:43:48,  8.17s/it]
 25%|██▍       | 9457/37962 [21:55:07<65:08:28,  8.23s/it]
 25%|██▍       | 9458/37962 [21:55:15<65:27:14,  8.27s/it]
 25%|██▍       | 9459/37962 [21:55:24<65:34:03,  8.28s/it]
 25%|██▍       | 9460/37962 [21:55:32<65:26:24,  8.27s/it]
                                                          
{'loss': 1.3526, 'grad_norm': 2.724869728088379, 'learning_rate': 1.7584174131901182e-05, 'epoch': 0.25}

 25%|██▍       | 9460/37962 [21:55:32<65:26:24,  8.27s/it]
 25%|██▍       | 9461/37962 [21:55:40<65:11:51,  8.24s/it]
 25%|██▍       | 9462/37962 [21:55:48<64:51:45,  8.19s/it]
 25%|██▍       | 9463/37962 [21:55:57<65:18:16,  8.25s/it]
 25%|██▍       | 9464/37962 [21:56:05<65:27:25,  8.27s/it]
 25%|██▍       | 9465/37962 [21:56:13<65:42:03,  8.30s/it]
 25%|██▍       | 9466/37962 [21:56:22<65:35:11,  8.29s/it]
 25%|██▍       | 9467/37962 [21:56:30<65:37:00,  8.29s/it]
 25%|██▍       | 9468/37962 [21:56:38<65:31:36,  8.28s/it]
 25%|██▍       | 9469/37962 [21:56:46<65:28:59,  8.27s/it]
 25%|██▍       | 9470/37962 [21:56:55<65:28:13,  8.27s/it]
                                                          
{'loss': 1.422, 'grad_norm': 13.814647674560547, 'learning_rate': 1.757861073587842e-05, 'epoch': 0.25}

 25%|██▍       | 9470/37962 [21:56:55<65:28:13,  8.27s/it]
 25%|██▍       | 9471/37962 [21:57:03<65:53:25,  8.33s/it]
 25%|██▍       | 9472/37962 [21:57:11<65:32:58,  8.28s/it]
 25%|██▍       | 9473/37962 [21:57:19<65:26:33,  8.27s/it]
 25%|██▍       | 9474/37962 [21:57:28<65:42:51,  8.30s/it]
 25%|██▍       | 9475/37962 [21:57:36<66:14:37,  8.37s/it]
 25%|██▍       | 9476/37962 [21:57:45<65:40:57,  8.30s/it]
 25%|██▍       | 9477/37962 [21:57:53<65:33:36,  8.29s/it]
 25%|██▍       | 9478/37962 [21:58:01<64:18:21,  8.13s/it]
 25%|██▍       | 9479/37962 [21:58:09<64:45:07,  8.18s/it]
 25%|██▍       | 9480/37962 [21:58:17<64:29:02,  8.15s/it]
                                                          
{'loss': 1.4093, 'grad_norm': 11.529662132263184, 'learning_rate': 1.7573041823516194e-05, 'epoch': 0.25}

 25%|██▍       | 9480/37962 [21:58:17<64:29:02,  8.15s/it]
 25%|██▍       | 9481/37962 [21:58:25<64:22:21,  8.14s/it]
 25%|██▍       | 9482/37962 [21:58:33<64:46:33,  8.19s/it]
 25%|██▍       | 9483/37962 [21:58:42<65:18:19,  8.26s/it]
 25%|██▍       | 9484/37962 [21:58:50<64:23:13,  8.14s/it]
 25%|██▍       | 9485/37962 [21:58:58<64:30:08,  8.15s/it]
 25%|██▍       | 9486/37962 [21:59:06<64:48:53,  8.19s/it]
 25%|██▍       | 9487/37962 [21:59:14<64:33:00,  8.16s/it]
 25%|██▍       | 9488/37962 [21:59:23<65:09:16,  8.24s/it]
 25%|██▍       | 9489/37962 [21:59:31<65:05:54,  8.23s/it]
 25%|██▍       | 9490/37962 [21:59:39<64:15:05,  8.12s/it]
                                                          
{'loss': 1.3466, 'grad_norm': 15.55040454864502, 'learning_rate': 1.7567467398868015e-05, 'epoch': 0.25}

 25%|██▍       | 9490/37962 [21:59:39<64:15:05,  8.12s/it]
 25%|██▌       | 9491/37962 [21:59:47<64:27:15,  8.15s/it]
 25%|██▌       | 9492/37962 [21:59:55<64:57:29,  8.21s/it]
 25%|██▌       | 9493/37962 [22:00:03<65:00:46,  8.22s/it]
 25%|██▌       | 9494/37962 [22:00:11<64:22:01,  8.14s/it]
 25%|██▌       | 9495/37962 [22:00:20<65:01:42,  8.22s/it]
 25%|██▌       | 9496/37962 [22:00:28<64:26:44,  8.15s/it]
 25%|██▌       | 9497/37962 [22:00:36<64:35:17,  8.17s/it]
 25%|██▌       | 9498/37962 [22:00:44<64:14:35,  8.13s/it]
 25%|██▌       | 9499/37962 [22:00:52<64:30:13,  8.16s/it]
 25%|██▌       | 9500/37962 [22:01:01<64:54:11,  8.21s/it]
                                                          
{'loss': 1.34, 'grad_norm': 8.819802284240723, 'learning_rate': 1.7561887465991418e-05, 'epoch': 0.25}

 25%|██▌       | 9500/37962 [22:01:01<64:54:11,  8.21s/it]
 25%|██▌       | 9501/37962 [22:01:09<65:29:52,  8.28s/it]
 25%|██▌       | 9502/37962 [22:01:17<65:14:47,  8.25s/it]
 25%|██▌       | 9503/37962 [22:01:25<65:00:55,  8.22s/it]
 25%|██▌       | 9504/37962 [22:01:34<65:00:04,  8.22s/it]
 25%|██▌       | 9505/37962 [22:01:42<65:04:55,  8.23s/it]
 25%|██▌       | 9506/37962 [22:01:50<64:16:28,  8.13s/it]
 25%|██▌       | 9507/37962 [22:01:58<64:38:58,  8.18s/it]
 25%|██▌       | 9508/37962 [22:02:06<64:59:37,  8.22s/it]
 25%|██▌       | 9509/37962 [22:02:15<65:00:35,  8.23s/it]
 25%|██▌       | 9510/37962 [22:02:23<65:14:26,  8.25s/it]
                                                          
{'loss': 1.4147, 'grad_norm': 11.171469688415527, 'learning_rate': 1.7556302028947932e-05, 'epoch': 0.25}

 25%|██▌       | 9510/37962 [22:02:23<65:14:26,  8.25s/it]
 25%|██▌       | 9511/37962 [22:02:31<65:07:25,  8.24s/it]
 25%|██▌       | 9512/37962 [22:02:39<64:59:07,  8.22s/it]
 25%|██▌       | 9513/37962 [22:02:48<65:19:10,  8.27s/it]
 25%|██▌       | 9514/37962 [22:02:56<65:31:01,  8.29s/it]
 25%|██▌       | 9515/37962 [22:03:04<65:14:47,  8.26s/it]
 25%|██▌       | 9516/37962 [22:03:13<65:41:02,  8.31s/it]
 25%|██▌       | 9517/37962 [22:03:21<65:04:24,  8.24s/it]
 25%|██▌       | 9518/37962 [22:03:29<64:58:54,  8.22s/it]
 25%|██▌       | 9519/37962 [22:03:38<65:46:47,  8.33s/it]
 25%|██▌       | 9520/37962 [22:03:45<64:18:28,  8.14s/it]
                                                          
{'loss': 1.3862, 'grad_norm': 12.487887382507324, 'learning_rate': 1.755071109180311e-05, 'epoch': 0.25}

 25%|██▌       | 9520/37962 [22:03:45<64:18:28,  8.14s/it]
 25%|██▌       | 9521/37962 [22:03:53<64:12:56,  8.13s/it]
 25%|██▌       | 9522/37962 [22:04:02<65:04:22,  8.24s/it]
 25%|██▌       | 9523/37962 [22:04:10<65:29:50,  8.29s/it]
 25%|██▌       | 9524/37962 [22:04:18<65:20:11,  8.27s/it]
 25%|██▌       | 9525/37962 [22:04:27<65:30:46,  8.29s/it]
 25%|██▌       | 9526/37962 [22:04:35<65:18:37,  8.27s/it]
 25%|██▌       | 9527/37962 [22:04:43<65:06:15,  8.24s/it]
 25%|██▌       | 9528/37962 [22:04:51<65:02:11,  8.23s/it]
 25%|██▌       | 9529/37962 [22:05:00<64:54:46,  8.22s/it]
 25%|██▌       | 9530/37962 [22:05:08<64:31:10,  8.17s/it]
                                                          
{'loss': 1.4067, 'grad_norm': 13.787839889526367, 'learning_rate': 1.754511465862648e-05, 'epoch': 0.25}

 25%|██▌       | 9530/37962 [22:05:08<64:31:10,  8.17s/it]
 25%|██▌       | 9531/37962 [22:05:16<64:35:12,  8.18s/it]
 25%|██▌       | 9532/37962 [22:05:24<64:39:38,  8.19s/it]
 25%|██▌       | 9533/37962 [22:05:32<65:11:32,  8.26s/it]
 25%|██▌       | 9534/37962 [22:05:41<64:54:40,  8.22s/it]
 25%|██▌       | 9535/37962 [22:05:49<65:10:22,  8.25s/it]
 25%|██▌       | 9536/37962 [22:05:57<64:43:50,  8.20s/it]
 25%|██▌       | 9537/37962 [22:06:05<65:20:53,  8.28s/it]
 25%|██▌       | 9538/37962 [22:06:13<64:46:45,  8.20s/it]
 25%|██▌       | 9539/37962 [22:06:22<64:51:31,  8.21s/it]
 25%|██▌       | 9540/37962 [22:06:30<64:50:45,  8.21s/it]
                                                          
{'loss': 1.378, 'grad_norm': 6.18507194519043, 'learning_rate': 1.753951273349161e-05, 'epoch': 0.25}

 25%|██▌       | 9540/37962 [22:06:30<64:50:45,  8.21s/it]
 25%|██▌       | 9541/37962 [22:06:38<64:55:59,  8.22s/it]
 25%|██▌       | 9542/37962 [22:06:46<64:31:35,  8.17s/it]
 25%|██▌       | 9543/37962 [22:06:54<64:32:39,  8.18s/it]
 25%|██▌       | 9544/37962 [22:07:03<64:40:40,  8.19s/it]
 25%|██▌       | 9545/37962 [22:07:11<65:29:11,  8.30s/it]
 25%|██▌       | 9546/37962 [22:07:19<64:21:02,  8.15s/it]
 25%|██▌       | 9547/37962 [22:07:27<64:39:16,  8.19s/it]
 25%|██▌       | 9548/37962 [22:07:36<64:56:38,  8.23s/it]
 25%|██▌       | 9549/37962 [22:07:44<64:28:59,  8.17s/it]
 25%|██▌       | 9550/37962 [22:07:52<64:58:22,  8.23s/it]
                                                          
{'loss': 1.4965, 'grad_norm': 4.558582305908203, 'learning_rate': 1.753390532047603e-05, 'epoch': 0.25}

 25%|██▌       | 9550/37962 [22:07:52<64:58:22,  8.23s/it]
 25%|██▌       | 9551/37962 [22:08:00<64:50:33,  8.22s/it]
 25%|██▌       | 9552/37962 [22:08:09<65:10:56,  8.26s/it]
 25%|██▌       | 9553/37962 [22:08:17<65:24:44,  8.29s/it]
 25%|██▌       | 9554/37962 [22:08:25<65:08:13,  8.25s/it]
 25%|██▌       | 9555/37962 [22:08:33<65:13:52,  8.27s/it]
 25%|██▌       | 9556/37962 [22:08:42<65:03:22,  8.24s/it]
 25%|██▌       | 9557/37962 [22:08:50<65:05:58,  8.25s/it]
 25%|██▌       | 9558/37962 [22:08:58<65:19:01,  8.28s/it]
 25%|██▌       | 9559/37962 [22:09:06<65:21:45,  8.28s/it]
 25%|██▌       | 9560/37962 [22:09:15<65:53:25,  8.35s/it]
                                                          
{'loss': 1.3626, 'grad_norm': 5.872384071350098, 'learning_rate': 1.7528292423661283e-05, 'epoch': 0.25}

 25%|██▌       | 9560/37962 [22:09:15<65:53:25,  8.35s/it]
 25%|██▌       | 9561/37962 [22:09:23<65:49:38,  8.34s/it]
 25%|██▌       | 9562/37962 [22:09:32<65:45:15,  8.34s/it]
 25%|██▌       | 9563/37962 [22:09:40<64:59:40,  8.24s/it]
 25%|██▌       | 9564/37962 [22:09:48<64:55:51,  8.23s/it]
 25%|██▌       | 9565/37962 [22:09:56<64:39:51,  8.20s/it]
 25%|██▌       | 9566/37962 [22:10:04<64:58:31,  8.24s/it]
 25%|██▌       | 9567/37962 [22:10:13<64:59:30,  8.24s/it]
 25%|██▌       | 9568/37962 [22:10:21<66:13:47,  8.40s/it]
 25%|██▌       | 9569/37962 [22:10:29<65:03:38,  8.25s/it]
 25%|██▌       | 9570/37962 [22:10:37<65:05:55,  8.25s/it]
                                                          
{'loss': 1.3496, 'grad_norm': 9.10808277130127, 'learning_rate': 1.75226740471329e-05, 'epoch': 0.25}

 25%|██▌       | 9570/37962 [22:10:37<65:05:55,  8.25s/it]
 25%|██▌       | 9571/37962 [22:10:46<65:12:46,  8.27s/it]
 25%|██▌       | 9572/37962 [22:10:54<65:29:54,  8.31s/it]
 25%|██▌       | 9573/37962 [22:11:02<65:21:48,  8.29s/it]
 25%|██▌       | 9574/37962 [22:11:11<65:38:19,  8.32s/it]
 25%|██▌       | 9575/37962 [22:11:19<65:29:59,  8.31s/it]
 25%|██▌       | 9576/37962 [22:11:27<65:20:13,  8.29s/it]
 25%|██▌       | 9577/37962 [22:11:36<65:17:49,  8.28s/it]
 25%|██▌       | 9578/37962 [22:11:44<65:41:16,  8.33s/it]
 25%|██▌       | 9579/37962 [22:11:52<65:26:52,  8.30s/it]
 25%|██▌       | 9580/37962 [22:12:00<65:11:55,  8.27s/it]
                                                          
{'loss': 1.4218, 'grad_norm': 9.000541687011719, 'learning_rate': 1.75170501949804e-05, 'epoch': 0.25}

 25%|██▌       | 9580/37962 [22:12:00<65:11:55,  8.27s/it]
 25%|██▌       | 9581/37962 [22:12:08<64:31:00,  8.18s/it]
 25%|██▌       | 9582/37962 [22:12:17<64:10:19,  8.14s/it]
 25%|██▌       | 9583/37962 [22:12:25<64:47:58,  8.22s/it]
 25%|██▌       | 9584/37962 [22:12:33<64:56:41,  8.24s/it]
 25%|██▌       | 9585/37962 [22:12:42<65:25:25,  8.30s/it]
 25%|██▌       | 9586/37962 [22:12:50<65:38:06,  8.33s/it]
 25%|██▌       | 9587/37962 [22:12:58<65:18:05,  8.28s/it]
 25%|██▌       | 9588/37962 [22:13:07<65:21:22,  8.29s/it]
 25%|██▌       | 9589/37962 [22:13:15<65:27:44,  8.31s/it]
 25%|██▌       | 9590/37962 [22:13:23<65:55:38,  8.37s/it]
                                                          
{'loss': 1.4401, 'grad_norm': 8.382094383239746, 'learning_rate': 1.7511420871297288e-05, 'epoch': 0.25}

 25%|██▌       | 9590/37962 [22:13:23<65:55:38,  8.37s/it]
 25%|██▌       | 9591/37962 [22:13:32<65:44:23,  8.34s/it]
 25%|██▌       | 9592/37962 [22:13:40<65:44:00,  8.34s/it]
 25%|██▌       | 9593/37962 [22:13:48<65:44:37,  8.34s/it]
 25%|██▌       | 9594/37962 [22:13:56<65:10:03,  8.27s/it]
 25%|██▌       | 9595/37962 [22:14:05<65:52:43,  8.36s/it]
 25%|██▌       | 9596/37962 [22:14:13<65:37:19,  8.33s/it]
 25%|██▌       | 9597/37962 [22:14:22<65:27:11,  8.31s/it]
 25%|██▌       | 9598/37962 [22:14:30<65:54:12,  8.36s/it]
 25%|██▌       | 9599/37962 [22:14:38<65:56:18,  8.37s/it]
 25%|██▌       | 9600/37962 [22:14:47<65:23:40,  8.30s/it]
                                                          
{'loss': 1.4416, 'grad_norm': 16.166828155517578, 'learning_rate': 1.7505786080181043e-05, 'epoch': 0.25}

 25%|██▌       | 9600/37962 [22:14:47<65:23:40,  8.30s/it]
 25%|██▌       | 9601/37962 [22:14:55<65:17:42,  8.29s/it]
 25%|██▌       | 9602/37962 [22:15:03<64:34:11,  8.20s/it]
 25%|██▌       | 9603/37962 [22:15:11<64:16:08,  8.16s/it]
 25%|██▌       | 9604/37962 [22:15:19<63:56:08,  8.12s/it]
 25%|██▌       | 9605/37962 [22:15:27<64:19:07,  8.17s/it]
 25%|██▌       | 9606/37962 [22:15:35<64:39:14,  8.21s/it]
 25%|██▌       | 9607/37962 [22:15:43<64:07:06,  8.14s/it]
 25%|██▌       | 9608/37962 [22:15:51<63:51:50,  8.11s/it]
 25%|██▌       | 9609/37962 [22:16:00<64:11:04,  8.15s/it]
 25%|██▌       | 9610/37962 [22:16:08<64:23:06,  8.18s/it]
                                                          
{'loss': 1.404, 'grad_norm': 21.841997146606445, 'learning_rate': 1.7500145825733147e-05, 'epoch': 0.25}

 25%|██▌       | 9610/37962 [22:16:08<64:23:06,  8.18s/it]
 25%|██▌       | 9611/37962 [22:16:16<65:03:24,  8.26s/it]
 25%|██▌       | 9612/37962 [22:16:24<64:21:45,  8.17s/it]
 25%|██▌       | 9613/37962 [22:16:33<64:26:32,  8.18s/it]
 25%|██▌       | 9614/37962 [22:16:41<63:57:07,  8.12s/it]
 25%|██▌       | 9615/37962 [22:16:49<64:29:33,  8.19s/it]
 25%|██▌       | 9616/37962 [22:16:57<64:37:12,  8.21s/it]
 25%|██▌       | 9617/37962 [22:17:05<64:43:05,  8.22s/it]
 25%|██▌       | 9618/37962 [22:17:14<64:43:42,  8.22s/it]
 25%|██▌       | 9619/37962 [22:17:22<65:11:49,  8.28s/it]
 25%|██▌       | 9620/37962 [22:17:30<65:20:59,  8.30s/it]
                                                          
{'loss': 1.5644, 'grad_norm': 10.182059288024902, 'learning_rate': 1.7494500112059033e-05, 'epoch': 0.25}

 25%|██▌       | 9620/37962 [22:17:30<65:20:59,  8.30s/it]
 25%|██▌       | 9621/37962 [22:17:39<65:59:49,  8.38s/it]
 25%|██▌       | 9622/37962 [22:17:47<65:43:54,  8.35s/it]
 25%|██▌       | 9623/37962 [22:17:56<65:38:04,  8.34s/it]
 25%|██▌       | 9624/37962 [22:18:04<65:24:10,  8.31s/it]
 25%|██▌       | 9625/37962 [22:18:12<65:17:44,  8.30s/it]
 25%|██▌       | 9626/37962 [22:18:20<65:07:52,  8.27s/it]
 25%|██▌       | 9627/37962 [22:18:28<64:32:35,  8.20s/it]
 25%|██▌       | 9628/37962 [22:18:37<65:25:14,  8.31s/it]
 25%|██▌       | 9629/37962 [22:18:45<65:13:05,  8.29s/it]
 25%|██▌       | 9630/37962 [22:18:53<64:56:24,  8.25s/it]
                                                          
{'loss': 1.4198, 'grad_norm': 22.30634117126465, 'learning_rate': 1.7488848943268124e-05, 'epoch': 0.25}

 25%|██▌       | 9630/37962 [22:18:53<64:56:24,  8.25s/it]
 25%|██▌       | 9631/37962 [22:19:02<65:35:47,  8.34s/it]
 25%|██▌       | 9632/37962 [22:19:10<65:24:50,  8.31s/it]
 25%|██▌       | 9633/37962 [22:19:18<64:19:08,  8.17s/it]
 25%|██▌       | 9634/37962 [22:19:26<64:12:43,  8.16s/it]
 25%|██▌       | 9635/37962 [22:19:34<64:42:09,  8.22s/it]
 25%|██▌       | 9636/37962 [22:19:43<64:56:02,  8.25s/it]
 25%|██▌       | 9637/37962 [22:19:51<64:48:01,  8.24s/it]
 25%|██▌       | 9638/37962 [22:19:59<64:22:08,  8.18s/it]
 25%|██▌       | 9639/37962 [22:20:07<64:28:16,  8.19s/it]
 25%|██▌       | 9640/37962 [22:20:16<64:40:23,  8.22s/it]
                                                          
{'loss': 1.4334, 'grad_norm': 9.819772720336914, 'learning_rate': 1.7483192323473802e-05, 'epoch': 0.25}

 25%|██▌       | 9640/37962 [22:20:16<64:40:23,  8.22s/it]
 25%|██▌       | 9641/37962 [22:20:24<64:28:29,  8.20s/it]
 25%|██▌       | 9642/37962 [22:20:32<64:32:50,  8.21s/it]
 25%|██▌       | 9643/37962 [22:20:40<64:26:28,  8.19s/it]
 25%|██▌       | 9644/37962 [22:20:48<64:15:03,  8.17s/it]
 25%|██▌       | 9645/37962 [22:20:56<63:57:47,  8.13s/it]
 25%|██▌       | 9646/37962 [22:21:05<64:27:04,  8.19s/it]
 25%|██▌       | 9647/37962 [22:21:13<65:06:09,  8.28s/it]
 25%|██▌       | 9648/37962 [22:21:21<65:26:42,  8.32s/it]
 25%|██▌       | 9649/37962 [22:21:30<65:29:30,  8.33s/it]
 25%|██▌       | 9650/37962 [22:21:38<65:34:54,  8.34s/it]
                                                          
{'loss': 1.3408, 'grad_norm': 9.199249267578125, 'learning_rate': 1.7477530256793422e-05, 'epoch': 0.25}

 25%|██▌       | 9650/37962 [22:21:38<65:34:54,  8.34s/it]
 25%|██▌       | 9651/37962 [22:21:46<64:42:09,  8.23s/it]
 25%|██▌       | 9652/37962 [22:21:55<65:18:16,  8.30s/it]
 25%|██▌       | 9653/37962 [22:22:03<65:35:25,  8.34s/it]
 25%|██▌       | 9654/37962 [22:22:11<65:48:42,  8.37s/it]
 25%|██▌       | 9655/37962 [22:22:20<65:38:17,  8.35s/it]
 25%|██▌       | 9656/37962 [22:22:28<65:22:40,  8.31s/it]
 25%|██▌       | 9657/37962 [22:22:36<65:40:51,  8.35s/it]
 25%|██▌       | 9658/37962 [22:22:45<65:18:19,  8.31s/it]
 25%|██▌       | 9659/37962 [22:22:53<65:04:41,  8.28s/it]
 25%|██▌       | 9660/37962 [22:23:01<64:55:13,  8.26s/it]
                                                          
{'loss': 1.4342, 'grad_norm': 10.712924003601074, 'learning_rate': 1.747186274734831e-05, 'epoch': 0.25}

 25%|██▌       | 9660/37962 [22:23:01<64:55:13,  8.26s/it]
 25%|██▌       | 9661/37962 [22:23:09<64:53:38,  8.25s/it]
 25%|██▌       | 9662/37962 [22:23:18<65:05:51,  8.28s/it]
 25%|██▌       | 9663/37962 [22:23:26<65:15:03,  8.30s/it]
 25%|██▌       | 9664/37962 [22:23:34<65:10:53,  8.29s/it]
 25%|██▌       | 9665/37962 [22:23:43<65:06:50,  8.28s/it]
 25%|██▌       | 9666/37962 [22:23:51<64:44:38,  8.24s/it]
 25%|██▌       | 9667/37962 [22:23:59<64:58:21,  8.27s/it]
 25%|██▌       | 9668/37962 [22:24:07<65:13:24,  8.30s/it]
 25%|██▌       | 9669/37962 [22:24:16<65:23:57,  8.32s/it]
 25%|██▌       | 9670/37962 [22:24:24<65:16:42,  8.31s/it]
                                                          
{'loss': 1.4293, 'grad_norm': 17.250553131103516, 'learning_rate': 1.7466189799263747e-05, 'epoch': 0.25}

 25%|██▌       | 9670/37962 [22:24:24<65:16:42,  8.31s/it]
 25%|██▌       | 9671/37962 [22:24:32<65:15:07,  8.30s/it]
 25%|██▌       | 9672/37962 [22:24:41<65:06:57,  8.29s/it]
 25%|██▌       | 9673/37962 [22:24:49<65:24:11,  8.32s/it]
 25%|██▌       | 9674/37962 [22:24:57<64:37:26,  8.22s/it]
 25%|██▌       | 9675/37962 [22:25:06<65:36:18,  8.35s/it]
 25%|██▌       | 9676/37962 [22:25:14<65:22:13,  8.32s/it]
 25%|██▌       | 9677/37962 [22:25:22<64:45:59,  8.24s/it]
 25%|██▌       | 9678/37962 [22:25:30<65:12:27,  8.30s/it]
 25%|██▌       | 9679/37962 [22:25:39<65:10:12,  8.30s/it]
 25%|██▌       | 9680/37962 [22:25:47<65:02:40,  8.28s/it]
                                                          
{'loss': 1.4058, 'grad_norm': 5.623974323272705, 'learning_rate': 1.746051141666897e-05, 'epoch': 0.25}

 25%|██▌       | 9680/37962 [22:25:47<65:02:40,  8.28s/it]
 26%|██▌       | 9681/37962 [22:25:55<64:57:46,  8.27s/it]
 26%|██▌       | 9682/37962 [22:26:04<65:19:11,  8.32s/it]
 26%|██▌       | 9683/37962 [22:26:12<65:03:19,  8.28s/it]
 26%|██▌       | 9684/37962 [22:26:20<65:15:11,  8.31s/it]
 26%|██▌       | 9685/37962 [22:26:28<65:01:04,  8.28s/it]
 26%|██▌       | 9686/37962 [22:26:37<65:16:21,  8.31s/it]
 26%|██▌       | 9687/37962 [22:26:45<65:16:20,  8.31s/it]
 26%|██▌       | 9688/37962 [22:26:53<65:01:39,  8.28s/it]
 26%|██▌       | 9689/37962 [22:27:02<65:06:54,  8.29s/it]
 26%|██▌       | 9690/37962 [22:27:10<65:30:44,  8.34s/it]
                                                          
{'loss': 1.5155, 'grad_norm': 24.104249954223633, 'learning_rate': 1.745482760369718e-05, 'epoch': 0.26}

 26%|██▌       | 9690/37962 [22:27:10<65:30:44,  8.34s/it]
 26%|██▌       | 9691/37962 [22:27:18<64:38:33,  8.23s/it]
 26%|██▌       | 9692/37962 [22:27:26<63:40:30,  8.11s/it]
 26%|██▌       | 9693/37962 [22:27:34<64:00:25,  8.15s/it]
 26%|██▌       | 9694/37962 [22:27:42<64:21:01,  8.20s/it]
 26%|██▌       | 9695/37962 [22:27:51<64:32:35,  8.22s/it]
 26%|██▌       | 9696/37962 [22:27:59<64:39:10,  8.23s/it]
 26%|██▌       | 9697/37962 [22:28:07<64:52:50,  8.26s/it]
 26%|██▌       | 9698/37962 [22:28:16<65:44:47,  8.37s/it]
 26%|██▌       | 9699/37962 [22:28:24<65:14:53,  8.31s/it]
 26%|██▌       | 9700/37962 [22:28:32<65:29:08,  8.34s/it]
                                                          
{'loss': 1.4951, 'grad_norm': 9.512042045593262, 'learning_rate': 1.7449138364485524e-05, 'epoch': 0.26}

 26%|██▌       | 9700/37962 [22:28:32<65:29:08,  8.34s/it]
 26%|██▌       | 9701/37962 [22:28:41<65:13:19,  8.31s/it]
 26%|██▌       | 9702/37962 [22:28:49<65:17:52,  8.32s/it]
 26%|██▌       | 9703/37962 [22:28:57<64:33:34,  8.22s/it]
 26%|██▌       | 9704/37962 [22:29:05<64:37:54,  8.23s/it]
 26%|██▌       | 9705/37962 [22:29:14<64:49:25,  8.26s/it]
 26%|██▌       | 9706/37962 [22:29:22<64:49:48,  8.26s/it]
 26%|██▌       | 9707/37962 [22:29:30<65:08:20,  8.30s/it]
 26%|██▌       | 9708/37962 [22:29:39<65:12:56,  8.31s/it]
 26%|██▌       | 9709/37962 [22:29:47<64:58:31,  8.28s/it]
 26%|██▌       | 9710/37962 [22:29:55<64:58:00,  8.28s/it]
                                                          
{'loss': 1.2372, 'grad_norm': 7.285160541534424, 'learning_rate': 1.7443443703175095e-05, 'epoch': 0.26}

 26%|██▌       | 9710/37962 [22:29:55<64:58:00,  8.28s/it]
 26%|██▌       | 9711/37962 [22:30:03<65:09:35,  8.30s/it]
 26%|██▌       | 9712/37962 [22:30:12<65:38:36,  8.37s/it]
 26%|██▌       | 9713/37962 [22:30:20<65:22:25,  8.33s/it]
 26%|██▌       | 9714/37962 [22:30:28<65:05:09,  8.29s/it]
 26%|██▌       | 9715/37962 [22:30:37<65:33:43,  8.36s/it]
 26%|██▌       | 9716/37962 [22:30:45<65:39:07,  8.37s/it]
 26%|██▌       | 9717/37962 [22:30:53<65:16:06,  8.32s/it]
 26%|██▌       | 9718/37962 [22:31:02<65:08:25,  8.30s/it]
 26%|██▌       | 9719/37962 [22:31:10<65:09:59,  8.31s/it]
 26%|██▌       | 9720/37962 [22:31:19<65:30:15,  8.35s/it]
                                                          
{'loss': 1.5012, 'grad_norm': 2.373140335083008, 'learning_rate': 1.743774362391095e-05, 'epoch': 0.26}

 26%|██▌       | 9720/37962 [22:31:19<65:30:15,  8.35s/it]
 26%|██▌       | 9721/37962 [22:31:27<64:41:15,  8.25s/it]
 26%|██▌       | 9722/37962 [22:31:35<64:12:23,  8.18s/it]
 26%|██▌       | 9723/37962 [22:31:43<64:39:01,  8.24s/it]
 26%|██▌       | 9724/37962 [22:31:51<64:36:16,  8.24s/it]
 26%|██▌       | 9725/37962 [22:31:59<64:44:30,  8.25s/it]
 26%|██▌       | 9726/37962 [22:32:07<64:01:28,  8.16s/it]
 26%|██▌       | 9727/37962 [22:32:16<64:03:40,  8.17s/it]
 26%|██▌       | 9728/37962 [22:32:24<65:02:12,  8.29s/it]
 26%|██▌       | 9729/37962 [22:32:32<64:55:14,  8.28s/it]
 26%|██▌       | 9730/37962 [22:32:40<64:01:26,  8.16s/it]
                                                          
{'loss': 1.4207, 'grad_norm': 16.646223068237305, 'learning_rate': 1.7432038130842063e-05, 'epoch': 0.26}

 26%|██▌       | 9730/37962 [22:32:40<64:01:26,  8.16s/it]
 26%|██▌       | 9731/37962 [22:32:48<63:39:24,  8.12s/it]
 26%|██▌       | 9732/37962 [22:32:57<64:02:45,  8.17s/it]
 26%|██▌       | 9733/37962 [22:33:05<64:45:20,  8.26s/it]
 26%|██▌       | 9734/37962 [22:33:13<64:28:49,  8.22s/it]
 26%|██▌       | 9735/37962 [22:33:21<64:29:11,  8.22s/it]
 26%|██▌       | 9736/37962 [22:33:30<64:55:48,  8.28s/it]
 26%|██▌       | 9737/37962 [22:33:38<65:08:58,  8.31s/it]
 26%|██▌       | 9738/37962 [22:33:47<65:02:41,  8.30s/it]
 26%|██▌       | 9739/37962 [22:33:55<65:09:09,  8.31s/it]
 26%|██▌       | 9740/37962 [22:34:03<65:48:09,  8.39s/it]
                                                          
{'loss': 1.4876, 'grad_norm': 15.934700012207031, 'learning_rate': 1.7426327228121375e-05, 'epoch': 0.26}

 26%|██▌       | 9740/37962 [22:34:03<65:48:09,  8.39s/it]
 26%|██▌       | 9741/37962 [22:34:12<65:20:22,  8.34s/it]
 26%|██▌       | 9742/37962 [22:34:20<65:09:19,  8.31s/it]
 26%|██▌       | 9743/37962 [22:34:28<65:17:56,  8.33s/it]
 26%|██▌       | 9744/37962 [22:34:36<64:58:27,  8.29s/it]
 26%|██▌       | 9745/37962 [22:34:45<65:17:06,  8.33s/it]
 26%|██▌       | 9746/37962 [22:34:53<65:05:22,  8.30s/it]
 26%|██▌       | 9747/37962 [22:35:01<64:52:55,  8.28s/it]
 26%|██▌       | 9748/37962 [22:35:10<65:11:27,  8.32s/it]
 26%|██▌       | 9749/37962 [22:35:17<63:41:51,  8.13s/it]
 26%|██▌       | 9750/37962 [22:35:26<64:10:26,  8.19s/it]
                                                          
{'loss': 1.2346, 'grad_norm': 11.875810623168945, 'learning_rate': 1.742061091990575e-05, 'epoch': 0.26}

 26%|██▌       | 9750/37962 [22:35:26<64:10:26,  8.19s/it]
 26%|██▌       | 9751/37962 [22:35:34<64:42:02,  8.26s/it]
 26%|██▌       | 9752/37962 [22:35:42<64:21:16,  8.21s/it]
 26%|██▌       | 9753/37962 [22:35:51<64:35:11,  8.24s/it]
 26%|██▌       | 9754/37962 [22:35:59<64:47:56,  8.27s/it]
 26%|██▌       | 9755/37962 [22:36:07<64:42:29,  8.26s/it]
 26%|██▌       | 9756/37962 [22:36:15<64:25:59,  8.22s/it]
 26%|██▌       | 9757/37962 [22:36:23<64:07:41,  8.19s/it]
 26%|██▌       | 9758/37962 [22:36:32<64:19:53,  8.21s/it]
 26%|██▌       | 9759/37962 [22:36:40<64:51:04,  8.28s/it]
 26%|██▌       | 9760/37962 [22:36:48<64:48:25,  8.27s/it]
                                                          
{'loss': 1.3497, 'grad_norm': 10.218939781188965, 'learning_rate': 1.7414889210355992e-05, 'epoch': 0.26}

 26%|██▌       | 9760/37962 [22:36:48<64:48:25,  8.27s/it]
 26%|██▌       | 9761/37962 [22:36:57<64:51:19,  8.28s/it]
 26%|██▌       | 9762/37962 [22:37:05<64:47:50,  8.27s/it]
 26%|██▌       | 9763/37962 [22:37:13<63:23:33,  8.09s/it]
 26%|██▌       | 9764/37962 [22:37:21<64:14:35,  8.20s/it]
 26%|██▌       | 9765/37962 [22:37:29<64:23:09,  8.22s/it]
 26%|██▌       | 9766/37962 [22:37:38<65:11:48,  8.32s/it]
 26%|██▌       | 9767/37962 [22:37:46<65:23:48,  8.35s/it]
 26%|██▌       | 9768/37962 [22:37:55<66:12:14,  8.45s/it]
 26%|██▌       | 9769/37962 [22:38:03<66:08:28,  8.45s/it]
 26%|██▌       | 9770/37962 [22:38:12<66:41:24,  8.52s/it]
                                                          
{'loss': 1.2256, 'grad_norm': 9.011862754821777, 'learning_rate': 1.740916210363683e-05, 'epoch': 0.26}

 26%|██▌       | 9770/37962 [22:38:12<66:41:24,  8.52s/it]
 26%|██▌       | 9771/37962 [22:38:21<66:50:22,  8.54s/it]
 26%|██▌       | 9772/37962 [22:38:29<66:40:08,  8.51s/it]
 26%|██▌       | 9773/37962 [22:38:37<65:57:15,  8.42s/it]
 26%|██▌       | 9774/37962 [22:38:46<65:47:19,  8.40s/it]
 26%|██▌       | 9775/37962 [22:38:54<65:45:50,  8.40s/it]
 26%|██▌       | 9776/37962 [22:39:02<65:37:18,  8.38s/it]
 26%|██▌       | 9777/37962 [22:39:11<65:39:14,  8.39s/it]
 26%|██▌       | 9778/37962 [22:39:19<65:33:16,  8.37s/it]
 26%|██▌       | 9779/37962 [22:39:27<65:14:24,  8.33s/it]
 26%|██▌       | 9780/37962 [22:39:36<65:45:55,  8.40s/it]
                                                          
{'loss': 1.4216, 'grad_norm': 5.259324073791504, 'learning_rate': 1.7403429603916927e-05, 'epoch': 0.26}

 26%|██▌       | 9780/37962 [22:39:36<65:45:55,  8.40s/it]
 26%|██▌       | 9781/37962 [22:39:44<65:41:49,  8.39s/it]
 26%|██▌       | 9782/37962 [22:39:53<66:03:57,  8.44s/it]
 26%|██▌       | 9783/37962 [22:40:01<65:15:38,  8.34s/it]
 26%|██▌       | 9784/37962 [22:40:09<65:28:41,  8.37s/it]
 26%|██▌       | 9785/37962 [22:40:17<64:21:38,  8.22s/it]
 26%|██▌       | 9786/37962 [22:40:25<64:03:24,  8.18s/it]
 26%|██▌       | 9787/37962 [22:40:34<64:17:56,  8.22s/it]
 26%|██▌       | 9788/37962 [22:40:42<64:30:31,  8.24s/it]
 26%|██▌       | 9789/37962 [22:40:50<64:31:51,  8.25s/it]
 26%|██▌       | 9790/37962 [22:40:58<64:27:37,  8.24s/it]
                                                          
{'loss': 1.3095, 'grad_norm': 10.163860321044922, 'learning_rate': 1.7397691715368875e-05, 'epoch': 0.26}

 26%|██▌       | 9790/37962 [22:40:58<64:27:37,  8.24s/it]
 26%|██▌       | 9791/37962 [22:41:07<64:48:51,  8.28s/it]
 26%|██▌       | 9792/37962 [22:41:15<64:14:13,  8.21s/it]
 26%|██▌       | 9793/37962 [22:41:23<64:33:09,  8.25s/it]
 26%|██▌       | 9794/37962 [22:41:31<64:10:20,  8.20s/it]
 26%|██▌       | 9795/37962 [22:41:40<64:44:15,  8.27s/it]
 26%|██▌       | 9796/37962 [22:41:48<64:08:30,  8.20s/it]
 26%|██▌       | 9797/37962 [22:41:56<64:15:27,  8.21s/it]
 26%|██▌       | 9798/37962 [22:42:04<64:13:44,  8.21s/it]
 26%|██▌       | 9799/37962 [22:42:12<63:55:38,  8.17s/it]
 26%|██▌       | 9800/37962 [22:42:21<64:17:21,  8.22s/it]
                                                          
{'loss': 1.424, 'grad_norm': 13.664591789245605, 'learning_rate': 1.7391948442169172e-05, 'epoch': 0.26}

 26%|██▌       | 9800/37962 [22:42:21<64:17:21,  8.22s/it]
 26%|██▌       | 9801/37962 [22:42:29<63:51:27,  8.16s/it]
 26%|██▌       | 9802/37962 [22:42:37<63:19:59,  8.10s/it]
 26%|██▌       | 9803/37962 [22:42:45<63:31:45,  8.12s/it]
 26%|██▌       | 9804/37962 [22:42:53<63:55:05,  8.17s/it]
 26%|██▌       | 9805/37962 [22:43:01<64:04:10,  8.19s/it]
 26%|██▌       | 9806/37962 [22:43:09<62:48:59,  8.03s/it]
 26%|██▌       | 9807/37962 [22:43:17<63:03:56,  8.06s/it]
 26%|██▌       | 9808/37962 [22:43:25<63:28:12,  8.12s/it]
 26%|██▌       | 9809/37962 [22:43:34<63:39:09,  8.14s/it]
 26%|██▌       | 9810/37962 [22:43:42<63:48:11,  8.16s/it]
                                                          
{'loss': 1.3193, 'grad_norm': 15.566182136535645, 'learning_rate': 1.738619978849826e-05, 'epoch': 0.26}

 26%|██▌       | 9810/37962 [22:43:42<63:48:11,  8.16s/it]
 26%|██▌       | 9811/37962 [22:43:50<63:21:15,  8.10s/it]
 26%|██▌       | 9812/37962 [22:43:58<63:17:25,  8.09s/it]
 26%|██▌       | 9813/37962 [22:44:06<63:58:46,  8.18s/it]
 26%|██▌       | 9814/37962 [22:44:14<64:05:17,  8.20s/it]
 26%|██▌       | 9815/37962 [22:44:23<64:18:07,  8.22s/it]
 26%|██▌       | 9816/37962 [22:44:31<64:16:13,  8.22s/it]
 26%|██▌       | 9817/37962 [22:44:39<64:50:34,  8.29s/it]
 26%|██▌       | 9818/37962 [22:44:48<64:59:12,  8.31s/it]
 26%|██▌       | 9819/37962 [22:44:56<64:43:31,  8.28s/it]
 26%|██▌       | 9820/37962 [22:45:04<64:44:18,  8.28s/it]
                                                          
{'loss': 1.2749, 'grad_norm': 6.85333251953125, 'learning_rate': 1.738044575854048e-05, 'epoch': 0.26}

 26%|██▌       | 9820/37962 [22:45:04<64:44:18,  8.28s/it]
 26%|██▌       | 9821/37962 [22:45:12<64:28:59,  8.25s/it]
 26%|██▌       | 9822/37962 [22:45:21<64:47:18,  8.29s/it]
 26%|██▌       | 9823/37962 [22:45:29<64:52:04,  8.30s/it]
 26%|██▌       | 9824/37962 [22:45:37<64:54:34,  8.30s/it]
 26%|██▌       | 9825/37962 [22:45:45<64:13:18,  8.22s/it]
 26%|██▌       | 9826/37962 [22:45:54<64:04:40,  8.20s/it]
 26%|██▌       | 9827/37962 [22:46:02<64:16:33,  8.22s/it]
 26%|██▌       | 9828/37962 [22:46:10<64:27:33,  8.25s/it]
 26%|██▌       | 9829/37962 [22:46:18<64:21:10,  8.23s/it]
 26%|██▌       | 9830/37962 [22:46:27<64:14:45,  8.22s/it]
                                                          
{'loss': 1.4487, 'grad_norm': 12.405961036682129, 'learning_rate': 1.737468635648409e-05, 'epoch': 0.26}

 26%|██▌       | 9830/37962 [22:46:27<64:14:45,  8.22s/it]
 26%|██▌       | 9831/37962 [22:46:35<64:35:34,  8.27s/it]
 26%|██▌       | 9832/37962 [22:46:43<63:59:11,  8.19s/it]
 26%|██▌       | 9833/37962 [22:46:51<64:02:00,  8.20s/it]
 26%|██▌       | 9834/37962 [22:46:59<63:56:05,  8.18s/it]
 26%|██▌       | 9835/37962 [22:47:07<63:17:54,  8.10s/it]
 26%|██▌       | 9836/37962 [22:47:16<63:50:39,  8.17s/it]
 26%|██▌       | 9837/37962 [22:47:24<63:32:49,  8.13s/it]
 26%|██▌       | 9838/37962 [22:47:32<63:36:43,  8.14s/it]
 26%|██▌       | 9839/37962 [22:47:40<64:22:17,  8.24s/it]
 26%|██▌       | 9840/37962 [22:47:48<63:51:57,  8.18s/it]
                                                          
{'loss': 1.3707, 'grad_norm': 2.9343132972717285, 'learning_rate': 1.7368921586521256e-05, 'epoch': 0.26}

 26%|██▌       | 9840/37962 [22:47:48<63:51:57,  8.18s/it]
 26%|██▌       | 9841/37962 [22:47:57<64:08:55,  8.21s/it]
 26%|██▌       | 9842/37962 [22:48:05<64:59:22,  8.32s/it]
 26%|██▌       | 9843/37962 [22:48:14<65:24:08,  8.37s/it]
 26%|██▌       | 9844/37962 [22:48:22<65:35:10,  8.40s/it]
 26%|██▌       | 9845/37962 [22:48:30<65:17:11,  8.36s/it]
 26%|██▌       | 9846/37962 [22:48:39<64:59:22,  8.32s/it]
 26%|██▌       | 9847/37962 [22:48:47<64:51:46,  8.31s/it]
 26%|██▌       | 9848/37962 [22:48:55<64:51:45,  8.31s/it]
 26%|██▌       | 9849/37962 [22:49:04<64:53:00,  8.31s/it]
 26%|██▌       | 9850/37962 [22:49:12<64:48:09,  8.30s/it]
                                                          
{'loss': 1.4033, 'grad_norm': 8.643600463867188, 'learning_rate': 1.736315145284806e-05, 'epoch': 0.26}

 26%|██▌       | 9850/37962 [22:49:12<64:48:09,  8.30s/it]
 26%|██▌       | 9851/37962 [22:49:20<64:32:03,  8.26s/it]
 26%|██▌       | 9852/37962 [22:49:28<64:52:01,  8.31s/it]
 26%|██▌       | 9853/37962 [22:49:37<64:59:00,  8.32s/it]
 26%|██▌       | 9854/37962 [22:49:45<64:44:46,  8.29s/it]
 26%|██▌       | 9855/37962 [22:49:53<64:23:21,  8.25s/it]
 26%|██▌       | 9856/37962 [22:50:02<64:53:40,  8.31s/it]
 26%|██▌       | 9857/37962 [22:50:10<64:49:06,  8.30s/it]
 26%|██▌       | 9858/37962 [22:50:18<64:33:32,  8.27s/it]
 26%|██▌       | 9859/37962 [22:50:26<64:24:04,  8.25s/it]
 26%|██▌       | 9860/37962 [22:50:34<64:09:18,  8.22s/it]
                                                          
{'loss': 1.2536, 'grad_norm': 15.15362548828125, 'learning_rate': 1.7357375959664476e-05, 'epoch': 0.26}

 26%|██▌       | 9860/37962 [22:50:34<64:09:18,  8.22s/it]
 26%|██▌       | 9861/37962 [22:50:43<64:20:51,  8.24s/it]
 26%|██▌       | 9862/37962 [22:50:51<64:27:18,  8.26s/it]
 26%|██▌       | 9863/37962 [22:50:59<64:31:39,  8.27s/it]
 26%|██▌       | 9864/37962 [22:51:07<64:21:39,  8.25s/it]
 26%|██▌       | 9865/37962 [22:51:16<64:25:15,  8.25s/it]
 26%|██▌       | 9866/37962 [22:51:24<63:18:43,  8.11s/it]
 26%|██▌       | 9867/37962 [22:51:32<63:46:34,  8.17s/it]
 26%|██▌       | 9868/37962 [22:51:40<63:54:42,  8.19s/it]
 26%|██▌       | 9869/37962 [22:51:48<64:18:57,  8.24s/it]
 26%|██▌       | 9870/37962 [22:51:57<64:17:57,  8.24s/it]
                                                          
{'loss': 1.4343, 'grad_norm': 10.478927612304688, 'learning_rate': 1.7351595111174387e-05, 'epoch': 0.26}

 26%|██▌       | 9870/37962 [22:51:57<64:17:57,  8.24s/it]
 26%|██▌       | 9871/37962 [22:52:05<64:27:40,  8.26s/it]
 26%|██▌       | 9872/37962 [22:52:13<64:00:46,  8.20s/it]
 26%|██▌       | 9873/37962 [22:52:21<64:17:29,  8.24s/it]
 26%|██▌       | 9874/37962 [22:52:29<63:51:49,  8.19s/it]
 26%|██▌       | 9875/37962 [22:52:38<64:20:28,  8.25s/it]
 26%|██▌       | 9876/37962 [22:52:46<64:13:09,  8.23s/it]
 26%|██▌       | 9877/37962 [22:52:54<64:24:47,  8.26s/it]
 26%|██▌       | 9878/37962 [22:53:03<64:44:40,  8.30s/it]
 26%|██▌       | 9879/37962 [22:53:11<64:46:51,  8.30s/it]
 26%|██▌       | 9880/37962 [22:53:19<64:40:53,  8.29s/it]
                                                          
{'loss': 1.4897, 'grad_norm': 8.922406196594238, 'learning_rate': 1.734580891158558e-05, 'epoch': 0.26}

 26%|██▌       | 9880/37962 [22:53:19<64:40:53,  8.29s/it]
 26%|██▌       | 9881/37962 [22:53:28<64:49:30,  8.31s/it]
 26%|██▌       | 9882/37962 [22:53:36<64:39:12,  8.29s/it]
 26%|██▌       | 9883/37962 [22:53:44<64:40:35,  8.29s/it]
 26%|██▌       | 9884/37962 [22:53:52<64:21:15,  8.25s/it]
 26%|██▌       | 9885/37962 [22:54:01<64:32:14,  8.27s/it]
 26%|██▌       | 9886/37962 [22:54:09<64:19:25,  8.25s/it]
 26%|██▌       | 9887/37962 [22:54:17<64:34:03,  8.28s/it]
 26%|██▌       | 9888/37962 [22:54:25<64:32:45,  8.28s/it]
 26%|██▌       | 9889/37962 [22:54:34<64:17:44,  8.25s/it]
 26%|██▌       | 9890/37962 [22:54:42<64:27:22,  8.27s/it]
                                                          
{'loss': 1.4025, 'grad_norm': 5.50809383392334, 'learning_rate': 1.734001736510972e-05, 'epoch': 0.26}

 26%|██▌       | 9890/37962 [22:54:42<64:27:22,  8.27s/it]
 26%|██▌       | 9891/37962 [22:54:50<64:21:27,  8.25s/it]
 26%|██▌       | 9892/37962 [22:54:58<63:45:31,  8.18s/it]
 26%|██▌       | 9893/37962 [22:55:06<63:57:47,  8.20s/it]
 26%|██▌       | 9894/37962 [22:55:15<63:55:55,  8.20s/it]
 26%|██▌       | 9895/37962 [22:55:23<63:58:02,  8.20s/it]
 26%|██▌       | 9896/37962 [22:55:31<64:17:20,  8.25s/it]
 26%|██▌       | 9897/37962 [22:55:39<64:10:59,  8.23s/it]
 26%|██▌       | 9898/37962 [22:55:48<64:11:51,  8.24s/it]
 26%|██▌       | 9899/37962 [22:55:56<64:18:51,  8.25s/it]
 26%|██▌       | 9900/37962 [22:56:04<64:03:26,  8.22s/it]
                                                          
{'loss': 1.4673, 'grad_norm': 13.72231674194336, 'learning_rate': 1.7334220475962377e-05, 'epoch': 0.26}

 26%|██▌       | 9900/37962 [22:56:04<64:03:26,  8.22s/it]
 26%|██▌       | 9901/37962 [22:56:12<64:22:48,  8.26s/it]
 26%|██▌       | 9902/37962 [22:56:21<64:00:36,  8.21s/it]
 26%|██▌       | 9903/37962 [22:56:29<64:15:23,  8.24s/it]
 26%|██▌       | 9904/37962 [22:56:37<63:54:35,  8.20s/it]
 26%|██▌       | 9905/37962 [22:56:45<64:04:35,  8.22s/it]
 26%|██▌       | 9906/37962 [22:56:53<64:04:41,  8.22s/it]
 26%|██▌       | 9907/37962 [22:57:02<63:57:58,  8.21s/it]
 26%|██▌       | 9908/37962 [22:57:10<64:24:39,  8.27s/it]
 26%|██▌       | 9909/37962 [22:57:18<64:52:35,  8.33s/it]
 26%|██▌       | 9910/37962 [22:57:27<64:51:13,  8.32s/it]
                                                          
{'loss': 1.4346, 'grad_norm': 7.844658851623535, 'learning_rate': 1.7328418248363007e-05, 'epoch': 0.26}

 26%|██▌       | 9910/37962 [22:57:27<64:51:13,  8.32s/it]
 26%|██▌       | 9911/37962 [22:57:35<65:05:45,  8.35s/it]
 26%|██▌       | 9912/37962 [22:57:43<64:49:25,  8.32s/it]
 26%|██▌       | 9913/37962 [22:57:52<64:42:31,  8.31s/it]
 26%|██▌       | 9914/37962 [22:58:00<64:25:10,  8.27s/it]
 26%|██▌       | 9915/37962 [22:58:08<63:56:59,  8.21s/it]
 26%|██▌       | 9916/37962 [22:58:16<63:53:01,  8.20s/it]
 26%|██▌       | 9917/37962 [22:58:25<64:33:29,  8.29s/it]
 26%|██▌       | 9918/37962 [22:58:33<64:28:09,  8.28s/it]
 26%|██▌       | 9919/37962 [22:58:41<64:21:47,  8.26s/it]
 26%|██▌       | 9920/37962 [22:58:49<64:10:10,  8.24s/it]
                                                          
{'loss': 1.3521, 'grad_norm': 5.5728440284729, 'learning_rate': 1.7322610686534952e-05, 'epoch': 0.26}

 26%|██▌       | 9920/37962 [22:58:49<64:10:10,  8.24s/it]
 26%|██▌       | 9921/37962 [22:58:57<62:58:16,  8.08s/it]
 26%|██▌       | 9922/37962 [22:59:05<63:12:17,  8.11s/it]
 26%|██▌       | 9923/37962 [22:59:13<63:15:54,  8.12s/it]
 26%|██▌       | 9924/37962 [22:59:22<63:43:58,  8.18s/it]
 26%|██▌       | 9925/37962 [22:59:30<63:11:18,  8.11s/it]
 26%|██▌       | 9926/37962 [22:59:38<63:27:07,  8.15s/it]
 26%|██▌       | 9927/37962 [22:59:46<63:03:40,  8.10s/it]
 26%|██▌       | 9928/37962 [22:59:54<62:51:05,  8.07s/it]
 26%|██▌       | 9929/37962 [23:00:02<63:14:51,  8.12s/it]
 26%|██▌       | 9930/37962 [23:00:10<62:56:23,  8.08s/it]
                                                          
{'loss': 1.5188, 'grad_norm': 9.070394515991211, 'learning_rate': 1.7316797794705432e-05, 'epoch': 0.26}

 26%|██▌       | 9930/37962 [23:00:10<62:56:23,  8.08s/it]
 26%|██▌       | 9931/37962 [23:00:18<63:19:55,  8.13s/it]
 26%|██▌       | 9932/37962 [23:00:27<63:59:53,  8.22s/it]
 26%|██▌       | 9933/37962 [23:00:35<64:36:49,  8.30s/it]
 26%|██▌       | 9934/37962 [23:00:44<64:29:01,  8.28s/it]
 26%|██▌       | 9935/37962 [23:00:51<63:33:31,  8.16s/it]
 26%|██▌       | 9936/37962 [23:00:59<62:59:22,  8.09s/it]
 26%|██▌       | 9937/37962 [23:01:08<63:27:26,  8.15s/it]
 26%|██▌       | 9938/37962 [23:01:16<63:58:13,  8.22s/it]
 26%|██▌       | 9939/37962 [23:01:24<64:09:42,  8.24s/it]
 26%|██▌       | 9940/37962 [23:01:33<64:24:06,  8.27s/it]
                                                          
{'loss': 1.3805, 'grad_norm': 7.583191871643066, 'learning_rate': 1.731097957710555e-05, 'epoch': 0.26}

 26%|██▌       | 9940/37962 [23:01:33<64:24:06,  8.27s/it]
 26%|██▌       | 9941/37962 [23:01:41<64:48:30,  8.33s/it]
 26%|██▌       | 9942/37962 [23:01:49<64:45:50,  8.32s/it]
 26%|██▌       | 9943/37962 [23:01:58<64:21:11,  8.27s/it]
 26%|██▌       | 9944/37962 [23:02:06<64:32:18,  8.29s/it]
 26%|██▌       | 9945/37962 [23:02:14<64:47:28,  8.33s/it]
 26%|██▌       | 9946/37962 [23:02:23<64:41:28,  8.31s/it]
 26%|██▌       | 9947/37962 [23:02:31<65:02:03,  8.36s/it]
 26%|██▌       | 9948/37962 [23:02:39<64:38:13,  8.31s/it]
 26%|██▌       | 9949/37962 [23:02:48<64:48:40,  8.33s/it]
 26%|██▌       | 9950/37962 [23:02:56<63:54:59,  8.21s/it]
                                                          
{'loss': 1.4032, 'grad_norm': 7.576006889343262, 'learning_rate': 1.7305156037970293e-05, 'epoch': 0.26}

 26%|██▌       | 9950/37962 [23:02:56<63:54:59,  8.21s/it]
 26%|██▌       | 9951/37962 [23:03:04<64:48:57,  8.33s/it]
 26%|██▌       | 9952/37962 [23:03:13<65:01:45,  8.36s/it]
 26%|██▌       | 9953/37962 [23:03:21<64:34:56,  8.30s/it]
 26%|██▌       | 9954/37962 [23:03:29<64:18:05,  8.26s/it]
 26%|██▌       | 9955/37962 [23:03:37<64:26:28,  8.28s/it]
 26%|██▌       | 9956/37962 [23:03:45<64:15:18,  8.26s/it]
 26%|██▌       | 9957/37962 [23:03:54<64:36:02,  8.30s/it]
 26%|██▌       | 9958/37962 [23:04:02<64:23:21,  8.28s/it]
 26%|██▌       | 9959/37962 [23:04:10<64:22:58,  8.28s/it]
 26%|██▌       | 9960/37962 [23:04:18<63:56:00,  8.22s/it]
                                                          
{'loss': 1.4457, 'grad_norm': 14.709942817687988, 'learning_rate': 1.7299327181538506e-05, 'epoch': 0.26}

 26%|██▌       | 9960/37962 [23:04:18<63:56:00,  8.22s/it]
 26%|██▌       | 9961/37962 [23:04:27<64:07:48,  8.25s/it]
 26%|██▌       | 9962/37962 [23:04:35<64:22:38,  8.28s/it]
 26%|██▌       | 9963/37962 [23:04:43<64:34:56,  8.30s/it]
 26%|██▌       | 9964/37962 [23:04:52<64:29:43,  8.29s/it]
 26%|██▌       | 9965/37962 [23:05:00<63:59:10,  8.23s/it]
 26%|██▋       | 9966/37962 [23:05:08<64:19:32,  8.27s/it]
 26%|██▋       | 9967/37962 [23:05:16<64:20:04,  8.27s/it]
 26%|██▋       | 9968/37962 [23:05:24<63:47:59,  8.20s/it]
 26%|██▋       | 9969/37962 [23:05:33<63:57:34,  8.23s/it]
 26%|██▋       | 9970/37962 [23:05:41<64:42:22,  8.32s/it]
                                                          
{'loss': 1.3365, 'grad_norm': 6.896272659301758, 'learning_rate': 1.7293493012052917e-05, 'epoch': 0.26}

 26%|██▋       | 9970/37962 [23:05:41<64:42:22,  8.32s/it]
 26%|██▋       | 9971/37962 [23:05:49<64:10:18,  8.25s/it]
 26%|██▋       | 9972/37962 [23:05:58<64:03:36,  8.24s/it]
 26%|██▋       | 9973/37962 [23:06:06<64:00:17,  8.23s/it]
 26%|██▋       | 9974/37962 [23:06:14<63:55:44,  8.22s/it]
 26%|██▋       | 9975/37962 [23:06:22<63:49:39,  8.21s/it]
 26%|██▋       | 9976/37962 [23:06:31<64:14:29,  8.26s/it]
 26%|██▋       | 9977/37962 [23:06:39<63:27:58,  8.16s/it]
 26%|██▋       | 9978/37962 [23:06:47<64:08:44,  8.25s/it]
 26%|██▋       | 9979/37962 [23:06:55<64:24:05,  8.29s/it]
 26%|██▋       | 9980/37962 [23:07:04<64:32:17,  8.30s/it]
                                                          
{'loss': 1.4193, 'grad_norm': 15.76815414428711, 'learning_rate': 1.7287653533760114e-05, 'epoch': 0.26}

 26%|██▋       | 9980/37962 [23:07:04<64:32:17,  8.30s/it]
 26%|██▋       | 9981/37962 [23:07:12<64:11:11,  8.26s/it]
 26%|██▋       | 9982/37962 [23:07:20<63:04:55,  8.12s/it]
 26%|██▋       | 9983/37962 [23:07:28<63:40:33,  8.19s/it]
 26%|██▋       | 9984/37962 [23:07:36<64:07:01,  8.25s/it]
 26%|██▋       | 9985/37962 [23:07:45<63:59:28,  8.23s/it]
 26%|██▋       | 9986/37962 [23:07:53<63:27:43,  8.17s/it]
 26%|██▋       | 9987/37962 [23:08:01<63:44:35,  8.20s/it]
 26%|██▋       | 9988/37962 [23:08:09<64:01:02,  8.24s/it]
 26%|██▋       | 9989/37962 [23:08:17<63:54:22,  8.22s/it]
 26%|██▋       | 9990/37962 [23:08:25<63:35:20,  8.18s/it]
                                                          
{'loss': 1.4703, 'grad_norm': 15.889115333557129, 'learning_rate': 1.7281808750910554e-05, 'epoch': 0.26}

 26%|██▋       | 9990/37962 [23:08:25<63:35:20,  8.18s/it]
 26%|██▋       | 9991/37962 [23:08:34<63:59:41,  8.24s/it]
 26%|██▋       | 9992/37962 [23:08:42<63:34:44,  8.18s/it]
 26%|██▋       | 9993/37962 [23:08:50<63:55:05,  8.23s/it]
 26%|██▋       | 9994/37962 [23:08:58<63:45:43,  8.21s/it]
 26%|██▋       | 9995/37962 [23:09:07<64:06:25,  8.25s/it]
 26%|██▋       | 9996/37962 [23:09:15<64:28:38,  8.30s/it]
 26%|██▋       | 9997/37962 [23:09:23<64:20:56,  8.28s/it]
 26%|██▋       | 9998/37962 [23:09:32<64:35:33,  8.32s/it]
 26%|██▋       | 9999/37962 [23:09:40<63:56:21,  8.23s/it]
 26%|██▋       | 10000/37962 [23:09:48<63:18:02,  8.15s/it]
                                                           
{'loss': 1.4275, 'grad_norm': 27.858835220336914, 'learning_rate': 1.7275958667758553e-05, 'epoch': 0.26}

 26%|██▋       | 10000/37962 [23:09:48<63:18:02,  8.15s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 26%|██▋       | 10001/37962 [23:10:38<160:13:01, 20.63s/it]
 26%|██▋       | 10002/37962 [23:10:45<130:38:42, 16.82s/it]
 26%|██▋       | 10003/37962 [23:10:54<110:40:26, 14.25s/it]
 26%|██▋       | 10004/37962 [23:11:02<97:05:03, 12.50s/it] 
 26%|██▋       | 10005/37962 [23:11:11<87:30:55, 11.27s/it]
 26%|██▋       | 10006/37962 [23:11:19<81:04:11, 10.44s/it]
 26%|██▋       | 10007/37962 [23:11:28<76:39:35,  9.87s/it]
 26%|██▋       | 10008/37962 [23:11:36<73:55:56,  9.52s/it]
 26%|██▋       | 10009/37962 [23:11:45<71:26:22,  9.20s/it]
 26%|██▋       | 10010/37962 [23:11:53<69:21:39,  8.93s/it]
                                                           
{'loss': 1.5337, 'grad_norm': 10.806760787963867, 'learning_rate': 1.7270103288562286e-05, 'epoch': 0.26}

 26%|██▋       | 10010/37962 [23:11:53<69:21:39,  8.93s/it]
 26%|██▋       | 10011/37962 [23:12:01<67:34:01,  8.70s/it]
 26%|██▋       | 10012/37962 [23:12:10<66:39:04,  8.58s/it]
 26%|██▋       | 10013/37962 [23:12:18<66:56:16,  8.62s/it]
 26%|██▋       | 10014/37962 [23:12:27<66:41:46,  8.59s/it]
 26%|██▋       | 10015/37962 [23:12:35<65:32:31,  8.44s/it]
 26%|██▋       | 10016/37962 [23:12:43<65:55:56,  8.49s/it]
 26%|██▋       | 10017/37962 [23:12:52<65:35:03,  8.45s/it]
 26%|██▋       | 10018/37962 [23:13:01<66:14:38,  8.53s/it]
 26%|██▋       | 10019/37962 [23:13:09<66:29:08,  8.57s/it]
 26%|██▋       | 10020/37962 [23:13:17<65:30:06,  8.44s/it]
                                                           
{'loss': 1.428, 'grad_norm': 10.30576229095459, 'learning_rate': 1.7264242617583778e-05, 'epoch': 0.26}

 26%|██▋       | 10020/37962 [23:13:17<65:30:06,  8.44s/it]
 26%|██▋       | 10021/37962 [23:13:26<65:51:31,  8.49s/it]
 26%|██▋       | 10022/37962 [23:13:35<66:31:13,  8.57s/it]
 26%|██▋       | 10023/37962 [23:13:43<65:53:23,  8.49s/it]
 26%|██▋       | 10024/37962 [23:13:51<65:42:11,  8.47s/it]
 26%|██▋       | 10025/37962 [23:14:00<65:48:11,  8.48s/it]
 26%|██▋       | 10026/37962 [23:14:09<66:10:06,  8.53s/it]
 26%|██▋       | 10027/37962 [23:14:17<65:43:15,  8.47s/it]
 26%|██▋       | 10028/37962 [23:14:25<64:43:45,  8.34s/it]
 26%|██▋       | 10029/37962 [23:14:34<65:23:20,  8.43s/it]
 26%|██▋       | 10030/37962 [23:14:42<66:07:19,  8.52s/it]
                                                           
{'loss': 1.3752, 'grad_norm': 9.926338195800781, 'learning_rate': 1.725837665908892e-05, 'epoch': 0.26}

 26%|██▋       | 10030/37962 [23:14:42<66:07:19,  8.52s/it]
 26%|██▋       | 10031/37962 [23:14:51<66:55:52,  8.63s/it]
 26%|██▋       | 10032/37962 [23:15:00<67:43:11,  8.73s/it]
 26%|██▋       | 10033/37962 [23:15:09<67:25:19,  8.69s/it]
 26%|██▋       | 10034/37962 [23:15:18<67:37:33,  8.72s/it]
 26%|██▋       | 10035/37962 [23:15:26<67:47:14,  8.74s/it]
 26%|██▋       | 10036/37962 [23:15:35<67:32:19,  8.71s/it]
 26%|██▋       | 10037/37962 [23:15:43<66:41:18,  8.60s/it]
 26%|██▋       | 10038/37962 [23:15:52<66:39:20,  8.59s/it]
 26%|██▋       | 10039/37962 [23:16:01<67:04:19,  8.65s/it]
 26%|██▋       | 10040/37962 [23:16:09<67:18:18,  8.68s/it]
                                                           
{'loss': 1.498, 'grad_norm': 7.84537935256958, 'learning_rate': 1.725250541734743e-05, 'epoch': 0.26}

 26%|██▋       | 10040/37962 [23:16:09<67:18:18,  8.68s/it]
 26%|██▋       | 10041/37962 [23:16:18<67:20:32,  8.68s/it]
 26%|██▋       | 10042/37962 [23:16:27<67:25:02,  8.69s/it]
 26%|██▋       | 10043/37962 [23:16:35<67:24:03,  8.69s/it]
 26%|██▋       | 10044/37962 [23:16:44<68:04:03,  8.78s/it]
 26%|██▋       | 10045/37962 [23:16:53<67:28:45,  8.70s/it]
 26%|██▋       | 10046/37962 [23:17:01<66:20:18,  8.55s/it]
 26%|██▋       | 10047/37962 [23:17:10<66:41:10,  8.60s/it]
 26%|██▋       | 10048/37962 [23:17:18<66:27:12,  8.57s/it]
 26%|██▋       | 10049/37962 [23:17:27<66:47:36,  8.61s/it]
 26%|██▋       | 10050/37962 [23:17:36<67:26:55,  8.70s/it]
                                                           
{'loss': 1.4404, 'grad_norm': 9.879533767700195, 'learning_rate': 1.7246628896632888e-05, 'epoch': 0.26}

 26%|██▋       | 10050/37962 [23:17:36<67:26:55,  8.70s/it]
 26%|██▋       | 10051/37962 [23:17:45<66:57:02,  8.64s/it]
 26%|██▋       | 10052/37962 [23:17:53<66:58:27,  8.64s/it]
 26%|██▋       | 10053/37962 [23:18:02<67:02:35,  8.65s/it]
 26%|██▋       | 10054/37962 [23:18:10<65:55:39,  8.50s/it]
 26%|██▋       | 10055/37962 [23:18:18<65:42:32,  8.48s/it]
 26%|██▋       | 10056/37962 [23:18:27<66:17:17,  8.55s/it]
 26%|██▋       | 10057/37962 [23:18:36<66:08:42,  8.53s/it]
 26%|██▋       | 10058/37962 [23:18:44<66:11:08,  8.54s/it]
 26%|██▋       | 10059/37962 [23:18:53<66:13:10,  8.54s/it]
 27%|██▋       | 10060/37962 [23:19:01<66:27:57,  8.58s/it]
                                                           
{'loss': 1.4357, 'grad_norm': 4.346865653991699, 'learning_rate': 1.724074710122271e-05, 'epoch': 0.26}

 27%|██▋       | 10060/37962 [23:19:01<66:27:57,  8.58s/it]
 27%|██▋       | 10061/37962 [23:19:10<65:56:16,  8.51s/it]
 27%|██▋       | 10062/37962 [23:19:18<65:46:13,  8.49s/it]
 27%|██▋       | 10063/37962 [23:19:27<65:57:31,  8.51s/it]
 27%|██▋       | 10064/37962 [23:19:35<65:46:07,  8.49s/it]
 27%|██▋       | 10065/37962 [23:19:44<65:38:38,  8.47s/it]
 27%|██▋       | 10066/37962 [23:19:52<65:36:52,  8.47s/it]
 27%|██▋       | 10067/37962 [23:20:01<65:46:45,  8.49s/it]
 27%|██▋       | 10068/37962 [23:20:09<65:40:32,  8.48s/it]
 27%|██▋       | 10069/37962 [23:20:18<65:39:07,  8.47s/it]
 27%|██▋       | 10070/37962 [23:20:26<65:42:45,  8.48s/it]
                                                           
{'loss': 1.3869, 'grad_norm': 3.60611629486084, 'learning_rate': 1.7234860035398155e-05, 'epoch': 0.27}

 27%|██▋       | 10070/37962 [23:20:26<65:42:45,  8.48s/it]
 27%|██▋       | 10071/37962 [23:20:35<65:59:30,  8.52s/it]
 27%|██▋       | 10072/37962 [23:20:43<65:51:26,  8.50s/it]
 27%|██▋       | 10073/37962 [23:20:51<65:29:45,  8.45s/it]
 27%|██▋       | 10074/37962 [23:20:59<63:55:29,  8.25s/it]
 27%|██▋       | 10075/37962 [23:21:07<64:01:10,  8.26s/it]
 27%|██▋       | 10076/37962 [23:21:16<64:02:45,  8.27s/it]
 27%|██▋       | 10077/37962 [23:21:24<63:58:23,  8.26s/it]
 27%|██▋       | 10078/37962 [23:21:33<64:33:05,  8.33s/it]
 27%|██▋       | 10079/37962 [23:21:41<64:35:18,  8.34s/it]
 27%|██▋       | 10080/37962 [23:21:49<64:00:41,  8.26s/it]
                                                           
{'loss': 1.5483, 'grad_norm': 8.183425903320312, 'learning_rate': 1.722896770344432e-05, 'epoch': 0.27}

 27%|██▋       | 10080/37962 [23:21:49<64:00:41,  8.26s/it]
 27%|██▋       | 10081/37962 [23:21:57<63:47:02,  8.24s/it]
 27%|██▋       | 10082/37962 [23:22:06<64:11:37,  8.29s/it]
 27%|██▋       | 10083/37962 [23:22:14<64:34:23,  8.34s/it]
 27%|██▋       | 10084/37962 [23:22:22<64:20:38,  8.31s/it]
 27%|██▋       | 10085/37962 [23:22:31<64:34:47,  8.34s/it]
 27%|██▋       | 10086/37962 [23:22:39<64:16:44,  8.30s/it]
 27%|██▋       | 10087/37962 [23:22:47<64:29:58,  8.33s/it]
 27%|██▋       | 10088/37962 [23:22:56<64:26:33,  8.32s/it]
 27%|██▋       | 10089/37962 [23:23:04<64:28:55,  8.33s/it]
 27%|██▋       | 10090/37962 [23:23:12<63:52:19,  8.25s/it]
                                                           
{'loss': 1.4294, 'grad_norm': 14.513091087341309, 'learning_rate': 1.722307010965012e-05, 'epoch': 0.27}

 27%|██▋       | 10090/37962 [23:23:12<63:52:19,  8.25s/it]
 27%|██▋       | 10091/37962 [23:23:20<64:01:35,  8.27s/it]
 27%|██▋       | 10092/37962 [23:23:29<64:14:39,  8.30s/it]
 27%|██▋       | 10093/37962 [23:23:37<64:25:35,  8.32s/it]
 27%|██▋       | 10094/37962 [23:23:45<63:39:08,  8.22s/it]
 27%|██▋       | 10095/37962 [23:23:54<64:15:01,  8.30s/it]
 27%|██▋       | 10096/37962 [23:24:02<64:03:28,  8.28s/it]
 27%|██▋       | 10097/37962 [23:24:10<64:03:31,  8.28s/it]
 27%|██▋       | 10098/37962 [23:24:18<64:01:15,  8.27s/it]
 27%|██▋       | 10099/37962 [23:24:26<63:37:46,  8.22s/it]
 27%|██▋       | 10100/37962 [23:24:35<64:03:05,  8.28s/it]
                                                           
{'loss': 1.3358, 'grad_norm': 4.75250244140625, 'learning_rate': 1.7217167258308323e-05, 'epoch': 0.27}

 27%|██▋       | 10100/37962 [23:24:35<64:03:05,  8.28s/it]
 27%|██▋       | 10101/37962 [23:24:43<63:59:19,  8.27s/it]
 27%|██▋       | 10102/37962 [23:24:51<64:11:44,  8.30s/it]
 27%|██▋       | 10103/37962 [23:25:00<64:07:06,  8.29s/it]
 27%|██▋       | 10104/37962 [23:25:08<64:26:43,  8.33s/it]
 27%|██▋       | 10105/37962 [23:25:16<64:34:38,  8.35s/it]
 27%|██▋       | 10106/37962 [23:25:25<64:51:18,  8.38s/it]
 27%|██▋       | 10107/37962 [23:25:33<64:25:23,  8.33s/it]
 27%|██▋       | 10108/37962 [23:25:41<64:07:23,  8.29s/it]
 27%|██▋       | 10109/37962 [23:25:50<64:21:44,  8.32s/it]
 27%|██▋       | 10110/37962 [23:25:58<64:26:46,  8.33s/it]
                                                           
{'loss': 1.4284, 'grad_norm': 17.50128173828125, 'learning_rate': 1.7211259153715504e-05, 'epoch': 0.27}

 27%|██▋       | 10110/37962 [23:25:58<64:26:46,  8.33s/it]
 27%|██▋       | 10111/37962 [23:26:06<64:28:04,  8.33s/it]
 27%|██▋       | 10112/37962 [23:26:15<64:13:03,  8.30s/it]
 27%|██▋       | 10113/37962 [23:26:23<64:13:38,  8.30s/it]
 27%|██▋       | 10114/37962 [23:26:32<64:51:23,  8.38s/it]
 27%|██▋       | 10115/37962 [23:26:40<64:27:04,  8.33s/it]
 27%|██▋       | 10116/37962 [23:26:48<64:18:36,  8.31s/it]
 27%|██▋       | 10117/37962 [23:26:56<63:33:23,  8.22s/it]
 27%|██▋       | 10118/37962 [23:27:04<63:58:06,  8.27s/it]
 27%|██▋       | 10119/37962 [23:27:13<63:58:54,  8.27s/it]
 27%|██▋       | 10120/37962 [23:27:21<63:55:40,  8.27s/it]
                                                           
{'loss': 1.4892, 'grad_norm': 14.623844146728516, 'learning_rate': 1.720534580017208e-05, 'epoch': 0.27}

 27%|██▋       | 10120/37962 [23:27:21<63:55:40,  8.27s/it]
 27%|██▋       | 10121/37962 [23:27:29<63:40:31,  8.23s/it]
 27%|██▋       | 10122/37962 [23:27:37<63:21:29,  8.19s/it]
 27%|██▋       | 10123/37962 [23:27:45<63:13:22,  8.18s/it]
 27%|██▋       | 10124/37962 [23:27:54<63:40:15,  8.23s/it]
 27%|██▋       | 10125/37962 [23:28:02<64:00:24,  8.28s/it]
 27%|██▋       | 10126/37962 [23:28:10<64:01:53,  8.28s/it]
 27%|██▋       | 10127/37962 [23:28:19<63:54:06,  8.26s/it]
 27%|██▋       | 10128/37962 [23:28:27<63:56:51,  8.27s/it]
 27%|██▋       | 10129/37962 [23:28:35<63:56:51,  8.27s/it]
 27%|██▋       | 10130/37962 [23:28:43<63:59:21,  8.28s/it]
                                                           
{'loss': 1.4276, 'grad_norm': 15.394331932067871, 'learning_rate': 1.7199427201982268e-05, 'epoch': 0.27}

 27%|██▋       | 10130/37962 [23:28:43<63:59:21,  8.28s/it]
 27%|██▋       | 10131/37962 [23:28:52<63:44:31,  8.25s/it]
 27%|██▋       | 10132/37962 [23:29:00<63:48:23,  8.25s/it]
 27%|██▋       | 10133/37962 [23:29:08<64:07:39,  8.30s/it]
 27%|██▋       | 10134/37962 [23:29:17<64:22:26,  8.33s/it]
 27%|██▋       | 10135/37962 [23:29:25<64:15:34,  8.31s/it]
 27%|██▋       | 10136/37962 [23:29:33<63:37:34,  8.23s/it]
 27%|██▋       | 10137/37962 [23:29:41<63:37:00,  8.23s/it]
 27%|██▋       | 10138/37962 [23:29:50<64:22:45,  8.33s/it]
 27%|██▋       | 10139/37962 [23:29:58<64:11:52,  8.31s/it]
 27%|██▋       | 10140/37962 [23:30:06<64:06:37,  8.30s/it]
                                                           
{'loss': 1.4053, 'grad_norm': 7.127656936645508, 'learning_rate': 1.7193503363454115e-05, 'epoch': 0.27}

 27%|██▋       | 10140/37962 [23:30:06<64:06:37,  8.30s/it]
 27%|██▋       | 10141/37962 [23:30:15<64:37:35,  8.36s/it]
 27%|██▋       | 10142/37962 [23:30:23<64:01:41,  8.29s/it]
 27%|██▋       | 10143/37962 [23:30:31<64:07:48,  8.30s/it]
 27%|██▋       | 10144/37962 [23:30:40<64:21:44,  8.33s/it]
 27%|██▋       | 10145/37962 [23:30:48<64:38:58,  8.37s/it]
 27%|██▋       | 10146/37962 [23:30:56<64:18:34,  8.32s/it]
 27%|██▋       | 10147/37962 [23:31:05<64:20:04,  8.33s/it]
 27%|██▋       | 10148/37962 [23:31:13<64:18:19,  8.32s/it]
 27%|██▋       | 10149/37962 [23:31:28<79:23:27, 10.28s/it]
 27%|██▋       | 10150/37962 [23:31:36<74:08:35,  9.60s/it]
                                                           
{'loss': 1.3784, 'grad_norm': 9.381247520446777, 'learning_rate': 1.7187574288899486e-05, 'epoch': 0.27}

 27%|██▋       | 10150/37962 [23:31:36<74:08:35,  9.60s/it]
 27%|██▋       | 10151/37962 [23:31:44<71:16:16,  9.23s/it]
 27%|██▋       | 10152/37962 [23:31:52<68:21:30,  8.85s/it]
 27%|██▋       | 10153/37962 [23:32:01<67:19:26,  8.72s/it]
 27%|██▋       | 10154/37962 [23:32:09<66:18:01,  8.58s/it]
 27%|██▋       | 10155/37962 [23:32:17<65:27:22,  8.47s/it]
 27%|██▋       | 10156/37962 [23:32:25<65:04:03,  8.42s/it]
 27%|██▋       | 10157/37962 [23:32:34<64:47:56,  8.39s/it]
 27%|██▋       | 10158/37962 [23:32:42<64:28:20,  8.35s/it]
 27%|██▋       | 10159/37962 [23:32:50<64:23:47,  8.34s/it]
 27%|██▋       | 10160/37962 [23:32:58<64:03:13,  8.29s/it]
                                                           
{'loss': 1.3601, 'grad_norm': 5.514267444610596, 'learning_rate': 1.7181639982634047e-05, 'epoch': 0.27}

 27%|██▋       | 10160/37962 [23:32:58<64:03:13,  8.29s/it]
 27%|██▋       | 10161/37962 [23:33:07<63:54:45,  8.28s/it]
 27%|██▋       | 10162/37962 [23:33:15<63:18:53,  8.20s/it]
 27%|██▋       | 10163/37962 [23:33:23<63:25:55,  8.21s/it]
 27%|██▋       | 10164/37962 [23:33:31<63:43:37,  8.25s/it]
 27%|██▋       | 10165/37962 [23:33:39<63:21:32,  8.21s/it]
 27%|██▋       | 10166/37962 [23:33:48<63:28:29,  8.22s/it]
 27%|██▋       | 10167/37962 [23:33:56<63:14:29,  8.19s/it]
 27%|██▋       | 10168/37962 [23:34:04<63:41:58,  8.25s/it]
 27%|██▋       | 10169/37962 [23:34:12<63:42:18,  8.25s/it]
 27%|██▋       | 10170/37962 [23:34:20<63:04:00,  8.17s/it]
                                                           
{'loss': 1.3722, 'grad_norm': 6.509953498840332, 'learning_rate': 1.7175700448977277e-05, 'epoch': 0.27}

 27%|██▋       | 10170/37962 [23:34:20<63:04:00,  8.17s/it]
 27%|██▋       | 10171/37962 [23:34:29<63:07:49,  8.18s/it]
 27%|██▋       | 10172/37962 [23:34:37<63:38:45,  8.24s/it]
 27%|██▋       | 10173/37962 [23:34:45<62:50:39,  8.14s/it]
 27%|██▋       | 10174/37962 [23:34:53<62:34:48,  8.11s/it]
 27%|██▋       | 10175/37962 [23:35:01<63:09:07,  8.18s/it]
 27%|██▋       | 10176/37962 [23:35:10<63:48:30,  8.27s/it]
 27%|██▋       | 10177/37962 [23:35:18<63:26:36,  8.22s/it]
 27%|██▋       | 10178/37962 [23:35:26<62:21:33,  8.08s/it]
 27%|██▋       | 10179/37962 [23:35:34<63:01:02,  8.17s/it]
 27%|██▋       | 10180/37962 [23:35:42<63:13:33,  8.19s/it]
                                                           
{'loss': 1.3553, 'grad_norm': 18.630632400512695, 'learning_rate': 1.716975569225246e-05, 'epoch': 0.27}

 27%|██▋       | 10180/37962 [23:35:42<63:13:33,  8.19s/it]
 27%|██▋       | 10181/37962 [23:35:50<62:52:10,  8.15s/it]
 27%|██▋       | 10182/37962 [23:35:58<62:42:48,  8.13s/it]
 27%|██▋       | 10183/37962 [23:36:07<62:58:20,  8.16s/it]
 27%|██▋       | 10184/37962 [23:36:15<63:16:24,  8.20s/it]
 27%|██▋       | 10185/37962 [23:36:23<63:28:38,  8.23s/it]
 27%|██▋       | 10186/37962 [23:36:31<62:44:03,  8.13s/it]
 27%|██▋       | 10187/37962 [23:36:39<63:06:47,  8.18s/it]
 27%|██▋       | 10188/37962 [23:36:48<63:18:57,  8.21s/it]
 27%|██▋       | 10189/37962 [23:36:56<63:41:03,  8.25s/it]
 27%|██▋       | 10190/37962 [23:37:04<64:01:09,  8.30s/it]
                                                           
{'loss': 1.3843, 'grad_norm': 5.28150749206543, 'learning_rate': 1.7163805716786687e-05, 'epoch': 0.27}

 27%|██▋       | 10190/37962 [23:37:04<64:01:09,  8.30s/it]
 27%|██▋       | 10191/37962 [23:37:13<64:01:33,  8.30s/it]
 27%|██▋       | 10192/37962 [23:37:21<63:41:49,  8.26s/it]
 27%|██▋       | 10193/37962 [23:37:29<64:10:40,  8.32s/it]
 27%|██▋       | 10194/37962 [23:37:38<64:35:37,  8.37s/it]
 27%|██▋       | 10195/37962 [23:37:46<64:08:18,  8.32s/it]
 27%|██▋       | 10196/37962 [23:37:54<64:00:10,  8.30s/it]
 27%|██▋       | 10197/37962 [23:38:03<64:10:07,  8.32s/it]
 27%|██▋       | 10198/37962 [23:38:11<64:05:52,  8.31s/it]
 27%|██▋       | 10199/37962 [23:38:19<64:11:20,  8.32s/it]
 27%|██▋       | 10200/37962 [23:38:28<64:09:01,  8.32s/it]
                                                           
{'loss': 1.4096, 'grad_norm': 12.375866889953613, 'learning_rate': 1.715785052691083e-05, 'epoch': 0.27}

 27%|██▋       | 10200/37962 [23:38:28<64:09:01,  8.32s/it]
 27%|██▋       | 10201/37962 [23:38:36<64:25:18,  8.35s/it]
 27%|██▋       | 10202/37962 [23:38:44<63:55:00,  8.29s/it]
 27%|██▋       | 10203/37962 [23:38:52<62:57:26,  8.16s/it]
 27%|██▋       | 10204/37962 [23:39:00<62:45:51,  8.14s/it]
 27%|██▋       | 10205/37962 [23:39:08<63:07:49,  8.19s/it]
 27%|██▋       | 10206/37962 [23:39:17<63:17:14,  8.21s/it]
 27%|██▋       | 10207/37962 [23:39:25<62:47:45,  8.15s/it]
 27%|██▋       | 10208/37962 [23:39:33<63:30:32,  8.24s/it]
 27%|██▋       | 10209/37962 [23:39:41<63:25:15,  8.23s/it]
 27%|██▋       | 10210/37962 [23:39:50<63:32:31,  8.24s/it]
                                                           
{'loss': 1.4068, 'grad_norm': 5.365541458129883, 'learning_rate': 1.7151890126959583e-05, 'epoch': 0.27}

 27%|██▋       | 10210/37962 [23:39:50<63:32:31,  8.24s/it]
 27%|██▋       | 10211/37962 [23:39:58<62:58:12,  8.17s/it]
 27%|██▋       | 10212/37962 [23:40:06<63:12:18,  8.20s/it]
 27%|██▋       | 10213/37962 [23:40:14<63:11:34,  8.20s/it]
 27%|██▋       | 10214/37962 [23:40:22<62:55:07,  8.16s/it]
 27%|██▋       | 10215/37962 [23:40:30<62:35:16,  8.12s/it]
 27%|██▋       | 10216/37962 [23:40:38<63:08:45,  8.19s/it]
 27%|██▋       | 10217/37962 [23:40:47<63:38:55,  8.26s/it]
 27%|██▋       | 10218/37962 [23:40:55<64:15:01,  8.34s/it]
 27%|██▋       | 10219/37962 [23:41:04<64:03:10,  8.31s/it]
 27%|██▋       | 10220/37962 [23:41:12<64:11:32,  8.33s/it]
                                                           
{'loss': 1.3548, 'grad_norm': 22.405471801757812, 'learning_rate': 1.7145924521271408e-05, 'epoch': 0.27}

 27%|██▋       | 10220/37962 [23:41:12<64:11:32,  8.33s/it]
 27%|██▋       | 10221/37962 [23:41:20<64:08:31,  8.32s/it]
 27%|██▋       | 10222/37962 [23:41:28<63:16:55,  8.21s/it]
 27%|██▋       | 10223/37962 [23:41:37<63:57:14,  8.30s/it]
 27%|██▋       | 10224/37962 [23:41:45<64:01:57,  8.31s/it]
 27%|██▋       | 10225/37962 [23:41:53<63:53:58,  8.29s/it]
 27%|██▋       | 10226/37962 [23:42:02<64:21:17,  8.35s/it]
 27%|██▋       | 10227/37962 [23:42:10<64:17:36,  8.35s/it]
 27%|██▋       | 10228/37962 [23:42:19<64:44:04,  8.40s/it]
 27%|██▋       | 10229/37962 [23:42:27<64:02:35,  8.31s/it]
 27%|██▋       | 10230/37962 [23:42:35<64:31:38,  8.38s/it]
                                                           
{'loss': 1.3288, 'grad_norm': 2.3575403690338135, 'learning_rate': 1.7139953714188565e-05, 'epoch': 0.27}

 27%|██▋       | 10230/37962 [23:42:35<64:31:38,  8.38s/it]
 27%|██▋       | 10231/37962 [23:42:44<64:34:12,  8.38s/it]
 27%|██▋       | 10232/37962 [23:42:52<64:21:56,  8.36s/it]
 27%|██▋       | 10233/37962 [23:43:00<64:29:11,  8.37s/it]
 27%|██▋       | 10234/37962 [23:43:09<64:19:56,  8.35s/it]
 27%|██▋       | 10235/37962 [23:43:17<63:39:26,  8.27s/it]
 27%|██▋       | 10236/37962 [23:43:25<63:05:19,  8.19s/it]
 27%|██▋       | 10237/37962 [23:43:33<63:24:21,  8.23s/it]
 27%|██▋       | 10238/37962 [23:43:42<63:57:08,  8.30s/it]
 27%|██▋       | 10239/37962 [23:43:50<64:23:24,  8.36s/it]
 27%|██▋       | 10240/37962 [23:43:58<64:16:18,  8.35s/it]
                                                           
{'loss': 1.3631, 'grad_norm': 6.551501750946045, 'learning_rate': 1.713397771005711e-05, 'epoch': 0.27}

 27%|██▋       | 10240/37962 [23:43:58<64:16:18,  8.35s/it]
 27%|██▋       | 10241/37962 [23:44:07<64:22:55,  8.36s/it]
 27%|██▋       | 10242/37962 [23:44:15<64:09:46,  8.33s/it]
 27%|██▋       | 10243/37962 [23:44:23<63:41:58,  8.27s/it]
 27%|██▋       | 10244/37962 [23:44:32<63:39:43,  8.27s/it]
 27%|██▋       | 10245/37962 [23:44:40<63:53:10,  8.30s/it]
 27%|██▋       | 10246/37962 [23:44:48<64:09:51,  8.33s/it]
 27%|██▋       | 10247/37962 [23:44:57<64:31:00,  8.38s/it]
 27%|██▋       | 10248/37962 [23:45:05<64:51:32,  8.43s/it]
 27%|██▋       | 10249/37962 [23:45:14<64:52:28,  8.43s/it]
 27%|██▋       | 10250/37962 [23:45:22<64:41:59,  8.41s/it]
                                                           
{'loss': 1.3245, 'grad_norm': 7.130479335784912, 'learning_rate': 1.7127996513226863e-05, 'epoch': 0.27}

 27%|██▋       | 10250/37962 [23:45:22<64:41:59,  8.41s/it]
 27%|██▋       | 10251/37962 [23:45:31<64:52:00,  8.43s/it]
 27%|██▋       | 10252/37962 [23:45:39<65:11:29,  8.47s/it]
 27%|██▋       | 10253/37962 [23:45:47<63:41:09,  8.27s/it]
 27%|██▋       | 10254/37962 [23:45:55<63:31:16,  8.25s/it]
 27%|██▋       | 10255/37962 [23:46:04<64:14:13,  8.35s/it]
 27%|██▋       | 10256/37962 [23:46:12<64:27:49,  8.38s/it]
 27%|██▋       | 10257/37962 [23:46:21<64:30:20,  8.38s/it]
 27%|██▋       | 10258/37962 [23:46:29<64:30:39,  8.38s/it]
 27%|██▋       | 10259/37962 [23:46:37<64:09:49,  8.34s/it]
 27%|██▋       | 10260/37962 [23:46:46<64:47:04,  8.42s/it]
                                                           
{'loss': 1.4094, 'grad_norm': 5.8388352394104, 'learning_rate': 1.712201012805144e-05, 'epoch': 0.27}

 27%|██▋       | 10260/37962 [23:46:46<64:47:04,  8.42s/it]
 27%|██▋       | 10261/37962 [23:46:54<64:31:14,  8.39s/it]
 27%|██▋       | 10262/37962 [23:47:03<65:06:48,  8.46s/it]
 27%|██▋       | 10263/37962 [23:47:11<64:49:58,  8.43s/it]
 27%|██▋       | 10264/37962 [23:47:19<64:36:46,  8.40s/it]
 27%|██▋       | 10265/37962 [23:47:28<65:07:39,  8.47s/it]
 27%|██▋       | 10266/37962 [23:47:36<64:39:27,  8.40s/it]
 27%|██▋       | 10267/37962 [23:47:45<65:17:04,  8.49s/it]
 27%|██▋       | 10268/37962 [23:47:54<65:39:19,  8.53s/it]
 27%|██▋       | 10269/37962 [23:48:03<66:23:52,  8.63s/it]
 27%|██▋       | 10270/37962 [23:48:11<65:41:59,  8.54s/it]
                                                           
{'loss': 1.3182, 'grad_norm': 25.963781356811523, 'learning_rate': 1.7116018558888225e-05, 'epoch': 0.27}

 27%|██▋       | 10270/37962 [23:48:11<65:41:59,  8.54s/it]
 27%|██▋       | 10271/37962 [23:48:19<65:12:11,  8.48s/it]
 27%|██▋       | 10272/37962 [23:48:28<65:29:50,  8.52s/it]
 27%|██▋       | 10273/37962 [23:48:36<65:03:31,  8.46s/it]
 27%|██▋       | 10274/37962 [23:48:44<64:26:12,  8.38s/it]
 27%|██▋       | 10275/37962 [23:48:53<64:28:07,  8.38s/it]
 27%|██▋       | 10276/37962 [23:49:01<64:53:56,  8.44s/it]
 27%|██▋       | 10277/37962 [23:49:10<64:30:43,  8.39s/it]
 27%|██▋       | 10278/37962 [23:49:18<64:16:31,  8.36s/it]
 27%|██▋       | 10279/37962 [23:49:26<63:23:37,  8.24s/it]
 27%|██▋       | 10280/37962 [23:49:34<63:46:20,  8.29s/it]
                                                           
{'loss': 1.3669, 'grad_norm': 6.962029933929443, 'learning_rate': 1.7110021810098382e-05, 'epoch': 0.27}

 27%|██▋       | 10280/37962 [23:49:34<63:46:20,  8.29s/it]
 27%|██▋       | 10281/37962 [23:49:43<64:50:31,  8.43s/it]
 27%|██▋       | 10282/37962 [23:49:51<64:31:04,  8.39s/it]
 27%|██▋       | 10283/37962 [23:49:59<64:09:36,  8.34s/it]
 27%|██▋       | 10284/37962 [23:50:08<64:06:14,  8.34s/it]
 27%|██▋       | 10285/37962 [23:50:17<65:10:12,  8.48s/it]
 27%|██▋       | 10286/37962 [23:50:25<64:37:56,  8.41s/it]
 27%|██▋       | 10287/37962 [23:50:33<64:22:35,  8.37s/it]
 27%|██▋       | 10288/37962 [23:50:41<64:18:00,  8.36s/it]
 27%|██▋       | 10289/37962 [23:50:50<64:23:39,  8.38s/it]
 27%|██▋       | 10290/37962 [23:50:59<65:10:12,  8.48s/it]
                                                           
{'loss': 1.4282, 'grad_norm': 9.447394371032715, 'learning_rate': 1.710401988604683e-05, 'epoch': 0.27}

 27%|██▋       | 10290/37962 [23:50:59<65:10:12,  8.48s/it]
 27%|██▋       | 10291/37962 [23:51:07<64:15:09,  8.36s/it]
 27%|██▋       | 10292/37962 [23:51:15<64:09:35,  8.35s/it]
 27%|██▋       | 10293/37962 [23:51:24<64:37:28,  8.41s/it]
 27%|██▋       | 10294/37962 [23:51:32<64:57:35,  8.45s/it]
 27%|██▋       | 10295/37962 [23:51:40<64:40:00,  8.41s/it]
 27%|██▋       | 10296/37962 [23:51:49<64:39:59,  8.41s/it]
 27%|██▋       | 10297/37962 [23:51:57<64:55:14,  8.45s/it]
 27%|██▋       | 10298/37962 [23:52:06<65:09:14,  8.48s/it]
 27%|██▋       | 10299/37962 [23:52:15<65:30:42,  8.53s/it]
 27%|██▋       | 10300/37962 [23:52:23<64:35:23,  8.41s/it]
                                                           
{'loss': 1.3482, 'grad_norm': 5.864366054534912, 'learning_rate': 1.7098012791102275e-05, 'epoch': 0.27}

 27%|██▋       | 10300/37962 [23:52:23<64:35:23,  8.41s/it]
 27%|██▋       | 10301/37962 [23:52:32<65:45:08,  8.56s/it]
 27%|██▋       | 10302/37962 [23:52:40<65:21:01,  8.51s/it]
 27%|██▋       | 10303/37962 [23:52:48<65:06:42,  8.47s/it]
 27%|██▋       | 10304/37962 [23:52:57<65:03:45,  8.47s/it]
 27%|██▋       | 10305/37962 [23:53:05<65:10:25,  8.48s/it]
 27%|██▋       | 10306/37962 [23:53:14<64:49:49,  8.44s/it]
 27%|██▋       | 10307/37962 [23:53:22<65:09:25,  8.48s/it]
 27%|██▋       | 10308/37962 [23:53:31<64:36:32,  8.41s/it]
 27%|██▋       | 10309/37962 [23:53:39<65:06:41,  8.48s/it]
 27%|██▋       | 10310/37962 [23:53:48<65:16:29,  8.50s/it]
                                                           
{'loss': 1.3161, 'grad_norm': 13.406704902648926, 'learning_rate': 1.709200052963717e-05, 'epoch': 0.27}

 27%|██▋       | 10310/37962 [23:53:48<65:16:29,  8.50s/it]
 27%|██▋       | 10311/37962 [23:53:56<64:46:40,  8.43s/it]
 27%|██▋       | 10312/37962 [23:54:05<65:01:19,  8.47s/it]
 27%|██▋       | 10313/37962 [23:54:12<63:37:40,  8.28s/it]
 27%|██▋       | 10314/37962 [23:54:21<63:33:44,  8.28s/it]
 27%|██▋       | 10315/37962 [23:54:29<63:20:55,  8.25s/it]
 27%|██▋       | 10316/37962 [23:54:37<63:31:48,  8.27s/it]
 27%|██▋       | 10317/37962 [23:54:45<63:16:17,  8.24s/it]
 27%|██▋       | 10318/37962 [23:54:54<63:18:57,  8.25s/it]
 27%|██▋       | 10319/37962 [23:55:02<63:32:42,  8.28s/it]
 27%|██▋       | 10320/37962 [23:55:10<62:39:15,  8.16s/it]
                                                           
{'loss': 1.4372, 'grad_norm': 6.410478591918945, 'learning_rate': 1.7085983106027744e-05, 'epoch': 0.27}

 27%|██▋       | 10320/37962 [23:55:10<62:39:15,  8.16s/it]
 27%|██▋       | 10321/37962 [23:55:18<63:02:12,  8.21s/it]
 27%|██▋       | 10322/37962 [23:55:26<62:27:44,  8.14s/it]
 27%|██▋       | 10323/37962 [23:55:34<62:22:41,  8.12s/it]
 27%|██▋       | 10324/37962 [23:55:42<61:34:57,  8.02s/it]
 27%|██▋       | 10325/37962 [23:55:50<61:20:18,  7.99s/it]
 27%|██▋       | 10326/37962 [23:55:58<62:08:16,  8.09s/it]
 27%|██▋       | 10327/37962 [23:56:07<62:33:16,  8.15s/it]
 27%|██▋       | 10328/37962 [23:56:15<62:21:38,  8.12s/it]
 27%|██▋       | 10329/37962 [23:56:23<62:44:25,  8.17s/it]
 27%|██▋       | 10330/37962 [23:56:31<63:01:28,  8.21s/it]
                                                           
{'loss': 1.3419, 'grad_norm': 6.863880634307861, 'learning_rate': 1.7079960524653972e-05, 'epoch': 0.27}

 27%|██▋       | 10330/37962 [23:56:31<63:01:28,  8.21s/it]
 27%|██▋       | 10331/37962 [23:56:39<63:09:28,  8.23s/it]
 27%|██▋       | 10332/37962 [23:56:48<63:39:41,  8.29s/it]
 27%|██▋       | 10333/37962 [23:56:56<63:31:43,  8.28s/it]
 27%|██▋       | 10334/37962 [23:57:04<63:24:07,  8.26s/it]
 27%|██▋       | 10335/37962 [23:57:13<64:09:09,  8.36s/it]
 27%|██▋       | 10336/37962 [23:57:21<63:46:04,  8.31s/it]
 27%|██▋       | 10337/37962 [23:57:30<63:59:19,  8.34s/it]
 27%|██▋       | 10338/37962 [23:57:38<63:58:40,  8.34s/it]
 27%|██▋       | 10339/37962 [23:57:46<64:24:07,  8.39s/it]
 27%|██▋       | 10340/37962 [23:57:55<64:03:27,  8.35s/it]
                                                           
{'loss': 1.3873, 'grad_norm': 8.327899932861328, 'learning_rate': 1.7073932789899582e-05, 'epoch': 0.27}

 27%|██▋       | 10340/37962 [23:57:55<64:03:27,  8.35s/it]
 27%|██▋       | 10341/37962 [23:58:03<63:53:35,  8.33s/it]
 27%|██▋       | 10342/37962 [23:58:11<63:39:01,  8.30s/it]
 27%|██▋       | 10343/37962 [23:58:19<63:45:00,  8.31s/it]
 27%|██▋       | 10344/37962 [23:58:28<63:38:05,  8.29s/it]
 27%|██▋       | 10345/37962 [23:58:36<64:09:32,  8.36s/it]
 27%|██▋       | 10346/37962 [23:58:45<64:00:28,  8.34s/it]
 27%|██▋       | 10347/37962 [23:58:53<63:56:07,  8.33s/it]
 27%|██▋       | 10348/37962 [23:59:01<63:57:50,  8.34s/it]
 27%|██▋       | 10349/37962 [23:59:10<63:53:43,  8.33s/it]
 27%|██▋       | 10350/37962 [23:59:18<63:40:40,  8.30s/it]
                                                           
{'loss': 1.2646, 'grad_norm': 3.1528546810150146, 'learning_rate': 1.7067899906152066e-05, 'epoch': 0.27}

 27%|██▋       | 10350/37962 [23:59:18<63:40:40,  8.30s/it]
 27%|██▋       | 10351/37962 [23:59:26<64:03:43,  8.35s/it]
 27%|██▋       | 10352/37962 [23:59:35<63:57:17,  8.34s/it]
 27%|██▋       | 10353/37962 [23:59:43<63:07:32,  8.23s/it]
 27%|██▋       | 10354/37962 [23:59:51<62:58:53,  8.21s/it]
 27%|██▋       | 10355/37962 [23:59:59<62:53:59,  8.20s/it]
 27%|██▋       | 10356/37962 [24:00:07<63:03:39,  8.22s/it]
 27%|██▋       | 10357/37962 [24:00:15<62:38:03,  8.17s/it]
 27%|██▋       | 10358/37962 [24:00:23<62:51:09,  8.20s/it]
 27%|██▋       | 10359/37962 [24:00:32<62:55:35,  8.21s/it]
 27%|██▋       | 10360/37962 [24:00:40<63:21:51,  8.26s/it]
                                                           
{'loss': 1.4475, 'grad_norm': 13.592677116394043, 'learning_rate': 1.706186187780265e-05, 'epoch': 0.27}

 27%|██▋       | 10360/37962 [24:00:40<63:21:51,  8.26s/it]
 27%|██▋       | 10361/37962 [24:00:48<62:19:06,  8.13s/it]
 27%|██▋       | 10362/37962 [24:00:56<62:58:27,  8.21s/it]
 27%|██▋       | 10363/37962 [24:01:05<63:23:06,  8.27s/it]
 27%|██▋       | 10364/37962 [24:01:13<63:26:47,  8.28s/it]
 27%|██▋       | 10365/37962 [24:01:21<62:42:12,  8.18s/it]
 27%|██▋       | 10366/37962 [24:01:29<62:39:05,  8.17s/it]
 27%|██▋       | 10367/37962 [24:01:37<62:48:49,  8.19s/it]
 27%|██▋       | 10368/37962 [24:01:46<63:14:26,  8.25s/it]
 27%|██▋       | 10369/37962 [24:02:01<78:40:06, 10.26s/it]
 27%|██▋       | 10370/37962 [24:02:09<74:02:04,  9.66s/it]
                                                           
{'loss': 1.4143, 'grad_norm': 3.0780158042907715, 'learning_rate': 1.7055818709246307e-05, 'epoch': 0.27}

 27%|██▋       | 10370/37962 [24:02:09<74:02:04,  9.66s/it]
 27%|██▋       | 10371/37962 [24:02:17<70:57:30,  9.26s/it]
 27%|██▋       | 10372/37962 [24:02:26<68:37:46,  8.95s/it]
 27%|██▋       | 10373/37962 [24:02:34<67:00:07,  8.74s/it]
 27%|██▋       | 10374/37962 [24:02:42<66:04:14,  8.62s/it]
 27%|██▋       | 10375/37962 [24:02:51<65:31:39,  8.55s/it]
 27%|██▋       | 10376/37962 [24:02:59<64:54:54,  8.47s/it]
 27%|██▋       | 10377/37962 [24:03:07<64:32:34,  8.42s/it]
 27%|██▋       | 10378/37962 [24:03:15<63:45:57,  8.32s/it]
 27%|██▋       | 10379/37962 [24:03:23<63:29:42,  8.29s/it]
 27%|██▋       | 10380/37962 [24:03:32<63:23:23,  8.27s/it]
                                                           
{'loss': 1.332, 'grad_norm': 18.138545989990234, 'learning_rate': 1.7049770404881764e-05, 'epoch': 0.27}

 27%|██▋       | 10380/37962 [24:03:32<63:23:23,  8.27s/it]
 27%|██▋       | 10381/37962 [24:03:40<63:39:51,  8.31s/it]
 27%|██▋       | 10382/37962 [24:03:48<63:47:48,  8.33s/it]
 27%|██▋       | 10383/37962 [24:03:57<63:33:29,  8.30s/it]
 27%|██▋       | 10384/37962 [24:04:05<63:34:27,  8.30s/it]
 27%|██▋       | 10385/37962 [24:04:13<62:47:56,  8.20s/it]
 27%|██▋       | 10386/37962 [24:04:21<63:12:15,  8.25s/it]
 27%|██▋       | 10387/37962 [24:04:30<63:35:22,  8.30s/it]
 27%|██▋       | 10388/37962 [24:04:38<63:46:08,  8.33s/it]
 27%|██▋       | 10389/37962 [24:04:46<63:52:39,  8.34s/it]
 27%|██▋       | 10390/37962 [24:04:55<64:00:38,  8.36s/it]
                                                           
{'loss': 1.4209, 'grad_norm': 14.070090293884277, 'learning_rate': 1.7043716969111474e-05, 'epoch': 0.27}

 27%|██▋       | 10390/37962 [24:04:55<64:00:38,  8.36s/it]
 27%|██▋       | 10391/37962 [24:05:03<63:58:13,  8.35s/it]
 27%|██▋       | 10392/37962 [24:05:12<64:09:34,  8.38s/it]
 27%|██▋       | 10393/37962 [24:05:20<64:14:57,  8.39s/it]
 27%|██▋       | 10394/37962 [24:05:28<64:18:26,  8.40s/it]
 27%|██▋       | 10395/37962 [24:05:37<64:09:38,  8.38s/it]
 27%|██▋       | 10396/37962 [24:05:45<63:55:06,  8.35s/it]
 27%|██▋       | 10397/37962 [24:05:53<64:01:08,  8.36s/it]
 27%|██▋       | 10398/37962 [24:06:02<63:58:36,  8.36s/it]
 27%|██▋       | 10399/37962 [24:06:10<64:01:18,  8.36s/it]
 27%|██▋       | 10400/37962 [24:06:18<63:34:05,  8.30s/it]
                                                           
{'loss': 1.4409, 'grad_norm': 14.453300476074219, 'learning_rate': 1.7037658406341623e-05, 'epoch': 0.27}

 27%|██▋       | 10400/37962 [24:06:18<63:34:05,  8.30s/it]
 27%|██▋       | 10401/37962 [24:06:27<63:23:29,  8.28s/it]
 27%|██▋       | 10402/37962 [24:06:35<63:21:56,  8.28s/it]
 27%|██▋       | 10403/37962 [24:06:51<80:28:31, 10.51s/it]
 27%|██▋       | 10404/37962 [24:06:59<75:47:53,  9.90s/it]
 27%|██▋       | 10405/37962 [24:07:07<71:44:21,  9.37s/it]
 27%|██▋       | 10406/37962 [24:07:15<69:12:36,  9.04s/it]
 27%|██▋       | 10407/37962 [24:07:24<67:44:58,  8.85s/it]
 27%|██▋       | 10408/37962 [24:07:32<67:05:40,  8.77s/it]
 27%|██▋       | 10409/37962 [24:07:41<65:57:30,  8.62s/it]
 27%|██▋       | 10410/37962 [24:07:49<64:52:57,  8.48s/it]
                                                           
{'loss': 1.4234, 'grad_norm': 9.975614547729492, 'learning_rate': 1.7031594720982134e-05, 'epoch': 0.27}

 27%|██▋       | 10410/37962 [24:07:49<64:52:57,  8.48s/it]
 27%|██▋       | 10411/37962 [24:07:57<64:17:23,  8.40s/it]
 27%|██▋       | 10412/37962 [24:08:05<64:21:23,  8.41s/it]
 27%|██▋       | 10413/37962 [24:08:14<64:12:49,  8.39s/it]
 27%|██▋       | 10414/37962 [24:08:22<62:56:02,  8.22s/it]
 27%|██▋       | 10415/37962 [24:08:30<63:27:53,  8.29s/it]
 27%|██▋       | 10416/37962 [24:08:38<63:26:06,  8.29s/it]
 27%|██▋       | 10417/37962 [24:08:47<63:15:03,  8.27s/it]
 27%|██▋       | 10418/37962 [24:08:54<62:10:32,  8.13s/it]
 27%|██▋       | 10419/37962 [24:09:03<62:42:48,  8.20s/it]
 27%|██▋       | 10420/37962 [24:09:11<62:52:59,  8.22s/it]
                                                           
{'loss': 1.3255, 'grad_norm': 23.27369499206543, 'learning_rate': 1.7025525917446667e-05, 'epoch': 0.27}

 27%|██▋       | 10420/37962 [24:09:11<62:52:59,  8.22s/it]
 27%|██▋       | 10421/37962 [24:09:19<62:58:28,  8.23s/it]
 27%|██▋       | 10422/37962 [24:09:28<63:01:07,  8.24s/it]
 27%|██▋       | 10423/37962 [24:09:36<62:47:24,  8.21s/it]
 27%|██▋       | 10424/37962 [24:09:44<62:57:39,  8.23s/it]
 27%|██▋       | 10425/37962 [24:09:52<62:54:26,  8.22s/it]
 27%|██▋       | 10426/37962 [24:10:01<63:12:23,  8.26s/it]
 27%|██▋       | 10427/37962 [24:10:09<63:01:17,  8.24s/it]
 27%|██▋       | 10428/37962 [24:10:17<62:29:40,  8.17s/it]
 27%|██▋       | 10429/37962 [24:10:25<61:37:44,  8.06s/it]
 27%|██▋       | 10430/37962 [24:10:33<62:31:49,  8.18s/it]
                                                           
{'loss': 1.4333, 'grad_norm': 19.014198303222656, 'learning_rate': 1.701945200015259e-05, 'epoch': 0.27}

 27%|██▋       | 10430/37962 [24:10:33<62:31:49,  8.18s/it]
 27%|██▋       | 10431/37962 [24:10:41<61:27:13,  8.04s/it]
 27%|██▋       | 10432/37962 [24:10:49<61:34:57,  8.05s/it]
 27%|██▋       | 10433/37962 [24:10:57<61:53:47,  8.09s/it]
 27%|██▋       | 10434/37962 [24:11:05<62:40:38,  8.20s/it]
 27%|██▋       | 10435/37962 [24:11:13<62:08:43,  8.13s/it]
 27%|██▋       | 10436/37962 [24:11:22<62:26:47,  8.17s/it]
 27%|██▋       | 10437/37962 [24:11:30<62:12:41,  8.14s/it]
 27%|██▋       | 10438/37962 [24:11:37<61:22:06,  8.03s/it]
 27%|██▋       | 10439/37962 [24:11:46<61:53:58,  8.10s/it]
 28%|██▊       | 10440/37962 [24:11:54<62:34:52,  8.19s/it]
                                                           
{'loss': 1.3894, 'grad_norm': 6.786633491516113, 'learning_rate': 1.701337297352101e-05, 'epoch': 0.28}

 28%|██▊       | 10440/37962 [24:11:54<62:34:52,  8.19s/it]
 28%|██▊       | 10441/37962 [24:12:02<62:55:03,  8.23s/it]
 28%|██▊       | 10442/37962 [24:12:11<62:45:34,  8.21s/it]
 28%|██▊       | 10443/37962 [24:12:19<63:13:50,  8.27s/it]
 28%|██▊       | 10444/37962 [24:12:27<62:18:57,  8.15s/it]
 28%|██▊       | 10445/37962 [24:12:35<62:36:35,  8.19s/it]
 28%|██▊       | 10446/37962 [24:12:44<63:24:33,  8.30s/it]
 28%|██▊       | 10447/37962 [24:12:52<63:30:31,  8.31s/it]
 28%|██▊       | 10448/37962 [24:13:01<64:01:31,  8.38s/it]
 28%|██▊       | 10449/37962 [24:13:09<63:46:02,  8.34s/it]
 28%|██▊       | 10450/37962 [24:13:17<63:30:14,  8.31s/it]
                                                           
{'loss': 1.3027, 'grad_norm': 7.586590766906738, 'learning_rate': 1.7007288841976736e-05, 'epoch': 0.28}

 28%|██▊       | 10450/37962 [24:13:17<63:30:14,  8.31s/it]
 28%|██▊       | 10451/37962 [24:13:25<63:17:45,  8.28s/it]
 28%|██▊       | 10452/37962 [24:13:34<63:09:27,  8.26s/it]
 28%|██▊       | 10453/37962 [24:13:42<63:15:05,  8.28s/it]
 28%|██▊       | 10454/37962 [24:13:50<63:01:02,  8.25s/it]
 28%|██▊       | 10455/37962 [24:13:58<63:06:49,  8.26s/it]
 28%|██▊       | 10456/37962 [24:14:07<63:19:54,  8.29s/it]
 28%|██▊       | 10457/37962 [24:14:15<63:57:12,  8.37s/it]
 28%|██▊       | 10458/37962 [24:14:24<63:56:50,  8.37s/it]
 28%|██▊       | 10459/37962 [24:14:32<63:42:11,  8.34s/it]
 28%|██▊       | 10460/37962 [24:14:40<63:10:10,  8.27s/it]
                                                           
{'loss': 1.4249, 'grad_norm': 10.669122695922852, 'learning_rate': 1.700119960994831e-05, 'epoch': 0.28}

 28%|██▊       | 10460/37962 [24:14:40<63:10:10,  8.27s/it]
 28%|██▊       | 10461/37962 [24:14:48<62:54:51,  8.24s/it]
 28%|██▊       | 10462/37962 [24:14:57<63:13:41,  8.28s/it]
 28%|██▊       | 10463/37962 [24:15:05<63:20:56,  8.29s/it]
 28%|██▊       | 10464/37962 [24:15:13<63:02:13,  8.25s/it]
 28%|██▊       | 10465/37962 [24:15:21<63:07:27,  8.26s/it]
 28%|██▊       | 10466/37962 [24:15:30<63:23:05,  8.30s/it]
 28%|██▊       | 10467/37962 [24:15:38<63:25:07,  8.30s/it]
 28%|██▊       | 10468/37962 [24:15:46<63:35:23,  8.33s/it]
 28%|██▊       | 10469/37962 [24:15:55<63:37:25,  8.33s/it]
 28%|██▊       | 10470/37962 [24:16:03<63:36:00,  8.33s/it]
                                                           
{'loss': 1.519, 'grad_norm': 5.211860179901123, 'learning_rate': 1.6995105281867973e-05, 'epoch': 0.28}

 28%|██▊       | 10470/37962 [24:16:03<63:36:00,  8.33s/it]
 28%|██▊       | 10471/37962 [24:16:11<63:48:01,  8.35s/it]
 28%|██▊       | 10472/37962 [24:16:20<63:31:03,  8.32s/it]
 28%|██▊       | 10473/37962 [24:16:28<62:57:59,  8.25s/it]
 28%|██▊       | 10474/37962 [24:16:36<62:33:14,  8.19s/it]
 28%|██▊       | 10475/37962 [24:16:44<62:46:25,  8.22s/it]
 28%|██▊       | 10476/37962 [24:16:52<62:28:15,  8.18s/it]
 28%|██▊       | 10477/37962 [24:17:00<62:41:32,  8.21s/it]
 28%|██▊       | 10478/37962 [24:17:09<63:04:40,  8.26s/it]
 28%|██▊       | 10479/37962 [24:17:17<63:09:52,  8.27s/it]
 28%|██▊       | 10480/37962 [24:17:25<62:41:01,  8.21s/it]
                                                           
{'loss': 1.4937, 'grad_norm': 11.10645866394043, 'learning_rate': 1.698900586217169e-05, 'epoch': 0.28}

 28%|██▊       | 10480/37962 [24:17:25<62:41:01,  8.21s/it]
 28%|██▊       | 10481/37962 [24:17:33<62:46:38,  8.22s/it]
 28%|██▊       | 10482/37962 [24:17:42<62:43:31,  8.22s/it]
 28%|██▊       | 10483/37962 [24:17:50<62:06:16,  8.14s/it]
 28%|██▊       | 10484/37962 [24:17:58<62:31:05,  8.19s/it]
 28%|██▊       | 10485/37962 [24:18:06<62:56:59,  8.25s/it]
 28%|██▊       | 10486/37962 [24:18:15<62:55:23,  8.24s/it]
 28%|██▊       | 10487/37962 [24:18:23<63:14:37,  8.29s/it]
 28%|██▊       | 10488/37962 [24:18:31<63:20:53,  8.30s/it]
 28%|██▊       | 10489/37962 [24:18:40<63:06:56,  8.27s/it]
 28%|██▊       | 10490/37962 [24:18:48<63:24:09,  8.31s/it]
                                                           
{'loss': 1.5356, 'grad_norm': 31.61640739440918, 'learning_rate': 1.6982901355299106e-05, 'epoch': 0.28}

 28%|██▊       | 10490/37962 [24:18:48<63:24:09,  8.31s/it]
 28%|██▊       | 10491/37962 [24:18:56<63:13:32,  8.29s/it]
 28%|██▊       | 10492/37962 [24:19:05<63:33:25,  8.33s/it]
 28%|██▊       | 10493/37962 [24:19:13<63:18:00,  8.30s/it]
 28%|██▊       | 10494/37962 [24:19:21<63:15:47,  8.29s/it]
 28%|██▊       | 10495/37962 [24:19:29<63:30:48,  8.32s/it]
 28%|██▊       | 10496/37962 [24:19:38<62:54:07,  8.24s/it]
 28%|██▊       | 10497/37962 [24:19:46<63:00:25,  8.26s/it]
 28%|██▊       | 10498/37962 [24:19:54<62:43:57,  8.22s/it]
 28%|██▊       | 10499/37962 [24:20:02<63:06:26,  8.27s/it]
 28%|██▊       | 10500/37962 [24:20:10<62:06:13,  8.14s/it]
                                                           
{'loss': 1.3284, 'grad_norm': 10.761563301086426, 'learning_rate': 1.6976791765693602e-05, 'epoch': 0.28}

 28%|██▊       | 10500/37962 [24:20:10<62:06:13,  8.14s/it]
 28%|██▊       | 10501/37962 [24:20:18<62:00:46,  8.13s/it]
 28%|██▊       | 10502/37962 [24:20:27<62:14:33,  8.16s/it]
 28%|██▊       | 10503/37962 [24:20:35<62:24:56,  8.18s/it]
 28%|██▊       | 10504/37962 [24:20:43<62:40:48,  8.22s/it]
 28%|██▊       | 10505/37962 [24:20:51<62:44:16,  8.23s/it]
 28%|██▊       | 10506/37962 [24:20:59<62:00:22,  8.13s/it]
 28%|██▊       | 10507/37962 [24:21:07<62:18:27,  8.17s/it]
 28%|██▊       | 10508/37962 [24:21:16<62:33:02,  8.20s/it]
 28%|██▊       | 10509/37962 [24:21:24<62:13:16,  8.16s/it]
 28%|██▊       | 10510/37962 [24:21:32<62:07:05,  8.15s/it]
                                                           
{'loss': 1.4126, 'grad_norm': 11.6333646774292, 'learning_rate': 1.6970677097802236e-05, 'epoch': 0.28}

 28%|██▊       | 10510/37962 [24:21:32<62:07:05,  8.15s/it]
 28%|██▊       | 10511/37962 [24:21:40<62:26:55,  8.19s/it]
 28%|██▊       | 10512/37962 [24:21:48<61:37:25,  8.08s/it]
 28%|██▊       | 10513/37962 [24:21:56<61:50:59,  8.11s/it]
 28%|██▊       | 10514/37962 [24:22:04<61:56:42,  8.12s/it]
 28%|██▊       | 10515/37962 [24:22:13<62:15:37,  8.17s/it]
 28%|██▊       | 10516/37962 [24:22:21<62:24:40,  8.19s/it]
 28%|██▊       | 10517/37962 [24:22:29<62:30:39,  8.20s/it]
 28%|██▊       | 10518/37962 [24:22:37<62:40:14,  8.22s/it]
 28%|██▊       | 10519/37962 [24:22:46<62:44:55,  8.23s/it]
 28%|██▊       | 10520/37962 [24:22:54<62:39:40,  8.22s/it]
                                                           
{'loss': 1.4142, 'grad_norm': 18.506364822387695, 'learning_rate': 1.6964557356075767e-05, 'epoch': 0.28}

 28%|██▊       | 10520/37962 [24:22:54<62:39:40,  8.22s/it]
 28%|██▊       | 10521/37962 [24:23:02<62:35:45,  8.21s/it]
 28%|██▊       | 10522/37962 [24:23:10<62:31:31,  8.20s/it]
 28%|██▊       | 10523/37962 [24:23:18<62:36:10,  8.21s/it]
 28%|██▊       | 10524/37962 [24:23:26<62:14:54,  8.17s/it]
 28%|██▊       | 10525/37962 [24:23:35<62:38:48,  8.22s/it]
 28%|██▊       | 10526/37962 [24:23:43<61:46:23,  8.11s/it]
 28%|██▊       | 10527/37962 [24:23:51<62:18:37,  8.18s/it]
 28%|██▊       | 10528/37962 [24:23:59<62:03:20,  8.14s/it]
 28%|██▊       | 10529/37962 [24:24:07<61:58:20,  8.13s/it]
 28%|██▊       | 10530/37962 [24:24:15<62:05:35,  8.15s/it]
                                                           
{'loss': 1.3807, 'grad_norm': 5.859353065490723, 'learning_rate': 1.6958432544968656e-05, 'epoch': 0.28}

 28%|██▊       | 10530/37962 [24:24:15<62:05:35,  8.15s/it]
 28%|██▊       | 10531/37962 [24:24:24<62:25:06,  8.19s/it]
 28%|██▊       | 10532/37962 [24:24:32<62:37:30,  8.22s/it]
 28%|██▊       | 10533/37962 [24:24:40<62:59:18,  8.27s/it]
 28%|██▊       | 10534/37962 [24:24:48<62:09:20,  8.16s/it]
 28%|██▊       | 10535/37962 [24:24:56<61:42:16,  8.10s/it]
 28%|██▊       | 10536/37962 [24:25:04<62:04:35,  8.15s/it]
 28%|██▊       | 10537/37962 [24:25:13<62:36:08,  8.22s/it]
 28%|██▊       | 10538/37962 [24:25:21<62:41:22,  8.23s/it]
 28%|██▊       | 10539/37962 [24:25:37<79:16:40, 10.41s/it]
 28%|██▊       | 10540/37962 [24:25:45<74:34:26,  9.79s/it]
                                                           
{'loss': 1.4916, 'grad_norm': 10.047774314880371, 'learning_rate': 1.6952302668939038e-05, 'epoch': 0.28}

 28%|██▊       | 10540/37962 [24:25:45<74:34:26,  9.79s/it]
 28%|██▊       | 10541/37962 [24:25:53<71:05:51,  9.33s/it]
 28%|██▊       | 10542/37962 [24:26:08<84:33:57, 11.10s/it]
 28%|██▊       | 10543/37962 [24:26:17<77:50:05, 10.22s/it]
 28%|██▊       | 10544/37962 [24:26:25<73:17:09,  9.62s/it]
 28%|██▊       | 10545/37962 [24:26:33<69:43:21,  9.15s/it]
 28%|██▊       | 10546/37962 [24:26:41<67:52:28,  8.91s/it]
 28%|██▊       | 10547/37962 [24:26:49<66:17:09,  8.70s/it]
 28%|██▊       | 10548/37962 [24:26:57<64:28:31,  8.47s/it]
 28%|██▊       | 10549/37962 [24:27:06<63:57:11,  8.40s/it]
 28%|██▊       | 10550/37962 [24:27:14<63:37:17,  8.36s/it]
                                                           
{'loss': 1.4542, 'grad_norm': 12.138692855834961, 'learning_rate': 1.694616773244875e-05, 'epoch': 0.28}

 28%|██▊       | 10550/37962 [24:27:14<63:37:17,  8.36s/it]
 28%|██▊       | 10551/37962 [24:27:22<63:29:35,  8.34s/it]
 28%|██▊       | 10552/37962 [24:27:31<64:07:56,  8.42s/it]
 28%|██▊       | 10553/37962 [24:27:39<64:15:52,  8.44s/it]
 28%|██▊       | 10554/37962 [24:27:47<63:20:28,  8.32s/it]
 28%|██▊       | 10555/37962 [24:27:56<63:08:35,  8.29s/it]
 28%|██▊       | 10556/37962 [24:28:04<63:05:08,  8.29s/it]
 28%|██▊       | 10557/37962 [24:28:12<62:50:31,  8.26s/it]
 28%|██▊       | 10558/37962 [24:28:20<62:50:59,  8.26s/it]
 28%|██▊       | 10559/37962 [24:28:29<63:14:57,  8.31s/it]
 28%|██▊       | 10560/37962 [24:28:37<63:43:23,  8.37s/it]
                                                           
{'loss': 1.3316, 'grad_norm': 13.369186401367188, 'learning_rate': 1.69400277399633e-05, 'epoch': 0.28}

 28%|██▊       | 10560/37962 [24:28:37<63:43:23,  8.37s/it]
 28%|██▊       | 10561/37962 [24:28:45<63:30:56,  8.34s/it]
 28%|██▊       | 10562/37962 [24:28:54<63:16:31,  8.31s/it]
 28%|██▊       | 10563/37962 [24:29:02<63:14:54,  8.31s/it]
 28%|██▊       | 10564/37962 [24:29:10<63:10:24,  8.30s/it]
 28%|██▊       | 10565/37962 [24:29:18<62:57:42,  8.27s/it]
 28%|██▊       | 10566/37962 [24:29:27<62:28:36,  8.21s/it]
 28%|██▊       | 10567/37962 [24:29:35<62:49:11,  8.26s/it]
 28%|██▊       | 10568/37962 [24:29:43<63:22:57,  8.33s/it]
 28%|██▊       | 10569/37962 [24:29:52<63:00:01,  8.28s/it]
 28%|██▊       | 10570/37962 [24:30:00<62:51:52,  8.26s/it]
                                                           
{'loss': 1.441, 'grad_norm': 8.96480655670166, 'learning_rate': 1.693388269595189e-05, 'epoch': 0.28}

 28%|██▊       | 10570/37962 [24:30:00<62:51:52,  8.26s/it]
 28%|██▊       | 10571/37962 [24:30:08<63:10:32,  8.30s/it]
 28%|██▊       | 10572/37962 [24:30:16<63:07:15,  8.30s/it]
 28%|██▊       | 10573/37962 [24:30:25<63:25:35,  8.34s/it]
 28%|██▊       | 10574/37962 [24:30:33<63:16:44,  8.32s/it]
 28%|██▊       | 10575/37962 [24:30:42<63:24:12,  8.33s/it]
 28%|██▊       | 10576/37962 [24:30:50<63:19:01,  8.32s/it]
 28%|██▊       | 10577/37962 [24:30:58<63:17:18,  8.32s/it]
 28%|██▊       | 10578/37962 [24:31:06<63:13:50,  8.31s/it]
 28%|██▊       | 10579/37962 [24:31:14<62:21:28,  8.20s/it]
 28%|██▊       | 10580/37962 [24:31:23<62:13:28,  8.18s/it]
                                                           
{'loss': 1.4222, 'grad_norm': 6.4151291847229, 'learning_rate': 1.6927732604887385e-05, 'epoch': 0.28}

 28%|██▊       | 10580/37962 [24:31:23<62:13:28,  8.18s/it]
 28%|██▊       | 10581/37962 [24:31:31<62:23:34,  8.20s/it]
 28%|██▊       | 10582/37962 [24:31:39<62:48:14,  8.26s/it]
 28%|██▊       | 10583/37962 [24:31:47<61:43:19,  8.12s/it]
 28%|██▊       | 10584/37962 [24:31:55<61:57:19,  8.15s/it]
 28%|██▊       | 10585/37962 [24:32:03<61:32:26,  8.09s/it]
 28%|██▊       | 10586/37962 [24:32:11<61:30:07,  8.09s/it]
 28%|██▊       | 10587/37962 [24:32:19<61:46:31,  8.12s/it]
 28%|██▊       | 10588/37962 [24:32:28<61:57:55,  8.15s/it]
 28%|██▊       | 10589/37962 [24:32:36<62:03:56,  8.16s/it]
 28%|██▊       | 10590/37962 [24:32:44<62:12:41,  8.18s/it]
                                                           
{'loss': 1.3837, 'grad_norm': 68.81958770751953, 'learning_rate': 1.692157747124633e-05, 'epoch': 0.28}

 28%|██▊       | 10590/37962 [24:32:44<62:12:41,  8.18s/it]
 28%|██▊       | 10591/37962 [24:32:52<62:09:13,  8.17s/it]
 28%|██▊       | 10592/37962 [24:33:00<62:16:21,  8.19s/it]
 28%|██▊       | 10593/37962 [24:33:09<62:44:53,  8.25s/it]
 28%|██▊       | 10594/37962 [24:33:17<62:38:36,  8.24s/it]
 28%|██▊       | 10595/37962 [24:33:25<62:45:39,  8.26s/it]
 28%|██▊       | 10596/37962 [24:33:34<62:49:43,  8.27s/it]
 28%|██▊       | 10597/37962 [24:33:42<62:16:20,  8.19s/it]
 28%|██▊       | 10598/37962 [24:33:50<62:23:44,  8.21s/it]
 28%|██▊       | 10599/37962 [24:33:58<62:28:01,  8.22s/it]
 28%|██▊       | 10600/37962 [24:34:06<62:33:14,  8.23s/it]
                                                           
{'loss': 1.3375, 'grad_norm': 48.135948181152344, 'learning_rate': 1.6915417299508947e-05, 'epoch': 0.28}

 28%|██▊       | 10600/37962 [24:34:06<62:33:14,  8.23s/it]
 28%|██▊       | 10601/37962 [24:34:15<62:34:28,  8.23s/it]
 28%|██▊       | 10602/37962 [24:34:23<62:50:40,  8.27s/it]
 28%|██▊       | 10603/37962 [24:34:31<62:32:48,  8.23s/it]
 28%|██▊       | 10604/37962 [24:34:39<62:42:44,  8.25s/it]
 28%|██▊       | 10605/37962 [24:34:48<62:40:40,  8.25s/it]
 28%|██▊       | 10606/37962 [24:34:55<61:34:18,  8.10s/it]
 28%|██▊       | 10607/37962 [24:35:04<62:03:48,  8.17s/it]
 28%|██▊       | 10608/37962 [24:35:12<61:40:51,  8.12s/it]
 28%|██▊       | 10609/37962 [24:35:20<61:53:26,  8.15s/it]
 28%|██▊       | 10610/37962 [24:35:28<62:00:38,  8.16s/it]
                                                           
{'loss': 1.4013, 'grad_norm': 6.188029766082764, 'learning_rate': 1.6909252094159106e-05, 'epoch': 0.28}

 28%|██▊       | 10610/37962 [24:35:28<62:00:38,  8.16s/it]
 28%|██▊       | 10611/37962 [24:35:36<62:13:23,  8.19s/it]
 28%|██▊       | 10612/37962 [24:35:45<62:28:50,  8.22s/it]
 28%|██▊       | 10613/37962 [24:35:53<62:36:32,  8.24s/it]
 28%|██▊       | 10614/37962 [24:36:01<62:55:37,  8.28s/it]
 28%|██▊       | 10615/37962 [24:36:10<62:58:54,  8.29s/it]
 28%|██▊       | 10616/37962 [24:36:18<63:03:40,  8.30s/it]
 28%|██▊       | 10617/37962 [24:36:26<62:10:17,  8.18s/it]
 28%|██▊       | 10618/37962 [24:36:34<62:27:42,  8.22s/it]
 28%|██▊       | 10619/37962 [24:36:43<62:43:51,  8.26s/it]
 28%|██▊       | 10620/37962 [24:36:51<62:33:03,  8.24s/it]
                                                           
{'loss': 1.5047, 'grad_norm': 16.440561294555664, 'learning_rate': 1.6903081859684362e-05, 'epoch': 0.28}

 28%|██▊       | 10620/37962 [24:36:51<62:33:03,  8.24s/it]
 28%|██▊       | 10621/37962 [24:36:58<61:21:05,  8.08s/it]
 28%|██▊       | 10622/37962 [24:37:07<61:47:28,  8.14s/it]
 28%|██▊       | 10623/37962 [24:37:15<61:45:43,  8.13s/it]
 28%|██▊       | 10624/37962 [24:37:23<62:12:30,  8.19s/it]
 28%|██▊       | 10625/37962 [24:37:32<62:49:27,  8.27s/it]
 28%|██▊       | 10626/37962 [24:37:40<62:38:30,  8.25s/it]
 28%|██▊       | 10627/37962 [24:37:48<61:17:40,  8.07s/it]
 28%|██▊       | 10628/37962 [24:37:55<60:58:24,  8.03s/it]
 28%|██▊       | 10629/37962 [24:38:04<61:26:34,  8.09s/it]
 28%|██▊       | 10630/37962 [24:38:12<61:44:13,  8.13s/it]
                                                           
{'loss': 1.4008, 'grad_norm': 10.713726043701172, 'learning_rate': 1.6896906600575925e-05, 'epoch': 0.28}

 28%|██▊       | 10630/37962 [24:38:12<61:44:13,  8.13s/it]
 28%|██▊       | 10631/37962 [24:38:20<62:10:56,  8.19s/it]
 28%|██▊       | 10632/37962 [24:38:28<62:09:08,  8.19s/it]
 28%|██▊       | 10633/37962 [24:38:37<62:25:47,  8.22s/it]
 28%|██▊       | 10634/37962 [24:38:45<61:42:37,  8.13s/it]
 28%|██▊       | 10635/37962 [24:38:53<62:09:29,  8.19s/it]
 28%|██▊       | 10636/37962 [24:39:01<61:23:10,  8.09s/it]
 28%|██▊       | 10637/37962 [24:39:09<61:50:31,  8.15s/it]
 28%|██▊       | 10638/37962 [24:39:17<62:15:36,  8.20s/it]
 28%|██▊       | 10639/37962 [24:39:26<62:13:46,  8.20s/it]
 28%|██▊       | 10640/37962 [24:39:34<62:35:47,  8.25s/it]
                                                           
{'loss': 1.3936, 'grad_norm': 6.877676963806152, 'learning_rate': 1.6890726321328652e-05, 'epoch': 0.28}

 28%|██▊       | 10640/37962 [24:39:34<62:35:47,  8.25s/it]
 28%|██▊       | 10641/37962 [24:39:42<62:56:53,  8.29s/it]
 28%|██▊       | 10642/37962 [24:39:51<63:05:19,  8.31s/it]
 28%|██▊       | 10643/37962 [24:39:59<63:18:01,  8.34s/it]
 28%|██▊       | 10644/37962 [24:40:08<63:17:48,  8.34s/it]
 28%|██▊       | 10645/37962 [24:40:16<62:50:17,  8.28s/it]
 28%|██▊       | 10646/37962 [24:40:24<62:44:40,  8.27s/it]
 28%|██▊       | 10647/37962 [24:40:32<62:51:40,  8.28s/it]
 28%|██▊       | 10648/37962 [24:40:41<62:58:59,  8.30s/it]
 28%|██▊       | 10649/37962 [24:40:49<62:24:03,  8.22s/it]
 28%|██▊       | 10650/37962 [24:40:57<62:21:22,  8.22s/it]
                                                           
{'loss': 1.4532, 'grad_norm': 7.687427043914795, 'learning_rate': 1.6884541026441058e-05, 'epoch': 0.28}

 28%|██▊       | 10650/37962 [24:40:57<62:21:22,  8.22s/it]
 28%|██▊       | 10651/37962 [24:41:05<62:39:53,  8.26s/it]
 28%|██▊       | 10652/37962 [24:41:21<78:52:14, 10.40s/it]
 28%|██▊       | 10653/37962 [24:41:29<73:35:56,  9.70s/it]
 28%|██▊       | 10654/37962 [24:41:37<70:19:02,  9.27s/it]
 28%|██▊       | 10655/37962 [24:41:45<67:36:37,  8.91s/it]
 28%|██▊       | 10656/37962 [24:41:53<65:52:51,  8.69s/it]
 28%|██▊       | 10657/37962 [24:42:01<65:07:08,  8.59s/it]
 28%|██▊       | 10658/37962 [24:42:10<64:56:59,  8.56s/it]
 28%|██▊       | 10659/37962 [24:42:18<63:44:12,  8.40s/it]
 28%|██▊       | 10660/37962 [24:42:26<63:25:16,  8.36s/it]
                                                           
{'loss': 1.3041, 'grad_norm': 7.978679180145264, 'learning_rate': 1.6878350720415326e-05, 'epoch': 0.28}

 28%|██▊       | 10660/37962 [24:42:26<63:25:16,  8.36s/it]
 28%|██▊       | 10661/37962 [24:42:35<63:04:28,  8.32s/it]
 28%|██▊       | 10662/37962 [24:42:43<63:13:36,  8.34s/it]
 28%|██▊       | 10663/37962 [24:42:51<63:21:55,  8.36s/it]
 28%|██▊       | 10664/37962 [24:43:00<63:22:05,  8.36s/it]
 28%|██▊       | 10665/37962 [24:43:08<63:19:11,  8.35s/it]
 28%|██▊       | 10666/37962 [24:43:16<62:58:38,  8.31s/it]
 28%|██▊       | 10667/37962 [24:43:24<62:42:16,  8.27s/it]
 28%|██▊       | 10668/37962 [24:43:33<62:37:52,  8.26s/it]
 28%|██▊       | 10669/37962 [24:43:41<62:56:18,  8.30s/it]
 28%|██▊       | 10670/37962 [24:43:49<62:41:19,  8.27s/it]
                                                           
{'loss': 1.482, 'grad_norm': 15.336507797241211, 'learning_rate': 1.6872155407757257e-05, 'epoch': 0.28}

 28%|██▊       | 10670/37962 [24:43:49<62:41:19,  8.27s/it]
 28%|██▊       | 10671/37962 [24:43:57<62:22:20,  8.23s/it]
 28%|██▊       | 10672/37962 [24:44:06<62:31:28,  8.25s/it]
 28%|██▊       | 10673/37962 [24:44:14<61:55:07,  8.17s/it]
 28%|██▊       | 10674/37962 [24:44:22<62:12:18,  8.21s/it]
 28%|██▊       | 10675/37962 [24:44:30<63:01:21,  8.31s/it]
 28%|██▊       | 10676/37962 [24:44:39<62:30:42,  8.25s/it]
 28%|██▊       | 10677/37962 [24:44:46<61:42:25,  8.14s/it]
 28%|██▊       | 10678/37962 [24:44:55<61:56:07,  8.17s/it]
 28%|██▊       | 10679/37962 [24:45:03<62:17:11,  8.22s/it]
 28%|██▊       | 10680/37962 [24:45:11<62:19:42,  8.22s/it]
                                                           
{'loss': 1.4064, 'grad_norm': 21.367015838623047, 'learning_rate': 1.6865955092976324e-05, 'epoch': 0.28}

 28%|██▊       | 10680/37962 [24:45:11<62:19:42,  8.22s/it]
 28%|██▊       | 10681/37962 [24:45:19<62:14:08,  8.21s/it]
 28%|██▊       | 10682/37962 [24:45:27<61:49:50,  8.16s/it]
 28%|██▊       | 10683/37962 [24:45:35<61:17:53,  8.09s/it]
 28%|██▊       | 10684/37962 [24:45:44<61:37:26,  8.13s/it]
 28%|██▊       | 10685/37962 [24:45:52<61:30:55,  8.12s/it]
 28%|██▊       | 10686/37962 [24:46:00<61:19:26,  8.09s/it]
 28%|██▊       | 10687/37962 [24:46:08<61:41:12,  8.14s/it]
 28%|██▊       | 10688/37962 [24:46:16<61:46:19,  8.15s/it]
 28%|██▊       | 10689/37962 [24:46:24<61:52:04,  8.17s/it]
 28%|██▊       | 10690/37962 [24:46:32<61:37:49,  8.14s/it]
                                                           
{'loss': 1.4977, 'grad_norm': 23.915796279907227, 'learning_rate': 1.685974978058562e-05, 'epoch': 0.28}

 28%|██▊       | 10690/37962 [24:46:32<61:37:49,  8.14s/it]
 28%|██▊       | 10691/37962 [24:46:41<62:12:50,  8.21s/it]
 28%|██▊       | 10692/37962 [24:46:49<61:53:05,  8.17s/it]
 28%|██▊       | 10693/37962 [24:46:57<62:06:30,  8.20s/it]
 28%|██▊       | 10694/37962 [24:47:06<62:24:35,  8.24s/it]
 28%|██▊       | 10695/37962 [24:47:14<63:16:15,  8.35s/it]
 28%|██▊       | 10696/37962 [24:47:22<63:13:43,  8.35s/it]
 28%|██▊       | 10697/37962 [24:47:31<62:33:02,  8.26s/it]
 28%|██▊       | 10698/37962 [24:47:39<62:51:58,  8.30s/it]
 28%|██▊       | 10699/37962 [24:47:47<61:54:51,  8.18s/it]
 28%|██▊       | 10700/37962 [24:47:55<62:26:21,  8.25s/it]
                                                           
{'loss': 1.3661, 'grad_norm': 9.73816204071045, 'learning_rate': 1.685353947510189e-05, 'epoch': 0.28}

 28%|██▊       | 10700/37962 [24:47:55<62:26:21,  8.25s/it]
 28%|██▊       | 10701/37962 [24:48:03<62:14:39,  8.22s/it]
 28%|██▊       | 10702/37962 [24:48:12<62:07:17,  8.20s/it]
 28%|██▊       | 10703/37962 [24:48:20<62:49:10,  8.30s/it]
 28%|██▊       | 10704/37962 [24:48:28<62:23:45,  8.24s/it]
 28%|██▊       | 10705/37962 [24:48:37<62:47:42,  8.29s/it]
 28%|██▊       | 10706/37962 [24:48:45<62:09:30,  8.21s/it]
 28%|██▊       | 10707/37962 [24:48:53<62:25:45,  8.25s/it]
 28%|██▊       | 10708/37962 [24:49:01<62:17:07,  8.23s/it]
 28%|██▊       | 10709/37962 [24:49:09<62:18:00,  8.23s/it]
 28%|██▊       | 10710/37962 [24:49:17<61:26:31,  8.12s/it]
                                                           
{'loss': 1.4091, 'grad_norm': 5.3168816566467285, 'learning_rate': 1.6847324181045503e-05, 'epoch': 0.28}

 28%|██▊       | 10710/37962 [24:49:17<61:26:31,  8.12s/it]
 28%|██▊       | 10711/37962 [24:49:26<61:56:09,  8.18s/it]
 28%|██▊       | 10712/37962 [24:49:34<61:42:35,  8.15s/it]
 28%|██▊       | 10713/37962 [24:49:42<61:53:43,  8.18s/it]
 28%|██▊       | 10714/37962 [24:49:50<62:25:53,  8.25s/it]
 28%|██▊       | 10715/37962 [24:49:58<61:52:35,  8.18s/it]
 28%|██▊       | 10716/37962 [24:50:06<61:56:24,  8.18s/it]
 28%|██▊       | 10717/37962 [24:50:14<61:14:03,  8.09s/it]
 28%|██▊       | 10718/37962 [24:50:22<61:00:50,  8.06s/it]
 28%|██▊       | 10719/37962 [24:50:31<61:40:04,  8.15s/it]
 28%|██▊       | 10720/37962 [24:50:39<61:17:06,  8.10s/it]
                                                           
{'loss': 1.2937, 'grad_norm': 3.060407876968384, 'learning_rate': 1.6841103902940464e-05, 'epoch': 0.28}

 28%|██▊       | 10720/37962 [24:50:39<61:17:06,  8.10s/it]
 28%|██▊       | 10721/37962 [24:50:47<61:44:17,  8.16s/it]
 28%|██▊       | 10722/37962 [24:50:55<62:04:19,  8.20s/it]
 28%|██▊       | 10723/37962 [24:51:04<62:36:41,  8.27s/it]
 28%|██▊       | 10724/37962 [24:51:12<62:57:25,  8.32s/it]
 28%|██▊       | 10725/37962 [24:51:20<62:48:33,  8.30s/it]
 28%|██▊       | 10726/37962 [24:51:29<62:21:21,  8.24s/it]
 28%|██▊       | 10727/37962 [24:51:37<62:22:20,  8.24s/it]
 28%|██▊       | 10728/37962 [24:51:45<62:23:17,  8.25s/it]
 28%|██▊       | 10729/37962 [24:51:53<62:14:55,  8.23s/it]
 28%|██▊       | 10730/37962 [24:52:02<62:41:21,  8.29s/it]
                                                           
{'loss': 1.2962, 'grad_norm': 11.058568000793457, 'learning_rate': 1.6834878645314413e-05, 'epoch': 0.28}

 28%|██▊       | 10730/37962 [24:52:02<62:41:21,  8.29s/it]
 28%|██▊       | 10731/37962 [24:52:09<61:26:51,  8.12s/it]
 28%|██▊       | 10732/37962 [24:52:17<61:18:10,  8.10s/it]
 28%|██▊       | 10733/37962 [24:52:26<61:23:52,  8.12s/it]
 28%|██▊       | 10734/37962 [24:52:34<61:45:31,  8.17s/it]
 28%|██▊       | 10735/37962 [24:52:42<62:03:59,  8.21s/it]
 28%|██▊       | 10736/37962 [24:52:50<61:38:39,  8.15s/it]
 28%|██▊       | 10737/37962 [24:52:58<61:53:12,  8.18s/it]
 28%|██▊       | 10738/37962 [24:53:07<61:36:10,  8.15s/it]
 28%|██▊       | 10739/37962 [24:53:15<61:51:52,  8.18s/it]
 28%|██▊       | 10740/37962 [24:53:23<61:58:43,  8.20s/it]
                                                           
{'loss': 1.4272, 'grad_norm': 5.227705001831055, 'learning_rate': 1.68286484126986e-05, 'epoch': 0.28}

 28%|██▊       | 10740/37962 [24:53:23<61:58:43,  8.20s/it]
 28%|██▊       | 10741/37962 [24:53:31<62:03:11,  8.21s/it]
 28%|██▊       | 10742/37962 [24:53:40<62:33:37,  8.27s/it]
 28%|██▊       | 10743/37962 [24:53:47<61:23:42,  8.12s/it]
 28%|██▊       | 10744/37962 [24:53:56<61:31:09,  8.14s/it]
 28%|██▊       | 10745/37962 [24:54:04<62:05:51,  8.21s/it]
 28%|██▊       | 10746/37962 [24:54:12<61:54:47,  8.19s/it]
 28%|██▊       | 10747/37962 [24:54:20<61:54:03,  8.19s/it]
 28%|██▊       | 10748/37962 [24:54:29<61:58:04,  8.20s/it]
 28%|██▊       | 10749/37962 [24:54:37<62:03:07,  8.21s/it]
 28%|██▊       | 10750/37962 [24:54:45<62:21:48,  8.25s/it]
                                                           
{'loss': 1.5406, 'grad_norm': 12.315330505371094, 'learning_rate': 1.6822413209627907e-05, 'epoch': 0.28}

 28%|██▊       | 10750/37962 [24:54:45<62:21:48,  8.25s/it]
 28%|██▊       | 10751/37962 [24:54:53<62:31:13,  8.27s/it]
 28%|██▊       | 10752/37962 [24:55:02<62:32:36,  8.27s/it]
 28%|██▊       | 10753/37962 [24:55:09<61:24:56,  8.13s/it]
 28%|██▊       | 10754/37962 [24:55:18<61:26:10,  8.13s/it]
 28%|██▊       | 10755/37962 [24:55:26<61:13:06,  8.10s/it]
 28%|██▊       | 10756/37962 [24:55:34<61:22:51,  8.12s/it]
 28%|██▊       | 10757/37962 [24:55:42<61:30:06,  8.14s/it]
 28%|██▊       | 10758/37962 [24:55:50<61:41:23,  8.16s/it]
 28%|██▊       | 10759/37962 [24:55:58<61:36:45,  8.15s/it]
 28%|██▊       | 10760/37962 [24:56:07<61:48:13,  8.18s/it]
                                                           
{'loss': 1.4798, 'grad_norm': 8.483112335205078, 'learning_rate': 1.6816173040640827e-05, 'epoch': 0.28}

 28%|██▊       | 10760/37962 [24:56:07<61:48:13,  8.18s/it]
 28%|██▊       | 10761/37962 [24:56:15<61:58:19,  8.20s/it]
 28%|██▊       | 10762/37962 [24:56:23<62:20:41,  8.25s/it]
 28%|██▊       | 10763/37962 [24:56:32<62:42:24,  8.30s/it]
 28%|██▊       | 10764/37962 [24:56:40<62:21:41,  8.25s/it]
 28%|██▊       | 10765/37962 [24:56:48<62:38:02,  8.29s/it]
 28%|██▊       | 10766/37962 [24:56:56<61:46:13,  8.18s/it]
 28%|██▊       | 10767/37962 [24:57:04<62:07:06,  8.22s/it]
 28%|██▊       | 10768/37962 [24:57:13<62:28:05,  8.27s/it]
 28%|██▊       | 10769/37962 [24:57:21<62:31:03,  8.28s/it]
 28%|██▊       | 10770/37962 [24:57:29<62:27:16,  8.27s/it]
                                                           
{'loss': 1.4791, 'grad_norm': 5.636556625366211, 'learning_rate': 1.6809927910279475e-05, 'epoch': 0.28}

 28%|██▊       | 10770/37962 [24:57:29<62:27:16,  8.27s/it]
 28%|██▊       | 10771/37962 [24:57:38<62:57:24,  8.34s/it]
 28%|██▊       | 10772/37962 [24:57:46<62:43:56,  8.31s/it]
 28%|██▊       | 10773/37962 [24:57:54<61:37:32,  8.16s/it]
 28%|██▊       | 10774/37962 [24:58:02<61:55:18,  8.20s/it]
 28%|██▊       | 10775/37962 [24:58:10<62:12:04,  8.24s/it]
 28%|██▊       | 10776/37962 [24:58:26<77:59:22, 10.33s/it]
 28%|██▊       | 10777/37962 [24:58:34<73:38:08,  9.75s/it]
 28%|██▊       | 10778/37962 [24:58:42<70:13:58,  9.30s/it]
 28%|██▊       | 10779/37962 [24:58:51<68:14:30,  9.04s/it]
 28%|██▊       | 10780/37962 [24:58:59<66:24:10,  8.79s/it]
                                                           
{'loss': 1.4543, 'grad_norm': 16.112831115722656, 'learning_rate': 1.680367782308957e-05, 'epoch': 0.28}

 28%|██▊       | 10780/37962 [24:58:59<66:24:10,  8.79s/it]
 28%|██▊       | 10781/37962 [24:59:07<65:09:43,  8.63s/it]
 28%|██▊       | 10782/37962 [24:59:16<64:23:42,  8.53s/it]
 28%|██▊       | 10783/37962 [24:59:24<63:47:40,  8.45s/it]
 28%|██▊       | 10784/37962 [24:59:32<63:13:24,  8.37s/it]
 28%|██▊       | 10785/37962 [24:59:40<63:03:42,  8.35s/it]
 28%|██▊       | 10786/37962 [24:59:49<63:12:04,  8.37s/it]
 28%|██▊       | 10787/37962 [24:59:57<63:12:18,  8.37s/it]
 28%|██▊       | 10788/37962 [25:00:05<62:59:21,  8.34s/it]
 28%|██▊       | 10789/37962 [25:00:14<62:38:58,  8.30s/it]
 28%|██▊       | 10790/37962 [25:00:21<61:38:41,  8.17s/it]
                                                           
{'loss': 1.3782, 'grad_norm': 21.118633270263672, 'learning_rate': 1.6797422783620442e-05, 'epoch': 0.28}

 28%|██▊       | 10790/37962 [25:00:21<61:38:41,  8.17s/it]
 28%|██▊       | 10791/37962 [25:00:30<61:32:33,  8.15s/it]
 28%|██▊       | 10792/37962 [25:00:38<61:42:16,  8.18s/it]
 28%|██▊       | 10793/37962 [25:00:46<61:29:58,  8.15s/it]
 28%|██▊       | 10794/37962 [25:00:54<61:29:01,  8.15s/it]
 28%|██▊       | 10795/37962 [25:01:02<62:05:52,  8.23s/it]
 28%|██▊       | 10796/37962 [25:01:11<61:51:15,  8.20s/it]
 28%|██▊       | 10797/37962 [25:01:19<61:48:49,  8.19s/it]
 28%|██▊       | 10798/37962 [25:01:27<61:47:28,  8.19s/it]
 28%|██▊       | 10799/37962 [25:01:35<61:41:19,  8.18s/it]
 28%|██▊       | 10800/37962 [25:01:44<62:38:43,  8.30s/it]
                                                           
{'loss': 1.4306, 'grad_norm': 16.596115112304688, 'learning_rate': 1.6791162796425028e-05, 'epoch': 0.28}

 28%|██▊       | 10800/37962 [25:01:44<62:38:43,  8.30s/it]
 28%|██▊       | 10801/37962 [25:01:52<62:40:41,  8.31s/it]
 28%|██▊       | 10802/37962 [25:02:00<61:42:42,  8.18s/it]
 28%|██▊       | 10803/37962 [25:02:08<61:54:50,  8.21s/it]
 28%|██▊       | 10804/37962 [25:02:16<61:45:38,  8.19s/it]
 28%|██▊       | 10805/37962 [25:02:25<61:53:34,  8.20s/it]
 28%|██▊       | 10806/37962 [25:02:33<62:00:43,  8.22s/it]
 28%|██▊       | 10807/37962 [25:02:41<62:00:10,  8.22s/it]
 28%|██▊       | 10808/37962 [25:02:49<61:28:01,  8.15s/it]
 28%|██▊       | 10809/37962 [25:02:57<61:08:25,  8.11s/it]
 28%|██▊       | 10810/37962 [25:03:05<60:34:04,  8.03s/it]
                                                           
{'loss': 1.4283, 'grad_norm': 14.366751670837402, 'learning_rate': 1.6784897866059863e-05, 'epoch': 0.28}

 28%|██▊       | 10810/37962 [25:03:05<60:34:04,  8.03s/it]
 28%|██▊       | 10811/37962 [25:03:13<60:57:55,  8.08s/it]
 28%|██▊       | 10812/37962 [25:03:21<61:08:16,  8.11s/it]
 28%|██▊       | 10813/37962 [25:03:29<61:30:35,  8.16s/it]
 28%|██▊       | 10814/37962 [25:03:38<62:19:26,  8.26s/it]
 28%|██▊       | 10815/37962 [25:03:46<61:52:31,  8.21s/it]
 28%|██▊       | 10816/37962 [25:03:54<62:00:51,  8.22s/it]
 28%|██▊       | 10817/37962 [25:04:02<61:35:53,  8.17s/it]
 28%|██▊       | 10818/37962 [25:04:11<61:34:28,  8.17s/it]
 28%|██▊       | 10819/37962 [25:04:19<61:33:56,  8.17s/it]
 29%|██▊       | 10820/37962 [25:04:27<61:18:55,  8.13s/it]
                                                           
{'loss': 1.3525, 'grad_norm': 12.539999008178711, 'learning_rate': 1.6778627997085084e-05, 'epoch': 0.29}

 29%|██▊       | 10820/37962 [25:04:27<61:18:55,  8.13s/it]
 29%|██▊       | 10821/37962 [25:04:35<62:06:53,  8.24s/it]
 29%|██▊       | 10822/37962 [25:04:43<61:52:07,  8.21s/it]
 29%|██▊       | 10823/37962 [25:04:52<62:01:37,  8.23s/it]
 29%|██▊       | 10824/37962 [25:05:00<62:00:28,  8.23s/it]
 29%|██▊       | 10825/37962 [25:05:08<62:37:31,  8.31s/it]
 29%|██▊       | 10826/37962 [25:05:17<62:28:30,  8.29s/it]
 29%|██▊       | 10827/37962 [25:05:25<62:45:45,  8.33s/it]
 29%|██▊       | 10828/37962 [25:05:33<63:01:14,  8.36s/it]
 29%|██▊       | 10829/37962 [25:05:42<63:11:00,  8.38s/it]
 29%|██▊       | 10830/37962 [25:05:50<62:52:14,  8.34s/it]
                                                           
{'loss': 1.4816, 'grad_norm': 19.15412712097168, 'learning_rate': 1.6772353194064413e-05, 'epoch': 0.29}

 29%|██▊       | 10830/37962 [25:05:50<62:52:14,  8.34s/it]
 29%|██▊       | 10831/37962 [25:05:58<62:47:33,  8.33s/it]
 29%|██▊       | 10832/37962 [25:06:07<62:54:55,  8.35s/it]
 29%|██▊       | 10833/37962 [25:06:15<62:50:26,  8.34s/it]
 29%|██▊       | 10834/37962 [25:06:23<62:50:36,  8.34s/it]
 29%|██▊       | 10835/37962 [25:06:32<62:36:43,  8.31s/it]
 29%|██▊       | 10836/37962 [25:06:40<62:04:41,  8.24s/it]
 29%|██▊       | 10837/37962 [25:06:48<61:44:13,  8.19s/it]
 29%|██▊       | 10838/37962 [25:06:56<61:50:59,  8.21s/it]
 29%|██▊       | 10839/37962 [25:07:05<62:12:36,  8.26s/it]
 29%|██▊       | 10840/37962 [25:07:13<61:54:26,  8.22s/it]
                                                           
{'loss': 1.3326, 'grad_norm': 22.995914459228516, 'learning_rate': 1.676607346156518e-05, 'epoch': 0.29}

 29%|██▊       | 10840/37962 [25:07:13<61:54:26,  8.22s/it]
 29%|██▊       | 10841/37962 [25:07:21<62:23:49,  8.28s/it]
 29%|██▊       | 10842/37962 [25:07:29<62:20:54,  8.28s/it]
 29%|██▊       | 10843/37962 [25:07:38<62:06:36,  8.25s/it]
 29%|██▊       | 10844/37962 [25:07:46<61:59:54,  8.23s/it]
 29%|██▊       | 10845/37962 [25:07:54<62:02:39,  8.24s/it]
 29%|██▊       | 10846/37962 [25:08:02<62:23:56,  8.28s/it]
 29%|██▊       | 10847/37962 [25:08:11<62:13:06,  8.26s/it]
 29%|██▊       | 10848/37962 [25:08:19<62:02:48,  8.24s/it]
 29%|██▊       | 10849/37962 [25:08:27<61:57:42,  8.23s/it]
 29%|██▊       | 10850/37962 [25:08:35<62:03:35,  8.24s/it]
                                                           
{'loss': 1.374, 'grad_norm': 8.671506881713867, 'learning_rate': 1.6759788804158286e-05, 'epoch': 0.29}

 29%|██▊       | 10850/37962 [25:08:35<62:03:35,  8.24s/it]
 29%|██▊       | 10851/37962 [25:08:43<61:39:41,  8.19s/it]
 29%|██▊       | 10852/37962 [25:08:51<60:59:45,  8.10s/it]
 29%|██▊       | 10853/37962 [25:08:59<61:11:52,  8.13s/it]
 29%|██▊       | 10854/37962 [25:09:08<61:56:11,  8.23s/it]
 29%|██▊       | 10855/37962 [25:09:16<62:02:16,  8.24s/it]
 29%|██▊       | 10856/37962 [25:09:24<61:43:21,  8.20s/it]
 29%|██▊       | 10857/37962 [25:09:32<61:46:03,  8.20s/it]
 29%|██▊       | 10858/37962 [25:09:41<62:03:20,  8.24s/it]
 29%|██▊       | 10859/37962 [25:09:49<61:55:57,  8.23s/it]
 29%|██▊       | 10860/37962 [25:09:57<61:40:03,  8.19s/it]
                                                           
{'loss': 1.4263, 'grad_norm': 9.748671531677246, 'learning_rate': 1.6753499226418228e-05, 'epoch': 0.29}

 29%|██▊       | 10860/37962 [25:09:57<61:40:03,  8.19s/it]
 29%|██▊       | 10861/37962 [25:10:05<61:41:39,  8.20s/it]
 29%|██▊       | 10862/37962 [25:10:13<61:49:46,  8.21s/it]
 29%|██▊       | 10863/37962 [25:10:21<61:11:39,  8.13s/it]
 29%|██▊       | 10864/37962 [25:10:30<61:37:55,  8.19s/it]
 29%|██▊       | 10865/37962 [25:10:38<61:38:44,  8.19s/it]
 29%|██▊       | 10866/37962 [25:10:46<61:41:40,  8.20s/it]
 29%|██▊       | 10867/37962 [25:10:54<61:45:15,  8.21s/it]
 29%|██▊       | 10868/37962 [25:11:03<62:38:45,  8.32s/it]
 29%|██▊       | 10869/37962 [25:11:11<62:44:56,  8.34s/it]
 29%|██▊       | 10870/37962 [25:11:19<62:14:54,  8.27s/it]
                                                           
{'loss': 1.4203, 'grad_norm': 3.421365976333618, 'learning_rate': 1.674720473292308e-05, 'epoch': 0.29}

 29%|██▊       | 10870/37962 [25:11:19<62:14:54,  8.27s/it]
 29%|██▊       | 10871/37962 [25:11:28<62:04:38,  8.25s/it]
 29%|██▊       | 10872/37962 [25:11:43<78:44:02, 10.46s/it]
 29%|██▊       | 10873/37962 [25:11:52<74:04:13,  9.84s/it]
 29%|██▊       | 10874/37962 [25:12:00<70:37:59,  9.39s/it]
 29%|██▊       | 10875/37962 [25:12:08<67:02:16,  8.91s/it]
 29%|██▊       | 10876/37962 [25:12:16<65:27:45,  8.70s/it]
 29%|██▊       | 10877/37962 [25:12:24<63:59:20,  8.51s/it]
 29%|██▊       | 10878/37962 [25:12:32<63:22:29,  8.42s/it]
 29%|██▊       | 10879/37962 [25:12:40<62:22:48,  8.29s/it]
 29%|██▊       | 10880/37962 [25:12:49<62:13:14,  8.27s/it]
                                                           
{'loss': 1.4973, 'grad_norm': 23.0704288482666, 'learning_rate': 1.674090532825449e-05, 'epoch': 0.29}

 29%|██▊       | 10880/37962 [25:12:49<62:13:14,  8.27s/it]
 29%|██▊       | 10881/37962 [25:12:57<61:59:16,  8.24s/it]
 29%|██▊       | 10882/37962 [25:13:05<62:45:04,  8.34s/it]
 29%|██▊       | 10883/37962 [25:13:14<62:33:32,  8.32s/it]
 29%|██▊       | 10884/37962 [25:13:22<62:23:44,  8.30s/it]
 29%|██▊       | 10885/37962 [25:13:30<62:22:18,  8.29s/it]
 29%|██▊       | 10886/37962 [25:13:39<62:55:48,  8.37s/it]
 29%|██▊       | 10887/37962 [25:13:47<62:33:53,  8.32s/it]
 29%|██▊       | 10888/37962 [25:13:55<62:39:17,  8.33s/it]
 29%|██▊       | 10889/37962 [25:14:03<62:33:26,  8.32s/it]
 29%|██▊       | 10890/37962 [25:14:12<62:58:36,  8.37s/it]
                                                           
{'loss': 1.4191, 'grad_norm': 11.372081756591797, 'learning_rate': 1.6734601016997695e-05, 'epoch': 0.29}

 29%|██▊       | 10890/37962 [25:14:12<62:58:36,  8.37s/it]
 29%|██▊       | 10891/37962 [25:14:20<62:41:01,  8.34s/it]
 29%|██▊       | 10892/37962 [25:14:28<61:57:21,  8.24s/it]
 29%|██▊       | 10893/37962 [25:14:36<62:01:47,  8.25s/it]
 29%|██▊       | 10894/37962 [25:14:45<61:55:34,  8.24s/it]
 29%|██▊       | 10895/37962 [25:14:53<62:12:00,  8.27s/it]
 29%|██▊       | 10896/37962 [25:15:01<62:14:34,  8.28s/it]
 29%|██▊       | 10897/37962 [25:15:09<61:50:44,  8.23s/it]
 29%|██▊       | 10898/37962 [25:15:18<61:41:02,  8.21s/it]
 29%|██▊       | 10899/37962 [25:15:26<61:02:44,  8.12s/it]
 29%|██▊       | 10900/37962 [25:15:34<61:28:30,  8.18s/it]
                                                           
{'loss': 1.4, 'grad_norm': 2.87218976020813, 'learning_rate': 1.6728291803741485e-05, 'epoch': 0.29}

 29%|██▊       | 10900/37962 [25:15:34<61:28:30,  8.18s/it]
 29%|██▊       | 10901/37962 [25:15:42<61:36:06,  8.20s/it]
 29%|██▊       | 10902/37962 [25:15:50<61:44:50,  8.21s/it]
 29%|██▊       | 10903/37962 [25:15:59<61:43:47,  8.21s/it]
 29%|██▊       | 10904/37962 [25:16:07<61:26:11,  8.17s/it]
 29%|██▊       | 10905/37962 [25:16:15<61:12:56,  8.14s/it]
 29%|██▊       | 10906/37962 [25:16:23<61:41:29,  8.21s/it]
 29%|██▊       | 10907/37962 [25:16:31<61:37:40,  8.20s/it]
 29%|██▊       | 10908/37962 [25:16:39<61:37:59,  8.20s/it]
 29%|██▊       | 10909/37962 [25:16:48<61:32:18,  8.19s/it]
 29%|██▊       | 10910/37962 [25:16:56<61:42:21,  8.21s/it]
                                                           
{'loss': 1.4073, 'grad_norm': 19.180843353271484, 'learning_rate': 1.672197769307823e-05, 'epoch': 0.29}

 29%|██▊       | 10910/37962 [25:16:56<61:42:21,  8.21s/it]
 29%|██▊       | 10911/37962 [25:17:04<61:49:29,  8.23s/it]
 29%|██▊       | 10912/37962 [25:17:12<61:55:40,  8.24s/it]
 29%|██▊       | 10913/37962 [25:17:20<60:59:59,  8.12s/it]
 29%|██▊       | 10914/37962 [25:17:29<61:21:34,  8.17s/it]
 29%|██▉       | 10915/37962 [25:17:37<61:35:18,  8.20s/it]
 29%|██▉       | 10916/37962 [25:17:45<61:57:43,  8.25s/it]
 29%|██▉       | 10917/37962 [25:17:53<61:58:53,  8.25s/it]
 29%|██▉       | 10918/37962 [25:18:02<61:55:29,  8.24s/it]
 29%|██▉       | 10919/37962 [25:18:10<62:15:19,  8.29s/it]
 29%|██▉       | 10920/37962 [25:18:18<62:37:31,  8.34s/it]
                                                           
{'loss': 1.4504, 'grad_norm': 2.8647992610931396, 'learning_rate': 1.6715658689603866e-05, 'epoch': 0.29}

 29%|██▉       | 10920/37962 [25:18:18<62:37:31,  8.34s/it]
 29%|██▉       | 10921/37962 [25:18:26<61:22:34,  8.17s/it]
 29%|██▉       | 10922/37962 [25:18:35<61:31:06,  8.19s/it]
 29%|██▉       | 10923/37962 [25:18:43<62:02:45,  8.26s/it]
 29%|██▉       | 10924/37962 [25:18:51<61:30:49,  8.19s/it]
 29%|██▉       | 10925/37962 [25:18:59<61:27:13,  8.18s/it]
 29%|██▉       | 10926/37962 [25:19:07<61:38:37,  8.21s/it]
 29%|██▉       | 10927/37962 [25:19:16<61:33:26,  8.20s/it]
 29%|██▉       | 10928/37962 [25:19:24<61:50:46,  8.24s/it]
 29%|██▉       | 10929/37962 [25:19:32<62:16:36,  8.29s/it]
 29%|██▉       | 10930/37962 [25:19:41<62:15:54,  8.29s/it]
                                                           
{'loss': 1.3366, 'grad_norm': 13.637640953063965, 'learning_rate': 1.670933479791788e-05, 'epoch': 0.29}

 29%|██▉       | 10930/37962 [25:19:41<62:15:54,  8.29s/it]
 29%|██▉       | 10931/37962 [25:19:49<62:04:27,  8.27s/it]
 29%|██▉       | 10932/37962 [25:19:57<62:09:18,  8.28s/it]
 29%|██▉       | 10933/37962 [25:20:05<61:44:10,  8.22s/it]
 29%|██▉       | 10934/37962 [25:20:14<61:57:52,  8.25s/it]
 29%|██▉       | 10935/37962 [25:20:22<62:41:09,  8.35s/it]
 29%|██▉       | 10936/37962 [25:20:31<63:08:24,  8.41s/it]
 29%|██▉       | 10937/37962 [25:20:39<63:04:28,  8.40s/it]
 29%|██▉       | 10938/37962 [25:20:49<66:04:15,  8.80s/it]
 29%|██▉       | 10939/37962 [25:20:57<64:17:38,  8.57s/it]
 29%|██▉       | 10940/37962 [25:21:05<63:27:41,  8.45s/it]
                                                           
{'loss': 1.3657, 'grad_norm': 17.84374237060547, 'learning_rate': 1.6703006022623327e-05, 'epoch': 0.29}

 29%|██▉       | 10940/37962 [25:21:05<63:27:41,  8.45s/it]
 29%|██▉       | 10941/37962 [25:21:13<63:24:39,  8.45s/it]
 29%|██▉       | 10942/37962 [25:21:22<63:47:53,  8.50s/it]
 29%|██▉       | 10943/37962 [25:21:30<63:29:38,  8.46s/it]
 29%|██▉       | 10944/37962 [25:21:39<63:18:40,  8.44s/it]
 29%|██▉       | 10945/37962 [25:21:47<62:49:07,  8.37s/it]
 29%|██▉       | 10946/37962 [25:21:55<63:05:46,  8.41s/it]
 29%|██▉       | 10947/37962 [25:22:04<63:02:53,  8.40s/it]
 29%|██▉       | 10948/37962 [25:22:12<62:35:11,  8.34s/it]
 29%|██▉       | 10949/37962 [25:22:20<62:35:48,  8.34s/it]
 29%|██▉       | 10950/37962 [25:22:29<62:15:46,  8.30s/it]
                                                           
{'loss': 1.2915, 'grad_norm': 9.04928970336914, 'learning_rate': 1.6696672368326814e-05, 'epoch': 0.29}

 29%|██▉       | 10950/37962 [25:22:29<62:15:46,  8.30s/it]
 29%|██▉       | 10951/37962 [25:22:37<62:47:21,  8.37s/it]
 29%|██▉       | 10952/37962 [25:22:45<62:41:06,  8.35s/it]
 29%|██▉       | 10953/37962 [25:22:54<62:28:14,  8.33s/it]
 29%|██▉       | 10954/37962 [25:23:02<61:50:54,  8.24s/it]
 29%|██▉       | 10955/37962 [25:23:10<61:27:48,  8.19s/it]
 29%|██▉       | 10956/37962 [25:23:18<61:45:54,  8.23s/it]
 29%|██▉       | 10957/37962 [25:23:26<61:02:20,  8.14s/it]
 29%|██▉       | 10958/37962 [25:23:34<61:32:33,  8.20s/it]
 29%|██▉       | 10959/37962 [25:23:43<61:25:04,  8.19s/it]
 29%|██▉       | 10960/37962 [25:23:51<61:52:10,  8.25s/it]
                                                           
{'loss': 1.2984, 'grad_norm': 16.57237434387207, 'learning_rate': 1.6690333839638494e-05, 'epoch': 0.29}

 29%|██▉       | 10960/37962 [25:23:51<61:52:10,  8.25s/it]
 29%|██▉       | 10961/37962 [25:24:00<62:44:59,  8.37s/it]
 29%|██▉       | 10962/37962 [25:24:08<62:41:38,  8.36s/it]
 29%|██▉       | 10963/37962 [25:24:16<63:00:12,  8.40s/it]
 29%|██▉       | 10964/37962 [25:24:25<62:55:45,  8.39s/it]
 29%|██▉       | 10965/37962 [25:24:33<62:41:14,  8.36s/it]
 29%|██▉       | 10966/37962 [25:24:41<61:53:07,  8.25s/it]
 29%|██▉       | 10967/37962 [25:24:49<61:24:24,  8.19s/it]
 29%|██▉       | 10968/37962 [25:24:57<61:37:05,  8.22s/it]
 29%|██▉       | 10969/37962 [25:25:06<61:59:51,  8.27s/it]
 29%|██▉       | 10970/37962 [25:25:14<62:00:20,  8.27s/it]
                                                           
{'loss': 1.3516, 'grad_norm': 5.867823123931885, 'learning_rate': 1.6683990441172073e-05, 'epoch': 0.29}

 29%|██▉       | 10970/37962 [25:25:14<62:00:20,  8.27s/it]
 29%|██▉       | 10971/37962 [25:25:22<61:18:35,  8.18s/it]
 29%|██▉       | 10972/37962 [25:25:30<61:43:41,  8.23s/it]
 29%|██▉       | 10973/37962 [25:25:39<61:53:21,  8.26s/it]
 29%|██▉       | 10974/37962 [25:25:47<62:03:02,  8.28s/it]
 29%|██▉       | 10975/37962 [25:25:56<62:20:30,  8.32s/it]
 29%|██▉       | 10976/37962 [25:26:04<62:41:31,  8.36s/it]
 29%|██▉       | 10977/37962 [25:26:12<62:25:20,  8.33s/it]
 29%|██▉       | 10978/37962 [25:26:20<62:08:04,  8.29s/it]
 29%|██▉       | 10979/37962 [25:26:29<62:10:13,  8.29s/it]
 29%|██▉       | 10980/37962 [25:26:37<62:07:58,  8.29s/it]
                                                           
{'loss': 1.4213, 'grad_norm': 5.3153252601623535, 'learning_rate': 1.6677642177544803e-05, 'epoch': 0.29}

 29%|██▉       | 10980/37962 [25:26:37<62:07:58,  8.29s/it]
 29%|██▉       | 10981/37962 [25:26:45<62:02:37,  8.28s/it]
 29%|██▉       | 10982/37962 [25:26:53<61:35:28,  8.22s/it]
 29%|██▉       | 10983/37962 [25:27:02<62:16:28,  8.31s/it]
 29%|██▉       | 10984/37962 [25:27:10<61:03:30,  8.15s/it]
 29%|██▉       | 10985/37962 [25:27:18<61:29:21,  8.21s/it]
 29%|██▉       | 10986/37962 [25:27:26<61:39:02,  8.23s/it]
 29%|██▉       | 10987/37962 [25:27:35<61:49:24,  8.25s/it]
 29%|██▉       | 10988/37962 [25:27:43<61:53:59,  8.26s/it]
 29%|██▉       | 10989/37962 [25:27:51<61:40:55,  8.23s/it]
 29%|██▉       | 10990/37962 [25:27:59<61:37:24,  8.23s/it]
                                                           
{'loss': 1.5141, 'grad_norm': 13.790233612060547, 'learning_rate': 1.6671289053377476e-05, 'epoch': 0.29}

 29%|██▉       | 10990/37962 [25:27:59<61:37:24,  8.23s/it]
 29%|██▉       | 10991/37962 [25:28:07<61:29:48,  8.21s/it]
 29%|██▉       | 10992/37962 [25:28:16<61:33:34,  8.22s/it]
 29%|██▉       | 10993/37962 [25:28:24<61:36:07,  8.22s/it]
 29%|██▉       | 10994/37962 [25:28:32<62:19:34,  8.32s/it]
 29%|██▉       | 10995/37962 [25:28:40<61:23:06,  8.19s/it]
 29%|██▉       | 10996/37962 [25:28:48<60:30:45,  8.08s/it]
 29%|██▉       | 10997/37962 [25:28:56<60:03:37,  8.02s/it]
 29%|██▉       | 10998/37962 [25:29:04<60:28:53,  8.07s/it]
 29%|██▉       | 10999/37962 [25:29:12<60:44:05,  8.11s/it]
 29%|██▉       | 11000/37962 [25:29:21<60:57:08,  8.14s/it]
                                                           
{'loss': 1.5027, 'grad_norm': 4.852967262268066, 'learning_rate': 1.6664931073294413e-05, 'epoch': 0.29}

 29%|██▉       | 11000/37962 [25:29:21<60:57:08,  8.14s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 29%|██▉       | 11001/37962 [25:30:12<158:00:17, 21.10s/it]
 29%|██▉       | 11002/37962 [25:30:20<129:24:23, 17.28s/it]
 29%|██▉       | 11003/37962 [25:30:29<109:05:26, 14.57s/it]
 29%|██▉       | 11004/37962 [25:30:37<95:13:57, 12.72s/it] 
 29%|██▉       | 11005/37962 [25:30:45<84:53:27, 11.34s/it]
 29%|██▉       | 11006/37962 [25:30:53<78:07:07, 10.43s/it]
 29%|██▉       | 11007/37962 [25:31:02<73:51:12,  9.86s/it]
 29%|██▉       | 11008/37962 [25:31:10<70:47:24,  9.45s/it]
 29%|██▉       | 11009/37962 [25:31:19<67:54:39,  9.07s/it]
 29%|██▉       | 11010/37962 [25:31:27<66:12:22,  8.84s/it]
                                                           
{'loss': 1.4078, 'grad_norm': 14.57627010345459, 'learning_rate': 1.6658568241923488e-05, 'epoch': 0.29}

 29%|██▉       | 11010/37962 [25:31:27<66:12:22,  8.84s/it]
 29%|██▉       | 11011/37962 [25:31:35<64:54:38,  8.67s/it]
 29%|██▉       | 11012/37962 [25:31:43<64:06:45,  8.56s/it]
 29%|██▉       | 11013/37962 [25:31:52<63:21:58,  8.46s/it]
 29%|██▉       | 11014/37962 [25:32:00<63:14:44,  8.45s/it]
 29%|██▉       | 11015/37962 [25:32:09<63:11:00,  8.44s/it]
 29%|██▉       | 11016/37962 [25:32:17<62:39:57,  8.37s/it]
 29%|██▉       | 11017/37962 [25:32:25<61:28:42,  8.21s/it]
 29%|██▉       | 11018/37962 [25:32:33<62:25:08,  8.34s/it]
 29%|██▉       | 11019/37962 [25:32:42<62:19:37,  8.33s/it]
 29%|██▉       | 11020/37962 [25:32:50<62:24:08,  8.34s/it]
                                                           
{'loss': 1.4368, 'grad_norm': 12.767474174499512, 'learning_rate': 1.6652200563896094e-05, 'epoch': 0.29}

 29%|██▉       | 11020/37962 [25:32:50<62:24:08,  8.34s/it]
 29%|██▉       | 11021/37962 [25:32:58<62:07:41,  8.30s/it]
 29%|██▉       | 11022/37962 [25:33:06<61:09:40,  8.17s/it]
 29%|██▉       | 11023/37962 [25:33:14<61:23:14,  8.20s/it]
 29%|██▉       | 11024/37962 [25:33:22<61:23:47,  8.21s/it]
 29%|██▉       | 11025/37962 [25:33:31<61:41:04,  8.24s/it]
 29%|██▉       | 11026/37962 [25:33:39<61:51:05,  8.27s/it]
 29%|██▉       | 11027/37962 [25:33:48<62:14:45,  8.32s/it]
 29%|██▉       | 11028/37962 [25:33:56<61:51:30,  8.27s/it]
 29%|██▉       | 11029/37962 [25:34:04<61:49:15,  8.26s/it]
 29%|██▉       | 11030/37962 [25:34:12<61:34:15,  8.23s/it]
                                                           
{'loss': 1.3787, 'grad_norm': 10.942769050598145, 'learning_rate': 1.6645828043847147e-05, 'epoch': 0.29}

 29%|██▉       | 11030/37962 [25:34:12<61:34:15,  8.23s/it]
 29%|██▉       | 11031/37962 [25:34:20<61:41:59,  8.25s/it]
 29%|██▉       | 11032/37962 [25:34:29<61:24:47,  8.21s/it]
 29%|██▉       | 11033/37962 [25:34:37<61:27:10,  8.22s/it]
 29%|██▉       | 11034/37962 [25:34:45<61:29:48,  8.22s/it]
 29%|██▉       | 11035/37962 [25:34:53<61:40:40,  8.25s/it]
 29%|██▉       | 11036/37962 [25:35:02<61:57:44,  8.28s/it]
 29%|██▉       | 11037/37962 [25:35:10<62:20:07,  8.33s/it]
 29%|██▉       | 11038/37962 [25:35:18<62:19:19,  8.33s/it]
 29%|██▉       | 11039/37962 [25:35:27<62:09:51,  8.31s/it]
 29%|██▉       | 11040/37962 [25:35:35<62:39:33,  8.38s/it]
                                                           
{'loss': 1.5008, 'grad_norm': 7.967550754547119, 'learning_rate': 1.6639450686415103e-05, 'epoch': 0.29}

 29%|██▉       | 11040/37962 [25:35:35<62:39:33,  8.38s/it]
 29%|██▉       | 11041/37962 [25:35:43<62:20:21,  8.34s/it]
 29%|██▉       | 11042/37962 [25:35:52<61:48:22,  8.27s/it]
 29%|██▉       | 11043/37962 [25:36:00<61:58:34,  8.29s/it]
 29%|██▉       | 11044/37962 [25:36:08<62:03:30,  8.30s/it]
 29%|██▉       | 11045/37962 [25:36:16<61:34:12,  8.23s/it]
 29%|██▉       | 11046/37962 [25:36:24<60:43:17,  8.12s/it]
 29%|██▉       | 11047/37962 [25:36:32<61:03:58,  8.17s/it]
 29%|██▉       | 11048/37962 [25:36:41<61:23:45,  8.21s/it]
 29%|██▉       | 11049/37962 [25:36:49<61:27:14,  8.22s/it]
 29%|██▉       | 11050/37962 [25:36:57<60:41:43,  8.12s/it]
                                                           
{'loss': 1.3686, 'grad_norm': 22.981107711791992, 'learning_rate': 1.663306849624192e-05, 'epoch': 0.29}

 29%|██▉       | 11050/37962 [25:36:57<60:41:43,  8.12s/it]
 29%|██▉       | 11051/37962 [25:37:05<60:47:14,  8.13s/it]
 29%|██▉       | 11052/37962 [25:37:13<61:10:31,  8.18s/it]
 29%|██▉       | 11053/37962 [25:37:22<61:39:58,  8.25s/it]
 29%|██▉       | 11054/37962 [25:37:30<61:53:29,  8.28s/it]
 29%|██▉       | 11055/37962 [25:37:38<61:54:39,  8.28s/it]
 29%|██▉       | 11056/37962 [25:37:47<61:58:08,  8.29s/it]
 29%|██▉       | 11057/37962 [25:37:55<62:06:36,  8.31s/it]
 29%|██▉       | 11058/37962 [25:38:04<62:30:47,  8.36s/it]
 29%|██▉       | 11059/37962 [25:38:12<62:50:45,  8.41s/it]
 29%|██▉       | 11060/37962 [25:38:20<62:32:33,  8.37s/it]
                                                           
{'loss': 1.4371, 'grad_norm': 6.2156982421875, 'learning_rate': 1.6626681477973092e-05, 'epoch': 0.29}

 29%|██▉       | 11060/37962 [25:38:20<62:32:33,  8.37s/it]
 29%|██▉       | 11061/37962 [25:38:29<62:37:39,  8.38s/it]
 29%|██▉       | 11062/37962 [25:38:37<62:32:38,  8.37s/it]
 29%|██▉       | 11063/37962 [25:38:45<62:12:47,  8.33s/it]
 29%|██▉       | 11064/37962 [25:38:53<61:47:38,  8.27s/it]
 29%|██▉       | 11065/37962 [25:39:02<61:42:17,  8.26s/it]
 29%|██▉       | 11066/37962 [25:39:10<62:01:55,  8.30s/it]
 29%|██▉       | 11067/37962 [25:39:19<62:17:49,  8.34s/it]
 29%|██▉       | 11068/37962 [25:39:27<62:10:34,  8.32s/it]
 29%|██▉       | 11069/37962 [25:39:35<62:24:18,  8.35s/it]
 29%|██▉       | 11070/37962 [25:39:44<62:50:31,  8.41s/it]
                                                           
{'loss': 1.4377, 'grad_norm': 3.8957228660583496, 'learning_rate': 1.662028963625761e-05, 'epoch': 0.29}

 29%|██▉       | 11070/37962 [25:39:44<62:50:31,  8.41s/it]
 29%|██▉       | 11071/37962 [25:39:52<62:08:34,  8.32s/it]
 29%|██▉       | 11072/37962 [25:40:00<62:15:27,  8.33s/it]
 29%|██▉       | 11073/37962 [25:40:09<62:21:19,  8.35s/it]
 29%|██▉       | 11074/37962 [25:40:17<62:17:28,  8.34s/it]
 29%|██▉       | 11075/37962 [25:40:25<62:06:35,  8.32s/it]
 29%|██▉       | 11076/37962 [25:40:33<61:14:25,  8.20s/it]
 29%|██▉       | 11077/37962 [25:40:41<61:25:48,  8.23s/it]
 29%|██▉       | 11078/37962 [25:40:50<61:37:13,  8.25s/it]
 29%|██▉       | 11079/37962 [25:40:58<61:40:15,  8.26s/it]
 29%|██▉       | 11080/37962 [25:41:06<61:23:33,  8.22s/it]
                                                           
{'loss': 1.4782, 'grad_norm': 12.599555015563965, 'learning_rate': 1.6613892975747998e-05, 'epoch': 0.29}

 29%|██▉       | 11080/37962 [25:41:06<61:23:33,  8.22s/it]
 29%|██▉       | 11081/37962 [25:41:14<61:13:03,  8.20s/it]
 29%|██▉       | 11082/37962 [25:41:23<61:56:01,  8.29s/it]
 29%|██▉       | 11083/37962 [25:41:32<62:59:36,  8.44s/it]
 29%|██▉       | 11084/37962 [25:41:40<62:40:03,  8.39s/it]
 29%|██▉       | 11085/37962 [25:41:48<61:46:00,  8.27s/it]
 29%|██▉       | 11086/37962 [25:41:56<61:14:48,  8.20s/it]
 29%|██▉       | 11087/37962 [25:42:04<61:37:16,  8.25s/it]
 29%|██▉       | 11088/37962 [25:42:13<61:45:10,  8.27s/it]
 29%|██▉       | 11089/37962 [25:42:21<61:37:49,  8.26s/it]
 29%|██▉       | 11090/37962 [25:42:29<61:35:04,  8.25s/it]
                                                           
{'loss': 1.3241, 'grad_norm': 12.195301055908203, 'learning_rate': 1.6607491501100262e-05, 'epoch': 0.29}

 29%|██▉       | 11090/37962 [25:42:29<61:35:04,  8.25s/it]
 29%|██▉       | 11091/37962 [25:42:38<62:12:25,  8.33s/it]
 29%|██▉       | 11092/37962 [25:42:46<62:20:11,  8.35s/it]
 29%|██▉       | 11093/37962 [25:42:54<62:02:54,  8.31s/it]
 29%|██▉       | 11094/37962 [25:43:02<61:49:25,  8.28s/it]
 29%|██▉       | 11095/37962 [25:43:11<62:04:09,  8.32s/it]
 29%|██▉       | 11096/37962 [25:43:19<61:39:28,  8.26s/it]
 29%|██▉       | 11097/37962 [25:43:27<61:43:34,  8.27s/it]
 29%|██▉       | 11098/37962 [25:43:36<62:19:37,  8.35s/it]
 29%|██▉       | 11099/37962 [25:43:44<62:31:25,  8.38s/it]
 29%|██▉       | 11100/37962 [25:43:52<62:09:30,  8.33s/it]
                                                           
{'loss': 1.3788, 'grad_norm': 6.27243709564209, 'learning_rate': 1.6601085216973932e-05, 'epoch': 0.29}

 29%|██▉       | 11100/37962 [25:43:52<62:09:30,  8.33s/it]
 29%|██▉       | 11101/37962 [25:44:01<61:52:48,  8.29s/it]
 29%|██▉       | 11102/37962 [25:44:09<62:02:18,  8.31s/it]
 29%|██▉       | 11103/37962 [25:44:17<62:10:07,  8.33s/it]
 29%|██▉       | 11104/37962 [25:44:26<62:05:29,  8.32s/it]
 29%|██▉       | 11105/37962 [25:44:34<61:42:16,  8.27s/it]
 29%|██▉       | 11106/37962 [25:44:42<61:09:43,  8.20s/it]
 29%|██▉       | 11107/37962 [25:44:50<61:28:41,  8.24s/it]
 29%|██▉       | 11108/37962 [25:44:59<61:41:38,  8.27s/it]
 29%|██▉       | 11109/37962 [25:45:07<61:40:17,  8.27s/it]
 29%|██▉       | 11110/37962 [25:45:15<61:26:10,  8.24s/it]
                                                           
{'loss': 1.4074, 'grad_norm': 6.370360374450684, 'learning_rate': 1.6594674128032026e-05, 'epoch': 0.29}

 29%|██▉       | 11110/37962 [25:45:15<61:26:10,  8.24s/it]
 29%|██▉       | 11111/37962 [25:45:24<62:10:46,  8.34s/it]
 29%|██▉       | 11112/37962 [25:45:32<62:41:29,  8.41s/it]
 29%|██▉       | 11113/37962 [25:45:40<61:19:02,  8.22s/it]
 29%|██▉       | 11114/37962 [25:45:48<61:51:26,  8.29s/it]
 29%|██▉       | 11115/37962 [25:45:56<60:56:42,  8.17s/it]
 29%|██▉       | 11116/37962 [25:46:05<61:23:18,  8.23s/it]
 29%|██▉       | 11117/37962 [25:46:13<61:28:30,  8.24s/it]
 29%|██▉       | 11118/37962 [25:46:21<61:56:09,  8.31s/it]
 29%|██▉       | 11119/37962 [25:46:30<62:03:24,  8.32s/it]
 29%|██▉       | 11120/37962 [25:46:38<62:24:51,  8.37s/it]
                                                           
{'loss': 1.4595, 'grad_norm': 8.430095672607422, 'learning_rate': 1.6588258238941066e-05, 'epoch': 0.29}

 29%|██▉       | 11120/37962 [25:46:38<62:24:51,  8.37s/it]
 29%|██▉       | 11121/37962 [25:46:46<61:36:28,  8.26s/it]
 29%|██▉       | 11122/37962 [25:46:54<61:31:31,  8.25s/it]
 29%|██▉       | 11123/37962 [25:47:03<61:18:22,  8.22s/it]
 29%|██▉       | 11124/37962 [25:47:11<61:36:33,  8.26s/it]
 29%|██▉       | 11125/37962 [25:47:19<61:23:05,  8.23s/it]
 29%|██▉       | 11126/37962 [25:47:27<61:29:07,  8.25s/it]
 29%|██▉       | 11127/37962 [25:47:36<61:46:17,  8.29s/it]
 29%|██▉       | 11128/37962 [25:47:44<62:02:17,  8.32s/it]
 29%|██▉       | 11129/37962 [25:47:52<61:54:36,  8.31s/it]
 29%|██▉       | 11130/37962 [25:48:01<62:02:28,  8.32s/it]
                                                           
{'loss': 1.3929, 'grad_norm': 7.276546478271484, 'learning_rate': 1.6581837554371072e-05, 'epoch': 0.29}

 29%|██▉       | 11130/37962 [25:48:01<62:02:28,  8.32s/it]
 29%|██▉       | 11131/37962 [25:48:09<61:36:59,  8.27s/it]
 29%|██▉       | 11132/37962 [25:48:17<61:44:38,  8.28s/it]
 29%|██▉       | 11133/37962 [25:48:26<62:00:24,  8.32s/it]
 29%|██▉       | 11134/37962 [25:48:34<60:54:02,  8.17s/it]
 29%|██▉       | 11135/37962 [25:48:42<61:02:13,  8.19s/it]
 29%|██▉       | 11136/37962 [25:48:50<61:06:31,  8.20s/it]
 29%|██▉       | 11137/37962 [25:48:58<61:43:13,  8.28s/it]
 29%|██▉       | 11138/37962 [25:49:07<61:35:35,  8.27s/it]
 29%|██▉       | 11139/37962 [25:49:15<61:23:58,  8.24s/it]
 29%|██▉       | 11140/37962 [25:49:23<61:29:35,  8.25s/it]
                                                           
{'loss': 1.4663, 'grad_norm': 5.935778617858887, 'learning_rate': 1.657541207899554e-05, 'epoch': 0.29}

 29%|██▉       | 11140/37962 [25:49:23<61:29:35,  8.25s/it]
 29%|██▉       | 11141/37962 [25:49:32<62:10:31,  8.35s/it]
 29%|██▉       | 11142/37962 [25:49:40<62:02:47,  8.33s/it]
 29%|██▉       | 11143/37962 [25:49:48<61:47:42,  8.29s/it]
 29%|██▉       | 11144/37962 [25:49:56<61:17:44,  8.23s/it]
 29%|██▉       | 11145/37962 [25:50:05<61:25:53,  8.25s/it]
 29%|██▉       | 11146/37962 [25:50:12<60:37:18,  8.14s/it]
 29%|██▉       | 11147/37962 [25:50:21<60:59:11,  8.19s/it]
 29%|██▉       | 11148/37962 [25:50:29<61:12:31,  8.22s/it]
 29%|██▉       | 11149/37962 [25:50:37<61:31:55,  8.26s/it]
 29%|██▉       | 11150/37962 [25:50:46<62:13:32,  8.35s/it]
                                                           
{'loss': 1.3986, 'grad_norm': 11.938579559326172, 'learning_rate': 1.656898181749147e-05, 'epoch': 0.29}

 29%|██▉       | 11150/37962 [25:50:46<62:13:32,  8.35s/it]
 29%|██▉       | 11151/37962 [25:50:54<61:54:02,  8.31s/it]
 29%|██▉       | 11152/37962 [25:51:03<62:12:11,  8.35s/it]
 29%|██▉       | 11153/37962 [25:51:11<62:25:00,  8.38s/it]
 29%|██▉       | 11154/37962 [25:51:19<62:13:50,  8.36s/it]
 29%|██▉       | 11155/37962 [25:51:28<61:50:22,  8.30s/it]
 29%|██▉       | 11156/37962 [25:51:36<61:42:33,  8.29s/it]
 29%|██▉       | 11157/37962 [25:51:44<62:07:32,  8.34s/it]
 29%|██▉       | 11158/37962 [25:51:52<61:37:49,  8.28s/it]
 29%|██▉       | 11159/37962 [25:52:01<61:19:47,  8.24s/it]
 29%|██▉       | 11160/37962 [25:52:09<61:14:57,  8.23s/it]
                                                           
{'loss': 1.391, 'grad_norm': 7.343424320220947, 'learning_rate': 1.6562546774539336e-05, 'epoch': 0.29}

 29%|██▉       | 11160/37962 [25:52:09<61:14:57,  8.23s/it]
 29%|██▉       | 11161/37962 [25:52:17<61:22:18,  8.24s/it]
 29%|██▉       | 11162/37962 [25:52:25<60:56:20,  8.19s/it]
 29%|██▉       | 11163/37962 [25:52:33<61:16:46,  8.23s/it]
 29%|██▉       | 11164/37962 [25:52:42<61:18:31,  8.24s/it]
 29%|██▉       | 11165/37962 [25:52:50<60:49:37,  8.17s/it]
 29%|██▉       | 11166/37962 [25:52:58<60:47:33,  8.17s/it]
 29%|██▉       | 11167/37962 [25:53:06<61:31:59,  8.27s/it]
 29%|██▉       | 11168/37962 [25:53:15<61:45:13,  8.30s/it]
 29%|██▉       | 11169/37962 [25:53:23<61:59:06,  8.33s/it]
 29%|██▉       | 11170/37962 [25:53:31<61:58:56,  8.33s/it]
                                                           
{'loss': 1.3937, 'grad_norm': 17.60485076904297, 'learning_rate': 1.655610695482309e-05, 'epoch': 0.29}

 29%|██▉       | 11170/37962 [25:53:31<61:58:56,  8.33s/it]
 29%|██▉       | 11171/37962 [25:53:39<60:59:27,  8.20s/it]
 29%|██▉       | 11172/37962 [25:53:48<61:01:56,  8.20s/it]
 29%|██▉       | 11173/37962 [25:53:56<61:19:33,  8.24s/it]
 29%|██▉       | 11174/37962 [25:54:05<62:07:45,  8.35s/it]
 29%|██▉       | 11175/37962 [25:54:13<62:20:08,  8.38s/it]
 29%|██▉       | 11176/37962 [25:54:21<62:02:24,  8.34s/it]
 29%|██▉       | 11177/37962 [25:54:29<61:07:33,  8.22s/it]
 29%|██▉       | 11178/37962 [25:54:37<61:18:46,  8.24s/it]
 29%|██▉       | 11179/37962 [25:54:46<61:52:56,  8.32s/it]
 29%|██▉       | 11180/37962 [25:54:54<62:00:35,  8.34s/it]
                                                           
{'loss': 1.4869, 'grad_norm': 14.37977123260498, 'learning_rate': 1.6549662363030167e-05, 'epoch': 0.29}

 29%|██▉       | 11180/37962 [25:54:54<62:00:35,  8.34s/it]
 29%|██▉       | 11181/37962 [25:55:03<61:52:37,  8.32s/it]
 29%|██▉       | 11182/37962 [25:55:11<61:20:56,  8.25s/it]
 29%|██▉       | 11183/37962 [25:55:19<61:29:36,  8.27s/it]
 29%|██▉       | 11184/37962 [25:55:27<62:03:02,  8.34s/it]
 29%|██▉       | 11185/37962 [25:55:36<61:53:17,  8.32s/it]
 29%|██▉       | 11186/37962 [25:55:44<61:51:21,  8.32s/it]
 29%|██▉       | 11187/37962 [25:55:52<61:25:21,  8.26s/it]
 29%|██▉       | 11188/37962 [25:56:00<60:55:12,  8.19s/it]
 29%|██▉       | 11189/37962 [25:56:09<61:18:05,  8.24s/it]
 29%|██▉       | 11190/37962 [25:56:17<61:38:43,  8.29s/it]
                                                           
{'loss': 1.4631, 'grad_norm': 12.51855754852295, 'learning_rate': 1.654321300385148e-05, 'epoch': 0.29}

 29%|██▉       | 11190/37962 [25:56:17<61:38:43,  8.29s/it]
 29%|██▉       | 11191/37962 [25:56:25<61:21:57,  8.25s/it]
 29%|██▉       | 11192/37962 [25:56:34<61:37:52,  8.29s/it]
 29%|██▉       | 11193/37962 [25:56:42<61:00:07,  8.20s/it]
 29%|██▉       | 11194/37962 [25:56:50<61:01:09,  8.21s/it]
 29%|██▉       | 11195/37962 [25:56:58<60:55:24,  8.19s/it]
 29%|██▉       | 11196/37962 [25:57:06<61:19:56,  8.25s/it]
 29%|██▉       | 11197/37962 [25:57:15<61:39:49,  8.29s/it]
 29%|██▉       | 11198/37962 [25:57:23<60:44:26,  8.17s/it]
 30%|██▉       | 11199/37962 [25:57:30<60:09:34,  8.09s/it]
 30%|██▉       | 11200/37962 [25:57:39<60:44:16,  8.17s/it]
                                                           
{'loss': 1.4011, 'grad_norm': 5.2395429611206055, 'learning_rate': 1.6536758881981397e-05, 'epoch': 0.3}

 30%|██▉       | 11200/37962 [25:57:39<60:44:16,  8.17s/it]
 30%|██▉       | 11201/37962 [25:57:47<60:54:07,  8.19s/it]
 30%|██▉       | 11202/37962 [25:57:55<61:19:33,  8.25s/it]
 30%|██▉       | 11203/37962 [25:58:04<61:09:44,  8.23s/it]
 30%|██▉       | 11204/37962 [25:58:12<61:37:05,  8.29s/it]
 30%|██▉       | 11205/37962 [25:58:20<61:35:26,  8.29s/it]
 30%|██▉       | 11206/37962 [25:58:29<61:44:53,  8.31s/it]
 30%|██▉       | 11207/37962 [25:58:37<61:54:47,  8.33s/it]
 30%|██▉       | 11208/37962 [25:58:45<61:54:23,  8.33s/it]
 30%|██▉       | 11209/37962 [25:58:54<61:51:16,  8.32s/it]
 30%|██▉       | 11210/37962 [25:59:02<62:07:39,  8.36s/it]
                                                           
{'loss': 1.444, 'grad_norm': 4.581459045410156, 'learning_rate': 1.653030000211777e-05, 'epoch': 0.3}

 30%|██▉       | 11210/37962 [25:59:02<62:07:39,  8.36s/it]
 30%|██▉       | 11211/37962 [25:59:10<61:05:05,  8.22s/it]
 30%|██▉       | 11212/37962 [25:59:18<61:23:27,  8.26s/it]
 30%|██▉       | 11213/37962 [25:59:26<60:52:43,  8.19s/it]
 30%|██▉       | 11214/37962 [25:59:35<61:27:16,  8.27s/it]
 30%|██▉       | 11215/37962 [25:59:43<61:32:48,  8.28s/it]
 30%|██▉       | 11216/37962 [25:59:51<61:26:33,  8.27s/it]
 30%|██▉       | 11217/37962 [26:00:00<61:16:43,  8.25s/it]
 30%|██▉       | 11218/37962 [26:00:08<61:26:28,  8.27s/it]
 30%|██▉       | 11219/37962 [26:00:16<60:37:18,  8.16s/it]
 30%|██▉       | 11220/37962 [26:00:24<60:41:19,  8.17s/it]
                                                           
{'loss': 1.4952, 'grad_norm': 7.901134490966797, 'learning_rate': 1.6523836368961895e-05, 'epoch': 0.3}

 30%|██▉       | 11220/37962 [26:00:24<60:41:19,  8.17s/it]
 30%|██▉       | 11221/37962 [26:00:33<61:39:42,  8.30s/it]
 30%|██▉       | 11222/37962 [26:00:41<61:33:38,  8.29s/it]
 30%|██▉       | 11223/37962 [26:00:49<61:25:02,  8.27s/it]
 30%|██▉       | 11224/37962 [26:00:57<60:44:48,  8.18s/it]
 30%|██▉       | 11225/37962 [26:01:05<60:59:43,  8.21s/it]
 30%|██▉       | 11226/37962 [26:01:14<61:04:39,  8.22s/it]
 30%|██▉       | 11227/37962 [26:01:22<61:25:25,  8.27s/it]
 30%|██▉       | 11228/37962 [26:01:30<61:29:40,  8.28s/it]
 30%|██▉       | 11229/37962 [26:01:39<61:40:21,  8.31s/it]
 30%|██▉       | 11230/37962 [26:01:47<62:16:45,  8.39s/it]
                                                           
{'loss': 1.4896, 'grad_norm': 33.4068489074707, 'learning_rate': 1.6517367987218552e-05, 'epoch': 0.3}

 30%|██▉       | 11230/37962 [26:01:47<62:16:45,  8.39s/it]
 30%|██▉       | 11231/37962 [26:01:55<61:40:06,  8.31s/it]
 30%|██▉       | 11232/37962 [26:02:04<61:34:29,  8.29s/it]
 30%|██▉       | 11233/37962 [26:02:12<61:37:06,  8.30s/it]
 30%|██▉       | 11234/37962 [26:02:20<61:25:04,  8.27s/it]
 30%|██▉       | 11235/37962 [26:02:28<61:17:46,  8.26s/it]
 30%|██▉       | 11236/37962 [26:02:37<61:20:13,  8.26s/it]
 30%|██▉       | 11237/37962 [26:02:45<61:14:35,  8.25s/it]
 30%|██▉       | 11238/37962 [26:02:53<61:51:51,  8.33s/it]
 30%|██▉       | 11239/37962 [26:03:02<62:50:39,  8.47s/it]
 30%|██▉       | 11240/37962 [26:03:10<62:17:56,  8.39s/it]
                                                           
{'loss': 1.5399, 'grad_norm': 14.283897399902344, 'learning_rate': 1.6510894861595953e-05, 'epoch': 0.3}

 30%|██▉       | 11240/37962 [26:03:10<62:17:56,  8.39s/it]
 30%|██▉       | 11241/37962 [26:03:19<62:07:17,  8.37s/it]
 30%|██▉       | 11242/37962 [26:03:27<62:10:12,  8.38s/it]
 30%|██▉       | 11243/37962 [26:03:35<61:59:54,  8.35s/it]
 30%|██▉       | 11244/37962 [26:03:44<61:41:44,  8.31s/it]
 30%|██▉       | 11245/37962 [26:03:52<61:21:32,  8.27s/it]
 30%|██▉       | 11246/37962 [26:04:00<60:57:10,  8.21s/it]
 30%|██▉       | 11247/37962 [26:04:08<61:23:39,  8.27s/it]
 30%|██▉       | 11248/37962 [26:04:17<61:16:45,  8.26s/it]
 30%|██▉       | 11249/37962 [26:04:24<60:24:36,  8.14s/it]
 30%|██▉       | 11250/37962 [26:04:33<60:39:21,  8.17s/it]
                                                           
{'loss': 1.3279, 'grad_norm': 19.251859664916992, 'learning_rate': 1.6504416996805775e-05, 'epoch': 0.3}

 30%|██▉       | 11250/37962 [26:04:33<60:39:21,  8.17s/it]
 30%|██▉       | 11251/37962 [26:04:41<61:26:16,  8.28s/it]
 30%|██▉       | 11252/37962 [26:04:49<61:17:27,  8.26s/it]
 30%|██▉       | 11253/37962 [26:04:58<61:07:00,  8.24s/it]
 30%|██▉       | 11254/37962 [26:05:06<61:47:22,  8.33s/it]
 30%|██▉       | 11255/37962 [26:05:14<61:35:50,  8.30s/it]
 30%|██▉       | 11256/37962 [26:05:23<62:05:53,  8.37s/it]
 30%|██▉       | 11257/37962 [26:05:31<61:48:13,  8.33s/it]
 30%|██▉       | 11258/37962 [26:05:39<61:23:30,  8.28s/it]
 30%|██▉       | 11259/37962 [26:05:48<61:53:20,  8.34s/it]
 30%|██▉       | 11260/37962 [26:05:56<61:51:32,  8.34s/it]
                                                           
{'loss': 1.4273, 'grad_norm': 16.487606048583984, 'learning_rate': 1.649793439756315e-05, 'epoch': 0.3}

 30%|██▉       | 11260/37962 [26:05:56<61:51:32,  8.34s/it]
 30%|██▉       | 11261/37962 [26:06:04<61:39:12,  8.31s/it]
 30%|██▉       | 11262/37962 [26:06:13<61:38:01,  8.31s/it]
 30%|██▉       | 11263/37962 [26:06:21<61:30:11,  8.29s/it]
 30%|██▉       | 11264/37962 [26:06:29<61:38:35,  8.31s/it]
 30%|██▉       | 11265/37962 [26:06:38<61:31:23,  8.30s/it]
 30%|██▉       | 11266/37962 [26:06:46<61:41:14,  8.32s/it]
 30%|██▉       | 11267/37962 [26:06:54<61:08:14,  8.24s/it]
 30%|██▉       | 11268/37962 [26:07:02<61:36:41,  8.31s/it]
 30%|██▉       | 11269/37962 [26:07:11<61:41:45,  8.32s/it]
 30%|██▉       | 11270/37962 [26:07:19<61:05:03,  8.24s/it]
                                                           
{'loss': 1.4234, 'grad_norm': 8.102100372314453, 'learning_rate': 1.6491447068586644e-05, 'epoch': 0.3}

 30%|██▉       | 11270/37962 [26:07:19<61:05:03,  8.24s/it]
 30%|██▉       | 11271/37962 [26:07:27<60:37:58,  8.18s/it]
 30%|██▉       | 11272/37962 [26:07:35<60:59:40,  8.23s/it]
 30%|██▉       | 11273/37962 [26:07:43<60:47:15,  8.20s/it]
 30%|██▉       | 11274/37962 [26:07:52<60:41:29,  8.19s/it]
 30%|██▉       | 11275/37962 [26:07:59<59:32:03,  8.03s/it]
 30%|██▉       | 11276/37962 [26:08:08<60:16:40,  8.13s/it]
 30%|██▉       | 11277/37962 [26:08:16<60:33:33,  8.17s/it]
 30%|██▉       | 11278/37962 [26:08:24<60:41:39,  8.19s/it]
 30%|██▉       | 11279/37962 [26:08:32<60:44:28,  8.20s/it]
 30%|██▉       | 11280/37962 [26:08:41<61:12:11,  8.26s/it]
                                                           
{'loss': 1.3522, 'grad_norm': 8.766030311584473, 'learning_rate': 1.6484955014598273e-05, 'epoch': 0.3}

 30%|██▉       | 11280/37962 [26:08:41<61:12:11,  8.26s/it]
 30%|██▉       | 11281/37962 [26:08:49<61:27:44,  8.29s/it]
 30%|██▉       | 11282/37962 [26:08:57<61:01:49,  8.23s/it]
 30%|██▉       | 11283/37962 [26:09:06<61:32:22,  8.30s/it]
 30%|██▉       | 11284/37962 [26:09:14<61:34:48,  8.31s/it]
 30%|██▉       | 11285/37962 [26:09:22<61:03:37,  8.24s/it]
 30%|██▉       | 11286/37962 [26:09:30<61:11:23,  8.26s/it]
 30%|██▉       | 11287/37962 [26:09:39<61:34:25,  8.31s/it]
 30%|██▉       | 11288/37962 [26:09:47<61:22:17,  8.28s/it]
 30%|██▉       | 11289/37962 [26:09:56<62:08:51,  8.39s/it]
 30%|██▉       | 11290/37962 [26:10:04<62:13:14,  8.40s/it]
                                                           
{'loss': 1.4133, 'grad_norm': 3.9384922981262207, 'learning_rate': 1.6478458240323495e-05, 'epoch': 0.3}

 30%|██▉       | 11290/37962 [26:10:04<62:13:14,  8.40s/it]
 30%|██▉       | 11291/37962 [26:10:12<61:29:10,  8.30s/it]
 30%|██▉       | 11292/37962 [26:10:20<61:31:07,  8.30s/it]
 30%|██▉       | 11293/37962 [26:10:29<61:15:55,  8.27s/it]
 30%|██▉       | 11294/37962 [26:10:37<60:40:17,  8.19s/it]
 30%|██▉       | 11295/37962 [26:10:45<61:02:29,  8.24s/it]
 30%|██▉       | 11296/37962 [26:10:53<60:38:03,  8.19s/it]
 30%|██▉       | 11297/37962 [26:11:01<60:59:50,  8.24s/it]
 30%|██▉       | 11298/37962 [26:11:10<61:15:16,  8.27s/it]
 30%|██▉       | 11299/37962 [26:11:18<61:40:17,  8.33s/it]
 30%|██▉       | 11300/37962 [26:11:26<61:16:33,  8.27s/it]
                                                           
{'loss': 1.2923, 'grad_norm': 9.32307243347168, 'learning_rate': 1.6471956750491192e-05, 'epoch': 0.3}

 30%|██▉       | 11300/37962 [26:11:26<61:16:33,  8.27s/it]
 30%|██▉       | 11301/37962 [26:11:35<61:43:50,  8.34s/it]
 30%|██▉       | 11302/37962 [26:11:43<61:47:09,  8.34s/it]
 30%|██▉       | 11303/37962 [26:11:51<61:04:03,  8.25s/it]
 30%|██▉       | 11304/37962 [26:11:59<60:25:22,  8.16s/it]
 30%|██▉       | 11305/37962 [26:12:07<60:51:18,  8.22s/it]
 30%|██▉       | 11306/37962 [26:12:16<61:04:10,  8.25s/it]
 30%|██▉       | 11307/37962 [26:12:24<61:13:54,  8.27s/it]
 30%|██▉       | 11308/37962 [26:12:33<61:31:35,  8.31s/it]
 30%|██▉       | 11309/37962 [26:12:41<61:24:17,  8.29s/it]
 30%|██▉       | 11310/37962 [26:12:49<61:38:44,  8.33s/it]
                                                           
{'loss': 1.527, 'grad_norm': 12.604999542236328, 'learning_rate': 1.6465450549833687e-05, 'epoch': 0.3}

 30%|██▉       | 11310/37962 [26:12:49<61:38:44,  8.33s/it]
 30%|██▉       | 11311/37962 [26:12:57<60:57:58,  8.24s/it]
 30%|██▉       | 11312/37962 [26:13:06<61:30:19,  8.31s/it]
 30%|██▉       | 11313/37962 [26:13:14<61:34:46,  8.32s/it]
 30%|██▉       | 11314/37962 [26:13:23<62:01:29,  8.38s/it]
 30%|██▉       | 11315/37962 [26:13:31<62:31:22,  8.45s/it]
 30%|██▉       | 11316/37962 [26:13:39<62:08:50,  8.40s/it]
 30%|██▉       | 11317/37962 [26:13:48<61:59:46,  8.38s/it]
 30%|██▉       | 11318/37962 [26:13:56<62:41:28,  8.47s/it]
 30%|██▉       | 11319/37962 [26:14:05<61:58:57,  8.38s/it]
 30%|██▉       | 11320/37962 [26:14:13<61:53:35,  8.36s/it]
                                                           
{'loss': 1.4863, 'grad_norm': 15.302648544311523, 'learning_rate': 1.645893964308674e-05, 'epoch': 0.3}

 30%|██▉       | 11320/37962 [26:14:13<61:53:35,  8.36s/it]
 30%|██▉       | 11321/37962 [26:14:21<61:37:01,  8.33s/it]
 30%|██▉       | 11322/37962 [26:14:29<61:35:15,  8.32s/it]
 30%|██▉       | 11323/37962 [26:14:38<61:37:47,  8.33s/it]
 30%|██▉       | 11324/37962 [26:14:46<61:40:08,  8.33s/it]
 30%|██▉       | 11325/37962 [26:14:54<61:31:32,  8.32s/it]
 30%|██▉       | 11326/37962 [26:15:03<62:00:15,  8.38s/it]
 30%|██▉       | 11327/37962 [26:15:11<61:58:05,  8.38s/it]
 30%|██▉       | 11328/37962 [26:15:20<61:38:09,  8.33s/it]
 30%|██▉       | 11329/37962 [26:15:28<62:00:25,  8.38s/it]
 30%|██▉       | 11330/37962 [26:15:36<62:00:21,  8.38s/it]
                                                           
{'loss': 1.3936, 'grad_norm': 24.4898681640625, 'learning_rate': 1.6452424034989512e-05, 'epoch': 0.3}

 30%|██▉       | 11330/37962 [26:15:36<62:00:21,  8.38s/it]
 30%|██▉       | 11331/37962 [26:15:45<61:59:23,  8.38s/it]
 30%|██▉       | 11332/37962 [26:15:53<61:29:50,  8.31s/it]
 30%|██▉       | 11333/37962 [26:16:01<61:43:39,  8.35s/it]
 30%|██▉       | 11334/37962 [26:16:10<61:42:04,  8.34s/it]
 30%|██▉       | 11335/37962 [26:16:19<62:40:14,  8.47s/it]
 30%|██▉       | 11336/37962 [26:16:27<62:19:42,  8.43s/it]
 30%|██▉       | 11337/37962 [26:16:35<62:00:02,  8.38s/it]
 30%|██▉       | 11338/37962 [26:16:43<61:36:43,  8.33s/it]
 30%|██▉       | 11339/37962 [26:16:52<61:52:42,  8.37s/it]
 30%|██▉       | 11340/37962 [26:17:00<62:03:53,  8.39s/it]
                                                           
{'loss': 1.3531, 'grad_norm': 7.22315788269043, 'learning_rate': 1.6445903730284617e-05, 'epoch': 0.3}

 30%|██▉       | 11340/37962 [26:17:00<62:03:53,  8.39s/it]
 30%|██▉       | 11341/37962 [26:17:09<61:54:55,  8.37s/it]
 30%|██▉       | 11342/37962 [26:17:17<61:46:42,  8.35s/it]
 30%|██▉       | 11343/37962 [26:17:25<61:20:40,  8.30s/it]
 30%|██▉       | 11344/37962 [26:17:33<60:47:17,  8.22s/it]
 30%|██▉       | 11345/37962 [26:17:41<61:03:38,  8.26s/it]
 30%|██▉       | 11346/37962 [26:17:50<60:54:55,  8.24s/it]
 30%|██▉       | 11347/37962 [26:17:58<60:35:01,  8.19s/it]
 30%|██▉       | 11348/37962 [26:18:06<61:07:47,  8.27s/it]
 30%|██▉       | 11349/37962 [26:18:15<61:19:58,  8.30s/it]
 30%|██▉       | 11350/37962 [26:18:23<61:08:54,  8.27s/it]
                                                           
{'loss': 1.4779, 'grad_norm': 15.466109275817871, 'learning_rate': 1.6439378733718062e-05, 'epoch': 0.3}

 30%|██▉       | 11350/37962 [26:18:23<61:08:54,  8.27s/it]
 30%|██▉       | 11351/37962 [26:18:31<61:26:12,  8.31s/it]
 30%|██▉       | 11352/37962 [26:18:40<61:41:22,  8.35s/it]
 30%|██▉       | 11353/37962 [26:18:48<61:56:43,  8.38s/it]
 30%|██▉       | 11354/37962 [26:18:56<60:19:01,  8.16s/it]
 30%|██▉       | 11355/37962 [26:19:04<60:27:23,  8.18s/it]
 30%|██▉       | 11356/37962 [26:19:12<60:45:32,  8.22s/it]
 30%|██▉       | 11357/37962 [26:19:20<60:41:15,  8.21s/it]
 30%|██▉       | 11358/37962 [26:19:29<60:44:06,  8.22s/it]
 30%|██▉       | 11359/37962 [26:19:37<60:50:50,  8.23s/it]
 30%|██▉       | 11360/37962 [26:19:45<61:05:57,  8.27s/it]
                                                           
{'loss': 1.4559, 'grad_norm': 2.7611029148101807, 'learning_rate': 1.6432849050039278e-05, 'epoch': 0.3}

 30%|██▉       | 11360/37962 [26:19:45<61:05:57,  8.27s/it]
 30%|██▉       | 11361/37962 [26:19:53<61:01:24,  8.26s/it]
 30%|██▉       | 11362/37962 [26:20:02<61:00:46,  8.26s/it]
 30%|██▉       | 11363/37962 [26:20:10<60:59:33,  8.25s/it]
 30%|██▉       | 11364/37962 [26:20:18<61:02:03,  8.26s/it]
 30%|██▉       | 11365/37962 [26:20:26<60:53:37,  8.24s/it]
 30%|██▉       | 11366/37962 [26:20:35<61:14:31,  8.29s/it]
 30%|██▉       | 11367/37962 [26:20:43<61:06:01,  8.27s/it]
 30%|██▉       | 11368/37962 [26:20:51<61:11:49,  8.28s/it]
 30%|██▉       | 11369/37962 [26:21:00<61:07:02,  8.27s/it]
 30%|██▉       | 11370/37962 [26:21:08<60:50:47,  8.24s/it]
                                                           
{'loss': 1.3732, 'grad_norm': 9.261701583862305, 'learning_rate': 1.6426314684001118e-05, 'epoch': 0.3}

 30%|██▉       | 11370/37962 [26:21:08<60:50:47,  8.24s/it]
 30%|██▉       | 11371/37962 [26:21:16<60:48:09,  8.23s/it]
 30%|██▉       | 11372/37962 [26:21:24<61:09:41,  8.28s/it]
 30%|██▉       | 11373/37962 [26:21:33<61:25:29,  8.32s/it]
 30%|██▉       | 11374/37962 [26:21:41<61:30:15,  8.33s/it]
 30%|██▉       | 11375/37962 [26:21:50<61:40:47,  8.35s/it]
 30%|██▉       | 11376/37962 [26:21:58<61:12:19,  8.29s/it]
 30%|██▉       | 11377/37962 [26:22:06<61:20:27,  8.31s/it]
 30%|██▉       | 11378/37962 [26:22:14<60:08:02,  8.14s/it]
 30%|██▉       | 11379/37962 [26:22:22<60:15:09,  8.16s/it]
 30%|██▉       | 11380/37962 [26:22:30<60:28:31,  8.19s/it]
                                                           
{'loss': 1.3382, 'grad_norm': 7.383176803588867, 'learning_rate': 1.6419775640359822e-05, 'epoch': 0.3}

 30%|██▉       | 11380/37962 [26:22:30<60:28:31,  8.19s/it]
 30%|██▉       | 11381/37962 [26:22:39<60:41:41,  8.22s/it]
 30%|██▉       | 11382/37962 [26:22:47<60:04:48,  8.14s/it]
 30%|██▉       | 11383/37962 [26:22:55<59:54:44,  8.11s/it]
 30%|██▉       | 11384/37962 [26:23:03<60:09:36,  8.15s/it]
 30%|██▉       | 11385/37962 [26:23:11<60:09:14,  8.15s/it]
 30%|██▉       | 11386/37962 [26:23:19<60:28:17,  8.19s/it]
 30%|██▉       | 11387/37962 [26:23:28<60:36:57,  8.21s/it]
 30%|██▉       | 11388/37962 [26:23:36<60:29:43,  8.20s/it]
 30%|███       | 11389/37962 [26:23:44<61:10:38,  8.29s/it]
 30%|███       | 11390/37962 [26:23:52<61:12:26,  8.29s/it]
                                                           
{'loss': 1.3494, 'grad_norm': 7.435217380523682, 'learning_rate': 1.6413231923875053e-05, 'epoch': 0.3}

 30%|███       | 11390/37962 [26:23:52<61:12:26,  8.29s/it]
 30%|███       | 11391/37962 [26:24:01<61:19:40,  8.31s/it]
 30%|███       | 11392/37962 [26:24:10<62:12:48,  8.43s/it]
 30%|███       | 11393/37962 [26:24:18<62:19:18,  8.44s/it]
 30%|███       | 11394/37962 [26:24:26<62:09:05,  8.42s/it]
 30%|███       | 11395/37962 [26:24:35<62:32:10,  8.47s/it]
 30%|███       | 11396/37962 [26:24:43<62:24:45,  8.46s/it]
 30%|███       | 11397/37962 [26:24:52<62:44:49,  8.50s/it]
 30%|███       | 11398/37962 [26:25:01<63:10:33,  8.56s/it]
 30%|███       | 11399/37962 [26:25:09<62:46:56,  8.51s/it]
 30%|███       | 11400/37962 [26:25:17<62:18:19,  8.44s/it]
                                                           
{'loss': 1.399, 'grad_norm': 8.045174598693848, 'learning_rate': 1.6406683539309872e-05, 'epoch': 0.3}

 30%|███       | 11400/37962 [26:25:17<62:18:19,  8.44s/it]
 30%|███       | 11401/37962 [26:25:26<62:14:08,  8.44s/it]
 30%|███       | 11402/37962 [26:25:34<62:05:02,  8.42s/it]
 30%|███       | 11403/37962 [26:25:42<61:41:41,  8.36s/it]
 30%|███       | 11404/37962 [26:25:51<61:42:14,  8.36s/it]
 30%|███       | 11405/37962 [26:25:59<62:04:01,  8.41s/it]
 30%|███       | 11406/37962 [26:26:08<61:59:16,  8.40s/it]
 30%|███       | 11407/37962 [26:26:16<60:52:52,  8.25s/it]
 30%|███       | 11408/37962 [26:26:24<61:07:13,  8.29s/it]
 30%|███       | 11409/37962 [26:26:32<60:54:55,  8.26s/it]
 30%|███       | 11410/37962 [26:26:40<60:55:21,  8.26s/it]
                                                           
{'loss': 1.3808, 'grad_norm': 6.5585150718688965, 'learning_rate': 1.6400130491430726e-05, 'epoch': 0.3}

 30%|███       | 11410/37962 [26:26:40<60:55:21,  8.26s/it]
 30%|███       | 11411/37962 [26:26:49<61:27:26,  8.33s/it]
 30%|███       | 11412/37962 [26:26:57<61:38:43,  8.36s/it]
 30%|███       | 11413/37962 [26:27:06<61:49:04,  8.38s/it]
 30%|███       | 11414/37962 [26:27:14<60:28:53,  8.20s/it]
 30%|███       | 11415/37962 [26:27:22<60:54:08,  8.26s/it]
 30%|███       | 11416/37962 [26:27:31<61:49:11,  8.38s/it]
 30%|███       | 11417/37962 [26:27:39<61:55:10,  8.40s/it]
 30%|███       | 11418/37962 [26:27:47<61:52:02,  8.39s/it]
 30%|███       | 11419/37962 [26:27:56<61:54:52,  8.40s/it]
 30%|███       | 11420/37962 [26:28:05<62:29:15,  8.48s/it]
                                                           
{'loss': 1.4771, 'grad_norm': 10.903836250305176, 'learning_rate': 1.6393572785007474e-05, 'epoch': 0.3}

 30%|███       | 11420/37962 [26:28:05<62:29:15,  8.48s/it]
 30%|███       | 11421/37962 [26:28:13<62:00:48,  8.41s/it]
 30%|███       | 11422/37962 [26:28:21<61:41:39,  8.37s/it]
 30%|███       | 11423/37962 [26:28:29<61:47:45,  8.38s/it]
 30%|███       | 11424/37962 [26:28:38<62:24:06,  8.47s/it]
 30%|███       | 11425/37962 [26:28:46<62:01:53,  8.42s/it]
 30%|███       | 11426/37962 [26:28:55<61:42:22,  8.37s/it]
 30%|███       | 11427/37962 [26:29:03<61:45:12,  8.38s/it]
 30%|███       | 11428/37962 [26:29:11<61:09:11,  8.30s/it]
 30%|███       | 11429/37962 [26:29:20<61:18:25,  8.32s/it]
 30%|███       | 11430/37962 [26:29:28<61:38:39,  8.36s/it]
                                                           
{'loss': 1.3633, 'grad_norm': 5.828615188598633, 'learning_rate': 1.6387010424813354e-05, 'epoch': 0.3}

 30%|███       | 11430/37962 [26:29:28<61:38:39,  8.36s/it]
 30%|███       | 11431/37962 [26:29:37<62:11:32,  8.44s/it]
 30%|███       | 11432/37962 [26:29:45<62:26:02,  8.47s/it]
 30%|███       | 11433/37962 [26:29:53<61:50:51,  8.39s/it]
 30%|███       | 11434/37962 [26:30:02<62:30:11,  8.48s/it]
 30%|███       | 11435/37962 [26:30:11<62:35:17,  8.49s/it]
 30%|███       | 11436/37962 [26:30:19<62:55:54,  8.54s/it]
 30%|███       | 11437/37962 [26:30:28<62:25:47,  8.47s/it]
 30%|███       | 11438/37962 [26:30:36<61:58:22,  8.41s/it]
 30%|███       | 11439/37962 [26:30:44<62:18:16,  8.46s/it]
 30%|███       | 11440/37962 [26:30:53<62:23:44,  8.47s/it]
                                                           
{'loss': 1.4437, 'grad_norm': 19.3684024810791, 'learning_rate': 1.638044341562499e-05, 'epoch': 0.3}

 30%|███       | 11440/37962 [26:30:53<62:23:44,  8.47s/it]
 30%|███       | 11441/37962 [26:31:01<62:26:47,  8.48s/it]
 30%|███       | 11442/37962 [26:31:10<62:19:20,  8.46s/it]
 30%|███       | 11443/37962 [26:31:18<62:40:47,  8.51s/it]
 30%|███       | 11444/37962 [26:31:27<62:50:50,  8.53s/it]
 30%|███       | 11445/37962 [26:31:36<62:44:45,  8.52s/it]
 30%|███       | 11446/37962 [26:31:44<63:00:44,  8.55s/it]
 30%|███       | 11447/37962 [26:31:53<63:14:01,  8.59s/it]
 30%|███       | 11448/37962 [26:32:01<63:13:21,  8.58s/it]
 30%|███       | 11449/37962 [26:32:10<63:11:23,  8.58s/it]
 30%|███       | 11450/37962 [26:32:18<63:02:35,  8.56s/it]
                                                           
{'loss': 1.3755, 'grad_norm': 13.786738395690918, 'learning_rate': 1.6373871762222398e-05, 'epoch': 0.3}

 30%|███       | 11450/37962 [26:32:18<63:02:35,  8.56s/it]
 30%|███       | 11451/37962 [26:32:27<62:42:55,  8.52s/it]
 30%|███       | 11452/37962 [26:32:36<63:39:44,  8.65s/it]
 30%|███       | 11453/37962 [26:32:44<63:13:47,  8.59s/it]
 30%|███       | 11454/37962 [26:32:53<63:28:02,  8.62s/it]
 30%|███       | 11455/37962 [26:33:02<63:48:21,  8.67s/it]
 30%|███       | 11456/37962 [26:33:10<63:13:50,  8.59s/it]
 30%|███       | 11457/37962 [26:33:18<61:51:08,  8.40s/it]
 30%|███       | 11458/37962 [26:33:27<62:32:46,  8.50s/it]
 30%|███       | 11459/37962 [26:33:35<62:24:38,  8.48s/it]
 30%|███       | 11460/37962 [26:33:44<62:40:49,  8.51s/it]
                                                           
{'loss': 1.4433, 'grad_norm': 6.257304668426514, 'learning_rate': 1.6367295469388967e-05, 'epoch': 0.3}

 30%|███       | 11460/37962 [26:33:44<62:40:49,  8.51s/it]
 30%|███       | 11461/37962 [26:33:52<62:55:57,  8.55s/it]
 30%|███       | 11462/37962 [26:34:01<63:33:48,  8.64s/it]
 30%|███       | 11463/37962 [26:34:10<62:36:02,  8.50s/it]
 30%|███       | 11464/37962 [26:34:18<62:27:19,  8.49s/it]
 30%|███       | 11465/37962 [26:34:26<62:15:10,  8.46s/it]
 30%|███       | 11466/37962 [26:34:35<63:08:35,  8.58s/it]
 30%|███       | 11467/37962 [26:34:44<64:07:42,  8.71s/it]
 30%|███       | 11468/37962 [26:34:53<63:18:21,  8.60s/it]
 30%|███       | 11469/37962 [26:35:02<64:02:46,  8.70s/it]
 30%|███       | 11470/37962 [26:35:10<64:34:55,  8.78s/it]
                                                           
{'loss': 1.5617, 'grad_norm': 4.551675796508789, 'learning_rate': 1.636071454191148e-05, 'epoch': 0.3}

 30%|███       | 11470/37962 [26:35:10<64:34:55,  8.78s/it]
 30%|███       | 11471/37962 [26:35:19<63:47:24,  8.67s/it]
 30%|███       | 11472/37962 [26:35:28<64:10:35,  8.72s/it]
 30%|███       | 11473/37962 [26:35:36<63:45:31,  8.67s/it]
 30%|███       | 11474/37962 [26:35:45<62:50:51,  8.54s/it]
 30%|███       | 11475/37962 [26:35:53<62:41:36,  8.52s/it]
 30%|███       | 11476/37962 [26:36:01<62:23:33,  8.48s/it]
 30%|███       | 11477/37962 [26:36:10<61:53:01,  8.41s/it]
 30%|███       | 11478/37962 [26:36:18<61:45:10,  8.39s/it]
 30%|███       | 11479/37962 [26:36:26<61:28:43,  8.36s/it]
 30%|███       | 11480/37962 [26:36:34<61:01:31,  8.30s/it]
                                                           
{'loss': 1.2626, 'grad_norm': 12.681303977966309, 'learning_rate': 1.635412898458006e-05, 'epoch': 0.3}

 30%|███       | 11480/37962 [26:36:34<61:01:31,  8.30s/it]
 30%|███       | 11481/37962 [26:36:43<61:40:20,  8.38s/it]
 30%|███       | 11482/37962 [26:36:51<61:37:05,  8.38s/it]
 30%|███       | 11483/37962 [26:36:59<60:59:18,  8.29s/it]
 30%|███       | 11484/37962 [26:37:08<61:10:04,  8.32s/it]
 30%|███       | 11485/37962 [26:37:16<61:24:40,  8.35s/it]
 30%|███       | 11486/37962 [26:37:25<62:05:15,  8.44s/it]
 30%|███       | 11487/37962 [26:37:33<62:20:39,  8.48s/it]
 30%|███       | 11488/37962 [26:37:42<62:04:17,  8.44s/it]
 30%|███       | 11489/37962 [26:37:50<62:33:19,  8.51s/it]
 30%|███       | 11490/37962 [26:37:59<62:25:53,  8.49s/it]
                                                           
{'loss': 1.4016, 'grad_norm': 2.4506781101226807, 'learning_rate': 1.634753880218823e-05, 'epoch': 0.3}

 30%|███       | 11490/37962 [26:37:59<62:25:53,  8.49s/it]
 30%|███       | 11491/37962 [26:38:07<62:23:23,  8.48s/it]
 30%|███       | 11492/37962 [26:38:16<62:02:36,  8.44s/it]
 30%|███       | 11493/37962 [26:38:24<62:26:05,  8.49s/it]
 30%|███       | 11494/37962 [26:38:33<62:11:16,  8.46s/it]
 30%|███       | 11495/37962 [26:38:41<61:19:30,  8.34s/it]
 30%|███       | 11496/37962 [26:38:50<62:06:17,  8.45s/it]
 30%|███       | 11497/37962 [26:38:58<62:27:05,  8.50s/it]
 30%|███       | 11498/37962 [26:39:07<62:27:06,  8.50s/it]
 30%|███       | 11499/37962 [26:39:15<62:41:17,  8.53s/it]
 30%|███       | 11500/37962 [26:39:24<62:08:44,  8.45s/it]
                                                           
{'loss': 1.4061, 'grad_norm': 7.696333885192871, 'learning_rate': 1.6340943999532873e-05, 'epoch': 0.3}

 30%|███       | 11500/37962 [26:39:24<62:08:44,  8.45s/it]
 30%|███       | 11501/37962 [26:39:32<62:30:20,  8.50s/it]
 30%|███       | 11502/37962 [26:39:41<62:15:27,  8.47s/it]
 30%|███       | 11503/37962 [26:39:49<61:54:26,  8.42s/it]
 30%|███       | 11504/37962 [26:39:57<62:19:13,  8.48s/it]
 30%|███       | 11505/37962 [26:40:06<62:39:27,  8.53s/it]
 30%|███       | 11506/37962 [26:40:15<62:34:22,  8.51s/it]
 30%|███       | 11507/37962 [26:40:23<62:17:36,  8.48s/it]
 30%|███       | 11508/37962 [26:40:31<61:33:46,  8.38s/it]
 30%|███       | 11509/37962 [26:40:39<61:32:56,  8.38s/it]
 30%|███       | 11510/37962 [26:40:48<61:50:06,  8.42s/it]
                                                           
{'loss': 1.4813, 'grad_norm': 16.586326599121094, 'learning_rate': 1.6334344581414224e-05, 'epoch': 0.3}

 30%|███       | 11510/37962 [26:40:48<61:50:06,  8.42s/it]
 30%|███       | 11511/37962 [26:40:56<60:30:52,  8.24s/it]
 30%|███       | 11512/37962 [26:41:04<60:52:26,  8.29s/it]
 30%|███       | 11513/37962 [26:41:13<61:03:22,  8.31s/it]
 30%|███       | 11514/37962 [26:41:21<61:27:25,  8.37s/it]
 30%|███       | 11515/37962 [26:41:29<61:10:16,  8.33s/it]
 30%|███       | 11516/37962 [26:41:38<61:04:18,  8.31s/it]
 30%|███       | 11517/37962 [26:41:46<60:50:31,  8.28s/it]
 30%|███       | 11518/37962 [26:41:54<60:56:23,  8.30s/it]
 30%|███       | 11519/37962 [26:42:02<60:45:19,  8.27s/it]
 30%|███       | 11520/37962 [26:42:11<60:54:42,  8.29s/it]
                                                           
{'loss': 1.4033, 'grad_norm': 16.241857528686523, 'learning_rate': 1.6327740552635895e-05, 'epoch': 0.3}

 30%|███       | 11520/37962 [26:42:11<60:54:42,  8.29s/it]
 30%|███       | 11521/37962 [26:42:19<61:30:18,  8.37s/it]
 30%|███       | 11522/37962 [26:42:28<61:36:34,  8.39s/it]
 30%|███       | 11523/37962 [26:42:36<61:32:38,  8.38s/it]
 30%|███       | 11524/37962 [26:42:45<61:56:18,  8.43s/it]
 30%|███       | 11525/37962 [26:42:53<62:21:49,  8.49s/it]
 30%|███       | 11526/37962 [26:43:02<62:14:52,  8.48s/it]
 30%|███       | 11527/37962 [26:43:10<61:58:07,  8.44s/it]
 30%|███       | 11528/37962 [26:43:18<61:35:48,  8.39s/it]
 30%|███       | 11529/37962 [26:43:27<61:23:28,  8.36s/it]
 30%|███       | 11530/37962 [26:43:35<61:46:02,  8.41s/it]
                                                           
{'loss': 1.3201, 'grad_norm': 8.393757820129395, 'learning_rate': 1.6321131918004835e-05, 'epoch': 0.3}

 30%|███       | 11530/37962 [26:43:35<61:46:02,  8.41s/it]
 30%|███       | 11531/37962 [26:43:43<61:26:58,  8.37s/it]
 30%|███       | 11532/37962 [26:43:52<61:27:55,  8.37s/it]
 30%|███       | 11533/37962 [26:44:00<61:14:16,  8.34s/it]
 30%|███       | 11534/37962 [26:44:08<61:18:29,  8.35s/it]
 30%|███       | 11535/37962 [26:44:16<60:40:26,  8.27s/it]
 30%|███       | 11536/37962 [26:44:24<59:58:03,  8.17s/it]
 30%|███       | 11537/37962 [26:44:33<59:49:46,  8.15s/it]
 30%|███       | 11538/37962 [26:44:41<60:05:30,  8.19s/it]
 30%|███       | 11539/37962 [26:44:49<60:30:29,  8.24s/it]
 30%|███       | 11540/37962 [26:44:57<60:25:32,  8.23s/it]
                                                           
{'loss': 1.3524, 'grad_norm': 4.901735305786133, 'learning_rate': 1.631451868233136e-05, 'epoch': 0.3}

 30%|███       | 11540/37962 [26:44:57<60:25:32,  8.23s/it]
 30%|███       | 11541/37962 [26:45:05<59:52:27,  8.16s/it]
 30%|███       | 11542/37962 [26:45:14<59:58:57,  8.17s/it]
 30%|███       | 11543/37962 [26:45:22<59:43:35,  8.14s/it]
 30%|███       | 11544/37962 [26:45:30<59:34:16,  8.12s/it]
 30%|███       | 11545/37962 [26:45:38<60:03:34,  8.18s/it]
 30%|███       | 11546/37962 [26:45:46<59:52:34,  8.16s/it]
 30%|███       | 11547/37962 [26:45:54<59:59:04,  8.18s/it]
 30%|███       | 11548/37962 [26:46:02<59:53:55,  8.16s/it]
 30%|███       | 11549/37962 [26:46:11<60:00:30,  8.18s/it]
 30%|███       | 11550/37962 [26:46:19<60:20:09,  8.22s/it]
                                                           
{'loss': 1.4204, 'grad_norm': 9.770957946777344, 'learning_rate': 1.6307900850429126e-05, 'epoch': 0.3}

 30%|███       | 11550/37962 [26:46:19<60:20:09,  8.22s/it]
 30%|███       | 11551/37962 [26:46:27<60:34:53,  8.26s/it]
 30%|███       | 11552/37962 [26:46:36<61:09:31,  8.34s/it]
 30%|███       | 11553/37962 [26:46:44<60:56:32,  8.31s/it]
 30%|███       | 11554/37962 [26:46:52<60:39:43,  8.27s/it]
 30%|███       | 11555/37962 [26:47:01<60:45:54,  8.28s/it]
 30%|███       | 11556/37962 [26:47:09<60:55:27,  8.31s/it]
 30%|███       | 11557/37962 [26:47:17<60:50:50,  8.30s/it]
 30%|███       | 11558/37962 [26:47:25<60:43:41,  8.28s/it]
 30%|███       | 11559/37962 [26:47:34<60:43:58,  8.28s/it]
 30%|███       | 11560/37962 [26:47:42<61:13:52,  8.35s/it]
                                                           
{'loss': 1.4936, 'grad_norm': 6.626745223999023, 'learning_rate': 1.630127842711514e-05, 'epoch': 0.3}

 30%|███       | 11560/37962 [26:47:42<61:13:52,  8.35s/it]
 30%|███       | 11561/37962 [26:47:50<60:56:52,  8.31s/it]
 30%|███       | 11562/37962 [26:47:59<60:44:01,  8.28s/it]
 30%|███       | 11563/37962 [26:48:07<60:49:54,  8.30s/it]
 30%|███       | 11564/37962 [26:48:15<60:53:42,  8.30s/it]
 30%|███       | 11565/37962 [26:48:24<60:43:10,  8.28s/it]
 30%|███       | 11566/37962 [26:48:32<60:04:36,  8.19s/it]
 30%|███       | 11567/37962 [26:48:40<60:10:13,  8.21s/it]
 30%|███       | 11568/37962 [26:48:48<59:59:34,  8.18s/it]
 30%|███       | 11569/37962 [26:48:56<59:58:50,  8.18s/it]
 30%|███       | 11570/37962 [26:49:04<59:49:13,  8.16s/it]
                                                           
{'loss': 1.4657, 'grad_norm': 9.699769973754883, 'learning_rate': 1.629465141720975e-05, 'epoch': 0.3}

 30%|███       | 11570/37962 [26:49:04<59:49:13,  8.16s/it]
 30%|███       | 11571/37962 [26:49:12<59:27:46,  8.11s/it]
 30%|███       | 11572/37962 [26:49:20<59:43:24,  8.15s/it]
 30%|███       | 11573/37962 [26:49:29<59:37:59,  8.14s/it]
 30%|███       | 11574/37962 [26:49:37<59:56:49,  8.18s/it]
 30%|███       | 11575/37962 [26:49:45<60:22:02,  8.24s/it]
 30%|███       | 11576/37962 [26:49:53<60:23:12,  8.24s/it]
 30%|███       | 11577/37962 [26:50:02<59:59:27,  8.19s/it]
 30%|███       | 11578/37962 [26:50:10<60:06:19,  8.20s/it]
 31%|███       | 11579/37962 [26:50:18<60:09:38,  8.21s/it]
 31%|███       | 11580/37962 [26:50:26<60:19:17,  8.23s/it]
                                                           
{'loss': 1.4673, 'grad_norm': 11.607088088989258, 'learning_rate': 1.628801982553664e-05, 'epoch': 0.31}

 31%|███       | 11580/37962 [26:50:26<60:19:17,  8.23s/it]
 31%|███       | 11581/37962 [26:50:34<60:03:37,  8.20s/it]
 31%|███       | 11582/37962 [26:50:43<60:23:11,  8.24s/it]
 31%|███       | 11583/37962 [26:50:51<59:49:10,  8.16s/it]
 31%|███       | 11584/37962 [26:50:59<60:06:08,  8.20s/it]
 31%|███       | 11585/37962 [26:51:07<60:21:03,  8.24s/it]
 31%|███       | 11586/37962 [26:51:16<60:24:19,  8.24s/it]
 31%|███       | 11587/37962 [26:51:24<59:56:46,  8.18s/it]
 31%|███       | 11588/37962 [26:51:32<60:17:25,  8.23s/it]
 31%|███       | 11589/37962 [26:51:40<59:24:16,  8.11s/it]
 31%|███       | 11590/37962 [26:51:48<59:49:36,  8.17s/it]
                                                           
{'loss': 1.4012, 'grad_norm': 22.222490310668945, 'learning_rate': 1.6281383656922833e-05, 'epoch': 0.31}

 31%|███       | 11590/37962 [26:51:48<59:49:36,  8.17s/it]
 31%|███       | 11591/37962 [26:51:56<60:05:01,  8.20s/it]
 31%|███       | 11592/37962 [26:52:05<60:06:30,  8.21s/it]
 31%|███       | 11593/37962 [26:52:13<60:01:30,  8.19s/it]
 31%|███       | 11594/37962 [26:52:21<60:03:03,  8.20s/it]
 31%|███       | 11595/37962 [26:52:29<59:59:44,  8.19s/it]
 31%|███       | 11596/37962 [26:52:38<60:34:03,  8.27s/it]
 31%|███       | 11597/37962 [26:52:46<59:48:32,  8.17s/it]
 31%|███       | 11598/37962 [26:52:53<59:02:40,  8.06s/it]
 31%|███       | 11599/37962 [26:53:01<58:53:51,  8.04s/it]
 31%|███       | 11600/37962 [26:53:10<59:33:36,  8.13s/it]
                                                           
{'loss': 1.3686, 'grad_norm': 3.7001383304595947, 'learning_rate': 1.6274742916198673e-05, 'epoch': 0.31}

 31%|███       | 11600/37962 [26:53:10<59:33:36,  8.13s/it]
 31%|███       | 11601/37962 [26:53:18<59:40:11,  8.15s/it]
 31%|███       | 11602/37962 [26:53:26<59:46:32,  8.16s/it]
 31%|███       | 11603/37962 [26:53:34<60:06:16,  8.21s/it]
 31%|███       | 11604/37962 [26:53:43<59:55:23,  8.18s/it]
 31%|███       | 11605/37962 [26:53:51<60:01:24,  8.20s/it]
 31%|███       | 11606/37962 [26:53:59<60:21:16,  8.24s/it]
 31%|███       | 11607/37962 [26:54:07<60:34:56,  8.28s/it]
 31%|███       | 11608/37962 [26:54:15<59:47:52,  8.17s/it]
 31%|███       | 11609/37962 [26:54:24<60:22:43,  8.25s/it]
 31%|███       | 11610/37962 [26:54:32<60:24:37,  8.25s/it]
                                                           
{'loss': 1.3965, 'grad_norm': 36.090576171875, 'learning_rate': 1.6268097608197855e-05, 'epoch': 0.31}

 31%|███       | 11610/37962 [26:54:32<60:24:37,  8.25s/it]
 31%|███       | 11611/37962 [26:54:40<60:10:59,  8.22s/it]
 31%|███       | 11612/37962 [26:54:48<60:12:15,  8.23s/it]
 31%|███       | 11613/37962 [26:54:57<60:29:57,  8.27s/it]
 31%|███       | 11614/37962 [26:55:05<60:37:23,  8.28s/it]
 31%|███       | 11615/37962 [26:55:14<60:59:29,  8.33s/it]
 31%|███       | 11616/37962 [26:55:22<60:34:54,  8.28s/it]
 31%|███       | 11617/37962 [26:55:30<60:28:02,  8.26s/it]
 31%|███       | 11618/37962 [26:55:38<60:50:06,  8.31s/it]
 31%|███       | 11619/37962 [26:55:46<59:54:53,  8.19s/it]
 31%|███       | 11620/37962 [26:55:55<60:40:22,  8.29s/it]
                                                           
{'loss': 1.5019, 'grad_norm': 5.05103063583374, 'learning_rate': 1.626144773775737e-05, 'epoch': 0.31}

 31%|███       | 11620/37962 [26:55:55<60:40:22,  8.29s/it]
 31%|███       | 11621/37962 [26:56:03<60:44:14,  8.30s/it]
 31%|███       | 11622/37962 [26:56:11<60:34:03,  8.28s/it]
 31%|███       | 11623/37962 [26:56:20<60:48:41,  8.31s/it]
 31%|███       | 11624/37962 [26:56:28<60:53:18,  8.32s/it]
 31%|███       | 11625/37962 [26:56:36<60:04:05,  8.21s/it]
 31%|███       | 11626/37962 [26:56:44<60:09:02,  8.22s/it]
 31%|███       | 11627/37962 [26:56:53<60:25:33,  8.26s/it]
 31%|███       | 11628/37962 [26:57:01<61:07:06,  8.36s/it]
 31%|███       | 11629/37962 [26:57:09<60:20:59,  8.25s/it]
 31%|███       | 11630/37962 [26:57:17<60:16:07,  8.24s/it]
                                                           
{'loss': 1.4254, 'grad_norm': 4.36926794052124, 'learning_rate': 1.6254793309717553e-05, 'epoch': 0.31}

 31%|███       | 11630/37962 [26:57:17<60:16:07,  8.24s/it]
 31%|███       | 11631/37962 [26:57:26<60:11:52,  8.23s/it]
 31%|███       | 11632/37962 [26:57:34<60:37:29,  8.29s/it]
 31%|███       | 11633/37962 [26:57:42<60:24:53,  8.26s/it]
 31%|███       | 11634/37962 [26:57:50<60:14:35,  8.24s/it]
 31%|███       | 11635/37962 [26:57:59<60:24:15,  8.26s/it]
 31%|███       | 11636/37962 [26:58:07<61:00:27,  8.34s/it]
 31%|███       | 11637/37962 [26:58:15<60:26:48,  8.27s/it]
 31%|███       | 11638/37962 [26:58:24<60:30:29,  8.27s/it]
 31%|███       | 11639/37962 [26:58:32<60:29:58,  8.27s/it]
 31%|███       | 11640/37962 [26:58:40<60:02:44,  8.21s/it]
                                                           
{'loss': 1.4926, 'grad_norm': 10.330196380615234, 'learning_rate': 1.624813432892204e-05, 'epoch': 0.31}

 31%|███       | 11640/37962 [26:58:40<60:02:44,  8.21s/it]
 31%|███       | 11641/37962 [26:58:49<60:40:04,  8.30s/it]
 31%|███       | 11642/37962 [26:58:57<60:17:27,  8.25s/it]
 31%|███       | 11643/37962 [26:59:05<60:48:18,  8.32s/it]
 31%|███       | 11644/37962 [26:59:14<61:05:31,  8.36s/it]
 31%|███       | 11645/37962 [26:59:21<59:55:44,  8.20s/it]
 31%|███       | 11646/37962 [26:59:30<60:14:57,  8.24s/it]
 31%|███       | 11647/37962 [26:59:38<60:35:46,  8.29s/it]
 31%|███       | 11648/37962 [26:59:46<60:29:25,  8.28s/it]
 31%|███       | 11649/37962 [26:59:55<60:39:00,  8.30s/it]
 31%|███       | 11650/37962 [27:00:02<59:14:03,  8.10s/it]
                                                           
{'loss': 1.3511, 'grad_norm': 10.573012351989746, 'learning_rate': 1.624147080021779e-05, 'epoch': 0.31}

 31%|███       | 11650/37962 [27:00:02<59:14:03,  8.10s/it]
 31%|███       | 11651/37962 [27:00:11<60:09:54,  8.23s/it]
 31%|███       | 11652/37962 [27:00:19<60:37:09,  8.29s/it]
 31%|███       | 11653/37962 [27:00:28<61:11:29,  8.37s/it]
 31%|███       | 11654/37962 [27:00:36<61:08:26,  8.37s/it]
 31%|███       | 11655/37962 [27:00:45<60:52:16,  8.33s/it]
 31%|███       | 11656/37962 [27:00:53<60:25:37,  8.27s/it]
 31%|███       | 11657/37962 [27:01:01<60:38:26,  8.30s/it]
 31%|███       | 11658/37962 [27:01:09<60:13:12,  8.24s/it]
 31%|███       | 11659/37962 [27:01:18<60:56:16,  8.34s/it]
 31%|███       | 11660/37962 [27:01:26<60:31:25,  8.28s/it]
                                                           
{'loss': 1.42, 'grad_norm': 14.491007804870605, 'learning_rate': 1.6234802728455075e-05, 'epoch': 0.31}

 31%|███       | 11660/37962 [27:01:26<60:31:25,  8.28s/it]
 31%|███       | 11661/37962 [27:01:34<60:09:55,  8.24s/it]
 31%|███       | 11662/37962 [27:01:42<60:12:41,  8.24s/it]
 31%|███       | 11663/37962 [27:01:50<59:48:48,  8.19s/it]
 31%|███       | 11664/37962 [27:01:59<60:14:08,  8.25s/it]
 31%|███       | 11665/37962 [27:02:07<59:46:07,  8.18s/it]
 31%|███       | 11666/37962 [27:02:15<60:04:12,  8.22s/it]
 31%|███       | 11667/37962 [27:02:23<59:58:56,  8.21s/it]
 31%|███       | 11668/37962 [27:02:32<60:25:42,  8.27s/it]
 31%|███       | 11669/37962 [27:02:40<60:23:04,  8.27s/it]
 31%|███       | 11670/37962 [27:02:48<60:03:01,  8.22s/it]
                                                           
{'loss': 1.4034, 'grad_norm': 10.059892654418945, 'learning_rate': 1.6228130118487458e-05, 'epoch': 0.31}

 31%|███       | 11670/37962 [27:02:48<60:03:01,  8.22s/it]
 31%|███       | 11671/37962 [27:02:56<60:21:02,  8.26s/it]
 31%|███       | 11672/37962 [27:03:05<60:48:06,  8.33s/it]
 31%|███       | 11673/37962 [27:03:13<60:39:52,  8.31s/it]
 31%|███       | 11674/37962 [27:03:22<60:53:50,  8.34s/it]
 31%|███       | 11675/37962 [27:03:30<60:43:34,  8.32s/it]
 31%|███       | 11676/37962 [27:03:38<61:03:00,  8.36s/it]
 31%|███       | 11677/37962 [27:03:47<61:15:56,  8.39s/it]
 31%|███       | 11678/37962 [27:03:55<61:04:05,  8.36s/it]
 31%|███       | 11679/37962 [27:04:03<61:05:28,  8.37s/it]
 31%|███       | 11680/37962 [27:04:12<60:46:20,  8.32s/it]
                                                           
{'loss': 1.359, 'grad_norm': 17.897216796875, 'learning_rate': 1.6221452975171826e-05, 'epoch': 0.31}

 31%|███       | 11680/37962 [27:04:12<60:46:20,  8.32s/it]
 31%|███       | 11681/37962 [27:04:20<60:41:43,  8.31s/it]
 31%|███       | 11682/37962 [27:04:28<59:40:29,  8.17s/it]
 31%|███       | 11683/37962 [27:04:36<60:25:32,  8.28s/it]
 31%|███       | 11684/37962 [27:04:45<60:21:44,  8.27s/it]
 31%|███       | 11685/37962 [27:04:53<60:24:32,  8.28s/it]
 31%|███       | 11686/37962 [27:05:01<60:23:42,  8.27s/it]
 31%|███       | 11687/37962 [27:05:09<60:28:59,  8.29s/it]
 31%|███       | 11688/37962 [27:05:18<60:14:43,  8.25s/it]
 31%|███       | 11689/37962 [27:05:26<60:18:38,  8.26s/it]
 31%|███       | 11690/37962 [27:05:34<60:45:51,  8.33s/it]
                                                           
{'loss': 1.3946, 'grad_norm': 15.310016632080078, 'learning_rate': 1.6214771303368357e-05, 'epoch': 0.31}

 31%|███       | 11690/37962 [27:05:34<60:45:51,  8.33s/it]
 31%|███       | 11691/37962 [27:05:43<60:40:44,  8.32s/it]
 31%|███       | 11692/37962 [27:05:51<60:30:05,  8.29s/it]
 31%|███       | 11693/37962 [27:05:59<60:52:25,  8.34s/it]
 31%|███       | 11694/37962 [27:06:07<60:29:39,  8.29s/it]
 31%|███       | 11695/37962 [27:06:16<60:31:16,  8.29s/it]
 31%|███       | 11696/37962 [27:06:24<60:19:46,  8.27s/it]
 31%|███       | 11697/37962 [27:06:32<60:34:28,  8.30s/it]
 31%|███       | 11698/37962 [27:06:41<60:55:55,  8.35s/it]
 31%|███       | 11699/37962 [27:06:49<60:13:31,  8.26s/it]
 31%|███       | 11700/37962 [27:06:57<60:20:29,  8.27s/it]
                                                           
{'loss': 1.4407, 'grad_norm': 10.818742752075195, 'learning_rate': 1.6208085107940516e-05, 'epoch': 0.31}

 31%|███       | 11700/37962 [27:06:57<60:20:29,  8.27s/it]
 31%|███       | 11701/37962 [27:07:06<60:45:36,  8.33s/it]
 31%|███       | 11702/37962 [27:07:14<60:11:19,  8.25s/it]
 31%|███       | 11703/37962 [27:07:22<60:30:11,  8.29s/it]
 31%|███       | 11704/37962 [27:07:31<60:51:52,  8.34s/it]
 31%|███       | 11705/37962 [27:07:39<60:57:59,  8.36s/it]
 31%|███       | 11706/37962 [27:07:47<60:58:23,  8.36s/it]
 31%|███       | 11707/37962 [27:07:55<60:30:20,  8.30s/it]
 31%|███       | 11708/37962 [27:08:04<60:14:14,  8.26s/it]
 31%|███       | 11709/37962 [27:08:12<60:23:41,  8.28s/it]
 31%|███       | 11710/37962 [27:08:20<60:34:48,  8.31s/it]
                                                           
{'loss': 1.2963, 'grad_norm': 6.188978672027588, 'learning_rate': 1.620139439375507e-05, 'epoch': 0.31}

 31%|███       | 11710/37962 [27:08:20<60:34:48,  8.31s/it]
 31%|███       | 11711/37962 [27:08:28<60:12:36,  8.26s/it]
 31%|███       | 11712/37962 [27:08:37<60:28:26,  8.29s/it]
 31%|███       | 11713/37962 [27:08:45<60:27:33,  8.29s/it]
 31%|███       | 11714/37962 [27:08:54<60:36:56,  8.31s/it]
 31%|███       | 11715/37962 [27:09:02<60:28:48,  8.30s/it]
 31%|███       | 11716/37962 [27:09:10<60:20:22,  8.28s/it]
 31%|███       | 11717/37962 [27:09:18<60:09:44,  8.25s/it]
 31%|███       | 11718/37962 [27:09:26<60:08:46,  8.25s/it]
 31%|███       | 11719/37962 [27:09:35<60:25:27,  8.29s/it]
 31%|███       | 11720/37962 [27:09:43<60:33:38,  8.31s/it]
                                                           
{'loss': 1.3941, 'grad_norm': 7.344741344451904, 'learning_rate': 1.6194699165682077e-05, 'epoch': 0.31}

 31%|███       | 11720/37962 [27:09:43<60:33:38,  8.31s/it]
 31%|███       | 11721/37962 [27:09:51<60:06:38,  8.25s/it]
 31%|███       | 11722/37962 [27:10:00<60:25:13,  8.29s/it]
 31%|███       | 11723/37962 [27:10:08<61:00:25,  8.37s/it]
 31%|███       | 11724/37962 [27:10:16<60:25:05,  8.29s/it]
 31%|███       | 11725/37962 [27:10:25<60:53:49,  8.36s/it]
 31%|███       | 11726/37962 [27:10:33<60:55:32,  8.36s/it]
 31%|███       | 11727/37962 [27:10:41<60:12:44,  8.26s/it]
 31%|███       | 11728/37962 [27:10:50<60:32:55,  8.31s/it]
 31%|███       | 11729/37962 [27:10:58<60:26:35,  8.29s/it]
 31%|███       | 11730/37962 [27:11:06<60:41:35,  8.33s/it]
                                                           
{'loss': 1.4899, 'grad_norm': 10.95267105102539, 'learning_rate': 1.6187999428594882e-05, 'epoch': 0.31}

 31%|███       | 11730/37962 [27:11:06<60:41:35,  8.33s/it]
 31%|███       | 11731/37962 [27:11:14<59:41:39,  8.19s/it]
 31%|███       | 11732/37962 [27:11:22<59:48:36,  8.21s/it]
 31%|███       | 11733/37962 [27:11:31<60:41:23,  8.33s/it]
 31%|███       | 11734/37962 [27:11:39<60:22:32,  8.29s/it]
 31%|███       | 11735/37962 [27:11:47<60:12:24,  8.26s/it]
 31%|███       | 11736/37962 [27:11:56<60:03:50,  8.24s/it]
 31%|███       | 11737/37962 [27:12:04<59:56:02,  8.23s/it]
 31%|███       | 11738/37962 [27:12:12<60:04:31,  8.25s/it]
 31%|███       | 11739/37962 [27:12:20<59:57:14,  8.23s/it]
 31%|███       | 11740/37962 [27:12:29<60:03:14,  8.24s/it]
                                                           
{'loss': 1.3278, 'grad_norm': 14.109932899475098, 'learning_rate': 1.6181295187370095e-05, 'epoch': 0.31}

 31%|███       | 11740/37962 [27:12:29<60:03:14,  8.24s/it]
 31%|███       | 11741/37962 [27:12:37<60:19:56,  8.28s/it]
 31%|███       | 11742/37962 [27:12:45<59:15:01,  8.14s/it]
 31%|███       | 11743/37962 [27:12:53<59:42:32,  8.20s/it]
 31%|███       | 11744/37962 [27:13:01<60:04:26,  8.25s/it]
 31%|███       | 11745/37962 [27:13:09<58:51:38,  8.08s/it]
 31%|███       | 11746/37962 [27:13:17<59:20:59,  8.15s/it]
 31%|███       | 11747/37962 [27:13:25<59:00:23,  8.10s/it]
 31%|███       | 11748/37962 [27:13:34<59:33:31,  8.18s/it]
 31%|███       | 11749/37962 [27:13:42<59:57:35,  8.23s/it]
 31%|███       | 11750/37962 [27:13:51<60:29:42,  8.31s/it]
                                                           
{'loss': 1.4507, 'grad_norm': 7.154019832611084, 'learning_rate': 1.617458644688763e-05, 'epoch': 0.31}

 31%|███       | 11750/37962 [27:13:51<60:29:42,  8.31s/it]
 31%|███       | 11751/37962 [27:13:59<60:34:05,  8.32s/it]
 31%|███       | 11752/37962 [27:14:07<60:13:51,  8.27s/it]
 31%|███       | 11753/37962 [27:14:15<60:07:40,  8.26s/it]
 31%|███       | 11754/37962 [27:14:24<59:50:22,  8.22s/it]
 31%|███       | 11755/37962 [27:14:32<60:33:33,  8.32s/it]
 31%|███       | 11756/37962 [27:14:40<60:21:59,  8.29s/it]
 31%|███       | 11757/37962 [27:14:49<60:12:39,  8.27s/it]
 31%|███       | 11758/37962 [27:14:57<60:25:22,  8.30s/it]
 31%|███       | 11759/37962 [27:15:05<60:28:22,  8.31s/it]
 31%|███       | 11760/37962 [27:15:13<60:10:33,  8.27s/it]
                                                           
{'loss': 1.4337, 'grad_norm': 11.458073616027832, 'learning_rate': 1.616787321203066e-05, 'epoch': 0.31}

 31%|███       | 11760/37962 [27:15:13<60:10:33,  8.27s/it]
 31%|███       | 11761/37962 [27:15:22<60:29:54,  8.31s/it]
 31%|███       | 11762/37962 [27:15:30<59:33:57,  8.18s/it]
 31%|███       | 11763/37962 [27:15:38<59:55:29,  8.23s/it]
 31%|███       | 11764/37962 [27:15:46<59:54:41,  8.23s/it]
 31%|███       | 11765/37962 [27:15:55<60:48:13,  8.36s/it]
 31%|███       | 11766/37962 [27:16:03<60:44:58,  8.35s/it]
 31%|███       | 11767/37962 [27:16:11<60:04:53,  8.26s/it]
 31%|███       | 11768/37962 [27:16:20<60:13:34,  8.28s/it]
 31%|███       | 11769/37962 [27:16:28<59:38:15,  8.20s/it]
 31%|███       | 11770/37962 [27:16:36<59:24:54,  8.17s/it]
                                                           
{'loss': 1.3787, 'grad_norm': 4.665543079376221, 'learning_rate': 1.6161155487685624e-05, 'epoch': 0.31}

 31%|███       | 11770/37962 [27:16:36<59:24:54,  8.17s/it]
 31%|███       | 11771/37962 [27:16:44<59:57:33,  8.24s/it]
 31%|███       | 11772/37962 [27:16:52<60:01:10,  8.25s/it]
 31%|███       | 11773/37962 [27:17:01<59:48:32,  8.22s/it]
 31%|███       | 11774/37962 [27:17:09<60:05:17,  8.26s/it]
 31%|███       | 11775/37962 [27:17:17<59:54:10,  8.24s/it]
 31%|███       | 11776/37962 [27:17:25<59:22:07,  8.16s/it]
 31%|███       | 11777/37962 [27:17:33<59:28:29,  8.18s/it]
 31%|███       | 11778/37962 [27:17:41<59:03:26,  8.12s/it]
 31%|███       | 11779/37962 [27:17:50<59:24:37,  8.17s/it]
 31%|███       | 11780/37962 [27:17:58<59:35:33,  8.19s/it]
                                                           
{'loss': 1.3792, 'grad_norm': 16.608346939086914, 'learning_rate': 1.615443327874225e-05, 'epoch': 0.31}

 31%|███       | 11780/37962 [27:17:58<59:35:33,  8.19s/it]
 31%|███       | 11781/37962 [27:18:06<60:12:52,  8.28s/it]
 31%|███       | 11782/37962 [27:18:15<60:19:36,  8.30s/it]
 31%|███       | 11783/37962 [27:18:23<60:18:27,  8.29s/it]
 31%|███       | 11784/37962 [27:18:31<59:22:52,  8.17s/it]
 31%|███       | 11785/37962 [27:18:39<59:12:31,  8.14s/it]
 31%|███       | 11786/37962 [27:18:47<58:18:04,  8.02s/it]
 31%|███       | 11787/37962 [27:18:55<58:49:21,  8.09s/it]
 31%|███       | 11788/37962 [27:19:03<59:18:25,  8.16s/it]
 31%|███       | 11789/37962 [27:19:11<59:30:06,  8.18s/it]
 31%|███       | 11790/37962 [27:19:20<59:33:00,  8.19s/it]
                                                           
{'loss': 1.2397, 'grad_norm': 5.418328762054443, 'learning_rate': 1.6147706590093506e-05, 'epoch': 0.31}

 31%|███       | 11790/37962 [27:19:20<59:33:00,  8.19s/it]
 31%|███       | 11791/37962 [27:19:28<59:34:40,  8.20s/it]
 31%|███       | 11792/37962 [27:19:36<59:52:36,  8.24s/it]
 31%|███       | 11793/37962 [27:19:44<59:23:02,  8.17s/it]
 31%|███       | 11794/37962 [27:19:52<59:34:31,  8.20s/it]
 31%|███       | 11795/37962 [27:20:01<60:16:40,  8.29s/it]
 31%|███       | 11796/37962 [27:20:09<59:13:56,  8.15s/it]
 31%|███       | 11797/37962 [27:20:17<59:31:49,  8.19s/it]
 31%|███       | 11798/37962 [27:20:25<59:49:31,  8.23s/it]
 31%|███       | 11799/37962 [27:20:34<60:12:32,  8.28s/it]
 31%|███       | 11800/37962 [27:20:42<60:08:44,  8.28s/it]
                                                           
{'loss': 1.3329, 'grad_norm': 12.209650993347168, 'learning_rate': 1.6140975426635637e-05, 'epoch': 0.31}

 31%|███       | 11800/37962 [27:20:42<60:08:44,  8.28s/it]
 31%|███       | 11801/37962 [27:20:50<60:13:57,  8.29s/it]
 31%|███       | 11802/37962 [27:20:59<60:18:48,  8.30s/it]
 31%|███       | 11803/37962 [27:21:07<60:27:40,  8.32s/it]
 31%|███       | 11804/37962 [27:21:15<60:23:30,  8.31s/it]
 31%|███       | 11805/37962 [27:21:24<60:21:22,  8.31s/it]
 31%|███       | 11806/37962 [27:21:32<60:40:04,  8.35s/it]
 31%|███       | 11807/37962 [27:21:40<59:11:36,  8.15s/it]
 31%|███       | 11808/37962 [27:21:48<59:21:52,  8.17s/it]
 31%|███       | 11809/37962 [27:21:56<60:01:43,  8.26s/it]
 31%|███       | 11810/37962 [27:22:05<60:19:52,  8.31s/it]
                                                           
{'loss': 1.454, 'grad_norm': 45.66841506958008, 'learning_rate': 1.613423979326814e-05, 'epoch': 0.31}

 31%|███       | 11810/37962 [27:22:05<60:19:52,  8.31s/it]
 31%|███       | 11811/37962 [27:22:13<59:14:10,  8.15s/it]
 31%|███       | 11812/37962 [27:22:21<59:21:10,  8.17s/it]
 31%|███       | 11813/37962 [27:22:29<59:53:42,  8.25s/it]
 31%|███       | 11814/37962 [27:22:38<60:01:57,  8.27s/it]
 31%|███       | 11815/37962 [27:22:46<59:58:31,  8.26s/it]
 31%|███       | 11816/37962 [27:22:54<59:25:36,  8.18s/it]
 31%|███       | 11817/37962 [27:23:02<60:07:40,  8.28s/it]
 31%|███       | 11818/37962 [27:23:11<60:10:21,  8.29s/it]
 31%|███       | 11819/37962 [27:23:19<60:12:36,  8.29s/it]
 31%|███       | 11820/37962 [27:23:27<59:58:58,  8.26s/it]
                                                           
{'loss': 1.2572, 'grad_norm': 5.743036270141602, 'learning_rate': 1.612749969489377e-05, 'epoch': 0.31}

 31%|███       | 11820/37962 [27:23:27<59:58:58,  8.26s/it]
 31%|███       | 11821/37962 [27:23:35<59:19:18,  8.17s/it]
 31%|███       | 11822/37962 [27:23:43<59:23:05,  8.18s/it]
 31%|███       | 11823/37962 [27:23:52<59:31:08,  8.20s/it]
 31%|███       | 11824/37962 [27:24:00<59:11:43,  8.15s/it]
 31%|███       | 11825/37962 [27:24:08<59:40:29,  8.22s/it]
 31%|███       | 11826/37962 [27:24:16<59:09:55,  8.15s/it]
 31%|███       | 11827/37962 [27:24:24<59:18:46,  8.17s/it]
 31%|███       | 11828/37962 [27:24:33<59:59:54,  8.26s/it]
 31%|███       | 11829/37962 [27:24:41<59:56:45,  8.26s/it]
 31%|███       | 11830/37962 [27:24:49<60:18:51,  8.31s/it]
                                                           
{'loss': 1.4759, 'grad_norm': 4.474371433258057, 'learning_rate': 1.612075513641852e-05, 'epoch': 0.31}

 31%|███       | 11830/37962 [27:24:49<60:18:51,  8.31s/it]
 31%|███       | 11831/37962 [27:24:58<60:09:27,  8.29s/it]
 31%|███       | 11832/37962 [27:25:06<60:07:02,  8.28s/it]
 31%|███       | 11833/37962 [27:25:14<60:12:51,  8.30s/it]
 31%|███       | 11834/37962 [27:25:22<59:47:07,  8.24s/it]
 31%|███       | 11835/37962 [27:25:31<60:18:21,  8.31s/it]
 31%|███       | 11836/37962 [27:25:39<60:07:59,  8.29s/it]
 31%|███       | 11837/37962 [27:25:47<59:57:03,  8.26s/it]
 31%|███       | 11838/37962 [27:25:55<59:53:55,  8.25s/it]
 31%|███       | 11839/37962 [27:26:04<60:05:57,  8.28s/it]
 31%|███       | 11840/37962 [27:26:12<59:53:18,  8.25s/it]
                                                           
{'loss': 1.3493, 'grad_norm': 11.682698249816895, 'learning_rate': 1.611400612275164e-05, 'epoch': 0.31}

 31%|███       | 11840/37962 [27:26:12<59:53:18,  8.25s/it]
 31%|███       | 11841/37962 [27:26:20<59:26:58,  8.19s/it]
 31%|███       | 11842/37962 [27:26:28<59:04:56,  8.14s/it]
 31%|███       | 11843/37962 [27:26:36<59:30:27,  8.20s/it]
 31%|███       | 11844/37962 [27:26:45<59:26:00,  8.19s/it]
 31%|███       | 11845/37962 [27:26:53<59:37:53,  8.22s/it]
 31%|███       | 11846/37962 [27:27:01<59:52:26,  8.25s/it]
 31%|███       | 11847/37962 [27:27:10<60:08:57,  8.29s/it]
 31%|███       | 11848/37962 [27:27:18<60:01:46,  8.28s/it]
 31%|███       | 11849/37962 [27:27:26<60:00:57,  8.27s/it]
 31%|███       | 11850/37962 [27:27:34<60:14:18,  8.30s/it]
                                                           
{'loss': 1.4582, 'grad_norm': 5.382815361022949, 'learning_rate': 1.610725265880562e-05, 'epoch': 0.31}

 31%|███       | 11850/37962 [27:27:34<60:14:18,  8.30s/it]
 31%|███       | 11851/37962 [27:27:43<60:34:29,  8.35s/it]
 31%|███       | 11852/37962 [27:27:51<60:34:25,  8.35s/it]
 31%|███       | 11853/37962 [27:27:59<60:15:52,  8.31s/it]
 31%|███       | 11854/37962 [27:28:08<60:00:56,  8.28s/it]
 31%|███       | 11855/37962 [27:28:16<60:10:28,  8.30s/it]
 31%|███       | 11856/37962 [27:28:24<60:05:25,  8.29s/it]
 31%|███       | 11857/37962 [27:28:32<59:16:04,  8.17s/it]
 31%|███       | 11858/37962 [27:28:41<60:01:45,  8.28s/it]
 31%|███       | 11859/37962 [27:28:49<59:55:22,  8.26s/it]
 31%|███       | 11860/37962 [27:28:57<59:54:46,  8.26s/it]
                                                           
{'loss': 1.3452, 'grad_norm': 20.161046981811523, 'learning_rate': 1.6100494749496195e-05, 'epoch': 0.31}

 31%|███       | 11860/37962 [27:28:57<59:54:46,  8.26s/it]
 31%|███       | 11861/37962 [27:29:06<60:12:12,  8.30s/it]
 31%|███       | 11862/37962 [27:29:14<59:39:06,  8.23s/it]
 31%|███       | 11863/37962 [27:29:22<59:45:35,  8.24s/it]
 31%|███▏      | 11864/37962 [27:29:30<60:10:00,  8.30s/it]
 31%|███▏      | 11865/37962 [27:29:39<60:22:48,  8.33s/it]
 31%|███▏      | 11866/37962 [27:29:47<60:10:36,  8.30s/it]
 31%|███▏      | 11867/37962 [27:29:55<59:26:20,  8.20s/it]
 31%|███▏      | 11868/37962 [27:30:03<59:53:39,  8.26s/it]
 31%|███▏      | 11869/37962 [27:30:12<59:47:42,  8.25s/it]
 31%|███▏      | 11870/37962 [27:30:20<59:22:19,  8.19s/it]
                                                           
{'loss': 1.303, 'grad_norm': 15.565869331359863, 'learning_rate': 1.6093732399742324e-05, 'epoch': 0.31}

 31%|███▏      | 11870/37962 [27:30:20<59:22:19,  8.19s/it]
 31%|███▏      | 11871/37962 [27:30:28<59:23:28,  8.19s/it]
 31%|███▏      | 11872/37962 [27:30:36<59:37:11,  8.23s/it]
 31%|███▏      | 11873/37962 [27:30:51<74:43:50, 10.31s/it]
 31%|███▏      | 11874/37962 [27:31:00<70:29:46,  9.73s/it]
 31%|███▏      | 11875/37962 [27:31:08<67:34:44,  9.33s/it]
 31%|███▏      | 11876/37962 [27:31:17<65:39:17,  9.06s/it]
 31%|███▏      | 11877/37962 [27:31:25<64:09:57,  8.86s/it]
 31%|███▏      | 11878/37962 [27:31:33<63:13:08,  8.73s/it]
 31%|███▏      | 11879/37962 [27:31:42<62:02:20,  8.56s/it]
 31%|███▏      | 11880/37962 [27:31:50<62:05:01,  8.57s/it]
                                                           
{'loss': 1.4329, 'grad_norm': 14.290550231933594, 'learning_rate': 1.6086965614466206e-05, 'epoch': 0.31}

 31%|███▏      | 11880/37962 [27:31:50<62:05:01,  8.57s/it]
 31%|███▏      | 11881/37962 [27:31:59<61:51:59,  8.54s/it]
 31%|███▏      | 11882/37962 [27:32:07<61:13:27,  8.45s/it]
 31%|███▏      | 11883/37962 [27:32:15<61:02:58,  8.43s/it]
 31%|███▏      | 11884/37962 [27:32:24<61:18:03,  8.46s/it]
 31%|███▏      | 11885/37962 [27:32:32<61:11:19,  8.45s/it]
 31%|███▏      | 11886/37962 [27:32:40<60:48:46,  8.40s/it]
 31%|███▏      | 11887/37962 [27:32:49<60:22:00,  8.33s/it]
 31%|███▏      | 11888/37962 [27:32:57<60:02:11,  8.29s/it]
 31%|███▏      | 11889/37962 [27:33:05<59:46:32,  8.25s/it]
 31%|███▏      | 11890/37962 [27:33:13<60:07:39,  8.30s/it]
                                                           
{'loss': 1.3956, 'grad_norm': 10.574429512023926, 'learning_rate': 1.6080194398593267e-05, 'epoch': 0.31}

 31%|███▏      | 11890/37962 [27:33:13<60:07:39,  8.30s/it]
 31%|███▏      | 11891/37962 [27:33:22<60:18:24,  8.33s/it]
 31%|███▏      | 11892/37962 [27:33:30<60:35:17,  8.37s/it]
 31%|███▏      | 11893/37962 [27:33:39<60:30:03,  8.35s/it]
 31%|███▏      | 11894/37962 [27:33:47<59:50:40,  8.26s/it]
 31%|███▏      | 11895/37962 [27:33:55<59:58:12,  8.28s/it]
 31%|███▏      | 11896/37962 [27:34:03<60:11:51,  8.31s/it]
 31%|███▏      | 11897/37962 [27:34:12<60:31:52,  8.36s/it]
 31%|███▏      | 11898/37962 [27:34:20<60:22:48,  8.34s/it]
 31%|███▏      | 11899/37962 [27:34:28<59:47:23,  8.26s/it]
 31%|███▏      | 11900/37962 [27:34:37<60:02:20,  8.29s/it]
                                                           
{'loss': 1.3009, 'grad_norm': 11.35422134399414, 'learning_rate': 1.6073418757052156e-05, 'epoch': 0.31}

 31%|███▏      | 11900/37962 [27:34:37<60:02:20,  8.29s/it]
 31%|███▏      | 11901/37962 [27:34:45<59:49:22,  8.26s/it]
 31%|███▏      | 11902/37962 [27:34:53<59:44:36,  8.25s/it]
 31%|███▏      | 11903/37962 [27:35:01<59:07:43,  8.17s/it]
 31%|███▏      | 11904/37962 [27:35:09<58:52:24,  8.13s/it]
 31%|███▏      | 11905/37962 [27:35:17<59:11:19,  8.18s/it]
 31%|███▏      | 11906/37962 [27:35:26<59:24:52,  8.21s/it]
 31%|███▏      | 11907/37962 [27:35:34<59:35:37,  8.23s/it]
 31%|███▏      | 11908/37962 [27:35:42<59:30:45,  8.22s/it]
 31%|███▏      | 11909/37962 [27:35:50<59:38:18,  8.24s/it]
 31%|███▏      | 11910/37962 [27:35:58<58:31:44,  8.09s/it]
                                                           
{'loss': 1.3234, 'grad_norm': 7.912886619567871, 'learning_rate': 1.6066638694774747e-05, 'epoch': 0.31}

 31%|███▏      | 11910/37962 [27:35:58<58:31:44,  8.09s/it]
 31%|███▏      | 11911/37962 [27:36:06<59:04:38,  8.16s/it]
 31%|███▏      | 11912/37962 [27:36:15<59:06:57,  8.17s/it]
 31%|███▏      | 11913/37962 [27:36:23<59:26:58,  8.22s/it]
 31%|███▏      | 11914/37962 [27:36:31<60:15:00,  8.33s/it]
 31%|███▏      | 11915/37962 [27:36:39<59:07:30,  8.17s/it]
 31%|███▏      | 11916/37962 [27:36:47<58:45:54,  8.12s/it]
 31%|███▏      | 11917/37962 [27:36:56<58:59:57,  8.16s/it]
 31%|███▏      | 11918/37962 [27:37:04<59:11:10,  8.18s/it]
 31%|███▏      | 11919/37962 [27:37:12<59:18:09,  8.20s/it]
 31%|███▏      | 11920/37962 [27:37:20<58:41:07,  8.11s/it]
                                                           
{'loss': 1.4081, 'grad_norm': 6.515002250671387, 'learning_rate': 1.605985421669613e-05, 'epoch': 0.31}

 31%|███▏      | 11920/37962 [27:37:20<58:41:07,  8.11s/it]
 31%|███▏      | 11921/37962 [27:37:28<59:02:26,  8.16s/it]
 31%|███▏      | 11922/37962 [27:37:37<59:22:04,  8.21s/it]
 31%|███▏      | 11923/37962 [27:37:45<59:22:25,  8.21s/it]
 31%|███▏      | 11924/37962 [27:37:53<59:48:15,  8.27s/it]
 31%|███▏      | 11925/37962 [27:38:01<59:48:13,  8.27s/it]
 31%|███▏      | 11926/37962 [27:38:10<60:23:35,  8.35s/it]
 31%|███▏      | 11927/37962 [27:38:18<60:15:12,  8.33s/it]
 31%|███▏      | 11928/37962 [27:38:26<59:14:05,  8.19s/it]
 31%|███▏      | 11929/37962 [27:38:34<59:37:43,  8.25s/it]
 31%|███▏      | 11930/37962 [27:38:43<59:49:01,  8.27s/it]
                                                           
{'loss': 1.38, 'grad_norm': 15.646952629089355, 'learning_rate': 1.605306532775461e-05, 'epoch': 0.31}

 31%|███▏      | 11930/37962 [27:38:43<59:49:01,  8.27s/it]
 31%|███▏      | 11931/37962 [27:38:51<60:11:58,  8.33s/it]
 31%|███▏      | 11932/37962 [27:38:59<59:51:44,  8.28s/it]
 31%|███▏      | 11933/37962 [27:39:07<59:16:46,  8.20s/it]
 31%|███▏      | 11934/37962 [27:39:16<59:23:58,  8.22s/it]
 31%|███▏      | 11935/37962 [27:39:24<59:23:29,  8.21s/it]
 31%|███▏      | 11936/37962 [27:39:32<59:26:52,  8.22s/it]
 31%|███▏      | 11937/37962 [27:39:40<58:58:48,  8.16s/it]
 31%|███▏      | 11938/37962 [27:39:48<58:04:10,  8.03s/it]
 31%|███▏      | 11939/37962 [27:39:56<58:31:54,  8.10s/it]
 31%|███▏      | 11940/37962 [27:40:05<59:20:46,  8.21s/it]
                                                           
{'loss': 1.3643, 'grad_norm': 15.696507453918457, 'learning_rate': 1.6046272032891698e-05, 'epoch': 0.31}

 31%|███▏      | 11940/37962 [27:40:05<59:20:46,  8.21s/it]
 31%|███▏      | 11941/37962 [27:40:13<59:16:34,  8.20s/it]
 31%|███▏      | 11942/37962 [27:40:21<59:11:03,  8.19s/it]
 31%|███▏      | 11943/37962 [27:40:29<59:40:54,  8.26s/it]
 31%|███▏      | 11944/37962 [27:40:38<59:31:10,  8.24s/it]
 31%|███▏      | 11945/37962 [27:40:46<59:31:28,  8.24s/it]
 31%|███▏      | 11946/37962 [27:40:54<58:56:06,  8.16s/it]
 31%|███▏      | 11947/37962 [27:41:02<59:15:37,  8.20s/it]
 31%|███▏      | 11948/37962 [27:41:10<59:28:59,  8.23s/it]
 31%|███▏      | 11949/37962 [27:41:19<59:17:47,  8.21s/it]
 31%|███▏      | 11950/37962 [27:41:27<59:08:06,  8.18s/it]
                                                           
{'loss': 1.4689, 'grad_norm': 13.89185619354248, 'learning_rate': 1.6039474337052116e-05, 'epoch': 0.31}

 31%|███▏      | 11950/37962 [27:41:27<59:08:06,  8.18s/it]
 31%|███▏      | 11951/37962 [27:41:35<59:54:46,  8.29s/it]
 31%|███▏      | 11952/37962 [27:41:43<59:34:17,  8.25s/it]
 31%|███▏      | 11953/37962 [27:41:52<59:51:05,  8.28s/it]
 31%|███▏      | 11954/37962 [27:42:00<60:01:44,  8.31s/it]
 31%|███▏      | 11955/37962 [27:42:09<60:22:11,  8.36s/it]
 31%|███▏      | 11956/37962 [27:42:17<60:36:34,  8.39s/it]
 31%|███▏      | 11957/37962 [27:42:25<60:16:27,  8.34s/it]
 31%|███▏      | 11958/37962 [27:42:33<59:59:36,  8.31s/it]
 32%|███▏      | 11959/37962 [27:42:42<59:57:23,  8.30s/it]
 32%|███▏      | 11960/37962 [27:42:50<59:26:26,  8.23s/it]
                                                           
{'loss': 1.2771, 'grad_norm': 6.697390079498291, 'learning_rate': 1.6032672245183793e-05, 'epoch': 0.32}

 32%|███▏      | 11960/37962 [27:42:50<59:26:26,  8.23s/it]
 32%|███▏      | 11961/37962 [27:42:58<59:26:08,  8.23s/it]
 32%|███▏      | 11962/37962 [27:43:06<59:50:21,  8.29s/it]
 32%|███▏      | 11963/37962 [27:43:15<59:59:00,  8.31s/it]
 32%|███▏      | 11964/37962 [27:43:23<58:42:39,  8.13s/it]
 32%|███▏      | 11965/37962 [27:43:31<58:48:14,  8.14s/it]
 32%|███▏      | 11966/37962 [27:43:39<59:04:27,  8.18s/it]
 32%|███▏      | 11967/37962 [27:43:47<59:01:01,  8.17s/it]
 32%|███▏      | 11968/37962 [27:43:55<59:12:47,  8.20s/it]
 32%|███▏      | 11969/37962 [27:44:04<59:02:52,  8.18s/it]
 32%|███▏      | 11970/37962 [27:44:12<59:26:04,  8.23s/it]
                                                           
{'loss': 1.4956, 'grad_norm': 7.575247287750244, 'learning_rate': 1.6025865762237854e-05, 'epoch': 0.32}

 32%|███▏      | 11970/37962 [27:44:12<59:26:04,  8.23s/it]
 32%|███▏      | 11971/37962 [27:44:20<59:15:06,  8.21s/it]
 32%|███▏      | 11972/37962 [27:44:28<59:06:41,  8.19s/it]
 32%|███▏      | 11973/37962 [27:44:37<59:38:09,  8.26s/it]
 32%|███▏      | 11974/37962 [27:44:45<59:39:07,  8.26s/it]
 32%|███▏      | 11975/37962 [27:44:53<59:58:36,  8.31s/it]
 32%|███▏      | 11976/37962 [27:45:01<59:30:13,  8.24s/it]
 32%|███▏      | 11977/37962 [27:45:10<59:42:04,  8.27s/it]
 32%|███▏      | 11978/37962 [27:45:18<59:55:10,  8.30s/it]
 32%|███▏      | 11979/37962 [27:45:26<59:46:33,  8.28s/it]
 32%|███▏      | 11980/37962 [27:45:34<59:19:36,  8.22s/it]
                                                           
{'loss': 1.4203, 'grad_norm': 15.326065063476562, 'learning_rate': 1.6019054893168614e-05, 'epoch': 0.32}

 32%|███▏      | 11980/37962 [27:45:34<59:19:36,  8.22s/it]
 32%|███▏      | 11981/37962 [27:45:43<59:23:59,  8.23s/it]
 32%|███▏      | 11982/37962 [27:45:51<59:17:43,  8.22s/it]
 32%|███▏      | 11983/37962 [27:45:59<59:19:54,  8.22s/it]
 32%|███▏      | 11984/37962 [27:46:07<59:46:36,  8.28s/it]
 32%|███▏      | 11985/37962 [27:46:16<60:25:39,  8.37s/it]
 32%|███▏      | 11986/37962 [27:46:24<60:22:48,  8.37s/it]
 32%|███▏      | 11987/37962 [27:46:32<59:37:48,  8.26s/it]
 32%|███▏      | 11988/37962 [27:46:41<59:39:01,  8.27s/it]
 32%|███▏      | 11989/37962 [27:46:49<60:19:50,  8.36s/it]
 32%|███▏      | 11990/37962 [27:46:58<60:50:29,  8.43s/it]
                                                           
{'loss': 1.4028, 'grad_norm': 16.28388214111328, 'learning_rate': 1.601223964293359e-05, 'epoch': 0.32}

 32%|███▏      | 11990/37962 [27:46:58<60:50:29,  8.43s/it]
 32%|███▏      | 11991/37962 [27:47:06<60:17:27,  8.36s/it]
 32%|███▏      | 11992/37962 [27:47:15<60:30:48,  8.39s/it]
 32%|███▏      | 11993/37962 [27:47:23<60:12:20,  8.35s/it]
 32%|███▏      | 11994/37962 [27:47:31<60:19:30,  8.36s/it]
 32%|███▏      | 11995/37962 [27:47:39<59:59:52,  8.32s/it]
 32%|███▏      | 11996/37962 [27:47:48<60:13:31,  8.35s/it]
 32%|███▏      | 11997/37962 [27:47:56<59:26:53,  8.24s/it]
 32%|███▏      | 11998/37962 [27:48:04<58:51:09,  8.16s/it]
 32%|███▏      | 11999/37962 [27:48:12<58:56:04,  8.17s/it]
 32%|███▏      | 12000/37962 [27:48:20<58:54:03,  8.17s/it]
                                                           
{'loss': 1.6952, 'grad_norm': 21.975643157958984, 'learning_rate': 1.6005420016493485e-05, 'epoch': 0.32}

 32%|███▏      | 12000/37962 [27:48:20<58:54:03,  8.17s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 32%|███▏      | 12001/37962 [27:49:11<151:00:20, 20.94s/it]
 32%|███▏      | 12002/37962 [27:49:19<123:50:58, 17.17s/it]
 32%|███▏      | 12003/37962 [27:49:27<103:59:15, 14.42s/it]
 32%|███▏      | 12004/37962 [27:49:35<90:23:32, 12.54s/it] 
 32%|███▏      | 12005/37962 [27:49:44<80:54:49, 11.22s/it]
 32%|███▏      | 12006/37962 [27:49:52<74:22:23, 10.32s/it]
 32%|███▏      | 12007/37962 [27:50:00<70:15:02,  9.74s/it]
 32%|███▏      | 12008/37962 [27:50:08<66:58:56,  9.29s/it]
 32%|███▏      | 12009/37962 [27:50:17<64:42:21,  8.98s/it]
 32%|███▏      | 12010/37962 [27:50:25<63:17:56,  8.78s/it]
                                                           
{'loss': 1.4561, 'grad_norm': 10.333736419677734, 'learning_rate': 1.599859601881219e-05, 'epoch': 0.32}

 32%|███▏      | 12010/37962 [27:50:25<63:17:56,  8.78s/it]
 32%|███▏      | 12011/37962 [27:50:34<62:46:25,  8.71s/it]
 32%|███▏      | 12012/37962 [27:50:42<61:32:30,  8.54s/it]
 32%|███▏      | 12013/37962 [27:50:50<60:51:59,  8.44s/it]
 32%|███▏      | 12014/37962 [27:50:58<60:21:37,  8.37s/it]
 32%|███▏      | 12015/37962 [27:51:06<59:42:33,  8.28s/it]
 32%|███▏      | 12016/37962 [27:51:14<59:44:32,  8.29s/it]
 32%|███▏      | 12017/37962 [27:51:23<59:37:07,  8.27s/it]
 32%|███▏      | 12018/37962 [27:51:31<59:52:36,  8.31s/it]
 32%|███▏      | 12019/37962 [27:51:39<59:47:55,  8.30s/it]
 32%|███▏      | 12020/37962 [27:51:48<59:56:09,  8.32s/it]
                                                           
{'loss': 1.4471, 'grad_norm': 5.278055191040039, 'learning_rate': 1.5991767654856772e-05, 'epoch': 0.32}

 32%|███▏      | 12020/37962 [27:51:48<59:56:09,  8.32s/it]
 32%|███▏      | 12021/37962 [27:51:56<59:43:33,  8.29s/it]
 32%|███▏      | 12022/37962 [27:52:04<59:02:04,  8.19s/it]
 32%|███▏      | 12023/37962 [27:52:12<59:28:27,  8.25s/it]
 32%|███▏      | 12024/37962 [27:52:20<59:16:42,  8.23s/it]
 32%|███▏      | 12025/37962 [27:52:28<58:25:52,  8.11s/it]
 32%|███▏      | 12026/37962 [27:52:37<58:40:25,  8.14s/it]
 32%|███▏      | 12027/37962 [27:52:44<58:01:39,  8.05s/it]
 32%|███▏      | 12028/37962 [27:52:53<58:39:48,  8.14s/it]
 32%|███▏      | 12029/37962 [27:53:01<59:07:40,  8.21s/it]
 32%|███▏      | 12030/37962 [27:53:09<58:52:31,  8.17s/it]
                                                           
{'loss': 1.2913, 'grad_norm': 33.18233108520508, 'learning_rate': 1.5984934929597473e-05, 'epoch': 0.32}

 32%|███▏      | 12030/37962 [27:53:09<58:52:31,  8.17s/it]
 32%|███▏      | 12031/37962 [27:53:17<59:05:04,  8.20s/it]
 32%|███▏      | 12032/37962 [27:53:26<59:17:57,  8.23s/it]
 32%|███▏      | 12033/37962 [27:53:34<59:36:17,  8.28s/it]
 32%|███▏      | 12034/37962 [27:53:43<60:22:57,  8.38s/it]
 32%|███▏      | 12035/37962 [27:53:51<59:56:59,  8.32s/it]
 32%|███▏      | 12036/37962 [27:54:00<60:47:43,  8.44s/it]
 32%|███▏      | 12037/37962 [27:54:08<59:30:21,  8.26s/it]
 32%|███▏      | 12038/37962 [27:54:16<59:35:14,  8.27s/it]
 32%|███▏      | 12039/37962 [27:54:24<59:06:49,  8.21s/it]
 32%|███▏      | 12040/37962 [27:54:32<59:28:07,  8.26s/it]
                                                           
{'loss': 1.4315, 'grad_norm': 52.71499252319336, 'learning_rate': 1.5978097848007723e-05, 'epoch': 0.32}

 32%|███▏      | 12040/37962 [27:54:32<59:28:07,  8.26s/it]
 32%|███▏      | 12041/37962 [27:54:41<59:42:26,  8.29s/it]
 32%|███▏      | 12042/37962 [27:54:49<59:51:41,  8.31s/it]
 32%|███▏      | 12043/37962 [27:54:57<59:54:38,  8.32s/it]
 32%|███▏      | 12044/37962 [27:55:06<59:50:06,  8.31s/it]
 32%|███▏      | 12045/37962 [27:55:14<60:00:48,  8.34s/it]
 32%|███▏      | 12046/37962 [27:55:22<59:50:32,  8.31s/it]
 32%|███▏      | 12047/37962 [27:55:31<59:58:39,  8.33s/it]
 32%|███▏      | 12048/37962 [27:55:39<60:06:58,  8.35s/it]
 32%|███▏      | 12049/37962 [27:55:48<60:20:56,  8.38s/it]
 32%|███▏      | 12050/37962 [27:55:56<59:59:12,  8.33s/it]
                                                           
{'loss': 1.4687, 'grad_norm': 10.75975227355957, 'learning_rate': 1.5971256415064115e-05, 'epoch': 0.32}

 32%|███▏      | 12050/37962 [27:55:56<59:59:12,  8.33s/it]
 32%|███▏      | 12051/37962 [27:56:04<59:37:09,  8.28s/it]
 32%|███▏      | 12052/37962 [27:56:12<59:21:39,  8.25s/it]
 32%|███▏      | 12053/37962 [27:56:20<59:32:13,  8.27s/it]
 32%|███▏      | 12054/37962 [27:56:29<59:50:50,  8.32s/it]
 32%|███▏      | 12055/37962 [27:56:37<60:06:25,  8.35s/it]
 32%|███▏      | 12056/37962 [27:56:45<59:46:20,  8.31s/it]
 32%|███▏      | 12057/37962 [27:56:53<58:36:41,  8.15s/it]
 32%|███▏      | 12058/37962 [27:57:02<59:09:13,  8.22s/it]
 32%|███▏      | 12059/37962 [27:57:10<58:35:50,  8.14s/it]
 32%|███▏      | 12060/37962 [27:57:18<58:49:39,  8.18s/it]
                                                           
{'loss': 1.3882, 'grad_norm': 8.186162948608398, 'learning_rate': 1.5964410635746406e-05, 'epoch': 0.32}

 32%|███▏      | 12060/37962 [27:57:18<58:49:39,  8.18s/it]
 32%|███▏      | 12061/37962 [27:57:26<58:28:40,  8.13s/it]
 32%|███▏      | 12062/37962 [27:57:34<59:04:34,  8.21s/it]
 32%|███▏      | 12063/37962 [27:57:42<57:43:08,  8.02s/it]
 32%|███▏      | 12064/37962 [27:57:50<58:21:20,  8.11s/it]
 32%|███▏      | 12065/37962 [27:57:58<58:38:22,  8.15s/it]
 32%|███▏      | 12066/37962 [27:58:07<58:57:50,  8.20s/it]
 32%|███▏      | 12067/37962 [27:58:15<59:01:30,  8.21s/it]
 32%|███▏      | 12068/37962 [27:58:23<58:57:31,  8.20s/it]
 32%|███▏      | 12069/37962 [27:58:31<59:11:34,  8.23s/it]
 32%|███▏      | 12070/37962 [27:58:40<59:33:54,  8.28s/it]
                                                           
{'loss': 1.4094, 'grad_norm': 63.93559265136719, 'learning_rate': 1.5957560515037518e-05, 'epoch': 0.32}

 32%|███▏      | 12070/37962 [27:58:40<59:33:54,  8.28s/it]
 32%|███▏      | 12071/37962 [27:58:48<59:34:02,  8.28s/it]
 32%|███▏      | 12072/37962 [27:58:56<59:25:03,  8.26s/it]
 32%|███▏      | 12073/37962 [27:59:04<58:52:29,  8.19s/it]
 32%|███▏      | 12074/37962 [27:59:13<58:58:36,  8.20s/it]
 32%|███▏      | 12075/37962 [27:59:21<59:22:13,  8.26s/it]
 32%|███▏      | 12076/37962 [27:59:29<58:39:06,  8.16s/it]
 32%|███▏      | 12077/37962 [27:59:37<58:45:14,  8.17s/it]
 32%|███▏      | 12078/37962 [27:59:45<58:34:28,  8.15s/it]
 32%|███▏      | 12079/37962 [27:59:53<58:53:49,  8.19s/it]
 32%|███▏      | 12080/37962 [28:00:02<58:52:11,  8.19s/it]
                                                           
{'loss': 1.5581, 'grad_norm': 10.444463729858398, 'learning_rate': 1.5950706057923544e-05, 'epoch': 0.32}

 32%|███▏      | 12080/37962 [28:00:02<58:52:11,  8.19s/it]
 32%|███▏      | 12081/37962 [28:00:10<59:04:33,  8.22s/it]
 32%|███▏      | 12082/37962 [28:00:18<58:29:43,  8.14s/it]
 32%|███▏      | 12083/37962 [28:00:26<58:52:29,  8.19s/it]
 32%|███▏      | 12084/37962 [28:00:34<59:02:51,  8.21s/it]
 32%|███▏      | 12085/37962 [28:00:43<59:10:14,  8.23s/it]
 32%|███▏      | 12086/37962 [28:00:51<58:45:59,  8.18s/it]
 32%|███▏      | 12087/37962 [28:00:59<59:07:59,  8.23s/it]
 32%|███▏      | 12088/37962 [28:01:08<59:34:01,  8.29s/it]
 32%|███▏      | 12089/37962 [28:01:16<59:16:51,  8.25s/it]
 32%|███▏      | 12090/37962 [28:01:24<59:29:46,  8.28s/it]
                                                           
{'loss': 1.4806, 'grad_norm': 13.02112102508545, 'learning_rate': 1.5943847269393716e-05, 'epoch': 0.32}

 32%|███▏      | 12090/37962 [28:01:24<59:29:46,  8.28s/it]
 32%|███▏      | 12091/37962 [28:01:32<59:10:25,  8.23s/it]
 32%|███▏      | 12092/37962 [28:01:41<59:29:50,  8.28s/it]
 32%|███▏      | 12093/37962 [28:01:49<60:10:10,  8.37s/it]
 32%|███▏      | 12094/37962 [28:01:57<59:18:41,  8.25s/it]
 32%|███▏      | 12095/37962 [28:02:06<59:35:30,  8.29s/it]
 32%|███▏      | 12096/37962 [28:02:14<59:59:59,  8.35s/it]
 32%|███▏      | 12097/37962 [28:02:29<74:32:34, 10.38s/it]
 32%|███▏      | 12098/37962 [28:02:37<69:51:31,  9.72s/it]
 32%|███▏      | 12099/37962 [28:02:46<66:41:57,  9.28s/it]
 32%|███▏      | 12100/37962 [28:02:54<64:21:17,  8.96s/it]
                                                           
{'loss': 1.3799, 'grad_norm': 8.401366233825684, 'learning_rate': 1.5936984154440435e-05, 'epoch': 0.32}

 32%|███▏      | 12100/37962 [28:02:54<64:21:17,  8.96s/it]
 32%|███▏      | 12101/37962 [28:03:02<62:56:17,  8.76s/it]
 32%|███▏      | 12102/37962 [28:03:10<62:05:44,  8.64s/it]
 32%|███▏      | 12103/37962 [28:03:19<61:28:44,  8.56s/it]
 32%|███▏      | 12104/37962 [28:03:27<61:04:18,  8.50s/it]
 32%|███▏      | 12105/37962 [28:03:35<60:41:34,  8.45s/it]
 32%|███▏      | 12106/37962 [28:03:44<60:41:08,  8.45s/it]
 32%|███▏      | 12107/37962 [28:03:52<60:15:44,  8.39s/it]
 32%|███▏      | 12108/37962 [28:04:01<60:13:47,  8.39s/it]
 32%|███▏      | 12109/37962 [28:04:09<60:21:12,  8.40s/it]
 32%|███▏      | 12110/37962 [28:04:17<60:11:41,  8.38s/it]
                                                           
{'loss': 1.3766, 'grad_norm': 6.218594074249268, 'learning_rate': 1.5930116718059237e-05, 'epoch': 0.32}

 32%|███▏      | 12110/37962 [28:04:17<60:11:41,  8.38s/it]
 32%|███▏      | 12111/37962 [28:04:26<60:01:21,  8.36s/it]
 32%|███▏      | 12112/37962 [28:04:34<59:19:10,  8.26s/it]
 32%|███▏      | 12113/37962 [28:04:42<59:29:57,  8.29s/it]
 32%|███▏      | 12114/37962 [28:04:50<58:48:24,  8.19s/it]
 32%|███▏      | 12115/37962 [28:04:58<59:04:38,  8.23s/it]
 32%|███▏      | 12116/37962 [28:05:07<59:14:52,  8.25s/it]
 32%|███▏      | 12117/37962 [28:05:14<58:22:15,  8.13s/it]
 32%|███▏      | 12118/37962 [28:05:23<58:51:01,  8.20s/it]
 32%|███▏      | 12119/37962 [28:05:31<58:39:43,  8.17s/it]
 32%|███▏      | 12120/37962 [28:05:39<58:57:22,  8.21s/it]
                                                           
{'loss': 1.3985, 'grad_norm': 10.2295560836792, 'learning_rate': 1.592324496524881e-05, 'epoch': 0.32}

 32%|███▏      | 12120/37962 [28:05:39<58:57:22,  8.21s/it]
 32%|███▏      | 12121/37962 [28:05:48<59:15:53,  8.26s/it]
 32%|███▏      | 12122/37962 [28:05:56<59:14:16,  8.25s/it]
 32%|███▏      | 12123/37962 [28:06:04<59:33:28,  8.30s/it]
 32%|███▏      | 12124/37962 [28:06:13<59:57:15,  8.35s/it]
 32%|███▏      | 12125/37962 [28:06:21<60:18:08,  8.40s/it]
 32%|███▏      | 12126/37962 [28:06:29<59:54:47,  8.35s/it]
 32%|███▏      | 12127/37962 [28:06:38<59:50:58,  8.34s/it]
 32%|███▏      | 12128/37962 [28:06:46<59:45:35,  8.33s/it]
 32%|███▏      | 12129/37962 [28:06:54<59:20:03,  8.27s/it]
 32%|███▏      | 12130/37962 [28:07:02<59:18:32,  8.27s/it]
                                                           
{'loss': 1.4876, 'grad_norm': 8.347479820251465, 'learning_rate': 1.5916368901010987e-05, 'epoch': 0.32}

 32%|███▏      | 12130/37962 [28:07:02<59:18:32,  8.27s/it]
 32%|███▏      | 12131/37962 [28:07:11<59:11:54,  8.25s/it]
 32%|███▏      | 12132/37962 [28:07:19<59:16:42,  8.26s/it]
 32%|███▏      | 12133/37962 [28:07:27<59:13:41,  8.26s/it]
 32%|███▏      | 12134/37962 [28:07:36<59:29:12,  8.29s/it]
 32%|███▏      | 12135/37962 [28:07:44<59:19:07,  8.27s/it]
 32%|███▏      | 12136/37962 [28:07:52<59:25:17,  8.28s/it]
 32%|███▏      | 12137/37962 [28:08:00<58:37:22,  8.17s/it]
 32%|███▏      | 12138/37962 [28:08:08<58:39:19,  8.18s/it]
 32%|███▏      | 12139/37962 [28:08:17<59:08:45,  8.25s/it]
 32%|███▏      | 12140/37962 [28:08:25<58:59:36,  8.22s/it]
                                                           
{'loss': 1.3582, 'grad_norm': 10.813390731811523, 'learning_rate': 1.5909488530350735e-05, 'epoch': 0.32}

 32%|███▏      | 12140/37962 [28:08:25<58:59:36,  8.22s/it]
 32%|███▏      | 12141/37962 [28:08:33<59:55:47,  8.36s/it]
 32%|███▏      | 12142/37962 [28:08:42<60:22:40,  8.42s/it]
 32%|███▏      | 12143/37962 [28:08:50<60:11:14,  8.39s/it]
 32%|███▏      | 12144/37962 [28:08:59<60:06:07,  8.38s/it]
 32%|███▏      | 12145/37962 [28:09:07<59:23:38,  8.28s/it]
 32%|███▏      | 12146/37962 [28:09:15<59:44:15,  8.33s/it]
 32%|███▏      | 12147/37962 [28:09:24<59:50:03,  8.34s/it]
 32%|███▏      | 12148/37962 [28:09:32<59:36:58,  8.31s/it]
 32%|███▏      | 12149/37962 [28:09:40<60:08:42,  8.39s/it]
 32%|███▏      | 12150/37962 [28:09:49<60:19:38,  8.41s/it]
                                                           
{'loss': 1.4061, 'grad_norm': 37.29164505004883, 'learning_rate': 1.5902603858276155e-05, 'epoch': 0.32}

 32%|███▏      | 12150/37962 [28:09:49<60:19:38,  8.41s/it]
 32%|███▏      | 12151/37962 [28:09:57<59:13:27,  8.26s/it]
 32%|███▏      | 12152/37962 [28:10:05<59:44:30,  8.33s/it]
 32%|███▏      | 12153/37962 [28:10:14<59:57:55,  8.36s/it]
 32%|███▏      | 12154/37962 [28:10:22<58:53:40,  8.22s/it]
 32%|███▏      | 12155/37962 [28:10:30<59:13:07,  8.26s/it]
 32%|███▏      | 12156/37962 [28:10:45<73:50:18, 10.30s/it]
 32%|███▏      | 12157/37962 [28:10:54<69:57:48,  9.76s/it]
 32%|███▏      | 12158/37962 [28:11:02<66:46:33,  9.32s/it]
 32%|███▏      | 12159/37962 [28:11:10<64:40:36,  9.02s/it]
 32%|███▏      | 12160/37962 [28:11:18<63:02:14,  8.80s/it]
                                                           
{'loss': 1.442, 'grad_norm': 684.0595092773438, 'learning_rate': 1.5895714889798485e-05, 'epoch': 0.32}

 32%|███▏      | 12160/37962 [28:11:18<63:02:14,  8.80s/it]
 32%|███▏      | 12161/37962 [28:11:27<61:42:27,  8.61s/it]
 32%|███▏      | 12162/37962 [28:11:35<61:08:21,  8.53s/it]
 32%|███▏      | 12163/37962 [28:11:43<60:45:56,  8.48s/it]
 32%|███▏      | 12164/37962 [28:11:51<59:47:33,  8.34s/it]
 32%|███▏      | 12165/37962 [28:12:00<59:49:23,  8.35s/it]
 32%|███▏      | 12166/37962 [28:12:08<59:38:40,  8.32s/it]
 32%|███▏      | 12167/37962 [28:12:16<59:52:50,  8.36s/it]
 32%|███▏      | 12168/37962 [28:12:25<59:57:11,  8.37s/it]
 32%|███▏      | 12169/37962 [28:12:33<60:06:56,  8.39s/it]
 32%|███▏      | 12170/37962 [28:12:41<59:51:07,  8.35s/it]
                                                           
{'loss': 1.488, 'grad_norm': 399.1373291015625, 'learning_rate': 1.5888821629932084e-05, 'epoch': 0.32}

 32%|███▏      | 12170/37962 [28:12:41<59:51:07,  8.35s/it]
 32%|███▏      | 12171/37962 [28:12:49<59:01:11,  8.24s/it]
 32%|███▏      | 12172/37962 [28:12:58<59:22:17,  8.29s/it]
 32%|███▏      | 12173/37962 [28:13:06<59:30:07,  8.31s/it]
 32%|███▏      | 12174/37962 [28:13:14<59:30:00,  8.31s/it]
 32%|███▏      | 12175/37962 [28:13:23<60:03:53,  8.39s/it]
 32%|███▏      | 12176/37962 [28:13:31<59:45:40,  8.34s/it]
 32%|███▏      | 12177/37962 [28:13:40<59:51:12,  8.36s/it]
 32%|███▏      | 12178/37962 [28:13:48<59:59:03,  8.38s/it]
 32%|███▏      | 12179/37962 [28:13:56<59:41:09,  8.33s/it]
 32%|███▏      | 12180/37962 [28:14:04<59:01:43,  8.24s/it]
                                                           
{'loss': 1.515, 'grad_norm': 25.75657844543457, 'learning_rate': 1.5881924083694435e-05, 'epoch': 0.32}

 32%|███▏      | 12180/37962 [28:14:04<59:01:43,  8.24s/it]
 32%|███▏      | 12181/37962 [28:14:12<58:42:56,  8.20s/it]
 32%|███▏      | 12182/37962 [28:14:21<59:08:13,  8.26s/it]
 32%|███▏      | 12183/37962 [28:14:29<58:12:13,  8.13s/it]
 32%|███▏      | 12184/37962 [28:14:37<59:23:51,  8.30s/it]
 32%|███▏      | 12185/37962 [28:14:46<59:08:13,  8.26s/it]
 32%|███▏      | 12186/37962 [28:14:54<59:48:11,  8.35s/it]
 32%|███▏      | 12187/37962 [28:15:03<60:11:04,  8.41s/it]
 32%|███▏      | 12188/37962 [28:15:11<59:43:50,  8.34s/it]
 32%|███▏      | 12189/37962 [28:15:19<58:34:31,  8.18s/it]
 32%|███▏      | 12190/37962 [28:15:27<58:39:46,  8.19s/it]
                                                           
{'loss': 1.4289, 'grad_norm': 4.991872310638428, 'learning_rate': 1.587502225610614e-05, 'epoch': 0.32}

 32%|███▏      | 12190/37962 [28:15:27<58:39:46,  8.19s/it]
 32%|███▏      | 12191/37962 [28:15:35<59:30:56,  8.31s/it]
 32%|███▏      | 12192/37962 [28:15:44<59:42:06,  8.34s/it]
 32%|███▏      | 12193/37962 [28:15:52<59:33:36,  8.32s/it]
 32%|███▏      | 12194/37962 [28:16:00<59:35:58,  8.33s/it]
 32%|███▏      | 12195/37962 [28:16:09<59:53:51,  8.37s/it]
 32%|███▏      | 12196/37962 [28:16:17<59:18:24,  8.29s/it]
 32%|███▏      | 12197/37962 [28:16:25<59:27:16,  8.31s/it]
 32%|███▏      | 12198/37962 [28:16:34<59:06:17,  8.26s/it]
 32%|███▏      | 12199/37962 [28:16:42<59:13:22,  8.28s/it]
 32%|███▏      | 12200/37962 [28:16:50<58:59:01,  8.24s/it]
                                                           
{'loss': 1.4483, 'grad_norm': 7.99547004699707, 'learning_rate': 1.5868116152190925e-05, 'epoch': 0.32}

 32%|███▏      | 12200/37962 [28:16:50<58:59:01,  8.24s/it]
 32%|███▏      | 12201/37962 [28:16:58<59:19:11,  8.29s/it]
 32%|███▏      | 12202/37962 [28:17:07<59:09:19,  8.27s/it]
 32%|███▏      | 12203/37962 [28:17:15<58:51:38,  8.23s/it]
 32%|███▏      | 12204/37962 [28:17:23<58:49:43,  8.22s/it]
 32%|███▏      | 12205/37962 [28:17:31<59:09:15,  8.27s/it]
 32%|███▏      | 12206/37962 [28:17:40<58:59:39,  8.25s/it]
 32%|███▏      | 12207/37962 [28:17:48<59:12:24,  8.28s/it]
 32%|███▏      | 12208/37962 [28:17:56<59:10:50,  8.27s/it]
 32%|███▏      | 12209/37962 [28:18:04<58:46:34,  8.22s/it]
 32%|███▏      | 12210/37962 [28:18:12<58:47:41,  8.22s/it]
                                                           
{'loss': 1.3018, 'grad_norm': 11.554422378540039, 'learning_rate': 1.586120577697562e-05, 'epoch': 0.32}

 32%|███▏      | 12210/37962 [28:18:12<58:47:41,  8.22s/it]
 32%|███▏      | 12211/37962 [28:18:21<59:18:10,  8.29s/it]
 32%|███▏      | 12212/37962 [28:18:29<59:03:19,  8.26s/it]
 32%|███▏      | 12213/37962 [28:18:37<59:14:07,  8.28s/it]
 32%|███▏      | 12214/37962 [28:18:46<59:02:55,  8.26s/it]
 32%|███▏      | 12215/37962 [28:18:54<58:56:02,  8.24s/it]
 32%|███▏      | 12216/37962 [28:19:03<59:50:12,  8.37s/it]
 32%|███▏      | 12217/37962 [28:19:11<60:15:36,  8.43s/it]
 32%|███▏      | 12218/37962 [28:19:19<59:50:09,  8.37s/it]
 32%|███▏      | 12219/37962 [28:19:28<59:28:43,  8.32s/it]
 32%|███▏      | 12220/37962 [28:19:36<59:09:38,  8.27s/it]
                                                           
{'loss': 1.5076, 'grad_norm': 7.3297014236450195, 'learning_rate': 1.5854291135490167e-05, 'epoch': 0.32}

 32%|███▏      | 12220/37962 [28:19:36<59:09:38,  8.27s/it]
 32%|███▏      | 12221/37962 [28:19:44<59:17:01,  8.29s/it]
 32%|███▏      | 12222/37962 [28:19:52<59:03:02,  8.26s/it]
 32%|███▏      | 12223/37962 [28:20:01<59:11:02,  8.28s/it]
 32%|███▏      | 12224/37962 [28:20:09<59:28:45,  8.32s/it]
 32%|███▏      | 12225/37962 [28:20:17<59:56:18,  8.38s/it]
 32%|███▏      | 12226/37962 [28:20:26<59:51:01,  8.37s/it]
 32%|███▏      | 12227/37962 [28:20:34<59:33:57,  8.33s/it]
 32%|███▏      | 12228/37962 [28:20:42<59:41:43,  8.35s/it]
 32%|███▏      | 12229/37962 [28:20:50<58:45:45,  8.22s/it]
 32%|███▏      | 12230/37962 [28:20:59<59:01:52,  8.26s/it]
                                                           
{'loss': 1.2163, 'grad_norm': 10.765825271606445, 'learning_rate': 1.584737223276762e-05, 'epoch': 0.32}

 32%|███▏      | 12230/37962 [28:20:59<59:01:52,  8.26s/it]
 32%|███▏      | 12231/37962 [28:21:07<58:25:14,  8.17s/it]
 32%|███▏      | 12232/37962 [28:21:15<58:38:11,  8.20s/it]
 32%|███▏      | 12233/37962 [28:21:23<58:56:22,  8.25s/it]
 32%|███▏      | 12234/37962 [28:21:32<59:18:11,  8.30s/it]
 32%|███▏      | 12235/37962 [28:21:40<59:43:26,  8.36s/it]
 32%|███▏      | 12236/37962 [28:21:48<59:30:49,  8.33s/it]
 32%|███▏      | 12237/37962 [28:21:57<59:37:29,  8.34s/it]
 32%|███▏      | 12238/37962 [28:22:05<59:33:28,  8.33s/it]
 32%|███▏      | 12239/37962 [28:22:14<59:31:38,  8.33s/it]
 32%|███▏      | 12240/37962 [28:22:22<59:18:58,  8.30s/it]
                                                           
{'loss': 1.4524, 'grad_norm': 49.778411865234375, 'learning_rate': 1.5840449073844118e-05, 'epoch': 0.32}

 32%|███▏      | 12240/37962 [28:22:22<59:18:58,  8.30s/it]
 32%|███▏      | 12241/37962 [28:22:30<58:36:57,  8.20s/it]
 32%|███▏      | 12242/37962 [28:22:38<58:55:54,  8.25s/it]
 32%|███▏      | 12243/37962 [28:22:46<58:05:56,  8.13s/it]
 32%|███▏      | 12244/37962 [28:22:54<57:59:05,  8.12s/it]
 32%|███▏      | 12245/37962 [28:23:02<58:37:49,  8.21s/it]
 32%|███▏      | 12246/37962 [28:23:11<58:32:39,  8.20s/it]
 32%|███▏      | 12247/37962 [28:23:19<57:59:36,  8.12s/it]
 32%|███▏      | 12248/37962 [28:23:27<58:36:50,  8.21s/it]
 32%|███▏      | 12249/37962 [28:23:36<59:28:49,  8.33s/it]
 32%|███▏      | 12250/37962 [28:23:44<59:28:08,  8.33s/it]
                                                           
{'loss': 1.5056, 'grad_norm': 8.963550567626953, 'learning_rate': 1.5833521663758915e-05, 'epoch': 0.32}

 32%|███▏      | 12250/37962 [28:23:44<59:28:08,  8.33s/it]
 32%|███▏      | 12251/37962 [28:23:52<59:20:53,  8.31s/it]
 32%|███▏      | 12252/37962 [28:24:01<59:28:21,  8.33s/it]
 32%|███▏      | 12253/37962 [28:24:09<59:12:42,  8.29s/it]
 32%|███▏      | 12254/37962 [28:24:17<59:02:27,  8.27s/it]
 32%|███▏      | 12255/37962 [28:24:25<59:13:56,  8.29s/it]
 32%|███▏      | 12256/37962 [28:24:34<59:21:51,  8.31s/it]
 32%|███▏      | 12257/37962 [28:24:42<59:07:40,  8.28s/it]
 32%|███▏      | 12258/37962 [28:24:50<59:21:24,  8.31s/it]
 32%|███▏      | 12259/37962 [28:24:58<59:05:44,  8.28s/it]
 32%|███▏      | 12260/37962 [28:25:07<59:19:01,  8.31s/it]
                                                           
{'loss': 1.4518, 'grad_norm': 6.211223125457764, 'learning_rate': 1.5826590007554356e-05, 'epoch': 0.32}

 32%|███▏      | 12260/37962 [28:25:07<59:19:01,  8.31s/it]
 32%|███▏      | 12261/37962 [28:25:15<58:15:51,  8.16s/it]
 32%|███▏      | 12262/37962 [28:25:23<58:35:59,  8.21s/it]
 32%|███▏      | 12263/37962 [28:25:31<59:02:15,  8.27s/it]
 32%|███▏      | 12264/37962 [28:25:40<59:03:45,  8.27s/it]
 32%|███▏      | 12265/37962 [28:25:48<58:59:59,  8.27s/it]
 32%|███▏      | 12266/37962 [28:25:56<59:04:23,  8.28s/it]
 32%|███▏      | 12267/37962 [28:26:04<59:03:36,  8.27s/it]
 32%|███▏      | 12268/37962 [28:26:13<59:33:27,  8.34s/it]
 32%|███▏      | 12269/37962 [28:26:21<58:36:33,  8.21s/it]
 32%|███▏      | 12270/37962 [28:26:29<58:55:56,  8.26s/it]
                                                           
{'loss': 1.2503, 'grad_norm': 8.3572359085083, 'learning_rate': 1.581965411027586e-05, 'epoch': 0.32}

 32%|███▏      | 12270/37962 [28:26:29<58:55:56,  8.26s/it]
 32%|███▏      | 12271/37962 [28:26:38<59:21:51,  8.32s/it]
 32%|███▏      | 12272/37962 [28:26:46<59:28:34,  8.33s/it]
 32%|███▏      | 12273/37962 [28:26:54<59:11:59,  8.30s/it]
 32%|███▏      | 12274/37962 [28:27:03<59:23:44,  8.32s/it]
 32%|███▏      | 12275/37962 [28:27:11<59:18:50,  8.31s/it]
 32%|███▏      | 12276/37962 [28:27:19<59:09:09,  8.29s/it]
 32%|███▏      | 12277/37962 [28:27:27<59:03:38,  8.28s/it]
 32%|███▏      | 12278/37962 [28:27:36<59:08:25,  8.29s/it]
 32%|███▏      | 12279/37962 [28:27:44<59:29:18,  8.34s/it]
 32%|███▏      | 12280/37962 [28:27:52<58:20:38,  8.18s/it]
                                                           
{'loss': 1.3305, 'grad_norm': 8.056180000305176, 'learning_rate': 1.5812713976971966e-05, 'epoch': 0.32}

 32%|███▏      | 12280/37962 [28:27:52<58:20:38,  8.18s/it]
 32%|███▏      | 12281/37962 [28:28:00<58:05:27,  8.14s/it]
 32%|███▏      | 12282/37962 [28:28:08<58:28:10,  8.20s/it]
 32%|███▏      | 12283/37962 [28:28:17<58:41:24,  8.23s/it]
 32%|███▏      | 12284/37962 [28:28:25<58:43:35,  8.23s/it]
 32%|███▏      | 12285/37962 [28:28:33<58:58:27,  8.27s/it]
 32%|███▏      | 12286/37962 [28:28:41<58:47:23,  8.24s/it]
 32%|███▏      | 12287/37962 [28:28:50<58:30:54,  8.20s/it]
 32%|███▏      | 12288/37962 [28:28:58<58:47:33,  8.24s/it]
 32%|███▏      | 12289/37962 [28:29:06<58:19:25,  8.18s/it]
 32%|███▏      | 12290/37962 [28:29:14<57:44:09,  8.10s/it]
                                                           
{'loss': 1.4634, 'grad_norm': 3.9289252758026123, 'learning_rate': 1.580576961269426e-05, 'epoch': 0.32}

 32%|███▏      | 12290/37962 [28:29:14<57:44:09,  8.10s/it]
 32%|███▏      | 12291/37962 [28:29:22<57:55:54,  8.12s/it]
 32%|███▏      | 12292/37962 [28:29:30<58:27:24,  8.20s/it]
 32%|███▏      | 12293/37962 [28:29:39<58:40:35,  8.23s/it]
 32%|███▏      | 12294/37962 [28:29:47<59:08:38,  8.30s/it]
 32%|███▏      | 12295/37962 [28:29:55<59:08:52,  8.30s/it]
 32%|███▏      | 12296/37962 [28:30:04<59:12:43,  8.31s/it]
 32%|███▏      | 12297/37962 [28:30:12<59:07:43,  8.29s/it]
 32%|███▏      | 12298/37962 [28:30:21<59:28:13,  8.34s/it]
 32%|███▏      | 12299/37962 [28:30:29<59:33:25,  8.35s/it]
 32%|███▏      | 12300/37962 [28:30:38<60:05:11,  8.43s/it]
                                                           
{'loss': 1.4865, 'grad_norm': 11.075051307678223, 'learning_rate': 1.5798821022497435e-05, 'epoch': 0.32}

 32%|███▏      | 12300/37962 [28:30:38<60:05:11,  8.43s/it]
 32%|███▏      | 12301/37962 [28:30:46<59:39:16,  8.37s/it]
 32%|███▏      | 12302/37962 [28:30:54<59:26:08,  8.34s/it]
 32%|███▏      | 12303/37962 [28:31:10<74:47:42, 10.49s/it]
 32%|███▏      | 12304/37962 [28:31:18<70:36:30,  9.91s/it]
 32%|███▏      | 12305/37962 [28:31:34<83:28:20, 11.71s/it]
 32%|███▏      | 12306/37962 [28:31:42<76:17:13, 10.70s/it]
 32%|███▏      | 12307/37962 [28:31:51<71:05:30,  9.98s/it]
 32%|███▏      | 12308/37962 [28:31:59<66:53:59,  9.39s/it]
 32%|███▏      | 12309/37962 [28:32:07<64:46:04,  9.09s/it]
 32%|███▏      | 12310/37962 [28:32:15<62:34:40,  8.78s/it]
                                                           
{'loss': 1.4686, 'grad_norm': 18.046520233154297, 'learning_rate': 1.5791868211439244e-05, 'epoch': 0.32}

 32%|███▏      | 12310/37962 [28:32:15<62:34:40,  8.78s/it]
 32%|███▏      | 12311/37962 [28:32:23<61:34:11,  8.64s/it]
 32%|███▏      | 12312/37962 [28:32:32<60:54:03,  8.55s/it]
 32%|███▏      | 12313/37962 [28:32:40<60:09:37,  8.44s/it]
 32%|███▏      | 12314/37962 [28:32:48<59:36:35,  8.37s/it]
 32%|███▏      | 12315/37962 [28:32:56<59:34:11,  8.36s/it]
 32%|███▏      | 12316/37962 [28:33:05<59:14:31,  8.32s/it]
 32%|███▏      | 12317/37962 [28:33:13<59:04:39,  8.29s/it]
 32%|███▏      | 12318/37962 [28:33:21<58:38:59,  8.23s/it]
 32%|███▏      | 12319/37962 [28:33:29<58:59:34,  8.28s/it]
 32%|███▏      | 12320/37962 [28:33:38<58:58:51,  8.28s/it]
                                                           
{'loss': 1.458, 'grad_norm': 25.2907772064209, 'learning_rate': 1.578491118458052e-05, 'epoch': 0.32}

 32%|███▏      | 12320/37962 [28:33:38<58:58:51,  8.28s/it]
 32%|███▏      | 12321/37962 [28:33:46<59:08:10,  8.30s/it]
 32%|███▏      | 12322/37962 [28:33:54<59:07:02,  8.30s/it]
 32%|███▏      | 12323/37962 [28:34:03<59:51:06,  8.40s/it]
 32%|███▏      | 12324/37962 [28:34:11<59:34:04,  8.36s/it]
 32%|███▏      | 12325/37962 [28:34:20<59:31:54,  8.36s/it]
 32%|███▏      | 12326/37962 [28:34:28<59:11:28,  8.31s/it]
 32%|███▏      | 12327/37962 [28:34:36<59:03:58,  8.29s/it]
 32%|███▏      | 12328/37962 [28:34:45<59:34:32,  8.37s/it]
 32%|███▏      | 12329/37962 [28:34:53<59:29:03,  8.35s/it]
 32%|███▏      | 12330/37962 [28:35:01<59:16:12,  8.32s/it]
                                                           
{'loss': 1.3756, 'grad_norm': 13.678783416748047, 'learning_rate': 1.5777949946985157e-05, 'epoch': 0.32}

 32%|███▏      | 12330/37962 [28:35:01<59:16:12,  8.32s/it]
 32%|███▏      | 12331/37962 [28:35:09<58:22:10,  8.20s/it]
 32%|███▏      | 12332/37962 [28:35:17<58:47:03,  8.26s/it]
 32%|███▏      | 12333/37962 [28:35:26<58:47:10,  8.26s/it]
 32%|███▏      | 12334/37962 [28:35:34<59:11:58,  8.32s/it]
 32%|███▏      | 12335/37962 [28:35:43<59:11:47,  8.32s/it]
 32%|███▏      | 12336/37962 [28:35:51<59:18:16,  8.33s/it]
 32%|███▏      | 12337/37962 [28:35:59<59:09:22,  8.31s/it]
 33%|███▎      | 12338/37962 [28:36:07<58:59:37,  8.29s/it]
 33%|███▎      | 12339/37962 [28:36:16<58:58:47,  8.29s/it]
 33%|███▎      | 12340/37962 [28:36:24<58:35:50,  8.23s/it]
                                                           
{'loss': 1.3323, 'grad_norm': 11.862297058105469, 'learning_rate': 1.577098450372013e-05, 'epoch': 0.33}

 33%|███▎      | 12340/37962 [28:36:24<58:35:50,  8.23s/it]
 33%|███▎      | 12341/37962 [28:36:32<58:59:50,  8.29s/it]
 33%|███▎      | 12342/37962 [28:36:40<58:18:52,  8.19s/it]
 33%|███▎      | 12343/37962 [28:36:48<58:24:10,  8.21s/it]
 33%|███▎      | 12344/37962 [28:36:57<58:39:27,  8.24s/it]
 33%|███▎      | 12345/37962 [28:37:05<58:32:40,  8.23s/it]
 33%|███▎      | 12346/37962 [28:37:13<58:32:06,  8.23s/it]
 33%|███▎      | 12347/37962 [28:37:21<58:35:12,  8.23s/it]
 33%|███▎      | 12348/37962 [28:37:30<58:25:08,  8.21s/it]
 33%|███▎      | 12349/37962 [28:37:38<58:13:04,  8.18s/it]
 33%|███▎      | 12350/37962 [28:37:46<58:09:01,  8.17s/it]
                                                           
{'loss': 1.5687, 'grad_norm': 4.0484747886657715, 'learning_rate': 1.5764014859855456e-05, 'epoch': 0.33}

 33%|███▎      | 12350/37962 [28:37:46<58:09:01,  8.17s/it]
 33%|███▎      | 12351/37962 [28:37:54<58:37:42,  8.24s/it]
 33%|███▎      | 12352/37962 [28:38:02<58:38:47,  8.24s/it]
 33%|███▎      | 12353/37962 [28:38:11<58:53:56,  8.28s/it]
 33%|███▎      | 12354/37962 [28:38:19<59:14:15,  8.33s/it]
 33%|███▎      | 12355/37962 [28:38:28<59:22:26,  8.35s/it]
 33%|███▎      | 12356/37962 [28:38:36<59:09:03,  8.32s/it]
 33%|███▎      | 12357/37962 [28:38:44<59:41:09,  8.39s/it]
 33%|███▎      | 12358/37962 [28:38:53<59:40:57,  8.39s/it]
 33%|███▎      | 12359/37962 [28:39:01<59:50:47,  8.41s/it]
 33%|███▎      | 12360/37962 [28:39:10<59:44:14,  8.40s/it]
                                                           
{'loss': 1.3572, 'grad_norm': 3.4585046768188477, 'learning_rate': 1.5757041020464215e-05, 'epoch': 0.33}

 33%|███▎      | 12360/37962 [28:39:10<59:44:14,  8.40s/it]
 33%|███▎      | 12361/37962 [28:39:18<59:41:46,  8.39s/it]
 33%|███▎      | 12362/37962 [28:39:27<59:48:29,  8.41s/it]
 33%|███▎      | 12363/37962 [28:39:35<59:32:35,  8.37s/it]
 33%|███▎      | 12364/37962 [28:39:43<60:10:53,  8.46s/it]
 33%|███▎      | 12365/37962 [28:39:52<60:07:26,  8.46s/it]
 33%|███▎      | 12366/37962 [28:40:01<60:34:48,  8.52s/it]
 33%|███▎      | 12367/37962 [28:40:09<60:34:49,  8.52s/it]
 33%|███▎      | 12368/37962 [28:40:18<60:29:15,  8.51s/it]
 33%|███▎      | 12369/37962 [28:40:26<60:34:42,  8.52s/it]
 33%|███▎      | 12370/37962 [28:40:35<61:19:20,  8.63s/it]
                                                           
{'loss': 1.4131, 'grad_norm': 12.318306922912598, 'learning_rate': 1.5750062990622558e-05, 'epoch': 0.33}

 33%|███▎      | 12370/37962 [28:40:35<61:19:20,  8.63s/it]
 33%|███▎      | 12371/37962 [28:40:44<61:09:44,  8.60s/it]
 33%|███▎      | 12372/37962 [28:40:52<60:43:39,  8.54s/it]
 33%|███▎      | 12373/37962 [28:41:00<60:04:39,  8.45s/it]
 33%|███▎      | 12374/37962 [28:41:09<60:18:26,  8.48s/it]
 33%|███▎      | 12375/37962 [28:41:17<59:50:19,  8.42s/it]
 33%|███▎      | 12376/37962 [28:41:25<59:54:07,  8.43s/it]
 33%|███▎      | 12377/37962 [28:41:34<59:32:15,  8.38s/it]
 33%|███▎      | 12378/37962 [28:41:42<59:16:18,  8.34s/it]
 33%|███▎      | 12379/37962 [28:41:50<59:00:33,  8.30s/it]
 33%|███▎      | 12380/37962 [28:41:59<59:02:19,  8.31s/it]
                                                           
{'loss': 1.4599, 'grad_norm': 10.13508129119873, 'learning_rate': 1.5743080775409654e-05, 'epoch': 0.33}

 33%|███▎      | 12380/37962 [28:41:59<59:02:19,  8.31s/it]
 33%|███▎      | 12381/37962 [28:42:07<59:25:00,  8.36s/it]
 33%|███▎      | 12382/37962 [28:42:15<59:26:51,  8.37s/it]
 33%|███▎      | 12383/37962 [28:42:24<59:00:40,  8.31s/it]
 33%|███▎      | 12384/37962 [28:42:32<58:51:53,  8.28s/it]
 33%|███▎      | 12385/37962 [28:42:40<59:35:51,  8.39s/it]
 33%|███▎      | 12386/37962 [28:42:49<59:42:00,  8.40s/it]
 33%|███▎      | 12387/37962 [28:42:57<59:19:52,  8.35s/it]
 33%|███▎      | 12388/37962 [28:43:06<59:40:05,  8.40s/it]
 33%|███▎      | 12389/37962 [28:43:14<59:41:08,  8.40s/it]
 33%|███▎      | 12390/37962 [28:43:22<59:16:47,  8.35s/it]
                                                           
{'loss': 1.3552, 'grad_norm': 7.670694828033447, 'learning_rate': 1.5736094379907755e-05, 'epoch': 0.33}

 33%|███▎      | 12390/37962 [28:43:22<59:16:47,  8.35s/it]
 33%|███▎      | 12391/37962 [28:43:30<58:04:44,  8.18s/it]
 33%|███▎      | 12392/37962 [28:43:38<58:29:22,  8.23s/it]
 33%|███▎      | 12393/37962 [28:43:47<58:44:35,  8.27s/it]
 33%|███▎      | 12394/37962 [28:43:55<59:10:23,  8.33s/it]
 33%|███▎      | 12395/37962 [28:44:03<58:26:52,  8.23s/it]
 33%|███▎      | 12396/37962 [28:44:12<58:48:05,  8.28s/it]
 33%|███▎      | 12397/37962 [28:44:20<58:46:40,  8.28s/it]
 33%|███▎      | 12398/37962 [28:44:28<58:36:12,  8.25s/it]
 33%|███▎      | 12399/37962 [28:44:36<58:31:48,  8.24s/it]
 33%|███▎      | 12400/37962 [28:44:45<58:31:59,  8.24s/it]
                                                           
{'loss': 1.2865, 'grad_norm': 5.777686595916748, 'learning_rate': 1.572910380920212e-05, 'epoch': 0.33}

 33%|███▎      | 12400/37962 [28:44:45<58:31:59,  8.24s/it]
 33%|███▎      | 12401/37962 [28:44:53<58:27:52,  8.23s/it]
 33%|███▎      | 12402/37962 [28:45:01<58:39:48,  8.26s/it]
 33%|███▎      | 12403/37962 [28:45:09<58:52:16,  8.29s/it]
 33%|███▎      | 12404/37962 [28:45:18<58:45:45,  8.28s/it]
 33%|███▎      | 12405/37962 [28:45:26<58:43:35,  8.27s/it]
 33%|███▎      | 12406/37962 [28:45:34<58:32:55,  8.25s/it]
 33%|███▎      | 12407/37962 [28:45:42<58:26:07,  8.23s/it]
 33%|███▎      | 12408/37962 [28:45:51<58:31:56,  8.25s/it]
 33%|███▎      | 12409/37962 [28:45:58<57:23:48,  8.09s/it]
 33%|███▎      | 12410/37962 [28:46:07<57:57:13,  8.17s/it]
                                                           
{'loss': 1.5236, 'grad_norm': 21.895729064941406, 'learning_rate': 1.5722109068381076e-05, 'epoch': 0.33}

 33%|███▎      | 12410/37962 [28:46:07<57:57:13,  8.17s/it]
 33%|███▎      | 12411/37962 [28:46:15<58:03:32,  8.18s/it]
 33%|███▎      | 12412/37962 [28:46:23<57:28:54,  8.10s/it]
 33%|███▎      | 12413/37962 [28:46:31<57:34:29,  8.11s/it]
 33%|███▎      | 12414/37962 [28:46:39<58:23:50,  8.23s/it]
 33%|███▎      | 12415/37962 [28:46:48<58:50:17,  8.29s/it]
 33%|███▎      | 12416/37962 [28:46:56<58:26:26,  8.24s/it]
 33%|███▎      | 12417/37962 [28:47:05<59:12:48,  8.34s/it]
 33%|███▎      | 12418/37962 [28:47:13<58:56:58,  8.31s/it]
 33%|███▎      | 12419/37962 [28:47:21<58:48:34,  8.29s/it]
 33%|███▎      | 12420/37962 [28:47:29<58:45:28,  8.28s/it]
                                                           
{'loss': 1.4118, 'grad_norm': 5.889650821685791, 'learning_rate': 1.5715110162535972e-05, 'epoch': 0.33}

 33%|███▎      | 12420/37962 [28:47:29<58:45:28,  8.28s/it]
 33%|███▎      | 12421/37962 [28:47:38<58:56:58,  8.31s/it]
 33%|███▎      | 12422/37962 [28:47:46<58:51:45,  8.30s/it]
 33%|███▎      | 12423/37962 [28:47:54<58:41:19,  8.27s/it]
 33%|███▎      | 12424/37962 [28:48:02<58:43:01,  8.28s/it]
 33%|███▎      | 12425/37962 [28:48:11<58:37:47,  8.27s/it]
 33%|███▎      | 12426/37962 [28:48:19<58:37:05,  8.26s/it]
 33%|███▎      | 12427/37962 [28:48:27<57:54:12,  8.16s/it]
 33%|███▎      | 12428/37962 [28:48:35<58:46:09,  8.29s/it]
 33%|███▎      | 12429/37962 [28:48:44<58:38:27,  8.27s/it]
 33%|███▎      | 12430/37962 [28:48:52<57:54:23,  8.16s/it]
                                                           
{'loss': 1.3995, 'grad_norm': 8.279088973999023, 'learning_rate': 1.5708107096761184e-05, 'epoch': 0.33}

 33%|███▎      | 12430/37962 [28:48:52<57:54:23,  8.16s/it]
 33%|███▎      | 12431/37962 [28:49:00<58:05:29,  8.19s/it]
 33%|███▎      | 12432/37962 [28:49:16<74:00:25, 10.44s/it]
 33%|███▎      | 12433/37962 [28:49:24<69:19:01,  9.77s/it]
 33%|███▎      | 12434/37962 [28:49:32<65:56:51,  9.30s/it]
 33%|███▎      | 12435/37962 [28:49:40<63:36:37,  8.97s/it]
 33%|███▎      | 12436/37962 [28:49:49<62:16:31,  8.78s/it]
 33%|███▎      | 12437/37962 [28:49:57<61:09:03,  8.62s/it]
 33%|███▎      | 12438/37962 [28:50:05<59:49:05,  8.44s/it]
 33%|███▎      | 12439/37962 [28:50:13<59:22:52,  8.38s/it]
 33%|███▎      | 12440/37962 [28:50:21<58:59:28,  8.32s/it]
                                                           
{'loss': 1.4041, 'grad_norm': 13.801074028015137, 'learning_rate': 1.570109987615413e-05, 'epoch': 0.33}

 33%|███▎      | 12440/37962 [28:50:21<58:59:28,  8.32s/it]
 33%|███▎      | 12441/37962 [28:50:30<59:14:01,  8.36s/it]
 33%|███▎      | 12442/37962 [28:50:38<59:10:48,  8.35s/it]
 33%|███▎      | 12443/37962 [28:50:46<59:11:15,  8.35s/it]
 33%|███▎      | 12444/37962 [28:50:55<58:56:02,  8.31s/it]
 33%|███▎      | 12445/37962 [28:51:03<59:02:22,  8.33s/it]
 33%|███▎      | 12446/37962 [28:51:11<58:41:39,  8.28s/it]
 33%|███▎      | 12447/37962 [28:51:19<58:44:28,  8.29s/it]
 33%|███▎      | 12448/37962 [28:51:28<58:31:55,  8.26s/it]
 33%|███▎      | 12449/37962 [28:51:36<58:52:42,  8.31s/it]
 33%|███▎      | 12450/37962 [28:51:44<58:33:21,  8.26s/it]
                                                           
{'loss': 1.4303, 'grad_norm': 9.493692398071289, 'learning_rate': 1.5694088505815238e-05, 'epoch': 0.33}

 33%|███▎      | 12450/37962 [28:51:44<58:33:21,  8.26s/it]
 33%|███▎      | 12451/37962 [28:51:52<58:32:53,  8.26s/it]
 33%|███▎      | 12452/37962 [28:52:01<58:17:43,  8.23s/it]
 33%|███▎      | 12453/37962 [28:52:09<58:45:13,  8.29s/it]
 33%|███▎      | 12454/37962 [28:52:17<58:45:18,  8.29s/it]
 33%|███▎      | 12455/37962 [28:52:26<58:40:36,  8.28s/it]
 33%|███▎      | 12456/37962 [28:52:34<58:46:28,  8.30s/it]
 33%|███▎      | 12457/37962 [28:52:42<58:52:20,  8.31s/it]
 33%|███▎      | 12458/37962 [28:52:50<58:45:16,  8.29s/it]
 33%|███▎      | 12459/37962 [28:52:58<58:08:55,  8.21s/it]
 33%|███▎      | 12460/37962 [28:53:07<58:34:16,  8.27s/it]
                                                           
{'loss': 1.3846, 'grad_norm': 12.441193580627441, 'learning_rate': 1.5687072990847965e-05, 'epoch': 0.33}

 33%|███▎      | 12460/37962 [28:53:07<58:34:16,  8.27s/it]
 33%|███▎      | 12461/37962 [28:53:15<57:53:02,  8.17s/it]
 33%|███▎      | 12462/37962 [28:53:23<58:01:54,  8.19s/it]
 33%|███▎      | 12463/37962 [28:53:31<58:03:19,  8.20s/it]
 33%|███▎      | 12464/37962 [28:53:40<58:20:12,  8.24s/it]
 33%|███▎      | 12465/37962 [28:53:48<57:44:20,  8.15s/it]
 33%|███▎      | 12466/37962 [28:53:56<58:03:58,  8.20s/it]
 33%|███▎      | 12467/37962 [28:54:04<58:02:18,  8.20s/it]
 33%|███▎      | 12468/37962 [28:54:12<58:14:37,  8.22s/it]
 33%|███▎      | 12469/37962 [28:54:20<58:00:31,  8.19s/it]
 33%|███▎      | 12470/37962 [28:54:29<57:41:37,  8.15s/it]
                                                           
{'loss': 1.2764, 'grad_norm': 13.371628761291504, 'learning_rate': 1.568005333635879e-05, 'epoch': 0.33}

 33%|███▎      | 12470/37962 [28:54:29<57:41:37,  8.15s/it]
 33%|███▎      | 12471/37962 [28:54:37<57:53:55,  8.18s/it]
 33%|███▎      | 12472/37962 [28:54:45<57:57:01,  8.18s/it]
 33%|███▎      | 12473/37962 [28:54:53<57:18:33,  8.09s/it]
 33%|███▎      | 12474/37962 [28:55:01<57:33:29,  8.13s/it]
 33%|███▎      | 12475/37962 [28:55:09<57:55:48,  8.18s/it]
 33%|███▎      | 12476/37962 [28:55:17<57:19:20,  8.10s/it]
 33%|███▎      | 12477/37962 [28:55:26<57:41:08,  8.15s/it]
 33%|███▎      | 12478/37962 [28:55:34<57:55:02,  8.18s/it]
 33%|███▎      | 12479/37962 [28:55:42<58:16:54,  8.23s/it]
 33%|███▎      | 12480/37962 [28:55:50<57:41:50,  8.15s/it]
                                                           
{'loss': 1.2931, 'grad_norm': 16.4624080657959, 'learning_rate': 1.5673029547457184e-05, 'epoch': 0.33}

 33%|███▎      | 12480/37962 [28:55:50<57:41:50,  8.15s/it]
 33%|███▎      | 12481/37962 [28:55:58<57:43:58,  8.16s/it]
 33%|███▎      | 12482/37962 [28:56:06<57:50:39,  8.17s/it]
 33%|███▎      | 12483/37962 [28:56:14<57:30:31,  8.13s/it]
 33%|███▎      | 12484/37962 [28:56:23<57:26:37,  8.12s/it]
 33%|███▎      | 12485/37962 [28:56:31<57:48:53,  8.17s/it]
 33%|███▎      | 12486/37962 [28:56:39<57:36:09,  8.14s/it]
 33%|███▎      | 12487/37962 [28:56:47<58:06:52,  8.21s/it]
 33%|███▎      | 12488/37962 [28:56:56<58:04:42,  8.21s/it]
 33%|███▎      | 12489/37962 [28:57:04<58:14:31,  8.23s/it]
 33%|███▎      | 12490/37962 [28:57:12<58:28:03,  8.26s/it]
                                                           
{'loss': 1.3712, 'grad_norm': 29.842636108398438, 'learning_rate': 1.5666001629255655e-05, 'epoch': 0.33}

 33%|███▎      | 12490/37962 [28:57:12<58:28:03,  8.26s/it]
 33%|███▎      | 12491/37962 [28:57:20<58:02:19,  8.20s/it]
 33%|███▎      | 12492/37962 [28:57:28<58:10:57,  8.22s/it]
 33%|███▎      | 12493/37962 [28:57:37<58:20:31,  8.25s/it]
 33%|███▎      | 12494/37962 [28:57:45<58:27:23,  8.26s/it]
 33%|███▎      | 12495/37962 [28:57:53<58:17:23,  8.24s/it]
 33%|███▎      | 12496/37962 [28:58:02<58:33:01,  8.28s/it]
 33%|███▎      | 12497/37962 [28:58:10<58:26:34,  8.26s/it]
 33%|███▎      | 12498/37962 [28:58:18<58:04:38,  8.21s/it]
 33%|███▎      | 12499/37962 [28:58:26<58:15:13,  8.24s/it]
 33%|███▎      | 12500/37962 [28:58:35<58:37:20,  8.29s/it]
                                                           
{'loss': 1.4381, 'grad_norm': 6.33497428894043, 'learning_rate': 1.5658969586869694e-05, 'epoch': 0.33}

 33%|███▎      | 12500/37962 [28:58:35<58:37:20,  8.29s/it]
 33%|███▎      | 12501/37962 [28:58:43<59:16:29,  8.38s/it]
 33%|███▎      | 12502/37962 [28:58:51<58:49:33,  8.32s/it]
 33%|███▎      | 12503/37962 [28:59:00<58:31:48,  8.28s/it]
 33%|███▎      | 12504/37962 [28:59:08<58:35:16,  8.28s/it]
 33%|███▎      | 12505/37962 [28:59:16<58:23:19,  8.26s/it]
 33%|███▎      | 12506/37962 [28:59:24<58:36:32,  8.29s/it]
 33%|███▎      | 12507/37962 [28:59:33<58:31:18,  8.28s/it]
 33%|███▎      | 12508/37962 [28:59:41<58:36:21,  8.29s/it]
 33%|███▎      | 12509/37962 [28:59:49<58:31:41,  8.28s/it]
 33%|███▎      | 12510/37962 [28:59:58<58:25:38,  8.26s/it]
                                                           
{'loss': 1.3508, 'grad_norm': 12.79800033569336, 'learning_rate': 1.5651933425417804e-05, 'epoch': 0.33}

 33%|███▎      | 12510/37962 [28:59:58<58:25:38,  8.26s/it]
 33%|███▎      | 12511/37962 [29:00:06<58:28:00,  8.27s/it]
 33%|███▎      | 12512/37962 [29:00:14<58:36:28,  8.29s/it]
 33%|███▎      | 12513/37962 [29:00:22<57:45:34,  8.17s/it]
 33%|███▎      | 12514/37962 [29:00:30<58:00:15,  8.21s/it]
 33%|███▎      | 12515/37962 [29:00:39<58:41:51,  8.30s/it]
 33%|███▎      | 12516/37962 [29:00:47<57:31:09,  8.14s/it]
 33%|███▎      | 12517/37962 [29:00:54<56:56:59,  8.06s/it]
 33%|███▎      | 12518/37962 [29:01:02<56:30:11,  7.99s/it]
 33%|███▎      | 12519/37962 [29:01:10<56:13:10,  7.95s/it]
 33%|███▎      | 12520/37962 [29:01:18<56:40:53,  8.02s/it]
                                                           
{'loss': 1.4031, 'grad_norm': 7.985075950622559, 'learning_rate': 1.564489315002149e-05, 'epoch': 0.33}

 33%|███▎      | 12520/37962 [29:01:18<56:40:53,  8.02s/it]
 33%|███▎      | 12521/37962 [29:01:27<57:12:26,  8.10s/it]
 33%|███▎      | 12522/37962 [29:01:35<57:48:02,  8.18s/it]
 33%|███▎      | 12523/37962 [29:01:43<57:59:44,  8.21s/it]
 33%|███▎      | 12524/37962 [29:01:52<58:26:06,  8.27s/it]
 33%|███▎      | 12525/37962 [29:02:00<58:31:34,  8.28s/it]
 33%|███▎      | 12526/37962 [29:02:08<58:54:36,  8.34s/it]
 33%|███▎      | 12527/37962 [29:02:17<58:47:55,  8.32s/it]
 33%|███▎      | 12528/37962 [29:02:25<58:44:04,  8.31s/it]
 33%|███▎      | 12529/37962 [29:02:34<59:21:18,  8.40s/it]
 33%|███▎      | 12530/37962 [29:02:42<59:05:03,  8.36s/it]
                                                           
{'loss': 1.413, 'grad_norm': 5.1234612464904785, 'learning_rate': 1.5637848765805238e-05, 'epoch': 0.33}

 33%|███▎      | 12530/37962 [29:02:42<59:05:03,  8.36s/it]
 33%|███▎      | 12531/37962 [29:02:50<58:59:11,  8.35s/it]
 33%|███▎      | 12532/37962 [29:02:58<58:21:14,  8.26s/it]
 33%|███▎      | 12533/37962 [29:03:07<58:57:43,  8.35s/it]
 33%|███▎      | 12534/37962 [29:03:15<58:49:02,  8.33s/it]
 33%|███▎      | 12535/37962 [29:03:23<58:48:20,  8.33s/it]
 33%|███▎      | 12536/37962 [29:03:32<59:07:43,  8.37s/it]
 33%|███▎      | 12537/37962 [29:03:40<59:00:34,  8.36s/it]
 33%|███▎      | 12538/37962 [29:03:49<59:04:32,  8.37s/it]
 33%|███▎      | 12539/37962 [29:03:56<57:59:13,  8.21s/it]
 33%|███▎      | 12540/37962 [29:04:05<58:18:02,  8.26s/it]
                                                           
{'loss': 1.4298, 'grad_norm': 5.010728359222412, 'learning_rate': 1.5630800277896544e-05, 'epoch': 0.33}

 33%|███▎      | 12540/37962 [29:04:05<58:18:02,  8.26s/it]
 33%|███▎      | 12541/37962 [29:04:13<58:10:36,  8.24s/it]
 33%|███▎      | 12542/37962 [29:04:21<57:37:42,  8.16s/it]
 33%|███▎      | 12543/37962 [29:04:29<58:04:07,  8.22s/it]
 33%|███▎      | 12544/37962 [29:04:37<57:30:50,  8.15s/it]
 33%|███▎      | 12545/37962 [29:04:46<57:32:32,  8.15s/it]
 33%|███▎      | 12546/37962 [29:04:54<57:50:46,  8.19s/it]
 33%|███▎      | 12547/37962 [29:05:02<58:16:23,  8.25s/it]
 33%|███▎      | 12548/37962 [29:05:11<58:47:59,  8.33s/it]
 33%|███▎      | 12549/37962 [29:05:19<58:44:56,  8.32s/it]
 33%|███▎      | 12550/37962 [29:05:27<57:44:51,  8.18s/it]
                                                           
{'loss': 1.3648, 'grad_norm': 7.344033241271973, 'learning_rate': 1.5623747691425872e-05, 'epoch': 0.33}

 33%|███▎      | 12550/37962 [29:05:27<57:44:51,  8.18s/it]
 33%|███▎      | 12551/37962 [29:05:35<57:55:56,  8.21s/it]
 33%|███▎      | 12552/37962 [29:05:43<58:02:36,  8.22s/it]
 33%|███▎      | 12553/37962 [29:05:52<57:51:04,  8.20s/it]
 33%|███▎      | 12554/37962 [29:06:00<58:02:58,  8.22s/it]
 33%|███▎      | 12555/37962 [29:06:08<58:21:14,  8.27s/it]
 33%|███▎      | 12556/37962 [29:06:17<58:38:57,  8.31s/it]
 33%|███▎      | 12557/37962 [29:06:25<59:35:56,  8.45s/it]
 33%|███▎      | 12558/37962 [29:06:34<59:16:15,  8.40s/it]
 33%|███▎      | 12559/37962 [29:06:42<59:02:48,  8.37s/it]
 33%|███▎      | 12560/37962 [29:06:50<58:53:26,  8.35s/it]
                                                           
{'loss': 1.3211, 'grad_norm': 10.601235389709473, 'learning_rate': 1.561669101152668e-05, 'epoch': 0.33}

 33%|███▎      | 12560/37962 [29:06:50<58:53:26,  8.35s/it]
 33%|███▎      | 12561/37962 [29:06:58<58:40:36,  8.32s/it]
 33%|███▎      | 12562/37962 [29:07:07<58:39:43,  8.31s/it]
 33%|███▎      | 12563/37962 [29:07:15<58:17:20,  8.26s/it]
 33%|███▎      | 12564/37962 [29:07:24<58:54:48,  8.35s/it]
 33%|███▎      | 12565/37962 [29:07:32<59:15:29,  8.40s/it]
 33%|███▎      | 12566/37962 [29:07:40<59:10:35,  8.39s/it]
 33%|███▎      | 12567/37962 [29:07:48<58:16:08,  8.26s/it]
 33%|███▎      | 12568/37962 [29:07:57<58:20:38,  8.27s/it]
 33%|███▎      | 12569/37962 [29:08:05<58:24:18,  8.28s/it]
 33%|███▎      | 12570/37962 [29:08:13<58:39:59,  8.32s/it]
                                                           
{'loss': 1.3769, 'grad_norm': 13.062161445617676, 'learning_rate': 1.5609630243335404e-05, 'epoch': 0.33}

 33%|███▎      | 12570/37962 [29:08:13<58:39:59,  8.32s/it]
 33%|███▎      | 12571/37962 [29:08:22<58:40:51,  8.32s/it]
 33%|███▎      | 12572/37962 [29:08:30<58:21:46,  8.28s/it]
 33%|███▎      | 12573/37962 [29:08:38<58:12:37,  8.25s/it]
 33%|███▎      | 12574/37962 [29:08:47<58:38:58,  8.32s/it]
 33%|███▎      | 12575/37962 [29:08:55<58:04:19,  8.23s/it]
 33%|███▎      | 12576/37962 [29:09:03<59:14:42,  8.40s/it]
 33%|███▎      | 12577/37962 [29:09:12<59:51:38,  8.49s/it]
 33%|███▎      | 12578/37962 [29:09:20<59:46:39,  8.48s/it]
 33%|███▎      | 12579/37962 [29:09:29<59:44:11,  8.47s/it]
 33%|███▎      | 12580/37962 [29:09:38<60:32:26,  8.59s/it]
                                                           
{'loss': 1.4366, 'grad_norm': 6.036024570465088, 'learning_rate': 1.560256539199146e-05, 'epoch': 0.33}

 33%|███▎      | 12580/37962 [29:09:38<60:32:26,  8.59s/it]
 33%|███▎      | 12581/37962 [29:09:53<75:22:36, 10.69s/it]
 33%|███▎      | 12582/37962 [29:10:02<70:57:36, 10.07s/it]
 33%|███▎      | 12583/37962 [29:10:10<67:13:36,  9.54s/it]
 33%|███▎      | 12584/37962 [29:10:19<64:42:45,  9.18s/it]
 33%|███▎      | 12585/37962 [29:10:27<62:53:53,  8.92s/it]
 33%|███▎      | 12586/37962 [29:10:35<61:22:12,  8.71s/it]
 33%|███▎      | 12587/37962 [29:10:43<60:12:32,  8.54s/it]
 33%|███▎      | 12588/37962 [29:10:52<59:24:00,  8.43s/it]
 33%|███▎      | 12589/37962 [29:11:00<58:52:07,  8.35s/it]
 33%|███▎      | 12590/37962 [29:11:08<58:44:25,  8.33s/it]
                                                           
{'loss': 1.3641, 'grad_norm': 14.684640884399414, 'learning_rate': 1.5595496462637215e-05, 'epoch': 0.33}

 33%|███▎      | 12590/37962 [29:11:08<58:44:25,  8.33s/it]
 33%|███▎      | 12591/37962 [29:11:16<58:13:35,  8.26s/it]
 33%|███▎      | 12592/37962 [29:11:24<58:18:05,  8.27s/it]
 33%|███▎      | 12593/37962 [29:11:33<58:18:31,  8.27s/it]
 33%|███▎      | 12594/37962 [29:11:41<57:37:02,  8.18s/it]
 33%|███▎      | 12595/37962 [29:11:49<57:42:27,  8.19s/it]
 33%|███▎      | 12596/37962 [29:11:57<57:30:07,  8.16s/it]
 33%|███▎      | 12597/37962 [29:12:05<58:12:11,  8.26s/it]
 33%|███▎      | 12598/37962 [29:12:14<57:55:27,  8.22s/it]
 33%|███▎      | 12599/37962 [29:12:22<58:09:51,  8.26s/it]
 33%|███▎      | 12600/37962 [29:12:30<58:19:45,  8.28s/it]
                                                           
{'loss': 1.3579, 'grad_norm': 7.113190650939941, 'learning_rate': 1.5588423460418038e-05, 'epoch': 0.33}

 33%|███▎      | 12600/37962 [29:12:30<58:19:45,  8.28s/it]
 33%|███▎      | 12601/37962 [29:12:39<58:40:20,  8.33s/it]
 33%|███▎      | 12602/37962 [29:12:47<58:19:47,  8.28s/it]
 33%|███▎      | 12603/37962 [29:12:55<57:58:13,  8.23s/it]
 33%|███▎      | 12604/37962 [29:13:03<58:11:11,  8.26s/it]
 33%|███▎      | 12605/37962 [29:13:11<58:08:08,  8.25s/it]
 33%|███▎      | 12606/37962 [29:13:20<57:56:13,  8.23s/it]
 33%|███▎      | 12607/37962 [29:13:28<57:23:40,  8.15s/it]
 33%|███▎      | 12608/37962 [29:13:36<57:51:09,  8.21s/it]
 33%|███▎      | 12609/37962 [29:13:44<57:55:07,  8.22s/it]
 33%|███▎      | 12610/37962 [29:13:52<57:55:48,  8.23s/it]
                                                           
{'loss': 1.5132, 'grad_norm': 8.90922737121582, 'learning_rate': 1.5581346390482228e-05, 'epoch': 0.33}

 33%|███▎      | 12610/37962 [29:13:52<57:55:48,  8.23s/it]
 33%|███▎      | 12611/37962 [29:14:01<58:14:58,  8.27s/it]
 33%|███▎      | 12612/37962 [29:14:09<58:18:18,  8.28s/it]
 33%|███▎      | 12613/37962 [29:14:17<58:00:57,  8.24s/it]
 33%|███▎      | 12614/37962 [29:14:26<58:18:05,  8.28s/it]
 33%|███▎      | 12615/37962 [29:14:34<58:19:13,  8.28s/it]
 33%|███▎      | 12616/37962 [29:14:42<57:21:52,  8.15s/it]
 33%|███▎      | 12617/37962 [29:14:50<57:56:54,  8.23s/it]
 33%|███▎      | 12618/37962 [29:14:59<58:19:28,  8.28s/it]
 33%|███▎      | 12619/37962 [29:15:07<58:12:49,  8.27s/it]
 33%|███▎      | 12620/37962 [29:15:15<58:16:07,  8.28s/it]
                                                           
{'loss': 1.284, 'grad_norm': 8.317419052124023, 'learning_rate': 1.5574265257981077e-05, 'epoch': 0.33}

 33%|███▎      | 12620/37962 [29:15:15<58:16:07,  8.28s/it]
 33%|███▎      | 12621/37962 [29:15:23<58:20:12,  8.29s/it]
 33%|███▎      | 12622/37962 [29:15:32<58:22:04,  8.29s/it]
 33%|███▎      | 12623/37962 [29:15:40<58:43:06,  8.34s/it]
 33%|███▎      | 12624/37962 [29:15:49<58:49:26,  8.36s/it]
 33%|███▎      | 12625/37962 [29:15:57<59:29:52,  8.45s/it]
 33%|███▎      | 12626/37962 [29:16:06<59:09:33,  8.41s/it]
 33%|███▎      | 12627/37962 [29:16:14<59:13:44,  8.42s/it]
 33%|███▎      | 12628/37962 [29:16:22<58:47:19,  8.35s/it]
 33%|███▎      | 12629/37962 [29:16:30<58:10:17,  8.27s/it]
 33%|███▎      | 12630/37962 [29:16:39<58:57:10,  8.38s/it]
                                                           
{'loss': 1.4225, 'grad_norm': 5.407451629638672, 'learning_rate': 1.5567180068068812e-05, 'epoch': 0.33}

 33%|███▎      | 12630/37962 [29:16:39<58:57:10,  8.38s/it]
 33%|███▎      | 12631/37962 [29:16:46<56:57:31,  8.09s/it]
 33%|███▎      | 12632/37962 [29:16:54<57:00:53,  8.10s/it]
 33%|███▎      | 12633/37962 [29:17:03<57:35:14,  8.18s/it]
 33%|███▎      | 12634/37962 [29:17:11<58:07:53,  8.26s/it]
 33%|███▎      | 12635/37962 [29:17:20<58:04:35,  8.26s/it]
 33%|███▎      | 12636/37962 [29:17:28<58:04:29,  8.26s/it]
 33%|███▎      | 12637/37962 [29:17:36<58:01:45,  8.25s/it]
 33%|███▎      | 12638/37962 [29:17:44<58:19:17,  8.29s/it]
 33%|███▎      | 12639/37962 [29:17:52<57:44:06,  8.21s/it]
 33%|███▎      | 12640/37962 [29:18:01<57:48:12,  8.22s/it]
                                                           
{'loss': 1.4021, 'grad_norm': 12.0491361618042, 'learning_rate': 1.556009082590262e-05, 'epoch': 0.33}

 33%|███▎      | 12640/37962 [29:18:01<57:48:12,  8.22s/it]
 33%|███▎      | 12641/37962 [29:18:09<58:22:42,  8.30s/it]
 33%|███▎      | 12642/37962 [29:18:18<58:27:47,  8.31s/it]
 33%|███▎      | 12643/37962 [29:18:26<58:16:06,  8.28s/it]
 33%|███▎      | 12644/37962 [29:18:34<57:29:20,  8.17s/it]
 33%|███▎      | 12645/37962 [29:18:42<57:43:34,  8.21s/it]
 33%|███▎      | 12646/37962 [29:18:50<57:04:38,  8.12s/it]
 33%|███▎      | 12647/37962 [29:18:58<57:12:07,  8.13s/it]
 33%|███▎      | 12648/37962 [29:19:06<57:28:34,  8.17s/it]
 33%|███▎      | 12649/37962 [29:19:14<57:26:30,  8.17s/it]
 33%|███▎      | 12650/37962 [29:19:23<57:29:28,  8.18s/it]
                                                           
{'loss': 1.4251, 'grad_norm': 14.125913619995117, 'learning_rate': 1.5552997536642637e-05, 'epoch': 0.33}

 33%|███▎      | 12650/37962 [29:19:23<57:29:28,  8.18s/it]
 33%|███▎      | 12651/37962 [29:19:31<57:49:47,  8.23s/it]
 33%|███▎      | 12652/37962 [29:19:39<57:15:45,  8.14s/it]
 33%|███▎      | 12653/37962 [29:19:47<56:55:50,  8.10s/it]
 33%|███▎      | 12654/37962 [29:19:55<57:17:44,  8.15s/it]
 33%|███▎      | 12655/37962 [29:20:04<57:52:39,  8.23s/it]
 33%|███▎      | 12656/37962 [29:20:12<58:06:30,  8.27s/it]
 33%|███▎      | 12657/37962 [29:20:20<57:47:47,  8.22s/it]
 33%|███▎      | 12658/37962 [29:20:28<57:58:41,  8.25s/it]
 33%|███▎      | 12659/37962 [29:20:37<58:53:54,  8.38s/it]
 33%|███▎      | 12660/37962 [29:20:45<58:47:27,  8.36s/it]
                                                           
{'loss': 1.3716, 'grad_norm': 6.6380510330200195, 'learning_rate': 1.5545900205451947e-05, 'epoch': 0.33}

 33%|███▎      | 12660/37962 [29:20:45<58:47:27,  8.36s/it]
 33%|███▎      | 12661/37962 [29:20:54<58:23:58,  8.31s/it]
 33%|███▎      | 12662/37962 [29:21:02<58:32:35,  8.33s/it]
 33%|███▎      | 12663/37962 [29:21:10<58:50:00,  8.37s/it]
 33%|███▎      | 12664/37962 [29:21:19<58:37:57,  8.34s/it]
 33%|███▎      | 12665/37962 [29:21:27<58:25:31,  8.31s/it]
 33%|███▎      | 12666/37962 [29:21:35<58:30:41,  8.33s/it]
 33%|███▎      | 12667/37962 [29:21:44<58:20:44,  8.30s/it]
 33%|███▎      | 12668/37962 [29:21:52<58:12:02,  8.28s/it]
 33%|███▎      | 12669/37962 [29:22:00<58:03:22,  8.26s/it]
 33%|███▎      | 12670/37962 [29:22:08<57:51:32,  8.24s/it]
                                                           
{'loss': 1.3759, 'grad_norm': 18.087665557861328, 'learning_rate': 1.5538798837496568e-05, 'epoch': 0.33}

 33%|███▎      | 12670/37962 [29:22:08<57:51:32,  8.24s/it]
 33%|███▎      | 12671/37962 [29:22:16<57:47:10,  8.23s/it]
 33%|███▎      | 12672/37962 [29:22:25<58:19:12,  8.30s/it]
 33%|███▎      | 12673/37962 [29:22:33<58:17:24,  8.30s/it]
 33%|███▎      | 12674/37962 [29:22:41<58:14:03,  8.29s/it]
 33%|███▎      | 12675/37962 [29:22:50<58:16:34,  8.30s/it]
 33%|███▎      | 12676/37962 [29:22:58<58:56:00,  8.39s/it]
 33%|███▎      | 12677/37962 [29:23:06<58:09:38,  8.28s/it]
 33%|███▎      | 12678/37962 [29:23:15<57:56:48,  8.25s/it]
 33%|███▎      | 12679/37962 [29:23:22<57:03:40,  8.12s/it]
 33%|███▎      | 12680/37962 [29:23:30<56:37:05,  8.06s/it]
                                                           
{'loss': 1.3794, 'grad_norm': 4.536417007446289, 'learning_rate': 1.5531693437945476e-05, 'epoch': 0.33}

 33%|███▎      | 12680/37962 [29:23:30<56:37:05,  8.06s/it]
 33%|███▎      | 12681/37962 [29:23:39<57:02:13,  8.12s/it]
 33%|███▎      | 12682/37962 [29:23:47<57:35:00,  8.20s/it]
 33%|███▎      | 12683/37962 [29:23:55<57:54:27,  8.25s/it]
 33%|███▎      | 12684/37962 [29:24:11<74:33:05, 10.62s/it]
 33%|███▎      | 12685/37962 [29:24:20<69:30:07,  9.90s/it]
 33%|███▎      | 12686/37962 [29:24:28<65:55:43,  9.39s/it]
 33%|███▎      | 12687/37962 [29:24:36<63:49:00,  9.09s/it]
 33%|███▎      | 12688/37962 [29:24:44<61:50:12,  8.81s/it]
 33%|███▎      | 12689/37962 [29:24:53<61:09:27,  8.71s/it]
 33%|███▎      | 12690/37962 [29:25:01<60:31:54,  8.62s/it]
                                                           
{'loss': 1.3874, 'grad_norm': 10.937272071838379, 'learning_rate': 1.5524584011970558e-05, 'epoch': 0.33}

 33%|███▎      | 12690/37962 [29:25:01<60:31:54,  8.62s/it]
 33%|███▎      | 12691/37962 [29:25:10<60:04:24,  8.56s/it]
 33%|███▎      | 12692/37962 [29:25:18<59:28:33,  8.47s/it]
 33%|███▎      | 12693/37962 [29:25:26<59:08:46,  8.43s/it]
 33%|███▎      | 12694/37962 [29:25:35<59:16:30,  8.45s/it]
 33%|███▎      | 12695/37962 [29:25:43<58:48:10,  8.38s/it]
 33%|███▎      | 12696/37962 [29:25:51<58:25:23,  8.32s/it]
 33%|███▎      | 12697/37962 [29:25:59<57:39:57,  8.22s/it]
 33%|███▎      | 12698/37962 [29:26:07<56:50:28,  8.10s/it]
 33%|███▎      | 12699/37962 [29:26:15<57:00:29,  8.12s/it]
 33%|███▎      | 12700/37962 [29:26:24<57:24:16,  8.18s/it]
                                                           
{'loss': 1.284, 'grad_norm': 8.105043411254883, 'learning_rate': 1.5517470564746644e-05, 'epoch': 0.33}

 33%|███▎      | 12700/37962 [29:26:24<57:24:16,  8.18s/it]
 33%|███▎      | 12701/37962 [29:26:32<57:50:14,  8.24s/it]
 33%|███▎      | 12702/37962 [29:26:40<58:02:38,  8.27s/it]
 33%|███▎      | 12703/37962 [29:26:48<57:53:36,  8.25s/it]
 33%|███▎      | 12704/37962 [29:26:57<58:00:10,  8.27s/it]
 33%|███▎      | 12705/37962 [29:27:05<58:30:07,  8.34s/it]
 33%|███▎      | 12706/37962 [29:27:14<58:28:59,  8.34s/it]
 33%|███▎      | 12707/37962 [29:27:22<58:31:28,  8.34s/it]
 33%|███▎      | 12708/37962 [29:27:30<58:22:01,  8.32s/it]
 33%|███▎      | 12709/37962 [29:27:39<58:17:46,  8.31s/it]
 33%|███▎      | 12710/37962 [29:27:47<58:20:11,  8.32s/it]
                                                           
{'loss': 1.3609, 'grad_norm': 8.407843589782715, 'learning_rate': 1.551035310145149e-05, 'epoch': 0.33}

 33%|███▎      | 12710/37962 [29:27:47<58:20:11,  8.32s/it]
 33%|███▎      | 12711/37962 [29:27:55<58:27:47,  8.34s/it]
 33%|███▎      | 12712/37962 [29:28:04<58:35:21,  8.35s/it]
 33%|███▎      | 12713/37962 [29:28:12<58:47:09,  8.38s/it]
 33%|███▎      | 12714/37962 [29:28:20<58:18:48,  8.31s/it]
 33%|███▎      | 12715/37962 [29:28:29<58:18:48,  8.31s/it]
 33%|███▎      | 12716/37962 [29:28:37<58:18:55,  8.32s/it]
 33%|███▎      | 12717/37962 [29:28:45<58:09:37,  8.29s/it]
 34%|███▎      | 12718/37962 [29:28:53<57:37:25,  8.22s/it]
 34%|███▎      | 12719/37962 [29:29:01<57:39:27,  8.22s/it]
 34%|███▎      | 12720/37962 [29:29:10<58:16:23,  8.31s/it]
                                                           
{'loss': 1.4181, 'grad_norm': 18.736358642578125, 'learning_rate': 1.5503231627265772e-05, 'epoch': 0.34}

 34%|███▎      | 12720/37962 [29:29:10<58:16:23,  8.31s/it]
 34%|███▎      | 12721/37962 [29:29:18<57:26:27,  8.19s/it]
 34%|███▎      | 12722/37962 [29:29:26<57:37:48,  8.22s/it]
 34%|███▎      | 12723/37962 [29:29:34<57:42:27,  8.23s/it]
 34%|███▎      | 12724/37962 [29:29:43<57:37:21,  8.22s/it]
 34%|███▎      | 12725/37962 [29:29:51<57:06:49,  8.15s/it]
 34%|███▎      | 12726/37962 [29:29:59<56:56:57,  8.12s/it]
 34%|███▎      | 12727/37962 [29:30:07<56:37:24,  8.08s/it]
 34%|███▎      | 12728/37962 [29:30:15<57:07:48,  8.15s/it]
 34%|███▎      | 12729/37962 [29:30:23<57:12:01,  8.16s/it]
 34%|███▎      | 12730/37962 [29:30:32<57:51:48,  8.26s/it]
                                                           
{'loss': 1.3641, 'grad_norm': 4.751344203948975, 'learning_rate': 1.5496106147373093e-05, 'epoch': 0.34}

 34%|███▎      | 12730/37962 [29:30:32<57:51:48,  8.26s/it]
 34%|███▎      | 12731/37962 [29:30:40<57:52:08,  8.26s/it]
 34%|███▎      | 12732/37962 [29:30:48<58:05:34,  8.29s/it]
 34%|███▎      | 12733/37962 [29:30:56<57:53:25,  8.26s/it]
 34%|███▎      | 12734/37962 [29:31:05<58:06:10,  8.29s/it]
 34%|███▎      | 12735/37962 [29:31:13<57:59:18,  8.28s/it]
 34%|███▎      | 12736/37962 [29:31:21<57:44:51,  8.24s/it]
 34%|███▎      | 12737/37962 [29:31:29<57:45:15,  8.24s/it]
 34%|███▎      | 12738/37962 [29:31:38<57:39:59,  8.23s/it]
 34%|███▎      | 12739/37962 [29:31:46<57:34:48,  8.22s/it]
 34%|███▎      | 12740/37962 [29:31:54<57:34:48,  8.22s/it]
                                                           
{'loss': 1.403, 'grad_norm': 6.319554328918457, 'learning_rate': 1.548897666695996e-05, 'epoch': 0.34}

 34%|███▎      | 12740/37962 [29:31:54<57:34:48,  8.22s/it]
 34%|███▎      | 12741/37962 [29:32:02<57:54:35,  8.27s/it]
 34%|███▎      | 12742/37962 [29:32:11<57:51:04,  8.26s/it]
 34%|███▎      | 12743/37962 [29:32:19<57:59:33,  8.28s/it]
 34%|███▎      | 12744/37962 [29:32:27<58:10:18,  8.30s/it]
 34%|███▎      | 12745/37962 [29:32:36<58:02:26,  8.29s/it]
 34%|███▎      | 12746/37962 [29:32:44<58:06:03,  8.29s/it]
 34%|███▎      | 12747/37962 [29:32:52<58:08:27,  8.30s/it]
 34%|███▎      | 12748/37962 [29:33:00<58:01:46,  8.29s/it]
 34%|███▎      | 12749/37962 [29:33:09<58:02:17,  8.29s/it]
 34%|███▎      | 12750/37962 [29:33:17<57:58:39,  8.28s/it]
                                                           
{'loss': 1.4101, 'grad_norm': 14.073752403259277, 'learning_rate': 1.5481843191215803e-05, 'epoch': 0.34}

 34%|███▎      | 12750/37962 [29:33:17<57:58:39,  8.28s/it]
 34%|███▎      | 12751/37962 [29:33:25<57:46:53,  8.25s/it]
 34%|███▎      | 12752/37962 [29:33:33<57:30:51,  8.21s/it]
 34%|███▎      | 12753/37962 [29:33:41<57:00:26,  8.14s/it]
 34%|███▎      | 12754/37962 [29:33:49<57:16:02,  8.18s/it]
 34%|███▎      | 12755/37962 [29:33:58<57:16:12,  8.18s/it]
 34%|███▎      | 12756/37962 [29:34:06<57:59:32,  8.28s/it]
 34%|███▎      | 12757/37962 [29:34:15<58:09:48,  8.31s/it]
 34%|███▎      | 12758/37962 [29:34:23<58:02:35,  8.29s/it]
 34%|███▎      | 12759/37962 [29:34:31<58:04:48,  8.30s/it]
 34%|███▎      | 12760/37962 [29:34:39<58:03:30,  8.29s/it]
                                                           
{'loss': 1.2445, 'grad_norm': 5.474279403686523, 'learning_rate': 1.5474705725332954e-05, 'epoch': 0.34}

 34%|███▎      | 12760/37962 [29:34:39<58:03:30,  8.29s/it]
 34%|███▎      | 12761/37962 [29:34:48<57:50:51,  8.26s/it]
 34%|███▎      | 12762/37962 [29:34:56<57:35:06,  8.23s/it]
 34%|███▎      | 12763/37962 [29:35:04<57:32:36,  8.22s/it]
 34%|███▎      | 12764/37962 [29:35:12<57:47:35,  8.26s/it]
 34%|███▎      | 12765/37962 [29:35:21<57:53:09,  8.27s/it]
 34%|███▎      | 12766/37962 [29:35:29<57:47:51,  8.26s/it]
 34%|███▎      | 12767/37962 [29:35:37<57:54:36,  8.27s/it]
 34%|███▎      | 12768/37962 [29:35:46<58:08:56,  8.31s/it]
 34%|███▎      | 12769/37962 [29:35:54<58:41:43,  8.39s/it]
 34%|███▎      | 12770/37962 [29:36:02<58:42:45,  8.39s/it]
                                                           
{'loss': 1.4834, 'grad_norm': 10.729292869567871, 'learning_rate': 1.5467564274506647e-05, 'epoch': 0.34}

 34%|███▎      | 12770/37962 [29:36:02<58:42:45,  8.39s/it]
 34%|███▎      | 12771/37962 [29:36:11<58:20:31,  8.34s/it]
 34%|███▎      | 12772/37962 [29:36:19<57:38:23,  8.24s/it]
 34%|███▎      | 12773/37962 [29:36:27<57:49:05,  8.26s/it]
 34%|███▎      | 12774/37962 [29:36:35<57:36:32,  8.23s/it]
 34%|███▎      | 12775/37962 [29:36:43<57:34:02,  8.23s/it]
 34%|███▎      | 12776/37962 [29:36:52<57:50:24,  8.27s/it]
 34%|███▎      | 12777/37962 [29:37:00<58:16:41,  8.33s/it]
 34%|███▎      | 12778/37962 [29:37:08<58:02:29,  8.30s/it]
 34%|███▎      | 12779/37962 [29:37:16<57:26:45,  8.21s/it]
 34%|███▎      | 12780/37962 [29:37:25<57:30:02,  8.22s/it]
                                                           
{'loss': 1.3707, 'grad_norm': 15.940821647644043, 'learning_rate': 1.5460418843935027e-05, 'epoch': 0.34}

 34%|███▎      | 12780/37962 [29:37:25<57:30:02,  8.22s/it]
 34%|███▎      | 12781/37962 [29:37:33<57:15:22,  8.19s/it]
 34%|███▎      | 12782/37962 [29:37:41<57:36:00,  8.24s/it]
 34%|███▎      | 12783/37962 [29:37:49<57:33:30,  8.23s/it]
 34%|███▎      | 12784/37962 [29:37:58<57:27:03,  8.21s/it]
 34%|███▎      | 12785/37962 [29:38:06<57:45:50,  8.26s/it]
 34%|███▎      | 12786/37962 [29:38:14<57:26:26,  8.21s/it]
 34%|███▎      | 12787/37962 [29:38:22<57:24:41,  8.21s/it]
 34%|███▎      | 12788/37962 [29:38:31<58:22:55,  8.35s/it]
 34%|███▎      | 12789/37962 [29:38:39<58:11:28,  8.32s/it]
 34%|███▎      | 12790/37962 [29:38:48<58:22:24,  8.35s/it]
                                                           
{'loss': 1.3665, 'grad_norm': 9.723480224609375, 'learning_rate': 1.5453269438819125e-05, 'epoch': 0.34}

 34%|███▎      | 12790/37962 [29:38:48<58:22:24,  8.35s/it]
 34%|███▎      | 12791/37962 [29:38:56<58:15:35,  8.33s/it]
 34%|███▎      | 12792/37962 [29:39:04<58:25:59,  8.36s/it]
 34%|███▎      | 12793/37962 [29:39:13<58:08:18,  8.32s/it]
 34%|███▎      | 12794/37962 [29:39:20<57:13:34,  8.19s/it]
 34%|███▎      | 12795/37962 [29:39:29<57:18:21,  8.20s/it]
 34%|███▎      | 12796/37962 [29:39:37<57:40:28,  8.25s/it]
 34%|███▎      | 12797/37962 [29:39:45<57:30:39,  8.23s/it]
 34%|███▎      | 12798/37962 [29:39:54<57:59:45,  8.30s/it]
 34%|███▎      | 12799/37962 [29:40:02<57:34:10,  8.24s/it]
 34%|███▎      | 12800/37962 [29:40:10<58:00:35,  8.30s/it]
                                                           
{'loss': 1.5055, 'grad_norm': 9.150084495544434, 'learning_rate': 1.5446116064362867e-05, 'epoch': 0.34}

 34%|███▎      | 12800/37962 [29:40:10<58:00:35,  8.30s/it]
 34%|███▎      | 12801/37962 [29:40:18<57:40:55,  8.25s/it]
 34%|███▎      | 12802/37962 [29:40:27<58:05:52,  8.31s/it]
 34%|███▎      | 12803/37962 [29:40:35<56:59:16,  8.15s/it]
 34%|███▎      | 12804/37962 [29:40:43<57:20:39,  8.21s/it]
 34%|███▎      | 12805/37962 [29:40:51<57:23:39,  8.21s/it]
 34%|███▎      | 12806/37962 [29:40:59<57:37:54,  8.25s/it]
 34%|███▎      | 12807/37962 [29:41:08<57:35:29,  8.24s/it]
 34%|███▎      | 12808/37962 [29:41:16<57:45:45,  8.27s/it]
 34%|███▎      | 12809/37962 [29:41:24<57:47:55,  8.27s/it]
 34%|███▎      | 12810/37962 [29:41:33<57:49:31,  8.28s/it]
                                                           
{'loss': 1.3847, 'grad_norm': 13.373900413513184, 'learning_rate': 1.5438958725773077e-05, 'epoch': 0.34}

 34%|███▎      | 12810/37962 [29:41:33<57:49:31,  8.28s/it]
 34%|███▎      | 12811/37962 [29:41:41<57:20:43,  8.21s/it]
 34%|███▎      | 12812/37962 [29:41:49<57:20:24,  8.21s/it]
 34%|███▍      | 12813/37962 [29:41:57<57:21:01,  8.21s/it]
 34%|███▍      | 12814/37962 [29:42:05<57:26:58,  8.22s/it]
 34%|███▍      | 12815/37962 [29:42:14<57:37:07,  8.25s/it]
 34%|███▍      | 12816/37962 [29:42:22<57:32:01,  8.24s/it]
 34%|███▍      | 12817/37962 [29:42:30<57:45:42,  8.27s/it]
 34%|███▍      | 12818/37962 [29:42:39<58:07:49,  8.32s/it]
 34%|███▍      | 12819/37962 [29:42:47<57:59:49,  8.30s/it]
 34%|███▍      | 12820/37962 [29:42:55<57:51:52,  8.29s/it]
                                                           
{'loss': 1.3438, 'grad_norm': 11.6533784866333, 'learning_rate': 1.543179742825945e-05, 'epoch': 0.34}

 34%|███▍      | 12820/37962 [29:42:55<57:51:52,  8.29s/it]
 34%|███▍      | 12821/37962 [29:43:03<57:07:37,  8.18s/it]
 34%|███▍      | 12822/37962 [29:43:11<56:42:16,  8.12s/it]
 34%|███▍      | 12823/37962 [29:43:19<57:14:32,  8.20s/it]
 34%|███▍      | 12824/37962 [29:43:27<56:59:09,  8.16s/it]
 34%|███▍      | 12825/37962 [29:43:36<57:15:07,  8.20s/it]
 34%|███▍      | 12826/37962 [29:43:44<57:23:51,  8.22s/it]
 34%|███▍      | 12827/37962 [29:43:52<57:26:43,  8.23s/it]
 34%|███▍      | 12828/37962 [29:44:01<57:27:29,  8.23s/it]
 34%|███▍      | 12829/37962 [29:44:09<57:34:15,  8.25s/it]
 34%|███▍      | 12830/37962 [29:44:17<57:03:39,  8.17s/it]
                                                           
{'loss': 1.3879, 'grad_norm': 2.24631404876709, 'learning_rate': 1.5424632177034577e-05, 'epoch': 0.34}

 34%|███▍      | 12830/37962 [29:44:17<57:03:39,  8.17s/it]
 34%|███▍      | 12831/37962 [29:44:25<57:23:14,  8.22s/it]
 34%|███▍      | 12832/37962 [29:44:33<57:29:19,  8.24s/it]
 34%|███▍      | 12833/37962 [29:44:42<57:30:45,  8.24s/it]
 34%|███▍      | 12834/37962 [29:44:50<57:26:58,  8.23s/it]
 34%|███▍      | 12835/37962 [29:44:58<58:09:48,  8.33s/it]
 34%|███▍      | 12836/37962 [29:45:07<58:12:11,  8.34s/it]
 34%|███▍      | 12837/37962 [29:45:15<57:24:37,  8.23s/it]
 34%|███▍      | 12838/37962 [29:45:23<56:30:07,  8.10s/it]
 34%|███▍      | 12839/37962 [29:45:31<56:13:57,  8.06s/it]
 34%|███▍      | 12840/37962 [29:45:39<56:33:03,  8.10s/it]
                                                           
{'loss': 1.3269, 'grad_norm': 4.449690341949463, 'learning_rate': 1.5417462977313918e-05, 'epoch': 0.34}

 34%|███▍      | 12840/37962 [29:45:39<56:33:03,  8.10s/it]
 34%|███▍      | 12841/37962 [29:45:47<56:43:01,  8.13s/it]
 34%|███▍      | 12842/37962 [29:45:55<56:54:43,  8.16s/it]
 34%|███▍      | 12843/37962 [29:46:04<57:24:32,  8.23s/it]
 34%|███▍      | 12844/37962 [29:46:12<57:36:53,  8.26s/it]
 34%|███▍      | 12845/37962 [29:46:20<57:39:09,  8.26s/it]
 34%|███▍      | 12846/37962 [29:46:28<56:41:37,  8.13s/it]
 34%|███▍      | 12847/37962 [29:46:36<57:01:22,  8.17s/it]
 34%|███▍      | 12848/37962 [29:46:45<57:30:33,  8.24s/it]
 34%|███▍      | 12849/37962 [29:46:53<57:53:19,  8.30s/it]
 34%|███▍      | 12850/37962 [29:47:01<57:40:52,  8.27s/it]
                                                           
{'loss': 1.306, 'grad_norm': 3.368565559387207, 'learning_rate': 1.5410289834315812e-05, 'epoch': 0.34}

 34%|███▍      | 12850/37962 [29:47:01<57:40:52,  8.27s/it]
 34%|███▍      | 12851/37962 [29:47:10<57:57:02,  8.31s/it]
 34%|███▍      | 12852/37962 [29:47:18<58:08:45,  8.34s/it]
 34%|███▍      | 12853/37962 [29:47:26<57:59:32,  8.31s/it]
 34%|███▍      | 12854/37962 [29:47:35<58:27:29,  8.38s/it]
 34%|███▍      | 12855/37962 [29:47:43<58:16:46,  8.36s/it]
 34%|███▍      | 12856/37962 [29:47:51<57:41:53,  8.27s/it]
 34%|███▍      | 12857/37962 [29:48:00<58:01:09,  8.32s/it]
 34%|███▍      | 12858/37962 [29:48:08<58:15:12,  8.35s/it]
 34%|███▍      | 12859/37962 [29:48:16<58:17:01,  8.36s/it]
 34%|███▍      | 12860/37962 [29:48:25<58:22:55,  8.37s/it]
                                                           
{'loss': 1.4553, 'grad_norm': 5.641608715057373, 'learning_rate': 1.5403112753261458e-05, 'epoch': 0.34}

 34%|███▍      | 12860/37962 [29:48:25<58:22:55,  8.37s/it]
 34%|███▍      | 12861/37962 [29:48:33<58:45:01,  8.43s/it]
 34%|███▍      | 12862/37962 [29:48:42<58:49:10,  8.44s/it]
 34%|███▍      | 12863/37962 [29:48:50<59:07:57,  8.48s/it]
 34%|███▍      | 12864/37962 [29:48:59<58:46:29,  8.43s/it]
 34%|███▍      | 12865/37962 [29:49:07<58:58:54,  8.46s/it]
 34%|███▍      | 12866/37962 [29:49:16<58:25:18,  8.38s/it]
 34%|███▍      | 12867/37962 [29:49:24<58:00:45,  8.32s/it]
 34%|███▍      | 12868/37962 [29:49:32<56:59:49,  8.18s/it]
 34%|███▍      | 12869/37962 [29:49:40<57:08:46,  8.20s/it]
 34%|███▍      | 12870/37962 [29:49:48<57:16:08,  8.22s/it]
                                                           
{'loss': 1.3824, 'grad_norm': 11.098250389099121, 'learning_rate': 1.5395931739374935e-05, 'epoch': 0.34}

 34%|███▍      | 12870/37962 [29:49:48<57:16:08,  8.22s/it]
 34%|███▍      | 12871/37962 [29:49:56<57:06:49,  8.19s/it]
 34%|███▍      | 12872/37962 [29:50:04<57:15:34,  8.22s/it]
 34%|███▍      | 12873/37962 [29:50:13<57:26:32,  8.24s/it]
 34%|███▍      | 12874/37962 [29:50:21<57:40:39,  8.28s/it]
 34%|███▍      | 12875/37962 [29:50:29<57:37:03,  8.27s/it]
 34%|███▍      | 12876/37962 [29:50:38<58:01:15,  8.33s/it]
 34%|███▍      | 12877/37962 [29:50:46<57:26:50,  8.24s/it]
 34%|███▍      | 12878/37962 [29:50:54<57:40:54,  8.28s/it]
 34%|███▍      | 12879/37962 [29:51:03<57:47:47,  8.30s/it]
 34%|███▍      | 12880/37962 [29:51:10<56:44:48,  8.14s/it]
                                                           
{'loss': 1.4015, 'grad_norm': 3.5989723205566406, 'learning_rate': 1.5388746797883172e-05, 'epoch': 0.34}

 34%|███▍      | 12880/37962 [29:51:10<56:44:48,  8.14s/it]
 34%|███▍      | 12881/37962 [29:51:19<57:00:43,  8.18s/it]
 34%|███▍      | 12882/37962 [29:51:27<56:37:27,  8.13s/it]
 34%|███▍      | 12883/37962 [29:51:35<57:31:34,  8.26s/it]
 34%|███▍      | 12884/37962 [29:51:44<57:45:12,  8.29s/it]
 34%|███▍      | 12885/37962 [29:51:52<57:47:11,  8.30s/it]
 34%|███▍      | 12886/37962 [29:52:00<57:36:38,  8.27s/it]
 34%|███▍      | 12887/37962 [29:52:08<57:37:40,  8.27s/it]
 34%|███▍      | 12888/37962 [29:52:17<57:42:54,  8.29s/it]
 34%|███▍      | 12889/37962 [29:52:25<57:43:27,  8.29s/it]
 34%|███▍      | 12890/37962 [29:52:33<57:53:17,  8.31s/it]
                                                           
{'loss': 1.3922, 'grad_norm': 2.5168631076812744, 'learning_rate': 1.5381557934015975e-05, 'epoch': 0.34}

 34%|███▍      | 12890/37962 [29:52:33<57:53:17,  8.31s/it]
 34%|███▍      | 12891/37962 [29:52:42<58:00:00,  8.33s/it]
 34%|███▍      | 12892/37962 [29:52:50<58:00:05,  8.33s/it]
 34%|███▍      | 12893/37962 [29:52:59<58:50:10,  8.45s/it]
 34%|███▍      | 12894/37962 [29:53:07<58:23:54,  8.39s/it]
 34%|███▍      | 12895/37962 [29:53:15<58:35:10,  8.41s/it]
 34%|███▍      | 12896/37962 [29:53:24<58:33:28,  8.41s/it]
 34%|███▍      | 12897/37962 [29:53:32<58:21:18,  8.38s/it]
 34%|███▍      | 12898/37962 [29:53:40<57:46:16,  8.30s/it]
 34%|███▍      | 12899/37962 [29:53:49<57:41:20,  8.29s/it]
 34%|███▍      | 12900/37962 [29:53:57<57:31:37,  8.26s/it]
                                                           
{'loss': 1.3922, 'grad_norm': 10.570174217224121, 'learning_rate': 1.5374365153005982e-05, 'epoch': 0.34}

 34%|███▍      | 12900/37962 [29:53:57<57:31:37,  8.26s/it]
 34%|███▍      | 12901/37962 [29:54:05<57:46:37,  8.30s/it]
 34%|███▍      | 12902/37962 [29:54:13<57:47:49,  8.30s/it]
 34%|███▍      | 12903/37962 [29:54:22<57:42:32,  8.29s/it]
 34%|███▍      | 12904/37962 [29:54:30<57:46:50,  8.30s/it]
 34%|███▍      | 12905/37962 [29:54:38<58:05:01,  8.35s/it]
 34%|███▍      | 12906/37962 [29:54:47<57:31:34,  8.27s/it]
 34%|███▍      | 12907/37962 [29:54:55<57:47:28,  8.30s/it]
 34%|███▍      | 12908/37962 [29:55:03<57:53:01,  8.32s/it]
 34%|███▍      | 12909/37962 [29:55:12<57:42:36,  8.29s/it]
 34%|███▍      | 12910/37962 [29:55:20<57:51:24,  8.31s/it]
                                                           
{'loss': 1.4506, 'grad_norm': 11.620294570922852, 'learning_rate': 1.5367168460088703e-05, 'epoch': 0.34}

 34%|███▍      | 12910/37962 [29:55:20<57:51:24,  8.31s/it]
 34%|███▍      | 12911/37962 [29:55:28<57:48:12,  8.31s/it]
 34%|███▍      | 12912/37962 [29:55:37<58:00:06,  8.34s/it]
 34%|███▍      | 12913/37962 [29:55:45<57:46:35,  8.30s/it]
 34%|███▍      | 12914/37962 [29:55:53<57:30:14,  8.26s/it]
 34%|███▍      | 12915/37962 [29:56:01<57:52:16,  8.32s/it]
 34%|███▍      | 12916/37962 [29:56:10<58:00:12,  8.34s/it]
 34%|███▍      | 12917/37962 [29:56:18<57:27:00,  8.26s/it]
 34%|███▍      | 12918/37962 [29:56:26<57:16:58,  8.23s/it]
 34%|███▍      | 12919/37962 [29:56:34<57:30:31,  8.27s/it]
 34%|███▍      | 12920/37962 [29:56:43<57:49:09,  8.31s/it]
                                                           
{'loss': 1.468, 'grad_norm': 17.21299171447754, 'learning_rate': 1.5359967860502484e-05, 'epoch': 0.34}

 34%|███▍      | 12920/37962 [29:56:43<57:49:09,  8.31s/it]
 34%|███▍      | 12921/37962 [29:56:51<58:00:45,  8.34s/it]
 34%|███▍      | 12922/37962 [29:56:59<57:23:51,  8.25s/it]
 34%|███▍      | 12923/37962 [29:57:08<57:37:42,  8.29s/it]
 34%|███▍      | 12924/37962 [29:57:16<57:12:50,  8.23s/it]
 34%|███▍      | 12925/37962 [29:57:24<57:19:25,  8.24s/it]
 34%|███▍      | 12926/37962 [29:57:32<57:46:24,  8.31s/it]
 34%|███▍      | 12927/37962 [29:57:41<57:22:06,  8.25s/it]
 34%|███▍      | 12928/37962 [29:57:49<57:38:24,  8.29s/it]
 34%|███▍      | 12929/37962 [29:57:57<57:16:06,  8.24s/it]
 34%|███▍      | 12930/37962 [29:58:05<57:13:06,  8.23s/it]
                                                           
{'loss': 1.4487, 'grad_norm': 6.778715133666992, 'learning_rate': 1.5352763359488506e-05, 'epoch': 0.34}

 34%|███▍      | 12930/37962 [29:58:05<57:13:06,  8.23s/it]
 34%|███▍      | 12931/37962 [29:58:13<57:07:04,  8.21s/it]
 34%|███▍      | 12932/37962 [29:58:22<57:31:25,  8.27s/it]
 34%|███▍      | 12933/37962 [29:58:30<57:05:41,  8.21s/it]
 34%|███▍      | 12934/37962 [29:58:38<57:20:42,  8.25s/it]
 34%|███▍      | 12935/37962 [29:58:46<56:21:22,  8.11s/it]
 34%|███▍      | 12936/37962 [29:58:54<56:15:03,  8.09s/it]
 34%|███▍      | 12937/37962 [29:59:03<57:01:37,  8.20s/it]
 34%|███▍      | 12938/37962 [29:59:11<56:58:33,  8.20s/it]
 34%|███▍      | 12939/37962 [29:59:19<57:06:51,  8.22s/it]
 34%|███▍      | 12940/37962 [29:59:27<57:19:02,  8.25s/it]
                                                           
{'loss': 1.4136, 'grad_norm': 15.515448570251465, 'learning_rate': 1.5345554962290816e-05, 'epoch': 0.34}

 34%|███▍      | 12940/37962 [29:59:27<57:19:02,  8.25s/it]
 34%|███▍      | 12941/37962 [29:59:36<57:32:39,  8.28s/it]
 34%|███▍      | 12942/37962 [29:59:44<57:23:43,  8.26s/it]
 34%|███▍      | 12943/37962 [29:59:52<57:18:49,  8.25s/it]
 34%|███▍      | 12944/37962 [30:00:01<57:32:17,  8.28s/it]
 34%|███▍      | 12945/37962 [30:00:09<57:46:19,  8.31s/it]
 34%|███▍      | 12946/37962 [30:00:17<57:30:45,  8.28s/it]
 34%|███▍      | 12947/37962 [30:00:25<57:18:34,  8.25s/it]
 34%|███▍      | 12948/37962 [30:00:33<56:44:57,  8.17s/it]
 34%|███▍      | 12949/37962 [30:00:41<56:53:23,  8.19s/it]
 34%|███▍      | 12950/37962 [30:00:50<56:52:37,  8.19s/it]
                                                           
{'loss': 1.3851, 'grad_norm': 25.95926284790039, 'learning_rate': 1.5338342674156274e-05, 'epoch': 0.34}

 34%|███▍      | 12950/37962 [30:00:50<56:52:37,  8.19s/it]
 34%|███▍      | 12951/37962 [30:00:58<56:16:22,  8.10s/it]
 34%|███▍      | 12952/37962 [30:01:05<55:40:53,  8.01s/it]
 34%|███▍      | 12953/37962 [30:01:14<56:40:46,  8.16s/it]
 34%|███▍      | 12954/37962 [30:01:22<56:50:50,  8.18s/it]
 34%|███▍      | 12955/37962 [30:01:30<56:47:23,  8.18s/it]
 34%|███▍      | 12956/37962 [30:01:39<57:13:07,  8.24s/it]
 34%|███▍      | 12957/37962 [30:01:47<57:07:24,  8.22s/it]
 34%|███▍      | 12958/37962 [30:01:55<56:57:01,  8.20s/it]
 34%|███▍      | 12959/37962 [30:02:03<56:11:02,  8.09s/it]
 34%|███▍      | 12960/37962 [30:02:11<56:21:37,  8.12s/it]
                                                           
{'loss': 1.3657, 'grad_norm': 2.595224380493164, 'learning_rate': 1.5331126500334573e-05, 'epoch': 0.34}

 34%|███▍      | 12960/37962 [30:02:11<56:21:37,  8.12s/it]
 34%|███▍      | 12961/37962 [30:02:19<56:54:07,  8.19s/it]
 34%|███▍      | 12962/37962 [30:02:28<57:03:50,  8.22s/it]
 34%|███▍      | 12963/37962 [30:02:36<57:08:40,  8.23s/it]
 34%|███▍      | 12964/37962 [30:02:44<57:39:53,  8.30s/it]
 34%|███▍      | 12965/37962 [30:02:52<57:01:43,  8.21s/it]
 34%|███▍      | 12966/37962 [30:03:01<56:59:00,  8.21s/it]
 34%|███▍      | 12967/37962 [30:03:09<57:08:47,  8.23s/it]
 34%|███▍      | 12968/37962 [30:03:17<57:01:25,  8.21s/it]
 34%|███▍      | 12969/37962 [30:03:25<57:21:10,  8.26s/it]
 34%|███▍      | 12970/37962 [30:03:34<58:00:19,  8.36s/it]
                                                           
{'loss': 1.4002, 'grad_norm': 5.58395528793335, 'learning_rate': 1.5323906446078254e-05, 'epoch': 0.34}

 34%|███▍      | 12970/37962 [30:03:34<58:00:19,  8.36s/it]
 34%|███▍      | 12971/37962 [30:03:42<57:37:10,  8.30s/it]
 34%|███▍      | 12972/37962 [30:03:51<57:45:14,  8.32s/it]
 34%|███▍      | 12973/37962 [30:03:59<57:41:02,  8.31s/it]
 34%|███▍      | 12974/37962 [30:04:07<58:01:28,  8.36s/it]
 34%|███▍      | 12975/37962 [30:04:16<58:09:38,  8.38s/it]
 34%|███▍      | 12976/37962 [30:04:24<58:00:42,  8.36s/it]
 34%|███▍      | 12977/37962 [30:04:32<57:28:56,  8.28s/it]
 34%|███▍      | 12978/37962 [30:04:41<57:43:56,  8.32s/it]
 34%|███▍      | 12979/37962 [30:04:49<57:32:19,  8.29s/it]
 34%|███▍      | 12980/37962 [30:04:57<57:37:11,  8.30s/it]
                                                           
{'loss': 1.3373, 'grad_norm': 13.379227638244629, 'learning_rate': 1.5316682516642663e-05, 'epoch': 0.34}

 34%|███▍      | 12980/37962 [30:04:57<57:37:11,  8.30s/it]
 34%|███▍      | 12981/37962 [30:05:05<57:39:24,  8.31s/it]
 34%|███▍      | 12982/37962 [30:05:14<57:36:08,  8.30s/it]
 34%|███▍      | 12983/37962 [30:05:22<57:20:11,  8.26s/it]
 34%|███▍      | 12984/37962 [30:05:30<57:35:34,  8.30s/it]
 34%|███▍      | 12985/37962 [30:05:39<57:51:30,  8.34s/it]
 34%|███▍      | 12986/37962 [30:05:47<57:30:38,  8.29s/it]
 34%|███▍      | 12987/37962 [30:05:55<57:38:34,  8.31s/it]
 34%|███▍      | 12988/37962 [30:06:04<57:47:35,  8.33s/it]
 34%|███▍      | 12989/37962 [30:06:12<57:01:12,  8.22s/it]
 34%|███▍      | 12990/37962 [30:06:20<56:33:11,  8.15s/it]
                                                           
{'loss': 1.3197, 'grad_norm': 12.631510734558105, 'learning_rate': 1.5309454717285972e-05, 'epoch': 0.34}

 34%|███▍      | 12990/37962 [30:06:20<56:33:11,  8.15s/it]
 34%|███▍      | 12991/37962 [30:06:28<56:43:18,  8.18s/it]
 34%|███▍      | 12992/37962 [30:06:36<57:07:46,  8.24s/it]
 34%|███▍      | 12993/37962 [30:06:44<56:47:34,  8.19s/it]
 34%|███▍      | 12994/37962 [30:06:52<56:22:56,  8.13s/it]
 34%|███▍      | 12995/37962 [30:07:00<56:23:32,  8.13s/it]
 34%|███▍      | 12996/37962 [30:07:09<56:58:41,  8.22s/it]
 34%|███▍      | 12997/37962 [30:07:17<57:00:33,  8.22s/it]
 34%|███▍      | 12998/37962 [30:07:25<57:07:22,  8.24s/it]
 34%|███▍      | 12999/37962 [30:07:34<57:09:20,  8.24s/it]
 34%|███▍      | 13000/37962 [30:07:42<57:14:10,  8.25s/it]
                                                           
{'loss': 1.3645, 'grad_norm': 5.114025592803955, 'learning_rate': 1.530222305326917e-05, 'epoch': 0.34}

 34%|███▍      | 13000/37962 [30:07:42<57:14:10,  8.25s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 34%|███▍      | 13001/37962 [30:08:31<141:24:25, 20.39s/it]
 34%|███▍      | 13002/37962 [30:08:39<116:25:09, 16.79s/it]
 34%|███▍      | 13003/37962 [30:08:47<98:23:25, 14.19s/it] 
 34%|███▍      | 13004/37962 [30:08:55<86:07:38, 12.42s/it]
 34%|███▍      | 13005/37962 [30:09:04<77:44:24, 11.21s/it]
 34%|███▍      | 13006/37962 [30:09:12<71:45:13, 10.35s/it]
 34%|███▍      | 13007/37962 [30:09:20<67:27:24,  9.73s/it]
 34%|███▍      | 13008/37962 [30:09:28<63:52:57,  9.22s/it]
 34%|███▍      | 13009/37962 [30:09:37<62:36:22,  9.03s/it]
 34%|███▍      | 13010/37962 [30:09:45<60:55:35,  8.79s/it]
                                                           
{'loss': 1.4057, 'grad_norm': 10.444599151611328, 'learning_rate': 1.5294987529856066e-05, 'epoch': 0.34}

 34%|███▍      | 13010/37962 [30:09:45<60:55:35,  8.79s/it]
 34%|███▍      | 13011/37962 [30:09:53<59:25:23,  8.57s/it]
 34%|███▍      | 13012/37962 [30:10:01<58:03:48,  8.38s/it]
 34%|███▍      | 13013/37962 [30:10:09<57:20:54,  8.28s/it]
 34%|███▍      | 13014/37962 [30:10:17<57:09:12,  8.25s/it]
 34%|███▍      | 13015/37962 [30:10:26<57:33:30,  8.31s/it]
 34%|███▍      | 13016/37962 [30:10:34<57:23:15,  8.28s/it]
 34%|███▍      | 13017/37962 [30:10:42<57:32:44,  8.30s/it]
 34%|███▍      | 13018/37962 [30:10:51<57:21:53,  8.28s/it]
 34%|███▍      | 13019/37962 [30:10:59<57:22:19,  8.28s/it]
 34%|███▍      | 13020/37962 [30:11:07<57:40:02,  8.32s/it]
                                                           
{'loss': 1.4123, 'grad_norm': 10.352643013000488, 'learning_rate': 1.528774815231327e-05, 'epoch': 0.34}

 34%|███▍      | 13020/37962 [30:11:07<57:40:02,  8.32s/it]
 34%|███▍      | 13021/37962 [30:11:16<57:20:50,  8.28s/it]
 34%|███▍      | 13022/37962 [30:11:24<57:09:25,  8.25s/it]
 34%|███▍      | 13023/37962 [30:11:32<57:29:18,  8.30s/it]
 34%|███▍      | 13024/37962 [30:11:41<57:58:06,  8.37s/it]
 34%|███▍      | 13025/37962 [30:11:49<57:51:37,  8.35s/it]
 34%|███▍      | 13026/37962 [30:11:57<56:54:13,  8.22s/it]
 34%|███▍      | 13027/37962 [30:12:05<56:23:52,  8.14s/it]
 34%|███▍      | 13028/37962 [30:12:13<56:08:18,  8.11s/it]
 34%|███▍      | 13029/37962 [30:12:21<56:41:38,  8.19s/it]
 34%|███▍      | 13030/37962 [30:12:30<56:54:24,  8.22s/it]
                                                           
{'loss': 1.6236, 'grad_norm': 11.244441032409668, 'learning_rate': 1.5280504925910195e-05, 'epoch': 0.34}

 34%|███▍      | 13030/37962 [30:12:30<56:54:24,  8.22s/it]
 34%|███▍      | 13031/37962 [30:12:38<57:19:55,  8.28s/it]
 34%|███▍      | 13032/37962 [30:12:46<57:43:52,  8.34s/it]
 34%|███▍      | 13033/37962 [30:12:55<57:40:45,  8.33s/it]
 34%|███▍      | 13034/37962 [30:13:03<57:37:18,  8.32s/it]
 34%|███▍      | 13035/37962 [30:13:11<57:22:28,  8.29s/it]
 34%|███▍      | 13036/37962 [30:13:20<57:45:25,  8.34s/it]
 34%|███▍      | 13037/37962 [30:13:28<57:49:25,  8.35s/it]
 34%|███▍      | 13038/37962 [30:13:36<57:54:17,  8.36s/it]
 34%|███▍      | 13039/37962 [30:13:45<57:14:24,  8.27s/it]
 34%|███▍      | 13040/37962 [30:13:53<57:25:21,  8.29s/it]
                                                           
{'loss': 1.3617, 'grad_norm': 13.948497772216797, 'learning_rate': 1.5273257855919062e-05, 'epoch': 0.34}

 34%|███▍      | 13040/37962 [30:13:53<57:25:21,  8.29s/it]
 34%|███▍      | 13041/37962 [30:14:01<57:31:59,  8.31s/it]
 34%|███▍      | 13042/37962 [30:14:09<57:25:09,  8.29s/it]
 34%|███▍      | 13043/37962 [30:14:18<57:28:00,  8.30s/it]
 34%|███▍      | 13044/37962 [30:14:26<57:24:19,  8.29s/it]
 34%|███▍      | 13045/37962 [30:14:35<57:50:06,  8.36s/it]
 34%|███▍      | 13046/37962 [30:14:43<58:01:48,  8.38s/it]
 34%|███▍      | 13047/37962 [30:14:51<57:41:54,  8.34s/it]
 34%|███▍      | 13048/37962 [30:15:00<57:31:13,  8.31s/it]
 34%|███▍      | 13049/37962 [30:15:08<57:29:19,  8.31s/it]
 34%|███▍      | 13050/37962 [30:15:16<57:32:10,  8.31s/it]
                                                           
{'loss': 1.3615, 'grad_norm': 13.996700286865234, 'learning_rate': 1.526600694761489e-05, 'epoch': 0.34}

 34%|███▍      | 13050/37962 [30:15:16<57:32:10,  8.31s/it]
 34%|███▍      | 13051/37962 [30:15:24<56:49:40,  8.21s/it]
 34%|███▍      | 13052/37962 [30:15:32<56:51:52,  8.22s/it]
 34%|███▍      | 13053/37962 [30:15:41<57:18:21,  8.28s/it]
 34%|███▍      | 13054/37962 [30:15:49<57:11:30,  8.27s/it]
 34%|███▍      | 13055/37962 [30:15:57<57:05:01,  8.25s/it]
 34%|███▍      | 13056/37962 [30:16:06<57:53:04,  8.37s/it]
 34%|███▍      | 13057/37962 [30:16:14<58:12:36,  8.41s/it]
 34%|███▍      | 13058/37962 [30:16:23<57:38:38,  8.33s/it]
 34%|███▍      | 13059/37962 [30:16:31<57:49:53,  8.36s/it]
 34%|███▍      | 13060/37962 [30:16:39<57:29:56,  8.31s/it]
                                                           
{'loss': 1.4132, 'grad_norm': 12.247322082519531, 'learning_rate': 1.5258752206275486e-05, 'epoch': 0.34}

 34%|███▍      | 13060/37962 [30:16:39<57:29:56,  8.31s/it]
 34%|███▍      | 13061/37962 [30:16:47<57:28:31,  8.31s/it]
 34%|███▍      | 13062/37962 [30:16:56<57:25:15,  8.30s/it]
 34%|███▍      | 13063/37962 [30:17:04<57:24:54,  8.30s/it]
 34%|███▍      | 13064/37962 [30:17:12<57:25:17,  8.30s/it]
 34%|███▍      | 13065/37962 [30:17:21<57:11:12,  8.27s/it]
 34%|███▍      | 13066/37962 [30:17:29<57:14:19,  8.28s/it]
 34%|███▍      | 13067/37962 [30:17:37<57:10:53,  8.27s/it]
 34%|███▍      | 13068/37962 [30:17:45<57:02:41,  8.25s/it]
 34%|███▍      | 13069/37962 [30:17:54<57:04:00,  8.25s/it]
 34%|███▍      | 13070/37962 [30:18:02<57:08:28,  8.26s/it]
                                                           
{'loss': 1.4402, 'grad_norm': 14.586034774780273, 'learning_rate': 1.5251493637181453e-05, 'epoch': 0.34}

 34%|███▍      | 13070/37962 [30:18:02<57:08:28,  8.26s/it]
 34%|███▍      | 13071/37962 [30:18:10<56:55:06,  8.23s/it]
 34%|███▍      | 13072/37962 [30:18:18<56:51:51,  8.22s/it]
 34%|███▍      | 13073/37962 [30:18:26<56:19:49,  8.15s/it]
 34%|███▍      | 13074/37962 [30:18:35<56:42:28,  8.20s/it]
 34%|███▍      | 13075/37962 [30:18:43<56:49:52,  8.22s/it]
 34%|███▍      | 13076/37962 [30:18:51<56:55:36,  8.24s/it]
 34%|███▍      | 13077/37962 [30:18:59<56:46:43,  8.21s/it]
 34%|███▍      | 13078/37962 [30:19:07<56:41:35,  8.20s/it]
 34%|███▍      | 13079/37962 [30:19:16<57:07:25,  8.26s/it]
 34%|███▍      | 13080/37962 [30:19:24<56:39:24,  8.20s/it]
                                                           
{'loss': 1.4002, 'grad_norm': 10.141186714172363, 'learning_rate': 1.5244231245616182e-05, 'epoch': 0.34}

 34%|███▍      | 13080/37962 [30:19:24<56:39:24,  8.20s/it]
 34%|███▍      | 13081/37962 [30:19:32<56:54:17,  8.23s/it]
 34%|███▍      | 13082/37962 [30:19:41<57:17:18,  8.29s/it]
 34%|███▍      | 13083/37962 [30:19:49<57:31:30,  8.32s/it]
 34%|███▍      | 13084/37962 [30:19:57<57:19:48,  8.30s/it]
 34%|███▍      | 13085/37962 [30:20:05<57:19:24,  8.30s/it]
 34%|███▍      | 13086/37962 [30:20:14<57:14:21,  8.28s/it]
 34%|███▍      | 13087/37962 [30:20:22<56:30:56,  8.18s/it]
 34%|███▍      | 13088/37962 [30:20:30<56:38:59,  8.20s/it]
 34%|███▍      | 13089/37962 [30:20:38<57:18:16,  8.29s/it]
 34%|███▍      | 13090/37962 [30:20:47<57:16:36,  8.29s/it]
                                                           
{'loss': 1.3957, 'grad_norm': 6.6555070877075195, 'learning_rate': 1.5236965036865834e-05, 'epoch': 0.34}

 34%|███▍      | 13090/37962 [30:20:47<57:16:36,  8.29s/it]
 34%|███▍      | 13091/37962 [30:20:55<57:35:46,  8.34s/it]
 34%|███▍      | 13092/37962 [30:21:04<57:51:51,  8.38s/it]
 34%|███▍      | 13093/37962 [30:21:12<56:58:43,  8.25s/it]
 34%|███▍      | 13094/37962 [30:21:20<56:49:19,  8.23s/it]
 34%|███▍      | 13095/37962 [30:21:28<56:58:00,  8.25s/it]
 34%|███▍      | 13096/37962 [30:21:36<57:17:07,  8.29s/it]
 35%|███▍      | 13097/37962 [30:21:45<57:08:53,  8.27s/it]
 35%|███▍      | 13098/37962 [30:21:53<57:05:58,  8.27s/it]
 35%|███▍      | 13099/37962 [30:22:01<57:18:47,  8.30s/it]
 35%|███▍      | 13100/37962 [30:22:10<57:09:19,  8.28s/it]
                                                           
{'loss': 1.3859, 'grad_norm': 2.5723233222961426, 'learning_rate': 1.5229695016219358e-05, 'epoch': 0.35}

 35%|███▍      | 13100/37962 [30:22:10<57:09:19,  8.28s/it]
 35%|███▍      | 13101/37962 [30:22:18<57:22:09,  8.31s/it]
 35%|███▍      | 13102/37962 [30:22:26<57:29:06,  8.32s/it]
 35%|███▍      | 13103/37962 [30:22:35<57:22:51,  8.31s/it]
 35%|███▍      | 13104/37962 [30:22:43<57:16:54,  8.30s/it]
 35%|███▍      | 13105/37962 [30:22:51<56:41:19,  8.21s/it]
 35%|███▍      | 13106/37962 [30:22:59<56:46:50,  8.22s/it]
 35%|███▍      | 13107/37962 [30:23:07<56:55:48,  8.25s/it]
 35%|███▍      | 13108/37962 [30:23:16<57:01:11,  8.26s/it]
 35%|███▍      | 13109/37962 [30:23:24<57:06:04,  8.27s/it]
 35%|███▍      | 13110/37962 [30:23:32<57:31:26,  8.33s/it]
                                                           
{'loss': 1.377, 'grad_norm': 18.0116024017334, 'learning_rate': 1.522242118896848e-05, 'epoch': 0.35}

 35%|███▍      | 13110/37962 [30:23:32<57:31:26,  8.33s/it]
 35%|███▍      | 13111/37962 [30:23:41<57:45:10,  8.37s/it]
 35%|███▍      | 13112/37962 [30:23:49<57:01:10,  8.26s/it]
 35%|███▍      | 13113/37962 [30:23:57<56:54:54,  8.25s/it]
 35%|███▍      | 13114/37962 [30:24:05<56:51:50,  8.24s/it]
 35%|███▍      | 13115/37962 [30:24:13<56:34:53,  8.20s/it]
 35%|███▍      | 13116/37962 [30:24:22<56:45:03,  8.22s/it]
 35%|███▍      | 13117/37962 [30:24:30<56:48:03,  8.23s/it]
 35%|███▍      | 13118/37962 [30:24:39<57:46:18,  8.37s/it]
 35%|███▍      | 13119/37962 [30:24:47<57:28:51,  8.33s/it]
 35%|███▍      | 13120/37962 [30:24:55<57:33:54,  8.34s/it]
                                                           
{'loss': 1.3487, 'grad_norm': 3.1253726482391357, 'learning_rate': 1.5215143560407693e-05, 'epoch': 0.35}

 35%|███▍      | 13120/37962 [30:24:55<57:33:54,  8.34s/it]
 35%|███▍      | 13121/37962 [30:25:04<57:26:49,  8.33s/it]
 35%|███▍      | 13122/37962 [30:25:12<57:13:25,  8.29s/it]
 35%|███▍      | 13123/37962 [30:25:20<57:04:49,  8.27s/it]
 35%|███▍      | 13124/37962 [30:25:28<56:51:14,  8.24s/it]
 35%|███▍      | 13125/37962 [30:25:36<57:01:19,  8.27s/it]
 35%|███▍      | 13126/37962 [30:25:45<57:03:47,  8.27s/it]
 35%|███▍      | 13127/37962 [30:25:53<56:43:32,  8.22s/it]
 35%|███▍      | 13128/37962 [30:26:01<56:45:44,  8.23s/it]
 35%|███▍      | 13129/37962 [30:26:09<57:02:44,  8.27s/it]
 35%|███▍      | 13130/37962 [30:26:18<56:56:34,  8.26s/it]
                                                           
{'loss': 1.4382, 'grad_norm': 4.386977672576904, 'learning_rate': 1.5207862135834252e-05, 'epoch': 0.35}

 35%|███▍      | 13130/37962 [30:26:18<56:56:34,  8.26s/it]
 35%|███▍      | 13131/37962 [30:26:26<56:55:14,  8.25s/it]
 35%|███▍      | 13132/37962 [30:26:34<57:10:25,  8.29s/it]
 35%|███▍      | 13133/37962 [30:26:43<57:20:05,  8.31s/it]
 35%|███▍      | 13134/37962 [30:26:51<57:13:41,  8.30s/it]
 35%|███▍      | 13135/37962 [30:26:59<57:11:29,  8.29s/it]
 35%|███▍      | 13136/37962 [30:27:08<57:23:38,  8.32s/it]
 35%|███▍      | 13137/37962 [30:27:16<57:50:52,  8.39s/it]
 35%|███▍      | 13138/37962 [30:27:24<57:34:11,  8.35s/it]
 35%|███▍      | 13139/37962 [30:27:33<57:08:39,  8.29s/it]
 35%|███▍      | 13140/37962 [30:27:41<57:03:56,  8.28s/it]
                                                           
{'loss': 1.2968, 'grad_norm': 5.462677001953125, 'learning_rate': 1.520057692054818e-05, 'epoch': 0.35}

 35%|███▍      | 13140/37962 [30:27:41<57:03:56,  8.28s/it]
 35%|███▍      | 13141/37962 [30:27:49<57:12:28,  8.30s/it]
 35%|███▍      | 13142/37962 [30:27:57<57:06:46,  8.28s/it]
 35%|███▍      | 13143/37962 [30:28:06<57:15:31,  8.31s/it]
 35%|███▍      | 13144/37962 [30:28:14<56:18:22,  8.17s/it]
 35%|███▍      | 13145/37962 [30:28:22<56:44:57,  8.23s/it]
 35%|███▍      | 13146/37962 [30:28:30<56:59:13,  8.27s/it]
 35%|███▍      | 13147/37962 [30:28:38<56:23:25,  8.18s/it]
 35%|███▍      | 13148/37962 [30:28:47<56:21:16,  8.18s/it]
 35%|███▍      | 13149/37962 [30:28:55<56:36:03,  8.21s/it]
 35%|███▍      | 13150/37962 [30:29:03<56:56:17,  8.26s/it]
                                                           
{'loss': 1.445, 'grad_norm': 12.846744537353516, 'learning_rate': 1.5193287919852267e-05, 'epoch': 0.35}

 35%|███▍      | 13150/37962 [30:29:03<56:56:17,  8.26s/it]
 35%|███▍      | 13151/37962 [30:29:12<57:07:11,  8.29s/it]
 35%|███▍      | 13152/37962 [30:29:20<56:55:12,  8.26s/it]
 35%|███▍      | 13153/37962 [30:29:28<57:04:49,  8.28s/it]
 35%|███▍      | 13154/37962 [30:29:37<57:26:18,  8.34s/it]
 35%|███▍      | 13155/37962 [30:29:45<57:08:52,  8.29s/it]
 35%|███▍      | 13156/37962 [30:29:53<56:51:08,  8.25s/it]
 35%|███▍      | 13157/37962 [30:30:01<57:01:49,  8.28s/it]
 35%|███▍      | 13158/37962 [30:30:10<57:20:04,  8.32s/it]
 35%|███▍      | 13159/37962 [30:30:18<57:04:46,  8.28s/it]
 35%|███▍      | 13160/37962 [30:30:26<56:55:33,  8.26s/it]
                                                           
{'loss': 1.3689, 'grad_norm': 7.607128620147705, 'learning_rate': 1.5185995139052044e-05, 'epoch': 0.35}

 35%|███▍      | 13160/37962 [30:30:26<56:55:33,  8.26s/it]
 35%|███▍      | 13161/37962 [30:30:34<56:35:09,  8.21s/it]
 35%|███▍      | 13162/37962 [30:30:42<56:32:49,  8.21s/it]
 35%|███▍      | 13163/37962 [30:30:50<56:13:48,  8.16s/it]
 35%|███▍      | 13164/37962 [30:30:59<56:39:59,  8.23s/it]
 35%|███▍      | 13165/37962 [30:31:07<56:39:11,  8.22s/it]
 35%|███▍      | 13166/37962 [30:31:15<56:41:40,  8.23s/it]
 35%|███▍      | 13167/37962 [30:31:24<57:19:41,  8.32s/it]
 35%|███▍      | 13168/37962 [30:31:32<56:42:53,  8.23s/it]
 35%|███▍      | 13169/37962 [30:31:40<57:16:24,  8.32s/it]
 35%|███▍      | 13170/37962 [30:31:49<57:32:36,  8.36s/it]
                                                           
{'loss': 1.2586, 'grad_norm': 9.580033302307129, 'learning_rate': 1.5178698583455795e-05, 'epoch': 0.35}

 35%|███▍      | 13170/37962 [30:31:49<57:32:36,  8.36s/it]
 35%|███▍      | 13171/37962 [30:31:57<57:44:53,  8.39s/it]
 35%|███▍      | 13172/37962 [30:32:05<57:22:16,  8.33s/it]
 35%|███▍      | 13173/37962 [30:32:14<56:56:53,  8.27s/it]
 35%|███▍      | 13174/37962 [30:32:22<56:34:26,  8.22s/it]
 35%|███▍      | 13175/37962 [30:32:30<57:08:07,  8.30s/it]
 35%|███▍      | 13176/37962 [30:32:38<57:06:56,  8.30s/it]
 35%|███▍      | 13177/37962 [30:32:47<56:58:15,  8.27s/it]
 35%|███▍      | 13178/37962 [30:32:55<57:14:58,  8.32s/it]
 35%|███▍      | 13179/37962 [30:33:03<56:38:39,  8.23s/it]
 35%|███▍      | 13180/37962 [30:33:11<56:53:38,  8.26s/it]
                                                           
{'loss': 1.369, 'grad_norm': 14.428675651550293, 'learning_rate': 1.5171398258374566e-05, 'epoch': 0.35}

 35%|███▍      | 13180/37962 [30:33:11<56:53:38,  8.26s/it]
 35%|███▍      | 13181/37962 [30:33:20<56:47:35,  8.25s/it]
 35%|███▍      | 13182/37962 [30:33:28<57:14:39,  8.32s/it]
 35%|███▍      | 13183/37962 [30:33:36<57:20:45,  8.33s/it]
 35%|███▍      | 13184/37962 [30:33:45<57:00:04,  8.28s/it]
 35%|███▍      | 13185/37962 [30:33:53<57:13:41,  8.32s/it]
 35%|███▍      | 13186/37962 [30:34:01<56:41:36,  8.24s/it]
 35%|███▍      | 13187/37962 [30:34:09<56:44:45,  8.25s/it]
 35%|███▍      | 13188/37962 [30:34:18<56:59:59,  8.28s/it]
 35%|███▍      | 13189/37962 [30:34:26<57:03:25,  8.29s/it]
 35%|███▍      | 13190/37962 [30:34:34<56:59:16,  8.28s/it]
                                                           
{'loss': 1.4269, 'grad_norm': 14.435626983642578, 'learning_rate': 1.5164094169122132e-05, 'epoch': 0.35}

 35%|███▍      | 13190/37962 [30:34:34<56:59:16,  8.28s/it]
 35%|███▍      | 13191/37962 [30:34:43<56:51:23,  8.26s/it]
 35%|███▍      | 13192/37962 [30:34:51<57:23:37,  8.34s/it]
 35%|███▍      | 13193/37962 [30:34:59<56:51:20,  8.26s/it]
 35%|███▍      | 13194/37962 [30:35:07<56:06:00,  8.15s/it]
 35%|███▍      | 13195/37962 [30:35:15<55:58:44,  8.14s/it]
 35%|███▍      | 13196/37962 [30:35:24<56:37:33,  8.23s/it]
 35%|███▍      | 13197/37962 [30:35:32<56:58:39,  8.28s/it]
 35%|███▍      | 13198/37962 [30:35:40<56:00:30,  8.14s/it]
 35%|███▍      | 13199/37962 [30:35:48<56:07:05,  8.16s/it]
 35%|███▍      | 13200/37962 [30:35:56<56:26:34,  8.21s/it]
                                                           
{'loss': 1.4491, 'grad_norm': 3.1616885662078857, 'learning_rate': 1.5156786321015013e-05, 'epoch': 0.35}

 35%|███▍      | 13200/37962 [30:35:56<56:26:34,  8.21s/it]
 35%|███▍      | 13201/37962 [30:36:05<56:42:45,  8.25s/it]
 35%|███▍      | 13202/37962 [30:36:13<56:28:48,  8.21s/it]
 35%|███▍      | 13203/37962 [30:36:21<56:36:49,  8.23s/it]
 35%|███▍      | 13204/37962 [30:36:29<56:11:18,  8.17s/it]
 35%|███▍      | 13205/37962 [30:36:37<56:08:53,  8.16s/it]
 35%|███▍      | 13206/37962 [30:36:45<55:34:49,  8.08s/it]
 35%|███▍      | 13207/37962 [30:36:54<56:25:10,  8.20s/it]
 35%|███▍      | 13208/37962 [30:37:02<56:45:03,  8.25s/it]
 35%|███▍      | 13209/37962 [30:37:10<56:52:40,  8.27s/it]
 35%|███▍      | 13210/37962 [30:37:19<56:47:21,  8.26s/it]
                                                           
{'loss': 1.2706, 'grad_norm': 12.13348388671875, 'learning_rate': 1.514947471937247e-05, 'epoch': 0.35}

 35%|███▍      | 13210/37962 [30:37:19<56:47:21,  8.26s/it]
 35%|███▍      | 13211/37962 [30:37:27<56:49:59,  8.27s/it]
 35%|███▍      | 13212/37962 [30:37:35<57:11:03,  8.32s/it]
 35%|███▍      | 13213/37962 [30:37:43<56:48:05,  8.26s/it]
 35%|███▍      | 13214/37962 [30:37:52<56:52:11,  8.27s/it]
 35%|███▍      | 13215/37962 [30:38:00<56:40:18,  8.24s/it]
 35%|███▍      | 13216/37962 [30:38:08<56:56:53,  8.28s/it]
 35%|███▍      | 13217/37962 [30:38:16<55:57:55,  8.14s/it]
 35%|███▍      | 13218/37962 [30:38:24<56:20:32,  8.20s/it]
 35%|███▍      | 13219/37962 [30:38:33<56:42:02,  8.25s/it]
 35%|███▍      | 13220/37962 [30:38:41<56:07:07,  8.17s/it]
                                                           
{'loss': 1.3064, 'grad_norm': 6.760093688964844, 'learning_rate': 1.5142159369516483e-05, 'epoch': 0.35}

 35%|███▍      | 13220/37962 [30:38:41<56:07:07,  8.17s/it]
 35%|███▍      | 13221/37962 [30:38:49<56:30:33,  8.22s/it]
 35%|███▍      | 13222/37962 [30:38:57<56:50:29,  8.27s/it]
 35%|███▍      | 13223/37962 [30:39:06<56:58:32,  8.29s/it]
 35%|███▍      | 13224/37962 [30:39:14<57:03:28,  8.30s/it]
 35%|███▍      | 13225/37962 [30:39:22<56:55:31,  8.28s/it]
 35%|███▍      | 13226/37962 [30:39:31<56:43:21,  8.26s/it]
 35%|███▍      | 13227/37962 [30:39:39<56:42:42,  8.25s/it]
 35%|███▍      | 13228/37962 [30:39:47<56:32:12,  8.23s/it]
 35%|███▍      | 13229/37962 [30:39:55<56:38:00,  8.24s/it]
 35%|███▍      | 13230/37962 [30:40:04<57:22:26,  8.35s/it]
                                                           
{'loss': 1.5262, 'grad_norm': 6.532286167144775, 'learning_rate': 1.5134840276771777e-05, 'epoch': 0.35}

 35%|███▍      | 13230/37962 [30:40:04<57:22:26,  8.35s/it]
 35%|███▍      | 13231/37962 [30:40:12<56:24:35,  8.21s/it]
 35%|███▍      | 13232/37962 [30:40:20<56:59:45,  8.30s/it]
 35%|███▍      | 13233/37962 [30:40:28<56:36:23,  8.24s/it]
 35%|███▍      | 13234/37962 [30:40:37<56:27:56,  8.22s/it]
 35%|███▍      | 13235/37962 [30:40:45<56:37:28,  8.24s/it]
 35%|███▍      | 13236/37962 [30:40:53<56:53:05,  8.28s/it]
 35%|███▍      | 13237/37962 [30:41:02<57:13:41,  8.33s/it]
 35%|███▍      | 13238/37962 [30:41:10<56:31:31,  8.23s/it]
 35%|███▍      | 13239/37962 [30:41:18<56:30:02,  8.23s/it]
 35%|███▍      | 13240/37962 [30:41:26<56:39:48,  8.25s/it]
                                                           
{'loss': 1.2846, 'grad_norm': 4.950694561004639, 'learning_rate': 1.5127517446465785e-05, 'epoch': 0.35}

 35%|███▍      | 13240/37962 [30:41:26<56:39:48,  8.25s/it]
 35%|███▍      | 13241/37962 [30:41:35<57:05:28,  8.31s/it]
 35%|███▍      | 13242/37962 [30:41:43<57:07:56,  8.32s/it]
 35%|███▍      | 13243/37962 [30:41:51<56:55:44,  8.29s/it]
 35%|███▍      | 13244/37962 [30:41:59<56:46:29,  8.27s/it]
 35%|███▍      | 13245/37962 [30:42:08<56:52:58,  8.28s/it]
 35%|███▍      | 13246/37962 [30:42:16<56:54:34,  8.29s/it]
 35%|███▍      | 13247/37962 [30:42:24<56:53:57,  8.29s/it]
 35%|███▍      | 13248/37962 [30:42:32<56:18:03,  8.20s/it]
 35%|███▍      | 13249/37962 [30:42:41<56:29:58,  8.23s/it]
 35%|███▍      | 13250/37962 [30:42:49<56:45:02,  8.27s/it]
                                                           
{'loss': 1.3826, 'grad_norm': 3.5839781761169434, 'learning_rate': 1.5120190883928671e-05, 'epoch': 0.35}

 35%|███▍      | 13250/37962 [30:42:49<56:45:02,  8.27s/it]
 35%|███▍      | 13251/37962 [30:42:57<56:39:38,  8.25s/it]
 35%|███▍      | 13252/37962 [30:43:06<57:15:57,  8.34s/it]
 35%|███▍      | 13253/37962 [30:43:14<57:04:35,  8.32s/it]
 35%|███▍      | 13254/37962 [30:43:22<56:51:57,  8.29s/it]
 35%|███▍      | 13255/37962 [30:43:30<56:49:49,  8.28s/it]
 35%|███▍      | 13256/37962 [30:43:38<56:18:49,  8.21s/it]
 35%|███▍      | 13257/37962 [30:43:47<56:19:38,  8.21s/it]
 35%|███▍      | 13258/37962 [30:43:55<55:42:30,  8.12s/it]
 35%|███▍      | 13259/37962 [30:44:03<56:14:36,  8.20s/it]
 35%|███▍      | 13260/37962 [30:44:11<56:50:58,  8.29s/it]
                                                           
{'loss': 1.3831, 'grad_norm': 4.57205057144165, 'learning_rate': 1.511286059449332e-05, 'epoch': 0.35}

 35%|███▍      | 13260/37962 [30:44:11<56:50:58,  8.29s/it]
 35%|███▍      | 13261/37962 [30:44:20<57:06:24,  8.32s/it]
 35%|███▍      | 13262/37962 [30:44:28<57:05:07,  8.32s/it]
 35%|███▍      | 13263/37962 [30:44:37<57:04:01,  8.32s/it]
 35%|███▍      | 13264/37962 [30:44:45<57:26:52,  8.37s/it]
 35%|███▍      | 13265/37962 [30:44:53<56:58:49,  8.31s/it]
 35%|███▍      | 13266/37962 [30:45:02<57:04:16,  8.32s/it]
 35%|███▍      | 13267/37962 [30:45:09<56:19:38,  8.21s/it]
 35%|███▍      | 13268/37962 [30:45:18<56:44:07,  8.27s/it]
 35%|███▍      | 13269/37962 [30:45:26<56:23:14,  8.22s/it]
 35%|███▍      | 13270/37962 [30:45:34<56:46:55,  8.28s/it]
                                                           
{'loss': 1.4039, 'grad_norm': 7.147394180297852, 'learning_rate': 1.5105526583495311e-05, 'epoch': 0.35}

 35%|███▍      | 13270/37962 [30:45:34<56:46:55,  8.28s/it]
 35%|███▍      | 13271/37962 [30:45:43<56:51:48,  8.29s/it]
 35%|███▍      | 13272/37962 [30:45:51<56:50:30,  8.29s/it]
 35%|███▍      | 13273/37962 [30:45:59<56:50:16,  8.29s/it]
 35%|███▍      | 13274/37962 [30:46:07<56:28:57,  8.24s/it]
 35%|███▍      | 13275/37962 [30:46:16<56:28:14,  8.23s/it]
 35%|███▍      | 13276/37962 [30:46:24<56:19:01,  8.21s/it]
 35%|███▍      | 13277/37962 [30:46:32<56:36:46,  8.26s/it]
 35%|███▍      | 13278/37962 [30:46:41<56:52:02,  8.29s/it]
 35%|███▍      | 13279/37962 [30:46:49<56:38:11,  8.26s/it]
 35%|███▍      | 13280/37962 [30:46:57<56:32:42,  8.25s/it]
                                                           
{'loss': 1.2891, 'grad_norm': 7.256117343902588, 'learning_rate': 1.5098188856272948e-05, 'epoch': 0.35}

 35%|███▍      | 13280/37962 [30:46:57<56:32:42,  8.25s/it]
 35%|███▍      | 13281/37962 [30:47:05<57:07:01,  8.33s/it]
 35%|███▍      | 13282/37962 [30:47:14<57:09:25,  8.34s/it]
 35%|███▍      | 13283/37962 [30:47:22<56:57:23,  8.31s/it]
 35%|███▍      | 13284/37962 [30:47:30<57:00:24,  8.32s/it]
 35%|███▍      | 13285/37962 [30:47:39<56:58:35,  8.31s/it]
 35%|███▍      | 13286/37962 [30:47:47<57:09:58,  8.34s/it]
 35%|███▌      | 13287/37962 [30:47:55<56:28:36,  8.24s/it]
 35%|███▌      | 13288/37962 [30:48:03<56:40:05,  8.27s/it]
 35%|███▌      | 13289/37962 [30:48:12<56:51:37,  8.30s/it]
 35%|███▌      | 13290/37962 [30:48:20<56:44:58,  8.28s/it]
                                                           
{'loss': 1.5507, 'grad_norm': 13.992640495300293, 'learning_rate': 1.509084741816724e-05, 'epoch': 0.35}

 35%|███▌      | 13290/37962 [30:48:20<56:44:58,  8.28s/it]
 35%|███▌      | 13291/37962 [30:48:28<56:59:52,  8.32s/it]
 35%|███▌      | 13292/37962 [30:48:37<57:17:07,  8.36s/it]
 35%|███▌      | 13293/37962 [30:48:45<57:33:26,  8.40s/it]
 35%|███▌      | 13294/37962 [30:48:54<57:26:24,  8.38s/it]
 35%|███▌      | 13295/37962 [30:49:02<57:20:57,  8.37s/it]
 35%|███▌      | 13296/37962 [30:49:10<56:44:49,  8.28s/it]
 35%|███▌      | 13297/37962 [30:49:19<56:59:05,  8.32s/it]
 35%|███▌      | 13298/37962 [30:49:27<57:29:12,  8.39s/it]
 35%|███▌      | 13299/37962 [30:49:36<57:31:41,  8.40s/it]
 35%|███▌      | 13300/37962 [30:49:44<57:22:53,  8.38s/it]
                                                           
{'loss': 1.3419, 'grad_norm': 14.541725158691406, 'learning_rate': 1.5083502274521888e-05, 'epoch': 0.35}

 35%|███▌      | 13300/37962 [30:49:44<57:22:53,  8.38s/it]
 35%|███▌      | 13301/37962 [30:49:52<56:25:54,  8.24s/it]
 35%|███▌      | 13302/37962 [30:50:00<56:24:25,  8.23s/it]
 35%|███▌      | 13303/37962 [30:50:08<56:50:54,  8.30s/it]
 35%|███▌      | 13304/37962 [30:50:17<56:35:59,  8.26s/it]
 35%|███▌      | 13305/37962 [30:50:25<56:28:14,  8.24s/it]
 35%|███▌      | 13306/37962 [30:50:33<56:28:47,  8.25s/it]
 35%|███▌      | 13307/37962 [30:50:41<56:04:03,  8.19s/it]
 35%|███▌      | 13308/37962 [30:50:49<56:15:48,  8.22s/it]
 35%|███▌      | 13309/37962 [30:50:58<56:03:53,  8.19s/it]
 35%|███▌      | 13310/37962 [30:51:06<56:32:36,  8.26s/it]
                                                           
{'loss': 1.3729, 'grad_norm': 12.46586799621582, 'learning_rate': 1.5076153430683295e-05, 'epoch': 0.35}

 35%|███▌      | 13310/37962 [30:51:06<56:32:36,  8.26s/it]
 35%|███▌      | 13311/37962 [30:51:14<56:12:10,  8.21s/it]
 35%|███▌      | 13312/37962 [30:51:22<56:25:00,  8.24s/it]
 35%|███▌      | 13313/37962 [30:51:31<56:30:34,  8.25s/it]
 35%|███▌      | 13314/37962 [30:51:39<56:24:19,  8.24s/it]
 35%|███▌      | 13315/37962 [30:51:47<56:25:27,  8.24s/it]
 35%|███▌      | 13316/37962 [30:51:55<56:18:49,  8.23s/it]
 35%|███▌      | 13317/37962 [30:52:03<56:12:42,  8.21s/it]
 35%|███▌      | 13318/37962 [30:52:12<56:23:48,  8.24s/it]
 35%|███▌      | 13319/37962 [30:52:20<55:33:58,  8.12s/it]
 35%|███▌      | 13320/37962 [30:52:28<56:07:19,  8.20s/it]
                                                           
{'loss': 1.3458, 'grad_norm': 8.058752059936523, 'learning_rate': 1.5068800892000561e-05, 'epoch': 0.35}

 35%|███▌      | 13320/37962 [30:52:28<56:07:19,  8.20s/it]
 35%|███▌      | 13321/37962 [30:52:36<56:06:35,  8.20s/it]
 35%|███▌      | 13322/37962 [30:52:44<56:08:02,  8.20s/it]
 35%|███▌      | 13323/37962 [30:52:52<55:35:43,  8.12s/it]
 35%|███▌      | 13324/37962 [30:53:01<56:03:53,  8.19s/it]
 35%|███▌      | 13325/37962 [30:53:09<55:38:46,  8.13s/it]
 35%|███▌      | 13326/37962 [30:53:17<55:45:49,  8.15s/it]
 35%|███▌      | 13327/37962 [30:53:25<56:05:51,  8.20s/it]
 35%|███▌      | 13328/37962 [30:53:33<56:11:53,  8.21s/it]
 35%|███▌      | 13329/37962 [30:53:42<56:37:33,  8.28s/it]
 35%|███▌      | 13330/37962 [30:53:50<56:13:37,  8.22s/it]
                                                           
{'loss': 1.5226, 'grad_norm': 18.819734573364258, 'learning_rate': 1.5061444663825467e-05, 'epoch': 0.35}

 35%|███▌      | 13330/37962 [30:53:50<56:13:37,  8.22s/it]
 35%|███▌      | 13331/37962 [30:53:58<56:30:09,  8.26s/it]
 35%|███▌      | 13332/37962 [30:54:07<56:29:15,  8.26s/it]
 35%|███▌      | 13333/37962 [30:54:15<56:19:40,  8.23s/it]
 35%|███▌      | 13334/37962 [30:54:23<56:47:22,  8.30s/it]
 35%|███▌      | 13335/37962 [30:54:31<56:31:21,  8.26s/it]
 35%|███▌      | 13336/37962 [30:54:40<56:28:45,  8.26s/it]
 35%|███▌      | 13337/37962 [30:54:48<56:31:54,  8.26s/it]
 35%|███▌      | 13338/37962 [30:54:56<55:58:35,  8.18s/it]
 35%|███▌      | 13339/37962 [30:55:04<56:07:36,  8.21s/it]
 35%|███▌      | 13340/37962 [30:55:12<55:38:38,  8.14s/it]
                                                           
{'loss': 1.2759, 'grad_norm': 7.524699687957764, 'learning_rate': 1.5054084751512483e-05, 'epoch': 0.35}

 35%|███▌      | 13340/37962 [30:55:12<55:38:38,  8.14s/it]
 35%|███▌      | 13341/37962 [30:55:21<56:14:05,  8.22s/it]
 35%|███▌      | 13342/37962 [30:55:29<56:01:46,  8.19s/it]
 35%|███▌      | 13343/37962 [30:55:37<56:17:56,  8.23s/it]
 35%|███▌      | 13344/37962 [30:55:45<56:25:27,  8.25s/it]
 35%|███▌      | 13345/37962 [30:55:53<56:20:25,  8.24s/it]
 35%|███▌      | 13346/37962 [30:56:01<55:39:55,  8.14s/it]
 35%|███▌      | 13347/37962 [30:56:10<55:47:21,  8.16s/it]
 35%|███▌      | 13348/37962 [30:56:18<55:40:23,  8.14s/it]
 35%|███▌      | 13349/37962 [30:56:26<56:01:42,  8.19s/it]
 35%|███▌      | 13350/37962 [30:56:34<56:07:21,  8.21s/it]
                                                           
{'loss': 1.4278, 'grad_norm': 7.355799198150635, 'learning_rate': 1.5046721160418769e-05, 'epoch': 0.35}

 35%|███▌      | 13350/37962 [30:56:34<56:07:21,  8.21s/it]
 35%|███▌      | 13351/37962 [30:56:43<56:45:54,  8.30s/it]
 35%|███▌      | 13352/37962 [30:56:51<57:04:06,  8.35s/it]
 35%|███▌      | 13353/37962 [30:56:59<56:42:26,  8.30s/it]
 35%|███▌      | 13354/37962 [30:57:08<56:24:25,  8.25s/it]
 35%|███▌      | 13355/37962 [30:57:16<55:55:46,  8.18s/it]
 35%|███▌      | 13356/37962 [30:57:24<55:57:07,  8.19s/it]
 35%|███▌      | 13357/37962 [30:57:32<55:57:05,  8.19s/it]
 35%|███▌      | 13358/37962 [30:57:40<55:25:45,  8.11s/it]
 35%|███▌      | 13359/37962 [30:57:48<55:57:54,  8.19s/it]
 35%|███▌      | 13360/37962 [30:57:57<56:04:23,  8.21s/it]
                                                           
{'loss': 1.3536, 'grad_norm': 7.913285255432129, 'learning_rate': 1.503935389590415e-05, 'epoch': 0.35}

 35%|███▌      | 13360/37962 [30:57:57<56:04:23,  8.21s/it]
 35%|███▌      | 13361/37962 [30:58:05<55:53:13,  8.18s/it]
 35%|███▌      | 13362/37962 [30:58:13<56:16:14,  8.23s/it]
 35%|███▌      | 13363/37962 [30:58:21<56:24:58,  8.26s/it]
 35%|███▌      | 13364/37962 [30:58:29<56:01:14,  8.20s/it]
 35%|███▌      | 13365/37962 [30:58:38<56:20:56,  8.25s/it]
 35%|███▌      | 13366/37962 [30:58:46<56:43:50,  8.30s/it]
 35%|███▌      | 13367/37962 [30:58:54<56:39:07,  8.29s/it]
 35%|███▌      | 13368/37962 [30:59:03<56:37:57,  8.29s/it]
 35%|███▌      | 13369/37962 [30:59:11<56:53:41,  8.33s/it]
 35%|███▌      | 13370/37962 [30:59:19<56:31:07,  8.27s/it]
                                                           
{'loss': 1.404, 'grad_norm': 7.3410234451293945, 'learning_rate': 1.503198296333113e-05, 'epoch': 0.35}

 35%|███▌      | 13370/37962 [30:59:19<56:31:07,  8.27s/it]
 35%|███▌      | 13371/37962 [30:59:28<56:29:50,  8.27s/it]
 35%|███▌      | 13372/37962 [30:59:36<56:57:32,  8.34s/it]
 35%|███▌      | 13373/37962 [30:59:44<57:13:21,  8.38s/it]
 35%|███▌      | 13374/37962 [30:59:53<57:00:52,  8.35s/it]
 35%|███▌      | 13375/37962 [31:00:01<57:04:44,  8.36s/it]
 35%|███▌      | 13376/37962 [31:00:09<57:01:17,  8.35s/it]
 35%|███▌      | 13377/37962 [31:00:18<56:56:45,  8.34s/it]
 35%|███▌      | 13378/37962 [31:00:26<56:56:02,  8.34s/it]
 35%|███▌      | 13379/37962 [31:00:35<57:00:04,  8.35s/it]
 35%|███▌      | 13380/37962 [31:00:43<56:44:23,  8.31s/it]
                                                           
{'loss': 1.5092, 'grad_norm': 4.592426776885986, 'learning_rate': 1.5024608368064883e-05, 'epoch': 0.35}

 35%|███▌      | 13380/37962 [31:00:43<56:44:23,  8.31s/it]
 35%|███▌      | 13381/37962 [31:00:51<57:00:15,  8.35s/it]
 35%|███▌      | 13382/37962 [31:01:00<57:00:44,  8.35s/it]
 35%|███▌      | 13383/37962 [31:01:08<57:06:36,  8.36s/it]
 35%|███▌      | 13384/37962 [31:01:16<57:05:04,  8.36s/it]
 35%|███▌      | 13385/37962 [31:01:25<56:55:08,  8.34s/it]
 35%|███▌      | 13386/37962 [31:01:32<56:04:18,  8.21s/it]
 35%|███▌      | 13387/37962 [31:01:41<56:19:03,  8.25s/it]
 35%|███▌      | 13388/37962 [31:01:49<56:09:20,  8.23s/it]
 35%|███▌      | 13389/37962 [31:01:57<56:25:10,  8.27s/it]
 35%|███▌      | 13390/37962 [31:02:06<56:22:07,  8.26s/it]
                                                           
{'loss': 1.3978, 'grad_norm': 8.372693061828613, 'learning_rate': 1.5017230115473252e-05, 'epoch': 0.35}

 35%|███▌      | 13390/37962 [31:02:06<56:22:07,  8.26s/it]
 35%|███▌      | 13391/37962 [31:02:13<55:34:22,  8.14s/it]
 35%|███▌      | 13392/37962 [31:02:22<55:56:58,  8.20s/it]
 35%|███▌      | 13393/37962 [31:02:30<56:17:56,  8.25s/it]
 35%|███▌      | 13394/37962 [31:02:39<56:31:11,  8.28s/it]
 35%|███▌      | 13395/37962 [31:02:47<56:24:06,  8.26s/it]
 35%|███▌      | 13396/37962 [31:02:55<56:00:45,  8.21s/it]
 35%|███▌      | 13397/37962 [31:03:03<56:22:18,  8.26s/it]
 35%|███▌      | 13398/37962 [31:03:11<56:17:04,  8.25s/it]
 35%|███▌      | 13399/37962 [31:03:20<56:40:44,  8.31s/it]
 35%|███▌      | 13400/37962 [31:03:28<56:43:22,  8.31s/it]
                                                           
{'loss': 1.3584, 'grad_norm': 5.466429710388184, 'learning_rate': 1.5009848210926736e-05, 'epoch': 0.35}

 35%|███▌      | 13400/37962 [31:03:28<56:43:22,  8.31s/it]
 35%|███▌      | 13401/37962 [31:03:37<57:18:41,  8.40s/it]
 35%|███▌      | 13402/37962 [31:03:45<57:04:45,  8.37s/it]
 35%|███▌      | 13403/37962 [31:03:54<57:29:56,  8.43s/it]
 35%|███▌      | 13404/37962 [31:04:02<57:37:37,  8.45s/it]
 35%|███▌      | 13405/37962 [31:04:11<57:41:33,  8.46s/it]
 35%|███▌      | 13406/37962 [31:04:19<57:19:52,  8.40s/it]
 35%|███▌      | 13407/37962 [31:04:27<57:13:34,  8.39s/it]
 35%|███▌      | 13408/37962 [31:04:35<56:14:37,  8.25s/it]
 35%|███▌      | 13409/37962 [31:04:43<56:15:12,  8.25s/it]
 35%|███▌      | 13410/37962 [31:04:52<56:07:51,  8.23s/it]
                                                           
{'loss': 1.3713, 'grad_norm': 12.244159698486328, 'learning_rate': 1.5002462659798498e-05, 'epoch': 0.35}

 35%|███▌      | 13410/37962 [31:04:52<56:07:51,  8.23s/it]
 35%|███▌      | 13411/37962 [31:05:00<56:31:44,  8.29s/it]
 35%|███▌      | 13412/37962 [31:05:08<56:28:12,  8.28s/it]
 35%|███▌      | 13413/37962 [31:05:16<56:17:37,  8.26s/it]
 35%|███▌      | 13414/37962 [31:05:25<56:16:48,  8.25s/it]
 35%|███▌      | 13415/37962 [31:05:33<56:10:07,  8.24s/it]
 35%|███▌      | 13416/37962 [31:05:41<56:37:38,  8.31s/it]
 35%|███▌      | 13417/37962 [31:05:50<56:25:28,  8.28s/it]
 35%|███▌      | 13418/37962 [31:05:58<56:19:01,  8.26s/it]
 35%|███▌      | 13419/37962 [31:06:06<56:23:36,  8.27s/it]
 35%|███▌      | 13420/37962 [31:06:15<56:36:58,  8.30s/it]
                                                           
{'loss': 1.4009, 'grad_norm': 18.255769729614258, 'learning_rate': 1.4995073467464352e-05, 'epoch': 0.35}

 35%|███▌      | 13420/37962 [31:06:15<56:36:58,  8.30s/it]
 35%|███▌      | 13421/37962 [31:06:23<55:58:23,  8.21s/it]
 35%|███▌      | 13422/37962 [31:06:31<55:36:33,  8.16s/it]
 35%|███▌      | 13423/37962 [31:06:39<55:45:20,  8.18s/it]
 35%|███▌      | 13424/37962 [31:06:47<56:02:34,  8.22s/it]
 35%|███▌      | 13425/37962 [31:06:55<56:03:36,  8.22s/it]
 35%|███▌      | 13426/37962 [31:07:04<56:17:47,  8.26s/it]
 35%|███▌      | 13427/37962 [31:07:12<56:04:13,  8.23s/it]
 35%|███▌      | 13428/37962 [31:07:20<55:37:52,  8.16s/it]
 35%|███▌      | 13429/37962 [31:07:28<55:03:11,  8.08s/it]
 35%|███▌      | 13430/37962 [31:07:36<55:37:03,  8.16s/it]
                                                           
{'loss': 1.3289, 'grad_norm': 10.710450172424316, 'learning_rate': 1.4987680639302761e-05, 'epoch': 0.35}

 35%|███▌      | 13430/37962 [31:07:36<55:37:03,  8.16s/it]
 35%|███▌      | 13431/37962 [31:07:44<55:27:00,  8.14s/it]
 35%|███▌      | 13432/37962 [31:07:52<55:51:23,  8.20s/it]
 35%|███▌      | 13433/37962 [31:08:01<55:50:03,  8.19s/it]
 35%|███▌      | 13434/37962 [31:08:09<55:55:29,  8.21s/it]
 35%|███▌      | 13435/37962 [31:08:17<55:39:42,  8.17s/it]
 35%|███▌      | 13436/37962 [31:08:25<55:56:06,  8.21s/it]
 35%|███▌      | 13437/37962 [31:08:34<56:25:09,  8.28s/it]
 35%|███▌      | 13438/37962 [31:08:42<56:19:44,  8.27s/it]
 35%|███▌      | 13439/37962 [31:08:50<56:11:08,  8.25s/it]
 35%|███▌      | 13440/37962 [31:08:58<55:59:16,  8.22s/it]
                                                           
{'loss': 1.312, 'grad_norm': 35.073429107666016, 'learning_rate': 1.4980284180694839e-05, 'epoch': 0.35}

 35%|███▌      | 13440/37962 [31:08:58<55:59:16,  8.22s/it]
 35%|███▌      | 13441/37962 [31:09:07<56:22:58,  8.28s/it]
 35%|███▌      | 13442/37962 [31:09:15<55:40:59,  8.18s/it]
 35%|███▌      | 13443/37962 [31:09:23<55:45:48,  8.19s/it]
 35%|███▌      | 13444/37962 [31:09:31<55:16:29,  8.12s/it]
 35%|███▌      | 13445/37962 [31:09:39<55:42:34,  8.18s/it]
 35%|███▌      | 13446/37962 [31:09:47<55:41:32,  8.18s/it]
 35%|███▌      | 13447/37962 [31:09:56<55:47:44,  8.19s/it]
 35%|███▌      | 13448/37962 [31:10:04<56:01:26,  8.23s/it]
 35%|███▌      | 13449/37962 [31:10:13<56:52:24,  8.35s/it]
 35%|███▌      | 13450/37962 [31:10:20<55:56:40,  8.22s/it]
                                                           
{'loss': 1.4485, 'grad_norm': 12.47243881225586, 'learning_rate': 1.497288409702434e-05, 'epoch': 0.35}

 35%|███▌      | 13450/37962 [31:10:20<55:56:40,  8.22s/it]
 35%|███▌      | 13451/37962 [31:10:29<56:03:53,  8.23s/it]
 35%|███▌      | 13452/37962 [31:10:37<56:08:23,  8.25s/it]
 35%|███▌      | 13453/37962 [31:10:45<56:20:04,  8.27s/it]
 35%|███▌      | 13454/37962 [31:10:54<56:27:34,  8.29s/it]
 35%|███▌      | 13455/37962 [31:11:02<56:11:52,  8.26s/it]
 35%|███▌      | 13456/37962 [31:11:10<55:56:19,  8.22s/it]
 35%|███▌      | 13457/37962 [31:11:18<56:25:13,  8.29s/it]
 35%|███▌      | 13458/37962 [31:11:27<56:21:00,  8.28s/it]
 35%|███▌      | 13459/37962 [31:11:35<56:32:56,  8.31s/it]
 35%|███▌      | 13460/37962 [31:11:43<55:28:22,  8.15s/it]
                                                           
{'loss': 1.3259, 'grad_norm': 16.22907066345215, 'learning_rate': 1.4965480393677656e-05, 'epoch': 0.35}

 35%|███▌      | 13460/37962 [31:11:43<55:28:22,  8.15s/it]
 35%|███▌      | 13461/37962 [31:11:51<56:00:03,  8.23s/it]
 35%|███▌      | 13462/37962 [31:12:00<56:05:55,  8.24s/it]
 35%|███▌      | 13463/37962 [31:12:08<56:21:02,  8.28s/it]
 35%|███▌      | 13464/37962 [31:12:16<56:27:26,  8.30s/it]
 35%|███▌      | 13465/37962 [31:12:24<56:15:50,  8.27s/it]
 35%|███▌      | 13466/37962 [31:12:33<56:33:34,  8.31s/it]
 35%|███▌      | 13467/37962 [31:12:41<56:37:03,  8.32s/it]
 35%|███▌      | 13468/37962 [31:12:49<56:00:13,  8.23s/it]
 35%|███▌      | 13469/37962 [31:12:57<55:56:41,  8.22s/it]
 35%|███▌      | 13470/37962 [31:13:06<55:57:20,  8.22s/it]
                                                           
{'loss': 1.429, 'grad_norm': 24.07572364807129, 'learning_rate': 1.4958073076043817e-05, 'epoch': 0.35}

 35%|███▌      | 13470/37962 [31:13:06<55:57:20,  8.22s/it]
 35%|███▌      | 13471/37962 [31:13:14<55:42:00,  8.19s/it]
 35%|███▌      | 13472/37962 [31:13:22<55:40:40,  8.18s/it]
 35%|███▌      | 13473/37962 [31:13:30<55:31:40,  8.16s/it]
 35%|███▌      | 13474/37962 [31:13:38<56:01:05,  8.24s/it]
 35%|███▌      | 13475/37962 [31:13:47<56:45:10,  8.34s/it]
 35%|███▌      | 13476/37962 [31:13:55<56:40:34,  8.33s/it]
 36%|███▌      | 13477/37962 [31:14:04<56:52:42,  8.36s/it]
 36%|███▌      | 13478/37962 [31:14:12<56:36:11,  8.32s/it]
 36%|███▌      | 13479/37962 [31:14:20<56:02:39,  8.24s/it]
 36%|███▌      | 13480/37962 [31:14:28<55:01:32,  8.09s/it]
                                                           
{'loss': 1.4567, 'grad_norm': 7.839529991149902, 'learning_rate': 1.4950662149514475e-05, 'epoch': 0.36}

 36%|███▌      | 13480/37962 [31:14:28<55:01:32,  8.09s/it]
 36%|███▌      | 13481/37962 [31:14:36<55:01:14,  8.09s/it]
 36%|███▌      | 13482/37962 [31:14:44<55:11:14,  8.12s/it]
 36%|███▌      | 13483/37962 [31:14:52<55:40:01,  8.19s/it]
 36%|███▌      | 13484/37962 [31:15:00<55:19:38,  8.14s/it]
 36%|███▌      | 13485/37962 [31:15:09<55:35:00,  8.18s/it]
 36%|███▌      | 13486/37962 [31:15:17<55:25:01,  8.15s/it]
 36%|███▌      | 13487/37962 [31:15:25<55:42:18,  8.19s/it]
 36%|███▌      | 13488/37962 [31:15:33<56:00:52,  8.24s/it]
 36%|███▌      | 13489/37962 [31:15:42<56:03:06,  8.25s/it]
 36%|███▌      | 13490/37962 [31:15:50<55:43:22,  8.20s/it]
                                                           
{'loss': 1.2798, 'grad_norm': 4.368978977203369, 'learning_rate': 1.494324761948392e-05, 'epoch': 0.36}

 36%|███▌      | 13490/37962 [31:15:50<55:43:22,  8.20s/it]
 36%|███▌      | 13491/37962 [31:15:58<56:00:49,  8.24s/it]
 36%|███▌      | 13492/37962 [31:16:07<56:19:21,  8.29s/it]
 36%|███▌      | 13493/37962 [31:16:15<55:51:00,  8.22s/it]
 36%|███▌      | 13494/37962 [31:16:23<56:38:38,  8.33s/it]
 36%|███▌      | 13495/37962 [31:16:31<56:31:04,  8.32s/it]
 36%|███▌      | 13496/37962 [31:16:40<56:43:06,  8.35s/it]
 36%|███▌      | 13497/37962 [31:16:48<56:04:19,  8.25s/it]
 36%|███▌      | 13498/37962 [31:16:56<56:13:28,  8.27s/it]
 36%|███▌      | 13499/37962 [31:17:05<56:20:23,  8.29s/it]
 36%|███▌      | 13500/37962 [31:17:13<56:25:26,  8.30s/it]
                                                           
{'loss': 1.263, 'grad_norm': 4.198204040527344, 'learning_rate': 1.4935829491349059e-05, 'epoch': 0.36}

 36%|███▌      | 13500/37962 [31:17:13<56:25:26,  8.30s/it]
 36%|███▌      | 13501/37962 [31:17:21<56:20:28,  8.29s/it]
 36%|███▌      | 13502/37962 [31:17:29<55:41:53,  8.20s/it]
 36%|███▌      | 13503/37962 [31:17:37<55:51:59,  8.22s/it]
 36%|███▌      | 13504/37962 [31:17:46<56:13:36,  8.28s/it]
 36%|███▌      | 13505/37962 [31:17:54<56:06:26,  8.26s/it]
 36%|███▌      | 13506/37962 [31:18:02<56:28:54,  8.31s/it]
 36%|███▌      | 13507/37962 [31:18:11<56:32:29,  8.32s/it]
 36%|███▌      | 13508/37962 [31:18:19<56:16:14,  8.28s/it]
 36%|███▌      | 13509/37962 [31:18:27<56:37:44,  8.34s/it]
 36%|███▌      | 13510/37962 [31:18:36<56:36:48,  8.34s/it]
                                                           
{'loss': 1.3813, 'grad_norm': 6.534206867218018, 'learning_rate': 1.4928407770509416e-05, 'epoch': 0.36}

 36%|███▌      | 13510/37962 [31:18:36<56:36:48,  8.34s/it]
 36%|███▌      | 13511/37962 [31:18:44<55:59:15,  8.24s/it]
 36%|███▌      | 13512/37962 [31:18:52<55:52:18,  8.23s/it]
 36%|███▌      | 13513/37962 [31:19:00<56:04:10,  8.26s/it]
 36%|███▌      | 13514/37962 [31:19:09<56:21:00,  8.30s/it]
 36%|███▌      | 13515/37962 [31:19:17<56:36:20,  8.34s/it]
 36%|███▌      | 13516/37962 [31:19:26<56:39:57,  8.34s/it]
 36%|███▌      | 13517/37962 [31:19:34<57:15:00,  8.43s/it]
 36%|███▌      | 13518/37962 [31:19:42<56:19:03,  8.29s/it]
 36%|███▌      | 13519/37962 [31:19:50<55:24:55,  8.16s/it]
 36%|███▌      | 13520/37962 [31:19:58<55:32:45,  8.18s/it]
                                                           
{'loss': 1.3221, 'grad_norm': 6.692596912384033, 'learning_rate': 1.492098246236714e-05, 'epoch': 0.36}

 36%|███▌      | 13520/37962 [31:19:58<55:32:45,  8.18s/it]
 36%|███▌      | 13521/37962 [31:20:07<56:06:02,  8.26s/it]
 36%|███▌      | 13522/37962 [31:20:15<56:01:48,  8.25s/it]
 36%|███▌      | 13523/37962 [31:20:23<56:09:37,  8.27s/it]
 36%|███▌      | 13524/37962 [31:20:31<55:51:21,  8.23s/it]
 36%|███▌      | 13525/37962 [31:20:40<56:03:50,  8.26s/it]
 36%|███▌      | 13526/37962 [31:20:48<56:05:08,  8.26s/it]
 36%|███▌      | 13527/37962 [31:20:56<56:08:06,  8.27s/it]
 36%|███▌      | 13528/37962 [31:21:04<56:00:35,  8.25s/it]
 36%|███▌      | 13529/37962 [31:21:13<55:58:26,  8.25s/it]
 36%|███▌      | 13530/37962 [31:21:21<56:24:52,  8.31s/it]
                                                           
{'loss': 1.4581, 'grad_norm': 9.253410339355469, 'learning_rate': 1.4913553572326978e-05, 'epoch': 0.36}

 36%|███▌      | 13530/37962 [31:21:21<56:24:52,  8.31s/it]
 36%|███▌      | 13531/37962 [31:21:29<56:12:55,  8.28s/it]
 36%|███▌      | 13532/37962 [31:21:38<56:11:57,  8.28s/it]
 36%|███▌      | 13533/37962 [31:21:46<55:50:22,  8.23s/it]
 36%|███▌      | 13534/37962 [31:21:54<55:54:13,  8.24s/it]
 36%|███▌      | 13535/37962 [31:22:02<55:56:10,  8.24s/it]
 36%|███▌      | 13536/37962 [31:22:10<55:54:52,  8.24s/it]
 36%|███▌      | 13537/37962 [31:22:19<55:36:52,  8.20s/it]
 36%|███▌      | 13538/37962 [31:22:27<55:40:57,  8.21s/it]
 36%|███▌      | 13539/37962 [31:22:35<55:45:39,  8.22s/it]
 36%|███▌      | 13540/37962 [31:22:43<55:48:30,  8.23s/it]
                                                           
{'loss': 1.3248, 'grad_norm': 8.166877746582031, 'learning_rate': 1.4906121105796289e-05, 'epoch': 0.36}

 36%|███▌      | 13540/37962 [31:22:43<55:48:30,  8.23s/it]
 36%|███▌      | 13541/37962 [31:22:51<55:21:45,  8.16s/it]
 36%|███▌      | 13542/37962 [31:23:00<55:46:07,  8.22s/it]
 36%|███▌      | 13543/37962 [31:23:08<55:50:40,  8.23s/it]
 36%|███▌      | 13544/37962 [31:23:16<55:45:55,  8.22s/it]
 36%|███▌      | 13545/37962 [31:23:24<55:43:35,  8.22s/it]
 36%|███▌      | 13546/37962 [31:23:33<56:22:31,  8.31s/it]
 36%|███▌      | 13547/37962 [31:23:41<55:59:00,  8.25s/it]
 36%|███▌      | 13548/37962 [31:23:49<55:55:44,  8.25s/it]
 36%|███▌      | 13549/37962 [31:23:58<56:08:23,  8.28s/it]
 36%|███▌      | 13550/37962 [31:24:06<56:22:06,  8.31s/it]
                                                           
{'loss': 1.3628, 'grad_norm': 4.618102073669434, 'learning_rate': 1.4898685068185043e-05, 'epoch': 0.36}

 36%|███▌      | 13550/37962 [31:24:06<56:22:06,  8.31s/it]
 36%|███▌      | 13551/37962 [31:24:14<56:16:13,  8.30s/it]
 36%|███▌      | 13552/37962 [31:24:23<56:17:26,  8.30s/it]
 36%|███▌      | 13553/37962 [31:24:30<55:25:50,  8.18s/it]
 36%|███▌      | 13554/37962 [31:24:39<55:23:06,  8.17s/it]
 36%|███▌      | 13555/37962 [31:24:47<55:45:54,  8.23s/it]
 36%|███▌      | 13556/37962 [31:24:55<56:04:16,  8.27s/it]
 36%|███▌      | 13557/37962 [31:25:04<56:03:59,  8.27s/it]
 36%|███▌      | 13558/37962 [31:25:12<56:40:15,  8.36s/it]
 36%|███▌      | 13559/37962 [31:25:20<56:30:07,  8.34s/it]
 36%|███▌      | 13560/37962 [31:25:29<56:53:54,  8.39s/it]
                                                           
{'loss': 1.2954, 'grad_norm': 16.70050811767578, 'learning_rate': 1.4891245464905799e-05, 'epoch': 0.36}

 36%|███▌      | 13560/37962 [31:25:29<56:53:54,  8.39s/it]
 36%|███▌      | 13561/37962 [31:25:37<56:44:22,  8.37s/it]
 36%|███▌      | 13562/37962 [31:25:46<56:31:15,  8.34s/it]
 36%|███▌      | 13563/37962 [31:25:54<56:43:50,  8.37s/it]
 36%|███▌      | 13564/37962 [31:26:02<55:22:43,  8.17s/it]
 36%|███▌      | 13565/37962 [31:26:10<55:38:58,  8.21s/it]
 36%|███▌      | 13566/37962 [31:26:25<69:07:30, 10.20s/it]
 36%|███▌      | 13567/37962 [31:26:33<64:20:57,  9.50s/it]
 36%|███▌      | 13568/37962 [31:26:41<61:55:21,  9.14s/it]
 36%|███▌      | 13569/37962 [31:26:49<60:04:11,  8.87s/it]
 36%|███▌      | 13570/37962 [31:26:58<59:11:30,  8.74s/it]
                                                           
{'loss': 1.2838, 'grad_norm': 5.224015712738037, 'learning_rate': 1.488380230137372e-05, 'epoch': 0.36}

 36%|███▌      | 13570/37962 [31:26:58<59:11:30,  8.74s/it]
 36%|███▌      | 13571/37962 [31:27:06<58:39:48,  8.66s/it]
 36%|███▌      | 13572/37962 [31:27:14<58:02:33,  8.57s/it]
 36%|███▌      | 13573/37962 [31:27:23<57:30:27,  8.49s/it]
 36%|███▌      | 13574/37962 [31:27:31<57:04:02,  8.42s/it]
 36%|███▌      | 13575/37962 [31:27:39<56:58:54,  8.41s/it]
 36%|███▌      | 13576/37962 [31:27:47<56:08:05,  8.29s/it]
 36%|███▌      | 13577/37962 [31:27:56<56:16:55,  8.31s/it]
 36%|███▌      | 13578/37962 [31:28:04<56:10:17,  8.29s/it]
 36%|███▌      | 13579/37962 [31:28:12<56:07:03,  8.29s/it]
 36%|███▌      | 13580/37962 [31:28:21<56:16:50,  8.31s/it]
                                                           
{'loss': 1.3631, 'grad_norm': 2.4615893363952637, 'learning_rate': 1.4876355583006546e-05, 'epoch': 0.36}

 36%|███▌      | 13580/37962 [31:28:21<56:16:50,  8.31s/it]
 36%|███▌      | 13581/37962 [31:28:29<56:06:50,  8.29s/it]
 36%|███▌      | 13582/37962 [31:28:37<56:10:52,  8.30s/it]
 36%|███▌      | 13583/37962 [31:28:45<56:04:34,  8.28s/it]
 36%|███▌      | 13584/37962 [31:28:54<56:01:10,  8.27s/it]
 36%|███▌      | 13585/37962 [31:29:02<56:06:26,  8.29s/it]
 36%|███▌      | 13586/37962 [31:29:10<55:53:45,  8.26s/it]
 36%|███▌      | 13587/37962 [31:29:18<55:50:58,  8.25s/it]
 36%|███▌      | 13588/37962 [31:29:27<56:10:00,  8.30s/it]
 36%|███▌      | 13589/37962 [31:29:35<56:08:15,  8.29s/it]
 36%|███▌      | 13590/37962 [31:29:44<56:21:43,  8.33s/it]
                                                           
{'loss': 1.4275, 'grad_norm': 10.01089096069336, 'learning_rate': 1.4868905315224622e-05, 'epoch': 0.36}

 36%|███▌      | 13590/37962 [31:29:44<56:21:43,  8.33s/it]
 36%|███▌      | 13591/37962 [31:29:52<56:01:14,  8.28s/it]
 36%|███▌      | 13592/37962 [31:30:00<56:10:40,  8.30s/it]
 36%|███▌      | 13593/37962 [31:30:08<56:04:55,  8.28s/it]
 36%|███▌      | 13594/37962 [31:30:16<55:42:41,  8.23s/it]
 36%|███▌      | 13595/37962 [31:30:25<55:52:39,  8.26s/it]
 36%|███▌      | 13596/37962 [31:30:33<56:03:50,  8.28s/it]
 36%|███▌      | 13597/37962 [31:30:41<55:55:13,  8.26s/it]
 36%|███▌      | 13598/37962 [31:30:50<56:06:38,  8.29s/it]
 36%|███▌      | 13599/37962 [31:30:58<56:09:11,  8.30s/it]
 36%|███▌      | 13600/37962 [31:31:06<56:16:04,  8.31s/it]
                                                           
{'loss': 1.4282, 'grad_norm': 7.35436487197876, 'learning_rate': 1.486145150345087e-05, 'epoch': 0.36}

 36%|███▌      | 13600/37962 [31:31:06<56:16:04,  8.31s/it]
 36%|███▌      | 13601/37962 [31:31:15<56:13:36,  8.31s/it]
 36%|███▌      | 13602/37962 [31:31:23<56:30:11,  8.35s/it]
 36%|███▌      | 13603/37962 [31:31:31<56:28:10,  8.35s/it]
 36%|███▌      | 13604/37962 [31:31:40<56:11:51,  8.31s/it]
 36%|███▌      | 13605/37962 [31:31:48<56:48:02,  8.40s/it]
 36%|███▌      | 13606/37962 [31:31:57<56:35:20,  8.36s/it]
 36%|███▌      | 13607/37962 [31:32:05<56:17:29,  8.32s/it]
 36%|███▌      | 13608/37962 [31:32:13<56:10:39,  8.30s/it]
 36%|███▌      | 13609/37962 [31:32:21<55:34:49,  8.22s/it]
 36%|███▌      | 13610/37962 [31:32:29<55:42:20,  8.24s/it]
                                                           
{'loss': 1.3901, 'grad_norm': 14.222929000854492, 'learning_rate': 1.4853994153110784e-05, 'epoch': 0.36}

 36%|███▌      | 13610/37962 [31:32:29<55:42:20,  8.24s/it]
 36%|███▌      | 13611/37962 [31:32:38<55:40:03,  8.23s/it]
 36%|███▌      | 13612/37962 [31:32:46<55:59:10,  8.28s/it]
 36%|███▌      | 13613/37962 [31:32:54<55:09:26,  8.15s/it]
 36%|███▌      | 13614/37962 [31:33:02<54:56:04,  8.12s/it]
 36%|███▌      | 13615/37962 [31:33:10<54:49:07,  8.11s/it]
 36%|███▌      | 13616/37962 [31:33:18<55:09:08,  8.16s/it]
 36%|███▌      | 13617/37962 [31:33:26<55:23:26,  8.19s/it]
 36%|███▌      | 13618/37962 [31:33:35<55:10:21,  8.16s/it]
 36%|███▌      | 13619/37962 [31:33:43<54:54:58,  8.12s/it]
 36%|███▌      | 13620/37962 [31:33:51<55:01:40,  8.14s/it]
                                                           
{'loss': 1.3364, 'grad_norm': 19.453901290893555, 'learning_rate': 1.4846533269632447e-05, 'epoch': 0.36}

 36%|███▌      | 13620/37962 [31:33:51<55:01:40,  8.14s/it]
 36%|███▌      | 13621/37962 [31:33:59<55:11:00,  8.16s/it]
 36%|███▌      | 13622/37962 [31:34:07<55:24:12,  8.19s/it]
 36%|███▌      | 13623/37962 [31:34:16<56:10:18,  8.31s/it]
 36%|███▌      | 13624/37962 [31:34:24<55:36:31,  8.23s/it]
 36%|███▌      | 13625/37962 [31:34:32<55:40:50,  8.24s/it]
 36%|███▌      | 13626/37962 [31:34:40<55:45:26,  8.25s/it]
 36%|███▌      | 13627/37962 [31:34:48<55:15:10,  8.17s/it]
 36%|███▌      | 13628/37962 [31:34:57<55:24:12,  8.20s/it]
 36%|███▌      | 13629/37962 [31:35:05<55:28:45,  8.21s/it]
 36%|███▌      | 13630/37962 [31:35:13<55:37:20,  8.23s/it]
                                                           
{'loss': 1.4418, 'grad_norm': 9.946592330932617, 'learning_rate': 1.4839068858446498e-05, 'epoch': 0.36}

 36%|███▌      | 13630/37962 [31:35:13<55:37:20,  8.23s/it]
 36%|███▌      | 13631/37962 [31:35:21<55:15:00,  8.17s/it]
 36%|███▌      | 13632/37962 [31:35:30<55:35:28,  8.23s/it]
 36%|███▌      | 13633/37962 [31:35:38<55:27:54,  8.21s/it]
 36%|███▌      | 13634/37962 [31:35:46<55:32:14,  8.22s/it]
 36%|███▌      | 13635/37962 [31:35:54<55:35:37,  8.23s/it]
 36%|███▌      | 13636/37962 [31:36:02<55:42:03,  8.24s/it]
 36%|███▌      | 13637/37962 [31:36:11<55:45:52,  8.25s/it]
 36%|███▌      | 13638/37962 [31:36:19<55:34:10,  8.22s/it]
 36%|███▌      | 13639/37962 [31:36:27<56:19:29,  8.34s/it]
 36%|███▌      | 13640/37962 [31:36:36<56:09:23,  8.31s/it]
                                                           
{'loss': 1.2256, 'grad_norm': 18.906896591186523, 'learning_rate': 1.4831600924986162e-05, 'epoch': 0.36}

 36%|███▌      | 13640/37962 [31:36:36<56:09:23,  8.31s/it]
 36%|███▌      | 13641/37962 [31:36:44<55:37:52,  8.23s/it]
 36%|███▌      | 13642/37962 [31:36:52<55:04:49,  8.15s/it]
 36%|███▌      | 13643/37962 [31:37:00<55:40:04,  8.24s/it]
 36%|███▌      | 13644/37962 [31:37:08<55:40:49,  8.24s/it]
 36%|███▌      | 13645/37962 [31:37:17<55:42:35,  8.25s/it]
 36%|███▌      | 13646/37962 [31:37:25<55:35:48,  8.23s/it]
 36%|███▌      | 13647/37962 [31:37:33<56:00:26,  8.29s/it]
 36%|███▌      | 13648/37962 [31:37:41<55:34:26,  8.23s/it]
 36%|███▌      | 13649/37962 [31:37:50<55:54:20,  8.28s/it]
 36%|███▌      | 13650/37962 [31:37:58<55:45:02,  8.26s/it]
                                                           
{'loss': 1.2316, 'grad_norm': 8.317885398864746, 'learning_rate': 1.4824129474687215e-05, 'epoch': 0.36}

 36%|███▌      | 13650/37962 [31:37:58<55:45:02,  8.26s/it]
 36%|███▌      | 13651/37962 [31:38:06<55:16:21,  8.18s/it]
 36%|███▌      | 13652/37962 [31:38:14<55:02:17,  8.15s/it]
 36%|███▌      | 13653/37962 [31:38:22<55:09:07,  8.17s/it]
 36%|███▌      | 13654/37962 [31:38:31<55:35:55,  8.23s/it]
 36%|███▌      | 13655/37962 [31:38:39<55:53:14,  8.28s/it]
 36%|███▌      | 13656/37962 [31:38:48<56:24:56,  8.36s/it]
 36%|███▌      | 13657/37962 [31:38:56<56:13:23,  8.33s/it]
 36%|███▌      | 13658/37962 [31:39:04<56:13:11,  8.33s/it]
 36%|███▌      | 13659/37962 [31:39:12<56:09:53,  8.32s/it]
 36%|███▌      | 13660/37962 [31:39:21<56:22:26,  8.35s/it]
                                                           
{'loss': 1.4655, 'grad_norm': 11.247236251831055, 'learning_rate': 1.4816654512987994e-05, 'epoch': 0.36}

 36%|███▌      | 13660/37962 [31:39:21<56:22:26,  8.35s/it]
 36%|███▌      | 13661/37962 [31:39:29<55:45:27,  8.26s/it]
 36%|███▌      | 13662/37962 [31:39:37<55:24:32,  8.21s/it]
 36%|███▌      | 13663/37962 [31:39:45<55:04:37,  8.16s/it]
 36%|███▌      | 13664/37962 [31:39:53<55:09:54,  8.17s/it]
 36%|███▌      | 13665/37962 [31:40:02<55:24:09,  8.21s/it]
 36%|███▌      | 13666/37962 [31:40:10<56:00:32,  8.30s/it]
 36%|███▌      | 13667/37962 [31:40:18<55:55:27,  8.29s/it]
 36%|███▌      | 13668/37962 [31:40:27<56:02:58,  8.31s/it]
 36%|███▌      | 13669/37962 [31:40:35<56:01:59,  8.30s/it]
 36%|███▌      | 13670/37962 [31:40:43<55:39:33,  8.25s/it]
                                                           
{'loss': 1.4404, 'grad_norm': 2.615150213241577, 'learning_rate': 1.4809176045329397e-05, 'epoch': 0.36}

 36%|███▌      | 13670/37962 [31:40:43<55:39:33,  8.25s/it]
 36%|███▌      | 13671/37962 [31:40:51<55:39:21,  8.25s/it]
 36%|███▌      | 13672/37962 [31:41:00<55:54:10,  8.29s/it]
 36%|███▌      | 13673/37962 [31:41:08<56:11:20,  8.33s/it]
 36%|███▌      | 13674/37962 [31:41:16<55:52:48,  8.28s/it]
 36%|███▌      | 13675/37962 [31:41:25<55:45:07,  8.26s/it]
 36%|███▌      | 13676/37962 [31:41:33<56:14:23,  8.34s/it]
 36%|███▌      | 13677/37962 [31:41:42<56:27:44,  8.37s/it]
 36%|███▌      | 13678/37962 [31:41:50<56:09:54,  8.33s/it]
 36%|███▌      | 13679/37962 [31:41:58<56:00:12,  8.30s/it]
 36%|███▌      | 13680/37962 [31:42:06<55:53:42,  8.29s/it]
                                                           
{'loss': 1.4244, 'grad_norm': 5.09364128112793, 'learning_rate': 1.480169407715487e-05, 'epoch': 0.36}

 36%|███▌      | 13680/37962 [31:42:06<55:53:42,  8.29s/it]
 36%|███▌      | 13681/37962 [31:42:15<56:18:41,  8.35s/it]
 36%|███▌      | 13682/37962 [31:42:23<55:34:29,  8.24s/it]
 36%|███▌      | 13683/37962 [31:42:31<55:36:51,  8.25s/it]
 36%|███▌      | 13684/37962 [31:42:39<55:00:21,  8.16s/it]
 36%|███▌      | 13685/37962 [31:42:47<55:06:11,  8.17s/it]
 36%|███▌      | 13686/37962 [31:42:55<55:09:34,  8.18s/it]
 36%|███▌      | 13687/37962 [31:43:04<55:35:32,  8.24s/it]
 36%|███▌      | 13688/37962 [31:43:12<55:39:35,  8.25s/it]
 36%|███▌      | 13689/37962 [31:43:20<55:52:00,  8.29s/it]
 36%|███▌      | 13690/37962 [31:43:29<56:05:41,  8.32s/it]
                                                           
{'loss': 1.4407, 'grad_norm': 12.840813636779785, 'learning_rate': 1.4794208613910405e-05, 'epoch': 0.36}

 36%|███▌      | 13690/37962 [31:43:29<56:05:41,  8.32s/it]
 36%|███▌      | 13691/37962 [31:43:37<55:56:31,  8.30s/it]
 36%|███▌      | 13692/37962 [31:43:45<55:45:33,  8.27s/it]
 36%|███▌      | 13693/37962 [31:43:54<55:48:10,  8.28s/it]
 36%|███▌      | 13694/37962 [31:44:02<55:58:31,  8.30s/it]
 36%|███▌      | 13695/37962 [31:44:10<56:16:13,  8.35s/it]
 36%|███▌      | 13696/37962 [31:44:19<55:58:19,  8.30s/it]
 36%|███▌      | 13697/37962 [31:44:27<55:46:26,  8.27s/it]
 36%|███▌      | 13698/37962 [31:44:35<55:17:26,  8.20s/it]
 36%|███▌      | 13699/37962 [31:44:43<55:15:24,  8.20s/it]
 36%|███▌      | 13700/37962 [31:44:51<55:14:45,  8.20s/it]
                                                           
{'loss': 1.3371, 'grad_norm': 7.254754066467285, 'learning_rate': 1.4786719661044548e-05, 'epoch': 0.36}

 36%|███▌      | 13700/37962 [31:44:51<55:14:45,  8.20s/it]
 36%|███▌      | 13701/37962 [31:44:59<55:17:41,  8.20s/it]
 36%|███▌      | 13702/37962 [31:45:08<56:03:41,  8.32s/it]
 36%|███▌      | 13703/37962 [31:45:16<55:45:10,  8.27s/it]
 36%|███▌      | 13704/37962 [31:45:24<55:47:37,  8.28s/it]
 36%|███▌      | 13705/37962 [31:45:33<56:22:41,  8.37s/it]
 36%|███▌      | 13706/37962 [31:45:41<56:27:54,  8.38s/it]
 36%|███▌      | 13707/37962 [31:45:50<56:12:47,  8.34s/it]
 36%|███▌      | 13708/37962 [31:45:58<55:39:13,  8.26s/it]
 36%|███▌      | 13709/37962 [31:46:06<55:52:09,  8.29s/it]
 36%|███▌      | 13710/37962 [31:46:14<55:53:35,  8.30s/it]
                                                           
{'loss': 1.3778, 'grad_norm': 20.475887298583984, 'learning_rate': 1.4779227224008372e-05, 'epoch': 0.36}

 36%|███▌      | 13710/37962 [31:46:14<55:53:35,  8.30s/it]
 36%|███▌      | 13711/37962 [31:46:23<55:43:26,  8.27s/it]
 36%|███▌      | 13712/37962 [31:46:31<55:38:45,  8.26s/it]
 36%|███▌      | 13713/37962 [31:46:39<55:31:50,  8.24s/it]
 36%|███▌      | 13714/37962 [31:46:47<55:29:45,  8.24s/it]
 36%|███▌      | 13715/37962 [31:46:55<55:15:43,  8.20s/it]
 36%|███▌      | 13716/37962 [31:47:04<55:49:55,  8.29s/it]
 36%|███▌      | 13717/37962 [31:47:12<55:48:09,  8.29s/it]
 36%|███▌      | 13718/37962 [31:47:21<55:55:07,  8.30s/it]
 36%|███▌      | 13719/37962 [31:47:29<55:48:07,  8.29s/it]
 36%|███▌      | 13720/37962 [31:47:37<55:40:05,  8.27s/it]
                                                           
{'loss': 1.4109, 'grad_norm': 5.804864883422852, 'learning_rate': 1.4771731308255492e-05, 'epoch': 0.36}

 36%|███▌      | 13720/37962 [31:47:37<55:40:05,  8.27s/it]
 36%|███▌      | 13721/37962 [31:47:45<55:44:04,  8.28s/it]
 36%|███▌      | 13722/37962 [31:47:54<55:48:38,  8.29s/it]
 36%|███▌      | 13723/37962 [31:48:02<56:04:51,  8.33s/it]
 36%|███▌      | 13724/37962 [31:48:10<56:07:01,  8.33s/it]
 36%|███▌      | 13725/37962 [31:48:19<55:46:24,  8.28s/it]
 36%|███▌      | 13726/37962 [31:48:27<55:46:05,  8.28s/it]
 36%|███▌      | 13727/37962 [31:48:35<55:49:49,  8.29s/it]
 36%|███▌      | 13728/37962 [31:48:44<55:59:19,  8.32s/it]
 36%|███▌      | 13729/37962 [31:48:52<55:51:28,  8.30s/it]
 36%|███▌      | 13730/37962 [31:49:00<55:50:33,  8.30s/it]
                                                           
{'loss': 1.4241, 'grad_norm': 7.684752941131592, 'learning_rate': 1.476423191924206e-05, 'epoch': 0.36}

 36%|███▌      | 13730/37962 [31:49:00<55:50:33,  8.30s/it]
 36%|███▌      | 13731/37962 [31:49:08<55:25:21,  8.23s/it]
 36%|███▌      | 13732/37962 [31:49:16<54:57:11,  8.16s/it]
 36%|███▌      | 13733/37962 [31:49:24<55:07:57,  8.19s/it]
 36%|███▌      | 13734/37962 [31:49:33<55:16:27,  8.21s/it]
 36%|███▌      | 13735/37962 [31:49:41<54:53:47,  8.16s/it]
 36%|███▌      | 13736/37962 [31:49:49<55:20:04,  8.22s/it]
 36%|███▌      | 13737/37962 [31:49:57<55:10:34,  8.20s/it]
 36%|███▌      | 13738/37962 [31:50:06<55:23:23,  8.23s/it]
 36%|███▌      | 13739/37962 [31:50:14<55:03:41,  8.18s/it]
 36%|███▌      | 13740/37962 [31:50:22<55:26:48,  8.24s/it]
                                                           
{'loss': 1.5095, 'grad_norm': 5.1446614265441895, 'learning_rate': 1.4756729062426748e-05, 'epoch': 0.36}

 36%|███▌      | 13740/37962 [31:50:22<55:26:48,  8.24s/it]
 36%|███▌      | 13741/37962 [31:50:30<55:26:19,  8.24s/it]
 36%|███▌      | 13742/37962 [31:50:38<55:19:59,  8.22s/it]
 36%|███▌      | 13743/37962 [31:50:47<55:25:39,  8.24s/it]
 36%|███▌      | 13744/37962 [31:50:55<55:33:05,  8.26s/it]
 36%|███▌      | 13745/37962 [31:51:03<55:29:55,  8.25s/it]
 36%|███▌      | 13746/37962 [31:51:11<54:43:20,  8.14s/it]
 36%|███▌      | 13747/37962 [31:51:19<54:59:07,  8.17s/it]
 36%|███▌      | 13748/37962 [31:51:28<55:11:46,  8.21s/it]
 36%|███▌      | 13749/37962 [31:51:36<55:17:04,  8.22s/it]
 36%|███▌      | 13750/37962 [31:51:44<54:53:13,  8.16s/it]
                                                           
{'loss': 1.3515, 'grad_norm': 5.480652809143066, 'learning_rate': 1.4749222743270757e-05, 'epoch': 0.36}

 36%|███▌      | 13750/37962 [31:51:44<54:53:13,  8.16s/it]
 36%|███▌      | 13751/37962 [31:51:52<54:55:59,  8.17s/it]
 36%|███▌      | 13752/37962 [31:52:00<55:23:55,  8.24s/it]
 36%|███▌      | 13753/37962 [31:52:09<56:28:24,  8.40s/it]
 36%|███▌      | 13754/37962 [31:52:17<55:47:16,  8.30s/it]
 36%|███▌      | 13755/37962 [31:52:26<56:03:47,  8.34s/it]
 36%|███▌      | 13756/37962 [31:52:34<55:54:25,  8.31s/it]
 36%|███▌      | 13757/37962 [31:52:43<56:30:49,  8.41s/it]
 36%|███▌      | 13758/37962 [31:52:51<55:27:35,  8.25s/it]
 36%|███▌      | 13759/37962 [31:52:59<55:19:38,  8.23s/it]
 36%|███▌      | 13760/37962 [31:53:07<55:21:46,  8.24s/it]
                                                           
{'loss': 1.4088, 'grad_norm': 9.661053657531738, 'learning_rate': 1.4741712967237804e-05, 'epoch': 0.36}

 36%|███▌      | 13760/37962 [31:53:07<55:21:46,  8.24s/it]
 36%|███▌      | 13761/37962 [31:53:16<56:02:02,  8.34s/it]
 36%|███▋      | 13762/37962 [31:53:24<55:31:34,  8.26s/it]
 36%|███▋      | 13763/37962 [31:53:32<54:48:40,  8.15s/it]
 36%|███▋      | 13764/37962 [31:53:40<54:54:27,  8.17s/it]
 36%|███▋      | 13765/37962 [31:53:48<55:16:37,  8.22s/it]
 36%|███▋      | 13766/37962 [31:53:56<54:52:39,  8.16s/it]
 36%|███▋      | 13767/37962 [31:54:04<55:04:11,  8.19s/it]
 36%|███▋      | 13768/37962 [31:54:13<55:28:08,  8.25s/it]
 36%|███▋      | 13769/37962 [31:54:21<55:41:47,  8.29s/it]
 36%|███▋      | 13770/37962 [31:54:29<55:36:37,  8.28s/it]
                                                           
{'loss': 1.404, 'grad_norm': 9.502453804016113, 'learning_rate': 1.4734199739794128e-05, 'epoch': 0.36}

 36%|███▋      | 13770/37962 [31:54:29<55:36:37,  8.28s/it]
 36%|███▋      | 13771/37962 [31:54:38<55:31:45,  8.26s/it]
 36%|███▋      | 13772/37962 [31:54:46<54:49:40,  8.16s/it]
 36%|███▋      | 13773/37962 [31:54:54<55:04:01,  8.20s/it]
 36%|███▋      | 13774/37962 [31:55:02<55:26:09,  8.25s/it]
 36%|███▋      | 13775/37962 [31:55:10<55:31:23,  8.26s/it]
 36%|███▋      | 13776/37962 [31:55:19<55:22:17,  8.24s/it]
 36%|███▋      | 13777/37962 [31:55:27<55:45:16,  8.30s/it]
 36%|███▋      | 13778/37962 [31:55:36<56:18:41,  8.38s/it]
 36%|███▋      | 13779/37962 [31:55:44<55:52:55,  8.32s/it]
 36%|███▋      | 13780/37962 [31:55:52<55:48:43,  8.31s/it]
                                                           
{'loss': 1.4646, 'grad_norm': 12.694108009338379, 'learning_rate': 1.4726683066408478e-05, 'epoch': 0.36}

 36%|███▋      | 13780/37962 [31:55:52<55:48:43,  8.31s/it]
 36%|███▋      | 13781/37962 [31:56:00<55:45:33,  8.30s/it]
 36%|███▋      | 13782/37962 [31:56:09<56:19:49,  8.39s/it]
 36%|███▋      | 13783/37962 [31:56:17<54:58:37,  8.19s/it]
 36%|███▋      | 13784/37962 [31:56:25<54:59:04,  8.19s/it]
 36%|███▋      | 13785/37962 [31:56:33<55:27:32,  8.26s/it]
 36%|███▋      | 13786/37962 [31:56:42<56:03:03,  8.35s/it]
 36%|███▋      | 13787/37962 [31:56:50<56:08:05,  8.36s/it]
 36%|███▋      | 13788/37962 [31:56:59<55:54:13,  8.33s/it]
 36%|███▋      | 13789/37962 [31:57:07<55:49:40,  8.31s/it]
 36%|███▋      | 13790/37962 [31:57:15<56:06:31,  8.36s/it]
                                                           
{'loss': 1.4172, 'grad_norm': 17.397258758544922, 'learning_rate': 1.4719162952552104e-05, 'epoch': 0.36}

 36%|███▋      | 13790/37962 [31:57:15<56:06:31,  8.36s/it]
 36%|███▋      | 13791/37962 [31:57:23<55:48:19,  8.31s/it]
 36%|███▋      | 13792/37962 [31:57:31<55:13:26,  8.23s/it]
 36%|███▋      | 13793/37962 [31:57:39<54:18:31,  8.09s/it]
 36%|███▋      | 13794/37962 [31:57:48<54:54:28,  8.18s/it]
 36%|███▋      | 13795/37962 [31:57:56<55:37:42,  8.29s/it]
 36%|███▋      | 13796/37962 [31:58:05<55:53:04,  8.33s/it]
 36%|███▋      | 13797/37962 [31:58:13<55:46:07,  8.31s/it]
 36%|███▋      | 13798/37962 [31:58:21<55:14:02,  8.23s/it]
 36%|███▋      | 13799/37962 [31:58:29<55:41:02,  8.30s/it]
 36%|███▋      | 13800/37962 [31:58:37<55:06:37,  8.21s/it]
                                                           
{'loss': 1.4191, 'grad_norm': 19.81020736694336, 'learning_rate': 1.4711639403698772e-05, 'epoch': 0.36}

 36%|███▋      | 13800/37962 [31:58:37<55:06:37,  8.21s/it]
 36%|███▋      | 13801/37962 [31:58:46<55:26:25,  8.26s/it]
 36%|███▋      | 13802/37962 [31:58:54<55:24:27,  8.26s/it]
 36%|███▋      | 13803/37962 [31:59:02<55:35:07,  8.28s/it]
 36%|███▋      | 13804/37962 [31:59:11<55:24:16,  8.26s/it]
 36%|███▋      | 13805/37962 [31:59:19<55:43:43,  8.30s/it]
 36%|███▋      | 13806/37962 [31:59:27<55:30:42,  8.27s/it]
 36%|███▋      | 13807/37962 [31:59:36<56:09:22,  8.37s/it]
 36%|███▋      | 13808/37962 [31:59:44<55:23:36,  8.26s/it]
 36%|███▋      | 13809/37962 [31:59:52<55:18:56,  8.24s/it]
 36%|███▋      | 13810/37962 [32:00:00<55:27:58,  8.27s/it]
                                                           
{'loss': 1.4809, 'grad_norm': 8.570978164672852, 'learning_rate': 1.4704112425324743e-05, 'epoch': 0.36}

 36%|███▋      | 13810/37962 [32:00:00<55:27:58,  8.27s/it]
 36%|███▋      | 13811/37962 [32:00:09<55:43:00,  8.31s/it]
 36%|███▋      | 13812/37962 [32:00:17<54:56:17,  8.19s/it]
 36%|███▋      | 13813/37962 [32:00:25<55:05:13,  8.21s/it]
 36%|███▋      | 13814/37962 [32:00:33<54:41:10,  8.15s/it]
 36%|███▋      | 13815/37962 [32:00:41<54:51:44,  8.18s/it]
 36%|███▋      | 13816/37962 [32:00:49<55:00:41,  8.20s/it]
 36%|███▋      | 13817/37962 [32:00:58<55:22:50,  8.26s/it]
 36%|███▋      | 13818/37962 [32:01:06<55:32:04,  8.28s/it]
 36%|███▋      | 13819/37962 [32:01:14<55:33:52,  8.29s/it]
 36%|███▋      | 13820/37962 [32:01:23<55:49:18,  8.32s/it]
                                                           
{'loss': 1.4388, 'grad_norm': 5.395107746124268, 'learning_rate': 1.4696582022908772e-05, 'epoch': 0.36}

 36%|███▋      | 13820/37962 [32:01:23<55:49:18,  8.32s/it]
 36%|███▋      | 13821/37962 [32:01:31<55:28:04,  8.27s/it]
 36%|███▋      | 13822/37962 [32:01:39<54:36:21,  8.14s/it]
 36%|███▋      | 13823/37962 [32:01:47<54:31:53,  8.13s/it]
 36%|███▋      | 13824/37962 [32:01:55<55:16:22,  8.24s/it]
 36%|███▋      | 13825/37962 [32:02:03<54:43:58,  8.16s/it]
 36%|███▋      | 13826/37962 [32:02:12<54:48:50,  8.18s/it]
 36%|███▋      | 13827/37962 [32:02:20<55:03:39,  8.21s/it]
 36%|███▋      | 13828/37962 [32:02:28<55:13:33,  8.24s/it]
 36%|███▋      | 13829/37962 [32:02:36<55:12:09,  8.23s/it]
 36%|███▋      | 13830/37962 [32:02:45<55:00:54,  8.21s/it]
                                                           
{'loss': 1.3243, 'grad_norm': 11.466837882995605, 'learning_rate': 1.468904820193211e-05, 'epoch': 0.36}

 36%|███▋      | 13830/37962 [32:02:45<55:00:54,  8.21s/it]
 36%|███▋      | 13831/37962 [32:02:53<55:24:36,  8.27s/it]
 36%|███▋      | 13832/37962 [32:03:01<55:26:09,  8.27s/it]
 36%|███▋      | 13833/37962 [32:03:10<55:38:23,  8.30s/it]
 36%|███▋      | 13834/37962 [32:03:18<55:16:17,  8.25s/it]
 36%|███▋      | 13835/37962 [32:03:26<55:16:57,  8.25s/it]
 36%|███▋      | 13836/37962 [32:03:34<55:14:52,  8.24s/it]
 36%|███▋      | 13837/37962 [32:03:42<55:12:07,  8.24s/it]
 36%|███▋      | 13838/37962 [32:03:51<55:31:05,  8.28s/it]
 36%|███▋      | 13839/37962 [32:03:59<55:49:36,  8.33s/it]
 36%|███▋      | 13840/37962 [32:04:07<54:34:00,  8.14s/it]
                                                           
{'loss': 1.4434, 'grad_norm': 6.657503128051758, 'learning_rate': 1.4681510967878491e-05, 'epoch': 0.36}

 36%|███▋      | 13840/37962 [32:04:07<54:34:00,  8.14s/it]
 36%|███▋      | 13841/37962 [32:04:15<54:47:29,  8.18s/it]
 36%|███▋      | 13842/37962 [32:04:24<55:10:31,  8.24s/it]
 36%|███▋      | 13843/37962 [32:04:39<69:28:18, 10.37s/it]
 36%|███▋      | 13844/37962 [32:04:47<65:09:15,  9.73s/it]
 36%|███▋      | 13845/37962 [32:04:55<62:17:05,  9.30s/it]
 36%|███▋      | 13846/37962 [32:05:04<60:09:29,  8.98s/it]
 36%|███▋      | 13847/37962 [32:05:12<59:06:19,  8.82s/it]
 36%|███▋      | 13848/37962 [32:05:21<58:14:39,  8.70s/it]
 36%|███▋      | 13849/37962 [32:05:29<57:25:58,  8.57s/it]
 36%|███▋      | 13850/37962 [32:05:37<56:39:34,  8.46s/it]
                                                           
{'loss': 1.4736, 'grad_norm': 10.612898826599121, 'learning_rate': 1.4673970326234143e-05, 'epoch': 0.36}

 36%|███▋      | 13850/37962 [32:05:37<56:39:34,  8.46s/it]
 36%|███▋      | 13851/37962 [32:05:45<56:09:45,  8.39s/it]
 36%|███▋      | 13852/37962 [32:05:53<55:17:19,  8.26s/it]
 36%|███▋      | 13853/37962 [32:06:02<55:37:42,  8.31s/it]
 36%|███▋      | 13854/37962 [32:06:10<55:29:52,  8.29s/it]
 36%|███▋      | 13855/37962 [32:06:18<55:21:04,  8.27s/it]
 36%|███▋      | 13856/37962 [32:06:26<55:19:31,  8.26s/it]
 37%|███▋      | 13857/37962 [32:06:35<55:32:45,  8.30s/it]
 37%|███▋      | 13858/37962 [32:06:43<55:23:19,  8.27s/it]
 37%|███▋      | 13859/37962 [32:06:51<55:14:37,  8.25s/it]
 37%|███▋      | 13860/37962 [32:06:59<54:04:27,  8.08s/it]
                                                           
{'loss': 1.4404, 'grad_norm': 13.192609786987305, 'learning_rate': 1.4666426282487765e-05, 'epoch': 0.37}

 37%|███▋      | 13860/37962 [32:06:59<54:04:27,  8.08s/it]
 37%|███▋      | 13861/37962 [32:07:07<54:32:12,  8.15s/it]
 37%|███▋      | 13862/37962 [32:07:15<54:46:59,  8.18s/it]
 37%|███▋      | 13863/37962 [32:07:24<55:03:47,  8.23s/it]
 37%|███▋      | 13864/37962 [32:07:32<55:34:35,  8.30s/it]
 37%|███▋      | 13865/37962 [32:07:41<55:47:16,  8.33s/it]
 37%|███▋      | 13866/37962 [32:07:49<56:00:03,  8.37s/it]
 37%|███▋      | 13867/37962 [32:07:57<55:45:13,  8.33s/it]
 37%|███▋      | 13868/37962 [32:08:05<54:56:36,  8.21s/it]
 37%|███▋      | 13869/37962 [32:08:14<55:05:12,  8.23s/it]
 37%|███▋      | 13870/37962 [32:08:22<55:27:56,  8.29s/it]
                                                           
{'loss': 1.4091, 'grad_norm': 5.312329292297363, 'learning_rate': 1.4658878842130535e-05, 'epoch': 0.37}

 37%|███▋      | 13870/37962 [32:08:22<55:27:56,  8.29s/it]
 37%|███▋      | 13871/37962 [32:08:30<55:22:38,  8.28s/it]
 37%|███▋      | 13872/37962 [32:08:38<55:14:02,  8.25s/it]
 37%|███▋      | 13873/37962 [32:08:46<54:56:33,  8.21s/it]
 37%|███▋      | 13874/37962 [32:08:55<55:12:18,  8.25s/it]
 37%|███▋      | 13875/37962 [32:09:03<54:59:29,  8.22s/it]
 37%|███▋      | 13876/37962 [32:09:11<54:56:55,  8.21s/it]
 37%|███▋      | 13877/37962 [32:09:19<55:02:24,  8.23s/it]
 37%|███▋      | 13878/37962 [32:09:28<55:00:31,  8.22s/it]
 37%|███▋      | 13879/37962 [32:09:36<55:02:24,  8.23s/it]
 37%|███▋      | 13880/37962 [32:09:44<55:11:41,  8.25s/it]
                                                           
{'loss': 1.3332, 'grad_norm': 20.19355583190918, 'learning_rate': 1.4651328010656105e-05, 'epoch': 0.37}

 37%|███▋      | 13880/37962 [32:09:44<55:11:41,  8.25s/it]
 37%|███▋      | 13881/37962 [32:09:52<55:02:37,  8.23s/it]
 37%|███▋      | 13882/37962 [32:10:01<54:57:47,  8.22s/it]
 37%|███▋      | 13883/37962 [32:10:09<55:17:00,  8.27s/it]
 37%|███▋      | 13884/37962 [32:10:17<55:45:09,  8.34s/it]
 37%|███▋      | 13885/37962 [32:10:26<55:29:39,  8.30s/it]
 37%|███▋      | 13886/37962 [32:10:34<55:27:46,  8.29s/it]
 37%|███▋      | 13887/37962 [32:10:42<55:31:16,  8.30s/it]
 37%|███▋      | 13888/37962 [32:10:51<55:29:01,  8.30s/it]
 37%|███▋      | 13889/37962 [32:10:59<55:18:04,  8.27s/it]
 37%|███▋      | 13890/37962 [32:11:07<55:13:39,  8.26s/it]
                                                           
{'loss': 1.3347, 'grad_norm': 5.760142803192139, 'learning_rate': 1.4643773793560599e-05, 'epoch': 0.37}

 37%|███▋      | 13890/37962 [32:11:07<55:13:39,  8.26s/it]
 37%|███▋      | 13891/37962 [32:11:15<54:54:09,  8.21s/it]
 37%|███▋      | 13892/37962 [32:11:23<54:59:33,  8.22s/it]
 37%|███▋      | 13893/37962 [32:11:32<55:02:39,  8.23s/it]
 37%|███▋      | 13894/37962 [32:11:40<55:17:07,  8.27s/it]
 37%|███▋      | 13895/37962 [32:11:48<54:45:55,  8.19s/it]
 37%|███▋      | 13896/37962 [32:11:56<54:55:08,  8.22s/it]
 37%|███▋      | 13897/37962 [32:12:12<69:15:33, 10.36s/it]
 37%|███▋      | 13898/37962 [32:12:20<65:02:19,  9.73s/it]
 37%|███▋      | 13899/37962 [32:12:28<62:29:22,  9.35s/it]
 37%|███▋      | 13900/37962 [32:12:37<60:35:16,  9.06s/it]
                                                           
{'loss': 1.2946, 'grad_norm': 5.5318827629089355, 'learning_rate': 1.4636216196342592e-05, 'epoch': 0.37}

 37%|███▋      | 13900/37962 [32:12:37<60:35:16,  9.06s/it]
 37%|███▋      | 13901/37962 [32:12:45<59:00:07,  8.83s/it]
 37%|███▋      | 13902/37962 [32:12:53<57:33:20,  8.61s/it]
 37%|███▋      | 13903/37962 [32:13:02<57:24:57,  8.59s/it]
 37%|███▋      | 13904/37962 [32:13:10<56:21:58,  8.43s/it]
 37%|███▋      | 13905/37962 [32:13:18<56:07:05,  8.40s/it]
 37%|███▋      | 13906/37962 [32:13:26<55:41:13,  8.33s/it]
 37%|███▋      | 13907/37962 [32:13:34<55:21:07,  8.28s/it]
 37%|███▋      | 13908/37962 [32:13:43<55:21:11,  8.28s/it]
 37%|███▋      | 13909/37962 [32:13:51<55:27:28,  8.30s/it]
 37%|███▋      | 13910/37962 [32:13:59<55:04:16,  8.24s/it]
                                                           
{'loss': 1.3397, 'grad_norm': 6.95716667175293, 'learning_rate': 1.4628655224503136e-05, 'epoch': 0.37}

 37%|███▋      | 13910/37962 [32:13:59<55:04:16,  8.24s/it]
 37%|███▋      | 13911/37962 [32:14:07<55:18:23,  8.28s/it]
 37%|███▋      | 13912/37962 [32:14:16<55:19:25,  8.28s/it]
 37%|███▋      | 13913/37962 [32:14:24<55:24:04,  8.29s/it]
 37%|███▋      | 13914/37962 [32:14:32<55:14:35,  8.27s/it]
 37%|███▋      | 13915/37962 [32:14:41<55:11:28,  8.26s/it]
 37%|███▋      | 13916/37962 [32:14:49<55:07:32,  8.25s/it]
 37%|███▋      | 13917/37962 [32:14:57<54:42:59,  8.19s/it]
 37%|███▋      | 13918/37962 [32:15:05<55:06:17,  8.25s/it]
 37%|███▋      | 13919/37962 [32:15:13<55:10:12,  8.26s/it]
 37%|███▋      | 13920/37962 [32:15:22<54:59:15,  8.23s/it]
                                                           
{'loss': 1.4193, 'grad_norm': 9.773702621459961, 'learning_rate': 1.4621090883545726e-05, 'epoch': 0.37}

 37%|███▋      | 13920/37962 [32:15:22<54:59:15,  8.23s/it]
 37%|███▋      | 13921/37962 [32:15:30<55:24:51,  8.30s/it]
 37%|███▋      | 13922/37962 [32:15:39<55:41:41,  8.34s/it]
 37%|███▋      | 13923/37962 [32:15:46<54:35:49,  8.18s/it]
 37%|███▋      | 13924/37962 [32:15:55<55:09:34,  8.26s/it]
 37%|███▋      | 13925/37962 [32:16:03<55:25:35,  8.30s/it]
 37%|███▋      | 13926/37962 [32:16:11<55:25:54,  8.30s/it]
 37%|███▋      | 13927/37962 [32:16:20<55:33:06,  8.32s/it]
 37%|███▋      | 13928/37962 [32:16:28<55:18:03,  8.28s/it]
 37%|███▋      | 13929/37962 [32:16:36<55:24:14,  8.30s/it]
 37%|███▋      | 13930/37962 [32:16:45<55:23:50,  8.30s/it]
                                                           
{'loss': 1.4935, 'grad_norm': 8.297050476074219, 'learning_rate': 1.4613523178976315e-05, 'epoch': 0.37}

 37%|███▋      | 13930/37962 [32:16:45<55:23:50,  8.30s/it]
 37%|███▋      | 13931/37962 [32:16:53<55:28:46,  8.31s/it]
 37%|███▋      | 13932/37962 [32:17:01<55:28:18,  8.31s/it]
 37%|███▋      | 13933/37962 [32:17:10<55:45:02,  8.35s/it]
 37%|███▋      | 13934/37962 [32:17:18<55:31:34,  8.32s/it]
 37%|███▋      | 13935/37962 [32:17:26<55:16:40,  8.28s/it]
 37%|███▋      | 13936/37962 [32:17:34<55:07:15,  8.26s/it]
 37%|███▋      | 13937/37962 [32:17:43<55:05:15,  8.25s/it]
 37%|███▋      | 13938/37962 [32:17:51<55:13:58,  8.28s/it]
 37%|███▋      | 13939/37962 [32:17:59<55:13:27,  8.28s/it]
 37%|███▋      | 13940/37962 [32:18:07<54:54:36,  8.23s/it]
                                                           
{'loss': 1.4406, 'grad_norm': 2.1775271892547607, 'learning_rate': 1.460595211630331e-05, 'epoch': 0.37}

 37%|███▋      | 13940/37962 [32:18:07<54:54:36,  8.23s/it]
 37%|███▋      | 13941/37962 [32:18:16<55:15:05,  8.28s/it]
 37%|███▋      | 13942/37962 [32:18:24<55:26:11,  8.31s/it]
 37%|███▋      | 13943/37962 [32:18:33<55:30:25,  8.32s/it]
 37%|███▋      | 13944/37962 [32:18:41<55:15:17,  8.28s/it]
 37%|███▋      | 13945/37962 [32:18:49<55:12:49,  8.28s/it]
 37%|███▋      | 13946/37962 [32:18:57<54:26:06,  8.16s/it]
 37%|███▋      | 13947/37962 [32:19:05<54:36:57,  8.19s/it]
 37%|███▋      | 13948/37962 [32:19:13<54:45:31,  8.21s/it]
 37%|███▋      | 13949/37962 [32:19:22<54:49:36,  8.22s/it]
 37%|███▋      | 13950/37962 [32:19:30<55:15:44,  8.29s/it]
                                                           
{'loss': 1.3526, 'grad_norm': 14.061629295349121, 'learning_rate': 1.459837770103755e-05, 'epoch': 0.37}

 37%|███▋      | 13950/37962 [32:19:30<55:15:44,  8.29s/it]
 37%|███▋      | 13951/37962 [32:19:38<54:31:34,  8.18s/it]
 37%|███▋      | 13952/37962 [32:19:46<54:34:45,  8.18s/it]
 37%|███▋      | 13953/37962 [32:19:54<54:34:24,  8.18s/it]
 37%|███▋      | 13954/37962 [32:20:03<54:53:20,  8.23s/it]
 37%|███▋      | 13955/37962 [32:20:11<55:18:41,  8.29s/it]
 37%|███▋      | 13956/37962 [32:20:20<55:32:39,  8.33s/it]
 37%|███▋      | 13957/37962 [32:20:27<54:40:46,  8.20s/it]
 37%|███▋      | 13958/37962 [32:20:36<54:35:18,  8.19s/it]
 37%|███▋      | 13959/37962 [32:20:44<54:34:11,  8.18s/it]
 37%|███▋      | 13960/37962 [32:20:52<54:59:21,  8.25s/it]
                                                           
{'loss': 1.3323, 'grad_norm': 5.127872467041016, 'learning_rate': 1.4590799938692322e-05, 'epoch': 0.37}

 37%|███▋      | 13960/37962 [32:20:52<54:59:21,  8.25s/it]
 37%|███▋      | 13961/37962 [32:21:00<55:05:39,  8.26s/it]
 37%|███▋      | 13962/37962 [32:21:09<55:22:56,  8.31s/it]
 37%|███▋      | 13963/37962 [32:21:17<55:03:52,  8.26s/it]
 37%|███▋      | 13964/37962 [32:21:25<54:48:03,  8.22s/it]
 37%|███▋      | 13965/37962 [32:21:34<55:04:21,  8.26s/it]
 37%|███▋      | 13966/37962 [32:21:42<55:20:52,  8.30s/it]
 37%|███▋      | 13967/37962 [32:21:50<55:18:27,  8.30s/it]
 37%|███▋      | 13968/37962 [32:21:58<55:11:46,  8.28s/it]
 37%|███▋      | 13969/37962 [32:22:07<55:22:38,  8.31s/it]
 37%|███▋      | 13970/37962 [32:22:15<55:52:48,  8.38s/it]
                                                           
{'loss': 1.4106, 'grad_norm': 7.264883518218994, 'learning_rate': 1.458321883478335e-05, 'epoch': 0.37}

 37%|███▋      | 13970/37962 [32:22:15<55:52:48,  8.38s/it]
 37%|███▋      | 13971/37962 [32:22:24<55:32:39,  8.33s/it]
 37%|███▋      | 13972/37962 [32:22:32<55:23:16,  8.31s/it]
 37%|███▋      | 13973/37962 [32:22:40<55:23:26,  8.31s/it]
 37%|███▋      | 13974/37962 [32:22:49<55:29:20,  8.33s/it]
 37%|███▋      | 13975/37962 [32:22:57<55:25:25,  8.32s/it]
 37%|███▋      | 13976/37962 [32:23:05<55:19:56,  8.30s/it]
 37%|███▋      | 13977/37962 [32:23:13<54:39:30,  8.20s/it]
 37%|███▋      | 13978/37962 [32:23:21<54:46:20,  8.22s/it]
 37%|███▋      | 13979/37962 [32:23:29<54:32:08,  8.19s/it]
 37%|███▋      | 13980/37962 [32:23:38<54:44:11,  8.22s/it]
                                                           
{'loss': 1.4023, 'grad_norm': 5.5468950271606445, 'learning_rate': 1.4575634394828785e-05, 'epoch': 0.37}

 37%|███▋      | 13980/37962 [32:23:38<54:44:11,  8.22s/it]
 37%|███▋      | 13981/37962 [32:23:46<54:50:27,  8.23s/it]
 37%|███▋      | 13982/37962 [32:23:54<54:56:07,  8.25s/it]
 37%|███▋      | 13983/37962 [32:24:03<55:05:22,  8.27s/it]
 37%|███▋      | 13984/37962 [32:24:11<55:09:22,  8.28s/it]
 37%|███▋      | 13985/37962 [32:24:19<55:19:34,  8.31s/it]
 37%|███▋      | 13986/37962 [32:24:27<54:27:58,  8.18s/it]
 37%|███▋      | 13987/37962 [32:24:36<55:00:03,  8.26s/it]
 37%|███▋      | 13988/37962 [32:24:44<54:39:10,  8.21s/it]
 37%|███▋      | 13989/37962 [32:24:52<54:49:40,  8.23s/it]
 37%|███▋      | 13990/37962 [32:25:00<55:02:28,  8.27s/it]
                                                           
{'loss': 1.4126, 'grad_norm': 4.172088146209717, 'learning_rate': 1.4568046624349215e-05, 'epoch': 0.37}

 37%|███▋      | 13990/37962 [32:25:00<55:02:28,  8.27s/it]
 37%|███▋      | 13991/37962 [32:25:09<54:59:56,  8.26s/it]
 37%|███▋      | 13992/37962 [32:25:17<54:50:22,  8.24s/it]
 37%|███▋      | 13993/37962 [32:25:25<54:49:24,  8.23s/it]
 37%|███▋      | 13994/37962 [32:25:33<55:00:36,  8.26s/it]
 37%|███▋      | 13995/37962 [32:25:41<54:21:33,  8.17s/it]
 37%|███▋      | 13996/37962 [32:25:49<53:47:09,  8.08s/it]
 37%|███▋      | 13997/37962 [32:25:57<54:19:27,  8.16s/it]
 37%|███▋      | 13998/37962 [32:26:06<54:23:39,  8.17s/it]
 37%|███▋      | 13999/37962 [32:26:14<55:02:05,  8.27s/it]
 37%|███▋      | 14000/37962 [32:26:22<54:28:40,  8.18s/it]
                                                           
{'loss': 1.3878, 'grad_norm': 2.0598316192626953, 'learning_rate': 1.456045552886764e-05, 'epoch': 0.37}

 37%|███▋      | 14000/37962 [32:26:22<54:28:40,  8.18s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 37%|███▋      | 14001/37962 [32:27:13<139:12:29, 20.92s/it]
 37%|███▋      | 14002/37962 [32:27:21<113:43:07, 17.09s/it]
 37%|███▋      | 14003/37962 [32:27:29<95:53:46, 14.41s/it] 
 37%|███▋      | 14004/37962 [32:27:38<84:03:43, 12.63s/it]
 37%|███▋      | 14005/37962 [32:27:46<75:28:12, 11.34s/it]
 37%|███▋      | 14006/37962 [32:27:54<68:55:46, 10.36s/it]
 37%|███▋      | 14007/37962 [32:28:02<65:11:25,  9.80s/it]
 37%|███▋      | 14008/37962 [32:28:11<62:08:48,  9.34s/it]
 37%|███▋      | 14009/37962 [32:28:19<59:35:48,  8.96s/it]
 37%|███▋      | 14010/37962 [32:28:27<58:06:22,  8.73s/it]
                                                           
{'loss': 1.5055, 'grad_norm': 7.383871078491211, 'learning_rate': 1.4552861113909491e-05, 'epoch': 0.37}

 37%|███▋      | 14010/37962 [32:28:27<58:06:22,  8.73s/it]
 37%|███▋      | 14011/37962 [32:28:36<57:45:21,  8.68s/it]
 37%|███▋      | 14012/37962 [32:28:44<56:46:21,  8.53s/it]
 37%|███▋      | 14013/37962 [32:28:52<56:18:59,  8.47s/it]
 37%|███▋      | 14014/37962 [32:29:00<55:53:36,  8.40s/it]
 37%|███▋      | 14015/37962 [32:29:09<55:37:58,  8.36s/it]
 37%|███▋      | 14016/37962 [32:29:16<54:41:32,  8.22s/it]
 37%|███▋      | 14017/37962 [32:29:25<54:51:53,  8.25s/it]
 37%|███▋      | 14018/37962 [32:29:33<55:12:45,  8.30s/it]
 37%|███▋      | 14019/37962 [32:29:42<55:23:01,  8.33s/it]
 37%|███▋      | 14020/37962 [32:29:50<55:57:05,  8.41s/it]
                                                           
{'loss': 1.3048, 'grad_norm': 3.057752847671509, 'learning_rate': 1.4545263385002612e-05, 'epoch': 0.37}

 37%|███▋      | 14020/37962 [32:29:50<55:57:05,  8.41s/it]
 37%|███▋      | 14021/37962 [32:29:59<56:30:04,  8.50s/it]
 37%|███▋      | 14022/37962 [32:30:07<56:16:17,  8.46s/it]
 37%|███▋      | 14023/37962 [32:30:15<55:36:42,  8.36s/it]
 37%|███▋      | 14024/37962 [32:30:24<55:37:43,  8.37s/it]
 37%|███▋      | 14025/37962 [32:30:32<55:38:54,  8.37s/it]
 37%|███▋      | 14026/37962 [32:30:40<55:02:02,  8.28s/it]
 37%|███▋      | 14027/37962 [32:30:48<54:34:27,  8.21s/it]
 37%|███▋      | 14028/37962 [32:30:57<54:34:52,  8.21s/it]
 37%|███▋      | 14029/37962 [32:31:05<54:50:15,  8.25s/it]
 37%|███▋      | 14030/37962 [32:31:13<54:31:26,  8.20s/it]
                                                           
{'loss': 1.3537, 'grad_norm': 10.646309852600098, 'learning_rate': 1.4537662347677252e-05, 'epoch': 0.37}

 37%|███▋      | 14030/37962 [32:31:13<54:31:26,  8.20s/it]
 37%|███▋      | 14031/37962 [32:31:21<54:14:52,  8.16s/it]
 37%|███▋      | 14032/37962 [32:31:29<53:48:32,  8.09s/it]
 37%|███▋      | 14033/37962 [32:31:37<54:22:14,  8.18s/it]
 37%|███▋      | 14034/37962 [32:31:46<54:36:59,  8.22s/it]
 37%|███▋      | 14035/37962 [32:31:54<54:36:44,  8.22s/it]
 37%|███▋      | 14036/37962 [32:32:02<54:48:17,  8.25s/it]
 37%|███▋      | 14037/37962 [32:32:11<55:14:24,  8.31s/it]
 37%|███▋      | 14038/37962 [32:32:19<55:27:49,  8.35s/it]
 37%|███▋      | 14039/37962 [32:32:28<55:42:15,  8.38s/it]
 37%|███▋      | 14040/37962 [32:32:36<55:56:12,  8.42s/it]
                                                           
{'loss': 1.3828, 'grad_norm': 12.763510704040527, 'learning_rate': 1.4530058007466086e-05, 'epoch': 0.37}

 37%|███▋      | 14040/37962 [32:32:36<55:56:12,  8.42s/it]
 37%|███▋      | 14041/37962 [32:32:44<55:44:18,  8.39s/it]
 37%|███▋      | 14042/37962 [32:32:53<55:48:23,  8.40s/it]
 37%|███▋      | 14043/37962 [32:33:01<55:39:42,  8.38s/it]
 37%|███▋      | 14044/37962 [32:33:09<55:26:09,  8.34s/it]
 37%|███▋      | 14045/37962 [32:33:18<55:28:54,  8.35s/it]
 37%|███▋      | 14046/37962 [32:33:26<54:37:22,  8.22s/it]
 37%|███▋      | 14047/37962 [32:33:33<53:49:58,  8.10s/it]
 37%|███▋      | 14048/37962 [32:33:42<53:42:33,  8.09s/it]
 37%|███▋      | 14049/37962 [32:33:50<54:10:15,  8.16s/it]
 37%|███▋      | 14050/37962 [32:33:58<54:33:46,  8.21s/it]
                                                           
{'loss': 1.5095, 'grad_norm': 9.680891036987305, 'learning_rate': 1.4522450369904169e-05, 'epoch': 0.37}

 37%|███▋      | 14050/37962 [32:33:58<54:33:46,  8.21s/it]
 37%|███▋      | 14051/37962 [32:34:06<54:40:36,  8.23s/it]
 37%|███▋      | 14052/37962 [32:34:15<54:56:05,  8.27s/it]
 37%|███▋      | 14053/37962 [32:34:23<55:15:05,  8.32s/it]
 37%|███▋      | 14054/37962 [32:34:32<55:13:17,  8.32s/it]
 37%|███▋      | 14055/37962 [32:34:40<55:17:10,  8.33s/it]
 37%|███▋      | 14056/37962 [32:34:48<55:21:00,  8.34s/it]
 37%|███▋      | 14057/37962 [32:34:57<55:39:17,  8.38s/it]
 37%|███▋      | 14058/37962 [32:35:05<55:44:45,  8.40s/it]
 37%|███▋      | 14059/37962 [32:35:13<55:23:11,  8.34s/it]
 37%|███▋      | 14060/37962 [32:35:22<55:06:42,  8.30s/it]
                                                           
{'loss': 1.3467, 'grad_norm': 12.855578422546387, 'learning_rate': 1.4514839440528973e-05, 'epoch': 0.37}

 37%|███▋      | 14060/37962 [32:35:22<55:06:42,  8.30s/it]
 37%|███▋      | 14061/37962 [32:35:30<55:00:45,  8.29s/it]
 37%|███▋      | 14062/37962 [32:35:38<55:08:32,  8.31s/it]
 37%|███▋      | 14063/37962 [32:35:46<54:56:53,  8.28s/it]
 37%|███▋      | 14064/37962 [32:35:55<54:45:12,  8.25s/it]
 37%|███▋      | 14065/37962 [32:36:10<69:12:45, 10.43s/it]
 37%|███▋      | 14066/37962 [32:36:18<65:05:34,  9.81s/it]
 37%|███▋      | 14067/37962 [32:36:26<61:27:30,  9.26s/it]
 37%|███▋      | 14068/37962 [32:36:35<59:43:34,  9.00s/it]
 37%|███▋      | 14069/37962 [32:36:50<72:46:42, 10.97s/it]
 37%|███▋      | 14070/37962 [32:36:59<67:43:38, 10.21s/it]
                                                           
{'loss': 1.2941, 'grad_norm': 10.740635871887207, 'learning_rate': 1.4507225224880363e-05, 'epoch': 0.37}

 37%|███▋      | 14070/37962 [32:36:59<67:43:38, 10.21s/it]
 37%|███▋      | 14071/37962 [32:37:07<63:52:58,  9.63s/it]
 37%|███▋      | 14072/37962 [32:37:15<61:25:51,  9.26s/it]
 37%|███▋      | 14073/37962 [32:37:24<59:39:31,  8.99s/it]
 37%|███▋      | 14074/37962 [32:37:32<58:37:42,  8.84s/it]
 37%|███▋      | 14075/37962 [32:37:41<57:27:46,  8.66s/it]
 37%|███▋      | 14076/37962 [32:37:49<56:51:45,  8.57s/it]
 37%|███▋      | 14077/37962 [32:37:57<56:30:57,  8.52s/it]
 37%|███▋      | 14078/37962 [32:38:06<56:06:09,  8.46s/it]
 37%|███▋      | 14079/37962 [32:38:14<55:43:09,  8.40s/it]
 37%|███▋      | 14080/37962 [32:38:22<55:30:39,  8.37s/it]
                                                           
{'loss': 1.4473, 'grad_norm': 10.499129295349121, 'learning_rate': 1.4499607728500598e-05, 'epoch': 0.37}

 37%|███▋      | 14080/37962 [32:38:22<55:30:39,  8.37s/it]
 37%|███▋      | 14081/37962 [32:38:31<55:28:49,  8.36s/it]
 37%|███▋      | 14082/37962 [32:38:39<55:12:15,  8.32s/it]
 37%|███▋      | 14083/37962 [32:38:47<55:01:13,  8.29s/it]
 37%|███▋      | 14084/37962 [32:38:55<54:38:02,  8.24s/it]
 37%|███▋      | 14085/37962 [32:39:03<54:49:30,  8.27s/it]
 37%|███▋      | 14086/37962 [32:39:12<54:52:18,  8.27s/it]
 37%|███▋      | 14087/37962 [32:39:20<55:00:56,  8.30s/it]
 37%|███▋      | 14088/37962 [32:39:29<55:16:24,  8.33s/it]
 37%|███▋      | 14089/37962 [32:39:37<55:27:59,  8.36s/it]
 37%|███▋      | 14090/37962 [32:39:45<54:48:43,  8.27s/it]
                                                           
{'loss': 1.3218, 'grad_norm': 10.872820854187012, 'learning_rate': 1.4491986956934315e-05, 'epoch': 0.37}

 37%|███▋      | 14090/37962 [32:39:45<54:48:43,  8.27s/it]
 37%|███▋      | 14091/37962 [32:39:53<54:48:49,  8.27s/it]
 37%|███▋      | 14092/37962 [32:40:02<55:06:52,  8.31s/it]
 37%|███▋      | 14093/37962 [32:40:10<54:57:45,  8.29s/it]
 37%|███▋      | 14094/37962 [32:40:18<54:13:04,  8.18s/it]
 37%|███▋      | 14095/37962 [32:40:26<54:13:50,  8.18s/it]
 37%|███▋      | 14096/37962 [32:40:34<53:32:31,  8.08s/it]
 37%|███▋      | 14097/37962 [32:40:42<53:14:15,  8.03s/it]
 37%|███▋      | 14098/37962 [32:40:50<53:57:39,  8.14s/it]
 37%|███▋      | 14099/37962 [32:40:58<53:59:59,  8.15s/it]
 37%|███▋      | 14100/37962 [32:41:07<54:21:34,  8.20s/it]
                                                           
{'loss': 1.3825, 'grad_norm': 8.625067710876465, 'learning_rate': 1.4484362915728543e-05, 'epoch': 0.37}

 37%|███▋      | 14100/37962 [32:41:07<54:21:34,  8.20s/it]
 37%|███▋      | 14101/37962 [32:41:15<54:28:07,  8.22s/it]
 37%|███▋      | 14102/37962 [32:41:23<54:46:57,  8.27s/it]
 37%|███▋      | 14103/37962 [32:41:32<55:07:21,  8.32s/it]
 37%|███▋      | 14104/37962 [32:41:40<55:06:49,  8.32s/it]
 37%|███▋      | 14105/37962 [32:41:48<54:34:52,  8.24s/it]
 37%|███▋      | 14106/37962 [32:41:57<55:00:44,  8.30s/it]
 37%|███▋      | 14107/37962 [32:42:05<55:25:29,  8.36s/it]
 37%|███▋      | 14108/37962 [32:42:13<54:59:15,  8.30s/it]
 37%|███▋      | 14109/37962 [32:42:21<54:53:03,  8.28s/it]
 37%|███▋      | 14110/37962 [32:42:30<55:04:04,  8.31s/it]
                                                           
{'loss': 1.3618, 'grad_norm': 13.40442180633545, 'learning_rate': 1.4476735610432693e-05, 'epoch': 0.37}

 37%|███▋      | 14110/37962 [32:42:30<55:04:04,  8.31s/it]
 37%|███▋      | 14111/37962 [32:42:38<54:57:55,  8.30s/it]
 37%|███▋      | 14112/37962 [32:42:46<54:51:39,  8.28s/it]
 37%|███▋      | 14113/37962 [32:42:55<54:43:33,  8.26s/it]
 37%|███▋      | 14114/37962 [32:43:03<54:42:24,  8.26s/it]
 37%|███▋      | 14115/37962 [32:43:11<54:36:25,  8.24s/it]
 37%|███▋      | 14116/37962 [32:43:19<54:19:18,  8.20s/it]
 37%|███▋      | 14117/37962 [32:43:27<54:36:33,  8.24s/it]
 37%|███▋      | 14118/37962 [32:43:36<54:36:39,  8.25s/it]
 37%|███▋      | 14119/37962 [32:43:44<55:00:02,  8.30s/it]
 37%|███▋      | 14120/37962 [32:43:52<54:39:51,  8.25s/it]
                                                           
{'loss': 1.4133, 'grad_norm': 15.46483325958252, 'learning_rate': 1.4469105046598546e-05, 'epoch': 0.37}

 37%|███▋      | 14120/37962 [32:43:52<54:39:51,  8.25s/it]
 37%|███▋      | 14121/37962 [32:44:00<54:23:48,  8.21s/it]
 37%|███▋      | 14122/37962 [32:44:09<54:58:09,  8.30s/it]
 37%|███▋      | 14123/37962 [32:44:17<54:59:28,  8.30s/it]
 37%|███▋      | 14124/37962 [32:44:25<54:39:32,  8.25s/it]
 37%|███▋      | 14125/37962 [32:44:34<54:49:43,  8.28s/it]
 37%|███▋      | 14126/37962 [32:44:42<54:55:57,  8.30s/it]
 37%|███▋      | 14127/37962 [32:44:50<54:56:53,  8.30s/it]
 37%|███▋      | 14128/37962 [32:44:58<53:54:49,  8.14s/it]
 37%|███▋      | 14129/37962 [32:45:06<54:21:19,  8.21s/it]
 37%|███▋      | 14130/37962 [32:45:14<53:43:39,  8.12s/it]
                                                           
{'loss': 1.3227, 'grad_norm': 10.819055557250977, 'learning_rate': 1.4461471229780255e-05, 'epoch': 0.37}

 37%|███▋      | 14130/37962 [32:45:14<53:43:39,  8.12s/it]
 37%|███▋      | 14131/37962 [32:45:23<53:53:55,  8.14s/it]
 37%|███▋      | 14132/37962 [32:45:31<54:06:03,  8.17s/it]
 37%|███▋      | 14133/37962 [32:45:39<54:31:53,  8.24s/it]
 37%|███▋      | 14134/37962 [32:45:47<54:34:42,  8.25s/it]
 37%|███▋      | 14135/37962 [32:45:56<54:20:41,  8.21s/it]
 37%|███▋      | 14136/37962 [32:46:04<54:10:05,  8.18s/it]
 37%|███▋      | 14137/37962 [32:46:12<54:18:38,  8.21s/it]
 37%|███▋      | 14138/37962 [32:46:20<53:42:52,  8.12s/it]
 37%|███▋      | 14139/37962 [32:46:28<54:04:36,  8.17s/it]
 37%|███▋      | 14140/37962 [32:46:37<54:28:32,  8.23s/it]
                                                           
{'loss': 1.3726, 'grad_norm': 90.15763092041016, 'learning_rate': 1.4453834165534348e-05, 'epoch': 0.37}

 37%|███▋      | 14140/37962 [32:46:37<54:28:32,  8.23s/it]
 37%|███▋      | 14141/37962 [32:46:45<54:33:36,  8.25s/it]
 37%|███▋      | 14142/37962 [32:46:53<54:27:24,  8.23s/it]
 37%|███▋      | 14143/37962 [32:47:01<54:31:51,  8.24s/it]
 37%|███▋      | 14144/37962 [32:47:10<54:35:09,  8.25s/it]
 37%|███▋      | 14145/37962 [32:47:18<54:35:40,  8.25s/it]
 37%|███▋      | 14146/37962 [32:47:26<54:12:03,  8.19s/it]
 37%|███▋      | 14147/37962 [32:47:34<54:14:34,  8.20s/it]
 37%|███▋      | 14148/37962 [32:47:42<54:11:23,  8.19s/it]
 37%|███▋      | 14149/37962 [32:47:50<53:41:22,  8.12s/it]
 37%|███▋      | 14150/37962 [32:47:58<53:59:12,  8.16s/it]
                                                           
{'loss': 1.3874, 'grad_norm': 121.56712341308594, 'learning_rate': 1.444619385941971e-05, 'epoch': 0.37}

 37%|███▋      | 14150/37962 [32:47:58<53:59:12,  8.16s/it]
 37%|███▋      | 14151/37962 [32:48:07<54:16:12,  8.21s/it]
 37%|███▋      | 14152/37962 [32:48:15<54:17:17,  8.21s/it]
 37%|███▋      | 14153/37962 [32:48:23<54:41:22,  8.27s/it]
 37%|███▋      | 14154/37962 [32:48:32<54:53:53,  8.30s/it]
 37%|███▋      | 14155/37962 [32:48:40<55:05:20,  8.33s/it]
 37%|███▋      | 14156/37962 [32:48:49<55:03:44,  8.33s/it]
 37%|███▋      | 14157/37962 [32:48:57<55:08:46,  8.34s/it]
 37%|███▋      | 14158/37962 [32:49:05<54:24:10,  8.23s/it]
 37%|███▋      | 14159/37962 [32:49:13<54:40:23,  8.27s/it]
 37%|███▋      | 14160/37962 [32:49:21<54:34:41,  8.25s/it]
                                                           
{'loss': 1.4003, 'grad_norm': 9.503137588500977, 'learning_rate': 1.4438550316997585e-05, 'epoch': 0.37}

 37%|███▋      | 14160/37962 [32:49:21<54:34:41,  8.25s/it]
 37%|███▋      | 14161/37962 [32:49:30<55:04:59,  8.33s/it]
 37%|███▋      | 14162/37962 [32:49:38<55:16:58,  8.36s/it]
 37%|███▋      | 14163/37962 [32:49:47<55:17:20,  8.36s/it]
 37%|███▋      | 14164/37962 [32:49:55<55:05:45,  8.33s/it]
 37%|███▋      | 14165/37962 [32:50:03<55:01:16,  8.32s/it]
 37%|███▋      | 14166/37962 [32:50:12<54:51:07,  8.30s/it]
 37%|███▋      | 14167/37962 [32:50:20<54:13:20,  8.20s/it]
 37%|███▋      | 14168/37962 [32:50:28<54:12:47,  8.20s/it]
 37%|███▋      | 14169/37962 [32:50:36<53:59:23,  8.17s/it]
 37%|███▋      | 14170/37962 [32:50:44<54:35:16,  8.26s/it]
                                                           
{'loss': 1.4342, 'grad_norm': 9.128656387329102, 'learning_rate': 1.4430903543831579e-05, 'epoch': 0.37}

 37%|███▋      | 14170/37962 [32:50:44<54:35:16,  8.26s/it]
 37%|███▋      | 14171/37962 [32:50:53<54:36:42,  8.26s/it]
 37%|███▋      | 14172/37962 [32:51:01<54:36:37,  8.26s/it]
 37%|███▋      | 14173/37962 [32:51:09<54:31:55,  8.25s/it]
 37%|███▋      | 14174/37962 [32:51:17<54:47:45,  8.29s/it]
 37%|███▋      | 14175/37962 [32:51:26<54:37:23,  8.27s/it]
 37%|███▋      | 14176/37962 [32:51:34<55:17:43,  8.37s/it]
 37%|███▋      | 14177/37962 [32:51:43<55:35:25,  8.41s/it]
 37%|███▋      | 14178/37962 [32:51:51<54:44:15,  8.29s/it]
 37%|███▋      | 14179/37962 [32:51:59<54:45:33,  8.29s/it]
 37%|███▋      | 14180/37962 [32:52:07<54:19:29,  8.22s/it]
                                                           
{'loss': 1.4971, 'grad_norm': 8.004496574401855, 'learning_rate': 1.442325354548764e-05, 'epoch': 0.37}

 37%|███▋      | 14180/37962 [32:52:07<54:19:29,  8.22s/it]
 37%|███▋      | 14181/37962 [32:52:15<54:15:03,  8.21s/it]
 37%|███▋      | 14182/37962 [32:52:24<54:20:07,  8.23s/it]
 37%|███▋      | 14183/37962 [32:52:32<54:48:29,  8.30s/it]
 37%|███▋      | 14184/37962 [32:52:40<54:36:25,  8.27s/it]
 37%|███▋      | 14185/37962 [32:52:49<54:38:51,  8.27s/it]
 37%|███▋      | 14186/37962 [32:52:57<54:46:35,  8.29s/it]
 37%|███▋      | 14187/37962 [32:53:05<54:48:44,  8.30s/it]
 37%|███▋      | 14188/37962 [32:53:14<55:04:39,  8.34s/it]
 37%|███▋      | 14189/37962 [32:53:22<54:59:31,  8.33s/it]
 37%|███▋      | 14190/37962 [32:53:30<55:08:38,  8.35s/it]
                                                           
{'loss': 1.2692, 'grad_norm': 18.990219116210938, 'learning_rate': 1.4415600327534076e-05, 'epoch': 0.37}

 37%|███▋      | 14190/37962 [32:53:30<55:08:38,  8.35s/it]
 37%|███▋      | 14191/37962 [32:53:39<54:51:32,  8.31s/it]
 37%|███▋      | 14192/37962 [32:53:47<54:43:17,  8.29s/it]
 37%|███▋      | 14193/37962 [32:53:55<54:09:17,  8.20s/it]
 37%|███▋      | 14194/37962 [32:54:03<54:21:43,  8.23s/it]
 37%|███▋      | 14195/37962 [32:54:12<54:53:15,  8.31s/it]
 37%|███▋      | 14196/37962 [32:54:20<55:08:52,  8.35s/it]
 37%|███▋      | 14197/37962 [32:54:28<55:08:24,  8.35s/it]
 37%|███▋      | 14198/37962 [32:54:37<55:01:47,  8.34s/it]
 37%|███▋      | 14199/37962 [32:54:45<54:58:34,  8.33s/it]
 37%|███▋      | 14200/37962 [32:54:53<54:50:13,  8.31s/it]
                                                           
{'loss': 1.4004, 'grad_norm': 3.104677200317383, 'learning_rate': 1.4407943895541527e-05, 'epoch': 0.37}

 37%|███▋      | 14200/37962 [32:54:53<54:50:13,  8.31s/it]
 37%|███▋      | 14201/37962 [32:55:01<53:47:33,  8.15s/it]
 37%|███▋      | 14202/37962 [32:55:09<53:22:54,  8.09s/it]
 37%|███▋      | 14203/37962 [32:55:25<68:07:08, 10.32s/it]
 37%|███▋      | 14204/37962 [32:55:33<64:35:29,  9.79s/it]
 37%|███▋      | 14205/37962 [32:55:41<61:23:44,  9.30s/it]
 37%|███▋      | 14206/37962 [32:55:50<59:29:26,  9.02s/it]
 37%|███▋      | 14207/37962 [32:55:58<58:09:17,  8.81s/it]
 37%|███▋      | 14208/37962 [32:56:06<57:35:17,  8.73s/it]
 37%|███▋      | 14209/37962 [32:56:15<56:36:25,  8.58s/it]
 37%|███▋      | 14210/37962 [32:56:23<56:13:45,  8.52s/it]
                                                           
{'loss': 1.3113, 'grad_norm': 12.727896690368652, 'learning_rate': 1.4400284255082979e-05, 'epoch': 0.37}

 37%|███▋      | 14210/37962 [32:56:23<56:13:45,  8.52s/it]
 37%|███▋      | 14211/37962 [32:56:31<55:47:23,  8.46s/it]
 37%|███▋      | 14212/37962 [32:56:40<55:33:36,  8.42s/it]
 37%|███▋      | 14213/37962 [32:56:48<55:18:27,  8.38s/it]
 37%|███▋      | 14214/37962 [32:56:56<54:55:28,  8.33s/it]
 37%|███▋      | 14215/37962 [32:57:05<54:57:24,  8.33s/it]
 37%|███▋      | 14216/37962 [32:57:13<54:46:09,  8.30s/it]
 37%|███▋      | 14217/37962 [32:57:21<54:47:18,  8.31s/it]
 37%|███▋      | 14218/37962 [32:57:29<54:42:03,  8.29s/it]
 37%|███▋      | 14219/37962 [32:57:38<54:53:21,  8.32s/it]
 37%|███▋      | 14220/37962 [32:57:46<54:34:44,  8.28s/it]
                                                           
{'loss': 1.4055, 'grad_norm': 3.225799798965454, 'learning_rate': 1.4392621411733753e-05, 'epoch': 0.37}

 37%|███▋      | 14220/37962 [32:57:46<54:34:44,  8.28s/it]
 37%|███▋      | 14221/37962 [32:57:54<54:26:34,  8.26s/it]
 37%|███▋      | 14222/37962 [32:58:02<54:32:52,  8.27s/it]
 37%|███▋      | 14223/37962 [32:58:11<54:56:14,  8.33s/it]
 37%|███▋      | 14224/37962 [32:58:19<54:53:30,  8.32s/it]
 37%|███▋      | 14225/37962 [32:58:27<54:39:34,  8.29s/it]
 37%|███▋      | 14226/37962 [32:58:36<54:45:59,  8.31s/it]
 37%|███▋      | 14227/37962 [32:58:44<54:56:55,  8.33s/it]
 37%|███▋      | 14228/37962 [32:58:52<54:40:35,  8.29s/it]
 37%|███▋      | 14229/37962 [32:59:00<54:17:34,  8.24s/it]
 37%|███▋      | 14230/37962 [32:59:08<53:41:35,  8.14s/it]
                                                           
{'loss': 1.3864, 'grad_norm': 8.016375541687012, 'learning_rate': 1.4384955371071501e-05, 'epoch': 0.37}

 37%|███▋      | 14230/37962 [32:59:08<53:41:35,  8.14s/it]
 37%|███▋      | 14231/37962 [32:59:17<53:55:36,  8.18s/it]
 37%|███▋      | 14232/37962 [32:59:25<54:36:35,  8.28s/it]
 37%|███▋      | 14233/37962 [32:59:33<54:02:48,  8.20s/it]
 37%|███▋      | 14234/37962 [32:59:41<54:06:16,  8.21s/it]
 37%|███▋      | 14235/37962 [32:59:49<53:17:10,  8.08s/it]
 38%|███▊      | 14236/37962 [32:59:58<53:42:13,  8.15s/it]
 38%|███▊      | 14237/37962 [33:00:06<54:00:57,  8.20s/it]
 38%|███▊      | 14238/37962 [33:00:14<54:09:24,  8.22s/it]
 38%|███▊      | 14239/37962 [33:00:22<54:23:27,  8.25s/it]
 38%|███▊      | 14240/37962 [33:00:31<54:33:02,  8.28s/it]
                                                           
{'loss': 1.3598, 'grad_norm': 2.8889567852020264, 'learning_rate': 1.437728613867619e-05, 'epoch': 0.38}

 38%|███▊      | 14240/37962 [33:00:31<54:33:02,  8.28s/it]
 38%|███▊      | 14241/37962 [33:00:39<53:50:40,  8.17s/it]
 38%|███▊      | 14242/37962 [33:00:47<53:30:26,  8.12s/it]
 38%|███▊      | 14243/37962 [33:00:55<53:48:22,  8.17s/it]
 38%|███▊      | 14244/37962 [33:01:03<54:32:48,  8.28s/it]
 38%|███▊      | 14245/37962 [33:01:12<54:26:26,  8.26s/it]
 38%|███▊      | 14246/37962 [33:01:20<54:25:02,  8.26s/it]
 38%|███▊      | 14247/37962 [33:01:28<54:32:37,  8.28s/it]
 38%|███▊      | 14248/37962 [33:01:37<54:32:54,  8.28s/it]
 38%|███▊      | 14249/37962 [33:01:45<54:46:45,  8.32s/it]
 38%|███▊      | 14250/37962 [33:01:53<54:45:29,  8.31s/it]
                                                           
{'loss': 1.4011, 'grad_norm': 5.612499237060547, 'learning_rate': 1.4369613720130138e-05, 'epoch': 0.38}

 38%|███▊      | 14250/37962 [33:01:53<54:45:29,  8.31s/it]
 38%|███▊      | 14251/37962 [33:02:02<54:41:42,  8.30s/it]
 38%|███▊      | 14252/37962 [33:02:10<54:47:27,  8.32s/it]
 38%|███▊      | 14253/37962 [33:02:18<54:45:21,  8.31s/it]
 38%|███▊      | 14254/37962 [33:02:27<54:59:05,  8.35s/it]
 38%|███▊      | 14255/37962 [33:02:35<55:01:29,  8.36s/it]
 38%|███▊      | 14256/37962 [33:02:43<54:21:01,  8.25s/it]
 38%|███▊      | 14257/37962 [33:02:51<54:33:48,  8.29s/it]
 38%|███▊      | 14258/37962 [33:03:00<54:29:30,  8.28s/it]
 38%|███▊      | 14259/37962 [33:03:08<53:55:02,  8.19s/it]
 38%|███▊      | 14260/37962 [33:03:16<53:54:17,  8.19s/it]
                                                           
{'loss': 1.3727, 'grad_norm': 4.765489101409912, 'learning_rate': 1.4361938121017953e-05, 'epoch': 0.38}

 38%|███▊      | 14260/37962 [33:03:16<53:54:17,  8.19s/it]
 38%|███▊      | 14261/37962 [33:03:24<54:10:41,  8.23s/it]
 38%|███▊      | 14262/37962 [33:03:32<53:55:38,  8.19s/it]
 38%|███▊      | 14263/37962 [33:03:41<54:01:01,  8.21s/it]
 38%|███▊      | 14264/37962 [33:03:49<54:11:03,  8.23s/it]
 38%|███▊      | 14265/37962 [33:03:57<53:42:42,  8.16s/it]
 38%|███▊      | 14266/37962 [33:04:05<53:52:47,  8.19s/it]
 38%|███▊      | 14267/37962 [33:04:13<54:08:03,  8.22s/it]
 38%|███▊      | 14268/37962 [33:04:22<54:33:01,  8.29s/it]
 38%|███▊      | 14269/37962 [33:04:30<54:27:10,  8.27s/it]
 38%|███▊      | 14270/37962 [33:04:38<54:40:23,  8.31s/it]
                                                           
{'loss': 1.3924, 'grad_norm': 6.395460605621338, 'learning_rate': 1.4354259346926576e-05, 'epoch': 0.38}

 38%|███▊      | 14270/37962 [33:04:38<54:40:23,  8.31s/it]
 38%|███▊      | 14271/37962 [33:04:47<54:44:35,  8.32s/it]
 38%|███▊      | 14272/37962 [33:04:55<54:46:36,  8.32s/it]
 38%|███▊      | 14273/37962 [33:05:03<54:52:24,  8.34s/it]
 38%|███▊      | 14274/37962 [33:05:12<55:16:09,  8.40s/it]
 38%|███▊      | 14275/37962 [33:05:20<55:00:56,  8.36s/it]
 38%|███▊      | 14276/37962 [33:05:29<55:08:09,  8.38s/it]
 38%|███▊      | 14277/37962 [33:05:37<54:50:50,  8.34s/it]
 38%|███▊      | 14278/37962 [33:05:45<54:13:44,  8.24s/it]
 38%|███▊      | 14279/37962 [33:05:53<54:41:15,  8.31s/it]
 38%|███▊      | 14280/37962 [33:06:02<54:14:47,  8.25s/it]
                                                           
{'loss': 1.5624, 'grad_norm': 6.062030792236328, 'learning_rate': 1.4346577403445255e-05, 'epoch': 0.38}

 38%|███▊      | 14280/37962 [33:06:02<54:14:47,  8.25s/it]
 38%|███▊      | 14281/37962 [33:06:10<54:04:01,  8.22s/it]
 38%|███▊      | 14282/37962 [33:06:18<54:03:17,  8.22s/it]
 38%|███▊      | 14283/37962 [33:06:26<54:14:08,  8.25s/it]
 38%|███▊      | 14284/37962 [33:06:34<54:16:26,  8.25s/it]
 38%|███▊      | 14285/37962 [33:06:43<54:21:04,  8.26s/it]
 38%|███▊      | 14286/37962 [33:06:51<53:28:47,  8.13s/it]
 38%|███▊      | 14287/37962 [33:06:59<54:14:50,  8.25s/it]
 38%|███▊      | 14288/37962 [33:07:08<55:12:14,  8.39s/it]
 38%|███▊      | 14289/37962 [33:07:16<54:35:52,  8.30s/it]
 38%|███▊      | 14290/37962 [33:07:24<54:36:35,  8.30s/it]
                                                           
{'loss': 1.2837, 'grad_norm': 5.744520664215088, 'learning_rate': 1.4338892296165538e-05, 'epoch': 0.38}

 38%|███▊      | 14290/37962 [33:07:24<54:36:35,  8.30s/it]
 38%|███▊      | 14291/37962 [33:07:33<55:06:05,  8.38s/it]
 38%|███▊      | 14292/37962 [33:07:41<54:58:56,  8.36s/it]
 38%|███▊      | 14293/37962 [33:07:49<54:42:30,  8.32s/it]
 38%|███▊      | 14294/37962 [33:07:58<54:36:41,  8.31s/it]
 38%|███▊      | 14295/37962 [33:08:06<55:04:54,  8.38s/it]
 38%|███▊      | 14296/37962 [33:08:14<54:40:49,  8.32s/it]
 38%|███▊      | 14297/37962 [33:08:23<54:32:24,  8.30s/it]
 38%|███▊      | 14298/37962 [33:08:31<54:29:44,  8.29s/it]
 38%|███▊      | 14299/37962 [33:08:39<54:47:50,  8.34s/it]
 38%|███▊      | 14300/37962 [33:08:48<54:52:16,  8.35s/it]
                                                           
{'loss': 1.4819, 'grad_norm': 16.997982025146484, 'learning_rate': 1.4331204030681285e-05, 'epoch': 0.38}

 38%|███▊      | 14300/37962 [33:08:48<54:52:16,  8.35s/it]
 38%|███▊      | 14301/37962 [33:08:56<54:51:30,  8.35s/it]
 38%|███▊      | 14302/37962 [33:09:04<54:39:16,  8.32s/it]
 38%|███▊      | 14303/37962 [33:09:13<54:56:37,  8.36s/it]
 38%|███▊      | 14304/37962 [33:09:21<54:36:29,  8.31s/it]
 38%|███▊      | 14305/37962 [33:09:29<54:10:56,  8.25s/it]
 38%|███▊      | 14306/37962 [33:09:37<53:58:46,  8.21s/it]
 38%|███▊      | 14307/37962 [33:09:45<54:09:31,  8.24s/it]
 38%|███▊      | 14308/37962 [33:09:54<54:04:33,  8.23s/it]
 38%|███▊      | 14309/37962 [33:10:02<54:07:02,  8.24s/it]
 38%|███▊      | 14310/37962 [33:10:10<54:09:30,  8.24s/it]
                                                           
{'loss': 1.4081, 'grad_norm': 7.562694549560547, 'learning_rate': 1.4323512612588655e-05, 'epoch': 0.38}

 38%|███▊      | 14310/37962 [33:10:10<54:09:30,  8.24s/it]
 38%|███▊      | 14311/37962 [33:10:18<53:25:07,  8.13s/it]
 38%|███▊      | 14312/37962 [33:10:27<54:06:30,  8.24s/it]
 38%|███▊      | 14313/37962 [33:10:35<54:25:16,  8.28s/it]
 38%|███▊      | 14314/37962 [33:10:43<53:33:44,  8.15s/it]
 38%|███▊      | 14315/37962 [33:10:51<53:33:37,  8.15s/it]
 38%|███▊      | 14316/37962 [33:10:59<53:52:48,  8.20s/it]
 38%|███▊      | 14317/37962 [33:11:07<53:52:33,  8.20s/it]
 38%|███▊      | 14318/37962 [33:11:16<53:50:05,  8.20s/it]
 38%|███▊      | 14319/37962 [33:11:24<53:27:36,  8.14s/it]
 38%|███▊      | 14320/37962 [33:11:32<54:09:42,  8.25s/it]
                                                           
{'loss': 1.3309, 'grad_norm': 5.358477592468262, 'learning_rate': 1.431581804748609e-05, 'epoch': 0.38}

 38%|███▊      | 14320/37962 [33:11:32<54:09:42,  8.25s/it]
 38%|███▊      | 14321/37962 [33:11:40<54:06:38,  8.24s/it]
 38%|███▊      | 14322/37962 [33:11:49<54:04:55,  8.24s/it]
 38%|███▊      | 14323/37962 [33:11:57<54:05:47,  8.24s/it]
 38%|███▊      | 14324/37962 [33:12:05<54:27:35,  8.29s/it]
 38%|███▊      | 14325/37962 [33:12:13<54:00:03,  8.22s/it]
 38%|███▊      | 14326/37962 [33:12:22<53:58:44,  8.22s/it]
 38%|███▊      | 14327/37962 [33:12:30<53:57:01,  8.22s/it]
 38%|███▊      | 14328/37962 [33:12:38<54:02:32,  8.23s/it]
 38%|███▊      | 14329/37962 [33:12:46<54:19:37,  8.28s/it]
 38%|███▊      | 14330/37962 [33:12:54<53:10:28,  8.10s/it]
                                                           
{'loss': 1.205, 'grad_norm': 3.4188365936279297, 'learning_rate': 1.4308120340974339e-05, 'epoch': 0.38}

 38%|███▊      | 14330/37962 [33:12:54<53:10:28,  8.10s/it]
 38%|███▊      | 14331/37962 [33:13:02<53:36:15,  8.17s/it]
 38%|███▊      | 14332/37962 [33:13:11<54:10:24,  8.25s/it]
 38%|███▊      | 14333/37962 [33:13:19<53:49:19,  8.20s/it]
 38%|███▊      | 14334/37962 [33:13:27<54:04:18,  8.24s/it]
 38%|███▊      | 14335/37962 [33:13:36<54:05:50,  8.24s/it]
 38%|███▊      | 14336/37962 [33:13:44<54:06:00,  8.24s/it]
 38%|███▊      | 14337/37962 [33:13:52<54:24:50,  8.29s/it]
 38%|███▊      | 14338/37962 [33:14:00<54:16:01,  8.27s/it]
 38%|███▊      | 14339/37962 [33:14:08<53:40:27,  8.18s/it]
 38%|███▊      | 14340/37962 [33:14:17<53:40:09,  8.18s/it]
                                                           
{'loss': 1.3466, 'grad_norm': 17.949207305908203, 'learning_rate': 1.4300419498656419e-05, 'epoch': 0.38}

 38%|███▊      | 14340/37962 [33:14:17<53:40:09,  8.18s/it]
 38%|███▊      | 14341/37962 [33:14:25<54:37:43,  8.33s/it]
 38%|███▊      | 14342/37962 [33:14:33<54:02:43,  8.24s/it]
 38%|███▊      | 14343/37962 [33:14:41<54:02:50,  8.24s/it]
 38%|███▊      | 14344/37962 [33:14:50<53:52:47,  8.21s/it]
 38%|███▊      | 14345/37962 [33:14:58<54:26:16,  8.30s/it]
 38%|███▊      | 14346/37962 [33:15:06<54:24:06,  8.29s/it]
 38%|███▊      | 14347/37962 [33:15:15<54:12:07,  8.26s/it]
 38%|███▊      | 14348/37962 [33:15:23<54:15:21,  8.27s/it]
 38%|███▊      | 14349/37962 [33:15:31<54:14:14,  8.27s/it]
 38%|███▊      | 14350/37962 [33:15:40<54:25:21,  8.30s/it]
                                                           
{'loss': 1.4071, 'grad_norm': 12.3236722946167, 'learning_rate': 1.4292715526137646e-05, 'epoch': 0.38}

 38%|███▊      | 14350/37962 [33:15:40<54:25:21,  8.30s/it]
 38%|███▊      | 14351/37962 [33:15:48<54:39:01,  8.33s/it]
 38%|███▊      | 14352/37962 [33:15:56<54:43:42,  8.34s/it]
 38%|███▊      | 14353/37962 [33:16:05<54:38:01,  8.33s/it]
 38%|███▊      | 14354/37962 [33:16:13<54:38:03,  8.33s/it]
 38%|███▊      | 14355/37962 [33:16:21<54:52:41,  8.37s/it]
 38%|███▊      | 14356/37962 [33:16:30<55:11:56,  8.42s/it]
 38%|███▊      | 14357/37962 [33:16:38<54:53:12,  8.37s/it]
 38%|███▊      | 14358/37962 [33:16:47<55:06:44,  8.41s/it]
 38%|███▊      | 14359/37962 [33:16:55<54:05:07,  8.25s/it]
 38%|███▊      | 14360/37962 [33:17:03<54:23:23,  8.30s/it]
                                                           
{'loss': 1.3305, 'grad_norm': 9.832478523254395, 'learning_rate': 1.4285008429025613e-05, 'epoch': 0.38}

 38%|███▊      | 14360/37962 [33:17:03<54:23:23,  8.30s/it]
 38%|███▊      | 14361/37962 [33:17:11<53:35:30,  8.17s/it]
 38%|███▊      | 14362/37962 [33:17:19<53:58:39,  8.23s/it]
 38%|███▊      | 14363/37962 [33:17:27<54:00:45,  8.24s/it]
 38%|███▊      | 14364/37962 [33:17:36<54:12:53,  8.27s/it]
 38%|███▊      | 14365/37962 [33:17:44<54:23:55,  8.30s/it]
 38%|███▊      | 14366/37962 [33:17:53<54:28:19,  8.31s/it]
 38%|███▊      | 14367/37962 [33:18:01<54:14:30,  8.28s/it]
 38%|███▊      | 14368/37962 [33:18:09<53:58:20,  8.24s/it]
 38%|███▊      | 14369/37962 [33:18:17<52:53:36,  8.07s/it]
 38%|███▊      | 14370/37962 [33:18:25<53:09:38,  8.11s/it]
                                                           
{'loss': 1.3967, 'grad_norm': 7.953855037689209, 'learning_rate': 1.4277298212930172e-05, 'epoch': 0.38}

 38%|███▊      | 14370/37962 [33:18:25<53:09:38,  8.11s/it]
 38%|███▊      | 14371/37962 [33:18:33<53:47:59,  8.21s/it]
 38%|███▊      | 14372/37962 [33:18:41<53:59:01,  8.24s/it]
 38%|███▊      | 14373/37962 [33:18:50<53:39:24,  8.19s/it]
 38%|███▊      | 14374/37962 [33:18:58<53:43:35,  8.20s/it]
 38%|███▊      | 14375/37962 [33:19:06<53:43:04,  8.20s/it]
 38%|███▊      | 14376/37962 [33:19:14<53:50:02,  8.22s/it]
 38%|███▊      | 14377/37962 [33:19:23<54:00:55,  8.24s/it]
 38%|███▊      | 14378/37962 [33:19:31<53:39:52,  8.19s/it]
 38%|███▊      | 14379/37962 [33:19:39<53:56:45,  8.23s/it]
 38%|███▊      | 14380/37962 [33:19:47<53:41:13,  8.20s/it]
                                                           
{'loss': 1.4285, 'grad_norm': 8.537074089050293, 'learning_rate': 1.4269584883463466e-05, 'epoch': 0.38}

 38%|███▊      | 14380/37962 [33:19:47<53:41:13,  8.20s/it]
 38%|███▊      | 14381/37962 [33:19:55<53:58:07,  8.24s/it]
 38%|███▊      | 14382/37962 [33:20:04<54:21:51,  8.30s/it]
 38%|███▊      | 14383/37962 [33:20:12<54:36:03,  8.34s/it]
 38%|███▊      | 14384/37962 [33:20:20<54:18:36,  8.29s/it]
 38%|███▊      | 14385/37962 [33:20:28<53:25:34,  8.16s/it]
 38%|███▊      | 14386/37962 [33:20:37<53:47:16,  8.21s/it]
 38%|███▊      | 14387/37962 [33:20:45<53:55:20,  8.23s/it]
 38%|███▊      | 14388/37962 [33:20:53<54:09:54,  8.27s/it]
 38%|███▊      | 14389/37962 [33:21:02<54:07:39,  8.27s/it]
 38%|███▊      | 14390/37962 [33:21:17<68:23:56, 10.45s/it]
                                                           
{'loss': 1.3092, 'grad_norm': 27.083322525024414, 'learning_rate': 1.4261868446239886e-05, 'epoch': 0.38}

 38%|███▊      | 14390/37962 [33:21:17<68:23:56, 10.45s/it]
 38%|███▊      | 14391/37962 [33:21:25<63:39:59,  9.72s/it]
 38%|███▊      | 14392/37962 [33:21:33<60:38:51,  9.26s/it]
 38%|███▊      | 14393/37962 [33:21:41<58:06:56,  8.88s/it]
 38%|███▊      | 14394/37962 [33:21:49<56:43:52,  8.67s/it]
 38%|███▊      | 14395/37962 [33:21:58<55:53:58,  8.54s/it]
 38%|███▊      | 14396/37962 [33:22:06<55:24:18,  8.46s/it]
 38%|███▊      | 14397/37962 [33:22:14<54:42:31,  8.36s/it]
 38%|███▊      | 14398/37962 [33:22:23<54:54:09,  8.39s/it]
 38%|███▊      | 14399/37962 [33:22:31<54:27:50,  8.32s/it]
 38%|███▊      | 14400/37962 [33:22:39<53:49:18,  8.22s/it]
                                                           
{'loss': 1.4678, 'grad_norm': 20.45146942138672, 'learning_rate': 1.4254148906876095e-05, 'epoch': 0.38}

 38%|███▊      | 14400/37962 [33:22:39<53:49:18,  8.22s/it]
 38%|███▊      | 14401/37962 [33:22:47<53:27:42,  8.17s/it]
 38%|███▊      | 14402/37962 [33:22:55<53:50:38,  8.23s/it]
 38%|███▊      | 14403/37962 [33:23:03<53:59:32,  8.25s/it]
 38%|███▊      | 14404/37962 [33:23:12<54:28:11,  8.32s/it]
 38%|███▊      | 14405/37962 [33:23:20<54:17:35,  8.30s/it]
 38%|███▊      | 14406/37962 [33:23:28<54:08:52,  8.28s/it]
 38%|███▊      | 14407/37962 [33:23:37<54:21:03,  8.31s/it]
 38%|███▊      | 14408/37962 [33:23:45<54:10:07,  8.28s/it]
 38%|███▊      | 14409/37962 [33:23:53<53:19:17,  8.15s/it]
 38%|███▊      | 14410/37962 [33:24:01<53:05:59,  8.12s/it]
                                                           
{'loss': 1.3874, 'grad_norm': 5.861148357391357, 'learning_rate': 1.4246426270991014e-05, 'epoch': 0.38}

 38%|███▊      | 14410/37962 [33:24:01<53:05:59,  8.12s/it]
 38%|███▊      | 14411/37962 [33:24:09<52:55:41,  8.09s/it]
 38%|███▊      | 14412/37962 [33:24:17<53:20:23,  8.15s/it]
 38%|███▊      | 14413/37962 [33:24:25<53:28:07,  8.17s/it]
 38%|███▊      | 14414/37962 [33:24:33<52:59:51,  8.10s/it]
 38%|███▊      | 14415/37962 [33:24:42<53:45:10,  8.22s/it]
 38%|███▊      | 14416/37962 [33:24:50<53:57:08,  8.25s/it]
 38%|███▊      | 14417/37962 [33:24:58<53:40:41,  8.21s/it]
 38%|███▊      | 14418/37962 [33:25:06<53:37:17,  8.20s/it]
 38%|███▊      | 14419/37962 [33:25:15<53:50:23,  8.23s/it]
 38%|███▊      | 14420/37962 [33:25:23<53:54:33,  8.24s/it]
                                                           
{'loss': 1.501, 'grad_norm': 43.94831085205078, 'learning_rate': 1.4238700544205814e-05, 'epoch': 0.38}

 38%|███▊      | 14420/37962 [33:25:23<53:54:33,  8.24s/it]
 38%|███▊      | 14421/37962 [33:25:31<54:10:24,  8.28s/it]
 38%|███▊      | 14422/37962 [33:25:40<54:18:21,  8.31s/it]
 38%|███▊      | 14423/37962 [33:25:48<53:58:37,  8.26s/it]
 38%|███▊      | 14424/37962 [33:25:56<54:01:36,  8.26s/it]
 38%|███▊      | 14425/37962 [33:26:04<54:08:12,  8.28s/it]
 38%|███▊      | 14426/37962 [33:26:13<53:55:24,  8.25s/it]
 38%|███▊      | 14427/37962 [33:26:21<54:09:06,  8.28s/it]
 38%|███▊      | 14428/37962 [33:26:29<54:00:25,  8.26s/it]
 38%|███▊      | 14429/37962 [33:26:38<54:06:05,  8.28s/it]
 38%|███▊      | 14430/37962 [33:26:46<53:57:32,  8.25s/it]
                                                           
{'loss': 1.3529, 'grad_norm': 94.89093780517578, 'learning_rate': 1.423097173214392e-05, 'epoch': 0.38}

 38%|███▊      | 14430/37962 [33:26:46<53:57:32,  8.25s/it]
 38%|███▊      | 14431/37962 [33:26:54<53:45:26,  8.22s/it]
 38%|███▊      | 14432/37962 [33:27:02<53:51:09,  8.24s/it]
 38%|███▊      | 14433/37962 [33:27:11<54:33:42,  8.35s/it]
 38%|███▊      | 14434/37962 [33:27:19<54:16:30,  8.30s/it]
 38%|███▊      | 14435/37962 [33:27:27<54:10:07,  8.29s/it]
 38%|███▊      | 14436/37962 [33:27:36<54:18:35,  8.31s/it]
 38%|███▊      | 14437/37962 [33:27:44<55:02:32,  8.42s/it]
 38%|███▊      | 14438/37962 [33:27:52<54:31:01,  8.34s/it]
 38%|███▊      | 14439/37962 [33:28:00<53:50:13,  8.24s/it]
 38%|███▊      | 14440/37962 [33:28:09<54:05:33,  8.28s/it]
                                                           
{'loss': 1.5298, 'grad_norm': 11.575323104858398, 'learning_rate': 1.4223239840430996e-05, 'epoch': 0.38}

 38%|███▊      | 14440/37962 [33:28:09<54:05:33,  8.28s/it]
 38%|███▊      | 14441/37962 [33:28:17<53:58:05,  8.26s/it]
 38%|███▊      | 14442/37962 [33:28:25<54:11:46,  8.30s/it]
 38%|███▊      | 14443/37962 [33:28:34<54:15:04,  8.30s/it]
 38%|███▊      | 14444/37962 [33:28:42<53:53:19,  8.25s/it]
 38%|███▊      | 14445/37962 [33:28:50<53:48:04,  8.24s/it]
 38%|███▊      | 14446/37962 [33:28:58<53:53:21,  8.25s/it]
 38%|███▊      | 14447/37962 [33:29:07<53:57:20,  8.26s/it]
 38%|███▊      | 14448/37962 [33:29:15<54:27:50,  8.34s/it]
 38%|███▊      | 14449/37962 [33:29:23<54:13:15,  8.30s/it]
 38%|███▊      | 14450/37962 [33:29:32<54:10:44,  8.30s/it]
                                                           
{'loss': 1.3726, 'grad_norm': 192.29547119140625, 'learning_rate': 1.4215504874694957e-05, 'epoch': 0.38}

 38%|███▊      | 14450/37962 [33:29:32<54:10:44,  8.30s/it]
 38%|███▊      | 14451/37962 [33:29:40<53:28:30,  8.19s/it]
 38%|███▊      | 14452/37962 [33:29:48<53:40:20,  8.22s/it]
 38%|███▊      | 14453/37962 [33:29:56<53:47:49,  8.24s/it]
 38%|███▊      | 14454/37962 [33:30:05<54:16:58,  8.31s/it]
 38%|███▊      | 14455/37962 [33:30:13<54:04:05,  8.28s/it]
 38%|███▊      | 14456/37962 [33:30:21<53:43:25,  8.23s/it]
 38%|███▊      | 14457/37962 [33:30:29<53:45:15,  8.23s/it]
 38%|███▊      | 14458/37962 [33:30:37<53:48:45,  8.24s/it]
 38%|███▊      | 14459/37962 [33:30:46<53:42:26,  8.23s/it]
 38%|███▊      | 14460/37962 [33:30:54<54:03:39,  8.28s/it]
                                                           
{'loss': 1.9216, 'grad_norm': 28.046993255615234, 'learning_rate': 1.4207766840565949e-05, 'epoch': 0.38}

 38%|███▊      | 14460/37962 [33:30:54<54:03:39,  8.28s/it]
 38%|███▊      | 14461/37962 [33:31:02<54:10:35,  8.30s/it]
 38%|███▊      | 14462/37962 [33:31:11<54:25:02,  8.34s/it]
 38%|███▊      | 14463/37962 [33:31:19<54:45:10,  8.39s/it]
 38%|███▊      | 14464/37962 [33:31:28<54:35:36,  8.36s/it]
 38%|███▊      | 14465/37962 [33:31:36<54:48:47,  8.40s/it]
 38%|███▊      | 14466/37962 [33:31:44<54:35:55,  8.37s/it]
 38%|███▊      | 14467/37962 [33:31:53<54:12:39,  8.31s/it]
 38%|███▊      | 14468/37962 [33:32:01<54:16:45,  8.32s/it]
 38%|███▊      | 14469/37962 [33:32:09<54:07:14,  8.29s/it]
 38%|███▊      | 14470/37962 [33:32:17<53:52:21,  8.26s/it]
                                                           
{'loss': 1.9393, 'grad_norm': 54.38966751098633, 'learning_rate': 1.4200025743676348e-05, 'epoch': 0.38}

 38%|███▊      | 14470/37962 [33:32:17<53:52:21,  8.26s/it]
 38%|███▊      | 14471/37962 [33:32:25<52:59:46,  8.12s/it]
 38%|███▊      | 14472/37962 [33:32:33<53:12:24,  8.15s/it]
 38%|███▊      | 14473/37962 [33:32:42<53:30:30,  8.20s/it]
 38%|███▊      | 14474/37962 [33:32:50<53:49:31,  8.25s/it]
 38%|███▊      | 14475/37962 [33:32:58<53:47:01,  8.24s/it]
 38%|███▊      | 14476/37962 [33:33:07<54:12:27,  8.31s/it]
 38%|███▊      | 14477/37962 [33:33:15<54:09:11,  8.30s/it]
 38%|███▊      | 14478/37962 [33:33:23<54:10:43,  8.31s/it]
 38%|███▊      | 14479/37962 [33:33:32<54:23:11,  8.34s/it]
 38%|███▊      | 14480/37962 [33:33:40<54:14:49,  8.32s/it]
                                                           
{'loss': 5.9251, 'grad_norm': 20.182350158691406, 'learning_rate': 1.4192281589660771e-05, 'epoch': 0.38}

 38%|███▊      | 14480/37962 [33:33:40<54:14:49,  8.32s/it]
 38%|███▊      | 14481/37962 [33:33:48<53:59:09,  8.28s/it]
 38%|███▊      | 14482/37962 [33:33:57<54:16:21,  8.32s/it]
 38%|███▊      | 14483/37962 [33:34:05<54:22:27,  8.34s/it]
 38%|███▊      | 14484/37962 [33:34:13<54:12:37,  8.31s/it]
 38%|███▊      | 14485/37962 [33:34:29<68:49:30, 10.55s/it]
 38%|███▊      | 14486/37962 [33:34:37<64:29:49,  9.89s/it]
 38%|███▊      | 14487/37962 [33:34:46<61:28:07,  9.43s/it]
 38%|███▊      | 14488/37962 [33:34:54<58:55:51,  9.04s/it]
 38%|███▊      | 14489/37962 [33:35:02<57:36:33,  8.84s/it]
 38%|███▊      | 14490/37962 [33:35:10<56:30:04,  8.67s/it]
                                                           
{'loss': 1.5837, 'grad_norm': 47.746761322021484, 'learning_rate': 1.4184534384156053e-05, 'epoch': 0.38}

 38%|███▊      | 14490/37962 [33:35:10<56:30:04,  8.67s/it]
 38%|███▊      | 14491/37962 [33:35:19<55:54:10,  8.57s/it]
 38%|███▊      | 14492/37962 [33:35:27<55:01:25,  8.44s/it]
 38%|███▊      | 14493/37962 [33:35:35<54:49:11,  8.41s/it]
 38%|███▊      | 14494/37962 [33:35:44<54:43:07,  8.39s/it]
 38%|███▊      | 14495/37962 [33:35:52<54:45:11,  8.40s/it]
 38%|███▊      | 14496/37962 [33:36:01<54:51:17,  8.42s/it]
 38%|███▊      | 14497/37962 [33:36:09<54:11:22,  8.31s/it]
 38%|███▊      | 14498/37962 [33:36:17<54:06:39,  8.30s/it]
 38%|███▊      | 14499/37962 [33:36:25<54:09:20,  8.31s/it]
 38%|███▊      | 14500/37962 [33:36:33<53:40:38,  8.24s/it]
                                                           
{'loss': 1.4632, 'grad_norm': 38.669559478759766, 'learning_rate': 1.4176784132801248e-05, 'epoch': 0.38}

 38%|███▊      | 14500/37962 [33:36:33<53:40:38,  8.24s/it]
 38%|███▊      | 14501/37962 [33:36:42<53:51:25,  8.26s/it]
 38%|███▊      | 14502/37962 [33:36:50<53:51:54,  8.27s/it]
 38%|███▊      | 14503/37962 [33:36:58<53:07:38,  8.15s/it]
 38%|███▊      | 14504/37962 [33:37:06<52:29:43,  8.06s/it]
 38%|███▊      | 14505/37962 [33:37:14<52:53:59,  8.12s/it]
 38%|███▊      | 14506/37962 [33:37:22<53:08:54,  8.16s/it]
 38%|███▊      | 14507/37962 [33:37:30<53:21:35,  8.19s/it]
 38%|███▊      | 14508/37962 [33:37:39<53:59:43,  8.29s/it]
 38%|███▊      | 14509/37962 [33:37:47<53:56:30,  8.28s/it]
 38%|███▊      | 14510/37962 [33:37:55<53:48:08,  8.26s/it]
                                                           
{'loss': 1.3411, 'grad_norm': 35.13742446899414, 'learning_rate': 1.416903084123763e-05, 'epoch': 0.38}

 38%|███▊      | 14510/37962 [33:37:55<53:48:08,  8.26s/it]
 38%|███▊      | 14511/37962 [33:38:04<53:50:22,  8.26s/it]
 38%|███▊      | 14512/37962 [33:38:12<53:53:42,  8.27s/it]
 38%|███▊      | 14513/37962 [33:38:20<53:29:20,  8.21s/it]
 38%|███▊      | 14514/37962 [33:38:28<53:05:04,  8.15s/it]
 38%|███▊      | 14515/37962 [33:38:36<52:49:40,  8.11s/it]
 38%|███▊      | 14516/37962 [33:38:44<53:21:40,  8.19s/it]
 38%|███▊      | 14517/37962 [33:38:53<53:35:14,  8.23s/it]
 38%|███▊      | 14518/37962 [33:39:01<54:02:12,  8.30s/it]
 38%|███▊      | 14519/37962 [33:39:09<53:24:17,  8.20s/it]
 38%|███▊      | 14520/37962 [33:39:17<53:24:20,  8.20s/it]
                                                           
{'loss': 1.5661, 'grad_norm': 26.411706924438477, 'learning_rate': 1.4161274515108686e-05, 'epoch': 0.38}

 38%|███▊      | 14520/37962 [33:39:17<53:24:20,  8.20s/it]
 38%|███▊      | 14521/37962 [33:39:25<53:05:21,  8.15s/it]
 38%|███▊      | 14522/37962 [33:39:34<53:19:36,  8.19s/it]
 38%|███▊      | 14523/37962 [33:39:42<53:21:00,  8.19s/it]
 38%|███▊      | 14524/37962 [33:39:50<53:31:36,  8.22s/it]
 38%|███▊      | 14525/37962 [33:39:59<53:48:38,  8.27s/it]
 38%|███▊      | 14526/37962 [33:40:07<53:46:26,  8.26s/it]
 38%|███▊      | 14527/37962 [33:40:15<53:55:30,  8.28s/it]
 38%|███▊      | 14528/37962 [33:40:23<54:04:20,  8.31s/it]
 38%|███▊      | 14529/37962 [33:40:32<54:14:46,  8.33s/it]
 38%|███▊      | 14530/37962 [33:40:40<54:11:43,  8.33s/it]
                                                           
{'loss': 1.3658, 'grad_norm': 22.797040939331055, 'learning_rate': 1.4153515160060111e-05, 'epoch': 0.38}

 38%|███▊      | 14530/37962 [33:40:40<54:11:43,  8.33s/it]
 38%|███▊      | 14531/37962 [33:40:49<54:13:10,  8.33s/it]
 38%|███▊      | 14532/37962 [33:40:56<53:28:00,  8.22s/it]
 38%|███▊      | 14533/37962 [33:41:05<53:56:09,  8.29s/it]
 38%|███▊      | 14534/37962 [33:41:13<54:10:08,  8.32s/it]
 38%|███▊      | 14535/37962 [33:41:21<53:38:24,  8.24s/it]
 38%|███▊      | 14536/37962 [33:41:30<53:43:11,  8.26s/it]
 38%|███▊      | 14537/37962 [33:41:38<53:36:23,  8.24s/it]
 38%|███▊      | 14538/37962 [33:41:46<53:35:22,  8.24s/it]
 38%|███▊      | 14539/37962 [33:41:54<53:40:42,  8.25s/it]
 38%|███▊      | 14540/37962 [33:42:03<53:51:35,  8.28s/it]
                                                           
{'loss': 1.3823, 'grad_norm': 19.911762237548828, 'learning_rate': 1.4145752781739805e-05, 'epoch': 0.38}

 38%|███▊      | 14540/37962 [33:42:03<53:51:35,  8.28s/it]
 38%|███▊      | 14541/37962 [33:42:11<53:21:10,  8.20s/it]
 38%|███▊      | 14542/37962 [33:42:19<53:30:47,  8.23s/it]
 38%|███▊      | 14543/37962 [33:42:27<53:37:26,  8.24s/it]
 38%|███▊      | 14544/37962 [33:42:36<54:03:02,  8.31s/it]
 38%|███▊      | 14545/37962 [33:42:44<53:49:59,  8.28s/it]
 38%|███▊      | 14546/37962 [33:42:52<54:04:25,  8.31s/it]
 38%|███▊      | 14547/37962 [33:43:01<53:57:50,  8.30s/it]
 38%|███▊      | 14548/37962 [33:43:09<53:53:41,  8.29s/it]
 38%|███▊      | 14549/37962 [33:43:17<53:48:36,  8.27s/it]
 38%|███▊      | 14550/37962 [33:43:25<53:56:31,  8.29s/it]
                                                           
{'loss': 1.5458, 'grad_norm': 5.983964920043945, 'learning_rate': 1.4137987385797871e-05, 'epoch': 0.38}

 38%|███▊      | 14550/37962 [33:43:25<53:56:31,  8.29s/it]
 38%|███▊      | 14551/37962 [33:43:33<52:51:18,  8.13s/it]
 38%|███▊      | 14552/37962 [33:43:42<53:17:46,  8.20s/it]
 38%|███▊      | 14553/37962 [33:43:50<53:23:35,  8.21s/it]
 38%|███▊      | 14554/37962 [33:43:58<53:04:45,  8.16s/it]
 38%|███▊      | 14555/37962 [33:44:06<53:28:25,  8.22s/it]
 38%|███▊      | 14556/37962 [33:44:14<53:26:29,  8.22s/it]
 38%|███▊      | 14557/37962 [33:44:22<53:00:38,  8.15s/it]
 38%|███▊      | 14558/37962 [33:44:30<52:47:57,  8.12s/it]
 38%|███▊      | 14559/37962 [33:44:39<52:54:45,  8.14s/it]
 38%|███▊      | 14560/37962 [33:44:47<52:39:37,  8.10s/it]
                                                           
{'loss': 1.5966, 'grad_norm': 9.547913551330566, 'learning_rate': 1.4130218977886605e-05, 'epoch': 0.38}

 38%|███▊      | 14560/37962 [33:44:47<52:39:37,  8.10s/it]
 38%|███▊      | 14561/37962 [33:44:55<52:53:14,  8.14s/it]
 38%|███▊      | 14562/37962 [33:45:03<53:36:54,  8.25s/it]
 38%|███▊      | 14563/37962 [33:45:12<53:45:48,  8.27s/it]
 38%|███▊      | 14564/37962 [33:45:20<53:48:08,  8.28s/it]
 38%|███▊      | 14565/37962 [33:45:28<53:46:10,  8.27s/it]
 38%|███▊      | 14566/37962 [33:45:36<53:13:47,  8.19s/it]
 38%|███▊      | 14567/37962 [33:45:45<53:24:33,  8.22s/it]
 38%|███▊      | 14568/37962 [33:45:53<53:30:51,  8.24s/it]
 38%|███▊      | 14569/37962 [33:46:01<54:00:13,  8.31s/it]
 38%|███▊      | 14570/37962 [33:46:10<54:13:37,  8.35s/it]
                                                           
{'loss': 1.3778, 'grad_norm': 7.954235553741455, 'learning_rate': 1.4122447563660498e-05, 'epoch': 0.38}

 38%|███▊      | 14570/37962 [33:46:10<54:13:37,  8.35s/it]
 38%|███▊      | 14571/37962 [33:46:18<54:01:18,  8.31s/it]
 38%|███▊      | 14572/37962 [33:46:26<53:48:14,  8.28s/it]
 38%|███▊      | 14573/37962 [33:46:34<53:46:01,  8.28s/it]
 38%|███▊      | 14574/37962 [33:46:43<53:37:04,  8.25s/it]
 38%|███▊      | 14575/37962 [33:46:51<53:34:32,  8.25s/it]
 38%|███▊      | 14576/37962 [33:46:59<53:45:39,  8.28s/it]
 38%|███▊      | 14577/37962 [33:47:08<53:46:57,  8.28s/it]
 38%|███▊      | 14578/37962 [33:47:16<53:37:58,  8.26s/it]
 38%|███▊      | 14579/37962 [33:47:24<53:37:12,  8.26s/it]
 38%|███▊      | 14580/37962 [33:47:32<53:40:50,  8.26s/it]
                                                           
{'loss': 1.5369, 'grad_norm': 30.239574432373047, 'learning_rate': 1.4114673148776221e-05, 'epoch': 0.38}

 38%|███▊      | 14580/37962 [33:47:32<53:40:50,  8.26s/it]
 38%|███▊      | 14581/37962 [33:47:41<53:39:20,  8.26s/it]
 38%|███▊      | 14582/37962 [33:47:49<53:31:22,  8.24s/it]
 38%|███▊      | 14583/37962 [33:47:57<53:54:36,  8.30s/it]
 38%|███▊      | 14584/37962 [33:48:06<54:03:48,  8.33s/it]
 38%|███▊      | 14585/37962 [33:48:14<54:09:54,  8.34s/it]
 38%|███▊      | 14586/37962 [33:48:22<53:57:13,  8.31s/it]
 38%|███▊      | 14587/37962 [33:48:30<53:58:55,  8.31s/it]
 38%|███▊      | 14588/37962 [33:48:39<54:35:17,  8.41s/it]
 38%|███▊      | 14589/37962 [33:48:47<53:49:02,  8.29s/it]
 38%|███▊      | 14590/37962 [33:48:55<53:37:04,  8.26s/it]
                                                           
{'loss': 1.4199, 'grad_norm': 10.088793754577637, 'learning_rate': 1.4106895738892647e-05, 'epoch': 0.38}

 38%|███▊      | 14590/37962 [33:48:55<53:37:04,  8.26s/it]
 38%|███▊      | 14591/37962 [33:49:04<53:37:01,  8.26s/it]
 38%|███▊      | 14592/37962 [33:49:12<53:41:58,  8.27s/it]
 38%|███▊      | 14593/37962 [33:49:20<53:34:21,  8.25s/it]
 38%|███▊      | 14594/37962 [33:49:28<53:39:28,  8.27s/it]
 38%|███▊      | 14595/37962 [33:49:37<53:40:05,  8.27s/it]
 38%|███▊      | 14596/37962 [33:49:45<53:30:48,  8.24s/it]
 38%|███▊      | 14597/37962 [33:49:53<53:40:26,  8.27s/it]
 38%|███▊      | 14598/37962 [33:50:02<54:10:35,  8.35s/it]
 38%|███▊      | 14599/37962 [33:50:10<54:33:23,  8.41s/it]
 38%|███▊      | 14600/37962 [33:50:19<54:21:37,  8.38s/it]
                                                           
{'loss': 1.4314, 'grad_norm': 14.220598220825195, 'learning_rate': 1.4099115339670808e-05, 'epoch': 0.38}

 38%|███▊      | 14600/37962 [33:50:19<54:21:37,  8.38s/it]
 38%|███▊      | 14601/37962 [33:50:27<54:39:12,  8.42s/it]
 38%|███▊      | 14602/37962 [33:50:35<53:51:28,  8.30s/it]
 38%|███▊      | 14603/37962 [33:50:43<53:39:06,  8.27s/it]
 38%|███▊      | 14604/37962 [33:50:51<52:46:57,  8.14s/it]
 38%|███▊      | 14605/37962 [33:50:59<52:56:58,  8.16s/it]
 38%|███▊      | 14606/37962 [33:51:07<52:35:53,  8.11s/it]
 38%|███▊      | 14607/37962 [33:51:16<53:12:18,  8.20s/it]
 38%|███▊      | 14608/37962 [33:51:24<52:55:46,  8.16s/it]
 38%|███▊      | 14609/37962 [33:51:32<53:14:08,  8.21s/it]
 38%|███▊      | 14610/37962 [33:51:40<53:27:20,  8.24s/it]
                                                           
{'loss': 1.4164, 'grad_norm': 58.917606353759766, 'learning_rate': 1.4091331956773924e-05, 'epoch': 0.38}

 38%|███▊      | 14610/37962 [33:51:40<53:27:20,  8.24s/it]
 38%|███▊      | 14611/37962 [33:51:49<53:23:13,  8.23s/it]
 38%|███▊      | 14612/37962 [33:51:57<53:13:41,  8.21s/it]
 38%|███▊      | 14613/37962 [33:52:05<52:39:42,  8.12s/it]
 38%|███▊      | 14614/37962 [33:52:13<53:08:06,  8.19s/it]
 38%|███▊      | 14615/37962 [33:52:21<52:20:49,  8.07s/it]
 39%|███▊      | 14616/37962 [33:52:29<52:37:57,  8.12s/it]
 39%|███▊      | 14617/37962 [33:52:37<52:51:57,  8.15s/it]
 39%|███▊      | 14618/37962 [33:52:45<52:28:43,  8.09s/it]
 39%|███▊      | 14619/37962 [33:52:53<52:18:53,  8.07s/it]
 39%|███▊      | 14620/37962 [33:53:01<52:09:32,  8.04s/it]
                                                           
{'loss': 1.4864, 'grad_norm': 30.70146942138672, 'learning_rate': 1.408354559586739e-05, 'epoch': 0.39}

 39%|███▊      | 14620/37962 [33:53:01<52:09:32,  8.04s/it]
 39%|███▊      | 14621/37962 [33:53:09<52:28:32,  8.09s/it]
 39%|███▊      | 14622/37962 [33:53:18<53:02:05,  8.18s/it]
 39%|███▊      | 14623/37962 [33:53:26<53:28:24,  8.25s/it]
 39%|███▊      | 14624/37962 [33:53:35<53:26:05,  8.24s/it]
 39%|███▊      | 14625/37962 [33:53:43<53:19:27,  8.23s/it]
 39%|███▊      | 14626/37962 [33:53:51<53:31:34,  8.26s/it]
 39%|███▊      | 14627/37962 [33:53:59<53:31:02,  8.26s/it]
 39%|███▊      | 14628/37962 [33:54:07<52:54:08,  8.16s/it]
 39%|███▊      | 14629/37962 [33:54:16<53:08:12,  8.20s/it]
 39%|███▊      | 14630/37962 [33:54:23<52:43:42,  8.14s/it]
                                                           
{'loss': 1.4774, 'grad_norm': 45.016849517822266, 'learning_rate': 1.4075756262618757e-05, 'epoch': 0.39}

 39%|███▊      | 14630/37962 [33:54:23<52:43:42,  8.14s/it]
 39%|███▊      | 14631/37962 [33:54:32<52:39:14,  8.12s/it]
 39%|███▊      | 14632/37962 [33:54:40<52:53:02,  8.16s/it]
 39%|███▊      | 14633/37962 [33:54:48<53:02:10,  8.18s/it]
 39%|███▊      | 14634/37962 [33:54:56<53:06:57,  8.20s/it]
 39%|███▊      | 14635/37962 [33:55:05<53:37:41,  8.28s/it]
 39%|███▊      | 14636/37962 [33:55:13<53:58:39,  8.33s/it]
 39%|███▊      | 14637/37962 [33:55:22<54:09:52,  8.36s/it]
 39%|███▊      | 14638/37962 [33:55:30<53:35:44,  8.27s/it]
 39%|███▊      | 14639/37962 [33:55:38<53:41:06,  8.29s/it]
 39%|███▊      | 14640/37962 [33:55:46<53:29:12,  8.26s/it]
                                                           
{'loss': 1.5272, 'grad_norm': 21.885793685913086, 'learning_rate': 1.4067963962697748e-05, 'epoch': 0.39}

 39%|███▊      | 14640/37962 [33:55:46<53:29:12,  8.26s/it]
 39%|███▊      | 14641/37962 [33:55:54<53:21:15,  8.24s/it]
 39%|███▊      | 14642/37962 [33:56:03<53:42:42,  8.29s/it]
 39%|███▊      | 14643/37962 [33:56:11<53:59:31,  8.34s/it]
 39%|███▊      | 14644/37962 [33:56:19<53:24:02,  8.24s/it]
 39%|███▊      | 14645/37962 [33:56:28<53:29:34,  8.26s/it]
 39%|███▊      | 14646/37962 [33:56:36<53:39:33,  8.29s/it]
 39%|███▊      | 14647/37962 [33:56:45<54:11:55,  8.37s/it]
 39%|███▊      | 14648/37962 [33:56:53<53:59:56,  8.34s/it]
 39%|███▊      | 14649/37962 [33:57:01<53:26:13,  8.25s/it]
 39%|███▊      | 14650/37962 [33:57:09<53:08:41,  8.21s/it]
                                                           
{'loss': 1.4403, 'grad_norm': 33.49884033203125, 'learning_rate': 1.4060168701776242e-05, 'epoch': 0.39}

 39%|███▊      | 14650/37962 [33:57:09<53:08:41,  8.21s/it]
 39%|███▊      | 14651/37962 [33:57:17<52:50:31,  8.16s/it]
 39%|███▊      | 14652/37962 [33:57:25<53:28:35,  8.26s/it]
 39%|███▊      | 14653/37962 [33:57:34<53:09:57,  8.21s/it]
 39%|███▊      | 14654/37962 [33:57:42<53:14:02,  8.22s/it]
 39%|███▊      | 14655/37962 [33:57:50<53:26:44,  8.26s/it]
 39%|███▊      | 14656/37962 [33:57:58<53:25:17,  8.25s/it]
 39%|███▊      | 14657/37962 [33:58:07<53:21:09,  8.24s/it]
 39%|███▊      | 14658/37962 [33:58:15<53:31:01,  8.27s/it]
 39%|███▊      | 14659/37962 [33:58:23<53:24:16,  8.25s/it]
 39%|███▊      | 14660/37962 [33:58:32<54:07:49,  8.36s/it]
                                                           
{'loss': 1.4147, 'grad_norm': 49.4317626953125, 'learning_rate': 1.4052370485528277e-05, 'epoch': 0.39}

 39%|███▊      | 14660/37962 [33:58:32<54:07:49,  8.36s/it]
 39%|███▊      | 14661/37962 [33:58:40<54:03:08,  8.35s/it]
 39%|███▊      | 14662/37962 [33:58:48<53:53:30,  8.33s/it]
 39%|███▊      | 14663/37962 [33:58:57<53:38:27,  8.29s/it]
 39%|███▊      | 14664/37962 [33:59:05<53:39:44,  8.29s/it]
 39%|███▊      | 14665/37962 [33:59:13<53:32:28,  8.27s/it]
 39%|███▊      | 14666/37962 [33:59:21<53:30:04,  8.27s/it]
 39%|███▊      | 14667/37962 [33:59:30<53:31:43,  8.27s/it]
 39%|███▊      | 14668/37962 [33:59:38<53:20:47,  8.24s/it]
 39%|███▊      | 14669/37962 [33:59:46<53:32:55,  8.28s/it]
 39%|███▊      | 14670/37962 [33:59:55<53:41:18,  8.30s/it]
                                                           
{'loss': 1.5087, 'grad_norm': 18.248889923095703, 'learning_rate': 1.4044569319630039e-05, 'epoch': 0.39}

 39%|███▊      | 14670/37962 [33:59:55<53:41:18,  8.30s/it]
 39%|███▊      | 14671/37962 [34:00:03<53:32:48,  8.28s/it]
 39%|███▊      | 14672/37962 [34:00:11<53:38:34,  8.29s/it]
 39%|███▊      | 14673/37962 [34:00:19<53:17:30,  8.24s/it]
 39%|███▊      | 14674/37962 [34:00:28<53:34:32,  8.28s/it]
 39%|███▊      | 14675/37962 [34:00:36<53:31:37,  8.27s/it]
 39%|███▊      | 14676/37962 [34:00:44<53:21:46,  8.25s/it]
 39%|███▊      | 14677/37962 [34:00:52<53:25:49,  8.26s/it]
 39%|███▊      | 14678/37962 [34:01:00<53:01:49,  8.20s/it]
 39%|███▊      | 14679/37962 [34:01:09<53:41:11,  8.30s/it]
 39%|███▊      | 14680/37962 [34:01:17<53:38:32,  8.29s/it]
                                                           
{'loss': 1.2942, 'grad_norm': 12.690756797790527, 'learning_rate': 1.403676520975986e-05, 'epoch': 0.39}

 39%|███▊      | 14680/37962 [34:01:17<53:38:32,  8.29s/it]
 39%|███▊      | 14681/37962 [34:01:25<53:34:00,  8.28s/it]
 39%|███▊      | 14682/37962 [34:01:34<53:56:50,  8.34s/it]
 39%|███▊      | 14683/37962 [34:01:42<53:42:25,  8.31s/it]
 39%|███▊      | 14684/37962 [34:01:50<53:32:06,  8.28s/it]
 39%|███▊      | 14685/37962 [34:01:59<53:26:03,  8.26s/it]
 39%|███▊      | 14686/37962 [34:02:07<53:48:49,  8.32s/it]
 39%|███▊      | 14687/37962 [34:02:15<53:44:43,  8.31s/it]
 39%|███▊      | 14688/37962 [34:02:24<53:44:01,  8.31s/it]
 39%|███▊      | 14689/37962 [34:02:32<53:54:38,  8.34s/it]
 39%|███▊      | 14690/37962 [34:02:40<53:47:38,  8.32s/it]
                                                           
{'loss': 1.3361, 'grad_norm': 10.565882682800293, 'learning_rate': 1.4028958161598217e-05, 'epoch': 0.39}

 39%|███▊      | 14690/37962 [34:02:40<53:47:38,  8.32s/it]
 39%|███▊      | 14691/37962 [34:02:49<53:50:07,  8.33s/it]
 39%|███▊      | 14692/37962 [34:02:57<53:52:32,  8.33s/it]
 39%|███▊      | 14693/37962 [34:03:05<53:47:21,  8.32s/it]
 39%|███▊      | 14694/37962 [34:03:14<53:55:51,  8.34s/it]
 39%|███▊      | 14695/37962 [34:03:22<53:49:18,  8.33s/it]
 39%|███▊      | 14696/37962 [34:03:30<53:18:22,  8.25s/it]
 39%|███▊      | 14697/37962 [34:03:38<53:31:39,  8.28s/it]
 39%|███▊      | 14698/37962 [34:03:47<53:47:29,  8.32s/it]
 39%|███▊      | 14699/37962 [34:03:55<53:18:35,  8.25s/it]
 39%|███▊      | 14700/37962 [34:04:03<53:23:34,  8.26s/it]
                                                           
{'loss': 1.2748, 'grad_norm': 11.384231567382812, 'learning_rate': 1.4021148180827726e-05, 'epoch': 0.39}

 39%|███▊      | 14700/37962 [34:04:03<53:23:34,  8.26s/it]
 39%|███▊      | 14701/37962 [34:04:12<53:48:41,  8.33s/it]
 39%|███▊      | 14702/37962 [34:04:20<54:02:41,  8.36s/it]
 39%|███▊      | 14703/37962 [34:04:28<53:11:36,  8.23s/it]
 39%|███▊      | 14704/37962 [34:04:36<53:18:16,  8.25s/it]
 39%|███▊      | 14705/37962 [34:04:45<53:32:42,  8.29s/it]
 39%|███▊      | 14706/37962 [34:04:53<53:44:18,  8.32s/it]
 39%|███▊      | 14707/37962 [34:05:01<53:35:33,  8.30s/it]
 39%|███▊      | 14708/37962 [34:05:10<53:55:08,  8.35s/it]
 39%|███▊      | 14709/37962 [34:05:18<53:44:07,  8.32s/it]
 39%|███▊      | 14710/37962 [34:05:27<54:12:10,  8.39s/it]
                                                           
{'loss': 1.648, 'grad_norm': 14.115474700927734, 'learning_rate': 1.4013335273133132e-05, 'epoch': 0.39}

 39%|███▊      | 14710/37962 [34:05:27<54:12:10,  8.39s/it]
 39%|███▉      | 14711/37962 [34:05:35<54:08:33,  8.38s/it]
 39%|███▉      | 14712/37962 [34:05:43<53:49:40,  8.33s/it]
 39%|███▉      | 14713/37962 [34:05:51<53:37:28,  8.30s/it]
 39%|███▉      | 14714/37962 [34:06:00<54:05:55,  8.38s/it]
 39%|███▉      | 14715/37962 [34:06:08<53:50:15,  8.34s/it]
 39%|███▉      | 14716/37962 [34:06:16<53:13:53,  8.24s/it]
 39%|███▉      | 14717/37962 [34:06:25<53:19:01,  8.26s/it]
 39%|███▉      | 14718/37962 [34:06:33<53:44:38,  8.32s/it]
 39%|███▉      | 14719/37962 [34:06:41<53:50:47,  8.34s/it]
 39%|███▉      | 14720/37962 [34:06:50<53:39:51,  8.31s/it]
                                                           
{'loss': 1.425, 'grad_norm': 90.19509887695312, 'learning_rate': 1.4005519444201323e-05, 'epoch': 0.39}

 39%|███▉      | 14720/37962 [34:06:50<53:39:51,  8.31s/it]
 39%|███▉      | 14721/37962 [34:06:58<53:33:38,  8.30s/it]
 39%|███▉      | 14722/37962 [34:07:06<53:27:33,  8.28s/it]
 39%|███▉      | 14723/37962 [34:07:14<53:10:34,  8.24s/it]
 39%|███▉      | 14724/37962 [34:07:22<52:27:59,  8.13s/it]
 39%|███▉      | 14725/37962 [34:07:30<52:43:18,  8.17s/it]
 39%|███▉      | 14726/37962 [34:07:39<53:11:46,  8.24s/it]
 39%|███▉      | 14727/37962 [34:07:47<52:51:43,  8.19s/it]
 39%|███▉      | 14728/37962 [34:07:55<53:31:07,  8.29s/it]
 39%|███▉      | 14729/37962 [34:08:04<53:10:52,  8.24s/it]
 39%|███▉      | 14730/37962 [34:08:12<53:23:47,  8.27s/it]
                                                           
{'loss': 1.3708, 'grad_norm': 12794.9755859375, 'learning_rate': 1.3997700699721302e-05, 'epoch': 0.39}

 39%|███▉      | 14730/37962 [34:08:12<53:23:47,  8.27s/it]
 39%|███▉      | 14731/37962 [34:08:20<53:24:22,  8.28s/it]
 39%|███▉      | 14732/37962 [34:08:29<53:33:09,  8.30s/it]
 39%|███▉      | 14733/37962 [34:08:37<53:40:27,  8.32s/it]
 39%|███▉      | 14734/37962 [34:08:45<53:04:18,  8.23s/it]
 39%|███▉      | 14735/37962 [34:08:53<53:05:05,  8.23s/it]
 39%|███▉      | 14736/37962 [34:09:02<53:19:06,  8.26s/it]
 39%|███▉      | 14737/37962 [34:09:10<53:03:14,  8.22s/it]
 39%|███▉      | 14738/37962 [34:09:18<53:04:56,  8.23s/it]
 39%|███▉      | 14739/37962 [34:09:26<53:02:39,  8.22s/it]
 39%|███▉      | 14740/37962 [34:09:35<53:33:46,  8.30s/it]
                                                           
{'loss': 1.4427, 'grad_norm': 14.234919548034668, 'learning_rate': 1.3989879045384194e-05, 'epoch': 0.39}

 39%|███▉      | 14740/37962 [34:09:35<53:33:46,  8.30s/it]
 39%|███▉      | 14741/37962 [34:09:43<53:30:52,  8.30s/it]
 39%|███▉      | 14742/37962 [34:09:51<52:56:06,  8.21s/it]
 39%|███▉      | 14743/37962 [34:09:59<52:55:58,  8.21s/it]
 39%|███▉      | 14744/37962 [34:10:07<53:13:50,  8.25s/it]
 39%|███▉      | 14745/37962 [34:10:16<53:22:08,  8.28s/it]
 39%|███▉      | 14746/37962 [34:10:24<53:00:01,  8.22s/it]
 39%|███▉      | 14747/37962 [34:10:32<53:08:52,  8.24s/it]
 39%|███▉      | 14748/37962 [34:10:40<52:36:32,  8.16s/it]
 39%|███▉      | 14749/37962 [34:10:48<52:36:44,  8.16s/it]
 39%|███▉      | 14750/37962 [34:10:57<53:04:36,  8.23s/it]
                                                           
{'loss': 1.5527, 'grad_norm': 12.258428573608398, 'learning_rate': 1.398205448688325e-05, 'epoch': 0.39}

 39%|███▉      | 14750/37962 [34:10:57<53:04:36,  8.23s/it]
 39%|███▉      | 14751/37962 [34:11:05<53:45:31,  8.34s/it]
 39%|███▉      | 14752/37962 [34:11:14<53:39:50,  8.32s/it]
 39%|███▉      | 14753/37962 [34:11:22<53:50:15,  8.35s/it]
 39%|███▉      | 14754/37962 [34:11:30<53:17:41,  8.27s/it]
 39%|███▉      | 14755/37962 [34:11:38<53:18:53,  8.27s/it]
 39%|███▉      | 14756/37962 [34:11:47<53:16:02,  8.26s/it]
 39%|███▉      | 14757/37962 [34:11:55<53:29:21,  8.30s/it]
 39%|███▉      | 14758/37962 [34:12:03<53:31:03,  8.30s/it]
 39%|███▉      | 14759/37962 [34:12:11<53:20:57,  8.28s/it]
 39%|███▉      | 14760/37962 [34:12:20<53:14:17,  8.26s/it]
                                                           
{'loss': 1.3268, 'grad_norm': 13.358673095703125, 'learning_rate': 1.3974227029913827e-05, 'epoch': 0.39}

 39%|███▉      | 14760/37962 [34:12:20<53:14:17,  8.26s/it]
 39%|███▉      | 14761/37962 [34:12:28<53:21:30,  8.28s/it]
 39%|███▉      | 14762/37962 [34:12:36<53:22:53,  8.28s/it]
 39%|███▉      | 14763/37962 [34:12:45<53:27:30,  8.30s/it]
 39%|███▉      | 14764/37962 [34:12:53<53:30:03,  8.30s/it]
 39%|███▉      | 14765/37962 [34:13:01<53:42:07,  8.33s/it]
 39%|███▉      | 14766/37962 [34:13:10<54:05:10,  8.39s/it]
 39%|███▉      | 14767/37962 [34:13:18<53:39:37,  8.33s/it]
 39%|███▉      | 14768/37962 [34:13:26<53:29:12,  8.30s/it]
 39%|███▉      | 14769/37962 [34:13:35<53:30:15,  8.30s/it]
 39%|███▉      | 14770/37962 [34:13:43<53:23:36,  8.29s/it]
                                                           
{'loss': 1.4112, 'grad_norm': 16.93588638305664, 'learning_rate': 1.3966396680173403e-05, 'epoch': 0.39}

 39%|███▉      | 14770/37962 [34:13:43<53:23:36,  8.29s/it]
 39%|███▉      | 14771/37962 [34:13:51<53:19:03,  8.28s/it]
 39%|███▉      | 14772/37962 [34:13:59<53:09:53,  8.25s/it]
 39%|███▉      | 14773/37962 [34:14:08<53:21:16,  8.28s/it]
 39%|███▉      | 14774/37962 [34:14:16<53:36:26,  8.32s/it]
 39%|███▉      | 14775/37962 [34:14:24<52:55:46,  8.22s/it]
 39%|███▉      | 14776/37962 [34:14:32<53:11:01,  8.26s/it]
 39%|███▉      | 14777/37962 [34:14:41<53:24:35,  8.29s/it]
 39%|███▉      | 14778/37962 [34:14:49<53:00:44,  8.23s/it]
 39%|███▉      | 14779/37962 [34:14:57<53:09:32,  8.25s/it]
 39%|███▉      | 14780/37962 [34:15:05<53:16:18,  8.27s/it]
                                                           
{'loss': 1.3377, 'grad_norm': 14.2131929397583, 'learning_rate': 1.3958563443361543e-05, 'epoch': 0.39}

 39%|███▉      | 14780/37962 [34:15:05<53:16:18,  8.27s/it]
 39%|███▉      | 14781/37962 [34:15:14<53:08:49,  8.25s/it]
 39%|███▉      | 14782/37962 [34:15:22<53:17:38,  8.28s/it]
 39%|███▉      | 14783/37962 [34:15:30<53:26:13,  8.30s/it]
 39%|███▉      | 14784/37962 [34:15:39<53:26:41,  8.30s/it]
 39%|███▉      | 14785/37962 [34:15:47<53:10:16,  8.26s/it]
 39%|███▉      | 14786/37962 [34:15:55<53:04:59,  8.25s/it]
 39%|███▉      | 14787/37962 [34:16:03<52:44:16,  8.19s/it]
 39%|███▉      | 14788/37962 [34:16:12<53:19:47,  8.28s/it]
 39%|███▉      | 14789/37962 [34:16:20<53:27:35,  8.31s/it]
 39%|███▉      | 14790/37962 [34:16:28<52:52:41,  8.22s/it]
                                                           
{'loss': 1.4165, 'grad_norm': 7.659984588623047, 'learning_rate': 1.3950727325179934e-05, 'epoch': 0.39}

 39%|███▉      | 14790/37962 [34:16:28<52:52:41,  8.22s/it]
 39%|███▉      | 14791/37962 [34:16:36<53:24:55,  8.30s/it]
 39%|███▉      | 14792/37962 [34:16:45<53:16:50,  8.28s/it]
 39%|███▉      | 14793/37962 [34:16:53<53:20:22,  8.29s/it]
 39%|███▉      | 14794/37962 [34:17:01<53:07:31,  8.25s/it]
 39%|███▉      | 14795/37962 [34:17:10<53:26:47,  8.31s/it]
 39%|███▉      | 14796/37962 [34:17:18<53:28:30,  8.31s/it]
 39%|███▉      | 14797/37962 [34:17:26<53:08:34,  8.26s/it]
 39%|███▉      | 14798/37962 [34:17:34<53:15:34,  8.28s/it]
 39%|███▉      | 14799/37962 [34:17:43<53:13:04,  8.27s/it]
 39%|███▉      | 14800/37962 [34:17:51<53:10:45,  8.27s/it]
                                                           
{'loss': 1.3507, 'grad_norm': 907.00146484375, 'learning_rate': 1.3942888331332347e-05, 'epoch': 0.39}

 39%|███▉      | 14800/37962 [34:17:51<53:10:45,  8.27s/it]
 39%|███▉      | 14801/37962 [34:17:59<53:00:59,  8.24s/it]
 39%|███▉      | 14802/37962 [34:18:07<53:06:25,  8.25s/it]
 39%|███▉      | 14803/37962 [34:18:16<53:29:45,  8.32s/it]
 39%|███▉      | 14804/37962 [34:18:24<53:32:12,  8.32s/it]
 39%|███▉      | 14805/37962 [34:18:32<53:23:21,  8.30s/it]
 39%|███▉      | 14806/37962 [34:18:41<53:27:40,  8.31s/it]
 39%|███▉      | 14807/37962 [34:18:49<52:58:50,  8.24s/it]
 39%|███▉      | 14808/37962 [34:18:57<52:25:56,  8.15s/it]
 39%|███▉      | 14809/37962 [34:19:05<52:44:48,  8.20s/it]
 39%|███▉      | 14810/37962 [34:19:13<52:50:12,  8.22s/it]
                                                           
{'loss': 1.4552, 'grad_norm': 19.37653923034668, 'learning_rate': 1.3935046467524647e-05, 'epoch': 0.39}

 39%|███▉      | 14810/37962 [34:19:13<52:50:12,  8.22s/it]
 39%|███▉      | 14811/37962 [34:19:21<52:19:45,  8.14s/it]
 39%|███▉      | 14812/37962 [34:19:30<52:25:43,  8.15s/it]
 39%|███▉      | 14813/37962 [34:19:38<52:58:17,  8.24s/it]
 39%|███▉      | 14814/37962 [34:19:46<53:09:37,  8.27s/it]
 39%|███▉      | 14815/37962 [34:19:54<52:45:52,  8.21s/it]
 39%|███▉      | 14816/37962 [34:20:03<53:22:22,  8.30s/it]
 39%|███▉      | 14817/37962 [34:20:11<53:21:12,  8.30s/it]
 39%|███▉      | 14818/37962 [34:20:19<53:21:56,  8.30s/it]
 39%|███▉      | 14819/37962 [34:20:28<53:18:54,  8.29s/it]
 39%|███▉      | 14820/37962 [34:20:36<53:25:13,  8.31s/it]
                                                           
{'loss': 1.4162, 'grad_norm': 339.8660888671875, 'learning_rate': 1.3927201739464793e-05, 'epoch': 0.39}

 39%|███▉      | 14820/37962 [34:20:36<53:25:13,  8.31s/it]
 39%|███▉      | 14821/37962 [34:20:44<53:16:30,  8.29s/it]
 39%|███▉      | 14822/37962 [34:20:53<53:30:05,  8.32s/it]
 39%|███▉      | 14823/37962 [34:21:01<52:48:42,  8.22s/it]
 39%|███▉      | 14824/37962 [34:21:09<52:56:39,  8.24s/it]
 39%|███▉      | 14825/37962 [34:21:17<53:12:40,  8.28s/it]
 39%|███▉      | 14826/37962 [34:21:26<53:37:25,  8.34s/it]
 39%|███▉      | 14827/37962 [34:21:34<53:01:26,  8.25s/it]
 39%|███▉      | 14828/37962 [34:21:42<53:09:35,  8.27s/it]
 39%|███▉      | 14829/37962 [34:21:51<53:22:37,  8.31s/it]
 39%|███▉      | 14830/37962 [34:21:59<53:22:02,  8.31s/it]
                                                           
{'loss': 1.3108, 'grad_norm': 42.60823440551758, 'learning_rate': 1.391935415286283e-05, 'epoch': 0.39}

 39%|███▉      | 14830/37962 [34:21:59<53:22:02,  8.31s/it]
 39%|███▉      | 14831/37962 [34:22:07<53:32:23,  8.33s/it]
 39%|███▉      | 14832/37962 [34:22:16<53:29:26,  8.33s/it]
 39%|███▉      | 14833/37962 [34:22:24<53:35:06,  8.34s/it]
 39%|███▉      | 14834/37962 [34:22:32<53:40:44,  8.36s/it]
 39%|███▉      | 14835/37962 [34:22:41<53:39:41,  8.35s/it]
 39%|███▉      | 14836/37962 [34:22:49<53:28:08,  8.32s/it]
 39%|███▉      | 14837/37962 [34:22:57<53:01:37,  8.26s/it]
 39%|███▉      | 14838/37962 [34:23:05<53:08:37,  8.27s/it]
 39%|███▉      | 14839/37962 [34:23:14<53:07:11,  8.27s/it]
 39%|███▉      | 14840/37962 [34:23:21<52:11:28,  8.13s/it]
                                                           
{'loss': 1.3716, 'grad_norm': 12.07181453704834, 'learning_rate': 1.3911503713430875e-05, 'epoch': 0.39}

 39%|███▉      | 14840/37962 [34:23:21<52:11:28,  8.13s/it]
 39%|███▉      | 14841/37962 [34:23:30<52:28:59,  8.17s/it]
 39%|███▉      | 14842/37962 [34:23:38<53:00:48,  8.25s/it]
 39%|███▉      | 14843/37962 [34:23:47<53:11:11,  8.28s/it]
 39%|███▉      | 14844/37962 [34:23:55<52:54:45,  8.24s/it]
 39%|███▉      | 14845/37962 [34:24:03<53:03:37,  8.26s/it]
 39%|███▉      | 14846/37962 [34:24:11<53:09:51,  8.28s/it]
 39%|███▉      | 14847/37962 [34:24:19<52:35:23,  8.19s/it]
 39%|███▉      | 14848/37962 [34:24:27<52:37:23,  8.20s/it]
 39%|███▉      | 14849/37962 [34:24:36<53:13:56,  8.29s/it]
 39%|███▉      | 14850/37962 [34:24:44<53:21:31,  8.31s/it]
                                                           
{'loss': 1.4417, 'grad_norm': 10.790331840515137, 'learning_rate': 1.3903650426883128e-05, 'epoch': 0.39}

 39%|███▉      | 14850/37962 [34:24:44<53:21:31,  8.31s/it]
 39%|███▉      | 14851/37962 [34:24:53<53:17:37,  8.30s/it]
 39%|███▉      | 14852/37962 [34:25:01<52:58:44,  8.25s/it]
 39%|███▉      | 14853/37962 [34:25:09<53:06:11,  8.27s/it]
 39%|███▉      | 14854/37962 [34:25:17<53:17:47,  8.30s/it]
 39%|███▉      | 14855/37962 [34:25:26<53:14:56,  8.30s/it]
 39%|███▉      | 14856/37962 [34:25:34<53:37:22,  8.35s/it]
 39%|███▉      | 14857/37962 [34:25:43<53:33:12,  8.34s/it]
 39%|███▉      | 14858/37962 [34:25:51<53:29:07,  8.33s/it]
 39%|███▉      | 14859/37962 [34:25:59<53:48:31,  8.38s/it]
 39%|███▉      | 14860/37962 [34:26:08<53:36:37,  8.35s/it]
                                                           
{'loss': 1.4093, 'grad_norm': 16.120529174804688, 'learning_rate': 1.3895794298935862e-05, 'epoch': 0.39}

 39%|███▉      | 14860/37962 [34:26:08<53:36:37,  8.35s/it]
 39%|███▉      | 14861/37962 [34:26:16<53:24:47,  8.32s/it]
 39%|███▉      | 14862/37962 [34:26:24<53:15:23,  8.30s/it]
 39%|███▉      | 14863/37962 [34:26:32<53:18:18,  8.31s/it]
 39%|███▉      | 14864/37962 [34:26:41<52:55:31,  8.25s/it]
 39%|███▉      | 14865/37962 [34:26:49<53:02:50,  8.27s/it]
 39%|███▉      | 14866/37962 [34:26:57<53:14:03,  8.30s/it]
 39%|███▉      | 14867/37962 [34:27:06<53:27:41,  8.33s/it]
 39%|███▉      | 14868/37962 [34:27:14<52:57:19,  8.25s/it]
 39%|███▉      | 14869/37962 [34:27:22<53:12:43,  8.30s/it]
 39%|███▉      | 14870/37962 [34:27:30<52:54:09,  8.25s/it]
                                                           
{'loss': 1.2992, 'grad_norm': 22.032482147216797, 'learning_rate': 1.3887935335307417e-05, 'epoch': 0.39}

 39%|███▉      | 14870/37962 [34:27:30<52:54:09,  8.25s/it]
 39%|███▉      | 14871/37962 [34:27:38<52:36:11,  8.20s/it]
 39%|███▉      | 14872/37962 [34:27:46<51:44:28,  8.07s/it]
 39%|███▉      | 14873/37962 [34:27:54<51:57:18,  8.10s/it]
 39%|███▉      | 14874/37962 [34:28:03<52:27:03,  8.18s/it]
 39%|███▉      | 14875/37962 [34:28:11<52:45:34,  8.23s/it]
 39%|███▉      | 14876/37962 [34:28:19<53:02:15,  8.27s/it]
 39%|███▉      | 14877/37962 [34:28:28<53:07:57,  8.29s/it]
 39%|███▉      | 14878/37962 [34:28:36<53:30:05,  8.34s/it]
 39%|███▉      | 14879/37962 [34:28:44<53:25:28,  8.33s/it]
 39%|███▉      | 14880/37962 [34:28:53<53:14:47,  8.30s/it]
                                                           
{'loss': 1.3545, 'grad_norm': 14.725638389587402, 'learning_rate': 1.3880073541718197e-05, 'epoch': 0.39}

 39%|███▉      | 14880/37962 [34:28:53<53:14:47,  8.30s/it]
 39%|███▉      | 14881/37962 [34:29:01<53:32:39,  8.35s/it]
 39%|███▉      | 14882/37962 [34:29:09<53:24:40,  8.33s/it]
 39%|███▉      | 14883/37962 [34:29:18<53:24:51,  8.33s/it]
 39%|███▉      | 14884/37962 [34:29:26<53:46:23,  8.39s/it]
 39%|███▉      | 14885/37962 [34:29:34<53:19:42,  8.32s/it]
 39%|███▉      | 14886/37962 [34:29:43<53:09:38,  8.29s/it]
 39%|███▉      | 14887/37962 [34:29:51<52:19:00,  8.16s/it]
 39%|███▉      | 14888/37962 [34:29:59<52:36:41,  8.21s/it]
 39%|███▉      | 14889/37962 [34:30:07<52:41:50,  8.22s/it]
 39%|███▉      | 14890/37962 [34:30:15<52:44:23,  8.23s/it]
                                                           
{'loss': 1.3432, 'grad_norm': 60.39795684814453, 'learning_rate': 1.387220892389066e-05, 'epoch': 0.39}

 39%|███▉      | 14890/37962 [34:30:15<52:44:23,  8.23s/it]
 39%|███▉      | 14891/37962 [34:30:24<52:45:30,  8.23s/it]
 39%|███▉      | 14892/37962 [34:30:32<53:10:48,  8.30s/it]
 39%|███▉      | 14893/37962 [34:30:40<53:06:35,  8.29s/it]
 39%|███▉      | 14894/37962 [34:30:48<52:21:20,  8.17s/it]
 39%|███▉      | 14895/37962 [34:30:56<52:27:45,  8.19s/it]
 39%|███▉      | 14896/37962 [34:31:05<52:50:09,  8.25s/it]
 39%|███▉      | 14897/37962 [34:31:13<52:48:22,  8.24s/it]
 39%|███▉      | 14898/37962 [34:31:21<53:06:39,  8.29s/it]
 39%|███▉      | 14899/37962 [34:31:30<53:08:52,  8.30s/it]
 39%|███▉      | 14900/37962 [34:31:38<53:16:38,  8.32s/it]
                                                           
{'loss': 1.3131, 'grad_norm': 10.220460891723633, 'learning_rate': 1.3864341487549331e-05, 'epoch': 0.39}

 39%|███▉      | 14900/37962 [34:31:38<53:16:38,  8.32s/it]
 39%|███▉      | 14901/37962 [34:31:46<53:12:31,  8.31s/it]
 39%|███▉      | 14902/37962 [34:31:55<53:11:17,  8.30s/it]
 39%|███▉      | 14903/37962 [34:32:03<53:26:52,  8.34s/it]
 39%|███▉      | 14904/37962 [34:32:11<52:34:02,  8.21s/it]
 39%|███▉      | 14905/37962 [34:32:19<52:36:19,  8.21s/it]
 39%|███▉      | 14906/37962 [34:32:28<52:36:44,  8.21s/it]
 39%|███▉      | 14907/37962 [34:32:36<52:16:47,  8.16s/it]
 39%|███▉      | 14908/37962 [34:32:44<52:20:23,  8.17s/it]
 39%|███▉      | 14909/37962 [34:32:52<52:54:10,  8.26s/it]
 39%|███▉      | 14910/37962 [34:33:00<52:52:26,  8.26s/it]
                                                           
{'loss': 1.5443, 'grad_norm': 56.70800018310547, 'learning_rate': 1.3856471238420778e-05, 'epoch': 0.39}

 39%|███▉      | 14910/37962 [34:33:00<52:52:26,  8.26s/it]
 39%|███▉      | 14911/37962 [34:33:09<53:15:36,  8.32s/it]
 39%|███▉      | 14912/37962 [34:33:17<53:03:37,  8.29s/it]
 39%|███▉      | 14913/37962 [34:33:26<53:16:19,  8.32s/it]
 39%|███▉      | 14914/37962 [34:33:34<53:30:09,  8.36s/it]
 39%|███▉      | 14915/37962 [34:33:42<53:01:02,  8.28s/it]
 39%|███▉      | 14916/37962 [34:33:50<52:51:26,  8.26s/it]
 39%|███▉      | 14917/37962 [34:33:59<52:51:30,  8.26s/it]
 39%|███▉      | 14918/37962 [34:34:07<52:56:32,  8.27s/it]
 39%|███▉      | 14919/37962 [34:34:15<52:44:21,  8.24s/it]
 39%|███▉      | 14920/37962 [34:34:23<52:06:43,  8.14s/it]
                                                           
{'loss': 1.3388, 'grad_norm': 13.991398811340332, 'learning_rate': 1.3848598182233617e-05, 'epoch': 0.39}

 39%|███▉      | 14920/37962 [34:34:23<52:06:43,  8.14s/it]
 39%|███▉      | 14921/37962 [34:34:31<51:30:31,  8.05s/it]
 39%|███▉      | 14922/37962 [34:34:39<51:59:24,  8.12s/it]
 39%|███▉      | 14923/37962 [34:34:47<52:04:04,  8.14s/it]
 39%|███▉      | 14924/37962 [34:34:56<52:27:12,  8.20s/it]
 39%|███▉      | 14925/37962 [34:35:04<52:41:13,  8.23s/it]
 39%|███▉      | 14926/37962 [34:35:12<52:51:27,  8.26s/it]
 39%|███▉      | 14927/37962 [34:35:21<53:06:21,  8.30s/it]
 39%|███▉      | 14928/37962 [34:35:29<52:37:54,  8.23s/it]
 39%|███▉      | 14929/37962 [34:35:37<53:08:06,  8.30s/it]
 39%|███▉      | 14930/37962 [34:35:45<52:56:12,  8.27s/it]
                                                           
{'loss': 1.3692, 'grad_norm': 11.70394229888916, 'learning_rate': 1.3840722324718516e-05, 'epoch': 0.39}

 39%|███▉      | 14930/37962 [34:35:45<52:56:12,  8.27s/it]
 39%|███▉      | 14931/37962 [34:35:53<52:14:50,  8.17s/it]
 39%|███▉      | 14932/37962 [34:36:01<51:59:01,  8.13s/it]
 39%|███▉      | 14933/37962 [34:36:09<52:07:24,  8.15s/it]
 39%|███▉      | 14934/37962 [34:36:17<51:32:00,  8.06s/it]
 39%|███▉      | 14935/37962 [34:36:25<51:35:06,  8.06s/it]
 39%|███▉      | 14936/37962 [34:36:33<51:29:29,  8.05s/it]
 39%|███▉      | 14937/37962 [34:36:41<51:23:41,  8.04s/it]
 39%|███▉      | 14938/37962 [34:36:50<51:54:59,  8.12s/it]
 39%|███▉      | 14939/37962 [34:36:58<52:41:58,  8.24s/it]
 39%|███▉      | 14940/37962 [34:37:07<53:01:00,  8.29s/it]
                                                           
{'loss': 1.348, 'grad_norm': 69.9419174194336, 'learning_rate': 1.3832843671608167e-05, 'epoch': 0.39}

 39%|███▉      | 14940/37962 [34:37:07<53:01:00,  8.29s/it]
 39%|███▉      | 14941/37962 [34:37:15<52:47:19,  8.26s/it]
 39%|███▉      | 14942/37962 [34:37:23<52:06:43,  8.15s/it]
 39%|███▉      | 14943/37962 [34:37:31<52:49:18,  8.26s/it]
 39%|███▉      | 14944/37962 [34:37:39<52:44:47,  8.25s/it]
 39%|███▉      | 14945/37962 [34:37:48<52:36:44,  8.23s/it]
 39%|███▉      | 14946/37962 [34:37:56<52:36:38,  8.23s/it]
 39%|███▉      | 14947/37962 [34:38:04<52:50:24,  8.27s/it]
 39%|███▉      | 14948/37962 [34:38:12<52:42:52,  8.25s/it]
 39%|███▉      | 14949/37962 [34:38:21<52:38:34,  8.24s/it]
 39%|███▉      | 14950/37962 [34:38:29<52:34:44,  8.23s/it]
                                                           
{'loss': 1.3852, 'grad_norm': 47.75214385986328, 'learning_rate': 1.3824962228637311e-05, 'epoch': 0.39}

 39%|███▉      | 14950/37962 [34:38:29<52:34:44,  8.23s/it]
 39%|███▉      | 14951/37962 [34:38:37<52:49:56,  8.27s/it]
 39%|███▉      | 14952/37962 [34:38:45<51:52:42,  8.12s/it]
 39%|███▉      | 14953/37962 [34:38:53<51:29:31,  8.06s/it]
 39%|███▉      | 14954/37962 [34:39:01<51:46:05,  8.10s/it]
 39%|███▉      | 14955/37962 [34:39:09<52:04:36,  8.15s/it]
 39%|███▉      | 14956/37962 [34:39:18<52:27:11,  8.21s/it]
 39%|███▉      | 14957/37962 [34:39:26<52:37:22,  8.23s/it]
 39%|███▉      | 14958/37962 [34:39:34<52:57:37,  8.29s/it]
 39%|███▉      | 14959/37962 [34:39:43<52:53:59,  8.28s/it]
 39%|███▉      | 14960/37962 [34:39:51<53:11:18,  8.32s/it]
                                                           
{'loss': 1.3647, 'grad_norm': 8.550277709960938, 'learning_rate': 1.3817078001542711e-05, 'epoch': 0.39}

 39%|███▉      | 14960/37962 [34:39:51<53:11:18,  8.32s/it]
 39%|███▉      | 14961/37962 [34:39:59<53:14:54,  8.33s/it]
 39%|███▉      | 14962/37962 [34:40:07<52:24:47,  8.20s/it]
 39%|███▉      | 14963/37962 [34:40:15<52:15:57,  8.18s/it]
 39%|███▉      | 14964/37962 [34:40:24<52:34:52,  8.23s/it]
 39%|███▉      | 14965/37962 [34:40:32<52:55:05,  8.28s/it]
 39%|███▉      | 14966/37962 [34:40:41<53:10:08,  8.32s/it]
 39%|███▉      | 14967/37962 [34:40:49<52:59:42,  8.30s/it]
 39%|███▉      | 14968/37962 [34:40:57<53:26:37,  8.37s/it]
 39%|███▉      | 14969/37962 [34:41:06<53:14:46,  8.34s/it]
 39%|███▉      | 14970/37962 [34:41:14<52:45:36,  8.26s/it]
                                                           
{'loss': 1.2506, 'grad_norm': 12.094663619995117, 'learning_rate': 1.380919099606316e-05, 'epoch': 0.39}

 39%|███▉      | 14970/37962 [34:41:14<52:45:36,  8.26s/it]
 39%|███▉      | 14971/37962 [34:41:22<52:59:43,  8.30s/it]
 39%|███▉      | 14972/37962 [34:41:31<53:30:54,  8.38s/it]
 39%|███▉      | 14973/37962 [34:41:39<53:16:20,  8.34s/it]
 39%|███▉      | 14974/37962 [34:41:47<52:55:02,  8.29s/it]
 39%|███▉      | 14975/37962 [34:41:55<52:43:06,  8.26s/it]
 39%|███▉      | 14976/37962 [34:42:03<52:14:31,  8.18s/it]
 39%|███▉      | 14977/37962 [34:42:12<52:54:21,  8.29s/it]
 39%|███▉      | 14978/37962 [34:42:20<52:48:52,  8.27s/it]
 39%|███▉      | 14979/37962 [34:42:28<52:37:32,  8.24s/it]
 39%|███▉      | 14980/37962 [34:42:36<52:06:49,  8.16s/it]
                                                           
{'loss': 1.3535, 'grad_norm': 29.14276695251465, 'learning_rate': 1.3801301217939474e-05, 'epoch': 0.39}

 39%|███▉      | 14980/37962 [34:42:36<52:06:49,  8.16s/it]
 39%|███▉      | 14981/37962 [34:42:44<51:49:21,  8.12s/it]
 39%|███▉      | 14982/37962 [34:42:52<51:57:47,  8.14s/it]
 39%|███▉      | 14983/37962 [34:43:00<51:41:52,  8.10s/it]
 39%|███▉      | 14984/37962 [34:43:09<52:01:24,  8.15s/it]
 39%|███▉      | 14985/37962 [34:43:17<52:32:45,  8.23s/it]
 39%|███▉      | 14986/37962 [34:43:25<52:31:05,  8.23s/it]
 39%|███▉      | 14987/37962 [34:43:34<52:25:16,  8.21s/it]
 39%|███▉      | 14988/37962 [34:43:42<52:36:32,  8.24s/it]
 39%|███▉      | 14989/37962 [34:43:50<52:04:19,  8.16s/it]
 39%|███▉      | 14990/37962 [34:43:58<52:09:41,  8.17s/it]
                                                           
{'loss': 1.4151, 'grad_norm': 6.793166160583496, 'learning_rate': 1.379340867291448e-05, 'epoch': 0.39}

 39%|███▉      | 14990/37962 [34:43:58<52:09:41,  8.17s/it]
 39%|███▉      | 14991/37962 [34:44:06<52:24:40,  8.21s/it]
 39%|███▉      | 14992/37962 [34:44:15<52:39:39,  8.25s/it]
 39%|███▉      | 14993/37962 [34:44:23<53:01:39,  8.31s/it]
 39%|███▉      | 14994/37962 [34:44:31<52:45:23,  8.27s/it]
 40%|███▉      | 14995/37962 [34:44:40<52:54:13,  8.29s/it]
 40%|███▉      | 14996/37962 [34:44:48<53:07:38,  8.33s/it]
 40%|███▉      | 14997/37962 [34:44:56<53:04:11,  8.32s/it]
 40%|███▉      | 14998/37962 [34:45:05<53:13:18,  8.34s/it]
 40%|███▉      | 14999/37962 [34:45:13<52:56:56,  8.30s/it]
 40%|███▉      | 15000/37962 [34:45:21<52:48:44,  8.28s/it]
                                                           
{'loss': 1.4206, 'grad_norm': 8.16550064086914, 'learning_rate': 1.3785513366733034e-05, 'epoch': 0.4}

 40%|███▉      | 15000/37962 [34:45:21<52:48:44,  8.28s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 40%|███▉      | 15001/37962 [34:46:09<128:28:41, 20.14s/it]
 40%|███▉      | 15002/37962 [34:46:17<106:06:19, 16.64s/it]
 40%|███▉      | 15003/37962 [34:46:26<90:20:44, 14.17s/it] 
 40%|███▉      | 15004/37962 [34:46:34<79:14:29, 12.43s/it]
 40%|███▉      | 15005/37962 [34:46:42<71:02:04, 11.14s/it]
 40%|███▉      | 15006/37962 [34:46:51<65:22:56, 10.25s/it]
 40%|███▉      | 15007/37962 [34:46:59<61:40:19,  9.67s/it]
 40%|███▉      | 15008/37962 [34:47:07<59:04:34,  9.27s/it]
 40%|███▉      | 15009/37962 [34:47:16<57:30:00,  9.02s/it]
 40%|███▉      | 15010/37962 [34:47:24<56:12:13,  8.82s/it]
                                                           
{'loss': 1.3404, 'grad_norm': 18.75669288635254, 'learning_rate': 1.3777615305141989e-05, 'epoch': 0.4}

 40%|███▉      | 15010/37962 [34:47:24<56:12:13,  8.82s/it]
 40%|███▉      | 15011/37962 [34:47:32<55:20:03,  8.68s/it]
 40%|███▉      | 15012/37962 [34:47:41<54:38:46,  8.57s/it]
 40%|███▉      | 15013/37962 [34:47:49<54:08:28,  8.49s/it]
 40%|███▉      | 15014/37962 [34:47:57<53:29:48,  8.39s/it]
 40%|███▉      | 15015/37962 [34:48:05<53:23:54,  8.38s/it]
 40%|███▉      | 15016/37962 [34:48:14<53:23:56,  8.38s/it]
 40%|███▉      | 15017/37962 [34:48:22<52:47:00,  8.28s/it]
 40%|███▉      | 15018/37962 [34:48:30<52:46:47,  8.28s/it]
 40%|███▉      | 15019/37962 [34:48:39<53:22:55,  8.38s/it]
 40%|███▉      | 15020/37962 [34:48:47<53:35:33,  8.41s/it]
                                                           
{'loss': 1.437, 'grad_norm': 12.622206687927246, 'learning_rate': 1.3769714493890203e-05, 'epoch': 0.4}

 40%|███▉      | 15020/37962 [34:48:47<53:35:33,  8.41s/it]
 40%|███▉      | 15021/37962 [34:48:55<52:43:41,  8.27s/it]
 40%|███▉      | 15022/37962 [34:49:03<52:39:20,  8.26s/it]
 40%|███▉      | 15023/37962 [34:49:12<53:01:35,  8.32s/it]
 40%|███▉      | 15024/37962 [34:49:20<53:03:28,  8.33s/it]
 40%|███▉      | 15025/37962 [34:49:28<52:48:11,  8.29s/it]
 40%|███▉      | 15026/37962 [34:49:37<52:48:05,  8.29s/it]
 40%|███▉      | 15027/37962 [34:49:45<52:02:49,  8.17s/it]
 40%|███▉      | 15028/37962 [34:49:53<52:35:39,  8.26s/it]
 40%|███▉      | 15029/37962 [34:50:01<52:20:04,  8.22s/it]
 40%|███▉      | 15030/37962 [34:50:10<52:58:16,  8.32s/it]
                                                           
{'loss': 1.4743, 'grad_norm': 9.983560562133789, 'learning_rate': 1.3761810938728541e-05, 'epoch': 0.4}

 40%|███▉      | 15030/37962 [34:50:10<52:58:16,  8.32s/it]
 40%|███▉      | 15031/37962 [34:50:18<52:51:51,  8.30s/it]
 40%|███▉      | 15032/37962 [34:50:26<52:52:09,  8.30s/it]
 40%|███▉      | 15033/37962 [34:50:35<52:36:06,  8.26s/it]
 40%|███▉      | 15034/37962 [34:50:43<52:34:41,  8.26s/it]
 40%|███▉      | 15035/37962 [34:50:51<52:40:27,  8.27s/it]
 40%|███▉      | 15036/37962 [34:50:59<52:42:16,  8.28s/it]
 40%|███▉      | 15037/37962 [34:51:08<52:30:14,  8.24s/it]
 40%|███▉      | 15038/37962 [34:51:16<52:10:46,  8.19s/it]
 40%|███▉      | 15039/37962 [34:51:24<52:16:17,  8.21s/it]
 40%|███▉      | 15040/37962 [34:51:32<52:39:21,  8.27s/it]
                                                           
{'loss': 1.4266, 'grad_norm': 19.950082778930664, 'learning_rate': 1.375390464540987e-05, 'epoch': 0.4}

 40%|███▉      | 15040/37962 [34:51:32<52:39:21,  8.27s/it]
 40%|███▉      | 15041/37962 [34:51:40<52:06:59,  8.19s/it]
 40%|███▉      | 15042/37962 [34:51:48<52:10:57,  8.20s/it]
 40%|███▉      | 15043/37962 [34:51:57<52:13:33,  8.20s/it]
 40%|███▉      | 15044/37962 [34:52:05<52:18:19,  8.22s/it]
 40%|███▉      | 15045/37962 [34:52:13<52:33:59,  8.26s/it]
 40%|███▉      | 15046/37962 [34:52:22<52:39:54,  8.27s/it]
 40%|███▉      | 15047/37962 [34:52:30<52:45:04,  8.29s/it]
 40%|███▉      | 15048/37962 [34:52:38<52:31:58,  8.25s/it]
 40%|███▉      | 15049/37962 [34:52:46<52:38:24,  8.27s/it]
 40%|███▉      | 15050/37962 [34:52:55<52:27:57,  8.24s/it]
                                                           
{'loss': 1.3316, 'grad_norm': 220.5886688232422, 'learning_rate': 1.3745995619689036e-05, 'epoch': 0.4}

 40%|███▉      | 15050/37962 [34:52:55<52:27:57,  8.24s/it]
 40%|███▉      | 15051/37962 [34:53:03<52:34:29,  8.26s/it]
 40%|███▉      | 15052/37962 [34:53:11<52:39:40,  8.28s/it]
 40%|███▉      | 15053/37962 [34:53:19<52:39:50,  8.28s/it]
 40%|███▉      | 15054/37962 [34:53:28<52:52:39,  8.31s/it]
 40%|███▉      | 15055/37962 [34:53:36<53:02:43,  8.34s/it]
 40%|███▉      | 15056/37962 [34:53:44<52:43:23,  8.29s/it]
 40%|███▉      | 15057/37962 [34:53:53<52:53:17,  8.31s/it]
 40%|███▉      | 15058/37962 [34:54:01<52:46:57,  8.30s/it]
 40%|███▉      | 15059/37962 [34:54:09<52:42:52,  8.29s/it]
 40%|███▉      | 15060/37962 [34:54:17<52:16:32,  8.22s/it]
                                                           
{'loss': 1.3816, 'grad_norm': 54.752262115478516, 'learning_rate': 1.3738083867322885e-05, 'epoch': 0.4}

 40%|███▉      | 15060/37962 [34:54:17<52:16:32,  8.22s/it]
 40%|███▉      | 15061/37962 [34:54:26<52:18:41,  8.22s/it]
 40%|███▉      | 15062/37962 [34:54:34<52:23:24,  8.24s/it]
 40%|███▉      | 15063/37962 [34:54:42<52:03:10,  8.18s/it]
 40%|███▉      | 15064/37962 [34:54:50<52:04:53,  8.19s/it]
 40%|███▉      | 15065/37962 [34:54:58<51:58:07,  8.17s/it]
 40%|███▉      | 15066/37962 [34:55:07<52:29:41,  8.25s/it]
 40%|███▉      | 15067/37962 [34:55:15<52:13:53,  8.21s/it]
 40%|███▉      | 15068/37962 [34:55:23<52:23:35,  8.24s/it]
 40%|███▉      | 15069/37962 [34:55:31<52:06:36,  8.19s/it]
 40%|███▉      | 15070/37962 [34:55:39<52:02:03,  8.18s/it]
                                                           
{'loss': 1.2748, 'grad_norm': 12.10908317565918, 'learning_rate': 1.373016939407025e-05, 'epoch': 0.4}

 40%|███▉      | 15070/37962 [34:55:39<52:02:03,  8.18s/it]
 40%|███▉      | 15071/37962 [34:55:48<51:56:20,  8.17s/it]
 40%|███▉      | 15072/37962 [34:55:56<52:21:22,  8.23s/it]
 40%|███▉      | 15073/37962 [34:56:04<52:28:42,  8.25s/it]
 40%|███▉      | 15074/37962 [34:56:12<52:07:49,  8.20s/it]
 40%|███▉      | 15075/37962 [34:56:20<51:43:42,  8.14s/it]
 40%|███▉      | 15076/37962 [34:56:29<52:06:56,  8.20s/it]
 40%|███▉      | 15077/37962 [34:56:37<52:13:40,  8.22s/it]
 40%|███▉      | 15078/37962 [34:56:45<51:35:34,  8.12s/it]
 40%|███▉      | 15079/37962 [34:56:53<52:02:36,  8.19s/it]
 40%|███▉      | 15080/37962 [34:57:02<52:41:20,  8.29s/it]
                                                           
{'loss': 1.4362, 'grad_norm': 25.632116317749023, 'learning_rate': 1.3722252205691929e-05, 'epoch': 0.4}

 40%|███▉      | 15080/37962 [34:57:02<52:41:20,  8.29s/it]
 40%|███▉      | 15081/37962 [34:57:10<52:35:14,  8.27s/it]
 40%|███▉      | 15082/37962 [34:57:18<52:49:56,  8.31s/it]
 40%|███▉      | 15083/37962 [34:57:27<53:13:49,  8.38s/it]
 40%|███▉      | 15084/37962 [34:57:35<53:42:52,  8.45s/it]
 40%|███▉      | 15085/37962 [34:57:44<53:34:46,  8.43s/it]
 40%|███▉      | 15086/37962 [34:57:52<52:21:44,  8.24s/it]
 40%|███▉      | 15087/37962 [34:58:00<52:54:29,  8.33s/it]
 40%|███▉      | 15088/37962 [34:58:08<52:57:12,  8.33s/it]
 40%|███▉      | 15089/37962 [34:58:17<52:42:41,  8.30s/it]
 40%|███▉      | 15090/37962 [34:58:25<52:50:28,  8.32s/it]
                                                           
{'loss': 1.3004, 'grad_norm': 15.033336639404297, 'learning_rate': 1.3714332307950713e-05, 'epoch': 0.4}

 40%|███▉      | 15090/37962 [34:58:25<52:50:28,  8.32s/it]
 40%|███▉      | 15091/37962 [34:58:33<52:39:43,  8.29s/it]
 40%|███▉      | 15092/37962 [34:58:41<51:27:55,  8.10s/it]
 40%|███▉      | 15093/37962 [34:58:49<51:45:09,  8.15s/it]
 40%|███▉      | 15094/37962 [34:58:57<51:31:11,  8.11s/it]
 40%|███▉      | 15095/37962 [34:59:06<51:51:38,  8.16s/it]
 40%|███▉      | 15096/37962 [34:59:14<52:17:44,  8.23s/it]
 40%|███▉      | 15097/37962 [34:59:22<52:16:10,  8.23s/it]
 40%|███▉      | 15098/37962 [34:59:31<52:47:19,  8.31s/it]
 40%|███▉      | 15099/37962 [34:59:39<52:09:28,  8.21s/it]
 40%|███▉      | 15100/37962 [34:59:47<52:48:50,  8.32s/it]
                                                           
{'loss': 1.4565, 'grad_norm': 17.965044021606445, 'learning_rate': 1.3706409706611354e-05, 'epoch': 0.4}

 40%|███▉      | 15100/37962 [34:59:47<52:48:50,  8.32s/it]
 40%|███▉      | 15101/37962 [34:59:56<52:57:35,  8.34s/it]
 40%|███▉      | 15102/37962 [35:00:04<52:31:49,  8.27s/it]
 40%|███▉      | 15103/37962 [35:00:12<52:25:45,  8.26s/it]
 40%|███▉      | 15104/37962 [35:00:20<52:15:11,  8.23s/it]
 40%|███▉      | 15105/37962 [35:00:28<52:20:59,  8.25s/it]
 40%|███▉      | 15106/37962 [35:00:37<52:48:25,  8.32s/it]
 40%|███▉      | 15107/37962 [35:00:45<52:32:58,  8.28s/it]
 40%|███▉      | 15108/37962 [35:00:53<52:50:30,  8.32s/it]
 40%|███▉      | 15109/37962 [35:01:02<52:42:12,  8.30s/it]
 40%|███▉      | 15110/37962 [35:01:10<52:50:01,  8.32s/it]
                                                           
{'loss': 1.3403, 'grad_norm': 10.019550323486328, 'learning_rate': 1.3698484407440582e-05, 'epoch': 0.4}

 40%|███▉      | 15110/37962 [35:01:10<52:50:01,  8.32s/it]
 40%|███▉      | 15111/37962 [35:01:18<52:40:44,  8.30s/it]
 40%|███▉      | 15112/37962 [35:01:27<52:58:45,  8.35s/it]
 40%|███▉      | 15113/37962 [35:01:35<53:00:11,  8.35s/it]
 40%|███▉      | 15114/37962 [35:01:44<53:40:08,  8.46s/it]
 40%|███▉      | 15115/37962 [35:01:52<53:08:34,  8.37s/it]
 40%|███▉      | 15116/37962 [35:02:01<53:50:51,  8.49s/it]
 40%|███▉      | 15117/37962 [35:02:09<53:20:11,  8.40s/it]
 40%|███▉      | 15118/37962 [35:02:17<53:14:03,  8.39s/it]
 40%|███▉      | 15119/37962 [35:02:26<52:51:30,  8.33s/it]
 40%|███▉      | 15120/37962 [35:02:34<52:41:21,  8.30s/it]
                                                           
{'loss': 1.4399, 'grad_norm': 21.99286460876465, 'learning_rate': 1.3690556416207081e-05, 'epoch': 0.4}

 40%|███▉      | 15120/37962 [35:02:34<52:41:21,  8.30s/it]
 40%|███▉      | 15121/37962 [35:02:42<52:42:48,  8.31s/it]
 40%|███▉      | 15122/37962 [35:02:50<52:52:47,  8.33s/it]
 40%|███▉      | 15123/37962 [35:02:59<52:36:56,  8.29s/it]
 40%|███▉      | 15124/37962 [35:03:07<52:34:05,  8.29s/it]
 40%|███▉      | 15125/37962 [35:03:15<52:43:30,  8.31s/it]
 40%|███▉      | 15126/37962 [35:03:24<52:45:55,  8.32s/it]
 40%|███▉      | 15127/37962 [35:03:32<52:51:34,  8.33s/it]
 40%|███▉      | 15128/37962 [35:03:40<52:46:33,  8.32s/it]
 40%|███▉      | 15129/37962 [35:03:49<52:55:51,  8.35s/it]
 40%|███▉      | 15130/37962 [35:03:57<52:25:57,  8.27s/it]
                                                           
{'loss': 1.2982, 'grad_norm': 16.343158721923828, 'learning_rate': 1.3682625738681496e-05, 'epoch': 0.4}

 40%|███▉      | 15130/37962 [35:03:57<52:25:57,  8.27s/it]
 40%|███▉      | 15131/37962 [35:04:05<52:29:44,  8.28s/it]
 40%|███▉      | 15132/37962 [35:04:13<52:28:54,  8.28s/it]
 40%|███▉      | 15133/37962 [35:04:22<52:18:51,  8.25s/it]
 40%|███▉      | 15134/37962 [35:04:30<52:42:13,  8.31s/it]
 40%|███▉      | 15135/37962 [35:04:38<52:39:07,  8.30s/it]
 40%|███▉      | 15136/37962 [35:04:47<52:38:24,  8.30s/it]
 40%|███▉      | 15137/37962 [35:04:55<52:29:13,  8.28s/it]
 40%|███▉      | 15138/37962 [35:05:03<52:46:23,  8.32s/it]
 40%|███▉      | 15139/37962 [35:05:11<52:28:50,  8.28s/it]
 40%|███▉      | 15140/37962 [35:05:20<52:24:06,  8.27s/it]
                                                           
{'loss': 1.3837, 'grad_norm': 37.03549575805664, 'learning_rate': 1.3674692380636438e-05, 'epoch': 0.4}

 40%|███▉      | 15140/37962 [35:05:20<52:24:06,  8.27s/it]
 40%|███▉      | 15141/37962 [35:05:28<52:17:23,  8.25s/it]
 40%|███▉      | 15142/37962 [35:05:36<52:50:18,  8.34s/it]
 40%|███▉      | 15143/37962 [35:05:44<51:59:50,  8.20s/it]
 40%|███▉      | 15144/37962 [35:05:53<52:07:14,  8.22s/it]
 40%|███▉      | 15145/37962 [35:06:01<52:02:21,  8.21s/it]
 40%|███▉      | 15146/37962 [35:06:09<52:16:28,  8.25s/it]
 40%|███▉      | 15147/37962 [35:06:17<52:18:18,  8.25s/it]
 40%|███▉      | 15148/37962 [35:06:26<52:05:08,  8.22s/it]
 40%|███▉      | 15149/37962 [35:06:34<52:18:40,  8.25s/it]
 40%|███▉      | 15150/37962 [35:06:42<52:25:45,  8.27s/it]
                                                           
{'loss': 1.4369, 'grad_norm': 84.72125244140625, 'learning_rate': 1.3666756347846452e-05, 'epoch': 0.4}

 40%|███▉      | 15150/37962 [35:06:42<52:25:45,  8.27s/it]
 40%|███▉      | 15151/37962 [35:06:51<52:37:34,  8.31s/it]
 40%|███▉      | 15152/37962 [35:06:59<52:29:19,  8.28s/it]
 40%|███▉      | 15153/37962 [35:07:07<52:50:52,  8.34s/it]
 40%|███▉      | 15154/37962 [35:07:16<53:10:44,  8.39s/it]
 40%|███▉      | 15155/37962 [35:07:24<52:45:31,  8.33s/it]
 40%|███▉      | 15156/37962 [35:07:32<52:41:16,  8.32s/it]
 40%|███▉      | 15157/37962 [35:07:40<51:29:46,  8.13s/it]
 40%|███▉      | 15158/37962 [35:07:48<51:43:31,  8.17s/it]
 40%|███▉      | 15159/37962 [35:07:56<51:32:29,  8.14s/it]
 40%|███▉      | 15160/37962 [35:08:04<51:39:10,  8.15s/it]
                                                           
{'loss': 1.4695, 'grad_norm': 25.428346633911133, 'learning_rate': 1.3658817646088045e-05, 'epoch': 0.4}

 40%|███▉      | 15160/37962 [35:08:04<51:39:10,  8.15s/it]
 40%|███▉      | 15161/37962 [35:08:12<50:49:48,  8.03s/it]
 40%|███▉      | 15162/37962 [35:08:20<51:17:35,  8.10s/it]
 40%|███▉      | 15163/37962 [35:08:29<51:54:58,  8.20s/it]
 40%|███▉      | 15164/37962 [35:08:37<51:22:38,  8.11s/it]
 40%|███▉      | 15165/37962 [35:08:45<51:51:38,  8.19s/it]
 40%|███▉      | 15166/37962 [35:08:53<51:01:49,  8.06s/it]
 40%|███▉      | 15167/37962 [35:09:01<51:45:53,  8.18s/it]
 40%|███▉      | 15168/37962 [35:09:09<51:35:39,  8.15s/it]
 40%|███▉      | 15169/37962 [35:09:17<51:14:26,  8.09s/it]
 40%|███▉      | 15170/37962 [35:09:26<51:25:13,  8.12s/it]
                                                           
{'loss': 1.3223, 'grad_norm': 32.85851287841797, 'learning_rate': 1.3650876281139657e-05, 'epoch': 0.4}

 40%|███▉      | 15170/37962 [35:09:26<51:25:13,  8.12s/it]
 40%|███▉      | 15171/37962 [35:09:34<51:54:21,  8.20s/it]
 40%|███▉      | 15172/37962 [35:09:42<51:25:43,  8.12s/it]
 40%|███▉      | 15173/37962 [35:09:50<51:34:47,  8.15s/it]
 40%|███▉      | 15174/37962 [35:09:58<51:43:31,  8.17s/it]
 40%|███▉      | 15175/37962 [35:10:07<52:03:32,  8.22s/it]
 40%|███▉      | 15176/37962 [35:10:15<51:50:52,  8.19s/it]
 40%|███▉      | 15177/37962 [35:10:23<51:53:28,  8.20s/it]
 40%|███▉      | 15178/37962 [35:10:31<52:06:31,  8.23s/it]
 40%|███▉      | 15179/37962 [35:10:39<51:27:48,  8.13s/it]
 40%|███▉      | 15180/37962 [35:10:48<52:00:03,  8.22s/it]
                                                           
{'loss': 1.3116, 'grad_norm': 10.869043350219727, 'learning_rate': 1.364293225878167e-05, 'epoch': 0.4}

 40%|███▉      | 15180/37962 [35:10:48<52:00:03,  8.22s/it]
 40%|███▉      | 15181/37962 [35:10:56<52:13:10,  8.25s/it]
 40%|███▉      | 15182/37962 [35:11:04<52:05:40,  8.23s/it]
 40%|███▉      | 15183/37962 [35:11:12<52:08:01,  8.24s/it]
 40%|███▉      | 15184/37962 [35:11:21<52:56:59,  8.37s/it]
 40%|████      | 15185/37962 [35:11:30<53:06:09,  8.39s/it]
 40%|████      | 15186/37962 [35:11:38<52:49:36,  8.35s/it]
 40%|████      | 15187/37962 [35:11:46<52:28:40,  8.30s/it]
 40%|████      | 15188/37962 [35:11:54<52:44:19,  8.34s/it]
 40%|████      | 15189/37962 [35:12:03<52:27:08,  8.29s/it]
 40%|████      | 15190/37962 [35:12:11<52:02:02,  8.23s/it]
                                                           
{'loss': 1.4307, 'grad_norm': 74.46104431152344, 'learning_rate': 1.3634985584796403e-05, 'epoch': 0.4}

 40%|████      | 15190/37962 [35:12:11<52:02:02,  8.23s/it]
 40%|████      | 15191/37962 [35:12:19<51:33:20,  8.15s/it]
 40%|████      | 15192/37962 [35:12:27<51:44:36,  8.18s/it]
 40%|████      | 15193/37962 [35:12:35<51:33:53,  8.15s/it]
 40%|████      | 15194/37962 [35:12:43<51:41:07,  8.17s/it]
 40%|████      | 15195/37962 [35:12:51<51:54:51,  8.21s/it]
 40%|████      | 15196/37962 [35:13:00<52:14:43,  8.26s/it]
 40%|████      | 15197/37962 [35:13:08<52:18:56,  8.27s/it]
 40%|████      | 15198/37962 [35:13:16<52:08:40,  8.25s/it]
 40%|████      | 15199/37962 [35:13:25<52:25:00,  8.29s/it]
 40%|████      | 15200/37962 [35:13:33<52:29:44,  8.30s/it]
                                                           
{'loss': 1.3973, 'grad_norm': 178.43858337402344, 'learning_rate': 1.3627036264968098e-05, 'epoch': 0.4}

 40%|████      | 15200/37962 [35:13:33<52:29:44,  8.30s/it]
 40%|████      | 15201/37962 [35:13:41<52:29:23,  8.30s/it]
 40%|████      | 15202/37962 [35:13:50<52:46:48,  8.35s/it]
 40%|████      | 15203/37962 [35:13:58<52:38:29,  8.33s/it]
 40%|████      | 15204/37962 [35:14:06<51:51:22,  8.20s/it]
 40%|████      | 15205/37962 [35:14:14<52:07:32,  8.25s/it]
 40%|████      | 15206/37962 [35:14:23<52:26:06,  8.30s/it]
 40%|████      | 15207/37962 [35:14:32<53:17:06,  8.43s/it]
 40%|████      | 15208/37962 [35:14:40<53:10:39,  8.41s/it]
 40%|████      | 15209/37962 [35:14:48<53:17:04,  8.43s/it]
 40%|████      | 15210/37962 [35:14:57<52:54:43,  8.37s/it]
                                                           
{'loss': 1.3751, 'grad_norm': 8.72211742401123, 'learning_rate': 1.3619084305082932e-05, 'epoch': 0.4}

 40%|████      | 15210/37962 [35:14:57<52:54:43,  8.37s/it]
 40%|████      | 15211/37962 [35:15:05<52:35:23,  8.32s/it]
 40%|████      | 15212/37962 [35:15:13<52:16:56,  8.27s/it]
 40%|████      | 15213/37962 [35:15:21<52:27:36,  8.30s/it]
 40%|████      | 15214/37962 [35:15:29<51:27:48,  8.14s/it]
 40%|████      | 15215/37962 [35:15:37<51:30:32,  8.15s/it]
 40%|████      | 15216/37962 [35:15:46<51:46:49,  8.20s/it]
 40%|████      | 15217/37962 [35:15:54<51:47:58,  8.20s/it]
 40%|████      | 15218/37962 [35:16:02<52:06:01,  8.25s/it]
 40%|████      | 15219/37962 [35:16:10<51:37:28,  8.17s/it]
 40%|████      | 15220/37962 [35:16:18<51:54:50,  8.22s/it]
                                                           
{'loss': 1.3297, 'grad_norm': 14.177505493164062, 'learning_rate': 1.3611129710928998e-05, 'epoch': 0.4}

 40%|████      | 15220/37962 [35:16:18<51:54:50,  8.22s/it]
 40%|████      | 15221/37962 [35:16:27<52:11:40,  8.26s/it]
 40%|████      | 15222/37962 [35:16:35<52:17:54,  8.28s/it]
 40%|████      | 15223/37962 [35:16:43<51:55:25,  8.22s/it]
 40%|████      | 15224/37962 [35:16:51<51:20:38,  8.13s/it]
 40%|████      | 15225/37962 [35:17:00<51:50:49,  8.21s/it]
 40%|████      | 15226/37962 [35:17:08<52:17:34,  8.28s/it]
 40%|████      | 15227/37962 [35:17:16<52:20:58,  8.29s/it]
 40%|████      | 15228/37962 [35:17:25<52:21:48,  8.29s/it]
 40%|████      | 15229/37962 [35:17:33<52:23:52,  8.30s/it]
 40%|████      | 15230/37962 [35:17:41<52:23:19,  8.30s/it]
                                                           
{'loss': 1.4046, 'grad_norm': 32.30446243286133, 'learning_rate': 1.3603172488296304e-05, 'epoch': 0.4}

 40%|████      | 15230/37962 [35:17:41<52:23:19,  8.30s/it]
 40%|████      | 15231/37962 [35:17:50<52:26:03,  8.30s/it]
 40%|████      | 15232/37962 [35:17:58<52:19:12,  8.29s/it]
 40%|████      | 15233/37962 [35:18:06<51:48:54,  8.21s/it]
 40%|████      | 15234/37962 [35:18:14<51:45:42,  8.20s/it]
 40%|████      | 15235/37962 [35:18:22<52:09:19,  8.26s/it]
 40%|████      | 15236/37962 [35:18:31<52:27:58,  8.31s/it]
 40%|████      | 15237/37962 [35:18:39<52:22:27,  8.30s/it]
 40%|████      | 15238/37962 [35:18:47<52:11:22,  8.27s/it]
 40%|████      | 15239/37962 [35:18:55<51:24:47,  8.15s/it]
 40%|████      | 15240/37962 [35:19:03<51:43:34,  8.20s/it]
                                                           
{'loss': 1.4261, 'grad_norm': 13.893932342529297, 'learning_rate': 1.3595212642976775e-05, 'epoch': 0.4}

 40%|████      | 15240/37962 [35:19:03<51:43:34,  8.20s/it]
 40%|████      | 15241/37962 [35:19:12<51:48:52,  8.21s/it]
 40%|████      | 15242/37962 [35:19:20<52:00:58,  8.24s/it]
 40%|████      | 15243/37962 [35:19:28<52:14:55,  8.28s/it]
 40%|████      | 15244/37962 [35:19:36<51:39:42,  8.19s/it]
 40%|████      | 15245/37962 [35:19:44<50:44:03,  8.04s/it]
 40%|████      | 15246/37962 [35:19:52<51:29:53,  8.16s/it]
 40%|████      | 15247/37962 [35:20:01<52:17:25,  8.29s/it]
 40%|████      | 15248/37962 [35:20:09<52:24:45,  8.31s/it]
 40%|████      | 15249/37962 [35:20:18<52:17:26,  8.29s/it]
 40%|████      | 15250/37962 [35:20:26<52:11:28,  8.27s/it]
                                                           
{'loss': 1.2977, 'grad_norm': 170.42160034179688, 'learning_rate': 1.358725018076425e-05, 'epoch': 0.4}

 40%|████      | 15250/37962 [35:20:26<52:11:28,  8.27s/it]
 40%|████      | 15251/37962 [35:20:34<52:37:05,  8.34s/it]
 40%|████      | 15252/37962 [35:20:43<52:16:27,  8.29s/it]
 40%|████      | 15253/37962 [35:20:51<52:12:26,  8.28s/it]
 40%|████      | 15254/37962 [35:20:59<52:06:31,  8.26s/it]
 40%|████      | 15255/37962 [35:21:07<52:07:32,  8.26s/it]
 40%|████      | 15256/37962 [35:21:16<52:04:26,  8.26s/it]
 40%|████      | 15257/37962 [35:21:24<52:19:49,  8.30s/it]
 40%|████      | 15258/37962 [35:21:32<52:18:34,  8.29s/it]
 40%|████      | 15259/37962 [35:21:40<52:00:47,  8.25s/it]
 40%|████      | 15260/37962 [35:21:49<51:49:59,  8.22s/it]
                                                           
{'loss': 1.3176, 'grad_norm': 37.966896057128906, 'learning_rate': 1.357928510745446e-05, 'epoch': 0.4}

 40%|████      | 15260/37962 [35:21:49<51:49:59,  8.22s/it]
 40%|████      | 15261/37962 [35:21:57<51:57:38,  8.24s/it]
 40%|████      | 15262/37962 [35:22:05<52:02:16,  8.25s/it]
 40%|████      | 15263/37962 [35:22:13<52:18:14,  8.30s/it]
 40%|████      | 15264/37962 [35:22:22<52:15:11,  8.29s/it]
 40%|████      | 15265/37962 [35:22:30<52:26:47,  8.32s/it]
 40%|████      | 15266/37962 [35:22:38<52:24:16,  8.31s/it]
 40%|████      | 15267/37962 [35:22:47<52:15:29,  8.29s/it]
 40%|████      | 15268/37962 [35:22:55<51:57:51,  8.24s/it]
 40%|████      | 15269/37962 [35:23:03<51:51:57,  8.23s/it]
 40%|████      | 15270/37962 [35:23:11<51:54:08,  8.23s/it]
                                                           
{'loss': 1.3153, 'grad_norm': 8.381155967712402, 'learning_rate': 1.3571317428845048e-05, 'epoch': 0.4}

 40%|████      | 15270/37962 [35:23:11<51:54:08,  8.23s/it]
 40%|████      | 15271/37962 [35:23:20<52:04:07,  8.26s/it]
 40%|████      | 15272/37962 [35:23:28<52:11:13,  8.28s/it]
 40%|████      | 15273/37962 [35:23:36<52:17:58,  8.30s/it]
 40%|████      | 15274/37962 [35:23:45<52:27:50,  8.32s/it]
 40%|████      | 15275/37962 [35:24:00<65:09:08, 10.34s/it]
 40%|████      | 15276/37962 [35:24:08<61:44:08,  9.80s/it]
 40%|████      | 15277/37962 [35:24:17<59:02:17,  9.37s/it]
 40%|████      | 15278/37962 [35:24:25<56:30:14,  8.97s/it]
 40%|████      | 15279/37962 [35:24:33<55:12:22,  8.76s/it]
 40%|████      | 15280/37962 [35:24:41<54:17:21,  8.62s/it]
                                                           
{'loss': 1.3636, 'grad_norm': 10.543867111206055, 'learning_rate': 1.356334715073555e-05, 'epoch': 0.4}

 40%|████      | 15280/37962 [35:24:41<54:17:21,  8.62s/it]
 40%|████      | 15281/37962 [35:24:49<53:40:19,  8.52s/it]
 40%|████      | 15282/37962 [35:24:58<53:05:00,  8.43s/it]
 40%|████      | 15283/37962 [35:25:06<52:54:01,  8.40s/it]
 40%|████      | 15284/37962 [35:25:14<52:49:51,  8.39s/it]
 40%|████      | 15285/37962 [35:25:22<52:12:13,  8.29s/it]
 40%|████      | 15286/37962 [35:25:31<52:25:26,  8.32s/it]
 40%|████      | 15287/37962 [35:25:39<52:22:12,  8.31s/it]
 40%|████      | 15288/37962 [35:25:47<52:17:10,  8.30s/it]
 40%|████      | 15289/37962 [35:25:56<52:27:15,  8.33s/it]
 40%|████      | 15290/37962 [35:26:04<52:28:11,  8.33s/it]
                                                           
{'loss': 1.2938, 'grad_norm': 12.562834739685059, 'learning_rate': 1.3555374278927392e-05, 'epoch': 0.4}

 40%|████      | 15290/37962 [35:26:04<52:28:11,  8.33s/it]
 40%|████      | 15291/37962 [35:26:12<52:17:50,  8.30s/it]
 40%|████      | 15292/37962 [35:26:21<52:21:57,  8.32s/it]
 40%|████      | 15293/37962 [35:26:29<52:23:55,  8.32s/it]
 40%|████      | 15294/37962 [35:26:37<52:11:27,  8.29s/it]
 40%|████      | 15295/37962 [35:26:45<52:02:52,  8.27s/it]
 40%|████      | 15296/37962 [35:26:54<52:10:37,  8.29s/it]
 40%|████      | 15297/37962 [35:27:02<51:31:57,  8.19s/it]
 40%|████      | 15298/37962 [35:27:10<51:38:19,  8.20s/it]
 40%|████      | 15299/37962 [35:27:18<51:03:20,  8.11s/it]
 40%|████      | 15300/37962 [35:27:26<50:58:06,  8.10s/it]
                                                           
{'loss': 1.2472, 'grad_norm': 21.459482192993164, 'learning_rate': 1.3547398819223885e-05, 'epoch': 0.4}

 40%|████      | 15300/37962 [35:27:26<50:58:06,  8.10s/it]
 40%|████      | 15301/37962 [35:27:34<51:29:54,  8.18s/it]
 40%|████      | 15302/37962 [35:27:43<51:53:26,  8.24s/it]
 40%|████      | 15303/37962 [35:27:51<52:16:31,  8.31s/it]
 40%|████      | 15304/37962 [35:28:00<52:37:40,  8.36s/it]
 40%|████      | 15305/37962 [35:28:08<52:01:04,  8.27s/it]
 40%|████      | 15306/37962 [35:28:16<52:08:39,  8.29s/it]
 40%|████      | 15307/37962 [35:28:24<52:02:26,  8.27s/it]
 40%|████      | 15308/37962 [35:28:32<51:58:43,  8.26s/it]
 40%|████      | 15309/37962 [35:28:41<51:47:47,  8.23s/it]
 40%|████      | 15310/37962 [35:28:49<51:13:34,  8.14s/it]
                                                           
{'loss': 1.5274, 'grad_norm': 8.907787322998047, 'learning_rate': 1.3539420777430235e-05, 'epoch': 0.4}

 40%|████      | 15310/37962 [35:28:49<51:13:34,  8.14s/it]
 40%|████      | 15311/37962 [35:28:57<51:13:53,  8.14s/it]
 40%|████      | 15312/37962 [35:29:05<51:23:21,  8.17s/it]
 40%|████      | 15313/37962 [35:29:13<51:47:49,  8.23s/it]
 40%|████      | 15314/37962 [35:29:22<51:53:09,  8.25s/it]
 40%|████      | 15315/37962 [35:29:30<52:14:45,  8.31s/it]
 40%|████      | 15316/37962 [35:29:38<52:09:22,  8.29s/it]
 40%|████      | 15317/37962 [35:29:46<51:41:53,  8.22s/it]
 40%|████      | 15318/37962 [35:29:55<51:50:31,  8.24s/it]
 40%|████      | 15319/37962 [35:30:03<51:33:22,  8.20s/it]
 40%|████      | 15320/37962 [35:30:11<51:54:17,  8.25s/it]
                                                           
{'loss': 1.4272, 'grad_norm': 10.799487113952637, 'learning_rate': 1.3531440159353518e-05, 'epoch': 0.4}

 40%|████      | 15320/37962 [35:30:11<51:54:17,  8.25s/it]
 40%|████      | 15321/37962 [35:30:19<51:47:07,  8.23s/it]
 40%|████      | 15322/37962 [35:30:27<51:24:32,  8.17s/it]
 40%|████      | 15323/37962 [35:30:35<51:16:50,  8.15s/it]
 40%|████      | 15324/37962 [35:30:44<51:20:19,  8.16s/it]
 40%|████      | 15325/37962 [35:30:52<51:49:12,  8.24s/it]
 40%|████      | 15326/37962 [35:31:00<52:01:13,  8.27s/it]
 40%|████      | 15327/37962 [35:31:09<52:02:18,  8.28s/it]
 40%|████      | 15328/37962 [35:31:17<52:00:38,  8.27s/it]
 40%|████      | 15329/37962 [35:31:25<51:54:43,  8.26s/it]
 40%|████      | 15330/37962 [35:31:33<51:59:08,  8.27s/it]
                                                           
{'loss': 1.4475, 'grad_norm': 6.930497646331787, 'learning_rate': 1.3523456970802684e-05, 'epoch': 0.4}

 40%|████      | 15330/37962 [35:31:33<51:59:08,  8.27s/it]
 40%|████      | 15331/37962 [35:31:42<51:48:58,  8.24s/it]
 40%|████      | 15332/37962 [35:31:50<51:42:12,  8.23s/it]
 40%|████      | 15333/37962 [35:31:58<51:45:09,  8.23s/it]
 40%|████      | 15334/37962 [35:32:06<51:49:07,  8.24s/it]
 40%|████      | 15335/37962 [35:32:15<51:39:19,  8.22s/it]
 40%|████      | 15336/37962 [35:32:23<51:55:16,  8.26s/it]
 40%|████      | 15337/37962 [35:32:31<51:58:14,  8.27s/it]
 40%|████      | 15338/37962 [35:32:39<51:35:29,  8.21s/it]
 40%|████      | 15339/37962 [35:32:48<51:54:39,  8.26s/it]
 40%|████      | 15340/37962 [35:32:56<51:24:10,  8.18s/it]
                                                           
{'loss': 1.3667, 'grad_norm': 7.383173942565918, 'learning_rate': 1.351547121758856e-05, 'epoch': 0.4}

 40%|████      | 15340/37962 [35:32:56<51:24:10,  8.18s/it]
 40%|████      | 15341/37962 [35:33:04<51:41:32,  8.23s/it]
 40%|████      | 15342/37962 [35:33:12<51:34:48,  8.21s/it]
 40%|████      | 15343/37962 [35:33:20<51:34:21,  8.21s/it]
 40%|████      | 15344/37962 [35:33:29<52:22:13,  8.34s/it]
 40%|████      | 15345/37962 [35:33:37<52:05:48,  8.29s/it]
 40%|████      | 15346/37962 [35:33:45<51:55:45,  8.27s/it]
 40%|████      | 15347/37962 [35:33:54<51:55:47,  8.27s/it]
 40%|████      | 15348/37962 [35:34:02<52:19:20,  8.33s/it]
 40%|████      | 15349/37962 [35:34:10<52:04:31,  8.29s/it]
 40%|████      | 15350/37962 [35:34:19<51:55:07,  8.27s/it]
                                                           
{'loss': 1.4454, 'grad_norm': 10.1578369140625, 'learning_rate': 1.3507482905523843e-05, 'epoch': 0.4}

 40%|████      | 15350/37962 [35:34:19<51:55:07,  8.27s/it]
 40%|████      | 15351/37962 [35:34:27<51:47:53,  8.25s/it]
 40%|████      | 15352/37962 [35:34:35<52:07:33,  8.30s/it]
 40%|████      | 15353/37962 [35:34:43<51:52:00,  8.26s/it]
 40%|████      | 15354/37962 [35:34:52<51:45:22,  8.24s/it]
 40%|████      | 15355/37962 [35:35:00<51:43:03,  8.24s/it]
 40%|████      | 15356/37962 [35:35:08<51:58:08,  8.28s/it]
 40%|████      | 15357/37962 [35:35:16<52:01:12,  8.28s/it]
 40%|████      | 15358/37962 [35:35:24<51:32:46,  8.21s/it]
 40%|████      | 15359/37962 [35:35:33<51:40:16,  8.23s/it]
 40%|████      | 15360/37962 [35:35:41<51:00:30,  8.12s/it]
                                                           
{'loss': 1.4197, 'grad_norm': 5.529361724853516, 'learning_rate': 1.3499492040423082e-05, 'epoch': 0.4}

 40%|████      | 15360/37962 [35:35:41<51:00:30,  8.12s/it]
 40%|████      | 15361/37962 [35:35:49<51:43:09,  8.24s/it]
 40%|████      | 15362/37962 [35:35:57<51:52:11,  8.26s/it]
 40%|████      | 15363/37962 [35:36:06<51:41:01,  8.23s/it]
 40%|████      | 15364/37962 [35:36:14<52:12:23,  8.32s/it]
 40%|████      | 15365/37962 [35:36:22<52:11:44,  8.32s/it]
 40%|████      | 15366/37962 [35:36:31<52:20:17,  8.34s/it]
 40%|████      | 15367/37962 [35:36:39<52:28:57,  8.36s/it]
 40%|████      | 15368/37962 [35:36:47<52:17:30,  8.33s/it]
 40%|████      | 15369/37962 [35:36:56<52:30:22,  8.37s/it]
 40%|████      | 15370/37962 [35:37:04<52:25:08,  8.35s/it]
                                                           
{'loss': 1.4365, 'grad_norm': 10.250508308410645, 'learning_rate': 1.3491498628102683e-05, 'epoch': 0.4}

 40%|████      | 15370/37962 [35:37:04<52:25:08,  8.35s/it]
 40%|████      | 15371/37962 [35:37:12<52:10:59,  8.32s/it]
 40%|████      | 15372/37962 [35:37:21<52:03:42,  8.30s/it]
 40%|████      | 15373/37962 [35:37:29<52:04:24,  8.30s/it]
 40%|████      | 15374/37962 [35:37:37<51:54:16,  8.27s/it]
 41%|████      | 15375/37962 [35:37:45<51:34:28,  8.22s/it]
 41%|████      | 15376/37962 [35:37:53<51:18:07,  8.18s/it]
 41%|████      | 15377/37962 [35:38:02<52:12:58,  8.32s/it]
 41%|████      | 15378/37962 [35:38:10<52:17:24,  8.34s/it]
 41%|████      | 15379/37962 [35:38:18<51:20:53,  8.19s/it]
 41%|████      | 15380/37962 [35:38:26<51:12:18,  8.16s/it]
                                                           
{'loss': 1.2701, 'grad_norm': 4.068673610687256, 'learning_rate': 1.3483502674380923e-05, 'epoch': 0.41}

 41%|████      | 15380/37962 [35:38:26<51:12:18,  8.16s/it]
 41%|████      | 15381/37962 [35:38:34<51:03:24,  8.14s/it]
 41%|████      | 15382/37962 [35:38:43<51:29:19,  8.21s/it]
 41%|████      | 15383/37962 [35:38:51<51:22:13,  8.19s/it]
 41%|████      | 15384/37962 [35:38:59<50:58:09,  8.13s/it]
 41%|████      | 15385/37962 [35:39:07<51:08:48,  8.16s/it]
 41%|████      | 15386/37962 [35:39:16<51:25:54,  8.20s/it]
 41%|████      | 15387/37962 [35:39:24<51:42:52,  8.25s/it]
 41%|████      | 15388/37962 [35:39:32<51:30:06,  8.21s/it]
 41%|████      | 15389/37962 [35:39:40<50:58:07,  8.13s/it]
 41%|████      | 15390/37962 [35:39:49<51:50:06,  8.27s/it]
                                                           
{'loss': 1.3523, 'grad_norm': 20.9726505279541, 'learning_rate': 1.3475504185077915e-05, 'epoch': 0.41}

 41%|████      | 15390/37962 [35:39:49<51:50:06,  8.27s/it]
 41%|████      | 15391/37962 [35:39:57<52:22:07,  8.35s/it]
 41%|████      | 15392/37962 [35:40:05<52:20:15,  8.35s/it]
 41%|████      | 15393/37962 [35:40:14<52:22:06,  8.35s/it]
 41%|████      | 15394/37962 [35:40:22<52:14:04,  8.33s/it]
 41%|████      | 15395/37962 [35:40:30<52:18:36,  8.34s/it]
 41%|████      | 15396/37962 [35:40:39<52:06:02,  8.31s/it]
 41%|████      | 15397/37962 [35:40:47<51:59:06,  8.29s/it]
 41%|████      | 15398/37962 [35:40:55<51:33:56,  8.23s/it]
 41%|████      | 15399/37962 [35:41:03<51:46:53,  8.26s/it]
 41%|████      | 15400/37962 [35:41:12<52:14:46,  8.34s/it]
                                                           
{'loss': 1.3059, 'grad_norm': 5.3065185546875, 'learning_rate': 1.3467503166015616e-05, 'epoch': 0.41}

 41%|████      | 15400/37962 [35:41:12<52:14:46,  8.34s/it]
 41%|████      | 15401/37962 [35:41:20<52:02:14,  8.30s/it]
 41%|████      | 15402/37962 [35:41:28<51:50:11,  8.27s/it]
 41%|████      | 15403/37962 [35:41:37<51:58:38,  8.29s/it]
 41%|████      | 15404/37962 [35:41:45<51:53:15,  8.28s/it]
 41%|████      | 15405/37962 [35:41:53<51:59:10,  8.30s/it]
 41%|████      | 15406/37962 [35:42:02<52:30:15,  8.38s/it]
 41%|████      | 15407/37962 [35:42:10<52:40:05,  8.41s/it]
 41%|████      | 15408/37962 [35:42:18<52:11:51,  8.33s/it]
 41%|████      | 15409/37962 [35:42:26<51:27:57,  8.22s/it]
 41%|████      | 15410/37962 [35:42:35<51:33:33,  8.23s/it]
                                                           
{'loss': 1.4898, 'grad_norm': 5.9524664878845215, 'learning_rate': 1.3459499623017835e-05, 'epoch': 0.41}

 41%|████      | 15410/37962 [35:42:35<51:33:33,  8.23s/it]
 41%|████      | 15411/37962 [35:42:43<51:36:11,  8.24s/it]
 41%|████      | 15412/37962 [35:42:51<51:32:16,  8.23s/it]
 41%|████      | 15413/37962 [35:42:59<51:24:49,  8.21s/it]
 41%|████      | 15414/37962 [35:43:08<51:36:21,  8.24s/it]
 41%|████      | 15415/37962 [35:43:16<51:42:35,  8.26s/it]
 41%|████      | 15416/37962 [35:43:24<51:18:45,  8.19s/it]
 41%|████      | 15417/37962 [35:43:32<51:24:44,  8.21s/it]
 41%|████      | 15418/37962 [35:43:40<51:22:41,  8.20s/it]
 41%|████      | 15419/37962 [35:43:48<51:13:28,  8.18s/it]
 41%|████      | 15420/37962 [35:43:57<51:26:19,  8.21s/it]
                                                           
{'loss': 1.3894, 'grad_norm': 5.131724834442139, 'learning_rate': 1.3451493561910211e-05, 'epoch': 0.41}

 41%|████      | 15420/37962 [35:43:57<51:26:19,  8.21s/it]
 41%|████      | 15421/37962 [35:44:05<51:30:48,  8.23s/it]
 41%|████      | 15422/37962 [35:44:13<51:25:46,  8.21s/it]
 41%|████      | 15423/37962 [35:44:21<50:53:55,  8.13s/it]
 41%|████      | 15424/37962 [35:44:29<51:08:31,  8.17s/it]
 41%|████      | 15425/37962 [35:44:38<51:26:59,  8.22s/it]
 41%|████      | 15426/37962 [35:44:46<51:30:32,  8.23s/it]
 41%|████      | 15427/37962 [35:44:54<51:44:01,  8.26s/it]
 41%|████      | 15428/37962 [35:45:03<52:13:57,  8.34s/it]
 41%|████      | 15429/37962 [35:45:11<52:06:35,  8.33s/it]
 41%|████      | 15430/37962 [35:45:19<51:52:33,  8.29s/it]
                                                           
{'loss': 1.3855, 'grad_norm': 11.42402172088623, 'learning_rate': 1.3443484988520217e-05, 'epoch': 0.41}

 41%|████      | 15430/37962 [35:45:19<51:52:33,  8.29s/it]
 41%|████      | 15431/37962 [35:45:28<51:57:22,  8.30s/it]
 41%|████      | 15432/37962 [35:45:36<52:04:06,  8.32s/it]
 41%|████      | 15433/37962 [35:45:44<52:01:08,  8.31s/it]
 41%|████      | 15434/37962 [35:45:52<51:23:42,  8.21s/it]
 41%|████      | 15435/37962 [35:46:01<51:30:04,  8.23s/it]
 41%|████      | 15436/37962 [35:46:09<51:16:06,  8.19s/it]
 41%|████      | 15437/37962 [35:46:17<51:32:12,  8.24s/it]
 41%|████      | 15438/37962 [35:46:25<51:35:43,  8.25s/it]
 41%|████      | 15439/37962 [35:46:34<51:51:22,  8.29s/it]
 41%|████      | 15440/37962 [35:46:42<51:13:05,  8.19s/it]
                                                           
{'loss': 1.3705, 'grad_norm': 11.634478569030762, 'learning_rate': 1.343547390867715e-05, 'epoch': 0.41}

 41%|████      | 15440/37962 [35:46:42<51:13:05,  8.19s/it]
 41%|████      | 15441/37962 [35:46:50<51:35:07,  8.25s/it]
 41%|████      | 15442/37962 [35:46:58<50:45:55,  8.12s/it]
 41%|████      | 15443/37962 [35:47:06<51:09:43,  8.18s/it]
 41%|████      | 15444/37962 [35:47:15<51:36:23,  8.25s/it]
 41%|████      | 15445/37962 [35:47:23<51:54:07,  8.30s/it]
 41%|████      | 15446/37962 [35:47:31<52:08:47,  8.34s/it]
 41%|████      | 15447/37962 [35:47:40<52:04:20,  8.33s/it]
 41%|████      | 15448/37962 [35:47:48<51:58:59,  8.31s/it]
 41%|████      | 15449/37962 [35:47:56<52:04:49,  8.33s/it]
 41%|████      | 15450/37962 [35:48:05<51:58:43,  8.31s/it]
                                                           
{'loss': 1.4243, 'grad_norm': 7.0573248863220215, 'learning_rate': 1.3427460328212143e-05, 'epoch': 0.41}

 41%|████      | 15450/37962 [35:48:05<51:58:43,  8.31s/it]
 41%|████      | 15451/37962 [35:48:13<51:56:22,  8.31s/it]
 41%|████      | 15452/37962 [35:48:21<52:15:19,  8.36s/it]
 41%|████      | 15453/37962 [35:48:30<52:19:50,  8.37s/it]
 41%|████      | 15454/37962 [35:48:38<51:25:32,  8.23s/it]
 41%|████      | 15455/37962 [35:48:46<51:21:57,  8.22s/it]
 41%|████      | 15456/37962 [35:48:54<51:13:20,  8.19s/it]
 41%|████      | 15457/37962 [35:49:02<51:18:42,  8.21s/it]
 41%|████      | 15458/37962 [35:49:11<51:31:07,  8.24s/it]
 41%|████      | 15459/37962 [35:49:19<51:31:34,  8.24s/it]
 41%|████      | 15460/37962 [35:49:27<51:52:47,  8.30s/it]
                                                           
{'loss': 1.4012, 'grad_norm': 19.362293243408203, 'learning_rate': 1.341944425295814e-05, 'epoch': 0.41}

 41%|████      | 15460/37962 [35:49:27<51:52:47,  8.30s/it]
 41%|████      | 15461/37962 [35:49:36<52:25:27,  8.39s/it]
 41%|████      | 15462/37962 [35:49:44<52:17:57,  8.37s/it]
 41%|████      | 15463/37962 [35:49:52<52:00:35,  8.32s/it]
 41%|████      | 15464/37962 [35:50:01<52:03:47,  8.33s/it]
 41%|████      | 15465/37962 [35:50:09<51:53:22,  8.30s/it]
 41%|████      | 15466/37962 [35:50:17<51:43:09,  8.28s/it]
 41%|████      | 15467/37962 [35:50:25<51:36:16,  8.26s/it]
 41%|████      | 15468/37962 [35:50:33<51:15:24,  8.20s/it]
 41%|████      | 15469/37962 [35:50:42<51:26:33,  8.23s/it]
 41%|████      | 15470/37962 [35:50:50<51:21:42,  8.22s/it]
                                                           
{'loss': 1.268, 'grad_norm': 6.649326324462891, 'learning_rate': 1.3411425688749901e-05, 'epoch': 0.41}

 41%|████      | 15470/37962 [35:50:50<51:21:42,  8.22s/it]
 41%|████      | 15471/37962 [35:50:58<50:45:50,  8.13s/it]
 41%|████      | 15472/37962 [35:51:06<50:43:17,  8.12s/it]
 41%|████      | 15473/37962 [35:51:14<50:44:12,  8.12s/it]
 41%|████      | 15474/37962 [35:51:22<50:26:21,  8.07s/it]
 41%|████      | 15475/37962 [35:51:30<50:57:09,  8.16s/it]
 41%|████      | 15476/37962 [35:51:39<51:11:30,  8.20s/it]
 41%|████      | 15477/37962 [35:51:47<50:50:11,  8.14s/it]
 41%|████      | 15478/37962 [35:51:55<51:02:13,  8.17s/it]
 41%|████      | 15479/37962 [35:52:03<51:40:11,  8.27s/it]
 41%|████      | 15480/37962 [35:52:12<51:23:57,  8.23s/it]
                                                           
{'loss': 1.3115, 'grad_norm': 8.10913372039795, 'learning_rate': 1.3403404641424007e-05, 'epoch': 0.41}

 41%|████      | 15480/37962 [35:52:12<51:23:57,  8.23s/it]
 41%|████      | 15481/37962 [35:52:20<51:32:32,  8.25s/it]
 41%|████      | 15482/37962 [35:52:28<51:22:37,  8.23s/it]
 41%|████      | 15483/37962 [35:52:36<51:42:41,  8.28s/it]
 41%|████      | 15484/37962 [35:52:45<51:22:09,  8.23s/it]
 41%|████      | 15485/37962 [35:52:53<51:03:17,  8.18s/it]
 41%|████      | 15486/37962 [35:53:01<50:55:23,  8.16s/it]
 41%|████      | 15487/37962 [35:53:09<51:09:22,  8.19s/it]
 41%|████      | 15488/37962 [35:53:17<51:08:14,  8.19s/it]
 41%|████      | 15489/37962 [35:53:25<51:17:29,  8.22s/it]
 41%|████      | 15490/37962 [35:53:34<51:16:50,  8.22s/it]
                                                           
{'loss': 1.3388, 'grad_norm': 5.259890556335449, 'learning_rate': 1.339538111681883e-05, 'epoch': 0.41}

 41%|████      | 15490/37962 [35:53:34<51:16:50,  8.22s/it]
 41%|████      | 15491/37962 [35:53:42<51:26:38,  8.24s/it]
 41%|████      | 15492/37962 [35:53:50<51:22:21,  8.23s/it]
 41%|████      | 15493/37962 [35:53:58<51:18:19,  8.22s/it]
 41%|████      | 15494/37962 [35:54:07<51:42:31,  8.29s/it]
 41%|████      | 15495/37962 [35:54:15<51:39:50,  8.28s/it]
 41%|████      | 15496/37962 [35:54:23<51:40:22,  8.28s/it]
 41%|████      | 15497/37962 [35:54:32<51:44:27,  8.29s/it]
 41%|████      | 15498/37962 [35:54:40<51:31:26,  8.26s/it]
 41%|████      | 15499/37962 [35:54:48<51:18:07,  8.22s/it]
 41%|████      | 15500/37962 [35:54:56<51:27:40,  8.25s/it]
                                                           
{'loss': 1.3575, 'grad_norm': 9.884650230407715, 'learning_rate': 1.338735512077456e-05, 'epoch': 0.41}

 41%|████      | 15500/37962 [35:54:56<51:27:40,  8.25s/it]
 41%|████      | 15501/37962 [35:55:05<51:35:41,  8.27s/it]
 41%|████      | 15502/37962 [35:55:13<51:54:25,  8.32s/it]
 41%|████      | 15503/37962 [35:55:21<51:09:50,  8.20s/it]
 41%|████      | 15504/37962 [35:55:30<51:46:00,  8.30s/it]
 41%|████      | 15505/37962 [35:55:38<51:15:26,  8.22s/it]
 41%|████      | 15506/37962 [35:55:46<51:14:47,  8.22s/it]
 41%|████      | 15507/37962 [35:55:54<51:21:12,  8.23s/it]
 41%|████      | 15508/37962 [35:56:02<51:43:41,  8.29s/it]
 41%|████      | 15509/37962 [35:56:11<51:41:28,  8.29s/it]
 41%|████      | 15510/37962 [35:56:19<51:44:00,  8.30s/it]
                                                           
{'loss': 1.3098, 'grad_norm': 9.518773078918457, 'learning_rate': 1.3379326659133181e-05, 'epoch': 0.41}

 41%|████      | 15510/37962 [35:56:19<51:44:00,  8.30s/it]
 41%|████      | 15511/37962 [35:56:27<51:49:08,  8.31s/it]
 41%|████      | 15512/37962 [35:56:36<51:46:23,  8.30s/it]
 41%|████      | 15513/37962 [35:56:44<51:14:18,  8.22s/it]
 41%|████      | 15514/37962 [35:56:52<51:06:51,  8.20s/it]
 41%|████      | 15515/37962 [35:57:00<51:15:03,  8.22s/it]
 41%|████      | 15516/37962 [35:57:08<50:44:14,  8.14s/it]
 41%|████      | 15517/37962 [35:57:17<51:38:44,  8.28s/it]
 41%|████      | 15518/37962 [35:57:25<51:41:17,  8.29s/it]
 41%|████      | 15519/37962 [35:57:33<51:53:42,  8.32s/it]
 41%|████      | 15520/37962 [35:57:41<51:13:07,  8.22s/it]
                                                           
{'loss': 1.3905, 'grad_norm': 9.161428451538086, 'learning_rate': 1.3371295737738468e-05, 'epoch': 0.41}

 41%|████      | 15520/37962 [35:57:41<51:13:07,  8.22s/it]
 41%|████      | 15521/37962 [35:57:50<51:25:33,  8.25s/it]
 41%|████      | 15522/37962 [35:57:58<51:22:10,  8.24s/it]
 41%|████      | 15523/37962 [35:58:06<51:37:55,  8.28s/it]
 41%|████      | 15524/37962 [35:58:15<51:32:04,  8.27s/it]
 41%|████      | 15525/37962 [35:58:23<51:48:11,  8.31s/it]
 41%|████      | 15526/37962 [35:58:31<52:12:18,  8.38s/it]
 41%|████      | 15527/37962 [35:58:40<52:07:04,  8.36s/it]
 41%|████      | 15528/37962 [35:58:48<51:51:39,  8.32s/it]
 41%|████      | 15529/37962 [35:58:56<51:21:51,  8.24s/it]
 41%|████      | 15530/37962 [35:59:05<51:41:54,  8.30s/it]
                                                           
{'loss': 1.4098, 'grad_norm': 12.229033470153809, 'learning_rate': 1.3363262362435992e-05, 'epoch': 0.41}

 41%|████      | 15530/37962 [35:59:05<51:41:54,  8.30s/it]
 41%|████      | 15531/37962 [35:59:13<51:29:16,  8.26s/it]
 41%|████      | 15532/37962 [35:59:21<51:07:27,  8.21s/it]
 41%|████      | 15533/37962 [35:59:29<50:52:03,  8.16s/it]
 41%|████      | 15534/37962 [35:59:37<51:15:41,  8.23s/it]
 41%|████      | 15535/37962 [35:59:45<51:06:50,  8.20s/it]
 41%|████      | 15536/37962 [35:59:54<51:14:52,  8.23s/it]
 41%|████      | 15537/37962 [36:00:02<51:01:35,  8.19s/it]
 41%|████      | 15538/37962 [36:00:10<51:26:48,  8.26s/it]
 41%|████      | 15539/37962 [36:00:19<51:44:40,  8.31s/it]
 41%|████      | 15540/37962 [36:00:27<51:36:22,  8.29s/it]
                                                           
{'loss': 1.4855, 'grad_norm': 11.670672416687012, 'learning_rate': 1.3355226539073104e-05, 'epoch': 0.41}

 41%|████      | 15540/37962 [36:00:27<51:36:22,  8.29s/it]
 41%|████      | 15541/37962 [36:00:35<51:22:52,  8.25s/it]
 41%|████      | 15542/37962 [36:00:43<51:39:07,  8.29s/it]
 41%|████      | 15543/37962 [36:00:51<51:14:42,  8.23s/it]
 41%|████      | 15544/37962 [36:01:00<51:28:06,  8.27s/it]
 41%|████      | 15545/37962 [36:01:08<51:27:14,  8.26s/it]
 41%|████      | 15546/37962 [36:01:16<51:34:43,  8.28s/it]
 41%|████      | 15547/37962 [36:01:25<51:30:31,  8.27s/it]
 41%|████      | 15548/37962 [36:01:33<51:04:24,  8.20s/it]
 41%|████      | 15549/37962 [36:01:41<50:26:17,  8.10s/it]
 41%|████      | 15550/37962 [36:01:49<50:50:07,  8.17s/it]
                                                           
{'loss': 1.3829, 'grad_norm': 3.850428819656372, 'learning_rate': 1.3347188273498948e-05, 'epoch': 0.41}

 41%|████      | 15550/37962 [36:01:49<50:50:07,  8.17s/it]
 41%|████      | 15551/37962 [36:01:57<51:07:55,  8.21s/it]
 41%|████      | 15552/37962 [36:02:06<51:26:51,  8.26s/it]
 41%|████      | 15553/37962 [36:02:14<51:23:16,  8.26s/it]
 41%|████      | 15554/37962 [36:02:22<51:29:34,  8.27s/it]
 41%|████      | 15555/37962 [36:02:31<51:41:45,  8.31s/it]
 41%|████      | 15556/37962 [36:02:39<51:58:57,  8.35s/it]
 41%|████      | 15557/37962 [36:02:47<51:22:48,  8.26s/it]
 41%|████      | 15558/37962 [36:02:55<51:18:34,  8.24s/it]
 41%|████      | 15559/37962 [36:03:04<51:31:02,  8.28s/it]
 41%|████      | 15560/37962 [36:03:12<51:31:54,  8.28s/it]
                                                           
{'loss': 1.3849, 'grad_norm': 9.473344802856445, 'learning_rate': 1.3339147571564432e-05, 'epoch': 0.41}

 41%|████      | 15560/37962 [36:03:12<51:31:54,  8.28s/it]
 41%|████      | 15561/37962 [36:03:20<51:32:16,  8.28s/it]
 41%|████      | 15562/37962 [36:03:28<51:32:21,  8.28s/it]
 41%|████      | 15563/37962 [36:03:37<51:40:38,  8.31s/it]
 41%|████      | 15564/37962 [36:03:45<51:21:47,  8.26s/it]
 41%|████      | 15565/37962 [36:03:53<51:07:20,  8.22s/it]
 41%|████      | 15566/37962 [36:04:01<51:13:46,  8.23s/it]
 41%|████      | 15567/37962 [36:04:10<51:05:56,  8.21s/it]
 41%|████      | 15568/37962 [36:04:18<51:19:37,  8.25s/it]
 41%|████      | 15569/37962 [36:04:26<50:51:41,  8.18s/it]
 41%|████      | 15570/37962 [36:04:34<51:10:10,  8.23s/it]
                                                           
{'loss': 1.3736, 'grad_norm': 1.9616492986679077, 'learning_rate': 1.3331104439122244e-05, 'epoch': 0.41}

 41%|████      | 15570/37962 [36:04:34<51:10:10,  8.23s/it]
 41%|████      | 15571/37962 [36:04:42<51:15:00,  8.24s/it]
 41%|████      | 15572/37962 [36:04:51<51:21:46,  8.26s/it]
 41%|████      | 15573/37962 [36:04:59<51:19:01,  8.25s/it]
 41%|████      | 15574/37962 [36:05:07<51:26:22,  8.27s/it]
 41%|████      | 15575/37962 [36:05:16<51:16:09,  8.24s/it]
 41%|████      | 15576/37962 [36:05:24<51:16:47,  8.25s/it]
 41%|████      | 15577/37962 [36:05:32<51:04:47,  8.21s/it]
 41%|████      | 15578/37962 [36:05:40<51:32:56,  8.29s/it]
 41%|████      | 15579/37962 [36:05:49<51:15:41,  8.24s/it]
 41%|████      | 15580/37962 [36:05:57<51:00:12,  8.20s/it]
                                                           
{'loss': 1.4675, 'grad_norm': 5.797630310058594, 'learning_rate': 1.3323058882026843e-05, 'epoch': 0.41}

 41%|████      | 15580/37962 [36:05:57<51:00:12,  8.20s/it]
 41%|████      | 15581/37962 [36:06:05<51:05:07,  8.22s/it]
 41%|████      | 15582/37962 [36:06:13<51:16:44,  8.25s/it]
 41%|████      | 15583/37962 [36:06:22<51:39:20,  8.31s/it]
 41%|████      | 15584/37962 [36:06:37<65:01:15, 10.46s/it]
 41%|████      | 15585/37962 [36:06:46<61:09:45,  9.84s/it]
 41%|████      | 15586/37962 [36:06:54<58:31:18,  9.42s/it]
 41%|████      | 15587/37962 [36:07:02<55:38:59,  8.95s/it]
 41%|████      | 15588/37962 [36:07:10<54:33:57,  8.78s/it]
 41%|████      | 15589/37962 [36:07:18<53:39:01,  8.63s/it]
 41%|████      | 15590/37962 [36:07:27<53:01:16,  8.53s/it]
                                                           
{'loss': 1.2048, 'grad_norm': 15.889266967773438, 'learning_rate': 1.3315010906134452e-05, 'epoch': 0.41}

 41%|████      | 15590/37962 [36:07:27<53:01:16,  8.53s/it]
 41%|████      | 15591/37962 [36:07:35<52:31:19,  8.45s/it]
 41%|████      | 15592/37962 [36:07:43<51:46:30,  8.33s/it]
 41%|████      | 15593/37962 [36:07:51<51:43:20,  8.32s/it]
 41%|████      | 15594/37962 [36:08:00<51:47:47,  8.34s/it]
 41%|████      | 15595/37962 [36:08:08<51:26:49,  8.28s/it]
 41%|████      | 15596/37962 [36:08:16<50:59:31,  8.21s/it]
 41%|████      | 15597/37962 [36:08:24<51:30:24,  8.29s/it]
 41%|████      | 15598/37962 [36:08:33<51:27:37,  8.28s/it]
 41%|████      | 15599/37962 [36:08:41<51:23:06,  8.27s/it]
 41%|████      | 15600/37962 [36:08:49<51:29:01,  8.29s/it]
                                                           
{'loss': 1.4179, 'grad_norm': 11.765287399291992, 'learning_rate': 1.3306960517303049e-05, 'epoch': 0.41}

 41%|████      | 15600/37962 [36:08:49<51:29:01,  8.29s/it]
 41%|████      | 15601/37962 [36:08:58<51:36:27,  8.31s/it]
 41%|████      | 15602/37962 [36:09:06<51:28:02,  8.29s/it]
 41%|████      | 15603/37962 [36:09:14<51:58:15,  8.37s/it]
 41%|████      | 15604/37962 [36:09:23<51:47:53,  8.34s/it]
 41%|████      | 15605/37962 [36:09:31<51:46:23,  8.34s/it]
 41%|████      | 15606/37962 [36:09:39<51:39:20,  8.32s/it]
 41%|████      | 15607/37962 [36:09:47<50:50:32,  8.19s/it]
 41%|████      | 15608/37962 [36:09:55<50:46:26,  8.18s/it]
 41%|████      | 15609/37962 [36:10:03<50:14:43,  8.09s/it]
 41%|████      | 15610/37962 [36:10:12<50:37:18,  8.15s/it]
                                                           
{'loss': 1.4878, 'grad_norm': 4.846033573150635, 'learning_rate': 1.3298907721392374e-05, 'epoch': 0.41}

 41%|████      | 15610/37962 [36:10:12<50:37:18,  8.15s/it]
 41%|████      | 15611/37962 [36:10:20<51:06:15,  8.23s/it]
 41%|████      | 15612/37962 [36:10:28<51:16:43,  8.26s/it]
 41%|████      | 15613/37962 [36:10:37<51:23:22,  8.28s/it]
 41%|████      | 15614/37962 [36:10:45<51:59:21,  8.37s/it]
 41%|████      | 15615/37962 [36:10:53<51:41:57,  8.33s/it]
 41%|████      | 15616/37962 [36:11:02<51:40:29,  8.32s/it]
 41%|████      | 15617/37962 [36:11:10<51:42:06,  8.33s/it]
 41%|████      | 15618/37962 [36:11:19<52:02:51,  8.39s/it]
 41%|████      | 15619/37962 [36:11:27<52:10:22,  8.41s/it]
 41%|████      | 15620/37962 [36:11:35<52:08:10,  8.40s/it]
                                                           
{'loss': 1.3669, 'grad_norm': 6.000058650970459, 'learning_rate': 1.329085252426392e-05, 'epoch': 0.41}

 41%|████      | 15620/37962 [36:11:35<52:08:10,  8.40s/it]
 41%|████      | 15621/37962 [36:11:44<51:58:58,  8.38s/it]
 41%|████      | 15622/37962 [36:11:52<51:27:11,  8.29s/it]
 41%|████      | 15623/37962 [36:12:00<51:30:37,  8.30s/it]
 41%|████      | 15624/37962 [36:12:09<51:50:55,  8.36s/it]
 41%|████      | 15625/37962 [36:12:17<51:56:18,  8.37s/it]
 41%|████      | 15626/37962 [36:12:25<51:05:21,  8.23s/it]
 41%|████      | 15627/37962 [36:12:33<50:48:21,  8.19s/it]
 41%|████      | 15628/37962 [36:12:41<50:59:45,  8.22s/it]
 41%|████      | 15629/37962 [36:12:50<51:08:39,  8.24s/it]
 41%|████      | 15630/37962 [36:12:58<51:31:17,  8.31s/it]
                                                           
{'loss': 1.3584, 'grad_norm': 6.277578353881836, 'learning_rate': 1.3282794931780924e-05, 'epoch': 0.41}

 41%|████      | 15630/37962 [36:12:58<51:31:17,  8.31s/it]
 41%|████      | 15631/37962 [36:13:06<51:31:50,  8.31s/it]
 41%|████      | 15632/37962 [36:13:15<51:26:27,  8.29s/it]
 41%|████      | 15633/37962 [36:13:22<50:29:29,  8.14s/it]
 41%|████      | 15634/37962 [36:13:31<50:23:36,  8.13s/it]
 41%|████      | 15635/37962 [36:13:39<50:50:19,  8.20s/it]
 41%|████      | 15636/37962 [36:13:47<51:07:53,  8.24s/it]
 41%|████      | 15637/37962 [36:14:03<65:30:17, 10.56s/it]
 41%|████      | 15638/37962 [36:14:12<61:29:57,  9.92s/it]
 41%|████      | 15639/37962 [36:14:20<58:04:50,  9.37s/it]
 41%|████      | 15640/37962 [36:14:28<56:01:23,  9.04s/it]
                                                           
{'loss': 1.3247, 'grad_norm': 3.944087505340576, 'learning_rate': 1.3274734949808366e-05, 'epoch': 0.41}

 41%|████      | 15640/37962 [36:14:28<56:01:23,  9.04s/it]
 41%|████      | 15641/37962 [36:14:36<54:45:29,  8.83s/it]
 41%|████      | 15642/37962 [36:14:45<53:59:43,  8.71s/it]
 41%|████      | 15643/37962 [36:14:53<53:01:25,  8.55s/it]
 41%|████      | 15644/37962 [36:15:01<51:50:41,  8.36s/it]
 41%|████      | 15645/37962 [36:15:09<51:17:45,  8.27s/it]
 41%|████      | 15646/37962 [36:15:17<51:29:19,  8.31s/it]
 41%|████      | 15647/37962 [36:15:26<51:29:57,  8.31s/it]
 41%|████      | 15648/37962 [36:15:34<51:41:35,  8.34s/it]
 41%|████      | 15649/37962 [36:15:42<51:39:02,  8.33s/it]
 41%|████      | 15650/37962 [36:15:51<51:58:58,  8.39s/it]
                                                           
{'loss': 1.4622, 'grad_norm': 12.023463249206543, 'learning_rate': 1.326667258421297e-05, 'epoch': 0.41}

 41%|████      | 15650/37962 [36:15:51<51:58:58,  8.39s/it]
 41%|████      | 15651/37962 [36:15:59<51:39:37,  8.34s/it]
 41%|████      | 15652/37962 [36:16:07<51:28:52,  8.31s/it]
 41%|████      | 15653/37962 [36:16:16<51:25:40,  8.30s/it]
 41%|████      | 15654/37962 [36:16:24<51:31:08,  8.31s/it]
 41%|████      | 15655/37962 [36:16:32<51:45:17,  8.35s/it]
 41%|████      | 15656/37962 [36:16:41<51:48:59,  8.36s/it]
 41%|████      | 15657/37962 [36:16:49<51:40:19,  8.34s/it]
 41%|████      | 15658/37962 [36:16:57<51:44:12,  8.35s/it]
 41%|████      | 15659/37962 [36:17:05<50:43:32,  8.19s/it]
 41%|████▏     | 15660/37962 [36:17:14<50:52:04,  8.21s/it]
                                                           
{'loss': 1.4178, 'grad_norm': 5.0492472648620605, 'learning_rate': 1.325860784086319e-05, 'epoch': 0.41}

 41%|████▏     | 15660/37962 [36:17:14<50:52:04,  8.21s/it]
 41%|████▏     | 15661/37962 [36:17:22<50:51:34,  8.21s/it]
 41%|████▏     | 15662/37962 [36:17:30<51:15:26,  8.27s/it]
 41%|████▏     | 15663/37962 [36:17:39<51:42:56,  8.35s/it]
 41%|████▏     | 15664/37962 [36:17:47<51:37:39,  8.34s/it]
 41%|████▏     | 15665/37962 [36:17:55<51:25:11,  8.30s/it]
 41%|████▏     | 15666/37962 [36:18:04<51:25:46,  8.30s/it]
 41%|████▏     | 15667/37962 [36:18:12<51:24:44,  8.30s/it]
 41%|████▏     | 15668/37962 [36:18:20<51:18:58,  8.29s/it]
 41%|████▏     | 15669/37962 [36:18:28<51:09:55,  8.26s/it]
 41%|████▏     | 15670/37962 [36:18:37<51:36:27,  8.33s/it]
                                                           
{'loss': 1.3226, 'grad_norm': 8.90405559539795, 'learning_rate': 1.3250540725629217e-05, 'epoch': 0.41}

 41%|████▏     | 15670/37962 [36:18:37<51:36:27,  8.33s/it]
 41%|████▏     | 15671/37962 [36:18:45<52:02:56,  8.41s/it]
 41%|████▏     | 15672/37962 [36:18:54<51:36:56,  8.34s/it]
 41%|████▏     | 15673/37962 [36:19:02<51:25:24,  8.31s/it]
 41%|████▏     | 15674/37962 [36:19:10<51:19:32,  8.29s/it]
 41%|████▏     | 15675/37962 [36:19:18<51:37:23,  8.34s/it]
 41%|████▏     | 15676/37962 [36:19:27<51:14:00,  8.28s/it]
 41%|████▏     | 15677/37962 [36:19:35<51:09:38,  8.26s/it]
 41%|████▏     | 15678/37962 [36:19:43<50:44:37,  8.20s/it]
 41%|████▏     | 15679/37962 [36:19:51<51:05:51,  8.26s/it]
 41%|████▏     | 15680/37962 [36:20:00<51:14:41,  8.28s/it]
                                                           
{'loss': 1.3729, 'grad_norm': 10.668108940124512, 'learning_rate': 1.3242471244382957e-05, 'epoch': 0.41}

 41%|████▏     | 15680/37962 [36:20:00<51:14:41,  8.28s/it]
 41%|████▏     | 15681/37962 [36:20:08<51:22:53,  8.30s/it]
 41%|████▏     | 15682/37962 [36:20:16<51:26:36,  8.31s/it]
 41%|████▏     | 15683/37962 [36:20:24<50:39:18,  8.19s/it]
 41%|████▏     | 15684/37962 [36:20:33<51:01:20,  8.24s/it]
 41%|████▏     | 15685/37962 [36:20:41<50:40:08,  8.19s/it]
 41%|████▏     | 15686/37962 [36:20:49<50:29:03,  8.16s/it]
 41%|████▏     | 15687/37962 [36:20:57<50:47:32,  8.21s/it]
 41%|████▏     | 15688/37962 [36:21:06<51:23:36,  8.31s/it]
 41%|████▏     | 15689/37962 [36:21:14<51:18:37,  8.29s/it]
 41%|████▏     | 15690/37962 [36:21:22<51:06:43,  8.26s/it]
                                                           
{'loss': 1.4294, 'grad_norm': 6.2647881507873535, 'learning_rate': 1.3234399402998053e-05, 'epoch': 0.41}

 41%|████▏     | 15690/37962 [36:21:22<51:06:43,  8.26s/it]
 41%|████▏     | 15691/37962 [36:21:30<50:43:46,  8.20s/it]
 41%|████▏     | 15692/37962 [36:21:38<50:46:02,  8.21s/it]
 41%|████▏     | 15693/37962 [36:21:46<50:46:28,  8.21s/it]
 41%|████▏     | 15694/37962 [36:21:55<50:37:43,  8.18s/it]
 41%|████▏     | 15695/37962 [36:22:03<50:50:34,  8.22s/it]
 41%|████▏     | 15696/37962 [36:22:11<50:50:10,  8.22s/it]
 41%|████▏     | 15697/37962 [36:22:19<50:55:50,  8.23s/it]
 41%|████▏     | 15698/37962 [36:22:28<50:54:21,  8.23s/it]
 41%|████▏     | 15699/37962 [36:22:36<50:21:26,  8.14s/it]
 41%|████▏     | 15700/37962 [36:22:44<50:43:48,  8.20s/it]
                                                           
{'loss': 1.4017, 'grad_norm': 4.72715950012207, 'learning_rate': 1.3226325207349855e-05, 'epoch': 0.41}

 41%|████▏     | 15700/37962 [36:22:44<50:43:48,  8.20s/it]
 41%|████▏     | 15701/37962 [36:22:52<51:18:19,  8.30s/it]
 41%|████▏     | 15702/37962 [36:23:01<51:12:06,  8.28s/it]
 41%|████▏     | 15703/37962 [36:23:09<50:47:44,  8.22s/it]
 41%|████▏     | 15704/37962 [36:23:17<51:00:40,  8.25s/it]
 41%|████▏     | 15705/37962 [36:23:26<51:21:11,  8.31s/it]
 41%|████▏     | 15706/37962 [36:23:34<52:00:02,  8.41s/it]
 41%|████▏     | 15707/37962 [36:23:43<52:13:44,  8.45s/it]
 41%|████▏     | 15708/37962 [36:23:51<51:55:24,  8.40s/it]
 41%|████▏     | 15709/37962 [36:23:59<52:02:40,  8.42s/it]
 41%|████▏     | 15710/37962 [36:24:08<52:12:03,  8.45s/it]
                                                           
{'loss': 1.2765, 'grad_norm': 13.50994873046875, 'learning_rate': 1.3218248663315429e-05, 'epoch': 0.41}

 41%|████▏     | 15710/37962 [36:24:08<52:12:03,  8.45s/it]
 41%|████▏     | 15711/37962 [36:24:16<51:45:28,  8.37s/it]
 41%|████▏     | 15712/37962 [36:24:24<51:09:12,  8.28s/it]
 41%|████▏     | 15713/37962 [36:24:32<50:49:04,  8.22s/it]
 41%|████▏     | 15714/37962 [36:24:41<51:03:19,  8.26s/it]
 41%|████▏     | 15715/37962 [36:24:49<51:03:15,  8.26s/it]
 41%|████▏     | 15716/37962 [36:24:57<51:09:38,  8.28s/it]
 41%|████▏     | 15717/37962 [36:25:06<51:21:00,  8.31s/it]
 41%|████▏     | 15718/37962 [36:25:14<50:58:09,  8.25s/it]
 41%|████▏     | 15719/37962 [36:25:22<50:58:09,  8.25s/it]
 41%|████▏     | 15720/37962 [36:25:30<51:08:07,  8.28s/it]
                                                           
{'loss': 1.4294, 'grad_norm': 5.650006294250488, 'learning_rate': 1.3210169776773558e-05, 'epoch': 0.41}

 41%|████▏     | 15720/37962 [36:25:30<51:08:07,  8.28s/it]
 41%|████▏     | 15721/37962 [36:25:39<51:14:39,  8.29s/it]
 41%|████▏     | 15722/37962 [36:25:47<51:19:28,  8.31s/it]
 41%|████▏     | 15723/37962 [36:25:55<51:07:55,  8.28s/it]
 41%|████▏     | 15724/37962 [36:26:04<51:12:43,  8.29s/it]
 41%|████▏     | 15725/37962 [36:26:12<51:07:03,  8.28s/it]
 41%|████▏     | 15726/37962 [36:26:20<51:15:59,  8.30s/it]
 41%|████▏     | 15727/37962 [36:26:29<51:32:21,  8.34s/it]
 41%|████▏     | 15728/37962 [36:26:37<51:22:18,  8.32s/it]
 41%|████▏     | 15729/37962 [36:26:45<51:14:51,  8.30s/it]
 41%|████▏     | 15730/37962 [36:26:53<51:27:35,  8.33s/it]
                                                           
{'loss': 1.4633, 'grad_norm': 8.880769729614258, 'learning_rate': 1.3202088553604718e-05, 'epoch': 0.41}

 41%|████▏     | 15730/37962 [36:26:54<51:27:35,  8.33s/it]
 41%|████▏     | 15731/37962 [36:27:01<50:44:36,  8.22s/it]
 41%|████▏     | 15732/37962 [36:27:10<50:46:46,  8.22s/it]
 41%|████▏     | 15733/37962 [36:27:18<50:50:26,  8.23s/it]
 41%|████▏     | 15734/37962 [36:27:26<51:02:05,  8.27s/it]
 41%|████▏     | 15735/37962 [36:27:35<51:17:43,  8.31s/it]
 41%|████▏     | 15736/37962 [36:27:43<51:37:07,  8.36s/it]
 41%|████▏     | 15737/37962 [36:27:51<51:26:16,  8.33s/it]
 41%|████▏     | 15738/37962 [36:28:00<50:57:34,  8.25s/it]
 41%|████▏     | 15739/37962 [36:28:08<51:00:30,  8.26s/it]
 41%|████▏     | 15740/37962 [36:28:16<50:54:58,  8.25s/it]
                                                           
{'loss': 1.408, 'grad_norm': 3.0964739322662354, 'learning_rate': 1.3194004999691089e-05, 'epoch': 0.41}

 41%|████▏     | 15740/37962 [36:28:16<50:54:58,  8.25s/it]
 41%|████▏     | 15741/37962 [36:28:24<50:42:16,  8.21s/it]
 41%|████▏     | 15742/37962 [36:28:32<50:53:43,  8.25s/it]
 41%|████▏     | 15743/37962 [36:28:41<51:13:21,  8.30s/it]
 41%|████▏     | 15744/37962 [36:28:49<51:29:41,  8.34s/it]
 41%|████▏     | 15745/37962 [36:28:58<51:22:09,  8.32s/it]
 41%|████▏     | 15746/37962 [36:29:06<51:02:16,  8.27s/it]
 41%|████▏     | 15747/37962 [36:29:14<51:01:48,  8.27s/it]
 41%|████▏     | 15748/37962 [36:29:22<51:10:28,  8.29s/it]
 41%|████▏     | 15749/37962 [36:29:31<50:53:14,  8.25s/it]
 41%|████▏     | 15750/37962 [36:29:39<50:58:26,  8.26s/it]
                                                           
{'loss': 1.3881, 'grad_norm': 30.252710342407227, 'learning_rate': 1.3185919120916554e-05, 'epoch': 0.41}

 41%|████▏     | 15750/37962 [36:29:39<50:58:26,  8.26s/it]
 41%|████▏     | 15751/37962 [36:29:47<51:22:35,  8.33s/it]
 41%|████▏     | 15752/37962 [36:29:56<51:13:08,  8.30s/it]
 41%|████▏     | 15753/37962 [36:30:03<50:22:26,  8.17s/it]
 41%|████▏     | 15754/37962 [36:30:12<50:28:10,  8.18s/it]
 42%|████▏     | 15755/37962 [36:30:20<50:22:49,  8.17s/it]
 42%|████▏     | 15756/37962 [36:30:28<50:39:34,  8.21s/it]
 42%|████▏     | 15757/37962 [36:30:36<50:18:53,  8.16s/it]
 42%|████▏     | 15758/37962 [36:30:44<50:44:28,  8.23s/it]
 42%|████▏     | 15759/37962 [36:30:53<50:28:24,  8.18s/it]
 42%|████▏     | 15760/37962 [36:31:01<50:24:21,  8.17s/it]
                                                           
{'loss': 1.4627, 'grad_norm': 16.142927169799805, 'learning_rate': 1.3177830923166684e-05, 'epoch': 0.42}

 42%|████▏     | 15760/37962 [36:31:01<50:24:21,  8.17s/it]
 42%|████▏     | 15761/37962 [36:31:09<50:02:40,  8.11s/it]
 42%|████▏     | 15762/37962 [36:31:17<50:12:21,  8.14s/it]
 42%|████▏     | 15763/37962 [36:31:25<50:06:18,  8.13s/it]
 42%|████▏     | 15764/37962 [36:31:33<50:14:08,  8.15s/it]
 42%|████▏     | 15765/37962 [36:31:41<50:29:47,  8.19s/it]
 42%|████▏     | 15766/37962 [36:31:50<50:26:39,  8.18s/it]
 42%|████▏     | 15767/37962 [36:31:58<50:04:46,  8.12s/it]
 42%|████▏     | 15768/37962 [36:32:06<50:25:45,  8.18s/it]
 42%|████▏     | 15769/37962 [36:32:14<50:18:56,  8.16s/it]
 42%|████▏     | 15770/37962 [36:32:23<51:10:19,  8.30s/it]
                                                           
{'loss': 1.4821, 'grad_norm': 4.905526638031006, 'learning_rate': 1.3169740412328737e-05, 'epoch': 0.42}

 42%|████▏     | 15770/37962 [36:32:23<51:10:19,  8.30s/it]
 42%|████▏     | 15771/37962 [36:32:31<51:16:19,  8.32s/it]
 42%|████▏     | 15772/37962 [36:32:39<51:14:50,  8.31s/it]
 42%|████▏     | 15773/37962 [36:32:48<51:22:49,  8.34s/it]
 42%|████▏     | 15774/37962 [36:32:56<51:17:40,  8.32s/it]
 42%|████▏     | 15775/37962 [36:33:04<51:00:31,  8.28s/it]
 42%|████▏     | 15776/37962 [36:33:13<51:16:09,  8.32s/it]
 42%|████▏     | 15777/37962 [36:33:21<51:05:36,  8.29s/it]
 42%|████▏     | 15778/37962 [36:33:29<50:22:38,  8.18s/it]
 42%|████▏     | 15779/37962 [36:33:37<50:47:35,  8.24s/it]
 42%|████▏     | 15780/37962 [36:33:45<50:51:17,  8.25s/it]
                                                           
{'loss': 1.3643, 'grad_norm': 4.244324207305908, 'learning_rate': 1.3161647594291656e-05, 'epoch': 0.42}

 42%|████▏     | 15780/37962 [36:33:45<50:51:17,  8.25s/it]
 42%|████▏     | 15781/37962 [36:33:54<50:39:56,  8.22s/it]
 42%|████▏     | 15782/37962 [36:34:02<50:37:44,  8.22s/it]
 42%|████▏     | 15783/37962 [36:34:10<50:41:38,  8.23s/it]
 42%|████▏     | 15784/37962 [36:34:18<50:29:12,  8.20s/it]
 42%|████▏     | 15785/37962 [36:34:27<50:58:26,  8.27s/it]
 42%|████▏     | 15786/37962 [36:34:35<50:52:28,  8.26s/it]
 42%|████▏     | 15787/37962 [36:34:43<51:02:32,  8.29s/it]
 42%|████▏     | 15788/37962 [36:34:51<50:35:27,  8.21s/it]
 42%|████▏     | 15789/37962 [36:35:00<50:47:19,  8.25s/it]
 42%|████▏     | 15790/37962 [36:35:08<50:26:55,  8.19s/it]
                                                           
{'loss': 1.4528, 'grad_norm': 7.677330017089844, 'learning_rate': 1.315355247494606e-05, 'epoch': 0.42}

 42%|████▏     | 15790/37962 [36:35:08<50:26:55,  8.19s/it]
 42%|████▏     | 15791/37962 [36:35:16<50:39:53,  8.23s/it]
 42%|████▏     | 15792/37962 [36:35:24<50:38:00,  8.22s/it]
 42%|████▏     | 15793/37962 [36:35:32<50:46:30,  8.25s/it]
 42%|████▏     | 15794/37962 [36:35:41<50:50:57,  8.26s/it]
 42%|████▏     | 15795/37962 [36:35:49<50:24:01,  8.19s/it]
 42%|████▏     | 15796/37962 [36:35:57<50:33:51,  8.21s/it]
 42%|████▏     | 15797/37962 [36:36:05<50:42:15,  8.24s/it]
 42%|████▏     | 15798/37962 [36:36:13<50:36:33,  8.22s/it]
 42%|████▏     | 15799/37962 [36:36:21<50:00:09,  8.12s/it]
 42%|████▏     | 15800/37962 [36:36:30<50:09:46,  8.15s/it]
                                                           
{'loss': 1.4125, 'grad_norm': 4.299878120422363, 'learning_rate': 1.3145455060184256e-05, 'epoch': 0.42}

 42%|████▏     | 15800/37962 [36:36:30<50:09:46,  8.15s/it]
 42%|████▏     | 15801/37962 [36:36:38<50:44:32,  8.24s/it]
 42%|████▏     | 15802/37962 [36:36:46<50:46:32,  8.25s/it]
 42%|████▏     | 15803/37962 [36:36:54<50:40:51,  8.23s/it]
 42%|████▏     | 15804/37962 [36:37:03<50:20:25,  8.18s/it]
 42%|████▏     | 15805/37962 [36:37:11<50:33:18,  8.21s/it]
 42%|████▏     | 15806/37962 [36:37:19<50:37:27,  8.23s/it]
 42%|████▏     | 15807/37962 [36:37:28<50:59:15,  8.29s/it]
 42%|████▏     | 15808/37962 [36:37:36<50:31:36,  8.21s/it]
 42%|████▏     | 15809/37962 [36:37:44<50:52:34,  8.27s/it]
 42%|████▏     | 15810/37962 [36:37:52<50:55:55,  8.28s/it]
                                                           
{'loss': 1.3351, 'grad_norm': 5.14202356338501, 'learning_rate': 1.31373553559002e-05, 'epoch': 0.42}

 42%|████▏     | 15810/37962 [36:37:52<50:55:55,  8.28s/it]
 42%|████▏     | 15811/37962 [36:38:01<50:56:13,  8.28s/it]
 42%|████▏     | 15812/37962 [36:38:09<51:12:14,  8.32s/it]
 42%|████▏     | 15813/37962 [36:38:17<50:32:03,  8.21s/it]
 42%|████▏     | 15814/37962 [36:38:25<50:42:09,  8.24s/it]
 42%|████▏     | 15815/37962 [36:38:33<50:33:49,  8.22s/it]
 42%|████▏     | 15816/37962 [36:38:42<50:21:47,  8.19s/it]
 42%|████▏     | 15817/37962 [36:38:50<51:02:54,  8.30s/it]
 42%|████▏     | 15818/37962 [36:38:58<50:57:47,  8.29s/it]
 42%|████▏     | 15819/37962 [36:39:07<51:14:17,  8.33s/it]
 42%|████▏     | 15820/37962 [36:39:15<50:43:18,  8.25s/it]
                                                           
{'loss': 1.4487, 'grad_norm': 13.069316864013672, 'learning_rate': 1.312925336798953e-05, 'epoch': 0.42}

 42%|████▏     | 15820/37962 [36:39:15<50:43:18,  8.25s/it]
 42%|████▏     | 15821/37962 [36:39:23<50:54:39,  8.28s/it]
 42%|████▏     | 15822/37962 [36:39:32<51:18:16,  8.34s/it]
 42%|████▏     | 15823/37962 [36:39:40<51:26:55,  8.37s/it]
 42%|████▏     | 15824/37962 [36:39:48<51:05:05,  8.31s/it]
 42%|████▏     | 15825/37962 [36:39:56<50:53:47,  8.28s/it]
 42%|████▏     | 15826/37962 [36:40:04<50:04:24,  8.14s/it]
 42%|████▏     | 15827/37962 [36:40:13<50:19:27,  8.18s/it]
 42%|████▏     | 15828/37962 [36:40:21<50:52:49,  8.28s/it]
 42%|████▏     | 15829/37962 [36:40:36<63:56:12, 10.40s/it]
 42%|████▏     | 15830/37962 [36:40:44<59:37:24,  9.70s/it]
                                                           
{'loss': 1.4011, 'grad_norm': 5.252227783203125, 'learning_rate': 1.3121149102349549e-05, 'epoch': 0.42}

 42%|████▏     | 15830/37962 [36:40:44<59:37:24,  9.70s/it]
 42%|████▏     | 15831/37962 [36:40:53<57:12:26,  9.31s/it]
 42%|████▏     | 15832/37962 [36:41:01<55:05:46,  8.96s/it]
 42%|████▏     | 15833/37962 [36:41:09<53:02:15,  8.63s/it]
 42%|████▏     | 15834/37962 [36:41:24<65:33:32, 10.67s/it]
 42%|████▏     | 15835/37962 [36:41:32<60:59:51,  9.92s/it]
 42%|████▏     | 15836/37962 [36:41:41<57:54:57,  9.42s/it]
 42%|████▏     | 15837/37962 [36:41:49<55:42:55,  9.07s/it]
 42%|████▏     | 15838/37962 [36:41:57<54:42:01,  8.90s/it]
 42%|████▏     | 15839/37962 [36:42:06<53:59:29,  8.79s/it]
 42%|████▏     | 15840/37962 [36:42:14<52:28:56,  8.54s/it]
                                                           
{'loss': 1.4049, 'grad_norm': 12.533105850219727, 'learning_rate': 1.3113042564879201e-05, 'epoch': 0.42}

 42%|████▏     | 15840/37962 [36:42:14<52:28:56,  8.54s/it]
 42%|████▏     | 15841/37962 [36:42:22<51:36:40,  8.40s/it]
 42%|████▏     | 15842/37962 [36:42:30<51:38:00,  8.40s/it]
 42%|████▏     | 15843/37962 [36:42:39<51:39:02,  8.41s/it]
 42%|████▏     | 15844/37962 [36:42:47<51:18:07,  8.35s/it]
 42%|████▏     | 15845/37962 [36:42:55<51:07:04,  8.32s/it]
 42%|████▏     | 15846/37962 [36:43:04<50:54:52,  8.29s/it]
 42%|████▏     | 15847/37962 [36:43:12<50:32:02,  8.23s/it]
 42%|████▏     | 15848/37962 [36:43:20<50:36:45,  8.24s/it]
 42%|████▏     | 15849/37962 [36:43:28<49:32:59,  8.07s/it]
 42%|████▏     | 15850/37962 [36:43:35<49:18:25,  8.03s/it]
                                                           
{'loss': 1.2646, 'grad_norm': 9.362533569335938, 'learning_rate': 1.3104933761479103e-05, 'epoch': 0.42}

 42%|████▏     | 15850/37962 [36:43:35<49:18:25,  8.03s/it]
 42%|████▏     | 15851/37962 [36:43:44<49:39:20,  8.08s/it]
 42%|████▏     | 15852/37962 [36:43:52<49:50:12,  8.11s/it]
 42%|████▏     | 15853/37962 [36:44:00<50:21:06,  8.20s/it]
 42%|████▏     | 15854/37962 [36:44:08<50:08:51,  8.17s/it]
 42%|████▏     | 15855/37962 [36:44:17<50:31:35,  8.23s/it]
 42%|████▏     | 15856/37962 [36:44:25<51:01:49,  8.31s/it]
 42%|████▏     | 15857/37962 [36:44:34<51:06:40,  8.32s/it]
 42%|████▏     | 15858/37962 [36:44:42<50:54:36,  8.29s/it]
 42%|████▏     | 15859/37962 [36:44:50<50:57:52,  8.30s/it]
 42%|████▏     | 15860/37962 [36:44:58<50:55:32,  8.29s/it]
                                                           
{'loss': 1.4254, 'grad_norm': 6.303677558898926, 'learning_rate': 1.309682269805151e-05, 'epoch': 0.42}

 42%|████▏     | 15860/37962 [36:44:58<50:55:32,  8.29s/it]
 42%|████▏     | 15861/37962 [36:45:07<50:35:48,  8.24s/it]
 42%|████▏     | 15862/37962 [36:45:15<50:40:51,  8.26s/it]
 42%|████▏     | 15863/37962 [36:45:23<50:35:54,  8.24s/it]
 42%|████▏     | 15864/37962 [36:45:31<50:49:50,  8.28s/it]
 42%|████▏     | 15865/37962 [36:45:39<50:26:08,  8.22s/it]
 42%|████▏     | 15866/37962 [36:45:48<50:25:52,  8.22s/it]
 42%|████▏     | 15867/37962 [36:45:56<50:22:30,  8.21s/it]
 42%|████▏     | 15868/37962 [36:46:04<49:52:52,  8.13s/it]
 42%|████▏     | 15869/37962 [36:46:12<50:16:50,  8.19s/it]
 42%|████▏     | 15870/37962 [36:46:20<50:07:22,  8.17s/it]
                                                           
{'loss': 1.3672, 'grad_norm': 12.081802368164062, 'learning_rate': 1.3088709380500322e-05, 'epoch': 0.42}

 42%|████▏     | 15870/37962 [36:46:20<50:07:22,  8.17s/it]
 42%|████▏     | 15871/37962 [36:46:28<49:55:05,  8.13s/it]
 42%|████▏     | 15872/37962 [36:46:37<50:19:18,  8.20s/it]
 42%|████▏     | 15873/37962 [36:46:45<50:17:40,  8.20s/it]
 42%|████▏     | 15874/37962 [36:46:53<50:22:17,  8.21s/it]
 42%|████▏     | 15875/37962 [36:47:01<50:26:26,  8.22s/it]
 42%|████▏     | 15876/37962 [36:47:10<50:40:07,  8.26s/it]
 42%|████▏     | 15877/37962 [36:47:18<50:50:17,  8.29s/it]
 42%|████▏     | 15878/37962 [36:47:26<50:53:19,  8.30s/it]
 42%|████▏     | 15879/37962 [36:47:35<51:06:00,  8.33s/it]
 42%|████▏     | 15880/37962 [36:47:43<51:03:54,  8.33s/it]
                                                           
{'loss': 1.3946, 'grad_norm': 6.973888397216797, 'learning_rate': 1.3080593814731084e-05, 'epoch': 0.42}

 42%|████▏     | 15880/37962 [36:47:43<51:03:54,  8.33s/it]
 42%|████▏     | 15881/37962 [36:47:51<50:51:44,  8.29s/it]
 42%|████▏     | 15882/37962 [36:48:00<50:46:09,  8.28s/it]
 42%|████▏     | 15883/37962 [36:48:08<51:13:57,  8.35s/it]
 42%|████▏     | 15884/37962 [36:48:16<51:00:40,  8.32s/it]
 42%|████▏     | 15885/37962 [36:48:24<50:18:52,  8.20s/it]
 42%|████▏     | 15886/37962 [36:48:33<50:24:45,  8.22s/it]
 42%|████▏     | 15887/37962 [36:48:40<49:40:38,  8.10s/it]
 42%|████▏     | 15888/37962 [36:48:49<49:52:53,  8.14s/it]
 42%|████▏     | 15889/37962 [36:48:57<50:22:32,  8.22s/it]
 42%|████▏     | 15890/37962 [36:49:05<50:29:41,  8.24s/it]
                                                           
{'loss': 1.4299, 'grad_norm': 11.671911239624023, 'learning_rate': 1.3072476006650976e-05, 'epoch': 0.42}

 42%|████▏     | 15890/37962 [36:49:05<50:29:41,  8.24s/it]
 42%|████▏     | 15891/37962 [36:49:14<50:36:50,  8.26s/it]
 42%|████▏     | 15892/37962 [36:49:22<50:42:02,  8.27s/it]
 42%|████▏     | 15893/37962 [36:49:30<50:56:16,  8.31s/it]
 42%|████▏     | 15894/37962 [36:49:39<51:09:26,  8.35s/it]
 42%|████▏     | 15895/37962 [36:49:47<50:57:26,  8.31s/it]
 42%|████▏     | 15896/37962 [36:49:55<50:07:09,  8.18s/it]
 42%|████▏     | 15897/37962 [36:50:03<50:07:02,  8.18s/it]
 42%|████▏     | 15898/37962 [36:50:11<49:49:56,  8.13s/it]
 42%|████▏     | 15899/37962 [36:50:19<49:30:37,  8.08s/it]
 42%|████▏     | 15900/37962 [36:50:27<49:20:45,  8.05s/it]
                                                           
{'loss': 1.447, 'grad_norm': 15.659972190856934, 'learning_rate': 1.3064355962168813e-05, 'epoch': 0.42}

 42%|████▏     | 15900/37962 [36:50:27<49:20:45,  8.05s/it]
 42%|████▏     | 15901/37962 [36:50:35<49:49:21,  8.13s/it]
 42%|████▏     | 15902/37962 [36:50:44<50:07:22,  8.18s/it]
 42%|████▏     | 15903/37962 [36:50:52<50:42:44,  8.28s/it]
 42%|████▏     | 15904/37962 [36:51:00<50:38:02,  8.26s/it]
 42%|████▏     | 15905/37962 [36:51:08<50:09:13,  8.19s/it]
 42%|████▏     | 15906/37962 [36:51:16<50:09:09,  8.19s/it]
 42%|████▏     | 15907/37962 [36:51:25<50:05:54,  8.18s/it]
 42%|████▏     | 15908/37962 [36:51:33<50:28:04,  8.24s/it]
 42%|████▏     | 15909/37962 [36:51:41<50:32:45,  8.25s/it]
 42%|████▏     | 15910/37962 [36:51:50<50:33:27,  8.25s/it]
                                                           
{'loss': 1.369, 'grad_norm': 7.726922035217285, 'learning_rate': 1.305623368719503e-05, 'epoch': 0.42}

 42%|████▏     | 15910/37962 [36:51:50<50:33:27,  8.25s/it]
 42%|████▏     | 15911/37962 [36:51:58<50:15:49,  8.21s/it]
 42%|████▏     | 15912/37962 [36:52:06<50:46:36,  8.29s/it]
 42%|████▏     | 15913/37962 [36:52:14<50:35:52,  8.26s/it]
 42%|████▏     | 15914/37962 [36:52:22<50:00:22,  8.17s/it]
 42%|████▏     | 15915/37962 [36:52:31<50:20:55,  8.22s/it]
 42%|████▏     | 15916/37962 [36:52:39<50:35:42,  8.26s/it]
 42%|████▏     | 15917/37962 [36:52:47<50:18:51,  8.22s/it]
 42%|████▏     | 15918/37962 [36:52:55<50:33:16,  8.26s/it]
 42%|████▏     | 15919/37962 [36:53:04<50:11:57,  8.20s/it]
 42%|████▏     | 15920/37962 [36:53:11<49:38:28,  8.11s/it]
                                                           
{'loss': 1.3382, 'grad_norm': 9.023506164550781, 'learning_rate': 1.3048109187641687e-05, 'epoch': 0.42}

 42%|████▏     | 15920/37962 [36:53:11<49:38:28,  8.11s/it]
 42%|████▏     | 15921/37962 [36:53:20<49:50:03,  8.14s/it]
 42%|████▏     | 15922/37962 [36:53:28<50:13:02,  8.20s/it]
 42%|████▏     | 15923/37962 [36:53:36<50:29:25,  8.25s/it]
 42%|████▏     | 15924/37962 [36:53:44<50:22:29,  8.23s/it]
 42%|████▏     | 15925/37962 [36:53:53<50:26:20,  8.24s/it]
 42%|████▏     | 15926/37962 [36:54:01<50:28:51,  8.25s/it]
 42%|████▏     | 15927/37962 [36:54:09<50:49:48,  8.30s/it]
 42%|████▏     | 15928/37962 [36:54:18<50:39:21,  8.28s/it]
 42%|████▏     | 15929/37962 [36:54:26<50:43:10,  8.29s/it]
 42%|████▏     | 15930/37962 [36:54:35<51:11:52,  8.37s/it]
                                                           
{'loss': 1.362, 'grad_norm': 8.783249855041504, 'learning_rate': 1.3039982469422476e-05, 'epoch': 0.42}

 42%|████▏     | 15930/37962 [36:54:35<51:11:52,  8.37s/it]
 42%|████▏     | 15931/37962 [36:54:43<51:15:25,  8.38s/it]
 42%|████▏     | 15932/37962 [36:54:51<51:13:48,  8.37s/it]
 42%|████▏     | 15933/37962 [36:55:00<51:00:43,  8.34s/it]
 42%|████▏     | 15934/37962 [36:55:08<50:57:28,  8.33s/it]
 42%|████▏     | 15935/37962 [36:55:16<50:52:14,  8.31s/it]
 42%|████▏     | 15936/37962 [36:55:24<50:52:14,  8.31s/it]
 42%|████▏     | 15937/37962 [36:55:33<50:40:31,  8.28s/it]
 42%|████▏     | 15938/37962 [36:55:41<50:36:40,  8.27s/it]
 42%|████▏     | 15939/37962 [36:55:49<50:38:00,  8.28s/it]
 42%|████▏     | 15940/37962 [36:55:57<50:26:12,  8.25s/it]
                                                           
{'loss': 1.3394, 'grad_norm': 3.572702407836914, 'learning_rate': 1.303185353845269e-05, 'epoch': 0.42}

 42%|████▏     | 15940/37962 [36:55:57<50:26:12,  8.25s/it]
 42%|████▏     | 15941/37962 [36:56:06<50:18:06,  8.22s/it]
 42%|████▏     | 15942/37962 [36:56:14<50:18:01,  8.22s/it]
 42%|████▏     | 15943/37962 [36:56:22<49:43:47,  8.13s/it]
 42%|████▏     | 15944/37962 [36:56:30<50:08:13,  8.20s/it]
 42%|████▏     | 15945/37962 [36:56:38<49:40:15,  8.12s/it]
 42%|████▏     | 15946/37962 [36:56:46<49:49:13,  8.15s/it]
 42%|████▏     | 15947/37962 [36:56:54<49:47:50,  8.14s/it]
 42%|████▏     | 15948/37962 [36:57:03<50:10:07,  8.20s/it]
 42%|████▏     | 15949/37962 [36:57:11<50:20:19,  8.23s/it]
 42%|████▏     | 15950/37962 [36:57:19<49:16:29,  8.06s/it]
                                                           
{'loss': 1.3894, 'grad_norm': 2.8077034950256348, 'learning_rate': 1.3023722400649234e-05, 'epoch': 0.42}

 42%|████▏     | 15950/37962 [36:57:19<49:16:29,  8.06s/it]
 42%|████▏     | 15951/37962 [36:57:27<49:34:20,  8.11s/it]
 42%|████▏     | 15952/37962 [36:57:35<49:52:36,  8.16s/it]
 42%|████▏     | 15953/37962 [36:57:43<49:26:52,  8.09s/it]
 42%|████▏     | 15954/37962 [36:57:51<49:55:09,  8.17s/it]
 42%|████▏     | 15955/37962 [36:58:00<49:52:42,  8.16s/it]
 42%|████▏     | 15956/37962 [36:58:08<50:04:49,  8.19s/it]
 42%|████▏     | 15957/37962 [36:58:16<50:34:20,  8.27s/it]
 42%|████▏     | 15958/37962 [36:58:24<50:21:58,  8.24s/it]
 42%|████▏     | 15959/37962 [36:58:33<50:30:24,  8.26s/it]
 42%|████▏     | 15960/37962 [36:58:41<50:32:00,  8.27s/it]
                                                           
{'loss': 1.4683, 'grad_norm': 6.189970016479492, 'learning_rate': 1.3015589061930627e-05, 'epoch': 0.42}

 42%|████▏     | 15960/37962 [36:58:41<50:32:00,  8.27s/it]
 42%|████▏     | 15961/37962 [36:58:49<50:46:46,  8.31s/it]
 42%|████▏     | 15962/37962 [36:58:58<50:45:08,  8.30s/it]
 42%|████▏     | 15963/37962 [36:59:06<51:20:13,  8.40s/it]
 42%|████▏     | 15964/37962 [36:59:15<51:16:43,  8.39s/it]
 42%|████▏     | 15965/37962 [36:59:23<51:08:38,  8.37s/it]
 42%|████▏     | 15966/37962 [36:59:31<50:55:22,  8.33s/it]
 42%|████▏     | 15967/37962 [36:59:40<50:50:43,  8.32s/it]
 42%|████▏     | 15968/37962 [36:59:48<50:56:50,  8.34s/it]
 42%|████▏     | 15969/37962 [36:59:56<50:57:38,  8.34s/it]
 42%|████▏     | 15970/37962 [37:00:05<51:01:13,  8.35s/it]
                                                           
{'loss': 1.4569, 'grad_norm': 3.104602575302124, 'learning_rate': 1.3007453528216981e-05, 'epoch': 0.42}

 42%|████▏     | 15970/37962 [37:00:05<51:01:13,  8.35s/it]
 42%|████▏     | 15971/37962 [37:00:13<50:58:29,  8.34s/it]
 42%|████▏     | 15972/37962 [37:00:21<50:45:32,  8.31s/it]
 42%|████▏     | 15973/37962 [37:00:30<50:43:47,  8.31s/it]
 42%|████▏     | 15974/37962 [37:00:38<50:36:52,  8.29s/it]
 42%|████▏     | 15975/37962 [37:00:46<50:44:06,  8.31s/it]
 42%|████▏     | 15976/37962 [37:00:54<50:38:29,  8.29s/it]
 42%|████▏     | 15977/37962 [37:01:03<50:29:59,  8.27s/it]
 42%|████▏     | 15978/37962 [37:01:11<50:11:39,  8.22s/it]
 42%|████▏     | 15979/37962 [37:01:19<49:50:06,  8.16s/it]
 42%|████▏     | 15980/37962 [37:01:27<49:57:12,  8.18s/it]
                                                           
{'loss': 1.407, 'grad_norm': 3.7802796363830566, 'learning_rate': 1.2999315805430015e-05, 'epoch': 0.42}

 42%|████▏     | 15980/37962 [37:01:27<49:57:12,  8.18s/it]
 42%|████▏     | 15981/37962 [37:01:35<50:07:50,  8.21s/it]
 42%|████▏     | 15982/37962 [37:01:44<50:20:01,  8.24s/it]
 42%|████▏     | 15983/37962 [37:01:52<50:25:13,  8.26s/it]
 42%|████▏     | 15984/37962 [37:02:00<50:29:58,  8.27s/it]
 42%|████▏     | 15985/37962 [37:02:08<50:24:50,  8.26s/it]
 42%|████▏     | 15986/37962 [37:02:17<50:47:24,  8.32s/it]
 42%|████▏     | 15987/37962 [37:02:32<63:02:43, 10.33s/it]
 42%|████▏     | 15988/37962 [37:02:40<58:26:00,  9.57s/it]
 42%|████▏     | 15989/37962 [37:02:48<56:18:35,  9.23s/it]
 42%|████▏     | 15990/37962 [37:02:56<54:19:27,  8.90s/it]
                                                           
{'loss': 1.3797, 'grad_norm': 4.91039514541626, 'learning_rate': 1.2991175899493032e-05, 'epoch': 0.42}

 42%|████▏     | 15990/37962 [37:02:56<54:19:27,  8.90s/it]
 42%|████▏     | 15991/37962 [37:03:05<53:16:15,  8.73s/it]
 42%|████▏     | 15992/37962 [37:03:13<52:28:20,  8.60s/it]
 42%|████▏     | 15993/37962 [37:03:21<51:50:33,  8.50s/it]
 42%|████▏     | 15994/37962 [37:03:29<51:35:10,  8.45s/it]
 42%|████▏     | 15995/37962 [37:03:38<51:27:27,  8.43s/it]
 42%|████▏     | 15996/37962 [37:03:46<51:06:37,  8.38s/it]
 42%|████▏     | 15997/37962 [37:03:54<50:25:54,  8.27s/it]
 42%|████▏     | 15998/37962 [37:04:02<50:01:35,  8.20s/it]
 42%|████▏     | 15999/37962 [37:04:10<50:09:40,  8.22s/it]
 42%|████▏     | 16000/37962 [37:04:18<49:46:42,  8.16s/it]
                                                           
{'loss': 1.3416, 'grad_norm': 7.9525146484375, 'learning_rate': 1.298303381633093e-05, 'epoch': 0.42}

 42%|████▏     | 16000/37962 [37:04:18<49:46:42,  8.16s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 42%|████▏     | 16001/37962 [37:05:08<126:13:47, 20.69s/it]
 42%|████▏     | 16002/37962 [37:05:17<103:33:41, 16.98s/it]
 42%|████▏     | 16003/37962 [37:05:25<87:35:14, 14.36s/it] 
 42%|████▏     | 16004/37962 [37:05:33<76:17:16, 12.51s/it]
 42%|████▏     | 16005/37962 [37:05:42<69:06:33, 11.33s/it]
 42%|████▏     | 16006/37962 [37:05:50<63:36:03, 10.43s/it]
 42%|████▏     | 16007/37962 [37:05:58<59:48:34,  9.81s/it]
 42%|████▏     | 16008/37962 [37:06:07<57:04:56,  9.36s/it]
 42%|████▏     | 16009/37962 [37:06:15<55:05:45,  9.03s/it]
 42%|████▏     | 16010/37962 [37:06:23<52:34:12,  8.62s/it]
                                                           
{'loss': 1.3937, 'grad_norm': 9.515143394470215, 'learning_rate': 1.2974889561870195e-05, 'epoch': 0.42}

 42%|████▏     | 16010/37962 [37:06:23<52:34:12,  8.62s/it]
 42%|████▏     | 16011/37962 [37:06:31<52:27:53,  8.60s/it]
 42%|████▏     | 16012/37962 [37:06:40<52:00:43,  8.53s/it]
 42%|████▏     | 16013/37962 [37:06:48<51:45:26,  8.49s/it]
 42%|████▏     | 16014/37962 [37:06:56<51:21:51,  8.42s/it]
 42%|████▏     | 16015/37962 [37:07:05<51:05:14,  8.38s/it]
 42%|████▏     | 16016/37962 [37:07:13<50:51:39,  8.34s/it]
 42%|████▏     | 16017/37962 [37:07:21<51:02:38,  8.37s/it]
 42%|████▏     | 16018/37962 [37:07:29<50:48:36,  8.34s/it]
 42%|████▏     | 16019/37962 [37:07:38<50:59:04,  8.36s/it]
 42%|████▏     | 16020/37962 [37:07:46<50:03:39,  8.21s/it]
                                                           
{'loss': 1.3005, 'grad_norm': 24.651504516601562, 'learning_rate': 1.2966743142038879e-05, 'epoch': 0.42}

 42%|████▏     | 16020/37962 [37:07:46<50:03:39,  8.21s/it]
 42%|████▏     | 16021/37962 [37:07:54<49:16:02,  8.08s/it]
 42%|████▏     | 16022/37962 [37:08:02<49:39:02,  8.15s/it]
 42%|████▏     | 16023/37962 [37:08:10<49:57:26,  8.20s/it]
 42%|████▏     | 16024/37962 [37:08:18<50:10:13,  8.23s/it]
 42%|████▏     | 16025/37962 [37:08:26<49:35:53,  8.14s/it]
 42%|████▏     | 16026/37962 [37:08:35<49:46:57,  8.17s/it]
 42%|████▏     | 16027/37962 [37:08:43<49:50:27,  8.18s/it]
 42%|████▏     | 16028/37962 [37:08:51<50:12:07,  8.24s/it]
 42%|████▏     | 16029/37962 [37:08:59<50:08:16,  8.23s/it]
 42%|████▏     | 16030/37962 [37:09:08<50:20:55,  8.26s/it]
                                                           
{'loss': 1.2954, 'grad_norm': 10.607142448425293, 'learning_rate': 1.2958594562766627e-05, 'epoch': 0.42}

 42%|████▏     | 16030/37962 [37:09:08<50:20:55,  8.26s/it]
 42%|████▏     | 16031/37962 [37:09:16<50:33:05,  8.30s/it]
 42%|████▏     | 16032/37962 [37:09:24<50:34:52,  8.30s/it]
 42%|████▏     | 16033/37962 [37:09:33<50:45:02,  8.33s/it]
 42%|████▏     | 16034/37962 [37:09:41<50:38:00,  8.31s/it]
 42%|████▏     | 16035/37962 [37:09:49<50:40:14,  8.32s/it]
 42%|████▏     | 16036/37962 [37:09:58<50:17:19,  8.26s/it]
 42%|████▏     | 16037/37962 [37:10:06<50:27:55,  8.29s/it]
 42%|████▏     | 16038/37962 [37:10:14<50:16:18,  8.25s/it]
 42%|████▏     | 16039/37962 [37:10:22<50:03:48,  8.22s/it]
 42%|████▏     | 16040/37962 [37:10:30<50:06:49,  8.23s/it]
                                                           
{'loss': 1.3965, 'grad_norm': 3.8237063884735107, 'learning_rate': 1.2950443829984649e-05, 'epoch': 0.42}

 42%|████▏     | 16040/37962 [37:10:30<50:06:49,  8.23s/it]
 42%|████▏     | 16041/37962 [37:10:39<50:15:42,  8.25s/it]
 42%|████▏     | 16042/37962 [37:10:47<50:10:06,  8.24s/it]
 42%|████▏     | 16043/37962 [37:10:55<50:11:12,  8.24s/it]
 42%|████▏     | 16044/37962 [37:11:03<50:06:17,  8.23s/it]
 42%|████▏     | 16045/37962 [37:11:12<50:14:08,  8.25s/it]
 42%|████▏     | 16046/37962 [37:11:20<49:59:09,  8.21s/it]
 42%|████▏     | 16047/37962 [37:11:28<50:08:19,  8.24s/it]
 42%|████▏     | 16048/37962 [37:11:36<50:06:48,  8.23s/it]
 42%|████▏     | 16049/37962 [37:11:45<50:26:50,  8.29s/it]
 42%|████▏     | 16050/37962 [37:11:53<50:13:18,  8.25s/it]
                                                           
{'loss': 1.4714, 'grad_norm': 9.423677444458008, 'learning_rate': 1.2942290949625717e-05, 'epoch': 0.42}

 42%|████▏     | 16050/37962 [37:11:53<50:13:18,  8.25s/it]
 42%|████▏     | 16051/37962 [37:12:01<50:22:26,  8.28s/it]
 42%|████▏     | 16052/37962 [37:12:10<50:24:23,  8.28s/it]
 42%|████▏     | 16053/37962 [37:12:18<50:48:07,  8.35s/it]
 42%|████▏     | 16054/37962 [37:12:26<50:35:35,  8.31s/it]
 42%|████▏     | 16055/37962 [37:12:35<50:25:46,  8.29s/it]
 42%|████▏     | 16056/37962 [37:12:43<50:32:54,  8.31s/it]
 42%|████▏     | 16057/37962 [37:12:51<50:28:32,  8.30s/it]
 42%|████▏     | 16058/37962 [37:12:59<49:48:59,  8.19s/it]
 42%|████▏     | 16059/37962 [37:13:07<49:42:36,  8.17s/it]
 42%|████▏     | 16060/37962 [37:13:15<49:35:53,  8.15s/it]
                                                           
{'loss': 1.4917, 'grad_norm': 13.82089614868164, 'learning_rate': 1.2934135927624176e-05, 'epoch': 0.42}

 42%|████▏     | 16060/37962 [37:13:15<49:35:53,  8.15s/it]
 42%|████▏     | 16061/37962 [37:13:23<49:29:17,  8.13s/it]
 42%|████▏     | 16062/37962 [37:13:32<49:34:44,  8.15s/it]
 42%|████▏     | 16063/37962 [37:13:40<49:51:06,  8.20s/it]
 42%|████▏     | 16064/37962 [37:13:48<50:02:00,  8.23s/it]
 42%|████▏     | 16065/37962 [37:13:57<50:08:20,  8.24s/it]
 42%|████▏     | 16066/37962 [37:14:05<50:22:49,  8.28s/it]
 42%|████▏     | 16067/37962 [37:14:13<50:26:47,  8.29s/it]
 42%|████▏     | 16068/37962 [37:14:22<50:31:55,  8.31s/it]
 42%|████▏     | 16069/37962 [37:14:30<50:42:19,  8.34s/it]
 42%|████▏     | 16070/37962 [37:14:38<50:47:17,  8.35s/it]
                                                           
{'loss': 1.3696, 'grad_norm': 2.9570960998535156, 'learning_rate': 1.2925978769915921e-05, 'epoch': 0.42}

 42%|████▏     | 16070/37962 [37:14:38<50:47:17,  8.35s/it]
 42%|████▏     | 16071/37962 [37:14:46<50:17:56,  8.27s/it]
 42%|████▏     | 16072/37962 [37:14:55<50:20:19,  8.28s/it]
 42%|████▏     | 16073/37962 [37:15:03<49:45:25,  8.18s/it]
 42%|████▏     | 16074/37962 [37:15:11<50:24:42,  8.29s/it]
 42%|████▏     | 16075/37962 [37:15:19<50:17:29,  8.27s/it]
 42%|████▏     | 16076/37962 [37:15:28<50:15:51,  8.27s/it]
 42%|████▏     | 16077/37962 [37:15:36<50:20:54,  8.28s/it]
 42%|████▏     | 16078/37962 [37:15:44<50:24:23,  8.29s/it]
 42%|████▏     | 16079/37962 [37:15:53<50:21:48,  8.29s/it]
 42%|████▏     | 16080/37962 [37:16:01<50:14:46,  8.27s/it]
                                                           
{'loss': 1.3548, 'grad_norm': 10.938511848449707, 'learning_rate': 1.2917819482438412e-05, 'epoch': 0.42}

 42%|████▏     | 16080/37962 [37:16:01<50:14:46,  8.27s/it]
 42%|████▏     | 16081/37962 [37:16:09<49:42:52,  8.18s/it]
 42%|████▏     | 16082/37962 [37:16:17<49:52:00,  8.20s/it]
 42%|████▏     | 16083/37962 [37:16:25<49:51:37,  8.20s/it]
 42%|████▏     | 16084/37962 [37:16:34<49:57:00,  8.22s/it]
 42%|████▏     | 16085/37962 [37:16:42<50:00:26,  8.23s/it]
 42%|████▏     | 16086/37962 [37:16:50<49:57:06,  8.22s/it]
 42%|████▏     | 16087/37962 [37:16:58<50:21:29,  8.29s/it]
 42%|████▏     | 16088/37962 [37:17:06<49:48:48,  8.20s/it]
 42%|████▏     | 16089/37962 [37:17:15<50:07:42,  8.25s/it]
 42%|████▏     | 16090/37962 [37:17:23<49:46:48,  8.19s/it]
                                                           
{'loss': 1.4372, 'grad_norm': 13.101497650146484, 'learning_rate': 1.2909658071130648e-05, 'epoch': 0.42}

 42%|████▏     | 16090/37962 [37:17:23<49:46:48,  8.19s/it]
 42%|████▏     | 16091/37962 [37:17:31<50:22:32,  8.29s/it]
 42%|████▏     | 16092/37962 [37:17:40<50:19:20,  8.28s/it]
 42%|████▏     | 16093/37962 [37:17:48<50:36:05,  8.33s/it]
 42%|████▏     | 16094/37962 [37:17:56<49:43:28,  8.19s/it]
 42%|████▏     | 16095/37962 [37:18:04<50:00:51,  8.23s/it]
 42%|████▏     | 16096/37962 [37:18:13<50:21:33,  8.29s/it]
 42%|████▏     | 16097/37962 [37:18:21<50:19:33,  8.29s/it]
 42%|████▏     | 16098/37962 [37:18:29<50:10:01,  8.26s/it]
 42%|████▏     | 16099/37962 [37:18:37<50:04:33,  8.25s/it]
 42%|████▏     | 16100/37962 [37:18:46<50:20:58,  8.29s/it]
                                                           
{'loss': 1.3544, 'grad_norm': 7.241787910461426, 'learning_rate': 1.2901494541933181e-05, 'epoch': 0.42}

 42%|████▏     | 16100/37962 [37:18:46<50:20:58,  8.29s/it]
 42%|████▏     | 16101/37962 [37:18:53<49:14:37,  8.11s/it]
 42%|████▏     | 16102/37962 [37:19:02<49:30:04,  8.15s/it]
 42%|████▏     | 16103/37962 [37:19:10<49:47:44,  8.20s/it]
 42%|████▏     | 16104/37962 [37:19:18<49:54:33,  8.22s/it]
 42%|████▏     | 16105/37962 [37:19:27<49:58:11,  8.23s/it]
 42%|████▏     | 16106/37962 [37:19:35<49:57:42,  8.23s/it]
 42%|████▏     | 16107/37962 [37:19:43<50:19:19,  8.29s/it]
 42%|████▏     | 16108/37962 [37:19:51<49:24:46,  8.14s/it]
 42%|████▏     | 16109/37962 [37:19:59<49:44:43,  8.19s/it]
 42%|████▏     | 16110/37962 [37:20:07<49:33:22,  8.16s/it]
                                                           
{'loss': 1.3894, 'grad_norm': 3.909675121307373, 'learning_rate': 1.2893328900788102e-05, 'epoch': 0.42}

 42%|████▏     | 16110/37962 [37:20:07<49:33:22,  8.16s/it]
 42%|████▏     | 16111/37962 [37:20:16<49:52:41,  8.22s/it]
 42%|████▏     | 16112/37962 [37:20:24<50:15:14,  8.28s/it]
 42%|████▏     | 16113/37962 [37:20:32<50:06:53,  8.26s/it]
 42%|████▏     | 16114/37962 [37:20:41<50:14:22,  8.28s/it]
 42%|████▏     | 16115/37962 [37:20:49<50:13:00,  8.27s/it]
 42%|████▏     | 16116/37962 [37:20:57<50:29:12,  8.32s/it]
 42%|████▏     | 16117/37962 [37:21:06<50:52:32,  8.38s/it]
 42%|████▏     | 16118/37962 [37:21:14<50:32:24,  8.33s/it]
 42%|████▏     | 16119/37962 [37:21:22<50:32:06,  8.33s/it]
 42%|████▏     | 16120/37962 [37:21:31<50:16:04,  8.29s/it]
                                                           
{'loss': 1.3848, 'grad_norm': 9.922338485717773, 'learning_rate': 1.2885161153639039e-05, 'epoch': 0.42}

 42%|████▏     | 16120/37962 [37:21:31<50:16:04,  8.29s/it]
 42%|████▏     | 16121/37962 [37:21:39<50:24:37,  8.31s/it]
 42%|████▏     | 16122/37962 [37:21:47<50:24:54,  8.31s/it]
 42%|████▏     | 16123/37962 [37:21:56<50:45:04,  8.37s/it]
 42%|████▏     | 16124/37962 [37:22:04<50:47:42,  8.37s/it]
 42%|████▏     | 16125/37962 [37:22:12<50:22:23,  8.30s/it]
 42%|████▏     | 16126/37962 [37:22:20<49:32:45,  8.17s/it]
 42%|████▏     | 16127/37962 [37:22:29<50:14:33,  8.28s/it]
 42%|████▏     | 16128/37962 [37:22:37<50:11:01,  8.27s/it]
 42%|████▏     | 16129/37962 [37:22:46<50:41:18,  8.36s/it]
 42%|████▏     | 16130/37962 [37:22:54<49:59:13,  8.24s/it]
                                                           
{'loss': 1.4313, 'grad_norm': 18.903465270996094, 'learning_rate': 1.2876991306431153e-05, 'epoch': 0.42}

 42%|████▏     | 16130/37962 [37:22:54<49:59:13,  8.24s/it]
 42%|████▏     | 16131/37962 [37:23:02<49:46:20,  8.21s/it]
 42%|████▏     | 16132/37962 [37:23:10<49:37:20,  8.18s/it]
 42%|████▏     | 16133/37962 [37:23:18<49:46:21,  8.21s/it]
 43%|████▎     | 16134/37962 [37:23:27<50:25:52,  8.32s/it]
 43%|████▎     | 16135/37962 [37:23:35<50:35:36,  8.34s/it]
 43%|████▎     | 16136/37962 [37:23:43<50:37:15,  8.35s/it]
 43%|████▎     | 16137/37962 [37:23:52<50:29:01,  8.33s/it]
 43%|████▎     | 16138/37962 [37:24:00<50:40:38,  8.36s/it]
 43%|████▎     | 16139/37962 [37:24:08<50:29:14,  8.33s/it]
 43%|████▎     | 16140/37962 [37:24:16<49:49:22,  8.22s/it]
                                                           
{'loss': 1.4513, 'grad_norm': 5.6014299392700195, 'learning_rate': 1.2868819365111138e-05, 'epoch': 0.43}

 43%|████▎     | 16140/37962 [37:24:16<49:49:22,  8.22s/it]
 43%|████▎     | 16141/37962 [37:24:25<49:49:27,  8.22s/it]
 43%|████▎     | 16142/37962 [37:24:33<50:13:43,  8.29s/it]
 43%|████▎     | 16143/37962 [37:24:41<50:17:49,  8.30s/it]
 43%|████▎     | 16144/37962 [37:24:50<50:37:45,  8.35s/it]
 43%|████▎     | 16145/37962 [37:24:58<50:01:43,  8.26s/it]
 43%|████▎     | 16146/37962 [37:25:06<50:18:46,  8.30s/it]
 43%|████▎     | 16147/37962 [37:25:14<50:00:21,  8.25s/it]
 43%|████▎     | 16148/37962 [37:25:23<50:09:25,  8.28s/it]
 43%|████▎     | 16149/37962 [37:25:31<50:23:24,  8.32s/it]
 43%|████▎     | 16150/37962 [37:25:39<50:27:12,  8.33s/it]
                                                           
{'loss': 1.3996, 'grad_norm': 19.40557098388672, 'learning_rate': 1.2860645335627207e-05, 'epoch': 0.43}

 43%|████▎     | 16150/37962 [37:25:39<50:27:12,  8.33s/it]
 43%|████▎     | 16151/37962 [37:25:48<50:37:25,  8.36s/it]
 43%|████▎     | 16152/37962 [37:25:56<50:17:13,  8.30s/it]
 43%|████▎     | 16153/37962 [37:26:04<50:22:08,  8.31s/it]
 43%|████▎     | 16154/37962 [37:26:13<50:16:39,  8.30s/it]
 43%|████▎     | 16155/37962 [37:26:21<50:09:12,  8.28s/it]
 43%|████▎     | 16156/37962 [37:26:29<49:59:24,  8.25s/it]
 43%|████▎     | 16157/37962 [37:26:37<49:57:22,  8.25s/it]
 43%|████▎     | 16158/37962 [37:26:46<49:59:24,  8.25s/it]
 43%|████▎     | 16159/37962 [37:26:54<50:05:24,  8.27s/it]
 43%|████▎     | 16160/37962 [37:27:02<50:01:28,  8.26s/it]
                                                           
{'loss': 1.5025, 'grad_norm': 6.557480335235596, 'learning_rate': 1.2852469223929087e-05, 'epoch': 0.43}

 43%|████▎     | 16160/37962 [37:27:02<50:01:28,  8.26s/it]
 43%|████▎     | 16161/37962 [37:27:10<49:51:23,  8.23s/it]
 43%|████▎     | 16162/37962 [37:27:18<49:32:06,  8.18s/it]
 43%|████▎     | 16163/37962 [37:27:27<49:26:14,  8.16s/it]
 43%|████▎     | 16164/37962 [37:27:35<49:34:12,  8.19s/it]
 43%|████▎     | 16165/37962 [37:27:43<49:44:36,  8.22s/it]
 43%|████▎     | 16166/37962 [37:27:51<50:04:16,  8.27s/it]
 43%|████▎     | 16167/37962 [37:27:59<49:40:52,  8.21s/it]
 43%|████▎     | 16168/37962 [37:28:08<50:15:39,  8.30s/it]
 43%|████▎     | 16169/37962 [37:28:16<49:59:55,  8.26s/it]
 43%|████▎     | 16170/37962 [37:28:24<49:58:36,  8.26s/it]
                                                           
{'loss': 1.3485, 'grad_norm': 9.43835163116455, 'learning_rate': 1.2844291035968042e-05, 'epoch': 0.43}

 43%|████▎     | 16170/37962 [37:28:24<49:58:36,  8.26s/it]
 43%|████▎     | 16171/37962 [37:28:33<50:06:38,  8.28s/it]
 43%|████▎     | 16172/37962 [37:28:41<50:16:04,  8.30s/it]
 43%|████▎     | 16173/37962 [37:28:49<50:10:03,  8.29s/it]
 43%|████▎     | 16174/37962 [37:28:58<50:05:57,  8.28s/it]
 43%|████▎     | 16175/37962 [37:29:06<50:09:32,  8.29s/it]
 43%|████▎     | 16176/37962 [37:29:14<50:12:23,  8.30s/it]
 43%|████▎     | 16177/37962 [37:29:22<50:02:38,  8.27s/it]
 43%|████▎     | 16178/37962 [37:29:31<49:58:21,  8.26s/it]
 43%|████▎     | 16179/37962 [37:29:39<49:53:41,  8.25s/it]
 43%|████▎     | 16180/37962 [37:29:47<49:58:20,  8.26s/it]
                                                           
{'loss': 1.32, 'grad_norm': 4.642200946807861, 'learning_rate': 1.2836110777696822e-05, 'epoch': 0.43}

 43%|████▎     | 16180/37962 [37:29:47<49:58:20,  8.26s/it]
 43%|████▎     | 16181/37962 [37:29:56<50:18:36,  8.32s/it]
 43%|████▎     | 16182/37962 [37:30:04<50:25:55,  8.34s/it]
 43%|████▎     | 16183/37962 [37:30:12<50:08:25,  8.29s/it]
 43%|████▎     | 16184/37962 [37:30:21<50:25:27,  8.34s/it]
 43%|████▎     | 16185/37962 [37:30:29<50:06:05,  8.28s/it]
 43%|████▎     | 16186/37962 [37:30:37<50:19:45,  8.32s/it]
 43%|████▎     | 16187/37962 [37:30:45<50:10:42,  8.30s/it]
 43%|████▎     | 16188/37962 [37:30:54<50:24:56,  8.34s/it]
 43%|████▎     | 16189/37962 [37:31:02<50:02:05,  8.27s/it]
 43%|████▎     | 16190/37962 [37:31:10<49:57:16,  8.26s/it]
                                                           
{'loss': 1.4316, 'grad_norm': 5.85961389541626, 'learning_rate': 1.28279284550697e-05, 'epoch': 0.43}

 43%|████▎     | 16190/37962 [37:31:10<49:57:16,  8.26s/it]
 43%|████▎     | 16191/37962 [37:31:19<50:03:34,  8.28s/it]
 43%|████▎     | 16192/37962 [37:31:27<49:55:20,  8.26s/it]
 43%|████▎     | 16193/37962 [37:31:42<63:23:46, 10.48s/it]
 43%|████▎     | 16194/37962 [37:31:51<59:28:19,  9.84s/it]
 43%|████▎     | 16195/37962 [37:31:59<56:54:30,  9.41s/it]
 43%|████▎     | 16196/37962 [37:32:07<54:47:37,  9.06s/it]
 43%|████▎     | 16197/37962 [37:32:16<53:14:55,  8.81s/it]
 43%|████▎     | 16198/37962 [37:32:24<52:10:14,  8.63s/it]
 43%|████▎     | 16199/37962 [37:32:32<51:28:00,  8.51s/it]
 43%|████▎     | 16200/37962 [37:32:40<50:24:50,  8.34s/it]
                                                           
{'loss': 1.4176, 'grad_norm': 12.451931953430176, 'learning_rate': 1.2819744074042446e-05, 'epoch': 0.43}

 43%|████▎     | 16200/37962 [37:32:40<50:24:50,  8.34s/it]
 43%|████▎     | 16201/37962 [37:32:48<50:21:07,  8.33s/it]
 43%|████▎     | 16202/37962 [37:32:56<50:01:27,  8.28s/it]
 43%|████▎     | 16203/37962 [37:33:05<49:56:20,  8.26s/it]
 43%|████▎     | 16204/37962 [37:33:13<49:12:04,  8.14s/it]
 43%|████▎     | 16205/37962 [37:33:21<49:06:22,  8.13s/it]
 43%|████▎     | 16206/37962 [37:33:29<48:57:48,  8.10s/it]
 43%|████▎     | 16207/37962 [37:33:37<49:17:47,  8.16s/it]
 43%|████▎     | 16208/37962 [37:33:45<48:57:30,  8.10s/it]
 43%|████▎     | 16209/37962 [37:33:53<49:14:14,  8.15s/it]
 43%|████▎     | 16210/37962 [37:34:01<49:23:54,  8.18s/it]
                                                           
{'loss': 1.4121, 'grad_norm': 6.895419120788574, 'learning_rate': 1.2811557640572329e-05, 'epoch': 0.43}

 43%|████▎     | 16210/37962 [37:34:01<49:23:54,  8.18s/it]
 43%|████▎     | 16211/37962 [37:34:10<49:24:33,  8.18s/it]
 43%|████▎     | 16212/37962 [37:34:18<49:44:24,  8.23s/it]
 43%|████▎     | 16213/37962 [37:34:26<49:54:58,  8.26s/it]
 43%|████▎     | 16214/37962 [37:34:35<49:51:52,  8.25s/it]
 43%|████▎     | 16215/37962 [37:34:43<49:40:45,  8.22s/it]
 43%|████▎     | 16216/37962 [37:34:51<49:37:47,  8.22s/it]
 43%|████▎     | 16217/37962 [37:34:59<49:43:25,  8.23s/it]
 43%|████▎     | 16218/37962 [37:35:07<49:26:01,  8.18s/it]
 43%|████▎     | 16219/37962 [37:35:15<49:28:11,  8.19s/it]
 43%|████▎     | 16220/37962 [37:35:24<49:55:08,  8.27s/it]
                                                           
{'loss': 1.2888, 'grad_norm': 7.075610637664795, 'learning_rate': 1.2803369160618115e-05, 'epoch': 0.43}

 43%|████▎     | 16220/37962 [37:35:24<49:55:08,  8.27s/it]
 43%|████▎     | 16221/37962 [37:35:32<50:16:13,  8.32s/it]
 43%|████▎     | 16222/37962 [37:35:41<50:02:10,  8.29s/it]
 43%|████▎     | 16223/37962 [37:35:48<49:09:37,  8.14s/it]
 43%|████▎     | 16224/37962 [37:35:56<48:51:30,  8.09s/it]
 43%|████▎     | 16225/37962 [37:36:05<49:18:15,  8.17s/it]
 43%|████▎     | 16226/37962 [37:36:13<49:14:17,  8.16s/it]
 43%|████▎     | 16227/37962 [37:36:21<49:22:04,  8.18s/it]
 43%|████▎     | 16228/37962 [37:36:30<50:14:25,  8.32s/it]
 43%|████▎     | 16229/37962 [37:36:38<50:14:36,  8.32s/it]
 43%|████▎     | 16230/37962 [37:36:46<50:23:51,  8.35s/it]
                                                           
{'loss': 1.3997, 'grad_norm': 10.624726295471191, 'learning_rate': 1.279517864014005e-05, 'epoch': 0.43}

 43%|████▎     | 16230/37962 [37:36:46<50:23:51,  8.35s/it]
 43%|████▎     | 16231/37962 [37:36:55<50:14:41,  8.32s/it]
 43%|████▎     | 16232/37962 [37:37:03<50:14:52,  8.32s/it]
 43%|████▎     | 16233/37962 [37:37:11<50:09:45,  8.31s/it]
 43%|████▎     | 16234/37962 [37:37:20<49:56:47,  8.28s/it]
 43%|████▎     | 16235/37962 [37:37:28<50:06:12,  8.30s/it]
 43%|████▎     | 16236/37962 [37:37:36<50:11:08,  8.32s/it]
 43%|████▎     | 16237/37962 [37:37:44<49:53:13,  8.27s/it]
 43%|████▎     | 16238/37962 [37:37:53<50:15:34,  8.33s/it]
 43%|████▎     | 16239/37962 [37:38:01<49:57:50,  8.28s/it]
 43%|████▎     | 16240/37962 [37:38:09<49:52:30,  8.27s/it]
                                                           
{'loss': 1.4445, 'grad_norm': 18.465593338012695, 'learning_rate': 1.2786986085099879e-05, 'epoch': 0.43}

 43%|████▎     | 16240/37962 [37:38:09<49:52:30,  8.27s/it]
 43%|████▎     | 16241/37962 [37:38:18<49:50:32,  8.26s/it]
 43%|████▎     | 16242/37962 [37:38:26<49:41:15,  8.24s/it]
 43%|████▎     | 16243/37962 [37:38:34<49:39:32,  8.23s/it]
 43%|████▎     | 16244/37962 [37:38:42<49:49:32,  8.26s/it]
 43%|████▎     | 16245/37962 [37:38:50<49:32:48,  8.21s/it]
 43%|████▎     | 16246/37962 [37:38:59<49:55:12,  8.28s/it]
 43%|████▎     | 16247/37962 [37:39:07<50:23:51,  8.36s/it]
 43%|████▎     | 16248/37962 [37:39:15<50:06:08,  8.31s/it]
 43%|████▎     | 16249/37962 [37:39:24<49:41:10,  8.24s/it]
 43%|████▎     | 16250/37962 [37:39:32<50:05:15,  8.30s/it]
                                                           
{'loss': 1.3344, 'grad_norm': 3.063359260559082, 'learning_rate': 1.2778791501460817e-05, 'epoch': 0.43}

 43%|████▎     | 16250/37962 [37:39:32<50:05:15,  8.30s/it]
 43%|████▎     | 16251/37962 [37:39:40<48:57:08,  8.12s/it]
 43%|████▎     | 16252/37962 [37:39:48<48:53:05,  8.11s/it]
 43%|████▎     | 16253/37962 [37:39:56<48:29:05,  8.04s/it]
 43%|████▎     | 16254/37962 [37:40:04<48:53:49,  8.11s/it]
 43%|████▎     | 16255/37962 [37:40:12<48:49:40,  8.10s/it]
 43%|████▎     | 16256/37962 [37:40:20<49:17:18,  8.17s/it]
 43%|████▎     | 16257/37962 [37:40:29<49:21:25,  8.19s/it]
 43%|████▎     | 16258/37962 [37:40:37<49:43:13,  8.25s/it]
 43%|████▎     | 16259/37962 [37:40:45<49:53:57,  8.28s/it]
 43%|████▎     | 16260/37962 [37:40:53<49:28:26,  8.21s/it]
                                                           
{'loss': 1.2595, 'grad_norm': 5.968415260314941, 'learning_rate': 1.2770594895187558e-05, 'epoch': 0.43}

 43%|████▎     | 16260/37962 [37:40:53<49:28:26,  8.21s/it]
 43%|████▎     | 16261/37962 [37:41:02<49:26:35,  8.20s/it]
 43%|████▎     | 16262/37962 [37:41:10<49:47:26,  8.26s/it]
 43%|████▎     | 16263/37962 [37:41:18<50:10:33,  8.32s/it]
 43%|████▎     | 16264/37962 [37:41:26<49:34:20,  8.22s/it]
 43%|████▎     | 16265/37962 [37:41:35<49:38:42,  8.24s/it]
 43%|████▎     | 16266/37962 [37:41:43<49:45:38,  8.26s/it]
 43%|████▎     | 16267/37962 [37:41:51<50:04:57,  8.31s/it]
 43%|████▎     | 16268/37962 [37:41:59<49:32:28,  8.22s/it]
 43%|████▎     | 16269/37962 [37:42:08<49:24:11,  8.20s/it]
 43%|████▎     | 16270/37962 [37:42:16<49:03:51,  8.14s/it]
                                                           
{'loss': 1.3247, 'grad_norm': 12.125001907348633, 'learning_rate': 1.2762396272246273e-05, 'epoch': 0.43}

 43%|████▎     | 16270/37962 [37:42:16<49:03:51,  8.14s/it]
 43%|████▎     | 16271/37962 [37:42:24<48:55:53,  8.12s/it]
 43%|████▎     | 16272/37962 [37:42:32<49:56:47,  8.29s/it]
 43%|████▎     | 16273/37962 [37:42:41<49:46:14,  8.26s/it]
 43%|████▎     | 16274/37962 [37:42:49<49:45:49,  8.26s/it]
 43%|████▎     | 16275/37962 [37:42:57<49:47:28,  8.27s/it]
 43%|████▎     | 16276/37962 [37:43:05<49:25:27,  8.20s/it]
 43%|████▎     | 16277/37962 [37:43:14<50:02:09,  8.31s/it]
 43%|████▎     | 16278/37962 [37:43:22<49:29:08,  8.22s/it]
 43%|████▎     | 16279/37962 [37:43:30<49:23:54,  8.20s/it]
 43%|████▎     | 16280/37962 [37:43:38<49:51:02,  8.28s/it]
                                                           
{'loss': 1.3405, 'grad_norm': 4.532658100128174, 'learning_rate': 1.27541956386046e-05, 'epoch': 0.43}

 43%|████▎     | 16280/37962 [37:43:38<49:51:02,  8.28s/it]
 43%|████▎     | 16281/37962 [37:43:47<49:43:07,  8.26s/it]
 43%|████▎     | 16282/37962 [37:43:55<49:48:47,  8.27s/it]
 43%|████▎     | 16283/37962 [37:44:03<49:26:20,  8.21s/it]
 43%|████▎     | 16284/37962 [37:44:11<49:15:36,  8.18s/it]
 43%|████▎     | 16285/37962 [37:44:19<49:33:03,  8.23s/it]
 43%|████▎     | 16286/37962 [37:44:28<49:45:05,  8.26s/it]
 43%|████▎     | 16287/37962 [37:44:36<50:01:11,  8.31s/it]
 43%|████▎     | 16288/37962 [37:44:45<50:21:09,  8.36s/it]
 43%|████▎     | 16289/37962 [37:44:53<50:37:09,  8.41s/it]
 43%|████▎     | 16290/37962 [37:45:01<49:57:47,  8.30s/it]
                                                           
{'loss': 1.4102, 'grad_norm': 11.607972145080566, 'learning_rate': 1.2745993000231632e-05, 'epoch': 0.43}

 43%|████▎     | 16290/37962 [37:45:01<49:57:47,  8.30s/it]
 43%|████▎     | 16291/37962 [37:45:09<49:36:20,  8.24s/it]
 43%|████▎     | 16292/37962 [37:45:18<49:44:06,  8.26s/it]
 43%|████▎     | 16293/37962 [37:45:26<50:31:54,  8.40s/it]
 43%|████▎     | 16294/37962 [37:45:34<49:15:55,  8.19s/it]
 43%|████▎     | 16295/37962 [37:45:42<49:20:53,  8.20s/it]
 43%|████▎     | 16296/37962 [37:45:51<49:35:32,  8.24s/it]
 43%|████▎     | 16297/37962 [37:45:59<49:28:37,  8.22s/it]
 43%|████▎     | 16298/37962 [37:46:07<49:30:57,  8.23s/it]
 43%|████▎     | 16299/37962 [37:46:15<49:15:54,  8.19s/it]
 43%|████▎     | 16300/37962 [37:46:23<49:25:31,  8.21s/it]
                                                           
{'loss': 1.3324, 'grad_norm': 6.874989986419678, 'learning_rate': 1.273778836309793e-05, 'epoch': 0.43}

 43%|████▎     | 16300/37962 [37:46:23<49:25:31,  8.21s/it]
 43%|████▎     | 16301/37962 [37:46:32<49:32:01,  8.23s/it]
 43%|████▎     | 16302/37962 [37:46:48<64:04:33, 10.65s/it]
 43%|████▎     | 16303/37962 [37:46:56<59:50:09,  9.95s/it]
 43%|████▎     | 16304/37962 [37:47:04<56:23:33,  9.37s/it]
 43%|████▎     | 16305/37962 [37:47:13<54:35:15,  9.07s/it]
 43%|████▎     | 16306/37962 [37:47:21<52:41:23,  8.76s/it]
 43%|████▎     | 16307/37962 [37:47:29<51:48:06,  8.61s/it]
 43%|████▎     | 16308/37962 [37:47:37<51:21:16,  8.54s/it]
 43%|████▎     | 16309/37962 [37:47:46<50:55:30,  8.47s/it]
 43%|████▎     | 16310/37962 [37:47:54<50:31:01,  8.40s/it]
                                                           
{'loss': 1.2742, 'grad_norm': 12.042860984802246, 'learning_rate': 1.2729581733175508e-05, 'epoch': 0.43}

 43%|████▎     | 16310/37962 [37:47:54<50:31:01,  8.40s/it]
 43%|████▎     | 16311/37962 [37:48:02<49:59:59,  8.31s/it]
 43%|████▎     | 16312/37962 [37:48:10<49:54:41,  8.30s/it]
 43%|████▎     | 16313/37962 [37:48:18<49:47:46,  8.28s/it]
 43%|████▎     | 16314/37962 [37:48:27<49:27:42,  8.23s/it]
 43%|████▎     | 16315/37962 [37:48:35<49:47:26,  8.28s/it]
 43%|████▎     | 16316/37962 [37:48:43<49:33:10,  8.24s/it]
 43%|████▎     | 16317/37962 [37:48:52<50:11:24,  8.35s/it]
 43%|████▎     | 16318/37962 [37:49:00<50:00:25,  8.32s/it]
 43%|████▎     | 16319/37962 [37:49:08<50:22:44,  8.38s/it]
 43%|████▎     | 16320/37962 [37:49:17<50:04:15,  8.33s/it]
                                                           
{'loss': 1.4552, 'grad_norm': 13.916406631469727, 'learning_rate': 1.2721373116437832e-05, 'epoch': 0.43}

 43%|████▎     | 16320/37962 [37:49:17<50:04:15,  8.33s/it]
 43%|████▎     | 16321/37962 [37:49:25<50:06:37,  8.34s/it]
 43%|████▎     | 16322/37962 [37:49:33<50:07:46,  8.34s/it]
 43%|████▎     | 16323/37962 [37:49:42<50:06:33,  8.34s/it]
 43%|████▎     | 16324/37962 [37:49:50<50:07:29,  8.34s/it]
 43%|████▎     | 16325/37962 [37:49:58<49:15:37,  8.20s/it]
 43%|████▎     | 16326/37962 [37:50:07<50:05:18,  8.33s/it]
 43%|████▎     | 16327/37962 [37:50:15<49:55:42,  8.31s/it]
 43%|████▎     | 16328/37962 [37:50:23<50:02:16,  8.33s/it]
 43%|████▎     | 16329/37962 [37:50:31<49:57:27,  8.31s/it]
 43%|████▎     | 16330/37962 [37:50:40<50:16:09,  8.37s/it]
                                                           
{'loss': 1.3336, 'grad_norm': 10.46174144744873, 'learning_rate': 1.2713162518859803e-05, 'epoch': 0.43}

 43%|████▎     | 16330/37962 [37:50:40<50:16:09,  8.37s/it]
 43%|████▎     | 16331/37962 [37:50:48<49:58:52,  8.32s/it]
 43%|████▎     | 16332/37962 [37:50:56<49:45:08,  8.28s/it]
 43%|████▎     | 16333/37962 [37:51:05<49:32:49,  8.25s/it]
 43%|████▎     | 16334/37962 [37:51:13<49:39:37,  8.27s/it]
 43%|████▎     | 16335/37962 [37:51:21<49:39:42,  8.27s/it]
 43%|████▎     | 16336/37962 [37:51:29<49:46:29,  8.29s/it]
 43%|████▎     | 16337/37962 [37:51:38<49:50:55,  8.30s/it]
 43%|████▎     | 16338/37962 [37:51:46<49:22:51,  8.22s/it]
 43%|████▎     | 16339/37962 [37:51:54<49:17:13,  8.21s/it]
 43%|████▎     | 16340/37962 [37:52:02<49:33:01,  8.25s/it]
                                                           
{'loss': 1.3482, 'grad_norm': 12.4313325881958, 'learning_rate': 1.2704949946417781e-05, 'epoch': 0.43}

 43%|████▎     | 16340/37962 [37:52:02<49:33:01,  8.25s/it]
 43%|████▎     | 16341/37962 [37:52:10<49:22:57,  8.22s/it]
 43%|████▎     | 16342/37962 [37:52:19<49:32:00,  8.25s/it]
 43%|████▎     | 16343/37962 [37:52:27<49:42:31,  8.28s/it]
 43%|████▎     | 16344/37962 [37:52:35<49:14:42,  8.20s/it]
 43%|████▎     | 16345/37962 [37:52:43<49:19:01,  8.21s/it]
 43%|████▎     | 16346/37962 [37:52:52<49:11:14,  8.19s/it]
 43%|████▎     | 16347/37962 [37:53:00<49:10:28,  8.19s/it]
 43%|████▎     | 16348/37962 [37:53:08<48:53:42,  8.14s/it]
 43%|████▎     | 16349/37962 [37:53:16<49:00:53,  8.16s/it]
 43%|████▎     | 16350/37962 [37:53:24<49:10:28,  8.19s/it]
                                                           
{'loss': 1.4838, 'grad_norm': 18.28696632385254, 'learning_rate': 1.2696735405089557e-05, 'epoch': 0.43}

 43%|████▎     | 16350/37962 [37:53:24<49:10:28,  8.19s/it]
 43%|████▎     | 16351/37962 [37:53:32<49:10:17,  8.19s/it]
 43%|████▎     | 16352/37962 [37:53:41<49:16:53,  8.21s/it]
 43%|████▎     | 16353/37962 [37:53:49<49:17:49,  8.21s/it]
 43%|████▎     | 16354/37962 [37:53:57<49:05:00,  8.18s/it]
 43%|████▎     | 16355/37962 [37:54:06<49:53:55,  8.31s/it]
 43%|████▎     | 16356/37962 [37:54:14<49:31:34,  8.25s/it]
 43%|████▎     | 16357/37962 [37:54:22<49:34:12,  8.26s/it]
 43%|████▎     | 16358/37962 [37:54:30<49:46:30,  8.29s/it]
 43%|████▎     | 16359/37962 [37:54:39<50:07:45,  8.35s/it]
 43%|████▎     | 16360/37962 [37:54:47<49:50:16,  8.31s/it]
                                                           
{'loss': 1.4805, 'grad_norm': 6.721951484680176, 'learning_rate': 1.2688518900854347e-05, 'epoch': 0.43}

 43%|████▎     | 16360/37962 [37:54:47<49:50:16,  8.31s/it]
 43%|████▎     | 16361/37962 [37:54:55<49:57:45,  8.33s/it]
 43%|████▎     | 16362/37962 [37:55:03<49:19:56,  8.22s/it]
 43%|████▎     | 16363/37962 [37:55:11<48:55:09,  8.15s/it]
 43%|████▎     | 16364/37962 [37:55:20<49:07:45,  8.19s/it]
 43%|████▎     | 16365/37962 [37:55:28<48:46:18,  8.13s/it]
 43%|████▎     | 16366/37962 [37:55:36<49:31:26,  8.26s/it]
 43%|████▎     | 16367/37962 [37:55:44<49:22:54,  8.23s/it]
 43%|████▎     | 16368/37962 [37:55:53<49:36:22,  8.27s/it]
 43%|████▎     | 16369/37962 [37:56:01<49:20:48,  8.23s/it]
 43%|████▎     | 16370/37962 [37:56:09<49:24:54,  8.24s/it]
                                                           
{'loss': 1.3802, 'grad_norm': 11.45789909362793, 'learning_rate': 1.2680300439692803e-05, 'epoch': 0.43}

 43%|████▎     | 16370/37962 [37:56:09<49:24:54,  8.24s/it]
 43%|████▎     | 16371/37962 [37:56:17<49:28:38,  8.25s/it]
 43%|████▎     | 16372/37962 [37:56:26<49:33:21,  8.26s/it]
 43%|████▎     | 16373/37962 [37:56:34<50:12:39,  8.37s/it]
 43%|████▎     | 16374/37962 [37:56:43<49:55:29,  8.33s/it]
 43%|████▎     | 16375/37962 [37:56:51<50:02:40,  8.35s/it]
 43%|████▎     | 16376/37962 [37:56:59<50:10:05,  8.37s/it]
 43%|████▎     | 16377/37962 [37:57:08<50:11:00,  8.37s/it]
 43%|████▎     | 16378/37962 [37:57:16<49:58:24,  8.34s/it]
 43%|████▎     | 16379/37962 [37:57:24<49:45:02,  8.30s/it]
 43%|████▎     | 16380/37962 [37:57:33<49:59:18,  8.34s/it]
                                                           
{'loss': 1.4597, 'grad_norm': 17.696706771850586, 'learning_rate': 1.2672080027587003e-05, 'epoch': 0.43}

 43%|████▎     | 16380/37962 [37:57:33<49:59:18,  8.34s/it]
 43%|████▎     | 16381/37962 [37:57:41<49:13:57,  8.21s/it]
 43%|████▎     | 16382/37962 [37:57:49<49:21:02,  8.23s/it]
 43%|████▎     | 16383/37962 [37:57:57<49:22:34,  8.24s/it]
 43%|████▎     | 16384/37962 [37:58:06<50:04:18,  8.35s/it]
 43%|████▎     | 16385/37962 [37:58:14<50:29:36,  8.42s/it]
 43%|████▎     | 16386/37962 [37:58:22<50:02:59,  8.35s/it]
 43%|████▎     | 16387/37962 [37:58:31<50:09:01,  8.37s/it]
 43%|████▎     | 16388/37962 [37:58:39<49:23:43,  8.24s/it]
 43%|████▎     | 16389/37962 [37:58:47<49:48:54,  8.31s/it]
 43%|████▎     | 16390/37962 [37:58:56<49:38:41,  8.28s/it]
                                                           
{'loss': 1.451, 'grad_norm': 8.663355827331543, 'learning_rate': 1.2663857670520442e-05, 'epoch': 0.43}

 43%|████▎     | 16390/37962 [37:58:56<49:38:41,  8.28s/it]
 43%|████▎     | 16391/37962 [37:59:04<49:32:32,  8.27s/it]
 43%|████▎     | 16392/37962 [37:59:12<49:38:51,  8.29s/it]
 43%|████▎     | 16393/37962 [37:59:21<50:00:51,  8.35s/it]
 43%|████▎     | 16394/37962 [37:59:29<49:42:33,  8.30s/it]
 43%|████▎     | 16395/37962 [37:59:37<49:58:25,  8.34s/it]
 43%|████▎     | 16396/37962 [37:59:45<49:47:44,  8.31s/it]
 43%|████▎     | 16397/37962 [37:59:54<49:52:21,  8.33s/it]
 43%|████▎     | 16398/37962 [38:00:02<49:25:38,  8.25s/it]
 43%|████▎     | 16399/37962 [38:00:10<49:34:28,  8.28s/it]
 43%|████▎     | 16400/37962 [38:00:19<49:52:39,  8.33s/it]
                                                           
{'loss': 1.4477, 'grad_norm': 25.208045959472656, 'learning_rate': 1.2655633374478031e-05, 'epoch': 0.43}

 43%|████▎     | 16400/37962 [38:00:19<49:52:39,  8.33s/it]
 43%|████▎     | 16401/37962 [38:00:27<50:11:12,  8.38s/it]
 43%|████▎     | 16402/37962 [38:00:35<49:30:39,  8.27s/it]
 43%|████▎     | 16403/37962 [38:00:43<49:11:17,  8.21s/it]
 43%|████▎     | 16404/37962 [38:00:52<49:29:45,  8.27s/it]
 43%|████▎     | 16405/37962 [38:01:00<49:22:45,  8.25s/it]
 43%|████▎     | 16406/37962 [38:01:08<48:28:53,  8.10s/it]
 43%|████▎     | 16407/37962 [38:01:16<48:42:22,  8.13s/it]
 43%|████▎     | 16408/37962 [38:01:24<48:51:46,  8.16s/it]
 43%|████▎     | 16409/37962 [38:01:32<49:04:12,  8.20s/it]
 43%|████▎     | 16410/37962 [38:01:40<48:53:16,  8.17s/it]
                                                           
{'loss': 1.2936, 'grad_norm': 8.942983627319336, 'learning_rate': 1.2647407145446092e-05, 'epoch': 0.43}

 43%|████▎     | 16410/37962 [38:01:40<48:53:16,  8.17s/it]
 43%|████▎     | 16411/37962 [38:01:49<49:03:08,  8.19s/it]
 43%|████▎     | 16412/37962 [38:01:57<49:13:35,  8.22s/it]
 43%|████▎     | 16413/37962 [38:02:05<49:26:10,  8.26s/it]
 43%|████▎     | 16414/37962 [38:02:13<49:15:57,  8.23s/it]
 43%|████▎     | 16415/37962 [38:02:21<48:51:27,  8.16s/it]
 43%|████▎     | 16416/37962 [38:02:30<48:37:37,  8.12s/it]
 43%|████▎     | 16417/37962 [38:02:38<49:35:17,  8.29s/it]
 43%|████▎     | 16418/37962 [38:02:47<49:41:01,  8.30s/it]
 43%|████▎     | 16419/37962 [38:02:55<49:34:04,  8.28s/it]
 43%|████▎     | 16420/37962 [38:03:03<49:43:04,  8.31s/it]
                                                           
{'loss': 1.3445, 'grad_norm': 6.198161602020264, 'learning_rate': 1.2639178989412356e-05, 'epoch': 0.43}

 43%|████▎     | 16420/37962 [38:03:03<49:43:04,  8.31s/it]
 43%|████▎     | 16421/37962 [38:03:11<49:40:28,  8.30s/it]
 43%|████▎     | 16422/37962 [38:03:20<49:38:14,  8.30s/it]
 43%|████▎     | 16423/37962 [38:03:28<49:49:16,  8.33s/it]
 43%|████▎     | 16424/37962 [38:03:36<49:54:17,  8.34s/it]
 43%|████▎     | 16425/37962 [38:03:45<49:50:15,  8.33s/it]
 43%|████▎     | 16426/37962 [38:03:53<49:38:36,  8.30s/it]
 43%|████▎     | 16427/37962 [38:04:01<49:37:16,  8.30s/it]
 43%|████▎     | 16428/37962 [38:04:09<49:22:11,  8.25s/it]
 43%|████▎     | 16429/37962 [38:04:18<49:30:31,  8.28s/it]
 43%|████▎     | 16430/37962 [38:04:26<49:11:35,  8.22s/it]
                                                           
{'loss': 1.3899, 'grad_norm': 2.4878838062286377, 'learning_rate': 1.263094891236595e-05, 'epoch': 0.43}

 43%|████▎     | 16430/37962 [38:04:26<49:11:35,  8.22s/it]
 43%|████▎     | 16431/37962 [38:04:34<49:52:16,  8.34s/it]
 43%|████▎     | 16432/37962 [38:04:43<49:23:01,  8.26s/it]
 43%|████▎     | 16433/37962 [38:04:50<48:44:07,  8.15s/it]
 43%|████▎     | 16434/37962 [38:04:59<48:52:23,  8.17s/it]
 43%|████▎     | 16435/37962 [38:05:07<49:10:40,  8.22s/it]
 43%|████▎     | 16436/37962 [38:05:15<49:14:09,  8.23s/it]
 43%|████▎     | 16437/37962 [38:05:23<48:53:28,  8.18s/it]
 43%|████▎     | 16438/37962 [38:05:31<48:25:25,  8.10s/it]
 43%|████▎     | 16439/37962 [38:05:39<48:35:17,  8.13s/it]
 43%|████▎     | 16440/37962 [38:05:48<48:34:46,  8.13s/it]
                                                           
{'loss': 1.4104, 'grad_norm': 6.758609294891357, 'learning_rate': 1.2622716920297413e-05, 'epoch': 0.43}

 43%|████▎     | 16440/37962 [38:05:48<48:34:46,  8.13s/it]
 43%|████▎     | 16441/37962 [38:05:55<48:04:43,  8.04s/it]
 43%|████▎     | 16442/37962 [38:06:04<48:27:50,  8.11s/it]
 43%|████▎     | 16443/37962 [38:06:12<48:24:08,  8.10s/it]
 43%|████▎     | 16444/37962 [38:06:20<49:08:51,  8.22s/it]
 43%|████▎     | 16445/37962 [38:06:29<49:12:58,  8.23s/it]
 43%|████▎     | 16446/37962 [38:06:37<49:28:48,  8.28s/it]
 43%|████▎     | 16447/37962 [38:06:45<49:35:19,  8.30s/it]
 43%|████▎     | 16448/37962 [38:06:53<49:10:52,  8.23s/it]
 43%|████▎     | 16449/37962 [38:07:02<49:32:25,  8.29s/it]
 43%|████▎     | 16450/37962 [38:07:10<49:30:23,  8.28s/it]
                                                           
{'loss': 1.3305, 'grad_norm': 8.211750984191895, 'learning_rate': 1.2614483019198667e-05, 'epoch': 0.43}

 43%|████▎     | 16450/37962 [38:07:10<49:30:23,  8.28s/it]
 43%|████▎     | 16451/37962 [38:07:18<49:07:02,  8.22s/it]
 43%|████▎     | 16452/37962 [38:07:26<48:51:23,  8.18s/it]
 43%|████▎     | 16453/37962 [38:07:35<49:31:53,  8.29s/it]
 43%|████▎     | 16454/37962 [38:07:43<49:33:26,  8.29s/it]
 43%|████▎     | 16455/37962 [38:07:51<49:27:29,  8.28s/it]
 43%|████▎     | 16456/37962 [38:07:59<49:20:11,  8.26s/it]
 43%|████▎     | 16457/37962 [38:08:08<49:30:07,  8.29s/it]
 43%|████▎     | 16458/37962 [38:08:16<49:40:38,  8.32s/it]
 43%|████▎     | 16459/37962 [38:08:25<49:50:18,  8.34s/it]
 43%|████▎     | 16460/37962 [38:08:33<49:40:20,  8.32s/it]
                                                           
{'loss': 1.4101, 'grad_norm': 18.20709991455078, 'learning_rate': 1.2606247215063019e-05, 'epoch': 0.43}

 43%|████▎     | 16460/37962 [38:08:33<49:40:20,  8.32s/it]
 43%|████▎     | 16461/37962 [38:08:41<49:40:42,  8.32s/it]
 43%|████▎     | 16462/37962 [38:08:49<49:28:23,  8.28s/it]
 43%|████▎     | 16463/37962 [38:08:57<48:42:47,  8.16s/it]
 43%|████▎     | 16464/37962 [38:09:05<48:50:06,  8.18s/it]
 43%|████▎     | 16465/37962 [38:09:14<49:19:19,  8.26s/it]
 43%|████▎     | 16466/37962 [38:09:22<48:39:37,  8.15s/it]
 43%|████▎     | 16467/37962 [38:09:30<48:58:35,  8.20s/it]
 43%|████▎     | 16468/37962 [38:09:38<48:31:53,  8.13s/it]
 43%|████▎     | 16469/37962 [38:09:46<48:56:54,  8.20s/it]
 43%|████▎     | 16470/37962 [38:09:55<48:58:18,  8.20s/it]
                                                           
{'loss': 1.4965, 'grad_norm': 2.9763343334198, 'learning_rate': 1.2598009513885174e-05, 'epoch': 0.43}

 43%|████▎     | 16470/37962 [38:09:55<48:58:18,  8.20s/it]
 43%|████▎     | 16471/37962 [38:10:03<49:00:03,  8.21s/it]
 43%|████▎     | 16472/37962 [38:10:12<49:54:35,  8.36s/it]
 43%|████▎     | 16473/37962 [38:10:20<50:05:47,  8.39s/it]
 43%|████▎     | 16474/37962 [38:10:28<49:48:31,  8.34s/it]
 43%|████▎     | 16475/37962 [38:10:37<49:40:28,  8.32s/it]
 43%|████▎     | 16476/37962 [38:10:45<49:30:12,  8.29s/it]
 43%|████▎     | 16477/37962 [38:10:53<49:25:13,  8.28s/it]
 43%|████▎     | 16478/37962 [38:11:01<49:16:42,  8.26s/it]
 43%|████▎     | 16479/37962 [38:11:10<49:31:14,  8.30s/it]
 43%|████▎     | 16480/37962 [38:11:18<49:18:17,  8.26s/it]
                                                           
{'loss': 1.2799, 'grad_norm': 7.870472431182861, 'learning_rate': 1.2589769921661215e-05, 'epoch': 0.43}

 43%|████▎     | 16480/37962 [38:11:18<49:18:17,  8.26s/it]
 43%|████▎     | 16481/37962 [38:11:26<49:10:35,  8.24s/it]
 43%|████▎     | 16482/37962 [38:11:34<49:26:51,  8.29s/it]
 43%|████▎     | 16483/37962 [38:11:42<48:42:10,  8.16s/it]
 43%|████▎     | 16484/37962 [38:11:50<48:25:48,  8.12s/it]
 43%|████▎     | 16485/37962 [38:11:59<48:44:53,  8.17s/it]
 43%|████▎     | 16486/37962 [38:12:07<49:10:36,  8.24s/it]
 43%|████▎     | 16487/37962 [38:12:15<49:05:32,  8.23s/it]
 43%|████▎     | 16488/37962 [38:12:23<49:08:13,  8.24s/it]
 43%|████▎     | 16489/37962 [38:12:32<48:55:20,  8.20s/it]
 43%|████▎     | 16490/37962 [38:12:40<49:10:12,  8.24s/it]
                                                           
{'loss': 1.49, 'grad_norm': 7.063776016235352, 'learning_rate': 1.2581528444388593e-05, 'epoch': 0.43}

 43%|████▎     | 16490/37962 [38:12:40<49:10:12,  8.24s/it]
 43%|████▎     | 16491/37962 [38:12:48<48:59:46,  8.22s/it]
 43%|████▎     | 16492/37962 [38:12:56<48:54:05,  8.20s/it]
 43%|████▎     | 16493/37962 [38:13:05<49:09:00,  8.24s/it]
 43%|████▎     | 16494/37962 [38:13:13<48:52:02,  8.19s/it]
 43%|████▎     | 16495/37962 [38:13:21<49:08:08,  8.24s/it]
 43%|████▎     | 16496/37962 [38:13:29<49:09:17,  8.24s/it]
 43%|████▎     | 16497/37962 [38:13:38<49:18:58,  8.27s/it]
 43%|████▎     | 16498/37962 [38:13:46<49:29:49,  8.30s/it]
 43%|████▎     | 16499/37962 [38:13:54<48:59:25,  8.22s/it]
 43%|████▎     | 16500/37962 [38:14:02<49:27:34,  8.30s/it]
                                                           
{'loss': 1.4383, 'grad_norm': 7.303397178649902, 'learning_rate': 1.2573285088066137e-05, 'epoch': 0.43}

 43%|████▎     | 16500/37962 [38:14:02<49:27:34,  8.30s/it]
 43%|████▎     | 16501/37962 [38:14:11<49:28:40,  8.30s/it]
 43%|████▎     | 16502/37962 [38:14:19<49:04:29,  8.23s/it]
 43%|████▎     | 16503/37962 [38:14:27<49:08:13,  8.24s/it]
 43%|████▎     | 16504/37962 [38:14:36<49:44:43,  8.35s/it]
 43%|████▎     | 16505/37962 [38:14:44<49:29:48,  8.30s/it]
 43%|████▎     | 16506/37962 [38:14:52<49:19:17,  8.28s/it]
 43%|████▎     | 16507/37962 [38:15:00<49:22:56,  8.29s/it]
 43%|████▎     | 16508/37962 [38:15:09<49:24:15,  8.29s/it]
 43%|████▎     | 16509/37962 [38:15:17<48:58:59,  8.22s/it]
 43%|████▎     | 16510/37962 [38:15:25<49:02:36,  8.23s/it]
                                                           
{'loss': 1.3957, 'grad_norm': 82.07682800292969, 'learning_rate': 1.2565039858694044e-05, 'epoch': 0.43}

 43%|████▎     | 16510/37962 [38:15:25<49:02:36,  8.23s/it]
 43%|████▎     | 16511/37962 [38:15:33<49:19:21,  8.28s/it]
 43%|████▎     | 16512/37962 [38:15:42<49:10:16,  8.25s/it]
 43%|████▎     | 16513/37962 [38:15:50<49:12:04,  8.26s/it]
 44%|████▎     | 16514/37962 [38:15:58<49:04:15,  8.24s/it]
 44%|████▎     | 16515/37962 [38:16:07<49:28:07,  8.30s/it]
 44%|████▎     | 16516/37962 [38:16:15<49:32:16,  8.32s/it]
 44%|████▎     | 16517/37962 [38:16:23<49:28:08,  8.30s/it]
 44%|████▎     | 16518/37962 [38:16:32<49:47:25,  8.36s/it]
 44%|████▎     | 16519/37962 [38:16:40<49:07:21,  8.25s/it]
 44%|████▎     | 16520/37962 [38:16:48<49:38:20,  8.33s/it]
                                                           
{'loss': 1.3496, 'grad_norm': 2.4122869968414307, 'learning_rate': 1.2556792762273875e-05, 'epoch': 0.44}

 44%|████▎     | 16520/37962 [38:16:48<49:38:20,  8.33s/it]
 44%|████▎     | 16521/37962 [38:16:56<48:33:20,  8.15s/it]
 44%|████▎     | 16522/37962 [38:17:04<48:42:42,  8.18s/it]
 44%|████▎     | 16523/37962 [38:17:12<48:32:26,  8.15s/it]
 44%|████▎     | 16524/37962 [38:17:20<48:43:04,  8.18s/it]
 44%|████▎     | 16525/37962 [38:17:29<48:58:35,  8.22s/it]
 44%|████▎     | 16526/37962 [38:17:37<48:39:11,  8.17s/it]
 44%|████▎     | 16527/37962 [38:17:45<48:21:36,  8.12s/it]
 44%|████▎     | 16528/37962 [38:17:53<48:41:28,  8.18s/it]
 44%|████▎     | 16529/37962 [38:18:02<48:59:18,  8.23s/it]
 44%|████▎     | 16530/37962 [38:18:10<49:12:48,  8.27s/it]
                                                           
{'loss': 1.3585, 'grad_norm': 5.706669330596924, 'learning_rate': 1.2548543804808549e-05, 'epoch': 0.44}

 44%|████▎     | 16530/37962 [38:18:10<49:12:48,  8.27s/it]
 44%|████▎     | 16531/37962 [38:18:18<49:16:27,  8.28s/it]
 44%|████▎     | 16532/37962 [38:18:26<49:20:46,  8.29s/it]
 44%|████▎     | 16533/37962 [38:18:35<49:27:52,  8.31s/it]
 44%|████▎     | 16534/37962 [38:18:43<49:34:12,  8.33s/it]
 44%|████▎     | 16535/37962 [38:18:51<49:22:13,  8.29s/it]
 44%|████▎     | 16536/37962 [38:19:00<49:29:15,  8.31s/it]
 44%|████▎     | 16537/37962 [38:19:08<49:54:00,  8.38s/it]
 44%|████▎     | 16538/37962 [38:19:17<50:04:53,  8.42s/it]
 44%|████▎     | 16539/37962 [38:19:25<49:41:36,  8.35s/it]
 44%|████▎     | 16540/37962 [38:19:33<49:34:47,  8.33s/it]
                                                           
{'loss': 1.2756, 'grad_norm': 6.834168434143066, 'learning_rate': 1.2540292992302336e-05, 'epoch': 0.44}

 44%|████▎     | 16540/37962 [38:19:33<49:34:47,  8.33s/it]
 44%|████▎     | 16541/37962 [38:19:42<49:47:00,  8.37s/it]
 44%|████▎     | 16542/37962 [38:19:50<49:47:42,  8.37s/it]
 44%|████▎     | 16543/37962 [38:19:58<49:35:12,  8.33s/it]
 44%|████▎     | 16544/37962 [38:20:07<49:24:19,  8.30s/it]
 44%|████▎     | 16545/37962 [38:20:15<49:10:03,  8.26s/it]
 44%|████▎     | 16546/37962 [38:20:23<49:13:52,  8.28s/it]
 44%|████▎     | 16547/37962 [38:20:31<49:01:44,  8.24s/it]
 44%|████▎     | 16548/37962 [38:20:40<49:14:18,  8.28s/it]
 44%|████▎     | 16549/37962 [38:20:48<49:29:38,  8.32s/it]
 44%|████▎     | 16550/37962 [38:20:56<49:21:51,  8.30s/it]
                                                           
{'loss': 1.264, 'grad_norm': 8.142898559570312, 'learning_rate': 1.2532040330760863e-05, 'epoch': 0.44}

 44%|████▎     | 16550/37962 [38:20:56<49:21:51,  8.30s/it]
 44%|████▎     | 16551/37962 [38:21:05<49:15:47,  8.28s/it]
 44%|████▎     | 16552/37962 [38:21:13<48:47:02,  8.20s/it]
 44%|████▎     | 16553/37962 [38:21:21<49:02:05,  8.25s/it]
 44%|████▎     | 16554/37962 [38:21:29<48:57:03,  8.23s/it]
 44%|████▎     | 16555/37962 [38:21:37<48:59:36,  8.24s/it]
 44%|████▎     | 16556/37962 [38:21:46<48:53:56,  8.22s/it]
 44%|████▎     | 16557/37962 [38:21:54<49:03:43,  8.25s/it]
 44%|████▎     | 16558/37962 [38:22:02<49:32:12,  8.33s/it]
 44%|████▎     | 16559/37962 [38:22:11<49:24:48,  8.31s/it]
 44%|████▎     | 16560/37962 [38:22:19<49:36:30,  8.34s/it]
                                                           
{'loss': 1.3304, 'grad_norm': 3.8998258113861084, 'learning_rate': 1.25237858261911e-05, 'epoch': 0.44}

 44%|████▎     | 16560/37962 [38:22:19<49:36:30,  8.34s/it]
 44%|████▎     | 16561/37962 [38:22:27<49:27:50,  8.32s/it]
 44%|████▎     | 16562/37962 [38:22:36<50:04:13,  8.42s/it]
 44%|████▎     | 16563/37962 [38:22:44<49:23:52,  8.31s/it]
 44%|████▎     | 16564/37962 [38:22:52<49:19:01,  8.30s/it]
 44%|████▎     | 16565/37962 [38:23:00<48:45:07,  8.20s/it]
 44%|████▎     | 16566/37962 [38:23:09<49:25:26,  8.32s/it]
 44%|████▎     | 16567/37962 [38:23:17<49:19:13,  8.30s/it]
 44%|████▎     | 16568/37962 [38:23:25<49:00:09,  8.25s/it]
 44%|████▎     | 16569/37962 [38:23:33<48:55:24,  8.23s/it]
 44%|████▎     | 16570/37962 [38:23:42<49:04:09,  8.26s/it]
                                                           
{'loss': 1.3694, 'grad_norm': 5.435478687286377, 'learning_rate': 1.2515529484601349e-05, 'epoch': 0.44}

 44%|████▎     | 16570/37962 [38:23:42<49:04:09,  8.26s/it]
 44%|████▎     | 16571/37962 [38:23:49<48:03:51,  8.09s/it]
 44%|████▎     | 16572/37962 [38:23:57<47:57:27,  8.07s/it]
 44%|████▎     | 16573/37962 [38:24:06<48:23:15,  8.14s/it]
 44%|████▎     | 16574/37962 [38:24:14<48:55:07,  8.23s/it]
 44%|████▎     | 16575/37962 [38:24:23<49:14:13,  8.29s/it]
 44%|████▎     | 16576/37962 [38:24:31<49:15:29,  8.29s/it]
 44%|████▎     | 16577/37962 [38:24:39<49:14:18,  8.29s/it]
 44%|████▎     | 16578/37962 [38:24:47<48:12:54,  8.12s/it]
 44%|████▎     | 16579/37962 [38:24:55<48:36:19,  8.18s/it]
 44%|████▎     | 16580/37962 [38:25:03<48:18:45,  8.13s/it]
                                                           
{'loss': 1.4248, 'grad_norm': 7.5678205490112305, 'learning_rate': 1.250727131200127e-05, 'epoch': 0.44}

 44%|████▎     | 16580/37962 [38:25:03<48:18:45,  8.13s/it]
 44%|████▎     | 16581/37962 [38:25:12<48:36:16,  8.18s/it]
 44%|████▎     | 16582/37962 [38:25:20<48:39:06,  8.19s/it]
 44%|████▎     | 16583/37962 [38:25:28<48:57:06,  8.24s/it]
 44%|████▎     | 16584/37962 [38:25:37<49:12:30,  8.29s/it]
 44%|████▎     | 16585/37962 [38:25:45<49:10:34,  8.28s/it]
 44%|████▎     | 16586/37962 [38:25:53<48:32:06,  8.17s/it]
 44%|████▎     | 16587/37962 [38:26:01<48:55:27,  8.24s/it]
 44%|████▎     | 16588/37962 [38:26:09<48:57:56,  8.25s/it]
 44%|████▎     | 16589/37962 [38:26:17<48:27:56,  8.16s/it]
 44%|████▎     | 16590/37962 [38:26:25<47:41:40,  8.03s/it]
                                                           
{'loss': 1.4421, 'grad_norm': 11.584285736083984, 'learning_rate': 1.2499011314401839e-05, 'epoch': 0.44}

 44%|████▎     | 16590/37962 [38:26:25<47:41:40,  8.03s/it]
 44%|████▎     | 16591/37962 [38:26:33<48:09:28,  8.11s/it]
 44%|████▎     | 16592/37962 [38:26:42<48:44:12,  8.21s/it]
 44%|████▎     | 16593/37962 [38:26:50<48:42:03,  8.20s/it]
 44%|████▎     | 16594/37962 [38:26:58<48:58:54,  8.25s/it]
 44%|████▎     | 16595/37962 [38:27:07<49:21:06,  8.32s/it]
 44%|████▎     | 16596/37962 [38:27:15<49:36:59,  8.36s/it]
 44%|████▎     | 16597/37962 [38:27:24<49:18:58,  8.31s/it]
 44%|████▎     | 16598/37962 [38:27:32<48:51:24,  8.23s/it]
 44%|████▎     | 16599/37962 [38:27:40<48:53:52,  8.24s/it]
 44%|████▎     | 16600/37962 [38:27:48<48:53:17,  8.24s/it]
                                                           
{'loss': 1.4728, 'grad_norm': 28.495189666748047, 'learning_rate': 1.2490749497815362e-05, 'epoch': 0.44}

 44%|████▎     | 16600/37962 [38:27:48<48:53:17,  8.24s/it]
 44%|████▎     | 16601/37962 [38:27:56<49:07:31,  8.28s/it]
 44%|████▎     | 16602/37962 [38:28:05<49:26:41,  8.33s/it]
 44%|████▎     | 16603/37962 [38:28:13<49:30:08,  8.34s/it]
 44%|████▎     | 16604/37962 [38:28:22<49:31:41,  8.35s/it]
 44%|████▎     | 16605/37962 [38:28:30<49:12:17,  8.29s/it]
 44%|████▎     | 16606/37962 [38:28:38<49:07:30,  8.28s/it]
 44%|████▎     | 16607/37962 [38:28:46<49:13:42,  8.30s/it]
 44%|████▎     | 16608/37962 [38:28:55<49:05:33,  8.28s/it]
 44%|████▍     | 16609/37962 [38:29:03<48:55:02,  8.25s/it]
 44%|████▍     | 16610/37962 [38:29:11<49:01:40,  8.27s/it]
                                                           
{'loss': 1.3512, 'grad_norm': 7.732114315032959, 'learning_rate': 1.2482485868255476e-05, 'epoch': 0.44}

 44%|████▍     | 16610/37962 [38:29:11<49:01:40,  8.27s/it]
 44%|████▍     | 16611/37962 [38:29:19<48:43:04,  8.21s/it]
 44%|████▍     | 16612/37962 [38:29:28<49:12:39,  8.30s/it]
 44%|████▍     | 16613/37962 [38:29:36<48:54:03,  8.25s/it]
 44%|████▍     | 16614/37962 [38:29:44<48:16:54,  8.14s/it]
 44%|████▍     | 16615/37962 [38:29:52<48:48:12,  8.23s/it]
 44%|████▍     | 16616/37962 [38:30:01<49:07:50,  8.29s/it]
 44%|████▍     | 16617/37962 [38:30:09<49:27:17,  8.34s/it]
 44%|████▍     | 16618/37962 [38:30:17<49:14:44,  8.31s/it]
 44%|████▍     | 16619/37962 [38:30:25<48:54:08,  8.25s/it]
 44%|████▍     | 16620/37962 [38:30:34<48:50:35,  8.24s/it]
                                                           
{'loss': 1.2919, 'grad_norm': 8.742936134338379, 'learning_rate': 1.2474220431737135e-05, 'epoch': 0.44}

 44%|████▍     | 16620/37962 [38:30:34<48:50:35,  8.24s/it]
 44%|████▍     | 16621/37962 [38:30:42<48:33:44,  8.19s/it]
 44%|████▍     | 16622/37962 [38:30:50<48:45:57,  8.23s/it]
 44%|████▍     | 16623/37962 [38:30:58<48:22:02,  8.16s/it]
 44%|████▍     | 16624/37962 [38:31:06<48:44:42,  8.22s/it]
 44%|████▍     | 16625/37962 [38:31:15<48:39:55,  8.21s/it]
 44%|████▍     | 16626/37962 [38:31:23<48:45:28,  8.23s/it]
 44%|████▍     | 16627/37962 [38:31:31<48:44:05,  8.22s/it]
 44%|████▍     | 16628/37962 [38:31:39<48:48:32,  8.24s/it]
 44%|████▍     | 16629/37962 [38:31:48<48:47:19,  8.23s/it]
 44%|████▍     | 16630/37962 [38:31:56<49:45:22,  8.40s/it]
                                                           
{'loss': 1.2377, 'grad_norm': 10.803218841552734, 'learning_rate': 1.24659531942766e-05, 'epoch': 0.44}

 44%|████▍     | 16630/37962 [38:31:56<49:45:22,  8.40s/it]
 44%|████▍     | 16631/37962 [38:32:05<49:42:47,  8.39s/it]
 44%|████▍     | 16632/37962 [38:32:13<49:26:11,  8.34s/it]
 44%|████▍     | 16633/37962 [38:32:21<49:21:19,  8.33s/it]
 44%|████▍     | 16634/37962 [38:32:30<49:17:42,  8.32s/it]
 44%|████▍     | 16635/37962 [38:32:38<49:12:32,  8.31s/it]
 44%|████▍     | 16636/37962 [38:32:46<48:53:24,  8.25s/it]
 44%|████▍     | 16637/37962 [38:32:54<48:53:50,  8.25s/it]
 44%|████▍     | 16638/37962 [38:33:02<48:33:00,  8.20s/it]
 44%|████▍     | 16639/37962 [38:33:10<48:34:22,  8.20s/it]
 44%|████▍     | 16640/37962 [38:33:19<48:48:28,  8.24s/it]
                                                           
{'loss': 1.3961, 'grad_norm': 7.6756744384765625, 'learning_rate': 1.2457684161891456e-05, 'epoch': 0.44}

 44%|████▍     | 16640/37962 [38:33:19<48:48:28,  8.24s/it]
 44%|████▍     | 16641/37962 [38:33:27<48:09:19,  8.13s/it]
 44%|████▍     | 16642/37962 [38:33:35<48:12:27,  8.14s/it]
 44%|████▍     | 16643/37962 [38:33:43<48:23:33,  8.17s/it]
 44%|████▍     | 16644/37962 [38:33:51<48:15:54,  8.15s/it]
 44%|████▍     | 16645/37962 [38:34:00<48:35:51,  8.21s/it]
 44%|████▍     | 16646/37962 [38:34:08<48:59:57,  8.28s/it]
 44%|████▍     | 16647/37962 [38:34:16<49:10:50,  8.31s/it]
 44%|████▍     | 16648/37962 [38:34:24<48:47:45,  8.24s/it]
 44%|████▍     | 16649/37962 [38:34:33<49:21:06,  8.34s/it]
 44%|████▍     | 16650/37962 [38:34:41<48:50:06,  8.25s/it]
                                                           
{'loss': 1.3993, 'grad_norm': 5.053336143493652, 'learning_rate': 1.2449413340600586e-05, 'epoch': 0.44}

 44%|████▍     | 16650/37962 [38:34:41<48:50:06,  8.25s/it]
 44%|████▍     | 16651/37962 [38:34:49<48:44:55,  8.23s/it]
 44%|████▍     | 16652/37962 [38:34:58<48:57:55,  8.27s/it]
 44%|████▍     | 16653/37962 [38:35:06<49:07:13,  8.30s/it]
 44%|████▍     | 16654/37962 [38:35:14<48:59:23,  8.28s/it]
 44%|████▍     | 16655/37962 [38:35:22<48:59:08,  8.28s/it]
 44%|████▍     | 16656/37962 [38:35:31<48:55:19,  8.27s/it]
 44%|████▍     | 16657/37962 [38:35:39<49:17:53,  8.33s/it]
 44%|████▍     | 16658/37962 [38:35:48<49:29:18,  8.36s/it]
 44%|████▍     | 16659/37962 [38:35:56<49:20:13,  8.34s/it]
 44%|████▍     | 16660/37962 [38:36:04<49:07:12,  8.30s/it]
                                                           
{'loss': 1.4214, 'grad_norm': 15.580668449401855, 'learning_rate': 1.2441140736424185e-05, 'epoch': 0.44}

 44%|████▍     | 16660/37962 [38:36:04<49:07:12,  8.30s/it]
 44%|████▍     | 16661/37962 [38:36:12<48:20:20,  8.17s/it]
 44%|████▍     | 16662/37962 [38:36:20<48:39:47,  8.22s/it]
 44%|████▍     | 16663/37962 [38:36:28<48:08:15,  8.14s/it]
 44%|████▍     | 16664/37962 [38:36:37<48:25:16,  8.18s/it]
 44%|████▍     | 16665/37962 [38:36:45<48:37:29,  8.22s/it]
 44%|████▍     | 16666/37962 [38:36:53<49:08:02,  8.31s/it]
 44%|████▍     | 16667/37962 [38:37:02<48:54:26,  8.27s/it]
 44%|████▍     | 16668/37962 [38:37:10<48:51:58,  8.26s/it]
 44%|████▍     | 16669/37962 [38:37:18<48:52:41,  8.26s/it]
 44%|████▍     | 16670/37962 [38:37:26<48:49:35,  8.26s/it]
                                                           
{'loss': 1.3514, 'grad_norm': 10.165223121643066, 'learning_rate': 1.2432866355383723e-05, 'epoch': 0.44}

 44%|████▍     | 16670/37962 [38:37:26<48:49:35,  8.26s/it]
 44%|████▍     | 16671/37962 [38:37:35<48:55:12,  8.27s/it]
 44%|████▍     | 16672/37962 [38:37:43<49:01:13,  8.29s/it]
 44%|████▍     | 16673/37962 [38:37:51<48:56:17,  8.28s/it]
 44%|████▍     | 16674/37962 [38:37:59<48:13:23,  8.15s/it]
 44%|████▍     | 16675/37962 [38:38:07<48:33:47,  8.21s/it]
 44%|████▍     | 16676/37962 [38:38:16<48:37:58,  8.23s/it]
 44%|████▍     | 16677/37962 [38:38:24<49:03:30,  8.30s/it]
 44%|████▍     | 16678/37962 [38:38:32<49:02:56,  8.30s/it]
 44%|████▍     | 16679/37962 [38:38:41<48:54:45,  8.27s/it]
 44%|████▍     | 16680/37962 [38:38:49<48:33:03,  8.21s/it]
                                                           
{'loss': 1.3138, 'grad_norm': 11.372960090637207, 'learning_rate': 1.2424590203501992e-05, 'epoch': 0.44}

 44%|████▍     | 16680/37962 [38:38:49<48:33:03,  8.21s/it]
 44%|████▍     | 16681/37962 [38:38:57<48:37:34,  8.23s/it]
 44%|████▍     | 16682/37962 [38:39:05<48:56:01,  8.28s/it]
 44%|████▍     | 16683/37962 [38:39:14<49:13:04,  8.33s/it]
 44%|████▍     | 16684/37962 [38:39:22<49:40:30,  8.40s/it]
 44%|████▍     | 16685/37962 [38:39:31<49:19:55,  8.35s/it]
 44%|████▍     | 16686/37962 [38:39:39<48:55:21,  8.28s/it]
 44%|████▍     | 16687/37962 [38:39:47<48:48:09,  8.26s/it]
 44%|████▍     | 16688/37962 [38:39:55<48:54:54,  8.28s/it]
 44%|████▍     | 16689/37962 [38:40:04<48:57:27,  8.29s/it]
 44%|████▍     | 16690/37962 [38:40:12<48:37:50,  8.23s/it]
                                                           
{'loss': 1.3031, 'grad_norm': 9.802559852600098, 'learning_rate': 1.2416312286803057e-05, 'epoch': 0.44}

 44%|████▍     | 16690/37962 [38:40:12<48:37:50,  8.23s/it]
 44%|████▍     | 16691/37962 [38:40:20<49:19:06,  8.35s/it]
 44%|████▍     | 16692/37962 [38:40:28<48:51:25,  8.27s/it]
 44%|████▍     | 16693/37962 [38:40:37<49:08:30,  8.32s/it]
 44%|████▍     | 16694/37962 [38:40:45<48:32:24,  8.22s/it]
 44%|████▍     | 16695/37962 [38:40:53<48:46:38,  8.26s/it]
 44%|████▍     | 16696/37962 [38:41:01<49:00:55,  8.30s/it]
 44%|████▍     | 16697/37962 [38:41:10<48:38:05,  8.23s/it]
 44%|████▍     | 16698/37962 [38:41:18<48:36:40,  8.23s/it]
 44%|████▍     | 16699/37962 [38:41:26<48:40:03,  8.24s/it]
 44%|████▍     | 16700/37962 [38:41:34<48:55:37,  8.28s/it]
                                                           
{'loss': 1.2657, 'grad_norm': 17.825084686279297, 'learning_rate': 1.2408032611312267e-05, 'epoch': 0.44}

 44%|████▍     | 16700/37962 [38:41:34<48:55:37,  8.28s/it]
 44%|████▍     | 16701/37962 [38:41:43<48:34:12,  8.22s/it]
 44%|████▍     | 16702/37962 [38:41:50<47:35:56,  8.06s/it]
 44%|████▍     | 16703/37962 [38:41:58<47:55:40,  8.12s/it]
 44%|████▍     | 16704/37962 [38:42:07<48:27:54,  8.21s/it]
 44%|████▍     | 16705/37962 [38:42:15<48:45:35,  8.26s/it]
 44%|████▍     | 16706/37962 [38:42:23<48:25:47,  8.20s/it]
 44%|████▍     | 16707/37962 [38:42:32<48:41:44,  8.25s/it]
 44%|████▍     | 16708/37962 [38:42:40<48:50:40,  8.27s/it]
 44%|████▍     | 16709/37962 [38:42:48<48:59:06,  8.30s/it]
 44%|████▍     | 16710/37962 [38:42:57<49:00:14,  8.30s/it]
                                                           
{'loss': 1.3587, 'grad_norm': 12.653438568115234, 'learning_rate': 1.239975118305625e-05, 'epoch': 0.44}

 44%|████▍     | 16710/37962 [38:42:57<49:00:14,  8.30s/it]
 44%|████▍     | 16711/37962 [38:43:05<48:59:42,  8.30s/it]
 44%|████▍     | 16712/37962 [38:43:13<48:47:57,  8.27s/it]
 44%|████▍     | 16713/37962 [38:43:21<48:21:05,  8.19s/it]
 44%|████▍     | 16714/37962 [38:43:30<48:35:09,  8.23s/it]
 44%|████▍     | 16715/37962 [38:43:38<48:22:49,  8.20s/it]
 44%|████▍     | 16716/37962 [38:43:46<48:20:31,  8.19s/it]
 44%|████▍     | 16717/37962 [38:43:54<48:23:10,  8.20s/it]
 44%|████▍     | 16718/37962 [38:44:02<48:12:26,  8.17s/it]
 44%|████▍     | 16719/37962 [38:44:10<48:33:31,  8.23s/it]
 44%|████▍     | 16720/37962 [38:44:19<48:16:36,  8.18s/it]
                                                           
{'loss': 1.4649, 'grad_norm': 100.38288879394531, 'learning_rate': 1.2391468008062926e-05, 'epoch': 0.44}

 44%|████▍     | 16720/37962 [38:44:19<48:16:36,  8.18s/it]
 44%|████▍     | 16721/37962 [38:44:27<48:26:19,  8.21s/it]
 44%|████▍     | 16722/37962 [38:44:35<48:38:52,  8.25s/it]
 44%|████▍     | 16723/37962 [38:44:43<48:26:56,  8.21s/it]
 44%|████▍     | 16724/37962 [38:44:51<47:42:56,  8.09s/it]
 44%|████▍     | 16725/37962 [38:44:59<47:51:31,  8.11s/it]
 44%|████▍     | 16726/37962 [38:45:08<48:31:21,  8.23s/it]
 44%|████▍     | 16727/37962 [38:45:16<48:37:29,  8.24s/it]
 44%|████▍     | 16728/37962 [38:45:24<48:42:49,  8.26s/it]
 44%|████▍     | 16729/37962 [38:45:33<49:00:33,  8.31s/it]
 44%|████▍     | 16730/37962 [38:45:41<49:02:09,  8.31s/it]
                                                           
{'loss': 1.3451, 'grad_norm': 8.964167594909668, 'learning_rate': 1.2383183092361466e-05, 'epoch': 0.44}

 44%|████▍     | 16730/37962 [38:45:41<49:02:09,  8.31s/it]
 44%|████▍     | 16731/37962 [38:45:49<49:10:26,  8.34s/it]
 44%|████▍     | 16732/37962 [38:45:58<49:10:06,  8.34s/it]
 44%|████▍     | 16733/37962 [38:46:06<49:01:40,  8.31s/it]
 44%|████▍     | 16734/37962 [38:46:14<48:55:51,  8.30s/it]
 44%|████▍     | 16735/37962 [38:46:22<48:30:24,  8.23s/it]
 44%|████▍     | 16736/37962 [38:46:31<48:41:15,  8.26s/it]
 44%|████▍     | 16737/37962 [38:46:39<48:46:53,  8.27s/it]
 44%|████▍     | 16738/37962 [38:46:48<49:08:20,  8.33s/it]
 44%|████▍     | 16739/37962 [38:46:56<49:23:23,  8.38s/it]
 44%|████▍     | 16740/37962 [38:47:04<49:18:33,  8.36s/it]
                                                           
{'loss': 1.4573, 'grad_norm': 9.696819305419922, 'learning_rate': 1.2374896441982319e-05, 'epoch': 0.44}

 44%|████▍     | 16740/37962 [38:47:04<49:18:33,  8.36s/it]
 44%|████▍     | 16741/37962 [38:47:13<49:22:03,  8.37s/it]
 44%|████▍     | 16742/37962 [38:47:21<48:51:38,  8.29s/it]
 44%|████▍     | 16743/37962 [38:47:29<49:00:14,  8.31s/it]
 44%|████▍     | 16744/37962 [38:47:38<49:21:23,  8.37s/it]
 44%|████▍     | 16745/37962 [38:47:46<49:06:10,  8.33s/it]
 44%|████▍     | 16746/37962 [38:47:54<48:23:57,  8.21s/it]
 44%|████▍     | 16747/37962 [38:48:02<48:16:03,  8.19s/it]
 44%|████▍     | 16748/37962 [38:48:10<48:20:08,  8.20s/it]
 44%|████▍     | 16749/37962 [38:48:18<48:21:38,  8.21s/it]
 44%|████▍     | 16750/37962 [38:48:27<48:25:23,  8.22s/it]
                                                           
{'loss': 1.3852, 'grad_norm': 5.018054962158203, 'learning_rate': 1.2366608062957196e-05, 'epoch': 0.44}

 44%|████▍     | 16750/37962 [38:48:27<48:25:23,  8.22s/it]
 44%|████▍     | 16751/37962 [38:48:35<49:11:10,  8.35s/it]
 44%|████▍     | 16752/37962 [38:48:44<49:03:06,  8.33s/it]
 44%|████▍     | 16753/37962 [38:48:52<48:39:37,  8.26s/it]
 44%|████▍     | 16754/37962 [38:49:00<48:40:23,  8.26s/it]
 44%|████▍     | 16755/37962 [38:49:08<48:37:46,  8.26s/it]
 44%|████▍     | 16756/37962 [38:49:17<48:52:58,  8.30s/it]
 44%|████▍     | 16757/37962 [38:49:25<48:24:46,  8.22s/it]
 44%|████▍     | 16758/37962 [38:49:33<48:38:48,  8.26s/it]
 44%|████▍     | 16759/37962 [38:49:41<48:41:32,  8.27s/it]
 44%|████▍     | 16760/37962 [38:49:50<48:57:41,  8.31s/it]
                                                           
{'loss': 1.375, 'grad_norm': 12.65772819519043, 'learning_rate': 1.2358317961319059e-05, 'epoch': 0.44}

 44%|████▍     | 16760/37962 [38:49:50<48:57:41,  8.31s/it]
 44%|████▍     | 16761/37962 [38:49:58<48:51:45,  8.30s/it]
 44%|████▍     | 16762/37962 [38:50:06<48:46:34,  8.28s/it]
 44%|████▍     | 16763/37962 [38:50:14<48:36:48,  8.26s/it]
 44%|████▍     | 16764/37962 [38:50:23<48:57:15,  8.31s/it]
 44%|████▍     | 16765/37962 [38:50:31<48:54:48,  8.31s/it]
 44%|████▍     | 16766/37962 [38:50:40<49:22:23,  8.39s/it]
 44%|████▍     | 16767/37962 [38:50:48<49:16:46,  8.37s/it]
 44%|████▍     | 16768/37962 [38:50:56<49:12:38,  8.36s/it]
 44%|████▍     | 16769/37962 [38:51:05<49:25:18,  8.40s/it]
 44%|████▍     | 16770/37962 [38:51:13<49:33:54,  8.42s/it]
                                                           
{'loss': 1.4334, 'grad_norm': 139.33087158203125, 'learning_rate': 1.2350026143102134e-05, 'epoch': 0.44}

 44%|████▍     | 16770/37962 [38:51:13<49:33:54,  8.42s/it]
 44%|████▍     | 16771/37962 [38:51:22<49:10:00,  8.35s/it]
 44%|████▍     | 16772/37962 [38:51:30<49:05:25,  8.34s/it]
 44%|████▍     | 16773/37962 [38:51:38<49:13:21,  8.36s/it]
 44%|████▍     | 16774/37962 [38:51:47<49:15:02,  8.37s/it]
 44%|████▍     | 16775/37962 [38:51:55<49:06:02,  8.34s/it]
 44%|████▍     | 16776/37962 [38:52:03<49:00:48,  8.33s/it]
 44%|████▍     | 16777/37962 [38:52:12<49:02:55,  8.33s/it]
 44%|████▍     | 16778/37962 [38:52:20<49:00:47,  8.33s/it]
 44%|████▍     | 16779/37962 [38:52:28<48:51:23,  8.30s/it]
 44%|████▍     | 16780/37962 [38:52:36<48:20:47,  8.22s/it]
                                                           
{'loss': 1.4002, 'grad_norm': 17.72840118408203, 'learning_rate': 1.2341732614341892e-05, 'epoch': 0.44}

 44%|████▍     | 16780/37962 [38:52:36<48:20:47,  8.22s/it]
 44%|████▍     | 16781/37962 [38:52:45<48:35:17,  8.26s/it]
 44%|████▍     | 16782/37962 [38:52:53<49:00:53,  8.33s/it]
 44%|████▍     | 16783/37962 [38:53:01<49:14:56,  8.37s/it]
 44%|████▍     | 16784/37962 [38:53:10<49:01:38,  8.33s/it]
 44%|████▍     | 16785/37962 [38:53:18<48:58:41,  8.33s/it]
 44%|████▍     | 16786/37962 [38:53:26<48:25:44,  8.23s/it]
 44%|████▍     | 16787/37962 [38:53:34<48:15:57,  8.21s/it]
 44%|████▍     | 16788/37962 [38:53:42<47:33:57,  8.09s/it]
 44%|████▍     | 16789/37962 [38:53:50<47:56:16,  8.15s/it]
 44%|████▍     | 16790/37962 [38:53:59<48:01:36,  8.17s/it]
                                                           
{'loss': 1.3396, 'grad_norm': 7.262829303741455, 'learning_rate': 1.2333437381075047e-05, 'epoch': 0.44}

 44%|████▍     | 16790/37962 [38:53:59<48:01:36,  8.17s/it]
 44%|████▍     | 16791/37962 [38:54:07<48:27:44,  8.24s/it]
 44%|████▍     | 16792/37962 [38:54:15<48:30:52,  8.25s/it]
 44%|████▍     | 16793/37962 [38:54:23<48:23:14,  8.23s/it]
 44%|████▍     | 16794/37962 [38:54:32<48:39:42,  8.28s/it]
 44%|████▍     | 16795/37962 [38:54:40<48:36:44,  8.27s/it]
 44%|████▍     | 16796/37962 [38:54:48<48:16:44,  8.21s/it]
 44%|████▍     | 16797/37962 [38:54:56<48:25:52,  8.24s/it]
 44%|████▍     | 16798/37962 [38:55:05<48:30:17,  8.25s/it]
 44%|████▍     | 16799/37962 [38:55:13<48:21:44,  8.23s/it]
 44%|████▍     | 16800/37962 [38:55:21<48:30:50,  8.25s/it]
                                                           
{'loss': 1.412, 'grad_norm': 12.905970573425293, 'learning_rate': 1.2325140449339558e-05, 'epoch': 0.44}

 44%|████▍     | 16800/37962 [38:55:21<48:30:50,  8.25s/it]
 44%|████▍     | 16801/37962 [38:55:29<48:26:25,  8.24s/it]
 44%|████▍     | 16802/37962 [38:55:38<48:41:58,  8.29s/it]
 44%|████▍     | 16803/37962 [38:55:46<48:49:55,  8.31s/it]
 44%|████▍     | 16804/37962 [38:55:54<48:34:44,  8.27s/it]
 44%|████▍     | 16805/37962 [38:56:03<48:34:59,  8.27s/it]
 44%|████▍     | 16806/37962 [38:56:11<48:53:52,  8.32s/it]
 44%|████▍     | 16807/37962 [38:56:19<48:41:10,  8.29s/it]
 44%|████▍     | 16808/37962 [38:56:27<48:21:19,  8.23s/it]
 44%|████▍     | 16809/37962 [38:56:36<48:50:22,  8.31s/it]
 44%|████▍     | 16810/37962 [38:56:44<48:39:16,  8.28s/it]
                                                           
{'loss': 1.3665, 'grad_norm': 5.6244635581970215, 'learning_rate': 1.2316841825174614e-05, 'epoch': 0.44}

 44%|████▍     | 16810/37962 [38:56:44<48:39:16,  8.28s/it]
 44%|████▍     | 16811/37962 [38:56:52<48:45:34,  8.30s/it]
 44%|████▍     | 16812/37962 [38:57:00<48:22:18,  8.23s/it]
 44%|████▍     | 16813/37962 [38:57:09<48:46:30,  8.30s/it]
 44%|████▍     | 16814/37962 [38:57:17<48:05:12,  8.19s/it]
 44%|████▍     | 16815/37962 [38:57:25<48:39:58,  8.28s/it]
 44%|████▍     | 16816/37962 [38:57:34<48:48:41,  8.31s/it]
 44%|████▍     | 16817/37962 [38:57:42<48:39:30,  8.28s/it]
 44%|████▍     | 16818/37962 [38:57:50<48:06:05,  8.19s/it]
 44%|████▍     | 16819/37962 [38:57:58<48:47:53,  8.31s/it]
 44%|████▍     | 16820/37962 [38:58:07<48:36:01,  8.28s/it]
                                                           
{'loss': 1.363, 'grad_norm': 10.447578430175781, 'learning_rate': 1.2308541514620645e-05, 'epoch': 0.44}

 44%|████▍     | 16820/37962 [38:58:07<48:36:01,  8.28s/it]
 44%|████▍     | 16821/37962 [38:58:15<48:40:09,  8.29s/it]
 44%|████▍     | 16822/37962 [38:58:23<48:37:53,  8.28s/it]
 44%|████▍     | 16823/37962 [38:58:32<48:47:21,  8.31s/it]
 44%|████▍     | 16824/37962 [38:58:40<48:07:56,  8.20s/it]
 44%|████▍     | 16825/37962 [38:58:48<47:54:37,  8.16s/it]
 44%|████▍     | 16826/37962 [38:58:56<47:42:45,  8.13s/it]
 44%|████▍     | 16827/37962 [38:59:04<47:39:26,  8.12s/it]
 44%|████▍     | 16828/37962 [38:59:12<47:58:30,  8.17s/it]
 44%|████▍     | 16829/37962 [38:59:20<48:01:31,  8.18s/it]
 44%|████▍     | 16830/37962 [38:59:29<48:18:24,  8.23s/it]
                                                           
{'loss': 1.4304, 'grad_norm': 8.445951461791992, 'learning_rate': 1.23002395237193e-05, 'epoch': 0.44}

 44%|████▍     | 16830/37962 [38:59:29<48:18:24,  8.23s/it]
 44%|████▍     | 16831/37962 [38:59:37<48:25:36,  8.25s/it]
 44%|████▍     | 16832/37962 [38:59:45<48:51:03,  8.32s/it]
 44%|████▍     | 16833/37962 [38:59:54<48:40:34,  8.29s/it]
 44%|████▍     | 16834/37962 [39:00:02<48:35:06,  8.28s/it]
 44%|████▍     | 16835/37962 [39:00:10<48:33:41,  8.27s/it]
 44%|████▍     | 16836/37962 [39:00:18<48:39:44,  8.29s/it]
 44%|████▍     | 16837/37962 [39:00:27<48:36:29,  8.28s/it]
 44%|████▍     | 16838/37962 [39:00:35<48:40:40,  8.30s/it]
 44%|████▍     | 16839/37962 [39:00:43<48:35:04,  8.28s/it]
 44%|████▍     | 16840/37962 [39:00:52<48:44:14,  8.31s/it]
                                                           
{'loss': 1.2957, 'grad_norm': 8.335794448852539, 'learning_rate': 1.2291935858513451e-05, 'epoch': 0.44}

 44%|████▍     | 16840/37962 [39:00:52<48:44:14,  8.31s/it]
 44%|████▍     | 16841/37962 [39:01:00<48:40:22,  8.30s/it]
 44%|████▍     | 16842/37962 [39:01:08<48:27:58,  8.26s/it]
 44%|████▍     | 16843/37962 [39:01:16<48:28:31,  8.26s/it]
 44%|████▍     | 16844/37962 [39:01:25<48:28:47,  8.26s/it]
 44%|████▍     | 16845/37962 [39:01:33<49:01:00,  8.36s/it]
 44%|████▍     | 16846/37962 [39:01:42<48:59:56,  8.35s/it]
 44%|████▍     | 16847/37962 [39:01:50<48:46:16,  8.32s/it]
 44%|████▍     | 16848/37962 [39:01:58<48:52:04,  8.33s/it]
 44%|████▍     | 16849/37962 [39:02:07<48:57:41,  8.35s/it]
 44%|████▍     | 16850/37962 [39:02:14<48:04:37,  8.20s/it]
                                                           
{'loss': 1.2718, 'grad_norm': 6.921783447265625, 'learning_rate': 1.2283630525047197e-05, 'epoch': 0.44}

 44%|████▍     | 16850/37962 [39:02:14<48:04:37,  8.20s/it]
 44%|████▍     | 16851/37962 [39:02:22<47:09:11,  8.04s/it]
 44%|████▍     | 16852/37962 [39:02:30<47:37:04,  8.12s/it]
 44%|████▍     | 16853/37962 [39:02:39<47:33:57,  8.11s/it]
 44%|████▍     | 16854/37962 [39:02:47<47:40:07,  8.13s/it]
 44%|████▍     | 16855/37962 [39:02:55<47:07:58,  8.04s/it]
 44%|████▍     | 16856/37962 [39:03:03<47:37:47,  8.12s/it]
 44%|████▍     | 16857/37962 [39:03:11<48:19:20,  8.24s/it]
 44%|████▍     | 16858/37962 [39:03:19<48:08:06,  8.21s/it]
 44%|████▍     | 16859/37962 [39:03:28<48:05:21,  8.20s/it]
 44%|████▍     | 16860/37962 [39:03:36<48:18:31,  8.24s/it]
                                                           
{'loss': 1.4568, 'grad_norm': 11.129327774047852, 'learning_rate': 1.2275323529365846e-05, 'epoch': 0.44}

 44%|████▍     | 16860/37962 [39:03:36<48:18:31,  8.24s/it]
 44%|████▍     | 16861/37962 [39:03:44<47:53:02,  8.17s/it]
 44%|████▍     | 16862/37962 [39:03:52<48:12:33,  8.23s/it]
 44%|████▍     | 16863/37962 [39:04:01<48:26:19,  8.26s/it]
 44%|████▍     | 16864/37962 [39:04:09<48:16:05,  8.24s/it]
 44%|████▍     | 16865/37962 [39:04:17<47:43:28,  8.14s/it]
 44%|████▍     | 16866/37962 [39:04:25<47:29:12,  8.10s/it]
 44%|████▍     | 16867/37962 [39:04:33<47:29:17,  8.10s/it]
 44%|████▍     | 16868/37962 [39:04:41<47:13:54,  8.06s/it]
 44%|████▍     | 16869/37962 [39:04:49<47:24:14,  8.09s/it]
 44%|████▍     | 16870/37962 [39:04:57<47:30:10,  8.11s/it]
                                                           
{'loss': 1.3401, 'grad_norm': 24.843137741088867, 'learning_rate': 1.2267014877515918e-05, 'epoch': 0.44}

 44%|████▍     | 16870/37962 [39:04:57<47:30:10,  8.11s/it]
 44%|████▍     | 16871/37962 [39:05:06<48:05:44,  8.21s/it]
 44%|████▍     | 16872/37962 [39:05:14<48:11:14,  8.23s/it]
 44%|████▍     | 16873/37962 [39:05:22<48:17:35,  8.24s/it]
 44%|████▍     | 16874/37962 [39:05:30<48:23:02,  8.26s/it]
 44%|████▍     | 16875/37962 [39:05:39<49:11:23,  8.40s/it]
 44%|████▍     | 16876/37962 [39:05:47<48:34:28,  8.29s/it]
 44%|████▍     | 16877/37962 [39:05:55<48:23:32,  8.26s/it]
 44%|████▍     | 16878/37962 [39:06:04<48:30:19,  8.28s/it]
 44%|████▍     | 16879/37962 [39:06:12<48:22:42,  8.26s/it]
 44%|████▍     | 16880/37962 [39:06:20<47:44:11,  8.15s/it]
                                                           
{'loss': 1.3668, 'grad_norm': 10.04791259765625, 'learning_rate': 1.2258704575545126e-05, 'epoch': 0.44}

 44%|████▍     | 16880/37962 [39:06:20<47:44:11,  8.15s/it]
 44%|████▍     | 16881/37962 [39:06:28<48:00:19,  8.20s/it]
 44%|████▍     | 16882/37962 [39:06:37<48:30:40,  8.28s/it]
 44%|████▍     | 16883/37962 [39:06:45<48:32:27,  8.29s/it]
 44%|████▍     | 16884/37962 [39:06:53<48:20:26,  8.26s/it]
 44%|████▍     | 16885/37962 [39:07:01<48:16:35,  8.25s/it]
 44%|████▍     | 16886/37962 [39:07:10<48:10:13,  8.23s/it]
 44%|████▍     | 16887/37962 [39:07:18<47:51:19,  8.17s/it]
 44%|████▍     | 16888/37962 [39:07:26<48:03:14,  8.21s/it]
 44%|████▍     | 16889/37962 [39:07:34<48:15:53,  8.25s/it]
 44%|████▍     | 16890/37962 [39:07:42<47:51:08,  8.18s/it]
                                                           
{'loss': 1.3379, 'grad_norm': 68.27047729492188, 'learning_rate': 1.225039262950241e-05, 'epoch': 0.44}

 44%|████▍     | 16890/37962 [39:07:42<47:51:08,  8.18s/it]
 44%|████▍     | 16891/37962 [39:07:51<48:21:50,  8.26s/it]
 44%|████▍     | 16892/37962 [39:07:59<48:20:56,  8.26s/it]
 44%|████▍     | 16893/37962 [39:08:07<48:20:18,  8.26s/it]
 45%|████▍     | 16894/37962 [39:08:15<48:14:45,  8.24s/it]
 45%|████▍     | 16895/37962 [39:08:24<48:02:43,  8.21s/it]
 45%|████▍     | 16896/37962 [39:08:32<48:05:10,  8.22s/it]
 45%|████▍     | 16897/37962 [39:08:40<48:02:14,  8.21s/it]
 45%|████▍     | 16898/37962 [39:08:48<48:12:49,  8.24s/it]
 45%|████▍     | 16899/37962 [39:08:57<48:41:26,  8.32s/it]
 45%|████▍     | 16900/37962 [39:09:05<48:32:27,  8.30s/it]
                                                           
{'loss': 1.3539, 'grad_norm': 5.000762462615967, 'learning_rate': 1.2242079045437883e-05, 'epoch': 0.45}

 45%|████▍     | 16900/37962 [39:09:05<48:32:27,  8.30s/it]
 45%|████▍     | 16901/37962 [39:09:13<48:35:07,  8.30s/it]
 45%|████▍     | 16902/37962 [39:09:22<48:27:54,  8.28s/it]
 45%|████▍     | 16903/37962 [39:09:30<48:25:08,  8.28s/it]
 45%|████▍     | 16904/37962 [39:09:38<48:46:54,  8.34s/it]
 45%|████▍     | 16905/37962 [39:09:47<48:28:15,  8.29s/it]
 45%|████▍     | 16906/37962 [39:09:55<48:25:41,  8.28s/it]
 45%|████▍     | 16907/37962 [39:10:03<47:59:52,  8.21s/it]
 45%|████▍     | 16908/37962 [39:10:11<48:19:50,  8.26s/it]
 45%|████▍     | 16909/37962 [39:10:19<48:16:43,  8.26s/it]
 45%|████▍     | 16910/37962 [39:10:28<48:01:53,  8.21s/it]
                                                           
{'loss': 1.438, 'grad_norm': 29.6456241607666, 'learning_rate': 1.2233763829402856e-05, 'epoch': 0.45}

 45%|████▍     | 16910/37962 [39:10:28<48:01:53,  8.21s/it]
 45%|████▍     | 16911/37962 [39:10:36<48:07:40,  8.23s/it]
 45%|████▍     | 16912/37962 [39:10:44<48:28:45,  8.29s/it]
 45%|████▍     | 16913/37962 [39:10:53<48:53:49,  8.36s/it]
 45%|████▍     | 16914/37962 [39:11:01<48:41:55,  8.33s/it]
 45%|████▍     | 16915/37962 [39:11:09<48:29:36,  8.29s/it]
 45%|████▍     | 16916/37962 [39:11:18<48:58:01,  8.38s/it]
 45%|████▍     | 16917/37962 [39:11:26<49:02:11,  8.39s/it]
 45%|████▍     | 16918/37962 [39:11:35<48:56:44,  8.37s/it]
 45%|████▍     | 16919/37962 [39:11:43<49:02:32,  8.39s/it]
 45%|████▍     | 16920/37962 [39:11:51<48:10:36,  8.24s/it]
                                                           
{'loss': 1.2982, 'grad_norm': 7.272820949554443, 'learning_rate': 1.2225446987449838e-05, 'epoch': 0.45}

 45%|████▍     | 16920/37962 [39:11:51<48:10:36,  8.24s/it]
 45%|████▍     | 16921/37962 [39:11:59<48:22:38,  8.28s/it]
 45%|████▍     | 16922/37962 [39:12:08<48:18:57,  8.27s/it]
 45%|████▍     | 16923/37962 [39:12:16<48:08:22,  8.24s/it]
 45%|████▍     | 16924/37962 [39:12:24<48:17:17,  8.26s/it]
 45%|████▍     | 16925/37962 [39:12:32<48:10:47,  8.24s/it]
 45%|████▍     | 16926/37962 [39:12:41<48:18:51,  8.27s/it]
 45%|████▍     | 16927/37962 [39:12:49<48:18:49,  8.27s/it]
 45%|████▍     | 16928/37962 [39:12:57<48:24:15,  8.28s/it]
 45%|████▍     | 16929/37962 [39:13:06<48:35:38,  8.32s/it]
 45%|████▍     | 16930/37962 [39:13:14<48:26:15,  8.29s/it]
                                                           
{'loss': 1.2816, 'grad_norm': 27.656280517578125, 'learning_rate': 1.2217128525632513e-05, 'epoch': 0.45}

 45%|████▍     | 16930/37962 [39:13:14<48:26:15,  8.29s/it]
 45%|████▍     | 16931/37962 [39:13:22<48:04:48,  8.23s/it]
 45%|████▍     | 16932/37962 [39:13:30<48:22:31,  8.28s/it]
 45%|████▍     | 16933/37962 [39:13:38<48:20:52,  8.28s/it]
 45%|████▍     | 16934/37962 [39:13:47<48:12:04,  8.25s/it]
 45%|████▍     | 16935/37962 [39:13:55<47:58:10,  8.21s/it]
 45%|████▍     | 16936/37962 [39:14:03<48:03:55,  8.23s/it]
 45%|████▍     | 16937/37962 [39:14:11<48:22:32,  8.28s/it]
 45%|████▍     | 16938/37962 [39:14:20<48:20:53,  8.28s/it]
 45%|████▍     | 16939/37962 [39:14:28<48:04:24,  8.23s/it]
 45%|████▍     | 16940/37962 [39:14:36<47:54:26,  8.20s/it]
                                                           
{'loss': 1.301, 'grad_norm': 13.898427963256836, 'learning_rate': 1.2208808450005743e-05, 'epoch': 0.45}

 45%|████▍     | 16940/37962 [39:14:36<47:54:26,  8.20s/it]
 45%|████▍     | 16941/37962 [39:14:44<47:53:27,  8.20s/it]
 45%|████▍     | 16942/37962 [39:14:53<48:12:43,  8.26s/it]
 45%|████▍     | 16943/37962 [39:15:01<48:20:29,  8.28s/it]
 45%|████▍     | 16944/37962 [39:15:09<48:29:51,  8.31s/it]
 45%|████▍     | 16945/37962 [39:15:18<48:29:29,  8.31s/it]
 45%|████▍     | 16946/37962 [39:15:26<48:37:55,  8.33s/it]
 45%|████▍     | 16947/37962 [39:15:34<48:23:28,  8.29s/it]
 45%|████▍     | 16948/37962 [39:15:42<48:08:33,  8.25s/it]
 45%|████▍     | 16949/37962 [39:15:51<48:27:18,  8.30s/it]
 45%|████▍     | 16950/37962 [39:15:59<48:28:58,  8.31s/it]
                                                           
{'loss': 1.3609, 'grad_norm': 21.38923454284668, 'learning_rate': 1.2200486766625566e-05, 'epoch': 0.45}

 45%|████▍     | 16950/37962 [39:15:59<48:28:58,  8.31s/it]
 45%|████▍     | 16951/37962 [39:16:07<48:16:00,  8.27s/it]
 45%|████▍     | 16952/37962 [39:16:16<48:19:31,  8.28s/it]
 45%|████▍     | 16953/37962 [39:16:24<48:36:00,  8.33s/it]
 45%|████▍     | 16954/37962 [39:16:32<48:37:55,  8.33s/it]
 45%|████▍     | 16955/37962 [39:16:40<47:39:05,  8.17s/it]
 45%|████▍     | 16956/37962 [39:16:49<48:02:01,  8.23s/it]
 45%|████▍     | 16957/37962 [39:16:57<48:16:13,  8.27s/it]
 45%|████▍     | 16958/37962 [39:17:05<48:02:57,  8.24s/it]
 45%|████▍     | 16959/37962 [39:17:13<48:06:10,  8.25s/it]
 45%|████▍     | 16960/37962 [39:17:22<48:16:01,  8.27s/it]
                                                           
{'loss': 1.3708, 'grad_norm': 6.022297382354736, 'learning_rate': 1.2192163481549192e-05, 'epoch': 0.45}

 45%|████▍     | 16960/37962 [39:17:22<48:16:01,  8.27s/it]
 45%|████▍     | 16961/37962 [39:17:30<48:51:24,  8.38s/it]
 45%|████▍     | 16962/37962 [39:17:39<48:46:56,  8.36s/it]
 45%|████▍     | 16963/37962 [39:17:47<48:49:34,  8.37s/it]
 45%|████▍     | 16964/37962 [39:17:55<48:13:42,  8.27s/it]
 45%|████▍     | 16965/37962 [39:18:03<47:30:50,  8.15s/it]
 45%|████▍     | 16966/37962 [39:18:11<47:45:45,  8.19s/it]
 45%|████▍     | 16967/37962 [39:18:20<48:08:03,  8.25s/it]
 45%|████▍     | 16968/37962 [39:18:28<48:15:54,  8.28s/it]
 45%|████▍     | 16969/37962 [39:18:36<48:43:25,  8.36s/it]
 45%|████▍     | 16970/37962 [39:18:45<48:34:28,  8.33s/it]
                                                           
{'loss': 1.292, 'grad_norm': 3.8119192123413086, 'learning_rate': 1.2183838600834997e-05, 'epoch': 0.45}

 45%|████▍     | 16970/37962 [39:18:45<48:34:28,  8.33s/it]
 45%|████▍     | 16971/37962 [39:18:53<48:27:30,  8.31s/it]
 45%|████▍     | 16972/37962 [39:19:01<48:34:11,  8.33s/it]
 45%|████▍     | 16973/37962 [39:19:10<48:46:12,  8.36s/it]
 45%|████▍     | 16974/37962 [39:19:18<48:20:48,  8.29s/it]
 45%|████▍     | 16975/37962 [39:19:26<47:48:18,  8.20s/it]
 45%|████▍     | 16976/37962 [39:19:34<48:04:39,  8.25s/it]
 45%|████▍     | 16977/37962 [39:19:43<48:05:28,  8.25s/it]
 45%|████▍     | 16978/37962 [39:19:51<47:59:10,  8.23s/it]
 45%|████▍     | 16979/37962 [39:19:59<48:08:59,  8.26s/it]
 45%|████▍     | 16980/37962 [39:20:07<48:06:05,  8.25s/it]
                                                           
{'loss': 1.3145, 'grad_norm': 6.753296852111816, 'learning_rate': 1.2175512130542516e-05, 'epoch': 0.45}

 45%|████▍     | 16980/37962 [39:20:07<48:06:05,  8.25s/it]
 45%|████▍     | 16981/37962 [39:20:15<47:54:47,  8.22s/it]
 45%|████▍     | 16982/37962 [39:20:24<48:03:01,  8.25s/it]
 45%|████▍     | 16983/37962 [39:20:32<47:42:57,  8.19s/it]
 45%|████▍     | 16984/37962 [39:20:40<48:09:47,  8.27s/it]
 45%|████▍     | 16985/37962 [39:20:48<47:50:44,  8.21s/it]
 45%|████▍     | 16986/37962 [39:20:57<47:51:46,  8.21s/it]
 45%|████▍     | 16987/37962 [39:21:05<47:43:10,  8.19s/it]
 45%|████▍     | 16988/37962 [39:21:20<60:32:23, 10.39s/it]
 45%|████▍     | 16989/37962 [39:21:28<56:44:01,  9.74s/it]
 45%|████▍     | 16990/37962 [39:21:37<54:17:40,  9.32s/it]
                                                           
{'loss': 1.345, 'grad_norm': 10.175582885742188, 'learning_rate': 1.2167184076732444e-05, 'epoch': 0.45}

 45%|████▍     | 16990/37962 [39:21:37<54:17:40,  9.32s/it]
 45%|████▍     | 16991/37962 [39:21:45<51:55:05,  8.91s/it]
 45%|████▍     | 16992/37962 [39:21:53<50:38:06,  8.69s/it]
 45%|████▍     | 16993/37962 [39:22:01<49:56:40,  8.57s/it]
 45%|████▍     | 16994/37962 [39:22:09<49:19:59,  8.47s/it]
 45%|████▍     | 16995/37962 [39:22:18<49:08:18,  8.44s/it]
 45%|████▍     | 16996/37962 [39:22:26<49:08:11,  8.44s/it]
 45%|████▍     | 16997/37962 [39:22:34<48:22:51,  8.31s/it]
 45%|████▍     | 16998/37962 [39:22:42<48:01:30,  8.25s/it]
 45%|████▍     | 16999/37962 [39:22:51<48:03:56,  8.25s/it]
 45%|████▍     | 17000/37962 [39:22:59<48:15:11,  8.29s/it]
                                                           
{'loss': 1.2474, 'grad_norm': 5.681181907653809, 'learning_rate': 1.2158854445466623e-05, 'epoch': 0.45}

 45%|████▍     | 17000/37962 [39:22:59<48:15:11,  8.29s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 45%|████▍     | 17001/37962 [39:23:49<120:57:28, 20.77s/it]
 45%|████▍     | 17002/37962 [39:23:57<99:06:42, 17.02s/it] 
 45%|████▍     | 17003/37962 [39:24:05<83:53:18, 14.41s/it]
 45%|████▍     | 17004/37962 [39:24:14<73:12:13, 12.57s/it]
 45%|████▍     | 17005/37962 [39:24:22<65:25:19, 11.24s/it]
 45%|████▍     | 17006/37962 [39:24:30<60:18:45, 10.36s/it]
 45%|████▍     | 17007/37962 [39:24:39<56:57:53,  9.79s/it]
 45%|████▍     | 17008/37962 [39:24:47<54:05:56,  9.29s/it]
 45%|████▍     | 17009/37962 [39:24:55<52:15:22,  8.98s/it]
 45%|████▍     | 17010/37962 [39:25:04<51:38:49,  8.87s/it]
                                                           
{'loss': 1.2891, 'grad_norm': 10.444528579711914, 'learning_rate': 1.2150523242808057e-05, 'epoch': 0.45}

 45%|████▍     | 17010/37962 [39:25:04<51:38:49,  8.87s/it]
 45%|████▍     | 17011/37962 [39:25:12<50:34:19,  8.69s/it]
 45%|████▍     | 17012/37962 [39:25:20<49:43:31,  8.54s/it]
 45%|████▍     | 17013/37962 [39:25:28<49:21:09,  8.48s/it]
 45%|████▍     | 17014/37962 [39:25:37<49:01:49,  8.43s/it]
 45%|████▍     | 17015/37962 [39:25:45<48:42:38,  8.37s/it]
 45%|████▍     | 17016/37962 [39:25:53<48:35:03,  8.35s/it]
 45%|████▍     | 17017/37962 [39:26:01<48:13:44,  8.29s/it]
 45%|████▍     | 17018/37962 [39:26:10<48:24:40,  8.32s/it]
 45%|████▍     | 17019/37962 [39:26:18<48:12:36,  8.29s/it]
 45%|████▍     | 17020/37962 [39:26:27<48:34:44,  8.35s/it]
                                                           
{'loss': 1.388, 'grad_norm': 8.15530776977539, 'learning_rate': 1.2142190474820873e-05, 'epoch': 0.45}

 45%|████▍     | 17020/37962 [39:26:27<48:34:44,  8.35s/it]
 45%|████▍     | 17021/37962 [39:26:35<48:30:55,  8.34s/it]
 45%|████▍     | 17022/37962 [39:26:43<48:27:23,  8.33s/it]
 45%|████▍     | 17023/37962 [39:26:52<48:29:21,  8.34s/it]
 45%|████▍     | 17024/37962 [39:27:00<48:16:50,  8.30s/it]
 45%|████▍     | 17025/37962 [39:27:08<47:53:58,  8.24s/it]
 45%|████▍     | 17026/37962 [39:27:16<47:54:35,  8.24s/it]
 45%|████▍     | 17027/37962 [39:27:24<48:02:11,  8.26s/it]
 45%|████▍     | 17028/37962 [39:27:33<48:17:50,  8.31s/it]
 45%|████▍     | 17029/37962 [39:27:41<47:57:45,  8.25s/it]
 45%|████▍     | 17030/37962 [39:27:49<47:41:39,  8.20s/it]
                                                           
{'loss': 1.4086, 'grad_norm': 9.996291160583496, 'learning_rate': 1.2133856147570358e-05, 'epoch': 0.45}

 45%|████▍     | 17030/37962 [39:27:49<47:41:39,  8.20s/it]
 45%|████▍     | 17031/37962 [39:27:57<48:07:03,  8.28s/it]
 45%|████▍     | 17032/37962 [39:28:06<47:51:32,  8.23s/it]
 45%|████▍     | 17033/37962 [39:28:14<48:01:15,  8.26s/it]
 45%|████▍     | 17034/37962 [39:28:22<47:54:58,  8.24s/it]
 45%|████▍     | 17035/37962 [39:28:30<48:01:04,  8.26s/it]
 45%|████▍     | 17036/37962 [39:28:39<48:01:10,  8.26s/it]
 45%|████▍     | 17037/37962 [39:28:47<48:08:23,  8.28s/it]
 45%|████▍     | 17038/37962 [39:28:55<48:18:28,  8.31s/it]
 45%|████▍     | 17039/37962 [39:29:03<47:57:08,  8.25s/it]
 45%|████▍     | 17040/37962 [39:29:12<47:44:55,  8.22s/it]
                                                           
{'loss': 1.3998, 'grad_norm': 5.583968639373779, 'learning_rate': 1.2125520267122915e-05, 'epoch': 0.45}

 45%|████▍     | 17040/37962 [39:29:12<47:44:55,  8.22s/it]
 45%|████▍     | 17041/37962 [39:29:20<47:58:17,  8.25s/it]
 45%|████▍     | 17042/37962 [39:29:28<48:00:29,  8.26s/it]
 45%|████▍     | 17043/37962 [39:29:36<47:21:31,  8.15s/it]
 45%|████▍     | 17044/37962 [39:29:45<47:47:17,  8.22s/it]
 45%|████▍     | 17045/37962 [39:29:53<48:14:20,  8.30s/it]
 45%|████▍     | 17046/37962 [39:30:01<47:56:38,  8.25s/it]
 45%|████▍     | 17047/37962 [39:30:10<48:11:43,  8.30s/it]
 45%|████▍     | 17048/37962 [39:30:18<48:13:55,  8.30s/it]
 45%|████▍     | 17049/37962 [39:30:26<47:50:44,  8.24s/it]
 45%|████▍     | 17050/37962 [39:30:34<47:54:47,  8.25s/it]
                                                           
{'loss': 1.4876, 'grad_norm': 9.561891555786133, 'learning_rate': 1.2117182839546091e-05, 'epoch': 0.45}

 45%|████▍     | 17050/37962 [39:30:34<47:54:47,  8.25s/it]
 45%|████▍     | 17051/37962 [39:30:42<47:58:16,  8.26s/it]
 45%|████▍     | 17052/37962 [39:30:51<48:09:44,  8.29s/it]
 45%|████▍     | 17053/37962 [39:30:59<47:51:19,  8.24s/it]
 45%|████▍     | 17054/37962 [39:31:07<47:21:23,  8.15s/it]
 45%|████▍     | 17055/37962 [39:31:15<47:22:48,  8.16s/it]
 45%|████▍     | 17056/37962 [39:31:23<47:28:42,  8.18s/it]
 45%|████▍     | 17057/37962 [39:31:32<47:41:14,  8.21s/it]
 45%|████▍     | 17058/37962 [39:31:40<47:27:02,  8.17s/it]
 45%|████▍     | 17059/37962 [39:31:48<47:21:45,  8.16s/it]
 45%|████▍     | 17060/37962 [39:31:56<47:33:37,  8.19s/it]
                                                           
{'loss': 1.388, 'grad_norm': 22.232030868530273, 'learning_rate': 1.2108843870908559e-05, 'epoch': 0.45}

 45%|████▍     | 17060/37962 [39:31:56<47:33:37,  8.19s/it]
 45%|████▍     | 17061/37962 [39:32:04<47:40:52,  8.21s/it]
 45%|████▍     | 17062/37962 [39:32:13<47:36:46,  8.20s/it]
 45%|████▍     | 17063/37962 [39:32:21<47:16:38,  8.14s/it]
 45%|████▍     | 17064/37962 [39:32:29<47:30:27,  8.18s/it]
 45%|████▍     | 17065/37962 [39:32:37<47:17:06,  8.15s/it]
 45%|████▍     | 17066/37962 [39:32:45<47:32:51,  8.19s/it]
 45%|████▍     | 17067/37962 [39:32:53<47:18:36,  8.15s/it]
 45%|████▍     | 17068/37962 [39:33:02<47:40:05,  8.21s/it]
 45%|████▍     | 17069/37962 [39:33:10<48:02:33,  8.28s/it]
 45%|████▍     | 17070/37962 [39:33:19<48:33:55,  8.37s/it]
                                                           
{'loss': 1.2969, 'grad_norm': 7.613271713256836, 'learning_rate': 1.2100503367280108e-05, 'epoch': 0.45}

 45%|████▍     | 17070/37962 [39:33:19<48:33:55,  8.37s/it]
 45%|████▍     | 17071/37962 [39:33:27<48:39:33,  8.39s/it]
 45%|████▍     | 17072/37962 [39:33:35<48:38:25,  8.38s/it]
 45%|████▍     | 17073/37962 [39:33:44<48:25:47,  8.35s/it]
 45%|████▍     | 17074/37962 [39:33:52<48:11:26,  8.31s/it]
 45%|████▍     | 17075/37962 [39:34:00<48:15:30,  8.32s/it]
 45%|████▍     | 17076/37962 [39:34:08<48:02:44,  8.28s/it]
 45%|████▍     | 17077/37962 [39:34:17<48:03:38,  8.28s/it]
 45%|████▍     | 17078/37962 [39:34:25<47:59:21,  8.27s/it]
 45%|████▍     | 17079/37962 [39:34:33<48:07:59,  8.30s/it]
 45%|████▍     | 17080/37962 [39:34:42<48:03:46,  8.29s/it]
                                                           
{'loss': 1.3251, 'grad_norm': 16.274017333984375, 'learning_rate': 1.2092161334731643e-05, 'epoch': 0.45}

 45%|████▍     | 17080/37962 [39:34:42<48:03:46,  8.29s/it]
 45%|████▍     | 17081/37962 [39:34:50<47:54:23,  8.26s/it]
 45%|████▍     | 17082/37962 [39:34:58<47:55:59,  8.26s/it]
 45%|████▌     | 17083/37962 [39:35:06<48:08:48,  8.30s/it]
 45%|████▌     | 17084/37962 [39:35:15<48:03:04,  8.29s/it]
 45%|████▌     | 17085/37962 [39:35:23<47:37:24,  8.21s/it]
 45%|████▌     | 17086/37962 [39:35:31<47:49:12,  8.25s/it]
 45%|████▌     | 17087/37962 [39:35:39<47:22:39,  8.17s/it]
 45%|████▌     | 17088/37962 [39:35:47<47:39:29,  8.22s/it]
 45%|████▌     | 17089/37962 [39:35:55<47:30:01,  8.19s/it]
 45%|████▌     | 17090/37962 [39:36:04<47:42:53,  8.23s/it]
                                                           
{'loss': 1.4244, 'grad_norm': 22.479900360107422, 'learning_rate': 1.2083817779335185e-05, 'epoch': 0.45}

 45%|████▌     | 17090/37962 [39:36:04<47:42:53,  8.23s/it]
 45%|████▌     | 17091/37962 [39:36:12<47:34:52,  8.21s/it]
 45%|████▌     | 17092/37962 [39:36:20<47:29:03,  8.19s/it]
 45%|████▌     | 17093/37962 [39:36:28<47:40:20,  8.22s/it]
 45%|████▌     | 17094/37962 [39:36:37<47:45:04,  8.24s/it]
 45%|████▌     | 17095/37962 [39:36:45<48:01:04,  8.28s/it]
 45%|████▌     | 17096/37962 [39:36:53<47:58:19,  8.28s/it]
 45%|████▌     | 17097/37962 [39:37:02<48:29:26,  8.37s/it]
 45%|████▌     | 17098/37962 [39:37:10<48:40:03,  8.40s/it]
 45%|████▌     | 17099/37962 [39:37:19<48:16:52,  8.33s/it]
 45%|████▌     | 17100/37962 [39:37:27<48:11:55,  8.32s/it]
                                                           
{'loss': 1.3025, 'grad_norm': 9.764890670776367, 'learning_rate': 1.2075472707163867e-05, 'epoch': 0.45}

 45%|████▌     | 17100/37962 [39:37:27<48:11:55,  8.32s/it]
 45%|████▌     | 17101/37962 [39:37:35<48:17:17,  8.33s/it]
 45%|████▌     | 17102/37962 [39:37:43<48:06:21,  8.30s/it]
 45%|████▌     | 17103/37962 [39:37:52<48:12:13,  8.32s/it]
 45%|████▌     | 17104/37962 [39:38:00<47:14:48,  8.15s/it]
 45%|████▌     | 17105/37962 [39:38:08<47:03:37,  8.12s/it]
 45%|████▌     | 17106/37962 [39:38:16<47:15:12,  8.16s/it]
 45%|████▌     | 17107/37962 [39:38:24<47:51:52,  8.26s/it]
 45%|████▌     | 17108/37962 [39:38:33<48:09:47,  8.31s/it]
 45%|████▌     | 17109/37962 [39:38:41<47:56:32,  8.28s/it]
 45%|████▌     | 17110/37962 [39:38:49<48:00:31,  8.29s/it]
                                                           
{'loss': 1.4526, 'grad_norm': 21.82045555114746, 'learning_rate': 1.2067126124291918e-05, 'epoch': 0.45}

 45%|████▌     | 17110/37962 [39:38:49<48:00:31,  8.29s/it]
 45%|████▌     | 17111/37962 [39:38:58<48:18:41,  8.34s/it]
 45%|████▌     | 17112/37962 [39:39:06<47:58:04,  8.28s/it]
 45%|████▌     | 17113/37962 [39:39:14<47:55:10,  8.27s/it]
 45%|████▌     | 17114/37962 [39:39:22<47:52:34,  8.27s/it]
 45%|████▌     | 17115/37962 [39:39:31<47:48:33,  8.26s/it]
 45%|████▌     | 17116/37962 [39:39:39<47:52:16,  8.27s/it]
 45%|████▌     | 17117/37962 [39:39:48<48:26:56,  8.37s/it]
 45%|████▌     | 17118/37962 [39:39:56<48:39:36,  8.40s/it]
 45%|████▌     | 17119/37962 [39:40:04<48:20:54,  8.35s/it]
 45%|████▌     | 17120/37962 [39:40:13<48:37:35,  8.40s/it]
                                                           
{'loss': 1.3843, 'grad_norm': 8.007939338684082, 'learning_rate': 1.2058778036794676e-05, 'epoch': 0.45}

 45%|████▌     | 17120/37962 [39:40:13<48:37:35,  8.40s/it]
 45%|████▌     | 17121/37962 [39:40:21<48:28:26,  8.37s/it]
 45%|████▌     | 17122/37962 [39:40:29<48:08:32,  8.32s/it]
 45%|████▌     | 17123/37962 [39:40:37<47:55:20,  8.28s/it]
 45%|████▌     | 17124/37962 [39:40:46<47:58:20,  8.29s/it]
 45%|████▌     | 17125/37962 [39:40:54<47:25:51,  8.19s/it]
 45%|████▌     | 17126/37962 [39:41:02<46:56:48,  8.11s/it]
 45%|████▌     | 17127/37962 [39:41:10<47:19:55,  8.18s/it]
 45%|████▌     | 17128/37962 [39:41:18<47:09:38,  8.15s/it]
 45%|████▌     | 17129/37962 [39:41:26<47:32:50,  8.22s/it]
 45%|████▌     | 17130/37962 [39:41:35<47:39:17,  8.24s/it]
                                                           
{'loss': 1.332, 'grad_norm': 18.004653930664062, 'learning_rate': 1.2050428450748567e-05, 'epoch': 0.45}

 45%|████▌     | 17130/37962 [39:41:35<47:39:17,  8.24s/it]
 45%|████▌     | 17131/37962 [39:41:43<47:54:17,  8.28s/it]
 45%|████▌     | 17132/37962 [39:41:51<47:56:09,  8.28s/it]
 45%|████▌     | 17133/37962 [39:42:00<48:00:07,  8.30s/it]
 45%|████▌     | 17134/37962 [39:42:08<48:04:02,  8.31s/it]
 45%|████▌     | 17135/37962 [39:42:16<47:33:00,  8.22s/it]
 45%|████▌     | 17136/37962 [39:42:24<47:32:26,  8.22s/it]
 45%|████▌     | 17137/37962 [39:42:33<47:39:34,  8.24s/it]
 45%|████▌     | 17138/37962 [39:42:41<47:54:20,  8.28s/it]
 45%|████▌     | 17139/37962 [39:42:49<47:57:09,  8.29s/it]
 45%|████▌     | 17140/37962 [39:42:57<47:39:36,  8.24s/it]
                                                           
{'loss': 1.2857, 'grad_norm': 3.740792751312256, 'learning_rate': 1.2042077372231109e-05, 'epoch': 0.45}

 45%|████▌     | 17140/37962 [39:42:57<47:39:36,  8.24s/it]
 45%|████▌     | 17141/37962 [39:43:06<47:46:13,  8.26s/it]
 45%|████▌     | 17142/37962 [39:43:14<47:48:20,  8.27s/it]
 45%|████▌     | 17143/37962 [39:43:22<47:42:18,  8.25s/it]
 45%|████▌     | 17144/37962 [39:43:31<47:52:04,  8.28s/it]
 45%|████▌     | 17145/37962 [39:43:39<48:03:07,  8.31s/it]
 45%|████▌     | 17146/37962 [39:43:47<47:53:58,  8.28s/it]
 45%|████▌     | 17147/37962 [39:43:56<48:08:29,  8.33s/it]
 45%|████▌     | 17148/37962 [39:44:04<47:52:28,  8.28s/it]
 45%|████▌     | 17149/37962 [39:44:12<47:30:44,  8.22s/it]
 45%|████▌     | 17150/37962 [39:44:20<47:45:11,  8.26s/it]
                                                           
{'loss': 1.3336, 'grad_norm': 5.3372321128845215, 'learning_rate': 1.203372480732091e-05, 'epoch': 0.45}

 45%|████▌     | 17150/37962 [39:44:20<47:45:11,  8.26s/it]
 45%|████▌     | 17151/37962 [39:44:29<48:05:00,  8.32s/it]
 45%|████▌     | 17152/37962 [39:44:37<48:01:50,  8.31s/it]
 45%|████▌     | 17153/37962 [39:44:45<47:57:13,  8.30s/it]
 45%|████▌     | 17154/37962 [39:44:53<47:18:55,  8.19s/it]
 45%|████▌     | 17155/37962 [39:45:02<47:40:12,  8.25s/it]
 45%|████▌     | 17156/37962 [39:45:10<47:24:28,  8.20s/it]
 45%|████▌     | 17157/37962 [39:45:18<47:20:07,  8.19s/it]
 45%|████▌     | 17158/37962 [39:45:26<47:25:59,  8.21s/it]
 45%|████▌     | 17159/37962 [39:45:35<47:54:51,  8.29s/it]
 45%|████▌     | 17160/37962 [39:45:43<47:59:51,  8.31s/it]
                                                           
{'loss': 1.2689, 'grad_norm': 14.75044059753418, 'learning_rate': 1.2025370762097652e-05, 'epoch': 0.45}

 45%|████▌     | 17160/37962 [39:45:43<47:59:51,  8.31s/it]
 45%|████▌     | 17161/37962 [39:45:51<47:14:18,  8.18s/it]
 45%|████▌     | 17162/37962 [39:45:59<47:15:17,  8.18s/it]
 45%|████▌     | 17163/37962 [39:46:07<47:29:05,  8.22s/it]
 45%|████▌     | 17164/37962 [39:46:15<47:05:25,  8.15s/it]
 45%|████▌     | 17165/37962 [39:46:23<47:14:46,  8.18s/it]
 45%|████▌     | 17166/37962 [39:46:32<47:35:03,  8.24s/it]
 45%|████▌     | 17167/37962 [39:46:40<47:42:01,  8.26s/it]
 45%|████▌     | 17168/37962 [39:46:49<48:04:29,  8.32s/it]
 45%|████▌     | 17169/37962 [39:46:57<47:56:38,  8.30s/it]
 45%|████▌     | 17170/37962 [39:47:05<48:04:03,  8.32s/it]
                                                           
{'loss': 1.2972, 'grad_norm': 8.255457878112793, 'learning_rate': 1.2017015242642106e-05, 'epoch': 0.45}

 45%|████▌     | 17170/37962 [39:47:05<48:04:03,  8.32s/it]
 45%|████▌     | 17171/37962 [39:47:13<47:36:37,  8.24s/it]
 45%|████▌     | 17172/37962 [39:47:22<47:46:46,  8.27s/it]
 45%|████▌     | 17173/37962 [39:47:30<47:46:14,  8.27s/it]
 45%|████▌     | 17174/37962 [39:47:38<47:19:33,  8.20s/it]
 45%|████▌     | 17175/37962 [39:47:46<47:29:45,  8.23s/it]
 45%|████▌     | 17176/37962 [39:47:55<47:47:35,  8.28s/it]
 45%|████▌     | 17177/37962 [39:48:03<48:01:34,  8.32s/it]
 45%|████▌     | 17178/37962 [39:48:11<48:05:20,  8.33s/it]
 45%|████▌     | 17179/37962 [39:48:20<47:43:25,  8.27s/it]
 45%|████▌     | 17180/37962 [39:48:28<47:41:36,  8.26s/it]
                                                           
{'loss': 1.2964, 'grad_norm': 5.61617374420166, 'learning_rate': 1.2008658255036107e-05, 'epoch': 0.45}

 45%|████▌     | 17180/37962 [39:48:28<47:41:36,  8.26s/it]
 45%|████▌     | 17181/37962 [39:48:36<47:56:41,  8.31s/it]
 45%|████▌     | 17182/37962 [39:48:44<47:48:33,  8.28s/it]
 45%|████▌     | 17183/37962 [39:48:53<47:43:18,  8.27s/it]
 45%|████▌     | 17184/37962 [39:49:01<47:26:13,  8.22s/it]
 45%|████▌     | 17185/37962 [39:49:09<47:08:02,  8.17s/it]
 45%|████▌     | 17186/37962 [39:49:17<47:09:19,  8.17s/it]
 45%|████▌     | 17187/37962 [39:49:25<47:27:07,  8.22s/it]
 45%|████▌     | 17188/37962 [39:49:34<47:29:23,  8.23s/it]
 45%|████▌     | 17189/37962 [39:49:42<47:22:50,  8.21s/it]
 45%|████▌     | 17190/37962 [39:49:50<46:45:16,  8.10s/it]
                                                           
{'loss': 1.2318, 'grad_norm': 16.12565803527832, 'learning_rate': 1.2000299805362562e-05, 'epoch': 0.45}

 45%|████▌     | 17190/37962 [39:49:50<46:45:16,  8.10s/it]
 45%|████▌     | 17191/37962 [39:49:58<46:56:10,  8.13s/it]
 45%|████▌     | 17192/37962 [39:50:06<47:43:52,  8.27s/it]
 45%|████▌     | 17193/37962 [39:50:15<47:58:43,  8.32s/it]
 45%|████▌     | 17194/37962 [39:50:23<47:52:41,  8.30s/it]
 45%|████▌     | 17195/37962 [39:50:31<47:53:09,  8.30s/it]
 45%|████▌     | 17196/37962 [39:50:39<47:30:55,  8.24s/it]
 45%|████▌     | 17197/37962 [39:50:48<47:56:34,  8.31s/it]
 45%|████▌     | 17198/37962 [39:50:56<47:48:47,  8.29s/it]
 45%|████▌     | 17199/37962 [39:51:04<47:48:16,  8.29s/it]
 45%|████▌     | 17200/37962 [39:51:12<47:08:10,  8.17s/it]
                                                           
{'loss': 1.3524, 'grad_norm': 10.079707145690918, 'learning_rate': 1.199193989970544e-05, 'epoch': 0.45}

 45%|████▌     | 17200/37962 [39:51:12<47:08:10,  8.17s/it]
 45%|████▌     | 17201/37962 [39:51:21<47:24:19,  8.22s/it]
 45%|████▌     | 17202/37962 [39:51:29<47:15:20,  8.19s/it]
 45%|████▌     | 17203/37962 [39:51:37<47:26:38,  8.23s/it]
 45%|████▌     | 17204/37962 [39:51:45<47:28:01,  8.23s/it]
 45%|████▌     | 17205/37962 [39:51:53<47:00:29,  8.15s/it]
 45%|████▌     | 17206/37962 [39:52:02<47:43:18,  8.28s/it]
 45%|████▌     | 17207/37962 [39:52:10<48:12:32,  8.36s/it]
 45%|████▌     | 17208/37962 [39:52:19<47:57:19,  8.32s/it]
 45%|████▌     | 17209/37962 [39:52:27<47:34:45,  8.25s/it]
 45%|████▌     | 17210/37962 [39:52:35<47:33:42,  8.25s/it]
                                                           
{'loss': 1.3439, 'grad_norm': 7.114057540893555, 'learning_rate': 1.1983578544149776e-05, 'epoch': 0.45}

 45%|████▌     | 17210/37962 [39:52:35<47:33:42,  8.25s/it]
 45%|████▌     | 17211/37962 [39:52:43<47:29:21,  8.24s/it]
 45%|████▌     | 17212/37962 [39:52:51<47:25:57,  8.23s/it]
 45%|████▌     | 17213/37962 [39:53:00<47:12:30,  8.19s/it]
 45%|████▌     | 17214/37962 [39:53:08<47:38:30,  8.27s/it]
 45%|████▌     | 17215/37962 [39:53:16<47:51:38,  8.30s/it]
 45%|████▌     | 17216/37962 [39:53:25<47:43:07,  8.28s/it]
 45%|████▌     | 17217/37962 [39:53:33<47:43:55,  8.28s/it]
 45%|████▌     | 17218/37962 [39:53:42<48:21:13,  8.39s/it]
 45%|████▌     | 17219/37962 [39:53:50<48:17:33,  8.38s/it]
 45%|████▌     | 17220/37962 [39:53:58<47:56:53,  8.32s/it]
                                                           
{'loss': 1.2414, 'grad_norm': 9.709219932556152, 'learning_rate': 1.197521574478165e-05, 'epoch': 0.45}

 45%|████▌     | 17220/37962 [39:53:58<47:56:53,  8.32s/it]
 45%|████▌     | 17221/37962 [39:54:06<47:46:00,  8.29s/it]
 45%|████▌     | 17222/37962 [39:54:15<48:33:29,  8.43s/it]
 45%|████▌     | 17223/37962 [39:54:23<48:16:07,  8.38s/it]
 45%|████▌     | 17224/37962 [39:54:32<48:12:49,  8.37s/it]
 45%|████▌     | 17225/37962 [39:54:40<48:01:18,  8.34s/it]
 45%|████▌     | 17226/37962 [39:54:48<48:15:02,  8.38s/it]
 45%|████▌     | 17227/37962 [39:54:57<48:15:01,  8.38s/it]
 45%|████▌     | 17228/37962 [39:55:05<48:21:21,  8.40s/it]
 45%|████▌     | 17229/37962 [39:55:13<47:29:17,  8.25s/it]
 45%|████▌     | 17230/37962 [39:55:22<48:04:12,  8.35s/it]
                                                           
{'loss': 1.2717, 'grad_norm': 7.369729995727539, 'learning_rate': 1.1966851507688203e-05, 'epoch': 0.45}

 45%|████▌     | 17230/37962 [39:55:22<48:04:12,  8.35s/it]
 45%|████▌     | 17231/37962 [39:55:30<47:58:47,  8.33s/it]
 45%|████▌     | 17232/37962 [39:55:38<47:37:25,  8.27s/it]
 45%|████▌     | 17233/37962 [39:55:46<47:28:46,  8.25s/it]
 45%|████▌     | 17234/37962 [39:55:55<47:38:13,  8.27s/it]
 45%|████▌     | 17235/37962 [39:56:03<47:52:20,  8.31s/it]
 45%|████▌     | 17236/37962 [39:56:11<47:00:50,  8.17s/it]
 45%|████▌     | 17237/37962 [39:56:19<47:37:30,  8.27s/it]
 45%|████▌     | 17238/37962 [39:56:28<47:33:25,  8.26s/it]
 45%|████▌     | 17239/37962 [39:56:36<47:40:05,  8.28s/it]
 45%|████▌     | 17240/37962 [39:56:44<47:28:39,  8.25s/it]
                                                           
{'loss': 1.4083, 'grad_norm': 12.259556770324707, 'learning_rate': 1.1958485838957618e-05, 'epoch': 0.45}

 45%|████▌     | 17240/37962 [39:56:44<47:28:39,  8.25s/it]
 45%|████▌     | 17241/37962 [39:56:52<47:23:26,  8.23s/it]
 45%|████▌     | 17242/37962 [39:57:01<47:45:45,  8.30s/it]
 45%|████▌     | 17243/37962 [39:57:09<48:12:41,  8.38s/it]
 45%|████▌     | 17244/37962 [39:57:17<47:29:44,  8.25s/it]
 45%|████▌     | 17245/37962 [39:57:26<47:41:09,  8.29s/it]
 45%|████▌     | 17246/37962 [39:57:34<47:45:46,  8.30s/it]
 45%|████▌     | 17247/37962 [39:57:42<48:04:11,  8.35s/it]
 45%|████▌     | 17248/37962 [39:57:51<47:47:06,  8.30s/it]
 45%|████▌     | 17249/37962 [39:57:59<47:51:27,  8.32s/it]
 45%|████▌     | 17250/37962 [39:58:07<47:24:45,  8.24s/it]
                                                           
{'loss': 1.3734, 'grad_norm': 8.01494026184082, 'learning_rate': 1.1950118744679118e-05, 'epoch': 0.45}

 45%|████▌     | 17250/37962 [39:58:07<47:24:45,  8.24s/it]
 45%|████▌     | 17251/37962 [39:58:15<47:31:56,  8.26s/it]
 45%|████▌     | 17252/37962 [39:58:23<47:01:34,  8.17s/it]
 45%|████▌     | 17253/37962 [39:58:31<46:39:47,  8.11s/it]
 45%|████▌     | 17254/37962 [39:58:40<46:49:45,  8.14s/it]
 45%|████▌     | 17255/37962 [39:58:48<46:58:21,  8.17s/it]
 45%|████▌     | 17256/37962 [39:58:56<47:12:14,  8.21s/it]
 45%|████▌     | 17257/37962 [39:59:04<47:17:30,  8.22s/it]
 45%|████▌     | 17258/37962 [39:59:13<47:24:23,  8.24s/it]
 45%|████▌     | 17259/37962 [39:59:21<47:25:49,  8.25s/it]
 45%|████▌     | 17260/37962 [39:59:29<46:35:17,  8.10s/it]
                                                           
{'loss': 1.3972, 'grad_norm': 17.103248596191406, 'learning_rate': 1.1941750230942963e-05, 'epoch': 0.45}

 45%|████▌     | 17260/37962 [39:59:29<46:35:17,  8.10s/it]
 45%|████▌     | 17261/37962 [39:59:37<46:27:19,  8.08s/it]
 45%|████▌     | 17262/37962 [39:59:45<46:19:21,  8.06s/it]
 45%|████▌     | 17263/37962 [39:59:53<46:45:57,  8.13s/it]
 45%|████▌     | 17264/37962 [40:00:01<47:14:34,  8.22s/it]
 45%|████▌     | 17265/37962 [40:00:10<47:28:25,  8.26s/it]
 45%|████▌     | 17266/37962 [40:00:18<47:27:46,  8.26s/it]
 45%|████▌     | 17267/37962 [40:00:26<47:14:21,  8.22s/it]
 45%|████▌     | 17268/37962 [40:00:35<47:36:24,  8.28s/it]
 45%|████▌     | 17269/37962 [40:00:43<47:27:20,  8.26s/it]
 45%|████▌     | 17270/37962 [40:00:51<47:26:24,  8.25s/it]
                                                           
{'loss': 1.4064, 'grad_norm': 72.14289855957031, 'learning_rate': 1.1933380303840461e-05, 'epoch': 0.45}

 45%|████▌     | 17270/37962 [40:00:51<47:26:24,  8.25s/it]
 45%|████▌     | 17271/37962 [40:00:59<47:23:24,  8.25s/it]
 45%|████▌     | 17272/37962 [40:01:08<47:31:39,  8.27s/it]
 46%|████▌     | 17273/37962 [40:01:16<47:29:51,  8.26s/it]
 46%|████▌     | 17274/37962 [40:01:24<47:25:57,  8.25s/it]
 46%|████▌     | 17275/37962 [40:01:32<47:27:58,  8.26s/it]
 46%|████▌     | 17276/37962 [40:01:41<47:26:01,  8.25s/it]
 46%|████▌     | 17277/37962 [40:01:49<47:03:17,  8.19s/it]
 46%|████▌     | 17278/37962 [40:01:57<46:50:54,  8.15s/it]
 46%|████▌     | 17279/37962 [40:02:05<46:57:45,  8.17s/it]
 46%|████▌     | 17280/37962 [40:02:13<47:17:13,  8.23s/it]
                                                           
{'loss': 1.3733, 'grad_norm': 4.074512004852295, 'learning_rate': 1.1925008969463925e-05, 'epoch': 0.46}

 46%|████▌     | 17280/37962 [40:02:13<47:17:13,  8.23s/it]
 46%|████▌     | 17281/37962 [40:02:22<47:27:52,  8.26s/it]
 46%|████▌     | 17282/37962 [40:02:30<47:41:28,  8.30s/it]
 46%|████▌     | 17283/37962 [40:02:38<47:35:37,  8.29s/it]
 46%|████▌     | 17284/37962 [40:02:47<47:38:52,  8.30s/it]
 46%|████▌     | 17285/37962 [40:02:55<47:54:35,  8.34s/it]
 46%|████▌     | 17286/37962 [40:03:03<47:53:37,  8.34s/it]
 46%|████▌     | 17287/37962 [40:03:12<48:25:38,  8.43s/it]
 46%|████▌     | 17288/37962 [40:03:20<47:56:48,  8.35s/it]
 46%|████▌     | 17289/37962 [40:03:28<46:45:26,  8.14s/it]
 46%|████▌     | 17290/37962 [40:03:36<46:59:26,  8.18s/it]
                                                           
{'loss': 1.4913, 'grad_norm': 11.39741325378418, 'learning_rate': 1.1916636233906709e-05, 'epoch': 0.46}

 46%|████▌     | 17290/37962 [40:03:36<46:59:26,  8.18s/it]
 46%|████▌     | 17291/37962 [40:03:44<47:12:43,  8.22s/it]
 46%|████▌     | 17292/37962 [40:03:52<46:49:15,  8.15s/it]
 46%|████▌     | 17293/37962 [40:04:01<47:01:35,  8.19s/it]
 46%|████▌     | 17294/37962 [40:04:09<47:34:30,  8.29s/it]
 46%|████▌     | 17295/37962 [40:04:17<47:25:45,  8.26s/it]
 46%|████▌     | 17296/37962 [40:04:26<47:23:07,  8.25s/it]
 46%|████▌     | 17297/37962 [40:04:34<47:37:14,  8.30s/it]
 46%|████▌     | 17298/37962 [40:04:42<47:26:11,  8.26s/it]
 46%|████▌     | 17299/37962 [40:04:50<47:20:51,  8.25s/it]
 46%|████▌     | 17300/37962 [40:04:59<47:13:28,  8.23s/it]
                                                           
{'loss': 1.496, 'grad_norm': 12.816291809082031, 'learning_rate': 1.1908262103263184e-05, 'epoch': 0.46}

 46%|████▌     | 17300/37962 [40:04:59<47:13:28,  8.23s/it]
 46%|████▌     | 17301/37962 [40:05:07<47:02:50,  8.20s/it]
 46%|████▌     | 17302/37962 [40:05:15<47:33:45,  8.29s/it]
 46%|████▌     | 17303/37962 [40:05:24<47:38:38,  8.30s/it]
 46%|████▌     | 17304/37962 [40:05:32<47:33:58,  8.29s/it]
 46%|████▌     | 17305/37962 [40:05:40<47:44:35,  8.32s/it]
 46%|████▌     | 17306/37962 [40:05:48<47:44:57,  8.32s/it]
 46%|████▌     | 17307/37962 [40:05:57<47:38:04,  8.30s/it]
 46%|████▌     | 17308/37962 [40:06:05<47:41:47,  8.31s/it]
 46%|████▌     | 17309/37962 [40:06:13<47:32:00,  8.29s/it]
 46%|████▌     | 17310/37962 [40:06:21<47:14:34,  8.24s/it]
                                                           
{'loss': 1.3381, 'grad_norm': 11.657978057861328, 'learning_rate': 1.1899886583628731e-05, 'epoch': 0.46}

 46%|████▌     | 17310/37962 [40:06:21<47:14:34,  8.24s/it]
 46%|████▌     | 17311/37962 [40:06:30<47:12:24,  8.23s/it]
 46%|████▌     | 17312/37962 [40:06:38<47:24:23,  8.26s/it]
 46%|████▌     | 17313/37962 [40:06:46<47:24:45,  8.27s/it]
 46%|████▌     | 17314/37962 [40:06:54<47:20:59,  8.26s/it]
 46%|████▌     | 17315/37962 [40:07:03<47:35:49,  8.30s/it]
 46%|████▌     | 17316/37962 [40:07:11<47:22:06,  8.26s/it]
 46%|████▌     | 17317/37962 [40:07:19<46:46:48,  8.16s/it]
 46%|████▌     | 17318/37962 [40:07:27<46:43:00,  8.15s/it]
 46%|████▌     | 17319/37962 [40:07:35<46:54:32,  8.18s/it]
 46%|████▌     | 17320/37962 [40:07:43<46:44:51,  8.15s/it]
                                                           
{'loss': 1.377, 'grad_norm': 9.631767272949219, 'learning_rate': 1.1891509681099744e-05, 'epoch': 0.46}

 46%|████▌     | 17320/37962 [40:07:43<46:44:51,  8.15s/it]
 46%|████▌     | 17321/37962 [40:07:52<46:48:24,  8.16s/it]
 46%|████▌     | 17322/37962 [40:08:00<46:56:53,  8.19s/it]
 46%|████▌     | 17323/37962 [40:08:08<47:20:18,  8.26s/it]
 46%|████▌     | 17324/37962 [40:08:17<47:24:02,  8.27s/it]
 46%|████▌     | 17325/37962 [40:08:25<47:44:25,  8.33s/it]
 46%|████▌     | 17326/37962 [40:08:40<59:51:03, 10.44s/it]
 46%|████▌     | 17327/37962 [40:08:49<56:28:37,  9.85s/it]
 46%|████▌     | 17328/37962 [40:08:57<53:48:51,  9.39s/it]
 46%|████▌     | 17329/37962 [40:09:05<51:26:38,  8.98s/it]
 46%|████▌     | 17330/37962 [40:09:14<50:14:29,  8.77s/it]
                                                           
{'loss': 1.292, 'grad_norm': 4.321954250335693, 'learning_rate': 1.188313140177363e-05, 'epoch': 0.46}

 46%|████▌     | 17330/37962 [40:09:14<50:14:29,  8.77s/it]
 46%|████▌     | 17331/37962 [40:09:21<48:44:21,  8.50s/it]
 46%|████▌     | 17332/37962 [40:09:30<48:19:19,  8.43s/it]
 46%|████▌     | 17333/37962 [40:09:38<47:59:53,  8.38s/it]
 46%|████▌     | 17334/37962 [40:09:46<47:40:37,  8.32s/it]
 46%|████▌     | 17335/37962 [40:09:54<47:16:03,  8.25s/it]
 46%|████▌     | 17336/37962 [40:10:03<47:37:36,  8.31s/it]
 46%|████▌     | 17337/37962 [40:10:11<47:50:28,  8.35s/it]
 46%|████▌     | 17338/37962 [40:10:19<47:52:37,  8.36s/it]
 46%|████▌     | 17339/37962 [40:10:28<47:26:43,  8.28s/it]
 46%|████▌     | 17340/37962 [40:10:36<47:26:46,  8.28s/it]
                                                           
{'loss': 1.2805, 'grad_norm': 5.757198810577393, 'learning_rate': 1.1874751751748794e-05, 'epoch': 0.46}

 46%|████▌     | 17340/37962 [40:10:36<47:26:46,  8.28s/it]
 46%|████▌     | 17341/37962 [40:10:44<47:32:54,  8.30s/it]
 46%|████▌     | 17342/37962 [40:10:53<47:40:16,  8.32s/it]
 46%|████▌     | 17343/37962 [40:11:01<47:59:08,  8.38s/it]
 46%|████▌     | 17344/37962 [40:11:09<47:45:03,  8.34s/it]
 46%|████▌     | 17345/37962 [40:11:18<47:43:00,  8.33s/it]
 46%|████▌     | 17346/37962 [40:11:26<47:32:42,  8.30s/it]
 46%|████▌     | 17347/37962 [40:11:34<47:32:31,  8.30s/it]
 46%|████▌     | 17348/37962 [40:11:42<47:17:20,  8.26s/it]
 46%|████▌     | 17349/37962 [40:11:51<47:26:23,  8.29s/it]
 46%|████▌     | 17350/37962 [40:11:59<47:27:20,  8.29s/it]
                                                           
{'loss': 1.3795, 'grad_norm': 12.165517807006836, 'learning_rate': 1.1866370737124637e-05, 'epoch': 0.46}

 46%|████▌     | 17350/37962 [40:11:59<47:27:20,  8.29s/it]
 46%|████▌     | 17351/37962 [40:12:07<47:01:52,  8.21s/it]
 46%|████▌     | 17352/37962 [40:12:15<47:10:08,  8.24s/it]
 46%|████▌     | 17353/37962 [40:12:24<47:06:15,  8.23s/it]
 46%|████▌     | 17354/37962 [40:12:32<47:18:34,  8.26s/it]
 46%|████▌     | 17355/37962 [40:12:40<47:00:00,  8.21s/it]
 46%|████▌     | 17356/37962 [40:12:48<47:09:49,  8.24s/it]
 46%|████▌     | 17357/37962 [40:12:56<47:05:50,  8.23s/it]
 46%|████▌     | 17358/37962 [40:13:05<46:50:22,  8.18s/it]
 46%|████▌     | 17359/37962 [40:13:12<46:16:19,  8.09s/it]
 46%|████▌     | 17360/37962 [40:13:21<46:58:55,  8.21s/it]
                                                           
{'loss': 1.4036, 'grad_norm': 2.6982247829437256, 'learning_rate': 1.185798836400155e-05, 'epoch': 0.46}

 46%|████▌     | 17360/37962 [40:13:21<46:58:55,  8.21s/it]
 46%|████▌     | 17361/37962 [40:13:29<46:33:42,  8.14s/it]
 46%|████▌     | 17362/37962 [40:13:37<47:02:29,  8.22s/it]
 46%|████▌     | 17363/37962 [40:13:45<46:57:00,  8.21s/it]
 46%|████▌     | 17364/37962 [40:13:54<46:51:17,  8.19s/it]
 46%|████▌     | 17365/37962 [40:14:02<47:12:51,  8.25s/it]
 46%|████▌     | 17366/37962 [40:14:10<47:10:11,  8.24s/it]
 46%|████▌     | 17367/37962 [40:14:19<47:14:18,  8.26s/it]
 46%|████▌     | 17368/37962 [40:14:27<47:14:40,  8.26s/it]
 46%|████▌     | 17369/37962 [40:14:35<47:24:10,  8.29s/it]
 46%|████▌     | 17370/37962 [40:14:43<47:13:06,  8.25s/it]
                                                           
{'loss': 1.3653, 'grad_norm': 10.534849166870117, 'learning_rate': 1.184960463848092e-05, 'epoch': 0.46}

 46%|████▌     | 17370/37962 [40:14:43<47:13:06,  8.25s/it]
 46%|████▌     | 17371/37962 [40:14:59<59:51:13, 10.46s/it]
 46%|████▌     | 17372/37962 [40:15:08<56:38:37,  9.90s/it]
 46%|████▌     | 17373/37962 [40:15:16<54:11:50,  9.48s/it]
 46%|████▌     | 17374/37962 [40:15:24<52:00:13,  9.09s/it]
 46%|████▌     | 17375/37962 [40:15:32<50:18:54,  8.80s/it]
 46%|████▌     | 17376/37962 [40:15:41<49:40:52,  8.69s/it]
 46%|████▌     | 17377/37962 [40:15:49<48:57:48,  8.56s/it]
 46%|████▌     | 17378/37962 [40:15:57<48:20:44,  8.46s/it]
 46%|████▌     | 17379/37962 [40:16:05<48:01:47,  8.40s/it]
 46%|████▌     | 17380/37962 [40:16:14<47:40:33,  8.34s/it]
                                                           
{'loss': 1.284, 'grad_norm': 9.679082870483398, 'learning_rate': 1.184121956666512e-05, 'epoch': 0.46}

 46%|████▌     | 17380/37962 [40:16:14<47:40:33,  8.34s/it]
 46%|████▌     | 17381/37962 [40:16:22<47:38:57,  8.33s/it]
 46%|████▌     | 17382/37962 [40:16:30<47:47:16,  8.36s/it]
 46%|████▌     | 17383/37962 [40:16:39<47:43:28,  8.35s/it]
 46%|████▌     | 17384/37962 [40:16:47<47:48:52,  8.36s/it]
 46%|████▌     | 17385/37962 [40:16:55<47:40:26,  8.34s/it]
 46%|████▌     | 17386/37962 [40:17:04<47:20:07,  8.28s/it]
 46%|████▌     | 17387/37962 [40:17:12<47:18:54,  8.28s/it]
 46%|████▌     | 17388/37962 [40:17:20<47:40:00,  8.34s/it]
 46%|████▌     | 17389/37962 [40:17:29<47:46:34,  8.36s/it]
 46%|████▌     | 17390/37962 [40:17:37<47:53:29,  8.38s/it]
                                                           
{'loss': 1.3792, 'grad_norm': 9.43470573425293, 'learning_rate': 1.1832833154657488e-05, 'epoch': 0.46}

 46%|████▌     | 17390/37962 [40:17:37<47:53:29,  8.38s/it]
 46%|████▌     | 17391/37962 [40:17:46<47:56:44,  8.39s/it]
 46%|████▌     | 17392/37962 [40:17:54<47:44:04,  8.35s/it]
 46%|████▌     | 17393/37962 [40:18:02<47:53:20,  8.38s/it]
 46%|████▌     | 17394/37962 [40:18:11<47:55:49,  8.39s/it]
 46%|████▌     | 17395/37962 [40:18:19<47:46:41,  8.36s/it]
 46%|████▌     | 17396/37962 [40:18:27<47:36:53,  8.33s/it]
 46%|████▌     | 17397/37962 [40:18:36<47:32:03,  8.32s/it]
 46%|████▌     | 17398/37962 [40:18:44<47:38:13,  8.34s/it]
 46%|████▌     | 17399/37962 [40:18:52<47:49:05,  8.37s/it]
 46%|████▌     | 17400/37962 [40:19:01<47:41:03,  8.35s/it]
                                                           
{'loss': 1.3408, 'grad_norm': 5.2622551918029785, 'learning_rate': 1.1824445408562357e-05, 'epoch': 0.46}

 46%|████▌     | 17400/37962 [40:19:01<47:41:03,  8.35s/it]
 46%|████▌     | 17401/37962 [40:19:09<47:33:08,  8.33s/it]
 46%|████▌     | 17402/37962 [40:19:17<47:32:59,  8.33s/it]
 46%|████▌     | 17403/37962 [40:19:26<47:30:24,  8.32s/it]
 46%|████▌     | 17404/37962 [40:19:34<47:26:12,  8.31s/it]
 46%|████▌     | 17405/37962 [40:19:42<47:07:53,  8.25s/it]
 46%|████▌     | 17406/37962 [40:19:50<47:02:10,  8.24s/it]
 46%|████▌     | 17407/37962 [40:19:58<46:57:41,  8.22s/it]
 46%|████▌     | 17408/37962 [40:20:07<46:56:24,  8.22s/it]
 46%|████▌     | 17409/37962 [40:20:15<46:35:26,  8.16s/it]
 46%|████▌     | 17410/37962 [40:20:23<46:52:29,  8.21s/it]
                                                           
{'loss': 1.3875, 'grad_norm': 7.97438383102417, 'learning_rate': 1.181605633448502e-05, 'epoch': 0.46}

 46%|████▌     | 17410/37962 [40:20:23<46:52:29,  8.21s/it]
 46%|████▌     | 17411/37962 [40:20:31<47:13:32,  8.27s/it]
 46%|████▌     | 17412/37962 [40:20:40<47:25:01,  8.31s/it]
 46%|████▌     | 17413/37962 [40:20:48<47:18:35,  8.29s/it]
 46%|████▌     | 17414/37962 [40:20:56<47:24:21,  8.31s/it]
 46%|████▌     | 17415/37962 [40:21:05<47:24:56,  8.31s/it]
 46%|████▌     | 17416/37962 [40:21:13<47:26:16,  8.31s/it]
 46%|████▌     | 17417/37962 [40:21:21<47:39:08,  8.35s/it]
 46%|████▌     | 17418/37962 [40:21:30<47:41:04,  8.36s/it]
 46%|████▌     | 17419/37962 [40:21:38<48:07:07,  8.43s/it]
 46%|████▌     | 17420/37962 [40:21:47<47:37:23,  8.35s/it]
                                                           
{'loss': 1.3575, 'grad_norm': 3.620469331741333, 'learning_rate': 1.180766593853174e-05, 'epoch': 0.46}

 46%|████▌     | 17420/37962 [40:21:47<47:37:23,  8.35s/it]
 46%|████▌     | 17421/37962 [40:21:55<47:38:47,  8.35s/it]
 46%|████▌     | 17422/37962 [40:22:03<48:01:19,  8.42s/it]
 46%|████▌     | 17423/37962 [40:22:12<47:42:56,  8.36s/it]
 46%|████▌     | 17424/37962 [40:22:20<47:43:08,  8.36s/it]
 46%|████▌     | 17425/37962 [40:22:28<47:32:45,  8.33s/it]
 46%|████▌     | 17426/37962 [40:22:37<47:36:20,  8.35s/it]
 46%|████▌     | 17427/37962 [40:22:45<47:40:03,  8.36s/it]
 46%|████▌     | 17428/37962 [40:22:53<47:26:07,  8.32s/it]
 46%|████▌     | 17429/37962 [40:23:02<47:22:27,  8.31s/it]
 46%|████▌     | 17430/37962 [40:23:10<46:52:57,  8.22s/it]
                                                           
{'loss': 1.348, 'grad_norm': 20.46158790588379, 'learning_rate': 1.1799274226809736e-05, 'epoch': 0.46}

 46%|████▌     | 17430/37962 [40:23:10<46:52:57,  8.22s/it]
 46%|████▌     | 17431/37962 [40:23:18<46:42:57,  8.19s/it]
 46%|████▌     | 17432/37962 [40:23:26<46:54:03,  8.22s/it]
 46%|████▌     | 17433/37962 [40:23:34<47:13:00,  8.28s/it]
 46%|████▌     | 17434/37962 [40:23:43<46:54:14,  8.23s/it]
 46%|████▌     | 17435/37962 [40:23:51<46:35:32,  8.17s/it]
 46%|████▌     | 17436/37962 [40:23:59<46:46:11,  8.20s/it]
 46%|████▌     | 17437/37962 [40:24:07<46:20:27,  8.13s/it]
 46%|████▌     | 17438/37962 [40:24:15<46:41:39,  8.19s/it]
 46%|████▌     | 17439/37962 [40:24:24<47:08:57,  8.27s/it]
 46%|████▌     | 17440/37962 [40:24:32<47:12:08,  8.28s/it]
                                                           
{'loss': 1.3412, 'grad_norm': 10.182454109191895, 'learning_rate': 1.1790881205427193e-05, 'epoch': 0.46}

 46%|████▌     | 17440/37962 [40:24:32<47:12:08,  8.28s/it]
 46%|████▌     | 17441/37962 [40:24:40<47:06:27,  8.26s/it]
 46%|████▌     | 17442/37962 [40:24:48<47:03:03,  8.25s/it]
 46%|████▌     | 17443/37962 [40:24:57<47:11:57,  8.28s/it]
 46%|████▌     | 17444/37962 [40:25:05<47:14:45,  8.29s/it]
 46%|████▌     | 17445/37962 [40:25:14<47:37:30,  8.36s/it]
 46%|████▌     | 17446/37962 [40:25:22<47:26:50,  8.33s/it]
 46%|████▌     | 17447/37962 [40:25:30<47:13:02,  8.29s/it]
 46%|████▌     | 17448/37962 [40:25:38<47:21:07,  8.31s/it]
 46%|████▌     | 17449/37962 [40:25:46<46:53:18,  8.23s/it]
 46%|████▌     | 17450/37962 [40:25:54<46:29:01,  8.16s/it]
                                                           
{'loss': 1.3644, 'grad_norm': 17.81354331970215, 'learning_rate': 1.1782486880493249e-05, 'epoch': 0.46}

 46%|████▌     | 17450/37962 [40:25:54<46:29:01,  8.16s/it]
 46%|████▌     | 17451/37962 [40:26:03<47:25:48,  8.32s/it]
 46%|████▌     | 17452/37962 [40:26:11<47:22:45,  8.32s/it]
 46%|████▌     | 17453/37962 [40:26:20<47:16:48,  8.30s/it]
 46%|████▌     | 17454/37962 [40:26:28<47:07:48,  8.27s/it]
 46%|████▌     | 17455/37962 [40:26:36<47:06:09,  8.27s/it]
 46%|████▌     | 17456/37962 [40:26:44<46:40:20,  8.19s/it]
 46%|████▌     | 17457/37962 [40:26:52<46:19:44,  8.13s/it]
 46%|████▌     | 17458/37962 [40:27:00<46:04:56,  8.09s/it]
 46%|████▌     | 17459/37962 [40:27:08<45:52:14,  8.05s/it]
 46%|████▌     | 17460/37962 [40:27:16<45:30:09,  7.99s/it]
                                                           
{'loss': 1.3352, 'grad_norm': 9.221073150634766, 'learning_rate': 1.177409125811798e-05, 'epoch': 0.46}

 46%|████▌     | 17460/37962 [40:27:16<45:30:09,  7.99s/it]
 46%|████▌     | 17461/37962 [40:27:24<46:10:05,  8.11s/it]
 46%|████▌     | 17462/37962 [40:27:33<46:32:17,  8.17s/it]
 46%|████▌     | 17463/37962 [40:27:41<46:38:28,  8.19s/it]
 46%|████▌     | 17464/37962 [40:27:49<46:46:16,  8.21s/it]
 46%|████▌     | 17465/37962 [40:27:57<46:54:17,  8.24s/it]
 46%|████▌     | 17466/37962 [40:28:06<47:00:46,  8.26s/it]
 46%|████▌     | 17467/37962 [40:28:14<46:07:24,  8.10s/it]
 46%|████▌     | 17468/37962 [40:28:22<46:40:01,  8.20s/it]
 46%|████▌     | 17469/37962 [40:28:30<46:56:16,  8.25s/it]
 46%|████▌     | 17470/37962 [40:28:39<47:19:44,  8.31s/it]
                                                           
{'loss': 1.4912, 'grad_norm': 6.749099254608154, 'learning_rate': 1.1765694344412417e-05, 'epoch': 0.46}

 46%|████▌     | 17470/37962 [40:28:39<47:19:44,  8.31s/it]
 46%|████▌     | 17471/37962 [40:28:47<47:32:19,  8.35s/it]
 46%|████▌     | 17472/37962 [40:28:55<47:18:04,  8.31s/it]
 46%|████▌     | 17473/37962 [40:29:04<47:17:31,  8.31s/it]
 46%|████▌     | 17474/37962 [40:29:12<47:31:22,  8.35s/it]
 46%|████▌     | 17475/37962 [40:29:20<47:14:31,  8.30s/it]
 46%|████▌     | 17476/37962 [40:29:29<47:08:53,  8.29s/it]
 46%|████▌     | 17477/37962 [40:29:37<47:13:26,  8.30s/it]
 46%|████▌     | 17478/37962 [40:29:45<47:15:09,  8.30s/it]
 46%|████▌     | 17479/37962 [40:29:53<47:07:55,  8.28s/it]
 46%|████▌     | 17480/37962 [40:30:02<46:56:59,  8.25s/it]
                                                           
{'loss': 1.4312, 'grad_norm': 7.90328311920166, 'learning_rate': 1.1757296145488535e-05, 'epoch': 0.46}

 46%|████▌     | 17480/37962 [40:30:02<46:56:59,  8.25s/it]
 46%|████▌     | 17481/37962 [40:30:10<47:01:25,  8.27s/it]
 46%|████▌     | 17482/37962 [40:30:18<47:08:03,  8.29s/it]
 46%|████▌     | 17483/37962 [40:30:27<47:08:32,  8.29s/it]
 46%|████▌     | 17484/37962 [40:30:35<47:32:33,  8.36s/it]
 46%|████▌     | 17485/37962 [40:30:43<46:40:20,  8.21s/it]
 46%|████▌     | 17486/37962 [40:30:51<46:24:55,  8.16s/it]
 46%|████▌     | 17487/37962 [40:30:59<46:39:50,  8.20s/it]
 46%|████▌     | 17488/37962 [40:31:07<46:28:00,  8.17s/it]
 46%|████▌     | 17489/37962 [40:31:15<46:14:28,  8.13s/it]
 46%|████▌     | 17490/37962 [40:31:24<46:44:35,  8.22s/it]
                                                           
{'loss': 1.2442, 'grad_norm': 6.748524188995361, 'learning_rate': 1.174889666745923e-05, 'epoch': 0.46}

 46%|████▌     | 17490/37962 [40:31:24<46:44:35,  8.22s/it]
 46%|████▌     | 17491/37962 [40:31:32<47:06:38,  8.28s/it]
 46%|████▌     | 17492/37962 [40:31:41<47:07:41,  8.29s/it]
 46%|████▌     | 17493/37962 [40:31:49<47:01:44,  8.27s/it]
 46%|████▌     | 17494/37962 [40:31:57<47:11:42,  8.30s/it]
 46%|████▌     | 17495/37962 [40:32:06<47:26:01,  8.34s/it]
 46%|████▌     | 17496/37962 [40:32:14<47:21:20,  8.33s/it]
 46%|████▌     | 17497/37962 [40:32:22<46:17:14,  8.14s/it]
 46%|████▌     | 17498/37962 [40:32:30<46:29:48,  8.18s/it]
 46%|████▌     | 17499/37962 [40:32:38<47:06:02,  8.29s/it]
 46%|████▌     | 17500/37962 [40:32:46<46:38:11,  8.21s/it]
                                                           
{'loss': 1.3163, 'grad_norm': 17.53350067138672, 'learning_rate': 1.1740495916438334e-05, 'epoch': 0.46}

 46%|████▌     | 17500/37962 [40:32:46<46:38:11,  8.21s/it]
 46%|████▌     | 17501/37962 [40:32:55<46:47:07,  8.23s/it]
 46%|████▌     | 17502/37962 [40:33:03<47:15:06,  8.31s/it]
 46%|████▌     | 17503/37962 [40:33:12<47:31:11,  8.36s/it]
 46%|████▌     | 17504/37962 [40:33:20<46:38:19,  8.21s/it]
 46%|████▌     | 17505/37962 [40:33:28<46:47:47,  8.24s/it]
 46%|████▌     | 17506/37962 [40:33:36<46:56:40,  8.26s/it]
 46%|████▌     | 17507/37962 [40:33:44<46:51:18,  8.25s/it]
 46%|████▌     | 17508/37962 [40:33:53<46:45:08,  8.23s/it]
 46%|████▌     | 17509/37962 [40:34:01<46:18:55,  8.15s/it]
 46%|████▌     | 17510/37962 [40:34:09<46:30:02,  8.19s/it]
                                                           
{'loss': 1.3427, 'grad_norm': 17.300928115844727, 'learning_rate': 1.1732093898540616e-05, 'epoch': 0.46}

 46%|████▌     | 17510/37962 [40:34:09<46:30:02,  8.19s/it]
 46%|████▌     | 17511/37962 [40:34:17<47:12:48,  8.31s/it]
 46%|████▌     | 17512/37962 [40:34:26<47:18:59,  8.33s/it]
 46%|████▌     | 17513/37962 [40:34:34<46:53:14,  8.25s/it]
 46%|████▌     | 17514/37962 [40:34:42<46:54:44,  8.26s/it]
 46%|████▌     | 17515/37962 [40:34:50<46:49:48,  8.25s/it]
 46%|████▌     | 17516/37962 [40:34:59<46:54:42,  8.26s/it]
 46%|████▌     | 17517/37962 [40:35:07<46:36:45,  8.21s/it]
 46%|████▌     | 17518/37962 [40:35:15<46:44:33,  8.23s/it]
 46%|████▌     | 17519/37962 [40:35:23<46:23:32,  8.17s/it]
 46%|████▌     | 17520/37962 [40:35:32<47:10:39,  8.31s/it]
                                                           
{'loss': 1.3124, 'grad_norm': 5.9709062576293945, 'learning_rate': 1.1723690619881754e-05, 'epoch': 0.46}

 46%|████▌     | 17520/37962 [40:35:32<47:10:39,  8.31s/it]
 46%|████▌     | 17521/37962 [40:35:40<47:05:42,  8.29s/it]
 46%|████▌     | 17522/37962 [40:35:48<46:43:57,  8.23s/it]
 46%|████▌     | 17523/37962 [40:35:56<46:45:50,  8.24s/it]
 46%|████▌     | 17524/37962 [40:36:04<46:38:28,  8.22s/it]
 46%|████▌     | 17525/37962 [40:36:13<46:44:30,  8.23s/it]
 46%|████▌     | 17526/37962 [40:36:21<46:48:48,  8.25s/it]
 46%|████▌     | 17527/37962 [40:36:29<46:29:11,  8.19s/it]
 46%|████▌     | 17528/37962 [40:36:38<46:54:46,  8.26s/it]
 46%|████▌     | 17529/37962 [40:36:46<46:26:35,  8.18s/it]
 46%|████▌     | 17530/37962 [40:36:53<46:03:44,  8.12s/it]
                                                           
{'loss': 1.4312, 'grad_norm': 6.436551570892334, 'learning_rate': 1.1715286086578352e-05, 'epoch': 0.46}

 46%|████▌     | 17530/37962 [40:36:53<46:03:44,  8.12s/it]
 46%|████▌     | 17531/37962 [40:37:02<46:18:20,  8.16s/it]
 46%|████▌     | 17532/37962 [40:37:10<46:57:36,  8.27s/it]
 46%|████▌     | 17533/37962 [40:37:18<46:50:39,  8.25s/it]
 46%|████▌     | 17534/37962 [40:37:27<46:52:14,  8.26s/it]
 46%|████▌     | 17535/37962 [40:37:35<46:23:29,  8.18s/it]
 46%|████▌     | 17536/37962 [40:37:43<46:48:08,  8.25s/it]
 46%|████▌     | 17537/37962 [40:37:51<46:54:52,  8.27s/it]
 46%|████▌     | 17538/37962 [40:38:00<46:49:46,  8.25s/it]
 46%|████▌     | 17539/37962 [40:38:08<46:44:22,  8.24s/it]
 46%|████▌     | 17540/37962 [40:38:16<46:51:54,  8.26s/it]
                                                           
{'loss': 1.3081, 'grad_norm': 5.119504928588867, 'learning_rate': 1.1706880304747923e-05, 'epoch': 0.46}

 46%|████▌     | 17540/37962 [40:38:16<46:51:54,  8.26s/it]
 46%|████▌     | 17541/37962 [40:38:24<46:23:39,  8.18s/it]
 46%|████▌     | 17542/37962 [40:38:32<46:28:52,  8.19s/it]
 46%|████▌     | 17543/37962 [40:38:41<46:34:48,  8.21s/it]
 46%|████▌     | 17544/37962 [40:38:49<46:40:20,  8.23s/it]
 46%|████▌     | 17545/37962 [40:38:57<46:55:08,  8.27s/it]
 46%|████▌     | 17546/37962 [40:39:05<46:32:21,  8.21s/it]
 46%|████▌     | 17547/37962 [40:39:14<46:31:39,  8.20s/it]
 46%|████▌     | 17548/37962 [40:39:22<46:35:20,  8.22s/it]
 46%|████▌     | 17549/37962 [40:39:30<46:46:47,  8.25s/it]
 46%|████▌     | 17550/37962 [40:39:38<46:15:16,  8.16s/it]
                                                           
{'loss': 1.29, 'grad_norm': 9.107444763183594, 'learning_rate': 1.1698473280508888e-05, 'epoch': 0.46}

 46%|████▌     | 17550/37962 [40:39:38<46:15:16,  8.16s/it]
 46%|████▌     | 17551/37962 [40:39:46<46:34:50,  8.22s/it]
 46%|████▌     | 17552/37962 [40:39:55<46:40:52,  8.23s/it]
 46%|████▌     | 17553/37962 [40:40:03<46:29:51,  8.20s/it]
 46%|████▌     | 17554/37962 [40:40:11<46:41:28,  8.24s/it]
 46%|████▌     | 17555/37962 [40:40:19<46:38:51,  8.23s/it]
 46%|████▌     | 17556/37962 [40:40:28<46:35:57,  8.22s/it]
 46%|████▌     | 17557/37962 [40:40:36<46:56:48,  8.28s/it]
 46%|████▋     | 17558/37962 [40:40:44<46:58:59,  8.29s/it]
 46%|████▋     | 17559/37962 [40:40:52<46:11:10,  8.15s/it]
 46%|████▋     | 17560/37962 [40:41:01<46:40:53,  8.24s/it]
                                                           
{'loss': 1.3612, 'grad_norm': 7.0895094871521, 'learning_rate': 1.1690065019980582e-05, 'epoch': 0.46}

 46%|████▋     | 17560/37962 [40:41:01<46:40:53,  8.24s/it]
 46%|████▋     | 17561/37962 [40:41:09<46:48:32,  8.26s/it]
 46%|████▋     | 17562/37962 [40:41:17<46:51:03,  8.27s/it]
 46%|████▋     | 17563/37962 [40:41:25<46:46:19,  8.25s/it]
 46%|████▋     | 17564/37962 [40:41:34<46:51:47,  8.27s/it]
 46%|████▋     | 17565/37962 [40:41:42<46:24:51,  8.19s/it]
 46%|████▋     | 17566/37962 [40:41:50<46:36:03,  8.23s/it]
 46%|████▋     | 17567/37962 [40:41:58<46:37:13,  8.23s/it]
 46%|████▋     | 17568/37962 [40:42:07<46:41:48,  8.24s/it]
 46%|████▋     | 17569/37962 [40:42:15<47:05:53,  8.31s/it]
 46%|████▋     | 17570/37962 [40:42:23<47:15:14,  8.34s/it]
                                                           
{'loss': 1.361, 'grad_norm': 5.812620639801025, 'learning_rate': 1.1681655529283218e-05, 'epoch': 0.46}

 46%|████▋     | 17570/37962 [40:42:23<47:15:14,  8.34s/it]
 46%|████▋     | 17571/37962 [40:42:32<47:12:51,  8.34s/it]
 46%|████▋     | 17572/37962 [40:42:40<46:37:57,  8.23s/it]
 46%|████▋     | 17573/37962 [40:42:48<46:54:47,  8.28s/it]
 46%|████▋     | 17574/37962 [40:42:56<46:50:39,  8.27s/it]
 46%|████▋     | 17575/37962 [40:43:05<47:07:59,  8.32s/it]
 46%|████▋     | 17576/37962 [40:43:13<47:05:02,  8.31s/it]
 46%|████▋     | 17577/37962 [40:43:21<46:55:56,  8.29s/it]
 46%|████▋     | 17578/37962 [40:43:30<47:08:30,  8.33s/it]
 46%|████▋     | 17579/37962 [40:43:38<47:17:45,  8.35s/it]
 46%|████▋     | 17580/37962 [40:43:47<47:35:58,  8.41s/it]
                                                           
{'loss': 1.3411, 'grad_norm': 15.018234252929688, 'learning_rate': 1.1673244814537933e-05, 'epoch': 0.46}

 46%|████▋     | 17580/37962 [40:43:47<47:35:58,  8.41s/it]
 46%|████▋     | 17581/37962 [40:43:55<47:22:47,  8.37s/it]
 46%|████▋     | 17582/37962 [40:44:03<47:30:00,  8.39s/it]
 46%|████▋     | 17583/37962 [40:44:11<46:48:35,  8.27s/it]
 46%|████▋     | 17584/37962 [40:44:19<46:25:28,  8.20s/it]
 46%|████▋     | 17585/37962 [40:44:28<46:17:04,  8.18s/it]
 46%|████▋     | 17586/37962 [40:44:36<46:38:12,  8.24s/it]
 46%|████▋     | 17587/37962 [40:44:44<45:55:36,  8.11s/it]
 46%|████▋     | 17588/37962 [40:44:52<45:30:18,  8.04s/it]
 46%|████▋     | 17589/37962 [40:45:00<45:55:31,  8.12s/it]
 46%|████▋     | 17590/37962 [40:45:08<46:30:43,  8.22s/it]
                                                           
{'loss': 1.4037, 'grad_norm': 15.493953704833984, 'learning_rate': 1.1664832881866731e-05, 'epoch': 0.46}

 46%|████▋     | 17590/37962 [40:45:08<46:30:43,  8.22s/it]
 46%|████▋     | 17591/37962 [40:45:17<46:45:28,  8.26s/it]
 46%|████▋     | 17592/37962 [40:45:25<46:36:01,  8.24s/it]
 46%|████▋     | 17593/37962 [40:45:33<46:37:52,  8.24s/it]
 46%|████▋     | 17594/37962 [40:45:42<46:48:19,  8.27s/it]
 46%|████▋     | 17595/37962 [40:45:50<47:05:50,  8.32s/it]
 46%|████▋     | 17596/37962 [40:45:58<47:09:52,  8.34s/it]
 46%|████▋     | 17597/37962 [40:46:07<46:56:22,  8.30s/it]
 46%|████▋     | 17598/37962 [40:46:22<58:34:22, 10.35s/it]
 46%|████▋     | 17599/37962 [40:46:30<55:22:46,  9.79s/it]
 46%|████▋     | 17600/37962 [40:46:46<64:49:42, 11.46s/it]
                                                           
{'loss': 1.2953, 'grad_norm': 8.683887481689453, 'learning_rate': 1.1656419737392514e-05, 'epoch': 0.46}

 46%|████▋     | 17600/37962 [40:46:46<64:49:42, 11.46s/it]
 46%|████▋     | 17601/37962 [40:46:54<59:16:42, 10.48s/it]
 46%|████▋     | 17602/37962 [40:47:02<55:33:03,  9.82s/it]
 46%|████▋     | 17603/37962 [40:47:10<52:51:29,  9.35s/it]
 46%|████▋     | 17604/37962 [40:47:19<51:03:59,  9.03s/it]
 46%|████▋     | 17605/37962 [40:47:27<49:48:08,  8.81s/it]
 46%|████▋     | 17606/37962 [40:47:35<48:35:54,  8.59s/it]
 46%|████▋     | 17607/37962 [40:47:43<48:26:55,  8.57s/it]
 46%|████▋     | 17608/37962 [40:47:52<47:52:50,  8.47s/it]
 46%|████▋     | 17609/37962 [40:48:00<47:30:02,  8.40s/it]
 46%|████▋     | 17610/37962 [40:48:08<47:18:45,  8.37s/it]
                                                           
{'loss': 1.3635, 'grad_norm': 4.542469501495361, 'learning_rate': 1.1648005387239072e-05, 'epoch': 0.46}

 46%|████▋     | 17610/37962 [40:48:08<47:18:45,  8.37s/it]
 46%|████▋     | 17611/37962 [40:48:17<47:26:30,  8.39s/it]
 46%|████▋     | 17612/37962 [40:48:25<47:11:03,  8.35s/it]
 46%|████▋     | 17613/37962 [40:48:33<46:45:22,  8.27s/it]
 46%|████▋     | 17614/37962 [40:48:41<46:47:14,  8.28s/it]
 46%|████▋     | 17615/37962 [40:48:49<46:21:45,  8.20s/it]
 46%|████▋     | 17616/37962 [40:48:58<46:28:43,  8.22s/it]
 46%|████▋     | 17617/37962 [40:49:06<46:31:50,  8.23s/it]
 46%|████▋     | 17618/37962 [40:49:14<46:00:00,  8.14s/it]
 46%|████▋     | 17619/37962 [40:49:22<46:08:32,  8.17s/it]
 46%|████▋     | 17620/37962 [40:49:30<46:33:19,  8.24s/it]
                                                           
{'loss': 1.4093, 'grad_norm': 11.061468124389648, 'learning_rate': 1.1639589837531056e-05, 'epoch': 0.46}

 46%|████▋     | 17620/37962 [40:49:30<46:33:19,  8.24s/it]
 46%|████▋     | 17621/37962 [40:49:38<46:08:29,  8.17s/it]
 46%|████▋     | 17622/37962 [40:49:47<46:13:28,  8.18s/it]
 46%|████▋     | 17623/37962 [40:49:55<46:32:49,  8.24s/it]
 46%|████▋     | 17624/37962 [40:50:03<46:36:32,  8.25s/it]
 46%|████▋     | 17625/37962 [40:50:12<47:06:50,  8.34s/it]
 46%|████▋     | 17626/37962 [40:50:20<46:02:34,  8.15s/it]
 46%|████▋     | 17627/37962 [40:50:28<46:00:02,  8.14s/it]
 46%|████▋     | 17628/37962 [40:50:36<46:43:55,  8.27s/it]
 46%|████▋     | 17629/37962 [40:50:44<46:35:40,  8.25s/it]
 46%|████▋     | 17630/37962 [40:50:52<46:11:56,  8.18s/it]
                                                           
{'loss': 1.2486, 'grad_norm': 13.695755958557129, 'learning_rate': 1.1631173094394e-05, 'epoch': 0.46}

 46%|████▋     | 17630/37962 [40:50:52<46:11:56,  8.18s/it]
 46%|████▋     | 17631/37962 [40:51:00<45:53:34,  8.13s/it]
 46%|████▋     | 17632/37962 [40:51:09<46:14:39,  8.19s/it]
 46%|████▋     | 17633/37962 [40:51:17<46:38:39,  8.26s/it]
 46%|████▋     | 17634/37962 [40:51:25<46:00:33,  8.15s/it]
 46%|████▋     | 17635/37962 [40:51:33<46:08:30,  8.17s/it]
 46%|████▋     | 17636/37962 [40:51:42<46:31:59,  8.24s/it]
 46%|████▋     | 17637/37962 [40:51:50<46:26:47,  8.23s/it]
 46%|████▋     | 17638/37962 [40:51:58<46:46:37,  8.29s/it]
 46%|████▋     | 17639/37962 [40:52:06<46:20:08,  8.21s/it]
 46%|████▋     | 17640/37962 [40:52:15<46:35:48,  8.25s/it]
                                                           
{'loss': 1.3665, 'grad_norm': 7.886596202850342, 'learning_rate': 1.1622755163954306e-05, 'epoch': 0.46}

 46%|████▋     | 17640/37962 [40:52:15<46:35:48,  8.25s/it]
 46%|████▋     | 17641/37962 [40:52:23<46:35:13,  8.25s/it]
 46%|████▋     | 17642/37962 [40:52:31<46:40:05,  8.27s/it]
 46%|████▋     | 17643/37962 [40:52:40<46:37:57,  8.26s/it]
 46%|████▋     | 17644/37962 [40:52:48<46:43:14,  8.28s/it]
 46%|████▋     | 17645/37962 [40:52:56<46:50:22,  8.30s/it]
 46%|████▋     | 17646/37962 [40:53:04<46:44:14,  8.28s/it]
 46%|████▋     | 17647/37962 [40:53:13<46:29:55,  8.24s/it]
 46%|████▋     | 17648/37962 [40:53:21<45:59:56,  8.15s/it]
 46%|████▋     | 17649/37962 [40:53:29<46:06:57,  8.17s/it]
 46%|████▋     | 17650/37962 [40:53:37<46:17:38,  8.20s/it]
                                                           
{'loss': 1.4029, 'grad_norm': 8.5760498046875, 'learning_rate': 1.1614336052339242e-05, 'epoch': 0.46}

 46%|████▋     | 17650/37962 [40:53:37<46:17:38,  8.20s/it]
 46%|████▋     | 17651/37962 [40:53:45<46:09:44,  8.18s/it]
 46%|████▋     | 17652/37962 [40:53:53<45:58:58,  8.15s/it]
 47%|████▋     | 17653/37962 [40:54:01<45:37:44,  8.09s/it]
 47%|████▋     | 17654/37962 [40:54:09<45:48:51,  8.12s/it]
 47%|████▋     | 17655/37962 [40:54:18<45:58:45,  8.15s/it]
 47%|████▋     | 17656/37962 [40:54:26<46:11:41,  8.19s/it]
 47%|████▋     | 17657/37962 [40:54:34<46:34:26,  8.26s/it]
 47%|████▋     | 17658/37962 [40:54:42<46:24:39,  8.23s/it]
 47%|████▋     | 17659/37962 [40:54:51<46:44:17,  8.29s/it]
 47%|████▋     | 17660/37962 [40:54:59<46:28:52,  8.24s/it]
                                                           
{'loss': 1.4252, 'grad_norm': 7.391860008239746, 'learning_rate': 1.1605915765676931e-05, 'epoch': 0.47}

 47%|████▋     | 17660/37962 [40:54:59<46:28:52,  8.24s/it]
 47%|████▋     | 17661/37962 [40:55:07<46:27:19,  8.24s/it]
 47%|████▋     | 17662/37962 [40:55:16<46:35:55,  8.26s/it]
 47%|████▋     | 17663/37962 [40:55:24<46:44:59,  8.29s/it]
 47%|████▋     | 17664/37962 [40:55:32<46:50:32,  8.31s/it]
 47%|████▋     | 17665/37962 [40:55:40<46:04:52,  8.17s/it]
 47%|████▋     | 17666/37962 [40:55:49<46:29:14,  8.25s/it]
 47%|████▋     | 17667/37962 [40:55:57<46:01:58,  8.17s/it]
 47%|████▋     | 17668/37962 [40:56:05<46:18:53,  8.22s/it]
 47%|████▋     | 17669/37962 [40:56:13<45:40:18,  8.10s/it]
 47%|████▋     | 17670/37962 [40:56:21<45:47:16,  8.12s/it]
                                                           
{'loss': 1.3425, 'grad_norm': 10.017147064208984, 'learning_rate': 1.1597494310096354e-05, 'epoch': 0.47}

 47%|████▋     | 17670/37962 [40:56:21<45:47:16,  8.12s/it]
 47%|████▋     | 17671/37962 [40:56:29<45:43:56,  8.11s/it]
 47%|████▋     | 17672/37962 [40:56:37<45:56:48,  8.15s/it]
 47%|████▋     | 17673/37962 [40:56:46<46:20:07,  8.22s/it]
 47%|████▋     | 17674/37962 [40:56:54<45:56:16,  8.15s/it]
 47%|████▋     | 17675/37962 [40:57:02<45:59:52,  8.16s/it]
 47%|████▋     | 17676/37962 [40:57:10<46:10:15,  8.19s/it]
 47%|████▋     | 17677/37962 [40:57:18<46:03:41,  8.17s/it]
 47%|████▋     | 17678/37962 [40:57:27<46:23:03,  8.23s/it]
 47%|████▋     | 17679/37962 [40:57:35<46:34:34,  8.27s/it]
 47%|████▋     | 17680/37962 [40:57:43<46:50:35,  8.31s/it]
                                                           
{'loss': 1.3727, 'grad_norm': 9.948430061340332, 'learning_rate': 1.1589071691727345e-05, 'epoch': 0.47}

 47%|████▋     | 17680/37962 [40:57:43<46:50:35,  8.31s/it]
 47%|████▋     | 17681/37962 [40:57:52<46:43:10,  8.29s/it]
 47%|████▋     | 17682/37962 [40:58:00<46:21:24,  8.23s/it]
 47%|████▋     | 17683/37962 [40:58:08<46:35:05,  8.27s/it]
 47%|████▋     | 17684/37962 [40:58:16<46:32:16,  8.26s/it]
 47%|████▋     | 17685/37962 [40:58:24<46:26:47,  8.25s/it]
 47%|████▋     | 17686/37962 [40:58:33<46:31:42,  8.26s/it]
 47%|████▋     | 17687/37962 [40:58:41<46:44:31,  8.30s/it]
 47%|████▋     | 17688/37962 [40:58:49<46:40:11,  8.29s/it]
 47%|████▋     | 17689/37962 [40:58:58<46:43:45,  8.30s/it]
 47%|████▋     | 17690/37962 [40:59:06<47:09:55,  8.38s/it]
                                                           
{'loss': 1.3159, 'grad_norm': 8.862388610839844, 'learning_rate': 1.158064791670058e-05, 'epoch': 0.47}

 47%|████▋     | 17690/37962 [40:59:06<47:09:55,  8.38s/it]
 47%|████▋     | 17691/37962 [40:59:15<47:05:20,  8.36s/it]
 47%|████▋     | 17692/37962 [40:59:23<47:12:24,  8.38s/it]
 47%|████▋     | 17693/37962 [40:59:31<47:09:08,  8.37s/it]
 47%|████▋     | 17694/37962 [40:59:39<46:34:39,  8.27s/it]
 47%|████▋     | 17695/37962 [40:59:48<46:46:16,  8.31s/it]
 47%|████▋     | 17696/37962 [40:59:56<46:37:40,  8.28s/it]
 47%|████▋     | 17697/37962 [41:00:04<46:45:40,  8.31s/it]
 47%|████▋     | 17698/37962 [41:00:12<46:12:34,  8.21s/it]
 47%|████▋     | 17699/37962 [41:00:21<46:17:51,  8.23s/it]
 47%|████▋     | 17700/37962 [41:00:29<46:18:37,  8.23s/it]
                                                           
{'loss': 1.3681, 'grad_norm': 17.487131118774414, 'learning_rate': 1.157222299114758e-05, 'epoch': 0.47}

 47%|████▋     | 17700/37962 [41:00:29<46:18:37,  8.23s/it]
 47%|████▋     | 17701/37962 [41:00:37<46:04:08,  8.19s/it]
 47%|████▋     | 17702/37962 [41:00:45<46:06:00,  8.19s/it]
 47%|████▋     | 17703/37962 [41:00:53<46:11:14,  8.21s/it]
 47%|████▋     | 17704/37962 [41:01:02<46:23:28,  8.24s/it]
 47%|████▋     | 17705/37962 [41:01:10<45:50:19,  8.15s/it]
 47%|████▋     | 17706/37962 [41:01:18<46:01:02,  8.18s/it]
 47%|████▋     | 17707/37962 [41:01:26<46:22:48,  8.24s/it]
 47%|████▋     | 17708/37962 [41:01:35<46:36:22,  8.28s/it]
 47%|████▋     | 17709/37962 [41:01:43<46:43:37,  8.31s/it]
 47%|████▋     | 17710/37962 [41:01:52<47:01:08,  8.36s/it]
                                                           
{'loss': 1.3573, 'grad_norm': 7.854982852935791, 'learning_rate': 1.15637969212007e-05, 'epoch': 0.47}

 47%|████▋     | 17710/37962 [41:01:52<47:01:08,  8.36s/it]
 47%|████▋     | 17711/37962 [41:02:00<46:55:17,  8.34s/it]
 47%|████▋     | 17712/37962 [41:02:08<47:08:58,  8.38s/it]
 47%|████▋     | 17713/37962 [41:02:17<46:59:30,  8.35s/it]
 47%|████▋     | 17714/37962 [41:02:25<47:07:05,  8.38s/it]
 47%|████▋     | 17715/37962 [41:02:33<46:03:04,  8.19s/it]
 47%|████▋     | 17716/37962 [41:02:41<46:11:05,  8.21s/it]
 47%|████▋     | 17717/37962 [41:02:49<46:29:57,  8.27s/it]
 47%|████▋     | 17718/37962 [41:02:58<46:55:59,  8.35s/it]
 47%|████▋     | 17719/37962 [41:03:06<46:44:48,  8.31s/it]
 47%|████▋     | 17720/37962 [41:03:14<46:30:38,  8.27s/it]
                                                           
{'loss': 1.2794, 'grad_norm': 11.474493026733398, 'learning_rate': 1.1555369712993132e-05, 'epoch': 0.47}

 47%|████▋     | 17720/37962 [41:03:14<46:30:38,  8.27s/it]
 47%|████▋     | 17721/37962 [41:03:23<46:33:36,  8.28s/it]
 47%|████▋     | 17722/37962 [41:03:31<46:28:53,  8.27s/it]
 47%|████▋     | 17723/37962 [41:03:39<46:17:22,  8.23s/it]
 47%|████▋     | 17724/37962 [41:03:47<45:56:07,  8.17s/it]
 47%|████▋     | 17725/37962 [41:03:55<46:11:30,  8.22s/it]
 47%|████▋     | 17726/37962 [41:04:04<46:12:24,  8.22s/it]
 47%|████▋     | 17727/37962 [41:04:12<46:11:16,  8.22s/it]
 47%|████▋     | 17728/37962 [41:04:20<46:05:46,  8.20s/it]
 47%|████▋     | 17729/37962 [41:04:28<45:47:28,  8.15s/it]
 47%|████▋     | 17730/37962 [41:04:37<46:34:07,  8.29s/it]
                                                           
{'loss': 1.3269, 'grad_norm': 15.570900917053223, 'learning_rate': 1.1546941372658896e-05, 'epoch': 0.47}

 47%|████▋     | 17730/37962 [41:04:37<46:34:07,  8.29s/it]
 47%|████▋     | 17731/37962 [41:04:45<46:08:22,  8.21s/it]
 47%|████▋     | 17732/37962 [41:04:53<46:08:28,  8.21s/it]
 47%|████▋     | 17733/37962 [41:05:01<46:26:06,  8.26s/it]
 47%|████▋     | 17734/37962 [41:05:09<46:17:56,  8.24s/it]
 47%|████▋     | 17735/37962 [41:05:18<46:17:44,  8.24s/it]
 47%|████▋     | 17736/37962 [41:05:26<46:15:40,  8.23s/it]
 47%|████▋     | 17737/37962 [41:05:34<46:40:52,  8.31s/it]
 47%|████▋     | 17738/37962 [41:05:43<46:27:05,  8.27s/it]
 47%|████▋     | 17739/37962 [41:05:51<46:52:47,  8.35s/it]
 47%|████▋     | 17740/37962 [41:05:59<46:23:44,  8.26s/it]
                                                           
{'loss': 1.3744, 'grad_norm': 4.000212669372559, 'learning_rate': 1.1538511906332834e-05, 'epoch': 0.47}

 47%|████▋     | 17740/37962 [41:05:59<46:23:44,  8.26s/it]
 47%|████▋     | 17741/37962 [41:06:07<45:50:47,  8.16s/it]
 47%|████▋     | 17742/37962 [41:06:15<46:01:00,  8.19s/it]
 47%|████▋     | 17743/37962 [41:06:24<46:06:15,  8.21s/it]
 47%|████▋     | 17744/37962 [41:06:32<46:14:05,  8.23s/it]
 47%|████▋     | 17745/37962 [41:06:40<46:18:40,  8.25s/it]
 47%|████▋     | 17746/37962 [41:06:49<46:47:01,  8.33s/it]
 47%|████▋     | 17747/37962 [41:06:57<46:19:55,  8.25s/it]
 47%|████▋     | 17748/37962 [41:07:05<46:23:48,  8.26s/it]
 47%|████▋     | 17749/37962 [41:07:13<46:17:27,  8.24s/it]
 47%|████▋     | 17750/37962 [41:07:22<46:23:28,  8.26s/it]
                                                           
{'loss': 1.294, 'grad_norm': 2.809767246246338, 'learning_rate': 1.1530081320150605e-05, 'epoch': 0.47}

 47%|████▋     | 17750/37962 [41:07:22<46:23:28,  8.26s/it]
 47%|████▋     | 17751/37962 [41:07:30<46:24:19,  8.27s/it]
 47%|████▋     | 17752/37962 [41:07:38<46:39:50,  8.31s/it]
 47%|████▋     | 17753/37962 [41:07:46<46:28:37,  8.28s/it]
 47%|████▋     | 17754/37962 [41:08:02<59:17:22, 10.56s/it]
 47%|████▋     | 17755/37962 [41:08:10<55:11:06,  9.83s/it]
 47%|████▋     | 17756/37962 [41:08:19<52:52:44,  9.42s/it]
 47%|████▋     | 17757/37962 [41:08:27<51:04:08,  9.10s/it]
 47%|████▋     | 17758/37962 [41:08:36<50:21:28,  8.97s/it]
 47%|████▋     | 17759/37962 [41:08:44<49:13:52,  8.77s/it]
 47%|████▋     | 17760/37962 [41:08:53<48:39:44,  8.67s/it]
                                                           
{'loss': 1.3266, 'grad_norm': 10.884756088256836, 'learning_rate': 1.1521649620248694e-05, 'epoch': 0.47}

 47%|████▋     | 17760/37962 [41:08:53<48:39:44,  8.67s/it]
 47%|████▋     | 17761/37962 [41:09:01<47:59:51,  8.55s/it]
 47%|████▋     | 17762/37962 [41:09:10<47:55:21,  8.54s/it]
 47%|████▋     | 17763/37962 [41:09:18<47:33:04,  8.47s/it]
 47%|████▋     | 17764/37962 [41:09:26<47:02:39,  8.38s/it]
 47%|████▋     | 17765/37962 [41:09:35<47:20:48,  8.44s/it]
 47%|████▋     | 17766/37962 [41:09:43<47:14:48,  8.42s/it]
 47%|████▋     | 17767/37962 [41:09:51<46:40:16,  8.32s/it]
 47%|████▋     | 17768/37962 [41:09:59<45:56:15,  8.19s/it]
 47%|████▋     | 17769/37962 [41:10:07<46:01:14,  8.20s/it]
 47%|████▋     | 17770/37962 [41:10:16<46:22:24,  8.27s/it]
                                                           
{'loss': 1.2521, 'grad_norm': 5.4316725730896, 'learning_rate': 1.1513216812764383e-05, 'epoch': 0.47}

 47%|████▋     | 17770/37962 [41:10:16<46:22:24,  8.27s/it]
 47%|████▋     | 17771/37962 [41:10:24<46:40:17,  8.32s/it]
 47%|████▋     | 17772/37962 [41:10:32<46:48:18,  8.35s/it]
 47%|████▋     | 17773/37962 [41:10:41<46:35:34,  8.31s/it]
 47%|████▋     | 17774/37962 [41:10:49<46:50:39,  8.35s/it]
 47%|████▋     | 17775/37962 [41:10:57<46:50:50,  8.35s/it]
 47%|████▋     | 17776/37962 [41:11:06<46:48:48,  8.35s/it]
 47%|████▋     | 17777/37962 [41:11:14<46:41:42,  8.33s/it]
 47%|████▋     | 17778/37962 [41:11:22<46:05:12,  8.22s/it]
 47%|████▋     | 17779/37962 [41:11:30<46:24:14,  8.28s/it]
 47%|████▋     | 17780/37962 [41:11:39<46:04:40,  8.22s/it]
                                                           
{'loss': 1.3743, 'grad_norm': 10.219120979309082, 'learning_rate': 1.1504782903835769e-05, 'epoch': 0.47}

 47%|████▋     | 17780/37962 [41:11:39<46:04:40,  8.22s/it]
 47%|████▋     | 17781/37962 [41:11:47<46:03:35,  8.22s/it]
 47%|████▋     | 17782/37962 [41:11:55<46:01:05,  8.21s/it]
 47%|████▋     | 17783/37962 [41:12:03<46:30:49,  8.30s/it]
 47%|████▋     | 17784/37962 [41:12:12<46:12:09,  8.24s/it]
 47%|████▋     | 17785/37962 [41:12:20<46:06:13,  8.23s/it]
 47%|████▋     | 17786/37962 [41:12:28<46:09:56,  8.24s/it]
 47%|████▋     | 17787/37962 [41:12:36<46:13:47,  8.25s/it]
 47%|████▋     | 17788/37962 [41:12:44<45:58:14,  8.20s/it]
 47%|████▋     | 17789/37962 [41:12:53<45:56:27,  8.20s/it]
 47%|████▋     | 17790/37962 [41:13:01<45:52:15,  8.19s/it]
                                                           
{'loss': 1.3234, 'grad_norm': 19.54829978942871, 'learning_rate': 1.1496347899601752e-05, 'epoch': 0.47}

 47%|████▋     | 17790/37962 [41:13:01<45:52:15,  8.19s/it]
 47%|████▋     | 17791/37962 [41:13:09<46:08:15,  8.23s/it]
 47%|████▋     | 17792/37962 [41:13:17<46:18:25,  8.27s/it]
 47%|████▋     | 17793/37962 [41:13:26<46:24:00,  8.28s/it]
 47%|████▋     | 17794/37962 [41:13:34<46:20:18,  8.27s/it]
 47%|████▋     | 17795/37962 [41:13:42<46:26:33,  8.29s/it]
 47%|████▋     | 17796/37962 [41:13:51<46:18:33,  8.27s/it]
 47%|████▋     | 17797/37962 [41:13:58<45:43:34,  8.16s/it]
 47%|████▋     | 17798/37962 [41:14:07<46:09:05,  8.24s/it]
 47%|████▋     | 17799/37962 [41:14:15<46:21:21,  8.28s/it]
 47%|████▋     | 17800/37962 [41:14:24<46:37:24,  8.32s/it]
                                                           
{'loss': 1.4258, 'grad_norm': 13.243267059326172, 'learning_rate': 1.1487911806202016e-05, 'epoch': 0.47}

 47%|████▋     | 17800/37962 [41:14:24<46:37:24,  8.32s/it]
 47%|████▋     | 17801/37962 [41:14:32<46:03:15,  8.22s/it]
 47%|████▋     | 17802/37962 [41:14:40<45:56:54,  8.21s/it]
 47%|████▋     | 17803/37962 [41:14:48<45:39:17,  8.15s/it]
 47%|████▋     | 17804/37962 [41:14:56<46:06:30,  8.23s/it]
 47%|████▋     | 17805/37962 [41:15:05<46:10:44,  8.25s/it]
 47%|████▋     | 17806/37962 [41:15:13<46:02:34,  8.22s/it]
 47%|████▋     | 17807/37962 [41:15:21<46:03:18,  8.23s/it]
 47%|████▋     | 17808/37962 [41:15:29<45:53:31,  8.20s/it]
 47%|████▋     | 17809/37962 [41:15:37<46:13:22,  8.26s/it]
 47%|████▋     | 17810/37962 [41:15:46<46:20:52,  8.28s/it]
                                                           
{'loss': 1.3668, 'grad_norm': 5.31924295425415, 'learning_rate': 1.1479474629777051e-05, 'epoch': 0.47}

 47%|████▋     | 17810/37962 [41:15:46<46:20:52,  8.28s/it]
 47%|████▋     | 17811/37962 [41:15:54<45:43:32,  8.17s/it]
 47%|████▋     | 17812/37962 [41:16:02<45:24:05,  8.11s/it]
 47%|████▋     | 17813/37962 [41:16:10<45:44:21,  8.17s/it]
 47%|████▋     | 17814/37962 [41:16:18<45:52:29,  8.20s/it]
 47%|████▋     | 17815/37962 [41:16:27<46:07:17,  8.24s/it]
 47%|████▋     | 17816/37962 [41:16:35<46:10:08,  8.25s/it]
 47%|████▋     | 17817/37962 [41:16:43<46:11:12,  8.25s/it]
 47%|████▋     | 17818/37962 [41:16:51<46:12:57,  8.26s/it]
 47%|████▋     | 17819/37962 [41:16:59<45:54:24,  8.20s/it]
 47%|████▋     | 17820/37962 [41:17:08<46:08:45,  8.25s/it]
                                                           
{'loss': 1.4023, 'grad_norm': 10.356910705566406, 'learning_rate': 1.1471036376468138e-05, 'epoch': 0.47}

 47%|████▋     | 17820/37962 [41:17:08<46:08:45,  8.25s/it]
 47%|████▋     | 17821/37962 [41:17:16<46:09:28,  8.25s/it]
 47%|████▋     | 17822/37962 [41:17:24<46:23:50,  8.29s/it]
 47%|████▋     | 17823/37962 [41:17:33<46:11:13,  8.26s/it]
 47%|████▋     | 17824/37962 [41:17:41<46:12:32,  8.26s/it]
 47%|████▋     | 17825/37962 [41:17:49<45:50:04,  8.19s/it]
 47%|████▋     | 17826/37962 [41:17:57<45:46:11,  8.18s/it]
 47%|████▋     | 17827/37962 [41:18:05<45:50:44,  8.20s/it]
 47%|████▋     | 17828/37962 [41:18:14<46:05:25,  8.24s/it]
 47%|████▋     | 17829/37962 [41:18:22<45:55:09,  8.21s/it]
 47%|████▋     | 17830/37962 [41:18:30<46:29:48,  8.31s/it]
                                                           
{'loss': 1.4303, 'grad_norm': 9.739974975585938, 'learning_rate': 1.146259705241733e-05, 'epoch': 0.47}

 47%|████▋     | 17830/37962 [41:18:30<46:29:48,  8.31s/it]
 47%|████▋     | 17831/37962 [41:18:39<46:22:46,  8.29s/it]
 47%|████▋     | 17832/37962 [41:18:47<46:11:12,  8.26s/it]
 47%|████▋     | 17833/37962 [41:18:55<46:35:31,  8.33s/it]
 47%|████▋     | 17834/37962 [41:19:04<46:47:39,  8.37s/it]
 47%|████▋     | 17835/37962 [41:19:12<46:41:28,  8.35s/it]
 47%|████▋     | 17836/37962 [41:19:20<46:29:33,  8.32s/it]
 47%|████▋     | 17837/37962 [41:19:29<46:27:33,  8.31s/it]
 47%|████▋     | 17838/37962 [41:19:37<45:47:00,  8.19s/it]
 47%|████▋     | 17839/37962 [41:19:45<45:57:10,  8.22s/it]
 47%|████▋     | 17840/37962 [41:19:53<45:34:06,  8.15s/it]
                                                           
{'loss': 1.3083, 'grad_norm': 13.037135124206543, 'learning_rate': 1.1454156663767465e-05, 'epoch': 0.47}

 47%|████▋     | 17840/37962 [41:19:53<45:34:06,  8.15s/it]
 47%|████▋     | 17841/37962 [41:20:01<45:13:26,  8.09s/it]
 47%|████▋     | 17842/37962 [41:20:09<45:56:11,  8.22s/it]
 47%|████▋     | 17843/37962 [41:20:18<46:16:20,  8.28s/it]
 47%|████▋     | 17844/37962 [41:20:26<46:20:00,  8.29s/it]
 47%|████▋     | 17845/37962 [41:20:34<46:20:51,  8.29s/it]
 47%|████▋     | 17846/37962 [41:20:43<46:22:39,  8.30s/it]
 47%|████▋     | 17847/37962 [41:20:51<46:10:02,  8.26s/it]
 47%|████▋     | 17848/37962 [41:20:59<45:29:25,  8.14s/it]
 47%|████▋     | 17849/37962 [41:21:07<45:41:21,  8.18s/it]
 47%|████▋     | 17850/37962 [41:21:15<45:57:32,  8.23s/it]
                                                           
{'loss': 1.3051, 'grad_norm': 10.98000717163086, 'learning_rate': 1.1445715216662155e-05, 'epoch': 0.47}

 47%|████▋     | 17850/37962 [41:21:15<45:57:32,  8.23s/it]
 47%|████▋     | 17851/37962 [41:21:24<46:03:28,  8.24s/it]
 47%|████▋     | 17852/37962 [41:21:32<46:01:47,  8.24s/it]
 47%|████▋     | 17853/37962 [41:21:40<45:58:00,  8.23s/it]
 47%|████▋     | 17854/37962 [41:21:48<46:16:59,  8.29s/it]
 47%|████▋     | 17855/37962 [41:21:57<46:12:05,  8.27s/it]
 47%|████▋     | 17856/37962 [41:22:05<46:18:18,  8.29s/it]
 47%|████▋     | 17857/37962 [41:22:13<45:53:25,  8.22s/it]
 47%|████▋     | 17858/37962 [41:22:21<45:53:30,  8.22s/it]
 47%|████▋     | 17859/37962 [41:22:30<46:15:02,  8.28s/it]
 47%|████▋     | 17860/37962 [41:22:38<45:51:58,  8.21s/it]
                                                           
{'loss': 1.3033, 'grad_norm': 2.5696017742156982, 'learning_rate': 1.1437272717245782e-05, 'epoch': 0.47}

 47%|████▋     | 17860/37962 [41:22:38<45:51:58,  8.21s/it]
 47%|████▋     | 17861/37962 [41:22:46<45:47:15,  8.20s/it]
 47%|████▋     | 17862/37962 [41:22:54<46:08:20,  8.26s/it]
 47%|████▋     | 17863/37962 [41:23:03<46:41:58,  8.36s/it]
 47%|████▋     | 17864/37962 [41:23:11<46:35:08,  8.34s/it]
 47%|████▋     | 17865/37962 [41:23:19<46:27:11,  8.32s/it]
 47%|████▋     | 17866/37962 [41:23:28<46:30:43,  8.33s/it]
 47%|████▋     | 17867/37962 [41:23:36<46:52:34,  8.40s/it]
 47%|████▋     | 17868/37962 [41:23:45<46:45:31,  8.38s/it]
 47%|████▋     | 17869/37962 [41:23:53<46:44:22,  8.37s/it]
 47%|████▋     | 17870/37962 [41:24:01<46:46:01,  8.38s/it]
                                                           
{'loss': 1.3264, 'grad_norm': 7.220703125, 'learning_rate': 1.1428829171663501e-05, 'epoch': 0.47}

 47%|████▋     | 17870/37962 [41:24:01<46:46:01,  8.38s/it]
 47%|████▋     | 17871/37962 [41:24:10<46:51:02,  8.39s/it]
 47%|████▋     | 17872/37962 [41:24:18<46:41:29,  8.37s/it]
 47%|████▋     | 17873/37962 [41:24:26<46:23:27,  8.31s/it]
 47%|████▋     | 17874/37962 [41:24:35<46:27:20,  8.33s/it]
 47%|████▋     | 17875/37962 [41:24:43<46:37:39,  8.36s/it]
 47%|████▋     | 17876/37962 [41:24:52<46:53:43,  8.41s/it]
 47%|████▋     | 17877/37962 [41:25:00<46:26:21,  8.32s/it]
 47%|████▋     | 17878/37962 [41:25:08<46:33:22,  8.35s/it]
 47%|████▋     | 17879/37962 [41:25:17<46:35:20,  8.35s/it]
 47%|████▋     | 17880/37962 [41:25:25<46:06:24,  8.27s/it]
                                                           
{'loss': 1.336, 'grad_norm': 8.379532814025879, 'learning_rate': 1.1420384586061215e-05, 'epoch': 0.47}

 47%|████▋     | 17880/37962 [41:25:25<46:06:24,  8.27s/it]
 47%|████▋     | 17881/37962 [41:25:33<46:47:27,  8.39s/it]
 47%|████▋     | 17882/37962 [41:25:42<46:36:17,  8.36s/it]
 47%|████▋     | 17883/37962 [41:25:50<46:21:00,  8.31s/it]
 47%|████▋     | 17884/37962 [41:25:58<46:20:28,  8.31s/it]
 47%|████▋     | 17885/37962 [41:26:06<46:14:43,  8.29s/it]
 47%|████▋     | 17886/37962 [41:26:15<45:59:59,  8.25s/it]
 47%|████▋     | 17887/37962 [41:26:23<45:58:19,  8.24s/it]
 47%|████▋     | 17888/37962 [41:26:31<46:02:53,  8.26s/it]
 47%|████▋     | 17889/37962 [41:26:39<45:57:09,  8.24s/it]
 47%|████▋     | 17890/37962 [41:26:47<45:49:37,  8.22s/it]
                                                           
{'loss': 1.422, 'grad_norm': 6.1946306228637695, 'learning_rate': 1.1411938966585599e-05, 'epoch': 0.47}

 47%|████▋     | 17890/37962 [41:26:47<45:49:37,  8.22s/it]
 47%|████▋     | 17891/37962 [41:26:56<45:56:13,  8.24s/it]
 47%|████▋     | 17892/37962 [41:27:04<45:46:07,  8.21s/it]
 47%|████▋     | 17893/37962 [41:27:12<45:40:06,  8.19s/it]
 47%|████▋     | 17894/37962 [41:27:20<45:36:30,  8.18s/it]
 47%|████▋     | 17895/37962 [41:27:28<45:39:12,  8.19s/it]
 47%|████▋     | 17896/37962 [41:27:37<45:42:10,  8.20s/it]
 47%|████▋     | 17897/37962 [41:27:45<45:58:28,  8.25s/it]
 47%|████▋     | 17898/37962 [41:27:53<45:48:27,  8.22s/it]
 47%|████▋     | 17899/37962 [41:28:01<45:33:17,  8.17s/it]
 47%|████▋     | 17900/37962 [41:28:09<45:32:54,  8.17s/it]
                                                           
{'loss': 1.3865, 'grad_norm': 3.783933401107788, 'learning_rate': 1.140349231938407e-05, 'epoch': 0.47}

 47%|████▋     | 17900/37962 [41:28:09<45:32:54,  8.17s/it]
 47%|████▋     | 17901/37962 [41:28:18<46:05:16,  8.27s/it]
 47%|████▋     | 17902/37962 [41:28:26<45:58:15,  8.25s/it]
 47%|████▋     | 17903/37962 [41:28:34<45:56:42,  8.25s/it]
 47%|████▋     | 17904/37962 [41:28:42<45:43:08,  8.21s/it]
 47%|████▋     | 17905/37962 [41:28:51<45:54:01,  8.24s/it]
 47%|████▋     | 17906/37962 [41:28:59<46:27:06,  8.34s/it]
 47%|████▋     | 17907/37962 [41:29:07<45:51:57,  8.23s/it]
 47%|████▋     | 17908/37962 [41:29:16<46:07:34,  8.28s/it]
 47%|████▋     | 17909/37962 [41:29:24<46:11:27,  8.29s/it]
 47%|████▋     | 17910/37962 [41:29:32<46:14:16,  8.30s/it]
                                                           
{'loss': 1.2952, 'grad_norm': 5.940703392028809, 'learning_rate': 1.139504465060479e-05, 'epoch': 0.47}

 47%|████▋     | 17910/37962 [41:29:32<46:14:16,  8.30s/it]
 47%|████▋     | 17911/37962 [41:29:41<46:21:20,  8.32s/it]
 47%|████▋     | 17912/37962 [41:29:49<46:12:43,  8.30s/it]
 47%|████▋     | 17913/37962 [41:29:57<46:10:58,  8.29s/it]
 47%|████▋     | 17914/37962 [41:30:06<46:33:08,  8.36s/it]
 47%|████▋     | 17915/37962 [41:30:14<45:38:44,  8.20s/it]
 47%|████▋     | 17916/37962 [41:30:22<45:58:59,  8.26s/it]
 47%|████▋     | 17917/37962 [41:30:30<45:55:22,  8.25s/it]
 47%|████▋     | 17918/37962 [41:30:38<45:53:07,  8.24s/it]
 47%|████▋     | 17919/37962 [41:30:46<45:35:29,  8.19s/it]
 47%|████▋     | 17920/37962 [41:30:55<45:52:44,  8.24s/it]
                                                           
{'loss': 1.3672, 'grad_norm': 11.068153381347656, 'learning_rate': 1.1386595966396681e-05, 'epoch': 0.47}

 47%|████▋     | 17920/37962 [41:30:55<45:52:44,  8.24s/it]
 47%|████▋     | 17921/37962 [41:31:03<45:59:28,  8.26s/it]
 47%|████▋     | 17922/37962 [41:31:11<46:12:51,  8.30s/it]
 47%|████▋     | 17923/37962 [41:31:20<46:19:46,  8.32s/it]
 47%|████▋     | 17924/37962 [41:31:28<45:40:18,  8.21s/it]
 47%|████▋     | 17925/37962 [41:31:36<45:45:27,  8.22s/it]
 47%|████▋     | 17926/37962 [41:31:44<45:35:40,  8.19s/it]
 47%|████▋     | 17927/37962 [41:31:52<45:12:24,  8.12s/it]
 47%|████▋     | 17928/37962 [41:32:01<45:38:06,  8.20s/it]
 47%|████▋     | 17929/37962 [41:32:09<45:55:12,  8.25s/it]
 47%|████▋     | 17930/37962 [41:32:17<45:49:18,  8.23s/it]
                                                           
{'loss': 1.3043, 'grad_norm': 14.5176362991333, 'learning_rate': 1.1378146272909387e-05, 'epoch': 0.47}

 47%|████▋     | 17930/37962 [41:32:17<45:49:18,  8.23s/it]
 47%|████▋     | 17931/37962 [41:32:25<45:53:23,  8.25s/it]
 47%|████▋     | 17932/37962 [41:32:33<45:40:11,  8.21s/it]
 47%|████▋     | 17933/37962 [41:32:42<45:48:24,  8.23s/it]
 47%|████▋     | 17934/37962 [41:32:50<45:34:58,  8.19s/it]
 47%|████▋     | 17935/37962 [41:32:58<45:46:19,  8.23s/it]
 47%|████▋     | 17936/37962 [41:33:06<45:32:07,  8.19s/it]
 47%|████▋     | 17937/37962 [41:33:14<45:33:05,  8.19s/it]
 47%|████▋     | 17938/37962 [41:33:23<45:37:58,  8.20s/it]
 47%|████▋     | 17939/37962 [41:33:31<45:35:21,  8.20s/it]
 47%|████▋     | 17940/37962 [41:33:39<45:40:07,  8.21s/it]
                                                           
{'loss': 1.2776, 'grad_norm': 13.596080780029297, 'learning_rate': 1.1369695576293298e-05, 'epoch': 0.47}

 47%|████▋     | 17940/37962 [41:33:39<45:40:07,  8.21s/it]
 47%|████▋     | 17941/37962 [41:33:47<45:37:53,  8.21s/it]
 47%|████▋     | 17942/37962 [41:33:56<45:36:45,  8.20s/it]
 47%|████▋     | 17943/37962 [41:34:04<45:45:17,  8.23s/it]
 47%|████▋     | 17944/37962 [41:34:12<45:40:17,  8.21s/it]
 47%|████▋     | 17945/37962 [41:34:20<45:38:24,  8.21s/it]
 47%|████▋     | 17946/37962 [41:34:28<45:09:31,  8.12s/it]
 47%|████▋     | 17947/37962 [41:34:37<45:53:59,  8.26s/it]
 47%|████▋     | 17948/37962 [41:34:45<46:04:29,  8.29s/it]
 47%|████▋     | 17949/37962 [41:34:53<46:03:41,  8.29s/it]
 47%|████▋     | 17950/37962 [41:35:02<46:17:08,  8.33s/it]
                                                           
{'loss': 1.4124, 'grad_norm': 11.712677955627441, 'learning_rate': 1.1361243882699524e-05, 'epoch': 0.47}

 47%|████▋     | 17950/37962 [41:35:02<46:17:08,  8.33s/it]
 47%|████▋     | 17951/37962 [41:35:10<46:00:48,  8.28s/it]
 47%|████▋     | 17952/37962 [41:35:18<46:19:51,  8.34s/it]
 47%|████▋     | 17953/37962 [41:35:27<46:05:23,  8.29s/it]
 47%|████▋     | 17954/37962 [41:35:35<46:08:35,  8.30s/it]
 47%|████▋     | 17955/37962 [41:35:43<46:20:22,  8.34s/it]
 47%|████▋     | 17956/37962 [41:35:52<46:12:02,  8.31s/it]
 47%|████▋     | 17957/37962 [41:36:00<46:04:55,  8.29s/it]
 47%|████▋     | 17958/37962 [41:36:08<45:36:01,  8.21s/it]
 47%|████▋     | 17959/37962 [41:36:16<45:41:09,  8.22s/it]
 47%|████▋     | 17960/37962 [41:36:24<45:54:17,  8.26s/it]
                                                           
{'loss': 1.191, 'grad_norm': 5.001216888427734, 'learning_rate': 1.1352791198279913e-05, 'epoch': 0.47}

 47%|████▋     | 17960/37962 [41:36:24<45:54:17,  8.26s/it]
 47%|████▋     | 17961/37962 [41:36:33<46:19:52,  8.34s/it]
 47%|████▋     | 17962/37962 [41:36:41<46:13:34,  8.32s/it]
 47%|████▋     | 17963/37962 [41:36:49<46:03:56,  8.29s/it]
 47%|████▋     | 17964/37962 [41:36:58<46:11:28,  8.32s/it]
 47%|████▋     | 17965/37962 [41:37:06<46:08:26,  8.31s/it]
 47%|████▋     | 17966/37962 [41:37:14<45:20:35,  8.16s/it]
 47%|████▋     | 17967/37962 [41:37:22<45:34:28,  8.21s/it]
 47%|████▋     | 17968/37962 [41:37:30<45:18:30,  8.16s/it]
 47%|████▋     | 17969/37962 [41:37:39<45:46:16,  8.24s/it]
 47%|████▋     | 17970/37962 [41:37:47<46:01:07,  8.29s/it]
                                                           
{'loss': 1.3725, 'grad_norm': 3.1461751461029053, 'learning_rate': 1.1344337529187022e-05, 'epoch': 0.47}

 47%|████▋     | 17970/37962 [41:37:47<46:01:07,  8.29s/it]
 47%|████▋     | 17971/37962 [41:37:55<45:56:24,  8.27s/it]
 47%|████▋     | 17972/37962 [41:38:04<45:55:13,  8.27s/it]
 47%|████▋     | 17973/37962 [41:38:12<45:48:19,  8.25s/it]
 47%|████▋     | 17974/37962 [41:38:20<45:32:43,  8.20s/it]
 47%|████▋     | 17975/37962 [41:38:28<45:40:52,  8.23s/it]
 47%|████▋     | 17976/37962 [41:38:37<45:48:03,  8.25s/it]
 47%|████▋     | 17977/37962 [41:38:45<45:46:58,  8.25s/it]
 47%|████▋     | 17978/37962 [41:38:53<45:46:21,  8.25s/it]
 47%|████▋     | 17979/37962 [41:39:01<45:08:26,  8.13s/it]
 47%|████▋     | 17980/37962 [41:39:09<45:23:49,  8.18s/it]
                                                           
{'loss': 1.3915, 'grad_norm': 13.747640609741211, 'learning_rate': 1.1335882881574134e-05, 'epoch': 0.47}

 47%|████▋     | 17980/37962 [41:39:09<45:23:49,  8.18s/it]
 47%|████▋     | 17981/37962 [41:39:17<45:41:13,  8.23s/it]
 47%|████▋     | 17982/37962 [41:39:26<45:40:15,  8.23s/it]
 47%|████▋     | 17983/37962 [41:39:34<45:20:18,  8.17s/it]
 47%|████▋     | 17984/37962 [41:39:42<45:22:38,  8.18s/it]
 47%|████▋     | 17985/37962 [41:39:50<45:37:49,  8.22s/it]
 47%|████▋     | 17986/37962 [41:39:59<45:37:58,  8.22s/it]
 47%|████▋     | 17987/37962 [41:40:07<46:04:41,  8.30s/it]
 47%|████▋     | 17988/37962 [41:40:15<45:20:02,  8.17s/it]
 47%|████▋     | 17989/37962 [41:40:23<45:19:37,  8.17s/it]
 47%|████▋     | 17990/37962 [41:40:31<45:09:38,  8.14s/it]
                                                           
{'loss': 1.3045, 'grad_norm': 5.127624988555908, 'learning_rate': 1.1327427261595238e-05, 'epoch': 0.47}

 47%|████▋     | 17990/37962 [41:40:31<45:09:38,  8.14s/it]
 47%|████▋     | 17991/37962 [41:40:39<45:15:07,  8.16s/it]
 47%|████▋     | 17992/37962 [41:40:48<45:25:31,  8.19s/it]
 47%|████▋     | 17993/37962 [41:40:56<45:53:14,  8.27s/it]
 47%|████▋     | 17994/37962 [41:41:04<46:07:30,  8.32s/it]
 47%|████▋     | 17995/37962 [41:41:13<46:05:11,  8.31s/it]
 47%|████▋     | 17996/37962 [41:41:21<45:23:25,  8.18s/it]
 47%|████▋     | 17997/37962 [41:41:29<45:29:59,  8.20s/it]
 47%|████▋     | 17998/37962 [41:41:37<45:40:06,  8.24s/it]
 47%|████▋     | 17999/37962 [41:41:46<45:49:28,  8.26s/it]
 47%|████▋     | 18000/37962 [41:42:01<58:05:24, 10.48s/it]
                                                           
{'loss': 1.3864, 'grad_norm': 9.164541244506836, 'learning_rate': 1.1318970675405037e-05, 'epoch': 0.47}

 47%|████▋     | 18000/37962 [41:42:01<58:05:24, 10.48s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 47%|████▋     | 18001/37962 [41:42:54<127:58:11, 23.08s/it]
 47%|████▋     | 18002/37962 [41:43:02<103:58:07, 18.75s/it]
 47%|████▋     | 18003/37962 [41:43:11<86:37:09, 15.62s/it] 
 47%|████▋     | 18004/37962 [41:43:19<73:45:01, 13.30s/it]
 47%|████▋     | 18005/37962 [41:43:27<65:25:48, 11.80s/it]
 47%|████▋     | 18006/37962 [41:43:35<60:01:59, 10.83s/it]
 47%|████▋     | 18007/37962 [41:43:44<55:47:27, 10.06s/it]
 47%|████▋     | 18008/37962 [41:43:52<52:41:28,  9.51s/it]
 47%|████▋     | 18009/37962 [41:44:00<50:16:30,  9.07s/it]
 47%|████▋     | 18010/37962 [41:44:08<48:30:21,  8.75s/it]
                                                           
{'loss': 1.4237, 'grad_norm': 18.0811767578125, 'learning_rate': 1.1310513129158926e-05, 'epoch': 0.47}

 47%|████▋     | 18010/37962 [41:44:08<48:30:21,  8.75s/it]
 47%|████▋     | 18011/37962 [41:44:16<47:44:05,  8.61s/it]
 47%|████▋     | 18012/37962 [41:44:24<47:10:25,  8.51s/it]
 47%|████▋     | 18013/37962 [41:44:33<47:01:55,  8.49s/it]
 47%|████▋     | 18014/37962 [41:44:41<46:37:52,  8.42s/it]
 47%|████▋     | 18015/37962 [41:44:49<46:22:44,  8.37s/it]
 47%|████▋     | 18016/37962 [41:44:57<45:36:39,  8.23s/it]
 47%|████▋     | 18017/37962 [41:45:06<45:38:19,  8.24s/it]
 47%|████▋     | 18018/37962 [41:45:14<45:49:35,  8.27s/it]
 47%|████▋     | 18019/37962 [41:45:22<46:03:28,  8.31s/it]
 47%|████▋     | 18020/37962 [41:45:31<45:59:40,  8.30s/it]
                                                           
{'loss': 1.299, 'grad_norm': 11.857624053955078, 'learning_rate': 1.1302054629013015e-05, 'epoch': 0.47}

 47%|████▋     | 18020/37962 [41:45:31<45:59:40,  8.30s/it]
 47%|████▋     | 18021/37962 [41:45:39<45:45:07,  8.26s/it]
 47%|████▋     | 18022/37962 [41:45:47<45:47:07,  8.27s/it]
 47%|████▋     | 18023/37962 [41:45:55<45:36:47,  8.24s/it]
 47%|████▋     | 18024/37962 [41:46:03<45:30:58,  8.22s/it]
 47%|████▋     | 18025/37962 [41:46:12<45:30:22,  8.22s/it]
 47%|████▋     | 18026/37962 [41:46:20<45:44:24,  8.26s/it]
 47%|████▋     | 18027/37962 [41:46:28<46:05:49,  8.32s/it]
 47%|████▋     | 18028/37962 [41:46:37<46:06:09,  8.33s/it]
 47%|████▋     | 18029/37962 [41:46:45<45:52:44,  8.29s/it]
 47%|████▋     | 18030/37962 [41:46:53<45:46:13,  8.27s/it]
                                                           
{'loss': 1.3542, 'grad_norm': 9.494019508361816, 'learning_rate': 1.1293595181124097e-05, 'epoch': 0.47}

 47%|████▋     | 18030/37962 [41:46:53<45:46:13,  8.27s/it]
 47%|████▋     | 18031/37962 [41:47:02<46:04:01,  8.32s/it]
 48%|████▊     | 18032/37962 [41:47:10<46:00:58,  8.31s/it]
 48%|████▊     | 18033/37962 [41:47:18<45:56:01,  8.30s/it]
 48%|████▊     | 18034/37962 [41:47:26<45:53:38,  8.29s/it]
 48%|████▊     | 18035/37962 [41:47:35<45:58:25,  8.31s/it]
 48%|████▊     | 18036/37962 [41:47:43<46:17:25,  8.36s/it]
 48%|████▊     | 18037/37962 [41:47:51<45:59:28,  8.31s/it]
 48%|████▊     | 18038/37962 [41:48:00<45:59:54,  8.31s/it]
 48%|████▊     | 18039/37962 [41:48:08<45:33:55,  8.23s/it]
 48%|████▊     | 18040/37962 [41:48:16<45:44:43,  8.27s/it]
                                                           
{'loss': 1.3622, 'grad_norm': 3.08787202835083, 'learning_rate': 1.1285134791649655e-05, 'epoch': 0.48}

 48%|████▊     | 18040/37962 [41:48:16<45:44:43,  8.27s/it]
 48%|████▊     | 18041/37962 [41:48:25<45:47:14,  8.27s/it]
 48%|████▊     | 18042/37962 [41:48:33<46:04:34,  8.33s/it]
 48%|████▊     | 18043/37962 [41:48:41<45:54:02,  8.30s/it]
 48%|████▊     | 18044/37962 [41:48:49<45:51:48,  8.29s/it]
 48%|████▊     | 18045/37962 [41:48:58<46:21:11,  8.38s/it]
 48%|████▊     | 18046/37962 [41:49:06<46:15:17,  8.36s/it]
 48%|████▊     | 18047/37962 [41:49:15<46:06:36,  8.34s/it]
 48%|████▊     | 18048/37962 [41:49:23<46:21:32,  8.38s/it]
 48%|████▊     | 18049/37962 [41:49:32<46:22:30,  8.38s/it]
 48%|████▊     | 18050/37962 [41:49:39<45:42:38,  8.26s/it]
                                                           
{'loss': 1.4552, 'grad_norm': 8.90296745300293, 'learning_rate': 1.127667346674786e-05, 'epoch': 0.48}

 48%|████▊     | 18050/37962 [41:49:39<45:42:38,  8.26s/it]
 48%|████▊     | 18051/37962 [41:49:48<46:06:15,  8.34s/it]
 48%|████▊     | 18052/37962 [41:49:56<46:08:18,  8.34s/it]
 48%|████▊     | 18053/37962 [41:50:05<46:20:56,  8.38s/it]
 48%|████▊     | 18054/37962 [41:50:13<46:13:23,  8.36s/it]
 48%|████▊     | 18055/37962 [41:50:21<46:00:33,  8.32s/it]
 48%|████▊     | 18056/37962 [41:50:29<45:39:02,  8.26s/it]
 48%|████▊     | 18057/37962 [41:50:38<45:43:16,  8.27s/it]
 48%|████▊     | 18058/37962 [41:50:46<45:36:04,  8.25s/it]
 48%|████▊     | 18059/37962 [41:50:54<45:35:15,  8.25s/it]
 48%|████▊     | 18060/37962 [41:51:03<45:40:14,  8.26s/it]
                                                           
{'loss': 1.3724, 'grad_norm': 9.801215171813965, 'learning_rate': 1.1268211212577567e-05, 'epoch': 0.48}

 48%|████▊     | 18060/37962 [41:51:03<45:40:14,  8.26s/it]
 48%|████▊     | 18061/37962 [41:51:11<45:59:59,  8.32s/it]
 48%|████▊     | 18062/37962 [41:51:19<45:53:31,  8.30s/it]
 48%|████▊     | 18063/37962 [41:51:27<45:39:56,  8.26s/it]
 48%|████▊     | 18064/37962 [41:51:36<45:58:25,  8.32s/it]
 48%|████▊     | 18065/37962 [41:51:44<45:57:22,  8.31s/it]
 48%|████▊     | 18066/37962 [41:51:52<45:53:48,  8.30s/it]
 48%|████▊     | 18067/37962 [41:52:01<45:50:52,  8.30s/it]
 48%|████▊     | 18068/37962 [41:52:09<46:00:07,  8.32s/it]
 48%|████▊     | 18069/37962 [41:52:17<45:46:21,  8.28s/it]
 48%|████▊     | 18070/37962 [41:52:26<45:56:29,  8.31s/it]
                                                           
{'loss': 1.2491, 'grad_norm': 9.670479774475098, 'learning_rate': 1.1259748035298304e-05, 'epoch': 0.48}

 48%|████▊     | 18070/37962 [41:52:26<45:56:29,  8.31s/it]
 48%|████▊     | 18071/37962 [41:52:34<45:50:54,  8.30s/it]
 48%|████▊     | 18072/37962 [41:52:42<45:51:09,  8.30s/it]
 48%|████▊     | 18073/37962 [41:52:50<45:16:51,  8.20s/it]
 48%|████▊     | 18074/37962 [41:52:59<45:41:32,  8.27s/it]
 48%|████▊     | 18075/37962 [41:53:07<45:48:01,  8.29s/it]
 48%|████▊     | 18076/37962 [41:53:15<45:52:01,  8.30s/it]
 48%|████▊     | 18077/37962 [41:53:24<45:56:10,  8.32s/it]
 48%|████▊     | 18078/37962 [41:53:32<45:18:18,  8.20s/it]
 48%|████▊     | 18079/37962 [41:53:40<45:23:18,  8.22s/it]
 48%|████▊     | 18080/37962 [41:53:48<44:46:12,  8.11s/it]
                                                           
{'loss': 1.3442, 'grad_norm': 13.312700271606445, 'learning_rate': 1.1251283941070268e-05, 'epoch': 0.48}

 48%|████▊     | 18080/37962 [41:53:48<44:46:12,  8.11s/it]
 48%|████▊     | 18081/37962 [41:53:56<45:00:20,  8.15s/it]
 48%|████▊     | 18082/37962 [41:54:04<45:18:00,  8.20s/it]
 48%|████▊     | 18083/37962 [41:54:13<45:35:08,  8.26s/it]
 48%|████▊     | 18084/37962 [41:54:21<45:37:32,  8.26s/it]
 48%|████▊     | 18085/37962 [41:54:29<45:35:44,  8.26s/it]
 48%|████▊     | 18086/37962 [41:54:37<45:38:33,  8.27s/it]
 48%|████▊     | 18087/37962 [41:54:46<45:43:53,  8.28s/it]
 48%|████▊     | 18088/37962 [41:54:54<45:47:53,  8.30s/it]
 48%|████▊     | 18089/37962 [41:55:02<45:54:03,  8.31s/it]
 48%|████▊     | 18090/37962 [41:55:11<45:57:21,  8.33s/it]
                                                           
{'loss': 1.3298, 'grad_norm': 18.127330780029297, 'learning_rate': 1.1242818936054327e-05, 'epoch': 0.48}

 48%|████▊     | 18090/37962 [41:55:11<45:57:21,  8.33s/it]
 48%|████▊     | 18091/37962 [41:55:19<46:02:24,  8.34s/it]
 48%|████▊     | 18092/37962 [41:55:27<45:40:18,  8.27s/it]
 48%|████▊     | 18093/37962 [41:55:35<45:10:52,  8.19s/it]
 48%|████▊     | 18094/37962 [41:55:44<45:27:40,  8.24s/it]
 48%|████▊     | 18095/37962 [41:55:52<45:43:43,  8.29s/it]
 48%|████▊     | 18096/37962 [41:56:00<45:20:57,  8.22s/it]
 48%|████▊     | 18097/37962 [41:56:08<44:52:37,  8.13s/it]
 48%|████▊     | 18098/37962 [41:56:16<45:05:22,  8.17s/it]
 48%|████▊     | 18099/37962 [41:56:25<45:13:57,  8.20s/it]
 48%|████▊     | 18100/37962 [41:56:33<45:57:06,  8.33s/it]
                                                           
{'loss': 1.3974, 'grad_norm': 7.370855808258057, 'learning_rate': 1.1234353026412015e-05, 'epoch': 0.48}

 48%|████▊     | 18100/37962 [41:56:33<45:57:06,  8.33s/it]
 48%|████▊     | 18101/37962 [41:56:42<45:56:49,  8.33s/it]
 48%|████▊     | 18102/37962 [41:56:49<45:03:52,  8.17s/it]
 48%|████▊     | 18103/37962 [41:56:58<45:22:21,  8.23s/it]
 48%|████▊     | 18104/37962 [41:57:06<45:41:46,  8.28s/it]
 48%|████▊     | 18105/37962 [41:57:14<45:49:39,  8.31s/it]
 48%|████▊     | 18106/37962 [41:57:22<45:14:14,  8.20s/it]
 48%|████▊     | 18107/37962 [41:57:30<45:00:02,  8.16s/it]
 48%|████▊     | 18108/37962 [41:57:39<45:22:41,  8.23s/it]
 48%|████▊     | 18109/37962 [41:57:47<45:14:01,  8.20s/it]
 48%|████▊     | 18110/37962 [41:57:55<45:28:32,  8.25s/it]
                                                           
{'loss': 1.2922, 'grad_norm': 13.715600967407227, 'learning_rate': 1.1225886218305517e-05, 'epoch': 0.48}

 48%|████▊     | 18110/37962 [41:57:55<45:28:32,  8.25s/it]
 48%|████▊     | 18111/37962 [41:58:11<58:31:06, 10.61s/it]
 48%|████▊     | 18112/37962 [41:58:20<54:31:42,  9.89s/it]
 48%|████▊     | 18113/37962 [41:58:28<51:28:03,  9.33s/it]
 48%|████▊     | 18114/37962 [41:58:36<50:07:22,  9.09s/it]
 48%|████▊     | 18115/37962 [41:58:45<49:04:58,  8.90s/it]
 48%|████▊     | 18116/37962 [41:58:53<48:09:23,  8.74s/it]
 48%|████▊     | 18117/37962 [41:59:01<47:34:13,  8.63s/it]
 48%|████▊     | 18118/37962 [41:59:10<47:03:13,  8.54s/it]
 48%|████▊     | 18119/37962 [41:59:18<46:36:16,  8.46s/it]
 48%|████▊     | 18120/37962 [41:59:26<46:27:12,  8.43s/it]
                                                           
{'loss': 1.3602, 'grad_norm': 4.454719066619873, 'learning_rate': 1.121741851789768e-05, 'epoch': 0.48}

 48%|████▊     | 18120/37962 [41:59:26<46:27:12,  8.43s/it]
 48%|████▊     | 18121/37962 [41:59:35<46:33:13,  8.45s/it]
 48%|████▊     | 18122/37962 [41:59:43<46:13:09,  8.39s/it]
 48%|████▊     | 18123/37962 [41:59:52<46:13:12,  8.39s/it]
 48%|████▊     | 18124/37962 [42:00:00<45:51:45,  8.32s/it]
 48%|████▊     | 18125/37962 [42:00:08<46:16:01,  8.40s/it]
 48%|████▊     | 18126/37962 [42:00:16<45:58:56,  8.35s/it]
 48%|████▊     | 18127/37962 [42:00:25<45:31:01,  8.26s/it]
 48%|████▊     | 18128/37962 [42:00:33<45:19:55,  8.23s/it]
 48%|████▊     | 18129/37962 [42:00:41<45:16:53,  8.22s/it]
 48%|████▊     | 18130/37962 [42:00:49<45:13:07,  8.21s/it]
                                                           
{'loss': 1.3888, 'grad_norm': 5.337070465087891, 'learning_rate': 1.1208949931351995e-05, 'epoch': 0.48}

 48%|████▊     | 18130/37962 [42:00:49<45:13:07,  8.21s/it]
 48%|████▊     | 18131/37962 [42:00:57<44:40:05,  8.11s/it]
 48%|████▊     | 18132/37962 [42:01:05<45:00:15,  8.17s/it]
 48%|████▊     | 18133/37962 [42:01:14<45:06:25,  8.19s/it]
 48%|████▊     | 18134/37962 [42:01:22<45:17:16,  8.22s/it]
 48%|████▊     | 18135/37962 [42:01:30<45:35:06,  8.28s/it]
 48%|████▊     | 18136/37962 [42:01:38<45:31:17,  8.27s/it]
 48%|████▊     | 18137/37962 [42:01:47<45:36:10,  8.28s/it]
 48%|████▊     | 18138/37962 [42:01:55<45:33:22,  8.27s/it]
 48%|████▊     | 18139/37962 [42:02:03<45:37:05,  8.28s/it]
 48%|████▊     | 18140/37962 [42:02:12<45:29:44,  8.26s/it]
                                                           
{'loss': 1.3646, 'grad_norm': 18.37483024597168, 'learning_rate': 1.1200480464832597e-05, 'epoch': 0.48}

 48%|████▊     | 18140/37962 [42:02:12<45:29:44,  8.26s/it]
 48%|████▊     | 18141/37962 [42:02:20<45:20:36,  8.24s/it]
 48%|████▊     | 18142/37962 [42:02:28<45:27:19,  8.26s/it]
 48%|████▊     | 18143/37962 [42:02:36<45:41:11,  8.30s/it]
 48%|████▊     | 18144/37962 [42:02:45<45:40:42,  8.30s/it]
 48%|████▊     | 18145/37962 [42:02:53<45:36:51,  8.29s/it]
 48%|████▊     | 18146/37962 [42:03:01<45:54:25,  8.34s/it]
 48%|████▊     | 18147/37962 [42:03:10<45:39:29,  8.30s/it]
 48%|████▊     | 18148/37962 [42:03:18<45:18:23,  8.23s/it]
 48%|████▊     | 18149/37962 [42:03:26<45:17:52,  8.23s/it]
 48%|████▊     | 18150/37962 [42:03:34<45:26:44,  8.26s/it]
                                                           
{'loss': 1.3423, 'grad_norm': 9.429910659790039, 'learning_rate': 1.1192010124504263e-05, 'epoch': 0.48}

 48%|████▊     | 18150/37962 [42:03:34<45:26:44,  8.26s/it]
 48%|████▊     | 18151/37962 [42:03:42<45:09:18,  8.21s/it]
 48%|████▊     | 18152/37962 [42:03:51<45:16:00,  8.23s/it]
 48%|████▊     | 18153/37962 [42:03:59<44:59:07,  8.18s/it]
 48%|████▊     | 18154/37962 [42:04:07<45:07:01,  8.20s/it]
 48%|████▊     | 18155/37962 [42:04:15<45:34:21,  8.28s/it]
 48%|████▊     | 18156/37962 [42:04:24<45:39:01,  8.30s/it]
 48%|████▊     | 18157/37962 [42:04:32<45:41:59,  8.31s/it]
 48%|████▊     | 18158/37962 [42:04:40<45:42:39,  8.31s/it]
 48%|████▊     | 18159/37962 [42:04:49<45:58:25,  8.36s/it]
 48%|████▊     | 18160/37962 [42:04:57<46:16:18,  8.41s/it]
                                                           
{'loss': 1.3409, 'grad_norm': 6.537646770477295, 'learning_rate': 1.1183538916532413e-05, 'epoch': 0.48}

 48%|████▊     | 18160/37962 [42:04:57<46:16:18,  8.41s/it]
 48%|████▊     | 18161/37962 [42:05:06<46:07:44,  8.39s/it]
 48%|████▊     | 18162/37962 [42:05:14<46:17:14,  8.42s/it]
 48%|████▊     | 18163/37962 [42:05:23<46:13:24,  8.40s/it]
 48%|████▊     | 18164/37962 [42:05:31<45:54:12,  8.35s/it]
 48%|████▊     | 18165/37962 [42:05:39<46:02:13,  8.37s/it]
 48%|████▊     | 18166/37962 [42:05:48<46:04:57,  8.38s/it]
 48%|████▊     | 18167/37962 [42:05:56<46:10:32,  8.40s/it]
 48%|████▊     | 18168/37962 [42:06:05<46:24:07,  8.44s/it]
 48%|████▊     | 18169/37962 [42:06:13<46:01:43,  8.37s/it]
 48%|████▊     | 18170/37962 [42:06:21<45:47:21,  8.33s/it]
                                                           
{'loss': 1.3802, 'grad_norm': 8.128035545349121, 'learning_rate': 1.1175066847083088e-05, 'epoch': 0.48}

 48%|████▊     | 18170/37962 [42:06:21<45:47:21,  8.33s/it]
 48%|████▊     | 18171/37962 [42:06:29<45:42:28,  8.31s/it]
 48%|████▊     | 18172/37962 [42:06:38<45:45:45,  8.32s/it]
 48%|████▊     | 18173/37962 [42:06:46<45:48:38,  8.33s/it]
 48%|████▊     | 18174/37962 [42:06:54<45:51:13,  8.34s/it]
 48%|████▊     | 18175/37962 [42:07:03<45:37:20,  8.30s/it]
 48%|████▊     | 18176/37962 [42:07:11<45:27:57,  8.27s/it]
 48%|████▊     | 18177/37962 [42:07:19<45:20:15,  8.25s/it]
 48%|████▊     | 18178/37962 [42:07:28<45:49:37,  8.34s/it]
 48%|████▊     | 18179/37962 [42:07:36<45:49:33,  8.34s/it]
 48%|████▊     | 18180/37962 [42:07:44<45:39:29,  8.31s/it]
                                                           
{'loss': 1.2532, 'grad_norm': 8.956013679504395, 'learning_rate': 1.1166593922322962e-05, 'epoch': 0.48}

 48%|████▊     | 18180/37962 [42:07:44<45:39:29,  8.31s/it]
 48%|████▊     | 18181/37962 [42:07:52<45:28:08,  8.28s/it]
 48%|████▊     | 18182/37962 [42:08:01<45:35:12,  8.30s/it]
 48%|████▊     | 18183/37962 [42:08:09<45:38:07,  8.31s/it]
 48%|████▊     | 18184/37962 [42:08:17<45:29:25,  8.28s/it]
 48%|████▊     | 18185/37962 [42:08:26<45:46:44,  8.33s/it]
 48%|████▊     | 18186/37962 [42:08:34<45:43:59,  8.33s/it]
 48%|████▊     | 18187/37962 [42:08:42<45:40:28,  8.31s/it]
 48%|████▊     | 18188/37962 [42:08:50<45:18:58,  8.25s/it]
 48%|████▊     | 18189/37962 [42:08:59<45:28:29,  8.28s/it]
 48%|████▊     | 18190/37962 [42:09:07<45:35:01,  8.30s/it]
                                                           
{'loss': 1.3569, 'grad_norm': 11.01744270324707, 'learning_rate': 1.115812014841933e-05, 'epoch': 0.48}

 48%|████▊     | 18190/37962 [42:09:07<45:35:01,  8.30s/it]
 48%|████▊     | 18191/37962 [42:09:15<45:41:49,  8.32s/it]
 48%|████▊     | 18192/37962 [42:09:24<45:34:57,  8.30s/it]
 48%|████▊     | 18193/37962 [42:09:32<45:25:20,  8.27s/it]
 48%|████▊     | 18194/37962 [42:09:40<45:20:58,  8.26s/it]
 48%|████▊     | 18195/37962 [42:09:48<45:22:57,  8.27s/it]
 48%|████▊     | 18196/37962 [42:09:57<45:21:40,  8.26s/it]
 48%|████▊     | 18197/37962 [42:10:05<45:21:23,  8.26s/it]
 48%|████▊     | 18198/37962 [42:10:13<45:26:30,  8.28s/it]
 48%|████▊     | 18199/37962 [42:10:21<45:05:30,  8.21s/it]
 48%|████▊     | 18200/37962 [42:10:29<44:55:12,  8.18s/it]
                                                           
{'loss': 1.4022, 'grad_norm': 22.02098274230957, 'learning_rate': 1.1149645531540104e-05, 'epoch': 0.48}

 48%|████▊     | 18200/37962 [42:10:29<44:55:12,  8.18s/it]
 48%|████▊     | 18201/37962 [42:10:38<44:58:05,  8.19s/it]
 48%|████▊     | 18202/37962 [42:10:46<45:11:51,  8.23s/it]
 48%|████▊     | 18203/37962 [42:10:54<45:05:49,  8.22s/it]
 48%|████▊     | 18204/37962 [42:11:02<45:19:57,  8.26s/it]
 48%|████▊     | 18205/37962 [42:11:11<45:24:45,  8.27s/it]
 48%|████▊     | 18206/37962 [42:11:19<45:17:01,  8.25s/it]
 48%|████▊     | 18207/37962 [42:11:27<45:25:05,  8.28s/it]
 48%|████▊     | 18208/37962 [42:11:36<45:45:50,  8.34s/it]
 48%|████▊     | 18209/37962 [42:11:44<45:36:17,  8.31s/it]
 48%|████▊     | 18210/37962 [42:11:52<44:51:14,  8.18s/it]
                                                           
{'loss': 1.3723, 'grad_norm': 8.176229476928711, 'learning_rate': 1.1141170077853814e-05, 'epoch': 0.48}

 48%|████▊     | 18210/37962 [42:11:52<44:51:14,  8.18s/it]
 48%|████▊     | 18211/37962 [42:12:00<45:07:08,  8.22s/it]
 48%|████▊     | 18212/37962 [42:12:08<44:43:11,  8.15s/it]
 48%|████▊     | 18213/37962 [42:12:16<44:43:49,  8.15s/it]
 48%|████▊     | 18214/37962 [42:12:25<45:03:42,  8.21s/it]
 48%|████▊     | 18215/37962 [42:12:33<44:42:56,  8.15s/it]
 48%|████▊     | 18216/37962 [42:12:41<44:51:55,  8.18s/it]
 48%|████▊     | 18217/37962 [42:12:49<45:05:17,  8.22s/it]
 48%|████▊     | 18218/37962 [42:12:57<44:58:57,  8.20s/it]
 48%|████▊     | 18219/37962 [42:13:06<45:18:56,  8.26s/it]
 48%|████▊     | 18220/37962 [42:13:14<45:22:47,  8.28s/it]
                                                           
{'loss': 1.4838, 'grad_norm': 6.5172247886657715, 'learning_rate': 1.1132693793529588e-05, 'epoch': 0.48}

 48%|████▊     | 18220/37962 [42:13:14<45:22:47,  8.28s/it]
 48%|████▊     | 18221/37962 [42:13:22<45:07:31,  8.23s/it]
 48%|████▊     | 18222/37962 [42:13:31<45:10:55,  8.24s/it]
 48%|████▊     | 18223/37962 [42:13:39<45:10:35,  8.24s/it]
 48%|████▊     | 18224/37962 [42:13:47<44:40:55,  8.15s/it]
 48%|████▊     | 18225/37962 [42:13:55<44:33:14,  8.13s/it]
 48%|████▊     | 18226/37962 [42:14:03<45:02:21,  8.22s/it]
 48%|████▊     | 18227/37962 [42:14:11<45:04:22,  8.22s/it]
 48%|████▊     | 18228/37962 [42:14:20<45:12:38,  8.25s/it]
 48%|████▊     | 18229/37962 [42:14:28<45:07:23,  8.23s/it]
 48%|████▊     | 18230/37962 [42:14:36<45:00:34,  8.21s/it]
                                                           
{'loss': 1.3874, 'grad_norm': 2.8326313495635986, 'learning_rate': 1.1124216684737177e-05, 'epoch': 0.48}

 48%|████▊     | 18230/37962 [42:14:36<45:00:34,  8.21s/it]
 48%|████▊     | 18231/37962 [42:14:44<45:02:46,  8.22s/it]
 48%|████▊     | 18232/37962 [42:14:53<45:02:54,  8.22s/it]
 48%|████▊     | 18233/37962 [42:15:01<45:03:09,  8.22s/it]
 48%|████▊     | 18234/37962 [42:15:09<45:07:25,  8.23s/it]
 48%|████▊     | 18235/37962 [42:15:17<44:06:16,  8.05s/it]
 48%|████▊     | 18236/37962 [42:15:25<44:54:00,  8.19s/it]
 48%|████▊     | 18237/37962 [42:15:34<45:05:29,  8.23s/it]
 48%|████▊     | 18238/37962 [42:15:42<44:53:00,  8.19s/it]
 48%|████▊     | 18239/37962 [42:15:50<44:58:12,  8.21s/it]
 48%|████▊     | 18240/37962 [42:15:58<45:05:11,  8.23s/it]
                                                           
{'loss': 1.3754, 'grad_norm': 4.887580394744873, 'learning_rate': 1.1115738757646917e-05, 'epoch': 0.48}

 48%|████▊     | 18240/37962 [42:15:58<45:05:11,  8.23s/it]
 48%|████▊     | 18241/37962 [42:16:06<45:07:09,  8.24s/it]
 48%|████▊     | 18242/37962 [42:16:15<45:25:03,  8.29s/it]
 48%|████▊     | 18243/37962 [42:16:23<45:15:02,  8.26s/it]
 48%|████▊     | 18244/37962 [42:16:31<45:22:30,  8.28s/it]
 48%|████▊     | 18245/37962 [42:16:40<45:23:19,  8.29s/it]
 48%|████▊     | 18246/37962 [42:16:48<45:29:09,  8.31s/it]
 48%|████▊     | 18247/37962 [42:16:56<45:40:43,  8.34s/it]
 48%|████▊     | 18248/37962 [42:17:05<45:35:17,  8.32s/it]
 48%|████▊     | 18249/37962 [42:17:13<45:18:55,  8.28s/it]
 48%|████▊     | 18250/37962 [42:17:21<45:40:16,  8.34s/it]
                                                           
{'loss': 1.2877, 'grad_norm': 7.576440811157227, 'learning_rate': 1.1107260018429741e-05, 'epoch': 0.48}

 48%|████▊     | 18250/37962 [42:17:21<45:40:16,  8.34s/it]
 48%|████▊     | 18251/37962 [42:17:30<45:25:43,  8.30s/it]
 48%|████▊     | 18252/37962 [42:17:38<45:26:40,  8.30s/it]
 48%|████▊     | 18253/37962 [42:17:46<45:25:12,  8.30s/it]
 48%|████▊     | 18254/37962 [42:17:54<45:19:29,  8.28s/it]
 48%|████▊     | 18255/37962 [42:18:02<44:50:50,  8.19s/it]
 48%|████▊     | 18256/37962 [42:18:11<45:09:19,  8.25s/it]
 48%|████▊     | 18257/37962 [42:18:19<44:41:43,  8.17s/it]
 48%|████▊     | 18258/37962 [42:18:27<44:37:17,  8.15s/it]
 48%|████▊     | 18259/37962 [42:18:35<45:13:31,  8.26s/it]
 48%|████▊     | 18260/37962 [42:18:44<45:05:01,  8.24s/it]
                                                           
{'loss': 1.3264, 'grad_norm': 5.947535991668701, 'learning_rate': 1.1098780473257184e-05, 'epoch': 0.48}

 48%|████▊     | 18260/37962 [42:18:44<45:05:01,  8.24s/it]
 48%|████▊     | 18261/37962 [42:18:52<44:43:03,  8.17s/it]
 48%|████▊     | 18262/37962 [42:19:00<44:50:52,  8.20s/it]
 48%|████▊     | 18263/37962 [42:19:08<45:27:43,  8.31s/it]
 48%|████▊     | 18264/37962 [42:19:17<45:16:04,  8.27s/it]
 48%|████▊     | 18265/37962 [42:19:25<45:10:40,  8.26s/it]
 48%|████▊     | 18266/37962 [42:19:33<45:34:28,  8.33s/it]
 48%|████▊     | 18267/37962 [42:19:42<45:38:55,  8.34s/it]
 48%|████▊     | 18268/37962 [42:19:50<45:27:58,  8.31s/it]
 48%|████▊     | 18269/37962 [42:19:58<45:32:29,  8.33s/it]
 48%|████▊     | 18270/37962 [42:20:06<45:01:08,  8.23s/it]
                                                           
{'loss': 1.3794, 'grad_norm': 15.813035011291504, 'learning_rate': 1.1090300128301352e-05, 'epoch': 0.48}

 48%|████▊     | 18270/37962 [42:20:06<45:01:08,  8.23s/it]
 48%|████▊     | 18271/37962 [42:20:15<45:11:44,  8.26s/it]
 48%|████▊     | 18272/37962 [42:20:23<45:19:03,  8.29s/it]
 48%|████▊     | 18273/37962 [42:20:31<45:06:20,  8.25s/it]
 48%|████▊     | 18274/37962 [42:20:39<44:57:16,  8.22s/it]
 48%|████▊     | 18275/37962 [42:20:48<45:06:58,  8.25s/it]
 48%|████▊     | 18276/37962 [42:20:56<44:53:06,  8.21s/it]
 48%|████▊     | 18277/37962 [42:21:04<45:01:51,  8.24s/it]
 48%|████▊     | 18278/37962 [42:21:12<44:56:19,  8.22s/it]
 48%|████▊     | 18279/37962 [42:21:20<44:28:27,  8.13s/it]
 48%|████▊     | 18280/37962 [42:21:29<44:48:40,  8.20s/it]
                                                           
{'loss': 1.3281, 'grad_norm': 3.2813379764556885, 'learning_rate': 1.1081818989734946e-05, 'epoch': 0.48}

 48%|████▊     | 18280/37962 [42:21:29<44:48:40,  8.20s/it]
 48%|████▊     | 18281/37962 [42:21:37<44:58:30,  8.23s/it]
 48%|████▊     | 18282/37962 [42:21:45<44:35:52,  8.16s/it]
 48%|████▊     | 18283/37962 [42:21:53<44:51:48,  8.21s/it]
 48%|████▊     | 18284/37962 [42:22:01<44:58:15,  8.23s/it]
 48%|████▊     | 18285/37962 [42:22:10<45:20:32,  8.30s/it]
 48%|████▊     | 18286/37962 [42:22:18<45:09:20,  8.26s/it]
 48%|████▊     | 18287/37962 [42:22:26<45:10:49,  8.27s/it]
 48%|████▊     | 18288/37962 [42:22:35<45:47:03,  8.38s/it]
 48%|████▊     | 18289/37962 [42:22:43<45:29:25,  8.32s/it]
 48%|████▊     | 18290/37962 [42:22:51<44:58:33,  8.23s/it]
                                                           
{'loss': 1.3367, 'grad_norm': 11.992558479309082, 'learning_rate': 1.1073337063731239e-05, 'epoch': 0.48}

 48%|████▊     | 18290/37962 [42:22:51<44:58:33,  8.23s/it]
 48%|████▊     | 18291/37962 [42:23:00<45:19:42,  8.30s/it]
 48%|████▊     | 18292/37962 [42:23:08<45:31:13,  8.33s/it]
 48%|████▊     | 18293/37962 [42:23:16<45:18:47,  8.29s/it]
 48%|████▊     | 18294/37962 [42:23:25<45:21:16,  8.30s/it]
 48%|████▊     | 18295/37962 [42:23:33<45:32:16,  8.34s/it]
 48%|████▊     | 18296/37962 [42:23:41<44:54:03,  8.22s/it]
 48%|████▊     | 18297/37962 [42:23:49<45:11:45,  8.27s/it]
 48%|████▊     | 18298/37962 [42:23:57<44:45:57,  8.20s/it]
 48%|████▊     | 18299/37962 [42:24:06<45:05:11,  8.25s/it]
 48%|████▊     | 18300/37962 [42:24:14<45:04:31,  8.25s/it]
                                                           
{'loss': 1.272, 'grad_norm': 6.668792247772217, 'learning_rate': 1.1064854356464079e-05, 'epoch': 0.48}

 48%|████▊     | 18300/37962 [42:24:14<45:04:31,  8.25s/it]
 48%|████▊     | 18301/37962 [42:24:22<45:01:36,  8.24s/it]
 48%|████▊     | 18302/37962 [42:24:30<44:25:33,  8.13s/it]
 48%|████▊     | 18303/37962 [42:24:38<44:15:59,  8.11s/it]
 48%|████▊     | 18304/37962 [42:24:46<43:58:28,  8.05s/it]
 48%|████▊     | 18305/37962 [42:24:54<44:19:44,  8.12s/it]
 48%|████▊     | 18306/37962 [42:25:03<44:29:29,  8.15s/it]
 48%|████▊     | 18307/37962 [42:25:11<44:37:05,  8.17s/it]
 48%|████▊     | 18308/37962 [42:25:19<44:49:51,  8.21s/it]
 48%|████▊     | 18309/37962 [42:25:27<44:38:23,  8.18s/it]
 48%|████▊     | 18310/37962 [42:25:36<45:07:20,  8.27s/it]
                                                           
{'loss': 1.2833, 'grad_norm': 5.277916431427002, 'learning_rate': 1.1056370874107879e-05, 'epoch': 0.48}

 48%|████▊     | 18310/37962 [42:25:36<45:07:20,  8.27s/it]
 48%|████▊     | 18311/37962 [42:25:44<45:06:47,  8.26s/it]
 48%|████▊     | 18312/37962 [42:25:52<45:13:37,  8.29s/it]
 48%|████▊     | 18313/37962 [42:26:01<45:13:59,  8.29s/it]
 48%|████▊     | 18314/37962 [42:26:09<45:14:54,  8.29s/it]
 48%|████▊     | 18315/37962 [42:26:17<45:22:09,  8.31s/it]
 48%|████▊     | 18316/37962 [42:26:25<44:56:20,  8.23s/it]
 48%|████▊     | 18317/37962 [42:26:33<44:52:48,  8.22s/it]
 48%|████▊     | 18318/37962 [42:26:42<44:48:24,  8.21s/it]
 48%|████▊     | 18319/37962 [42:26:50<44:40:13,  8.19s/it]
 48%|████▊     | 18320/37962 [42:26:58<45:06:10,  8.27s/it]
                                                           
{'loss': 1.2934, 'grad_norm': 4.3763909339904785, 'learning_rate': 1.1047886622837619e-05, 'epoch': 0.48}

 48%|████▊     | 18320/37962 [42:26:58<45:06:10,  8.27s/it]
 48%|████▊     | 18321/37962 [42:27:07<45:23:27,  8.32s/it]
 48%|████▊     | 18322/37962 [42:27:15<45:22:59,  8.32s/it]
 48%|████▊     | 18323/37962 [42:27:23<45:29:52,  8.34s/it]
 48%|████▊     | 18324/37962 [42:27:32<45:21:20,  8.31s/it]
 48%|████▊     | 18325/37962 [42:27:40<45:08:18,  8.28s/it]
 48%|████▊     | 18326/37962 [42:27:48<44:32:07,  8.16s/it]
 48%|████▊     | 18327/37962 [42:27:56<44:46:39,  8.21s/it]
 48%|████▊     | 18328/37962 [42:28:04<44:54:30,  8.23s/it]
 48%|████▊     | 18329/37962 [42:28:13<44:56:18,  8.24s/it]
 48%|████▊     | 18330/37962 [42:28:21<44:56:44,  8.24s/it]
                                                           
{'loss': 1.3728, 'grad_norm': 3.4746158123016357, 'learning_rate': 1.103940160882884e-05, 'epoch': 0.48}

 48%|████▊     | 18330/37962 [42:28:21<44:56:44,  8.24s/it]
 48%|████▊     | 18331/37962 [42:28:29<44:50:28,  8.22s/it]
 48%|████▊     | 18332/37962 [42:28:37<44:56:10,  8.24s/it]
 48%|████▊     | 18333/37962 [42:28:46<45:05:46,  8.27s/it]
 48%|████▊     | 18334/37962 [42:28:54<45:03:02,  8.26s/it]
 48%|████▊     | 18335/37962 [42:29:02<45:20:04,  8.32s/it]
 48%|████▊     | 18336/37962 [42:29:10<45:06:03,  8.27s/it]
 48%|████▊     | 18337/37962 [42:29:18<44:38:14,  8.19s/it]
 48%|████▊     | 18338/37962 [42:29:26<43:54:16,  8.05s/it]
 48%|████▊     | 18339/37962 [42:29:35<44:21:05,  8.14s/it]
 48%|████▊     | 18340/37962 [42:29:43<44:08:58,  8.10s/it]
                                                           
{'loss': 1.4169, 'grad_norm': 5.771236896514893, 'learning_rate': 1.103091583825764e-05, 'epoch': 0.48}

 48%|████▊     | 18340/37962 [42:29:43<44:08:58,  8.10s/it]
 48%|████▊     | 18341/37962 [42:29:51<44:25:15,  8.15s/it]
 48%|████▊     | 18342/37962 [42:29:59<44:36:43,  8.19s/it]
 48%|████▊     | 18343/37962 [42:30:07<44:46:10,  8.22s/it]
 48%|████▊     | 18344/37962 [42:30:16<44:50:10,  8.23s/it]
 48%|████▊     | 18345/37962 [42:30:23<43:57:05,  8.07s/it]
 48%|████▊     | 18346/37962 [42:30:32<44:16:30,  8.13s/it]
 48%|████▊     | 18347/37962 [42:30:40<44:39:45,  8.20s/it]
 48%|████▊     | 18348/37962 [42:30:48<44:27:36,  8.16s/it]
 48%|████▊     | 18349/37962 [42:30:57<45:16:12,  8.31s/it]
 48%|████▊     | 18350/37962 [42:31:05<45:14:39,  8.31s/it]
                                                           
{'loss': 1.3383, 'grad_norm': 19.925865173339844, 'learning_rate': 1.1022429317300655e-05, 'epoch': 0.48}

 48%|████▊     | 18350/37962 [42:31:05<45:14:39,  8.31s/it]
 48%|████▊     | 18351/37962 [42:31:13<45:21:08,  8.33s/it]
 48%|████▊     | 18352/37962 [42:31:21<44:32:44,  8.18s/it]
 48%|████▊     | 18353/37962 [42:31:29<44:37:13,  8.19s/it]
 48%|████▊     | 18354/37962 [42:31:38<44:56:32,  8.25s/it]
 48%|████▊     | 18355/37962 [42:31:46<45:05:29,  8.28s/it]
 48%|████▊     | 18356/37962 [42:31:55<45:16:07,  8.31s/it]
 48%|████▊     | 18357/37962 [42:32:03<44:46:45,  8.22s/it]
 48%|████▊     | 18358/37962 [42:32:11<45:04:00,  8.28s/it]
 48%|████▊     | 18359/37962 [42:32:19<44:58:36,  8.26s/it]
 48%|████▊     | 18360/37962 [42:32:27<44:59:42,  8.26s/it]
                                                           
{'loss': 1.3163, 'grad_norm': 5.696928024291992, 'learning_rate': 1.1013942052135082e-05, 'epoch': 0.48}

 48%|████▊     | 18360/37962 [42:32:27<44:59:42,  8.26s/it]
 48%|████▊     | 18361/37962 [42:32:36<44:59:33,  8.26s/it]
 48%|████▊     | 18362/37962 [42:32:44<44:51:00,  8.24s/it]
 48%|████▊     | 18363/37962 [42:32:52<45:03:56,  8.28s/it]
 48%|████▊     | 18364/37962 [42:33:01<45:06:31,  8.29s/it]
 48%|████▊     | 18365/37962 [42:33:09<45:21:36,  8.33s/it]
 48%|████▊     | 18366/37962 [42:33:17<45:39:21,  8.39s/it]
 48%|████▊     | 18367/37962 [42:33:26<45:26:12,  8.35s/it]
 48%|████▊     | 18368/37962 [42:33:34<45:01:37,  8.27s/it]
 48%|████▊     | 18369/37962 [42:33:42<44:30:14,  8.18s/it]
 48%|████▊     | 18370/37962 [42:33:50<44:31:00,  8.18s/it]
                                                           
{'loss': 1.2002, 'grad_norm': 5.728933334350586, 'learning_rate': 1.1005454048938659e-05, 'epoch': 0.48}

 48%|████▊     | 18370/37962 [42:33:50<44:31:00,  8.18s/it]
 48%|████▊     | 18371/37962 [42:33:58<44:40:21,  8.21s/it]
 48%|████▊     | 18372/37962 [42:34:06<44:08:34,  8.11s/it]
 48%|████▊     | 18373/37962 [42:34:14<44:27:14,  8.17s/it]
 48%|████▊     | 18374/37962 [42:34:23<44:40:28,  8.21s/it]
 48%|████▊     | 18375/37962 [42:34:31<44:39:35,  8.21s/it]
 48%|████▊     | 18376/37962 [42:34:39<44:18:17,  8.14s/it]
 48%|████▊     | 18377/37962 [42:34:47<44:54:30,  8.25s/it]
 48%|████▊     | 18378/37962 [42:34:56<44:40:20,  8.21s/it]
 48%|████▊     | 18379/37962 [42:35:04<44:13:50,  8.13s/it]
 48%|████▊     | 18380/37962 [42:35:12<44:14:58,  8.13s/it]
                                                           
{'loss': 1.2748, 'grad_norm': 8.316256523132324, 'learning_rate': 1.0996965313889652e-05, 'epoch': 0.48}

 48%|████▊     | 18380/37962 [42:35:12<44:14:58,  8.13s/it]
 48%|████▊     | 18381/37962 [42:35:20<44:20:22,  8.15s/it]
 48%|████▊     | 18382/37962 [42:35:28<43:59:56,  8.09s/it]
 48%|████▊     | 18383/37962 [42:35:36<44:08:04,  8.12s/it]
 48%|████▊     | 18384/37962 [42:35:44<44:37:38,  8.21s/it]
 48%|████▊     | 18385/37962 [42:35:53<44:43:54,  8.23s/it]
 48%|████▊     | 18386/37962 [42:36:01<44:59:14,  8.27s/it]
 48%|████▊     | 18387/37962 [42:36:09<44:53:20,  8.26s/it]
 48%|████▊     | 18388/37962 [42:36:17<44:27:28,  8.18s/it]
 48%|████▊     | 18389/37962 [42:36:25<44:24:25,  8.17s/it]
 48%|████▊     | 18390/37962 [42:36:34<44:49:01,  8.24s/it]
                                                           
{'loss': 1.3693, 'grad_norm': 6.635368824005127, 'learning_rate': 1.098847585316686e-05, 'epoch': 0.48}

 48%|████▊     | 18390/37962 [42:36:34<44:49:01,  8.24s/it]
 48%|████▊     | 18391/37962 [42:36:42<45:08:46,  8.30s/it]
 48%|████▊     | 18392/37962 [42:36:51<45:05:19,  8.29s/it]
 48%|████▊     | 18393/37962 [42:36:59<45:04:46,  8.29s/it]
 48%|████▊     | 18394/37962 [42:37:07<45:14:41,  8.32s/it]
 48%|████▊     | 18395/37962 [42:37:15<44:55:35,  8.27s/it]
 48%|████▊     | 18396/37962 [42:37:24<45:05:40,  8.30s/it]
 48%|████▊     | 18397/37962 [42:37:32<45:10:00,  8.31s/it]
 48%|████▊     | 18398/37962 [42:37:40<45:11:15,  8.32s/it]
 48%|████▊     | 18399/37962 [42:37:49<45:07:52,  8.31s/it]
 48%|████▊     | 18400/37962 [42:37:57<45:05:09,  8.30s/it]
                                                           
{'loss': 1.4099, 'grad_norm': 10.656562805175781, 'learning_rate': 1.0979985672949622e-05, 'epoch': 0.48}

 48%|████▊     | 18400/37962 [42:37:57<45:05:09,  8.30s/it]
 48%|████▊     | 18401/37962 [42:38:05<45:25:27,  8.36s/it]
 48%|████▊     | 18402/37962 [42:38:14<45:22:20,  8.35s/it]
 48%|████▊     | 18403/37962 [42:38:22<44:54:48,  8.27s/it]
 48%|████▊     | 18404/37962 [42:38:30<45:07:50,  8.31s/it]
 48%|████▊     | 18405/37962 [42:38:39<45:21:06,  8.35s/it]
 48%|████▊     | 18406/37962 [42:38:47<45:46:54,  8.43s/it]
 48%|████▊     | 18407/37962 [42:38:55<45:18:50,  8.34s/it]
 48%|████▊     | 18408/37962 [42:39:04<45:01:50,  8.29s/it]
 48%|████▊     | 18409/37962 [42:39:11<44:15:42,  8.15s/it]
 48%|████▊     | 18410/37962 [42:39:20<44:17:05,  8.15s/it]
                                                           
{'loss': 1.3079, 'grad_norm': 6.002817630767822, 'learning_rate': 1.0971494779417789e-05, 'epoch': 0.48}

 48%|████▊     | 18410/37962 [42:39:20<44:17:05,  8.15s/it]
 48%|████▊     | 18411/37962 [42:39:28<44:33:49,  8.21s/it]
 49%|████▊     | 18412/37962 [42:39:36<44:53:22,  8.27s/it]
 49%|████▊     | 18413/37962 [42:39:45<44:59:46,  8.29s/it]
 49%|████▊     | 18414/37962 [42:39:53<44:50:28,  8.26s/it]
 49%|████▊     | 18415/37962 [42:40:01<44:26:26,  8.18s/it]
 49%|████▊     | 18416/37962 [42:40:09<44:59:48,  8.29s/it]
 49%|████▊     | 18417/37962 [42:40:18<44:39:59,  8.23s/it]
 49%|████▊     | 18418/37962 [42:40:26<44:43:42,  8.24s/it]
 49%|████▊     | 18419/37962 [42:40:34<44:44:10,  8.24s/it]
 49%|████▊     | 18420/37962 [42:40:42<44:00:58,  8.11s/it]
                                                           
{'loss': 1.3305, 'grad_norm': 8.822344779968262, 'learning_rate': 1.0963003178751736e-05, 'epoch': 0.49}

 49%|████▊     | 18420/37962 [42:40:42<44:00:58,  8.11s/it]
 49%|████▊     | 18421/37962 [42:40:50<44:18:24,  8.16s/it]
 49%|████▊     | 18422/37962 [42:40:58<44:24:01,  8.18s/it]
 49%|████▊     | 18423/37962 [42:41:07<44:41:43,  8.23s/it]
 49%|████▊     | 18424/37962 [42:41:15<44:51:17,  8.26s/it]
 49%|████▊     | 18425/37962 [42:41:23<44:26:00,  8.19s/it]
 49%|████▊     | 18426/37962 [42:41:31<44:28:16,  8.19s/it]
 49%|████▊     | 18427/37962 [42:41:40<44:43:11,  8.24s/it]
 49%|████▊     | 18428/37962 [42:41:48<44:37:21,  8.22s/it]
 49%|████▊     | 18429/37962 [42:41:56<44:37:39,  8.23s/it]
 49%|████▊     | 18430/37962 [42:42:05<45:05:04,  8.31s/it]
                                                           
{'loss': 1.3176, 'grad_norm': 5.804863452911377, 'learning_rate': 1.0954510877132352e-05, 'epoch': 0.49}

 49%|████▊     | 18430/37962 [42:42:05<45:05:04,  8.31s/it]
 49%|████▊     | 18431/37962 [42:42:13<44:50:35,  8.27s/it]
 49%|████▊     | 18432/37962 [42:42:21<44:51:49,  8.27s/it]
 49%|████▊     | 18433/37962 [42:42:29<44:54:02,  8.28s/it]
 49%|████▊     | 18434/37962 [42:42:38<44:57:19,  8.29s/it]
 49%|████▊     | 18435/37962 [42:42:46<44:40:56,  8.24s/it]
 49%|████▊     | 18436/37962 [42:42:54<44:47:21,  8.26s/it]
 49%|████▊     | 18437/37962 [42:43:02<44:49:56,  8.27s/it]
 49%|████▊     | 18438/37962 [42:43:10<44:43:43,  8.25s/it]
 49%|████▊     | 18439/37962 [42:43:19<44:49:52,  8.27s/it]
 49%|████▊     | 18440/37962 [42:43:27<44:57:57,  8.29s/it]
                                                           
{'loss': 1.1872, 'grad_norm': 18.32676124572754, 'learning_rate': 1.0946017880741035e-05, 'epoch': 0.49}

 49%|████▊     | 18440/37962 [42:43:27<44:57:57,  8.29s/it]
 49%|████▊     | 18441/37962 [42:43:36<45:06:20,  8.32s/it]
 49%|████▊     | 18442/37962 [42:43:43<44:13:05,  8.15s/it]
 49%|████▊     | 18443/37962 [42:43:52<44:24:17,  8.19s/it]
 49%|████▊     | 18444/37962 [42:44:00<44:41:42,  8.24s/it]
 49%|████▊     | 18445/37962 [42:44:08<45:12:15,  8.34s/it]
 49%|████▊     | 18446/37962 [42:44:17<45:32:46,  8.40s/it]
 49%|████▊     | 18447/37962 [42:44:25<45:14:48,  8.35s/it]
 49%|████▊     | 18448/37962 [42:44:33<44:41:05,  8.24s/it]
 49%|████▊     | 18449/37962 [42:44:41<44:37:36,  8.23s/it]
 49%|████▊     | 18450/37962 [42:44:49<44:00:09,  8.12s/it]
                                                           
{'loss': 1.4034, 'grad_norm': 5.125613689422607, 'learning_rate': 1.0937524195759696e-05, 'epoch': 0.49}

 49%|████▊     | 18450/37962 [42:44:49<44:00:09,  8.12s/it]
 49%|████▊     | 18451/37962 [42:44:57<44:01:55,  8.12s/it]
 49%|████▊     | 18452/37962 [42:45:06<44:15:38,  8.17s/it]
 49%|████▊     | 18453/37962 [42:45:14<44:35:35,  8.23s/it]
 49%|████▊     | 18454/37962 [42:45:22<44:13:57,  8.16s/it]
 49%|████▊     | 18455/37962 [42:45:30<44:20:48,  8.18s/it]
 49%|████▊     | 18456/37962 [42:45:39<44:37:11,  8.23s/it]
 49%|████▊     | 18457/37962 [42:45:47<44:57:25,  8.30s/it]
 49%|████▊     | 18458/37962 [42:45:55<44:02:49,  8.13s/it]
 49%|████▊     | 18459/37962 [42:46:03<43:48:18,  8.09s/it]
 49%|████▊     | 18460/37962 [42:46:11<43:32:51,  8.04s/it]
                                                           
{'loss': 1.2514, 'grad_norm': 6.892393112182617, 'learning_rate': 1.0929029828370732e-05, 'epoch': 0.49}

 49%|████▊     | 18460/37962 [42:46:11<43:32:51,  8.04s/it]
 49%|████▊     | 18461/37962 [42:46:19<44:09:32,  8.15s/it]
 49%|████▊     | 18462/37962 [42:46:28<44:34:39,  8.23s/it]
 49%|████▊     | 18463/37962 [42:46:36<44:47:56,  8.27s/it]
 49%|████▊     | 18464/37962 [42:46:44<44:44:58,  8.26s/it]
 49%|████▊     | 18465/37962 [42:46:52<44:36:44,  8.24s/it]
 49%|████▊     | 18466/37962 [42:47:01<44:28:41,  8.21s/it]
 49%|████▊     | 18467/37962 [42:47:08<43:40:30,  8.07s/it]
 49%|████▊     | 18468/37962 [42:47:17<44:03:16,  8.14s/it]
 49%|████▊     | 18469/37962 [42:47:25<43:58:25,  8.12s/it]
 49%|████▊     | 18470/37962 [42:47:33<44:33:48,  8.23s/it]
                                                           
{'loss': 1.3142, 'grad_norm': 8.154943466186523, 'learning_rate': 1.0920534784757053e-05, 'epoch': 0.49}

 49%|████▊     | 18470/37962 [42:47:33<44:33:48,  8.23s/it]
 49%|████▊     | 18471/37962 [42:47:42<44:47:18,  8.27s/it]
 49%|████▊     | 18472/37962 [42:47:50<44:53:21,  8.29s/it]
 49%|████▊     | 18473/37962 [42:47:58<44:33:49,  8.23s/it]
 49%|████▊     | 18474/37962 [42:48:06<44:48:30,  8.28s/it]
 49%|████▊     | 18475/37962 [42:48:15<44:52:45,  8.29s/it]
 49%|████▊     | 18476/37962 [42:48:23<44:49:11,  8.28s/it]
 49%|████▊     | 18477/37962 [42:48:31<44:38:34,  8.25s/it]
 49%|████▊     | 18478/37962 [42:48:39<44:39:12,  8.25s/it]
 49%|████▊     | 18479/37962 [42:48:48<44:46:06,  8.27s/it]
 49%|████▊     | 18480/37962 [42:48:56<44:37:45,  8.25s/it]
                                                           
{'loss': 1.3768, 'grad_norm': 14.054874420166016, 'learning_rate': 1.091203907110205e-05, 'epoch': 0.49}

 49%|████▊     | 18480/37962 [42:48:56<44:37:45,  8.25s/it]
 49%|████▊     | 18481/37962 [42:49:04<44:37:31,  8.25s/it]
 49%|████▊     | 18482/37962 [42:49:12<44:44:54,  8.27s/it]
 49%|████▊     | 18483/37962 [42:49:21<44:37:40,  8.25s/it]
 49%|████▊     | 18484/37962 [42:49:29<44:40:04,  8.26s/it]
 49%|████▊     | 18485/37962 [42:49:37<44:06:48,  8.15s/it]
 49%|████▊     | 18486/37962 [42:49:45<44:30:19,  8.23s/it]
 49%|████▊     | 18487/37962 [42:49:54<44:47:29,  8.28s/it]
 49%|████▊     | 18488/37962 [42:50:02<44:26:12,  8.21s/it]
 49%|████▊     | 18489/37962 [42:50:10<44:16:32,  8.19s/it]
 49%|████▊     | 18490/37962 [42:50:18<43:49:51,  8.10s/it]
                                                           
{'loss': 1.4095, 'grad_norm': 7.000333786010742, 'learning_rate': 1.0903542693589607e-05, 'epoch': 0.49}

 49%|████▊     | 18490/37962 [42:50:18<43:49:51,  8.10s/it]
 49%|████▊     | 18491/37962 [42:50:26<44:19:00,  8.19s/it]
 49%|████▊     | 18492/37962 [42:50:35<44:57:27,  8.31s/it]
 49%|████▊     | 18493/37962 [42:50:43<44:59:20,  8.32s/it]
 49%|████▊     | 18494/37962 [42:50:51<44:59:58,  8.32s/it]
 49%|████▊     | 18495/37962 [42:51:00<44:59:57,  8.32s/it]
 49%|████▊     | 18496/37962 [42:51:08<44:20:41,  8.20s/it]
 49%|████▊     | 18497/37962 [42:51:16<44:24:56,  8.21s/it]
 49%|████▊     | 18498/37962 [42:51:24<44:24:22,  8.21s/it]
 49%|████▊     | 18499/37962 [42:51:32<44:45:15,  8.28s/it]
 49%|████▊     | 18500/37962 [42:51:41<44:51:41,  8.30s/it]
                                                           
{'loss': 1.4148, 'grad_norm': 10.079047203063965, 'learning_rate': 1.0895045658404088e-05, 'epoch': 0.49}

 49%|████▊     | 18500/37962 [42:51:41<44:51:41,  8.30s/it]
 49%|████▊     | 18501/37962 [42:51:49<45:05:06,  8.34s/it]
 49%|████▊     | 18502/37962 [42:51:57<44:52:11,  8.30s/it]
 49%|████▊     | 18503/37962 [42:52:06<44:42:16,  8.27s/it]
 49%|████▊     | 18504/37962 [42:52:14<44:53:01,  8.30s/it]
 49%|████▊     | 18505/37962 [42:52:23<45:07:04,  8.35s/it]
 49%|████▊     | 18506/37962 [42:52:31<44:51:02,  8.30s/it]
 49%|████▉     | 18507/37962 [42:52:39<44:26:25,  8.22s/it]
 49%|████▉     | 18508/37962 [42:52:47<43:58:44,  8.14s/it]
 49%|████▉     | 18509/37962 [42:52:55<44:11:12,  8.18s/it]
 49%|████▉     | 18510/37962 [42:53:03<44:20:43,  8.21s/it]
                                                           
{'loss': 1.2723, 'grad_norm': 8.294957160949707, 'learning_rate': 1.0886547971730341e-05, 'epoch': 0.49}

 49%|████▉     | 18510/37962 [42:53:03<44:20:43,  8.21s/it]
 49%|████▉     | 18511/37962 [42:53:12<44:46:55,  8.29s/it]
 49%|████▉     | 18512/37962 [42:53:20<44:40:45,  8.27s/it]
 49%|████▉     | 18513/37962 [42:53:28<44:51:19,  8.30s/it]
 49%|████▉     | 18514/37962 [42:53:37<44:41:13,  8.27s/it]
 49%|████▉     | 18515/37962 [42:53:45<44:50:25,  8.30s/it]
 49%|████▉     | 18516/37962 [42:53:53<44:54:42,  8.31s/it]
 49%|████▉     | 18517/37962 [42:54:02<44:59:31,  8.33s/it]
 49%|████▉     | 18518/37962 [42:54:10<44:51:40,  8.31s/it]
 49%|████▉     | 18519/37962 [42:54:18<44:44:30,  8.28s/it]
 49%|████▉     | 18520/37962 [42:54:27<45:00:21,  8.33s/it]
                                                           
{'loss': 1.2843, 'grad_norm': 5.005089282989502, 'learning_rate': 1.0878049639753686e-05, 'epoch': 0.49}

 49%|████▉     | 18520/37962 [42:54:27<45:00:21,  8.33s/it]
 49%|████▉     | 18521/37962 [42:54:35<45:07:37,  8.36s/it]
 49%|████▉     | 18522/37962 [42:54:43<44:04:39,  8.16s/it]
 49%|████▉     | 18523/37962 [42:54:51<44:14:36,  8.19s/it]
 49%|████▉     | 18524/37962 [42:54:59<43:57:16,  8.14s/it]
 49%|████▉     | 18525/37962 [42:55:07<44:18:14,  8.21s/it]
 49%|████▉     | 18526/37962 [42:55:16<44:22:02,  8.22s/it]
 49%|████▉     | 18527/37962 [42:55:24<44:26:36,  8.23s/it]
 49%|████▉     | 18528/37962 [42:55:32<44:36:21,  8.26s/it]
 49%|████▉     | 18529/37962 [42:55:41<45:03:56,  8.35s/it]
 49%|████▉     | 18530/37962 [42:55:49<45:25:39,  8.42s/it]
                                                           
{'loss': 1.3356, 'grad_norm': 7.013195514678955, 'learning_rate': 1.0869550668659908e-05, 'epoch': 0.49}

 49%|████▉     | 18530/37962 [42:55:49<45:25:39,  8.42s/it]
 49%|████▉     | 18531/37962 [42:55:57<44:53:23,  8.32s/it]
 49%|████▉     | 18532/37962 [42:56:06<44:51:37,  8.31s/it]
 49%|████▉     | 18533/37962 [42:56:14<44:31:25,  8.25s/it]
 49%|████▉     | 18534/37962 [42:56:22<45:00:31,  8.34s/it]
 49%|████▉     | 18535/37962 [42:56:31<44:59:30,  8.34s/it]
 49%|████▉     | 18536/37962 [42:56:39<45:07:46,  8.36s/it]
 49%|████▉     | 18537/37962 [42:56:47<44:52:54,  8.32s/it]
 49%|████▉     | 18538/37962 [42:56:56<44:54:49,  8.32s/it]
 49%|████▉     | 18539/37962 [42:57:04<44:59:21,  8.34s/it]
 49%|████▉     | 18540/37962 [42:57:12<44:48:35,  8.31s/it]
                                                           
{'loss': 1.3048, 'grad_norm': 8.040790557861328, 'learning_rate': 1.0861051064635262e-05, 'epoch': 0.49}

 49%|████▉     | 18540/37962 [42:57:12<44:48:35,  8.31s/it]
 49%|████▉     | 18541/37962 [42:57:21<45:08:31,  8.37s/it]
 49%|████▉     | 18542/37962 [42:57:29<45:02:59,  8.35s/it]
 49%|████▉     | 18543/37962 [42:57:37<44:50:19,  8.31s/it]
 49%|████▉     | 18544/37962 [42:57:45<44:39:55,  8.28s/it]
 49%|████▉     | 18545/37962 [42:57:54<44:25:40,  8.24s/it]
 49%|████▉     | 18546/37962 [42:58:02<44:35:18,  8.27s/it]
 49%|████▉     | 18547/37962 [42:58:10<44:23:58,  8.23s/it]
 49%|████▉     | 18548/37962 [42:58:18<44:06:24,  8.18s/it]
 49%|████▉     | 18549/37962 [42:58:26<44:11:36,  8.20s/it]
 49%|████▉     | 18550/37962 [42:58:35<44:35:30,  8.27s/it]
                                                           
{'loss': 1.3405, 'grad_norm': 8.393134117126465, 'learning_rate': 1.0852550833866464e-05, 'epoch': 0.49}

 49%|████▉     | 18550/37962 [42:58:35<44:35:30,  8.27s/it]
 49%|████▉     | 18551/37962 [42:58:43<44:28:07,  8.25s/it]
 49%|████▉     | 18552/37962 [42:58:51<44:32:56,  8.26s/it]
 49%|████▉     | 18553/37962 [42:59:00<44:32:51,  8.26s/it]
 49%|████▉     | 18554/37962 [42:59:08<44:44:08,  8.30s/it]
 49%|████▉     | 18555/37962 [42:59:16<45:08:41,  8.37s/it]
 49%|████▉     | 18556/37962 [42:59:24<44:20:13,  8.22s/it]
 49%|████▉     | 18557/37962 [42:59:33<44:36:29,  8.28s/it]
 49%|████▉     | 18558/37962 [42:59:41<44:33:17,  8.27s/it]
 49%|████▉     | 18559/37962 [42:59:49<44:10:26,  8.20s/it]
 49%|████▉     | 18560/37962 [42:59:57<44:15:33,  8.21s/it]
                                                           
{'loss': 1.3283, 'grad_norm': 15.457324028015137, 'learning_rate': 1.084404998254068e-05, 'epoch': 0.49}

 49%|████▉     | 18560/37962 [42:59:57<44:15:33,  8.21s/it]
 49%|████▉     | 18561/37962 [43:00:06<44:15:10,  8.21s/it]
 49%|████▉     | 18562/37962 [43:00:14<44:14:49,  8.21s/it]
 49%|████▉     | 18563/37962 [43:00:22<44:15:41,  8.21s/it]
 49%|████▉     | 18564/37962 [43:00:30<44:21:08,  8.23s/it]
 49%|████▉     | 18565/37962 [43:00:38<44:18:12,  8.22s/it]
 49%|████▉     | 18566/37962 [43:00:46<44:03:19,  8.18s/it]
 49%|████▉     | 18567/37962 [43:00:55<44:11:05,  8.20s/it]
 49%|████▉     | 18568/37962 [43:01:03<44:18:48,  8.23s/it]
 49%|████▉     | 18569/37962 [43:01:11<44:18:50,  8.23s/it]
 49%|████▉     | 18570/37962 [43:01:20<44:21:34,  8.24s/it]
                                                           
{'loss': 1.3588, 'grad_norm': 6.33058500289917, 'learning_rate': 1.083554851684554e-05, 'epoch': 0.49}

 49%|████▉     | 18570/37962 [43:01:20<44:21:34,  8.24s/it]
 49%|████▉     | 18571/37962 [43:01:28<44:16:08,  8.22s/it]
 49%|████▉     | 18572/37962 [43:01:36<44:43:47,  8.30s/it]
 49%|████▉     | 18573/37962 [43:01:44<43:56:17,  8.16s/it]
 49%|████▉     | 18574/37962 [43:01:52<44:06:38,  8.19s/it]
 49%|████▉     | 18575/37962 [43:02:00<43:47:59,  8.13s/it]
 49%|████▉     | 18576/37962 [43:02:08<43:37:01,  8.10s/it]
 49%|████▉     | 18577/37962 [43:02:17<43:49:36,  8.14s/it]
 49%|████▉     | 18578/37962 [43:02:25<43:36:27,  8.10s/it]
 49%|████▉     | 18579/37962 [43:02:33<43:48:00,  8.14s/it]
 49%|████▉     | 18580/37962 [43:02:41<44:01:55,  8.18s/it]
                                                           
{'loss': 1.293, 'grad_norm': 3.9698903560638428, 'learning_rate': 1.082704644296911e-05, 'epoch': 0.49}

 49%|████▉     | 18580/37962 [43:02:41<44:01:55,  8.18s/it]
 49%|████▉     | 18581/37962 [43:02:49<43:43:28,  8.12s/it]
 49%|████▉     | 18582/37962 [43:02:57<43:49:10,  8.14s/it]
 49%|████▉     | 18583/37962 [43:03:05<44:03:35,  8.18s/it]
 49%|████▉     | 18584/37962 [43:03:14<44:30:49,  8.27s/it]
 49%|████▉     | 18585/37962 [43:03:22<44:30:45,  8.27s/it]
 49%|████▉     | 18586/37962 [43:03:31<44:41:00,  8.30s/it]
 49%|████▉     | 18587/37962 [43:03:39<44:51:18,  8.33s/it]
 49%|████▉     | 18588/37962 [43:03:47<44:38:00,  8.29s/it]
 49%|████▉     | 18589/37962 [43:03:55<44:34:45,  8.28s/it]
 49%|████▉     | 18590/37962 [43:04:04<44:19:10,  8.24s/it]
                                                           
{'loss': 1.3361, 'grad_norm': 13.630831718444824, 'learning_rate': 1.0818543767099899e-05, 'epoch': 0.49}

 49%|████▉     | 18590/37962 [43:04:04<44:19:10,  8.24s/it]
 49%|████▉     | 18591/37962 [43:04:12<44:15:51,  8.23s/it]
 49%|████▉     | 18592/37962 [43:04:20<43:44:20,  8.13s/it]
 49%|████▉     | 18593/37962 [43:04:28<43:37:42,  8.11s/it]
 49%|████▉     | 18594/37962 [43:04:36<43:47:22,  8.14s/it]
 49%|████▉     | 18595/37962 [43:04:44<43:57:52,  8.17s/it]
 49%|████▉     | 18596/37962 [43:04:52<44:03:10,  8.19s/it]
 49%|████▉     | 18597/37962 [43:05:01<44:14:01,  8.22s/it]
 49%|████▉     | 18598/37962 [43:05:09<44:00:43,  8.18s/it]
 49%|████▉     | 18599/37962 [43:05:17<43:49:06,  8.15s/it]
 49%|████▉     | 18600/37962 [43:05:25<44:13:06,  8.22s/it]
                                                           
{'loss': 1.3256, 'grad_norm': 6.548192501068115, 'learning_rate': 1.081004049542686e-05, 'epoch': 0.49}

 49%|████▉     | 18600/37962 [43:05:25<44:13:06,  8.22s/it]
 49%|████▉     | 18601/37962 [43:05:34<44:20:00,  8.24s/it]
 49%|████▉     | 18602/37962 [43:05:42<44:30:38,  8.28s/it]
 49%|████▉     | 18603/37962 [43:05:50<44:29:41,  8.27s/it]
 49%|████▉     | 18604/37962 [43:05:59<44:36:18,  8.30s/it]
 49%|████▉     | 18605/37962 [43:06:07<44:35:25,  8.29s/it]
 49%|████▉     | 18606/37962 [43:06:15<44:22:29,  8.25s/it]
 49%|████▉     | 18607/37962 [43:06:23<44:27:00,  8.27s/it]
 49%|████▉     | 18608/37962 [43:06:32<44:27:50,  8.27s/it]
 49%|████▉     | 18609/37962 [43:06:40<44:08:54,  8.21s/it]
 49%|████▉     | 18610/37962 [43:06:48<44:18:40,  8.24s/it]
                                                           
{'loss': 1.3426, 'grad_norm': 4.088810920715332, 'learning_rate': 1.0801536634139378e-05, 'epoch': 0.49}

 49%|████▉     | 18610/37962 [43:06:48<44:18:40,  8.24s/it]
 49%|████▉     | 18611/37962 [43:06:56<44:24:57,  8.26s/it]
 49%|████▉     | 18612/37962 [43:07:05<44:33:54,  8.29s/it]
 49%|████▉     | 18613/37962 [43:07:13<44:15:33,  8.23s/it]
 49%|████▉     | 18614/37962 [43:07:21<44:20:14,  8.25s/it]
 49%|████▉     | 18615/37962 [43:07:29<44:04:26,  8.20s/it]
 49%|████▉     | 18616/37962 [43:07:37<43:48:03,  8.15s/it]
 49%|████▉     | 18617/37962 [43:07:46<44:12:02,  8.23s/it]
 49%|████▉     | 18618/37962 [43:07:54<44:23:52,  8.26s/it]
 49%|████▉     | 18619/37962 [43:08:02<44:28:59,  8.28s/it]
 49%|████▉     | 18620/37962 [43:08:10<43:32:04,  8.10s/it]
                                                           
{'loss': 1.3614, 'grad_norm': 15.684075355529785, 'learning_rate': 1.0793032189427266e-05, 'epoch': 0.49}

 49%|████▉     | 18620/37962 [43:08:10<43:32:04,  8.10s/it]
 49%|████▉     | 18621/37962 [43:08:18<43:43:29,  8.14s/it]
 49%|████▉     | 18622/37962 [43:08:27<44:31:21,  8.29s/it]
 49%|████▉     | 18623/37962 [43:08:35<44:37:09,  8.31s/it]
 49%|████▉     | 18624/37962 [43:08:43<44:29:17,  8.28s/it]
 49%|████▉     | 18625/37962 [43:08:52<44:21:02,  8.26s/it]
 49%|████▉     | 18626/37962 [43:08:59<43:48:34,  8.16s/it]
 49%|████▉     | 18627/37962 [43:09:08<44:10:19,  8.22s/it]
 49%|████▉     | 18628/37962 [43:09:16<43:38:46,  8.13s/it]
 49%|████▉     | 18629/37962 [43:09:24<43:43:40,  8.14s/it]
 49%|████▉     | 18630/37962 [43:09:32<43:55:48,  8.18s/it]
                                                           
{'loss': 1.2587, 'grad_norm': 5.512744903564453, 'learning_rate': 1.078452716748076e-05, 'epoch': 0.49}

 49%|████▉     | 18630/37962 [43:09:32<43:55:48,  8.18s/it]
 49%|████▉     | 18631/37962 [43:09:41<44:16:37,  8.25s/it]
 49%|████▉     | 18632/37962 [43:09:49<44:04:18,  8.21s/it]
 49%|████▉     | 18633/37962 [43:09:57<44:01:19,  8.20s/it]
 49%|████▉     | 18634/37962 [43:10:05<44:10:21,  8.23s/it]
 49%|████▉     | 18635/37962 [43:10:14<45:08:03,  8.41s/it]
 49%|████▉     | 18636/37962 [43:10:22<44:51:44,  8.36s/it]
 49%|████▉     | 18637/37962 [43:10:31<44:40:45,  8.32s/it]
 49%|████▉     | 18638/37962 [43:10:39<44:37:12,  8.31s/it]
 49%|████▉     | 18639/37962 [43:10:47<44:17:47,  8.25s/it]
 49%|████▉     | 18640/37962 [43:10:55<44:19:07,  8.26s/it]
                                                           
{'loss': 1.2922, 'grad_norm': 7.062557697296143, 'learning_rate': 1.077602157449052e-05, 'epoch': 0.49}

 49%|████▉     | 18640/37962 [43:10:55<44:19:07,  8.26s/it]
 49%|████▉     | 18641/37962 [43:11:03<44:18:54,  8.26s/it]
 49%|████▉     | 18642/37962 [43:11:12<44:26:26,  8.28s/it]
 49%|████▉     | 18643/37962 [43:11:20<44:40:40,  8.33s/it]
 49%|████▉     | 18644/37962 [43:11:29<44:55:27,  8.37s/it]
 49%|████▉     | 18645/37962 [43:11:37<44:21:52,  8.27s/it]
 49%|████▉     | 18646/37962 [43:11:45<44:17:17,  8.25s/it]
 49%|████▉     | 18647/37962 [43:11:53<43:45:59,  8.16s/it]
 49%|████▉     | 18648/37962 [43:12:01<44:26:28,  8.28s/it]
 49%|████▉     | 18649/37962 [43:12:10<44:20:16,  8.26s/it]
 49%|████▉     | 18650/37962 [43:12:18<44:33:47,  8.31s/it]
                                                           
{'loss': 1.3275, 'grad_norm': 6.211926460266113, 'learning_rate': 1.0767515416647622e-05, 'epoch': 0.49}

 49%|████▉     | 18650/37962 [43:12:18<44:33:47,  8.31s/it]
 49%|████▉     | 18651/37962 [43:12:26<44:32:37,  8.30s/it]
 49%|████▉     | 18652/37962 [43:12:35<44:20:23,  8.27s/it]
 49%|████▉     | 18653/37962 [43:12:42<43:50:52,  8.18s/it]
 49%|████▉     | 18654/37962 [43:12:50<43:24:08,  8.09s/it]
 49%|████▉     | 18655/37962 [43:12:59<43:35:00,  8.13s/it]
 49%|████▉     | 18656/37962 [43:13:07<43:45:33,  8.16s/it]
 49%|████▉     | 18657/37962 [43:13:15<43:57:36,  8.20s/it]
 49%|████▉     | 18658/37962 [43:13:23<43:48:02,  8.17s/it]
 49%|████▉     | 18659/37962 [43:13:32<44:00:13,  8.21s/it]
 49%|████▉     | 18660/37962 [43:13:40<44:31:54,  8.31s/it]
                                                           
{'loss': 1.3268, 'grad_norm': 5.405052661895752, 'learning_rate': 1.0759008700143548e-05, 'epoch': 0.49}

 49%|████▉     | 18660/37962 [43:13:40<44:31:54,  8.31s/it]
 49%|████▉     | 18661/37962 [43:13:48<44:36:05,  8.32s/it]
 49%|████▉     | 18662/37962 [43:13:57<44:43:41,  8.34s/it]
 49%|████▉     | 18663/37962 [43:14:05<44:50:13,  8.36s/it]
 49%|████▉     | 18664/37962 [43:14:14<45:01:03,  8.40s/it]
 49%|████▉     | 18665/37962 [43:14:22<44:50:34,  8.37s/it]
 49%|████▉     | 18666/37962 [43:14:30<45:02:14,  8.40s/it]
 49%|████▉     | 18667/37962 [43:14:39<44:47:41,  8.36s/it]
 49%|████▉     | 18668/37962 [43:14:47<44:45:56,  8.35s/it]
 49%|████▉     | 18669/37962 [43:14:55<44:32:59,  8.31s/it]
 49%|████▉     | 18670/37962 [43:15:04<44:28:21,  8.30s/it]
                                                           
{'loss': 1.3938, 'grad_norm': 7.603171348571777, 'learning_rate': 1.0750501431170188e-05, 'epoch': 0.49}

 49%|████▉     | 18670/37962 [43:15:04<44:28:21,  8.30s/it]
 49%|████▉     | 18671/37962 [43:15:12<44:39:18,  8.33s/it]
 49%|████▉     | 18672/37962 [43:15:20<44:40:26,  8.34s/it]
 49%|████▉     | 18673/37962 [43:15:28<44:22:42,  8.28s/it]
 49%|████▉     | 18674/37962 [43:15:37<44:29:34,  8.30s/it]
 49%|████▉     | 18675/37962 [43:15:45<43:55:29,  8.20s/it]
 49%|████▉     | 18676/37962 [43:15:53<43:59:54,  8.21s/it]
 49%|████▉     | 18677/37962 [43:16:01<44:10:23,  8.25s/it]
 49%|████▉     | 18678/37962 [43:16:10<44:09:16,  8.24s/it]
 49%|████▉     | 18679/37962 [43:16:18<43:58:19,  8.21s/it]
 49%|████▉     | 18680/37962 [43:16:26<44:33:14,  8.32s/it]
                                                           
{'loss': 1.3844, 'grad_norm': 15.55841064453125, 'learning_rate': 1.074199361591984e-05, 'epoch': 0.49}

 49%|████▉     | 18680/37962 [43:16:26<44:33:14,  8.32s/it]
 49%|████▉     | 18681/37962 [43:16:34<44:16:16,  8.27s/it]
 49%|████▉     | 18682/37962 [43:16:43<44:15:24,  8.26s/it]
 49%|████▉     | 18683/37962 [43:16:51<44:13:17,  8.26s/it]
 49%|████▉     | 18684/37962 [43:16:59<44:20:36,  8.28s/it]
 49%|████▉     | 18685/37962 [43:17:08<44:24:58,  8.29s/it]
 49%|████▉     | 18686/37962 [43:17:16<44:31:54,  8.32s/it]
 49%|████▉     | 18687/37962 [43:17:24<44:22:17,  8.29s/it]
 49%|████▉     | 18688/37962 [43:17:33<44:33:01,  8.32s/it]
 49%|████▉     | 18689/37962 [43:17:41<44:27:34,  8.30s/it]
 49%|████▉     | 18690/37962 [43:17:49<44:11:30,  8.26s/it]
                                                           
{'loss': 1.3844, 'grad_norm': 6.230996608734131, 'learning_rate': 1.0733485260585194e-05, 'epoch': 0.49}

 49%|████▉     | 18690/37962 [43:17:49<44:11:30,  8.26s/it]
 49%|████▉     | 18691/37962 [43:17:57<44:18:37,  8.28s/it]
 49%|████▉     | 18692/37962 [43:18:06<44:21:29,  8.29s/it]
 49%|████▉     | 18693/37962 [43:18:14<44:32:40,  8.32s/it]
 49%|████▉     | 18694/37962 [43:18:22<44:10:36,  8.25s/it]
 49%|████▉     | 18695/37962 [43:18:30<44:07:45,  8.25s/it]
 49%|████▉     | 18696/37962 [43:18:39<43:59:48,  8.22s/it]
 49%|████▉     | 18697/37962 [43:18:47<43:56:03,  8.21s/it]
 49%|████▉     | 18698/37962 [43:18:55<44:05:08,  8.24s/it]
 49%|████▉     | 18699/37962 [43:19:03<44:13:20,  8.26s/it]
 49%|████▉     | 18700/37962 [43:19:12<44:14:25,  8.27s/it]
                                                           
{'loss': 1.315, 'grad_norm': 9.104446411132812, 'learning_rate': 1.072497637135933e-05, 'epoch': 0.49}

 49%|████▉     | 18700/37962 [43:19:12<44:14:25,  8.27s/it]
 49%|████▉     | 18701/37962 [43:19:19<43:34:52,  8.15s/it]
 49%|████▉     | 18702/37962 [43:19:27<43:24:05,  8.11s/it]
 49%|████▉     | 18703/37962 [43:19:42<54:02:52, 10.10s/it]
 49%|████▉     | 18704/37962 [43:19:51<51:04:41,  9.55s/it]
 49%|████▉     | 18705/37962 [43:19:59<49:03:59,  9.17s/it]
 49%|████▉     | 18706/37962 [43:20:07<47:13:05,  8.83s/it]
 49%|████▉     | 18707/37962 [43:20:15<46:15:33,  8.65s/it]
 49%|████▉     | 18708/37962 [43:20:23<45:54:07,  8.58s/it]
 49%|████▉     | 18709/37962 [43:20:32<45:22:26,  8.48s/it]
 49%|████▉     | 18710/37962 [43:20:40<45:04:04,  8.43s/it]
                                                           
{'loss': 1.2455, 'grad_norm': 8.425771713256836, 'learning_rate': 1.0716466954435729e-05, 'epoch': 0.49}

 49%|████▉     | 18710/37962 [43:20:40<45:04:04,  8.43s/it]
 49%|████▉     | 18711/37962 [43:20:49<45:27:52,  8.50s/it]
 49%|████▉     | 18712/37962 [43:20:57<45:20:52,  8.48s/it]
 49%|████▉     | 18713/37962 [43:21:05<44:57:56,  8.41s/it]
 49%|████▉     | 18714/37962 [43:21:14<44:42:52,  8.36s/it]
 49%|████▉     | 18715/37962 [43:21:22<44:09:18,  8.26s/it]
 49%|████▉     | 18716/37962 [43:21:29<43:27:45,  8.13s/it]
 49%|████▉     | 18717/37962 [43:21:38<43:41:17,  8.17s/it]
 49%|████▉     | 18718/37962 [43:21:46<43:53:55,  8.21s/it]
 49%|████▉     | 18719/37962 [43:21:54<44:12:20,  8.27s/it]
 49%|████▉     | 18720/37962 [43:22:03<44:09:00,  8.26s/it]
                                                           
{'loss': 1.294, 'grad_norm': 11.723322868347168, 'learning_rate': 1.070795701600824e-05, 'epoch': 0.49}

 49%|████▉     | 18720/37962 [43:22:03<44:09:00,  8.26s/it]
 49%|████▉     | 18721/37962 [43:22:11<44:25:05,  8.31s/it]
 49%|████▉     | 18722/37962 [43:22:19<44:12:25,  8.27s/it]
 49%|████▉     | 18723/37962 [43:22:28<44:19:44,  8.29s/it]
 49%|████▉     | 18724/37962 [43:22:36<44:27:38,  8.32s/it]
 49%|████▉     | 18725/37962 [43:22:44<44:13:34,  8.28s/it]
 49%|████▉     | 18726/37962 [43:22:52<43:41:40,  8.18s/it]
 49%|████▉     | 18727/37962 [43:23:01<44:00:01,  8.24s/it]
 49%|████▉     | 18728/37962 [43:23:09<43:46:43,  8.19s/it]
 49%|████▉     | 18729/37962 [43:23:17<43:38:08,  8.17s/it]
 49%|████▉     | 18730/37962 [43:23:25<43:47:24,  8.20s/it]
                                                           
{'loss': 1.2993, 'grad_norm': 13.477933883666992, 'learning_rate': 1.0699446562271104e-05, 'epoch': 0.49}

 49%|████▉     | 18730/37962 [43:23:25<43:47:24,  8.20s/it]
 49%|████▉     | 18731/37962 [43:23:33<43:50:46,  8.21s/it]
 49%|████▉     | 18732/37962 [43:23:41<43:44:57,  8.19s/it]
 49%|████▉     | 18733/37962 [43:23:50<43:56:22,  8.23s/it]
 49%|████▉     | 18734/37962 [43:23:58<43:21:19,  8.12s/it]
 49%|████▉     | 18735/37962 [43:24:06<43:17:30,  8.11s/it]
 49%|████▉     | 18736/37962 [43:24:14<43:33:29,  8.16s/it]
 49%|████▉     | 18737/37962 [43:24:22<43:12:06,  8.09s/it]
 49%|████▉     | 18738/37962 [43:24:30<43:28:42,  8.14s/it]
 49%|████▉     | 18739/37962 [43:24:38<43:21:22,  8.12s/it]
 49%|████▉     | 18740/37962 [43:24:47<43:46:39,  8.20s/it]
                                                           
{'loss': 1.3694, 'grad_norm': 12.618258476257324, 'learning_rate': 1.0690935599418935e-05, 'epoch': 0.49}

 49%|████▉     | 18740/37962 [43:24:47<43:46:39,  8.20s/it]
 49%|████▉     | 18741/37962 [43:24:55<43:48:07,  8.20s/it]
 49%|████▉     | 18742/37962 [43:25:03<43:50:23,  8.21s/it]
 49%|████▉     | 18743/37962 [43:25:11<43:57:58,  8.24s/it]
 49%|████▉     | 18744/37962 [43:25:20<43:59:02,  8.24s/it]
 49%|████▉     | 18745/37962 [43:25:28<44:09:17,  8.27s/it]
 49%|████▉     | 18746/37962 [43:25:36<44:02:15,  8.25s/it]
 49%|████▉     | 18747/37962 [43:25:44<43:42:21,  8.19s/it]
 49%|████▉     | 18748/37962 [43:25:53<44:04:54,  8.26s/it]
 49%|████▉     | 18749/37962 [43:26:01<43:49:00,  8.21s/it]
 49%|████▉     | 18750/37962 [43:26:09<43:36:55,  8.17s/it]
                                                           
{'loss': 1.2826, 'grad_norm': 6.200601100921631, 'learning_rate': 1.0682424133646712e-05, 'epoch': 0.49}

 49%|████▉     | 18750/37962 [43:26:09<43:36:55,  8.17s/it]
 49%|████▉     | 18751/37962 [43:26:17<43:38:51,  8.18s/it]
 49%|████▉     | 18752/37962 [43:26:25<43:20:17,  8.12s/it]
 49%|████▉     | 18753/37962 [43:26:33<43:36:53,  8.17s/it]
 49%|████▉     | 18754/37962 [43:26:41<43:00:41,  8.06s/it]
 49%|████▉     | 18755/37962 [43:26:49<43:17:53,  8.12s/it]
 49%|████▉     | 18756/37962 [43:26:58<43:31:30,  8.16s/it]
 49%|████▉     | 18757/37962 [43:27:06<43:58:28,  8.24s/it]
 49%|████▉     | 18758/37962 [43:27:14<44:03:14,  8.26s/it]
 49%|████▉     | 18759/37962 [43:27:22<44:01:42,  8.25s/it]
 49%|████▉     | 18760/37962 [43:27:31<43:51:29,  8.22s/it]
                                                           
{'loss': 1.3296, 'grad_norm': 360.7298278808594, 'learning_rate': 1.0673912171149786e-05, 'epoch': 0.49}

 49%|████▉     | 18760/37962 [43:27:31<43:51:29,  8.22s/it]
 49%|████▉     | 18761/37962 [43:27:39<44:04:24,  8.26s/it]
 49%|████▉     | 18762/37962 [43:27:47<43:40:51,  8.19s/it]
 49%|████▉     | 18763/37962 [43:27:55<43:49:09,  8.22s/it]
 49%|████▉     | 18764/37962 [43:28:04<44:14:37,  8.30s/it]
 49%|████▉     | 18765/37962 [43:28:12<44:20:07,  8.31s/it]
 49%|████▉     | 18766/37962 [43:28:20<43:31:20,  8.16s/it]
 49%|████▉     | 18767/37962 [43:28:28<43:48:51,  8.22s/it]
 49%|████▉     | 18768/37962 [43:28:37<43:57:34,  8.25s/it]
 49%|████▉     | 18769/37962 [43:28:45<44:10:08,  8.28s/it]
 49%|████▉     | 18770/37962 [43:28:53<44:08:16,  8.28s/it]
                                                           
{'loss': 1.3509, 'grad_norm': 11.332207679748535, 'learning_rate': 1.0665399718123859e-05, 'epoch': 0.49}

 49%|████▉     | 18770/37962 [43:28:53<44:08:16,  8.28s/it]
 49%|████▉     | 18771/37962 [43:29:01<44:02:03,  8.26s/it]
 49%|████▉     | 18772/37962 [43:29:10<44:15:33,  8.30s/it]
 49%|████▉     | 18773/37962 [43:29:18<44:13:51,  8.30s/it]
 49%|████▉     | 18774/37962 [43:29:27<44:26:38,  8.34s/it]
 49%|████▉     | 18775/37962 [43:29:35<44:48:45,  8.41s/it]
 49%|████▉     | 18776/37962 [43:29:43<44:27:40,  8.34s/it]
 49%|████▉     | 18777/37962 [43:29:52<44:33:50,  8.36s/it]
 49%|████▉     | 18778/37962 [43:30:00<44:42:03,  8.39s/it]
 49%|████▉     | 18779/37962 [43:30:08<44:29:06,  8.35s/it]
 49%|████▉     | 18780/37962 [43:30:17<44:41:24,  8.39s/it]
                                                           
{'loss': 1.4382, 'grad_norm': 11.473334312438965, 'learning_rate': 1.0656886780765007e-05, 'epoch': 0.49}

 49%|████▉     | 18780/37962 [43:30:17<44:41:24,  8.39s/it]
 49%|████▉     | 18781/37962 [43:30:25<44:27:30,  8.34s/it]
 49%|████▉     | 18782/37962 [43:30:33<44:23:29,  8.33s/it]
 49%|████▉     | 18783/37962 [43:30:41<43:42:11,  8.20s/it]
 49%|████▉     | 18784/37962 [43:30:50<43:49:48,  8.23s/it]
 49%|████▉     | 18785/37962 [43:30:58<43:53:01,  8.24s/it]
 49%|████▉     | 18786/37962 [43:31:06<44:04:16,  8.27s/it]
 49%|████▉     | 18787/37962 [43:31:14<43:51:35,  8.23s/it]
 49%|████▉     | 18788/37962 [43:31:23<43:45:47,  8.22s/it]
 49%|████▉     | 18789/37962 [43:31:31<43:38:08,  8.19s/it]
 49%|████▉     | 18790/37962 [43:31:39<43:27:21,  8.16s/it]
                                                           
{'loss': 1.3778, 'grad_norm': 6.741174221038818, 'learning_rate': 1.064837336526965e-05, 'epoch': 0.49}

 49%|████▉     | 18790/37962 [43:31:39<43:27:21,  8.16s/it]
 49%|████▉     | 18791/37962 [43:31:47<43:40:11,  8.20s/it]
 50%|████▉     | 18792/37962 [43:31:55<43:36:41,  8.19s/it]
 50%|████▉     | 18793/37962 [43:32:04<43:42:19,  8.21s/it]
 50%|████▉     | 18794/37962 [43:32:11<42:55:32,  8.06s/it]
 50%|████▉     | 18795/37962 [43:32:20<43:16:06,  8.13s/it]
 50%|████▉     | 18796/37962 [43:32:28<43:32:01,  8.18s/it]
 50%|████▉     | 18797/37962 [43:32:36<43:51:37,  8.24s/it]
 50%|████▉     | 18798/37962 [43:32:44<43:48:37,  8.23s/it]
 50%|████▉     | 18799/37962 [43:32:53<44:20:33,  8.33s/it]
 50%|████▉     | 18800/37962 [43:33:01<43:58:07,  8.26s/it]
                                                           
{'loss': 1.3624, 'grad_norm': 4.8239665031433105, 'learning_rate': 1.0639859477834551e-05, 'epoch': 0.5}

 50%|████▉     | 18800/37962 [43:33:01<43:58:07,  8.26s/it]
 50%|████▉     | 18801/37962 [43:33:10<44:18:11,  8.32s/it]
 50%|████▉     | 18802/37962 [43:33:18<43:57:17,  8.26s/it]
 50%|████▉     | 18803/37962 [43:33:26<44:11:19,  8.30s/it]
 50%|████▉     | 18804/37962 [43:33:35<44:24:47,  8.35s/it]
 50%|████▉     | 18805/37962 [43:33:43<44:19:29,  8.33s/it]
 50%|████▉     | 18806/37962 [43:33:51<44:24:25,  8.35s/it]
 50%|████▉     | 18807/37962 [43:34:00<44:22:37,  8.34s/it]
 50%|████▉     | 18808/37962 [43:34:08<44:26:49,  8.35s/it]
 50%|████▉     | 18809/37962 [43:34:16<44:19:39,  8.33s/it]
 50%|████▉     | 18810/37962 [43:34:24<43:58:25,  8.27s/it]
                                                           
{'loss': 1.4217, 'grad_norm': 9.023909568786621, 'learning_rate': 1.0631345124656825e-05, 'epoch': 0.5}

 50%|████▉     | 18810/37962 [43:34:24<43:58:25,  8.27s/it]
 50%|████▉     | 18811/37962 [43:34:33<44:09:04,  8.30s/it]
 50%|████▉     | 18812/37962 [43:34:41<43:44:26,  8.22s/it]
 50%|████▉     | 18813/37962 [43:34:49<43:48:04,  8.23s/it]
 50%|████▉     | 18814/37962 [43:34:57<43:43:13,  8.22s/it]
 50%|████▉     | 18815/37962 [43:35:05<43:42:10,  8.22s/it]
 50%|████▉     | 18816/37962 [43:35:14<44:09:59,  8.30s/it]
 50%|████▉     | 18817/37962 [43:35:22<43:48:22,  8.24s/it]
 50%|████▉     | 18818/37962 [43:35:30<43:27:19,  8.17s/it]
 50%|████▉     | 18819/37962 [43:35:38<43:28:36,  8.18s/it]
 50%|████▉     | 18820/37962 [43:35:47<43:47:48,  8.24s/it]
                                                           
{'loss': 1.3897, 'grad_norm': 6.240799903869629, 'learning_rate': 1.0622830311933918e-05, 'epoch': 0.5}

 50%|████▉     | 18820/37962 [43:35:47<43:47:48,  8.24s/it]
 50%|████▉     | 18821/37962 [43:35:55<43:44:21,  8.23s/it]
 50%|████▉     | 18822/37962 [43:36:03<43:41:52,  8.22s/it]
 50%|████▉     | 18823/37962 [43:36:11<43:45:06,  8.23s/it]
 50%|████▉     | 18824/37962 [43:36:19<43:12:54,  8.13s/it]
 50%|████▉     | 18825/37962 [43:36:27<43:22:08,  8.16s/it]
 50%|████▉     | 18826/37962 [43:36:35<43:18:55,  8.15s/it]
 50%|████▉     | 18827/37962 [43:36:44<43:51:30,  8.25s/it]
 50%|████▉     | 18828/37962 [43:36:52<43:48:42,  8.24s/it]
 50%|████▉     | 18829/37962 [43:37:00<43:55:24,  8.26s/it]
 50%|████▉     | 18830/37962 [43:37:09<43:48:43,  8.24s/it]
                                                           
{'loss': 1.2823, 'grad_norm': 26.3687744140625, 'learning_rate': 1.0614315045863617e-05, 'epoch': 0.5}

 50%|████▉     | 18830/37962 [43:37:09<43:48:43,  8.24s/it]
 50%|████▉     | 18831/37962 [43:37:17<43:54:41,  8.26s/it]
 50%|████▉     | 18832/37962 [43:37:25<43:56:36,  8.27s/it]
 50%|████▉     | 18833/37962 [43:37:34<44:17:42,  8.34s/it]
 50%|████▉     | 18834/37962 [43:37:42<44:03:53,  8.29s/it]
 50%|████▉     | 18835/37962 [43:37:50<43:25:23,  8.17s/it]
 50%|████▉     | 18836/37962 [43:37:58<43:47:09,  8.24s/it]
 50%|████▉     | 18837/37962 [43:38:07<44:08:01,  8.31s/it]
 50%|████▉     | 18838/37962 [43:38:15<44:04:34,  8.30s/it]
 50%|████▉     | 18839/37962 [43:38:23<43:43:54,  8.23s/it]
 50%|████▉     | 18840/37962 [43:38:32<44:06:09,  8.30s/it]
                                                           
{'loss': 1.3288, 'grad_norm': 7.929842948913574, 'learning_rate': 1.0605799332644036e-05, 'epoch': 0.5}

 50%|████▉     | 18840/37962 [43:38:32<44:06:09,  8.30s/it]
 50%|████▉     | 18841/37962 [43:38:40<44:10:32,  8.32s/it]
 50%|████▉     | 18842/37962 [43:38:48<44:02:02,  8.29s/it]
 50%|████▉     | 18843/37962 [43:38:56<43:35:56,  8.21s/it]
 50%|████▉     | 18844/37962 [43:39:05<44:03:37,  8.30s/it]
 50%|████▉     | 18845/37962 [43:39:13<44:04:49,  8.30s/it]
 50%|████▉     | 18846/37962 [43:39:21<43:54:54,  8.27s/it]
 50%|████▉     | 18847/37962 [43:39:29<44:01:51,  8.29s/it]
 50%|████▉     | 18848/37962 [43:39:38<44:15:11,  8.33s/it]
 50%|████▉     | 18849/37962 [43:39:46<44:07:09,  8.31s/it]
 50%|████▉     | 18850/37962 [43:39:54<43:32:42,  8.20s/it]
                                                           
{'loss': 1.2215, 'grad_norm': 15.663352966308594, 'learning_rate': 1.0597283178473614e-05, 'epoch': 0.5}

 50%|████▉     | 18850/37962 [43:39:54<43:32:42,  8.20s/it]
 50%|████▉     | 18851/37962 [43:40:02<43:32:41,  8.20s/it]
 50%|████▉     | 18852/37962 [43:40:11<43:32:40,  8.20s/it]
 50%|████▉     | 18853/37962 [43:40:19<43:12:07,  8.14s/it]
 50%|████▉     | 18854/37962 [43:40:27<43:29:44,  8.19s/it]
 50%|████▉     | 18855/37962 [43:40:35<44:05:00,  8.31s/it]
 50%|████▉     | 18856/37962 [43:40:44<44:19:19,  8.35s/it]
 50%|████▉     | 18857/37962 [43:40:52<44:08:45,  8.32s/it]
 50%|████▉     | 18858/37962 [43:41:01<44:19:07,  8.35s/it]
 50%|████▉     | 18859/37962 [43:41:09<43:52:55,  8.27s/it]
 50%|████▉     | 18860/37962 [43:41:17<44:00:18,  8.29s/it]
                                                           
{'loss': 1.3198, 'grad_norm': 11.104543685913086, 'learning_rate': 1.0588766589551116e-05, 'epoch': 0.5}

 50%|████▉     | 18860/37962 [43:41:17<44:00:18,  8.29s/it]
 50%|████▉     | 18861/37962 [43:41:25<43:40:44,  8.23s/it]
 50%|████▉     | 18862/37962 [43:41:33<43:56:43,  8.28s/it]
 50%|████▉     | 18863/37962 [43:41:42<43:39:32,  8.23s/it]
 50%|████▉     | 18864/37962 [43:41:50<43:55:01,  8.28s/it]
 50%|████▉     | 18865/37962 [43:41:58<43:46:48,  8.25s/it]
 50%|████▉     | 18866/37962 [43:42:06<43:22:10,  8.18s/it]
 50%|████▉     | 18867/37962 [43:42:14<43:25:48,  8.19s/it]
 50%|████▉     | 18868/37962 [43:42:23<43:42:50,  8.24s/it]
 50%|████▉     | 18869/37962 [43:42:31<43:30:37,  8.20s/it]
 50%|████▉     | 18870/37962 [43:42:39<43:43:38,  8.25s/it]
                                                           
{'loss': 1.3991, 'grad_norm': 11.175535202026367, 'learning_rate': 1.0580249572075615e-05, 'epoch': 0.5}

 50%|████▉     | 18870/37962 [43:42:39<43:43:38,  8.25s/it]
 50%|████▉     | 18871/37962 [43:42:47<43:31:36,  8.21s/it]
 50%|████▉     | 18872/37962 [43:42:55<43:19:35,  8.17s/it]
 50%|████▉     | 18873/37962 [43:43:04<43:28:00,  8.20s/it]
 50%|████▉     | 18874/37962 [43:43:12<43:25:06,  8.19s/it]
 50%|████▉     | 18875/37962 [43:43:20<43:43:26,  8.25s/it]
 50%|████▉     | 18876/37962 [43:43:28<43:40:44,  8.24s/it]
 50%|████▉     | 18877/37962 [43:43:37<43:51:20,  8.27s/it]
 50%|████▉     | 18878/37962 [43:43:45<43:49:23,  8.27s/it]
 50%|████▉     | 18879/37962 [43:43:53<44:04:24,  8.31s/it]
 50%|████▉     | 18880/37962 [43:44:02<43:57:29,  8.29s/it]
                                                           
{'loss': 1.4064, 'grad_norm': 4.537485599517822, 'learning_rate': 1.0571732132246504e-05, 'epoch': 0.5}

 50%|████▉     | 18880/37962 [43:44:02<43:57:29,  8.29s/it]
 50%|████▉     | 18881/37962 [43:44:10<43:39:35,  8.24s/it]
 50%|████▉     | 18882/37962 [43:44:18<43:34:25,  8.22s/it]
 50%|████▉     | 18883/37962 [43:44:26<43:57:34,  8.29s/it]
 50%|████▉     | 18884/37962 [43:44:35<43:50:02,  8.27s/it]
 50%|████▉     | 18885/37962 [43:44:43<43:43:18,  8.25s/it]
 50%|████▉     | 18886/37962 [43:44:51<43:38:25,  8.24s/it]
 50%|████▉     | 18887/37962 [43:44:59<43:22:03,  8.18s/it]
 50%|████▉     | 18888/37962 [43:45:07<43:32:34,  8.22s/it]
 50%|████▉     | 18889/37962 [43:45:16<43:30:11,  8.21s/it]
 50%|████▉     | 18890/37962 [43:45:24<43:26:36,  8.20s/it]
                                                           
{'loss': 1.3479, 'grad_norm': 3.0803215503692627, 'learning_rate': 1.0563214276263481e-05, 'epoch': 0.5}

 50%|████▉     | 18890/37962 [43:45:24<43:26:36,  8.20s/it]
 50%|████▉     | 18891/37962 [43:45:32<43:41:46,  8.25s/it]
 50%|████▉     | 18892/37962 [43:45:41<43:54:47,  8.29s/it]
 50%|████▉     | 18893/37962 [43:45:49<43:47:53,  8.27s/it]
 50%|████▉     | 18894/37962 [43:45:57<43:52:33,  8.28s/it]
 50%|████▉     | 18895/37962 [43:46:05<43:52:18,  8.28s/it]
 50%|████▉     | 18896/37962 [43:46:14<43:59:06,  8.31s/it]
 50%|████▉     | 18897/37962 [43:46:22<43:48:58,  8.27s/it]
 50%|████▉     | 18898/37962 [43:46:30<43:56:13,  8.30s/it]
 50%|████▉     | 18899/37962 [43:46:39<43:59:18,  8.31s/it]
 50%|████▉     | 18900/37962 [43:46:47<44:04:05,  8.32s/it]
                                                           
{'loss': 1.3053, 'grad_norm': 6.9684295654296875, 'learning_rate': 1.055469601032654e-05, 'epoch': 0.5}

 50%|████▉     | 18900/37962 [43:46:47<44:04:05,  8.32s/it]
 50%|████▉     | 18901/37962 [43:46:55<44:07:04,  8.33s/it]
 50%|████▉     | 18902/37962 [43:47:04<44:47:10,  8.46s/it]
 50%|████▉     | 18903/37962 [43:47:12<44:35:22,  8.42s/it]
 50%|████▉     | 18904/37962 [43:47:21<44:37:55,  8.43s/it]
 50%|████▉     | 18905/37962 [43:47:29<44:22:33,  8.38s/it]
 50%|████▉     | 18906/37962 [43:47:37<44:10:54,  8.35s/it]
 50%|████▉     | 18907/37962 [43:47:45<43:34:32,  8.23s/it]
 50%|████▉     | 18908/37962 [43:47:54<43:37:43,  8.24s/it]
 50%|████▉     | 18909/37962 [43:48:02<43:55:07,  8.30s/it]
 50%|████▉     | 18910/37962 [43:48:10<43:45:11,  8.27s/it]
                                                           
{'loss': 1.2095, 'grad_norm': 7.465551376342773, 'learning_rate': 1.0546177340635981e-05, 'epoch': 0.5}

 50%|████▉     | 18910/37962 [43:48:10<43:45:11,  8.27s/it]
 50%|████▉     | 18911/37962 [43:48:19<43:54:59,  8.30s/it]
 50%|████▉     | 18912/37962 [43:48:27<43:41:54,  8.26s/it]
 50%|████▉     | 18913/37962 [43:48:35<43:45:05,  8.27s/it]
 50%|████▉     | 18914/37962 [43:48:43<43:12:03,  8.16s/it]
 50%|████▉     | 18915/37962 [43:48:51<42:32:48,  8.04s/it]
 50%|████▉     | 18916/37962 [43:48:59<42:52:14,  8.10s/it]
 50%|████▉     | 18917/37962 [43:49:07<43:11:42,  8.16s/it]
 50%|████▉     | 18918/37962 [43:49:16<43:37:22,  8.25s/it]
 50%|████▉     | 18919/37962 [43:49:24<43:04:44,  8.14s/it]
 50%|████▉     | 18920/37962 [43:49:32<43:23:43,  8.20s/it]
                                                           
{'loss': 1.3669, 'grad_norm': 11.208521842956543, 'learning_rate': 1.0537658273392404e-05, 'epoch': 0.5}

 50%|████▉     | 18920/37962 [43:49:32<43:23:43,  8.20s/it]
 50%|████▉     | 18921/37962 [43:49:40<43:11:59,  8.17s/it]
 50%|████▉     | 18922/37962 [43:49:48<43:16:43,  8.18s/it]
 50%|████▉     | 18923/37962 [43:49:56<43:07:26,  8.15s/it]
 50%|████▉     | 18924/37962 [43:50:05<43:23:30,  8.21s/it]
 50%|████▉     | 18925/37962 [43:50:13<43:37:26,  8.25s/it]
 50%|████▉     | 18926/37962 [43:50:21<43:42:18,  8.27s/it]
 50%|████▉     | 18927/37962 [43:50:30<43:38:53,  8.25s/it]
 50%|████▉     | 18928/37962 [43:50:38<43:55:15,  8.31s/it]
 50%|████▉     | 18929/37962 [43:50:47<44:19:55,  8.39s/it]
 50%|████▉     | 18930/37962 [43:50:55<44:14:01,  8.37s/it]
                                                           
{'loss': 1.3737, 'grad_norm': 12.768182754516602, 'learning_rate': 1.052913881479668e-05, 'epoch': 0.5}

 50%|████▉     | 18930/37962 [43:50:55<44:14:01,  8.37s/it]
 50%|████▉     | 18931/37962 [43:51:03<44:03:13,  8.33s/it]
 50%|████▉     | 18932/37962 [43:51:11<43:50:07,  8.29s/it]
 50%|████▉     | 18933/37962 [43:51:19<43:03:49,  8.15s/it]
 50%|████▉     | 18934/37962 [43:51:27<43:17:55,  8.19s/it]
 50%|████▉     | 18935/37962 [43:51:35<42:54:06,  8.12s/it]
 50%|████▉     | 18936/37962 [43:51:44<43:05:05,  8.15s/it]
 50%|████▉     | 18937/37962 [43:51:52<43:33:05,  8.24s/it]
 50%|████▉     | 18938/37962 [43:52:00<43:19:02,  8.20s/it]
 50%|████▉     | 18939/37962 [43:52:08<43:20:01,  8.20s/it]
 50%|████▉     | 18940/37962 [43:52:17<43:19:41,  8.20s/it]
                                                           
{'loss': 1.3527, 'grad_norm': 16.34889030456543, 'learning_rate': 1.0520618971049978e-05, 'epoch': 0.5}

 50%|████▉     | 18940/37962 [43:52:17<43:19:41,  8.20s/it]
 50%|████▉     | 18941/37962 [43:52:25<42:55:45,  8.12s/it]
 50%|████▉     | 18942/37962 [43:52:33<43:12:44,  8.18s/it]
 50%|████▉     | 18943/37962 [43:52:41<43:15:27,  8.19s/it]
 50%|████▉     | 18944/37962 [43:52:49<42:48:32,  8.10s/it]
 50%|████▉     | 18945/37962 [43:52:57<42:55:08,  8.12s/it]
 50%|████▉     | 18946/37962 [43:53:05<43:15:56,  8.19s/it]
 50%|████▉     | 18947/37962 [43:53:14<43:37:15,  8.26s/it]
 50%|████▉     | 18948/37962 [43:53:22<43:36:08,  8.26s/it]
 50%|████▉     | 18949/37962 [43:53:30<43:43:04,  8.28s/it]
 50%|████▉     | 18950/37962 [43:53:39<43:39:38,  8.27s/it]
                                                           
{'loss': 1.3147, 'grad_norm': 2.9364967346191406, 'learning_rate': 1.0512098748353748e-05, 'epoch': 0.5}

 50%|████▉     | 18950/37962 [43:53:39<43:39:38,  8.27s/it]
 50%|████▉     | 18951/37962 [43:53:47<44:03:13,  8.34s/it]
 50%|████▉     | 18952/37962 [43:53:56<43:59:13,  8.33s/it]
 50%|████▉     | 18953/37962 [43:54:04<44:04:05,  8.35s/it]
 50%|████▉     | 18954/37962 [43:54:12<43:33:05,  8.25s/it]
 50%|████▉     | 18955/37962 [43:54:20<43:35:29,  8.26s/it]
 50%|████▉     | 18956/37962 [43:54:28<43:30:41,  8.24s/it]
 50%|████▉     | 18957/37962 [43:54:37<43:59:49,  8.33s/it]
 50%|████▉     | 18958/37962 [43:54:45<43:47:02,  8.29s/it]
 50%|████▉     | 18959/37962 [43:54:54<43:53:21,  8.31s/it]
 50%|████▉     | 18960/37962 [43:55:02<43:57:48,  8.33s/it]
                                                           
{'loss': 1.3696, 'grad_norm': 9.796316146850586, 'learning_rate': 1.050357815290971e-05, 'epoch': 0.5}

 50%|████▉     | 18960/37962 [43:55:02<43:57:48,  8.33s/it]
 50%|████▉     | 18961/37962 [43:55:10<44:08:33,  8.36s/it]
 50%|████▉     | 18962/37962 [43:55:19<44:10:11,  8.37s/it]
 50%|████▉     | 18963/37962 [43:55:27<43:54:59,  8.32s/it]
 50%|████▉     | 18964/37962 [43:55:36<44:22:03,  8.41s/it]
 50%|████▉     | 18965/37962 [43:55:44<44:08:40,  8.37s/it]
 50%|████▉     | 18966/37962 [43:55:52<44:08:24,  8.37s/it]
 50%|████▉     | 18967/37962 [43:56:00<43:58:37,  8.33s/it]
 50%|████▉     | 18968/37962 [43:56:09<44:06:00,  8.36s/it]
 50%|████▉     | 18969/37962 [43:56:17<43:54:36,  8.32s/it]
 50%|████▉     | 18970/37962 [43:56:25<43:51:45,  8.31s/it]
                                                           
{'loss': 1.3308, 'grad_norm': 4.13497257232666, 'learning_rate': 1.0495057190919859e-05, 'epoch': 0.5}

 50%|████▉     | 18970/37962 [43:56:25<43:51:45,  8.31s/it]
 50%|████▉     | 18971/37962 [43:56:34<43:55:07,  8.33s/it]
 50%|████▉     | 18972/37962 [43:56:42<44:14:04,  8.39s/it]
 50%|████▉     | 18973/37962 [43:56:51<44:01:23,  8.35s/it]
 50%|████▉     | 18974/37962 [43:56:59<43:43:11,  8.29s/it]
 50%|████▉     | 18975/37962 [43:57:07<44:00:51,  8.35s/it]
 50%|████▉     | 18976/37962 [43:57:15<43:54:16,  8.32s/it]
 50%|████▉     | 18977/37962 [43:57:24<43:51:00,  8.31s/it]
 50%|████▉     | 18978/37962 [43:57:32<43:48:13,  8.31s/it]
 50%|████▉     | 18979/37962 [43:57:40<43:47:04,  8.30s/it]
 50%|████▉     | 18980/37962 [43:57:49<43:50:26,  8.31s/it]
                                                           
{'loss': 1.3738, 'grad_norm': 4.553013801574707, 'learning_rate': 1.0486535868586454e-05, 'epoch': 0.5}

 50%|████▉     | 18980/37962 [43:57:49<43:50:26,  8.31s/it]
 50%|█████     | 18981/37962 [43:57:57<43:41:15,  8.29s/it]
 50%|█████     | 18982/37962 [43:58:05<43:46:45,  8.30s/it]
 50%|█████     | 18983/37962 [43:58:13<43:41:32,  8.29s/it]
 50%|█████     | 18984/37962 [43:58:22<43:33:42,  8.26s/it]
 50%|█████     | 18985/37962 [43:58:30<43:37:42,  8.28s/it]
 50%|█████     | 18986/37962 [43:58:38<43:48:31,  8.31s/it]
 50%|█████     | 18987/37962 [43:58:47<43:34:02,  8.27s/it]
 50%|█████     | 18988/37962 [43:58:55<43:28:00,  8.25s/it]
 50%|█████     | 18989/37962 [43:59:03<43:21:39,  8.23s/it]
 50%|█████     | 18990/37962 [43:59:11<43:39:45,  8.29s/it]
                                                           
{'loss': 1.3732, 'grad_norm': 7.501701831817627, 'learning_rate': 1.0478014192112021e-05, 'epoch': 0.5}

 50%|█████     | 18990/37962 [43:59:11<43:39:45,  8.29s/it]
 50%|█████     | 18991/37962 [43:59:20<43:43:10,  8.30s/it]
 50%|█████     | 18992/37962 [43:59:28<43:31:30,  8.26s/it]
 50%|█████     | 18993/37962 [43:59:36<43:37:21,  8.28s/it]
 50%|█████     | 18994/37962 [43:59:44<43:41:58,  8.29s/it]
 50%|█████     | 18995/37962 [43:59:53<43:37:10,  8.28s/it]
 50%|█████     | 18996/37962 [44:00:01<43:34:33,  8.27s/it]
 50%|█████     | 18997/37962 [44:00:09<43:29:54,  8.26s/it]
 50%|█████     | 18998/37962 [44:00:17<42:47:28,  8.12s/it]
 50%|█████     | 18999/37962 [44:00:25<42:57:28,  8.16s/it]
 50%|█████     | 19000/37962 [44:00:34<43:15:10,  8.21s/it]
                                                           
{'loss': 1.2931, 'grad_norm': 6.131885528564453, 'learning_rate': 1.0469492167699337e-05, 'epoch': 0.5}

 50%|█████     | 19000/37962 [44:00:34<43:15:10,  8.21s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 50%|█████     | 19001/37962 [44:01:25<111:17:34, 21.13s/it]
 50%|█████     | 19002/37962 [44:01:33<91:08:33, 17.31s/it] 
 50%|█████     | 19003/37962 [44:01:41<76:47:42, 14.58s/it]
 50%|█████     | 19004/37962 [44:01:50<66:55:20, 12.71s/it]
 50%|█████     | 19005/37962 [44:01:58<59:18:09, 11.26s/it]
 50%|█████     | 19006/37962 [44:02:06<54:40:02, 10.38s/it]
 50%|█████     | 19007/37962 [44:02:14<51:20:29,  9.75s/it]
 50%|█████     | 19008/37962 [44:02:23<48:58:50,  9.30s/it]
 50%|█████     | 19009/37962 [44:02:31<47:34:11,  9.04s/it]
 50%|█████     | 19010/37962 [44:02:39<46:15:01,  8.79s/it]
                                                           
{'loss': 1.3346, 'grad_norm': 13.974555969238281, 'learning_rate': 1.0460969801551437e-05, 'epoch': 0.5}

 50%|█████     | 19010/37962 [44:02:39<46:15:01,  8.79s/it]
 50%|█████     | 19011/37962 [44:02:48<45:36:03,  8.66s/it]
 50%|█████     | 19012/37962 [44:02:56<45:09:46,  8.58s/it]
 50%|█████     | 19013/37962 [44:03:04<44:59:42,  8.55s/it]
 50%|█████     | 19014/37962 [44:03:13<44:35:33,  8.47s/it]
 50%|█████     | 19015/37962 [44:03:21<44:24:34,  8.44s/it]
 50%|█████     | 19016/37962 [44:03:30<44:39:13,  8.48s/it]
 50%|█████     | 19017/37962 [44:03:38<43:55:54,  8.35s/it]
 50%|█████     | 19018/37962 [44:03:46<43:42:04,  8.30s/it]
 50%|█████     | 19019/37962 [44:03:54<43:40:23,  8.30s/it]
 50%|█████     | 19020/37962 [44:04:02<43:37:34,  8.29s/it]
                                                           
{'loss': 1.3629, 'grad_norm': 8.073199272155762, 'learning_rate': 1.0452447099871605e-05, 'epoch': 0.5}

 50%|█████     | 19020/37962 [44:04:02<43:37:34,  8.29s/it]
 50%|█████     | 19021/37962 [44:04:11<43:42:33,  8.31s/it]
 50%|█████     | 19022/37962 [44:04:19<43:39:27,  8.30s/it]
 50%|█████     | 19023/37962 [44:04:27<43:39:30,  8.30s/it]
 50%|█████     | 19024/37962 [44:04:36<43:31:05,  8.27s/it]
 50%|█████     | 19025/37962 [44:04:44<43:28:50,  8.27s/it]
 50%|█████     | 19026/37962 [44:04:52<43:21:59,  8.24s/it]
 50%|█████     | 19027/37962 [44:05:00<42:52:49,  8.15s/it]
 50%|█████     | 19028/37962 [44:05:08<42:58:16,  8.17s/it]
 50%|█████     | 19029/37962 [44:05:16<42:49:57,  8.14s/it]
 50%|█████     | 19030/37962 [44:05:25<42:59:28,  8.17s/it]
                                                           
{'loss': 1.3221, 'grad_norm': 7.913654804229736, 'learning_rate': 1.044392406886337e-05, 'epoch': 0.5}

 50%|█████     | 19030/37962 [44:05:25<42:59:28,  8.17s/it]
 50%|█████     | 19031/37962 [44:05:33<43:21:06,  8.24s/it]
 50%|█████     | 19032/37962 [44:05:41<43:25:00,  8.26s/it]
 50%|█████     | 19033/37962 [44:05:49<43:24:40,  8.26s/it]
 50%|█████     | 19034/37962 [44:05:58<43:07:09,  8.20s/it]
 50%|█████     | 19035/37962 [44:06:06<43:31:04,  8.28s/it]
 50%|█████     | 19036/37962 [44:06:14<42:50:11,  8.15s/it]
 50%|█████     | 19037/37962 [44:06:22<42:32:08,  8.09s/it]
 50%|█████     | 19038/37962 [44:06:30<42:57:12,  8.17s/it]
 50%|█████     | 19039/37962 [44:06:39<43:17:15,  8.24s/it]
 50%|█████     | 19040/37962 [44:06:47<43:14:21,  8.23s/it]
                                                           
{'loss': 1.3289, 'grad_norm': 11.810351371765137, 'learning_rate': 1.0435400714730489e-05, 'epoch': 0.5}

 50%|█████     | 19040/37962 [44:06:47<43:14:21,  8.23s/it]
 50%|█████     | 19041/37962 [44:06:55<43:34:52,  8.29s/it]
 50%|█████     | 19042/37962 [44:07:03<43:30:36,  8.28s/it]
 50%|█████     | 19043/37962 [44:07:12<43:15:01,  8.23s/it]
 50%|█████     | 19044/37962 [44:07:20<43:06:24,  8.20s/it]
 50%|█████     | 19045/37962 [44:07:28<43:29:20,  8.28s/it]
 50%|█████     | 19046/37962 [44:07:36<43:10:12,  8.22s/it]
 50%|█████     | 19047/37962 [44:07:45<43:29:22,  8.28s/it]
 50%|█████     | 19048/37962 [44:07:53<43:33:02,  8.29s/it]
 50%|█████     | 19049/37962 [44:08:01<43:42:15,  8.32s/it]
 50%|█████     | 19050/37962 [44:08:10<43:47:23,  8.34s/it]
                                                           
{'loss': 1.3315, 'grad_norm': 7.955169677734375, 'learning_rate': 1.0426877043676975e-05, 'epoch': 0.5}

 50%|█████     | 19050/37962 [44:08:10<43:47:23,  8.34s/it]
 50%|█████     | 19051/37962 [44:08:18<43:32:33,  8.29s/it]
 50%|█████     | 19052/37962 [44:08:26<43:29:05,  8.28s/it]
 50%|█████     | 19053/37962 [44:08:34<43:30:47,  8.28s/it]
 50%|█████     | 19054/37962 [44:08:50<54:18:21, 10.34s/it]
 50%|█████     | 19055/37962 [44:08:58<50:39:18,  9.65s/it]
 50%|█████     | 19056/37962 [44:09:06<48:19:13,  9.20s/it]
 50%|█████     | 19057/37962 [44:09:14<47:02:47,  8.96s/it]
 50%|█████     | 19058/37962 [44:09:22<45:54:04,  8.74s/it]
 50%|█████     | 19059/37962 [44:09:31<45:11:53,  8.61s/it]
 50%|█████     | 19060/37962 [44:09:39<44:42:48,  8.52s/it]
                                                           
{'loss': 1.3215, 'grad_norm': 12.637120246887207, 'learning_rate': 1.0418353061907056e-05, 'epoch': 0.5}

 50%|█████     | 19060/37962 [44:09:39<44:42:48,  8.52s/it]
 50%|█████     | 19061/37962 [44:09:47<44:17:37,  8.44s/it]
 50%|█████     | 19062/37962 [44:09:55<43:58:32,  8.38s/it]
 50%|█████     | 19063/37962 [44:10:04<43:45:13,  8.33s/it]
 50%|█████     | 19064/37962 [44:10:12<43:38:01,  8.31s/it]
 50%|█████     | 19065/37962 [44:10:20<43:43:04,  8.33s/it]
 50%|█████     | 19066/37962 [44:10:28<43:00:32,  8.19s/it]
 50%|█████     | 19067/37962 [44:10:37<43:10:51,  8.23s/it]
 50%|█████     | 19068/37962 [44:10:45<42:57:20,  8.18s/it]
 50%|█████     | 19069/37962 [44:10:53<43:12:43,  8.23s/it]
 50%|█████     | 19070/37962 [44:11:01<43:39:29,  8.32s/it]
                                                           
{'loss': 1.4934, 'grad_norm': 13.099584579467773, 'learning_rate': 1.040982877562519e-05, 'epoch': 0.5}

 50%|█████     | 19070/37962 [44:11:01<43:39:29,  8.32s/it]
 50%|█████     | 19071/37962 [44:11:10<43:58:21,  8.38s/it]
 50%|█████     | 19072/37962 [44:11:18<43:42:27,  8.33s/it]
 50%|█████     | 19073/37962 [44:11:26<43:16:47,  8.25s/it]
 50%|█████     | 19074/37962 [44:11:35<43:29:41,  8.29s/it]
 50%|█████     | 19075/37962 [44:11:43<43:51:27,  8.36s/it]
 50%|█████     | 19076/37962 [44:11:51<43:04:26,  8.21s/it]
 50%|█████     | 19077/37962 [44:11:59<42:57:56,  8.19s/it]
 50%|█████     | 19078/37962 [44:12:08<43:23:40,  8.27s/it]
 50%|█████     | 19079/37962 [44:12:16<43:18:06,  8.26s/it]
 50%|█████     | 19080/37962 [44:12:24<43:19:26,  8.26s/it]
                                                           
{'loss': 1.3403, 'grad_norm': 13.352666854858398, 'learning_rate': 1.0401304191036056e-05, 'epoch': 0.5}

 50%|█████     | 19080/37962 [44:12:24<43:19:26,  8.26s/it]
 50%|█████     | 19081/37962 [44:12:32<43:16:27,  8.25s/it]
 50%|█████     | 19082/37962 [44:12:41<43:27:30,  8.29s/it]
 50%|█████     | 19083/37962 [44:12:49<42:51:15,  8.17s/it]
 50%|█████     | 19084/37962 [44:12:57<42:59:19,  8.20s/it]
 50%|█████     | 19085/37962 [44:13:05<43:00:48,  8.20s/it]
 50%|█████     | 19086/37962 [44:13:14<43:24:24,  8.28s/it]
 50%|█████     | 19087/37962 [44:13:21<42:47:25,  8.16s/it]
 50%|█████     | 19088/37962 [44:13:30<43:03:36,  8.21s/it]
 50%|█████     | 19089/37962 [44:13:38<43:24:25,  8.28s/it]
 50%|█████     | 19090/37962 [44:13:46<43:15:17,  8.25s/it]
                                                           
{'loss': 1.3111, 'grad_norm': 7.168139457702637, 'learning_rate': 1.0392779314344556e-05, 'epoch': 0.5}

 50%|█████     | 19090/37962 [44:13:46<43:15:17,  8.25s/it]
 50%|█████     | 19091/37962 [44:13:55<43:17:50,  8.26s/it]
 50%|█████     | 19092/37962 [44:14:03<43:17:54,  8.26s/it]
 50%|█████     | 19093/37962 [44:14:11<42:52:25,  8.18s/it]
 50%|█████     | 19094/37962 [44:14:19<42:23:57,  8.09s/it]
 50%|█████     | 19095/37962 [44:14:27<41:57:22,  8.01s/it]
 50%|█████     | 19096/37962 [44:14:35<42:23:52,  8.09s/it]
 50%|█████     | 19097/37962 [44:14:43<42:34:18,  8.12s/it]
 50%|█████     | 19098/37962 [44:14:51<42:21:26,  8.08s/it]
 50%|█████     | 19099/37962 [44:14:59<42:37:01,  8.13s/it]
 50%|█████     | 19100/37962 [44:15:07<42:19:27,  8.08s/it]
                                                           
{'loss': 1.2771, 'grad_norm': 10.92310905456543, 'learning_rate': 1.0384254151755799e-05, 'epoch': 0.5}

 50%|█████     | 19100/37962 [44:15:07<42:19:27,  8.08s/it]
 50%|█████     | 19101/37962 [44:15:15<42:15:01,  8.06s/it]
 50%|█████     | 19102/37962 [44:15:23<41:54:33,  8.00s/it]
 50%|█████     | 19103/37962 [44:15:32<42:29:27,  8.11s/it]
 50%|█████     | 19104/37962 [44:15:40<42:26:32,  8.10s/it]
 50%|█████     | 19105/37962 [44:15:48<42:41:15,  8.15s/it]
 50%|█████     | 19106/37962 [44:15:56<42:49:06,  8.17s/it]
 50%|█████     | 19107/37962 [44:16:04<42:17:14,  8.07s/it]
 50%|█████     | 19108/37962 [44:16:13<43:07:41,  8.23s/it]
 50%|█████     | 19109/37962 [44:16:20<42:27:52,  8.11s/it]
 50%|█████     | 19110/37962 [44:16:29<42:46:15,  8.17s/it]
                                                           
{'loss': 1.3583, 'grad_norm': 3.1108086109161377, 'learning_rate': 1.03757287094751e-05, 'epoch': 0.5}

 50%|█████     | 19110/37962 [44:16:29<42:46:15,  8.17s/it]
 50%|█████     | 19111/37962 [44:16:44<54:44:25, 10.45s/it]
 50%|█████     | 19112/37962 [44:16:53<51:09:35,  9.77s/it]
 50%|█████     | 19113/37962 [44:17:01<48:40:54,  9.30s/it]
 50%|█████     | 19114/37962 [44:17:09<47:15:20,  9.03s/it]
 50%|█████     | 19115/37962 [44:17:17<45:16:35,  8.65s/it]
 50%|█████     | 19116/37962 [44:17:25<44:24:13,  8.48s/it]
 50%|█████     | 19117/37962 [44:17:33<43:56:06,  8.39s/it]
 50%|█████     | 19118/37962 [44:17:42<43:42:19,  8.35s/it]
 50%|█████     | 19119/37962 [44:17:50<43:37:20,  8.33s/it]
 50%|█████     | 19120/37962 [44:17:58<42:51:09,  8.19s/it]
                                                           
{'loss': 1.2637, 'grad_norm': 9.135017395019531, 'learning_rate': 1.0367202993707981e-05, 'epoch': 0.5}

 50%|█████     | 19120/37962 [44:17:58<42:51:09,  8.19s/it]
 50%|█████     | 19121/37962 [44:18:06<43:01:43,  8.22s/it]
 50%|█████     | 19122/37962 [44:18:14<43:10:34,  8.25s/it]
 50%|█████     | 19123/37962 [44:18:23<43:14:27,  8.26s/it]
 50%|█████     | 19124/37962 [44:18:31<43:15:28,  8.27s/it]
 50%|█████     | 19125/37962 [44:18:39<43:23:01,  8.29s/it]
 50%|█████     | 19126/37962 [44:18:48<43:33:18,  8.32s/it]
 50%|█████     | 19127/37962 [44:18:56<43:50:16,  8.38s/it]
 50%|█████     | 19128/37962 [44:19:05<44:04:44,  8.43s/it]
 50%|█████     | 19129/37962 [44:19:13<44:05:34,  8.43s/it]
 50%|█████     | 19130/37962 [44:19:21<43:29:59,  8.32s/it]
                                                           
{'loss': 1.2925, 'grad_norm': 23.910722732543945, 'learning_rate': 1.0358677010660167e-05, 'epoch': 0.5}

 50%|█████     | 19130/37962 [44:19:21<43:29:59,  8.32s/it]
 50%|█████     | 19131/37962 [44:19:29<43:17:39,  8.28s/it]
 50%|█████     | 19132/37962 [44:19:38<43:21:27,  8.29s/it]
 50%|█████     | 19133/37962 [44:19:46<42:49:27,  8.19s/it]
 50%|█████     | 19134/37962 [44:19:54<42:56:23,  8.21s/it]
 50%|█████     | 19135/37962 [44:20:02<42:59:51,  8.22s/it]
 50%|█████     | 19136/37962 [44:20:10<43:12:05,  8.26s/it]
 50%|█████     | 19137/37962 [44:20:19<43:02:58,  8.23s/it]
 50%|█████     | 19138/37962 [44:20:27<43:11:37,  8.26s/it]
 50%|█████     | 19139/37962 [44:20:35<43:00:24,  8.23s/it]
 50%|█████     | 19140/37962 [44:20:43<43:14:39,  8.27s/it]
                                                           
{'loss': 1.3327, 'grad_norm': 5.586996078491211, 'learning_rate': 1.0350150766537568e-05, 'epoch': 0.5}

 50%|█████     | 19140/37962 [44:20:43<43:14:39,  8.27s/it]
 50%|█████     | 19141/37962 [44:20:52<43:22:58,  8.30s/it]
 50%|█████     | 19142/37962 [44:21:00<42:52:39,  8.20s/it]
 50%|█████     | 19143/37962 [44:21:08<42:25:37,  8.12s/it]
 50%|█████     | 19144/37962 [44:21:16<42:37:14,  8.15s/it]
 50%|█████     | 19145/37962 [44:21:24<42:46:00,  8.18s/it]
 50%|█████     | 19146/37962 [44:21:33<43:08:09,  8.25s/it]
 50%|█████     | 19147/37962 [44:21:41<43:18:13,  8.29s/it]
 50%|█████     | 19148/37962 [44:21:50<43:40:16,  8.36s/it]
 50%|█████     | 19149/37962 [44:21:58<43:34:25,  8.34s/it]
 50%|█████     | 19150/37962 [44:22:06<43:37:24,  8.35s/it]
                                                           
{'loss': 1.4213, 'grad_norm': 17.66317367553711, 'learning_rate': 1.0341624267546293e-05, 'epoch': 0.5}

 50%|█████     | 19150/37962 [44:22:06<43:37:24,  8.35s/it]
 50%|█████     | 19151/37962 [44:22:15<43:37:21,  8.35s/it]
 50%|█████     | 19152/37962 [44:22:23<43:37:14,  8.35s/it]
 50%|█████     | 19153/37962 [44:22:31<43:43:27,  8.37s/it]
 50%|█████     | 19154/37962 [44:22:40<43:30:31,  8.33s/it]
 50%|█████     | 19155/37962 [44:22:48<43:20:30,  8.30s/it]
 50%|█████     | 19156/37962 [44:22:56<43:18:34,  8.29s/it]
 50%|█████     | 19157/37962 [44:23:04<43:26:25,  8.32s/it]
 50%|█████     | 19158/37962 [44:23:13<43:25:48,  8.31s/it]
 50%|█████     | 19159/37962 [44:23:21<43:19:33,  8.30s/it]
 50%|█████     | 19160/37962 [44:23:29<43:16:48,  8.29s/it]
                                                           
{'loss': 1.2873, 'grad_norm': 6.6890177726745605, 'learning_rate': 1.0333097519892632e-05, 'epoch': 0.5}

 50%|█████     | 19160/37962 [44:23:29<43:16:48,  8.29s/it]
 50%|█████     | 19161/37962 [44:23:38<43:30:29,  8.33s/it]
 50%|█████     | 19162/37962 [44:23:46<43:20:10,  8.30s/it]
 50%|█████     | 19163/37962 [44:23:54<43:13:54,  8.28s/it]
 50%|█████     | 19164/37962 [44:24:03<43:25:49,  8.32s/it]
 50%|█████     | 19165/37962 [44:24:11<43:21:16,  8.30s/it]
 50%|█████     | 19166/37962 [44:24:19<43:25:51,  8.32s/it]
 50%|█████     | 19167/37962 [44:24:27<43:06:34,  8.26s/it]
 50%|█████     | 19168/37962 [44:24:36<43:09:40,  8.27s/it]
 50%|█████     | 19169/37962 [44:24:44<42:53:31,  8.22s/it]
 50%|█████     | 19170/37962 [44:24:52<42:56:08,  8.23s/it]
                                                           
{'loss': 1.226, 'grad_norm': 8.560919761657715, 'learning_rate': 1.0324570529783056e-05, 'epoch': 0.5}

 50%|█████     | 19170/37962 [44:24:52<42:56:08,  8.23s/it]
 51%|█████     | 19171/37962 [44:25:00<43:16:58,  8.29s/it]
 51%|█████     | 19172/37962 [44:25:09<43:19:12,  8.30s/it]
 51%|█████     | 19173/37962 [44:25:17<43:28:18,  8.33s/it]
 51%|█████     | 19174/37962 [44:25:26<43:47:16,  8.39s/it]
 51%|█████     | 19175/37962 [44:25:34<43:48:46,  8.40s/it]
 51%|█████     | 19176/37962 [44:25:42<43:32:15,  8.34s/it]
 51%|█████     | 19177/37962 [44:25:51<43:31:12,  8.34s/it]
 51%|█████     | 19178/37962 [44:25:59<43:42:43,  8.38s/it]
 51%|█████     | 19179/37962 [44:26:07<43:34:15,  8.35s/it]
 51%|█████     | 19180/37962 [44:26:16<43:21:13,  8.31s/it]
                                                           
{'loss': 1.3746, 'grad_norm': 8.851001739501953, 'learning_rate': 1.0316043303424215e-05, 'epoch': 0.51}

 51%|█████     | 19180/37962 [44:26:16<43:21:13,  8.31s/it]
 51%|█████     | 19181/37962 [44:26:24<43:28:09,  8.33s/it]
 51%|█████     | 19182/37962 [44:26:32<43:40:11,  8.37s/it]
 51%|█████     | 19183/37962 [44:26:41<43:32:03,  8.35s/it]
 51%|█████     | 19184/37962 [44:26:49<43:36:13,  8.36s/it]
 51%|█████     | 19185/37962 [44:26:57<43:20:22,  8.31s/it]
 51%|█████     | 19186/37962 [44:27:06<43:58:04,  8.43s/it]
 51%|█████     | 19187/37962 [44:27:14<43:48:44,  8.40s/it]
 51%|█████     | 19188/37962 [44:27:23<43:39:24,  8.37s/it]
 51%|█████     | 19189/37962 [44:27:31<43:45:42,  8.39s/it]
 51%|█████     | 19190/37962 [44:27:39<43:48:28,  8.40s/it]
                                                           
{'loss': 1.4172, 'grad_norm': 7.846219062805176, 'learning_rate': 1.0307515847022929e-05, 'epoch': 0.51}

 51%|█████     | 19190/37962 [44:27:39<43:48:28,  8.40s/it]
 51%|█████     | 19191/37962 [44:27:48<44:02:38,  8.45s/it]
 51%|█████     | 19192/37962 [44:27:56<43:56:31,  8.43s/it]
 51%|█████     | 19193/37962 [44:28:05<43:41:24,  8.38s/it]
 51%|█████     | 19194/37962 [44:28:13<43:27:21,  8.34s/it]
 51%|█████     | 19195/37962 [44:28:21<43:25:24,  8.33s/it]
 51%|█████     | 19196/37962 [44:28:29<43:13:27,  8.29s/it]
 51%|█████     | 19197/37962 [44:28:38<43:05:19,  8.27s/it]
 51%|█████     | 19198/37962 [44:28:46<43:17:47,  8.31s/it]
 51%|█████     | 19199/37962 [44:28:54<43:31:09,  8.35s/it]
 51%|█████     | 19200/37962 [44:29:03<43:22:31,  8.32s/it]
                                                           
{'loss': 1.3775, 'grad_norm': 8.423254013061523, 'learning_rate': 1.0298988166786184e-05, 'epoch': 0.51}

 51%|█████     | 19200/37962 [44:29:03<43:22:31,  8.32s/it]
 51%|█████     | 19201/37962 [44:29:11<43:16:05,  8.30s/it]
 51%|█████     | 19202/37962 [44:29:19<43:01:16,  8.26s/it]
 51%|█████     | 19203/37962 [44:29:27<42:44:23,  8.20s/it]
 51%|█████     | 19204/37962 [44:29:36<42:51:02,  8.22s/it]
 51%|█████     | 19205/37962 [44:29:44<42:53:13,  8.23s/it]
 51%|█████     | 19206/37962 [44:29:52<42:52:19,  8.23s/it]
 51%|█████     | 19207/37962 [44:30:00<42:47:57,  8.22s/it]
 51%|█████     | 19208/37962 [44:30:09<43:12:43,  8.29s/it]
 51%|█████     | 19209/37962 [44:30:17<42:35:46,  8.18s/it]
 51%|█████     | 19210/37962 [44:30:25<42:47:22,  8.21s/it]
                                                           
{'loss': 1.291, 'grad_norm': 2.2972495555877686, 'learning_rate': 1.0290460268921133e-05, 'epoch': 0.51}

 51%|█████     | 19210/37962 [44:30:25<42:47:22,  8.21s/it]
 51%|█████     | 19211/37962 [44:30:33<42:22:59,  8.14s/it]
 51%|█████     | 19212/37962 [44:30:41<42:36:02,  8.18s/it]
 51%|█████     | 19213/37962 [44:30:49<42:36:11,  8.18s/it]
 51%|█████     | 19214/37962 [44:30:57<42:39:11,  8.19s/it]
 51%|█████     | 19215/37962 [44:31:06<43:11:34,  8.29s/it]
 51%|█████     | 19216/37962 [44:31:14<43:17:40,  8.31s/it]
 51%|█████     | 19217/37962 [44:31:23<43:07:54,  8.28s/it]
 51%|█████     | 19218/37962 [44:31:31<43:05:32,  8.28s/it]
 51%|█████     | 19219/37962 [44:31:39<42:52:20,  8.23s/it]
 51%|█████     | 19220/37962 [44:31:47<42:57:22,  8.25s/it]
                                                           
{'loss': 1.3582, 'grad_norm': 11.780354499816895, 'learning_rate': 1.0281932159635083e-05, 'epoch': 0.51}

 51%|█████     | 19220/37962 [44:31:47<42:57:22,  8.25s/it]
 51%|█████     | 19221/37962 [44:31:56<42:59:42,  8.26s/it]
 51%|█████     | 19222/37962 [44:32:04<43:01:02,  8.26s/it]
 51%|█████     | 19223/37962 [44:32:12<43:04:19,  8.27s/it]
 51%|█████     | 19224/37962 [44:32:21<43:22:10,  8.33s/it]
 51%|█████     | 19225/37962 [44:32:29<43:09:40,  8.29s/it]
 51%|█████     | 19226/37962 [44:32:37<43:06:17,  8.28s/it]
 51%|█████     | 19227/37962 [44:32:45<43:01:05,  8.27s/it]
 51%|█████     | 19228/37962 [44:32:54<43:07:43,  8.29s/it]
 51%|█████     | 19229/37962 [44:33:02<42:51:25,  8.24s/it]
 51%|█████     | 19230/37962 [44:33:10<42:43:09,  8.21s/it]
                                                           
{'loss': 1.4108, 'grad_norm': 3.0510873794555664, 'learning_rate': 1.02734038451355e-05, 'epoch': 0.51}

 51%|█████     | 19230/37962 [44:33:10<42:43:09,  8.21s/it]
 51%|█████     | 19231/37962 [44:33:18<42:55:25,  8.25s/it]
 51%|█████     | 19232/37962 [44:33:27<43:03:15,  8.28s/it]
 51%|█████     | 19233/37962 [44:33:35<43:45:15,  8.41s/it]
 51%|█████     | 19234/37962 [44:33:44<43:34:04,  8.37s/it]
 51%|█████     | 19235/37962 [44:33:52<43:26:54,  8.35s/it]
 51%|█████     | 19236/37962 [44:34:00<43:25:44,  8.35s/it]
 51%|█████     | 19237/37962 [44:34:09<43:29:08,  8.36s/it]
 51%|█████     | 19238/37962 [44:34:17<43:34:17,  8.38s/it]
 51%|█████     | 19239/37962 [44:34:25<43:17:18,  8.32s/it]
 51%|█████     | 19240/37962 [44:34:34<43:15:38,  8.32s/it]
                                                           
{'loss': 1.3422, 'grad_norm': 11.734567642211914, 'learning_rate': 1.0264875331629993e-05, 'epoch': 0.51}

 51%|█████     | 19240/37962 [44:34:34<43:15:38,  8.32s/it]
 51%|█████     | 19241/37962 [44:34:42<43:44:13,  8.41s/it]
 51%|█████     | 19242/37962 [44:34:50<43:34:23,  8.38s/it]
 51%|█████     | 19243/37962 [44:34:59<43:13:39,  8.31s/it]
 51%|█████     | 19244/37962 [44:35:07<42:35:16,  8.19s/it]
 51%|█████     | 19245/37962 [44:35:15<42:51:20,  8.24s/it]
 51%|█████     | 19246/37962 [44:35:23<41:59:14,  8.08s/it]
 51%|█████     | 19247/37962 [44:35:31<42:14:29,  8.13s/it]
 51%|█████     | 19248/37962 [44:35:39<42:35:50,  8.19s/it]
 51%|█████     | 19249/37962 [44:35:47<42:35:29,  8.19s/it]
 51%|█████     | 19250/37962 [44:35:56<42:48:14,  8.24s/it]
                                                           
{'loss': 1.3607, 'grad_norm': 11.266237258911133, 'learning_rate': 1.0256346625326322e-05, 'epoch': 0.51}

 51%|█████     | 19250/37962 [44:35:56<42:48:14,  8.24s/it]
 51%|█████     | 19251/37962 [44:36:04<42:20:05,  8.15s/it]
 51%|█████     | 19252/37962 [44:36:12<42:49:53,  8.24s/it]
 51%|█████     | 19253/37962 [44:36:20<42:50:56,  8.25s/it]
 51%|█████     | 19254/37962 [44:36:28<42:34:04,  8.19s/it]
 51%|█████     | 19255/37962 [44:36:37<42:39:22,  8.21s/it]
 51%|█████     | 19256/37962 [44:36:45<42:41:22,  8.22s/it]
 51%|█████     | 19257/37962 [44:36:53<42:55:51,  8.26s/it]
 51%|█████     | 19258/37962 [44:37:01<42:33:32,  8.19s/it]
 51%|█████     | 19259/37962 [44:37:10<42:36:22,  8.20s/it]
 51%|█████     | 19260/37962 [44:37:18<42:58:57,  8.27s/it]
                                                           
{'loss': 1.3699, 'grad_norm': 7.599793910980225, 'learning_rate': 1.0247817732432383e-05, 'epoch': 0.51}

 51%|█████     | 19260/37962 [44:37:18<42:58:57,  8.27s/it]
 51%|█████     | 19261/37962 [44:37:26<42:54:11,  8.26s/it]
 51%|█████     | 19262/37962 [44:37:35<43:00:51,  8.28s/it]
 51%|█████     | 19263/37962 [44:37:43<43:04:10,  8.29s/it]
 51%|█████     | 19264/37962 [44:37:51<43:09:31,  8.31s/it]
 51%|█████     | 19265/37962 [44:37:59<42:55:51,  8.27s/it]
 51%|█████     | 19266/37962 [44:38:08<42:51:39,  8.25s/it]
 51%|█████     | 19267/37962 [44:38:16<42:57:18,  8.27s/it]
 51%|█████     | 19268/37962 [44:38:24<42:50:58,  8.25s/it]
 51%|█████     | 19269/37962 [44:38:32<42:12:12,  8.13s/it]
 51%|█████     | 19270/37962 [44:38:40<42:24:24,  8.17s/it]
                                                           
{'loss': 1.213, 'grad_norm': 42.46408462524414, 'learning_rate': 1.0239288659156207e-05, 'epoch': 0.51}

 51%|█████     | 19270/37962 [44:38:40<42:24:24,  8.17s/it]
 51%|█████     | 19271/37962 [44:38:49<42:39:24,  8.22s/it]
 51%|█████     | 19272/37962 [44:38:57<42:33:16,  8.20s/it]
 51%|█████     | 19273/37962 [44:39:05<42:44:01,  8.23s/it]
 51%|█████     | 19274/37962 [44:39:13<42:13:19,  8.13s/it]
 51%|█████     | 19275/37962 [44:39:21<42:30:13,  8.19s/it]
 51%|█████     | 19276/37962 [44:39:29<42:34:39,  8.20s/it]
 51%|█████     | 19277/37962 [44:39:38<42:41:35,  8.23s/it]
 51%|█████     | 19278/37962 [44:39:46<42:38:16,  8.22s/it]
 51%|█████     | 19279/37962 [44:39:54<42:52:39,  8.26s/it]
 51%|█████     | 19280/37962 [44:40:02<42:44:57,  8.24s/it]
                                                           
{'loss': 1.337, 'grad_norm': 7.755181789398193, 'learning_rate': 1.0230759411705966e-05, 'epoch': 0.51}

 51%|█████     | 19280/37962 [44:40:02<42:44:57,  8.24s/it]
 51%|█████     | 19281/37962 [44:40:11<43:01:12,  8.29s/it]
 51%|█████     | 19282/37962 [44:40:19<42:58:29,  8.28s/it]
 51%|█████     | 19283/37962 [44:40:27<42:59:04,  8.28s/it]
 51%|█████     | 19284/37962 [44:40:36<42:57:20,  8.28s/it]
 51%|█████     | 19285/37962 [44:40:44<42:54:46,  8.27s/it]
 51%|█████     | 19286/37962 [44:40:52<42:45:24,  8.24s/it]
 51%|█████     | 19287/37962 [44:41:00<42:32:45,  8.20s/it]
 51%|█████     | 19288/37962 [44:41:09<42:48:26,  8.25s/it]
 51%|█████     | 19289/37962 [44:41:17<42:37:37,  8.22s/it]
 51%|█████     | 19290/37962 [44:41:25<42:46:51,  8.25s/it]
                                                           
{'loss': 1.2605, 'grad_norm': 6.284817695617676, 'learning_rate': 1.0222229996289944e-05, 'epoch': 0.51}

 51%|█████     | 19290/37962 [44:41:25<42:46:51,  8.25s/it]
 51%|█████     | 19291/37962 [44:41:33<42:38:58,  8.22s/it]
 51%|█████     | 19292/37962 [44:41:42<43:14:48,  8.34s/it]
 51%|█████     | 19293/37962 [44:41:50<43:31:37,  8.39s/it]
 51%|█████     | 19294/37962 [44:41:59<43:22:48,  8.37s/it]
 51%|█████     | 19295/37962 [44:42:07<42:53:59,  8.27s/it]
 51%|█████     | 19296/37962 [44:42:15<42:50:27,  8.26s/it]
 51%|█████     | 19297/37962 [44:42:23<42:46:18,  8.25s/it]
 51%|█████     | 19298/37962 [44:42:31<42:51:30,  8.27s/it]
 51%|█████     | 19299/37962 [44:42:40<42:47:42,  8.25s/it]
 51%|█████     | 19300/37962 [44:42:48<43:02:55,  8.30s/it]
                                                           
{'loss': 1.3846, 'grad_norm': 6.445411205291748, 'learning_rate': 1.0213700419116556e-05, 'epoch': 0.51}

 51%|█████     | 19300/37962 [44:42:48<43:02:55,  8.30s/it]
 51%|█████     | 19301/37962 [44:42:56<42:52:49,  8.27s/it]
 51%|█████     | 19302/37962 [44:43:05<42:53:49,  8.28s/it]
 51%|█████     | 19303/37962 [44:43:13<42:45:43,  8.25s/it]
 51%|█████     | 19304/37962 [44:43:21<42:46:23,  8.25s/it]
 51%|█████     | 19305/37962 [44:43:30<43:08:39,  8.32s/it]
 51%|█████     | 19306/37962 [44:43:38<43:37:12,  8.42s/it]
 51%|█████     | 19307/37962 [44:43:46<43:24:45,  8.38s/it]
 51%|█████     | 19308/37962 [44:43:55<43:26:43,  8.38s/it]
 51%|█████     | 19309/37962 [44:44:03<43:35:07,  8.41s/it]
 51%|█████     | 19310/37962 [44:44:12<43:32:46,  8.40s/it]
                                                           
{'loss': 1.4485, 'grad_norm': 5.422335624694824, 'learning_rate': 1.020517068639434e-05, 'epoch': 0.51}

 51%|█████     | 19310/37962 [44:44:12<43:32:46,  8.40s/it]
 51%|█████     | 19311/37962 [44:44:20<43:31:27,  8.40s/it]
 51%|█████     | 19312/37962 [44:44:28<43:18:42,  8.36s/it]
 51%|█████     | 19313/37962 [44:44:37<43:07:55,  8.33s/it]
 51%|█████     | 19314/37962 [44:44:45<43:00:53,  8.30s/it]
 51%|█████     | 19315/37962 [44:44:53<43:03:29,  8.31s/it]
 51%|█████     | 19316/37962 [44:45:02<43:06:17,  8.32s/it]
 51%|█████     | 19317/37962 [44:45:10<43:03:42,  8.31s/it]
 51%|█████     | 19318/37962 [44:45:18<42:59:03,  8.30s/it]
 51%|█████     | 19319/37962 [44:45:26<42:59:49,  8.30s/it]
 51%|█████     | 19320/37962 [44:45:35<42:55:33,  8.29s/it]
                                                           
{'loss': 1.3307, 'grad_norm': 2.1947340965270996, 'learning_rate': 1.019664080433193e-05, 'epoch': 0.51}

 51%|█████     | 19320/37962 [44:45:35<42:55:33,  8.29s/it]
 51%|█████     | 19321/37962 [44:45:43<43:07:05,  8.33s/it]
 51%|█████     | 19322/37962 [44:45:51<42:46:26,  8.26s/it]
 51%|█████     | 19323/37962 [44:45:59<42:41:32,  8.25s/it]
 51%|█████     | 19324/37962 [44:46:08<42:53:01,  8.28s/it]
 51%|█████     | 19325/37962 [44:46:16<43:05:56,  8.33s/it]
 51%|█████     | 19326/37962 [44:46:24<42:38:06,  8.24s/it]
 51%|█████     | 19327/37962 [44:46:33<42:52:45,  8.28s/it]
 51%|█████     | 19328/37962 [44:46:40<41:58:21,  8.11s/it]
 51%|█████     | 19329/37962 [44:46:48<41:46:29,  8.07s/it]
 51%|█████     | 19330/37962 [44:46:57<42:06:17,  8.14s/it]
                                                           
{'loss': 1.3519, 'grad_norm': 15.02253246307373, 'learning_rate': 1.018811077913809e-05, 'epoch': 0.51}

 51%|█████     | 19330/37962 [44:46:57<42:06:17,  8.14s/it]
 51%|█████     | 19331/37962 [44:47:05<42:23:22,  8.19s/it]
 51%|█████     | 19332/37962 [44:47:13<41:43:47,  8.06s/it]
 51%|█████     | 19333/37962 [44:47:21<42:04:23,  8.13s/it]
 51%|█████     | 19334/37962 [44:47:29<42:01:26,  8.12s/it]
 51%|█████     | 19335/37962 [44:47:37<42:17:43,  8.17s/it]
 51%|█████     | 19336/37962 [44:47:46<42:24:40,  8.20s/it]
 51%|█████     | 19337/37962 [44:47:54<42:37:22,  8.24s/it]
 51%|█████     | 19338/37962 [44:48:02<43:00:22,  8.31s/it]
 51%|█████     | 19339/37962 [44:48:10<42:33:37,  8.23s/it]
 51%|█████     | 19340/37962 [44:48:19<42:53:47,  8.29s/it]
                                                           
{'loss': 1.2922, 'grad_norm': 4.987725257873535, 'learning_rate': 1.017958061702167e-05, 'epoch': 0.51}

 51%|█████     | 19340/37962 [44:48:19<42:53:47,  8.29s/it]
 51%|█████     | 19341/37962 [44:48:27<42:47:37,  8.27s/it]
 51%|█████     | 19342/37962 [44:48:36<42:57:48,  8.31s/it]
 51%|█████     | 19343/37962 [44:48:44<42:34:17,  8.23s/it]
 51%|█████     | 19344/37962 [44:48:52<42:25:56,  8.20s/it]
 51%|█████     | 19345/37962 [44:49:00<42:11:36,  8.16s/it]
 51%|█████     | 19346/37962 [44:49:08<42:19:50,  8.19s/it]
 51%|█████     | 19347/37962 [44:49:16<42:28:23,  8.21s/it]
 51%|█████     | 19348/37962 [44:49:25<42:35:43,  8.24s/it]
 51%|█████     | 19349/37962 [44:49:33<42:46:43,  8.27s/it]
 51%|█████     | 19350/37962 [44:49:42<43:11:12,  8.35s/it]
                                                           
{'loss': 1.3173, 'grad_norm': 10.358150482177734, 'learning_rate': 1.017105032419163e-05, 'epoch': 0.51}

 51%|█████     | 19350/37962 [44:49:42<43:11:12,  8.35s/it]
 51%|█████     | 19351/37962 [44:49:50<42:46:04,  8.27s/it]
 51%|█████     | 19352/37962 [44:49:58<42:42:54,  8.26s/it]
 51%|█████     | 19353/37962 [44:50:06<43:08:46,  8.35s/it]
 51%|█████     | 19354/37962 [44:50:15<43:18:40,  8.38s/it]
 51%|█████     | 19355/37962 [44:50:23<43:12:35,  8.36s/it]
 51%|█████     | 19356/37962 [44:50:31<43:05:29,  8.34s/it]
 51%|█████     | 19357/37962 [44:50:40<42:56:53,  8.31s/it]
 51%|█████     | 19358/37962 [44:50:56<54:42:14, 10.59s/it]
 51%|█████     | 19359/37962 [44:51:04<51:36:21,  9.99s/it]
 51%|█████     | 19360/37962 [44:51:13<49:02:55,  9.49s/it]
                                                           
{'loss': 1.2817, 'grad_norm': 7.917600154876709, 'learning_rate': 1.0162519906857024e-05, 'epoch': 0.51}

 51%|█████     | 19360/37962 [44:51:13<49:02:55,  9.49s/it]
 51%|█████     | 19361/37962 [44:51:28<58:01:04, 11.23s/it]
 51%|█████     | 19362/37962 [44:51:36<53:44:09, 10.40s/it]
 51%|█████     | 19363/37962 [44:51:44<49:42:48,  9.62s/it]
 51%|█████     | 19364/37962 [44:51:52<47:35:48,  9.21s/it]
 51%|█████     | 19365/37962 [44:52:01<46:07:16,  8.93s/it]
 51%|█████     | 19366/37962 [44:52:09<45:10:58,  8.75s/it]
 51%|█████     | 19367/37962 [44:52:17<44:23:35,  8.59s/it]
 51%|█████     | 19368/37962 [44:52:25<43:32:59,  8.43s/it]
 51%|█████     | 19369/37962 [44:52:33<42:51:50,  8.30s/it]
 51%|█████     | 19370/37962 [44:52:41<42:20:14,  8.20s/it]
                                                           
{'loss': 1.3676, 'grad_norm': 11.015604019165039, 'learning_rate': 1.0153989371226991e-05, 'epoch': 0.51}

 51%|█████     | 19370/37962 [44:52:41<42:20:14,  8.20s/it]
 51%|█████     | 19371/37962 [44:52:49<42:05:59,  8.15s/it]
 51%|█████     | 19372/37962 [44:52:57<42:07:50,  8.16s/it]
 51%|█████     | 19373/37962 [44:53:06<42:27:47,  8.22s/it]
 51%|█████     | 19374/37962 [44:53:14<42:23:34,  8.21s/it]
 51%|█████     | 19375/37962 [44:53:22<42:41:07,  8.27s/it]
 51%|█████     | 19376/37962 [44:53:31<42:35:18,  8.25s/it]
 51%|█████     | 19377/37962 [44:53:39<42:53:17,  8.31s/it]
 51%|█████     | 19378/37962 [44:53:47<42:31:53,  8.24s/it]
 51%|█████     | 19379/37962 [44:53:55<42:34:14,  8.25s/it]
 51%|█████     | 19380/37962 [44:54:03<42:11:48,  8.18s/it]
                                                           
{'loss': 1.2738, 'grad_norm': 7.387674331665039, 'learning_rate': 1.0145458723510764e-05, 'epoch': 0.51}

 51%|█████     | 19380/37962 [44:54:03<42:11:48,  8.18s/it]
 51%|█████     | 19381/37962 [44:54:12<42:31:57,  8.24s/it]
 51%|█████     | 19382/37962 [44:54:20<42:22:54,  8.21s/it]
 51%|█████     | 19383/37962 [44:54:28<42:32:23,  8.24s/it]
 51%|█████     | 19384/37962 [44:54:37<42:56:30,  8.32s/it]
 51%|█████     | 19385/37962 [44:54:45<42:38:36,  8.26s/it]
 51%|█████     | 19386/37962 [44:54:53<42:34:12,  8.25s/it]
 51%|█████     | 19387/37962 [44:55:01<42:46:41,  8.29s/it]
 51%|█████     | 19388/37962 [44:55:10<42:48:31,  8.30s/it]
 51%|█████     | 19389/37962 [44:55:18<42:58:02,  8.33s/it]
 51%|█████     | 19390/37962 [44:55:26<42:32:05,  8.24s/it]
                                                           
{'loss': 1.3773, 'grad_norm': 9.793679237365723, 'learning_rate': 1.0136927969917651e-05, 'epoch': 0.51}

 51%|█████     | 19390/37962 [44:55:26<42:32:05,  8.24s/it]
 51%|█████     | 19391/37962 [44:55:34<41:56:30,  8.13s/it]
 51%|█████     | 19392/37962 [44:55:42<42:14:29,  8.19s/it]
 51%|█████     | 19393/37962 [44:55:50<41:52:37,  8.12s/it]
 51%|█████     | 19394/37962 [44:55:59<42:12:34,  8.18s/it]
 51%|█████     | 19395/37962 [44:56:07<42:22:02,  8.21s/it]
 51%|█████     | 19396/37962 [44:56:15<42:33:02,  8.25s/it]
 51%|█████     | 19397/37962 [44:56:24<42:34:42,  8.26s/it]
 51%|█████     | 19398/37962 [44:56:32<42:36:15,  8.26s/it]
 51%|█████     | 19399/37962 [44:56:40<42:21:27,  8.21s/it]
 51%|█████     | 19400/37962 [44:56:48<42:20:47,  8.21s/it]
                                                           
{'loss': 1.3782, 'grad_norm': 8.170072555541992, 'learning_rate': 1.0128397116657041e-05, 'epoch': 0.51}

 51%|█████     | 19400/37962 [44:56:48<42:20:47,  8.21s/it]
 51%|█████     | 19401/37962 [44:56:57<42:34:49,  8.26s/it]
 51%|█████     | 19402/37962 [44:57:05<42:35:42,  8.26s/it]
 51%|█████     | 19403/37962 [44:57:13<42:29:39,  8.24s/it]
 51%|█████     | 19404/37962 [44:57:21<42:51:08,  8.31s/it]
 51%|█████     | 19405/37962 [44:57:30<42:48:08,  8.30s/it]
 51%|█████     | 19406/37962 [44:57:38<42:59:15,  8.34s/it]
 51%|█████     | 19407/37962 [44:57:46<42:47:18,  8.30s/it]
 51%|█████     | 19408/37962 [44:57:55<42:48:03,  8.30s/it]
 51%|█████     | 19409/37962 [44:58:03<42:58:46,  8.34s/it]
 51%|█████     | 19410/37962 [44:58:11<42:32:45,  8.26s/it]
                                                           
{'loss': 1.2558, 'grad_norm': 11.991969108581543, 'learning_rate': 1.0119866169938395e-05, 'epoch': 0.51}

 51%|█████     | 19410/37962 [44:58:11<42:32:45,  8.26s/it]
 51%|█████     | 19411/37962 [44:58:20<42:42:48,  8.29s/it]
 51%|█████     | 19412/37962 [44:58:28<42:47:06,  8.30s/it]
 51%|█████     | 19413/37962 [44:58:36<43:01:02,  8.35s/it]
 51%|█████     | 19414/37962 [44:58:45<42:54:57,  8.33s/it]
 51%|█████     | 19415/37962 [44:58:53<42:50:37,  8.32s/it]
 51%|█████     | 19416/37962 [44:59:01<42:58:19,  8.34s/it]
 51%|█████     | 19417/37962 [44:59:09<42:20:02,  8.22s/it]
 51%|█████     | 19418/37962 [44:59:17<42:23:25,  8.23s/it]
 51%|█████     | 19419/37962 [44:59:26<42:19:31,  8.22s/it]
 51%|█████     | 19420/37962 [44:59:34<42:30:10,  8.25s/it]
                                                           
{'loss': 1.242, 'grad_norm': 12.346480369567871, 'learning_rate': 1.0111335135971238e-05, 'epoch': 0.51}

 51%|█████     | 19420/37962 [44:59:34<42:30:10,  8.25s/it]
 51%|█████     | 19421/37962 [44:59:42<42:12:06,  8.19s/it]
 51%|█████     | 19422/37962 [44:59:50<42:09:23,  8.19s/it]
 51%|█████     | 19423/37962 [44:59:58<41:48:15,  8.12s/it]
 51%|█████     | 19424/37962 [45:00:06<42:01:11,  8.16s/it]
 51%|█████     | 19425/37962 [45:00:15<42:27:41,  8.25s/it]
 51%|█████     | 19426/37962 [45:00:23<42:01:19,  8.16s/it]
 51%|█████     | 19427/37962 [45:00:31<41:45:06,  8.11s/it]
 51%|█████     | 19428/37962 [45:00:39<42:12:39,  8.20s/it]
 51%|█████     | 19429/37962 [45:00:47<41:58:06,  8.15s/it]
 51%|█████     | 19430/37962 [45:00:56<42:26:19,  8.24s/it]
                                                           
{'loss': 1.2919, 'grad_norm': 7.305009365081787, 'learning_rate': 1.0102804020965167e-05, 'epoch': 0.51}

 51%|█████     | 19430/37962 [45:00:56<42:26:19,  8.24s/it]
 51%|█████     | 19431/37962 [45:01:04<42:34:58,  8.27s/it]
 51%|█████     | 19432/37962 [45:01:12<42:05:27,  8.18s/it]
 51%|█████     | 19433/37962 [45:01:20<42:18:55,  8.22s/it]
 51%|█████     | 19434/37962 [45:01:29<42:32:18,  8.27s/it]
 51%|█████     | 19435/37962 [45:01:37<42:29:48,  8.26s/it]
 51%|█████     | 19436/37962 [45:01:45<42:24:54,  8.24s/it]
 51%|█████     | 19437/37962 [45:01:53<42:01:57,  8.17s/it]
 51%|█████     | 19438/37962 [45:02:02<42:27:09,  8.25s/it]
 51%|█████     | 19439/37962 [45:02:10<42:18:20,  8.22s/it]
 51%|█████     | 19440/37962 [45:02:18<42:35:48,  8.28s/it]
                                                           
{'loss': 1.2355, 'grad_norm': 13.351061820983887, 'learning_rate': 1.009427283112983e-05, 'epoch': 0.51}

 51%|█████     | 19440/37962 [45:02:18<42:35:48,  8.28s/it]
 51%|█████     | 19441/37962 [45:02:26<42:38:04,  8.29s/it]
 51%|█████     | 19442/37962 [45:02:35<42:28:27,  8.26s/it]
 51%|█████     | 19443/37962 [45:02:43<42:35:44,  8.28s/it]
 51%|█████     | 19444/37962 [45:02:51<42:28:13,  8.26s/it]
 51%|█████     | 19445/37962 [45:02:59<42:24:19,  8.24s/it]
 51%|█████     | 19446/37962 [45:03:08<42:30:43,  8.27s/it]
 51%|█████     | 19447/37962 [45:03:16<42:27:38,  8.26s/it]
 51%|█████     | 19448/37962 [45:03:24<42:38:51,  8.29s/it]
 51%|█████     | 19449/37962 [45:03:33<42:35:07,  8.28s/it]
 51%|█████     | 19450/37962 [45:03:41<42:30:18,  8.27s/it]
                                                           
{'loss': 1.4516, 'grad_norm': 11.127525329589844, 'learning_rate': 1.008574157267493e-05, 'epoch': 0.51}

 51%|█████     | 19450/37962 [45:03:41<42:30:18,  8.27s/it]
 51%|█████     | 19451/37962 [45:03:49<42:50:44,  8.33s/it]
 51%|█████     | 19452/37962 [45:03:57<42:25:55,  8.25s/it]
 51%|█████     | 19453/37962 [45:04:05<42:03:13,  8.18s/it]
 51%|█████     | 19454/37962 [45:04:13<41:40:08,  8.11s/it]
 51%|█████     | 19455/37962 [45:04:22<41:54:31,  8.15s/it]
 51%|█████▏    | 19456/37962 [45:04:30<41:41:35,  8.11s/it]
 51%|█████▏    | 19457/37962 [45:04:38<41:54:06,  8.15s/it]
 51%|█████▏    | 19458/37962 [45:04:46<41:59:19,  8.17s/it]
 51%|█████▏    | 19459/37962 [45:04:55<42:29:32,  8.27s/it]
 51%|█████▏    | 19460/37962 [45:05:03<42:32:25,  8.28s/it]
                                                           
{'loss': 1.3915, 'grad_norm': 12.154691696166992, 'learning_rate': 1.0077210251810225e-05, 'epoch': 0.51}

 51%|█████▏    | 19460/37962 [45:05:03<42:32:25,  8.28s/it]
 51%|█████▏    | 19461/37962 [45:05:11<42:36:17,  8.29s/it]
 51%|█████▏    | 19462/37962 [45:05:20<42:39:11,  8.30s/it]
 51%|█████▏    | 19463/37962 [45:05:28<43:00:12,  8.37s/it]
 51%|█████▏    | 19464/37962 [45:05:36<42:51:18,  8.34s/it]
 51%|█████▏    | 19465/37962 [45:05:44<42:33:07,  8.28s/it]
 51%|█████▏    | 19466/37962 [45:05:53<42:44:50,  8.32s/it]
 51%|█████▏    | 19467/37962 [45:06:01<42:34:35,  8.29s/it]
 51%|█████▏    | 19468/37962 [45:06:09<42:41:09,  8.31s/it]
 51%|█████▏    | 19469/37962 [45:06:18<42:32:24,  8.28s/it]
 51%|█████▏    | 19470/37962 [45:06:26<42:22:55,  8.25s/it]
                                                           
{'loss': 1.4071, 'grad_norm': 7.51495885848999, 'learning_rate': 1.006867887474552e-05, 'epoch': 0.51}

 51%|█████▏    | 19470/37962 [45:06:26<42:22:55,  8.25s/it]
 51%|█████▏    | 19471/37962 [45:06:34<42:28:26,  8.27s/it]
 51%|█████▏    | 19472/37962 [45:06:42<42:33:06,  8.28s/it]
 51%|█████▏    | 19473/37962 [45:06:50<42:08:13,  8.20s/it]
 51%|█████▏    | 19474/37962 [45:06:59<42:12:52,  8.22s/it]
 51%|█████▏    | 19475/37962 [45:07:07<42:03:49,  8.19s/it]
 51%|█████▏    | 19476/37962 [45:07:15<42:11:03,  8.22s/it]
 51%|█████▏    | 19477/37962 [45:07:23<42:15:40,  8.23s/it]
 51%|█████▏    | 19478/37962 [45:07:32<42:29:18,  8.28s/it]
 51%|█████▏    | 19479/37962 [45:07:40<42:23:52,  8.26s/it]
 51%|█████▏    | 19480/37962 [45:07:48<42:34:28,  8.29s/it]
                                                           
{'loss': 1.3156, 'grad_norm': 9.554969787597656, 'learning_rate': 1.0060147447690647e-05, 'epoch': 0.51}

 51%|█████▏    | 19480/37962 [45:07:48<42:34:28,  8.29s/it]
 51%|█████▏    | 19481/37962 [45:07:56<41:53:56,  8.16s/it]
 51%|█████▏    | 19482/37962 [45:08:05<42:06:50,  8.20s/it]
 51%|█████▏    | 19483/37962 [45:08:13<42:09:59,  8.21s/it]
 51%|█████▏    | 19484/37962 [45:08:21<42:21:54,  8.25s/it]
 51%|█████▏    | 19485/37962 [45:08:29<42:29:45,  8.28s/it]
 51%|█████▏    | 19486/37962 [45:08:38<42:07:36,  8.21s/it]
 51%|█████▏    | 19487/37962 [45:08:45<41:32:25,  8.09s/it]
 51%|█████▏    | 19488/37962 [45:08:54<41:49:34,  8.15s/it]
 51%|█████▏    | 19489/37962 [45:09:02<42:02:55,  8.19s/it]
 51%|█████▏    | 19490/37962 [45:09:10<41:53:14,  8.16s/it]
                                                           
{'loss': 1.231, 'grad_norm': 18.020851135253906, 'learning_rate': 1.0051615976855488e-05, 'epoch': 0.51}

 51%|█████▏    | 19490/37962 [45:09:10<41:53:14,  8.16s/it]
 51%|█████▏    | 19491/37962 [45:09:18<41:40:08,  8.12s/it]
 51%|█████▏    | 19492/37962 [45:09:26<41:49:47,  8.15s/it]
 51%|█████▏    | 19493/37962 [45:09:35<42:04:26,  8.20s/it]
 51%|█████▏    | 19494/37962 [45:09:43<42:20:58,  8.26s/it]
 51%|█████▏    | 19495/37962 [45:09:51<42:19:47,  8.25s/it]
 51%|█████▏    | 19496/37962 [45:10:00<42:41:41,  8.32s/it]
 51%|█████▏    | 19497/37962 [45:10:08<42:54:24,  8.37s/it]
 51%|█████▏    | 19498/37962 [45:10:16<42:31:02,  8.29s/it]
 51%|█████▏    | 19499/37962 [45:10:24<42:16:36,  8.24s/it]
 51%|█████▏    | 19500/37962 [45:10:32<41:59:39,  8.19s/it]
                                                           
{'loss': 1.2869, 'grad_norm': 5.085447788238525, 'learning_rate': 1.0043084468449957e-05, 'epoch': 0.51}

 51%|█████▏    | 19500/37962 [45:10:32<41:59:39,  8.19s/it]
 51%|█████▏    | 19501/37962 [45:10:41<42:09:39,  8.22s/it]
 51%|█████▏    | 19502/37962 [45:10:49<42:17:53,  8.25s/it]
 51%|█████▏    | 19503/37962 [45:10:57<42:08:30,  8.22s/it]
 51%|█████▏    | 19504/37962 [45:11:05<41:38:41,  8.12s/it]
 51%|█████▏    | 19505/37962 [45:11:13<41:41:45,  8.13s/it]
 51%|█████▏    | 19506/37962 [45:11:22<42:00:22,  8.19s/it]
 51%|█████▏    | 19507/37962 [45:11:30<42:19:39,  8.26s/it]
 51%|█████▏    | 19508/37962 [45:11:38<42:22:59,  8.27s/it]
 51%|█████▏    | 19509/37962 [45:11:47<42:19:08,  8.26s/it]
 51%|█████▏    | 19510/37962 [45:11:54<41:50:04,  8.16s/it]
                                                           
{'loss': 1.331, 'grad_norm': 11.203155517578125, 'learning_rate': 1.0034552928683987e-05, 'epoch': 0.51}

 51%|█████▏    | 19510/37962 [45:11:54<41:50:04,  8.16s/it]
 51%|█████▏    | 19511/37962 [45:12:03<41:57:34,  8.19s/it]
 51%|█████▏    | 19512/37962 [45:12:11<42:13:40,  8.24s/it]
 51%|█████▏    | 19513/37962 [45:12:19<41:33:16,  8.11s/it]
 51%|█████▏    | 19514/37962 [45:12:27<41:20:39,  8.07s/it]
 51%|█████▏    | 19515/37962 [45:12:35<41:22:40,  8.08s/it]
 51%|█████▏    | 19516/37962 [45:12:43<41:38:58,  8.13s/it]
 51%|█████▏    | 19517/37962 [45:12:52<41:57:16,  8.19s/it]
 51%|█████▏    | 19518/37962 [45:13:00<42:22:15,  8.27s/it]
 51%|█████▏    | 19519/37962 [45:13:08<42:31:08,  8.30s/it]
 51%|█████▏    | 19520/37962 [45:13:17<42:24:30,  8.28s/it]
                                                           
{'loss': 1.3495, 'grad_norm': 8.833269119262695, 'learning_rate': 1.0026021363767538e-05, 'epoch': 0.51}

 51%|█████▏    | 19520/37962 [45:13:17<42:24:30,  8.28s/it]
 51%|█████▏    | 19521/37962 [45:13:25<42:22:01,  8.27s/it]
 51%|█████▏    | 19522/37962 [45:13:33<42:22:36,  8.27s/it]
 51%|█████▏    | 19523/37962 [45:13:42<42:32:44,  8.31s/it]
 51%|█████▏    | 19524/37962 [45:13:50<42:26:40,  8.29s/it]
 51%|█████▏    | 19525/37962 [45:13:58<42:24:42,  8.28s/it]
 51%|█████▏    | 19526/37962 [45:14:06<41:52:23,  8.18s/it]
 51%|█████▏    | 19527/37962 [45:14:14<41:58:54,  8.20s/it]
 51%|█████▏    | 19528/37962 [45:14:23<42:09:43,  8.23s/it]
 51%|█████▏    | 19529/37962 [45:14:31<42:29:56,  8.30s/it]
 51%|█████▏    | 19530/37962 [45:14:39<42:38:51,  8.33s/it]
                                                           
{'loss': 1.4133, 'grad_norm': 8.810002326965332, 'learning_rate': 1.0017489779910589e-05, 'epoch': 0.51}

 51%|█████▏    | 19530/37962 [45:14:39<42:38:51,  8.33s/it]
 51%|█████▏    | 19531/37962 [45:14:48<43:04:41,  8.41s/it]
 51%|█████▏    | 19532/37962 [45:14:56<42:36:10,  8.32s/it]
 51%|█████▏    | 19533/37962 [45:15:04<42:16:06,  8.26s/it]
 51%|█████▏    | 19534/37962 [45:15:12<42:13:03,  8.25s/it]
 51%|█████▏    | 19535/37962 [45:15:21<42:17:15,  8.26s/it]
 51%|█████▏    | 19536/37962 [45:15:29<42:04:30,  8.22s/it]
 51%|█████▏    | 19537/37962 [45:15:37<42:05:15,  8.22s/it]
 51%|█████▏    | 19538/37962 [45:15:45<41:22:57,  8.09s/it]
 51%|█████▏    | 19539/37962 [45:15:53<41:35:09,  8.13s/it]
 51%|█████▏    | 19540/37962 [45:16:01<41:45:09,  8.16s/it]
                                                           
{'loss': 1.3367, 'grad_norm': 6.781354904174805, 'learning_rate': 1.0008958183323134e-05, 'epoch': 0.51}

 51%|█████▏    | 19540/37962 [45:16:01<41:45:09,  8.16s/it]
 51%|█████▏    | 19541/37962 [45:16:09<41:42:02,  8.15s/it]
 51%|█████▏    | 19542/37962 [45:16:18<42:18:34,  8.27s/it]
 51%|█████▏    | 19543/37962 [45:16:33<53:26:34, 10.45s/it]
 51%|█████▏    | 19544/37962 [45:16:42<50:12:30,  9.81s/it]
 51%|█████▏    | 19545/37962 [45:16:50<47:47:07,  9.34s/it]
 51%|█████▏    | 19546/37962 [45:16:58<46:03:39,  9.00s/it]
 51%|█████▏    | 19547/37962 [45:17:06<44:47:19,  8.76s/it]
 51%|█████▏    | 19548/37962 [45:17:15<44:13:44,  8.65s/it]
 51%|█████▏    | 19549/37962 [45:17:23<43:47:13,  8.56s/it]
 51%|█████▏    | 19550/37962 [45:17:31<43:17:47,  8.47s/it]
                                                           
{'loss': 1.3715, 'grad_norm': 5.674666881561279, 'learning_rate': 1.0000426580215173e-05, 'epoch': 0.51}

 51%|█████▏    | 19550/37962 [45:17:31<43:17:47,  8.47s/it]
 52%|█████▏    | 19551/37962 [45:17:40<43:19:00,  8.47s/it]
 52%|█████▏    | 19552/37962 [45:17:48<42:57:24,  8.40s/it]
 52%|█████▏    | 19553/37962 [45:17:56<41:59:11,  8.21s/it]
 52%|█████▏    | 19554/37962 [45:18:04<42:23:12,  8.29s/it]
 52%|█████▏    | 19555/37962 [45:18:13<42:33:20,  8.32s/it]
 52%|█████▏    | 19556/37962 [45:18:21<41:56:06,  8.20s/it]
 52%|█████▏    | 19557/37962 [45:18:29<41:35:59,  8.14s/it]
 52%|█████▏    | 19558/37962 [45:18:37<41:25:02,  8.10s/it]
 52%|█████▏    | 19559/37962 [45:18:45<41:34:09,  8.13s/it]
 52%|█████▏    | 19560/37962 [45:18:53<41:53:51,  8.20s/it]
                                                           
{'loss': 1.3588, 'grad_norm': 11.790881156921387, 'learning_rate': 9.99189497679671e-06, 'epoch': 0.52}

 52%|█████▏    | 19560/37962 [45:18:53<41:53:51,  8.20s/it]
 52%|█████▏    | 19561/37962 [45:19:01<41:50:22,  8.19s/it]
 52%|█████▏    | 19562/37962 [45:19:10<41:55:02,  8.20s/it]
 52%|█████▏    | 19563/37962 [45:19:18<42:02:13,  8.23s/it]
 52%|█████▏    | 19564/37962 [45:19:26<41:58:00,  8.21s/it]
 52%|█████▏    | 19565/37962 [45:19:35<42:30:28,  8.32s/it]
 52%|█████▏    | 19566/37962 [45:19:43<42:20:12,  8.29s/it]
 52%|█████▏    | 19567/37962 [45:19:51<42:18:58,  8.28s/it]
 52%|█████▏    | 19568/37962 [45:19:59<42:18:07,  8.28s/it]
 52%|█████▏    | 19569/37962 [45:20:08<42:18:06,  8.28s/it]
 52%|█████▏    | 19570/37962 [45:20:16<42:28:30,  8.31s/it]
                                                           
{'loss': 1.3065, 'grad_norm': 4.221320152282715, 'learning_rate': 9.983363379277753e-06, 'epoch': 0.52}

 52%|█████▏    | 19570/37962 [45:20:16<42:28:30,  8.31s/it]
 52%|█████▏    | 19571/37962 [45:20:24<42:20:32,  8.29s/it]
 52%|█████▏    | 19572/37962 [45:20:33<42:21:40,  8.29s/it]
 52%|█████▏    | 19573/37962 [45:20:41<41:58:36,  8.22s/it]
 52%|█████▏    | 19574/37962 [45:20:49<41:59:37,  8.22s/it]
 52%|█████▏    | 19575/37962 [45:20:57<41:59:48,  8.22s/it]
 52%|█████▏    | 19576/37962 [45:21:05<42:07:38,  8.25s/it]
 52%|█████▏    | 19577/37962 [45:21:14<42:12:11,  8.26s/it]
 52%|█████▏    | 19578/37962 [45:21:22<41:49:14,  8.19s/it]
 52%|█████▏    | 19579/37962 [45:21:30<41:51:23,  8.20s/it]
 52%|█████▏    | 19580/37962 [45:21:38<41:59:45,  8.22s/it]
                                                           
{'loss': 1.4121, 'grad_norm': 19.112224578857422, 'learning_rate': 9.974831793868302e-06, 'epoch': 0.52}

 52%|█████▏    | 19580/37962 [45:21:38<41:59:45,  8.22s/it]
 52%|█████▏    | 19581/37962 [45:21:47<42:05:39,  8.24s/it]
 52%|█████▏    | 19582/37962 [45:21:55<41:49:56,  8.19s/it]
 52%|█████▏    | 19583/37962 [45:22:03<41:56:09,  8.21s/it]
 52%|█████▏    | 19584/37962 [45:22:11<41:36:44,  8.15s/it]
 52%|█████▏    | 19585/37962 [45:22:19<41:53:57,  8.21s/it]
 52%|█████▏    | 19586/37962 [45:22:27<41:55:14,  8.21s/it]
 52%|█████▏    | 19587/37962 [45:22:36<42:01:25,  8.23s/it]
 52%|█████▏    | 19588/37962 [45:22:44<41:58:58,  8.23s/it]
 52%|█████▏    | 19589/37962 [45:22:52<41:57:39,  8.22s/it]
 52%|█████▏    | 19590/37962 [45:23:01<42:26:31,  8.32s/it]
                                                           
{'loss': 1.4038, 'grad_norm': 6.683890342712402, 'learning_rate': 9.966300226778352e-06, 'epoch': 0.52}

 52%|█████▏    | 19590/37962 [45:23:01<42:26:31,  8.32s/it]
 52%|█████▏    | 19591/37962 [45:23:09<42:17:32,  8.29s/it]
 52%|█████▏    | 19592/37962 [45:23:17<42:37:07,  8.35s/it]
 52%|█████▏    | 19593/37962 [45:23:26<42:10:18,  8.26s/it]
 52%|█████▏    | 19594/37962 [45:23:34<42:37:05,  8.35s/it]
 52%|█████▏    | 19595/37962 [45:23:42<41:51:13,  8.20s/it]
 52%|█████▏    | 19596/37962 [45:23:50<41:43:30,  8.18s/it]
 52%|█████▏    | 19597/37962 [45:23:58<42:03:59,  8.25s/it]
 52%|█████▏    | 19598/37962 [45:24:07<42:07:57,  8.26s/it]
 52%|█████▏    | 19599/37962 [45:24:15<41:42:23,  8.18s/it]
 52%|█████▏    | 19600/37962 [45:24:23<41:58:41,  8.23s/it]
                                                           
{'loss': 1.3573, 'grad_norm': 2.4628565311431885, 'learning_rate': 9.957768684217884e-06, 'epoch': 0.52}

 52%|█████▏    | 19600/37962 [45:24:23<41:58:41,  8.23s/it]
 52%|█████▏    | 19601/37962 [45:24:31<42:13:45,  8.28s/it]
 52%|█████▏    | 19602/37962 [45:24:40<42:25:39,  8.32s/it]
 52%|█████▏    | 19603/37962 [45:24:48<42:17:50,  8.29s/it]
 52%|█████▏    | 19604/37962 [45:24:56<41:55:05,  8.22s/it]
 52%|█████▏    | 19605/37962 [45:25:04<41:29:18,  8.14s/it]
 52%|█████▏    | 19606/37962 [45:25:13<41:53:06,  8.21s/it]
 52%|█████▏    | 19607/37962 [45:25:20<41:09:16,  8.07s/it]
 52%|█████▏    | 19608/37962 [45:25:28<41:25:37,  8.13s/it]
 52%|█████▏    | 19609/37962 [45:25:37<41:42:40,  8.18s/it]
 52%|█████▏    | 19610/37962 [45:25:45<41:47:55,  8.20s/it]
                                                           
{'loss': 1.4354, 'grad_norm': 5.177793025970459, 'learning_rate': 9.949237172396866e-06, 'epoch': 0.52}

 52%|█████▏    | 19610/37962 [45:25:45<41:47:55,  8.20s/it]
 52%|█████▏    | 19611/37962 [45:25:53<41:15:59,  8.10s/it]
 52%|█████▏    | 19612/37962 [45:26:02<42:01:50,  8.25s/it]
 52%|█████▏    | 19613/37962 [45:26:09<41:36:52,  8.16s/it]
 52%|█████▏    | 19614/37962 [45:26:17<41:20:14,  8.11s/it]
 52%|█████▏    | 19615/37962 [45:26:26<42:03:46,  8.25s/it]
 52%|█████▏    | 19616/37962 [45:26:34<42:10:11,  8.27s/it]
 52%|█████▏    | 19617/37962 [45:26:43<42:00:45,  8.24s/it]
 52%|█████▏    | 19618/37962 [45:26:51<41:49:49,  8.21s/it]
 52%|█████▏    | 19619/37962 [45:26:59<42:23:05,  8.32s/it]
 52%|█████▏    | 19620/37962 [45:27:08<42:37:53,  8.37s/it]
                                                           
{'loss': 1.3068, 'grad_norm': 4.745413780212402, 'learning_rate': 9.940705697525223e-06, 'epoch': 0.52}

 52%|█████▏    | 19620/37962 [45:27:08<42:37:53,  8.37s/it]
 52%|█████▏    | 19621/37962 [45:27:16<42:18:35,  8.30s/it]
 52%|█████▏    | 19622/37962 [45:27:24<42:19:46,  8.31s/it]
 52%|█████▏    | 19623/37962 [45:27:33<42:32:25,  8.35s/it]
 52%|█████▏    | 19624/37962 [45:27:41<42:24:04,  8.32s/it]
 52%|█████▏    | 19625/37962 [45:27:49<42:18:16,  8.31s/it]
 52%|█████▏    | 19626/37962 [45:27:57<42:16:58,  8.30s/it]
 52%|█████▏    | 19627/37962 [45:28:06<42:42:39,  8.39s/it]
 52%|█████▏    | 19628/37962 [45:28:14<42:37:09,  8.37s/it]
 52%|█████▏    | 19629/37962 [45:28:23<42:25:08,  8.33s/it]
 52%|█████▏    | 19630/37962 [45:28:31<42:10:59,  8.28s/it]
                                                           
{'loss': 1.39, 'grad_norm': 9.126092910766602, 'learning_rate': 9.932174265812876e-06, 'epoch': 0.52}

 52%|█████▏    | 19630/37962 [45:28:31<42:10:59,  8.28s/it]
 52%|█████▏    | 19631/37962 [45:28:39<42:13:28,  8.29s/it]
 52%|█████▏    | 19632/37962 [45:28:47<42:07:06,  8.27s/it]
 52%|█████▏    | 19633/37962 [45:28:56<42:13:56,  8.29s/it]
 52%|█████▏    | 19634/37962 [45:29:04<42:15:41,  8.30s/it]
 52%|█████▏    | 19635/37962 [45:29:12<42:24:04,  8.33s/it]
 52%|█████▏    | 19636/37962 [45:29:21<42:43:44,  8.39s/it]
 52%|█████▏    | 19637/37962 [45:29:29<41:56:04,  8.24s/it]
 52%|█████▏    | 19638/37962 [45:29:37<41:56:04,  8.24s/it]
 52%|█████▏    | 19639/37962 [45:29:45<41:59:25,  8.25s/it]
 52%|█████▏    | 19640/37962 [45:29:54<42:07:03,  8.28s/it]
                                                           
{'loss': 1.255, 'grad_norm': 6.925912380218506, 'learning_rate': 9.923642883469709e-06, 'epoch': 0.52}

 52%|█████▏    | 19640/37962 [45:29:54<42:07:03,  8.28s/it]
 52%|█████▏    | 19641/37962 [45:30:02<42:12:47,  8.29s/it]
 52%|█████▏    | 19642/37962 [45:30:10<42:18:05,  8.31s/it]
 52%|█████▏    | 19643/37962 [45:30:19<42:17:23,  8.31s/it]
 52%|█████▏    | 19644/37962 [45:30:27<42:26:26,  8.34s/it]
 52%|█████▏    | 19645/37962 [45:30:35<41:47:26,  8.21s/it]
 52%|█████▏    | 19646/37962 [45:30:43<41:53:25,  8.23s/it]
 52%|█████▏    | 19647/37962 [45:30:52<42:22:52,  8.33s/it]
 52%|█████▏    | 19648/37962 [45:31:00<42:12:24,  8.30s/it]
 52%|█████▏    | 19649/37962 [45:31:08<41:52:10,  8.23s/it]
 52%|█████▏    | 19650/37962 [45:31:16<41:26:36,  8.15s/it]
                                                           
{'loss': 1.3306, 'grad_norm': 4.835154056549072, 'learning_rate': 9.915111556705562e-06, 'epoch': 0.52}

 52%|█████▏    | 19650/37962 [45:31:16<41:26:36,  8.15s/it]
 52%|█████▏    | 19651/37962 [45:31:25<41:52:46,  8.23s/it]
 52%|█████▏    | 19652/37962 [45:31:33<42:16:39,  8.31s/it]
 52%|█████▏    | 19653/37962 [45:31:41<42:21:03,  8.33s/it]
 52%|█████▏    | 19654/37962 [45:31:50<42:28:46,  8.35s/it]
 52%|█████▏    | 19655/37962 [45:31:58<42:21:54,  8.33s/it]
 52%|█████▏    | 19656/37962 [45:32:07<42:43:25,  8.40s/it]
 52%|█████▏    | 19657/37962 [45:32:15<42:29:36,  8.36s/it]
 52%|█████▏    | 19658/37962 [45:32:23<42:26:39,  8.35s/it]
 52%|█████▏    | 19659/37962 [45:32:32<42:30:50,  8.36s/it]
 52%|█████▏    | 19660/37962 [45:32:40<42:03:44,  8.27s/it]
                                                           
{'loss': 1.3293, 'grad_norm': 9.339128494262695, 'learning_rate': 9.906580291730244e-06, 'epoch': 0.52}

 52%|█████▏    | 19660/37962 [45:32:40<42:03:44,  8.27s/it]
 52%|█████▏    | 19661/37962 [45:32:48<42:04:37,  8.28s/it]
 52%|█████▏    | 19662/37962 [45:32:56<41:53:08,  8.24s/it]
 52%|█████▏    | 19663/37962 [45:33:04<41:46:11,  8.22s/it]
 52%|█████▏    | 19664/37962 [45:33:12<41:35:29,  8.18s/it]
 52%|█████▏    | 19665/37962 [45:33:21<41:32:16,  8.17s/it]
 52%|█████▏    | 19666/37962 [45:33:29<41:32:37,  8.17s/it]
 52%|█████▏    | 19667/37962 [45:33:37<41:30:25,  8.17s/it]
 52%|█████▏    | 19668/37962 [45:33:45<41:55:45,  8.25s/it]
 52%|█████▏    | 19669/37962 [45:33:54<41:59:05,  8.26s/it]
 52%|█████▏    | 19670/37962 [45:34:02<42:21:32,  8.34s/it]
                                                           
{'loss': 1.4013, 'grad_norm': 5.046908855438232, 'learning_rate': 9.898049094753513e-06, 'epoch': 0.52}

 52%|█████▏    | 19670/37962 [45:34:02<42:21:32,  8.34s/it]
 52%|█████▏    | 19671/37962 [45:34:10<42:04:13,  8.28s/it]
 52%|█████▏    | 19672/37962 [45:34:18<41:57:50,  8.26s/it]
 52%|█████▏    | 19673/37962 [45:34:27<42:05:03,  8.28s/it]
 52%|█████▏    | 19674/37962 [45:34:35<42:09:21,  8.30s/it]
 52%|█████▏    | 19675/37962 [45:34:44<42:17:58,  8.33s/it]
 52%|█████▏    | 19676/37962 [45:34:51<41:40:54,  8.21s/it]
 52%|█████▏    | 19677/37962 [45:34:59<41:20:59,  8.14s/it]
 52%|█████▏    | 19678/37962 [45:35:08<41:38:27,  8.20s/it]
 52%|█████▏    | 19679/37962 [45:35:16<41:13:39,  8.12s/it]
 52%|█████▏    | 19680/37962 [45:35:24<41:34:07,  8.19s/it]
                                                           
{'loss': 1.2559, 'grad_norm': 5.9152655601501465, 'learning_rate': 9.88951797198508e-06, 'epoch': 0.52}

 52%|█████▏    | 19680/37962 [45:35:24<41:34:07,  8.19s/it]
 52%|█████▏    | 19681/37962 [45:35:32<41:49:17,  8.24s/it]
 52%|█████▏    | 19682/37962 [45:35:41<42:00:42,  8.27s/it]
 52%|█████▏    | 19683/37962 [45:35:49<42:02:02,  8.28s/it]
 52%|█████▏    | 19684/37962 [45:35:57<42:02:26,  8.28s/it]
 52%|█████▏    | 19685/37962 [45:36:06<42:02:31,  8.28s/it]
 52%|█████▏    | 19686/37962 [45:36:14<42:09:43,  8.31s/it]
 52%|█████▏    | 19687/37962 [45:36:22<41:51:57,  8.25s/it]
 52%|█████▏    | 19688/37962 [45:36:30<42:03:53,  8.29s/it]
 52%|█████▏    | 19689/37962 [45:36:39<42:13:26,  8.32s/it]
 52%|█████▏    | 19690/37962 [45:36:47<42:14:02,  8.32s/it]
                                                           
{'loss': 1.2487, 'grad_norm': 10.951078414916992, 'learning_rate': 9.8809869296346e-06, 'epoch': 0.52}

 52%|█████▏    | 19690/37962 [45:36:47<42:14:02,  8.32s/it]
 52%|█████▏    | 19691/37962 [45:36:55<42:09:23,  8.31s/it]
 52%|█████▏    | 19692/37962 [45:37:04<42:16:33,  8.33s/it]
 52%|█████▏    | 19693/37962 [45:37:12<41:54:32,  8.26s/it]
 52%|█████▏    | 19694/37962 [45:37:20<42:05:48,  8.30s/it]
 52%|█████▏    | 19695/37962 [45:37:29<42:25:51,  8.36s/it]
 52%|█████▏    | 19696/37962 [45:37:37<42:25:08,  8.36s/it]
 52%|█████▏    | 19697/37962 [45:37:46<42:20:41,  8.35s/it]
 52%|█████▏    | 19698/37962 [45:37:54<41:56:59,  8.27s/it]
 52%|█████▏    | 19699/37962 [45:38:02<42:13:45,  8.32s/it]
 52%|█████▏    | 19700/37962 [45:38:10<42:05:31,  8.30s/it]
                                                           
{'loss': 1.2949, 'grad_norm': 7.968634128570557, 'learning_rate': 9.87245597391167e-06, 'epoch': 0.52}

 52%|█████▏    | 19700/37962 [45:38:10<42:05:31,  8.30s/it]
 52%|█████▏    | 19701/37962 [45:38:18<41:44:01,  8.23s/it]
 52%|█████▏    | 19702/37962 [45:38:27<41:38:25,  8.21s/it]
 52%|█████▏    | 19703/37962 [45:38:35<41:46:51,  8.24s/it]
 52%|█████▏    | 19704/37962 [45:38:43<41:56:26,  8.27s/it]
 52%|█████▏    | 19705/37962 [45:38:51<41:51:58,  8.26s/it]
 52%|█████▏    | 19706/37962 [45:39:00<41:50:29,  8.25s/it]
 52%|█████▏    | 19707/37962 [45:39:08<42:19:35,  8.35s/it]
 52%|█████▏    | 19708/37962 [45:39:16<42:15:06,  8.33s/it]
 52%|█████▏    | 19709/37962 [45:39:25<42:08:03,  8.31s/it]
 52%|█████▏    | 19710/37962 [45:39:33<41:41:45,  8.22s/it]
                                                           
{'loss': 1.3985, 'grad_norm': 6.710391044616699, 'learning_rate': 9.86392511102583e-06, 'epoch': 0.52}

 52%|█████▏    | 19710/37962 [45:39:33<41:41:45,  8.22s/it]
 52%|█████▏    | 19711/37962 [45:39:41<41:42:47,  8.23s/it]
 52%|█████▏    | 19712/37962 [45:39:49<41:37:40,  8.21s/it]
 52%|█████▏    | 19713/37962 [45:39:57<41:35:01,  8.20s/it]
 52%|█████▏    | 19714/37962 [45:40:06<41:34:57,  8.20s/it]
 52%|█████▏    | 19715/37962 [45:40:14<41:32:16,  8.20s/it]
 52%|█████▏    | 19716/37962 [45:40:22<41:38:14,  8.22s/it]
 52%|█████▏    | 19717/37962 [45:40:30<41:40:43,  8.22s/it]
 52%|█████▏    | 19718/37962 [45:40:38<41:38:06,  8.22s/it]
 52%|█████▏    | 19719/37962 [45:40:47<41:38:53,  8.22s/it]
 52%|█████▏    | 19720/37962 [45:40:55<41:40:48,  8.23s/it]
                                                           
{'loss': 1.2565, 'grad_norm': 17.539810180664062, 'learning_rate': 9.855394347186546e-06, 'epoch': 0.52}

 52%|█████▏    | 19720/37962 [45:40:55<41:40:48,  8.23s/it]
 52%|█████▏    | 19721/37962 [45:41:03<41:52:30,  8.26s/it]
 52%|█████▏    | 19722/37962 [45:41:12<42:08:59,  8.32s/it]
 52%|█████▏    | 19723/37962 [45:41:20<42:03:06,  8.30s/it]
 52%|█████▏    | 19724/37962 [45:41:28<42:20:51,  8.36s/it]
 52%|█████▏    | 19725/37962 [45:41:37<42:36:48,  8.41s/it]
 52%|█████▏    | 19726/37962 [45:41:45<42:26:33,  8.38s/it]
 52%|█████▏    | 19727/37962 [45:41:54<42:13:38,  8.34s/it]
 52%|█████▏    | 19728/37962 [45:42:02<42:10:42,  8.33s/it]
 52%|█████▏    | 19729/37962 [45:42:10<42:12:19,  8.33s/it]
 52%|█████▏    | 19730/37962 [45:42:18<42:00:37,  8.30s/it]
                                                           
{'loss': 1.2962, 'grad_norm': 4.416016578674316, 'learning_rate': 9.84686368860321e-06, 'epoch': 0.52}

 52%|█████▏    | 19730/37962 [45:42:18<42:00:37,  8.30s/it]
 52%|█████▏    | 19731/37962 [45:42:26<41:23:38,  8.17s/it]
 52%|█████▏    | 19732/37962 [45:42:35<41:40:19,  8.23s/it]
 52%|█████▏    | 19733/37962 [45:42:43<41:55:19,  8.28s/it]
 52%|█████▏    | 19734/37962 [45:42:51<41:51:14,  8.27s/it]
 52%|█████▏    | 19735/37962 [45:43:00<42:21:03,  8.36s/it]
 52%|█████▏    | 19736/37962 [45:43:08<41:34:39,  8.21s/it]
 52%|█████▏    | 19737/37962 [45:43:16<41:41:27,  8.24s/it]
 52%|█████▏    | 19738/37962 [45:43:24<41:51:16,  8.27s/it]
 52%|█████▏    | 19739/37962 [45:43:33<41:49:26,  8.26s/it]
 52%|█████▏    | 19740/37962 [45:43:41<41:26:06,  8.19s/it]
                                                           
{'loss': 1.3806, 'grad_norm': 13.236982345581055, 'learning_rate': 9.83833314148514e-06, 'epoch': 0.52}

 52%|█████▏    | 19740/37962 [45:43:41<41:26:06,  8.19s/it]
 52%|█████▏    | 19741/37962 [45:43:49<41:41:33,  8.24s/it]
 52%|█████▏    | 19742/37962 [45:43:57<41:37:18,  8.22s/it]
 52%|█████▏    | 19743/37962 [45:44:05<41:42:20,  8.24s/it]
 52%|█████▏    | 19744/37962 [45:44:14<41:44:15,  8.25s/it]
 52%|█████▏    | 19745/37962 [45:44:22<41:54:44,  8.28s/it]
 52%|█████▏    | 19746/37962 [45:44:31<42:25:10,  8.38s/it]
 52%|█████▏    | 19747/37962 [45:44:39<42:09:47,  8.33s/it]
 52%|█████▏    | 19748/37962 [45:44:47<42:18:36,  8.36s/it]
 52%|█████▏    | 19749/37962 [45:44:56<42:06:29,  8.32s/it]
 52%|█████▏    | 19750/37962 [45:45:04<42:20:11,  8.37s/it]
                                                           
{'loss': 1.4437, 'grad_norm': 10.089315414428711, 'learning_rate': 9.829802712041577e-06, 'epoch': 0.52}

 52%|█████▏    | 19750/37962 [45:45:04<42:20:11,  8.37s/it]
 52%|█████▏    | 19751/37962 [45:45:12<42:12:45,  8.34s/it]
 52%|█████▏    | 19752/37962 [45:45:20<41:35:33,  8.22s/it]
 52%|█████▏    | 19753/37962 [45:45:29<41:39:42,  8.24s/it]
 52%|█████▏    | 19754/37962 [45:45:37<42:01:57,  8.31s/it]
 52%|█████▏    | 19755/37962 [45:45:45<41:37:42,  8.23s/it]
 52%|█████▏    | 19756/37962 [45:45:53<41:40:35,  8.24s/it]
 52%|█████▏    | 19757/37962 [45:46:02<41:46:43,  8.26s/it]
 52%|█████▏    | 19758/37962 [45:46:10<41:49:25,  8.27s/it]
 52%|█████▏    | 19759/37962 [45:46:18<41:40:42,  8.24s/it]
 52%|█████▏    | 19760/37962 [45:46:26<41:46:19,  8.26s/it]
                                                           
{'loss': 1.2536, 'grad_norm': 11.692655563354492, 'learning_rate': 9.821272406481672e-06, 'epoch': 0.52}

 52%|█████▏    | 19760/37962 [45:46:26<41:46:19,  8.26s/it]
 52%|█████▏    | 19761/37962 [45:46:35<41:46:57,  8.26s/it]
 52%|█████▏    | 19762/37962 [45:46:43<41:51:29,  8.28s/it]
 52%|█████▏    | 19763/37962 [45:46:51<41:47:16,  8.27s/it]
 52%|█████▏    | 19764/37962 [45:46:59<41:33:02,  8.22s/it]
 52%|█████▏    | 19765/37962 [45:47:08<41:59:55,  8.31s/it]
 52%|█████▏    | 19766/37962 [45:47:16<41:51:25,  8.28s/it]
 52%|█████▏    | 19767/37962 [45:47:24<41:39:21,  8.24s/it]
 52%|█████▏    | 19768/37962 [45:47:32<41:37:29,  8.24s/it]
 52%|█████▏    | 19769/37962 [45:47:41<41:20:52,  8.18s/it]
 52%|█████▏    | 19770/37962 [45:47:49<41:37:23,  8.24s/it]
                                                           
{'loss': 1.2328, 'grad_norm': 9.34780502319336, 'learning_rate': 9.812742231014489e-06, 'epoch': 0.52}

 52%|█████▏    | 19770/37962 [45:47:49<41:37:23,  8.24s/it]
 52%|█████▏    | 19771/37962 [45:47:57<41:16:57,  8.17s/it]
 52%|█████▏    | 19772/37962 [45:48:05<41:07:28,  8.14s/it]
 52%|█████▏    | 19773/37962 [45:48:13<41:23:00,  8.19s/it]
 52%|█████▏    | 19774/37962 [45:48:22<41:30:40,  8.22s/it]
 52%|█████▏    | 19775/37962 [45:48:38<53:17:13, 10.55s/it]
 52%|█████▏    | 19776/37962 [45:48:46<49:54:32,  9.88s/it]
 52%|█████▏    | 19777/37962 [45:48:54<47:28:47,  9.40s/it]
 52%|█████▏    | 19778/37962 [45:49:02<45:45:46,  9.06s/it]
 52%|█████▏    | 19779/37962 [45:49:11<44:47:22,  8.87s/it]
 52%|█████▏    | 19780/37962 [45:49:19<43:48:11,  8.67s/it]
                                                           
{'loss': 1.2737, 'grad_norm': 7.544678688049316, 'learning_rate': 9.804212191848988e-06, 'epoch': 0.52}

 52%|█████▏    | 19780/37962 [45:49:19<43:48:11,  8.67s/it]
 52%|█████▏    | 19781/37962 [45:49:27<43:12:27,  8.56s/it]
 52%|█████▏    | 19782/37962 [45:49:36<42:45:49,  8.47s/it]
 52%|█████▏    | 19783/37962 [45:49:44<42:38:57,  8.45s/it]
 52%|█████▏    | 19784/37962 [45:49:52<42:36:03,  8.44s/it]
 52%|█████▏    | 19785/37962 [45:50:00<41:40:07,  8.25s/it]
 52%|█████▏    | 19786/37962 [45:50:09<41:51:43,  8.29s/it]
 52%|█████▏    | 19787/37962 [45:50:17<41:52:05,  8.29s/it]
 52%|█████▏    | 19788/37962 [45:50:25<41:43:00,  8.26s/it]
 52%|█████▏    | 19789/37962 [45:50:33<41:07:47,  8.15s/it]
 52%|█████▏    | 19790/37962 [45:50:41<40:17:24,  7.98s/it]
                                                           
{'loss': 1.2825, 'grad_norm': 4.627278804779053, 'learning_rate': 9.79568229519404e-06, 'epoch': 0.52}

 52%|█████▏    | 19790/37962 [45:50:41<40:17:24,  7.98s/it]
 52%|█████▏    | 19791/37962 [45:50:49<40:50:56,  8.09s/it]
 52%|█████▏    | 19792/37962 [45:50:57<40:34:39,  8.04s/it]
 52%|█████▏    | 19793/37962 [45:51:05<40:59:32,  8.12s/it]
 52%|█████▏    | 19794/37962 [45:51:13<41:15:04,  8.17s/it]
 52%|█████▏    | 19795/37962 [45:51:22<41:06:12,  8.15s/it]
 52%|█████▏    | 19796/37962 [45:51:30<41:19:18,  8.19s/it]
 52%|█████▏    | 19797/37962 [45:51:38<41:22:46,  8.20s/it]
 52%|█████▏    | 19798/37962 [45:51:46<41:33:45,  8.24s/it]
 52%|█████▏    | 19799/37962 [45:51:55<41:30:01,  8.23s/it]
 52%|█████▏    | 19800/37962 [45:52:03<41:42:20,  8.27s/it]
                                                           
{'loss': 1.3214, 'grad_norm': 6.48158597946167, 'learning_rate': 9.787152547258411e-06, 'epoch': 0.52}

 52%|█████▏    | 19800/37962 [45:52:03<41:42:20,  8.27s/it]
 52%|█████▏    | 19801/37962 [45:52:11<41:01:41,  8.13s/it]
 52%|█████▏    | 19802/37962 [45:52:19<41:12:16,  8.17s/it]
 52%|█████▏    | 19803/37962 [45:52:27<41:12:25,  8.17s/it]
 52%|█████▏    | 19804/37962 [45:52:36<41:43:35,  8.27s/it]
 52%|█████▏    | 19805/37962 [45:52:44<41:41:43,  8.27s/it]
 52%|█████▏    | 19806/37962 [45:52:52<41:22:15,  8.20s/it]
 52%|█████▏    | 19807/37962 [45:53:00<41:13:30,  8.17s/it]
 52%|█████▏    | 19808/37962 [45:53:08<41:29:10,  8.23s/it]
 52%|█████▏    | 19809/37962 [45:53:17<41:50:06,  8.30s/it]
 52%|█████▏    | 19810/37962 [45:53:25<41:52:55,  8.31s/it]
                                                           
{'loss': 1.2974, 'grad_norm': 7.222095489501953, 'learning_rate': 9.778622954250756e-06, 'epoch': 0.52}

 52%|█████▏    | 19810/37962 [45:53:25<41:52:55,  8.31s/it]
 52%|█████▏    | 19811/37962 [45:53:34<41:50:09,  8.30s/it]
 52%|█████▏    | 19812/37962 [45:53:42<41:55:36,  8.32s/it]
 52%|█████▏    | 19813/37962 [45:53:50<41:54:46,  8.31s/it]
 52%|█████▏    | 19814/37962 [45:53:58<41:47:29,  8.29s/it]
 52%|█████▏    | 19815/37962 [45:54:07<41:46:02,  8.29s/it]
 52%|█████▏    | 19816/37962 [45:54:15<41:52:50,  8.31s/it]
 52%|█████▏    | 19817/37962 [45:54:23<41:26:17,  8.22s/it]
 52%|█████▏    | 19818/37962 [45:54:31<41:20:45,  8.20s/it]
 52%|█████▏    | 19819/37962 [45:54:40<41:31:41,  8.24s/it]
 52%|█████▏    | 19820/37962 [45:54:48<41:40:44,  8.27s/it]
                                                           
{'loss': 1.2679, 'grad_norm': 6.295620918273926, 'learning_rate': 9.77009352237962e-06, 'epoch': 0.52}

 52%|█████▏    | 19820/37962 [45:54:48<41:40:44,  8.27s/it]
 52%|█████▏    | 19821/37962 [45:54:56<41:49:04,  8.30s/it]
 52%|█████▏    | 19822/37962 [45:55:05<42:04:55,  8.35s/it]
 52%|█████▏    | 19823/37962 [45:55:13<41:56:25,  8.32s/it]
 52%|█████▏    | 19824/37962 [45:55:21<41:58:29,  8.33s/it]
 52%|█████▏    | 19825/37962 [45:55:30<42:01:40,  8.34s/it]
 52%|█████▏    | 19826/37962 [45:55:38<42:02:57,  8.35s/it]
 52%|█████▏    | 19827/37962 [45:55:46<41:56:44,  8.33s/it]
 52%|█████▏    | 19828/37962 [45:55:55<41:56:49,  8.33s/it]
 52%|█████▏    | 19829/37962 [45:56:04<42:42:37,  8.48s/it]
 52%|█████▏    | 19830/37962 [45:56:11<41:56:41,  8.33s/it]
                                                           
{'loss': 1.3917, 'grad_norm': 8.712936401367188, 'learning_rate': 9.761564257853422e-06, 'epoch': 0.52}

 52%|█████▏    | 19830/37962 [45:56:11<41:56:41,  8.33s/it]
 52%|█████▏    | 19831/37962 [45:56:20<41:42:42,  8.28s/it]
 52%|█████▏    | 19832/37962 [45:56:28<41:21:02,  8.21s/it]
 52%|█████▏    | 19833/37962 [45:56:36<41:13:40,  8.19s/it]
 52%|█████▏    | 19834/37962 [45:56:44<41:24:28,  8.22s/it]
 52%|█████▏    | 19835/37962 [45:56:52<41:01:50,  8.15s/it]
 52%|█████▏    | 19836/37962 [45:57:00<40:55:00,  8.13s/it]
 52%|█████▏    | 19837/37962 [45:57:09<41:14:34,  8.19s/it]
 52%|█████▏    | 19838/37962 [45:57:17<41:32:23,  8.25s/it]
 52%|█████▏    | 19839/37962 [45:57:25<41:47:25,  8.30s/it]
 52%|█████▏    | 19840/37962 [45:57:33<41:18:09,  8.20s/it]
                                                           
{'loss': 1.2978, 'grad_norm': 12.404751777648926, 'learning_rate': 9.753035166880472e-06, 'epoch': 0.52}

 52%|█████▏    | 19840/37962 [45:57:33<41:18:09,  8.20s/it]
 52%|█████▏    | 19841/37962 [45:57:42<41:23:54,  8.22s/it]
 52%|█████▏    | 19842/37962 [45:57:50<41:35:14,  8.26s/it]
 52%|█████▏    | 19843/37962 [45:57:58<41:01:29,  8.15s/it]
 52%|█████▏    | 19844/37962 [45:58:06<41:05:32,  8.16s/it]
 52%|█████▏    | 19845/37962 [45:58:14<41:12:54,  8.19s/it]
 52%|█████▏    | 19846/37962 [45:58:23<41:26:34,  8.24s/it]
 52%|█████▏    | 19847/37962 [45:58:31<41:38:06,  8.27s/it]
 52%|█████▏    | 19848/37962 [45:58:39<41:27:31,  8.24s/it]
 52%|█████▏    | 19849/37962 [45:58:47<41:27:10,  8.24s/it]
 52%|█████▏    | 19850/37962 [45:58:55<41:02:56,  8.16s/it]
                                                           
{'loss': 1.258, 'grad_norm': 3.395648717880249, 'learning_rate': 9.744506255668946e-06, 'epoch': 0.52}

 52%|█████▏    | 19850/37962 [45:58:55<41:02:56,  8.16s/it]
 52%|█████▏    | 19851/37962 [45:59:04<41:20:12,  8.22s/it]
 52%|█████▏    | 19852/37962 [45:59:12<40:52:06,  8.12s/it]
 52%|█████▏    | 19853/37962 [45:59:20<40:58:10,  8.14s/it]
 52%|█████▏    | 19854/37962 [45:59:28<40:54:12,  8.13s/it]
 52%|█████▏    | 19855/37962 [45:59:36<41:00:36,  8.15s/it]
 52%|█████▏    | 19856/37962 [45:59:44<41:02:54,  8.16s/it]
 52%|█████▏    | 19857/37962 [45:59:53<41:07:01,  8.18s/it]
 52%|█████▏    | 19858/37962 [46:00:01<41:12:17,  8.19s/it]
 52%|█████▏    | 19859/37962 [46:00:09<41:28:51,  8.25s/it]
 52%|█████▏    | 19860/37962 [46:00:17<41:36:12,  8.27s/it]
                                                           
{'loss': 1.3101, 'grad_norm': 4.312768936157227, 'learning_rate': 9.735977530426889e-06, 'epoch': 0.52}

 52%|█████▏    | 19860/37962 [46:00:17<41:36:12,  8.27s/it]
 52%|█████▏    | 19861/37962 [46:00:26<41:31:37,  8.26s/it]
 52%|█████▏    | 19862/37962 [46:00:34<41:47:34,  8.31s/it]
 52%|█████▏    | 19863/37962 [46:00:43<42:00:23,  8.36s/it]
 52%|█████▏    | 19864/37962 [46:00:51<41:28:22,  8.25s/it]
 52%|█████▏    | 19865/37962 [46:00:59<41:26:19,  8.24s/it]
 52%|█████▏    | 19866/37962 [46:01:07<41:48:10,  8.32s/it]
 52%|█████▏    | 19867/37962 [46:01:16<41:47:42,  8.32s/it]
 52%|█████▏    | 19868/37962 [46:01:24<41:59:10,  8.35s/it]
 52%|█████▏    | 19869/37962 [46:01:32<42:04:26,  8.37s/it]
 52%|█████▏    | 19870/37962 [46:01:41<42:12:23,  8.40s/it]
                                                           
{'loss': 1.325, 'grad_norm': 5.7269439697265625, 'learning_rate': 9.72744899736221e-06, 'epoch': 0.52}

 52%|█████▏    | 19870/37962 [46:01:41<42:12:23,  8.40s/it]
 52%|█████▏    | 19871/37962 [46:01:49<42:03:36,  8.37s/it]
 52%|█████▏    | 19872/37962 [46:01:57<41:32:05,  8.27s/it]
 52%|█████▏    | 19873/37962 [46:02:06<41:49:12,  8.32s/it]
 52%|█████▏    | 19874/37962 [46:02:14<41:39:33,  8.29s/it]
 52%|█████▏    | 19875/37962 [46:02:22<41:01:47,  8.17s/it]
 52%|█████▏    | 19876/37962 [46:02:30<41:04:32,  8.18s/it]
 52%|█████▏    | 19877/37962 [46:02:38<40:58:23,  8.16s/it]
 52%|█████▏    | 19878/37962 [46:02:46<41:18:56,  8.22s/it]
 52%|█████▏    | 19879/37962 [46:02:55<41:19:04,  8.23s/it]
 52%|█████▏    | 19880/37962 [46:03:03<41:35:59,  8.28s/it]
                                                           
{'loss': 1.3277, 'grad_norm': 10.166214942932129, 'learning_rate': 9.718920662682695e-06, 'epoch': 0.52}

 52%|█████▏    | 19880/37962 [46:03:03<41:35:59,  8.28s/it]
 52%|█████▏    | 19881/37962 [46:03:12<41:51:02,  8.33s/it]
 52%|█████▏    | 19882/37962 [46:03:20<41:34:07,  8.28s/it]
 52%|█████▏    | 19883/37962 [46:03:28<41:34:11,  8.28s/it]
 52%|█████▏    | 19884/37962 [46:03:37<42:14:39,  8.41s/it]
 52%|█████▏    | 19885/37962 [46:03:45<41:56:11,  8.35s/it]
 52%|█████▏    | 19886/37962 [46:03:53<41:33:10,  8.28s/it]
 52%|█████▏    | 19887/37962 [46:04:01<41:45:31,  8.32s/it]
 52%|█████▏    | 19888/37962 [46:04:10<41:52:39,  8.34s/it]
 52%|█████▏    | 19889/37962 [46:04:18<41:38:30,  8.29s/it]
 52%|█████▏    | 19890/37962 [46:04:26<41:29:33,  8.27s/it]
                                                           
{'loss': 1.3079, 'grad_norm': 4.362672805786133, 'learning_rate': 9.710392532595954e-06, 'epoch': 0.52}

 52%|█████▏    | 19890/37962 [46:04:26<41:29:33,  8.27s/it]
 52%|█████▏    | 19891/37962 [46:04:35<41:54:04,  8.35s/it]
 52%|█████▏    | 19892/37962 [46:04:43<42:02:32,  8.38s/it]
 52%|█████▏    | 19893/37962 [46:04:52<41:55:04,  8.35s/it]
 52%|█████▏    | 19894/37962 [46:05:00<41:41:29,  8.31s/it]
 52%|█████▏    | 19895/37962 [46:05:08<41:57:45,  8.36s/it]
 52%|█████▏    | 19896/37962 [46:05:17<41:55:33,  8.35s/it]
 52%|█████▏    | 19897/37962 [46:05:25<41:48:54,  8.33s/it]
 52%|█████▏    | 19898/37962 [46:05:33<41:40:44,  8.31s/it]
 52%|█████▏    | 19899/37962 [46:05:41<41:20:35,  8.24s/it]
 52%|█████▏    | 19900/37962 [46:05:49<40:57:22,  8.16s/it]
                                                           
{'loss': 1.3062, 'grad_norm': 10.407474517822266, 'learning_rate': 9.701864613309468e-06, 'epoch': 0.52}

 52%|█████▏    | 19900/37962 [46:05:49<40:57:22,  8.16s/it]
 52%|█████▏    | 19901/37962 [46:05:57<40:49:18,  8.14s/it]
 52%|█████▏    | 19902/37962 [46:06:06<41:09:08,  8.20s/it]
 52%|█████▏    | 19903/37962 [46:06:14<41:09:08,  8.20s/it]
 52%|█████▏    | 19904/37962 [46:06:22<41:37:39,  8.30s/it]
 52%|█████▏    | 19905/37962 [46:06:31<41:45:08,  8.32s/it]
 52%|█████▏    | 19906/37962 [46:06:39<41:03:08,  8.19s/it]
 52%|█████▏    | 19907/37962 [46:06:47<41:06:49,  8.20s/it]
 52%|█████▏    | 19908/37962 [46:06:55<40:54:50,  8.16s/it]
 52%|█████▏    | 19909/37962 [46:07:03<41:06:05,  8.20s/it]
 52%|█████▏    | 19910/37962 [46:07:11<40:51:36,  8.15s/it]
                                                           
{'loss': 1.3597, 'grad_norm': 10.98146915435791, 'learning_rate': 9.693336911030568e-06, 'epoch': 0.52}

 52%|█████▏    | 19910/37962 [46:07:11<40:51:36,  8.15s/it]
 52%|█████▏    | 19911/37962 [46:07:19<40:58:52,  8.17s/it]
 52%|█████▏    | 19912/37962 [46:07:28<41:13:37,  8.22s/it]
 52%|█████▏    | 19913/37962 [46:07:36<41:34:43,  8.29s/it]
 52%|█████▏    | 19914/37962 [46:07:44<41:29:43,  8.28s/it]
 52%|█████▏    | 19915/37962 [46:07:53<41:33:32,  8.29s/it]
 52%|█████▏    | 19916/37962 [46:08:01<41:32:47,  8.29s/it]
 52%|█████▏    | 19917/37962 [46:08:09<41:35:33,  8.30s/it]
 52%|█████▏    | 19918/37962 [46:08:18<41:37:50,  8.31s/it]
 52%|█████▏    | 19919/37962 [46:08:26<41:35:35,  8.30s/it]
 52%|█████▏    | 19920/37962 [46:08:34<41:46:57,  8.34s/it]
                                                           
{'loss': 1.4415, 'grad_norm': 7.723073482513428, 'learning_rate': 9.684809431966419e-06, 'epoch': 0.52}

 52%|█████▏    | 19920/37962 [46:08:34<41:46:57,  8.34s/it]
 52%|█████▏    | 19921/37962 [46:08:42<41:12:01,  8.22s/it]
 52%|█████▏    | 19922/37962 [46:08:51<41:27:04,  8.27s/it]
 52%|█████▏    | 19923/37962 [46:08:59<41:25:00,  8.27s/it]
 52%|█████▏    | 19924/37962 [46:09:07<41:29:07,  8.28s/it]
 52%|█████▏    | 19925/37962 [46:09:15<41:16:18,  8.24s/it]
 52%|█████▏    | 19926/37962 [46:09:24<41:26:05,  8.27s/it]
 52%|█████▏    | 19927/37962 [46:09:32<41:05:10,  8.20s/it]
 52%|█████▏    | 19928/37962 [46:09:40<40:54:57,  8.17s/it]
 52%|█████▏    | 19929/37962 [46:09:48<41:06:16,  8.21s/it]
 52%|█████▏    | 19930/37962 [46:09:57<41:23:04,  8.26s/it]
                                                           
{'loss': 1.3069, 'grad_norm': 7.867602348327637, 'learning_rate': 9.676282182324025e-06, 'epoch': 0.52}

 52%|█████▏    | 19930/37962 [46:09:57<41:23:04,  8.26s/it]
 53%|█████▎    | 19931/37962 [46:10:05<41:49:19,  8.35s/it]
 53%|█████▎    | 19932/37962 [46:10:14<42:01:51,  8.39s/it]
 53%|█████▎    | 19933/37962 [46:10:22<41:45:27,  8.34s/it]
 53%|█████▎    | 19934/37962 [46:10:30<41:57:45,  8.38s/it]
 53%|█████▎    | 19935/37962 [46:10:39<41:42:04,  8.33s/it]
 53%|█████▎    | 19936/37962 [46:10:47<41:49:21,  8.35s/it]
 53%|█████▎    | 19937/37962 [46:10:55<41:43:33,  8.33s/it]
 53%|█████▎    | 19938/37962 [46:11:04<41:49:29,  8.35s/it]
 53%|█████▎    | 19939/37962 [46:11:12<41:49:11,  8.35s/it]
 53%|█████▎    | 19940/37962 [46:11:20<41:40:24,  8.32s/it]
                                                           
{'loss': 1.3097, 'grad_norm': 7.802581787109375, 'learning_rate': 9.667755168310222e-06, 'epoch': 0.53}

 53%|█████▎    | 19940/37962 [46:11:20<41:40:24,  8.32s/it]
 53%|█████▎    | 19941/37962 [46:11:28<41:32:01,  8.30s/it]
 53%|█████▎    | 19942/37962 [46:11:37<41:27:27,  8.28s/it]
 53%|█████▎    | 19943/37962 [46:11:45<41:05:54,  8.21s/it]
 53%|█████▎    | 19944/37962 [46:11:53<41:21:32,  8.26s/it]
 53%|█████▎    | 19945/37962 [46:12:01<40:53:09,  8.17s/it]
 53%|█████▎    | 19946/37962 [46:12:09<41:05:24,  8.21s/it]
 53%|█████▎    | 19947/37962 [46:12:18<41:01:53,  8.20s/it]
 53%|█████▎    | 19948/37962 [46:12:26<41:03:54,  8.21s/it]
 53%|█████▎    | 19949/37962 [46:12:34<41:12:32,  8.24s/it]
 53%|█████▎    | 19950/37962 [46:12:42<41:14:32,  8.24s/it]
                                                           
{'loss': 1.4422, 'grad_norm': 5.681435585021973, 'learning_rate': 9.659228396131675e-06, 'epoch': 0.53}

 53%|█████▎    | 19950/37962 [46:12:42<41:14:32,  8.24s/it]
 53%|█████▎    | 19951/37962 [46:12:51<41:20:53,  8.26s/it]
 53%|█████▎    | 19952/37962 [46:12:59<41:30:28,  8.30s/it]
 53%|█████▎    | 19953/37962 [46:13:07<41:14:19,  8.24s/it]
 53%|█████▎    | 19954/37962 [46:13:15<41:18:09,  8.26s/it]
 53%|█████▎    | 19955/37962 [46:13:24<41:21:08,  8.27s/it]
 53%|█████▎    | 19956/37962 [46:13:32<41:08:09,  8.22s/it]
 53%|█████▎    | 19957/37962 [46:13:40<41:27:48,  8.29s/it]
 53%|█████▎    | 19958/37962 [46:13:49<41:33:51,  8.31s/it]
 53%|█████▎    | 19959/37962 [46:13:57<41:21:43,  8.27s/it]
 53%|█████▎    | 19960/37962 [46:14:13<53:04:47, 10.61s/it]
                                                           
{'loss': 1.2761, 'grad_norm': 4.71826696395874, 'learning_rate': 9.650701871994874e-06, 'epoch': 0.53}

 53%|█████▎    | 19960/37962 [46:14:13<53:04:47, 10.61s/it]
 53%|█████▎    | 19961/37962 [46:14:21<49:49:02,  9.96s/it]
 53%|█████▎    | 19962/37962 [46:14:30<47:03:30,  9.41s/it]
 53%|█████▎    | 19963/37962 [46:14:38<45:43:33,  9.15s/it]
 53%|█████▎    | 19964/37962 [46:14:46<44:20:10,  8.87s/it]
 53%|█████▎    | 19965/37962 [46:14:55<43:28:54,  8.70s/it]
 53%|█████▎    | 19966/37962 [46:15:03<43:11:11,  8.64s/it]
 53%|█████▎    | 19967/37962 [46:15:11<42:35:21,  8.52s/it]
 53%|█████▎    | 19968/37962 [46:15:20<42:06:17,  8.42s/it]
 53%|█████▎    | 19969/37962 [46:15:28<41:54:17,  8.38s/it]
 53%|█████▎    | 19970/37962 [46:15:36<41:34:12,  8.32s/it]
                                                           
{'loss': 1.2943, 'grad_norm': 24.492860794067383, 'learning_rate': 9.64217560210613e-06, 'epoch': 0.53}

 53%|█████▎    | 19970/37962 [46:15:36<41:34:12,  8.32s/it]
 53%|█████▎    | 19971/37962 [46:15:44<41:31:33,  8.31s/it]
 53%|█████▎    | 19972/37962 [46:15:53<41:40:02,  8.34s/it]
 53%|█████▎    | 19973/37962 [46:16:01<41:39:48,  8.34s/it]
 53%|█████▎    | 19974/37962 [46:16:09<41:27:39,  8.30s/it]
 53%|█████▎    | 19975/37962 [46:16:17<41:21:26,  8.28s/it]
 53%|█████▎    | 19976/37962 [46:16:26<41:18:19,  8.27s/it]
 53%|█████▎    | 19977/37962 [46:16:34<41:30:58,  8.31s/it]
 53%|█████▎    | 19978/37962 [46:16:42<41:38:38,  8.34s/it]
 53%|█████▎    | 19979/37962 [46:16:51<41:30:07,  8.31s/it]
 53%|█████▎    | 19980/37962 [46:16:59<41:33:54,  8.32s/it]
                                                           
{'loss': 1.424, 'grad_norm': 13.512646675109863, 'learning_rate': 9.633649592671568e-06, 'epoch': 0.53}

 53%|█████▎    | 19980/37962 [46:16:59<41:33:54,  8.32s/it]
 53%|█████▎    | 19981/37962 [46:17:07<41:40:18,  8.34s/it]
 53%|█████▎    | 19982/37962 [46:17:16<41:30:34,  8.31s/it]
 53%|█████▎    | 19983/37962 [46:17:24<41:21:04,  8.28s/it]
 53%|█████▎    | 19984/37962 [46:17:32<41:31:16,  8.31s/it]
 53%|█████▎    | 19985/37962 [46:17:41<41:41:48,  8.35s/it]
 53%|█████▎    | 19986/37962 [46:17:49<41:33:39,  8.32s/it]
 53%|█████▎    | 19987/37962 [46:17:57<41:26:53,  8.30s/it]
 53%|█████▎    | 19988/37962 [46:18:05<41:13:41,  8.26s/it]
 53%|█████▎    | 19989/37962 [46:18:14<41:26:18,  8.30s/it]
 53%|█████▎    | 19990/37962 [46:18:22<41:21:28,  8.28s/it]
                                                           
{'loss': 1.4138, 'grad_norm': 8.41482162475586, 'learning_rate': 9.625123849897121e-06, 'epoch': 0.53}

 53%|█████▎    | 19990/37962 [46:18:22<41:21:28,  8.28s/it]
 53%|█████▎    | 19991/37962 [46:18:30<41:25:30,  8.30s/it]
 53%|█████▎    | 19992/37962 [46:18:39<41:31:05,  8.32s/it]
 53%|█████▎    | 19993/37962 [46:18:47<41:20:44,  8.28s/it]
 53%|█████▎    | 19994/37962 [46:18:55<40:53:37,  8.19s/it]
 53%|█████▎    | 19995/37962 [46:19:03<40:26:09,  8.10s/it]
 53%|█████▎    | 19996/37962 [46:19:11<40:10:14,  8.05s/it]
 53%|█████▎    | 19997/37962 [46:19:19<40:37:10,  8.14s/it]
 53%|█████▎    | 19998/37962 [46:19:27<40:17:43,  8.08s/it]
 53%|█████▎    | 19999/37962 [46:19:35<40:41:26,  8.15s/it]
 53%|█████▎    | 20000/37962 [46:19:44<40:46:59,  8.17s/it]
                                                           
{'loss': 1.2497, 'grad_norm': 5.459377765655518, 'learning_rate': 9.616598379988526e-06, 'epoch': 0.53}

 53%|█████▎    | 20000/37962 [46:19:44<40:46:59,  8.17s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 53%|█████▎    | 20001/37962 [46:20:34<104:07:38, 20.87s/it]
 53%|█████▎    | 20002/37962 [46:20:42<85:21:59, 17.11s/it] 
 53%|█████▎    | 20003/37962 [46:20:51<72:04:54, 14.45s/it]
 53%|█████▎    | 20004/37962 [46:20:59<62:18:27, 12.49s/it]
 53%|█████▎    | 20005/37962 [46:21:07<55:34:04, 11.14s/it]
 53%|█████▎    | 20006/37962 [46:21:15<51:09:10, 10.26s/it]
 53%|█████▎    | 20007/37962 [46:21:23<48:30:07,  9.72s/it]
 53%|█████▎    | 20008/37962 [46:21:31<46:02:45,  9.23s/it]
 53%|█████▎    | 20009/37962 [46:21:40<44:37:55,  8.95s/it]
 53%|█████▎    | 20010/37962 [46:21:48<43:02:27,  8.63s/it]
                                                           
{'loss': 1.2553, 'grad_norm': 11.476192474365234, 'learning_rate': 9.60807318915133e-06, 'epoch': 0.53}

 53%|█████▎    | 20010/37962 [46:21:48<43:02:27,  8.63s/it]
 53%|█████▎    | 20011/37962 [46:21:56<42:51:59,  8.60s/it]
 53%|█████▎    | 20012/37962 [46:22:04<42:34:30,  8.54s/it]
 53%|█████▎    | 20013/37962 [46:22:13<42:13:18,  8.47s/it]
 53%|█████▎    | 20014/37962 [46:22:21<41:51:40,  8.40s/it]
 53%|█████▎    | 20015/37962 [46:22:29<41:57:16,  8.42s/it]
 53%|█████▎    | 20016/37962 [46:22:38<41:40:57,  8.36s/it]
 53%|█████▎    | 20017/37962 [46:22:46<41:44:08,  8.37s/it]
 53%|█████▎    | 20018/37962 [46:22:54<41:09:38,  8.26s/it]
 53%|█████▎    | 20019/37962 [46:23:02<41:13:00,  8.27s/it]
 53%|█████▎    | 20020/37962 [46:23:11<41:10:25,  8.26s/it]
                                                           
{'loss': 1.3957, 'grad_norm': 17.393442153930664, 'learning_rate': 9.599548283590869e-06, 'epoch': 0.53}

 53%|█████▎    | 20020/37962 [46:23:11<41:10:25,  8.26s/it]
 53%|█████▎    | 20021/37962 [46:23:19<41:19:28,  8.29s/it]
 53%|█████▎    | 20022/37962 [46:23:27<41:16:30,  8.28s/it]
 53%|█████▎    | 20023/37962 [46:23:36<41:20:23,  8.30s/it]
 53%|█████▎    | 20024/37962 [46:23:44<41:24:49,  8.31s/it]
 53%|█████▎    | 20025/37962 [46:23:52<41:30:04,  8.33s/it]
 53%|█████▎    | 20026/37962 [46:24:00<40:45:45,  8.18s/it]
 53%|█████▎    | 20027/37962 [46:24:08<40:38:59,  8.16s/it]
 53%|█████▎    | 20028/37962 [46:24:17<41:13:53,  8.28s/it]
 53%|█████▎    | 20029/37962 [46:24:25<41:08:24,  8.26s/it]
 53%|█████▎    | 20030/37962 [46:24:33<40:40:33,  8.17s/it]
                                                           
{'loss': 1.1821, 'grad_norm': 8.594742774963379, 'learning_rate': 9.59102366951228e-06, 'epoch': 0.53}

 53%|█████▎    | 20030/37962 [46:24:33<40:40:33,  8.17s/it]
 53%|█████▎    | 20031/37962 [46:24:41<40:46:40,  8.19s/it]
 53%|█████▎    | 20032/37962 [46:24:50<41:06:51,  8.25s/it]
 53%|█████▎    | 20033/37962 [46:24:58<40:57:25,  8.22s/it]
 53%|█████▎    | 20034/37962 [46:25:06<41:21:30,  8.30s/it]
 53%|█████▎    | 20035/37962 [46:25:14<41:12:09,  8.27s/it]
 53%|█████▎    | 20036/37962 [46:25:23<41:17:49,  8.29s/it]
 53%|█████▎    | 20037/37962 [46:25:31<41:23:37,  8.31s/it]
 53%|█████▎    | 20038/37962 [46:25:39<41:17:22,  8.29s/it]
 53%|█████▎    | 20039/37962 [46:25:48<41:17:20,  8.29s/it]
 53%|█████▎    | 20040/37962 [46:25:56<41:25:15,  8.32s/it]
                                                           
{'loss': 1.431, 'grad_norm': 10.096039772033691, 'learning_rate': 9.582499353120472e-06, 'epoch': 0.53}

 53%|█████▎    | 20040/37962 [46:25:56<41:25:15,  8.32s/it]
 53%|█████▎    | 20041/37962 [46:26:04<41:14:44,  8.29s/it]
 53%|█████▎    | 20042/37962 [46:26:12<40:49:30,  8.20s/it]
 53%|█████▎    | 20043/37962 [46:26:20<40:47:34,  8.20s/it]
 53%|█████▎    | 20044/37962 [46:26:29<40:52:07,  8.21s/it]
 53%|█████▎    | 20045/37962 [46:26:37<41:00:22,  8.24s/it]
 53%|█████▎    | 20046/37962 [46:26:45<40:52:34,  8.21s/it]
 53%|█████▎    | 20047/37962 [46:26:53<40:42:21,  8.18s/it]
 53%|█████▎    | 20048/37962 [46:27:01<40:47:49,  8.20s/it]
 53%|█████▎    | 20049/37962 [46:27:10<41:12:41,  8.28s/it]
 53%|█████▎    | 20050/37962 [46:27:18<40:37:52,  8.17s/it]
                                                           
{'loss': 1.4164, 'grad_norm': 6.465191841125488, 'learning_rate': 9.573975340620153e-06, 'epoch': 0.53}

 53%|█████▎    | 20050/37962 [46:27:18<40:37:52,  8.17s/it]
 53%|█████▎    | 20051/37962 [46:27:26<40:49:34,  8.21s/it]
 53%|█████▎    | 20052/37962 [46:27:35<41:17:19,  8.30s/it]
 53%|█████▎    | 20053/37962 [46:27:43<41:01:59,  8.25s/it]
 53%|█████▎    | 20054/37962 [46:27:51<41:12:51,  8.29s/it]
 53%|█████▎    | 20055/37962 [46:27:59<41:04:11,  8.26s/it]
 53%|█████▎    | 20056/37962 [46:28:08<41:22:15,  8.32s/it]
 53%|█████▎    | 20057/37962 [46:28:16<41:38:11,  8.37s/it]
 53%|█████▎    | 20058/37962 [46:28:25<41:30:30,  8.35s/it]
 53%|█████▎    | 20059/37962 [46:28:33<41:12:42,  8.29s/it]
 53%|█████▎    | 20060/37962 [46:28:41<41:17:33,  8.30s/it]
                                                           
{'loss': 1.3888, 'grad_norm': 6.256454944610596, 'learning_rate': 9.565451638215803e-06, 'epoch': 0.53}

 53%|█████▎    | 20060/37962 [46:28:41<41:17:33,  8.30s/it]
 53%|█████▎    | 20061/37962 [46:28:49<41:12:41,  8.29s/it]
 53%|█████▎    | 20062/37962 [46:28:58<41:05:11,  8.26s/it]
 53%|█████▎    | 20063/37962 [46:29:06<41:03:22,  8.26s/it]
 53%|█████▎    | 20064/37962 [46:29:14<41:36:53,  8.37s/it]
 53%|█████▎    | 20065/37962 [46:29:23<41:31:49,  8.35s/it]
 53%|█████▎    | 20066/37962 [46:29:31<41:31:41,  8.35s/it]
 53%|█████▎    | 20067/37962 [46:29:40<41:42:52,  8.39s/it]
 53%|█████▎    | 20068/37962 [46:29:48<41:27:07,  8.34s/it]
 53%|█████▎    | 20069/37962 [46:29:56<41:16:35,  8.30s/it]
 53%|█████▎    | 20070/37962 [46:30:04<41:05:23,  8.27s/it]
                                                           
{'loss': 1.4147, 'grad_norm': 3.5726234912872314, 'learning_rate': 9.556928252111679e-06, 'epoch': 0.53}

 53%|█████▎    | 20070/37962 [46:30:04<41:05:23,  8.27s/it]
 53%|█████▎    | 20071/37962 [46:30:12<41:03:22,  8.26s/it]
 53%|█████▎    | 20072/37962 [46:30:21<41:18:13,  8.31s/it]
 53%|█████▎    | 20073/37962 [46:30:29<41:14:31,  8.30s/it]
 53%|█████▎    | 20074/37962 [46:30:37<40:55:57,  8.24s/it]
 53%|█████▎    | 20075/37962 [46:30:46<41:04:47,  8.27s/it]
 53%|█████▎    | 20076/37962 [46:30:54<41:13:14,  8.30s/it]
 53%|█████▎    | 20077/37962 [46:31:02<41:27:31,  8.35s/it]
 53%|█████▎    | 20078/37962 [46:31:11<41:18:06,  8.31s/it]
 53%|█████▎    | 20079/37962 [46:31:19<41:13:36,  8.30s/it]
 53%|█████▎    | 20080/37962 [46:31:27<41:09:06,  8.28s/it]
                                                           
{'loss': 1.3382, 'grad_norm': 12.142322540283203, 'learning_rate': 9.548405188511806e-06, 'epoch': 0.53}

 53%|█████▎    | 20080/37962 [46:31:27<41:09:06,  8.28s/it]
 53%|█████▎    | 20081/37962 [46:31:35<40:33:49,  8.17s/it]
 53%|█████▎    | 20082/37962 [46:31:43<40:24:23,  8.14s/it]
 53%|█████▎    | 20083/37962 [46:31:52<40:53:16,  8.23s/it]
 53%|█████▎    | 20084/37962 [46:32:00<41:07:26,  8.28s/it]
 53%|█████▎    | 20085/37962 [46:32:08<41:16:13,  8.31s/it]
 53%|█████▎    | 20086/37962 [46:32:16<41:01:50,  8.26s/it]
 53%|█████▎    | 20087/37962 [46:32:25<41:03:32,  8.27s/it]
 53%|█████▎    | 20088/37962 [46:32:33<41:22:20,  8.33s/it]
 53%|█████▎    | 20089/37962 [46:32:42<41:30:39,  8.36s/it]
 53%|█████▎    | 20090/37962 [46:32:50<41:28:41,  8.36s/it]
                                                           
{'loss': 1.3794, 'grad_norm': 9.740056037902832, 'learning_rate': 9.53988245361997e-06, 'epoch': 0.53}

 53%|█████▎    | 20090/37962 [46:32:50<41:28:41,  8.36s/it]
 53%|█████▎    | 20091/37962 [46:32:58<41:27:58,  8.35s/it]
 53%|█████▎    | 20092/37962 [46:33:07<41:22:33,  8.34s/it]
 53%|█████▎    | 20093/37962 [46:33:15<41:01:02,  8.26s/it]
 53%|█████▎    | 20094/37962 [46:33:23<41:10:20,  8.30s/it]
 53%|█████▎    | 20095/37962 [46:33:32<41:23:06,  8.34s/it]
 53%|█████▎    | 20096/37962 [46:33:40<41:09:47,  8.29s/it]
 53%|█████▎    | 20097/37962 [46:33:48<41:04:31,  8.28s/it]
 53%|█████▎    | 20098/37962 [46:33:56<40:48:26,  8.22s/it]
 53%|█████▎    | 20099/37962 [46:34:04<40:57:39,  8.26s/it]
 53%|█████▎    | 20100/37962 [46:34:13<40:55:54,  8.25s/it]
                                                           
{'loss': 1.3674, 'grad_norm': 5.354048728942871, 'learning_rate': 9.531360053639724e-06, 'epoch': 0.53}

 53%|█████▎    | 20100/37962 [46:34:13<40:55:54,  8.25s/it]
 53%|█████▎    | 20101/37962 [46:34:21<41:03:26,  8.28s/it]
 53%|█████▎    | 20102/37962 [46:34:29<40:33:26,  8.18s/it]
 53%|█████▎    | 20103/37962 [46:34:37<40:46:47,  8.22s/it]
 53%|█████▎    | 20104/37962 [46:34:46<40:49:50,  8.23s/it]
 53%|█████▎    | 20105/37962 [46:34:54<40:27:19,  8.16s/it]
 53%|█████▎    | 20106/37962 [46:35:02<41:00:17,  8.27s/it]
 53%|█████▎    | 20107/37962 [46:35:10<41:13:57,  8.31s/it]
 53%|█████▎    | 20108/37962 [46:35:19<41:19:48,  8.33s/it]
 53%|█████▎    | 20109/37962 [46:35:27<41:08:59,  8.30s/it]
 53%|█████▎    | 20110/37962 [46:35:35<41:06:00,  8.29s/it]
                                                           
{'loss': 1.2795, 'grad_norm': 4.187978744506836, 'learning_rate': 9.522837994774376e-06, 'epoch': 0.53}

 53%|█████▎    | 20110/37962 [46:35:35<41:06:00,  8.29s/it]
 53%|█████▎    | 20111/37962 [46:35:44<41:01:11,  8.27s/it]
 53%|█████▎    | 20112/37962 [46:35:52<41:02:44,  8.28s/it]
 53%|█████▎    | 20113/37962 [46:36:00<40:56:27,  8.26s/it]
 53%|█████▎    | 20114/37962 [46:36:09<41:14:01,  8.32s/it]
 53%|█████▎    | 20115/37962 [46:36:17<41:14:17,  8.32s/it]
 53%|█████▎    | 20116/37962 [46:36:25<41:19:57,  8.34s/it]
 53%|█████▎    | 20117/37962 [46:36:33<41:12:16,  8.31s/it]
 53%|█████▎    | 20118/37962 [46:36:42<41:18:32,  8.33s/it]
 53%|█████▎    | 20119/37962 [46:36:50<41:06:39,  8.29s/it]
 53%|█████▎    | 20120/37962 [46:36:58<40:36:17,  8.19s/it]
                                                           
{'loss': 1.3634, 'grad_norm': 8.987088203430176, 'learning_rate': 9.514316283226987e-06, 'epoch': 0.53}

 53%|█████▎    | 20120/37962 [46:36:58<40:36:17,  8.19s/it]
 53%|█████▎    | 20121/37962 [46:37:06<40:55:47,  8.26s/it]
 53%|█████▎    | 20122/37962 [46:37:15<40:51:28,  8.24s/it]
 53%|█████▎    | 20123/37962 [46:37:23<40:53:02,  8.25s/it]
 53%|█████▎    | 20124/37962 [46:37:31<40:58:01,  8.27s/it]
 53%|█████▎    | 20125/37962 [46:37:40<41:03:48,  8.29s/it]
 53%|█████▎    | 20126/37962 [46:37:48<40:56:30,  8.26s/it]
 53%|█████▎    | 20127/37962 [46:37:56<41:09:00,  8.31s/it]
 53%|█████▎    | 20128/37962 [46:38:04<41:00:37,  8.28s/it]
 53%|█████▎    | 20129/37962 [46:38:13<41:12:14,  8.32s/it]
 53%|█████▎    | 20130/37962 [46:38:21<41:24:58,  8.36s/it]
                                                           
{'loss': 1.5045, 'grad_norm': 15.204462051391602, 'learning_rate': 9.505794925200363e-06, 'epoch': 0.53}

 53%|█████▎    | 20130/37962 [46:38:21<41:24:58,  8.36s/it]
 53%|█████▎    | 20131/37962 [46:38:29<41:12:59,  8.32s/it]
 53%|█████▎    | 20132/37962 [46:38:38<41:14:58,  8.33s/it]
 53%|█████▎    | 20133/37962 [46:38:46<41:27:46,  8.37s/it]
 53%|█████▎    | 20134/37962 [46:38:55<41:19:52,  8.35s/it]
 53%|█████▎    | 20135/37962 [46:39:03<41:14:09,  8.33s/it]
 53%|█████▎    | 20136/37962 [46:39:11<41:05:12,  8.30s/it]
 53%|█████▎    | 20137/37962 [46:39:19<41:02:52,  8.29s/it]
 53%|█████▎    | 20138/37962 [46:39:28<41:04:20,  8.30s/it]
 53%|█████▎    | 20139/37962 [46:39:36<41:06:56,  8.30s/it]
 53%|█████▎    | 20140/37962 [46:39:44<40:45:29,  8.23s/it]
                                                           
{'loss': 1.2509, 'grad_norm': 5.814431667327881, 'learning_rate': 9.497273926897052e-06, 'epoch': 0.53}

 53%|█████▎    | 20140/37962 [46:39:44<40:45:29,  8.23s/it]
 53%|█████▎    | 20141/37962 [46:39:52<40:49:35,  8.25s/it]
 53%|█████▎    | 20142/37962 [46:40:00<40:34:27,  8.20s/it]
 53%|█████▎    | 20143/37962 [46:40:09<40:39:19,  8.21s/it]
 53%|█████▎    | 20144/37962 [46:40:17<40:42:40,  8.23s/it]
 53%|█████▎    | 20145/37962 [46:40:25<40:33:11,  8.19s/it]
 53%|█████▎    | 20146/37962 [46:40:33<40:51:28,  8.26s/it]
 53%|█████▎    | 20147/37962 [46:40:42<41:03:36,  8.30s/it]
 53%|█████▎    | 20148/37962 [46:40:50<40:28:42,  8.18s/it]
 53%|█████▎    | 20149/37962 [46:40:58<40:27:57,  8.18s/it]
 53%|█████▎    | 20150/37962 [46:41:06<40:44:10,  8.23s/it]
                                                           
{'loss': 1.3408, 'grad_norm': 4.966856479644775, 'learning_rate': 9.488753294519339e-06, 'epoch': 0.53}

 53%|█████▎    | 20150/37962 [46:41:06<40:44:10,  8.23s/it]
 53%|█████▎    | 20151/37962 [46:41:14<40:14:25,  8.13s/it]
 53%|█████▎    | 20152/37962 [46:41:23<40:40:07,  8.22s/it]
 53%|█████▎    | 20153/37962 [46:41:31<41:09:25,  8.32s/it]
 53%|█████▎    | 20154/37962 [46:41:39<41:05:42,  8.31s/it]
 53%|█████▎    | 20155/37962 [46:41:48<40:58:04,  8.28s/it]
 53%|█████▎    | 20156/37962 [46:41:56<40:28:29,  8.18s/it]
 53%|█████▎    | 20157/37962 [46:42:04<40:19:51,  8.15s/it]
 53%|█████▎    | 20158/37962 [46:42:12<40:05:14,  8.11s/it]
 53%|█████▎    | 20159/37962 [46:42:20<40:05:02,  8.11s/it]
 53%|█████▎    | 20160/37962 [46:42:28<40:19:57,  8.16s/it]
                                                           
{'loss': 1.3895, 'grad_norm': 6.494147777557373, 'learning_rate': 9.480233034269245e-06, 'epoch': 0.53}

 53%|█████▎    | 20160/37962 [46:42:28<40:19:57,  8.16s/it]
 53%|█████▎    | 20161/37962 [46:42:36<40:34:32,  8.21s/it]
 53%|█████▎    | 20162/37962 [46:42:45<40:48:29,  8.25s/it]
 53%|█████▎    | 20163/37962 [46:42:53<40:32:28,  8.20s/it]
 53%|█████▎    | 20164/37962 [46:43:01<40:39:56,  8.23s/it]
 53%|█████▎    | 20165/37962 [46:43:09<40:45:32,  8.24s/it]
 53%|█████▎    | 20166/37962 [46:43:18<40:45:01,  8.24s/it]
 53%|█████▎    | 20167/37962 [46:43:26<40:46:27,  8.25s/it]
 53%|█████▎    | 20168/37962 [46:43:34<40:49:51,  8.26s/it]
 53%|█████▎    | 20169/37962 [46:43:42<40:46:18,  8.25s/it]
 53%|█████▎    | 20170/37962 [46:43:51<40:49:13,  8.26s/it]
                                                           
{'loss': 1.321, 'grad_norm': 15.840327262878418, 'learning_rate': 9.471713152348522e-06, 'epoch': 0.53}

 53%|█████▎    | 20170/37962 [46:43:51<40:49:13,  8.26s/it]
 53%|█████▎    | 20171/37962 [46:43:59<40:51:24,  8.27s/it]
 53%|█████▎    | 20172/37962 [46:44:07<40:54:23,  8.28s/it]
 53%|█████▎    | 20173/37962 [46:44:16<40:52:24,  8.27s/it]
 53%|█████▎    | 20174/37962 [46:44:24<40:48:44,  8.26s/it]
 53%|█████▎    | 20175/37962 [46:44:32<41:04:48,  8.31s/it]
 53%|█████▎    | 20176/37962 [46:44:40<40:58:24,  8.29s/it]
 53%|█████▎    | 20177/37962 [46:44:49<41:05:08,  8.32s/it]
 53%|█████▎    | 20178/37962 [46:44:57<40:58:33,  8.29s/it]
 53%|█████▎    | 20179/37962 [46:45:05<41:01:48,  8.31s/it]
 53%|█████▎    | 20180/37962 [46:45:13<40:35:30,  8.22s/it]
                                                           
{'loss': 1.2217, 'grad_norm': 5.250042915344238, 'learning_rate': 9.463193654958642e-06, 'epoch': 0.53}

 53%|█████▎    | 20180/37962 [46:45:13<40:35:30,  8.22s/it]
 53%|█████▎    | 20181/37962 [46:45:22<40:57:43,  8.29s/it]
 53%|█████▎    | 20182/37962 [46:45:30<40:43:12,  8.24s/it]
 53%|█████▎    | 20183/37962 [46:45:38<40:23:45,  8.18s/it]
 53%|█████▎    | 20184/37962 [46:45:46<40:37:14,  8.23s/it]
 53%|█████▎    | 20185/37962 [46:45:55<40:41:19,  8.24s/it]
 53%|█████▎    | 20186/37962 [46:46:03<40:49:43,  8.27s/it]
 53%|█████▎    | 20187/37962 [46:46:11<41:06:22,  8.33s/it]
 53%|█████▎    | 20188/37962 [46:46:20<40:54:48,  8.29s/it]
 53%|█████▎    | 20189/37962 [46:46:28<40:48:31,  8.27s/it]
 53%|█████▎    | 20190/37962 [46:46:36<40:59:37,  8.30s/it]
                                                           
{'loss': 1.2964, 'grad_norm': 612.7410888671875, 'learning_rate': 9.454674548300805e-06, 'epoch': 0.53}

 53%|█████▎    | 20190/37962 [46:46:36<40:59:37,  8.30s/it]
 53%|█████▎    | 20191/37962 [46:46:44<40:37:19,  8.23s/it]
 53%|█████▎    | 20192/37962 [46:46:53<40:38:51,  8.23s/it]
 53%|█████▎    | 20193/37962 [46:47:01<40:37:20,  8.23s/it]
 53%|█████▎    | 20194/37962 [46:47:08<39:51:11,  8.07s/it]
 53%|█████▎    | 20195/37962 [46:47:17<40:04:36,  8.12s/it]
 53%|█████▎    | 20196/37962 [46:47:25<40:12:00,  8.15s/it]
 53%|█████▎    | 20197/37962 [46:47:33<39:54:44,  8.09s/it]
 53%|█████▎    | 20198/37962 [46:47:41<40:10:42,  8.14s/it]
 53%|█████▎    | 20199/37962 [46:47:49<40:15:11,  8.16s/it]
 53%|█████▎    | 20200/37962 [46:47:57<40:12:15,  8.15s/it]
                                                           
{'loss': 1.3933, 'grad_norm': 16.88507080078125, 'learning_rate': 9.446155838575914e-06, 'epoch': 0.53}

 53%|█████▎    | 20200/37962 [46:47:57<40:12:15,  8.15s/it]
 53%|█████▎    | 20201/37962 [46:48:06<40:33:36,  8.22s/it]
 53%|█████▎    | 20202/37962 [46:48:14<40:27:01,  8.20s/it]
 53%|█████▎    | 20203/37962 [46:48:22<40:26:42,  8.20s/it]
 53%|█████▎    | 20204/37962 [46:48:30<40:24:53,  8.19s/it]
 53%|█████▎    | 20205/37962 [46:48:39<40:58:41,  8.31s/it]
 53%|█████▎    | 20206/37962 [46:48:47<40:39:13,  8.24s/it]
 53%|█████▎    | 20207/37962 [46:48:55<40:44:45,  8.26s/it]
 53%|█████▎    | 20208/37962 [46:49:04<40:44:35,  8.26s/it]
 53%|█████▎    | 20209/37962 [46:49:12<40:52:18,  8.29s/it]
 53%|█████▎    | 20210/37962 [46:49:20<40:55:14,  8.30s/it]
                                                           
{'loss': 1.2719, 'grad_norm': 10.91889762878418, 'learning_rate': 9.437637531984593e-06, 'epoch': 0.53}

 53%|█████▎    | 20210/37962 [46:49:20<40:55:14,  8.30s/it]
 53%|█████▎    | 20211/37962 [46:49:28<40:28:56,  8.21s/it]
 53%|█████▎    | 20212/37962 [46:49:37<40:42:59,  8.26s/it]
 53%|█████▎    | 20213/37962 [46:49:45<41:13:35,  8.36s/it]
 53%|█████▎    | 20214/37962 [46:49:54<41:15:04,  8.37s/it]
 53%|█████▎    | 20215/37962 [46:50:02<41:03:59,  8.33s/it]
 53%|█████▎    | 20216/37962 [46:50:10<41:08:32,  8.35s/it]
 53%|█████▎    | 20217/37962 [46:50:18<40:55:44,  8.30s/it]
 53%|█████▎    | 20218/37962 [46:50:26<40:28:46,  8.21s/it]
 53%|█████▎    | 20219/37962 [46:50:34<39:56:25,  8.10s/it]
 53%|█████▎    | 20220/37962 [46:50:43<40:10:58,  8.15s/it]
                                                           
{'loss': 1.3065, 'grad_norm': 10.554426193237305, 'learning_rate': 9.429119634727171e-06, 'epoch': 0.53}

 53%|█████▎    | 20220/37962 [46:50:43<40:10:58,  8.15s/it]
 53%|█████▎    | 20221/37962 [46:50:51<40:43:51,  8.27s/it]
 53%|█████▎    | 20222/37962 [46:50:59<40:39:48,  8.25s/it]
 53%|█████▎    | 20223/37962 [46:51:08<41:04:03,  8.33s/it]
 53%|█████▎    | 20224/37962 [46:51:16<41:08:49,  8.35s/it]
 53%|█████▎    | 20225/37962 [46:51:25<41:00:02,  8.32s/it]
 53%|█████▎    | 20226/37962 [46:51:33<41:06:56,  8.35s/it]
 53%|█████▎    | 20227/37962 [46:51:41<40:45:52,  8.27s/it]
 53%|█████▎    | 20228/37962 [46:51:49<40:45:47,  8.27s/it]
 53%|█████▎    | 20229/37962 [46:51:58<40:43:37,  8.27s/it]
 53%|█████▎    | 20230/37962 [46:52:06<40:48:42,  8.29s/it]
                                                           
{'loss': 1.3798, 'grad_norm': 116.7707290649414, 'learning_rate': 9.420602153003676e-06, 'epoch': 0.53}

 53%|█████▎    | 20230/37962 [46:52:06<40:48:42,  8.29s/it]
 53%|█████▎    | 20231/37962 [46:52:14<40:58:54,  8.32s/it]
 53%|█████▎    | 20232/37962 [46:52:22<40:39:32,  8.26s/it]
 53%|█████▎    | 20233/37962 [46:52:31<40:53:31,  8.30s/it]
 53%|█████▎    | 20234/37962 [46:52:39<40:41:15,  8.26s/it]
 53%|█████▎    | 20235/37962 [46:52:47<40:41:51,  8.26s/it]
 53%|█████▎    | 20236/37962 [46:52:56<40:45:30,  8.28s/it]
 53%|█████▎    | 20237/37962 [46:53:04<40:55:16,  8.31s/it]
 53%|█████▎    | 20238/37962 [46:53:12<40:59:02,  8.32s/it]
 53%|█████▎    | 20239/37962 [46:53:21<41:06:24,  8.35s/it]
 53%|█████▎    | 20240/37962 [46:53:29<41:01:33,  8.33s/it]
                                                           
{'loss': 1.2545, 'grad_norm': 47.651519775390625, 'learning_rate': 9.412085093013841e-06, 'epoch': 0.53}

 53%|█████▎    | 20240/37962 [46:53:29<41:01:33,  8.33s/it]
 53%|█████▎    | 20241/37962 [46:53:37<40:29:18,  8.23s/it]
 53%|█████▎    | 20242/37962 [46:53:45<40:31:43,  8.23s/it]
 53%|█████▎    | 20243/37962 [46:53:53<40:12:00,  8.17s/it]
 53%|█████▎    | 20244/37962 [46:54:02<40:24:14,  8.21s/it]
 53%|█████▎    | 20245/37962 [46:54:09<39:57:38,  8.12s/it]
 53%|█████▎    | 20246/37962 [46:54:18<39:56:40,  8.12s/it]
 53%|█████▎    | 20247/37962 [46:54:26<40:19:04,  8.19s/it]
 53%|█████▎    | 20248/37962 [46:54:34<40:07:57,  8.16s/it]
 53%|█████▎    | 20249/37962 [46:54:42<40:04:20,  8.14s/it]
 53%|█████▎    | 20250/37962 [46:54:50<40:14:51,  8.18s/it]
                                                           
{'loss': 1.3333, 'grad_norm': 9.32347297668457, 'learning_rate': 9.403568460957078e-06, 'epoch': 0.53}

 53%|█████▎    | 20250/37962 [46:54:50<40:14:51,  8.18s/it]
 53%|█████▎    | 20251/37962 [46:54:59<40:09:49,  8.16s/it]
 53%|█████▎    | 20252/37962 [46:55:07<40:18:05,  8.19s/it]
 53%|█████▎    | 20253/37962 [46:55:15<40:18:36,  8.19s/it]
 53%|█████▎    | 20254/37962 [46:55:23<40:32:12,  8.24s/it]
 53%|█████▎    | 20255/37962 [46:55:32<40:33:17,  8.25s/it]
 53%|█████▎    | 20256/37962 [46:55:40<40:49:49,  8.30s/it]
 53%|█████▎    | 20257/37962 [46:55:49<41:11:39,  8.38s/it]
 53%|█████▎    | 20258/37962 [46:55:57<40:40:24,  8.27s/it]
 53%|█████▎    | 20259/37962 [46:56:05<40:40:05,  8.27s/it]
 53%|█████▎    | 20260/37962 [46:56:13<40:52:48,  8.31s/it]
                                                           
{'loss': 1.4144, 'grad_norm': 9.226522445678711, 'learning_rate': 9.3950522630325e-06, 'epoch': 0.53}

 53%|█████▎    | 20260/37962 [46:56:13<40:52:48,  8.31s/it]
 53%|█████▎    | 20261/37962 [46:56:22<40:50:11,  8.31s/it]
 53%|█████▎    | 20262/37962 [46:56:30<40:39:14,  8.27s/it]
 53%|█████▎    | 20263/37962 [46:56:38<40:42:07,  8.28s/it]
 53%|█████▎    | 20264/37962 [46:56:46<40:44:11,  8.29s/it]
 53%|█████▎    | 20265/37962 [46:56:54<40:15:41,  8.19s/it]
 53%|█████▎    | 20266/37962 [46:57:03<40:22:36,  8.21s/it]
 53%|█████▎    | 20267/37962 [46:57:11<40:21:36,  8.21s/it]
 53%|█████▎    | 20268/37962 [46:57:19<40:23:28,  8.22s/it]
 53%|█████▎    | 20269/37962 [46:57:27<40:34:44,  8.26s/it]
 53%|█████▎    | 20270/37962 [46:57:36<40:43:37,  8.29s/it]
                                                           
{'loss': 1.2635, 'grad_norm': 20.721189498901367, 'learning_rate': 9.3865365054389e-06, 'epoch': 0.53}

 53%|█████▎    | 20270/37962 [46:57:36<40:43:37,  8.29s/it]
 53%|█████▎    | 20271/37962 [46:57:44<40:30:06,  8.24s/it]
 53%|█████▎    | 20272/37962 [46:57:52<40:43:37,  8.29s/it]
 53%|█████▎    | 20273/37962 [46:58:01<40:42:24,  8.28s/it]
 53%|█████▎    | 20274/37962 [46:58:09<40:29:42,  8.24s/it]
 53%|█████▎    | 20275/37962 [46:58:17<40:29:19,  8.24s/it]
 53%|█████▎    | 20276/37962 [46:58:25<40:16:27,  8.20s/it]
 53%|█████▎    | 20277/37962 [46:58:33<40:23:47,  8.22s/it]
 53%|█████▎    | 20278/37962 [46:58:42<40:24:50,  8.23s/it]
 53%|█████▎    | 20279/37962 [46:58:49<39:52:08,  8.12s/it]
 53%|█████▎    | 20280/37962 [46:58:58<40:11:56,  8.18s/it]
                                                           
{'loss': 1.3063, 'grad_norm': 19.534564971923828, 'learning_rate': 9.378021194374747e-06, 'epoch': 0.53}

 53%|█████▎    | 20280/37962 [46:58:58<40:11:56,  8.18s/it]
 53%|█████▎    | 20281/37962 [46:59:06<40:40:46,  8.28s/it]
 53%|█████▎    | 20282/37962 [46:59:14<39:59:21,  8.14s/it]
 53%|█████▎    | 20283/37962 [46:59:22<40:05:08,  8.16s/it]
 53%|█████▎    | 20284/37962 [46:59:31<40:13:41,  8.19s/it]
 53%|█████▎    | 20285/37962 [46:59:39<39:57:16,  8.14s/it]
 53%|█████▎    | 20286/37962 [46:59:47<40:22:30,  8.22s/it]
 53%|█████▎    | 20287/37962 [46:59:55<40:14:58,  8.20s/it]
 53%|█████▎    | 20288/37962 [47:00:03<40:00:05,  8.15s/it]
 53%|█████▎    | 20289/37962 [47:00:11<39:27:46,  8.04s/it]
 53%|█████▎    | 20290/37962 [47:00:19<39:46:37,  8.10s/it]
                                                           
{'loss': 1.3987, 'grad_norm': 80.9044189453125, 'learning_rate': 9.3695063360382e-06, 'epoch': 0.53}

 53%|█████▎    | 20290/37962 [47:00:19<39:46:37,  8.10s/it]
 53%|█████▎    | 20291/37962 [47:00:28<40:23:58,  8.23s/it]
 53%|█████▎    | 20292/37962 [47:00:36<40:15:18,  8.20s/it]
 53%|█████▎    | 20293/37962 [47:00:44<40:15:15,  8.20s/it]
 53%|█████▎    | 20294/37962 [47:00:52<40:21:25,  8.22s/it]
 53%|█████▎    | 20295/37962 [47:01:00<40:03:04,  8.16s/it]
 53%|█████▎    | 20296/37962 [47:01:08<39:46:16,  8.10s/it]
 53%|█████▎    | 20297/37962 [47:01:17<39:55:55,  8.14s/it]
 53%|█████▎    | 20298/37962 [47:01:25<40:02:02,  8.16s/it]
 53%|█████▎    | 20299/37962 [47:01:33<40:20:45,  8.22s/it]
 53%|█████▎    | 20300/37962 [47:01:41<40:25:57,  8.24s/it]
                                                           
{'loss': 1.3448, 'grad_norm': 3.3335559368133545, 'learning_rate': 9.360991936627064e-06, 'epoch': 0.53}

 53%|█████▎    | 20300/37962 [47:01:41<40:25:57,  8.24s/it]
 53%|█████▎    | 20301/37962 [47:01:50<40:26:56,  8.25s/it]
 53%|█████▎    | 20302/37962 [47:01:58<40:45:32,  8.31s/it]
 53%|█████▎    | 20303/37962 [47:02:07<40:55:51,  8.34s/it]
 53%|█████▎    | 20304/37962 [47:02:15<40:41:53,  8.30s/it]
 53%|█████▎    | 20305/37962 [47:02:23<40:34:40,  8.27s/it]
 53%|█████▎    | 20306/37962 [47:02:31<40:33:54,  8.27s/it]
 53%|█████▎    | 20307/37962 [47:02:40<40:48:55,  8.32s/it]
 53%|█████▎    | 20308/37962 [47:02:48<40:32:09,  8.27s/it]
 53%|█████▎    | 20309/37962 [47:02:56<40:42:20,  8.30s/it]
 54%|█████▎    | 20310/37962 [47:03:04<40:26:03,  8.25s/it]
                                                           
{'loss': 1.2929, 'grad_norm': 5.73553466796875, 'learning_rate': 9.352478002338823e-06, 'epoch': 0.53}

 54%|█████▎    | 20310/37962 [47:03:04<40:26:03,  8.25s/it]
 54%|█████▎    | 20311/37962 [47:03:12<40:14:45,  8.21s/it]
 54%|█████▎    | 20312/37962 [47:03:20<39:57:52,  8.15s/it]
 54%|█████▎    | 20313/37962 [47:03:28<39:44:51,  8.11s/it]
 54%|█████▎    | 20314/37962 [47:03:37<39:53:55,  8.14s/it]
 54%|█████▎    | 20315/37962 [47:03:45<40:05:51,  8.18s/it]
 54%|█████▎    | 20316/37962 [47:03:53<39:52:23,  8.13s/it]
 54%|█████▎    | 20317/37962 [47:04:01<39:26:36,  8.05s/it]
 54%|█████▎    | 20318/37962 [47:04:09<40:02:30,  8.17s/it]
 54%|█████▎    | 20319/37962 [47:04:17<40:10:53,  8.20s/it]
 54%|█████▎    | 20320/37962 [47:04:26<40:20:17,  8.23s/it]
                                                           
{'loss': 1.25, 'grad_norm': 24.02547264099121, 'learning_rate': 9.34396453937063e-06, 'epoch': 0.54}

 54%|█████▎    | 20320/37962 [47:04:26<40:20:17,  8.23s/it]
 54%|█████▎    | 20321/37962 [47:04:34<40:39:41,  8.30s/it]
 54%|█████▎    | 20322/37962 [47:04:42<40:35:17,  8.28s/it]
 54%|█████▎    | 20323/37962 [47:04:51<40:22:50,  8.24s/it]
 54%|█████▎    | 20324/37962 [47:04:59<40:26:42,  8.26s/it]
 54%|█████▎    | 20325/37962 [47:05:07<40:05:03,  8.18s/it]
 54%|█████▎    | 20326/37962 [47:05:15<39:47:29,  8.12s/it]
 54%|█████▎    | 20327/37962 [47:05:23<39:24:43,  8.05s/it]
 54%|█████▎    | 20328/37962 [47:05:31<39:49:39,  8.13s/it]
 54%|█████▎    | 20329/37962 [47:05:39<40:11:38,  8.21s/it]
 54%|█████▎    | 20330/37962 [47:05:48<40:31:38,  8.27s/it]
                                                           
{'loss': 1.3628, 'grad_norm': 8.770973205566406, 'learning_rate': 9.335451553919283e-06, 'epoch': 0.54}

 54%|█████▎    | 20330/37962 [47:05:48<40:31:38,  8.27s/it]
 54%|█████▎    | 20331/37962 [47:05:56<40:22:16,  8.24s/it]
 54%|█████▎    | 20332/37962 [47:06:05<41:06:13,  8.39s/it]
 54%|█████▎    | 20333/37962 [47:06:13<40:36:43,  8.29s/it]
 54%|█████▎    | 20334/37962 [47:06:21<40:34:27,  8.29s/it]
 54%|█████▎    | 20335/37962 [47:06:29<39:58:02,  8.16s/it]
 54%|█████▎    | 20336/37962 [47:06:37<40:06:55,  8.19s/it]
 54%|█████▎    | 20337/37962 [47:06:46<40:06:57,  8.19s/it]
 54%|█████▎    | 20338/37962 [47:06:53<39:42:56,  8.11s/it]
 54%|█████▎    | 20339/37962 [47:07:01<39:31:59,  8.08s/it]
 54%|█████▎    | 20340/37962 [47:07:10<40:07:56,  8.20s/it]
                                                           
{'loss': 1.3167, 'grad_norm': 6.259378433227539, 'learning_rate': 9.326939052181238e-06, 'epoch': 0.54}

 54%|█████▎    | 20340/37962 [47:07:10<40:07:56,  8.20s/it]
 54%|█████▎    | 20341/37962 [47:07:18<40:10:38,  8.21s/it]
 54%|█████▎    | 20342/37962 [47:07:26<39:55:24,  8.16s/it]
 54%|█████▎    | 20343/37962 [47:07:35<40:20:35,  8.24s/it]
 54%|█████▎    | 20344/37962 [47:07:43<40:13:03,  8.22s/it]
 54%|█████▎    | 20345/37962 [47:07:51<40:21:01,  8.25s/it]
 54%|█████▎    | 20346/37962 [47:07:59<40:29:02,  8.27s/it]
 54%|█████▎    | 20347/37962 [47:08:08<40:28:49,  8.27s/it]
 54%|█████▎    | 20348/37962 [47:08:16<40:22:54,  8.25s/it]
 54%|█████▎    | 20349/37962 [47:08:24<40:35:06,  8.30s/it]
 54%|█████▎    | 20350/37962 [47:08:33<40:41:11,  8.32s/it]
                                                           
{'loss': 1.4321, 'grad_norm': 13.662786483764648, 'learning_rate': 9.3184270403526e-06, 'epoch': 0.54}

 54%|█████▎    | 20350/37962 [47:08:33<40:41:11,  8.32s/it]
 54%|█████▎    | 20351/37962 [47:08:41<40:22:37,  8.25s/it]
 54%|█████▎    | 20352/37962 [47:08:49<40:06:29,  8.20s/it]
 54%|█████▎    | 20353/37962 [47:08:57<40:23:11,  8.26s/it]
 54%|█████▎    | 20354/37962 [47:09:06<40:25:28,  8.26s/it]
 54%|█████▎    | 20355/37962 [47:09:14<40:20:40,  8.25s/it]
 54%|█████▎    | 20356/37962 [47:09:22<40:20:41,  8.25s/it]
 54%|█████▎    | 20357/37962 [47:09:30<40:24:06,  8.26s/it]
 54%|█████▎    | 20358/37962 [47:09:39<40:32:35,  8.29s/it]
 54%|█████▎    | 20359/37962 [47:09:47<40:29:04,  8.28s/it]
 54%|█████▎    | 20360/37962 [47:09:55<40:19:59,  8.25s/it]
                                                           
{'loss': 1.3401, 'grad_norm': 34.336181640625, 'learning_rate': 9.309915524629109e-06, 'epoch': 0.54}

 54%|█████▎    | 20360/37962 [47:09:55<40:19:59,  8.25s/it]
 54%|█████▎    | 20361/37962 [47:10:03<40:14:28,  8.23s/it]
 54%|█████▎    | 20362/37962 [47:10:11<40:03:53,  8.20s/it]
 54%|█████▎    | 20363/37962 [47:10:19<39:57:49,  8.17s/it]
 54%|█████▎    | 20364/37962 [47:10:28<39:59:11,  8.18s/it]
 54%|█████▎    | 20365/37962 [47:10:36<40:10:33,  8.22s/it]
 54%|█████▎    | 20366/37962 [47:10:44<39:52:42,  8.16s/it]
 54%|█████▎    | 20367/37962 [47:10:52<39:41:29,  8.12s/it]
 54%|█████▎    | 20368/37962 [47:11:00<39:41:51,  8.12s/it]
 54%|█████▎    | 20369/37962 [47:11:08<39:41:30,  8.12s/it]
 54%|█████▎    | 20370/37962 [47:11:17<40:07:36,  8.21s/it]
                                                           
{'loss': 1.2822, 'grad_norm': 12.778273582458496, 'learning_rate': 9.301404511206152e-06, 'epoch': 0.54}

 54%|█████▎    | 20370/37962 [47:11:17<40:07:36,  8.21s/it]
 54%|█████▎    | 20371/37962 [47:11:25<40:22:32,  8.26s/it]
 54%|█████▎    | 20372/37962 [47:11:33<40:20:41,  8.26s/it]
 54%|█████▎    | 20373/37962 [47:11:42<40:22:17,  8.26s/it]
 54%|█████▎    | 20374/37962 [47:11:50<40:27:46,  8.28s/it]
 54%|█████▎    | 20375/37962 [47:11:58<40:26:29,  8.28s/it]
 54%|█████▎    | 20376/37962 [47:12:07<40:38:47,  8.32s/it]
 54%|█████▎    | 20377/37962 [47:12:15<40:48:49,  8.36s/it]
 54%|█████▎    | 20378/37962 [47:12:23<40:41:18,  8.33s/it]
 54%|█████▎    | 20379/37962 [47:12:32<40:53:13,  8.37s/it]
 54%|█████▎    | 20380/37962 [47:12:40<40:51:38,  8.37s/it]
                                                           
{'loss': 1.4229, 'grad_norm': 6.733778953552246, 'learning_rate': 9.292894006278748e-06, 'epoch': 0.54}

 54%|█████▎    | 20380/37962 [47:12:40<40:51:38,  8.37s/it]
 54%|█████▎    | 20381/37962 [47:12:48<40:23:51,  8.27s/it]
 54%|█████▎    | 20382/37962 [47:12:56<39:38:45,  8.12s/it]
 54%|█████▎    | 20383/37962 [47:13:04<40:02:51,  8.20s/it]
 54%|█████▎    | 20384/37962 [47:13:13<40:08:15,  8.22s/it]
 54%|█████▎    | 20385/37962 [47:13:21<40:05:20,  8.21s/it]
 54%|█████▎    | 20386/37962 [47:13:29<40:05:09,  8.21s/it]
 54%|█████▎    | 20387/37962 [47:13:37<40:14:47,  8.24s/it]
 54%|█████▎    | 20388/37962 [47:13:46<40:11:23,  8.23s/it]
 54%|█████▎    | 20389/37962 [47:13:54<40:12:20,  8.24s/it]
 54%|█████▎    | 20390/37962 [47:14:02<40:20:59,  8.27s/it]
                                                           
{'loss': 1.2162, 'grad_norm': 5.283506870269775, 'learning_rate': 9.284384016041549e-06, 'epoch': 0.54}

 54%|█████▎    | 20390/37962 [47:14:02<40:20:59,  8.27s/it]
 54%|█████▎    | 20391/37962 [47:14:11<40:29:20,  8.30s/it]
 54%|█████▎    | 20392/37962 [47:14:19<40:04:21,  8.21s/it]
 54%|█████▎    | 20393/37962 [47:14:27<40:09:44,  8.23s/it]
 54%|█████▎    | 20394/37962 [47:14:35<40:16:08,  8.25s/it]
 54%|█████▎    | 20395/37962 [47:14:43<40:19:02,  8.26s/it]
 54%|█████▎    | 20396/37962 [47:14:52<40:41:50,  8.34s/it]
 54%|█████▎    | 20397/37962 [47:15:00<39:47:06,  8.15s/it]
 54%|█████▎    | 20398/37962 [47:15:08<40:04:04,  8.21s/it]
 54%|█████▎    | 20399/37962 [47:15:16<40:12:09,  8.24s/it]
 54%|█████▎    | 20400/37962 [47:15:24<40:03:47,  8.21s/it]
                                                           
{'loss': 1.3475, 'grad_norm': 9.889086723327637, 'learning_rate': 9.275874546688831e-06, 'epoch': 0.54}

 54%|█████▎    | 20400/37962 [47:15:24<40:03:47,  8.21s/it]
 54%|█████▎    | 20401/37962 [47:15:33<39:59:51,  8.20s/it]
 54%|█████▎    | 20402/37962 [47:15:41<40:03:13,  8.21s/it]
 54%|█████▎    | 20403/37962 [47:15:56<50:49:24, 10.42s/it]
 54%|█████▎    | 20404/37962 [47:16:05<47:53:21,  9.82s/it]
 54%|█████▍    | 20405/37962 [47:16:13<45:47:02,  9.39s/it]
 54%|█████▍    | 20406/37962 [47:16:21<43:57:09,  9.01s/it]
 54%|█████▍    | 20407/37962 [47:16:30<42:43:55,  8.76s/it]
 54%|█████▍    | 20408/37962 [47:16:38<41:58:18,  8.61s/it]
 54%|█████▍    | 20409/37962 [47:16:46<41:27:00,  8.50s/it]
 54%|█████▍    | 20410/37962 [47:16:54<40:38:49,  8.34s/it]
                                                           
{'loss': 1.3162, 'grad_norm': 6.22169828414917, 'learning_rate': 9.26736560441448e-06, 'epoch': 0.54}

 54%|█████▍    | 20410/37962 [47:16:54<40:38:49,  8.34s/it]
 54%|█████▍    | 20411/37962 [47:17:02<40:36:02,  8.33s/it]
 54%|█████▍    | 20412/37962 [47:17:11<40:45:57,  8.36s/it]
 54%|█████▍    | 20413/37962 [47:17:19<40:29:41,  8.31s/it]
 54%|█████▍    | 20414/37962 [47:17:27<40:32:49,  8.32s/it]
 54%|█████▍    | 20415/37962 [47:17:35<40:12:35,  8.25s/it]
 54%|█████▍    | 20416/37962 [47:17:44<40:29:51,  8.31s/it]
 54%|█████▍    | 20417/37962 [47:17:52<40:15:08,  8.26s/it]
 54%|█████▍    | 20418/37962 [47:18:00<40:25:35,  8.30s/it]
 54%|█████▍    | 20419/37962 [47:18:08<39:58:17,  8.20s/it]
 54%|█████▍    | 20420/37962 [47:18:16<39:46:09,  8.16s/it]
                                                           
{'loss': 1.2931, 'grad_norm': 6.661123752593994, 'learning_rate': 9.258857195412015e-06, 'epoch': 0.54}

 54%|█████▍    | 20420/37962 [47:18:16<39:46:09,  8.16s/it]
 54%|█████▍    | 20421/37962 [47:18:25<40:04:54,  8.23s/it]
 54%|█████▍    | 20422/37962 [47:18:33<39:55:52,  8.20s/it]
 54%|█████▍    | 20423/37962 [47:18:41<40:05:26,  8.23s/it]
 54%|█████▍    | 20424/37962 [47:18:49<40:12:16,  8.25s/it]
 54%|█████▍    | 20425/37962 [47:18:58<40:16:30,  8.27s/it]
 54%|█████▍    | 20426/37962 [47:19:06<40:14:48,  8.26s/it]
 54%|█████▍    | 20427/37962 [47:19:14<40:16:35,  8.27s/it]
 54%|█████▍    | 20428/37962 [47:19:23<40:26:18,  8.30s/it]
 54%|█████▍    | 20429/37962 [47:19:31<40:42:03,  8.36s/it]
 54%|█████▍    | 20430/37962 [47:19:39<40:33:16,  8.33s/it]
                                                           
{'loss': 1.3809, 'grad_norm': 12.40054702758789, 'learning_rate': 9.250349325874558e-06, 'epoch': 0.54}

 54%|█████▍    | 20430/37962 [47:19:39<40:33:16,  8.33s/it]
 54%|█████▍    | 20431/37962 [47:19:48<40:37:40,  8.34s/it]
 54%|█████▍    | 20432/37962 [47:19:56<40:38:17,  8.35s/it]
 54%|█████▍    | 20433/37962 [47:20:05<40:56:10,  8.41s/it]
 54%|█████▍    | 20434/37962 [47:20:13<40:31:17,  8.32s/it]
 54%|█████▍    | 20435/37962 [47:20:21<39:33:01,  8.12s/it]
 54%|█████▍    | 20436/37962 [47:20:28<39:18:50,  8.08s/it]
 54%|█████▍    | 20437/37962 [47:20:37<39:42:50,  8.16s/it]
 54%|█████▍    | 20438/37962 [47:20:45<39:53:13,  8.19s/it]
 54%|█████▍    | 20439/37962 [47:20:53<39:56:27,  8.21s/it]
 54%|█████▍    | 20440/37962 [47:21:02<40:01:23,  8.22s/it]
                                                           
{'loss': 1.3467, 'grad_norm': 8.174386024475098, 'learning_rate': 9.241842001994842e-06, 'epoch': 0.54}

 54%|█████▍    | 20440/37962 [47:21:02<40:01:23,  8.22s/it]
 54%|█████▍    | 20441/37962 [47:21:10<40:13:11,  8.26s/it]
 54%|█████▍    | 20442/37962 [47:21:18<40:07:47,  8.25s/it]
 54%|█████▍    | 20443/37962 [47:21:26<40:02:52,  8.23s/it]
 54%|█████▍    | 20444/37962 [47:21:35<40:07:06,  8.24s/it]
 54%|█████▍    | 20445/37962 [47:21:43<40:25:00,  8.31s/it]
 54%|█████▍    | 20446/37962 [47:21:51<40:13:53,  8.27s/it]
 54%|█████▍    | 20447/37962 [47:22:00<40:31:39,  8.33s/it]
 54%|█████▍    | 20448/37962 [47:22:08<40:34:27,  8.34s/it]
 54%|█████▍    | 20449/37962 [47:22:16<40:24:04,  8.30s/it]
 54%|█████▍    | 20450/37962 [47:22:25<40:27:44,  8.32s/it]
                                                           
{'loss': 1.353, 'grad_norm': 15.328058242797852, 'learning_rate': 9.233335229965199e-06, 'epoch': 0.54}

 54%|█████▍    | 20450/37962 [47:22:25<40:27:44,  8.32s/it]
 54%|█████▍    | 20451/37962 [47:22:33<39:52:04,  8.20s/it]
 54%|█████▍    | 20452/37962 [47:22:41<39:58:37,  8.22s/it]
 54%|█████▍    | 20453/37962 [47:22:49<40:03:30,  8.24s/it]
 54%|█████▍    | 20454/37962 [47:22:57<40:08:22,  8.25s/it]
 54%|█████▍    | 20455/37962 [47:23:06<40:38:19,  8.36s/it]
 54%|█████▍    | 20456/37962 [47:23:14<40:35:07,  8.35s/it]
 54%|█████▍    | 20457/37962 [47:23:23<40:29:23,  8.33s/it]
 54%|█████▍    | 20458/37962 [47:23:31<40:24:21,  8.31s/it]
 54%|█████▍    | 20459/37962 [47:23:39<40:22:30,  8.30s/it]
 54%|█████▍    | 20460/37962 [47:23:47<40:02:29,  8.24s/it]
                                                           
{'loss': 1.3483, 'grad_norm': 6.412385940551758, 'learning_rate': 9.224829015977558e-06, 'epoch': 0.54}

 54%|█████▍    | 20460/37962 [47:23:47<40:02:29,  8.24s/it]
 54%|█████▍    | 20461/37962 [47:23:56<40:17:58,  8.29s/it]
 54%|█████▍    | 20462/37962 [47:24:04<40:25:29,  8.32s/it]
 54%|█████▍    | 20463/37962 [47:24:12<40:17:30,  8.29s/it]
 54%|█████▍    | 20464/37962 [47:24:20<39:25:25,  8.11s/it]
 54%|█████▍    | 20465/37962 [47:24:28<39:23:23,  8.10s/it]
 54%|█████▍    | 20466/37962 [47:24:36<39:36:05,  8.15s/it]
 54%|█████▍    | 20467/37962 [47:24:45<39:42:33,  8.17s/it]
 54%|█████▍    | 20468/37962 [47:24:53<39:48:44,  8.19s/it]
 54%|█████▍    | 20469/37962 [47:25:01<39:41:06,  8.17s/it]
 54%|█████▍    | 20470/37962 [47:25:09<39:56:21,  8.22s/it]
                                                           
{'loss': 1.3242, 'grad_norm': 4.807486057281494, 'learning_rate': 9.21632336622345e-06, 'epoch': 0.54}

 54%|█████▍    | 20470/37962 [47:25:09<39:56:21,  8.22s/it]
 54%|█████▍    | 20471/37962 [47:25:17<39:27:07,  8.12s/it]
 54%|█████▍    | 20472/37962 [47:25:25<39:27:30,  8.12s/it]
 54%|█████▍    | 20473/37962 [47:25:33<39:21:35,  8.10s/it]
 54%|█████▍    | 20474/37962 [47:25:42<39:51:22,  8.20s/it]
 54%|█████▍    | 20475/37962 [47:25:50<39:56:12,  8.22s/it]
 54%|█████▍    | 20476/37962 [47:25:58<39:52:58,  8.21s/it]
 54%|█████▍    | 20477/37962 [47:26:06<40:00:36,  8.24s/it]
 54%|█████▍    | 20478/37962 [47:26:15<39:49:40,  8.20s/it]
 54%|█████▍    | 20479/37962 [47:26:23<40:12:56,  8.28s/it]
 54%|█████▍    | 20480/37962 [47:26:31<39:54:32,  8.22s/it]
                                                           
{'loss': 1.3244, 'grad_norm': 2.8902223110198975, 'learning_rate': 9.207818286893984e-06, 'epoch': 0.54}

 54%|█████▍    | 20480/37962 [47:26:31<39:54:32,  8.22s/it]
 54%|█████▍    | 20481/37962 [47:26:40<40:09:40,  8.27s/it]
 54%|█████▍    | 20482/37962 [47:26:48<39:57:18,  8.23s/it]
 54%|█████▍    | 20483/37962 [47:26:56<40:05:30,  8.26s/it]
 54%|█████▍    | 20484/37962 [47:27:04<40:10:10,  8.27s/it]
 54%|█████▍    | 20485/37962 [47:27:12<40:01:15,  8.24s/it]
 54%|█████▍    | 20486/37962 [47:27:21<39:49:37,  8.20s/it]
 54%|█████▍    | 20487/37962 [47:27:29<39:50:01,  8.21s/it]
 54%|█████▍    | 20488/37962 [47:27:37<40:07:24,  8.27s/it]
 54%|█████▍    | 20489/37962 [47:27:45<40:02:14,  8.25s/it]
 54%|█████▍    | 20490/37962 [47:27:54<39:58:32,  8.24s/it]
                                                           
{'loss': 1.2437, 'grad_norm': 3.055560827255249, 'learning_rate': 9.199313784179863e-06, 'epoch': 0.54}

 54%|█████▍    | 20490/37962 [47:27:54<39:58:32,  8.24s/it]
 54%|█████▍    | 20491/37962 [47:28:02<40:05:31,  8.26s/it]
 54%|█████▍    | 20492/37962 [47:28:10<39:44:02,  8.19s/it]
 54%|█████▍    | 20493/37962 [47:28:18<39:51:45,  8.21s/it]
 54%|█████▍    | 20494/37962 [47:28:27<39:56:17,  8.23s/it]
 54%|█████▍    | 20495/37962 [47:28:35<40:09:07,  8.28s/it]
 54%|█████▍    | 20496/37962 [47:28:43<40:10:21,  8.28s/it]
 54%|█████▍    | 20497/37962 [47:28:52<40:20:02,  8.31s/it]
 54%|█████▍    | 20498/37962 [47:29:00<40:15:39,  8.30s/it]
 54%|█████▍    | 20499/37962 [47:29:08<40:20:02,  8.31s/it]
 54%|█████▍    | 20500/37962 [47:29:16<39:49:18,  8.21s/it]
                                                           
{'loss': 1.3443, 'grad_norm': 11.85980224609375, 'learning_rate': 9.190809864271372e-06, 'epoch': 0.54}

 54%|█████▍    | 20500/37962 [47:29:16<39:49:18,  8.21s/it]
 54%|█████▍    | 20501/37962 [47:29:24<39:43:29,  8.19s/it]
 54%|█████▍    | 20502/37962 [47:29:32<39:22:34,  8.12s/it]
 54%|█████▍    | 20503/37962 [47:29:40<39:32:25,  8.15s/it]
 54%|█████▍    | 20504/37962 [47:29:49<39:36:44,  8.17s/it]
 54%|█████▍    | 20505/37962 [47:29:57<39:28:22,  8.14s/it]
 54%|█████▍    | 20506/37962 [47:30:05<39:50:25,  8.22s/it]
 54%|█████▍    | 20507/37962 [47:30:14<40:11:03,  8.29s/it]
 54%|█████▍    | 20508/37962 [47:30:22<40:07:48,  8.28s/it]
 54%|█████▍    | 20509/37962 [47:30:30<40:12:25,  8.29s/it]
 54%|█████▍    | 20510/37962 [47:30:38<39:43:41,  8.20s/it]
                                                           
{'loss': 1.3776, 'grad_norm': 18.41385841369629, 'learning_rate': 9.182306533358358e-06, 'epoch': 0.54}

 54%|█████▍    | 20510/37962 [47:30:38<39:43:41,  8.20s/it]
 54%|█████▍    | 20511/37962 [47:30:46<39:50:22,  8.22s/it]
 54%|█████▍    | 20512/37962 [47:30:54<39:17:00,  8.10s/it]
 54%|█████▍    | 20513/37962 [47:31:02<39:17:01,  8.10s/it]
 54%|█████▍    | 20514/37962 [47:31:11<39:40:16,  8.19s/it]
 54%|█████▍    | 20515/37962 [47:31:19<39:56:40,  8.24s/it]
 54%|█████▍    | 20516/37962 [47:31:27<39:58:05,  8.25s/it]
 54%|█████▍    | 20517/37962 [47:31:36<39:54:15,  8.23s/it]
 54%|█████▍    | 20518/37962 [47:31:44<40:06:25,  8.28s/it]
 54%|█████▍    | 20519/37962 [47:31:53<40:32:58,  8.37s/it]
 54%|█████▍    | 20520/37962 [47:32:01<40:00:23,  8.26s/it]
                                                           
{'loss': 1.3802, 'grad_norm': 7.31540060043335, 'learning_rate': 9.173803797630255e-06, 'epoch': 0.54}

 54%|█████▍    | 20520/37962 [47:32:01<40:00:23,  8.26s/it]
 54%|█████▍    | 20521/37962 [47:32:09<40:14:16,  8.31s/it]
 54%|█████▍    | 20522/37962 [47:32:17<40:24:52,  8.34s/it]
 54%|█████▍    | 20523/37962 [47:32:26<40:22:20,  8.33s/it]
 54%|█████▍    | 20524/37962 [47:32:34<40:17:38,  8.32s/it]
 54%|█████▍    | 20525/37962 [47:32:42<40:06:25,  8.28s/it]
 54%|█████▍    | 20526/37962 [47:32:51<40:18:49,  8.32s/it]
 54%|█████▍    | 20527/37962 [47:32:59<39:54:57,  8.24s/it]
 54%|█████▍    | 20528/37962 [47:33:07<39:54:08,  8.24s/it]
 54%|█████▍    | 20529/37962 [47:33:15<39:30:37,  8.16s/it]
 54%|█████▍    | 20530/37962 [47:33:23<39:51:49,  8.23s/it]
                                                           
{'loss': 1.3786, 'grad_norm': 10.61247444152832, 'learning_rate': 9.165301663276056e-06, 'epoch': 0.54}

 54%|█████▍    | 20530/37962 [47:33:23<39:51:49,  8.23s/it]
 54%|█████▍    | 20531/37962 [47:33:31<39:23:31,  8.14s/it]
 54%|█████▍    | 20532/37962 [47:33:40<39:41:02,  8.20s/it]
 54%|█████▍    | 20533/37962 [47:33:47<39:21:30,  8.13s/it]
 54%|█████▍    | 20534/37962 [47:33:55<39:05:23,  8.07s/it]
 54%|█████▍    | 20535/37962 [47:34:04<39:23:11,  8.14s/it]
 54%|█████▍    | 20536/37962 [47:34:12<39:36:49,  8.18s/it]
 54%|█████▍    | 20537/37962 [47:34:20<39:39:55,  8.19s/it]
 54%|█████▍    | 20538/37962 [47:34:29<40:00:11,  8.27s/it]
 54%|█████▍    | 20539/37962 [47:34:37<39:58:21,  8.26s/it]
 54%|█████▍    | 20540/37962 [47:34:45<40:03:13,  8.28s/it]
                                                           
{'loss': 1.3429, 'grad_norm': 11.754375457763672, 'learning_rate': 9.156800136484315e-06, 'epoch': 0.54}

 54%|█████▍    | 20540/37962 [47:34:45<40:03:13,  8.28s/it]
 54%|█████▍    | 20541/37962 [47:34:53<39:50:53,  8.23s/it]
 54%|█████▍    | 20542/37962 [47:35:02<39:50:03,  8.23s/it]
 54%|█████▍    | 20543/37962 [47:35:10<40:00:35,  8.27s/it]
 54%|█████▍    | 20544/37962 [47:35:18<40:24:48,  8.35s/it]
 54%|█████▍    | 20545/37962 [47:35:27<40:24:33,  8.35s/it]
 54%|█████▍    | 20546/37962 [47:35:35<40:15:41,  8.32s/it]
 54%|█████▍    | 20547/37962 [47:35:43<40:18:28,  8.33s/it]
 54%|█████▍    | 20548/37962 [47:35:52<40:13:12,  8.31s/it]
 54%|█████▍    | 20549/37962 [47:36:00<40:12:50,  8.31s/it]
 54%|█████▍    | 20550/37962 [47:36:08<39:53:22,  8.25s/it]
                                                           
{'loss': 1.3342, 'grad_norm': 9.424421310424805, 'learning_rate': 9.148299223443145e-06, 'epoch': 0.54}

 54%|█████▍    | 20550/37962 [47:36:08<39:53:22,  8.25s/it]
 54%|█████▍    | 20551/37962 [47:36:17<40:08:11,  8.30s/it]
 54%|█████▍    | 20552/37962 [47:36:25<40:19:20,  8.34s/it]
 54%|█████▍    | 20553/37962 [47:36:33<40:24:59,  8.36s/it]
 54%|█████▍    | 20554/37962 [47:36:42<40:32:12,  8.38s/it]
 54%|█████▍    | 20555/37962 [47:36:50<40:37:12,  8.40s/it]
 54%|█████▍    | 20556/37962 [47:36:58<40:21:18,  8.35s/it]
 54%|█████▍    | 20557/37962 [47:37:07<40:09:39,  8.31s/it]
 54%|█████▍    | 20558/37962 [47:37:15<40:16:35,  8.33s/it]
 54%|█████▍    | 20559/37962 [47:37:23<40:07:53,  8.30s/it]
 54%|█████▍    | 20560/37962 [47:37:32<39:59:18,  8.27s/it]
                                                           
{'loss': 1.3249, 'grad_norm': 3.3435277938842773, 'learning_rate': 9.13979893034022e-06, 'epoch': 0.54}

 54%|█████▍    | 20560/37962 [47:37:32<39:59:18,  8.27s/it]
 54%|█████▍    | 20561/37962 [47:37:40<39:59:07,  8.27s/it]
 54%|█████▍    | 20562/37962 [47:37:48<39:48:32,  8.24s/it]
 54%|█████▍    | 20563/37962 [47:37:56<39:17:28,  8.13s/it]
 54%|█████▍    | 20564/37962 [47:38:04<39:33:49,  8.19s/it]
 54%|█████▍    | 20565/37962 [47:38:13<39:54:58,  8.26s/it]
 54%|█████▍    | 20566/37962 [47:38:21<39:54:57,  8.26s/it]
 54%|█████▍    | 20567/37962 [47:38:29<39:21:02,  8.14s/it]
 54%|█████▍    | 20568/37962 [47:38:37<39:37:49,  8.20s/it]
 54%|█████▍    | 20569/37962 [47:38:45<39:30:18,  8.18s/it]
 54%|█████▍    | 20570/37962 [47:38:53<39:43:31,  8.22s/it]
                                                           
{'loss': 1.2849, 'grad_norm': 7.513850212097168, 'learning_rate': 9.13129926336275e-06, 'epoch': 0.54}

 54%|█████▍    | 20570/37962 [47:38:53<39:43:31,  8.22s/it]
 54%|█████▍    | 20571/37962 [47:39:02<39:41:32,  8.22s/it]
 54%|█████▍    | 20572/37962 [47:39:10<40:06:19,  8.30s/it]
 54%|█████▍    | 20573/37962 [47:39:19<40:07:39,  8.31s/it]
 54%|█████▍    | 20574/37962 [47:39:27<40:08:52,  8.31s/it]
 54%|█████▍    | 20575/37962 [47:39:35<40:01:57,  8.29s/it]
 54%|█████▍    | 20576/37962 [47:39:44<40:15:15,  8.34s/it]
 54%|█████▍    | 20577/37962 [47:39:52<39:49:41,  8.25s/it]
 54%|█████▍    | 20578/37962 [47:40:00<39:31:48,  8.19s/it]
 54%|█████▍    | 20579/37962 [47:40:08<39:36:24,  8.20s/it]
 54%|█████▍    | 20580/37962 [47:40:16<39:35:54,  8.20s/it]
                                                           
{'loss': 1.2955, 'grad_norm': 11.312723159790039, 'learning_rate': 9.122800228697495e-06, 'epoch': 0.54}

 54%|█████▍    | 20580/37962 [47:40:16<39:35:54,  8.20s/it]
 54%|█████▍    | 20581/37962 [47:40:24<39:32:54,  8.19s/it]
 54%|█████▍    | 20582/37962 [47:40:32<39:14:57,  8.13s/it]
 54%|█████▍    | 20583/37962 [47:40:41<39:32:22,  8.19s/it]
 54%|█████▍    | 20584/37962 [47:40:49<39:45:29,  8.24s/it]
 54%|█████▍    | 20585/37962 [47:40:57<39:29:56,  8.18s/it]
 54%|█████▍    | 20586/37962 [47:41:05<39:34:13,  8.20s/it]
 54%|█████▍    | 20587/37962 [47:41:13<39:45:56,  8.24s/it]
 54%|█████▍    | 20588/37962 [47:41:22<39:50:44,  8.26s/it]
 54%|█████▍    | 20589/37962 [47:41:30<40:23:57,  8.37s/it]
 54%|█████▍    | 20590/37962 [47:41:39<40:04:00,  8.30s/it]
                                                           
{'loss': 1.369, 'grad_norm': 9.243160247802734, 'learning_rate': 9.114301832530755e-06, 'epoch': 0.54}

 54%|█████▍    | 20590/37962 [47:41:39<40:04:00,  8.30s/it]
 54%|█████▍    | 20591/37962 [47:41:47<39:57:37,  8.28s/it]
 54%|█████▍    | 20592/37962 [47:41:55<39:18:49,  8.15s/it]
 54%|█████▍    | 20593/37962 [47:42:03<39:36:42,  8.21s/it]
 54%|█████▍    | 20594/37962 [47:42:11<39:42:54,  8.23s/it]
 54%|█████▍    | 20595/37962 [47:42:20<39:59:27,  8.29s/it]
 54%|█████▍    | 20596/37962 [47:42:28<39:57:41,  8.28s/it]
 54%|█████▍    | 20597/37962 [47:42:36<39:42:47,  8.23s/it]
 54%|█████▍    | 20598/37962 [47:42:44<39:53:29,  8.27s/it]
 54%|█████▍    | 20599/37962 [47:42:53<39:35:04,  8.21s/it]
 54%|█████▍    | 20600/37962 [47:43:01<39:37:02,  8.21s/it]
                                                           
{'loss': 1.3163, 'grad_norm': 6.749199867248535, 'learning_rate': 9.105804081048372e-06, 'epoch': 0.54}

 54%|█████▍    | 20600/37962 [47:43:01<39:37:02,  8.21s/it]
 54%|█████▍    | 20601/37962 [47:43:09<39:32:31,  8.20s/it]
 54%|█████▍    | 20602/37962 [47:43:17<39:47:02,  8.25s/it]
 54%|█████▍    | 20603/37962 [47:43:26<40:00:50,  8.30s/it]
 54%|█████▍    | 20604/37962 [47:43:34<40:15:14,  8.35s/it]
 54%|█████▍    | 20605/37962 [47:43:42<39:52:03,  8.27s/it]
 54%|█████▍    | 20606/37962 [47:43:51<39:54:17,  8.28s/it]
 54%|█████▍    | 20607/37962 [47:43:59<39:59:07,  8.29s/it]
 54%|█████▍    | 20608/37962 [47:44:07<39:29:30,  8.19s/it]
 54%|█████▍    | 20609/37962 [47:44:15<39:38:22,  8.22s/it]
 54%|█████▍    | 20610/37962 [47:44:23<39:43:09,  8.24s/it]
                                                           
{'loss': 1.2846, 'grad_norm': 4.227711200714111, 'learning_rate': 9.09730698043571e-06, 'epoch': 0.54}

 54%|█████▍    | 20610/37962 [47:44:23<39:43:09,  8.24s/it]
 54%|█████▍    | 20611/37962 [47:44:32<39:47:13,  8.26s/it]
 54%|█████▍    | 20612/37962 [47:44:40<39:51:37,  8.27s/it]
 54%|█████▍    | 20613/37962 [47:44:48<39:49:53,  8.27s/it]
 54%|█████▍    | 20614/37962 [47:44:57<40:02:19,  8.31s/it]
 54%|█████▍    | 20615/37962 [47:45:05<39:40:53,  8.24s/it]
 54%|█████▍    | 20616/37962 [47:45:13<39:50:26,  8.27s/it]
 54%|█████▍    | 20617/37962 [47:45:21<39:47:09,  8.26s/it]
 54%|█████▍    | 20618/37962 [47:45:30<39:45:19,  8.25s/it]
 54%|█████▍    | 20619/37962 [47:45:38<40:03:05,  8.31s/it]
 54%|█████▍    | 20620/37962 [47:45:46<39:58:11,  8.30s/it]
                                                           
{'loss': 1.2877, 'grad_norm': 14.402875900268555, 'learning_rate': 9.088810536877652e-06, 'epoch': 0.54}

 54%|█████▍    | 20620/37962 [47:45:46<39:58:11,  8.30s/it]
 54%|█████▍    | 20621/37962 [47:45:54<39:51:04,  8.27s/it]
 54%|█████▍    | 20622/37962 [47:46:03<39:35:40,  8.22s/it]
 54%|█████▍    | 20623/37962 [47:46:11<39:55:40,  8.29s/it]
 54%|█████▍    | 20624/37962 [47:46:19<39:59:56,  8.31s/it]
 54%|█████▍    | 20625/37962 [47:46:28<39:47:01,  8.26s/it]
 54%|█████▍    | 20626/37962 [47:46:36<39:43:38,  8.25s/it]
 54%|█████▍    | 20627/37962 [47:46:44<40:13:44,  8.35s/it]
 54%|█████▍    | 20628/37962 [47:46:53<40:12:47,  8.35s/it]
 54%|█████▍    | 20629/37962 [47:47:01<40:06:01,  8.33s/it]
 54%|█████▍    | 20630/37962 [47:47:09<39:58:09,  8.30s/it]
                                                           
{'loss': 1.3569, 'grad_norm': 12.266899108886719, 'learning_rate': 9.080314756558622e-06, 'epoch': 0.54}

 54%|█████▍    | 20630/37962 [47:47:09<39:58:09,  8.30s/it]
 54%|█████▍    | 20631/37962 [47:47:18<40:05:18,  8.33s/it]
 54%|█████▍    | 20632/37962 [47:47:26<40:06:01,  8.33s/it]
 54%|█████▍    | 20633/37962 [47:47:34<39:34:45,  8.22s/it]
 54%|█████▍    | 20634/37962 [47:47:42<39:54:28,  8.29s/it]
 54%|█████▍    | 20635/37962 [47:47:51<39:52:31,  8.28s/it]
 54%|█████▍    | 20636/37962 [47:47:59<39:53:03,  8.29s/it]
 54%|█████▍    | 20637/37962 [47:48:07<39:27:53,  8.20s/it]
 54%|█████▍    | 20638/37962 [47:48:15<39:06:52,  8.13s/it]
 54%|█████▍    | 20639/37962 [47:48:23<39:16:22,  8.16s/it]
 54%|█████▍    | 20640/37962 [47:48:31<39:33:36,  8.22s/it]
                                                           
{'loss': 1.3087, 'grad_norm': 7.241451263427734, 'learning_rate': 9.071819645662548e-06, 'epoch': 0.54}

 54%|█████▍    | 20640/37962 [47:48:31<39:33:36,  8.22s/it]
 54%|█████▍    | 20641/37962 [47:48:40<39:35:06,  8.23s/it]
 54%|█████▍    | 20642/37962 [47:48:48<39:36:50,  8.23s/it]
 54%|█████▍    | 20643/37962 [47:48:56<39:47:40,  8.27s/it]
 54%|█████▍    | 20644/37962 [47:49:05<40:22:31,  8.39s/it]
 54%|█████▍    | 20645/37962 [47:49:13<40:02:41,  8.32s/it]
 54%|█████▍    | 20646/37962 [47:49:22<40:10:28,  8.35s/it]
 54%|█████▍    | 20647/37962 [47:49:30<40:06:14,  8.34s/it]
 54%|█████▍    | 20648/37962 [47:49:38<40:08:49,  8.35s/it]
 54%|█████▍    | 20649/37962 [47:49:46<39:36:40,  8.24s/it]
 54%|█████▍    | 20650/37962 [47:49:54<39:33:47,  8.23s/it]
                                                           
{'loss': 1.2729, 'grad_norm': 5.761120319366455, 'learning_rate': 9.063325210372876e-06, 'epoch': 0.54}

 54%|█████▍    | 20650/37962 [47:49:54<39:33:47,  8.23s/it]
 54%|█████▍    | 20651/37962 [47:50:03<39:43:08,  8.26s/it]
 54%|█████▍    | 20652/37962 [47:50:11<39:43:41,  8.26s/it]
 54%|█████▍    | 20653/37962 [47:50:20<40:03:47,  8.33s/it]
 54%|█████▍    | 20654/37962 [47:50:28<39:52:36,  8.29s/it]
 54%|█████▍    | 20655/37962 [47:50:36<39:50:41,  8.29s/it]
 54%|█████▍    | 20656/37962 [47:50:44<39:45:08,  8.27s/it]
 54%|█████▍    | 20657/37962 [47:50:52<39:06:52,  8.14s/it]
 54%|█████▍    | 20658/37962 [47:51:00<39:23:30,  8.20s/it]
 54%|█████▍    | 20659/37962 [47:51:09<39:34:03,  8.23s/it]
 54%|█████▍    | 20660/37962 [47:51:17<40:06:00,  8.34s/it]
                                                           
{'loss': 1.3793, 'grad_norm': 6.2109761238098145, 'learning_rate': 9.054831456872556e-06, 'epoch': 0.54}

 54%|█████▍    | 20660/37962 [47:51:17<40:06:00,  8.34s/it]
 54%|█████▍    | 20661/37962 [47:51:25<39:42:18,  8.26s/it]
 54%|█████▍    | 20662/37962 [47:51:33<39:00:23,  8.12s/it]
 54%|█████▍    | 20663/37962 [47:51:42<39:22:24,  8.19s/it]
 54%|█████▍    | 20664/37962 [47:51:50<39:22:01,  8.19s/it]
 54%|█████▍    | 20665/37962 [47:51:58<39:31:32,  8.23s/it]
 54%|█████▍    | 20666/37962 [47:52:06<39:25:40,  8.21s/it]
 54%|█████▍    | 20667/37962 [47:52:14<39:33:38,  8.23s/it]
 54%|█████▍    | 20668/37962 [47:52:23<39:37:04,  8.25s/it]
 54%|█████▍    | 20669/37962 [47:52:31<39:17:30,  8.18s/it]
 54%|█████▍    | 20670/37962 [47:52:39<39:33:57,  8.24s/it]
                                                           
{'loss': 1.3159, 'grad_norm': 10.001215934753418, 'learning_rate': 9.046338391344044e-06, 'epoch': 0.54}

 54%|█████▍    | 20670/37962 [47:52:39<39:33:57,  8.24s/it]
 54%|█████▍    | 20671/37962 [47:52:47<39:37:08,  8.25s/it]
 54%|█████▍    | 20672/37962 [47:52:56<39:22:23,  8.20s/it]
 54%|█████▍    | 20673/37962 [47:53:04<39:34:52,  8.24s/it]
 54%|█████▍    | 20674/37962 [47:53:12<39:41:10,  8.26s/it]
 54%|█████▍    | 20675/37962 [47:53:20<39:33:21,  8.24s/it]
 54%|█████▍    | 20676/37962 [47:53:28<39:18:50,  8.19s/it]
 54%|█████▍    | 20677/37962 [47:53:37<39:25:58,  8.21s/it]
 54%|█████▍    | 20678/37962 [47:53:45<39:35:26,  8.25s/it]
 54%|█████▍    | 20679/37962 [47:53:53<39:21:55,  8.20s/it]
 54%|█████▍    | 20680/37962 [47:54:01<39:08:19,  8.15s/it]
                                                           
{'loss': 1.3992, 'grad_norm': 4.342574119567871, 'learning_rate': 9.037846019969296e-06, 'epoch': 0.54}

 54%|█████▍    | 20680/37962 [47:54:01<39:08:19,  8.15s/it]
 54%|█████▍    | 20681/37962 [47:54:09<39:08:15,  8.15s/it]
 54%|█████▍    | 20682/37962 [47:54:17<39:10:38,  8.16s/it]
 54%|█████▍    | 20683/37962 [47:54:26<39:43:23,  8.28s/it]
 54%|█████▍    | 20684/37962 [47:54:34<39:39:34,  8.26s/it]
 54%|█████▍    | 20685/37962 [47:54:42<39:16:54,  8.19s/it]
 54%|█████▍    | 20686/37962 [47:54:50<38:36:12,  8.04s/it]
 54%|█████▍    | 20687/37962 [47:54:58<39:11:46,  8.17s/it]
 54%|█████▍    | 20688/37962 [47:55:07<39:19:30,  8.20s/it]
 54%|█████▍    | 20689/37962 [47:55:15<39:26:15,  8.22s/it]
 55%|█████▍    | 20690/37962 [47:55:23<39:44:59,  8.29s/it]
                                                           
{'loss': 1.2675, 'grad_norm': 12.889762878417969, 'learning_rate': 9.029354348929759e-06, 'epoch': 0.55}

 55%|█████▍    | 20690/37962 [47:55:23<39:44:59,  8.29s/it]
 55%|█████▍    | 20691/37962 [47:55:32<39:40:09,  8.27s/it]
 55%|█████▍    | 20692/37962 [47:55:40<39:06:28,  8.15s/it]
 55%|█████▍    | 20693/37962 [47:55:48<39:14:26,  8.18s/it]
 55%|█████▍    | 20694/37962 [47:55:56<39:32:14,  8.24s/it]
 55%|█████▍    | 20695/37962 [47:56:05<39:42:35,  8.28s/it]
 55%|█████▍    | 20696/37962 [47:56:13<39:40:53,  8.27s/it]
 55%|█████▍    | 20697/37962 [47:56:21<39:15:36,  8.19s/it]
 55%|█████▍    | 20698/37962 [47:56:29<39:13:56,  8.18s/it]
 55%|█████▍    | 20699/37962 [47:56:37<39:19:28,  8.20s/it]
 55%|█████▍    | 20700/37962 [47:56:45<38:58:23,  8.13s/it]
                                                           
{'loss': 1.3296, 'grad_norm': 7.007455348968506, 'learning_rate': 9.020863384406378e-06, 'epoch': 0.55}

 55%|█████▍    | 20700/37962 [47:56:45<38:58:23,  8.13s/it]
 55%|█████▍    | 20701/37962 [47:56:53<39:10:21,  8.17s/it]
 55%|█████▍    | 20702/37962 [47:57:01<38:51:56,  8.11s/it]
 55%|█████▍    | 20703/37962 [47:57:10<39:04:51,  8.15s/it]
 55%|█████▍    | 20704/37962 [47:57:18<39:24:52,  8.22s/it]
 55%|█████▍    | 20705/37962 [47:57:26<39:42:28,  8.28s/it]
 55%|█████▍    | 20706/37962 [47:57:35<39:36:08,  8.26s/it]
 55%|█████▍    | 20707/37962 [47:57:43<39:40:38,  8.28s/it]
 55%|█████▍    | 20708/37962 [47:57:52<40:06:55,  8.37s/it]
 55%|█████▍    | 20709/37962 [47:58:00<39:52:21,  8.32s/it]
 55%|█████▍    | 20710/37962 [47:58:08<40:11:57,  8.39s/it]
                                                           
{'loss': 1.2592, 'grad_norm': 22.111038208007812, 'learning_rate': 9.012373132579582e-06, 'epoch': 0.55}

 55%|█████▍    | 20710/37962 [47:58:08<40:11:57,  8.39s/it]
 55%|█████▍    | 20711/37962 [47:58:17<40:02:04,  8.35s/it]
 55%|█████▍    | 20712/37962 [47:58:25<39:58:27,  8.34s/it]
 55%|█████▍    | 20713/37962 [47:58:33<40:09:39,  8.38s/it]
 55%|█████▍    | 20714/37962 [47:58:42<40:15:02,  8.40s/it]
 55%|█████▍    | 20715/37962 [47:58:50<39:40:24,  8.28s/it]
 55%|█████▍    | 20716/37962 [47:58:58<39:35:15,  8.26s/it]
 55%|█████▍    | 20717/37962 [47:59:07<39:55:09,  8.33s/it]
 55%|█████▍    | 20718/37962 [47:59:15<39:34:49,  8.26s/it]
 55%|█████▍    | 20719/37962 [47:59:23<39:10:20,  8.18s/it]
 55%|█████▍    | 20720/37962 [47:59:31<39:15:15,  8.20s/it]
                                                           
{'loss': 1.3844, 'grad_norm': 14.419602394104004, 'learning_rate': 9.003883599629266e-06, 'epoch': 0.55}

 55%|█████▍    | 20720/37962 [47:59:31<39:15:15,  8.20s/it]
 55%|█████▍    | 20721/37962 [47:59:39<38:53:04,  8.12s/it]
 55%|█████▍    | 20722/37962 [47:59:47<39:13:13,  8.19s/it]
 55%|█████▍    | 20723/37962 [47:59:55<39:14:19,  8.19s/it]
 55%|█████▍    | 20724/37962 [48:00:04<39:27:10,  8.24s/it]
 55%|█████▍    | 20725/37962 [48:00:12<39:15:18,  8.20s/it]
 55%|█████▍    | 20726/37962 [48:00:20<38:38:07,  8.07s/it]
 55%|█████▍    | 20727/37962 [48:00:28<39:06:56,  8.17s/it]
 55%|█████▍    | 20728/37962 [48:00:36<39:15:18,  8.20s/it]
 55%|█████▍    | 20729/37962 [48:00:44<39:08:48,  8.18s/it]
 55%|█████▍    | 20730/37962 [48:00:52<38:53:59,  8.13s/it]
                                                           
{'loss': 1.3155, 'grad_norm': 2.2722208499908447, 'learning_rate': 8.995394791734825e-06, 'epoch': 0.55}

 55%|█████▍    | 20730/37962 [48:00:52<38:53:59,  8.13s/it]
 55%|█████▍    | 20731/37962 [48:01:01<38:59:50,  8.15s/it]
 55%|█████▍    | 20732/37962 [48:01:09<39:09:53,  8.18s/it]
 55%|█████▍    | 20733/37962 [48:01:17<39:18:10,  8.21s/it]
 55%|█████▍    | 20734/37962 [48:01:25<39:14:17,  8.20s/it]
 55%|█████▍    | 20735/37962 [48:01:34<39:45:10,  8.31s/it]
 55%|█████▍    | 20736/37962 [48:01:42<39:19:17,  8.22s/it]
 55%|█████▍    | 20737/37962 [48:01:50<39:11:58,  8.19s/it]
 55%|█████▍    | 20738/37962 [48:01:58<39:18:08,  8.21s/it]
 55%|█████▍    | 20739/37962 [48:02:07<39:33:25,  8.27s/it]
 55%|█████▍    | 20740/37962 [48:02:15<39:03:14,  8.16s/it]
                                                           
{'loss': 1.2926, 'grad_norm': 6.060754776000977, 'learning_rate': 8.98690671507511e-06, 'epoch': 0.55}

 55%|█████▍    | 20740/37962 [48:02:15<39:03:14,  8.16s/it]
 55%|█████▍    | 20741/37962 [48:02:23<39:08:51,  8.18s/it]
 55%|█████▍    | 20742/37962 [48:02:31<39:26:50,  8.25s/it]
 55%|█████▍    | 20743/37962 [48:02:39<39:22:51,  8.23s/it]
 55%|█████▍    | 20744/37962 [48:02:48<39:33:37,  8.27s/it]
 55%|█████▍    | 20745/37962 [48:02:56<39:23:50,  8.24s/it]
 55%|█████▍    | 20746/37962 [48:03:04<39:35:58,  8.28s/it]
 55%|█████▍    | 20747/37962 [48:03:13<39:31:41,  8.27s/it]
 55%|█████▍    | 20748/37962 [48:03:21<39:07:15,  8.18s/it]
 55%|█████▍    | 20749/37962 [48:03:29<39:32:55,  8.27s/it]
 55%|█████▍    | 20750/37962 [48:03:37<39:39:03,  8.29s/it]
                                                           
{'loss': 1.4526, 'grad_norm': 9.068933486938477, 'learning_rate': 8.978419375828448e-06, 'epoch': 0.55}

 55%|█████▍    | 20750/37962 [48:03:37<39:39:03,  8.29s/it]
 55%|█████▍    | 20751/37962 [48:03:46<39:41:01,  8.30s/it]
 55%|█████▍    | 20752/37962 [48:03:54<39:43:28,  8.31s/it]
 55%|█████▍    | 20753/37962 [48:04:02<39:50:43,  8.34s/it]
 55%|█████▍    | 20754/37962 [48:04:11<39:39:55,  8.30s/it]
 55%|█████▍    | 20755/37962 [48:04:19<39:33:00,  8.27s/it]
 55%|█████▍    | 20756/37962 [48:04:27<39:43:14,  8.31s/it]
 55%|█████▍    | 20757/37962 [48:04:35<39:26:27,  8.25s/it]
 55%|█████▍    | 20758/37962 [48:04:44<39:33:57,  8.28s/it]
 55%|█████▍    | 20759/37962 [48:04:52<39:25:40,  8.25s/it]
 55%|█████▍    | 20760/37962 [48:05:00<39:29:26,  8.26s/it]
                                                           
{'loss': 1.2994, 'grad_norm': 11.597220420837402, 'learning_rate': 8.969932780172623e-06, 'epoch': 0.55}

 55%|█████▍    | 20760/37962 [48:05:00<39:29:26,  8.26s/it]
 55%|█████▍    | 20761/37962 [48:05:08<39:33:04,  8.28s/it]
 55%|█████▍    | 20762/37962 [48:05:17<39:34:33,  8.28s/it]
 55%|█████▍    | 20763/37962 [48:05:25<39:44:40,  8.32s/it]
 55%|█████▍    | 20764/37962 [48:05:34<39:57:21,  8.36s/it]
 55%|█████▍    | 20765/37962 [48:05:42<39:51:38,  8.34s/it]
 55%|█████▍    | 20766/37962 [48:05:50<39:39:41,  8.30s/it]
 55%|█████▍    | 20767/37962 [48:05:58<39:16:02,  8.22s/it]
 55%|█████▍    | 20768/37962 [48:06:06<39:20:07,  8.24s/it]
 55%|█████▍    | 20769/37962 [48:06:15<39:32:52,  8.28s/it]
 55%|█████▍    | 20770/37962 [48:06:23<39:38:14,  8.30s/it]
                                                           
{'loss': 1.2906, 'grad_norm': 8.490080833435059, 'learning_rate': 8.961446934284886e-06, 'epoch': 0.55}

 55%|█████▍    | 20770/37962 [48:06:23<39:38:14,  8.30s/it]
 55%|█████▍    | 20771/37962 [48:06:31<39:37:51,  8.30s/it]
 55%|█████▍    | 20772/37962 [48:06:40<39:30:13,  8.27s/it]
 55%|█████▍    | 20773/37962 [48:06:48<39:07:06,  8.19s/it]
 55%|█████▍    | 20774/37962 [48:06:56<39:07:45,  8.20s/it]
 55%|█████▍    | 20775/37962 [48:07:04<39:13:47,  8.22s/it]
 55%|█████▍    | 20776/37962 [48:07:12<39:09:34,  8.20s/it]
 55%|█████▍    | 20777/37962 [48:07:20<39:07:54,  8.20s/it]
 55%|█████▍    | 20778/37962 [48:07:29<39:11:49,  8.21s/it]
 55%|█████▍    | 20779/37962 [48:07:37<39:13:18,  8.22s/it]
 55%|█████▍    | 20780/37962 [48:07:45<39:29:05,  8.27s/it]
                                                           
{'loss': 1.3979, 'grad_norm': 17.67237091064453, 'learning_rate': 8.952961844341933e-06, 'epoch': 0.55}

 55%|█████▍    | 20780/37962 [48:07:45<39:29:05,  8.27s/it]
 55%|█████▍    | 20781/37962 [48:07:54<39:27:45,  8.27s/it]
 55%|█████▍    | 20782/37962 [48:08:02<39:50:52,  8.35s/it]
 55%|█████▍    | 20783/37962 [48:08:10<39:23:46,  8.26s/it]
 55%|█████▍    | 20784/37962 [48:08:18<39:26:20,  8.27s/it]
 55%|█████▍    | 20785/37962 [48:08:27<39:23:27,  8.26s/it]
 55%|█████▍    | 20786/37962 [48:08:42<49:41:01, 10.41s/it]
 55%|█████▍    | 20787/37962 [48:08:50<46:37:39,  9.77s/it]
 55%|█████▍    | 20788/37962 [48:08:59<44:35:24,  9.35s/it]
 55%|█████▍    | 20789/37962 [48:09:07<43:05:47,  9.03s/it]
 55%|█████▍    | 20790/37962 [48:09:15<41:55:12,  8.79s/it]
                                                           
{'loss': 1.3617, 'grad_norm': 7.586197376251221, 'learning_rate': 8.94447751651991e-06, 'epoch': 0.55}

 55%|█████▍    | 20790/37962 [48:09:15<41:55:12,  8.79s/it]
 55%|█████▍    | 20791/37962 [48:09:24<41:13:08,  8.64s/it]
 55%|█████▍    | 20792/37962 [48:09:32<41:04:47,  8.61s/it]
 55%|█████▍    | 20793/37962 [48:09:40<40:34:45,  8.51s/it]
 55%|█████▍    | 20794/37962 [48:09:49<40:15:09,  8.44s/it]
 55%|█████▍    | 20795/37962 [48:09:57<40:14:22,  8.44s/it]
 55%|█████▍    | 20796/37962 [48:10:05<39:55:35,  8.37s/it]
 55%|█████▍    | 20797/37962 [48:10:14<39:40:51,  8.32s/it]
 55%|█████▍    | 20798/37962 [48:10:22<39:25:10,  8.27s/it]
 55%|█████▍    | 20799/37962 [48:10:30<38:45:29,  8.13s/it]
 55%|█████▍    | 20800/37962 [48:10:38<39:24:27,  8.27s/it]
                                                           
{'loss': 1.2362, 'grad_norm': 10.99543571472168, 'learning_rate': 8.93599395699442e-06, 'epoch': 0.55}

 55%|█████▍    | 20800/37962 [48:10:38<39:24:27,  8.27s/it]
 55%|█████▍    | 20801/37962 [48:10:46<39:19:01,  8.25s/it]
 55%|█████▍    | 20802/37962 [48:10:54<39:11:05,  8.22s/it]
 55%|█████▍    | 20803/37962 [48:11:03<39:19:56,  8.25s/it]
 55%|█████▍    | 20804/37962 [48:11:11<39:24:03,  8.27s/it]
 55%|█████▍    | 20805/37962 [48:11:19<39:26:14,  8.27s/it]
 55%|█████▍    | 20806/37962 [48:11:28<39:25:53,  8.27s/it]
 55%|█████▍    | 20807/37962 [48:11:36<39:30:52,  8.29s/it]
 55%|█████▍    | 20808/37962 [48:11:44<39:35:19,  8.31s/it]
 55%|█████▍    | 20809/37962 [48:11:53<39:38:38,  8.32s/it]
 55%|█████▍    | 20810/37962 [48:12:01<39:21:37,  8.26s/it]
                                                           
{'loss': 1.1337, 'grad_norm': 3.9197394847869873, 'learning_rate': 8.927511171940496e-06, 'epoch': 0.55}

 55%|█████▍    | 20810/37962 [48:12:01<39:21:37,  8.26s/it]
 55%|█████▍    | 20811/37962 [48:12:09<39:19:22,  8.25s/it]
 55%|█████▍    | 20812/37962 [48:12:17<39:17:19,  8.25s/it]
 55%|█████▍    | 20813/37962 [48:12:26<39:27:12,  8.28s/it]
 55%|█████▍    | 20814/37962 [48:12:34<39:48:19,  8.36s/it]
 55%|█████▍    | 20815/37962 [48:12:42<39:33:25,  8.30s/it]
 55%|█████▍    | 20816/37962 [48:12:51<39:28:22,  8.29s/it]
 55%|█████▍    | 20817/37962 [48:12:59<39:35:33,  8.31s/it]
 55%|█████▍    | 20818/37962 [48:13:07<39:06:02,  8.21s/it]
 55%|█████▍    | 20819/37962 [48:13:15<39:12:46,  8.23s/it]
 55%|█████▍    | 20820/37962 [48:13:23<38:49:22,  8.15s/it]
                                                           
{'loss': 1.2125, 'grad_norm': 10.230262756347656, 'learning_rate': 8.919029167532612e-06, 'epoch': 0.55}

 55%|█████▍    | 20820/37962 [48:13:23<38:49:22,  8.15s/it]
 55%|█████▍    | 20821/37962 [48:13:31<38:46:38,  8.14s/it]
 55%|█████▍    | 20822/37962 [48:13:40<38:53:45,  8.17s/it]
 55%|█████▍    | 20823/37962 [48:13:48<38:54:42,  8.17s/it]
 55%|█████▍    | 20824/37962 [48:13:56<38:57:43,  8.18s/it]
 55%|█████▍    | 20825/37962 [48:14:04<39:18:31,  8.26s/it]
 55%|█████▍    | 20826/37962 [48:14:13<39:20:51,  8.27s/it]
 55%|█████▍    | 20827/37962 [48:14:21<39:24:16,  8.28s/it]
 55%|█████▍    | 20828/37962 [48:14:29<39:25:25,  8.28s/it]
 55%|█████▍    | 20829/37962 [48:14:38<39:28:58,  8.30s/it]
 55%|█████▍    | 20830/37962 [48:14:46<39:43:28,  8.35s/it]
                                                           
{'loss': 1.3553, 'grad_norm': 9.550813674926758, 'learning_rate': 8.91054794994467e-06, 'epoch': 0.55}

 55%|█████▍    | 20830/37962 [48:14:46<39:43:28,  8.35s/it]
 55%|█████▍    | 20831/37962 [48:14:54<39:37:46,  8.33s/it]
 55%|█████▍    | 20832/37962 [48:15:03<39:44:58,  8.35s/it]
 55%|█████▍    | 20833/37962 [48:15:11<39:33:35,  8.31s/it]
 55%|█████▍    | 20834/37962 [48:15:19<39:48:08,  8.37s/it]
 55%|█████▍    | 20835/37962 [48:15:28<39:38:47,  8.33s/it]
 55%|█████▍    | 20836/37962 [48:15:36<39:51:25,  8.38s/it]
 55%|█████▍    | 20837/37962 [48:15:44<39:19:34,  8.27s/it]
 55%|█████▍    | 20838/37962 [48:15:53<39:20:46,  8.27s/it]
 55%|█████▍    | 20839/37962 [48:16:01<39:24:59,  8.29s/it]
 55%|█████▍    | 20840/37962 [48:16:09<39:18:30,  8.26s/it]
                                                           
{'loss': 1.367, 'grad_norm': 6.1724324226379395, 'learning_rate': 8.90206752535e-06, 'epoch': 0.55}

 55%|█████▍    | 20840/37962 [48:16:09<39:18:30,  8.26s/it]
 55%|█████▍    | 20841/37962 [48:16:17<39:02:17,  8.21s/it]
 55%|█████▍    | 20842/37962 [48:16:25<39:04:07,  8.22s/it]
 55%|█████▍    | 20843/37962 [48:16:34<39:30:59,  8.31s/it]
 55%|█████▍    | 20844/37962 [48:16:42<39:40:43,  8.34s/it]
 55%|█████▍    | 20845/37962 [48:16:51<39:28:11,  8.30s/it]
 55%|█████▍    | 20846/37962 [48:16:59<39:24:12,  8.29s/it]
 55%|█████▍    | 20847/37962 [48:17:07<39:32:04,  8.32s/it]
 55%|█████▍    | 20848/37962 [48:17:15<39:31:13,  8.31s/it]
 55%|█████▍    | 20849/37962 [48:17:31<50:10:40, 10.56s/it]
 55%|█████▍    | 20850/37962 [48:17:40<47:22:30,  9.97s/it]
                                                           
{'loss': 1.322, 'grad_norm': 3.3362905979156494, 'learning_rate': 8.89358789992136e-06, 'epoch': 0.55}

 55%|█████▍    | 20850/37962 [48:17:40<47:22:30,  9.97s/it]
 55%|█████▍    | 20851/37962 [48:17:48<44:51:20,  9.44s/it]
 55%|█████▍    | 20852/37962 [48:17:56<43:24:20,  9.13s/it]
 55%|█████▍    | 20853/37962 [48:18:05<42:12:13,  8.88s/it]
 55%|█████▍    | 20854/37962 [48:18:13<41:12:27,  8.67s/it]
 55%|█████▍    | 20855/37962 [48:18:21<40:30:05,  8.52s/it]
 55%|█████▍    | 20856/37962 [48:18:29<40:09:32,  8.45s/it]
 55%|█████▍    | 20857/37962 [48:18:37<39:30:07,  8.31s/it]
 55%|█████▍    | 20858/37962 [48:18:46<39:44:27,  8.36s/it]
 55%|█████▍    | 20859/37962 [48:18:54<39:21:24,  8.28s/it]
 55%|█████▍    | 20860/37962 [48:19:02<39:29:33,  8.31s/it]
                                                           
{'loss': 1.3633, 'grad_norm': 9.346821784973145, 'learning_rate': 8.885109079830918e-06, 'epoch': 0.55}

 55%|█████▍    | 20860/37962 [48:19:02<39:29:33,  8.31s/it]
 55%|█████▍    | 20861/37962 [48:19:11<39:38:58,  8.35s/it]
 55%|█████▍    | 20862/37962 [48:19:19<39:43:00,  8.36s/it]
 55%|█████▍    | 20863/37962 [48:19:27<39:30:22,  8.32s/it]
 55%|█████▍    | 20864/37962 [48:19:36<39:30:38,  8.32s/it]
 55%|█████▍    | 20865/37962 [48:19:44<39:32:58,  8.33s/it]
 55%|█████▍    | 20866/37962 [48:19:52<39:26:19,  8.30s/it]
 55%|█████▍    | 20867/37962 [48:20:01<39:27:51,  8.31s/it]
 55%|█████▍    | 20868/37962 [48:20:09<39:22:07,  8.29s/it]
 55%|█████▍    | 20869/37962 [48:20:17<39:24:06,  8.30s/it]
 55%|█████▍    | 20870/37962 [48:20:25<38:51:23,  8.18s/it]
                                                           
{'loss': 1.2528, 'grad_norm': 8.440089225769043, 'learning_rate': 8.876631071250266e-06, 'epoch': 0.55}

 55%|█████▍    | 20870/37962 [48:20:25<38:51:23,  8.18s/it]
 55%|█████▍    | 20871/37962 [48:20:34<39:18:50,  8.28s/it]
 55%|█████▍    | 20872/37962 [48:20:42<39:25:27,  8.30s/it]
 55%|█████▍    | 20873/37962 [48:20:50<39:26:04,  8.31s/it]
 55%|█████▍    | 20874/37962 [48:20:58<39:11:54,  8.26s/it]
 55%|█████▍    | 20875/37962 [48:21:07<39:16:40,  8.28s/it]
 55%|█████▍    | 20876/37962 [48:21:15<39:22:18,  8.30s/it]
 55%|█████▍    | 20877/37962 [48:21:23<39:18:39,  8.28s/it]
 55%|█████▍    | 20878/37962 [48:21:32<39:31:40,  8.33s/it]
 55%|█████▍    | 20879/37962 [48:21:40<39:06:27,  8.24s/it]
 55%|█████▌    | 20880/37962 [48:21:48<39:13:49,  8.27s/it]
                                                           
{'loss': 1.3402, 'grad_norm': 12.390538215637207, 'learning_rate': 8.868153880350394e-06, 'epoch': 0.55}

 55%|█████▌    | 20880/37962 [48:21:48<39:13:49,  8.27s/it]
 55%|█████▌    | 20881/37962 [48:21:56<39:14:08,  8.27s/it]
 55%|█████▌    | 20882/37962 [48:22:05<39:17:28,  8.28s/it]
 55%|█████▌    | 20883/37962 [48:22:13<39:10:48,  8.26s/it]
 55%|█████▌    | 20884/37962 [48:22:21<39:10:25,  8.26s/it]
 55%|█████▌    | 20885/37962 [48:22:29<39:00:25,  8.22s/it]
 55%|█████▌    | 20886/37962 [48:22:38<38:57:12,  8.21s/it]
 55%|█████▌    | 20887/37962 [48:22:46<39:02:16,  8.23s/it]
 55%|█████▌    | 20888/37962 [48:22:54<39:08:07,  8.25s/it]
 55%|█████▌    | 20889/37962 [48:23:02<39:00:02,  8.22s/it]
 55%|█████▌    | 20890/37962 [48:23:11<39:18:05,  8.29s/it]
                                                           
{'loss': 1.1498, 'grad_norm': 13.170918464660645, 'learning_rate': 8.859677513301705e-06, 'epoch': 0.55}

 55%|█████▌    | 20890/37962 [48:23:11<39:18:05,  8.29s/it]
 55%|█████▌    | 20891/37962 [48:23:19<39:18:42,  8.29s/it]
 55%|█████▌    | 20892/37962 [48:23:27<39:20:11,  8.30s/it]
 55%|█████▌    | 20893/37962 [48:23:36<39:23:08,  8.31s/it]
 55%|█████▌    | 20894/37962 [48:23:44<39:17:53,  8.29s/it]
 55%|█████▌    | 20895/37962 [48:23:52<39:03:52,  8.24s/it]
 55%|█████▌    | 20896/37962 [48:24:00<39:10:01,  8.26s/it]
 55%|█████▌    | 20897/37962 [48:24:08<38:55:54,  8.21s/it]
 55%|█████▌    | 20898/37962 [48:24:17<39:00:05,  8.23s/it]
 55%|█████▌    | 20899/37962 [48:24:25<39:05:38,  8.25s/it]
 55%|█████▌    | 20900/37962 [48:24:33<39:24:29,  8.31s/it]
                                                           
{'loss': 1.3826, 'grad_norm': 9.257023811340332, 'learning_rate': 8.851201976273997e-06, 'epoch': 0.55}

 55%|█████▌    | 20900/37962 [48:24:33<39:24:29,  8.31s/it]
 55%|█████▌    | 20901/37962 [48:24:42<39:19:45,  8.30s/it]
 55%|█████▌    | 20902/37962 [48:24:50<39:13:49,  8.28s/it]
 55%|█████▌    | 20903/37962 [48:24:58<39:12:56,  8.28s/it]
 55%|█████▌    | 20904/37962 [48:25:06<39:05:53,  8.25s/it]
 55%|█████▌    | 20905/37962 [48:25:14<38:52:09,  8.20s/it]
 55%|█████▌    | 20906/37962 [48:25:23<38:51:13,  8.20s/it]
 55%|█████▌    | 20907/37962 [48:25:31<39:04:09,  8.25s/it]
 55%|█████▌    | 20908/37962 [48:25:39<39:07:35,  8.26s/it]
 55%|█████▌    | 20909/37962 [48:25:48<39:12:49,  8.28s/it]
 55%|█████▌    | 20910/37962 [48:25:56<38:43:18,  8.17s/it]
                                                           
{'loss': 1.2814, 'grad_norm': 10.819096565246582, 'learning_rate': 8.84272727543647e-06, 'epoch': 0.55}

 55%|█████▌    | 20910/37962 [48:25:56<38:43:18,  8.17s/it]
 55%|█████▌    | 20911/37962 [48:26:04<39:04:02,  8.25s/it]
 55%|█████▌    | 20912/37962 [48:26:12<39:11:37,  8.28s/it]
 55%|█████▌    | 20913/37962 [48:26:20<38:54:11,  8.21s/it]
 55%|█████▌    | 20914/37962 [48:26:29<39:19:49,  8.31s/it]
 55%|█████▌    | 20915/37962 [48:26:37<38:52:01,  8.21s/it]
 55%|█████▌    | 20916/37962 [48:26:45<38:55:50,  8.22s/it]
 55%|█████▌    | 20917/37962 [48:26:54<39:29:53,  8.34s/it]
 55%|█████▌    | 20918/37962 [48:27:02<39:27:14,  8.33s/it]
 55%|█████▌    | 20919/37962 [48:27:10<39:19:36,  8.31s/it]
 55%|█████▌    | 20920/37962 [48:27:18<38:54:38,  8.22s/it]
                                                           
{'loss': 1.2574, 'grad_norm': 9.847457885742188, 'learning_rate': 8.834253416957714e-06, 'epoch': 0.55}

 55%|█████▌    | 20920/37962 [48:27:18<38:54:38,  8.22s/it]
 55%|█████▌    | 20921/37962 [48:27:27<39:03:12,  8.25s/it]
 55%|█████▌    | 20922/37962 [48:27:35<39:09:21,  8.27s/it]
 55%|█████▌    | 20923/37962 [48:27:44<39:30:07,  8.35s/it]
 55%|█████▌    | 20924/37962 [48:27:52<39:20:19,  8.31s/it]
 55%|█████▌    | 20925/37962 [48:28:00<39:24:07,  8.33s/it]
 55%|█████▌    | 20926/37962 [48:28:09<39:47:08,  8.41s/it]
 55%|█████▌    | 20927/37962 [48:28:17<39:26:09,  8.33s/it]
 55%|█████▌    | 20928/37962 [48:28:25<38:51:51,  8.21s/it]
 55%|█████▌    | 20929/37962 [48:28:33<38:36:48,  8.16s/it]
 55%|█████▌    | 20930/37962 [48:28:41<38:27:34,  8.13s/it]
                                                           
{'loss': 1.1774, 'grad_norm': 7.28328800201416, 'learning_rate': 8.825780407005697e-06, 'epoch': 0.55}

 55%|█████▌    | 20930/37962 [48:28:41<38:27:34,  8.13s/it]
 55%|█████▌    | 20931/37962 [48:28:49<39:01:33,  8.25s/it]
 55%|█████▌    | 20932/37962 [48:28:58<39:09:52,  8.28s/it]
 55%|█████▌    | 20933/37962 [48:29:06<39:31:56,  8.36s/it]
 55%|█████▌    | 20934/37962 [48:29:15<39:35:42,  8.37s/it]
 55%|█████▌    | 20935/37962 [48:29:23<39:30:52,  8.35s/it]
 55%|█████▌    | 20936/37962 [48:29:31<39:35:23,  8.37s/it]
 55%|█████▌    | 20937/37962 [48:29:39<39:03:37,  8.26s/it]
 55%|█████▌    | 20938/37962 [48:29:48<39:16:59,  8.31s/it]
 55%|█████▌    | 20939/37962 [48:29:56<39:06:35,  8.27s/it]
 55%|█████▌    | 20940/37962 [48:30:04<39:13:07,  8.29s/it]
                                                           
{'loss': 1.1998, 'grad_norm': 3.793949604034424, 'learning_rate': 8.817308251747782e-06, 'epoch': 0.55}

 55%|█████▌    | 20940/37962 [48:30:04<39:13:07,  8.29s/it]
 55%|█████▌    | 20941/37962 [48:30:13<39:18:27,  8.31s/it]
 55%|█████▌    | 20942/37962 [48:30:21<39:15:07,  8.30s/it]
 55%|█████▌    | 20943/37962 [48:30:29<39:06:30,  8.27s/it]
 55%|█████▌    | 20944/37962 [48:30:38<39:17:09,  8.31s/it]
 55%|█████▌    | 20945/37962 [48:30:46<38:57:30,  8.24s/it]
 55%|█████▌    | 20946/37962 [48:30:54<38:57:26,  8.24s/it]
 55%|█████▌    | 20947/37962 [48:31:02<39:10:03,  8.29s/it]
 55%|█████▌    | 20948/37962 [48:31:11<39:04:54,  8.27s/it]
 55%|█████▌    | 20949/37962 [48:31:19<39:08:54,  8.28s/it]
 55%|█████▌    | 20950/37962 [48:31:27<39:13:41,  8.30s/it]
                                                           
{'loss': 1.245, 'grad_norm': 7.004604339599609, 'learning_rate': 8.8088369573507e-06, 'epoch': 0.55}

 55%|█████▌    | 20950/37962 [48:31:27<39:13:41,  8.30s/it]
 55%|█████▌    | 20951/37962 [48:31:36<39:23:13,  8.34s/it]
 55%|█████▌    | 20952/37962 [48:31:44<39:18:33,  8.32s/it]
 55%|█████▌    | 20953/37962 [48:31:52<39:17:13,  8.32s/it]
 55%|█████▌    | 20954/37962 [48:32:01<39:12:54,  8.30s/it]
 55%|█████▌    | 20955/37962 [48:32:09<39:27:28,  8.35s/it]
 55%|█████▌    | 20956/37962 [48:32:17<39:20:04,  8.33s/it]
 55%|█████▌    | 20957/37962 [48:32:25<39:12:30,  8.30s/it]
 55%|█████▌    | 20958/37962 [48:32:34<39:19:35,  8.33s/it]
 55%|█████▌    | 20959/37962 [48:32:42<39:20:26,  8.33s/it]
 55%|█████▌    | 20960/37962 [48:32:50<39:11:48,  8.30s/it]
                                                           
{'loss': 1.4016, 'grad_norm': 13.110543251037598, 'learning_rate': 8.800366529980563e-06, 'epoch': 0.55}

 55%|█████▌    | 20960/37962 [48:32:50<39:11:48,  8.30s/it]
 55%|█████▌    | 20961/37962 [48:32:59<39:19:50,  8.33s/it]
 55%|█████▌    | 20962/37962 [48:33:07<39:00:11,  8.26s/it]
 55%|█████▌    | 20963/37962 [48:33:15<39:01:03,  8.26s/it]
 55%|█████▌    | 20964/37962 [48:33:23<38:54:05,  8.24s/it]
 55%|█████▌    | 20965/37962 [48:33:32<38:59:53,  8.26s/it]
 55%|█████▌    | 20966/37962 [48:33:40<39:03:33,  8.27s/it]
 55%|█████▌    | 20967/37962 [48:33:49<39:29:17,  8.36s/it]
 55%|█████▌    | 20968/37962 [48:33:57<39:15:15,  8.32s/it]
 55%|█████▌    | 20969/37962 [48:34:05<39:10:01,  8.30s/it]
 55%|█████▌    | 20970/37962 [48:34:13<38:39:02,  8.19s/it]
                                                           
{'loss': 1.2877, 'grad_norm': 3.2150449752807617, 'learning_rate': 8.791896975802843e-06, 'epoch': 0.55}

 55%|█████▌    | 20970/37962 [48:34:13<38:39:02,  8.19s/it]
 55%|█████▌    | 20971/37962 [48:34:21<38:45:43,  8.21s/it]
 55%|█████▌    | 20972/37962 [48:34:30<38:56:46,  8.25s/it]
 55%|█████▌    | 20973/37962 [48:34:38<39:06:46,  8.29s/it]
 55%|█████▌    | 20974/37962 [48:34:46<39:04:31,  8.28s/it]
 55%|█████▌    | 20975/37962 [48:34:54<39:00:17,  8.27s/it]
 55%|█████▌    | 20976/37962 [48:35:03<38:59:12,  8.26s/it]
 55%|█████▌    | 20977/37962 [48:35:11<39:00:55,  8.27s/it]
 55%|█████▌    | 20978/37962 [48:35:20<39:24:29,  8.35s/it]
 55%|█████▌    | 20979/37962 [48:35:28<39:13:57,  8.32s/it]
 55%|█████▌    | 20980/37962 [48:35:36<39:01:10,  8.27s/it]
                                                           
{'loss': 1.2929, 'grad_norm': 13.143166542053223, 'learning_rate': 8.783428300982393e-06, 'epoch': 0.55}

 55%|█████▌    | 20980/37962 [48:35:36<39:01:10,  8.27s/it]
 55%|█████▌    | 20981/37962 [48:35:44<39:00:57,  8.27s/it]
 55%|█████▌    | 20982/37962 [48:35:52<38:55:28,  8.25s/it]
 55%|█████▌    | 20983/37962 [48:36:00<38:37:06,  8.19s/it]
 55%|█████▌    | 20984/37962 [48:36:08<38:13:41,  8.11s/it]
 55%|█████▌    | 20985/37962 [48:36:17<38:32:02,  8.17s/it]
 55%|█████▌    | 20986/37962 [48:36:25<38:09:51,  8.09s/it]
 55%|█████▌    | 20987/37962 [48:36:33<38:30:28,  8.17s/it]
 55%|█████▌    | 20988/37962 [48:36:41<38:41:46,  8.21s/it]
 55%|█████▌    | 20989/37962 [48:36:49<38:24:27,  8.15s/it]
 55%|█████▌    | 20990/37962 [48:36:57<38:03:34,  8.07s/it]
                                                           
{'loss': 1.1662, 'grad_norm': 3.1480088233947754, 'learning_rate': 8.774960511683402e-06, 'epoch': 0.55}

 55%|█████▌    | 20990/37962 [48:36:57<38:03:34,  8.07s/it]
 55%|█████▌    | 20991/37962 [48:37:05<38:12:50,  8.11s/it]
 55%|█████▌    | 20992/37962 [48:37:13<38:07:34,  8.09s/it]
 55%|█████▌    | 20993/37962 [48:37:22<38:15:50,  8.12s/it]
 55%|█████▌    | 20994/37962 [48:37:30<38:43:01,  8.21s/it]
 55%|█████▌    | 20995/37962 [48:37:38<38:32:45,  8.18s/it]
 55%|█████▌    | 20996/37962 [48:37:46<38:40:51,  8.21s/it]
 55%|█████▌    | 20997/37962 [48:37:54<38:25:50,  8.16s/it]
 55%|█████▌    | 20998/37962 [48:38:02<37:56:24,  8.05s/it]
 55%|█████▌    | 20999/37962 [48:38:10<37:52:15,  8.04s/it]
 55%|█████▌    | 21000/37962 [48:38:18<38:07:14,  8.09s/it]
                                                           
{'loss': 1.2509, 'grad_norm': 11.486973762512207, 'learning_rate': 8.76649361406943e-06, 'epoch': 0.55}

 55%|█████▌    | 21000/37962 [48:38:18<38:07:14,  8.09s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 55%|█████▌    | 21001/37962 [48:39:10<99:11:34, 21.05s/it]
 55%|█████▌    | 21002/37962 [48:39:18<81:16:31, 17.25s/it]
 55%|█████▌    | 21003/37962 [48:39:27<68:56:01, 14.63s/it]
 55%|█████▌    | 21004/37962 [48:39:35<59:50:26, 12.70s/it]
 55%|█████▌    | 21005/37962 [48:39:43<53:31:15, 11.36s/it]
 55%|█████▌    | 21006/37962 [48:39:51<49:08:09, 10.43s/it]
 55%|█████▌    | 21007/37962 [48:40:00<46:08:39,  9.80s/it]
 55%|█████▌    | 21008/37962 [48:40:08<44:10:54,  9.38s/it]
 55%|█████▌    | 21009/37962 [48:40:16<42:29:09,  9.02s/it]
 55%|█████▌    | 21010/37962 [48:40:24<40:56:54,  8.70s/it]
                                                           
{'loss': 1.3359, 'grad_norm': 7.297770977020264, 'learning_rate': 8.75802761430339e-06, 'epoch': 0.55}

 55%|█████▌    | 21010/37962 [48:40:24<40:56:54,  8.70s/it]
 55%|█████▌    | 21011/37962 [48:40:33<40:31:39,  8.61s/it]
 55%|█████▌    | 21012/37962 [48:40:41<40:21:54,  8.57s/it]
 55%|█████▌    | 21013/37962 [48:40:49<39:46:37,  8.45s/it]
 55%|█████▌    | 21014/37962 [48:40:58<39:40:52,  8.43s/it]
 55%|█████▌    | 21015/37962 [48:41:06<39:33:27,  8.40s/it]
 55%|█████▌    | 21016/37962 [48:41:14<39:40:36,  8.43s/it]
 55%|█████▌    | 21017/37962 [48:41:22<38:49:33,  8.25s/it]
 55%|█████▌    | 21018/37962 [48:41:31<39:06:10,  8.31s/it]
 55%|█████▌    | 21019/37962 [48:41:39<39:17:08,  8.35s/it]
 55%|█████▌    | 21020/37962 [48:41:47<39:06:31,  8.31s/it]
                                                           
{'loss': 1.2418, 'grad_norm': 3.477478504180908, 'learning_rate': 8.749562518547532e-06, 'epoch': 0.55}

 55%|█████▌    | 21020/37962 [48:41:47<39:06:31,  8.31s/it]
 55%|█████▌    | 21021/37962 [48:41:56<38:57:42,  8.28s/it]
 55%|█████▌    | 21022/37962 [48:42:04<38:58:44,  8.28s/it]
 55%|█████▌    | 21023/37962 [48:42:12<38:25:46,  8.17s/it]
 55%|█████▌    | 21024/37962 [48:42:20<38:39:18,  8.22s/it]
 55%|█████▌    | 21025/37962 [48:42:28<38:17:14,  8.14s/it]
 55%|█████▌    | 21026/37962 [48:42:36<38:06:09,  8.10s/it]
 55%|█████▌    | 21027/37962 [48:42:44<38:24:08,  8.16s/it]
 55%|█████▌    | 21028/37962 [48:42:53<38:50:31,  8.26s/it]
 55%|█████▌    | 21029/37962 [48:43:01<38:50:45,  8.26s/it]
 55%|█████▌    | 21030/37962 [48:43:10<39:01:27,  8.30s/it]
                                                           
{'loss': 1.3224, 'grad_norm': 6.981030464172363, 'learning_rate': 8.741098332963459e-06, 'epoch': 0.55}

 55%|█████▌    | 21030/37962 [48:43:10<39:01:27,  8.30s/it]
 55%|█████▌    | 21031/37962 [48:43:18<38:59:18,  8.29s/it]
 55%|█████▌    | 21032/37962 [48:43:26<39:12:19,  8.34s/it]
 55%|█████▌    | 21033/37962 [48:43:35<39:41:05,  8.44s/it]
 55%|█████▌    | 21034/37962 [48:43:43<39:20:37,  8.37s/it]
 55%|█████▌    | 21035/37962 [48:43:51<38:39:37,  8.22s/it]
 55%|█████▌    | 21036/37962 [48:43:59<38:53:13,  8.27s/it]
 55%|█████▌    | 21037/37962 [48:44:08<38:51:01,  8.26s/it]
 55%|█████▌    | 21038/37962 [48:44:16<39:03:24,  8.31s/it]
 55%|█████▌    | 21039/37962 [48:44:24<38:37:07,  8.22s/it]
 55%|█████▌    | 21040/37962 [48:44:32<38:45:22,  8.25s/it]
                                                           
{'loss': 1.2956, 'grad_norm': 7.039678573608398, 'learning_rate': 8.732635063712096e-06, 'epoch': 0.55}

 55%|█████▌    | 21040/37962 [48:44:32<38:45:22,  8.25s/it]
 55%|█████▌    | 21041/37962 [48:44:41<38:58:20,  8.29s/it]
 55%|█████▌    | 21042/37962 [48:44:49<39:00:50,  8.30s/it]
 55%|█████▌    | 21043/37962 [48:44:57<39:03:21,  8.31s/it]
 55%|█████▌    | 21044/37962 [48:45:06<39:05:33,  8.32s/it]
 55%|█████▌    | 21045/37962 [48:45:14<39:02:38,  8.31s/it]
 55%|█████▌    | 21046/37962 [48:45:22<39:00:07,  8.30s/it]
 55%|█████▌    | 21047/37962 [48:45:31<38:55:33,  8.28s/it]
 55%|█████▌    | 21048/37962 [48:45:38<38:13:52,  8.14s/it]
 55%|█████▌    | 21049/37962 [48:45:47<38:23:28,  8.17s/it]
 55%|█████▌    | 21050/37962 [48:45:55<38:41:02,  8.23s/it]
                                                           
{'loss': 1.2812, 'grad_norm': 3.488987922668457, 'learning_rate': 8.724172716953717e-06, 'epoch': 0.55}

 55%|█████▌    | 21050/37962 [48:45:55<38:41:02,  8.23s/it]
 55%|█████▌    | 21051/37962 [48:46:03<38:55:12,  8.29s/it]
 55%|█████▌    | 21052/37962 [48:46:12<38:48:05,  8.26s/it]
 55%|█████▌    | 21053/37962 [48:46:20<38:36:02,  8.22s/it]
 55%|█████▌    | 21054/37962 [48:46:28<38:40:07,  8.23s/it]
 55%|█████▌    | 21055/37962 [48:46:36<38:42:00,  8.24s/it]
 55%|█████▌    | 21056/37962 [48:46:45<38:45:11,  8.25s/it]
 55%|█████▌    | 21057/37962 [48:46:52<38:15:58,  8.15s/it]
 55%|█████▌    | 21058/37962 [48:47:01<38:41:56,  8.24s/it]
 55%|█████▌    | 21059/37962 [48:47:09<38:40:12,  8.24s/it]
 55%|█████▌    | 21060/37962 [48:47:17<38:50:28,  8.27s/it]
                                                           
{'loss': 1.3684, 'grad_norm': 9.06523323059082, 'learning_rate': 8.715711298847913e-06, 'epoch': 0.55}

 55%|█████▌    | 21060/37962 [48:47:17<38:50:28,  8.27s/it]
 55%|█████▌    | 21061/37962 [48:47:26<38:52:50,  8.28s/it]
 55%|█████▌    | 21062/37962 [48:47:34<38:45:01,  8.25s/it]
 55%|█████▌    | 21063/37962 [48:47:42<38:51:36,  8.28s/it]
 55%|█████▌    | 21064/37962 [48:47:51<38:55:09,  8.29s/it]
 55%|█████▌    | 21065/37962 [48:47:59<39:16:51,  8.37s/it]
 55%|█████▌    | 21066/37962 [48:48:08<39:22:42,  8.39s/it]
 55%|█████▌    | 21067/37962 [48:48:16<38:48:44,  8.27s/it]
 55%|█████▌    | 21068/37962 [48:48:23<38:15:23,  8.15s/it]
 56%|█████▌    | 21069/37962 [48:48:32<38:15:55,  8.15s/it]
 56%|█████▌    | 21070/37962 [48:48:40<38:31:42,  8.21s/it]
                                                           
{'loss': 1.3058, 'grad_norm': 6.24688196182251, 'learning_rate': 8.707250815553605e-06, 'epoch': 0.56}

 56%|█████▌    | 21070/37962 [48:48:40<38:31:42,  8.21s/it]
 56%|█████▌    | 21071/37962 [48:48:48<38:54:08,  8.29s/it]
 56%|█████▌    | 21072/37962 [48:48:57<38:59:23,  8.31s/it]
 56%|█████▌    | 21073/37962 [48:49:05<39:14:56,  8.37s/it]
 56%|█████▌    | 21074/37962 [48:49:14<39:19:51,  8.38s/it]
 56%|█████▌    | 21075/37962 [48:49:22<38:51:59,  8.29s/it]
 56%|█████▌    | 21076/37962 [48:49:30<38:55:20,  8.30s/it]
 56%|█████▌    | 21077/37962 [48:49:39<39:03:09,  8.33s/it]
 56%|█████▌    | 21078/37962 [48:49:47<39:18:19,  8.38s/it]
 56%|█████▌    | 21079/37962 [48:49:55<39:06:44,  8.34s/it]
 56%|█████▌    | 21080/37962 [48:50:03<38:55:30,  8.30s/it]
                                                           
{'loss': 1.1723, 'grad_norm': 3.984243154525757, 'learning_rate': 8.69879127322904e-06, 'epoch': 0.56}

 56%|█████▌    | 21080/37962 [48:50:03<38:55:30,  8.30s/it]
 56%|█████▌    | 21081/37962 [48:50:12<38:47:26,  8.27s/it]
 56%|█████▌    | 21082/37962 [48:50:20<38:47:09,  8.27s/it]
 56%|█████▌    | 21083/37962 [48:50:28<38:55:16,  8.30s/it]
 56%|█████▌    | 21084/37962 [48:50:37<38:57:27,  8.31s/it]
 56%|█████▌    | 21085/37962 [48:50:45<38:49:06,  8.28s/it]
 56%|█████▌    | 21086/37962 [48:50:53<38:13:25,  8.15s/it]
 56%|█████▌    | 21087/37962 [48:51:01<38:00:42,  8.11s/it]
 56%|█████▌    | 21088/37962 [48:51:09<38:23:02,  8.19s/it]
 56%|█████▌    | 21089/37962 [48:51:17<38:24:20,  8.19s/it]
 56%|█████▌    | 21090/37962 [48:51:26<38:26:19,  8.20s/it]
                                                           
{'loss': 1.3182, 'grad_norm': 3.7519986629486084, 'learning_rate': 8.69033267803176e-06, 'epoch': 0.56}

 56%|█████▌    | 21090/37962 [48:51:26<38:26:19,  8.20s/it]
 56%|█████▌    | 21091/37962 [48:51:34<38:15:45,  8.16s/it]
 56%|█████▌    | 21092/37962 [48:51:42<38:28:48,  8.21s/it]
 56%|█████▌    | 21093/37962 [48:51:50<38:34:25,  8.23s/it]
 56%|█████▌    | 21094/37962 [48:51:58<38:24:52,  8.20s/it]
 56%|█████▌    | 21095/37962 [48:52:07<38:34:40,  8.23s/it]
 56%|█████▌    | 21096/37962 [48:52:15<38:37:49,  8.25s/it]
 56%|█████▌    | 21097/37962 [48:52:23<38:54:47,  8.31s/it]
 56%|█████▌    | 21098/37962 [48:52:32<38:50:26,  8.29s/it]
 56%|█████▌    | 21099/37962 [48:52:40<38:44:10,  8.27s/it]
 56%|█████▌    | 21100/37962 [48:52:48<38:41:21,  8.26s/it]
                                                           
{'loss': 1.3108, 'grad_norm': 9.409341812133789, 'learning_rate': 8.681875036118632e-06, 'epoch': 0.56}

 56%|█████▌    | 21100/37962 [48:52:48<38:41:21,  8.26s/it]
 56%|█████▌    | 21101/37962 [48:52:56<38:34:03,  8.23s/it]
 56%|█████▌    | 21102/37962 [48:53:04<38:31:25,  8.23s/it]
 56%|█████▌    | 21103/37962 [48:53:13<38:35:48,  8.24s/it]
 56%|█████▌    | 21104/37962 [48:53:21<38:49:38,  8.29s/it]
 56%|█████▌    | 21105/37962 [48:53:29<38:51:58,  8.30s/it]
 56%|█████▌    | 21106/37962 [48:53:38<38:55:41,  8.31s/it]
 56%|█████▌    | 21107/37962 [48:53:46<39:14:11,  8.38s/it]
 56%|█████▌    | 21108/37962 [48:53:55<39:06:57,  8.36s/it]
 56%|█████▌    | 21109/37962 [48:54:03<38:59:18,  8.33s/it]
 56%|█████▌    | 21110/37962 [48:54:11<39:03:26,  8.34s/it]
                                                           
{'loss': 1.288, 'grad_norm': 8.394383430480957, 'learning_rate': 8.673418353645834e-06, 'epoch': 0.56}

 56%|█████▌    | 21110/37962 [48:54:11<39:03:26,  8.34s/it]
 56%|█████▌    | 21111/37962 [48:54:19<38:39:30,  8.26s/it]
 56%|█████▌    | 21112/37962 [48:54:28<38:40:18,  8.26s/it]
 56%|█████▌    | 21113/37962 [48:54:36<38:53:19,  8.31s/it]
 56%|█████▌    | 21114/37962 [48:54:44<38:48:55,  8.29s/it]
 56%|█████▌    | 21115/37962 [48:54:52<38:36:14,  8.25s/it]
 56%|█████▌    | 21116/37962 [48:55:01<38:33:31,  8.24s/it]
 56%|█████▌    | 21117/37962 [48:55:17<49:32:55, 10.59s/it]
 56%|█████▌    | 21118/37962 [48:55:25<46:22:01,  9.91s/it]
 56%|█████▌    | 21119/37962 [48:55:33<44:18:09,  9.47s/it]
 56%|█████▌    | 21120/37962 [48:55:42<42:37:56,  9.11s/it]
                                                           
{'loss': 1.3643, 'grad_norm': 9.494107246398926, 'learning_rate': 8.664962636768834e-06, 'epoch': 0.56}

 56%|█████▌    | 21120/37962 [48:55:42<42:37:56,  9.11s/it]
 56%|█████▌    | 21121/37962 [48:55:50<41:32:54,  8.88s/it]
 56%|█████▌    | 21122/37962 [48:55:58<40:45:51,  8.71s/it]
 56%|█████▌    | 21123/37962 [48:56:14<50:10:01, 10.73s/it]
 56%|█████▌    | 21124/37962 [48:56:22<46:47:07, 10.00s/it]
 56%|█████▌    | 21125/37962 [48:56:30<44:21:07,  9.48s/it]
 56%|█████▌    | 21126/37962 [48:56:39<42:29:07,  9.08s/it]
 56%|█████▌    | 21127/37962 [48:56:47<41:18:59,  8.84s/it]
 56%|█████▌    | 21128/37962 [48:56:55<40:08:09,  8.58s/it]
 56%|█████▌    | 21129/37962 [48:57:03<39:07:08,  8.37s/it]
 56%|█████▌    | 21130/37962 [48:57:11<38:30:29,  8.24s/it]
                                                           
{'loss': 1.4254, 'grad_norm': 4.449470520019531, 'learning_rate': 8.656507891642405e-06, 'epoch': 0.56}

 56%|█████▌    | 21130/37962 [48:57:11<38:30:29,  8.24s/it]
 56%|█████▌    | 21131/37962 [48:57:19<38:31:38,  8.24s/it]
 56%|█████▌    | 21132/37962 [48:57:27<38:19:24,  8.20s/it]
 56%|█████▌    | 21133/37962 [48:57:35<38:29:56,  8.24s/it]
 56%|█████▌    | 21134/37962 [48:57:43<38:22:47,  8.21s/it]
 56%|█████▌    | 21135/37962 [48:57:52<38:08:03,  8.16s/it]
 56%|█████▌    | 21136/37962 [48:58:00<38:35:51,  8.26s/it]
 56%|█████▌    | 21137/37962 [48:58:08<38:23:00,  8.21s/it]
 56%|█████▌    | 21138/37962 [48:58:16<38:33:51,  8.25s/it]
 56%|█████▌    | 21139/37962 [48:58:25<38:36:51,  8.26s/it]
 56%|█████▌    | 21140/37962 [48:58:33<38:43:29,  8.29s/it]
                                                           
{'loss': 1.3365, 'grad_norm': 5.849430561065674, 'learning_rate': 8.648054124420606e-06, 'epoch': 0.56}

 56%|█████▌    | 21140/37962 [48:58:33<38:43:29,  8.29s/it]
 56%|█████▌    | 21141/37962 [48:58:41<38:38:42,  8.27s/it]
 56%|█████▌    | 21142/37962 [48:58:50<38:46:06,  8.30s/it]
 56%|█████▌    | 21143/37962 [48:58:58<38:42:22,  8.28s/it]
 56%|█████▌    | 21144/37962 [48:59:06<38:42:08,  8.28s/it]
 56%|█████▌    | 21145/37962 [48:59:14<38:39:40,  8.28s/it]
 56%|█████▌    | 21146/37962 [48:59:23<38:40:12,  8.28s/it]
 56%|█████▌    | 21147/37962 [48:59:31<38:48:02,  8.31s/it]
 56%|█████▌    | 21148/37962 [48:59:40<38:58:24,  8.34s/it]
 56%|█████▌    | 21149/37962 [48:59:48<39:18:31,  8.42s/it]
 56%|█████▌    | 21150/37962 [48:59:56<38:59:25,  8.35s/it]
                                                           
{'loss': 1.2583, 'grad_norm': 9.65716552734375, 'learning_rate': 8.639601341256788e-06, 'epoch': 0.56}

 56%|█████▌    | 21150/37962 [48:59:56<38:59:25,  8.35s/it]
 56%|█████▌    | 21151/37962 [49:00:05<38:49:29,  8.31s/it]
 56%|█████▌    | 21152/37962 [49:00:13<38:42:20,  8.29s/it]
 56%|█████▌    | 21153/37962 [49:00:21<38:32:48,  8.26s/it]
 56%|█████▌    | 21154/37962 [49:00:29<38:31:49,  8.25s/it]
 56%|█████▌    | 21155/37962 [49:00:38<38:39:14,  8.28s/it]
 56%|█████▌    | 21156/37962 [49:00:46<38:34:34,  8.26s/it]
 56%|█████▌    | 21157/37962 [49:00:54<38:46:03,  8.30s/it]
 56%|█████▌    | 21158/37962 [49:01:02<38:24:34,  8.23s/it]
 56%|█████▌    | 21159/37962 [49:01:11<38:28:37,  8.24s/it]
 56%|█████▌    | 21160/37962 [49:01:19<38:29:39,  8.25s/it]
                                                           
{'loss': 1.3508, 'grad_norm': 10.847428321838379, 'learning_rate': 8.631149548303588e-06, 'epoch': 0.56}

 56%|█████▌    | 21160/37962 [49:01:19<38:29:39,  8.25s/it]
 56%|█████▌    | 21161/37962 [49:01:27<38:18:31,  8.21s/it]
 56%|█████▌    | 21162/37962 [49:01:35<38:27:01,  8.24s/it]
 56%|█████▌    | 21163/37962 [49:01:44<38:39:59,  8.29s/it]
 56%|█████▌    | 21164/37962 [49:01:52<38:37:15,  8.28s/it]
 56%|█████▌    | 21165/37962 [49:02:00<38:41:09,  8.29s/it]
 56%|█████▌    | 21166/37962 [49:02:09<39:06:51,  8.38s/it]
 56%|█████▌    | 21167/37962 [49:02:17<38:54:34,  8.34s/it]
 56%|█████▌    | 21168/37962 [49:02:25<38:44:19,  8.30s/it]
 56%|█████▌    | 21169/37962 [49:02:34<39:01:35,  8.37s/it]
 56%|█████▌    | 21170/37962 [49:02:42<38:35:38,  8.27s/it]
                                                           
{'loss': 1.2268, 'grad_norm': 9.496957778930664, 'learning_rate': 8.622698751712915e-06, 'epoch': 0.56}

 56%|█████▌    | 21170/37962 [49:02:42<38:35:38,  8.27s/it]
 56%|█████▌    | 21171/37962 [49:02:50<38:39:50,  8.29s/it]
 56%|█████▌    | 21172/37962 [49:02:58<38:05:41,  8.17s/it]
 56%|█████▌    | 21173/37962 [49:03:06<38:17:43,  8.21s/it]
 56%|█████▌    | 21174/37962 [49:03:14<38:06:58,  8.17s/it]
 56%|█████▌    | 21175/37962 [49:03:23<38:07:26,  8.18s/it]
 56%|█████▌    | 21176/37962 [49:03:31<38:27:51,  8.25s/it]
 56%|█████▌    | 21177/37962 [49:03:39<38:15:38,  8.21s/it]
 56%|█████▌    | 21178/37962 [49:03:47<38:27:27,  8.25s/it]
 56%|█████▌    | 21179/37962 [49:03:56<38:20:27,  8.22s/it]
 56%|█████▌    | 21180/37962 [49:04:04<38:35:17,  8.28s/it]
                                                           
{'loss': 1.1914, 'grad_norm': 10.514594078063965, 'learning_rate': 8.614248957635961e-06, 'epoch': 0.56}

 56%|█████▌    | 21180/37962 [49:04:04<38:35:17,  8.28s/it]
 56%|█████▌    | 21181/37962 [49:04:12<38:12:31,  8.20s/it]
 56%|█████▌    | 21182/37962 [49:04:20<38:20:30,  8.23s/it]
 56%|█████▌    | 21183/37962 [49:04:29<38:34:23,  8.28s/it]
 56%|█████▌    | 21184/37962 [49:04:37<38:42:38,  8.31s/it]
 56%|█████▌    | 21185/37962 [49:04:45<38:48:54,  8.33s/it]
 56%|█████▌    | 21186/37962 [49:04:54<38:55:49,  8.35s/it]
 56%|█████▌    | 21187/37962 [49:05:02<38:51:03,  8.34s/it]
 56%|█████▌    | 21188/37962 [49:05:10<38:32:16,  8.27s/it]
 56%|█████▌    | 21189/37962 [49:05:19<38:36:27,  8.29s/it]
 56%|█████▌    | 21190/37962 [49:05:27<38:03:58,  8.17s/it]
                                                           
{'loss': 1.375, 'grad_norm': 9.496868133544922, 'learning_rate': 8.60580017222319e-06, 'epoch': 0.56}

 56%|█████▌    | 21190/37962 [49:05:27<38:03:58,  8.17s/it]
 56%|█████▌    | 21191/37962 [49:05:35<38:16:17,  8.22s/it]
 56%|█████▌    | 21192/37962 [49:05:43<38:23:35,  8.24s/it]
 56%|█████▌    | 21193/37962 [49:05:52<38:34:38,  8.28s/it]
 56%|█████▌    | 21194/37962 [49:06:00<38:37:40,  8.29s/it]
 56%|█████▌    | 21195/37962 [49:06:09<39:11:14,  8.41s/it]
 56%|█████▌    | 21196/37962 [49:06:17<38:59:17,  8.37s/it]
 56%|█████▌    | 21197/37962 [49:06:25<38:59:06,  8.37s/it]
 56%|█████▌    | 21198/37962 [49:06:34<39:13:27,  8.42s/it]
 56%|█████▌    | 21199/37962 [49:06:42<38:51:45,  8.35s/it]
 56%|█████▌    | 21200/37962 [49:06:50<38:35:14,  8.29s/it]
                                                           
{'loss': 1.406, 'grad_norm': 5.875173091888428, 'learning_rate': 8.597352401624315e-06, 'epoch': 0.56}

 56%|█████▌    | 21200/37962 [49:06:50<38:35:14,  8.29s/it]
 56%|█████▌    | 21201/37962 [49:06:58<38:29:53,  8.27s/it]
 56%|█████▌    | 21202/37962 [49:07:07<38:31:51,  8.28s/it]
 56%|█████▌    | 21203/37962 [49:07:15<38:24:25,  8.25s/it]
 56%|█████▌    | 21204/37962 [49:07:23<38:45:28,  8.33s/it]
 56%|█████▌    | 21205/37962 [49:07:32<38:44:53,  8.32s/it]
 56%|█████▌    | 21206/37962 [49:07:40<38:11:45,  8.21s/it]
 56%|█████▌    | 21207/37962 [49:07:48<38:17:50,  8.23s/it]
 56%|█████▌    | 21208/37962 [49:07:56<38:14:22,  8.22s/it]
 56%|█████▌    | 21209/37962 [49:08:04<38:18:23,  8.23s/it]
 56%|█████▌    | 21210/37962 [49:08:12<38:15:02,  8.22s/it]
                                                           
{'loss': 1.2289, 'grad_norm': 3.749242067337036, 'learning_rate': 8.588905651988328e-06, 'epoch': 0.56}

 56%|█████▌    | 21210/37962 [49:08:12<38:15:02,  8.22s/it]
 56%|█████▌    | 21211/37962 [49:08:20<37:48:35,  8.13s/it]
 56%|█████▌    | 21212/37962 [49:08:29<37:56:03,  8.15s/it]
 56%|█████▌    | 21213/37962 [49:08:37<38:13:53,  8.22s/it]
 56%|█████▌    | 21214/37962 [49:08:45<37:55:29,  8.15s/it]
 56%|█████▌    | 21215/37962 [49:08:53<38:10:45,  8.21s/it]
 56%|█████▌    | 21216/37962 [49:09:02<38:16:55,  8.23s/it]
 56%|█████▌    | 21217/37962 [49:09:10<38:19:02,  8.24s/it]
 56%|█████▌    | 21218/37962 [49:09:18<37:43:58,  8.11s/it]
 56%|█████▌    | 21219/37962 [49:09:26<37:52:32,  8.14s/it]
 56%|█████▌    | 21220/37962 [49:09:34<38:04:29,  8.19s/it]
                                                           
{'loss': 1.2991, 'grad_norm': 12.955780982971191, 'learning_rate': 8.580459929463474e-06, 'epoch': 0.56}

 56%|█████▌    | 21220/37962 [49:09:34<38:04:29,  8.19s/it]
 56%|█████▌    | 21221/37962 [49:09:42<38:07:19,  8.20s/it]
 56%|█████▌    | 21222/37962 [49:09:50<38:01:14,  8.18s/it]
 56%|█████▌    | 21223/37962 [49:09:59<38:03:54,  8.19s/it]
 56%|█████▌    | 21224/37962 [49:10:07<38:24:31,  8.26s/it]
 56%|█████▌    | 21225/37962 [49:10:15<38:23:33,  8.26s/it]
 56%|█████▌    | 21226/37962 [49:10:23<37:47:13,  8.13s/it]
 56%|█████▌    | 21227/37962 [49:10:31<37:58:22,  8.17s/it]
 56%|█████▌    | 21228/37962 [49:10:40<38:02:08,  8.18s/it]
 56%|█████▌    | 21229/37962 [49:10:48<37:59:08,  8.17s/it]
 56%|█████▌    | 21230/37962 [49:10:56<37:47:46,  8.13s/it]
                                                           
{'loss': 1.3229, 'grad_norm': 5.630696773529053, 'learning_rate': 8.572015240197244e-06, 'epoch': 0.56}

 56%|█████▌    | 21230/37962 [49:10:56<37:47:46,  8.13s/it]
 56%|█████▌    | 21231/37962 [49:11:04<37:53:05,  8.15s/it]
 56%|█████▌    | 21232/37962 [49:11:12<37:54:58,  8.16s/it]
 56%|█████▌    | 21233/37962 [49:11:21<38:12:48,  8.22s/it]
 56%|█████▌    | 21234/37962 [49:11:29<38:20:12,  8.25s/it]
 56%|█████▌    | 21235/37962 [49:11:37<38:09:08,  8.21s/it]
 56%|█████▌    | 21236/37962 [49:11:45<38:21:10,  8.25s/it]
 56%|█████▌    | 21237/37962 [49:11:54<38:32:46,  8.30s/it]
 56%|█████▌    | 21238/37962 [49:12:02<38:58:00,  8.39s/it]
 56%|█████▌    | 21239/37962 [49:12:11<38:56:04,  8.38s/it]
 56%|█████▌    | 21240/37962 [49:12:19<38:47:39,  8.35s/it]
                                                           
{'loss': 1.2925, 'grad_norm': 7.698151111602783, 'learning_rate': 8.563571590336383e-06, 'epoch': 0.56}

 56%|█████▌    | 21240/37962 [49:12:19<38:47:39,  8.35s/it]
 56%|█████▌    | 21241/37962 [49:12:27<38:21:59,  8.26s/it]
 56%|█████▌    | 21242/37962 [49:12:35<38:26:50,  8.28s/it]
 56%|█████▌    | 21243/37962 [49:12:44<38:26:10,  8.28s/it]
 56%|█████▌    | 21244/37962 [49:12:52<38:20:30,  8.26s/it]
 56%|█████▌    | 21245/37962 [49:13:00<37:46:29,  8.13s/it]
 56%|█████▌    | 21246/37962 [49:13:08<38:12:49,  8.23s/it]
 56%|█████▌    | 21247/37962 [49:13:17<38:27:38,  8.28s/it]
 56%|█████▌    | 21248/37962 [49:13:25<38:30:46,  8.30s/it]
 56%|█████▌    | 21249/37962 [49:13:33<38:13:33,  8.23s/it]
 56%|█████▌    | 21250/37962 [49:13:41<38:00:52,  8.19s/it]
                                                           
{'loss': 1.2078, 'grad_norm': 10.453741073608398, 'learning_rate': 8.555128986026877e-06, 'epoch': 0.56}

 56%|█████▌    | 21250/37962 [49:13:41<38:00:52,  8.19s/it]
 56%|█████▌    | 21251/37962 [49:13:49<37:54:34,  8.17s/it]
 56%|█████▌    | 21252/37962 [49:13:58<38:07:38,  8.21s/it]
 56%|█████▌    | 21253/37962 [49:14:06<38:17:33,  8.25s/it]
 56%|█████▌    | 21254/37962 [49:14:14<38:06:23,  8.21s/it]
 56%|█████▌    | 21255/37962 [49:14:22<38:13:07,  8.24s/it]
 56%|█████▌    | 21256/37962 [49:14:31<38:19:17,  8.26s/it]
 56%|█████▌    | 21257/37962 [49:14:39<38:32:22,  8.31s/it]
 56%|█████▌    | 21258/37962 [49:14:47<38:33:08,  8.31s/it]
 56%|█████▌    | 21259/37962 [49:14:56<38:45:26,  8.35s/it]
 56%|█████▌    | 21260/37962 [49:15:04<37:59:57,  8.19s/it]
                                                           
{'loss': 1.3802, 'grad_norm': 13.10828971862793, 'learning_rate': 8.546687433413952e-06, 'epoch': 0.56}

 56%|█████▌    | 21260/37962 [49:15:04<37:59:57,  8.19s/it]
 56%|█████▌    | 21261/37962 [49:15:11<37:33:01,  8.09s/it]
 56%|█████▌    | 21262/37962 [49:15:20<37:48:55,  8.15s/it]
 56%|█████▌    | 21263/37962 [49:15:28<37:26:23,  8.07s/it]
 56%|█████▌    | 21264/37962 [49:15:36<38:01:43,  8.20s/it]
 56%|█████▌    | 21265/37962 [49:15:44<38:11:38,  8.23s/it]
 56%|█████▌    | 21266/37962 [49:15:53<38:14:19,  8.25s/it]
 56%|█████▌    | 21267/37962 [49:16:01<38:21:27,  8.27s/it]
 56%|█████▌    | 21268/37962 [49:16:09<38:28:36,  8.30s/it]
 56%|█████▌    | 21269/37962 [49:16:18<38:22:50,  8.28s/it]
 56%|█████▌    | 21270/37962 [49:16:26<38:19:07,  8.26s/it]
                                                           
{'loss': 1.3378, 'grad_norm': 2.817138671875, 'learning_rate': 8.538246938642066e-06, 'epoch': 0.56}

 56%|█████▌    | 21270/37962 [49:16:26<38:19:07,  8.26s/it]
 56%|█████▌    | 21271/37962 [49:16:34<38:37:33,  8.33s/it]
 56%|█████▌    | 21272/37962 [49:16:43<38:38:05,  8.33s/it]
 56%|█████▌    | 21273/37962 [49:16:51<38:44:40,  8.36s/it]
 56%|█████▌    | 21274/37962 [49:17:00<38:56:17,  8.40s/it]
 56%|█████▌    | 21275/37962 [49:17:08<39:08:46,  8.45s/it]
 56%|█████▌    | 21276/37962 [49:17:16<38:40:35,  8.34s/it]
 56%|█████▌    | 21277/37962 [49:17:24<38:27:32,  8.30s/it]
 56%|█████▌    | 21278/37962 [49:17:33<38:15:10,  8.25s/it]
 56%|█████▌    | 21279/37962 [49:17:41<38:23:52,  8.29s/it]
 56%|█████▌    | 21280/37962 [49:17:49<38:32:02,  8.32s/it]
                                                           
{'loss': 1.2919, 'grad_norm': 7.206291675567627, 'learning_rate': 8.529807507854907e-06, 'epoch': 0.56}

 56%|█████▌    | 21280/37962 [49:17:49<38:32:02,  8.32s/it]
 56%|█████▌    | 21281/37962 [49:17:57<37:59:31,  8.20s/it]
 56%|█████▌    | 21282/37962 [49:18:06<38:23:24,  8.29s/it]
 56%|█████▌    | 21283/37962 [49:18:14<38:26:17,  8.30s/it]
 56%|█████▌    | 21284/37962 [49:18:22<38:20:35,  8.28s/it]
 56%|█████▌    | 21285/37962 [49:18:31<38:15:26,  8.26s/it]
 56%|█████▌    | 21286/37962 [49:18:39<38:11:19,  8.24s/it]
 56%|█████▌    | 21287/37962 [49:18:47<38:15:01,  8.26s/it]
 56%|█████▌    | 21288/37962 [49:18:55<38:02:16,  8.21s/it]
 56%|█████▌    | 21289/37962 [49:19:03<37:57:33,  8.20s/it]
 56%|█████▌    | 21290/37962 [49:19:12<37:57:12,  8.20s/it]
                                                           
{'loss': 1.1976, 'grad_norm': 5.667418956756592, 'learning_rate': 8.521369147195403e-06, 'epoch': 0.56}

 56%|█████▌    | 21290/37962 [49:19:12<37:57:12,  8.20s/it]
 56%|█████▌    | 21291/37962 [49:19:20<38:01:59,  8.21s/it]
 56%|█████▌    | 21292/37962 [49:19:28<38:43:06,  8.36s/it]
 56%|█████▌    | 21293/37962 [49:19:37<38:44:42,  8.37s/it]
 56%|█████▌    | 21294/37962 [49:19:45<38:37:03,  8.34s/it]
 56%|█████▌    | 21295/37962 [49:19:53<38:28:04,  8.31s/it]
 56%|█████▌    | 21296/37962 [49:20:02<38:32:51,  8.33s/it]
 56%|█████▌    | 21297/37962 [49:20:10<38:28:23,  8.31s/it]
 56%|█████▌    | 21298/37962 [49:20:18<38:22:08,  8.29s/it]
 56%|█████▌    | 21299/37962 [49:20:26<38:02:34,  8.22s/it]
 56%|█████▌    | 21300/37962 [49:20:35<38:02:33,  8.22s/it]
                                                           
{'loss': 1.2713, 'grad_norm': 11.784053802490234, 'learning_rate': 8.512931862805674e-06, 'epoch': 0.56}

 56%|█████▌    | 21300/37962 [49:20:35<38:02:33,  8.22s/it]
 56%|█████▌    | 21301/37962 [49:20:43<38:19:27,  8.28s/it]
 56%|█████▌    | 21302/37962 [49:20:51<38:16:13,  8.27s/it]
 56%|█████▌    | 21303/37962 [49:20:59<38:14:10,  8.26s/it]
 56%|█████▌    | 21304/37962 [49:21:15<48:42:44, 10.53s/it]
 56%|█████▌    | 21305/37962 [49:21:24<46:05:02,  9.96s/it]
 56%|█████▌    | 21306/37962 [49:21:32<43:20:34,  9.37s/it]
 56%|█████▌    | 21307/37962 [49:21:40<41:42:03,  9.01s/it]
 56%|█████▌    | 21308/37962 [49:21:48<40:41:31,  8.80s/it]
 56%|█████▌    | 21309/37962 [49:21:57<40:03:48,  8.66s/it]
 56%|█████▌    | 21310/37962 [49:22:05<39:29:07,  8.54s/it]
                                                           
{'loss': 1.1995, 'grad_norm': 4.39980411529541, 'learning_rate': 8.504495660827081e-06, 'epoch': 0.56}

 56%|█████▌    | 21310/37962 [49:22:05<39:29:07,  8.54s/it]
 56%|█████▌    | 21311/37962 [49:22:13<38:27:29,  8.31s/it]
 56%|█████▌    | 21312/37962 [49:22:21<38:41:39,  8.37s/it]
 56%|█████▌    | 21313/37962 [49:22:30<38:44:06,  8.38s/it]
 56%|█████▌    | 21314/37962 [49:22:38<38:32:18,  8.33s/it]
 56%|█████▌    | 21315/37962 [49:22:46<38:28:16,  8.32s/it]
 56%|█████▌    | 21316/37962 [49:22:54<38:18:28,  8.28s/it]
 56%|█████▌    | 21317/37962 [49:23:03<38:29:44,  8.33s/it]
 56%|█████▌    | 21318/37962 [49:23:11<38:34:23,  8.34s/it]
 56%|█████▌    | 21319/37962 [49:23:19<37:58:19,  8.21s/it]
 56%|█████▌    | 21320/37962 [49:23:27<38:02:37,  8.23s/it]
                                                           
{'loss': 1.2898, 'grad_norm': 11.938528060913086, 'learning_rate': 8.49606054740019e-06, 'epoch': 0.56}

 56%|█████▌    | 21320/37962 [49:23:27<38:02:37,  8.23s/it]
 56%|█████▌    | 21321/37962 [49:23:36<38:28:18,  8.32s/it]
 56%|█████▌    | 21322/37962 [49:23:44<38:28:13,  8.32s/it]
 56%|█████▌    | 21323/37962 [49:23:52<38:20:13,  8.29s/it]
 56%|█████▌    | 21324/37962 [49:24:01<38:02:03,  8.23s/it]
 56%|█████▌    | 21325/37962 [49:24:09<38:20:25,  8.30s/it]
 56%|█████▌    | 21326/37962 [49:24:17<37:46:17,  8.17s/it]
 56%|█████▌    | 21327/37962 [49:24:25<37:52:40,  8.20s/it]
 56%|█████▌    | 21328/37962 [49:24:33<37:35:55,  8.14s/it]
 56%|█████▌    | 21329/37962 [49:24:42<38:05:03,  8.24s/it]
 56%|█████▌    | 21330/37962 [49:24:50<38:14:14,  8.28s/it]
                                                           
{'loss': 1.2358, 'grad_norm': 4.576653957366943, 'learning_rate': 8.487626528664771e-06, 'epoch': 0.56}

 56%|█████▌    | 21330/37962 [49:24:50<38:14:14,  8.28s/it]
 56%|█████▌    | 21331/37962 [49:24:58<38:16:52,  8.29s/it]
 56%|█████▌    | 21332/37962 [49:25:07<38:15:46,  8.28s/it]
 56%|█████▌    | 21333/37962 [49:25:15<38:16:34,  8.29s/it]
 56%|█████▌    | 21334/37962 [49:25:23<38:08:14,  8.26s/it]
 56%|█████▌    | 21335/37962 [49:25:32<38:27:57,  8.33s/it]
 56%|█████▌    | 21336/37962 [49:25:40<38:27:08,  8.33s/it]
 56%|█████▌    | 21337/37962 [49:25:48<38:13:25,  8.28s/it]
 56%|█████▌    | 21338/37962 [49:25:56<38:10:51,  8.27s/it]
 56%|█████▌    | 21339/37962 [49:26:04<38:08:39,  8.26s/it]
 56%|█████▌    | 21340/37962 [49:26:13<38:08:37,  8.26s/it]
                                                           
{'loss': 1.2551, 'grad_norm': 3.9316813945770264, 'learning_rate': 8.479193610759804e-06, 'epoch': 0.56}

 56%|█████▌    | 21340/37962 [49:26:13<38:08:37,  8.26s/it]
 56%|█████▌    | 21341/37962 [49:26:21<38:04:37,  8.25s/it]
 56%|█████▌    | 21342/37962 [49:26:29<37:54:22,  8.21s/it]
 56%|█████▌    | 21343/37962 [49:26:37<37:16:01,  8.07s/it]
 56%|█████▌    | 21344/37962 [49:26:45<37:39:39,  8.16s/it]
 56%|█████▌    | 21345/37962 [49:26:54<37:55:07,  8.21s/it]
 56%|█████▌    | 21346/37962 [49:27:02<38:07:21,  8.26s/it]
 56%|█████▌    | 21347/37962 [49:27:10<38:05:24,  8.25s/it]
 56%|█████▌    | 21348/37962 [49:27:18<38:02:45,  8.24s/it]
 56%|█████▌    | 21349/37962 [49:27:27<38:01:21,  8.24s/it]
 56%|█████▌    | 21350/37962 [49:27:35<38:30:13,  8.34s/it]
                                                           
{'loss': 1.3201, 'grad_norm': 4.396370887756348, 'learning_rate': 8.470761799823461e-06, 'epoch': 0.56}

 56%|█████▌    | 21350/37962 [49:27:35<38:30:13,  8.34s/it]
 56%|█████▌    | 21351/37962 [49:27:43<38:17:07,  8.30s/it]
 56%|█████▌    | 21352/37962 [49:27:52<38:17:45,  8.30s/it]
 56%|█████▌    | 21353/37962 [49:28:00<38:51:28,  8.42s/it]
 56%|█████▋    | 21354/37962 [49:28:09<38:51:25,  8.42s/it]
 56%|█████▋    | 21355/37962 [49:28:17<38:50:41,  8.42s/it]
 56%|█████▋    | 21356/37962 [49:28:25<38:37:26,  8.37s/it]
 56%|█████▋    | 21357/37962 [49:28:34<38:25:27,  8.33s/it]
 56%|█████▋    | 21358/37962 [49:28:42<38:15:47,  8.30s/it]
 56%|█████▋    | 21359/37962 [49:28:50<38:22:31,  8.32s/it]
 56%|█████▋    | 21360/37962 [49:28:58<38:09:19,  8.27s/it]
                                                           
{'loss': 1.274, 'grad_norm': 9.86244010925293, 'learning_rate': 8.462331101993111e-06, 'epoch': 0.56}

 56%|█████▋    | 21360/37962 [49:28:58<38:09:19,  8.27s/it]
 56%|█████▋    | 21361/37962 [49:29:07<38:24:34,  8.33s/it]
 56%|█████▋    | 21362/37962 [49:29:15<38:11:49,  8.28s/it]
 56%|█████▋    | 21363/37962 [49:29:23<38:17:20,  8.30s/it]
 56%|█████▋    | 21364/37962 [49:29:32<37:56:21,  8.23s/it]
 56%|█████▋    | 21365/37962 [49:29:40<37:57:27,  8.23s/it]
 56%|█████▋    | 21366/37962 [49:29:48<38:01:52,  8.25s/it]
 56%|█████▋    | 21367/37962 [49:29:56<38:12:13,  8.29s/it]
 56%|█████▋    | 21368/37962 [49:30:04<37:50:38,  8.21s/it]
 56%|█████▋    | 21369/37962 [49:30:13<37:46:44,  8.20s/it]
 56%|█████▋    | 21370/37962 [49:30:21<37:53:31,  8.22s/it]
                                                           
{'loss': 1.292, 'grad_norm': 8.69291877746582, 'learning_rate': 8.453901523405314e-06, 'epoch': 0.56}

 56%|█████▋    | 21370/37962 [49:30:21<37:53:31,  8.22s/it]
 56%|█████▋    | 21371/37962 [49:30:29<38:02:53,  8.26s/it]
 56%|█████▋    | 21372/37962 [49:30:38<38:23:29,  8.33s/it]
 56%|█████▋    | 21373/37962 [49:30:46<37:58:48,  8.24s/it]
 56%|█████▋    | 21374/37962 [49:30:54<37:58:07,  8.24s/it]
 56%|█████▋    | 21375/37962 [49:31:02<38:08:16,  8.28s/it]
 56%|█████▋    | 21376/37962 [49:31:11<38:06:15,  8.27s/it]
 56%|█████▋    | 21377/37962 [49:31:19<38:13:35,  8.30s/it]
 56%|█████▋    | 21378/37962 [49:31:27<38:08:57,  8.28s/it]
 56%|█████▋    | 21379/37962 [49:31:35<37:44:35,  8.19s/it]
 56%|█████▋    | 21380/37962 [49:31:43<37:27:51,  8.13s/it]
                                                           
{'loss': 1.3404, 'grad_norm': 15.838998794555664, 'learning_rate': 8.445473070195811e-06, 'epoch': 0.56}

 56%|█████▋    | 21380/37962 [49:31:43<37:27:51,  8.13s/it]
 56%|█████▋    | 21381/37962 [49:31:52<37:48:02,  8.21s/it]
 56%|█████▋    | 21382/37962 [49:32:00<37:33:47,  8.16s/it]
 56%|█████▋    | 21383/37962 [49:32:08<37:59:57,  8.25s/it]
 56%|█████▋    | 21384/37962 [49:32:16<37:32:42,  8.15s/it]
 56%|█████▋    | 21385/37962 [49:32:24<37:51:40,  8.22s/it]
 56%|█████▋    | 21386/37962 [49:32:33<38:08:08,  8.28s/it]
 56%|█████▋    | 21387/37962 [49:32:41<38:16:44,  8.31s/it]
 56%|█████▋    | 21388/37962 [49:32:49<37:53:44,  8.23s/it]
 56%|█████▋    | 21389/37962 [49:32:58<38:01:04,  8.26s/it]
 56%|█████▋    | 21390/37962 [49:33:06<38:12:11,  8.30s/it]
                                                           
{'loss': 1.1974, 'grad_norm': 10.592498779296875, 'learning_rate': 8.437045748499531e-06, 'epoch': 0.56}

 56%|█████▋    | 21390/37962 [49:33:06<38:12:11,  8.30s/it]
 56%|█████▋    | 21391/37962 [49:33:14<38:02:58,  8.27s/it]
 56%|█████▋    | 21392/37962 [49:33:23<38:12:06,  8.30s/it]
 56%|█████▋    | 21393/37962 [49:33:31<38:12:25,  8.30s/it]
 56%|█████▋    | 21394/37962 [49:33:39<37:41:56,  8.19s/it]
 56%|█████▋    | 21395/37962 [49:33:47<37:53:21,  8.23s/it]
 56%|█████▋    | 21396/37962 [49:33:55<38:01:31,  8.26s/it]
 56%|█████▋    | 21397/37962 [49:34:04<38:14:09,  8.31s/it]
 56%|█████▋    | 21398/37962 [49:34:12<38:28:23,  8.36s/it]
 56%|█████▋    | 21399/37962 [49:34:21<38:35:56,  8.39s/it]
 56%|█████▋    | 21400/37962 [49:34:29<38:21:40,  8.34s/it]
                                                           
{'loss': 1.4133, 'grad_norm': 7.39174222946167, 'learning_rate': 8.42861956445058e-06, 'epoch': 0.56}

 56%|█████▋    | 21400/37962 [49:34:29<38:21:40,  8.34s/it]
 56%|█████▋    | 21401/37962 [49:34:37<38:03:52,  8.27s/it]
 56%|█████▋    | 21402/37962 [49:34:45<38:01:05,  8.26s/it]
 56%|█████▋    | 21403/37962 [49:34:54<38:10:28,  8.30s/it]
 56%|█████▋    | 21404/37962 [49:35:02<38:11:51,  8.30s/it]
 56%|█████▋    | 21405/37962 [49:35:11<38:35:57,  8.39s/it]
 56%|█████▋    | 21406/37962 [49:35:19<37:59:21,  8.26s/it]
 56%|█████▋    | 21407/37962 [49:35:27<37:43:10,  8.20s/it]
 56%|█████▋    | 21408/37962 [49:35:35<37:51:14,  8.23s/it]
 56%|█████▋    | 21409/37962 [49:35:43<38:07:08,  8.29s/it]
 56%|█████▋    | 21410/37962 [49:35:52<37:53:53,  8.24s/it]
                                                           
{'loss': 1.42, 'grad_norm': 10.300704956054688, 'learning_rate': 8.420194524182219e-06, 'epoch': 0.56}

 56%|█████▋    | 21410/37962 [49:35:52<37:53:53,  8.24s/it]
 56%|█████▋    | 21411/37962 [49:36:00<37:51:16,  8.23s/it]
 56%|█████▋    | 21412/37962 [49:36:08<37:54:26,  8.25s/it]
 56%|█████▋    | 21413/37962 [49:36:16<38:05:21,  8.29s/it]
 56%|█████▋    | 21414/37962 [49:36:25<38:03:19,  8.28s/it]
 56%|█████▋    | 21415/37962 [49:36:33<38:06:35,  8.29s/it]
 56%|█████▋    | 21416/37962 [49:36:41<38:00:22,  8.27s/it]
 56%|█████▋    | 21417/37962 [49:36:49<37:43:54,  8.21s/it]
 56%|█████▋    | 21418/37962 [49:36:58<37:54:17,  8.25s/it]
 56%|█████▋    | 21419/37962 [49:37:06<37:46:10,  8.22s/it]
 56%|█████▋    | 21420/37962 [49:37:14<37:50:53,  8.24s/it]
                                                           
{'loss': 1.3035, 'grad_norm': 8.795065879821777, 'learning_rate': 8.411770633826898e-06, 'epoch': 0.56}

 56%|█████▋    | 21420/37962 [49:37:14<37:50:53,  8.24s/it]
 56%|█████▋    | 21421/37962 [49:37:22<37:47:43,  8.23s/it]
 56%|█████▋    | 21422/37962 [49:37:31<37:55:56,  8.26s/it]
 56%|█████▋    | 21423/37962 [49:37:39<37:52:13,  8.24s/it]
 56%|█████▋    | 21424/37962 [49:37:47<37:48:27,  8.23s/it]
 56%|█████▋    | 21425/37962 [49:37:55<37:36:16,  8.19s/it]
 56%|█████▋    | 21426/37962 [49:38:03<37:31:19,  8.17s/it]
 56%|█████▋    | 21427/37962 [49:38:11<37:34:53,  8.18s/it]
 56%|█████▋    | 21428/37962 [49:38:20<37:55:15,  8.26s/it]
 56%|█████▋    | 21429/37962 [49:38:28<38:00:42,  8.28s/it]
 56%|█████▋    | 21430/37962 [49:38:36<38:02:44,  8.28s/it]
                                                           
{'loss': 1.337, 'grad_norm': 7.380778789520264, 'learning_rate': 8.403347899516218e-06, 'epoch': 0.56}

 56%|█████▋    | 21430/37962 [49:38:36<38:02:44,  8.28s/it]
 56%|█████▋    | 21431/37962 [49:38:45<38:11:19,  8.32s/it]
 56%|█████▋    | 21432/37962 [49:38:53<38:09:31,  8.31s/it]
 56%|█████▋    | 21433/37962 [49:39:02<38:18:05,  8.34s/it]
 56%|█████▋    | 21434/37962 [49:39:10<38:35:48,  8.41s/it]
 56%|█████▋    | 21435/37962 [49:39:18<38:00:01,  8.28s/it]
 56%|█████▋    | 21436/37962 [49:39:26<38:02:26,  8.29s/it]
 56%|█████▋    | 21437/37962 [49:39:35<38:07:56,  8.31s/it]
 56%|█████▋    | 21438/37962 [49:39:43<37:59:37,  8.28s/it]
 56%|█████▋    | 21439/37962 [49:39:51<38:15:52,  8.34s/it]
 56%|█████▋    | 21440/37962 [49:40:00<38:11:53,  8.32s/it]
                                                           
{'loss': 1.3854, 'grad_norm': 3.6607167720794678, 'learning_rate': 8.394926327380942e-06, 'epoch': 0.56}

 56%|█████▋    | 21440/37962 [49:40:00<38:11:53,  8.32s/it]
 56%|█████▋    | 21441/37962 [49:40:08<38:00:53,  8.28s/it]
 56%|█████▋    | 21442/37962 [49:40:16<37:45:20,  8.23s/it]
 56%|█████▋    | 21443/37962 [49:40:24<37:55:41,  8.27s/it]
 56%|█████▋    | 21444/37962 [49:40:33<37:54:26,  8.26s/it]
 56%|█████▋    | 21445/37962 [49:40:41<38:03:34,  8.30s/it]
 56%|█████▋    | 21446/37962 [49:40:49<37:27:53,  8.17s/it]
 56%|█████▋    | 21447/37962 [49:40:57<37:35:41,  8.20s/it]
 56%|█████▋    | 21448/37962 [49:41:05<37:41:51,  8.22s/it]
 57%|█████▋    | 21449/37962 [49:41:13<37:26:39,  8.16s/it]
 57%|█████▋    | 21450/37962 [49:41:21<37:11:24,  8.11s/it]
                                                           
{'loss': 1.1598, 'grad_norm': 8.759010314941406, 'learning_rate': 8.386505923550989e-06, 'epoch': 0.57}

 57%|█████▋    | 21450/37962 [49:41:21<37:11:24,  8.11s/it]
 57%|█████▋    | 21451/37962 [49:41:30<37:11:20,  8.11s/it]
 57%|█████▋    | 21452/37962 [49:41:38<37:21:36,  8.15s/it]
 57%|█████▋    | 21453/37962 [49:41:46<37:35:04,  8.20s/it]
 57%|█████▋    | 21454/37962 [49:41:54<37:40:57,  8.22s/it]
 57%|█████▋    | 21455/37962 [49:42:03<37:50:57,  8.25s/it]
 57%|█████▋    | 21456/37962 [49:42:11<37:51:42,  8.26s/it]
 57%|█████▋    | 21457/37962 [49:42:19<37:51:38,  8.26s/it]
 57%|█████▋    | 21458/37962 [49:42:28<37:52:31,  8.26s/it]
 57%|█████▋    | 21459/37962 [49:42:36<37:31:00,  8.18s/it]
 57%|█████▋    | 21460/37962 [49:42:44<37:29:20,  8.18s/it]
                                                           
{'loss': 1.4663, 'grad_norm': 12.915225982666016, 'learning_rate': 8.37808669415542e-06, 'epoch': 0.57}

 57%|█████▋    | 21460/37962 [49:42:44<37:29:20,  8.18s/it]
 57%|█████▋    | 21461/37962 [49:42:52<37:37:55,  8.21s/it]
 57%|█████▋    | 21462/37962 [49:43:00<37:46:37,  8.24s/it]
 57%|█████▋    | 21463/37962 [49:43:09<37:45:32,  8.24s/it]
 57%|█████▋    | 21464/37962 [49:43:17<37:57:10,  8.28s/it]
 57%|█████▋    | 21465/37962 [49:43:25<37:53:29,  8.27s/it]
 57%|█████▋    | 21466/37962 [49:43:33<38:01:19,  8.30s/it]
 57%|█████▋    | 21467/37962 [49:43:42<37:52:05,  8.26s/it]
 57%|█████▋    | 21468/37962 [49:43:50<37:55:37,  8.28s/it]
 57%|█████▋    | 21469/37962 [49:43:58<37:31:38,  8.19s/it]
 57%|█████▋    | 21470/37962 [49:44:06<37:43:54,  8.24s/it]
                                                           
{'loss': 1.3152, 'grad_norm': 3.9171478748321533, 'learning_rate': 8.369668645322449e-06, 'epoch': 0.57}

 57%|█████▋    | 21470/37962 [49:44:06<37:43:54,  8.24s/it]
 57%|█████▋    | 21471/37962 [49:44:15<37:49:15,  8.26s/it]
 57%|█████▋    | 21472/37962 [49:44:23<37:48:08,  8.25s/it]
 57%|█████▋    | 21473/37962 [49:44:31<37:52:21,  8.27s/it]
 57%|█████▋    | 21474/37962 [49:44:39<37:36:22,  8.21s/it]
 57%|█████▋    | 21475/37962 [49:44:47<37:34:35,  8.20s/it]
 57%|█████▋    | 21476/37962 [49:44:56<37:38:18,  8.22s/it]
 57%|█████▋    | 21477/37962 [49:45:04<37:48:09,  8.26s/it]
 57%|█████▋    | 21478/37962 [49:45:12<37:51:05,  8.27s/it]
 57%|█████▋    | 21479/37962 [49:45:21<37:51:44,  8.27s/it]
 57%|█████▋    | 21480/37962 [49:45:29<37:45:39,  8.25s/it]
                                                           
{'loss': 1.2279, 'grad_norm': 17.44007110595703, 'learning_rate': 8.361251783179424e-06, 'epoch': 0.57}

 57%|█████▋    | 21480/37962 [49:45:29<37:45:39,  8.25s/it]
 57%|█████▋    | 21481/37962 [49:45:37<37:38:15,  8.22s/it]
 57%|█████▋    | 21482/37962 [49:45:45<37:31:02,  8.20s/it]
 57%|█████▋    | 21483/37962 [49:45:54<37:48:54,  8.26s/it]
 57%|█████▋    | 21484/37962 [49:46:02<37:30:29,  8.19s/it]
 57%|█████▋    | 21485/37962 [49:46:10<37:39:24,  8.23s/it]
 57%|█████▋    | 21486/37962 [49:46:18<37:30:37,  8.20s/it]
 57%|█████▋    | 21487/37962 [49:46:26<37:31:47,  8.20s/it]
 57%|█████▋    | 21488/37962 [49:46:34<37:27:45,  8.19s/it]
 57%|█████▋    | 21489/37962 [49:46:43<37:48:07,  8.26s/it]
 57%|█████▋    | 21490/37962 [49:46:51<37:30:51,  8.20s/it]
                                                           
{'loss': 1.308, 'grad_norm': 17.90239143371582, 'learning_rate': 8.352836113852838e-06, 'epoch': 0.57}

 57%|█████▋    | 21490/37962 [49:46:51<37:30:51,  8.20s/it]
 57%|█████▋    | 21491/37962 [49:46:59<37:44:03,  8.25s/it]
 57%|█████▋    | 21492/37962 [49:47:07<37:10:51,  8.13s/it]
 57%|█████▋    | 21493/37962 [49:47:15<37:17:14,  8.15s/it]
 57%|█████▋    | 21494/37962 [49:47:24<37:30:36,  8.20s/it]
 57%|█████▋    | 21495/37962 [49:47:32<37:39:49,  8.23s/it]
 57%|█████▋    | 21496/37962 [49:47:40<37:37:38,  8.23s/it]
 57%|█████▋    | 21497/37962 [49:47:48<37:37:52,  8.23s/it]
 57%|█████▋    | 21498/37962 [49:47:56<37:27:50,  8.19s/it]
 57%|█████▋    | 21499/37962 [49:48:05<37:30:18,  8.20s/it]
 57%|█████▋    | 21500/37962 [49:48:13<37:46:19,  8.26s/it]
                                                           
{'loss': 1.2626, 'grad_norm': 8.316226959228516, 'learning_rate': 8.344421643468316e-06, 'epoch': 0.57}

 57%|█████▋    | 21500/37962 [49:48:13<37:46:19,  8.26s/it]
 57%|█████▋    | 21501/37962 [49:48:21<37:33:12,  8.21s/it]
 57%|█████▋    | 21502/37962 [49:48:29<37:39:21,  8.24s/it]
 57%|█████▋    | 21503/37962 [49:48:38<37:58:30,  8.31s/it]
 57%|█████▋    | 21504/37962 [49:48:46<37:16:48,  8.15s/it]
 57%|█████▋    | 21505/37962 [49:48:54<37:22:38,  8.18s/it]
 57%|█████▋    | 21506/37962 [49:49:03<38:09:29,  8.35s/it]
 57%|█████▋    | 21507/37962 [49:49:11<38:15:38,  8.37s/it]
 57%|█████▋    | 21508/37962 [49:49:19<38:04:05,  8.33s/it]
 57%|█████▋    | 21509/37962 [49:49:28<37:51:41,  8.28s/it]
 57%|█████▋    | 21510/37962 [49:49:36<38:04:56,  8.33s/it]
                                                           
{'loss': 1.261, 'grad_norm': 11.291923522949219, 'learning_rate': 8.336008378150593e-06, 'epoch': 0.57}

 57%|█████▋    | 21510/37962 [49:49:36<38:04:56,  8.33s/it]
 57%|█████▋    | 21511/37962 [49:49:44<37:58:54,  8.31s/it]
 57%|█████▋    | 21512/37962 [49:49:53<37:57:33,  8.31s/it]
 57%|█████▋    | 21513/37962 [49:50:01<37:31:31,  8.21s/it]
 57%|█████▋    | 21514/37962 [49:50:09<37:41:12,  8.25s/it]
 57%|█████▋    | 21515/37962 [49:50:17<37:44:08,  8.26s/it]
 57%|█████▋    | 21516/37962 [49:50:25<37:39:14,  8.24s/it]
 57%|█████▋    | 21517/37962 [49:50:34<37:46:23,  8.27s/it]
 57%|█████▋    | 21518/37962 [49:50:41<37:08:49,  8.13s/it]
 57%|█████▋    | 21519/37962 [49:50:50<37:23:30,  8.19s/it]
 57%|█████▋    | 21520/37962 [49:50:58<37:09:23,  8.14s/it]
                                                           
{'loss': 1.538, 'grad_norm': 6.602450847625732, 'learning_rate': 8.327596324023545e-06, 'epoch': 0.57}

 57%|█████▋    | 21520/37962 [49:50:58<37:09:23,  8.14s/it]
 57%|█████▋    | 21521/37962 [49:51:06<36:55:22,  8.08s/it]
 57%|█████▋    | 21522/37962 [49:51:14<37:05:07,  8.12s/it]
 57%|█████▋    | 21523/37962 [49:51:22<37:13:07,  8.15s/it]
 57%|█████▋    | 21524/37962 [49:51:31<37:46:53,  8.27s/it]
 57%|█████▋    | 21525/37962 [49:51:39<38:07:29,  8.35s/it]
 57%|█████▋    | 21526/37962 [49:51:47<37:55:29,  8.31s/it]
 57%|█████▋    | 21527/37962 [49:51:56<38:00:38,  8.33s/it]
 57%|█████▋    | 21528/37962 [49:52:04<38:23:11,  8.41s/it]
 57%|█████▋    | 21529/37962 [49:52:13<38:09:43,  8.36s/it]
 57%|█████▋    | 21530/37962 [49:52:21<38:14:02,  8.38s/it]
                                                           
{'loss': 1.267, 'grad_norm': 9.672451972961426, 'learning_rate': 8.319185487210162e-06, 'epoch': 0.57}

 57%|█████▋    | 21530/37962 [49:52:21<38:14:02,  8.38s/it]
 57%|█████▋    | 21531/37962 [49:52:29<38:08:47,  8.36s/it]
 57%|█████▋    | 21532/37962 [49:52:38<38:17:19,  8.39s/it]
 57%|█████▋    | 21533/37962 [49:52:46<38:03:24,  8.34s/it]
 57%|█████▋    | 21534/37962 [49:52:54<37:42:29,  8.26s/it]
 57%|█████▋    | 21535/37962 [49:53:02<37:18:59,  8.18s/it]
 57%|█████▋    | 21536/37962 [49:53:10<37:20:59,  8.19s/it]
 57%|█████▋    | 21537/37962 [49:53:19<37:30:58,  8.22s/it]
 57%|█████▋    | 21538/37962 [49:53:27<37:37:38,  8.25s/it]
 57%|█████▋    | 21539/37962 [49:53:35<37:51:52,  8.30s/it]
 57%|█████▋    | 21540/37962 [49:53:44<38:02:16,  8.34s/it]
                                                           
{'loss': 1.2924, 'grad_norm': 11.05896282196045, 'learning_rate': 8.310775873832545e-06, 'epoch': 0.57}

 57%|█████▋    | 21540/37962 [49:53:44<38:02:16,  8.34s/it]
 57%|█████▋    | 21541/37962 [49:53:52<37:29:16,  8.22s/it]
 57%|█████▋    | 21542/37962 [49:54:00<37:24:37,  8.20s/it]
 57%|█████▋    | 21543/37962 [49:54:08<37:39:38,  8.26s/it]
 57%|█████▋    | 21544/37962 [49:54:17<37:40:58,  8.26s/it]
 57%|█████▋    | 21545/37962 [49:54:25<37:12:12,  8.16s/it]
 57%|█████▋    | 21546/37962 [49:54:33<37:40:59,  8.26s/it]
 57%|█████▋    | 21547/37962 [49:54:41<37:18:18,  8.18s/it]
 57%|█████▋    | 21548/37962 [49:54:49<37:19:43,  8.19s/it]
 57%|█████▋    | 21549/37962 [49:54:58<37:36:13,  8.25s/it]
 57%|█████▋    | 21550/37962 [49:55:06<37:27:12,  8.22s/it]
                                                           
{'loss': 1.3202, 'grad_norm': 5.1027374267578125, 'learning_rate': 8.302367490011904e-06, 'epoch': 0.57}

 57%|█████▋    | 21550/37962 [49:55:06<37:27:12,  8.22s/it]
 57%|█████▋    | 21551/37962 [49:55:14<37:27:10,  8.22s/it]
 57%|█████▋    | 21552/37962 [49:55:22<37:05:54,  8.14s/it]
 57%|█████▋    | 21553/37962 [49:55:30<37:33:02,  8.24s/it]
 57%|█████▋    | 21554/37962 [49:55:38<36:51:16,  8.09s/it]
 57%|█████▋    | 21555/37962 [49:55:46<36:33:46,  8.02s/it]
 57%|█████▋    | 21556/37962 [49:55:54<36:55:41,  8.10s/it]
 57%|█████▋    | 21557/37962 [49:56:03<37:35:16,  8.25s/it]
 57%|█████▋    | 21558/37962 [49:56:11<37:49:51,  8.30s/it]
 57%|█████▋    | 21559/37962 [49:56:19<37:38:33,  8.26s/it]
 57%|█████▋    | 21560/37962 [49:56:27<37:08:30,  8.15s/it]
                                                           
{'loss': 1.3297, 'grad_norm': 6.766063690185547, 'learning_rate': 8.293960341868558e-06, 'epoch': 0.57}

 57%|█████▋    | 21560/37962 [49:56:27<37:08:30,  8.15s/it]
 57%|█████▋    | 21561/37962 [49:56:35<37:03:13,  8.13s/it]
 57%|█████▋    | 21562/37962 [49:56:44<37:24:49,  8.21s/it]
 57%|█████▋    | 21563/37962 [49:56:52<37:20:20,  8.20s/it]
 57%|█████▋    | 21564/37962 [49:57:00<36:45:07,  8.07s/it]
 57%|█████▋    | 21565/37962 [49:57:08<36:26:03,  8.00s/it]
 57%|█████▋    | 21566/37962 [49:57:16<36:58:16,  8.12s/it]
 57%|█████▋    | 21567/37962 [49:57:24<37:09:16,  8.16s/it]
 57%|█████▋    | 21568/37962 [49:57:33<37:28:09,  8.23s/it]
 57%|█████▋    | 21569/37962 [49:57:41<37:25:04,  8.22s/it]
 57%|█████▋    | 21570/37962 [49:57:57<47:49:41, 10.50s/it]
                                                           
{'loss': 1.2257, 'grad_norm': 9.693469047546387, 'learning_rate': 8.285554435521926e-06, 'epoch': 0.57}

 57%|█████▋    | 21570/37962 [49:57:57<47:49:41, 10.50s/it]
 57%|█████▋    | 21571/37962 [49:58:05<45:03:31,  9.90s/it]
 57%|█████▋    | 21572/37962 [49:58:13<42:48:22,  9.40s/it]
 57%|█████▋    | 21573/37962 [49:58:22<41:27:05,  9.11s/it]
 57%|█████▋    | 21574/37962 [49:58:30<40:20:36,  8.86s/it]
 57%|█████▋    | 21575/37962 [49:58:38<39:37:34,  8.71s/it]
 57%|█████▋    | 21576/37962 [49:58:47<38:52:17,  8.54s/it]
 57%|█████▋    | 21577/37962 [49:58:55<38:28:36,  8.45s/it]
 57%|█████▋    | 21578/37962 [49:59:03<38:23:17,  8.43s/it]
 57%|█████▋    | 21579/37962 [49:59:11<37:44:15,  8.29s/it]
 57%|█████▋    | 21580/37962 [49:59:19<37:33:31,  8.25s/it]
                                                           
{'loss': 1.2374, 'grad_norm': 4.547384262084961, 'learning_rate': 8.277149777090514e-06, 'epoch': 0.57}

 57%|█████▋    | 21580/37962 [49:59:19<37:33:31,  8.25s/it]
 57%|█████▋    | 21581/37962 [49:59:28<37:33:42,  8.25s/it]
 57%|█████▋    | 21582/37962 [49:59:36<38:04:00,  8.37s/it]
 57%|█████▋    | 21583/37962 [49:59:45<37:54:29,  8.33s/it]
 57%|█████▋    | 21584/37962 [49:59:53<37:44:06,  8.29s/it]
 57%|█████▋    | 21585/37962 [50:00:01<37:47:50,  8.31s/it]
 57%|█████▋    | 21586/37962 [50:00:09<37:32:07,  8.25s/it]
 57%|█████▋    | 21587/37962 [50:00:17<37:29:43,  8.24s/it]
 57%|█████▋    | 21588/37962 [50:00:26<37:29:26,  8.24s/it]
 57%|█████▋    | 21589/37962 [50:00:34<37:29:41,  8.24s/it]
 57%|█████▋    | 21590/37962 [50:00:42<37:39:01,  8.28s/it]
                                                           
{'loss': 1.2828, 'grad_norm': 11.056100845336914, 'learning_rate': 8.268746372691937e-06, 'epoch': 0.57}

 57%|█████▋    | 21590/37962 [50:00:42<37:39:01,  8.28s/it]
 57%|█████▋    | 21591/37962 [50:00:51<37:34:38,  8.26s/it]
 57%|█████▋    | 21592/37962 [50:00:59<37:26:48,  8.24s/it]
 57%|█████▋    | 21593/37962 [50:01:07<37:35:02,  8.27s/it]
 57%|█████▋    | 21594/37962 [50:01:15<37:46:21,  8.31s/it]
 57%|█████▋    | 21595/37962 [50:01:24<37:32:17,  8.26s/it]
 57%|█████▋    | 21596/37962 [50:01:32<37:23:57,  8.23s/it]
 57%|█████▋    | 21597/37962 [50:01:40<37:30:07,  8.25s/it]
 57%|█████▋    | 21598/37962 [50:01:48<37:35:12,  8.27s/it]
 57%|█████▋    | 21599/37962 [50:01:57<37:55:12,  8.34s/it]
 57%|█████▋    | 21600/37962 [50:02:05<37:42:15,  8.30s/it]
                                                           
{'loss': 1.2188, 'grad_norm': 6.4510297775268555, 'learning_rate': 8.260344228442883e-06, 'epoch': 0.57}

 57%|█████▋    | 21600/37962 [50:02:05<37:42:15,  8.30s/it]
 57%|█████▋    | 21601/37962 [50:02:13<37:28:41,  8.25s/it]
 57%|█████▋    | 21602/37962 [50:02:21<37:21:46,  8.22s/it]
 57%|█████▋    | 21603/37962 [50:02:30<37:32:34,  8.26s/it]
 57%|█████▋    | 21604/37962 [50:02:38<37:18:27,  8.21s/it]
 57%|█████▋    | 21605/37962 [50:02:46<37:33:18,  8.27s/it]
 57%|█████▋    | 21606/37962 [50:02:55<37:58:16,  8.36s/it]
 57%|█████▋    | 21607/37962 [50:03:03<38:16:01,  8.42s/it]
 57%|█████▋    | 21608/37962 [50:03:12<38:32:16,  8.48s/it]
 57%|█████▋    | 21609/37962 [50:03:20<38:14:09,  8.42s/it]
 57%|█████▋    | 21610/37962 [50:03:28<37:56:13,  8.35s/it]
                                                           
{'loss': 1.2733, 'grad_norm': 6.645703315734863, 'learning_rate': 8.25194335045913e-06, 'epoch': 0.57}

 57%|█████▋    | 21610/37962 [50:03:28<37:56:13,  8.35s/it]
 57%|█████▋    | 21611/37962 [50:03:37<38:05:53,  8.39s/it]
 57%|█████▋    | 21612/37962 [50:03:45<37:49:24,  8.33s/it]
 57%|█████▋    | 21613/37962 [50:03:53<37:58:06,  8.36s/it]
 57%|█████▋    | 21614/37962 [50:04:02<37:48:17,  8.33s/it]
 57%|█████▋    | 21615/37962 [50:04:10<37:31:51,  8.27s/it]
 57%|█████▋    | 21616/37962 [50:04:18<37:01:17,  8.15s/it]
 57%|█████▋    | 21617/37962 [50:04:26<37:06:23,  8.17s/it]
 57%|█████▋    | 21618/37962 [50:04:34<37:26:15,  8.25s/it]
 57%|█████▋    | 21619/37962 [50:04:42<36:53:22,  8.13s/it]
 57%|█████▋    | 21620/37962 [50:04:51<37:07:27,  8.18s/it]
                                                           
{'loss': 1.2992, 'grad_norm': 5.12797212600708, 'learning_rate': 8.243543744855526e-06, 'epoch': 0.57}

 57%|█████▋    | 21620/37962 [50:04:51<37:07:27,  8.18s/it]
 57%|█████▋    | 21621/37962 [50:04:59<37:07:08,  8.18s/it]
 57%|█████▋    | 21622/37962 [50:05:07<37:19:49,  8.22s/it]
 57%|█████▋    | 21623/37962 [50:05:15<36:57:18,  8.14s/it]
 57%|█████▋    | 21624/37962 [50:05:23<37:08:00,  8.18s/it]
 57%|█████▋    | 21625/37962 [50:05:31<37:05:45,  8.17s/it]
 57%|█████▋    | 21626/37962 [50:05:40<37:14:47,  8.21s/it]
 57%|█████▋    | 21627/37962 [50:05:48<37:04:35,  8.17s/it]
 57%|█████▋    | 21628/37962 [50:05:56<36:49:12,  8.12s/it]
 57%|█████▋    | 21629/37962 [50:06:04<37:09:15,  8.19s/it]
 57%|█████▋    | 21630/37962 [50:06:13<37:23:31,  8.24s/it]
                                                           
{'loss': 1.3106, 'grad_norm': 6.631440162658691, 'learning_rate': 8.235145417746004e-06, 'epoch': 0.57}

 57%|█████▋    | 21630/37962 [50:06:13<37:23:31,  8.24s/it]
 57%|█████▋    | 21631/37962 [50:06:21<37:25:24,  8.25s/it]
 57%|█████▋    | 21632/37962 [50:06:29<37:34:44,  8.28s/it]
 57%|█████▋    | 21633/37962 [50:06:37<37:34:45,  8.28s/it]
 57%|█████▋    | 21634/37962 [50:06:46<37:53:04,  8.35s/it]
 57%|█████▋    | 21635/37962 [50:06:54<37:53:09,  8.35s/it]
 57%|█████▋    | 21636/37962 [50:07:02<37:39:20,  8.30s/it]
 57%|█████▋    | 21637/37962 [50:07:11<37:37:15,  8.30s/it]
 57%|█████▋    | 21638/37962 [50:07:19<37:39:19,  8.30s/it]
 57%|█████▋    | 21639/37962 [50:07:27<37:04:49,  8.18s/it]
 57%|█████▋    | 21640/37962 [50:07:35<37:12:41,  8.21s/it]
                                                           
{'loss': 1.3325, 'grad_norm': 9.748980522155762, 'learning_rate': 8.226748375243556e-06, 'epoch': 0.57}

 57%|█████▋    | 21640/37962 [50:07:35<37:12:41,  8.21s/it]
 57%|█████▋    | 21641/37962 [50:07:43<36:58:56,  8.16s/it]
 57%|█████▋    | 21642/37962 [50:07:52<37:04:20,  8.18s/it]
 57%|█████▋    | 21643/37962 [50:08:00<37:11:11,  8.20s/it]
 57%|█████▋    | 21644/37962 [50:08:08<37:13:42,  8.21s/it]
 57%|█████▋    | 21645/37962 [50:08:16<37:32:03,  8.28s/it]
 57%|█████▋    | 21646/37962 [50:08:25<37:29:15,  8.27s/it]
 57%|█████▋    | 21647/37962 [50:08:33<37:39:55,  8.31s/it]
 57%|█████▋    | 21648/37962 [50:08:41<37:28:02,  8.27s/it]
 57%|█████▋    | 21649/37962 [50:08:50<37:26:10,  8.26s/it]
 57%|█████▋    | 21650/37962 [50:08:58<37:20:41,  8.24s/it]
                                                           
{'loss': 1.2433, 'grad_norm': 8.703916549682617, 'learning_rate': 8.218352623460247e-06, 'epoch': 0.57}

 57%|█████▋    | 21650/37962 [50:08:58<37:20:41,  8.24s/it]
 57%|█████▋    | 21651/37962 [50:09:06<37:23:40,  8.25s/it]
 57%|█████▋    | 21652/37962 [50:09:14<37:29:45,  8.28s/it]
 57%|█████▋    | 21653/37962 [50:09:23<37:41:13,  8.32s/it]
 57%|█████▋    | 21654/37962 [50:09:31<37:03:50,  8.18s/it]
 57%|█████▋    | 21655/37962 [50:09:39<37:11:24,  8.21s/it]
 57%|█████▋    | 21656/37962 [50:09:47<37:17:06,  8.23s/it]
 57%|█████▋    | 21657/37962 [50:09:56<37:27:00,  8.27s/it]
 57%|█████▋    | 21658/37962 [50:10:04<38:03:19,  8.40s/it]
 57%|█████▋    | 21659/37962 [50:10:12<37:50:57,  8.36s/it]
 57%|█████▋    | 21660/37962 [50:10:21<37:45:13,  8.34s/it]
                                                           
{'loss': 1.2976, 'grad_norm': 6.609623908996582, 'learning_rate': 8.2099581685072e-06, 'epoch': 0.57}

 57%|█████▋    | 21660/37962 [50:10:21<37:45:13,  8.34s/it]
 57%|█████▋    | 21661/37962 [50:10:29<37:36:43,  8.31s/it]
 57%|█████▋    | 21662/37962 [50:10:37<37:43:16,  8.33s/it]
 57%|█████▋    | 21663/37962 [50:10:45<37:08:29,  8.20s/it]
 57%|█████▋    | 21664/37962 [50:10:53<37:04:19,  8.19s/it]
 57%|█████▋    | 21665/37962 [50:11:02<37:05:41,  8.19s/it]
 57%|█████▋    | 21666/37962 [50:11:10<37:23:19,  8.26s/it]
 57%|█████▋    | 21667/37962 [50:11:19<37:37:34,  8.31s/it]
 57%|█████▋    | 21668/37962 [50:11:27<37:48:37,  8.35s/it]
 57%|█████▋    | 21669/37962 [50:11:35<37:54:19,  8.38s/it]
 57%|█████▋    | 21670/37962 [50:11:44<37:49:14,  8.36s/it]
                                                           
{'loss': 1.3113, 'grad_norm': 11.439959526062012, 'learning_rate': 8.201565016494592e-06, 'epoch': 0.57}

 57%|█████▋    | 21670/37962 [50:11:44<37:49:14,  8.36s/it]
 57%|█████▋    | 21671/37962 [50:11:52<37:45:44,  8.34s/it]
 57%|█████▋    | 21672/37962 [50:12:01<38:00:48,  8.40s/it]
 57%|█████▋    | 21673/37962 [50:12:09<38:01:33,  8.40s/it]
 57%|█████▋    | 21674/37962 [50:12:17<37:54:43,  8.38s/it]
 57%|█████▋    | 21675/37962 [50:12:25<37:38:42,  8.32s/it]
 57%|█████▋    | 21676/37962 [50:12:34<37:54:28,  8.38s/it]
 57%|█████▋    | 21677/37962 [50:12:42<37:49:36,  8.36s/it]
 57%|█████▋    | 21678/37962 [50:12:51<37:37:07,  8.32s/it]
 57%|█████▋    | 21679/37962 [50:12:59<37:36:50,  8.32s/it]
 57%|█████▋    | 21680/37962 [50:13:07<37:32:25,  8.30s/it]
                                                           
{'loss': 1.2526, 'grad_norm': 17.232013702392578, 'learning_rate': 8.19317317353165e-06, 'epoch': 0.57}

 57%|█████▋    | 21680/37962 [50:13:07<37:32:25,  8.30s/it]
 57%|█████▋    | 21681/37962 [50:13:15<37:29:00,  8.29s/it]
 57%|█████▋    | 21682/37962 [50:13:24<37:29:01,  8.29s/it]
 57%|█████▋    | 21683/37962 [50:13:32<37:38:23,  8.32s/it]
 57%|█████▋    | 21684/37962 [50:13:40<37:36:00,  8.32s/it]
 57%|█████▋    | 21685/37962 [50:13:49<37:38:50,  8.33s/it]
 57%|█████▋    | 21686/37962 [50:13:57<37:25:50,  8.28s/it]
 57%|█████▋    | 21687/37962 [50:14:05<37:35:44,  8.32s/it]
 57%|█████▋    | 21688/37962 [50:14:14<37:36:24,  8.32s/it]
 57%|█████▋    | 21689/37962 [50:14:22<37:36:37,  8.32s/it]
 57%|█████▋    | 21690/37962 [50:14:30<37:16:17,  8.25s/it]
                                                           
{'loss': 1.3091, 'grad_norm': 10.785612106323242, 'learning_rate': 8.184782645726654e-06, 'epoch': 0.57}

 57%|█████▋    | 21690/37962 [50:14:30<37:16:17,  8.25s/it]
 57%|█████▋    | 21691/37962 [50:14:38<36:51:14,  8.15s/it]
 57%|█████▋    | 21692/37962 [50:14:46<36:59:04,  8.18s/it]
 57%|█████▋    | 21693/37962 [50:14:54<37:07:20,  8.21s/it]
 57%|█████▋    | 21694/37962 [50:15:03<37:16:53,  8.25s/it]
 57%|█████▋    | 21695/37962 [50:15:11<37:20:11,  8.26s/it]
 57%|█████▋    | 21696/37962 [50:15:19<37:30:30,  8.30s/it]
 57%|█████▋    | 21697/37962 [50:15:28<37:30:45,  8.30s/it]
 57%|█████▋    | 21698/37962 [50:15:36<37:45:06,  8.36s/it]
 57%|█████▋    | 21699/37962 [50:15:44<37:20:59,  8.27s/it]
 57%|█████▋    | 21700/37962 [50:15:53<37:25:27,  8.28s/it]
                                                           
{'loss': 1.348, 'grad_norm': 18.17803192138672, 'learning_rate': 8.176393439186923e-06, 'epoch': 0.57}

 57%|█████▋    | 21700/37962 [50:15:53<37:25:27,  8.28s/it]
 57%|█████▋    | 21701/37962 [50:16:01<37:10:55,  8.23s/it]
 57%|█████▋    | 21702/37962 [50:16:09<37:28:40,  8.30s/it]
 57%|█████▋    | 21703/37962 [50:16:17<37:11:17,  8.23s/it]
 57%|█████▋    | 21704/37962 [50:16:26<37:08:28,  8.22s/it]
 57%|█████▋    | 21705/37962 [50:16:33<36:44:42,  8.14s/it]
 57%|█████▋    | 21706/37962 [50:16:41<36:29:07,  8.08s/it]
 57%|█████▋    | 21707/37962 [50:16:49<36:29:55,  8.08s/it]
 57%|█████▋    | 21708/37962 [50:16:58<36:57:39,  8.19s/it]
 57%|█████▋    | 21709/37962 [50:17:06<37:08:51,  8.23s/it]
 57%|█████▋    | 21710/37962 [50:17:14<37:03:19,  8.21s/it]
                                                           
{'loss': 1.2567, 'grad_norm': 7.16942834854126, 'learning_rate': 8.16800556001882e-06, 'epoch': 0.57}

 57%|█████▋    | 21710/37962 [50:17:14<37:03:19,  8.21s/it]
 57%|█████▋    | 21711/37962 [50:17:23<37:10:27,  8.24s/it]
 57%|█████▋    | 21712/37962 [50:17:31<37:16:22,  8.26s/it]
 57%|█████▋    | 21713/37962 [50:17:39<36:59:18,  8.19s/it]
 57%|█████▋    | 21714/37962 [50:17:47<36:53:02,  8.17s/it]
 57%|█████▋    | 21715/37962 [50:17:55<36:52:17,  8.17s/it]
 57%|█████▋    | 21716/37962 [50:18:04<36:56:03,  8.18s/it]
 57%|█████▋    | 21717/37962 [50:18:12<36:42:50,  8.14s/it]
 57%|█████▋    | 21718/37962 [50:18:20<36:40:38,  8.13s/it]
 57%|█████▋    | 21719/37962 [50:18:28<36:50:45,  8.17s/it]
 57%|█████▋    | 21720/37962 [50:18:36<36:26:18,  8.08s/it]
                                                           
{'loss': 1.3366, 'grad_norm': 10.197978019714355, 'learning_rate': 8.15961901432773e-06, 'epoch': 0.57}

 57%|█████▋    | 21720/37962 [50:18:36<36:26:18,  8.08s/it]
 57%|█████▋    | 21721/37962 [50:18:44<36:17:34,  8.04s/it]
 57%|█████▋    | 21722/37962 [50:18:52<36:39:42,  8.13s/it]
 57%|█████▋    | 21723/37962 [50:19:00<36:39:58,  8.13s/it]
 57%|█████▋    | 21724/37962 [50:19:08<36:45:09,  8.15s/it]
 57%|█████▋    | 21725/37962 [50:19:16<36:35:38,  8.11s/it]
 57%|█████▋    | 21726/37962 [50:19:24<36:25:38,  8.08s/it]
 57%|█████▋    | 21727/37962 [50:19:33<36:25:26,  8.08s/it]
 57%|█████▋    | 21728/37962 [50:19:40<36:01:08,  7.99s/it]
 57%|█████▋    | 21729/37962 [50:19:49<37:00:50,  8.21s/it]
 57%|█████▋    | 21730/37962 [50:19:57<36:57:05,  8.20s/it]
                                                           
{'loss': 1.2391, 'grad_norm': 8.756180763244629, 'learning_rate': 8.151233808218077e-06, 'epoch': 0.57}

 57%|█████▋    | 21730/37962 [50:19:57<36:57:05,  8.20s/it]
 57%|█████▋    | 21731/37962 [50:20:05<36:42:32,  8.14s/it]
 57%|█████▋    | 21732/37962 [50:20:13<36:51:41,  8.18s/it]
 57%|█████▋    | 21733/37962 [50:20:22<36:53:18,  8.18s/it]
 57%|█████▋    | 21734/37962 [50:20:30<36:26:22,  8.08s/it]
 57%|█████▋    | 21735/37962 [50:20:38<37:08:43,  8.24s/it]
 57%|█████▋    | 21736/37962 [50:20:46<37:06:54,  8.23s/it]
 57%|█████▋    | 21737/37962 [50:20:55<37:15:10,  8.27s/it]
 57%|█████▋    | 21738/37962 [50:21:03<37:26:07,  8.31s/it]
 57%|█████▋    | 21739/37962 [50:21:11<37:33:04,  8.33s/it]
 57%|█████▋    | 21740/37962 [50:21:20<37:35:28,  8.34s/it]
                                                           
{'loss': 1.2503, 'grad_norm': 5.928918838500977, 'learning_rate': 8.142849947793308e-06, 'epoch': 0.57}

 57%|█████▋    | 21740/37962 [50:21:20<37:35:28,  8.34s/it]
 57%|█████▋    | 21741/37962 [50:21:28<37:13:43,  8.26s/it]
 57%|█████▋    | 21742/37962 [50:21:36<37:12:58,  8.26s/it]
 57%|█████▋    | 21743/37962 [50:21:44<37:16:21,  8.27s/it]
 57%|█████▋    | 21744/37962 [50:21:53<37:18:18,  8.28s/it]
 57%|█████▋    | 21745/37962 [50:22:01<37:21:17,  8.29s/it]
 57%|█████▋    | 21746/37962 [50:22:09<37:26:03,  8.31s/it]
 57%|█████▋    | 21747/37962 [50:22:18<37:35:45,  8.35s/it]
 57%|█████▋    | 21748/37962 [50:22:26<37:45:27,  8.38s/it]
 57%|█████▋    | 21749/37962 [50:22:35<37:31:59,  8.33s/it]
 57%|█████▋    | 21750/37962 [50:22:43<37:22:21,  8.30s/it]
                                                           
{'loss': 1.357, 'grad_norm': 3.1544532775878906, 'learning_rate': 8.134467439155888e-06, 'epoch': 0.57}

 57%|█████▋    | 21750/37962 [50:22:43<37:22:21,  8.30s/it]
 57%|█████▋    | 21751/37962 [50:22:51<36:36:37,  8.13s/it]
 57%|█████▋    | 21752/37962 [50:22:59<36:45:24,  8.16s/it]
 57%|█████▋    | 21753/37962 [50:23:07<37:06:52,  8.24s/it]
 57%|█████▋    | 21754/37962 [50:23:15<37:04:23,  8.23s/it]
 57%|█████▋    | 21755/37962 [50:23:24<37:12:09,  8.26s/it]
 57%|█████▋    | 21756/37962 [50:23:32<37:21:34,  8.30s/it]
 57%|█████▋    | 21757/37962 [50:23:40<37:09:54,  8.26s/it]
 57%|█████▋    | 21758/37962 [50:23:49<37:26:12,  8.32s/it]
 57%|█████▋    | 21759/37962 [50:24:05<48:50:08, 10.85s/it]
 57%|█████▋    | 21760/37962 [50:24:14<45:19:08, 10.07s/it]
                                                           
{'loss': 1.3025, 'grad_norm': 9.561430931091309, 'learning_rate': 8.126086288407304e-06, 'epoch': 0.57}

 57%|█████▋    | 21760/37962 [50:24:14<45:19:08, 10.07s/it]
 57%|█████▋    | 21761/37962 [50:24:22<42:48:50,  9.51s/it]
 57%|█████▋    | 21762/37962 [50:24:30<41:12:50,  9.16s/it]
 57%|█████▋    | 21763/37962 [50:24:38<39:37:58,  8.81s/it]
 57%|█████▋    | 21764/37962 [50:24:46<38:44:58,  8.61s/it]
 57%|█████▋    | 21765/37962 [50:24:55<38:13:06,  8.49s/it]
 57%|█████▋    | 21766/37962 [50:25:03<38:01:23,  8.45s/it]
 57%|█████▋    | 21767/37962 [50:25:11<37:56:36,  8.43s/it]
 57%|█████▋    | 21768/37962 [50:25:20<37:49:05,  8.41s/it]
 57%|█████▋    | 21769/37962 [50:25:28<37:42:13,  8.38s/it]
 57%|█████▋    | 21770/37962 [50:25:37<37:57:00,  8.44s/it]
                                                           
{'loss': 1.3337, 'grad_norm': 6.879429340362549, 'learning_rate': 8.117706501648045e-06, 'epoch': 0.57}

 57%|█████▋    | 21770/37962 [50:25:37<37:57:00,  8.44s/it]
 57%|█████▋    | 21771/37962 [50:25:45<37:53:03,  8.42s/it]
 57%|█████▋    | 21772/37962 [50:25:53<37:30:23,  8.34s/it]
 57%|█████▋    | 21773/37962 [50:26:01<37:22:27,  8.31s/it]
 57%|█████▋    | 21774/37962 [50:26:10<37:38:33,  8.37s/it]
 57%|█████▋    | 21775/37962 [50:26:18<37:37:58,  8.37s/it]
 57%|█████▋    | 21776/37962 [50:26:27<37:46:36,  8.40s/it]
 57%|█████▋    | 21777/37962 [50:26:35<37:56:56,  8.44s/it]
 57%|█████▋    | 21778/37962 [50:26:44<37:47:58,  8.41s/it]
 57%|█████▋    | 21779/37962 [50:26:52<37:44:23,  8.40s/it]
 57%|█████▋    | 21780/37962 [50:27:00<37:33:34,  8.36s/it]
                                                           
{'loss': 1.3466, 'grad_norm': 6.050500869750977, 'learning_rate': 8.109328084977615e-06, 'epoch': 0.57}

 57%|█████▋    | 21780/37962 [50:27:00<37:33:34,  8.36s/it]
 57%|█████▋    | 21781/37962 [50:27:09<37:27:54,  8.34s/it]
 57%|█████▋    | 21782/37962 [50:27:17<37:13:01,  8.28s/it]
 57%|█████▋    | 21783/37962 [50:27:25<37:16:00,  8.29s/it]
 57%|█████▋    | 21784/37962 [50:27:33<37:25:30,  8.33s/it]
 57%|█████▋    | 21785/37962 [50:27:41<36:42:27,  8.17s/it]
 57%|█████▋    | 21786/37962 [50:27:49<36:42:22,  8.17s/it]
 57%|█████▋    | 21787/37962 [50:27:58<36:58:28,  8.23s/it]
 57%|█████▋    | 21788/37962 [50:28:06<37:07:55,  8.26s/it]
 57%|█████▋    | 21789/37962 [50:28:15<37:17:20,  8.30s/it]
 57%|█████▋    | 21790/37962 [50:28:23<37:11:57,  8.28s/it]
                                                           
{'loss': 1.3284, 'grad_norm': 5.530920505523682, 'learning_rate': 8.100951044494516e-06, 'epoch': 0.57}

 57%|█████▋    | 21790/37962 [50:28:23<37:11:57,  8.28s/it]
 57%|█████▋    | 21791/37962 [50:28:31<37:23:54,  8.33s/it]
 57%|█████▋    | 21792/37962 [50:28:39<37:20:35,  8.31s/it]
 57%|█████▋    | 21793/37962 [50:28:48<37:17:43,  8.30s/it]
 57%|█████▋    | 21794/37962 [50:28:56<37:09:28,  8.27s/it]
 57%|█████▋    | 21795/37962 [50:29:04<37:12:51,  8.29s/it]
 57%|█████▋    | 21796/37962 [50:29:13<37:14:40,  8.29s/it]
 57%|█████▋    | 21797/37962 [50:29:21<37:23:02,  8.33s/it]
 57%|█████▋    | 21798/37962 [50:29:29<37:02:04,  8.25s/it]
 57%|█████▋    | 21799/37962 [50:29:38<37:22:14,  8.32s/it]
 57%|█████▋    | 21800/37962 [50:29:46<37:31:11,  8.36s/it]
                                                           
{'loss': 1.4156, 'grad_norm': 5.658797264099121, 'learning_rate': 8.092575386296255e-06, 'epoch': 0.57}

 57%|█████▋    | 21800/37962 [50:29:46<37:31:11,  8.36s/it]
 57%|█████▋    | 21801/37962 [50:29:54<37:08:18,  8.27s/it]
 57%|█████▋    | 21802/37962 [50:30:02<37:13:24,  8.29s/it]
 57%|█████▋    | 21803/37962 [50:30:10<36:46:26,  8.19s/it]
 57%|█████▋    | 21804/37962 [50:30:19<37:35:48,  8.38s/it]
 57%|█████▋    | 21805/37962 [50:30:27<37:31:04,  8.36s/it]
 57%|█████▋    | 21806/37962 [50:30:36<37:29:39,  8.35s/it]
 57%|█████▋    | 21807/37962 [50:30:44<37:19:24,  8.32s/it]
 57%|█████▋    | 21808/37962 [50:30:52<37:24:52,  8.34s/it]
 57%|█████▋    | 21809/37962 [50:31:01<37:16:38,  8.31s/it]
 57%|█████▋    | 21810/37962 [50:31:09<37:22:26,  8.33s/it]
                                                           
{'loss': 1.3163, 'grad_norm': 9.025680541992188, 'learning_rate': 8.08420111647933e-06, 'epoch': 0.57}

 57%|█████▋    | 21810/37962 [50:31:09<37:22:26,  8.33s/it]
 57%|█████▋    | 21811/37962 [50:31:17<37:11:53,  8.29s/it]
 57%|█████▋    | 21812/37962 [50:31:26<37:10:55,  8.29s/it]
 57%|█████▋    | 21813/37962 [50:31:34<36:59:48,  8.25s/it]
 57%|█████▋    | 21814/37962 [50:31:42<37:19:57,  8.32s/it]
 57%|█████▋    | 21815/37962 [50:31:50<36:58:25,  8.24s/it]
 57%|█████▋    | 21816/37962 [50:31:58<36:45:53,  8.20s/it]
 57%|█████▋    | 21817/37962 [50:32:07<36:53:55,  8.23s/it]
 57%|█████▋    | 21818/37962 [50:32:15<36:48:52,  8.21s/it]
 57%|█████▋    | 21819/37962 [50:32:23<36:53:38,  8.23s/it]
 57%|█████▋    | 21820/37962 [50:32:31<37:07:54,  8.28s/it]
                                                           
{'loss': 1.2727, 'grad_norm': 9.012530326843262, 'learning_rate': 8.075828241139218e-06, 'epoch': 0.57}

 57%|█████▋    | 21820/37962 [50:32:31<37:07:54,  8.28s/it]
 57%|█████▋    | 21821/37962 [50:32:40<36:56:16,  8.24s/it]
 57%|█████▋    | 21822/37962 [50:32:48<37:10:42,  8.29s/it]
 57%|█████▋    | 21823/37962 [50:32:56<36:59:54,  8.25s/it]
 57%|█████▋    | 21824/37962 [50:33:04<36:57:43,  8.25s/it]
 57%|█████▋    | 21825/37962 [50:33:13<37:04:40,  8.27s/it]
 57%|█████▋    | 21826/37962 [50:33:21<37:03:41,  8.27s/it]
 57%|█████▋    | 21827/37962 [50:33:29<37:02:48,  8.27s/it]
 57%|█████▋    | 21828/37962 [50:33:38<37:08:32,  8.29s/it]
 58%|█████▊    | 21829/37962 [50:33:46<37:20:04,  8.33s/it]
 58%|█████▊    | 21830/37962 [50:33:55<37:40:47,  8.41s/it]
                                                           
{'loss': 1.3664, 'grad_norm': 5.21568489074707, 'learning_rate': 8.067456766370398e-06, 'epoch': 0.58}

 58%|█████▊    | 21830/37962 [50:33:55<37:40:47,  8.41s/it]
 58%|█████▊    | 21831/37962 [50:34:03<38:06:41,  8.51s/it]
 58%|█████▊    | 21832/37962 [50:34:12<37:42:01,  8.41s/it]
 58%|█████▊    | 21833/37962 [50:34:20<37:49:15,  8.44s/it]
 58%|█████▊    | 21834/37962 [50:34:28<37:43:48,  8.42s/it]
 58%|█████▊    | 21835/37962 [50:34:36<37:10:41,  8.30s/it]
 58%|█████▊    | 21836/37962 [50:34:45<37:07:40,  8.29s/it]
 58%|█████▊    | 21837/37962 [50:34:53<37:03:20,  8.27s/it]
 58%|█████▊    | 21838/37962 [50:35:01<37:16:55,  8.32s/it]
 58%|█████▊    | 21839/37962 [50:35:09<36:36:42,  8.17s/it]
 58%|█████▊    | 21840/37962 [50:35:18<36:48:25,  8.22s/it]
                                                           
{'loss': 1.3179, 'grad_norm': 7.612646102905273, 'learning_rate': 8.059086698266317e-06, 'epoch': 0.58}

 58%|█████▊    | 21840/37962 [50:35:18<36:48:25,  8.22s/it]
 58%|█████▊    | 21841/37962 [50:35:26<37:13:51,  8.31s/it]
 58%|█████▊    | 21842/37962 [50:35:35<37:23:18,  8.35s/it]
 58%|█████▊    | 21843/37962 [50:35:43<37:14:05,  8.32s/it]
 58%|█████▊    | 21844/37962 [50:35:51<37:07:42,  8.29s/it]
 58%|█████▊    | 21845/37962 [50:35:59<37:07:04,  8.29s/it]
 58%|█████▊    | 21846/37962 [50:36:08<37:58:47,  8.48s/it]
 58%|█████▊    | 21847/37962 [50:36:17<37:57:55,  8.48s/it]
 58%|█████▊    | 21848/37962 [50:36:25<37:46:15,  8.44s/it]
 58%|█████▊    | 21849/37962 [50:36:33<37:47:58,  8.45s/it]
 58%|█████▊    | 21850/37962 [50:36:42<37:26:49,  8.37s/it]
                                                           
{'loss': 1.3834, 'grad_norm': 5.655656337738037, 'learning_rate': 8.050718042919405e-06, 'epoch': 0.58}

 58%|█████▊    | 21850/37962 [50:36:42<37:26:49,  8.37s/it]
 58%|█████▊    | 21851/37962 [50:36:50<37:13:31,  8.32s/it]
 58%|█████▊    | 21852/37962 [50:36:58<37:20:51,  8.35s/it]
 58%|█████▊    | 21853/37962 [50:37:07<37:30:01,  8.38s/it]
 58%|█████▊    | 21854/37962 [50:37:15<37:24:01,  8.36s/it]
 58%|█████▊    | 21855/37962 [50:37:23<37:29:05,  8.38s/it]
 58%|█████▊    | 21856/37962 [50:37:32<37:17:46,  8.34s/it]
 58%|█████▊    | 21857/37962 [50:37:40<37:05:03,  8.29s/it]
 58%|█████▊    | 21858/37962 [50:37:48<37:10:16,  8.31s/it]
 58%|█████▊    | 21859/37962 [50:37:56<37:02:38,  8.28s/it]
 58%|█████▊    | 21860/37962 [50:38:05<36:54:43,  8.25s/it]
                                                           
{'loss': 1.3612, 'grad_norm': 10.597208023071289, 'learning_rate': 8.042350806421059e-06, 'epoch': 0.58}

 58%|█████▊    | 21860/37962 [50:38:05<36:54:43,  8.25s/it]
 58%|█████▊    | 21861/37962 [50:38:13<36:56:17,  8.26s/it]
 58%|█████▊    | 21862/37962 [50:38:21<36:32:26,  8.17s/it]
 58%|█████▊    | 21863/37962 [50:38:29<36:42:56,  8.21s/it]
 58%|█████▊    | 21864/37962 [50:38:37<36:43:14,  8.21s/it]
 58%|█████▊    | 21865/37962 [50:38:46<36:41:59,  8.21s/it]
 58%|█████▊    | 21866/37962 [50:38:54<36:41:30,  8.21s/it]
 58%|█████▊    | 21867/37962 [50:39:02<37:05:59,  8.30s/it]
 58%|█████▊    | 21868/37962 [50:39:11<37:18:41,  8.35s/it]
 58%|█████▊    | 21869/37962 [50:39:19<37:05:42,  8.30s/it]
 58%|█████▊    | 21870/37962 [50:39:27<37:06:18,  8.30s/it]
                                                           
{'loss': 1.235, 'grad_norm': 4.16780424118042, 'learning_rate': 8.033984994861653e-06, 'epoch': 0.58}

 58%|█████▊    | 21870/37962 [50:39:27<37:06:18,  8.30s/it]
 58%|█████▊    | 21871/37962 [50:39:36<37:10:58,  8.32s/it]
 58%|█████▊    | 21872/37962 [50:39:44<37:05:13,  8.30s/it]
 58%|█████▊    | 21873/37962 [50:39:52<37:10:52,  8.32s/it]
 58%|█████▊    | 21874/37962 [50:40:00<36:44:04,  8.22s/it]
 58%|█████▊    | 21875/37962 [50:40:09<36:51:39,  8.25s/it]
 58%|█████▊    | 21876/37962 [50:40:16<36:15:45,  8.12s/it]
 58%|█████▊    | 21877/37962 [50:40:25<36:34:41,  8.19s/it]
 58%|█████▊    | 21878/37962 [50:40:33<36:38:46,  8.20s/it]
 58%|█████▊    | 21879/37962 [50:40:41<36:53:11,  8.26s/it]
 58%|█████▊    | 21880/37962 [50:40:50<36:54:39,  8.26s/it]
                                                           
{'loss': 1.2194, 'grad_norm': 12.96108627319336, 'learning_rate': 8.025620614330506e-06, 'epoch': 0.58}

 58%|█████▊    | 21880/37962 [50:40:50<36:54:39,  8.26s/it]
 58%|█████▊    | 21881/37962 [50:40:58<36:59:59,  8.28s/it]
 58%|█████▊    | 21882/37962 [50:41:06<36:59:08,  8.28s/it]
 58%|█████▊    | 21883/37962 [50:41:14<36:43:09,  8.22s/it]
 58%|█████▊    | 21884/37962 [50:41:23<36:49:03,  8.24s/it]
 58%|█████▊    | 21885/37962 [50:41:31<36:46:37,  8.24s/it]
 58%|█████▊    | 21886/37962 [50:41:39<36:46:41,  8.24s/it]
 58%|█████▊    | 21887/37962 [50:41:47<36:21:50,  8.14s/it]
 58%|█████▊    | 21888/37962 [50:41:55<36:38:06,  8.20s/it]
 58%|█████▊    | 21889/37962 [50:42:04<36:39:20,  8.21s/it]
 58%|█████▊    | 21890/37962 [50:42:12<36:52:36,  8.26s/it]
                                                           
{'loss': 1.404, 'grad_norm': 14.43004322052002, 'learning_rate': 8.017257670915908e-06, 'epoch': 0.58}

 58%|█████▊    | 21890/37962 [50:42:12<36:52:36,  8.26s/it]
 58%|█████▊    | 21891/37962 [50:42:20<36:48:08,  8.24s/it]
 58%|█████▊    | 21892/37962 [50:42:28<36:48:36,  8.25s/it]
 58%|█████▊    | 21893/37962 [50:42:37<36:56:40,  8.28s/it]
 58%|█████▊    | 21894/37962 [50:42:45<36:53:59,  8.27s/it]
 58%|█████▊    | 21895/37962 [50:42:53<36:51:55,  8.26s/it]
 58%|█████▊    | 21896/37962 [50:43:02<36:58:25,  8.28s/it]
 58%|█████▊    | 21897/37962 [50:43:10<37:05:18,  8.31s/it]
 58%|█████▊    | 21898/37962 [50:43:19<37:36:52,  8.43s/it]
 58%|█████▊    | 21899/37962 [50:43:27<37:20:27,  8.37s/it]
 58%|█████▊    | 21900/37962 [50:43:35<37:18:41,  8.36s/it]
                                                           
{'loss': 1.2659, 'grad_norm': 5.399217128753662, 'learning_rate': 8.008896170705106e-06, 'epoch': 0.58}

 58%|█████▊    | 21900/37962 [50:43:35<37:18:41,  8.36s/it]
 58%|█████▊    | 21901/37962 [50:43:44<37:21:07,  8.37s/it]
 58%|█████▊    | 21902/37962 [50:43:52<37:09:48,  8.33s/it]
 58%|█████▊    | 21903/37962 [50:44:00<37:10:49,  8.33s/it]
 58%|█████▊    | 21904/37962 [50:44:09<37:09:15,  8.33s/it]
 58%|█████▊    | 21905/37962 [50:44:17<37:07:08,  8.32s/it]
 58%|█████▊    | 21906/37962 [50:44:25<37:08:31,  8.33s/it]
 58%|█████▊    | 21907/37962 [50:44:34<37:26:12,  8.39s/it]
 58%|█████▊    | 21908/37962 [50:44:42<37:06:19,  8.32s/it]
 58%|█████▊    | 21909/37962 [50:44:50<36:57:01,  8.29s/it]
 58%|█████▊    | 21910/37962 [50:44:58<36:50:35,  8.26s/it]
                                                           
{'loss': 1.3241, 'grad_norm': 6.441407203674316, 'learning_rate': 8.000536119784285e-06, 'epoch': 0.58}

 58%|█████▊    | 21910/37962 [50:44:58<36:50:35,  8.26s/it]
 58%|█████▊    | 21911/37962 [50:45:07<36:52:07,  8.27s/it]
 58%|█████▊    | 21912/37962 [50:45:15<36:55:37,  8.28s/it]
 58%|█████▊    | 21913/37962 [50:45:23<37:03:08,  8.31s/it]
 58%|█████▊    | 21914/37962 [50:45:32<37:01:10,  8.30s/it]
 58%|█████▊    | 21915/37962 [50:45:40<37:08:00,  8.33s/it]
 58%|█████▊    | 21916/37962 [50:45:48<37:19:48,  8.38s/it]
 58%|█████▊    | 21917/37962 [50:45:57<37:21:41,  8.38s/it]
 58%|█████▊    | 21918/37962 [50:46:05<37:20:31,  8.38s/it]
 58%|█████▊    | 21919/37962 [50:46:13<36:49:12,  8.26s/it]
 58%|█████▊    | 21920/37962 [50:46:21<36:48:56,  8.26s/it]
                                                           
{'loss': 1.299, 'grad_norm': 9.41280746459961, 'learning_rate': 7.992177524238588e-06, 'epoch': 0.58}

 58%|█████▊    | 21920/37962 [50:46:21<36:48:56,  8.26s/it]
 58%|█████▊    | 21921/37962 [50:46:30<36:45:32,  8.25s/it]
 58%|█████▊    | 21922/37962 [50:46:38<36:51:22,  8.27s/it]
 58%|█████▊    | 21923/37962 [50:46:46<36:45:28,  8.25s/it]
 58%|█████▊    | 21924/37962 [50:46:54<36:50:33,  8.27s/it]
 58%|█████▊    | 21925/37962 [50:47:03<36:52:50,  8.28s/it]
 58%|█████▊    | 21926/37962 [50:47:11<36:50:32,  8.27s/it]
 58%|█████▊    | 21927/37962 [50:47:19<37:04:12,  8.32s/it]
 58%|█████▊    | 21928/37962 [50:47:28<37:07:25,  8.34s/it]
 58%|█████▊    | 21929/37962 [50:47:36<36:49:11,  8.27s/it]
 58%|█████▊    | 21930/37962 [50:47:44<36:20:19,  8.16s/it]
                                                           
{'loss': 1.2155, 'grad_norm': 7.931628227233887, 'learning_rate': 7.983820390152086e-06, 'epoch': 0.58}

 58%|█████▊    | 21930/37962 [50:47:44<36:20:19,  8.16s/it]
 58%|█████▊    | 21931/37962 [50:47:52<36:15:45,  8.14s/it]
 58%|█████▊    | 21932/37962 [50:48:00<35:54:40,  8.06s/it]
 58%|█████▊    | 21933/37962 [50:48:08<36:08:12,  8.12s/it]
 58%|█████▊    | 21934/37962 [50:48:16<36:29:32,  8.20s/it]
 58%|█████▊    | 21935/37962 [50:48:25<36:51:21,  8.28s/it]
 58%|█████▊    | 21936/37962 [50:48:34<37:15:31,  8.37s/it]
 58%|█████▊    | 21937/37962 [50:48:42<37:06:35,  8.34s/it]
 58%|█████▊    | 21938/37962 [50:48:50<37:09:08,  8.35s/it]
 58%|█████▊    | 21939/37962 [50:48:58<36:54:15,  8.29s/it]
 58%|█████▊    | 21940/37962 [50:49:06<36:35:24,  8.22s/it]
                                                           
{'loss': 1.4778, 'grad_norm': 13.773015975952148, 'learning_rate': 7.975464723607794e-06, 'epoch': 0.58}

 58%|█████▊    | 21940/37962 [50:49:06<36:35:24,  8.22s/it]
 58%|█████▊    | 21941/37962 [50:49:15<36:50:12,  8.28s/it]
 58%|█████▊    | 21942/37962 [50:49:23<36:47:41,  8.27s/it]
 58%|█████▊    | 21943/37962 [50:49:31<36:51:26,  8.28s/it]
 58%|█████▊    | 21944/37962 [50:49:40<36:47:29,  8.27s/it]
 58%|█████▊    | 21945/37962 [50:49:48<36:27:50,  8.20s/it]
 58%|█████▊    | 21946/37962 [50:49:56<36:43:27,  8.25s/it]
 58%|█████▊    | 21947/37962 [50:50:04<36:48:46,  8.28s/it]
 58%|█████▊    | 21948/37962 [50:50:13<36:48:43,  8.28s/it]
 58%|█████▊    | 21949/37962 [50:50:21<36:48:03,  8.27s/it]
 58%|█████▊    | 21950/37962 [50:50:29<36:13:29,  8.14s/it]
                                                           
{'loss': 1.194, 'grad_norm': 9.561707496643066, 'learning_rate': 7.967110530687655e-06, 'epoch': 0.58}

 58%|█████▊    | 21950/37962 [50:50:29<36:13:29,  8.14s/it]
 58%|█████▊    | 21951/37962 [50:50:37<36:51:21,  8.29s/it]
 58%|█████▊    | 21952/37962 [50:50:46<36:48:17,  8.28s/it]
 58%|█████▊    | 21953/37962 [50:50:54<36:43:36,  8.26s/it]
 58%|█████▊    | 21954/37962 [50:51:02<36:45:13,  8.27s/it]
 58%|█████▊    | 21955/37962 [50:51:10<36:51:46,  8.29s/it]
 58%|█████▊    | 21956/37962 [50:51:19<36:51:26,  8.29s/it]
 58%|█████▊    | 21957/37962 [50:51:27<36:31:19,  8.21s/it]
 58%|█████▊    | 21958/37962 [50:51:35<36:31:13,  8.22s/it]
 58%|█████▊    | 21959/37962 [50:51:43<36:27:59,  8.20s/it]
 58%|█████▊    | 21960/37962 [50:51:51<36:20:42,  8.18s/it]
                                                           
{'loss': 1.3109, 'grad_norm': 10.50223159790039, 'learning_rate': 7.958757817472546e-06, 'epoch': 0.58}

 58%|█████▊    | 21960/37962 [50:51:51<36:20:42,  8.18s/it]
 58%|█████▊    | 21961/37962 [50:52:00<36:47:09,  8.28s/it]
 58%|█████▊    | 21962/37962 [50:52:08<36:38:32,  8.24s/it]
 58%|█████▊    | 21963/37962 [50:52:16<36:51:07,  8.29s/it]
 58%|█████▊    | 21964/37962 [50:52:25<36:46:15,  8.27s/it]
 58%|█████▊    | 21965/37962 [50:52:33<36:40:10,  8.25s/it]
 58%|█████▊    | 21966/37962 [50:52:41<36:41:52,  8.26s/it]
 58%|█████▊    | 21967/37962 [50:52:49<35:57:02,  8.09s/it]
 58%|█████▊    | 21968/37962 [50:52:57<36:08:18,  8.13s/it]
 58%|█████▊    | 21969/37962 [50:53:05<36:25:32,  8.20s/it]
 58%|█████▊    | 21970/37962 [50:53:14<36:30:53,  8.22s/it]
                                                           
{'loss': 1.3248, 'grad_norm': 6.874154567718506, 'learning_rate': 7.95040659004226e-06, 'epoch': 0.58}

 58%|█████▊    | 21970/37962 [50:53:14<36:30:53,  8.22s/it]
 58%|█████▊    | 21971/37962 [50:53:22<36:43:09,  8.27s/it]
 58%|█████▊    | 21972/37962 [50:53:30<36:51:45,  8.30s/it]
 58%|█████▊    | 21973/37962 [50:53:39<36:51:27,  8.30s/it]
 58%|█████▊    | 21974/37962 [50:53:47<36:54:06,  8.31s/it]
 58%|█████▊    | 21975/37962 [50:53:55<36:47:38,  8.29s/it]
 58%|█████▊    | 21976/37962 [50:54:04<36:47:26,  8.29s/it]
 58%|█████▊    | 21977/37962 [50:54:12<36:51:15,  8.30s/it]
 58%|█████▊    | 21978/37962 [50:54:20<36:27:02,  8.21s/it]
 58%|█████▊    | 21979/37962 [50:54:28<36:30:25,  8.22s/it]
 58%|█████▊    | 21980/37962 [50:54:36<36:23:30,  8.20s/it]
                                                           
{'loss': 1.2976, 'grad_norm': 8.245305061340332, 'learning_rate': 7.94205685447551e-06, 'epoch': 0.58}

 58%|█████▊    | 21980/37962 [50:54:36<36:23:30,  8.20s/it]
 58%|█████▊    | 21981/37962 [50:54:44<36:03:23,  8.12s/it]
 58%|█████▊    | 21982/37962 [50:54:52<36:14:53,  8.17s/it]
 58%|█████▊    | 21983/37962 [50:55:00<36:00:38,  8.11s/it]
 58%|█████▊    | 21984/37962 [50:55:08<35:40:37,  8.04s/it]
 58%|█████▊    | 21985/37962 [50:55:16<35:42:20,  8.05s/it]
 58%|█████▊    | 21986/37962 [50:55:25<36:25:08,  8.21s/it]
 58%|█████▊    | 21987/37962 [50:55:33<36:36:44,  8.25s/it]
 58%|█████▊    | 21988/37962 [50:55:42<36:46:19,  8.29s/it]
 58%|█████▊    | 21989/37962 [50:55:50<36:49:59,  8.30s/it]
 58%|█████▊    | 21990/37962 [50:55:58<36:47:30,  8.29s/it]
                                                           
{'loss': 1.2739, 'grad_norm': 5.68314790725708, 'learning_rate': 7.933708616849923e-06, 'epoch': 0.58}

 58%|█████▊    | 21990/37962 [50:55:58<36:47:30,  8.29s/it]
 58%|█████▊    | 21991/37962 [50:56:07<36:43:41,  8.28s/it]
 58%|█████▊    | 21992/37962 [50:56:15<36:40:58,  8.27s/it]
 58%|█████▊    | 21993/37962 [50:56:23<36:36:08,  8.25s/it]
 58%|█████▊    | 21994/37962 [50:56:31<36:44:58,  8.29s/it]
 58%|█████▊    | 21995/37962 [50:56:40<37:07:39,  8.37s/it]
 58%|█████▊    | 21996/37962 [50:56:49<37:25:32,  8.44s/it]
 58%|█████▊    | 21997/37962 [50:56:57<37:10:19,  8.38s/it]
 58%|█████▊    | 21998/37962 [50:57:05<37:06:46,  8.37s/it]
 58%|█████▊    | 21999/37962 [50:57:13<37:04:50,  8.36s/it]
 58%|█████▊    | 22000/37962 [50:57:22<36:47:54,  8.30s/it]
                                                           
{'loss': 1.3234, 'grad_norm': 6.084063529968262, 'learning_rate': 7.925361883242036e-06, 'epoch': 0.58}

 58%|█████▊    | 22000/37962 [50:57:22<36:47:54,  8.30s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 58%|█████▊    | 22001/37962 [50:58:13<93:44:29, 21.14s/it]
 58%|█████▊    | 22002/37962 [50:58:21<76:31:41, 17.26s/it]
 58%|█████▊    | 22003/37962 [50:58:29<64:36:29, 14.57s/it]
 58%|█████▊    | 22004/37962 [50:58:37<56:11:59, 12.68s/it]
 58%|█████▊    | 22005/37962 [50:58:46<50:00:38, 11.28s/it]
 58%|█████▊    | 22006/37962 [50:58:54<46:07:16, 10.41s/it]
 58%|█████▊    | 22007/37962 [50:59:02<43:18:38,  9.77s/it]
 58%|█████▊    | 22008/37962 [50:59:10<41:23:38,  9.34s/it]
 58%|█████▊    | 22009/37962 [50:59:19<40:02:17,  9.04s/it]
 58%|█████▊    | 22010/37962 [50:59:27<38:54:23,  8.78s/it]
                                                           
{'loss': 1.3888, 'grad_norm': 12.827095031738281, 'learning_rate': 7.917016659727297e-06, 'epoch': 0.58}

 58%|█████▊    | 22010/37962 [50:59:27<38:54:23,  8.78s/it]
 58%|█████▊    | 22011/37962 [50:59:35<38:28:43,  8.68s/it]
 58%|█████▊    | 22012/37962 [50:59:44<37:50:15,  8.54s/it]
 58%|█████▊    | 22013/37962 [50:59:52<37:16:06,  8.41s/it]
 58%|█████▊    | 22014/37962 [51:00:00<37:13:42,  8.40s/it]
 58%|█████▊    | 22015/37962 [51:00:08<37:01:11,  8.36s/it]
 58%|█████▊    | 22016/37962 [51:00:17<36:49:57,  8.32s/it]
 58%|█████▊    | 22017/37962 [51:00:25<36:44:11,  8.29s/it]
 58%|█████▊    | 22018/37962 [51:00:33<36:37:07,  8.27s/it]
 58%|█████▊    | 22019/37962 [51:00:42<37:00:54,  8.36s/it]
 58%|█████▊    | 22020/37962 [51:00:49<36:16:08,  8.19s/it]
                                                           
{'loss': 1.3301, 'grad_norm': 9.698895454406738, 'learning_rate': 7.908672952380045e-06, 'epoch': 0.58}

 58%|█████▊    | 22020/37962 [51:00:49<36:16:08,  8.19s/it]
 58%|█████▊    | 22021/37962 [51:00:58<36:17:52,  8.20s/it]
 58%|█████▊    | 22022/37962 [51:01:06<36:31:46,  8.25s/it]
 58%|█████▊    | 22023/37962 [51:01:14<36:27:18,  8.23s/it]
 58%|█████▊    | 22024/37962 [51:01:23<36:38:59,  8.28s/it]
 58%|█████▊    | 22025/37962 [51:01:31<36:22:50,  8.22s/it]
 58%|█████▊    | 22026/37962 [51:01:39<36:22:52,  8.22s/it]
 58%|█████▊    | 22027/37962 [51:01:47<36:29:43,  8.24s/it]
 58%|█████▊    | 22028/37962 [51:01:56<36:42:12,  8.29s/it]
 58%|█████▊    | 22029/37962 [51:02:04<36:43:40,  8.30s/it]
 58%|█████▊    | 22030/37962 [51:02:12<36:51:07,  8.33s/it]
                                                           
{'loss': 1.2687, 'grad_norm': 7.428830623626709, 'learning_rate': 7.900330767273521e-06, 'epoch': 0.58}

 58%|█████▊    | 22030/37962 [51:02:12<36:51:07,  8.33s/it]
 58%|█████▊    | 22031/37962 [51:02:21<36:57:34,  8.35s/it]
 58%|█████▊    | 22032/37962 [51:02:29<36:52:15,  8.33s/it]
 58%|█████▊    | 22033/37962 [51:02:37<36:54:37,  8.34s/it]
 58%|█████▊    | 22034/37962 [51:02:46<36:53:29,  8.34s/it]
 58%|█████▊    | 22035/37962 [51:02:54<36:31:19,  8.26s/it]
 58%|█████▊    | 22036/37962 [51:03:02<36:38:57,  8.28s/it]
 58%|█████▊    | 22037/37962 [51:03:10<36:33:32,  8.26s/it]
 58%|█████▊    | 22038/37962 [51:03:18<36:17:40,  8.21s/it]
 58%|█████▊    | 22039/37962 [51:03:26<36:06:58,  8.17s/it]
 58%|█████▊    | 22040/37962 [51:03:35<36:26:58,  8.24s/it]
                                                           
{'loss': 1.3227, 'grad_norm': 9.695491790771484, 'learning_rate': 7.891990110479858e-06, 'epoch': 0.58}

 58%|█████▊    | 22040/37962 [51:03:35<36:26:58,  8.24s/it]
 58%|█████▊    | 22041/37962 [51:03:43<36:35:41,  8.27s/it]
 58%|█████▊    | 22042/37962 [51:03:51<36:27:38,  8.24s/it]
 58%|█████▊    | 22043/37962 [51:04:00<36:27:16,  8.24s/it]
 58%|█████▊    | 22044/37962 [51:04:08<36:34:49,  8.27s/it]
 58%|█████▊    | 22045/37962 [51:04:16<36:46:15,  8.32s/it]
 58%|█████▊    | 22046/37962 [51:04:25<36:44:37,  8.31s/it]
 58%|█████▊    | 22047/37962 [51:04:33<36:42:29,  8.30s/it]
 58%|█████▊    | 22048/37962 [51:04:41<36:30:02,  8.26s/it]
 58%|█████▊    | 22049/37962 [51:04:49<36:20:26,  8.22s/it]
 58%|█████▊    | 22050/37962 [51:04:57<36:01:09,  8.15s/it]
                                                           
{'loss': 1.3641, 'grad_norm': 6.558418273925781, 'learning_rate': 7.883650988070074e-06, 'epoch': 0.58}

 58%|█████▊    | 22050/37962 [51:04:57<36:01:09,  8.15s/it]
 58%|█████▊    | 22051/37962 [51:05:05<35:36:07,  8.06s/it]
 58%|█████▊    | 22052/37962 [51:05:14<36:12:09,  8.19s/it]
 58%|█████▊    | 22053/37962 [51:05:22<36:27:38,  8.25s/it]
 58%|█████▊    | 22054/37962 [51:05:30<36:31:09,  8.26s/it]
 58%|█████▊    | 22055/37962 [51:05:39<36:30:55,  8.26s/it]
 58%|█████▊    | 22056/37962 [51:05:47<36:27:06,  8.25s/it]
 58%|█████▊    | 22057/37962 [51:05:55<36:25:04,  8.24s/it]
 58%|█████▊    | 22058/37962 [51:06:03<36:30:17,  8.26s/it]
 58%|█████▊    | 22059/37962 [51:06:12<36:29:12,  8.26s/it]
 58%|█████▊    | 22060/37962 [51:06:20<36:25:43,  8.25s/it]
                                                           
{'loss': 1.4201, 'grad_norm': 12.280397415161133, 'learning_rate': 7.875313406114073e-06, 'epoch': 0.58}

 58%|█████▊    | 22060/37962 [51:06:20<36:25:43,  8.25s/it]
 58%|█████▊    | 22061/37962 [51:06:28<36:01:06,  8.15s/it]
 58%|█████▊    | 22062/37962 [51:06:36<36:13:28,  8.20s/it]
 58%|█████▊    | 22063/37962 [51:06:44<36:19:30,  8.23s/it]
 58%|█████▊    | 22064/37962 [51:06:53<36:32:45,  8.28s/it]
 58%|█████▊    | 22065/37962 [51:07:01<36:45:45,  8.33s/it]
 58%|█████▊    | 22066/37962 [51:07:09<36:20:16,  8.23s/it]
 58%|█████▊    | 22067/37962 [51:07:17<36:17:42,  8.22s/it]
 58%|█████▊    | 22068/37962 [51:07:25<35:53:55,  8.13s/it]
 58%|█████▊    | 22069/37962 [51:07:34<36:03:10,  8.17s/it]
 58%|█████▊    | 22070/37962 [51:07:42<36:23:32,  8.24s/it]
                                                           
{'loss': 1.301, 'grad_norm': 4.086822032928467, 'learning_rate': 7.866977370680633e-06, 'epoch': 0.58}

 58%|█████▊    | 22070/37962 [51:07:42<36:23:32,  8.24s/it]
 58%|█████▊    | 22071/37962 [51:07:50<36:00:29,  8.16s/it]
 58%|█████▊    | 22072/37962 [51:07:58<36:11:08,  8.20s/it]
 58%|█████▊    | 22073/37962 [51:08:06<36:02:51,  8.17s/it]
 58%|█████▊    | 22074/37962 [51:08:14<35:46:07,  8.10s/it]
 58%|█████▊    | 22075/37962 [51:08:23<36:01:17,  8.16s/it]
 58%|█████▊    | 22076/37962 [51:08:31<36:12:48,  8.21s/it]
 58%|█████▊    | 22077/37962 [51:08:39<36:20:52,  8.24s/it]
 58%|█████▊    | 22078/37962 [51:08:47<36:16:10,  8.22s/it]
 58%|█████▊    | 22079/37962 [51:08:56<36:17:50,  8.23s/it]
 58%|█████▊    | 22080/37962 [51:09:04<36:19:20,  8.23s/it]
                                                           
{'loss': 1.3656, 'grad_norm': 4.18739652633667, 'learning_rate': 7.85864288783742e-06, 'epoch': 0.58}

 58%|█████▊    | 22080/37962 [51:09:04<36:19:20,  8.23s/it]
 58%|█████▊    | 22081/37962 [51:09:12<36:29:00,  8.27s/it]
 58%|█████▊    | 22082/37962 [51:09:20<36:19:20,  8.23s/it]
 58%|█████▊    | 22083/37962 [51:09:29<36:28:04,  8.27s/it]
 58%|█████▊    | 22084/37962 [51:09:37<36:30:20,  8.28s/it]
 58%|█████▊    | 22085/37962 [51:09:45<36:39:06,  8.31s/it]
 58%|█████▊    | 22086/37962 [51:09:54<36:34:02,  8.29s/it]
 58%|█████▊    | 22087/37962 [51:10:02<36:38:28,  8.31s/it]
 58%|█████▊    | 22088/37962 [51:10:10<36:34:19,  8.29s/it]
 58%|█████▊    | 22089/37962 [51:10:18<35:59:44,  8.16s/it]
 58%|█████▊    | 22090/37962 [51:10:26<36:12:51,  8.21s/it]
                                                           
{'loss': 1.3468, 'grad_norm': 4.360587120056152, 'learning_rate': 7.850309963650947e-06, 'epoch': 0.58}

 58%|█████▊    | 22090/37962 [51:10:26<36:12:51,  8.21s/it]
 58%|█████▊    | 22091/37962 [51:10:35<36:19:04,  8.24s/it]
 58%|█████▊    | 22092/37962 [51:10:43<36:32:29,  8.29s/it]
 58%|█████▊    | 22093/37962 [51:10:51<36:22:52,  8.25s/it]
 58%|█████▊    | 22094/37962 [51:11:00<36:25:00,  8.26s/it]
 58%|█████▊    | 22095/37962 [51:11:08<36:13:06,  8.22s/it]
 58%|█████▊    | 22096/37962 [51:11:16<36:27:20,  8.27s/it]
 58%|█████▊    | 22097/37962 [51:11:24<36:10:12,  8.21s/it]
 58%|█████▊    | 22098/37962 [51:11:33<36:22:41,  8.26s/it]
 58%|█████▊    | 22099/37962 [51:11:40<35:54:44,  8.15s/it]
 58%|█████▊    | 22100/37962 [51:11:49<36:01:18,  8.18s/it]
                                                           
{'loss': 1.3492, 'grad_norm': 11.275213241577148, 'learning_rate': 7.84197860418661e-06, 'epoch': 0.58}

 58%|█████▊    | 22100/37962 [51:11:49<36:01:18,  8.18s/it]
 58%|█████▊    | 22101/37962 [51:11:57<36:28:24,  8.28s/it]
 58%|█████▊    | 22102/37962 [51:12:06<36:34:48,  8.30s/it]
 58%|█████▊    | 22103/37962 [51:12:14<36:45:51,  8.35s/it]
 58%|█████▊    | 22104/37962 [51:12:22<36:50:43,  8.36s/it]
 58%|█████▊    | 22105/37962 [51:12:31<36:59:28,  8.40s/it]
 58%|█████▊    | 22106/37962 [51:12:39<36:49:05,  8.36s/it]
 58%|█████▊    | 22107/37962 [51:12:47<36:17:41,  8.24s/it]
 58%|█████▊    | 22108/37962 [51:12:55<36:22:25,  8.26s/it]
 58%|█████▊    | 22109/37962 [51:13:04<36:17:41,  8.24s/it]
 58%|█████▊    | 22110/37962 [51:13:12<35:57:25,  8.17s/it]
                                                           
{'loss': 1.3176, 'grad_norm': 3.603940963745117, 'learning_rate': 7.833648815508663e-06, 'epoch': 0.58}

 58%|█████▊    | 22110/37962 [51:13:12<35:57:25,  8.17s/it]
 58%|█████▊    | 22111/37962 [51:13:20<35:48:45,  8.13s/it]
 58%|█████▊    | 22112/37962 [51:13:28<35:57:03,  8.17s/it]
 58%|█████▊    | 22113/37962 [51:13:36<35:32:49,  8.07s/it]
 58%|█████▊    | 22114/37962 [51:13:44<35:55:29,  8.16s/it]
 58%|█████▊    | 22115/37962 [51:13:52<35:59:30,  8.18s/it]
 58%|█████▊    | 22116/37962 [51:14:01<36:06:30,  8.20s/it]
 58%|█████▊    | 22117/37962 [51:14:09<36:13:11,  8.23s/it]
 58%|█████▊    | 22118/37962 [51:14:17<36:13:09,  8.23s/it]
 58%|█████▊    | 22119/37962 [51:14:25<36:12:39,  8.23s/it]
 58%|█████▊    | 22120/37962 [51:14:34<36:15:04,  8.24s/it]
                                                           
{'loss': 1.3373, 'grad_norm': 5.707281112670898, 'learning_rate': 7.825320603680213e-06, 'epoch': 0.58}

 58%|█████▊    | 22120/37962 [51:14:34<36:15:04,  8.24s/it]
 58%|█████▊    | 22121/37962 [51:14:49<46:10:00, 10.49s/it]
 58%|█████▊    | 22122/37962 [51:14:57<43:01:00,  9.78s/it]
 58%|█████▊    | 22123/37962 [51:15:06<41:02:04,  9.33s/it]
 58%|█████▊    | 22124/37962 [51:15:14<39:41:30,  9.02s/it]
 58%|█████▊    | 22125/37962 [51:15:22<38:32:36,  8.76s/it]
 58%|█████▊    | 22126/37962 [51:15:31<37:57:39,  8.63s/it]
 58%|█████▊    | 22127/37962 [51:15:39<37:32:29,  8.53s/it]
 58%|█████▊    | 22128/37962 [51:15:47<37:10:32,  8.45s/it]
 58%|█████▊    | 22129/37962 [51:15:55<36:24:45,  8.28s/it]
 58%|█████▊    | 22130/37962 [51:16:03<36:19:48,  8.26s/it]
                                                           
{'loss': 1.2925, 'grad_norm': 2.9216716289520264, 'learning_rate': 7.816993974763224e-06, 'epoch': 0.58}

 58%|█████▊    | 22130/37962 [51:16:03<36:19:48,  8.26s/it]
 58%|█████▊    | 22131/37962 [51:16:12<36:41:44,  8.34s/it]
 58%|█████▊    | 22132/37962 [51:16:20<36:08:21,  8.22s/it]
 58%|█████▊    | 22133/37962 [51:16:27<35:25:34,  8.06s/it]
 58%|█████▊    | 22134/37962 [51:16:36<35:43:51,  8.13s/it]
 58%|█████▊    | 22135/37962 [51:16:44<35:36:01,  8.10s/it]
 58%|█████▊    | 22136/37962 [51:16:52<35:43:03,  8.12s/it]
 58%|█████▊    | 22137/37962 [51:17:00<35:56:50,  8.18s/it]
 58%|█████▊    | 22138/37962 [51:17:09<36:29:20,  8.30s/it]
 58%|█████▊    | 22139/37962 [51:17:17<36:23:48,  8.28s/it]
 58%|█████▊    | 22140/37962 [51:17:25<36:20:46,  8.27s/it]
                                                           
{'loss': 1.2506, 'grad_norm': 7.1156158447265625, 'learning_rate': 7.808668934818503e-06, 'epoch': 0.58}

 58%|█████▊    | 22140/37962 [51:17:25<36:20:46,  8.27s/it]
 58%|█████▊    | 22141/37962 [51:17:34<36:25:51,  8.29s/it]
 58%|█████▊    | 22142/37962 [51:17:42<36:18:54,  8.26s/it]
 58%|█████▊    | 22143/37962 [51:17:50<36:27:21,  8.30s/it]
 58%|█████▊    | 22144/37962 [51:17:59<36:40:10,  8.35s/it]
 58%|█████▊    | 22145/37962 [51:18:07<36:42:52,  8.36s/it]
 58%|█████▊    | 22146/37962 [51:18:15<36:02:06,  8.20s/it]
 58%|█████▊    | 22147/37962 [51:18:23<35:37:57,  8.11s/it]
 58%|█████▊    | 22148/37962 [51:18:31<35:55:04,  8.18s/it]
 58%|█████▊    | 22149/37962 [51:18:39<35:59:58,  8.20s/it]
 58%|█████▊    | 22150/37962 [51:18:48<36:07:30,  8.22s/it]
                                                           
{'loss': 1.339, 'grad_norm': 9.864612579345703, 'learning_rate': 7.8003454899057e-06, 'epoch': 0.58}

 58%|█████▊    | 22150/37962 [51:18:48<36:07:30,  8.22s/it]
 58%|█████▊    | 22151/37962 [51:18:56<36:04:39,  8.21s/it]
 58%|█████▊    | 22152/37962 [51:19:04<36:08:06,  8.23s/it]
 58%|█████▊    | 22153/37962 [51:19:12<36:10:02,  8.24s/it]
 58%|█████▊    | 22154/37962 [51:19:21<36:10:16,  8.24s/it]
 58%|█████▊    | 22155/37962 [51:19:29<36:12:15,  8.25s/it]
 58%|█████▊    | 22156/37962 [51:19:37<36:13:28,  8.25s/it]
 58%|█████▊    | 22157/37962 [51:19:45<36:26:44,  8.30s/it]
 58%|█████▊    | 22158/37962 [51:19:54<36:51:46,  8.40s/it]
 58%|█████▊    | 22159/37962 [51:20:02<36:49:10,  8.39s/it]
 58%|█████▊    | 22160/37962 [51:20:11<36:38:32,  8.35s/it]
                                                           
{'loss': 1.2476, 'grad_norm': 10.434876441955566, 'learning_rate': 7.79202364608331e-06, 'epoch': 0.58}

 58%|█████▊    | 22160/37962 [51:20:11<36:38:32,  8.35s/it]
 58%|█████▊    | 22161/37962 [51:20:19<36:09:56,  8.24s/it]
 58%|█████▊    | 22162/37962 [51:20:27<36:14:16,  8.26s/it]
 58%|█████▊    | 22163/37962 [51:20:35<36:30:46,  8.32s/it]
 58%|█████▊    | 22164/37962 [51:20:44<36:17:48,  8.27s/it]
 58%|█████▊    | 22165/37962 [51:20:52<36:09:04,  8.24s/it]
 58%|█████▊    | 22166/37962 [51:21:00<36:13:15,  8.25s/it]
 58%|█████▊    | 22167/37962 [51:21:08<36:02:21,  8.21s/it]
 58%|█████▊    | 22168/37962 [51:21:16<36:05:20,  8.23s/it]
 58%|█████▊    | 22169/37962 [51:21:25<36:10:15,  8.25s/it]
 58%|█████▊    | 22170/37962 [51:21:33<36:24:04,  8.30s/it]
                                                           
{'loss': 1.3085, 'grad_norm': 7.231067180633545, 'learning_rate': 7.783703409408655e-06, 'epoch': 0.58}

 58%|█████▊    | 22170/37962 [51:21:33<36:24:04,  8.30s/it]
 58%|█████▊    | 22171/37962 [51:21:41<36:23:10,  8.30s/it]
 58%|█████▊    | 22172/37962 [51:21:50<36:10:43,  8.25s/it]
 58%|█████▊    | 22173/37962 [51:21:58<36:12:18,  8.26s/it]
 58%|█████▊    | 22174/37962 [51:22:06<35:55:08,  8.19s/it]
 58%|█████▊    | 22175/37962 [51:22:14<36:06:05,  8.23s/it]
 58%|█████▊    | 22176/37962 [51:22:22<36:00:36,  8.21s/it]
 58%|█████▊    | 22177/37962 [51:22:31<36:02:01,  8.22s/it]
 58%|█████▊    | 22178/37962 [51:22:39<36:15:03,  8.27s/it]
 58%|█████▊    | 22179/37962 [51:22:48<36:34:15,  8.34s/it]
 58%|█████▊    | 22180/37962 [51:22:56<36:24:02,  8.30s/it]
                                                           
{'loss': 1.235, 'grad_norm': 11.487489700317383, 'learning_rate': 7.775384785937899e-06, 'epoch': 0.58}

 58%|█████▊    | 22180/37962 [51:22:56<36:24:02,  8.30s/it]
 58%|█████▊    | 22181/37962 [51:23:04<36:32:32,  8.34s/it]
 58%|█████▊    | 22182/37962 [51:23:12<36:14:34,  8.27s/it]
 58%|█████▊    | 22183/37962 [51:23:21<36:21:16,  8.29s/it]
 58%|█████▊    | 22184/37962 [51:23:29<36:24:56,  8.31s/it]
 58%|█████▊    | 22185/37962 [51:23:37<36:20:11,  8.29s/it]
 58%|█████▊    | 22186/37962 [51:23:45<35:44:46,  8.16s/it]
 58%|█████▊    | 22187/37962 [51:23:53<35:56:08,  8.20s/it]
 58%|█████▊    | 22188/37962 [51:24:02<35:59:23,  8.21s/it]
 58%|█████▊    | 22189/37962 [51:24:10<36:03:00,  8.23s/it]
 58%|█████▊    | 22190/37962 [51:24:18<36:11:42,  8.26s/it]
                                                           
{'loss': 1.3632, 'grad_norm': 6.1279144287109375, 'learning_rate': 7.767067781726016e-06, 'epoch': 0.58}

 58%|█████▊    | 22190/37962 [51:24:18<36:11:42,  8.26s/it]
 58%|█████▊    | 22191/37962 [51:24:26<35:56:29,  8.20s/it]
 58%|█████▊    | 22192/37962 [51:24:35<36:34:06,  8.35s/it]
 58%|█████▊    | 22193/37962 [51:24:43<36:13:08,  8.27s/it]
 58%|█████▊    | 22194/37962 [51:24:51<36:09:19,  8.25s/it]
 58%|█████▊    | 22195/37962 [51:24:59<36:01:40,  8.23s/it]
 58%|█████▊    | 22196/37962 [51:25:08<36:17:37,  8.29s/it]
 58%|█████▊    | 22197/37962 [51:25:16<36:24:57,  8.32s/it]
 58%|█████▊    | 22198/37962 [51:25:25<36:23:19,  8.31s/it]
 58%|█████▊    | 22199/37962 [51:25:33<36:00:19,  8.22s/it]
 58%|█████▊    | 22200/37962 [51:25:41<36:15:19,  8.28s/it]
                                                           
{'loss': 1.1362, 'grad_norm': 6.82244348526001, 'learning_rate': 7.758752402826808e-06, 'epoch': 0.58}

 58%|█████▊    | 22200/37962 [51:25:41<36:15:19,  8.28s/it]
 58%|█████▊    | 22201/37962 [51:25:50<36:45:00,  8.39s/it]
 58%|█████▊    | 22202/37962 [51:25:58<36:32:36,  8.35s/it]
 58%|█████▊    | 22203/37962 [51:26:06<36:34:40,  8.36s/it]
 58%|█████▊    | 22204/37962 [51:26:15<36:41:26,  8.38s/it]
 58%|█████▊    | 22205/37962 [51:26:22<35:44:44,  8.17s/it]
 58%|█████▊    | 22206/37962 [51:26:31<35:47:22,  8.18s/it]
 58%|█████▊    | 22207/37962 [51:26:39<36:04:10,  8.24s/it]
 59%|█████▊    | 22208/37962 [51:26:47<35:47:50,  8.18s/it]
 59%|█████▊    | 22209/37962 [51:26:55<35:53:41,  8.20s/it]
 59%|█████▊    | 22210/37962 [51:27:04<35:58:44,  8.22s/it]
                                                           
{'loss': 1.2972, 'grad_norm': 5.009152412414551, 'learning_rate': 7.750438655292901e-06, 'epoch': 0.59}

 59%|█████▊    | 22210/37962 [51:27:04<35:58:44,  8.22s/it]
 59%|█████▊    | 22211/37962 [51:27:12<35:52:06,  8.20s/it]
 59%|█████▊    | 22212/37962 [51:27:20<36:07:49,  8.26s/it]
 59%|█████▊    | 22213/37962 [51:27:28<36:12:37,  8.28s/it]
 59%|█████▊    | 22214/37962 [51:27:36<35:45:56,  8.18s/it]
 59%|█████▊    | 22215/37962 [51:27:44<35:46:49,  8.18s/it]
 59%|█████▊    | 22216/37962 [51:27:53<36:01:56,  8.24s/it]
 59%|█████▊    | 22217/37962 [51:28:01<36:14:14,  8.29s/it]
 59%|█████▊    | 22218/37962 [51:28:09<36:07:06,  8.26s/it]
 59%|█████▊    | 22219/37962 [51:28:18<36:03:14,  8.24s/it]
 59%|█████▊    | 22220/37962 [51:28:26<36:00:05,  8.23s/it]
                                                           
{'loss': 1.2455, 'grad_norm': 4.370853900909424, 'learning_rate': 7.742126545175727e-06, 'epoch': 0.59}

 59%|█████▊    | 22220/37962 [51:28:26<36:00:05,  8.23s/it]
 59%|█████▊    | 22221/37962 [51:28:34<36:15:20,  8.29s/it]
 59%|█████▊    | 22222/37962 [51:28:43<36:08:15,  8.27s/it]
 59%|█████▊    | 22223/37962 [51:28:51<35:56:50,  8.22s/it]
 59%|█████▊    | 22224/37962 [51:28:59<35:52:42,  8.21s/it]
 59%|█████▊    | 22225/37962 [51:29:07<36:05:31,  8.26s/it]
 59%|█████▊    | 22226/37962 [51:29:15<36:03:53,  8.25s/it]
 59%|█████▊    | 22227/37962 [51:29:24<36:01:51,  8.24s/it]
 59%|█████▊    | 22228/37962 [51:29:32<36:08:55,  8.27s/it]
 59%|█████▊    | 22229/37962 [51:29:40<35:46:39,  8.19s/it]
 59%|█████▊    | 22230/37962 [51:29:48<36:06:22,  8.26s/it]
                                                           
{'loss': 1.2228, 'grad_norm': 12.881572723388672, 'learning_rate': 7.733816078525527e-06, 'epoch': 0.59}

 59%|█████▊    | 22230/37962 [51:29:48<36:06:22,  8.26s/it]
 59%|█████▊    | 22231/37962 [51:29:57<36:10:04,  8.28s/it]
 59%|█████▊    | 22232/37962 [51:30:05<36:24:40,  8.33s/it]
 59%|█████▊    | 22233/37962 [51:30:13<35:55:06,  8.22s/it]
 59%|█████▊    | 22234/37962 [51:30:22<36:07:36,  8.27s/it]
 59%|█████▊    | 22235/37962 [51:30:30<36:26:43,  8.34s/it]
 59%|█████▊    | 22236/37962 [51:30:38<36:05:07,  8.26s/it]
 59%|█████▊    | 22237/37962 [51:30:46<35:59:58,  8.24s/it]
 59%|█████▊    | 22238/37962 [51:30:55<36:22:33,  8.33s/it]
 59%|█████▊    | 22239/37962 [51:31:03<36:13:01,  8.29s/it]
 59%|█████▊    | 22240/37962 [51:31:11<36:07:22,  8.27s/it]
                                                           
{'loss': 1.2054, 'grad_norm': 6.866861820220947, 'learning_rate': 7.725507261391344e-06, 'epoch': 0.59}

 59%|█████▊    | 22240/37962 [51:31:11<36:07:22,  8.27s/it]
 59%|█████▊    | 22241/37962 [51:31:19<35:48:55,  8.20s/it]
 59%|█████▊    | 22242/37962 [51:31:28<35:58:12,  8.24s/it]
 59%|█████▊    | 22243/37962 [51:31:36<35:36:14,  8.15s/it]
 59%|█████▊    | 22244/37962 [51:31:44<35:41:39,  8.18s/it]
 59%|█████▊    | 22245/37962 [51:31:52<35:49:54,  8.21s/it]
 59%|█████▊    | 22246/37962 [51:32:00<36:02:25,  8.26s/it]
 59%|█████▊    | 22247/37962 [51:32:09<35:46:02,  8.19s/it]
 59%|█████▊    | 22248/37962 [51:32:17<35:51:44,  8.22s/it]
 59%|█████▊    | 22249/37962 [51:32:25<35:58:36,  8.24s/it]
 59%|█████▊    | 22250/37962 [51:32:33<36:03:19,  8.26s/it]
                                                           
{'loss': 1.286, 'grad_norm': 9.818111419677734, 'learning_rate': 7.71720009982102e-06, 'epoch': 0.59}

 59%|█████▊    | 22250/37962 [51:32:33<36:03:19,  8.26s/it]
 59%|█████▊    | 22251/37962 [51:32:42<36:19:52,  8.32s/it]
 59%|█████▊    | 22252/37962 [51:32:50<36:08:53,  8.28s/it]
 59%|█████▊    | 22253/37962 [51:32:58<36:00:47,  8.25s/it]
 59%|█████▊    | 22254/37962 [51:33:07<36:06:45,  8.28s/it]
 59%|█████▊    | 22255/37962 [51:33:15<36:16:42,  8.31s/it]
 59%|█████▊    | 22256/37962 [51:33:23<35:54:15,  8.23s/it]
 59%|█████▊    | 22257/37962 [51:33:31<35:44:36,  8.19s/it]
 59%|█████▊    | 22258/37962 [51:33:39<35:20:46,  8.10s/it]
 59%|█████▊    | 22259/37962 [51:33:47<35:25:26,  8.12s/it]
 59%|█████▊    | 22260/37962 [51:33:55<35:25:24,  8.12s/it]
                                                           
{'loss': 1.2043, 'grad_norm': 8.556487083435059, 'learning_rate': 7.708894599861198e-06, 'epoch': 0.59}

 59%|█████▊    | 22260/37962 [51:33:55<35:25:24,  8.12s/it]
 59%|█████▊    | 22261/37962 [51:34:04<35:47:56,  8.21s/it]
 59%|█████▊    | 22262/37962 [51:34:12<35:52:56,  8.23s/it]
 59%|█████▊    | 22263/37962 [51:34:20<35:52:44,  8.23s/it]
 59%|█████▊    | 22264/37962 [51:34:28<35:25:06,  8.12s/it]
 59%|█████▊    | 22265/37962 [51:34:36<35:34:22,  8.16s/it]
 59%|█████▊    | 22266/37962 [51:34:44<35:11:05,  8.07s/it]
 59%|█████▊    | 22267/37962 [51:34:52<35:19:49,  8.10s/it]
 59%|█████▊    | 22268/37962 [51:35:01<35:24:20,  8.12s/it]
 59%|█████▊    | 22269/37962 [51:35:08<35:12:08,  8.08s/it]
 59%|█████▊    | 22270/37962 [51:35:17<35:21:12,  8.11s/it]
                                                           
{'loss': 1.2298, 'grad_norm': 7.827797889709473, 'learning_rate': 7.700590767557302e-06, 'epoch': 0.59}

 59%|█████▊    | 22270/37962 [51:35:17<35:21:12,  8.11s/it]
 59%|█████▊    | 22271/37962 [51:35:25<35:35:30,  8.17s/it]
 59%|█████▊    | 22272/37962 [51:35:33<35:39:45,  8.18s/it]
 59%|█████▊    | 22273/37962 [51:35:41<35:33:51,  8.16s/it]
 59%|█████▊    | 22274/37962 [51:35:50<35:37:10,  8.17s/it]
 59%|█████▊    | 22275/37962 [51:35:58<35:49:29,  8.22s/it]
 59%|█████▊    | 22276/37962 [51:36:06<36:00:54,  8.27s/it]
 59%|█████▊    | 22277/37962 [51:36:15<36:17:18,  8.33s/it]
 59%|█████▊    | 22278/37962 [51:36:23<36:13:04,  8.31s/it]
 59%|█████▊    | 22279/37962 [51:36:31<36:03:29,  8.28s/it]
 59%|█████▊    | 22280/37962 [51:36:39<35:51:45,  8.23s/it]
                                                           
{'loss': 1.3703, 'grad_norm': 4.906430721282959, 'learning_rate': 7.692288608953556e-06, 'epoch': 0.59}

 59%|█████▊    | 22280/37962 [51:36:39<35:51:45,  8.23s/it]
 59%|█████▊    | 22281/37962 [51:36:47<35:44:42,  8.21s/it]
 59%|█████▊    | 22282/37962 [51:36:56<35:56:19,  8.25s/it]
 59%|█████▊    | 22283/37962 [51:37:04<35:43:12,  8.20s/it]
 59%|█████▊    | 22284/37962 [51:37:12<36:09:55,  8.30s/it]
 59%|█████▊    | 22285/37962 [51:37:21<36:05:24,  8.29s/it]
 59%|█████▊    | 22286/37962 [51:37:29<35:52:07,  8.24s/it]
 59%|█████▊    | 22287/37962 [51:37:37<35:35:40,  8.17s/it]
 59%|█████▊    | 22288/37962 [51:37:45<35:45:05,  8.21s/it]
 59%|█████▊    | 22289/37962 [51:37:54<36:06:18,  8.29s/it]
 59%|█████▊    | 22290/37962 [51:38:02<36:08:46,  8.30s/it]
                                                           
{'loss': 1.3155, 'grad_norm': 5.548553466796875, 'learning_rate': 7.683988130092958e-06, 'epoch': 0.59}

 59%|█████▊    | 22290/37962 [51:38:02<36:08:46,  8.30s/it]
 59%|█████▊    | 22291/37962 [51:38:10<35:55:12,  8.25s/it]
 59%|█████▊    | 22292/37962 [51:38:18<35:34:11,  8.17s/it]
 59%|█████▊    | 22293/37962 [51:38:27<36:07:40,  8.30s/it]
 59%|█████▊    | 22294/37962 [51:38:35<36:10:18,  8.31s/it]
 59%|█████▊    | 22295/37962 [51:38:43<35:53:01,  8.25s/it]
 59%|█████▊    | 22296/37962 [51:38:51<35:41:55,  8.20s/it]
 59%|█████▊    | 22297/37962 [51:38:59<35:48:05,  8.23s/it]
 59%|█████▊    | 22298/37962 [51:39:07<35:32:00,  8.17s/it]
 59%|█████▊    | 22299/37962 [51:39:16<35:38:03,  8.19s/it]
 59%|█████▊    | 22300/37962 [51:39:24<35:26:02,  8.14s/it]
                                                           
{'loss': 1.221, 'grad_norm': 9.5966796875, 'learning_rate': 7.67568933701727e-06, 'epoch': 0.59}

 59%|█████▊    | 22300/37962 [51:39:24<35:26:02,  8.14s/it]
 59%|█████▊    | 22301/37962 [51:39:32<35:51:11,  8.24s/it]
 59%|█████▊    | 22302/37962 [51:39:40<35:40:15,  8.20s/it]
 59%|█████▉    | 22303/37962 [51:39:48<35:37:08,  8.19s/it]
 59%|█████▉    | 22304/37962 [51:39:57<35:44:39,  8.22s/it]
 59%|█████▉    | 22305/37962 [51:40:05<35:38:34,  8.20s/it]
 59%|█████▉    | 22306/37962 [51:40:13<35:50:29,  8.24s/it]
 59%|█████▉    | 22307/37962 [51:40:21<35:45:16,  8.22s/it]
 59%|█████▉    | 22308/37962 [51:40:30<35:33:36,  8.18s/it]
 59%|█████▉    | 22309/37962 [51:40:38<36:05:25,  8.30s/it]
 59%|█████▉    | 22310/37962 [51:40:46<35:44:41,  8.22s/it]
                                                           
{'loss': 1.1998, 'grad_norm': 14.21728515625, 'learning_rate': 7.66739223576705e-06, 'epoch': 0.59}

 59%|█████▉    | 22310/37962 [51:40:46<35:44:41,  8.22s/it]
 59%|█████▉    | 22311/37962 [51:40:54<35:34:58,  8.18s/it]
 59%|█████▉    | 22312/37962 [51:41:03<35:43:38,  8.22s/it]
 59%|█████▉    | 22313/37962 [51:41:11<35:59:50,  8.28s/it]
 59%|█████▉    | 22314/37962 [51:41:19<36:15:57,  8.34s/it]
 59%|█████▉    | 22315/37962 [51:41:28<36:06:04,  8.31s/it]
 59%|█████▉    | 22316/37962 [51:41:36<36:02:24,  8.29s/it]
 59%|█████▉    | 22317/37962 [51:41:44<35:54:58,  8.26s/it]
 59%|█████▉    | 22318/37962 [51:41:52<35:50:35,  8.25s/it]
 59%|█████▉    | 22319/37962 [51:42:00<35:22:03,  8.14s/it]
 59%|█████▉    | 22320/37962 [51:42:08<35:30:55,  8.17s/it]
                                                           
{'loss': 1.3749, 'grad_norm': 3.2985284328460693, 'learning_rate': 7.659096832381613e-06, 'epoch': 0.59}

 59%|█████▉    | 22320/37962 [51:42:08<35:30:55,  8.17s/it]
 59%|█████▉    | 22321/37962 [51:42:17<35:35:22,  8.19s/it]
 59%|█████▉    | 22322/37962 [51:42:25<35:40:12,  8.21s/it]
 59%|█████▉    | 22323/37962 [51:42:33<35:21:22,  8.14s/it]
 59%|█████▉    | 22324/37962 [51:42:41<35:28:18,  8.17s/it]
 59%|█████▉    | 22325/37962 [51:42:49<35:35:34,  8.19s/it]
 59%|█████▉    | 22326/37962 [51:42:58<35:45:45,  8.23s/it]
 59%|█████▉    | 22327/37962 [51:43:06<35:56:56,  8.28s/it]
 59%|█████▉    | 22328/37962 [51:43:14<36:01:43,  8.30s/it]
 59%|█████▉    | 22329/37962 [51:43:23<36:01:04,  8.29s/it]
 59%|█████▉    | 22330/37962 [51:43:31<36:03:30,  8.30s/it]
                                                           
{'loss': 1.2763, 'grad_norm': 9.1517972946167, 'learning_rate': 7.650803132899039e-06, 'epoch': 0.59}

 59%|█████▉    | 22330/37962 [51:43:31<36:03:30,  8.30s/it]
 59%|█████▉    | 22331/37962 [51:43:39<36:09:51,  8.33s/it]
 59%|█████▉    | 22332/37962 [51:43:48<36:00:46,  8.29s/it]
 59%|█████▉    | 22333/37962 [51:43:56<35:57:11,  8.28s/it]
 59%|█████▉    | 22334/37962 [51:44:04<36:09:36,  8.33s/it]
 59%|█████▉    | 22335/37962 [51:44:13<36:36:42,  8.43s/it]
 59%|█████▉    | 22336/37962 [51:44:22<36:50:55,  8.49s/it]
 59%|█████▉    | 22337/37962 [51:44:30<37:03:01,  8.54s/it]
 59%|█████▉    | 22338/37962 [51:44:39<36:34:09,  8.43s/it]
 59%|█████▉    | 22339/37962 [51:44:47<36:37:47,  8.44s/it]
 59%|█████▉    | 22340/37962 [51:44:55<36:01:54,  8.30s/it]
                                                           
{'loss': 1.1611, 'grad_norm': 2.767810106277466, 'learning_rate': 7.642511143356168e-06, 'epoch': 0.59}

 59%|█████▉    | 22340/37962 [51:44:55<36:01:54,  8.30s/it]
 59%|█████▉    | 22341/37962 [51:45:03<36:04:08,  8.31s/it]
 59%|█████▉    | 22342/37962 [51:45:12<36:11:52,  8.34s/it]
 59%|█████▉    | 22343/37962 [51:45:20<36:06:27,  8.32s/it]
 59%|█████▉    | 22344/37962 [51:45:28<36:07:36,  8.33s/it]
 59%|█████▉    | 22345/37962 [51:45:37<36:12:48,  8.35s/it]
 59%|█████▉    | 22346/37962 [51:45:45<36:20:56,  8.38s/it]
 59%|█████▉    | 22347/37962 [51:45:53<36:09:14,  8.34s/it]
 59%|█████▉    | 22348/37962 [51:46:02<36:16:10,  8.36s/it]
 59%|█████▉    | 22349/37962 [51:46:10<36:00:46,  8.30s/it]
 59%|█████▉    | 22350/37962 [51:46:18<35:55:53,  8.29s/it]
                                                           
{'loss': 1.1701, 'grad_norm': 7.031129360198975, 'learning_rate': 7.634220869788593e-06, 'epoch': 0.59}

 59%|█████▉    | 22350/37962 [51:46:18<35:55:53,  8.29s/it]
 59%|█████▉    | 22351/37962 [51:46:26<35:52:52,  8.27s/it]
 59%|█████▉    | 22352/37962 [51:46:35<36:00:19,  8.30s/it]
 59%|█████▉    | 22353/37962 [51:46:43<35:51:32,  8.27s/it]
 59%|█████▉    | 22354/37962 [51:46:51<35:47:50,  8.26s/it]
 59%|█████▉    | 22355/37962 [51:46:59<35:35:14,  8.21s/it]
 59%|█████▉    | 22356/37962 [51:47:08<35:39:50,  8.23s/it]
 59%|█████▉    | 22357/37962 [51:47:16<35:38:27,  8.22s/it]
 59%|█████▉    | 22358/37962 [51:47:24<35:34:20,  8.21s/it]
 59%|█████▉    | 22359/37962 [51:47:32<35:50:10,  8.27s/it]
 59%|█████▉    | 22360/37962 [51:47:40<35:29:45,  8.19s/it]
                                                           
{'loss': 1.2993, 'grad_norm': 11.872005462646484, 'learning_rate': 7.625932318230662e-06, 'epoch': 0.59}

 59%|█████▉    | 22360/37962 [51:47:40<35:29:45,  8.19s/it]
 59%|█████▉    | 22361/37962 [51:47:49<35:33:37,  8.21s/it]
 59%|█████▉    | 22362/37962 [51:47:57<35:33:30,  8.21s/it]
 59%|█████▉    | 22363/37962 [51:48:05<35:46:25,  8.26s/it]
 59%|█████▉    | 22364/37962 [51:48:14<35:57:28,  8.30s/it]
 59%|█████▉    | 22365/37962 [51:48:22<36:09:14,  8.34s/it]
 59%|█████▉    | 22366/37962 [51:48:31<36:18:58,  8.38s/it]
 59%|█████▉    | 22367/37962 [51:48:39<35:57:08,  8.30s/it]
 59%|█████▉    | 22368/37962 [51:48:47<35:55:50,  8.29s/it]
 59%|█████▉    | 22369/37962 [51:48:55<36:02:26,  8.32s/it]
 59%|█████▉    | 22370/37962 [51:49:04<36:09:00,  8.35s/it]
                                                           
{'loss': 1.2245, 'grad_norm': 6.16968297958374, 'learning_rate': 7.617645494715467e-06, 'epoch': 0.59}

 59%|█████▉    | 22370/37962 [51:49:04<36:09:00,  8.35s/it]
 59%|█████▉    | 22371/37962 [51:49:12<36:16:32,  8.38s/it]
 59%|█████▉    | 22372/37962 [51:49:20<35:53:26,  8.29s/it]
 59%|█████▉    | 22373/37962 [51:49:28<35:37:34,  8.23s/it]
 59%|█████▉    | 22374/37962 [51:49:37<35:43:13,  8.25s/it]
 59%|█████▉    | 22375/37962 [51:49:45<35:43:52,  8.25s/it]
 59%|█████▉    | 22376/37962 [51:49:53<35:42:07,  8.25s/it]
 59%|█████▉    | 22377/37962 [51:50:02<36:08:39,  8.35s/it]
 59%|█████▉    | 22378/37962 [51:50:10<36:01:49,  8.32s/it]
 59%|█████▉    | 22379/37962 [51:50:19<36:15:46,  8.38s/it]
 59%|█████▉    | 22380/37962 [51:50:27<36:03:24,  8.33s/it]
                                                           
{'loss': 1.1943, 'grad_norm': 8.109123229980469, 'learning_rate': 7.609360405274848e-06, 'epoch': 0.59}

 59%|█████▉    | 22380/37962 [51:50:27<36:03:24,  8.33s/it]
 59%|█████▉    | 22381/37962 [51:50:35<36:18:06,  8.39s/it]
 59%|█████▉    | 22382/37962 [51:50:43<35:47:49,  8.27s/it]
 59%|█████▉    | 22383/37962 [51:50:51<35:28:24,  8.20s/it]
 59%|█████▉    | 22384/37962 [51:51:00<35:35:44,  8.23s/it]
 59%|█████▉    | 22385/37962 [51:51:08<35:36:13,  8.23s/it]
 59%|█████▉    | 22386/37962 [51:51:16<35:41:33,  8.25s/it]
 59%|█████▉    | 22387/37962 [51:51:24<35:02:23,  8.10s/it]
 59%|█████▉    | 22388/37962 [51:51:32<35:11:12,  8.13s/it]
 59%|█████▉    | 22389/37962 [51:51:40<34:44:53,  8.03s/it]
 59%|█████▉    | 22390/37962 [51:51:48<35:18:01,  8.16s/it]
                                                           
{'loss': 1.1714, 'grad_norm': 4.905035495758057, 'learning_rate': 7.6010770559393784e-06, 'epoch': 0.59}

 59%|█████▉    | 22390/37962 [51:51:48<35:18:01,  8.16s/it]
 59%|█████▉    | 22391/37962 [51:51:57<35:28:12,  8.20s/it]
 59%|█████▉    | 22392/37962 [51:52:05<35:24:40,  8.19s/it]
 59%|█████▉    | 22393/37962 [51:52:13<35:44:08,  8.26s/it]
 59%|█████▉    | 22394/37962 [51:52:22<35:47:16,  8.28s/it]
 59%|█████▉    | 22395/37962 [51:52:30<35:47:35,  8.28s/it]
 59%|█████▉    | 22396/37962 [51:52:38<35:46:23,  8.27s/it]
 59%|█████▉    | 22397/37962 [51:52:46<35:42:11,  8.26s/it]
 59%|█████▉    | 22398/37962 [51:52:54<35:14:41,  8.15s/it]
 59%|█████▉    | 22399/37962 [51:53:02<35:16:52,  8.16s/it]
 59%|█████▉    | 22400/37962 [51:53:11<35:35:43,  8.23s/it]
                                                           
{'loss': 1.252, 'grad_norm': 18.849170684814453, 'learning_rate': 7.592795452738355e-06, 'epoch': 0.59}

 59%|█████▉    | 22400/37962 [51:53:11<35:35:43,  8.23s/it]
 59%|█████▉    | 22401/37962 [51:53:19<35:36:08,  8.24s/it]
 59%|█████▉    | 22402/37962 [51:53:27<35:40:22,  8.25s/it]
 59%|█████▉    | 22403/37962 [51:53:36<36:02:57,  8.34s/it]
 59%|█████▉    | 22404/37962 [51:53:44<35:50:27,  8.29s/it]
 59%|█████▉    | 22405/37962 [51:53:53<36:22:38,  8.42s/it]
 59%|█████▉    | 22406/37962 [51:54:01<35:48:45,  8.29s/it]
 59%|█████▉    | 22407/37962 [51:54:09<36:04:25,  8.35s/it]
 59%|█████▉    | 22408/37962 [51:54:18<35:57:58,  8.32s/it]
 59%|█████▉    | 22409/37962 [51:54:26<35:58:02,  8.33s/it]
 59%|█████▉    | 22410/37962 [51:54:34<36:10:38,  8.37s/it]
                                                           
{'loss': 1.3357, 'grad_norm': 14.629880905151367, 'learning_rate': 7.584515601699822e-06, 'epoch': 0.59}

 59%|█████▉    | 22410/37962 [51:54:34<36:10:38,  8.37s/it]
 59%|█████▉    | 22411/37962 [51:54:43<36:03:50,  8.35s/it]
 59%|█████▉    | 22412/37962 [51:54:51<36:21:10,  8.42s/it]
 59%|█████▉    | 22413/37962 [51:55:00<36:26:27,  8.44s/it]
 59%|█████▉    | 22414/37962 [51:55:08<36:11:38,  8.38s/it]
 59%|█████▉    | 22415/37962 [51:55:16<36:05:48,  8.36s/it]
 59%|█████▉    | 22416/37962 [51:55:24<35:17:06,  8.17s/it]
 59%|█████▉    | 22417/37962 [51:55:32<35:37:26,  8.25s/it]
 59%|█████▉    | 22418/37962 [51:55:41<35:33:01,  8.23s/it]
 59%|█████▉    | 22419/37962 [51:55:49<35:33:47,  8.24s/it]
 59%|█████▉    | 22420/37962 [51:55:57<35:17:30,  8.17s/it]
                                                           
{'loss': 1.3252, 'grad_norm': 8.929216384887695, 'learning_rate': 7.576237508850537e-06, 'epoch': 0.59}

 59%|█████▉    | 22420/37962 [51:55:57<35:17:30,  8.17s/it]
 59%|█████▉    | 22421/37962 [51:56:05<35:37:22,  8.25s/it]
 59%|█████▉    | 22422/37962 [51:56:14<35:34:32,  8.24s/it]
 59%|█████▉    | 22423/37962 [51:56:22<35:36:01,  8.25s/it]
 59%|█████▉    | 22424/37962 [51:56:30<35:34:38,  8.24s/it]
 59%|█████▉    | 22425/37962 [51:56:38<35:47:59,  8.30s/it]
 59%|█████▉    | 22426/37962 [51:56:47<35:38:01,  8.26s/it]
 59%|█████▉    | 22427/37962 [51:56:55<35:41:52,  8.27s/it]
 59%|█████▉    | 22428/37962 [51:57:03<35:56:26,  8.33s/it]
 59%|█████▉    | 22429/37962 [51:57:12<35:48:56,  8.30s/it]
 59%|█████▉    | 22430/37962 [51:57:20<35:42:22,  8.28s/it]
                                                           
{'loss': 1.2437, 'grad_norm': 15.515727996826172, 'learning_rate': 7.56796118021598e-06, 'epoch': 0.59}

 59%|█████▉    | 22430/37962 [51:57:20<35:42:22,  8.28s/it]
 59%|█████▉    | 22431/37962 [51:57:28<35:46:28,  8.29s/it]
 59%|█████▉    | 22432/37962 [51:57:37<36:09:32,  8.38s/it]
 59%|█████▉    | 22433/37962 [51:57:45<36:03:12,  8.36s/it]
 59%|█████▉    | 22434/37962 [51:57:53<35:55:19,  8.33s/it]
 59%|█████▉    | 22435/37962 [51:58:02<35:54:55,  8.33s/it]
 59%|█████▉    | 22436/37962 [51:58:10<36:09:38,  8.38s/it]
 59%|█████▉    | 22437/37962 [51:58:18<36:02:37,  8.36s/it]
 59%|█████▉    | 22438/37962 [51:58:27<36:03:44,  8.36s/it]
 59%|█████▉    | 22439/37962 [51:58:35<35:57:39,  8.34s/it]
 59%|█████▉    | 22440/37962 [51:58:44<36:14:03,  8.40s/it]
                                                           
{'loss': 1.4, 'grad_norm': 7.28492546081543, 'learning_rate': 7.55968662182035e-06, 'epoch': 0.59}

 59%|█████▉    | 22440/37962 [51:58:44<36:14:03,  8.40s/it]
 59%|█████▉    | 22441/37962 [51:58:52<35:58:27,  8.34s/it]
 59%|█████▉    | 22442/37962 [51:59:00<35:53:34,  8.33s/it]
 59%|█████▉    | 22443/37962 [51:59:09<35:56:41,  8.34s/it]
 59%|█████▉    | 22444/37962 [51:59:17<36:02:42,  8.36s/it]
 59%|█████▉    | 22445/37962 [51:59:25<35:25:12,  8.22s/it]
 59%|█████▉    | 22446/37962 [51:59:33<35:30:32,  8.24s/it]
 59%|█████▉    | 22447/37962 [51:59:41<35:33:15,  8.25s/it]
 59%|█████▉    | 22448/37962 [51:59:49<35:17:00,  8.19s/it]
 59%|█████▉    | 22449/37962 [51:59:58<36:18:09,  8.42s/it]
 59%|█████▉    | 22450/37962 [52:00:07<36:13:37,  8.41s/it]
                                                           
{'loss': 1.3336, 'grad_norm': 6.34182071685791, 'learning_rate': 7.551413839686551e-06, 'epoch': 0.59}

 59%|█████▉    | 22450/37962 [52:00:07<36:13:37,  8.41s/it]
 59%|█████▉    | 22451/37962 [52:00:15<36:01:29,  8.36s/it]
 59%|█████▉    | 22452/37962 [52:00:23<35:48:17,  8.31s/it]
 59%|█████▉    | 22453/37962 [52:00:32<36:11:49,  8.40s/it]
 59%|█████▉    | 22454/37962 [52:00:40<35:49:21,  8.32s/it]
 59%|█████▉    | 22455/37962 [52:00:48<35:54:48,  8.34s/it]
 59%|█████▉    | 22456/37962 [52:00:56<35:20:37,  8.21s/it]
 59%|█████▉    | 22457/37962 [52:01:05<35:33:38,  8.26s/it]
 59%|█████▉    | 22458/37962 [52:01:13<35:31:29,  8.25s/it]
 59%|█████▉    | 22459/37962 [52:01:21<35:15:19,  8.19s/it]
 59%|█████▉    | 22460/37962 [52:01:29<35:17:37,  8.20s/it]
                                                           
{'loss': 1.3027, 'grad_norm': 6.427453517913818, 'learning_rate': 7.543142839836199e-06, 'epoch': 0.59}

 59%|█████▉    | 22460/37962 [52:01:29<35:17:37,  8.20s/it]
 59%|█████▉    | 22461/37962 [52:01:38<35:47:22,  8.31s/it]
 59%|█████▉    | 22462/37962 [52:01:46<35:44:38,  8.30s/it]
 59%|█████▉    | 22463/37962 [52:01:54<35:45:04,  8.30s/it]
 59%|█████▉    | 22464/37962 [52:02:03<35:47:40,  8.31s/it]
 59%|█████▉    | 22465/37962 [52:02:11<35:44:33,  8.30s/it]
 59%|█████▉    | 22466/37962 [52:02:19<35:39:02,  8.28s/it]
 59%|█████▉    | 22467/37962 [52:02:27<35:07:54,  8.16s/it]
 59%|█████▉    | 22468/37962 [52:02:35<35:21:29,  8.22s/it]
 59%|█████▉    | 22469/37962 [52:02:43<34:58:45,  8.13s/it]
 59%|█████▉    | 22470/37962 [52:02:52<35:19:23,  8.21s/it]
                                                           
{'loss': 1.3486, 'grad_norm': 2.2451975345611572, 'learning_rate': 7.534873628289612e-06, 'epoch': 0.59}

 59%|█████▉    | 22470/37962 [52:02:52<35:19:23,  8.21s/it]
 59%|█████▉    | 22471/37962 [52:03:00<35:30:18,  8.25s/it]
 59%|█████▉    | 22472/37962 [52:03:09<35:52:30,  8.34s/it]
 59%|█████▉    | 22473/37962 [52:03:17<35:57:04,  8.36s/it]
 59%|█████▉    | 22474/37962 [52:03:25<35:53:37,  8.34s/it]
 59%|█████▉    | 22475/37962 [52:03:34<35:51:13,  8.33s/it]
 59%|█████▉    | 22476/37962 [52:03:42<35:49:12,  8.33s/it]
 59%|█████▉    | 22477/37962 [52:03:50<35:37:22,  8.28s/it]
 59%|█████▉    | 22478/37962 [52:03:58<35:37:51,  8.28s/it]
 59%|█████▉    | 22479/37962 [52:04:07<35:37:15,  8.28s/it]
 59%|█████▉    | 22480/37962 [52:04:15<35:31:12,  8.26s/it]
                                                           
{'loss': 1.2321, 'grad_norm': 11.31552791595459, 'learning_rate': 7.526606211065803e-06, 'epoch': 0.59}

 59%|█████▉    | 22480/37962 [52:04:15<35:31:12,  8.26s/it]
 59%|█████▉    | 22481/37962 [52:04:23<35:41:08,  8.30s/it]
 59%|█████▉    | 22482/37962 [52:04:32<35:47:15,  8.32s/it]
 59%|█████▉    | 22483/37962 [52:04:40<35:48:29,  8.33s/it]
 59%|█████▉    | 22484/37962 [52:04:48<35:03:41,  8.15s/it]
 59%|█████▉    | 22485/37962 [52:04:56<34:54:05,  8.12s/it]
 59%|█████▉    | 22486/37962 [52:05:04<35:18:22,  8.21s/it]
 59%|█████▉    | 22487/37962 [52:05:13<35:32:14,  8.27s/it]
 59%|█████▉    | 22488/37962 [52:05:21<35:36:31,  8.28s/it]
 59%|█████▉    | 22489/37962 [52:05:29<35:29:07,  8.26s/it]
 59%|█████▉    | 22490/37962 [52:05:37<35:40:23,  8.30s/it]
                                                           
{'loss': 1.312, 'grad_norm': 6.722403526306152, 'learning_rate': 7.5183405941824875e-06, 'epoch': 0.59}

 59%|█████▉    | 22490/37962 [52:05:37<35:40:23,  8.30s/it]
 59%|█████▉    | 22491/37962 [52:05:46<35:42:23,  8.31s/it]
 59%|█████▉    | 22492/37962 [52:05:54<35:37:37,  8.29s/it]
 59%|█████▉    | 22493/37962 [52:06:02<35:46:01,  8.32s/it]
 59%|█████▉    | 22494/37962 [52:06:11<35:41:11,  8.31s/it]
 59%|█████▉    | 22495/37962 [52:06:19<35:40:29,  8.30s/it]
 59%|█████▉    | 22496/37962 [52:06:27<35:34:45,  8.28s/it]
 59%|█████▉    | 22497/37962 [52:06:35<35:29:28,  8.26s/it]
 59%|█████▉    | 22498/37962 [52:06:44<35:29:19,  8.26s/it]
 59%|█████▉    | 22499/37962 [52:06:52<35:37:15,  8.29s/it]
 59%|█████▉    | 22500/37962 [52:07:00<35:35:06,  8.29s/it]
                                                           
{'loss': 1.2261, 'grad_norm': 5.227752208709717, 'learning_rate': 7.510076783656061e-06, 'epoch': 0.59}

 59%|█████▉    | 22500/37962 [52:07:00<35:35:06,  8.29s/it]
 59%|█████▉    | 22501/37962 [52:07:09<35:48:27,  8.34s/it]
 59%|█████▉    | 22502/37962 [52:07:17<35:44:07,  8.32s/it]
 59%|█████▉    | 22503/37962 [52:07:25<35:36:37,  8.29s/it]
 59%|█████▉    | 22504/37962 [52:07:34<35:37:44,  8.30s/it]
 59%|█████▉    | 22505/37962 [52:07:42<35:31:08,  8.27s/it]
 59%|█████▉    | 22506/37962 [52:07:50<35:29:35,  8.27s/it]
 59%|█████▉    | 22507/37962 [52:07:58<35:35:17,  8.29s/it]
 59%|█████▉    | 22508/37962 [52:08:07<35:36:27,  8.29s/it]
 59%|█████▉    | 22509/37962 [52:08:15<35:39:24,  8.31s/it]
 59%|█████▉    | 22510/37962 [52:08:23<35:27:17,  8.26s/it]
                                                           
{'loss': 1.2546, 'grad_norm': 7.010986804962158, 'learning_rate': 7.50181478550161e-06, 'epoch': 0.59}

 59%|█████▉    | 22510/37962 [52:08:23<35:27:17,  8.26s/it]
 59%|█████▉    | 22511/37962 [52:08:32<35:28:36,  8.27s/it]
 59%|█████▉    | 22512/37962 [52:08:40<35:14:55,  8.21s/it]
 59%|█████▉    | 22513/37962 [52:08:48<34:51:20,  8.12s/it]
 59%|█████▉    | 22514/37962 [52:08:56<35:08:28,  8.19s/it]
 59%|█████▉    | 22515/37962 [52:09:04<34:50:44,  8.12s/it]
 59%|█████▉    | 22516/37962 [52:09:12<35:13:24,  8.21s/it]
 59%|█████▉    | 22517/37962 [52:09:21<35:22:57,  8.25s/it]
 59%|█████▉    | 22518/37962 [52:09:36<44:44:50, 10.43s/it]
 59%|█████▉    | 22519/37962 [52:09:44<41:44:09,  9.73s/it]
 59%|█████▉    | 22520/37962 [52:09:53<40:02:30,  9.33s/it]
                                                           
{'loss': 1.263, 'grad_norm': 8.845512390136719, 'learning_rate': 7.493554605732897e-06, 'epoch': 0.59}

 59%|█████▉    | 22520/37962 [52:09:53<40:02:30,  9.33s/it]
 59%|█████▉    | 22521/37962 [52:10:01<38:34:28,  8.99s/it]
 59%|█████▉    | 22522/37962 [52:10:09<37:32:51,  8.75s/it]
 59%|█████▉    | 22523/37962 [52:10:17<36:55:56,  8.61s/it]
 59%|█████▉    | 22524/37962 [52:10:25<36:23:29,  8.49s/it]
 59%|█████▉    | 22525/37962 [52:10:34<36:03:07,  8.41s/it]
 59%|█████▉    | 22526/37962 [52:10:42<35:50:58,  8.36s/it]
 59%|█████▉    | 22527/37962 [52:10:50<35:56:14,  8.38s/it]
 59%|█████▉    | 22528/37962 [52:10:59<35:40:58,  8.32s/it]
 59%|█████▉    | 22529/37962 [52:11:07<35:45:44,  8.34s/it]
 59%|█████▉    | 22530/37962 [52:11:15<35:40:43,  8.32s/it]
                                                           
{'loss': 1.2805, 'grad_norm': 5.356457710266113, 'learning_rate': 7.485296250362365e-06, 'epoch': 0.59}

 59%|█████▉    | 22530/37962 [52:11:15<35:40:43,  8.32s/it]
 59%|█████▉    | 22531/37962 [52:11:24<35:51:05,  8.36s/it]
 59%|█████▉    | 22532/37962 [52:11:32<35:52:45,  8.37s/it]
 59%|█████▉    | 22533/37962 [52:11:40<35:42:23,  8.33s/it]
 59%|█████▉    | 22534/37962 [52:11:49<35:32:46,  8.29s/it]
 59%|█████▉    | 22535/37962 [52:11:57<35:24:47,  8.26s/it]
 59%|█████▉    | 22536/37962 [52:12:05<35:48:41,  8.36s/it]
 59%|█████▉    | 22537/37962 [52:12:14<35:37:53,  8.32s/it]
 59%|█████▉    | 22538/37962 [52:12:22<35:45:58,  8.35s/it]
 59%|█████▉    | 22539/37962 [52:12:30<35:31:50,  8.29s/it]
 59%|█████▉    | 22540/37962 [52:12:38<35:29:21,  8.28s/it]
                                                           
{'loss': 1.2362, 'grad_norm': 9.55726432800293, 'learning_rate': 7.477039725401128e-06, 'epoch': 0.59}

 59%|█████▉    | 22540/37962 [52:12:38<35:29:21,  8.28s/it]
 59%|█████▉    | 22541/37962 [52:12:47<35:41:15,  8.33s/it]
 59%|█████▉    | 22542/37962 [52:12:55<35:35:21,  8.31s/it]
 59%|█████▉    | 22543/37962 [52:13:03<35:35:53,  8.31s/it]
 59%|█████▉    | 22544/37962 [52:13:12<35:41:26,  8.33s/it]
 59%|█████▉    | 22545/37962 [52:13:20<35:31:57,  8.30s/it]
 59%|█████▉    | 22546/37962 [52:13:28<35:21:55,  8.26s/it]
 59%|█████▉    | 22547/37962 [52:13:36<35:27:20,  8.28s/it]
 59%|█████▉    | 22548/37962 [52:13:44<34:59:59,  8.17s/it]
 59%|█████▉    | 22549/37962 [52:13:52<34:49:48,  8.14s/it]
 59%|█████▉    | 22550/37962 [52:14:01<35:02:06,  8.18s/it]
                                                           
{'loss': 1.3074, 'grad_norm': 5.571455001831055, 'learning_rate': 7.46878503685897e-06, 'epoch': 0.59}

 59%|█████▉    | 22550/37962 [52:14:01<35:02:06,  8.18s/it]
 59%|█████▉    | 22551/37962 [52:14:09<35:13:30,  8.23s/it]
 59%|█████▉    | 22552/37962 [52:14:17<35:13:18,  8.23s/it]
 59%|█████▉    | 22553/37962 [52:14:26<35:24:38,  8.27s/it]
 59%|█████▉    | 22554/37962 [52:14:34<35:28:24,  8.29s/it]
 59%|█████▉    | 22555/37962 [52:14:42<35:38:33,  8.33s/it]
 59%|█████▉    | 22556/37962 [52:14:51<35:29:44,  8.29s/it]
 59%|█████▉    | 22557/37962 [52:14:59<35:23:24,  8.27s/it]
 59%|█████▉    | 22558/37962 [52:15:07<35:21:49,  8.26s/it]
 59%|█████▉    | 22559/37962 [52:15:15<35:18:10,  8.25s/it]
 59%|█████▉    | 22560/37962 [52:15:24<35:25:23,  8.28s/it]
                                                           
{'loss': 1.3609, 'grad_norm': 12.443653106689453, 'learning_rate': 7.460532190744332e-06, 'epoch': 0.59}

 59%|█████▉    | 22560/37962 [52:15:24<35:25:23,  8.28s/it]
 59%|█████▉    | 22561/37962 [52:15:32<35:23:38,  8.27s/it]
 59%|█████▉    | 22562/37962 [52:15:40<35:44:22,  8.35s/it]
 59%|█████▉    | 22563/37962 [52:15:49<35:21:56,  8.27s/it]
 59%|█████▉    | 22564/37962 [52:15:57<35:29:33,  8.30s/it]
 59%|█████▉    | 22565/37962 [52:16:05<35:32:17,  8.31s/it]
 59%|█████▉    | 22566/37962 [52:16:14<35:44:34,  8.36s/it]
 59%|█████▉    | 22567/37962 [52:16:22<35:34:34,  8.32s/it]
 59%|█████▉    | 22568/37962 [52:16:30<35:34:06,  8.32s/it]
 59%|█████▉    | 22569/37962 [52:16:38<35:25:46,  8.29s/it]
 59%|█████▉    | 22570/37962 [52:16:47<35:30:05,  8.30s/it]
                                                           
{'loss': 1.2594, 'grad_norm': 10.531793594360352, 'learning_rate': 7.452281193064318e-06, 'epoch': 0.59}

 59%|█████▉    | 22570/37962 [52:16:47<35:30:05,  8.30s/it]
 59%|█████▉    | 22571/37962 [52:16:55<35:31:20,  8.31s/it]
 59%|█████▉    | 22572/37962 [52:17:04<35:43:05,  8.36s/it]
 59%|█████▉    | 22573/37962 [52:17:12<35:36:09,  8.33s/it]
 59%|█████▉    | 22574/37962 [52:17:20<35:38:11,  8.34s/it]
 59%|█████▉    | 22575/37962 [52:17:29<35:34:55,  8.32s/it]
 59%|█████▉    | 22576/37962 [52:17:36<35:04:59,  8.21s/it]
 59%|█████▉    | 22577/37962 [52:17:45<35:18:25,  8.26s/it]
 59%|█████▉    | 22578/37962 [52:17:53<35:21:13,  8.27s/it]
 59%|█████▉    | 22579/37962 [52:18:01<35:22:26,  8.28s/it]
 59%|█████▉    | 22580/37962 [52:18:10<35:24:39,  8.29s/it]
                                                           
{'loss': 1.2709, 'grad_norm': 7.4541215896606445, 'learning_rate': 7.444032049824686e-06, 'epoch': 0.59}

 59%|█████▉    | 22580/37962 [52:18:10<35:24:39,  8.29s/it]
 59%|█████▉    | 22581/37962 [52:18:18<35:27:55,  8.30s/it]
 59%|█████▉    | 22582/37962 [52:18:26<35:30:12,  8.31s/it]
 59%|█████▉    | 22583/37962 [52:18:35<35:29:30,  8.31s/it]
 59%|█████▉    | 22584/37962 [52:18:43<35:10:01,  8.23s/it]
 59%|█████▉    | 22585/37962 [52:18:51<35:19:41,  8.27s/it]
 59%|█████▉    | 22586/37962 [52:19:07<45:35:37, 10.67s/it]
 59%|█████▉    | 22587/37962 [52:19:16<42:41:42, 10.00s/it]
 60%|█████▉    | 22588/37962 [52:19:24<40:22:51,  9.46s/it]
 60%|█████▉    | 22589/37962 [52:19:32<38:54:39,  9.11s/it]
 60%|█████▉    | 22590/37962 [52:19:40<37:41:31,  8.83s/it]
                                                           
{'loss': 1.3949, 'grad_norm': 7.200699329376221, 'learning_rate': 7.435784767029848e-06, 'epoch': 0.6}

 60%|█████▉    | 22590/37962 [52:19:40<37:41:31,  8.83s/it]
 60%|█████▉    | 22591/37962 [52:19:49<36:57:02,  8.65s/it]
 60%|█████▉    | 22592/37962 [52:19:57<36:27:11,  8.54s/it]
 60%|█████▉    | 22593/37962 [52:20:05<36:04:43,  8.45s/it]
 60%|█████▉    | 22594/37962 [52:20:14<35:52:18,  8.40s/it]
 60%|█████▉    | 22595/37962 [52:20:22<35:40:33,  8.36s/it]
 60%|█████▉    | 22596/37962 [52:20:30<35:33:15,  8.33s/it]
 60%|█████▉    | 22597/37962 [52:20:38<35:22:38,  8.29s/it]
 60%|█████▉    | 22598/37962 [52:20:46<35:12:35,  8.25s/it]
 60%|█████▉    | 22599/37962 [52:20:55<35:18:45,  8.27s/it]
 60%|█████▉    | 22600/37962 [52:21:03<35:42:16,  8.37s/it]
                                                           
{'loss': 1.2772, 'grad_norm': 8.660252571105957, 'learning_rate': 7.42753935068286e-06, 'epoch': 0.6}

 60%|█████▉    | 22600/37962 [52:21:03<35:42:16,  8.37s/it]
 60%|█████▉    | 22601/37962 [52:21:12<35:33:05,  8.33s/it]
 60%|█████▉    | 22602/37962 [52:21:20<35:23:34,  8.30s/it]
 60%|█████▉    | 22603/37962 [52:21:28<35:01:37,  8.21s/it]
 60%|█████▉    | 22604/37962 [52:21:36<35:28:26,  8.32s/it]
 60%|█████▉    | 22605/37962 [52:21:44<35:11:47,  8.25s/it]
 60%|█████▉    | 22606/37962 [52:21:53<35:00:47,  8.21s/it]
 60%|█████▉    | 22607/37962 [52:22:01<35:08:10,  8.24s/it]
 60%|█████▉    | 22608/37962 [52:22:09<35:04:36,  8.22s/it]
 60%|█████▉    | 22609/37962 [52:22:17<35:05:10,  8.23s/it]
 60%|█████▉    | 22610/37962 [52:22:25<34:59:20,  8.20s/it]
                                                           
{'loss': 1.2971, 'grad_norm': 7.5081787109375, 'learning_rate': 7.419295806785407e-06, 'epoch': 0.6}

 60%|█████▉    | 22610/37962 [52:22:25<34:59:20,  8.20s/it]
 60%|█████▉    | 22611/37962 [52:22:34<35:18:45,  8.28s/it]
 60%|█████▉    | 22612/37962 [52:22:42<35:17:22,  8.28s/it]
 60%|█████▉    | 22613/37962 [52:22:51<35:28:45,  8.32s/it]
 60%|█████▉    | 22614/37962 [52:22:59<35:22:02,  8.30s/it]
 60%|█████▉    | 22615/37962 [52:23:07<35:15:00,  8.27s/it]
 60%|█████▉    | 22616/37962 [52:23:15<35:02:31,  8.22s/it]
 60%|█████▉    | 22617/37962 [52:23:24<35:17:03,  8.28s/it]
 60%|█████▉    | 22618/37962 [52:23:32<35:16:02,  8.27s/it]
 60%|█████▉    | 22619/37962 [52:23:40<35:17:01,  8.28s/it]
 60%|█████▉    | 22620/37962 [52:23:49<35:43:03,  8.38s/it]
                                                           
{'loss': 1.2775, 'grad_norm': 8.609156608581543, 'learning_rate': 7.411054141337831e-06, 'epoch': 0.6}

 60%|█████▉    | 22620/37962 [52:23:49<35:43:03,  8.38s/it]
 60%|█████▉    | 22621/37962 [52:23:57<35:44:43,  8.39s/it]
 60%|█████▉    | 22622/37962 [52:24:06<35:44:44,  8.39s/it]
 60%|█████▉    | 22623/37962 [52:24:14<35:32:16,  8.34s/it]
 60%|█████▉    | 22624/37962 [52:24:22<35:47:27,  8.40s/it]
 60%|█████▉    | 22625/37962 [52:24:31<35:56:39,  8.44s/it]
 60%|█████▉    | 22626/37962 [52:24:39<35:31:13,  8.34s/it]
 60%|█████▉    | 22627/37962 [52:24:47<35:23:14,  8.31s/it]
 60%|█████▉    | 22628/37962 [52:24:56<35:36:50,  8.36s/it]
 60%|█████▉    | 22629/37962 [52:25:04<35:38:20,  8.37s/it]
 60%|█████▉    | 22630/37962 [52:25:12<35:27:30,  8.33s/it]
                                                           
{'loss': 1.2311, 'grad_norm': 5.893675804138184, 'learning_rate': 7.4028143603390945e-06, 'epoch': 0.6}

 60%|█████▉    | 22630/37962 [52:25:12<35:27:30,  8.33s/it]
 60%|█████▉    | 22631/37962 [52:25:21<35:22:20,  8.31s/it]
 60%|█████▉    | 22632/37962 [52:25:29<35:21:49,  8.30s/it]
 60%|█████▉    | 22633/37962 [52:25:37<35:20:55,  8.30s/it]
 60%|█████▉    | 22634/37962 [52:25:46<35:27:56,  8.33s/it]
 60%|█████▉    | 22635/37962 [52:25:54<35:26:32,  8.32s/it]
 60%|█████▉    | 22636/37962 [52:26:02<34:57:33,  8.21s/it]
 60%|█████▉    | 22637/37962 [52:26:10<35:07:36,  8.25s/it]
 60%|█████▉    | 22638/37962 [52:26:19<35:20:22,  8.30s/it]
 60%|█████▉    | 22639/37962 [52:26:27<35:11:52,  8.27s/it]
 60%|█████▉    | 22640/37962 [52:26:35<34:47:54,  8.18s/it]
                                                           
{'loss': 1.2673, 'grad_norm': 10.977543830871582, 'learning_rate': 7.394576469786794e-06, 'epoch': 0.6}

 60%|█████▉    | 22640/37962 [52:26:35<34:47:54,  8.18s/it]
 60%|█████▉    | 22641/37962 [52:26:43<34:49:33,  8.18s/it]
 60%|█████▉    | 22642/37962 [52:26:51<35:02:43,  8.24s/it]
 60%|█████▉    | 22643/37962 [52:26:59<35:02:42,  8.24s/it]
 60%|█████▉    | 22644/37962 [52:27:08<35:00:19,  8.23s/it]
 60%|█████▉    | 22645/37962 [52:27:16<34:59:43,  8.23s/it]
 60%|█████▉    | 22646/37962 [52:27:25<35:31:58,  8.35s/it]
 60%|█████▉    | 22647/37962 [52:27:33<35:36:46,  8.37s/it]
 60%|█████▉    | 22648/37962 [52:27:41<35:24:26,  8.32s/it]
 60%|█████▉    | 22649/37962 [52:27:49<35:19:44,  8.31s/it]
 60%|█████▉    | 22650/37962 [52:27:58<35:21:08,  8.31s/it]
                                                           
{'loss': 1.1948, 'grad_norm': 5.313388347625732, 'learning_rate': 7.386340475677145e-06, 'epoch': 0.6}

 60%|█████▉    | 22650/37962 [52:27:58<35:21:08,  8.31s/it]
 60%|█████▉    | 22651/37962 [52:28:06<35:19:07,  8.30s/it]
 60%|█████▉    | 22652/37962 [52:28:14<35:09:56,  8.27s/it]
 60%|█████▉    | 22653/37962 [52:28:22<35:02:00,  8.24s/it]
 60%|█████▉    | 22654/37962 [52:28:31<35:23:44,  8.32s/it]
 60%|█████▉    | 22655/37962 [52:28:39<35:15:46,  8.29s/it]
 60%|█████▉    | 22656/37962 [52:28:47<35:13:42,  8.29s/it]
 60%|█████▉    | 22657/37962 [52:28:56<35:03:20,  8.25s/it]
 60%|█████▉    | 22658/37962 [52:29:04<34:45:45,  8.18s/it]
 60%|█████▉    | 22659/37962 [52:29:12<34:55:44,  8.22s/it]
 60%|█████▉    | 22660/37962 [52:29:20<35:00:00,  8.23s/it]
                                                           
{'loss': 1.2582, 'grad_norm': 9.700843811035156, 'learning_rate': 7.378106384004985e-06, 'epoch': 0.6}

 60%|█████▉    | 22660/37962 [52:29:20<35:00:00,  8.23s/it]
 60%|█████▉    | 22661/37962 [52:29:28<34:58:42,  8.23s/it]
 60%|█████▉    | 22662/37962 [52:29:37<34:54:15,  8.21s/it]
 60%|█████▉    | 22663/37962 [52:29:45<35:11:41,  8.28s/it]
 60%|█████▉    | 22664/37962 [52:29:53<35:17:13,  8.30s/it]
 60%|█████▉    | 22665/37962 [52:30:01<34:53:43,  8.21s/it]
 60%|█████▉    | 22666/37962 [52:30:09<34:37:04,  8.15s/it]
 60%|█████▉    | 22667/37962 [52:30:18<34:46:26,  8.18s/it]
 60%|█████▉    | 22668/37962 [52:30:26<34:40:46,  8.16s/it]
 60%|█████▉    | 22669/37962 [52:30:34<34:54:06,  8.22s/it]
 60%|█████▉    | 22670/37962 [52:30:43<35:10:27,  8.28s/it]
                                                           
{'loss': 1.2738, 'grad_norm': 5.491483688354492, 'learning_rate': 7.369874200763766e-06, 'epoch': 0.6}

 60%|█████▉    | 22670/37962 [52:30:43<35:10:27,  8.28s/it]
 60%|█████▉    | 22671/37962 [52:30:51<35:04:40,  8.26s/it]
 60%|█████▉    | 22672/37962 [52:30:59<35:06:16,  8.27s/it]
 60%|█████▉    | 22673/37962 [52:31:07<34:39:57,  8.16s/it]
 60%|█████▉    | 22674/37962 [52:31:15<34:53:31,  8.22s/it]
 60%|█████▉    | 22675/37962 [52:31:24<35:17:24,  8.31s/it]
 60%|█████▉    | 22676/37962 [52:31:32<35:28:20,  8.35s/it]
 60%|█████▉    | 22677/37962 [52:31:40<35:17:07,  8.31s/it]
 60%|█████▉    | 22678/37962 [52:31:49<35:02:01,  8.25s/it]
 60%|█████▉    | 22679/37962 [52:31:57<35:10:24,  8.29s/it]
 60%|█████▉    | 22680/37962 [52:32:05<35:15:23,  8.31s/it]
                                                           
{'loss': 1.2314, 'grad_norm': 9.734855651855469, 'learning_rate': 7.36164393194555e-06, 'epoch': 0.6}

 60%|█████▉    | 22680/37962 [52:32:05<35:15:23,  8.31s/it]
 60%|█████▉    | 22681/37962 [52:32:13<35:06:32,  8.27s/it]
 60%|█████▉    | 22682/37962 [52:32:21<34:45:40,  8.19s/it]
 60%|█████▉    | 22683/37962 [52:32:30<34:58:14,  8.24s/it]
 60%|█████▉    | 22684/37962 [52:32:38<35:04:51,  8.27s/it]
 60%|█████▉    | 22685/37962 [52:32:46<34:57:10,  8.24s/it]
 60%|█████▉    | 22686/37962 [52:32:55<35:01:48,  8.26s/it]
 60%|█████▉    | 22687/37962 [52:33:03<34:59:32,  8.25s/it]
 60%|█████▉    | 22688/37962 [52:33:11<35:01:39,  8.26s/it]
 60%|█████▉    | 22689/37962 [52:33:20<35:21:50,  8.34s/it]
 60%|█████▉    | 22690/37962 [52:33:28<34:47:16,  8.20s/it]
                                                           
{'loss': 1.2533, 'grad_norm': 8.397748947143555, 'learning_rate': 7.353415583541013e-06, 'epoch': 0.6}

 60%|█████▉    | 22690/37962 [52:33:28<34:47:16,  8.20s/it]
 60%|█████▉    | 22691/37962 [52:33:36<35:10:46,  8.29s/it]
 60%|█████▉    | 22692/37962 [52:33:44<34:53:53,  8.23s/it]
 60%|█████▉    | 22693/37962 [52:33:52<34:56:52,  8.24s/it]
 60%|█████▉    | 22694/37962 [52:34:01<35:06:23,  8.28s/it]
 60%|█████▉    | 22695/37962 [52:34:09<35:01:55,  8.26s/it]
 60%|█████▉    | 22696/37962 [52:34:17<35:06:36,  8.28s/it]
 60%|█████▉    | 22697/37962 [52:34:26<35:00:59,  8.26s/it]
 60%|█████▉    | 22698/37962 [52:34:34<35:01:30,  8.26s/it]
 60%|█████▉    | 22699/37962 [52:34:42<34:59:23,  8.25s/it]
 60%|█████▉    | 22700/37962 [52:34:51<35:17:16,  8.32s/it]
                                                           
{'loss': 1.1657, 'grad_norm': 16.1705265045166, 'learning_rate': 7.345189161539422e-06, 'epoch': 0.6}

 60%|█████▉    | 22700/37962 [52:34:51<35:17:16,  8.32s/it]
 60%|█████▉    | 22701/37962 [52:34:59<35:07:41,  8.29s/it]
 60%|█████▉    | 22702/37962 [52:35:07<35:18:15,  8.33s/it]
 60%|█████▉    | 22703/37962 [52:35:15<35:10:44,  8.30s/it]
 60%|█████▉    | 22704/37962 [52:35:24<35:09:26,  8.30s/it]
 60%|█████▉    | 22705/37962 [52:35:31<34:33:28,  8.15s/it]
 60%|█████▉    | 22706/37962 [52:35:40<34:57:41,  8.25s/it]
 60%|█████▉    | 22707/37962 [52:35:48<34:54:16,  8.24s/it]
 60%|█████▉    | 22708/37962 [52:35:56<34:56:31,  8.25s/it]
 60%|█████▉    | 22709/37962 [52:36:05<34:53:26,  8.23s/it]
 60%|█████▉    | 22710/37962 [52:36:13<34:24:33,  8.12s/it]
                                                           
{'loss': 1.3113, 'grad_norm': 6.640005588531494, 'learning_rate': 7.336964671928653e-06, 'epoch': 0.6}

 60%|█████▉    | 22710/37962 [52:36:13<34:24:33,  8.12s/it]
 60%|█████▉    | 22711/37962 [52:36:21<34:36:26,  8.17s/it]
 60%|█████▉    | 22712/37962 [52:36:29<35:03:01,  8.27s/it]
 60%|█████▉    | 22713/37962 [52:36:38<35:18:59,  8.34s/it]
 60%|█████▉    | 22714/37962 [52:36:46<35:07:47,  8.29s/it]
 60%|█████▉    | 22715/37962 [52:36:54<34:45:45,  8.21s/it]
 60%|█████▉    | 22716/37962 [52:37:02<35:03:49,  8.28s/it]
 60%|█████▉    | 22717/37962 [52:37:11<35:06:21,  8.29s/it]
 60%|█████▉    | 22718/37962 [52:37:19<34:58:54,  8.26s/it]
 60%|█████▉    | 22719/37962 [52:37:27<35:09:50,  8.30s/it]
 60%|█████▉    | 22720/37962 [52:37:36<35:06:41,  8.29s/it]
                                                           
{'loss': 1.3902, 'grad_norm': 18.024147033691406, 'learning_rate': 7.328742120695162e-06, 'epoch': 0.6}

 60%|█████▉    | 22720/37962 [52:37:36<35:06:41,  8.29s/it]
 60%|█████▉    | 22721/37962 [52:37:44<35:08:48,  8.30s/it]
 60%|█████▉    | 22722/37962 [52:37:52<34:39:14,  8.19s/it]
 60%|█████▉    | 22723/37962 [52:38:00<34:43:38,  8.20s/it]
 60%|█████▉    | 22724/37962 [52:38:09<35:03:53,  8.28s/it]
 60%|█████▉    | 22725/37962 [52:38:17<35:08:41,  8.30s/it]
 60%|█████▉    | 22726/37962 [52:38:25<35:10:41,  8.31s/it]
 60%|█████▉    | 22727/37962 [52:38:34<35:12:20,  8.32s/it]
 60%|█████▉    | 22728/37962 [52:38:42<35:19:34,  8.35s/it]
 60%|█████▉    | 22729/37962 [52:38:50<35:15:06,  8.33s/it]
 60%|█████▉    | 22730/37962 [52:38:58<34:57:31,  8.26s/it]
                                                           
{'loss': 1.161, 'grad_norm': 9.143914222717285, 'learning_rate': 7.320521513824004e-06, 'epoch': 0.6}

 60%|█████▉    | 22730/37962 [52:38:58<34:57:31,  8.26s/it]
 60%|█████▉    | 22731/37962 [52:39:07<34:46:53,  8.22s/it]
 60%|█████▉    | 22732/37962 [52:39:15<34:50:48,  8.24s/it]
 60%|█████▉    | 22733/37962 [52:39:23<34:55:25,  8.26s/it]
 60%|█████▉    | 22734/37962 [52:39:31<35:01:39,  8.28s/it]
 60%|█████▉    | 22735/37962 [52:39:40<35:02:26,  8.28s/it]
 60%|█████▉    | 22736/37962 [52:39:48<34:54:33,  8.25s/it]
 60%|█████▉    | 22737/37962 [52:39:56<34:59:33,  8.27s/it]
 60%|█████▉    | 22738/37962 [52:40:05<35:17:06,  8.34s/it]
 60%|█████▉    | 22739/37962 [52:40:13<35:06:17,  8.30s/it]
 60%|█████▉    | 22740/37962 [52:40:21<34:56:18,  8.26s/it]
                                                           
{'loss': 1.1857, 'grad_norm': 5.367995738983154, 'learning_rate': 7.312302857298819e-06, 'epoch': 0.6}

 60%|█████▉    | 22740/37962 [52:40:21<34:56:18,  8.26s/it]
 60%|█████▉    | 22741/37962 [52:40:29<34:58:11,  8.27s/it]
 60%|█████▉    | 22742/37962 [52:40:38<34:59:47,  8.28s/it]
 60%|█████▉    | 22743/37962 [52:40:46<35:09:03,  8.31s/it]
 60%|█████▉    | 22744/37962 [52:40:54<34:58:39,  8.27s/it]
 60%|█████▉    | 22745/37962 [52:41:03<34:56:01,  8.26s/it]
 60%|█████▉    | 22746/37962 [52:41:11<34:50:10,  8.24s/it]
 60%|█████▉    | 22747/37962 [52:41:19<34:40:57,  8.21s/it]
 60%|█████▉    | 22748/37962 [52:41:27<34:52:37,  8.25s/it]
 60%|█████▉    | 22749/37962 [52:41:35<34:32:22,  8.17s/it]
 60%|█████▉    | 22750/37962 [52:41:43<34:28:51,  8.16s/it]
                                                           
{'loss': 1.2639, 'grad_norm': 4.838736057281494, 'learning_rate': 7.3040861571018216e-06, 'epoch': 0.6}

 60%|█████▉    | 22750/37962 [52:41:43<34:28:51,  8.16s/it]
 60%|█████▉    | 22751/37962 [52:41:52<34:43:11,  8.22s/it]
 60%|█████▉    | 22752/37962 [52:42:00<34:44:02,  8.22s/it]
 60%|█████▉    | 22753/37962 [52:42:08<34:52:32,  8.26s/it]
 60%|█████▉    | 22754/37962 [52:42:16<34:50:39,  8.25s/it]
 60%|█████▉    | 22755/37962 [52:42:25<35:00:08,  8.29s/it]
 60%|█████▉    | 22756/37962 [52:42:33<35:06:44,  8.31s/it]
 60%|█████▉    | 22757/37962 [52:42:42<35:17:17,  8.35s/it]
 60%|█████▉    | 22758/37962 [52:42:50<35:20:14,  8.37s/it]
 60%|█████▉    | 22759/37962 [52:42:58<34:56:47,  8.28s/it]
 60%|█████▉    | 22760/37962 [52:43:06<34:58:14,  8.28s/it]
                                                           
{'loss': 1.3902, 'grad_norm': 9.055205345153809, 'learning_rate': 7.29587141921381e-06, 'epoch': 0.6}

 60%|█████▉    | 22760/37962 [52:43:06<34:58:14,  8.28s/it]
 60%|█████▉    | 22761/37962 [52:43:15<34:54:58,  8.27s/it]
 60%|█████▉    | 22762/37962 [52:43:23<34:59:39,  8.29s/it]
 60%|█████▉    | 22763/37962 [52:43:31<34:28:38,  8.17s/it]
 60%|█████▉    | 22764/37962 [52:43:39<34:43:28,  8.23s/it]
 60%|█████▉    | 22765/37962 [52:43:48<34:53:21,  8.26s/it]
 60%|█████▉    | 22766/37962 [52:43:56<34:49:47,  8.25s/it]
 60%|█████▉    | 22767/37962 [52:44:04<34:51:47,  8.26s/it]
 60%|█████▉    | 22768/37962 [52:44:12<35:01:42,  8.30s/it]
 60%|█████▉    | 22769/37962 [52:44:21<35:02:30,  8.30s/it]
 60%|█████▉    | 22770/37962 [52:44:29<34:36:19,  8.20s/it]
                                                           
{'loss': 1.2344, 'grad_norm': 7.2745137214660645, 'learning_rate': 7.2876586496141446e-06, 'epoch': 0.6}

 60%|█████▉    | 22770/37962 [52:44:29<34:36:19,  8.20s/it]
 60%|█████▉    | 22771/37962 [52:44:37<34:48:56,  8.25s/it]
 60%|█████▉    | 22772/37962 [52:44:45<34:50:17,  8.26s/it]
 60%|█████▉    | 22773/37962 [52:44:54<34:53:03,  8.27s/it]
 60%|█████▉    | 22774/37962 [52:45:02<35:01:38,  8.30s/it]
 60%|█████▉    | 22775/37962 [52:45:10<34:59:31,  8.29s/it]
 60%|█████▉    | 22776/37962 [52:45:19<34:58:37,  8.29s/it]
 60%|█████▉    | 22777/37962 [52:45:27<34:50:01,  8.26s/it]
 60%|██████    | 22778/37962 [52:45:35<35:01:43,  8.31s/it]
 60%|██████    | 22779/37962 [52:45:43<34:40:33,  8.22s/it]
 60%|██████    | 22780/37962 [52:45:52<34:49:08,  8.26s/it]
                                                           
{'loss': 1.2757, 'grad_norm': 6.434030055999756, 'learning_rate': 7.279447854280758e-06, 'epoch': 0.6}

 60%|██████    | 22780/37962 [52:45:52<34:49:08,  8.26s/it]
 60%|██████    | 22781/37962 [52:45:59<34:15:50,  8.13s/it]
 60%|██████    | 22782/37962 [52:46:08<34:40:14,  8.22s/it]
 60%|██████    | 22783/37962 [52:46:16<34:43:14,  8.23s/it]
 60%|██████    | 22784/37962 [52:46:24<34:51:34,  8.27s/it]
 60%|██████    | 22785/37962 [52:46:33<34:57:52,  8.29s/it]
 60%|██████    | 22786/37962 [52:46:41<34:58:33,  8.30s/it]
 60%|██████    | 22787/37962 [52:46:49<34:51:02,  8.27s/it]
 60%|██████    | 22788/37962 [52:46:58<34:54:19,  8.28s/it]
 60%|██████    | 22789/37962 [52:47:06<34:41:23,  8.23s/it]
 60%|██████    | 22790/37962 [52:47:14<34:24:14,  8.16s/it]
                                                           
{'loss': 1.3027, 'grad_norm': 11.080344200134277, 'learning_rate': 7.27123903919015e-06, 'epoch': 0.6}

 60%|██████    | 22790/37962 [52:47:14<34:24:14,  8.16s/it]
 60%|██████    | 22791/37962 [52:47:22<34:29:02,  8.18s/it]
 60%|██████    | 22792/37962 [52:47:30<34:28:52,  8.18s/it]
 60%|██████    | 22793/37962 [52:47:38<34:25:29,  8.17s/it]
 60%|██████    | 22794/37962 [52:47:47<34:35:54,  8.21s/it]
 60%|██████    | 22795/37962 [52:47:55<34:36:18,  8.21s/it]
 60%|██████    | 22796/37962 [52:48:03<34:35:42,  8.21s/it]
 60%|██████    | 22797/37962 [52:48:11<34:50:28,  8.27s/it]
 60%|██████    | 22798/37962 [52:48:20<34:52:46,  8.28s/it]
 60%|██████    | 22799/37962 [52:48:28<34:44:16,  8.25s/it]
 60%|██████    | 22800/37962 [52:48:36<34:51:34,  8.28s/it]
                                                           
{'loss': 1.3054, 'grad_norm': 9.357545852661133, 'learning_rate': 7.263032210317371e-06, 'epoch': 0.6}

 60%|██████    | 22800/37962 [52:48:36<34:51:34,  8.28s/it]
 60%|██████    | 22801/37962 [52:48:44<34:40:03,  8.23s/it]
 60%|██████    | 22802/37962 [52:48:53<34:44:03,  8.25s/it]
 60%|██████    | 22803/37962 [52:49:01<34:45:45,  8.26s/it]
 60%|██████    | 22804/37962 [52:49:09<34:44:01,  8.25s/it]
 60%|██████    | 22805/37962 [52:49:18<34:52:04,  8.28s/it]
 60%|██████    | 22806/37962 [52:49:26<35:05:24,  8.33s/it]
 60%|██████    | 22807/37962 [52:49:34<34:37:06,  8.22s/it]
 60%|██████    | 22808/37962 [52:49:42<34:27:13,  8.18s/it]
 60%|██████    | 22809/37962 [52:49:50<34:41:47,  8.24s/it]
 60%|██████    | 22810/37962 [52:49:59<34:41:17,  8.24s/it]
                                                           
{'loss': 1.2021, 'grad_norm': 9.832756996154785, 'learning_rate': 7.254827373636035e-06, 'epoch': 0.6}

 60%|██████    | 22810/37962 [52:49:59<34:41:17,  8.24s/it]
 60%|██████    | 22811/37962 [52:50:07<34:50:20,  8.28s/it]
 60%|██████    | 22812/37962 [52:50:15<34:43:58,  8.25s/it]
 60%|██████    | 22813/37962 [52:50:23<34:33:35,  8.21s/it]
 60%|██████    | 22814/37962 [52:50:32<34:58:26,  8.31s/it]
 60%|██████    | 22815/37962 [52:50:40<34:52:08,  8.29s/it]
 60%|██████    | 22816/37962 [52:50:48<34:48:45,  8.27s/it]
 60%|██████    | 22817/37962 [52:50:57<34:48:50,  8.28s/it]
 60%|██████    | 22818/37962 [52:51:05<34:57:46,  8.31s/it]
 60%|██████    | 22819/37962 [52:51:13<35:07:29,  8.35s/it]
 60%|██████    | 22820/37962 [52:51:22<35:01:46,  8.33s/it]
                                                           
{'loss': 1.34, 'grad_norm': 10.783754348754883, 'learning_rate': 7.2466245351182966e-06, 'epoch': 0.6}

 60%|██████    | 22820/37962 [52:51:22<35:01:46,  8.33s/it]
 60%|██████    | 22821/37962 [52:51:30<34:42:57,  8.25s/it]
 60%|██████    | 22822/37962 [52:51:38<34:44:15,  8.26s/it]
 60%|██████    | 22823/37962 [52:51:46<34:47:29,  8.27s/it]
 60%|██████    | 22824/37962 [52:51:54<34:30:55,  8.21s/it]
 60%|██████    | 22825/37962 [52:52:03<34:52:55,  8.30s/it]
 60%|██████    | 22826/37962 [52:52:11<34:45:13,  8.27s/it]
 60%|██████    | 22827/37962 [52:52:20<35:03:52,  8.34s/it]
 60%|██████    | 22828/37962 [52:52:28<35:00:19,  8.33s/it]
 60%|██████    | 22829/37962 [52:52:36<35:00:28,  8.33s/it]
 60%|██████    | 22830/37962 [52:52:45<34:53:30,  8.30s/it]
                                                           
{'loss': 1.2609, 'grad_norm': 14.342830657958984, 'learning_rate': 7.238423700734858e-06, 'epoch': 0.6}

 60%|██████    | 22830/37962 [52:52:45<34:53:30,  8.30s/it]
 60%|██████    | 22831/37962 [52:52:53<34:49:15,  8.28s/it]
 60%|██████    | 22832/37962 [52:53:01<34:52:24,  8.30s/it]
 60%|██████    | 22833/37962 [52:53:09<34:32:29,  8.22s/it]
 60%|██████    | 22834/37962 [52:53:17<34:15:15,  8.15s/it]
 60%|██████    | 22835/37962 [52:53:25<34:26:45,  8.20s/it]
 60%|██████    | 22836/37962 [52:53:34<34:40:26,  8.25s/it]
 60%|██████    | 22837/37962 [52:53:42<34:43:59,  8.27s/it]
 60%|██████    | 22838/37962 [52:53:50<34:45:50,  8.27s/it]
 60%|██████    | 22839/37962 [52:53:59<34:46:03,  8.28s/it]
 60%|██████    | 22840/37962 [52:54:07<34:44:53,  8.27s/it]
                                                           
{'loss': 1.1743, 'grad_norm': 4.901958465576172, 'learning_rate': 7.230224876454966e-06, 'epoch': 0.6}

 60%|██████    | 22840/37962 [52:54:07<34:44:53,  8.27s/it]
 60%|██████    | 22841/37962 [52:54:15<34:44:29,  8.27s/it]
 60%|██████    | 22842/37962 [52:54:23<34:23:55,  8.19s/it]
 60%|██████    | 22843/37962 [52:54:32<34:43:58,  8.27s/it]
 60%|██████    | 22844/37962 [52:54:40<34:10:56,  8.14s/it]
 60%|██████    | 22845/37962 [52:54:48<34:43:12,  8.27s/it]
 60%|██████    | 22846/37962 [52:54:57<34:52:17,  8.30s/it]
 60%|██████    | 22847/37962 [52:55:05<34:42:54,  8.27s/it]
 60%|██████    | 22848/37962 [52:55:13<34:38:32,  8.25s/it]
 60%|██████    | 22849/37962 [52:55:21<34:42:00,  8.27s/it]
 60%|██████    | 22850/37962 [52:55:29<34:23:47,  8.19s/it]
                                                           
{'loss': 1.34, 'grad_norm': 12.286541938781738, 'learning_rate': 7.222028068246405e-06, 'epoch': 0.6}

 60%|██████    | 22850/37962 [52:55:29<34:23:47,  8.19s/it]
 60%|██████    | 22851/37962 [52:55:37<33:52:14,  8.07s/it]
 60%|██████    | 22852/37962 [52:55:45<33:52:58,  8.07s/it]
 60%|██████    | 22853/37962 [52:55:53<34:06:19,  8.13s/it]
 60%|██████    | 22854/37962 [52:56:02<34:19:42,  8.18s/it]
 60%|██████    | 22855/37962 [52:56:10<34:31:31,  8.23s/it]
 60%|██████    | 22856/37962 [52:56:18<34:52:28,  8.31s/it]
 60%|██████    | 22857/37962 [52:56:27<34:43:48,  8.28s/it]
 60%|██████    | 22858/37962 [52:56:35<35:00:36,  8.34s/it]
 60%|██████    | 22859/37962 [52:56:43<34:29:21,  8.22s/it]
 60%|██████    | 22860/37962 [52:56:51<34:27:24,  8.21s/it]
                                                           
{'loss': 1.2289, 'grad_norm': 5.526609897613525, 'learning_rate': 7.213833282075489e-06, 'epoch': 0.6}

 60%|██████    | 22860/37962 [52:56:51<34:27:24,  8.21s/it]
 60%|██████    | 22861/37962 [52:57:00<34:33:08,  8.24s/it]
 60%|██████    | 22862/37962 [52:57:08<34:38:59,  8.26s/it]
 60%|██████    | 22863/37962 [52:57:16<34:40:46,  8.27s/it]
 60%|██████    | 22864/37962 [52:57:24<34:14:53,  8.17s/it]
 60%|██████    | 22865/37962 [52:57:32<34:13:42,  8.16s/it]
 60%|██████    | 22866/37962 [52:57:41<34:21:33,  8.19s/it]
 60%|██████    | 22867/37962 [52:57:49<34:26:52,  8.22s/it]
 60%|██████    | 22868/37962 [52:57:57<34:30:30,  8.23s/it]
 60%|██████    | 22869/37962 [52:58:05<34:35:28,  8.25s/it]
 60%|██████    | 22870/37962 [52:58:14<34:34:17,  8.25s/it]
                                                           
{'loss': 1.2791, 'grad_norm': 11.265115737915039, 'learning_rate': 7.205640523907058e-06, 'epoch': 0.6}

 60%|██████    | 22870/37962 [52:58:14<34:34:17,  8.25s/it]
 60%|██████    | 22871/37962 [52:58:22<34:26:04,  8.21s/it]
 60%|██████    | 22872/37962 [52:58:30<34:24:27,  8.21s/it]
 60%|██████    | 22873/37962 [52:58:38<34:37:45,  8.26s/it]
 60%|██████    | 22874/37962 [52:58:47<34:39:38,  8.27s/it]
 60%|██████    | 22875/37962 [52:58:55<34:30:58,  8.24s/it]
 60%|██████    | 22876/37962 [52:59:11<44:24:52, 10.60s/it]
 60%|██████    | 22877/37962 [52:59:19<41:25:48,  9.89s/it]
 60%|██████    | 22878/37962 [52:59:28<39:42:01,  9.48s/it]
 60%|██████    | 22879/37962 [52:59:36<38:07:55,  9.10s/it]
 60%|██████    | 22880/37962 [52:59:44<37:26:04,  8.94s/it]
                                                           
{'loss': 1.3153, 'grad_norm': 6.980140686035156, 'learning_rate': 7.197449799704481e-06, 'epoch': 0.6}

 60%|██████    | 22880/37962 [52:59:44<37:26:04,  8.94s/it]
 60%|██████    | 22881/37962 [52:59:53<36:28:56,  8.71s/it]
 60%|██████    | 22882/37962 [53:00:01<36:11:45,  8.64s/it]
 60%|██████    | 22883/37962 [53:00:09<35:19:32,  8.43s/it]
 60%|██████    | 22884/37962 [53:00:17<34:37:03,  8.27s/it]
 60%|██████    | 22885/37962 [53:00:25<34:44:07,  8.29s/it]
 60%|██████    | 22886/37962 [53:00:41<43:31:10, 10.39s/it]
 60%|██████    | 22887/37962 [53:00:48<40:19:55,  9.63s/it]
 60%|██████    | 22888/37962 [53:00:57<38:39:22,  9.23s/it]
 60%|██████    | 22889/37962 [53:01:05<37:24:44,  8.94s/it]
 60%|██████    | 22890/37962 [53:01:13<36:32:20,  8.73s/it]
                                                           
{'loss': 1.2374, 'grad_norm': 3.178445816040039, 'learning_rate': 7.189261115429642e-06, 'epoch': 0.6}

 60%|██████    | 22890/37962 [53:01:13<36:32:20,  8.73s/it]
 60%|██████    | 22891/37962 [53:01:22<36:05:04,  8.62s/it]
 60%|██████    | 22892/37962 [53:01:30<35:48:51,  8.56s/it]
 60%|██████    | 22893/37962 [53:01:38<35:24:55,  8.46s/it]
 60%|██████    | 22894/37962 [53:01:46<35:06:31,  8.39s/it]
 60%|██████    | 22895/37962 [53:01:54<34:18:37,  8.20s/it]
 60%|██████    | 22896/37962 [53:02:03<34:38:00,  8.28s/it]
 60%|██████    | 22897/37962 [53:02:11<34:30:45,  8.25s/it]
 60%|██████    | 22898/37962 [53:02:19<34:30:49,  8.25s/it]
 60%|██████    | 22899/37962 [53:02:27<34:41:11,  8.29s/it]
 60%|██████    | 22900/37962 [53:02:36<34:42:09,  8.29s/it]
                                                           
{'loss': 1.278, 'grad_norm': 5.948081016540527, 'learning_rate': 7.1810744770429465e-06, 'epoch': 0.6}

 60%|██████    | 22900/37962 [53:02:36<34:42:09,  8.29s/it]
 60%|██████    | 22901/37962 [53:02:44<34:21:57,  8.21s/it]
 60%|██████    | 22902/37962 [53:02:52<33:58:16,  8.12s/it]
 60%|██████    | 22903/37962 [53:03:00<34:05:01,  8.15s/it]
 60%|██████    | 22904/37962 [53:03:08<33:39:05,  8.05s/it]
 60%|██████    | 22905/37962 [53:03:16<34:05:45,  8.15s/it]
 60%|██████    | 22906/37962 [53:03:25<34:31:57,  8.26s/it]
 60%|██████    | 22907/37962 [53:03:33<34:37:32,  8.28s/it]
 60%|██████    | 22908/37962 [53:03:41<34:32:54,  8.26s/it]
 60%|██████    | 22909/37962 [53:03:49<34:31:00,  8.25s/it]
 60%|██████    | 22910/37962 [53:03:58<34:35:15,  8.27s/it]
                                                           
{'loss': 1.3081, 'grad_norm': 7.235008716583252, 'learning_rate': 7.172889890503303e-06, 'epoch': 0.6}

 60%|██████    | 22910/37962 [53:03:58<34:35:15,  8.27s/it]
 60%|██████    | 22911/37962 [53:04:06<34:36:01,  8.28s/it]
 60%|██████    | 22912/37962 [53:04:14<34:38:44,  8.29s/it]
 60%|██████    | 22913/37962 [53:04:23<35:01:59,  8.38s/it]
 60%|██████    | 22914/37962 [53:04:31<34:50:53,  8.34s/it]
 60%|██████    | 22915/37962 [53:04:39<34:49:32,  8.33s/it]
 60%|██████    | 22916/37962 [53:04:48<34:29:50,  8.25s/it]
 60%|██████    | 22917/37962 [53:04:56<34:53:30,  8.35s/it]
 60%|██████    | 22918/37962 [53:05:04<34:48:15,  8.33s/it]
 60%|██████    | 22919/37962 [53:05:13<34:40:45,  8.30s/it]
 60%|██████    | 22920/37962 [53:05:21<34:42:23,  8.31s/it]
                                                           
{'loss': 1.2099, 'grad_norm': 3.4584312438964844, 'learning_rate': 7.164707361768136e-06, 'epoch': 0.6}

 60%|██████    | 22920/37962 [53:05:21<34:42:23,  8.31s/it]
 60%|██████    | 22921/37962 [53:05:29<34:35:43,  8.28s/it]
 60%|██████    | 22922/37962 [53:05:37<34:21:18,  8.22s/it]
 60%|██████    | 22923/37962 [53:05:46<34:42:23,  8.31s/it]
 60%|██████    | 22924/37962 [53:05:54<34:46:55,  8.33s/it]
 60%|██████    | 22925/37962 [53:06:03<34:53:41,  8.35s/it]
 60%|██████    | 22926/37962 [53:06:11<35:11:32,  8.43s/it]
 60%|██████    | 22927/37962 [53:06:20<35:17:03,  8.45s/it]
 60%|██████    | 22928/37962 [53:06:28<35:06:29,  8.41s/it]
 60%|██████    | 22929/37962 [53:06:36<34:52:59,  8.35s/it]
 60%|██████    | 22930/37962 [53:06:45<34:59:31,  8.38s/it]
                                                           
{'loss': 1.258, 'grad_norm': 8.108735084533691, 'learning_rate': 7.156526896793359e-06, 'epoch': 0.6}

 60%|██████    | 22930/37962 [53:06:45<34:59:31,  8.38s/it]
 60%|██████    | 22931/37962 [53:06:53<34:53:27,  8.36s/it]
 60%|██████    | 22932/37962 [53:07:01<35:04:18,  8.40s/it]
 60%|██████    | 22933/37962 [53:07:10<34:52:51,  8.36s/it]
 60%|██████    | 22934/37962 [53:07:18<35:02:12,  8.39s/it]
 60%|██████    | 22935/37962 [53:07:26<34:46:18,  8.33s/it]
 60%|██████    | 22936/37962 [53:07:34<34:30:46,  8.27s/it]
 60%|██████    | 22937/37962 [53:07:43<34:25:18,  8.25s/it]
 60%|██████    | 22938/37962 [53:07:51<34:38:07,  8.30s/it]
 60%|██████    | 22939/37962 [53:07:59<34:20:02,  8.23s/it]
 60%|██████    | 22940/37962 [53:08:07<34:29:12,  8.26s/it]
                                                           
{'loss': 1.2228, 'grad_norm': 13.977660179138184, 'learning_rate': 7.148348501533396e-06, 'epoch': 0.6}

 60%|██████    | 22940/37962 [53:08:08<34:29:12,  8.26s/it]
 60%|██████    | 22941/37962 [53:08:16<34:27:20,  8.26s/it]
 60%|██████    | 22942/37962 [53:08:24<34:27:12,  8.26s/it]
 60%|██████    | 22943/37962 [53:08:32<34:16:07,  8.21s/it]
 60%|██████    | 22944/37962 [53:08:40<34:07:21,  8.18s/it]
 60%|██████    | 22945/37962 [53:08:48<34:10:24,  8.19s/it]
 60%|██████    | 22946/37962 [53:08:57<34:01:18,  8.16s/it]
 60%|██████    | 22947/37962 [53:09:05<34:20:02,  8.23s/it]
 60%|██████    | 22948/37962 [53:09:13<34:21:06,  8.24s/it]
 60%|██████    | 22949/37962 [53:09:21<33:47:51,  8.10s/it]
 60%|██████    | 22950/37962 [53:09:29<33:59:21,  8.15s/it]
                                                           
{'loss': 1.2565, 'grad_norm': 7.721965312957764, 'learning_rate': 7.140172181941154e-06, 'epoch': 0.6}

 60%|██████    | 22950/37962 [53:09:29<33:59:21,  8.15s/it]
 60%|██████    | 22951/37962 [53:09:38<34:15:34,  8.22s/it]
 60%|██████    | 22952/37962 [53:09:46<34:25:21,  8.26s/it]
 60%|██████    | 22953/37962 [53:09:54<34:10:03,  8.20s/it]
 60%|██████    | 22954/37962 [53:10:02<34:21:18,  8.24s/it]
 60%|██████    | 22955/37962 [53:10:11<34:40:27,  8.32s/it]
 60%|██████    | 22956/37962 [53:10:19<34:20:11,  8.24s/it]
 60%|██████    | 22957/37962 [53:10:27<34:27:55,  8.27s/it]
 60%|██████    | 22958/37962 [53:10:36<34:31:23,  8.28s/it]
 60%|██████    | 22959/37962 [53:10:44<34:24:11,  8.26s/it]
 60%|██████    | 22960/37962 [53:10:52<34:29:32,  8.28s/it]
                                                           
{'loss': 1.3464, 'grad_norm': 2.3341660499572754, 'learning_rate': 7.131997943968039e-06, 'epoch': 0.6}

 60%|██████    | 22960/37962 [53:10:52<34:29:32,  8.28s/it]
 60%|██████    | 22961/37962 [53:11:00<34:27:48,  8.27s/it]
 60%|██████    | 22962/37962 [53:11:09<34:44:06,  8.34s/it]
 60%|██████    | 22963/37962 [53:11:17<34:38:54,  8.32s/it]
 60%|██████    | 22964/37962 [53:11:25<34:30:53,  8.28s/it]
 60%|██████    | 22965/37962 [53:11:34<34:27:25,  8.27s/it]
 60%|██████    | 22966/37962 [53:11:42<34:24:04,  8.26s/it]
 60%|██████    | 22967/37962 [53:11:50<34:25:09,  8.26s/it]
 61%|██████    | 22968/37962 [53:11:58<34:31:59,  8.29s/it]
 61%|██████    | 22969/37962 [53:12:06<34:15:10,  8.22s/it]
 61%|██████    | 22970/37962 [53:12:15<34:13:32,  8.22s/it]
                                                           
{'loss': 1.2324, 'grad_norm': 15.479608535766602, 'learning_rate': 7.123825793563939e-06, 'epoch': 0.61}

 61%|██████    | 22970/37962 [53:12:15<34:13:32,  8.22s/it]
 61%|██████    | 22971/37962 [53:12:23<34:15:46,  8.23s/it]
 61%|██████    | 22972/37962 [53:12:31<34:40:24,  8.33s/it]
 61%|██████    | 22973/37962 [53:12:40<34:45:10,  8.35s/it]
 61%|██████    | 22974/37962 [53:12:48<34:33:22,  8.30s/it]
 61%|██████    | 22975/37962 [53:12:56<34:44:01,  8.34s/it]
 61%|██████    | 22976/37962 [53:13:05<34:55:59,  8.39s/it]
 61%|██████    | 22977/37962 [53:13:13<34:41:46,  8.34s/it]
 61%|██████    | 22978/37962 [53:13:21<34:34:07,  8.31s/it]
 61%|██████    | 22979/37962 [53:13:30<34:16:52,  8.24s/it]
 61%|██████    | 22980/37962 [53:13:38<34:04:16,  8.19s/it]
                                                           
{'loss': 1.3919, 'grad_norm': 8.016624450683594, 'learning_rate': 7.115655736677216e-06, 'epoch': 0.61}

 61%|██████    | 22980/37962 [53:13:38<34:04:16,  8.19s/it]
 61%|██████    | 22981/37962 [53:13:46<34:09:46,  8.21s/it]
 61%|██████    | 22982/37962 [53:13:54<34:14:48,  8.23s/it]
 61%|██████    | 22983/37962 [53:14:02<34:21:11,  8.26s/it]
 61%|██████    | 22984/37962 [53:14:11<34:20:03,  8.25s/it]
 61%|██████    | 22985/37962 [53:14:19<34:35:47,  8.32s/it]
 61%|██████    | 22986/37962 [53:14:28<34:38:21,  8.33s/it]
 61%|██████    | 22987/37962 [53:14:36<34:32:23,  8.30s/it]
 61%|██████    | 22988/37962 [53:14:44<34:28:39,  8.29s/it]
 61%|██████    | 22989/37962 [53:14:52<34:42:39,  8.35s/it]
 61%|██████    | 22990/37962 [53:15:01<34:35:13,  8.32s/it]
                                                           
{'loss': 1.1698, 'grad_norm': 6.168022155761719, 'learning_rate': 7.10748777925471e-06, 'epoch': 0.61}

 61%|██████    | 22990/37962 [53:15:01<34:35:13,  8.32s/it]
 61%|██████    | 22991/37962 [53:15:09<34:33:27,  8.31s/it]
 61%|██████    | 22992/37962 [53:15:17<34:30:42,  8.30s/it]
 61%|██████    | 22993/37962 [53:15:25<34:17:20,  8.25s/it]
 61%|██████    | 22994/37962 [53:15:34<34:29:06,  8.29s/it]
 61%|██████    | 22995/37962 [53:15:42<34:33:05,  8.31s/it]
 61%|██████    | 22996/37962 [53:15:50<34:24:54,  8.28s/it]
 61%|██████    | 22997/37962 [53:15:59<34:19:53,  8.26s/it]
 61%|██████    | 22998/37962 [53:16:07<34:16:29,  8.25s/it]
 61%|██████    | 22999/37962 [53:16:15<34:04:21,  8.20s/it]
 61%|██████    | 23000/37962 [53:16:23<34:02:16,  8.19s/it]
                                                           
{'loss': 1.1989, 'grad_norm': 3.46389102935791, 'learning_rate': 7.0993219272417444e-06, 'epoch': 0.61}

 61%|██████    | 23000/37962 [53:16:23<34:02:16,  8.19s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 61%|██████    | 23001/37962 [53:17:15<88:14:24, 21.23s/it]
 61%|██████    | 23002/37962 [53:17:23<71:49:26, 17.28s/it]
 61%|██████    | 23003/37962 [53:17:31<60:17:11, 14.51s/it]
 61%|██████    | 23004/37962 [53:17:39<52:32:18, 12.64s/it]
 61%|██████    | 23005/37962 [53:17:47<47:03:45, 11.33s/it]
 61%|██████    | 23006/37962 [53:17:56<43:11:59, 10.40s/it]
 61%|██████    | 23007/37962 [53:18:04<40:30:41,  9.75s/it]
 61%|██████    | 23008/37962 [53:18:12<38:41:37,  9.32s/it]
 61%|██████    | 23009/37962 [53:18:20<37:06:05,  8.93s/it]
 61%|██████    | 23010/37962 [53:18:29<36:20:26,  8.75s/it]
                                                           
{'loss': 1.2705, 'grad_norm': 5.359439373016357, 'learning_rate': 7.091158186582095e-06, 'epoch': 0.61}

 61%|██████    | 23010/37962 [53:18:29<36:20:26,  8.75s/it]
 61%|██████    | 23011/37962 [53:18:37<35:56:27,  8.65s/it]
 61%|██████    | 23012/37962 [53:18:45<35:27:50,  8.54s/it]
 61%|██████    | 23013/37962 [53:18:54<35:18:10,  8.50s/it]
 61%|██████    | 23014/37962 [53:19:02<34:57:23,  8.42s/it]
 61%|██████    | 23015/37962 [53:19:10<34:49:33,  8.39s/it]
 61%|██████    | 23016/37962 [53:19:18<34:36:05,  8.33s/it]
 61%|██████    | 23017/37962 [53:19:27<34:31:34,  8.32s/it]
 61%|██████    | 23018/37962 [53:19:35<34:29:25,  8.31s/it]
 61%|██████    | 23019/37962 [53:19:43<34:31:23,  8.32s/it]
 61%|██████    | 23020/37962 [53:19:51<34:20:24,  8.27s/it]
                                                           
{'loss': 1.3928, 'grad_norm': 9.111003875732422, 'learning_rate': 7.082996563218013e-06, 'epoch': 0.61}

 61%|██████    | 23020/37962 [53:19:51<34:20:24,  8.27s/it]
 61%|██████    | 23021/37962 [53:19:59<33:51:40,  8.16s/it]
 61%|██████    | 23022/37962 [53:20:08<34:01:45,  8.20s/it]
 61%|██████    | 23023/37962 [53:20:16<34:07:01,  8.22s/it]
 61%|██████    | 23024/37962 [53:20:24<34:23:10,  8.29s/it]
 61%|██████    | 23025/37962 [53:20:32<34:11:48,  8.24s/it]
 61%|██████    | 23026/37962 [53:20:41<34:20:43,  8.28s/it]
 61%|██████    | 23027/37962 [53:20:49<34:17:36,  8.27s/it]
 61%|██████    | 23028/37962 [53:20:57<34:25:34,  8.30s/it]
 61%|██████    | 23029/37962 [53:21:06<34:28:08,  8.31s/it]
 61%|██████    | 23030/37962 [53:21:14<34:20:39,  8.28s/it]
                                                           
{'loss': 1.2596, 'grad_norm': 11.575864791870117, 'learning_rate': 7.074837063090196e-06, 'epoch': 0.61}

 61%|██████    | 23030/37962 [53:21:14<34:20:39,  8.28s/it]
 61%|██████    | 23031/37962 [53:21:22<34:15:30,  8.26s/it]
 61%|██████    | 23032/37962 [53:21:31<34:24:16,  8.30s/it]
 61%|██████    | 23033/37962 [53:21:39<34:18:56,  8.27s/it]
 61%|██████    | 23034/37962 [53:21:47<34:17:34,  8.27s/it]
 61%|██████    | 23035/37962 [53:21:55<33:50:37,  8.16s/it]
 61%|██████    | 23036/37962 [53:22:03<34:02:48,  8.21s/it]
 61%|██████    | 23037/37962 [53:22:11<33:53:45,  8.18s/it]
 61%|██████    | 23038/37962 [53:22:20<33:46:33,  8.15s/it]
 61%|██████    | 23039/37962 [53:22:28<33:48:33,  8.16s/it]
 61%|██████    | 23040/37962 [53:22:36<34:08:23,  8.24s/it]
                                                           
{'loss': 1.2704, 'grad_norm': 7.501400470733643, 'learning_rate': 7.066679692137808e-06, 'epoch': 0.61}

 61%|██████    | 23040/37962 [53:22:36<34:08:23,  8.24s/it]
 61%|██████    | 23041/37962 [53:22:45<34:41:12,  8.37s/it]
 61%|██████    | 23042/37962 [53:22:53<34:33:50,  8.34s/it]
 61%|██████    | 23043/37962 [53:23:02<34:44:17,  8.38s/it]
 61%|██████    | 23044/37962 [53:23:10<34:33:35,  8.34s/it]
 61%|██████    | 23045/37962 [53:23:18<34:05:25,  8.23s/it]
 61%|██████    | 23046/37962 [53:23:26<34:10:36,  8.25s/it]
 61%|██████    | 23047/37962 [53:23:34<34:16:38,  8.27s/it]
 61%|██████    | 23048/37962 [53:23:43<34:16:17,  8.27s/it]
 61%|██████    | 23049/37962 [53:23:51<34:19:59,  8.29s/it]
 61%|██████    | 23050/37962 [53:23:59<34:13:27,  8.26s/it]
                                                           
{'loss': 1.2956, 'grad_norm': 14.277399063110352, 'learning_rate': 7.058524456298454e-06, 'epoch': 0.61}

 61%|██████    | 23050/37962 [53:23:59<34:13:27,  8.26s/it]
 61%|██████    | 23051/37962 [53:24:07<34:01:46,  8.22s/it]
 61%|██████    | 23052/37962 [53:24:15<34:01:28,  8.22s/it]
 61%|██████    | 23053/37962 [53:24:24<34:23:29,  8.30s/it]
 61%|██████    | 23054/37962 [53:24:32<34:12:24,  8.26s/it]
 61%|██████    | 23055/37962 [53:24:40<33:37:03,  8.12s/it]
 61%|██████    | 23056/37962 [53:24:48<33:42:49,  8.14s/it]
 61%|██████    | 23057/37962 [53:24:56<33:50:22,  8.17s/it]
 61%|██████    | 23058/37962 [53:25:04<33:38:14,  8.12s/it]
 61%|██████    | 23059/37962 [53:25:13<33:43:48,  8.15s/it]
 61%|██████    | 23060/37962 [53:25:21<33:49:11,  8.17s/it]
                                                           
{'loss': 1.3149, 'grad_norm': 8.56878662109375, 'learning_rate': 7.0503713615081885e-06, 'epoch': 0.61}

 61%|██████    | 23060/37962 [53:25:21<33:49:11,  8.17s/it]
 61%|██████    | 23061/37962 [53:25:29<33:27:48,  8.08s/it]
 61%|██████    | 23062/37962 [53:25:37<33:54:26,  8.19s/it]
 61%|██████    | 23063/37962 [53:25:45<33:59:10,  8.21s/it]
 61%|██████    | 23064/37962 [53:25:54<34:07:33,  8.25s/it]
 61%|██████    | 23065/37962 [53:26:02<33:51:46,  8.18s/it]
 61%|██████    | 23066/37962 [53:26:10<34:03:30,  8.23s/it]
 61%|██████    | 23067/37962 [53:26:18<34:02:32,  8.23s/it]
 61%|██████    | 23068/37962 [53:26:27<34:07:09,  8.25s/it]
 61%|██████    | 23069/37962 [53:26:35<33:40:23,  8.14s/it]
 61%|██████    | 23070/37962 [53:26:43<33:54:46,  8.20s/it]
                                                           
{'loss': 1.1996, 'grad_norm': 7.688422679901123, 'learning_rate': 7.042220413701516e-06, 'epoch': 0.61}

 61%|██████    | 23070/37962 [53:26:43<33:54:46,  8.20s/it]
 61%|██████    | 23071/37962 [53:26:51<33:40:46,  8.14s/it]
 61%|██████    | 23072/37962 [53:26:59<33:16:58,  8.05s/it]
 61%|██████    | 23073/37962 [53:27:07<33:39:15,  8.14s/it]
 61%|██████    | 23074/37962 [53:27:15<33:41:19,  8.15s/it]
 61%|██████    | 23075/37962 [53:27:24<34:09:39,  8.26s/it]
 61%|██████    | 23076/37962 [53:27:32<34:10:53,  8.27s/it]
 61%|██████    | 23077/37962 [53:27:40<34:03:42,  8.24s/it]
 61%|██████    | 23078/37962 [53:27:48<34:05:09,  8.24s/it]
 61%|██████    | 23079/37962 [53:27:57<34:11:04,  8.27s/it]
 61%|██████    | 23080/37962 [53:28:05<34:24:42,  8.32s/it]
                                                           
{'loss': 1.2929, 'grad_norm': 3.858163356781006, 'learning_rate': 7.034071618811361e-06, 'epoch': 0.61}

 61%|██████    | 23080/37962 [53:28:05<34:24:42,  8.32s/it]
 61%|██████    | 23081/37962 [53:28:14<34:26:52,  8.33s/it]
 61%|██████    | 23082/37962 [53:28:22<34:13:12,  8.28s/it]
 61%|██████    | 23083/37962 [53:28:30<34:12:33,  8.28s/it]
 61%|██████    | 23084/37962 [53:28:38<34:13:09,  8.28s/it]
 61%|██████    | 23085/37962 [53:28:47<34:29:07,  8.34s/it]
 61%|██████    | 23086/37962 [53:28:55<34:30:42,  8.35s/it]
 61%|██████    | 23087/37962 [53:29:03<34:12:11,  8.28s/it]
 61%|██████    | 23088/37962 [53:29:12<34:17:21,  8.30s/it]
 61%|██████    | 23089/37962 [53:29:27<43:06:32, 10.43s/it]
 61%|██████    | 23090/37962 [53:29:36<40:47:26,  9.87s/it]
                                                           
{'loss': 1.2255, 'grad_norm': 14.03501033782959, 'learning_rate': 7.025924982769091e-06, 'epoch': 0.61}

 61%|██████    | 23090/37962 [53:29:36<40:47:26,  9.87s/it]
 61%|██████    | 23091/37962 [53:29:44<38:30:44,  9.32s/it]
 61%|██████    | 23092/37962 [53:29:52<37:17:53,  9.03s/it]
 61%|██████    | 23093/37962 [53:30:00<36:05:05,  8.74s/it]
 61%|██████    | 23094/37962 [53:30:08<35:21:49,  8.56s/it]
 61%|██████    | 23095/37962 [53:30:16<34:57:25,  8.46s/it]
 61%|██████    | 23096/37962 [53:30:25<34:54:02,  8.45s/it]
 61%|██████    | 23097/37962 [53:30:33<34:42:47,  8.41s/it]
 61%|██████    | 23098/37962 [53:30:41<34:12:53,  8.29s/it]
 61%|██████    | 23099/37962 [53:30:49<34:14:52,  8.30s/it]
 61%|██████    | 23100/37962 [53:30:58<34:07:54,  8.27s/it]
                                                           
{'loss': 1.3797, 'grad_norm': 9.552128791809082, 'learning_rate': 7.0177805115045065e-06, 'epoch': 0.61}

 61%|██████    | 23100/37962 [53:30:58<34:07:54,  8.27s/it]
 61%|██████    | 23101/37962 [53:31:06<34:20:04,  8.32s/it]
 61%|██████    | 23102/37962 [53:31:14<34:14:07,  8.29s/it]
 61%|██████    | 23103/37962 [53:31:23<34:24:38,  8.34s/it]
 61%|██████    | 23104/37962 [53:31:31<34:24:28,  8.34s/it]
 61%|██████    | 23105/37962 [53:31:39<34:09:35,  8.28s/it]
 61%|██████    | 23106/37962 [53:31:47<33:55:54,  8.22s/it]
 61%|██████    | 23107/37962 [53:31:56<34:00:32,  8.24s/it]
 61%|██████    | 23108/37962 [53:32:04<34:13:08,  8.29s/it]
 61%|██████    | 23109/37962 [53:32:12<34:08:11,  8.27s/it]
 61%|██████    | 23110/37962 [53:32:21<34:21:25,  8.33s/it]
                                                           
{'loss': 1.2575, 'grad_norm': 4.1083269119262695, 'learning_rate': 7.009638210945824e-06, 'epoch': 0.61}

 61%|██████    | 23110/37962 [53:32:21<34:21:25,  8.33s/it]
 61%|██████    | 23111/37962 [53:32:29<34:17:48,  8.31s/it]
 61%|██████    | 23112/37962 [53:32:37<34:16:31,  8.31s/it]
 61%|██████    | 23113/37962 [53:32:46<34:18:29,  8.32s/it]
 61%|██████    | 23114/37962 [53:32:54<34:03:02,  8.26s/it]
 61%|██████    | 23115/37962 [53:33:02<34:38:31,  8.40s/it]
 61%|██████    | 23116/37962 [53:33:11<34:16:02,  8.31s/it]
 61%|██████    | 23117/37962 [53:33:19<33:54:26,  8.22s/it]
 61%|██████    | 23118/37962 [53:33:27<33:53:45,  8.22s/it]
 61%|██████    | 23119/37962 [53:33:35<33:58:19,  8.24s/it]
 61%|██████    | 23120/37962 [53:33:43<33:59:43,  8.25s/it]
                                                           
{'loss': 1.1832, 'grad_norm': 6.741252899169922, 'learning_rate': 7.001498087019683e-06, 'epoch': 0.61}

 61%|██████    | 23120/37962 [53:33:43<33:59:43,  8.25s/it]
 61%|██████    | 23121/37962 [53:33:52<34:07:05,  8.28s/it]
 61%|██████    | 23122/37962 [53:34:00<34:16:49,  8.32s/it]
 61%|██████    | 23123/37962 [53:34:08<34:08:23,  8.28s/it]
 61%|██████    | 23124/37962 [53:34:17<34:18:53,  8.33s/it]
 61%|██████    | 23125/37962 [53:34:25<34:17:47,  8.32s/it]
 61%|██████    | 23126/37962 [53:34:34<34:28:31,  8.37s/it]
 61%|██████    | 23127/37962 [53:34:42<34:18:20,  8.32s/it]
 61%|██████    | 23128/37962 [53:34:50<34:12:37,  8.30s/it]
 61%|██████    | 23129/37962 [53:34:58<34:05:27,  8.27s/it]
 61%|██████    | 23130/37962 [53:35:06<34:01:25,  8.26s/it]
                                                           
{'loss': 1.158, 'grad_norm': 8.085142135620117, 'learning_rate': 6.99336014565114e-06, 'epoch': 0.61}

 61%|██████    | 23130/37962 [53:35:06<34:01:25,  8.26s/it]
 61%|██████    | 23131/37962 [53:35:15<34:15:33,  8.32s/it]
 61%|██████    | 23132/37962 [53:35:23<34:17:11,  8.32s/it]
 61%|██████    | 23133/37962 [53:35:32<34:39:08,  8.41s/it]
 61%|██████    | 23134/37962 [53:35:40<34:32:27,  8.39s/it]
 61%|██████    | 23135/37962 [53:35:48<34:18:59,  8.33s/it]
 61%|██████    | 23136/37962 [53:35:57<34:07:35,  8.29s/it]
 61%|██████    | 23137/37962 [53:36:05<33:47:14,  8.20s/it]
 61%|██████    | 23138/37962 [53:36:13<33:52:00,  8.22s/it]
 61%|██████    | 23139/37962 [53:36:21<33:34:44,  8.16s/it]
 61%|██████    | 23140/37962 [53:36:29<33:35:52,  8.16s/it]
                                                           
{'loss': 1.309, 'grad_norm': 9.445575714111328, 'learning_rate': 6.9852243927636595e-06, 'epoch': 0.61}

 61%|██████    | 23140/37962 [53:36:29<33:35:52,  8.16s/it]
 61%|██████    | 23141/37962 [53:36:37<33:35:19,  8.16s/it]
 61%|██████    | 23142/37962 [53:36:45<33:45:04,  8.20s/it]
 61%|██████    | 23143/37962 [53:36:54<33:48:48,  8.21s/it]
 61%|██████    | 23144/37962 [53:37:02<33:57:14,  8.25s/it]
 61%|██████    | 23145/37962 [53:37:10<34:10:46,  8.30s/it]
 61%|██████    | 23146/37962 [53:37:19<34:14:36,  8.32s/it]
 61%|██████    | 23147/37962 [53:37:27<34:09:33,  8.30s/it]
 61%|██████    | 23148/37962 [53:37:35<34:15:42,  8.33s/it]
 61%|██████    | 23149/37962 [53:37:44<34:23:31,  8.36s/it]
 61%|██████    | 23150/37962 [53:37:52<34:31:57,  8.39s/it]
                                                           
{'loss': 1.431, 'grad_norm': 6.4371819496154785, 'learning_rate': 6.977090834279117e-06, 'epoch': 0.61}

 61%|██████    | 23150/37962 [53:37:52<34:31:57,  8.39s/it]
 61%|██████    | 23151/37962 [53:38:01<34:20:52,  8.35s/it]
 61%|██████    | 23152/37962 [53:38:09<34:12:46,  8.32s/it]
 61%|██████    | 23153/37962 [53:38:17<34:21:12,  8.35s/it]
 61%|██████    | 23154/37962 [53:38:26<34:17:44,  8.34s/it]
 61%|██████    | 23155/37962 [53:38:34<33:44:55,  8.21s/it]
 61%|██████    | 23156/37962 [53:38:42<33:48:40,  8.22s/it]
 61%|██████    | 23157/37962 [53:38:50<33:53:37,  8.24s/it]
 61%|██████    | 23158/37962 [53:38:58<33:23:05,  8.12s/it]
 61%|██████    | 23159/37962 [53:39:07<34:08:32,  8.30s/it]
 61%|██████    | 23160/37962 [53:39:15<33:42:41,  8.20s/it]
                                                           
{'loss': 1.3927, 'grad_norm': 11.130436897277832, 'learning_rate': 6.968959476117787e-06, 'epoch': 0.61}

 61%|██████    | 23160/37962 [53:39:15<33:42:41,  8.20s/it]
 61%|██████    | 23161/37962 [53:39:23<33:48:35,  8.22s/it]
 61%|██████    | 23162/37962 [53:39:31<34:02:11,  8.28s/it]
 61%|██████    | 23163/37962 [53:39:39<33:40:30,  8.19s/it]
 61%|██████    | 23164/37962 [53:39:48<33:45:29,  8.21s/it]
 61%|██████    | 23165/37962 [53:39:56<33:45:43,  8.21s/it]
 61%|██████    | 23166/37962 [53:40:04<33:59:30,  8.27s/it]
 61%|██████    | 23167/37962 [53:40:13<34:18:18,  8.35s/it]
 61%|██████    | 23168/37962 [53:40:21<33:55:44,  8.26s/it]
 61%|██████    | 23169/37962 [53:40:29<34:00:21,  8.28s/it]
 61%|██████    | 23170/37962 [53:40:37<33:30:24,  8.15s/it]
                                                           
{'loss': 1.1526, 'grad_norm': 6.79194974899292, 'learning_rate': 6.960830324198342e-06, 'epoch': 0.61}

 61%|██████    | 23170/37962 [53:40:37<33:30:24,  8.15s/it]
 61%|██████    | 23171/37962 [53:40:45<33:39:23,  8.19s/it]
 61%|██████    | 23172/37962 [53:40:53<33:20:02,  8.11s/it]
 61%|██████    | 23173/37962 [53:41:01<33:30:50,  8.16s/it]
 61%|██████    | 23174/37962 [53:41:10<33:39:33,  8.19s/it]
 61%|██████    | 23175/37962 [53:41:18<33:38:44,  8.19s/it]
 61%|██████    | 23176/37962 [53:41:26<33:11:46,  8.08s/it]
 61%|██████    | 23177/37962 [53:41:34<33:23:50,  8.13s/it]
 61%|██████    | 23178/37962 [53:41:42<33:20:50,  8.12s/it]
 61%|██████    | 23179/37962 [53:41:50<33:45:05,  8.22s/it]
 61%|██████    | 23180/37962 [53:41:59<33:51:48,  8.25s/it]
                                                           
{'loss': 1.2336, 'grad_norm': 6.564202308654785, 'learning_rate': 6.952703384437863e-06, 'epoch': 0.61}

 61%|██████    | 23180/37962 [53:41:59<33:51:48,  8.25s/it]
 61%|██████    | 23181/37962 [53:42:07<33:53:33,  8.25s/it]
 61%|██████    | 23182/37962 [53:42:15<33:43:19,  8.21s/it]
 61%|██████    | 23183/37962 [53:42:23<33:48:27,  8.24s/it]
 61%|██████    | 23184/37962 [53:42:32<33:56:51,  8.27s/it]
 61%|██████    | 23185/37962 [53:42:40<34:12:52,  8.34s/it]
 61%|██████    | 23186/37962 [53:42:49<34:07:28,  8.31s/it]
 61%|██████    | 23187/37962 [53:42:57<34:18:23,  8.36s/it]
 61%|██████    | 23188/37962 [53:43:06<34:38:44,  8.44s/it]
 61%|██████    | 23189/37962 [53:43:14<34:20:53,  8.37s/it]
 61%|██████    | 23190/37962 [53:43:22<33:57:29,  8.28s/it]
                                                           
{'loss': 1.3147, 'grad_norm': 7.813756465911865, 'learning_rate': 6.944578662751791e-06, 'epoch': 0.61}

 61%|██████    | 23190/37962 [53:43:22<33:57:29,  8.28s/it]
 61%|██████    | 23191/37962 [53:43:30<34:02:16,  8.30s/it]
 61%|██████    | 23192/37962 [53:43:39<34:17:49,  8.36s/it]
 61%|██████    | 23193/37962 [53:43:47<34:22:26,  8.38s/it]
 61%|██████    | 23194/37962 [53:43:55<34:10:40,  8.33s/it]
 61%|██████    | 23195/37962 [53:44:04<34:05:45,  8.31s/it]
 61%|██████    | 23196/37962 [53:44:12<34:09:42,  8.33s/it]
 61%|██████    | 23197/37962 [53:44:20<33:43:46,  8.22s/it]
 61%|██████    | 23198/37962 [53:44:28<33:28:42,  8.16s/it]
 61%|██████    | 23199/37962 [53:44:36<33:40:55,  8.21s/it]
 61%|██████    | 23200/37962 [53:44:45<33:50:58,  8.25s/it]
                                                           
{'loss': 1.2858, 'grad_norm': 3.7447543144226074, 'learning_rate': 6.936456165053982e-06, 'epoch': 0.61}

 61%|██████    | 23200/37962 [53:44:45<33:50:58,  8.25s/it]
 61%|██████    | 23201/37962 [53:44:53<33:30:02,  8.17s/it]
 61%|██████    | 23202/37962 [53:45:01<33:41:00,  8.22s/it]
 61%|██████    | 23203/37962 [53:45:09<33:51:53,  8.26s/it]
 61%|██████    | 23204/37962 [53:45:18<34:05:57,  8.32s/it]
 61%|██████    | 23205/37962 [53:45:26<33:57:23,  8.28s/it]
 61%|██████    | 23206/37962 [53:45:34<33:51:19,  8.26s/it]
 61%|██████    | 23207/37962 [53:45:42<33:48:24,  8.25s/it]
 61%|██████    | 23208/37962 [53:45:50<33:07:31,  8.08s/it]
 61%|██████    | 23209/37962 [53:45:58<33:26:03,  8.16s/it]
 61%|██████    | 23210/37962 [53:46:07<33:39:21,  8.21s/it]
                                                           
{'loss': 1.2691, 'grad_norm': 10.101070404052734, 'learning_rate': 6.928335897256657e-06, 'epoch': 0.61}

 61%|██████    | 23210/37962 [53:46:07<33:39:21,  8.21s/it]
 61%|██████    | 23211/37962 [53:46:15<33:50:27,  8.26s/it]
 61%|██████    | 23212/37962 [53:46:23<33:47:37,  8.25s/it]
 61%|██████    | 23213/37962 [53:46:32<33:56:45,  8.29s/it]
 61%|██████    | 23214/37962 [53:46:40<34:01:15,  8.30s/it]
 61%|██████    | 23215/37962 [53:46:48<33:57:48,  8.29s/it]
 61%|██████    | 23216/37962 [53:46:56<33:31:31,  8.18s/it]
 61%|██████    | 23217/37962 [53:47:05<33:44:00,  8.24s/it]
 61%|██████    | 23218/37962 [53:47:13<33:42:27,  8.23s/it]
 61%|██████    | 23219/37962 [53:47:21<33:39:03,  8.22s/it]
 61%|██████    | 23220/37962 [53:47:29<33:52:25,  8.27s/it]
                                                           
{'loss': 1.3508, 'grad_norm': 6.772805213928223, 'learning_rate': 6.920217865270421e-06, 'epoch': 0.61}

 61%|██████    | 23220/37962 [53:47:29<33:52:25,  8.27s/it]
 61%|██████    | 23221/37962 [53:47:38<33:56:15,  8.29s/it]
 61%|██████    | 23222/37962 [53:47:46<33:58:52,  8.30s/it]
 61%|██████    | 23223/37962 [53:47:55<34:05:39,  8.33s/it]
 61%|██████    | 23224/37962 [53:48:03<34:04:37,  8.32s/it]
 61%|██████    | 23225/37962 [53:48:11<34:07:55,  8.34s/it]
 61%|██████    | 23226/37962 [53:48:20<34:06:07,  8.33s/it]
 61%|██████    | 23227/37962 [53:48:27<33:34:04,  8.20s/it]
 61%|██████    | 23228/37962 [53:48:36<33:45:48,  8.25s/it]
 61%|██████    | 23229/37962 [53:48:44<33:57:57,  8.30s/it]
 61%|██████    | 23230/37962 [53:48:53<34:14:11,  8.37s/it]
                                                           
{'loss': 1.2964, 'grad_norm': 7.834812641143799, 'learning_rate': 6.912102075004249e-06, 'epoch': 0.61}

 61%|██████    | 23230/37962 [53:48:53<34:14:11,  8.37s/it]
 61%|██████    | 23231/37962 [53:49:01<34:04:58,  8.33s/it]
 61%|██████    | 23232/37962 [53:49:09<34:14:01,  8.37s/it]
 61%|██████    | 23233/37962 [53:49:18<33:59:49,  8.31s/it]
 61%|██████    | 23234/37962 [53:49:26<34:12:15,  8.36s/it]
 61%|██████    | 23235/37962 [53:49:34<33:35:15,  8.21s/it]
 61%|██████    | 23236/37962 [53:49:42<33:41:34,  8.24s/it]
 61%|██████    | 23237/37962 [53:49:51<33:56:18,  8.30s/it]
 61%|██████    | 23238/37962 [53:49:59<33:45:19,  8.25s/it]
 61%|██████    | 23239/37962 [53:50:07<34:01:06,  8.32s/it]
 61%|██████    | 23240/37962 [53:50:16<33:55:06,  8.29s/it]
                                                           
{'loss': 1.2855, 'grad_norm': 7.67508602142334, 'learning_rate': 6.90398853236548e-06, 'epoch': 0.61}

 61%|██████    | 23240/37962 [53:50:16<33:55:06,  8.29s/it]
 61%|██████    | 23241/37962 [53:50:24<33:47:33,  8.26s/it]
 61%|██████    | 23242/37962 [53:50:32<34:09:36,  8.35s/it]
 61%|██████    | 23243/37962 [53:50:41<34:08:06,  8.35s/it]
 61%|██████    | 23244/37962 [53:50:49<33:53:39,  8.29s/it]
 61%|██████    | 23245/37962 [53:50:57<33:47:48,  8.27s/it]
 61%|██████    | 23246/37962 [53:51:05<33:52:59,  8.29s/it]
 61%|██████    | 23247/37962 [53:51:14<33:52:07,  8.29s/it]
 61%|██████    | 23248/37962 [53:51:22<33:53:14,  8.29s/it]
 61%|██████    | 23249/37962 [53:51:30<33:59:25,  8.32s/it]
 61%|██████    | 23250/37962 [53:51:38<33:28:14,  8.19s/it]
                                                           
{'loss': 1.2013, 'grad_norm': 11.748907089233398, 'learning_rate': 6.895877243259823e-06, 'epoch': 0.61}

 61%|██████    | 23250/37962 [53:51:38<33:28:14,  8.19s/it]
 61%|██████    | 23251/37962 [53:51:46<33:34:34,  8.22s/it]
 61%|██████▏   | 23252/37962 [53:51:55<33:27:43,  8.19s/it]
 61%|██████▏   | 23253/37962 [53:52:02<33:00:09,  8.08s/it]
 61%|██████▏   | 23254/37962 [53:52:11<33:39:21,  8.24s/it]
 61%|██████▏   | 23255/37962 [53:52:19<33:37:01,  8.23s/it]
 61%|██████▏   | 23256/37962 [53:52:27<33:25:04,  8.18s/it]
 61%|██████▏   | 23257/37962 [53:52:36<33:27:32,  8.19s/it]
 61%|██████▏   | 23258/37962 [53:52:44<33:28:21,  8.20s/it]
 61%|██████▏   | 23259/37962 [53:52:52<33:26:18,  8.19s/it]
 61%|██████▏   | 23260/37962 [53:53:00<33:40:23,  8.25s/it]
                                                           
{'loss': 1.307, 'grad_norm': 4.221116065979004, 'learning_rate': 6.887768213591346e-06, 'epoch': 0.61}

 61%|██████▏   | 23260/37962 [53:53:00<33:40:23,  8.25s/it]
 61%|██████▏   | 23261/37962 [53:53:09<33:43:55,  8.26s/it]
 61%|██████▏   | 23262/37962 [53:53:17<33:39:12,  8.24s/it]
 61%|██████▏   | 23263/37962 [53:53:25<33:52:31,  8.30s/it]
 61%|██████▏   | 23264/37962 [53:53:34<33:59:06,  8.32s/it]
 61%|██████▏   | 23265/37962 [53:53:42<34:00:04,  8.33s/it]
 61%|██████▏   | 23266/37962 [53:53:50<33:36:31,  8.23s/it]
 61%|██████▏   | 23267/37962 [53:53:58<33:18:05,  8.16s/it]
 61%|██████▏   | 23268/37962 [53:54:06<33:36:19,  8.23s/it]
 61%|██████▏   | 23269/37962 [53:54:14<33:20:55,  8.17s/it]
 61%|██████▏   | 23270/37962 [53:54:23<33:27:02,  8.20s/it]
                                                           
{'loss': 1.3123, 'grad_norm': 6.391451835632324, 'learning_rate': 6.879661449262469e-06, 'epoch': 0.61}

 61%|██████▏   | 23270/37962 [53:54:23<33:27:02,  8.20s/it]
 61%|██████▏   | 23271/37962 [53:54:31<33:23:24,  8.18s/it]
 61%|██████▏   | 23272/37962 [53:54:39<33:11:42,  8.13s/it]
 61%|██████▏   | 23273/37962 [53:54:47<33:22:26,  8.18s/it]
 61%|██████▏   | 23274/37962 [53:54:55<33:20:16,  8.17s/it]
 61%|██████▏   | 23275/37962 [53:55:04<33:30:03,  8.21s/it]
 61%|██████▏   | 23276/37962 [53:55:12<33:43:02,  8.27s/it]
 61%|██████▏   | 23277/37962 [53:55:20<33:44:46,  8.27s/it]
 61%|██████▏   | 23278/37962 [53:55:28<33:43:30,  8.27s/it]
 61%|██████▏   | 23279/37962 [53:55:37<33:54:10,  8.31s/it]
 61%|██████▏   | 23280/37962 [53:55:45<33:36:45,  8.24s/it]
                                                           
{'loss': 1.2691, 'grad_norm': 5.364327907562256, 'learning_rate': 6.871556956173973e-06, 'epoch': 0.61}

 61%|██████▏   | 23280/37962 [53:55:45<33:36:45,  8.24s/it]
 61%|██████▏   | 23281/37962 [53:55:53<33:46:24,  8.28s/it]
 61%|██████▏   | 23282/37962 [53:56:02<33:51:35,  8.30s/it]
 61%|██████▏   | 23283/37962 [53:56:10<33:56:23,  8.32s/it]
 61%|██████▏   | 23284/37962 [53:56:18<33:45:02,  8.28s/it]
 61%|██████▏   | 23285/37962 [53:56:27<33:44:42,  8.28s/it]
 61%|██████▏   | 23286/37962 [53:56:35<33:49:59,  8.30s/it]
 61%|██████▏   | 23287/37962 [53:56:43<33:52:27,  8.31s/it]
 61%|██████▏   | 23288/37962 [53:56:51<33:36:56,  8.25s/it]
 61%|██████▏   | 23289/37962 [53:56:59<33:18:26,  8.17s/it]
 61%|██████▏   | 23290/37962 [53:57:07<33:00:54,  8.10s/it]
                                                           
{'loss': 1.253, 'grad_norm': 10.242194175720215, 'learning_rate': 6.863454740224965e-06, 'epoch': 0.61}

 61%|██████▏   | 23290/37962 [53:57:07<33:00:54,  8.10s/it]
 61%|██████▏   | 23291/37962 [53:57:16<33:24:59,  8.20s/it]
 61%|██████▏   | 23292/37962 [53:57:24<33:40:52,  8.27s/it]
 61%|██████▏   | 23293/37962 [53:57:33<33:55:29,  8.33s/it]
 61%|██████▏   | 23294/37962 [53:57:41<33:50:40,  8.31s/it]
 61%|██████▏   | 23295/37962 [53:57:49<33:27:15,  8.21s/it]
 61%|██████▏   | 23296/37962 [53:57:57<33:19:28,  8.18s/it]
 61%|██████▏   | 23297/37962 [53:58:05<33:35:28,  8.25s/it]
 61%|██████▏   | 23298/37962 [53:58:14<33:36:34,  8.25s/it]
 61%|██████▏   | 23299/37962 [53:58:22<33:36:46,  8.25s/it]
 61%|██████▏   | 23300/37962 [53:58:30<33:43:33,  8.28s/it]
                                                           
{'loss': 1.3808, 'grad_norm': 8.30744457244873, 'learning_rate': 6.855354807312917e-06, 'epoch': 0.61}

 61%|██████▏   | 23300/37962 [53:58:30<33:43:33,  8.28s/it]
 61%|██████▏   | 23301/37962 [53:58:38<33:40:17,  8.27s/it]
 61%|██████▏   | 23302/37962 [53:58:47<33:42:11,  8.28s/it]
 61%|██████▏   | 23303/37962 [53:58:55<33:44:49,  8.29s/it]
 61%|██████▏   | 23304/37962 [53:59:03<33:42:49,  8.28s/it]
 61%|██████▏   | 23305/37962 [53:59:11<33:25:06,  8.21s/it]
 61%|██████▏   | 23306/37962 [53:59:19<33:03:46,  8.12s/it]
 61%|██████▏   | 23307/37962 [53:59:28<33:16:08,  8.17s/it]
 61%|██████▏   | 23308/37962 [53:59:36<33:29:06,  8.23s/it]
 61%|██████▏   | 23309/37962 [53:59:44<33:18:35,  8.18s/it]
 61%|██████▏   | 23310/37962 [53:59:52<33:41:39,  8.28s/it]
                                                           
{'loss': 1.2559, 'grad_norm': 4.349042892456055, 'learning_rate': 6.847257163333628e-06, 'epoch': 0.61}

 61%|██████▏   | 23310/37962 [53:59:52<33:41:39,  8.28s/it]
 61%|██████▏   | 23311/37962 [54:00:01<33:47:49,  8.30s/it]
 61%|██████▏   | 23312/37962 [54:00:09<33:46:56,  8.30s/it]
 61%|██████▏   | 23313/37962 [54:00:18<33:57:42,  8.35s/it]
 61%|██████▏   | 23314/37962 [54:00:26<34:10:01,  8.40s/it]
 61%|██████▏   | 23315/37962 [54:00:34<33:59:42,  8.36s/it]
 61%|██████▏   | 23316/37962 [54:00:43<33:52:55,  8.33s/it]
 61%|██████▏   | 23317/37962 [54:00:51<33:45:50,  8.30s/it]
 61%|██████▏   | 23318/37962 [54:00:59<33:52:23,  8.33s/it]
 61%|██████▏   | 23319/37962 [54:01:08<33:51:09,  8.32s/it]
 61%|██████▏   | 23320/37962 [54:01:16<33:58:10,  8.35s/it]
                                                           
{'loss': 1.2363, 'grad_norm': 3.6985018253326416, 'learning_rate': 6.839161814181232e-06, 'epoch': 0.61}

 61%|██████▏   | 23320/37962 [54:01:16<33:58:10,  8.35s/it]
 61%|██████▏   | 23321/37962 [54:01:24<33:49:49,  8.32s/it]
 61%|██████▏   | 23322/37962 [54:01:33<33:48:17,  8.31s/it]
 61%|██████▏   | 23323/37962 [54:01:41<34:03:34,  8.38s/it]
 61%|██████▏   | 23324/37962 [54:01:49<33:58:16,  8.35s/it]
 61%|██████▏   | 23325/37962 [54:01:58<33:51:38,  8.33s/it]
 61%|██████▏   | 23326/37962 [54:02:06<34:12:19,  8.41s/it]
 61%|██████▏   | 23327/37962 [54:02:15<34:08:31,  8.40s/it]
 61%|██████▏   | 23328/37962 [54:02:23<33:58:09,  8.36s/it]
 61%|██████▏   | 23329/37962 [54:02:31<33:51:45,  8.33s/it]
 61%|██████▏   | 23330/37962 [54:02:39<33:50:44,  8.33s/it]
                                                           
{'loss': 1.3495, 'grad_norm': 6.641138076782227, 'learning_rate': 6.831068765748194e-06, 'epoch': 0.61}

 61%|██████▏   | 23330/37962 [54:02:39<33:50:44,  8.33s/it]
 61%|██████▏   | 23331/37962 [54:02:48<33:47:33,  8.31s/it]
 61%|██████▏   | 23332/37962 [54:02:56<33:37:10,  8.27s/it]
 61%|██████▏   | 23333/37962 [54:03:04<33:40:35,  8.29s/it]
 61%|██████▏   | 23334/37962 [54:03:12<33:37:42,  8.28s/it]
 61%|██████▏   | 23335/37962 [54:03:21<33:37:00,  8.27s/it]
 61%|██████▏   | 23336/37962 [54:03:29<33:10:44,  8.17s/it]
 61%|██████▏   | 23337/37962 [54:03:37<33:18:40,  8.20s/it]
 61%|██████▏   | 23338/37962 [54:03:45<33:22:14,  8.21s/it]
 61%|██████▏   | 23339/37962 [54:03:53<33:21:18,  8.21s/it]
 61%|██████▏   | 23340/37962 [54:04:02<33:27:05,  8.24s/it]
                                                           
{'loss': 1.3818, 'grad_norm': 7.832810878753662, 'learning_rate': 6.822978023925305e-06, 'epoch': 0.61}

 61%|██████▏   | 23340/37962 [54:04:02<33:27:05,  8.24s/it]
 61%|██████▏   | 23341/37962 [54:04:10<33:30:59,  8.25s/it]
 61%|██████▏   | 23342/37962 [54:04:18<33:15:00,  8.19s/it]
 61%|██████▏   | 23343/37962 [54:04:26<33:18:52,  8.20s/it]
 61%|██████▏   | 23344/37962 [54:04:35<33:24:42,  8.23s/it]
 61%|██████▏   | 23345/37962 [54:04:43<33:25:20,  8.23s/it]
 61%|██████▏   | 23346/37962 [54:04:51<33:43:51,  8.31s/it]
 62%|██████▏   | 23347/37962 [54:05:00<33:52:10,  8.34s/it]
 62%|██████▏   | 23348/37962 [54:05:08<34:06:00,  8.40s/it]
 62%|██████▏   | 23349/37962 [54:05:16<33:56:06,  8.36s/it]
 62%|██████▏   | 23350/37962 [54:05:25<33:51:38,  8.34s/it]
                                                           
{'loss': 1.3798, 'grad_norm': 7.324801445007324, 'learning_rate': 6.814889594601673e-06, 'epoch': 0.62}

 62%|██████▏   | 23350/37962 [54:05:25<33:51:38,  8.34s/it]
 62%|██████▏   | 23351/37962 [54:05:33<33:54:38,  8.36s/it]
 62%|██████▏   | 23352/37962 [54:05:41<33:49:04,  8.33s/it]
 62%|██████▏   | 23353/37962 [54:05:50<33:34:39,  8.27s/it]
 62%|██████▏   | 23354/37962 [54:05:58<33:31:32,  8.26s/it]
 62%|██████▏   | 23355/37962 [54:06:06<33:35:10,  8.28s/it]
 62%|██████▏   | 23356/37962 [54:06:14<33:39:02,  8.29s/it]
 62%|██████▏   | 23357/37962 [54:06:23<33:36:42,  8.29s/it]
 62%|██████▏   | 23358/37962 [54:06:31<33:35:18,  8.28s/it]
 62%|██████▏   | 23359/37962 [54:06:39<33:52:11,  8.35s/it]
 62%|██████▏   | 23360/37962 [54:06:48<33:48:00,  8.33s/it]
                                                           
{'loss': 1.3829, 'grad_norm': 17.30986976623535, 'learning_rate': 6.806803483664726e-06, 'epoch': 0.62}

 62%|██████▏   | 23360/37962 [54:06:48<33:48:00,  8.33s/it]
 62%|██████▏   | 23361/37962 [54:06:56<33:36:27,  8.29s/it]
 62%|██████▏   | 23362/37962 [54:07:04<33:32:38,  8.27s/it]
 62%|██████▏   | 23363/37962 [54:07:20<42:45:32, 10.54s/it]
 62%|██████▏   | 23364/37962 [54:07:28<39:52:48,  9.83s/it]
 62%|██████▏   | 23365/37962 [54:07:36<37:35:43,  9.27s/it]
 62%|██████▏   | 23366/37962 [54:07:45<36:26:54,  8.99s/it]
 62%|██████▏   | 23367/37962 [54:07:53<35:19:29,  8.71s/it]
 62%|██████▏   | 23368/37962 [54:08:01<34:50:02,  8.59s/it]
 62%|██████▏   | 23369/37962 [54:08:09<34:24:34,  8.49s/it]
 62%|██████▏   | 23370/37962 [54:08:17<34:03:53,  8.40s/it]
                                                           
{'loss': 1.3172, 'grad_norm': 5.366345405578613, 'learning_rate': 6.798719697000202e-06, 'epoch': 0.62}

 62%|██████▏   | 23370/37962 [54:08:17<34:03:53,  8.40s/it]
 62%|██████▏   | 23371/37962 [54:08:25<33:36:10,  8.29s/it]
 62%|██████▏   | 23372/37962 [54:08:34<33:59:43,  8.39s/it]
 62%|██████▏   | 23373/37962 [54:08:42<33:47:13,  8.34s/it]
 62%|██████▏   | 23374/37962 [54:08:50<33:37:06,  8.30s/it]
 62%|██████▏   | 23375/37962 [54:08:58<33:02:44,  8.16s/it]
 62%|██████▏   | 23376/37962 [54:09:07<33:35:35,  8.29s/it]
 62%|██████▏   | 23377/37962 [54:09:15<33:19:05,  8.22s/it]
 62%|██████▏   | 23378/37962 [54:09:23<33:09:58,  8.19s/it]
 62%|██████▏   | 23379/37962 [54:09:31<33:06:07,  8.17s/it]
 62%|██████▏   | 23380/37962 [54:09:39<33:08:25,  8.18s/it]
                                                           
{'loss': 1.2709, 'grad_norm': 8.090523719787598, 'learning_rate': 6.7906382404921554e-06, 'epoch': 0.62}

 62%|██████▏   | 23380/37962 [54:09:39<33:08:25,  8.18s/it]
 62%|██████▏   | 23381/37962 [54:09:48<33:24:08,  8.25s/it]
 62%|██████▏   | 23382/37962 [54:09:56<33:23:37,  8.25s/it]
 62%|██████▏   | 23383/37962 [54:10:04<32:53:19,  8.12s/it]
 62%|██████▏   | 23384/37962 [54:10:12<33:03:09,  8.16s/it]
 62%|██████▏   | 23385/37962 [54:10:21<33:23:40,  8.25s/it]
 62%|██████▏   | 23386/37962 [54:10:29<33:34:22,  8.29s/it]
 62%|██████▏   | 23387/37962 [54:10:37<33:31:27,  8.28s/it]
 62%|██████▏   | 23388/37962 [54:10:45<33:20:13,  8.23s/it]
 62%|██████▏   | 23389/37962 [54:10:54<33:27:29,  8.27s/it]
 62%|██████▏   | 23390/37962 [54:11:02<33:29:51,  8.28s/it]
                                                           
{'loss': 1.2771, 'grad_norm': 9.028170585632324, 'learning_rate': 6.78255912002294e-06, 'epoch': 0.62}

 62%|██████▏   | 23390/37962 [54:11:02<33:29:51,  8.28s/it]
 62%|██████▏   | 23391/37962 [54:11:10<33:34:29,  8.30s/it]
 62%|██████▏   | 23392/37962 [54:11:19<33:40:45,  8.32s/it]
 62%|██████▏   | 23393/37962 [54:11:27<33:22:13,  8.25s/it]
 62%|██████▏   | 23394/37962 [54:11:35<33:24:41,  8.26s/it]
 62%|██████▏   | 23395/37962 [54:11:43<33:26:50,  8.27s/it]
 62%|██████▏   | 23396/37962 [54:11:51<33:08:00,  8.19s/it]
 62%|██████▏   | 23397/37962 [54:11:59<32:56:12,  8.14s/it]
 62%|██████▏   | 23398/37962 [54:12:07<32:50:36,  8.12s/it]
 62%|██████▏   | 23399/37962 [54:12:16<32:55:39,  8.14s/it]
 62%|██████▏   | 23400/37962 [54:12:24<33:09:40,  8.20s/it]
                                                           
{'loss': 1.2674, 'grad_norm': 12.425600051879883, 'learning_rate': 6.774482341473196e-06, 'epoch': 0.62}

 62%|██████▏   | 23400/37962 [54:12:24<33:09:40,  8.20s/it]
 62%|██████▏   | 23401/37962 [54:12:32<33:25:30,  8.26s/it]
 62%|██████▏   | 23402/37962 [54:12:41<33:25:49,  8.27s/it]
 62%|██████▏   | 23403/37962 [54:12:49<33:32:03,  8.29s/it]
 62%|██████▏   | 23404/37962 [54:12:57<33:31:57,  8.29s/it]
 62%|██████▏   | 23405/37962 [54:13:06<33:46:16,  8.35s/it]
 62%|██████▏   | 23406/37962 [54:13:14<33:45:24,  8.35s/it]
 62%|██████▏   | 23407/37962 [54:13:23<33:50:55,  8.37s/it]
 62%|██████▏   | 23408/37962 [54:13:31<33:40:21,  8.33s/it]
 62%|██████▏   | 23409/37962 [54:13:39<33:46:34,  8.36s/it]
 62%|██████▏   | 23410/37962 [54:13:47<33:28:10,  8.28s/it]
                                                           
{'loss': 1.2337, 'grad_norm': 11.2600679397583, 'learning_rate': 6.76640791072188e-06, 'epoch': 0.62}

 62%|██████▏   | 23410/37962 [54:13:47<33:28:10,  8.28s/it]
 62%|██████▏   | 23411/37962 [54:13:55<33:23:04,  8.26s/it]
 62%|██████▏   | 23412/37962 [54:14:04<33:40:53,  8.33s/it]
 62%|██████▏   | 23413/37962 [54:14:12<33:32:47,  8.30s/it]
 62%|██████▏   | 23414/37962 [54:14:21<33:37:36,  8.32s/it]
 62%|██████▏   | 23415/37962 [54:14:29<33:38:47,  8.33s/it]
 62%|██████▏   | 23416/37962 [54:14:37<33:52:04,  8.38s/it]
 62%|██████▏   | 23417/37962 [54:14:46<33:53:51,  8.39s/it]
 62%|██████▏   | 23418/37962 [54:14:54<33:44:41,  8.35s/it]
 62%|██████▏   | 23419/37962 [54:15:02<33:40:43,  8.34s/it]
 62%|██████▏   | 23420/37962 [54:15:11<33:42:16,  8.34s/it]
                                                           
{'loss': 1.2993, 'grad_norm': 9.535510063171387, 'learning_rate': 6.758335833646226e-06, 'epoch': 0.62}

 62%|██████▏   | 23420/37962 [54:15:11<33:42:16,  8.34s/it]
 62%|██████▏   | 23421/37962 [54:15:19<33:32:30,  8.30s/it]
 62%|██████▏   | 23422/37962 [54:15:27<33:38:32,  8.33s/it]
 62%|██████▏   | 23423/37962 [54:15:36<33:45:48,  8.36s/it]
 62%|██████▏   | 23424/37962 [54:15:44<33:36:28,  8.32s/it]
 62%|██████▏   | 23425/37962 [54:15:52<33:29:31,  8.29s/it]
 62%|██████▏   | 23426/37962 [54:16:00<33:00:49,  8.18s/it]
 62%|██████▏   | 23427/37962 [54:16:08<33:09:58,  8.21s/it]
 62%|██████▏   | 23428/37962 [54:16:17<33:02:19,  8.18s/it]
 62%|██████▏   | 23429/37962 [54:16:24<32:34:57,  8.07s/it]
 62%|██████▏   | 23430/37962 [54:16:33<32:43:16,  8.11s/it]
                                                           
{'loss': 1.34, 'grad_norm': 11.369451522827148, 'learning_rate': 6.750266116121762e-06, 'epoch': 0.62}

 62%|██████▏   | 23430/37962 [54:16:33<32:43:16,  8.11s/it]
 62%|██████▏   | 23431/37962 [54:16:41<33:05:20,  8.20s/it]
 62%|██████▏   | 23432/37962 [54:16:49<33:06:42,  8.20s/it]
 62%|██████▏   | 23433/37962 [54:16:58<33:24:47,  8.28s/it]
 62%|██████▏   | 23434/37962 [54:17:06<33:47:58,  8.38s/it]
 62%|██████▏   | 23435/37962 [54:17:15<33:43:44,  8.36s/it]
 62%|██████▏   | 23436/37962 [54:17:23<33:30:52,  8.31s/it]
 62%|██████▏   | 23437/37962 [54:17:31<33:39:10,  8.34s/it]
 62%|██████▏   | 23438/37962 [54:17:39<33:18:24,  8.26s/it]
 62%|██████▏   | 23439/37962 [54:17:47<33:14:29,  8.24s/it]
 62%|██████▏   | 23440/37962 [54:17:56<33:25:47,  8.29s/it]
                                                           
{'loss': 1.2988, 'grad_norm': 11.331439018249512, 'learning_rate': 6.742198764022296e-06, 'epoch': 0.62}

 62%|██████▏   | 23440/37962 [54:17:56<33:25:47,  8.29s/it]
 62%|██████▏   | 23441/37962 [54:18:04<33:28:29,  8.30s/it]
 62%|██████▏   | 23442/37962 [54:18:13<33:38:49,  8.34s/it]
 62%|██████▏   | 23443/37962 [54:18:21<33:37:17,  8.34s/it]
 62%|██████▏   | 23444/37962 [54:18:30<33:58:52,  8.43s/it]
 62%|██████▏   | 23445/37962 [54:18:38<33:45:47,  8.37s/it]
 62%|██████▏   | 23446/37962 [54:18:46<33:35:45,  8.33s/it]
 62%|██████▏   | 23447/37962 [54:18:54<33:38:06,  8.34s/it]
 62%|██████▏   | 23448/37962 [54:19:03<33:41:30,  8.36s/it]
 62%|██████▏   | 23449/37962 [54:19:11<33:20:56,  8.27s/it]
 62%|██████▏   | 23450/37962 [54:19:19<33:18:42,  8.26s/it]
                                                           
{'loss': 1.2382, 'grad_norm': 10.294694900512695, 'learning_rate': 6.734133783219909e-06, 'epoch': 0.62}

 62%|██████▏   | 23450/37962 [54:19:19<33:18:42,  8.26s/it]
 62%|██████▏   | 23451/37962 [54:19:27<33:14:59,  8.25s/it]
 62%|██████▏   | 23452/37962 [54:19:36<33:42:02,  8.36s/it]
 62%|██████▏   | 23453/37962 [54:19:44<33:30:26,  8.31s/it]
 62%|██████▏   | 23454/37962 [54:19:52<33:01:39,  8.20s/it]
 62%|██████▏   | 23455/37962 [54:20:00<33:10:28,  8.23s/it]
 62%|██████▏   | 23456/37962 [54:20:09<33:22:51,  8.28s/it]
 62%|██████▏   | 23457/37962 [54:20:17<33:19:53,  8.27s/it]
 62%|██████▏   | 23458/37962 [54:20:25<33:21:37,  8.28s/it]
 62%|██████▏   | 23459/37962 [54:20:33<33:08:27,  8.23s/it]
 62%|██████▏   | 23460/37962 [54:20:42<33:13:37,  8.25s/it]
                                                           
{'loss': 1.3111, 'grad_norm': 6.10054874420166, 'learning_rate': 6.726071179584966e-06, 'epoch': 0.62}

 62%|██████▏   | 23460/37962 [54:20:42<33:13:37,  8.25s/it]
 62%|██████▏   | 23461/37962 [54:20:50<33:18:00,  8.27s/it]
 62%|██████▏   | 23462/37962 [54:20:58<33:01:04,  8.20s/it]
 62%|██████▏   | 23463/37962 [54:21:06<33:08:30,  8.23s/it]
 62%|██████▏   | 23464/37962 [54:21:15<33:21:10,  8.28s/it]
 62%|██████▏   | 23465/37962 [54:21:23<33:30:33,  8.32s/it]
 62%|██████▏   | 23466/37962 [54:21:32<33:41:08,  8.37s/it]
 62%|██████▏   | 23467/37962 [54:21:40<33:23:00,  8.29s/it]
 62%|██████▏   | 23468/37962 [54:21:48<33:22:28,  8.29s/it]
 62%|██████▏   | 23469/37962 [54:21:56<33:20:50,  8.28s/it]
 62%|██████▏   | 23470/37962 [54:22:05<33:28:06,  8.31s/it]
                                                           
{'loss': 1.3063, 'grad_norm': 6.449000358581543, 'learning_rate': 6.718010958986091e-06, 'epoch': 0.62}

 62%|██████▏   | 23470/37962 [54:22:05<33:28:06,  8.31s/it]
 62%|██████▏   | 23471/37962 [54:22:13<32:59:35,  8.20s/it]
 62%|██████▏   | 23472/37962 [54:22:21<32:56:30,  8.18s/it]
 62%|██████▏   | 23473/37962 [54:22:29<32:50:21,  8.16s/it]
 62%|██████▏   | 23474/37962 [54:22:37<33:12:05,  8.25s/it]
 62%|██████▏   | 23475/37962 [54:22:46<33:10:27,  8.24s/it]
 62%|██████▏   | 23476/37962 [54:22:54<33:14:43,  8.26s/it]
 62%|██████▏   | 23477/37962 [54:23:02<33:02:21,  8.21s/it]
 62%|██████▏   | 23478/37962 [54:23:10<33:16:00,  8.27s/it]
 62%|██████▏   | 23479/37962 [54:23:18<32:54:38,  8.18s/it]
 62%|██████▏   | 23480/37962 [54:23:27<33:06:38,  8.23s/it]
                                                           
{'loss': 1.1899, 'grad_norm': 8.075651168823242, 'learning_rate': 6.709953127290185e-06, 'epoch': 0.62}

 62%|██████▏   | 23480/37962 [54:23:27<33:06:38,  8.23s/it]
 62%|██████▏   | 23481/37962 [54:23:35<33:27:26,  8.32s/it]
 62%|██████▏   | 23482/37962 [54:23:44<33:30:45,  8.33s/it]
 62%|██████▏   | 23483/37962 [54:23:52<33:24:08,  8.31s/it]
 62%|██████▏   | 23484/37962 [54:24:00<33:14:03,  8.26s/it]
 62%|██████▏   | 23485/37962 [54:24:08<32:54:40,  8.18s/it]
 62%|██████▏   | 23486/37962 [54:24:16<33:12:45,  8.26s/it]
 62%|██████▏   | 23487/37962 [54:24:24<32:43:21,  8.14s/it]
 62%|██████▏   | 23488/37962 [54:24:33<32:54:37,  8.19s/it]
 62%|██████▏   | 23489/37962 [54:24:40<32:30:35,  8.09s/it]
 62%|██████▏   | 23490/37962 [54:24:49<32:50:52,  8.17s/it]
                                                           
{'loss': 1.3529, 'grad_norm': 6.170894622802734, 'learning_rate': 6.701897690362405e-06, 'epoch': 0.62}

 62%|██████▏   | 23490/37962 [54:24:49<32:50:52,  8.17s/it]
 62%|██████▏   | 23491/37962 [54:24:57<33:03:36,  8.22s/it]
 62%|██████▏   | 23492/37962 [54:25:06<33:08:35,  8.25s/it]
 62%|██████▏   | 23493/37962 [54:25:13<32:41:42,  8.13s/it]
 62%|██████▏   | 23494/37962 [54:25:22<32:53:06,  8.18s/it]
 62%|██████▏   | 23495/37962 [54:25:30<33:08:51,  8.25s/it]
 62%|██████▏   | 23496/37962 [54:25:38<33:17:45,  8.29s/it]
 62%|██████▏   | 23497/37962 [54:25:47<33:09:30,  8.25s/it]
 62%|██████▏   | 23498/37962 [54:25:55<33:18:51,  8.29s/it]
 62%|██████▏   | 23499/37962 [54:26:04<33:38:28,  8.37s/it]
 62%|██████▏   | 23500/37962 [54:26:12<33:25:27,  8.32s/it]
                                                           
{'loss': 1.2523, 'grad_norm': 5.929863929748535, 'learning_rate': 6.693844654066155e-06, 'epoch': 0.62}

 62%|██████▏   | 23500/37962 [54:26:12<33:25:27,  8.32s/it]
 62%|██████▏   | 23501/37962 [54:26:20<33:21:56,  8.31s/it]
 62%|██████▏   | 23502/37962 [54:26:28<33:16:05,  8.28s/it]
 62%|██████▏   | 23503/37962 [54:26:37<33:31:02,  8.35s/it]
 62%|██████▏   | 23504/37962 [54:26:45<33:41:39,  8.39s/it]
 62%|██████▏   | 23505/37962 [54:26:54<33:35:29,  8.36s/it]
 62%|██████▏   | 23506/37962 [54:27:02<33:25:52,  8.33s/it]
 62%|██████▏   | 23507/37962 [54:27:10<33:22:34,  8.31s/it]
 62%|██████▏   | 23508/37962 [54:27:19<33:34:20,  8.36s/it]
 62%|██████▏   | 23509/37962 [54:27:27<33:28:13,  8.34s/it]
 62%|██████▏   | 23510/37962 [54:27:35<33:28:01,  8.34s/it]
                                                           
{'loss': 1.2907, 'grad_norm': 7.320000648498535, 'learning_rate': 6.685794024263108e-06, 'epoch': 0.62}

 62%|██████▏   | 23510/37962 [54:27:35<33:28:01,  8.34s/it]
 62%|██████▏   | 23511/37962 [54:27:44<33:39:07,  8.38s/it]
 62%|██████▏   | 23512/37962 [54:27:52<33:34:54,  8.37s/it]
 62%|██████▏   | 23513/37962 [54:28:00<33:18:49,  8.30s/it]
 62%|██████▏   | 23514/37962 [54:28:08<32:58:03,  8.21s/it]
 62%|██████▏   | 23515/37962 [54:28:17<33:14:42,  8.28s/it]
 62%|██████▏   | 23516/37962 [54:28:25<33:32:26,  8.36s/it]
 62%|██████▏   | 23517/37962 [54:28:34<33:36:58,  8.38s/it]
 62%|██████▏   | 23518/37962 [54:28:42<33:32:38,  8.36s/it]
 62%|██████▏   | 23519/37962 [54:28:50<33:34:49,  8.37s/it]
 62%|██████▏   | 23520/37962 [54:28:59<33:31:54,  8.36s/it]
                                                           
{'loss': 1.3619, 'grad_norm': 9.73523998260498, 'learning_rate': 6.677745806813176e-06, 'epoch': 0.62}

 62%|██████▏   | 23520/37962 [54:28:59<33:31:54,  8.36s/it]
 62%|██████▏   | 23521/37962 [54:29:07<33:15:07,  8.29s/it]
 62%|██████▏   | 23522/37962 [54:29:15<33:25:02,  8.33s/it]
 62%|██████▏   | 23523/37962 [54:29:23<33:16:48,  8.30s/it]
 62%|██████▏   | 23524/37962 [54:29:31<33:04:32,  8.25s/it]
 62%|██████▏   | 23525/37962 [54:29:39<32:39:47,  8.14s/it]
 62%|██████▏   | 23526/37962 [54:29:48<32:43:55,  8.16s/it]
 62%|██████▏   | 23527/37962 [54:29:56<32:37:16,  8.14s/it]
 62%|██████▏   | 23528/37962 [54:30:04<32:55:59,  8.21s/it]
 62%|██████▏   | 23529/37962 [54:30:12<32:54:23,  8.21s/it]
 62%|██████▏   | 23530/37962 [54:30:20<32:53:29,  8.20s/it]
                                                           
{'loss': 1.2824, 'grad_norm': 4.73565673828125, 'learning_rate': 6.669700007574517e-06, 'epoch': 0.62}

 62%|██████▏   | 23530/37962 [54:30:20<32:53:29,  8.20s/it]
 62%|██████▏   | 23531/37962 [54:30:29<32:52:50,  8.20s/it]
 62%|██████▏   | 23532/37962 [54:30:37<32:56:32,  8.22s/it]
 62%|██████▏   | 23533/37962 [54:30:45<33:00:15,  8.23s/it]
 62%|██████▏   | 23534/37962 [54:30:53<32:50:19,  8.19s/it]
 62%|██████▏   | 23535/37962 [54:31:02<32:55:52,  8.22s/it]
 62%|██████▏   | 23536/37962 [54:31:10<32:58:37,  8.23s/it]
 62%|██████▏   | 23537/37962 [54:31:18<33:17:58,  8.31s/it]
 62%|██████▏   | 23538/37962 [54:31:27<33:14:11,  8.30s/it]
 62%|██████▏   | 23539/37962 [54:31:35<32:57:25,  8.23s/it]
 62%|██████▏   | 23540/37962 [54:31:43<32:45:38,  8.18s/it]
                                                           
{'loss': 1.1931, 'grad_norm': 11.902194023132324, 'learning_rate': 6.6616566324035285e-06, 'epoch': 0.62}

 62%|██████▏   | 23540/37962 [54:31:43<32:45:38,  8.18s/it]
 62%|██████▏   | 23541/37962 [54:31:51<33:00:59,  8.24s/it]
 62%|██████▏   | 23542/37962 [54:31:59<32:45:43,  8.18s/it]
 62%|██████▏   | 23543/37962 [54:32:07<32:52:40,  8.21s/it]
 62%|██████▏   | 23544/37962 [54:32:16<32:57:16,  8.23s/it]
 62%|██████▏   | 23545/37962 [54:32:24<32:59:39,  8.24s/it]
 62%|██████▏   | 23546/37962 [54:32:32<33:05:27,  8.26s/it]
 62%|██████▏   | 23547/37962 [54:32:40<33:02:35,  8.25s/it]
 62%|██████▏   | 23548/37962 [54:32:49<32:53:03,  8.21s/it]
 62%|██████▏   | 23549/37962 [54:32:57<32:58:26,  8.24s/it]
 62%|██████▏   | 23550/37962 [54:33:05<32:56:37,  8.23s/it]
                                                           
{'loss': 1.3449, 'grad_norm': 9.387982368469238, 'learning_rate': 6.653615687154846e-06, 'epoch': 0.62}

 62%|██████▏   | 23550/37962 [54:33:05<32:56:37,  8.23s/it]
 62%|██████▏   | 23551/37962 [54:33:13<32:26:12,  8.10s/it]
 62%|██████▏   | 23552/37962 [54:33:21<32:37:14,  8.15s/it]
 62%|██████▏   | 23553/37962 [54:33:29<32:36:20,  8.15s/it]
 62%|██████▏   | 23554/37962 [54:33:38<32:59:39,  8.24s/it]
 62%|██████▏   | 23555/37962 [54:33:46<32:44:10,  8.18s/it]
 62%|██████▏   | 23556/37962 [54:33:54<32:41:26,  8.17s/it]
 62%|██████▏   | 23557/37962 [54:34:02<32:40:06,  8.16s/it]
 62%|██████▏   | 23558/37962 [54:34:11<32:56:08,  8.23s/it]
 62%|██████▏   | 23559/37962 [54:34:27<42:16:51, 10.57s/it]
 62%|██████▏   | 23560/37962 [54:34:35<39:53:48,  9.97s/it]
                                                           
{'loss': 1.2027, 'grad_norm': 10.049251556396484, 'learning_rate': 6.645577177681333e-06, 'epoch': 0.62}

 62%|██████▏   | 23560/37962 [54:34:35<39:53:48,  9.97s/it]
 62%|██████▏   | 23561/37962 [54:34:44<38:03:24,  9.51s/it]
 62%|██████▏   | 23562/37962 [54:34:52<36:49:59,  9.21s/it]
 62%|██████▏   | 23563/37962 [54:35:00<35:45:48,  8.94s/it]
 62%|██████▏   | 23564/37962 [54:35:09<34:56:22,  8.74s/it]
 62%|██████▏   | 23565/37962 [54:35:17<34:18:24,  8.58s/it]
 62%|██████▏   | 23566/37962 [54:35:25<34:09:20,  8.54s/it]
 62%|██████▏   | 23567/37962 [54:35:33<33:31:58,  8.39s/it]
 62%|██████▏   | 23568/37962 [54:35:41<33:05:56,  8.28s/it]
 62%|██████▏   | 23569/37962 [54:35:50<33:03:10,  8.27s/it]
 62%|██████▏   | 23570/37962 [54:35:58<33:06:45,  8.28s/it]
                                                           
{'loss': 1.3259, 'grad_norm': 5.822140693664551, 'learning_rate': 6.637541109834077e-06, 'epoch': 0.62}

 62%|██████▏   | 23570/37962 [54:35:58<33:06:45,  8.28s/it]
 62%|██████▏   | 23571/37962 [54:36:06<32:51:58,  8.22s/it]
 62%|██████▏   | 23572/37962 [54:36:14<33:10:19,  8.30s/it]
 62%|██████▏   | 23573/37962 [54:36:23<33:15:25,  8.32s/it]
 62%|██████▏   | 23574/37962 [54:36:31<33:19:32,  8.34s/it]
 62%|██████▏   | 23575/37962 [54:36:39<33:05:02,  8.28s/it]
 62%|██████▏   | 23576/37962 [54:36:48<33:09:35,  8.30s/it]
 62%|██████▏   | 23577/37962 [54:36:56<33:03:12,  8.27s/it]
 62%|██████▏   | 23578/37962 [54:37:04<32:48:07,  8.21s/it]
 62%|██████▏   | 23579/37962 [54:37:12<32:40:38,  8.18s/it]
 62%|██████▏   | 23580/37962 [54:37:20<32:38:34,  8.17s/it]
                                                           
{'loss': 1.2612, 'grad_norm': 9.476832389831543, 'learning_rate': 6.629507489462399e-06, 'epoch': 0.62}

 62%|██████▏   | 23580/37962 [54:37:20<32:38:34,  8.17s/it]
 62%|██████▏   | 23581/37962 [54:37:28<32:42:03,  8.19s/it]
 62%|██████▏   | 23582/37962 [54:37:36<32:31:06,  8.14s/it]
 62%|██████▏   | 23583/37962 [54:37:45<32:58:30,  8.26s/it]
 62%|██████▏   | 23584/37962 [54:37:53<32:55:18,  8.24s/it]
 62%|██████▏   | 23585/37962 [54:38:01<32:53:48,  8.24s/it]
 62%|██████▏   | 23586/37962 [54:38:10<32:54:01,  8.24s/it]
 62%|██████▏   | 23587/37962 [54:38:18<32:57:57,  8.26s/it]
 62%|██████▏   | 23588/37962 [54:38:26<33:09:34,  8.30s/it]
 62%|██████▏   | 23589/37962 [54:38:35<33:11:48,  8.31s/it]
 62%|██████▏   | 23590/37962 [54:38:43<33:09:41,  8.31s/it]
                                                           
{'loss': 1.1804, 'grad_norm': 10.403910636901855, 'learning_rate': 6.62147632241383e-06, 'epoch': 0.62}

 62%|██████▏   | 23590/37962 [54:38:43<33:09:41,  8.31s/it]
 62%|██████▏   | 23591/37962 [54:38:51<33:19:49,  8.35s/it]
 62%|██████▏   | 23592/37962 [54:39:00<33:14:29,  8.33s/it]
 62%|██████▏   | 23593/37962 [54:39:08<33:05:00,  8.29s/it]
 62%|██████▏   | 23594/37962 [54:39:16<32:58:52,  8.26s/it]
 62%|██████▏   | 23595/37962 [54:39:24<32:56:25,  8.25s/it]
 62%|██████▏   | 23596/37962 [54:39:33<32:53:34,  8.24s/it]
 62%|██████▏   | 23597/37962 [54:39:41<32:59:22,  8.27s/it]
 62%|██████▏   | 23598/37962 [54:39:49<32:55:22,  8.25s/it]
 62%|██████▏   | 23599/37962 [54:39:57<32:55:11,  8.25s/it]
 62%|██████▏   | 23600/37962 [54:40:05<32:38:17,  8.18s/it]
                                                           
{'loss': 1.2668, 'grad_norm': 9.095616340637207, 'learning_rate': 6.613447614534119e-06, 'epoch': 0.62}

 62%|██████▏   | 23600/37962 [54:40:05<32:38:17,  8.18s/it]
 62%|██████▏   | 23601/37962 [54:40:14<32:50:13,  8.23s/it]
 62%|██████▏   | 23602/37962 [54:40:22<32:47:29,  8.22s/it]
 62%|██████▏   | 23603/37962 [54:40:30<32:46:48,  8.22s/it]
 62%|██████▏   | 23604/37962 [54:40:38<32:44:00,  8.21s/it]
 62%|██████▏   | 23605/37962 [54:40:47<32:49:13,  8.23s/it]
 62%|██████▏   | 23606/37962 [54:40:55<32:38:10,  8.18s/it]
 62%|██████▏   | 23607/37962 [54:41:03<32:53:57,  8.25s/it]
 62%|██████▏   | 23608/37962 [54:41:12<33:07:30,  8.31s/it]
 62%|██████▏   | 23609/37962 [54:41:20<33:01:34,  8.28s/it]
 62%|██████▏   | 23610/37962 [54:41:28<32:48:52,  8.23s/it]
                                                           
{'loss': 1.2621, 'grad_norm': 14.865009307861328, 'learning_rate': 6.605421371667219e-06, 'epoch': 0.62}

 62%|██████▏   | 23610/37962 [54:41:28<32:48:52,  8.23s/it]
 62%|██████▏   | 23611/37962 [54:41:36<33:07:18,  8.31s/it]
 62%|██████▏   | 23612/37962 [54:41:44<32:50:48,  8.24s/it]
 62%|██████▏   | 23613/37962 [54:41:52<32:30:54,  8.16s/it]
 62%|██████▏   | 23614/37962 [54:42:01<32:38:56,  8.19s/it]
 62%|██████▏   | 23615/37962 [54:42:09<32:45:10,  8.22s/it]
 62%|██████▏   | 23616/37962 [54:42:18<33:09:22,  8.32s/it]
 62%|██████▏   | 23617/37962 [54:42:26<33:16:20,  8.35s/it]
 62%|██████▏   | 23618/37962 [54:42:34<33:19:46,  8.36s/it]
 62%|██████▏   | 23619/37962 [54:42:43<33:23:08,  8.38s/it]
 62%|██████▏   | 23620/37962 [54:42:51<33:13:36,  8.34s/it]
                                                           
{'loss': 1.2812, 'grad_norm': 8.125730514526367, 'learning_rate': 6.597397599655295e-06, 'epoch': 0.62}

 62%|██████▏   | 23620/37962 [54:42:51<33:13:36,  8.34s/it]
 62%|██████▏   | 23621/37962 [54:42:59<33:06:11,  8.31s/it]
 62%|██████▏   | 23622/37962 [54:43:08<33:06:14,  8.31s/it]
 62%|██████▏   | 23623/37962 [54:43:16<33:01:05,  8.29s/it]
 62%|██████▏   | 23624/37962 [54:43:24<33:01:08,  8.29s/it]
 62%|██████▏   | 23625/37962 [54:43:33<33:20:58,  8.37s/it]
 62%|██████▏   | 23626/37962 [54:43:41<33:17:28,  8.36s/it]
 62%|██████▏   | 23627/37962 [54:43:50<33:26:53,  8.40s/it]
 62%|██████▏   | 23628/37962 [54:43:58<33:20:43,  8.37s/it]
 62%|██████▏   | 23629/37962 [54:44:06<33:27:35,  8.40s/it]
 62%|██████▏   | 23630/37962 [54:44:14<32:54:12,  8.26s/it]
                                                           
{'loss': 1.2496, 'grad_norm': 3.3977854251861572, 'learning_rate': 6.589376304338713e-06, 'epoch': 0.62}

 62%|██████▏   | 23630/37962 [54:44:14<32:54:12,  8.26s/it]
 62%|██████▏   | 23631/37962 [54:44:23<33:03:27,  8.30s/it]
 62%|██████▏   | 23632/37962 [54:44:31<33:05:57,  8.32s/it]
 62%|██████▏   | 23633/37962 [54:44:39<33:08:44,  8.33s/it]
 62%|██████▏   | 23634/37962 [54:44:48<33:09:20,  8.33s/it]
 62%|██████▏   | 23635/37962 [54:44:56<33:05:08,  8.31s/it]
 62%|██████▏   | 23636/37962 [54:45:04<33:08:30,  8.33s/it]
 62%|██████▏   | 23637/37962 [54:45:13<33:24:32,  8.40s/it]
 62%|██████▏   | 23638/37962 [54:45:21<33:14:01,  8.35s/it]
 62%|██████▏   | 23639/37962 [54:45:29<33:05:09,  8.32s/it]
 62%|██████▏   | 23640/37962 [54:45:37<32:47:43,  8.24s/it]
                                                           
{'loss': 1.2761, 'grad_norm': 5.114952087402344, 'learning_rate': 6.581357491556034e-06, 'epoch': 0.62}

 62%|██████▏   | 23640/37962 [54:45:37<32:47:43,  8.24s/it]
 62%|██████▏   | 23641/37962 [54:45:45<32:18:16,  8.12s/it]
 62%|██████▏   | 23642/37962 [54:45:54<32:33:43,  8.19s/it]
 62%|██████▏   | 23643/37962 [54:46:02<32:44:11,  8.23s/it]
 62%|██████▏   | 23644/37962 [54:46:10<32:44:13,  8.23s/it]
 62%|██████▏   | 23645/37962 [54:46:18<32:43:43,  8.23s/it]
 62%|██████▏   | 23646/37962 [54:46:27<32:59:58,  8.30s/it]
 62%|██████▏   | 23647/37962 [54:46:35<32:55:14,  8.28s/it]
 62%|██████▏   | 23648/37962 [54:46:43<32:58:26,  8.29s/it]
 62%|██████▏   | 23649/37962 [54:46:52<33:03:50,  8.32s/it]
 62%|██████▏   | 23650/37962 [54:47:00<32:52:51,  8.27s/it]
                                                           
{'loss': 1.2129, 'grad_norm': 3.1746456623077393, 'learning_rate': 6.573341167144015e-06, 'epoch': 0.62}

 62%|██████▏   | 23650/37962 [54:47:00<32:52:51,  8.27s/it]
 62%|██████▏   | 23651/37962 [54:47:08<32:49:21,  8.26s/it]
 62%|██████▏   | 23652/37962 [54:47:17<32:56:52,  8.29s/it]
 62%|██████▏   | 23653/37962 [54:47:25<32:52:01,  8.27s/it]
 62%|██████▏   | 23654/37962 [54:47:33<32:57:01,  8.29s/it]
 62%|██████▏   | 23655/37962 [54:47:41<32:57:46,  8.29s/it]
 62%|██████▏   | 23656/37962 [54:47:50<33:01:41,  8.31s/it]
 62%|██████▏   | 23657/37962 [54:47:58<33:00:16,  8.31s/it]
 62%|██████▏   | 23658/37962 [54:48:06<33:03:48,  8.32s/it]
 62%|██████▏   | 23659/37962 [54:48:15<33:02:11,  8.32s/it]
 62%|██████▏   | 23660/37962 [54:48:23<32:51:17,  8.27s/it]
                                                           
{'loss': 1.2632, 'grad_norm': 5.782456398010254, 'learning_rate': 6.565327336937596e-06, 'epoch': 0.62}

 62%|██████▏   | 23660/37962 [54:48:23<32:51:17,  8.27s/it]
 62%|██████▏   | 23661/37962 [54:48:31<32:24:27,  8.16s/it]
 62%|██████▏   | 23662/37962 [54:48:39<32:16:35,  8.13s/it]
 62%|██████▏   | 23663/37962 [54:48:47<32:39:15,  8.22s/it]
 62%|██████▏   | 23664/37962 [54:48:55<32:37:00,  8.21s/it]
 62%|██████▏   | 23665/37962 [54:49:04<32:39:35,  8.22s/it]
 62%|██████▏   | 23666/37962 [54:49:12<32:32:45,  8.20s/it]
 62%|██████▏   | 23667/37962 [54:49:20<32:50:56,  8.27s/it]
 62%|██████▏   | 23668/37962 [54:49:29<32:55:28,  8.29s/it]
 62%|██████▏   | 23669/37962 [54:49:37<33:07:04,  8.34s/it]
 62%|██████▏   | 23670/37962 [54:49:45<33:01:55,  8.32s/it]
                                                           
{'loss': 1.1876, 'grad_norm': 9.44813346862793, 'learning_rate': 6.557316006769906e-06, 'epoch': 0.62}

 62%|██████▏   | 23670/37962 [54:49:45<33:01:55,  8.32s/it]
 62%|██████▏   | 23671/37962 [54:49:54<33:04:18,  8.33s/it]
 62%|██████▏   | 23672/37962 [54:50:02<32:59:59,  8.31s/it]
 62%|██████▏   | 23673/37962 [54:50:10<32:54:02,  8.29s/it]
 62%|██████▏   | 23674/37962 [54:50:18<32:27:44,  8.18s/it]
 62%|██████▏   | 23675/37962 [54:50:26<32:18:56,  8.14s/it]
 62%|██████▏   | 23676/37962 [54:50:34<32:30:00,  8.19s/it]
 62%|██████▏   | 23677/37962 [54:50:43<32:30:10,  8.19s/it]
 62%|██████▏   | 23678/37962 [54:50:51<32:52:39,  8.29s/it]
 62%|██████▏   | 23679/37962 [54:50:59<32:51:17,  8.28s/it]
 62%|██████▏   | 23680/37962 [54:51:08<32:58:08,  8.31s/it]
                                                           
{'loss': 1.1504, 'grad_norm': 4.75942325592041, 'learning_rate': 6.549307182472253e-06, 'epoch': 0.62}

 62%|██████▏   | 23680/37962 [54:51:08<32:58:08,  8.31s/it]
 62%|██████▏   | 23681/37962 [54:51:16<32:53:29,  8.29s/it]
 62%|██████▏   | 23682/37962 [54:51:24<32:52:00,  8.29s/it]
 62%|██████▏   | 23683/37962 [54:51:33<32:51:31,  8.28s/it]
 62%|██████▏   | 23684/37962 [54:51:41<32:51:58,  8.29s/it]
 62%|██████▏   | 23685/37962 [54:51:49<32:53:08,  8.29s/it]
 62%|██████▏   | 23686/37962 [54:51:58<32:55:21,  8.30s/it]
 62%|██████▏   | 23687/37962 [54:52:05<32:22:19,  8.16s/it]
 62%|██████▏   | 23688/37962 [54:52:14<32:35:29,  8.22s/it]
 62%|██████▏   | 23689/37962 [54:52:22<32:39:05,  8.24s/it]
 62%|██████▏   | 23690/37962 [54:52:30<32:40:59,  8.24s/it]
                                                           
{'loss': 1.3393, 'grad_norm': 24.049909591674805, 'learning_rate': 6.541300869874123e-06, 'epoch': 0.62}

 62%|██████▏   | 23690/37962 [54:52:30<32:40:59,  8.24s/it]
 62%|██████▏   | 23691/37962 [54:52:38<32:12:20,  8.12s/it]
 62%|██████▏   | 23692/37962 [54:52:46<32:22:51,  8.17s/it]
 62%|██████▏   | 23693/37962 [54:52:55<32:36:32,  8.23s/it]
 62%|██████▏   | 23694/37962 [54:53:03<32:38:59,  8.24s/it]
 62%|██████▏   | 23695/37962 [54:53:11<32:46:21,  8.27s/it]
 62%|██████▏   | 23696/37962 [54:53:20<32:56:37,  8.31s/it]
 62%|██████▏   | 23697/37962 [54:53:28<32:59:57,  8.33s/it]
 62%|██████▏   | 23698/37962 [54:53:36<32:59:16,  8.33s/it]
 62%|██████▏   | 23699/37962 [54:53:45<32:57:57,  8.32s/it]
 62%|██████▏   | 23700/37962 [54:53:53<33:20:13,  8.41s/it]
                                                           
{'loss': 1.1778, 'grad_norm': 5.5013556480407715, 'learning_rate': 6.533297074803175e-06, 'epoch': 0.62}

 62%|██████▏   | 23700/37962 [54:53:53<33:20:13,  8.41s/it]
 62%|██████▏   | 23701/37962 [54:54:02<33:25:09,  8.44s/it]
 62%|██████▏   | 23702/37962 [54:54:10<33:04:51,  8.35s/it]
 62%|██████▏   | 23703/37962 [54:54:18<33:05:16,  8.35s/it]
 62%|██████▏   | 23704/37962 [54:54:27<33:11:13,  8.38s/it]
 62%|██████▏   | 23705/37962 [54:54:35<33:20:26,  8.42s/it]
 62%|██████▏   | 23706/37962 [54:54:44<33:07:22,  8.36s/it]
 62%|██████▏   | 23707/37962 [54:54:52<32:56:08,  8.32s/it]
 62%|██████▏   | 23708/37962 [54:55:00<32:49:15,  8.29s/it]
 62%|██████▏   | 23709/37962 [54:55:08<32:39:53,  8.25s/it]
 62%|██████▏   | 23710/37962 [54:55:16<32:39:37,  8.25s/it]
                                                           
{'loss': 1.3862, 'grad_norm': 4.614253997802734, 'learning_rate': 6.525295803085229e-06, 'epoch': 0.62}

 62%|██████▏   | 23710/37962 [54:55:16<32:39:37,  8.25s/it]
 62%|██████▏   | 23711/37962 [54:55:25<32:39:01,  8.25s/it]
 62%|██████▏   | 23712/37962 [54:55:33<32:39:51,  8.25s/it]
 62%|██████▏   | 23713/37962 [54:55:41<32:32:53,  8.22s/it]
 62%|██████▏   | 23714/37962 [54:55:49<32:37:38,  8.24s/it]
 62%|██████▏   | 23715/37962 [54:55:57<32:26:23,  8.20s/it]
 62%|██████▏   | 23716/37962 [54:56:06<32:36:13,  8.24s/it]
 62%|██████▏   | 23717/37962 [54:56:14<32:43:29,  8.27s/it]
 62%|██████▏   | 23718/37962 [54:56:22<32:39:25,  8.25s/it]
 62%|██████▏   | 23719/37962 [54:56:31<32:45:03,  8.28s/it]
 62%|██████▏   | 23720/37962 [54:56:39<32:26:47,  8.20s/it]
                                                           
{'loss': 1.2778, 'grad_norm': 14.510383605957031, 'learning_rate': 6.517297060544272e-06, 'epoch': 0.62}

 62%|██████▏   | 23720/37962 [54:56:39<32:26:47,  8.20s/it]
 62%|██████▏   | 23721/37962 [54:56:47<32:41:29,  8.26s/it]
 62%|██████▏   | 23722/37962 [54:56:55<32:39:04,  8.25s/it]
 62%|██████▏   | 23723/37962 [54:57:04<32:46:58,  8.29s/it]
 62%|██████▏   | 23724/37962 [54:57:12<32:47:01,  8.29s/it]
 62%|██████▏   | 23725/37962 [54:57:20<32:54:04,  8.32s/it]
 62%|██████▏   | 23726/37962 [54:57:29<33:03:39,  8.36s/it]
 63%|██████▎   | 23727/37962 [54:57:37<32:59:50,  8.34s/it]
 63%|██████▎   | 23728/37962 [54:57:46<32:59:14,  8.34s/it]
 63%|██████▎   | 23729/37962 [54:57:53<32:26:56,  8.21s/it]
 63%|██████▎   | 23730/37962 [54:58:02<32:39:14,  8.26s/it]
                                                           
{'loss': 1.1987, 'grad_norm': 11.528109550476074, 'learning_rate': 6.509300853002451e-06, 'epoch': 0.63}

 63%|██████▎   | 23730/37962 [54:58:02<32:39:14,  8.26s/it]
 63%|██████▎   | 23731/37962 [54:58:10<32:37:15,  8.25s/it]
 63%|██████▎   | 23732/37962 [54:58:18<32:37:00,  8.25s/it]
 63%|██████▎   | 23733/37962 [54:58:27<32:38:14,  8.26s/it]
 63%|██████▎   | 23734/37962 [54:58:35<32:40:48,  8.27s/it]
 63%|██████▎   | 23735/37962 [54:58:43<32:45:01,  8.29s/it]
 63%|██████▎   | 23736/37962 [54:58:51<32:22:06,  8.19s/it]
 63%|██████▎   | 23737/37962 [54:58:59<32:27:29,  8.21s/it]
 63%|██████▎   | 23738/37962 [54:59:08<32:49:24,  8.31s/it]
 63%|██████▎   | 23739/37962 [54:59:16<32:53:40,  8.33s/it]
 63%|██████▎   | 23740/37962 [54:59:25<32:47:18,  8.30s/it]
                                                           
{'loss': 1.2856, 'grad_norm': 9.3656644821167, 'learning_rate': 6.501307186280066e-06, 'epoch': 0.63}

 63%|██████▎   | 23740/37962 [54:59:25<32:47:18,  8.30s/it]
 63%|██████▎   | 23741/37962 [54:59:33<32:47:50,  8.30s/it]
 63%|██████▎   | 23742/37962 [54:59:41<32:48:23,  8.31s/it]
 63%|██████▎   | 23743/37962 [54:59:49<32:46:46,  8.30s/it]
 63%|██████▎   | 23744/37962 [54:59:57<32:15:28,  8.17s/it]
 63%|██████▎   | 23745/37962 [55:00:05<31:53:54,  8.08s/it]
 63%|██████▎   | 23746/37962 [55:00:14<32:13:13,  8.16s/it]
 63%|██████▎   | 23747/37962 [55:00:22<32:33:44,  8.25s/it]
 63%|██████▎   | 23748/37962 [55:00:30<32:37:24,  8.26s/it]
 63%|██████▎   | 23749/37962 [55:00:38<32:31:18,  8.24s/it]
 63%|██████▎   | 23750/37962 [55:00:47<33:00:38,  8.36s/it]
                                                           
{'loss': 1.3209, 'grad_norm': 14.30043888092041, 'learning_rate': 6.493316066195572e-06, 'epoch': 0.63}

 63%|██████▎   | 23750/37962 [55:00:47<33:00:38,  8.36s/it]
 63%|██████▎   | 23751/37962 [55:00:55<32:40:11,  8.28s/it]
 63%|██████▎   | 23752/37962 [55:01:04<32:58:15,  8.35s/it]
 63%|██████▎   | 23753/37962 [55:01:12<32:59:25,  8.36s/it]
 63%|██████▎   | 23754/37962 [55:01:20<32:23:40,  8.21s/it]
 63%|██████▎   | 23755/37962 [55:01:29<32:57:03,  8.35s/it]
 63%|██████▎   | 23756/37962 [55:01:37<32:48:20,  8.31s/it]
 63%|██████▎   | 23757/37962 [55:01:45<32:52:45,  8.33s/it]
 63%|██████▎   | 23758/37962 [55:01:53<32:43:04,  8.29s/it]
 63%|██████▎   | 23759/37962 [55:02:02<32:58:34,  8.36s/it]
 63%|██████▎   | 23760/37962 [55:02:10<32:58:52,  8.36s/it]
                                                           
{'loss': 1.3194, 'grad_norm': 6.584784984588623, 'learning_rate': 6.485327498565562e-06, 'epoch': 0.63}

 63%|██████▎   | 23760/37962 [55:02:10<32:58:52,  8.36s/it]
 63%|██████▎   | 23761/37962 [55:02:18<32:36:02,  8.26s/it]
 63%|██████▎   | 23762/37962 [55:02:26<32:05:03,  8.13s/it]
 63%|██████▎   | 23763/37962 [55:02:35<32:22:21,  8.21s/it]
 63%|██████▎   | 23764/37962 [55:02:43<32:30:42,  8.24s/it]
 63%|██████▎   | 23765/37962 [55:02:51<32:27:29,  8.23s/it]
 63%|██████▎   | 23766/37962 [55:02:59<32:36:32,  8.27s/it]
 63%|██████▎   | 23767/37962 [55:03:08<32:26:55,  8.23s/it]
 63%|██████▎   | 23768/37962 [55:03:16<32:22:34,  8.21s/it]
 63%|██████▎   | 23769/37962 [55:03:24<32:28:58,  8.24s/it]
 63%|██████▎   | 23770/37962 [55:03:32<32:40:28,  8.29s/it]
                                                           
{'loss': 1.3032, 'grad_norm': 10.42960262298584, 'learning_rate': 6.4773414892047785e-06, 'epoch': 0.63}

 63%|██████▎   | 23770/37962 [55:03:32<32:40:28,  8.29s/it]
 63%|██████▎   | 23771/37962 [55:03:40<32:07:19,  8.15s/it]
 63%|██████▎   | 23772/37962 [55:03:49<32:17:36,  8.19s/it]
 63%|██████▎   | 23773/37962 [55:03:56<31:56:30,  8.10s/it]
 63%|██████▎   | 23774/37962 [55:04:04<31:51:03,  8.08s/it]
 63%|██████▎   | 23775/37962 [55:04:13<32:14:55,  8.18s/it]
 63%|██████▎   | 23776/37962 [55:04:21<32:40:55,  8.29s/it]
 63%|██████▎   | 23777/37962 [55:04:30<32:31:32,  8.25s/it]
 63%|██████▎   | 23778/37962 [55:04:38<32:27:16,  8.24s/it]
 63%|██████▎   | 23779/37962 [55:04:46<32:29:42,  8.25s/it]
 63%|██████▎   | 23780/37962 [55:04:55<32:54:37,  8.35s/it]
                                                           
{'loss': 1.2705, 'grad_norm': 9.03256893157959, 'learning_rate': 6.469358043926097e-06, 'epoch': 0.63}

 63%|██████▎   | 23780/37962 [55:04:55<32:54:37,  8.35s/it]
 63%|██████▎   | 23781/37962 [55:05:03<32:39:21,  8.29s/it]
 63%|██████▎   | 23782/37962 [55:05:11<32:36:45,  8.28s/it]
 63%|██████▎   | 23783/37962 [55:05:19<32:30:31,  8.25s/it]
 63%|██████▎   | 23784/37962 [55:05:28<32:39:33,  8.29s/it]
 63%|██████▎   | 23785/37962 [55:05:36<32:44:27,  8.31s/it]
 63%|██████▎   | 23786/37962 [55:05:44<32:40:19,  8.30s/it]
 63%|██████▎   | 23787/37962 [55:05:53<32:33:48,  8.27s/it]
 63%|██████▎   | 23788/37962 [55:06:01<32:32:22,  8.26s/it]
 63%|██████▎   | 23789/37962 [55:06:09<32:43:32,  8.31s/it]
 63%|██████▎   | 23790/37962 [55:06:17<32:38:59,  8.29s/it]
                                                           
{'loss': 1.3129, 'grad_norm': 5.9911885261535645, 'learning_rate': 6.461377168540534e-06, 'epoch': 0.63}

 63%|██████▎   | 23790/37962 [55:06:17<32:38:59,  8.29s/it]
 63%|██████▎   | 23791/37962 [55:06:26<32:38:12,  8.29s/it]
 63%|██████▎   | 23792/37962 [55:06:34<32:41:14,  8.30s/it]
 63%|██████▎   | 23793/37962 [55:06:43<32:56:48,  8.37s/it]
 63%|██████▎   | 23794/37962 [55:06:51<32:32:03,  8.27s/it]
 63%|██████▎   | 23795/37962 [55:06:59<32:25:44,  8.24s/it]
 63%|██████▎   | 23796/37962 [55:07:07<32:40:48,  8.30s/it]
 63%|██████▎   | 23797/37962 [55:07:15<32:30:00,  8.26s/it]
 63%|██████▎   | 23798/37962 [55:07:24<32:41:07,  8.31s/it]
 63%|██████▎   | 23799/37962 [55:07:32<32:38:51,  8.30s/it]
 63%|██████▎   | 23800/37962 [55:07:40<32:41:28,  8.31s/it]
                                                           
{'loss': 1.2359, 'grad_norm': 6.663418292999268, 'learning_rate': 6.453398868857226e-06, 'epoch': 0.63}

 63%|██████▎   | 23800/37962 [55:07:40<32:41:28,  8.31s/it]
 63%|██████▎   | 23801/37962 [55:07:49<32:40:04,  8.30s/it]
 63%|██████▎   | 23802/37962 [55:07:57<32:13:58,  8.19s/it]
 63%|██████▎   | 23803/37962 [55:08:05<32:24:57,  8.24s/it]
 63%|██████▎   | 23804/37962 [55:08:14<32:45:47,  8.33s/it]
 63%|██████▎   | 23805/37962 [55:08:22<32:34:10,  8.28s/it]
 63%|██████▎   | 23806/37962 [55:08:30<32:31:39,  8.27s/it]
 63%|██████▎   | 23807/37962 [55:08:38<32:35:27,  8.29s/it]
 63%|██████▎   | 23808/37962 [55:08:46<32:23:58,  8.24s/it]
 63%|██████▎   | 23809/37962 [55:08:54<32:07:38,  8.17s/it]
 63%|██████▎   | 23810/37962 [55:09:03<32:14:23,  8.20s/it]
                                                           
{'loss': 1.1724, 'grad_norm': 5.57323694229126, 'learning_rate': 6.445423150683444e-06, 'epoch': 0.63}

 63%|██████▎   | 23810/37962 [55:09:03<32:14:23,  8.20s/it]
 63%|██████▎   | 23811/37962 [55:09:11<32:15:40,  8.21s/it]
 63%|██████▎   | 23812/37962 [55:09:19<32:02:41,  8.15s/it]
 63%|██████▎   | 23813/37962 [55:09:27<32:11:44,  8.19s/it]
 63%|██████▎   | 23814/37962 [55:09:36<32:30:43,  8.27s/it]
 63%|██████▎   | 23815/37962 [55:09:44<32:04:59,  8.16s/it]
 63%|██████▎   | 23816/37962 [55:09:52<32:14:24,  8.20s/it]
 63%|██████▎   | 23817/37962 [55:10:00<32:14:50,  8.21s/it]
 63%|██████▎   | 23818/37962 [55:10:08<32:27:10,  8.26s/it]
 63%|██████▎   | 23819/37962 [55:10:17<32:25:39,  8.25s/it]
 63%|██████▎   | 23820/37962 [55:10:25<32:22:42,  8.24s/it]
                                                           
{'loss': 1.2486, 'grad_norm': 6.364928245544434, 'learning_rate': 6.4374500198245695e-06, 'epoch': 0.63}

 63%|██████▎   | 23820/37962 [55:10:25<32:22:42,  8.24s/it]
 63%|██████▎   | 23821/37962 [55:10:33<32:30:04,  8.27s/it]
 63%|██████▎   | 23822/37962 [55:10:42<32:36:04,  8.30s/it]
 63%|██████▎   | 23823/37962 [55:10:50<32:29:42,  8.27s/it]
 63%|██████▎   | 23824/37962 [55:10:58<32:23:42,  8.25s/it]
 63%|██████▎   | 23825/37962 [55:11:06<32:11:10,  8.20s/it]
 63%|██████▎   | 23826/37962 [55:11:14<32:12:05,  8.20s/it]
 63%|██████▎   | 23827/37962 [55:11:23<32:14:23,  8.21s/it]
 63%|██████▎   | 23828/37962 [55:11:31<32:24:03,  8.25s/it]
 63%|██████▎   | 23829/37962 [55:11:39<31:54:32,  8.13s/it]
 63%|██████▎   | 23830/37962 [55:11:47<31:38:30,  8.06s/it]
                                                           
{'loss': 1.306, 'grad_norm': 8.837688446044922, 'learning_rate': 6.429479482084109e-06, 'epoch': 0.63}

 63%|██████▎   | 23830/37962 [55:11:47<31:38:30,  8.06s/it]
 63%|██████▎   | 23831/37962 [55:11:55<31:58:18,  8.15s/it]
 63%|██████▎   | 23832/37962 [55:12:03<31:49:17,  8.11s/it]
 63%|██████▎   | 23833/37962 [55:12:11<31:58:16,  8.15s/it]
 63%|██████▎   | 23834/37962 [55:12:19<32:01:55,  8.16s/it]
 63%|██████▎   | 23835/37962 [55:12:28<32:11:56,  8.21s/it]
 63%|██████▎   | 23836/37962 [55:12:36<32:14:17,  8.22s/it]
 63%|██████▎   | 23837/37962 [55:12:51<40:24:39, 10.30s/it]
 63%|██████▎   | 23838/37962 [55:13:00<38:08:34,  9.72s/it]
 63%|██████▎   | 23839/37962 [55:13:08<36:14:53,  9.24s/it]
 63%|██████▎   | 23840/37962 [55:13:16<35:06:52,  8.95s/it]
                                                           
{'loss': 1.2597, 'grad_norm': 11.236944198608398, 'learning_rate': 6.4215115432636774e-06, 'epoch': 0.63}

 63%|██████▎   | 23840/37962 [55:13:16<35:06:52,  8.95s/it]
 63%|██████▎   | 23841/37962 [55:13:24<34:16:22,  8.74s/it]
 63%|██████▎   | 23842/37962 [55:13:33<33:51:15,  8.63s/it]
 63%|██████▎   | 23843/37962 [55:13:41<33:31:52,  8.55s/it]
 63%|██████▎   | 23844/37962 [55:13:49<32:59:15,  8.41s/it]
 63%|██████▎   | 23845/37962 [55:13:57<32:49:19,  8.37s/it]
 63%|██████▎   | 23846/37962 [55:14:06<32:41:21,  8.34s/it]
 63%|██████▎   | 23847/37962 [55:14:14<32:45:29,  8.35s/it]
 63%|██████▎   | 23848/37962 [55:14:22<32:24:52,  8.27s/it]
 63%|██████▎   | 23849/37962 [55:14:30<32:39:07,  8.33s/it]
 63%|██████▎   | 23850/37962 [55:14:39<32:36:59,  8.32s/it]
                                                           
{'loss': 1.2744, 'grad_norm': 11.27929973602295, 'learning_rate': 6.4135462091629995e-06, 'epoch': 0.63}

 63%|██████▎   | 23850/37962 [55:14:39<32:36:59,  8.32s/it]
 63%|██████▎   | 23851/37962 [55:14:47<32:24:57,  8.27s/it]
 63%|██████▎   | 23852/37962 [55:14:55<32:35:28,  8.32s/it]
 63%|██████▎   | 23853/37962 [55:15:04<32:38:33,  8.33s/it]
 63%|██████▎   | 23854/37962 [55:15:12<32:32:54,  8.31s/it]
 63%|██████▎   | 23855/37962 [55:15:20<32:31:08,  8.30s/it]
 63%|██████▎   | 23856/37962 [55:15:29<32:28:49,  8.29s/it]
 63%|██████▎   | 23857/37962 [55:15:37<32:19:22,  8.25s/it]
 63%|██████▎   | 23858/37962 [55:15:45<32:13:53,  8.23s/it]
 63%|██████▎   | 23859/37962 [55:15:53<32:17:09,  8.24s/it]
 63%|██████▎   | 23860/37962 [55:16:01<32:25:32,  8.28s/it]
                                                           
{'loss': 1.1776, 'grad_norm': 18.471120834350586, 'learning_rate': 6.4055834855799095e-06, 'epoch': 0.63}

 63%|██████▎   | 23860/37962 [55:16:01<32:25:32,  8.28s/it]
 63%|██████▎   | 23861/37962 [55:16:10<32:23:57,  8.27s/it]
 63%|██████▎   | 23862/37962 [55:16:18<32:28:10,  8.29s/it]
 63%|██████▎   | 23863/37962 [55:16:26<32:26:45,  8.28s/it]
 63%|██████▎   | 23864/37962 [55:16:35<32:31:36,  8.31s/it]
 63%|██████▎   | 23865/37962 [55:16:43<32:38:15,  8.33s/it]
 63%|██████▎   | 23866/37962 [55:16:51<32:29:03,  8.30s/it]
 63%|██████▎   | 23867/37962 [55:17:00<32:38:33,  8.34s/it]
 63%|██████▎   | 23868/37962 [55:17:08<32:11:40,  8.22s/it]
 63%|██████▎   | 23869/37962 [55:17:16<31:41:20,  8.09s/it]
 63%|██████▎   | 23870/37962 [55:17:24<31:35:04,  8.07s/it]
                                                           
{'loss': 1.3147, 'grad_norm': 6.142937660217285, 'learning_rate': 6.397623378310329e-06, 'epoch': 0.63}

 63%|██████▎   | 23870/37962 [55:17:24<31:35:04,  8.07s/it]
 63%|██████▎   | 23871/37962 [55:17:32<32:09:59,  8.22s/it]
 63%|██████▎   | 23872/37962 [55:17:40<32:11:14,  8.22s/it]
 63%|██████▎   | 23873/37962 [55:17:49<32:16:06,  8.25s/it]
 63%|██████▎   | 23874/37962 [55:17:57<32:19:28,  8.26s/it]
 63%|██████▎   | 23875/37962 [55:18:05<32:28:48,  8.30s/it]
 63%|██████▎   | 23876/37962 [55:18:13<32:17:46,  8.25s/it]
 63%|██████▎   | 23877/37962 [55:18:22<32:34:04,  8.32s/it]
 63%|██████▎   | 23878/37962 [55:18:30<32:45:02,  8.37s/it]
 63%|██████▎   | 23879/37962 [55:18:39<32:50:54,  8.40s/it]
 63%|██████▎   | 23880/37962 [55:18:47<32:45:18,  8.37s/it]
                                                           
{'loss': 1.148, 'grad_norm': 15.222465515136719, 'learning_rate': 6.389665893148281e-06, 'epoch': 0.63}

 63%|██████▎   | 23880/37962 [55:18:47<32:45:18,  8.37s/it]
 63%|██████▎   | 23881/37962 [55:18:55<32:38:58,  8.35s/it]
 63%|██████▎   | 23882/37962 [55:19:04<32:35:37,  8.33s/it]
 63%|██████▎   | 23883/37962 [55:19:12<32:21:05,  8.27s/it]
 63%|██████▎   | 23884/37962 [55:19:20<32:25:26,  8.29s/it]
 63%|██████▎   | 23885/37962 [55:19:29<32:26:14,  8.30s/it]
 63%|██████▎   | 23886/37962 [55:19:37<32:28:43,  8.31s/it]
 63%|██████▎   | 23887/37962 [55:19:45<32:33:35,  8.33s/it]
 63%|██████▎   | 23888/37962 [55:19:54<32:32:14,  8.32s/it]
 63%|██████▎   | 23889/37962 [55:20:02<32:18:55,  8.27s/it]
 63%|██████▎   | 23890/37962 [55:20:10<32:19:40,  8.27s/it]
                                                           
{'loss': 1.2741, 'grad_norm': 6.507455348968506, 'learning_rate': 6.381711035885887e-06, 'epoch': 0.63}

 63%|██████▎   | 23890/37962 [55:20:10<32:19:40,  8.27s/it]
 63%|██████▎   | 23891/37962 [55:20:18<31:49:42,  8.14s/it]
 63%|██████▎   | 23892/37962 [55:20:26<32:05:53,  8.21s/it]
 63%|██████▎   | 23893/37962 [55:20:35<32:17:04,  8.26s/it]
 63%|██████▎   | 23894/37962 [55:20:43<32:25:52,  8.30s/it]
 63%|██████▎   | 23895/37962 [55:20:51<32:26:53,  8.30s/it]
 63%|██████▎   | 23896/37962 [55:20:59<32:20:14,  8.28s/it]
 63%|██████▎   | 23897/37962 [55:21:08<32:26:02,  8.30s/it]
 63%|██████▎   | 23898/37962 [55:21:16<32:27:06,  8.31s/it]
 63%|██████▎   | 23899/37962 [55:21:24<32:20:52,  8.28s/it]
 63%|██████▎   | 23900/37962 [55:21:33<32:31:29,  8.33s/it]
                                                           
{'loss': 1.2669, 'grad_norm': 10.206660270690918, 'learning_rate': 6.373758812313349e-06, 'epoch': 0.63}

 63%|██████▎   | 23900/37962 [55:21:33<32:31:29,  8.33s/it]
 63%|██████▎   | 23901/37962 [55:21:41<32:36:44,  8.35s/it]
 63%|██████▎   | 23902/37962 [55:21:49<32:16:12,  8.26s/it]
 63%|██████▎   | 23903/37962 [55:21:58<32:24:29,  8.30s/it]
 63%|██████▎   | 23904/37962 [55:22:06<32:38:17,  8.36s/it]
 63%|██████▎   | 23905/37962 [55:22:14<32:34:35,  8.34s/it]
 63%|██████▎   | 23906/37962 [55:22:23<32:41:11,  8.37s/it]
 63%|██████▎   | 23907/37962 [55:22:31<32:40:50,  8.37s/it]
 63%|██████▎   | 23908/37962 [55:22:40<32:34:35,  8.34s/it]
 63%|██████▎   | 23909/37962 [55:22:48<32:27:37,  8.32s/it]
 63%|██████▎   | 23910/37962 [55:22:56<32:27:14,  8.31s/it]
                                                           
{'loss': 1.3955, 'grad_norm': 14.38365364074707, 'learning_rate': 6.365809228218951e-06, 'epoch': 0.63}

 63%|██████▎   | 23910/37962 [55:22:56<32:27:14,  8.31s/it]
 63%|██████▎   | 23911/37962 [55:23:04<32:24:19,  8.30s/it]
 63%|██████▎   | 23912/37962 [55:23:13<32:39:42,  8.37s/it]
 63%|██████▎   | 23913/37962 [55:23:21<32:28:16,  8.32s/it]
 63%|██████▎   | 23914/37962 [55:23:29<32:29:24,  8.33s/it]
 63%|██████▎   | 23915/37962 [55:23:38<32:30:41,  8.33s/it]
 63%|██████▎   | 23916/37962 [55:23:46<32:33:55,  8.35s/it]
 63%|██████▎   | 23917/37962 [55:23:54<32:01:38,  8.21s/it]
 63%|██████▎   | 23918/37962 [55:24:02<32:06:17,  8.23s/it]
 63%|██████▎   | 23919/37962 [55:24:11<32:09:16,  8.24s/it]
 63%|██████▎   | 23920/37962 [55:24:19<32:08:52,  8.24s/it]
                                                           
{'loss': 1.1234, 'grad_norm': 7.23480749130249, 'learning_rate': 6.357862289389056e-06, 'epoch': 0.63}

 63%|██████▎   | 23920/37962 [55:24:19<32:08:52,  8.24s/it]
 63%|██████▎   | 23921/37962 [55:24:27<32:09:56,  8.25s/it]
 63%|██████▎   | 23922/37962 [55:24:36<32:22:50,  8.30s/it]
 63%|██████▎   | 23923/37962 [55:24:44<32:35:24,  8.36s/it]
 63%|██████▎   | 23924/37962 [55:24:53<32:43:36,  8.39s/it]
 63%|██████▎   | 23925/37962 [55:25:01<32:39:12,  8.37s/it]
 63%|██████▎   | 23926/37962 [55:25:09<32:30:39,  8.34s/it]
 63%|██████▎   | 23927/37962 [55:25:17<32:31:23,  8.34s/it]
 63%|██████▎   | 23928/37962 [55:25:26<32:32:17,  8.35s/it]
 63%|██████▎   | 23929/37962 [55:25:34<32:31:39,  8.34s/it]
 63%|██████▎   | 23930/37962 [55:25:42<32:27:51,  8.33s/it]
                                                           
{'loss': 1.3205, 'grad_norm': 11.297053337097168, 'learning_rate': 6.3499180016081065e-06, 'epoch': 0.63}

 63%|██████▎   | 23930/37962 [55:25:42<32:27:51,  8.33s/it]
 63%|██████▎   | 23931/37962 [55:25:51<32:25:39,  8.32s/it]
 63%|██████▎   | 23932/37962 [55:25:59<32:21:43,  8.30s/it]
 63%|██████▎   | 23933/37962 [55:26:07<32:24:44,  8.32s/it]
 63%|██████▎   | 23934/37962 [55:26:16<32:28:17,  8.33s/it]
 63%|██████▎   | 23935/37962 [55:26:24<32:21:32,  8.30s/it]
 63%|██████▎   | 23936/37962 [55:26:32<32:33:16,  8.36s/it]
 63%|██████▎   | 23937/37962 [55:26:41<32:25:03,  8.32s/it]
 63%|██████▎   | 23938/37962 [55:26:49<32:17:49,  8.29s/it]
 63%|██████▎   | 23939/37962 [55:26:57<32:19:18,  8.30s/it]
 63%|██████▎   | 23940/37962 [55:27:06<32:21:28,  8.31s/it]
                                                           
{'loss': 1.3233, 'grad_norm': 6.937431335449219, 'learning_rate': 6.341976370658609e-06, 'epoch': 0.63}

 63%|██████▎   | 23940/37962 [55:27:06<32:21:28,  8.31s/it]
 63%|██████▎   | 23941/37962 [55:27:14<32:30:31,  8.35s/it]
 63%|██████▎   | 23942/37962 [55:27:22<32:10:20,  8.26s/it]
 63%|██████▎   | 23943/37962 [55:27:30<32:08:52,  8.26s/it]
 63%|██████▎   | 23944/37962 [55:27:39<32:20:40,  8.31s/it]
 63%|██████▎   | 23945/37962 [55:27:47<32:15:16,  8.28s/it]
 63%|██████▎   | 23946/37962 [55:27:55<32:08:23,  8.26s/it]
 63%|██████▎   | 23947/37962 [55:28:03<32:13:10,  8.28s/it]
 63%|██████▎   | 23948/37962 [55:28:12<32:16:53,  8.29s/it]
 63%|██████▎   | 23949/37962 [55:28:20<32:13:47,  8.28s/it]
 63%|██████▎   | 23950/37962 [55:28:28<32:06:43,  8.25s/it]
                                                           
{'loss': 1.3714, 'grad_norm': 8.801004409790039, 'learning_rate': 6.3340374023211405e-06, 'epoch': 0.63}

 63%|██████▎   | 23950/37962 [55:28:28<32:06:43,  8.25s/it]
 63%|██████▎   | 23951/37962 [55:28:36<32:06:56,  8.25s/it]
 63%|██████▎   | 23952/37962 [55:28:45<31:50:26,  8.18s/it]
 63%|██████▎   | 23953/37962 [55:28:53<31:44:12,  8.16s/it]
 63%|██████▎   | 23954/37962 [55:29:01<32:00:20,  8.23s/it]
 63%|██████▎   | 23955/37962 [55:29:09<31:57:56,  8.22s/it]
 63%|██████▎   | 23956/37962 [55:29:17<31:58:28,  8.22s/it]
 63%|██████▎   | 23957/37962 [55:29:25<31:46:32,  8.17s/it]
 63%|██████▎   | 23958/37962 [55:29:34<31:59:18,  8.22s/it]
 63%|██████▎   | 23959/37962 [55:29:42<31:53:41,  8.20s/it]
 63%|██████▎   | 23960/37962 [55:29:50<31:59:41,  8.23s/it]
                                                           
{'loss': 1.2661, 'grad_norm': 10.611309051513672, 'learning_rate': 6.326101102374341e-06, 'epoch': 0.63}

 63%|██████▎   | 23960/37962 [55:29:50<31:59:41,  8.23s/it]
 63%|██████▎   | 23961/37962 [55:29:59<32:08:07,  8.26s/it]
 63%|██████▎   | 23962/37962 [55:30:07<32:18:19,  8.31s/it]
 63%|██████▎   | 23963/37962 [55:30:15<32:17:48,  8.31s/it]
 63%|██████▎   | 23964/37962 [55:30:23<31:52:53,  8.20s/it]
 63%|██████▎   | 23965/37962 [55:30:32<32:10:36,  8.28s/it]
 63%|██████▎   | 23966/37962 [55:30:40<32:12:39,  8.29s/it]
 63%|██████▎   | 23967/37962 [55:30:48<32:02:20,  8.24s/it]
 63%|██████▎   | 23968/37962 [55:30:56<32:00:12,  8.23s/it]
 63%|██████▎   | 23969/37962 [55:31:05<32:12:27,  8.29s/it]
 63%|██████▎   | 23970/37962 [55:31:13<32:08:54,  8.27s/it]
                                                           
{'loss': 1.2127, 'grad_norm': 14.521102905273438, 'learning_rate': 6.318167476594901e-06, 'epoch': 0.63}

 63%|██████▎   | 23970/37962 [55:31:13<32:08:54,  8.27s/it]
 63%|██████▎   | 23971/37962 [55:31:21<32:06:51,  8.26s/it]
 63%|██████▎   | 23972/37962 [55:31:29<32:04:12,  8.25s/it]
 63%|██████▎   | 23973/37962 [55:31:38<32:10:52,  8.28s/it]
 63%|██████▎   | 23974/37962 [55:31:46<32:27:43,  8.35s/it]
 63%|██████▎   | 23975/37962 [55:31:55<32:27:55,  8.36s/it]
 63%|██████▎   | 23976/37962 [55:32:03<32:18:47,  8.32s/it]
 63%|██████▎   | 23977/37962 [55:32:11<32:18:56,  8.32s/it]
 63%|██████▎   | 23978/37962 [55:32:20<32:29:22,  8.36s/it]
 63%|██████▎   | 23979/37962 [55:32:28<32:23:34,  8.34s/it]
 63%|██████▎   | 23980/37962 [55:32:36<32:25:38,  8.35s/it]
                                                           
{'loss': 1.2591, 'grad_norm': 4.533242225646973, 'learning_rate': 6.31023653075757e-06, 'epoch': 0.63}

 63%|██████▎   | 23980/37962 [55:32:36<32:25:38,  8.35s/it]
 63%|██████▎   | 23981/37962 [55:32:45<32:22:22,  8.34s/it]
 63%|██████▎   | 23982/37962 [55:32:53<32:26:30,  8.35s/it]
 63%|██████▎   | 23983/37962 [55:33:01<32:23:15,  8.34s/it]
 63%|██████▎   | 23984/37962 [55:33:09<31:50:48,  8.20s/it]
 63%|██████▎   | 23985/37962 [55:33:18<31:52:19,  8.21s/it]
 63%|██████▎   | 23986/37962 [55:33:26<32:01:36,  8.25s/it]
 63%|██████▎   | 23987/37962 [55:33:34<32:03:33,  8.26s/it]
 63%|██████▎   | 23988/37962 [55:33:42<31:58:30,  8.24s/it]
 63%|██████▎   | 23989/37962 [55:33:51<32:01:09,  8.25s/it]
 63%|██████▎   | 23990/37962 [55:33:59<31:47:32,  8.19s/it]
                                                           
{'loss': 1.2679, 'grad_norm': 4.9792890548706055, 'learning_rate': 6.302308270635147e-06, 'epoch': 0.63}

 63%|██████▎   | 23990/37962 [55:33:59<31:47:32,  8.19s/it]
 63%|██████▎   | 23991/37962 [55:34:07<31:51:42,  8.21s/it]
 63%|██████▎   | 23992/37962 [55:34:15<31:57:30,  8.24s/it]
 63%|██████▎   | 23993/37962 [55:34:23<31:36:16,  8.14s/it]
 63%|██████▎   | 23994/37962 [55:34:31<31:46:17,  8.19s/it]
 63%|██████▎   | 23995/37962 [55:34:40<31:55:07,  8.23s/it]
 63%|██████▎   | 23996/37962 [55:34:48<31:48:03,  8.20s/it]
 63%|██████▎   | 23997/37962 [55:34:56<31:17:03,  8.06s/it]
 63%|██████▎   | 23998/37962 [55:35:04<31:23:00,  8.09s/it]
 63%|██████▎   | 23999/37962 [55:35:12<31:49:36,  8.21s/it]
 63%|██████▎   | 24000/37962 [55:35:20<31:34:56,  8.14s/it]
                                                           
{'loss': 1.1929, 'grad_norm': 8.655173301696777, 'learning_rate': 6.294382701998478e-06, 'epoch': 0.63}

 63%|██████▎   | 24000/37962 [55:35:20<31:34:56,  8.14s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 63%|██████▎   | 24001/37962 [55:36:10<80:15:48, 20.70s/it]
 63%|██████▎   | 24002/37962 [55:36:19<65:51:44, 16.98s/it]
 63%|██████▎   | 24003/37962 [55:36:27<55:49:26, 14.40s/it]
 63%|██████▎   | 24004/37962 [55:36:35<48:45:06, 12.57s/it]
 63%|██████▎   | 24005/37962 [55:36:44<43:57:28, 11.34s/it]
 63%|██████▎   | 24006/37962 [55:36:52<40:12:40, 10.37s/it]
 63%|██████▎   | 24007/37962 [55:37:00<37:46:39,  9.75s/it]
 63%|██████▎   | 24008/37962 [55:37:08<35:52:16,  9.25s/it]
 63%|██████▎   | 24009/37962 [55:37:16<34:37:32,  8.93s/it]
 63%|██████▎   | 24010/37962 [55:37:25<33:49:16,  8.73s/it]
                                                           
{'loss': 1.2689, 'grad_norm': 17.959636688232422, 'learning_rate': 6.2864598306164435e-06, 'epoch': 0.63}

 63%|██████▎   | 24010/37962 [55:37:25<33:49:16,  8.73s/it]
 63%|██████▎   | 24011/37962 [55:37:33<33:19:29,  8.60s/it]
 63%|██████▎   | 24012/37962 [55:37:41<32:42:41,  8.44s/it]
 63%|██████▎   | 24013/37962 [55:37:49<32:19:33,  8.34s/it]
 63%|██████▎   | 24014/37962 [55:37:57<32:15:15,  8.32s/it]
 63%|██████▎   | 24015/37962 [55:38:06<32:27:11,  8.38s/it]
 63%|██████▎   | 24016/37962 [55:38:14<32:13:29,  8.32s/it]
 63%|██████▎   | 24017/37962 [55:38:22<31:56:12,  8.24s/it]
 63%|██████▎   | 24018/37962 [55:38:31<32:05:25,  8.28s/it]
 63%|██████▎   | 24019/37962 [55:38:39<32:13:18,  8.32s/it]
 63%|██████▎   | 24020/37962 [55:38:47<32:16:03,  8.33s/it]
                                                           
{'loss': 1.314, 'grad_norm': 16.479385375976562, 'learning_rate': 6.278539662255969e-06, 'epoch': 0.63}

 63%|██████▎   | 24020/37962 [55:38:47<32:16:03,  8.33s/it]
 63%|██████▎   | 24021/37962 [55:38:55<31:46:46,  8.21s/it]
 63%|██████▎   | 24022/37962 [55:39:03<31:51:46,  8.23s/it]
 63%|██████▎   | 24023/37962 [55:39:12<32:09:11,  8.30s/it]
 63%|██████▎   | 24024/37962 [55:39:20<32:05:22,  8.29s/it]
 63%|██████▎   | 24025/37962 [55:39:28<31:54:42,  8.24s/it]
 63%|██████▎   | 24026/37962 [55:39:37<32:08:47,  8.30s/it]
 63%|██████▎   | 24027/37962 [55:39:45<32:12:03,  8.32s/it]
 63%|██████▎   | 24028/37962 [55:39:54<32:16:13,  8.34s/it]
 63%|██████▎   | 24029/37962 [55:40:02<32:26:59,  8.38s/it]
 63%|██████▎   | 24030/37962 [55:40:10<32:10:53,  8.32s/it]
                                                           
{'loss': 1.3056, 'grad_norm': 9.303955078125, 'learning_rate': 6.270622202682002e-06, 'epoch': 0.63}

 63%|██████▎   | 24030/37962 [55:40:10<32:10:53,  8.32s/it]
 63%|██████▎   | 24031/37962 [55:40:18<32:07:38,  8.30s/it]
 63%|██████▎   | 24032/37962 [55:40:27<32:13:29,  8.33s/it]
 63%|██████▎   | 24033/37962 [55:40:35<32:03:38,  8.29s/it]
 63%|██████▎   | 24034/37962 [55:40:43<31:42:41,  8.20s/it]
 63%|██████▎   | 24035/37962 [55:40:51<31:49:16,  8.23s/it]
 63%|██████▎   | 24036/37962 [55:41:00<32:08:46,  8.31s/it]
 63%|██████▎   | 24037/37962 [55:41:08<32:09:40,  8.31s/it]
 63%|██████▎   | 24038/37962 [55:41:17<32:12:04,  8.33s/it]
 63%|██████▎   | 24039/37962 [55:41:25<32:12:10,  8.33s/it]
 63%|██████▎   | 24040/37962 [55:41:33<32:09:27,  8.32s/it]
                                                           
{'loss': 1.3237, 'grad_norm': 4.5777411460876465, 'learning_rate': 6.262707457657527e-06, 'epoch': 0.63}

 63%|██████▎   | 24040/37962 [55:41:33<32:09:27,  8.32s/it]
 63%|██████▎   | 24041/37962 [55:41:41<32:02:26,  8.29s/it]
 63%|██████▎   | 24042/37962 [55:41:50<32:03:36,  8.29s/it]
 63%|██████▎   | 24043/37962 [55:41:58<31:49:54,  8.23s/it]
 63%|██████▎   | 24044/37962 [55:42:06<31:53:06,  8.25s/it]
 63%|██████▎   | 24045/37962 [55:42:14<32:05:58,  8.30s/it]
 63%|██████▎   | 24046/37962 [55:42:23<32:09:04,  8.32s/it]
 63%|██████▎   | 24047/37962 [55:42:31<32:09:31,  8.32s/it]
 63%|██████▎   | 24048/37962 [55:42:39<32:11:19,  8.33s/it]
 63%|██████▎   | 24049/37962 [55:42:48<32:12:15,  8.33s/it]
 63%|██████▎   | 24050/37962 [55:42:56<32:16:23,  8.35s/it]
                                                           
{'loss': 1.3178, 'grad_norm': 5.1746978759765625, 'learning_rate': 6.2547954329435475e-06, 'epoch': 0.63}

 63%|██████▎   | 24050/37962 [55:42:56<32:16:23,  8.35s/it]
 63%|██████▎   | 24051/37962 [55:43:05<32:17:57,  8.36s/it]
 63%|██████▎   | 24052/37962 [55:43:13<32:17:42,  8.36s/it]
 63%|██████▎   | 24053/37962 [55:43:21<32:09:47,  8.32s/it]
 63%|██████▎   | 24054/37962 [55:43:30<32:09:36,  8.32s/it]
 63%|██████▎   | 24055/37962 [55:43:38<31:58:18,  8.28s/it]
 63%|██████▎   | 24056/37962 [55:43:46<31:57:07,  8.27s/it]
 63%|██████▎   | 24057/37962 [55:43:54<32:04:01,  8.30s/it]
 63%|██████▎   | 24058/37962 [55:44:03<31:59:56,  8.29s/it]
 63%|██████▎   | 24059/37962 [55:44:11<31:53:17,  8.26s/it]
 63%|██████▎   | 24060/37962 [55:44:19<32:00:01,  8.29s/it]
                                                           
{'loss': 1.3369, 'grad_norm': 6.559526443481445, 'learning_rate': 6.246886134299091e-06, 'epoch': 0.63}

 63%|██████▎   | 24060/37962 [55:44:19<32:00:01,  8.29s/it]
 63%|██████▎   | 24061/37962 [55:44:27<31:55:17,  8.27s/it]
 63%|██████▎   | 24062/37962 [55:44:36<31:59:28,  8.29s/it]
 63%|██████▎   | 24063/37962 [55:44:44<32:08:49,  8.33s/it]
 63%|██████▎   | 24064/37962 [55:44:52<31:58:09,  8.28s/it]
 63%|██████▎   | 24065/37962 [55:45:00<31:40:54,  8.21s/it]
 63%|██████▎   | 24066/37962 [55:45:09<31:44:13,  8.22s/it]
 63%|██████▎   | 24067/37962 [55:45:16<31:21:30,  8.12s/it]
 63%|██████▎   | 24068/37962 [55:45:25<31:33:39,  8.18s/it]
 63%|██████▎   | 24069/37962 [55:45:33<31:46:56,  8.24s/it]
 63%|██████▎   | 24070/37962 [55:45:42<32:05:25,  8.32s/it]
                                                           
{'loss': 1.2553, 'grad_norm': 5.893487930297852, 'learning_rate': 6.238979567481204e-06, 'epoch': 0.63}

 63%|██████▎   | 24070/37962 [55:45:42<32:05:25,  8.32s/it]
 63%|██████▎   | 24071/37962 [55:45:50<31:57:40,  8.28s/it]
 63%|██████▎   | 24072/37962 [55:45:58<31:50:06,  8.25s/it]
 63%|██████▎   | 24073/37962 [55:46:06<31:54:07,  8.27s/it]
 63%|██████▎   | 24074/37962 [55:46:15<31:58:37,  8.29s/it]
 63%|██████▎   | 24075/37962 [55:46:23<32:03:54,  8.31s/it]
 63%|██████▎   | 24076/37962 [55:46:31<31:57:15,  8.28s/it]
 63%|██████▎   | 24077/37962 [55:46:39<31:55:27,  8.28s/it]
 63%|██████▎   | 24078/37962 [55:46:48<31:49:01,  8.25s/it]
 63%|██████▎   | 24079/37962 [55:46:55<31:12:25,  8.09s/it]
 63%|██████▎   | 24080/37962 [55:47:04<31:31:42,  8.18s/it]
                                                           
{'loss': 1.1475, 'grad_norm': 9.396418571472168, 'learning_rate': 6.231075738244931e-06, 'epoch': 0.63}

 63%|██████▎   | 24080/37962 [55:47:04<31:31:42,  8.18s/it]
 63%|██████▎   | 24081/37962 [55:47:12<31:24:59,  8.15s/it]
 63%|██████▎   | 24082/37962 [55:47:20<31:28:37,  8.16s/it]
 63%|██████▎   | 24083/37962 [55:47:28<31:24:11,  8.15s/it]
 63%|██████▎   | 24084/37962 [55:47:36<31:36:11,  8.20s/it]
 63%|██████▎   | 24085/37962 [55:47:45<31:36:21,  8.20s/it]
 63%|██████▎   | 24086/37962 [55:47:53<31:30:01,  8.17s/it]
 63%|██████▎   | 24087/37962 [55:48:01<31:44:58,  8.24s/it]
 63%|██████▎   | 24088/37962 [55:48:09<31:46:14,  8.24s/it]
 63%|██████▎   | 24089/37962 [55:48:18<31:38:39,  8.21s/it]
 63%|██████▎   | 24090/37962 [55:48:26<31:56:31,  8.29s/it]
                                                           
{'loss': 1.1815, 'grad_norm': 18.963613510131836, 'learning_rate': 6.223174652343334e-06, 'epoch': 0.63}

 63%|██████▎   | 24090/37962 [55:48:26<31:56:31,  8.29s/it]
 63%|██████▎   | 24091/37962 [55:48:34<31:54:28,  8.28s/it]
 63%|██████▎   | 24092/37962 [55:48:43<31:48:43,  8.26s/it]
 63%|██████▎   | 24093/37962 [55:48:51<31:43:19,  8.23s/it]
 63%|██████▎   | 24094/37962 [55:48:59<31:56:53,  8.29s/it]
 63%|██████▎   | 24095/37962 [55:49:07<31:37:00,  8.21s/it]
 63%|██████▎   | 24096/37962 [55:49:15<31:41:45,  8.23s/it]
 63%|██████▎   | 24097/37962 [55:49:24<31:37:33,  8.21s/it]
 63%|██████▎   | 24098/37962 [55:49:32<31:45:45,  8.25s/it]
 63%|██████▎   | 24099/37962 [55:49:40<31:53:08,  8.28s/it]
 63%|██████▎   | 24100/37962 [55:49:49<31:49:40,  8.27s/it]
                                                           
{'loss': 1.2455, 'grad_norm': 11.102027893066406, 'learning_rate': 6.215276315527481e-06, 'epoch': 0.63}

 63%|██████▎   | 24100/37962 [55:49:49<31:49:40,  8.27s/it]
 63%|██████▎   | 24101/37962 [55:49:57<31:32:48,  8.19s/it]
 63%|██████▎   | 24102/37962 [55:50:05<31:38:48,  8.22s/it]
 63%|██████▎   | 24103/37962 [55:50:13<31:21:09,  8.14s/it]
 63%|██████▎   | 24104/37962 [55:50:21<31:25:30,  8.16s/it]
 63%|██████▎   | 24105/37962 [55:50:29<31:27:36,  8.17s/it]
 64%|██████▎   | 24106/37962 [55:50:37<31:34:22,  8.20s/it]
 64%|██████▎   | 24107/37962 [55:50:46<31:45:14,  8.25s/it]
 64%|██████▎   | 24108/37962 [55:50:54<31:12:43,  8.11s/it]
 64%|██████▎   | 24109/37962 [55:51:02<31:25:36,  8.17s/it]
 64%|██████▎   | 24110/37962 [55:51:10<31:34:01,  8.20s/it]
                                                           
{'loss': 1.2805, 'grad_norm': 11.724854469299316, 'learning_rate': 6.207380733546431e-06, 'epoch': 0.64}

 64%|██████▎   | 24110/37962 [55:51:10<31:34:01,  8.20s/it]
 64%|██████▎   | 24111/37962 [55:51:18<31:30:24,  8.19s/it]
 64%|██████▎   | 24112/37962 [55:51:27<31:58:10,  8.31s/it]
 64%|██████▎   | 24113/37962 [55:51:35<32:05:23,  8.34s/it]
 64%|██████▎   | 24114/37962 [55:51:44<31:59:36,  8.32s/it]
 64%|██████▎   | 24115/37962 [55:51:52<31:52:25,  8.29s/it]
 64%|██████▎   | 24116/37962 [55:52:00<31:56:00,  8.30s/it]
 64%|██████▎   | 24117/37962 [55:52:08<31:52:13,  8.29s/it]
 64%|██████▎   | 24118/37962 [55:52:17<31:48:20,  8.27s/it]
 64%|██████▎   | 24119/37962 [55:52:25<31:52:25,  8.29s/it]
 64%|██████▎   | 24120/37962 [55:52:33<32:02:56,  8.34s/it]
                                                           
{'loss': 1.2309, 'grad_norm': 12.711962699890137, 'learning_rate': 6.199487912147245e-06, 'epoch': 0.64}

 64%|██████▎   | 24120/37962 [55:52:33<32:02:56,  8.34s/it]
 64%|██████▎   | 24121/37962 [55:52:42<31:53:14,  8.29s/it]
 64%|██████▎   | 24122/37962 [55:52:50<32:01:13,  8.33s/it]
 64%|██████▎   | 24123/37962 [55:52:58<31:57:04,  8.31s/it]
 64%|██████▎   | 24124/37962 [55:53:06<31:38:32,  8.23s/it]
 64%|██████▎   | 24125/37962 [55:53:15<31:47:31,  8.27s/it]
 64%|██████▎   | 24126/37962 [55:53:23<31:16:20,  8.14s/it]
 64%|██████▎   | 24127/37962 [55:53:31<31:24:31,  8.17s/it]
 64%|██████▎   | 24128/37962 [55:53:39<31:28:44,  8.19s/it]
 64%|██████▎   | 24129/37962 [55:53:47<31:24:18,  8.17s/it]
 64%|██████▎   | 24130/37962 [55:53:56<31:52:07,  8.29s/it]
                                                           
{'loss': 1.3245, 'grad_norm': 11.612534523010254, 'learning_rate': 6.191597857074967e-06, 'epoch': 0.64}

 64%|██████▎   | 24130/37962 [55:53:56<31:52:07,  8.29s/it]
 64%|██████▎   | 24131/37962 [55:54:04<32:03:54,  8.35s/it]
 64%|██████▎   | 24132/37962 [55:54:12<32:00:12,  8.33s/it]
 64%|██████▎   | 24133/37962 [55:54:21<31:56:47,  8.32s/it]
 64%|██████▎   | 24134/37962 [55:54:29<32:02:04,  8.34s/it]
 64%|██████▎   | 24135/37962 [55:54:37<31:30:29,  8.20s/it]
 64%|██████▎   | 24136/37962 [55:54:45<31:36:08,  8.23s/it]
 64%|██████▎   | 24137/37962 [55:54:54<31:31:37,  8.21s/it]
 64%|██████▎   | 24138/37962 [55:55:02<31:40:55,  8.25s/it]
 64%|██████▎   | 24139/37962 [55:55:10<31:49:08,  8.29s/it]
 64%|██████▎   | 24140/37962 [55:55:19<31:51:02,  8.30s/it]
                                                           
{'loss': 1.2914, 'grad_norm': 71.12506866455078, 'learning_rate': 6.1837105740726324e-06, 'epoch': 0.64}

 64%|██████▎   | 24140/37962 [55:55:19<31:51:02,  8.30s/it]
 64%|██████▎   | 24141/37962 [55:55:27<31:56:03,  8.32s/it]
 64%|██████▎   | 24142/37962 [55:55:35<32:01:33,  8.34s/it]
 64%|██████▎   | 24143/37962 [55:55:44<31:51:53,  8.30s/it]
 64%|██████▎   | 24144/37962 [55:55:52<31:45:49,  8.28s/it]
 64%|██████▎   | 24145/37962 [55:56:00<31:47:11,  8.28s/it]
 64%|██████▎   | 24146/37962 [55:56:09<32:04:08,  8.36s/it]
 64%|██████▎   | 24147/37962 [55:56:17<31:55:06,  8.32s/it]
 64%|██████▎   | 24148/37962 [55:56:25<31:40:06,  8.25s/it]
 64%|██████▎   | 24149/37962 [55:56:33<31:41:23,  8.26s/it]
 64%|██████▎   | 24150/37962 [55:56:41<31:40:50,  8.26s/it]
                                                           
{'loss': 1.1951, 'grad_norm': 13.837928771972656, 'learning_rate': 6.175826068881258e-06, 'epoch': 0.64}

 64%|██████▎   | 24150/37962 [55:56:41<31:40:50,  8.26s/it]
 64%|██████▎   | 24151/37962 [55:56:50<31:30:06,  8.21s/it]
 64%|██████▎   | 24152/37962 [55:56:58<31:26:27,  8.20s/it]
 64%|██████▎   | 24153/37962 [55:57:06<31:29:26,  8.21s/it]
 64%|██████▎   | 24154/37962 [55:57:14<31:01:31,  8.09s/it]
 64%|██████▎   | 24155/37962 [55:57:22<31:16:40,  8.16s/it]
 64%|██████▎   | 24156/37962 [55:57:30<31:24:26,  8.19s/it]
 64%|██████▎   | 24157/37962 [55:57:39<31:43:46,  8.27s/it]
 64%|██████▎   | 24158/37962 [55:57:47<31:33:02,  8.23s/it]
 64%|██████▎   | 24159/37962 [55:57:55<31:34:23,  8.23s/it]
 64%|██████▎   | 24160/37962 [55:58:04<31:47:11,  8.29s/it]
                                                           
{'loss': 1.3062, 'grad_norm': 11.606605529785156, 'learning_rate': 6.1679443472398356e-06, 'epoch': 0.64}

 64%|██████▎   | 24160/37962 [55:58:04<31:47:11,  8.29s/it]
 64%|██████▎   | 24161/37962 [55:58:12<31:41:35,  8.27s/it]
 64%|██████▎   | 24162/37962 [55:58:20<31:43:58,  8.28s/it]
 64%|██████▎   | 24163/37962 [55:58:28<31:36:56,  8.25s/it]
 64%|██████▎   | 24164/37962 [55:58:37<31:42:59,  8.28s/it]
 64%|██████▎   | 24165/37962 [55:58:45<31:43:42,  8.28s/it]
 64%|██████▎   | 24166/37962 [55:58:53<31:55:29,  8.33s/it]
 64%|██████▎   | 24167/37962 [55:59:02<31:46:43,  8.29s/it]
 64%|██████▎   | 24168/37962 [55:59:10<31:39:05,  8.26s/it]
 64%|██████▎   | 24169/37962 [55:59:18<31:43:25,  8.28s/it]
 64%|██████▎   | 24170/37962 [55:59:26<31:42:40,  8.28s/it]
                                                           
{'loss': 1.3061, 'grad_norm': 17.915681838989258, 'learning_rate': 6.160065414885346e-06, 'epoch': 0.64}

 64%|██████▎   | 24170/37962 [55:59:26<31:42:40,  8.28s/it]
 64%|██████▎   | 24171/37962 [55:59:35<31:47:05,  8.30s/it]
 64%|██████▎   | 24172/37962 [55:59:43<31:47:30,  8.30s/it]
 64%|██████▎   | 24173/37962 [55:59:51<31:44:36,  8.29s/it]
 64%|██████▎   | 24174/37962 [55:59:59<31:42:14,  8.28s/it]
 64%|██████▎   | 24175/37962 [56:00:08<31:58:29,  8.35s/it]
 64%|██████▎   | 24176/37962 [56:00:16<31:58:36,  8.35s/it]
 64%|██████▎   | 24177/37962 [56:00:25<31:51:10,  8.32s/it]
 64%|██████▎   | 24178/37962 [56:00:33<31:47:05,  8.30s/it]
 64%|██████▎   | 24179/37962 [56:00:41<31:54:55,  8.34s/it]
 64%|██████▎   | 24180/37962 [56:00:50<31:47:47,  8.31s/it]
                                                           
{'loss': 1.2559, 'grad_norm': 27.647504806518555, 'learning_rate': 6.152189277552711e-06, 'epoch': 0.64}

 64%|██████▎   | 24180/37962 [56:00:50<31:47:47,  8.31s/it]
 64%|██████▎   | 24181/37962 [56:00:58<31:41:01,  8.28s/it]
 64%|██████▎   | 24182/37962 [56:01:06<31:46:28,  8.30s/it]
 64%|██████▎   | 24183/37962 [56:01:15<31:54:57,  8.34s/it]
 64%|██████▎   | 24184/37962 [56:01:23<31:32:04,  8.24s/it]
 64%|██████▎   | 24185/37962 [56:01:31<31:33:10,  8.24s/it]
 64%|██████▎   | 24186/37962 [56:01:39<31:34:49,  8.25s/it]
 64%|██████▎   | 24187/37962 [56:01:47<31:37:32,  8.27s/it]
 64%|██████▎   | 24188/37962 [56:01:56<31:35:14,  8.26s/it]
 64%|██████▎   | 24189/37962 [56:02:04<31:37:22,  8.27s/it]
 64%|██████▎   | 24190/37962 [56:02:12<31:31:18,  8.24s/it]
                                                           
{'loss': 1.2777, 'grad_norm': 20.00558853149414, 'learning_rate': 6.144315940974843e-06, 'epoch': 0.64}

 64%|██████▎   | 24190/37962 [56:02:12<31:31:18,  8.24s/it]
 64%|██████▎   | 24191/37962 [56:02:20<31:30:54,  8.24s/it]
 64%|██████▎   | 24192/37962 [56:02:29<31:46:01,  8.31s/it]
 64%|██████▎   | 24193/37962 [56:02:37<31:51:31,  8.33s/it]
 64%|██████▎   | 24194/37962 [56:02:45<31:53:42,  8.34s/it]
 64%|██████▎   | 24195/37962 [56:02:53<31:24:00,  8.21s/it]
 64%|██████▎   | 24196/37962 [56:03:02<31:32:22,  8.25s/it]
 64%|██████▎   | 24197/37962 [56:03:10<31:12:24,  8.16s/it]
 64%|██████▎   | 24198/37962 [56:03:18<31:00:03,  8.11s/it]
 64%|██████▎   | 24199/37962 [56:03:26<31:15:03,  8.17s/it]
 64%|██████▎   | 24200/37962 [56:03:35<31:40:50,  8.29s/it]
                                                           
{'loss': 1.2114, 'grad_norm': 7.387813091278076, 'learning_rate': 6.136445410882608e-06, 'epoch': 0.64}

 64%|██████▎   | 24200/37962 [56:03:35<31:40:50,  8.29s/it]
 64%|██████▍   | 24201/37962 [56:03:43<31:34:29,  8.26s/it]
 64%|██████▍   | 24202/37962 [56:03:51<31:37:57,  8.28s/it]
 64%|██████▍   | 24203/37962 [56:03:59<31:25:04,  8.22s/it]
 64%|██████▍   | 24204/37962 [56:04:07<31:05:00,  8.13s/it]
 64%|██████▍   | 24205/37962 [56:04:15<30:51:28,  8.08s/it]
 64%|██████▍   | 24206/37962 [56:04:23<31:04:58,  8.13s/it]
 64%|██████▍   | 24207/37962 [56:04:32<31:09:54,  8.16s/it]
 64%|██████▍   | 24208/37962 [56:04:40<31:09:35,  8.16s/it]
 64%|██████▍   | 24209/37962 [56:04:48<31:03:01,  8.13s/it]
 64%|██████▍   | 24210/37962 [56:04:56<31:11:24,  8.16s/it]
                                                           
{'loss': 1.2635, 'grad_norm': 20.884267807006836, 'learning_rate': 6.128577693004826e-06, 'epoch': 0.64}

 64%|██████▍   | 24210/37962 [56:04:56<31:11:24,  8.16s/it]
 64%|██████▍   | 24211/37962 [56:05:04<31:21:36,  8.21s/it]
 64%|██████▍   | 24212/37962 [56:05:13<31:33:50,  8.26s/it]
 64%|██████▍   | 24213/37962 [56:05:21<31:28:32,  8.24s/it]
 64%|██████▍   | 24214/37962 [56:05:29<31:27:47,  8.24s/it]
 64%|██████▍   | 24215/37962 [56:05:37<31:28:47,  8.24s/it]
 64%|██████▍   | 24216/37962 [56:05:46<31:28:33,  8.24s/it]
 64%|██████▍   | 24217/37962 [56:05:54<31:44:01,  8.31s/it]
 64%|██████▍   | 24218/37962 [56:06:02<31:34:19,  8.27s/it]
 64%|██████▍   | 24219/37962 [56:06:11<31:41:28,  8.30s/it]
 64%|██████▍   | 24220/37962 [56:06:19<31:38:33,  8.29s/it]
                                                           
{'loss': 1.1912, 'grad_norm': 433.44134521484375, 'learning_rate': 6.120712793068273e-06, 'epoch': 0.64}

 64%|██████▍   | 24220/37962 [56:06:19<31:38:33,  8.29s/it]
 64%|██████▍   | 24221/37962 [56:06:27<31:44:13,  8.31s/it]
 64%|██████▍   | 24222/37962 [56:06:36<31:49:17,  8.34s/it]
 64%|██████▍   | 24223/37962 [56:06:44<31:42:04,  8.31s/it]
 64%|██████▍   | 24224/37962 [56:06:52<31:38:04,  8.29s/it]
 64%|██████▍   | 24225/37962 [56:07:01<31:48:36,  8.34s/it]
 64%|██████▍   | 24226/37962 [56:07:09<31:47:24,  8.33s/it]
 64%|██████▍   | 24227/37962 [56:07:17<31:55:08,  8.37s/it]
 64%|██████▍   | 24228/37962 [56:07:26<32:03:54,  8.41s/it]
 64%|██████▍   | 24229/37962 [56:07:34<31:37:25,  8.29s/it]
 64%|██████▍   | 24230/37962 [56:07:42<31:40:12,  8.30s/it]
                                                           
{'loss': 1.3594, 'grad_norm': 6.703336238861084, 'learning_rate': 6.112850716797678e-06, 'epoch': 0.64}

 64%|██████▍   | 24230/37962 [56:07:42<31:40:12,  8.30s/it]
 64%|██████▍   | 24231/37962 [56:07:51<31:39:48,  8.30s/it]
 64%|██████▍   | 24232/37962 [56:07:59<31:41:05,  8.31s/it]
 64%|██████▍   | 24233/37962 [56:08:07<31:39:35,  8.30s/it]
 64%|██████▍   | 24234/37962 [56:08:15<31:43:23,  8.32s/it]
 64%|██████▍   | 24235/37962 [56:08:24<32:06:05,  8.42s/it]
 64%|██████▍   | 24236/37962 [56:08:32<31:46:25,  8.33s/it]
 64%|██████▍   | 24237/37962 [56:08:40<31:29:20,  8.26s/it]
 64%|██████▍   | 24238/37962 [56:08:49<31:30:02,  8.26s/it]
 64%|██████▍   | 24239/37962 [56:08:57<31:13:10,  8.19s/it]
 64%|██████▍   | 24240/37962 [56:09:05<31:24:25,  8.24s/it]
                                                           
{'loss': 1.1724, 'grad_norm': 104.52894592285156, 'learning_rate': 6.104991469915702e-06, 'epoch': 0.64}

 64%|██████▍   | 24240/37962 [56:09:05<31:24:25,  8.24s/it]
 64%|██████▍   | 24241/37962 [56:09:20<39:29:44, 10.36s/it]
 64%|██████▍   | 24242/37962 [56:09:28<36:58:07,  9.70s/it]
 64%|██████▍   | 24243/37962 [56:09:36<34:50:55,  9.14s/it]
 64%|██████▍   | 24244/37962 [56:09:45<33:53:16,  8.89s/it]
 64%|██████▍   | 24245/37962 [56:09:53<33:10:25,  8.71s/it]
 64%|██████▍   | 24246/37962 [56:10:01<32:32:21,  8.54s/it]
 64%|██████▍   | 24247/37962 [56:10:09<31:48:32,  8.35s/it]
 64%|██████▍   | 24248/37962 [56:10:17<31:44:32,  8.33s/it]
 64%|██████▍   | 24249/37962 [56:10:26<31:48:23,  8.35s/it]
 64%|██████▍   | 24250/37962 [56:10:34<31:46:00,  8.34s/it]
                                                           
{'loss': 1.268, 'grad_norm': 35.40473175048828, 'learning_rate': 6.09713505814296e-06, 'epoch': 0.64}

 64%|██████▍   | 24250/37962 [56:10:34<31:46:00,  8.34s/it]
 64%|██████▍   | 24251/37962 [56:10:42<31:39:10,  8.31s/it]
 64%|██████▍   | 24252/37962 [56:10:51<31:44:01,  8.33s/it]
 64%|██████▍   | 24253/37962 [56:10:59<31:28:33,  8.27s/it]
 64%|██████▍   | 24254/37962 [56:11:07<31:18:32,  8.22s/it]
 64%|██████▍   | 24255/37962 [56:11:15<31:23:21,  8.24s/it]
 64%|██████▍   | 24256/37962 [56:11:23<31:11:46,  8.19s/it]
 64%|██████▍   | 24257/37962 [56:11:32<31:30:03,  8.27s/it]
 64%|██████▍   | 24258/37962 [56:11:40<31:40:42,  8.32s/it]
 64%|██████▍   | 24259/37962 [56:11:48<31:33:34,  8.29s/it]
 64%|██████▍   | 24260/37962 [56:11:57<31:35:04,  8.30s/it]
                                                           
{'loss': 1.2303, 'grad_norm': 4.9983439445495605, 'learning_rate': 6.089281487197994e-06, 'epoch': 0.64}

 64%|██████▍   | 24260/37962 [56:11:57<31:35:04,  8.30s/it]
 64%|██████▍   | 24261/37962 [56:12:05<31:46:40,  8.35s/it]
 64%|██████▍   | 24262/37962 [56:12:13<31:38:02,  8.31s/it]
 64%|██████▍   | 24263/37962 [56:12:22<31:35:56,  8.30s/it]
 64%|██████▍   | 24264/37962 [56:12:30<31:33:09,  8.29s/it]
 64%|██████▍   | 24265/37962 [56:12:38<31:06:27,  8.18s/it]
 64%|██████▍   | 24266/37962 [56:12:46<31:08:51,  8.19s/it]
 64%|██████▍   | 24267/37962 [56:12:54<31:11:35,  8.20s/it]
 64%|██████▍   | 24268/37962 [56:13:02<31:18:01,  8.23s/it]
 64%|██████▍   | 24269/37962 [56:13:11<31:13:22,  8.21s/it]
 64%|██████▍   | 24270/37962 [56:13:19<31:29:51,  8.28s/it]
                                                           
{'loss': 1.299, 'grad_norm': 21.454208374023438, 'learning_rate': 6.081430762797287e-06, 'epoch': 0.64}

 64%|██████▍   | 24270/37962 [56:13:19<31:29:51,  8.28s/it]
 64%|██████▍   | 24271/37962 [56:13:27<31:36:24,  8.31s/it]
 64%|██████▍   | 24272/37962 [56:13:36<31:30:35,  8.29s/it]
 64%|██████▍   | 24273/37962 [56:13:44<31:29:04,  8.28s/it]
 64%|██████▍   | 24274/37962 [56:13:52<31:36:54,  8.31s/it]
 64%|██████▍   | 24275/37962 [56:14:01<31:31:36,  8.29s/it]
 64%|██████▍   | 24276/37962 [56:14:09<31:36:33,  8.31s/it]
 64%|██████▍   | 24277/37962 [56:14:17<31:30:56,  8.29s/it]
 64%|██████▍   | 24278/37962 [56:14:25<31:25:35,  8.27s/it]
 64%|██████▍   | 24279/37962 [56:14:34<31:22:53,  8.26s/it]
 64%|██████▍   | 24280/37962 [56:14:42<31:22:25,  8.26s/it]
                                                           
{'loss': 1.306, 'grad_norm': 4.333522796630859, 'learning_rate': 6.073582890655246e-06, 'epoch': 0.64}

 64%|██████▍   | 24280/37962 [56:14:42<31:22:25,  8.26s/it]
 64%|██████▍   | 24281/37962 [56:14:50<31:20:47,  8.25s/it]
 64%|██████▍   | 24282/37962 [56:14:59<31:32:49,  8.30s/it]
 64%|██████▍   | 24283/37962 [56:15:07<31:32:47,  8.30s/it]
 64%|██████▍   | 24284/37962 [56:15:15<31:15:08,  8.23s/it]
 64%|██████▍   | 24285/37962 [56:15:23<31:22:16,  8.26s/it]
 64%|██████▍   | 24286/37962 [56:15:32<31:25:34,  8.27s/it]
 64%|██████▍   | 24287/37962 [56:15:40<31:17:38,  8.24s/it]
 64%|██████▍   | 24288/37962 [56:15:48<31:26:49,  8.28s/it]
 64%|██████▍   | 24289/37962 [56:15:56<31:27:09,  8.28s/it]
 64%|██████▍   | 24290/37962 [56:16:05<31:25:11,  8.27s/it]
                                                           
{'loss': 1.2388, 'grad_norm': 44.91768264770508, 'learning_rate': 6.0657378764841945e-06, 'epoch': 0.64}

 64%|██████▍   | 24290/37962 [56:16:05<31:25:11,  8.27s/it]
 64%|██████▍   | 24291/37962 [56:16:13<31:20:00,  8.25s/it]
 64%|██████▍   | 24292/37962 [56:16:21<31:20:42,  8.25s/it]
 64%|██████▍   | 24293/37962 [56:16:29<31:16:43,  8.24s/it]
 64%|██████▍   | 24294/37962 [56:16:38<31:35:32,  8.32s/it]
 64%|██████▍   | 24295/37962 [56:16:46<31:29:59,  8.30s/it]
 64%|██████▍   | 24296/37962 [56:16:55<31:42:03,  8.35s/it]
 64%|██████▍   | 24297/37962 [56:17:03<31:23:05,  8.27s/it]
 64%|██████▍   | 24298/37962 [56:17:11<31:33:40,  8.32s/it]
 64%|██████▍   | 24299/37962 [56:17:19<31:27:43,  8.29s/it]
 64%|██████▍   | 24300/37962 [56:17:27<31:21:31,  8.26s/it]
                                                           
{'loss': 1.257, 'grad_norm': 58.400604248046875, 'learning_rate': 6.057895725994387e-06, 'epoch': 0.64}

 64%|██████▍   | 24300/37962 [56:17:27<31:21:31,  8.26s/it]
 64%|██████▍   | 24301/37962 [56:17:36<31:16:22,  8.24s/it]
 64%|██████▍   | 24302/37962 [56:17:44<31:15:49,  8.24s/it]
 64%|██████▍   | 24303/37962 [56:17:52<31:18:03,  8.25s/it]
 64%|██████▍   | 24304/37962 [56:18:01<31:29:35,  8.30s/it]
 64%|██████▍   | 24305/37962 [56:18:09<31:29:08,  8.30s/it]
 64%|██████▍   | 24306/37962 [56:18:17<31:22:49,  8.27s/it]
 64%|██████▍   | 24307/37962 [56:18:25<31:21:07,  8.27s/it]
 64%|██████▍   | 24308/37962 [56:18:33<31:07:39,  8.21s/it]
 64%|██████▍   | 24309/37962 [56:18:42<31:23:01,  8.28s/it]
 64%|██████▍   | 24310/37962 [56:18:50<31:11:16,  8.22s/it]
                                                           
{'loss': 1.2417, 'grad_norm': 13.313076972961426, 'learning_rate': 6.050056444893988e-06, 'epoch': 0.64}

 64%|██████▍   | 24310/37962 [56:18:50<31:11:16,  8.22s/it]
 64%|██████▍   | 24311/37962 [56:18:58<31:15:13,  8.24s/it]
 64%|██████▍   | 24312/37962 [56:19:06<31:07:52,  8.21s/it]
 64%|██████▍   | 24313/37962 [56:19:15<31:17:12,  8.25s/it]
 64%|██████▍   | 24314/37962 [56:19:23<31:09:56,  8.22s/it]
 64%|██████▍   | 24315/37962 [56:19:31<31:13:20,  8.24s/it]
 64%|██████▍   | 24316/37962 [56:19:40<31:32:48,  8.32s/it]
 64%|██████▍   | 24317/37962 [56:19:48<31:07:48,  8.21s/it]
 64%|██████▍   | 24318/37962 [56:19:56<31:14:12,  8.24s/it]
 64%|██████▍   | 24319/37962 [56:20:04<31:10:07,  8.22s/it]
 64%|██████▍   | 24320/37962 [56:20:12<30:57:47,  8.17s/it]
                                                           
{'loss': 1.3466, 'grad_norm': 20.55388832092285, 'learning_rate': 6.042220038889075e-06, 'epoch': 0.64}

 64%|██████▍   | 24320/37962 [56:20:12<30:57:47,  8.17s/it]
 64%|██████▍   | 24321/37962 [56:20:20<31:06:05,  8.21s/it]
 64%|██████▍   | 24322/37962 [56:20:29<31:16:32,  8.25s/it]
 64%|██████▍   | 24323/37962 [56:20:37<31:05:32,  8.21s/it]
 64%|██████▍   | 24324/37962 [56:20:46<31:32:03,  8.32s/it]
 64%|██████▍   | 24325/37962 [56:20:54<31:37:56,  8.35s/it]
 64%|██████▍   | 24326/37962 [56:21:02<31:10:13,  8.23s/it]
 64%|██████▍   | 24327/37962 [56:21:10<31:14:52,  8.25s/it]
 64%|██████▍   | 24328/37962 [56:21:26<39:53:32, 10.53s/it]
 64%|██████▍   | 24329/37962 [56:21:34<37:25:19,  9.88s/it]
 64%|██████▍   | 24330/37962 [56:21:43<35:49:22,  9.46s/it]
                                                           
{'loss': 1.1568, 'grad_norm': 6.284143447875977, 'learning_rate': 6.034386513683634e-06, 'epoch': 0.64}

 64%|██████▍   | 24330/37962 [56:21:43<35:49:22,  9.46s/it]
 64%|██████▍   | 24331/37962 [56:21:51<34:33:03,  9.13s/it]
 64%|██████▍   | 24332/37962 [56:21:59<33:35:48,  8.87s/it]
 64%|██████▍   | 24333/37962 [56:22:08<33:10:01,  8.76s/it]
 64%|██████▍   | 24334/37962 [56:22:16<32:32:21,  8.60s/it]
 64%|██████▍   | 24335/37962 [56:22:25<32:13:12,  8.51s/it]
 64%|██████▍   | 24336/37962 [56:22:33<31:43:01,  8.38s/it]
 64%|██████▍   | 24337/37962 [56:22:41<31:40:01,  8.37s/it]
 64%|██████▍   | 24338/37962 [56:22:49<31:41:16,  8.37s/it]
 64%|██████▍   | 24339/37962 [56:22:57<31:03:44,  8.21s/it]
 64%|██████▍   | 24340/37962 [56:23:05<31:12:05,  8.25s/it]
                                                           
{'loss': 1.2753, 'grad_norm': 17.66733741760254, 'learning_rate': 6.026555874979549e-06, 'epoch': 0.64}

 64%|██████▍   | 24340/37962 [56:23:05<31:12:05,  8.25s/it]
 64%|██████▍   | 24341/37962 [56:23:14<31:24:44,  8.30s/it]
 64%|██████▍   | 24342/37962 [56:23:22<31:02:33,  8.21s/it]
 64%|██████▍   | 24343/37962 [56:23:30<30:59:39,  8.19s/it]
 64%|██████▍   | 24344/37962 [56:23:38<31:14:08,  8.26s/it]
 64%|██████▍   | 24345/37962 [56:23:46<30:51:37,  8.16s/it]
 64%|██████▍   | 24346/37962 [56:23:54<30:28:22,  8.06s/it]
 64%|██████▍   | 24347/37962 [56:24:02<30:29:47,  8.06s/it]
 64%|██████▍   | 24348/37962 [56:24:11<30:48:47,  8.15s/it]
 64%|██████▍   | 24349/37962 [56:24:19<30:41:47,  8.12s/it]
 64%|██████▍   | 24350/37962 [56:24:27<30:54:04,  8.17s/it]
                                                           
{'loss': 1.3009, 'grad_norm': 66.92606353759766, 'learning_rate': 6.018728128476606e-06, 'epoch': 0.64}

 64%|██████▍   | 24350/37962 [56:24:27<30:54:04,  8.17s/it]
 64%|██████▍   | 24351/37962 [56:24:35<30:52:57,  8.17s/it]
 64%|██████▍   | 24352/37962 [56:24:43<31:02:49,  8.21s/it]
 64%|██████▍   | 24353/37962 [56:24:52<31:04:24,  8.22s/it]
 64%|██████▍   | 24354/37962 [56:25:00<31:15:02,  8.27s/it]
 64%|██████▍   | 24355/37962 [56:25:08<31:25:26,  8.31s/it]
 64%|██████▍   | 24356/37962 [56:25:17<31:24:56,  8.31s/it]
 64%|██████▍   | 24357/37962 [56:25:25<31:27:52,  8.33s/it]
 64%|██████▍   | 24358/37962 [56:25:34<31:38:52,  8.37s/it]
 64%|██████▍   | 24359/37962 [56:25:42<31:30:34,  8.34s/it]
 64%|██████▍   | 24360/37962 [56:25:50<31:26:33,  8.32s/it]
                                                           
{'loss': 1.2513, 'grad_norm': 58.176361083984375, 'learning_rate': 6.010903279872489e-06, 'epoch': 0.64}

 64%|██████▍   | 24360/37962 [56:25:50<31:26:33,  8.32s/it]
 64%|██████▍   | 24361/37962 [56:25:58<30:54:27,  8.18s/it]
 64%|██████▍   | 24362/37962 [56:26:06<30:57:40,  8.20s/it]
 64%|██████▍   | 24363/37962 [56:26:15<31:07:21,  8.24s/it]
 64%|██████▍   | 24364/37962 [56:26:23<31:15:26,  8.28s/it]
 64%|██████▍   | 24365/37962 [56:26:31<30:50:17,  8.16s/it]
 64%|██████▍   | 24366/37962 [56:26:39<30:59:14,  8.20s/it]
 64%|██████▍   | 24367/37962 [56:26:47<30:42:51,  8.13s/it]
 64%|██████▍   | 24368/37962 [56:26:55<30:47:27,  8.15s/it]
 64%|██████▍   | 24369/37962 [56:27:04<30:55:43,  8.19s/it]
 64%|██████▍   | 24370/37962 [56:27:12<31:09:41,  8.25s/it]
                                                           
{'loss': 1.1387, 'grad_norm': 7.7237420082092285, 'learning_rate': 6.0030813348627645e-06, 'epoch': 0.64}

 64%|██████▍   | 24370/37962 [56:27:12<31:09:41,  8.25s/it]
 64%|██████▍   | 24371/37962 [56:27:20<31:13:18,  8.27s/it]
 64%|██████▍   | 24372/37962 [56:27:28<30:57:25,  8.20s/it]
 64%|██████▍   | 24373/37962 [56:27:37<31:02:07,  8.22s/it]
 64%|██████▍   | 24374/37962 [56:27:45<30:40:05,  8.13s/it]
 64%|██████▍   | 24375/37962 [56:27:53<31:05:15,  8.24s/it]
 64%|██████▍   | 24376/37962 [56:28:01<31:13:24,  8.27s/it]
 64%|██████▍   | 24377/37962 [56:28:10<31:12:59,  8.27s/it]
 64%|██████▍   | 24378/37962 [56:28:18<31:14:27,  8.28s/it]
 64%|██████▍   | 24379/37962 [56:28:26<31:04:09,  8.23s/it]
 64%|██████▍   | 24380/37962 [56:28:34<31:12:52,  8.27s/it]
                                                           
{'loss': 1.1578, 'grad_norm': 109.33341217041016, 'learning_rate': 5.995262299140901e-06, 'epoch': 0.64}

 64%|██████▍   | 24380/37962 [56:28:34<31:12:52,  8.27s/it]
 64%|██████▍   | 24381/37962 [56:28:43<30:58:30,  8.21s/it]
 64%|██████▍   | 24382/37962 [56:28:51<31:21:07,  8.31s/it]
 64%|██████▍   | 24383/37962 [56:28:59<31:21:24,  8.31s/it]
 64%|██████▍   | 24384/37962 [56:29:08<31:32:07,  8.36s/it]
 64%|██████▍   | 24385/37962 [56:29:16<30:48:30,  8.17s/it]
 64%|██████▍   | 24386/37962 [56:29:24<31:16:26,  8.29s/it]
 64%|██████▍   | 24387/37962 [56:29:32<31:13:08,  8.28s/it]
 64%|██████▍   | 24388/37962 [56:29:41<31:20:14,  8.31s/it]
 64%|██████▍   | 24389/37962 [56:29:49<30:53:37,  8.19s/it]
 64%|██████▍   | 24390/37962 [56:29:57<30:53:05,  8.19s/it]
                                                           
{'loss': 1.3084, 'grad_norm': 6.186466217041016, 'learning_rate': 5.9874461783982275e-06, 'epoch': 0.64}

 64%|██████▍   | 24390/37962 [56:29:57<30:53:05,  8.19s/it]
 64%|██████▍   | 24391/37962 [56:30:06<31:22:49,  8.32s/it]
 64%|██████▍   | 24392/37962 [56:30:14<31:21:11,  8.32s/it]
 64%|██████▍   | 24393/37962 [56:30:22<30:59:23,  8.22s/it]
 64%|██████▍   | 24394/37962 [56:30:30<31:05:53,  8.25s/it]
 64%|██████▍   | 24395/37962 [56:30:39<31:14:34,  8.29s/it]
 64%|██████▍   | 24396/37962 [56:30:47<31:12:40,  8.28s/it]
 64%|██████▍   | 24397/37962 [56:30:55<31:18:11,  8.31s/it]
 64%|██████▍   | 24398/37962 [56:31:04<31:49:24,  8.45s/it]
 64%|██████▍   | 24399/37962 [56:31:12<31:41:11,  8.41s/it]
 64%|██████▍   | 24400/37962 [56:31:21<31:32:21,  8.37s/it]
                                                           
{'loss': 1.2303, 'grad_norm': 22.49488639831543, 'learning_rate': 5.979632978323969e-06, 'epoch': 0.64}

 64%|██████▍   | 24400/37962 [56:31:21<31:32:21,  8.37s/it]
 64%|██████▍   | 24401/37962 [56:31:28<31:03:59,  8.25s/it]
 64%|██████▍   | 24402/37962 [56:31:36<30:42:53,  8.15s/it]
 64%|██████▍   | 24403/37962 [56:31:45<30:38:27,  8.14s/it]
 64%|██████▍   | 24404/37962 [56:31:53<30:42:53,  8.16s/it]
 64%|██████▍   | 24405/37962 [56:32:01<30:54:39,  8.21s/it]
 64%|██████▍   | 24406/37962 [56:32:09<30:46:45,  8.17s/it]
 64%|██████▍   | 24407/37962 [56:32:17<30:54:18,  8.21s/it]
 64%|██████▍   | 24408/37962 [56:32:26<30:45:55,  8.17s/it]
 64%|██████▍   | 24409/37962 [56:32:34<30:57:15,  8.22s/it]
 64%|██████▍   | 24410/37962 [56:32:42<31:05:03,  8.26s/it]
                                                           
{'loss': 1.2891, 'grad_norm': 10.102582931518555, 'learning_rate': 5.971822704605219e-06, 'epoch': 0.64}

 64%|██████▍   | 24410/37962 [56:32:42<31:05:03,  8.26s/it]
 64%|██████▍   | 24411/37962 [56:32:51<31:14:12,  8.30s/it]
 64%|██████▍   | 24412/37962 [56:32:59<31:18:01,  8.32s/it]
 64%|██████▍   | 24413/37962 [56:33:07<31:00:31,  8.24s/it]
 64%|██████▍   | 24414/37962 [56:33:15<30:59:13,  8.23s/it]
 64%|██████▍   | 24415/37962 [56:33:24<31:06:08,  8.27s/it]
 64%|██████▍   | 24416/37962 [56:33:32<31:13:38,  8.30s/it]
 64%|██████▍   | 24417/37962 [56:33:40<31:08:24,  8.28s/it]
 64%|██████▍   | 24418/37962 [56:33:49<31:12:47,  8.30s/it]
 64%|██████▍   | 24419/37962 [56:33:57<30:53:03,  8.21s/it]
 64%|██████▍   | 24420/37962 [56:34:05<31:03:10,  8.26s/it]
                                                           
{'loss': 1.1173, 'grad_norm': 11.14193058013916, 'learning_rate': 5.964015362926937e-06, 'epoch': 0.64}

 64%|██████▍   | 24420/37962 [56:34:05<31:03:10,  8.26s/it]
 64%|██████▍   | 24421/37962 [56:34:13<30:28:46,  8.10s/it]
 64%|██████▍   | 24422/37962 [56:34:21<30:40:02,  8.15s/it]
 64%|██████▍   | 24423/37962 [56:34:29<30:44:59,  8.18s/it]
 64%|██████▍   | 24424/37962 [56:34:37<30:53:14,  8.21s/it]
 64%|██████▍   | 24425/37962 [56:34:46<30:54:29,  8.22s/it]
 64%|██████▍   | 24426/37962 [56:34:54<30:58:57,  8.24s/it]
 64%|██████▍   | 24427/37962 [56:35:02<30:41:55,  8.17s/it]
 64%|██████▍   | 24428/37962 [56:35:10<30:25:21,  8.09s/it]
 64%|██████▍   | 24429/37962 [56:35:18<30:43:51,  8.17s/it]
 64%|██████▍   | 24430/37962 [56:35:26<30:29:05,  8.11s/it]
                                                           
{'loss': 1.2933, 'grad_norm': 14.617886543273926, 'learning_rate': 5.956210958971955e-06, 'epoch': 0.64}

 64%|██████▍   | 24430/37962 [56:35:26<30:29:05,  8.11s/it]
 64%|██████▍   | 24431/37962 [56:35:35<30:52:08,  8.21s/it]
 64%|██████▍   | 24432/37962 [56:35:43<30:59:51,  8.25s/it]
 64%|██████▍   | 24433/37962 [56:35:51<30:56:02,  8.23s/it]
 64%|██████▍   | 24434/37962 [56:35:59<31:01:59,  8.26s/it]
 64%|██████▍   | 24435/37962 [56:36:08<31:15:05,  8.32s/it]
 64%|██████▍   | 24436/37962 [56:36:16<31:06:36,  8.28s/it]
 64%|██████▍   | 24437/37962 [56:36:24<30:59:39,  8.25s/it]
 64%|██████▍   | 24438/37962 [56:36:32<30:43:50,  8.18s/it]
 64%|██████▍   | 24439/37962 [56:36:40<30:30:00,  8.12s/it]
 64%|██████▍   | 24440/37962 [56:36:48<30:27:31,  8.11s/it]
                                                           
{'loss': 1.1829, 'grad_norm': 8.63116455078125, 'learning_rate': 5.9484094984209635e-06, 'epoch': 0.64}

 64%|██████▍   | 24440/37962 [56:36:48<30:27:31,  8.11s/it]
 64%|██████▍   | 24441/37962 [56:36:57<30:37:56,  8.16s/it]
 64%|██████▍   | 24442/37962 [56:37:05<30:45:19,  8.19s/it]
 64%|██████▍   | 24443/37962 [56:37:13<30:56:07,  8.24s/it]
 64%|██████▍   | 24444/37962 [56:37:21<30:26:44,  8.11s/it]
 64%|██████▍   | 24445/37962 [56:37:29<30:44:38,  8.19s/it]
 64%|██████▍   | 24446/37962 [56:37:38<30:54:47,  8.23s/it]
 64%|██████▍   | 24447/37962 [56:37:46<30:56:19,  8.24s/it]
 64%|██████▍   | 24448/37962 [56:37:54<30:56:05,  8.24s/it]
 64%|██████▍   | 24449/37962 [56:38:03<31:04:07,  8.28s/it]
 64%|██████▍   | 24450/37962 [56:38:11<30:56:24,  8.24s/it]
                                                           
{'loss': 1.2726, 'grad_norm': 15.653342247009277, 'learning_rate': 5.9406109869525065e-06, 'epoch': 0.64}

 64%|██████▍   | 24450/37962 [56:38:11<30:56:24,  8.24s/it]
 64%|██████▍   | 24451/37962 [56:38:19<30:53:29,  8.23s/it]
 64%|██████▍   | 24452/37962 [56:38:27<30:40:35,  8.17s/it]
 64%|██████▍   | 24453/37962 [56:38:36<30:57:53,  8.25s/it]
 64%|██████▍   | 24454/37962 [56:38:44<30:55:16,  8.24s/it]
 64%|██████▍   | 24455/37962 [56:38:52<31:10:35,  8.31s/it]
 64%|██████▍   | 24456/37962 [56:39:00<30:52:27,  8.23s/it]
 64%|██████▍   | 24457/37962 [56:39:09<31:12:41,  8.32s/it]
 64%|██████▍   | 24458/37962 [56:39:17<31:15:21,  8.33s/it]
 64%|██████▍   | 24459/37962 [56:39:25<31:08:05,  8.30s/it]
 64%|██████▍   | 24460/37962 [56:39:34<30:58:06,  8.26s/it]
                                                           
{'loss': 1.297, 'grad_norm': 10.092753410339355, 'learning_rate': 5.932815430242989e-06, 'epoch': 0.64}

 64%|██████▍   | 24460/37962 [56:39:34<30:58:06,  8.26s/it]
 64%|██████▍   | 24461/37962 [56:39:42<31:04:52,  8.29s/it]
 64%|██████▍   | 24462/37962 [56:39:50<30:53:33,  8.24s/it]
 64%|██████▍   | 24463/37962 [56:39:58<30:49:59,  8.22s/it]
 64%|██████▍   | 24464/37962 [56:40:07<30:59:23,  8.27s/it]
 64%|██████▍   | 24465/37962 [56:40:15<30:55:13,  8.25s/it]
 64%|██████▍   | 24466/37962 [56:40:23<31:03:40,  8.29s/it]
 64%|██████▍   | 24467/37962 [56:40:31<31:08:29,  8.31s/it]
 64%|██████▍   | 24468/37962 [56:40:40<31:11:47,  8.32s/it]
 64%|██████▍   | 24469/37962 [56:40:48<30:41:14,  8.19s/it]
 64%|██████▍   | 24470/37962 [56:40:56<30:36:09,  8.17s/it]
                                                           
{'loss': 1.309, 'grad_norm': 13.414009094238281, 'learning_rate': 5.925022833966658e-06, 'epoch': 0.64}

 64%|██████▍   | 24470/37962 [56:40:56<30:36:09,  8.17s/it]
 64%|██████▍   | 24471/37962 [56:41:04<30:20:02,  8.09s/it]
 64%|██████▍   | 24472/37962 [56:41:12<30:27:17,  8.13s/it]
 64%|██████▍   | 24473/37962 [56:41:21<30:59:25,  8.27s/it]
 64%|██████▍   | 24474/37962 [56:41:29<30:54:56,  8.25s/it]
 64%|██████▍   | 24475/37962 [56:41:37<30:48:34,  8.22s/it]
 64%|██████▍   | 24476/37962 [56:41:45<30:48:50,  8.23s/it]
 64%|██████▍   | 24477/37962 [56:41:54<30:57:47,  8.27s/it]
 64%|██████▍   | 24478/37962 [56:42:02<31:01:36,  8.28s/it]
 64%|██████▍   | 24479/37962 [56:42:10<31:05:22,  8.30s/it]
 64%|██████▍   | 24480/37962 [56:42:18<31:00:23,  8.28s/it]
                                                           
{'loss': 1.2707, 'grad_norm': 9.195425987243652, 'learning_rate': 5.917233203795616e-06, 'epoch': 0.64}

 64%|██████▍   | 24480/37962 [56:42:18<31:00:23,  8.28s/it]
 64%|██████▍   | 24481/37962 [56:42:27<30:56:29,  8.26s/it]
 64%|██████▍   | 24482/37962 [56:42:35<31:18:00,  8.36s/it]
 64%|██████▍   | 24483/37962 [56:42:43<30:46:55,  8.22s/it]
 64%|██████▍   | 24484/37962 [56:42:51<30:46:07,  8.22s/it]
 64%|██████▍   | 24485/37962 [56:43:00<30:49:10,  8.23s/it]
 65%|██████▍   | 24486/37962 [56:43:08<30:44:10,  8.21s/it]
 65%|██████▍   | 24487/37962 [56:43:16<30:51:56,  8.25s/it]
 65%|██████▍   | 24488/37962 [56:43:24<30:57:21,  8.27s/it]
 65%|██████▍   | 24489/37962 [56:43:33<30:54:01,  8.26s/it]
 65%|██████▍   | 24490/37962 [56:43:41<30:44:14,  8.21s/it]
                                                           
{'loss': 1.3124, 'grad_norm': 19.154054641723633, 'learning_rate': 5.909446545399795e-06, 'epoch': 0.65}

 65%|██████▍   | 24490/37962 [56:43:41<30:44:14,  8.21s/it]
 65%|██████▍   | 24491/37962 [56:43:49<30:29:29,  8.15s/it]
 65%|██████▍   | 24492/37962 [56:43:57<30:31:26,  8.16s/it]
 65%|██████▍   | 24493/37962 [56:44:05<30:29:13,  8.15s/it]
 65%|██████▍   | 24494/37962 [56:44:13<30:39:21,  8.19s/it]
 65%|██████▍   | 24495/37962 [56:44:22<30:46:03,  8.22s/it]
 65%|██████▍   | 24496/37962 [56:44:30<30:50:44,  8.25s/it]
 65%|██████▍   | 24497/37962 [56:44:38<30:52:30,  8.25s/it]
 65%|██████▍   | 24498/37962 [56:44:47<30:55:38,  8.27s/it]
 65%|██████▍   | 24499/37962 [56:44:55<30:52:46,  8.26s/it]
 65%|██████▍   | 24500/37962 [56:45:03<30:58:16,  8.28s/it]
                                                           
{'loss': 1.3942, 'grad_norm': 12.192293167114258, 'learning_rate': 5.901662864446969e-06, 'epoch': 0.65}

 65%|██████▍   | 24500/37962 [56:45:03<30:58:16,  8.28s/it]
 65%|██████▍   | 24501/37962 [56:45:11<30:40:18,  8.20s/it]
 65%|██████▍   | 24502/37962 [56:45:19<30:46:26,  8.23s/it]
 65%|██████▍   | 24503/37962 [56:45:28<31:00:27,  8.29s/it]
 65%|██████▍   | 24504/37962 [56:45:36<31:03:23,  8.31s/it]
 65%|██████▍   | 24505/37962 [56:45:45<31:18:33,  8.38s/it]
 65%|██████▍   | 24506/37962 [56:45:53<31:26:54,  8.41s/it]
 65%|██████▍   | 24507/37962 [56:46:02<31:37:57,  8.46s/it]
 65%|██████▍   | 24508/37962 [56:46:10<31:18:06,  8.38s/it]
 65%|██████▍   | 24509/37962 [56:46:18<31:11:28,  8.35s/it]
 65%|██████▍   | 24510/37962 [56:46:26<30:34:49,  8.18s/it]
                                                           
{'loss': 1.1983, 'grad_norm': 5.344665050506592, 'learning_rate': 5.893882166602745e-06, 'epoch': 0.65}

 65%|██████▍   | 24510/37962 [56:46:26<30:34:49,  8.18s/it]
 65%|██████▍   | 24511/37962 [56:46:35<31:07:20,  8.33s/it]
 65%|██████▍   | 24512/37962 [56:46:43<30:47:38,  8.24s/it]
 65%|██████▍   | 24513/37962 [56:46:51<30:45:00,  8.23s/it]
 65%|██████▍   | 24514/37962 [56:46:59<30:44:01,  8.23s/it]
 65%|██████▍   | 24515/37962 [56:47:08<30:56:24,  8.28s/it]
 65%|██████▍   | 24516/37962 [56:47:16<31:00:23,  8.30s/it]
 65%|██████▍   | 24517/37962 [56:47:24<30:59:04,  8.30s/it]
 65%|██████▍   | 24518/37962 [56:47:33<31:04:42,  8.32s/it]
 65%|██████▍   | 24519/37962 [56:47:41<31:15:35,  8.37s/it]
 65%|██████▍   | 24520/37962 [56:47:49<31:08:50,  8.34s/it]
                                                           
{'loss': 1.4042, 'grad_norm': 84.73094940185547, 'learning_rate': 5.886104457530559e-06, 'epoch': 0.65}

 65%|██████▍   | 24520/37962 [56:47:49<31:08:50,  8.34s/it]
 65%|██████▍   | 24521/37962 [56:47:58<30:59:59,  8.30s/it]
 65%|██████▍   | 24522/37962 [56:48:06<31:16:38,  8.38s/it]
 65%|██████▍   | 24523/37962 [56:48:15<31:30:30,  8.44s/it]
 65%|██████▍   | 24524/37962 [56:48:23<30:59:48,  8.30s/it]
 65%|██████▍   | 24525/37962 [56:48:31<31:00:02,  8.31s/it]
 65%|██████▍   | 24526/37962 [56:48:39<30:58:10,  8.30s/it]
 65%|██████▍   | 24527/37962 [56:48:48<31:02:59,  8.32s/it]
 65%|██████▍   | 24528/37962 [56:48:56<30:44:09,  8.24s/it]
 65%|██████▍   | 24529/37962 [56:49:04<30:42:43,  8.23s/it]
 65%|██████▍   | 24530/37962 [56:49:12<30:11:59,  8.09s/it]
                                                           
{'loss': 1.2496, 'grad_norm': 24.291522979736328, 'learning_rate': 5.878329742891669e-06, 'epoch': 0.65}

 65%|██████▍   | 24530/37962 [56:49:12<30:11:59,  8.09s/it]
 65%|██████▍   | 24531/37962 [56:49:20<30:16:30,  8.11s/it]
 65%|██████▍   | 24532/37962 [56:49:28<30:39:24,  8.22s/it]
 65%|██████▍   | 24533/37962 [56:49:37<30:50:34,  8.27s/it]
 65%|██████▍   | 24534/37962 [56:49:45<30:45:53,  8.25s/it]
 65%|██████▍   | 24535/37962 [56:49:53<30:02:05,  8.05s/it]
 65%|██████▍   | 24536/37962 [56:50:01<30:27:30,  8.17s/it]
 65%|██████▍   | 24537/37962 [56:50:09<30:25:48,  8.16s/it]
 65%|██████▍   | 24538/37962 [56:50:17<30:38:33,  8.22s/it]
 65%|██████▍   | 24539/37962 [56:50:26<30:45:59,  8.25s/it]
 65%|██████▍   | 24540/37962 [56:50:34<31:00:21,  8.32s/it]
                                                           
{'loss': 1.3037, 'grad_norm': 7.835628509521484, 'learning_rate': 5.870558028345159e-06, 'epoch': 0.65}

 65%|██████▍   | 24540/37962 [56:50:34<31:00:21,  8.32s/it]
 65%|██████▍   | 24541/37962 [56:50:42<30:52:29,  8.28s/it]
 65%|██████▍   | 24542/37962 [56:50:51<30:49:48,  8.27s/it]
 65%|██████▍   | 24543/37962 [56:50:59<30:46:24,  8.26s/it]
 65%|██████▍   | 24544/37962 [56:51:07<30:20:25,  8.14s/it]
 65%|██████▍   | 24545/37962 [56:51:15<30:29:48,  8.18s/it]
 65%|██████▍   | 24546/37962 [56:51:23<30:20:26,  8.14s/it]
 65%|██████▍   | 24547/37962 [56:51:31<30:29:29,  8.18s/it]
 65%|██████▍   | 24548/37962 [56:51:40<30:44:11,  8.25s/it]
 65%|██████▍   | 24549/37962 [56:51:48<30:48:50,  8.27s/it]
 65%|██████▍   | 24550/37962 [56:51:56<30:55:14,  8.30s/it]
                                                           
{'loss': 1.2818, 'grad_norm': 7.984575271606445, 'learning_rate': 5.862789319547919e-06, 'epoch': 0.65}

 65%|██████▍   | 24550/37962 [56:51:56<30:55:14,  8.30s/it]
 65%|██████▍   | 24551/37962 [56:52:05<30:56:30,  8.31s/it]
 65%|██████▍   | 24552/37962 [56:52:13<31:00:51,  8.33s/it]
 65%|██████▍   | 24553/37962 [56:52:21<30:52:08,  8.29s/it]
 65%|██████▍   | 24554/37962 [56:52:30<30:59:26,  8.32s/it]
 65%|██████▍   | 24555/37962 [56:52:38<30:50:49,  8.28s/it]
 65%|██████▍   | 24556/37962 [56:52:46<30:43:59,  8.25s/it]
 65%|██████▍   | 24557/37962 [56:52:55<30:59:59,  8.33s/it]
 65%|██████▍   | 24558/37962 [56:53:03<30:47:05,  8.27s/it]
 65%|██████▍   | 24559/37962 [56:53:11<30:47:44,  8.27s/it]
 65%|██████▍   | 24560/37962 [56:53:19<30:46:20,  8.27s/it]
                                                           
{'loss': 1.2166, 'grad_norm': 3.5392191410064697, 'learning_rate': 5.855023622154663e-06, 'epoch': 0.65}

 65%|██████▍   | 24560/37962 [56:53:19<30:46:20,  8.27s/it]
 65%|██████▍   | 24561/37962 [56:53:28<30:49:36,  8.28s/it]
 65%|██████▍   | 24562/37962 [56:53:36<30:59:27,  8.33s/it]
 65%|██████▍   | 24563/37962 [56:53:44<30:54:35,  8.30s/it]
 65%|██████▍   | 24564/37962 [56:53:52<30:22:42,  8.16s/it]
 65%|██████▍   | 24565/37962 [56:54:00<30:33:03,  8.21s/it]
 65%|██████▍   | 24566/37962 [56:54:09<30:42:55,  8.25s/it]
 65%|██████▍   | 24567/37962 [56:54:17<30:42:24,  8.25s/it]
 65%|██████▍   | 24568/37962 [56:54:25<30:44:25,  8.26s/it]
 65%|██████▍   | 24569/37962 [56:54:34<30:54:27,  8.31s/it]
 65%|██████▍   | 24570/37962 [56:54:42<30:57:56,  8.32s/it]
                                                           
{'loss': 1.333, 'grad_norm': 14.24640941619873, 'learning_rate': 5.847260941817903e-06, 'epoch': 0.65}

 65%|██████▍   | 24570/37962 [56:54:42<30:57:56,  8.32s/it]
 65%|██████▍   | 24571/37962 [56:54:50<30:42:49,  8.26s/it]
 65%|██████▍   | 24572/37962 [56:54:59<30:50:12,  8.29s/it]
 65%|██████▍   | 24573/37962 [56:55:07<30:54:03,  8.31s/it]
 65%|██████▍   | 24574/37962 [56:55:15<31:07:32,  8.37s/it]
 65%|██████▍   | 24575/37962 [56:55:23<30:44:53,  8.27s/it]
 65%|██████▍   | 24576/37962 [56:55:32<30:49:48,  8.29s/it]
 65%|██████▍   | 24577/37962 [56:55:40<30:45:42,  8.27s/it]
 65%|██████▍   | 24578/37962 [56:55:48<30:51:29,  8.30s/it]
 65%|██████▍   | 24579/37962 [56:55:57<30:37:58,  8.24s/it]
 65%|██████▍   | 24580/37962 [56:56:05<31:01:25,  8.35s/it]
                                                           
{'loss': 1.2871, 'grad_norm': 5.820329666137695, 'learning_rate': 5.839501284187963e-06, 'epoch': 0.65}

 65%|██████▍   | 24580/37962 [56:56:05<31:01:25,  8.35s/it]
 65%|██████▍   | 24581/37962 [56:56:14<31:09:01,  8.38s/it]
 65%|██████▍   | 24582/37962 [56:56:22<31:18:10,  8.42s/it]
 65%|██████▍   | 24583/37962 [56:56:30<31:10:55,  8.39s/it]
 65%|██████▍   | 24584/37962 [56:56:39<31:06:48,  8.37s/it]
 65%|██████▍   | 24585/37962 [56:56:47<30:36:37,  8.24s/it]
 65%|██████▍   | 24586/37962 [56:56:55<30:29:23,  8.21s/it]
 65%|██████▍   | 24587/37962 [56:57:03<30:55:43,  8.32s/it]
 65%|██████▍   | 24588/37962 [56:57:12<30:57:37,  8.33s/it]
 65%|██████▍   | 24589/37962 [56:57:20<30:54:54,  8.32s/it]
 65%|██████▍   | 24590/37962 [56:57:28<30:40:01,  8.26s/it]
                                                           
{'loss': 1.2092, 'grad_norm': 103.34264373779297, 'learning_rate': 5.831744654912965e-06, 'epoch': 0.65}

 65%|██████▍   | 24590/37962 [56:57:28<30:40:01,  8.26s/it]
 65%|██████▍   | 24591/37962 [56:57:37<30:46:23,  8.29s/it]
 65%|██████▍   | 24592/37962 [56:57:45<30:51:36,  8.31s/it]
 65%|██████▍   | 24593/37962 [56:57:53<30:22:22,  8.18s/it]
 65%|██████▍   | 24594/37962 [56:58:01<30:24:19,  8.19s/it]
 65%|██████▍   | 24595/37962 [56:58:10<30:48:41,  8.30s/it]
 65%|██████▍   | 24596/37962 [56:58:18<30:44:12,  8.28s/it]
 65%|██████▍   | 24597/37962 [56:58:26<30:41:42,  8.27s/it]
 65%|██████▍   | 24598/37962 [56:58:34<30:50:43,  8.31s/it]
 65%|██████▍   | 24599/37962 [56:58:43<30:43:57,  8.28s/it]
 65%|██████▍   | 24600/37962 [56:58:51<30:48:11,  8.30s/it]
                                                           
{'loss': 1.2191, 'grad_norm': 8.607818603515625, 'learning_rate': 5.8239910596388224e-06, 'epoch': 0.65}

 65%|██████▍   | 24600/37962 [56:58:51<30:48:11,  8.30s/it]
 65%|██████▍   | 24601/37962 [56:58:59<30:33:08,  8.23s/it]
 65%|██████▍   | 24602/37962 [56:59:07<30:31:45,  8.23s/it]
 65%|██████▍   | 24603/37962 [56:59:16<30:35:08,  8.24s/it]
 65%|██████▍   | 24604/37962 [56:59:24<30:41:24,  8.27s/it]
 65%|██████▍   | 24605/37962 [56:59:32<30:46:42,  8.30s/it]
 65%|██████▍   | 24606/37962 [56:59:41<30:51:25,  8.32s/it]
 65%|██████▍   | 24607/37962 [56:59:49<30:34:01,  8.24s/it]
 65%|██████▍   | 24608/37962 [56:59:57<30:34:33,  8.24s/it]
 65%|██████▍   | 24609/37962 [57:00:05<30:17:07,  8.16s/it]
 65%|██████▍   | 24610/37962 [57:00:13<30:09:37,  8.13s/it]
                                                           
{'loss': 1.3354, 'grad_norm': 10.013311386108398, 'learning_rate': 5.816240504009241e-06, 'epoch': 0.65}

 65%|██████▍   | 24610/37962 [57:00:13<30:09:37,  8.13s/it]
 65%|██████▍   | 24611/37962 [57:00:21<30:19:41,  8.18s/it]
 65%|██████▍   | 24612/37962 [57:00:30<30:28:24,  8.22s/it]
 65%|██████▍   | 24613/37962 [57:00:38<30:31:55,  8.23s/it]
 65%|██████▍   | 24614/37962 [57:00:46<30:43:29,  8.29s/it]
 65%|██████▍   | 24615/37962 [57:00:55<30:45:11,  8.29s/it]
 65%|██████▍   | 24616/37962 [57:01:03<30:47:18,  8.31s/it]
 65%|██████▍   | 24617/37962 [57:01:11<30:42:43,  8.29s/it]
 65%|██████▍   | 24618/37962 [57:01:19<30:36:27,  8.26s/it]
 65%|██████▍   | 24619/37962 [57:01:27<30:31:59,  8.24s/it]
 65%|██████▍   | 24620/37962 [57:01:36<30:39:42,  8.27s/it]
                                                           
{'loss': 1.2892, 'grad_norm': 8.941361427307129, 'learning_rate': 5.808492993665715e-06, 'epoch': 0.65}

 65%|██████▍   | 24620/37962 [57:01:36<30:39:42,  8.27s/it]
 65%|██████▍   | 24621/37962 [57:01:44<30:36:29,  8.26s/it]
 65%|██████▍   | 24622/37962 [57:01:52<30:46:04,  8.30s/it]
 65%|██████▍   | 24623/37962 [57:02:01<30:58:20,  8.36s/it]
 65%|██████▍   | 24624/37962 [57:02:09<30:25:28,  8.21s/it]
 65%|██████▍   | 24625/37962 [57:02:17<30:26:58,  8.22s/it]
 65%|██████▍   | 24626/37962 [57:02:25<30:16:18,  8.17s/it]
 65%|██████▍   | 24627/37962 [57:02:33<30:22:00,  8.20s/it]
 65%|██████▍   | 24628/37962 [57:02:42<30:25:39,  8.22s/it]
 65%|██████▍   | 24629/37962 [57:02:50<30:13:27,  8.16s/it]
 65%|██████▍   | 24630/37962 [57:02:58<30:35:13,  8.26s/it]
                                                           
{'loss': 1.2331, 'grad_norm': 10.48447036743164, 'learning_rate': 5.800748534247533e-06, 'epoch': 0.65}

 65%|██████▍   | 24630/37962 [57:02:58<30:35:13,  8.26s/it]
 65%|██████▍   | 24631/37962 [57:03:06<30:37:33,  8.27s/it]
 65%|██████▍   | 24632/37962 [57:03:15<30:34:03,  8.26s/it]
 65%|██████▍   | 24633/37962 [57:03:23<30:36:58,  8.27s/it]
 65%|██████▍   | 24634/37962 [57:03:31<30:14:26,  8.17s/it]
 65%|██████▍   | 24635/37962 [57:03:39<30:19:58,  8.19s/it]
 65%|██████▍   | 24636/37962 [57:03:47<30:20:33,  8.20s/it]
 65%|██████▍   | 24637/37962 [57:03:56<30:27:31,  8.23s/it]
 65%|██████▍   | 24638/37962 [57:04:04<30:37:48,  8.28s/it]
 65%|██████▍   | 24639/37962 [57:04:12<30:35:09,  8.26s/it]
 65%|██████▍   | 24640/37962 [57:04:21<30:56:10,  8.36s/it]
                                                           
{'loss': 1.3333, 'grad_norm': 15.687726020812988, 'learning_rate': 5.79300713139174e-06, 'epoch': 0.65}

 65%|██████▍   | 24640/37962 [57:04:21<30:56:10,  8.36s/it]
 65%|██████▍   | 24641/37962 [57:04:29<30:39:25,  8.29s/it]
 65%|██████▍   | 24642/37962 [57:04:37<30:24:20,  8.22s/it]
 65%|██████▍   | 24643/37962 [57:05:00<46:41:41, 12.62s/it]
 65%|██████▍   | 24644/37962 [57:05:09<42:12:34, 11.41s/it]
 65%|██████▍   | 24645/37962 [57:05:17<38:41:31, 10.46s/it]
 65%|██████▍   | 24646/37962 [57:05:25<36:25:56,  9.85s/it]
 65%|██████▍   | 24647/37962 [57:05:33<34:24:04,  9.30s/it]
 65%|██████▍   | 24648/37962 [57:05:41<32:57:44,  8.91s/it]
 65%|██████▍   | 24649/37962 [57:05:49<32:09:53,  8.70s/it]
 65%|██████▍   | 24650/37962 [57:05:58<31:38:42,  8.56s/it]
                                                           
{'loss': 1.3121, 'grad_norm': 11.545320510864258, 'learning_rate': 5.785268790733181e-06, 'epoch': 0.65}

 65%|██████▍   | 24650/37962 [57:05:58<31:38:42,  8.56s/it]
 65%|██████▍   | 24651/37962 [57:06:06<31:19:16,  8.47s/it]
 65%|██████▍   | 24652/37962 [57:06:14<31:15:21,  8.45s/it]
 65%|██████▍   | 24653/37962 [57:06:23<30:58:26,  8.38s/it]
 65%|██████▍   | 24654/37962 [57:06:31<30:44:25,  8.32s/it]
 65%|██████▍   | 24655/37962 [57:06:39<30:42:44,  8.31s/it]
 65%|██████▍   | 24656/37962 [57:06:47<30:30:59,  8.26s/it]
 65%|██████▍   | 24657/37962 [57:06:55<30:28:21,  8.25s/it]
 65%|██████▍   | 24658/37962 [57:07:03<30:16:46,  8.19s/it]
 65%|██████▍   | 24659/37962 [57:07:12<30:26:39,  8.24s/it]
 65%|██████▍   | 24660/37962 [57:07:20<30:17:11,  8.20s/it]
                                                           
{'loss': 1.2488, 'grad_norm': 363.5295104980469, 'learning_rate': 5.777533517904451e-06, 'epoch': 0.65}

 65%|██████▍   | 24660/37962 [57:07:20<30:17:11,  8.20s/it]
 65%|██████▍   | 24661/37962 [57:07:28<30:23:19,  8.22s/it]
 65%|██████▍   | 24662/37962 [57:07:36<30:23:50,  8.23s/it]
 65%|██████▍   | 24663/37962 [57:07:45<30:25:45,  8.24s/it]
 65%|██████▍   | 24664/37962 [57:07:53<30:28:28,  8.25s/it]
 65%|██████▍   | 24665/37962 [57:08:01<30:32:08,  8.27s/it]
 65%|██████▍   | 24666/37962 [57:08:09<30:29:22,  8.26s/it]
 65%|██████▍   | 24667/37962 [57:08:18<30:31:50,  8.27s/it]
 65%|██████▍   | 24668/37962 [57:08:26<30:42:08,  8.31s/it]
 65%|██████▍   | 24669/37962 [57:08:35<30:49:36,  8.35s/it]
 65%|██████▍   | 24670/37962 [57:08:43<30:32:01,  8.27s/it]
                                                           
{'loss': 1.275, 'grad_norm': 7.749255180358887, 'learning_rate': 5.769801318535921e-06, 'epoch': 0.65}

 65%|██████▍   | 24670/37962 [57:08:43<30:32:01,  8.27s/it]
 65%|██████▍   | 24671/37962 [57:08:51<30:30:10,  8.26s/it]
 65%|██████▍   | 24672/37962 [57:08:59<30:42:11,  8.32s/it]
 65%|██████▍   | 24673/37962 [57:09:08<30:35:00,  8.29s/it]
 65%|██████▍   | 24674/37962 [57:09:16<30:33:00,  8.28s/it]
 65%|██████▍   | 24675/37962 [57:09:24<30:27:05,  8.25s/it]
 65%|██████▌   | 24676/37962 [57:09:32<30:33:49,  8.28s/it]
 65%|██████▌   | 24677/37962 [57:09:41<30:32:29,  8.28s/it]
 65%|██████▌   | 24678/37962 [57:09:49<30:32:02,  8.27s/it]
 65%|██████▌   | 24679/37962 [57:09:57<30:47:18,  8.34s/it]
 65%|██████▌   | 24680/37962 [57:10:06<30:39:11,  8.31s/it]
                                                           
{'loss': 1.3136, 'grad_norm': 28.032316207885742, 'learning_rate': 5.762072198255734e-06, 'epoch': 0.65}

 65%|██████▌   | 24680/37962 [57:10:06<30:39:11,  8.31s/it]
 65%|██████▌   | 24681/37962 [57:10:14<30:41:16,  8.32s/it]
 65%|██████▌   | 24682/37962 [57:10:22<30:40:23,  8.31s/it]
 65%|██████▌   | 24683/37962 [57:10:31<30:46:46,  8.34s/it]
 65%|██████▌   | 24684/37962 [57:10:39<30:47:37,  8.35s/it]
 65%|██████▌   | 24685/37962 [57:10:47<30:42:58,  8.33s/it]
 65%|██████▌   | 24686/37962 [57:10:56<30:44:20,  8.34s/it]
 65%|██████▌   | 24687/37962 [57:11:04<30:32:31,  8.28s/it]
 65%|██████▌   | 24688/37962 [57:11:12<30:19:31,  8.22s/it]
 65%|██████▌   | 24689/37962 [57:11:20<30:19:17,  8.22s/it]
 65%|██████▌   | 24690/37962 [57:11:29<30:32:05,  8.28s/it]
                                                           
{'loss': 1.1822, 'grad_norm': 7.7889084815979, 'learning_rate': 5.754346162689772e-06, 'epoch': 0.65}

 65%|██████▌   | 24690/37962 [57:11:29<30:32:05,  8.28s/it]
 65%|██████▌   | 24691/37962 [57:11:36<30:05:37,  8.16s/it]
 65%|██████▌   | 24692/37962 [57:11:45<29:59:26,  8.14s/it]
 65%|██████▌   | 24693/37962 [57:11:52<29:45:56,  8.08s/it]
 65%|██████▌   | 24694/37962 [57:12:01<30:06:23,  8.17s/it]
 65%|██████▌   | 24695/37962 [57:12:09<30:19:44,  8.23s/it]
 65%|██████▌   | 24696/37962 [57:12:18<30:23:33,  8.25s/it]
 65%|██████▌   | 24697/37962 [57:12:26<30:06:43,  8.17s/it]
 65%|██████▌   | 24698/37962 [57:12:34<29:54:51,  8.12s/it]
 65%|██████▌   | 24699/37962 [57:12:42<30:10:00,  8.19s/it]
 65%|██████▌   | 24700/37962 [57:12:50<30:27:17,  8.27s/it]
                                                           
{'loss': 1.2164, 'grad_norm': 8.536368370056152, 'learning_rate': 5.74662321746169e-06, 'epoch': 0.65}

 65%|██████▌   | 24700/37962 [57:12:50<30:27:17,  8.27s/it]
 65%|██████▌   | 24701/37962 [57:12:59<30:24:24,  8.25s/it]
 65%|██████▌   | 24702/37962 [57:13:07<30:22:47,  8.25s/it]
 65%|██████▌   | 24703/37962 [57:13:15<30:25:08,  8.26s/it]
 65%|██████▌   | 24704/37962 [57:13:23<30:25:33,  8.26s/it]
 65%|██████▌   | 24705/37962 [57:13:31<30:17:58,  8.23s/it]
 65%|██████▌   | 24706/37962 [57:13:40<30:19:48,  8.24s/it]
 65%|██████▌   | 24707/37962 [57:13:48<30:31:03,  8.29s/it]
 65%|██████▌   | 24708/37962 [57:13:56<30:28:02,  8.28s/it]
 65%|██████▌   | 24709/37962 [57:14:05<30:29:48,  8.28s/it]
 65%|██████▌   | 24710/37962 [57:14:13<30:35:02,  8.31s/it]
                                                           
{'loss': 1.1822, 'grad_norm': 12.466568946838379, 'learning_rate': 5.738903368192884e-06, 'epoch': 0.65}

 65%|██████▌   | 24710/37962 [57:14:13<30:35:02,  8.31s/it]
 65%|██████▌   | 24711/37962 [57:14:21<30:38:55,  8.33s/it]
 65%|██████▌   | 24712/37962 [57:14:30<30:37:16,  8.32s/it]
 65%|██████▌   | 24713/37962 [57:14:38<30:22:46,  8.25s/it]
 65%|██████▌   | 24714/37962 [57:14:46<30:21:25,  8.25s/it]
 65%|██████▌   | 24715/37962 [57:14:54<30:19:03,  8.24s/it]
 65%|██████▌   | 24716/37962 [57:15:03<30:24:02,  8.26s/it]
 65%|██████▌   | 24717/37962 [57:15:11<30:22:25,  8.26s/it]
 65%|██████▌   | 24718/37962 [57:15:19<29:56:49,  8.14s/it]
 65%|██████▌   | 24719/37962 [57:15:27<29:46:15,  8.09s/it]
 65%|██████▌   | 24720/37962 [57:15:35<29:58:06,  8.15s/it]
                                                           
{'loss': 1.3404, 'grad_norm': 6.191939830780029, 'learning_rate': 5.7311866205024934e-06, 'epoch': 0.65}

 65%|██████▌   | 24720/37962 [57:15:35<29:58:06,  8.15s/it]
 65%|██████▌   | 24721/37962 [57:15:43<30:17:06,  8.23s/it]
 65%|██████▌   | 24722/37962 [57:15:52<30:26:15,  8.28s/it]
 65%|██████▌   | 24723/37962 [57:16:00<30:35:05,  8.32s/it]
 65%|██████▌   | 24724/37962 [57:16:08<30:23:32,  8.27s/it]
 65%|██████▌   | 24725/37962 [57:16:16<30:08:11,  8.20s/it]
 65%|██████▌   | 24726/37962 [57:16:25<30:10:39,  8.21s/it]
 65%|██████▌   | 24727/37962 [57:16:33<30:18:43,  8.25s/it]
 65%|██████▌   | 24728/37962 [57:16:41<30:27:45,  8.29s/it]
 65%|██████▌   | 24729/37962 [57:16:49<29:47:06,  8.10s/it]
 65%|██████▌   | 24730/37962 [57:16:57<30:11:33,  8.21s/it]
                                                           
{'loss': 1.2371, 'grad_norm': 15.161687850952148, 'learning_rate': 5.72347298000741e-06, 'epoch': 0.65}

 65%|██████▌   | 24730/37962 [57:16:57<30:11:33,  8.21s/it]
 65%|██████▌   | 24731/37962 [57:17:06<30:00:31,  8.17s/it]
 65%|██████▌   | 24732/37962 [57:17:14<30:06:03,  8.19s/it]
 65%|██████▌   | 24733/37962 [57:17:22<30:19:22,  8.25s/it]
 65%|██████▌   | 24734/37962 [57:17:30<30:02:01,  8.17s/it]
 65%|██████▌   | 24735/37962 [57:17:38<30:02:13,  8.18s/it]
 65%|██████▌   | 24736/37962 [57:17:47<30:03:31,  8.18s/it]
 65%|██████▌   | 24737/37962 [57:17:55<30:17:50,  8.25s/it]
 65%|██████▌   | 24738/37962 [57:18:03<30:25:13,  8.28s/it]
 65%|██████▌   | 24739/37962 [57:18:11<30:06:51,  8.20s/it]
 65%|██████▌   | 24740/37962 [57:18:20<30:14:55,  8.24s/it]
                                                           
{'loss': 1.3787, 'grad_norm': 8.813163757324219, 'learning_rate': 5.715762452322263e-06, 'epoch': 0.65}

 65%|██████▌   | 24740/37962 [57:18:20<30:14:55,  8.24s/it]
 65%|██████▌   | 24741/37962 [57:18:28<30:13:19,  8.23s/it]
 65%|██████▌   | 24742/37962 [57:18:36<30:15:19,  8.24s/it]
 65%|██████▌   | 24743/37962 [57:18:44<29:47:59,  8.12s/it]
 65%|██████▌   | 24744/37962 [57:18:52<29:56:50,  8.16s/it]
 65%|██████▌   | 24745/37962 [57:19:00<30:03:38,  8.19s/it]
 65%|██████▌   | 24746/37962 [57:19:09<30:20:04,  8.26s/it]
 65%|██████▌   | 24747/37962 [57:19:17<30:23:20,  8.28s/it]
 65%|██████▌   | 24748/37962 [57:19:25<30:21:24,  8.27s/it]
 65%|██████▌   | 24749/37962 [57:19:34<30:21:20,  8.27s/it]
 65%|██████▌   | 24750/37962 [57:19:42<30:18:48,  8.26s/it]
                                                           
{'loss': 1.2142, 'grad_norm': 8.468017578125, 'learning_rate': 5.7080550430594085e-06, 'epoch': 0.65}

 65%|██████▌   | 24750/37962 [57:19:42<30:18:48,  8.26s/it]
 65%|██████▌   | 24751/37962 [57:19:50<30:19:38,  8.26s/it]
 65%|██████▌   | 24752/37962 [57:19:58<30:09:43,  8.22s/it]
 65%|██████▌   | 24753/37962 [57:20:07<30:11:41,  8.23s/it]
 65%|██████▌   | 24754/37962 [57:20:15<30:20:32,  8.27s/it]
 65%|██████▌   | 24755/37962 [57:20:23<30:25:45,  8.29s/it]
 65%|██████▌   | 24756/37962 [57:20:32<30:27:01,  8.30s/it]
 65%|██████▌   | 24757/37962 [57:20:40<30:21:42,  8.28s/it]
 65%|██████▌   | 24758/37962 [57:20:48<30:19:12,  8.27s/it]
 65%|██████▌   | 24759/37962 [57:20:56<30:10:17,  8.23s/it]
 65%|██████▌   | 24760/37962 [57:21:04<30:11:31,  8.23s/it]
                                                           
{'loss': 1.3829, 'grad_norm': 6.7396626472473145, 'learning_rate': 5.70035075782893e-06, 'epoch': 0.65}

 65%|██████▌   | 24760/37962 [57:21:04<30:11:31,  8.23s/it]
 65%|██████▌   | 24761/37962 [57:21:13<30:17:58,  8.26s/it]
 65%|██████▌   | 24762/37962 [57:21:21<30:19:36,  8.27s/it]
 65%|██████▌   | 24763/37962 [57:21:29<30:03:35,  8.20s/it]
 65%|██████▌   | 24764/37962 [57:21:37<29:47:38,  8.13s/it]
 65%|██████▌   | 24765/37962 [57:21:46<30:10:02,  8.23s/it]
 65%|██████▌   | 24766/37962 [57:21:54<30:13:29,  8.25s/it]
 65%|██████▌   | 24767/37962 [57:22:02<30:20:26,  8.28s/it]
 65%|██████▌   | 24768/37962 [57:22:11<30:21:35,  8.28s/it]
 65%|██████▌   | 24769/37962 [57:22:19<30:28:37,  8.32s/it]
 65%|██████▌   | 24770/37962 [57:22:27<30:16:16,  8.26s/it]
                                                           
{'loss': 1.2607, 'grad_norm': 8.925597190856934, 'learning_rate': 5.692649602238652e-06, 'epoch': 0.65}

 65%|██████▌   | 24770/37962 [57:22:27<30:16:16,  8.26s/it]
 65%|██████▌   | 24771/37962 [57:22:35<30:28:21,  8.32s/it]
 65%|██████▌   | 24772/37962 [57:22:44<30:24:06,  8.30s/it]
 65%|██████▌   | 24773/37962 [57:22:52<29:50:33,  8.15s/it]
 65%|██████▌   | 24774/37962 [57:23:00<29:54:50,  8.17s/it]
 65%|██████▌   | 24775/37962 [57:23:08<30:12:00,  8.24s/it]
 65%|██████▌   | 24776/37962 [57:23:16<30:08:14,  8.23s/it]
 65%|██████▌   | 24777/37962 [57:23:25<30:06:10,  8.22s/it]
 65%|██████▌   | 24778/37962 [57:23:33<30:17:01,  8.27s/it]
 65%|██████▌   | 24779/37962 [57:23:41<30:24:20,  8.30s/it]
 65%|██████▌   | 24780/37962 [57:23:50<30:29:58,  8.33s/it]
                                                           
{'loss': 1.1922, 'grad_norm': 9.139082908630371, 'learning_rate': 5.684951581894106e-06, 'epoch': 0.65}

 65%|██████▌   | 24780/37962 [57:23:50<30:29:58,  8.33s/it]
 65%|██████▌   | 24781/37962 [57:23:58<30:27:55,  8.32s/it]
 65%|██████▌   | 24782/37962 [57:24:06<30:23:12,  8.30s/it]
 65%|██████▌   | 24783/37962 [57:24:15<30:29:01,  8.33s/it]
 65%|██████▌   | 24784/37962 [57:24:23<30:24:01,  8.30s/it]
 65%|██████▌   | 24785/37962 [57:24:31<30:22:06,  8.30s/it]
 65%|██████▌   | 24786/37962 [57:24:39<30:23:21,  8.30s/it]
 65%|██████▌   | 24787/37962 [57:24:48<30:29:31,  8.33s/it]
 65%|██████▌   | 24788/37962 [57:24:56<30:27:47,  8.32s/it]
 65%|██████▌   | 24789/37962 [57:25:05<30:27:45,  8.33s/it]
 65%|██████▌   | 24790/37962 [57:25:13<30:23:47,  8.31s/it]
                                                           
{'loss': 1.2189, 'grad_norm': 22.831769943237305, 'learning_rate': 5.677256702398553e-06, 'epoch': 0.65}

 65%|██████▌   | 24790/37962 [57:25:13<30:23:47,  8.31s/it]
 65%|██████▌   | 24791/37962 [57:25:21<30:09:33,  8.24s/it]
 65%|██████▌   | 24792/37962 [57:25:29<30:24:27,  8.31s/it]
 65%|██████▌   | 24793/37962 [57:25:38<30:35:11,  8.36s/it]
 65%|██████▌   | 24794/37962 [57:25:46<30:31:01,  8.34s/it]
 65%|██████▌   | 24795/37962 [57:25:55<30:34:11,  8.36s/it]
 65%|██████▌   | 24796/37962 [57:26:03<30:40:46,  8.39s/it]
 65%|██████▌   | 24797/37962 [57:26:11<30:33:01,  8.35s/it]
 65%|██████▌   | 24798/37962 [57:26:19<30:02:35,  8.22s/it]
 65%|██████▌   | 24799/37962 [57:26:27<30:05:08,  8.23s/it]
 65%|██████▌   | 24800/37962 [57:26:36<29:56:23,  8.19s/it]
                                                           
{'loss': 1.2554, 'grad_norm': 5.69135046005249, 'learning_rate': 5.6695649693529605e-06, 'epoch': 0.65}

 65%|██████▌   | 24800/37962 [57:26:36<29:56:23,  8.19s/it]
 65%|██████▌   | 24801/37962 [57:26:44<29:53:09,  8.17s/it]
 65%|██████▌   | 24802/37962 [57:26:52<29:56:45,  8.19s/it]
 65%|██████▌   | 24803/37962 [57:27:00<30:07:34,  8.24s/it]
 65%|██████▌   | 24804/37962 [57:27:08<30:07:32,  8.24s/it]
 65%|██████▌   | 24805/37962 [57:27:17<30:12:54,  8.27s/it]
 65%|██████▌   | 24806/37962 [57:27:25<30:01:43,  8.22s/it]
 65%|██████▌   | 24807/37962 [57:27:33<29:47:14,  8.15s/it]
 65%|██████▌   | 24808/37962 [57:27:41<29:56:36,  8.19s/it]
 65%|██████▌   | 24809/37962 [57:27:50<30:12:25,  8.27s/it]
 65%|██████▌   | 24810/37962 [57:27:58<30:13:52,  8.27s/it]
                                                           
{'loss': 1.2776, 'grad_norm': 19.217451095581055, 'learning_rate': 5.661876388356005e-06, 'epoch': 0.65}

 65%|██████▌   | 24810/37962 [57:27:58<30:13:52,  8.27s/it]
 65%|██████▌   | 24811/37962 [57:28:06<30:15:28,  8.28s/it]
 65%|██████▌   | 24812/37962 [57:28:14<29:58:00,  8.20s/it]
 65%|██████▌   | 24813/37962 [57:28:23<30:06:50,  8.24s/it]
 65%|██████▌   | 24814/37962 [57:28:31<30:10:16,  8.26s/it]
 65%|██████▌   | 24815/37962 [57:28:39<30:10:08,  8.26s/it]
 65%|██████▌   | 24816/37962 [57:28:47<30:11:28,  8.27s/it]
 65%|██████▌   | 24817/37962 [57:28:56<30:34:20,  8.37s/it]
 65%|██████▌   | 24818/37962 [57:29:04<30:29:24,  8.35s/it]
 65%|██████▌   | 24819/37962 [57:29:13<30:31:23,  8.36s/it]
 65%|██████▌   | 24820/37962 [57:29:21<30:35:14,  8.38s/it]
                                                           
{'loss': 1.4224, 'grad_norm': 4.157867431640625, 'learning_rate': 5.654190965004076e-06, 'epoch': 0.65}

 65%|██████▌   | 24820/37962 [57:29:21<30:35:14,  8.38s/it]
 65%|██████▌   | 24821/37962 [57:29:30<30:52:00,  8.46s/it]
 65%|██████▌   | 24822/37962 [57:29:38<30:58:45,  8.49s/it]
 65%|██████▌   | 24823/37962 [57:29:47<30:51:14,  8.45s/it]
 65%|██████▌   | 24824/37962 [57:29:55<30:36:35,  8.39s/it]
 65%|██████▌   | 24825/37962 [57:30:03<30:17:59,  8.30s/it]
 65%|██████▌   | 24826/37962 [57:30:11<30:12:13,  8.28s/it]
 65%|██████▌   | 24827/37962 [57:30:19<29:59:58,  8.22s/it]
 65%|██████▌   | 24828/37962 [57:30:28<30:02:23,  8.23s/it]
 65%|██████▌   | 24829/37962 [57:30:36<30:19:03,  8.31s/it]
 65%|██████▌   | 24830/37962 [57:30:44<30:19:50,  8.31s/it]
                                                           
{'loss': 1.218, 'grad_norm': 6.543351173400879, 'learning_rate': 5.6465087048912534e-06, 'epoch': 0.65}

 65%|██████▌   | 24830/37962 [57:30:44<30:19:50,  8.31s/it]
 65%|██████▌   | 24831/37962 [57:30:52<29:54:54,  8.20s/it]
 65%|██████▌   | 24832/37962 [57:31:01<30:01:25,  8.23s/it]
 65%|██████▌   | 24833/37962 [57:31:09<29:44:50,  8.16s/it]
 65%|██████▌   | 24834/37962 [57:31:17<29:59:21,  8.22s/it]
 65%|██████▌   | 24835/37962 [57:31:25<30:10:42,  8.28s/it]
 65%|██████▌   | 24836/37962 [57:31:34<30:15:22,  8.30s/it]
 65%|██████▌   | 24837/37962 [57:31:42<30:08:01,  8.27s/it]
 65%|██████▌   | 24838/37962 [57:31:50<30:19:46,  8.32s/it]
 65%|██████▌   | 24839/37962 [57:31:59<30:06:02,  8.26s/it]
 65%|██████▌   | 24840/37962 [57:32:07<30:24:37,  8.34s/it]
                                                           
{'loss': 1.2798, 'grad_norm': 10.412494659423828, 'learning_rate': 5.6388296136093266e-06, 'epoch': 0.65}

 65%|██████▌   | 24840/37962 [57:32:07<30:24:37,  8.34s/it]
 65%|██████▌   | 24841/37962 [57:32:15<30:14:42,  8.30s/it]
 65%|██████▌   | 24842/37962 [57:32:24<30:17:51,  8.31s/it]
 65%|██████▌   | 24843/37962 [57:32:32<30:24:25,  8.34s/it]
 65%|██████▌   | 24844/37962 [57:32:40<30:14:57,  8.30s/it]
 65%|██████▌   | 24845/37962 [57:32:49<30:13:34,  8.30s/it]
 65%|██████▌   | 24846/37962 [57:32:57<30:13:09,  8.29s/it]
 65%|██████▌   | 24847/37962 [57:33:05<30:12:49,  8.29s/it]
 65%|██████▌   | 24848/37962 [57:33:13<29:58:26,  8.23s/it]
 65%|██████▌   | 24849/37962 [57:33:21<29:38:58,  8.14s/it]
 65%|██████▌   | 24850/37962 [57:33:29<29:46:58,  8.18s/it]
                                                           
{'loss': 1.2827, 'grad_norm': 5.017049312591553, 'learning_rate': 5.631153696747775e-06, 'epoch': 0.65}

 65%|██████▌   | 24850/37962 [57:33:29<29:46:58,  8.18s/it]
 65%|██████▌   | 24851/37962 [57:33:38<30:20:04,  8.33s/it]
 65%|██████▌   | 24852/37962 [57:33:53<37:54:34, 10.41s/it]
 65%|██████▌   | 24853/37962 [57:34:02<35:41:31,  9.80s/it]
 65%|██████▌   | 24854/37962 [57:34:10<33:52:17,  9.30s/it]
 65%|██████▌   | 24855/37962 [57:34:18<32:15:34,  8.86s/it]
 65%|██████▌   | 24856/37962 [57:34:26<31:28:44,  8.65s/it]
 65%|██████▌   | 24857/37962 [57:34:34<31:20:50,  8.61s/it]
 65%|██████▌   | 24858/37962 [57:34:43<30:58:41,  8.51s/it]
 65%|██████▌   | 24859/37962 [57:34:51<30:24:29,  8.35s/it]
 65%|██████▌   | 24860/37962 [57:34:59<30:16:57,  8.32s/it]
                                                           
{'loss': 1.2146, 'grad_norm': 11.597733497619629, 'learning_rate': 5.623480959893763e-06, 'epoch': 0.65}

 65%|██████▌   | 24860/37962 [57:34:59<30:16:57,  8.32s/it]
 65%|██████▌   | 24861/37962 [57:35:07<30:09:31,  8.29s/it]
 65%|██████▌   | 24862/37962 [57:35:15<30:15:11,  8.31s/it]
 65%|██████▌   | 24863/37962 [57:35:24<30:10:43,  8.29s/it]
 65%|██████▌   | 24864/37962 [57:35:32<30:28:17,  8.38s/it]
 65%|██████▌   | 24865/37962 [57:35:40<30:16:31,  8.32s/it]
 66%|██████▌   | 24866/37962 [57:35:49<30:07:56,  8.28s/it]
 66%|██████▌   | 24867/37962 [57:35:57<30:04:21,  8.27s/it]
 66%|██████▌   | 24868/37962 [57:36:05<30:08:19,  8.29s/it]
 66%|██████▌   | 24869/37962 [57:36:13<30:04:25,  8.27s/it]
 66%|██████▌   | 24870/37962 [57:36:21<29:41:33,  8.16s/it]
                                                           
{'loss': 1.3387, 'grad_norm': 5.675497055053711, 'learning_rate': 5.615811408632137e-06, 'epoch': 0.66}

 66%|██████▌   | 24870/37962 [57:36:21<29:41:33,  8.16s/it]
 66%|██████▌   | 24871/37962 [57:36:30<29:56:54,  8.24s/it]
 66%|██████▌   | 24872/37962 [57:36:38<29:56:47,  8.24s/it]
 66%|██████▌   | 24873/37962 [57:36:46<29:52:33,  8.22s/it]
 66%|██████▌   | 24874/37962 [57:36:54<29:53:18,  8.22s/it]
 66%|██████▌   | 24875/37962 [57:37:03<29:56:17,  8.24s/it]
 66%|██████▌   | 24876/37962 [57:37:11<29:56:07,  8.24s/it]
 66%|██████▌   | 24877/37962 [57:37:19<30:00:27,  8.26s/it]
 66%|██████▌   | 24878/37962 [57:37:27<29:58:43,  8.25s/it]
 66%|██████▌   | 24879/37962 [57:37:36<30:05:58,  8.28s/it]
 66%|██████▌   | 24880/37962 [57:37:44<29:54:11,  8.23s/it]
                                                           
{'loss': 1.223, 'grad_norm': 7.64119815826416, 'learning_rate': 5.6081450485454414e-06, 'epoch': 0.66}

 66%|██████▌   | 24880/37962 [57:37:44<29:54:11,  8.23s/it]
 66%|██████▌   | 24881/37962 [57:37:52<29:50:56,  8.21s/it]
 66%|██████▌   | 24882/37962 [57:38:00<29:54:02,  8.23s/it]
 66%|██████▌   | 24883/37962 [57:38:09<29:58:10,  8.25s/it]
 66%|██████▌   | 24884/37962 [57:38:17<30:02:55,  8.27s/it]
 66%|██████▌   | 24885/37962 [57:38:25<30:14:31,  8.33s/it]
 66%|██████▌   | 24886/37962 [57:38:33<29:55:32,  8.24s/it]
 66%|██████▌   | 24887/37962 [57:38:42<30:05:55,  8.29s/it]
 66%|██████▌   | 24888/37962 [57:38:50<29:48:36,  8.21s/it]
 66%|██████▌   | 24889/37962 [57:38:58<29:51:36,  8.22s/it]
 66%|██████▌   | 24890/37962 [57:39:06<29:58:28,  8.25s/it]
                                                           
{'loss': 1.3209, 'grad_norm': 8.187503814697266, 'learning_rate': 5.600481885213875e-06, 'epoch': 0.66}

 66%|██████▌   | 24890/37962 [57:39:06<29:58:28,  8.25s/it]
 66%|██████▌   | 24891/37962 [57:39:15<30:09:46,  8.31s/it]
 66%|██████▌   | 24892/37962 [57:39:23<30:17:43,  8.34s/it]
 66%|██████▌   | 24893/37962 [57:39:31<29:52:54,  8.23s/it]
 66%|██████▌   | 24894/37962 [57:39:39<29:48:27,  8.21s/it]
 66%|██████▌   | 24895/37962 [57:39:48<30:00:57,  8.27s/it]
 66%|██████▌   | 24896/37962 [57:39:56<30:04:27,  8.29s/it]
 66%|██████▌   | 24897/37962 [57:40:05<30:16:52,  8.34s/it]
 66%|██████▌   | 24898/37962 [57:40:13<30:01:47,  8.28s/it]
 66%|██████▌   | 24899/37962 [57:40:21<30:18:09,  8.35s/it]
 66%|██████▌   | 24900/37962 [57:40:29<29:56:20,  8.25s/it]
                                                           
{'loss': 1.3173, 'grad_norm': 14.66977596282959, 'learning_rate': 5.5928219242153306e-06, 'epoch': 0.66}

 66%|██████▌   | 24900/37962 [57:40:29<29:56:20,  8.25s/it]
 66%|██████▌   | 24901/37962 [57:40:37<29:45:09,  8.20s/it]
 66%|██████▌   | 24902/37962 [57:40:45<29:27:39,  8.12s/it]
 66%|██████▌   | 24903/37962 [57:40:54<29:33:26,  8.15s/it]
 66%|██████▌   | 24904/37962 [57:41:02<29:43:15,  8.19s/it]
 66%|██████▌   | 24905/37962 [57:41:10<29:31:28,  8.14s/it]
 66%|██████▌   | 24906/37962 [57:41:18<29:38:06,  8.17s/it]
 66%|██████▌   | 24907/37962 [57:41:26<29:42:01,  8.19s/it]
 66%|██████▌   | 24908/37962 [57:41:35<29:38:55,  8.18s/it]
 66%|██████▌   | 24909/37962 [57:41:43<29:47:20,  8.22s/it]
 66%|██████▌   | 24910/37962 [57:41:51<29:53:00,  8.24s/it]
                                                           
{'loss': 1.2257, 'grad_norm': 7.250889778137207, 'learning_rate': 5.585165171125365e-06, 'epoch': 0.66}

 66%|██████▌   | 24910/37962 [57:41:51<29:53:00,  8.24s/it]
 66%|██████▌   | 24911/37962 [57:41:59<29:42:17,  8.19s/it]
 66%|██████▌   | 24912/37962 [57:42:07<29:33:54,  8.16s/it]
 66%|██████▌   | 24913/37962 [57:42:16<29:50:03,  8.23s/it]
 66%|██████▌   | 24914/37962 [57:42:24<29:48:17,  8.22s/it]
 66%|██████▌   | 24915/37962 [57:42:32<30:11:46,  8.33s/it]
 66%|██████▌   | 24916/37962 [57:42:40<29:49:07,  8.23s/it]
 66%|██████▌   | 24917/37962 [57:42:48<29:35:26,  8.17s/it]
 66%|██████▌   | 24918/37962 [57:42:57<29:46:32,  8.22s/it]
 66%|██████▌   | 24919/37962 [57:43:05<29:51:06,  8.24s/it]
 66%|██████▌   | 24920/37962 [57:43:13<29:30:54,  8.15s/it]
                                                           
{'loss': 1.2273, 'grad_norm': 14.946112632751465, 'learning_rate': 5.577511631517183e-06, 'epoch': 0.66}

 66%|██████▌   | 24920/37962 [57:43:13<29:30:54,  8.15s/it]
 66%|██████▌   | 24921/37962 [57:43:21<29:47:51,  8.23s/it]
 66%|██████▌   | 24922/37962 [57:43:30<29:55:50,  8.26s/it]
 66%|██████▌   | 24923/37962 [57:43:38<29:59:19,  8.28s/it]
 66%|██████▌   | 24924/37962 [57:43:46<29:49:26,  8.23s/it]
 66%|██████▌   | 24925/37962 [57:43:54<29:46:59,  8.22s/it]
 66%|██████▌   | 24926/37962 [57:44:03<29:53:02,  8.25s/it]
 66%|██████▌   | 24927/37962 [57:44:11<29:48:11,  8.23s/it]
 66%|██████▌   | 24928/37962 [57:44:19<29:57:39,  8.28s/it]
 66%|██████▌   | 24929/37962 [57:44:28<29:55:04,  8.26s/it]
 66%|██████▌   | 24930/37962 [57:44:36<30:05:36,  8.31s/it]
                                                           
{'loss': 1.3115, 'grad_norm': 28.251611709594727, 'learning_rate': 5.569861310961668e-06, 'epoch': 0.66}

 66%|██████▌   | 24930/37962 [57:44:36<30:05:36,  8.31s/it]
 66%|██████▌   | 24931/37962 [57:44:44<29:56:19,  8.27s/it]
 66%|██████▌   | 24932/37962 [57:44:52<29:52:07,  8.25s/it]
 66%|██████▌   | 24933/37962 [57:45:01<29:53:36,  8.26s/it]
 66%|██████▌   | 24934/37962 [57:45:09<29:58:31,  8.28s/it]
 66%|██████▌   | 24935/37962 [57:45:17<29:51:55,  8.25s/it]
 66%|██████▌   | 24936/37962 [57:45:25<29:50:21,  8.25s/it]
 66%|██████▌   | 24937/37962 [57:45:34<29:40:24,  8.20s/it]
 66%|██████▌   | 24938/37962 [57:45:41<29:25:37,  8.13s/it]
 66%|██████▌   | 24939/37962 [57:45:50<29:38:55,  8.20s/it]
 66%|██████▌   | 24940/37962 [57:45:58<29:32:03,  8.16s/it]
                                                           
{'loss': 1.2561, 'grad_norm': 21.359912872314453, 'learning_rate': 5.562214215027363e-06, 'epoch': 0.66}

 66%|██████▌   | 24940/37962 [57:45:58<29:32:03,  8.16s/it]
 66%|██████▌   | 24941/37962 [57:46:06<29:37:46,  8.19s/it]
 66%|██████▌   | 24942/37962 [57:46:15<29:48:35,  8.24s/it]
 66%|██████▌   | 24943/37962 [57:46:23<29:43:34,  8.22s/it]
 66%|██████▌   | 24944/37962 [57:46:31<29:52:46,  8.26s/it]
 66%|██████▌   | 24945/37962 [57:46:39<30:00:26,  8.30s/it]
 66%|██████▌   | 24946/37962 [57:46:48<29:55:27,  8.28s/it]
 66%|██████▌   | 24947/37962 [57:46:56<29:56:34,  8.28s/it]
 66%|██████▌   | 24948/37962 [57:47:04<29:50:05,  8.25s/it]
 66%|██████▌   | 24949/37962 [57:47:12<29:53:45,  8.27s/it]
 66%|██████▌   | 24950/37962 [57:47:21<29:52:15,  8.26s/it]
                                                           
{'loss': 1.3305, 'grad_norm': 14.089552879333496, 'learning_rate': 5.554570349280447e-06, 'epoch': 0.66}

 66%|██████▌   | 24950/37962 [57:47:21<29:52:15,  8.26s/it]
 66%|██████▌   | 24951/37962 [57:47:29<29:54:24,  8.27s/it]
 66%|██████▌   | 24952/37962 [57:47:37<29:34:19,  8.18s/it]
 66%|██████▌   | 24953/37962 [57:47:45<29:35:00,  8.19s/it]
 66%|██████▌   | 24954/37962 [57:47:53<29:42:47,  8.22s/it]
 66%|██████▌   | 24955/37962 [57:48:02<30:02:32,  8.31s/it]
 66%|██████▌   | 24956/37962 [57:48:10<29:58:48,  8.30s/it]
 66%|██████▌   | 24957/37962 [57:48:19<29:56:57,  8.29s/it]
 66%|██████▌   | 24958/37962 [57:48:27<29:39:21,  8.21s/it]
 66%|██████▌   | 24959/37962 [57:48:35<29:47:26,  8.25s/it]
 66%|██████▌   | 24960/37962 [57:48:43<29:39:13,  8.21s/it]
                                                           
{'loss': 1.2375, 'grad_norm': 3.4369020462036133, 'learning_rate': 5.546929719284766e-06, 'epoch': 0.66}

 66%|██████▌   | 24960/37962 [57:48:43<29:39:13,  8.21s/it]
 66%|██████▌   | 24961/37962 [57:48:51<29:48:08,  8.25s/it]
 66%|██████▌   | 24962/37962 [57:49:00<29:50:09,  8.26s/it]
 66%|██████▌   | 24963/37962 [57:49:08<29:55:39,  8.29s/it]
 66%|██████▌   | 24964/37962 [57:49:16<29:50:48,  8.27s/it]
 66%|██████▌   | 24965/37962 [57:49:24<29:35:20,  8.20s/it]
 66%|██████▌   | 24966/37962 [57:49:33<29:40:21,  8.22s/it]
 66%|██████▌   | 24967/37962 [57:49:41<29:40:51,  8.22s/it]
 66%|██████▌   | 24968/37962 [57:49:49<29:58:37,  8.31s/it]
 66%|██████▌   | 24969/37962 [57:49:58<29:56:21,  8.30s/it]
 66%|██████▌   | 24970/37962 [57:50:06<29:57:11,  8.30s/it]
                                                           
{'loss': 1.2634, 'grad_norm': 159.31581115722656, 'learning_rate': 5.539292330601799e-06, 'epoch': 0.66}

 66%|██████▌   | 24970/37962 [57:50:06<29:57:11,  8.30s/it]
 66%|██████▌   | 24971/37962 [57:50:14<29:48:24,  8.26s/it]
 66%|██████▌   | 24972/37962 [57:50:22<29:56:51,  8.30s/it]
 66%|██████▌   | 24973/37962 [57:50:31<29:54:50,  8.29s/it]
 66%|██████▌   | 24974/37962 [57:50:39<29:58:04,  8.31s/it]
 66%|██████▌   | 24975/37962 [57:50:47<29:56:11,  8.30s/it]
 66%|██████▌   | 24976/37962 [57:50:55<29:41:56,  8.23s/it]
 66%|██████▌   | 24977/37962 [57:51:04<29:42:43,  8.24s/it]
 66%|██████▌   | 24978/37962 [57:51:12<29:35:40,  8.21s/it]
 66%|██████▌   | 24979/37962 [57:51:20<29:38:42,  8.22s/it]
 66%|██████▌   | 24980/37962 [57:51:28<29:46:42,  8.26s/it]
                                                           
{'loss': 1.2459, 'grad_norm': 5.253540992736816, 'learning_rate': 5.531658188790663e-06, 'epoch': 0.66}

 66%|██████▌   | 24980/37962 [57:51:28<29:46:42,  8.26s/it]
 66%|██████▌   | 24981/37962 [57:51:37<30:13:43,  8.38s/it]
 66%|██████▌   | 24982/37962 [57:51:45<29:58:36,  8.31s/it]
 66%|██████▌   | 24983/37962 [57:51:53<29:58:48,  8.32s/it]
 66%|██████▌   | 24984/37962 [57:52:02<29:54:43,  8.30s/it]
 66%|██████▌   | 24985/37962 [57:52:10<29:43:59,  8.25s/it]
 66%|██████▌   | 24986/37962 [57:52:18<29:33:53,  8.20s/it]
 66%|██████▌   | 24987/37962 [57:52:26<29:37:47,  8.22s/it]
 66%|██████▌   | 24988/37962 [57:52:34<29:11:33,  8.10s/it]
 66%|██████▌   | 24989/37962 [57:52:43<29:34:55,  8.21s/it]
 66%|██████▌   | 24990/37962 [57:52:51<29:37:14,  8.22s/it]
                                                           
{'loss': 1.1287, 'grad_norm': 9.876240730285645, 'learning_rate': 5.5240272994081276e-06, 'epoch': 0.66}

 66%|██████▌   | 24990/37962 [57:52:51<29:37:14,  8.22s/it]
 66%|██████▌   | 24991/37962 [57:52:59<29:32:47,  8.20s/it]
 66%|██████▌   | 24992/37962 [57:53:07<29:35:35,  8.21s/it]
 66%|██████▌   | 24993/37962 [57:53:15<29:41:08,  8.24s/it]
 66%|██████▌   | 24994/37962 [57:53:24<29:43:04,  8.25s/it]
 66%|██████▌   | 24995/37962 [57:53:32<29:37:27,  8.22s/it]
 66%|██████▌   | 24996/37962 [57:53:40<29:27:45,  8.18s/it]
 66%|██████▌   | 24997/37962 [57:53:48<29:34:22,  8.21s/it]
 66%|██████▌   | 24998/37962 [57:53:57<29:46:14,  8.27s/it]
 66%|██████▌   | 24999/37962 [57:54:05<29:48:36,  8.28s/it]
 66%|██████▌   | 25000/37962 [57:54:13<29:44:18,  8.26s/it]
                                                           
{'loss': 1.244, 'grad_norm': 8.079313278198242, 'learning_rate': 5.5163996680085765e-06, 'epoch': 0.66}

 66%|██████▌   | 25000/37962 [57:54:13<29:44:18,  8.26s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 66%|██████▌   | 25001/37962 [57:55:07<78:50:27, 21.90s/it]
 66%|██████▌   | 25002/37962 [57:55:15<64:15:50, 17.85s/it]
 66%|██████▌   | 25003/37962 [57:55:24<53:56:27, 14.98s/it]
 66%|██████▌   | 25004/37962 [57:55:32<46:42:24, 12.98s/it]
 66%|██████▌   | 25005/37962 [57:55:40<41:09:15, 11.43s/it]
 66%|██████▌   | 25006/37962 [57:55:48<37:56:40, 10.54s/it]
 66%|██████▌   | 25007/37962 [57:55:56<35:26:44,  9.85s/it]
 66%|██████▌   | 25008/37962 [57:56:05<33:48:45,  9.40s/it]
 66%|██████▌   | 25009/37962 [57:56:13<32:35:25,  9.06s/it]
 66%|██████▌   | 25010/37962 [57:56:21<31:40:13,  8.80s/it]
                                                           
{'loss': 1.2519, 'grad_norm': 7.221560478210449, 'learning_rate': 5.508775300144036e-06, 'epoch': 0.66}

 66%|██████▌   | 25010/37962 [57:56:21<31:40:13,  8.80s/it]
 66%|██████▌   | 25011/37962 [57:56:30<31:05:48,  8.64s/it]
 66%|██████▌   | 25012/37962 [57:56:38<30:40:14,  8.53s/it]
 66%|██████▌   | 25013/37962 [57:56:46<30:21:11,  8.44s/it]
 66%|██████▌   | 25014/37962 [57:56:54<30:15:13,  8.41s/it]
 66%|██████▌   | 25015/37962 [57:57:03<30:00:56,  8.35s/it]
 66%|██████▌   | 25016/37962 [57:57:11<29:55:16,  8.32s/it]
 66%|██████▌   | 25017/37962 [57:57:19<30:08:40,  8.38s/it]
 66%|██████▌   | 25018/37962 [57:57:28<29:56:21,  8.33s/it]
 66%|██████▌   | 25019/37962 [57:57:36<29:50:19,  8.30s/it]
 66%|██████▌   | 25020/37962 [57:57:44<30:04:32,  8.37s/it]
                                                           
{'loss': 1.2583, 'grad_norm': 7.600493907928467, 'learning_rate': 5.501154201364151e-06, 'epoch': 0.66}

 66%|██████▌   | 25020/37962 [57:57:44<30:04:32,  8.37s/it]
 66%|██████▌   | 25021/37962 [57:57:53<29:55:44,  8.33s/it]
 66%|██████▌   | 25022/37962 [57:58:01<29:51:16,  8.31s/it]
 66%|██████▌   | 25023/37962 [57:58:09<29:40:40,  8.26s/it]
 66%|██████▌   | 25024/37962 [57:58:17<29:45:12,  8.28s/it]
 66%|██████▌   | 25025/37962 [57:58:25<29:38:51,  8.25s/it]
 66%|██████▌   | 25026/37962 [57:58:34<29:35:32,  8.24s/it]
 66%|██████▌   | 25027/37962 [57:58:42<29:21:32,  8.17s/it]
 66%|██████▌   | 25028/37962 [57:58:50<29:28:20,  8.20s/it]
 66%|██████▌   | 25029/37962 [57:58:58<29:33:30,  8.23s/it]
 66%|██████▌   | 25030/37962 [57:59:06<29:21:55,  8.17s/it]
                                                           
{'loss': 1.2936, 'grad_norm': 6.292059421539307, 'learning_rate': 5.493536377216181e-06, 'epoch': 0.66}

 66%|██████▌   | 25030/37962 [57:59:06<29:21:55,  8.17s/it]
 66%|██████▌   | 25031/37962 [57:59:14<28:58:26,  8.07s/it]
 66%|██████▌   | 25032/37962 [57:59:22<28:56:50,  8.06s/it]
 66%|██████▌   | 25033/37962 [57:59:30<29:13:21,  8.14s/it]
 66%|██████▌   | 25034/37962 [57:59:39<29:21:41,  8.18s/it]
 66%|██████▌   | 25035/37962 [57:59:47<29:29:26,  8.21s/it]
 66%|██████▌   | 25036/37962 [57:59:56<29:49:18,  8.31s/it]
 66%|██████▌   | 25037/37962 [58:00:03<29:24:56,  8.19s/it]
 66%|██████▌   | 25038/37962 [58:00:12<29:25:51,  8.20s/it]
 66%|██████▌   | 25039/37962 [58:00:20<29:40:51,  8.27s/it]
 66%|██████▌   | 25040/37962 [58:00:28<29:25:46,  8.20s/it]
                                                           
{'loss': 1.2549, 'grad_norm': 5.1319451332092285, 'learning_rate': 5.485921833245014e-06, 'epoch': 0.66}

 66%|██████▌   | 25040/37962 [58:00:28<29:25:46,  8.20s/it]
 66%|██████▌   | 25041/37962 [58:00:36<29:04:05,  8.10s/it]
 66%|██████▌   | 25042/37962 [58:00:44<29:24:22,  8.19s/it]
 66%|██████▌   | 25043/37962 [58:00:53<29:23:44,  8.19s/it]
 66%|██████▌   | 25044/37962 [58:01:01<29:37:45,  8.26s/it]
 66%|██████▌   | 25045/37962 [58:01:09<29:42:03,  8.28s/it]
 66%|██████▌   | 25046/37962 [58:01:18<29:39:54,  8.27s/it]
 66%|██████▌   | 25047/37962 [58:01:26<29:24:38,  8.20s/it]
 66%|██████▌   | 25048/37962 [58:01:34<29:35:21,  8.25s/it]
 66%|██████▌   | 25049/37962 [58:01:42<29:48:19,  8.31s/it]
 66%|██████▌   | 25050/37962 [58:01:51<29:50:01,  8.32s/it]
                                                           
{'loss': 1.2017, 'grad_norm': 7.589489936828613, 'learning_rate': 5.478310574993146e-06, 'epoch': 0.66}

 66%|██████▌   | 25050/37962 [58:01:51<29:50:01,  8.32s/it]
 66%|██████▌   | 25051/37962 [58:01:59<29:41:41,  8.28s/it]
 66%|██████▌   | 25052/37962 [58:02:07<29:40:22,  8.27s/it]
 66%|██████▌   | 25053/37962 [58:02:16<30:03:53,  8.38s/it]
 66%|██████▌   | 25054/37962 [58:02:24<29:56:17,  8.35s/it]
 66%|██████▌   | 25055/37962 [58:02:32<29:50:12,  8.32s/it]
 66%|██████▌   | 25056/37962 [58:02:40<29:19:23,  8.18s/it]
 66%|██████▌   | 25057/37962 [58:02:49<29:33:47,  8.25s/it]
 66%|██████▌   | 25058/37962 [58:02:57<29:31:12,  8.24s/it]
 66%|██████▌   | 25059/37962 [58:03:05<29:40:34,  8.28s/it]
 66%|██████▌   | 25060/37962 [58:03:14<29:41:40,  8.29s/it]
                                                           
{'loss': 1.2164, 'grad_norm': 11.390473365783691, 'learning_rate': 5.4707026080006735e-06, 'epoch': 0.66}

 66%|██████▌   | 25060/37962 [58:03:14<29:41:40,  8.29s/it]
 66%|██████▌   | 25061/37962 [58:03:22<29:39:51,  8.28s/it]
 66%|██████▌   | 25062/37962 [58:03:30<29:57:23,  8.36s/it]
 66%|██████▌   | 25063/37962 [58:03:39<29:50:45,  8.33s/it]
 66%|██████▌   | 25064/37962 [58:03:47<29:52:09,  8.34s/it]
 66%|██████▌   | 25065/37962 [58:03:55<29:42:04,  8.29s/it]
 66%|██████▌   | 25066/37962 [58:04:04<29:51:34,  8.34s/it]
 66%|██████▌   | 25067/37962 [58:04:11<29:10:45,  8.15s/it]
 66%|██████▌   | 25068/37962 [58:04:20<29:19:22,  8.19s/it]
 66%|██████▌   | 25069/37962 [58:04:28<29:21:24,  8.20s/it]
 66%|██████▌   | 25070/37962 [58:04:36<29:37:57,  8.27s/it]
                                                           
{'loss': 1.2473, 'grad_norm': 3.9240665435791016, 'learning_rate': 5.463097937805313e-06, 'epoch': 0.66}

 66%|██████▌   | 25070/37962 [58:04:36<29:37:57,  8.27s/it]
 66%|██████▌   | 25071/37962 [58:04:45<29:40:42,  8.29s/it]
 66%|██████▌   | 25072/37962 [58:04:53<29:37:07,  8.27s/it]
 66%|██████▌   | 25073/37962 [58:05:01<29:36:50,  8.27s/it]
 66%|██████▌   | 25074/37962 [58:05:09<29:43:24,  8.30s/it]
 66%|██████▌   | 25075/37962 [58:05:18<29:25:57,  8.22s/it]
 66%|██████▌   | 25076/37962 [58:05:26<29:28:50,  8.24s/it]
 66%|██████▌   | 25077/37962 [58:05:34<29:31:09,  8.25s/it]
 66%|██████▌   | 25078/37962 [58:05:42<29:17:10,  8.18s/it]
 66%|██████▌   | 25079/37962 [58:05:50<29:01:21,  8.11s/it]
 66%|██████▌   | 25080/37962 [58:05:58<29:16:29,  8.18s/it]
                                                           
{'loss': 1.2763, 'grad_norm': 6.549188613891602, 'learning_rate': 5.455496569942367e-06, 'epoch': 0.66}

 66%|██████▌   | 25080/37962 [58:05:58<29:16:29,  8.18s/it]
 66%|██████▌   | 25081/37962 [58:06:07<29:13:51,  8.17s/it]
 66%|██████▌   | 25082/37962 [58:06:15<29:27:03,  8.23s/it]
 66%|██████▌   | 25083/37962 [58:06:23<29:37:34,  8.28s/it]
 66%|██████▌   | 25084/37962 [58:06:32<29:33:41,  8.26s/it]
 66%|██████▌   | 25085/37962 [58:06:40<29:32:43,  8.26s/it]
 66%|██████▌   | 25086/37962 [58:06:48<29:37:00,  8.28s/it]
 66%|██████▌   | 25087/37962 [58:06:56<29:39:43,  8.29s/it]
 66%|██████▌   | 25088/37962 [58:07:05<29:36:23,  8.28s/it]
 66%|██████▌   | 25089/37962 [58:07:12<29:06:33,  8.14s/it]
 66%|██████▌   | 25090/37962 [58:07:21<29:14:56,  8.18s/it]
                                                           
{'loss': 1.2546, 'grad_norm': 4.976312637329102, 'learning_rate': 5.447898509944736e-06, 'epoch': 0.66}

 66%|██████▌   | 25090/37962 [58:07:21<29:14:56,  8.18s/it]
 66%|██████▌   | 25091/37962 [58:07:29<29:31:28,  8.26s/it]
 66%|██████▌   | 25092/37962 [58:07:38<29:37:01,  8.28s/it]
 66%|██████▌   | 25093/37962 [58:07:46<29:38:41,  8.29s/it]
 66%|██████▌   | 25094/37962 [58:07:54<29:06:31,  8.14s/it]
 66%|██████▌   | 25095/37962 [58:08:02<29:13:10,  8.18s/it]
 66%|██████▌   | 25096/37962 [58:08:10<29:17:15,  8.19s/it]
 66%|██████▌   | 25097/37962 [58:08:18<29:18:31,  8.20s/it]
 66%|██████▌   | 25098/37962 [58:08:27<29:28:36,  8.25s/it]
 66%|██████▌   | 25099/37962 [58:08:35<29:34:01,  8.27s/it]
 66%|██████▌   | 25100/37962 [58:08:43<29:20:00,  8.21s/it]
                                                           
{'loss': 1.2405, 'grad_norm': 18.180500030517578, 'learning_rate': 5.4403037633429175e-06, 'epoch': 0.66}

 66%|██████▌   | 25100/37962 [58:08:43<29:20:00,  8.21s/it]
 66%|██████▌   | 25101/37962 [58:08:52<29:32:43,  8.27s/it]
 66%|██████▌   | 25102/37962 [58:09:00<29:27:26,  8.25s/it]
 66%|██████▌   | 25103/37962 [58:09:08<29:51:33,  8.36s/it]
 66%|██████▌   | 25104/37962 [58:09:16<29:35:27,  8.28s/it]
 66%|██████▌   | 25105/37962 [58:09:25<29:28:33,  8.25s/it]
 66%|██████▌   | 25106/37962 [58:09:33<29:28:06,  8.25s/it]
 66%|██████▌   | 25107/37962 [58:09:41<29:26:59,  8.25s/it]
 66%|██████▌   | 25108/37962 [58:09:49<29:02:51,  8.14s/it]
 66%|██████▌   | 25109/37962 [58:09:57<28:55:58,  8.10s/it]
 66%|██████▌   | 25110/37962 [58:10:05<29:11:36,  8.18s/it]
                                                           
{'loss': 1.1637, 'grad_norm': 8.363852500915527, 'learning_rate': 5.432712335665005e-06, 'epoch': 0.66}

 66%|██████▌   | 25110/37962 [58:10:05<29:11:36,  8.18s/it]
 66%|██████▌   | 25111/37962 [58:10:14<29:23:14,  8.23s/it]
 66%|██████▌   | 25112/37962 [58:10:22<29:31:08,  8.27s/it]
 66%|██████▌   | 25113/37962 [58:10:30<29:28:51,  8.26s/it]
 66%|██████▌   | 25114/37962 [58:10:39<29:26:55,  8.25s/it]
 66%|██████▌   | 25115/37962 [58:10:47<29:23:36,  8.24s/it]
 66%|██████▌   | 25116/37962 [58:10:55<29:26:43,  8.25s/it]
 66%|██████▌   | 25117/37962 [58:11:03<29:15:10,  8.20s/it]
 66%|██████▌   | 25118/37962 [58:11:11<29:20:38,  8.22s/it]
 66%|██████▌   | 25119/37962 [58:11:20<29:26:07,  8.25s/it]
 66%|██████▌   | 25120/37962 [58:11:28<29:01:16,  8.14s/it]
                                                           
{'loss': 1.2357, 'grad_norm': 8.820781707763672, 'learning_rate': 5.42512423243666e-06, 'epoch': 0.66}

 66%|██████▌   | 25120/37962 [58:11:28<29:01:16,  8.14s/it]
 66%|██████▌   | 25121/37962 [58:11:36<29:22:18,  8.23s/it]
 66%|██████▌   | 25122/37962 [58:11:44<29:25:22,  8.25s/it]
 66%|██████▌   | 25123/37962 [58:11:53<29:21:55,  8.23s/it]
 66%|██████▌   | 25124/37962 [58:12:00<28:56:23,  8.12s/it]
 66%|██████▌   | 25125/37962 [58:12:09<29:04:30,  8.15s/it]
 66%|██████▌   | 25126/37962 [58:12:17<29:11:46,  8.19s/it]
 66%|██████▌   | 25127/37962 [58:12:25<29:29:46,  8.27s/it]
 66%|██████▌   | 25128/37962 [58:12:33<29:22:36,  8.24s/it]
 66%|██████▌   | 25129/37962 [58:12:42<29:23:22,  8.24s/it]
 66%|██████▌   | 25130/37962 [58:12:50<29:25:16,  8.25s/it]
                                                           
{'loss': 1.3383, 'grad_norm': 8.861018180847168, 'learning_rate': 5.417539459181127e-06, 'epoch': 0.66}

 66%|██████▌   | 25130/37962 [58:12:50<29:25:16,  8.25s/it]
 66%|██████▌   | 25131/37962 [58:12:58<29:34:57,  8.30s/it]
 66%|██████▌   | 25132/37962 [58:13:07<29:32:43,  8.29s/it]
 66%|██████▌   | 25133/37962 [58:13:15<29:27:13,  8.27s/it]
 66%|██████▌   | 25134/37962 [58:13:23<29:18:08,  8.22s/it]
 66%|██████▌   | 25135/37962 [58:13:31<29:07:16,  8.17s/it]
 66%|██████▌   | 25136/37962 [58:13:39<29:21:42,  8.24s/it]
 66%|██████▌   | 25137/37962 [58:13:48<29:28:24,  8.27s/it]
 66%|██████▌   | 25138/37962 [58:13:56<29:28:27,  8.27s/it]
 66%|██████▌   | 25139/37962 [58:14:04<29:16:46,  8.22s/it]
 66%|██████▌   | 25140/37962 [58:14:13<29:25:43,  8.26s/it]
                                                           
{'loss': 1.2333, 'grad_norm': 7.214784622192383, 'learning_rate': 5.409958021419241e-06, 'epoch': 0.66}

 66%|██████▌   | 25140/37962 [58:14:13<29:25:43,  8.26s/it]
 66%|██████▌   | 25141/37962 [58:14:21<29:04:25,  8.16s/it]
 66%|██████▌   | 25142/37962 [58:14:29<29:17:54,  8.23s/it]
 66%|██████▌   | 25143/37962 [58:14:37<29:16:14,  8.22s/it]
 66%|██████▌   | 25144/37962 [58:14:45<29:19:56,  8.24s/it]
 66%|██████▌   | 25145/37962 [58:14:53<29:03:57,  8.16s/it]
 66%|██████▌   | 25146/37962 [58:15:02<29:30:41,  8.29s/it]
 66%|██████▌   | 25147/37962 [58:15:10<29:33:03,  8.30s/it]
 66%|██████▌   | 25148/37962 [58:15:18<29:25:38,  8.27s/it]
 66%|██████▌   | 25149/37962 [58:15:27<29:24:37,  8.26s/it]
 66%|██████▋   | 25150/37962 [58:15:35<29:30:43,  8.29s/it]
                                                           
{'loss': 1.2782, 'grad_norm': 8.073467254638672, 'learning_rate': 5.402379924669389e-06, 'epoch': 0.66}

 66%|██████▋   | 25150/37962 [58:15:35<29:30:43,  8.29s/it]
 66%|██████▋   | 25151/37962 [58:15:43<29:25:49,  8.27s/it]
 66%|██████▋   | 25152/37962 [58:15:51<28:58:13,  8.14s/it]
 66%|██████▋   | 25153/37962 [58:15:59<29:04:00,  8.17s/it]
 66%|██████▋   | 25154/37962 [58:16:08<29:06:55,  8.18s/it]
 66%|██████▋   | 25155/37962 [58:16:16<29:16:43,  8.23s/it]
 66%|██████▋   | 25156/37962 [58:16:24<28:54:49,  8.13s/it]
 66%|██████▋   | 25157/37962 [58:16:32<29:19:40,  8.25s/it]
 66%|██████▋   | 25158/37962 [58:16:40<28:46:14,  8.09s/it]
 66%|██████▋   | 25159/37962 [58:16:48<28:59:42,  8.15s/it]
 66%|██████▋   | 25160/37962 [58:16:57<29:01:23,  8.16s/it]
                                                           
{'loss': 1.145, 'grad_norm': 8.478598594665527, 'learning_rate': 5.394805174447546e-06, 'epoch': 0.66}

 66%|██████▋   | 25160/37962 [58:16:57<29:01:23,  8.16s/it]
 66%|██████▋   | 25161/37962 [58:17:05<29:16:28,  8.23s/it]
 66%|██████▋   | 25162/37962 [58:17:13<29:19:09,  8.25s/it]
 66%|██████▋   | 25163/37962 [58:17:21<29:18:22,  8.24s/it]
 66%|██████▋   | 25164/37962 [58:17:30<29:24:48,  8.27s/it]
 66%|██████▋   | 25165/37962 [58:17:38<29:27:31,  8.29s/it]
 66%|██████▋   | 25166/37962 [58:17:46<29:25:05,  8.28s/it]
 66%|██████▋   | 25167/37962 [58:17:55<29:24:04,  8.27s/it]
 66%|██████▋   | 25168/37962 [58:18:03<29:26:29,  8.28s/it]
 66%|██████▋   | 25169/37962 [58:18:11<29:28:19,  8.29s/it]
 66%|██████▋   | 25170/37962 [58:18:19<29:06:58,  8.19s/it]
                                                           
{'loss': 1.3947, 'grad_norm': 10.074535369873047, 'learning_rate': 5.387233776267238e-06, 'epoch': 0.66}

 66%|██████▋   | 25170/37962 [58:18:19<29:06:58,  8.19s/it]
 66%|██████▋   | 25171/37962 [58:18:27<29:07:11,  8.20s/it]
 66%|██████▋   | 25172/37962 [58:18:36<29:31:35,  8.31s/it]
 66%|██████▋   | 25173/37962 [58:18:44<29:16:42,  8.24s/it]
 66%|██████▋   | 25174/37962 [58:18:52<29:14:42,  8.23s/it]
 66%|██████▋   | 25175/37962 [58:19:01<29:14:50,  8.23s/it]
 66%|██████▋   | 25176/37962 [58:19:09<29:17:56,  8.25s/it]
 66%|██████▋   | 25177/37962 [58:19:17<29:24:29,  8.28s/it]
 66%|██████▋   | 25178/37962 [58:19:25<29:22:36,  8.27s/it]
 66%|██████▋   | 25179/37962 [58:19:34<29:25:55,  8.29s/it]
 66%|██████▋   | 25180/37962 [58:19:50<37:41:26, 10.62s/it]
                                                           
{'loss': 1.3397, 'grad_norm': 7.660836696624756, 'learning_rate': 5.379665735639547e-06, 'epoch': 0.66}

 66%|██████▋   | 25180/37962 [58:19:50<37:41:26, 10.62s/it]
 66%|██████▋   | 25181/37962 [58:19:58<35:02:02,  9.87s/it]
 66%|██████▋   | 25182/37962 [58:20:06<33:21:23,  9.40s/it]
 66%|██████▋   | 25183/37962 [58:20:15<32:17:56,  9.10s/it]
 66%|██████▋   | 25184/37962 [58:20:23<31:26:20,  8.86s/it]
 66%|██████▋   | 25185/37962 [58:20:31<31:03:51,  8.75s/it]
 66%|██████▋   | 25186/37962 [58:20:40<30:31:18,  8.60s/it]
 66%|██████▋   | 25187/37962 [58:20:48<30:24:19,  8.57s/it]
 66%|██████▋   | 25188/37962 [58:20:56<29:40:54,  8.36s/it]
 66%|██████▋   | 25189/37962 [58:21:04<29:28:02,  8.31s/it]
 66%|██████▋   | 25190/37962 [58:21:12<29:23:20,  8.28s/it]
                                                           
{'loss': 1.0633, 'grad_norm': 3.5005226135253906, 'learning_rate': 5.372101058073131e-06, 'epoch': 0.66}

 66%|██████▋   | 25190/37962 [58:21:12<29:23:20,  8.28s/it]
 66%|██████▋   | 25191/37962 [58:21:21<29:26:06,  8.30s/it]
 66%|██████▋   | 25192/37962 [58:21:29<29:23:01,  8.28s/it]
 66%|██████▋   | 25193/37962 [58:21:37<29:06:58,  8.21s/it]
 66%|██████▋   | 25194/37962 [58:21:45<29:07:37,  8.21s/it]
 66%|██████▋   | 25195/37962 [58:21:53<28:44:43,  8.11s/it]
 66%|██████▋   | 25196/37962 [58:22:01<28:59:00,  8.17s/it]
 66%|██████▋   | 25197/37962 [58:22:10<29:12:57,  8.24s/it]
 66%|██████▋   | 25198/37962 [58:22:18<28:59:36,  8.18s/it]
 66%|██████▋   | 25199/37962 [58:22:26<29:01:38,  8.19s/it]
 66%|██████▋   | 25200/37962 [58:22:35<29:26:58,  8.31s/it]
                                                           
{'loss': 1.3575, 'grad_norm': 16.8642578125, 'learning_rate': 5.364539749074178e-06, 'epoch': 0.66}

 66%|██████▋   | 25200/37962 [58:22:35<29:26:58,  8.31s/it]
 66%|██████▋   | 25201/37962 [58:22:43<29:22:53,  8.29s/it]
 66%|██████▋   | 25202/37962 [58:22:51<29:07:49,  8.22s/it]
 66%|██████▋   | 25203/37962 [58:22:59<29:05:23,  8.21s/it]
 66%|██████▋   | 25204/37962 [58:23:07<28:58:45,  8.18s/it]
 66%|██████▋   | 25205/37962 [58:23:16<29:18:25,  8.27s/it]
 66%|██████▋   | 25206/37962 [58:23:24<28:50:45,  8.14s/it]
 66%|██████▋   | 25207/37962 [58:23:32<29:10:59,  8.24s/it]
 66%|██████▋   | 25208/37962 [58:23:40<29:10:46,  8.24s/it]
 66%|██████▋   | 25209/37962 [58:23:49<29:16:41,  8.26s/it]
 66%|██████▋   | 25210/37962 [58:23:57<29:14:47,  8.26s/it]
                                                           
{'loss': 1.1864, 'grad_norm': 12.858173370361328, 'learning_rate': 5.356981814146438e-06, 'epoch': 0.66}

 66%|██████▋   | 25210/37962 [58:23:57<29:14:47,  8.26s/it]
 66%|██████▋   | 25211/37962 [58:24:05<29:27:02,  8.31s/it]
 66%|██████▋   | 25212/37962 [58:24:14<29:37:46,  8.37s/it]
 66%|██████▋   | 25213/37962 [58:24:22<29:39:40,  8.38s/it]
 66%|██████▋   | 25214/37962 [58:24:31<29:43:28,  8.39s/it]
 66%|██████▋   | 25215/37962 [58:24:39<29:40:49,  8.38s/it]
 66%|██████▋   | 25216/37962 [58:24:47<29:31:48,  8.34s/it]
 66%|██████▋   | 25217/37962 [58:24:56<29:30:53,  8.34s/it]
 66%|██████▋   | 25218/37962 [58:25:04<29:28:28,  8.33s/it]
 66%|██████▋   | 25219/37962 [58:25:12<29:21:22,  8.29s/it]
 66%|██████▋   | 25220/37962 [58:25:20<29:25:32,  8.31s/it]
                                                           
{'loss': 1.2545, 'grad_norm': 5.893404006958008, 'learning_rate': 5.349427258791209e-06, 'epoch': 0.66}

 66%|██████▋   | 25220/37962 [58:25:20<29:25:32,  8.31s/it]
 66%|██████▋   | 25221/37962 [58:25:29<29:26:25,  8.32s/it]
 66%|██████▋   | 25222/37962 [58:25:37<29:23:23,  8.30s/it]
 66%|██████▋   | 25223/37962 [58:25:46<29:33:51,  8.35s/it]
 66%|██████▋   | 25224/37962 [58:25:54<29:32:38,  8.35s/it]
 66%|██████▋   | 25225/37962 [58:26:02<29:34:25,  8.36s/it]
 66%|██████▋   | 25226/37962 [58:26:10<29:25:19,  8.32s/it]
 66%|██████▋   | 25227/37962 [58:26:19<29:20:28,  8.29s/it]
 66%|██████▋   | 25228/37962 [58:26:27<29:08:30,  8.24s/it]
 66%|██████▋   | 25229/37962 [58:26:35<29:09:46,  8.25s/it]
 66%|██████▋   | 25230/37962 [58:26:43<29:09:09,  8.24s/it]
                                                           
{'loss': 1.3927, 'grad_norm': 9.388603210449219, 'learning_rate': 5.3418760885073026e-06, 'epoch': 0.66}

 66%|██████▋   | 25230/37962 [58:26:43<29:09:09,  8.24s/it]
 66%|██████▋   | 25231/37962 [58:26:51<28:58:42,  8.19s/it]
 66%|██████▋   | 25232/37962 [58:27:00<29:11:27,  8.26s/it]
 66%|██████▋   | 25233/37962 [58:27:08<29:13:53,  8.27s/it]
 66%|██████▋   | 25234/37962 [58:27:17<29:24:25,  8.32s/it]
 66%|██████▋   | 25235/37962 [58:27:24<28:59:57,  8.20s/it]
 66%|██████▋   | 25236/37962 [58:27:33<29:15:47,  8.28s/it]
 66%|██████▋   | 25237/37962 [58:27:41<29:11:58,  8.26s/it]
 66%|██████▋   | 25238/37962 [58:27:49<29:13:21,  8.27s/it]
 66%|██████▋   | 25239/37962 [58:27:58<29:06:31,  8.24s/it]
 66%|██████▋   | 25240/37962 [58:28:06<29:26:22,  8.33s/it]
                                                           
{'loss': 1.2672, 'grad_norm': 7.871059894561768, 'learning_rate': 5.334328308791097e-06, 'epoch': 0.66}

 66%|██████▋   | 25240/37962 [58:28:06<29:26:22,  8.33s/it]
 66%|██████▋   | 25241/37962 [58:28:14<29:29:13,  8.34s/it]
 66%|██████▋   | 25242/37962 [58:28:23<29:29:21,  8.35s/it]
 66%|██████▋   | 25243/37962 [58:28:31<29:41:48,  8.41s/it]
 66%|██████▋   | 25244/37962 [58:28:40<30:02:10,  8.50s/it]
 67%|██████▋   | 25245/37962 [58:28:48<29:50:33,  8.45s/it]
 67%|██████▋   | 25246/37962 [58:28:57<29:41:54,  8.41s/it]
 67%|██████▋   | 25247/37962 [58:29:05<29:34:08,  8.37s/it]
 67%|██████▋   | 25248/37962 [58:29:13<29:16:35,  8.29s/it]
 67%|██████▋   | 25249/37962 [58:29:22<29:20:53,  8.31s/it]
 67%|██████▋   | 25250/37962 [58:29:30<29:29:45,  8.35s/it]
                                                           
{'loss': 1.2542, 'grad_norm': 7.373490810394287, 'learning_rate': 5.326783925136491e-06, 'epoch': 0.67}

 67%|██████▋   | 25250/37962 [58:29:30<29:29:45,  8.35s/it]
 67%|██████▋   | 25251/37962 [58:29:38<29:39:41,  8.40s/it]
 67%|██████▋   | 25252/37962 [58:29:47<29:37:17,  8.39s/it]
 67%|██████▋   | 25253/37962 [58:29:55<29:32:42,  8.37s/it]
 67%|██████▋   | 25254/37962 [58:30:03<29:27:38,  8.35s/it]
 67%|██████▋   | 25255/37962 [58:30:12<29:26:48,  8.34s/it]
 67%|██████▋   | 25256/37962 [58:30:20<29:20:09,  8.31s/it]
 67%|██████▋   | 25257/37962 [58:30:28<29:21:11,  8.32s/it]
 67%|██████▋   | 25258/37962 [58:30:36<29:01:37,  8.23s/it]
 67%|██████▋   | 25259/37962 [58:30:45<29:05:50,  8.25s/it]
 67%|██████▋   | 25260/37962 [58:30:53<29:13:38,  8.28s/it]
                                                           
{'loss': 1.2704, 'grad_norm': 16.083660125732422, 'learning_rate': 5.319242943034906e-06, 'epoch': 0.67}

 67%|██████▋   | 25260/37962 [58:30:53<29:13:38,  8.28s/it]
 67%|██████▋   | 25261/37962 [58:31:01<29:21:36,  8.32s/it]
 67%|██████▋   | 25262/37962 [58:31:10<29:22:24,  8.33s/it]
 67%|██████▋   | 25263/37962 [58:31:18<29:23:15,  8.33s/it]
 67%|██████▋   | 25264/37962 [58:31:26<29:22:41,  8.33s/it]
 67%|██████▋   | 25265/37962 [58:31:35<29:30:03,  8.36s/it]
 67%|██████▋   | 25266/37962 [58:31:43<29:43:41,  8.43s/it]
 67%|██████▋   | 25267/37962 [58:31:52<29:41:53,  8.42s/it]
 67%|██████▋   | 25268/37962 [58:32:00<29:32:29,  8.38s/it]
 67%|██████▋   | 25269/37962 [58:32:08<28:56:21,  8.21s/it]
 67%|██████▋   | 25270/37962 [58:32:16<28:59:05,  8.22s/it]
                                                           
{'loss': 1.2866, 'grad_norm': 7.1136250495910645, 'learning_rate': 5.311705367975296e-06, 'epoch': 0.67}

 67%|██████▋   | 25270/37962 [58:32:16<28:59:05,  8.22s/it]
 67%|██████▋   | 25271/37962 [58:32:25<29:03:24,  8.24s/it]
 67%|██████▋   | 25272/37962 [58:32:33<29:03:42,  8.24s/it]
 67%|██████▋   | 25273/37962 [58:32:41<28:56:06,  8.21s/it]
 67%|██████▋   | 25274/37962 [58:32:49<29:07:30,  8.26s/it]
 67%|██████▋   | 25275/37962 [58:32:57<29:02:17,  8.24s/it]
 67%|██████▋   | 25276/37962 [58:33:06<29:16:27,  8.31s/it]
 67%|██████▋   | 25277/37962 [58:33:14<29:14:15,  8.30s/it]
 67%|██████▋   | 25278/37962 [58:33:23<29:18:49,  8.32s/it]
 67%|██████▋   | 25279/37962 [58:33:31<29:40:57,  8.43s/it]
 67%|██████▋   | 25280/37962 [58:33:40<29:36:22,  8.40s/it]
                                                           
{'loss': 1.2343, 'grad_norm': 5.501832008361816, 'learning_rate': 5.304171205444131e-06, 'epoch': 0.67}

 67%|██████▋   | 25280/37962 [58:33:40<29:36:22,  8.40s/it]
 67%|██████▋   | 25281/37962 [58:33:48<29:24:36,  8.35s/it]
 67%|██████▋   | 25282/37962 [58:33:56<29:28:39,  8.37s/it]
 67%|██████▋   | 25283/37962 [58:34:05<29:24:31,  8.35s/it]
 67%|██████▋   | 25284/37962 [58:34:13<29:25:01,  8.35s/it]
 67%|██████▋   | 25285/37962 [58:34:21<29:05:11,  8.26s/it]
 67%|██████▋   | 25286/37962 [58:34:29<29:15:10,  8.31s/it]
 67%|██████▋   | 25287/37962 [58:34:38<29:19:02,  8.33s/it]
 67%|██████▋   | 25288/37962 [58:34:46<29:19:41,  8.33s/it]
 67%|██████▋   | 25289/37962 [58:34:54<29:11:44,  8.29s/it]
 67%|██████▋   | 25290/37962 [58:35:03<29:11:19,  8.29s/it]
                                                           
{'loss': 1.2347, 'grad_norm': 5.678886413574219, 'learning_rate': 5.29664046092539e-06, 'epoch': 0.67}

 67%|██████▋   | 25290/37962 [58:35:03<29:11:19,  8.29s/it]
 67%|██████▋   | 25291/37962 [58:35:10<28:43:28,  8.16s/it]
 67%|██████▋   | 25292/37962 [58:35:19<28:54:10,  8.21s/it]
 67%|██████▋   | 25293/37962 [58:35:27<28:44:55,  8.17s/it]
 67%|██████▋   | 25294/37962 [58:35:35<28:58:39,  8.23s/it]
 67%|██████▋   | 25295/37962 [58:35:43<28:43:53,  8.17s/it]
 67%|██████▋   | 25296/37962 [58:35:52<28:59:39,  8.24s/it]
 67%|██████▋   | 25297/37962 [58:36:00<29:05:31,  8.27s/it]
 67%|██████▋   | 25298/37962 [58:36:08<29:08:16,  8.28s/it]
 67%|██████▋   | 25299/37962 [58:36:17<29:08:09,  8.28s/it]
 67%|██████▋   | 25300/37962 [58:36:25<29:03:58,  8.26s/it]
                                                           
{'loss': 1.2253, 'grad_norm': 10.199728965759277, 'learning_rate': 5.2891131399005834e-06, 'epoch': 0.67}

 67%|██████▋   | 25300/37962 [58:36:25<29:03:58,  8.26s/it]
 67%|██████▋   | 25301/37962 [58:36:33<29:27:00,  8.37s/it]
 67%|██████▋   | 25302/37962 [58:36:42<29:21:11,  8.35s/it]
 67%|██████▋   | 25303/37962 [58:36:50<29:18:45,  8.34s/it]
 67%|██████▋   | 25304/37962 [58:36:58<29:15:10,  8.32s/it]
 67%|██████▋   | 25305/37962 [58:37:06<29:00:22,  8.25s/it]
 67%|██████▋   | 25306/37962 [58:37:15<28:57:33,  8.24s/it]
 67%|██████▋   | 25307/37962 [58:37:23<29:14:57,  8.32s/it]
 67%|██████▋   | 25308/37962 [58:37:31<28:59:09,  8.25s/it]
 67%|██████▋   | 25309/37962 [58:37:39<28:59:58,  8.25s/it]
 67%|██████▋   | 25310/37962 [58:37:48<28:53:48,  8.22s/it]
                                                           
{'loss': 1.282, 'grad_norm': 7.2226433753967285, 'learning_rate': 5.281589247848707e-06, 'epoch': 0.67}

 67%|██████▋   | 25310/37962 [58:37:48<28:53:48,  8.22s/it]
 67%|██████▋   | 25311/37962 [58:37:56<28:47:18,  8.19s/it]
 67%|██████▋   | 25312/37962 [58:38:04<28:55:58,  8.23s/it]
 67%|██████▋   | 25313/37962 [58:38:12<28:43:26,  8.18s/it]
 67%|██████▋   | 25314/37962 [58:38:20<28:20:04,  8.06s/it]
 67%|██████▋   | 25315/37962 [58:38:28<28:17:40,  8.05s/it]
 67%|██████▋   | 25316/37962 [58:38:36<28:30:20,  8.11s/it]
 67%|██████▋   | 25317/37962 [58:38:44<28:21:22,  8.07s/it]
 67%|██████▋   | 25318/37962 [58:38:53<28:44:37,  8.18s/it]
 67%|██████▋   | 25319/37962 [58:39:01<28:37:30,  8.15s/it]
 67%|██████▋   | 25320/37962 [58:39:09<28:47:51,  8.20s/it]
                                                           
{'loss': 1.1734, 'grad_norm': 8.089058876037598, 'learning_rate': 5.274068790246276e-06, 'epoch': 0.67}

 67%|██████▋   | 25320/37962 [58:39:09<28:47:51,  8.20s/it]
 67%|██████▋   | 25321/37962 [58:39:17<28:44:38,  8.19s/it]
 67%|██████▋   | 25322/37962 [58:39:25<28:34:11,  8.14s/it]
 67%|██████▋   | 25323/37962 [58:39:33<28:40:50,  8.17s/it]
 67%|██████▋   | 25324/37962 [58:39:42<28:46:06,  8.19s/it]
 67%|██████▋   | 25325/37962 [58:39:50<28:58:41,  8.26s/it]
 67%|██████▋   | 25326/37962 [58:39:58<29:01:10,  8.27s/it]
 67%|██████▋   | 25327/37962 [58:40:07<29:02:44,  8.28s/it]
 67%|██████▋   | 25328/37962 [58:40:15<28:59:52,  8.26s/it]
 67%|██████▋   | 25329/37962 [58:40:23<29:02:53,  8.28s/it]
 67%|██████▋   | 25330/37962 [58:40:32<29:09:20,  8.31s/it]
                                                           
{'loss': 1.2023, 'grad_norm': 13.787459373474121, 'learning_rate': 5.2665517725673084e-06, 'epoch': 0.67}

 67%|██████▋   | 25330/37962 [58:40:32<29:09:20,  8.31s/it]
 67%|██████▋   | 25331/37962 [58:40:40<29:00:06,  8.27s/it]
 67%|██████▋   | 25332/37962 [58:40:48<28:51:06,  8.22s/it]
 67%|██████▋   | 25333/37962 [58:40:56<28:52:28,  8.23s/it]
 67%|██████▋   | 25334/37962 [58:41:04<28:53:35,  8.24s/it]
 67%|██████▋   | 25335/37962 [58:41:12<28:47:24,  8.21s/it]
 67%|██████▋   | 25336/37962 [58:41:21<28:48:21,  8.21s/it]
 67%|██████▋   | 25337/37962 [58:41:29<28:53:59,  8.24s/it]
 67%|██████▋   | 25338/37962 [58:41:37<28:56:13,  8.25s/it]
 67%|██████▋   | 25339/37962 [58:41:46<28:58:14,  8.26s/it]
 67%|██████▋   | 25340/37962 [58:41:54<28:42:55,  8.19s/it]
                                                           
{'loss': 1.2364, 'grad_norm': 6.604301452636719, 'learning_rate': 5.259038200283297e-06, 'epoch': 0.67}

 67%|██████▋   | 25340/37962 [58:41:54<28:42:55,  8.19s/it]
 67%|██████▋   | 25341/37962 [58:42:02<28:45:59,  8.21s/it]
 67%|██████▋   | 25342/37962 [58:42:10<28:42:09,  8.19s/it]
 67%|██████▋   | 25343/37962 [58:42:18<28:57:05,  8.26s/it]
 67%|██████▋   | 25344/37962 [58:42:27<29:03:29,  8.29s/it]
 67%|██████▋   | 25345/37962 [58:42:35<29:01:27,  8.28s/it]
 67%|██████▋   | 25346/37962 [58:42:43<28:48:00,  8.22s/it]
 67%|██████▋   | 25347/37962 [58:42:51<28:53:17,  8.24s/it]
 67%|██████▋   | 25348/37962 [58:42:59<28:29:39,  8.13s/it]
 67%|██████▋   | 25349/37962 [58:43:08<28:37:54,  8.17s/it]
 67%|██████▋   | 25350/37962 [58:43:16<28:41:20,  8.19s/it]
                                                           
{'loss': 1.3174, 'grad_norm': 22.778919219970703, 'learning_rate': 5.2515280788632494e-06, 'epoch': 0.67}

 67%|██████▋   | 25350/37962 [58:43:16<28:41:20,  8.19s/it]
 67%|██████▋   | 25351/37962 [58:43:24<28:41:23,  8.19s/it]
 67%|██████▋   | 25352/37962 [58:43:32<28:51:56,  8.24s/it]
 67%|██████▋   | 25353/37962 [58:43:41<28:47:08,  8.22s/it]
 67%|██████▋   | 25354/37962 [58:43:49<28:50:02,  8.23s/it]
 67%|██████▋   | 25355/37962 [58:43:57<28:56:07,  8.26s/it]
 67%|██████▋   | 25356/37962 [58:44:05<29:00:59,  8.29s/it]
 67%|██████▋   | 25357/37962 [58:44:14<28:57:13,  8.27s/it]
 67%|██████▋   | 25358/37962 [58:44:22<28:57:57,  8.27s/it]
 67%|██████▋   | 25359/37962 [58:44:30<29:07:50,  8.32s/it]
 67%|██████▋   | 25360/37962 [58:44:39<29:06:20,  8.31s/it]
                                                           
{'loss': 1.1681, 'grad_norm': 10.793685913085938, 'learning_rate': 5.244021413773654e-06, 'epoch': 0.67}

 67%|██████▋   | 25360/37962 [58:44:39<29:06:20,  8.31s/it]
 67%|██████▋   | 25361/37962 [58:44:47<29:13:49,  8.35s/it]
 67%|██████▋   | 25362/37962 [58:44:55<29:05:21,  8.31s/it]
 67%|██████▋   | 25363/37962 [58:45:03<28:54:50,  8.26s/it]
 67%|██████▋   | 25364/37962 [58:45:12<28:54:48,  8.26s/it]
 67%|██████▋   | 25365/37962 [58:45:20<28:53:18,  8.26s/it]
 67%|██████▋   | 25366/37962 [58:45:28<28:56:33,  8.27s/it]
 67%|██████▋   | 25367/37962 [58:45:45<37:23:04, 10.69s/it]
 67%|██████▋   | 25368/37962 [58:45:53<34:50:36,  9.96s/it]
 67%|██████▋   | 25369/37962 [58:46:01<33:21:07,  9.53s/it]
 67%|██████▋   | 25370/37962 [58:46:09<31:46:45,  9.09s/it]
                                                           
{'loss': 1.3077, 'grad_norm': 8.898537635803223, 'learning_rate': 5.236518210478477e-06, 'epoch': 0.67}

 67%|██████▋   | 25370/37962 [58:46:09<31:46:45,  9.09s/it]
 67%|██████▋   | 25371/37962 [58:46:18<31:02:47,  8.88s/it]
 67%|██████▋   | 25372/37962 [58:46:27<30:53:35,  8.83s/it]
 67%|██████▋   | 25373/37962 [58:46:35<30:20:16,  8.68s/it]
 67%|██████▋   | 25374/37962 [58:46:43<29:59:48,  8.58s/it]
 67%|██████▋   | 25375/37962 [58:46:52<29:41:05,  8.49s/it]
 67%|██████▋   | 25376/37962 [58:47:00<29:33:09,  8.45s/it]
 67%|██████▋   | 25377/37962 [58:47:08<29:31:48,  8.45s/it]
 67%|██████▋   | 25378/37962 [58:47:17<29:24:47,  8.41s/it]
 67%|██████▋   | 25379/37962 [58:47:25<29:12:26,  8.36s/it]
 67%|██████▋   | 25380/37962 [58:47:33<28:49:00,  8.25s/it]
                                                           
{'loss': 1.2737, 'grad_norm': 10.747381210327148, 'learning_rate': 5.229018474439176e-06, 'epoch': 0.67}

 67%|██████▋   | 25380/37962 [58:47:33<28:49:00,  8.25s/it]
 67%|██████▋   | 25381/37962 [58:47:41<28:56:40,  8.28s/it]
 67%|██████▋   | 25382/37962 [58:47:49<28:36:48,  8.19s/it]
 67%|██████▋   | 25383/37962 [58:47:57<28:38:40,  8.20s/it]
 67%|██████▋   | 25384/37962 [58:48:06<28:53:06,  8.27s/it]
 67%|██████▋   | 25385/37962 [58:48:14<28:55:13,  8.28s/it]
 67%|██████▋   | 25386/37962 [58:48:22<28:55:55,  8.28s/it]
 67%|██████▋   | 25387/37962 [58:48:30<28:33:34,  8.18s/it]
 67%|██████▋   | 25388/37962 [58:48:39<28:48:50,  8.25s/it]
 67%|██████▋   | 25389/37962 [58:48:47<28:58:13,  8.30s/it]
 67%|██████▋   | 25390/37962 [58:48:55<28:56:08,  8.29s/it]
                                                           
{'loss': 1.1539, 'grad_norm': 8.030888557434082, 'learning_rate': 5.221522211114676e-06, 'epoch': 0.67}

 67%|██████▋   | 25390/37962 [58:48:55<28:56:08,  8.29s/it]
 67%|██████▋   | 25391/37962 [58:49:04<29:14:11,  8.37s/it]
 67%|██████▋   | 25392/37962 [58:49:12<29:12:34,  8.37s/it]
 67%|██████▋   | 25393/37962 [58:49:21<29:05:27,  8.33s/it]
 67%|██████▋   | 25394/37962 [58:49:29<28:55:11,  8.28s/it]
 67%|██████▋   | 25395/37962 [58:49:37<28:52:50,  8.27s/it]
 67%|██████▋   | 25396/37962 [58:49:45<28:50:28,  8.26s/it]
 67%|██████▋   | 25397/37962 [58:49:54<29:03:17,  8.32s/it]
 67%|██████▋   | 25398/37962 [58:50:02<29:09:09,  8.35s/it]
 67%|██████▋   | 25399/37962 [58:50:11<29:21:14,  8.41s/it]
 67%|██████▋   | 25400/37962 [58:50:19<28:54:09,  8.28s/it]
                                                           
{'loss': 1.252, 'grad_norm': 13.118841171264648, 'learning_rate': 5.214029425961373e-06, 'epoch': 0.67}

 67%|██████▋   | 25400/37962 [58:50:19<28:54:09,  8.28s/it]
 67%|██████▋   | 25401/37962 [58:50:27<28:57:48,  8.30s/it]
 67%|██████▋   | 25402/37962 [58:50:35<28:54:21,  8.29s/it]
 67%|██████▋   | 25403/37962 [58:50:43<28:45:21,  8.24s/it]
 67%|██████▋   | 25404/37962 [58:50:52<28:42:52,  8.23s/it]
 67%|██████▋   | 25405/37962 [58:51:00<28:48:46,  8.26s/it]
 67%|██████▋   | 25406/37962 [58:51:09<29:05:11,  8.34s/it]
 67%|██████▋   | 25407/37962 [58:51:17<28:58:50,  8.31s/it]
 67%|██████▋   | 25408/37962 [58:51:25<28:40:52,  8.22s/it]
 67%|██████▋   | 25409/37962 [58:51:33<28:49:35,  8.27s/it]
 67%|██████▋   | 25410/37962 [58:51:41<28:48:28,  8.26s/it]
                                                           
{'loss': 1.2297, 'grad_norm': 6.408081531524658, 'learning_rate': 5.206540124433139e-06, 'epoch': 0.67}

 67%|██████▋   | 25410/37962 [58:51:41<28:48:28,  8.26s/it]
 67%|██████▋   | 25411/37962 [58:51:50<28:52:05,  8.28s/it]
 67%|██████▋   | 25412/37962 [58:51:58<28:46:50,  8.26s/it]
 67%|██████▋   | 25413/37962 [58:52:06<28:50:21,  8.27s/it]
 67%|██████▋   | 25414/37962 [58:52:15<28:58:47,  8.31s/it]
 67%|██████▋   | 25415/37962 [58:52:23<28:54:53,  8.30s/it]
 67%|██████▋   | 25416/37962 [58:52:31<28:54:45,  8.30s/it]
 67%|██████▋   | 25417/37962 [58:52:39<28:50:44,  8.28s/it]
 67%|██████▋   | 25418/37962 [58:52:48<28:56:29,  8.31s/it]
 67%|██████▋   | 25419/37962 [58:52:56<28:49:27,  8.27s/it]
 67%|██████▋   | 25420/37962 [58:53:04<28:50:23,  8.28s/it]
                                                           
{'loss': 1.2744, 'grad_norm': 11.952485084533691, 'learning_rate': 5.199054311981313e-06, 'epoch': 0.67}

 67%|██████▋   | 25420/37962 [58:53:04<28:50:23,  8.28s/it]
 67%|██████▋   | 25421/37962 [58:53:12<28:39:33,  8.23s/it]
 67%|██████▋   | 25422/37962 [58:53:20<28:27:27,  8.17s/it]
 67%|██████▋   | 25423/37962 [58:53:29<28:36:36,  8.21s/it]
 67%|██████▋   | 25424/37962 [58:53:37<28:38:47,  8.23s/it]
 67%|██████▋   | 25425/37962 [58:53:45<28:40:47,  8.24s/it]
 67%|██████▋   | 25426/37962 [58:53:54<28:48:33,  8.27s/it]
 67%|██████▋   | 25427/37962 [58:54:02<28:47:09,  8.27s/it]
 67%|██████▋   | 25428/37962 [58:54:10<28:39:11,  8.23s/it]
 67%|██████▋   | 25429/37962 [58:54:18<28:39:34,  8.23s/it]
 67%|██████▋   | 25430/37962 [58:54:26<28:39:03,  8.23s/it]
                                                           
{'loss': 1.2639, 'grad_norm': 13.3470458984375, 'learning_rate': 5.191571994054681e-06, 'epoch': 0.67}

 67%|██████▋   | 25430/37962 [58:54:26<28:39:03,  8.23s/it]
 67%|██████▋   | 25431/37962 [58:54:35<28:34:22,  8.21s/it]
 67%|██████▋   | 25432/37962 [58:54:43<28:49:42,  8.28s/it]
 67%|██████▋   | 25433/37962 [58:54:51<28:48:43,  8.28s/it]
 67%|██████▋   | 25434/37962 [58:54:59<28:25:58,  8.17s/it]
 67%|██████▋   | 25435/37962 [58:55:08<28:34:12,  8.21s/it]
 67%|██████▋   | 25436/37962 [58:55:16<28:41:34,  8.25s/it]
 67%|██████▋   | 25437/37962 [58:55:24<28:30:01,  8.19s/it]
 67%|██████▋   | 25438/37962 [58:55:32<28:29:42,  8.19s/it]
 67%|██████▋   | 25439/37962 [58:55:41<28:44:42,  8.26s/it]
 67%|██████▋   | 25440/37962 [58:55:49<28:44:09,  8.26s/it]
                                                           
{'loss': 1.2104, 'grad_norm': 4.612419128417969, 'learning_rate': 5.1840931760994915e-06, 'epoch': 0.67}

 67%|██████▋   | 25440/37962 [58:55:49<28:44:09,  8.26s/it]
 67%|██████▋   | 25441/37962 [58:55:57<28:46:42,  8.27s/it]
 67%|██████▋   | 25442/37962 [58:56:06<28:57:41,  8.33s/it]
 67%|██████▋   | 25443/37962 [58:56:14<29:00:07,  8.34s/it]
 67%|██████▋   | 25444/37962 [58:56:22<28:47:36,  8.28s/it]
 67%|██████▋   | 25445/37962 [58:56:31<29:03:18,  8.36s/it]
 67%|██████▋   | 25446/37962 [58:56:39<28:57:20,  8.33s/it]
 67%|██████▋   | 25447/37962 [58:56:47<29:02:35,  8.35s/it]
 67%|██████▋   | 25448/37962 [58:56:56<28:54:21,  8.32s/it]
 67%|██████▋   | 25449/37962 [58:57:04<28:50:11,  8.30s/it]
 67%|██████▋   | 25450/37962 [58:57:12<28:47:20,  8.28s/it]
                                                           
{'loss': 1.2359, 'grad_norm': 7.1024088859558105, 'learning_rate': 5.176617863559453e-06, 'epoch': 0.67}

 67%|██████▋   | 25450/37962 [58:57:12<28:47:20,  8.28s/it]
 67%|██████▋   | 25451/37962 [58:57:20<28:49:58,  8.30s/it]
 67%|██████▋   | 25452/37962 [58:57:29<28:48:10,  8.29s/it]
 67%|██████▋   | 25453/37962 [58:57:37<28:40:53,  8.25s/it]
 67%|██████▋   | 25454/37962 [58:57:45<28:20:46,  8.16s/it]
 67%|██████▋   | 25455/37962 [58:57:53<28:14:29,  8.13s/it]
 67%|██████▋   | 25456/37962 [58:58:01<28:28:46,  8.20s/it]
 67%|██████▋   | 25457/37962 [58:58:10<28:39:03,  8.25s/it]
 67%|██████▋   | 25458/37962 [58:58:18<28:42:22,  8.26s/it]
 67%|██████▋   | 25459/37962 [58:58:26<28:28:42,  8.20s/it]
 67%|██████▋   | 25460/37962 [58:58:34<28:44:28,  8.28s/it]
                                                           
{'loss': 1.2624, 'grad_norm': 7.2683515548706055, 'learning_rate': 5.169146061875709e-06, 'epoch': 0.67}

 67%|██████▋   | 25460/37962 [58:58:34<28:44:28,  8.28s/it]
 67%|██████▋   | 25461/37962 [58:58:43<28:47:43,  8.29s/it]
 67%|██████▋   | 25462/37962 [58:58:51<28:42:58,  8.27s/it]
 67%|██████▋   | 25463/37962 [58:58:59<28:37:21,  8.24s/it]
 67%|██████▋   | 25464/37962 [58:59:07<28:39:12,  8.25s/it]
 67%|██████▋   | 25465/37962 [58:59:16<28:34:22,  8.23s/it]
 67%|██████▋   | 25466/37962 [58:59:24<28:29:34,  8.21s/it]
 67%|██████▋   | 25467/37962 [58:59:32<28:26:40,  8.20s/it]
 67%|██████▋   | 25468/37962 [58:59:40<28:36:14,  8.24s/it]
 67%|██████▋   | 25469/37962 [58:59:48<28:36:52,  8.25s/it]
 67%|██████▋   | 25470/37962 [58:59:57<28:35:08,  8.24s/it]
                                                           
{'loss': 1.2968, 'grad_norm': 6.228296756744385, 'learning_rate': 5.16167777648686e-06, 'epoch': 0.67}

 67%|██████▋   | 25470/37962 [58:59:57<28:35:08,  8.24s/it]
 67%|██████▋   | 25471/37962 [59:00:05<28:40:00,  8.26s/it]
 67%|██████▋   | 25472/37962 [59:00:13<28:24:23,  8.19s/it]
 67%|██████▋   | 25473/37962 [59:00:21<28:37:47,  8.25s/it]
 67%|██████▋   | 25474/37962 [59:00:30<28:33:52,  8.23s/it]
 67%|██████▋   | 25475/37962 [59:00:38<28:49:44,  8.31s/it]
 67%|██████▋   | 25476/37962 [59:00:46<28:49:47,  8.31s/it]
 67%|██████▋   | 25477/37962 [59:00:55<28:51:08,  8.32s/it]
 67%|██████▋   | 25478/37962 [59:01:03<28:47:21,  8.30s/it]
 67%|██████▋   | 25479/37962 [59:01:11<28:45:21,  8.29s/it]
 67%|██████▋   | 25480/37962 [59:01:19<28:32:42,  8.23s/it]
                                                           
{'loss': 1.3243, 'grad_norm': 5.3247270584106445, 'learning_rate': 5.154213012828939e-06, 'epoch': 0.67}

 67%|██████▋   | 25480/37962 [59:01:19<28:32:42,  8.23s/it]
 67%|██████▋   | 25481/37962 [59:01:28<28:42:20,  8.28s/it]
 67%|██████▋   | 25482/37962 [59:01:36<28:41:07,  8.27s/it]
 67%|██████▋   | 25483/37962 [59:01:44<28:39:38,  8.27s/it]
 67%|██████▋   | 25484/37962 [59:01:53<28:44:06,  8.29s/it]
 67%|██████▋   | 25485/37962 [59:02:01<28:36:23,  8.25s/it]
 67%|██████▋   | 25486/37962 [59:02:09<28:44:27,  8.29s/it]
 67%|██████▋   | 25487/37962 [59:02:17<28:37:54,  8.26s/it]
 67%|██████▋   | 25488/37962 [59:02:26<28:45:07,  8.30s/it]
 67%|██████▋   | 25489/37962 [59:02:34<28:54:07,  8.34s/it]
 67%|██████▋   | 25490/37962 [59:02:43<28:51:06,  8.33s/it]
                                                           
{'loss': 1.1415, 'grad_norm': 4.917222023010254, 'learning_rate': 5.1467517763354166e-06, 'epoch': 0.67}

 67%|██████▋   | 25490/37962 [59:02:43<28:51:06,  8.33s/it]
 67%|██████▋   | 25491/37962 [59:02:51<28:49:24,  8.32s/it]
 67%|██████▋   | 25492/37962 [59:02:59<28:51:16,  8.33s/it]
 67%|██████▋   | 25493/37962 [59:03:07<28:45:43,  8.30s/it]
 67%|██████▋   | 25494/37962 [59:03:15<28:30:11,  8.23s/it]
 67%|██████▋   | 25495/37962 [59:03:24<28:42:27,  8.29s/it]
 67%|██████▋   | 25496/37962 [59:03:32<28:39:51,  8.28s/it]
 67%|██████▋   | 25497/37962 [59:03:40<28:36:25,  8.26s/it]
 67%|██████▋   | 25498/37962 [59:03:49<28:46:52,  8.31s/it]
 67%|██████▋   | 25499/37962 [59:03:57<28:38:25,  8.27s/it]
 67%|██████▋   | 25500/37962 [59:04:06<28:53:58,  8.35s/it]
                                                           
{'loss': 1.2499, 'grad_norm': 9.65062141418457, 'learning_rate': 5.139294072437202e-06, 'epoch': 0.67}

 67%|██████▋   | 25500/37962 [59:04:06<28:53:58,  8.35s/it]
 67%|██████▋   | 25501/37962 [59:04:14<28:48:58,  8.33s/it]
 67%|██████▋   | 25502/37962 [59:04:22<28:25:06,  8.21s/it]
 67%|██████▋   | 25503/37962 [59:04:30<28:28:42,  8.23s/it]
 67%|██████▋   | 25504/37962 [59:04:38<28:39:40,  8.28s/it]
 67%|██████▋   | 25505/37962 [59:04:47<28:33:49,  8.25s/it]
 67%|██████▋   | 25506/37962 [59:04:55<28:31:55,  8.25s/it]
 67%|██████▋   | 25507/37962 [59:05:03<28:33:17,  8.25s/it]
 67%|██████▋   | 25508/37962 [59:05:12<28:49:27,  8.33s/it]
 67%|██████▋   | 25509/37962 [59:05:20<28:42:17,  8.30s/it]
 67%|██████▋   | 25510/37962 [59:05:28<28:40:38,  8.29s/it]
                                                           
{'loss': 1.3418, 'grad_norm': 7.8579840660095215, 'learning_rate': 5.131839906562623e-06, 'epoch': 0.67}

 67%|██████▋   | 25510/37962 [59:05:28<28:40:38,  8.29s/it]
 67%|██████▋   | 25511/37962 [59:05:36<28:19:03,  8.19s/it]
 67%|██████▋   | 25512/37962 [59:05:44<27:55:27,  8.07s/it]
 67%|██████▋   | 25513/37962 [59:05:52<28:19:54,  8.19s/it]
 67%|██████▋   | 25514/37962 [59:06:01<28:34:34,  8.26s/it]
 67%|██████▋   | 25515/37962 [59:06:09<28:18:12,  8.19s/it]
 67%|██████▋   | 25516/37962 [59:06:17<28:23:59,  8.21s/it]
 67%|██████▋   | 25517/37962 [59:06:25<28:03:46,  8.12s/it]
 67%|██████▋   | 25518/37962 [59:06:33<28:25:07,  8.22s/it]
 67%|██████▋   | 25519/37962 [59:06:42<28:32:50,  8.26s/it]
 67%|██████▋   | 25520/37962 [59:06:50<28:36:22,  8.28s/it]
                                                           
{'loss': 1.1668, 'grad_norm': 10.6325044631958, 'learning_rate': 5.12438928413744e-06, 'epoch': 0.67}

 67%|██████▋   | 25520/37962 [59:06:50<28:36:22,  8.28s/it]
 67%|██████▋   | 25521/37962 [59:06:58<28:34:09,  8.27s/it]
 67%|██████▋   | 25522/37962 [59:07:07<28:34:57,  8.27s/it]
 67%|██████▋   | 25523/37962 [59:07:15<28:43:05,  8.31s/it]
 67%|██████▋   | 25524/37962 [59:07:23<28:39:25,  8.29s/it]
 67%|██████▋   | 25525/37962 [59:07:32<28:54:18,  8.37s/it]
 67%|██████▋   | 25526/37962 [59:07:40<28:45:09,  8.32s/it]
 67%|██████▋   | 25527/37962 [59:07:48<28:42:04,  8.31s/it]
 67%|██████▋   | 25528/37962 [59:07:57<28:42:11,  8.31s/it]
 67%|██████▋   | 25529/37962 [59:08:05<28:32:03,  8.26s/it]
 67%|██████▋   | 25530/37962 [59:08:13<28:25:32,  8.23s/it]
                                                           
{'loss': 1.3203, 'grad_norm': 17.93938446044922, 'learning_rate': 5.116942210584836e-06, 'epoch': 0.67}

 67%|██████▋   | 25530/37962 [59:08:13<28:25:32,  8.23s/it]
 67%|██████▋   | 25531/37962 [59:08:21<28:31:10,  8.26s/it]
 67%|██████▋   | 25532/37962 [59:08:30<28:42:31,  8.31s/it]
 67%|██████▋   | 25533/37962 [59:08:38<28:51:40,  8.36s/it]
 67%|██████▋   | 25534/37962 [59:08:46<28:37:47,  8.29s/it]
 67%|██████▋   | 25535/37962 [59:08:55<28:40:09,  8.31s/it]
 67%|██████▋   | 25536/37962 [59:09:03<28:50:25,  8.36s/it]
 67%|██████▋   | 25537/37962 [59:09:11<28:43:49,  8.32s/it]
 67%|██████▋   | 25538/37962 [59:09:20<28:55:15,  8.38s/it]
 67%|██████▋   | 25539/37962 [59:09:28<28:51:25,  8.36s/it]
 67%|██████▋   | 25540/37962 [59:09:37<29:04:44,  8.43s/it]
                                                           
{'loss': 1.3064, 'grad_norm': 14.400424003601074, 'learning_rate': 5.109498691325407e-06, 'epoch': 0.67}

 67%|██████▋   | 25540/37962 [59:09:37<29:04:44,  8.43s/it]
 67%|██████▋   | 25541/37962 [59:09:45<28:32:41,  8.27s/it]
 67%|██████▋   | 25542/37962 [59:10:00<36:09:06, 10.48s/it]
 67%|██████▋   | 25543/37962 [59:10:09<34:10:45,  9.91s/it]
 67%|██████▋   | 25544/37962 [59:10:17<32:23:48,  9.39s/it]
 67%|██████▋   | 25545/37962 [59:10:26<31:26:24,  9.12s/it]
 67%|██████▋   | 25546/37962 [59:10:34<30:36:40,  8.88s/it]
 67%|██████▋   | 25547/37962 [59:10:42<30:08:55,  8.74s/it]
 67%|██████▋   | 25548/37962 [59:10:50<29:23:59,  8.53s/it]
 67%|██████▋   | 25549/37962 [59:10:59<29:13:17,  8.47s/it]
 67%|██████▋   | 25550/37962 [59:11:07<28:39:21,  8.31s/it]
                                                           
{'loss': 1.2811, 'grad_norm': 4.8820719718933105, 'learning_rate': 5.102058731777152e-06, 'epoch': 0.67}

 67%|██████▋   | 25550/37962 [59:11:07<28:39:21,  8.31s/it]
 67%|██████▋   | 25551/37962 [59:11:14<28:14:03,  8.19s/it]
 67%|██████▋   | 25552/37962 [59:11:23<28:27:22,  8.25s/it]
 67%|██████▋   | 25553/37962 [59:11:31<28:20:40,  8.22s/it]
 67%|██████▋   | 25554/37962 [59:11:39<28:29:14,  8.27s/it]
 67%|██████▋   | 25555/37962 [59:11:48<28:30:25,  8.27s/it]
 67%|██████▋   | 25556/37962 [59:11:56<28:18:49,  8.22s/it]
 67%|██████▋   | 25557/37962 [59:12:04<28:28:17,  8.26s/it]
 67%|██████▋   | 25558/37962 [59:12:12<28:17:22,  8.21s/it]
 67%|██████▋   | 25559/37962 [59:12:20<28:15:39,  8.20s/it]
 67%|██████▋   | 25560/37962 [59:12:29<28:20:15,  8.23s/it]
                                                           
{'loss': 1.2357, 'grad_norm': 14.821746826171875, 'learning_rate': 5.094622337355499e-06, 'epoch': 0.67}

 67%|██████▋   | 25560/37962 [59:12:29<28:20:15,  8.23s/it]
 67%|██████▋   | 25561/37962 [59:12:37<28:31:34,  8.28s/it]
 67%|██████▋   | 25562/37962 [59:12:45<28:30:25,  8.28s/it]
 67%|██████▋   | 25563/37962 [59:12:54<28:29:07,  8.27s/it]
 67%|██████▋   | 25564/37962 [59:13:02<28:34:46,  8.30s/it]
 67%|██████▋   | 25565/37962 [59:13:10<28:31:02,  8.28s/it]
 67%|██████▋   | 25566/37962 [59:13:18<28:22:06,  8.24s/it]
 67%|██████▋   | 25567/37962 [59:13:27<28:19:13,  8.23s/it]
 67%|██████▋   | 25568/37962 [59:13:35<28:35:17,  8.30s/it]
 67%|██████▋   | 25569/37962 [59:13:44<28:44:58,  8.35s/it]
 67%|██████▋   | 25570/37962 [59:13:51<28:20:20,  8.23s/it]
                                                           
{'loss': 1.2265, 'grad_norm': 5.6725263595581055, 'learning_rate': 5.087189513473263e-06, 'epoch': 0.67}

 67%|██████▋   | 25570/37962 [59:13:51<28:20:20,  8.23s/it]
 67%|██████▋   | 25571/37962 [59:14:00<28:18:28,  8.22s/it]
 67%|██████▋   | 25572/37962 [59:14:08<28:10:09,  8.18s/it]
 67%|██████▋   | 25573/37962 [59:14:16<28:14:30,  8.21s/it]
 67%|██████▋   | 25574/37962 [59:14:24<28:16:04,  8.21s/it]
 67%|██████▋   | 25575/37962 [59:14:32<28:17:12,  8.22s/it]
 67%|██████▋   | 25576/37962 [59:14:41<28:15:45,  8.21s/it]
 67%|██████▋   | 25577/37962 [59:14:49<28:05:18,  8.16s/it]
 67%|██████▋   | 25578/37962 [59:14:57<27:57:46,  8.13s/it]
 67%|██████▋   | 25579/37962 [59:15:05<28:18:00,  8.23s/it]
 67%|██████▋   | 25580/37962 [59:15:13<28:19:47,  8.24s/it]
                                                           
{'loss': 1.181, 'grad_norm': 9.489623069763184, 'learning_rate': 5.079760265540677e-06, 'epoch': 0.67}

 67%|██████▋   | 25580/37962 [59:15:14<28:19:47,  8.24s/it]
 67%|██████▋   | 25581/37962 [59:15:22<28:23:13,  8.25s/it]
 67%|██████▋   | 25582/37962 [59:15:30<28:29:43,  8.29s/it]
 67%|██████▋   | 25583/37962 [59:15:38<28:26:57,  8.27s/it]
 67%|██████▋   | 25584/37962 [59:15:47<28:22:21,  8.25s/it]
 67%|██████▋   | 25585/37962 [59:15:55<28:34:25,  8.31s/it]
 67%|██████▋   | 25586/37962 [59:16:03<28:34:40,  8.31s/it]
 67%|██████▋   | 25587/37962 [59:16:12<28:48:48,  8.38s/it]
 67%|██████▋   | 25588/37962 [59:16:20<28:47:40,  8.38s/it]
 67%|██████▋   | 25589/37962 [59:16:28<28:36:01,  8.32s/it]
 67%|██████▋   | 25590/37962 [59:16:37<28:37:33,  8.33s/it]
                                                           
{'loss': 1.3302, 'grad_norm': 12.100436210632324, 'learning_rate': 5.072334598965355e-06, 'epoch': 0.67}

 67%|██████▋   | 25590/37962 [59:16:37<28:37:33,  8.33s/it]
 67%|██████▋   | 25591/37962 [59:16:45<28:35:31,  8.32s/it]
 67%|██████▋   | 25592/37962 [59:16:53<28:30:35,  8.30s/it]
 67%|██████▋   | 25593/37962 [59:17:02<28:33:51,  8.31s/it]
 67%|██████▋   | 25594/37962 [59:17:10<28:30:13,  8.30s/it]
 67%|██████▋   | 25595/37962 [59:17:18<28:39:23,  8.34s/it]
 67%|██████▋   | 25596/37962 [59:17:26<28:20:11,  8.25s/it]
 67%|██████▋   | 25597/37962 [59:17:35<28:21:17,  8.26s/it]
 67%|██████▋   | 25598/37962 [59:17:43<28:27:27,  8.29s/it]
 67%|██████▋   | 25599/37962 [59:17:51<28:27:45,  8.29s/it]
 67%|██████▋   | 25600/37962 [59:17:59<28:12:31,  8.21s/it]
                                                           
{'loss': 1.187, 'grad_norm': 3.5319018363952637, 'learning_rate': 5.064912519152308e-06, 'epoch': 0.67}

 67%|██████▋   | 25600/37962 [59:17:59<28:12:31,  8.21s/it]
 67%|██████▋   | 25601/37962 [59:18:08<28:07:59,  8.19s/it]
 67%|██████▋   | 25602/37962 [59:18:16<27:58:43,  8.15s/it]
 67%|██████▋   | 25603/37962 [59:18:24<27:44:25,  8.08s/it]
 67%|██████▋   | 25604/37962 [59:18:32<27:54:47,  8.13s/it]
 67%|██████▋   | 25605/37962 [59:18:40<27:32:55,  8.03s/it]
 67%|██████▋   | 25606/37962 [59:18:48<27:39:41,  8.06s/it]
 67%|██████▋   | 25607/37962 [59:18:56<27:57:53,  8.15s/it]
 67%|██████▋   | 25608/37962 [59:19:04<27:52:37,  8.12s/it]
 67%|██████▋   | 25609/37962 [59:19:12<27:57:47,  8.15s/it]
 67%|██████▋   | 25610/37962 [59:19:21<28:06:28,  8.19s/it]
                                                           
{'loss': 1.2882, 'grad_norm': 7.3396453857421875, 'learning_rate': 5.0574940315039485e-06, 'epoch': 0.67}

 67%|██████▋   | 25610/37962 [59:19:21<28:06:28,  8.19s/it]
 67%|██████▋   | 25611/37962 [59:19:29<27:59:40,  8.16s/it]
 67%|██████▋   | 25612/37962 [59:19:37<28:13:40,  8.23s/it]
 67%|██████▋   | 25613/37962 [59:19:45<28:18:51,  8.25s/it]
 67%|██████▋   | 25614/37962 [59:19:53<28:04:40,  8.19s/it]
 67%|██████▋   | 25615/37962 [59:20:02<28:14:46,  8.24s/it]
 67%|██████▋   | 25616/37962 [59:20:10<28:11:31,  8.22s/it]
 67%|██████▋   | 25617/37962 [59:20:18<28:11:00,  8.22s/it]
 67%|██████▋   | 25618/37962 [59:20:26<27:53:48,  8.14s/it]
 67%|██████▋   | 25619/37962 [59:20:34<27:57:23,  8.15s/it]
 67%|██████▋   | 25620/37962 [59:20:43<28:05:32,  8.19s/it]
                                                           
{'loss': 1.2141, 'grad_norm': 9.092619895935059, 'learning_rate': 5.050079141420057e-06, 'epoch': 0.67}

 67%|██████▋   | 25620/37962 [59:20:43<28:05:32,  8.19s/it]
 67%|██████▋   | 25621/37962 [59:20:51<27:50:00,  8.12s/it]
 67%|██████▋   | 25622/37962 [59:20:59<28:00:56,  8.17s/it]
 67%|██████▋   | 25623/37962 [59:21:07<28:01:03,  8.17s/it]
 67%|██████▋   | 25624/37962 [59:21:15<28:09:59,  8.22s/it]
 68%|██████▊   | 25625/37962 [59:21:24<28:22:31,  8.28s/it]
 68%|██████▊   | 25626/37962 [59:21:32<28:14:21,  8.24s/it]
 68%|██████▊   | 25627/37962 [59:21:40<28:23:04,  8.28s/it]
 68%|██████▊   | 25628/37962 [59:21:49<28:27:12,  8.30s/it]
 68%|██████▊   | 25629/37962 [59:21:57<28:29:12,  8.32s/it]
 68%|██████▊   | 25630/37962 [59:22:05<28:22:04,  8.28s/it]
                                                           
{'loss': 1.245, 'grad_norm': 6.60184383392334, 'learning_rate': 5.042667854297807e-06, 'epoch': 0.68}

 68%|██████▊   | 25630/37962 [59:22:05<28:22:04,  8.28s/it]
 68%|██████▊   | 25631/37962 [59:22:13<28:11:18,  8.23s/it]
 68%|██████▊   | 25632/37962 [59:22:22<28:17:41,  8.26s/it]
 68%|██████▊   | 25633/37962 [59:22:30<28:23:08,  8.29s/it]
 68%|██████▊   | 25634/37962 [59:22:38<28:12:44,  8.24s/it]
 68%|██████▊   | 25635/37962 [59:22:46<27:55:57,  8.16s/it]
 68%|██████▊   | 25636/37962 [59:22:54<28:07:47,  8.22s/it]
 68%|██████▊   | 25637/37962 [59:23:03<28:08:16,  8.22s/it]
 68%|██████▊   | 25638/37962 [59:23:11<28:03:11,  8.19s/it]
 68%|██████▊   | 25639/37962 [59:23:19<28:05:01,  8.20s/it]
 68%|██████▊   | 25640/37962 [59:23:27<27:48:49,  8.13s/it]
                                                           
{'loss': 1.2344, 'grad_norm': 14.064559936523438, 'learning_rate': 5.035260175531749e-06, 'epoch': 0.68}

 68%|██████▊   | 25640/37962 [59:23:27<27:48:49,  8.13s/it]
 68%|██████▊   | 25641/37962 [59:23:35<28:01:08,  8.19s/it]
 68%|██████▊   | 25642/37962 [59:23:44<28:12:07,  8.24s/it]
 68%|██████▊   | 25643/37962 [59:23:52<27:59:12,  8.18s/it]
 68%|██████▊   | 25644/37962 [59:24:00<28:05:27,  8.21s/it]
 68%|██████▊   | 25645/37962 [59:24:09<28:25:33,  8.31s/it]
 68%|██████▊   | 25646/37962 [59:24:17<28:15:53,  8.26s/it]
 68%|██████▊   | 25647/37962 [59:24:25<28:14:56,  8.26s/it]
 68%|██████▊   | 25648/37962 [59:24:33<28:08:09,  8.23s/it]
 68%|██████▊   | 25649/37962 [59:24:41<28:00:55,  8.19s/it]
 68%|██████▊   | 25650/37962 [59:24:50<28:12:23,  8.25s/it]
                                                           
{'loss': 1.2411, 'grad_norm': 4.638503551483154, 'learning_rate': 5.027856110513802e-06, 'epoch': 0.68}

 68%|██████▊   | 25650/37962 [59:24:50<28:12:23,  8.25s/it]
 68%|██████▊   | 25651/37962 [59:24:58<28:08:31,  8.23s/it]
 68%|██████▊   | 25652/37962 [59:25:06<28:11:12,  8.24s/it]
 68%|██████▊   | 25653/37962 [59:25:14<28:20:21,  8.29s/it]
 68%|██████▊   | 25654/37962 [59:25:23<28:13:10,  8.25s/it]
 68%|██████▊   | 25655/37962 [59:25:31<28:21:37,  8.30s/it]
 68%|██████▊   | 25656/37962 [59:25:39<28:03:23,  8.21s/it]
 68%|██████▊   | 25657/37962 [59:25:48<28:27:18,  8.32s/it]
 68%|██████▊   | 25658/37962 [59:25:56<28:31:29,  8.35s/it]
 68%|██████▊   | 25659/37962 [59:26:04<28:28:15,  8.33s/it]
 68%|██████▊   | 25660/37962 [59:26:12<28:15:56,  8.27s/it]
                                                           
{'loss': 1.1698, 'grad_norm': 13.772846221923828, 'learning_rate': 5.02045566463325e-06, 'epoch': 0.68}

 68%|██████▊   | 25660/37962 [59:26:12<28:15:56,  8.27s/it]
 68%|██████▊   | 25661/37962 [59:26:21<28:14:20,  8.26s/it]
 68%|██████▊   | 25662/37962 [59:26:29<28:16:24,  8.28s/it]
 68%|██████▊   | 25663/37962 [59:26:37<28:19:20,  8.29s/it]
 68%|██████▊   | 25664/37962 [59:26:45<28:12:23,  8.26s/it]
 68%|██████▊   | 25665/37962 [59:26:53<27:51:20,  8.15s/it]
 68%|██████▊   | 25666/37962 [59:27:02<28:06:28,  8.23s/it]
 68%|██████▊   | 25667/37962 [59:27:10<28:07:45,  8.24s/it]
 68%|██████▊   | 25668/37962 [59:27:18<28:15:51,  8.28s/it]
 68%|██████▊   | 25669/37962 [59:27:26<28:00:57,  8.20s/it]
 68%|██████▊   | 25670/37962 [59:27:35<28:06:47,  8.23s/it]
                                                           
{'loss': 1.2528, 'grad_norm': 4.584293842315674, 'learning_rate': 5.01305884327676e-06, 'epoch': 0.68}

 68%|██████▊   | 25670/37962 [59:27:35<28:06:47,  8.23s/it]
 68%|██████▊   | 25671/37962 [59:27:43<27:50:38,  8.16s/it]
 68%|██████▊   | 25672/37962 [59:27:51<28:01:26,  8.21s/it]
 68%|██████▊   | 25673/37962 [59:27:59<27:59:10,  8.20s/it]
 68%|██████▊   | 25674/37962 [59:28:07<27:35:58,  8.09s/it]
 68%|██████▊   | 25675/37962 [59:28:15<27:53:36,  8.17s/it]
 68%|██████▊   | 25676/37962 [59:28:24<27:55:28,  8.18s/it]
 68%|██████▊   | 25677/37962 [59:28:32<27:40:08,  8.11s/it]
 68%|██████▊   | 25678/37962 [59:28:40<27:38:16,  8.10s/it]
 68%|██████▊   | 25679/37962 [59:28:48<27:46:10,  8.14s/it]
 68%|██████▊   | 25680/37962 [59:28:56<28:08:44,  8.25s/it]
                                                           
{'loss': 1.3255, 'grad_norm': 6.553313732147217, 'learning_rate': 5.005665651828342e-06, 'epoch': 0.68}

 68%|██████▊   | 25680/37962 [59:28:56<28:08:44,  8.25s/it]
 68%|██████▊   | 25681/37962 [59:29:05<28:19:59,  8.31s/it]
 68%|██████▊   | 25682/37962 [59:29:13<28:18:57,  8.30s/it]
 68%|██████▊   | 25683/37962 [59:29:21<28:17:05,  8.29s/it]
 68%|██████▊   | 25684/37962 [59:29:30<28:15:03,  8.28s/it]
 68%|██████▊   | 25685/37962 [59:29:38<28:14:45,  8.28s/it]
 68%|██████▊   | 25686/37962 [59:29:46<28:12:03,  8.27s/it]
 68%|██████▊   | 25687/37962 [59:29:55<28:20:51,  8.31s/it]
 68%|██████▊   | 25688/37962 [59:30:03<28:24:24,  8.33s/it]
 68%|██████▊   | 25689/37962 [59:30:11<28:25:15,  8.34s/it]
 68%|██████▊   | 25690/37962 [59:30:20<28:20:24,  8.31s/it]
                                                           
{'loss': 1.244, 'grad_norm': 5.754970550537109, 'learning_rate': 4.998276095669377e-06, 'epoch': 0.68}

 68%|██████▊   | 25690/37962 [59:30:20<28:20:24,  8.31s/it]
 68%|██████▊   | 25691/37962 [59:30:28<28:13:52,  8.28s/it]
 68%|██████▊   | 25692/37962 [59:30:36<28:16:45,  8.30s/it]
 68%|██████▊   | 25693/37962 [59:30:44<28:22:40,  8.33s/it]
 68%|██████▊   | 25694/37962 [59:30:53<28:21:17,  8.32s/it]
 68%|██████▊   | 25695/37962 [59:31:01<28:18:28,  8.31s/it]
 68%|██████▊   | 25696/37962 [59:31:09<28:23:15,  8.33s/it]
 68%|██████▊   | 25697/37962 [59:31:18<28:20:01,  8.32s/it]
 68%|██████▊   | 25698/37962 [59:31:26<28:14:53,  8.29s/it]
 68%|██████▊   | 25699/37962 [59:31:34<28:13:11,  8.28s/it]
 68%|██████▊   | 25700/37962 [59:31:43<28:36:05,  8.40s/it]
                                                           
{'loss': 1.2931, 'grad_norm': 4.153213024139404, 'learning_rate': 4.990890180178595e-06, 'epoch': 0.68}

 68%|██████▊   | 25700/37962 [59:31:43<28:36:05,  8.40s/it]
 68%|██████▊   | 25701/37962 [59:31:51<28:03:30,  8.24s/it]
 68%|██████▊   | 25702/37962 [59:31:59<27:53:23,  8.19s/it]
 68%|██████▊   | 25703/37962 [59:32:07<27:58:44,  8.22s/it]
 68%|██████▊   | 25704/37962 [59:32:15<27:57:49,  8.21s/it]
 68%|██████▊   | 25705/37962 [59:32:24<27:56:02,  8.20s/it]
 68%|██████▊   | 25706/37962 [59:32:32<28:02:58,  8.24s/it]
 68%|██████▊   | 25707/37962 [59:32:40<27:48:16,  8.17s/it]
 68%|██████▊   | 25708/37962 [59:32:48<27:55:03,  8.20s/it]
 68%|██████▊   | 25709/37962 [59:32:56<27:57:37,  8.21s/it]
 68%|██████▊   | 25710/37962 [59:33:04<27:44:59,  8.15s/it]
                                                           
{'loss': 1.2207, 'grad_norm': 5.867581367492676, 'learning_rate': 4.983507910732069e-06, 'epoch': 0.68}

 68%|██████▊   | 25710/37962 [59:33:04<27:44:59,  8.15s/it]
 68%|██████▊   | 25711/37962 [59:33:12<27:27:34,  8.07s/it]
 68%|██████▊   | 25712/37962 [59:33:21<27:50:47,  8.18s/it]
 68%|██████▊   | 25713/37962 [59:33:29<28:03:23,  8.25s/it]
 68%|██████▊   | 25714/37962 [59:33:38<28:16:55,  8.31s/it]
 68%|██████▊   | 25715/37962 [59:33:46<28:15:23,  8.31s/it]
 68%|██████▊   | 25716/37962 [59:33:54<28:21:58,  8.34s/it]
 68%|██████▊   | 25717/37962 [59:34:03<28:25:20,  8.36s/it]
 68%|██████▊   | 25718/37962 [59:34:11<28:31:54,  8.39s/it]
 68%|██████▊   | 25719/37962 [59:34:19<28:21:34,  8.34s/it]
 68%|██████▊   | 25720/37962 [59:34:27<27:53:13,  8.20s/it]
                                                           
{'loss': 1.1769, 'grad_norm': 7.608166694641113, 'learning_rate': 4.976129292703231e-06, 'epoch': 0.68}

 68%|██████▊   | 25720/37962 [59:34:27<27:53:13,  8.20s/it]
 68%|██████▊   | 25721/37962 [59:34:36<28:09:45,  8.28s/it]
 68%|██████▊   | 25722/37962 [59:34:44<27:59:35,  8.23s/it]
 68%|██████▊   | 25723/37962 [59:34:52<28:01:27,  8.24s/it]
 68%|██████▊   | 25724/37962 [59:35:00<27:48:51,  8.18s/it]
 68%|██████▊   | 25725/37962 [59:35:09<28:01:12,  8.24s/it]
 68%|██████▊   | 25726/37962 [59:35:16<27:37:42,  8.13s/it]
 68%|██████▊   | 25727/37962 [59:35:25<27:44:37,  8.16s/it]
 68%|██████▊   | 25728/37962 [59:35:33<27:46:18,  8.17s/it]
 68%|██████▊   | 25729/37962 [59:35:41<27:45:58,  8.17s/it]
 68%|██████▊   | 25730/37962 [59:35:49<27:32:35,  8.11s/it]
                                                           
{'loss': 1.2564, 'grad_norm': 6.769532203674316, 'learning_rate': 4.968754331462852e-06, 'epoch': 0.68}

 68%|██████▊   | 25730/37962 [59:35:49<27:32:35,  8.11s/it]
 68%|██████▊   | 25731/37962 [59:35:57<27:30:30,  8.10s/it]
 68%|██████▊   | 25732/37962 [59:36:05<27:31:46,  8.10s/it]
 68%|██████▊   | 25733/37962 [59:36:13<27:41:15,  8.15s/it]
 68%|██████▊   | 25734/37962 [59:36:22<27:51:04,  8.20s/it]
 68%|██████▊   | 25735/37962 [59:36:30<27:52:43,  8.21s/it]
 68%|██████▊   | 25736/37962 [59:36:38<27:54:45,  8.22s/it]
 68%|██████▊   | 25737/37962 [59:36:46<27:54:34,  8.22s/it]
 68%|██████▊   | 25738/37962 [59:36:54<27:46:50,  8.18s/it]
 68%|██████▊   | 25739/37962 [59:37:03<27:58:14,  8.24s/it]
 68%|██████▊   | 25740/37962 [59:37:11<27:55:12,  8.22s/it]
                                                           
{'loss': 1.1354, 'grad_norm': 7.936191558837891, 'learning_rate': 4.961383032379029e-06, 'epoch': 0.68}

 68%|██████▊   | 25740/37962 [59:37:11<27:55:12,  8.22s/it]
 68%|██████▊   | 25741/37962 [59:37:19<27:27:12,  8.09s/it]
 68%|██████▊   | 25742/37962 [59:37:27<27:41:26,  8.16s/it]
 68%|██████▊   | 25743/37962 [59:37:35<27:34:37,  8.12s/it]
 68%|██████▊   | 25744/37962 [59:37:43<27:40:02,  8.15s/it]
 68%|██████▊   | 25745/37962 [59:37:52<27:43:45,  8.17s/it]
 68%|██████▊   | 25746/37962 [59:38:00<28:00:05,  8.25s/it]
 68%|██████▊   | 25747/37962 [59:38:08<28:00:23,  8.25s/it]
 68%|██████▊   | 25748/37962 [59:38:16<27:54:35,  8.23s/it]
 68%|██████▊   | 25749/37962 [59:38:25<27:58:26,  8.25s/it]
 68%|██████▊   | 25750/37962 [59:38:33<27:40:52,  8.16s/it]
                                                           
{'loss': 1.2458, 'grad_norm': 8.05709171295166, 'learning_rate': 4.954015400817215e-06, 'epoch': 0.68}

 68%|██████▊   | 25750/37962 [59:38:33<27:40:52,  8.16s/it]
 68%|██████▊   | 25751/37962 [59:38:41<27:53:42,  8.22s/it]
 68%|██████▊   | 25752/37962 [59:38:49<27:54:13,  8.23s/it]
 68%|██████▊   | 25753/37962 [59:38:58<27:51:59,  8.22s/it]
 68%|██████▊   | 25754/37962 [59:39:06<28:14:34,  8.33s/it]
 68%|██████▊   | 25755/37962 [59:39:14<28:14:00,  8.33s/it]
 68%|██████▊   | 25756/37962 [59:39:23<27:59:38,  8.26s/it]
 68%|██████▊   | 25757/37962 [59:39:31<27:58:48,  8.25s/it]
 68%|██████▊   | 25758/37962 [59:39:39<27:58:50,  8.25s/it]
 68%|██████▊   | 25759/37962 [59:39:47<28:06:48,  8.29s/it]
 68%|██████▊   | 25760/37962 [59:39:55<27:49:58,  8.21s/it]
                                                           
{'loss': 1.3451, 'grad_norm': 9.296412467956543, 'learning_rate': 4.946651442140173e-06, 'epoch': 0.68}

 68%|██████▊   | 25760/37962 [59:39:55<27:49:58,  8.21s/it]
 68%|██████▊   | 25761/37962 [59:40:03<27:27:41,  8.10s/it]
 68%|██████▊   | 25762/37962 [59:40:12<27:35:39,  8.14s/it]
 68%|██████▊   | 25763/37962 [59:40:20<27:50:33,  8.22s/it]
 68%|██████▊   | 25764/37962 [59:40:28<27:51:22,  8.22s/it]
 68%|██████▊   | 25765/37962 [59:40:36<27:52:30,  8.23s/it]
 68%|██████▊   | 25766/37962 [59:40:45<27:51:25,  8.22s/it]
 68%|██████▊   | 25767/37962 [59:40:53<27:58:13,  8.26s/it]
 68%|██████▊   | 25768/37962 [59:41:01<28:12:42,  8.33s/it]
 68%|██████▊   | 25769/37962 [59:41:10<28:08:04,  8.31s/it]
 68%|██████▊   | 25770/37962 [59:41:18<28:05:07,  8.29s/it]
                                                           
{'loss': 1.3016, 'grad_norm': 14.730242729187012, 'learning_rate': 4.939291161708001e-06, 'epoch': 0.68}

 68%|██████▊   | 25770/37962 [59:41:18<28:05:07,  8.29s/it]
 68%|██████▊   | 25771/37962 [59:41:26<28:01:15,  8.27s/it]
 68%|██████▊   | 25772/37962 [59:41:35<28:08:46,  8.31s/it]
 68%|██████▊   | 25773/37962 [59:41:43<28:12:16,  8.33s/it]
 68%|██████▊   | 25774/37962 [59:41:51<28:07:27,  8.31s/it]
 68%|██████▊   | 25775/37962 [59:41:59<27:41:31,  8.18s/it]
 68%|██████▊   | 25776/37962 [59:42:08<27:55:08,  8.25s/it]
 68%|██████▊   | 25777/37962 [59:42:16<27:42:15,  8.19s/it]
 68%|██████▊   | 25778/37962 [59:42:24<27:40:29,  8.18s/it]
 68%|██████▊   | 25779/37962 [59:42:32<27:45:41,  8.20s/it]
 68%|██████▊   | 25780/37962 [59:42:41<28:07:06,  8.31s/it]
                                                           
{'loss': 1.1254, 'grad_norm': 3.7692058086395264, 'learning_rate': 4.931934564878124e-06, 'epoch': 0.68}

 68%|██████▊   | 25780/37962 [59:42:41<28:07:06,  8.31s/it]
 68%|██████▊   | 25781/37962 [59:42:49<27:53:28,  8.24s/it]
 68%|██████▊   | 25782/37962 [59:42:57<27:58:35,  8.27s/it]
 68%|██████▊   | 25783/37962 [59:43:05<28:02:16,  8.29s/it]
 68%|██████▊   | 25784/37962 [59:43:14<28:03:59,  8.30s/it]
 68%|██████▊   | 25785/37962 [59:43:22<27:43:43,  8.20s/it]
 68%|██████▊   | 25786/37962 [59:43:30<27:44:20,  8.20s/it]
 68%|██████▊   | 25787/37962 [59:43:38<27:52:14,  8.24s/it]
 68%|██████▊   | 25788/37962 [59:43:46<27:52:14,  8.24s/it]
 68%|██████▊   | 25789/37962 [59:43:55<27:50:45,  8.24s/it]
 68%|██████▊   | 25790/37962 [59:44:03<27:42:27,  8.19s/it]
                                                           
{'loss': 1.227, 'grad_norm': 8.029806137084961, 'learning_rate': 4.924581657005276e-06, 'epoch': 0.68}

 68%|██████▊   | 25790/37962 [59:44:03<27:42:27,  8.19s/it]
 68%|██████▊   | 25791/37962 [59:44:11<27:45:25,  8.21s/it]
 68%|██████▊   | 25792/37962 [59:44:19<28:02:28,  8.29s/it]
 68%|██████▊   | 25793/37962 [59:44:28<27:51:31,  8.24s/it]
 68%|██████▊   | 25794/37962 [59:44:36<27:56:33,  8.27s/it]
 68%|██████▊   | 25795/37962 [59:44:44<27:57:19,  8.27s/it]
 68%|██████▊   | 25796/37962 [59:44:53<28:06:44,  8.32s/it]
 68%|██████▊   | 25797/37962 [59:45:01<28:10:23,  8.34s/it]
 68%|██████▊   | 25798/37962 [59:45:09<28:04:49,  8.31s/it]
 68%|██████▊   | 25799/37962 [59:45:17<27:57:19,  8.27s/it]
 68%|██████▊   | 25800/37962 [59:45:26<27:51:56,  8.25s/it]
                                                           
{'loss': 1.275, 'grad_norm': 5.416784763336182, 'learning_rate': 4.917232443441518e-06, 'epoch': 0.68}

 68%|██████▊   | 25800/37962 [59:45:26<27:51:56,  8.25s/it]
 68%|██████▊   | 25801/37962 [59:45:34<27:57:20,  8.28s/it]
 68%|██████▊   | 25802/37962 [59:45:42<27:23:17,  8.11s/it]
 68%|██████▊   | 25803/37962 [59:45:50<27:36:32,  8.17s/it]
 68%|██████▊   | 25804/37962 [59:45:58<27:44:49,  8.22s/it]
 68%|██████▊   | 25805/37962 [59:46:06<27:39:54,  8.19s/it]
 68%|██████▊   | 25806/37962 [59:46:15<27:48:20,  8.23s/it]
 68%|██████▊   | 25807/37962 [59:46:23<27:35:05,  8.17s/it]
 68%|██████▊   | 25808/37962 [59:46:31<27:12:45,  8.06s/it]
 68%|██████▊   | 25809/37962 [59:46:39<27:23:43,  8.12s/it]
 68%|██████▊   | 25810/37962 [59:46:47<27:39:26,  8.19s/it]
                                                           
{'loss': 1.1591, 'grad_norm': 9.923258781433105, 'learning_rate': 4.9098869295362105e-06, 'epoch': 0.68}

 68%|██████▊   | 25810/37962 [59:46:47<27:39:26,  8.19s/it]
 68%|██████▊   | 25811/37962 [59:46:55<27:38:47,  8.19s/it]
 68%|██████▊   | 25812/37962 [59:47:04<27:37:41,  8.19s/it]
 68%|██████▊   | 25813/37962 [59:47:12<27:42:26,  8.21s/it]
 68%|██████▊   | 25814/37962 [59:47:20<27:45:10,  8.22s/it]
 68%|██████▊   | 25815/37962 [59:47:28<27:31:08,  8.16s/it]
 68%|██████▊   | 25816/37962 [59:47:36<27:40:13,  8.20s/it]
 68%|██████▊   | 25817/37962 [59:47:45<27:41:58,  8.21s/it]
 68%|██████▊   | 25818/37962 [59:47:52<27:20:29,  8.11s/it]
 68%|██████▊   | 25819/37962 [59:48:00<27:16:11,  8.08s/it]
 68%|██████▊   | 25820/37962 [59:48:09<27:24:05,  8.12s/it]
                                                           
{'loss': 1.2427, 'grad_norm': 17.738723754882812, 'learning_rate': 4.902545120636024e-06, 'epoch': 0.68}

 68%|██████▊   | 25820/37962 [59:48:09<27:24:05,  8.12s/it]
 68%|██████▊   | 25821/37962 [59:48:17<27:31:54,  8.16s/it]
 68%|██████▊   | 25822/37962 [59:48:25<27:33:31,  8.17s/it]
 68%|██████▊   | 25823/37962 [59:48:34<27:52:24,  8.27s/it]
 68%|██████▊   | 25824/37962 [59:48:42<28:03:09,  8.32s/it]
 68%|██████▊   | 25825/37962 [59:48:50<28:03:18,  8.32s/it]
 68%|██████▊   | 25826/37962 [59:48:59<28:11:08,  8.36s/it]
 68%|██████▊   | 25827/37962 [59:49:07<28:13:39,  8.37s/it]
 68%|██████▊   | 25828/37962 [59:49:16<28:09:16,  8.35s/it]
 68%|██████▊   | 25829/37962 [59:49:24<28:07:23,  8.34s/it]
 68%|██████▊   | 25830/37962 [59:49:32<28:02:10,  8.32s/it]
                                                           
{'loss': 1.2402, 'grad_norm': 8.052451133728027, 'learning_rate': 4.895207022084937e-06, 'epoch': 0.68}

 68%|██████▊   | 25830/37962 [59:49:32<28:02:10,  8.32s/it]
 68%|██████▊   | 25831/37962 [59:49:41<28:07:58,  8.35s/it]
 68%|██████▊   | 25832/37962 [59:49:49<27:57:48,  8.30s/it]
 68%|██████▊   | 25833/37962 [59:49:57<27:52:49,  8.28s/it]
 68%|██████▊   | 25834/37962 [59:50:05<27:29:53,  8.16s/it]
 68%|██████▊   | 25835/37962 [59:50:13<27:34:40,  8.19s/it]
 68%|██████▊   | 25836/37962 [59:50:21<27:35:44,  8.19s/it]
 68%|██████▊   | 25837/37962 [59:50:30<27:46:59,  8.25s/it]
 68%|██████▊   | 25838/37962 [59:50:38<27:53:43,  8.28s/it]
 68%|██████▊   | 25839/37962 [59:50:46<28:02:01,  8.32s/it]
 68%|██████▊   | 25840/37962 [59:50:55<27:45:28,  8.24s/it]
                                                           
{'loss': 1.2684, 'grad_norm': 6.196129322052002, 'learning_rate': 4.887872639224227e-06, 'epoch': 0.68}

 68%|██████▊   | 25840/37962 [59:50:55<27:45:28,  8.24s/it]
 68%|██████▊   | 25841/37962 [59:51:03<27:55:05,  8.29s/it]
 68%|██████▊   | 25842/37962 [59:51:11<27:56:13,  8.30s/it]
 68%|██████▊   | 25843/37962 [59:51:19<27:47:35,  8.26s/it]
 68%|██████▊   | 25844/37962 [59:51:28<27:40:40,  8.22s/it]
 68%|██████▊   | 25845/37962 [59:51:36<27:48:39,  8.26s/it]
 68%|██████▊   | 25846/37962 [59:51:44<27:50:09,  8.27s/it]
 68%|██████▊   | 25847/37962 [59:51:52<27:36:26,  8.20s/it]
 68%|██████▊   | 25848/37962 [59:52:00<27:34:49,  8.20s/it]
 68%|██████▊   | 25849/37962 [59:52:09<27:45:52,  8.25s/it]
 68%|██████▊   | 25850/37962 [59:52:17<27:50:47,  8.28s/it]
                                                           
{'loss': 1.2936, 'grad_norm': 11.164350509643555, 'learning_rate': 4.880541977392462e-06, 'epoch': 0.68}

 68%|██████▊   | 25850/37962 [59:52:17<27:50:47,  8.28s/it]
 68%|██████▊   | 25851/37962 [59:52:25<27:49:13,  8.27s/it]
 68%|██████▊   | 25852/37962 [59:52:34<28:02:00,  8.33s/it]
 68%|██████▊   | 25853/37962 [59:52:42<27:55:00,  8.30s/it]
 68%|██████▊   | 25854/37962 [59:52:50<27:57:15,  8.31s/it]
 68%|██████▊   | 25855/37962 [59:52:59<27:53:44,  8.29s/it]
 68%|██████▊   | 25856/37962 [59:53:07<27:55:17,  8.30s/it]
 68%|██████▊   | 25857/37962 [59:53:15<27:51:28,  8.28s/it]
 68%|██████▊   | 25858/37962 [59:53:24<27:56:18,  8.31s/it]
 68%|██████▊   | 25859/37962 [59:53:32<27:54:09,  8.30s/it]
 68%|██████▊   | 25860/37962 [59:53:40<27:53:10,  8.30s/it]
                                                           
{'loss': 1.1448, 'grad_norm': 17.020627975463867, 'learning_rate': 4.8732150419254996e-06, 'epoch': 0.68}

 68%|██████▊   | 25860/37962 [59:53:40<27:53:10,  8.30s/it]
 68%|██████▊   | 25861/37962 [59:53:48<27:47:59,  8.27s/it]
 68%|██████▊   | 25862/37962 [59:53:57<27:48:21,  8.27s/it]
 68%|██████▊   | 25863/37962 [59:54:05<27:41:14,  8.24s/it]
 68%|██████▊   | 25864/37962 [59:54:13<27:46:41,  8.27s/it]
 68%|██████▊   | 25865/37962 [59:54:21<27:47:24,  8.27s/it]
 68%|██████▊   | 25866/37962 [59:54:30<27:39:10,  8.23s/it]
 68%|██████▊   | 25867/37962 [59:54:38<27:44:37,  8.26s/it]
 68%|██████▊   | 25868/37962 [59:54:46<27:41:54,  8.24s/it]
 68%|██████▊   | 25869/37962 [59:54:54<27:37:41,  8.22s/it]
 68%|██████▊   | 25870/37962 [59:55:03<27:40:01,  8.24s/it]
                                                           
{'loss': 1.238, 'grad_norm': 3.2352802753448486, 'learning_rate': 4.865891838156497e-06, 'epoch': 0.68}

 68%|██████▊   | 25870/37962 [59:55:03<27:40:01,  8.24s/it]
 68%|██████▊   | 25871/37962 [59:55:11<27:51:12,  8.29s/it]
 68%|██████▊   | 25872/37962 [59:55:19<27:44:16,  8.26s/it]
 68%|██████▊   | 25873/37962 [59:55:28<27:48:16,  8.28s/it]
 68%|██████▊   | 25874/37962 [59:55:36<27:39:14,  8.24s/it]
 68%|██████▊   | 25875/37962 [59:55:44<27:39:09,  8.24s/it]
 68%|██████▊   | 25876/37962 [59:55:52<27:25:02,  8.17s/it]
 68%|██████▊   | 25877/37962 [59:56:00<27:33:24,  8.21s/it]
 68%|██████▊   | 25878/37962 [59:56:08<27:30:02,  8.19s/it]
 68%|██████▊   | 25879/37962 [59:56:17<27:51:05,  8.30s/it]
 68%|██████▊   | 25880/37962 [59:56:25<27:43:36,  8.26s/it]
                                                           
{'loss': 1.4112, 'grad_norm': 8.99996280670166, 'learning_rate': 4.858572371415878e-06, 'epoch': 0.68}

 68%|██████▊   | 25880/37962 [59:56:25<27:43:36,  8.26s/it]
 68%|██████▊   | 25881/37962 [59:56:33<27:37:44,  8.23s/it]
 68%|██████▊   | 25882/37962 [59:56:41<27:38:19,  8.24s/it]
 68%|██████▊   | 25883/37962 [59:56:50<27:37:37,  8.23s/it]
 68%|██████▊   | 25884/37962 [59:56:58<27:25:01,  8.17s/it]
 68%|██████▊   | 25885/37962 [59:57:06<27:29:08,  8.19s/it]
 68%|██████▊   | 25886/37962 [59:57:14<27:40:08,  8.25s/it]
 68%|██████▊   | 25887/37962 [59:57:23<27:41:46,  8.26s/it]
 68%|██████▊   | 25888/37962 [59:57:31<27:43:56,  8.27s/it]
 68%|██████▊   | 25889/37962 [59:57:39<27:44:29,  8.27s/it]
 68%|██████▊   | 25890/37962 [59:57:47<27:45:07,  8.28s/it]
                                                           
{'loss': 1.2233, 'grad_norm': 6.6104350090026855, 'learning_rate': 4.851256647031365e-06, 'epoch': 0.68}

 68%|██████▊   | 25890/37962 [59:57:47<27:45:07,  8.28s/it]
 68%|██████▊   | 25891/37962 [59:57:56<27:46:24,  8.28s/it]
 68%|██████▊   | 25892/37962 [59:58:04<27:56:41,  8.33s/it]
 68%|██████▊   | 25893/37962 [59:58:13<27:53:36,  8.32s/it]
 68%|██████▊   | 25894/37962 [59:58:21<27:52:33,  8.32s/it]
 68%|██████▊   | 25895/37962 [59:58:29<27:51:57,  8.31s/it]
 68%|██████▊   | 25896/37962 [59:58:37<27:41:44,  8.26s/it]
 68%|██████▊   | 25897/37962 [59:58:45<27:20:01,  8.16s/it]
 68%|██████▊   | 25898/37962 [59:58:53<27:16:17,  8.14s/it]
 68%|██████▊   | 25899/37962 [59:59:01<26:52:21,  8.02s/it]
 68%|██████▊   | 25900/37962 [59:59:10<27:25:43,  8.19s/it]
                                                           
{'loss': 1.1534, 'grad_norm': 10.195953369140625, 'learning_rate': 4.843944670327942e-06, 'epoch': 0.68}

 68%|██████▊   | 25900/37962 [59:59:10<27:25:43,  8.19s/it]
 68%|██████▊   | 25901/37962 [59:59:18<27:26:22,  8.19s/it]
 68%|██████▊   | 25902/37962 [59:59:26<27:15:25,  8.14s/it]
 68%|██████▊   | 25903/37962 [59:59:34<27:38:51,  8.25s/it]
 68%|██████▊   | 25904/37962 [59:59:43<27:40:37,  8.26s/it]
 68%|██████▊   | 25905/37962 [59:59:51<27:44:58,  8.29s/it]
 68%|██████▊   | 25906/37962 [59:59:59<27:45:45,  8.29s/it]
 68%|██████▊   | 25907/37962 [60:00:07<27:40:45,  8.27s/it]
 68%|██████▊   | 25908/37962 [60:00:16<27:37:49,  8.25s/it]
 68%|██████▊   | 25909/37962 [60:00:24<27:32:47,  8.23s/it]
 68%|██████▊   | 25910/37962 [60:00:32<27:46:51,  8.30s/it]
                                                           
{'loss': 1.2197, 'grad_norm': 6.369474411010742, 'learning_rate': 4.836636446627868e-06, 'epoch': 0.68}

 68%|██████▊   | 25910/37962 [60:00:32<27:46:51,  8.30s/it]
 68%|██████▊   | 25911/37962 [60:00:41<27:42:31,  8.28s/it]
 68%|██████▊   | 25912/37962 [60:00:49<27:47:41,  8.30s/it]
 68%|██████▊   | 25913/37962 [60:00:57<27:44:06,  8.29s/it]
 68%|██████▊   | 25914/37962 [60:01:05<27:36:03,  8.25s/it]
 68%|██████▊   | 25915/37962 [60:01:14<27:54:20,  8.34s/it]
 68%|██████▊   | 25916/37962 [60:01:22<27:44:46,  8.29s/it]
 68%|██████▊   | 25917/37962 [60:01:30<27:52:13,  8.33s/it]
 68%|██████▊   | 25918/37962 [60:01:39<27:48:07,  8.31s/it]
 68%|██████▊   | 25919/37962 [60:01:47<27:47:51,  8.31s/it]
 68%|██████▊   | 25920/37962 [60:01:55<27:38:41,  8.26s/it]
                                                           
{'loss': 1.2241, 'grad_norm': 10.754475593566895, 'learning_rate': 4.8293319812506755e-06, 'epoch': 0.68}

 68%|██████▊   | 25920/37962 [60:01:55<27:38:41,  8.26s/it]
 68%|██████▊   | 25921/37962 [60:02:03<27:33:25,  8.24s/it]
 68%|██████▊   | 25922/37962 [60:02:12<27:42:02,  8.28s/it]
 68%|██████▊   | 25923/37962 [60:02:20<27:31:18,  8.23s/it]
 68%|██████▊   | 25924/37962 [60:02:28<27:32:45,  8.24s/it]
 68%|██████▊   | 25925/37962 [60:02:37<27:50:01,  8.32s/it]
 68%|██████▊   | 25926/37962 [60:02:45<27:45:57,  8.30s/it]
 68%|██████▊   | 25927/37962 [60:02:53<27:20:37,  8.18s/it]
 68%|██████▊   | 25928/37962 [60:03:01<27:32:05,  8.24s/it]
 68%|██████▊   | 25929/37962 [60:03:10<27:38:10,  8.27s/it]
 68%|██████▊   | 25930/37962 [60:03:18<27:34:55,  8.25s/it]
                                                           
{'loss': 1.2638, 'grad_norm': 8.090713500976562, 'learning_rate': 4.822031279513162e-06, 'epoch': 0.68}

 68%|██████▊   | 25930/37962 [60:03:18<27:34:55,  8.25s/it]
 68%|██████▊   | 25931/37962 [60:03:26<27:21:13,  8.18s/it]
 68%|██████▊   | 25932/37962 [60:03:34<27:30:06,  8.23s/it]
 68%|██████▊   | 25933/37962 [60:03:42<27:29:40,  8.23s/it]
 68%|██████▊   | 25934/37962 [60:03:50<27:20:09,  8.18s/it]
 68%|██████▊   | 25935/37962 [60:03:59<27:26:34,  8.21s/it]
 68%|██████▊   | 25936/37962 [60:04:07<27:17:57,  8.17s/it]
 68%|██████▊   | 25937/37962 [60:04:15<27:14:10,  8.15s/it]
 68%|██████▊   | 25938/37962 [60:04:23<27:23:17,  8.20s/it]
 68%|██████▊   | 25939/37962 [60:04:32<27:35:46,  8.26s/it]
 68%|██████▊   | 25940/37962 [60:04:40<27:36:16,  8.27s/it]
                                                           
{'loss': 1.1602, 'grad_norm': 4.408010959625244, 'learning_rate': 4.814734346729374e-06, 'epoch': 0.68}

 68%|██████▊   | 25940/37962 [60:04:40<27:36:16,  8.27s/it]
 68%|██████▊   | 25941/37962 [60:04:48<27:29:40,  8.23s/it]
 68%|██████▊   | 25942/37962 [60:04:56<27:29:51,  8.24s/it]
 68%|██████▊   | 25943/37962 [60:05:05<27:46:40,  8.32s/it]
 68%|██████▊   | 25944/37962 [60:05:13<27:32:41,  8.25s/it]
 68%|██████▊   | 25945/37962 [60:05:21<27:28:43,  8.23s/it]
 68%|██████▊   | 25946/37962 [60:05:29<27:36:17,  8.27s/it]
 68%|██████▊   | 25947/37962 [60:05:38<27:41:05,  8.30s/it]
 68%|██████▊   | 25948/37962 [60:05:46<27:39:24,  8.29s/it]
 68%|██████▊   | 25949/37962 [60:05:54<27:41:07,  8.30s/it]
 68%|██████▊   | 25950/37962 [60:06:03<27:39:44,  8.29s/it]
                                                           
{'loss': 1.2971, 'grad_norm': 10.687789916992188, 'learning_rate': 4.80744118821063e-06, 'epoch': 0.68}

 68%|██████▊   | 25950/37962 [60:06:03<27:39:44,  8.29s/it]
 68%|██████▊   | 25951/37962 [60:06:11<27:38:54,  8.29s/it]
 68%|██████▊   | 25952/37962 [60:06:19<27:35:03,  8.27s/it]
 68%|██████▊   | 25953/37962 [60:06:27<27:35:11,  8.27s/it]
 68%|██████▊   | 25954/37962 [60:06:36<27:43:34,  8.31s/it]
 68%|██████▊   | 25955/37962 [60:06:44<27:45:36,  8.32s/it]
 68%|██████▊   | 25956/37962 [60:06:52<27:38:54,  8.29s/it]
 68%|██████▊   | 25957/37962 [60:07:01<27:34:01,  8.27s/it]
 68%|██████▊   | 25958/37962 [60:07:09<27:32:05,  8.26s/it]
 68%|██████▊   | 25959/37962 [60:07:17<27:18:48,  8.19s/it]
 68%|██████▊   | 25960/37962 [60:07:25<27:16:37,  8.18s/it]
                                                           
{'loss': 1.2278, 'grad_norm': 5.895262718200684, 'learning_rate': 4.800151809265491e-06, 'epoch': 0.68}

 68%|██████▊   | 25960/37962 [60:07:25<27:16:37,  8.18s/it]
 68%|██████▊   | 25961/37962 [60:07:33<27:25:38,  8.23s/it]
 68%|██████▊   | 25962/37962 [60:07:42<27:30:13,  8.25s/it]
 68%|██████▊   | 25963/37962 [60:07:50<27:29:46,  8.25s/it]
 68%|██████▊   | 25964/37962 [60:07:58<27:28:15,  8.24s/it]
 68%|██████▊   | 25965/37962 [60:08:07<27:38:24,  8.29s/it]
 68%|██████▊   | 25966/37962 [60:08:15<27:30:35,  8.26s/it]
 68%|██████▊   | 25967/37962 [60:08:23<27:26:36,  8.24s/it]
 68%|██████▊   | 25968/37962 [60:08:31<27:26:50,  8.24s/it]
 68%|██████▊   | 25969/37962 [60:08:39<27:29:38,  8.25s/it]
 68%|██████▊   | 25970/37962 [60:08:47<27:08:59,  8.15s/it]
                                                           
{'loss': 1.1605, 'grad_norm': 9.807201385498047, 'learning_rate': 4.792866215199764e-06, 'epoch': 0.68}

 68%|██████▊   | 25970/37962 [60:08:47<27:08:59,  8.15s/it]
 68%|██████▊   | 25971/37962 [60:08:56<27:10:57,  8.16s/it]
 68%|██████▊   | 25972/37962 [60:09:04<27:15:43,  8.19s/it]
 68%|██████▊   | 25973/37962 [60:09:12<27:17:49,  8.20s/it]
 68%|██████▊   | 25974/37962 [60:09:20<26:57:30,  8.10s/it]
 68%|██████▊   | 25975/37962 [60:09:28<27:10:55,  8.16s/it]
 68%|██████▊   | 25976/37962 [60:09:36<26:58:54,  8.10s/it]
 68%|██████▊   | 25977/37962 [60:09:45<27:22:09,  8.22s/it]
 68%|██████▊   | 25978/37962 [60:09:53<27:21:34,  8.22s/it]
 68%|██████▊   | 25979/37962 [60:10:01<27:17:53,  8.20s/it]
 68%|██████▊   | 25980/37962 [60:10:17<34:56:20, 10.50s/it]
                                                           
{'loss': 1.3415, 'grad_norm': 7.876060485839844, 'learning_rate': 4.785584411316515e-06, 'epoch': 0.68}

 68%|██████▊   | 25980/37962 [60:10:17<34:56:20, 10.50s/it]
 68%|██████▊   | 25981/37962 [60:10:25<33:02:25,  9.93s/it]
 68%|██████▊   | 25982/37962 [60:10:34<31:40:24,  9.52s/it]
 68%|██████▊   | 25983/37962 [60:10:42<30:19:06,  9.11s/it]
 68%|██████▊   | 25984/37962 [60:10:50<29:01:32,  8.72s/it]
 68%|██████▊   | 25985/37962 [60:10:58<28:31:07,  8.57s/it]
 68%|██████▊   | 25986/37962 [60:11:07<28:22:40,  8.53s/it]
 68%|██████▊   | 25987/37962 [60:11:15<28:05:40,  8.45s/it]
 68%|██████▊   | 25988/37962 [60:11:23<27:57:42,  8.41s/it]
 68%|██████▊   | 25989/37962 [60:11:32<27:52:26,  8.38s/it]
 68%|██████▊   | 25990/37962 [60:11:40<27:58:33,  8.41s/it]
                                                           
{'loss': 1.2953, 'grad_norm': 11.125879287719727, 'learning_rate': 4.778306402916036e-06, 'epoch': 0.68}

 68%|██████▊   | 25990/37962 [60:11:40<27:58:33,  8.41s/it]
 68%|██████▊   | 25991/37962 [60:11:48<27:50:27,  8.37s/it]
 68%|██████▊   | 25992/37962 [60:11:56<27:32:26,  8.28s/it]
 68%|██████▊   | 25993/37962 [60:12:04<27:17:15,  8.21s/it]
 68%|██████▊   | 25994/37962 [60:12:13<27:26:20,  8.25s/it]
 68%|██████▊   | 25995/37962 [60:12:21<27:31:19,  8.28s/it]
 68%|██████▊   | 25996/37962 [60:12:29<27:05:36,  8.15s/it]
 68%|██████▊   | 25997/37962 [60:12:37<26:51:06,  8.08s/it]
 68%|██████▊   | 25998/37962 [60:12:45<26:56:12,  8.11s/it]
 68%|██████▊   | 25999/37962 [60:12:54<27:25:16,  8.25s/it]
 68%|██████▊   | 26000/37962 [60:13:02<27:32:44,  8.29s/it]
                                                           
{'loss': 1.2004, 'grad_norm': 6.682347774505615, 'learning_rate': 4.771032195295868e-06, 'epoch': 0.68}

 68%|██████▊   | 26000/37962 [60:13:02<27:32:44,  8.29s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 68%|██████▊   | 26001/37962 [60:13:48<65:01:56, 19.57s/it]
 68%|██████▊   | 26002/37962 [60:13:56<53:39:42, 16.15s/it]
 68%|██████▊   | 26003/37962 [60:14:05<45:57:07, 13.83s/it]
 69%|██████▊   | 26004/37962 [60:14:13<40:34:08, 12.21s/it]
 69%|██████▊   | 26005/37962 [60:14:21<36:48:27, 11.08s/it]
 69%|██████▊   | 26006/37962 [60:14:30<34:07:08, 10.27s/it]
 69%|██████▊   | 26007/37962 [60:14:38<31:52:14,  9.60s/it]
 69%|██████▊   | 26008/37962 [60:14:46<30:31:33,  9.19s/it]
 69%|██████▊   | 26009/37962 [60:14:54<29:45:05,  8.96s/it]
 69%|██████▊   | 26010/37962 [60:15:02<28:46:04,  8.67s/it]
                                                           
{'loss': 1.2084, 'grad_norm': 6.505674839019775, 'learning_rate': 4.763761793750785e-06, 'epoch': 0.69}

 69%|██████▊   | 26010/37962 [60:15:02<28:46:04,  8.67s/it]
 69%|██████▊   | 26011/37962 [60:15:11<28:11:35,  8.49s/it]
 69%|██████▊   | 26012/37962 [60:15:19<27:56:57,  8.42s/it]
 69%|██████▊   | 26013/37962 [60:15:27<27:49:54,  8.39s/it]
 69%|██████▊   | 26014/37962 [60:15:35<27:38:04,  8.33s/it]
 69%|██████▊   | 26015/37962 [60:15:43<27:18:35,  8.23s/it]
 69%|██████▊   | 26016/37962 [60:15:52<27:25:11,  8.26s/it]
 69%|██████▊   | 26017/37962 [60:16:00<27:04:24,  8.16s/it]
 69%|██████▊   | 26018/37962 [60:16:08<27:25:02,  8.26s/it]
 69%|██████▊   | 26019/37962 [60:16:16<27:06:27,  8.17s/it]
 69%|██████▊   | 26020/37962 [60:16:24<27:11:38,  8.20s/it]
                                                           
{'loss': 1.2385, 'grad_norm': 10.68661117553711, 'learning_rate': 4.756495203572775e-06, 'epoch': 0.69}

 69%|██████▊   | 26020/37962 [60:16:24<27:11:38,  8.20s/it]
 69%|██████▊   | 26021/37962 [60:16:33<27:19:47,  8.24s/it]
 69%|██████▊   | 26022/37962 [60:16:41<27:23:24,  8.26s/it]
 69%|██████▊   | 26023/37962 [60:16:49<27:20:58,  8.25s/it]
 69%|██████▊   | 26024/37962 [60:16:57<27:27:57,  8.28s/it]
 69%|██████▊   | 26025/37962 [60:17:06<27:35:16,  8.32s/it]
 69%|██████▊   | 26026/37962 [60:17:14<27:28:45,  8.29s/it]
 69%|██████▊   | 26027/37962 [60:17:22<27:33:53,  8.31s/it]
 69%|██████▊   | 26028/37962 [60:17:31<27:39:20,  8.34s/it]
 69%|██████▊   | 26029/37962 [60:17:39<27:18:55,  8.24s/it]
 69%|██████▊   | 26030/37962 [60:17:47<27:24:02,  8.27s/it]
                                                           
{'loss': 1.2114, 'grad_norm': 7.965817928314209, 'learning_rate': 4.749232430051067e-06, 'epoch': 0.69}

 69%|██████▊   | 26030/37962 [60:17:47<27:24:02,  8.27s/it]
 69%|██████▊   | 26031/37962 [60:17:55<27:13:41,  8.22s/it]
 69%|██████▊   | 26032/37962 [60:18:04<27:24:35,  8.27s/it]
 69%|██████▊   | 26033/37962 [60:18:12<27:25:48,  8.28s/it]
 69%|██████▊   | 26034/37962 [60:18:21<27:44:48,  8.37s/it]
 69%|██████▊   | 26035/37962 [60:18:29<27:36:08,  8.33s/it]
 69%|██████▊   | 26036/37962 [60:18:37<27:35:56,  8.33s/it]
 69%|██████▊   | 26037/37962 [60:18:45<27:30:18,  8.30s/it]
 69%|██████▊   | 26038/37962 [60:18:54<27:19:09,  8.25s/it]
 69%|██████▊   | 26039/37962 [60:19:02<27:14:53,  8.23s/it]
 69%|██████▊   | 26040/37962 [60:19:10<27:10:23,  8.21s/it]
                                                           
{'loss': 1.3824, 'grad_norm': 23.90220069885254, 'learning_rate': 4.741973478472115e-06, 'epoch': 0.69}

 69%|██████▊   | 26040/37962 [60:19:10<27:10:23,  8.21s/it]
 69%|██████▊   | 26041/37962 [60:19:18<27:19:16,  8.25s/it]
 69%|██████▊   | 26042/37962 [60:19:26<27:18:42,  8.25s/it]
 69%|██████▊   | 26043/37962 [60:19:35<27:18:08,  8.25s/it]
 69%|██████▊   | 26044/37962 [60:19:43<27:25:54,  8.29s/it]
 69%|██████▊   | 26045/37962 [60:19:51<27:22:35,  8.27s/it]
 69%|██████▊   | 26046/37962 [60:19:59<27:07:17,  8.19s/it]
 69%|██████▊   | 26047/37962 [60:20:08<27:12:36,  8.22s/it]
 69%|██████▊   | 26048/37962 [60:20:16<27:20:44,  8.26s/it]
 69%|██████▊   | 26049/37962 [60:20:24<27:17:05,  8.25s/it]
 69%|██████▊   | 26050/37962 [60:20:32<27:05:42,  8.19s/it]
                                                           
{'loss': 1.2672, 'grad_norm': 10.84085464477539, 'learning_rate': 4.734718354119575e-06, 'epoch': 0.69}

 69%|██████▊   | 26050/37962 [60:20:32<27:05:42,  8.19s/it]
 69%|██████▊   | 26051/37962 [60:20:48<34:11:06, 10.33s/it]
 69%|██████▊   | 26052/37962 [60:20:56<32:15:19,  9.75s/it]
 69%|██████▊   | 26053/37962 [60:21:04<30:55:46,  9.35s/it]
 69%|██████▊   | 26054/37962 [60:21:13<29:59:53,  9.07s/it]
 69%|██████▊   | 26055/37962 [60:21:21<29:11:14,  8.82s/it]
 69%|██████▊   | 26056/37962 [60:21:29<28:36:56,  8.65s/it]
 69%|██████▊   | 26057/37962 [60:21:38<28:20:26,  8.57s/it]
 69%|██████▊   | 26058/37962 [60:21:46<27:59:33,  8.47s/it]
 69%|██████▊   | 26059/37962 [60:21:54<27:48:54,  8.41s/it]
 69%|██████▊   | 26060/37962 [60:22:03<27:43:37,  8.39s/it]
                                                           
{'loss': 1.1759, 'grad_norm': 7.181231498718262, 'learning_rate': 4.727467062274334e-06, 'epoch': 0.69}

 69%|██████▊   | 26060/37962 [60:22:03<27:43:37,  8.39s/it]
 69%|██████▊   | 26061/37962 [60:22:11<27:21:03,  8.27s/it]
 69%|██████▊   | 26062/37962 [60:22:18<26:56:11,  8.15s/it]
 69%|██████▊   | 26063/37962 [60:22:27<26:53:47,  8.14s/it]
 69%|██████▊   | 26064/37962 [60:22:35<27:10:10,  8.22s/it]
 69%|██████▊   | 26065/37962 [60:22:43<27:03:23,  8.19s/it]
 69%|██████▊   | 26066/37962 [60:22:51<27:05:54,  8.20s/it]
 69%|██████▊   | 26067/37962 [60:22:59<26:59:57,  8.17s/it]
 69%|██████▊   | 26068/37962 [60:23:08<27:13:44,  8.24s/it]
 69%|██████▊   | 26069/37962 [60:23:16<27:20:30,  8.28s/it]
 69%|██████▊   | 26070/37962 [60:23:24<27:23:28,  8.29s/it]
                                                           
{'loss': 1.3251, 'grad_norm': 11.738880157470703, 'learning_rate': 4.7202196082144765e-06, 'epoch': 0.69}

 69%|██████▊   | 26070/37962 [60:23:24<27:23:28,  8.29s/it]
 69%|██████▊   | 26071/37962 [60:23:33<27:18:55,  8.27s/it]
 69%|██████▊   | 26072/37962 [60:23:41<27:16:25,  8.26s/it]
 69%|██████▊   | 26073/37962 [60:23:49<27:24:06,  8.30s/it]
 69%|██████▊   | 26074/37962 [60:23:58<27:21:37,  8.29s/it]
 69%|██████▊   | 26075/37962 [60:24:06<27:23:08,  8.29s/it]
 69%|██████▊   | 26076/37962 [60:24:14<27:30:40,  8.33s/it]
 69%|██████▊   | 26077/37962 [60:24:23<27:40:13,  8.38s/it]
 69%|██████▊   | 26078/37962 [60:24:31<27:44:00,  8.40s/it]
 69%|██████▊   | 26079/37962 [60:24:39<27:32:34,  8.34s/it]
 69%|██████▊   | 26080/37962 [60:24:48<27:42:38,  8.40s/it]
                                                           
{'loss': 1.2819, 'grad_norm': 6.119255542755127, 'learning_rate': 4.7129759972152985e-06, 'epoch': 0.69}

 69%|██████▊   | 26080/37962 [60:24:48<27:42:38,  8.40s/it]
 69%|██████▊   | 26081/37962 [60:24:56<27:11:03,  8.24s/it]
 69%|██████▊   | 26082/37962 [60:25:04<27:23:11,  8.30s/it]
 69%|██████▊   | 26083/37962 [60:25:13<27:24:20,  8.31s/it]
 69%|██████▊   | 26084/37962 [60:25:21<27:17:04,  8.27s/it]
 69%|██████▊   | 26085/37962 [60:25:29<27:17:58,  8.27s/it]
 69%|██████▊   | 26086/37962 [60:25:37<27:14:02,  8.26s/it]
 69%|██████▊   | 26087/37962 [60:25:46<27:18:43,  8.28s/it]
 69%|██████▊   | 26088/37962 [60:25:54<27:16:08,  8.27s/it]
 69%|██████▊   | 26089/37962 [60:26:02<27:15:46,  8.27s/it]
 69%|██████▊   | 26090/37962 [60:26:11<27:27:22,  8.33s/it]
                                                           
{'loss': 1.2304, 'grad_norm': 8.480938911437988, 'learning_rate': 4.705736234549304e-06, 'epoch': 0.69}

 69%|██████▊   | 26090/37962 [60:26:11<27:27:22,  8.33s/it]
 69%|██████▊   | 26091/37962 [60:26:19<27:30:07,  8.34s/it]
 69%|██████▊   | 26092/37962 [60:26:27<27:11:21,  8.25s/it]
 69%|██████▊   | 26093/37962 [60:26:35<27:06:44,  8.22s/it]
 69%|██████▊   | 26094/37962 [60:26:43<27:11:30,  8.25s/it]
 69%|██████▊   | 26095/37962 [60:26:52<27:09:53,  8.24s/it]
 69%|██████▊   | 26096/37962 [60:27:00<27:11:52,  8.25s/it]
 69%|██████▊   | 26097/37962 [60:27:08<27:14:31,  8.27s/it]
 69%|██████▊   | 26098/37962 [60:27:17<27:16:02,  8.27s/it]
 69%|██████▉   | 26099/37962 [60:27:25<27:20:13,  8.30s/it]
 69%|██████▉   | 26100/37962 [60:27:33<27:20:51,  8.30s/it]
                                                           
{'loss': 1.1382, 'grad_norm': 7.583914279937744, 'learning_rate': 4.698500325486184e-06, 'epoch': 0.69}

 69%|██████▉   | 26100/37962 [60:27:33<27:20:51,  8.30s/it]
 69%|██████▉   | 26101/37962 [60:27:42<27:21:43,  8.30s/it]
 69%|██████▉   | 26102/37962 [60:27:49<26:53:01,  8.16s/it]
 69%|██████▉   | 26103/37962 [60:27:58<26:59:14,  8.19s/it]
 69%|██████▉   | 26104/37962 [60:28:06<26:47:44,  8.13s/it]
 69%|██████▉   | 26105/37962 [60:28:14<26:56:08,  8.18s/it]
 69%|██████▉   | 26106/37962 [60:28:22<27:05:31,  8.23s/it]
 69%|██████▉   | 26107/37962 [60:28:31<27:18:59,  8.30s/it]
 69%|██████▉   | 26108/37962 [60:28:39<27:07:55,  8.24s/it]
 69%|██████▉   | 26109/37962 [60:28:47<27:08:17,  8.24s/it]
 69%|██████▉   | 26110/37962 [60:28:55<27:04:42,  8.23s/it]
                                                           
{'loss': 1.184, 'grad_norm': 10.518040657043457, 'learning_rate': 4.691268275292838e-06, 'epoch': 0.69}

 69%|██████▉   | 26110/37962 [60:28:55<27:04:42,  8.23s/it]
 69%|██████▉   | 26111/37962 [60:29:03<26:49:14,  8.15s/it]
 69%|██████▉   | 26112/37962 [60:29:12<27:04:22,  8.22s/it]
 69%|██████▉   | 26113/37962 [60:29:20<27:04:15,  8.22s/it]
 69%|██████▉   | 26114/37962 [60:29:28<27:10:04,  8.25s/it]
 69%|██████▉   | 26115/37962 [60:29:37<27:18:48,  8.30s/it]
 69%|██████▉   | 26116/37962 [60:29:45<27:17:03,  8.29s/it]
 69%|██████▉   | 26117/37962 [60:29:53<27:13:36,  8.27s/it]
 69%|██████▉   | 26118/37962 [60:30:01<27:15:01,  8.28s/it]
 69%|██████▉   | 26119/37962 [60:30:10<27:21:56,  8.32s/it]
 69%|██████▉   | 26120/37962 [60:30:18<27:17:09,  8.30s/it]
                                                           
{'loss': 1.4255, 'grad_norm': 10.468603134155273, 'learning_rate': 4.684040089233347e-06, 'epoch': 0.69}

 69%|██████▉   | 26120/37962 [60:30:18<27:17:09,  8.30s/it]
 69%|██████▉   | 26121/37962 [60:30:26<27:24:16,  8.33s/it]
 69%|██████▉   | 26122/37962 [60:30:35<27:30:48,  8.37s/it]
 69%|██████▉   | 26123/37962 [60:30:43<27:28:19,  8.35s/it]
 69%|██████▉   | 26124/37962 [60:30:51<27:24:53,  8.34s/it]
 69%|██████▉   | 26125/37962 [60:31:00<27:24:23,  8.34s/it]
 69%|██████▉   | 26126/37962 [60:31:08<27:29:57,  8.36s/it]
 69%|██████▉   | 26127/37962 [60:31:17<27:25:54,  8.34s/it]
 69%|██████▉   | 26128/37962 [60:31:25<27:22:04,  8.33s/it]
 69%|██████▉   | 26129/37962 [60:31:33<27:20:50,  8.32s/it]
 69%|██████▉   | 26130/37962 [60:31:41<27:16:28,  8.30s/it]
                                                           
{'loss': 1.2506, 'grad_norm': 8.233214378356934, 'learning_rate': 4.676815772568978e-06, 'epoch': 0.69}

 69%|██████▉   | 26130/37962 [60:31:41<27:16:28,  8.30s/it]
 69%|██████▉   | 26131/37962 [60:31:50<27:29:24,  8.36s/it]
 69%|██████▉   | 26132/37962 [60:31:58<27:36:45,  8.40s/it]
 69%|██████▉   | 26133/37962 [60:32:07<27:34:05,  8.39s/it]
 69%|██████▉   | 26134/37962 [60:32:15<27:23:02,  8.33s/it]
 69%|██████▉   | 26135/37962 [60:32:23<27:20:15,  8.32s/it]
 69%|██████▉   | 26136/37962 [60:32:31<27:13:34,  8.29s/it]
 69%|██████▉   | 26137/37962 [60:32:40<27:20:14,  8.32s/it]
 69%|██████▉   | 26138/37962 [60:32:48<27:20:47,  8.33s/it]
 69%|██████▉   | 26139/37962 [60:32:56<27:16:56,  8.31s/it]
 69%|██████▉   | 26140/37962 [60:33:05<27:21:33,  8.33s/it]
                                                           
{'loss': 1.2317, 'grad_norm': 9.4558744430542, 'learning_rate': 4.669595330558191e-06, 'epoch': 0.69}

 69%|██████▉   | 26140/37962 [60:33:05<27:21:33,  8.33s/it]
 69%|██████▉   | 26141/37962 [60:33:13<27:14:19,  8.30s/it]
 69%|██████▉   | 26142/37962 [60:33:21<27:18:27,  8.32s/it]
 69%|██████▉   | 26143/37962 [60:33:29<26:56:31,  8.21s/it]
 69%|██████▉   | 26144/37962 [60:33:38<27:08:17,  8.27s/it]
 69%|██████▉   | 26145/37962 [60:33:46<27:02:49,  8.24s/it]
 69%|██████▉   | 26146/37962 [60:33:54<27:05:40,  8.25s/it]
 69%|██████▉   | 26147/37962 [60:34:03<27:05:44,  8.26s/it]
 69%|██████▉   | 26148/37962 [60:34:11<27:01:51,  8.24s/it]
 69%|██████▉   | 26149/37962 [60:34:19<26:49:56,  8.18s/it]
 69%|██████▉   | 26150/37962 [60:34:27<26:50:55,  8.18s/it]
                                                           
{'loss': 1.281, 'grad_norm': 15.419086456298828, 'learning_rate': 4.662378768456623e-06, 'epoch': 0.69}

 69%|██████▉   | 26150/37962 [60:34:27<26:50:55,  8.18s/it]
 69%|██████▉   | 26151/37962 [60:34:35<27:04:30,  8.25s/it]
 69%|██████▉   | 26152/37962 [60:34:43<26:34:20,  8.10s/it]
 69%|██████▉   | 26153/37962 [60:34:51<26:45:33,  8.16s/it]
 69%|██████▉   | 26154/37962 [60:35:00<26:49:23,  8.18s/it]
 69%|██████▉   | 26155/37962 [60:35:08<26:55:14,  8.21s/it]
 69%|██████▉   | 26156/37962 [60:35:16<26:58:17,  8.22s/it]
 69%|██████▉   | 26157/37962 [60:35:24<27:03:16,  8.25s/it]
 69%|██████▉   | 26158/37962 [60:35:33<27:10:42,  8.29s/it]
 69%|██████▉   | 26159/37962 [60:35:41<27:06:54,  8.27s/it]
 69%|██████▉   | 26160/37962 [60:35:50<27:21:25,  8.34s/it]
                                                           
{'loss': 1.1414, 'grad_norm': 12.238937377929688, 'learning_rate': 4.6551660915170796e-06, 'epoch': 0.69}

 69%|██████▉   | 26160/37962 [60:35:50<27:21:25,  8.34s/it]
 69%|██████▉   | 26161/37962 [60:35:58<27:06:41,  8.27s/it]
 69%|██████▉   | 26162/37962 [60:36:06<27:11:05,  8.29s/it]
 69%|██████▉   | 26163/37962 [60:36:14<27:08:10,  8.28s/it]
 69%|██████▉   | 26164/37962 [60:36:23<27:05:35,  8.27s/it]
 69%|██████▉   | 26165/37962 [60:36:31<27:09:26,  8.29s/it]
 69%|██████▉   | 26166/37962 [60:36:39<27:19:48,  8.34s/it]
 69%|██████▉   | 26167/37962 [60:36:48<27:14:06,  8.31s/it]
 69%|██████▉   | 26168/37962 [60:36:56<27:09:24,  8.29s/it]
 69%|██████▉   | 26169/37962 [60:37:04<27:07:24,  8.28s/it]
 69%|██████▉   | 26170/37962 [60:37:12<27:04:31,  8.27s/it]
                                                           
{'loss': 1.3172, 'grad_norm': 8.086023330688477, 'learning_rate': 4.6479573049895474e-06, 'epoch': 0.69}

 69%|██████▉   | 26170/37962 [60:37:12<27:04:31,  8.27s/it]
 69%|██████▉   | 26171/37962 [60:37:20<26:53:09,  8.21s/it]
 69%|██████▉   | 26172/37962 [60:37:29<26:55:46,  8.22s/it]
 69%|██████▉   | 26173/37962 [60:37:37<26:56:32,  8.23s/it]
 69%|██████▉   | 26174/37962 [60:37:45<26:59:44,  8.24s/it]
 69%|██████▉   | 26175/37962 [60:37:53<26:37:11,  8.13s/it]
 69%|██████▉   | 26176/37962 [60:38:01<26:34:19,  8.12s/it]
 69%|██████▉   | 26177/37962 [60:38:09<26:26:21,  8.08s/it]
 69%|██████▉   | 26178/37962 [60:38:17<26:37:01,  8.13s/it]
 69%|██████▉   | 26179/37962 [60:38:26<26:51:41,  8.21s/it]
 69%|██████▉   | 26180/37962 [60:38:34<27:02:01,  8.26s/it]
                                                           
{'loss': 1.1635, 'grad_norm': 12.26124382019043, 'learning_rate': 4.640752414121176e-06, 'epoch': 0.69}

 69%|██████▉   | 26180/37962 [60:38:34<27:02:01,  8.26s/it]
 69%|██████▉   | 26181/37962 [60:38:42<26:43:05,  8.16s/it]
 69%|██████▉   | 26182/37962 [60:38:51<27:00:13,  8.25s/it]
 69%|██████▉   | 26183/37962 [60:38:59<26:58:15,  8.24s/it]
 69%|██████▉   | 26184/37962 [60:39:07<27:03:14,  8.27s/it]
 69%|██████▉   | 26185/37962 [60:39:15<27:04:55,  8.28s/it]
 69%|██████▉   | 26186/37962 [60:39:24<27:13:50,  8.32s/it]
 69%|██████▉   | 26187/37962 [60:39:32<27:16:14,  8.34s/it]
 69%|██████▉   | 26188/37962 [60:39:40<26:51:57,  8.21s/it]
 69%|██████▉   | 26189/37962 [60:39:48<26:55:02,  8.23s/it]
 69%|██████▉   | 26190/37962 [60:39:57<27:00:42,  8.26s/it]
                                                           
{'loss': 1.1649, 'grad_norm': 12.110819816589355, 'learning_rate': 4.633551424156275e-06, 'epoch': 0.69}

 69%|██████▉   | 26190/37962 [60:39:57<27:00:42,  8.26s/it]
 69%|██████▉   | 26191/37962 [60:40:05<27:09:42,  8.31s/it]
 69%|██████▉   | 26192/37962 [60:40:13<26:45:35,  8.18s/it]
 69%|██████▉   | 26193/37962 [60:40:21<26:56:59,  8.24s/it]
 69%|██████▉   | 26194/37962 [60:40:30<26:59:09,  8.26s/it]
 69%|██████▉   | 26195/37962 [60:40:38<27:04:43,  8.28s/it]
 69%|██████▉   | 26196/37962 [60:40:46<26:53:28,  8.23s/it]
 69%|██████▉   | 26197/37962 [60:40:54<26:52:42,  8.22s/it]
 69%|██████▉   | 26198/37962 [60:41:03<26:58:13,  8.25s/it]
 69%|██████▉   | 26199/37962 [60:41:11<26:57:12,  8.25s/it]
 69%|██████▉   | 26200/37962 [60:41:19<26:53:02,  8.23s/it]
                                                           
{'loss': 1.175, 'grad_norm': 11.336727142333984, 'learning_rate': 4.626354340336326e-06, 'epoch': 0.69}

 69%|██████▉   | 26200/37962 [60:41:19<26:53:02,  8.23s/it]
 69%|██████▉   | 26201/37962 [60:41:27<26:55:06,  8.24s/it]
 69%|██████▉   | 26202/37962 [60:41:36<26:54:59,  8.24s/it]
 69%|██████▉   | 26203/37962 [60:41:44<26:52:40,  8.23s/it]
 69%|██████▉   | 26204/37962 [60:41:52<26:41:46,  8.17s/it]
 69%|██████▉   | 26205/37962 [60:42:00<26:47:55,  8.21s/it]
 69%|██████▉   | 26206/37962 [60:42:08<26:40:01,  8.17s/it]
 69%|██████▉   | 26207/37962 [60:42:16<26:37:14,  8.15s/it]
 69%|██████▉   | 26208/37962 [60:42:25<26:45:53,  8.20s/it]
 69%|██████▉   | 26209/37962 [60:42:33<27:07:35,  8.31s/it]
 69%|██████▉   | 26210/37962 [60:42:42<27:09:36,  8.32s/it]
                                                           
{'loss': 1.2481, 'grad_norm': 6.492225646972656, 'learning_rate': 4.619161167899964e-06, 'epoch': 0.69}

 69%|██████▉   | 26210/37962 [60:42:42<27:09:36,  8.32s/it]
 69%|██████▉   | 26211/37962 [60:42:50<27:04:38,  8.30s/it]
 69%|██████▉   | 26212/37962 [60:42:58<26:43:56,  8.19s/it]
 69%|██████▉   | 26213/37962 [60:43:06<26:23:45,  8.09s/it]
 69%|██████▉   | 26214/37962 [60:43:14<26:46:13,  8.20s/it]
 69%|██████▉   | 26215/37962 [60:43:22<26:49:44,  8.22s/it]
 69%|██████▉   | 26216/37962 [60:43:31<27:05:08,  8.30s/it]
 69%|██████▉   | 26217/37962 [60:43:39<27:07:07,  8.31s/it]
 69%|██████▉   | 26218/37962 [60:43:47<27:05:28,  8.30s/it]
 69%|██████▉   | 26219/37962 [60:43:55<26:51:29,  8.23s/it]
 69%|██████▉   | 26220/37962 [60:44:04<26:58:35,  8.27s/it]
                                                           
{'loss': 1.0758, 'grad_norm': 5.356593608856201, 'learning_rate': 4.611971912082971e-06, 'epoch': 0.69}

 69%|██████▉   | 26220/37962 [60:44:04<26:58:35,  8.27s/it]
 69%|██████▉   | 26221/37962 [60:44:12<26:50:03,  8.23s/it]
 69%|██████▉   | 26222/37962 [60:44:20<26:57:19,  8.27s/it]
 69%|██████▉   | 26223/37962 [60:44:29<27:14:18,  8.35s/it]
 69%|██████▉   | 26224/37962 [60:44:37<27:06:46,  8.32s/it]
 69%|██████▉   | 26225/37962 [60:44:45<27:05:42,  8.31s/it]
 69%|██████▉   | 26226/37962 [60:44:54<27:07:29,  8.32s/it]
 69%|██████▉   | 26227/37962 [60:45:02<26:48:41,  8.23s/it]
 69%|██████▉   | 26228/37962 [60:45:10<26:48:56,  8.23s/it]
 69%|██████▉   | 26229/37962 [60:45:18<26:52:25,  8.25s/it]
 69%|██████▉   | 26230/37962 [60:45:26<26:51:21,  8.24s/it]
                                                           
{'loss': 1.2235, 'grad_norm': 7.507218360900879, 'learning_rate': 4.604786578118278e-06, 'epoch': 0.69}

 69%|██████▉   | 26230/37962 [60:45:26<26:51:21,  8.24s/it]
 69%|██████▉   | 26231/37962 [60:45:35<26:53:50,  8.25s/it]
 69%|██████▉   | 26232/37962 [60:45:43<26:56:22,  8.27s/it]
 69%|██████▉   | 26233/37962 [60:45:51<26:58:34,  8.28s/it]
 69%|██████▉   | 26234/37962 [60:46:00<26:54:22,  8.26s/it]
 69%|██████▉   | 26235/37962 [60:46:08<27:01:19,  8.30s/it]
 69%|██████▉   | 26236/37962 [60:46:16<26:43:48,  8.21s/it]
 69%|██████▉   | 26237/37962 [60:46:24<26:37:55,  8.18s/it]
 69%|██████▉   | 26238/37962 [60:46:32<26:47:44,  8.23s/it]
 69%|██████▉   | 26239/37962 [60:46:41<26:45:18,  8.22s/it]
 69%|██████▉   | 26240/37962 [60:46:49<26:51:44,  8.25s/it]
                                                           
{'loss': 1.1808, 'grad_norm': 11.786884307861328, 'learning_rate': 4.597605171235974e-06, 'epoch': 0.69}

 69%|██████▉   | 26240/37962 [60:46:49<26:51:44,  8.25s/it]
 69%|██████▉   | 26241/37962 [60:46:57<27:05:35,  8.32s/it]
 69%|██████▉   | 26242/37962 [60:47:06<27:07:32,  8.33s/it]
 69%|██████▉   | 26243/37962 [60:47:14<27:22:20,  8.41s/it]
 69%|██████▉   | 26244/37962 [60:47:22<27:01:20,  8.30s/it]
 69%|██████▉   | 26245/37962 [60:47:31<27:08:58,  8.34s/it]
 69%|██████▉   | 26246/37962 [60:47:39<27:02:55,  8.31s/it]
 69%|██████▉   | 26247/37962 [60:47:47<26:48:23,  8.24s/it]
 69%|██████▉   | 26248/37962 [60:47:55<26:50:41,  8.25s/it]
 69%|██████▉   | 26249/37962 [60:48:04<26:51:25,  8.25s/it]
 69%|██████▉   | 26250/37962 [60:48:12<26:50:45,  8.25s/it]
                                                           
{'loss': 1.1944, 'grad_norm': 8.464447021484375, 'learning_rate': 4.590427696663271e-06, 'epoch': 0.69}

 69%|██████▉   | 26250/37962 [60:48:12<26:50:45,  8.25s/it]
 69%|██████▉   | 26251/37962 [60:48:20<26:51:04,  8.25s/it]
 69%|██████▉   | 26252/37962 [60:48:28<26:48:52,  8.24s/it]
 69%|██████▉   | 26253/37962 [60:48:37<26:48:44,  8.24s/it]
 69%|██████▉   | 26254/37962 [60:48:45<27:02:54,  8.32s/it]
 69%|██████▉   | 26255/37962 [60:48:53<27:01:44,  8.31s/it]
 69%|██████▉   | 26256/37962 [60:49:02<27:00:18,  8.31s/it]
 69%|██████▉   | 26257/37962 [60:49:10<27:03:13,  8.32s/it]
 69%|██████▉   | 26258/37962 [60:49:18<27:05:42,  8.33s/it]
 69%|██████▉   | 26259/37962 [60:49:27<26:59:48,  8.30s/it]
 69%|██████▉   | 26260/37962 [60:49:35<26:53:46,  8.27s/it]
                                                           
{'loss': 1.2996, 'grad_norm': 6.287354946136475, 'learning_rate': 4.583254159624537e-06, 'epoch': 0.69}

 69%|██████▉   | 26260/37962 [60:49:35<26:53:46,  8.27s/it]
 69%|██████▉   | 26261/37962 [60:49:43<26:50:37,  8.26s/it]
 69%|██████▉   | 26262/37962 [60:49:51<26:48:02,  8.25s/it]
 69%|██████▉   | 26263/37962 [60:49:59<26:39:37,  8.20s/it]
 69%|██████▉   | 26264/37962 [60:50:08<26:38:50,  8.20s/it]
 69%|██████▉   | 26265/37962 [60:50:16<26:39:35,  8.21s/it]
 69%|██████▉   | 26266/37962 [60:50:24<26:47:23,  8.25s/it]
 69%|██████▉   | 26267/37962 [60:50:33<27:07:54,  8.35s/it]
 69%|██████▉   | 26268/37962 [60:50:41<27:09:19,  8.36s/it]
 69%|██████▉   | 26269/37962 [60:50:50<27:16:02,  8.40s/it]
 69%|██████▉   | 26270/37962 [60:50:58<27:02:42,  8.33s/it]
                                                           
{'loss': 1.1879, 'grad_norm': 6.0503458976745605, 'learning_rate': 4.576084565341262e-06, 'epoch': 0.69}

 69%|██████▉   | 26270/37962 [60:50:58<27:02:42,  8.33s/it]
 69%|██████▉   | 26271/37962 [60:51:06<26:59:33,  8.31s/it]
 69%|██████▉   | 26272/37962 [60:51:14<26:52:27,  8.28s/it]
 69%|██████▉   | 26273/37962 [60:51:23<26:54:05,  8.29s/it]
 69%|██████▉   | 26274/37962 [60:51:31<26:41:08,  8.22s/it]
 69%|██████▉   | 26275/37962 [60:51:39<26:19:08,  8.11s/it]
 69%|██████▉   | 26276/37962 [60:51:46<25:58:16,  8.00s/it]
 69%|██████▉   | 26277/37962 [60:51:54<26:08:56,  8.06s/it]
 69%|██████▉   | 26278/37962 [60:52:03<26:32:50,  8.18s/it]
 69%|██████▉   | 26279/37962 [60:52:11<26:37:21,  8.20s/it]
 69%|██████▉   | 26280/37962 [60:52:19<26:37:54,  8.21s/it]
                                                           
{'loss': 1.2748, 'grad_norm': 38.46751403808594, 'learning_rate': 4.5689189190320646e-06, 'epoch': 0.69}

 69%|██████▉   | 26280/37962 [60:52:19<26:37:54,  8.21s/it]
 69%|██████▉   | 26281/37962 [60:52:28<26:48:21,  8.26s/it]
 69%|██████▉   | 26282/37962 [60:52:36<26:58:45,  8.32s/it]
 69%|██████▉   | 26283/37962 [60:52:45<26:56:37,  8.31s/it]
 69%|██████▉   | 26284/37962 [60:52:53<26:55:19,  8.30s/it]
 69%|██████▉   | 26285/37962 [60:53:01<26:54:30,  8.30s/it]
 69%|██████▉   | 26286/37962 [60:53:09<26:45:34,  8.25s/it]
 69%|██████▉   | 26287/37962 [60:53:17<26:43:16,  8.24s/it]
 69%|██████▉   | 26288/37962 [60:53:26<26:46:26,  8.26s/it]
 69%|██████▉   | 26289/37962 [60:53:34<26:56:29,  8.31s/it]
 69%|██████▉   | 26290/37962 [60:53:42<26:51:04,  8.28s/it]
                                                           
{'loss': 1.2878, 'grad_norm': 9.215950965881348, 'learning_rate': 4.561757225912703e-06, 'epoch': 0.69}

 69%|██████▉   | 26290/37962 [60:53:42<26:51:04,  8.28s/it]
 69%|██████▉   | 26291/37962 [60:53:51<27:05:17,  8.36s/it]
 69%|██████▉   | 26292/37962 [60:53:59<27:10:56,  8.39s/it]
 69%|██████▉   | 26293/37962 [60:54:08<27:11:54,  8.39s/it]
 69%|██████▉   | 26294/37962 [60:54:16<27:01:13,  8.34s/it]
 69%|██████▉   | 26295/37962 [60:54:24<26:58:07,  8.32s/it]
 69%|██████▉   | 26296/37962 [60:54:33<26:58:52,  8.33s/it]
 69%|██████▉   | 26297/37962 [60:54:41<26:51:52,  8.29s/it]
 69%|██████▉   | 26298/37962 [60:54:49<26:48:40,  8.28s/it]
 69%|██████▉   | 26299/37962 [60:54:57<26:50:23,  8.28s/it]
 69%|██████▉   | 26300/37962 [60:55:06<26:54:07,  8.30s/it]
                                                           
{'loss': 1.2687, 'grad_norm': 18.78495216369629, 'learning_rate': 4.554599491196042e-06, 'epoch': 0.69}

 69%|██████▉   | 26300/37962 [60:55:06<26:54:07,  8.30s/it]
 69%|██████▉   | 26301/37962 [60:55:14<26:48:27,  8.28s/it]
 69%|██████▉   | 26302/37962 [60:55:22<26:46:03,  8.26s/it]
 69%|██████▉   | 26303/37962 [60:55:31<26:52:40,  8.30s/it]
 69%|██████▉   | 26304/37962 [60:55:39<27:05:32,  8.37s/it]
 69%|██████▉   | 26305/37962 [60:55:48<27:11:49,  8.40s/it]
 69%|██████▉   | 26306/37962 [60:55:56<27:10:38,  8.39s/it]
 69%|██████▉   | 26307/37962 [60:56:04<27:02:41,  8.35s/it]
 69%|██████▉   | 26308/37962 [60:56:12<26:56:19,  8.32s/it]
 69%|██████▉   | 26309/37962 [60:56:21<26:53:17,  8.31s/it]
 69%|██████▉   | 26310/37962 [60:56:29<26:54:32,  8.31s/it]
                                                           
{'loss': 1.2576, 'grad_norm': 111.27274322509766, 'learning_rate': 4.547445720092076e-06, 'epoch': 0.69}

 69%|██████▉   | 26310/37962 [60:56:29<26:54:32,  8.31s/it]
 69%|██████▉   | 26311/37962 [60:56:37<26:52:58,  8.31s/it]
 69%|██████▉   | 26312/37962 [60:56:46<26:52:47,  8.31s/it]
 69%|██████▉   | 26313/37962 [60:56:54<26:56:12,  8.32s/it]
 69%|██████▉   | 26314/37962 [60:57:02<27:01:13,  8.35s/it]
 69%|██████▉   | 26315/37962 [60:57:11<26:54:15,  8.32s/it]
 69%|██████▉   | 26316/37962 [60:57:19<26:58:58,  8.34s/it]
 69%|██████▉   | 26317/37962 [60:57:27<26:23:58,  8.16s/it]
 69%|██████▉   | 26318/37962 [60:57:35<26:26:36,  8.18s/it]
 69%|██████▉   | 26319/37962 [60:57:43<26:07:39,  8.08s/it]
 69%|██████▉   | 26320/37962 [60:57:51<26:05:19,  8.07s/it]
                                                           
{'loss': 1.2667, 'grad_norm': 19.893360137939453, 'learning_rate': 4.540295917807919e-06, 'epoch': 0.69}

 69%|██████▉   | 26320/37962 [60:57:51<26:05:19,  8.07s/it]
 69%|██████▉   | 26321/37962 [60:57:59<26:20:23,  8.15s/it]
 69%|██████▉   | 26322/37962 [60:58:08<26:28:49,  8.19s/it]
 69%|██████▉   | 26323/37962 [60:58:16<26:33:38,  8.22s/it]
 69%|██████▉   | 26324/37962 [60:58:24<26:36:45,  8.23s/it]
 69%|██████▉   | 26325/37962 [60:58:32<26:32:25,  8.21s/it]
 69%|██████▉   | 26326/37962 [60:58:41<26:41:23,  8.26s/it]
 69%|██████▉   | 26327/37962 [60:58:49<26:43:02,  8.27s/it]
 69%|██████▉   | 26328/37962 [60:58:57<26:44:11,  8.27s/it]
 69%|██████▉   | 26329/37962 [60:59:05<26:43:59,  8.27s/it]
 69%|██████▉   | 26330/37962 [60:59:14<26:59:10,  8.35s/it]
                                                           
{'loss': 1.2948, 'grad_norm': 10.132737159729004, 'learning_rate': 4.533150089547773e-06, 'epoch': 0.69}

 69%|██████▉   | 26330/37962 [60:59:14<26:59:10,  8.35s/it]
 69%|██████▉   | 26331/37962 [60:59:22<26:53:11,  8.32s/it]
 69%|██████▉   | 26332/37962 [60:59:30<26:26:07,  8.18s/it]
 69%|██████▉   | 26333/37962 [60:59:38<26:32:10,  8.21s/it]
 69%|██████▉   | 26334/37962 [60:59:46<26:15:49,  8.13s/it]
 69%|██████▉   | 26335/37962 [60:59:55<26:22:07,  8.16s/it]
 69%|██████▉   | 26336/37962 [61:00:03<26:30:41,  8.21s/it]
 69%|██████▉   | 26337/37962 [61:00:11<26:34:19,  8.23s/it]
 69%|██████▉   | 26338/37962 [61:00:19<26:29:07,  8.20s/it]
 69%|██████▉   | 26339/37962 [61:00:27<26:29:11,  8.20s/it]
 69%|██████▉   | 26340/37962 [61:00:36<26:30:31,  8.21s/it]
                                                           
{'loss': 1.3479, 'grad_norm': 5.417535781860352, 'learning_rate': 4.5260082405129705e-06, 'epoch': 0.69}

 69%|██████▉   | 26340/37962 [61:00:36<26:30:31,  8.21s/it]
 69%|██████▉   | 26341/37962 [61:00:44<26:10:43,  8.11s/it]
 69%|██████▉   | 26342/37962 [61:00:52<26:19:32,  8.16s/it]
 69%|██████▉   | 26343/37962 [61:01:00<26:27:58,  8.20s/it]
 69%|██████▉   | 26344/37962 [61:01:09<26:40:17,  8.26s/it]
 69%|██████▉   | 26345/37962 [61:01:17<26:43:07,  8.28s/it]
 69%|██████▉   | 26346/37962 [61:01:25<26:38:19,  8.26s/it]
 69%|██████▉   | 26347/37962 [61:01:33<26:45:01,  8.29s/it]
 69%|██████▉   | 26348/37962 [61:01:42<26:40:34,  8.27s/it]
 69%|██████▉   | 26349/37962 [61:01:50<26:42:21,  8.28s/it]
 69%|██████▉   | 26350/37962 [61:01:58<26:43:02,  8.28s/it]
                                                           
{'loss': 1.262, 'grad_norm': 14.898469924926758, 'learning_rate': 4.5188703759019425e-06, 'epoch': 0.69}

 69%|██████▉   | 26350/37962 [61:01:58<26:43:02,  8.28s/it]
 69%|██████▉   | 26351/37962 [61:02:07<27:04:06,  8.39s/it]
 69%|██████▉   | 26352/37962 [61:02:15<26:54:05,  8.34s/it]
 69%|██████▉   | 26353/37962 [61:02:23<26:51:41,  8.33s/it]
 69%|██████▉   | 26354/37962 [61:02:31<26:35:33,  8.25s/it]
 69%|██████▉   | 26355/37962 [61:02:40<26:35:38,  8.25s/it]
 69%|██████▉   | 26356/37962 [61:02:48<26:32:51,  8.23s/it]
 69%|██████▉   | 26357/37962 [61:02:56<26:30:03,  8.22s/it]
 69%|██████▉   | 26358/37962 [61:03:05<26:45:56,  8.30s/it]
 69%|██████▉   | 26359/37962 [61:03:13<26:59:20,  8.37s/it]
 69%|██████▉   | 26360/37962 [61:03:21<26:46:04,  8.31s/it]
                                                           
{'loss': 1.2263, 'grad_norm': 6.699496269226074, 'learning_rate': 4.511736500910211e-06, 'epoch': 0.69}

 69%|██████▉   | 26360/37962 [61:03:21<26:46:04,  8.31s/it]
 69%|██████▉   | 26361/37962 [61:03:30<26:45:38,  8.30s/it]
 69%|██████▉   | 26362/37962 [61:03:38<26:53:10,  8.34s/it]
 69%|██████▉   | 26363/37962 [61:03:46<26:34:41,  8.25s/it]
 69%|██████▉   | 26364/37962 [61:03:54<26:32:08,  8.24s/it]
 69%|██████▉   | 26365/37962 [61:04:03<26:36:54,  8.26s/it]
 69%|██████▉   | 26366/37962 [61:04:10<26:11:57,  8.13s/it]
 69%|██████▉   | 26367/37962 [61:04:19<26:22:04,  8.19s/it]
 69%|██████▉   | 26368/37962 [61:04:27<26:29:20,  8.22s/it]
 69%|██████▉   | 26369/37962 [61:04:35<26:32:17,  8.24s/it]
 69%|██████▉   | 26370/37962 [61:04:44<26:35:33,  8.26s/it]
                                                           
{'loss': 1.2499, 'grad_norm': 9.356054306030273, 'learning_rate': 4.504606620730405e-06, 'epoch': 0.69}

 69%|██████▉   | 26370/37962 [61:04:44<26:35:33,  8.26s/it]
 69%|██████▉   | 26371/37962 [61:04:52<26:36:29,  8.26s/it]
 69%|██████▉   | 26372/37962 [61:05:00<26:34:20,  8.25s/it]
 69%|██████▉   | 26373/37962 [61:05:09<26:44:25,  8.31s/it]
 69%|██████▉   | 26374/37962 [61:05:17<26:39:12,  8.28s/it]
 69%|██████▉   | 26375/37962 [61:05:25<26:26:57,  8.22s/it]
 69%|██████▉   | 26376/37962 [61:05:33<26:46:13,  8.32s/it]
 69%|██████▉   | 26377/37962 [61:05:42<26:41:45,  8.30s/it]
 69%|██████▉   | 26378/37962 [61:05:50<26:41:25,  8.29s/it]
 69%|██████▉   | 26379/37962 [61:05:58<26:02:52,  8.10s/it]
 69%|██████▉   | 26380/37962 [61:06:06<26:21:27,  8.19s/it]
                                                           
{'loss': 1.2819, 'grad_norm': 4.419393062591553, 'learning_rate': 4.497480740552241e-06, 'epoch': 0.69}

 69%|██████▉   | 26380/37962 [61:06:06<26:21:27,  8.19s/it]
 69%|██████▉   | 26381/37962 [61:06:14<26:26:40,  8.22s/it]
 69%|██████▉   | 26382/37962 [61:06:23<26:36:48,  8.27s/it]
 69%|██████▉   | 26383/37962 [61:06:31<26:20:29,  8.19s/it]
 70%|██████▉   | 26384/37962 [61:06:39<26:06:40,  8.12s/it]
 70%|██████▉   | 26385/37962 [61:06:47<26:18:28,  8.18s/it]
 70%|██████▉   | 26386/37962 [61:06:55<26:04:54,  8.11s/it]
 70%|██████▉   | 26387/37962 [61:07:03<25:54:32,  8.06s/it]
 70%|██████▉   | 26388/37962 [61:07:11<26:07:53,  8.13s/it]
 70%|██████▉   | 26389/37962 [61:07:20<26:31:40,  8.25s/it]
 70%|██████▉   | 26390/37962 [61:07:28<26:17:52,  8.18s/it]
                                                           
{'loss': 1.2072, 'grad_norm': 11.327170372009277, 'learning_rate': 4.490358865562515e-06, 'epoch': 0.7}

 70%|██████▉   | 26390/37962 [61:07:28<26:17:52,  8.18s/it]
 70%|██████▉   | 26391/37962 [61:07:36<26:24:57,  8.22s/it]
 70%|██████▉   | 26392/37962 [61:07:44<26:39:10,  8.29s/it]
 70%|██████▉   | 26393/37962 [61:07:53<26:29:47,  8.25s/it]
 70%|██████▉   | 26394/37962 [61:08:01<26:35:16,  8.27s/it]
 70%|██████▉   | 26395/37962 [61:08:09<26:18:03,  8.19s/it]
 70%|██████▉   | 26396/37962 [61:08:17<26:21:41,  8.21s/it]
 70%|██████▉   | 26397/37962 [61:08:26<26:30:15,  8.25s/it]
 70%|██████▉   | 26398/37962 [61:08:41<33:18:00, 10.37s/it]
 70%|██████▉   | 26399/37962 [61:08:49<31:18:52,  9.75s/it]
 70%|██████▉   | 26400/37962 [61:08:57<29:43:47,  9.26s/it]
                                                           
{'loss': 1.2838, 'grad_norm': 13.916216850280762, 'learning_rate': 4.483241000945128e-06, 'epoch': 0.7}

 70%|██████▉   | 26400/37962 [61:08:57<29:43:47,  9.26s/it]
 70%|██████▉   | 26401/37962 [61:09:06<28:52:00,  8.99s/it]
 70%|██████▉   | 26402/37962 [61:09:14<28:17:21,  8.81s/it]
 70%|██████▉   | 26403/37962 [61:09:22<27:44:16,  8.64s/it]
 70%|██████▉   | 26404/37962 [61:09:31<27:27:39,  8.55s/it]
 70%|██████▉   | 26405/37962 [61:09:39<27:18:10,  8.50s/it]
 70%|██████▉   | 26406/37962 [61:09:47<27:12:33,  8.48s/it]
 70%|██████▉   | 26407/37962 [61:10:03<34:13:24, 10.66s/it]
 70%|██████▉   | 26408/37962 [61:10:12<32:01:31,  9.98s/it]
 70%|██████▉   | 26409/37962 [61:10:20<30:23:14,  9.47s/it]
 70%|██████▉   | 26410/37962 [61:10:28<29:10:52,  9.09s/it]
                                                           
{'loss': 1.2076, 'grad_norm': 8.555780410766602, 'learning_rate': 4.4761271518810415e-06, 'epoch': 0.7}

 70%|██████▉   | 26410/37962 [61:10:28<29:10:52,  9.09s/it]
 70%|██████▉   | 26411/37962 [61:10:36<28:26:58,  8.87s/it]
 70%|██████▉   | 26412/37962 [61:10:45<27:53:00,  8.69s/it]
 70%|██████▉   | 26413/37962 [61:10:53<27:27:07,  8.56s/it]
 70%|██████▉   | 26414/37962 [61:11:01<27:06:23,  8.45s/it]
 70%|██████▉   | 26415/37962 [61:11:10<27:04:13,  8.44s/it]
 70%|██████▉   | 26416/37962 [61:11:18<26:41:40,  8.32s/it]
 70%|██████▉   | 26417/37962 [61:11:26<26:43:48,  8.34s/it]
 70%|██████▉   | 26418/37962 [61:11:34<26:39:02,  8.31s/it]
 70%|██████▉   | 26419/37962 [61:11:43<26:39:58,  8.32s/it]
 70%|██████▉   | 26420/37962 [61:11:51<26:41:08,  8.32s/it]
                                                           
{'loss': 1.1717, 'grad_norm': 3.542222738265991, 'learning_rate': 4.469017323548305e-06, 'epoch': 0.7}

 70%|██████▉   | 26420/37962 [61:11:51<26:41:08,  8.32s/it]
 70%|██████▉   | 26421/37962 [61:11:59<26:44:22,  8.34s/it]
 70%|██████▉   | 26422/37962 [61:12:08<26:43:17,  8.34s/it]
 70%|██████▉   | 26423/37962 [61:12:16<26:37:19,  8.31s/it]
 70%|██████▉   | 26424/37962 [61:12:24<26:32:40,  8.28s/it]
 70%|██████▉   | 26425/37962 [61:12:32<26:35:01,  8.30s/it]
 70%|██████▉   | 26426/37962 [61:12:41<26:31:17,  8.28s/it]
 70%|██████▉   | 26427/37962 [61:12:49<26:16:12,  8.20s/it]
 70%|██████▉   | 26428/37962 [61:12:57<26:12:24,  8.18s/it]
 70%|██████▉   | 26429/37962 [61:13:05<26:22:51,  8.23s/it]
 70%|██████▉   | 26430/37962 [61:13:13<26:13:57,  8.19s/it]
                                                           
{'loss': 1.1913, 'grad_norm': 10.457159996032715, 'learning_rate': 4.461911521122047e-06, 'epoch': 0.7}

 70%|██████▉   | 26430/37962 [61:13:13<26:13:57,  8.19s/it]
 70%|██████▉   | 26431/37962 [61:13:21<26:10:55,  8.17s/it]
 70%|██████▉   | 26432/37962 [61:13:29<25:59:40,  8.12s/it]
 70%|██████▉   | 26433/37962 [61:13:38<26:09:36,  8.17s/it]
 70%|██████▉   | 26434/37962 [61:13:46<26:21:15,  8.23s/it]
 70%|██████▉   | 26435/37962 [61:13:54<26:21:16,  8.23s/it]
 70%|██████▉   | 26436/37962 [61:14:03<26:30:44,  8.28s/it]
 70%|██████▉   | 26437/37962 [61:14:11<26:21:47,  8.23s/it]
 70%|██████▉   | 26438/37962 [61:14:19<26:29:22,  8.28s/it]
 70%|██████▉   | 26439/37962 [61:14:27<26:33:53,  8.30s/it]
 70%|██████▉   | 26440/37962 [61:14:36<26:37:31,  8.32s/it]
                                                           
{'loss': 1.2599, 'grad_norm': 6.690451622009277, 'learning_rate': 4.454809749774449e-06, 'epoch': 0.7}

 70%|██████▉   | 26440/37962 [61:14:36<26:37:31,  8.32s/it]
 70%|██████▉   | 26441/37962 [61:14:44<26:39:33,  8.33s/it]
 70%|██████▉   | 26442/37962 [61:14:53<26:42:08,  8.34s/it]
 70%|██████▉   | 26443/37962 [61:15:01<26:31:17,  8.29s/it]
 70%|██████▉   | 26444/37962 [61:15:09<26:24:38,  8.25s/it]
 70%|██████▉   | 26445/37962 [61:15:17<26:29:33,  8.28s/it]
 70%|██████▉   | 26446/37962 [61:15:26<26:30:14,  8.29s/it]
 70%|██████▉   | 26447/37962 [61:15:34<26:36:57,  8.32s/it]
 70%|██████▉   | 26448/37962 [61:15:42<26:31:45,  8.29s/it]
 70%|██████▉   | 26449/37962 [61:15:50<26:33:28,  8.30s/it]
 70%|██████▉   | 26450/37962 [61:15:59<26:33:57,  8.31s/it]
                                                           
{'loss': 1.3214, 'grad_norm': 14.857367515563965, 'learning_rate': 4.447712014674769e-06, 'epoch': 0.7}

 70%|██████▉   | 26450/37962 [61:15:59<26:33:57,  8.31s/it]
 70%|██████▉   | 26451/37962 [61:16:07<26:03:08,  8.15s/it]
 70%|██████▉   | 26452/37962 [61:16:15<25:54:37,  8.10s/it]
 70%|██████▉   | 26453/37962 [61:16:23<26:08:57,  8.18s/it]
 70%|██████▉   | 26454/37962 [61:16:31<26:16:05,  8.22s/it]
 70%|██████▉   | 26455/37962 [61:16:40<26:19:49,  8.24s/it]
 70%|██████▉   | 26456/37962 [61:16:48<26:07:21,  8.17s/it]
 70%|██████▉   | 26457/37962 [61:16:56<26:24:07,  8.26s/it]
 70%|██████▉   | 26458/37962 [61:17:04<26:24:44,  8.27s/it]
 70%|██████▉   | 26459/37962 [61:17:12<26:09:18,  8.19s/it]
 70%|██████▉   | 26460/37962 [61:17:20<26:04:55,  8.16s/it]
                                                           
{'loss': 1.2782, 'grad_norm': 5.9561662673950195, 'learning_rate': 4.440618320989329e-06, 'epoch': 0.7}

 70%|██████▉   | 26460/37962 [61:17:20<26:04:55,  8.16s/it]
 70%|██████▉   | 26461/37962 [61:17:28<25:53:30,  8.10s/it]
 70%|██████▉   | 26462/37962 [61:17:37<26:18:33,  8.24s/it]
 70%|██████▉   | 26463/37962 [61:17:45<25:59:10,  8.14s/it]
 70%|██████▉   | 26464/37962 [61:17:53<25:56:55,  8.12s/it]
 70%|██████▉   | 26465/37962 [61:18:01<25:59:27,  8.14s/it]
 70%|██████▉   | 26466/37962 [61:18:09<26:11:22,  8.20s/it]
 70%|██████▉   | 26467/37962 [61:18:18<26:24:19,  8.27s/it]
 70%|██████▉   | 26468/37962 [61:18:26<26:28:29,  8.29s/it]
 70%|██████▉   | 26469/37962 [61:18:34<26:13:09,  8.21s/it]
 70%|██████▉   | 26470/37962 [61:18:43<26:24:17,  8.27s/it]
                                                           
{'loss': 1.2418, 'grad_norm': 7.599686145782471, 'learning_rate': 4.4335286738815005e-06, 'epoch': 0.7}

 70%|██████▉   | 26470/37962 [61:18:43<26:24:17,  8.27s/it]
 70%|██████▉   | 26471/37962 [61:18:51<26:25:18,  8.28s/it]
 70%|██████▉   | 26472/37962 [61:18:59<26:32:05,  8.31s/it]
 70%|██████▉   | 26473/37962 [61:19:08<26:40:32,  8.36s/it]
 70%|██████▉   | 26474/37962 [61:19:16<26:20:46,  8.26s/it]
 70%|██████▉   | 26475/37962 [61:19:24<26:14:24,  8.22s/it]
 70%|██████▉   | 26476/37962 [61:19:32<26:26:41,  8.29s/it]
 70%|██████▉   | 26477/37962 [61:19:40<26:11:04,  8.21s/it]
 70%|██████▉   | 26478/37962 [61:19:49<26:14:26,  8.23s/it]
 70%|██████▉   | 26479/37962 [61:19:57<26:10:18,  8.21s/it]
 70%|██████▉   | 26480/37962 [61:20:05<26:24:45,  8.28s/it]
                                                           
{'loss': 1.1854, 'grad_norm': 13.278491020202637, 'learning_rate': 4.426443078511722e-06, 'epoch': 0.7}

 70%|██████▉   | 26480/37962 [61:20:05<26:24:45,  8.28s/it]
 70%|██████▉   | 26481/37962 [61:20:14<26:27:38,  8.30s/it]
 70%|██████▉   | 26482/37962 [61:20:22<26:23:36,  8.28s/it]
 70%|██████▉   | 26483/37962 [61:20:30<26:24:50,  8.28s/it]
 70%|██████▉   | 26484/37962 [61:20:39<26:28:42,  8.30s/it]
 70%|██████▉   | 26485/37962 [61:20:47<26:11:16,  8.21s/it]
 70%|██████▉   | 26486/37962 [61:20:55<25:58:48,  8.15s/it]
 70%|██████▉   | 26487/37962 [61:21:03<26:07:32,  8.20s/it]
 70%|██████▉   | 26488/37962 [61:21:11<26:04:27,  8.18s/it]
 70%|██████▉   | 26489/37962 [61:21:19<26:08:03,  8.20s/it]
 70%|██████▉   | 26490/37962 [61:21:27<26:09:15,  8.21s/it]
                                                           
{'loss': 1.2792, 'grad_norm': 15.866960525512695, 'learning_rate': 4.419361540037471e-06, 'epoch': 0.7}

 70%|██████▉   | 26490/37962 [61:21:27<26:09:15,  8.21s/it]
 70%|██████▉   | 26491/37962 [61:21:36<26:11:07,  8.22s/it]
 70%|██████▉   | 26492/37962 [61:21:44<26:11:00,  8.22s/it]
 70%|██████▉   | 26493/37962 [61:21:52<26:18:38,  8.26s/it]
 70%|██████▉   | 26494/37962 [61:22:01<26:28:24,  8.31s/it]
 70%|██████▉   | 26495/37962 [61:22:09<26:29:56,  8.32s/it]
 70%|██████▉   | 26496/37962 [61:22:17<26:23:53,  8.29s/it]
 70%|██████▉   | 26497/37962 [61:22:26<26:30:47,  8.33s/it]
 70%|██████▉   | 26498/37962 [61:22:33<25:58:11,  8.16s/it]
 70%|██████▉   | 26499/37962 [61:22:42<25:57:49,  8.15s/it]
 70%|██████▉   | 26500/37962 [61:22:50<26:07:07,  8.20s/it]
                                                           
{'loss': 1.2913, 'grad_norm': 12.472566604614258, 'learning_rate': 4.4122840636132734e-06, 'epoch': 0.7}

 70%|██████▉   | 26500/37962 [61:22:50<26:07:07,  8.20s/it]
 70%|██████▉   | 26501/37962 [61:22:58<26:03:02,  8.18s/it]
 70%|██████▉   | 26502/37962 [61:23:06<26:10:46,  8.22s/it]
 70%|██████▉   | 26503/37962 [61:23:15<26:12:52,  8.24s/it]
 70%|██████▉   | 26504/37962 [61:23:23<26:12:34,  8.23s/it]
 70%|██████▉   | 26505/37962 [61:23:31<26:28:41,  8.32s/it]
 70%|██████▉   | 26506/37962 [61:23:40<26:30:23,  8.33s/it]
 70%|██████▉   | 26507/37962 [61:23:48<26:32:32,  8.34s/it]
 70%|██████▉   | 26508/37962 [61:23:56<26:35:19,  8.36s/it]
 70%|██████▉   | 26509/37962 [61:24:04<26:12:12,  8.24s/it]
 70%|██████▉   | 26510/37962 [61:24:13<26:29:28,  8.33s/it]
                                                           
{'loss': 1.1481, 'grad_norm': 12.251481056213379, 'learning_rate': 4.405210654390705e-06, 'epoch': 0.7}

 70%|██████▉   | 26510/37962 [61:24:13<26:29:28,  8.33s/it]
 70%|██████▉   | 26511/37962 [61:24:21<26:29:17,  8.33s/it]
 70%|██████▉   | 26512/37962 [61:24:30<26:33:02,  8.35s/it]
 70%|██████▉   | 26513/37962 [61:24:38<26:24:56,  8.31s/it]
 70%|██████▉   | 26514/37962 [61:24:46<26:22:49,  8.30s/it]
 70%|██████▉   | 26515/37962 [61:24:54<26:14:41,  8.25s/it]
 70%|██████▉   | 26516/37962 [61:25:03<26:17:06,  8.27s/it]
 70%|██████▉   | 26517/37962 [61:25:11<26:13:49,  8.25s/it]
 70%|██████▉   | 26518/37962 [61:25:19<26:24:39,  8.31s/it]
 70%|██████▉   | 26519/37962 [61:25:27<26:15:56,  8.26s/it]
 70%|██████▉   | 26520/37962 [61:25:36<26:08:12,  8.22s/it]
                                                           
{'loss': 1.186, 'grad_norm': 3.742027521133423, 'learning_rate': 4.398141317518382e-06, 'epoch': 0.7}

 70%|██████▉   | 26520/37962 [61:25:36<26:08:12,  8.22s/it]
 70%|██████▉   | 26521/37962 [61:25:44<26:06:48,  8.22s/it]
 70%|██████▉   | 26522/37962 [61:25:52<26:08:07,  8.22s/it]
 70%|██████▉   | 26523/37962 [61:26:00<26:08:43,  8.23s/it]
 70%|██████▉   | 26524/37962 [61:26:08<26:08:50,  8.23s/it]
 70%|██████▉   | 26525/37962 [61:26:17<26:11:52,  8.25s/it]
 70%|██████▉   | 26526/37962 [61:26:25<26:18:01,  8.28s/it]
 70%|██████▉   | 26527/37962 [61:26:33<26:09:06,  8.23s/it]
 70%|██████▉   | 26528/37962 [61:26:42<26:12:44,  8.25s/it]
 70%|██████▉   | 26529/37962 [61:26:50<26:22:57,  8.31s/it]
 70%|██████▉   | 26530/37962 [61:26:58<26:26:47,  8.33s/it]
                                                           
{'loss': 1.1703, 'grad_norm': 10.487710952758789, 'learning_rate': 4.391076058141949e-06, 'epoch': 0.7}

 70%|██████▉   | 26530/37962 [61:26:58<26:26:47,  8.33s/it]
 70%|██████▉   | 26531/37962 [61:27:07<26:27:29,  8.33s/it]
 70%|██████▉   | 26532/37962 [61:27:15<26:24:31,  8.32s/it]
 70%|██████▉   | 26533/37962 [61:27:23<26:23:04,  8.31s/it]
 70%|██████▉   | 26534/37962 [61:27:32<26:19:29,  8.29s/it]
 70%|██████▉   | 26535/37962 [61:27:40<26:24:31,  8.32s/it]
 70%|██████▉   | 26536/37962 [61:27:48<26:20:15,  8.30s/it]
 70%|██████▉   | 26537/37962 [61:27:56<26:18:21,  8.29s/it]
 70%|██████▉   | 26538/37962 [61:28:05<26:16:28,  8.28s/it]
 70%|██████▉   | 26539/37962 [61:28:13<26:04:01,  8.22s/it]
 70%|██████▉   | 26540/37962 [61:28:21<26:07:34,  8.23s/it]
                                                           
{'loss': 1.1961, 'grad_norm': 12.822783470153809, 'learning_rate': 4.38401488140408e-06, 'epoch': 0.7}

 70%|██████▉   | 26540/37962 [61:28:21<26:07:34,  8.23s/it]
 70%|██████▉   | 26541/37962 [61:28:29<26:06:34,  8.23s/it]
 70%|██████▉   | 26542/37962 [61:28:38<26:10:25,  8.25s/it]
 70%|██████▉   | 26543/37962 [61:28:46<26:15:06,  8.28s/it]
 70%|██████▉   | 26544/37962 [61:28:54<25:51:46,  8.15s/it]
 70%|██████▉   | 26545/37962 [61:29:02<26:02:32,  8.21s/it]
 70%|██████▉   | 26546/37962 [61:29:10<26:00:05,  8.20s/it]
 70%|██████▉   | 26547/37962 [61:29:19<26:12:02,  8.26s/it]
 70%|██████▉   | 26548/37962 [61:29:27<26:22:33,  8.32s/it]
 70%|██████▉   | 26549/37962 [61:29:35<26:20:00,  8.31s/it]
 70%|██████▉   | 26550/37962 [61:29:44<26:14:15,  8.28s/it]
                                                           
{'loss': 1.1991, 'grad_norm': 14.203514099121094, 'learning_rate': 4.376957792444494e-06, 'epoch': 0.7}

 70%|██████▉   | 26550/37962 [61:29:44<26:14:15,  8.28s/it]
 70%|██████▉   | 26551/37962 [61:29:52<26:12:00,  8.27s/it]
 70%|██████▉   | 26552/37962 [61:30:00<26:17:11,  8.29s/it]
 70%|██████▉   | 26553/37962 [61:30:08<26:11:26,  8.26s/it]
 70%|██████▉   | 26554/37962 [61:30:17<26:11:44,  8.27s/it]
 70%|██████▉   | 26555/37962 [61:30:25<26:02:55,  8.22s/it]
 70%|██████▉   | 26556/37962 [61:30:33<26:09:20,  8.26s/it]
 70%|██████▉   | 26557/37962 [61:30:41<26:06:33,  8.24s/it]
 70%|██████▉   | 26558/37962 [61:30:49<25:59:54,  8.21s/it]
 70%|██████▉   | 26559/37962 [61:30:58<26:06:30,  8.24s/it]
 70%|██████▉   | 26560/37962 [61:31:06<26:12:53,  8.28s/it]
                                                           
{'loss': 1.2545, 'grad_norm': 6.6220173835754395, 'learning_rate': 4.369904796399914e-06, 'epoch': 0.7}

 70%|██████▉   | 26560/37962 [61:31:06<26:12:53,  8.28s/it]
 70%|██████▉   | 26561/37962 [61:31:14<25:56:31,  8.19s/it]
 70%|██████▉   | 26562/37962 [61:31:22<26:00:24,  8.21s/it]
 70%|██████▉   | 26563/37962 [61:31:31<25:59:56,  8.21s/it]
 70%|██████▉   | 26564/37962 [61:31:39<26:20:00,  8.32s/it]
 70%|██████▉   | 26565/37962 [61:31:48<26:20:52,  8.32s/it]
 70%|██████▉   | 26566/37962 [61:31:56<26:13:49,  8.29s/it]
 70%|██████▉   | 26567/37962 [61:32:04<26:21:33,  8.33s/it]
 70%|██████▉   | 26568/37962 [61:32:12<26:15:03,  8.29s/it]
 70%|██████▉   | 26569/37962 [61:32:21<26:09:57,  8.27s/it]
 70%|██████▉   | 26570/37962 [61:32:28<25:48:56,  8.16s/it]
                                                           
{'loss': 1.0459, 'grad_norm': 7.648800849914551, 'learning_rate': 4.362855898404102e-06, 'epoch': 0.7}

 70%|██████▉   | 26570/37962 [61:32:28<25:48:56,  8.16s/it]
 70%|██████▉   | 26571/37962 [61:32:36<25:37:37,  8.10s/it]
 70%|██████▉   | 26572/37962 [61:32:45<25:39:26,  8.11s/it]
 70%|██████▉   | 26573/37962 [61:32:52<25:27:24,  8.05s/it]
 70%|███████   | 26574/37962 [61:33:01<25:39:27,  8.11s/it]
 70%|███████   | 26575/37962 [61:33:09<25:35:58,  8.09s/it]
 70%|███████   | 26576/37962 [61:33:17<25:44:31,  8.14s/it]
 70%|███████   | 26577/37962 [61:33:25<25:39:59,  8.12s/it]
 70%|███████   | 26578/37962 [61:33:33<25:40:39,  8.12s/it]
 70%|███████   | 26579/37962 [61:33:41<25:34:48,  8.09s/it]
 70%|███████   | 26580/37962 [61:33:50<25:51:26,  8.18s/it]
                                                           
{'loss': 1.2009, 'grad_norm': 10.704607009887695, 'learning_rate': 4.3558111035878206e-06, 'epoch': 0.7}

 70%|███████   | 26580/37962 [61:33:50<25:51:26,  8.18s/it]
 70%|███████   | 26581/37962 [61:33:58<25:57:36,  8.21s/it]
 70%|███████   | 26582/37962 [61:34:06<25:49:06,  8.17s/it]
 70%|███████   | 26583/37962 [61:34:14<25:50:39,  8.18s/it]
 70%|███████   | 26584/37962 [61:34:22<25:51:26,  8.18s/it]
 70%|███████   | 26585/37962 [61:34:30<25:35:28,  8.10s/it]
 70%|███████   | 26586/37962 [61:34:39<25:49:42,  8.17s/it]
 70%|███████   | 26587/37962 [61:34:47<25:47:46,  8.16s/it]
 70%|███████   | 26588/37962 [61:34:55<25:39:00,  8.12s/it]
 70%|███████   | 26589/37962 [61:35:03<25:46:04,  8.16s/it]
 70%|███████   | 26590/37962 [61:35:11<26:01:44,  8.24s/it]
                                                           
{'loss': 1.2185, 'grad_norm': 15.9629487991333, 'learning_rate': 4.348770417078862e-06, 'epoch': 0.7}

 70%|███████   | 26590/37962 [61:35:11<26:01:44,  8.24s/it]
 70%|███████   | 26591/37962 [61:35:19<25:42:38,  8.14s/it]
 70%|███████   | 26592/37962 [61:35:28<25:44:32,  8.15s/it]
 70%|███████   | 26593/37962 [61:35:36<25:45:41,  8.16s/it]
 70%|███████   | 26594/37962 [61:35:44<25:37:31,  8.12s/it]
 70%|███████   | 26595/37962 [61:35:52<25:54:03,  8.20s/it]
 70%|███████   | 26596/37962 [61:36:00<25:58:49,  8.23s/it]
 70%|███████   | 26597/37962 [61:36:09<26:10:46,  8.29s/it]
 70%|███████   | 26598/37962 [61:36:17<26:14:10,  8.31s/it]
 70%|███████   | 26599/37962 [61:36:25<26:10:39,  8.29s/it]
 70%|███████   | 26600/37962 [61:36:34<26:09:21,  8.29s/it]
                                                           
{'loss': 1.2104, 'grad_norm': 9.63800048828125, 'learning_rate': 4.341733844002015e-06, 'epoch': 0.7}

 70%|███████   | 26600/37962 [61:36:34<26:09:21,  8.29s/it]
 70%|███████   | 26601/37962 [61:36:42<25:55:12,  8.21s/it]
 70%|███████   | 26602/37962 [61:36:50<26:09:29,  8.29s/it]
 70%|███████   | 26603/37962 [61:36:58<26:02:28,  8.25s/it]
 70%|███████   | 26604/37962 [61:37:07<26:02:20,  8.25s/it]
 70%|███████   | 26605/37962 [61:37:15<26:03:21,  8.26s/it]
 70%|███████   | 26606/37962 [61:37:24<26:22:00,  8.36s/it]
 70%|███████   | 26607/37962 [61:37:32<26:25:36,  8.38s/it]
 70%|███████   | 26608/37962 [61:37:40<26:07:28,  8.28s/it]
 70%|███████   | 26609/37962 [61:37:48<26:02:59,  8.26s/it]
 70%|███████   | 26610/37962 [61:37:56<25:51:13,  8.20s/it]
                                                           
{'loss': 1.1035, 'grad_norm': 7.120676040649414, 'learning_rate': 4.334701389479078e-06, 'epoch': 0.7}

 70%|███████   | 26610/37962 [61:37:56<25:51:13,  8.20s/it]
 70%|███████   | 26611/37962 [61:38:05<25:57:21,  8.23s/it]
 70%|███████   | 26612/37962 [61:38:13<25:57:45,  8.23s/it]
 70%|███████   | 26613/37962 [61:38:21<25:59:26,  8.24s/it]
 70%|███████   | 26614/37962 [61:38:29<25:58:38,  8.24s/it]
 70%|███████   | 26615/37962 [61:38:38<26:10:21,  8.30s/it]
 70%|███████   | 26616/37962 [61:38:46<25:57:24,  8.24s/it]
 70%|███████   | 26617/37962 [61:38:54<26:03:00,  8.27s/it]
 70%|███████   | 26618/37962 [61:39:03<26:08:40,  8.30s/it]
 70%|███████   | 26619/37962 [61:39:11<26:07:11,  8.29s/it]
 70%|███████   | 26620/37962 [61:39:19<26:01:32,  8.26s/it]
                                                           
{'loss': 1.1586, 'grad_norm': 8.560417175292969, 'learning_rate': 4.327673058628854e-06, 'epoch': 0.7}

 70%|███████   | 26620/37962 [61:39:19<26:01:32,  8.26s/it]
 70%|███████   | 26621/37962 [61:39:27<26:09:19,  8.30s/it]
 70%|███████   | 26622/37962 [61:39:36<26:05:33,  8.28s/it]
 70%|███████   | 26623/37962 [61:39:44<26:06:21,  8.29s/it]
 70%|███████   | 26624/37962 [61:39:52<26:05:26,  8.28s/it]
 70%|███████   | 26625/37962 [61:40:01<26:05:21,  8.28s/it]
 70%|███████   | 26626/37962 [61:40:09<26:00:37,  8.26s/it]
 70%|███████   | 26627/37962 [61:40:17<25:46:36,  8.19s/it]
 70%|███████   | 26628/37962 [61:40:25<25:52:50,  8.22s/it]
 70%|███████   | 26629/37962 [61:40:33<25:56:32,  8.24s/it]
 70%|███████   | 26630/37962 [61:40:42<25:58:24,  8.25s/it]
                                                           
{'loss': 1.3308, 'grad_norm': 16.64198875427246, 'learning_rate': 4.320648856567148e-06, 'epoch': 0.7}

 70%|███████   | 26630/37962 [61:40:42<25:58:24,  8.25s/it]
 70%|███████   | 26631/37962 [61:40:58<33:12:34, 10.55s/it]
 70%|███████   | 26632/37962 [61:41:06<31:09:47,  9.90s/it]
 70%|███████   | 26633/37962 [61:41:14<29:39:17,  9.42s/it]
 70%|███████   | 26634/37962 [61:41:23<28:36:55,  9.09s/it]
 70%|███████   | 26635/37962 [61:41:31<27:56:43,  8.88s/it]
 70%|███████   | 26636/37962 [61:41:39<27:22:22,  8.70s/it]
 70%|███████   | 26637/37962 [61:41:48<27:15:54,  8.67s/it]
 70%|███████   | 26638/37962 [61:41:56<26:47:01,  8.51s/it]
 70%|███████   | 26639/37962 [61:42:04<26:38:08,  8.47s/it]
 70%|███████   | 26640/37962 [61:42:12<26:14:51,  8.35s/it]
                                                           
{'loss': 1.209, 'grad_norm': 11.895329475402832, 'learning_rate': 4.3136287884067515e-06, 'epoch': 0.7}

 70%|███████   | 26640/37962 [61:42:12<26:14:51,  8.35s/it]
 70%|███████   | 26641/37962 [61:42:20<25:51:36,  8.22s/it]
 70%|███████   | 26642/37962 [61:42:28<25:48:33,  8.21s/it]
 70%|███████   | 26643/37962 [61:42:37<25:51:13,  8.22s/it]
 70%|███████   | 26644/37962 [61:42:45<25:58:42,  8.26s/it]
 70%|███████   | 26645/37962 [61:42:53<25:58:24,  8.26s/it]
 70%|███████   | 26646/37962 [61:43:01<25:51:07,  8.22s/it]
 70%|███████   | 26647/37962 [61:43:10<25:43:17,  8.18s/it]
 70%|███████   | 26648/37962 [61:43:18<25:55:14,  8.25s/it]
 70%|███████   | 26649/37962 [61:43:26<25:50:57,  8.23s/it]
 70%|███████   | 26650/37962 [61:43:34<25:48:53,  8.22s/it]
                                                           
{'loss': 1.1209, 'grad_norm': 10.462653160095215, 'learning_rate': 4.306612859257446e-06, 'epoch': 0.7}

 70%|███████   | 26650/37962 [61:43:34<25:48:53,  8.22s/it]
 70%|███████   | 26651/37962 [61:43:43<26:00:43,  8.28s/it]
 70%|███████   | 26652/37962 [61:43:51<25:57:49,  8.26s/it]
 70%|███████   | 26653/37962 [61:43:59<25:49:44,  8.22s/it]
 70%|███████   | 26654/37962 [61:44:07<25:22:12,  8.08s/it]
 70%|███████   | 26655/37962 [61:44:15<25:26:59,  8.10s/it]
 70%|███████   | 26656/37962 [61:44:23<25:32:13,  8.13s/it]
 70%|███████   | 26657/37962 [61:44:32<25:48:55,  8.22s/it]
 70%|███████   | 26658/37962 [61:44:40<25:59:28,  8.28s/it]
 70%|███████   | 26659/37962 [61:44:48<26:02:06,  8.29s/it]
 70%|███████   | 26660/37962 [61:44:56<25:50:17,  8.23s/it]
                                                           
{'loss': 1.2627, 'grad_norm': 6.716972827911377, 'learning_rate': 4.299601074226014e-06, 'epoch': 0.7}

 70%|███████   | 26660/37962 [61:44:56<25:50:17,  8.23s/it]
 70%|███████   | 26661/37962 [61:45:05<25:55:45,  8.26s/it]
 70%|███████   | 26662/37962 [61:45:13<25:34:32,  8.15s/it]
 70%|███████   | 26663/37962 [61:45:21<25:20:18,  8.07s/it]
 70%|███████   | 26664/37962 [61:45:29<25:29:49,  8.12s/it]
 70%|███████   | 26665/37962 [61:45:37<25:52:54,  8.25s/it]
 70%|███████   | 26666/37962 [61:45:46<25:58:21,  8.28s/it]
 70%|███████   | 26667/37962 [61:45:54<26:08:28,  8.33s/it]
 70%|███████   | 26668/37962 [61:46:02<26:06:32,  8.32s/it]
 70%|███████   | 26669/37962 [61:46:11<26:23:24,  8.41s/it]
 70%|███████   | 26670/37962 [61:46:19<26:11:29,  8.35s/it]
                                                           
{'loss': 1.3103, 'grad_norm': 13.060144424438477, 'learning_rate': 4.2925934384162035e-06, 'epoch': 0.7}

 70%|███████   | 26670/37962 [61:46:19<26:11:29,  8.35s/it]
 70%|███████   | 26671/37962 [61:46:28<26:14:34,  8.37s/it]
 70%|███████   | 26672/37962 [61:46:36<25:58:21,  8.28s/it]
 70%|███████   | 26673/37962 [61:46:44<25:51:33,  8.25s/it]
 70%|███████   | 26674/37962 [61:46:52<25:52:52,  8.25s/it]
 70%|███████   | 26675/37962 [61:47:01<25:59:59,  8.29s/it]
 70%|███████   | 26676/37962 [61:47:09<25:52:58,  8.26s/it]
 70%|███████   | 26677/37962 [61:47:17<25:59:28,  8.29s/it]
 70%|███████   | 26678/37962 [61:47:25<25:43:21,  8.21s/it]
 70%|███████   | 26679/37962 [61:47:33<25:41:02,  8.19s/it]
 70%|███████   | 26680/37962 [61:47:42<25:40:07,  8.19s/it]
                                                           
{'loss': 1.2486, 'grad_norm': 13.93526840209961, 'learning_rate': 4.285589956928759e-06, 'epoch': 0.7}

 70%|███████   | 26680/37962 [61:47:42<25:40:07,  8.19s/it]
 70%|███████   | 26681/37962 [61:47:50<25:49:10,  8.24s/it]
 70%|███████   | 26682/37962 [61:47:58<25:52:56,  8.26s/it]
 70%|███████   | 26683/37962 [61:48:06<25:52:18,  8.26s/it]
 70%|███████   | 26684/37962 [61:48:15<25:48:23,  8.24s/it]
 70%|███████   | 26685/37962 [61:48:23<25:49:56,  8.25s/it]
 70%|███████   | 26686/37962 [61:48:31<26:02:08,  8.31s/it]
 70%|███████   | 26687/37962 [61:48:40<25:59:28,  8.30s/it]
 70%|███████   | 26688/37962 [61:48:48<25:59:58,  8.30s/it]
 70%|███████   | 26689/37962 [61:48:56<25:58:36,  8.30s/it]
 70%|███████   | 26690/37962 [61:49:05<26:01:05,  8.31s/it]
                                                           
{'loss': 1.2954, 'grad_norm': 8.40973949432373, 'learning_rate': 4.278590634861388e-06, 'epoch': 0.7}

 70%|███████   | 26690/37962 [61:49:05<26:01:05,  8.31s/it]
 70%|███████   | 26691/37962 [61:49:13<25:57:57,  8.29s/it]
 70%|███████   | 26692/37962 [61:49:21<25:53:39,  8.27s/it]
 70%|███████   | 26693/37962 [61:49:29<25:54:30,  8.28s/it]
 70%|███████   | 26694/37962 [61:49:38<25:55:22,  8.28s/it]
 70%|███████   | 26695/37962 [61:49:46<25:39:32,  8.20s/it]
 70%|███████   | 26696/37962 [61:49:53<25:13:02,  8.06s/it]
 70%|███████   | 26697/37962 [61:50:02<25:32:17,  8.16s/it]
 70%|███████   | 26698/37962 [61:50:10<25:39:07,  8.20s/it]
 70%|███████   | 26699/37962 [61:50:18<25:35:13,  8.18s/it]
 70%|███████   | 26700/37962 [61:50:26<25:41:29,  8.21s/it]
                                                           
{'loss': 1.0858, 'grad_norm': 7.413169860839844, 'learning_rate': 4.271595477308775e-06, 'epoch': 0.7}

 70%|███████   | 26700/37962 [61:50:26<25:41:29,  8.21s/it]
 70%|███████   | 26701/37962 [61:50:35<25:51:00,  8.26s/it]
 70%|███████   | 26702/37962 [61:50:43<25:47:31,  8.25s/it]
 70%|███████   | 26703/37962 [61:50:51<25:55:09,  8.29s/it]
 70%|███████   | 26704/37962 [61:51:00<25:49:01,  8.26s/it]
 70%|███████   | 26705/37962 [61:51:08<25:58:00,  8.30s/it]
 70%|███████   | 26706/37962 [61:51:16<25:38:44,  8.20s/it]
 70%|███████   | 26707/37962 [61:51:24<25:41:48,  8.22s/it]
 70%|███████   | 26708/37962 [61:51:32<25:30:32,  8.16s/it]
 70%|███████   | 26709/37962 [61:51:40<25:14:57,  8.08s/it]
 70%|███████   | 26710/37962 [61:51:48<25:23:10,  8.12s/it]
                                                           
{'loss': 1.092, 'grad_norm': 3.585737705230713, 'learning_rate': 4.264604489362576e-06, 'epoch': 0.7}

 70%|███████   | 26710/37962 [61:51:48<25:23:10,  8.12s/it]
 70%|███████   | 26711/37962 [61:51:57<25:32:14,  8.17s/it]
 70%|███████   | 26712/37962 [61:52:05<25:44:26,  8.24s/it]
 70%|███████   | 26713/37962 [61:52:13<25:50:42,  8.27s/it]
 70%|███████   | 26714/37962 [61:52:21<25:39:35,  8.21s/it]
 70%|███████   | 26715/37962 [61:52:30<25:34:40,  8.19s/it]
 70%|███████   | 26716/37962 [61:52:38<25:51:29,  8.28s/it]
 70%|███████   | 26717/37962 [61:52:46<25:51:05,  8.28s/it]
 70%|███████   | 26718/37962 [61:52:55<25:45:14,  8.25s/it]
 70%|███████   | 26719/37962 [61:53:03<25:49:09,  8.27s/it]
 70%|███████   | 26720/37962 [61:53:11<25:59:10,  8.32s/it]
                                                           
{'loss': 1.3224, 'grad_norm': 10.374383926391602, 'learning_rate': 4.257617676111413e-06, 'epoch': 0.7}

 70%|███████   | 26720/37962 [61:53:11<25:59:10,  8.32s/it]
 70%|███████   | 26721/37962 [61:53:20<25:55:56,  8.31s/it]
 70%|███████   | 26722/37962 [61:53:28<25:46:57,  8.26s/it]
 70%|███████   | 26723/37962 [61:53:36<25:52:56,  8.29s/it]
 70%|███████   | 26724/37962 [61:53:44<25:52:10,  8.29s/it]
 70%|███████   | 26725/37962 [61:53:53<25:46:59,  8.26s/it]
 70%|███████   | 26726/37962 [61:54:01<25:43:25,  8.24s/it]
 70%|███████   | 26727/37962 [61:54:09<25:44:55,  8.25s/it]
 70%|███████   | 26728/37962 [61:54:17<25:29:37,  8.17s/it]
 70%|███████   | 26729/37962 [61:54:25<25:38:23,  8.22s/it]
 70%|███████   | 26730/37962 [61:54:34<25:47:49,  8.27s/it]
                                                           
{'loss': 1.3247, 'grad_norm': 14.501031875610352, 'learning_rate': 4.25063504264086e-06, 'epoch': 0.7}

 70%|███████   | 26730/37962 [61:54:34<25:47:49,  8.27s/it]
 70%|███████   | 26731/37962 [61:54:42<25:47:32,  8.27s/it]
 70%|███████   | 26732/37962 [61:54:50<25:40:06,  8.23s/it]
 70%|███████   | 26733/37962 [61:54:58<25:42:20,  8.24s/it]
 70%|███████   | 26734/37962 [61:55:07<25:57:28,  8.32s/it]
 70%|███████   | 26735/37962 [61:55:15<25:51:06,  8.29s/it]
 70%|███████   | 26736/37962 [61:55:23<25:46:24,  8.27s/it]
 70%|███████   | 26737/37962 [61:55:32<25:49:45,  8.28s/it]
 70%|███████   | 26738/37962 [61:55:40<25:44:41,  8.26s/it]
 70%|███████   | 26739/37962 [61:55:48<25:46:29,  8.27s/it]
 70%|███████   | 26740/37962 [61:55:56<25:47:55,  8.28s/it]
                                                           
{'loss': 1.3065, 'grad_norm': 17.889558792114258, 'learning_rate': 4.243656594033464e-06, 'epoch': 0.7}

 70%|███████   | 26740/37962 [61:55:56<25:47:55,  8.28s/it]
 70%|███████   | 26741/37962 [61:56:05<25:45:09,  8.26s/it]
 70%|███████   | 26742/37962 [61:56:13<25:38:52,  8.23s/it]
 70%|███████   | 26743/37962 [61:56:21<25:44:45,  8.26s/it]
 70%|███████   | 26744/37962 [61:56:29<25:36:36,  8.22s/it]
 70%|███████   | 26745/37962 [61:56:38<25:39:00,  8.23s/it]
 70%|███████   | 26746/37962 [61:56:46<25:37:39,  8.23s/it]
 70%|███████   | 26747/37962 [61:56:54<25:43:14,  8.26s/it]
 70%|███████   | 26748/37962 [61:57:02<25:46:30,  8.27s/it]
 70%|███████   | 26749/37962 [61:57:11<25:40:26,  8.24s/it]
 70%|███████   | 26750/37962 [61:57:19<25:40:30,  8.24s/it]
                                                           
{'loss': 1.2566, 'grad_norm': 10.550854682922363, 'learning_rate': 4.2366823353687105e-06, 'epoch': 0.7}

 70%|███████   | 26750/37962 [61:57:19<25:40:30,  8.24s/it]
 70%|███████   | 26751/37962 [61:57:27<25:42:06,  8.25s/it]
 70%|███████   | 26752/37962 [61:57:35<25:42:50,  8.26s/it]
 70%|███████   | 26753/37962 [61:57:44<25:45:26,  8.27s/it]
 70%|███████   | 26754/37962 [61:57:52<25:47:09,  8.28s/it]
 70%|███████   | 26755/37962 [61:58:00<25:39:33,  8.24s/it]
 70%|███████   | 26756/37962 [61:58:08<25:44:22,  8.27s/it]
 70%|███████   | 26757/37962 [61:58:17<25:49:32,  8.30s/it]
 70%|███████   | 26758/37962 [61:58:25<25:31:25,  8.20s/it]
 70%|███████   | 26759/37962 [61:58:33<25:18:02,  8.13s/it]
 70%|███████   | 26760/37962 [61:58:41<24:59:00,  8.03s/it]
                                                           
{'loss': 1.2376, 'grad_norm': 7.20853328704834, 'learning_rate': 4.229712271723038e-06, 'epoch': 0.7}

 70%|███████   | 26760/37962 [61:58:41<24:59:00,  8.03s/it]
 70%|███████   | 26761/37962 [61:58:49<25:16:32,  8.12s/it]
 70%|███████   | 26762/37962 [61:58:57<25:31:25,  8.20s/it]
 70%|███████   | 26763/37962 [61:59:05<25:30:31,  8.20s/it]
 71%|███████   | 26764/37962 [61:59:14<25:23:09,  8.16s/it]
 71%|███████   | 26765/37962 [61:59:22<25:14:25,  8.12s/it]
 71%|███████   | 26766/37962 [61:59:30<25:29:46,  8.20s/it]
 71%|███████   | 26767/37962 [61:59:38<25:28:48,  8.19s/it]
 71%|███████   | 26768/37962 [61:59:46<25:29:37,  8.20s/it]
 71%|███████   | 26769/37962 [61:59:55<25:36:44,  8.24s/it]
 71%|███████   | 26770/37962 [62:00:03<25:39:05,  8.25s/it]
                                                           
{'loss': 1.172, 'grad_norm': 5.194488525390625, 'learning_rate': 4.222746408169844e-06, 'epoch': 0.71}

 71%|███████   | 26770/37962 [62:00:03<25:39:05,  8.25s/it]
 71%|███████   | 26771/37962 [62:00:11<25:39:43,  8.26s/it]
 71%|███████   | 26772/37962 [62:00:19<25:39:18,  8.25s/it]
 71%|███████   | 26773/37962 [62:00:27<25:18:56,  8.15s/it]
 71%|███████   | 26774/37962 [62:00:35<25:05:00,  8.07s/it]
 71%|███████   | 26775/37962 [62:00:43<25:13:03,  8.12s/it]
 71%|███████   | 26776/37962 [62:00:52<25:14:33,  8.12s/it]
 71%|███████   | 26777/37962 [62:01:00<25:26:19,  8.19s/it]
 71%|███████   | 26778/37962 [62:01:08<25:17:53,  8.14s/it]
 71%|███████   | 26779/37962 [62:01:16<25:26:03,  8.19s/it]
 71%|███████   | 26780/37962 [62:01:25<25:36:46,  8.25s/it]
                                                           
{'loss': 1.2937, 'grad_norm': 8.624528884887695, 'learning_rate': 4.2157847497794516e-06, 'epoch': 0.71}

 71%|███████   | 26780/37962 [62:01:25<25:36:46,  8.25s/it]
 71%|███████   | 26781/37962 [62:01:33<25:32:47,  8.23s/it]
 71%|███████   | 26782/37962 [62:01:41<25:32:09,  8.22s/it]
 71%|███████   | 26783/37962 [62:01:49<25:35:06,  8.24s/it]
 71%|███████   | 26784/37962 [62:01:58<25:41:14,  8.27s/it]
 71%|███████   | 26785/37962 [62:02:06<25:47:13,  8.31s/it]
 71%|███████   | 26786/37962 [62:02:14<25:26:10,  8.19s/it]
 71%|███████   | 26787/37962 [62:02:22<25:39:51,  8.27s/it]
 71%|███████   | 26788/37962 [62:02:31<25:45:33,  8.30s/it]
 71%|███████   | 26789/37962 [62:02:39<25:38:16,  8.26s/it]
 71%|███████   | 26790/37962 [62:02:47<25:30:04,  8.22s/it]
                                                           
{'loss': 1.2639, 'grad_norm': 5.13702917098999, 'learning_rate': 4.208827301619135e-06, 'epoch': 0.71}

 71%|███████   | 26790/37962 [62:02:47<25:30:04,  8.22s/it]
 71%|███████   | 26791/37962 [62:02:56<25:42:10,  8.28s/it]
 71%|███████   | 26792/37962 [62:03:04<25:47:55,  8.31s/it]
 71%|███████   | 26793/37962 [62:03:12<25:42:22,  8.29s/it]
 71%|███████   | 26794/37962 [62:03:20<25:42:06,  8.28s/it]
 71%|███████   | 26795/37962 [62:03:29<25:44:41,  8.30s/it]
 71%|███████   | 26796/37962 [62:03:37<25:41:49,  8.28s/it]
 71%|███████   | 26797/37962 [62:03:45<25:24:54,  8.19s/it]
 71%|███████   | 26798/37962 [62:03:53<25:20:27,  8.17s/it]
 71%|███████   | 26799/37962 [62:04:01<25:28:53,  8.22s/it]
 71%|███████   | 26800/37962 [62:04:10<25:25:09,  8.20s/it]
                                                           
{'loss': 1.2405, 'grad_norm': 16.630889892578125, 'learning_rate': 4.201874068753108e-06, 'epoch': 0.71}

 71%|███████   | 26800/37962 [62:04:10<25:25:09,  8.20s/it]
 71%|███████   | 26801/37962 [62:04:18<25:29:35,  8.22s/it]
 71%|███████   | 26802/37962 [62:04:26<25:35:09,  8.25s/it]
 71%|███████   | 26803/37962 [62:04:34<25:33:56,  8.25s/it]
 71%|███████   | 26804/37962 [62:04:43<25:36:26,  8.26s/it]
 71%|███████   | 26805/37962 [62:04:51<25:28:11,  8.22s/it]
 71%|███████   | 26806/37962 [62:04:59<25:31:06,  8.23s/it]
 71%|███████   | 26807/37962 [62:05:07<25:36:58,  8.27s/it]
 71%|███████   | 26808/37962 [62:05:16<25:37:35,  8.27s/it]
 71%|███████   | 26809/37962 [62:05:24<25:39:22,  8.28s/it]
 71%|███████   | 26810/37962 [62:05:32<25:38:59,  8.28s/it]
                                                           
{'loss': 1.3355, 'grad_norm': 6.863016605377197, 'learning_rate': 4.1949250562424925e-06, 'epoch': 0.71}

 71%|███████   | 26810/37962 [62:05:32<25:38:59,  8.28s/it]
 71%|███████   | 26811/37962 [62:05:41<25:35:34,  8.26s/it]
 71%|███████   | 26812/37962 [62:05:49<25:33:37,  8.25s/it]
 71%|███████   | 26813/37962 [62:05:57<25:38:01,  8.28s/it]
 71%|███████   | 26814/37962 [62:06:05<25:32:48,  8.25s/it]
 71%|███████   | 26815/37962 [62:06:14<25:38:40,  8.28s/it]
 71%|███████   | 26816/37962 [62:06:22<25:21:15,  8.19s/it]
 71%|███████   | 26817/37962 [62:06:30<25:30:44,  8.24s/it]
 71%|███████   | 26818/37962 [62:06:38<25:23:31,  8.20s/it]
 71%|███████   | 26819/37962 [62:06:46<25:21:39,  8.19s/it]
 71%|███████   | 26820/37962 [62:06:54<25:18:03,  8.17s/it]
                                                           
{'loss': 1.3134, 'grad_norm': 4.846906661987305, 'learning_rate': 4.187980269145363e-06, 'epoch': 0.71}

 71%|███████   | 26820/37962 [62:06:54<25:18:03,  8.17s/it]
 71%|███████   | 26821/37962 [62:07:03<25:34:45,  8.27s/it]
 71%|███████   | 26822/37962 [62:07:11<25:32:06,  8.25s/it]
 71%|███████   | 26823/37962 [62:07:19<25:29:40,  8.24s/it]
 71%|███████   | 26824/37962 [62:07:28<25:32:36,  8.26s/it]
 71%|███████   | 26825/37962 [62:07:36<25:23:57,  8.21s/it]
 71%|███████   | 26826/37962 [62:07:44<25:28:15,  8.23s/it]
 71%|███████   | 26827/37962 [62:07:52<25:22:54,  8.21s/it]
 71%|███████   | 26828/37962 [62:08:01<25:34:31,  8.27s/it]
 71%|███████   | 26829/37962 [62:08:09<25:23:18,  8.21s/it]
 71%|███████   | 26830/37962 [62:08:17<25:30:31,  8.25s/it]
                                                           
{'loss': 1.2702, 'grad_norm': 12.716536521911621, 'learning_rate': 4.181039712516713e-06, 'epoch': 0.71}

 71%|███████   | 26830/37962 [62:08:17<25:30:31,  8.25s/it]
 71%|███████   | 26831/37962 [62:08:25<25:37:03,  8.29s/it]
 71%|███████   | 26832/37962 [62:08:34<25:35:33,  8.28s/it]
 71%|███████   | 26833/37962 [62:08:42<25:34:53,  8.28s/it]
 71%|███████   | 26834/37962 [62:08:50<25:40:22,  8.31s/it]
 71%|███████   | 26835/37962 [62:08:58<25:18:13,  8.19s/it]
 71%|███████   | 26836/37962 [62:09:06<25:24:25,  8.22s/it]
 71%|███████   | 26837/37962 [62:09:15<25:25:46,  8.23s/it]
 71%|███████   | 26838/37962 [62:09:23<25:25:08,  8.23s/it]
 71%|███████   | 26839/37962 [62:09:31<25:22:30,  8.21s/it]
 71%|███████   | 26840/37962 [62:09:39<25:28:26,  8.25s/it]
                                                           
{'loss': 1.1471, 'grad_norm': 7.544820308685303, 'learning_rate': 4.174103391408446e-06, 'epoch': 0.71}

 71%|███████   | 26840/37962 [62:09:39<25:28:26,  8.25s/it]
 71%|███████   | 26841/37962 [62:09:48<25:39:09,  8.30s/it]
 71%|███████   | 26842/37962 [62:09:56<25:42:41,  8.32s/it]
 71%|███████   | 26843/37962 [62:10:05<25:40:44,  8.31s/it]
 71%|███████   | 26844/37962 [62:10:13<25:50:26,  8.37s/it]
 71%|███████   | 26845/37962 [62:10:21<25:52:37,  8.38s/it]
 71%|███████   | 26846/37962 [62:10:30<25:40:00,  8.31s/it]
 71%|███████   | 26847/37962 [62:10:38<25:31:15,  8.27s/it]
 71%|███████   | 26848/37962 [62:10:46<25:43:24,  8.33s/it]
 71%|███████   | 26849/37962 [62:10:54<25:38:22,  8.31s/it]
 71%|███████   | 26850/37962 [62:11:03<25:42:09,  8.33s/it]
                                                           
{'loss': 1.142, 'grad_norm': 10.287220001220703, 'learning_rate': 4.167171310869398e-06, 'epoch': 0.71}

 71%|███████   | 26850/37962 [62:11:03<25:42:09,  8.33s/it]
 71%|███████   | 26851/37962 [62:11:11<25:48:09,  8.36s/it]
 71%|███████   | 26852/37962 [62:11:20<25:45:01,  8.34s/it]
 71%|███████   | 26853/37962 [62:11:28<25:40:38,  8.32s/it]
 71%|███████   | 26854/37962 [62:11:36<25:38:31,  8.31s/it]
 71%|███████   | 26855/37962 [62:11:44<25:39:36,  8.32s/it]
 71%|███████   | 26856/37962 [62:11:52<25:21:34,  8.22s/it]
 71%|███████   | 26857/37962 [62:12:00<25:03:20,  8.12s/it]
 71%|███████   | 26858/37962 [62:12:09<25:17:33,  8.20s/it]
 71%|███████   | 26859/37962 [62:12:17<25:14:58,  8.19s/it]
 71%|███████   | 26860/37962 [62:12:25<25:09:25,  8.16s/it]
                                                           
{'loss': 1.1947, 'grad_norm': 9.600506782531738, 'learning_rate': 4.160243475945305e-06, 'epoch': 0.71}

 71%|███████   | 26860/37962 [62:12:25<25:09:25,  8.16s/it]
 71%|███████   | 26861/37962 [62:12:33<25:17:09,  8.20s/it]
 71%|███████   | 26862/37962 [62:12:42<25:18:15,  8.21s/it]
 71%|███████   | 26863/37962 [62:12:50<25:28:39,  8.26s/it]
 71%|███████   | 26864/37962 [62:12:58<25:30:02,  8.27s/it]
 71%|███████   | 26865/37962 [62:13:07<25:32:48,  8.29s/it]
 71%|███████   | 26866/37962 [62:13:15<25:35:21,  8.30s/it]
 71%|███████   | 26867/37962 [62:13:23<25:19:46,  8.22s/it]
 71%|███████   | 26868/37962 [62:13:31<25:25:23,  8.25s/it]
 71%|███████   | 26869/37962 [62:13:39<25:25:07,  8.25s/it]
 71%|███████   | 26870/37962 [62:13:48<25:29:36,  8.27s/it]
                                                           
{'loss': 1.2426, 'grad_norm': 19.258453369140625, 'learning_rate': 4.1533198916788155e-06, 'epoch': 0.71}

 71%|███████   | 26870/37962 [62:13:48<25:29:36,  8.27s/it]
 71%|███████   | 26871/37962 [62:13:56<25:27:18,  8.26s/it]
 71%|███████   | 26872/37962 [62:14:05<25:40:22,  8.33s/it]
 71%|███████   | 26873/37962 [62:14:12<25:14:45,  8.20s/it]
 71%|███████   | 26874/37962 [62:14:21<25:22:55,  8.24s/it]
 71%|███████   | 26875/37962 [62:14:29<25:27:06,  8.26s/it]
 71%|███████   | 26876/37962 [62:14:37<25:28:38,  8.27s/it]
 71%|███████   | 26877/37962 [62:14:46<25:31:29,  8.29s/it]
 71%|███████   | 26878/37962 [62:14:54<25:29:08,  8.28s/it]
 71%|███████   | 26879/37962 [62:15:02<25:30:37,  8.29s/it]
 71%|███████   | 26880/37962 [62:15:10<25:03:09,  8.14s/it]
                                                           
{'loss': 1.2521, 'grad_norm': 9.592848777770996, 'learning_rate': 4.146400563109494e-06, 'epoch': 0.71}

 71%|███████   | 26880/37962 [62:15:10<25:03:09,  8.14s/it]
 71%|███████   | 26881/37962 [62:15:18<25:12:04,  8.19s/it]
 71%|███████   | 26882/37962 [62:15:27<25:14:30,  8.20s/it]
 71%|███████   | 26883/37962 [62:15:35<25:23:12,  8.25s/it]
 71%|███████   | 26884/37962 [62:15:43<25:24:18,  8.26s/it]
 71%|███████   | 26885/37962 [62:15:52<25:29:17,  8.28s/it]
 71%|███████   | 26886/37962 [62:16:00<25:23:20,  8.25s/it]
 71%|███████   | 26887/37962 [62:16:08<25:25:09,  8.26s/it]
 71%|███████   | 26888/37962 [62:16:16<25:15:35,  8.21s/it]
 71%|███████   | 26889/37962 [62:16:25<25:25:27,  8.27s/it]
 71%|███████   | 26890/37962 [62:16:32<25:02:43,  8.14s/it]
                                                           
{'loss': 1.043, 'grad_norm': 7.419201850891113, 'learning_rate': 4.139485495273792e-06, 'epoch': 0.71}

 71%|███████   | 26890/37962 [62:16:32<25:02:43,  8.14s/it]
 71%|███████   | 26891/37962 [62:16:41<25:11:50,  8.19s/it]
 71%|███████   | 26892/37962 [62:16:49<25:21:31,  8.25s/it]
 71%|███████   | 26893/37962 [62:16:57<25:27:20,  8.28s/it]
 71%|███████   | 26894/37962 [62:17:06<25:26:15,  8.27s/it]
 71%|███████   | 26895/37962 [62:17:14<25:30:06,  8.30s/it]
 71%|███████   | 26896/37962 [62:17:22<25:08:42,  8.18s/it]
 71%|███████   | 26897/37962 [62:17:30<25:16:23,  8.22s/it]
 71%|███████   | 26898/37962 [62:17:38<25:17:02,  8.23s/it]
 71%|███████   | 26899/37962 [62:17:47<25:13:10,  8.21s/it]
 71%|███████   | 26900/37962 [62:17:55<25:12:37,  8.20s/it]
                                                           
{'loss': 1.249, 'grad_norm': 11.28185749053955, 'learning_rate': 4.132574693205076e-06, 'epoch': 0.71}

 71%|███████   | 26900/37962 [62:17:55<25:12:37,  8.20s/it]
 71%|███████   | 26901/37962 [62:18:03<25:03:24,  8.16s/it]
 71%|███████   | 26902/37962 [62:18:11<25:15:27,  8.22s/it]
 71%|███████   | 26903/37962 [62:18:20<25:21:02,  8.25s/it]
 71%|███████   | 26904/37962 [62:18:28<25:13:49,  8.21s/it]
 71%|███████   | 26905/37962 [62:18:36<25:05:05,  8.17s/it]
 71%|███████   | 26906/37962 [62:18:44<25:13:15,  8.21s/it]
 71%|███████   | 26907/37962 [62:18:52<25:22:40,  8.26s/it]
 71%|███████   | 26908/37962 [62:19:00<24:45:28,  8.06s/it]
 71%|███████   | 26909/37962 [62:19:08<24:55:04,  8.12s/it]
 71%|███████   | 26910/37962 [62:19:17<25:08:25,  8.19s/it]
                                                           
{'loss': 1.1904, 'grad_norm': 3.7899718284606934, 'learning_rate': 4.125668161933594e-06, 'epoch': 0.71}

 71%|███████   | 26910/37962 [62:19:17<25:08:25,  8.19s/it]
 71%|███████   | 26911/37962 [62:19:25<25:09:32,  8.20s/it]
 71%|███████   | 26912/37962 [62:19:33<25:26:24,  8.29s/it]
 71%|███████   | 26913/37962 [62:19:41<25:02:44,  8.16s/it]
 71%|███████   | 26914/37962 [62:19:50<25:09:07,  8.20s/it]
 71%|███████   | 26915/37962 [62:19:58<25:03:13,  8.16s/it]
 71%|███████   | 26916/37962 [62:20:06<24:57:10,  8.13s/it]
 71%|███████   | 26917/37962 [62:20:14<25:29:44,  8.31s/it]
 71%|███████   | 26918/37962 [62:20:23<25:20:00,  8.26s/it]
 71%|███████   | 26919/37962 [62:20:31<25:04:52,  8.18s/it]
 71%|███████   | 26920/37962 [62:20:39<25:16:47,  8.24s/it]
                                                           
{'loss': 1.2099, 'grad_norm': 9.760143280029297, 'learning_rate': 4.1187659064864875e-06, 'epoch': 0.71}

 71%|███████   | 26920/37962 [62:20:39<25:16:47,  8.24s/it]
 71%|███████   | 26921/37962 [62:20:47<25:28:25,  8.31s/it]
 71%|███████   | 26922/37962 [62:20:55<25:02:42,  8.17s/it]
 71%|███████   | 26923/37962 [62:21:03<24:51:32,  8.11s/it]
 71%|███████   | 26924/37962 [62:21:12<25:04:34,  8.18s/it]
 71%|███████   | 26925/37962 [62:21:20<25:12:17,  8.22s/it]
 71%|███████   | 26926/37962 [62:21:28<25:05:03,  8.18s/it]
 71%|███████   | 26927/37962 [62:21:36<25:21:57,  8.28s/it]
 71%|███████   | 26928/37962 [62:21:45<25:31:53,  8.33s/it]
 71%|███████   | 26929/37962 [62:21:53<25:31:12,  8.33s/it]
 71%|███████   | 26930/37962 [62:22:02<25:32:31,  8.34s/it]
                                                           
{'loss': 1.1547, 'grad_norm': 11.602234840393066, 'learning_rate': 4.1118679318877915e-06, 'epoch': 0.71}

 71%|███████   | 26930/37962 [62:22:02<25:32:31,  8.34s/it]
 71%|███████   | 26931/37962 [62:22:10<25:28:49,  8.32s/it]
 71%|███████   | 26932/37962 [62:22:18<25:26:10,  8.30s/it]
 71%|███████   | 26933/37962 [62:22:26<25:23:25,  8.29s/it]
 71%|███████   | 26934/37962 [62:22:35<25:22:29,  8.28s/it]
 71%|███████   | 26935/37962 [62:22:43<25:31:11,  8.33s/it]
 71%|███████   | 26936/37962 [62:22:51<25:29:29,  8.32s/it]
 71%|███████   | 26937/37962 [62:23:00<25:35:55,  8.36s/it]
 71%|███████   | 26938/37962 [62:23:08<25:17:16,  8.26s/it]
 71%|███████   | 26939/37962 [62:23:16<25:20:47,  8.28s/it]
 71%|███████   | 26940/37962 [62:23:25<25:28:16,  8.32s/it]
                                                           
{'loss': 1.2753, 'grad_norm': 8.613829612731934, 'learning_rate': 4.104974243158427e-06, 'epoch': 0.71}

 71%|███████   | 26940/37962 [62:23:25<25:28:16,  8.32s/it]
 71%|███████   | 26941/37962 [62:23:33<25:38:08,  8.37s/it]
 71%|███████   | 26942/37962 [62:23:41<25:01:09,  8.17s/it]
 71%|███████   | 26943/37962 [62:23:49<25:07:20,  8.21s/it]
 71%|███████   | 26944/37962 [62:23:57<25:12:34,  8.24s/it]
 71%|███████   | 26945/37962 [62:24:06<25:24:18,  8.30s/it]
 71%|███████   | 26946/37962 [62:24:14<25:27:57,  8.32s/it]
 71%|███████   | 26947/37962 [62:24:22<25:15:05,  8.25s/it]
 71%|███████   | 26948/37962 [62:24:30<25:09:57,  8.23s/it]
 71%|███████   | 26949/37962 [62:24:39<25:18:53,  8.28s/it]
 71%|███████   | 26950/37962 [62:24:47<25:26:30,  8.32s/it]
                                                           
{'loss': 1.1203, 'grad_norm': 6.963531970977783, 'learning_rate': 4.098084845316187e-06, 'epoch': 0.71}

 71%|███████   | 26950/37962 [62:24:47<25:26:30,  8.32s/it]
 71%|███████   | 26951/37962 [62:24:55<25:19:08,  8.28s/it]
 71%|███████   | 26952/37962 [62:25:04<25:06:58,  8.21s/it]
 71%|███████   | 26953/37962 [62:25:12<25:01:39,  8.18s/it]
 71%|███████   | 26954/37962 [62:25:20<25:07:46,  8.22s/it]
 71%|███████   | 26955/37962 [62:25:28<24:52:42,  8.14s/it]
 71%|███████   | 26956/37962 [62:25:36<24:58:12,  8.17s/it]
 71%|███████   | 26957/37962 [62:25:44<25:05:58,  8.21s/it]
 71%|███████   | 26958/37962 [62:25:52<24:52:19,  8.14s/it]
 71%|███████   | 26959/37962 [62:26:01<24:52:54,  8.14s/it]
 71%|███████   | 26960/37962 [62:26:09<24:57:50,  8.17s/it]
                                                           
{'loss': 1.1829, 'grad_norm': 12.72823715209961, 'learning_rate': 4.091199743375741e-06, 'epoch': 0.71}

 71%|███████   | 26960/37962 [62:26:09<24:57:50,  8.17s/it]
 71%|███████   | 26961/37962 [62:26:17<25:06:16,  8.22s/it]
 71%|███████   | 26962/37962 [62:26:25<25:05:27,  8.21s/it]
 71%|███████   | 26963/37962 [62:26:33<25:01:07,  8.19s/it]
 71%|███████   | 26964/37962 [62:26:42<25:03:38,  8.20s/it]
 71%|███████   | 26965/37962 [62:26:50<25:11:23,  8.25s/it]
 71%|███████   | 26966/37962 [62:26:58<25:13:35,  8.26s/it]
 71%|███████   | 26967/37962 [62:27:07<25:20:01,  8.29s/it]
 71%|███████   | 26968/37962 [62:27:15<25:17:13,  8.28s/it]
 71%|███████   | 26969/37962 [62:27:23<25:19:23,  8.29s/it]
 71%|███████   | 26970/37962 [62:27:32<25:21:02,  8.30s/it]
                                                           
{'loss': 1.2105, 'grad_norm': 11.918359756469727, 'learning_rate': 4.084318942348642e-06, 'epoch': 0.71}

 71%|███████   | 26970/37962 [62:27:32<25:21:02,  8.30s/it]
 71%|███████   | 26971/37962 [62:27:40<25:18:42,  8.29s/it]
 71%|███████   | 26972/37962 [62:27:48<24:54:03,  8.16s/it]
 71%|███████   | 26973/37962 [62:27:56<24:54:29,  8.16s/it]
 71%|███████   | 26974/37962 [62:28:04<25:06:58,  8.23s/it]
 71%|███████   | 26975/37962 [62:28:20<32:02:46, 10.50s/it]
 71%|███████   | 26976/37962 [62:28:28<29:57:26,  9.82s/it]
 71%|███████   | 26977/37962 [62:28:36<28:21:40,  9.29s/it]
 71%|███████   | 26978/37962 [62:28:44<27:13:14,  8.92s/it]
 71%|███████   | 26979/37962 [62:28:53<26:45:43,  8.77s/it]
 71%|███████   | 26980/37962 [62:29:01<26:14:45,  8.60s/it]
                                                           
{'loss': 1.2699, 'grad_norm': 8.371241569519043, 'learning_rate': 4.0774424472433e-06, 'epoch': 0.71}

 71%|███████   | 26980/37962 [62:29:01<26:14:45,  8.60s/it]
 71%|███████   | 26981/37962 [62:29:09<26:04:59,  8.55s/it]
 71%|███████   | 26982/37962 [62:29:18<25:48:32,  8.46s/it]
 71%|███████   | 26983/37962 [62:29:26<25:33:09,  8.38s/it]
 71%|███████   | 26984/37962 [62:29:34<25:43:07,  8.43s/it]
 71%|███████   | 26985/37962 [62:29:43<25:39:07,  8.41s/it]
 71%|███████   | 26986/37962 [62:29:51<25:18:31,  8.30s/it]
 71%|███████   | 26987/37962 [62:29:59<25:25:11,  8.34s/it]
 71%|███████   | 26988/37962 [62:30:07<25:14:07,  8.28s/it]
 71%|███████   | 26989/37962 [62:30:16<25:19:41,  8.31s/it]
 71%|███████   | 26990/37962 [62:30:24<25:26:15,  8.35s/it]
                                                           
{'loss': 1.2558, 'grad_norm': 16.880403518676758, 'learning_rate': 4.070570263064998e-06, 'epoch': 0.71}

 71%|███████   | 26990/37962 [62:30:24<25:26:15,  8.35s/it]
 71%|███████   | 26991/37962 [62:30:33<25:28:36,  8.36s/it]
 71%|███████   | 26992/37962 [62:30:41<25:35:10,  8.40s/it]
 71%|███████   | 26993/37962 [62:30:49<25:30:52,  8.37s/it]
 71%|███████   | 26994/37962 [62:30:58<25:31:07,  8.38s/it]
 71%|███████   | 26995/37962 [62:31:06<25:28:56,  8.36s/it]
 71%|███████   | 26996/37962 [62:31:14<25:25:45,  8.35s/it]
 71%|███████   | 26997/37962 [62:31:23<25:16:03,  8.30s/it]
 71%|███████   | 26998/37962 [62:31:31<25:07:33,  8.25s/it]
 71%|███████   | 26999/37962 [62:31:39<24:59:49,  8.21s/it]
 71%|███████   | 27000/37962 [62:31:47<24:47:00,  8.14s/it]
                                                           
{'loss': 1.1944, 'grad_norm': 21.592655181884766, 'learning_rate': 4.0637023948158864e-06, 'epoch': 0.71}

 71%|███████   | 27000/37962 [62:31:47<24:47:00,  8.14s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 71%|███████   | 27001/37962 [62:32:37<63:25:39, 20.83s/it]
 71%|███████   | 27002/37962 [62:32:46<51:58:16, 17.07s/it]
 71%|███████   | 27003/37962 [62:32:54<43:55:28, 14.43s/it]
 71%|███████   | 27004/37962 [62:33:02<38:19:44, 12.59s/it]
 71%|███████   | 27005/37962 [62:33:10<34:12:36, 11.24s/it]
 71%|███████   | 27006/37962 [62:33:18<31:05:13, 10.21s/it]
 71%|███████   | 27007/37962 [62:33:27<29:33:45,  9.71s/it]
 71%|███████   | 27008/37962 [62:33:35<28:15:49,  9.29s/it]
 71%|███████   | 27009/37962 [62:33:43<27:20:21,  8.99s/it]
 71%|███████   | 27010/37962 [62:33:51<26:36:20,  8.75s/it]
                                                           
{'loss': 1.2168, 'grad_norm': 6.380605697631836, 'learning_rate': 4.05683884749496e-06, 'epoch': 0.71}

 71%|███████   | 27010/37962 [62:33:51<26:36:20,  8.75s/it]
 71%|███████   | 27011/37962 [62:34:00<26:16:00,  8.63s/it]
 71%|███████   | 27012/37962 [62:34:08<25:51:20,  8.50s/it]
 71%|███████   | 27013/37962 [62:34:16<25:29:06,  8.38s/it]
 71%|███████   | 27014/37962 [62:34:24<25:19:54,  8.33s/it]
 71%|███████   | 27015/37962 [62:34:33<25:19:29,  8.33s/it]
 71%|███████   | 27016/37962 [62:34:41<25:16:50,  8.31s/it]
 71%|███████   | 27017/37962 [62:34:49<25:25:07,  8.36s/it]
 71%|███████   | 27018/37962 [62:34:58<25:15:27,  8.31s/it]
 71%|███████   | 27019/37962 [62:35:06<25:06:35,  8.26s/it]
 71%|███████   | 27020/37962 [62:35:14<25:10:30,  8.28s/it]
                                                           
{'loss': 1.2473, 'grad_norm': 4.519338607788086, 'learning_rate': 4.049979626098076e-06, 'epoch': 0.71}

 71%|███████   | 27020/37962 [62:35:14<25:10:30,  8.28s/it]
 71%|███████   | 27021/37962 [62:35:23<25:27:09,  8.37s/it]
 71%|███████   | 27022/37962 [62:35:31<25:20:05,  8.34s/it]
 71%|███████   | 27023/37962 [62:35:39<25:25:55,  8.37s/it]
 71%|███████   | 27024/37962 [62:35:48<25:26:36,  8.37s/it]
 71%|███████   | 27025/37962 [62:35:56<25:13:35,  8.30s/it]
 71%|███████   | 27026/37962 [62:36:04<25:12:31,  8.30s/it]
 71%|███████   | 27027/37962 [62:36:12<25:16:59,  8.32s/it]
 71%|███████   | 27028/37962 [62:36:21<25:01:36,  8.24s/it]
 71%|███████   | 27029/37962 [62:36:29<24:59:41,  8.23s/it]
 71%|███████   | 27030/37962 [62:36:37<25:06:26,  8.27s/it]
                                                           
{'loss': 1.1218, 'grad_norm': 5.7306108474731445, 'learning_rate': 4.0431247356179494e-06, 'epoch': 0.71}

 71%|███████   | 27030/37962 [62:36:37<25:06:26,  8.27s/it]
 71%|███████   | 27031/37962 [62:36:45<25:00:52,  8.24s/it]
 71%|███████   | 27032/37962 [62:36:53<24:39:25,  8.12s/it]
 71%|███████   | 27033/37962 [62:37:02<24:55:51,  8.21s/it]
 71%|███████   | 27034/37962 [62:37:10<24:59:52,  8.24s/it]
 71%|███████   | 27035/37962 [62:37:18<24:58:01,  8.23s/it]
 71%|███████   | 27036/37962 [62:37:27<25:11:14,  8.30s/it]
 71%|███████   | 27037/37962 [62:37:35<25:04:28,  8.26s/it]
 71%|███████   | 27038/37962 [62:37:43<25:06:56,  8.28s/it]
 71%|███████   | 27039/37962 [62:37:51<25:07:50,  8.28s/it]
 71%|███████   | 27040/37962 [62:38:00<25:08:46,  8.29s/it]
                                                           
{'loss': 1.2691, 'grad_norm': 8.584068298339844, 'learning_rate': 4.036274181044127e-06, 'epoch': 0.71}

 71%|███████   | 27040/37962 [62:38:00<25:08:46,  8.29s/it]
 71%|███████   | 27041/37962 [62:38:08<25:16:42,  8.33s/it]
 71%|███████   | 27042/37962 [62:38:16<25:16:02,  8.33s/it]
 71%|███████   | 27043/37962 [62:38:25<25:15:24,  8.33s/it]
 71%|███████   | 27044/37962 [62:38:33<25:21:07,  8.36s/it]
 71%|███████   | 27045/37962 [62:38:42<25:32:00,  8.42s/it]
 71%|███████   | 27046/37962 [62:38:50<25:31:27,  8.42s/it]
 71%|███████   | 27047/37962 [62:38:58<25:23:35,  8.38s/it]
 71%|███████▏  | 27048/37962 [62:39:06<25:10:19,  8.30s/it]
 71%|███████▏  | 27049/37962 [62:39:15<25:09:55,  8.30s/it]
 71%|███████▏  | 27050/37962 [62:39:23<24:42:37,  8.15s/it]
                                                           
{'loss': 1.382, 'grad_norm': 16.393827438354492, 'learning_rate': 4.029427967363019e-06, 'epoch': 0.71}

 71%|███████▏  | 27050/37962 [62:39:23<24:42:37,  8.15s/it]
 71%|███████▏  | 27051/37962 [62:39:31<24:47:35,  8.18s/it]
 71%|███████▏  | 27052/37962 [62:39:39<24:51:15,  8.20s/it]
 71%|███████▏  | 27053/37962 [62:39:48<25:07:07,  8.29s/it]
 71%|███████▏  | 27054/37962 [62:39:56<25:04:10,  8.27s/it]
 71%|███████▏  | 27055/37962 [62:40:04<25:17:07,  8.35s/it]
 71%|███████▏  | 27056/37962 [62:40:12<24:58:02,  8.24s/it]
 71%|███████▏  | 27057/37962 [62:40:21<25:12:35,  8.32s/it]
 71%|███████▏  | 27058/37962 [62:40:29<24:50:16,  8.20s/it]
 71%|███████▏  | 27059/37962 [62:40:37<25:03:07,  8.27s/it]
 71%|███████▏  | 27060/37962 [62:40:45<25:00:44,  8.26s/it]
                                                           
{'loss': 1.2275, 'grad_norm': 17.976877212524414, 'learning_rate': 4.022586099557859e-06, 'epoch': 0.71}

 71%|███████▏  | 27060/37962 [62:40:45<25:00:44,  8.26s/it]
 71%|███████▏  | 27061/37962 [62:40:54<25:01:11,  8.26s/it]
 71%|███████▏  | 27062/37962 [62:41:02<25:08:29,  8.30s/it]
 71%|███████▏  | 27063/37962 [62:41:10<25:02:06,  8.27s/it]
 71%|███████▏  | 27064/37962 [62:41:19<25:02:19,  8.27s/it]
 71%|███████▏  | 27065/37962 [62:41:27<24:45:09,  8.18s/it]
 71%|███████▏  | 27066/37962 [62:41:35<24:57:01,  8.24s/it]
 71%|███████▏  | 27067/37962 [62:41:43<24:59:31,  8.26s/it]
 71%|███████▏  | 27068/37962 [62:41:52<25:09:31,  8.31s/it]
 71%|███████▏  | 27069/37962 [62:42:00<25:00:04,  8.26s/it]
 71%|███████▏  | 27070/37962 [62:42:08<24:57:43,  8.25s/it]
                                                           
{'loss': 1.0862, 'grad_norm': 8.9276704788208, 'learning_rate': 4.0157485826087225e-06, 'epoch': 0.71}

 71%|███████▏  | 27070/37962 [62:42:08<24:57:43,  8.25s/it]
 71%|███████▏  | 27071/37962 [62:42:16<24:30:21,  8.10s/it]
 71%|███████▏  | 27072/37962 [62:42:23<24:08:09,  7.98s/it]
 71%|███████▏  | 27073/37962 [62:42:31<24:02:38,  7.95s/it]
 71%|███████▏  | 27074/37962 [62:42:40<24:23:33,  8.07s/it]
 71%|███████▏  | 27075/37962 [62:42:48<24:29:08,  8.10s/it]
 71%|███████▏  | 27076/37962 [62:42:56<24:35:07,  8.13s/it]
 71%|███████▏  | 27077/37962 [62:43:04<24:40:13,  8.16s/it]
 71%|███████▏  | 27078/37962 [62:43:13<24:52:05,  8.23s/it]
 71%|███████▏  | 27079/37962 [62:43:21<24:55:14,  8.24s/it]
 71%|███████▏  | 27080/37962 [62:43:29<24:49:27,  8.21s/it]
                                                           
{'loss': 1.2812, 'grad_norm': 7.40914249420166, 'learning_rate': 4.0089154214925266e-06, 'epoch': 0.71}

 71%|███████▏  | 27080/37962 [62:43:29<24:49:27,  8.21s/it]
 71%|███████▏  | 27081/37962 [62:43:37<24:37:41,  8.15s/it]
 71%|███████▏  | 27082/37962 [62:43:45<24:40:42,  8.17s/it]
 71%|███████▏  | 27083/37962 [62:43:54<24:52:24,  8.23s/it]
 71%|███████▏  | 27084/37962 [62:44:02<25:00:05,  8.27s/it]
 71%|███████▏  | 27085/37962 [62:44:10<24:58:34,  8.27s/it]
 71%|███████▏  | 27086/37962 [62:44:18<24:44:58,  8.19s/it]
 71%|███████▏  | 27087/37962 [62:44:27<25:01:11,  8.28s/it]
 71%|███████▏  | 27088/37962 [62:44:35<25:07:24,  8.32s/it]
 71%|███████▏  | 27089/37962 [62:44:44<25:12:05,  8.34s/it]
 71%|███████▏  | 27090/37962 [62:44:52<24:48:01,  8.21s/it]
                                                           
{'loss': 1.3035, 'grad_norm': 5.1152567863464355, 'learning_rate': 4.002086621183002e-06, 'epoch': 0.71}

 71%|███████▏  | 27090/37962 [62:44:52<24:48:01,  8.21s/it]
 71%|███████▏  | 27091/37962 [62:45:00<24:52:07,  8.24s/it]
 71%|███████▏  | 27092/37962 [62:45:08<24:58:01,  8.27s/it]
 71%|███████▏  | 27093/37962 [62:45:16<24:55:22,  8.25s/it]
 71%|███████▏  | 27094/37962 [62:45:25<24:57:20,  8.27s/it]
 71%|███████▏  | 27095/37962 [62:45:33<24:51:44,  8.24s/it]
 71%|███████▏  | 27096/37962 [62:45:41<24:50:55,  8.23s/it]
 71%|███████▏  | 27097/37962 [62:45:49<24:48:38,  8.22s/it]
 71%|███████▏  | 27098/37962 [62:45:57<24:48:04,  8.22s/it]
 71%|███████▏  | 27099/37962 [62:46:06<24:55:05,  8.26s/it]
 71%|███████▏  | 27100/37962 [62:46:14<24:56:51,  8.27s/it]
                                                           
{'loss': 1.3142, 'grad_norm': 8.336875915527344, 'learning_rate': 3.99526218665072e-06, 'epoch': 0.71}

 71%|███████▏  | 27100/37962 [62:46:14<24:56:51,  8.27s/it]
 71%|███████▏  | 27101/37962 [62:46:22<25:01:21,  8.29s/it]
 71%|███████▏  | 27102/37962 [62:46:30<24:40:48,  8.18s/it]
 71%|███████▏  | 27103/37962 [62:46:39<24:37:20,  8.16s/it]
 71%|███████▏  | 27104/37962 [62:46:47<24:34:39,  8.15s/it]
 71%|███████▏  | 27105/37962 [62:46:55<24:43:53,  8.20s/it]
 71%|███████▏  | 27106/37962 [62:47:03<24:46:06,  8.21s/it]
 71%|███████▏  | 27107/37962 [62:47:11<24:47:36,  8.22s/it]
 71%|███████▏  | 27108/37962 [62:47:20<24:49:14,  8.23s/it]
 71%|███████▏  | 27109/37962 [62:47:28<24:53:12,  8.26s/it]
 71%|███████▏  | 27110/37962 [62:47:36<24:58:17,  8.28s/it]
                                                           
{'loss': 1.2744, 'grad_norm': 12.616101264953613, 'learning_rate': 3.988442122863076e-06, 'epoch': 0.71}

 71%|███████▏  | 27110/37962 [62:47:36<24:58:17,  8.28s/it]
 71%|███████▏  | 27111/37962 [62:47:45<25:03:41,  8.31s/it]
 71%|███████▏  | 27112/37962 [62:47:53<24:48:04,  8.23s/it]
 71%|███████▏  | 27113/37962 [62:48:01<24:47:27,  8.23s/it]
 71%|███████▏  | 27114/37962 [62:48:09<24:39:58,  8.19s/it]
 71%|███████▏  | 27115/37962 [62:48:17<24:28:30,  8.12s/it]
 71%|███████▏  | 27116/37962 [62:48:25<24:31:23,  8.14s/it]
 71%|███████▏  | 27117/37962 [62:48:34<24:44:15,  8.21s/it]
 71%|███████▏  | 27118/37962 [62:48:42<24:31:51,  8.14s/it]
 71%|███████▏  | 27119/37962 [62:48:50<24:31:13,  8.14s/it]
 71%|███████▏  | 27120/37962 [62:48:58<24:35:46,  8.17s/it]
                                                           
{'loss': 1.1754, 'grad_norm': 5.930954456329346, 'learning_rate': 3.981626434784258e-06, 'epoch': 0.71}

 71%|███████▏  | 27120/37962 [62:48:58<24:35:46,  8.17s/it]
 71%|███████▏  | 27121/37962 [62:49:06<24:47:03,  8.23s/it]
 71%|███████▏  | 27122/37962 [62:49:15<24:55:11,  8.28s/it]
 71%|███████▏  | 27123/37962 [62:49:23<24:58:15,  8.29s/it]
 71%|███████▏  | 27124/37962 [62:49:31<24:58:39,  8.30s/it]
 71%|███████▏  | 27125/37962 [62:49:40<24:58:49,  8.30s/it]
 71%|███████▏  | 27126/37962 [62:49:48<25:00:46,  8.31s/it]
 71%|███████▏  | 27127/37962 [62:49:56<25:02:51,  8.32s/it]
 71%|███████▏  | 27128/37962 [62:50:05<24:59:11,  8.30s/it]
 71%|███████▏  | 27129/37962 [62:50:13<25:11:09,  8.37s/it]
 71%|███████▏  | 27130/37962 [62:50:21<25:00:05,  8.31s/it]
                                                           
{'loss': 1.2693, 'grad_norm': 7.624250888824463, 'learning_rate': 3.974815127375297e-06, 'epoch': 0.71}

 71%|███████▏  | 27130/37962 [62:50:21<25:00:05,  8.31s/it]
 71%|███████▏  | 27131/37962 [62:50:30<24:56:12,  8.29s/it]
 71%|███████▏  | 27132/37962 [62:50:38<25:01:02,  8.32s/it]
 71%|███████▏  | 27133/37962 [62:50:46<24:50:50,  8.26s/it]
 71%|███████▏  | 27134/37962 [62:50:54<24:52:08,  8.27s/it]
 71%|███████▏  | 27135/37962 [62:51:03<24:54:38,  8.28s/it]
 71%|███████▏  | 27136/37962 [62:51:10<24:29:32,  8.14s/it]
 71%|███████▏  | 27137/37962 [62:51:19<24:41:06,  8.21s/it]
 71%|███████▏  | 27138/37962 [62:51:27<24:46:39,  8.24s/it]
 71%|███████▏  | 27139/37962 [62:51:35<24:47:20,  8.25s/it]
 71%|███████▏  | 27140/37962 [62:51:44<24:42:48,  8.22s/it]
                                                           
{'loss': 1.3082, 'grad_norm': 5.490057468414307, 'learning_rate': 3.968008205594032e-06, 'epoch': 0.71}

 71%|███████▏  | 27140/37962 [62:51:44<24:42:48,  8.22s/it]
 71%|███████▏  | 27141/37962 [62:51:52<24:42:04,  8.22s/it]
 71%|███████▏  | 27142/37962 [62:52:00<24:44:59,  8.23s/it]
 72%|███████▏  | 27143/37962 [62:52:08<24:49:38,  8.26s/it]
 72%|███████▏  | 27144/37962 [62:52:17<24:53:00,  8.28s/it]
 72%|███████▏  | 27145/37962 [62:52:24<24:20:39,  8.10s/it]
 72%|███████▏  | 27146/37962 [62:52:33<24:40:13,  8.21s/it]
 72%|███████▏  | 27147/37962 [62:52:41<24:51:17,  8.27s/it]
 72%|███████▏  | 27148/37962 [62:52:50<24:55:10,  8.30s/it]
 72%|███████▏  | 27149/37962 [62:52:57<24:31:37,  8.17s/it]
 72%|███████▏  | 27150/37962 [62:53:06<24:38:12,  8.20s/it]
                                                           
{'loss': 1.2517, 'grad_norm': 19.924509048461914, 'learning_rate': 3.961205674395091e-06, 'epoch': 0.72}

 72%|███████▏  | 27150/37962 [62:53:06<24:38:12,  8.20s/it]
 72%|███████▏  | 27151/37962 [62:53:14<24:54:43,  8.30s/it]
 72%|███████▏  | 27152/37962 [62:53:22<24:28:59,  8.15s/it]
 72%|███████▏  | 27153/37962 [62:53:30<24:33:18,  8.18s/it]
 72%|███████▏  | 27154/37962 [62:53:39<24:44:27,  8.24s/it]
 72%|███████▏  | 27155/37962 [62:53:47<24:59:43,  8.33s/it]
 72%|███████▏  | 27156/37962 [62:53:56<25:02:48,  8.34s/it]
 72%|███████▏  | 27157/37962 [62:54:04<25:02:41,  8.34s/it]
 72%|███████▏  | 27158/37962 [62:54:12<25:01:11,  8.34s/it]
 72%|███████▏  | 27159/37962 [62:54:20<24:49:27,  8.27s/it]
 72%|███████▏  | 27160/37962 [62:54:29<24:46:28,  8.26s/it]
                                                           
{'loss': 1.2262, 'grad_norm': 9.019606590270996, 'learning_rate': 3.95440753872993e-06, 'epoch': 0.72}

 72%|███████▏  | 27160/37962 [62:54:29<24:46:28,  8.26s/it]
 72%|███████▏  | 27161/37962 [62:54:37<24:34:06,  8.19s/it]
 72%|███████▏  | 27162/37962 [62:54:45<24:36:39,  8.20s/it]
 72%|███████▏  | 27163/37962 [62:54:53<24:26:07,  8.15s/it]
 72%|███████▏  | 27164/37962 [62:55:01<24:30:55,  8.17s/it]
 72%|███████▏  | 27165/37962 [62:55:10<24:40:42,  8.23s/it]
 72%|███████▏  | 27166/37962 [62:55:18<24:29:39,  8.17s/it]
 72%|███████▏  | 27167/37962 [62:55:26<24:35:02,  8.20s/it]
 72%|███████▏  | 27168/37962 [62:55:34<24:43:25,  8.25s/it]
 72%|███████▏  | 27169/37962 [62:55:43<25:03:22,  8.36s/it]
 72%|███████▏  | 27170/37962 [62:55:51<25:07:30,  8.38s/it]
                                                           
{'loss': 1.1618, 'grad_norm': 8.202618598937988, 'learning_rate': 3.947613803546787e-06, 'epoch': 0.72}

 72%|███████▏  | 27170/37962 [62:55:51<25:07:30,  8.38s/it]
 72%|███████▏  | 27171/37962 [62:56:00<25:12:09,  8.41s/it]
 72%|███████▏  | 27172/37962 [62:56:08<25:11:22,  8.40s/it]
 72%|███████▏  | 27173/37962 [62:56:16<25:00:36,  8.35s/it]
 72%|███████▏  | 27174/37962 [62:56:25<24:57:20,  8.33s/it]
 72%|███████▏  | 27175/37962 [62:56:33<24:49:36,  8.29s/it]
 72%|███████▏  | 27176/37962 [62:56:41<24:50:59,  8.29s/it]
 72%|███████▏  | 27177/37962 [62:56:49<24:41:38,  8.24s/it]
 72%|███████▏  | 27178/37962 [62:56:57<24:39:54,  8.23s/it]
 72%|███████▏  | 27179/37962 [62:57:06<24:50:52,  8.30s/it]
 72%|███████▏  | 27180/37962 [62:57:14<24:35:03,  8.21s/it]
                                                           
{'loss': 1.262, 'grad_norm': 12.091175079345703, 'learning_rate': 3.940824473790703e-06, 'epoch': 0.72}

 72%|███████▏  | 27180/37962 [62:57:14<24:35:03,  8.21s/it]
 72%|███████▏  | 27181/37962 [62:57:22<24:52:31,  8.31s/it]
 72%|███████▏  | 27182/37962 [62:57:30<24:41:00,  8.24s/it]
 72%|███████▏  | 27183/37962 [62:57:38<24:23:04,  8.14s/it]
 72%|███████▏  | 27184/37962 [62:57:47<24:24:52,  8.15s/it]
 72%|███████▏  | 27185/37962 [62:57:55<24:32:09,  8.20s/it]
 72%|███████▏  | 27186/37962 [62:58:03<24:33:03,  8.20s/it]
 72%|███████▏  | 27187/37962 [62:58:11<24:33:26,  8.20s/it]
 72%|███████▏  | 27188/37962 [62:58:20<24:51:16,  8.30s/it]
 72%|███████▏  | 27189/37962 [62:58:28<24:56:13,  8.33s/it]
 72%|███████▏  | 27190/37962 [62:58:37<25:02:11,  8.37s/it]
                                                           
{'loss': 1.2811, 'grad_norm': 10.056657791137695, 'learning_rate': 3.934039554403521e-06, 'epoch': 0.72}

 72%|███████▏  | 27190/37962 [62:58:37<25:02:11,  8.37s/it]
 72%|███████▏  | 27191/37962 [62:58:53<32:01:07, 10.70s/it]
 72%|███████▏  | 27192/37962 [62:59:01<29:48:58,  9.97s/it]
 72%|███████▏  | 27193/37962 [62:59:10<28:29:42,  9.53s/it]
 72%|███████▏  | 27194/37962 [62:59:18<27:14:36,  9.11s/it]
 72%|███████▏  | 27195/37962 [62:59:26<26:41:05,  8.92s/it]
 72%|███████▏  | 27196/37962 [62:59:35<26:11:56,  8.76s/it]
 72%|███████▏  | 27197/37962 [62:59:43<25:42:42,  8.60s/it]
 72%|███████▏  | 27198/37962 [62:59:51<25:19:55,  8.47s/it]
 72%|███████▏  | 27199/37962 [62:59:59<24:51:00,  8.31s/it]
 72%|███████▏  | 27200/37962 [63:00:07<24:48:34,  8.30s/it]
                                                           
{'loss': 1.143, 'grad_norm': 6.162553310394287, 'learning_rate': 3.927259050323857e-06, 'epoch': 0.72}

 72%|███████▏  | 27200/37962 [63:00:07<24:48:34,  8.30s/it]
 72%|███████▏  | 27201/37962 [63:00:15<24:40:27,  8.25s/it]
 72%|███████▏  | 27202/37962 [63:00:24<24:38:46,  8.25s/it]
 72%|███████▏  | 27203/37962 [63:00:32<24:41:40,  8.26s/it]
 72%|███████▏  | 27204/37962 [63:00:40<24:44:23,  8.28s/it]
 72%|███████▏  | 27205/37962 [63:00:49<24:48:38,  8.30s/it]
 72%|███████▏  | 27206/37962 [63:00:57<24:43:48,  8.28s/it]
 72%|███████▏  | 27207/37962 [63:01:05<24:40:56,  8.26s/it]
 72%|███████▏  | 27208/37962 [63:01:13<24:38:02,  8.25s/it]
 72%|███████▏  | 27209/37962 [63:01:22<24:44:06,  8.28s/it]
 72%|███████▏  | 27210/37962 [63:01:30<24:43:43,  8.28s/it]
                                                           
{'loss': 1.2152, 'grad_norm': 8.208976745605469, 'learning_rate': 3.9204829664871314e-06, 'epoch': 0.72}

 72%|███████▏  | 27210/37962 [63:01:30<24:43:43,  8.28s/it]
 72%|███████▏  | 27211/37962 [63:01:38<24:45:52,  8.29s/it]
 72%|███████▏  | 27212/37962 [63:01:46<24:35:35,  8.24s/it]
 72%|███████▏  | 27213/37962 [63:01:55<25:00:28,  8.38s/it]
 72%|███████▏  | 27214/37962 [63:02:03<25:05:34,  8.40s/it]
 72%|███████▏  | 27215/37962 [63:02:12<24:54:14,  8.34s/it]
 72%|███████▏  | 27216/37962 [63:02:20<24:45:17,  8.29s/it]
 72%|███████▏  | 27217/37962 [63:02:28<24:49:20,  8.32s/it]
 72%|███████▏  | 27218/37962 [63:02:37<24:51:04,  8.33s/it]
 72%|███████▏  | 27219/37962 [63:02:45<24:40:54,  8.27s/it]
 72%|███████▏  | 27220/37962 [63:02:53<24:37:37,  8.25s/it]
                                                           
{'loss': 1.2598, 'grad_norm': 6.672464847564697, 'learning_rate': 3.913711307825531e-06, 'epoch': 0.72}

 72%|███████▏  | 27220/37962 [63:02:53<24:37:37,  8.25s/it]
 72%|███████▏  | 27221/37962 [63:03:01<24:40:49,  8.27s/it]
 72%|███████▏  | 27222/37962 [63:03:09<24:40:42,  8.27s/it]
 72%|███████▏  | 27223/37962 [63:03:18<24:40:29,  8.27s/it]
 72%|███████▏  | 27224/37962 [63:03:26<24:47:55,  8.31s/it]
 72%|███████▏  | 27225/37962 [63:03:35<24:51:34,  8.34s/it]
 72%|███████▏  | 27226/37962 [63:03:43<25:00:04,  8.38s/it]
 72%|███████▏  | 27227/37962 [63:03:51<24:50:25,  8.33s/it]
 72%|███████▏  | 27228/37962 [63:03:59<24:38:29,  8.26s/it]
 72%|███████▏  | 27229/37962 [63:04:08<24:37:38,  8.26s/it]
 72%|███████▏  | 27230/37962 [63:04:16<24:50:12,  8.33s/it]
                                                           
{'loss': 1.3071, 'grad_norm': 5.915503978729248, 'learning_rate': 3.9069440792680336e-06, 'epoch': 0.72}

 72%|███████▏  | 27230/37962 [63:04:16<24:50:12,  8.33s/it]
 72%|███████▏  | 27231/37962 [63:04:24<24:51:54,  8.34s/it]
 72%|███████▏  | 27232/37962 [63:04:32<24:31:50,  8.23s/it]
 72%|███████▏  | 27233/37962 [63:04:41<24:30:18,  8.22s/it]
 72%|███████▏  | 27234/37962 [63:04:49<24:41:37,  8.29s/it]
 72%|███████▏  | 27235/37962 [63:04:57<24:43:22,  8.30s/it]
 72%|███████▏  | 27236/37962 [63:05:06<24:52:23,  8.35s/it]
 72%|███████▏  | 27237/37962 [63:05:14<24:52:18,  8.35s/it]
 72%|███████▏  | 27238/37962 [63:05:23<24:51:36,  8.35s/it]
 72%|███████▏  | 27239/37962 [63:05:38<31:26:15, 10.55s/it]
 72%|███████▏  | 27240/37962 [63:05:46<29:14:43,  9.82s/it]
                                                           
{'loss': 1.1622, 'grad_norm': 16.773019790649414, 'learning_rate': 3.900181285740384e-06, 'epoch': 0.72}

 72%|███████▏  | 27240/37962 [63:05:46<29:14:43,  9.82s/it]
 72%|███████▏  | 27241/37962 [63:05:55<27:53:29,  9.37s/it]
 72%|███████▏  | 27242/37962 [63:06:03<26:53:37,  9.03s/it]
 72%|███████▏  | 27243/37962 [63:06:11<26:08:58,  8.78s/it]
 72%|███████▏  | 27244/37962 [63:06:19<25:40:44,  8.63s/it]
 72%|███████▏  | 27245/37962 [63:06:28<25:24:23,  8.53s/it]
 72%|███████▏  | 27246/37962 [63:06:36<25:18:13,  8.50s/it]
 72%|███████▏  | 27247/37962 [63:06:45<25:19:20,  8.51s/it]
 72%|███████▏  | 27248/37962 [63:06:53<25:11:54,  8.47s/it]
 72%|███████▏  | 27249/37962 [63:07:01<25:00:46,  8.41s/it]
 72%|███████▏  | 27250/37962 [63:07:10<25:01:27,  8.41s/it]
                                                           
{'loss': 1.26, 'grad_norm': 7.119536399841309, 'learning_rate': 3.8934229321651075e-06, 'epoch': 0.72}

 72%|███████▏  | 27250/37962 [63:07:10<25:01:27,  8.41s/it]
 72%|███████▏  | 27251/37962 [63:07:18<24:47:45,  8.33s/it]
 72%|███████▏  | 27252/37962 [63:07:26<24:43:41,  8.31s/it]
 72%|███████▏  | 27253/37962 [63:07:34<24:13:55,  8.15s/it]
 72%|███████▏  | 27254/37962 [63:07:42<24:28:01,  8.23s/it]
 72%|███████▏  | 27255/37962 [63:07:50<24:22:27,  8.20s/it]
 72%|███████▏  | 27256/37962 [63:07:59<24:26:08,  8.22s/it]
 72%|███████▏  | 27257/37962 [63:08:07<24:32:57,  8.26s/it]
 72%|███████▏  | 27258/37962 [63:08:15<24:34:39,  8.27s/it]
 72%|███████▏  | 27259/37962 [63:08:24<24:38:09,  8.29s/it]
 72%|███████▏  | 27260/37962 [63:08:32<24:40:00,  8.30s/it]
                                                           
{'loss': 1.3047, 'grad_norm': 4.531097412109375, 'learning_rate': 3.886669023461486e-06, 'epoch': 0.72}

 72%|███████▏  | 27260/37962 [63:08:32<24:40:00,  8.30s/it]
 72%|███████▏  | 27261/37962 [63:08:40<24:31:44,  8.25s/it]
 72%|███████▏  | 27262/37962 [63:08:49<24:38:22,  8.29s/it]
 72%|███████▏  | 27263/37962 [63:08:57<24:44:43,  8.33s/it]
 72%|███████▏  | 27264/37962 [63:09:05<24:36:10,  8.28s/it]
 72%|███████▏  | 27265/37962 [63:09:13<24:38:25,  8.29s/it]
 72%|███████▏  | 27266/37962 [63:09:22<24:37:28,  8.29s/it]
 72%|███████▏  | 27267/37962 [63:09:30<24:45:59,  8.34s/it]
 72%|███████▏  | 27268/37962 [63:09:38<24:37:35,  8.29s/it]
 72%|███████▏  | 27269/37962 [63:09:47<24:34:19,  8.27s/it]
 72%|███████▏  | 27270/37962 [63:09:55<24:16:38,  8.17s/it]
                                                           
{'loss': 1.1749, 'grad_norm': 8.247146606445312, 'learning_rate': 3.87991956454558e-06, 'epoch': 0.72}

 72%|███████▏  | 27270/37962 [63:09:55<24:16:38,  8.17s/it]
 72%|███████▏  | 27271/37962 [63:10:03<24:07:50,  8.13s/it]
 72%|███████▏  | 27272/37962 [63:10:11<24:26:14,  8.23s/it]
 72%|███████▏  | 27273/37962 [63:10:19<24:27:13,  8.24s/it]
 72%|███████▏  | 27274/37962 [63:10:27<24:17:46,  8.18s/it]
 72%|███████▏  | 27275/37962 [63:10:36<24:22:26,  8.21s/it]
 72%|███████▏  | 27276/37962 [63:10:44<24:11:42,  8.15s/it]
 72%|███████▏  | 27277/37962 [63:10:52<24:02:09,  8.10s/it]
 72%|███████▏  | 27278/37962 [63:11:00<24:06:15,  8.12s/it]
 72%|███████▏  | 27279/37962 [63:11:08<24:08:02,  8.13s/it]
 72%|███████▏  | 27280/37962 [63:11:16<24:03:30,  8.11s/it]
                                                           
{'loss': 1.1969, 'grad_norm': 7.823760509490967, 'learning_rate': 3.873174560330199e-06, 'epoch': 0.72}

 72%|███████▏  | 27280/37962 [63:11:16<24:03:30,  8.11s/it]
 72%|███████▏  | 27281/37962 [63:11:24<24:11:45,  8.16s/it]
 72%|███████▏  | 27282/37962 [63:11:33<24:22:51,  8.22s/it]
 72%|███████▏  | 27283/37962 [63:11:41<24:19:39,  8.20s/it]
 72%|███████▏  | 27284/37962 [63:11:49<24:20:36,  8.21s/it]
 72%|███████▏  | 27285/37962 [63:11:57<24:23:54,  8.23s/it]
 72%|███████▏  | 27286/37962 [63:12:06<24:29:51,  8.26s/it]
 72%|███████▏  | 27287/37962 [63:12:14<24:26:07,  8.24s/it]
 72%|███████▏  | 27288/37962 [63:12:22<24:41:40,  8.33s/it]
 72%|███████▏  | 27289/37962 [63:12:30<24:15:45,  8.18s/it]
 72%|███████▏  | 27290/37962 [63:12:38<24:20:23,  8.21s/it]
                                                           
{'loss': 1.2317, 'grad_norm': 8.736512184143066, 'learning_rate': 3.866434015724912e-06, 'epoch': 0.72}

 72%|███████▏  | 27290/37962 [63:12:38<24:20:23,  8.21s/it]
 72%|███████▏  | 27291/37962 [63:12:47<24:22:04,  8.22s/it]
 72%|███████▏  | 27292/37962 [63:12:55<24:26:36,  8.25s/it]
 72%|███████▏  | 27293/37962 [63:13:03<24:34:50,  8.29s/it]
 72%|███████▏  | 27294/37962 [63:13:12<24:30:47,  8.27s/it]
 72%|███████▏  | 27295/37962 [63:13:20<24:24:01,  8.23s/it]
 72%|███████▏  | 27296/37962 [63:13:28<24:46:46,  8.36s/it]
 72%|███████▏  | 27297/37962 [63:13:37<24:42:28,  8.34s/it]
 72%|███████▏  | 27298/37962 [63:13:45<24:35:12,  8.30s/it]
 72%|███████▏  | 27299/37962 [63:13:53<24:31:33,  8.28s/it]
 72%|███████▏  | 27300/37962 [63:14:02<24:57:31,  8.43s/it]
                                                           
{'loss': 1.1847, 'grad_norm': 32.64698791503906, 'learning_rate': 3.85969793563605e-06, 'epoch': 0.72}

 72%|███████▏  | 27300/37962 [63:14:02<24:57:31,  8.43s/it]
 72%|███████▏  | 27301/37962 [63:14:10<24:51:14,  8.39s/it]
 72%|███████▏  | 27302/37962 [63:14:18<24:37:05,  8.31s/it]
 72%|███████▏  | 27303/37962 [63:14:26<24:26:30,  8.26s/it]
 72%|███████▏  | 27304/37962 [63:14:35<24:33:54,  8.30s/it]
 72%|███████▏  | 27305/37962 [63:14:43<24:38:56,  8.33s/it]
 72%|███████▏  | 27306/37962 [63:14:51<24:08:45,  8.16s/it]
 72%|███████▏  | 27307/37962 [63:14:59<24:13:50,  8.19s/it]
 72%|███████▏  | 27308/37962 [63:15:08<24:26:14,  8.26s/it]
 72%|███████▏  | 27309/37962 [63:15:16<24:17:47,  8.21s/it]
 72%|███████▏  | 27310/37962 [63:15:24<24:22:17,  8.24s/it]
                                                           
{'loss': 1.2246, 'grad_norm': 9.443113327026367, 'learning_rate': 3.8529663249666895e-06, 'epoch': 0.72}

 72%|███████▏  | 27310/37962 [63:15:24<24:22:17,  8.24s/it]
 72%|███████▏  | 27311/37962 [63:15:32<24:22:39,  8.24s/it]
 72%|███████▏  | 27312/37962 [63:15:41<24:18:27,  8.22s/it]
 72%|███████▏  | 27313/37962 [63:15:49<24:08:50,  8.16s/it]
 72%|███████▏  | 27314/37962 [63:15:57<24:00:04,  8.11s/it]
 72%|███████▏  | 27315/37962 [63:16:05<24:10:04,  8.17s/it]
 72%|███████▏  | 27316/37962 [63:16:13<24:17:58,  8.22s/it]
 72%|███████▏  | 27317/37962 [63:16:21<24:13:35,  8.19s/it]
 72%|███████▏  | 27318/37962 [63:16:30<24:14:16,  8.20s/it]
 72%|███████▏  | 27319/37962 [63:16:37<23:55:07,  8.09s/it]
 72%|███████▏  | 27320/37962 [63:16:46<24:11:02,  8.18s/it]
                                                           
{'loss': 1.2331, 'grad_norm': 16.1470890045166, 'learning_rate': 3.846239188616651e-06, 'epoch': 0.72}

 72%|███████▏  | 27320/37962 [63:16:46<24:11:02,  8.18s/it]
 72%|███████▏  | 27321/37962 [63:16:54<24:11:08,  8.18s/it]
 72%|███████▏  | 27322/37962 [63:17:02<24:29:20,  8.29s/it]
 72%|███████▏  | 27323/37962 [63:17:11<24:24:33,  8.26s/it]
 72%|███████▏  | 27324/37962 [63:17:19<24:18:09,  8.22s/it]
 72%|███████▏  | 27325/37962 [63:17:27<24:15:55,  8.21s/it]
 72%|███████▏  | 27326/37962 [63:17:35<24:24:36,  8.26s/it]
 72%|███████▏  | 27327/37962 [63:17:44<24:31:51,  8.30s/it]
 72%|███████▏  | 27328/37962 [63:17:52<24:27:26,  8.28s/it]
 72%|███████▏  | 27329/37962 [63:18:00<24:26:13,  8.27s/it]
 72%|███████▏  | 27330/37962 [63:18:09<24:28:27,  8.29s/it]
                                                           
{'loss': 1.1908, 'grad_norm': 5.221819877624512, 'learning_rate': 3.839516531482496e-06, 'epoch': 0.72}

 72%|███████▏  | 27330/37962 [63:18:09<24:28:27,  8.29s/it]
 72%|███████▏  | 27331/37962 [63:18:17<24:12:52,  8.20s/it]
 72%|███████▏  | 27332/37962 [63:18:25<24:09:04,  8.18s/it]
 72%|███████▏  | 27333/37962 [63:18:33<24:24:03,  8.26s/it]
 72%|███████▏  | 27334/37962 [63:18:42<24:44:47,  8.38s/it]
 72%|███████▏  | 27335/37962 [63:18:50<24:35:35,  8.33s/it]
 72%|███████▏  | 27336/37962 [63:18:58<24:27:59,  8.29s/it]
 72%|███████▏  | 27337/37962 [63:19:06<24:25:03,  8.27s/it]
 72%|███████▏  | 27338/37962 [63:19:15<24:16:09,  8.22s/it]
 72%|███████▏  | 27339/37962 [63:19:22<23:58:25,  8.12s/it]
 72%|███████▏  | 27340/37962 [63:19:31<24:06:46,  8.17s/it]
                                                           
{'loss': 1.2274, 'grad_norm': 8.22486686706543, 'learning_rate': 3.832798358457539e-06, 'epoch': 0.72}

 72%|███████▏  | 27340/37962 [63:19:31<24:06:46,  8.17s/it]
 72%|███████▏  | 27341/37962 [63:19:39<24:03:28,  8.15s/it]
 72%|███████▏  | 27342/37962 [63:19:47<24:09:11,  8.19s/it]
 72%|███████▏  | 27343/37962 [63:19:55<24:04:23,  8.16s/it]
 72%|███████▏  | 27344/37962 [63:20:03<24:01:10,  8.14s/it]
 72%|███████▏  | 27345/37962 [63:20:12<24:21:27,  8.26s/it]
 72%|███████▏  | 27346/37962 [63:20:20<24:20:54,  8.26s/it]
 72%|███████▏  | 27347/37962 [63:20:28<24:16:43,  8.23s/it]
 72%|███████▏  | 27348/37962 [63:20:36<24:15:20,  8.23s/it]
 72%|███████▏  | 27349/37962 [63:20:45<24:17:27,  8.24s/it]
 72%|███████▏  | 27350/37962 [63:20:53<24:09:38,  8.20s/it]
                                                           
{'loss': 1.3021, 'grad_norm': 9.44852352142334, 'learning_rate': 3.826084674431815e-06, 'epoch': 0.72}

 72%|███████▏  | 27350/37962 [63:20:53<24:09:38,  8.20s/it]
 72%|███████▏  | 27351/37962 [63:21:01<24:01:35,  8.15s/it]
 72%|███████▏  | 27352/37962 [63:21:09<24:06:28,  8.18s/it]
 72%|███████▏  | 27353/37962 [63:21:17<24:11:27,  8.21s/it]
 72%|███████▏  | 27354/37962 [63:21:26<24:14:43,  8.23s/it]
 72%|███████▏  | 27355/37962 [63:21:34<24:26:32,  8.30s/it]
 72%|███████▏  | 27356/37962 [63:21:42<24:24:08,  8.28s/it]
 72%|███████▏  | 27357/37962 [63:21:51<24:27:25,  8.30s/it]
 72%|███████▏  | 27358/37962 [63:21:59<24:16:49,  8.24s/it]
 72%|███████▏  | 27359/37962 [63:22:07<24:23:19,  8.28s/it]
 72%|███████▏  | 27360/37962 [63:22:15<23:57:30,  8.14s/it]
                                                           
{'loss': 1.243, 'grad_norm': 10.664459228515625, 'learning_rate': 3.8193754842921025e-06, 'epoch': 0.72}

 72%|███████▏  | 27360/37962 [63:22:15<23:57:30,  8.14s/it]
 72%|███████▏  | 27361/37962 [63:22:23<23:57:30,  8.14s/it]
 72%|███████▏  | 27362/37962 [63:22:31<23:45:51,  8.07s/it]
 72%|███████▏  | 27363/37962 [63:22:39<24:02:55,  8.17s/it]
 72%|███████▏  | 27364/37962 [63:22:48<24:11:19,  8.22s/it]
 72%|███████▏  | 27365/37962 [63:22:56<24:00:08,  8.15s/it]
 72%|███████▏  | 27366/37962 [63:23:04<24:14:19,  8.24s/it]
 72%|███████▏  | 27367/37962 [63:23:12<24:03:23,  8.17s/it]
 72%|███████▏  | 27368/37962 [63:23:21<24:07:30,  8.20s/it]
 72%|███████▏  | 27369/37962 [63:23:29<23:56:34,  8.14s/it]
 72%|███████▏  | 27370/37962 [63:23:37<24:14:41,  8.24s/it]
                                                           
{'loss': 1.1209, 'grad_norm': 8.17872428894043, 'learning_rate': 3.812670792921904e-06, 'epoch': 0.72}

 72%|███████▏  | 27370/37962 [63:23:37<24:14:41,  8.24s/it]
 72%|███████▏  | 27371/37962 [63:23:45<24:16:26,  8.25s/it]
 72%|███████▏  | 27372/37962 [63:23:53<23:54:00,  8.12s/it]
 72%|███████▏  | 27373/37962 [63:24:01<24:06:35,  8.20s/it]
 72%|███████▏  | 27374/37962 [63:24:10<24:13:12,  8.24s/it]
 72%|███████▏  | 27375/37962 [63:24:18<24:15:22,  8.25s/it]
 72%|███████▏  | 27376/37962 [63:24:26<24:14:08,  8.24s/it]
 72%|███████▏  | 27377/37962 [63:24:35<24:24:00,  8.30s/it]
 72%|███████▏  | 27378/37962 [63:24:43<24:20:06,  8.28s/it]
 72%|███████▏  | 27379/37962 [63:24:51<24:29:18,  8.33s/it]
 72%|███████▏  | 27380/37962 [63:25:00<24:17:55,  8.27s/it]
                                                           
{'loss': 1.3766, 'grad_norm': 12.051916122436523, 'learning_rate': 3.8059706052014436e-06, 'epoch': 0.72}

 72%|███████▏  | 27380/37962 [63:25:00<24:17:55,  8.27s/it]
 72%|███████▏  | 27381/37962 [63:25:08<24:33:22,  8.35s/it]
 72%|███████▏  | 27382/37962 [63:25:16<24:27:32,  8.32s/it]
 72%|███████▏  | 27383/37962 [63:25:25<24:22:07,  8.29s/it]
 72%|███████▏  | 27384/37962 [63:25:33<24:16:54,  8.26s/it]
 72%|███████▏  | 27385/37962 [63:25:41<24:00:36,  8.17s/it]
 72%|███████▏  | 27386/37962 [63:25:49<24:10:34,  8.23s/it]
 72%|███████▏  | 27387/37962 [63:25:57<24:11:21,  8.23s/it]
 72%|███████▏  | 27388/37962 [63:26:06<24:17:46,  8.27s/it]
 72%|███████▏  | 27389/37962 [63:26:14<24:23:02,  8.30s/it]
 72%|███████▏  | 27390/37962 [63:26:22<24:28:46,  8.34s/it]
                                                           
{'loss': 1.1954, 'grad_norm': 11.754029273986816, 'learning_rate': 3.799274926007679e-06, 'epoch': 0.72}

 72%|███████▏  | 27390/37962 [63:26:22<24:28:46,  8.34s/it]
 72%|███████▏  | 27391/37962 [63:26:31<24:26:49,  8.33s/it]
 72%|███████▏  | 27392/37962 [63:26:39<24:11:22,  8.24s/it]
 72%|███████▏  | 27393/37962 [63:26:47<24:09:59,  8.23s/it]
 72%|███████▏  | 27394/37962 [63:26:55<24:03:54,  8.20s/it]
 72%|███████▏  | 27395/37962 [63:27:03<23:59:43,  8.17s/it]
 72%|███████▏  | 27396/37962 [63:27:12<24:04:23,  8.20s/it]
 72%|███████▏  | 27397/37962 [63:27:20<24:10:54,  8.24s/it]
 72%|███████▏  | 27398/37962 [63:27:28<24:14:39,  8.26s/it]
 72%|███████▏  | 27399/37962 [63:27:36<23:50:41,  8.13s/it]
 72%|███████▏  | 27400/37962 [63:27:44<23:53:52,  8.15s/it]
                                                           
{'loss': 1.1508, 'grad_norm': 7.017317771911621, 'learning_rate': 3.7925837602142712e-06, 'epoch': 0.72}

 72%|███████▏  | 27400/37962 [63:27:44<23:53:52,  8.15s/it]
 72%|███████▏  | 27401/37962 [63:27:52<23:59:22,  8.18s/it]
 72%|███████▏  | 27402/37962 [63:28:01<24:02:51,  8.20s/it]
 72%|███████▏  | 27403/37962 [63:28:09<24:12:41,  8.25s/it]
 72%|███████▏  | 27404/37962 [63:28:17<23:49:15,  8.12s/it]
 72%|███████▏  | 27405/37962 [63:28:25<23:54:25,  8.15s/it]
 72%|███████▏  | 27406/37962 [63:28:34<24:13:00,  8.26s/it]
 72%|███████▏  | 27407/37962 [63:28:42<24:20:34,  8.30s/it]
 72%|███████▏  | 27408/37962 [63:28:50<24:16:20,  8.28s/it]
 72%|███████▏  | 27409/37962 [63:28:58<24:13:36,  8.26s/it]
 72%|███████▏  | 27410/37962 [63:29:06<23:55:02,  8.16s/it]
                                                           
{'loss': 1.2186, 'grad_norm': 16.88252830505371, 'learning_rate': 3.785897112691609e-06, 'epoch': 0.72}

 72%|███████▏  | 27410/37962 [63:29:06<23:55:02,  8.16s/it]
 72%|███████▏  | 27411/37962 [63:29:15<23:55:19,  8.16s/it]
 72%|███████▏  | 27412/37962 [63:29:23<24:02:16,  8.20s/it]
 72%|███████▏  | 27413/37962 [63:29:31<23:50:48,  8.14s/it]
 72%|███████▏  | 27414/37962 [63:29:39<23:54:52,  8.16s/it]
 72%|███████▏  | 27415/37962 [63:29:47<23:52:58,  8.15s/it]
 72%|███████▏  | 27416/37962 [63:29:55<23:58:04,  8.18s/it]
 72%|███████▏  | 27417/37962 [63:30:04<24:09:25,  8.25s/it]
 72%|███████▏  | 27418/37962 [63:30:12<23:58:48,  8.19s/it]
 72%|███████▏  | 27419/37962 [63:30:20<24:03:50,  8.22s/it]
 72%|███████▏  | 27420/37962 [63:30:29<24:11:34,  8.26s/it]
                                                           
{'loss': 1.3978, 'grad_norm': 4.564225673675537, 'learning_rate': 3.779214988306794e-06, 'epoch': 0.72}

 72%|███████▏  | 27420/37962 [63:30:29<24:11:34,  8.26s/it]
 72%|███████▏  | 27421/37962 [63:30:36<23:53:37,  8.16s/it]
 72%|███████▏  | 27422/37962 [63:30:45<23:54:01,  8.16s/it]
 72%|███████▏  | 27423/37962 [63:30:53<23:59:19,  8.19s/it]
 72%|███████▏  | 27424/37962 [63:31:01<23:34:08,  8.05s/it]
 72%|███████▏  | 27425/37962 [63:31:09<23:47:24,  8.13s/it]
 72%|███████▏  | 27426/37962 [63:31:17<23:55:52,  8.18s/it]
 72%|███████▏  | 27427/37962 [63:31:25<23:37:47,  8.07s/it]
 72%|███████▏  | 27428/37962 [63:31:34<24:00:53,  8.21s/it]
 72%|███████▏  | 27429/37962 [63:31:42<24:05:50,  8.24s/it]
 72%|███████▏  | 27430/37962 [63:31:50<24:04:15,  8.23s/it]
                                                           
{'loss': 1.3138, 'grad_norm': 7.190403461456299, 'learning_rate': 3.772537391923613e-06, 'epoch': 0.72}

 72%|███████▏  | 27430/37962 [63:31:50<24:04:15,  8.23s/it]
 72%|███████▏  | 27431/37962 [63:31:58<24:05:54,  8.24s/it]
 72%|███████▏  | 27432/37962 [63:32:07<24:13:16,  8.28s/it]
 72%|███████▏  | 27433/37962 [63:32:15<24:08:31,  8.25s/it]
 72%|███████▏  | 27434/37962 [63:32:23<24:04:05,  8.23s/it]
 72%|███████▏  | 27435/37962 [63:32:31<23:49:44,  8.15s/it]
 72%|███████▏  | 27436/37962 [63:32:39<23:47:36,  8.14s/it]
 72%|███████▏  | 27437/37962 [63:32:48<24:00:28,  8.21s/it]
 72%|███████▏  | 27438/37962 [63:32:56<24:05:57,  8.24s/it]
 72%|███████▏  | 27439/37962 [63:33:04<24:07:16,  8.25s/it]
 72%|███████▏  | 27440/37962 [63:33:13<24:16:32,  8.31s/it]
                                                           
{'loss': 1.2387, 'grad_norm': 4.266348361968994, 'learning_rate': 3.765864328402582e-06, 'epoch': 0.72}

 72%|███████▏  | 27440/37962 [63:33:13<24:16:32,  8.31s/it]
 72%|███████▏  | 27441/37962 [63:33:21<24:19:56,  8.33s/it]
 72%|███████▏  | 27442/37962 [63:33:29<24:26:25,  8.36s/it]
 72%|███████▏  | 27443/37962 [63:33:38<24:35:33,  8.42s/it]
 72%|███████▏  | 27444/37962 [63:33:46<24:19:56,  8.33s/it]
 72%|███████▏  | 27445/37962 [63:33:54<23:58:58,  8.21s/it]
 72%|███████▏  | 27446/37962 [63:34:02<24:07:22,  8.26s/it]
 72%|███████▏  | 27447/37962 [63:34:11<24:15:14,  8.30s/it]
 72%|███████▏  | 27448/37962 [63:34:19<24:02:47,  8.23s/it]
 72%|███████▏  | 27449/37962 [63:34:27<24:21:00,  8.34s/it]
 72%|███████▏  | 27450/37962 [63:34:36<24:08:40,  8.27s/it]
                                                           
{'loss': 1.1954, 'grad_norm': 13.613656997680664, 'learning_rate': 3.75919580260091e-06, 'epoch': 0.72}

 72%|███████▏  | 27450/37962 [63:34:36<24:08:40,  8.27s/it]
 72%|███████▏  | 27451/37962 [63:34:44<24:09:18,  8.27s/it]
 72%|███████▏  | 27452/37962 [63:34:52<24:10:08,  8.28s/it]
 72%|███████▏  | 27453/37962 [63:35:01<24:17:08,  8.32s/it]
 72%|███████▏  | 27454/37962 [63:35:09<24:18:59,  8.33s/it]
 72%|███████▏  | 27455/37962 [63:35:17<24:16:40,  8.32s/it]
 72%|███████▏  | 27456/37962 [63:35:25<24:10:31,  8.28s/it]
 72%|███████▏  | 27457/37962 [63:35:34<24:05:58,  8.26s/it]
 72%|███████▏  | 27458/37962 [63:35:42<24:06:00,  8.26s/it]
 72%|███████▏  | 27459/37962 [63:35:50<24:06:17,  8.26s/it]
 72%|███████▏  | 27460/37962 [63:35:58<24:00:43,  8.23s/it]
                                                           
{'loss': 1.1263, 'grad_norm': 7.081454277038574, 'learning_rate': 3.7525318193724967e-06, 'epoch': 0.72}

 72%|███████▏  | 27460/37962 [63:35:58<24:00:43,  8.23s/it]
 72%|███████▏  | 27461/37962 [63:36:06<23:55:27,  8.20s/it]
 72%|███████▏  | 27462/37962 [63:36:15<24:02:23,  8.24s/it]
 72%|███████▏  | 27463/37962 [63:36:23<24:10:37,  8.29s/it]
 72%|███████▏  | 27464/37962 [63:36:32<24:17:51,  8.33s/it]
 72%|███████▏  | 27465/37962 [63:36:40<24:06:01,  8.27s/it]
 72%|███████▏  | 27466/37962 [63:36:48<24:23:20,  8.37s/it]
 72%|███████▏  | 27467/37962 [63:36:57<24:18:58,  8.34s/it]
 72%|███████▏  | 27468/37962 [63:37:05<24:11:45,  8.30s/it]
 72%|███████▏  | 27469/37962 [63:37:13<24:11:32,  8.30s/it]
 72%|███████▏  | 27470/37962 [63:37:22<24:20:32,  8.35s/it]
                                                           
{'loss': 1.2437, 'grad_norm': 17.199438095092773, 'learning_rate': 3.7458723835679444e-06, 'epoch': 0.72}

 72%|███████▏  | 27470/37962 [63:37:22<24:20:32,  8.35s/it]
 72%|███████▏  | 27471/37962 [63:37:30<24:12:24,  8.31s/it]
 72%|███████▏  | 27472/37962 [63:37:38<24:12:26,  8.31s/it]
 72%|███████▏  | 27473/37962 [63:37:46<24:11:18,  8.30s/it]
 72%|███████▏  | 27474/37962 [63:37:55<24:23:10,  8.37s/it]
 72%|███████▏  | 27475/37962 [63:38:03<24:20:55,  8.36s/it]
 72%|███████▏  | 27476/37962 [63:38:11<24:07:52,  8.28s/it]
 72%|███████▏  | 27477/37962 [63:38:20<24:04:21,  8.27s/it]
 72%|███████▏  | 27478/37962 [63:38:28<24:12:03,  8.31s/it]
 72%|███████▏  | 27479/37962 [63:38:36<24:17:34,  8.34s/it]
 72%|███████▏  | 27480/37962 [63:38:45<24:24:13,  8.38s/it]
                                                           
{'loss': 1.3021, 'grad_norm': 90.51868438720703, 'learning_rate': 3.7392175000345387e-06, 'epoch': 0.72}

 72%|███████▏  | 27480/37962 [63:38:45<24:24:13,  8.38s/it]
 72%|███████▏  | 27481/37962 [63:38:53<24:20:07,  8.36s/it]
 72%|███████▏  | 27482/37962 [63:39:02<24:24:42,  8.39s/it]
 72%|███████▏  | 27483/37962 [63:39:10<24:16:34,  8.34s/it]
 72%|███████▏  | 27484/37962 [63:39:18<24:08:48,  8.30s/it]
 72%|███████▏  | 27485/37962 [63:39:26<23:54:23,  8.21s/it]
 72%|███████▏  | 27486/37962 [63:39:35<24:17:03,  8.35s/it]
 72%|███████▏  | 27487/37962 [63:39:43<24:06:55,  8.29s/it]
 72%|███████▏  | 27488/37962 [63:39:51<24:13:07,  8.32s/it]
 72%|███████▏  | 27489/37962 [63:40:00<24:10:34,  8.31s/it]
 72%|███████▏  | 27490/37962 [63:40:08<24:09:43,  8.31s/it]
                                                           
{'loss': 1.2069, 'grad_norm': 11.227408409118652, 'learning_rate': 3.7325671736162515e-06, 'epoch': 0.72}

 72%|███████▏  | 27490/37962 [63:40:08<24:09:43,  8.31s/it]
 72%|███████▏  | 27491/37962 [63:40:16<24:18:20,  8.36s/it]
 72%|███████▏  | 27492/37962 [63:40:24<23:52:03,  8.21s/it]
 72%|███████▏  | 27493/37962 [63:40:32<23:53:15,  8.21s/it]
 72%|███████▏  | 27494/37962 [63:40:41<24:01:53,  8.26s/it]
 72%|███████▏  | 27495/37962 [63:40:49<24:05:29,  8.29s/it]
 72%|███████▏  | 27496/37962 [63:40:57<23:55:31,  8.23s/it]
 72%|███████▏  | 27497/37962 [63:41:06<24:08:13,  8.30s/it]
 72%|███████▏  | 27498/37962 [63:41:14<24:03:09,  8.28s/it]
 72%|███████▏  | 27499/37962 [63:41:22<24:14:34,  8.34s/it]
 72%|███████▏  | 27500/37962 [63:41:31<24:08:44,  8.31s/it]
                                                           
{'loss': 1.1013, 'grad_norm': 9.376782417297363, 'learning_rate': 3.7259214091537422e-06, 'epoch': 0.72}

 72%|███████▏  | 27500/37962 [63:41:31<24:08:44,  8.31s/it]
 72%|███████▏  | 27501/37962 [63:41:39<24:05:19,  8.29s/it]
 72%|███████▏  | 27502/37962 [63:41:47<23:43:31,  8.17s/it]
 72%|███████▏  | 27503/37962 [63:41:55<23:51:21,  8.21s/it]
 72%|███████▏  | 27504/37962 [63:42:03<23:48:01,  8.19s/it]
 72%|███████▏  | 27505/37962 [63:42:11<23:33:14,  8.11s/it]
 72%|███████▏  | 27506/37962 [63:42:19<23:46:32,  8.19s/it]
 72%|███████▏  | 27507/37962 [63:42:28<24:01:06,  8.27s/it]
 72%|███████▏  | 27508/37962 [63:42:36<24:00:33,  8.27s/it]
 72%|███████▏  | 27509/37962 [63:42:44<23:44:50,  8.18s/it]
 72%|███████▏  | 27510/37962 [63:42:52<23:43:36,  8.17s/it]
                                                           
{'loss': 1.1985, 'grad_norm': 8.863065719604492, 'learning_rate': 3.7192802114843517e-06, 'epoch': 0.72}

 72%|███████▏  | 27510/37962 [63:42:52<23:43:36,  8.17s/it]
 72%|███████▏  | 27511/37962 [63:43:01<23:53:43,  8.23s/it]
 72%|███████▏  | 27512/37962 [63:43:09<23:51:53,  8.22s/it]
 72%|███████▏  | 27513/37962 [63:43:17<23:41:30,  8.16s/it]
 72%|███████▏  | 27514/37962 [63:43:25<23:43:19,  8.17s/it]
 72%|███████▏  | 27515/37962 [63:43:33<23:53:07,  8.23s/it]
 72%|███████▏  | 27516/37962 [63:43:42<23:50:12,  8.21s/it]
 72%|███████▏  | 27517/37962 [63:43:50<23:53:00,  8.23s/it]
 72%|███████▏  | 27518/37962 [63:43:58<23:54:20,  8.24s/it]
 72%|███████▏  | 27519/37962 [63:44:07<24:02:46,  8.29s/it]
 72%|███████▏  | 27520/37962 [63:44:15<24:02:32,  8.29s/it]
                                                           
{'loss': 1.2011, 'grad_norm': 5.11561393737793, 'learning_rate': 3.7126435854420863e-06, 'epoch': 0.72}

 72%|███████▏  | 27520/37962 [63:44:15<24:02:32,  8.29s/it]
 72%|███████▏  | 27521/37962 [63:44:23<23:52:33,  8.23s/it]
 72%|███████▏  | 27522/37962 [63:44:31<23:54:12,  8.24s/it]
 73%|███████▎  | 27523/37962 [63:44:40<24:01:08,  8.28s/it]
 73%|███████▎  | 27524/37962 [63:44:48<24:00:21,  8.28s/it]
 73%|███████▎  | 27525/37962 [63:44:56<23:55:46,  8.25s/it]
 73%|███████▎  | 27526/37962 [63:45:05<24:15:31,  8.37s/it]
 73%|███████▎  | 27527/37962 [63:45:13<23:59:48,  8.28s/it]
 73%|███████▎  | 27528/37962 [63:45:21<23:57:51,  8.27s/it]
 73%|███████▎  | 27529/37962 [63:45:29<23:48:57,  8.22s/it]
 73%|███████▎  | 27530/37962 [63:45:37<23:51:09,  8.23s/it]
                                                           
{'loss': 1.2471, 'grad_norm': 8.133282661437988, 'learning_rate': 3.7060115358576363e-06, 'epoch': 0.73}

 73%|███████▎  | 27530/37962 [63:45:37<23:51:09,  8.23s/it]
 73%|███████▎  | 27531/37962 [63:45:46<23:48:57,  8.22s/it]
 73%|███████▎  | 27532/37962 [63:45:54<23:51:45,  8.24s/it]
 73%|███████▎  | 27533/37962 [63:46:02<23:54:09,  8.25s/it]
 73%|███████▎  | 27534/37962 [63:46:11<24:03:58,  8.31s/it]
 73%|███████▎  | 27535/37962 [63:46:19<23:47:40,  8.22s/it]
 73%|███████▎  | 27536/37962 [63:46:27<23:52:38,  8.24s/it]
 73%|███████▎  | 27537/37962 [63:46:35<23:47:52,  8.22s/it]
 73%|███████▎  | 27538/37962 [63:46:44<24:05:00,  8.32s/it]
 73%|███████▎  | 27539/37962 [63:46:52<24:04:18,  8.31s/it]
 73%|███████▎  | 27540/37962 [63:47:00<24:16:46,  8.39s/it]
                                                           
{'loss': 1.1695, 'grad_norm': 4.464186668395996, 'learning_rate': 3.699384067558355e-06, 'epoch': 0.73}

 73%|███████▎  | 27540/37962 [63:47:00<24:16:46,  8.39s/it]
 73%|███████▎  | 27541/37962 [63:47:09<24:17:41,  8.39s/it]
 73%|███████▎  | 27542/37962 [63:47:17<24:07:16,  8.33s/it]
 73%|███████▎  | 27543/37962 [63:47:25<24:01:19,  8.30s/it]
 73%|███████▎  | 27544/37962 [63:47:34<23:58:30,  8.28s/it]
 73%|███████▎  | 27545/37962 [63:47:42<23:56:49,  8.28s/it]
 73%|███████▎  | 27546/37962 [63:47:50<23:44:16,  8.20s/it]
 73%|███████▎  | 27547/37962 [63:47:58<23:47:36,  8.22s/it]
 73%|███████▎  | 27548/37962 [63:48:07<23:56:56,  8.28s/it]
 73%|███████▎  | 27549/37962 [63:48:15<24:03:40,  8.32s/it]
 73%|███████▎  | 27550/37962 [63:48:23<24:02:04,  8.31s/it]
                                                           
{'loss': 1.156, 'grad_norm': 12.826943397521973, 'learning_rate': 3.6927611853682554e-06, 'epoch': 0.73}

 73%|███████▎  | 27550/37962 [63:48:23<24:02:04,  8.31s/it]
 73%|███████▎  | 27551/37962 [63:48:31<23:56:51,  8.28s/it]
 73%|███████▎  | 27552/37962 [63:48:40<23:56:08,  8.28s/it]
 73%|███████▎  | 27553/37962 [63:48:48<23:44:24,  8.21s/it]
 73%|███████▎  | 27554/37962 [63:48:56<24:01:39,  8.31s/it]
 73%|███████▎  | 27555/37962 [63:49:04<23:48:55,  8.24s/it]
 73%|███████▎  | 27556/37962 [63:49:13<23:56:17,  8.28s/it]
 73%|███████▎  | 27557/37962 [63:49:21<23:57:52,  8.29s/it]
 73%|███████▎  | 27558/37962 [63:49:29<23:58:19,  8.29s/it]
 73%|███████▎  | 27559/37962 [63:49:38<24:07:57,  8.35s/it]
 73%|███████▎  | 27560/37962 [63:49:46<24:06:30,  8.34s/it]
                                                           
{'loss': 1.1632, 'grad_norm': 11.135642051696777, 'learning_rate': 3.6861428941080277e-06, 'epoch': 0.73}

 73%|███████▎  | 27560/37962 [63:49:46<24:06:30,  8.34s/it]
 73%|███████▎  | 27561/37962 [63:49:54<23:58:04,  8.30s/it]
 73%|███████▎  | 27562/37962 [63:50:03<23:55:32,  8.28s/it]
 73%|███████▎  | 27563/37962 [63:50:11<23:49:43,  8.25s/it]
 73%|███████▎  | 27564/37962 [63:50:19<23:52:59,  8.27s/it]
 73%|███████▎  | 27565/37962 [63:50:27<23:51:09,  8.26s/it]
 73%|███████▎  | 27566/37962 [63:50:36<24:00:40,  8.31s/it]
 73%|███████▎  | 27567/37962 [63:50:44<23:45:17,  8.23s/it]
 73%|███████▎  | 27568/37962 [63:50:52<23:51:46,  8.26s/it]
 73%|███████▎  | 27569/37962 [63:51:00<23:50:15,  8.26s/it]
 73%|███████▎  | 27570/37962 [63:51:09<23:53:29,  8.28s/it]
                                                           
{'loss': 1.2144, 'grad_norm': 6.3439764976501465, 'learning_rate': 3.679529198595004e-06, 'epoch': 0.73}

 73%|███████▎  | 27570/37962 [63:51:09<23:53:29,  8.28s/it]
 73%|███████▎  | 27571/37962 [63:51:17<23:50:09,  8.26s/it]
 73%|███████▎  | 27572/37962 [63:51:25<23:48:38,  8.25s/it]
 73%|███████▎  | 27573/37962 [63:51:33<23:51:03,  8.26s/it]
 73%|███████▎  | 27574/37962 [63:51:42<23:40:23,  8.20s/it]
 73%|███████▎  | 27575/37962 [63:51:50<23:47:15,  8.24s/it]
 73%|███████▎  | 27576/37962 [63:51:58<23:41:22,  8.21s/it]
 73%|███████▎  | 27577/37962 [63:52:06<23:45:25,  8.24s/it]
 73%|███████▎  | 27578/37962 [63:52:14<23:43:07,  8.22s/it]
 73%|███████▎  | 27579/37962 [63:52:23<23:53:35,  8.28s/it]
 73%|███████▎  | 27580/37962 [63:52:31<23:51:12,  8.27s/it]
                                                           
{'loss': 1.2348, 'grad_norm': 8.332908630371094, 'learning_rate': 3.672920103643185e-06, 'epoch': 0.73}

 73%|███████▎  | 27580/37962 [63:52:31<23:51:12,  8.27s/it]
 73%|███████▎  | 27581/37962 [63:52:39<23:52:35,  8.28s/it]
 73%|███████▎  | 27582/37962 [63:52:48<24:00:58,  8.33s/it]
 73%|███████▎  | 27583/37962 [63:52:56<23:47:36,  8.25s/it]
 73%|███████▎  | 27584/37962 [63:53:04<23:51:18,  8.28s/it]
 73%|███████▎  | 27585/37962 [63:53:13<23:49:24,  8.26s/it]
 73%|███████▎  | 27586/37962 [63:53:21<23:47:27,  8.25s/it]
 73%|███████▎  | 27587/37962 [63:53:29<23:54:06,  8.29s/it]
 73%|███████▎  | 27588/37962 [63:53:37<23:52:44,  8.29s/it]
 73%|███████▎  | 27589/37962 [63:53:46<23:55:48,  8.31s/it]
 73%|███████▎  | 27590/37962 [63:53:54<23:51:03,  8.28s/it]
                                                           
{'loss': 1.2726, 'grad_norm': 4.791558742523193, 'learning_rate': 3.6663156140632137e-06, 'epoch': 0.73}

 73%|███████▎  | 27590/37962 [63:53:54<23:51:03,  8.28s/it]
 73%|███████▎  | 27591/37962 [63:54:02<23:51:30,  8.28s/it]
 73%|███████▎  | 27592/37962 [63:54:11<24:01:57,  8.34s/it]
 73%|███████▎  | 27593/37962 [63:54:19<23:59:19,  8.33s/it]
 73%|███████▎  | 27594/37962 [63:54:27<23:58:35,  8.33s/it]
 73%|███████▎  | 27595/37962 [63:54:36<23:56:12,  8.31s/it]
 73%|███████▎  | 27596/37962 [63:54:44<23:59:22,  8.33s/it]
 73%|███████▎  | 27597/37962 [63:54:52<23:46:39,  8.26s/it]
 73%|███████▎  | 27598/37962 [63:55:00<23:38:58,  8.21s/it]
 73%|███████▎  | 27599/37962 [63:55:08<23:38:08,  8.21s/it]
 73%|███████▎  | 27600/37962 [63:55:17<23:46:39,  8.26s/it]
                                                           
{'loss': 1.2408, 'grad_norm': 10.161201477050781, 'learning_rate': 3.65971573466238e-06, 'epoch': 0.73}

 73%|███████▎  | 27600/37962 [63:55:17<23:46:39,  8.26s/it]
 73%|███████▎  | 27601/37962 [63:55:25<23:30:34,  8.17s/it]
 73%|███████▎  | 27602/37962 [63:55:33<23:41:50,  8.23s/it]
 73%|███████▎  | 27603/37962 [63:55:41<23:29:48,  8.17s/it]
 73%|███████▎  | 27604/37962 [63:55:49<23:35:33,  8.20s/it]
 73%|███████▎  | 27605/37962 [63:55:58<23:42:59,  8.24s/it]
 73%|███████▎  | 27606/37962 [63:56:06<23:39:15,  8.22s/it]
 73%|███████▎  | 27607/37962 [63:56:14<23:43:14,  8.25s/it]
 73%|███████▎  | 27608/37962 [63:56:22<23:34:19,  8.20s/it]
 73%|███████▎  | 27609/37962 [63:56:31<23:40:11,  8.23s/it]
 73%|███████▎  | 27610/37962 [63:56:39<23:27:24,  8.16s/it]
                                                           
{'loss': 1.0737, 'grad_norm': 6.7998857498168945, 'learning_rate': 3.6531204702446245e-06, 'epoch': 0.73}

 73%|███████▎  | 27610/37962 [63:56:39<23:27:24,  8.16s/it]
 73%|███████▎  | 27611/37962 [63:56:47<23:34:43,  8.20s/it]
 73%|███████▎  | 27612/37962 [63:56:55<23:43:51,  8.25s/it]
 73%|███████▎  | 27613/37962 [63:57:03<23:35:45,  8.21s/it]
 73%|███████▎  | 27614/37962 [63:57:12<23:42:01,  8.25s/it]
 73%|███████▎  | 27615/37962 [63:57:20<23:24:09,  8.14s/it]
 73%|███████▎  | 27616/37962 [63:57:28<23:28:45,  8.17s/it]
 73%|███████▎  | 27617/37962 [63:57:36<23:39:55,  8.24s/it]
 73%|███████▎  | 27618/37962 [63:57:45<23:43:19,  8.26s/it]
 73%|███████▎  | 27619/37962 [63:57:53<23:41:56,  8.25s/it]
 73%|███████▎  | 27620/37962 [63:58:01<23:45:03,  8.27s/it]
                                                           
{'loss': 1.1997, 'grad_norm': 7.088876724243164, 'learning_rate': 3.646529825610531e-06, 'epoch': 0.73}

 73%|███████▎  | 27620/37962 [63:58:01<23:45:03,  8.27s/it]
 73%|███████▎  | 27621/37962 [63:58:09<23:48:50,  8.29s/it]
 73%|███████▎  | 27622/37962 [63:58:18<23:52:56,  8.31s/it]
 73%|███████▎  | 27623/37962 [63:58:26<23:25:43,  8.16s/it]
 73%|███████▎  | 27624/37962 [63:58:34<23:40:39,  8.25s/it]
 73%|███████▎  | 27625/37962 [63:58:42<23:42:14,  8.26s/it]
 73%|███████▎  | 27626/37962 [63:58:51<23:53:04,  8.32s/it]
 73%|███████▎  | 27627/37962 [63:58:59<23:52:20,  8.32s/it]
 73%|███████▎  | 27628/37962 [63:59:07<23:50:09,  8.30s/it]
 73%|███████▎  | 27629/37962 [63:59:16<23:55:22,  8.33s/it]
 73%|███████▎  | 27630/37962 [63:59:24<23:55:48,  8.34s/it]
                                                           
{'loss': 1.2839, 'grad_norm': 8.473071098327637, 'learning_rate': 3.63994380555731e-06, 'epoch': 0.73}

 73%|███████▎  | 27630/37962 [63:59:24<23:55:48,  8.34s/it]
 73%|███████▎  | 27631/37962 [63:59:32<23:53:25,  8.33s/it]
 73%|███████▎  | 27632/37962 [63:59:41<23:57:50,  8.35s/it]
 73%|███████▎  | 27633/37962 [63:59:49<23:59:02,  8.36s/it]
 73%|███████▎  | 27634/37962 [63:59:58<23:58:03,  8.35s/it]
 73%|███████▎  | 27635/37962 [64:00:06<23:57:39,  8.35s/it]
 73%|███████▎  | 27636/37962 [64:00:14<23:51:26,  8.32s/it]
 73%|███████▎  | 27637/37962 [64:00:22<23:49:26,  8.31s/it]
 73%|███████▎  | 27638/37962 [64:00:31<24:01:10,  8.38s/it]
 73%|███████▎  | 27639/37962 [64:00:39<23:53:52,  8.33s/it]
 73%|███████▎  | 27640/37962 [64:00:48<23:50:17,  8.31s/it]
                                                           
{'loss': 1.2391, 'grad_norm': 6.355103969573975, 'learning_rate': 3.6333624148788094e-06, 'epoch': 0.73}

 73%|███████▎  | 27640/37962 [64:00:48<23:50:17,  8.31s/it]
 73%|███████▎  | 27641/37962 [64:00:56<23:46:23,  8.29s/it]
 73%|███████▎  | 27642/37962 [64:01:04<23:47:07,  8.30s/it]
 73%|███████▎  | 27643/37962 [64:01:12<23:47:44,  8.30s/it]
 73%|███████▎  | 27644/37962 [64:01:20<23:38:01,  8.25s/it]
 73%|███████▎  | 27645/37962 [64:01:29<23:33:58,  8.22s/it]
 73%|███████▎  | 27646/37962 [64:01:37<23:37:52,  8.25s/it]
 73%|███████▎  | 27647/37962 [64:01:45<23:43:52,  8.28s/it]
 73%|███████▎  | 27648/37962 [64:01:53<23:37:10,  8.24s/it]
 73%|███████▎  | 27649/37962 [64:02:02<23:45:20,  8.29s/it]
 73%|███████▎  | 27650/37962 [64:02:10<23:40:16,  8.26s/it]
                                                           
{'loss': 1.2885, 'grad_norm': 17.870521545410156, 'learning_rate': 3.626785658365517e-06, 'epoch': 0.73}

 73%|███████▎  | 27650/37962 [64:02:10<23:40:16,  8.26s/it]
 73%|███████▎  | 27651/37962 [64:02:18<23:42:42,  8.28s/it]
 73%|███████▎  | 27652/37962 [64:02:26<23:33:20,  8.23s/it]
 73%|███████▎  | 27653/37962 [64:02:34<23:16:32,  8.13s/it]
 73%|███████▎  | 27654/37962 [64:02:43<23:28:24,  8.20s/it]
 73%|███████▎  | 27655/37962 [64:02:51<23:44:11,  8.29s/it]
 73%|███████▎  | 27656/37962 [64:03:00<23:44:46,  8.29s/it]
 73%|███████▎  | 27657/37962 [64:03:08<23:42:18,  8.28s/it]
 73%|███████▎  | 27658/37962 [64:03:16<23:42:00,  8.28s/it]
 73%|███████▎  | 27659/37962 [64:03:24<23:43:39,  8.29s/it]
 73%|███████▎  | 27660/37962 [64:03:33<23:43:44,  8.29s/it]
                                                           
{'loss': 1.3269, 'grad_norm': 8.336323738098145, 'learning_rate': 3.6202135408045314e-06, 'epoch': 0.73}

 73%|███████▎  | 27660/37962 [64:03:33<23:43:44,  8.29s/it]
 73%|███████▎  | 27661/37962 [64:03:41<23:36:03,  8.25s/it]
 73%|███████▎  | 27662/37962 [64:03:49<23:29:43,  8.21s/it]
 73%|███████▎  | 27663/37962 [64:03:57<23:03:14,  8.06s/it]
 73%|███████▎  | 27664/37962 [64:04:05<23:21:27,  8.17s/it]
 73%|███████▎  | 27665/37962 [64:04:13<23:28:22,  8.21s/it]
 73%|███████▎  | 27666/37962 [64:04:22<23:31:54,  8.23s/it]
 73%|███████▎  | 27667/37962 [64:04:30<23:36:45,  8.26s/it]
 73%|███████▎  | 27668/37962 [64:04:39<23:50:41,  8.34s/it]
 73%|███████▎  | 27669/37962 [64:04:47<23:51:27,  8.34s/it]
 73%|███████▎  | 27670/37962 [64:04:55<23:41:25,  8.29s/it]
                                                           
{'loss': 1.2479, 'grad_norm': 8.01080322265625, 'learning_rate': 3.6136460669795903e-06, 'epoch': 0.73}

 73%|███████▎  | 27670/37962 [64:04:55<23:41:25,  8.29s/it]
 73%|███████▎  | 27671/37962 [64:05:03<23:44:04,  8.30s/it]
 73%|███████▎  | 27672/37962 [64:05:12<23:41:47,  8.29s/it]
 73%|███████▎  | 27673/37962 [64:05:20<23:46:01,  8.32s/it]
 73%|███████▎  | 27674/37962 [64:05:28<23:41:53,  8.29s/it]
 73%|███████▎  | 27675/37962 [64:05:37<23:48:58,  8.33s/it]
 73%|███████▎  | 27676/37962 [64:05:45<23:47:12,  8.33s/it]
 73%|███████▎  | 27677/37962 [64:05:53<23:44:09,  8.31s/it]
 73%|███████▎  | 27678/37962 [64:06:01<23:27:53,  8.21s/it]
 73%|███████▎  | 27679/37962 [64:06:10<23:31:32,  8.24s/it]
 73%|███████▎  | 27680/37962 [64:06:18<23:32:52,  8.24s/it]
                                                           
{'loss': 1.3053, 'grad_norm': 15.984819412231445, 'learning_rate': 3.6070832416710375e-06, 'epoch': 0.73}

 73%|███████▎  | 27680/37962 [64:06:18<23:32:52,  8.24s/it]
 73%|███████▎  | 27681/37962 [64:06:26<23:13:08,  8.13s/it]
 73%|███████▎  | 27682/37962 [64:06:33<22:57:36,  8.04s/it]
 73%|███████▎  | 27683/37962 [64:06:42<22:57:36,  8.04s/it]
 73%|███████▎  | 27684/37962 [64:06:50<22:59:31,  8.05s/it]
 73%|███████▎  | 27685/37962 [64:06:58<23:11:40,  8.13s/it]
 73%|███████▎  | 27686/37962 [64:07:06<23:19:42,  8.17s/it]
 73%|███████▎  | 27687/37962 [64:07:14<23:17:53,  8.16s/it]
 73%|███████▎  | 27688/37962 [64:07:22<23:12:29,  8.13s/it]
 73%|███████▎  | 27689/37962 [64:07:31<23:23:24,  8.20s/it]
 73%|███████▎  | 27690/37962 [64:07:39<23:26:54,  8.22s/it]
                                                           
{'loss': 1.1353, 'grad_norm': 4.790171146392822, 'learning_rate': 3.600525069655847e-06, 'epoch': 0.73}

 73%|███████▎  | 27690/37962 [64:07:39<23:26:54,  8.22s/it]
 73%|███████▎  | 27691/37962 [64:07:47<23:28:51,  8.23s/it]
 73%|███████▎  | 27692/37962 [64:07:56<23:32:21,  8.25s/it]
 73%|███████▎  | 27693/37962 [64:08:04<23:41:46,  8.31s/it]
 73%|███████▎  | 27694/37962 [64:08:12<23:36:33,  8.28s/it]
 73%|███████▎  | 27695/37962 [64:08:20<23:35:41,  8.27s/it]
 73%|███████▎  | 27696/37962 [64:08:29<23:30:44,  8.25s/it]
 73%|███████▎  | 27697/37962 [64:08:37<23:43:53,  8.32s/it]
 73%|███████▎  | 27698/37962 [64:08:45<23:40:09,  8.30s/it]
 73%|███████▎  | 27699/37962 [64:08:54<23:34:17,  8.27s/it]
 73%|███████▎  | 27700/37962 [64:09:02<23:44:04,  8.33s/it]
                                                           
{'loss': 1.3046, 'grad_norm': 19.164060592651367, 'learning_rate': 3.593971555707595e-06, 'epoch': 0.73}

 73%|███████▎  | 27700/37962 [64:09:02<23:44:04,  8.33s/it]
 73%|███████▎  | 27701/37962 [64:09:10<23:43:35,  8.32s/it]
 73%|███████▎  | 27702/37962 [64:09:19<23:45:40,  8.34s/it]
 73%|███████▎  | 27703/37962 [64:09:27<23:38:14,  8.29s/it]
 73%|███████▎  | 27704/37962 [64:09:35<23:50:23,  8.37s/it]
 73%|███████▎  | 27705/37962 [64:09:51<30:04:12, 10.55s/it]
 73%|███████▎  | 27706/37962 [64:09:59<28:07:18,  9.87s/it]
 73%|███████▎  | 27707/37962 [64:10:08<26:51:11,  9.43s/it]
 73%|███████▎  | 27708/37962 [64:10:16<25:51:45,  9.08s/it]
 73%|███████▎  | 27709/37962 [64:10:25<25:19:29,  8.89s/it]
 73%|███████▎  | 27710/37962 [64:10:33<24:47:35,  8.71s/it]
                                                           
{'loss': 1.1326, 'grad_norm': 4.733276844024658, 'learning_rate': 3.5874227045964683e-06, 'epoch': 0.73}

 73%|███████▎  | 27710/37962 [64:10:33<24:47:35,  8.71s/it]
 73%|███████▎  | 27711/37962 [64:10:41<24:06:21,  8.47s/it]
 73%|███████▎  | 27712/37962 [64:10:49<24:05:45,  8.46s/it]
 73%|███████▎  | 27713/37962 [64:10:58<24:03:50,  8.45s/it]
 73%|███████▎  | 27714/37962 [64:11:06<23:52:20,  8.39s/it]
 73%|███████▎  | 27715/37962 [64:11:14<23:48:25,  8.36s/it]
 73%|███████▎  | 27716/37962 [64:11:22<23:18:59,  8.19s/it]
 73%|███████▎  | 27717/37962 [64:11:30<23:24:45,  8.23s/it]
 73%|███████▎  | 27718/37962 [64:11:39<23:34:49,  8.29s/it]
 73%|███████▎  | 27719/37962 [64:11:47<23:21:55,  8.21s/it]
 73%|███████▎  | 27720/37962 [64:11:55<23:22:47,  8.22s/it]
                                                           
{'loss': 1.245, 'grad_norm': 14.153071403503418, 'learning_rate': 3.580878521089264e-06, 'epoch': 0.73}

 73%|███████▎  | 27720/37962 [64:11:55<23:22:47,  8.22s/it]
 73%|███████▎  | 27721/37962 [64:12:03<23:33:55,  8.28s/it]
 73%|███████▎  | 27722/37962 [64:12:12<23:41:08,  8.33s/it]
 73%|███████▎  | 27723/37962 [64:12:20<23:29:09,  8.26s/it]
 73%|███████▎  | 27724/37962 [64:12:28<23:28:29,  8.25s/it]
 73%|███████▎  | 27725/37962 [64:12:36<23:29:41,  8.26s/it]
 73%|███████▎  | 27726/37962 [64:12:45<23:35:48,  8.30s/it]
 73%|███████▎  | 27727/37962 [64:12:53<23:39:53,  8.32s/it]
 73%|███████▎  | 27728/37962 [64:13:01<23:31:19,  8.27s/it]
 73%|███████▎  | 27729/37962 [64:13:10<23:37:40,  8.31s/it]
 73%|███████▎  | 27730/37962 [64:13:18<23:34:12,  8.29s/it]
                                                           
{'loss': 1.1434, 'grad_norm': 69.43119812011719, 'learning_rate': 3.5743390099493835e-06, 'epoch': 0.73}

 73%|███████▎  | 27730/37962 [64:13:18<23:34:12,  8.29s/it]
 73%|███████▎  | 27731/37962 [64:13:26<23:30:59,  8.27s/it]
 73%|███████▎  | 27732/37962 [64:13:35<23:41:54,  8.34s/it]
 73%|███████▎  | 27733/37962 [64:13:43<23:34:36,  8.30s/it]
 73%|███████▎  | 27734/37962 [64:13:51<23:36:46,  8.31s/it]
 73%|███████▎  | 27735/37962 [64:13:59<23:30:19,  8.27s/it]
 73%|███████▎  | 27736/37962 [64:14:08<23:33:54,  8.30s/it]
 73%|███████▎  | 27737/37962 [64:14:16<23:12:35,  8.17s/it]
 73%|███████▎  | 27738/37962 [64:14:24<23:14:58,  8.19s/it]
 73%|███████▎  | 27739/37962 [64:14:32<23:23:15,  8.24s/it]
 73%|███████▎  | 27740/37962 [64:14:41<23:33:29,  8.30s/it]
                                                           
{'loss': 1.3792, 'grad_norm': 12.831033706665039, 'learning_rate': 3.567804175936821e-06, 'epoch': 0.73}

 73%|███████▎  | 27740/37962 [64:14:41<23:33:29,  8.30s/it]
 73%|███████▎  | 27741/37962 [64:14:49<23:20:49,  8.22s/it]
 73%|███████▎  | 27742/37962 [64:14:57<23:12:06,  8.17s/it]
 73%|███████▎  | 27743/37962 [64:15:05<23:27:16,  8.26s/it]
 73%|███████▎  | 27744/37962 [64:15:13<23:23:09,  8.24s/it]
 73%|███████▎  | 27745/37962 [64:15:22<23:28:02,  8.27s/it]
 73%|███████▎  | 27746/37962 [64:15:30<23:17:47,  8.21s/it]
 73%|███████▎  | 27747/37962 [64:15:38<23:36:41,  8.32s/it]
 73%|███████▎  | 27748/37962 [64:15:47<23:32:44,  8.30s/it]
 73%|███████▎  | 27749/37962 [64:15:55<23:36:04,  8.32s/it]
 73%|███████▎  | 27750/37962 [64:16:03<23:33:20,  8.30s/it]
                                                           
{'loss': 1.2688, 'grad_norm': 11.46628475189209, 'learning_rate': 3.5612740238081643e-06, 'epoch': 0.73}

 73%|███████▎  | 27750/37962 [64:16:03<23:33:20,  8.30s/it]
 73%|███████▎  | 27751/37962 [64:16:12<23:34:39,  8.31s/it]
 73%|███████▎  | 27752/37962 [64:16:20<23:22:40,  8.24s/it]
 73%|███████▎  | 27753/37962 [64:16:28<23:10:48,  8.17s/it]
 73%|███████▎  | 27754/37962 [64:16:36<23:16:32,  8.21s/it]
 73%|███████▎  | 27755/37962 [64:16:44<23:23:17,  8.25s/it]
 73%|███████▎  | 27756/37962 [64:16:53<23:19:11,  8.23s/it]
 73%|███████▎  | 27757/37962 [64:17:01<23:28:44,  8.28s/it]
 73%|███████▎  | 27758/37962 [64:17:09<23:28:57,  8.28s/it]
 73%|███████▎  | 27759/37962 [64:17:18<23:33:53,  8.31s/it]
 73%|███████▎  | 27760/37962 [64:17:26<23:32:02,  8.30s/it]
                                                           
{'loss': 1.1629, 'grad_norm': 6.316324234008789, 'learning_rate': 3.5547485583166076e-06, 'epoch': 0.73}

 73%|███████▎  | 27760/37962 [64:17:26<23:32:02,  8.30s/it]
 73%|███████▎  | 27761/37962 [64:17:34<23:08:26,  8.17s/it]
 73%|███████▎  | 27762/37962 [64:17:42<23:22:02,  8.25s/it]
 73%|███████▎  | 27763/37962 [64:17:51<23:27:53,  8.28s/it]
 73%|███████▎  | 27764/37962 [64:17:59<23:36:04,  8.33s/it]
 73%|███████▎  | 27765/37962 [64:18:07<23:34:29,  8.32s/it]
 73%|███████▎  | 27766/37962 [64:18:16<23:29:45,  8.30s/it]
 73%|███████▎  | 27767/37962 [64:18:23<23:10:04,  8.18s/it]
 73%|███████▎  | 27768/37962 [64:18:32<23:19:28,  8.24s/it]
 73%|███████▎  | 27769/37962 [64:18:40<23:20:39,  8.24s/it]
 73%|███████▎  | 27770/37962 [64:18:48<23:13:16,  8.20s/it]
                                                           
{'loss': 1.1249, 'grad_norm': 7.9033284187316895, 'learning_rate': 3.548227784211914e-06, 'epoch': 0.73}

 73%|███████▎  | 27770/37962 [64:18:48<23:13:16,  8.20s/it]
 73%|███████▎  | 27771/37962 [64:18:56<23:03:36,  8.15s/it]
 73%|███████▎  | 27772/37962 [64:19:05<23:27:35,  8.29s/it]
 73%|███████▎  | 27773/37962 [64:19:13<23:30:04,  8.30s/it]
 73%|███████▎  | 27774/37962 [64:19:21<23:27:11,  8.29s/it]
 73%|███████▎  | 27775/37962 [64:19:30<23:27:12,  8.29s/it]
 73%|███████▎  | 27776/37962 [64:19:38<23:34:44,  8.33s/it]
 73%|███████▎  | 27777/37962 [64:19:46<23:23:11,  8.27s/it]
 73%|███████▎  | 27778/37962 [64:19:55<23:23:02,  8.27s/it]
 73%|███████▎  | 27779/37962 [64:20:03<23:19:41,  8.25s/it]
 73%|███████▎  | 27780/37962 [64:20:11<23:23:55,  8.27s/it]
                                                           
{'loss': 1.2182, 'grad_norm': 17.419740676879883, 'learning_rate': 3.5417117062404483e-06, 'epoch': 0.73}

 73%|███████▎  | 27780/37962 [64:20:11<23:23:55,  8.27s/it]
 73%|███████▎  | 27781/37962 [64:20:20<23:34:46,  8.34s/it]
 73%|███████▎  | 27782/37962 [64:20:28<23:29:10,  8.31s/it]
 73%|███████▎  | 27783/37962 [64:20:36<23:27:45,  8.30s/it]
 73%|███████▎  | 27784/37962 [64:20:44<23:23:59,  8.28s/it]
 73%|███████▎  | 27785/37962 [64:20:53<23:23:36,  8.28s/it]
 73%|███████▎  | 27786/37962 [64:21:01<23:25:34,  8.29s/it]
 73%|███████▎  | 27787/37962 [64:21:09<23:21:35,  8.26s/it]
 73%|███████▎  | 27788/37962 [64:21:17<23:18:45,  8.25s/it]
 73%|███████▎  | 27789/37962 [64:21:26<23:26:38,  8.30s/it]
 73%|███████▎  | 27790/37962 [64:21:34<23:30:42,  8.32s/it]
                                                           
{'loss': 1.3151, 'grad_norm': 12.671055793762207, 'learning_rate': 3.5352003291451555e-06, 'epoch': 0.73}

 73%|███████▎  | 27790/37962 [64:21:34<23:30:42,  8.32s/it]
 73%|███████▎  | 27791/37962 [64:21:42<23:34:30,  8.34s/it]
 73%|███████▎  | 27792/37962 [64:21:51<23:24:26,  8.29s/it]
 73%|███████▎  | 27793/37962 [64:21:59<23:34:58,  8.35s/it]
 73%|███████▎  | 27794/37962 [64:22:07<23:32:26,  8.33s/it]
 73%|███████▎  | 27795/37962 [64:22:16<23:32:26,  8.34s/it]
 73%|███████▎  | 27796/37962 [64:22:24<23:32:36,  8.34s/it]
 73%|███████▎  | 27797/37962 [64:22:32<23:34:11,  8.35s/it]
 73%|███████▎  | 27798/37962 [64:22:41<23:26:33,  8.30s/it]
 73%|███████▎  | 27799/37962 [64:22:49<23:22:03,  8.28s/it]
 73%|███████▎  | 27800/37962 [64:22:57<23:23:35,  8.29s/it]
                                                           
{'loss': 1.3401, 'grad_norm': 11.4597749710083, 'learning_rate': 3.5286936576655404e-06, 'epoch': 0.73}

 73%|███████▎  | 27800/37962 [64:22:57<23:23:35,  8.29s/it]
 73%|███████▎  | 27801/37962 [64:23:06<23:25:45,  8.30s/it]
 73%|███████▎  | 27802/37962 [64:23:14<23:23:48,  8.29s/it]
 73%|███████▎  | 27803/37962 [64:23:22<23:26:21,  8.31s/it]
 73%|███████▎  | 27804/37962 [64:23:30<23:25:16,  8.30s/it]
 73%|███████▎  | 27805/37962 [64:23:38<23:11:57,  8.22s/it]
 73%|███████▎  | 27806/37962 [64:23:54<29:47:01, 10.56s/it]
 73%|███████▎  | 27807/37962 [64:24:03<27:56:21,  9.90s/it]
 73%|███████▎  | 27808/37962 [64:24:11<26:18:18,  9.33s/it]
 73%|███████▎  | 27809/37962 [64:24:19<25:36:27,  9.08s/it]
 73%|███████▎  | 27810/37962 [64:24:28<24:57:13,  8.85s/it]
                                                           
{'loss': 1.3419, 'grad_norm': 8.529892921447754, 'learning_rate': 3.5221916965377057e-06, 'epoch': 0.73}

 73%|███████▎  | 27810/37962 [64:24:28<24:57:13,  8.85s/it]
 73%|███████▎  | 27811/37962 [64:24:36<24:29:13,  8.68s/it]
 73%|███████▎  | 27812/37962 [64:24:44<24:11:22,  8.58s/it]
 73%|███████▎  | 27813/37962 [64:24:53<24:01:27,  8.52s/it]
 73%|███████▎  | 27814/37962 [64:25:01<24:01:39,  8.52s/it]
 73%|███████▎  | 27815/37962 [64:25:09<23:27:24,  8.32s/it]
 73%|███████▎  | 27816/37962 [64:25:17<23:22:00,  8.29s/it]
 73%|███████▎  | 27817/37962 [64:25:26<23:20:54,  8.29s/it]
 73%|███████▎  | 27818/37962 [64:25:34<23:23:11,  8.30s/it]
 73%|███████▎  | 27819/37962 [64:25:42<23:10:12,  8.22s/it]
 73%|███████▎  | 27820/37962 [64:25:50<23:07:33,  8.21s/it]
                                                           
{'loss': 1.2133, 'grad_norm': 6.525213718414307, 'learning_rate': 3.515694450494318e-06, 'epoch': 0.73}

 73%|███████▎  | 27820/37962 [64:25:50<23:07:33,  8.21s/it]
 73%|███████▎  | 27821/37962 [64:25:58<23:02:17,  8.18s/it]
 73%|███████▎  | 27822/37962 [64:26:07<23:10:08,  8.23s/it]
 73%|███████▎  | 27823/37962 [64:26:15<22:58:52,  8.16s/it]
 73%|███████▎  | 27824/37962 [64:26:23<23:02:56,  8.18s/it]
 73%|███████▎  | 27825/37962 [64:26:31<23:15:58,  8.26s/it]
 73%|███████▎  | 27826/37962 [64:26:40<23:18:26,  8.28s/it]
 73%|███████▎  | 27827/37962 [64:26:48<23:15:51,  8.26s/it]
 73%|███████▎  | 27828/37962 [64:26:56<23:13:46,  8.25s/it]
 73%|███████▎  | 27829/37962 [64:27:04<23:22:01,  8.30s/it]
 73%|███████▎  | 27830/37962 [64:27:13<23:23:23,  8.31s/it]
                                                           
{'loss': 1.281, 'grad_norm': 6.203141689300537, 'learning_rate': 3.5092019242646025e-06, 'epoch': 0.73}

 73%|███████▎  | 27830/37962 [64:27:13<23:23:23,  8.31s/it]
 73%|███████▎  | 27831/37962 [64:27:21<23:01:11,  8.18s/it]
 73%|███████▎  | 27832/37962 [64:27:29<23:01:28,  8.18s/it]
 73%|███████▎  | 27833/37962 [64:27:37<23:11:00,  8.24s/it]
 73%|███████▎  | 27834/37962 [64:27:45<23:10:23,  8.24s/it]
 73%|███████▎  | 27835/37962 [64:27:54<23:05:00,  8.21s/it]
 73%|███████▎  | 27836/37962 [64:28:02<23:09:33,  8.23s/it]
 73%|███████▎  | 27837/37962 [64:28:10<22:53:25,  8.14s/it]
 73%|███████▎  | 27838/37962 [64:28:18<22:55:34,  8.15s/it]
 73%|███████▎  | 27839/37962 [64:28:26<23:11:12,  8.25s/it]
 73%|███████▎  | 27840/37962 [64:28:35<23:16:30,  8.28s/it]
                                                           
{'loss': 1.1847, 'grad_norm': 4.599928855895996, 'learning_rate': 3.5027141225743656e-06, 'epoch': 0.73}

 73%|███████▎  | 27840/37962 [64:28:35<23:16:30,  8.28s/it]
 73%|███████▎  | 27841/37962 [64:28:43<23:12:55,  8.26s/it]
 73%|███████▎  | 27842/37962 [64:28:51<22:53:55,  8.15s/it]
 73%|███████▎  | 27843/37962 [64:28:59<23:01:20,  8.19s/it]
 73%|███████▎  | 27844/37962 [64:29:07<23:04:28,  8.21s/it]
 73%|███████▎  | 27845/37962 [64:29:16<23:01:27,  8.19s/it]
 73%|███████▎  | 27846/37962 [64:29:24<23:11:00,  8.25s/it]
 73%|███████▎  | 27847/37962 [64:29:32<23:10:26,  8.25s/it]
 73%|███████▎  | 27848/37962 [64:29:40<23:08:53,  8.24s/it]
 73%|███████▎  | 27849/37962 [64:29:49<23:07:34,  8.23s/it]
 73%|███████▎  | 27850/37962 [64:29:57<22:59:56,  8.19s/it]
                                                           
{'loss': 1.1693, 'grad_norm': 6.271599769592285, 'learning_rate': 3.4962310501459606e-06, 'epoch': 0.73}

 73%|███████▎  | 27850/37962 [64:29:57<22:59:56,  8.19s/it]
 73%|███████▎  | 27851/37962 [64:30:05<22:50:39,  8.13s/it]
 73%|███████▎  | 27852/37962 [64:30:13<22:49:11,  8.13s/it]
 73%|███████▎  | 27853/37962 [64:30:21<23:01:02,  8.20s/it]
 73%|███████▎  | 27854/37962 [64:30:29<23:04:26,  8.22s/it]
 73%|███████▎  | 27855/37962 [64:30:38<23:04:10,  8.22s/it]
 73%|███████▎  | 27856/37962 [64:30:46<23:05:54,  8.23s/it]
 73%|███████▎  | 27857/37962 [64:30:54<22:59:17,  8.19s/it]
 73%|███████▎  | 27858/37962 [64:31:02<23:05:17,  8.23s/it]
 73%|███████▎  | 27859/37962 [64:31:11<23:13:32,  8.28s/it]
 73%|███████▎  | 27860/37962 [64:31:19<23:09:54,  8.26s/it]
                                                           
{'loss': 1.2112, 'grad_norm': 7.340092182159424, 'learning_rate': 3.489752711698301e-06, 'epoch': 0.73}

 73%|███████▎  | 27860/37962 [64:31:19<23:09:54,  8.26s/it]
 73%|███████▎  | 27861/37962 [64:31:27<23:08:07,  8.25s/it]
 73%|███████▎  | 27862/37962 [64:31:35<23:10:50,  8.26s/it]
 73%|███████▎  | 27863/37962 [64:31:43<22:57:55,  8.19s/it]
 73%|███████▎  | 27864/37962 [64:31:52<23:03:04,  8.22s/it]
 73%|███████▎  | 27865/37962 [64:32:00<23:11:47,  8.27s/it]
 73%|███████▎  | 27866/37962 [64:32:08<23:12:52,  8.28s/it]
 73%|███████▎  | 27867/37962 [64:32:17<23:11:12,  8.27s/it]
 73%|███████▎  | 27868/37962 [64:32:25<23:24:22,  8.35s/it]
 73%|███████▎  | 27869/37962 [64:32:34<23:32:06,  8.39s/it]
 73%|███████▎  | 27870/37962 [64:32:42<23:34:34,  8.41s/it]
                                                           
{'loss': 1.2962, 'grad_norm': 7.582787036895752, 'learning_rate': 3.4832791119468624e-06, 'epoch': 0.73}

 73%|███████▎  | 27870/37962 [64:32:42<23:34:34,  8.41s/it]
 73%|███████▎  | 27871/37962 [64:32:51<23:31:29,  8.39s/it]
 73%|███████▎  | 27872/37962 [64:32:59<23:17:29,  8.31s/it]
 73%|███████▎  | 27873/37962 [64:33:07<23:16:35,  8.31s/it]
 73%|███████▎  | 27874/37962 [64:33:15<23:12:12,  8.28s/it]
 73%|███████▎  | 27875/37962 [64:33:23<22:58:34,  8.20s/it]
 73%|███████▎  | 27876/37962 [64:33:32<23:05:38,  8.24s/it]
 73%|███████▎  | 27877/37962 [64:33:40<23:24:17,  8.35s/it]
 73%|███████▎  | 27878/37962 [64:33:48<23:17:28,  8.32s/it]
 73%|███████▎  | 27879/37962 [64:33:56<23:03:35,  8.23s/it]
 73%|███████▎  | 27880/37962 [64:34:05<23:08:27,  8.26s/it]
                                                           
{'loss': 1.2808, 'grad_norm': 8.689005851745605, 'learning_rate': 3.476810255603662e-06, 'epoch': 0.73}

 73%|███████▎  | 27880/37962 [64:34:05<23:08:27,  8.26s/it]
 73%|███████▎  | 27881/37962 [64:34:13<23:13:30,  8.29s/it]
 73%|███████▎  | 27882/37962 [64:34:21<23:17:42,  8.32s/it]
 73%|███████▎  | 27883/37962 [64:34:30<23:29:18,  8.39s/it]
 73%|███████▎  | 27884/37962 [64:34:38<23:22:44,  8.35s/it]
 73%|███████▎  | 27885/37962 [64:34:47<23:25:08,  8.37s/it]
 73%|███████▎  | 27886/37962 [64:34:55<23:16:25,  8.32s/it]
 73%|███████▎  | 27887/37962 [64:35:03<23:28:20,  8.39s/it]
 73%|███████▎  | 27888/37962 [64:35:12<23:14:17,  8.30s/it]
 73%|███████▎  | 27889/37962 [64:35:20<23:14:31,  8.31s/it]
 73%|███████▎  | 27890/37962 [64:35:28<23:20:38,  8.34s/it]
                                                           
{'loss': 1.3084, 'grad_norm': 14.772737503051758, 'learning_rate': 3.470346147377269e-06, 'epoch': 0.73}

 73%|███████▎  | 27890/37962 [64:35:28<23:20:38,  8.34s/it]
 73%|███████▎  | 27891/37962 [64:35:37<23:13:11,  8.30s/it]
 73%|███████▎  | 27892/37962 [64:35:45<23:08:29,  8.27s/it]
 73%|███████▎  | 27893/37962 [64:35:53<23:09:30,  8.28s/it]
 73%|███████▎  | 27894/37962 [64:36:01<23:09:11,  8.28s/it]
 73%|███████▎  | 27895/37962 [64:36:09<23:02:02,  8.24s/it]
 73%|███████▎  | 27896/37962 [64:36:18<23:07:38,  8.27s/it]
 73%|███████▎  | 27897/37962 [64:36:26<23:00:56,  8.23s/it]
 73%|███████▎  | 27898/37962 [64:36:34<23:09:37,  8.28s/it]
 73%|███████▎  | 27899/37962 [64:36:43<23:04:41,  8.26s/it]
 73%|███████▎  | 27900/37962 [64:36:51<23:07:10,  8.27s/it]
                                                           
{'loss': 1.1936, 'grad_norm': 8.314057350158691, 'learning_rate': 3.463886791972804e-06, 'epoch': 0.73}

 73%|███████▎  | 27900/37962 [64:36:51<23:07:10,  8.27s/it]
 73%|███████▎  | 27901/37962 [64:36:59<22:53:38,  8.19s/it]
 73%|███████▎  | 27902/37962 [64:37:07<22:54:06,  8.20s/it]
 74%|███████▎  | 27903/37962 [64:37:15<22:59:26,  8.23s/it]
 74%|███████▎  | 27904/37962 [64:37:23<22:41:32,  8.12s/it]
 74%|███████▎  | 27905/37962 [64:37:32<22:54:30,  8.20s/it]
 74%|███████▎  | 27906/37962 [64:37:40<22:57:42,  8.22s/it]
 74%|███████▎  | 27907/37962 [64:37:48<22:58:02,  8.22s/it]
 74%|███████▎  | 27908/37962 [64:37:56<23:00:25,  8.24s/it]
 74%|███████▎  | 27909/37962 [64:38:05<23:00:31,  8.24s/it]
 74%|███████▎  | 27910/37962 [64:38:13<23:04:05,  8.26s/it]
                                                           
{'loss': 1.2585, 'grad_norm': 6.526559352874756, 'learning_rate': 3.4574321940919054e-06, 'epoch': 0.74}

 74%|███████▎  | 27910/37962 [64:38:13<23:04:05,  8.26s/it]
 74%|███████▎  | 27911/37962 [64:38:21<23:03:43,  8.26s/it]
 74%|███████▎  | 27912/37962 [64:38:29<23:05:02,  8.27s/it]
 74%|███████▎  | 27913/37962 [64:38:38<23:12:00,  8.31s/it]
 74%|███████▎  | 27914/37962 [64:38:46<23:06:17,  8.28s/it]
 74%|███████▎  | 27915/37962 [64:38:54<23:06:48,  8.28s/it]
 74%|███████▎  | 27916/37962 [64:39:03<23:09:26,  8.30s/it]
 74%|███████▎  | 27917/37962 [64:39:11<23:01:00,  8.25s/it]
 74%|███████▎  | 27918/37962 [64:39:19<23:02:13,  8.26s/it]
 74%|███████▎  | 27919/37962 [64:39:27<23:04:07,  8.27s/it]
 74%|███████▎  | 27920/37962 [64:39:36<22:57:01,  8.23s/it]
                                                           
{'loss': 1.1911, 'grad_norm': 9.90458869934082, 'learning_rate': 3.4509823584327706e-06, 'epoch': 0.74}

 74%|███████▎  | 27920/37962 [64:39:36<22:57:01,  8.23s/it]
 74%|███████▎  | 27921/37962 [64:39:44<23:02:50,  8.26s/it]
 74%|███████▎  | 27922/37962 [64:39:52<23:06:04,  8.28s/it]
 74%|███████▎  | 27923/37962 [64:40:00<23:04:47,  8.28s/it]
 74%|███████▎  | 27924/37962 [64:40:09<22:59:45,  8.25s/it]
 74%|███████▎  | 27925/37962 [64:40:17<22:40:34,  8.13s/it]
 74%|███████▎  | 27926/37962 [64:40:25<22:45:07,  8.16s/it]
 74%|███████▎  | 27927/37962 [64:40:33<22:54:08,  8.22s/it]
 74%|███████▎  | 27928/37962 [64:40:41<22:44:24,  8.16s/it]
 74%|███████▎  | 27929/37962 [64:40:49<22:44:30,  8.16s/it]
 74%|███████▎  | 27930/37962 [64:40:57<22:45:19,  8.17s/it]
                                                           
{'loss': 1.1673, 'grad_norm': 7.201440811157227, 'learning_rate': 3.4445372896901276e-06, 'epoch': 0.74}

 74%|███████▎  | 27930/37962 [64:40:57<22:45:19,  8.17s/it]
 74%|███████▎  | 27931/37962 [64:41:06<22:55:51,  8.23s/it]
 74%|███████▎  | 27932/37962 [64:41:14<22:56:12,  8.23s/it]
 74%|███████▎  | 27933/37962 [64:41:22<23:01:37,  8.27s/it]
 74%|███████▎  | 27934/37962 [64:41:31<22:59:53,  8.26s/it]
 74%|███████▎  | 27935/37962 [64:41:39<22:59:02,  8.25s/it]
 74%|███████▎  | 27936/37962 [64:41:47<23:01:30,  8.27s/it]
 74%|███████▎  | 27937/37962 [64:41:55<22:59:00,  8.25s/it]
 74%|███████▎  | 27938/37962 [64:42:04<23:03:50,  8.28s/it]
 74%|███████▎  | 27939/37962 [64:42:12<23:00:21,  8.26s/it]
 74%|███████▎  | 27940/37962 [64:42:20<23:08:49,  8.31s/it]
                                                           
{'loss': 1.3175, 'grad_norm': 13.639850616455078, 'learning_rate': 3.438096992555221e-06, 'epoch': 0.74}

 74%|███████▎  | 27940/37962 [64:42:20<23:08:49,  8.31s/it]
 74%|███████▎  | 27941/37962 [64:42:28<22:49:51,  8.20s/it]
 74%|███████▎  | 27942/37962 [64:42:36<22:38:57,  8.14s/it]
 74%|███████▎  | 27943/37962 [64:42:45<22:55:35,  8.24s/it]
 74%|███████▎  | 27944/37962 [64:42:53<22:55:49,  8.24s/it]
 74%|███████▎  | 27945/37962 [64:43:01<22:58:57,  8.26s/it]
 74%|███████▎  | 27946/37962 [64:43:10<22:58:41,  8.26s/it]
 74%|███████▎  | 27947/37962 [64:43:18<22:58:34,  8.26s/it]
 74%|███████▎  | 27948/37962 [64:43:26<23:04:48,  8.30s/it]
 74%|███████▎  | 27949/37962 [64:43:35<23:06:37,  8.31s/it]
 74%|███████▎  | 27950/37962 [64:43:43<22:59:27,  8.27s/it]
                                                           
{'loss': 1.3027, 'grad_norm': 7.421821117401123, 'learning_rate': 3.4316614717158405e-06, 'epoch': 0.74}

 74%|███████▎  | 27950/37962 [64:43:43<22:59:27,  8.27s/it]
 74%|███████▎  | 27951/37962 [64:43:51<22:45:56,  8.19s/it]
 74%|███████▎  | 27952/37962 [64:43:59<22:53:29,  8.23s/it]
 74%|███████▎  | 27953/37962 [64:44:08<23:02:23,  8.29s/it]
 74%|███████▎  | 27954/37962 [64:44:16<22:57:59,  8.26s/it]
 74%|███████▎  | 27955/37962 [64:44:24<23:05:14,  8.31s/it]
 74%|███████▎  | 27956/37962 [64:44:32<23:02:59,  8.29s/it]
 74%|███████▎  | 27957/37962 [64:44:41<23:02:17,  8.29s/it]
 74%|███████▎  | 27958/37962 [64:44:49<23:03:20,  8.30s/it]
 74%|███████▎  | 27959/37962 [64:44:57<23:01:36,  8.29s/it]
 74%|███████▎  | 27960/37962 [64:45:05<22:52:45,  8.23s/it]
                                                           
{'loss': 1.2021, 'grad_norm': 9.729570388793945, 'learning_rate': 3.425230731856286e-06, 'epoch': 0.74}

 74%|███████▎  | 27960/37962 [64:45:05<22:52:45,  8.23s/it]
 74%|███████▎  | 27961/37962 [64:45:13<22:41:51,  8.17s/it]
 74%|███████▎  | 27962/37962 [64:45:22<22:49:24,  8.22s/it]
 74%|███████▎  | 27963/37962 [64:45:30<22:54:26,  8.25s/it]
 74%|███████▎  | 27964/37962 [64:45:39<23:18:51,  8.39s/it]
 74%|███████▎  | 27965/37962 [64:45:47<23:18:43,  8.39s/it]
 74%|███████▎  | 27966/37962 [64:45:56<23:15:47,  8.38s/it]
 74%|███████▎  | 27967/37962 [64:46:04<23:00:44,  8.29s/it]
 74%|███████▎  | 27968/37962 [64:46:12<22:55:57,  8.26s/it]
 74%|███████▎  | 27969/37962 [64:46:20<22:55:58,  8.26s/it]
 74%|███████▎  | 27970/37962 [64:46:29<23:04:45,  8.32s/it]
                                                           
{'loss': 1.1681, 'grad_norm': 6.487699031829834, 'learning_rate': 3.4188047776573785e-06, 'epoch': 0.74}

 74%|███████▎  | 27970/37962 [64:46:29<23:04:45,  8.32s/it]
 74%|███████▎  | 27971/37962 [64:46:37<23:04:18,  8.31s/it]
 74%|███████▎  | 27972/37962 [64:46:45<22:52:28,  8.24s/it]
 74%|███████▎  | 27973/37962 [64:46:53<22:59:28,  8.29s/it]
 74%|███████▎  | 27974/37962 [64:47:02<23:07:10,  8.33s/it]
 74%|███████▎  | 27975/37962 [64:47:10<23:04:59,  8.32s/it]
 74%|███████▎  | 27976/37962 [64:47:18<22:58:25,  8.28s/it]
 74%|███████▎  | 27977/37962 [64:47:27<23:00:47,  8.30s/it]
 74%|███████▎  | 27978/37962 [64:47:35<23:10:52,  8.36s/it]
 74%|███████▎  | 27979/37962 [64:47:43<23:12:20,  8.37s/it]
 74%|███████▎  | 27980/37962 [64:47:52<23:05:52,  8.33s/it]
                                                           
{'loss': 1.2198, 'grad_norm': 7.422901153564453, 'learning_rate': 3.412383613796465e-06, 'epoch': 0.74}

 74%|███████▎  | 27980/37962 [64:47:52<23:05:52,  8.33s/it]
 74%|███████▎  | 27981/37962 [64:48:00<23:01:32,  8.30s/it]
 74%|███████▎  | 27982/37962 [64:48:08<23:02:13,  8.31s/it]
 74%|███████▎  | 27983/37962 [64:48:16<22:53:07,  8.26s/it]
 74%|███████▎  | 27984/37962 [64:48:24<22:30:10,  8.12s/it]
 74%|███████▎  | 27985/37962 [64:48:33<22:44:37,  8.21s/it]
 74%|███████▎  | 27986/37962 [64:48:41<22:55:04,  8.27s/it]
 74%|███████▎  | 27987/37962 [64:48:49<22:51:57,  8.25s/it]
 74%|███████▎  | 27988/37962 [64:48:57<22:29:52,  8.12s/it]
 74%|███████▎  | 27989/37962 [64:49:05<22:17:59,  8.05s/it]
 74%|███████▎  | 27990/37962 [64:49:13<22:26:38,  8.10s/it]
                                                           
{'loss': 1.2278, 'grad_norm': 5.198636054992676, 'learning_rate': 3.4059672449473922e-06, 'epoch': 0.74}

 74%|███████▎  | 27990/37962 [64:49:13<22:26:38,  8.10s/it]
 74%|███████▎  | 27991/37962 [64:49:21<22:32:51,  8.14s/it]
 74%|███████▎  | 27992/37962 [64:49:30<22:44:04,  8.21s/it]
 74%|███████▎  | 27993/37962 [64:49:38<22:40:44,  8.19s/it]
 74%|███████▎  | 27994/37962 [64:49:46<22:46:49,  8.23s/it]
 74%|███████▎  | 27995/37962 [64:49:55<22:54:26,  8.27s/it]
 74%|███████▎  | 27996/37962 [64:50:03<22:38:15,  8.18s/it]
 74%|███████▍  | 27997/37962 [64:50:11<22:49:15,  8.24s/it]
 74%|███████▍  | 27998/37962 [64:50:19<22:29:51,  8.13s/it]
 74%|███████▍  | 27999/37962 [64:50:27<22:39:37,  8.19s/it]
 74%|███████▍  | 28000/37962 [64:50:35<22:44:33,  8.22s/it]
                                                           
{'loss': 1.1354, 'grad_norm': 7.601644992828369, 'learning_rate': 3.3995556757805327e-06, 'epoch': 0.74}

 74%|███████▍  | 28000/37962 [64:50:35<22:44:33,  8.22s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 74%|███████▍  | 28001/37962 [64:51:25<56:53:53, 20.56s/it]
 74%|███████▍  | 28002/37962 [64:51:33<46:45:01, 16.90s/it]
 74%|███████▍  | 28003/37962 [64:51:42<39:46:53, 14.38s/it]
 74%|███████▍  | 28004/37962 [64:51:50<34:54:36, 12.62s/it]
 74%|███████▍  | 28005/37962 [64:51:58<31:13:10, 11.29s/it]
 74%|███████▍  | 28006/37962 [64:52:07<28:40:59, 10.37s/it]
 74%|███████▍  | 28007/37962 [64:52:15<27:02:48,  9.78s/it]
 74%|███████▍  | 28008/37962 [64:52:23<25:38:42,  9.27s/it]
 74%|███████▍  | 28009/37962 [64:52:32<25:05:07,  9.07s/it]
 74%|███████▍  | 28010/37962 [64:52:40<24:20:15,  8.80s/it]
                                                           
{'loss': 1.2167, 'grad_norm': 9.192791938781738, 'learning_rate': 3.393148910962751e-06, 'epoch': 0.74}

 74%|███████▍  | 28010/37962 [64:52:40<24:20:15,  8.80s/it]
 74%|███████▍  | 28011/37962 [64:52:48<23:50:16,  8.62s/it]
 74%|███████▍  | 28012/37962 [64:52:56<23:19:06,  8.44s/it]
 74%|███████▍  | 28013/37962 [64:53:04<23:14:02,  8.41s/it]
 74%|███████▍  | 28014/37962 [64:53:13<23:08:51,  8.38s/it]
 74%|███████▍  | 28015/37962 [64:53:21<22:56:51,  8.31s/it]
 74%|███████▍  | 28016/37962 [64:53:29<22:58:21,  8.32s/it]
 74%|███████▍  | 28017/37962 [64:53:37<22:57:12,  8.31s/it]
 74%|███████▍  | 28018/37962 [64:53:46<22:58:47,  8.32s/it]
 74%|███████▍  | 28019/37962 [64:53:54<22:44:12,  8.23s/it]
 74%|███████▍  | 28020/37962 [64:54:02<22:48:08,  8.26s/it]
                                                           
{'loss': 1.1059, 'grad_norm': 15.241044998168945, 'learning_rate': 3.3867469551574197e-06, 'epoch': 0.74}

 74%|███████▍  | 28020/37962 [64:54:02<22:48:08,  8.26s/it]
 74%|███████▍  | 28021/37962 [64:54:10<22:48:31,  8.26s/it]
 74%|███████▍  | 28022/37962 [64:54:19<22:55:39,  8.30s/it]
 74%|███████▍  | 28023/37962 [64:54:27<22:45:36,  8.24s/it]
 74%|███████▍  | 28024/37962 [64:54:35<23:01:38,  8.34s/it]
 74%|███████▍  | 28025/37962 [64:54:44<23:02:39,  8.35s/it]
 74%|███████▍  | 28026/37962 [64:54:52<23:09:34,  8.39s/it]
 74%|███████▍  | 28027/37962 [64:55:01<22:59:57,  8.33s/it]
 74%|███████▍  | 28028/37962 [64:55:09<22:54:40,  8.30s/it]
 74%|███████▍  | 28029/37962 [64:55:17<22:49:22,  8.27s/it]
 74%|███████▍  | 28030/37962 [64:55:25<22:51:20,  8.28s/it]
                                                           
{'loss': 1.2194, 'grad_norm': 239.47024536132812, 'learning_rate': 3.3803498130244105e-06, 'epoch': 0.74}

 74%|███████▍  | 28030/37962 [64:55:25<22:51:20,  8.28s/it]
 74%|███████▍  | 28031/37962 [64:55:34<22:49:21,  8.27s/it]
 74%|███████▍  | 28032/37962 [64:55:42<22:46:04,  8.25s/it]
 74%|███████▍  | 28033/37962 [64:55:50<22:54:02,  8.30s/it]
 74%|███████▍  | 28034/37962 [64:55:58<22:51:11,  8.29s/it]
 74%|███████▍  | 28035/37962 [64:56:06<22:39:14,  8.22s/it]
 74%|███████▍  | 28036/37962 [64:56:15<22:46:15,  8.26s/it]
 74%|███████▍  | 28037/37962 [64:56:23<22:49:28,  8.28s/it]
 74%|███████▍  | 28038/37962 [64:56:31<22:51:56,  8.29s/it]
 74%|███████▍  | 28039/37962 [64:56:40<22:53:41,  8.31s/it]
 74%|███████▍  | 28040/37962 [64:56:48<22:53:01,  8.30s/it]
                                                           
{'loss': 1.1658, 'grad_norm': 5.6883463859558105, 'learning_rate': 3.3739574892201e-06, 'epoch': 0.74}

 74%|███████▍  | 28040/37962 [64:56:48<22:53:01,  8.30s/it]
 74%|███████▍  | 28041/37962 [64:56:56<22:49:48,  8.28s/it]
 74%|███████▍  | 28042/37962 [64:57:04<22:41:39,  8.24s/it]
 74%|███████▍  | 28043/37962 [64:57:13<22:45:56,  8.26s/it]
 74%|███████▍  | 28044/37962 [64:57:21<22:22:07,  8.12s/it]
 74%|███████▍  | 28045/37962 [64:57:29<22:33:13,  8.19s/it]
 74%|███████▍  | 28046/37962 [64:57:37<22:31:56,  8.18s/it]
 74%|███████▍  | 28047/37962 [64:57:45<22:37:21,  8.21s/it]
 74%|███████▍  | 28048/37962 [64:57:54<22:40:14,  8.23s/it]
 74%|███████▍  | 28049/37962 [64:58:02<22:42:05,  8.24s/it]
 74%|███████▍  | 28050/37962 [64:58:10<22:25:29,  8.14s/it]
                                                           
{'loss': 1.2525, 'grad_norm': 9.623893737792969, 'learning_rate': 3.3675699883973433e-06, 'epoch': 0.74}

 74%|███████▍  | 28050/37962 [64:58:10<22:25:29,  8.14s/it]
 74%|███████▍  | 28051/37962 [64:58:18<22:15:54,  8.09s/it]
 74%|███████▍  | 28052/37962 [64:58:26<22:36:46,  8.21s/it]
 74%|███████▍  | 28053/37962 [64:58:35<22:47:54,  8.28s/it]
 74%|███████▍  | 28054/37962 [64:58:43<22:51:55,  8.31s/it]
 74%|███████▍  | 28055/37962 [64:58:51<22:43:41,  8.26s/it]
 74%|███████▍  | 28056/37962 [64:58:59<22:34:34,  8.20s/it]
 74%|███████▍  | 28057/37962 [64:59:08<22:45:05,  8.27s/it]
 74%|███████▍  | 28058/37962 [64:59:16<22:48:22,  8.29s/it]
 74%|███████▍  | 28059/37962 [64:59:24<22:33:19,  8.20s/it]
 74%|███████▍  | 28060/37962 [64:59:32<22:43:01,  8.26s/it]
                                                           
{'loss': 1.2901, 'grad_norm': 19.118925094604492, 'learning_rate': 3.361187315205491e-06, 'epoch': 0.74}

 74%|███████▍  | 28060/37962 [64:59:32<22:43:01,  8.26s/it]
 74%|███████▍  | 28061/37962 [64:59:41<23:00:57,  8.37s/it]
 74%|███████▍  | 28062/37962 [64:59:49<22:54:57,  8.33s/it]
 74%|███████▍  | 28063/37962 [64:59:58<22:56:46,  8.34s/it]
 74%|███████▍  | 28064/37962 [65:00:06<22:55:21,  8.34s/it]
 74%|███████▍  | 28065/37962 [65:00:14<22:49:20,  8.30s/it]
 74%|███████▍  | 28066/37962 [65:00:22<22:43:01,  8.26s/it]
 74%|███████▍  | 28067/37962 [65:00:31<22:40:20,  8.25s/it]
 74%|███████▍  | 28068/37962 [65:00:39<22:46:28,  8.29s/it]
 74%|███████▍  | 28069/37962 [65:00:47<22:31:31,  8.20s/it]
 74%|███████▍  | 28070/37962 [65:00:55<22:30:59,  8.19s/it]
                                                           
{'loss': 1.2783, 'grad_norm': 5.679615020751953, 'learning_rate': 3.354809474290385e-06, 'epoch': 0.74}

 74%|███████▍  | 28070/37962 [65:00:55<22:30:59,  8.19s/it]
 74%|███████▍  | 28071/37962 [65:01:03<22:14:46,  8.10s/it]
 74%|███████▍  | 28072/37962 [65:01:11<22:21:30,  8.14s/it]
 74%|███████▍  | 28073/37962 [65:01:20<22:33:31,  8.21s/it]
 74%|███████▍  | 28074/37962 [65:01:28<22:44:38,  8.28s/it]
 74%|███████▍  | 28075/37962 [65:01:36<22:43:55,  8.28s/it]
 74%|███████▍  | 28076/37962 [65:01:45<22:42:19,  8.27s/it]
 74%|███████▍  | 28077/37962 [65:01:53<22:44:26,  8.28s/it]
 74%|███████▍  | 28078/37962 [65:02:01<22:43:12,  8.28s/it]
 74%|███████▍  | 28079/37962 [65:02:10<22:44:16,  8.28s/it]
 74%|███████▍  | 28080/37962 [65:02:18<22:44:02,  8.28s/it]
                                                           
{'loss': 1.1817, 'grad_norm': 11.751688957214355, 'learning_rate': 3.3484364702943397e-06, 'epoch': 0.74}

 74%|███████▍  | 28080/37962 [65:02:18<22:44:02,  8.28s/it]
 74%|███████▍  | 28081/37962 [65:02:26<22:48:05,  8.31s/it]
 74%|███████▍  | 28082/37962 [65:02:35<22:57:49,  8.37s/it]
 74%|███████▍  | 28083/37962 [65:02:43<22:40:14,  8.26s/it]
 74%|███████▍  | 28084/37962 [65:02:51<22:50:51,  8.33s/it]
 74%|███████▍  | 28085/37962 [65:02:59<22:42:34,  8.28s/it]
 74%|███████▍  | 28086/37962 [65:03:08<22:48:56,  8.32s/it]
 74%|███████▍  | 28087/37962 [65:03:16<22:33:58,  8.23s/it]
 74%|███████▍  | 28088/37962 [65:03:24<22:28:38,  8.20s/it]
 74%|███████▍  | 28089/37962 [65:03:32<22:40:44,  8.27s/it]
 74%|███████▍  | 28090/37962 [65:03:41<22:48:39,  8.32s/it]
                                                           
{'loss': 1.1387, 'grad_norm': 8.829705238342285, 'learning_rate': 3.342068307856157e-06, 'epoch': 0.74}

 74%|███████▍  | 28090/37962 [65:03:41<22:48:39,  8.32s/it]
 74%|███████▍  | 28091/37962 [65:03:49<22:47:13,  8.31s/it]
 74%|███████▍  | 28092/37962 [65:03:57<22:36:03,  8.24s/it]
 74%|███████▍  | 28093/37962 [65:04:05<22:29:54,  8.21s/it]
 74%|███████▍  | 28094/37962 [65:04:14<22:35:18,  8.24s/it]
 74%|███████▍  | 28095/37962 [65:04:22<22:35:30,  8.24s/it]
 74%|███████▍  | 28096/37962 [65:04:30<22:33:31,  8.23s/it]
 74%|███████▍  | 28097/37962 [65:04:38<22:34:54,  8.24s/it]
 74%|███████▍  | 28098/37962 [65:04:47<22:56:38,  8.37s/it]
 74%|███████▍  | 28099/37962 [65:04:55<22:57:02,  8.38s/it]
 74%|███████▍  | 28100/37962 [65:05:04<22:51:07,  8.34s/it]
                                                           
{'loss': 1.1294, 'grad_norm': 10.12314510345459, 'learning_rate': 3.335704991611115e-06, 'epoch': 0.74}

 74%|███████▍  | 28100/37962 [65:05:04<22:51:07,  8.34s/it]
 74%|███████▍  | 28101/37962 [65:05:12<22:44:32,  8.30s/it]
 74%|███████▍  | 28102/37962 [65:05:20<22:47:08,  8.32s/it]
 74%|███████▍  | 28103/37962 [65:05:28<22:30:41,  8.22s/it]
 74%|███████▍  | 28104/37962 [65:05:36<22:14:57,  8.13s/it]
 74%|███████▍  | 28105/37962 [65:05:44<22:24:46,  8.19s/it]
 74%|███████▍  | 28106/37962 [65:05:52<22:16:45,  8.14s/it]
 74%|███████▍  | 28107/37962 [65:06:01<22:24:26,  8.19s/it]
 74%|███████▍  | 28108/37962 [65:06:09<22:25:07,  8.19s/it]
 74%|███████▍  | 28109/37962 [65:06:17<22:24:26,  8.19s/it]
 74%|███████▍  | 28110/37962 [65:06:25<22:20:56,  8.17s/it]
                                                           
{'loss': 1.2984, 'grad_norm': 17.57094955444336, 'learning_rate': 3.32934652619096e-06, 'epoch': 0.74}

 74%|███████▍  | 28110/37962 [65:06:25<22:20:56,  8.17s/it]
 74%|███████▍  | 28111/37962 [65:06:34<22:30:06,  8.22s/it]
 74%|███████▍  | 28112/37962 [65:06:42<22:38:15,  8.27s/it]
 74%|███████▍  | 28113/37962 [65:06:50<22:21:03,  8.17s/it]
 74%|███████▍  | 28114/37962 [65:06:58<22:22:34,  8.18s/it]
 74%|███████▍  | 28115/37962 [65:07:07<22:37:59,  8.27s/it]
 74%|███████▍  | 28116/37962 [65:07:15<22:33:56,  8.25s/it]
 74%|███████▍  | 28117/37962 [65:07:23<22:46:50,  8.33s/it]
 74%|███████▍  | 28118/37962 [65:07:32<22:43:16,  8.31s/it]
 74%|███████▍  | 28119/37962 [65:07:40<22:48:01,  8.34s/it]
 74%|███████▍  | 28120/37962 [65:07:48<22:52:18,  8.37s/it]
                                                           
{'loss': 1.2661, 'grad_norm': 4.487314224243164, 'learning_rate': 3.322992916223903e-06, 'epoch': 0.74}

 74%|███████▍  | 28120/37962 [65:07:48<22:52:18,  8.37s/it]
 74%|███████▍  | 28121/37962 [65:07:57<22:50:37,  8.36s/it]
 74%|███████▍  | 28122/37962 [65:08:05<22:45:29,  8.33s/it]
 74%|███████▍  | 28123/37962 [65:08:13<22:52:19,  8.37s/it]
 74%|███████▍  | 28124/37962 [65:08:22<22:44:24,  8.32s/it]
 74%|███████▍  | 28125/37962 [65:08:30<22:42:23,  8.31s/it]
 74%|███████▍  | 28126/37962 [65:08:38<22:48:35,  8.35s/it]
 74%|███████▍  | 28127/37962 [65:08:47<22:42:15,  8.31s/it]
 74%|███████▍  | 28128/37962 [65:08:55<22:41:42,  8.31s/it]
 74%|███████▍  | 28129/37962 [65:09:03<22:36:25,  8.28s/it]
 74%|███████▍  | 28130/37962 [65:09:12<22:46:01,  8.34s/it]
                                                           
{'loss': 1.2303, 'grad_norm': 6.947227954864502, 'learning_rate': 3.316644166334634e-06, 'epoch': 0.74}

 74%|███████▍  | 28130/37962 [65:09:12<22:46:01,  8.34s/it]
 74%|███████▍  | 28131/37962 [65:09:19<22:19:13,  8.17s/it]
 74%|███████▍  | 28132/37962 [65:09:28<22:29:23,  8.24s/it]
 74%|███████▍  | 28133/37962 [65:09:36<22:30:06,  8.24s/it]
 74%|███████▍  | 28134/37962 [65:09:44<22:30:20,  8.24s/it]
 74%|███████▍  | 28135/37962 [65:09:52<22:22:49,  8.20s/it]
 74%|███████▍  | 28136/37962 [65:10:01<22:38:30,  8.30s/it]
 74%|███████▍  | 28137/37962 [65:10:09<22:42:08,  8.32s/it]
 74%|███████▍  | 28138/37962 [65:10:17<22:36:16,  8.28s/it]
 74%|███████▍  | 28139/37962 [65:10:26<22:25:45,  8.22s/it]
 74%|███████▍  | 28140/37962 [65:10:34<22:33:24,  8.27s/it]
                                                           
{'loss': 1.2491, 'grad_norm': 7.996845245361328, 'learning_rate': 3.310300281144293e-06, 'epoch': 0.74}

 74%|███████▍  | 28140/37962 [65:10:34<22:33:24,  8.27s/it]
 74%|███████▍  | 28141/37962 [65:10:42<22:23:51,  8.21s/it]
 74%|███████▍  | 28142/37962 [65:10:50<22:25:31,  8.22s/it]
 74%|███████▍  | 28143/37962 [65:10:58<22:09:03,  8.12s/it]
 74%|███████▍  | 28144/37962 [65:11:06<22:10:46,  8.13s/it]
 74%|███████▍  | 28145/37962 [65:11:14<22:13:11,  8.15s/it]
 74%|███████▍  | 28146/37962 [65:11:23<22:24:26,  8.22s/it]
 74%|███████▍  | 28147/37962 [65:11:31<22:32:56,  8.27s/it]
 74%|███████▍  | 28148/37962 [65:11:39<22:31:50,  8.26s/it]
 74%|███████▍  | 28149/37962 [65:11:48<22:33:49,  8.28s/it]
 74%|███████▍  | 28150/37962 [65:11:56<22:31:00,  8.26s/it]
                                                           
{'loss': 1.1199, 'grad_norm': 12.407132148742676, 'learning_rate': 3.303961265270489e-06, 'epoch': 0.74}

 74%|███████▍  | 28150/37962 [65:11:56<22:31:00,  8.26s/it]
 74%|███████▍  | 28151/37962 [65:12:04<22:36:23,  8.30s/it]
 74%|███████▍  | 28152/37962 [65:12:12<22:26:21,  8.23s/it]
 74%|███████▍  | 28153/37962 [65:12:21<22:32:30,  8.27s/it]
 74%|███████▍  | 28154/37962 [65:12:29<22:34:41,  8.29s/it]
 74%|███████▍  | 28155/37962 [65:12:37<22:34:06,  8.28s/it]
 74%|███████▍  | 28156/37962 [65:12:46<22:29:38,  8.26s/it]
 74%|███████▍  | 28157/37962 [65:12:54<22:16:11,  8.18s/it]
 74%|███████▍  | 28158/37962 [65:13:02<22:26:14,  8.24s/it]
 74%|███████▍  | 28159/37962 [65:13:10<22:28:25,  8.25s/it]
 74%|███████▍  | 28160/37962 [65:13:19<22:27:50,  8.25s/it]
                                                           
{'loss': 1.0999, 'grad_norm': 11.668746948242188, 'learning_rate': 3.2976271233272794e-06, 'epoch': 0.74}

 74%|███████▍  | 28160/37962 [65:13:19<22:27:50,  8.25s/it]
 74%|███████▍  | 28161/37962 [65:13:27<22:34:56,  8.29s/it]
 74%|███████▍  | 28162/37962 [65:13:43<28:50:50, 10.60s/it]
 74%|███████▍  | 28163/37962 [65:13:51<26:58:19,  9.91s/it]
 74%|███████▍  | 28164/37962 [65:13:59<25:36:14,  9.41s/it]
 74%|███████▍  | 28165/37962 [65:14:08<24:38:48,  9.06s/it]
 74%|███████▍  | 28166/37962 [65:14:16<24:04:56,  8.85s/it]
 74%|███████▍  | 28167/37962 [65:14:24<23:43:59,  8.72s/it]
 74%|███████▍  | 28168/37962 [65:14:33<23:28:33,  8.63s/it]
 74%|███████▍  | 28169/37962 [65:14:41<23:16:13,  8.55s/it]
 74%|███████▍  | 28170/37962 [65:14:50<23:21:19,  8.59s/it]
                                                           
{'loss': 1.213, 'grad_norm': 16.312633514404297, 'learning_rate': 3.2912978599251723e-06, 'epoch': 0.74}

 74%|███████▍  | 28170/37962 [65:14:50<23:21:19,  8.59s/it]
 74%|███████▍  | 28171/37962 [65:14:58<22:57:03,  8.44s/it]
 74%|███████▍  | 28172/37962 [65:15:14<28:52:04, 10.62s/it]
 74%|███████▍  | 28173/37962 [65:15:22<26:54:32,  9.90s/it]
 74%|███████▍  | 28174/37962 [65:15:30<25:28:38,  9.37s/it]
 74%|███████▍  | 28175/37962 [65:15:38<24:35:26,  9.05s/it]
 74%|███████▍  | 28176/37962 [65:15:47<23:55:53,  8.80s/it]
 74%|███████▍  | 28177/37962 [65:15:55<23:37:20,  8.69s/it]
 74%|███████▍  | 28178/37962 [65:16:03<23:15:27,  8.56s/it]
 74%|███████▍  | 28179/37962 [65:16:12<22:59:47,  8.46s/it]
 74%|███████▍  | 28180/37962 [65:16:20<22:48:10,  8.39s/it]
                                                           
{'loss': 1.1734, 'grad_norm': 9.091997146606445, 'learning_rate': 3.284973479671135e-06, 'epoch': 0.74}

 74%|███████▍  | 28180/37962 [65:16:20<22:48:10,  8.39s/it]
 74%|███████▍  | 28181/37962 [65:16:28<22:27:06,  8.26s/it]
 74%|███████▍  | 28182/37962 [65:16:36<22:32:45,  8.30s/it]
 74%|███████▍  | 28183/37962 [65:16:44<22:29:43,  8.28s/it]
 74%|███████▍  | 28184/37962 [65:16:53<22:29:24,  8.28s/it]
 74%|███████▍  | 28185/37962 [65:17:01<22:34:26,  8.31s/it]
 74%|███████▍  | 28186/37962 [65:17:09<22:32:16,  8.30s/it]
 74%|███████▍  | 28187/37962 [65:17:17<22:22:29,  8.24s/it]
 74%|███████▍  | 28188/37962 [65:17:26<22:32:11,  8.30s/it]
 74%|███████▍  | 28189/37962 [65:17:34<22:32:50,  8.31s/it]
 74%|███████▍  | 28190/37962 [65:17:42<22:32:21,  8.30s/it]
                                                           
{'loss': 1.1941, 'grad_norm': 10.747648239135742, 'learning_rate': 3.2786539871685707e-06, 'epoch': 0.74}

 74%|███████▍  | 28190/37962 [65:17:42<22:32:21,  8.30s/it]
 74%|███████▍  | 28191/37962 [65:17:50<22:16:59,  8.21s/it]
 74%|███████▍  | 28192/37962 [65:17:59<22:18:26,  8.22s/it]
 74%|███████▍  | 28193/37962 [65:18:07<22:05:51,  8.14s/it]
 74%|███████▍  | 28194/37962 [65:18:15<21:53:36,  8.07s/it]
 74%|███████▍  | 28195/37962 [65:18:23<22:00:38,  8.11s/it]
 74%|███████▍  | 28196/37962 [65:18:31<22:10:08,  8.17s/it]
 74%|███████▍  | 28197/37962 [65:18:39<22:17:14,  8.22s/it]
 74%|███████▍  | 28198/37962 [65:18:48<22:17:03,  8.22s/it]
 74%|███████▍  | 28199/37962 [65:18:56<22:16:48,  8.22s/it]
 74%|███████▍  | 28200/37962 [65:19:04<22:21:44,  8.25s/it]
                                                           
{'loss': 1.1475, 'grad_norm': 13.304875373840332, 'learning_rate': 3.2723393870173293e-06, 'epoch': 0.74}

 74%|███████▍  | 28200/37962 [65:19:04<22:21:44,  8.25s/it]
 74%|███████▍  | 28201/37962 [65:19:12<22:26:33,  8.28s/it]
 74%|███████▍  | 28202/37962 [65:19:21<22:28:25,  8.29s/it]
 74%|███████▍  | 28203/37962 [65:19:29<22:36:53,  8.34s/it]
 74%|███████▍  | 28204/37962 [65:19:37<22:31:07,  8.31s/it]
 74%|███████▍  | 28205/37962 [65:19:46<22:17:33,  8.23s/it]
 74%|███████▍  | 28206/37962 [65:19:54<22:22:10,  8.25s/it]
 74%|███████▍  | 28207/37962 [65:20:02<22:22:36,  8.26s/it]
 74%|███████▍  | 28208/37962 [65:20:10<22:19:21,  8.24s/it]
 74%|███████▍  | 28209/37962 [65:20:19<22:22:58,  8.26s/it]
 74%|███████▍  | 28210/37962 [65:20:27<22:38:51,  8.36s/it]
                                                           
{'loss': 1.2475, 'grad_norm': 23.057933807373047, 'learning_rate': 3.266029683813705e-06, 'epoch': 0.74}

 74%|███████▍  | 28210/37962 [65:20:27<22:38:51,  8.36s/it]
 74%|███████▍  | 28211/37962 [65:20:35<22:27:35,  8.29s/it]
 74%|███████▍  | 28212/37962 [65:20:44<22:25:11,  8.28s/it]
 74%|███████▍  | 28213/37962 [65:20:52<22:23:40,  8.27s/it]
 74%|███████▍  | 28214/37962 [65:21:00<22:31:23,  8.32s/it]
 74%|███████▍  | 28215/37962 [65:21:09<22:29:12,  8.31s/it]
 74%|███████▍  | 28216/37962 [65:21:17<22:23:21,  8.27s/it]
 74%|███████▍  | 28217/37962 [65:21:25<22:21:47,  8.26s/it]
 74%|███████▍  | 28218/37962 [65:21:33<22:11:41,  8.20s/it]
 74%|███████▍  | 28219/37962 [65:21:41<22:21:38,  8.26s/it]
 74%|███████▍  | 28220/37962 [65:21:50<22:17:39,  8.24s/it]
                                                           
{'loss': 1.209, 'grad_norm': 17.446489334106445, 'learning_rate': 3.2597248821504103e-06, 'epoch': 0.74}

 74%|███████▍  | 28220/37962 [65:21:50<22:17:39,  8.24s/it]
 74%|███████▍  | 28221/37962 [65:21:58<22:20:20,  8.26s/it]
 74%|███████▍  | 28222/37962 [65:22:06<22:31:06,  8.32s/it]
 74%|███████▍  | 28223/37962 [65:22:15<22:29:11,  8.31s/it]
 74%|███████▍  | 28224/37962 [65:22:23<22:27:06,  8.30s/it]
 74%|███████▍  | 28225/37962 [65:22:31<22:20:14,  8.26s/it]
 74%|███████▍  | 28226/37962 [65:22:39<22:23:36,  8.28s/it]
 74%|███████▍  | 28227/37962 [65:22:48<22:27:19,  8.30s/it]
 74%|███████▍  | 28228/37962 [65:22:56<22:29:28,  8.32s/it]
 74%|███████▍  | 28229/37962 [65:23:04<22:30:05,  8.32s/it]
 74%|███████▍  | 28230/37962 [65:23:13<22:25:29,  8.30s/it]
                                                           
{'loss': 1.2108, 'grad_norm': 12.732297897338867, 'learning_rate': 3.2534249866166055e-06, 'epoch': 0.74}

 74%|███████▍  | 28230/37962 [65:23:13<22:25:29,  8.30s/it]
 74%|███████▍  | 28231/37962 [65:23:21<22:35:22,  8.36s/it]
 74%|███████▍  | 28232/37962 [65:23:29<22:30:02,  8.33s/it]
 74%|███████▍  | 28233/37962 [65:23:38<22:24:44,  8.29s/it]
 74%|███████▍  | 28234/37962 [65:23:46<22:15:03,  8.23s/it]
 74%|███████▍  | 28235/37962 [65:23:54<22:29:23,  8.32s/it]
 74%|███████▍  | 28236/37962 [65:24:03<22:30:27,  8.33s/it]
 74%|███████▍  | 28237/37962 [65:24:11<22:16:36,  8.25s/it]
 74%|███████▍  | 28238/37962 [65:24:19<22:15:24,  8.24s/it]
 74%|███████▍  | 28239/37962 [65:24:27<22:06:56,  8.19s/it]
 74%|███████▍  | 28240/37962 [65:24:35<22:12:37,  8.22s/it]
                                                           
{'loss': 1.0834, 'grad_norm': 10.92783260345459, 'learning_rate': 3.24713000179788e-06, 'epoch': 0.74}

 74%|███████▍  | 28240/37962 [65:24:35<22:12:37,  8.22s/it]
 74%|███████▍  | 28241/37962 [65:24:44<22:14:46,  8.24s/it]
 74%|███████▍  | 28242/37962 [65:24:52<22:33:23,  8.35s/it]
 74%|███████▍  | 28243/37962 [65:25:00<22:06:51,  8.19s/it]
 74%|███████▍  | 28244/37962 [65:25:08<22:13:11,  8.23s/it]
 74%|███████▍  | 28245/37962 [65:25:17<22:17:40,  8.26s/it]
 74%|███████▍  | 28246/37962 [65:25:25<22:18:34,  8.27s/it]
 74%|███████▍  | 28247/37962 [65:25:33<22:14:45,  8.24s/it]
 74%|███████▍  | 28248/37962 [65:25:41<22:17:28,  8.26s/it]
 74%|███████▍  | 28249/37962 [65:25:50<22:12:13,  8.23s/it]
 74%|███████▍  | 28250/37962 [65:25:58<22:12:52,  8.23s/it]
                                                           
{'loss': 1.1568, 'grad_norm': 6.13063383102417, 'learning_rate': 3.2408399322762386e-06, 'epoch': 0.74}

 74%|███████▍  | 28250/37962 [65:25:58<22:12:52,  8.23s/it]
 74%|███████▍  | 28251/37962 [65:26:06<22:23:24,  8.30s/it]
 74%|███████▍  | 28252/37962 [65:26:14<22:13:51,  8.24s/it]
 74%|███████▍  | 28253/37962 [65:26:23<22:19:25,  8.28s/it]
 74%|███████▍  | 28254/37962 [65:26:31<22:18:38,  8.27s/it]
 74%|███████▍  | 28255/37962 [65:26:39<22:17:16,  8.27s/it]
 74%|███████▍  | 28256/37962 [65:26:47<21:57:32,  8.14s/it]
 74%|███████▍  | 28257/37962 [65:26:56<22:07:32,  8.21s/it]
 74%|███████▍  | 28258/37962 [65:27:04<22:06:02,  8.20s/it]
 74%|███████▍  | 28259/37962 [65:27:12<22:06:22,  8.20s/it]
 74%|███████▍  | 28260/37962 [65:27:20<22:14:57,  8.26s/it]
                                                           
{'loss': 1.2044, 'grad_norm': 5.784814834594727, 'learning_rate': 3.2345547826301184e-06, 'epoch': 0.74}

 74%|███████▍  | 28260/37962 [65:27:20<22:14:57,  8.26s/it]
 74%|███████▍  | 28261/37962 [65:27:29<22:13:22,  8.25s/it]
 74%|███████▍  | 28262/37962 [65:27:37<22:18:03,  8.28s/it]
 74%|███████▍  | 28263/37962 [65:27:45<22:11:07,  8.23s/it]
 74%|███████▍  | 28264/37962 [65:27:53<22:11:42,  8.24s/it]
 74%|███████▍  | 28265/37962 [65:28:02<22:19:35,  8.29s/it]
 74%|███████▍  | 28266/37962 [65:28:10<22:08:53,  8.22s/it]
 74%|███████▍  | 28267/37962 [65:28:18<22:12:52,  8.25s/it]
 74%|███████▍  | 28268/37962 [65:28:26<22:13:17,  8.25s/it]
 74%|███████▍  | 28269/37962 [65:28:34<22:07:37,  8.22s/it]
 74%|███████▍  | 28270/37962 [65:28:43<22:03:44,  8.19s/it]
                                                           
{'loss': 1.2105, 'grad_norm': 9.758633613586426, 'learning_rate': 3.2282745574343687e-06, 'epoch': 0.74}

 74%|███████▍  | 28270/37962 [65:28:43<22:03:44,  8.19s/it]
 74%|███████▍  | 28271/37962 [65:28:51<22:11:43,  8.25s/it]
 74%|███████▍  | 28272/37962 [65:28:59<22:15:20,  8.27s/it]
 74%|███████▍  | 28273/37962 [65:29:08<22:26:31,  8.34s/it]
 74%|███████▍  | 28274/37962 [65:29:16<22:06:14,  8.21s/it]
 74%|███████▍  | 28275/37962 [65:29:24<22:15:39,  8.27s/it]
 74%|███████▍  | 28276/37962 [65:29:32<22:10:59,  8.24s/it]
 74%|███████▍  | 28277/37962 [65:29:41<22:21:27,  8.31s/it]
 74%|███████▍  | 28278/37962 [65:29:49<22:19:56,  8.30s/it]
 74%|███████▍  | 28279/37962 [65:29:57<22:28:51,  8.36s/it]
 74%|███████▍  | 28280/37962 [65:30:06<22:22:43,  8.32s/it]
                                                           
{'loss': 1.1666, 'grad_norm': 5.850472927093506, 'learning_rate': 3.2219992612602534e-06, 'epoch': 0.74}

 74%|███████▍  | 28280/37962 [65:30:06<22:22:43,  8.32s/it]
 74%|███████▍  | 28281/37962 [65:30:13<21:55:58,  8.16s/it]
 75%|███████▍  | 28282/37962 [65:30:22<22:07:53,  8.23s/it]
 75%|███████▍  | 28283/37962 [65:30:30<22:16:00,  8.28s/it]
 75%|███████▍  | 28284/37962 [65:30:39<22:22:29,  8.32s/it]
 75%|███████▍  | 28285/37962 [65:30:47<22:16:39,  8.29s/it]
 75%|███████▍  | 28286/37962 [65:30:55<22:27:35,  8.36s/it]
 75%|███████▍  | 28287/37962 [65:31:04<22:14:55,  8.28s/it]
 75%|███████▍  | 28288/37962 [65:31:12<22:15:08,  8.28s/it]
 75%|███████▍  | 28289/37962 [65:31:20<21:56:02,  8.16s/it]
 75%|███████▍  | 28290/37962 [65:31:28<21:44:30,  8.09s/it]
                                                           
{'loss': 1.1458, 'grad_norm': 12.295234680175781, 'learning_rate': 3.2157288986754575e-06, 'epoch': 0.75}

 75%|███████▍  | 28290/37962 [65:31:28<21:44:30,  8.09s/it]
 75%|███████▍  | 28291/37962 [65:31:36<22:01:02,  8.20s/it]
 75%|███████▍  | 28292/37962 [65:31:44<22:01:02,  8.20s/it]
 75%|███████▍  | 28293/37962 [65:31:53<22:03:47,  8.21s/it]
 75%|███████▍  | 28294/37962 [65:32:01<22:01:13,  8.20s/it]
 75%|███████▍  | 28295/37962 [65:32:09<22:09:30,  8.25s/it]
 75%|███████▍  | 28296/37962 [65:32:17<22:07:56,  8.24s/it]
 75%|███████▍  | 28297/37962 [65:32:25<22:04:52,  8.22s/it]
 75%|███████▍  | 28298/37962 [65:32:34<22:11:57,  8.27s/it]
 75%|███████▍  | 28299/37962 [65:32:42<22:08:28,  8.25s/it]
 75%|███████▍  | 28300/37962 [65:32:50<21:45:41,  8.11s/it]
                                                           
{'loss': 1.0723, 'grad_norm': 14.055192947387695, 'learning_rate': 3.2094634742440643e-06, 'epoch': 0.75}

 75%|███████▍  | 28300/37962 [65:32:50<21:45:41,  8.11s/it]
 75%|███████▍  | 28301/37962 [65:32:58<21:59:24,  8.19s/it]
 75%|███████▍  | 28302/37962 [65:33:07<22:11:14,  8.27s/it]
 75%|███████▍  | 28303/37962 [65:33:15<22:09:00,  8.26s/it]
 75%|███████▍  | 28304/37962 [65:33:23<21:52:47,  8.16s/it]
 75%|███████▍  | 28305/37962 [65:33:31<21:56:04,  8.18s/it]
 75%|███████▍  | 28306/37962 [65:33:39<21:58:48,  8.19s/it]
 75%|███████▍  | 28307/37962 [65:33:47<21:58:35,  8.19s/it]
 75%|███████▍  | 28308/37962 [65:33:56<22:02:41,  8.22s/it]
 75%|███████▍  | 28309/37962 [65:34:04<22:15:46,  8.30s/it]
 75%|███████▍  | 28310/37962 [65:34:12<22:05:27,  8.24s/it]
                                                           
{'loss': 1.2262, 'grad_norm': 7.126120090484619, 'learning_rate': 3.2032029925265707e-06, 'epoch': 0.75}

 75%|███████▍  | 28310/37962 [65:34:12<22:05:27,  8.24s/it]
 75%|███████▍  | 28311/37962 [65:34:21<22:13:20,  8.29s/it]
 75%|███████▍  | 28312/37962 [65:34:29<22:02:48,  8.22s/it]
 75%|███████▍  | 28313/37962 [65:34:37<21:59:54,  8.21s/it]
 75%|███████▍  | 28314/37962 [65:34:45<22:04:19,  8.24s/it]
 75%|███████▍  | 28315/37962 [65:34:54<22:05:48,  8.25s/it]
 75%|███████▍  | 28316/37962 [65:35:02<22:11:24,  8.28s/it]
 75%|███████▍  | 28317/37962 [65:35:10<22:06:32,  8.25s/it]
 75%|███████▍  | 28318/37962 [65:35:18<22:05:38,  8.25s/it]
 75%|███████▍  | 28319/37962 [65:35:26<21:56:41,  8.19s/it]
 75%|███████▍  | 28320/37962 [65:35:35<22:10:35,  8.28s/it]
                                                           
{'loss': 1.2049, 'grad_norm': 10.581390380859375, 'learning_rate': 3.1969474580798755e-06, 'epoch': 0.75}

 75%|███████▍  | 28320/37962 [65:35:35<22:10:35,  8.28s/it]
 75%|███████▍  | 28321/37962 [65:35:43<22:16:20,  8.32s/it]
 75%|███████▍  | 28322/37962 [65:35:52<22:20:35,  8.34s/it]
 75%|███████▍  | 28323/37962 [65:36:00<22:00:46,  8.22s/it]
 75%|███████▍  | 28324/37962 [65:36:08<22:01:05,  8.22s/it]
 75%|███████▍  | 28325/37962 [65:36:16<21:44:56,  8.12s/it]
 75%|███████▍  | 28326/37962 [65:36:24<21:50:00,  8.16s/it]
 75%|███████▍  | 28327/37962 [65:36:32<21:47:02,  8.14s/it]
 75%|███████▍  | 28328/37962 [65:36:40<21:51:34,  8.17s/it]
 75%|███████▍  | 28329/37962 [65:36:49<21:58:25,  8.21s/it]
 75%|███████▍  | 28330/37962 [65:36:57<21:51:07,  8.17s/it]
                                                           
{'loss': 1.3099, 'grad_norm': 91.3353042602539, 'learning_rate': 3.19069687545727e-06, 'epoch': 0.75}

 75%|███████▍  | 28330/37962 [65:36:57<21:51:07,  8.17s/it]
 75%|███████▍  | 28331/37962 [65:37:05<21:58:32,  8.21s/it]
 75%|███████▍  | 28332/37962 [65:37:13<21:55:20,  8.20s/it]
 75%|███████▍  | 28333/37962 [65:37:21<21:56:57,  8.21s/it]
 75%|███████▍  | 28334/37962 [65:37:30<21:56:46,  8.21s/it]
 75%|███████▍  | 28335/37962 [65:37:38<22:03:11,  8.25s/it]
 75%|███████▍  | 28336/37962 [65:37:46<21:47:46,  8.15s/it]
 75%|███████▍  | 28337/37962 [65:37:54<21:54:47,  8.20s/it]
 75%|███████▍  | 28338/37962 [65:38:02<21:54:18,  8.19s/it]
 75%|███████▍  | 28339/37962 [65:38:11<22:01:44,  8.24s/it]
 75%|███████▍  | 28340/37962 [65:38:19<21:57:20,  8.21s/it]
                                                           
{'loss': 1.2608, 'grad_norm': 8.80888843536377, 'learning_rate': 3.184451249208443e-06, 'epoch': 0.75}

 75%|███████▍  | 28340/37962 [65:38:19<21:57:20,  8.21s/it]
 75%|███████▍  | 28341/37962 [65:38:27<22:00:32,  8.24s/it]
 75%|███████▍  | 28342/37962 [65:38:35<22:01:30,  8.24s/it]
 75%|███████▍  | 28343/37962 [65:38:44<21:55:18,  8.20s/it]
 75%|███████▍  | 28344/37962 [65:38:51<21:41:15,  8.12s/it]
 75%|███████▍  | 28345/37962 [65:39:00<21:49:42,  8.17s/it]
 75%|███████▍  | 28346/37962 [65:39:08<21:57:24,  8.22s/it]
 75%|███████▍  | 28347/37962 [65:39:17<22:08:28,  8.29s/it]
 75%|███████▍  | 28348/37962 [65:39:25<22:06:49,  8.28s/it]
 75%|███████▍  | 28349/37962 [65:39:33<22:15:19,  8.33s/it]
 75%|███████▍  | 28350/37962 [65:39:42<22:12:19,  8.32s/it]
                                                           
{'loss': 1.1929, 'grad_norm': 8.281060218811035, 'learning_rate': 3.178210583879483e-06, 'epoch': 0.75}

 75%|███████▍  | 28350/37962 [65:39:42<22:12:19,  8.32s/it]
 75%|███████▍  | 28351/37962 [65:39:50<22:11:41,  8.31s/it]
 75%|███████▍  | 28352/37962 [65:39:58<22:13:53,  8.33s/it]
 75%|███████▍  | 28353/37962 [65:40:07<22:12:33,  8.32s/it]
 75%|███████▍  | 28354/37962 [65:40:15<22:10:21,  8.31s/it]
 75%|███████▍  | 28355/37962 [65:40:23<22:09:50,  8.31s/it]
 75%|███████▍  | 28356/37962 [65:40:31<22:10:06,  8.31s/it]
 75%|███████▍  | 28357/37962 [65:40:39<21:54:57,  8.21s/it]
 75%|███████▍  | 28358/37962 [65:40:48<21:59:49,  8.25s/it]
 75%|███████▍  | 28359/37962 [65:40:56<22:02:14,  8.26s/it]
 75%|███████▍  | 28360/37962 [65:41:04<22:04:43,  8.28s/it]
                                                           
{'loss': 1.2491, 'grad_norm': 29.88004493713379, 'learning_rate': 3.1719748840128574e-06, 'epoch': 0.75}

 75%|███████▍  | 28360/37962 [65:41:04<22:04:43,  8.28s/it]
 75%|███████▍  | 28361/37962 [65:41:12<21:51:24,  8.20s/it]
 75%|███████▍  | 28362/37962 [65:41:21<21:56:58,  8.23s/it]
 75%|███████▍  | 28363/37962 [65:41:28<21:36:55,  8.11s/it]
 75%|███████▍  | 28364/37962 [65:41:37<21:46:14,  8.17s/it]
 75%|███████▍  | 28365/37962 [65:41:45<21:50:41,  8.19s/it]
 75%|███████▍  | 28366/37962 [65:41:53<22:02:40,  8.27s/it]
 75%|███████▍  | 28367/37962 [65:42:02<22:06:57,  8.30s/it]
 75%|███████▍  | 28368/37962 [65:42:10<22:05:39,  8.29s/it]
 75%|███████▍  | 28369/37962 [65:42:18<21:48:45,  8.19s/it]
 75%|███████▍  | 28370/37962 [65:42:26<21:53:02,  8.21s/it]
                                                           
{'loss': 1.1833, 'grad_norm': 4.409859657287598, 'learning_rate': 3.165744154147429e-06, 'epoch': 0.75}

 75%|███████▍  | 28370/37962 [65:42:26<21:53:02,  8.21s/it]
 75%|███████▍  | 28371/37962 [65:42:35<22:02:51,  8.28s/it]
 75%|███████▍  | 28372/37962 [65:42:43<21:59:01,  8.25s/it]
 75%|███████▍  | 28373/37962 [65:42:51<22:08:09,  8.31s/it]
 75%|███████▍  | 28374/37962 [65:43:00<22:08:02,  8.31s/it]
 75%|███████▍  | 28375/37962 [65:43:08<22:11:17,  8.33s/it]
 75%|███████▍  | 28376/37962 [65:43:16<22:09:08,  8.32s/it]
 75%|███████▍  | 28377/37962 [65:43:25<22:05:14,  8.30s/it]
 75%|███████▍  | 28378/37962 [65:43:33<22:08:13,  8.32s/it]
 75%|███████▍  | 28379/37962 [65:43:41<22:11:21,  8.34s/it]
 75%|███████▍  | 28380/37962 [65:43:49<22:01:33,  8.28s/it]
                                                           
{'loss': 1.2773, 'grad_norm': 5.630708694458008, 'learning_rate': 3.1595183988184354e-06, 'epoch': 0.75}

 75%|███████▍  | 28380/37962 [65:43:49<22:01:33,  8.28s/it]
 75%|███████▍  | 28381/37962 [65:43:58<22:03:22,  8.29s/it]
 75%|███████▍  | 28382/37962 [65:44:06<22:03:03,  8.29s/it]
 75%|███████▍  | 28383/37962 [65:44:14<22:02:53,  8.29s/it]
 75%|███████▍  | 28384/37962 [65:44:22<21:46:00,  8.18s/it]
 75%|███████▍  | 28385/37962 [65:44:31<21:49:19,  8.20s/it]
 75%|███████▍  | 28386/37962 [65:44:39<21:47:42,  8.19s/it]
 75%|███████▍  | 28387/37962 [65:44:47<21:53:02,  8.23s/it]
 75%|███████▍  | 28388/37962 [65:44:55<21:49:21,  8.21s/it]
 75%|███████▍  | 28389/37962 [65:45:04<22:00:18,  8.28s/it]
 75%|███████▍  | 28390/37962 [65:45:12<22:03:52,  8.30s/it]
                                                           
{'loss': 1.0849, 'grad_norm': 8.95604133605957, 'learning_rate': 3.153297622557493e-06, 'epoch': 0.75}

 75%|███████▍  | 28390/37962 [65:45:12<22:03:52,  8.30s/it]
 75%|███████▍  | 28391/37962 [65:45:20<22:09:06,  8.33s/it]
 75%|███████▍  | 28392/37962 [65:45:29<22:04:09,  8.30s/it]
 75%|███████▍  | 28393/37962 [65:45:37<22:03:08,  8.30s/it]
 75%|███████▍  | 28394/37962 [65:45:45<21:58:48,  8.27s/it]
 75%|███████▍  | 28395/37962 [65:45:53<21:55:52,  8.25s/it]
 75%|███████▍  | 28396/37962 [65:46:02<22:03:05,  8.30s/it]
 75%|███████▍  | 28397/37962 [65:46:10<21:59:37,  8.28s/it]
 75%|███████▍  | 28398/37962 [65:46:18<21:50:16,  8.22s/it]
 75%|███████▍  | 28399/37962 [65:46:26<21:52:42,  8.24s/it]
 75%|███████▍  | 28400/37962 [65:46:35<21:51:35,  8.23s/it]
                                                           
{'loss': 1.1724, 'grad_norm': 11.138555526733398, 'learning_rate': 3.1470818298926e-06, 'epoch': 0.75}

 75%|███████▍  | 28400/37962 [65:46:35<21:51:35,  8.23s/it]
 75%|███████▍  | 28401/37962 [65:46:42<21:35:23,  8.13s/it]
 75%|███████▍  | 28402/37962 [65:46:51<21:41:22,  8.17s/it]
 75%|███████▍  | 28403/37962 [65:46:59<21:45:11,  8.19s/it]
 75%|███████▍  | 28404/37962 [65:47:07<22:00:07,  8.29s/it]
 75%|███████▍  | 28405/37962 [65:47:16<21:56:47,  8.27s/it]
 75%|███████▍  | 28406/37962 [65:47:24<21:51:20,  8.23s/it]
 75%|███████▍  | 28407/37962 [65:47:32<21:49:49,  8.22s/it]
 75%|███████▍  | 28408/37962 [65:47:40<21:52:35,  8.24s/it]
 75%|███████▍  | 28409/37962 [65:47:49<21:50:20,  8.23s/it]
 75%|███████▍  | 28410/37962 [65:47:57<21:56:08,  8.27s/it]
                                                           
{'loss': 1.2353, 'grad_norm': 9.953782081604004, 'learning_rate': 3.1408710253481277e-06, 'epoch': 0.75}

 75%|███████▍  | 28410/37962 [65:47:57<21:56:08,  8.27s/it]
 75%|███████▍  | 28411/37962 [65:48:12<27:34:51, 10.40s/it]
 75%|███████▍  | 28412/37962 [65:48:21<26:01:02,  9.81s/it]
 75%|███████▍  | 28413/37962 [65:48:29<24:44:07,  9.33s/it]
 75%|███████▍  | 28414/37962 [65:48:37<23:45:31,  8.96s/it]
 75%|███████▍  | 28415/37962 [65:48:45<23:12:47,  8.75s/it]
 75%|███████▍  | 28416/37962 [65:48:54<22:49:06,  8.61s/it]
 75%|███████▍  | 28417/37962 [65:49:02<22:33:30,  8.51s/it]
 75%|███████▍  | 28418/37962 [65:49:10<22:19:10,  8.42s/it]
 75%|███████▍  | 28419/37962 [65:49:18<22:16:03,  8.40s/it]
 75%|███████▍  | 28420/37962 [65:49:26<22:01:05,  8.31s/it]
                                                           
{'loss': 1.2747, 'grad_norm': 7.278004169464111, 'learning_rate': 3.1346652134448098e-06, 'epoch': 0.75}

 75%|███████▍  | 28420/37962 [65:49:26<22:01:05,  8.31s/it]
 75%|███████▍  | 28421/37962 [65:49:34<21:43:01,  8.19s/it]
 75%|███████▍  | 28422/37962 [65:49:43<21:51:03,  8.25s/it]
 75%|███████▍  | 28423/37962 [65:49:51<21:48:39,  8.23s/it]
 75%|███████▍  | 28424/37962 [65:49:59<21:38:10,  8.17s/it]
 75%|███████▍  | 28425/37962 [65:50:07<21:41:30,  8.19s/it]
 75%|███████▍  | 28426/37962 [65:50:15<21:38:45,  8.17s/it]
 75%|███████▍  | 28427/37962 [65:50:24<21:45:19,  8.21s/it]
 75%|███████▍  | 28428/37962 [65:50:32<21:49:43,  8.24s/it]
 75%|███████▍  | 28429/37962 [65:50:40<21:53:34,  8.27s/it]
 75%|███████▍  | 28430/37962 [65:50:48<21:29:25,  8.12s/it]
                                                           
{'loss': 1.3521, 'grad_norm': 7.015289306640625, 'learning_rate': 3.1284643986997454e-06, 'epoch': 0.75}

 75%|███████▍  | 28430/37962 [65:50:48<21:29:25,  8.12s/it]
 75%|███████▍  | 28431/37962 [65:50:56<21:26:10,  8.10s/it]
 75%|███████▍  | 28432/37962 [65:51:04<21:14:42,  8.03s/it]
 75%|███████▍  | 28433/37962 [65:51:12<21:20:28,  8.06s/it]
 75%|███████▍  | 28434/37962 [65:51:20<21:32:13,  8.14s/it]
 75%|███████▍  | 28435/37962 [65:51:29<21:50:31,  8.25s/it]
 75%|███████▍  | 28436/37962 [65:51:37<21:37:21,  8.17s/it]
 75%|███████▍  | 28437/37962 [65:51:45<21:41:26,  8.20s/it]
 75%|███████▍  | 28438/37962 [65:51:53<21:44:36,  8.22s/it]
 75%|███████▍  | 28439/37962 [65:52:02<21:50:46,  8.26s/it]
 75%|███████▍  | 28440/37962 [65:52:10<21:42:52,  8.21s/it]
                                                           
{'loss': 1.1604, 'grad_norm': 31.50489044189453, 'learning_rate': 3.122268585626408e-06, 'epoch': 0.75}

 75%|███████▍  | 28440/37962 [65:52:10<21:42:52,  8.21s/it]
 75%|███████▍  | 28441/37962 [65:52:18<21:38:38,  8.18s/it]
 75%|███████▍  | 28442/37962 [65:52:26<21:47:53,  8.24s/it]
 75%|███████▍  | 28443/37962 [65:52:35<21:47:38,  8.24s/it]
 75%|███████▍  | 28444/37962 [65:52:43<21:43:47,  8.22s/it]
 75%|███████▍  | 28445/37962 [65:52:51<21:43:54,  8.22s/it]
 75%|███████▍  | 28446/37962 [65:52:59<21:52:57,  8.28s/it]
 75%|███████▍  | 28447/37962 [65:53:08<21:54:51,  8.29s/it]
 75%|███████▍  | 28448/37962 [65:53:16<21:56:13,  8.30s/it]
 75%|███████▍  | 28449/37962 [65:53:24<21:44:18,  8.23s/it]
 75%|███████▍  | 28450/37962 [65:53:32<21:49:10,  8.26s/it]
                                                           
{'loss': 1.1961, 'grad_norm': 9.79479694366455, 'learning_rate': 3.1160777787346153e-06, 'epoch': 0.75}

 75%|███████▍  | 28450/37962 [65:53:32<21:49:10,  8.26s/it]
 75%|███████▍  | 28451/37962 [65:53:41<21:48:56,  8.26s/it]
 75%|███████▍  | 28452/37962 [65:53:49<21:46:25,  8.24s/it]
 75%|███████▍  | 28453/37962 [65:53:57<21:48:17,  8.26s/it]
 75%|███████▍  | 28454/37962 [65:54:06<21:52:23,  8.28s/it]
 75%|███████▍  | 28455/37962 [65:54:14<21:50:26,  8.27s/it]
 75%|███████▍  | 28456/37962 [65:54:22<21:44:01,  8.23s/it]
 75%|███████▍  | 28457/37962 [65:54:30<21:51:38,  8.28s/it]
 75%|███████▍  | 28458/37962 [65:54:39<22:00:25,  8.34s/it]
 75%|███████▍  | 28459/37962 [65:54:47<21:57:31,  8.32s/it]
 75%|███████▍  | 28460/37962 [65:54:55<21:51:58,  8.28s/it]
                                                           
{'loss': 1.1244, 'grad_norm': 10.374861717224121, 'learning_rate': 3.109891982530554e-06, 'epoch': 0.75}

 75%|███████▍  | 28460/37962 [65:54:55<21:51:58,  8.28s/it]
 75%|███████▍  | 28461/37962 [65:55:04<21:55:05,  8.31s/it]
 75%|███████▍  | 28462/37962 [65:55:12<21:57:38,  8.32s/it]
 75%|███████▍  | 28463/37962 [65:55:20<22:03:49,  8.36s/it]
 75%|███████▍  | 28464/37962 [65:55:29<21:54:07,  8.30s/it]
 75%|███████▍  | 28465/37962 [65:55:37<21:51:22,  8.29s/it]
 75%|███████▍  | 28466/37962 [65:55:45<21:42:07,  8.23s/it]
 75%|███████▍  | 28467/37962 [65:55:53<21:48:44,  8.27s/it]
 75%|███████▍  | 28468/37962 [65:56:01<21:31:07,  8.16s/it]
 75%|███████▍  | 28469/37962 [65:56:09<21:27:16,  8.14s/it]
 75%|███████▍  | 28470/37962 [65:56:17<21:20:12,  8.09s/it]
                                                           
{'loss': 1.2418, 'grad_norm': 22.22563362121582, 'learning_rate': 3.103711201516757e-06, 'epoch': 0.75}

 75%|███████▍  | 28470/37962 [65:56:17<21:20:12,  8.09s/it]
 75%|███████▍  | 28471/37962 [65:56:25<21:18:58,  8.09s/it]
 75%|███████▌  | 28472/37962 [65:56:34<21:40:48,  8.22s/it]
 75%|███████▌  | 28473/37962 [65:56:42<21:41:54,  8.23s/it]
 75%|███████▌  | 28474/37962 [65:56:50<21:39:04,  8.22s/it]
 75%|███████▌  | 28475/37962 [65:56:59<21:40:38,  8.23s/it]
 75%|███████▌  | 28476/37962 [65:57:07<21:28:23,  8.15s/it]
 75%|███████▌  | 28477/37962 [65:57:14<20:58:45,  7.96s/it]
 75%|███████▌  | 28478/37962 [65:57:22<21:02:22,  7.99s/it]
 75%|███████▌  | 28479/37962 [65:57:30<21:11:55,  8.05s/it]
 75%|███████▌  | 28480/37962 [65:57:39<21:28:39,  8.15s/it]
                                                           
{'loss': 1.1937, 'grad_norm': 181.2485809326172, 'learning_rate': 3.0975354401921022e-06, 'epoch': 0.75}

 75%|███████▌  | 28480/37962 [65:57:39<21:28:39,  8.15s/it]
 75%|███████▌  | 28481/37962 [65:57:47<21:22:45,  8.12s/it]
 75%|███████▌  | 28482/37962 [65:57:55<21:25:37,  8.14s/it]
 75%|███████▌  | 28483/37962 [65:58:03<21:32:43,  8.18s/it]
 75%|███████▌  | 28484/37962 [65:58:12<21:37:55,  8.22s/it]
 75%|███████▌  | 28485/37962 [65:58:20<21:52:26,  8.31s/it]
 75%|███████▌  | 28486/37962 [65:58:28<21:49:54,  8.29s/it]
 75%|███████▌  | 28487/37962 [65:58:37<21:48:50,  8.29s/it]
 75%|███████▌  | 28488/37962 [65:58:45<21:50:01,  8.30s/it]
 75%|███████▌  | 28489/37962 [65:58:53<21:52:33,  8.31s/it]
 75%|███████▌  | 28490/37962 [65:59:01<21:45:32,  8.27s/it]
                                                           
{'loss': 1.307, 'grad_norm': 19.17630386352539, 'learning_rate': 3.0913647030518267e-06, 'epoch': 0.75}

 75%|███████▌  | 28490/37962 [65:59:01<21:45:32,  8.27s/it]
 75%|███████▌  | 28491/37962 [65:59:10<21:43:55,  8.26s/it]
 75%|███████▌  | 28492/37962 [65:59:18<21:48:22,  8.29s/it]
 75%|███████▌  | 28493/37962 [65:59:26<21:50:18,  8.30s/it]
 75%|███████▌  | 28494/37962 [65:59:35<21:51:20,  8.31s/it]
 75%|███████▌  | 28495/37962 [65:59:43<21:58:01,  8.35s/it]
 75%|███████▌  | 28496/37962 [65:59:51<21:42:01,  8.25s/it]
 75%|███████▌  | 28497/37962 [66:00:00<21:47:03,  8.29s/it]
 75%|███████▌  | 28498/37962 [66:00:08<21:35:31,  8.21s/it]
 75%|███████▌  | 28499/37962 [66:00:16<21:31:38,  8.19s/it]
 75%|███████▌  | 28500/37962 [66:00:24<21:28:04,  8.17s/it]
                                                           
{'loss': 1.1995, 'grad_norm': 6.6467108726501465, 'learning_rate': 3.085198994587497e-06, 'epoch': 0.75}

 75%|███████▌  | 28500/37962 [66:00:24<21:28:04,  8.17s/it]
 75%|███████▌  | 28501/37962 [66:00:32<21:40:00,  8.24s/it]
 75%|███████▌  | 28502/37962 [66:00:41<21:51:35,  8.32s/it]
 75%|███████▌  | 28503/37962 [66:00:49<21:48:00,  8.30s/it]
 75%|███████▌  | 28504/37962 [66:00:57<21:37:21,  8.23s/it]
 75%|███████▌  | 28505/37962 [66:01:05<21:41:37,  8.26s/it]
 75%|███████▌  | 28506/37962 [66:01:13<21:32:12,  8.20s/it]
 75%|███████▌  | 28507/37962 [66:01:22<21:44:08,  8.28s/it]
 75%|███████▌  | 28508/37962 [66:01:30<21:37:53,  8.24s/it]
 75%|███████▌  | 28509/37962 [66:01:38<21:47:26,  8.30s/it]
 75%|███████▌  | 28510/37962 [66:01:47<21:47:50,  8.30s/it]
                                                           
{'loss': 1.2474, 'grad_norm': 8.172994613647461, 'learning_rate': 3.079038319287029e-06, 'epoch': 0.75}

 75%|███████▌  | 28510/37962 [66:01:47<21:47:50,  8.30s/it]
 75%|███████▌  | 28511/37962 [66:01:55<21:45:16,  8.29s/it]
 75%|███████▌  | 28512/37962 [66:02:03<21:42:53,  8.27s/it]
 75%|███████▌  | 28513/37962 [66:02:12<21:51:17,  8.33s/it]
 75%|███████▌  | 28514/37962 [66:02:20<21:46:50,  8.30s/it]
 75%|███████▌  | 28515/37962 [66:02:28<21:36:27,  8.23s/it]
 75%|███████▌  | 28516/37962 [66:02:36<21:30:01,  8.19s/it]
 75%|███████▌  | 28517/37962 [66:02:44<21:35:19,  8.23s/it]
 75%|███████▌  | 28518/37962 [66:02:53<21:37:03,  8.24s/it]
 75%|███████▌  | 28519/37962 [66:03:01<21:39:48,  8.26s/it]
 75%|███████▌  | 28520/37962 [66:03:09<21:35:04,  8.23s/it]
                                                           
{'loss': 1.1796, 'grad_norm': 18.543067932128906, 'learning_rate': 3.0728826816346733e-06, 'epoch': 0.75}

 75%|███████▌  | 28520/37962 [66:03:09<21:35:04,  8.23s/it]
 75%|███████▌  | 28521/37962 [66:03:17<21:22:12,  8.15s/it]
 75%|███████▌  | 28522/37962 [66:03:26<21:32:26,  8.21s/it]
 75%|███████▌  | 28523/37962 [66:03:34<21:34:06,  8.23s/it]
 75%|███████▌  | 28524/37962 [66:03:42<21:39:34,  8.26s/it]
 75%|███████▌  | 28525/37962 [66:03:50<21:22:58,  8.16s/it]
 75%|███████▌  | 28526/37962 [66:03:58<21:22:34,  8.16s/it]
 75%|███████▌  | 28527/37962 [66:04:07<21:35:03,  8.24s/it]
 75%|███████▌  | 28528/37962 [66:04:15<21:38:45,  8.26s/it]
 75%|███████▌  | 28529/37962 [66:04:23<21:35:15,  8.24s/it]
 75%|███████▌  | 28530/37962 [66:04:31<21:31:21,  8.21s/it]
                                                           
{'loss': 1.2926, 'grad_norm': 8.461370468139648, 'learning_rate': 3.066732086111011e-06, 'epoch': 0.75}

 75%|███████▌  | 28530/37962 [66:04:31<21:31:21,  8.21s/it]
 75%|███████▌  | 28531/37962 [66:04:39<21:27:57,  8.19s/it]
 75%|███████▌  | 28532/37962 [66:04:47<21:16:34,  8.12s/it]
 75%|███████▌  | 28533/37962 [66:04:56<21:18:32,  8.14s/it]
 75%|███████▌  | 28534/37962 [66:05:04<21:20:53,  8.15s/it]
 75%|███████▌  | 28535/37962 [66:05:12<21:30:23,  8.21s/it]
 75%|███████▌  | 28536/37962 [66:05:20<21:39:59,  8.27s/it]
 75%|███████▌  | 28537/37962 [66:05:29<21:40:53,  8.28s/it]
 75%|███████▌  | 28538/37962 [66:05:37<21:43:01,  8.30s/it]
 75%|███████▌  | 28539/37962 [66:05:45<21:40:07,  8.28s/it]
 75%|███████▌  | 28540/37962 [66:05:54<21:39:58,  8.28s/it]
                                                           
{'loss': 1.2061, 'grad_norm': 32.905635833740234, 'learning_rate': 3.060586537192948e-06, 'epoch': 0.75}

 75%|███████▌  | 28540/37962 [66:05:54<21:39:58,  8.28s/it]
 75%|███████▌  | 28541/37962 [66:06:02<21:26:52,  8.20s/it]
 75%|███████▌  | 28542/37962 [66:06:10<21:30:26,  8.22s/it]
 75%|███████▌  | 28543/37962 [66:06:18<21:37:43,  8.27s/it]
 75%|███████▌  | 28544/37962 [66:06:26<21:33:48,  8.24s/it]
 75%|███████▌  | 28545/37962 [66:06:35<21:39:33,  8.28s/it]
 75%|███████▌  | 28546/37962 [66:06:43<21:23:29,  8.18s/it]
 75%|███████▌  | 28547/37962 [66:06:51<21:31:33,  8.23s/it]
 75%|███████▌  | 28548/37962 [66:06:59<21:31:54,  8.23s/it]
 75%|███████▌  | 28549/37962 [66:07:07<21:20:18,  8.16s/it]
 75%|███████▌  | 28550/37962 [66:07:16<21:22:31,  8.18s/it]
                                                           
{'loss': 1.3205, 'grad_norm': 7.27686882019043, 'learning_rate': 3.05444603935373e-06, 'epoch': 0.75}

 75%|███████▌  | 28550/37962 [66:07:16<21:22:31,  8.18s/it]
 75%|███████▌  | 28551/37962 [66:07:24<21:28:17,  8.21s/it]
 75%|███████▌  | 28552/37962 [66:07:32<21:28:10,  8.21s/it]
 75%|███████▌  | 28553/37962 [66:07:40<21:33:13,  8.25s/it]
 75%|███████▌  | 28554/37962 [66:07:49<21:33:50,  8.25s/it]
 75%|███████▌  | 28555/37962 [66:07:57<21:28:33,  8.22s/it]
 75%|███████▌  | 28556/37962 [66:08:05<21:11:52,  8.11s/it]
 75%|███████▌  | 28557/37962 [66:08:13<21:19:15,  8.16s/it]
 75%|███████▌  | 28558/37962 [66:08:21<21:21:02,  8.17s/it]
 75%|███████▌  | 28559/37962 [66:08:29<21:22:00,  8.18s/it]
 75%|███████▌  | 28560/37962 [66:08:38<21:35:45,  8.27s/it]
                                                           
{'loss': 1.2253, 'grad_norm': 16.090869903564453, 'learning_rate': 3.0483105970629145e-06, 'epoch': 0.75}

 75%|███████▌  | 28560/37962 [66:08:38<21:35:45,  8.27s/it]
 75%|███████▌  | 28561/37962 [66:08:46<21:11:47,  8.12s/it]
 75%|███████▌  | 28562/37962 [66:08:54<21:17:45,  8.16s/it]
 75%|███████▌  | 28563/37962 [66:09:02<21:16:35,  8.15s/it]
 75%|███████▌  | 28564/37962 [66:09:10<21:22:21,  8.19s/it]
 75%|███████▌  | 28565/37962 [66:09:19<21:25:35,  8.21s/it]
 75%|███████▌  | 28566/37962 [66:09:27<21:24:57,  8.21s/it]
 75%|███████▌  | 28567/37962 [66:09:35<21:28:17,  8.23s/it]
 75%|███████▌  | 28568/37962 [66:09:43<21:32:15,  8.25s/it]
 75%|███████▌  | 28569/37962 [66:09:52<21:29:35,  8.24s/it]
 75%|███████▌  | 28570/37962 [66:10:00<21:32:32,  8.26s/it]
                                                           
{'loss': 1.2639, 'grad_norm': 15.524304389953613, 'learning_rate': 3.0421802147863857e-06, 'epoch': 0.75}

 75%|███████▌  | 28570/37962 [66:10:00<21:32:32,  8.26s/it]
 75%|███████▌  | 28571/37962 [66:10:08<21:37:18,  8.29s/it]
 75%|███████▌  | 28572/37962 [66:10:17<21:39:46,  8.31s/it]
 75%|███████▌  | 28573/37962 [66:10:25<21:30:48,  8.25s/it]
 75%|███████▌  | 28574/37962 [66:10:33<21:34:34,  8.27s/it]
 75%|███████▌  | 28575/37962 [66:10:41<21:36:14,  8.29s/it]
 75%|███████▌  | 28576/37962 [66:10:50<21:33:22,  8.27s/it]
 75%|███████▌  | 28577/37962 [66:10:58<21:32:21,  8.26s/it]
 75%|███████▌  | 28578/37962 [66:11:06<21:30:49,  8.25s/it]
 75%|███████▌  | 28579/37962 [66:11:14<21:35:58,  8.29s/it]
 75%|███████▌  | 28580/37962 [66:11:23<21:33:31,  8.27s/it]
                                                           
{'loss': 1.1626, 'grad_norm': 7.1260762214660645, 'learning_rate': 3.0360548969863477e-06, 'epoch': 0.75}

 75%|███████▌  | 28580/37962 [66:11:23<21:33:31,  8.27s/it]
 75%|███████▌  | 28581/37962 [66:11:31<21:33:29,  8.27s/it]
 75%|███████▌  | 28582/37962 [66:11:39<21:22:14,  8.20s/it]
 75%|███████▌  | 28583/37962 [66:11:47<21:26:25,  8.23s/it]
 75%|███████▌  | 28584/37962 [66:11:55<21:26:34,  8.23s/it]
 75%|███████▌  | 28585/37962 [66:12:04<21:33:08,  8.27s/it]
 75%|███████▌  | 28586/37962 [66:12:12<21:25:17,  8.23s/it]
 75%|███████▌  | 28587/37962 [66:12:20<21:23:39,  8.22s/it]
 75%|███████▌  | 28588/37962 [66:12:29<21:32:20,  8.27s/it]
 75%|███████▌  | 28589/37962 [66:12:37<21:46:04,  8.36s/it]
 75%|███████▌  | 28590/37962 [66:12:45<21:41:45,  8.33s/it]
                                                           
{'loss': 1.2278, 'grad_norm': 11.451452255249023, 'learning_rate': 3.029934648121303e-06, 'epoch': 0.75}

 75%|███████▌  | 28590/37962 [66:12:45<21:41:45,  8.33s/it]
 75%|███████▌  | 28591/37962 [66:12:54<21:33:41,  8.28s/it]
 75%|███████▌  | 28592/37962 [66:13:02<21:40:39,  8.33s/it]
 75%|███████▌  | 28593/37962 [66:13:10<21:38:03,  8.31s/it]
 75%|███████▌  | 28594/37962 [66:13:18<21:28:31,  8.25s/it]
 75%|███████▌  | 28595/37962 [66:13:27<21:26:17,  8.24s/it]
 75%|███████▌  | 28596/37962 [66:13:35<21:22:26,  8.22s/it]
 75%|███████▌  | 28597/37962 [66:13:43<21:14:16,  8.16s/it]
 75%|███████▌  | 28598/37962 [66:13:51<20:59:57,  8.07s/it]
 75%|███████▌  | 28599/37962 [66:13:59<21:09:17,  8.13s/it]
 75%|███████▌  | 28600/37962 [66:14:07<21:02:27,  8.09s/it]
                                                           
{'loss': 1.2497, 'grad_norm': 8.683025360107422, 'learning_rate': 3.023819472646079e-06, 'epoch': 0.75}

 75%|███████▌  | 28600/37962 [66:14:07<21:02:27,  8.09s/it]
 75%|███████▌  | 28601/37962 [66:14:15<21:00:42,  8.08s/it]
 75%|███████▌  | 28602/37962 [66:14:23<21:08:42,  8.13s/it]
 75%|███████▌  | 28603/37962 [66:14:32<21:17:19,  8.19s/it]
 75%|███████▌  | 28604/37962 [66:14:40<21:24:56,  8.24s/it]
 75%|███████▌  | 28605/37962 [66:14:48<21:33:27,  8.29s/it]
 75%|███████▌  | 28606/37962 [66:14:57<21:48:43,  8.39s/it]
 75%|███████▌  | 28607/37962 [66:15:05<21:32:46,  8.29s/it]
 75%|███████▌  | 28608/37962 [66:15:13<21:24:34,  8.24s/it]
 75%|███████▌  | 28609/37962 [66:15:21<21:16:27,  8.19s/it]
 75%|███████▌  | 28610/37962 [66:15:29<21:01:56,  8.10s/it]
                                                           
{'loss': 1.1293, 'grad_norm': 5.360408306121826, 'learning_rate': 3.0177093750118103e-06, 'epoch': 0.75}

 75%|███████▌  | 28610/37962 [66:15:29<21:01:56,  8.10s/it]
 75%|███████▌  | 28611/37962 [66:15:37<21:15:10,  8.18s/it]
 75%|███████▌  | 28612/37962 [66:15:46<21:25:18,  8.25s/it]
 75%|███████▌  | 28613/37962 [66:15:54<21:30:42,  8.28s/it]
 75%|███████▌  | 28614/37962 [66:16:03<21:45:34,  8.38s/it]
 75%|███████▌  | 28615/37962 [66:16:11<21:42:27,  8.36s/it]
 75%|███████▌  | 28616/37962 [66:16:19<21:37:46,  8.33s/it]
 75%|███████▌  | 28617/37962 [66:16:28<21:35:29,  8.32s/it]
 75%|███████▌  | 28618/37962 [66:16:36<21:32:50,  8.30s/it]
 75%|███████▌  | 28619/37962 [66:16:44<21:22:04,  8.23s/it]
 75%|███████▌  | 28620/37962 [66:16:52<21:20:34,  8.22s/it]
                                                           
{'loss': 1.1803, 'grad_norm': 14.305228233337402, 'learning_rate': 3.011604359665925e-06, 'epoch': 0.75}

 75%|███████▌  | 28620/37962 [66:16:52<21:20:34,  8.22s/it]
 75%|███████▌  | 28621/37962 [66:17:00<21:22:05,  8.24s/it]
 75%|███████▌  | 28622/37962 [66:17:09<21:31:04,  8.29s/it]
 75%|███████▌  | 28623/37962 [66:17:17<21:33:21,  8.31s/it]
 75%|███████▌  | 28624/37962 [66:17:26<21:39:06,  8.35s/it]
 75%|███████▌  | 28625/37962 [66:17:34<21:42:20,  8.37s/it]
 75%|███████▌  | 28626/37962 [66:17:42<21:37:53,  8.34s/it]
 75%|███████▌  | 28627/37962 [66:17:51<21:35:19,  8.33s/it]
 75%|███████▌  | 28628/37962 [66:17:59<21:41:12,  8.36s/it]
 75%|███████▌  | 28629/37962 [66:18:07<21:37:01,  8.34s/it]
 75%|███████▌  | 28630/37962 [66:18:16<21:33:08,  8.31s/it]
                                                           
{'loss': 1.163, 'grad_norm': 12.235884666442871, 'learning_rate': 3.005504431052163e-06, 'epoch': 0.75}

 75%|███████▌  | 28630/37962 [66:18:16<21:33:08,  8.31s/it]
 75%|███████▌  | 28631/37962 [66:18:24<21:19:08,  8.23s/it]
 75%|███████▌  | 28632/37962 [66:18:32<21:36:26,  8.34s/it]
 75%|███████▌  | 28633/37962 [66:18:41<21:33:33,  8.32s/it]
 75%|███████▌  | 28634/37962 [66:18:49<21:29:14,  8.29s/it]
 75%|███████▌  | 28635/37962 [66:18:57<21:31:56,  8.31s/it]
 75%|███████▌  | 28636/37962 [66:19:05<21:33:46,  8.32s/it]
 75%|███████▌  | 28637/37962 [66:19:14<21:34:39,  8.33s/it]
 75%|███████▌  | 28638/37962 [66:19:22<21:38:06,  8.35s/it]
 75%|███████▌  | 28639/37962 [66:19:30<21:19:01,  8.23s/it]
 75%|███████▌  | 28640/37962 [66:19:38<21:22:28,  8.25s/it]
                                                           
{'loss': 1.117, 'grad_norm': 13.817008018493652, 'learning_rate': 2.999409593610555e-06, 'epoch': 0.75}

 75%|███████▌  | 28640/37962 [66:19:38<21:22:28,  8.25s/it]
 75%|███████▌  | 28641/37962 [66:19:47<21:22:17,  8.25s/it]
 75%|███████▌  | 28642/37962 [66:19:55<21:17:30,  8.22s/it]
 75%|███████▌  | 28643/37962 [66:20:03<21:22:35,  8.26s/it]
 75%|███████▌  | 28644/37962 [66:20:11<21:16:30,  8.22s/it]
 75%|███████▌  | 28645/37962 [66:20:20<21:14:07,  8.21s/it]
 75%|███████▌  | 28646/37962 [66:20:28<21:12:55,  8.20s/it]
 75%|███████▌  | 28647/37962 [66:20:36<21:14:30,  8.21s/it]
 75%|███████▌  | 28648/37962 [66:20:44<21:13:03,  8.20s/it]
 75%|███████▌  | 28649/37962 [66:20:52<20:53:26,  8.08s/it]
 75%|███████▌  | 28650/37962 [66:21:00<21:05:25,  8.15s/it]
                                                           
{'loss': 1.2165, 'grad_norm': 5.548008918762207, 'learning_rate': 2.9933198517774233e-06, 'epoch': 0.75}

 75%|███████▌  | 28650/37962 [66:21:00<21:05:25,  8.15s/it]
 75%|███████▌  | 28651/37962 [66:21:09<21:12:57,  8.20s/it]
 75%|███████▌  | 28652/37962 [66:21:17<21:10:18,  8.19s/it]
 75%|███████▌  | 28653/37962 [66:21:25<21:15:52,  8.22s/it]
 75%|███████▌  | 28654/37962 [66:21:34<21:32:57,  8.33s/it]
 75%|███████▌  | 28655/37962 [66:21:42<21:22:58,  8.27s/it]
 75%|███████▌  | 28656/37962 [66:21:50<21:20:17,  8.25s/it]
 75%|███████▌  | 28657/37962 [66:21:58<21:25:12,  8.29s/it]
 75%|███████▌  | 28658/37962 [66:22:07<21:24:34,  8.28s/it]
 75%|███████▌  | 28659/37962 [66:22:15<21:23:55,  8.28s/it]
 75%|███████▌  | 28660/37962 [66:22:23<21:27:19,  8.30s/it]
                                                           
{'loss': 1.2513, 'grad_norm': 7.306845664978027, 'learning_rate': 2.987235209985393e-06, 'epoch': 0.75}

 75%|███████▌  | 28660/37962 [66:22:23<21:27:19,  8.30s/it]
 75%|███████▌  | 28661/37962 [66:22:32<21:32:41,  8.34s/it]
 76%|███████▌  | 28662/37962 [66:22:40<21:23:45,  8.28s/it]
 76%|███████▌  | 28663/37962 [66:22:48<21:30:44,  8.33s/it]
 76%|███████▌  | 28664/37962 [66:22:56<21:18:56,  8.25s/it]
 76%|███████▌  | 28665/37962 [66:23:05<21:22:11,  8.27s/it]
 76%|███████▌  | 28666/37962 [66:23:13<21:32:27,  8.34s/it]
 76%|███████▌  | 28667/37962 [66:23:21<21:25:52,  8.30s/it]
 76%|███████▌  | 28668/37962 [66:23:30<21:27:12,  8.31s/it]
 76%|███████▌  | 28669/37962 [66:23:38<21:30:27,  8.33s/it]
 76%|███████▌  | 28670/37962 [66:23:46<21:20:21,  8.27s/it]
                                                           
{'loss': 1.1674, 'grad_norm': 10.334417343139648, 'learning_rate': 2.9811556726633615e-06, 'epoch': 0.76}

 76%|███████▌  | 28670/37962 [66:23:46<21:20:21,  8.27s/it]
 76%|███████▌  | 28671/37962 [66:23:54<21:19:50,  8.26s/it]
 76%|███████▌  | 28672/37962 [66:24:02<21:03:32,  8.16s/it]
 76%|███████▌  | 28673/37962 [66:24:11<21:07:54,  8.19s/it]
 76%|███████▌  | 28674/37962 [66:24:19<21:00:22,  8.14s/it]
 76%|███████▌  | 28675/37962 [66:24:27<21:09:54,  8.20s/it]
 76%|███████▌  | 28676/37962 [66:24:35<21:20:44,  8.28s/it]
 76%|███████▌  | 28677/37962 [66:24:43<21:03:33,  8.17s/it]
 76%|███████▌  | 28678/37962 [66:24:52<21:17:38,  8.26s/it]
 76%|███████▌  | 28679/37962 [66:25:00<21:11:33,  8.22s/it]
 76%|███████▌  | 28680/37962 [66:25:08<21:17:53,  8.26s/it]
                                                           
{'loss': 1.2057, 'grad_norm': 7.412516117095947, 'learning_rate': 2.975081244236526e-06, 'epoch': 0.76}

 76%|███████▌  | 28680/37962 [66:25:08<21:17:53,  8.26s/it]
 76%|███████▌  | 28681/37962 [66:25:17<21:15:34,  8.25s/it]
 76%|███████▌  | 28682/37962 [66:25:25<21:21:21,  8.28s/it]
 76%|███████▌  | 28683/37962 [66:25:33<21:25:52,  8.31s/it]
 76%|███████▌  | 28684/37962 [66:25:41<21:20:47,  8.28s/it]
 76%|███████▌  | 28685/37962 [66:25:49<21:01:40,  8.16s/it]
 76%|███████▌  | 28686/37962 [66:25:57<20:57:12,  8.13s/it]
 76%|███████▌  | 28687/37962 [66:26:06<21:01:08,  8.16s/it]
 76%|███████▌  | 28688/37962 [66:26:14<21:02:52,  8.17s/it]
 76%|███████▌  | 28689/37962 [66:26:22<21:09:05,  8.21s/it]
 76%|███████▌  | 28690/37962 [66:26:30<21:08:06,  8.21s/it]
                                                           
{'loss': 1.1513, 'grad_norm': 9.014336585998535, 'learning_rate': 2.9690119291263543e-06, 'epoch': 0.76}

 76%|███████▌  | 28690/37962 [66:26:30<21:08:06,  8.21s/it]
 76%|███████▌  | 28691/37962 [66:26:38<21:01:20,  8.16s/it]
 76%|███████▌  | 28692/37962 [66:26:47<21:14:19,  8.25s/it]
 76%|███████▌  | 28693/37962 [66:26:55<21:11:27,  8.23s/it]
 76%|███████▌  | 28694/37962 [66:27:03<21:12:40,  8.24s/it]
 76%|███████▌  | 28695/37962 [66:27:12<21:18:09,  8.28s/it]
 76%|███████▌  | 28696/37962 [66:27:20<21:15:24,  8.26s/it]
 76%|███████▌  | 28697/37962 [66:27:28<21:14:25,  8.25s/it]
 76%|███████▌  | 28698/37962 [66:27:36<21:11:59,  8.24s/it]
 76%|███████▌  | 28699/37962 [66:27:45<21:14:45,  8.26s/it]
 76%|███████▌  | 28700/37962 [66:27:53<21:13:29,  8.25s/it]
                                                           
{'loss': 1.2096, 'grad_norm': 14.351969718933105, 'learning_rate': 2.962947731750593e-06, 'epoch': 0.76}

 76%|███████▌  | 28700/37962 [66:27:53<21:13:29,  8.25s/it]
 76%|███████▌  | 28701/37962 [66:28:01<21:16:00,  8.27s/it]
 76%|███████▌  | 28702/37962 [66:28:09<21:14:17,  8.26s/it]
 76%|███████▌  | 28703/37962 [66:28:18<21:22:57,  8.31s/it]
 76%|███████▌  | 28704/37962 [66:28:26<21:19:14,  8.29s/it]
 76%|███████▌  | 28705/37962 [66:28:34<21:20:20,  8.30s/it]
 76%|███████▌  | 28706/37962 [66:28:42<21:10:48,  8.24s/it]
 76%|███████▌  | 28707/37962 [66:28:51<21:15:46,  8.27s/it]
 76%|███████▌  | 28708/37962 [66:28:59<21:24:48,  8.33s/it]
 76%|███████▌  | 28709/37962 [66:29:08<21:25:39,  8.34s/it]
 76%|███████▌  | 28710/37962 [66:29:16<21:19:16,  8.30s/it]
                                                           
{'loss': 1.1669, 'grad_norm': 6.201923370361328, 'learning_rate': 2.956888656523269e-06, 'epoch': 0.76}

 76%|███████▌  | 28710/37962 [66:29:16<21:19:16,  8.30s/it]
 76%|███████▌  | 28711/37962 [66:29:24<21:08:01,  8.22s/it]
 76%|███████▌  | 28712/37962 [66:29:32<21:24:32,  8.33s/it]
 76%|███████▌  | 28713/37962 [66:29:41<21:13:05,  8.26s/it]
 76%|███████▌  | 28714/37962 [66:29:49<21:06:48,  8.22s/it]
 76%|███████▌  | 28715/37962 [66:29:57<21:05:14,  8.21s/it]
 76%|███████▌  | 28716/37962 [66:30:05<21:13:24,  8.26s/it]
 76%|███████▌  | 28717/37962 [66:30:14<21:22:37,  8.32s/it]
 76%|███████▌  | 28718/37962 [66:30:22<21:21:48,  8.32s/it]
 76%|███████▌  | 28719/37962 [66:30:30<21:14:50,  8.28s/it]
 76%|███████▌  | 28720/37962 [66:30:39<21:19:08,  8.30s/it]
                                                           
{'loss': 1.1903, 'grad_norm': 6.256604194641113, 'learning_rate': 2.9508347078546816e-06, 'epoch': 0.76}

 76%|███████▌  | 28720/37962 [66:30:39<21:19:08,  8.30s/it]
 76%|███████▌  | 28721/37962 [66:30:47<21:21:23,  8.32s/it]
 76%|███████▌  | 28722/37962 [66:30:55<21:14:58,  8.28s/it]
 76%|███████▌  | 28723/37962 [66:31:03<21:00:35,  8.19s/it]
 76%|███████▌  | 28724/37962 [66:31:11<21:05:38,  8.22s/it]
 76%|███████▌  | 28725/37962 [66:31:19<20:55:22,  8.15s/it]
 76%|███████▌  | 28726/37962 [66:31:28<21:05:07,  8.22s/it]
 76%|███████▌  | 28727/37962 [66:31:36<21:12:14,  8.27s/it]
 76%|███████▌  | 28728/37962 [66:31:44<21:11:55,  8.26s/it]
 76%|███████▌  | 28729/37962 [66:31:53<21:06:43,  8.23s/it]
 76%|███████▌  | 28730/37962 [66:32:00<20:51:38,  8.13s/it]
                                                           
{'loss': 1.1866, 'grad_norm': 5.771930694580078, 'learning_rate': 2.9447858901513894e-06, 'epoch': 0.76}

 76%|███████▌  | 28730/37962 [66:32:00<20:51:38,  8.13s/it]
 76%|███████▌  | 28731/37962 [66:32:09<21:03:06,  8.21s/it]
 76%|███████▌  | 28732/37962 [66:32:17<20:44:47,  8.09s/it]
 76%|███████▌  | 28733/37962 [66:32:25<20:55:01,  8.16s/it]
 76%|███████▌  | 28734/37962 [66:32:33<20:59:59,  8.19s/it]
 76%|███████▌  | 28735/37962 [66:32:42<21:10:59,  8.26s/it]
 76%|███████▌  | 28736/37962 [66:32:50<21:14:19,  8.29s/it]
 76%|███████▌  | 28737/37962 [66:32:58<20:55:13,  8.16s/it]
 76%|███████▌  | 28738/37962 [66:33:06<21:00:54,  8.20s/it]
 76%|███████▌  | 28739/37962 [66:33:15<21:10:54,  8.27s/it]
 76%|███████▌  | 28740/37962 [66:33:23<21:07:37,  8.25s/it]
                                                           
{'loss': 1.218, 'grad_norm': 16.40662956237793, 'learning_rate': 2.9387422078162266e-06, 'epoch': 0.76}

 76%|███████▌  | 28740/37962 [66:33:23<21:07:37,  8.25s/it]
 76%|███████▌  | 28741/37962 [66:33:31<21:17:42,  8.31s/it]
 76%|███████▌  | 28742/37962 [66:33:40<21:13:42,  8.29s/it]
 76%|███████▌  | 28743/37962 [66:33:48<21:10:32,  8.27s/it]
 76%|███████▌  | 28744/37962 [66:33:56<21:04:40,  8.23s/it]
 76%|███████▌  | 28745/37962 [66:34:04<21:17:38,  8.32s/it]
 76%|███████▌  | 28746/37962 [66:34:13<21:13:12,  8.29s/it]
 76%|███████▌  | 28747/37962 [66:34:21<21:15:59,  8.31s/it]
 76%|███████▌  | 28748/37962 [66:34:29<21:02:29,  8.22s/it]
 76%|███████▌  | 28749/37962 [66:34:37<21:03:45,  8.23s/it]
 76%|███████▌  | 28750/37962 [66:34:46<21:11:29,  8.28s/it]
                                                           
{'loss': 1.2242, 'grad_norm': 15.207331657409668, 'learning_rate': 2.932703665248283e-06, 'epoch': 0.76}

 76%|███████▌  | 28750/37962 [66:34:46<21:11:29,  8.28s/it]
 76%|███████▌  | 28751/37962 [66:34:54<20:56:32,  8.19s/it]
 76%|███████▌  | 28752/37962 [66:35:01<20:39:38,  8.08s/it]
 76%|███████▌  | 28753/37962 [66:35:10<20:46:28,  8.12s/it]
 76%|███████▌  | 28754/37962 [66:35:18<20:58:40,  8.20s/it]
 76%|███████▌  | 28755/37962 [66:35:26<21:00:46,  8.22s/it]
 76%|███████▌  | 28756/37962 [66:35:34<20:57:42,  8.20s/it]
 76%|███████▌  | 28757/37962 [66:35:43<21:06:38,  8.26s/it]
 76%|███████▌  | 28758/37962 [66:35:51<21:07:19,  8.26s/it]
 76%|███████▌  | 28759/37962 [66:35:59<21:02:05,  8.23s/it]
 76%|███████▌  | 28760/37962 [66:36:08<21:11:31,  8.29s/it]
                                                           
{'loss': 1.2411, 'grad_norm': 8.450349807739258, 'learning_rate': 2.926670266842906e-06, 'epoch': 0.76}

 76%|███████▌  | 28760/37962 [66:36:08<21:11:31,  8.29s/it]
 76%|███████▌  | 28761/37962 [66:36:16<20:58:45,  8.21s/it]
 76%|███████▌  | 28762/37962 [66:36:24<21:05:37,  8.25s/it]
 76%|███████▌  | 28763/37962 [66:36:32<21:05:37,  8.25s/it]
 76%|███████▌  | 28764/37962 [66:36:40<20:51:29,  8.16s/it]
 76%|███████▌  | 28765/37962 [66:36:49<20:57:48,  8.21s/it]
 76%|███████▌  | 28766/37962 [66:36:57<20:58:50,  8.21s/it]
 76%|███████▌  | 28767/37962 [66:37:05<21:11:19,  8.30s/it]
 76%|███████▌  | 28768/37962 [66:37:14<21:11:35,  8.30s/it]
 76%|███████▌  | 28769/37962 [66:37:22<21:14:11,  8.32s/it]
 76%|███████▌  | 28770/37962 [66:37:30<21:11:58,  8.30s/it]
                                                           
{'loss': 1.1494, 'grad_norm': 5.260526180267334, 'learning_rate': 2.9206420169917083e-06, 'epoch': 0.76}

 76%|███████▌  | 28770/37962 [66:37:30<21:11:58,  8.30s/it]
 76%|███████▌  | 28771/37962 [66:37:39<21:09:20,  8.29s/it]
 76%|███████▌  | 28772/37962 [66:37:47<21:00:49,  8.23s/it]
 76%|███████▌  | 28773/37962 [66:37:55<21:04:13,  8.25s/it]
 76%|███████▌  | 28774/37962 [66:38:03<21:10:41,  8.30s/it]
 76%|███████▌  | 28775/37962 [66:38:12<21:15:31,  8.33s/it]
 76%|███████▌  | 28776/37962 [66:38:20<20:56:39,  8.21s/it]
 76%|███████▌  | 28777/37962 [66:38:28<20:52:27,  8.18s/it]
 76%|███████▌  | 28778/37962 [66:38:36<20:55:40,  8.20s/it]
 76%|███████▌  | 28779/37962 [66:38:44<20:58:27,  8.22s/it]
 76%|███████▌  | 28780/37962 [66:38:53<21:03:10,  8.25s/it]
                                                           
{'loss': 1.1618, 'grad_norm': 15.014945983886719, 'learning_rate': 2.9146189200825413e-06, 'epoch': 0.76}

 76%|███████▌  | 28780/37962 [66:38:53<21:03:10,  8.25s/it]
 76%|███████▌  | 28781/37962 [66:39:01<21:13:34,  8.32s/it]
 76%|███████▌  | 28782/37962 [66:39:09<21:13:46,  8.33s/it]
 76%|███████▌  | 28783/37962 [66:39:18<21:07:30,  8.29s/it]
 76%|███████▌  | 28784/37962 [66:39:26<21:15:09,  8.34s/it]
 76%|███████▌  | 28785/37962 [66:39:34<20:59:50,  8.24s/it]
 76%|███████▌  | 28786/37962 [66:39:43<21:12:07,  8.32s/it]
 76%|███████▌  | 28787/37962 [66:39:51<21:10:25,  8.31s/it]
 76%|███████▌  | 28788/37962 [66:39:59<20:56:13,  8.22s/it]
 76%|███████▌  | 28789/37962 [66:40:07<20:46:40,  8.15s/it]
 76%|███████▌  | 28790/37962 [66:40:23<26:48:15, 10.52s/it]
                                                           
{'loss': 1.1926, 'grad_norm': 7.597702503204346, 'learning_rate': 2.9086009804995196e-06, 'epoch': 0.76}

 76%|███████▌  | 28790/37962 [66:40:23<26:48:15, 10.52s/it]
 76%|███████▌  | 28791/37962 [66:40:31<25:10:41,  9.88s/it]
 76%|███████▌  | 28792/37962 [66:40:40<23:56:35,  9.40s/it]
 76%|███████▌  | 28793/37962 [66:40:48<23:02:43,  9.05s/it]
 76%|███████▌  | 28794/37962 [66:40:56<22:32:28,  8.85s/it]
 76%|███████▌  | 28795/37962 [66:41:05<22:14:39,  8.74s/it]
 76%|███████▌  | 28796/37962 [66:41:13<21:48:17,  8.56s/it]
 76%|███████▌  | 28797/37962 [66:41:21<21:23:28,  8.40s/it]
 76%|███████▌  | 28798/37962 [66:41:29<21:19:37,  8.38s/it]
 76%|███████▌  | 28799/37962 [66:41:37<21:14:52,  8.35s/it]
 76%|███████▌  | 28800/37962 [66:41:46<21:17:08,  8.36s/it]
                                                           
{'loss': 1.163, 'grad_norm': 10.684842109680176, 'learning_rate': 2.9025882026229946e-06, 'epoch': 0.76}

 76%|███████▌  | 28800/37962 [66:41:46<21:17:08,  8.36s/it]
 76%|███████▌  | 28801/37962 [66:41:54<20:54:56,  8.22s/it]
 76%|███████▌  | 28802/37962 [66:42:02<20:52:54,  8.21s/it]
 76%|███████▌  | 28803/37962 [66:42:10<20:37:47,  8.11s/it]
 76%|███████▌  | 28804/37962 [66:42:18<20:49:06,  8.18s/it]
 76%|███████▌  | 28805/37962 [66:42:27<21:00:59,  8.26s/it]
 76%|███████▌  | 28806/37962 [66:42:35<21:10:05,  8.32s/it]
 76%|███████▌  | 28807/37962 [66:42:43<20:49:54,  8.19s/it]
 76%|███████▌  | 28808/37962 [66:42:51<21:01:39,  8.27s/it]
 76%|███████▌  | 28809/37962 [66:43:00<20:56:08,  8.23s/it]
 76%|███████▌  | 28810/37962 [66:43:08<20:57:44,  8.25s/it]
                                                           
{'loss': 1.2202, 'grad_norm': 6.940189838409424, 'learning_rate': 2.8965805908295595e-06, 'epoch': 0.76}

 76%|███████▌  | 28810/37962 [66:43:08<20:57:44,  8.25s/it]
 76%|███████▌  | 28811/37962 [66:43:16<20:49:58,  8.20s/it]
 76%|███████▌  | 28812/37962 [66:43:24<20:58:13,  8.25s/it]
 76%|███████▌  | 28813/37962 [66:43:32<20:50:28,  8.20s/it]
 76%|███████▌  | 28814/37962 [66:43:41<20:58:04,  8.25s/it]
 76%|███████▌  | 28815/37962 [66:43:49<20:52:37,  8.22s/it]
 76%|███████▌  | 28816/37962 [66:43:57<20:56:16,  8.24s/it]
 76%|███████▌  | 28817/37962 [66:44:05<20:52:38,  8.22s/it]
 76%|███████▌  | 28818/37962 [66:44:14<20:59:39,  8.27s/it]
 76%|███████▌  | 28819/37962 [66:44:22<20:55:53,  8.24s/it]
 76%|███████▌  | 28820/37962 [66:44:30<20:54:59,  8.24s/it]
                                                           
{'loss': 1.2313, 'grad_norm': 15.791461944580078, 'learning_rate': 2.890578149492054e-06, 'epoch': 0.76}

 76%|███████▌  | 28820/37962 [66:44:30<20:54:59,  8.24s/it]
 76%|███████▌  | 28821/37962 [66:44:38<20:59:20,  8.27s/it]
 76%|███████▌  | 28822/37962 [66:44:47<21:02:41,  8.29s/it]
 76%|███████▌  | 28823/37962 [66:44:55<21:10:20,  8.34s/it]
 76%|███████▌  | 28824/37962 [66:45:04<21:15:09,  8.37s/it]
 76%|███████▌  | 28825/37962 [66:45:12<21:11:50,  8.35s/it]
 76%|███████▌  | 28826/37962 [66:45:20<21:10:02,  8.34s/it]
 76%|███████▌  | 28827/37962 [66:45:29<21:04:36,  8.31s/it]
 76%|███████▌  | 28828/37962 [66:45:37<21:03:01,  8.30s/it]
 76%|███████▌  | 28829/37962 [66:45:45<21:04:15,  8.31s/it]
 76%|███████▌  | 28830/37962 [66:45:54<21:05:31,  8.31s/it]
                                                           
{'loss': 1.2363, 'grad_norm': 5.58223819732666, 'learning_rate': 2.884580882979554e-06, 'epoch': 0.76}

 76%|███████▌  | 28830/37962 [66:45:54<21:05:31,  8.31s/it]
 76%|███████▌  | 28831/37962 [66:46:02<21:09:47,  8.34s/it]
 76%|███████▌  | 28832/37962 [66:46:10<21:04:10,  8.31s/it]
 76%|███████▌  | 28833/37962 [66:46:19<21:12:00,  8.36s/it]
 76%|███████▌  | 28834/37962 [66:46:27<21:15:13,  8.38s/it]
 76%|███████▌  | 28835/37962 [66:46:35<21:08:23,  8.34s/it]
 76%|███████▌  | 28836/37962 [66:46:43<20:57:52,  8.27s/it]
 76%|███████▌  | 28837/37962 [66:46:51<20:49:03,  8.21s/it]
 76%|███████▌  | 28838/37962 [66:47:00<20:48:15,  8.21s/it]
 76%|███████▌  | 28839/37962 [66:47:08<20:48:35,  8.21s/it]
 76%|███████▌  | 28840/37962 [66:47:16<20:41:30,  8.17s/it]
                                                           
{'loss': 1.2575, 'grad_norm': 12.057272911071777, 'learning_rate': 2.878588795657363e-06, 'epoch': 0.76}

 76%|███████▌  | 28840/37962 [66:47:16<20:41:30,  8.17s/it]
 76%|███████▌  | 28841/37962 [66:47:24<20:46:37,  8.20s/it]
 76%|███████▌  | 28842/37962 [66:47:33<20:59:02,  8.28s/it]
 76%|███████▌  | 28843/37962 [66:47:41<20:54:27,  8.25s/it]
 76%|███████▌  | 28844/37962 [66:47:49<20:53:26,  8.25s/it]
 76%|███████▌  | 28845/37962 [66:47:57<20:40:48,  8.17s/it]
 76%|███████▌  | 28846/37962 [66:48:06<20:54:50,  8.26s/it]
 76%|███████▌  | 28847/37962 [66:48:14<20:58:08,  8.28s/it]
 76%|███████▌  | 28848/37962 [66:48:22<20:55:15,  8.26s/it]
 76%|███████▌  | 28849/37962 [66:48:30<20:55:49,  8.27s/it]
 76%|███████▌  | 28850/37962 [66:48:39<20:57:14,  8.28s/it]
                                                           
{'loss': 1.1674, 'grad_norm': 7.4193196296691895, 'learning_rate': 2.8726018918870146e-06, 'epoch': 0.76}

 76%|███████▌  | 28850/37962 [66:48:39<20:57:14,  8.28s/it]
 76%|███████▌  | 28851/37962 [66:48:47<20:52:27,  8.25s/it]
 76%|███████▌  | 28852/37962 [66:48:55<21:03:23,  8.32s/it]
 76%|███████▌  | 28853/37962 [66:49:04<21:03:29,  8.32s/it]
 76%|███████▌  | 28854/37962 [66:49:12<21:04:46,  8.33s/it]
 76%|███████▌  | 28855/37962 [66:49:20<21:03:40,  8.33s/it]
 76%|███████▌  | 28856/37962 [66:49:29<20:57:31,  8.29s/it]
 76%|███████▌  | 28857/37962 [66:49:37<20:53:53,  8.26s/it]
 76%|███████▌  | 28858/37962 [66:49:45<20:56:33,  8.28s/it]
 76%|███████▌  | 28859/37962 [66:49:53<20:52:42,  8.26s/it]
 76%|███████▌  | 28860/37962 [66:50:02<20:52:11,  8.25s/it]
                                                           
{'loss': 1.1734, 'grad_norm': 7.747781753540039, 'learning_rate': 2.866620176026279e-06, 'epoch': 0.76}

 76%|███████▌  | 28860/37962 [66:50:02<20:52:11,  8.25s/it]
 76%|███████▌  | 28861/37962 [66:50:09<20:35:50,  8.15s/it]
 76%|███████▌  | 28862/37962 [66:50:18<20:34:34,  8.14s/it]
 76%|███████▌  | 28863/37962 [66:50:26<20:25:36,  8.08s/it]
 76%|███████▌  | 28864/37962 [66:50:33<20:17:50,  8.03s/it]
 76%|███████▌  | 28865/37962 [66:50:42<20:24:54,  8.08s/it]
 76%|███████▌  | 28866/37962 [66:50:50<20:32:27,  8.13s/it]
 76%|███████▌  | 28867/37962 [66:50:58<20:52:12,  8.26s/it]
 76%|███████▌  | 28868/37962 [66:51:06<20:34:11,  8.14s/it]
 76%|███████▌  | 28869/37962 [66:51:14<20:23:19,  8.07s/it]
 76%|███████▌  | 28870/37962 [66:51:23<20:35:24,  8.15s/it]
                                                           
{'loss': 1.204, 'grad_norm': 9.720925331115723, 'learning_rate': 2.860643652429137e-06, 'epoch': 0.76}

 76%|███████▌  | 28870/37962 [66:51:23<20:35:24,  8.15s/it]
 76%|███████▌  | 28871/37962 [66:51:31<20:53:03,  8.27s/it]
 76%|███████▌  | 28872/37962 [66:51:39<20:51:03,  8.26s/it]
 76%|███████▌  | 28873/37962 [66:51:48<20:49:04,  8.25s/it]
 76%|███████▌  | 28874/37962 [66:51:56<20:49:12,  8.25s/it]
 76%|███████▌  | 28875/37962 [66:52:04<20:53:17,  8.28s/it]
 76%|███████▌  | 28876/37962 [66:52:12<20:55:15,  8.29s/it]
 76%|███████▌  | 28877/37962 [66:52:21<20:50:23,  8.26s/it]
 76%|███████▌  | 28878/37962 [66:52:29<20:53:46,  8.28s/it]
 76%|███████▌  | 28879/37962 [66:52:37<20:55:31,  8.29s/it]
 76%|███████▌  | 28880/37962 [66:52:46<20:53:54,  8.28s/it]
                                                           
{'loss': 1.1502, 'grad_norm': 5.958893775939941, 'learning_rate': 2.8546723254457988e-06, 'epoch': 0.76}

 76%|███████▌  | 28880/37962 [66:52:46<20:53:54,  8.28s/it]
 76%|███████▌  | 28881/37962 [66:52:54<20:53:20,  8.28s/it]
 76%|███████▌  | 28882/37962 [66:53:02<20:33:51,  8.15s/it]
 76%|███████▌  | 28883/37962 [66:53:10<20:37:16,  8.18s/it]
 76%|███████▌  | 28884/37962 [66:53:18<20:44:59,  8.23s/it]
 76%|███████▌  | 28885/37962 [66:53:27<20:47:20,  8.25s/it]
 76%|███████▌  | 28886/37962 [66:53:35<20:43:06,  8.22s/it]
 76%|███████▌  | 28887/37962 [66:53:43<20:48:56,  8.26s/it]
 76%|███████▌  | 28888/37962 [66:53:51<20:48:51,  8.26s/it]
 76%|███████▌  | 28889/37962 [66:54:00<20:46:47,  8.25s/it]
 76%|███████▌  | 28890/37962 [66:54:08<20:53:33,  8.29s/it]
                                                           
{'loss': 1.1615, 'grad_norm': 6.58300256729126, 'learning_rate': 2.848706199422696e-06, 'epoch': 0.76}

 76%|███████▌  | 28890/37962 [66:54:08<20:53:33,  8.29s/it]
 76%|███████▌  | 28891/37962 [66:54:16<20:52:33,  8.29s/it]
 76%|███████▌  | 28892/37962 [66:54:24<20:47:55,  8.26s/it]
 76%|███████▌  | 28893/37962 [66:54:32<20:38:00,  8.19s/it]
 76%|███████▌  | 28894/37962 [66:54:40<20:25:39,  8.11s/it]
 76%|███████▌  | 28895/37962 [66:54:49<20:33:29,  8.16s/it]
 76%|███████▌  | 28896/37962 [66:54:57<20:41:57,  8.22s/it]
 76%|███████▌  | 28897/37962 [66:55:05<20:47:05,  8.25s/it]
 76%|███████▌  | 28898/37962 [66:55:14<20:45:24,  8.24s/it]
 76%|███████▌  | 28899/37962 [66:55:22<20:44:12,  8.24s/it]
 76%|███████▌  | 28900/37962 [66:55:30<20:45:58,  8.25s/it]
                                                           
{'loss': 1.0715, 'grad_norm': 6.317964553833008, 'learning_rate': 2.842745278702458e-06, 'epoch': 0.76}

 76%|███████▌  | 28900/37962 [66:55:30<20:45:58,  8.25s/it]
 76%|███████▌  | 28901/37962 [66:55:38<20:49:31,  8.27s/it]
 76%|███████▌  | 28902/37962 [66:55:47<20:47:57,  8.26s/it]
 76%|███████▌  | 28903/37962 [66:55:55<20:47:01,  8.26s/it]
 76%|███████▌  | 28904/37962 [66:56:03<20:39:28,  8.21s/it]
 76%|███████▌  | 28905/37962 [66:56:11<20:44:31,  8.24s/it]
 76%|███████▌  | 28906/37962 [66:56:20<20:45:06,  8.25s/it]
 76%|███████▌  | 28907/37962 [66:56:28<20:45:32,  8.25s/it]
 76%|███████▌  | 28908/37962 [66:56:36<20:39:05,  8.21s/it]
 76%|███████▌  | 28909/37962 [66:56:44<20:49:27,  8.28s/it]
 76%|███████▌  | 28910/37962 [66:56:52<20:38:43,  8.21s/it]
                                                           
{'loss': 1.1557, 'grad_norm': 9.768638610839844, 'learning_rate': 2.836789567623941e-06, 'epoch': 0.76}

 76%|███████▌  | 28910/37962 [66:56:52<20:38:43,  8.21s/it]
 76%|███████▌  | 28911/37962 [66:57:01<20:42:11,  8.23s/it]
 76%|███████▌  | 28912/37962 [66:57:09<20:22:43,  8.11s/it]
 76%|███████▌  | 28913/37962 [66:57:17<20:28:04,  8.14s/it]
 76%|███████▌  | 28914/37962 [66:57:25<20:22:39,  8.11s/it]
 76%|███████▌  | 28915/37962 [66:57:33<20:10:18,  8.03s/it]
 76%|███████▌  | 28916/37962 [66:57:41<20:30:29,  8.16s/it]
 76%|███████▌  | 28917/37962 [66:57:49<20:34:32,  8.19s/it]
 76%|███████▌  | 28918/37962 [66:57:58<20:44:47,  8.26s/it]
 76%|███████▌  | 28919/37962 [66:58:06<20:45:32,  8.26s/it]
 76%|███████▌  | 28920/37962 [66:58:14<20:34:50,  8.19s/it]
                                                           
{'loss': 1.1307, 'grad_norm': 6.416312217712402, 'learning_rate': 2.8308390705222057e-06, 'epoch': 0.76}

 76%|███████▌  | 28920/37962 [66:58:14<20:34:50,  8.19s/it]
 76%|███████▌  | 28921/37962 [66:58:22<20:31:03,  8.17s/it]
 76%|███████▌  | 28922/37962 [66:58:30<20:28:02,  8.15s/it]
 76%|███████▌  | 28923/37962 [66:58:39<20:38:28,  8.22s/it]
 76%|███████▌  | 28924/37962 [66:58:47<20:24:04,  8.13s/it]
 76%|███████▌  | 28925/37962 [66:58:55<20:27:47,  8.15s/it]
 76%|███████▌  | 28926/37962 [66:59:03<20:38:18,  8.22s/it]
 76%|███████▌  | 28927/37962 [66:59:11<20:20:20,  8.10s/it]
 76%|███████▌  | 28928/37962 [66:59:19<20:04:14,  8.00s/it]
 76%|███████▌  | 28929/37962 [66:59:27<20:16:58,  8.08s/it]
 76%|███████▌  | 28930/37962 [66:59:35<20:27:50,  8.16s/it]
                                                           
{'loss': 1.1914, 'grad_norm': 6.566009998321533, 'learning_rate': 2.8248937917285113e-06, 'epoch': 0.76}

 76%|███████▌  | 28930/37962 [66:59:35<20:27:50,  8.16s/it]
 76%|███████▌  | 28931/37962 [66:59:43<20:25:18,  8.14s/it]
 76%|███████▌  | 28932/37962 [66:59:52<20:34:56,  8.21s/it]
 76%|███████▌  | 28933/37962 [67:00:00<20:37:25,  8.22s/it]
 76%|███████▌  | 28934/37962 [67:00:09<20:45:35,  8.28s/it]
 76%|███████▌  | 28935/37962 [67:00:17<20:43:10,  8.26s/it]
 76%|███████▌  | 28936/37962 [67:00:25<20:42:21,  8.26s/it]
 76%|███████▌  | 28937/37962 [67:00:33<20:48:50,  8.30s/it]
 76%|███████▌  | 28938/37962 [67:00:42<20:47:30,  8.29s/it]
 76%|███████▌  | 28939/37962 [67:00:50<20:37:33,  8.23s/it]
 76%|███████▌  | 28940/37962 [67:00:58<20:38:24,  8.24s/it]
                                                           
{'loss': 1.2963, 'grad_norm': 8.535893440246582, 'learning_rate': 2.818953735570329e-06, 'epoch': 0.76}

 76%|███████▌  | 28940/37962 [67:00:58<20:38:24,  8.24s/it]
 76%|███████▌  | 28941/37962 [67:01:13<26:03:24, 10.40s/it]
 76%|███████▌  | 28942/37962 [67:01:22<24:35:37,  9.82s/it]
 76%|███████▌  | 28943/37962 [67:01:30<23:26:33,  9.36s/it]
 76%|███████▌  | 28944/37962 [67:01:38<22:36:54,  9.03s/it]
 76%|███████▌  | 28945/37962 [67:01:47<22:01:19,  8.79s/it]
 76%|███████▌  | 28946/37962 [67:01:55<21:40:07,  8.65s/it]
 76%|███████▋  | 28947/37962 [67:02:03<21:17:15,  8.50s/it]
 76%|███████▋  | 28948/37962 [67:02:12<21:10:47,  8.46s/it]
 76%|███████▋  | 28949/37962 [67:02:20<21:01:52,  8.40s/it]
 76%|███████▋  | 28950/37962 [67:02:28<20:42:48,  8.27s/it]
                                                           
{'loss': 1.258, 'grad_norm': 8.804706573486328, 'learning_rate': 2.8130189063713187e-06, 'epoch': 0.76}

 76%|███████▋  | 28950/37962 [67:02:28<20:42:48,  8.27s/it]
 76%|███████▋  | 28951/37962 [67:02:36<20:55:39,  8.36s/it]
 76%|███████▋  | 28952/37962 [67:02:45<20:48:31,  8.31s/it]
 76%|███████▋  | 28953/37962 [67:02:53<20:43:43,  8.28s/it]
 76%|███████▋  | 28954/37962 [67:03:01<20:43:15,  8.28s/it]
 76%|███████▋  | 28955/37962 [67:03:09<20:26:25,  8.17s/it]
 76%|███████▋  | 28956/37962 [67:03:17<20:40:48,  8.27s/it]
 76%|███████▋  | 28957/37962 [67:03:26<20:37:44,  8.25s/it]
 76%|███████▋  | 28958/37962 [67:03:34<20:31:04,  8.20s/it]
 76%|███████▋  | 28959/37962 [67:03:42<20:13:05,  8.08s/it]
 76%|███████▋  | 28960/37962 [67:03:49<19:58:59,  7.99s/it]
                                                           
{'loss': 1.2119, 'grad_norm': 10.209192276000977, 'learning_rate': 2.8070893084513372e-06, 'epoch': 0.76}

 76%|███████▋  | 28960/37962 [67:03:49<19:58:59,  7.99s/it]
 76%|███████▋  | 28961/37962 [67:03:57<19:55:00,  7.97s/it]
 76%|███████▋  | 28962/37962 [67:04:05<20:09:35,  8.06s/it]
 76%|███████▋  | 28963/37962 [67:04:14<20:14:01,  8.09s/it]
 76%|███████▋  | 28964/37962 [67:04:22<20:23:26,  8.16s/it]
 76%|███████▋  | 28965/37962 [67:04:30<20:24:35,  8.17s/it]
 76%|███████▋  | 28966/37962 [67:04:38<20:32:11,  8.22s/it]
 76%|███████▋  | 28967/37962 [67:04:47<20:30:13,  8.21s/it]
 76%|███████▋  | 28968/37962 [67:04:55<20:26:28,  8.18s/it]
 76%|███████▋  | 28969/37962 [67:05:03<20:16:51,  8.12s/it]
 76%|███████▋  | 28970/37962 [67:05:11<20:23:04,  8.16s/it]
                                                           
{'loss': 1.2349, 'grad_norm': 8.886204719543457, 'learning_rate': 2.8011649461264435e-06, 'epoch': 0.76}

 76%|███████▋  | 28970/37962 [67:05:11<20:23:04,  8.16s/it]
 76%|███████▋  | 28971/37962 [67:05:19<20:24:35,  8.17s/it]
 76%|███████▋  | 28972/37962 [67:05:27<20:28:18,  8.20s/it]
 76%|███████▋  | 28973/37962 [67:05:36<20:33:57,  8.24s/it]
 76%|███████▋  | 28974/37962 [67:05:44<20:36:36,  8.26s/it]
 76%|███████▋  | 28975/37962 [67:05:52<20:40:16,  8.28s/it]
 76%|███████▋  | 28976/37962 [67:06:01<20:43:09,  8.30s/it]
 76%|███████▋  | 28977/37962 [67:06:09<20:37:46,  8.27s/it]
 76%|███████▋  | 28978/37962 [67:06:17<20:35:37,  8.25s/it]
 76%|███████▋  | 28979/37962 [67:06:25<20:36:12,  8.26s/it]
 76%|███████▋  | 28980/37962 [67:06:34<20:44:14,  8.31s/it]
                                                           
{'loss': 1.2566, 'grad_norm': 10.782876014709473, 'learning_rate': 2.7952458237088707e-06, 'epoch': 0.76}

 76%|███████▋  | 28980/37962 [67:06:34<20:44:14,  8.31s/it]
 76%|███████▋  | 28981/37962 [67:06:42<20:43:25,  8.31s/it]
 76%|███████▋  | 28982/37962 [67:06:50<20:40:49,  8.29s/it]
 76%|███████▋  | 28983/37962 [67:06:59<20:38:39,  8.28s/it]
 76%|███████▋  | 28984/37962 [67:07:07<20:35:51,  8.26s/it]
 76%|███████▋  | 28985/37962 [67:07:15<20:45:01,  8.32s/it]
 76%|███████▋  | 28986/37962 [67:07:23<20:31:50,  8.23s/it]
 76%|███████▋  | 28987/37962 [67:07:32<20:43:07,  8.31s/it]
 76%|███████▋  | 28988/37962 [67:07:40<20:46:38,  8.34s/it]
 76%|███████▋  | 28989/37962 [67:07:49<20:45:30,  8.33s/it]
 76%|███████▋  | 28990/37962 [67:07:57<20:40:48,  8.30s/it]
                                                           
{'loss': 1.2707, 'grad_norm': 8.938876152038574, 'learning_rate': 2.789331945507049e-06, 'epoch': 0.76}

 76%|███████▋  | 28990/37962 [67:07:57<20:40:48,  8.30s/it]
 76%|███████▋  | 28991/37962 [67:08:05<20:33:15,  8.25s/it]
 76%|███████▋  | 28992/37962 [67:08:13<20:36:28,  8.27s/it]
 76%|███████▋  | 28993/37962 [67:08:22<20:38:23,  8.28s/it]
 76%|███████▋  | 28994/37962 [67:08:30<20:35:00,  8.26s/it]
 76%|███████▋  | 28995/37962 [67:08:38<20:31:42,  8.24s/it]
 76%|███████▋  | 28996/37962 [67:08:46<20:29:33,  8.23s/it]
 76%|███████▋  | 28997/37962 [67:08:55<20:35:55,  8.27s/it]
 76%|███████▋  | 28998/37962 [67:09:03<20:37:19,  8.28s/it]
 76%|███████▋  | 28999/37962 [67:09:11<20:42:46,  8.32s/it]
 76%|███████▋  | 29000/37962 [67:09:19<20:37:18,  8.28s/it]
                                                           
{'loss': 1.1739, 'grad_norm': 20.352174758911133, 'learning_rate': 2.783423315825594e-06, 'epoch': 0.76}

 76%|███████▋  | 29000/37962 [67:09:19<20:37:18,  8.28s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 76%|███████▋  | 29001/37962 [67:10:11<52:38:35, 21.15s/it]
 76%|███████▋  | 29002/37962 [67:10:19<43:04:23, 17.31s/it]
 76%|███████▋  | 29003/37962 [67:10:27<36:19:31, 14.60s/it]
 76%|███████▋  | 29004/37962 [67:10:36<31:41:04, 12.73s/it]
 76%|███████▋  | 29005/37962 [67:10:44<28:22:22, 11.40s/it]
 76%|███████▋  | 29006/37962 [67:10:52<26:06:33, 10.50s/it]
 76%|███████▋  | 29007/37962 [67:11:01<24:31:54,  9.86s/it]
 76%|███████▋  | 29008/37962 [67:11:09<23:10:52,  9.32s/it]
 76%|███████▋  | 29009/37962 [67:11:17<22:18:47,  8.97s/it]
 76%|███████▋  | 29010/37962 [67:11:25<21:41:39,  8.72s/it]
                                                           
{'loss': 1.2797, 'grad_norm': 4.359538555145264, 'learning_rate': 2.7775199389652828e-06, 'epoch': 0.76}

 76%|███████▋  | 29010/37962 [67:11:25<21:41:39,  8.72s/it]
 76%|███████▋  | 29011/37962 [67:11:34<21:28:18,  8.64s/it]
 76%|███████▋  | 29012/37962 [67:11:42<21:09:27,  8.51s/it]
 76%|███████▋  | 29013/37962 [67:11:50<20:52:17,  8.40s/it]
 76%|███████▋  | 29014/37962 [67:11:58<20:37:55,  8.30s/it]
 76%|███████▋  | 29015/37962 [67:12:06<20:42:39,  8.33s/it]
 76%|███████▋  | 29016/37962 [67:12:15<20:34:46,  8.28s/it]
 76%|███████▋  | 29017/37962 [67:12:31<26:33:45, 10.69s/it]
 76%|███████▋  | 29018/37962 [67:12:39<24:51:50, 10.01s/it]
 76%|███████▋  | 29019/37962 [67:12:47<23:31:57,  9.47s/it]
 76%|███████▋  | 29020/37962 [67:12:56<22:36:37,  9.10s/it]
                                                           
{'loss': 1.2643, 'grad_norm': 10.400784492492676, 'learning_rate': 2.771621819223088e-06, 'epoch': 0.76}

 76%|███████▋  | 29020/37962 [67:12:56<22:36:37,  9.10s/it]
 76%|███████▋  | 29021/37962 [67:13:04<21:59:34,  8.86s/it]
 76%|███████▋  | 29022/37962 [67:13:12<21:32:44,  8.68s/it]
 76%|███████▋  | 29023/37962 [67:13:20<21:11:39,  8.54s/it]
 76%|███████▋  | 29024/37962 [67:13:29<21:04:25,  8.49s/it]
 76%|███████▋  | 29025/37962 [67:13:37<20:53:52,  8.42s/it]
 76%|███████▋  | 29026/37962 [67:13:45<20:36:16,  8.30s/it]
 76%|███████▋  | 29027/37962 [67:13:53<20:31:47,  8.27s/it]
 76%|███████▋  | 29028/37962 [67:14:01<20:22:16,  8.21s/it]
 76%|███████▋  | 29029/37962 [67:14:10<20:22:19,  8.21s/it]
 76%|███████▋  | 29030/37962 [67:14:18<20:26:08,  8.24s/it]
                                                           
{'loss': 1.2051, 'grad_norm': 10.794914245605469, 'learning_rate': 2.7657289608921502e-06, 'epoch': 0.76}

 76%|███████▋  | 29030/37962 [67:14:18<20:26:08,  8.24s/it]
 76%|███████▋  | 29031/37962 [67:14:26<20:20:59,  8.20s/it]
 76%|███████▋  | 29032/37962 [67:14:34<20:09:27,  8.13s/it]
 76%|███████▋  | 29033/37962 [67:14:42<20:18:05,  8.19s/it]
 76%|███████▋  | 29034/37962 [67:14:50<20:18:08,  8.19s/it]
 76%|███████▋  | 29035/37962 [67:14:59<20:19:30,  8.20s/it]
 76%|███████▋  | 29036/37962 [67:15:07<20:22:10,  8.22s/it]
 76%|███████▋  | 29037/37962 [67:15:15<20:22:33,  8.22s/it]
 76%|███████▋  | 29038/37962 [67:15:24<20:36:26,  8.31s/it]
 76%|███████▋  | 29039/37962 [67:15:32<20:40:09,  8.34s/it]
 76%|███████▋  | 29040/37962 [67:15:41<20:53:52,  8.43s/it]
                                                           
{'loss': 1.207, 'grad_norm': 6.815999507904053, 'learning_rate': 2.759841368261773e-06, 'epoch': 0.76}

 76%|███████▋  | 29040/37962 [67:15:41<20:53:52,  8.43s/it]
 77%|███████▋  | 29041/37962 [67:15:49<20:45:22,  8.38s/it]
 77%|███████▋  | 29042/37962 [67:15:57<20:35:17,  8.31s/it]
 77%|███████▋  | 29043/37962 [67:16:05<20:14:22,  8.17s/it]
 77%|███████▋  | 29044/37962 [67:16:13<20:08:09,  8.13s/it]
 77%|███████▋  | 29045/37962 [67:16:21<20:12:26,  8.16s/it]
 77%|███████▋  | 29046/37962 [67:16:29<20:14:15,  8.17s/it]
 77%|███████▋  | 29047/37962 [67:16:38<20:10:22,  8.15s/it]
 77%|███████▋  | 29048/37962 [67:16:46<20:24:27,  8.24s/it]
 77%|███████▋  | 29049/37962 [67:16:54<20:19:27,  8.21s/it]
 77%|███████▋  | 29050/37962 [67:17:02<20:22:04,  8.23s/it]
                                                           
{'loss': 1.1434, 'grad_norm': 10.733039855957031, 'learning_rate': 2.7539590456174402e-06, 'epoch': 0.77}

 77%|███████▋  | 29050/37962 [67:17:02<20:22:04,  8.23s/it]
 77%|███████▋  | 29051/37962 [67:17:11<20:25:31,  8.25s/it]
 77%|███████▋  | 29052/37962 [67:17:19<20:26:13,  8.26s/it]
 77%|███████▋  | 29053/37962 [67:17:27<20:25:56,  8.26s/it]
 77%|███████▋  | 29054/37962 [67:17:36<20:32:17,  8.30s/it]
 77%|███████▋  | 29055/37962 [67:17:44<20:31:26,  8.30s/it]
 77%|███████▋  | 29056/37962 [67:17:52<20:28:02,  8.27s/it]
 77%|███████▋  | 29057/37962 [67:18:01<20:35:12,  8.32s/it]
 77%|███████▋  | 29058/37962 [67:18:09<20:42:23,  8.37s/it]
 77%|███████▋  | 29059/37962 [67:18:17<20:41:25,  8.37s/it]
 77%|███████▋  | 29060/37962 [67:18:26<20:35:38,  8.33s/it]
                                                           
{'loss': 1.1787, 'grad_norm': 10.242233276367188, 'learning_rate': 2.7480819972407893e-06, 'epoch': 0.77}

 77%|███████▋  | 29060/37962 [67:18:26<20:35:38,  8.33s/it]
 77%|███████▋  | 29061/37962 [67:18:34<20:42:01,  8.37s/it]
 77%|███████▋  | 29062/37962 [67:18:42<20:33:51,  8.32s/it]
 77%|███████▋  | 29063/37962 [67:18:51<20:30:16,  8.29s/it]
 77%|███████▋  | 29064/37962 [67:18:59<20:22:06,  8.24s/it]
 77%|███████▋  | 29065/37962 [67:19:07<20:09:24,  8.16s/it]
 77%|███████▋  | 29066/37962 [67:19:15<20:06:01,  8.13s/it]
 77%|███████▋  | 29067/37962 [67:19:23<20:10:45,  8.17s/it]
 77%|███████▋  | 29068/37962 [67:19:32<20:26:57,  8.28s/it]
 77%|███████▋  | 29069/37962 [67:19:40<20:25:04,  8.27s/it]
 77%|███████▋  | 29070/37962 [67:19:48<20:23:04,  8.25s/it]
                                                           
{'loss': 1.0793, 'grad_norm': 5.773466110229492, 'learning_rate': 2.7422102274096185e-06, 'epoch': 0.77}

 77%|███████▋  | 29070/37962 [67:19:48<20:23:04,  8.25s/it]
 77%|███████▋  | 29071/37962 [67:19:56<20:26:13,  8.28s/it]
 77%|███████▋  | 29072/37962 [67:20:05<20:23:57,  8.26s/it]
 77%|███████▋  | 29073/37962 [67:20:13<20:22:02,  8.25s/it]
 77%|███████▋  | 29074/37962 [67:20:21<20:24:03,  8.26s/it]
 77%|███████▋  | 29075/37962 [67:20:29<20:11:55,  8.18s/it]
 77%|███████▋  | 29076/37962 [67:20:37<20:15:11,  8.21s/it]
 77%|███████▋  | 29077/37962 [67:20:46<20:18:38,  8.23s/it]
 77%|███████▋  | 29078/37962 [67:20:54<20:27:51,  8.29s/it]
 77%|███████▋  | 29079/37962 [67:21:03<20:37:10,  8.36s/it]
 77%|███████▋  | 29080/37962 [67:21:11<20:42:29,  8.39s/it]
                                                           
{'loss': 1.169, 'grad_norm': 7.101656913757324, 'learning_rate': 2.736343740397893e-06, 'epoch': 0.77}

 77%|███████▋  | 29080/37962 [67:21:11<20:42:29,  8.39s/it]
 77%|███████▋  | 29081/37962 [67:21:19<20:21:08,  8.25s/it]
 77%|███████▋  | 29082/37962 [67:21:27<20:25:23,  8.28s/it]
 77%|███████▋  | 29083/37962 [67:21:36<20:23:50,  8.27s/it]
 77%|███████▋  | 29084/37962 [67:21:44<20:20:41,  8.25s/it]
 77%|███████▋  | 29085/37962 [67:21:52<20:20:09,  8.25s/it]
 77%|███████▋  | 29086/37962 [67:22:00<20:19:15,  8.24s/it]
 77%|███████▋  | 29087/37962 [67:22:08<20:15:14,  8.22s/it]
 77%|███████▋  | 29088/37962 [67:22:17<20:19:09,  8.24s/it]
 77%|███████▋  | 29089/37962 [67:22:25<20:15:02,  8.22s/it]
 77%|███████▋  | 29090/37962 [67:22:33<20:18:46,  8.24s/it]
                                                           
{'loss': 1.1255, 'grad_norm': 8.725702285766602, 'learning_rate': 2.730482540475722e-06, 'epoch': 0.77}

 77%|███████▋  | 29090/37962 [67:22:33<20:18:46,  8.24s/it]
 77%|███████▋  | 29091/37962 [67:22:41<20:19:00,  8.24s/it]
 77%|███████▋  | 29092/37962 [67:22:49<20:02:44,  8.14s/it]
 77%|███████▋  | 29093/37962 [67:22:58<20:11:17,  8.19s/it]
 77%|███████▋  | 29094/37962 [67:23:06<19:59:35,  8.12s/it]
 77%|███████▋  | 29095/37962 [67:23:14<20:03:45,  8.15s/it]
 77%|███████▋  | 29096/37962 [67:23:22<20:06:12,  8.16s/it]
 77%|███████▋  | 29097/37962 [67:23:30<20:04:50,  8.15s/it]
 77%|███████▋  | 29098/37962 [67:23:38<20:08:03,  8.18s/it]
 77%|███████▋  | 29099/37962 [67:23:47<20:15:30,  8.23s/it]
 77%|███████▋  | 29100/37962 [67:23:55<20:11:05,  8.20s/it]
                                                           
{'loss': 1.2055, 'grad_norm': 9.821623802185059, 'learning_rate': 2.7246266319093774e-06, 'epoch': 0.77}

 77%|███████▋  | 29100/37962 [67:23:55<20:11:05,  8.20s/it]
 77%|███████▋  | 29101/37962 [67:24:03<20:05:06,  8.16s/it]
 77%|███████▋  | 29102/37962 [67:24:11<19:59:38,  8.12s/it]
 77%|███████▋  | 29103/37962 [67:24:19<20:08:06,  8.18s/it]
 77%|███████▋  | 29104/37962 [67:24:28<20:18:33,  8.25s/it]
 77%|███████▋  | 29105/37962 [67:24:36<20:20:46,  8.27s/it]
 77%|███████▋  | 29106/37962 [67:24:44<20:13:47,  8.22s/it]
 77%|███████▋  | 29107/37962 [67:24:53<20:28:26,  8.32s/it]
 77%|███████▋  | 29108/37962 [67:25:01<20:28:12,  8.32s/it]
 77%|███████▋  | 29109/37962 [67:25:09<20:24:05,  8.30s/it]
 77%|███████▋  | 29110/37962 [67:25:17<20:23:45,  8.29s/it]
                                                           
{'loss': 1.2679, 'grad_norm': 10.057585716247559, 'learning_rate': 2.7187760189612665e-06, 'epoch': 0.77}

 77%|███████▋  | 29110/37962 [67:25:17<20:23:45,  8.29s/it]
 77%|███████▋  | 29111/37962 [67:25:26<20:29:21,  8.33s/it]
 77%|███████▋  | 29112/37962 [67:25:34<20:36:02,  8.38s/it]
 77%|███████▋  | 29113/37962 [67:25:43<20:33:40,  8.36s/it]
 77%|███████▋  | 29114/37962 [67:25:51<20:32:14,  8.36s/it]
 77%|███████▋  | 29115/37962 [67:25:59<20:31:51,  8.35s/it]
 77%|███████▋  | 29116/37962 [67:26:08<20:29:25,  8.34s/it]
 77%|███████▋  | 29117/37962 [67:26:16<20:29:56,  8.34s/it]
 77%|███████▋  | 29118/37962 [67:26:24<20:25:54,  8.32s/it]
 77%|███████▋  | 29119/37962 [67:26:32<20:06:34,  8.19s/it]
 77%|███████▋  | 29120/37962 [67:26:40<19:55:20,  8.11s/it]
                                                           
{'loss': 1.111, 'grad_norm': 12.664830207824707, 'learning_rate': 2.712930705889957e-06, 'epoch': 0.77}

 77%|███████▋  | 29120/37962 [67:26:40<19:55:20,  8.11s/it]
 77%|███████▋  | 29121/37962 [67:26:48<20:01:32,  8.15s/it]
 77%|███████▋  | 29122/37962 [67:26:57<20:05:11,  8.18s/it]
 77%|███████▋  | 29123/37962 [67:27:05<20:14:43,  8.25s/it]
 77%|███████▋  | 29124/37962 [67:27:13<20:18:32,  8.27s/it]
 77%|███████▋  | 29125/37962 [67:27:21<19:58:40,  8.14s/it]
 77%|███████▋  | 29126/37962 [67:27:30<20:09:21,  8.21s/it]
 77%|███████▋  | 29127/37962 [67:27:38<20:20:09,  8.29s/it]
 77%|███████▋  | 29128/37962 [67:27:46<20:26:14,  8.33s/it]
 77%|███████▋  | 29129/37962 [67:27:55<20:18:56,  8.28s/it]
 77%|███████▋  | 29130/37962 [67:28:03<20:17:40,  8.27s/it]
                                                           
{'loss': 1.256, 'grad_norm': 6.862995624542236, 'learning_rate': 2.707090696950143e-06, 'epoch': 0.77}

 77%|███████▋  | 29130/37962 [67:28:03<20:17:40,  8.27s/it]
 77%|███████▋  | 29131/37962 [67:28:11<20:15:14,  8.26s/it]
 77%|███████▋  | 29132/37962 [67:28:19<20:19:22,  8.29s/it]
 77%|███████▋  | 29133/37962 [67:28:27<20:08:35,  8.21s/it]
 77%|███████▋  | 29134/37962 [67:28:36<20:09:31,  8.22s/it]
 77%|███████▋  | 29135/37962 [67:28:44<20:02:41,  8.18s/it]
 77%|███████▋  | 29136/37962 [67:28:52<20:05:23,  8.19s/it]
 77%|███████▋  | 29137/37962 [67:29:00<20:09:57,  8.23s/it]
 77%|███████▋  | 29138/37962 [67:29:09<20:09:34,  8.22s/it]
 77%|███████▋  | 29139/37962 [67:29:17<20:18:24,  8.29s/it]
 77%|███████▋  | 29140/37962 [67:29:25<20:05:58,  8.20s/it]
                                                           
{'loss': 1.2801, 'grad_norm': 6.412499904632568, 'learning_rate': 2.701255996392672e-06, 'epoch': 0.77}

 77%|███████▋  | 29140/37962 [67:29:25<20:05:58,  8.20s/it]
 77%|███████▋  | 29141/37962 [67:29:33<20:12:18,  8.25s/it]
 77%|███████▋  | 29142/37962 [67:29:41<20:02:36,  8.18s/it]
 77%|███████▋  | 29143/37962 [67:29:50<20:04:15,  8.19s/it]
 77%|███████▋  | 29144/37962 [67:29:58<20:05:09,  8.20s/it]
 77%|███████▋  | 29145/37962 [67:30:06<20:10:19,  8.24s/it]
 77%|███████▋  | 29146/37962 [67:30:14<20:02:12,  8.18s/it]
 77%|███████▋  | 29147/37962 [67:30:22<20:05:52,  8.21s/it]
 77%|███████▋  | 29148/37962 [67:30:31<20:20:33,  8.31s/it]
 77%|███████▋  | 29149/37962 [67:30:39<20:23:51,  8.33s/it]
 77%|███████▋  | 29150/37962 [67:30:48<20:23:20,  8.33s/it]
                                                           
{'loss': 1.2615, 'grad_norm': 7.135316848754883, 'learning_rate': 2.695426608464521e-06, 'epoch': 0.77}

 77%|███████▋  | 29150/37962 [67:30:48<20:23:20,  8.33s/it]
 77%|███████▋  | 29151/37962 [67:30:56<20:23:53,  8.33s/it]
 77%|███████▋  | 29152/37962 [67:31:04<20:25:59,  8.35s/it]
 77%|███████▋  | 29153/37962 [67:31:12<20:10:29,  8.24s/it]
 77%|███████▋  | 29154/37962 [67:31:20<20:02:53,  8.19s/it]
 77%|███████▋  | 29155/37962 [67:31:29<20:05:58,  8.22s/it]
 77%|███████▋  | 29156/37962 [67:31:37<20:16:28,  8.29s/it]
 77%|███████▋  | 29157/37962 [67:31:46<20:17:12,  8.29s/it]
 77%|███████▋  | 29158/37962 [67:31:54<20:17:41,  8.30s/it]
 77%|███████▋  | 29159/37962 [67:32:02<20:07:18,  8.23s/it]
 77%|███████▋  | 29160/37962 [67:32:10<20:08:42,  8.24s/it]
                                                           
{'loss': 1.1688, 'grad_norm': 10.006275177001953, 'learning_rate': 2.689602537408794e-06, 'epoch': 0.77}

 77%|███████▋  | 29160/37962 [67:32:10<20:08:42,  8.24s/it]
 77%|███████▋  | 29161/37962 [67:32:18<20:08:15,  8.24s/it]
 77%|███████▋  | 29162/37962 [67:32:27<20:15:33,  8.29s/it]
 77%|███████▋  | 29163/37962 [67:32:35<20:16:16,  8.29s/it]
 77%|███████▋  | 29164/37962 [67:32:43<20:12:04,  8.27s/it]
 77%|███████▋  | 29165/37962 [67:32:52<20:13:59,  8.28s/it]
 77%|███████▋  | 29166/37962 [67:33:00<20:19:02,  8.32s/it]
 77%|███████▋  | 29167/37962 [67:33:08<20:19:39,  8.32s/it]
 77%|███████▋  | 29168/37962 [67:33:17<20:13:37,  8.28s/it]
 77%|███████▋  | 29169/37962 [67:33:25<20:13:21,  8.28s/it]
 77%|███████▋  | 29170/37962 [67:33:33<20:10:12,  8.26s/it]
                                                           
{'loss': 1.1008, 'grad_norm': 8.497063636779785, 'learning_rate': 2.68378378746474e-06, 'epoch': 0.77}

 77%|███████▋  | 29170/37962 [67:33:33<20:10:12,  8.26s/it]
 77%|███████▋  | 29171/37962 [67:33:41<20:19:21,  8.32s/it]
 77%|███████▋  | 29172/37962 [67:33:50<20:05:35,  8.23s/it]
 77%|███████▋  | 29173/37962 [67:33:57<19:53:49,  8.15s/it]
 77%|███████▋  | 29174/37962 [67:34:06<19:48:49,  8.12s/it]
 77%|███████▋  | 29175/37962 [67:34:14<19:58:20,  8.18s/it]
 77%|███████▋  | 29176/37962 [67:34:22<20:02:00,  8.21s/it]
 77%|███████▋  | 29177/37962 [67:34:30<20:02:45,  8.21s/it]
 77%|███████▋  | 29178/37962 [67:34:39<20:16:23,  8.31s/it]
 77%|███████▋  | 29179/37962 [67:34:47<20:24:50,  8.37s/it]
 77%|███████▋  | 29180/37962 [67:34:55<20:07:12,  8.25s/it]
                                                           
{'loss': 1.3264, 'grad_norm': 149.290771484375, 'learning_rate': 2.677970362867719e-06, 'epoch': 0.77}

 77%|███████▋  | 29180/37962 [67:34:55<20:07:12,  8.25s/it]
 77%|███████▋  | 29181/37962 [67:35:04<20:09:58,  8.27s/it]
 77%|███████▋  | 29182/37962 [67:35:12<20:06:31,  8.25s/it]
 77%|███████▋  | 29183/37962 [67:35:20<20:06:17,  8.24s/it]
 77%|███████▋  | 29184/37962 [67:35:28<20:09:52,  8.27s/it]
 77%|███████▋  | 29185/37962 [67:35:37<20:09:33,  8.27s/it]
 77%|███████▋  | 29186/37962 [67:35:45<20:08:33,  8.26s/it]
 77%|███████▋  | 29187/37962 [67:35:53<20:16:52,  8.32s/it]
 77%|███████▋  | 29188/37962 [67:36:01<20:01:33,  8.22s/it]
 77%|███████▋  | 29189/37962 [67:36:10<20:04:36,  8.24s/it]
 77%|███████▋  | 29190/37962 [67:36:18<20:05:25,  8.25s/it]
                                                           
{'loss': 1.1417, 'grad_norm': 13.960959434509277, 'learning_rate': 2.6721622678492255e-06, 'epoch': 0.77}

 77%|███████▋  | 29190/37962 [67:36:18<20:05:25,  8.25s/it]
 77%|███████▋  | 29191/37962 [67:36:26<20:04:21,  8.24s/it]
 77%|███████▋  | 29192/37962 [67:36:35<20:11:40,  8.29s/it]
 77%|███████▋  | 29193/37962 [67:36:43<20:05:18,  8.25s/it]
 77%|███████▋  | 29194/37962 [67:36:51<20:02:50,  8.23s/it]
 77%|███████▋  | 29195/37962 [67:36:59<20:02:05,  8.23s/it]
 77%|███████▋  | 29196/37962 [67:37:07<20:03:30,  8.24s/it]
 77%|███████▋  | 29197/37962 [67:37:15<19:52:01,  8.16s/it]
 77%|███████▋  | 29198/37962 [67:37:24<19:53:11,  8.17s/it]
 77%|███████▋  | 29199/37962 [67:37:32<19:51:55,  8.16s/it]
 77%|███████▋  | 29200/37962 [67:37:40<20:03:42,  8.24s/it]
                                                           
{'loss': 1.1165, 'grad_norm': 19.05010223388672, 'learning_rate': 2.666359506636874e-06, 'epoch': 0.77}

 77%|███████▋  | 29200/37962 [67:37:40<20:03:42,  8.24s/it]
 77%|███████▋  | 29201/37962 [67:37:48<19:48:30,  8.14s/it]
 77%|███████▋  | 29202/37962 [67:37:56<19:53:03,  8.17s/it]
 77%|███████▋  | 29203/37962 [67:38:05<19:59:46,  8.22s/it]
 77%|███████▋  | 29204/37962 [67:38:13<19:54:48,  8.19s/it]
 77%|███████▋  | 29205/37962 [67:38:21<20:00:32,  8.23s/it]
 77%|███████▋  | 29206/37962 [67:38:29<20:01:47,  8.24s/it]
 77%|███████▋  | 29207/37962 [67:38:38<20:03:38,  8.25s/it]
 77%|███████▋  | 29208/37962 [67:38:46<20:03:59,  8.25s/it]
 77%|███████▋  | 29209/37962 [67:38:54<20:00:30,  8.23s/it]
 77%|███████▋  | 29210/37962 [67:39:02<20:03:50,  8.25s/it]
                                                           
{'loss': 1.2419, 'grad_norm': 13.392934799194336, 'learning_rate': 2.660562083454391e-06, 'epoch': 0.77}

 77%|███████▋  | 29210/37962 [67:39:02<20:03:50,  8.25s/it]
 77%|███████▋  | 29211/37962 [67:39:10<19:55:40,  8.20s/it]
 77%|███████▋  | 29212/37962 [67:39:18<19:35:57,  8.06s/it]
 77%|███████▋  | 29213/37962 [67:39:26<19:32:46,  8.04s/it]
 77%|███████▋  | 29214/37962 [67:39:34<19:34:26,  8.06s/it]
 77%|███████▋  | 29215/37962 [67:39:42<19:44:34,  8.13s/it]
 77%|███████▋  | 29216/37962 [67:39:51<19:48:11,  8.15s/it]
 77%|███████▋  | 29217/37962 [67:39:59<20:01:39,  8.24s/it]
 77%|███████▋  | 29218/37962 [67:40:07<20:03:52,  8.26s/it]
 77%|███████▋  | 29219/37962 [67:40:16<20:05:50,  8.28s/it]
 77%|███████▋  | 29220/37962 [67:40:24<20:02:54,  8.26s/it]
                                                           
{'loss': 1.2393, 'grad_norm': 9.286114692687988, 'learning_rate': 2.654770002521617e-06, 'epoch': 0.77}

 77%|███████▋  | 29220/37962 [67:40:24<20:02:54,  8.26s/it]
 77%|███████▋  | 29221/37962 [67:40:32<20:06:01,  8.28s/it]
 77%|███████▋  | 29222/37962 [67:40:40<19:57:59,  8.22s/it]
 77%|███████▋  | 29223/37962 [67:40:49<19:59:15,  8.23s/it]
 77%|███████▋  | 29224/37962 [67:40:57<19:58:34,  8.23s/it]
 77%|███████▋  | 29225/37962 [67:41:05<20:00:12,  8.24s/it]
 77%|███████▋  | 29226/37962 [67:41:14<20:07:25,  8.29s/it]
 77%|███████▋  | 29227/37962 [67:41:22<20:06:38,  8.29s/it]
 77%|███████▋  | 29228/37962 [67:41:30<20:15:04,  8.35s/it]
 77%|███████▋  | 29229/37962 [67:41:39<20:15:33,  8.35s/it]
 77%|███████▋  | 29230/37962 [67:41:47<19:58:38,  8.24s/it]
                                                           
{'loss': 1.1537, 'grad_norm': 11.260431289672852, 'learning_rate': 2.648983268054515e-06, 'epoch': 0.77}

 77%|███████▋  | 29230/37962 [67:41:47<19:58:38,  8.24s/it]
 77%|███████▋  | 29231/37962 [67:41:55<20:00:50,  8.25s/it]
 77%|███████▋  | 29232/37962 [67:42:03<20:05:57,  8.29s/it]
 77%|███████▋  | 29233/37962 [67:42:12<20:03:29,  8.27s/it]
 77%|███████▋  | 29234/37962 [67:42:20<20:12:53,  8.34s/it]
 77%|███████▋  | 29235/37962 [67:42:28<20:14:03,  8.35s/it]
 77%|███████▋  | 29236/37962 [67:42:37<20:12:46,  8.34s/it]
 77%|███████▋  | 29237/37962 [67:42:45<20:10:09,  8.32s/it]
 77%|███████▋  | 29238/37962 [67:42:54<20:17:05,  8.37s/it]
 77%|███████▋  | 29239/37962 [67:43:02<20:07:54,  8.31s/it]
 77%|███████▋  | 29240/37962 [67:43:10<20:06:06,  8.30s/it]
                                                           
{'loss': 1.168, 'grad_norm': 6.862528324127197, 'learning_rate': 2.6432018842651395e-06, 'epoch': 0.77}

 77%|███████▋  | 29240/37962 [67:43:10<20:06:06,  8.30s/it]
 77%|███████▋  | 29241/37962 [67:43:18<20:11:51,  8.34s/it]
 77%|███████▋  | 29242/37962 [67:43:27<20:04:57,  8.29s/it]
 77%|███████▋  | 29243/37962 [67:43:35<20:05:53,  8.30s/it]
 77%|███████▋  | 29244/37962 [67:43:43<19:48:42,  8.18s/it]
 77%|███████▋  | 29245/37962 [67:43:51<19:51:39,  8.20s/it]
 77%|███████▋  | 29246/37962 [67:43:59<19:53:54,  8.22s/it]
 77%|███████▋  | 29247/37962 [67:44:08<19:58:22,  8.25s/it]
 77%|███████▋  | 29248/37962 [67:44:16<19:58:38,  8.25s/it]
 77%|███████▋  | 29249/37962 [67:44:24<20:01:41,  8.28s/it]
 77%|███████▋  | 29250/37962 [67:44:32<19:50:47,  8.20s/it]
                                                           
{'loss': 1.1334, 'grad_norm': 7.785955905914307, 'learning_rate': 2.6374258553616693e-06, 'epoch': 0.77}

 77%|███████▋  | 29250/37962 [67:44:32<19:50:47,  8.20s/it]
 77%|███████▋  | 29251/37962 [67:44:41<19:57:54,  8.25s/it]
 77%|███████▋  | 29252/37962 [67:44:49<20:09:38,  8.33s/it]
 77%|███████▋  | 29253/37962 [67:44:57<20:03:17,  8.29s/it]
 77%|███████▋  | 29254/37962 [67:45:06<20:06:16,  8.31s/it]
 77%|███████▋  | 29255/37962 [67:45:14<20:01:25,  8.28s/it]
 77%|███████▋  | 29256/37962 [67:45:22<19:39:06,  8.13s/it]
 77%|███████▋  | 29257/37962 [67:45:30<19:42:28,  8.15s/it]
 77%|███████▋  | 29258/37962 [67:45:38<19:44:44,  8.17s/it]
 77%|███████▋  | 29259/37962 [67:45:46<19:44:42,  8.17s/it]
 77%|███████▋  | 29260/37962 [67:45:54<19:32:54,  8.09s/it]
                                                           
{'loss': 1.1102, 'grad_norm': 4.850693225860596, 'learning_rate': 2.631655185548371e-06, 'epoch': 0.77}

 77%|███████▋  | 29260/37962 [67:45:54<19:32:54,  8.09s/it]
 77%|███████▋  | 29261/37962 [67:46:02<19:31:19,  8.08s/it]
 77%|███████▋  | 29262/37962 [67:46:10<19:37:59,  8.12s/it]
 77%|███████▋  | 29263/37962 [67:46:19<19:38:12,  8.13s/it]
 77%|███████▋  | 29264/37962 [67:46:27<19:46:43,  8.19s/it]
 77%|███████▋  | 29265/37962 [67:46:35<19:50:22,  8.21s/it]
 77%|███████▋  | 29266/37962 [67:46:43<19:42:16,  8.16s/it]
 77%|███████▋  | 29267/37962 [67:46:51<19:44:10,  8.17s/it]
 77%|███████▋  | 29268/37962 [67:47:00<19:52:55,  8.23s/it]
 77%|███████▋  | 29269/37962 [67:47:08<19:58:35,  8.27s/it]
 77%|███████▋  | 29270/37962 [67:47:16<19:57:41,  8.27s/it]
                                                           
{'loss': 1.108, 'grad_norm': 5.961513042449951, 'learning_rate': 2.6258898790256117e-06, 'epoch': 0.77}

 77%|███████▋  | 29270/37962 [67:47:16<19:57:41,  8.27s/it]
 77%|███████▋  | 29271/37962 [67:47:25<19:52:55,  8.24s/it]
 77%|███████▋  | 29272/37962 [67:47:33<19:58:09,  8.27s/it]
 77%|███████▋  | 29273/37962 [67:47:41<19:41:05,  8.16s/it]
 77%|███████▋  | 29274/37962 [67:47:49<19:49:47,  8.22s/it]
 77%|███████▋  | 29275/37962 [67:47:57<19:47:22,  8.20s/it]
 77%|███████▋  | 29276/37962 [67:48:06<19:56:10,  8.26s/it]
 77%|███████▋  | 29277/37962 [67:48:14<19:57:12,  8.27s/it]
 77%|███████▋  | 29278/37962 [67:48:22<19:50:15,  8.22s/it]
 77%|███████▋  | 29279/37962 [67:48:30<19:52:19,  8.24s/it]
 77%|███████▋  | 29280/37962 [67:48:39<19:58:44,  8.28s/it]
                                                           
{'loss': 1.2256, 'grad_norm': 7.133718013763428, 'learning_rate': 2.620129939989865e-06, 'epoch': 0.77}

 77%|███████▋  | 29280/37962 [67:48:39<19:58:44,  8.28s/it]
 77%|███████▋  | 29281/37962 [67:48:47<20:08:13,  8.35s/it]
 77%|███████▋  | 29282/37962 [67:48:56<20:06:52,  8.34s/it]
 77%|███████▋  | 29283/37962 [67:49:04<20:09:39,  8.36s/it]
 77%|███████▋  | 29284/37962 [67:49:12<20:03:00,  8.32s/it]
 77%|███████▋  | 29285/37962 [67:49:20<19:59:46,  8.30s/it]
 77%|███████▋  | 29286/37962 [67:49:29<20:01:45,  8.31s/it]
 77%|███████▋  | 29287/37962 [67:49:37<19:58:06,  8.29s/it]
 77%|███████▋  | 29288/37962 [67:49:46<20:06:00,  8.34s/it]
 77%|███████▋  | 29289/37962 [67:49:54<20:04:27,  8.33s/it]
 77%|███████▋  | 29290/37962 [67:50:02<19:52:54,  8.25s/it]
                                                           
{'loss': 1.2005, 'grad_norm': 7.862963676452637, 'learning_rate': 2.614375372633685e-06, 'epoch': 0.77}

 77%|███████▋  | 29290/37962 [67:50:02<19:52:54,  8.25s/it]
 77%|███████▋  | 29291/37962 [67:50:10<19:43:55,  8.19s/it]
 77%|███████▋  | 29292/37962 [67:50:18<19:51:07,  8.24s/it]
 77%|███████▋  | 29293/37962 [67:50:27<20:00:34,  8.31s/it]
 77%|███████▋  | 29294/37962 [67:50:35<20:07:26,  8.36s/it]
 77%|███████▋  | 29295/37962 [67:50:44<20:06:06,  8.35s/it]
 77%|███████▋  | 29296/37962 [67:50:52<20:07:17,  8.36s/it]
 77%|███████▋  | 29297/37962 [67:51:00<20:12:11,  8.39s/it]
 77%|███████▋  | 29298/37962 [67:51:08<19:57:01,  8.29s/it]
 77%|███████▋  | 29299/37962 [67:51:16<19:44:33,  8.20s/it]
 77%|███████▋  | 29300/37962 [67:51:25<19:42:41,  8.19s/it]
                                                           
{'loss': 1.145, 'grad_norm': 11.230714797973633, 'learning_rate': 2.6086261811457234e-06, 'epoch': 0.77}

 77%|███████▋  | 29300/37962 [67:51:25<19:42:41,  8.19s/it]
 77%|███████▋  | 29301/37962 [67:51:33<19:44:30,  8.21s/it]
 77%|███████▋  | 29302/37962 [67:51:41<19:52:38,  8.26s/it]
 77%|███████▋  | 29303/37962 [67:51:49<19:33:51,  8.13s/it]
 77%|███████▋  | 29304/37962 [67:51:57<19:39:58,  8.18s/it]
 77%|███████▋  | 29305/37962 [67:52:06<19:48:32,  8.24s/it]
 77%|███████▋  | 29306/37962 [67:52:14<19:48:01,  8.23s/it]
 77%|███████▋  | 29307/37962 [67:52:22<19:45:35,  8.22s/it]
 77%|███████▋  | 29308/37962 [67:52:30<19:49:04,  8.24s/it]
 77%|███████▋  | 29309/37962 [67:52:38<19:32:13,  8.13s/it]
 77%|███████▋  | 29310/37962 [67:52:47<19:33:34,  8.14s/it]
                                                           
{'loss': 1.2441, 'grad_norm': 10.648667335510254, 'learning_rate': 2.602882369710724e-06, 'epoch': 0.77}

 77%|███████▋  | 29310/37962 [67:52:47<19:33:34,  8.14s/it]
 77%|███████▋  | 29311/37962 [67:52:55<19:36:43,  8.16s/it]
 77%|███████▋  | 29312/37962 [67:53:03<19:41:53,  8.20s/it]
 77%|███████▋  | 29313/37962 [67:53:11<19:45:18,  8.22s/it]
 77%|███████▋  | 29314/37962 [67:53:20<19:47:17,  8.24s/it]
 77%|███████▋  | 29315/37962 [67:53:28<19:50:39,  8.26s/it]
 77%|███████▋  | 29316/37962 [67:53:36<19:53:53,  8.29s/it]
 77%|███████▋  | 29317/37962 [67:53:44<19:48:50,  8.25s/it]
 77%|███████▋  | 29318/37962 [67:53:53<19:46:11,  8.23s/it]
 77%|███████▋  | 29319/37962 [67:54:01<19:58:41,  8.32s/it]
 77%|███████▋  | 29320/37962 [67:54:09<19:51:58,  8.28s/it]
                                                           
{'loss': 1.1544, 'grad_norm': 7.801405906677246, 'learning_rate': 2.597143942509496e-06, 'epoch': 0.77}

 77%|███████▋  | 29320/37962 [67:54:09<19:51:58,  8.28s/it]
 77%|███████▋  | 29321/37962 [67:54:17<19:48:20,  8.25s/it]
 77%|███████▋  | 29322/37962 [67:54:26<19:50:28,  8.27s/it]
 77%|███████▋  | 29323/37962 [67:54:34<19:58:02,  8.32s/it]
 77%|███████▋  | 29324/37962 [67:54:43<20:03:32,  8.36s/it]
 77%|███████▋  | 29325/37962 [67:54:51<19:59:34,  8.33s/it]
 77%|███████▋  | 29326/37962 [67:54:59<19:52:45,  8.29s/it]
 77%|███████▋  | 29327/37962 [67:55:07<19:37:40,  8.18s/it]
 77%|███████▋  | 29328/37962 [67:55:15<19:38:31,  8.19s/it]
 77%|███████▋  | 29329/37962 [67:55:23<19:35:58,  8.17s/it]
 77%|███████▋  | 29330/37962 [67:55:32<19:40:37,  8.21s/it]
                                                           
{'loss': 1.1998, 'grad_norm': 17.442501068115234, 'learning_rate': 2.591410903718946e-06, 'epoch': 0.77}

 77%|███████▋  | 29330/37962 [67:55:32<19:40:37,  8.21s/it]
 77%|███████▋  | 29331/37962 [67:55:40<19:51:11,  8.28s/it]
 77%|███████▋  | 29332/37962 [67:55:48<19:47:42,  8.26s/it]
 77%|███████▋  | 29333/37962 [67:55:57<19:52:48,  8.29s/it]
 77%|███████▋  | 29334/37962 [67:56:05<19:52:43,  8.29s/it]
 77%|███████▋  | 29335/37962 [67:56:13<19:53:06,  8.30s/it]
 77%|███████▋  | 29336/37962 [67:56:21<19:40:48,  8.21s/it]
 77%|███████▋  | 29337/37962 [67:56:30<19:40:18,  8.21s/it]
 77%|███████▋  | 29338/37962 [67:56:38<19:46:25,  8.25s/it]
 77%|███████▋  | 29339/37962 [67:56:46<19:39:08,  8.20s/it]
 77%|███████▋  | 29340/37962 [67:56:54<19:47:47,  8.27s/it]
                                                           
{'loss': 1.1808, 'grad_norm': 8.34951114654541, 'learning_rate': 2.585683257512057e-06, 'epoch': 0.77}

 77%|███████▋  | 29340/37962 [67:56:54<19:47:47,  8.27s/it]
 77%|███████▋  | 29341/37962 [67:57:03<19:44:33,  8.24s/it]
 77%|███████▋  | 29342/37962 [67:57:11<19:47:06,  8.26s/it]
 77%|███████▋  | 29343/37962 [67:57:19<19:51:19,  8.29s/it]
 77%|███████▋  | 29344/37962 [67:57:28<19:50:17,  8.29s/it]
 77%|███████▋  | 29345/37962 [67:57:36<19:48:42,  8.28s/it]
 77%|███████▋  | 29346/37962 [67:57:44<19:48:06,  8.27s/it]
 77%|███████▋  | 29347/37962 [67:57:52<19:42:11,  8.23s/it]
 77%|███████▋  | 29348/37962 [67:58:01<19:51:47,  8.30s/it]
 77%|███████▋  | 29349/37962 [67:58:09<19:43:30,  8.24s/it]
 77%|███████▋  | 29350/37962 [67:58:17<19:40:50,  8.23s/it]
                                                           
{'loss': 1.1223, 'grad_norm': 7.824574947357178, 'learning_rate': 2.579961008057877e-06, 'epoch': 0.77}

 77%|███████▋  | 29350/37962 [67:58:17<19:40:50,  8.23s/it]
 77%|███████▋  | 29351/37962 [67:58:25<19:41:59,  8.24s/it]
 77%|███████▋  | 29352/37962 [67:58:34<19:54:44,  8.33s/it]
 77%|███████▋  | 29353/37962 [67:58:42<19:48:04,  8.28s/it]
 77%|███████▋  | 29354/37962 [67:58:50<19:56:15,  8.34s/it]
 77%|███████▋  | 29355/37962 [67:58:59<19:53:15,  8.32s/it]
 77%|███████▋  | 29356/37962 [67:59:07<19:54:28,  8.33s/it]
 77%|███████▋  | 29357/37962 [67:59:15<19:45:28,  8.27s/it]
 77%|███████▋  | 29358/37962 [67:59:23<19:45:03,  8.26s/it]
 77%|███████▋  | 29359/37962 [67:59:32<19:44:59,  8.26s/it]
 77%|███████▋  | 29360/37962 [67:59:40<19:48:13,  8.29s/it]
                                                           
{'loss': 1.1924, 'grad_norm': 12.186628341674805, 'learning_rate': 2.5742441595215394e-06, 'epoch': 0.77}

 77%|███████▋  | 29360/37962 [67:59:40<19:48:13,  8.29s/it]
 77%|███████▋  | 29361/37962 [67:59:48<19:53:51,  8.33s/it]
 77%|███████▋  | 29362/37962 [67:59:57<19:51:33,  8.31s/it]
 77%|███████▋  | 29363/37962 [68:00:05<19:48:08,  8.29s/it]
 77%|███████▋  | 29364/37962 [68:00:13<19:42:37,  8.25s/it]
 77%|███████▋  | 29365/37962 [68:00:22<19:48:52,  8.30s/it]
 77%|███████▋  | 29366/37962 [68:00:30<19:46:34,  8.28s/it]
 77%|███████▋  | 29367/37962 [68:00:38<19:52:09,  8.32s/it]
 77%|███████▋  | 29368/37962 [68:00:46<19:48:26,  8.30s/it]
 77%|███████▋  | 29369/37962 [68:00:55<19:48:34,  8.30s/it]
 77%|███████▋  | 29370/37962 [68:01:04<20:11:24,  8.46s/it]
                                                           
{'loss': 1.1141, 'grad_norm': 7.276239395141602, 'learning_rate': 2.5685327160642357e-06, 'epoch': 0.77}

 77%|███████▋  | 29370/37962 [68:01:04<20:11:24,  8.46s/it]
 77%|███████▋  | 29371/37962 [68:01:12<20:10:46,  8.46s/it]
 77%|███████▋  | 29372/37962 [68:01:20<19:55:07,  8.35s/it]
 77%|███████▋  | 29373/37962 [68:01:28<19:56:25,  8.36s/it]
 77%|███████▋  | 29374/37962 [68:01:37<19:59:52,  8.38s/it]
 77%|███████▋  | 29375/37962 [68:01:45<19:57:28,  8.37s/it]
 77%|███████▋  | 29376/37962 [68:01:54<19:56:51,  8.36s/it]
 77%|███████▋  | 29377/37962 [68:02:02<19:52:13,  8.33s/it]
 77%|███████▋  | 29378/37962 [68:02:10<19:55:32,  8.36s/it]
 77%|███████▋  | 29379/37962 [68:02:19<19:53:45,  8.35s/it]
 77%|███████▋  | 29380/37962 [68:02:27<19:48:16,  8.31s/it]
                                                           
{'loss': 1.2721, 'grad_norm': 10.465269088745117, 'learning_rate': 2.5628266818432246e-06, 'epoch': 0.77}

 77%|███████▋  | 29380/37962 [68:02:27<19:48:16,  8.31s/it]
 77%|███████▋  | 29381/37962 [68:02:35<19:53:13,  8.34s/it]
 77%|███████▋  | 29382/37962 [68:02:44<19:51:15,  8.33s/it]
 77%|███████▋  | 29383/37962 [68:02:52<19:49:47,  8.32s/it]
 77%|███████▋  | 29384/37962 [68:03:00<19:54:27,  8.35s/it]
 77%|███████▋  | 29385/37962 [68:03:09<19:59:45,  8.39s/it]
 77%|███████▋  | 29386/37962 [68:03:17<19:46:23,  8.30s/it]
 77%|███████▋  | 29387/37962 [68:03:25<19:39:07,  8.25s/it]
 77%|███████▋  | 29388/37962 [68:03:33<19:41:16,  8.27s/it]
 77%|███████▋  | 29389/37962 [68:03:42<19:40:41,  8.26s/it]
 77%|███████▋  | 29390/37962 [68:03:50<19:32:40,  8.21s/it]
                                                           
{'loss': 1.2609, 'grad_norm': 14.45191478729248, 'learning_rate': 2.557126061011831e-06, 'epoch': 0.77}

 77%|███████▋  | 29390/37962 [68:03:50<19:32:40,  8.21s/it]
 77%|███████▋  | 29391/37962 [68:03:58<19:35:23,  8.23s/it]
 77%|███████▋  | 29392/37962 [68:04:06<19:23:31,  8.15s/it]
 77%|███████▋  | 29393/37962 [68:04:14<19:25:54,  8.16s/it]
 77%|███████▋  | 29394/37962 [68:04:22<19:29:38,  8.19s/it]
 77%|███████▋  | 29395/37962 [68:04:30<19:27:54,  8.18s/it]
 77%|███████▋  | 29396/37962 [68:04:39<19:28:43,  8.19s/it]
 77%|███████▋  | 29397/37962 [68:04:47<19:31:10,  8.20s/it]
 77%|███████▋  | 29398/37962 [68:04:55<19:42:59,  8.29s/it]
 77%|███████▋  | 29399/37962 [68:05:04<19:41:55,  8.28s/it]
 77%|███████▋  | 29400/37962 [68:05:12<19:46:44,  8.32s/it]
                                                           
{'loss': 1.1946, 'grad_norm': 19.268863677978516, 'learning_rate': 2.5514308577194413e-06, 'epoch': 0.77}

 77%|███████▋  | 29400/37962 [68:05:12<19:46:44,  8.32s/it]
 77%|███████▋  | 29401/37962 [68:05:20<19:44:33,  8.30s/it]
 77%|███████▋  | 29402/37962 [68:05:28<19:24:17,  8.16s/it]
 77%|███████▋  | 29403/37962 [68:05:37<19:35:35,  8.24s/it]
 77%|███████▋  | 29404/37962 [68:05:45<19:30:01,  8.20s/it]
 77%|███████▋  | 29405/37962 [68:05:53<19:19:39,  8.13s/it]
 77%|███████▋  | 29406/37962 [68:06:01<19:31:31,  8.22s/it]
 77%|███████▋  | 29407/37962 [68:06:09<19:30:25,  8.21s/it]
 77%|███████▋  | 29408/37962 [68:06:17<19:31:21,  8.22s/it]
 77%|███████▋  | 29409/37962 [68:06:26<19:30:01,  8.21s/it]
 77%|███████▋  | 29410/37962 [68:06:34<19:21:38,  8.15s/it]
                                                           
{'loss': 1.1466, 'grad_norm': 6.567240238189697, 'learning_rate': 2.54574107611149e-06, 'epoch': 0.77}

 77%|███████▋  | 29410/37962 [68:06:34<19:21:38,  8.15s/it]
 77%|███████▋  | 29411/37962 [68:06:42<19:23:28,  8.16s/it]
 77%|███████▋  | 29412/37962 [68:06:50<19:25:37,  8.18s/it]
 77%|███████▋  | 29413/37962 [68:06:58<19:16:27,  8.12s/it]
 77%|███████▋  | 29414/37962 [68:07:06<19:23:18,  8.17s/it]
 77%|███████▋  | 29415/37962 [68:07:15<19:31:12,  8.22s/it]
 77%|███████▋  | 29416/37962 [68:07:23<19:32:35,  8.23s/it]
 77%|███████▋  | 29417/37962 [68:07:31<19:31:54,  8.23s/it]
 77%|███████▋  | 29418/37962 [68:07:40<19:37:02,  8.27s/it]
 77%|███████▋  | 29419/37962 [68:07:48<19:33:26,  8.24s/it]
 77%|███████▋  | 29420/37962 [68:07:56<19:37:37,  8.27s/it]
                                                           
{'loss': 1.2487, 'grad_norm': 17.037254333496094, 'learning_rate': 2.5400567203294757e-06, 'epoch': 0.77}

 77%|███████▋  | 29420/37962 [68:07:56<19:37:37,  8.27s/it]
 78%|███████▊  | 29421/37962 [68:08:11<24:35:32, 10.37s/it]
 78%|███████▊  | 29422/37962 [68:08:20<23:03:56,  9.72s/it]
 78%|███████▊  | 29423/37962 [68:08:28<22:05:59,  9.32s/it]
 78%|███████▊  | 29424/37962 [68:08:36<21:17:21,  8.98s/it]
 78%|███████▊  | 29425/37962 [68:08:44<20:41:54,  8.73s/it]
 78%|███████▊  | 29426/37962 [68:08:53<20:26:30,  8.62s/it]
 78%|███████▊  | 29427/37962 [68:09:01<20:07:37,  8.49s/it]
 78%|███████▊  | 29428/37962 [68:09:09<19:55:12,  8.40s/it]
 78%|███████▊  | 29429/37962 [68:09:17<19:38:02,  8.28s/it]
 78%|███████▊  | 29430/37962 [68:09:25<19:38:01,  8.28s/it]
                                                           
{'loss': 1.1965, 'grad_norm': 6.074851989746094, 'learning_rate': 2.5343777945109404e-06, 'epoch': 0.78}

 78%|███████▊  | 29430/37962 [68:09:25<19:38:01,  8.28s/it]
 78%|███████▊  | 29431/37962 [68:09:34<19:41:52,  8.31s/it]
 78%|███████▊  | 29432/37962 [68:09:42<19:39:09,  8.29s/it]
 78%|███████▊  | 29433/37962 [68:09:50<19:19:52,  8.16s/it]
 78%|███████▊  | 29434/37962 [68:09:58<19:21:51,  8.17s/it]
 78%|███████▊  | 29435/37962 [68:10:06<19:27:35,  8.22s/it]
 78%|███████▊  | 29436/37962 [68:10:15<19:39:04,  8.30s/it]
 78%|███████▊  | 29437/37962 [68:10:23<19:40:50,  8.31s/it]
 78%|███████▊  | 29438/37962 [68:10:31<19:39:23,  8.30s/it]
 78%|███████▊  | 29439/37962 [68:10:39<19:17:02,  8.15s/it]
 78%|███████▊  | 29440/37962 [68:10:47<19:24:25,  8.20s/it]
                                                           
{'loss': 1.3025, 'grad_norm': 15.437520980834961, 'learning_rate': 2.528704302789472e-06, 'epoch': 0.78}

 78%|███████▊  | 29440/37962 [68:10:47<19:24:25,  8.20s/it]
 78%|███████▊  | 29441/37962 [68:10:56<19:25:13,  8.20s/it]
 78%|███████▊  | 29442/37962 [68:11:04<19:17:12,  8.15s/it]
 78%|███████▊  | 29443/37962 [68:11:12<19:21:11,  8.18s/it]
 78%|███████▊  | 29444/37962 [68:11:20<19:27:17,  8.22s/it]
 78%|███████▊  | 29445/37962 [68:11:29<19:30:58,  8.25s/it]
 78%|███████▊  | 29446/37962 [68:11:37<19:31:42,  8.26s/it]
 78%|███████▊  | 29447/37962 [68:11:45<19:31:44,  8.26s/it]
 78%|███████▊  | 29448/37962 [68:11:54<19:37:02,  8.29s/it]
 78%|███████▊  | 29449/37962 [68:12:02<19:24:48,  8.21s/it]
 78%|███████▊  | 29450/37962 [68:12:10<19:34:14,  8.28s/it]
                                                           
{'loss': 1.2582, 'grad_norm': 7.9437713623046875, 'learning_rate': 2.5230362492947136e-06, 'epoch': 0.78}

 78%|███████▊  | 29450/37962 [68:12:10<19:34:14,  8.28s/it]
 78%|███████▊  | 29451/37962 [68:12:18<19:18:55,  8.17s/it]
 78%|███████▊  | 29452/37962 [68:12:26<19:23:00,  8.20s/it]
 78%|███████▊  | 29453/37962 [68:12:34<19:26:33,  8.23s/it]
 78%|███████▊  | 29454/37962 [68:12:43<19:26:32,  8.23s/it]
 78%|███████▊  | 29455/37962 [68:12:51<19:27:08,  8.23s/it]
 78%|███████▊  | 29456/37962 [68:12:59<19:27:06,  8.23s/it]
 78%|███████▊  | 29457/37962 [68:13:08<19:33:52,  8.28s/it]
 78%|███████▊  | 29458/37962 [68:13:16<19:37:10,  8.31s/it]
 78%|███████▊  | 29459/37962 [68:13:24<19:27:16,  8.24s/it]
 78%|███████▊  | 29460/37962 [68:13:32<19:19:45,  8.18s/it]
                                                           
{'loss': 1.1951, 'grad_norm': 8.013522148132324, 'learning_rate': 2.5173736381523373e-06, 'epoch': 0.78}

 78%|███████▊  | 29460/37962 [68:13:32<19:19:45,  8.18s/it]
 78%|███████▊  | 29461/37962 [68:13:40<19:23:37,  8.21s/it]
 78%|███████▊  | 29462/37962 [68:13:49<19:27:52,  8.24s/it]
 78%|███████▊  | 29463/37962 [68:13:57<19:28:19,  8.25s/it]
 78%|███████▊  | 29464/37962 [68:14:05<19:22:10,  8.21s/it]
 78%|███████▊  | 29465/37962 [68:14:13<19:27:37,  8.24s/it]
 78%|███████▊  | 29466/37962 [68:14:22<19:32:30,  8.28s/it]
 78%|███████▊  | 29467/37962 [68:14:30<19:34:25,  8.29s/it]
 78%|███████▊  | 29468/37962 [68:14:38<19:30:55,  8.27s/it]
 78%|███████▊  | 29469/37962 [68:14:46<19:15:46,  8.17s/it]
 78%|███████▊  | 29470/37962 [68:14:55<19:27:39,  8.25s/it]
                                                           
{'loss': 1.1582, 'grad_norm': 12.880861282348633, 'learning_rate': 2.5117164734840636e-06, 'epoch': 0.78}

 78%|███████▊  | 29470/37962 [68:14:55<19:27:39,  8.25s/it]
 78%|███████▊  | 29471/37962 [68:15:03<19:44:16,  8.37s/it]
 78%|███████▊  | 29472/37962 [68:15:12<19:40:30,  8.34s/it]
 78%|███████▊  | 29473/37962 [68:15:20<19:46:57,  8.39s/it]
 78%|███████▊  | 29474/37962 [68:15:28<19:44:26,  8.37s/it]
 78%|███████▊  | 29475/37962 [68:15:37<19:42:32,  8.36s/it]
 78%|███████▊  | 29476/37962 [68:15:45<19:39:50,  8.34s/it]
 78%|███████▊  | 29477/37962 [68:15:53<19:18:54,  8.20s/it]
 78%|███████▊  | 29478/37962 [68:16:01<19:26:39,  8.25s/it]
 78%|███████▊  | 29479/37962 [68:16:10<19:38:24,  8.33s/it]
 78%|███████▊  | 29480/37962 [68:16:18<19:42:13,  8.36s/it]
                                                           
{'loss': 1.1545, 'grad_norm': 12.082415580749512, 'learning_rate': 2.506064759407645e-06, 'epoch': 0.78}

 78%|███████▊  | 29480/37962 [68:16:18<19:42:13,  8.36s/it]
 78%|███████▊  | 29481/37962 [68:16:27<19:42:01,  8.36s/it]
 78%|███████▊  | 29482/37962 [68:16:35<19:39:06,  8.34s/it]
 78%|███████▊  | 29483/37962 [68:16:43<19:39:30,  8.35s/it]
 78%|███████▊  | 29484/37962 [68:16:51<19:36:38,  8.33s/it]
 78%|███████▊  | 29485/37962 [68:17:00<19:33:59,  8.31s/it]
 78%|███████▊  | 29486/37962 [68:17:08<19:37:52,  8.34s/it]
 78%|███████▊  | 29487/37962 [68:17:16<19:29:55,  8.28s/it]
 78%|███████▊  | 29488/37962 [68:17:24<19:16:25,  8.19s/it]
 78%|███████▊  | 29489/37962 [68:17:33<19:24:33,  8.25s/it]
 78%|███████▊  | 29490/37962 [68:17:41<19:28:18,  8.27s/it]
                                                           
{'loss': 1.271, 'grad_norm': 5.783811092376709, 'learning_rate': 2.5004185000368608e-06, 'epoch': 0.78}

 78%|███████▊  | 29490/37962 [68:17:41<19:28:18,  8.27s/it]
 78%|███████▊  | 29491/37962 [68:17:49<19:07:35,  8.13s/it]
 78%|███████▊  | 29492/37962 [68:17:57<19:11:52,  8.16s/it]
 78%|███████▊  | 29493/37962 [68:18:05<18:58:21,  8.06s/it]
 78%|███████▊  | 29494/37962 [68:18:13<19:11:23,  8.16s/it]
 78%|███████▊  | 29495/37962 [68:18:22<19:16:53,  8.20s/it]
 78%|███████▊  | 29496/37962 [68:18:29<19:05:34,  8.12s/it]
 78%|███████▊  | 29497/37962 [68:18:38<19:14:21,  8.18s/it]
 78%|███████▊  | 29498/37962 [68:18:46<19:17:43,  8.21s/it]
 78%|███████▊  | 29499/37962 [68:18:54<19:22:56,  8.24s/it]
 78%|███████▊  | 29500/37962 [68:19:03<19:30:19,  8.30s/it]
                                                           
{'loss': 1.2646, 'grad_norm': 8.522090911865234, 'learning_rate': 2.494777699481529e-06, 'epoch': 0.78}

 78%|███████▊  | 29500/37962 [68:19:03<19:30:19,  8.30s/it]
 78%|███████▊  | 29501/37962 [68:19:11<19:27:15,  8.28s/it]
 78%|███████▊  | 29502/37962 [68:19:19<19:28:22,  8.29s/it]
 78%|███████▊  | 29503/37962 [68:19:28<19:36:53,  8.35s/it]
 78%|███████▊  | 29504/37962 [68:19:36<19:35:15,  8.34s/it]
 78%|███████▊  | 29505/37962 [68:19:45<19:35:28,  8.34s/it]
 78%|███████▊  | 29506/37962 [68:19:53<19:35:39,  8.34s/it]
 78%|███████▊  | 29507/37962 [68:20:01<19:34:10,  8.33s/it]
 78%|███████▊  | 29508/37962 [68:20:09<19:31:07,  8.31s/it]
 78%|███████▊  | 29509/37962 [68:20:18<19:28:11,  8.29s/it]
 78%|███████▊  | 29510/37962 [68:20:26<19:28:07,  8.29s/it]
                                                           
{'loss': 1.1694, 'grad_norm': 8.932174682617188, 'learning_rate': 2.489142361847493e-06, 'epoch': 0.78}

 78%|███████▊  | 29510/37962 [68:20:26<19:28:07,  8.29s/it]
 78%|███████▊  | 29511/37962 [68:20:34<19:21:54,  8.25s/it]
 78%|███████▊  | 29512/37962 [68:20:43<19:33:14,  8.33s/it]
 78%|███████▊  | 29513/37962 [68:20:51<19:25:03,  8.27s/it]
 78%|███████▊  | 29514/37962 [68:20:59<19:29:46,  8.31s/it]
 78%|███████▊  | 29515/37962 [68:21:08<19:34:15,  8.34s/it]
 78%|███████▊  | 29516/37962 [68:21:16<19:30:09,  8.31s/it]
 78%|███████▊  | 29517/37962 [68:21:24<19:12:04,  8.19s/it]
 78%|███████▊  | 29518/37962 [68:21:32<19:15:38,  8.21s/it]
 78%|███████▊  | 29519/37962 [68:21:41<19:27:55,  8.30s/it]
 78%|███████▊  | 29520/37962 [68:21:48<19:10:38,  8.18s/it]
                                                           
{'loss': 1.2839, 'grad_norm': 7.147249698638916, 'learning_rate': 2.483512491236616e-06, 'epoch': 0.78}

 78%|███████▊  | 29520/37962 [68:21:48<19:10:38,  8.18s/it]
 78%|███████▊  | 29521/37962 [68:21:56<19:04:06,  8.13s/it]
 78%|███████▊  | 29522/37962 [68:22:05<19:10:17,  8.18s/it]
 78%|███████▊  | 29523/37962 [68:22:13<19:20:57,  8.25s/it]
 78%|███████▊  | 29524/37962 [68:22:21<19:11:30,  8.19s/it]
 78%|███████▊  | 29525/37962 [68:22:29<19:07:29,  8.16s/it]
 78%|███████▊  | 29526/37962 [68:22:38<19:14:29,  8.21s/it]
 78%|███████▊  | 29527/37962 [68:22:46<19:21:02,  8.26s/it]
 78%|███████▊  | 29528/37962 [68:22:55<19:32:32,  8.34s/it]
 78%|███████▊  | 29529/37962 [68:23:03<19:30:45,  8.33s/it]
 78%|███████▊  | 29530/37962 [68:23:11<19:25:21,  8.29s/it]
                                                           
{'loss': 1.1721, 'grad_norm': 9.566771507263184, 'learning_rate': 2.4778880917467795e-06, 'epoch': 0.78}

 78%|███████▊  | 29530/37962 [68:23:11<19:25:21,  8.29s/it]
 78%|███████▊  | 29531/37962 [68:23:19<19:21:01,  8.26s/it]
 78%|███████▊  | 29532/37962 [68:23:28<19:24:31,  8.29s/it]
 78%|███████▊  | 29533/37962 [68:23:36<19:26:51,  8.31s/it]
 78%|███████▊  | 29534/37962 [68:23:44<19:24:35,  8.29s/it]
 78%|███████▊  | 29535/37962 [68:23:53<19:28:15,  8.32s/it]
 78%|███████▊  | 29536/37962 [68:24:01<19:34:16,  8.36s/it]
 78%|███████▊  | 29537/37962 [68:24:09<19:37:10,  8.38s/it]
 78%|███████▊  | 29538/37962 [68:24:17<19:20:20,  8.26s/it]
 78%|███████▊  | 29539/37962 [68:24:26<19:14:22,  8.22s/it]
 78%|███████▊  | 29540/37962 [68:24:34<19:15:31,  8.23s/it]
                                                           
{'loss': 1.1978, 'grad_norm': 9.908282279968262, 'learning_rate': 2.4722691674718923e-06, 'epoch': 0.78}

 78%|███████▊  | 29540/37962 [68:24:34<19:15:31,  8.23s/it]
 78%|███████▊  | 29541/37962 [68:24:42<19:11:27,  8.20s/it]
 78%|███████▊  | 29542/37962 [68:24:50<19:11:07,  8.20s/it]
 78%|███████▊  | 29543/37962 [68:24:58<19:11:44,  8.21s/it]
 78%|███████▊  | 29544/37962 [68:25:06<19:07:50,  8.18s/it]
 78%|███████▊  | 29545/37962 [68:25:15<19:17:20,  8.25s/it]
 78%|███████▊  | 29546/37962 [68:25:23<19:20:19,  8.27s/it]
 78%|███████▊  | 29547/37962 [68:25:32<19:26:15,  8.32s/it]
 78%|███████▊  | 29548/37962 [68:25:40<19:24:27,  8.30s/it]
 78%|███████▊  | 29549/37962 [68:25:48<19:24:15,  8.30s/it]
 78%|███████▊  | 29550/37962 [68:25:56<19:20:28,  8.28s/it]
                                                           
{'loss': 1.1948, 'grad_norm': 7.336192607879639, 'learning_rate': 2.4666557225018663e-06, 'epoch': 0.78}

 78%|███████▊  | 29550/37962 [68:25:56<19:20:28,  8.28s/it]
 78%|███████▊  | 29551/37962 [68:26:05<19:16:40,  8.25s/it]
 78%|███████▊  | 29552/37962 [68:26:13<19:15:39,  8.24s/it]
 78%|███████▊  | 29553/37962 [68:26:21<19:19:11,  8.27s/it]
 78%|███████▊  | 29554/37962 [68:26:30<19:27:38,  8.33s/it]
 78%|███████▊  | 29555/37962 [68:26:38<19:32:35,  8.37s/it]
 78%|███████▊  | 29556/37962 [68:26:46<19:29:43,  8.35s/it]
 78%|███████▊  | 29557/37962 [68:26:55<19:27:06,  8.33s/it]
 78%|███████▊  | 29558/37962 [68:27:03<19:27:49,  8.34s/it]
 78%|███████▊  | 29559/37962 [68:27:19<24:49:08, 10.63s/it]
 78%|███████▊  | 29560/37962 [68:27:27<23:14:58,  9.96s/it]
                                                           
{'loss': 1.1967, 'grad_norm': 5.521805286407471, 'learning_rate': 2.4610477609226356e-06, 'epoch': 0.78}

 78%|███████▊  | 29560/37962 [68:27:27<23:14:58,  9.96s/it]
 78%|███████▊  | 29561/37962 [68:27:36<21:58:34,  9.42s/it]
 78%|███████▊  | 29562/37962 [68:27:44<21:12:50,  9.09s/it]
 78%|███████▊  | 29563/37962 [68:27:52<20:31:22,  8.80s/it]
 78%|███████▊  | 29564/37962 [68:28:00<20:16:01,  8.69s/it]
 78%|███████▊  | 29565/37962 [68:28:09<20:00:18,  8.58s/it]
 78%|███████▊  | 29566/37962 [68:28:17<19:44:09,  8.46s/it]
 78%|███████▊  | 29567/37962 [68:28:25<19:34:55,  8.40s/it]
 78%|███████▊  | 29568/37962 [68:28:33<19:24:45,  8.33s/it]
 78%|███████▊  | 29569/37962 [68:28:42<19:20:20,  8.30s/it]
 78%|███████▊  | 29570/37962 [68:28:50<19:11:11,  8.23s/it]
                                                           
{'loss': 1.346, 'grad_norm': 8.708727836608887, 'learning_rate': 2.455445286816137e-06, 'epoch': 0.78}

 78%|███████▊  | 29570/37962 [68:28:50<19:11:11,  8.23s/it]
 78%|███████▊  | 29571/37962 [68:28:58<19:12:06,  8.24s/it]
 78%|███████▊  | 29572/37962 [68:29:06<19:23:18,  8.32s/it]
 78%|███████▊  | 29573/37962 [68:29:15<19:18:02,  8.28s/it]
 78%|███████▊  | 29574/37962 [68:29:23<19:24:29,  8.33s/it]
 78%|███████▊  | 29575/37962 [68:29:32<19:28:15,  8.36s/it]
 78%|███████▊  | 29576/37962 [68:29:40<19:18:16,  8.29s/it]
 78%|███████▊  | 29577/37962 [68:29:48<19:19:55,  8.30s/it]
 78%|███████▊  | 29578/37962 [68:29:56<19:17:54,  8.29s/it]
 78%|███████▊  | 29579/37962 [68:30:04<19:03:41,  8.19s/it]
 78%|███████▊  | 29580/37962 [68:30:12<19:02:07,  8.18s/it]
                                                           
{'loss': 1.1213, 'grad_norm': 11.33962345123291, 'learning_rate': 2.4498483042603103e-06, 'epoch': 0.78}

 78%|███████▊  | 29580/37962 [68:30:12<19:02:07,  8.18s/it]
 78%|███████▊  | 29581/37962 [68:30:21<19:05:00,  8.20s/it]
 78%|███████▊  | 29582/37962 [68:30:29<19:05:23,  8.20s/it]
 78%|███████▊  | 29583/37962 [68:30:37<19:05:52,  8.21s/it]
 78%|███████▊  | 29584/37962 [68:30:45<19:01:49,  8.18s/it]
 78%|███████▊  | 29585/37962 [68:30:54<19:12:17,  8.25s/it]
 78%|███████▊  | 29586/37962 [68:31:02<19:11:02,  8.25s/it]
 78%|███████▊  | 29587/37962 [68:31:10<19:14:41,  8.27s/it]
 78%|███████▊  | 29588/37962 [68:31:18<19:13:20,  8.26s/it]
 78%|███████▊  | 29589/37962 [68:31:27<19:12:50,  8.26s/it]
 78%|███████▊  | 29590/37962 [68:31:35<19:21:08,  8.32s/it]
                                                           
{'loss': 1.0929, 'grad_norm': 10.437017440795898, 'learning_rate': 2.444256817329109e-06, 'epoch': 0.78}

 78%|███████▊  | 29590/37962 [68:31:35<19:21:08,  8.32s/it]
 78%|███████▊  | 29591/37962 [68:31:43<19:24:51,  8.35s/it]
 78%|███████▊  | 29592/37962 [68:31:52<19:20:26,  8.32s/it]
 78%|███████▊  | 29593/37962 [68:32:00<19:13:31,  8.27s/it]
 78%|███████▊  | 29594/37962 [68:32:08<19:03:07,  8.20s/it]
 78%|███████▊  | 29595/37962 [68:32:16<19:12:50,  8.27s/it]
 78%|███████▊  | 29596/37962 [68:32:24<18:57:31,  8.16s/it]
 78%|███████▊  | 29597/37962 [68:32:32<18:48:34,  8.09s/it]
 78%|███████▊  | 29598/37962 [68:32:40<18:55:12,  8.14s/it]
 78%|███████▊  | 29599/37962 [68:32:49<19:02:54,  8.20s/it]
 78%|███████▊  | 29600/37962 [68:32:57<19:01:23,  8.19s/it]
                                                           
{'loss': 1.169, 'grad_norm': 5.898515701293945, 'learning_rate': 2.4386708300924735e-06, 'epoch': 0.78}

 78%|███████▊  | 29600/37962 [68:32:57<19:01:23,  8.19s/it]
 78%|███████▊  | 29601/37962 [68:33:05<18:57:49,  8.17s/it]
 78%|███████▊  | 29602/37962 [68:33:13<18:52:02,  8.12s/it]
 78%|███████▊  | 29603/37962 [68:33:21<18:45:15,  8.08s/it]
 78%|███████▊  | 29604/37962 [68:33:29<18:36:03,  8.01s/it]
 78%|███████▊  | 29605/37962 [68:33:37<18:32:52,  7.99s/it]
 78%|███████▊  | 29606/37962 [68:33:45<18:24:44,  7.93s/it]
 78%|███████▊  | 29607/37962 [68:33:53<18:43:24,  8.07s/it]
 78%|███████▊  | 29608/37962 [68:34:01<18:57:04,  8.17s/it]
 78%|███████▊  | 29609/37962 [68:34:10<18:56:55,  8.17s/it]
 78%|███████▊  | 29610/37962 [68:34:18<18:47:26,  8.10s/it]
                                                           
{'loss': 1.2477, 'grad_norm': 7.564632415771484, 'learning_rate': 2.433090346616348e-06, 'epoch': 0.78}

 78%|███████▊  | 29610/37962 [68:34:18<18:47:26,  8.10s/it]
 78%|███████▊  | 29611/37962 [68:34:26<18:57:11,  8.17s/it]
 78%|███████▊  | 29612/37962 [68:34:34<19:06:26,  8.24s/it]
 78%|███████▊  | 29613/37962 [68:34:42<18:46:43,  8.10s/it]
 78%|███████▊  | 29614/37962 [68:34:50<18:55:12,  8.16s/it]
 78%|███████▊  | 29615/37962 [68:34:58<18:51:06,  8.13s/it]
 78%|███████▊  | 29616/37962 [68:35:07<19:03:26,  8.22s/it]
 78%|███████▊  | 29617/37962 [68:35:15<19:05:13,  8.23s/it]
 78%|███████▊  | 29618/37962 [68:35:23<19:09:43,  8.27s/it]
 78%|███████▊  | 29619/37962 [68:35:32<19:10:35,  8.27s/it]
 78%|███████▊  | 29620/37962 [68:35:40<19:10:23,  8.27s/it]
                                                           
{'loss': 1.161, 'grad_norm': 6.833156108856201, 'learning_rate': 2.4275153709626764e-06, 'epoch': 0.78}

 78%|███████▊  | 29620/37962 [68:35:40<19:10:23,  8.27s/it]
 78%|███████▊  | 29621/37962 [68:35:48<19:06:16,  8.25s/it]
 78%|███████▊  | 29622/37962 [68:35:56<19:05:05,  8.24s/it]
 78%|███████▊  | 29623/37962 [68:36:05<19:11:12,  8.28s/it]
 78%|███████▊  | 29624/37962 [68:36:13<19:07:41,  8.26s/it]
 78%|███████▊  | 29625/37962 [68:36:21<19:03:52,  8.23s/it]
 78%|███████▊  | 29626/37962 [68:36:29<18:59:54,  8.20s/it]
 78%|███████▊  | 29627/37962 [68:36:37<18:52:28,  8.15s/it]
 78%|███████▊  | 29628/37962 [68:36:46<18:55:24,  8.17s/it]
 78%|███████▊  | 29629/37962 [68:36:54<19:00:19,  8.21s/it]
 78%|███████▊  | 29630/37962 [68:37:02<19:10:07,  8.28s/it]
                                                           
{'loss': 1.1873, 'grad_norm': 7.021008014678955, 'learning_rate': 2.4219459071893803e-06, 'epoch': 0.78}

 78%|███████▊  | 29630/37962 [68:37:02<19:10:07,  8.28s/it]
 78%|███████▊  | 29631/37962 [68:37:11<19:07:03,  8.26s/it]
 78%|███████▊  | 29632/37962 [68:37:19<19:04:59,  8.25s/it]
 78%|███████▊  | 29633/37962 [68:37:27<19:02:06,  8.23s/it]
 78%|███████▊  | 29634/37962 [68:37:35<19:09:40,  8.28s/it]
 78%|███████▊  | 29635/37962 [68:37:44<19:06:41,  8.26s/it]
 78%|███████▊  | 29636/37962 [68:37:52<19:06:49,  8.26s/it]
 78%|███████▊  | 29637/37962 [68:38:00<19:02:27,  8.23s/it]
 78%|███████▊  | 29638/37962 [68:38:08<19:00:43,  8.22s/it]
 78%|███████▊  | 29639/37962 [68:38:16<19:00:58,  8.23s/it]
 78%|███████▊  | 29640/37962 [68:38:25<19:06:05,  8.26s/it]
                                                           
{'loss': 1.142, 'grad_norm': 9.238037109375, 'learning_rate': 2.4163819593503758e-06, 'epoch': 0.78}

 78%|███████▊  | 29640/37962 [68:38:25<19:06:05,  8.26s/it]
 78%|███████▊  | 29641/37962 [68:38:33<19:07:35,  8.27s/it]
 78%|███████▊  | 29642/37962 [68:38:41<19:03:41,  8.25s/it]
 78%|███████▊  | 29643/37962 [68:38:50<19:08:53,  8.29s/it]
 78%|███████▊  | 29644/37962 [68:38:58<19:05:41,  8.26s/it]
 78%|███████▊  | 29645/37962 [68:39:06<19:07:52,  8.28s/it]
 78%|███████▊  | 29646/37962 [68:39:14<19:08:52,  8.29s/it]
 78%|███████▊  | 29647/37962 [68:39:23<18:59:46,  8.22s/it]
 78%|███████▊  | 29648/37962 [68:39:31<19:13:11,  8.32s/it]
 78%|███████▊  | 29649/37962 [68:39:39<18:58:55,  8.22s/it]
 78%|███████▊  | 29650/37962 [68:39:47<19:07:00,  8.28s/it]
                                                           
{'loss': 1.1803, 'grad_norm': 12.667337417602539, 'learning_rate': 2.4108235314955687e-06, 'epoch': 0.78}

 78%|███████▊  | 29650/37962 [68:39:47<19:07:00,  8.28s/it]
 78%|███████▊  | 29651/37962 [68:39:55<18:48:05,  8.14s/it]
 78%|███████▊  | 29652/37962 [68:40:04<18:53:09,  8.18s/it]
 78%|███████▊  | 29653/37962 [68:40:12<19:02:14,  8.25s/it]
 78%|███████▊  | 29654/37962 [68:40:20<18:59:44,  8.23s/it]
 78%|███████▊  | 29655/37962 [68:40:29<19:05:18,  8.27s/it]
 78%|███████▊  | 29656/37962 [68:40:37<19:12:44,  8.33s/it]
 78%|███████▊  | 29657/37962 [68:40:45<19:09:06,  8.30s/it]
 78%|███████▊  | 29658/37962 [68:40:53<19:06:32,  8.28s/it]
 78%|███████▊  | 29659/37962 [68:41:02<19:02:12,  8.25s/it]
 78%|███████▊  | 29660/37962 [68:41:10<19:06:58,  8.29s/it]
                                                           
{'loss': 1.2027, 'grad_norm': 8.852887153625488, 'learning_rate': 2.405270627670836e-06, 'epoch': 0.78}

 78%|███████▊  | 29660/37962 [68:41:10<19:06:58,  8.29s/it]
 78%|███████▊  | 29661/37962 [68:41:18<19:02:51,  8.26s/it]
 78%|███████▊  | 29662/37962 [68:41:26<18:48:13,  8.16s/it]
 78%|███████▊  | 29663/37962 [68:41:35<18:57:37,  8.22s/it]
 78%|███████▊  | 29664/37962 [68:41:43<19:02:52,  8.26s/it]
 78%|███████▊  | 29665/37962 [68:41:51<18:59:11,  8.24s/it]
 78%|███████▊  | 29666/37962 [68:41:59<18:56:17,  8.22s/it]
 78%|███████▊  | 29667/37962 [68:42:08<19:01:47,  8.26s/it]
 78%|███████▊  | 29668/37962 [68:42:16<18:58:35,  8.24s/it]
 78%|███████▊  | 29669/37962 [68:42:24<19:00:03,  8.25s/it]
 78%|███████▊  | 29670/37962 [68:42:32<18:45:40,  8.15s/it]
                                                           
{'loss': 1.1204, 'grad_norm': 9.440607070922852, 'learning_rate': 2.3997232519180446e-06, 'epoch': 0.78}

 78%|███████▊  | 29670/37962 [68:42:32<18:45:40,  8.15s/it]
 78%|███████▊  | 29671/37962 [68:42:40<18:51:52,  8.19s/it]
 78%|███████▊  | 29672/37962 [68:42:48<18:52:10,  8.19s/it]
 78%|███████▊  | 29673/37962 [68:42:57<19:03:34,  8.28s/it]
 78%|███████▊  | 29674/37962 [68:43:05<19:11:25,  8.34s/it]
 78%|███████▊  | 29675/37962 [68:43:14<19:11:32,  8.34s/it]
 78%|███████▊  | 29676/37962 [68:43:22<19:10:43,  8.33s/it]
 78%|███████▊  | 29677/37962 [68:43:30<19:05:44,  8.30s/it]
 78%|███████▊  | 29678/37962 [68:43:38<19:02:18,  8.27s/it]
 78%|███████▊  | 29679/37962 [68:43:47<19:00:43,  8.26s/it]
 78%|███████▊  | 29680/37962 [68:43:55<19:06:32,  8.31s/it]
                                                           
{'loss': 1.249, 'grad_norm': 6.9737091064453125, 'learning_rate': 2.394181408275037e-06, 'epoch': 0.78}

 78%|███████▊  | 29680/37962 [68:43:55<19:06:32,  8.31s/it]
 78%|███████▊  | 29681/37962 [68:44:03<19:08:11,  8.32s/it]
 78%|███████▊  | 29682/37962 [68:44:12<19:07:36,  8.32s/it]
 78%|███████▊  | 29683/37962 [68:44:20<19:07:47,  8.32s/it]
 78%|███████▊  | 29684/37962 [68:44:28<19:06:43,  8.31s/it]
 78%|███████▊  | 29685/37962 [68:44:37<19:03:05,  8.29s/it]
 78%|███████▊  | 29686/37962 [68:44:45<18:58:39,  8.26s/it]
 78%|███████▊  | 29687/37962 [68:44:53<18:59:08,  8.26s/it]
 78%|███████▊  | 29688/37962 [68:45:01<19:02:04,  8.28s/it]
 78%|███████▊  | 29689/37962 [68:45:10<18:58:51,  8.26s/it]
 78%|███████▊  | 29690/37962 [68:45:18<19:02:58,  8.29s/it]
                                                           
{'loss': 1.2429, 'grad_norm': 11.870588302612305, 'learning_rate': 2.3886451007756162e-06, 'epoch': 0.78}

 78%|███████▊  | 29690/37962 [68:45:18<19:02:58,  8.29s/it]
 78%|███████▊  | 29691/37962 [68:45:26<18:57:51,  8.25s/it]
 78%|███████▊  | 29692/37962 [68:45:35<19:03:58,  8.30s/it]
 78%|███████▊  | 29693/37962 [68:45:43<19:01:48,  8.28s/it]
 78%|███████▊  | 29694/37962 [68:45:51<19:01:05,  8.28s/it]
 78%|███████▊  | 29695/37962 [68:45:59<18:59:19,  8.27s/it]
 78%|███████▊  | 29696/37962 [68:46:08<18:56:53,  8.25s/it]
 78%|███████▊  | 29697/37962 [68:46:16<18:58:18,  8.26s/it]
 78%|███████▊  | 29698/37962 [68:46:24<19:00:39,  8.28s/it]
 78%|███████▊  | 29699/37962 [68:46:32<18:59:20,  8.27s/it]
 78%|███████▊  | 29700/37962 [68:46:41<19:01:09,  8.29s/it]
                                                           
{'loss': 1.2367, 'grad_norm': 92.65271759033203, 'learning_rate': 2.3831143334495677e-06, 'epoch': 0.78}

 78%|███████▊  | 29700/37962 [68:46:41<19:01:09,  8.29s/it]
 78%|███████▊  | 29701/37962 [68:46:49<19:10:37,  8.36s/it]
 78%|███████▊  | 29702/37962 [68:46:57<18:48:02,  8.19s/it]
 78%|███████▊  | 29703/37962 [68:47:05<18:49:51,  8.21s/it]
 78%|███████▊  | 29704/37962 [68:47:14<18:51:05,  8.22s/it]
 78%|███████▊  | 29705/37962 [68:47:22<18:54:12,  8.24s/it]
 78%|███████▊  | 29706/37962 [68:47:30<18:54:02,  8.24s/it]
 78%|███████▊  | 29707/37962 [68:47:38<18:54:04,  8.24s/it]
 78%|███████▊  | 29708/37962 [68:47:47<18:51:50,  8.23s/it]
 78%|███████▊  | 29709/37962 [68:47:55<18:43:51,  8.17s/it]
 78%|███████▊  | 29710/37962 [68:48:03<18:49:52,  8.22s/it]
                                                           
{'loss': 1.323, 'grad_norm': 7.394147872924805, 'learning_rate': 2.3775891103226454e-06, 'epoch': 0.78}

 78%|███████▊  | 29710/37962 [68:48:03<18:49:52,  8.22s/it]
 78%|███████▊  | 29711/37962 [68:48:11<18:48:36,  8.21s/it]
 78%|███████▊  | 29712/37962 [68:48:19<18:50:34,  8.22s/it]
 78%|███████▊  | 29713/37962 [68:48:27<18:44:48,  8.18s/it]
 78%|███████▊  | 29714/37962 [68:48:36<19:00:55,  8.30s/it]
 78%|███████▊  | 29715/37962 [68:48:44<19:00:56,  8.30s/it]
 78%|███████▊  | 29716/37962 [68:48:53<18:57:19,  8.28s/it]
 78%|███████▊  | 29717/37962 [68:49:01<18:51:42,  8.24s/it]
 78%|███████▊  | 29718/37962 [68:49:09<18:52:46,  8.24s/it]
 78%|███████▊  | 29719/37962 [68:49:17<18:48:49,  8.22s/it]
 78%|███████▊  | 29720/37962 [68:49:25<18:47:17,  8.21s/it]
                                                           
{'loss': 1.3226, 'grad_norm': 13.865079879760742, 'learning_rate': 2.3720694354165584e-06, 'epoch': 0.78}

 78%|███████▊  | 29720/37962 [68:49:25<18:47:17,  8.21s/it]
 78%|███████▊  | 29721/37962 [68:49:34<18:52:05,  8.24s/it]
 78%|███████▊  | 29722/37962 [68:49:42<18:54:33,  8.26s/it]
 78%|███████▊  | 29723/37962 [68:49:50<18:57:18,  8.28s/it]
 78%|███████▊  | 29724/37962 [68:49:58<18:49:07,  8.22s/it]
 78%|███████▊  | 29725/37962 [68:50:06<18:39:28,  8.15s/it]
 78%|███████▊  | 29726/37962 [68:50:15<18:48:56,  8.22s/it]
 78%|███████▊  | 29727/37962 [68:50:23<19:00:27,  8.31s/it]
 78%|███████▊  | 29728/37962 [68:50:31<19:00:04,  8.31s/it]
 78%|███████▊  | 29729/37962 [68:50:40<18:57:59,  8.29s/it]
 78%|███████▊  | 29730/37962 [68:50:48<19:01:43,  8.32s/it]
                                                           
{'loss': 1.1864, 'grad_norm': 8.259990692138672, 'learning_rate': 2.366555312748986e-06, 'epoch': 0.78}

 78%|███████▊  | 29730/37962 [68:50:48<19:01:43,  8.32s/it]
 78%|███████▊  | 29731/37962 [68:50:56<18:48:15,  8.22s/it]
 78%|███████▊  | 29732/37962 [68:51:04<18:51:28,  8.25s/it]
 78%|███████▊  | 29733/37962 [68:51:12<18:41:38,  8.18s/it]
 78%|███████▊  | 29734/37962 [68:51:21<18:36:26,  8.14s/it]
 78%|███████▊  | 29735/37962 [68:51:29<18:41:47,  8.18s/it]
 78%|███████▊  | 29736/37962 [68:51:37<18:53:18,  8.27s/it]
 78%|███████▊  | 29737/37962 [68:51:45<18:37:53,  8.15s/it]
 78%|███████▊  | 29738/37962 [68:51:53<18:38:25,  8.16s/it]
 78%|███████▊  | 29739/37962 [68:52:02<18:48:23,  8.23s/it]
 78%|███████▊  | 29740/37962 [68:52:09<18:28:35,  8.09s/it]
                                                           
{'loss': 1.1787, 'grad_norm': 11.749423027038574, 'learning_rate': 2.3610467463335627e-06, 'epoch': 0.78}

 78%|███████▊  | 29740/37962 [68:52:09<18:28:35,  8.09s/it]
 78%|███████▊  | 29741/37962 [68:52:18<18:32:45,  8.12s/it]
 78%|███████▊  | 29742/37962 [68:52:26<18:25:02,  8.07s/it]
 78%|███████▊  | 29743/37962 [68:52:34<18:42:37,  8.20s/it]
 78%|███████▊  | 29744/37962 [68:52:42<18:43:40,  8.20s/it]
 78%|███████▊  | 29745/37962 [68:52:51<18:48:01,  8.24s/it]
 78%|███████▊  | 29746/37962 [68:52:59<18:52:25,  8.27s/it]
 78%|███████▊  | 29747/37962 [68:53:07<18:55:48,  8.30s/it]
 78%|███████▊  | 29748/37962 [68:53:16<18:55:23,  8.29s/it]
 78%|███████▊  | 29749/37962 [68:53:24<18:52:41,  8.27s/it]
 78%|███████▊  | 29750/37962 [68:53:32<18:55:34,  8.30s/it]
                                                           
{'loss': 1.2101, 'grad_norm': 6.162956714630127, 'learning_rate': 2.355543740179873e-06, 'epoch': 0.78}

 78%|███████▊  | 29750/37962 [68:53:32<18:55:34,  8.30s/it]
 78%|███████▊  | 29751/37962 [68:53:41<18:58:26,  8.32s/it]
 78%|███████▊  | 29752/37962 [68:53:49<18:54:01,  8.29s/it]
 78%|███████▊  | 29753/37962 [68:53:57<18:43:32,  8.21s/it]
 78%|███████▊  | 29754/37962 [68:54:05<18:46:56,  8.24s/it]
 78%|███████▊  | 29755/37962 [68:54:13<18:46:54,  8.24s/it]
 78%|███████▊  | 29756/37962 [68:54:22<18:50:25,  8.27s/it]
 78%|███████▊  | 29757/37962 [68:54:30<18:52:49,  8.28s/it]
 78%|███████▊  | 29758/37962 [68:54:38<18:47:58,  8.25s/it]
 78%|███████▊  | 29759/37962 [68:54:47<18:50:24,  8.27s/it]
 78%|███████▊  | 29760/37962 [68:54:55<18:48:24,  8.25s/it]
                                                           
{'loss': 1.1573, 'grad_norm': 5.713232040405273, 'learning_rate': 2.350046298293467e-06, 'epoch': 0.78}

 78%|███████▊  | 29760/37962 [68:54:55<18:48:24,  8.25s/it]
 78%|███████▊  | 29761/37962 [68:55:03<18:48:28,  8.26s/it]
 78%|███████▊  | 29762/37962 [68:55:11<18:47:45,  8.25s/it]
 78%|███████▊  | 29763/37962 [68:55:19<18:47:22,  8.25s/it]
 78%|███████▊  | 29764/37962 [68:55:28<18:51:40,  8.28s/it]
 78%|███████▊  | 29765/37962 [68:55:36<18:43:50,  8.23s/it]
 78%|███████▊  | 29766/37962 [68:55:44<18:43:19,  8.22s/it]
 78%|███████▊  | 29767/37962 [68:55:52<18:42:08,  8.22s/it]
 78%|███████▊  | 29768/37962 [68:56:01<18:48:22,  8.26s/it]
 78%|███████▊  | 29769/37962 [68:56:09<18:52:56,  8.30s/it]
 78%|███████▊  | 29770/37962 [68:56:17<18:35:33,  8.17s/it]
                                                           
{'loss': 1.2584, 'grad_norm': 15.664694786071777, 'learning_rate': 2.3445544246758313e-06, 'epoch': 0.78}

 78%|███████▊  | 29770/37962 [68:56:17<18:35:33,  8.17s/it]
 78%|███████▊  | 29771/37962 [68:56:25<18:39:53,  8.20s/it]
 78%|███████▊  | 29772/37962 [68:56:34<18:51:58,  8.29s/it]
 78%|███████▊  | 29773/37962 [68:56:42<18:47:53,  8.26s/it]
 78%|███████▊  | 29774/37962 [68:56:50<18:34:01,  8.16s/it]
 78%|███████▊  | 29775/37962 [68:56:58<18:33:18,  8.16s/it]
 78%|███████▊  | 29776/37962 [68:57:06<18:40:46,  8.21s/it]
 78%|███████▊  | 29777/37962 [68:57:15<18:43:16,  8.23s/it]
 78%|███████▊  | 29778/37962 [68:57:22<18:21:53,  8.08s/it]
 78%|███████▊  | 29779/37962 [68:57:31<18:24:13,  8.10s/it]
 78%|███████▊  | 29780/37962 [68:57:39<18:41:06,  8.22s/it]
                                                           
{'loss': 1.2127, 'grad_norm': 9.85895824432373, 'learning_rate': 2.3390681233244105e-06, 'epoch': 0.78}

 78%|███████▊  | 29780/37962 [68:57:39<18:41:06,  8.22s/it]
 78%|███████▊  | 29781/37962 [68:57:47<18:41:44,  8.23s/it]
 78%|███████▊  | 29782/37962 [68:57:55<18:40:27,  8.22s/it]
 78%|███████▊  | 29783/37962 [68:58:04<18:44:24,  8.25s/it]
 78%|███████▊  | 29784/37962 [68:58:12<18:42:51,  8.24s/it]
 78%|███████▊  | 29785/37962 [68:58:20<18:41:53,  8.23s/it]
 78%|███████▊  | 29786/37962 [68:58:28<18:34:25,  8.18s/it]
 78%|███████▊  | 29787/37962 [68:58:36<18:31:33,  8.16s/it]
 78%|███████▊  | 29788/37962 [68:58:45<18:30:32,  8.15s/it]
 78%|███████▊  | 29789/37962 [68:58:53<18:47:57,  8.28s/it]
 78%|███████▊  | 29790/37962 [68:59:02<18:58:29,  8.36s/it]
                                                           
{'loss': 1.1112, 'grad_norm': 12.996437072753906, 'learning_rate': 2.333587398232586e-06, 'epoch': 0.78}

 78%|███████▊  | 29790/37962 [68:59:02<18:58:29,  8.36s/it]
 78%|███████▊  | 29791/37962 [68:59:10<18:51:10,  8.31s/it]
 78%|███████▊  | 29792/37962 [68:59:18<18:52:21,  8.32s/it]
 78%|███████▊  | 29793/37962 [68:59:26<18:48:51,  8.29s/it]
 78%|███████▊  | 29794/37962 [68:59:35<18:50:04,  8.30s/it]
 78%|███████▊  | 29795/37962 [68:59:43<18:47:35,  8.28s/it]
 78%|███████▊  | 29796/37962 [68:59:51<18:41:51,  8.24s/it]
 78%|███████▊  | 29797/37962 [68:59:59<18:43:35,  8.26s/it]
 78%|███████▊  | 29798/37962 [69:00:08<18:50:01,  8.30s/it]
 78%|███████▊  | 29799/37962 [69:00:16<18:31:59,  8.17s/it]
 78%|███████▊  | 29800/37962 [69:00:24<18:37:40,  8.22s/it]
                                                           
{'loss': 1.154, 'grad_norm': 4.515507698059082, 'learning_rate': 2.3281122533896814e-06, 'epoch': 0.78}

 78%|███████▊  | 29800/37962 [69:00:24<18:37:40,  8.22s/it]
 79%|███████▊  | 29801/37962 [69:00:32<18:39:11,  8.23s/it]
 79%|███████▊  | 29802/37962 [69:00:40<18:35:51,  8.20s/it]
 79%|███████▊  | 29803/37962 [69:00:49<18:40:19,  8.24s/it]
 79%|███████▊  | 29804/37962 [69:00:57<18:42:51,  8.26s/it]
 79%|███████▊  | 29805/37962 [69:01:06<18:52:23,  8.33s/it]
 79%|███████▊  | 29806/37962 [69:01:13<18:36:33,  8.21s/it]
 79%|███████▊  | 29807/37962 [69:01:22<18:39:46,  8.24s/it]
 79%|███████▊  | 29808/37962 [69:01:30<18:33:18,  8.19s/it]
 79%|███████▊  | 29809/37962 [69:01:38<18:29:30,  8.17s/it]
 79%|███████▊  | 29810/37962 [69:01:46<18:39:01,  8.24s/it]
                                                           
{'loss': 1.1639, 'grad_norm': 14.356064796447754, 'learning_rate': 2.322642692780961e-06, 'epoch': 0.79}

 79%|███████▊  | 29810/37962 [69:01:46<18:39:01,  8.24s/it]
 79%|███████▊  | 29811/37962 [69:01:54<18:22:37,  8.12s/it]
 79%|███████▊  | 29812/37962 [69:02:03<18:32:56,  8.19s/it]
 79%|███████▊  | 29813/37962 [69:02:11<18:30:25,  8.18s/it]
 79%|███████▊  | 29814/37962 [69:02:19<18:31:50,  8.19s/it]
 79%|███████▊  | 29815/37962 [69:02:27<18:43:24,  8.27s/it]
 79%|███████▊  | 29816/37962 [69:02:36<18:56:37,  8.37s/it]
 79%|███████▊  | 29817/37962 [69:02:44<18:55:06,  8.36s/it]
 79%|███████▊  | 29818/37962 [69:02:53<18:49:34,  8.32s/it]
 79%|███████▊  | 29819/37962 [69:03:01<18:54:41,  8.36s/it]
 79%|███████▊  | 29820/37962 [69:03:09<18:51:22,  8.34s/it]
                                                           
{'loss': 1.203, 'grad_norm': 12.914161682128906, 'learning_rate': 2.317178720387625e-06, 'epoch': 0.79}

 79%|███████▊  | 29820/37962 [69:03:09<18:51:22,  8.34s/it]
 79%|███████▊  | 29821/37962 [69:03:18<18:49:19,  8.32s/it]
 79%|███████▊  | 29822/37962 [69:03:25<18:33:00,  8.20s/it]
 79%|███████▊  | 29823/37962 [69:03:34<18:38:34,  8.25s/it]
 79%|███████▊  | 29824/37962 [69:03:42<18:41:48,  8.27s/it]
 79%|███████▊  | 29825/37962 [69:03:50<18:31:37,  8.20s/it]
 79%|███████▊  | 29826/37962 [69:03:59<18:37:52,  8.24s/it]
 79%|███████▊  | 29827/37962 [69:04:07<18:41:06,  8.27s/it]
 79%|███████▊  | 29828/37962 [69:04:15<18:37:45,  8.25s/it]
 79%|███████▊  | 29829/37962 [69:04:24<18:45:52,  8.31s/it]
 79%|███████▊  | 29830/37962 [69:04:32<18:48:34,  8.33s/it]
                                                           
{'loss': 1.1648, 'grad_norm': 8.196971893310547, 'learning_rate': 2.3117203401868003e-06, 'epoch': 0.79}

 79%|███████▊  | 29830/37962 [69:04:32<18:48:34,  8.33s/it]
 79%|███████▊  | 29831/37962 [69:04:40<18:47:14,  8.32s/it]
 79%|███████▊  | 29832/37962 [69:04:49<18:51:08,  8.35s/it]
 79%|███████▊  | 29833/37962 [69:04:57<18:49:30,  8.34s/it]
 79%|███████▊  | 29834/37962 [69:05:05<18:46:04,  8.31s/it]
 79%|███████▊  | 29835/37962 [69:05:13<18:42:46,  8.29s/it]
 79%|███████▊  | 29836/37962 [69:05:22<18:51:44,  8.36s/it]
 79%|███████▊  | 29837/37962 [69:05:30<18:38:50,  8.26s/it]
 79%|███████▊  | 29838/37962 [69:05:38<18:43:06,  8.29s/it]
 79%|███████▊  | 29839/37962 [69:05:47<18:46:37,  8.32s/it]
 79%|███████▊  | 29840/37962 [69:05:55<18:44:29,  8.31s/it]
                                                           
{'loss': 1.2294, 'grad_norm': 14.145028114318848, 'learning_rate': 2.3062675561515536e-06, 'epoch': 0.79}

 79%|███████▊  | 29840/37962 [69:05:55<18:44:29,  8.31s/it]
 79%|███████▊  | 29841/37962 [69:06:04<18:55:15,  8.39s/it]
 79%|███████▊  | 29842/37962 [69:06:12<18:55:22,  8.39s/it]
 79%|███████▊  | 29843/37962 [69:06:20<18:52:08,  8.37s/it]
 79%|███████▊  | 29844/37962 [69:06:29<18:46:53,  8.33s/it]
 79%|███████▊  | 29845/37962 [69:06:37<18:45:38,  8.32s/it]
 79%|███████▊  | 29846/37962 [69:06:45<18:46:49,  8.33s/it]
 79%|███████▊  | 29847/37962 [69:06:53<18:37:14,  8.26s/it]
 79%|███████▊  | 29848/37962 [69:07:01<18:31:52,  8.22s/it]
 79%|███████▊  | 29849/37962 [69:07:10<18:41:06,  8.29s/it]
 79%|███████▊  | 29850/37962 [69:07:18<18:40:43,  8.29s/it]
                                                           
{'loss': 1.2127, 'grad_norm': 8.980778694152832, 'learning_rate': 2.300820372250867e-06, 'epoch': 0.79}

 79%|███████▊  | 29850/37962 [69:07:18<18:40:43,  8.29s/it]
 79%|███████▊  | 29851/37962 [69:07:26<18:36:48,  8.26s/it]
 79%|███████▊  | 29852/37962 [69:07:35<18:43:02,  8.31s/it]
 79%|███████▊  | 29853/37962 [69:07:43<18:36:07,  8.26s/it]
 79%|███████▊  | 29854/37962 [69:07:51<18:18:44,  8.13s/it]
 79%|███████▊  | 29855/37962 [69:07:59<18:09:46,  8.07s/it]
 79%|███████▊  | 29856/37962 [69:08:07<18:17:05,  8.12s/it]
 79%|███████▊  | 29857/37962 [69:08:15<18:15:41,  8.11s/it]
 79%|███████▊  | 29858/37962 [69:08:23<18:20:43,  8.15s/it]
 79%|███████▊  | 29859/37962 [69:08:31<18:24:54,  8.18s/it]
 79%|███████▊  | 29860/37962 [69:08:40<18:27:29,  8.20s/it]
                                                           
{'loss': 1.1653, 'grad_norm': 10.700285911560059, 'learning_rate': 2.2953787924496494e-06, 'epoch': 0.79}

 79%|███████▊  | 29860/37962 [69:08:40<18:27:29,  8.20s/it]
 79%|███████▊  | 29861/37962 [69:08:48<18:33:18,  8.25s/it]
 79%|███████▊  | 29862/37962 [69:08:56<18:34:36,  8.26s/it]
 79%|███████▊  | 29863/37962 [69:09:05<18:34:14,  8.25s/it]
 79%|███████▊  | 29864/37962 [69:09:13<18:27:33,  8.21s/it]
 79%|███████▊  | 29865/37962 [69:09:21<18:33:40,  8.25s/it]
 79%|███████▊  | 29866/37962 [69:09:29<18:32:15,  8.24s/it]
 79%|███████▊  | 29867/37962 [69:09:38<18:40:37,  8.31s/it]
 79%|███████▊  | 29868/37962 [69:09:46<18:32:25,  8.25s/it]
 79%|███████▊  | 29869/37962 [69:09:54<18:44:10,  8.33s/it]
 79%|███████▊  | 29870/37962 [69:10:03<18:41:51,  8.32s/it]
                                                           
{'loss': 1.2301, 'grad_norm': 7.736145496368408, 'learning_rate': 2.289942820708738e-06, 'epoch': 0.79}

 79%|███████▊  | 29870/37962 [69:10:03<18:41:51,  8.32s/it]
 79%|███████▊  | 29871/37962 [69:10:11<18:35:56,  8.28s/it]
 79%|███████▊  | 29872/37962 [69:10:19<18:36:10,  8.28s/it]
 79%|███████▊  | 29873/37962 [69:10:27<18:37:18,  8.29s/it]
 79%|███████▊  | 29874/37962 [69:10:36<18:45:33,  8.35s/it]
 79%|███████▊  | 29875/37962 [69:10:44<18:33:42,  8.26s/it]
 79%|███████▊  | 29876/37962 [69:10:52<18:41:34,  8.32s/it]
 79%|███████▊  | 29877/37962 [69:11:01<18:39:30,  8.31s/it]
 79%|███████▊  | 29878/37962 [69:11:09<18:38:05,  8.30s/it]
 79%|███████▊  | 29879/37962 [69:11:17<18:35:14,  8.28s/it]
 79%|███████▊  | 29880/37962 [69:11:25<18:33:58,  8.27s/it]
                                                           
{'loss': 1.1063, 'grad_norm': 7.432765483856201, 'learning_rate': 2.284512460984878e-06, 'epoch': 0.79}

 79%|███████▊  | 29880/37962 [69:11:25<18:33:58,  8.27s/it]
 79%|███████▊  | 29881/37962 [69:11:34<18:31:54,  8.26s/it]
 79%|███████▊  | 29882/37962 [69:11:42<18:15:06,  8.13s/it]
 79%|███████▊  | 29883/37962 [69:11:50<18:17:56,  8.15s/it]
 79%|███████▊  | 29884/37962 [69:11:58<18:15:46,  8.14s/it]
 79%|███████▊  | 29885/37962 [69:12:06<18:25:37,  8.21s/it]
 79%|███████▊  | 29886/37962 [69:12:14<18:21:58,  8.19s/it]
 79%|███████▊  | 29887/37962 [69:12:23<18:33:33,  8.27s/it]
 79%|███████▊  | 29888/37962 [69:12:31<18:40:15,  8.32s/it]
 79%|███████▊  | 29889/37962 [69:12:39<18:29:53,  8.25s/it]
 79%|███████▊  | 29890/37962 [69:12:48<18:28:50,  8.24s/it]
                                                           
{'loss': 1.2144, 'grad_norm': 5.816494941711426, 'learning_rate': 2.279087717230738e-06, 'epoch': 0.79}

 79%|███████▊  | 29890/37962 [69:12:48<18:28:50,  8.24s/it]
 79%|███████▊  | 29891/37962 [69:12:55<18:15:04,  8.14s/it]
 79%|███████▊  | 29892/37962 [69:13:04<18:15:49,  8.15s/it]
 79%|███████▊  | 29893/37962 [69:13:12<18:25:34,  8.22s/it]
 79%|███████▊  | 29894/37962 [69:13:20<18:22:54,  8.20s/it]
 79%|███████▊  | 29895/37962 [69:13:29<18:31:37,  8.27s/it]
 79%|███████▉  | 29896/37962 [69:13:37<18:37:08,  8.31s/it]
 79%|███████▉  | 29897/37962 [69:13:45<18:37:00,  8.31s/it]
 79%|███████▉  | 29898/37962 [69:13:54<18:35:31,  8.30s/it]
 79%|███████▉  | 29899/37962 [69:14:02<18:38:12,  8.32s/it]
 79%|███████▉  | 29900/37962 [69:14:10<18:39:35,  8.33s/it]
                                                           
{'loss': 1.1636, 'grad_norm': 11.162300109863281, 'learning_rate': 2.27366859339489e-06, 'epoch': 0.79}

 79%|███████▉  | 29900/37962 [69:14:10<18:39:35,  8.33s/it]
 79%|███████▉  | 29901/37962 [69:14:19<18:33:33,  8.29s/it]
 79%|███████▉  | 29902/37962 [69:14:27<18:37:12,  8.32s/it]
 79%|███████▉  | 29903/37962 [69:14:36<18:51:14,  8.42s/it]
 79%|███████▉  | 29904/37962 [69:14:44<18:38:49,  8.33s/it]
 79%|███████▉  | 29905/37962 [69:14:52<18:35:27,  8.31s/it]
 79%|███████▉  | 29906/37962 [69:15:00<18:33:40,  8.29s/it]
 79%|███████▉  | 29907/37962 [69:15:08<18:25:46,  8.24s/it]
 79%|███████▉  | 29908/37962 [69:15:17<18:24:29,  8.23s/it]
 79%|███████▉  | 29909/37962 [69:15:25<18:23:13,  8.22s/it]
 79%|███████▉  | 29910/37962 [69:15:33<18:09:30,  8.12s/it]
                                                           
{'loss': 1.136, 'grad_norm': 8.670113563537598, 'learning_rate': 2.2682550934218262e-06, 'epoch': 0.79}

 79%|███████▉  | 29910/37962 [69:15:33<18:09:30,  8.12s/it]
 79%|███████▉  | 29911/37962 [69:15:40<18:00:25,  8.05s/it]
 79%|███████▉  | 29912/37962 [69:15:49<18:16:15,  8.17s/it]
 79%|███████▉  | 29913/37962 [69:15:57<18:02:46,  8.07s/it]
 79%|███████▉  | 29914/37962 [69:16:05<18:13:36,  8.15s/it]
 79%|███████▉  | 29915/37962 [69:16:13<18:16:36,  8.18s/it]
 79%|███████▉  | 29916/37962 [69:16:22<18:17:24,  8.18s/it]
 79%|███████▉  | 29917/37962 [69:16:38<23:42:29, 10.61s/it]
 79%|███████▉  | 29918/37962 [69:16:46<22:19:06,  9.99s/it]
 79%|███████▉  | 29919/37962 [69:16:55<21:13:43,  9.50s/it]
 79%|███████▉  | 29920/37962 [69:17:03<20:26:18,  9.15s/it]
                                                           
{'loss': 1.2706, 'grad_norm': 8.336100578308105, 'learning_rate': 2.262847221251935e-06, 'epoch': 0.79}

 79%|███████▉  | 29920/37962 [69:17:03<20:26:18,  9.15s/it]
 79%|███████▉  | 29921/37962 [69:17:11<19:47:13,  8.86s/it]
 79%|███████▉  | 29922/37962 [69:17:19<19:22:29,  8.68s/it]
 79%|███████▉  | 29923/37962 [69:17:28<19:04:50,  8.54s/it]
 79%|███████▉  | 29924/37962 [69:17:36<18:58:08,  8.50s/it]
 79%|███████▉  | 29925/37962 [69:17:44<18:41:25,  8.37s/it]
 79%|███████▉  | 29926/37962 [69:17:52<18:36:41,  8.34s/it]
 79%|███████▉  | 29927/37962 [69:18:01<18:32:23,  8.31s/it]
 79%|███████▉  | 29928/37962 [69:18:09<18:35:19,  8.33s/it]
 79%|███████▉  | 29929/37962 [69:18:17<18:36:34,  8.34s/it]
 79%|███████▉  | 29930/37962 [69:18:26<18:28:56,  8.28s/it]
                                                           
{'loss': 1.1808, 'grad_norm': 15.423498153686523, 'learning_rate': 2.257444980821516e-06, 'epoch': 0.79}

 79%|███████▉  | 29930/37962 [69:18:26<18:28:56,  8.28s/it]
 79%|███████▉  | 29931/37962 [69:18:33<18:12:10,  8.16s/it]
 79%|███████▉  | 29932/37962 [69:18:42<18:20:50,  8.23s/it]
 79%|███████▉  | 29933/37962 [69:18:50<18:11:57,  8.16s/it]
 79%|███████▉  | 29934/37962 [69:18:58<17:57:45,  8.05s/it]
 79%|███████▉  | 29935/37962 [69:19:06<18:02:07,  8.09s/it]
 79%|███████▉  | 29936/37962 [69:19:14<18:10:08,  8.15s/it]
 79%|███████▉  | 29937/37962 [69:19:22<18:17:29,  8.21s/it]
 79%|███████▉  | 29938/37962 [69:19:31<18:22:12,  8.24s/it]
 79%|███████▉  | 29939/37962 [69:19:39<18:22:52,  8.25s/it]
 79%|███████▉  | 29940/37962 [69:19:47<18:25:29,  8.27s/it]
                                                           
{'loss': 1.365, 'grad_norm': 7.66867733001709, 'learning_rate': 2.2520483760627642e-06, 'epoch': 0.79}

 79%|███████▉  | 29940/37962 [69:19:47<18:25:29,  8.27s/it]
 79%|███████▉  | 29941/37962 [69:20:03<23:28:14, 10.53s/it]
 79%|███████▉  | 29942/37962 [69:20:12<22:03:34,  9.90s/it]
 79%|███████▉  | 29943/37962 [69:20:20<20:59:21,  9.42s/it]
 79%|███████▉  | 29944/37962 [69:20:28<20:06:15,  9.03s/it]
 79%|███████▉  | 29945/37962 [69:20:36<19:43:52,  8.86s/it]
 79%|███████▉  | 29946/37962 [69:20:45<19:17:51,  8.67s/it]
 79%|███████▉  | 29947/37962 [69:20:53<18:59:37,  8.53s/it]
 79%|███████▉  | 29948/37962 [69:21:01<18:51:59,  8.48s/it]
 79%|███████▉  | 29949/37962 [69:21:10<18:49:28,  8.46s/it]
 79%|███████▉  | 29950/37962 [69:21:18<18:35:59,  8.36s/it]
                                                           
{'loss': 1.1725, 'grad_norm': 5.870673656463623, 'learning_rate': 2.2466574109037743e-06, 'epoch': 0.79}

 79%|███████▉  | 29950/37962 [69:21:18<18:35:59,  8.36s/it]
 79%|███████▉  | 29951/37962 [69:21:26<18:30:55,  8.32s/it]
 79%|███████▉  | 29952/37962 [69:21:34<18:35:22,  8.35s/it]
 79%|███████▉  | 29953/37962 [69:21:43<18:30:01,  8.32s/it]
 79%|███████▉  | 29954/37962 [69:21:51<18:26:31,  8.29s/it]
 79%|███████▉  | 29955/37962 [69:21:59<18:14:35,  8.20s/it]
 79%|███████▉  | 29956/37962 [69:22:07<18:28:03,  8.30s/it]
 79%|███████▉  | 29957/37962 [69:22:16<18:27:42,  8.30s/it]
 79%|███████▉  | 29958/37962 [69:22:24<18:26:12,  8.29s/it]
 79%|███████▉  | 29959/37962 [69:22:32<18:26:01,  8.29s/it]
 79%|███████▉  | 29960/37962 [69:22:41<18:22:03,  8.26s/it]
                                                           
{'loss': 1.1429, 'grad_norm': 8.252575874328613, 'learning_rate': 2.241272089268539e-06, 'epoch': 0.79}

 79%|███████▉  | 29960/37962 [69:22:41<18:22:03,  8.26s/it]
 79%|███████▉  | 29961/37962 [69:22:49<18:17:33,  8.23s/it]
 79%|███████▉  | 29962/37962 [69:22:57<18:19:10,  8.24s/it]
 79%|███████▉  | 29963/37962 [69:23:05<18:27:58,  8.31s/it]
 79%|███████▉  | 29964/37962 [69:23:14<18:22:55,  8.27s/it]
 79%|███████▉  | 29965/37962 [69:23:22<18:26:52,  8.30s/it]
 79%|███████▉  | 29966/37962 [69:23:30<18:27:35,  8.31s/it]
 79%|███████▉  | 29967/37962 [69:23:39<18:24:04,  8.29s/it]
 79%|███████▉  | 29968/37962 [69:23:47<18:22:31,  8.28s/it]
 79%|███████▉  | 29969/37962 [69:23:55<18:25:44,  8.30s/it]
 79%|███████▉  | 29970/37962 [69:24:03<18:15:38,  8.23s/it]
                                                           
{'loss': 1.1883, 'grad_norm': 14.552165031433105, 'learning_rate': 2.235892415076938e-06, 'epoch': 0.79}

 79%|███████▉  | 29970/37962 [69:24:03<18:15:38,  8.23s/it]
 79%|███████▉  | 29971/37962 [69:24:11<18:15:32,  8.23s/it]
 79%|███████▉  | 29972/37962 [69:24:19<18:04:54,  8.15s/it]
 79%|███████▉  | 29973/37962 [69:24:28<18:11:29,  8.20s/it]
 79%|███████▉  | 29974/37962 [69:24:36<18:24:23,  8.30s/it]
 79%|███████▉  | 29975/37962 [69:24:44<18:10:05,  8.19s/it]
 79%|███████▉  | 29976/37962 [69:24:52<18:11:15,  8.20s/it]
 79%|███████▉  | 29977/37962 [69:25:01<18:14:38,  8.23s/it]
 79%|███████▉  | 29978/37962 [69:25:09<18:19:31,  8.26s/it]
 79%|███████▉  | 29979/37962 [69:25:17<18:12:06,  8.21s/it]
 79%|███████▉  | 29980/37962 [69:25:25<18:10:20,  8.20s/it]
                                                           
{'loss': 1.1468, 'grad_norm': 7.39256477355957, 'learning_rate': 2.230518392244743e-06, 'epoch': 0.79}

 79%|███████▉  | 29980/37962 [69:25:25<18:10:20,  8.20s/it]
 79%|███████▉  | 29981/37962 [69:25:34<18:21:29,  8.28s/it]
 79%|███████▉  | 29982/37962 [69:25:42<18:23:35,  8.30s/it]
 79%|███████▉  | 29983/37962 [69:25:50<18:22:35,  8.29s/it]
 79%|███████▉  | 29984/37962 [69:25:59<18:19:22,  8.27s/it]
 79%|███████▉  | 29985/37962 [69:26:07<18:14:01,  8.23s/it]
 79%|███████▉  | 29986/37962 [69:26:15<18:12:11,  8.22s/it]
 79%|███████▉  | 29987/37962 [69:26:23<18:05:04,  8.16s/it]
 79%|███████▉  | 29988/37962 [69:26:31<18:09:53,  8.20s/it]
 79%|███████▉  | 29989/37962 [69:26:39<18:08:28,  8.19s/it]
 79%|███████▉  | 29990/37962 [69:26:47<18:04:33,  8.16s/it]
                                                           
{'loss': 1.1685, 'grad_norm': 7.381988525390625, 'learning_rate': 2.225150024683619e-06, 'epoch': 0.79}

 79%|███████▉  | 29990/37962 [69:26:47<18:04:33,  8.16s/it]
 79%|███████▉  | 29991/37962 [69:26:56<18:09:06,  8.20s/it]
 79%|███████▉  | 29992/37962 [69:27:04<18:04:34,  8.16s/it]
 79%|███████▉  | 29993/37962 [69:27:12<18:06:09,  8.18s/it]
 79%|███████▉  | 29994/37962 [69:27:20<18:02:51,  8.15s/it]
 79%|███████▉  | 29995/37962 [69:27:28<18:06:40,  8.18s/it]
 79%|███████▉  | 29996/37962 [69:27:37<18:08:12,  8.20s/it]
 79%|███████▉  | 29997/37962 [69:27:45<18:09:22,  8.21s/it]
 79%|███████▉  | 29998/37962 [69:27:53<18:10:25,  8.22s/it]
 79%|███████▉  | 29999/37962 [69:28:01<18:11:55,  8.23s/it]
 79%|███████▉  | 30000/37962 [69:28:10<18:13:37,  8.24s/it]
                                                           
{'loss': 1.27, 'grad_norm': 10.897844314575195, 'learning_rate': 2.219787316301096e-06, 'epoch': 0.79}

 79%|███████▉  | 30000/37962 [69:28:10<18:13:37,  8.24s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 79%|███████▉  | 30001/37962 [69:28:59<45:42:03, 20.67s/it]
 79%|███████▉  | 30002/37962 [69:29:08<37:30:40, 16.96s/it]
 79%|███████▉  | 30003/37962 [69:29:16<31:44:33, 14.36s/it]
 79%|███████▉  | 30004/37962 [69:29:24<27:44:48, 12.55s/it]
 79%|███████▉  | 30005/37962 [69:29:33<24:55:46, 11.28s/it]
 79%|███████▉  | 30006/37962 [69:29:41<23:02:08, 10.42s/it]
 79%|███████▉  | 30007/37962 [69:29:49<21:39:31,  9.80s/it]
 79%|███████▉  | 30008/37962 [69:29:58<20:43:25,  9.38s/it]
 79%|███████▉  | 30009/37962 [69:30:06<20:10:13,  9.13s/it]
 79%|███████▉  | 30010/37962 [69:30:14<19:33:35,  8.86s/it]
                                                           
{'loss': 1.2735, 'grad_norm': 7.39321756362915, 'learning_rate': 2.214430271000603e-06, 'epoch': 0.79}

 79%|███████▉  | 30010/37962 [69:30:14<19:33:35,  8.86s/it]
 79%|███████▉  | 30011/37962 [69:30:23<19:15:01,  8.72s/it]
 79%|███████▉  | 30012/37962 [69:30:31<19:03:26,  8.63s/it]
 79%|███████▉  | 30013/37962 [69:30:40<18:56:07,  8.58s/it]
 79%|███████▉  | 30014/37962 [69:30:48<18:37:32,  8.44s/it]
 79%|███████▉  | 30015/37962 [69:30:56<18:28:36,  8.37s/it]
 79%|███████▉  | 30016/37962 [69:31:04<18:25:20,  8.35s/it]
 79%|███████▉  | 30017/37962 [69:31:13<18:31:28,  8.39s/it]
 79%|███████▉  | 30018/37962 [69:31:21<18:20:57,  8.32s/it]
 79%|███████▉  | 30019/37962 [69:31:29<18:20:03,  8.31s/it]
 79%|███████▉  | 30020/37962 [69:31:37<18:08:00,  8.22s/it]
                                                           
{'loss': 1.142, 'grad_norm': 7.9408416748046875, 'learning_rate': 2.209078892681442e-06, 'epoch': 0.79}

 79%|███████▉  | 30020/37962 [69:31:37<18:08:00,  8.22s/it]
 79%|███████▉  | 30021/37962 [69:31:45<18:04:40,  8.20s/it]
 79%|███████▉  | 30022/37962 [69:31:54<18:08:55,  8.23s/it]
 79%|███████▉  | 30023/37962 [69:32:02<18:08:26,  8.23s/it]
 79%|███████▉  | 30024/37962 [69:32:10<18:13:43,  8.27s/it]
 79%|███████▉  | 30025/37962 [69:32:19<18:14:23,  8.27s/it]
 79%|███████▉  | 30026/37962 [69:32:27<18:13:58,  8.27s/it]
 79%|███████▉  | 30027/37962 [69:32:35<18:18:57,  8.31s/it]
 79%|███████▉  | 30028/37962 [69:32:44<18:17:49,  8.30s/it]
 79%|███████▉  | 30029/37962 [69:32:52<18:07:04,  8.22s/it]
 79%|███████▉  | 30030/37962 [69:33:00<18:18:01,  8.31s/it]
                                                           
{'loss': 1.1792, 'grad_norm': 12.226337432861328, 'learning_rate': 2.2037331852387843e-06, 'epoch': 0.79}

 79%|███████▉  | 30030/37962 [69:33:00<18:18:01,  8.31s/it]
 79%|███████▉  | 30031/37962 [69:33:08<18:15:38,  8.29s/it]
 79%|███████▉  | 30032/37962 [69:33:17<18:14:42,  8.28s/it]
 79%|███████▉  | 30033/37962 [69:33:25<18:08:32,  8.24s/it]
 79%|███████▉  | 30034/37962 [69:33:33<18:19:28,  8.32s/it]
 79%|███████▉  | 30035/37962 [69:33:42<18:23:42,  8.35s/it]
 79%|███████▉  | 30036/37962 [69:33:50<18:25:43,  8.37s/it]
 79%|███████▉  | 30037/37962 [69:33:59<18:28:30,  8.39s/it]
 79%|███████▉  | 30038/37962 [69:34:07<18:29:12,  8.40s/it]
 79%|███████▉  | 30039/37962 [69:34:15<18:26:32,  8.38s/it]
 79%|███████▉  | 30040/37962 [69:34:24<18:24:37,  8.37s/it]
                                                           
{'loss': 1.1798, 'grad_norm': 6.207977294921875, 'learning_rate': 2.1983931525636836e-06, 'epoch': 0.79}

 79%|███████▉  | 30040/37962 [69:34:24<18:24:37,  8.37s/it]
 79%|███████▉  | 30041/37962 [69:34:32<18:22:30,  8.35s/it]
 79%|███████▉  | 30042/37962 [69:34:40<18:15:00,  8.30s/it]
 79%|███████▉  | 30043/37962 [69:34:48<18:17:11,  8.31s/it]
 79%|███████▉  | 30044/37962 [69:34:57<18:18:31,  8.32s/it]
 79%|███████▉  | 30045/37962 [69:35:05<18:22:35,  8.36s/it]
 79%|███████▉  | 30046/37962 [69:35:13<18:11:42,  8.27s/it]
 79%|███████▉  | 30047/37962 [69:35:22<18:10:15,  8.26s/it]
 79%|███████▉  | 30048/37962 [69:35:30<18:13:04,  8.29s/it]
 79%|███████▉  | 30049/37962 [69:35:38<18:06:30,  8.24s/it]
 79%|███████▉  | 30050/37962 [69:35:46<17:47:42,  8.10s/it]
                                                           
{'loss': 1.2528, 'grad_norm': 27.126054763793945, 'learning_rate': 2.1930587985430528e-06, 'epoch': 0.79}

 79%|███████▉  | 30050/37962 [69:35:46<17:47:42,  8.10s/it]
 79%|███████▉  | 30051/37962 [69:35:54<17:55:05,  8.15s/it]
 79%|███████▉  | 30052/37962 [69:36:02<18:00:17,  8.19s/it]
 79%|███████▉  | 30053/37962 [69:36:11<18:05:06,  8.23s/it]
 79%|███████▉  | 30054/37962 [69:36:19<18:07:43,  8.25s/it]
 79%|███████▉  | 30055/37962 [69:36:27<18:08:50,  8.26s/it]
 79%|███████▉  | 30056/37962 [69:36:36<18:19:56,  8.35s/it]
 79%|███████▉  | 30057/37962 [69:36:44<18:17:44,  8.33s/it]
 79%|███████▉  | 30058/37962 [69:36:52<18:14:32,  8.31s/it]
 79%|███████▉  | 30059/37962 [69:37:01<18:17:32,  8.33s/it]
 79%|███████▉  | 30060/37962 [69:37:09<18:20:20,  8.35s/it]
                                                           
{'loss': 1.1347, 'grad_norm': 5.866193771362305, 'learning_rate': 2.187730127059674e-06, 'epoch': 0.79}

 79%|███████▉  | 30060/37962 [69:37:09<18:20:20,  8.35s/it]
 79%|███████▉  | 30061/37962 [69:37:17<18:15:15,  8.32s/it]
 79%|███████▉  | 30062/37962 [69:37:26<18:09:56,  8.28s/it]
 79%|███████▉  | 30063/37962 [69:37:34<18:17:47,  8.34s/it]
 79%|███████▉  | 30064/37962 [69:37:42<18:02:07,  8.22s/it]
 79%|███████▉  | 30065/37962 [69:37:50<18:08:20,  8.27s/it]
 79%|███████▉  | 30066/37962 [69:37:59<18:08:01,  8.27s/it]
 79%|███████▉  | 30067/37962 [69:38:07<18:06:56,  8.26s/it]
 79%|███████▉  | 30068/37962 [69:38:15<18:14:10,  8.32s/it]
 79%|███████▉  | 30069/37962 [69:38:24<18:10:36,  8.29s/it]
 79%|███████▉  | 30070/37962 [69:38:32<18:11:27,  8.30s/it]
                                                           
{'loss': 1.1463, 'grad_norm': 7.08782958984375, 'learning_rate': 2.1824071419922e-06, 'epoch': 0.79}

 79%|███████▉  | 30070/37962 [69:38:32<18:11:27,  8.30s/it]
 79%|███████▉  | 30071/37962 [69:38:40<18:13:30,  8.31s/it]
 79%|███████▉  | 30072/37962 [69:38:49<18:15:27,  8.33s/it]
 79%|███████▉  | 30073/37962 [69:38:57<17:57:38,  8.20s/it]
 79%|███████▉  | 30074/37962 [69:39:05<17:52:03,  8.15s/it]
 79%|███████▉  | 30075/37962 [69:39:13<17:52:50,  8.16s/it]
 79%|███████▉  | 30076/37962 [69:39:21<17:56:40,  8.19s/it]
 79%|███████▉  | 30077/37962 [69:39:29<17:56:49,  8.19s/it]
 79%|███████▉  | 30078/37962 [69:39:38<18:00:22,  8.22s/it]
 79%|███████▉  | 30079/37962 [69:39:46<17:59:48,  8.22s/it]
 79%|███████▉  | 30080/37962 [69:39:54<18:03:24,  8.25s/it]
                                                           
{'loss': 1.0605, 'grad_norm': 7.024013519287109, 'learning_rate': 2.1770898472151337e-06, 'epoch': 0.79}

 79%|███████▉  | 30080/37962 [69:39:54<18:03:24,  8.25s/it]
 79%|███████▉  | 30081/37962 [69:40:02<17:56:09,  8.19s/it]
 79%|███████▉  | 30082/37962 [69:40:11<18:08:01,  8.28s/it]
 79%|███████▉  | 30083/37962 [69:40:19<18:04:48,  8.26s/it]
 79%|███████▉  | 30084/37962 [69:40:27<18:03:16,  8.25s/it]
 79%|███████▉  | 30085/37962 [69:40:35<18:04:24,  8.26s/it]
 79%|███████▉  | 30086/37962 [69:40:44<18:01:42,  8.24s/it]
 79%|███████▉  | 30087/37962 [69:40:51<17:47:20,  8.13s/it]
 79%|███████▉  | 30088/37962 [69:40:59<17:41:42,  8.09s/it]
 79%|███████▉  | 30089/37962 [69:41:08<17:57:28,  8.21s/it]
 79%|███████▉  | 30090/37962 [69:41:16<18:03:24,  8.26s/it]
                                                           
{'loss': 1.1988, 'grad_norm': 7.243834018707275, 'learning_rate': 2.1717782465988457e-06, 'epoch': 0.79}

 79%|███████▉  | 30090/37962 [69:41:16<18:03:24,  8.26s/it]
 79%|███████▉  | 30091/37962 [69:41:24<17:49:43,  8.15s/it]
 79%|███████▉  | 30092/37962 [69:41:32<17:47:50,  8.14s/it]
 79%|███████▉  | 30093/37962 [69:41:41<17:53:20,  8.18s/it]
 79%|███████▉  | 30094/37962 [69:41:49<17:58:30,  8.22s/it]
 79%|███████▉  | 30095/37962 [69:41:57<17:55:22,  8.20s/it]
 79%|███████▉  | 30096/37962 [69:42:05<18:03:50,  8.27s/it]
 79%|███████▉  | 30097/37962 [69:42:14<18:08:52,  8.31s/it]
 79%|███████▉  | 30098/37962 [69:42:22<18:15:09,  8.36s/it]
 79%|███████▉  | 30099/37962 [69:42:31<18:18:08,  8.38s/it]
 79%|███████▉  | 30100/37962 [69:42:38<17:50:43,  8.17s/it]
                                                           
{'loss': 1.1962, 'grad_norm': 6.959171295166016, 'learning_rate': 2.1664723440095626e-06, 'epoch': 0.79}

 79%|███████▉  | 30100/37962 [69:42:38<17:50:43,  8.17s/it]
 79%|███████▉  | 30101/37962 [69:42:46<17:46:17,  8.14s/it]
 79%|███████▉  | 30102/37962 [69:42:55<17:54:28,  8.20s/it]
 79%|███████▉  | 30103/37962 [69:43:03<18:03:49,  8.27s/it]
 79%|███████▉  | 30104/37962 [69:43:11<17:58:15,  8.23s/it]
 79%|███████▉  | 30105/37962 [69:43:20<18:03:15,  8.27s/it]
 79%|███████▉  | 30106/37962 [69:43:28<18:05:32,  8.29s/it]
 79%|███████▉  | 30107/37962 [69:43:36<18:03:39,  8.28s/it]
 79%|███████▉  | 30108/37962 [69:43:45<18:02:01,  8.27s/it]
 79%|███████▉  | 30109/37962 [69:43:53<17:59:15,  8.25s/it]
 79%|███████▉  | 30110/37962 [69:44:01<18:01:08,  8.26s/it]
                                                           
{'loss': 1.1998, 'grad_norm': 10.731844902038574, 'learning_rate': 2.161172143309348e-06, 'epoch': 0.79}

 79%|███████▉  | 30110/37962 [69:44:01<18:01:08,  8.26s/it]
 79%|███████▉  | 30111/37962 [69:44:09<18:05:14,  8.29s/it]
 79%|███████▉  | 30112/37962 [69:44:18<18:07:45,  8.31s/it]
 79%|███████▉  | 30113/37962 [69:44:26<18:06:24,  8.30s/it]
 79%|███████▉  | 30114/37962 [69:44:34<18:08:23,  8.32s/it]
 79%|███████▉  | 30115/37962 [69:44:43<18:01:05,  8.27s/it]
 79%|███████▉  | 30116/37962 [69:44:51<18:04:08,  8.29s/it]
 79%|███████▉  | 30117/37962 [69:44:59<18:02:51,  8.28s/it]
 79%|███████▉  | 30118/37962 [69:45:08<18:04:19,  8.29s/it]
 79%|███████▉  | 30119/37962 [69:45:15<17:42:43,  8.13s/it]
 79%|███████▉  | 30120/37962 [69:45:23<17:40:42,  8.12s/it]
                                                           
{'loss': 1.1448, 'grad_norm': 8.12711238861084, 'learning_rate': 2.155877648356132e-06, 'epoch': 0.79}

 79%|███████▉  | 30120/37962 [69:45:23<17:40:42,  8.12s/it]
 79%|███████▉  | 30121/37962 [69:45:32<17:48:37,  8.18s/it]
 79%|███████▉  | 30122/37962 [69:45:40<17:53:44,  8.22s/it]
 79%|███████▉  | 30123/37962 [69:45:48<17:57:07,  8.24s/it]
 79%|███████▉  | 30124/37962 [69:45:57<17:57:21,  8.25s/it]
 79%|███████▉  | 30125/37962 [69:46:05<18:05:14,  8.31s/it]
 79%|███████▉  | 30126/37962 [69:46:13<18:03:49,  8.30s/it]
 79%|███████▉  | 30127/37962 [69:46:22<18:02:37,  8.29s/it]
 79%|███████▉  | 30128/37962 [69:46:29<17:43:46,  8.15s/it]
 79%|███████▉  | 30129/37962 [69:46:38<17:48:09,  8.18s/it]
 79%|███████▉  | 30130/37962 [69:46:46<17:53:07,  8.22s/it]
                                                           
{'loss': 1.2241, 'grad_norm': 9.20328140258789, 'learning_rate': 2.1505888630036877e-06, 'epoch': 0.79}

 79%|███████▉  | 30130/37962 [69:46:46<17:53:07,  8.22s/it]
 79%|███████▉  | 30131/37962 [69:46:54<17:55:52,  8.24s/it]
 79%|███████▉  | 30132/37962 [69:47:02<17:48:43,  8.19s/it]
 79%|███████▉  | 30133/37962 [69:47:11<17:53:39,  8.23s/it]
 79%|███████▉  | 30134/37962 [69:47:19<17:54:53,  8.24s/it]
 79%|███████▉  | 30135/37962 [69:47:27<18:00:21,  8.28s/it]
 79%|███████▉  | 30136/37962 [69:47:35<17:45:23,  8.17s/it]
 79%|███████▉  | 30137/37962 [69:47:43<17:46:32,  8.18s/it]
 79%|███████▉  | 30138/37962 [69:47:52<17:45:03,  8.17s/it]
 79%|███████▉  | 30139/37962 [69:48:00<17:43:31,  8.16s/it]
 79%|███████▉  | 30140/37962 [69:48:08<17:47:04,  8.19s/it]
                                                           
{'loss': 1.1203, 'grad_norm': 12.966363906860352, 'learning_rate': 2.145305791101626e-06, 'epoch': 0.79}

 79%|███████▉  | 30140/37962 [69:48:08<17:47:04,  8.19s/it]
 79%|███████▉  | 30141/37962 [69:48:16<17:55:04,  8.25s/it]
 79%|███████▉  | 30142/37962 [69:48:25<17:54:51,  8.25s/it]
 79%|███████▉  | 30143/37962 [69:48:33<17:54:17,  8.24s/it]
 79%|███████▉  | 30144/37962 [69:48:41<17:58:03,  8.27s/it]
 79%|███████▉  | 30145/37962 [69:48:49<17:54:14,  8.25s/it]
 79%|███████▉  | 30146/37962 [69:48:58<17:59:02,  8.28s/it]
 79%|███████▉  | 30147/37962 [69:49:06<17:50:02,  8.22s/it]
 79%|███████▉  | 30148/37962 [69:49:14<18:05:17,  8.33s/it]
 79%|███████▉  | 30149/37962 [69:49:23<18:05:58,  8.34s/it]
 79%|███████▉  | 30150/37962 [69:49:31<17:59:12,  8.29s/it]
                                                           
{'loss': 1.1471, 'grad_norm': 11.48041820526123, 'learning_rate': 2.1400284364954093e-06, 'epoch': 0.79}

 79%|███████▉  | 30150/37962 [69:49:31<17:59:12,  8.29s/it]
 79%|███████▉  | 30151/37962 [69:49:39<17:53:32,  8.25s/it]
 79%|███████▉  | 30152/37962 [69:49:48<18:05:02,  8.34s/it]
 79%|███████▉  | 30153/37962 [69:49:56<18:03:25,  8.32s/it]
 79%|███████▉  | 30154/37962 [69:50:04<18:00:58,  8.31s/it]
 79%|███████▉  | 30155/37962 [69:50:12<17:56:26,  8.27s/it]
 79%|███████▉  | 30156/37962 [69:50:21<18:02:44,  8.32s/it]
 79%|███████▉  | 30157/37962 [69:50:29<17:59:26,  8.30s/it]
 79%|███████▉  | 30158/37962 [69:50:37<17:57:44,  8.29s/it]
 79%|███████▉  | 30159/37962 [69:50:45<17:53:55,  8.26s/it]
 79%|███████▉  | 30160/37962 [69:50:54<17:58:36,  8.29s/it]
                                                           
{'loss': 1.2408, 'grad_norm': 5.0922675132751465, 'learning_rate': 2.134756803026328e-06, 'epoch': 0.79}

 79%|███████▉  | 30160/37962 [69:50:54<17:58:36,  8.29s/it]
 79%|███████▉  | 30161/37962 [69:51:02<18:04:13,  8.34s/it]
 79%|███████▉  | 30162/37962 [69:51:10<17:57:40,  8.29s/it]
 79%|███████▉  | 30163/37962 [69:51:18<17:46:52,  8.21s/it]
 79%|███████▉  | 30164/37962 [69:51:27<17:50:13,  8.23s/it]
 79%|███████▉  | 30165/37962 [69:51:35<17:57:02,  8.29s/it]
 79%|███████▉  | 30166/37962 [69:51:43<17:52:00,  8.25s/it]
 79%|███████▉  | 30167/37962 [69:51:51<17:40:37,  8.16s/it]
 79%|███████▉  | 30168/37962 [69:52:00<17:47:26,  8.22s/it]
 79%|███████▉  | 30169/37962 [69:52:08<17:57:58,  8.30s/it]
 79%|███████▉  | 30170/37962 [69:52:16<17:54:29,  8.27s/it]
                                                           
{'loss': 1.0427, 'grad_norm': 12.124212265014648, 'learning_rate': 2.129490894531512e-06, 'epoch': 0.79}

 79%|███████▉  | 30170/37962 [69:52:16<17:54:29,  8.27s/it]
 79%|███████▉  | 30171/37962 [69:52:25<17:52:26,  8.26s/it]
 79%|███████▉  | 30172/37962 [69:52:33<18:01:04,  8.33s/it]
 79%|███████▉  | 30173/37962 [69:52:42<18:07:00,  8.37s/it]
 79%|███████▉  | 30174/37962 [69:52:50<17:56:54,  8.30s/it]
 79%|███████▉  | 30175/37962 [69:52:58<17:52:12,  8.26s/it]
 79%|███████▉  | 30176/37962 [69:53:06<18:01:14,  8.33s/it]
 79%|███████▉  | 30177/37962 [69:53:14<17:40:40,  8.17s/it]
 79%|███████▉  | 30178/37962 [69:53:22<17:45:52,  8.22s/it]
 79%|███████▉  | 30179/37962 [69:53:31<17:46:46,  8.22s/it]
 80%|███████▉  | 30180/37962 [69:53:39<17:44:09,  8.20s/it]
                                                           
{'loss': 1.2521, 'grad_norm': 7.154329299926758, 'learning_rate': 2.1242307148439258e-06, 'epoch': 0.79}

 80%|███████▉  | 30180/37962 [69:53:39<17:44:09,  8.20s/it]
 80%|███████▉  | 30181/37962 [69:53:47<17:53:09,  8.28s/it]
 80%|███████▉  | 30182/37962 [69:53:56<18:00:58,  8.34s/it]
 80%|███████▉  | 30183/37962 [69:54:04<18:03:21,  8.36s/it]
 80%|███████▉  | 30184/37962 [69:54:12<18:00:22,  8.33s/it]
 80%|███████▉  | 30185/37962 [69:54:21<17:53:32,  8.28s/it]
 80%|███████▉  | 30186/37962 [69:54:29<17:56:49,  8.31s/it]
 80%|███████▉  | 30187/37962 [69:54:44<22:16:46, 10.32s/it]
 80%|███████▉  | 30188/37962 [69:54:52<21:02:35,  9.74s/it]
 80%|███████▉  | 30189/37962 [69:55:01<20:14:49,  9.38s/it]
 80%|███████▉  | 30190/37962 [69:55:09<19:37:47,  9.09s/it]
                                                           
{'loss': 1.1745, 'grad_norm': 11.58328914642334, 'learning_rate': 2.1189762677923662e-06, 'epoch': 0.8}

 80%|███████▉  | 30190/37962 [69:55:09<19:37:47,  9.09s/it]
 80%|███████▉  | 30191/37962 [69:55:18<19:04:19,  8.84s/it]
 80%|███████▉  | 30192/37962 [69:55:26<18:30:05,  8.57s/it]
 80%|███████▉  | 30193/37962 [69:55:34<18:17:31,  8.48s/it]
 80%|███████▉  | 30194/37962 [69:55:42<18:08:30,  8.41s/it]
 80%|███████▉  | 30195/37962 [69:55:50<18:01:32,  8.35s/it]
 80%|███████▉  | 30196/37962 [69:55:58<17:52:07,  8.28s/it]
 80%|███████▉  | 30197/37962 [69:56:07<17:46:53,  8.24s/it]
 80%|███████▉  | 30198/37962 [69:56:15<17:49:23,  8.26s/it]
 80%|███████▉  | 30199/37962 [69:56:23<17:48:55,  8.26s/it]
 80%|███████▉  | 30200/37962 [69:56:31<17:45:38,  8.24s/it]
                                                           
{'loss': 1.21, 'grad_norm': 7.410404205322266, 'learning_rate': 2.1137275572014525e-06, 'epoch': 0.8}

 80%|███████▉  | 30200/37962 [69:56:31<17:45:38,  8.24s/it]
 80%|███████▉  | 30201/37962 [69:56:40<17:52:15,  8.29s/it]
 80%|███████▉  | 30202/37962 [69:56:48<17:54:19,  8.31s/it]
 80%|███████▉  | 30203/37962 [69:56:56<17:45:14,  8.24s/it]
 80%|███████▉  | 30204/37962 [69:57:04<17:35:22,  8.16s/it]
 80%|███████▉  | 30205/37962 [69:57:12<17:37:47,  8.18s/it]
 80%|███████▉  | 30206/37962 [69:57:21<17:41:13,  8.21s/it]
 80%|███████▉  | 30207/37962 [69:57:29<17:44:46,  8.24s/it]
 80%|███████▉  | 30208/37962 [69:57:37<17:50:43,  8.29s/it]
 80%|███████▉  | 30209/37962 [69:57:46<17:48:23,  8.27s/it]
 80%|███████▉  | 30210/37962 [69:57:54<17:52:21,  8.30s/it]
                                                           
{'loss': 1.2256, 'grad_norm': 9.289090156555176, 'learning_rate': 2.108484586891627e-06, 'epoch': 0.8}

 80%|███████▉  | 30210/37962 [69:57:54<17:52:21,  8.30s/it]
 80%|███████▉  | 30211/37962 [69:58:02<17:53:09,  8.31s/it]
 80%|███████▉  | 30212/37962 [69:58:11<17:55:31,  8.33s/it]
 80%|███████▉  | 30213/37962 [69:58:19<17:51:43,  8.30s/it]
 80%|███████▉  | 30214/37962 [69:58:27<17:47:31,  8.27s/it]
 80%|███████▉  | 30215/37962 [69:58:35<17:51:22,  8.30s/it]
 80%|███████▉  | 30216/37962 [69:58:43<17:39:18,  8.21s/it]
 80%|███████▉  | 30217/37962 [69:58:52<17:44:53,  8.25s/it]
 80%|███████▉  | 30218/37962 [69:59:00<17:49:58,  8.29s/it]
 80%|███████▉  | 30219/37962 [69:59:09<17:54:06,  8.32s/it]
 80%|███████▉  | 30220/37962 [69:59:17<17:53:06,  8.32s/it]
                                                           
{'loss': 1.1164, 'grad_norm': 10.899478912353516, 'learning_rate': 2.1032473606791628e-06, 'epoch': 0.8}

 80%|███████▉  | 30220/37962 [69:59:17<17:53:06,  8.32s/it]
 80%|███████▉  | 30221/37962 [69:59:25<17:52:00,  8.31s/it]
 80%|███████▉  | 30222/37962 [69:59:34<17:59:44,  8.37s/it]
 80%|███████▉  | 30223/37962 [69:59:42<17:57:35,  8.35s/it]
 80%|███████▉  | 30224/37962 [69:59:50<17:56:37,  8.35s/it]
 80%|███████▉  | 30225/37962 [69:59:58<17:45:26,  8.26s/it]
 80%|███████▉  | 30226/37962 [70:00:06<17:32:05,  8.16s/it]
 80%|███████▉  | 30227/37962 [70:00:15<17:42:56,  8.25s/it]
 80%|███████▉  | 30228/37962 [70:00:23<17:40:00,  8.22s/it]
 80%|███████▉  | 30229/37962 [70:00:31<17:50:54,  8.31s/it]
 80%|███████▉  | 30230/37962 [70:00:40<17:46:06,  8.27s/it]
                                                           
{'loss': 1.2263, 'grad_norm': 6.359890937805176, 'learning_rate': 2.0980158823761397e-06, 'epoch': 0.8}

 80%|███████▉  | 30230/37962 [70:00:40<17:46:06,  8.27s/it]
 80%|███████▉  | 30231/37962 [70:00:48<17:33:53,  8.18s/it]
 80%|███████▉  | 30232/37962 [70:00:56<17:35:28,  8.19s/it]
 80%|███████▉  | 30233/37962 [70:01:04<17:29:17,  8.15s/it]
 80%|███████▉  | 30234/37962 [70:01:12<17:33:10,  8.18s/it]
 80%|███████▉  | 30235/37962 [70:01:20<17:19:46,  8.07s/it]
 80%|███████▉  | 30236/37962 [70:01:28<17:36:32,  8.21s/it]
 80%|███████▉  | 30237/37962 [70:01:37<17:38:06,  8.22s/it]
 80%|███████▉  | 30238/37962 [70:01:45<17:44:27,  8.27s/it]
 80%|███████▉  | 30239/37962 [70:01:53<17:44:14,  8.27s/it]
 80%|███████▉  | 30240/37962 [70:02:01<17:39:37,  8.23s/it]
                                                           
{'loss': 1.1782, 'grad_norm': 9.507186889648438, 'learning_rate': 2.092790155790465e-06, 'epoch': 0.8}

 80%|███████▉  | 30240/37962 [70:02:01<17:39:37,  8.23s/it]
 80%|███████▉  | 30241/37962 [70:02:10<17:42:56,  8.26s/it]
 80%|███████▉  | 30242/37962 [70:02:18<17:43:36,  8.27s/it]
 80%|███████▉  | 30243/37962 [70:02:26<17:41:32,  8.25s/it]
 80%|███████▉  | 30244/37962 [70:02:35<17:44:04,  8.27s/it]
 80%|███████▉  | 30245/37962 [70:02:43<17:44:21,  8.28s/it]
 80%|███████▉  | 30246/37962 [70:02:51<17:43:08,  8.27s/it]
 80%|███████▉  | 30247/37962 [70:02:59<17:40:55,  8.25s/it]
 80%|███████▉  | 30248/37962 [70:03:08<17:52:12,  8.34s/it]
 80%|███████▉  | 30249/37962 [70:03:16<17:39:50,  8.24s/it]
 80%|███████▉  | 30250/37962 [70:03:24<17:30:07,  8.17s/it]
                                                           
{'loss': 1.1979, 'grad_norm': 11.528604507446289, 'learning_rate': 2.0875701847258513e-06, 'epoch': 0.8}

 80%|███████▉  | 30250/37962 [70:03:24<17:30:07,  8.17s/it]
 80%|███████▉  | 30251/37962 [70:03:32<17:36:04,  8.22s/it]
 80%|███████▉  | 30252/37962 [70:03:40<17:25:58,  8.14s/it]
 80%|███████▉  | 30253/37962 [70:03:48<17:32:03,  8.19s/it]
 80%|███████▉  | 30254/37962 [70:03:57<17:40:36,  8.26s/it]
 80%|███████▉  | 30255/37962 [70:04:05<17:41:27,  8.26s/it]
 80%|███████▉  | 30256/37962 [70:04:13<17:29:42,  8.17s/it]
 80%|███████▉  | 30257/37962 [70:04:22<17:38:02,  8.24s/it]
 80%|███████▉  | 30258/37962 [70:04:30<17:37:15,  8.23s/it]
 80%|███████▉  | 30259/37962 [70:04:38<17:40:13,  8.26s/it]
 80%|███████▉  | 30260/37962 [70:04:46<17:37:22,  8.24s/it]
                                                           
{'loss': 1.2236, 'grad_norm': 11.746885299682617, 'learning_rate': 2.08235597298183e-06, 'epoch': 0.8}

 80%|███████▉  | 30260/37962 [70:04:46<17:37:22,  8.24s/it]
 80%|███████▉  | 30261/37962 [70:04:55<17:40:27,  8.26s/it]
 80%|███████▉  | 30262/37962 [70:05:03<17:38:42,  8.25s/it]
 80%|███████▉  | 30263/37962 [70:05:11<17:29:30,  8.18s/it]
 80%|███████▉  | 30264/37962 [70:05:19<17:33:20,  8.21s/it]
 80%|███████▉  | 30265/37962 [70:05:28<17:41:55,  8.28s/it]
 80%|███████▉  | 30266/37962 [70:05:36<17:45:51,  8.31s/it]
 80%|███████▉  | 30267/37962 [70:05:44<17:40:24,  8.27s/it]
 80%|███████▉  | 30268/37962 [70:05:53<17:47:11,  8.32s/it]
 80%|███████▉  | 30269/37962 [70:06:01<17:40:02,  8.27s/it]
 80%|███████▉  | 30270/37962 [70:06:09<17:31:44,  8.20s/it]
                                                           
{'loss': 1.1462, 'grad_norm': 11.046670913696289, 'learning_rate': 2.0771475243537307e-06, 'epoch': 0.8}

 80%|███████▉  | 30270/37962 [70:06:09<17:31:44,  8.20s/it]
 80%|███████▉  | 30271/37962 [70:06:17<17:31:56,  8.21s/it]
 80%|███████▉  | 30272/37962 [70:06:25<17:34:19,  8.23s/it]
 80%|███████▉  | 30273/37962 [70:06:34<17:43:02,  8.30s/it]
 80%|███████▉  | 30274/37962 [70:06:42<17:42:58,  8.30s/it]
 80%|███████▉  | 30275/37962 [70:06:50<17:38:33,  8.26s/it]
 80%|███████▉  | 30276/37962 [70:06:58<17:36:57,  8.25s/it]
 80%|███████▉  | 30277/37962 [70:07:07<17:32:03,  8.21s/it]
 80%|███████▉  | 30278/37962 [70:07:15<17:37:12,  8.26s/it]
 80%|███████▉  | 30279/37962 [70:07:23<17:37:40,  8.26s/it]
 80%|███████▉  | 30280/37962 [70:07:31<17:28:22,  8.19s/it]
                                                           
{'loss': 1.238, 'grad_norm': 12.598626136779785, 'learning_rate': 2.071944842632693e-06, 'epoch': 0.8}

 80%|███████▉  | 30280/37962 [70:07:31<17:28:22,  8.19s/it]
 80%|███████▉  | 30281/37962 [70:07:39<17:16:00,  8.09s/it]
 80%|███████▉  | 30282/37962 [70:07:47<17:27:52,  8.19s/it]
 80%|███████▉  | 30283/37962 [70:07:56<17:32:02,  8.22s/it]
 80%|███████▉  | 30284/37962 [70:08:04<17:17:12,  8.11s/it]
 80%|███████▉  | 30285/37962 [70:08:12<17:19:03,  8.12s/it]
 80%|███████▉  | 30286/37962 [70:08:20<17:26:48,  8.18s/it]
 80%|███████▉  | 30287/37962 [70:08:28<17:19:28,  8.13s/it]
 80%|███████▉  | 30288/37962 [70:08:36<17:14:51,  8.09s/it]
 80%|███████▉  | 30289/37962 [70:08:44<17:19:43,  8.13s/it]
 80%|███████▉  | 30290/37962 [70:08:52<17:13:14,  8.08s/it]
                                                           
{'loss': 1.0968, 'grad_norm': 7.527684211730957, 'learning_rate': 2.0667479316056594e-06, 'epoch': 0.8}

 80%|███████▉  | 30290/37962 [70:08:52<17:13:14,  8.08s/it]
 80%|███████▉  | 30291/37962 [70:09:01<17:21:58,  8.15s/it]
 80%|███████▉  | 30292/37962 [70:09:09<17:32:24,  8.23s/it]
 80%|███████▉  | 30293/37962 [70:09:17<17:39:41,  8.29s/it]
 80%|███████▉  | 30294/37962 [70:09:26<17:37:36,  8.28s/it]
 80%|███████▉  | 30295/37962 [70:09:34<17:41:12,  8.30s/it]
 80%|███████▉  | 30296/37962 [70:09:42<17:37:27,  8.28s/it]
 80%|███████▉  | 30297/37962 [70:09:51<17:44:22,  8.33s/it]
 80%|███████▉  | 30298/37962 [70:09:59<17:42:39,  8.32s/it]
 80%|███████▉  | 30299/37962 [70:10:07<17:43:59,  8.33s/it]
 80%|███████▉  | 30300/37962 [70:10:16<17:41:33,  8.31s/it]
                                                           
{'loss': 1.2431, 'grad_norm': 5.641901969909668, 'learning_rate': 2.061556795055375e-06, 'epoch': 0.8}

 80%|███████▉  | 30300/37962 [70:10:16<17:41:33,  8.31s/it]
 80%|███████▉  | 30301/37962 [70:10:24<17:41:48,  8.32s/it]
 80%|███████▉  | 30302/37962 [70:10:32<17:36:31,  8.28s/it]
 80%|███████▉  | 30303/37962 [70:10:41<17:45:43,  8.35s/it]
 80%|███████▉  | 30304/37962 [70:10:49<17:39:08,  8.30s/it]
 80%|███████▉  | 30305/37962 [70:10:57<17:36:40,  8.28s/it]
 80%|███████▉  | 30306/37962 [70:11:06<17:47:17,  8.36s/it]
 80%|███████▉  | 30307/37962 [70:11:14<17:48:45,  8.38s/it]
 80%|███████▉  | 30308/37962 [70:11:22<17:30:50,  8.24s/it]
 80%|███████▉  | 30309/37962 [70:11:30<17:29:46,  8.23s/it]
 80%|███████▉  | 30310/37962 [70:11:38<17:31:14,  8.24s/it]
                                                           
{'loss': 1.1648, 'grad_norm': 6.267887115478516, 'learning_rate': 2.0563714367603748e-06, 'epoch': 0.8}

 80%|███████▉  | 30310/37962 [70:11:38<17:31:14,  8.24s/it]
 80%|███████▉  | 30311/37962 [70:11:47<17:33:35,  8.26s/it]
 80%|███████▉  | 30312/37962 [70:11:55<17:36:54,  8.29s/it]
 80%|███████▉  | 30313/37962 [70:12:03<17:31:58,  8.25s/it]
 80%|███████▉  | 30314/37962 [70:12:11<17:27:27,  8.22s/it]
 80%|███████▉  | 30315/37962 [70:12:19<17:21:49,  8.17s/it]
 80%|███████▉  | 30316/37962 [70:12:27<17:09:50,  8.08s/it]
 80%|███████▉  | 30317/37962 [70:12:36<17:16:52,  8.14s/it]
 80%|███████▉  | 30318/37962 [70:12:44<17:24:16,  8.20s/it]
 80%|███████▉  | 30319/37962 [70:12:52<17:16:20,  8.14s/it]
 80%|███████▉  | 30320/37962 [70:13:00<17:23:55,  8.20s/it]
                                                           
{'loss': 1.1743, 'grad_norm': 6.753127098083496, 'learning_rate': 2.05119186049499e-06, 'epoch': 0.8}

 80%|███████▉  | 30320/37962 [70:13:00<17:23:55,  8.20s/it]
 80%|███████▉  | 30321/37962 [70:13:09<17:33:48,  8.27s/it]
 80%|███████▉  | 30322/37962 [70:13:17<17:33:43,  8.28s/it]
 80%|███████▉  | 30323/37962 [70:13:25<17:34:33,  8.28s/it]
 80%|███████▉  | 30324/37962 [70:13:34<17:38:19,  8.31s/it]
 80%|███████▉  | 30325/37962 [70:13:42<17:33:21,  8.28s/it]
 80%|███████▉  | 30326/37962 [70:13:50<17:29:15,  8.24s/it]
 80%|███████▉  | 30327/37962 [70:13:58<17:34:11,  8.28s/it]
 80%|███████▉  | 30328/37962 [70:14:07<17:32:32,  8.27s/it]
 80%|███████▉  | 30329/37962 [70:14:15<17:29:24,  8.25s/it]
 80%|███████▉  | 30330/37962 [70:14:23<17:17:19,  8.16s/it]
                                                           
{'loss': 1.1304, 'grad_norm': 9.947144508361816, 'learning_rate': 2.0460180700293485e-06, 'epoch': 0.8}

 80%|███████▉  | 30330/37962 [70:14:23<17:17:19,  8.16s/it]
 80%|███████▉  | 30331/37962 [70:14:31<17:23:07,  8.20s/it]
 80%|███████▉  | 30332/37962 [70:14:39<17:28:48,  8.25s/it]
 80%|███████▉  | 30333/37962 [70:14:47<17:18:50,  8.17s/it]
 80%|███████▉  | 30334/37962 [70:14:56<17:19:15,  8.17s/it]
 80%|███████▉  | 30335/37962 [70:15:04<17:24:07,  8.21s/it]
 80%|███████▉  | 30336/37962 [70:15:12<17:15:31,  8.15s/it]
 80%|███████▉  | 30337/37962 [70:15:20<17:25:28,  8.23s/it]
 80%|███████▉  | 30338/37962 [70:15:28<17:16:23,  8.16s/it]
 80%|███████▉  | 30339/37962 [70:15:37<17:22:10,  8.20s/it]
 80%|███████▉  | 30340/37962 [70:15:45<17:25:34,  8.23s/it]
                                                           
{'loss': 1.2436, 'grad_norm': 9.089081764221191, 'learning_rate': 2.0408500691293587e-06, 'epoch': 0.8}

 80%|███████▉  | 30340/37962 [70:15:45<17:25:34,  8.23s/it]
 80%|███████▉  | 30341/37962 [70:15:53<17:22:22,  8.21s/it]
 80%|███████▉  | 30342/37962 [70:16:01<17:23:45,  8.22s/it]
 80%|███████▉  | 30343/37962 [70:16:10<17:28:22,  8.26s/it]
 80%|███████▉  | 30344/37962 [70:16:18<17:18:40,  8.18s/it]
 80%|███████▉  | 30345/37962 [70:16:26<17:20:18,  8.19s/it]
 80%|███████▉  | 30346/37962 [70:16:34<17:28:18,  8.26s/it]
 80%|███████▉  | 30347/37962 [70:16:43<17:30:21,  8.28s/it]
 80%|███████▉  | 30348/37962 [70:16:51<17:31:01,  8.28s/it]
 80%|███████▉  | 30349/37962 [70:16:59<17:17:10,  8.17s/it]
 80%|███████▉  | 30350/37962 [70:17:07<17:14:44,  8.16s/it]
                                                           
{'loss': 1.1603, 'grad_norm': 14.342866897583008, 'learning_rate': 2.0356878615567223e-06, 'epoch': 0.8}

 80%|███████▉  | 30350/37962 [70:17:07<17:14:44,  8.16s/it]
 80%|███████▉  | 30351/37962 [70:17:15<17:13:03,  8.14s/it]
 80%|███████▉  | 30352/37962 [70:17:23<17:16:25,  8.17s/it]
 80%|███████▉  | 30353/37962 [70:17:31<17:12:41,  8.14s/it]
 80%|███████▉  | 30354/37962 [70:17:40<17:17:18,  8.18s/it]
 80%|███████▉  | 30355/37962 [70:17:48<17:19:04,  8.20s/it]
 80%|███████▉  | 30356/37962 [70:17:56<17:24:30,  8.24s/it]
 80%|███████▉  | 30357/37962 [70:18:05<17:28:57,  8.28s/it]
 80%|███████▉  | 30358/37962 [70:18:13<17:27:06,  8.26s/it]
 80%|███████▉  | 30359/37962 [70:18:21<17:27:21,  8.27s/it]
 80%|███████▉  | 30360/37962 [70:18:29<17:29:08,  8.28s/it]
                                                           
{'loss': 1.154, 'grad_norm': 6.698002815246582, 'learning_rate': 2.03053145106892e-06, 'epoch': 0.8}

 80%|███████▉  | 30360/37962 [70:18:29<17:29:08,  8.28s/it]
 80%|███████▉  | 30361/37962 [70:18:38<17:25:11,  8.25s/it]
 80%|███████▉  | 30362/37962 [70:18:46<17:26:22,  8.26s/it]
 80%|███████▉  | 30363/37962 [70:18:54<17:28:36,  8.28s/it]
 80%|███████▉  | 30364/37962 [70:19:02<17:27:57,  8.28s/it]
 80%|███████▉  | 30365/37962 [70:19:11<17:22:00,  8.23s/it]
 80%|███████▉  | 30366/37962 [70:19:19<17:21:54,  8.23s/it]
 80%|███████▉  | 30367/37962 [70:19:27<17:25:43,  8.26s/it]
 80%|███████▉  | 30368/37962 [70:19:35<17:14:22,  8.17s/it]
 80%|███████▉  | 30369/37962 [70:19:44<17:32:43,  8.32s/it]
 80%|████████  | 30370/37962 [70:19:52<17:30:35,  8.30s/it]
                                                           
{'loss': 1.2414, 'grad_norm': 13.20721435546875, 'learning_rate': 2.02538084141921e-06, 'epoch': 0.8}

 80%|████████  | 30370/37962 [70:19:52<17:30:35,  8.30s/it]
 80%|████████  | 30371/37962 [70:20:00<17:35:17,  8.34s/it]
 80%|████████  | 30372/37962 [70:20:09<17:34:16,  8.33s/it]
 80%|████████  | 30373/37962 [70:20:17<17:45:32,  8.42s/it]
 80%|████████  | 30374/37962 [70:20:26<17:38:01,  8.37s/it]
 80%|████████  | 30375/37962 [70:20:34<17:34:23,  8.34s/it]
 80%|████████  | 30376/37962 [70:20:42<17:30:08,  8.31s/it]
 80%|████████  | 30377/37962 [70:20:51<17:33:36,  8.33s/it]
 80%|████████  | 30378/37962 [70:20:59<17:25:12,  8.27s/it]
 80%|████████  | 30379/37962 [70:21:07<17:31:46,  8.32s/it]
 80%|████████  | 30380/37962 [70:21:15<17:32:17,  8.33s/it]
                                                           
{'loss': 1.1799, 'grad_norm': 8.23073959350586, 'learning_rate': 2.0202360363566377e-06, 'epoch': 0.8}

 80%|████████  | 30380/37962 [70:21:15<17:32:17,  8.33s/it]
 80%|████████  | 30381/37962 [70:21:24<17:35:13,  8.35s/it]
 80%|████████  | 30382/37962 [70:21:32<17:28:35,  8.30s/it]
 80%|████████  | 30383/37962 [70:21:40<17:29:55,  8.31s/it]
 80%|████████  | 30384/37962 [70:21:49<17:35:54,  8.36s/it]
 80%|████████  | 30385/37962 [70:21:57<17:35:28,  8.36s/it]
 80%|████████  | 30386/37962 [70:22:05<17:31:26,  8.33s/it]
 80%|████████  | 30387/37962 [70:22:14<17:27:04,  8.29s/it]
 80%|████████  | 30388/37962 [70:22:22<17:27:25,  8.30s/it]
 80%|████████  | 30389/37962 [70:22:31<17:36:32,  8.37s/it]
 80%|████████  | 30390/37962 [70:22:39<17:39:17,  8.39s/it]
                                                           
{'loss': 1.2781, 'grad_norm': 8.671072959899902, 'learning_rate': 2.0150970396260106e-06, 'epoch': 0.8}

 80%|████████  | 30390/37962 [70:22:39<17:39:17,  8.39s/it]
 80%|████████  | 30391/37962 [70:22:47<17:33:35,  8.35s/it]
 80%|████████  | 30392/37962 [70:22:55<17:29:22,  8.32s/it]
 80%|████████  | 30393/37962 [70:23:04<17:27:45,  8.31s/it]
 80%|████████  | 30394/37962 [70:23:12<17:15:27,  8.21s/it]
 80%|████████  | 30395/37962 [70:23:20<17:12:42,  8.19s/it]
 80%|████████  | 30396/37962 [70:23:28<17:12:28,  8.19s/it]
 80%|████████  | 30397/37962 [70:23:36<17:10:38,  8.17s/it]
 80%|████████  | 30398/37962 [70:23:44<17:13:13,  8.20s/it]
 80%|████████  | 30399/37962 [70:23:53<17:14:52,  8.21s/it]
 80%|████████  | 30400/37962 [70:24:01<17:18:20,  8.24s/it]
                                                           
{'loss': 1.2991, 'grad_norm': 10.397076606750488, 'learning_rate': 2.0099638549679213e-06, 'epoch': 0.8}

 80%|████████  | 30400/37962 [70:24:01<17:18:20,  8.24s/it]
 80%|████████  | 30401/37962 [70:24:09<17:16:32,  8.23s/it]
 80%|████████  | 30402/37962 [70:24:17<17:14:19,  8.21s/it]
 80%|████████  | 30403/37962 [70:24:26<17:20:03,  8.26s/it]
 80%|████████  | 30404/37962 [70:24:34<17:36:55,  8.39s/it]
 80%|████████  | 30405/37962 [70:24:43<17:35:42,  8.38s/it]
 80%|████████  | 30406/37962 [70:24:51<17:28:52,  8.33s/it]
 80%|████████  | 30407/37962 [70:24:59<17:22:42,  8.28s/it]
 80%|████████  | 30408/37962 [70:25:07<17:20:02,  8.26s/it]
 80%|████████  | 30409/37962 [70:25:16<17:21:20,  8.27s/it]
 80%|████████  | 30410/37962 [70:25:24<17:21:15,  8.27s/it]
                                                           
{'loss': 1.2025, 'grad_norm': 9.48668098449707, 'learning_rate': 2.0048364861187276e-06, 'epoch': 0.8}

 80%|████████  | 30410/37962 [70:25:24<17:21:15,  8.27s/it]
 80%|████████  | 30411/37962 [70:25:32<17:30:23,  8.35s/it]
 80%|████████  | 30412/37962 [70:25:41<17:28:35,  8.33s/it]
 80%|████████  | 30413/37962 [70:25:49<17:36:06,  8.39s/it]
 80%|████████  | 30414/37962 [70:25:58<17:29:39,  8.34s/it]
 80%|████████  | 30415/37962 [70:26:06<17:35:01,  8.39s/it]
 80%|████████  | 30416/37962 [70:26:14<17:33:25,  8.38s/it]
 80%|████████  | 30417/37962 [70:26:23<17:40:10,  8.43s/it]
 80%|████████  | 30418/37962 [70:26:31<17:28:48,  8.34s/it]
 80%|████████  | 30419/37962 [70:26:39<17:21:43,  8.29s/it]
 80%|████████  | 30420/37962 [70:26:47<17:10:40,  8.20s/it]
                                                           
{'loss': 1.1281, 'grad_norm': 5.827513217926025, 'learning_rate': 1.999714936810545e-06, 'epoch': 0.8}

 80%|████████  | 30420/37962 [70:26:47<17:10:40,  8.20s/it]
 80%|████████  | 30421/37962 [70:26:55<17:12:16,  8.21s/it]
 80%|████████  | 30422/37962 [70:27:03<17:04:27,  8.15s/it]
 80%|████████  | 30423/37962 [70:27:12<17:08:39,  8.19s/it]
 80%|████████  | 30424/37962 [70:27:20<16:59:33,  8.12s/it]
 80%|████████  | 30425/37962 [70:27:28<17:01:49,  8.13s/it]
 80%|████████  | 30426/37962 [70:27:36<17:05:46,  8.17s/it]
 80%|████████  | 30427/37962 [70:27:44<17:11:53,  8.22s/it]
 80%|████████  | 30428/37962 [70:27:53<17:12:44,  8.22s/it]
 80%|████████  | 30429/37962 [70:28:01<17:01:57,  8.14s/it]
 80%|████████  | 30430/37962 [70:28:09<17:07:21,  8.18s/it]
                                                           
{'loss': 1.1126, 'grad_norm': 8.886529922485352, 'learning_rate': 1.994599210771263e-06, 'epoch': 0.8}

 80%|████████  | 30430/37962 [70:28:09<17:07:21,  8.18s/it]
 80%|████████  | 30431/37962 [70:28:17<17:06:15,  8.18s/it]
 80%|████████  | 30432/37962 [70:28:25<17:09:56,  8.21s/it]
 80%|████████  | 30433/37962 [70:28:34<17:25:25,  8.33s/it]
 80%|████████  | 30434/37962 [70:28:42<17:25:47,  8.34s/it]
 80%|████████  | 30435/37962 [70:28:51<17:23:48,  8.32s/it]
 80%|████████  | 30436/37962 [70:28:59<17:19:21,  8.29s/it]
 80%|████████  | 30437/37962 [70:29:07<17:16:49,  8.27s/it]
 80%|████████  | 30438/37962 [70:29:15<17:11:26,  8.23s/it]
 80%|████████  | 30439/37962 [70:29:23<17:12:29,  8.23s/it]
 80%|████████  | 30440/37962 [70:29:32<17:14:33,  8.25s/it]
                                                           
{'loss': 1.0692, 'grad_norm': 7.082366943359375, 'learning_rate': 1.989489311724535e-06, 'epoch': 0.8}

 80%|████████  | 30440/37962 [70:29:32<17:14:33,  8.25s/it]
 80%|████████  | 30441/37962 [70:29:40<17:11:46,  8.23s/it]
 80%|████████  | 30442/37962 [70:29:48<17:07:51,  8.20s/it]
 80%|████████  | 30443/37962 [70:29:56<17:16:00,  8.27s/it]
 80%|████████  | 30444/37962 [70:30:04<17:03:07,  8.17s/it]
 80%|████████  | 30445/37962 [70:30:13<17:06:41,  8.20s/it]
 80%|████████  | 30446/37962 [70:30:21<17:15:19,  8.27s/it]
 80%|████████  | 30447/37962 [70:30:29<17:17:41,  8.28s/it]
 80%|████████  | 30448/37962 [70:30:38<17:21:06,  8.31s/it]
 80%|████████  | 30449/37962 [70:30:46<17:20:38,  8.31s/it]
 80%|████████  | 30450/37962 [70:30:54<17:15:41,  8.27s/it]
                                                           
{'loss': 1.16, 'grad_norm': 11.16085433959961, 'learning_rate': 1.9843852433897626e-06, 'epoch': 0.8}

 80%|████████  | 30450/37962 [70:30:54<17:15:41,  8.27s/it]
 80%|████████  | 30451/37962 [70:31:02<17:03:44,  8.18s/it]
 80%|████████  | 30452/37962 [70:31:10<17:00:17,  8.15s/it]
 80%|████████  | 30453/37962 [70:31:18<17:00:37,  8.16s/it]
 80%|████████  | 30454/37962 [70:31:27<17:02:44,  8.17s/it]
 80%|████████  | 30455/37962 [70:31:35<17:13:07,  8.26s/it]
 80%|████████  | 30456/37962 [70:31:43<17:12:01,  8.25s/it]
 80%|████████  | 30457/37962 [70:31:52<17:20:11,  8.32s/it]
 80%|████████  | 30458/37962 [70:32:00<17:18:51,  8.31s/it]
 80%|████████  | 30459/37962 [70:32:08<17:15:29,  8.28s/it]
 80%|████████  | 30460/37962 [70:32:17<17:19:09,  8.31s/it]
                                                           
{'loss': 1.1018, 'grad_norm': 10.1416654586792, 'learning_rate': 1.979287009482109e-06, 'epoch': 0.8}

 80%|████████  | 30460/37962 [70:32:17<17:19:09,  8.31s/it]
 80%|████████  | 30461/37962 [70:32:25<17:24:29,  8.35s/it]
 80%|████████  | 30462/37962 [70:32:33<17:17:12,  8.30s/it]
 80%|████████  | 30463/37962 [70:32:42<17:19:56,  8.32s/it]
 80%|████████  | 30464/37962 [70:32:50<17:24:50,  8.36s/it]
 80%|████████  | 30465/37962 [70:32:58<17:17:10,  8.30s/it]
 80%|████████  | 30466/37962 [70:33:06<17:09:12,  8.24s/it]
 80%|████████  | 30467/37962 [70:33:15<17:14:20,  8.28s/it]
 80%|████████  | 30468/37962 [70:33:23<17:13:59,  8.28s/it]
 80%|████████  | 30469/37962 [70:33:31<17:12:39,  8.27s/it]
 80%|████████  | 30470/37962 [70:33:40<17:11:02,  8.26s/it]
                                                           
{'loss': 1.2392, 'grad_norm': 11.169856071472168, 'learning_rate': 1.9741946137124978e-06, 'epoch': 0.8}

 80%|████████  | 30470/37962 [70:33:40<17:11:02,  8.26s/it]
 80%|████████  | 30471/37962 [70:33:48<17:06:41,  8.22s/it]
 80%|████████  | 30472/37962 [70:33:56<17:12:17,  8.27s/it]
 80%|████████  | 30473/37962 [70:34:04<17:16:52,  8.31s/it]
 80%|████████  | 30474/37962 [70:34:13<17:14:54,  8.29s/it]
 80%|████████  | 30475/37962 [70:34:21<17:19:40,  8.33s/it]
 80%|████████  | 30476/37962 [70:34:29<17:18:40,  8.32s/it]
 80%|████████  | 30477/37962 [70:34:38<17:22:47,  8.36s/it]
 80%|████████  | 30478/37962 [70:34:46<17:06:13,  8.23s/it]
 80%|████████  | 30479/37962 [70:34:54<17:05:27,  8.22s/it]
 80%|████████  | 30480/37962 [70:35:02<17:06:57,  8.24s/it]
                                                           
{'loss': 1.1052, 'grad_norm': 8.751354217529297, 'learning_rate': 1.9691080597875844e-06, 'epoch': 0.8}

 80%|████████  | 30480/37962 [70:35:02<17:06:57,  8.24s/it]
 80%|████████  | 30481/37962 [70:35:11<17:09:52,  8.26s/it]
 80%|████████  | 30482/37962 [70:35:19<16:59:55,  8.18s/it]
 80%|████████  | 30483/37962 [70:35:27<16:59:51,  8.18s/it]
 80%|████████  | 30484/37962 [70:35:35<17:06:11,  8.23s/it]
 80%|████████  | 30485/37962 [70:35:44<17:10:54,  8.27s/it]
 80%|████████  | 30486/37962 [70:35:52<17:01:26,  8.20s/it]
 80%|████████  | 30487/37962 [70:35:59<16:48:01,  8.09s/it]
 80%|████████  | 30488/37962 [70:36:08<16:57:37,  8.17s/it]
 80%|████████  | 30489/37962 [70:36:16<17:00:45,  8.20s/it]
 80%|████████  | 30490/37962 [70:36:24<17:00:52,  8.20s/it]
                                                           
{'loss': 1.178, 'grad_norm': 21.854312896728516, 'learning_rate': 1.9640273514097886e-06, 'epoch': 0.8}

 80%|████████  | 30490/37962 [70:36:24<17:00:52,  8.20s/it]
 80%|████████  | 30491/37962 [70:36:32<16:54:41,  8.15s/it]
 80%|████████  | 30492/37962 [70:36:40<16:53:21,  8.14s/it]
 80%|████████  | 30493/37962 [70:36:49<17:06:57,  8.25s/it]
 80%|████████  | 30494/37962 [70:36:57<17:05:51,  8.24s/it]
 80%|████████  | 30495/37962 [70:37:05<17:07:38,  8.26s/it]
 80%|████████  | 30496/37962 [70:37:14<17:06:56,  8.25s/it]
 80%|████████  | 30497/37962 [70:37:22<17:09:45,  8.28s/it]
 80%|████████  | 30498/37962 [70:37:31<17:21:13,  8.37s/it]
 80%|████████  | 30499/37962 [70:37:39<17:14:13,  8.31s/it]
 80%|████████  | 30500/37962 [70:37:47<17:01:22,  8.21s/it]
                                                           
{'loss': 1.0936, 'grad_norm': 7.638960838317871, 'learning_rate': 1.958952492277273e-06, 'epoch': 0.8}

 80%|████████  | 30500/37962 [70:37:47<17:01:22,  8.21s/it]
 80%|████████  | 30501/37962 [70:37:55<16:57:18,  8.18s/it]
 80%|████████  | 30502/37962 [70:38:03<17:04:10,  8.24s/it]
 80%|████████  | 30503/37962 [70:38:11<16:55:56,  8.17s/it]
 80%|████████  | 30504/37962 [70:38:19<16:59:29,  8.20s/it]
 80%|████████  | 30505/37962 [70:38:28<16:59:52,  8.21s/it]
 80%|████████  | 30506/37962 [70:38:36<17:08:27,  8.28s/it]
 80%|████████  | 30507/37962 [70:38:44<16:59:30,  8.21s/it]
 80%|████████  | 30508/37962 [70:38:52<17:01:07,  8.22s/it]
 80%|████████  | 30509/37962 [70:39:01<17:02:03,  8.23s/it]
 80%|████████  | 30510/37962 [70:39:09<17:11:33,  8.31s/it]
                                                           
{'loss': 1.2324, 'grad_norm': 6.640834808349609, 'learning_rate': 1.953883486083934e-06, 'epoch': 0.8}

 80%|████████  | 30510/37962 [70:39:09<17:11:33,  8.31s/it]
 80%|████████  | 30511/37962 [70:39:17<17:06:57,  8.27s/it]
 80%|████████  | 30512/37962 [70:39:25<17:02:50,  8.24s/it]
 80%|████████  | 30513/37962 [70:39:34<17:10:44,  8.30s/it]
 80%|████████  | 30514/37962 [70:39:42<16:51:43,  8.15s/it]
 80%|████████  | 30515/37962 [70:39:50<16:50:30,  8.14s/it]
 80%|████████  | 30516/37962 [70:39:58<16:50:55,  8.15s/it]
 80%|████████  | 30517/37962 [70:40:06<16:54:21,  8.17s/it]
 80%|████████  | 30518/37962 [70:40:15<17:00:13,  8.22s/it]
 80%|████████  | 30519/37962 [70:40:23<17:08:27,  8.29s/it]
 80%|████████  | 30520/37962 [70:40:31<17:03:04,  8.25s/it]
                                                           
{'loss': 1.1237, 'grad_norm': 9.722416877746582, 'learning_rate': 1.948820336519418e-06, 'epoch': 0.8}

 80%|████████  | 30520/37962 [70:40:31<17:03:04,  8.25s/it]
 80%|████████  | 30521/37962 [70:40:40<17:05:43,  8.27s/it]
 80%|████████  | 30522/37962 [70:40:48<17:03:13,  8.25s/it]
 80%|████████  | 30523/37962 [70:40:56<17:11:56,  8.32s/it]
 80%|████████  | 30524/37962 [70:41:04<17:07:02,  8.28s/it]
 80%|████████  | 30525/37962 [70:41:13<17:03:55,  8.26s/it]
 80%|████████  | 30526/37962 [70:41:21<17:06:12,  8.28s/it]
 80%|████████  | 30527/37962 [70:41:29<17:13:12,  8.34s/it]
 80%|████████  | 30528/37962 [70:41:38<17:13:27,  8.34s/it]
 80%|████████  | 30529/37962 [70:41:46<17:07:59,  8.30s/it]
 80%|████████  | 30530/37962 [70:41:54<17:03:22,  8.26s/it]
                                                           
{'loss': 1.1802, 'grad_norm': 8.796442985534668, 'learning_rate': 1.9437630472691026e-06, 'epoch': 0.8}

 80%|████████  | 30530/37962 [70:41:54<17:03:22,  8.26s/it]
 80%|████████  | 30531/37962 [70:42:03<17:08:18,  8.30s/it]
 80%|████████  | 30532/37962 [70:42:10<16:49:38,  8.15s/it]
 80%|████████  | 30533/37962 [70:42:19<16:51:57,  8.17s/it]
 80%|████████  | 30534/37962 [70:42:27<16:52:46,  8.18s/it]
 80%|████████  | 30535/37962 [70:42:35<16:48:30,  8.15s/it]
 80%|████████  | 30536/37962 [70:42:43<16:43:29,  8.11s/it]
 80%|████████  | 30537/37962 [70:42:51<16:50:29,  8.17s/it]
 80%|████████  | 30538/37962 [70:42:59<16:54:29,  8.20s/it]
 80%|████████  | 30539/37962 [70:43:08<17:00:20,  8.25s/it]
 80%|████████  | 30540/37962 [70:43:16<17:05:43,  8.29s/it]
                                                           
{'loss': 1.1418, 'grad_norm': 13.944660186767578, 'learning_rate': 1.9387116220140965e-06, 'epoch': 0.8}

 80%|████████  | 30540/37962 [70:43:16<17:05:43,  8.29s/it]
 80%|████████  | 30541/37962 [70:43:24<16:51:04,  8.17s/it]
 80%|████████  | 30542/37962 [70:43:32<16:48:59,  8.16s/it]
 80%|████████  | 30543/37962 [70:43:40<16:48:21,  8.15s/it]
 80%|████████  | 30544/37962 [70:43:49<16:48:23,  8.16s/it]
 80%|████████  | 30545/37962 [70:43:57<16:50:07,  8.17s/it]
 80%|████████  | 30546/37962 [70:44:05<16:54:02,  8.20s/it]
 80%|████████  | 30547/37962 [70:44:13<16:55:43,  8.22s/it]
 80%|████████  | 30548/37962 [70:44:21<16:53:07,  8.20s/it]
 80%|████████  | 30549/37962 [70:44:30<16:57:10,  8.23s/it]
 80%|████████  | 30550/37962 [70:44:38<16:59:38,  8.25s/it]
                                                           
{'loss': 1.1231, 'grad_norm': 7.697389125823975, 'learning_rate': 1.9336660644312524e-06, 'epoch': 0.8}

 80%|████████  | 30550/37962 [70:44:38<16:59:38,  8.25s/it]
 80%|████████  | 30551/37962 [70:44:46<16:58:36,  8.25s/it]
 80%|████████  | 30552/37962 [70:44:55<17:02:15,  8.28s/it]
 80%|████████  | 30553/37962 [70:45:03<17:08:35,  8.33s/it]
 80%|████████  | 30554/37962 [70:45:11<17:07:46,  8.32s/it]
 80%|████████  | 30555/37962 [70:45:19<16:55:28,  8.23s/it]
 80%|████████  | 30556/37962 [70:45:28<17:01:44,  8.28s/it]
 80%|████████  | 30557/37962 [70:45:36<17:06:38,  8.32s/it]
 80%|████████  | 30558/37962 [70:45:44<17:02:29,  8.29s/it]
 80%|████████  | 30559/37962 [70:45:53<17:00:21,  8.27s/it]
 81%|████████  | 30560/37962 [70:46:01<17:02:00,  8.28s/it]
                                                           
{'loss': 1.1298, 'grad_norm': 11.009883880615234, 'learning_rate': 1.928626378193138e-06, 'epoch': 0.8}

 81%|████████  | 30560/37962 [70:46:01<17:02:00,  8.28s/it]
 81%|████████  | 30561/37962 [70:46:09<17:08:12,  8.34s/it]
 81%|████████  | 30562/37962 [70:46:18<17:03:06,  8.30s/it]
 81%|████████  | 30563/37962 [70:46:26<17:00:44,  8.28s/it]
 81%|████████  | 30564/37962 [70:46:34<16:58:36,  8.26s/it]
 81%|████████  | 30565/37962 [70:46:42<16:55:04,  8.23s/it]
 81%|████████  | 30566/37962 [70:46:50<16:43:47,  8.14s/it]
 81%|████████  | 30567/37962 [70:46:59<16:52:06,  8.21s/it]
 81%|████████  | 30568/37962 [70:47:06<16:35:21,  8.08s/it]
 81%|████████  | 30569/37962 [70:47:14<16:21:56,  7.97s/it]
 81%|████████  | 30570/37962 [70:47:22<16:36:03,  8.08s/it]
                                                           
{'loss': 1.1393, 'grad_norm': 5.224260330200195, 'learning_rate': 1.923592566968058e-06, 'epoch': 0.81}

 81%|████████  | 30570/37962 [70:47:22<16:36:03,  8.08s/it]
 81%|████████  | 30571/37962 [70:47:31<16:40:56,  8.13s/it]
 81%|████████  | 30572/37962 [70:47:39<16:37:46,  8.10s/it]
 81%|████████  | 30573/37962 [70:47:47<16:33:14,  8.07s/it]
 81%|████████  | 30574/37962 [70:47:55<16:33:26,  8.07s/it]
 81%|████████  | 30575/37962 [70:48:03<16:40:08,  8.12s/it]
 81%|████████  | 30576/37962 [70:48:11<16:44:19,  8.16s/it]
 81%|████████  | 30577/37962 [70:48:19<16:37:26,  8.10s/it]
 81%|████████  | 30578/37962 [70:48:27<16:43:40,  8.16s/it]
 81%|████████  | 30579/37962 [70:48:36<16:46:32,  8.18s/it]
 81%|████████  | 30580/37962 [70:48:44<16:49:10,  8.20s/it]
                                                           
{'loss': 1.1475, 'grad_norm': 10.568594932556152, 'learning_rate': 1.918564634420034e-06, 'epoch': 0.81}

 81%|████████  | 30580/37962 [70:48:44<16:49:10,  8.20s/it]
 81%|████████  | 30581/37962 [70:48:52<16:55:15,  8.25s/it]
 81%|████████  | 30582/37962 [70:49:01<17:03:34,  8.32s/it]
 81%|████████  | 30583/37962 [70:49:09<16:59:14,  8.29s/it]
 81%|████████  | 30584/37962 [70:49:17<17:01:53,  8.31s/it]
 81%|████████  | 30585/37962 [70:49:26<16:59:42,  8.29s/it]
 81%|████████  | 30586/37962 [70:49:34<17:07:42,  8.36s/it]
 81%|████████  | 30587/37962 [70:49:42<17:05:14,  8.34s/it]
 81%|████████  | 30588/37962 [70:49:58<21:40:39, 10.58s/it]
 81%|████████  | 30589/37962 [70:50:07<20:17:51,  9.91s/it]
 81%|████████  | 30590/37962 [70:50:15<19:14:59,  9.40s/it]
                                                           
{'loss': 1.1887, 'grad_norm': 13.124283790588379, 'learning_rate': 1.9135425842088083e-06, 'epoch': 0.81}

 81%|████████  | 30590/37962 [70:50:15<19:14:59,  9.40s/it]
 81%|████████  | 30591/37962 [70:50:23<18:36:10,  9.09s/it]
 81%|████████  | 30592/37962 [70:50:31<18:08:15,  8.86s/it]
 81%|████████  | 30593/37962 [70:50:40<17:43:36,  8.66s/it]
 81%|████████  | 30594/37962 [70:50:48<17:31:18,  8.56s/it]
 81%|████████  | 30595/37962 [70:50:56<17:24:16,  8.50s/it]
 81%|████████  | 30596/37962 [70:51:05<17:13:52,  8.42s/it]
 81%|████████  | 30597/37962 [70:51:13<17:11:24,  8.40s/it]
 81%|████████  | 30598/37962 [70:51:21<16:59:06,  8.30s/it]
 81%|████████  | 30599/37962 [70:51:29<16:58:42,  8.30s/it]
 81%|████████  | 30600/37962 [70:51:38<16:58:21,  8.30s/it]
                                                           
{'loss': 1.1704, 'grad_norm': 5.619673728942871, 'learning_rate': 1.908526419989848e-06, 'epoch': 0.81}

 81%|████████  | 30600/37962 [70:51:38<16:58:21,  8.30s/it]
 81%|████████  | 30601/37962 [70:51:46<17:02:23,  8.33s/it]
 81%|████████  | 30602/37962 [70:51:54<17:01:08,  8.32s/it]
 81%|████████  | 30603/37962 [70:52:03<17:04:58,  8.36s/it]
 81%|████████  | 30604/37962 [70:52:11<16:53:42,  8.27s/it]
 81%|████████  | 30605/37962 [70:52:19<16:52:32,  8.26s/it]
 81%|████████  | 30606/37962 [70:52:27<16:56:17,  8.29s/it]
 81%|████████  | 30607/37962 [70:52:36<17:03:11,  8.35s/it]
 81%|████████  | 30608/37962 [70:52:44<16:54:14,  8.28s/it]
 81%|████████  | 30609/37962 [70:52:52<16:54:08,  8.28s/it]
 81%|████████  | 30610/37962 [70:53:01<16:53:30,  8.27s/it]
                                                           
{'loss': 1.1284, 'grad_norm': 5.663410663604736, 'learning_rate': 1.9035161454143347e-06, 'epoch': 0.81}

 81%|████████  | 30610/37962 [70:53:01<16:53:30,  8.27s/it]
 81%|████████  | 30611/37962 [70:53:09<16:55:27,  8.29s/it]
 81%|████████  | 30612/37962 [70:53:17<16:55:30,  8.29s/it]
 81%|████████  | 30613/37962 [70:53:26<16:59:29,  8.32s/it]
 81%|████████  | 30614/37962 [70:53:34<16:56:02,  8.30s/it]
 81%|████████  | 30615/37962 [70:53:42<16:56:19,  8.30s/it]
 81%|████████  | 30616/37962 [70:53:51<17:01:18,  8.34s/it]
 81%|████████  | 30617/37962 [70:53:59<16:56:17,  8.30s/it]
 81%|████████  | 30618/37962 [70:54:07<16:39:57,  8.17s/it]
 81%|████████  | 30619/37962 [70:54:15<16:43:55,  8.20s/it]
 81%|████████  | 30620/37962 [70:54:23<16:43:00,  8.20s/it]
                                                           
{'loss': 1.221, 'grad_norm': 11.685273170471191, 'learning_rate': 1.8985117641291562e-06, 'epoch': 0.81}

 81%|████████  | 30620/37962 [70:54:23<16:43:00,  8.20s/it]
 81%|████████  | 30621/37962 [70:54:31<16:34:12,  8.13s/it]
 81%|████████  | 30622/37962 [70:54:39<16:38:10,  8.16s/it]
 81%|████████  | 30623/37962 [70:54:48<16:45:45,  8.22s/it]
 81%|████████  | 30624/37962 [70:54:56<16:48:24,  8.25s/it]
 81%|████████  | 30625/37962 [70:55:04<16:48:39,  8.25s/it]
 81%|████████  | 30626/37962 [70:55:13<16:50:48,  8.27s/it]
 81%|████████  | 30627/37962 [70:55:21<16:47:47,  8.24s/it]
 81%|████████  | 30628/37962 [70:55:29<16:51:59,  8.28s/it]
 81%|████████  | 30629/37962 [70:55:37<16:51:21,  8.28s/it]
 81%|████████  | 30630/37962 [70:55:46<16:49:02,  8.26s/it]
                                                           
{'loss': 1.286, 'grad_norm': 10.362303733825684, 'learning_rate': 1.893513279776913e-06, 'epoch': 0.81}

 81%|████████  | 30630/37962 [70:55:46<16:49:02,  8.26s/it]
 81%|████████  | 30631/37962 [70:55:54<16:55:12,  8.31s/it]
 81%|████████  | 30632/37962 [70:56:02<16:49:07,  8.26s/it]
 81%|████████  | 30633/37962 [70:56:10<16:53:12,  8.29s/it]
 81%|████████  | 30634/37962 [70:56:18<16:38:05,  8.17s/it]
 81%|████████  | 30635/37962 [70:56:27<16:42:27,  8.21s/it]
 81%|████████  | 30636/37962 [70:56:35<16:54:08,  8.31s/it]
 81%|████████  | 30637/37962 [70:56:43<16:53:52,  8.30s/it]
 81%|████████  | 30638/37962 [70:56:59<21:21:39, 10.50s/it]
 81%|████████  | 30639/37962 [70:57:07<20:03:05,  9.86s/it]
 81%|████████  | 30640/37962 [70:57:16<19:02:46,  9.36s/it]
                                                           
{'loss': 1.2271, 'grad_norm': 7.370004653930664, 'learning_rate': 1.8885206959959212e-06, 'epoch': 0.81}

 81%|████████  | 30640/37962 [70:57:16<19:02:46,  9.36s/it]
 81%|████████  | 30641/37962 [70:57:24<18:20:11,  9.02s/it]
 81%|████████  | 30642/37962 [70:57:32<17:53:00,  8.80s/it]
 81%|████████  | 30643/37962 [70:57:40<17:33:11,  8.63s/it]
 81%|████████  | 30644/37962 [70:57:49<17:13:12,  8.47s/it]
 81%|████████  | 30645/37962 [70:57:57<17:07:38,  8.43s/it]
 81%|████████  | 30646/37962 [70:58:05<17:04:32,  8.40s/it]
 81%|████████  | 30647/37962 [70:58:14<17:01:16,  8.38s/it]
 81%|████████  | 30648/37962 [70:58:22<17:00:15,  8.37s/it]
 81%|████████  | 30649/37962 [70:58:30<16:58:56,  8.36s/it]
 81%|████████  | 30650/37962 [70:58:38<16:53:30,  8.32s/it]
                                                           
{'loss': 1.1456, 'grad_norm': 10.271496772766113, 'learning_rate': 1.883534016420191e-06, 'epoch': 0.81}

 81%|████████  | 30650/37962 [70:58:38<16:53:30,  8.32s/it]
 81%|████████  | 30651/37962 [70:58:47<16:51:39,  8.30s/it]
 81%|████████  | 30652/37962 [70:58:55<16:54:15,  8.32s/it]
 81%|████████  | 30653/37962 [70:59:03<16:55:38,  8.34s/it]
 81%|████████  | 30654/37962 [70:59:12<16:50:48,  8.30s/it]
 81%|████████  | 30655/37962 [70:59:20<16:53:02,  8.32s/it]
 81%|████████  | 30656/37962 [70:59:28<16:45:54,  8.26s/it]
 81%|████████  | 30657/37962 [70:59:37<16:52:31,  8.32s/it]
 81%|████████  | 30658/37962 [70:59:45<16:38:28,  8.20s/it]
 81%|████████  | 30659/37962 [70:59:52<16:24:23,  8.09s/it]
 81%|████████  | 30660/37962 [71:00:01<16:35:21,  8.18s/it]
                                                           
{'loss': 1.1787, 'grad_norm': 6.52615213394165, 'learning_rate': 1.8785532446794442e-06, 'epoch': 0.81}

 81%|████████  | 30660/37962 [71:00:01<16:35:21,  8.18s/it]
 81%|████████  | 30661/37962 [71:00:09<16:28:46,  8.13s/it]
 81%|████████  | 30662/37962 [71:00:17<16:38:00,  8.20s/it]
 81%|████████  | 30663/37962 [71:00:25<16:25:32,  8.10s/it]
 81%|████████  | 30664/37962 [71:00:33<16:32:06,  8.16s/it]
 81%|████████  | 30665/37962 [71:00:42<16:37:37,  8.20s/it]
 81%|████████  | 30666/37962 [71:00:50<16:39:31,  8.22s/it]
 81%|████████  | 30667/37962 [71:00:58<16:45:30,  8.27s/it]
 81%|████████  | 30668/37962 [71:01:06<16:44:28,  8.26s/it]
 81%|████████  | 30669/37962 [71:01:15<16:45:55,  8.28s/it]
 81%|████████  | 30670/37962 [71:01:23<16:38:30,  8.22s/it]
                                                           
{'loss': 1.2581, 'grad_norm': 12.689484596252441, 'learning_rate': 1.8735783843990962e-06, 'epoch': 0.81}

 81%|████████  | 30670/37962 [71:01:23<16:38:30,  8.22s/it]
 81%|████████  | 30671/37962 [71:01:31<16:43:56,  8.26s/it]
 81%|████████  | 30672/37962 [71:01:40<16:50:10,  8.31s/it]
 81%|████████  | 30673/37962 [71:01:48<16:50:13,  8.32s/it]
 81%|████████  | 30674/37962 [71:01:56<16:48:11,  8.30s/it]
 81%|████████  | 30675/37962 [71:02:04<16:38:00,  8.22s/it]
 81%|████████  | 30676/37962 [71:02:12<16:38:21,  8.22s/it]
 81%|████████  | 30677/37962 [71:02:21<16:42:28,  8.26s/it]
 81%|████████  | 30678/37962 [71:02:29<16:37:48,  8.22s/it]
 81%|████████  | 30679/37962 [71:02:37<16:41:33,  8.25s/it]
 81%|████████  | 30680/37962 [71:02:45<16:37:23,  8.22s/it]
                                                           
{'loss': 1.1537, 'grad_norm': 14.378952026367188, 'learning_rate': 1.8686094392002585e-06, 'epoch': 0.81}

 81%|████████  | 30680/37962 [71:02:45<16:37:23,  8.22s/it]
 81%|████████  | 30681/37962 [71:02:54<16:41:08,  8.25s/it]
 81%|████████  | 30682/37962 [71:03:02<16:46:02,  8.29s/it]
 81%|████████  | 30683/37962 [71:03:10<16:38:59,  8.23s/it]
 81%|████████  | 30684/37962 [71:03:18<16:32:47,  8.18s/it]
 81%|████████  | 30685/37962 [71:03:27<16:33:00,  8.19s/it]
 81%|████████  | 30686/37962 [71:03:35<16:40:49,  8.25s/it]
 81%|████████  | 30687/37962 [71:03:43<16:35:32,  8.21s/it]
 81%|████████  | 30688/37962 [71:03:51<16:37:14,  8.23s/it]
 81%|████████  | 30689/37962 [71:04:00<16:40:53,  8.26s/it]
 81%|████████  | 30690/37962 [71:04:08<16:31:11,  8.18s/it]
                                                           
{'loss': 1.0988, 'grad_norm': 12.024587631225586, 'learning_rate': 1.8636464126997443e-06, 'epoch': 0.81}

 81%|████████  | 30690/37962 [71:04:08<16:31:11,  8.18s/it]
 81%|████████  | 30691/37962 [71:04:16<16:36:57,  8.23s/it]
 81%|████████  | 30692/37962 [71:04:24<16:33:44,  8.20s/it]
 81%|████████  | 30693/37962 [71:04:32<16:35:22,  8.22s/it]
 81%|████████  | 30694/37962 [71:04:41<16:38:04,  8.24s/it]
 81%|████████  | 30695/37962 [71:04:49<16:35:21,  8.22s/it]
 81%|████████  | 30696/37962 [71:04:57<16:36:53,  8.23s/it]
 81%|████████  | 30697/37962 [71:05:05<16:37:58,  8.24s/it]
 81%|████████  | 30698/37962 [71:05:14<16:36:12,  8.23s/it]
 81%|████████  | 30699/37962 [71:05:22<16:36:30,  8.23s/it]
 81%|████████  | 30700/37962 [71:05:30<16:25:33,  8.14s/it]
                                                           
{'loss': 1.2898, 'grad_norm': 16.845104217529297, 'learning_rate': 1.8586893085100556e-06, 'epoch': 0.81}

 81%|████████  | 30700/37962 [71:05:30<16:25:33,  8.14s/it]
 81%|████████  | 30701/37962 [71:05:38<16:18:37,  8.09s/it]
 81%|████████  | 30702/37962 [71:05:46<16:25:48,  8.15s/it]
 81%|████████  | 30703/37962 [71:05:54<16:29:51,  8.18s/it]
 81%|████████  | 30704/37962 [71:06:03<16:34:34,  8.22s/it]
 81%|████████  | 30705/37962 [71:06:11<16:33:40,  8.22s/it]
 81%|████████  | 30706/37962 [71:06:19<16:35:54,  8.24s/it]
 81%|████████  | 30707/37962 [71:06:27<16:24:21,  8.14s/it]
 81%|████████  | 30708/37962 [71:06:35<16:31:20,  8.20s/it]
 81%|████████  | 30709/37962 [71:06:44<16:37:33,  8.25s/it]
 81%|████████  | 30710/37962 [71:06:51<16:20:37,  8.11s/it]
                                                           
{'loss': 1.2093, 'grad_norm': 6.919190406799316, 'learning_rate': 1.8537381302393787e-06, 'epoch': 0.81}

 81%|████████  | 30710/37962 [71:06:51<16:20:37,  8.11s/it]
 81%|████████  | 30711/37962 [71:07:00<16:26:17,  8.16s/it]
 81%|████████  | 30712/37962 [71:07:08<16:34:32,  8.23s/it]
 81%|████████  | 30713/37962 [71:07:16<16:28:48,  8.18s/it]
 81%|████████  | 30714/37962 [71:07:25<16:33:59,  8.23s/it]
 81%|████████  | 30715/37962 [71:07:33<16:28:01,  8.18s/it]
 81%|████████  | 30716/37962 [71:07:41<16:30:30,  8.20s/it]
 81%|████████  | 30717/37962 [71:07:49<16:28:23,  8.19s/it]
 81%|████████  | 30718/37962 [71:07:57<16:31:42,  8.21s/it]
 81%|████████  | 30719/37962 [71:08:06<16:36:20,  8.25s/it]
 81%|████████  | 30720/37962 [71:08:14<16:35:49,  8.25s/it]
                                                           
{'loss': 1.1422, 'grad_norm': 5.981680870056152, 'learning_rate': 1.8487928814915945e-06, 'epoch': 0.81}

 81%|████████  | 30720/37962 [71:08:14<16:35:49,  8.25s/it]
 81%|████████  | 30721/37962 [71:08:22<16:40:04,  8.29s/it]
 81%|████████  | 30722/37962 [71:08:31<16:47:21,  8.35s/it]
 81%|████████  | 30723/37962 [71:08:39<16:46:03,  8.34s/it]
 81%|████████  | 30724/37962 [71:08:47<16:48:34,  8.36s/it]
 81%|████████  | 30725/37962 [71:08:56<16:48:34,  8.36s/it]
 81%|████████  | 30726/37962 [71:09:04<16:49:57,  8.37s/it]
 81%|████████  | 30727/37962 [71:09:12<16:42:07,  8.31s/it]
 81%|████████  | 30728/37962 [71:09:21<16:42:12,  8.31s/it]
 81%|████████  | 30729/37962 [71:09:29<16:29:27,  8.21s/it]
 81%|████████  | 30730/37962 [71:09:37<16:35:02,  8.26s/it]
                                                           
{'loss': 1.2616, 'grad_norm': 13.231403350830078, 'learning_rate': 1.8438535658662615e-06, 'epoch': 0.81}

 81%|████████  | 30730/37962 [71:09:37<16:35:02,  8.26s/it]
 81%|████████  | 30731/37962 [71:09:45<16:40:07,  8.30s/it]
 81%|████████  | 30732/37962 [71:09:54<16:44:14,  8.33s/it]
 81%|████████  | 30733/37962 [71:10:02<16:40:12,  8.30s/it]
 81%|████████  | 30734/37962 [71:10:10<16:38:55,  8.29s/it]
 81%|████████  | 30735/37962 [71:10:19<16:43:49,  8.33s/it]
 81%|████████  | 30736/37962 [71:10:27<16:44:15,  8.34s/it]
 81%|████████  | 30737/37962 [71:10:35<16:38:36,  8.29s/it]
 81%|████████  | 30738/37962 [71:10:44<16:41:20,  8.32s/it]
 81%|████████  | 30739/37962 [71:10:52<16:28:13,  8.21s/it]
 81%|████████  | 30740/37962 [71:11:00<16:24:21,  8.18s/it]
                                                           
{'loss': 1.1545, 'grad_norm': 11.066880226135254, 'learning_rate': 1.8389201869586205e-06, 'epoch': 0.81}

 81%|████████  | 30740/37962 [71:11:00<16:24:21,  8.18s/it]
 81%|████████  | 30741/37962 [71:11:08<16:38:07,  8.29s/it]
 81%|████████  | 30742/37962 [71:11:16<16:28:59,  8.22s/it]
 81%|████████  | 30743/37962 [71:11:25<16:37:58,  8.29s/it]
 81%|████████  | 30744/37962 [71:11:33<16:38:46,  8.30s/it]
 81%|████████  | 30745/37962 [71:11:41<16:38:44,  8.30s/it]
 81%|████████  | 30746/37962 [71:11:50<16:49:47,  8.40s/it]
 81%|████████  | 30747/37962 [71:11:58<16:43:59,  8.35s/it]
 81%|████████  | 30748/37962 [71:12:07<16:43:06,  8.34s/it]
 81%|████████  | 30749/37962 [71:12:15<16:43:02,  8.34s/it]
 81%|████████  | 30750/37962 [71:12:23<16:40:43,  8.33s/it]
                                                           
{'loss': 1.2404, 'grad_norm': 6.695717811584473, 'learning_rate': 1.8339927483595965e-06, 'epoch': 0.81}

 81%|████████  | 30750/37962 [71:12:23<16:40:43,  8.33s/it]
 81%|████████  | 30751/37962 [71:12:32<16:38:53,  8.31s/it]
 81%|████████  | 30752/37962 [71:12:39<16:20:42,  8.16s/it]
 81%|████████  | 30753/37962 [71:12:48<16:30:22,  8.24s/it]
 81%|████████  | 30754/37962 [71:12:56<16:38:08,  8.31s/it]
 81%|████████  | 30755/37962 [71:13:05<16:41:18,  8.34s/it]
 81%|████████  | 30756/37962 [71:13:13<16:42:07,  8.34s/it]
 81%|████████  | 30757/37962 [71:13:21<16:34:26,  8.28s/it]
 81%|████████  | 30758/37962 [71:13:29<16:35:44,  8.29s/it]
 81%|████████  | 30759/37962 [71:13:38<16:38:22,  8.32s/it]
 81%|████████  | 30760/37962 [71:13:46<16:41:07,  8.34s/it]
                                                           
{'loss': 1.1426, 'grad_norm': 8.397920608520508, 'learning_rate': 1.829071253655782e-06, 'epoch': 0.81}

 81%|████████  | 30760/37962 [71:13:46<16:41:07,  8.34s/it]
 81%|████████  | 30761/37962 [71:13:55<16:40:45,  8.34s/it]
 81%|████████  | 30762/37962 [71:14:03<16:42:26,  8.35s/it]
 81%|████████  | 30763/37962 [71:14:11<16:36:54,  8.31s/it]
 81%|████████  | 30764/37962 [71:14:19<16:32:46,  8.28s/it]
 81%|████████  | 30765/37962 [71:14:28<16:29:47,  8.25s/it]
 81%|████████  | 30766/37962 [71:14:36<16:39:30,  8.33s/it]
 81%|████████  | 30767/37962 [71:14:44<16:38:25,  8.33s/it]
 81%|████████  | 30768/37962 [71:14:53<16:33:38,  8.29s/it]
 81%|████████  | 30769/37962 [71:15:01<16:34:33,  8.30s/it]
 81%|████████  | 30770/37962 [71:15:09<16:39:00,  8.33s/it]
                                                           
{'loss': 1.2197, 'grad_norm': 9.284100532531738, 'learning_rate': 1.8241557064294513e-06, 'epoch': 0.81}

 81%|████████  | 30770/37962 [71:15:09<16:39:00,  8.33s/it]
 81%|████████  | 30771/37962 [71:15:18<16:40:02,  8.34s/it]
 81%|████████  | 30772/37962 [71:15:26<16:34:32,  8.30s/it]
 81%|████████  | 30773/37962 [71:15:34<16:33:09,  8.29s/it]
 81%|████████  | 30774/37962 [71:15:42<16:34:18,  8.30s/it]
 81%|████████  | 30775/37962 [71:15:51<16:30:49,  8.27s/it]
 81%|████████  | 30776/37962 [71:15:59<16:27:06,  8.24s/it]
 81%|████████  | 30777/37962 [71:16:07<16:27:54,  8.25s/it]
 81%|████████  | 30778/37962 [71:16:15<16:28:19,  8.25s/it]
 81%|████████  | 30779/37962 [71:16:24<16:27:57,  8.25s/it]
 81%|████████  | 30780/37962 [71:16:32<16:26:00,  8.24s/it]
                                                           
{'loss': 1.2595, 'grad_norm': 10.304032325744629, 'learning_rate': 1.819246110258549e-06, 'epoch': 0.81}

 81%|████████  | 30780/37962 [71:16:32<16:26:00,  8.24s/it]
 81%|████████  | 30781/37962 [71:16:40<16:29:51,  8.27s/it]
 81%|████████  | 30782/37962 [71:16:48<16:28:57,  8.26s/it]
 81%|████████  | 30783/37962 [71:16:57<16:34:08,  8.31s/it]
 81%|████████  | 30784/37962 [71:17:05<16:34:11,  8.31s/it]
 81%|████████  | 30785/37962 [71:17:14<16:36:30,  8.33s/it]
 81%|████████  | 30786/37962 [71:17:22<16:31:09,  8.29s/it]
 81%|████████  | 30787/37962 [71:17:30<16:35:17,  8.32s/it]
 81%|████████  | 30788/37962 [71:17:38<16:29:25,  8.28s/it]
 81%|████████  | 30789/37962 [71:17:47<16:31:25,  8.29s/it]
 81%|████████  | 30790/37962 [71:17:55<16:31:05,  8.29s/it]
                                                           
{'loss': 1.2529, 'grad_norm': 8.334758758544922, 'learning_rate': 1.8143424687166767e-06, 'epoch': 0.81}

 81%|████████  | 30790/37962 [71:17:55<16:31:05,  8.29s/it]
 81%|████████  | 30791/37962 [71:18:03<16:32:11,  8.30s/it]
 81%|████████  | 30792/37962 [71:18:11<16:11:33,  8.13s/it]
 81%|████████  | 30793/37962 [71:18:19<16:21:21,  8.21s/it]
 81%|████████  | 30794/37962 [71:18:28<16:21:29,  8.22s/it]
 81%|████████  | 30795/37962 [71:18:36<16:14:57,  8.16s/it]
 81%|████████  | 30796/37962 [71:18:44<16:18:08,  8.19s/it]
 81%|████████  | 30797/37962 [71:18:52<16:22:34,  8.23s/it]
 81%|████████  | 30798/37962 [71:19:00<16:20:01,  8.21s/it]
 81%|████████  | 30799/37962 [71:19:09<16:22:44,  8.23s/it]
 81%|████████  | 30800/37962 [71:19:17<16:24:41,  8.25s/it]
                                                           
{'loss': 1.1339, 'grad_norm': 7.986595153808594, 'learning_rate': 1.8094447853731145e-06, 'epoch': 0.81}

 81%|████████  | 30800/37962 [71:19:17<16:24:41,  8.25s/it]
 81%|████████  | 30801/37962 [71:19:25<16:16:14,  8.18s/it]
 81%|████████  | 30802/37962 [71:19:33<16:19:47,  8.21s/it]
 81%|████████  | 30803/37962 [71:19:42<16:23:33,  8.24s/it]
 81%|████████  | 30804/37962 [71:19:50<16:28:34,  8.29s/it]
 81%|████████  | 30805/37962 [71:19:58<16:28:21,  8.29s/it]
 81%|████████  | 30806/37962 [71:20:06<16:28:06,  8.28s/it]
 81%|████████  | 30807/37962 [71:20:15<16:26:55,  8.28s/it]
 81%|████████  | 30808/37962 [71:20:23<16:27:19,  8.28s/it]
 81%|████████  | 30809/37962 [71:20:31<16:15:34,  8.18s/it]
 81%|████████  | 30810/37962 [71:20:39<16:16:15,  8.19s/it]
                                                           
{'loss': 1.2074, 'grad_norm': 5.3527655601501465, 'learning_rate': 1.8045530637928033e-06, 'epoch': 0.81}

 81%|████████  | 30810/37962 [71:20:39<16:16:15,  8.19s/it]
 81%|████████  | 30811/37962 [71:20:47<16:17:09,  8.20s/it]
 81%|████████  | 30812/37962 [71:20:56<16:28:19,  8.29s/it]
 81%|████████  | 30813/37962 [71:21:04<16:27:13,  8.29s/it]
 81%|████████  | 30814/37962 [71:21:12<16:23:29,  8.26s/it]
 81%|████████  | 30815/37962 [71:21:21<16:22:20,  8.25s/it]
 81%|████████  | 30816/37962 [71:21:29<16:10:31,  8.15s/it]
 81%|████████  | 30817/37962 [71:21:37<16:21:46,  8.24s/it]
 81%|████████  | 30818/37962 [71:21:45<16:20:45,  8.24s/it]
 81%|████████  | 30819/37962 [71:21:54<16:22:30,  8.25s/it]
 81%|████████  | 30820/37962 [71:22:02<16:27:16,  8.29s/it]
                                                           
{'loss': 1.2215, 'grad_norm': 7.946804046630859, 'learning_rate': 1.7996673075363391e-06, 'epoch': 0.81}

 81%|████████  | 30820/37962 [71:22:02<16:27:16,  8.29s/it]
 81%|████████  | 30821/37962 [71:22:10<16:27:14,  8.29s/it]
 81%|████████  | 30822/37962 [71:22:18<16:16:31,  8.21s/it]
 81%|████████  | 30823/37962 [71:22:26<16:16:10,  8.20s/it]
 81%|████████  | 30824/37962 [71:22:35<16:31:27,  8.33s/it]
 81%|████████  | 30825/37962 [71:22:43<16:31:02,  8.33s/it]
 81%|████████  | 30826/37962 [71:22:52<16:25:56,  8.29s/it]
 81%|████████  | 30827/37962 [71:23:00<16:20:26,  8.24s/it]
 81%|████████  | 30828/37962 [71:23:08<16:23:02,  8.27s/it]
 81%|████████  | 30829/37962 [71:23:16<16:25:12,  8.29s/it]
 81%|████████  | 30830/37962 [71:23:25<16:23:07,  8.27s/it]
                                                           
{'loss': 1.0723, 'grad_norm': 7.503734111785889, 'learning_rate': 1.794787520159983e-06, 'epoch': 0.81}

 81%|████████  | 30830/37962 [71:23:25<16:23:07,  8.27s/it]
 81%|████████  | 30831/37962 [71:23:33<16:24:42,  8.29s/it]
 81%|████████  | 30832/37962 [71:23:41<16:21:52,  8.26s/it]
 81%|████████  | 30833/37962 [71:23:50<16:31:21,  8.34s/it]
 81%|████████  | 30834/37962 [71:23:58<16:28:46,  8.32s/it]
 81%|████████  | 30835/37962 [71:24:06<16:27:04,  8.31s/it]
 81%|████████  | 30836/37962 [71:24:14<16:15:13,  8.21s/it]
 81%|████████  | 30837/37962 [71:24:23<16:21:59,  8.27s/it]
 81%|████████  | 30838/37962 [71:24:31<16:30:00,  8.34s/it]
 81%|████████  | 30839/37962 [71:24:39<16:29:44,  8.34s/it]
 81%|████████  | 30840/37962 [71:24:48<16:24:34,  8.29s/it]
                                                           
{'loss': 1.1157, 'grad_norm': 6.672111511230469, 'learning_rate': 1.7899137052156445e-06, 'epoch': 0.81}

 81%|████████  | 30840/37962 [71:24:48<16:24:34,  8.29s/it]
 81%|████████  | 30841/37962 [71:24:56<16:25:14,  8.30s/it]
 81%|████████  | 30842/37962 [71:25:04<16:24:02,  8.29s/it]
 81%|████████  | 30843/37962 [71:25:12<16:21:18,  8.27s/it]
 81%|████████  | 30844/37962 [71:25:21<16:21:13,  8.27s/it]
 81%|████████▏ | 30845/37962 [71:25:29<16:23:53,  8.29s/it]
 81%|████████▏ | 30846/37962 [71:25:37<16:28:26,  8.33s/it]
 81%|████████▏ | 30847/37962 [71:25:46<16:18:45,  8.25s/it]
 81%|████████▏ | 30848/37962 [71:25:54<16:22:29,  8.29s/it]
 81%|████████▏ | 30849/37962 [71:26:02<16:26:19,  8.32s/it]
 81%|████████▏ | 30850/37962 [71:26:11<16:36:50,  8.41s/it]
                                                           
{'loss': 1.2212, 'grad_norm': 8.291959762573242, 'learning_rate': 1.7850458662508884e-06, 'epoch': 0.81}

 81%|████████▏ | 30850/37962 [71:26:11<16:36:50,  8.41s/it]
 81%|████████▏ | 30851/37962 [71:26:19<16:30:15,  8.36s/it]
 81%|████████▏ | 30852/37962 [71:26:27<16:27:41,  8.33s/it]
 81%|████████▏ | 30853/37962 [71:26:35<16:14:18,  8.22s/it]
 81%|████████▏ | 30854/37962 [71:26:44<16:15:55,  8.24s/it]
 81%|████████▏ | 30855/37962 [71:26:52<16:20:31,  8.28s/it]
 81%|████████▏ | 30856/37962 [71:27:09<21:19:51, 10.81s/it]
 81%|████████▏ | 30857/37962 [71:27:17<19:41:17,  9.98s/it]
 81%|████████▏ | 30858/37962 [71:27:25<18:42:20,  9.48s/it]
 81%|████████▏ | 30859/37962 [71:27:33<18:01:52,  9.14s/it]
 81%|████████▏ | 30860/37962 [71:27:42<17:32:53,  8.90s/it]
                                                           
{'loss': 1.1289, 'grad_norm': 12.518600463867188, 'learning_rate': 1.7801840068089337e-06, 'epoch': 0.81}

 81%|████████▏ | 30860/37962 [71:27:42<17:32:53,  8.90s/it]
 81%|████████▏ | 30861/37962 [71:27:50<16:58:34,  8.61s/it]
 81%|████████▏ | 30862/37962 [71:27:58<16:39:52,  8.45s/it]
 81%|████████▏ | 30863/37962 [71:28:06<16:29:06,  8.36s/it]
 81%|████████▏ | 30864/37962 [71:28:14<16:23:34,  8.31s/it]
 81%|████████▏ | 30865/37962 [71:28:22<16:23:04,  8.31s/it]
 81%|████████▏ | 30866/37962 [71:28:31<16:15:53,  8.25s/it]
 81%|████████▏ | 30867/37962 [71:28:39<16:21:16,  8.30s/it]
 81%|████████▏ | 30868/37962 [71:28:47<16:18:39,  8.28s/it]
 81%|████████▏ | 30869/37962 [71:28:55<16:16:20,  8.26s/it]
 81%|████████▏ | 30870/37962 [71:29:03<16:04:36,  8.16s/it]
                                                           
{'loss': 1.1697, 'grad_norm': 6.536609172821045, 'learning_rate': 1.7753281304286397e-06, 'epoch': 0.81}

 81%|████████▏ | 30870/37962 [71:29:03<16:04:36,  8.16s/it]
 81%|████████▏ | 30871/37962 [71:29:12<16:09:29,  8.20s/it]
 81%|████████▏ | 30872/37962 [71:29:20<16:05:47,  8.17s/it]
 81%|████████▏ | 30873/37962 [71:29:28<16:09:41,  8.21s/it]
 81%|████████▏ | 30874/37962 [71:29:36<16:04:21,  8.16s/it]
 81%|████████▏ | 30875/37962 [71:29:44<16:11:50,  8.23s/it]
 81%|████████▏ | 30876/37962 [71:29:53<16:06:04,  8.18s/it]
 81%|████████▏ | 30877/37962 [71:30:01<16:05:12,  8.17s/it]
 81%|████████▏ | 30878/37962 [71:30:09<16:06:04,  8.18s/it]
 81%|████████▏ | 30879/37962 [71:30:17<16:01:53,  8.15s/it]
 81%|████████▏ | 30880/37962 [71:30:25<16:03:08,  8.16s/it]
                                                           
{'loss': 1.1448, 'grad_norm': 8.193613052368164, 'learning_rate': 1.7704782406445197e-06, 'epoch': 0.81}

 81%|████████▏ | 30880/37962 [71:30:25<16:03:08,  8.16s/it]
 81%|████████▏ | 30881/37962 [71:30:33<16:08:56,  8.21s/it]
 81%|████████▏ | 30882/37962 [71:30:42<16:10:57,  8.23s/it]
 81%|████████▏ | 30883/37962 [71:30:50<16:14:10,  8.26s/it]
 81%|████████▏ | 30884/37962 [71:30:58<16:12:03,  8.24s/it]
 81%|████████▏ | 30885/37962 [71:31:06<16:07:38,  8.20s/it]
 81%|████████▏ | 30886/37962 [71:31:14<15:56:13,  8.11s/it]
 81%|████████▏ | 30887/37962 [71:31:23<16:04:31,  8.18s/it]
 81%|████████▏ | 30888/37962 [71:31:31<16:14:24,  8.26s/it]
 81%|████████▏ | 30889/37962 [71:31:39<16:13:34,  8.26s/it]
 81%|████████▏ | 30890/37962 [71:31:48<16:12:44,  8.25s/it]
                                                           
{'loss': 1.201, 'grad_norm': 13.966492652893066, 'learning_rate': 1.7656343409867215e-06, 'epoch': 0.81}

 81%|████████▏ | 30890/37962 [71:31:48<16:12:44,  8.25s/it]
 81%|████████▏ | 30891/37962 [71:31:56<16:10:29,  8.24s/it]
 81%|████████▏ | 30892/37962 [71:32:04<16:12:13,  8.25s/it]
 81%|████████▏ | 30893/37962 [71:32:12<16:04:15,  8.18s/it]
 81%|████████▏ | 30894/37962 [71:32:20<16:07:53,  8.22s/it]
 81%|████████▏ | 30895/37962 [71:32:29<16:09:00,  8.23s/it]
 81%|████████▏ | 30896/37962 [71:32:37<16:16:29,  8.29s/it]
 81%|████████▏ | 30897/37962 [71:32:45<16:13:26,  8.27s/it]
 81%|████████▏ | 30898/37962 [71:32:53<16:10:12,  8.24s/it]
 81%|████████▏ | 30899/37962 [71:33:02<16:09:32,  8.24s/it]
 81%|████████▏ | 30900/37962 [71:33:10<16:12:33,  8.26s/it]
                                                           
{'loss': 1.2047, 'grad_norm': 20.347383499145508, 'learning_rate': 1.7607964349810337e-06, 'epoch': 0.81}

 81%|████████▏ | 30900/37962 [71:33:10<16:12:33,  8.26s/it]
 81%|████████▏ | 30901/37962 [71:33:19<16:20:18,  8.33s/it]
 81%|████████▏ | 30902/37962 [71:33:26<16:03:13,  8.19s/it]
 81%|████████▏ | 30903/37962 [71:33:35<16:09:15,  8.24s/it]
 81%|████████▏ | 30904/37962 [71:33:43<16:09:24,  8.24s/it]
 81%|████████▏ | 30905/37962 [71:33:51<16:19:17,  8.33s/it]
 81%|████████▏ | 30906/37962 [71:33:59<16:06:30,  8.22s/it]
 81%|████████▏ | 30907/37962 [71:34:08<16:12:32,  8.27s/it]
 81%|████████▏ | 30908/37962 [71:34:16<16:15:09,  8.29s/it]
 81%|████████▏ | 30909/37962 [71:34:24<16:01:22,  8.18s/it]
 81%|████████▏ | 30910/37962 [71:34:32<16:05:54,  8.22s/it]
                                                           
{'loss': 1.1768, 'grad_norm': 7.036605358123779, 'learning_rate': 1.7559645261488856e-06, 'epoch': 0.81}

 81%|████████▏ | 30910/37962 [71:34:32<16:05:54,  8.22s/it]
 81%|████████▏ | 30911/37962 [71:34:41<16:05:46,  8.22s/it]
 81%|████████▏ | 30912/37962 [71:34:49<16:09:34,  8.25s/it]
 81%|████████▏ | 30913/37962 [71:34:57<16:06:21,  8.23s/it]
 81%|████████▏ | 30914/37962 [71:35:06<16:12:29,  8.28s/it]
 81%|████████▏ | 30915/37962 [71:35:14<16:09:21,  8.25s/it]
 81%|████████▏ | 30916/37962 [71:35:22<16:06:16,  8.23s/it]
 81%|████████▏ | 30917/37962 [71:35:30<16:04:16,  8.21s/it]
 81%|████████▏ | 30918/37962 [71:35:38<16:03:04,  8.20s/it]
 81%|████████▏ | 30919/37962 [71:35:46<15:59:37,  8.18s/it]
 81%|████████▏ | 30920/37962 [71:35:55<16:08:36,  8.25s/it]
                                                           
{'loss': 1.1316, 'grad_norm': 7.031517028808594, 'learning_rate': 1.7511386180073442e-06, 'epoch': 0.81}

 81%|████████▏ | 30920/37962 [71:35:55<16:08:36,  8.25s/it]
 81%|████████▏ | 30921/37962 [71:36:03<16:10:57,  8.27s/it]
 81%|████████▏ | 30922/37962 [71:36:12<16:19:44,  8.35s/it]
 81%|████████▏ | 30923/37962 [71:36:19<15:56:44,  8.16s/it]
 81%|████████▏ | 30924/37962 [71:36:27<15:50:52,  8.11s/it]
 81%|████████▏ | 30925/37962 [71:36:36<16:12:47,  8.29s/it]
 81%|████████▏ | 30926/37962 [71:36:44<16:13:40,  8.30s/it]
 81%|████████▏ | 30927/37962 [71:36:52<16:05:48,  8.24s/it]
 81%|████████▏ | 30928/37962 [71:37:01<16:14:56,  8.32s/it]
 81%|████████▏ | 30929/37962 [71:37:09<16:06:11,  8.24s/it]
 81%|████████▏ | 30930/37962 [71:37:17<16:09:58,  8.28s/it]
                                                           
{'loss': 1.1893, 'grad_norm': 16.96132469177246, 'learning_rate': 1.7463187140690974e-06, 'epoch': 0.81}

 81%|████████▏ | 30930/37962 [71:37:17<16:09:58,  8.28s/it]
 81%|████████▏ | 30931/37962 [71:37:26<16:09:37,  8.27s/it]
 81%|████████▏ | 30932/37962 [71:37:34<16:11:36,  8.29s/it]
 81%|████████▏ | 30933/37962 [71:37:42<16:04:59,  8.24s/it]
 81%|████████▏ | 30934/37962 [71:37:50<16:04:49,  8.24s/it]
 81%|████████▏ | 30935/37962 [71:37:59<16:13:41,  8.31s/it]
 81%|████████▏ | 30936/37962 [71:38:07<16:10:17,  8.29s/it]
 81%|████████▏ | 30937/37962 [71:38:15<16:06:59,  8.26s/it]
 81%|████████▏ | 30938/37962 [71:38:23<15:57:19,  8.18s/it]
 81%|████████▏ | 30939/37962 [71:38:32<16:08:41,  8.28s/it]
 82%|████████▏ | 30940/37962 [71:38:40<16:08:35,  8.28s/it]
                                                           
{'loss': 1.2012, 'grad_norm': 8.5442476272583, 'learning_rate': 1.7415048178424765e-06, 'epoch': 0.82}

 82%|████████▏ | 30940/37962 [71:38:40<16:08:35,  8.28s/it]
 82%|████████▏ | 30941/37962 [71:38:48<16:09:45,  8.29s/it]
 82%|████████▏ | 30942/37962 [71:38:57<16:15:56,  8.34s/it]
 82%|████████▏ | 30943/37962 [71:39:05<16:12:15,  8.31s/it]
 82%|████████▏ | 30944/37962 [71:39:13<16:08:02,  8.28s/it]
 82%|████████▏ | 30945/37962 [71:39:21<16:04:01,  8.24s/it]
 82%|████████▏ | 30946/37962 [71:39:30<16:07:20,  8.27s/it]
 82%|████████▏ | 30947/37962 [71:39:38<16:15:37,  8.34s/it]
 82%|████████▏ | 30948/37962 [71:39:47<16:15:51,  8.35s/it]
 82%|████████▏ | 30949/37962 [71:39:55<16:18:16,  8.37s/it]
 82%|████████▏ | 30950/37962 [71:40:03<16:18:55,  8.38s/it]
                                                           
{'loss': 1.1133, 'grad_norm': 11.770572662353516, 'learning_rate': 1.7366969328314299e-06, 'epoch': 0.82}

 82%|████████▏ | 30950/37962 [71:40:03<16:18:55,  8.38s/it]
 82%|████████▏ | 30951/37962 [71:40:12<16:17:27,  8.37s/it]
 82%|████████▏ | 30952/37962 [71:40:20<16:12:19,  8.32s/it]
 82%|████████▏ | 30953/37962 [71:40:29<16:18:43,  8.38s/it]
 82%|████████▏ | 30954/37962 [71:40:37<16:12:44,  8.33s/it]
 82%|████████▏ | 30955/37962 [71:40:45<16:13:18,  8.33s/it]
 82%|████████▏ | 30956/37962 [71:40:53<16:16:04,  8.36s/it]
 82%|████████▏ | 30957/37962 [71:41:02<16:11:36,  8.32s/it]
 82%|████████▏ | 30958/37962 [71:41:10<16:12:01,  8.33s/it]
 82%|████████▏ | 30959/37962 [71:41:18<15:57:34,  8.20s/it]
 82%|████████▏ | 30960/37962 [71:41:26<15:56:28,  8.20s/it]
                                                           
{'loss': 1.229, 'grad_norm': 7.239218235015869, 'learning_rate': 1.7318950625355323e-06, 'epoch': 0.82}

 82%|████████▏ | 30960/37962 [71:41:26<15:56:28,  8.20s/it]
 82%|████████▏ | 30961/37962 [71:41:34<16:01:07,  8.24s/it]
 82%|████████▏ | 30962/37962 [71:41:42<15:49:48,  8.14s/it]
 82%|████████▏ | 30963/37962 [71:41:51<15:54:20,  8.18s/it]
 82%|████████▏ | 30964/37962 [71:41:59<15:49:42,  8.14s/it]
 82%|████████▏ | 30965/37962 [71:42:07<15:57:55,  8.21s/it]
 82%|████████▏ | 30966/37962 [71:42:15<15:33:35,  8.01s/it]
 82%|████████▏ | 30967/37962 [71:42:23<15:35:26,  8.02s/it]
 82%|████████▏ | 30968/37962 [71:42:31<15:29:24,  7.97s/it]
 82%|████████▏ | 30969/37962 [71:42:39<15:38:28,  8.05s/it]
 82%|████████▏ | 30970/37962 [71:42:47<15:42:06,  8.08s/it]
                                                           
{'loss': 1.1614, 'grad_norm': 8.630463600158691, 'learning_rate': 1.727099210449983e-06, 'epoch': 0.82}

 82%|████████▏ | 30970/37962 [71:42:47<15:42:06,  8.08s/it]
 82%|████████▏ | 30971/37962 [71:42:55<15:56:02,  8.21s/it]
 82%|████████▏ | 30972/37962 [71:43:04<15:54:05,  8.19s/it]
 82%|████████▏ | 30973/37962 [71:43:11<15:36:59,  8.04s/it]
 82%|████████▏ | 30974/37962 [71:43:20<15:48:36,  8.14s/it]
 82%|████████▏ | 30975/37962 [71:43:28<15:51:31,  8.17s/it]
 82%|████████▏ | 30976/37962 [71:43:36<15:57:18,  8.22s/it]
 82%|████████▏ | 30977/37962 [71:43:44<15:54:36,  8.20s/it]
 82%|████████▏ | 30978/37962 [71:43:53<15:53:49,  8.19s/it]
 82%|████████▏ | 30979/37962 [71:44:01<15:49:35,  8.16s/it]
 82%|████████▏ | 30980/37962 [71:44:09<15:55:21,  8.21s/it]
                                                           
{'loss': 1.1189, 'grad_norm': 7.736532688140869, 'learning_rate': 1.722309380065602e-06, 'epoch': 0.82}

 82%|████████▏ | 30980/37962 [71:44:09<15:55:21,  8.21s/it]
 82%|████████▏ | 30981/37962 [71:44:17<15:59:53,  8.25s/it]
 82%|████████▏ | 30982/37962 [71:44:26<16:01:16,  8.26s/it]
 82%|████████▏ | 30983/37962 [71:44:34<16:05:09,  8.30s/it]
 82%|████████▏ | 30984/37962 [71:44:42<15:57:21,  8.23s/it]
 82%|████████▏ | 30985/37962 [71:44:50<15:59:26,  8.25s/it]
 82%|████████▏ | 30986/37962 [71:44:59<15:59:01,  8.25s/it]
 82%|████████▏ | 30987/37962 [71:45:07<15:58:38,  8.25s/it]
 82%|████████▏ | 30988/37962 [71:45:15<16:02:45,  8.28s/it]
 82%|████████▏ | 30989/37962 [71:45:24<16:03:40,  8.29s/it]
 82%|████████▏ | 30990/37962 [71:45:31<15:48:09,  8.16s/it]
                                                           
{'loss': 1.153, 'grad_norm': 7.92650032043457, 'learning_rate': 1.7175255748688236e-06, 'epoch': 0.82}

 82%|████████▏ | 30990/37962 [71:45:31<15:48:09,  8.16s/it]
 82%|████████▏ | 30991/37962 [71:45:39<15:45:51,  8.14s/it]
 82%|████████▏ | 30992/37962 [71:45:48<15:43:49,  8.12s/it]
 82%|████████▏ | 30993/37962 [71:45:56<15:47:46,  8.16s/it]
 82%|████████▏ | 30994/37962 [71:46:04<15:51:17,  8.19s/it]
 82%|████████▏ | 30995/37962 [71:46:12<15:49:19,  8.18s/it]
 82%|████████▏ | 30996/37962 [71:46:20<15:51:30,  8.20s/it]
 82%|████████▏ | 30997/37962 [71:46:28<15:40:26,  8.10s/it]
 82%|████████▏ | 30998/37962 [71:46:36<15:41:11,  8.11s/it]
 82%|████████▏ | 30999/37962 [71:46:45<15:48:54,  8.18s/it]
 82%|████████▏ | 31000/37962 [71:46:53<15:47:21,  8.16s/it]
                                                           
{'loss': 1.1043, 'grad_norm': 8.156909942626953, 'learning_rate': 1.7127477983416917e-06, 'epoch': 0.82}

 82%|████████▏ | 31000/37962 [71:46:53<15:47:21,  8.16s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 82%|████████▏ | 31001/37962 [71:47:44<40:30:42, 20.95s/it]
 82%|████████▏ | 31002/37962 [71:47:52<32:53:12, 17.01s/it]
 82%|████████▏ | 31003/37962 [71:48:00<27:47:27, 14.38s/it]
 82%|████████▏ | 31004/37962 [71:48:08<24:05:30, 12.46s/it]
 82%|████████▏ | 31005/37962 [71:48:16<21:45:29, 11.26s/it]
 82%|████████▏ | 31006/37962 [71:48:25<20:03:35, 10.38s/it]
 82%|████████▏ | 31007/37962 [71:48:33<18:48:38,  9.74s/it]
 82%|████████▏ | 31008/37962 [71:48:41<17:46:24,  9.20s/it]
 82%|████████▏ | 31009/37962 [71:48:49<17:06:50,  8.86s/it]
 82%|████████▏ | 31010/37962 [71:48:57<16:39:40,  8.63s/it]
                                                           
{'loss': 1.151, 'grad_norm': 7.331878185272217, 'learning_rate': 1.7079760539618729e-06, 'epoch': 0.82}

 82%|████████▏ | 31010/37962 [71:48:57<16:39:40,  8.63s/it]
 82%|████████▏ | 31011/37962 [71:49:05<16:29:58,  8.55s/it]
 82%|████████▏ | 31012/37962 [71:49:14<16:19:48,  8.46s/it]
 82%|████████▏ | 31013/37962 [71:49:22<16:12:49,  8.40s/it]
 82%|████████▏ | 31014/37962 [71:49:30<16:08:24,  8.36s/it]
 82%|████████▏ | 31015/37962 [71:49:38<16:10:40,  8.38s/it]
 82%|████████▏ | 31016/37962 [71:49:47<16:08:38,  8.37s/it]
 82%|████████▏ | 31017/37962 [71:49:55<16:02:03,  8.31s/it]
 82%|████████▏ | 31018/37962 [71:50:03<15:55:41,  8.26s/it]
 82%|████████▏ | 31019/37962 [71:50:11<15:59:27,  8.29s/it]
 82%|████████▏ | 31020/37962 [71:50:20<15:55:48,  8.26s/it]
                                                           
{'loss': 1.1558, 'grad_norm': 7.292756080627441, 'learning_rate': 1.7032103452026327e-06, 'epoch': 0.82}

 82%|████████▏ | 31020/37962 [71:50:20<15:55:48,  8.26s/it]
 82%|████████▏ | 31021/37962 [71:50:28<15:55:43,  8.26s/it]
 82%|████████▏ | 31022/37962 [71:50:36<15:58:48,  8.29s/it]
 82%|████████▏ | 31023/37962 [71:50:45<15:56:19,  8.27s/it]
 82%|████████▏ | 31024/37962 [71:50:53<15:59:57,  8.30s/it]
 82%|████████▏ | 31025/37962 [71:51:01<15:52:08,  8.24s/it]
 82%|████████▏ | 31026/37962 [71:51:09<16:00:50,  8.31s/it]
 82%|████████▏ | 31027/37962 [71:51:18<15:57:06,  8.28s/it]
 82%|████████▏ | 31028/37962 [71:51:26<15:54:25,  8.26s/it]
 82%|████████▏ | 31029/37962 [71:51:34<16:02:06,  8.33s/it]
 82%|████████▏ | 31030/37962 [71:51:43<16:03:28,  8.34s/it]
                                                           
{'loss': 1.1534, 'grad_norm': 12.1723051071167, 'learning_rate': 1.6984506755328522e-06, 'epoch': 0.82}

 82%|████████▏ | 31030/37962 [71:51:43<16:03:28,  8.34s/it]
 82%|████████▏ | 31031/37962 [71:51:51<15:59:56,  8.31s/it]
 82%|████████▏ | 31032/37962 [71:51:59<16:00:12,  8.31s/it]
 82%|████████▏ | 31033/37962 [71:52:08<16:01:35,  8.33s/it]
 82%|████████▏ | 31034/37962 [71:52:16<16:00:57,  8.32s/it]
 82%|████████▏ | 31035/37962 [71:52:24<16:01:30,  8.33s/it]
 82%|████████▏ | 31036/37962 [71:52:32<15:48:07,  8.21s/it]
 82%|████████▏ | 31037/37962 [71:52:40<15:48:29,  8.22s/it]
 82%|████████▏ | 31038/37962 [71:52:48<15:39:34,  8.14s/it]
 82%|████████▏ | 31039/37962 [71:52:57<15:42:11,  8.17s/it]
 82%|████████▏ | 31040/37962 [71:53:04<15:28:24,  8.05s/it]
                                                           
{'loss': 1.0971, 'grad_norm': 7.21678352355957, 'learning_rate': 1.6936970484170111e-06, 'epoch': 0.82}

 82%|████████▏ | 31040/37962 [71:53:04<15:28:24,  8.05s/it]
 82%|████████▏ | 31041/37962 [71:53:13<15:34:23,  8.10s/it]
 82%|████████▏ | 31042/37962 [71:53:21<15:29:20,  8.06s/it]
 82%|████████▏ | 31043/37962 [71:53:29<15:31:13,  8.08s/it]
 82%|████████▏ | 31044/37962 [71:53:37<15:26:27,  8.04s/it]
 82%|████████▏ | 31045/37962 [71:53:45<15:37:29,  8.13s/it]
 82%|████████▏ | 31046/37962 [71:53:54<15:50:38,  8.25s/it]
 82%|████████▏ | 31047/37962 [71:54:02<15:53:26,  8.27s/it]
 82%|████████▏ | 31048/37962 [71:54:10<15:47:47,  8.22s/it]
 82%|████████▏ | 31049/37962 [71:54:18<15:48:59,  8.24s/it]
 82%|████████▏ | 31050/37962 [71:54:27<15:53:34,  8.28s/it]
                                                           
{'loss': 1.1066, 'grad_norm': 9.061594009399414, 'learning_rate': 1.6889494673151897e-06, 'epoch': 0.82}

 82%|████████▏ | 31050/37962 [71:54:27<15:53:34,  8.28s/it]
 82%|████████▏ | 31051/37962 [71:54:35<15:56:18,  8.30s/it]
 82%|████████▏ | 31052/37962 [71:54:43<15:41:58,  8.18s/it]
 82%|████████▏ | 31053/37962 [71:54:51<15:44:20,  8.20s/it]
 82%|████████▏ | 31054/37962 [71:54:59<15:49:24,  8.25s/it]
 82%|████████▏ | 31055/37962 [71:55:08<15:48:36,  8.24s/it]
 82%|████████▏ | 31056/37962 [71:55:16<15:41:19,  8.18s/it]
 82%|████████▏ | 31057/37962 [71:55:24<15:46:59,  8.23s/it]
 82%|████████▏ | 31058/37962 [71:55:32<15:52:10,  8.27s/it]
 82%|████████▏ | 31059/37962 [71:55:41<15:50:07,  8.26s/it]
 82%|████████▏ | 31060/37962 [71:55:49<15:49:56,  8.26s/it]
                                                           
{'loss': 1.2331, 'grad_norm': 6.573923587799072, 'learning_rate': 1.6842079356830732e-06, 'epoch': 0.82}

 82%|████████▏ | 31060/37962 [71:55:49<15:49:56,  8.26s/it]
 82%|████████▏ | 31061/37962 [71:55:57<15:50:21,  8.26s/it]
 82%|████████▏ | 31062/37962 [71:56:05<15:49:20,  8.26s/it]
 82%|████████▏ | 31063/37962 [71:56:14<15:52:39,  8.29s/it]
 82%|████████▏ | 31064/37962 [71:56:22<15:51:14,  8.27s/it]
 82%|████████▏ | 31065/37962 [71:56:30<15:48:31,  8.25s/it]
 82%|████████▏ | 31066/37962 [71:56:39<16:00:19,  8.36s/it]
 82%|████████▏ | 31067/37962 [71:56:47<15:58:44,  8.34s/it]
 82%|████████▏ | 31068/37962 [71:56:55<15:53:01,  8.29s/it]
 82%|████████▏ | 31069/37962 [71:57:04<15:54:28,  8.31s/it]
 82%|████████▏ | 31070/37962 [71:57:12<15:43:15,  8.21s/it]
                                                           
{'loss': 1.1589, 'grad_norm': 9.935912132263184, 'learning_rate': 1.679472456971938e-06, 'epoch': 0.82}

 82%|████████▏ | 31070/37962 [71:57:12<15:43:15,  8.21s/it]
 82%|████████▏ | 31071/37962 [71:57:20<15:49:25,  8.27s/it]
 82%|████████▏ | 31072/37962 [71:57:28<15:52:11,  8.29s/it]
 82%|████████▏ | 31073/37962 [71:57:37<15:49:24,  8.27s/it]
 82%|████████▏ | 31074/37962 [71:57:45<15:52:12,  8.29s/it]
 82%|████████▏ | 31075/37962 [71:57:53<15:48:23,  8.26s/it]
 82%|████████▏ | 31076/37962 [71:58:02<15:49:44,  8.28s/it]
 82%|████████▏ | 31077/37962 [71:58:10<15:51:25,  8.29s/it]
 82%|████████▏ | 31078/37962 [71:58:18<15:47:40,  8.26s/it]
 82%|████████▏ | 31079/37962 [71:58:26<15:46:32,  8.25s/it]
 82%|████████▏ | 31080/37962 [71:58:35<15:51:44,  8.30s/it]
                                                           
{'loss': 1.1589, 'grad_norm': 10.924025535583496, 'learning_rate': 1.6747430346286576e-06, 'epoch': 0.82}

 82%|████████▏ | 31080/37962 [71:58:35<15:51:44,  8.30s/it]
 82%|████████▏ | 31081/37962 [71:58:42<15:35:44,  8.16s/it]
 82%|████████▏ | 31082/37962 [71:58:51<15:36:46,  8.17s/it]
 82%|████████▏ | 31083/37962 [71:58:59<15:46:28,  8.26s/it]
 82%|████████▏ | 31084/37962 [71:59:07<15:37:12,  8.18s/it]
 82%|████████▏ | 31085/37962 [71:59:15<15:29:15,  8.11s/it]
 82%|████████▏ | 31086/37962 [71:59:23<15:34:33,  8.15s/it]
 82%|████████▏ | 31087/37962 [71:59:32<15:42:46,  8.23s/it]
 82%|████████▏ | 31088/37962 [71:59:40<15:46:39,  8.26s/it]
 82%|████████▏ | 31089/37962 [71:59:48<15:50:40,  8.30s/it]
 82%|████████▏ | 31090/37962 [71:59:56<15:40:39,  8.21s/it]
                                                           
{'loss': 1.1754, 'grad_norm': 7.645543575286865, 'learning_rate': 1.6700196720957006e-06, 'epoch': 0.82}

 82%|████████▏ | 31090/37962 [71:59:56<15:40:39,  8.21s/it]
 82%|████████▏ | 31091/37962 [72:00:05<15:46:37,  8.27s/it]
 82%|████████▏ | 31092/37962 [72:00:13<15:45:25,  8.26s/it]
 82%|████████▏ | 31093/37962 [72:00:21<15:44:00,  8.25s/it]
 82%|████████▏ | 31094/37962 [72:00:30<15:41:56,  8.23s/it]
 82%|████████▏ | 31095/37962 [72:00:38<15:47:53,  8.28s/it]
 82%|████████▏ | 31096/37962 [72:00:46<15:45:05,  8.26s/it]
 82%|████████▏ | 31097/37962 [72:00:55<15:49:14,  8.30s/it]
 82%|████████▏ | 31098/37962 [72:01:03<15:49:50,  8.30s/it]
 82%|████████▏ | 31099/37962 [72:01:11<15:40:21,  8.22s/it]
 82%|████████▏ | 31100/37962 [72:01:19<15:40:53,  8.23s/it]
                                                           
{'loss': 1.1601, 'grad_norm': 6.837041854858398, 'learning_rate': 1.665302372811114e-06, 'epoch': 0.82}

 82%|████████▏ | 31100/37962 [72:01:19<15:40:53,  8.23s/it]
 82%|████████▏ | 31101/37962 [72:01:27<15:37:20,  8.20s/it]
 82%|████████▏ | 31102/37962 [72:01:35<15:27:25,  8.11s/it]
 82%|████████▏ | 31103/37962 [72:01:44<15:37:42,  8.20s/it]
 82%|████████▏ | 31104/37962 [72:01:52<15:42:22,  8.24s/it]
 82%|████████▏ | 31105/37962 [72:02:00<15:33:11,  8.17s/it]
 82%|████████▏ | 31106/37962 [72:02:08<15:37:26,  8.20s/it]
 82%|████████▏ | 31107/37962 [72:02:17<15:44:46,  8.27s/it]
 82%|████████▏ | 31108/37962 [72:02:25<15:45:07,  8.27s/it]
 82%|████████▏ | 31109/37962 [72:02:33<15:40:38,  8.24s/it]
 82%|████████▏ | 31110/37962 [72:02:41<15:40:26,  8.23s/it]
                                                           
{'loss': 1.0987, 'grad_norm': 9.495421409606934, 'learning_rate': 1.6605911402085407e-06, 'epoch': 0.82}

 82%|████████▏ | 31110/37962 [72:02:41<15:40:26,  8.23s/it]
 82%|████████▏ | 31111/37962 [72:02:49<15:37:52,  8.21s/it]
 82%|████████▏ | 31112/37962 [72:02:58<15:46:08,  8.29s/it]
 82%|████████▏ | 31113/37962 [72:03:06<15:43:49,  8.27s/it]
 82%|████████▏ | 31114/37962 [72:03:14<15:45:16,  8.28s/it]
 82%|████████▏ | 31115/37962 [72:03:23<15:41:17,  8.25s/it]
 82%|████████▏ | 31116/37962 [72:03:31<15:44:58,  8.28s/it]
 82%|████████▏ | 31117/37962 [72:03:39<15:38:41,  8.23s/it]
 82%|████████▏ | 31118/37962 [72:03:47<15:41:01,  8.25s/it]
 82%|████████▏ | 31119/37962 [72:03:56<15:42:36,  8.26s/it]
 82%|████████▏ | 31120/37962 [72:04:04<15:27:54,  8.14s/it]
                                                           
{'loss': 1.102, 'grad_norm': 6.673137187957764, 'learning_rate': 1.6558859777172088e-06, 'epoch': 0.82}

 82%|████████▏ | 31120/37962 [72:04:04<15:27:54,  8.14s/it]
 82%|████████▏ | 31121/37962 [72:04:12<15:31:05,  8.17s/it]
 82%|████████▏ | 31122/37962 [72:04:20<15:34:39,  8.20s/it]
 82%|████████▏ | 31123/37962 [72:04:28<15:39:43,  8.24s/it]
 82%|████████▏ | 31124/37962 [72:04:36<15:32:13,  8.18s/it]
 82%|████████▏ | 31125/37962 [72:04:45<15:37:21,  8.23s/it]
 82%|████████▏ | 31126/37962 [72:04:53<15:26:53,  8.14s/it]
 82%|████████▏ | 31127/37962 [72:05:01<15:28:48,  8.15s/it]
 82%|████████▏ | 31128/37962 [72:05:09<15:31:07,  8.17s/it]
 82%|████████▏ | 31129/37962 [72:05:17<15:35:33,  8.22s/it]
 82%|████████▏ | 31130/37962 [72:05:26<15:36:48,  8.23s/it]
                                                           
{'loss': 1.118, 'grad_norm': 11.590087890625, 'learning_rate': 1.6511868887619197e-06, 'epoch': 0.82}

 82%|████████▏ | 31130/37962 [72:05:26<15:36:48,  8.23s/it]
 82%|████████▏ | 31131/37962 [72:05:34<15:26:39,  8.14s/it]
 82%|████████▏ | 31132/37962 [72:05:42<15:27:49,  8.15s/it]
 82%|████████▏ | 31133/37962 [72:05:50<15:30:39,  8.18s/it]
 82%|████████▏ | 31134/37962 [72:05:58<15:22:44,  8.11s/it]
 82%|████████▏ | 31135/37962 [72:06:06<15:29:42,  8.17s/it]
 82%|████████▏ | 31136/37962 [72:06:14<15:31:46,  8.19s/it]
 82%|████████▏ | 31137/37962 [72:06:23<15:36:14,  8.23s/it]
 82%|████████▏ | 31138/37962 [72:06:31<15:38:51,  8.25s/it]
 82%|████████▏ | 31139/37962 [72:06:40<15:43:09,  8.29s/it]
 82%|████████▏ | 31140/37962 [72:06:48<15:44:01,  8.30s/it]
                                                           
{'loss': 1.1177, 'grad_norm': 9.012484550476074, 'learning_rate': 1.6464938767630634e-06, 'epoch': 0.82}

 82%|████████▏ | 31140/37962 [72:06:48<15:44:01,  8.30s/it]
 82%|████████▏ | 31141/37962 [72:06:56<15:41:23,  8.28s/it]
 82%|████████▏ | 31142/37962 [72:07:04<15:44:15,  8.31s/it]
 82%|████████▏ | 31143/37962 [72:07:13<15:43:46,  8.30s/it]
 82%|████████▏ | 31144/37962 [72:07:21<15:30:51,  8.19s/it]
 82%|████████▏ | 31145/37962 [72:07:29<15:32:39,  8.21s/it]
 82%|████████▏ | 31146/37962 [72:07:37<15:37:21,  8.25s/it]
 82%|████████▏ | 31147/37962 [72:07:46<15:38:34,  8.26s/it]
 82%|████████▏ | 31148/37962 [72:07:54<15:36:12,  8.24s/it]
 82%|████████▏ | 31149/37962 [72:08:02<15:34:49,  8.23s/it]
 82%|████████▏ | 31150/37962 [72:08:10<15:35:16,  8.24s/it]
                                                           
{'loss': 1.2369, 'grad_norm': 5.9406418800354, 'learning_rate': 1.6418069451365993e-06, 'epoch': 0.82}

 82%|████████▏ | 31150/37962 [72:08:10<15:35:16,  8.24s/it]
 82%|████████▏ | 31151/37962 [72:08:19<15:42:49,  8.31s/it]
 82%|████████▏ | 31152/37962 [72:08:27<15:44:33,  8.32s/it]
 82%|████████▏ | 31153/37962 [72:08:35<15:47:05,  8.35s/it]
 82%|████████▏ | 31154/37962 [72:08:43<15:37:38,  8.26s/it]
 82%|████████▏ | 31155/37962 [72:08:52<15:40:58,  8.29s/it]
 82%|████████▏ | 31156/37962 [72:09:00<15:41:15,  8.30s/it]
 82%|████████▏ | 31157/37962 [72:09:08<15:38:00,  8.27s/it]
 82%|████████▏ | 31158/37962 [72:09:16<15:29:55,  8.20s/it]
 82%|████████▏ | 31159/37962 [72:09:25<15:35:11,  8.25s/it]
 82%|████████▏ | 31160/37962 [72:09:33<15:44:08,  8.33s/it]
                                                           
{'loss': 1.1546, 'grad_norm': 6.872724533081055, 'learning_rate': 1.6371260972940618e-06, 'epoch': 0.82}

 82%|████████▏ | 31160/37962 [72:09:33<15:44:08,  8.33s/it]
 82%|████████▏ | 31161/37962 [72:09:41<15:35:53,  8.26s/it]
 82%|████████▏ | 31162/37962 [72:09:50<15:34:05,  8.24s/it]
 82%|████████▏ | 31163/37962 [72:09:58<15:36:57,  8.27s/it]
 82%|████████▏ | 31164/37962 [72:10:06<15:38:54,  8.29s/it]
 82%|████████▏ | 31165/37962 [72:10:15<15:41:14,  8.31s/it]
 82%|████████▏ | 31166/37962 [72:10:23<15:38:34,  8.29s/it]
 82%|████████▏ | 31167/37962 [72:10:31<15:41:15,  8.31s/it]
 82%|████████▏ | 31168/37962 [72:10:39<15:38:57,  8.29s/it]
 82%|████████▏ | 31169/37962 [72:10:48<15:39:43,  8.30s/it]
 82%|████████▏ | 31170/37962 [72:10:56<15:39:08,  8.30s/it]
                                                           
{'loss': 1.1608, 'grad_norm': 12.503777503967285, 'learning_rate': 1.6324513366425631e-06, 'epoch': 0.82}

 82%|████████▏ | 31170/37962 [72:10:56<15:39:08,  8.30s/it]
 82%|████████▏ | 31171/37962 [72:11:04<15:39:11,  8.30s/it]
 82%|████████▏ | 31172/37962 [72:11:13<15:47:05,  8.37s/it]
 82%|████████▏ | 31173/37962 [72:11:21<15:43:03,  8.33s/it]
 82%|████████▏ | 31174/37962 [72:11:30<15:44:14,  8.35s/it]
 82%|████████▏ | 31175/37962 [72:11:46<20:05:48, 10.66s/it]
 82%|████████▏ | 31176/37962 [72:11:54<18:41:24,  9.92s/it]
 82%|████████▏ | 31177/37962 [72:12:02<17:37:18,  9.35s/it]
 82%|████████▏ | 31178/37962 [72:12:10<17:00:32,  9.03s/it]
 82%|████████▏ | 31179/37962 [72:12:18<16:30:22,  8.76s/it]
 82%|████████▏ | 31180/37962 [72:12:26<16:12:04,  8.60s/it]
                                                           
{'loss': 1.1954, 'grad_norm': 9.994277954101562, 'learning_rate': 1.6277826665847762e-06, 'epoch': 0.82}

 82%|████████▏ | 31180/37962 [72:12:26<16:12:04,  8.60s/it]
 82%|████████▏ | 31181/37962 [72:12:34<15:48:57,  8.40s/it]
 82%|████████▏ | 31182/37962 [72:12:43<15:42:08,  8.34s/it]
 82%|████████▏ | 31183/37962 [72:12:51<15:33:23,  8.26s/it]
 82%|████████▏ | 31184/37962 [72:12:59<15:34:17,  8.27s/it]
 82%|████████▏ | 31185/37962 [72:13:07<15:35:31,  8.28s/it]
 82%|████████▏ | 31186/37962 [72:13:15<15:33:13,  8.26s/it]
 82%|████████▏ | 31187/37962 [72:13:24<15:33:31,  8.27s/it]
 82%|████████▏ | 31188/37962 [72:13:32<15:33:17,  8.27s/it]
 82%|████████▏ | 31189/37962 [72:13:40<15:32:54,  8.26s/it]
 82%|████████▏ | 31190/37962 [72:13:48<15:28:13,  8.22s/it]
                                                           
{'loss': 1.1555, 'grad_norm': 15.936970710754395, 'learning_rate': 1.623120090518948e-06, 'epoch': 0.82}

 82%|████████▏ | 31190/37962 [72:13:48<15:28:13,  8.22s/it]
 82%|████████▏ | 31191/37962 [72:13:57<15:30:04,  8.24s/it]
 82%|████████▏ | 31192/37962 [72:14:05<15:34:29,  8.28s/it]
 82%|████████▏ | 31193/37962 [72:14:13<15:34:03,  8.28s/it]
 82%|████████▏ | 31194/37962 [72:14:22<15:31:06,  8.25s/it]
 82%|████████▏ | 31195/37962 [72:14:30<15:29:22,  8.24s/it]
 82%|████████▏ | 31196/37962 [72:14:38<15:28:11,  8.23s/it]
 82%|████████▏ | 31197/37962 [72:14:46<15:23:33,  8.19s/it]
 82%|████████▏ | 31198/37962 [72:14:54<15:24:43,  8.20s/it]
 82%|████████▏ | 31199/37962 [72:15:02<15:18:13,  8.15s/it]
 82%|████████▏ | 31200/37962 [72:15:11<15:30:15,  8.25s/it]
                                                           
{'loss': 1.1344, 'grad_norm': 13.967419624328613, 'learning_rate': 1.618463611838892e-06, 'epoch': 0.82}

 82%|████████▏ | 31200/37962 [72:15:11<15:30:15,  8.25s/it]
 82%|████████▏ | 31201/37962 [72:15:19<15:17:55,  8.15s/it]
 82%|████████▏ | 31202/37962 [72:15:27<15:27:32,  8.23s/it]
 82%|████████▏ | 31203/37962 [72:15:36<15:34:56,  8.30s/it]
 82%|████████▏ | 31204/37962 [72:15:44<15:27:09,  8.23s/it]
 82%|████████▏ | 31205/37962 [72:15:52<15:27:36,  8.24s/it]
 82%|████████▏ | 31206/37962 [72:16:00<15:31:12,  8.27s/it]
 82%|████████▏ | 31207/37962 [72:16:08<15:30:02,  8.26s/it]
 82%|████████▏ | 31208/37962 [72:16:17<15:28:00,  8.24s/it]
 82%|████████▏ | 31209/37962 [72:16:25<15:34:02,  8.30s/it]
 82%|████████▏ | 31210/37962 [72:16:33<15:12:13,  8.11s/it]
                                                           
{'loss': 1.2307, 'grad_norm': 6.840645790100098, 'learning_rate': 1.6138132339339662e-06, 'epoch': 0.82}

 82%|████████▏ | 31210/37962 [72:16:33<15:12:13,  8.11s/it]
 82%|████████▏ | 31211/37962 [72:16:41<15:16:34,  8.15s/it]
 82%|████████▏ | 31212/37962 [72:16:49<15:14:14,  8.13s/it]
 82%|████████▏ | 31213/37962 [72:16:58<15:24:58,  8.22s/it]
 82%|████████▏ | 31214/37962 [72:17:06<15:21:37,  8.19s/it]
 82%|████████▏ | 31215/37962 [72:17:14<15:22:24,  8.20s/it]
 82%|████████▏ | 31216/37962 [72:17:22<15:25:23,  8.23s/it]
 82%|████████▏ | 31217/37962 [72:17:31<15:31:41,  8.29s/it]
 82%|████████▏ | 31218/37962 [72:17:39<15:33:43,  8.31s/it]
 82%|████████▏ | 31219/37962 [72:17:47<15:26:57,  8.25s/it]
 82%|████████▏ | 31220/37962 [72:17:55<15:30:43,  8.28s/it]
                                                           
{'loss': 1.1939, 'grad_norm': 11.96902084350586, 'learning_rate': 1.609168960189108e-06, 'epoch': 0.82}

 82%|████████▏ | 31220/37962 [72:17:55<15:30:43,  8.28s/it]
 82%|████████▏ | 31221/37962 [72:18:04<15:31:23,  8.29s/it]
 82%|████████▏ | 31222/37962 [72:18:12<15:29:25,  8.27s/it]
 82%|████████▏ | 31223/37962 [72:18:20<15:29:03,  8.27s/it]
 82%|████████▏ | 31224/37962 [72:18:28<15:25:27,  8.24s/it]
 82%|████████▏ | 31225/37962 [72:18:37<15:24:17,  8.23s/it]
 82%|████████▏ | 31226/37962 [72:18:45<15:28:33,  8.27s/it]
 82%|████████▏ | 31227/37962 [72:18:53<15:25:31,  8.25s/it]
 82%|████████▏ | 31228/37962 [72:19:01<15:24:36,  8.24s/it]
 82%|████████▏ | 31229/37962 [72:19:10<15:22:54,  8.22s/it]
 82%|████████▏ | 31230/37962 [72:19:18<15:25:19,  8.25s/it]
                                                           
{'loss': 1.2028, 'grad_norm': 5.833017349243164, 'learning_rate': 1.6045307939848042e-06, 'epoch': 0.82}

 82%|████████▏ | 31230/37962 [72:19:18<15:25:19,  8.25s/it]
 82%|████████▏ | 31231/37962 [72:19:26<15:25:58,  8.25s/it]
 82%|████████▏ | 31232/37962 [72:19:34<15:26:45,  8.26s/it]
 82%|████████▏ | 31233/37962 [72:19:43<15:21:37,  8.22s/it]
 82%|████████▏ | 31234/37962 [72:19:50<15:11:19,  8.13s/it]
 82%|████████▏ | 31235/37962 [72:19:59<15:14:32,  8.16s/it]
 82%|████████▏ | 31236/37962 [72:20:07<15:16:28,  8.18s/it]
 82%|████████▏ | 31237/37962 [72:20:15<15:09:06,  8.11s/it]
 82%|████████▏ | 31238/37962 [72:20:23<15:06:13,  8.09s/it]
 82%|████████▏ | 31239/37962 [72:20:31<15:18:22,  8.20s/it]
 82%|████████▏ | 31240/37962 [72:20:40<15:24:43,  8.25s/it]
                                                           
{'loss': 1.2014, 'grad_norm': 8.82912826538086, 'learning_rate': 1.599898738697092e-06, 'epoch': 0.82}

 82%|████████▏ | 31240/37962 [72:20:40<15:24:43,  8.25s/it]
 82%|████████▏ | 31241/37962 [72:20:48<15:28:33,  8.29s/it]
 82%|████████▏ | 31242/37962 [72:20:56<15:26:23,  8.27s/it]
 82%|████████▏ | 31243/37962 [72:21:05<15:29:35,  8.30s/it]
 82%|████████▏ | 31244/37962 [72:21:13<15:27:33,  8.28s/it]
 82%|████████▏ | 31245/37962 [72:21:21<15:27:10,  8.28s/it]
 82%|████████▏ | 31246/37962 [72:21:30<15:28:39,  8.30s/it]
 82%|████████▏ | 31247/37962 [72:21:38<15:31:40,  8.32s/it]
 82%|████████▏ | 31248/37962 [72:21:46<15:28:44,  8.30s/it]
 82%|████████▏ | 31249/37962 [72:21:54<15:24:54,  8.27s/it]
 82%|████████▏ | 31250/37962 [72:22:03<15:24:48,  8.27s/it]
                                                           
{'loss': 1.1353, 'grad_norm': 7.119807720184326, 'learning_rate': 1.595272797697569e-06, 'epoch': 0.82}

 82%|████████▏ | 31250/37962 [72:22:03<15:24:48,  8.27s/it]
 82%|████████▏ | 31251/37962 [72:22:11<15:28:21,  8.30s/it]
 82%|████████▏ | 31252/37962 [72:22:19<15:14:03,  8.17s/it]
 82%|████████▏ | 31253/37962 [72:22:27<15:19:39,  8.22s/it]
 82%|████████▏ | 31254/37962 [72:22:36<15:20:46,  8.24s/it]
 82%|████████▏ | 31255/37962 [72:22:44<15:25:36,  8.28s/it]
 82%|████████▏ | 31256/37962 [72:22:52<15:25:36,  8.28s/it]
 82%|████████▏ | 31257/37962 [72:23:00<15:26:43,  8.29s/it]
 82%|████████▏ | 31258/37962 [72:23:09<15:24:14,  8.27s/it]
 82%|████████▏ | 31259/37962 [72:23:17<15:24:23,  8.27s/it]
 82%|████████▏ | 31260/37962 [72:23:25<15:20:32,  8.24s/it]
                                                           
{'loss': 1.2312, 'grad_norm': 10.981313705444336, 'learning_rate': 1.5906529743533738e-06, 'epoch': 0.82}

 82%|████████▏ | 31260/37962 [72:23:25<15:20:32,  8.24s/it]
 82%|████████▏ | 31261/37962 [72:23:33<15:17:31,  8.22s/it]
 82%|████████▏ | 31262/37962 [72:23:41<15:02:48,  8.08s/it]
 82%|████████▏ | 31263/37962 [72:23:49<15:09:02,  8.14s/it]
 82%|████████▏ | 31264/37962 [72:23:58<15:10:03,  8.15s/it]
 82%|████████▏ | 31265/37962 [72:24:06<15:14:44,  8.20s/it]
 82%|████████▏ | 31266/37962 [72:24:14<15:17:56,  8.23s/it]
 82%|████████▏ | 31267/37962 [72:24:22<15:19:32,  8.24s/it]
 82%|████████▏ | 31268/37962 [72:24:30<15:09:01,  8.15s/it]
 82%|████████▏ | 31269/37962 [72:24:39<15:12:28,  8.18s/it]
 82%|████████▏ | 31270/37962 [72:24:47<15:15:14,  8.21s/it]
                                                           
{'loss': 1.0894, 'grad_norm': 7.623471736907959, 'learning_rate': 1.5860392720271945e-06, 'epoch': 0.82}

 82%|████████▏ | 31270/37962 [72:24:47<15:15:14,  8.21s/it]
 82%|████████▏ | 31271/37962 [72:24:55<15:01:52,  8.09s/it]
 82%|████████▏ | 31272/37962 [72:25:03<15:07:46,  8.14s/it]
 82%|████████▏ | 31273/37962 [72:25:11<15:16:22,  8.22s/it]
 82%|████████▏ | 31274/37962 [72:25:20<15:14:13,  8.20s/it]
 82%|████████▏ | 31275/37962 [72:25:28<15:21:28,  8.27s/it]
 82%|████████▏ | 31276/37962 [72:25:36<15:15:24,  8.21s/it]
 82%|████████▏ | 31277/37962 [72:25:44<15:17:25,  8.23s/it]
 82%|████████▏ | 31278/37962 [72:25:53<15:18:07,  8.24s/it]
 82%|████████▏ | 31279/37962 [72:26:01<15:22:04,  8.28s/it]
 82%|████████▏ | 31280/37962 [72:26:09<15:24:53,  8.30s/it]
                                                           
{'loss': 1.2434, 'grad_norm': 13.504652976989746, 'learning_rate': 1.5814316940772668e-06, 'epoch': 0.82}

 82%|████████▏ | 31280/37962 [72:26:09<15:24:53,  8.30s/it]
 82%|████████▏ | 31281/37962 [72:26:18<15:28:36,  8.34s/it]
 82%|████████▏ | 31282/37962 [72:26:26<15:29:34,  8.35s/it]
 82%|████████▏ | 31283/37962 [72:26:34<15:22:09,  8.28s/it]
 82%|████████▏ | 31284/37962 [72:26:43<15:26:51,  8.33s/it]
 82%|████████▏ | 31285/37962 [72:26:51<15:28:06,  8.34s/it]
 82%|████████▏ | 31286/37962 [72:26:59<15:24:01,  8.30s/it]
 82%|████████▏ | 31287/37962 [72:27:08<15:22:36,  8.29s/it]
 82%|████████▏ | 31288/37962 [72:27:16<15:25:23,  8.32s/it]
 82%|████████▏ | 31289/37962 [72:27:24<15:22:21,  8.29s/it]
 82%|████████▏ | 31290/37962 [72:27:32<15:24:49,  8.32s/it]
                                                           
{'loss': 1.0537, 'grad_norm': 7.754531383514404, 'learning_rate': 1.5768302438573701e-06, 'epoch': 0.82}

 82%|████████▏ | 31290/37962 [72:27:32<15:24:49,  8.32s/it]
 82%|████████▏ | 31291/37962 [72:27:41<15:29:03,  8.36s/it]
 82%|████████▏ | 31292/37962 [72:27:50<15:40:04,  8.46s/it]
 82%|████████▏ | 31293/37962 [72:27:58<15:39:25,  8.45s/it]
 82%|████████▏ | 31294/37962 [72:28:07<15:39:45,  8.46s/it]
 82%|████████▏ | 31295/37962 [72:28:15<15:35:25,  8.42s/it]
 82%|████████▏ | 31296/37962 [72:28:23<15:29:22,  8.37s/it]
 82%|████████▏ | 31297/37962 [72:28:31<15:30:08,  8.37s/it]
 82%|████████▏ | 31298/37962 [72:28:40<15:26:00,  8.34s/it]
 82%|████████▏ | 31299/37962 [72:28:48<15:22:13,  8.30s/it]
 82%|████████▏ | 31300/37962 [72:28:56<15:19:43,  8.28s/it]
                                                           
{'loss': 1.1731, 'grad_norm': 9.678184509277344, 'learning_rate': 1.5722349247168167e-06, 'epoch': 0.82}

 82%|████████▏ | 31300/37962 [72:28:56<15:19:43,  8.28s/it]
 82%|████████▏ | 31301/37962 [72:29:04<15:19:38,  8.28s/it]
 82%|████████▏ | 31302/37962 [72:29:21<19:38:18, 10.62s/it]
 82%|████████▏ | 31303/37962 [72:29:29<18:14:07,  9.86s/it]
 82%|████████▏ | 31304/37962 [72:29:37<17:23:44,  9.41s/it]
 82%|████████▏ | 31305/37962 [72:29:45<16:49:29,  9.10s/it]
 82%|████████▏ | 31306/37962 [72:29:53<16:12:58,  8.77s/it]
 82%|████████▏ | 31307/37962 [72:30:02<15:55:28,  8.61s/it]
 82%|████████▏ | 31308/37962 [72:30:10<15:36:46,  8.45s/it]
 82%|████████▏ | 31309/37962 [72:30:18<15:34:08,  8.42s/it]
 82%|████████▏ | 31310/37962 [72:30:26<15:29:13,  8.38s/it]
                                                           
{'loss': 1.1228, 'grad_norm': 9.057154655456543, 'learning_rate': 1.5676457400004597e-06, 'epoch': 0.82}

 82%|████████▏ | 31310/37962 [72:30:26<15:29:13,  8.38s/it]
 82%|████████▏ | 31311/37962 [72:30:35<15:40:01,  8.48s/it]
 82%|████████▏ | 31312/37962 [72:30:43<15:29:35,  8.39s/it]
 82%|████████▏ | 31313/37962 [72:30:52<15:28:55,  8.38s/it]
 82%|████████▏ | 31314/37962 [72:31:00<15:28:12,  8.38s/it]
 82%|████████▏ | 31315/37962 [72:31:08<15:18:06,  8.29s/it]
 82%|████████▏ | 31316/37962 [72:31:16<15:18:48,  8.29s/it]
 82%|████████▏ | 31317/37962 [72:31:25<15:15:11,  8.26s/it]
 82%|████████▏ | 31318/37962 [72:31:33<15:18:25,  8.29s/it]
 83%|████████▎ | 31319/37962 [72:31:41<15:16:33,  8.28s/it]
 83%|████████▎ | 31320/37962 [72:31:49<15:16:30,  8.28s/it]
                                                           
{'loss': 1.1123, 'grad_norm': 5.831321716308594, 'learning_rate': 1.5630626930486891e-06, 'epoch': 0.83}

 83%|████████▎ | 31320/37962 [72:31:49<15:16:30,  8.28s/it]
 83%|████████▎ | 31321/37962 [72:31:58<15:20:53,  8.32s/it]
 83%|████████▎ | 31322/37962 [72:32:06<15:21:41,  8.33s/it]
 83%|████████▎ | 31323/37962 [72:32:15<15:24:02,  8.35s/it]
 83%|████████▎ | 31324/37962 [72:32:23<15:12:28,  8.25s/it]
 83%|████████▎ | 31325/37962 [72:32:31<15:09:44,  8.22s/it]
 83%|████████▎ | 31326/37962 [72:32:39<15:16:05,  8.28s/it]
 83%|████████▎ | 31327/37962 [72:32:48<15:18:09,  8.30s/it]
 83%|████████▎ | 31328/37962 [72:32:56<15:19:35,  8.32s/it]
 83%|████████▎ | 31329/37962 [72:33:04<15:19:53,  8.32s/it]
 83%|████████▎ | 31330/37962 [72:33:13<15:19:15,  8.32s/it]
                                                           
{'loss': 1.2498, 'grad_norm': 10.18830394744873, 'learning_rate': 1.5584857871974224e-06, 'epoch': 0.83}

 83%|████████▎ | 31330/37962 [72:33:13<15:19:15,  8.32s/it]
 83%|████████▎ | 31331/37962 [72:33:21<15:12:21,  8.26s/it]
 83%|████████▎ | 31332/37962 [72:33:29<15:10:11,  8.24s/it]
 83%|████████▎ | 31333/37962 [72:33:37<15:12:05,  8.26s/it]
 83%|████████▎ | 31334/37962 [72:33:45<15:05:48,  8.20s/it]
 83%|████████▎ | 31335/37962 [72:33:53<15:07:19,  8.21s/it]
 83%|████████▎ | 31336/37962 [72:34:02<15:14:17,  8.28s/it]
 83%|████████▎ | 31337/37962 [72:34:10<15:11:49,  8.26s/it]
 83%|████████▎ | 31338/37962 [72:34:18<15:10:23,  8.25s/it]
 83%|████████▎ | 31339/37962 [72:34:27<15:13:23,  8.27s/it]
 83%|████████▎ | 31340/37962 [72:34:35<15:19:14,  8.33s/it]
                                                           
{'loss': 1.1883, 'grad_norm': 14.060593605041504, 'learning_rate': 1.5539150257781144e-06, 'epoch': 0.83}

 83%|████████▎ | 31340/37962 [72:34:35<15:19:14,  8.33s/it]
 83%|████████▎ | 31341/37962 [72:34:43<15:16:24,  8.30s/it]
 83%|████████▎ | 31342/37962 [72:34:52<15:18:11,  8.32s/it]
 83%|████████▎ | 31343/37962 [72:35:00<15:18:37,  8.33s/it]
 83%|████████▎ | 31344/37962 [72:35:08<15:16:29,  8.31s/it]
 83%|████████▎ | 31345/37962 [72:35:17<15:14:26,  8.29s/it]
 83%|████████▎ | 31346/37962 [72:35:25<15:11:57,  8.27s/it]
 83%|████████▎ | 31347/37962 [72:35:33<15:16:12,  8.31s/it]
 83%|████████▎ | 31348/37962 [72:35:42<15:16:06,  8.31s/it]
 83%|████████▎ | 31349/37962 [72:35:50<15:16:40,  8.32s/it]
 83%|████████▎ | 31350/37962 [72:35:58<15:17:49,  8.33s/it]
                                                           
{'loss': 1.1684, 'grad_norm': 13.48275375366211, 'learning_rate': 1.5493504121177383e-06, 'epoch': 0.83}

 83%|████████▎ | 31350/37962 [72:35:58<15:17:49,  8.33s/it]
 83%|████████▎ | 31351/37962 [72:36:06<15:16:28,  8.32s/it]
 83%|████████▎ | 31352/37962 [72:36:15<15:13:53,  8.30s/it]
 83%|████████▎ | 31353/37962 [72:36:23<15:21:07,  8.36s/it]
 83%|████████▎ | 31354/37962 [72:36:31<15:11:32,  8.28s/it]
 83%|████████▎ | 31355/37962 [72:36:40<15:11:17,  8.28s/it]
 83%|████████▎ | 31356/37962 [72:36:48<15:08:10,  8.25s/it]
 83%|████████▎ | 31357/37962 [72:36:56<15:08:12,  8.25s/it]
 83%|████████▎ | 31358/37962 [72:37:04<15:08:17,  8.25s/it]
 83%|████████▎ | 31359/37962 [72:37:12<15:02:14,  8.20s/it]
 83%|████████▎ | 31360/37962 [72:37:21<15:07:27,  8.25s/it]
                                                           
{'loss': 1.0894, 'grad_norm': 12.0016450881958, 'learning_rate': 1.544791949538802e-06, 'epoch': 0.83}

 83%|████████▎ | 31360/37962 [72:37:21<15:07:27,  8.25s/it]
 83%|████████▎ | 31361/37962 [72:37:29<15:06:15,  8.24s/it]
 83%|████████▎ | 31362/37962 [72:37:37<15:08:52,  8.26s/it]
 83%|████████▎ | 31363/37962 [72:37:46<15:09:07,  8.27s/it]
 83%|████████▎ | 31364/37962 [72:37:54<15:09:08,  8.27s/it]
 83%|████████▎ | 31365/37962 [72:38:02<15:11:06,  8.29s/it]
 83%|████████▎ | 31366/37962 [72:38:10<15:10:59,  8.29s/it]
 83%|████████▎ | 31367/37962 [72:38:18<15:00:54,  8.20s/it]
 83%|████████▎ | 31368/37962 [72:38:27<15:14:11,  8.32s/it]
 83%|████████▎ | 31369/37962 [72:38:35<15:11:13,  8.29s/it]
 83%|████████▎ | 31370/37962 [72:38:44<15:12:47,  8.31s/it]
                                                           
{'loss': 1.073, 'grad_norm': 10.265583038330078, 'learning_rate': 1.540239641359329e-06, 'epoch': 0.83}

 83%|████████▎ | 31370/37962 [72:38:44<15:12:47,  8.31s/it]
 83%|████████▎ | 31371/37962 [72:38:52<15:12:54,  8.31s/it]
 83%|████████▎ | 31372/37962 [72:39:00<15:16:39,  8.35s/it]
 83%|████████▎ | 31373/37962 [72:39:09<15:18:16,  8.36s/it]
 83%|████████▎ | 31374/37962 [72:39:17<15:04:26,  8.24s/it]
 83%|████████▎ | 31375/37962 [72:39:25<15:02:21,  8.22s/it]
 83%|████████▎ | 31376/37962 [72:39:33<15:11:12,  8.30s/it]
 83%|████████▎ | 31377/37962 [72:39:42<15:10:44,  8.30s/it]
 83%|████████▎ | 31378/37962 [72:39:50<15:10:59,  8.30s/it]
 83%|████████▎ | 31379/37962 [72:39:58<15:08:30,  8.28s/it]
 83%|████████▎ | 31380/37962 [72:40:07<15:13:32,  8.33s/it]
                                                           
{'loss': 1.1769, 'grad_norm': 8.88713550567627, 'learning_rate': 1.5356934908928634e-06, 'epoch': 0.83}

 83%|████████▎ | 31380/37962 [72:40:07<15:13:32,  8.33s/it]
 83%|████████▎ | 31381/37962 [72:40:15<15:01:42,  8.22s/it]
 83%|████████▎ | 31382/37962 [72:40:23<14:53:36,  8.15s/it]
 83%|████████▎ | 31383/37962 [72:40:30<14:45:17,  8.07s/it]
 83%|████████▎ | 31384/37962 [72:40:39<14:53:07,  8.15s/it]
 83%|████████▎ | 31385/37962 [72:40:47<14:58:58,  8.20s/it]
 83%|████████▎ | 31386/37962 [72:40:56<15:04:58,  8.26s/it]
 83%|████████▎ | 31387/37962 [72:41:04<15:10:47,  8.31s/it]
 83%|████████▎ | 31388/37962 [72:41:12<15:08:21,  8.29s/it]
 83%|████████▎ | 31389/37962 [72:41:20<15:06:07,  8.27s/it]
 83%|████████▎ | 31390/37962 [72:41:28<14:57:17,  8.19s/it]
                                                           
{'loss': 1.092, 'grad_norm': 8.522123336791992, 'learning_rate': 1.5311535014484724e-06, 'epoch': 0.83}

 83%|████████▎ | 31390/37962 [72:41:28<14:57:17,  8.19s/it]
 83%|████████▎ | 31391/37962 [72:41:37<14:58:46,  8.21s/it]
 83%|████████▎ | 31392/37962 [72:41:45<15:07:20,  8.29s/it]
 83%|████████▎ | 31393/37962 [72:41:53<15:06:30,  8.28s/it]
 83%|████████▎ | 31394/37962 [72:42:02<15:07:27,  8.29s/it]
 83%|████████▎ | 31395/37962 [72:42:10<15:09:46,  8.31s/it]
 83%|████████▎ | 31396/37962 [72:42:18<15:00:19,  8.23s/it]
 83%|████████▎ | 31397/37962 [72:42:26<14:59:30,  8.22s/it]
 83%|████████▎ | 31398/37962 [72:42:35<15:05:18,  8.28s/it]
 83%|████████▎ | 31399/37962 [72:42:43<15:15:16,  8.37s/it]
 83%|████████▎ | 31400/37962 [72:42:51<15:09:08,  8.31s/it]
                                                           
{'loss': 1.256, 'grad_norm': 8.581297874450684, 'learning_rate': 1.5266196763307362e-06, 'epoch': 0.83}

 83%|████████▎ | 31400/37962 [72:42:51<15:09:08,  8.31s/it]
 83%|████████▎ | 31401/37962 [72:43:00<15:17:39,  8.39s/it]
 83%|████████▎ | 31402/37962 [72:43:09<15:19:20,  8.41s/it]
 83%|████████▎ | 31403/37962 [72:43:17<15:18:16,  8.40s/it]
 83%|████████▎ | 31404/37962 [72:43:25<15:13:56,  8.36s/it]
 83%|████████▎ | 31405/37962 [72:43:33<15:12:54,  8.35s/it]
 83%|████████▎ | 31406/37962 [72:43:42<15:09:13,  8.32s/it]
 83%|████████▎ | 31407/37962 [72:43:50<15:09:00,  8.32s/it]
 83%|████████▎ | 31408/37962 [72:43:59<15:16:01,  8.39s/it]
 83%|████████▎ | 31409/37962 [72:44:07<15:13:59,  8.37s/it]
 83%|████████▎ | 31410/37962 [72:44:15<15:10:12,  8.34s/it]
                                                           
{'loss': 1.2262, 'grad_norm': 9.425699234008789, 'learning_rate': 1.522092018839747e-06, 'epoch': 0.83}

 83%|████████▎ | 31410/37962 [72:44:15<15:10:12,  8.34s/it]
 83%|████████▎ | 31411/37962 [72:44:23<15:05:41,  8.30s/it]
 83%|████████▎ | 31412/37962 [72:44:32<15:07:20,  8.31s/it]
 83%|████████▎ | 31413/37962 [72:44:40<15:01:44,  8.26s/it]
 83%|████████▎ | 31414/37962 [72:44:48<15:01:20,  8.26s/it]
 83%|████████▎ | 31415/37962 [72:44:56<15:01:51,  8.27s/it]
 83%|████████▎ | 31416/37962 [72:45:05<15:00:55,  8.26s/it]
 83%|████████▎ | 31417/37962 [72:45:13<14:58:26,  8.24s/it]
 83%|████████▎ | 31418/37962 [72:45:21<14:52:15,  8.18s/it]
 83%|████████▎ | 31419/37962 [72:45:29<14:55:16,  8.21s/it]
 83%|████████▎ | 31420/37962 [72:45:37<14:58:28,  8.24s/it]
                                                           
{'loss': 1.149, 'grad_norm': 6.643393516540527, 'learning_rate': 1.5175705322711053e-06, 'epoch': 0.83}

 83%|████████▎ | 31420/37962 [72:45:37<14:58:28,  8.24s/it]
 83%|████████▎ | 31421/37962 [72:45:46<15:03:35,  8.29s/it]
 83%|████████▎ | 31422/37962 [72:45:54<14:50:33,  8.17s/it]
 83%|████████▎ | 31423/37962 [72:46:02<14:45:24,  8.12s/it]
 83%|████████▎ | 31424/37962 [72:46:10<14:44:58,  8.12s/it]
 83%|████████▎ | 31425/37962 [72:46:18<14:51:40,  8.18s/it]
 83%|████████▎ | 31426/37962 [72:46:26<14:53:10,  8.20s/it]
 83%|████████▎ | 31427/37962 [72:46:35<14:49:33,  8.17s/it]
 83%|████████▎ | 31428/37962 [72:46:43<14:42:32,  8.10s/it]
 83%|████████▎ | 31429/37962 [72:46:51<14:54:20,  8.21s/it]
 83%|████████▎ | 31430/37962 [72:46:59<14:55:27,  8.23s/it]
                                                           
{'loss': 1.1955, 'grad_norm': 9.888055801391602, 'learning_rate': 1.5130552199159277e-06, 'epoch': 0.83}

 83%|████████▎ | 31430/37962 [72:46:59<14:55:27,  8.23s/it]
 83%|████████▎ | 31431/37962 [72:47:07<14:55:45,  8.23s/it]
 83%|████████▎ | 31432/37962 [72:47:16<14:59:12,  8.26s/it]
 83%|████████▎ | 31433/37962 [72:47:24<15:05:26,  8.32s/it]
 83%|████████▎ | 31434/37962 [72:47:32<15:00:45,  8.28s/it]
 83%|████████▎ | 31435/37962 [72:47:41<15:00:48,  8.28s/it]
 83%|████████▎ | 31436/37962 [72:47:49<14:54:24,  8.22s/it]
 83%|████████▎ | 31437/37962 [72:47:57<14:59:33,  8.27s/it]
 83%|████████▎ | 31438/37962 [72:48:05<14:58:58,  8.27s/it]
 83%|████████▎ | 31439/37962 [72:48:14<15:02:20,  8.30s/it]
 83%|████████▎ | 31440/37962 [72:48:22<15:03:41,  8.31s/it]
                                                           
{'loss': 1.1433, 'grad_norm': 9.682290077209473, 'learning_rate': 1.508546085060827e-06, 'epoch': 0.83}

 83%|████████▎ | 31440/37962 [72:48:22<15:03:41,  8.31s/it]
 83%|████████▎ | 31441/37962 [72:48:31<15:03:41,  8.31s/it]
 83%|████████▎ | 31442/37962 [72:48:39<15:02:15,  8.30s/it]
 83%|████████▎ | 31443/37962 [72:48:47<14:55:03,  8.24s/it]
 83%|████████▎ | 31444/37962 [72:48:55<14:56:41,  8.25s/it]
 83%|████████▎ | 31445/37962 [72:49:03<14:50:23,  8.20s/it]
 83%|████████▎ | 31446/37962 [72:49:12<14:52:48,  8.22s/it]
 83%|████████▎ | 31447/37962 [72:49:20<14:55:02,  8.24s/it]
 83%|████████▎ | 31448/37962 [72:49:28<14:54:07,  8.24s/it]
 83%|████████▎ | 31449/37962 [72:49:37<15:02:41,  8.32s/it]
 83%|████████▎ | 31450/37962 [72:49:45<14:59:37,  8.29s/it]
                                                           
{'loss': 1.2235, 'grad_norm': 8.159740447998047, 'learning_rate': 1.504043130987929e-06, 'epoch': 0.83}

 83%|████████▎ | 31450/37962 [72:49:45<14:59:37,  8.29s/it]
 83%|████████▎ | 31451/37962 [72:49:53<14:45:24,  8.16s/it]
 83%|████████▎ | 31452/37962 [72:50:01<14:47:49,  8.18s/it]
 83%|████████▎ | 31453/37962 [72:50:09<14:51:00,  8.21s/it]
 83%|████████▎ | 31454/37962 [72:50:17<14:37:24,  8.09s/it]
 83%|████████▎ | 31455/37962 [72:50:25<14:41:13,  8.13s/it]
 83%|████████▎ | 31456/37962 [72:50:33<14:47:38,  8.19s/it]
 83%|████████▎ | 31457/37962 [72:50:42<14:53:14,  8.24s/it]
 83%|████████▎ | 31458/37962 [72:50:50<14:52:54,  8.24s/it]
 83%|████████▎ | 31459/37962 [72:50:58<14:52:48,  8.24s/it]
 83%|████████▎ | 31460/37962 [72:51:07<14:59:40,  8.30s/it]
                                                           
{'loss': 1.1531, 'grad_norm': 7.289303302764893, 'learning_rate': 1.4995463609748528e-06, 'epoch': 0.83}

 83%|████████▎ | 31460/37962 [72:51:07<14:59:40,  8.30s/it]
 83%|████████▎ | 31461/37962 [72:51:15<14:59:22,  8.30s/it]
 83%|████████▎ | 31462/37962 [72:51:23<15:01:57,  8.33s/it]
 83%|████████▎ | 31463/37962 [72:51:32<14:56:01,  8.27s/it]
 83%|████████▎ | 31464/37962 [72:51:40<15:01:21,  8.32s/it]
 83%|████████▎ | 31465/37962 [72:51:48<14:58:02,  8.29s/it]
 83%|████████▎ | 31466/37962 [72:51:57<15:03:21,  8.34s/it]
 83%|████████▎ | 31467/37962 [72:52:05<15:07:29,  8.38s/it]
 83%|████████▎ | 31468/37962 [72:52:13<15:02:24,  8.34s/it]
 83%|████████▎ | 31469/37962 [72:52:22<14:57:15,  8.29s/it]
 83%|████████▎ | 31470/37962 [72:52:30<14:57:56,  8.30s/it]
                                                           
{'loss': 1.1545, 'grad_norm': 8.978445053100586, 'learning_rate': 1.4950557782947183e-06, 'epoch': 0.83}

 83%|████████▎ | 31470/37962 [72:52:30<14:57:56,  8.30s/it]
 83%|████████▎ | 31471/37962 [72:52:38<14:48:49,  8.22s/it]
 83%|████████▎ | 31472/37962 [72:52:46<14:51:54,  8.25s/it]
 83%|████████▎ | 31473/37962 [72:52:54<14:49:57,  8.23s/it]
 83%|████████▎ | 31474/37962 [72:53:03<14:52:00,  8.25s/it]
 83%|████████▎ | 31475/37962 [72:53:11<14:42:29,  8.16s/it]
 83%|████████▎ | 31476/37962 [72:53:19<14:38:38,  8.13s/it]
 83%|████████▎ | 31477/37962 [72:53:27<14:46:35,  8.20s/it]
 83%|████████▎ | 31478/37962 [72:53:35<14:40:36,  8.15s/it]
 83%|████████▎ | 31479/37962 [72:53:44<14:48:23,  8.22s/it]
 83%|████████▎ | 31480/37962 [72:53:52<14:48:53,  8.23s/it]
                                                           
{'loss': 1.0461, 'grad_norm': 6.11592960357666, 'learning_rate': 1.4905713862161464e-06, 'epoch': 0.83}

 83%|████████▎ | 31480/37962 [72:53:52<14:48:53,  8.23s/it]
 83%|████████▎ | 31481/37962 [72:54:00<14:50:11,  8.24s/it]
 83%|████████▎ | 31482/37962 [72:54:08<14:54:03,  8.28s/it]
 83%|████████▎ | 31483/37962 [72:54:17<14:56:59,  8.31s/it]
 83%|████████▎ | 31484/37962 [72:54:25<14:55:50,  8.30s/it]
 83%|████████▎ | 31485/37962 [72:54:33<14:44:26,  8.19s/it]
 83%|████████▎ | 31486/37962 [72:54:41<14:42:39,  8.18s/it]
 83%|████████▎ | 31487/37962 [72:54:50<14:51:27,  8.26s/it]
 83%|████████▎ | 31488/37962 [72:54:58<14:48:59,  8.24s/it]
 83%|████████▎ | 31489/37962 [72:55:06<14:49:13,  8.24s/it]
 83%|████████▎ | 31490/37962 [72:55:14<14:39:19,  8.15s/it]
                                                           
{'loss': 1.0722, 'grad_norm': 18.631071090698242, 'learning_rate': 1.4860931880032447e-06, 'epoch': 0.83}

 83%|████████▎ | 31490/37962 [72:55:14<14:39:19,  8.15s/it]
 83%|████████▎ | 31491/37962 [72:55:22<14:45:01,  8.21s/it]
 83%|████████▎ | 31492/37962 [72:55:30<14:34:31,  8.11s/it]
 83%|████████▎ | 31493/37962 [72:55:38<14:39:07,  8.15s/it]
 83%|████████▎ | 31494/37962 [72:55:46<14:31:56,  8.09s/it]
 83%|████████▎ | 31495/37962 [72:55:55<14:38:16,  8.15s/it]
 83%|████████▎ | 31496/37962 [72:56:03<14:41:55,  8.18s/it]
 83%|████████▎ | 31497/37962 [72:56:11<14:47:24,  8.24s/it]
 83%|████████▎ | 31498/37962 [72:56:19<14:36:38,  8.14s/it]
 83%|████████▎ | 31499/37962 [72:56:27<14:37:58,  8.15s/it]
 83%|████████▎ | 31500/37962 [72:56:36<14:47:54,  8.24s/it]
                                                           
{'loss': 1.0993, 'grad_norm': 7.065129280090332, 'learning_rate': 1.4816211869156194e-06, 'epoch': 0.83}

 83%|████████▎ | 31500/37962 [72:56:36<14:47:54,  8.24s/it]
 83%|████████▎ | 31501/37962 [72:56:44<14:53:29,  8.30s/it]
 83%|████████▎ | 31502/37962 [72:56:52<14:48:39,  8.25s/it]
 83%|████████▎ | 31503/37962 [72:57:01<14:47:35,  8.25s/it]
 83%|████████▎ | 31504/37962 [72:57:09<14:49:02,  8.26s/it]
 83%|████████▎ | 31505/37962 [72:57:17<14:48:52,  8.26s/it]
 83%|████████▎ | 31506/37962 [72:57:25<14:48:26,  8.26s/it]
 83%|████████▎ | 31507/37962 [72:57:34<14:52:29,  8.30s/it]
 83%|████████▎ | 31508/37962 [72:57:42<14:53:09,  8.30s/it]
 83%|████████▎ | 31509/37962 [72:57:51<14:55:33,  8.33s/it]
 83%|████████▎ | 31510/37962 [72:57:58<14:37:52,  8.16s/it]
                                                           
{'loss': 1.1809, 'grad_norm': 6.307096481323242, 'learning_rate': 1.477155386208362e-06, 'epoch': 0.83}

 83%|████████▎ | 31510/37962 [72:57:58<14:37:52,  8.16s/it]
 83%|████████▎ | 31511/37962 [72:58:07<14:45:07,  8.23s/it]
 83%|████████▎ | 31512/37962 [72:58:15<14:50:18,  8.28s/it]
 83%|████████▎ | 31513/37962 [72:58:23<14:49:50,  8.28s/it]
 83%|████████▎ | 31514/37962 [72:58:32<14:47:52,  8.26s/it]
 83%|████████▎ | 31515/37962 [72:58:40<14:40:28,  8.19s/it]
 83%|████████▎ | 31516/37962 [72:58:48<14:37:55,  8.17s/it]
 83%|████████▎ | 31517/37962 [72:58:56<14:41:33,  8.21s/it]
 83%|████████▎ | 31518/37962 [72:59:05<14:48:53,  8.28s/it]
 83%|████████▎ | 31519/37962 [72:59:13<14:55:42,  8.34s/it]
 83%|████████▎ | 31520/37962 [72:59:21<14:46:08,  8.25s/it]
                                                           
{'loss': 1.1323, 'grad_norm': 9.172913551330566, 'learning_rate': 1.4726957891320516e-06, 'epoch': 0.83}

 83%|████████▎ | 31520/37962 [72:59:21<14:46:08,  8.25s/it]
 83%|████████▎ | 31521/37962 [72:59:29<14:51:02,  8.30s/it]
 83%|████████▎ | 31522/37962 [72:59:38<14:53:48,  8.33s/it]
 83%|████████▎ | 31523/37962 [72:59:46<14:53:49,  8.33s/it]
 83%|████████▎ | 31524/37962 [72:59:55<14:56:21,  8.35s/it]
 83%|████████▎ | 31525/37962 [73:00:03<15:08:19,  8.47s/it]
 83%|████████▎ | 31526/37962 [73:00:12<15:05:20,  8.44s/it]
 83%|████████▎ | 31527/37962 [73:00:20<14:59:00,  8.38s/it]
 83%|████████▎ | 31528/37962 [73:00:28<14:43:05,  8.24s/it]
 83%|████████▎ | 31529/37962 [73:00:36<14:47:01,  8.27s/it]
 83%|████████▎ | 31530/37962 [73:00:44<14:44:12,  8.25s/it]
                                                           
{'loss': 1.1859, 'grad_norm': 9.846457481384277, 'learning_rate': 1.4682423989327487e-06, 'epoch': 0.83}

 83%|████████▎ | 31530/37962 [73:00:44<14:44:12,  8.25s/it]
 83%|████████▎ | 31531/37962 [73:00:52<14:35:41,  8.17s/it]
 83%|████████▎ | 31532/37962 [73:01:01<14:41:28,  8.23s/it]
 83%|████████▎ | 31533/37962 [73:01:09<14:47:28,  8.28s/it]
 83%|████████▎ | 31534/37962 [73:01:17<14:46:06,  8.27s/it]
 83%|████████▎ | 31535/37962 [73:01:26<14:46:00,  8.27s/it]
 83%|████████▎ | 31536/37962 [73:01:34<14:46:56,  8.28s/it]
 83%|████████▎ | 31537/37962 [73:01:42<14:47:03,  8.28s/it]
 83%|████████▎ | 31538/37962 [73:01:51<14:46:54,  8.28s/it]
 83%|████████▎ | 31539/37962 [73:01:58<14:33:48,  8.16s/it]
 83%|████████▎ | 31540/37962 [73:02:07<14:39:46,  8.22s/it]
                                                           
{'loss': 1.2412, 'grad_norm': 13.42812442779541, 'learning_rate': 1.4637952188520032e-06, 'epoch': 0.83}

 83%|████████▎ | 31540/37962 [73:02:07<14:39:46,  8.22s/it]
 83%|████████▎ | 31541/37962 [73:02:15<14:41:43,  8.24s/it]
 83%|████████▎ | 31542/37962 [73:02:23<14:45:15,  8.27s/it]
 83%|████████▎ | 31543/37962 [73:02:32<14:51:45,  8.34s/it]
 83%|████████▎ | 31544/37962 [73:02:40<14:41:45,  8.24s/it]
 83%|████████▎ | 31545/37962 [73:02:48<14:40:10,  8.23s/it]
 83%|████████▎ | 31546/37962 [73:02:57<14:49:04,  8.31s/it]
 83%|████████▎ | 31547/37962 [73:03:05<14:38:19,  8.22s/it]
 83%|████████▎ | 31548/37962 [73:03:13<14:38:58,  8.22s/it]
 83%|████████▎ | 31549/37962 [73:03:21<14:39:06,  8.22s/it]
 83%|████████▎ | 31550/37962 [73:03:29<14:44:46,  8.28s/it]
                                                           
{'loss': 1.2043, 'grad_norm': 9.484488487243652, 'learning_rate': 1.4593542521268356e-06, 'epoch': 0.83}

 83%|████████▎ | 31550/37962 [73:03:29<14:44:46,  8.28s/it]
 83%|████████▎ | 31551/37962 [73:03:38<14:46:58,  8.30s/it]
 83%|████████▎ | 31552/37962 [73:03:46<14:42:23,  8.26s/it]
 83%|████████▎ | 31553/37962 [73:03:54<14:43:33,  8.27s/it]
 83%|████████▎ | 31554/37962 [73:04:03<14:58:07,  8.41s/it]
 83%|████████▎ | 31555/37962 [73:04:11<14:48:12,  8.32s/it]
 83%|████████▎ | 31556/37962 [73:04:19<14:46:34,  8.30s/it]
 83%|████████▎ | 31557/37962 [73:04:27<14:33:34,  8.18s/it]
 83%|████████▎ | 31558/37962 [73:04:36<14:43:05,  8.27s/it]
 83%|████████▎ | 31559/37962 [73:04:44<14:43:29,  8.28s/it]
 83%|████████▎ | 31560/37962 [73:04:52<14:40:19,  8.25s/it]
                                                           
{'loss': 1.0403, 'grad_norm': 6.812854766845703, 'learning_rate': 1.4549195019897512e-06, 'epoch': 0.83}

 83%|████████▎ | 31560/37962 [73:04:52<14:40:19,  8.25s/it]
 83%|████████▎ | 31561/37962 [73:05:00<14:36:06,  8.21s/it]
 83%|████████▎ | 31562/37962 [73:05:09<14:42:31,  8.27s/it]
 83%|████████▎ | 31563/37962 [73:05:17<14:45:35,  8.30s/it]
 83%|████████▎ | 31564/37962 [73:05:25<14:42:11,  8.27s/it]
 83%|████████▎ | 31565/37962 [73:05:34<14:40:22,  8.26s/it]
 83%|████████▎ | 31566/37962 [73:05:42<14:38:47,  8.24s/it]
 83%|████████▎ | 31567/37962 [73:05:50<14:30:08,  8.16s/it]
 83%|████████▎ | 31568/37962 [73:05:58<14:32:25,  8.19s/it]
 83%|████████▎ | 31569/37962 [73:06:06<14:30:04,  8.17s/it]
 83%|████████▎ | 31570/37962 [73:06:14<14:27:37,  8.14s/it]
                                                           
{'loss': 1.1561, 'grad_norm': 9.425481796264648, 'learning_rate': 1.4504909716687332e-06, 'epoch': 0.83}

 83%|████████▎ | 31570/37962 [73:06:14<14:27:37,  8.14s/it]
 83%|████████▎ | 31571/37962 [73:06:23<14:32:25,  8.19s/it]
 83%|████████▎ | 31572/37962 [73:06:31<14:33:11,  8.20s/it]
 83%|████████▎ | 31573/37962 [73:06:39<14:33:33,  8.20s/it]
 83%|████████▎ | 31574/37962 [73:06:47<14:35:39,  8.22s/it]
 83%|████████▎ | 31575/37962 [73:06:56<14:40:06,  8.27s/it]
 83%|████████▎ | 31576/37962 [73:07:04<14:45:49,  8.32s/it]
 83%|████████▎ | 31577/37962 [73:07:12<14:45:13,  8.32s/it]
 83%|████████▎ | 31578/37962 [73:07:21<14:48:45,  8.35s/it]
 83%|████████▎ | 31579/37962 [73:07:29<14:46:08,  8.33s/it]
 83%|████████▎ | 31580/37962 [73:07:37<14:46:26,  8.33s/it]
                                                           
{'loss': 1.0601, 'grad_norm': 6.462778568267822, 'learning_rate': 1.4460686643872212e-06, 'epoch': 0.83}

 83%|████████▎ | 31580/37962 [73:07:37<14:46:26,  8.33s/it]
 83%|████████▎ | 31581/37962 [73:07:46<14:44:49,  8.32s/it]
 83%|████████▎ | 31582/37962 [73:07:54<14:34:48,  8.23s/it]
 83%|████████▎ | 31583/37962 [73:08:02<14:33:53,  8.22s/it]
 83%|████████▎ | 31584/37962 [73:08:10<14:32:33,  8.21s/it]
 83%|████████▎ | 31585/37962 [73:08:18<14:22:42,  8.12s/it]
 83%|████████▎ | 31586/37962 [73:08:26<14:29:25,  8.18s/it]
 83%|████████▎ | 31587/37962 [73:08:35<14:31:16,  8.20s/it]
 83%|████████▎ | 31588/37962 [73:08:43<14:25:39,  8.15s/it]
 83%|████████▎ | 31589/37962 [73:08:50<14:14:40,  8.05s/it]
 83%|████████▎ | 31590/37962 [73:08:59<14:22:33,  8.12s/it]
                                                           
{'loss': 1.1366, 'grad_norm': 10.865950584411621, 'learning_rate': 1.4416525833641426e-06, 'epoch': 0.83}

 83%|████████▎ | 31590/37962 [73:08:59<14:22:33,  8.12s/it]
 83%|████████▎ | 31591/37962 [73:09:07<14:25:02,  8.15s/it]
 83%|████████▎ | 31592/37962 [73:09:15<14:32:50,  8.22s/it]
 83%|████████▎ | 31593/37962 [73:09:23<14:30:43,  8.20s/it]
 83%|████████▎ | 31594/37962 [73:09:32<14:26:35,  8.17s/it]
 83%|████████▎ | 31595/37962 [73:09:40<14:35:05,  8.25s/it]
 83%|████████▎ | 31596/37962 [73:09:48<14:40:32,  8.30s/it]
 83%|████████▎ | 31597/37962 [73:09:57<14:47:20,  8.36s/it]
 83%|████████▎ | 31598/37962 [73:10:05<14:45:01,  8.34s/it]
 83%|████████▎ | 31599/37962 [73:10:13<14:35:41,  8.26s/it]
 83%|████████▎ | 31600/37962 [73:10:22<14:37:56,  8.28s/it]
                                                           
{'loss': 1.3141, 'grad_norm': 6.0995869636535645, 'learning_rate': 1.4372427318138882e-06, 'epoch': 0.83}

 83%|████████▎ | 31600/37962 [73:10:22<14:37:56,  8.28s/it]
 83%|████████▎ | 31601/37962 [73:10:30<14:45:03,  8.35s/it]
 83%|████████▎ | 31602/37962 [73:10:38<14:37:24,  8.28s/it]
 83%|████████▎ | 31603/37962 [73:10:46<14:34:26,  8.25s/it]
 83%|████████▎ | 31604/37962 [73:10:55<14:33:06,  8.24s/it]
 83%|████████▎ | 31605/37962 [73:11:03<14:41:52,  8.32s/it]
 83%|████████▎ | 31606/37962 [73:11:12<14:44:59,  8.35s/it]
 83%|████████▎ | 31607/37962 [73:11:20<14:39:55,  8.31s/it]
 83%|████████▎ | 31608/37962 [73:11:28<14:37:04,  8.28s/it]
 83%|████████▎ | 31609/37962 [73:11:36<14:21:23,  8.14s/it]
 83%|████████▎ | 31610/37962 [73:11:44<14:25:14,  8.17s/it]
                                                           
{'loss': 1.1443, 'grad_norm': 13.180307388305664, 'learning_rate': 1.432839112946306e-06, 'epoch': 0.83}

 83%|████████▎ | 31610/37962 [73:11:44<14:25:14,  8.17s/it]
 83%|████████▎ | 31611/37962 [73:11:52<14:27:08,  8.19s/it]
 83%|████████▎ | 31612/37962 [73:12:01<14:29:01,  8.21s/it]
 83%|████████▎ | 31613/37962 [73:12:09<14:33:06,  8.25s/it]
 83%|████████▎ | 31614/37962 [73:12:17<14:32:02,  8.24s/it]
 83%|████████▎ | 31615/37962 [73:12:25<14:24:23,  8.17s/it]
 83%|████████▎ | 31616/37962 [73:12:33<14:20:38,  8.14s/it]
 83%|████████▎ | 31617/37962 [73:12:41<14:25:26,  8.18s/it]
 83%|████████▎ | 31618/37962 [73:12:50<14:21:27,  8.15s/it]
 83%|████████▎ | 31619/37962 [73:12:58<14:23:58,  8.17s/it]
 83%|████████▎ | 31620/37962 [73:13:06<14:28:26,  8.22s/it]
                                                           
{'loss': 1.1565, 'grad_norm': 9.40456485748291, 'learning_rate': 1.4284417299667208e-06, 'epoch': 0.83}

 83%|████████▎ | 31620/37962 [73:13:06<14:28:26,  8.22s/it]
 83%|████████▎ | 31621/37962 [73:13:14<14:28:44,  8.22s/it]
 83%|████████▎ | 31622/37962 [73:13:22<14:25:35,  8.19s/it]
 83%|████████▎ | 31623/37962 [73:13:30<14:18:04,  8.12s/it]
 83%|████████▎ | 31624/37962 [73:13:39<14:25:56,  8.20s/it]
 83%|████████▎ | 31625/37962 [73:13:47<14:29:45,  8.24s/it]
 83%|████████▎ | 31626/37962 [73:13:56<14:41:48,  8.35s/it]
 83%|████████▎ | 31627/37962 [73:14:04<14:44:55,  8.38s/it]
 83%|████████▎ | 31628/37962 [73:14:12<14:39:30,  8.33s/it]
 83%|████████▎ | 31629/37962 [73:14:21<14:36:46,  8.31s/it]
 83%|████████▎ | 31630/37962 [73:14:29<14:37:45,  8.32s/it]
                                                           
{'loss': 1.1625, 'grad_norm': 9.118408203125, 'learning_rate': 1.4240505860759079e-06, 'epoch': 0.83}

 83%|████████▎ | 31630/37962 [73:14:29<14:37:45,  8.32s/it]
 83%|████████▎ | 31631/37962 [73:14:37<14:33:17,  8.28s/it]
 83%|████████▎ | 31632/37962 [73:14:45<14:32:58,  8.27s/it]
 83%|████████▎ | 31633/37962 [73:14:54<14:38:32,  8.33s/it]
 83%|████████▎ | 31634/37962 [73:15:02<14:45:12,  8.39s/it]
 83%|████████▎ | 31635/37962 [73:15:11<14:40:30,  8.35s/it]
 83%|████████▎ | 31636/37962 [73:15:19<14:25:01,  8.20s/it]
 83%|████████▎ | 31637/37962 [73:15:27<14:27:32,  8.23s/it]
 83%|████████▎ | 31638/37962 [73:15:35<14:21:30,  8.17s/it]
 83%|████████▎ | 31639/37962 [73:15:43<14:25:09,  8.21s/it]
 83%|████████▎ | 31640/37962 [73:15:51<14:25:47,  8.22s/it]
                                                           
{'loss': 1.1086, 'grad_norm': 12.277379989624023, 'learning_rate': 1.4196656844701028e-06, 'epoch': 0.83}

 83%|████████▎ | 31640/37962 [73:15:51<14:25:47,  8.22s/it]
 83%|████████▎ | 31641/37962 [73:16:00<14:36:26,  8.32s/it]
 83%|████████▎ | 31642/37962 [73:16:08<14:38:19,  8.34s/it]
 83%|████████▎ | 31643/37962 [73:16:17<14:32:29,  8.28s/it]
 83%|████████▎ | 31644/37962 [73:16:25<14:32:16,  8.28s/it]
 83%|████████▎ | 31645/37962 [73:16:33<14:32:36,  8.29s/it]
 83%|████████▎ | 31646/37962 [73:16:41<14:33:41,  8.30s/it]
 83%|████████▎ | 31647/37962 [73:16:50<14:35:04,  8.31s/it]
 83%|████████▎ | 31648/37962 [73:16:58<14:30:54,  8.28s/it]
 83%|████████▎ | 31649/37962 [73:17:06<14:26:42,  8.24s/it]
 83%|████████▎ | 31650/37962 [73:17:14<14:24:46,  8.22s/it]
                                                           
{'loss': 1.1474, 'grad_norm': 11.576123237609863, 'learning_rate': 1.4152870283410046e-06, 'epoch': 0.83}

 83%|████████▎ | 31650/37962 [73:17:14<14:24:46,  8.22s/it]
 83%|████████▎ | 31651/37962 [73:17:23<14:24:40,  8.22s/it]
 83%|████████▎ | 31652/37962 [73:17:31<14:17:26,  8.15s/it]
 83%|████████▎ | 31653/37962 [73:17:39<14:19:23,  8.17s/it]
 83%|████████▎ | 31654/37962 [73:17:47<14:23:45,  8.22s/it]
 83%|████████▎ | 31655/37962 [73:17:55<14:25:30,  8.23s/it]
 83%|████████▎ | 31656/37962 [73:18:04<14:34:54,  8.32s/it]
 83%|████████▎ | 31657/37962 [73:18:12<14:24:59,  8.23s/it]
 83%|████████▎ | 31658/37962 [73:18:20<14:27:53,  8.26s/it]
 83%|████████▎ | 31659/37962 [73:18:28<14:25:49,  8.24s/it]
 83%|████████▎ | 31660/37962 [73:18:37<14:30:43,  8.29s/it]
                                                           
{'loss': 1.1369, 'grad_norm': 8.516620635986328, 'learning_rate': 1.4109146208757563e-06, 'epoch': 0.83}

 83%|████████▎ | 31660/37962 [73:18:37<14:30:43,  8.29s/it]
 83%|████████▎ | 31661/37962 [73:18:45<14:28:14,  8.27s/it]
 83%|████████▎ | 31662/37962 [73:18:53<14:26:53,  8.26s/it]
 83%|████████▎ | 31663/37962 [73:19:02<14:28:49,  8.28s/it]
 83%|████████▎ | 31664/37962 [73:19:10<14:33:41,  8.32s/it]
 83%|████████▎ | 31665/37962 [73:19:18<14:34:07,  8.33s/it]
 83%|████████▎ | 31666/37962 [73:19:26<14:25:51,  8.25s/it]
 83%|████████▎ | 31667/37962 [73:19:35<14:21:45,  8.21s/it]
 83%|████████▎ | 31668/37962 [73:19:43<14:26:50,  8.26s/it]
 83%|████████▎ | 31669/37962 [73:19:51<14:23:34,  8.23s/it]
 83%|████████▎ | 31670/37962 [73:19:59<14:20:29,  8.21s/it]
                                                           
{'loss': 1.0954, 'grad_norm': 7.281489372253418, 'learning_rate': 1.4065484652569616e-06, 'epoch': 0.83}

 83%|████████▎ | 31670/37962 [73:19:59<14:20:29,  8.21s/it]
 83%|████████▎ | 31671/37962 [73:20:07<14:15:21,  8.16s/it]
 83%|████████▎ | 31672/37962 [73:20:16<14:25:05,  8.25s/it]
 83%|████████▎ | 31673/37962 [73:20:24<14:26:38,  8.27s/it]
 83%|████████▎ | 31674/37962 [73:20:32<14:26:12,  8.27s/it]
 83%|████████▎ | 31675/37962 [73:20:41<14:25:36,  8.26s/it]
 83%|████████▎ | 31676/37962 [73:20:49<14:23:47,  8.24s/it]
 83%|████████▎ | 31677/37962 [73:20:57<14:26:21,  8.27s/it]
 83%|████████▎ | 31678/37962 [73:21:05<14:25:26,  8.26s/it]
 83%|████████▎ | 31679/37962 [73:21:13<14:17:36,  8.19s/it]
 83%|████████▎ | 31680/37962 [73:21:22<14:17:31,  8.19s/it]
                                                           
{'loss': 1.2363, 'grad_norm': 8.071185111999512, 'learning_rate': 1.4021885646626677e-06, 'epoch': 0.83}

 83%|████████▎ | 31680/37962 [73:21:22<14:17:31,  8.19s/it]
 83%|████████▎ | 31681/37962 [73:21:30<14:26:56,  8.28s/it]
 83%|████████▎ | 31682/37962 [73:21:46<18:31:27, 10.62s/it]
 83%|████████▎ | 31683/37962 [73:21:54<17:18:00,  9.92s/it]
 83%|████████▎ | 31684/37962 [73:22:03<16:32:56,  9.49s/it]
 83%|████████▎ | 31685/37962 [73:22:11<15:54:13,  9.12s/it]
 83%|████████▎ | 31686/37962 [73:22:19<15:27:32,  8.87s/it]
 83%|████████▎ | 31687/37962 [73:22:28<15:07:10,  8.67s/it]
 83%|████████▎ | 31688/37962 [73:22:36<14:56:24,  8.57s/it]
 83%|████████▎ | 31689/37962 [73:22:44<14:50:16,  8.52s/it]
 83%|████████▎ | 31690/37962 [73:22:53<14:46:00,  8.48s/it]
                                                           
{'loss': 1.2071, 'grad_norm': 13.013749122619629, 'learning_rate': 1.397834922266369e-06, 'epoch': 0.83}

 83%|████████▎ | 31690/37962 [73:22:53<14:46:00,  8.48s/it]
 83%|████████▎ | 31691/37962 [73:23:01<14:31:39,  8.34s/it]
 83%|████████▎ | 31692/37962 [73:23:09<14:37:33,  8.40s/it]
 83%|████████▎ | 31693/37962 [73:23:18<14:35:12,  8.38s/it]
 83%|████████▎ | 31694/37962 [73:23:26<14:25:07,  8.28s/it]
 83%|████████▎ | 31695/37962 [73:23:34<14:20:14,  8.24s/it]
 83%|████████▎ | 31696/37962 [73:23:42<14:23:54,  8.27s/it]
 83%|████████▎ | 31697/37962 [73:23:50<14:18:57,  8.23s/it]
 83%|████████▎ | 31698/37962 [73:23:59<14:19:27,  8.23s/it]
 84%|████████▎ | 31699/37962 [73:24:07<14:26:39,  8.30s/it]
 84%|████████▎ | 31700/37962 [73:24:15<14:30:12,  8.34s/it]
                                                           
{'loss': 1.1243, 'grad_norm': 9.298771858215332, 'learning_rate': 1.3934875412370075e-06, 'epoch': 0.84}

 84%|████████▎ | 31700/37962 [73:24:15<14:30:12,  8.34s/it]
 84%|████████▎ | 31701/37962 [73:24:24<14:28:53,  8.33s/it]
 84%|████████▎ | 31702/37962 [73:24:32<14:36:25,  8.40s/it]
 84%|████████▎ | 31703/37962 [73:24:41<14:37:45,  8.41s/it]
 84%|████████▎ | 31704/37962 [73:24:49<14:29:12,  8.33s/it]
 84%|████████▎ | 31705/37962 [73:24:57<14:30:35,  8.35s/it]
 84%|████████▎ | 31706/37962 [73:25:05<14:17:47,  8.23s/it]
 84%|████████▎ | 31707/37962 [73:25:14<14:21:11,  8.26s/it]
 84%|████████▎ | 31708/37962 [73:25:22<14:19:38,  8.25s/it]
 84%|████████▎ | 31709/37962 [73:25:38<18:18:24, 10.54s/it]
 84%|████████▎ | 31710/37962 [73:25:46<17:05:36,  9.84s/it]
                                                           
{'loss': 1.157, 'grad_norm': 11.528572082519531, 'learning_rate': 1.3891464247389696e-06, 'epoch': 0.84}

 84%|████████▎ | 31710/37962 [73:25:46<17:05:36,  9.84s/it]
 84%|████████▎ | 31711/37962 [73:25:54<16:11:35,  9.33s/it]
 84%|████████▎ | 31712/37962 [73:26:02<15:41:12,  9.04s/it]
 84%|████████▎ | 31713/37962 [73:26:11<15:20:26,  8.84s/it]
 84%|████████▎ | 31714/37962 [73:26:19<15:01:38,  8.66s/it]
 84%|████████▎ | 31715/37962 [73:26:27<14:54:18,  8.59s/it]
 84%|████████▎ | 31716/37962 [73:26:36<14:47:17,  8.52s/it]
 84%|████████▎ | 31717/37962 [73:26:44<14:35:08,  8.41s/it]
 84%|████████▎ | 31718/37962 [73:26:52<14:19:57,  8.26s/it]
 84%|████████▎ | 31719/37962 [73:27:00<14:19:40,  8.26s/it]
 84%|████████▎ | 31720/37962 [73:27:08<14:14:23,  8.21s/it]
                                                           
{'loss': 1.1696, 'grad_norm': 9.760581970214844, 'learning_rate': 1.3848115759320757e-06, 'epoch': 0.84}

 84%|████████▎ | 31720/37962 [73:27:08<14:14:23,  8.21s/it]
 84%|████████▎ | 31721/37962 [73:27:16<14:16:16,  8.23s/it]
 84%|████████▎ | 31722/37962 [73:27:25<14:13:36,  8.21s/it]
 84%|████████▎ | 31723/37962 [73:27:33<14:17:16,  8.24s/it]
 84%|████████▎ | 31724/37962 [73:27:41<14:14:16,  8.22s/it]
 84%|████████▎ | 31725/37962 [73:27:49<14:15:15,  8.23s/it]
 84%|████████▎ | 31726/37962 [73:27:58<14:13:15,  8.21s/it]
 84%|████████▎ | 31727/37962 [73:28:06<14:14:52,  8.23s/it]
 84%|████████▎ | 31728/37962 [73:28:14<14:22:07,  8.30s/it]
 84%|████████▎ | 31729/37962 [73:28:23<14:26:13,  8.34s/it]
 84%|████████▎ | 31730/37962 [73:28:31<14:22:34,  8.30s/it]
                                                           
{'loss': 1.1982, 'grad_norm': 8.217482566833496, 'learning_rate': 1.3804829979715862e-06, 'epoch': 0.84}

 84%|████████▎ | 31730/37962 [73:28:31<14:22:34,  8.30s/it]
 84%|████████▎ | 31731/37962 [73:28:39<14:20:10,  8.28s/it]
 84%|████████▎ | 31732/37962 [73:28:47<14:18:56,  8.27s/it]
 84%|████████▎ | 31733/37962 [73:28:56<14:15:50,  8.24s/it]
 84%|████████▎ | 31734/37962 [73:29:04<14:17:44,  8.26s/it]
 84%|████████▎ | 31735/37962 [73:29:12<14:15:37,  8.24s/it]
 84%|████████▎ | 31736/37962 [73:29:20<14:17:08,  8.26s/it]
 84%|████████▎ | 31737/37962 [73:29:28<14:04:17,  8.14s/it]
 84%|████████▎ | 31738/37962 [73:29:37<14:11:11,  8.21s/it]
 84%|████████▎ | 31739/37962 [73:29:45<14:02:46,  8.13s/it]
 84%|████████▎ | 31740/37962 [73:29:53<14:05:28,  8.15s/it]
                                                           
{'loss': 1.2329, 'grad_norm': 6.121707439422607, 'learning_rate': 1.3761606940082018e-06, 'epoch': 0.84}

 84%|████████▎ | 31740/37962 [73:29:53<14:05:28,  8.15s/it]
 84%|████████▎ | 31741/37962 [73:30:01<14:00:38,  8.11s/it]
 84%|████████▎ | 31742/37962 [73:30:09<14:09:23,  8.19s/it]
 84%|████████▎ | 31743/37962 [73:30:17<13:59:08,  8.10s/it]
 84%|████████▎ | 31744/37962 [73:30:25<14:02:52,  8.13s/it]
 84%|████████▎ | 31745/37962 [73:30:34<14:08:30,  8.19s/it]
 84%|████████▎ | 31746/37962 [73:30:42<14:12:19,  8.23s/it]
 84%|████████▎ | 31747/37962 [73:30:50<14:14:45,  8.25s/it]
 84%|████████▎ | 31748/37962 [73:30:59<14:18:02,  8.28s/it]
 84%|████████▎ | 31749/37962 [73:31:07<14:19:10,  8.30s/it]
 84%|████████▎ | 31750/37962 [73:31:15<14:19:41,  8.30s/it]
                                                           
{'loss': 1.1399, 'grad_norm': 9.802854537963867, 'learning_rate': 1.3718446671880482e-06, 'epoch': 0.84}

 84%|████████▎ | 31750/37962 [73:31:15<14:19:41,  8.30s/it]
 84%|████████▎ | 31751/37962 [73:31:24<14:21:32,  8.32s/it]
 84%|████████▎ | 31752/37962 [73:31:31<14:08:30,  8.20s/it]
 84%|████████▎ | 31753/37962 [73:31:40<14:15:19,  8.27s/it]
 84%|████████▎ | 31754/37962 [73:31:48<14:10:18,  8.22s/it]
 84%|████████▎ | 31755/37962 [73:31:56<14:11:17,  8.23s/it]
 84%|████████▎ | 31756/37962 [73:32:04<14:07:38,  8.20s/it]
 84%|████████▎ | 31757/37962 [73:32:12<14:02:11,  8.14s/it]
 84%|████████▎ | 31758/37962 [73:32:20<13:49:53,  8.03s/it]
 84%|████████▎ | 31759/37962 [73:32:28<13:59:29,  8.12s/it]
 84%|████████▎ | 31760/37962 [73:32:36<13:55:02,  8.08s/it]
                                                           
{'loss': 1.1493, 'grad_norm': 9.835196495056152, 'learning_rate': 1.367534920652689e-06, 'epoch': 0.84}

 84%|████████▎ | 31760/37962 [73:32:36<13:55:02,  8.08s/it]
 84%|████████▎ | 31761/37962 [73:32:45<14:01:10,  8.14s/it]
 84%|████████▎ | 31762/37962 [73:32:53<14:10:34,  8.23s/it]
 84%|████████▎ | 31763/37962 [73:33:02<14:17:41,  8.30s/it]
 84%|████████▎ | 31764/37962 [73:33:10<14:18:09,  8.31s/it]
 84%|████████▎ | 31765/37962 [73:33:18<14:18:34,  8.31s/it]
 84%|████████▎ | 31766/37962 [73:33:26<14:09:07,  8.22s/it]
 84%|████████▎ | 31767/37962 [73:33:35<14:10:04,  8.23s/it]
 84%|████████▎ | 31768/37962 [73:33:43<14:11:23,  8.25s/it]
 84%|████████▎ | 31769/37962 [73:33:51<14:18:20,  8.32s/it]
 84%|████████▎ | 31770/37962 [73:34:00<14:14:30,  8.28s/it]
                                                           
{'loss': 1.1631, 'grad_norm': 5.81845235824585, 'learning_rate': 1.363231457539116e-06, 'epoch': 0.84}

 84%|████████▎ | 31770/37962 [73:34:00<14:14:30,  8.28s/it]
 84%|████████▎ | 31771/37962 [73:34:08<14:12:35,  8.26s/it]
 84%|████████▎ | 31772/37962 [73:34:16<14:16:45,  8.30s/it]
 84%|████████▎ | 31773/37962 [73:34:24<14:13:08,  8.27s/it]
 84%|████████▎ | 31774/37962 [73:34:33<14:16:31,  8.30s/it]
 84%|████████▎ | 31775/37962 [73:34:41<14:19:33,  8.34s/it]
 84%|████████▎ | 31776/37962 [73:34:49<14:15:00,  8.29s/it]
 84%|████████▎ | 31777/37962 [73:34:57<14:11:19,  8.26s/it]
 84%|████████▎ | 31778/37962 [73:35:06<14:17:51,  8.32s/it]
 84%|████████▎ | 31779/37962 [73:35:14<14:07:44,  8.23s/it]
 84%|████████▎ | 31780/37962 [73:35:22<14:10:48,  8.26s/it]
                                                           
{'loss': 1.1535, 'grad_norm': 10.56919002532959, 'learning_rate': 1.3589342809797423e-06, 'epoch': 0.84}

 84%|████████▎ | 31780/37962 [73:35:22<14:10:48,  8.26s/it]
 84%|████████▎ | 31781/37962 [73:35:30<14:01:51,  8.17s/it]
 84%|████████▎ | 31782/37962 [73:35:39<14:09:10,  8.24s/it]
 84%|████████▎ | 31783/37962 [73:35:47<14:06:58,  8.22s/it]
 84%|████████▎ | 31784/37962 [73:35:55<14:03:46,  8.19s/it]
 84%|████████▎ | 31785/37962 [73:36:03<14:10:06,  8.26s/it]
 84%|████████▎ | 31786/37962 [73:36:12<14:16:40,  8.32s/it]
 84%|████████▎ | 31787/37962 [73:36:20<14:14:03,  8.30s/it]
 84%|████████▎ | 31788/37962 [73:36:28<14:12:10,  8.28s/it]
 84%|████████▎ | 31789/37962 [73:36:37<14:17:47,  8.34s/it]
 84%|████████▎ | 31790/37962 [73:36:45<14:05:18,  8.22s/it]
                                                           
{'loss': 1.0689, 'grad_norm': 6.401373863220215, 'learning_rate': 1.3546433941024084e-06, 'epoch': 0.84}

 84%|████████▎ | 31790/37962 [73:36:45<14:05:18,  8.22s/it]
 84%|████████▎ | 31791/37962 [73:36:53<14:03:50,  8.20s/it]
 84%|████████▎ | 31792/37962 [73:37:01<14:05:11,  8.22s/it]
 84%|████████▎ | 31793/37962 [73:37:09<13:50:16,  8.08s/it]
 84%|████████▍ | 31794/37962 [73:37:17<13:56:27,  8.14s/it]
 84%|████████▍ | 31795/37962 [73:37:26<14:01:13,  8.18s/it]
 84%|████████▍ | 31796/37962 [73:37:34<14:03:20,  8.21s/it]
 84%|████████▍ | 31797/37962 [73:37:42<14:05:47,  8.23s/it]
 84%|████████▍ | 31798/37962 [73:37:50<14:09:15,  8.27s/it]
 84%|████████▍ | 31799/37962 [73:37:59<14:08:01,  8.26s/it]
 84%|████████▍ | 31800/37962 [73:38:07<13:56:48,  8.15s/it]
                                                           
{'loss': 1.217, 'grad_norm': 9.20201587677002, 'learning_rate': 1.3503588000303791e-06, 'epoch': 0.84}

 84%|████████▍ | 31800/37962 [73:38:07<13:56:48,  8.15s/it]
 84%|████████▍ | 31801/37962 [73:38:14<13:47:02,  8.05s/it]
 84%|████████▍ | 31802/37962 [73:38:22<13:45:49,  8.04s/it]
 84%|████████▍ | 31803/37962 [73:38:31<13:49:34,  8.08s/it]
 84%|████████▍ | 31804/37962 [73:38:39<13:47:06,  8.06s/it]
 84%|████████▍ | 31805/37962 [73:38:46<13:37:16,  7.96s/it]
 84%|████████▍ | 31806/37962 [73:38:55<13:54:01,  8.13s/it]
 84%|████████▍ | 31807/37962 [73:39:03<14:01:52,  8.21s/it]
 84%|████████▍ | 31808/37962 [73:39:11<14:02:20,  8.21s/it]
 84%|████████▍ | 31809/37962 [73:39:20<14:04:06,  8.23s/it]
 84%|████████▍ | 31810/37962 [73:39:28<14:10:25,  8.29s/it]
                                                           
{'loss': 1.2464, 'grad_norm': 15.267066955566406, 'learning_rate': 1.346080501882332e-06, 'epoch': 0.84}

 84%|████████▍ | 31810/37962 [73:39:28<14:10:25,  8.29s/it]
 84%|████████▍ | 31811/37962 [73:39:36<14:10:30,  8.30s/it]
 84%|████████▍ | 31812/37962 [73:39:45<14:13:40,  8.33s/it]
 84%|████████▍ | 31813/37962 [73:39:53<14:11:16,  8.31s/it]
 84%|████████▍ | 31814/37962 [73:40:02<14:16:18,  8.36s/it]
 84%|████████▍ | 31815/37962 [73:40:10<14:05:49,  8.26s/it]
 84%|████████▍ | 31816/37962 [73:40:18<14:08:35,  8.28s/it]
 84%|████████▍ | 31817/37962 [73:40:26<14:05:22,  8.25s/it]
 84%|████████▍ | 31818/37962 [73:40:35<14:16:07,  8.36s/it]
 84%|████████▍ | 31819/37962 [73:40:43<14:13:02,  8.33s/it]
 84%|████████▍ | 31820/37962 [73:40:51<14:12:09,  8.32s/it]
                                                           
{'loss': 1.1573, 'grad_norm': 13.017342567443848, 'learning_rate': 1.3418085027723703e-06, 'epoch': 0.84}

 84%|████████▍ | 31820/37962 [73:40:51<14:12:09,  8.32s/it]
 84%|████████▍ | 31821/37962 [73:41:00<14:09:34,  8.30s/it]
 84%|████████▍ | 31822/37962 [73:41:08<14:06:23,  8.27s/it]
 84%|████████▍ | 31823/37962 [73:41:16<14:14:58,  8.36s/it]
 84%|████████▍ | 31824/37962 [73:41:24<14:06:10,  8.27s/it]
 84%|████████▍ | 31825/37962 [73:41:33<14:01:38,  8.23s/it]
 84%|████████▍ | 31826/37962 [73:41:41<13:59:16,  8.21s/it]
 84%|████████▍ | 31827/37962 [73:41:49<13:55:34,  8.17s/it]
 84%|████████▍ | 31828/37962 [73:41:57<13:59:38,  8.21s/it]
 84%|████████▍ | 31829/37962 [73:42:05<14:00:14,  8.22s/it]
 84%|████████▍ | 31830/37962 [73:42:14<14:03:10,  8.25s/it]
                                                           
{'loss': 1.0973, 'grad_norm': 7.63369083404541, 'learning_rate': 1.3375428058100081e-06, 'epoch': 0.84}

 84%|████████▍ | 31830/37962 [73:42:14<14:03:10,  8.25s/it]
 84%|████████▍ | 31831/37962 [73:42:22<14:03:26,  8.25s/it]
 84%|████████▍ | 31832/37962 [73:42:30<14:04:58,  8.27s/it]
 84%|████████▍ | 31833/37962 [73:42:38<14:01:26,  8.24s/it]
 84%|████████▍ | 31834/37962 [73:42:47<14:05:25,  8.28s/it]
 84%|████████▍ | 31835/37962 [73:42:55<14:09:47,  8.32s/it]
 84%|████████▍ | 31836/37962 [73:43:03<14:08:20,  8.31s/it]
 84%|████████▍ | 31837/37962 [73:43:12<14:02:22,  8.25s/it]
 84%|████████▍ | 31838/37962 [73:43:20<14:01:43,  8.25s/it]
 84%|████████▍ | 31839/37962 [73:43:28<14:05:09,  8.28s/it]
 84%|████████▍ | 31840/37962 [73:43:37<14:10:05,  8.33s/it]
                                                           
{'loss': 1.1093, 'grad_norm': 9.179638862609863, 'learning_rate': 1.3332834141001671e-06, 'epoch': 0.84}

 84%|████████▍ | 31840/37962 [73:43:37<14:10:05,  8.33s/it]
 84%|████████▍ | 31841/37962 [73:43:45<14:07:28,  8.31s/it]
 84%|████████▍ | 31842/37962 [73:43:53<14:04:21,  8.28s/it]
 84%|████████▍ | 31843/37962 [73:44:01<14:06:11,  8.30s/it]
 84%|████████▍ | 31844/37962 [73:44:10<14:08:53,  8.33s/it]
 84%|████████▍ | 31845/37962 [73:44:18<14:05:59,  8.30s/it]
 84%|████████▍ | 31846/37962 [73:44:26<13:59:00,  8.23s/it]
 84%|████████▍ | 31847/37962 [73:44:34<13:55:08,  8.19s/it]
 84%|████████▍ | 31848/37962 [73:44:42<13:55:28,  8.20s/it]
 84%|████████▍ | 31849/37962 [73:44:51<13:54:42,  8.19s/it]
 84%|████████▍ | 31850/37962 [73:44:59<13:52:12,  8.17s/it]
                                                           
{'loss': 1.1711, 'grad_norm': 14.927624702453613, 'learning_rate': 1.3290303307431918e-06, 'epoch': 0.84}

 84%|████████▍ | 31850/37962 [73:44:59<13:52:12,  8.17s/it]
 84%|████████▍ | 31851/37962 [73:45:07<13:54:45,  8.20s/it]
 84%|████████▍ | 31852/37962 [73:45:15<13:58:56,  8.24s/it]
 84%|████████▍ | 31853/37962 [73:45:24<13:57:55,  8.23s/it]
 84%|████████▍ | 31854/37962 [73:45:31<13:49:50,  8.15s/it]
 84%|████████▍ | 31855/37962 [73:45:40<13:50:38,  8.16s/it]
 84%|████████▍ | 31856/37962 [73:45:48<13:57:00,  8.22s/it]
 84%|████████▍ | 31857/37962 [73:45:56<13:46:22,  8.12s/it]
 84%|████████▍ | 31858/37962 [73:46:04<13:54:49,  8.21s/it]
 84%|████████▍ | 31859/37962 [73:46:13<14:05:21,  8.31s/it]
 84%|████████▍ | 31860/37962 [73:46:21<14:08:43,  8.35s/it]
                                                           
{'loss': 1.291, 'grad_norm': 8.259521484375, 'learning_rate': 1.3247835588348211e-06, 'epoch': 0.84}

 84%|████████▍ | 31860/37962 [73:46:21<14:08:43,  8.35s/it]
 84%|████████▍ | 31861/37962 [73:46:30<14:11:24,  8.37s/it]
 84%|████████▍ | 31862/37962 [73:46:38<14:08:53,  8.35s/it]
 84%|████████▍ | 31863/37962 [73:46:46<14:07:02,  8.33s/it]
 84%|████████▍ | 31864/37962 [73:46:54<14:01:40,  8.28s/it]
 84%|████████▍ | 31865/37962 [73:47:03<13:59:33,  8.26s/it]
 84%|████████▍ | 31866/37962 [73:47:11<14:01:48,  8.29s/it]
 84%|████████▍ | 31867/37962 [73:47:19<13:53:53,  8.21s/it]
 84%|████████▍ | 31868/37962 [73:47:27<13:52:39,  8.20s/it]
 84%|████████▍ | 31869/37962 [73:47:36<13:54:41,  8.22s/it]
 84%|████████▍ | 31870/37962 [73:47:44<13:56:14,  8.24s/it]
                                                           
{'loss': 1.1318, 'grad_norm': 12.4782075881958, 'learning_rate': 1.3205431014662106e-06, 'epoch': 0.84}

 84%|████████▍ | 31870/37962 [73:47:44<13:56:14,  8.24s/it]
 84%|████████▍ | 31871/37962 [73:47:52<13:54:13,  8.22s/it]
 84%|████████▍ | 31872/37962 [73:48:00<13:55:51,  8.24s/it]
 84%|████████▍ | 31873/37962 [73:48:08<13:51:36,  8.19s/it]
 84%|████████▍ | 31874/37962 [73:48:17<13:53:09,  8.21s/it]
 84%|████████▍ | 31875/37962 [73:48:25<13:54:13,  8.22s/it]
 84%|████████▍ | 31876/37962 [73:48:33<14:01:29,  8.30s/it]
 84%|████████▍ | 31877/37962 [73:48:42<13:58:46,  8.27s/it]
 84%|████████▍ | 31878/37962 [73:48:50<14:02:19,  8.31s/it]
 84%|████████▍ | 31879/37962 [73:48:58<13:58:34,  8.27s/it]
 84%|████████▍ | 31880/37962 [73:49:06<14:00:55,  8.30s/it]
                                                           
{'loss': 1.1766, 'grad_norm': 8.319181442260742, 'learning_rate': 1.3163089617239199e-06, 'epoch': 0.84}

 84%|████████▍ | 31880/37962 [73:49:06<14:00:55,  8.30s/it]
 84%|████████▍ | 31881/37962 [73:49:15<14:02:11,  8.31s/it]
 84%|████████▍ | 31882/37962 [73:49:23<14:01:12,  8.30s/it]
 84%|████████▍ | 31883/37962 [73:49:31<13:55:52,  8.25s/it]
 84%|████████▍ | 31884/37962 [73:49:39<13:55:07,  8.24s/it]
 84%|████████▍ | 31885/37962 [73:49:47<13:47:50,  8.17s/it]
 84%|████████▍ | 31886/37962 [73:49:56<13:44:33,  8.14s/it]
 84%|████████▍ | 31887/37962 [73:50:04<13:45:00,  8.15s/it]
 84%|████████▍ | 31888/37962 [73:50:12<13:42:21,  8.12s/it]
 84%|████████▍ | 31889/37962 [73:50:20<13:45:42,  8.16s/it]
 84%|████████▍ | 31890/37962 [73:50:28<13:47:49,  8.18s/it]
                                                           
{'loss': 1.2385, 'grad_norm': 9.566555976867676, 'learning_rate': 1.3120811426898972e-06, 'epoch': 0.84}

 84%|████████▍ | 31890/37962 [73:50:28<13:47:49,  8.18s/it]
 84%|████████▍ | 31891/37962 [73:50:36<13:49:42,  8.20s/it]
 84%|████████▍ | 31892/37962 [73:50:45<13:45:25,  8.16s/it]
 84%|████████▍ | 31893/37962 [73:50:53<13:46:25,  8.17s/it]
 84%|████████▍ | 31894/37962 [73:51:01<13:54:58,  8.26s/it]
 84%|████████▍ | 31895/37962 [73:51:09<13:49:04,  8.20s/it]
 84%|████████▍ | 31896/37962 [73:51:18<13:53:37,  8.25s/it]
 84%|████████▍ | 31897/37962 [73:51:26<13:54:05,  8.25s/it]
 84%|████████▍ | 31898/37962 [73:51:34<13:54:34,  8.26s/it]
 84%|████████▍ | 31899/37962 [73:51:43<13:57:48,  8.29s/it]
 84%|████████▍ | 31900/37962 [73:51:51<13:51:59,  8.23s/it]
                                                           
{'loss': 1.2096, 'grad_norm': 7.779786109924316, 'learning_rate': 1.3078596474415029e-06, 'epoch': 0.84}

 84%|████████▍ | 31900/37962 [73:51:51<13:51:59,  8.23s/it]
 84%|████████▍ | 31901/37962 [73:51:59<13:41:52,  8.14s/it]
 84%|████████▍ | 31902/37962 [73:52:07<13:47:16,  8.19s/it]
 84%|████████▍ | 31903/37962 [73:52:15<13:51:22,  8.23s/it]
 84%|████████▍ | 31904/37962 [73:52:23<13:49:44,  8.22s/it]
 84%|████████▍ | 31905/37962 [73:52:31<13:46:58,  8.19s/it]
 84%|████████▍ | 31906/37962 [73:52:40<13:47:56,  8.20s/it]
 84%|████████▍ | 31907/37962 [73:52:48<13:47:02,  8.20s/it]
 84%|████████▍ | 31908/37962 [73:52:56<13:50:38,  8.23s/it]
 84%|████████▍ | 31909/37962 [73:53:04<13:51:46,  8.24s/it]
 84%|████████▍ | 31910/37962 [73:53:13<13:50:43,  8.24s/it]
                                                           
{'loss': 1.1596, 'grad_norm': 13.955842018127441, 'learning_rate': 1.3036444790514913e-06, 'epoch': 0.84}

 84%|████████▍ | 31910/37962 [73:53:13<13:50:43,  8.24s/it]
 84%|████████▍ | 31911/37962 [73:53:21<13:40:25,  8.14s/it]
 84%|████████▍ | 31912/37962 [73:53:29<13:43:48,  8.17s/it]
 84%|████████▍ | 31913/37962 [73:53:37<13:38:23,  8.12s/it]
 84%|████████▍ | 31914/37962 [73:53:45<13:49:29,  8.23s/it]
 84%|████████▍ | 31915/37962 [73:53:54<13:49:06,  8.23s/it]
 84%|████████▍ | 31916/37962 [73:54:02<13:51:24,  8.25s/it]
 84%|████████▍ | 31917/37962 [73:54:10<13:51:33,  8.25s/it]
 84%|████████▍ | 31918/37962 [73:54:18<13:52:11,  8.26s/it]
 84%|████████▍ | 31919/37962 [73:54:27<13:47:45,  8.22s/it]
 84%|████████▍ | 31920/37962 [73:54:35<13:50:57,  8.25s/it]
                                                           
{'loss': 1.2116, 'grad_norm': 5.74229097366333, 'learning_rate': 1.2994356405880083e-06, 'epoch': 0.84}

 84%|████████▍ | 31920/37962 [73:54:35<13:50:57,  8.25s/it]
 84%|████████▍ | 31921/37962 [73:54:43<13:48:18,  8.23s/it]
 84%|████████▍ | 31922/37962 [73:54:51<13:44:09,  8.19s/it]
 84%|████████▍ | 31923/37962 [73:55:00<13:52:10,  8.27s/it]
 84%|████████▍ | 31924/37962 [73:55:08<13:57:12,  8.32s/it]
 84%|████████▍ | 31925/37962 [73:55:16<13:54:36,  8.29s/it]
 84%|████████▍ | 31926/37962 [73:55:24<13:37:33,  8.13s/it]
 84%|████████▍ | 31927/37962 [73:55:32<13:41:55,  8.17s/it]
 84%|████████▍ | 31928/37962 [73:55:41<13:48:05,  8.23s/it]
 84%|████████▍ | 31929/37962 [73:55:49<13:45:43,  8.21s/it]
 84%|████████▍ | 31930/37962 [73:55:57<13:48:04,  8.24s/it]
                                                           
{'loss': 1.1322, 'grad_norm': 6.232998847961426, 'learning_rate': 1.2952331351145976e-06, 'epoch': 0.84}

 84%|████████▍ | 31930/37962 [73:55:57<13:48:04,  8.24s/it]
 84%|████████▍ | 31931/37962 [73:56:05<13:48:22,  8.24s/it]
 84%|████████▍ | 31932/37962 [73:56:14<13:46:06,  8.22s/it]
 84%|████████▍ | 31933/37962 [73:56:22<13:51:00,  8.27s/it]
 84%|████████▍ | 31934/37962 [73:56:30<13:51:07,  8.27s/it]
 84%|████████▍ | 31935/37962 [73:56:39<13:53:37,  8.30s/it]
 84%|████████▍ | 31936/37962 [73:56:47<13:45:02,  8.21s/it]
 84%|████████▍ | 31937/37962 [73:56:55<13:47:53,  8.24s/it]
 84%|████████▍ | 31938/37962 [73:57:03<13:51:38,  8.28s/it]
 84%|████████▍ | 31939/37962 [73:57:12<13:58:24,  8.35s/it]
 84%|████████▍ | 31940/37962 [73:57:20<13:54:38,  8.32s/it]
                                                           
{'loss': 1.0641, 'grad_norm': 16.023740768432617, 'learning_rate': 1.2910369656901878e-06, 'epoch': 0.84}

 84%|████████▍ | 31940/37962 [73:57:20<13:54:38,  8.32s/it]
 84%|████████▍ | 31941/37962 [73:57:28<13:55:20,  8.32s/it]
 84%|████████▍ | 31942/37962 [73:57:37<13:53:40,  8.31s/it]
 84%|████████▍ | 31943/37962 [73:57:45<13:52:08,  8.30s/it]
 84%|████████▍ | 31944/37962 [73:57:53<13:46:47,  8.24s/it]
 84%|████████▍ | 31945/37962 [73:58:01<13:48:48,  8.26s/it]
 84%|████████▍ | 31946/37962 [73:58:10<13:50:02,  8.28s/it]
 84%|████████▍ | 31947/37962 [73:58:18<13:49:09,  8.27s/it]
 84%|████████▍ | 31948/37962 [73:58:26<13:55:02,  8.33s/it]
 84%|████████▍ | 31949/37962 [73:58:35<13:58:22,  8.37s/it]
 84%|████████▍ | 31950/37962 [73:58:43<13:54:08,  8.32s/it]
                                                           
{'loss': 1.2735, 'grad_norm': 7.124833583831787, 'learning_rate': 1.286847135369096e-06, 'epoch': 0.84}

 84%|████████▍ | 31950/37962 [73:58:43<13:54:08,  8.32s/it]
 84%|████████▍ | 31951/37962 [73:58:51<13:47:29,  8.26s/it]
 84%|████████▍ | 31952/37962 [73:58:59<13:46:18,  8.25s/it]
 84%|████████▍ | 31953/37962 [73:59:08<13:55:08,  8.34s/it]
 84%|████████▍ | 31954/37962 [73:59:16<13:52:42,  8.32s/it]
 84%|████████▍ | 31955/37962 [73:59:24<13:49:51,  8.29s/it]
 84%|████████▍ | 31956/37962 [73:59:33<13:54:28,  8.34s/it]
 84%|████████▍ | 31957/37962 [73:59:41<13:51:36,  8.31s/it]
 84%|████████▍ | 31958/37962 [73:59:49<13:53:13,  8.33s/it]
 84%|████████▍ | 31959/37962 [73:59:58<13:56:17,  8.36s/it]
 84%|████████▍ | 31960/37962 [74:00:06<13:56:59,  8.37s/it]
                                                           
{'loss': 1.2098, 'grad_norm': 5.208352565765381, 'learning_rate': 1.2826636472010312e-06, 'epoch': 0.84}

 84%|████████▍ | 31960/37962 [74:00:06<13:56:59,  8.37s/it]
 84%|████████▍ | 31961/37962 [74:00:14<13:47:18,  8.27s/it]
 84%|████████▍ | 31962/37962 [74:00:22<13:41:49,  8.22s/it]
 84%|████████▍ | 31963/37962 [74:00:31<13:41:36,  8.22s/it]
 84%|████████▍ | 31964/37962 [74:00:39<13:40:45,  8.21s/it]
 84%|████████▍ | 31965/37962 [74:00:47<13:33:59,  8.14s/it]
 84%|████████▍ | 31966/37962 [74:00:55<13:25:31,  8.06s/it]
 84%|████████▍ | 31967/37962 [74:01:03<13:35:59,  8.17s/it]
 84%|████████▍ | 31968/37962 [74:01:11<13:37:56,  8.19s/it]
 84%|████████▍ | 31969/37962 [74:01:27<17:22:41, 10.44s/it]
 84%|████████▍ | 31970/37962 [74:01:36<16:27:37,  9.89s/it]
                                                           
{'loss': 1.1647, 'grad_norm': 7.167943477630615, 'learning_rate': 1.2784865042310779e-06, 'epoch': 0.84}

 84%|████████▍ | 31970/37962 [74:01:36<16:27:37,  9.89s/it]
 84%|████████▍ | 31971/37962 [74:01:44<15:39:18,  9.41s/it]
 84%|████████▍ | 31972/37962 [74:01:52<15:02:16,  9.04s/it]
 84%|████████▍ | 31973/37962 [74:02:01<14:45:50,  8.87s/it]
 84%|████████▍ | 31974/37962 [74:02:09<14:18:27,  8.60s/it]
 84%|████████▍ | 31975/37962 [74:02:17<14:06:06,  8.48s/it]
 84%|████████▍ | 31976/37962 [74:02:25<13:53:46,  8.36s/it]
 84%|████████▍ | 31977/37962 [74:02:33<13:50:32,  8.33s/it]
 84%|████████▍ | 31978/37962 [74:02:41<13:45:48,  8.28s/it]
 84%|████████▍ | 31979/37962 [74:02:50<13:50:03,  8.32s/it]
 84%|████████▍ | 31980/37962 [74:02:58<13:53:38,  8.36s/it]
                                                           
{'loss': 1.167, 'grad_norm': 7.650773048400879, 'learning_rate': 1.2743157094997083e-06, 'epoch': 0.84}

 84%|████████▍ | 31980/37962 [74:02:58<13:53:38,  8.36s/it]
 84%|████████▍ | 31981/37962 [74:03:06<13:49:14,  8.32s/it]
 84%|████████▍ | 31982/37962 [74:03:14<13:40:01,  8.23s/it]
 84%|████████▍ | 31983/37962 [74:03:23<13:41:45,  8.25s/it]
 84%|████████▍ | 31984/37962 [74:03:31<13:49:01,  8.32s/it]
 84%|████████▍ | 31985/37962 [74:03:39<13:44:38,  8.28s/it]
 84%|████████▍ | 31986/37962 [74:03:48<13:48:36,  8.32s/it]
 84%|████████▍ | 31987/37962 [74:03:56<13:47:34,  8.31s/it]
 84%|████████▍ | 31988/37962 [74:04:04<13:46:05,  8.30s/it]
 84%|████████▍ | 31989/37962 [74:04:13<13:44:21,  8.28s/it]
 84%|████████▍ | 31990/37962 [74:04:21<13:43:17,  8.27s/it]
                                                           
{'loss': 1.1989, 'grad_norm': 5.722565650939941, 'learning_rate': 1.2701512660427772e-06, 'epoch': 0.84}

 84%|████████▍ | 31990/37962 [74:04:21<13:43:17,  8.27s/it]
 84%|████████▍ | 31991/37962 [74:04:29<13:48:20,  8.32s/it]
 84%|████████▍ | 31992/37962 [74:04:37<13:43:44,  8.28s/it]
 84%|████████▍ | 31993/37962 [74:04:46<13:46:57,  8.31s/it]
 84%|████████▍ | 31994/37962 [74:04:54<13:32:16,  8.17s/it]
 84%|████████▍ | 31995/37962 [74:05:02<13:31:45,  8.16s/it]
 84%|████████▍ | 31996/37962 [74:05:10<13:35:16,  8.20s/it]
 84%|████████▍ | 31997/37962 [74:05:18<13:41:50,  8.27s/it]
 84%|████████▍ | 31998/37962 [74:05:27<13:42:19,  8.27s/it]
 84%|████████▍ | 31999/37962 [74:05:35<13:49:01,  8.34s/it]
 84%|████████▍ | 32000/37962 [74:05:43<13:43:58,  8.29s/it]
                                                           
{'loss': 1.216, 'grad_norm': 6.9415283203125, 'learning_rate': 1.2659931768915001e-06, 'epoch': 0.84}

 84%|████████▍ | 32000/37962 [74:05:43<13:43:58,  8.29s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 84%|████████▍ | 32001/37962 [74:06:34<34:44:31, 20.98s/it]
 84%|████████▍ | 32002/37962 [74:06:42<28:29:08, 17.21s/it]
 84%|████████▍ | 32003/37962 [74:06:51<24:00:13, 14.50s/it]
 84%|████████▍ | 32004/37962 [74:06:59<20:54:49, 12.64s/it]
 84%|████████▍ | 32005/37962 [74:07:07<18:43:41, 11.32s/it]
 84%|████████▍ | 32006/37962 [74:07:15<17:02:10, 10.30s/it]
 84%|████████▍ | 32007/37962 [74:07:23<16:04:02,  9.71s/it]
 84%|████████▍ | 32008/37962 [74:07:32<15:23:08,  9.30s/it]
 84%|████████▍ | 32009/37962 [74:07:40<14:53:05,  9.00s/it]
 84%|████████▍ | 32010/37962 [74:07:48<14:26:41,  8.74s/it]
                                                           
{'loss': 1.1671, 'grad_norm': 7.182684898376465, 'learning_rate': 1.2618414450724825e-06, 'epoch': 0.84}

 84%|████████▍ | 32010/37962 [74:07:48<14:26:41,  8.74s/it]
 84%|████████▍ | 32011/37962 [74:07:56<14:10:31,  8.58s/it]
 84%|████████▍ | 32012/37962 [74:08:05<14:05:19,  8.52s/it]
 84%|████████▍ | 32013/37962 [74:08:13<13:59:00,  8.46s/it]
 84%|████████▍ | 32014/37962 [74:08:22<13:58:22,  8.46s/it]
 84%|████████▍ | 32015/37962 [74:08:30<13:53:39,  8.41s/it]
 84%|████████▍ | 32016/37962 [74:08:38<13:49:00,  8.37s/it]
 84%|████████▍ | 32017/37962 [74:08:46<13:38:53,  8.26s/it]
 84%|████████▍ | 32018/37962 [74:08:54<13:39:57,  8.28s/it]
 84%|████████▍ | 32019/37962 [74:09:03<13:40:22,  8.28s/it]
 84%|████████▍ | 32020/37962 [74:09:11<13:39:09,  8.27s/it]
                                                           
{'loss': 1.1655, 'grad_norm': 9.801552772521973, 'learning_rate': 1.2576960736077016e-06, 'epoch': 0.84}

 84%|████████▍ | 32020/37962 [74:09:11<13:39:09,  8.27s/it]
 84%|████████▍ | 32021/37962 [74:09:20<13:48:23,  8.37s/it]
 84%|████████▍ | 32022/37962 [74:09:28<13:48:35,  8.37s/it]
 84%|████████▍ | 32023/37962 [74:09:36<13:49:53,  8.38s/it]
 84%|████████▍ | 32024/37962 [74:09:45<13:46:21,  8.35s/it]
 84%|████████▍ | 32025/37962 [74:09:53<13:40:47,  8.30s/it]
 84%|████████▍ | 32026/37962 [74:10:01<13:33:59,  8.23s/it]
 84%|████████▍ | 32027/37962 [74:10:09<13:37:44,  8.27s/it]
 84%|████████▍ | 32028/37962 [74:10:18<13:40:15,  8.29s/it]
 84%|████████▍ | 32029/37962 [74:10:26<13:39:47,  8.29s/it]
 84%|████████▍ | 32030/37962 [74:10:34<13:40:56,  8.30s/it]
                                                           
{'loss': 1.2447, 'grad_norm': 9.985540390014648, 'learning_rate': 1.2535570655144969e-06, 'epoch': 0.84}

 84%|████████▍ | 32030/37962 [74:10:34<13:40:56,  8.30s/it]
 84%|████████▍ | 32031/37962 [74:10:42<13:33:29,  8.23s/it]
 84%|████████▍ | 32032/37962 [74:10:50<13:28:38,  8.18s/it]
 84%|████████▍ | 32033/37962 [74:10:59<13:28:40,  8.18s/it]
 84%|████████▍ | 32034/37962 [74:11:07<13:32:18,  8.22s/it]
 84%|████████▍ | 32035/37962 [74:11:15<13:33:50,  8.24s/it]
 84%|████████▍ | 32036/37962 [74:11:24<13:38:19,  8.29s/it]
 84%|████████▍ | 32037/37962 [74:11:32<13:37:03,  8.27s/it]
 84%|████████▍ | 32038/37962 [74:11:40<13:35:18,  8.26s/it]
 84%|████████▍ | 32039/37962 [74:11:48<13:36:33,  8.27s/it]
 84%|████████▍ | 32040/37962 [74:11:56<13:33:39,  8.24s/it]
                                                           
{'loss': 1.137, 'grad_norm': 15.624730110168457, 'learning_rate': 1.249424423805584e-06, 'epoch': 0.84}

 84%|████████▍ | 32040/37962 [74:11:56<13:33:39,  8.24s/it]
 84%|████████▍ | 32041/37962 [74:12:05<13:30:43,  8.22s/it]
 84%|████████▍ | 32042/37962 [74:12:13<13:30:07,  8.21s/it]
 84%|████████▍ | 32043/37962 [74:12:21<13:32:20,  8.23s/it]
 84%|████████▍ | 32044/37962 [74:12:29<13:30:48,  8.22s/it]
 84%|████████▍ | 32045/37962 [74:12:38<13:33:58,  8.25s/it]
 84%|████████▍ | 32046/37962 [74:12:46<13:32:54,  8.24s/it]
 84%|████████▍ | 32047/37962 [74:12:54<13:35:52,  8.28s/it]
 84%|████████▍ | 32048/37962 [74:13:02<13:34:45,  8.27s/it]
 84%|████████▍ | 32049/37962 [74:13:10<13:23:28,  8.15s/it]
 84%|████████▍ | 32050/37962 [74:13:18<13:17:04,  8.09s/it]
                                                           
{'loss': 1.2742, 'grad_norm': 49.671630859375, 'learning_rate': 1.2452981514890405e-06, 'epoch': 0.84}

 84%|████████▍ | 32050/37962 [74:13:18<13:17:04,  8.09s/it]
 84%|████████▍ | 32051/37962 [74:13:26<13:20:11,  8.12s/it]
 84%|████████▍ | 32052/37962 [74:13:35<13:25:41,  8.18s/it]
 84%|████████▍ | 32053/37962 [74:13:43<13:33:38,  8.26s/it]
 84%|████████▍ | 32054/37962 [74:13:51<13:33:43,  8.26s/it]
 84%|████████▍ | 32055/37962 [74:13:59<13:24:47,  8.17s/it]
 84%|████████▍ | 32056/37962 [74:14:08<13:28:59,  8.22s/it]
 84%|████████▍ | 32057/37962 [74:14:16<13:32:23,  8.25s/it]
 84%|████████▍ | 32058/37962 [74:14:25<13:37:00,  8.30s/it]
 84%|████████▍ | 32059/37962 [74:14:33<13:39:26,  8.33s/it]
 84%|████████▍ | 32060/37962 [74:14:41<13:36:13,  8.30s/it]
                                                           
{'loss': 1.1929, 'grad_norm': 11.738286018371582, 'learning_rate': 1.2411782515683045e-06, 'epoch': 0.84}

 84%|████████▍ | 32060/37962 [74:14:41<13:36:13,  8.30s/it]
 84%|████████▍ | 32061/37962 [74:14:49<13:37:22,  8.31s/it]
 84%|████████▍ | 32062/37962 [74:14:58<13:35:04,  8.29s/it]
 84%|████████▍ | 32063/37962 [74:15:06<13:29:36,  8.23s/it]
 84%|████████▍ | 32064/37962 [74:15:14<13:28:42,  8.23s/it]
 84%|████████▍ | 32065/37962 [74:15:22<13:21:26,  8.15s/it]
 84%|████████▍ | 32066/37962 [74:15:30<13:28:57,  8.23s/it]
 84%|████████▍ | 32067/37962 [74:15:39<13:29:53,  8.24s/it]
 84%|████████▍ | 32068/37962 [74:15:47<13:30:28,  8.25s/it]
 84%|████████▍ | 32069/37962 [74:15:55<13:18:56,  8.13s/it]
 84%|████████▍ | 32070/37962 [74:16:03<13:16:49,  8.11s/it]
                                                           
{'loss': 1.1553, 'grad_norm': 9.703991889953613, 'learning_rate': 1.2370647270421831e-06, 'epoch': 0.84}

 84%|████████▍ | 32070/37962 [74:16:03<13:16:49,  8.11s/it]
 84%|████████▍ | 32071/37962 [74:16:11<13:20:36,  8.15s/it]
 84%|████████▍ | 32072/37962 [74:16:19<13:17:49,  8.13s/it]
 84%|████████▍ | 32073/37962 [74:16:28<13:22:21,  8.17s/it]
 84%|████████▍ | 32074/37962 [74:16:36<13:27:56,  8.23s/it]
 84%|████████▍ | 32075/37962 [74:16:44<13:26:49,  8.22s/it]
 84%|████████▍ | 32076/37962 [74:16:52<13:30:14,  8.26s/it]
 84%|████████▍ | 32077/37962 [74:17:00<13:20:35,  8.16s/it]
 85%|████████▍ | 32078/37962 [74:17:09<13:29:08,  8.25s/it]
 85%|████████▍ | 32079/37962 [74:17:17<13:30:06,  8.26s/it]
 85%|████████▍ | 32080/37962 [74:17:26<13:34:26,  8.31s/it]
                                                           
{'loss': 1.1089, 'grad_norm': 8.218047142028809, 'learning_rate': 1.2329575809048423e-06, 'epoch': 0.85}

 85%|████████▍ | 32080/37962 [74:17:26<13:34:26,  8.31s/it]
 85%|████████▍ | 32081/37962 [74:17:34<13:33:03,  8.30s/it]
 85%|████████▍ | 32082/37962 [74:17:42<13:27:32,  8.24s/it]
 85%|████████▍ | 32083/37962 [74:17:50<13:30:32,  8.27s/it]
 85%|████████▍ | 32084/37962 [74:17:58<13:20:05,  8.17s/it]
 85%|████████▍ | 32085/37962 [74:18:07<13:29:54,  8.27s/it]
 85%|████████▍ | 32086/37962 [74:18:15<13:28:35,  8.26s/it]
 85%|████████▍ | 32087/37962 [74:18:24<13:40:43,  8.38s/it]
 85%|████████▍ | 32088/37962 [74:18:32<13:44:05,  8.42s/it]
 85%|████████▍ | 32089/37962 [74:18:40<13:30:35,  8.28s/it]
 85%|████████▍ | 32090/37962 [74:18:48<13:30:12,  8.28s/it]
                                                           
{'loss': 1.0829, 'grad_norm': 10.105463981628418, 'learning_rate': 1.2288568161457992e-06, 'epoch': 0.85}

 85%|████████▍ | 32090/37962 [74:18:48<13:30:12,  8.28s/it]
 85%|████████▍ | 32091/37962 [74:18:57<13:35:38,  8.34s/it]
 85%|████████▍ | 32092/37962 [74:19:05<13:39:20,  8.37s/it]
 85%|████████▍ | 32093/37962 [74:19:14<13:35:52,  8.34s/it]
 85%|████████▍ | 32094/37962 [74:19:22<13:34:36,  8.33s/it]
 85%|████████▍ | 32095/37962 [74:19:30<13:33:06,  8.32s/it]
 85%|████████▍ | 32096/37962 [74:19:39<13:38:31,  8.37s/it]
 85%|████████▍ | 32097/37962 [74:19:47<13:27:18,  8.26s/it]
 85%|████████▍ | 32098/37962 [74:19:54<13:16:01,  8.14s/it]
 85%|████████▍ | 32099/37962 [74:20:03<13:22:33,  8.21s/it]
 85%|████████▍ | 32100/37962 [74:20:11<13:25:54,  8.25s/it]
                                                           
{'loss': 1.0326, 'grad_norm': 14.89763355255127, 'learning_rate': 1.2247624357499288e-06, 'epoch': 0.85}

 85%|████████▍ | 32100/37962 [74:20:11<13:25:54,  8.25s/it]
 85%|████████▍ | 32101/37962 [74:20:20<13:29:41,  8.29s/it]
 85%|████████▍ | 32102/37962 [74:20:28<13:21:44,  8.21s/it]
 85%|████████▍ | 32103/37962 [74:20:36<13:24:16,  8.24s/it]
 85%|████████▍ | 32104/37962 [74:20:44<13:27:53,  8.27s/it]
 85%|████████▍ | 32105/37962 [74:20:53<13:28:57,  8.29s/it]
 85%|████████▍ | 32106/37962 [74:21:01<13:27:20,  8.27s/it]
 85%|████████▍ | 32107/37962 [74:21:09<13:26:36,  8.27s/it]
 85%|████████▍ | 32108/37962 [74:21:17<13:22:22,  8.22s/it]
 85%|████████▍ | 32109/37962 [74:21:25<13:23:29,  8.24s/it]
 85%|████████▍ | 32110/37962 [74:21:34<13:26:16,  8.27s/it]
                                                           
{'loss': 1.1758, 'grad_norm': 11.415549278259277, 'learning_rate': 1.2206744426974616e-06, 'epoch': 0.85}

 85%|████████▍ | 32110/37962 [74:21:34<13:26:16,  8.27s/it]
 85%|████████▍ | 32111/37962 [74:21:42<13:26:20,  8.27s/it]
 85%|████████▍ | 32112/37962 [74:21:50<13:30:46,  8.32s/it]
 85%|████████▍ | 32113/37962 [74:21:59<13:26:01,  8.27s/it]
 85%|████████▍ | 32114/37962 [74:22:07<13:30:03,  8.31s/it]
 85%|████████▍ | 32115/37962 [74:22:15<13:14:32,  8.15s/it]
 85%|████████▍ | 32116/37962 [74:22:23<13:16:47,  8.18s/it]
 85%|████████▍ | 32117/37962 [74:22:32<13:24:26,  8.26s/it]
 85%|████████▍ | 32118/37962 [74:22:40<13:20:00,  8.21s/it]
 85%|████████▍ | 32119/37962 [74:22:48<13:19:44,  8.21s/it]
 85%|████████▍ | 32120/37962 [74:22:56<13:22:42,  8.24s/it]
                                                           
{'loss': 1.1349, 'grad_norm': 9.858724594116211, 'learning_rate': 1.2165928399639748e-06, 'epoch': 0.85}

 85%|████████▍ | 32120/37962 [74:22:56<13:22:42,  8.24s/it]
 85%|████████▍ | 32121/37962 [74:23:04<13:24:43,  8.27s/it]
 85%|████████▍ | 32122/37962 [74:23:13<13:22:36,  8.25s/it]
 85%|████████▍ | 32123/37962 [74:23:21<13:17:54,  8.20s/it]
 85%|████████▍ | 32124/37962 [74:23:28<13:02:20,  8.04s/it]
 85%|████████▍ | 32125/37962 [74:23:37<13:04:36,  8.07s/it]
 85%|████████▍ | 32126/37962 [74:23:45<13:08:57,  8.11s/it]
 85%|████████▍ | 32127/37962 [74:23:53<13:10:54,  8.13s/it]
 85%|████████▍ | 32128/37962 [74:24:01<13:07:03,  8.09s/it]
 85%|████████▍ | 32129/37962 [74:24:09<13:09:35,  8.12s/it]
 85%|████████▍ | 32130/37962 [74:24:17<13:13:06,  8.16s/it]
                                                           
{'loss': 1.0875, 'grad_norm': 9.781495094299316, 'learning_rate': 1.2125176305204001e-06, 'epoch': 0.85}

 85%|████████▍ | 32130/37962 [74:24:17<13:13:06,  8.16s/it]
 85%|████████▍ | 32131/37962 [74:24:26<13:13:48,  8.17s/it]
 85%|████████▍ | 32132/37962 [74:24:34<13:18:04,  8.21s/it]
 85%|████████▍ | 32133/37962 [74:24:42<13:22:29,  8.26s/it]
 85%|████████▍ | 32134/37962 [74:24:51<13:22:47,  8.26s/it]
 85%|████████▍ | 32135/37962 [74:24:59<13:20:39,  8.24s/it]
 85%|████████▍ | 32136/37962 [74:25:07<13:23:14,  8.27s/it]
 85%|████████▍ | 32137/37962 [74:25:15<13:21:11,  8.25s/it]
 85%|████████▍ | 32138/37962 [74:25:23<13:15:18,  8.19s/it]
 85%|████████▍ | 32139/37962 [74:25:32<13:17:14,  8.21s/it]
 85%|████████▍ | 32140/37962 [74:25:40<13:18:34,  8.23s/it]
                                                           
{'loss': 1.1417, 'grad_norm': 12.387809753417969, 'learning_rate': 1.208448817333009e-06, 'epoch': 0.85}

 85%|████████▍ | 32140/37962 [74:25:40<13:18:34,  8.23s/it]
 85%|████████▍ | 32141/37962 [74:25:48<13:17:42,  8.22s/it]
 85%|████████▍ | 32142/37962 [74:25:56<13:20:47,  8.26s/it]
 85%|████████▍ | 32143/37962 [74:26:05<13:20:16,  8.25s/it]
 85%|████████▍ | 32144/37962 [74:26:13<13:18:28,  8.23s/it]
 85%|████████▍ | 32145/37962 [74:26:21<13:17:42,  8.23s/it]
 85%|████████▍ | 32146/37962 [74:26:29<13:22:06,  8.27s/it]
 85%|████████▍ | 32147/37962 [74:26:38<13:20:59,  8.26s/it]
 85%|████████▍ | 32148/37962 [74:26:46<13:20:02,  8.26s/it]
 85%|████████▍ | 32149/37962 [74:26:54<13:21:23,  8.27s/it]
 85%|████████▍ | 32150/37962 [74:27:03<13:22:49,  8.29s/it]
                                                           
{'loss': 1.1496, 'grad_norm': 14.604401588439941, 'learning_rate': 1.2043864033634178e-06, 'epoch': 0.85}

 85%|████████▍ | 32150/37962 [74:27:03<13:22:49,  8.29s/it]
 85%|████████▍ | 32151/37962 [74:27:11<13:22:15,  8.28s/it]
 85%|████████▍ | 32152/37962 [74:27:19<13:20:15,  8.26s/it]
 85%|████████▍ | 32153/37962 [74:27:27<13:24:29,  8.31s/it]
 85%|████████▍ | 32154/37962 [74:27:36<13:31:49,  8.39s/it]
 85%|████████▍ | 32155/37962 [74:27:44<13:27:59,  8.35s/it]
 85%|████████▍ | 32156/37962 [74:27:53<13:24:26,  8.31s/it]
 85%|████████▍ | 32157/37962 [74:28:01<13:16:43,  8.23s/it]
 85%|████████▍ | 32158/37962 [74:28:09<13:17:53,  8.25s/it]
 85%|████████▍ | 32159/37962 [74:28:17<13:21:42,  8.29s/it]
 85%|████████▍ | 32160/37962 [74:28:25<13:19:19,  8.27s/it]
                                                           
{'loss': 1.0867, 'grad_norm': 18.11266326904297, 'learning_rate': 1.2003303915685937e-06, 'epoch': 0.85}

 85%|████████▍ | 32160/37962 [74:28:25<13:19:19,  8.27s/it]
 85%|████████▍ | 32161/37962 [74:28:34<13:19:54,  8.27s/it]
 85%|████████▍ | 32162/37962 [74:28:42<13:18:05,  8.26s/it]
 85%|████████▍ | 32163/37962 [74:28:50<13:20:52,  8.29s/it]
 85%|████████▍ | 32164/37962 [74:28:59<13:19:20,  8.27s/it]
 85%|████████▍ | 32165/37962 [74:29:07<13:20:18,  8.28s/it]
 85%|████████▍ | 32166/37962 [74:29:15<13:13:25,  8.21s/it]
 85%|████████▍ | 32167/37962 [74:29:23<13:15:45,  8.24s/it]
 85%|████████▍ | 32168/37962 [74:29:31<13:15:25,  8.24s/it]
 85%|████████▍ | 32169/37962 [74:29:40<13:18:58,  8.28s/it]
 85%|████████▍ | 32170/37962 [74:29:48<13:15:21,  8.24s/it]
                                                           
{'loss': 1.1521, 'grad_norm': 6.647791385650635, 'learning_rate': 1.1962807849008306e-06, 'epoch': 0.85}

 85%|████████▍ | 32170/37962 [74:29:48<13:15:21,  8.24s/it]
 85%|████████▍ | 32171/37962 [74:29:56<13:19:14,  8.28s/it]
 85%|████████▍ | 32172/37962 [74:30:04<13:15:19,  8.24s/it]
 85%|████████▍ | 32173/37962 [74:30:13<13:17:24,  8.26s/it]
 85%|████████▍ | 32174/37962 [74:30:21<13:23:21,  8.33s/it]
 85%|████████▍ | 32175/37962 [74:30:30<13:21:38,  8.31s/it]
 85%|████████▍ | 32176/37962 [74:30:38<13:22:12,  8.32s/it]
 85%|████████▍ | 32177/37962 [74:30:46<13:13:44,  8.23s/it]
 85%|████████▍ | 32178/37962 [74:30:54<13:16:22,  8.26s/it]
 85%|████████▍ | 32179/37962 [74:31:03<13:19:03,  8.29s/it]
 85%|████████▍ | 32180/37962 [74:31:11<13:19:54,  8.30s/it]
                                                           
{'loss': 1.1745, 'grad_norm': 8.048619270324707, 'learning_rate': 1.1922375863077707e-06, 'epoch': 0.85}

 85%|████████▍ | 32180/37962 [74:31:11<13:19:54,  8.30s/it]
 85%|████████▍ | 32181/37962 [74:31:19<13:18:46,  8.29s/it]
 85%|████████▍ | 32182/37962 [74:31:27<13:18:03,  8.28s/it]
 85%|████████▍ | 32183/37962 [74:31:36<13:13:14,  8.24s/it]
 85%|████████▍ | 32184/37962 [74:31:44<13:22:23,  8.33s/it]
 85%|████████▍ | 32185/37962 [74:31:52<13:22:13,  8.33s/it]
 85%|████████▍ | 32186/37962 [74:32:01<13:15:19,  8.26s/it]
 85%|████████▍ | 32187/37962 [74:32:09<13:19:19,  8.30s/it]
 85%|████████▍ | 32188/37962 [74:32:17<13:22:13,  8.34s/it]
 85%|████████▍ | 32189/37962 [74:32:26<13:24:05,  8.36s/it]
 85%|████████▍ | 32190/37962 [74:32:34<13:25:39,  8.37s/it]
                                                           
{'loss': 1.1516, 'grad_norm': 6.168464660644531, 'learning_rate': 1.1882007987323886e-06, 'epoch': 0.85}

 85%|████████▍ | 32190/37962 [74:32:34<13:25:39,  8.37s/it]
 85%|████████▍ | 32191/37962 [74:32:42<13:21:52,  8.34s/it]
 85%|████████▍ | 32192/37962 [74:32:51<13:25:20,  8.37s/it]
 85%|████████▍ | 32193/37962 [74:32:59<13:27:00,  8.39s/it]
 85%|████████▍ | 32194/37962 [74:33:08<13:25:54,  8.38s/it]
 85%|████████▍ | 32195/37962 [74:33:16<13:23:44,  8.36s/it]
 85%|████████▍ | 32196/37962 [74:33:24<13:18:25,  8.31s/it]
 85%|████████▍ | 32197/37962 [74:33:32<13:14:57,  8.27s/it]
 85%|████████▍ | 32198/37962 [74:33:41<13:12:08,  8.25s/it]
 85%|████████▍ | 32199/37962 [74:33:48<13:02:12,  8.14s/it]
 85%|████████▍ | 32200/37962 [74:33:57<13:05:00,  8.17s/it]
                                                           
{'loss': 1.0721, 'grad_norm': 6.283260822296143, 'learning_rate': 1.1841704251129927e-06, 'epoch': 0.85}

 85%|████████▍ | 32200/37962 [74:33:57<13:05:00,  8.17s/it]
 85%|████████▍ | 32201/37962 [74:34:05<13:09:41,  8.22s/it]
 85%|████████▍ | 32202/37962 [74:34:13<12:59:49,  8.12s/it]
 85%|████████▍ | 32203/37962 [74:34:21<13:01:32,  8.14s/it]
 85%|████████▍ | 32204/37962 [74:34:29<13:06:59,  8.20s/it]
 85%|████████▍ | 32205/37962 [74:34:38<13:06:54,  8.20s/it]
 85%|████████▍ | 32206/37962 [74:34:45<12:53:51,  8.07s/it]
 85%|████████▍ | 32207/37962 [74:34:53<12:50:15,  8.03s/it]
 85%|████████▍ | 32208/37962 [74:35:02<13:04:12,  8.18s/it]
 85%|████████▍ | 32209/37962 [74:35:10<13:05:36,  8.19s/it]
 85%|████████▍ | 32210/37962 [74:35:18<13:08:51,  8.23s/it]
                                                           
{'loss': 1.0989, 'grad_norm': 16.147626876831055, 'learning_rate': 1.1801464683832175e-06, 'epoch': 0.85}

 85%|████████▍ | 32210/37962 [74:35:18<13:08:51,  8.23s/it]
 85%|████████▍ | 32211/37962 [74:35:27<13:09:06,  8.23s/it]
 85%|████████▍ | 32212/37962 [74:35:35<13:18:21,  8.33s/it]
 85%|████████▍ | 32213/37962 [74:35:43<13:08:39,  8.23s/it]
 85%|████████▍ | 32214/37962 [74:35:52<13:14:39,  8.29s/it]
 85%|████████▍ | 32215/37962 [74:36:00<13:20:11,  8.35s/it]
 85%|████████▍ | 32216/37962 [74:36:09<13:18:58,  8.34s/it]
 85%|████████▍ | 32217/37962 [74:36:17<13:19:51,  8.35s/it]
 85%|████████▍ | 32218/37962 [74:36:25<13:18:11,  8.34s/it]
 85%|████████▍ | 32219/37962 [74:36:34<13:18:44,  8.34s/it]
 85%|████████▍ | 32220/37962 [74:36:42<13:20:49,  8.37s/it]
                                                           
{'loss': 1.1807, 'grad_norm': 9.637327194213867, 'learning_rate': 1.176128931472037e-06, 'epoch': 0.85}

 85%|████████▍ | 32220/37962 [74:36:42<13:20:49,  8.37s/it]
 85%|████████▍ | 32221/37962 [74:36:50<13:16:59,  8.33s/it]
 85%|████████▍ | 32222/37962 [74:36:58<13:15:47,  8.32s/it]
 85%|████████▍ | 32223/37962 [74:37:07<13:16:18,  8.33s/it]
 85%|████████▍ | 32224/37962 [74:37:15<13:08:36,  8.25s/it]
 85%|████████▍ | 32225/37962 [74:37:23<13:07:29,  8.24s/it]
 85%|████████▍ | 32226/37962 [74:37:31<12:59:53,  8.16s/it]
 85%|████████▍ | 32227/37962 [74:37:39<12:58:06,  8.14s/it]
 85%|████████▍ | 32228/37962 [74:37:47<12:56:40,  8.13s/it]
 85%|████████▍ | 32229/37962 [74:37:55<12:56:39,  8.13s/it]
 85%|████████▍ | 32230/37962 [74:38:04<13:00:09,  8.17s/it]
                                                           
{'loss': 1.2392, 'grad_norm': 7.452266216278076, 'learning_rate': 1.1721178173037417e-06, 'epoch': 0.85}

 85%|████████▍ | 32230/37962 [74:38:04<13:00:09,  8.17s/it]
 85%|████████▍ | 32231/37962 [74:38:12<13:04:35,  8.21s/it]
 85%|████████▍ | 32232/37962 [74:38:20<13:04:05,  8.21s/it]
 85%|████████▍ | 32233/37962 [74:38:28<13:05:50,  8.23s/it]
 85%|████████▍ | 32234/37962 [74:38:37<13:07:52,  8.25s/it]
 85%|████████▍ | 32235/37962 [74:38:45<13:06:18,  8.24s/it]
 85%|████████▍ | 32236/37962 [74:38:53<12:57:19,  8.15s/it]
 85%|████████▍ | 32237/37962 [74:39:01<12:59:52,  8.17s/it]
 85%|████████▍ | 32238/37962 [74:39:10<13:04:58,  8.23s/it]
 85%|████████▍ | 32239/37962 [74:39:18<13:07:43,  8.26s/it]
 85%|████████▍ | 32240/37962 [74:39:26<13:14:50,  8.33s/it]
                                                           
{'loss': 1.1803, 'grad_norm': 13.176429748535156, 'learning_rate': 1.1681131287979563e-06, 'epoch': 0.85}

 85%|████████▍ | 32240/37962 [74:39:26<13:14:50,  8.33s/it]
 85%|████████▍ | 32241/37962 [74:39:35<13:11:37,  8.30s/it]
 85%|████████▍ | 32242/37962 [74:39:43<13:16:12,  8.35s/it]
 85%|████████▍ | 32243/37962 [74:39:51<13:16:36,  8.36s/it]
 85%|████████▍ | 32244/37962 [74:40:00<13:16:45,  8.36s/it]
 85%|████████▍ | 32245/37962 [74:40:08<13:13:37,  8.33s/it]
 85%|████████▍ | 32246/37962 [74:40:16<13:06:04,  8.25s/it]
 85%|████████▍ | 32247/37962 [74:40:24<13:04:43,  8.24s/it]
 85%|████████▍ | 32248/37962 [74:40:32<12:57:46,  8.17s/it]
 85%|████████▍ | 32249/37962 [74:40:40<12:51:31,  8.10s/it]
 85%|████████▍ | 32250/37962 [74:40:48<12:47:24,  8.06s/it]
                                                           
{'loss': 1.1163, 'grad_norm': 7.590648651123047, 'learning_rate': 1.1641148688696235e-06, 'epoch': 0.85}

 85%|████████▍ | 32250/37962 [74:40:48<12:47:24,  8.06s/it]
 85%|████████▍ | 32251/37962 [74:40:56<12:53:09,  8.12s/it]
 85%|████████▍ | 32252/37962 [74:41:05<12:50:03,  8.09s/it]
 85%|████████▍ | 32253/37962 [74:41:13<12:48:09,  8.07s/it]
 85%|████████▍ | 32254/37962 [74:41:21<12:56:38,  8.16s/it]
 85%|████████▍ | 32255/37962 [74:41:29<13:08:15,  8.29s/it]
 85%|████████▍ | 32256/37962 [74:41:38<13:01:48,  8.22s/it]
 85%|████████▍ | 32257/37962 [74:41:46<12:58:46,  8.19s/it]
 85%|████████▍ | 32258/37962 [74:41:54<13:00:19,  8.21s/it]
 85%|████████▍ | 32259/37962 [74:42:02<13:08:42,  8.30s/it]
 85%|████████▍ | 32260/37962 [74:42:11<13:08:25,  8.30s/it]
                                                           
{'loss': 1.1477, 'grad_norm': 17.394153594970703, 'learning_rate': 1.1601230404290042e-06, 'epoch': 0.85}

 85%|████████▍ | 32260/37962 [74:42:11<13:08:25,  8.30s/it]
 85%|████████▍ | 32261/37962 [74:42:19<13:10:11,  8.32s/it]
 85%|████████▍ | 32262/37962 [74:42:27<13:10:37,  8.32s/it]
 85%|████████▍ | 32263/37962 [74:42:36<13:10:13,  8.32s/it]
 85%|████████▍ | 32264/37962 [74:42:44<13:10:59,  8.33s/it]
 85%|████████▍ | 32265/37962 [74:42:52<13:04:05,  8.26s/it]
 85%|████████▍ | 32266/37962 [74:43:00<13:02:51,  8.25s/it]
 85%|████████▍ | 32267/37962 [74:43:09<13:04:54,  8.27s/it]
 85%|████████▌ | 32268/37962 [74:43:17<12:57:43,  8.20s/it]
 85%|████████▌ | 32269/37962 [74:43:25<13:01:56,  8.24s/it]
 85%|████████▌ | 32270/37962 [74:43:33<13:05:25,  8.28s/it]
                                                           
{'loss': 1.1167, 'grad_norm': 8.838793754577637, 'learning_rate': 1.1561376463816843e-06, 'epoch': 0.85}

 85%|████████▌ | 32270/37962 [74:43:33<13:05:25,  8.28s/it]
 85%|████████▌ | 32271/37962 [74:43:42<13:04:42,  8.27s/it]
 85%|████████▌ | 32272/37962 [74:43:50<13:07:11,  8.30s/it]
 85%|████████▌ | 32273/37962 [74:43:58<12:55:55,  8.18s/it]
 85%|████████▌ | 32274/37962 [74:44:06<12:59:51,  8.23s/it]
 85%|████████▌ | 32275/37962 [74:44:15<13:00:24,  8.23s/it]
 85%|████████▌ | 32276/37962 [74:44:23<12:58:12,  8.21s/it]
 85%|████████▌ | 32277/37962 [74:44:31<13:01:02,  8.24s/it]
 85%|████████▌ | 32278/37962 [74:44:39<13:01:51,  8.25s/it]
 85%|████████▌ | 32279/37962 [74:44:48<13:03:12,  8.27s/it]
 85%|████████▌ | 32280/37962 [74:44:56<13:03:26,  8.27s/it]
                                                           
{'loss': 1.1269, 'grad_norm': 8.388134002685547, 'learning_rate': 1.1521586896285608e-06, 'epoch': 0.85}

 85%|████████▌ | 32280/37962 [74:44:56<13:03:26,  8.27s/it]
 85%|████████▌ | 32281/37962 [74:45:04<12:54:53,  8.18s/it]
 85%|████████▌ | 32282/37962 [74:45:12<12:57:00,  8.21s/it]
 85%|████████▌ | 32283/37962 [74:45:20<12:51:34,  8.15s/it]
 85%|████████▌ | 32284/37962 [74:45:28<12:53:27,  8.17s/it]
 85%|████████▌ | 32285/37962 [74:45:37<12:59:12,  8.24s/it]
 85%|████████▌ | 32286/37962 [74:45:45<13:04:30,  8.29s/it]
 85%|████████▌ | 32287/37962 [74:45:53<13:00:34,  8.25s/it]
 85%|████████▌ | 32288/37962 [74:46:01<12:48:55,  8.13s/it]
 85%|████████▌ | 32289/37962 [74:46:09<12:47:30,  8.12s/it]
 85%|████████▌ | 32290/37962 [74:46:18<12:53:33,  8.18s/it]
                                                           
{'loss': 1.1771, 'grad_norm': 6.998394966125488, 'learning_rate': 1.1481861730658494e-06, 'epoch': 0.85}

 85%|████████▌ | 32290/37962 [74:46:18<12:53:33,  8.18s/it]
 85%|████████▌ | 32291/37962 [74:46:26<12:50:07,  8.15s/it]
 85%|████████▌ | 32292/37962 [74:46:34<12:51:30,  8.16s/it]
 85%|████████▌ | 32293/37962 [74:46:42<12:46:39,  8.11s/it]
 85%|████████▌ | 32294/37962 [74:46:50<12:51:24,  8.17s/it]
 85%|████████▌ | 32295/37962 [74:46:58<12:54:38,  8.20s/it]
 85%|████████▌ | 32296/37962 [74:47:07<13:04:46,  8.31s/it]
 85%|████████▌ | 32297/37962 [74:47:15<12:58:02,  8.24s/it]
 85%|████████▌ | 32298/37962 [74:47:23<12:57:37,  8.24s/it]
 85%|████████▌ | 32299/37962 [74:47:32<12:57:22,  8.24s/it]
 85%|████████▌ | 32300/37962 [74:47:40<13:00:11,  8.27s/it]
                                                           
{'loss': 1.0813, 'grad_norm': 9.368738174438477, 'learning_rate': 1.1442200995850772e-06, 'epoch': 0.85}

 85%|████████▌ | 32300/37962 [74:47:40<13:00:11,  8.27s/it]
 85%|████████▌ | 32301/37962 [74:47:48<12:50:06,  8.16s/it]
 85%|████████▌ | 32302/37962 [74:47:56<12:51:08,  8.17s/it]
 85%|████████▌ | 32303/37962 [74:48:04<12:48:52,  8.15s/it]
 85%|████████▌ | 32304/37962 [74:48:12<12:44:01,  8.10s/it]
 85%|████████▌ | 32305/37962 [74:48:20<12:47:43,  8.14s/it]
 85%|████████▌ | 32306/37962 [74:48:29<12:52:34,  8.20s/it]
 85%|████████▌ | 32307/37962 [74:48:37<12:54:54,  8.22s/it]
 85%|████████▌ | 32308/37962 [74:48:45<12:54:58,  8.22s/it]
 85%|████████▌ | 32309/37962 [74:48:53<12:51:05,  8.18s/it]
 85%|████████▌ | 32310/37962 [74:49:02<12:53:38,  8.21s/it]
                                                           
{'loss': 1.1932, 'grad_norm': 14.990349769592285, 'learning_rate': 1.140260472073078e-06, 'epoch': 0.85}

 85%|████████▌ | 32310/37962 [74:49:02<12:53:38,  8.21s/it]
 85%|████████▌ | 32311/37962 [74:49:10<12:55:18,  8.23s/it]
 85%|████████▌ | 32312/37962 [74:49:18<12:55:27,  8.23s/it]
 85%|████████▌ | 32313/37962 [74:49:26<12:55:38,  8.24s/it]
 85%|████████▌ | 32314/37962 [74:49:35<12:59:00,  8.28s/it]
 85%|████████▌ | 32315/37962 [74:49:43<12:59:10,  8.28s/it]
 85%|████████▌ | 32316/37962 [74:49:51<12:54:04,  8.23s/it]
 85%|████████▌ | 32317/37962 [74:49:59<12:56:49,  8.26s/it]
 85%|████████▌ | 32318/37962 [74:50:08<12:59:46,  8.29s/it]
 85%|████████▌ | 32319/37962 [74:50:16<12:58:49,  8.28s/it]
 85%|████████▌ | 32320/37962 [74:50:24<13:00:19,  8.30s/it]
                                                           
{'loss': 1.1258, 'grad_norm': 15.991227149963379, 'learning_rate': 1.1363072934119957e-06, 'epoch': 0.85}

 85%|████████▌ | 32320/37962 [74:50:24<13:00:19,  8.30s/it]
 85%|████████▌ | 32321/37962 [74:50:33<12:57:36,  8.27s/it]
 85%|████████▌ | 32322/37962 [74:50:41<12:59:29,  8.29s/it]
 85%|████████▌ | 32323/37962 [74:50:49<12:57:05,  8.27s/it]
 85%|████████▌ | 32324/37962 [74:50:57<12:57:53,  8.28s/it]
 85%|████████▌ | 32325/37962 [74:51:06<12:58:33,  8.29s/it]
 85%|████████▌ | 32326/37962 [74:51:14<12:53:06,  8.23s/it]
 85%|████████▌ | 32327/37962 [74:51:22<12:51:45,  8.22s/it]
 85%|████████▌ | 32328/37962 [74:51:30<12:52:58,  8.23s/it]
 85%|████████▌ | 32329/37962 [74:51:39<12:56:14,  8.27s/it]
 85%|████████▌ | 32330/37962 [74:51:47<13:02:40,  8.34s/it]
                                                           
{'loss': 1.1251, 'grad_norm': 8.31185245513916, 'learning_rate': 1.1323605664792824e-06, 'epoch': 0.85}

 85%|████████▌ | 32330/37962 [74:51:47<13:02:40,  8.34s/it]
 85%|████████▌ | 32331/37962 [74:52:03<16:29:47, 10.55s/it]
 85%|████████▌ | 32332/37962 [74:52:11<15:24:00,  9.85s/it]
 85%|████████▌ | 32333/37962 [74:52:20<14:45:20,  9.44s/it]
 85%|████████▌ | 32334/37962 [74:52:28<14:05:28,  9.01s/it]
 85%|████████▌ | 32335/37962 [74:52:36<13:38:25,  8.73s/it]
 85%|████████▌ | 32336/37962 [74:52:44<13:27:30,  8.61s/it]
 85%|████████▌ | 32337/37962 [74:52:52<13:15:35,  8.49s/it]
 85%|████████▌ | 32338/37962 [74:53:00<13:09:10,  8.42s/it]
 85%|████████▌ | 32339/37962 [74:53:09<13:05:09,  8.38s/it]
 85%|████████▌ | 32340/37962 [74:53:17<13:03:53,  8.37s/it]
                                                           
{'loss': 1.27, 'grad_norm': 17.082847595214844, 'learning_rate': 1.1284202941476908e-06, 'epoch': 0.85}

 85%|████████▌ | 32340/37962 [74:53:17<13:03:53,  8.37s/it]
 85%|████████▌ | 32341/37962 [74:53:25<12:54:34,  8.27s/it]
 85%|████████▌ | 32342/37962 [74:53:33<12:49:27,  8.21s/it]
 85%|████████▌ | 32343/37962 [74:53:42<12:54:59,  8.28s/it]
 85%|████████▌ | 32344/37962 [74:53:49<12:43:04,  8.15s/it]
 85%|████████▌ | 32345/37962 [74:53:58<12:42:15,  8.14s/it]
 85%|████████▌ | 32346/37962 [74:54:06<12:47:53,  8.20s/it]
 85%|████████▌ | 32347/37962 [74:54:14<12:47:25,  8.20s/it]
 85%|████████▌ | 32348/37962 [74:54:22<12:47:55,  8.21s/it]
 85%|████████▌ | 32349/37962 [74:54:31<12:48:18,  8.21s/it]
 85%|████████▌ | 32350/37962 [74:54:39<12:42:36,  8.15s/it]
                                                           
{'loss': 1.2204, 'grad_norm': 8.107457160949707, 'learning_rate': 1.1244864792852783e-06, 'epoch': 0.85}

 85%|████████▌ | 32350/37962 [74:54:39<12:42:36,  8.15s/it]
 85%|████████▌ | 32351/37962 [74:54:47<12:43:54,  8.17s/it]
 85%|████████▌ | 32352/37962 [74:54:55<12:44:11,  8.17s/it]
 85%|████████▌ | 32353/37962 [74:55:03<12:48:53,  8.22s/it]
 85%|████████▌ | 32354/37962 [74:55:11<12:44:04,  8.17s/it]
 85%|████████▌ | 32355/37962 [74:55:20<12:48:05,  8.22s/it]
 85%|████████▌ | 32356/37962 [74:55:28<12:48:02,  8.22s/it]
 85%|████████▌ | 32357/37962 [74:55:36<12:55:41,  8.30s/it]
 85%|████████▌ | 32358/37962 [74:55:45<12:53:31,  8.28s/it]
 85%|████████▌ | 32359/37962 [74:55:53<12:49:51,  8.24s/it]
 85%|████████▌ | 32360/37962 [74:56:01<12:45:11,  8.20s/it]
                                                           
{'loss': 1.1258, 'grad_norm': 9.99099063873291, 'learning_rate': 1.1205591247554004e-06, 'epoch': 0.85}

 85%|████████▌ | 32360/37962 [74:56:01<12:45:11,  8.20s/it]
 85%|████████▌ | 32361/37962 [74:56:09<12:49:16,  8.24s/it]
 85%|████████▌ | 32362/37962 [74:56:17<12:47:20,  8.22s/it]
 85%|████████▌ | 32363/37962 [74:56:25<12:41:16,  8.16s/it]
 85%|████████▌ | 32364/37962 [74:56:34<12:48:27,  8.24s/it]
 85%|████████▌ | 32365/37962 [74:56:42<12:58:35,  8.35s/it]
 85%|████████▌ | 32366/37962 [74:56:51<13:02:23,  8.39s/it]
 85%|████████▌ | 32367/37962 [74:56:59<13:00:08,  8.37s/it]
 85%|████████▌ | 32368/37962 [74:57:08<12:58:18,  8.35s/it]
 85%|████████▌ | 32369/37962 [74:57:16<12:58:10,  8.35s/it]
 85%|████████▌ | 32370/37962 [74:57:24<12:56:37,  8.33s/it]
                                                           
{'loss': 1.1274, 'grad_norm': 5.88415002822876, 'learning_rate': 1.1166382334167081e-06, 'epoch': 0.85}

 85%|████████▌ | 32370/37962 [74:57:24<12:56:37,  8.33s/it]
 85%|████████▌ | 32371/37962 [74:57:32<12:42:59,  8.19s/it]
 85%|████████▌ | 32372/37962 [74:57:40<12:39:15,  8.15s/it]
 85%|████████▌ | 32373/37962 [74:57:48<12:38:33,  8.14s/it]
 85%|████████▌ | 32374/37962 [74:57:57<12:48:54,  8.26s/it]
 85%|████████▌ | 32375/37962 [74:58:05<12:51:21,  8.28s/it]
 85%|████████▌ | 32376/37962 [74:58:14<12:59:23,  8.37s/it]
 85%|████████▌ | 32377/37962 [74:58:22<13:00:05,  8.38s/it]
 85%|████████▌ | 32378/37962 [74:58:30<12:50:36,  8.28s/it]
 85%|████████▌ | 32379/37962 [74:58:38<12:48:26,  8.26s/it]
 85%|████████▌ | 32380/37962 [74:58:47<12:49:09,  8.27s/it]
                                                           
{'loss': 1.1829, 'grad_norm': 7.295480728149414, 'learning_rate': 1.1127238081231517e-06, 'epoch': 0.85}

 85%|████████▌ | 32380/37962 [74:58:47<12:49:09,  8.27s/it]
 85%|████████▌ | 32381/37962 [74:58:55<12:49:21,  8.27s/it]
 85%|████████▌ | 32382/37962 [74:59:03<12:45:21,  8.23s/it]
 85%|████████▌ | 32383/37962 [74:59:11<12:45:39,  8.23s/it]
 85%|████████▌ | 32384/37962 [74:59:20<12:46:04,  8.24s/it]
 85%|████████▌ | 32385/37962 [74:59:28<12:45:43,  8.24s/it]
 85%|████████▌ | 32386/37962 [74:59:36<12:47:25,  8.26s/it]
 85%|████████▌ | 32387/37962 [74:59:44<12:45:26,  8.24s/it]
 85%|████████▌ | 32388/37962 [74:59:52<12:39:36,  8.18s/it]
 85%|████████▌ | 32389/37962 [75:00:01<12:45:13,  8.24s/it]
 85%|████████▌ | 32390/37962 [75:00:09<12:47:16,  8.26s/it]
                                                           
{'loss': 1.0533, 'grad_norm': 7.197556495666504, 'learning_rate': 1.1088158517239755e-06, 'epoch': 0.85}

 85%|████████▌ | 32390/37962 [75:00:09<12:47:16,  8.26s/it]
 85%|████████▌ | 32391/37962 [75:00:17<12:45:36,  8.25s/it]
 85%|████████▌ | 32392/37962 [75:00:25<12:44:06,  8.23s/it]
 85%|████████▌ | 32393/37962 [75:00:34<12:43:43,  8.23s/it]
 85%|████████▌ | 32394/37962 [75:00:49<16:15:23, 10.51s/it]
 85%|████████▌ | 32395/37962 [75:00:58<15:07:36,  9.78s/it]
 85%|████████▌ | 32396/37962 [75:01:06<14:26:14,  9.34s/it]
 85%|████████▌ | 32397/37962 [75:01:14<13:44:00,  8.88s/it]
 85%|████████▌ | 32398/37962 [75:01:22<13:20:25,  8.63s/it]
 85%|████████▌ | 32399/37962 [75:01:30<13:02:35,  8.44s/it]
 85%|████████▌ | 32400/37962 [75:01:38<12:56:57,  8.38s/it]
                                                           
{'loss': 1.0898, 'grad_norm': 7.740823268890381, 'learning_rate': 1.1049143670637118e-06, 'epoch': 0.85}

 85%|████████▌ | 32400/37962 [75:01:38<12:56:57,  8.38s/it]
 85%|████████▌ | 32401/37962 [75:01:46<12:56:08,  8.37s/it]
 85%|████████▌ | 32402/37962 [75:01:55<12:54:28,  8.36s/it]
 85%|████████▌ | 32403/37962 [75:02:03<12:51:26,  8.33s/it]
 85%|████████▌ | 32404/37962 [75:02:11<12:51:49,  8.33s/it]
 85%|████████▌ | 32405/37962 [75:02:20<12:51:52,  8.33s/it]
 85%|████████▌ | 32406/37962 [75:02:28<12:51:07,  8.33s/it]
 85%|████████▌ | 32407/37962 [75:02:36<12:51:12,  8.33s/it]
 85%|████████▌ | 32408/37962 [75:02:44<12:40:22,  8.21s/it]
 85%|████████▌ | 32409/37962 [75:02:52<12:42:56,  8.24s/it]
 85%|████████▌ | 32410/37962 [75:03:01<12:43:35,  8.25s/it]
                                                           
{'loss': 1.1399, 'grad_norm': 8.950605392456055, 'learning_rate': 1.1010193569821813e-06, 'epoch': 0.85}

 85%|████████▌ | 32410/37962 [75:03:01<12:43:35,  8.25s/it]
 85%|████████▌ | 32411/37962 [75:03:09<12:46:49,  8.29s/it]
 85%|████████▌ | 32412/37962 [75:03:17<12:47:14,  8.29s/it]
 85%|████████▌ | 32413/37962 [75:03:26<12:50:30,  8.33s/it]
 85%|████████▌ | 32414/37962 [75:03:34<12:49:48,  8.33s/it]
 85%|████████▌ | 32415/37962 [75:03:42<12:43:53,  8.26s/it]
 85%|████████▌ | 32416/37962 [75:03:50<12:38:25,  8.21s/it]
 85%|████████▌ | 32417/37962 [75:03:59<12:37:37,  8.20s/it]
 85%|████████▌ | 32418/37962 [75:04:07<12:42:50,  8.26s/it]
 85%|████████▌ | 32419/37962 [75:04:15<12:43:49,  8.27s/it]
 85%|████████▌ | 32420/37962 [75:04:24<12:52:22,  8.36s/it]
                                                           
{'loss': 1.2119, 'grad_norm': 8.77425479888916, 'learning_rate': 1.0971308243144985e-06, 'epoch': 0.85}

 85%|████████▌ | 32420/37962 [75:04:24<12:52:22,  8.36s/it]
 85%|████████▌ | 32421/37962 [75:04:32<12:47:21,  8.31s/it]
 85%|████████▌ | 32422/37962 [75:04:40<12:51:14,  8.35s/it]
 85%|████████▌ | 32423/37962 [75:04:49<12:49:49,  8.34s/it]
 85%|████████▌ | 32424/37962 [75:04:57<12:51:21,  8.36s/it]
 85%|████████▌ | 32425/37962 [75:05:05<12:50:01,  8.34s/it]
 85%|████████▌ | 32426/37962 [75:05:14<12:48:33,  8.33s/it]
 85%|████████▌ | 32427/37962 [75:05:22<12:50:56,  8.36s/it]
 85%|████████▌ | 32428/37962 [75:05:31<12:52:04,  8.37s/it]
 85%|████████▌ | 32429/37962 [75:05:39<12:50:31,  8.36s/it]
 85%|████████▌ | 32430/37962 [75:05:47<12:44:57,  8.30s/it]
                                                           
{'loss': 1.1894, 'grad_norm': 8.838737487792969, 'learning_rate': 1.0932487718910555e-06, 'epoch': 0.85}

 85%|████████▌ | 32430/37962 [75:05:47<12:44:57,  8.30s/it]
 85%|████████▌ | 32431/37962 [75:05:55<12:45:04,  8.30s/it]
 85%|████████▌ | 32432/37962 [75:06:04<12:46:11,  8.31s/it]
 85%|████████▌ | 32433/37962 [75:06:12<12:42:43,  8.28s/it]
 85%|████████▌ | 32434/37962 [75:06:20<12:37:39,  8.22s/it]
 85%|████████▌ | 32435/37962 [75:06:28<12:37:20,  8.22s/it]
 85%|████████▌ | 32436/37962 [75:06:37<12:39:36,  8.25s/it]
 85%|████████▌ | 32437/37962 [75:06:45<12:43:34,  8.29s/it]
 85%|████████▌ | 32438/37962 [75:06:53<12:40:54,  8.26s/it]
 85%|████████▌ | 32439/37962 [75:07:02<12:46:36,  8.33s/it]
 85%|████████▌ | 32440/37962 [75:07:10<12:38:47,  8.24s/it]
                                                           
{'loss': 1.0847, 'grad_norm': 5.256621360778809, 'learning_rate': 1.0893732025375348e-06, 'epoch': 0.85}

 85%|████████▌ | 32440/37962 [75:07:10<12:38:47,  8.24s/it]
 85%|████████▌ | 32441/37962 [75:07:18<12:37:31,  8.23s/it]
 85%|████████▌ | 32442/37962 [75:07:26<12:39:22,  8.25s/it]
 85%|████████▌ | 32443/37962 [75:07:34<12:35:43,  8.22s/it]
 85%|████████▌ | 32444/37962 [75:07:42<12:35:30,  8.22s/it]
 85%|████████▌ | 32445/37962 [75:07:51<12:40:15,  8.27s/it]
 85%|████████▌ | 32446/37962 [75:07:59<12:40:26,  8.27s/it]
 85%|████████▌ | 32447/37962 [75:08:07<12:37:04,  8.24s/it]
 85%|████████▌ | 32448/37962 [75:08:16<12:37:58,  8.25s/it]
 85%|████████▌ | 32449/37962 [75:08:24<12:41:14,  8.28s/it]
 85%|████████▌ | 32450/37962 [75:08:32<12:41:45,  8.29s/it]
                                                           
{'loss': 1.1446, 'grad_norm': 12.977181434631348, 'learning_rate': 1.0855041190748915e-06, 'epoch': 0.85}

 85%|████████▌ | 32450/37962 [75:08:32<12:41:45,  8.29s/it]
 85%|████████▌ | 32451/37962 [75:08:41<12:41:34,  8.29s/it]
 85%|████████▌ | 32452/37962 [75:08:49<12:47:08,  8.35s/it]
 85%|████████▌ | 32453/37962 [75:08:57<12:42:14,  8.30s/it]
 85%|████████▌ | 32454/37962 [75:09:06<12:41:42,  8.30s/it]
 85%|████████▌ | 32455/37962 [75:09:14<12:33:14,  8.21s/it]
 85%|████████▌ | 32456/37962 [75:09:22<12:29:25,  8.17s/it]
 85%|████████▌ | 32457/37962 [75:09:30<12:30:51,  8.18s/it]
 86%|████████▌ | 32458/37962 [75:09:38<12:32:17,  8.20s/it]
 86%|████████▌ | 32459/37962 [75:09:46<12:29:33,  8.17s/it]
 86%|████████▌ | 32460/37962 [75:09:55<12:36:26,  8.25s/it]
                                                           
{'loss': 1.1205, 'grad_norm': 5.654670238494873, 'learning_rate': 1.081641524319369e-06, 'epoch': 0.86}

 86%|████████▌ | 32460/37962 [75:09:55<12:36:26,  8.25s/it]
 86%|████████▌ | 32461/37962 [75:10:03<12:33:19,  8.22s/it]
 86%|████████▌ | 32462/37962 [75:10:11<12:33:51,  8.22s/it]
 86%|████████▌ | 32463/37962 [75:10:19<12:34:13,  8.23s/it]
 86%|████████▌ | 32464/37962 [75:10:28<12:35:55,  8.25s/it]
 86%|████████▌ | 32465/37962 [75:10:36<12:40:52,  8.30s/it]
 86%|████████▌ | 32466/37962 [75:10:44<12:44:47,  8.35s/it]
 86%|████████▌ | 32467/37962 [75:10:53<12:41:12,  8.31s/it]
 86%|████████▌ | 32468/37962 [75:11:01<12:41:29,  8.32s/it]
 86%|████████▌ | 32469/37962 [75:11:09<12:38:18,  8.28s/it]
 86%|████████▌ | 32470/37962 [75:11:17<12:37:55,  8.28s/it]
                                                           
{'loss': 1.1704, 'grad_norm': 20.45656967163086, 'learning_rate': 1.07778542108248e-06, 'epoch': 0.86}

 86%|████████▌ | 32470/37962 [75:11:17<12:37:55,  8.28s/it]
 86%|████████▌ | 32471/37962 [75:11:26<12:37:02,  8.27s/it]
 86%|████████▌ | 32472/37962 [75:11:34<12:37:28,  8.28s/it]
 86%|████████▌ | 32473/37962 [75:11:42<12:39:04,  8.30s/it]
 86%|████████▌ | 32474/37962 [75:11:51<12:39:34,  8.30s/it]
 86%|████████▌ | 32475/37962 [75:11:59<12:40:11,  8.31s/it]
 86%|████████▌ | 32476/37962 [75:12:07<12:38:10,  8.29s/it]
 86%|████████▌ | 32477/37962 [75:12:15<12:36:24,  8.27s/it]
 86%|████████▌ | 32478/37962 [75:12:24<12:36:02,  8.27s/it]
 86%|████████▌ | 32479/37962 [75:12:32<12:40:49,  8.33s/it]
 86%|████████▌ | 32480/37962 [75:12:40<12:37:27,  8.29s/it]
                                                           
{'loss': 1.1356, 'grad_norm': 20.9130916595459, 'learning_rate': 1.0739358121710152e-06, 'epoch': 0.86}

 86%|████████▌ | 32480/37962 [75:12:40<12:37:27,  8.29s/it]
 86%|████████▌ | 32481/37962 [75:12:49<12:40:09,  8.32s/it]
 86%|████████▌ | 32482/37962 [75:12:57<12:40:45,  8.33s/it]
 86%|████████▌ | 32483/37962 [75:13:06<12:43:48,  8.36s/it]
 86%|████████▌ | 32484/37962 [75:13:14<12:39:35,  8.32s/it]
 86%|████████▌ | 32485/37962 [75:13:22<12:38:59,  8.31s/it]
 86%|████████▌ | 32486/37962 [75:13:31<12:42:07,  8.35s/it]
 86%|████████▌ | 32487/37962 [75:13:39<12:38:53,  8.32s/it]
 86%|████████▌ | 32488/37962 [75:13:47<12:38:29,  8.31s/it]
 86%|████████▌ | 32489/37962 [75:13:55<12:38:30,  8.32s/it]
 86%|████████▌ | 32490/37962 [75:14:04<12:38:00,  8.31s/it]
                                                           
{'loss': 1.325, 'grad_norm': 6.402110576629639, 'learning_rate': 1.0700927003870398e-06, 'epoch': 0.86}

 86%|████████▌ | 32490/37962 [75:14:04<12:38:00,  8.31s/it]
 86%|████████▌ | 32491/37962 [75:14:12<12:37:00,  8.30s/it]
 86%|████████▌ | 32492/37962 [75:14:20<12:37:25,  8.31s/it]
 86%|████████▌ | 32493/37962 [75:14:28<12:34:46,  8.28s/it]
 86%|████████▌ | 32494/37962 [75:14:37<12:36:44,  8.30s/it]
 86%|████████▌ | 32495/37962 [75:14:45<12:35:23,  8.29s/it]
 86%|████████▌ | 32496/37962 [75:14:53<12:32:43,  8.26s/it]
 86%|████████▌ | 32497/37962 [75:15:02<12:33:12,  8.27s/it]
 86%|████████▌ | 32498/37962 [75:15:10<12:32:45,  8.27s/it]
 86%|████████▌ | 32499/37962 [75:15:18<12:35:00,  8.29s/it]
 86%|████████▌ | 32500/37962 [75:15:26<12:34:05,  8.28s/it]
                                                           
{'loss': 1.0658, 'grad_norm': 8.539005279541016, 'learning_rate': 1.0662560885278883e-06, 'epoch': 0.86}

 86%|████████▌ | 32500/37962 [75:15:26<12:34:05,  8.28s/it]
 86%|████████▌ | 32501/37962 [75:15:35<12:31:35,  8.26s/it]
 86%|████████▌ | 32502/37962 [75:15:43<12:30:10,  8.24s/it]
 86%|████████▌ | 32503/37962 [75:15:51<12:31:06,  8.26s/it]
 86%|████████▌ | 32504/37962 [75:15:59<12:29:20,  8.24s/it]
 86%|████████▌ | 32505/37962 [75:16:07<12:24:45,  8.19s/it]
 86%|████████▌ | 32506/37962 [75:16:15<12:18:06,  8.12s/it]
 86%|████████▌ | 32507/37962 [75:16:24<12:19:20,  8.13s/it]
 86%|████████▌ | 32508/37962 [75:16:32<12:27:52,  8.23s/it]
 86%|████████▌ | 32509/37962 [75:16:40<12:34:11,  8.30s/it]
 86%|████████▌ | 32510/37962 [75:16:49<12:32:25,  8.28s/it]
                                                           
{'loss': 1.0978, 'grad_norm': 9.427186012268066, 'learning_rate': 1.0624259793861647e-06, 'epoch': 0.86}

 86%|████████▌ | 32510/37962 [75:16:49<12:32:25,  8.28s/it]
 86%|████████▌ | 32511/37962 [75:16:57<12:36:03,  8.32s/it]
 86%|████████▌ | 32512/37962 [75:17:06<12:40:13,  8.37s/it]
 86%|████████▌ | 32513/37962 [75:17:14<12:40:29,  8.37s/it]
 86%|████████▌ | 32514/37962 [75:17:22<12:33:06,  8.29s/it]
 86%|████████▌ | 32515/37962 [75:17:30<12:35:40,  8.32s/it]
 86%|████████▌ | 32516/37962 [75:17:39<12:38:40,  8.36s/it]
 86%|████████▌ | 32517/37962 [75:17:47<12:29:07,  8.25s/it]
 86%|████████▌ | 32518/37962 [75:17:55<12:31:57,  8.29s/it]
 86%|████████▌ | 32519/37962 [75:18:04<12:31:02,  8.28s/it]
 86%|████████▌ | 32520/37962 [75:18:12<12:33:24,  8.31s/it]
                                                           
{'loss': 1.1965, 'grad_norm': 18.484560012817383, 'learning_rate': 1.0586023757497355e-06, 'epoch': 0.86}

 86%|████████▌ | 32520/37962 [75:18:12<12:33:24,  8.31s/it]
 86%|████████▌ | 32521/37962 [75:18:20<12:31:13,  8.28s/it]
 86%|████████▌ | 32522/37962 [75:18:29<12:39:00,  8.37s/it]
 86%|████████▌ | 32523/37962 [75:18:37<12:39:45,  8.38s/it]
 86%|████████▌ | 32524/37962 [75:18:45<12:37:04,  8.35s/it]
 86%|████████▌ | 32525/37962 [75:18:53<12:28:49,  8.26s/it]
 86%|████████▌ | 32526/37962 [75:19:02<12:29:06,  8.27s/it]
 86%|████████▌ | 32527/37962 [75:19:10<12:34:47,  8.33s/it]
 86%|████████▌ | 32528/37962 [75:19:18<12:23:00,  8.20s/it]
 86%|████████▌ | 32529/37962 [75:19:26<12:23:53,  8.22s/it]
 86%|████████▌ | 32530/37962 [75:19:35<12:25:11,  8.23s/it]
                                                           
{'loss': 1.2014, 'grad_norm': 13.262128829956055, 'learning_rate': 1.0547852804017388e-06, 'epoch': 0.86}

 86%|████████▌ | 32530/37962 [75:19:35<12:25:11,  8.23s/it]
 86%|████████▌ | 32531/37962 [75:19:43<12:25:02,  8.23s/it]
 86%|████████▌ | 32532/37962 [75:19:51<12:27:35,  8.26s/it]
 86%|████████▌ | 32533/37962 [75:19:59<12:17:09,  8.15s/it]
 86%|████████▌ | 32534/37962 [75:20:07<12:09:48,  8.07s/it]
 86%|████████▌ | 32535/37962 [75:20:15<12:09:39,  8.07s/it]
 86%|████████▌ | 32536/37962 [75:20:23<12:13:13,  8.11s/it]
 86%|████████▌ | 32537/37962 [75:20:31<12:15:02,  8.13s/it]
 86%|████████▌ | 32538/37962 [75:20:40<12:18:18,  8.17s/it]
 86%|████████▌ | 32539/37962 [75:20:48<12:19:22,  8.18s/it]
 86%|████████▌ | 32540/37962 [75:20:56<12:09:10,  8.07s/it]
                                                           
{'loss': 1.1143, 'grad_norm': 12.884847640991211, 'learning_rate': 1.0509746961205701e-06, 'epoch': 0.86}

 86%|████████▌ | 32540/37962 [75:20:56<12:09:10,  8.07s/it]
 86%|████████▌ | 32541/37962 [75:21:04<12:16:28,  8.15s/it]
 86%|████████▌ | 32542/37962 [75:21:12<12:18:41,  8.18s/it]
 86%|████████▌ | 32543/37962 [75:21:21<12:21:30,  8.21s/it]
 86%|████████▌ | 32544/37962 [75:21:29<12:22:16,  8.22s/it]
 86%|████████▌ | 32545/37962 [75:21:37<12:23:25,  8.23s/it]
 86%|████████▌ | 32546/37962 [75:21:45<12:27:12,  8.28s/it]
 86%|████████▌ | 32547/37962 [75:21:53<12:17:34,  8.17s/it]
 86%|████████▌ | 32548/37962 [75:22:01<12:06:27,  8.05s/it]
 86%|████████▌ | 32549/37962 [75:22:09<12:13:03,  8.13s/it]
 86%|████████▌ | 32550/37962 [75:22:18<12:19:12,  8.20s/it]
                                                           
{'loss': 1.1649, 'grad_norm': 6.777890682220459, 'learning_rate': 1.047170625679892e-06, 'epoch': 0.86}

 86%|████████▌ | 32550/37962 [75:22:18<12:19:12,  8.20s/it]
 86%|████████▌ | 32551/37962 [75:22:26<12:27:02,  8.28s/it]
 86%|████████▌ | 32552/37962 [75:22:35<12:25:54,  8.27s/it]
 86%|████████▌ | 32553/37962 [75:22:43<12:29:25,  8.31s/it]
 86%|████████▌ | 32554/37962 [75:22:51<12:29:06,  8.31s/it]
 86%|████████▌ | 32555/37962 [75:22:59<12:21:02,  8.22s/it]
 86%|████████▌ | 32556/37962 [75:23:08<12:22:33,  8.24s/it]
 86%|████████▌ | 32557/37962 [75:23:16<12:14:44,  8.16s/it]
 86%|████████▌ | 32558/37962 [75:23:24<12:20:21,  8.22s/it]
 86%|████████▌ | 32559/37962 [75:23:32<12:22:01,  8.24s/it]
 86%|████████▌ | 32560/37962 [75:23:40<12:23:40,  8.26s/it]
                                                           
{'loss': 1.1533, 'grad_norm': 8.76011848449707, 'learning_rate': 1.0433730718486168e-06, 'epoch': 0.86}

 86%|████████▌ | 32560/37962 [75:23:40<12:23:40,  8.26s/it]
 86%|████████▌ | 32561/37962 [75:23:49<12:23:13,  8.26s/it]
 86%|████████▌ | 32562/37962 [75:23:57<12:23:54,  8.27s/it]
 86%|████████▌ | 32563/37962 [75:24:05<12:20:10,  8.23s/it]
 86%|████████▌ | 32564/37962 [75:24:13<12:22:20,  8.25s/it]
 86%|████████▌ | 32565/37962 [75:24:22<12:22:43,  8.26s/it]
 86%|████████▌ | 32566/37962 [75:24:30<12:21:52,  8.25s/it]
 86%|████████▌ | 32567/37962 [75:24:38<12:23:44,  8.27s/it]
 86%|████████▌ | 32568/37962 [75:24:46<12:15:16,  8.18s/it]
 86%|████████▌ | 32569/37962 [75:24:54<12:05:43,  8.07s/it]
 86%|████████▌ | 32570/37962 [75:25:02<12:08:44,  8.11s/it]
                                                           
{'loss': 1.2923, 'grad_norm': 8.164602279663086, 'learning_rate': 1.0395820373909183e-06, 'epoch': 0.86}

 86%|████████▌ | 32570/37962 [75:25:02<12:08:44,  8.11s/it]
 86%|████████▌ | 32571/37962 [75:25:11<12:14:31,  8.17s/it]
 86%|████████▌ | 32572/37962 [75:25:19<12:18:30,  8.22s/it]
 86%|████████▌ | 32573/37962 [75:25:27<12:20:02,  8.24s/it]
 86%|████████▌ | 32574/37962 [75:25:35<12:20:00,  8.24s/it]
 86%|████████▌ | 32575/37962 [75:25:44<12:16:55,  8.21s/it]
 86%|████████▌ | 32576/37962 [75:25:52<12:15:54,  8.20s/it]
 86%|████████▌ | 32577/37962 [75:26:00<12:20:05,  8.25s/it]
 86%|████████▌ | 32578/37962 [75:26:09<12:24:20,  8.30s/it]
 86%|████████▌ | 32579/37962 [75:26:17<12:17:06,  8.22s/it]
 86%|████████▌ | 32580/37962 [75:26:25<12:19:23,  8.24s/it]
                                                           
{'loss': 1.2673, 'grad_norm': 21.255773544311523, 'learning_rate': 1.0357975250662256e-06, 'epoch': 0.86}

 86%|████████▌ | 32580/37962 [75:26:25<12:19:23,  8.24s/it]
 86%|████████▌ | 32581/37962 [75:26:33<12:23:49,  8.29s/it]
 86%|████████▌ | 32582/37962 [75:26:41<12:14:57,  8.20s/it]
 86%|████████▌ | 32583/37962 [75:26:49<12:15:51,  8.21s/it]
 86%|████████▌ | 32584/37962 [75:26:58<12:19:25,  8.25s/it]
 86%|████████▌ | 32585/37962 [75:27:06<12:24:28,  8.31s/it]
 86%|████████▌ | 32586/37962 [75:27:14<12:20:58,  8.27s/it]
 86%|████████▌ | 32587/37962 [75:27:23<12:26:52,  8.34s/it]
 86%|████████▌ | 32588/37962 [75:27:32<12:32:43,  8.40s/it]
 86%|████████▌ | 32589/37962 [75:27:40<12:28:25,  8.36s/it]
 86%|████████▌ | 32590/37962 [75:27:48<12:28:02,  8.35s/it]
                                                           
{'loss': 1.1797, 'grad_norm': 13.915183067321777, 'learning_rate': 1.032019537629223e-06, 'epoch': 0.86}

 86%|████████▌ | 32590/37962 [75:27:48<12:28:02,  8.35s/it]
 86%|████████▌ | 32591/37962 [75:27:56<12:23:41,  8.31s/it]
 86%|████████▌ | 32592/37962 [75:28:05<12:21:35,  8.29s/it]
 86%|████████▌ | 32593/37962 [75:28:12<12:09:31,  8.15s/it]
 86%|████████▌ | 32594/37962 [75:28:21<12:13:03,  8.19s/it]
 86%|████████▌ | 32595/37962 [75:28:29<12:05:32,  8.11s/it]
 86%|████████▌ | 32596/37962 [75:28:36<11:58:01,  8.03s/it]
 86%|████████▌ | 32597/37962 [75:28:45<12:03:03,  8.09s/it]
 86%|████████▌ | 32598/37962 [75:28:53<12:10:18,  8.17s/it]
 86%|████████▌ | 32599/37962 [75:29:01<12:04:20,  8.10s/it]
 86%|████████▌ | 32600/37962 [75:29:09<12:14:57,  8.22s/it]
                                                           
{'loss': 1.1785, 'grad_norm': 6.027990818023682, 'learning_rate': 1.0282480778298365e-06, 'epoch': 0.86}

 86%|████████▌ | 32600/37962 [75:29:09<12:14:57,  8.22s/it]
 86%|████████▌ | 32601/37962 [75:29:18<12:20:27,  8.29s/it]
 86%|████████▌ | 32602/37962 [75:29:26<12:20:54,  8.29s/it]
 86%|████████▌ | 32603/37962 [75:29:35<12:23:22,  8.32s/it]
 86%|████████▌ | 32604/37962 [75:29:43<12:20:03,  8.29s/it]
 86%|████████▌ | 32605/37962 [75:29:51<12:10:23,  8.18s/it]
 86%|████████▌ | 32606/37962 [75:29:59<12:08:32,  8.16s/it]
 86%|████████▌ | 32607/37962 [75:30:07<12:14:36,  8.23s/it]
 86%|████████▌ | 32608/37962 [75:30:15<12:13:58,  8.23s/it]
 86%|████████▌ | 32609/37962 [75:30:24<12:18:27,  8.28s/it]
 86%|████████▌ | 32610/37962 [75:30:32<12:17:30,  8.27s/it]
                                                           
{'loss': 1.2101, 'grad_norm': 14.588454246520996, 'learning_rate': 1.0244831484132523e-06, 'epoch': 0.86}

 86%|████████▌ | 32610/37962 [75:30:32<12:17:30,  8.27s/it]
 86%|████████▌ | 32611/37962 [75:30:40<12:16:30,  8.26s/it]
 86%|████████▌ | 32612/37962 [75:30:48<12:07:20,  8.16s/it]
 86%|████████▌ | 32613/37962 [75:30:56<12:08:16,  8.17s/it]
 86%|████████▌ | 32614/37962 [75:31:05<12:08:01,  8.17s/it]
 86%|████████▌ | 32615/37962 [75:31:13<12:09:05,  8.18s/it]
 86%|████████▌ | 32616/37962 [75:31:21<12:09:52,  8.19s/it]
 86%|████████▌ | 32617/37962 [75:31:29<12:13:11,  8.23s/it]
 86%|████████▌ | 32618/37962 [75:31:38<12:18:36,  8.29s/it]
 86%|████████▌ | 32619/37962 [75:31:46<12:13:31,  8.24s/it]
 86%|████████▌ | 32620/37962 [75:31:54<12:05:43,  8.15s/it]
                                                           
{'loss': 1.0428, 'grad_norm': 6.58778715133667, 'learning_rate': 1.0207247521198937e-06, 'epoch': 0.86}

 86%|████████▌ | 32620/37962 [75:31:54<12:05:43,  8.15s/it]
 86%|████████▌ | 32621/37962 [75:32:02<12:07:30,  8.17s/it]
 86%|████████▌ | 32622/37962 [75:32:10<12:07:22,  8.17s/it]
 86%|████████▌ | 32623/37962 [75:32:19<12:08:49,  8.19s/it]
 86%|████████▌ | 32624/37962 [75:32:27<12:05:46,  8.16s/it]
 86%|████████▌ | 32625/37962 [75:32:35<12:11:50,  8.23s/it]
 86%|████████▌ | 32626/37962 [75:32:43<12:11:13,  8.22s/it]
 86%|████████▌ | 32627/37962 [75:32:52<12:13:34,  8.25s/it]
 86%|████████▌ | 32628/37962 [75:33:00<12:13:05,  8.25s/it]
 86%|████████▌ | 32629/37962 [75:33:08<12:16:19,  8.28s/it]
 86%|████████▌ | 32630/37962 [75:33:17<12:21:48,  8.35s/it]
                                                           
{'loss': 1.1358, 'grad_norm': 9.921648979187012, 'learning_rate': 1.0169728916854305e-06, 'epoch': 0.86}

 86%|████████▌ | 32630/37962 [75:33:17<12:21:48,  8.35s/it]
 86%|████████▌ | 32631/37962 [75:33:25<12:17:03,  8.30s/it]
 86%|████████▌ | 32632/37962 [75:33:33<12:22:54,  8.36s/it]
 86%|████████▌ | 32633/37962 [75:33:42<12:19:17,  8.32s/it]
 86%|████████▌ | 32634/37962 [75:33:50<12:24:33,  8.38s/it]
 86%|████████▌ | 32635/37962 [75:33:58<12:22:27,  8.36s/it]
 86%|████████▌ | 32636/37962 [75:34:07<12:18:04,  8.31s/it]
 86%|████████▌ | 32637/37962 [75:34:15<12:18:34,  8.32s/it]
 86%|████████▌ | 32638/37962 [75:34:23<12:16:17,  8.30s/it]
 86%|████████▌ | 32639/37962 [75:34:31<12:09:10,  8.22s/it]
 86%|████████▌ | 32640/37962 [75:34:39<12:11:14,  8.24s/it]
                                                           
{'loss': 1.1278, 'grad_norm': 8.758599281311035, 'learning_rate': 1.0132275698407813e-06, 'epoch': 0.86}

 86%|████████▌ | 32640/37962 [75:34:39<12:11:14,  8.24s/it]
 86%|████████▌ | 32641/37962 [75:34:48<12:11:13,  8.25s/it]
 86%|████████▌ | 32642/37962 [75:34:56<12:10:03,  8.23s/it]
 86%|████████▌ | 32643/37962 [75:35:04<12:01:30,  8.14s/it]
 86%|████████▌ | 32644/37962 [75:35:12<12:06:13,  8.19s/it]
 86%|████████▌ | 32645/37962 [75:35:20<12:07:42,  8.21s/it]
 86%|████████▌ | 32646/37962 [75:35:29<12:07:07,  8.21s/it]
 86%|████████▌ | 32647/37962 [75:35:37<12:12:53,  8.27s/it]
 86%|████████▌ | 32648/37962 [75:35:45<12:15:45,  8.31s/it]
 86%|████████▌ | 32649/37962 [75:35:54<12:16:19,  8.32s/it]
 86%|████████▌ | 32650/37962 [75:36:02<12:13:40,  8.29s/it]
                                                           
{'loss': 1.1941, 'grad_norm': 7.8956804275512695, 'learning_rate': 1.0094887893120964e-06, 'epoch': 0.86}

 86%|████████▌ | 32650/37962 [75:36:02<12:13:40,  8.29s/it]
 86%|████████▌ | 32651/37962 [75:36:10<12:08:16,  8.23s/it]
 86%|████████▌ | 32652/37962 [75:36:18<12:09:38,  8.24s/it]
 86%|████████▌ | 32653/37962 [75:36:27<12:08:30,  8.23s/it]
 86%|████████▌ | 32654/37962 [75:36:35<12:07:58,  8.23s/it]
 86%|████████▌ | 32655/37962 [75:36:43<12:04:10,  8.19s/it]
 86%|████████▌ | 32656/37962 [75:36:51<12:06:26,  8.21s/it]
 86%|████████▌ | 32657/37962 [75:36:59<11:59:56,  8.14s/it]
 86%|████████▌ | 32658/37962 [75:37:07<12:00:52,  8.15s/it]
 86%|████████▌ | 32659/37962 [75:37:16<12:06:18,  8.22s/it]
 86%|████████▌ | 32660/37962 [75:37:24<12:00:44,  8.16s/it]
                                                           
{'loss': 1.1498, 'grad_norm': 9.253400802612305, 'learning_rate': 1.0057565528207725e-06, 'epoch': 0.86}

 86%|████████▌ | 32660/37962 [75:37:24<12:00:44,  8.16s/it]
 86%|████████▌ | 32661/37962 [75:37:32<12:03:49,  8.19s/it]
 86%|████████▌ | 32662/37962 [75:37:40<12:02:02,  8.17s/it]
 86%|████████▌ | 32663/37962 [75:37:49<12:07:45,  8.24s/it]
 86%|████████▌ | 32664/37962 [75:37:57<12:10:00,  8.27s/it]
 86%|████████▌ | 32665/37962 [75:38:05<12:10:16,  8.27s/it]
 86%|████████▌ | 32666/37962 [75:38:13<12:08:16,  8.25s/it]
 86%|████████▌ | 32667/37962 [75:38:29<15:28:24, 10.52s/it]
 86%|████████▌ | 32668/37962 [75:38:38<14:36:12,  9.93s/it]
 86%|████████▌ | 32669/37962 [75:38:46<13:48:10,  9.39s/it]
 86%|████████▌ | 32670/37962 [75:38:54<13:16:06,  9.03s/it]
                                                           
{'loss': 1.1951, 'grad_norm': 8.5781888961792, 'learning_rate': 1.002030863083443e-06, 'epoch': 0.86}

 86%|████████▌ | 32670/37962 [75:38:54<13:16:06,  9.03s/it]
 86%|████████▌ | 32671/37962 [75:39:02<12:50:24,  8.74s/it]
 86%|████████▌ | 32672/37962 [75:39:10<12:36:16,  8.58s/it]
 86%|████████▌ | 32673/37962 [75:39:19<12:29:47,  8.51s/it]
 86%|████████▌ | 32674/37962 [75:39:27<12:20:34,  8.40s/it]
 86%|████████▌ | 32675/37962 [75:39:35<12:08:31,  8.27s/it]
 86%|████████▌ | 32676/37962 [75:39:43<12:11:02,  8.30s/it]
 86%|████████▌ | 32677/37962 [75:39:51<11:57:54,  8.15s/it]
 86%|████████▌ | 32678/37962 [75:39:59<11:59:08,  8.17s/it]
 86%|████████▌ | 32679/37962 [75:40:07<11:55:16,  8.12s/it]
 86%|████████▌ | 32680/37962 [75:40:15<11:59:32,  8.17s/it]
                                                           
{'loss': 1.2482, 'grad_norm': 11.353352546691895, 'learning_rate': 9.983117228119643e-07, 'epoch': 0.86}

 86%|████████▌ | 32680/37962 [75:40:15<11:59:32,  8.17s/it]
 86%|████████▌ | 32681/37962 [75:40:24<12:01:09,  8.19s/it]
 86%|████████▌ | 32682/37962 [75:40:32<11:55:32,  8.13s/it]
 86%|████████▌ | 32683/37962 [75:40:40<12:02:03,  8.21s/it]
 86%|████████▌ | 32684/37962 [75:40:48<12:03:35,  8.23s/it]
 86%|████████▌ | 32685/37962 [75:40:57<12:06:24,  8.26s/it]
 86%|████████▌ | 32686/37962 [75:41:05<12:07:11,  8.27s/it]
 86%|████████▌ | 32687/37962 [75:41:13<12:05:10,  8.25s/it]
 86%|████████▌ | 32688/37962 [75:41:21<12:08:06,  8.28s/it]
 86%|████████▌ | 32689/37962 [75:41:30<12:02:25,  8.22s/it]
 86%|████████▌ | 32690/37962 [75:41:38<12:11:44,  8.33s/it]
                                                           
{'loss': 1.1426, 'grad_norm': 10.097792625427246, 'learning_rate': 9.945991347134398e-07, 'epoch': 0.86}

 86%|████████▌ | 32690/37962 [75:41:38<12:11:44,  8.33s/it]
 86%|████████▌ | 32691/37962 [75:41:46<12:09:50,  8.31s/it]
 86%|████████▌ | 32692/37962 [75:41:55<12:11:54,  8.33s/it]
 86%|████████▌ | 32693/37962 [75:42:03<12:06:59,  8.28s/it]
 86%|████████▌ | 32694/37962 [75:42:11<12:06:45,  8.28s/it]
 86%|████████▌ | 32695/37962 [75:42:20<12:15:21,  8.38s/it]
 86%|████████▌ | 32696/37962 [75:42:28<12:07:38,  8.29s/it]
 86%|████████▌ | 32697/37962 [75:42:36<12:11:42,  8.34s/it]
 86%|████████▌ | 32698/37962 [75:42:45<12:08:56,  8.31s/it]
 86%|████████▌ | 32699/37962 [75:42:53<12:05:55,  8.28s/it]
 86%|████████▌ | 32700/37962 [75:43:01<12:09:57,  8.32s/it]
                                                           
{'loss': 1.0248, 'grad_norm': 17.720996856689453, 'learning_rate': 9.908931014901991e-07, 'epoch': 0.86}

 86%|████████▌ | 32700/37962 [75:43:01<12:09:57,  8.32s/it]
 86%|████████▌ | 32701/37962 [75:43:10<12:10:15,  8.33s/it]
 86%|████████▌ | 32702/37962 [75:43:18<12:13:07,  8.36s/it]
 86%|████████▌ | 32703/37962 [75:43:26<12:10:17,  8.33s/it]
 86%|████████▌ | 32704/37962 [75:43:34<11:59:30,  8.21s/it]
 86%|████████▌ | 32705/37962 [75:43:43<12:02:43,  8.25s/it]
 86%|████████▌ | 32706/37962 [75:43:51<12:06:39,  8.30s/it]
 86%|████████▌ | 32707/37962 [75:43:59<12:03:46,  8.26s/it]
 86%|████████▌ | 32708/37962 [75:44:08<12:07:54,  8.31s/it]
 86%|████████▌ | 32709/37962 [75:44:16<12:11:47,  8.36s/it]
 86%|████████▌ | 32710/37962 [75:44:24<12:10:58,  8.35s/it]
                                                           
{'loss': 1.2, 'grad_norm': 22.136411666870117, 'learning_rate': 9.871936258397962e-07, 'epoch': 0.86}

 86%|████████▌ | 32710/37962 [75:44:24<12:10:58,  8.35s/it]
 86%|████████▌ | 32711/37962 [75:44:33<12:08:31,  8.32s/it]
 86%|████████▌ | 32712/37962 [75:44:41<12:06:08,  8.30s/it]
 86%|████████▌ | 32713/37962 [75:44:49<12:04:42,  8.28s/it]
 86%|████████▌ | 32714/37962 [75:44:58<12:08:10,  8.33s/it]
 86%|████████▌ | 32715/37962 [75:45:06<12:00:36,  8.24s/it]
 86%|████████▌ | 32716/37962 [75:45:14<11:52:21,  8.15s/it]
 86%|████████▌ | 32717/37962 [75:45:22<11:54:10,  8.17s/it]
 86%|████████▌ | 32718/37962 [75:45:30<12:01:12,  8.25s/it]
 86%|████████▌ | 32719/37962 [75:45:38<11:51:27,  8.14s/it]
 86%|████████▌ | 32720/37962 [75:45:46<11:53:45,  8.17s/it]
                                                           
{'loss': 1.137, 'grad_norm': 9.966521263122559, 'learning_rate': 9.835007104550187e-07, 'epoch': 0.86}

 86%|████████▌ | 32720/37962 [75:45:46<11:53:45,  8.17s/it]
 86%|████████▌ | 32721/37962 [75:45:54<11:48:16,  8.11s/it]
 86%|████████▌ | 32722/37962 [75:46:03<11:55:41,  8.19s/it]
 86%|████████▌ | 32723/37962 [75:46:11<11:57:48,  8.22s/it]
 86%|████████▌ | 32724/37962 [75:46:19<11:56:00,  8.20s/it]
 86%|████████▌ | 32725/37962 [75:46:27<11:57:27,  8.22s/it]
 86%|████████▌ | 32726/37962 [75:46:35<11:52:47,  8.17s/it]
 86%|████████▌ | 32727/37962 [75:46:44<12:03:03,  8.29s/it]
 86%|████████▌ | 32728/37962 [75:46:52<11:58:57,  8.24s/it]
 86%|████████▌ | 32729/37962 [75:47:00<12:01:10,  8.27s/it]
 86%|████████▌ | 32730/37962 [75:47:08<11:53:11,  8.18s/it]
                                                           
{'loss': 1.0849, 'grad_norm': 7.82261848449707, 'learning_rate': 9.79814358023874e-07, 'epoch': 0.86}

 86%|████████▌ | 32730/37962 [75:47:08<11:53:11,  8.18s/it]
 86%|████████▌ | 32731/37962 [75:47:17<11:57:10,  8.23s/it]
 86%|████████▌ | 32732/37962 [75:47:25<11:56:29,  8.22s/it]
 86%|████████▌ | 32733/37962 [75:47:33<12:00:26,  8.27s/it]
 86%|████████▌ | 32734/37962 [75:47:42<11:58:34,  8.25s/it]
 86%|████████▌ | 32735/37962 [75:47:50<12:11:57,  8.40s/it]
 86%|████████▌ | 32736/37962 [75:47:59<12:07:46,  8.36s/it]
 86%|████████▌ | 32737/37962 [75:48:07<12:00:48,  8.28s/it]
 86%|████████▌ | 32738/37962 [75:48:15<11:58:18,  8.25s/it]
 86%|████████▌ | 32739/37962 [75:48:23<12:05:50,  8.34s/it]
 86%|████████▌ | 32740/37962 [75:48:32<12:02:54,  8.31s/it]
                                                           
{'loss': 1.2001, 'grad_norm': 7.936743259429932, 'learning_rate': 9.76134571229591e-07, 'epoch': 0.86}

 86%|████████▌ | 32740/37962 [75:48:32<12:02:54,  8.31s/it]
 86%|████████▌ | 32741/37962 [75:48:40<11:57:09,  8.24s/it]
 86%|████████▌ | 32742/37962 [75:48:48<11:57:04,  8.24s/it]
 86%|████████▋ | 32743/37962 [75:48:56<12:00:04,  8.28s/it]
 86%|████████▋ | 32744/37962 [75:49:05<12:01:13,  8.29s/it]
 86%|████████▋ | 32745/37962 [75:49:13<12:01:30,  8.30s/it]
 86%|████████▋ | 32746/37962 [75:49:21<11:59:29,  8.28s/it]
 86%|████████▋ | 32747/37962 [75:49:29<11:58:17,  8.26s/it]
 86%|████████▋ | 32748/37962 [75:49:38<12:05:53,  8.35s/it]
 86%|████████▋ | 32749/37962 [75:49:46<12:02:39,  8.32s/it]
 86%|████████▋ | 32750/37962 [75:49:54<12:01:01,  8.30s/it]
                                                           
{'loss': 1.1676, 'grad_norm': 10.880586624145508, 'learning_rate': 9.724613527506287e-07, 'epoch': 0.86}

 86%|████████▋ | 32750/37962 [75:49:54<12:01:01,  8.30s/it]
 86%|████████▋ | 32751/37962 [75:50:03<12:01:06,  8.30s/it]
 86%|████████▋ | 32752/37962 [75:50:11<12:00:32,  8.30s/it]
 86%|████████▋ | 32753/37962 [75:50:19<11:56:33,  8.25s/it]
 86%|████████▋ | 32754/37962 [75:50:28<11:58:17,  8.28s/it]
 86%|████████▋ | 32755/37962 [75:50:36<11:53:52,  8.23s/it]
 86%|████████▋ | 32756/37962 [75:50:44<11:57:44,  8.27s/it]
 86%|████████▋ | 32757/37962 [75:50:52<11:57:50,  8.27s/it]
 86%|████████▋ | 32758/37962 [75:51:01<11:59:30,  8.30s/it]
 86%|████████▋ | 32759/37962 [75:51:09<12:00:21,  8.31s/it]
 86%|████████▋ | 32760/37962 [75:51:17<12:00:38,  8.31s/it]
                                                           
{'loss': 1.119, 'grad_norm': 9.771432876586914, 'learning_rate': 9.68794705260655e-07, 'epoch': 0.86}

 86%|████████▋ | 32760/37962 [75:51:17<12:00:38,  8.31s/it]
 86%|████████▋ | 32761/37962 [75:51:26<12:00:09,  8.31s/it]
 86%|████████▋ | 32762/37962 [75:51:34<12:00:51,  8.32s/it]
 86%|████████▋ | 32763/37962 [75:51:42<12:00:10,  8.31s/it]
 86%|████████▋ | 32764/37962 [75:51:51<12:02:05,  8.34s/it]
 86%|████████▋ | 32765/37962 [75:51:59<12:02:29,  8.34s/it]
 86%|████████▋ | 32766/37962 [75:52:07<11:58:05,  8.29s/it]
 86%|████████▋ | 32767/37962 [75:52:15<11:46:01,  8.15s/it]
 86%|████████▋ | 32768/37962 [75:52:23<11:48:10,  8.18s/it]
 86%|████████▋ | 32769/37962 [75:52:32<11:53:13,  8.24s/it]
 86%|████████▋ | 32770/37962 [75:52:40<11:46:02,  8.16s/it]
                                                           
{'loss': 1.2368, 'grad_norm': 15.077018737792969, 'learning_rate': 9.651346314285614e-07, 'epoch': 0.86}

 86%|████████▋ | 32770/37962 [75:52:40<11:46:02,  8.16s/it]
 86%|████████▋ | 32771/37962 [75:52:48<11:49:02,  8.20s/it]
 86%|████████▋ | 32772/37962 [75:52:56<11:50:28,  8.21s/it]
 86%|████████▋ | 32773/37962 [75:53:05<11:55:41,  8.28s/it]
 86%|████████▋ | 32774/37962 [75:53:13<11:57:54,  8.30s/it]
 86%|████████▋ | 32775/37962 [75:53:21<11:55:59,  8.28s/it]
 86%|████████▋ | 32776/37962 [75:53:29<11:55:48,  8.28s/it]
 86%|████████▋ | 32777/37962 [75:53:38<11:56:12,  8.29s/it]
 86%|████████▋ | 32778/37962 [75:53:46<11:47:16,  8.19s/it]
 86%|████████▋ | 32779/37962 [75:53:54<11:52:29,  8.25s/it]
 86%|████████▋ | 32780/37962 [75:54:02<11:53:54,  8.27s/it]
                                                           
{'loss': 1.2019, 'grad_norm': 15.520211219787598, 'learning_rate': 9.614811339184515e-07, 'epoch': 0.86}

 86%|████████▋ | 32780/37962 [75:54:02<11:53:54,  8.27s/it]
 86%|████████▋ | 32781/37962 [75:54:11<11:56:09,  8.29s/it]
 86%|████████▋ | 32782/37962 [75:54:19<11:48:07,  8.20s/it]
 86%|████████▋ | 32783/37962 [75:54:27<11:47:14,  8.19s/it]
 86%|████████▋ | 32784/37962 [75:54:35<11:54:54,  8.28s/it]
 86%|████████▋ | 32785/37962 [75:54:43<11:47:19,  8.20s/it]
 86%|████████▋ | 32786/37962 [75:54:52<11:49:23,  8.22s/it]
 86%|████████▋ | 32787/37962 [75:55:00<11:50:54,  8.24s/it]
 86%|████████▋ | 32788/37962 [75:55:08<11:55:14,  8.29s/it]
 86%|████████▋ | 32789/37962 [75:55:17<11:54:51,  8.29s/it]
 86%|████████▋ | 32790/37962 [75:55:25<11:56:18,  8.31s/it]
                                                           
{'loss': 1.2731, 'grad_norm': 7.021361351013184, 'learning_rate': 9.578342153896403e-07, 'epoch': 0.86}

 86%|████████▋ | 32790/37962 [75:55:25<11:56:18,  8.31s/it]
 86%|████████▋ | 32791/37962 [75:55:33<11:58:12,  8.33s/it]
 86%|████████▋ | 32792/37962 [75:55:42<11:57:47,  8.33s/it]
 86%|████████▋ | 32793/37962 [75:55:50<11:58:14,  8.34s/it]
 86%|████████▋ | 32794/37962 [75:55:58<11:59:42,  8.36s/it]
 86%|████████▋ | 32795/37962 [75:56:07<11:57:34,  8.33s/it]
 86%|████████▋ | 32796/37962 [75:56:15<11:54:30,  8.30s/it]
 86%|████████▋ | 32797/37962 [75:56:23<11:53:35,  8.29s/it]
 86%|████████▋ | 32798/37962 [75:56:31<11:48:07,  8.23s/it]
 86%|████████▋ | 32799/37962 [75:56:39<11:41:00,  8.15s/it]
 86%|████████▋ | 32800/37962 [75:56:48<11:46:34,  8.21s/it]
                                                           
{'loss': 1.1596, 'grad_norm': 10.737369537353516, 'learning_rate': 9.541938784966598e-07, 'epoch': 0.86}

 86%|████████▋ | 32800/37962 [75:56:48<11:46:34,  8.21s/it]
 86%|████████▋ | 32801/37962 [75:56:56<11:46:54,  8.22s/it]
 86%|████████▋ | 32802/37962 [75:57:04<11:51:31,  8.27s/it]
 86%|████████▋ | 32803/37962 [75:57:13<11:51:45,  8.28s/it]
 86%|████████▋ | 32804/37962 [75:57:21<11:49:42,  8.26s/it]
 86%|████████▋ | 32805/37962 [75:57:29<11:48:50,  8.25s/it]
 86%|████████▋ | 32806/37962 [75:57:37<11:47:05,  8.23s/it]
 86%|████████▋ | 32807/37962 [75:57:45<11:48:34,  8.25s/it]
 86%|████████▋ | 32808/37962 [75:57:54<11:46:24,  8.22s/it]
 86%|████████▋ | 32809/37962 [75:58:02<11:49:24,  8.26s/it]
 86%|████████▋ | 32810/37962 [75:58:10<11:50:39,  8.28s/it]
                                                           
{'loss': 1.1587, 'grad_norm': 8.75694465637207, 'learning_rate': 9.505601258892483e-07, 'epoch': 0.86}

 86%|████████▋ | 32810/37962 [75:58:10<11:50:39,  8.28s/it]
 86%|████████▋ | 32811/37962 [75:58:19<11:50:48,  8.28s/it]
 86%|████████▋ | 32812/37962 [75:58:27<11:44:31,  8.21s/it]
 86%|████████▋ | 32813/37962 [75:58:35<11:50:30,  8.28s/it]
 86%|████████▋ | 32814/37962 [75:58:43<11:53:17,  8.31s/it]
 86%|████████▋ | 32815/37962 [75:58:52<11:56:18,  8.35s/it]
 86%|████████▋ | 32816/37962 [75:59:00<11:57:29,  8.37s/it]
 86%|████████▋ | 32817/37962 [75:59:09<11:57:08,  8.36s/it]
 86%|████████▋ | 32818/37962 [75:59:17<11:54:21,  8.33s/it]
 86%|████████▋ | 32819/37962 [75:59:25<11:53:27,  8.32s/it]
 86%|████████▋ | 32820/37962 [75:59:33<11:50:44,  8.29s/it]
                                                           
{'loss': 1.1878, 'grad_norm': 7.977107524871826, 'learning_rate': 9.469329602123512e-07, 'epoch': 0.86}

 86%|████████▋ | 32820/37962 [75:59:33<11:50:44,  8.29s/it]
 86%|████████▋ | 32821/37962 [75:59:42<11:49:39,  8.28s/it]
 86%|████████▋ | 32822/37962 [75:59:50<11:47:22,  8.26s/it]
 86%|████████▋ | 32823/37962 [75:59:58<11:45:43,  8.24s/it]
 86%|████████▋ | 32824/37962 [76:00:06<11:48:05,  8.27s/it]
 86%|████████▋ | 32825/37962 [76:00:15<11:51:09,  8.31s/it]
 86%|████████▋ | 32826/37962 [76:00:23<11:59:58,  8.41s/it]
 86%|████████▋ | 32827/37962 [76:00:32<11:56:46,  8.38s/it]
 86%|████████▋ | 32828/37962 [76:00:40<11:58:42,  8.40s/it]
 86%|████████▋ | 32829/37962 [76:00:48<11:53:47,  8.34s/it]
 86%|████████▋ | 32830/37962 [76:00:57<11:49:05,  8.29s/it]
                                                           
{'loss': 1.2563, 'grad_norm': 19.376012802124023, 'learning_rate': 9.433123841061165e-07, 'epoch': 0.86}

 86%|████████▋ | 32830/37962 [76:00:57<11:49:05,  8.29s/it]
 86%|████████▋ | 32831/37962 [76:01:05<11:50:46,  8.31s/it]
 86%|████████▋ | 32832/37962 [76:01:13<11:43:30,  8.23s/it]
 86%|████████▋ | 32833/37962 [76:01:21<11:48:43,  8.29s/it]
 86%|████████▋ | 32834/37962 [76:01:30<11:46:10,  8.26s/it]
 86%|████████▋ | 32835/37962 [76:01:38<11:48:04,  8.29s/it]
 86%|████████▋ | 32836/37962 [76:01:46<11:46:41,  8.27s/it]
 86%|████████▋ | 32837/37962 [76:01:54<11:44:52,  8.25s/it]
 87%|████████▋ | 32838/37962 [76:02:03<11:46:11,  8.27s/it]
 87%|████████▋ | 32839/37962 [76:02:11<11:47:00,  8.28s/it]
 87%|████████▋ | 32840/37962 [76:02:19<11:47:20,  8.29s/it]
                                                           
{'loss': 1.0984, 'grad_norm': 13.567084312438965, 'learning_rate': 9.396984002059029e-07, 'epoch': 0.87}

 87%|████████▋ | 32840/37962 [76:02:19<11:47:20,  8.29s/it]
 87%|████████▋ | 32841/37962 [76:02:28<11:46:41,  8.28s/it]
 87%|████████▋ | 32842/37962 [76:02:36<11:53:18,  8.36s/it]
 87%|████████▋ | 32843/37962 [76:02:44<11:46:10,  8.28s/it]
 87%|████████▋ | 32844/37962 [76:02:53<11:45:59,  8.28s/it]
 87%|████████▋ | 32845/37962 [76:03:01<11:49:36,  8.32s/it]
 87%|████████▋ | 32846/37962 [76:03:09<11:49:45,  8.32s/it]
 87%|████████▋ | 32847/37962 [76:03:18<11:49:23,  8.32s/it]
 87%|████████▋ | 32848/37962 [76:03:26<11:45:30,  8.28s/it]
 87%|████████▋ | 32849/37962 [76:03:34<11:47:31,  8.30s/it]
 87%|████████▋ | 32850/37962 [76:03:42<11:44:27,  8.27s/it]
                                                           
{'loss': 1.2243, 'grad_norm': 12.21002197265625, 'learning_rate': 9.360910111422627e-07, 'epoch': 0.87}

 87%|████████▋ | 32850/37962 [76:03:42<11:44:27,  8.27s/it]
 87%|████████▋ | 32851/37962 [76:03:51<11:43:11,  8.26s/it]
 87%|████████▋ | 32852/37962 [76:03:59<11:40:39,  8.23s/it]
 87%|████████▋ | 32853/37962 [76:04:07<11:40:27,  8.23s/it]
 87%|████████▋ | 32854/37962 [76:04:15<11:36:45,  8.18s/it]
 87%|████████▋ | 32855/37962 [76:04:23<11:39:16,  8.22s/it]
 87%|████████▋ | 32856/37962 [76:04:32<11:42:21,  8.25s/it]
 87%|████████▋ | 32857/37962 [76:04:40<11:41:45,  8.25s/it]
 87%|████████▋ | 32858/37962 [76:04:48<11:42:19,  8.26s/it]
 87%|████████▋ | 32859/37962 [76:04:56<11:43:10,  8.27s/it]
 87%|████████▋ | 32860/37962 [76:05:05<11:42:35,  8.26s/it]
                                                           
{'loss': 1.1744, 'grad_norm': 14.12449836730957, 'learning_rate': 9.324902195409535e-07, 'epoch': 0.87}

 87%|████████▋ | 32860/37962 [76:05:05<11:42:35,  8.26s/it]
 87%|████████▋ | 32861/37962 [76:05:13<11:47:40,  8.32s/it]
 87%|████████▋ | 32862/37962 [76:05:21<11:47:52,  8.33s/it]
 87%|████████▋ | 32863/37962 [76:05:30<11:55:31,  8.42s/it]
 87%|████████▋ | 32864/37962 [76:05:38<11:43:41,  8.28s/it]
 87%|████████▋ | 32865/37962 [76:05:46<11:42:56,  8.27s/it]
 87%|████████▋ | 32866/37962 [76:05:55<11:49:18,  8.35s/it]
 87%|████████▋ | 32867/37962 [76:06:03<11:53:03,  8.40s/it]
 87%|████████▋ | 32868/37962 [76:06:12<11:51:03,  8.38s/it]
 87%|████████▋ | 32869/37962 [76:06:20<11:48:04,  8.34s/it]
 87%|████████▋ | 32870/37962 [76:06:28<11:50:00,  8.37s/it]
                                                           
{'loss': 1.1798, 'grad_norm': 7.414440631866455, 'learning_rate': 9.288960280229331e-07, 'epoch': 0.87}

 87%|████████▋ | 32870/37962 [76:06:28<11:50:00,  8.37s/it]
 87%|████████▋ | 32871/37962 [76:06:37<11:47:37,  8.34s/it]
 87%|████████▋ | 32872/37962 [76:06:45<11:44:52,  8.31s/it]
 87%|████████▋ | 32873/37962 [76:06:53<11:41:46,  8.27s/it]
 87%|████████▋ | 32874/37962 [76:07:01<11:42:32,  8.28s/it]
 87%|████████▋ | 32875/37962 [76:07:10<11:44:59,  8.32s/it]
 87%|████████▋ | 32876/37962 [76:07:18<11:44:03,  8.31s/it]
 87%|████████▋ | 32877/37962 [76:07:26<11:45:28,  8.32s/it]
 87%|████████▋ | 32878/37962 [76:07:35<11:47:19,  8.35s/it]
 87%|████████▋ | 32879/37962 [76:07:43<11:44:40,  8.32s/it]
 87%|████████▋ | 32880/37962 [76:07:51<11:42:54,  8.30s/it]
                                                           
{'loss': 1.0936, 'grad_norm': 7.436563491821289, 'learning_rate': 9.253084392043466e-07, 'epoch': 0.87}

 87%|████████▋ | 32880/37962 [76:07:51<11:42:54,  8.30s/it]
 87%|████████▋ | 32881/37962 [76:08:00<11:40:08,  8.27s/it]
 87%|████████▋ | 32882/37962 [76:08:08<11:43:22,  8.31s/it]
 87%|████████▋ | 32883/37962 [76:08:16<11:43:30,  8.31s/it]
 87%|████████▋ | 32884/37962 [76:08:24<11:33:45,  8.20s/it]
 87%|████████▋ | 32885/37962 [76:08:33<11:38:00,  8.25s/it]
 87%|████████▋ | 32886/37962 [76:08:41<11:41:06,  8.29s/it]
 87%|████████▋ | 32887/37962 [76:08:49<11:33:20,  8.20s/it]
 87%|████████▋ | 32888/37962 [76:08:57<11:34:00,  8.21s/it]
 87%|████████▋ | 32889/37962 [76:09:05<11:36:52,  8.24s/it]
 87%|████████▋ | 32890/37962 [76:09:14<11:36:37,  8.24s/it]
                                                           
{'loss': 1.2198, 'grad_norm': 11.315138816833496, 'learning_rate': 9.217274556965372e-07, 'epoch': 0.87}

 87%|████████▋ | 32890/37962 [76:09:14<11:36:37,  8.24s/it]
 87%|████████▋ | 32891/37962 [76:09:22<11:38:59,  8.27s/it]
 87%|████████▋ | 32892/37962 [76:09:30<11:32:04,  8.19s/it]
 87%|████████▋ | 32893/37962 [76:09:38<11:33:23,  8.21s/it]
 87%|████████▋ | 32894/37962 [76:09:46<11:32:56,  8.20s/it]
 87%|████████▋ | 32895/37962 [76:09:55<11:32:38,  8.20s/it]
 87%|████████▋ | 32896/37962 [76:10:03<11:34:19,  8.22s/it]
 87%|████████▋ | 32897/37962 [76:10:11<11:35:13,  8.24s/it]
 87%|████████▋ | 32898/37962 [76:10:19<11:30:53,  8.19s/it]
 87%|████████▋ | 32899/37962 [76:10:28<11:42:06,  8.32s/it]
 87%|████████▋ | 32900/37962 [76:10:37<11:49:31,  8.41s/it]
                                                           
{'loss': 1.1023, 'grad_norm': 7.812894344329834, 'learning_rate': 9.181530801060445e-07, 'epoch': 0.87}

 87%|████████▋ | 32900/37962 [76:10:37<11:49:31,  8.41s/it]
 87%|████████▋ | 32901/37962 [76:10:45<11:46:46,  8.38s/it]
 87%|████████▋ | 32902/37962 [76:10:53<11:42:23,  8.33s/it]
 87%|████████▋ | 32903/37962 [76:11:01<11:42:23,  8.33s/it]
 87%|████████▋ | 32904/37962 [76:11:10<11:40:05,  8.30s/it]
 87%|████████▋ | 32905/37962 [76:11:18<11:42:16,  8.33s/it]
 87%|████████▋ | 32906/37962 [76:11:26<11:37:52,  8.28s/it]
 87%|████████▋ | 32907/37962 [76:11:35<11:38:41,  8.29s/it]
 87%|████████▋ | 32908/37962 [76:11:50<14:39:24, 10.44s/it]
 87%|████████▋ | 32909/37962 [76:11:58<13:43:06,  9.77s/it]
 87%|████████▋ | 32910/37962 [76:12:07<13:08:39,  9.37s/it]
                                                           
{'loss': 1.1526, 'grad_norm': 6.790420055389404, 'learning_rate': 9.145853150345907e-07, 'epoch': 0.87}

 87%|████████▋ | 32910/37962 [76:12:07<13:08:39,  9.37s/it]
 87%|████████▋ | 32911/37962 [76:12:15<12:36:50,  8.99s/it]
 87%|████████▋ | 32912/37962 [76:12:23<12:16:35,  8.75s/it]
 87%|████████▋ | 32913/37962 [76:12:31<11:55:51,  8.51s/it]
 87%|████████▋ | 32914/37962 [76:12:39<11:50:02,  8.44s/it]
 87%|████████▋ | 32915/37962 [76:12:47<11:27:27,  8.17s/it]
 87%|████████▋ | 32916/37962 [76:12:54<11:16:11,  8.04s/it]
 87%|████████▋ | 32917/37962 [76:13:03<11:23:45,  8.13s/it]
 87%|████████▋ | 32918/37962 [76:13:11<11:28:59,  8.20s/it]
 87%|████████▋ | 32919/37962 [76:13:19<11:31:03,  8.22s/it]
 87%|████████▋ | 32920/37962 [76:13:28<11:31:26,  8.23s/it]
                                                           
{'loss': 1.1071, 'grad_norm': 11.079233169555664, 'learning_rate': 9.11024163079094e-07, 'epoch': 0.87}

 87%|████████▋ | 32920/37962 [76:13:28<11:31:26,  8.23s/it]
 87%|████████▋ | 32921/37962 [76:13:36<11:29:48,  8.21s/it]
 87%|████████▋ | 32922/37962 [76:13:44<11:33:56,  8.26s/it]
 87%|████████▋ | 32923/37962 [76:13:53<11:37:33,  8.31s/it]
 87%|████████▋ | 32924/37962 [76:14:01<11:39:42,  8.33s/it]
 87%|████████▋ | 32925/37962 [76:14:09<11:41:22,  8.35s/it]
 87%|████████▋ | 32926/37962 [76:14:18<11:36:44,  8.30s/it]
 87%|████████▋ | 32927/37962 [76:14:26<11:37:49,  8.32s/it]
 87%|████████▋ | 32928/37962 [76:14:34<11:38:21,  8.32s/it]
 87%|████████▋ | 32929/37962 [76:14:43<11:36:15,  8.30s/it]
 87%|████████▋ | 32930/37962 [76:14:51<11:35:20,  8.29s/it]
                                                           
{'loss': 1.1058, 'grad_norm': 7.48032808303833, 'learning_rate': 9.07469626831653e-07, 'epoch': 0.87}

 87%|████████▋ | 32930/37962 [76:14:51<11:35:20,  8.29s/it]
 87%|████████▋ | 32931/37962 [76:14:59<11:34:10,  8.28s/it]
 87%|████████▋ | 32932/37962 [76:15:07<11:37:18,  8.32s/it]
 87%|████████▋ | 32933/37962 [76:15:16<11:34:31,  8.29s/it]
 87%|████████▋ | 32934/37962 [76:15:24<11:32:45,  8.27s/it]
 87%|████████▋ | 32935/37962 [76:15:32<11:27:27,  8.21s/it]
 87%|████████▋ | 32936/37962 [76:15:40<11:31:02,  8.25s/it]
 87%|████████▋ | 32937/37962 [76:15:48<11:29:26,  8.23s/it]
 87%|████████▋ | 32938/37962 [76:15:57<11:27:33,  8.21s/it]
 87%|████████▋ | 32939/37962 [76:16:05<11:18:33,  8.11s/it]
 87%|████████▋ | 32940/37962 [76:16:12<11:13:46,  8.05s/it]
                                                           
{'loss': 1.1675, 'grad_norm': 9.705724716186523, 'learning_rate': 9.039217088795515e-07, 'epoch': 0.87}

 87%|████████▋ | 32940/37962 [76:16:12<11:13:46,  8.05s/it]
 87%|████████▋ | 32941/37962 [76:16:20<11:09:17,  8.00s/it]
 87%|████████▋ | 32942/37962 [76:16:29<11:16:03,  8.08s/it]
 87%|████████▋ | 32943/37962 [76:16:37<11:23:04,  8.17s/it]
 87%|████████▋ | 32944/37962 [76:16:45<11:29:36,  8.25s/it]
 87%|████████▋ | 32945/37962 [76:16:54<11:30:38,  8.26s/it]
 87%|████████▋ | 32946/37962 [76:17:02<11:29:34,  8.25s/it]
 87%|████████▋ | 32947/37962 [76:17:10<11:27:36,  8.23s/it]
 87%|████████▋ | 32948/37962 [76:17:18<11:28:00,  8.23s/it]
 87%|████████▋ | 32949/37962 [76:17:27<11:28:56,  8.25s/it]
 87%|████████▋ | 32950/37962 [76:17:35<11:36:46,  8.34s/it]
                                                           
{'loss': 1.1198, 'grad_norm': 5.903825759887695, 'learning_rate': 9.003804118052617e-07, 'epoch': 0.87}

 87%|████████▋ | 32950/37962 [76:17:35<11:36:46,  8.34s/it]
 87%|████████▋ | 32951/37962 [76:17:43<11:32:46,  8.30s/it]
 87%|████████▋ | 32952/37962 [76:17:52<11:36:36,  8.34s/it]
 87%|████████▋ | 32953/37962 [76:18:00<11:28:11,  8.24s/it]
 87%|████████▋ | 32954/37962 [76:18:08<11:28:26,  8.25s/it]
 87%|████████▋ | 32955/37962 [76:18:16<11:26:55,  8.23s/it]
 87%|████████▋ | 32956/37962 [76:18:25<11:28:44,  8.25s/it]
 87%|████████▋ | 32957/37962 [76:18:33<11:33:36,  8.31s/it]
 87%|████████▋ | 32958/37962 [76:18:41<11:33:38,  8.32s/it]
 87%|████████▋ | 32959/37962 [76:18:49<11:26:24,  8.23s/it]
 87%|████████▋ | 32960/37962 [76:18:58<11:28:24,  8.26s/it]
                                                           
{'loss': 1.2655, 'grad_norm': 11.331876754760742, 'learning_rate': 8.968457381864282e-07, 'epoch': 0.87}

 87%|████████▋ | 32960/37962 [76:18:58<11:28:24,  8.26s/it]
 87%|████████▋ | 32961/37962 [76:19:06<11:28:19,  8.26s/it]
 87%|████████▋ | 32962/37962 [76:19:14<11:29:03,  8.27s/it]
 87%|████████▋ | 32963/37962 [76:19:23<11:29:23,  8.27s/it]
 87%|████████▋ | 32964/37962 [76:19:31<11:29:58,  8.28s/it]
 87%|████████▋ | 32965/37962 [76:19:39<11:22:53,  8.20s/it]
 87%|████████▋ | 32966/37962 [76:19:47<11:26:37,  8.25s/it]
 87%|████████▋ | 32967/37962 [76:19:55<11:25:41,  8.24s/it]
 87%|████████▋ | 32968/37962 [76:20:04<11:26:04,  8.24s/it]
 87%|████████▋ | 32969/37962 [76:20:12<11:21:23,  8.19s/it]
 87%|████████▋ | 32970/37962 [76:20:20<11:26:11,  8.25s/it]
                                                           
{'loss': 1.2225, 'grad_norm': 13.345621109008789, 'learning_rate': 8.933176905958796e-07, 'epoch': 0.87}

 87%|████████▋ | 32970/37962 [76:20:20<11:26:11,  8.25s/it]
 87%|████████▋ | 32971/37962 [76:20:28<11:27:38,  8.27s/it]
 87%|████████▋ | 32972/37962 [76:20:37<11:31:05,  8.31s/it]
 87%|████████▋ | 32973/37962 [76:20:45<11:31:09,  8.31s/it]
 87%|████████▋ | 32974/37962 [76:20:54<11:33:18,  8.34s/it]
 87%|████████▋ | 32975/37962 [76:21:02<11:32:54,  8.34s/it]
 87%|████████▋ | 32976/37962 [76:21:10<11:31:24,  8.32s/it]
 87%|████████▋ | 32977/37962 [76:21:18<11:25:39,  8.25s/it]
 87%|████████▋ | 32978/37962 [76:21:26<11:22:57,  8.22s/it]
 87%|████████▋ | 32979/37962 [76:21:35<11:29:16,  8.30s/it]
 87%|████████▋ | 32980/37962 [76:21:43<11:30:52,  8.32s/it]
                                                           
{'loss': 1.111, 'grad_norm': 9.482664108276367, 'learning_rate': 8.897962716016262e-07, 'epoch': 0.87}

 87%|████████▋ | 32980/37962 [76:21:43<11:30:52,  8.32s/it]
 87%|████████▋ | 32981/37962 [76:21:51<11:24:41,  8.25s/it]
 87%|████████▋ | 32982/37962 [76:22:00<11:23:22,  8.23s/it]
 87%|████████▋ | 32983/37962 [76:22:08<11:25:20,  8.26s/it]
 87%|████████▋ | 32984/37962 [76:22:16<11:23:50,  8.24s/it]
 87%|████████▋ | 32985/37962 [76:22:24<11:15:06,  8.14s/it]
 87%|████████▋ | 32986/37962 [76:22:32<11:18:17,  8.18s/it]
 87%|████████▋ | 32987/37962 [76:22:41<11:28:09,  8.30s/it]
 87%|████████▋ | 32988/37962 [76:22:49<11:26:50,  8.29s/it]
 87%|████████▋ | 32989/37962 [76:22:57<11:26:02,  8.28s/it]
 87%|████████▋ | 32990/37962 [76:23:06<11:25:07,  8.27s/it]
                                                           
{'loss': 1.1319, 'grad_norm': 10.475421905517578, 'learning_rate': 8.862814837668388e-07, 'epoch': 0.87}

 87%|████████▋ | 32990/37962 [76:23:06<11:25:07,  8.27s/it]
 87%|████████▋ | 32991/37962 [76:23:14<11:25:30,  8.27s/it]
 87%|████████▋ | 32992/37962 [76:23:22<11:24:05,  8.26s/it]
 87%|████████▋ | 32993/37962 [76:23:30<11:23:57,  8.26s/it]
 87%|████████▋ | 32994/37962 [76:23:38<11:20:41,  8.22s/it]
 87%|████████▋ | 32995/37962 [76:23:47<11:26:09,  8.29s/it]
 87%|████████▋ | 32996/37962 [76:23:55<11:16:12,  8.17s/it]
 87%|████████▋ | 32997/37962 [76:24:03<11:21:04,  8.23s/it]
 87%|████████▋ | 32998/37962 [76:24:12<11:25:00,  8.28s/it]
 87%|████████▋ | 32999/37962 [76:24:20<11:22:39,  8.25s/it]
 87%|████████▋ | 33000/37962 [76:24:28<11:25:07,  8.28s/it]
                                                           
{'loss': 1.156, 'grad_norm': 12.020146369934082, 'learning_rate': 8.827733296498753e-07, 'epoch': 0.87}

 87%|████████▋ | 33000/37962 [76:24:28<11:25:07,  8.28s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 87%|████████▋ | 33001/37962 [76:25:17<28:10:04, 20.44s/it]
 87%|████████▋ | 33002/37962 [76:25:25<23:05:21, 16.76s/it]
 87%|████████▋ | 33003/37962 [76:25:33<19:26:31, 14.11s/it]
 87%|████████▋ | 33004/37962 [76:25:41<17:03:34, 12.39s/it]
 87%|████████▋ | 33005/37962 [76:25:50<15:22:21, 11.16s/it]
 87%|████████▋ | 33006/37962 [76:25:58<14:05:23, 10.23s/it]
 87%|████████▋ | 33007/37962 [76:26:06<13:15:03,  9.63s/it]
 87%|████████▋ | 33008/37962 [76:26:14<12:41:35,  9.22s/it]
 87%|████████▋ | 33009/37962 [76:26:22<12:11:39,  8.86s/it]
 87%|████████▋ | 33010/37962 [76:26:31<11:56:07,  8.68s/it]
                                                           
{'loss': 1.054, 'grad_norm': 8.262234687805176, 'learning_rate': 8.79271811804262e-07, 'epoch': 0.87}

 87%|████████▋ | 33010/37962 [76:26:31<11:56:07,  8.68s/it]
 87%|████████▋ | 33011/37962 [76:26:39<11:52:55,  8.64s/it]
 87%|████████▋ | 33012/37962 [76:26:47<11:44:25,  8.54s/it]
 87%|████████▋ | 33013/37962 [76:26:55<11:31:41,  8.39s/it]
 87%|████████▋ | 33014/37962 [76:27:04<11:26:44,  8.33s/it]
 87%|████████▋ | 33015/37962 [76:27:12<11:26:35,  8.33s/it]
 87%|████████▋ | 33016/37962 [76:27:20<11:24:45,  8.31s/it]
 87%|████████▋ | 33017/37962 [76:27:28<11:22:57,  8.29s/it]
 87%|████████▋ | 33018/37962 [76:27:37<11:24:45,  8.31s/it]
 87%|████████▋ | 33019/37962 [76:27:45<11:13:57,  8.18s/it]
 87%|████████▋ | 33020/37962 [76:27:52<11:04:34,  8.07s/it]
                                                           
{'loss': 1.2027, 'grad_norm': 7.831429481506348, 'learning_rate': 8.757769327786892e-07, 'epoch': 0.87}

 87%|████████▋ | 33020/37962 [76:27:52<11:04:34,  8.07s/it]
 87%|████████▋ | 33021/37962 [76:28:01<11:08:14,  8.11s/it]
 87%|████████▋ | 33022/37962 [76:28:09<11:20:19,  8.26s/it]
 87%|████████▋ | 33023/37962 [76:28:18<11:18:10,  8.24s/it]
 87%|████████▋ | 33024/37962 [76:28:26<11:21:59,  8.29s/it]
 87%|████████▋ | 33025/37962 [76:28:34<11:27:43,  8.36s/it]
 87%|████████▋ | 33026/37962 [76:28:43<11:23:51,  8.31s/it]
 87%|████████▋ | 33027/37962 [76:28:51<11:20:56,  8.28s/it]
 87%|████████▋ | 33028/37962 [76:28:59<11:14:34,  8.20s/it]
 87%|████████▋ | 33029/37962 [76:29:07<11:17:51,  8.24s/it]
 87%|████████▋ | 33030/37962 [76:29:15<11:17:56,  8.25s/it]
                                                           
{'loss': 1.1941, 'grad_norm': 13.1566162109375, 'learning_rate': 8.722886951170229e-07, 'epoch': 0.87}

 87%|████████▋ | 33030/37962 [76:29:15<11:17:56,  8.25s/it]
 87%|████████▋ | 33031/37962 [76:29:24<11:16:35,  8.23s/it]
 87%|████████▋ | 33032/37962 [76:29:32<11:20:49,  8.29s/it]
 87%|████████▋ | 33033/37962 [76:29:40<11:18:09,  8.26s/it]
 87%|████████▋ | 33034/37962 [76:29:48<11:13:08,  8.20s/it]
 87%|████████▋ | 33035/37962 [76:29:57<11:18:56,  8.27s/it]
 87%|████████▋ | 33036/37962 [76:30:05<11:22:03,  8.31s/it]
 87%|████████▋ | 33037/37962 [76:30:13<11:22:55,  8.32s/it]
 87%|████████▋ | 33038/37962 [76:30:22<11:20:21,  8.29s/it]
 87%|████████▋ | 33039/37962 [76:30:30<11:18:52,  8.27s/it]
 87%|████████▋ | 33040/37962 [76:30:38<11:21:07,  8.30s/it]
                                                           
{'loss': 1.2146, 'grad_norm': 9.970398902893066, 'learning_rate': 8.688071013582889e-07, 'epoch': 0.87}

 87%|████████▋ | 33040/37962 [76:30:38<11:21:07,  8.30s/it]
 87%|████████▋ | 33041/37962 [76:30:47<11:19:06,  8.28s/it]
 87%|████████▋ | 33042/37962 [76:30:55<11:11:45,  8.19s/it]
 87%|████████▋ | 33043/37962 [76:31:03<11:14:42,  8.23s/it]
 87%|████████▋ | 33044/37962 [76:31:11<11:17:59,  8.27s/it]
 87%|████████▋ | 33045/37962 [76:31:19<11:16:36,  8.26s/it]
 87%|████████▋ | 33046/37962 [76:31:27<11:10:03,  8.18s/it]
 87%|████████▋ | 33047/37962 [76:31:35<10:59:06,  8.05s/it]
 87%|████████▋ | 33048/37962 [76:31:44<11:05:59,  8.13s/it]
 87%|████████▋ | 33049/37962 [76:31:52<11:03:59,  8.11s/it]
 87%|████████▋ | 33050/37962 [76:32:00<11:07:21,  8.15s/it]
                                                           
{'loss': 1.1333, 'grad_norm': 11.162369728088379, 'learning_rate': 8.653321540366754e-07, 'epoch': 0.87}

 87%|████████▋ | 33050/37962 [76:32:00<11:07:21,  8.15s/it]
 87%|████████▋ | 33051/37962 [76:32:08<11:07:24,  8.15s/it]
 87%|████████▋ | 33052/37962 [76:32:24<14:22:35, 10.54s/it]
 87%|████████▋ | 33053/37962 [76:32:32<13:24:50,  9.84s/it]
 87%|████████▋ | 33054/37962 [76:32:41<12:48:36,  9.40s/it]
 87%|████████▋ | 33055/37962 [76:32:49<12:18:44,  9.03s/it]
 87%|████████▋ | 33056/37962 [76:32:57<12:02:27,  8.84s/it]
 87%|████████▋ | 33057/37962 [76:33:05<11:47:27,  8.65s/it]
 87%|████████▋ | 33058/37962 [76:33:14<11:37:56,  8.54s/it]
 87%|████████▋ | 33059/37962 [76:33:22<11:33:04,  8.48s/it]
 87%|████████▋ | 33060/37962 [76:33:30<11:30:37,  8.45s/it]
                                                           
{'loss': 1.1461, 'grad_norm': 11.680798530578613, 'learning_rate': 8.618638556815429e-07, 'epoch': 0.87}

 87%|████████▋ | 33060/37962 [76:33:30<11:30:37,  8.45s/it]
 87%|████████▋ | 33061/37962 [76:33:39<11:27:19,  8.41s/it]
 87%|████████▋ | 33062/37962 [76:33:47<11:24:07,  8.38s/it]
 87%|████████▋ | 33063/37962 [76:33:55<11:19:30,  8.32s/it]
 87%|████████▋ | 33064/37962 [76:34:03<11:09:16,  8.20s/it]
 87%|████████▋ | 33065/37962 [76:34:12<11:15:30,  8.28s/it]
 87%|████████▋ | 33066/37962 [76:34:20<11:14:54,  8.27s/it]
 87%|████████▋ | 33067/37962 [76:34:28<11:16:35,  8.29s/it]
 87%|████████▋ | 33068/37962 [76:34:36<11:15:39,  8.28s/it]
 87%|████████▋ | 33069/37962 [76:34:45<11:16:56,  8.30s/it]
 87%|████████▋ | 33070/37962 [76:34:53<11:15:20,  8.28s/it]
                                                           
{'loss': 1.157, 'grad_norm': 13.578390121459961, 'learning_rate': 8.584022088174004e-07, 'epoch': 0.87}

 87%|████████▋ | 33070/37962 [76:34:53<11:15:20,  8.28s/it]
 87%|████████▋ | 33071/37962 [76:35:01<11:14:10,  8.27s/it]
 87%|████████▋ | 33072/37962 [76:35:09<11:09:59,  8.22s/it]
 87%|████████▋ | 33073/37962 [76:35:18<11:12:14,  8.25s/it]
 87%|████████▋ | 33074/37962 [76:35:26<11:11:36,  8.24s/it]
 87%|████████▋ | 33075/37962 [76:35:34<11:09:50,  8.22s/it]
 87%|████████▋ | 33076/37962 [76:35:42<11:09:36,  8.22s/it]
 87%|████████▋ | 33077/37962 [76:35:51<11:14:22,  8.28s/it]
 87%|████████▋ | 33078/37962 [76:35:59<11:13:39,  8.28s/it]
 87%|████████▋ | 33079/37962 [76:36:07<11:12:52,  8.27s/it]
 87%|████████▋ | 33080/37962 [76:36:16<11:12:55,  8.27s/it]
                                                           
{'loss': 1.1288, 'grad_norm': 10.638934135437012, 'learning_rate': 8.549472159639216e-07, 'epoch': 0.87}

 87%|████████▋ | 33080/37962 [76:36:16<11:12:55,  8.27s/it]
 87%|████████▋ | 33081/37962 [76:36:24<11:15:32,  8.30s/it]
 87%|████████▋ | 33082/37962 [76:36:32<11:14:08,  8.29s/it]
 87%|████████▋ | 33083/37962 [76:36:40<11:14:08,  8.29s/it]
 87%|████████▋ | 33084/37962 [76:36:48<11:05:50,  8.19s/it]
 87%|████████▋ | 33085/37962 [76:36:57<11:06:58,  8.21s/it]
 87%|████████▋ | 33086/37962 [76:37:05<11:11:12,  8.26s/it]
 87%|████████▋ | 33087/37962 [76:37:13<11:11:10,  8.26s/it]
 87%|████████▋ | 33088/37962 [76:37:21<11:06:04,  8.20s/it]
 87%|████████▋ | 33089/37962 [76:37:30<11:08:12,  8.23s/it]
 87%|████████▋ | 33090/37962 [76:37:38<11:08:14,  8.23s/it]
                                                           
{'loss': 1.1521, 'grad_norm': 8.945267677307129, 'learning_rate': 8.514988796359402e-07, 'epoch': 0.87}

 87%|████████▋ | 33090/37962 [76:37:38<11:08:14,  8.23s/it]
 87%|████████▋ | 33091/37962 [76:37:46<11:06:35,  8.21s/it]
 87%|████████▋ | 33092/37962 [76:37:54<11:08:55,  8.24s/it]
 87%|████████▋ | 33093/37962 [76:38:02<11:04:52,  8.19s/it]
 87%|████████▋ | 33094/37962 [76:38:11<11:02:16,  8.16s/it]
 87%|████████▋ | 33095/37962 [76:38:19<11:05:34,  8.21s/it]
 87%|████████▋ | 33096/37962 [76:38:27<10:55:17,  8.08s/it]
 87%|████████▋ | 33097/37962 [76:38:35<11:00:51,  8.15s/it]
 87%|████████▋ | 33098/37962 [76:38:43<11:02:29,  8.17s/it]
 87%|████████▋ | 33099/37962 [76:38:51<11:03:05,  8.18s/it]
 87%|████████▋ | 33100/37962 [76:39:00<11:10:34,  8.28s/it]
                                                           
{'loss': 1.1185, 'grad_norm': 7.636659145355225, 'learning_rate': 8.480572023434375e-07, 'epoch': 0.87}

 87%|████████▋ | 33100/37962 [76:39:00<11:10:34,  8.28s/it]
 87%|████████▋ | 33101/37962 [76:39:08<11:07:04,  8.23s/it]
 87%|████████▋ | 33102/37962 [76:39:16<11:08:44,  8.26s/it]
 87%|████████▋ | 33103/37962 [76:39:25<11:10:47,  8.28s/it]
 87%|████████▋ | 33104/37962 [76:39:33<11:13:40,  8.32s/it]
 87%|████████▋ | 33105/37962 [76:39:41<11:08:02,  8.25s/it]
 87%|████████▋ | 33106/37962 [76:39:49<11:08:29,  8.26s/it]
 87%|████████▋ | 33107/37962 [76:39:58<11:04:03,  8.21s/it]
 87%|████████▋ | 33108/37962 [76:40:06<11:00:56,  8.17s/it]
 87%|████████▋ | 33109/37962 [76:40:14<11:01:18,  8.18s/it]
 87%|████████▋ | 33110/37962 [76:40:22<11:02:41,  8.19s/it]
                                                           
{'loss': 1.166, 'grad_norm': 10.493993759155273, 'learning_rate': 8.446221865915483e-07, 'epoch': 0.87}

 87%|████████▋ | 33110/37962 [76:40:22<11:02:41,  8.19s/it]
 87%|████████▋ | 33111/37962 [76:40:30<11:06:53,  8.25s/it]
 87%|████████▋ | 33112/37962 [76:40:39<11:14:49,  8.35s/it]
 87%|████████▋ | 33113/37962 [76:40:47<11:08:02,  8.27s/it]
 87%|████████▋ | 33114/37962 [76:40:55<11:07:12,  8.26s/it]
 87%|████████▋ | 33115/37962 [76:41:04<11:07:41,  8.27s/it]
 87%|████████▋ | 33116/37962 [76:41:12<11:12:29,  8.33s/it]
 87%|████████▋ | 33117/37962 [76:41:20<11:09:20,  8.29s/it]
 87%|████████▋ | 33118/37962 [76:41:28<11:03:27,  8.22s/it]
 87%|████████▋ | 33119/37962 [76:41:37<11:06:54,  8.26s/it]
 87%|████████▋ | 33120/37962 [76:41:45<11:07:28,  8.27s/it]
                                                           
{'loss': 1.1564, 'grad_norm': 11.306140899658203, 'learning_rate': 8.411938348805648e-07, 'epoch': 0.87}

 87%|████████▋ | 33120/37962 [76:41:45<11:07:28,  8.27s/it]
 87%|████████▋ | 33121/37962 [76:41:53<11:03:14,  8.22s/it]
 87%|████████▋ | 33122/37962 [76:42:01<11:04:49,  8.24s/it]
 87%|████████▋ | 33123/37962 [76:42:10<11:03:31,  8.23s/it]
 87%|████████▋ | 33124/37962 [76:42:18<11:06:57,  8.27s/it]
 87%|████████▋ | 33125/37962 [76:42:26<11:07:25,  8.28s/it]
 87%|████████▋ | 33126/37962 [76:42:35<11:17:57,  8.41s/it]
 87%|████████▋ | 33127/37962 [76:42:43<11:12:00,  8.34s/it]
 87%|████████▋ | 33128/37962 [76:42:51<10:57:38,  8.16s/it]
 87%|████████▋ | 33129/37962 [76:42:59<10:59:38,  8.19s/it]
 87%|████████▋ | 33130/37962 [76:43:07<10:56:24,  8.15s/it]
                                                           
{'loss': 1.1599, 'grad_norm': 9.309571266174316, 'learning_rate': 8.377721497059222e-07, 'epoch': 0.87}

 87%|████████▋ | 33130/37962 [76:43:07<10:56:24,  8.15s/it]
 87%|████████▋ | 33131/37962 [76:43:15<10:54:41,  8.13s/it]
 87%|████████▋ | 33132/37962 [76:43:24<11:03:18,  8.24s/it]
 87%|████████▋ | 33133/37962 [76:43:32<10:57:54,  8.17s/it]
 87%|████████▋ | 33134/37962 [76:43:40<11:03:32,  8.25s/it]
 87%|████████▋ | 33135/37962 [76:43:48<10:59:36,  8.20s/it]
 87%|████████▋ | 33136/37962 [76:43:57<11:00:03,  8.21s/it]
 87%|████████▋ | 33137/37962 [76:44:05<11:00:25,  8.21s/it]
 87%|████████▋ | 33138/37962 [76:44:13<11:02:20,  8.24s/it]
 87%|████████▋ | 33139/37962 [76:44:21<11:04:49,  8.27s/it]
 87%|████████▋ | 33140/37962 [76:44:29<10:55:43,  8.16s/it]
                                                           
{'loss': 1.1623, 'grad_norm': 8.761427879333496, 'learning_rate': 8.34357133558209e-07, 'epoch': 0.87}

 87%|████████▋ | 33140/37962 [76:44:29<10:55:43,  8.16s/it]
 87%|████████▋ | 33141/37962 [76:44:38<11:10:44,  8.35s/it]
 87%|████████▋ | 33142/37962 [76:44:46<11:06:48,  8.30s/it]
 87%|████████▋ | 33143/37962 [76:44:54<10:57:15,  8.18s/it]
 87%|████████▋ | 33144/37962 [76:45:02<10:58:58,  8.21s/it]
 87%|████████▋ | 33145/37962 [76:45:11<11:02:32,  8.25s/it]
 87%|████████▋ | 33146/37962 [76:45:19<10:59:49,  8.22s/it]
 87%|████████▋ | 33147/37962 [76:45:27<11:00:27,  8.23s/it]
 87%|████████▋ | 33148/37962 [76:45:36<11:03:45,  8.27s/it]
 87%|████████▋ | 33149/37962 [76:45:44<11:04:59,  8.29s/it]
 87%|████████▋ | 33150/37962 [76:45:52<11:00:01,  8.23s/it]
                                                           
{'loss': 1.1039, 'grad_norm': 8.290459632873535, 'learning_rate': 8.309487889231549e-07, 'epoch': 0.87}

 87%|████████▋ | 33150/37962 [76:45:52<11:00:01,  8.23s/it]
 87%|████████▋ | 33151/37962 [76:46:00<10:59:10,  8.22s/it]
 87%|████████▋ | 33152/37962 [76:46:08<10:59:42,  8.23s/it]
 87%|████████▋ | 33153/37962 [76:46:17<11:05:50,  8.31s/it]
 87%|████████▋ | 33154/37962 [76:46:25<11:06:22,  8.32s/it]
 87%|████████▋ | 33155/37962 [76:46:33<11:01:56,  8.26s/it]
 87%|████████▋ | 33156/37962 [76:46:42<11:02:05,  8.27s/it]
 87%|████████▋ | 33157/37962 [76:46:50<10:52:52,  8.15s/it]
 87%|████████▋ | 33158/37962 [76:46:58<10:56:43,  8.20s/it]
 87%|████████▋ | 33159/37962 [76:47:06<11:04:15,  8.30s/it]
 87%|████████▋ | 33160/37962 [76:47:15<11:00:58,  8.26s/it]
                                                           
{'loss': 1.1608, 'grad_norm': 6.5263590812683105, 'learning_rate': 8.275471182816319e-07, 'epoch': 0.87}

 87%|████████▋ | 33160/37962 [76:47:15<11:00:58,  8.26s/it]
 87%|████████▋ | 33161/37962 [76:47:23<10:58:48,  8.23s/it]
 87%|████████▋ | 33162/37962 [76:47:31<11:04:10,  8.30s/it]
 87%|████████▋ | 33163/37962 [76:47:39<11:02:16,  8.28s/it]
 87%|████████▋ | 33164/37962 [76:47:48<11:00:28,  8.26s/it]
 87%|████████▋ | 33165/37962 [76:47:56<10:59:21,  8.25s/it]
 87%|████████▋ | 33166/37962 [76:48:04<11:00:34,  8.26s/it]
 87%|████████▋ | 33167/37962 [76:48:13<11:03:42,  8.30s/it]
 87%|████████▋ | 33168/37962 [76:48:21<11:00:35,  8.27s/it]
 87%|████████▋ | 33169/37962 [76:48:29<11:02:17,  8.29s/it]
 87%|████████▋ | 33170/37962 [76:48:37<10:55:05,  8.20s/it]
                                                           
{'loss': 1.1158, 'grad_norm': 6.421176910400391, 'learning_rate': 8.241521241096595e-07, 'epoch': 0.87}

 87%|████████▋ | 33170/37962 [76:48:37<10:55:05,  8.20s/it]
 87%|████████▋ | 33171/37962 [76:48:45<10:55:49,  8.21s/it]
 87%|████████▋ | 33172/37962 [76:48:54<10:56:48,  8.23s/it]
 87%|████████▋ | 33173/37962 [76:49:02<10:55:08,  8.21s/it]
 87%|████████▋ | 33174/37962 [76:49:10<10:50:17,  8.15s/it]
 87%|████████▋ | 33175/37962 [76:49:18<10:53:44,  8.19s/it]
 87%|████████▋ | 33176/37962 [76:49:26<10:52:46,  8.18s/it]
 87%|████████▋ | 33177/37962 [76:49:34<10:54:01,  8.20s/it]
 87%|████████▋ | 33178/37962 [76:49:42<10:43:52,  8.08s/it]
 87%|████████▋ | 33179/37962 [76:49:51<10:48:48,  8.14s/it]
 87%|████████▋ | 33180/37962 [76:49:59<10:53:23,  8.20s/it]
                                                           
{'loss': 1.1064, 'grad_norm': 10.207563400268555, 'learning_rate': 8.207638088783976e-07, 'epoch': 0.87}

 87%|████████▋ | 33180/37962 [76:49:59<10:53:23,  8.20s/it]
 87%|████████▋ | 33181/37962 [76:50:07<10:53:54,  8.21s/it]
 87%|████████▋ | 33182/37962 [76:50:15<10:54:57,  8.22s/it]
 87%|████████▋ | 33183/37962 [76:50:24<10:56:38,  8.24s/it]
 87%|████████▋ | 33184/37962 [76:50:32<10:57:14,  8.25s/it]
 87%|████████▋ | 33185/37962 [76:50:40<10:56:40,  8.25s/it]
 87%|████████▋ | 33186/37962 [76:50:48<10:56:01,  8.24s/it]
 87%|████████▋ | 33187/37962 [76:50:57<10:57:27,  8.26s/it]
 87%|████████▋ | 33188/37962 [76:51:05<10:57:25,  8.26s/it]
 87%|████████▋ | 33189/37962 [76:51:13<10:55:10,  8.24s/it]
 87%|████████▋ | 33190/37962 [76:51:21<10:56:36,  8.26s/it]
                                                           
{'loss': 1.1368, 'grad_norm': 7.8762736320495605, 'learning_rate': 8.173821750541389e-07, 'epoch': 0.87}

 87%|████████▋ | 33190/37962 [76:51:21<10:56:36,  8.26s/it]
 87%|████████▋ | 33191/37962 [76:51:30<10:57:13,  8.27s/it]
 87%|████████▋ | 33192/37962 [76:51:38<10:55:41,  8.25s/it]
 87%|████████▋ | 33193/37962 [76:51:46<10:54:33,  8.24s/it]
 87%|████████▋ | 33194/37962 [76:51:54<10:51:02,  8.19s/it]
 87%|████████▋ | 33195/37962 [76:52:02<10:47:37,  8.15s/it]
 87%|████████▋ | 33196/37962 [76:52:11<10:53:43,  8.23s/it]
 87%|████████▋ | 33197/37962 [76:52:19<10:57:40,  8.28s/it]
 87%|████████▋ | 33198/37962 [76:52:27<10:55:52,  8.26s/it]
 87%|████████▋ | 33199/37962 [76:52:36<11:02:23,  8.34s/it]
 87%|████████▋ | 33200/37962 [76:52:44<11:01:52,  8.34s/it]
                                                           
{'loss': 1.2476, 'grad_norm': 10.871671676635742, 'learning_rate': 8.140072250983166e-07, 'epoch': 0.87}

 87%|████████▋ | 33200/37962 [76:52:44<11:01:52,  8.34s/it]
 87%|████████▋ | 33201/37962 [76:52:53<11:01:53,  8.34s/it]
 87%|████████▋ | 33202/37962 [76:53:01<10:55:51,  8.27s/it]
 87%|████████▋ | 33203/37962 [76:53:09<10:56:36,  8.28s/it]
 87%|████████▋ | 33204/37962 [76:53:17<10:52:09,  8.22s/it]
 87%|████████▋ | 33205/37962 [76:53:25<10:50:53,  8.21s/it]
 87%|████████▋ | 33206/37962 [76:53:33<10:51:15,  8.22s/it]
 87%|████████▋ | 33207/37962 [76:53:42<10:50:43,  8.21s/it]
 87%|████████▋ | 33208/37962 [76:53:50<10:42:46,  8.11s/it]
 87%|████████▋ | 33209/37962 [76:53:58<10:49:18,  8.20s/it]
 87%|████████▋ | 33210/37962 [76:54:06<10:51:24,  8.22s/it]
                                                           
{'loss': 1.1057, 'grad_norm': 10.234232902526855, 'learning_rate': 8.106389614674992e-07, 'epoch': 0.87}

 87%|████████▋ | 33210/37962 [76:54:06<10:51:24,  8.22s/it]
 87%|████████▋ | 33211/37962 [76:54:14<10:42:47,  8.12s/it]
 87%|████████▋ | 33212/37962 [76:54:22<10:41:05,  8.10s/it]
 87%|████████▋ | 33213/37962 [76:54:30<10:47:13,  8.18s/it]
 87%|████████▋ | 33214/37962 [76:54:39<10:50:22,  8.22s/it]
 87%|████████▋ | 33215/37962 [76:54:47<10:51:16,  8.23s/it]
 87%|████████▋ | 33216/37962 [76:54:55<10:56:00,  8.29s/it]
 88%|████████▊ | 33217/37962 [76:55:04<11:00:17,  8.35s/it]
 88%|████████▊ | 33218/37962 [76:55:12<10:58:49,  8.33s/it]
 88%|████████▊ | 33219/37962 [76:55:20<10:55:58,  8.30s/it]
 88%|████████▊ | 33220/37962 [76:55:29<10:55:36,  8.30s/it]
                                                           
{'loss': 1.175, 'grad_norm': 10.188984870910645, 'learning_rate': 8.072773866133854e-07, 'epoch': 0.88}

 88%|████████▊ | 33220/37962 [76:55:29<10:55:36,  8.30s/it]
 88%|████████▊ | 33221/37962 [76:55:37<10:55:43,  8.30s/it]
 88%|████████▊ | 33222/37962 [76:55:45<10:52:21,  8.26s/it]
 88%|████████▊ | 33223/37962 [76:55:53<10:50:04,  8.23s/it]
 88%|████████▊ | 33224/37962 [76:56:02<10:51:47,  8.25s/it]
 88%|████████▊ | 33225/37962 [76:56:10<10:55:18,  8.30s/it]
 88%|████████▊ | 33226/37962 [76:56:18<10:55:49,  8.31s/it]
 88%|████████▊ | 33227/37962 [76:56:27<10:53:48,  8.28s/it]
 88%|████████▊ | 33228/37962 [76:56:35<10:53:16,  8.28s/it]
 88%|████████▊ | 33229/37962 [76:56:43<10:52:52,  8.28s/it]
 88%|████████▊ | 33230/37962 [76:56:52<10:55:22,  8.31s/it]
                                                           
{'loss': 1.1081, 'grad_norm': 6.831078052520752, 'learning_rate': 8.039225029828113e-07, 'epoch': 0.88}

 88%|████████▊ | 33230/37962 [76:56:52<10:55:22,  8.31s/it]
 88%|████████▊ | 33231/37962 [76:57:00<10:52:45,  8.28s/it]
 88%|████████▊ | 33232/37962 [76:57:08<10:49:44,  8.24s/it]
 88%|████████▊ | 33233/37962 [76:57:16<10:39:57,  8.12s/it]
 88%|████████▊ | 33234/37962 [76:57:24<10:45:16,  8.19s/it]
 88%|████████▊ | 33235/37962 [76:57:32<10:48:09,  8.23s/it]
 88%|████████▊ | 33236/37962 [76:57:41<10:47:06,  8.22s/it]
 88%|████████▊ | 33237/37962 [76:57:49<10:46:28,  8.21s/it]
 88%|████████▊ | 33238/37962 [76:57:57<10:39:29,  8.12s/it]
 88%|████████▊ | 33239/37962 [76:58:05<10:34:58,  8.07s/it]
 88%|████████▊ | 33240/37962 [76:58:13<10:36:35,  8.09s/it]
                                                           
{'loss': 1.1407, 'grad_norm': 14.557104110717773, 'learning_rate': 8.005743130177357e-07, 'epoch': 0.88}

 88%|████████▊ | 33240/37962 [76:58:13<10:36:35,  8.09s/it]
 88%|████████▊ | 33241/37962 [76:58:21<10:40:13,  8.14s/it]
 88%|████████▊ | 33242/37962 [76:58:29<10:42:23,  8.17s/it]
 88%|████████▊ | 33243/37962 [76:58:37<10:39:39,  8.13s/it]
 88%|████████▊ | 33244/37962 [76:58:46<10:42:07,  8.17s/it]
 88%|████████▊ | 33245/37962 [76:58:54<10:42:52,  8.18s/it]
 88%|████████▊ | 33246/37962 [76:59:02<10:49:32,  8.26s/it]
 88%|████████▊ | 33247/37962 [76:59:11<10:49:52,  8.27s/it]
 88%|████████▊ | 33248/37962 [76:59:19<10:47:40,  8.24s/it]
 88%|████████▊ | 33249/37962 [76:59:27<10:49:19,  8.27s/it]
 88%|████████▊ | 33250/37962 [76:59:35<10:38:47,  8.13s/it]
                                                           
{'loss': 1.1935, 'grad_norm': 12.310924530029297, 'learning_rate': 7.97232819155247e-07, 'epoch': 0.88}

 88%|████████▊ | 33250/37962 [76:59:35<10:38:47,  8.13s/it]
 88%|████████▊ | 33251/37962 [76:59:43<10:45:59,  8.23s/it]
 88%|████████▊ | 33252/37962 [76:59:52<10:45:03,  8.22s/it]
 88%|████████▊ | 33253/37962 [77:00:00<10:49:01,  8.27s/it]
 88%|████████▊ | 33254/37962 [77:00:08<10:49:31,  8.28s/it]
 88%|████████▊ | 33255/37962 [77:00:16<10:38:14,  8.14s/it]
 88%|████████▊ | 33256/37962 [77:00:24<10:43:48,  8.21s/it]
 88%|████████▊ | 33257/37962 [77:00:32<10:36:50,  8.12s/it]
 88%|████████▊ | 33258/37962 [77:00:41<10:40:24,  8.17s/it]
 88%|████████▊ | 33259/37962 [77:00:49<10:41:55,  8.19s/it]
 88%|████████▊ | 33260/37962 [77:00:57<10:43:38,  8.21s/it]
                                                           
{'loss': 1.1896, 'grad_norm': 12.728763580322266, 'learning_rate': 7.93898023827564e-07, 'epoch': 0.88}

 88%|████████▊ | 33260/37962 [77:00:57<10:43:38,  8.21s/it]
 88%|████████▊ | 33261/37962 [77:01:05<10:40:35,  8.18s/it]
 88%|████████▊ | 33262/37962 [77:01:14<10:44:46,  8.23s/it]
 88%|████████▊ | 33263/37962 [77:01:22<10:43:26,  8.22s/it]
 88%|████████▊ | 33264/37962 [77:01:30<10:46:27,  8.26s/it]
 88%|████████▊ | 33265/37962 [77:01:38<10:44:59,  8.24s/it]
 88%|████████▊ | 33266/37962 [77:01:47<10:44:16,  8.23s/it]
 88%|████████▊ | 33267/37962 [77:01:55<10:40:43,  8.19s/it]
 88%|████████▊ | 33268/37962 [77:02:03<10:46:21,  8.26s/it]
 88%|████████▊ | 33269/37962 [77:02:11<10:47:51,  8.28s/it]
 88%|████████▊ | 33270/37962 [77:02:20<10:47:07,  8.28s/it]
                                                           
{'loss': 1.1816, 'grad_norm': 10.874765396118164, 'learning_rate': 7.905699294620228e-07, 'epoch': 0.88}

 88%|████████▊ | 33270/37962 [77:02:20<10:47:07,  8.28s/it]
 88%|████████▊ | 33271/37962 [77:02:28<10:45:57,  8.26s/it]
 88%|████████▊ | 33272/37962 [77:02:36<10:48:08,  8.29s/it]
 88%|████████▊ | 33273/37962 [77:02:44<10:46:24,  8.27s/it]
 88%|████████▊ | 33274/37962 [77:02:53<10:43:40,  8.24s/it]
 88%|████████▊ | 33275/37962 [77:03:01<10:45:20,  8.26s/it]
 88%|████████▊ | 33276/37962 [77:03:09<10:52:27,  8.35s/it]
 88%|████████▊ | 33277/37962 [77:03:18<10:54:22,  8.38s/it]
 88%|████████▊ | 33278/37962 [77:03:26<10:47:22,  8.29s/it]
 88%|████████▊ | 33279/37962 [77:03:34<10:43:42,  8.25s/it]
 88%|████████▊ | 33280/37962 [77:03:42<10:43:44,  8.25s/it]
                                                           
{'loss': 1.1601, 'grad_norm': 15.096443176269531, 'learning_rate': 7.87248538481088e-07, 'epoch': 0.88}

 88%|████████▊ | 33280/37962 [77:03:42<10:43:44,  8.25s/it]
 88%|████████▊ | 33281/37962 [77:03:51<10:46:51,  8.29s/it]
 88%|████████▊ | 33282/37962 [77:03:59<10:48:13,  8.31s/it]
 88%|████████▊ | 33283/37962 [77:04:08<10:49:08,  8.32s/it]
 88%|████████▊ | 33284/37962 [77:04:16<10:47:43,  8.31s/it]
 88%|████████▊ | 33285/37962 [77:04:24<10:47:37,  8.31s/it]
 88%|████████▊ | 33286/37962 [77:04:32<10:36:19,  8.17s/it]
 88%|████████▊ | 33287/37962 [77:04:40<10:27:55,  8.06s/it]
 88%|████████▊ | 33288/37962 [77:04:48<10:28:49,  8.07s/it]
 88%|████████▊ | 33289/37962 [77:04:56<10:37:52,  8.19s/it]
 88%|████████▊ | 33290/37962 [77:05:05<10:39:07,  8.21s/it]
                                                           
{'loss': 1.0653, 'grad_norm': 13.569689750671387, 'learning_rate': 7.839338533023433e-07, 'epoch': 0.88}

 88%|████████▊ | 33290/37962 [77:05:05<10:39:07,  8.21s/it]
 88%|████████▊ | 33291/37962 [77:05:12<10:29:39,  8.09s/it]
 88%|████████▊ | 33292/37962 [77:05:21<10:33:14,  8.14s/it]
 88%|████████▊ | 33293/37962 [77:05:29<10:37:35,  8.19s/it]
 88%|████████▊ | 33294/37962 [77:05:37<10:29:31,  8.09s/it]
 88%|████████▊ | 33295/37962 [77:05:45<10:34:33,  8.16s/it]
 88%|████████▊ | 33296/37962 [77:05:53<10:36:15,  8.18s/it]
 88%|████████▊ | 33297/37962 [77:06:02<10:40:34,  8.24s/it]
 88%|████████▊ | 33298/37962 [77:06:10<10:42:40,  8.27s/it]
 88%|████████▊ | 33299/37962 [77:06:18<10:42:07,  8.26s/it]
 88%|████████▊ | 33300/37962 [77:06:26<10:29:05,  8.10s/it]
                                                           
{'loss': 0.9433, 'grad_norm': 7.381311893463135, 'learning_rate': 7.806258763384888e-07, 'epoch': 0.88}

 88%|████████▊ | 33300/37962 [77:06:26<10:29:05,  8.10s/it]
 88%|████████▊ | 33301/37962 [77:06:34<10:32:09,  8.14s/it]
 88%|████████▊ | 33302/37962 [77:06:42<10:28:15,  8.09s/it]
 88%|████████▊ | 33303/37962 [77:06:51<10:32:27,  8.14s/it]
 88%|████████▊ | 33304/37962 [77:06:59<10:36:30,  8.20s/it]
 88%|████████▊ | 33305/37962 [77:07:07<10:42:37,  8.28s/it]
 88%|████████▊ | 33306/37962 [77:07:15<10:37:54,  8.22s/it]
 88%|████████▊ | 33307/37962 [77:07:24<10:37:35,  8.22s/it]
 88%|████████▊ | 33308/37962 [77:07:32<10:37:36,  8.22s/it]
 88%|████████▊ | 33309/37962 [77:07:40<10:38:37,  8.24s/it]
 88%|████████▊ | 33310/37962 [77:07:48<10:41:12,  8.27s/it]
                                                           
{'loss': 1.0483, 'grad_norm': 14.047696113586426, 'learning_rate': 7.77324609997343e-07, 'epoch': 0.88}

 88%|████████▊ | 33310/37962 [77:07:48<10:41:12,  8.27s/it]
 88%|████████▊ | 33311/37962 [77:07:57<10:49:38,  8.38s/it]
 88%|████████▊ | 33312/37962 [77:08:05<10:46:13,  8.34s/it]
 88%|████████▊ | 33313/37962 [77:08:14<10:44:04,  8.31s/it]
 88%|████████▊ | 33314/37962 [77:08:22<10:35:40,  8.21s/it]
 88%|████████▊ | 33315/37962 [77:08:30<10:42:39,  8.30s/it]
 88%|████████▊ | 33316/37962 [77:08:38<10:42:51,  8.30s/it]
 88%|████████▊ | 33317/37962 [77:08:47<10:41:41,  8.29s/it]
 88%|████████▊ | 33318/37962 [77:08:55<10:41:53,  8.29s/it]
 88%|████████▊ | 33319/37962 [77:09:03<10:35:12,  8.21s/it]
 88%|████████▊ | 33320/37962 [77:09:11<10:32:43,  8.18s/it]
                                                           
{'loss': 1.1856, 'grad_norm': 20.547494888305664, 'learning_rate': 7.740300566818437e-07, 'epoch': 0.88}

 88%|████████▊ | 33320/37962 [77:09:11<10:32:43,  8.18s/it]
 88%|████████▊ | 33321/37962 [77:09:19<10:35:22,  8.21s/it]
 88%|████████▊ | 33322/37962 [77:09:28<10:37:34,  8.24s/it]
 88%|████████▊ | 33323/37962 [77:09:36<10:39:42,  8.27s/it]
 88%|████████▊ | 33324/37962 [77:09:44<10:34:41,  8.21s/it]
 88%|████████▊ | 33325/37962 [77:09:52<10:33:36,  8.20s/it]
 88%|████████▊ | 33326/37962 [77:10:00<10:31:09,  8.17s/it]
 88%|████████▊ | 33327/37962 [77:10:08<10:27:57,  8.13s/it]
 88%|████████▊ | 33328/37962 [77:10:17<10:29:58,  8.16s/it]
 88%|████████▊ | 33329/37962 [77:10:25<10:31:33,  8.18s/it]
 88%|████████▊ | 33330/37962 [77:10:33<10:41:54,  8.31s/it]
                                                           
{'loss': 1.245, 'grad_norm': 7.950221538543701, 'learning_rate': 7.707422187900349e-07, 'epoch': 0.88}

 88%|████████▊ | 33330/37962 [77:10:33<10:41:54,  8.31s/it]
 88%|████████▊ | 33331/37962 [77:10:42<10:40:16,  8.30s/it]
 88%|████████▊ | 33332/37962 [77:10:50<10:43:36,  8.34s/it]
 88%|████████▊ | 33333/37962 [77:10:58<10:43:05,  8.34s/it]
 88%|████████▊ | 33334/37962 [77:11:07<10:41:39,  8.32s/it]
 88%|████████▊ | 33335/37962 [77:11:15<10:37:55,  8.27s/it]
 88%|████████▊ | 33336/37962 [77:11:23<10:40:27,  8.31s/it]
 88%|████████▊ | 33337/37962 [77:11:32<10:40:51,  8.31s/it]
 88%|████████▊ | 33338/37962 [77:11:40<10:41:59,  8.33s/it]
 88%|████████▊ | 33339/37962 [77:11:48<10:39:39,  8.30s/it]
 88%|████████▊ | 33340/37962 [77:11:57<10:39:19,  8.30s/it]
                                                           
{'loss': 1.1519, 'grad_norm': 90.43871307373047, 'learning_rate': 7.67461098715081e-07, 'epoch': 0.88}

 88%|████████▊ | 33340/37962 [77:11:57<10:39:19,  8.30s/it]
 88%|████████▊ | 33341/37962 [77:12:05<10:39:38,  8.31s/it]
 88%|████████▊ | 33342/37962 [77:12:13<10:29:36,  8.18s/it]
 88%|████████▊ | 33343/37962 [77:12:21<10:33:19,  8.23s/it]
 88%|████████▊ | 33344/37962 [77:12:29<10:35:19,  8.25s/it]
 88%|████████▊ | 33345/37962 [77:12:38<10:32:40,  8.22s/it]
 88%|████████▊ | 33346/37962 [77:12:46<10:32:20,  8.22s/it]
 88%|████████▊ | 33347/37962 [77:12:54<10:33:56,  8.24s/it]
 88%|████████▊ | 33348/37962 [77:13:02<10:36:17,  8.27s/it]
 88%|████████▊ | 33349/37962 [77:13:11<10:40:45,  8.33s/it]
 88%|████████▊ | 33350/37962 [77:13:19<10:38:02,  8.30s/it]
                                                           
{'loss': 1.1112, 'grad_norm': 8.950170516967773, 'learning_rate': 7.641866988452506e-07, 'epoch': 0.88}

 88%|████████▊ | 33350/37962 [77:13:19<10:38:02,  8.30s/it]
 88%|████████▊ | 33351/37962 [77:13:27<10:38:56,  8.31s/it]
 88%|████████▊ | 33352/37962 [77:13:36<10:41:34,  8.35s/it]
 88%|████████▊ | 33353/37962 [77:13:44<10:42:38,  8.37s/it]
 88%|████████▊ | 33354/37962 [77:13:53<10:41:22,  8.35s/it]
 88%|████████▊ | 33355/37962 [77:14:01<10:43:16,  8.38s/it]
 88%|████████▊ | 33356/37962 [77:14:09<10:38:35,  8.32s/it]
 88%|████████▊ | 33357/37962 [77:14:18<10:45:21,  8.41s/it]
 88%|████████▊ | 33358/37962 [77:14:26<10:43:37,  8.39s/it]
 88%|████████▊ | 33359/37962 [77:14:34<10:40:24,  8.35s/it]
 88%|████████▊ | 33360/37962 [77:14:43<10:39:49,  8.34s/it]
                                                           
{'loss': 0.997, 'grad_norm': 9.244402885437012, 'learning_rate': 7.609190215639217e-07, 'epoch': 0.88}

 88%|████████▊ | 33360/37962 [77:14:43<10:39:49,  8.34s/it]
 88%|████████▊ | 33361/37962 [77:14:51<10:38:07,  8.32s/it]
 88%|████████▊ | 33362/37962 [77:14:59<10:32:46,  8.25s/it]
 88%|████████▊ | 33363/37962 [77:15:07<10:31:59,  8.25s/it]
 88%|████████▊ | 33364/37962 [77:15:16<10:31:13,  8.24s/it]
 88%|████████▊ | 33365/37962 [77:15:23<10:24:03,  8.15s/it]
 88%|████████▊ | 33366/37962 [77:15:32<10:25:49,  8.17s/it]
 88%|████████▊ | 33367/37962 [77:15:40<10:26:33,  8.18s/it]
 88%|████████▊ | 33368/37962 [77:15:48<10:26:30,  8.18s/it]
 88%|████████▊ | 33369/37962 [77:15:56<10:28:30,  8.21s/it]
 88%|████████▊ | 33370/37962 [77:16:05<10:34:12,  8.29s/it]
                                                           
{'loss': 1.1845, 'grad_norm': 9.939420700073242, 'learning_rate': 7.576580692495794e-07, 'epoch': 0.88}

 88%|████████▊ | 33370/37962 [77:16:05<10:34:12,  8.29s/it]
 88%|████████▊ | 33371/37962 [77:16:13<10:28:58,  8.22s/it]
 88%|████████▊ | 33372/37962 [77:16:21<10:29:37,  8.23s/it]
 88%|████████▊ | 33373/37962 [77:16:29<10:31:06,  8.25s/it]
 88%|████████▊ | 33374/37962 [77:16:38<10:27:46,  8.21s/it]
 88%|████████▊ | 33375/37962 [77:16:46<10:22:29,  8.14s/it]
 88%|████████▊ | 33376/37962 [77:16:54<10:28:29,  8.22s/it]
 88%|████████▊ | 33377/37962 [77:17:02<10:31:22,  8.26s/it]
 88%|████████▊ | 33378/37962 [77:17:11<10:37:22,  8.34s/it]
 88%|████████▊ | 33379/37962 [77:17:19<10:37:51,  8.35s/it]
 88%|████████▊ | 33380/37962 [77:17:27<10:29:07,  8.24s/it]
                                                           
{'loss': 1.135, 'grad_norm': 9.907596588134766, 'learning_rate': 7.544038442758184e-07, 'epoch': 0.88}

 88%|████████▊ | 33380/37962 [77:17:27<10:29:07,  8.24s/it]
 88%|████████▊ | 33381/37962 [77:17:36<10:32:42,  8.29s/it]
 88%|████████▊ | 33382/37962 [77:17:44<10:33:19,  8.30s/it]
 88%|████████▊ | 33383/37962 [77:17:52<10:32:18,  8.29s/it]
 88%|████████▊ | 33384/37962 [77:18:00<10:32:10,  8.29s/it]
 88%|████████▊ | 33385/37962 [77:18:08<10:26:08,  8.21s/it]
 88%|████████▊ | 33386/37962 [77:18:17<10:31:20,  8.28s/it]
 88%|████████▊ | 33387/37962 [77:18:25<10:32:02,  8.29s/it]
 88%|████████▊ | 33388/37962 [77:18:34<10:35:31,  8.34s/it]
 88%|████████▊ | 33389/37962 [77:18:42<10:33:13,  8.31s/it]
 88%|████████▊ | 33390/37962 [77:18:50<10:33:59,  8.32s/it]
                                                           
{'loss': 1.177, 'grad_norm': 9.714869499206543, 'learning_rate': 7.511563490113283e-07, 'epoch': 0.88}

 88%|████████▊ | 33390/37962 [77:18:50<10:33:59,  8.32s/it]
 88%|████████▊ | 33391/37962 [77:18:59<10:37:28,  8.37s/it]
 88%|████████▊ | 33392/37962 [77:19:07<10:35:26,  8.34s/it]
 88%|████████▊ | 33393/37962 [77:19:15<10:29:59,  8.27s/it]
 88%|████████▊ | 33394/37962 [77:19:23<10:28:55,  8.26s/it]
 88%|████████▊ | 33395/37962 [77:19:32<10:29:50,  8.27s/it]
 88%|████████▊ | 33396/37962 [77:19:40<10:27:54,  8.25s/it]
 88%|████████▊ | 33397/37962 [77:19:48<10:19:23,  8.14s/it]
 88%|████████▊ | 33398/37962 [77:19:56<10:17:55,  8.12s/it]
 88%|████████▊ | 33399/37962 [77:20:04<10:29:17,  8.27s/it]
 88%|████████▊ | 33400/37962 [77:20:13<10:24:47,  8.22s/it]
                                                           
{'loss': 1.2004, 'grad_norm': 25.097639083862305, 'learning_rate': 7.479155858199083e-07, 'epoch': 0.88}

 88%|████████▊ | 33400/37962 [77:20:13<10:24:47,  8.22s/it]
 88%|████████▊ | 33401/37962 [77:20:21<10:25:46,  8.23s/it]
 88%|████████▊ | 33402/37962 [77:20:29<10:25:47,  8.23s/it]
 88%|████████▊ | 33403/37962 [77:20:37<10:26:57,  8.25s/it]
 88%|████████▊ | 33404/37962 [77:20:45<10:22:28,  8.19s/it]
 88%|████████▊ | 33405/37962 [77:20:54<10:22:34,  8.20s/it]
 88%|████████▊ | 33406/37962 [77:21:02<10:22:02,  8.19s/it]
 88%|████████▊ | 33407/37962 [77:21:10<10:24:42,  8.23s/it]
 88%|████████▊ | 33408/37962 [77:21:18<10:22:28,  8.20s/it]
 88%|████████▊ | 33409/37962 [77:21:27<10:25:37,  8.24s/it]
 88%|████████▊ | 33410/37962 [77:21:35<10:32:49,  8.34s/it]
                                                           
{'loss': 1.1779, 'grad_norm': 12.760353088378906, 'learning_rate': 7.446815570604527e-07, 'epoch': 0.88}

 88%|████████▊ | 33410/37962 [77:21:35<10:32:49,  8.34s/it]
 88%|████████▊ | 33411/37962 [77:21:44<10:35:04,  8.37s/it]
 88%|████████▊ | 33412/37962 [77:21:52<10:35:45,  8.38s/it]
 88%|████████▊ | 33413/37962 [77:22:00<10:34:29,  8.37s/it]
 88%|████████▊ | 33414/37962 [77:22:09<10:35:21,  8.38s/it]
 88%|████████▊ | 33415/37962 [77:22:17<10:34:28,  8.37s/it]
 88%|████████▊ | 33416/37962 [77:22:25<10:30:36,  8.32s/it]
 88%|████████▊ | 33417/37962 [77:22:34<10:29:07,  8.31s/it]
 88%|████████▊ | 33418/37962 [77:22:42<10:35:48,  8.40s/it]
 88%|████████▊ | 33419/37962 [77:22:51<10:33:56,  8.37s/it]
 88%|████████▊ | 33420/37962 [77:22:59<10:30:14,  8.33s/it]
                                                           
{'loss': 1.3113, 'grad_norm': 6.628438949584961, 'learning_rate': 7.414542650869517e-07, 'epoch': 0.88}

 88%|████████▊ | 33420/37962 [77:22:59<10:30:14,  8.33s/it]
 88%|████████▊ | 33421/37962 [77:23:07<10:29:44,  8.32s/it]
 88%|████████▊ | 33422/37962 [77:23:15<10:30:59,  8.34s/it]
 88%|████████▊ | 33423/37962 [77:23:24<10:32:40,  8.36s/it]
 88%|████████▊ | 33424/37962 [77:23:32<10:35:49,  8.41s/it]
 88%|████████▊ | 33425/37962 [77:23:41<10:31:22,  8.35s/it]
 88%|████████▊ | 33426/37962 [77:23:49<10:22:04,  8.23s/it]
 88%|████████▊ | 33427/37962 [77:23:57<10:22:31,  8.24s/it]
 88%|████████▊ | 33428/37962 [77:24:05<10:18:10,  8.18s/it]
 88%|████████▊ | 33429/37962 [77:24:13<10:18:25,  8.19s/it]
 88%|████████▊ | 33430/37962 [77:24:21<10:22:38,  8.24s/it]
                                                           
{'loss': 1.2333, 'grad_norm': 8.37238883972168, 'learning_rate': 7.382337122484995e-07, 'epoch': 0.88}

 88%|████████▊ | 33430/37962 [77:24:21<10:22:38,  8.24s/it]
 88%|████████▊ | 33431/37962 [77:24:30<10:26:25,  8.30s/it]
 88%|████████▊ | 33432/37962 [77:24:38<10:26:20,  8.30s/it]
 88%|████████▊ | 33433/37962 [77:24:46<10:24:58,  8.28s/it]
 88%|████████▊ | 33434/37962 [77:24:55<10:25:39,  8.29s/it]
 88%|████████▊ | 33435/37962 [77:25:03<10:29:19,  8.34s/it]
 88%|████████▊ | 33436/37962 [77:25:11<10:28:01,  8.33s/it]
 88%|████████▊ | 33437/37962 [77:25:20<10:25:27,  8.29s/it]
 88%|████████▊ | 33438/37962 [77:25:28<10:18:50,  8.21s/it]
 88%|████████▊ | 33439/37962 [77:25:36<10:19:32,  8.22s/it]
 88%|████████▊ | 33440/37962 [77:25:44<10:17:25,  8.19s/it]
                                                           
{'loss': 1.1481, 'grad_norm': 9.207893371582031, 'learning_rate': 7.350199008892778e-07, 'epoch': 0.88}

 88%|████████▊ | 33440/37962 [77:25:44<10:17:25,  8.19s/it]
 88%|████████▊ | 33441/37962 [77:25:52<10:14:14,  8.15s/it]
 88%|████████▊ | 33442/37962 [77:26:08<13:20:48, 10.63s/it]
 88%|████████▊ | 33443/37962 [77:26:17<12:27:32,  9.93s/it]
 88%|████████▊ | 33444/37962 [77:26:25<11:44:27,  9.36s/it]
 88%|████████▊ | 33445/37962 [77:26:33<11:20:31,  9.04s/it]
 88%|████████▊ | 33446/37962 [77:26:41<11:04:56,  8.83s/it]
 88%|████████▊ | 33447/37962 [77:26:50<10:50:27,  8.64s/it]
 88%|████████▊ | 33448/37962 [77:26:58<10:34:12,  8.43s/it]
 88%|████████▊ | 33449/37962 [77:27:06<10:30:55,  8.39s/it]
 88%|████████▊ | 33450/37962 [77:27:14<10:27:32,  8.35s/it]
                                                           
{'loss': 1.1524, 'grad_norm': 10.915332794189453, 'learning_rate': 7.318128333485663e-07, 'epoch': 0.88}

 88%|████████▊ | 33450/37962 [77:27:14<10:27:32,  8.35s/it]
 88%|████████▊ | 33451/37962 [77:27:22<10:25:41,  8.32s/it]
 88%|████████▊ | 33452/37962 [77:27:31<10:23:30,  8.30s/it]
 88%|████████▊ | 33453/37962 [77:27:39<10:24:02,  8.30s/it]
 88%|████████▊ | 33454/37962 [77:27:47<10:18:35,  8.23s/it]
 88%|████████▊ | 33455/37962 [77:27:55<10:21:51,  8.28s/it]
 88%|████████▊ | 33456/37962 [77:28:04<10:21:21,  8.27s/it]
 88%|████████▊ | 33457/37962 [77:28:12<10:22:29,  8.29s/it]
 88%|████████▊ | 33458/37962 [77:28:20<10:20:58,  8.27s/it]
 88%|████████▊ | 33459/37962 [77:28:28<10:19:14,  8.25s/it]
 88%|████████▊ | 33460/37962 [77:28:37<10:19:11,  8.25s/it]
                                                           
{'loss': 1.0823, 'grad_norm': 14.058781623840332, 'learning_rate': 7.286125119607346e-07, 'epoch': 0.88}

 88%|████████▊ | 33460/37962 [77:28:37<10:19:11,  8.25s/it]
 88%|████████▊ | 33461/37962 [77:28:45<10:18:00,  8.24s/it]
 88%|████████▊ | 33462/37962 [77:28:53<10:18:47,  8.25s/it]
 88%|████████▊ | 33463/37962 [77:29:02<10:23:45,  8.32s/it]
 88%|████████▊ | 33464/37962 [77:29:10<10:23:52,  8.32s/it]
 88%|████████▊ | 33465/37962 [77:29:18<10:22:39,  8.31s/it]
 88%|████████▊ | 33466/37962 [77:29:34<13:08:16, 10.52s/it]
 88%|████████▊ | 33467/37962 [77:29:42<12:18:44,  9.86s/it]
 88%|████████▊ | 33468/37962 [77:29:50<11:41:58,  9.37s/it]
 88%|████████▊ | 33469/37962 [77:29:59<11:17:53,  9.05s/it]
 88%|████████▊ | 33470/37962 [77:30:07<10:59:53,  8.81s/it]
                                                           
{'loss': 1.1943, 'grad_norm': 8.432967185974121, 'learning_rate': 7.254189390552369e-07, 'epoch': 0.88}

 88%|████████▊ | 33470/37962 [77:30:07<10:59:53,  8.81s/it]
 88%|████████▊ | 33471/37962 [77:30:15<10:47:51,  8.66s/it]
 88%|████████▊ | 33472/37962 [77:30:24<10:38:05,  8.53s/it]
 88%|████████▊ | 33473/37962 [77:30:32<10:36:55,  8.51s/it]
 88%|████████▊ | 33474/37962 [77:30:40<10:31:23,  8.44s/it]
 88%|████████▊ | 33475/37962 [77:30:48<10:22:27,  8.32s/it]
 88%|████████▊ | 33476/37962 [77:30:57<10:23:06,  8.33s/it]
 88%|████████▊ | 33477/37962 [77:31:05<10:25:30,  8.37s/it]
 88%|████████▊ | 33478/37962 [77:31:13<10:23:13,  8.34s/it]
 88%|████████▊ | 33479/37962 [77:31:21<10:14:26,  8.22s/it]
 88%|████████▊ | 33480/37962 [77:31:29<10:07:54,  8.14s/it]
                                                           
{'loss': 1.145, 'grad_norm': 8.47348690032959, 'learning_rate': 7.222321169566227e-07, 'epoch': 0.88}

 88%|████████▊ | 33480/37962 [77:31:29<10:07:54,  8.14s/it]
 88%|████████▊ | 33481/37962 [77:31:38<10:11:32,  8.19s/it]
 88%|████████▊ | 33482/37962 [77:31:45<10:02:00,  8.06s/it]
 88%|████████▊ | 33483/37962 [77:31:54<10:07:43,  8.14s/it]
 88%|████████▊ | 33484/37962 [77:32:02<10:07:49,  8.14s/it]
 88%|████████▊ | 33485/37962 [77:32:10<10:11:55,  8.20s/it]
 88%|████████▊ | 33486/37962 [77:32:18<10:13:11,  8.22s/it]
 88%|████████▊ | 33487/37962 [77:32:27<10:14:45,  8.24s/it]
 88%|████████▊ | 33488/37962 [77:32:35<10:14:50,  8.25s/it]
 88%|████████▊ | 33489/37962 [77:32:43<10:12:48,  8.22s/it]
 88%|████████▊ | 33490/37962 [77:32:51<10:11:07,  8.20s/it]
                                                           
{'loss': 1.2015, 'grad_norm': 10.444677352905273, 'learning_rate': 7.190520479845253e-07, 'epoch': 0.88}

 88%|████████▊ | 33490/37962 [77:32:51<10:11:07,  8.20s/it]
 88%|████████▊ | 33491/37962 [77:33:00<10:12:30,  8.22s/it]
 88%|████████▊ | 33492/37962 [77:33:08<10:12:33,  8.22s/it]
 88%|████████▊ | 33493/37962 [77:33:16<10:12:23,  8.22s/it]
 88%|████████▊ | 33494/37962 [77:33:24<10:13:44,  8.24s/it]
 88%|████████▊ | 33495/37962 [77:33:33<10:15:00,  8.26s/it]
 88%|████████▊ | 33496/37962 [77:33:41<10:12:03,  8.22s/it]
 88%|████████▊ | 33497/37962 [77:33:49<10:11:46,  8.22s/it]
 88%|████████▊ | 33498/37962 [77:33:57<10:12:41,  8.24s/it]
 88%|████████▊ | 33499/37962 [77:34:06<10:14:02,  8.26s/it]
 88%|████████▊ | 33500/37962 [77:34:14<10:18:44,  8.32s/it]
                                                           
{'loss': 1.0062, 'grad_norm': 16.779993057250977, 'learning_rate': 7.1587873445366e-07, 'epoch': 0.88}

 88%|████████▊ | 33500/37962 [77:34:14<10:18:44,  8.32s/it]
 88%|████████▊ | 33501/37962 [77:34:22<10:17:11,  8.30s/it]
 88%|████████▊ | 33502/37962 [77:34:31<10:19:58,  8.34s/it]
 88%|████████▊ | 33503/37962 [77:34:39<10:17:21,  8.31s/it]
 88%|████████▊ | 33504/37962 [77:34:47<10:16:11,  8.29s/it]
 88%|████████▊ | 33505/37962 [77:34:55<10:12:34,  8.25s/it]
 88%|████████▊ | 33506/37962 [77:35:04<10:12:21,  8.25s/it]
 88%|████████▊ | 33507/37962 [77:35:12<10:11:58,  8.24s/it]
 88%|████████▊ | 33508/37962 [77:35:20<10:12:38,  8.25s/it]
 88%|████████▊ | 33509/37962 [77:35:28<10:12:31,  8.25s/it]
 88%|████████▊ | 33510/37962 [77:35:37<10:16:15,  8.31s/it]
                                                           
{'loss': 1.2153, 'grad_norm': 14.761578559875488, 'learning_rate': 7.127121786738278e-07, 'epoch': 0.88}

 88%|████████▊ | 33510/37962 [77:35:37<10:16:15,  8.31s/it]
 88%|████████▊ | 33511/37962 [77:35:45<10:16:26,  8.31s/it]
 88%|████████▊ | 33512/37962 [77:35:53<10:06:31,  8.18s/it]
 88%|████████▊ | 33513/37962 [77:36:01<10:07:40,  8.20s/it]
 88%|████████▊ | 33514/37962 [77:36:09<10:00:30,  8.10s/it]
 88%|████████▊ | 33515/37962 [77:36:17<10:04:49,  8.16s/it]
 88%|████████▊ | 33516/37962 [77:36:26<10:08:21,  8.21s/it]
 88%|████████▊ | 33517/37962 [77:36:34<10:09:33,  8.23s/it]
 88%|████████▊ | 33518/37962 [77:36:42<10:11:22,  8.25s/it]
 88%|████████▊ | 33519/37962 [77:36:50<10:09:04,  8.23s/it]
 88%|████████▊ | 33520/37962 [77:36:59<10:08:22,  8.22s/it]
                                                           
{'loss': 1.1257, 'grad_norm': 11.269522666931152, 'learning_rate': 7.095523829499107e-07, 'epoch': 0.88}

 88%|████████▊ | 33520/37962 [77:36:59<10:08:22,  8.22s/it]
 88%|████████▊ | 33521/37962 [77:37:07<10:10:01,  8.24s/it]
 88%|████████▊ | 33522/37962 [77:37:15<10:10:03,  8.24s/it]
 88%|████████▊ | 33523/37962 [77:37:24<10:16:10,  8.33s/it]
 88%|████████▊ | 33524/37962 [77:37:32<10:17:18,  8.35s/it]
 88%|████████▊ | 33525/37962 [77:37:40<10:13:47,  8.30s/it]
 88%|████████▊ | 33526/37962 [77:37:49<10:17:49,  8.36s/it]
 88%|████████▊ | 33527/37962 [77:37:57<10:18:11,  8.36s/it]
 88%|████████▊ | 33528/37962 [77:38:05<10:16:20,  8.34s/it]
 88%|████████▊ | 33529/37962 [77:38:14<10:21:33,  8.41s/it]
 88%|████████▊ | 33530/37962 [77:38:22<10:19:35,  8.39s/it]
                                                           
{'loss': 1.1598, 'grad_norm': 9.841747283935547, 'learning_rate': 7.063993495818666e-07, 'epoch': 0.88}

 88%|████████▊ | 33530/37962 [77:38:22<10:19:35,  8.39s/it]
 88%|████████▊ | 33531/37962 [77:38:31<10:16:15,  8.34s/it]
 88%|████████▊ | 33532/37962 [77:38:39<10:17:08,  8.36s/it]
 88%|████████▊ | 33533/37962 [77:38:47<10:18:24,  8.38s/it]
 88%|████████▊ | 33534/37962 [77:38:55<10:10:21,  8.27s/it]
 88%|████████▊ | 33535/37962 [77:39:04<10:11:28,  8.29s/it]
 88%|████████▊ | 33536/37962 [77:39:12<10:04:36,  8.20s/it]
 88%|████████▊ | 33537/37962 [77:39:20<10:03:49,  8.19s/it]
 88%|████████▊ | 33538/37962 [77:39:28<10:03:46,  8.19s/it]
 88%|████████▊ | 33539/37962 [77:39:36<10:07:12,  8.24s/it]
 88%|████████▊ | 33540/37962 [77:39:45<10:04:43,  8.21s/it]
                                                           
{'loss': 1.1321, 'grad_norm': 13.81451416015625, 'learning_rate': 7.032530808647364e-07, 'epoch': 0.88}

 88%|████████▊ | 33540/37962 [77:39:45<10:04:43,  8.21s/it]
 88%|████████▊ | 33541/37962 [77:39:53<10:05:10,  8.21s/it]
 88%|████████▊ | 33542/37962 [77:40:01<10:08:04,  8.25s/it]
 88%|████████▊ | 33543/37962 [77:40:09<10:01:54,  8.17s/it]
 88%|████████▊ | 33544/37962 [77:40:17<10:02:10,  8.18s/it]
 88%|████████▊ | 33545/37962 [77:40:26<10:02:43,  8.19s/it]
 88%|████████▊ | 33546/37962 [77:40:34<9:59:25,  8.14s/it] 
 88%|████████▊ | 33547/37962 [77:40:42<10:01:02,  8.17s/it]
 88%|████████▊ | 33548/37962 [77:40:50<9:59:51,  8.15s/it] 
 88%|████████▊ | 33549/37962 [77:40:58<10:01:01,  8.17s/it]
 88%|████████▊ | 33550/37962 [77:41:07<10:09:21,  8.29s/it]
                                                           
{'loss': 1.154, 'grad_norm': 10.095303535461426, 'learning_rate': 7.001135790886315e-07, 'epoch': 0.88}

 88%|████████▊ | 33550/37962 [77:41:07<10:09:21,  8.29s/it]
 88%|████████▊ | 33551/37962 [77:41:15<10:09:09,  8.29s/it]
 88%|████████▊ | 33552/37962 [77:41:23<10:07:34,  8.27s/it]
 88%|████████▊ | 33553/37962 [77:41:32<10:12:28,  8.33s/it]
 88%|████████▊ | 33554/37962 [77:41:40<10:05:28,  8.24s/it]
 88%|████████▊ | 33555/37962 [77:41:48<10:01:49,  8.19s/it]
 88%|████████▊ | 33556/37962 [77:41:56<9:57:56,  8.14s/it] 
 88%|████████▊ | 33557/37962 [77:42:04<10:01:32,  8.19s/it]
 88%|████████▊ | 33558/37962 [77:42:12<10:02:48,  8.21s/it]
 88%|████████▊ | 33559/37962 [77:42:21<10:05:36,  8.25s/it]
 88%|████████▊ | 33560/37962 [77:42:29<10:05:33,  8.25s/it]
                                                           
{'loss': 1.2709, 'grad_norm': 8.073668479919434, 'learning_rate': 6.969808465387451e-07, 'epoch': 0.88}

 88%|████████▊ | 33560/37962 [77:42:29<10:05:33,  8.25s/it]
 88%|████████▊ | 33561/37962 [77:42:37<10:08:04,  8.29s/it]
 88%|████████▊ | 33562/37962 [77:42:46<10:06:07,  8.27s/it]
 88%|████████▊ | 33563/37962 [77:42:54<10:03:09,  8.23s/it]
 88%|████████▊ | 33564/37962 [77:43:02<9:56:15,  8.13s/it] 
 88%|████████▊ | 33565/37962 [77:43:10<9:59:44,  8.18s/it]
 88%|████████▊ | 33566/37962 [77:43:18<10:06:48,  8.28s/it]
 88%|████████▊ | 33567/37962 [77:43:27<10:05:52,  8.27s/it]
 88%|████████▊ | 33568/37962 [77:43:35<10:07:29,  8.30s/it]
 88%|████████▊ | 33569/37962 [77:43:43<10:07:58,  8.30s/it]
 88%|████████▊ | 33570/37962 [77:43:52<10:10:02,  8.33s/it]
                                                           
{'loss': 1.1518, 'grad_norm': 9.129645347595215, 'learning_rate': 6.938548854953364e-07, 'epoch': 0.88}

 88%|████████▊ | 33570/37962 [77:43:52<10:10:02,  8.33s/it]
 88%|████████▊ | 33571/37962 [77:44:00<10:08:09,  8.31s/it]
 88%|████████▊ | 33572/37962 [77:44:08<10:05:41,  8.28s/it]
 88%|████████▊ | 33573/37962 [77:44:17<10:05:02,  8.27s/it]
 88%|████████▊ | 33574/37962 [77:44:25<10:05:08,  8.27s/it]
 88%|████████▊ | 33575/37962 [77:44:33<10:10:48,  8.35s/it]
 88%|████████▊ | 33576/37962 [77:44:41<10:06:07,  8.29s/it]
 88%|████████▊ | 33577/37962 [77:44:49<9:58:32,  8.19s/it] 
 88%|████████▊ | 33578/37962 [77:44:58<9:58:27,  8.19s/it]
 88%|████████▊ | 33579/37962 [77:45:06<10:01:02,  8.23s/it]
 88%|████████▊ | 33580/37962 [77:45:14<10:07:03,  8.31s/it]
                                                           
{'loss': 1.1327, 'grad_norm': 8.365315437316895, 'learning_rate': 6.907356982337377e-07, 'epoch': 0.88}

 88%|████████▊ | 33580/37962 [77:45:14<10:07:03,  8.31s/it]
 88%|████████▊ | 33581/37962 [77:45:23<10:07:49,  8.32s/it]
 88%|████████▊ | 33582/37962 [77:45:31<10:09:40,  8.35s/it]
 88%|████████▊ | 33583/37962 [77:45:40<10:12:31,  8.39s/it]
 88%|████████▊ | 33584/37962 [77:45:48<10:07:51,  8.33s/it]
 88%|████████▊ | 33585/37962 [77:45:56<10:02:09,  8.25s/it]
 88%|████████▊ | 33586/37962 [77:46:04<10:07:20,  8.33s/it]
 88%|████████▊ | 33587/37962 [77:46:13<10:11:13,  8.38s/it]
 88%|████████▊ | 33588/37962 [77:46:21<10:08:54,  8.35s/it]
 88%|████████▊ | 33589/37962 [77:46:30<10:10:07,  8.37s/it]
 88%|████████▊ | 33590/37962 [77:46:38<10:09:02,  8.36s/it]
                                                           
{'loss': 1.28, 'grad_norm': 12.932005882263184, 'learning_rate': 6.876232870243515e-07, 'epoch': 0.88}

 88%|████████▊ | 33590/37962 [77:46:38<10:09:02,  8.36s/it]
 88%|████████▊ | 33591/37962 [77:46:46<10:01:01,  8.25s/it]
 88%|████████▊ | 33592/37962 [77:46:54<9:59:20,  8.23s/it] 
 88%|████████▊ | 33593/37962 [77:47:02<9:56:27,  8.19s/it]
 88%|████████▊ | 33594/37962 [77:47:11<10:02:15,  8.27s/it]
 88%|████████▊ | 33595/37962 [77:47:19<10:02:48,  8.28s/it]
 88%|████████▊ | 33596/37962 [77:47:27<10:00:44,  8.26s/it]
 89%|████████▊ | 33597/37962 [77:47:35<9:58:37,  8.23s/it] 
 89%|████████▊ | 33598/37962 [77:47:44<10:00:50,  8.26s/it]
 89%|████████▊ | 33599/37962 [77:47:52<10:02:26,  8.28s/it]
 89%|████████▊ | 33600/37962 [77:48:00<10:02:03,  8.28s/it]
                                                           
{'loss': 1.1187, 'grad_norm': 10.146544456481934, 'learning_rate': 6.845176541326504e-07, 'epoch': 0.89}

 89%|████████▊ | 33600/37962 [77:48:00<10:02:03,  8.28s/it]
 89%|████████▊ | 33601/37962 [77:48:09<10:01:42,  8.28s/it]
 89%|████████▊ | 33602/37962 [77:48:17<9:59:43,  8.25s/it] 
 89%|████████▊ | 33603/37962 [77:48:25<9:54:32,  8.18s/it]
 89%|████████▊ | 33604/37962 [77:48:33<9:58:27,  8.24s/it]
 89%|████████▊ | 33605/37962 [77:48:41<9:51:21,  8.14s/it]
 89%|████████▊ | 33606/37962 [77:48:49<9:53:53,  8.18s/it]
 89%|████████▊ | 33607/37962 [77:48:58<9:55:43,  8.21s/it]
 89%|████████▊ | 33608/37962 [77:49:06<9:53:36,  8.18s/it]
 89%|████████▊ | 33609/37962 [77:49:14<9:53:37,  8.18s/it]
 89%|████████▊ | 33610/37962 [77:49:22<9:55:47,  8.21s/it]
                                                          
{'loss': 1.1366, 'grad_norm': 7.70832633972168, 'learning_rate': 6.814188018191703e-07, 'epoch': 0.89}

 89%|████████▊ | 33610/37962 [77:49:22<9:55:47,  8.21s/it]
 89%|████████▊ | 33611/37962 [77:49:30<9:52:02,  8.16s/it]
 89%|████████▊ | 33612/37962 [77:49:39<10:01:19,  8.29s/it]
 89%|████████▊ | 33613/37962 [77:49:47<9:58:46,  8.26s/it] 
 89%|████████▊ | 33614/37962 [77:49:55<9:58:27,  8.26s/it]
 89%|████████▊ | 33615/37962 [77:50:04<10:00:43,  8.29s/it]
 89%|████████▊ | 33616/37962 [77:50:12<10:01:03,  8.30s/it]
 89%|████████▊ | 33617/37962 [77:50:20<9:59:34,  8.28s/it] 
 89%|████████▊ | 33618/37962 [77:50:29<10:05:04,  8.36s/it]
 89%|████████▊ | 33619/37962 [77:50:37<10:06:13,  8.38s/it]
 89%|████████▊ | 33620/37962 [77:50:45<9:58:26,  8.27s/it] 
                                                          
{'loss': 1.0771, 'grad_norm': 8.484920501708984, 'learning_rate': 6.783267323395082e-07, 'epoch': 0.89}

 89%|████████▊ | 33620/37962 [77:50:45<9:58:26,  8.27s/it]
 89%|████████▊ | 33621/37962 [77:50:53<9:52:53,  8.19s/it]
 89%|████████▊ | 33622/37962 [77:51:02<9:54:17,  8.22s/it]
 89%|████████▊ | 33623/37962 [77:51:10<9:50:29,  8.17s/it]
 89%|████████▊ | 33624/37962 [77:51:18<9:52:37,  8.20s/it]
 89%|████████▊ | 33625/37962 [77:51:26<9:55:58,  8.25s/it]
 89%|████████▊ | 33626/37962 [77:51:35<9:57:40,  8.27s/it]
 89%|████████▊ | 33627/37962 [77:51:43<9:58:47,  8.29s/it]
 89%|████████▊ | 33628/37962 [77:51:51<9:55:00,  8.24s/it]
 89%|████████▊ | 33629/37962 [77:51:59<9:53:59,  8.23s/it]
 89%|████████▊ | 33630/37962 [77:52:07<9:49:33,  8.17s/it]
                                                          
{'loss': 1.1287, 'grad_norm': 7.143806457519531, 'learning_rate': 6.752414479443326e-07, 'epoch': 0.89}

 89%|████████▊ | 33630/37962 [77:52:07<9:49:33,  8.17s/it]
 89%|████████▊ | 33631/37962 [77:52:15<9:50:28,  8.18s/it]
 89%|████████▊ | 33632/37962 [77:52:23<9:41:29,  8.06s/it]
 89%|████████▊ | 33633/37962 [77:52:32<9:51:11,  8.19s/it]
 89%|████████▊ | 33634/37962 [77:52:40<9:56:31,  8.27s/it]
 89%|████████▊ | 33635/37962 [77:52:48<9:55:59,  8.26s/it]
 89%|████████▊ | 33636/37962 [77:52:57<9:58:21,  8.30s/it]
 89%|████████▊ | 33637/37962 [77:53:05<10:01:38,  8.35s/it]
 89%|████████▊ | 33638/37962 [77:53:13<9:58:46,  8.31s/it] 
 89%|████████▊ | 33639/37962 [77:53:21<9:52:28,  8.22s/it]
 89%|████████▊ | 33640/37962 [77:53:30<9:53:03,  8.23s/it]
                                                          
{'loss': 1.1925, 'grad_norm': 14.096282958984375, 'learning_rate': 6.721629508793648e-07, 'epoch': 0.89}

 89%|████████▊ | 33640/37962 [77:53:30<9:53:03,  8.23s/it]
 89%|████████▊ | 33641/37962 [77:53:38<9:55:52,  8.27s/it]
 89%|████████▊ | 33642/37962 [77:53:46<9:55:21,  8.27s/it]
 89%|████████▊ | 33643/37962 [77:53:55<9:57:02,  8.29s/it]
 89%|████████▊ | 33644/37962 [77:54:03<9:53:04,  8.24s/it]
 89%|████████▊ | 33645/37962 [77:54:11<9:54:17,  8.26s/it]
 89%|████████▊ | 33646/37962 [77:54:19<9:54:32,  8.27s/it]
 89%|████████▊ | 33647/37962 [77:54:27<9:48:34,  8.18s/it]
 89%|████████▊ | 33648/37962 [77:54:35<9:44:49,  8.13s/it]
 89%|████████▊ | 33649/37962 [77:54:44<9:45:54,  8.15s/it]
 89%|████████▊ | 33650/37962 [77:54:52<9:42:31,  8.11s/it]
                                                          
{'loss': 1.1855, 'grad_norm': 9.240187644958496, 'learning_rate': 6.690912433853924e-07, 'epoch': 0.89}

 89%|████████▊ | 33650/37962 [77:54:52<9:42:31,  8.11s/it]
 89%|████████▊ | 33651/37962 [77:55:00<9:46:16,  8.16s/it]
 89%|████████▊ | 33652/37962 [77:55:08<9:50:37,  8.22s/it]
 89%|████████▊ | 33653/37962 [77:55:17<9:50:58,  8.23s/it]
 89%|████████▊ | 33654/37962 [77:55:25<9:52:52,  8.26s/it]
 89%|████████▊ | 33655/37962 [77:55:33<9:52:45,  8.26s/it]
 89%|████████▊ | 33656/37962 [77:55:41<9:52:02,  8.25s/it]
 89%|████████▊ | 33657/37962 [77:55:50<9:52:37,  8.26s/it]
 89%|████████▊ | 33658/37962 [77:55:58<9:52:41,  8.26s/it]
 89%|████████▊ | 33659/37962 [77:56:06<9:50:30,  8.23s/it]
 89%|████████▊ | 33660/37962 [77:56:14<9:52:41,  8.27s/it]
                                                          
{'loss': 1.2127, 'grad_norm': 11.503029823303223, 'learning_rate': 6.660263276982593e-07, 'epoch': 0.89}

 89%|████████▊ | 33660/37962 [77:56:14<9:52:41,  8.27s/it]
 89%|████████▊ | 33661/37962 [77:56:22<9:43:10,  8.14s/it]
 89%|████████▊ | 33662/37962 [77:56:30<9:45:29,  8.17s/it]
 89%|████████▊ | 33663/37962 [77:56:39<9:52:56,  8.28s/it]
 89%|████████▊ | 33664/37962 [77:56:47<9:51:31,  8.26s/it]
 89%|████████▊ | 33665/37962 [77:56:55<9:51:00,  8.25s/it]
 89%|████████▊ | 33666/37962 [77:57:04<9:51:40,  8.26s/it]
 89%|████████▊ | 33667/37962 [77:57:12<9:47:07,  8.20s/it]
 89%|████████▊ | 33668/37962 [77:57:20<9:48:11,  8.22s/it]
 89%|████████▊ | 33669/37962 [77:57:28<9:47:28,  8.21s/it]
 89%|████████▊ | 33670/37962 [77:57:36<9:46:57,  8.21s/it]
                                                          
{'loss': 1.1358, 'grad_norm': 7.248349189758301, 'learning_rate': 6.629682060488596e-07, 'epoch': 0.89}

 89%|████████▊ | 33670/37962 [77:57:36<9:46:57,  8.21s/it]
 89%|████████▊ | 33671/37962 [77:57:45<9:52:41,  8.29s/it]
 89%|████████▊ | 33672/37962 [77:57:53<9:50:30,  8.26s/it]
 89%|████████▊ | 33673/37962 [77:58:01<9:52:40,  8.29s/it]
 89%|████████▊ | 33674/37962 [77:58:10<9:53:31,  8.30s/it]
 89%|████████▊ | 33675/37962 [77:58:18<9:53:53,  8.31s/it]
 89%|████████▊ | 33676/37962 [77:58:26<9:46:20,  8.21s/it]
 89%|████████▊ | 33677/37962 [77:58:35<9:51:04,  8.28s/it]
 89%|████████▊ | 33678/37962 [77:58:43<9:51:25,  8.28s/it]
 89%|████████▊ | 33679/37962 [77:58:51<9:49:46,  8.26s/it]
 89%|████████▊ | 33680/37962 [77:58:59<9:50:11,  8.27s/it]
                                                          
{'loss': 1.1697, 'grad_norm': 13.219300270080566, 'learning_rate': 6.599168806631496e-07, 'epoch': 0.89}

 89%|████████▊ | 33680/37962 [77:58:59<9:50:11,  8.27s/it]
 89%|████████▊ | 33681/37962 [77:59:08<9:53:25,  8.32s/it]
 89%|████████▊ | 33682/37962 [77:59:16<9:50:40,  8.28s/it]
 89%|████████▊ | 33683/37962 [77:59:24<9:52:16,  8.30s/it]
 89%|████████▊ | 33684/37962 [77:59:33<9:50:33,  8.28s/it]
 89%|████████▊ | 33685/37962 [77:59:41<9:52:33,  8.31s/it]
 89%|████████▊ | 33686/37962 [77:59:49<9:50:53,  8.29s/it]
 89%|████████▊ | 33687/37962 [77:59:58<9:51:41,  8.30s/it]
 89%|████████▊ | 33688/37962 [78:00:06<9:54:29,  8.35s/it]
 89%|████████▊ | 33689/37962 [78:00:14<9:52:40,  8.32s/it]
 89%|████████▊ | 33690/37962 [78:00:22<9:50:00,  8.29s/it]
                                                          
{'loss': 1.1618, 'grad_norm': 6.271262168884277, 'learning_rate': 6.568723537621368e-07, 'epoch': 0.89}

 89%|████████▊ | 33690/37962 [78:00:22<9:50:00,  8.29s/it]
 89%|████████▊ | 33691/37962 [78:00:30<9:40:55,  8.16s/it]
 89%|████████▉ | 33692/37962 [78:00:38<9:36:27,  8.10s/it]
 89%|████████▉ | 33693/37962 [78:00:46<9:38:36,  8.13s/it]
 89%|████████▉ | 33694/37962 [78:00:55<9:40:21,  8.16s/it]
 89%|████████▉ | 33695/37962 [78:01:03<9:41:32,  8.18s/it]
 89%|████████▉ | 33696/37962 [78:01:11<9:43:22,  8.21s/it]
 89%|████████▉ | 33697/37962 [78:01:19<9:43:21,  8.21s/it]
 89%|████████▉ | 33698/37962 [78:01:28<9:43:40,  8.21s/it]
 89%|████████▉ | 33699/37962 [78:01:36<9:44:54,  8.23s/it]
 89%|████████▉ | 33700/37962 [78:01:44<9:44:22,  8.23s/it]
                                                          
{'loss': 1.1378, 'grad_norm': 8.143566131591797, 'learning_rate': 6.53834627561879e-07, 'epoch': 0.89}

 89%|████████▉ | 33700/37962 [78:01:44<9:44:22,  8.23s/it]
 89%|████████▉ | 33701/37962 [78:01:52<9:44:26,  8.23s/it]
 89%|████████▉ | 33702/37962 [78:02:01<9:48:15,  8.29s/it]
 89%|████████▉ | 33703/37962 [78:02:09<9:52:14,  8.34s/it]
 89%|████████▉ | 33704/37962 [78:02:17<9:46:01,  8.26s/it]
 89%|████████▉ | 33705/37962 [78:02:26<9:47:19,  8.28s/it]
 89%|████████▉ | 33706/37962 [78:02:34<9:49:09,  8.31s/it]
 89%|████████▉ | 33707/37962 [78:02:42<9:47:49,  8.29s/it]
 89%|████████▉ | 33708/37962 [78:02:50<9:46:12,  8.27s/it]
 89%|████████▉ | 33709/37962 [78:02:59<9:46:15,  8.27s/it]
 89%|████████▉ | 33710/37962 [78:03:07<9:49:06,  8.31s/it]
                                                          
{'loss': 1.1235, 'grad_norm': 9.58728313446045, 'learning_rate': 6.50803704273485e-07, 'epoch': 0.89}

 89%|████████▉ | 33710/37962 [78:03:07<9:49:06,  8.31s/it]
 89%|████████▉ | 33711/37962 [78:03:15<9:47:32,  8.29s/it]
 89%|████████▉ | 33712/37962 [78:03:24<9:48:15,  8.30s/it]
 89%|████████▉ | 33713/37962 [78:03:32<9:41:16,  8.21s/it]
 89%|████████▉ | 33714/37962 [78:03:40<9:43:07,  8.24s/it]
 89%|████████▉ | 33715/37962 [78:03:48<9:44:54,  8.26s/it]
 89%|████████▉ | 33716/37962 [78:03:57<9:48:59,  8.32s/it]
 89%|████████▉ | 33717/37962 [78:04:05<9:48:48,  8.32s/it]
 89%|████████▉ | 33718/37962 [78:04:13<9:48:04,  8.31s/it]
 89%|████████▉ | 33719/37962 [78:04:22<9:51:16,  8.36s/it]
 89%|████████▉ | 33720/37962 [78:04:30<9:51:04,  8.36s/it]
                                                          
{'loss': 1.1556, 'grad_norm': 17.685007095336914, 'learning_rate': 6.477795861031122e-07, 'epoch': 0.89}

 89%|████████▉ | 33720/37962 [78:04:30<9:51:04,  8.36s/it]
 89%|████████▉ | 33721/37962 [78:04:39<9:49:24,  8.34s/it]
 89%|████████▉ | 33722/37962 [78:04:47<9:46:23,  8.30s/it]
 89%|████████▉ | 33723/37962 [78:04:55<9:41:56,  8.24s/it]
 89%|████████▉ | 33724/37962 [78:05:03<9:46:57,  8.31s/it]
 89%|████████▉ | 33725/37962 [78:05:11<9:39:43,  8.21s/it]
 89%|████████▉ | 33726/37962 [78:05:20<9:43:42,  8.27s/it]
 89%|████████▉ | 33727/37962 [78:05:28<9:44:56,  8.29s/it]
 89%|████████▉ | 33728/37962 [78:05:36<9:43:51,  8.27s/it]
 89%|████████▉ | 33729/37962 [78:05:45<9:43:25,  8.27s/it]
 89%|████████▉ | 33730/37962 [78:05:53<9:44:59,  8.29s/it]
                                                          
{'loss': 1.0611, 'grad_norm': 15.417120933532715, 'learning_rate': 6.4476227525196e-07, 'epoch': 0.89}

 89%|████████▉ | 33730/37962 [78:05:53<9:44:59,  8.29s/it]
 89%|████████▉ | 33731/37962 [78:06:01<9:43:29,  8.27s/it]
 89%|████████▉ | 33732/37962 [78:06:09<9:41:30,  8.25s/it]
 89%|████████▉ | 33733/37962 [78:06:17<9:38:37,  8.21s/it]
 89%|████████▉ | 33734/37962 [78:06:25<9:34:03,  8.15s/it]
 89%|████████▉ | 33735/37962 [78:06:34<9:37:19,  8.19s/it]
 89%|████████▉ | 33736/37962 [78:06:42<9:37:23,  8.20s/it]
 89%|████████▉ | 33737/37962 [78:06:51<9:46:18,  8.33s/it]
 89%|████████▉ | 33738/37962 [78:06:59<9:47:09,  8.34s/it]
 89%|████████▉ | 33739/37962 [78:07:07<9:45:48,  8.32s/it]
 89%|████████▉ | 33740/37962 [78:07:16<9:45:06,  8.32s/it]
                                                          
{'loss': 1.2015, 'grad_norm': 12.311787605285645, 'learning_rate': 6.417517739162804e-07, 'epoch': 0.89}

 89%|████████▉ | 33740/37962 [78:07:16<9:45:06,  8.32s/it]
 89%|████████▉ | 33741/37962 [78:07:24<9:40:14,  8.25s/it]
 89%|████████▉ | 33742/37962 [78:07:32<9:41:02,  8.26s/it]
 89%|████████▉ | 33743/37962 [78:07:40<9:43:41,  8.30s/it]
 89%|████████▉ | 33744/37962 [78:07:49<9:42:28,  8.29s/it]
 89%|████████▉ | 33745/37962 [78:07:57<9:40:38,  8.26s/it]
 89%|████████▉ | 33746/37962 [78:08:05<9:42:36,  8.29s/it]
 89%|████████▉ | 33747/37962 [78:08:14<9:44:54,  8.33s/it]
 89%|████████▉ | 33748/37962 [78:08:21<9:34:42,  8.18s/it]
 89%|████████▉ | 33749/37962 [78:08:37<12:16:46, 10.49s/it]
 89%|████████▉ | 33750/37962 [78:08:45<11:26:47,  9.78s/it]
                                                           
{'loss': 1.0907, 'grad_norm': 11.058486938476562, 'learning_rate': 6.387480842873639e-07, 'epoch': 0.89}

 89%|████████▉ | 33750/37962 [78:08:45<11:26:47,  9.78s/it]
 89%|████████▉ | 33751/37962 [78:08:54<11:00:26,  9.41s/it]
 89%|████████▉ | 33752/37962 [78:09:02<10:27:35,  8.94s/it]
 89%|████████▉ | 33753/37962 [78:09:10<10:11:13,  8.71s/it]
 89%|████████▉ | 33754/37962 [78:09:18<9:49:11,  8.40s/it] 
 89%|████████▉ | 33755/37962 [78:09:26<9:49:17,  8.40s/it]
 89%|████████▉ | 33756/37962 [78:09:34<9:42:52,  8.31s/it]
 89%|████████▉ | 33757/37962 [78:09:42<9:43:21,  8.32s/it]
 89%|████████▉ | 33758/37962 [78:09:51<9:41:05,  8.29s/it]
 89%|████████▉ | 33759/37962 [78:09:59<9:40:46,  8.29s/it]
 89%|████████▉ | 33760/37962 [78:10:07<9:41:05,  8.30s/it]
                                                          
{'loss': 1.14, 'grad_norm': 11.436271667480469, 'learning_rate': 6.357512085515439e-07, 'epoch': 0.89}

 89%|████████▉ | 33760/37962 [78:10:07<9:41:05,  8.30s/it]
 89%|████████▉ | 33761/37962 [78:10:15<9:38:21,  8.26s/it]
 89%|████████▉ | 33762/37962 [78:10:24<9:42:46,  8.33s/it]
 89%|████████▉ | 33763/37962 [78:10:32<9:45:24,  8.36s/it]
 89%|████████▉ | 33764/37962 [78:10:41<9:39:51,  8.29s/it]
 89%|████████▉ | 33765/37962 [78:10:49<9:38:41,  8.27s/it]
 89%|████████▉ | 33766/37962 [78:10:57<9:36:00,  8.24s/it]
 89%|████████▉ | 33767/37962 [78:11:05<9:36:22,  8.24s/it]
 89%|████████▉ | 33768/37962 [78:11:13<9:35:13,  8.23s/it]
 89%|████████▉ | 33769/37962 [78:11:22<9:34:19,  8.22s/it]
 89%|████████▉ | 33770/37962 [78:11:30<9:30:16,  8.16s/it]
                                                          
{'loss': 1.1333, 'grad_norm': 14.99983024597168, 'learning_rate': 6.327611488901964e-07, 'epoch': 0.89}

 89%|████████▉ | 33770/37962 [78:11:30<9:30:16,  8.16s/it]
 89%|████████▉ | 33771/37962 [78:11:37<9:21:12,  8.03s/it]
 89%|████████▉ | 33772/37962 [78:11:46<9:24:13,  8.08s/it]
 89%|████████▉ | 33773/37962 [78:11:54<9:23:26,  8.07s/it]
 89%|████████▉ | 33774/37962 [78:12:02<9:25:08,  8.10s/it]
 89%|████████▉ | 33775/37962 [78:12:10<9:28:16,  8.14s/it]
 89%|████████▉ | 33776/37962 [78:12:18<9:32:03,  8.20s/it]
 89%|████████▉ | 33777/37962 [78:12:27<9:34:41,  8.24s/it]
 89%|████████▉ | 33778/37962 [78:12:35<9:38:57,  8.30s/it]
 89%|████████▉ | 33779/37962 [78:12:43<9:30:32,  8.18s/it]
 89%|████████▉ | 33780/37962 [78:12:51<9:33:43,  8.23s/it]
                                                          
{'loss': 1.1686, 'grad_norm': 10.018074035644531, 'learning_rate': 6.297779074797306e-07, 'epoch': 0.89}

 89%|████████▉ | 33780/37962 [78:12:51<9:33:43,  8.23s/it]
 89%|████████▉ | 33781/37962 [78:13:00<9:35:16,  8.26s/it]
 89%|████████▉ | 33782/37962 [78:13:08<9:37:33,  8.29s/it]
 89%|████████▉ | 33783/37962 [78:13:16<9:36:18,  8.27s/it]
 89%|████████▉ | 33784/37962 [78:13:24<9:35:17,  8.26s/it]
 89%|████████▉ | 33785/37962 [78:13:33<9:40:06,  8.33s/it]
 89%|████████▉ | 33786/37962 [78:13:41<9:37:52,  8.30s/it]
 89%|████████▉ | 33787/37962 [78:13:49<9:34:51,  8.26s/it]
 89%|████████▉ | 33788/37962 [78:13:58<9:37:03,  8.30s/it]
 89%|████████▉ | 33789/37962 [78:14:06<9:43:41,  8.39s/it]
 89%|████████▉ | 33790/37962 [78:14:14<9:37:57,  8.31s/it]
                                                          
{'loss': 1.1251, 'grad_norm': 12.41059684753418, 'learning_rate': 6.268014864915972e-07, 'epoch': 0.89}

 89%|████████▉ | 33790/37962 [78:14:14<9:37:57,  8.31s/it]
 89%|████████▉ | 33791/37962 [78:14:23<9:35:03,  8.27s/it]
 89%|████████▉ | 33792/37962 [78:14:31<9:35:49,  8.29s/it]
 89%|████████▉ | 33793/37962 [78:14:39<9:33:57,  8.26s/it]
 89%|████████▉ | 33794/37962 [78:14:47<9:28:49,  8.19s/it]
 89%|████████▉ | 33795/37962 [78:14:56<9:33:18,  8.25s/it]
 89%|████████▉ | 33796/37962 [78:15:04<9:33:12,  8.26s/it]
 89%|████████▉ | 33797/37962 [78:15:12<9:30:54,  8.22s/it]
 89%|████████▉ | 33798/37962 [78:15:20<9:25:53,  8.15s/it]
 89%|████████▉ | 33799/37962 [78:15:28<9:30:27,  8.22s/it]
 89%|████████▉ | 33800/37962 [78:15:37<9:35:02,  8.29s/it]
                                                          
{'loss': 1.1535, 'grad_norm': 10.920182228088379, 'learning_rate': 6.238318880922845e-07, 'epoch': 0.89}

 89%|████████▉ | 33800/37962 [78:15:37<9:35:02,  8.29s/it]
 89%|████████▉ | 33801/37962 [78:15:45<9:37:25,  8.33s/it]
 89%|████████▉ | 33802/37962 [78:15:53<9:34:53,  8.29s/it]
 89%|████████▉ | 33803/37962 [78:16:01<9:24:43,  8.15s/it]
 89%|████████▉ | 33804/37962 [78:16:09<9:22:30,  8.12s/it]
 89%|████████▉ | 33805/37962 [78:16:18<9:24:48,  8.15s/it]
 89%|████████▉ | 33806/37962 [78:16:26<9:26:57,  8.19s/it]
 89%|████████▉ | 33807/37962 [78:16:34<9:28:02,  8.20s/it]
 89%|████████▉ | 33808/37962 [78:16:42<9:24:13,  8.15s/it]
 89%|████████▉ | 33809/37962 [78:16:50<9:26:44,  8.19s/it]
 89%|████████▉ | 33810/37962 [78:16:59<9:26:19,  8.18s/it]
                                                          
{'loss': 1.0321, 'grad_norm': 8.647796630859375, 'learning_rate': 6.208691144433077e-07, 'epoch': 0.89}

 89%|████████▉ | 33810/37962 [78:16:59<9:26:19,  8.18s/it]
 89%|████████▉ | 33811/37962 [78:17:06<9:18:34,  8.07s/it]
 89%|████████▉ | 33812/37962 [78:17:15<9:23:17,  8.14s/it]
 89%|████████▉ | 33813/37962 [78:17:23<9:27:13,  8.20s/it]
 89%|████████▉ | 33814/37962 [78:17:32<9:34:57,  8.32s/it]
 89%|████████▉ | 33815/37962 [78:17:40<9:32:34,  8.28s/it]
 89%|████████▉ | 33816/37962 [78:17:48<9:33:31,  8.30s/it]
 89%|████████▉ | 33817/37962 [78:17:56<9:32:13,  8.28s/it]
 89%|████████▉ | 33818/37962 [78:18:05<9:34:03,  8.31s/it]
 89%|████████▉ | 33819/37962 [78:18:13<9:28:18,  8.23s/it]
 89%|████████▉ | 33820/37962 [78:18:21<9:29:42,  8.25s/it]
                                                          
{'loss': 1.1692, 'grad_norm': 17.016326904296875, 'learning_rate': 6.179131677012218e-07, 'epoch': 0.89}

 89%|████████▉ | 33820/37962 [78:18:21<9:29:42,  8.25s/it]
 89%|████████▉ | 33821/37962 [78:18:29<9:20:25,  8.12s/it]
 89%|████████▉ | 33822/37962 [78:18:37<9:23:51,  8.17s/it]
 89%|████████▉ | 33823/37962 [78:18:45<9:18:30,  8.10s/it]
 89%|████████▉ | 33824/37962 [78:18:53<9:22:28,  8.16s/it]
 89%|████████▉ | 33825/37962 [78:19:02<9:23:51,  8.18s/it]
 89%|████████▉ | 33826/37962 [78:19:10<9:20:41,  8.13s/it]
 89%|████████▉ | 33827/37962 [78:19:18<9:25:43,  8.21s/it]
 89%|████████▉ | 33828/37962 [78:19:26<9:21:53,  8.16s/it]
 89%|████████▉ | 33829/37962 [78:19:34<9:16:32,  8.08s/it]
 89%|████████▉ | 33830/37962 [78:19:42<9:20:51,  8.14s/it]
                                                          
{'loss': 1.1825, 'grad_norm': 6.892329692840576, 'learning_rate': 6.149640500176068e-07, 'epoch': 0.89}

 89%|████████▉ | 33830/37962 [78:19:42<9:20:51,  8.14s/it]
 89%|████████▉ | 33831/37962 [78:19:51<9:24:08,  8.19s/it]
 89%|████████▉ | 33832/37962 [78:19:59<9:26:00,  8.22s/it]
 89%|████████▉ | 33833/37962 [78:20:07<9:25:58,  8.22s/it]
 89%|████████▉ | 33834/37962 [78:20:15<9:20:07,  8.14s/it]
 89%|████████▉ | 33835/37962 [78:20:24<9:26:01,  8.23s/it]
 89%|████████▉ | 33836/37962 [78:20:32<9:26:21,  8.24s/it]
 89%|████████▉ | 33837/37962 [78:20:40<9:26:26,  8.24s/it]
 89%|████████▉ | 33838/37962 [78:20:48<9:29:56,  8.29s/it]
 89%|████████▉ | 33839/37962 [78:20:57<9:30:38,  8.30s/it]
 89%|████████▉ | 33840/37962 [78:21:05<9:29:47,  8.29s/it]
                                                          
{'loss': 1.121, 'grad_norm': 9.525321960449219, 'learning_rate': 6.120217635390746e-07, 'epoch': 0.89}

 89%|████████▉ | 33840/37962 [78:21:05<9:29:47,  8.29s/it]
 89%|████████▉ | 33841/37962 [78:21:13<9:27:11,  8.26s/it]
 89%|████████▉ | 33842/37962 [78:21:21<9:26:26,  8.25s/it]
 89%|████████▉ | 33843/37962 [78:21:30<9:26:05,  8.25s/it]
 89%|████████▉ | 33844/37962 [78:21:38<9:30:32,  8.31s/it]
 89%|████████▉ | 33845/37962 [78:21:46<9:26:05,  8.25s/it]
 89%|████████▉ | 33846/37962 [78:21:55<9:27:30,  8.27s/it]
 89%|████████▉ | 33847/37962 [78:22:03<9:21:04,  8.18s/it]
 89%|████████▉ | 33848/37962 [78:22:11<9:25:55,  8.25s/it]
 89%|████████▉ | 33849/37962 [78:22:19<9:25:15,  8.25s/it]
 89%|████████▉ | 33850/37962 [78:22:27<9:25:43,  8.25s/it]
                                                          
{'loss': 1.1909, 'grad_norm': 13.722214698791504, 'learning_rate': 6.090863104072664e-07, 'epoch': 0.89}

 89%|████████▉ | 33850/37962 [78:22:27<9:25:43,  8.25s/it]
 89%|████████▉ | 33851/37962 [78:22:36<9:26:22,  8.27s/it]
 89%|████████▉ | 33852/37962 [78:22:44<9:25:27,  8.25s/it]
 89%|████████▉ | 33853/37962 [78:22:52<9:19:44,  8.17s/it]
 89%|████████▉ | 33854/37962 [78:23:00<9:20:18,  8.18s/it]
 89%|████████▉ | 33855/37962 [78:23:08<9:19:33,  8.17s/it]
 89%|████████▉ | 33856/37962 [78:23:16<9:17:43,  8.15s/it]
 89%|████████▉ | 33857/37962 [78:23:25<9:20:25,  8.19s/it]
 89%|████████▉ | 33858/37962 [78:23:33<9:25:00,  8.26s/it]
 89%|████████▉ | 33859/37962 [78:23:41<9:22:12,  8.22s/it]
 89%|████████▉ | 33860/37962 [78:23:49<9:19:20,  8.18s/it]
                                                          
{'loss': 1.1035, 'grad_norm': 11.378910064697266, 'learning_rate': 6.061576927588452e-07, 'epoch': 0.89}

 89%|████████▉ | 33860/37962 [78:23:49<9:19:20,  8.18s/it]
 89%|████████▉ | 33861/37962 [78:23:58<9:25:29,  8.27s/it]
 89%|████████▉ | 33862/37962 [78:24:06<9:24:37,  8.26s/it]
 89%|████████▉ | 33863/37962 [78:24:14<9:18:26,  8.17s/it]
 89%|████████▉ | 33864/37962 [78:24:22<9:23:43,  8.25s/it]
 89%|████████▉ | 33865/37962 [78:24:31<9:24:41,  8.27s/it]
 89%|████████▉ | 33866/37962 [78:24:39<9:21:15,  8.22s/it]
 89%|████████▉ | 33867/37962 [78:24:47<9:20:25,  8.21s/it]
 89%|████████▉ | 33868/37962 [78:24:56<9:24:48,  8.28s/it]
 89%|████████▉ | 33869/37962 [78:25:04<9:27:25,  8.32s/it]
 89%|████████▉ | 33870/37962 [78:25:12<9:27:31,  8.32s/it]
                                                          
{'loss': 1.1253, 'grad_norm': 16.482786178588867, 'learning_rate': 6.032359127255027e-07, 'epoch': 0.89}

 89%|████████▉ | 33870/37962 [78:25:12<9:27:31,  8.32s/it]
 89%|████████▉ | 33871/37962 [78:25:21<9:31:36,  8.38s/it]
 89%|████████▉ | 33872/37962 [78:25:29<9:26:03,  8.30s/it]
 89%|████████▉ | 33873/37962 [78:25:37<9:25:56,  8.30s/it]
 89%|████████▉ | 33874/37962 [78:25:45<9:15:48,  8.16s/it]
 89%|████████▉ | 33875/37962 [78:25:53<9:15:12,  8.15s/it]
 89%|████████▉ | 33876/37962 [78:26:01<9:16:46,  8.18s/it]
 89%|████████▉ | 33877/37962 [78:26:09<9:15:05,  8.15s/it]
 89%|████████▉ | 33878/37962 [78:26:18<9:14:58,  8.15s/it]
 89%|████████▉ | 33879/37962 [78:26:26<9:18:39,  8.21s/it]
 89%|████████▉ | 33880/37962 [78:26:34<9:20:35,  8.24s/it]
                                                          
{'loss': 1.1326, 'grad_norm': 14.005863189697266, 'learning_rate': 6.003209724339531e-07, 'epoch': 0.89}

 89%|████████▉ | 33880/37962 [78:26:34<9:20:35,  8.24s/it]
 89%|████████▉ | 33881/37962 [78:26:43<9:24:12,  8.30s/it]
 89%|████████▉ | 33882/37962 [78:26:51<9:24:55,  8.31s/it]
 89%|████████▉ | 33883/37962 [78:26:59<9:25:15,  8.31s/it]
 89%|████████▉ | 33884/37962 [78:27:07<9:19:07,  8.23s/it]
 89%|████████▉ | 33885/37962 [78:27:16<9:19:18,  8.23s/it]
 89%|████████▉ | 33886/37962 [78:27:24<9:19:26,  8.24s/it]
 89%|████████▉ | 33887/37962 [78:27:32<9:15:31,  8.18s/it]
 89%|████████▉ | 33888/37962 [78:27:40<9:13:44,  8.16s/it]
 89%|████████▉ | 33889/37962 [78:27:48<9:18:21,  8.23s/it]
 89%|████████▉ | 33890/37962 [78:27:57<9:18:58,  8.24s/it]
                                                          
{'loss': 1.1292, 'grad_norm': 10.064102172851562, 'learning_rate': 5.974128740059271e-07, 'epoch': 0.89}

 89%|████████▉ | 33890/37962 [78:27:57<9:18:58,  8.24s/it]
 89%|████████▉ | 33891/37962 [78:28:05<9:21:16,  8.27s/it]
 89%|████████▉ | 33892/37962 [78:28:13<9:18:35,  8.23s/it]
 89%|████████▉ | 33893/37962 [78:28:21<9:13:03,  8.16s/it]
 89%|████████▉ | 33894/37962 [78:28:29<9:15:46,  8.20s/it]
 89%|████████▉ | 33895/37962 [78:28:38<9:18:13,  8.24s/it]
 89%|████████▉ | 33896/37962 [78:28:46<9:17:41,  8.23s/it]
 89%|████████▉ | 33897/37962 [78:28:54<9:17:28,  8.23s/it]
 89%|████████▉ | 33898/37962 [78:29:03<9:20:16,  8.27s/it]
 89%|████████▉ | 33899/37962 [78:29:11<9:19:57,  8.27s/it]
 89%|████████▉ | 33900/37962 [78:29:19<9:15:06,  8.20s/it]
                                                          
{'loss': 1.1384, 'grad_norm': 9.63267993927002, 'learning_rate': 5.945116195581813e-07, 'epoch': 0.89}

 89%|████████▉ | 33900/37962 [78:29:19<9:15:06,  8.20s/it]
 89%|████████▉ | 33901/37962 [78:29:27<9:15:08,  8.20s/it]
 89%|████████▉ | 33902/37962 [78:29:35<9:11:10,  8.15s/it]
 89%|████████▉ | 33903/37962 [78:29:44<9:16:51,  8.23s/it]
 89%|████████▉ | 33904/37962 [78:29:52<9:16:32,  8.23s/it]
 89%|████████▉ | 33905/37962 [78:30:00<9:20:25,  8.29s/it]
 89%|████████▉ | 33906/37962 [78:30:09<9:23:12,  8.33s/it]
 89%|████████▉ | 33907/37962 [78:30:17<9:19:46,  8.28s/it]
 89%|████████▉ | 33908/37962 [78:30:25<9:20:02,  8.29s/it]
 89%|████████▉ | 33909/37962 [78:30:33<9:16:22,  8.24s/it]
 89%|████████▉ | 33910/37962 [78:30:42<9:18:31,  8.27s/it]
                                                          
{'loss': 1.1708, 'grad_norm': 15.928169250488281, 'learning_rate': 5.916172112024898e-07, 'epoch': 0.89}

 89%|████████▉ | 33910/37962 [78:30:42<9:18:31,  8.27s/it]
 89%|████████▉ | 33911/37962 [78:30:50<9:19:00,  8.28s/it]
 89%|████████▉ | 33912/37962 [78:30:58<9:18:06,  8.27s/it]
 89%|████████▉ | 33913/37962 [78:31:07<9:20:48,  8.31s/it]
 89%|████████▉ | 33914/37962 [78:31:15<9:19:42,  8.30s/it]
 89%|████████▉ | 33915/37962 [78:31:23<9:19:03,  8.29s/it]
 89%|████████▉ | 33916/37962 [78:31:31<9:20:56,  8.32s/it]
 89%|████████▉ | 33917/37962 [78:31:40<9:18:43,  8.29s/it]
 89%|████████▉ | 33918/37962 [78:31:48<9:15:58,  8.25s/it]
 89%|████████▉ | 33919/37962 [78:31:56<9:07:02,  8.12s/it]
 89%|████████▉ | 33920/37962 [78:32:04<9:08:58,  8.15s/it]
                                                          
{'loss': 1.2091, 'grad_norm': 12.025712013244629, 'learning_rate': 5.887296510456408e-07, 'epoch': 0.89}

 89%|████████▉ | 33920/37962 [78:32:04<9:08:58,  8.15s/it]
 89%|████████▉ | 33921/37962 [78:32:12<9:09:33,  8.16s/it]
 89%|████████▉ | 33922/37962 [78:32:20<9:09:53,  8.17s/it]
 89%|████████▉ | 33923/37962 [78:32:28<9:06:50,  8.12s/it]
 89%|████████▉ | 33924/37962 [78:32:37<9:15:40,  8.26s/it]
 89%|████████▉ | 33925/37962 [78:32:45<9:16:26,  8.27s/it]
 89%|████████▉ | 33926/37962 [78:32:53<9:16:21,  8.27s/it]
 89%|████████▉ | 33927/37962 [78:33:02<9:13:50,  8.24s/it]
 89%|████████▉ | 33928/37962 [78:33:10<9:16:23,  8.28s/it]
 89%|████████▉ | 33929/37962 [78:33:18<9:13:16,  8.23s/it]
 89%|████████▉ | 33930/37962 [78:33:26<9:14:04,  8.25s/it]
                                                          
{'loss': 1.1342, 'grad_norm': 7.9270920753479, 'learning_rate': 5.858489411894397e-07, 'epoch': 0.89}

 89%|████████▉ | 33930/37962 [78:33:26<9:14:04,  8.25s/it]
 89%|████████▉ | 33931/37962 [78:33:35<9:16:24,  8.28s/it]
 89%|████████▉ | 33932/37962 [78:33:43<9:12:47,  8.23s/it]
 89%|████████▉ | 33933/37962 [78:33:51<9:13:57,  8.25s/it]
 89%|████████▉ | 33934/37962 [78:33:59<9:10:13,  8.20s/it]
 89%|████████▉ | 33935/37962 [78:34:07<9:10:05,  8.20s/it]
 89%|████████▉ | 33936/37962 [78:34:16<9:12:05,  8.23s/it]
 89%|████████▉ | 33937/37962 [78:34:24<9:16:36,  8.30s/it]
 89%|████████▉ | 33938/37962 [78:34:32<9:11:31,  8.22s/it]
 89%|████████▉ | 33939/37962 [78:34:41<9:13:49,  8.26s/it]
 89%|████████▉ | 33940/37962 [78:34:49<9:10:43,  8.22s/it]
                                                          
{'loss': 1.1861, 'grad_norm': 16.032176971435547, 'learning_rate': 5.829750837307068e-07, 'epoch': 0.89}

 89%|████████▉ | 33940/37962 [78:34:49<9:10:43,  8.22s/it]
 89%|████████▉ | 33941/37962 [78:34:57<9:12:32,  8.24s/it]
 89%|████████▉ | 33942/37962 [78:35:05<9:12:38,  8.25s/it]
 89%|████████▉ | 33943/37962 [78:35:13<9:10:28,  8.22s/it]
 89%|████████▉ | 33944/37962 [78:35:21<9:05:45,  8.15s/it]
 89%|████████▉ | 33945/37962 [78:35:30<9:09:30,  8.21s/it]
 89%|████████▉ | 33946/37962 [78:35:38<9:12:21,  8.25s/it]
 89%|████████▉ | 33947/37962 [78:35:46<9:14:18,  8.28s/it]
 89%|████████▉ | 33948/37962 [78:35:55<9:15:29,  8.30s/it]
 89%|████████▉ | 33949/37962 [78:36:03<9:16:25,  8.32s/it]
 89%|████████▉ | 33950/37962 [78:36:11<9:14:08,  8.29s/it]
                                                          
{'loss': 1.0836, 'grad_norm': 6.138912677764893, 'learning_rate': 5.801080807612691e-07, 'epoch': 0.89}

 89%|████████▉ | 33950/37962 [78:36:11<9:14:08,  8.29s/it]
 89%|████████▉ | 33951/37962 [78:36:20<9:13:10,  8.27s/it]
 89%|████████▉ | 33952/37962 [78:36:28<9:13:04,  8.28s/it]
 89%|████████▉ | 33953/37962 [78:36:36<9:13:52,  8.29s/it]
 89%|████████▉ | 33954/37962 [78:36:44<9:13:22,  8.28s/it]
 89%|████████▉ | 33955/37962 [78:36:53<9:11:05,  8.25s/it]
 89%|████████▉ | 33956/37962 [78:37:01<9:09:21,  8.23s/it]
 89%|████████▉ | 33957/37962 [78:37:09<9:08:43,  8.22s/it]
 89%|████████▉ | 33958/37962 [78:37:17<9:07:34,  8.21s/it]
 89%|████████▉ | 33959/37962 [78:37:26<9:13:08,  8.29s/it]
 89%|████████▉ | 33960/37962 [78:37:34<9:11:58,  8.28s/it]
                                                          
{'loss': 1.0801, 'grad_norm': 8.317326545715332, 'learning_rate': 5.772479343679715e-07, 'epoch': 0.89}

 89%|████████▉ | 33960/37962 [78:37:34<9:11:58,  8.28s/it]
 89%|████████▉ | 33961/37962 [78:37:42<9:03:43,  8.15s/it]
 89%|████████▉ | 33962/37962 [78:37:50<9:07:01,  8.21s/it]
 89%|████████▉ | 33963/37962 [78:37:58<9:09:43,  8.25s/it]
 89%|████████▉ | 33964/37962 [78:38:07<9:07:23,  8.22s/it]
 89%|████████▉ | 33965/37962 [78:38:15<9:14:30,  8.32s/it]
 89%|████████▉ | 33966/37962 [78:38:23<9:12:36,  8.30s/it]
 89%|████████▉ | 33967/37962 [78:38:32<9:16:36,  8.36s/it]
 89%|████████▉ | 33968/37962 [78:38:40<9:15:31,  8.35s/it]
 89%|████████▉ | 33969/37962 [78:38:49<9:15:08,  8.34s/it]
 89%|████████▉ | 33970/37962 [78:38:57<9:10:38,  8.28s/it]
                                                          
{'loss': 1.1327, 'grad_norm': 12.230846405029297, 'learning_rate': 5.743946466326666e-07, 'epoch': 0.89}

 89%|████████▉ | 33970/37962 [78:38:57<9:10:38,  8.28s/it]
 89%|████████▉ | 33971/37962 [78:39:05<9:14:07,  8.33s/it]
 89%|████████▉ | 33972/37962 [78:39:13<9:11:48,  8.30s/it]
 89%|████████▉ | 33973/37962 [78:39:22<9:13:27,  8.32s/it]
 89%|████████▉ | 33974/37962 [78:39:30<9:11:46,  8.30s/it]
 89%|████████▉ | 33975/37962 [78:39:38<9:13:46,  8.33s/it]
 90%|████████▉ | 33976/37962 [78:39:47<9:11:54,  8.31s/it]
 90%|████████▉ | 33977/37962 [78:39:55<9:12:42,  8.32s/it]
 90%|████████▉ | 33978/37962 [78:40:03<9:05:52,  8.22s/it]
 90%|████████▉ | 33979/37962 [78:40:11<9:08:41,  8.27s/it]
 90%|████████▉ | 33980/37962 [78:40:19<9:06:30,  8.23s/it]
                                                          
{'loss': 1.1457, 'grad_norm': 13.969659805297852, 'learning_rate': 5.715482196322098e-07, 'epoch': 0.9}

 90%|████████▉ | 33980/37962 [78:40:19<9:06:30,  8.23s/it]
 90%|████████▉ | 33981/37962 [78:40:27<9:00:01,  8.14s/it]
 90%|████████▉ | 33982/37962 [78:40:36<9:01:13,  8.16s/it]
 90%|████████▉ | 33983/37962 [78:40:44<9:02:07,  8.17s/it]
 90%|████████▉ | 33984/37962 [78:40:52<9:05:20,  8.23s/it]
 90%|████████▉ | 33985/37962 [78:41:01<9:07:36,  8.26s/it]
 90%|████████▉ | 33986/37962 [78:41:09<9:13:56,  8.36s/it]
 90%|████████▉ | 33987/37962 [78:41:18<9:15:05,  8.38s/it]
 90%|████████▉ | 33988/37962 [78:41:26<9:11:55,  8.33s/it]
 90%|████████▉ | 33989/37962 [78:41:34<9:13:31,  8.36s/it]
 90%|████████▉ | 33990/37962 [78:41:42<9:00:34,  8.17s/it]
                                                          
{'loss': 1.1134, 'grad_norm': 15.057814598083496, 'learning_rate': 5.687086554384658e-07, 'epoch': 0.9}

 90%|████████▉ | 33990/37962 [78:41:42<9:00:34,  8.17s/it]
 90%|████████▉ | 33991/37962 [78:41:50<9:01:51,  8.19s/it]
 90%|████████▉ | 33992/37962 [78:41:58<9:03:22,  8.21s/it]
 90%|████████▉ | 33993/37962 [78:42:07<9:10:29,  8.32s/it]
 90%|████████▉ | 33994/37962 [78:42:15<9:04:05,  8.23s/it]
 90%|████████▉ | 33995/37962 [78:42:23<9:03:54,  8.23s/it]
 90%|████████▉ | 33996/37962 [78:42:32<9:08:19,  8.30s/it]
 90%|████████▉ | 33997/37962 [78:42:40<9:05:41,  8.26s/it]
 90%|████████▉ | 33998/37962 [78:42:48<9:05:50,  8.26s/it]
 90%|████████▉ | 33999/37962 [78:42:56<9:05:03,  8.25s/it]
 90%|████████▉ | 34000/37962 [78:43:04<9:00:02,  8.18s/it]
                                                          
{'loss': 1.0638, 'grad_norm': 8.711353302001953, 'learning_rate': 5.65875956118308e-07, 'epoch': 0.9}

 90%|████████▉ | 34000/37962 [78:43:04<9:00:02,  8.18s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 90%|████████▉ | 34001/37962 [78:43:55<23:05:37, 20.99s/it]
 90%|████████▉ | 34002/37962 [78:44:03<18:53:20, 17.17s/it]
 90%|████████▉ | 34003/37962 [78:44:12<15:56:17, 14.49s/it]
 90%|████████▉ | 34004/37962 [78:44:20<13:53:51, 12.64s/it]
 90%|████████▉ | 34005/37962 [78:44:28<12:31:04, 11.39s/it]
 90%|████████▉ | 34006/37962 [78:44:37<11:31:29, 10.49s/it]
 90%|████████▉ | 34007/37962 [78:44:45<10:47:48,  9.83s/it]
 90%|████████▉ | 34008/37962 [78:44:54<10:22:08,  9.44s/it]
 90%|████████▉ | 34009/37962 [78:45:02<9:59:34,  9.10s/it] 
 90%|████████▉ | 34010/37962 [78:45:10<9:42:01,  8.84s/it]
                                                          
{'loss': 1.1945, 'grad_norm': 7.880745887756348, 'learning_rate': 5.63050123733605e-07, 'epoch': 0.9}

 90%|████████▉ | 34010/37962 [78:45:10<9:42:01,  8.84s/it]
 90%|████████▉ | 34011/37962 [78:45:19<9:33:44,  8.71s/it]
 90%|████████▉ | 34012/37962 [78:45:27<9:23:34,  8.56s/it]
 90%|████████▉ | 34013/37962 [78:45:35<9:12:10,  8.39s/it]
 90%|████████▉ | 34014/37962 [78:45:43<9:09:02,  8.34s/it]
 90%|████████▉ | 34015/37962 [78:45:51<9:08:29,  8.34s/it]
 90%|████████▉ | 34016/37962 [78:45:59<9:00:54,  8.22s/it]
 90%|████████▉ | 34017/37962 [78:46:08<9:02:21,  8.25s/it]
 90%|████████▉ | 34018/37962 [78:46:16<9:01:08,  8.23s/it]
 90%|████████▉ | 34019/37962 [78:46:24<8:59:31,  8.21s/it]
 90%|████████▉ | 34020/37962 [78:46:32<9:00:48,  8.23s/it]
                                                          
{'loss': 1.0468, 'grad_norm': 10.515129089355469, 'learning_rate': 5.602311603412336e-07, 'epoch': 0.9}

 90%|████████▉ | 34020/37962 [78:46:32<9:00:48,  8.23s/it]
 90%|████████▉ | 34021/37962 [78:46:41<9:01:46,  8.25s/it]
 90%|████████▉ | 34022/37962 [78:46:56<11:30:18, 10.51s/it]
 90%|████████▉ | 34023/37962 [78:47:05<10:46:10,  9.84s/it]
 90%|████████▉ | 34024/37962 [78:47:13<10:15:12,  9.37s/it]
 90%|████████▉ | 34025/37962 [78:47:21<9:52:58,  9.04s/it] 
 90%|████████▉ | 34026/37962 [78:47:30<9:39:47,  8.84s/it]
 90%|████████▉ | 34027/37962 [78:47:38<9:27:19,  8.65s/it]
 90%|████████▉ | 34028/37962 [78:47:46<9:23:27,  8.59s/it]
 90%|████████▉ | 34029/37962 [78:47:54<9:15:27,  8.47s/it]
 90%|████████▉ | 34030/37962 [78:48:03<9:11:39,  8.42s/it]
                                                          
{'loss': 1.0661, 'grad_norm': 9.075164794921875, 'learning_rate': 5.57419067993068e-07, 'epoch': 0.9}

 90%|████████▉ | 34030/37962 [78:48:03<9:11:39,  8.42s/it]
 90%|████████▉ | 34031/37962 [78:48:11<9:08:44,  8.38s/it]
 90%|████████▉ | 34032/37962 [78:48:19<9:05:50,  8.33s/it]
 90%|████████▉ | 34033/37962 [78:48:27<9:00:32,  8.25s/it]
 90%|████████▉ | 34034/37962 [78:48:36<9:00:22,  8.25s/it]
 90%|████████▉ | 34035/37962 [78:48:44<9:01:40,  8.28s/it]
 90%|████████▉ | 34036/37962 [78:48:52<9:02:22,  8.29s/it]
 90%|████████▉ | 34037/37962 [78:49:01<9:04:26,  8.32s/it]
 90%|████████▉ | 34038/37962 [78:49:09<9:04:04,  8.32s/it]
 90%|████████▉ | 34039/37962 [78:49:17<9:04:53,  8.33s/it]
 90%|████████▉ | 34040/37962 [78:49:25<8:54:41,  8.18s/it]
                                                          
{'loss': 1.0438, 'grad_norm': 16.623886108398438, 'learning_rate': 5.546138487359809e-07, 'epoch': 0.9}

 90%|████████▉ | 34040/37962 [78:49:25<8:54:41,  8.18s/it]
 90%|████████▉ | 34041/37962 [78:49:33<8:55:42,  8.20s/it]
 90%|████████▉ | 34042/37962 [78:49:42<8:58:01,  8.24s/it]
 90%|████████▉ | 34043/37962 [78:49:50<8:57:54,  8.24s/it]
 90%|████████▉ | 34044/37962 [78:49:58<9:01:27,  8.29s/it]
 90%|████████▉ | 34045/37962 [78:50:07<9:03:03,  8.32s/it]
 90%|████████▉ | 34046/37962 [78:50:15<9:00:30,  8.28s/it]
 90%|████████▉ | 34047/37962 [78:50:23<8:58:51,  8.26s/it]
 90%|████████▉ | 34048/37962 [78:50:31<8:59:40,  8.27s/it]
 90%|████████▉ | 34049/37962 [78:50:40<8:56:00,  8.22s/it]
 90%|████████▉ | 34050/37962 [78:50:48<8:57:37,  8.25s/it]
                                                          
{'loss': 1.2021, 'grad_norm': 14.254822731018066, 'learning_rate': 5.518155046118434e-07, 'epoch': 0.9}

 90%|████████▉ | 34050/37962 [78:50:48<8:57:37,  8.25s/it]
 90%|████████▉ | 34051/37962 [78:50:56<8:56:55,  8.24s/it]
 90%|████████▉ | 34052/37962 [78:51:04<8:59:38,  8.28s/it]
 90%|████████▉ | 34053/37962 [78:51:13<8:59:37,  8.28s/it]
 90%|████████▉ | 34054/37962 [78:51:21<9:01:29,  8.31s/it]
 90%|████████▉ | 34055/37962 [78:51:29<8:57:55,  8.26s/it]
 90%|████████▉ | 34056/37962 [78:51:38<8:58:51,  8.28s/it]
 90%|████████▉ | 34057/37962 [78:51:46<9:00:29,  8.30s/it]
 90%|████████▉ | 34058/37962 [78:51:54<8:52:23,  8.18s/it]
 90%|████████▉ | 34059/37962 [78:52:02<8:54:36,  8.22s/it]
 90%|████████▉ | 34060/37962 [78:52:10<8:52:05,  8.18s/it]
                                                          
{'loss': 1.1711, 'grad_norm': 5.599329948425293, 'learning_rate': 5.490240376575207e-07, 'epoch': 0.9}

 90%|████████▉ | 34060/37962 [78:52:10<8:52:05,  8.18s/it]
 90%|████████▉ | 34061/37962 [78:52:18<8:51:53,  8.18s/it]
 90%|████████▉ | 34062/37962 [78:52:27<8:53:06,  8.20s/it]
 90%|████████▉ | 34063/37962 [78:52:35<8:50:13,  8.16s/it]
 90%|████████▉ | 34064/37962 [78:52:43<8:51:43,  8.18s/it]
 90%|████████▉ | 34065/37962 [78:52:51<8:53:23,  8.21s/it]
 90%|████████▉ | 34066/37962 [78:53:00<8:54:37,  8.23s/it]
 90%|████████▉ | 34067/37962 [78:53:08<9:00:08,  8.32s/it]
 90%|████████▉ | 34068/37962 [78:53:16<8:58:08,  8.29s/it]
 90%|████████▉ | 34069/37962 [78:53:24<8:56:51,  8.27s/it]
 90%|████████▉ | 34070/37962 [78:53:33<9:00:01,  8.33s/it]
                                                          
{'loss': 1.1248, 'grad_norm': 22.06282615661621, 'learning_rate': 5.46239449904874e-07, 'epoch': 0.9}

 90%|████████▉ | 34070/37962 [78:53:33<9:00:01,  8.33s/it]
 90%|████████▉ | 34071/37962 [78:53:41<8:55:52,  8.26s/it]
 90%|████████▉ | 34072/37962 [78:53:49<8:50:42,  8.19s/it]
 90%|████████▉ | 34073/37962 [78:53:57<8:53:02,  8.22s/it]
 90%|████████▉ | 34074/37962 [78:54:06<8:58:09,  8.30s/it]
 90%|████████▉ | 34075/37962 [78:54:14<8:59:37,  8.33s/it]
 90%|████████▉ | 34076/37962 [78:54:23<8:58:21,  8.31s/it]
 90%|████████▉ | 34077/37962 [78:54:30<8:50:40,  8.20s/it]
 90%|████████▉ | 34078/37962 [78:54:39<8:53:56,  8.25s/it]
 90%|████████▉ | 34079/37962 [78:54:47<8:55:31,  8.27s/it]
 90%|████████▉ | 34080/37962 [78:54:56<8:57:22,  8.31s/it]
                                                          
{'loss': 1.2427, 'grad_norm': 11.838096618652344, 'learning_rate': 5.434617433807598e-07, 'epoch': 0.9}

 90%|████████▉ | 34080/37962 [78:54:56<8:57:22,  8.31s/it]
 90%|████████▉ | 34081/37962 [78:55:04<8:56:57,  8.30s/it]
 90%|████████▉ | 34082/37962 [78:55:13<9:04:32,  8.42s/it]
 90%|████████▉ | 34083/37962 [78:55:21<9:03:17,  8.40s/it]
 90%|████████▉ | 34084/37962 [78:55:29<9:02:41,  8.40s/it]
 90%|████████▉ | 34085/37962 [78:55:38<8:59:26,  8.35s/it]
 90%|████████▉ | 34086/37962 [78:55:46<8:54:28,  8.27s/it]
 90%|████████▉ | 34087/37962 [78:55:54<8:49:28,  8.20s/it]
 90%|████████▉ | 34088/37962 [78:56:02<8:47:26,  8.17s/it]
 90%|████████▉ | 34089/37962 [78:56:10<8:42:45,  8.10s/it]
 90%|████████▉ | 34090/37962 [78:56:18<8:50:12,  8.22s/it]
                                                          
{'loss': 1.069, 'grad_norm': 12.729964256286621, 'learning_rate': 5.406909201070154e-07, 'epoch': 0.9}

 90%|████████▉ | 34090/37962 [78:56:18<8:50:12,  8.22s/it]
 90%|████████▉ | 34091/37962 [78:56:27<8:53:04,  8.26s/it]
 90%|████████▉ | 34092/37962 [78:56:35<8:59:09,  8.36s/it]
 90%|████████▉ | 34093/37962 [78:56:43<8:56:07,  8.31s/it]
 90%|████████▉ | 34094/37962 [78:56:52<8:56:54,  8.33s/it]
 90%|████████▉ | 34095/37962 [78:57:00<8:59:16,  8.37s/it]
 90%|████████▉ | 34096/37962 [78:57:08<8:58:36,  8.36s/it]
 90%|████████▉ | 34097/37962 [78:57:17<8:52:11,  8.26s/it]
 90%|████████▉ | 34098/37962 [78:57:25<8:53:00,  8.28s/it]
 90%|████████▉ | 34099/37962 [78:57:33<8:54:21,  8.30s/it]
 90%|████████▉ | 34100/37962 [78:57:42<8:55:05,  8.31s/it]
                                                          
{'loss': 1.1839, 'grad_norm': 12.28861141204834, 'learning_rate': 5.379269821004795e-07, 'epoch': 0.9}

 90%|████████▉ | 34100/37962 [78:57:42<8:55:05,  8.31s/it]
 90%|████████▉ | 34101/37962 [78:57:50<8:53:13,  8.29s/it]
 90%|████████▉ | 34102/37962 [78:57:58<8:50:23,  8.24s/it]
 90%|████████▉ | 34103/37962 [78:58:06<8:48:25,  8.22s/it]
 90%|████████▉ | 34104/37962 [78:58:14<8:51:22,  8.26s/it]
 90%|████████▉ | 34105/37962 [78:58:23<8:51:21,  8.27s/it]
 90%|████████▉ | 34106/37962 [78:58:31<8:50:57,  8.26s/it]
 90%|████████▉ | 34107/37962 [78:58:40<8:56:57,  8.36s/it]
 90%|████████▉ | 34108/37962 [78:58:48<8:56:01,  8.34s/it]
 90%|████████▉ | 34109/37962 [78:58:56<8:53:03,  8.30s/it]
 90%|████████▉ | 34110/37962 [78:59:04<8:52:39,  8.30s/it]
                                                          
{'loss': 1.2031, 'grad_norm': 9.865238189697266, 'learning_rate': 5.351699313729763e-07, 'epoch': 0.9}

 90%|████████▉ | 34110/37962 [78:59:04<8:52:39,  8.30s/it]
 90%|████████▉ | 34111/37962 [78:59:13<8:55:07,  8.34s/it]
 90%|████████▉ | 34112/37962 [78:59:20<8:43:04,  8.15s/it]
 90%|████████▉ | 34113/37962 [78:59:29<8:46:53,  8.21s/it]
 90%|████████▉ | 34114/37962 [78:59:37<8:50:27,  8.27s/it]
 90%|████████▉ | 34115/37962 [78:59:46<8:50:48,  8.28s/it]
 90%|████████▉ | 34116/37962 [78:59:54<8:56:21,  8.37s/it]
 90%|████████▉ | 34117/37962 [79:00:02<8:54:07,  8.33s/it]
 90%|████████▉ | 34118/37962 [79:00:11<8:51:30,  8.30s/it]
 90%|████████▉ | 34119/37962 [79:00:19<8:53:00,  8.32s/it]
 90%|████████▉ | 34120/37962 [79:00:27<8:52:12,  8.31s/it]
                                                          
{'loss': 1.2736, 'grad_norm': 15.585233688354492, 'learning_rate': 5.324197699313105e-07, 'epoch': 0.9}

 90%|████████▉ | 34120/37962 [79:00:27<8:52:12,  8.31s/it]
 90%|████████▉ | 34121/37962 [79:00:36<8:57:36,  8.40s/it]
 90%|████████▉ | 34122/37962 [79:00:44<8:54:11,  8.35s/it]
 90%|████████▉ | 34123/37962 [79:00:52<8:52:48,  8.33s/it]
 90%|████████▉ | 34124/37962 [79:01:01<8:51:27,  8.31s/it]
 90%|████████▉ | 34125/37962 [79:01:09<8:49:04,  8.27s/it]
 90%|████████▉ | 34126/37962 [79:01:17<8:49:23,  8.28s/it]
 90%|████████▉ | 34127/37962 [79:01:25<8:49:02,  8.28s/it]
 90%|████████▉ | 34128/37962 [79:01:34<8:54:12,  8.36s/it]
 90%|████████▉ | 34129/37962 [79:01:42<8:51:06,  8.31s/it]
 90%|████████▉ | 34130/37962 [79:01:50<8:49:05,  8.28s/it]
                                                          
{'loss': 1.1695, 'grad_norm': 9.083556175231934, 'learning_rate': 5.296764997772819e-07, 'epoch': 0.9}

 90%|████████▉ | 34130/37962 [79:01:50<8:49:05,  8.28s/it]
 90%|████████▉ | 34131/37962 [79:01:58<8:43:56,  8.21s/it]
 90%|████████▉ | 34132/37962 [79:02:07<8:45:28,  8.23s/it]
 90%|████████▉ | 34133/37962 [79:02:15<8:45:35,  8.24s/it]
 90%|████████▉ | 34134/37962 [79:02:23<8:47:08,  8.26s/it]
 90%|████████▉ | 34135/37962 [79:02:31<8:46:31,  8.25s/it]
 90%|████████▉ | 34136/37962 [79:02:40<8:48:28,  8.29s/it]
 90%|████████▉ | 34137/37962 [79:02:48<8:47:11,  8.27s/it]
 90%|████████▉ | 34138/37962 [79:02:56<8:38:41,  8.14s/it]
 90%|████████▉ | 34139/37962 [79:03:04<8:44:13,  8.23s/it]
 90%|████████▉ | 34140/37962 [79:03:13<8:45:37,  8.25s/it]
                                                          
{'loss': 1.181, 'grad_norm': 8.976278305053711, 'learning_rate': 5.269401229076665e-07, 'epoch': 0.9}

 90%|████████▉ | 34140/37962 [79:03:13<8:45:37,  8.25s/it]
 90%|████████▉ | 34141/37962 [79:03:21<8:46:49,  8.27s/it]
 90%|████████▉ | 34142/37962 [79:03:29<8:45:06,  8.25s/it]
 90%|████████▉ | 34143/37962 [79:03:37<8:45:18,  8.25s/it]
 90%|████████▉ | 34144/37962 [79:03:46<8:46:40,  8.28s/it]
 90%|████████▉ | 34145/37962 [79:03:54<8:48:27,  8.31s/it]
 90%|████████▉ | 34146/37962 [79:04:02<8:48:23,  8.31s/it]
 90%|████████▉ | 34147/37962 [79:04:11<8:45:59,  8.27s/it]
 90%|████████▉ | 34148/37962 [79:04:19<8:46:49,  8.29s/it]
 90%|████████▉ | 34149/37962 [79:04:27<8:46:42,  8.29s/it]
 90%|████████▉ | 34150/37962 [79:04:35<8:45:59,  8.28s/it]
                                                          
{'loss': 1.1615, 'grad_norm': 9.39061164855957, 'learning_rate': 5.242106413142245e-07, 'epoch': 0.9}

 90%|████████▉ | 34150/37962 [79:04:35<8:45:59,  8.28s/it]
 90%|████████▉ | 34151/37962 [79:04:44<8:47:08,  8.30s/it]
 90%|████████▉ | 34152/37962 [79:04:52<8:48:31,  8.32s/it]
 90%|████████▉ | 34153/37962 [79:05:00<8:47:24,  8.31s/it]
 90%|████████▉ | 34154/37962 [79:05:09<8:45:20,  8.28s/it]
 90%|████████▉ | 34155/37962 [79:05:17<8:48:04,  8.32s/it]
 90%|████████▉ | 34156/37962 [79:05:25<8:41:00,  8.21s/it]
 90%|████████▉ | 34157/37962 [79:05:34<8:46:20,  8.30s/it]
 90%|████████▉ | 34158/37962 [79:05:42<8:45:44,  8.29s/it]
 90%|████████▉ | 34159/37962 [79:05:50<8:45:26,  8.29s/it]
 90%|████████▉ | 34160/37962 [79:05:58<8:46:21,  8.31s/it]
                                                          
{'loss': 1.1567, 'grad_norm': 9.694668769836426, 'learning_rate': 5.214880569837e-07, 'epoch': 0.9}

 90%|████████▉ | 34160/37962 [79:05:58<8:46:21,  8.31s/it]
 90%|████████▉ | 34161/37962 [79:06:07<8:48:56,  8.35s/it]
 90%|████████▉ | 34162/37962 [79:06:15<8:46:35,  8.31s/it]
 90%|████████▉ | 34163/37962 [79:06:23<8:40:46,  8.22s/it]
 90%|████████▉ | 34164/37962 [79:06:31<8:39:09,  8.20s/it]
 90%|████████▉ | 34165/37962 [79:06:40<8:42:00,  8.25s/it]
 90%|█████████ | 34166/37962 [79:06:48<8:46:01,  8.31s/it]
 90%|█████████ | 34167/37962 [79:06:56<8:43:41,  8.28s/it]
 90%|█████████ | 34168/37962 [79:07:05<8:45:16,  8.31s/it]
 90%|█████████ | 34169/37962 [79:07:13<8:36:57,  8.18s/it]
 90%|█████████ | 34170/37962 [79:07:21<8:39:00,  8.21s/it]
                                                          
{'loss': 1.1277, 'grad_norm': 8.248828887939453, 'learning_rate': 5.187723718978165e-07, 'epoch': 0.9}

 90%|█████████ | 34170/37962 [79:07:21<8:39:00,  8.21s/it]
 90%|█████████ | 34171/37962 [79:07:29<8:37:26,  8.19s/it]
 90%|█████████ | 34172/37962 [79:07:38<8:42:49,  8.28s/it]
 90%|█████████ | 34173/37962 [79:07:46<8:45:23,  8.32s/it]
 90%|█████████ | 34174/37962 [79:07:54<8:44:46,  8.31s/it]
 90%|█████████ | 34175/37962 [79:08:03<8:49:21,  8.39s/it]
 90%|█████████ | 34176/37962 [79:08:11<8:47:06,  8.35s/it]
 90%|█████████ | 34177/37962 [79:08:19<8:44:42,  8.32s/it]
 90%|█████████ | 34178/37962 [79:08:28<8:43:54,  8.31s/it]
 90%|█████████ | 34179/37962 [79:08:36<8:42:10,  8.28s/it]
 90%|█████████ | 34180/37962 [79:08:44<8:41:12,  8.27s/it]
                                                          
{'loss': 1.193, 'grad_norm': 9.504439353942871, 'learning_rate': 5.160635880332709e-07, 'epoch': 0.9}

 90%|█████████ | 34180/37962 [79:08:44<8:41:12,  8.27s/it]
 90%|█████████ | 34181/37962 [79:08:52<8:41:34,  8.28s/it]
 90%|█████████ | 34182/37962 [79:09:01<8:42:15,  8.29s/it]
 90%|█████████ | 34183/37962 [79:09:08<8:32:14,  8.13s/it]
 90%|█████████ | 34184/37962 [79:09:17<8:33:57,  8.16s/it]
 90%|█████████ | 34185/37962 [79:09:25<8:37:35,  8.22s/it]
 90%|█████████ | 34186/37962 [79:09:33<8:41:36,  8.29s/it]
 90%|█████████ | 34187/37962 [79:09:42<8:41:32,  8.29s/it]
 90%|█████████ | 34188/37962 [79:09:50<8:41:56,  8.30s/it]
 90%|█████████ | 34189/37962 [79:09:58<8:28:46,  8.09s/it]
 90%|█████████ | 34190/37962 [79:10:06<8:31:14,  8.13s/it]
                                                          
{'loss': 1.1369, 'grad_norm': 10.71901798248291, 'learning_rate': 5.133617073617425e-07, 'epoch': 0.9}

 90%|█████████ | 34190/37962 [79:10:06<8:31:14,  8.13s/it]
 90%|█████████ | 34191/37962 [79:10:14<8:27:33,  8.08s/it]
 90%|█████████ | 34192/37962 [79:10:22<8:24:17,  8.03s/it]
 90%|█████████ | 34193/37962 [79:10:30<8:33:25,  8.17s/it]
 90%|█████████ | 34194/37962 [79:10:39<8:36:16,  8.22s/it]
 90%|█████████ | 34195/37962 [79:10:47<8:36:51,  8.23s/it]
 90%|█████████ | 34196/37962 [79:10:55<8:36:08,  8.22s/it]
 90%|█████████ | 34197/37962 [79:11:03<8:35:26,  8.21s/it]
 90%|█████████ | 34198/37962 [79:11:12<8:35:39,  8.22s/it]
 90%|█████████ | 34199/37962 [79:11:20<8:37:15,  8.25s/it]
 90%|█████████ | 34200/37962 [79:11:36<11:06:53, 10.64s/it]
                                                           
{'loss': 1.1649, 'grad_norm': 7.78312873840332, 'learning_rate': 5.106667318498837e-07, 'epoch': 0.9}

 90%|█████████ | 34200/37962 [79:11:36<11:06:53, 10.64s/it]
 90%|█████████ | 34201/37962 [79:11:44<10:18:27,  9.87s/it]
 90%|█████████ | 34202/37962 [79:11:52<9:49:45,  9.41s/it] 
 90%|█████████ | 34203/37962 [79:12:01<9:28:03,  9.07s/it]
 90%|█████████ | 34204/37962 [79:12:09<9:13:14,  8.83s/it]
 90%|█████████ | 34205/37962 [79:12:17<9:05:40,  8.71s/it]
 90%|█████████ | 34206/37962 [79:12:25<8:50:31,  8.47s/it]
 90%|█████████ | 34207/37962 [79:12:34<8:47:35,  8.43s/it]
 90%|█████████ | 34208/37962 [79:12:42<8:42:17,  8.35s/it]
 90%|█████████ | 34209/37962 [79:12:50<8:39:30,  8.31s/it]
 90%|█████████ | 34210/37962 [79:12:58<8:30:53,  8.17s/it]
                                                          
{'loss': 1.0707, 'grad_norm': 11.043210983276367, 'learning_rate': 5.079786634593175e-07, 'epoch': 0.9}

 90%|█████████ | 34210/37962 [79:12:58<8:30:53,  8.17s/it]
 90%|█████████ | 34211/37962 [79:13:06<8:36:05,  8.26s/it]
 90%|█████████ | 34212/37962 [79:13:15<8:35:03,  8.24s/it]
 90%|█████████ | 34213/37962 [79:13:23<8:32:42,  8.21s/it]
 90%|█████████ | 34214/37962 [79:13:31<8:29:23,  8.15s/it]
 90%|█████████ | 34215/37962 [79:13:39<8:23:07,  8.06s/it]
 90%|█████████ | 34216/37962 [79:13:47<8:29:23,  8.16s/it]
 90%|█████████ | 34217/37962 [79:13:55<8:33:13,  8.22s/it]
 90%|█████████ | 34218/37962 [79:14:04<8:33:41,  8.23s/it]
 90%|█████████ | 34219/37962 [79:14:12<8:32:44,  8.22s/it]
 90%|█████████ | 34220/37962 [79:14:20<8:34:02,  8.24s/it]
                                                          
{'loss': 1.0984, 'grad_norm': 8.47862434387207, 'learning_rate': 5.052975041466457e-07, 'epoch': 0.9}

 90%|█████████ | 34220/37962 [79:14:20<8:34:02,  8.24s/it]
 90%|█████████ | 34221/37962 [79:14:28<8:31:40,  8.21s/it]
 90%|█████████ | 34222/37962 [79:14:36<8:32:01,  8.21s/it]
 90%|█████████ | 34223/37962 [79:14:45<8:32:32,  8.22s/it]
 90%|█████████ | 34224/37962 [79:14:53<8:34:07,  8.25s/it]
 90%|█████████ | 34225/37962 [79:15:01<8:36:38,  8.29s/it]
 90%|█████████ | 34226/37962 [79:15:10<8:35:03,  8.27s/it]
 90%|█████████ | 34227/37962 [79:15:18<8:36:56,  8.30s/it]
 90%|█████████ | 34228/37962 [79:15:26<8:38:44,  8.34s/it]
 90%|█████████ | 34229/37962 [79:15:35<8:35:34,  8.29s/it]
 90%|█████████ | 34230/37962 [79:15:43<8:32:23,  8.24s/it]
                                                          
{'loss': 1.125, 'grad_norm': 10.428630828857422, 'learning_rate': 5.026232558634337e-07, 'epoch': 0.9}

 90%|█████████ | 34230/37962 [79:15:43<8:32:23,  8.24s/it]
 90%|█████████ | 34231/37962 [79:15:51<8:31:55,  8.23s/it]
 90%|█████████ | 34232/37962 [79:15:59<8:31:51,  8.23s/it]
 90%|█████████ | 34233/37962 [79:16:07<8:28:17,  8.18s/it]
 90%|█████████ | 34234/37962 [79:16:16<8:32:29,  8.25s/it]
 90%|█████████ | 34235/37962 [79:16:24<8:34:03,  8.28s/it]
 90%|█████████ | 34236/37962 [79:16:32<8:34:04,  8.28s/it]
 90%|█████████ | 34237/37962 [79:16:40<8:33:08,  8.27s/it]
 90%|█████████ | 34238/37962 [79:16:48<8:24:14,  8.12s/it]
 90%|█████████ | 34239/37962 [79:16:56<8:22:09,  8.09s/it]
 90%|█████████ | 34240/37962 [79:17:04<8:22:27,  8.10s/it]
                                                          
{'loss': 1.241, 'grad_norm': 11.397324562072754, 'learning_rate': 4.999559205562254e-07, 'epoch': 0.9}

 90%|█████████ | 34240/37962 [79:17:04<8:22:27,  8.10s/it]
 90%|█████████ | 34241/37962 [79:17:13<8:24:19,  8.13s/it]
 90%|█████████ | 34242/37962 [79:17:21<8:25:54,  8.16s/it]
 90%|█████████ | 34243/37962 [79:17:29<8:27:55,  8.19s/it]
 90%|█████████ | 34244/37962 [79:17:38<8:32:46,  8.27s/it]
 90%|█████████ | 34245/37962 [79:17:46<8:35:00,  8.31s/it]
 90%|█████████ | 34246/37962 [79:17:54<8:35:21,  8.32s/it]
 90%|█████████ | 34247/37962 [79:18:03<8:36:29,  8.34s/it]
 90%|█████████ | 34248/37962 [79:18:11<8:35:40,  8.33s/it]
 90%|█████████ | 34249/37962 [79:18:19<8:33:47,  8.30s/it]
 90%|█████████ | 34250/37962 [79:18:27<8:33:06,  8.29s/it]
                                                          
{'loss': 1.1933, 'grad_norm': 10.000570297241211, 'learning_rate': 4.972955001665258e-07, 'epoch': 0.9}

 90%|█████████ | 34250/37962 [79:18:27<8:33:06,  8.29s/it]
 90%|█████████ | 34251/37962 [79:18:36<8:31:35,  8.27s/it]
 90%|█████████ | 34252/37962 [79:18:44<8:31:19,  8.27s/it]
 90%|█████████ | 34253/37962 [79:18:52<8:28:16,  8.22s/it]
 90%|█████████ | 34254/37962 [79:19:00<8:28:33,  8.23s/it]
 90%|█████████ | 34255/37962 [79:19:09<8:33:48,  8.32s/it]
 90%|█████████ | 34256/37962 [79:19:17<8:26:10,  8.19s/it]
 90%|█████████ | 34257/37962 [79:19:25<8:29:39,  8.25s/it]
 90%|█████████ | 34258/37962 [79:19:33<8:30:39,  8.27s/it]
 90%|█████████ | 34259/37962 [79:19:42<8:31:48,  8.29s/it]
 90%|█████████ | 34260/37962 [79:19:50<8:29:46,  8.26s/it]
                                                          
{'loss': 1.1676, 'grad_norm': 7.003478050231934, 'learning_rate': 4.946419966308058e-07, 'epoch': 0.9}

 90%|█████████ | 34260/37962 [79:19:50<8:29:46,  8.26s/it]
 90%|█████████ | 34261/37962 [79:19:58<8:26:29,  8.21s/it]
 90%|█████████ | 34262/37962 [79:20:06<8:25:30,  8.20s/it]
 90%|█████████ | 34263/37962 [79:20:15<8:26:41,  8.22s/it]
 90%|█████████ | 34264/37962 [79:20:23<8:23:17,  8.17s/it]
 90%|█████████ | 34265/37962 [79:20:31<8:24:15,  8.18s/it]
 90%|█████████ | 34266/37962 [79:20:39<8:29:16,  8.27s/it]
 90%|█████████ | 34267/37962 [79:20:47<8:27:39,  8.24s/it]
 90%|█████████ | 34268/37962 [79:20:56<8:28:20,  8.26s/it]
 90%|█████████ | 34269/37962 [79:21:04<8:29:41,  8.28s/it]
 90%|█████████ | 34270/37962 [79:21:12<8:32:14,  8.32s/it]
                                                          
{'loss': 1.048, 'grad_norm': 12.588276863098145, 'learning_rate': 4.919954118805093e-07, 'epoch': 0.9}

 90%|█████████ | 34270/37962 [79:21:13<8:32:14,  8.32s/it]
 90%|█████████ | 34271/37962 [79:21:21<8:29:41,  8.29s/it]
 90%|█████████ | 34272/37962 [79:21:29<8:28:24,  8.27s/it]
 90%|█████████ | 34273/37962 [79:21:37<8:24:23,  8.20s/it]
 90%|█████████ | 34274/37962 [79:21:45<8:26:55,  8.25s/it]
 90%|█████████ | 34275/37962 [79:21:53<8:23:22,  8.19s/it]
 90%|█████████ | 34276/37962 [79:22:02<8:24:32,  8.21s/it]
 90%|█████████ | 34277/37962 [79:22:10<8:21:58,  8.17s/it]
 90%|█████████ | 34278/37962 [79:22:18<8:20:29,  8.15s/it]
 90%|█████████ | 34279/37962 [79:22:26<8:22:03,  8.18s/it]
 90%|█████████ | 34280/37962 [79:22:35<8:27:39,  8.27s/it]
                                                          
{'loss': 1.1383, 'grad_norm': 9.712860107421875, 'learning_rate': 4.893557478420375e-07, 'epoch': 0.9}

 90%|█████████ | 34280/37962 [79:22:35<8:27:39,  8.27s/it]
 90%|█████████ | 34281/37962 [79:22:43<8:31:46,  8.34s/it]
 90%|█████████ | 34282/37962 [79:22:51<8:26:06,  8.25s/it]
 90%|█████████ | 34283/37962 [79:22:59<8:26:01,  8.25s/it]
 90%|█████████ | 34284/37962 [79:23:08<8:25:23,  8.24s/it]
 90%|█████████ | 34285/37962 [79:23:16<8:24:43,  8.24s/it]
 90%|█████████ | 34286/37962 [79:23:24<8:23:33,  8.22s/it]
 90%|█████████ | 34287/37962 [79:23:32<8:19:15,  8.15s/it]
 90%|█████████ | 34288/37962 [79:23:40<8:21:22,  8.19s/it]
 90%|█████████ | 34289/37962 [79:23:48<8:20:20,  8.17s/it]
 90%|█████████ | 34290/37962 [79:23:57<8:20:38,  8.18s/it]
                                                          
{'loss': 1.1227, 'grad_norm': 13.447404861450195, 'learning_rate': 4.867230064367567e-07, 'epoch': 0.9}

 90%|█████████ | 34290/37962 [79:23:57<8:20:38,  8.18s/it]
 90%|█████████ | 34291/37962 [79:24:05<8:21:21,  8.19s/it]
 90%|█████████ | 34292/37962 [79:24:13<8:14:36,  8.09s/it]
 90%|█████████ | 34293/37962 [79:24:20<8:09:27,  8.00s/it]
 90%|█████████ | 34294/37962 [79:24:29<8:15:47,  8.11s/it]
 90%|█████████ | 34295/37962 [79:24:37<8:17:33,  8.14s/it]
 90%|█████████ | 34296/37962 [79:24:45<8:20:19,  8.19s/it]
 90%|█████████ | 34297/37962 [79:24:53<8:18:47,  8.17s/it]
 90%|█████████ | 34298/37962 [79:25:02<8:20:37,  8.20s/it]
 90%|█████████ | 34299/37962 [79:25:10<8:21:16,  8.21s/it]
 90%|█████████ | 34300/37962 [79:25:18<8:22:27,  8.23s/it]
                                                          
{'loss': 1.1254, 'grad_norm': 14.642350196838379, 'learning_rate': 4.840971895809921e-07, 'epoch': 0.9}

 90%|█████████ | 34300/37962 [79:25:18<8:22:27,  8.23s/it]
 90%|█████████ | 34301/37962 [79:25:26<8:22:30,  8.24s/it]
 90%|█████████ | 34302/37962 [79:25:35<8:26:59,  8.31s/it]
 90%|█████████ | 34303/37962 [79:25:43<8:25:07,  8.28s/it]
 90%|█████████ | 34304/37962 [79:25:52<8:25:57,  8.30s/it]
 90%|█████████ | 34305/37962 [79:26:00<8:25:33,  8.29s/it]
 90%|█████████ | 34306/37962 [79:26:08<8:27:00,  8.32s/it]
 90%|█████████ | 34307/37962 [79:26:16<8:25:27,  8.30s/it]
 90%|█████████ | 34308/37962 [79:26:25<8:25:23,  8.30s/it]
 90%|█████████ | 34309/37962 [79:26:33<8:22:23,  8.25s/it]
 90%|█████████ | 34310/37962 [79:26:41<8:23:51,  8.28s/it]
                                                          
{'loss': 1.2321, 'grad_norm': 11.322317123413086, 'learning_rate': 4.814782991860323e-07, 'epoch': 0.9}

 90%|█████████ | 34310/37962 [79:26:41<8:23:51,  8.28s/it]
 90%|█████████ | 34311/37962 [79:26:49<8:21:09,  8.24s/it]
 90%|█████████ | 34312/37962 [79:26:58<8:20:45,  8.23s/it]
 90%|█████████ | 34313/37962 [79:27:06<8:22:55,  8.27s/it]
 90%|█████████ | 34314/37962 [79:27:14<8:21:39,  8.25s/it]
 90%|█████████ | 34315/37962 [79:27:22<8:22:42,  8.27s/it]
 90%|█████████ | 34316/37962 [79:27:31<8:22:34,  8.27s/it]
 90%|█████████ | 34317/37962 [79:27:39<8:25:42,  8.32s/it]
 90%|█████████ | 34318/37962 [79:27:48<8:28:14,  8.37s/it]
 90%|█████████ | 34319/37962 [79:27:56<8:24:39,  8.31s/it]
 90%|█████████ | 34320/37962 [79:28:04<8:24:06,  8.30s/it]
                                                          
{'loss': 1.1621, 'grad_norm': 13.353717803955078, 'learning_rate': 4.788663371581204e-07, 'epoch': 0.9}

 90%|█████████ | 34320/37962 [79:28:04<8:24:06,  8.30s/it]
 90%|█████████ | 34321/37962 [79:28:12<8:21:31,  8.26s/it]
 90%|█████████ | 34322/37962 [79:28:20<8:20:00,  8.24s/it]
 90%|█████████ | 34323/37962 [79:28:29<8:20:39,  8.25s/it]
 90%|█████████ | 34324/37962 [79:28:37<8:19:27,  8.24s/it]
 90%|█████████ | 34325/37962 [79:28:45<8:21:45,  8.28s/it]
 90%|█████████ | 34326/37962 [79:28:54<8:23:21,  8.31s/it]
 90%|█████████ | 34327/37962 [79:29:02<8:18:36,  8.23s/it]
 90%|█████████ | 34328/37962 [79:29:10<8:18:27,  8.23s/it]
 90%|█████████ | 34329/37962 [79:29:18<8:19:34,  8.25s/it]
 90%|█████████ | 34330/37962 [79:29:27<8:24:24,  8.33s/it]
                                                          
{'loss': 1.221, 'grad_norm': 13.853574752807617, 'learning_rate': 4.7626130539846105e-07, 'epoch': 0.9}

 90%|█████████ | 34330/37962 [79:29:27<8:24:24,  8.33s/it]
 90%|█████████ | 34331/37962 [79:29:35<8:27:11,  8.38s/it]
 90%|█████████ | 34332/37962 [79:29:44<8:25:23,  8.35s/it]
 90%|█████████ | 34333/37962 [79:29:52<8:29:15,  8.42s/it]
 90%|█████████ | 34334/37962 [79:30:00<8:26:27,  8.38s/it]
 90%|█████████ | 34335/37962 [79:30:08<8:19:38,  8.27s/it]
 90%|█████████ | 34336/37962 [79:30:17<8:17:41,  8.24s/it]
 90%|█████████ | 34337/37962 [79:30:24<8:09:51,  8.11s/it]
 90%|█████████ | 34338/37962 [79:30:33<8:14:08,  8.18s/it]
 90%|█████████ | 34339/37962 [79:30:41<8:13:48,  8.18s/it]
 90%|█████████ | 34340/37962 [79:30:49<8:14:34,  8.19s/it]
                                                          
{'loss': 1.1177, 'grad_norm': 17.469995498657227, 'learning_rate': 4.736632058032098e-07, 'epoch': 0.9}

 90%|█████████ | 34340/37962 [79:30:49<8:14:34,  8.19s/it]
 90%|█████████ | 34341/37962 [79:30:57<8:15:04,  8.20s/it]
 90%|█████████ | 34342/37962 [79:31:06<8:17:10,  8.24s/it]
 90%|█████████ | 34343/37962 [79:31:14<8:18:58,  8.27s/it]
 90%|█████████ | 34344/37962 [79:31:22<8:19:22,  8.28s/it]
 90%|█████████ | 34345/37962 [79:31:31<8:18:26,  8.27s/it]
 90%|█████████ | 34346/37962 [79:31:39<8:21:43,  8.32s/it]
 90%|█████████ | 34347/37962 [79:31:47<8:19:48,  8.30s/it]
 90%|█████████ | 34348/37962 [79:31:55<8:16:09,  8.24s/it]
 90%|█████████ | 34349/37962 [79:32:04<8:13:36,  8.20s/it]
 90%|█████████ | 34350/37962 [79:32:12<8:15:20,  8.23s/it]
                                                          
{'loss': 1.1688, 'grad_norm': 8.299126625061035, 'learning_rate': 4.710720402634772e-07, 'epoch': 0.9}

 90%|█████████ | 34350/37962 [79:32:12<8:15:20,  8.23s/it]
 90%|█████████ | 34351/37962 [79:32:20<8:18:09,  8.28s/it]
 90%|█████████ | 34352/37962 [79:32:29<8:19:14,  8.30s/it]
 90%|█████████ | 34353/37962 [79:32:37<8:18:05,  8.28s/it]
 90%|█████████ | 34354/37962 [79:32:45<8:16:30,  8.26s/it]
 90%|█████████ | 34355/37962 [79:32:53<8:17:32,  8.28s/it]
 91%|█████████ | 34356/37962 [79:33:02<8:17:15,  8.27s/it]
 91%|█████████ | 34357/37962 [79:33:10<8:16:34,  8.26s/it]
 91%|█████████ | 34358/37962 [79:33:18<8:15:38,  8.25s/it]
 91%|█████████ | 34359/37962 [79:33:26<8:15:40,  8.25s/it]
 91%|█████████ | 34360/37962 [79:33:35<8:15:19,  8.25s/it]
                                                          
{'loss': 1.1105, 'grad_norm': 8.922969818115234, 'learning_rate': 4.684878106653301e-07, 'epoch': 0.91}

 91%|█████████ | 34360/37962 [79:33:35<8:15:19,  8.25s/it]
 91%|█████████ | 34361/37962 [79:33:42<8:07:26,  8.12s/it]
 91%|█████████ | 34362/37962 [79:33:51<8:10:24,  8.17s/it]
 91%|█████████ | 34363/37962 [79:33:59<8:10:33,  8.18s/it]
 91%|█████████ | 34364/37962 [79:34:07<8:06:59,  8.12s/it]
 91%|█████████ | 34365/37962 [79:34:15<8:10:32,  8.18s/it]
 91%|█████████ | 34366/37962 [79:34:23<8:12:18,  8.21s/it]
 91%|█████████ | 34367/37962 [79:34:32<8:12:45,  8.22s/it]
 91%|█████████ | 34368/37962 [79:34:40<8:09:20,  8.17s/it]
 91%|█████████ | 34369/37962 [79:34:48<8:11:56,  8.21s/it]
 91%|█████████ | 34370/37962 [79:34:56<8:10:17,  8.19s/it]
                                                          
{'loss': 1.1551, 'grad_norm': 7.330361843109131, 'learning_rate': 4.659105188897817e-07, 'epoch': 0.91}

 91%|█████████ | 34370/37962 [79:34:56<8:10:17,  8.19s/it]
 91%|█████████ | 34371/37962 [79:35:05<8:13:08,  8.24s/it]
 91%|█████████ | 34372/37962 [79:35:13<8:17:05,  8.31s/it]
 91%|█████████ | 34373/37962 [79:35:21<8:19:48,  8.36s/it]
 91%|█████████ | 34374/37962 [79:35:30<8:24:02,  8.43s/it]
 91%|█████████ | 34375/37962 [79:35:38<8:20:48,  8.38s/it]
 91%|█████████ | 34376/37962 [79:35:47<8:25:03,  8.45s/it]
 91%|█████████ | 34377/37962 [79:35:55<8:22:30,  8.41s/it]
 91%|█████████ | 34378/37962 [79:36:04<8:23:43,  8.43s/it]
 91%|█████████ | 34379/37962 [79:36:12<8:20:52,  8.39s/it]
 91%|█████████ | 34380/37962 [79:36:20<8:18:22,  8.35s/it]
                                                          
{'loss': 1.2517, 'grad_norm': 11.607035636901855, 'learning_rate': 4.6334016681280126e-07, 'epoch': 0.91}

 91%|█████████ | 34380/37962 [79:36:20<8:18:22,  8.35s/it]
 91%|█████████ | 34381/37962 [79:36:29<8:19:21,  8.37s/it]
 91%|█████████ | 34382/37962 [79:36:37<8:16:43,  8.32s/it]
 91%|█████████ | 34383/37962 [79:36:45<8:15:41,  8.31s/it]
 91%|█████████ | 34384/37962 [79:36:54<8:15:16,  8.31s/it]
 91%|█████████ | 34385/37962 [79:37:02<8:13:45,  8.28s/it]
 91%|█████████ | 34386/37962 [79:37:10<8:15:50,  8.32s/it]
 91%|█████████ | 34387/37962 [79:37:18<8:14:00,  8.29s/it]
 91%|█████████ | 34388/37962 [79:37:27<8:15:19,  8.32s/it]
 91%|█████████ | 34389/37962 [79:37:35<8:15:40,  8.32s/it]
 91%|█████████ | 34390/37962 [79:37:43<8:07:28,  8.19s/it]
                                                          
{'loss': 1.1116, 'grad_norm': 8.082463264465332, 'learning_rate': 4.607767563053023e-07, 'epoch': 0.91}

 91%|█████████ | 34390/37962 [79:37:43<8:07:28,  8.19s/it]
 91%|█████████ | 34391/37962 [79:37:51<8:09:23,  8.22s/it]
 91%|█████████ | 34392/37962 [79:38:00<8:12:32,  8.28s/it]
 91%|█████████ | 34393/37962 [79:38:08<8:08:32,  8.21s/it]
 91%|█████████ | 34394/37962 [79:38:16<8:09:12,  8.23s/it]
 91%|█████████ | 34395/37962 [79:38:24<8:09:12,  8.23s/it]
 91%|█████████ | 34396/37962 [79:38:33<8:10:06,  8.25s/it]
 91%|█████████ | 34397/37962 [79:38:41<8:11:51,  8.28s/it]
 91%|█████████ | 34398/37962 [79:38:49<8:11:17,  8.27s/it]
 91%|█████████ | 34399/37962 [79:38:57<8:10:05,  8.25s/it]
 91%|█████████ | 34400/37962 [79:39:05<8:04:50,  8.17s/it]
                                                          
{'loss': 1.2351, 'grad_norm': 8.443191528320312, 'learning_rate': 4.5822028923314664e-07, 'epoch': 0.91}

 91%|█████████ | 34400/37962 [79:39:05<8:04:50,  8.17s/it]
 91%|█████████ | 34401/37962 [79:39:14<8:07:22,  8.21s/it]
 91%|█████████ | 34402/37962 [79:39:22<8:02:42,  8.14s/it]
 91%|█████████ | 34403/37962 [79:39:30<8:04:27,  8.17s/it]
 91%|█████████ | 34404/37962 [79:39:38<8:04:33,  8.17s/it]
 91%|█████████ | 34405/37962 [79:39:46<8:07:17,  8.22s/it]
 91%|█████████ | 34406/37962 [79:39:55<8:07:27,  8.22s/it]
 91%|█████████ | 34407/37962 [79:40:03<8:08:24,  8.24s/it]
 91%|█████████ | 34408/37962 [79:40:11<8:08:35,  8.25s/it]
 91%|█████████ | 34409/37962 [79:40:19<8:08:09,  8.24s/it]
 91%|█████████ | 34410/37962 [79:40:28<8:09:34,  8.27s/it]
                                                          
{'loss': 1.3143, 'grad_norm': 8.197968482971191, 'learning_rate': 4.5567076745714144e-07, 'epoch': 0.91}

 91%|█████████ | 34410/37962 [79:40:28<8:09:34,  8.27s/it]
 91%|█████████ | 34411/37962 [79:40:36<8:07:16,  8.23s/it]
 91%|█████████ | 34412/37962 [79:40:44<8:02:45,  8.16s/it]
 91%|█████████ | 34413/37962 [79:40:52<7:55:40,  8.04s/it]
 91%|█████████ | 34414/37962 [79:41:00<8:02:27,  8.16s/it]
 91%|█████████ | 34415/37962 [79:41:08<8:04:01,  8.19s/it]
 91%|█████████ | 34416/37962 [79:41:17<8:06:37,  8.23s/it]
 91%|█████████ | 34417/37962 [79:41:25<8:09:11,  8.28s/it]
 91%|█████████ | 34418/37962 [79:41:33<8:05:08,  8.21s/it]
 91%|█████████ | 34419/37962 [79:41:41<8:05:29,  8.22s/it]
 91%|█████████ | 34420/37962 [79:41:49<8:01:25,  8.16s/it]
                                                          
{'loss': 1.2197, 'grad_norm': 8.638983726501465, 'learning_rate': 4.5312819283304223e-07, 'epoch': 0.91}

 91%|█████████ | 34420/37962 [79:41:49<8:01:25,  8.16s/it]
 91%|█████████ | 34421/37962 [79:41:58<8:03:21,  8.19s/it]
 91%|█████████ | 34422/37962 [79:42:06<8:09:16,  8.29s/it]
 91%|█████████ | 34423/37962 [79:42:14<8:07:27,  8.26s/it]
 91%|█████████ | 34424/37962 [79:42:23<8:09:22,  8.30s/it]
 91%|█████████ | 34425/37962 [79:42:31<8:06:58,  8.26s/it]
 91%|█████████ | 34426/37962 [79:42:39<8:09:44,  8.31s/it]
 91%|█████████ | 34427/37962 [79:42:48<8:10:19,  8.32s/it]
 91%|█████████ | 34428/37962 [79:42:56<8:10:29,  8.33s/it]
 91%|█████████ | 34429/37962 [79:43:04<8:09:22,  8.31s/it]
 91%|█████████ | 34430/37962 [79:43:12<8:02:26,  8.20s/it]
                                                          
{'loss': 1.1241, 'grad_norm': 10.592315673828125, 'learning_rate': 4.505925672115419e-07, 'epoch': 0.91}

 91%|█████████ | 34430/37962 [79:43:12<8:02:26,  8.20s/it]
 91%|█████████ | 34431/37962 [79:43:20<8:02:00,  8.19s/it]
 91%|█████████ | 34432/37962 [79:43:28<7:57:12,  8.11s/it]
 91%|█████████ | 34433/37962 [79:43:37<8:02:56,  8.21s/it]
 91%|█████████ | 34434/37962 [79:43:45<8:04:21,  8.24s/it]
 91%|█████████ | 34435/37962 [79:43:53<8:01:16,  8.19s/it]
 91%|█████████ | 34436/37962 [79:44:01<8:02:05,  8.20s/it]
 91%|█████████ | 34437/37962 [79:44:09<8:00:15,  8.17s/it]
 91%|█████████ | 34438/37962 [79:44:18<8:01:27,  8.20s/it]
 91%|█████████ | 34439/37962 [79:44:26<8:06:45,  8.29s/it]
 91%|█████████ | 34440/37962 [79:44:35<8:08:29,  8.32s/it]
                                                          
{'loss': 1.1288, 'grad_norm': 7.927863121032715, 'learning_rate': 4.480638924382796e-07, 'epoch': 0.91}

 91%|█████████ | 34440/37962 [79:44:35<8:08:29,  8.32s/it]
 91%|█████████ | 34441/37962 [79:44:43<8:06:17,  8.29s/it]
 91%|█████████ | 34442/37962 [79:44:51<8:03:30,  8.24s/it]
 91%|█████████ | 34443/37962 [79:44:59<8:04:18,  8.26s/it]
 91%|█████████ | 34444/37962 [79:45:07<7:57:47,  8.15s/it]
 91%|█████████ | 34445/37962 [79:45:15<8:00:26,  8.20s/it]
 91%|█████████ | 34446/37962 [79:45:24<8:01:30,  8.22s/it]
 91%|█████████ | 34447/37962 [79:45:32<8:01:24,  8.22s/it]
 91%|█████████ | 34448/37962 [79:45:41<8:08:34,  8.34s/it]
 91%|█████████ | 34449/37962 [79:45:49<8:03:01,  8.25s/it]
 91%|█████████ | 34450/37962 [79:45:57<8:00:34,  8.21s/it]
                                                          
{'loss': 1.1285, 'grad_norm': 8.67530345916748, 'learning_rate': 4.455421703538354e-07, 'epoch': 0.91}

 91%|█████████ | 34450/37962 [79:45:57<8:00:34,  8.21s/it]
 91%|█████████ | 34451/37962 [79:46:05<8:05:10,  8.29s/it]
 91%|█████████ | 34452/37962 [79:46:13<8:02:16,  8.24s/it]
 91%|█████████ | 34453/37962 [79:46:22<8:03:42,  8.27s/it]
 91%|█████████ | 34454/37962 [79:46:30<8:03:22,  8.27s/it]
 91%|█████████ | 34455/37962 [79:46:38<8:04:20,  8.29s/it]
 91%|█████████ | 34456/37962 [79:46:46<8:01:21,  8.24s/it]
 91%|█████████ | 34457/37962 [79:46:55<8:00:28,  8.22s/it]
 91%|█████████ | 34458/37962 [79:47:03<7:57:04,  8.17s/it]
 91%|█████████ | 34459/37962 [79:47:11<8:01:12,  8.24s/it]
 91%|█████████ | 34460/37962 [79:47:19<8:04:14,  8.30s/it]
                                                          
{'loss': 1.2415, 'grad_norm': 11.051576614379883, 'learning_rate': 4.4302740279372426e-07, 'epoch': 0.91}

 91%|█████████ | 34460/37962 [79:47:19<8:04:14,  8.30s/it]
 91%|█████████ | 34461/37962 [79:47:28<8:05:00,  8.31s/it]
 91%|█████████ | 34462/37962 [79:47:36<8:06:09,  8.33s/it]
 91%|█████████ | 34463/37962 [79:47:44<8:01:59,  8.27s/it]
 91%|█████████ | 34464/37962 [79:47:52<8:01:22,  8.26s/it]
 91%|█████████ | 34465/37962 [79:48:01<8:04:19,  8.31s/it]
 91%|█████████ | 34466/37962 [79:48:09<7:59:24,  8.23s/it]
 91%|█████████ | 34467/37962 [79:48:17<7:58:38,  8.22s/it]
 91%|█████████ | 34468/37962 [79:48:25<7:57:29,  8.20s/it]
 91%|█████████ | 34469/37962 [79:48:34<8:02:23,  8.29s/it]
 91%|█████████ | 34470/37962 [79:48:42<7:56:55,  8.19s/it]
                                                          
{'loss': 1.2447, 'grad_norm': 7.310313701629639, 'learning_rate': 4.4051959158840417e-07, 'epoch': 0.91}

 91%|█████████ | 34470/37962 [79:48:42<7:56:55,  8.19s/it]
 91%|█████████ | 34471/37962 [79:48:50<7:59:11,  8.24s/it]
 91%|█████████ | 34472/37962 [79:49:07<10:25:55, 10.76s/it]
 91%|█████████ | 34473/37962 [79:49:15<9:33:25,  9.86s/it] 
 91%|█████████ | 34474/37962 [79:49:23<9:05:07,  9.38s/it]
 91%|█████████ | 34475/37962 [79:49:31<8:42:32,  8.99s/it]
 91%|█████████ | 34476/37962 [79:49:39<8:28:23,  8.75s/it]
 91%|█████████ | 34477/37962 [79:49:47<8:19:20,  8.60s/it]
 91%|█████████ | 34478/37962 [79:49:55<8:07:47,  8.40s/it]
 91%|█████████ | 34479/37962 [79:50:04<8:07:11,  8.39s/it]
 91%|█████████ | 34480/37962 [79:50:12<8:03:50,  8.34s/it]
                                                          
{'loss': 1.1648, 'grad_norm': 13.749628067016602, 'learning_rate': 4.3801873856326725e-07, 'epoch': 0.91}

 91%|█████████ | 34480/37962 [79:50:12<8:03:50,  8.34s/it]
 91%|█████████ | 34481/37962 [79:50:20<8:03:28,  8.33s/it]
 91%|█████████ | 34482/37962 [79:50:28<8:01:10,  8.30s/it]
 91%|█████████ | 34483/37962 [79:50:37<8:03:41,  8.34s/it]
 91%|█████████ | 34484/37962 [79:50:45<8:01:54,  8.31s/it]
 91%|█████████ | 34485/37962 [79:50:53<8:00:23,  8.29s/it]
 91%|█████████ | 34486/37962 [79:51:02<8:00:28,  8.29s/it]
 91%|█████████ | 34487/37962 [79:51:10<8:02:04,  8.32s/it]
 91%|█████████ | 34488/37962 [79:51:18<8:02:34,  8.33s/it]
 91%|█████████ | 34489/37962 [79:51:26<7:59:02,  8.28s/it]
 91%|█████████ | 34490/37962 [79:51:35<8:02:11,  8.33s/it]
                                                          
{'loss': 1.1078, 'grad_norm': 11.915290832519531, 'learning_rate': 4.355248455386385e-07, 'epoch': 0.91}

 91%|█████████ | 34490/37962 [79:51:35<8:02:11,  8.33s/it]
 91%|█████████ | 34491/37962 [79:51:43<8:00:20,  8.30s/it]
 91%|█████████ | 34492/37962 [79:51:51<7:57:02,  8.25s/it]
 91%|█████████ | 34493/37962 [79:52:00<7:57:06,  8.25s/it]
 91%|█████████ | 34494/37962 [79:52:08<8:01:38,  8.33s/it]
 91%|█████████ | 34495/37962 [79:52:17<8:04:11,  8.38s/it]
 91%|█████████ | 34496/37962 [79:52:25<8:03:49,  8.38s/it]
 91%|█████████ | 34497/37962 [79:52:33<8:02:11,  8.35s/it]
 91%|█████████ | 34498/37962 [79:52:41<8:00:04,  8.32s/it]
 91%|█████████ | 34499/37962 [79:52:50<8:01:45,  8.35s/it]
 91%|█████████ | 34500/37962 [79:52:58<7:59:29,  8.31s/it]
                                                          
{'loss': 1.2672, 'grad_norm': 9.331755638122559, 'learning_rate': 4.330379143297836e-07, 'epoch': 0.91}

 91%|█████████ | 34500/37962 [79:52:58<7:59:29,  8.31s/it]
 91%|█████████ | 34501/37962 [79:53:06<7:57:36,  8.28s/it]
 91%|█████████ | 34502/37962 [79:53:15<7:57:55,  8.29s/it]
 91%|█████████ | 34503/37962 [79:53:23<7:59:03,  8.31s/it]
 91%|█████████ | 34504/37962 [79:53:31<7:57:18,  8.28s/it]
 91%|█████████ | 34505/37962 [79:53:40<7:57:57,  8.30s/it]
 91%|█████████ | 34506/37962 [79:53:48<7:56:33,  8.27s/it]
 91%|█████████ | 34507/37962 [79:53:56<7:55:50,  8.26s/it]
 91%|█████████ | 34508/37962 [79:54:05<8:01:28,  8.36s/it]
 91%|█████████ | 34509/37962 [79:54:13<7:58:38,  8.32s/it]
 91%|█████████ | 34510/37962 [79:54:21<7:55:02,  8.26s/it]
                                                          
{'loss': 1.1114, 'grad_norm': 11.98525333404541, 'learning_rate': 4.3055794674689457e-07, 'epoch': 0.91}

 91%|█████████ | 34510/37962 [79:54:21<7:55:02,  8.26s/it]
 91%|█████████ | 34511/37962 [79:54:29<7:49:27,  8.16s/it]
 91%|█████████ | 34512/37962 [79:54:37<7:47:48,  8.14s/it]
 91%|█████████ | 34513/37962 [79:54:45<7:51:11,  8.20s/it]
 91%|█████████ | 34514/37962 [79:54:54<7:53:23,  8.24s/it]
 91%|█████████ | 34515/37962 [79:55:02<7:55:51,  8.28s/it]
 91%|█████████ | 34516/37962 [79:55:10<7:55:12,  8.27s/it]
 91%|█████████ | 34517/37962 [79:55:18<7:54:08,  8.26s/it]
 91%|█████████ | 34518/37962 [79:55:27<7:54:17,  8.26s/it]
 91%|█████████ | 34519/37962 [79:55:35<7:55:44,  8.29s/it]
 91%|█████████ | 34520/37962 [79:55:43<7:55:16,  8.28s/it]
                                                          
{'loss': 1.1328, 'grad_norm': 11.972107887268066, 'learning_rate': 4.280849445950952e-07, 'epoch': 0.91}

 91%|█████████ | 34520/37962 [79:55:43<7:55:16,  8.28s/it]
 91%|█████████ | 34521/37962 [79:55:51<7:50:39,  8.21s/it]
 91%|█████████ | 34522/37962 [79:55:59<7:48:10,  8.17s/it]
 91%|█████████ | 34523/37962 [79:56:08<7:51:26,  8.23s/it]
 91%|█████████ | 34524/37962 [79:56:16<7:56:10,  8.31s/it]
 91%|█████████ | 34525/37962 [79:56:25<7:55:53,  8.31s/it]
 91%|█████████ | 34526/37962 [79:56:33<7:52:27,  8.25s/it]
 91%|█████████ | 34527/37962 [79:56:41<7:50:39,  8.22s/it]
 91%|█████████ | 34528/37962 [79:56:49<7:52:52,  8.26s/it]
 91%|█████████ | 34529/37962 [79:56:57<7:46:44,  8.16s/it]
 91%|█████████ | 34530/37962 [79:57:06<7:52:31,  8.26s/it]
                                                          
{'loss': 1.102, 'grad_norm': 9.174764633178711, 'learning_rate': 4.256189096744445e-07, 'epoch': 0.91}

 91%|█████████ | 34530/37962 [79:57:06<7:52:31,  8.26s/it]
 91%|█████████ | 34531/37962 [79:57:14<7:51:04,  8.24s/it]
 91%|█████████ | 34532/37962 [79:57:22<7:53:12,  8.28s/it]
 91%|█████████ | 34533/37962 [79:57:30<7:52:25,  8.27s/it]
 91%|█████████ | 34534/37962 [79:57:39<7:52:45,  8.27s/it]
 91%|█████████ | 34535/37962 [79:57:47<7:51:22,  8.25s/it]
 91%|█████████ | 34536/37962 [79:57:55<7:48:47,  8.21s/it]
 91%|█████████ | 34537/37962 [79:58:04<7:54:26,  8.31s/it]
 91%|█████████ | 34538/37962 [79:58:12<7:49:59,  8.24s/it]
 91%|█████████ | 34539/37962 [79:58:20<7:51:02,  8.26s/it]
 91%|█████████ | 34540/37962 [79:58:28<7:43:20,  8.12s/it]
                                                          
{'loss': 1.0238, 'grad_norm': 12.22992992401123, 'learning_rate': 4.2315984377992226e-07, 'epoch': 0.91}

 91%|█████████ | 34540/37962 [79:58:28<7:43:20,  8.12s/it]
 91%|█████████ | 34541/37962 [79:58:36<7:47:19,  8.20s/it]
 91%|█████████ | 34542/37962 [79:58:44<7:49:39,  8.24s/it]
 91%|█████████ | 34543/37962 [79:58:53<7:49:17,  8.24s/it]
 91%|█████████ | 34544/37962 [79:59:01<7:52:32,  8.30s/it]
 91%|█████████ | 34545/37962 [79:59:10<7:53:49,  8.32s/it]
 91%|█████████ | 34546/37962 [79:59:18<7:52:12,  8.29s/it]
 91%|█████████ | 34547/37962 [79:59:26<7:50:19,  8.26s/it]
 91%|█████████ | 34548/37962 [79:59:34<7:50:10,  8.26s/it]
 91%|█████████ | 34549/37962 [79:59:43<7:50:50,  8.28s/it]
 91%|█████████ | 34550/37962 [79:59:51<7:49:47,  8.26s/it]
                                                          
{'loss': 1.1902, 'grad_norm': 8.637992858886719, 'learning_rate': 4.207077487014444e-07, 'epoch': 0.91}

 91%|█████████ | 34550/37962 [79:59:51<7:49:47,  8.26s/it]
 91%|█████████ | 34551/37962 [79:59:59<7:51:54,  8.30s/it]
 91%|█████████ | 34552/37962 [80:00:07<7:46:46,  8.21s/it]
 91%|█████████ | 34553/37962 [80:00:16<7:50:18,  8.28s/it]
 91%|█████████ | 34554/37962 [80:00:24<7:46:28,  8.21s/it]
 91%|█████████ | 34555/37962 [80:00:32<7:47:12,  8.23s/it]
 91%|█████████ | 34556/37962 [80:00:40<7:50:54,  8.30s/it]
 91%|█████████ | 34557/37962 [80:00:49<7:51:39,  8.31s/it]
 91%|█████████ | 34558/37962 [80:00:57<7:44:24,  8.19s/it]
 91%|█████████ | 34559/37962 [80:01:05<7:47:09,  8.24s/it]
 91%|█████████ | 34560/37962 [80:01:13<7:48:34,  8.26s/it]
                                                          
{'loss': 1.1797, 'grad_norm': 10.551188468933105, 'learning_rate': 4.182626262238465e-07, 'epoch': 0.91}

 91%|█████████ | 34560/37962 [80:01:13<7:48:34,  8.26s/it]
 91%|█████████ | 34561/37962 [80:01:21<7:46:53,  8.24s/it]
 91%|█████████ | 34562/37962 [80:01:30<7:48:01,  8.26s/it]
 91%|█████████ | 34563/37962 [80:01:38<7:49:07,  8.28s/it]
 91%|█████████ | 34564/37962 [80:01:46<7:45:49,  8.23s/it]
 91%|█████████ | 34565/37962 [80:01:54<7:40:58,  8.14s/it]
 91%|█████████ | 34566/37962 [80:02:02<7:43:46,  8.19s/it]
 91%|█████████ | 34567/37962 [80:02:11<7:44:15,  8.20s/it]
 91%|█████████ | 34568/37962 [80:02:19<7:39:44,  8.13s/it]
 91%|█████████ | 34569/37962 [80:02:27<7:38:47,  8.11s/it]
 91%|█████████ | 34570/37962 [80:02:35<7:45:22,  8.23s/it]
                                                          
{'loss': 1.1283, 'grad_norm': 10.139881134033203, 'learning_rate': 4.158244781268894e-07, 'epoch': 0.91}

 91%|█████████ | 34570/37962 [80:02:35<7:45:22,  8.23s/it]
 91%|█████████ | 34571/37962 [80:02:43<7:41:01,  8.16s/it]
 91%|█████████ | 34572/37962 [80:02:51<7:39:57,  8.14s/it]
 91%|█████████ | 34573/37962 [80:03:00<7:42:08,  8.18s/it]
 91%|█████████ | 34574/37962 [80:03:08<7:44:50,  8.23s/it]
 91%|█████████ | 34575/37962 [80:03:16<7:44:05,  8.22s/it]
 91%|█████████ | 34576/37962 [80:03:24<7:43:11,  8.21s/it]
 91%|█████████ | 34577/37962 [80:03:33<7:43:17,  8.21s/it]
 91%|█████████ | 34578/37962 [80:03:41<7:45:41,  8.26s/it]
 91%|█████████ | 34579/37962 [80:03:49<7:46:54,  8.28s/it]
 91%|█████████ | 34580/37962 [80:03:57<7:41:34,  8.19s/it]
                                                          
{'loss': 1.3071, 'grad_norm': 9.849143028259277, 'learning_rate': 4.133933061852602e-07, 'epoch': 0.91}

 91%|█████████ | 34580/37962 [80:03:57<7:41:34,  8.19s/it]
 91%|█████████ | 34581/37962 [80:04:05<7:36:13,  8.10s/it]
 91%|█████████ | 34582/37962 [80:04:13<7:33:03,  8.04s/it]
 91%|█████████ | 34583/37962 [80:04:21<7:38:15,  8.14s/it]
 91%|█████████ | 34584/37962 [80:04:30<7:39:04,  8.15s/it]
 91%|█████████ | 34585/37962 [80:04:38<7:43:04,  8.23s/it]
 91%|█████████ | 34586/37962 [80:04:46<7:46:49,  8.30s/it]
 91%|█████████ | 34587/37962 [80:04:55<7:49:03,  8.34s/it]
 91%|█████████ | 34588/37962 [80:05:03<7:48:31,  8.33s/it]
 91%|█████████ | 34589/37962 [80:05:11<7:44:52,  8.27s/it]
 91%|█████████ | 34590/37962 [80:05:20<7:44:57,  8.27s/it]
                                                          
{'loss': 1.099, 'grad_norm': 10.614712715148926, 'learning_rate': 4.109691121685677e-07, 'epoch': 0.91}

 91%|█████████ | 34590/37962 [80:05:20<7:44:57,  8.27s/it]
 91%|█████████ | 34591/37962 [80:05:28<7:45:49,  8.29s/it]
 91%|█████████ | 34592/37962 [80:05:36<7:45:28,  8.29s/it]
 91%|█████████ | 34593/37962 [80:05:44<7:45:03,  8.28s/it]
 91%|█████████ | 34594/37962 [80:05:53<7:47:58,  8.34s/it]
 91%|█████████ | 34595/37962 [80:06:01<7:49:09,  8.36s/it]
 91%|█████████ | 34596/37962 [80:06:09<7:43:51,  8.27s/it]
 91%|█████████ | 34597/37962 [80:06:17<7:39:58,  8.20s/it]
 91%|█████████ | 34598/37962 [80:06:26<7:39:06,  8.19s/it]
 91%|█████████ | 34599/37962 [80:06:34<7:39:19,  8.19s/it]
 91%|█████████ | 34600/37962 [80:06:42<7:41:37,  8.24s/it]
                                                          
{'loss': 1.2214, 'grad_norm': 9.403322219848633, 'learning_rate': 4.085518978413394e-07, 'epoch': 0.91}

 91%|█████████ | 34600/37962 [80:06:42<7:41:37,  8.24s/it]
 91%|█████████ | 34601/37962 [80:06:50<7:40:35,  8.22s/it]
 91%|█████████ | 34602/37962 [80:06:59<7:39:52,  8.21s/it]
 91%|█████████ | 34603/37962 [80:07:07<7:40:08,  8.22s/it]
 91%|█████████ | 34604/37962 [80:07:15<7:39:07,  8.20s/it]
 91%|█████████ | 34605/37962 [80:07:23<7:36:08,  8.15s/it]
 91%|█████████ | 34606/37962 [80:07:31<7:39:07,  8.21s/it]
 91%|█████████ | 34607/37962 [80:07:39<7:36:43,  8.17s/it]
 91%|█████████ | 34608/37962 [80:07:48<7:39:49,  8.23s/it]
 91%|█████████ | 34609/37962 [80:07:56<7:39:54,  8.23s/it]
 91%|█████████ | 34610/37962 [80:08:04<7:41:16,  8.26s/it]
                                                          
{'loss': 1.2258, 'grad_norm': 12.368501663208008, 'learning_rate': 4.0614166496302454e-07, 'epoch': 0.91}

 91%|█████████ | 34610/37962 [80:08:04<7:41:16,  8.26s/it]
 91%|█████████ | 34611/37962 [80:08:13<7:42:45,  8.29s/it]
 91%|█████████ | 34612/37962 [80:08:21<7:45:39,  8.34s/it]
 91%|█████████ | 34613/37962 [80:08:29<7:42:29,  8.29s/it]
 91%|█████████ | 34614/37962 [80:08:38<7:44:44,  8.33s/it]
 91%|█████████ | 34615/37962 [80:08:46<7:44:03,  8.32s/it]
 91%|█████████ | 34616/37962 [80:08:54<7:42:12,  8.29s/it]
 91%|█████████ | 34617/37962 [80:09:02<7:38:46,  8.23s/it]
 91%|█████████ | 34618/37962 [80:09:11<7:38:43,  8.23s/it]
 91%|█████████ | 34619/37962 [80:09:19<7:38:59,  8.24s/it]
 91%|█████████ | 34620/37962 [80:09:27<7:37:24,  8.21s/it]
                                                          
{'loss': 1.2282, 'grad_norm': 8.237711906433105, 'learning_rate': 4.0373841528799084e-07, 'epoch': 0.91}

 91%|█████████ | 34620/37962 [80:09:27<7:37:24,  8.21s/it]
 91%|█████████ | 34621/37962 [80:09:35<7:39:26,  8.25s/it]
 91%|█████████ | 34622/37962 [80:09:43<7:38:49,  8.24s/it]
 91%|█████████ | 34623/37962 [80:09:52<7:39:09,  8.25s/it]
 91%|█████████ | 34624/37962 [80:10:00<7:42:41,  8.32s/it]
 91%|█████████ | 34625/37962 [80:10:09<7:42:45,  8.32s/it]
 91%|█████████ | 34626/37962 [80:10:17<7:38:18,  8.24s/it]
 91%|█████████ | 34627/37962 [80:10:25<7:36:32,  8.21s/it]
 91%|█████████ | 34628/37962 [80:10:33<7:32:22,  8.14s/it]
 91%|█████████ | 34629/37962 [80:10:41<7:38:13,  8.25s/it]
 91%|█████████ | 34630/37962 [80:10:49<7:37:09,  8.23s/it]
                                                          
{'loss': 1.1052, 'grad_norm': 22.661710739135742, 'learning_rate': 4.0134215056551793e-07, 'epoch': 0.91}

 91%|█████████ | 34630/37962 [80:10:49<7:37:09,  8.23s/it]
 91%|█████████ | 34631/37962 [80:10:57<7:33:10,  8.16s/it]
 91%|█████████ | 34632/37962 [80:11:06<7:32:32,  8.15s/it]
 91%|█████████ | 34633/37962 [80:11:21<9:40:43, 10.47s/it]
 91%|█████████ | 34634/37962 [80:11:30<9:02:47,  9.79s/it]
 91%|█████████ | 34635/37962 [80:11:38<8:40:17,  9.38s/it]
 91%|█████████ | 34636/37962 [80:11:46<8:23:45,  9.09s/it]
 91%|█████████ | 34637/37962 [80:11:55<8:08:22,  8.81s/it]
 91%|█████████ | 34638/37962 [80:12:03<7:58:41,  8.64s/it]
 91%|█████████ | 34639/37962 [80:12:11<7:42:58,  8.36s/it]
 91%|█████████ | 34640/37962 [80:12:19<7:43:47,  8.38s/it]
                                                          
{'loss': 1.22, 'grad_norm': 9.472006797790527, 'learning_rate': 3.9895287253981065e-07, 'epoch': 0.91}

 91%|█████████ | 34640/37962 [80:12:19<7:43:47,  8.38s/it]
 91%|█████████▏| 34641/37962 [80:12:27<7:45:16,  8.41s/it]
 91%|█████████▏| 34642/37962 [80:12:36<7:40:29,  8.32s/it]
 91%|█████████▏| 34643/37962 [80:12:44<7:39:04,  8.30s/it]
 91%|█████████▏| 34644/37962 [80:12:52<7:41:29,  8.35s/it]
 91%|█████████▏| 34645/37962 [80:13:01<7:43:06,  8.38s/it]
 91%|█████████▏| 34646/37962 [80:13:09<7:40:27,  8.33s/it]
 91%|█████████▏| 34647/37962 [80:13:17<7:40:23,  8.33s/it]
 91%|█████████▏| 34648/37962 [80:13:26<7:38:15,  8.30s/it]
 91%|█████████▏| 34649/37962 [80:13:34<7:37:32,  8.29s/it]
 91%|█████████▏| 34650/37962 [80:13:42<7:36:31,  8.27s/it]
                                                          
{'loss': 1.3375, 'grad_norm': 8.794376373291016, 'learning_rate': 3.9657058294997885e-07, 'epoch': 0.91}

 91%|█████████▏| 34650/37962 [80:13:42<7:36:31,  8.27s/it]
 91%|█████████▏| 34651/37962 [80:13:50<7:32:09,  8.19s/it]
 91%|█████████▏| 34652/37962 [80:13:58<7:34:34,  8.24s/it]
 91%|█████████▏| 34653/37962 [80:14:07<7:34:22,  8.24s/it]
 91%|█████████▏| 34654/37962 [80:14:15<7:35:31,  8.26s/it]
 91%|█████████▏| 34655/37962 [80:14:23<7:28:46,  8.14s/it]
 91%|█████████▏| 34656/37962 [80:14:31<7:33:57,  8.24s/it]
 91%|█████████▏| 34657/37962 [80:14:39<7:32:29,  8.21s/it]
 91%|█████████▏| 34658/37962 [80:14:48<7:33:44,  8.24s/it]
 91%|█████████▏| 34659/37962 [80:14:56<7:28:10,  8.14s/it]
 91%|█████████▏| 34660/37962 [80:15:04<7:30:27,  8.19s/it]
                                                          
{'loss': 1.0883, 'grad_norm': 11.453802108764648, 'learning_rate': 3.941952835300533e-07, 'epoch': 0.91}

 91%|█████████▏| 34660/37962 [80:15:04<7:30:27,  8.19s/it]
 91%|█████████▏| 34661/37962 [80:15:12<7:28:35,  8.15s/it]
 91%|█████████▏| 34662/37962 [80:15:20<7:30:40,  8.19s/it]
 91%|█████████▏| 34663/37962 [80:15:29<7:31:16,  8.21s/it]
 91%|█████████▏| 34664/37962 [80:15:37<7:31:56,  8.22s/it]
 91%|█████████▏| 34665/37962 [80:15:45<7:34:22,  8.27s/it]
 91%|█████████▏| 34666/37962 [80:15:53<7:32:28,  8.24s/it]
 91%|█████████▏| 34667/37962 [80:16:02<7:31:54,  8.23s/it]
 91%|█████████▏| 34668/37962 [80:16:10<7:30:37,  8.21s/it]
 91%|█████████▏| 34669/37962 [80:16:18<7:34:50,  8.29s/it]
 91%|█████████▏| 34670/37962 [80:16:26<7:32:30,  8.25s/it]
                                                          
{'loss': 1.0843, 'grad_norm': 8.617049217224121, 'learning_rate': 3.918269760089721e-07, 'epoch': 0.91}

 91%|█████████▏| 34670/37962 [80:16:26<7:32:30,  8.25s/it]
 91%|█████████▏| 34671/37962 [80:16:35<7:34:41,  8.29s/it]
 91%|█████████▏| 34672/37962 [80:16:43<7:35:36,  8.31s/it]
 91%|█████████▏| 34673/37962 [80:16:51<7:33:18,  8.27s/it]
 91%|█████████▏| 34674/37962 [80:17:00<7:35:05,  8.30s/it]
 91%|█████████▏| 34675/37962 [80:17:08<7:32:30,  8.26s/it]
 91%|█████████▏| 34676/37962 [80:17:16<7:30:41,  8.23s/it]
 91%|█████████▏| 34677/37962 [80:17:24<7:31:49,  8.25s/it]
 91%|█████████▏| 34678/37962 [80:17:33<7:34:49,  8.31s/it]
 91%|█████████▏| 34679/37962 [80:17:41<7:37:34,  8.36s/it]
 91%|█████████▏| 34680/37962 [80:17:50<7:37:02,  8.36s/it]
                                                          
{'loss': 1.2083, 'grad_norm': 14.568305969238281, 'learning_rate': 3.89465662110583e-07, 'epoch': 0.91}

 91%|█████████▏| 34680/37962 [80:17:50<7:37:02,  8.36s/it]
 91%|█████████▏| 34681/37962 [80:17:58<7:32:11,  8.27s/it]
 91%|█████████▏| 34682/37962 [80:18:06<7:32:09,  8.27s/it]
 91%|█████████▏| 34683/37962 [80:18:14<7:26:58,  8.18s/it]
 91%|█████████▏| 34684/37962 [80:18:22<7:29:29,  8.23s/it]
 91%|█████████▏| 34685/37962 [80:18:30<7:30:53,  8.26s/it]
 91%|█████████▏| 34686/37962 [80:18:39<7:33:49,  8.31s/it]
 91%|█████████▏| 34687/37962 [80:18:47<7:36:02,  8.35s/it]
 91%|█████████▏| 34688/37962 [80:18:55<7:30:58,  8.26s/it]
 91%|█████████▏| 34689/37962 [80:19:04<7:29:00,  8.23s/it]
 91%|█████████▏| 34690/37962 [80:19:12<7:28:28,  8.22s/it]
                                                          
{'loss': 1.1827, 'grad_norm': 6.929413795471191, 'learning_rate': 3.8711134355364665e-07, 'epoch': 0.91}

 91%|█████████▏| 34690/37962 [80:19:12<7:28:28,  8.22s/it]
 91%|█████████▏| 34691/37962 [80:19:20<7:34:33,  8.34s/it]
 91%|█████████▏| 34692/37962 [80:19:29<7:37:52,  8.40s/it]
 91%|█████████▏| 34693/37962 [80:19:37<7:32:25,  8.30s/it]
 91%|█████████▏| 34694/37962 [80:19:45<7:31:55,  8.30s/it]
 91%|█████████▏| 34695/37962 [80:19:54<7:30:52,  8.28s/it]
 91%|█████████▏| 34696/37962 [80:20:02<7:36:03,  8.38s/it]
 91%|█████████▏| 34697/37962 [80:20:11<7:36:55,  8.40s/it]
 91%|█████████▏| 34698/37962 [80:20:19<7:36:50,  8.40s/it]
 91%|█████████▏| 34699/37962 [80:20:27<7:35:02,  8.37s/it]
 91%|█████████▏| 34700/37962 [80:20:36<7:37:11,  8.41s/it]
                                                          
{'loss': 1.1858, 'grad_norm': 13.545747756958008, 'learning_rate': 3.847640220518345e-07, 'epoch': 0.91}

 91%|█████████▏| 34700/37962 [80:20:36<7:37:11,  8.41s/it]
 91%|█████████▏| 34701/37962 [80:20:44<7:33:07,  8.34s/it]
 91%|█████████▏| 34702/37962 [80:20:52<7:28:27,  8.25s/it]
 91%|█████████▏| 34703/37962 [80:21:00<7:27:48,  8.24s/it]
 91%|█████████▏| 34704/37962 [80:21:09<7:32:26,  8.33s/it]
 91%|█████████▏| 34705/37962 [80:21:17<7:30:42,  8.30s/it]
 91%|█████████▏| 34706/37962 [80:21:25<7:29:37,  8.29s/it]
 91%|█████████▏| 34707/37962 [80:21:34<7:28:59,  8.28s/it]
 91%|█████████▏| 34708/37962 [80:21:42<7:30:59,  8.32s/it]
 91%|█████████▏| 34709/37962 [80:21:50<7:30:31,  8.31s/it]
 91%|█████████▏| 34710/37962 [80:21:58<7:28:29,  8.27s/it]
                                                          
{'loss': 1.1733, 'grad_norm': 10.864808082580566, 'learning_rate': 3.8242369931371647e-07, 'epoch': 0.91}

 91%|█████████▏| 34710/37962 [80:21:58<7:28:29,  8.27s/it]
 91%|█████████▏| 34711/37962 [80:22:07<7:28:10,  8.27s/it]
 91%|█████████▏| 34712/37962 [80:22:15<7:30:04,  8.31s/it]
 91%|█████████▏| 34713/37962 [80:22:24<7:32:55,  8.36s/it]
 91%|█████████▏| 34714/37962 [80:22:32<7:30:49,  8.33s/it]
 91%|█████████▏| 34715/37962 [80:22:40<7:25:52,  8.24s/it]
 91%|█████████▏| 34716/37962 [80:22:48<7:26:20,  8.25s/it]
 91%|█████████▏| 34717/37962 [80:22:56<7:26:46,  8.26s/it]
 91%|█████████▏| 34718/37962 [80:23:05<7:31:26,  8.35s/it]
 91%|█████████▏| 34719/37962 [80:23:13<7:29:13,  8.31s/it]
 91%|█████████▏| 34720/37962 [80:23:21<7:28:10,  8.29s/it]
                                                          
{'loss': 1.1124, 'grad_norm': 11.020844459533691, 'learning_rate': 3.800903770427744e-07, 'epoch': 0.91}

 91%|█████████▏| 34720/37962 [80:23:21<7:28:10,  8.29s/it]
 91%|█████████▏| 34721/37962 [80:23:30<7:28:27,  8.30s/it]
 91%|█████████▏| 34722/37962 [80:23:38<7:29:19,  8.32s/it]
 91%|█████████▏| 34723/37962 [80:23:46<7:26:26,  8.27s/it]
 91%|█████████▏| 34724/37962 [80:23:54<7:25:13,  8.25s/it]
 91%|█████████▏| 34725/37962 [80:24:03<7:29:07,  8.32s/it]
 91%|█████████▏| 34726/37962 [80:24:11<7:29:18,  8.33s/it]
 91%|█████████▏| 34727/37962 [80:24:20<7:29:09,  8.33s/it]
 91%|█████████▏| 34728/37962 [80:24:28<7:30:50,  8.36s/it]
 91%|█████████▏| 34729/37962 [80:24:37<7:32:20,  8.39s/it]
 91%|█████████▏| 34730/37962 [80:24:45<7:26:12,  8.28s/it]
                                                          
{'loss': 1.1573, 'grad_norm': 12.68433666229248, 'learning_rate': 3.7776405693739417e-07, 'epoch': 0.91}

 91%|█████████▏| 34730/37962 [80:24:45<7:26:12,  8.28s/it]
 91%|█████████▏| 34731/37962 [80:24:52<7:18:54,  8.15s/it]
 91%|█████████▏| 34732/37962 [80:25:00<7:15:13,  8.08s/it]
 91%|█████████▏| 34733/37962 [80:25:09<7:17:45,  8.13s/it]
 91%|█████████▏| 34734/37962 [80:25:17<7:23:43,  8.25s/it]
 91%|█████████▏| 34735/37962 [80:25:25<7:23:06,  8.24s/it]
 92%|█████████▏| 34736/37962 [80:25:34<7:23:31,  8.25s/it]
 92%|█████████▏| 34737/37962 [80:25:42<7:26:15,  8.30s/it]
 92%|█████████▏| 34738/37962 [80:25:50<7:26:45,  8.31s/it]
 92%|█████████▏| 34739/37962 [80:25:59<7:25:45,  8.30s/it]
 92%|█████████▏| 34740/37962 [80:26:07<7:19:51,  8.19s/it]
                                                          
{'loss': 1.1885, 'grad_norm': 8.306726455688477, 'learning_rate': 3.754447406908612e-07, 'epoch': 0.92}

 92%|█████████▏| 34740/37962 [80:26:07<7:19:51,  8.19s/it]
 92%|█████████▏| 34741/37962 [80:26:15<7:21:41,  8.23s/it]
 92%|█████████▏| 34742/37962 [80:26:23<7:25:30,  8.30s/it]
 92%|█████████▏| 34743/37962 [80:26:32<7:23:49,  8.27s/it]
 92%|█████████▏| 34744/37962 [80:26:40<7:22:24,  8.25s/it]
 92%|█████████▏| 34745/37962 [80:26:48<7:16:42,  8.15s/it]
 92%|█████████▏| 34746/37962 [80:26:56<7:15:44,  8.13s/it]
 92%|█████████▏| 34747/37962 [80:27:04<7:16:26,  8.15s/it]
 92%|█████████▏| 34748/37962 [80:27:12<7:18:05,  8.18s/it]
 92%|█████████▏| 34749/37962 [80:27:21<7:19:53,  8.21s/it]
 92%|█████████▏| 34750/37962 [80:27:29<7:20:49,  8.23s/it]
                                                          
{'loss': 1.1713, 'grad_norm': 15.842392921447754, 'learning_rate': 3.7313242999136856e-07, 'epoch': 0.92}

 92%|█████████▏| 34750/37962 [80:27:29<7:20:49,  8.23s/it]
 92%|█████████▏| 34751/37962 [80:27:37<7:21:40,  8.25s/it]
 92%|█████████▏| 34752/37962 [80:27:45<7:21:45,  8.26s/it]
 92%|█████████▏| 34753/37962 [80:27:54<7:21:05,  8.25s/it]
 92%|█████████▏| 34754/37962 [80:28:02<7:21:55,  8.27s/it]
 92%|█████████▏| 34755/37962 [80:28:10<7:22:41,  8.28s/it]
 92%|█████████▏| 34756/37962 [80:28:18<7:20:58,  8.25s/it]
 92%|█████████▏| 34757/37962 [80:28:27<7:22:24,  8.28s/it]
 92%|█████████▏| 34758/37962 [80:28:35<7:22:50,  8.29s/it]
 92%|█████████▏| 34759/37962 [80:28:43<7:23:43,  8.31s/it]
 92%|█████████▏| 34760/37962 [80:28:52<7:21:45,  8.28s/it]
                                                          
{'loss': 1.0568, 'grad_norm': 6.022691249847412, 'learning_rate': 3.708271265220087e-07, 'epoch': 0.92}

 92%|█████████▏| 34760/37962 [80:28:52<7:21:45,  8.28s/it]
 92%|█████████▏| 34761/37962 [80:29:00<7:20:18,  8.25s/it]
 92%|█████████▏| 34762/37962 [80:29:08<7:15:46,  8.17s/it]
 92%|█████████▏| 34763/37962 [80:29:16<7:10:23,  8.07s/it]
 92%|█████████▏| 34764/37962 [80:29:24<7:12:42,  8.12s/it]
 92%|█████████▏| 34765/37962 [80:29:32<7:15:51,  8.18s/it]
 92%|█████████▏| 34766/37962 [80:29:40<7:17:13,  8.21s/it]
 92%|█████████▏| 34767/37962 [80:29:49<7:19:35,  8.26s/it]
 92%|█████████▏| 34768/37962 [80:29:57<7:18:55,  8.25s/it]
 92%|█████████▏| 34769/37962 [80:30:05<7:18:31,  8.24s/it]
 92%|█████████▏| 34770/37962 [80:30:13<7:17:21,  8.22s/it]
                                                          
{'loss': 1.1169, 'grad_norm': 6.986219882965088, 'learning_rate': 3.6852883196076605e-07, 'epoch': 0.92}

 92%|█████████▏| 34770/37962 [80:30:13<7:17:21,  8.22s/it]
 92%|█████████▏| 34771/37962 [80:30:22<7:18:06,  8.24s/it]
 92%|█████████▏| 34772/37962 [80:30:30<7:21:55,  8.31s/it]
 92%|█████████▏| 34773/37962 [80:30:38<7:20:16,  8.28s/it]
 92%|█████████▏| 34774/37962 [80:30:47<7:20:12,  8.28s/it]
 92%|█████████▏| 34775/37962 [80:30:55<7:19:27,  8.27s/it]
 92%|█████████▏| 34776/37962 [80:31:03<7:21:48,  8.32s/it]
 92%|█████████▏| 34777/37962 [80:31:12<7:19:18,  8.28s/it]
 92%|█████████▏| 34778/37962 [80:31:20<7:16:55,  8.23s/it]
 92%|█████████▏| 34779/37962 [80:31:28<7:18:08,  8.26s/it]
 92%|█████████▏| 34780/37962 [80:31:36<7:14:18,  8.19s/it]
                                                          
{'loss': 1.1288, 'grad_norm': 7.124515533447266, 'learning_rate': 3.6623754798053357e-07, 'epoch': 0.92}

 92%|█████████▏| 34780/37962 [80:31:36<7:14:18,  8.19s/it]
 92%|█████████▏| 34781/37962 [80:31:44<7:10:21,  8.12s/it]
 92%|█████████▏| 34782/37962 [80:31:52<7:11:51,  8.15s/it]
 92%|█████████▏| 34783/37962 [80:32:00<7:11:29,  8.14s/it]
 92%|█████████▏| 34784/37962 [80:32:09<7:13:05,  8.18s/it]
 92%|█████████▏| 34785/37962 [80:32:17<7:12:51,  8.17s/it]
 92%|█████████▏| 34786/37962 [80:32:25<7:08:23,  8.09s/it]
 92%|█████████▏| 34787/37962 [80:32:33<7:11:51,  8.16s/it]
 92%|█████████▏| 34788/37962 [80:32:41<7:12:52,  8.18s/it]
 92%|█████████▏| 34789/37962 [80:32:50<7:15:33,  8.24s/it]
 92%|█████████▏| 34790/37962 [80:32:57<7:09:17,  8.12s/it]
                                                          
{'loss': 1.151, 'grad_norm': 8.407670974731445, 'learning_rate': 3.6395327624909714e-07, 'epoch': 0.92}

 92%|█████████▏| 34790/37962 [80:32:57<7:09:17,  8.12s/it]
 92%|█████████▏| 34791/37962 [80:33:06<7:12:47,  8.19s/it]
 92%|█████████▏| 34792/37962 [80:33:14<7:09:55,  8.14s/it]
 92%|█████████▏| 34793/37962 [80:33:30<9:18:36, 10.58s/it]
 92%|█████████▏| 34794/37962 [80:33:38<8:43:34,  9.92s/it]
 92%|█████████▏| 34795/37962 [80:33:47<8:14:54,  9.38s/it]
 92%|█████████▏| 34796/37962 [80:33:55<7:59:09,  9.08s/it]
 92%|█████████▏| 34797/37962 [80:34:03<7:45:20,  8.82s/it]
 92%|█████████▏| 34798/37962 [80:34:11<7:33:34,  8.60s/it]
 92%|█████████▏| 34799/37962 [80:34:20<7:30:06,  8.54s/it]
 92%|█████████▏| 34800/37962 [80:34:28<7:28:22,  8.51s/it]
                                                          
{'loss': 1.0792, 'grad_norm': 9.635004997253418, 'learning_rate': 3.616760184291368e-07, 'epoch': 0.92}

 92%|█████████▏| 34800/37962 [80:34:28<7:28:22,  8.51s/it]
 92%|█████████▏| 34801/37962 [80:34:36<7:25:21,  8.45s/it]
 92%|█████████▏| 34802/37962 [80:34:45<7:19:44,  8.35s/it]
 92%|█████████▏| 34803/37962 [80:34:53<7:21:41,  8.39s/it]
 92%|█████████▏| 34804/37962 [80:35:02<7:23:39,  8.43s/it]
 92%|█████████▏| 34805/37962 [80:35:10<7:19:24,  8.35s/it]
 92%|█████████▏| 34806/37962 [80:35:18<7:16:16,  8.29s/it]
 92%|█████████▏| 34807/37962 [80:35:26<7:16:21,  8.30s/it]
 92%|█████████▏| 34808/37962 [80:35:34<7:15:34,  8.29s/it]
 92%|█████████▏| 34809/37962 [80:35:43<7:15:44,  8.29s/it]
 92%|█████████▏| 34810/37962 [80:35:51<7:07:31,  8.14s/it]
                                                          
{'loss': 1.038, 'grad_norm': 14.190855979919434, 'learning_rate': 3.59405776178231e-07, 'epoch': 0.92}

 92%|█████████▏| 34810/37962 [80:35:51<7:07:31,  8.14s/it]
 92%|█████████▏| 34811/37962 [80:35:58<7:02:46,  8.05s/it]
 92%|█████████▏| 34812/37962 [80:36:07<7:06:43,  8.13s/it]
 92%|█████████▏| 34813/37962 [80:36:15<7:08:57,  8.17s/it]
 92%|█████████▏| 34814/37962 [80:36:23<7:10:56,  8.21s/it]
 92%|█████████▏| 34815/37962 [80:36:32<7:13:17,  8.26s/it]
 92%|█████████▏| 34816/37962 [80:36:40<7:12:22,  8.25s/it]
 92%|█████████▏| 34817/37962 [80:36:48<7:14:33,  8.29s/it]
 92%|█████████▏| 34818/37962 [80:36:57<7:14:35,  8.29s/it]
 92%|█████████▏| 34819/37962 [80:37:05<7:19:24,  8.39s/it]
 92%|█████████▏| 34820/37962 [80:37:13<7:14:16,  8.29s/it]
                                                          
{'loss': 1.13, 'grad_norm': 17.446435928344727, 'learning_rate': 3.571425511488491e-07, 'epoch': 0.92}

 92%|█████████▏| 34820/37962 [80:37:13<7:14:16,  8.29s/it]
 92%|█████████▏| 34821/37962 [80:37:22<7:14:53,  8.31s/it]
 92%|█████████▏| 34822/37962 [80:37:30<7:13:07,  8.28s/it]
 92%|█████████▏| 34823/37962 [80:37:38<7:14:40,  8.31s/it]
 92%|█████████▏| 34824/37962 [80:37:46<7:14:37,  8.31s/it]
 92%|█████████▏| 34825/37962 [80:37:55<7:14:57,  8.32s/it]
 92%|█████████▏| 34826/37962 [80:38:03<7:13:48,  8.30s/it]
 92%|█████████▏| 34827/37962 [80:38:11<7:13:33,  8.30s/it]
 92%|█████████▏| 34828/37962 [80:38:19<7:09:18,  8.22s/it]
 92%|█████████▏| 34829/37962 [80:38:28<7:10:30,  8.24s/it]
 92%|█████████▏| 34830/37962 [80:38:36<7:12:39,  8.29s/it]
                                                          
{'loss': 1.1885, 'grad_norm': 15.851126670837402, 'learning_rate': 3.548863449883522e-07, 'epoch': 0.92}

 92%|█████████▏| 34830/37962 [80:38:36<7:12:39,  8.29s/it]
 92%|█████████▏| 34831/37962 [80:38:44<7:11:35,  8.27s/it]
 92%|█████████▏| 34832/37962 [80:38:52<7:07:44,  8.20s/it]
 92%|█████████▏| 34833/37962 [80:39:01<7:07:16,  8.19s/it]
 92%|█████████▏| 34834/37962 [80:39:09<7:07:20,  8.20s/it]
 92%|█████████▏| 34835/37962 [80:39:17<7:09:27,  8.24s/it]
 92%|█████████▏| 34836/37962 [80:39:25<7:08:17,  8.22s/it]
 92%|█████████▏| 34837/37962 [80:39:33<7:07:57,  8.22s/it]
 92%|█████████▏| 34838/37962 [80:39:41<7:03:05,  8.13s/it]
 92%|█████████▏| 34839/37962 [80:39:50<7:05:22,  8.17s/it]
 92%|█████████▏| 34840/37962 [80:39:58<7:07:10,  8.21s/it]
                                                          
{'loss': 1.0151, 'grad_norm': 12.545223236083984, 'learning_rate': 3.5263715933899543e-07, 'epoch': 0.92}

 92%|█████████▏| 34840/37962 [80:39:58<7:07:10,  8.21s/it]
 92%|█████████▏| 34841/37962 [80:40:06<7:07:51,  8.23s/it]
 92%|█████████▏| 34842/37962 [80:40:14<7:06:31,  8.20s/it]
 92%|█████████▏| 34843/37962 [80:40:23<7:10:15,  8.28s/it]
 92%|█████████▏| 34844/37962 [80:40:31<7:10:52,  8.29s/it]
 92%|█████████▏| 34845/37962 [80:40:39<7:09:00,  8.26s/it]
 92%|█████████▏| 34846/37962 [80:40:48<7:09:16,  8.27s/it]
 92%|█████████▏| 34847/37962 [80:40:56<7:10:13,  8.29s/it]
 92%|█████████▏| 34848/37962 [80:41:04<7:10:04,  8.29s/it]
 92%|█████████▏| 34849/37962 [80:41:13<7:11:37,  8.32s/it]
 92%|█████████▏| 34850/37962 [80:41:21<7:09:55,  8.29s/it]
                                                          
{'loss': 1.0801, 'grad_norm': 10.077058792114258, 'learning_rate': 3.503949958379205e-07, 'epoch': 0.92}

 92%|█████████▏| 34850/37962 [80:41:21<7:09:55,  8.29s/it]
 92%|█████████▏| 34851/37962 [80:41:29<7:04:19,  8.18s/it]
 92%|█████████▏| 34852/37962 [80:41:37<7:00:21,  8.11s/it]
 92%|█████████▏| 34853/37962 [80:41:45<7:02:17,  8.15s/it]
 92%|█████████▏| 34854/37962 [80:41:53<7:03:33,  8.18s/it]
 92%|█████████▏| 34855/37962 [80:42:01<7:04:47,  8.20s/it]
 92%|█████████▏| 34856/37962 [80:42:10<7:07:06,  8.25s/it]
 92%|█████████▏| 34857/37962 [80:42:18<7:08:28,  8.28s/it]
 92%|█████████▏| 34858/37962 [80:42:27<7:10:20,  8.32s/it]
 92%|█████████▏| 34859/37962 [80:42:35<7:11:38,  8.35s/it]
 92%|█████████▏| 34860/37962 [80:42:43<7:10:08,  8.32s/it]
                                                          
{'loss': 1.1137, 'grad_norm': 4.87405252456665, 'learning_rate': 3.481598561171595e-07, 'epoch': 0.92}

 92%|█████████▏| 34860/37962 [80:42:43<7:10:08,  8.32s/it]
 92%|█████████▏| 34861/37962 [80:42:51<7:08:28,  8.29s/it]
 92%|█████████▏| 34862/37962 [80:43:00<7:08:59,  8.30s/it]
 92%|█████████▏| 34863/37962 [80:43:08<7:12:13,  8.37s/it]
 92%|█████████▏| 34864/37962 [80:43:17<7:10:10,  8.33s/it]
 92%|█████████▏| 34865/37962 [80:43:25<7:09:32,  8.32s/it]
 92%|█████████▏| 34866/37962 [80:43:33<7:10:43,  8.35s/it]
 92%|█████████▏| 34867/37962 [80:43:41<7:03:05,  8.20s/it]
 92%|█████████▏| 34868/37962 [80:43:49<7:04:27,  8.23s/it]
 92%|█████████▏| 34869/37962 [80:43:58<7:07:08,  8.29s/it]
 92%|█████████▏| 34870/37962 [80:44:06<7:06:13,  8.27s/it]
                                                          
{'loss': 1.1275, 'grad_norm': 15.157437324523926, 'learning_rate': 3.4593174180363563e-07, 'epoch': 0.92}

 92%|█████████▏| 34870/37962 [80:44:06<7:06:13,  8.27s/it]
 92%|█████████▏| 34871/37962 [80:44:14<7:01:36,  8.18s/it]
 92%|█████████▏| 34872/37962 [80:44:22<7:05:12,  8.26s/it]
 92%|█████████▏| 34873/37962 [80:44:31<7:05:31,  8.27s/it]
 92%|█████████▏| 34874/37962 [80:44:39<7:04:20,  8.24s/it]
 92%|█████████▏| 34875/37962 [80:44:47<7:05:14,  8.27s/it]
 92%|█████████▏| 34876/37962 [80:44:56<7:05:44,  8.28s/it]
 92%|█████████▏| 34877/37962 [80:45:04<7:07:07,  8.31s/it]
 92%|█████████▏| 34878/37962 [80:45:12<7:08:46,  8.34s/it]
 92%|█████████▏| 34879/37962 [80:45:21<7:06:14,  8.30s/it]
 92%|█████████▏| 34880/37962 [80:45:29<7:07:41,  8.33s/it]
                                                          
{'loss': 1.1791, 'grad_norm': 8.567394256591797, 'learning_rate': 3.437106545191493e-07, 'epoch': 0.92}

 92%|█████████▏| 34880/37962 [80:45:29<7:07:41,  8.33s/it]
 92%|█████████▏| 34881/37962 [80:45:37<7:07:12,  8.32s/it]
 92%|█████████▏| 34882/37962 [80:45:45<7:04:52,  8.28s/it]
 92%|█████████▏| 34883/37962 [80:45:53<6:58:03,  8.15s/it]
 92%|█████████▏| 34884/37962 [80:46:01<6:58:31,  8.16s/it]
 92%|█████████▏| 34885/37962 [80:46:10<7:00:33,  8.20s/it]
 92%|█████████▏| 34886/37962 [80:46:18<7:02:03,  8.23s/it]
 92%|█████████▏| 34887/37962 [80:46:26<7:04:18,  8.28s/it]
 92%|█████████▏| 34888/37962 [80:46:35<7:07:38,  8.35s/it]
 92%|█████████▏| 34889/37962 [80:46:43<7:07:59,  8.36s/it]
 92%|█████████▏| 34890/37962 [80:46:52<7:05:24,  8.31s/it]
                                                          
{'loss': 1.0996, 'grad_norm': 13.740571022033691, 'learning_rate': 3.414965958803929e-07, 'epoch': 0.92}

 92%|█████████▏| 34890/37962 [80:46:52<7:05:24,  8.31s/it]
 92%|█████████▏| 34891/37962 [80:47:00<7:04:58,  8.30s/it]
 92%|█████████▏| 34892/37962 [80:47:08<7:05:39,  8.32s/it]
 92%|█████████▏| 34893/37962 [80:47:16<7:01:52,  8.25s/it]
 92%|█████████▏| 34894/37962 [80:47:24<6:55:38,  8.13s/it]
 92%|█████████▏| 34895/37962 [80:47:32<6:57:55,  8.18s/it]
 92%|█████████▏| 34896/37962 [80:47:41<6:59:12,  8.20s/it]
 92%|█████████▏| 34897/37962 [80:47:49<7:04:06,  8.30s/it]
 92%|█████████▏| 34898/37962 [80:47:57<7:02:45,  8.28s/it]
 92%|█████████▏| 34899/37962 [80:48:06<7:02:57,  8.29s/it]
 92%|█████████▏| 34900/37962 [80:48:14<7:00:52,  8.25s/it]
                                                          
{'loss': 1.1445, 'grad_norm': 12.134849548339844, 'learning_rate': 3.3928956749894384e-07, 'epoch': 0.92}

 92%|█████████▏| 34900/37962 [80:48:14<7:00:52,  8.25s/it]
 92%|█████████▏| 34901/37962 [80:48:22<7:02:49,  8.29s/it]
 92%|█████████▏| 34902/37962 [80:48:30<7:01:43,  8.27s/it]
 92%|█████████▏| 34903/37962 [80:48:38<6:55:37,  8.15s/it]
 92%|█████████▏| 34904/37962 [80:48:47<6:57:59,  8.20s/it]
 92%|█████████▏| 34905/37962 [80:48:55<6:59:37,  8.24s/it]
 92%|█████████▏| 34906/37962 [80:49:03<7:01:23,  8.27s/it]
 92%|█████████▏| 34907/37962 [80:49:12<7:00:19,  8.26s/it]
 92%|█████████▏| 34908/37962 [80:49:20<6:58:19,  8.22s/it]
 92%|█████████▏| 34909/37962 [80:49:28<7:00:07,  8.26s/it]
 92%|█████████▏| 34910/37962 [80:49:37<7:04:06,  8.34s/it]
                                                          
{'loss': 1.1691, 'grad_norm': 6.4669036865234375, 'learning_rate': 3.3708957098125716e-07, 'epoch': 0.92}

 92%|█████████▏| 34910/37962 [80:49:37<7:04:06,  8.34s/it]
 92%|█████████▏| 34911/37962 [80:49:45<7:03:24,  8.33s/it]
 92%|█████████▏| 34912/37962 [80:49:53<7:02:18,  8.31s/it]
 92%|█████████▏| 34913/37962 [80:50:01<7:02:42,  8.32s/it]
 92%|█████████▏| 34914/37962 [80:50:10<7:02:45,  8.32s/it]
 92%|█████████▏| 34915/37962 [80:50:18<6:57:53,  8.23s/it]
 92%|█████████▏| 34916/37962 [80:50:26<6:56:30,  8.20s/it]
 92%|█████████▏| 34917/37962 [80:50:34<6:56:43,  8.21s/it]
 92%|█████████▏| 34918/37962 [80:50:42<6:51:47,  8.12s/it]
 92%|█████████▏| 34919/37962 [80:50:50<6:49:57,  8.08s/it]
 92%|█████████▏| 34920/37962 [80:50:58<6:49:39,  8.08s/it]
                                                          
{'loss': 1.0953, 'grad_norm': 16.00926399230957, 'learning_rate': 3.3489660792867526e-07, 'epoch': 0.92}

 92%|█████████▏| 34920/37962 [80:50:58<6:49:39,  8.08s/it]
 92%|█████████▏| 34921/37962 [80:51:06<6:48:30,  8.06s/it]
 92%|█████████▏| 34922/37962 [80:51:14<6:46:19,  8.02s/it]
 92%|█████████▏| 34923/37962 [80:51:22<6:47:37,  8.05s/it]
 92%|█████████▏| 34924/37962 [80:51:30<6:46:52,  8.04s/it]
 92%|█████████▏| 34925/37962 [80:51:39<6:52:59,  8.16s/it]
 92%|█████████▏| 34926/37962 [80:51:47<6:54:31,  8.19s/it]
 92%|█████████▏| 34927/37962 [80:51:55<6:56:10,  8.23s/it]
 92%|█████████▏| 34928/37962 [80:52:04<7:00:18,  8.31s/it]
 92%|█████████▏| 34929/37962 [80:52:12<6:59:42,  8.30s/it]
 92%|█████████▏| 34930/37962 [80:52:20<6:58:18,  8.28s/it]
                                                          
{'loss': 1.1562, 'grad_norm': 10.941378593444824, 'learning_rate': 3.327106799374147e-07, 'epoch': 0.92}

 92%|█████████▏| 34930/37962 [80:52:20<6:58:18,  8.28s/it]
 92%|█████████▏| 34931/37962 [80:52:29<6:58:09,  8.28s/it]
 92%|█████████▏| 34932/37962 [80:52:37<6:59:20,  8.30s/it]
 92%|█████████▏| 34933/37962 [80:52:45<6:58:44,  8.29s/it]
 92%|█████████▏| 34934/37962 [80:52:53<6:55:30,  8.23s/it]
 92%|█████████▏| 34935/37962 [80:53:02<6:56:55,  8.26s/it]
 92%|█████████▏| 34936/37962 [80:53:09<6:50:33,  8.14s/it]
 92%|█████████▏| 34937/37962 [80:53:18<6:51:51,  8.17s/it]
 92%|█████████▏| 34938/37962 [80:53:26<6:54:10,  8.22s/it]
 92%|█████████▏| 34939/37962 [80:53:34<6:55:27,  8.25s/it]
 92%|█████████▏| 34940/37962 [80:53:43<7:00:18,  8.34s/it]
                                                          
{'loss': 1.2504, 'grad_norm': 18.4073429107666, 'learning_rate': 3.305317885985759e-07, 'epoch': 0.92}

 92%|█████████▏| 34940/37962 [80:53:43<7:00:18,  8.34s/it]
 92%|█████████▏| 34941/37962 [80:53:51<6:59:43,  8.34s/it]
 92%|█████████▏| 34942/37962 [80:53:59<6:57:34,  8.30s/it]
 92%|█████████▏| 34943/37962 [80:54:08<6:57:07,  8.29s/it]
 92%|█████████▏| 34944/37962 [80:54:16<6:54:33,  8.24s/it]
 92%|█████████▏| 34945/37962 [80:54:24<6:52:01,  8.19s/it]
 92%|█████████▏| 34946/37962 [80:54:32<6:53:27,  8.23s/it]
 92%|█████████▏| 34947/37962 [80:54:40<6:53:40,  8.23s/it]
 92%|█████████▏| 34948/37962 [80:54:49<6:56:10,  8.28s/it]
 92%|█████████▏| 34949/37962 [80:54:57<6:59:19,  8.35s/it]
 92%|█████████▏| 34950/37962 [80:55:06<6:56:18,  8.29s/it]
                                                          
{'loss': 1.0611, 'grad_norm': 7.866973400115967, 'learning_rate': 3.2835993549813593e-07, 'epoch': 0.92}

 92%|█████████▏| 34950/37962 [80:55:06<6:56:18,  8.29s/it]
 92%|█████████▏| 34951/37962 [80:55:14<6:55:07,  8.27s/it]
 92%|█████████▏| 34952/37962 [80:55:22<6:54:37,  8.26s/it]
 92%|█████████▏| 34953/37962 [80:55:30<6:51:09,  8.20s/it]
 92%|█████████▏| 34954/37962 [80:55:38<6:53:22,  8.25s/it]
 92%|█████████▏| 34955/37962 [80:55:47<6:55:00,  8.28s/it]
 92%|█████████▏| 34956/37962 [80:55:55<6:54:39,  8.28s/it]
 92%|█████████▏| 34957/37962 [80:56:03<6:56:15,  8.31s/it]
 92%|█████████▏| 34958/37962 [80:56:11<6:50:37,  8.20s/it]
 92%|█████████▏| 34959/37962 [80:56:20<6:50:53,  8.21s/it]
 92%|█████████▏| 34960/37962 [80:56:28<6:55:14,  8.30s/it]
                                                          
{'loss': 1.1868, 'grad_norm': 9.33503532409668, 'learning_rate': 3.2619512221694905e-07, 'epoch': 0.92}

 92%|█████████▏| 34960/37962 [80:56:28<6:55:14,  8.30s/it]
 92%|█████████▏| 34961/37962 [80:56:37<6:56:14,  8.32s/it]
 92%|█████████▏| 34962/37962 [80:56:45<6:57:30,  8.35s/it]
 92%|█████████▏| 34963/37962 [80:56:53<6:59:12,  8.39s/it]
 92%|█████████▏| 34964/37962 [80:57:01<6:53:09,  8.27s/it]
 92%|█████████▏| 34965/37962 [80:57:10<6:53:00,  8.27s/it]
 92%|█████████▏| 34966/37962 [80:57:18<6:52:32,  8.26s/it]
 92%|█████████▏| 34967/37962 [80:57:26<6:51:47,  8.25s/it]
 92%|█████████▏| 34968/37962 [80:57:34<6:52:54,  8.27s/it]
 92%|█████████▏| 34969/37962 [80:57:43<6:52:03,  8.26s/it]
 92%|█████████▏| 34970/37962 [80:57:51<6:54:23,  8.31s/it]
                                                          
{'loss': 1.0021, 'grad_norm': 8.085517883300781, 'learning_rate': 3.24037350330747e-07, 'epoch': 0.92}

 92%|█████████▏| 34970/37962 [80:57:51<6:54:23,  8.31s/it]
 92%|█████████▏| 34971/37962 [80:57:59<6:47:19,  8.17s/it]
 92%|█████████▏| 34972/37962 [80:58:07<6:48:05,  8.19s/it]
 92%|█████████▏| 34973/37962 [80:58:16<6:51:24,  8.26s/it]
 92%|█████████▏| 34974/37962 [80:58:24<6:51:57,  8.27s/it]
 92%|█████████▏| 34975/37962 [80:58:32<6:53:54,  8.31s/it]
 92%|█████████▏| 34976/37962 [80:58:41<6:52:06,  8.28s/it]
 92%|█████████▏| 34977/37962 [80:58:49<6:53:34,  8.31s/it]
 92%|█████████▏| 34978/37962 [80:58:57<6:53:01,  8.30s/it]
 92%|█████████▏| 34979/37962 [80:59:05<6:52:40,  8.30s/it]
 92%|█████████▏| 34980/37962 [80:59:14<6:53:50,  8.33s/it]
                                                          
{'loss': 1.1625, 'grad_norm': 10.626887321472168, 'learning_rate': 3.218866214101313e-07, 'epoch': 0.92}

 92%|█████████▏| 34980/37962 [80:59:14<6:53:50,  8.33s/it]
 92%|█████████▏| 34981/37962 [80:59:22<6:51:18,  8.28s/it]
 92%|█████████▏| 34982/37962 [80:59:30<6:51:47,  8.29s/it]
 92%|█████████▏| 34983/37962 [80:59:38<6:48:28,  8.23s/it]
 92%|█████████▏| 34984/37962 [80:59:47<6:49:02,  8.24s/it]
 92%|█████████▏| 34985/37962 [80:59:55<6:51:37,  8.30s/it]
 92%|█████████▏| 34986/37962 [81:00:03<6:49:06,  8.25s/it]
 92%|█████████▏| 34987/37962 [81:00:12<6:49:02,  8.25s/it]
 92%|█████████▏| 34988/37962 [81:00:20<6:49:38,  8.26s/it]
 92%|█████████▏| 34989/37962 [81:00:28<6:48:09,  8.24s/it]
 92%|█████████▏| 34990/37962 [81:00:36<6:45:21,  8.18s/it]
                                                          
{'loss': 1.1755, 'grad_norm': 9.1748046875, 'learning_rate': 3.197429370205829e-07, 'epoch': 0.92}

 92%|█████████▏| 34990/37962 [81:00:36<6:45:21,  8.18s/it]
 92%|█████████▏| 34991/37962 [81:00:44<6:46:54,  8.22s/it]
 92%|█████████▏| 34992/37962 [81:00:53<6:46:32,  8.21s/it]
 92%|█████████▏| 34993/37962 [81:01:01<6:48:07,  8.25s/it]
 92%|█████████▏| 34994/37962 [81:01:09<6:42:55,  8.15s/it]
 92%|█████████▏| 34995/37962 [81:01:17<6:44:35,  8.18s/it]
 92%|█████████▏| 34996/37962 [81:01:25<6:48:11,  8.26s/it]
 92%|█████████▏| 34997/37962 [81:01:34<6:48:52,  8.27s/it]
 92%|█████████▏| 34998/37962 [81:01:42<6:48:08,  8.26s/it]
 92%|█████████▏| 34999/37962 [81:01:51<6:53:40,  8.38s/it]
 92%|█████████▏| 35000/37962 [81:01:59<6:47:49,  8.26s/it]
                                                          
{'loss': 1.0913, 'grad_norm': 12.799250602722168, 'learning_rate': 3.176062987224504e-07, 'epoch': 0.92}

 92%|█████████▏| 35000/37962 [81:01:59<6:47:49,  8.26s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 92%|█████████▏| 35001/37962 [81:02:48<16:56:48, 20.60s/it]
 92%|█████████▏| 35002/37962 [81:02:56<13:52:54, 16.88s/it]
 92%|█████████▏| 35003/37962 [81:03:05<11:47:57, 14.36s/it]
 92%|█████████▏| 35004/37962 [81:03:13<10:16:48, 12.51s/it]
 92%|█████████▏| 35005/37962 [81:03:21<9:14:32, 11.25s/it] 
 92%|█████████▏| 35006/37962 [81:03:29<8:23:48, 10.23s/it]
 92%|█████████▏| 35007/37962 [81:03:37<7:55:06,  9.65s/it]
 92%|█████████▏| 35008/37962 [81:03:46<7:35:49,  9.26s/it]
 92%|█████████▏| 35009/37962 [81:03:54<7:17:19,  8.89s/it]
 92%|█████████▏| 35010/37962 [81:04:02<7:08:28,  8.71s/it]
                                                          
{'loss': 1.2066, 'grad_norm': 8.19770336151123, 'learning_rate': 3.1547670807095864e-07, 'epoch': 0.92}

 92%|█████████▏| 35010/37962 [81:04:02<7:08:28,  8.71s/it]
 92%|█████████▏| 35011/37962 [81:04:10<7:02:25,  8.59s/it]
 92%|█████████▏| 35012/37962 [81:04:19<6:58:20,  8.51s/it]
 92%|█████████▏| 35013/37962 [81:04:27<6:55:17,  8.45s/it]
 92%|█████████▏| 35014/37962 [81:04:35<6:52:19,  8.39s/it]
 92%|█████████▏| 35015/37962 [81:04:44<6:51:09,  8.37s/it]
 92%|█████████▏| 35016/37962 [81:04:51<6:43:12,  8.21s/it]
 92%|█████████▏| 35017/37962 [81:05:00<6:42:49,  8.21s/it]
 92%|█████████▏| 35018/37962 [81:05:08<6:43:05,  8.22s/it]
 92%|█████████▏| 35019/37962 [81:05:16<6:43:46,  8.23s/it]
 92%|█████████▏| 35020/37962 [81:05:24<6:43:12,  8.22s/it]
                                                          
{'loss': 1.1312, 'grad_norm': 13.099564552307129, 'learning_rate': 3.133541666161977e-07, 'epoch': 0.92}

 92%|█████████▏| 35020/37962 [81:05:24<6:43:12,  8.22s/it]
 92%|█████████▏| 35021/37962 [81:05:32<6:37:24,  8.11s/it]
 92%|█████████▏| 35022/37962 [81:05:40<6:38:23,  8.13s/it]
 92%|█████████▏| 35023/37962 [81:05:49<6:40:49,  8.18s/it]
 92%|█████████▏| 35024/37962 [81:05:57<6:42:25,  8.22s/it]
 92%|█████████▏| 35025/37962 [81:06:05<6:44:47,  8.27s/it]
 92%|█████████▏| 35026/37962 [81:06:14<6:44:39,  8.27s/it]
 92%|█████████▏| 35027/37962 [81:06:22<6:45:45,  8.29s/it]
 92%|█████████▏| 35028/37962 [81:06:30<6:46:13,  8.31s/it]
 92%|█████████▏| 35029/37962 [81:06:39<6:45:59,  8.31s/it]
 92%|█████████▏| 35030/37962 [81:06:47<6:43:36,  8.26s/it]
                                                          
{'loss': 1.1215, 'grad_norm': 8.530671119689941, 'learning_rate': 3.112386759031294e-07, 'epoch': 0.92}

 92%|█████████▏| 35030/37962 [81:06:47<6:43:36,  8.26s/it]
 92%|█████████▏| 35031/37962 [81:06:55<6:38:35,  8.16s/it]
 92%|█████████▏| 35032/37962 [81:07:03<6:42:38,  8.25s/it]
 92%|█████████▏| 35033/37962 [81:07:11<6:41:58,  8.23s/it]
 92%|█████████▏| 35034/37962 [81:07:20<6:41:50,  8.23s/it]
 92%|█████████▏| 35035/37962 [81:07:28<6:41:53,  8.24s/it]
 92%|█████████▏| 35036/37962 [81:07:36<6:43:17,  8.27s/it]
 92%|█████████▏| 35037/37962 [81:07:44<6:40:39,  8.22s/it]
 92%|█████████▏| 35038/37962 [81:07:52<6:38:15,  8.17s/it]
 92%|█████████▏| 35039/37962 [81:08:01<6:39:52,  8.21s/it]
 92%|█████████▏| 35040/37962 [81:08:09<6:41:04,  8.24s/it]
                                                          
{'loss': 1.0338, 'grad_norm': 15.649774551391602, 'learning_rate': 3.0913023747158324e-07, 'epoch': 0.92}

 92%|█████████▏| 35040/37962 [81:08:09<6:41:04,  8.24s/it]
 92%|█████████▏| 35041/37962 [81:08:17<6:39:19,  8.20s/it]
 92%|█████████▏| 35042/37962 [81:08:25<6:42:18,  8.27s/it]
 92%|█████████▏| 35043/37962 [81:08:34<6:43:02,  8.28s/it]
 92%|█████████▏| 35044/37962 [81:08:42<6:44:04,  8.31s/it]
 92%|█████████▏| 35045/37962 [81:08:50<6:43:49,  8.31s/it]
 92%|█████████▏| 35046/37962 [81:08:59<6:40:01,  8.23s/it]
 92%|█████████▏| 35047/37962 [81:09:07<6:40:35,  8.25s/it]
 92%|█████████▏| 35048/37962 [81:09:15<6:40:41,  8.25s/it]
 92%|█████████▏| 35049/37962 [81:09:24<6:44:25,  8.33s/it]
 92%|█████████▏| 35050/37962 [81:09:32<6:45:00,  8.35s/it]
                                                          
{'loss': 1.2182, 'grad_norm': 12.086459159851074, 'learning_rate': 3.070288528562526e-07, 'epoch': 0.92}

 92%|█████████▏| 35050/37962 [81:09:32<6:45:00,  8.35s/it]
 92%|█████████▏| 35051/37962 [81:09:40<6:43:22,  8.31s/it]
 92%|█████████▏| 35052/37962 [81:09:48<6:43:08,  8.31s/it]
 92%|█████████▏| 35053/37962 [81:09:57<6:42:41,  8.31s/it]
 92%|█████████▏| 35054/37962 [81:10:05<6:42:28,  8.30s/it]
 92%|█████████▏| 35055/37962 [81:10:13<6:41:27,  8.29s/it]
 92%|█████████▏| 35056/37962 [81:10:22<6:40:46,  8.27s/it]
 92%|█████████▏| 35057/37962 [81:10:30<6:40:57,  8.28s/it]
 92%|█████████▏| 35058/37962 [81:10:38<6:38:08,  8.23s/it]
 92%|█████████▏| 35059/37962 [81:10:46<6:37:55,  8.22s/it]
 92%|█████████▏| 35060/37962 [81:10:54<6:39:01,  8.25s/it]
                                                          
{'loss': 1.137, 'grad_norm': 25.359580993652344, 'learning_rate': 3.0493452358670183e-07, 'epoch': 0.92}

 92%|█████████▏| 35060/37962 [81:10:54<6:39:01,  8.25s/it]
 92%|█████████▏| 35061/37962 [81:11:03<6:40:17,  8.28s/it]
 92%|█████████▏| 35062/37962 [81:11:11<6:39:07,  8.26s/it]
 92%|█████████▏| 35063/37962 [81:11:19<6:38:30,  8.25s/it]
 92%|█████████▏| 35064/37962 [81:11:28<6:40:29,  8.29s/it]
 92%|█████████▏| 35065/37962 [81:11:36<6:36:30,  8.21s/it]
 92%|█████████▏| 35066/37962 [81:11:44<6:37:18,  8.23s/it]
 92%|█████████▏| 35067/37962 [81:11:52<6:39:59,  8.29s/it]
 92%|█████████▏| 35068/37962 [81:12:01<6:41:05,  8.32s/it]
 92%|█████████▏| 35069/37962 [81:12:09<6:40:53,  8.31s/it]
 92%|█████████▏| 35070/37962 [81:12:17<6:39:38,  8.29s/it]
                                                          
{'loss': 1.1399, 'grad_norm': 30.076513290405273, 'learning_rate': 3.0284725118735593e-07, 'epoch': 0.92}

 92%|█████████▏| 35070/37962 [81:12:17<6:39:38,  8.29s/it]
 92%|█████████▏| 35071/37962 [81:12:25<6:35:14,  8.20s/it]
 92%|█████████▏| 35072/37962 [81:12:34<6:37:43,  8.26s/it]
 92%|█████████▏| 35073/37962 [81:12:42<6:36:42,  8.24s/it]
 92%|█████████▏| 35074/37962 [81:12:50<6:35:35,  8.22s/it]
 92%|█████████▏| 35075/37962 [81:12:58<6:36:12,  8.23s/it]
 92%|█████████▏| 35076/37962 [81:13:07<6:36:14,  8.24s/it]
 92%|█████████▏| 35077/37962 [81:13:15<6:36:22,  8.24s/it]
 92%|█████████▏| 35078/37962 [81:13:23<6:34:08,  8.20s/it]
 92%|█████████▏| 35079/37962 [81:13:31<6:36:54,  8.26s/it]
 92%|█████████▏| 35080/37962 [81:13:40<6:36:17,  8.25s/it]
                                                          
{'loss': 1.2501, 'grad_norm': 11.856143951416016, 'learning_rate': 3.007670371775051e-07, 'epoch': 0.92}

 92%|█████████▏| 35080/37962 [81:13:40<6:36:17,  8.25s/it]
 92%|█████████▏| 35081/37962 [81:13:48<6:36:13,  8.25s/it]
 92%|█████████▏| 35082/37962 [81:13:56<6:31:30,  8.16s/it]
 92%|█████████▏| 35083/37962 [81:14:04<6:30:19,  8.13s/it]
 92%|█████████▏| 35084/37962 [81:14:12<6:32:14,  8.18s/it]
 92%|█████████▏| 35085/37962 [81:14:20<6:32:44,  8.19s/it]
 92%|█████████▏| 35086/37962 [81:14:28<6:29:59,  8.14s/it]
 92%|█████████▏| 35087/37962 [81:14:37<6:36:28,  8.27s/it]
 92%|█████████▏| 35088/37962 [81:14:45<6:36:38,  8.28s/it]
 92%|█████████▏| 35089/37962 [81:14:53<6:36:06,  8.27s/it]
 92%|█████████▏| 35090/37962 [81:15:02<6:39:24,  8.34s/it]
                                                          
{'loss': 1.0819, 'grad_norm': 7.266757011413574, 'learning_rate': 2.986938830712982e-07, 'epoch': 0.92}

 92%|█████████▏| 35090/37962 [81:15:02<6:39:24,  8.34s/it]
 92%|█████████▏| 35091/37962 [81:15:10<6:36:39,  8.29s/it]
 92%|█████████▏| 35092/37962 [81:15:19<6:37:05,  8.30s/it]
 92%|█████████▏| 35093/37962 [81:15:26<6:29:21,  8.14s/it]
 92%|█████████▏| 35094/37962 [81:15:35<6:30:35,  8.17s/it]
 92%|█████████▏| 35095/37962 [81:15:43<6:30:05,  8.16s/it]
 92%|█████████▏| 35096/37962 [81:15:51<6:29:47,  8.16s/it]
 92%|█████████▏| 35097/37962 [81:15:59<6:27:47,  8.12s/it]
 92%|█████████▏| 35098/37962 [81:16:07<6:30:22,  8.18s/it]
 92%|█████████▏| 35099/37962 [81:16:15<6:31:57,  8.21s/it]
 92%|█████████▏| 35100/37962 [81:16:24<6:31:53,  8.22s/it]
                                                          
{'loss': 1.1701, 'grad_norm': 14.82813835144043, 'learning_rate': 2.9662779037775145e-07, 'epoch': 0.92}

 92%|█████████▏| 35100/37962 [81:16:24<6:31:53,  8.22s/it]
 92%|█████████▏| 35101/37962 [81:16:32<6:34:06,  8.27s/it]
 92%|█████████▏| 35102/37962 [81:16:40<6:33:29,  8.26s/it]
 92%|█████████▏| 35103/37962 [81:16:48<6:29:36,  8.18s/it]
 92%|█████████▏| 35104/37962 [81:16:57<6:31:27,  8.22s/it]
 92%|█████████▏| 35105/37962 [81:17:05<6:32:09,  8.24s/it]
 92%|█████████▏| 35106/37962 [81:17:13<6:31:54,  8.23s/it]
 92%|█████████▏| 35107/37962 [81:17:21<6:32:36,  8.25s/it]
 92%|█████████▏| 35108/37962 [81:17:30<6:34:33,  8.29s/it]
 92%|█████████▏| 35109/37962 [81:17:38<6:34:49,  8.30s/it]
 92%|█████████▏| 35110/37962 [81:17:46<6:34:11,  8.29s/it]
                                                          
{'loss': 1.079, 'grad_norm': 9.683534622192383, 'learning_rate': 2.945687606007364e-07, 'epoch': 0.92}

 92%|█████████▏| 35110/37962 [81:17:46<6:34:11,  8.29s/it]
 92%|█████████▏| 35111/37962 [81:17:55<6:35:04,  8.31s/it]
 92%|█████████▏| 35112/37962 [81:18:03<6:35:53,  8.33s/it]
 92%|█████████▏| 35113/37962 [81:18:11<6:32:45,  8.27s/it]
 92%|█████████▏| 35114/37962 [81:18:20<6:32:30,  8.27s/it]
 93%|█████████▎| 35115/37962 [81:18:28<6:30:00,  8.22s/it]
 93%|█████████▎| 35116/37962 [81:18:36<6:32:00,  8.26s/it]
 93%|█████████▎| 35117/37962 [81:18:44<6:34:36,  8.32s/it]
 93%|█████████▎| 35118/37962 [81:18:53<6:32:34,  8.28s/it]
 93%|█████████▎| 35119/37962 [81:19:01<6:32:44,  8.29s/it]
 93%|█████████▎| 35120/37962 [81:19:09<6:33:57,  8.32s/it]
                                                          
{'loss': 1.2272, 'grad_norm': 13.479316711425781, 'learning_rate': 2.9251679523898627e-07, 'epoch': 0.93}

 93%|█████████▎| 35120/37962 [81:19:09<6:33:57,  8.32s/it]
 93%|█████████▎| 35121/37962 [81:19:18<6:33:41,  8.31s/it]
 93%|█████████▎| 35122/37962 [81:19:26<6:32:23,  8.29s/it]
 93%|█████████▎| 35123/37962 [81:19:34<6:36:18,  8.38s/it]
 93%|█████████▎| 35124/37962 [81:19:43<6:33:40,  8.32s/it]
 93%|█████████▎| 35125/37962 [81:19:51<6:30:35,  8.26s/it]
 93%|█████████▎| 35126/37962 [81:19:59<6:23:34,  8.12s/it]
 93%|█████████▎| 35127/37962 [81:20:07<6:26:34,  8.18s/it]
 93%|█████████▎| 35128/37962 [81:20:15<6:20:27,  8.05s/it]
 93%|█████████▎| 35129/37962 [81:20:23<6:23:50,  8.13s/it]
 93%|█████████▎| 35130/37962 [81:20:31<6:25:20,  8.16s/it]
                                                          
{'loss': 1.0942, 'grad_norm': 11.112903594970703, 'learning_rate': 2.904718957860897e-07, 'epoch': 0.93}

 93%|█████████▎| 35130/37962 [81:20:31<6:25:20,  8.16s/it]
 93%|█████████▎| 35131/37962 [81:20:39<6:26:14,  8.19s/it]
 93%|█████████▎| 35132/37962 [81:20:48<6:26:57,  8.20s/it]
 93%|█████████▎| 35133/37962 [81:20:56<6:28:00,  8.23s/it]
 93%|█████████▎| 35134/37962 [81:21:04<6:28:13,  8.24s/it]
 93%|█████████▎| 35135/37962 [81:21:12<6:28:46,  8.25s/it]
 93%|█████████▎| 35136/37962 [81:21:21<6:28:56,  8.26s/it]
 93%|█████████▎| 35137/37962 [81:21:29<6:28:24,  8.25s/it]
 93%|█████████▎| 35138/37962 [81:21:37<6:24:18,  8.17s/it]
 93%|█████████▎| 35139/37962 [81:21:45<6:21:59,  8.12s/it]
 93%|█████████▎| 35140/37962 [81:21:53<6:21:10,  8.10s/it]
                                                          
{'loss': 1.1549, 'grad_norm': 8.884119033813477, 'learning_rate': 2.8843406373049277e-07, 'epoch': 0.93}

 93%|█████████▎| 35140/37962 [81:21:53<6:21:10,  8.10s/it]
 93%|█████████▎| 35141/37962 [81:22:01<6:25:24,  8.20s/it]
 93%|█████████▎| 35142/37962 [81:22:10<6:23:23,  8.16s/it]
 93%|█████████▎| 35143/37962 [81:22:18<6:22:51,  8.15s/it]
 93%|█████████▎| 35144/37962 [81:22:26<6:20:52,  8.11s/it]
 93%|█████████▎| 35145/37962 [81:22:34<6:23:00,  8.16s/it]
 93%|█████████▎| 35146/37962 [81:22:42<6:24:08,  8.18s/it]
 93%|█████████▎| 35147/37962 [81:22:50<6:24:23,  8.19s/it]
 93%|█████████▎| 35148/37962 [81:22:59<6:25:42,  8.22s/it]
 93%|█████████▎| 35149/37962 [81:23:07<6:23:35,  8.18s/it]
 93%|█████████▎| 35150/37962 [81:23:15<6:25:13,  8.22s/it]
                                                          
{'loss': 1.0823, 'grad_norm': 7.286893367767334, 'learning_rate': 2.864033005555011e-07, 'epoch': 0.93}

 93%|█████████▎| 35150/37962 [81:23:15<6:25:13,  8.22s/it]
 93%|█████████▎| 35151/37962 [81:23:23<6:25:00,  8.22s/it]
 93%|█████████▎| 35152/37962 [81:23:32<6:27:00,  8.26s/it]
 93%|█████████▎| 35153/37962 [81:23:39<6:20:47,  8.13s/it]
 93%|█████████▎| 35154/37962 [81:23:48<6:22:55,  8.18s/it]
 93%|█████████▎| 35155/37962 [81:23:56<6:23:44,  8.20s/it]
 93%|█████████▎| 35156/37962 [81:24:04<6:25:49,  8.25s/it]
 93%|█████████▎| 35157/37962 [81:24:13<6:29:34,  8.33s/it]
 93%|█████████▎| 35158/37962 [81:24:21<6:30:43,  8.36s/it]
 93%|█████████▎| 35159/37962 [81:24:30<6:30:50,  8.37s/it]
 93%|█████████▎| 35160/37962 [81:24:38<6:30:05,  8.35s/it]
                                                          
{'loss': 1.1133, 'grad_norm': 15.727523803710938, 'learning_rate': 2.843796077392691e-07, 'epoch': 0.93}

 93%|█████████▎| 35160/37962 [81:24:38<6:30:05,  8.35s/it]
 93%|█████████▎| 35161/37962 [81:24:46<6:28:27,  8.32s/it]
 93%|█████████▎| 35162/37962 [81:24:54<6:26:40,  8.29s/it]
 93%|█████████▎| 35163/37962 [81:25:03<6:28:29,  8.33s/it]
 93%|█████████▎| 35164/37962 [81:25:11<6:26:40,  8.29s/it]
 93%|█████████▎| 35165/37962 [81:25:19<6:26:14,  8.29s/it]
 93%|█████████▎| 35166/37962 [81:25:28<6:26:22,  8.29s/it]
 93%|█████████▎| 35167/37962 [81:25:36<6:27:15,  8.31s/it]
 93%|█████████▎| 35168/37962 [81:25:44<6:27:39,  8.32s/it]
 93%|█████████▎| 35169/37962 [81:25:53<6:27:07,  8.32s/it]
 93%|█████████▎| 35170/37962 [81:26:01<6:29:44,  8.38s/it]
                                                          
{'loss': 1.1411, 'grad_norm': 10.252971649169922, 'learning_rate': 2.8236298675480946e-07, 'epoch': 0.93}

 93%|█████████▎| 35170/37962 [81:26:01<6:29:44,  8.38s/it]
 93%|█████████▎| 35171/37962 [81:26:09<6:24:07,  8.26s/it]
 93%|█████████▎| 35172/37962 [81:26:17<6:22:34,  8.23s/it]
 93%|█████████▎| 35173/37962 [81:26:25<6:20:41,  8.19s/it]
 93%|█████████▎| 35174/37962 [81:26:34<6:21:17,  8.21s/it]
 93%|█████████▎| 35175/37962 [81:26:42<6:21:34,  8.21s/it]
 93%|█████████▎| 35176/37962 [81:26:50<6:21:34,  8.22s/it]
 93%|█████████▎| 35177/37962 [81:26:58<6:21:13,  8.21s/it]
 93%|█████████▎| 35178/37962 [81:27:07<6:22:21,  8.24s/it]
 93%|█████████▎| 35179/37962 [81:27:15<6:22:21,  8.24s/it]
 93%|█████████▎| 35180/37962 [81:27:23<6:25:20,  8.31s/it]
                                                          
{'loss': 1.188, 'grad_norm': 7.825354099273682, 'learning_rate': 2.803534390699869e-07, 'epoch': 0.93}

 93%|█████████▎| 35180/37962 [81:27:23<6:25:20,  8.31s/it]
 93%|█████████▎| 35181/37962 [81:27:31<6:21:38,  8.23s/it]
 93%|█████████▎| 35182/37962 [81:27:40<6:21:50,  8.24s/it]
 93%|█████████▎| 35183/37962 [81:27:48<6:20:50,  8.22s/it]
 93%|█████████▎| 35184/37962 [81:27:56<6:21:48,  8.25s/it]
 93%|█████████▎| 35185/37962 [81:28:04<6:21:54,  8.25s/it]
 93%|█████████▎| 35186/37962 [81:28:13<6:22:02,  8.26s/it]
 93%|█████████▎| 35187/37962 [81:28:21<6:24:30,  8.31s/it]
 93%|█████████▎| 35188/37962 [81:28:29<6:24:14,  8.31s/it]
 93%|█████████▎| 35189/37962 [81:28:38<6:21:49,  8.26s/it]
 93%|█████████▎| 35190/37962 [81:28:46<6:21:15,  8.25s/it]
                                                          
{'loss': 1.2233, 'grad_norm': 10.369479179382324, 'learning_rate': 2.7835096614751366e-07, 'epoch': 0.93}

 93%|█████████▎| 35190/37962 [81:28:46<6:21:15,  8.25s/it]
 93%|█████████▎| 35191/37962 [81:28:54<6:21:45,  8.27s/it]
 93%|█████████▎| 35192/37962 [81:29:03<6:24:06,  8.32s/it]
 93%|█████████▎| 35193/37962 [81:29:11<6:21:04,  8.26s/it]
 93%|█████████▎| 35194/37962 [81:29:19<6:22:22,  8.29s/it]
 93%|█████████▎| 35195/37962 [81:29:27<6:21:30,  8.27s/it]
 93%|█████████▎| 35196/37962 [81:29:36<6:23:37,  8.32s/it]
 93%|█████████▎| 35197/37962 [81:29:44<6:22:26,  8.30s/it]
 93%|█████████▎| 35198/37962 [81:29:52<6:20:52,  8.27s/it]
 93%|█████████▎| 35199/37962 [81:30:01<6:22:46,  8.31s/it]
 93%|█████████▎| 35200/37962 [81:30:09<6:26:02,  8.39s/it]
                                                          
{'loss': 1.0858, 'grad_norm': 13.462965965270996, 'learning_rate': 2.763555694449549e-07, 'epoch': 0.93}

 93%|█████████▎| 35200/37962 [81:30:09<6:26:02,  8.39s/it]
 93%|█████████▎| 35201/37962 [81:30:18<6:25:24,  8.38s/it]
 93%|█████████▎| 35202/37962 [81:30:26<6:25:32,  8.38s/it]
 93%|█████████▎| 35203/37962 [81:30:42<8:14:52, 10.76s/it]
 93%|█████████▎| 35204/37962 [81:30:51<7:43:56, 10.09s/it]
 93%|█████████▎| 35205/37962 [81:30:59<7:17:53,  9.53s/it]
 93%|█████████▎| 35206/37962 [81:31:07<7:01:47,  9.18s/it]
 93%|█████████▎| 35207/37962 [81:31:16<6:49:53,  8.93s/it]
 93%|█████████▎| 35208/37962 [81:31:23<6:34:34,  8.60s/it]
 93%|█████████▎| 35209/37962 [81:31:32<6:29:20,  8.49s/it]
 93%|█████████▎| 35210/37962 [81:31:40<6:26:39,  8.43s/it]
                                                          
{'loss': 1.0888, 'grad_norm': 14.940919876098633, 'learning_rate': 2.743672504147277e-07, 'epoch': 0.93}

 93%|█████████▎| 35210/37962 [81:31:40<6:26:39,  8.43s/it]
 93%|█████████▎| 35211/37962 [81:31:48<6:27:03,  8.44s/it]
 93%|█████████▎| 35212/37962 [81:31:57<6:25:27,  8.41s/it]
 93%|█████████▎| 35213/37962 [81:32:05<6:23:27,  8.37s/it]
 93%|█████████▎| 35214/37962 [81:32:13<6:21:40,  8.33s/it]
 93%|█████████▎| 35215/37962 [81:32:22<6:19:41,  8.29s/it]
 93%|█████████▎| 35216/37962 [81:32:30<6:18:37,  8.27s/it]
 93%|█████████▎| 35217/37962 [81:32:38<6:19:22,  8.29s/it]
 93%|█████████▎| 35218/37962 [81:32:46<6:13:54,  8.18s/it]
 93%|█████████▎| 35219/37962 [81:32:54<6:16:53,  8.24s/it]
 93%|█████████▎| 35220/37962 [81:33:03<6:17:51,  8.27s/it]
                                                          
{'loss': 1.2033, 'grad_norm': 9.79853630065918, 'learning_rate': 2.723860105040932e-07, 'epoch': 0.93}

 93%|█████████▎| 35220/37962 [81:33:03<6:17:51,  8.27s/it]
 93%|█████████▎| 35221/37962 [81:33:11<6:19:40,  8.31s/it]
 93%|█████████▎| 35222/37962 [81:33:19<6:18:15,  8.28s/it]
 93%|█████████▎| 35223/37962 [81:33:35<8:01:41, 10.55s/it]
 93%|█████████▎| 35224/37962 [81:33:43<7:25:17,  9.76s/it]
 93%|█████████▎| 35225/37962 [81:33:51<7:05:29,  9.33s/it]
 93%|█████████▎| 35226/37962 [81:34:00<6:51:53,  9.03s/it]
 93%|█████████▎| 35227/37962 [81:34:08<6:42:30,  8.83s/it]
 93%|█████████▎| 35228/37962 [81:34:16<6:34:36,  8.66s/it]
 93%|█████████▎| 35229/37962 [81:34:25<6:30:41,  8.58s/it]
 93%|█████████▎| 35230/37962 [81:34:33<6:25:52,  8.47s/it]
                                                          
{'loss': 1.2043, 'grad_norm': 8.61441421508789, 'learning_rate': 2.7041185115516234e-07, 'epoch': 0.93}

 93%|█████████▎| 35230/37962 [81:34:33<6:25:52,  8.47s/it]
 93%|█████████▎| 35231/37962 [81:34:41<6:25:05,  8.46s/it]
 93%|█████████▎| 35232/37962 [81:34:50<6:22:06,  8.40s/it]
 93%|█████████▎| 35233/37962 [81:34:58<6:19:48,  8.35s/it]
 93%|█████████▎| 35234/37962 [81:35:06<6:21:48,  8.40s/it]
 93%|█████████▎| 35235/37962 [81:35:15<6:17:47,  8.31s/it]
 93%|█████████▎| 35236/37962 [81:35:23<6:16:44,  8.29s/it]
 93%|█████████▎| 35237/37962 [81:35:31<6:17:14,  8.31s/it]
 93%|█████████▎| 35238/37962 [81:35:40<6:19:21,  8.36s/it]
 93%|█████████▎| 35239/37962 [81:35:48<6:19:22,  8.36s/it]
 93%|█████████▎| 35240/37962 [81:35:56<6:16:35,  8.30s/it]
                                                          
{'loss': 1.0968, 'grad_norm': 7.251466751098633, 'learning_rate': 2.684447738048934e-07, 'epoch': 0.93}

 93%|█████████▎| 35240/37962 [81:35:56<6:16:35,  8.30s/it]
 93%|█████████▎| 35241/37962 [81:36:04<6:15:42,  8.28s/it]
 93%|█████████▎| 35242/37962 [81:36:12<6:11:16,  8.19s/it]
 93%|█████████▎| 35243/37962 [81:36:21<6:12:25,  8.22s/it]
 93%|█████████▎| 35244/37962 [81:36:29<6:10:50,  8.19s/it]
 93%|█████████▎| 35245/37962 [81:36:37<6:12:47,  8.23s/it]
 93%|█████████▎| 35246/37962 [81:36:45<6:14:26,  8.27s/it]
 93%|█████████▎| 35247/37962 [81:36:54<6:11:11,  8.20s/it]
 93%|█████████▎| 35248/37962 [81:37:02<6:12:22,  8.23s/it]
 93%|█████████▎| 35249/37962 [81:37:10<6:13:41,  8.26s/it]
 93%|█████████▎| 35250/37962 [81:37:18<6:12:48,  8.25s/it]
                                                          
{'loss': 1.0821, 'grad_norm': 12.215846061706543, 'learning_rate': 2.664847798850834e-07, 'epoch': 0.93}

 93%|█████████▎| 35250/37962 [81:37:18<6:12:48,  8.25s/it]
 93%|█████████▎| 35251/37962 [81:37:27<6:12:27,  8.24s/it]
 93%|█████████▎| 35252/37962 [81:37:35<6:12:31,  8.25s/it]
 93%|█████████▎| 35253/37962 [81:37:43<6:11:51,  8.24s/it]
 93%|█████████▎| 35254/37962 [81:37:51<6:10:46,  8.21s/it]
 93%|█████████▎| 35255/37962 [81:37:59<6:09:15,  8.18s/it]
 93%|█████████▎| 35256/37962 [81:38:07<6:06:55,  8.14s/it]
 93%|█████████▎| 35257/37962 [81:38:15<6:04:13,  8.08s/it]
 93%|█████████▎| 35258/37962 [81:38:24<6:07:21,  8.15s/it]
 93%|█████████▎| 35259/37962 [81:38:32<6:08:36,  8.18s/it]
 93%|█████████▎| 35260/37962 [81:38:40<6:09:14,  8.20s/it]
                                                          
{'loss': 1.118, 'grad_norm': 9.009248733520508, 'learning_rate': 2.6453187082238096e-07, 'epoch': 0.93}

 93%|█████████▎| 35260/37962 [81:38:40<6:09:14,  8.20s/it]
 93%|█████████▎| 35261/37962 [81:38:48<6:09:02,  8.20s/it]
 93%|█████████▎| 35262/37962 [81:38:57<6:09:53,  8.22s/it]
 93%|█████████▎| 35263/37962 [81:39:05<6:09:29,  8.21s/it]
 93%|█████████▎| 35264/37962 [81:39:13<6:05:52,  8.14s/it]
 93%|█████████▎| 35265/37962 [81:39:21<6:06:21,  8.15s/it]
 93%|█████████▎| 35266/37962 [81:39:29<6:08:29,  8.20s/it]
 93%|█████████▎| 35267/37962 [81:39:38<6:09:47,  8.23s/it]
 93%|█████████▎| 35268/37962 [81:39:46<6:09:28,  8.23s/it]
 93%|█████████▎| 35269/37962 [81:39:54<6:09:35,  8.23s/it]
 93%|█████████▎| 35270/37962 [81:40:02<6:09:33,  8.24s/it]
                                                          
{'loss': 1.1162, 'grad_norm': 6.85825252532959, 'learning_rate': 2.625860480382758e-07, 'epoch': 0.93}

 93%|█████████▎| 35270/37962 [81:40:02<6:09:33,  8.24s/it]
 93%|█████████▎| 35271/37962 [81:40:11<6:10:38,  8.26s/it]
 93%|█████████▎| 35272/37962 [81:40:19<6:11:55,  8.30s/it]
 93%|█████████▎| 35273/37962 [81:40:28<6:15:16,  8.37s/it]
 93%|█████████▎| 35274/37962 [81:40:36<6:17:09,  8.42s/it]
 93%|█████████▎| 35275/37962 [81:40:44<6:15:21,  8.38s/it]
 93%|█████████▎| 35276/37962 [81:40:53<6:13:19,  8.34s/it]
 93%|█████████▎| 35277/37962 [81:41:01<6:10:03,  8.27s/it]
 93%|█████████▎| 35278/37962 [81:41:09<6:07:41,  8.22s/it]
 93%|█████████▎| 35279/37962 [81:41:17<6:08:58,  8.25s/it]
 93%|█████████▎| 35280/37962 [81:41:26<6:10:35,  8.29s/it]
                                                          
{'loss': 1.1468, 'grad_norm': 11.131779670715332, 'learning_rate': 2.6064731294909606e-07, 'epoch': 0.93}

 93%|█████████▎| 35280/37962 [81:41:26<6:10:35,  8.29s/it]
 93%|█████████▎| 35281/37962 [81:41:34<6:12:10,  8.33s/it]
 93%|█████████▎| 35282/37962 [81:41:42<6:12:04,  8.33s/it]
 93%|█████████▎| 35283/37962 [81:41:51<6:11:53,  8.33s/it]
 93%|█████████▎| 35284/37962 [81:41:59<6:09:16,  8.27s/it]
 93%|█████████▎| 35285/37962 [81:42:07<6:09:37,  8.28s/it]
 93%|█████████▎| 35286/37962 [81:42:15<6:10:54,  8.32s/it]
 93%|█████████▎| 35287/37962 [81:42:24<6:08:03,  8.26s/it]
 93%|█████████▎| 35288/37962 [81:42:32<6:10:08,  8.31s/it]
 93%|█████████▎| 35289/37962 [81:42:40<6:10:11,  8.31s/it]
 93%|█████████▎| 35290/37962 [81:42:49<6:10:21,  8.32s/it]
                                                          
{'loss': 1.0536, 'grad_norm': 9.182358741760254, 'learning_rate': 2.587156669660151e-07, 'epoch': 0.93}

 93%|█████████▎| 35290/37962 [81:42:49<6:10:21,  8.32s/it]
 93%|█████████▎| 35291/37962 [81:42:57<6:10:59,  8.33s/it]
 93%|█████████▎| 35292/37962 [81:43:05<6:11:15,  8.34s/it]
 93%|█████████▎| 35293/37962 [81:43:14<6:11:12,  8.34s/it]
 93%|█████████▎| 35294/37962 [81:43:22<6:11:12,  8.35s/it]
 93%|█████████▎| 35295/37962 [81:43:30<6:08:57,  8.30s/it]
 93%|█████████▎| 35296/37962 [81:43:38<6:07:32,  8.27s/it]
 93%|█████████▎| 35297/37962 [81:43:47<6:08:13,  8.29s/it]
 93%|█████████▎| 35298/37962 [81:43:55<6:07:19,  8.27s/it]
 93%|█████████▎| 35299/37962 [81:44:03<6:07:21,  8.28s/it]
 93%|█████████▎| 35300/37962 [81:44:12<6:07:06,  8.27s/it]
                                                          
{'loss': 1.1114, 'grad_norm': 9.992034912109375, 'learning_rate': 2.5679111149504387e-07, 'epoch': 0.93}

 93%|█████████▎| 35300/37962 [81:44:12<6:07:06,  8.27s/it]
 93%|█████████▎| 35301/37962 [81:44:20<6:06:01,  8.25s/it]
 93%|█████████▎| 35302/37962 [81:44:28<6:04:56,  8.23s/it]
 93%|█████████▎| 35303/37962 [81:44:36<6:07:52,  8.30s/it]
 93%|█████████▎| 35304/37962 [81:44:45<6:09:05,  8.33s/it]
 93%|█████████▎| 35305/37962 [81:44:53<6:10:19,  8.36s/it]
 93%|█████████▎| 35306/37962 [81:45:02<6:10:19,  8.37s/it]
 93%|█████████▎| 35307/37962 [81:45:10<6:08:48,  8.33s/it]
 93%|█████████▎| 35308/37962 [81:45:18<6:11:47,  8.41s/it]
 93%|█████████▎| 35309/37962 [81:45:27<6:10:37,  8.38s/it]
 93%|█████████▎| 35310/37962 [81:45:35<6:10:37,  8.39s/it]
                                                          
{'loss': 1.0908, 'grad_norm': 17.44693374633789, 'learning_rate': 2.5487364793703173e-07, 'epoch': 0.93}

 93%|█████████▎| 35310/37962 [81:45:35<6:10:37,  8.39s/it]
 93%|█████████▎| 35311/37962 [81:45:44<6:10:15,  8.38s/it]
 93%|█████████▎| 35312/37962 [81:45:52<6:06:01,  8.29s/it]
 93%|█████████▎| 35313/37962 [81:46:00<6:07:48,  8.33s/it]
 93%|█████████▎| 35314/37962 [81:46:09<6:09:18,  8.37s/it]
 93%|█████████▎| 35315/37962 [81:46:17<6:08:40,  8.36s/it]
 93%|█████████▎| 35316/37962 [81:46:25<6:02:14,  8.21s/it]
 93%|█████████▎| 35317/37962 [81:46:33<6:07:07,  8.33s/it]
 93%|█████████▎| 35318/37962 [81:46:42<6:05:45,  8.30s/it]
 93%|█████████▎| 35319/37962 [81:46:50<6:04:36,  8.28s/it]
 93%|█████████▎| 35320/37962 [81:46:58<6:05:45,  8.31s/it]
                                                          
{'loss': 1.0439, 'grad_norm': 7.336899280548096, 'learning_rate': 2.5296327768767004e-07, 'epoch': 0.93}

 93%|█████████▎| 35320/37962 [81:46:58<6:05:45,  8.31s/it]
 93%|█████████▎| 35321/37962 [81:47:06<6:03:15,  8.25s/it]
 93%|█████████▎| 35322/37962 [81:47:14<5:58:37,  8.15s/it]
 93%|█████████▎| 35323/37962 [81:47:22<6:00:29,  8.20s/it]
 93%|█████████▎| 35324/37962 [81:47:31<5:58:23,  8.15s/it]
 93%|█████████▎| 35325/37962 [81:47:39<6:02:57,  8.26s/it]
 93%|█████████▎| 35326/37962 [81:47:47<6:05:15,  8.31s/it]
 93%|█████████▎| 35327/37962 [81:47:56<6:04:55,  8.31s/it]
 93%|█████████▎| 35328/37962 [81:48:04<6:06:46,  8.35s/it]
 93%|█████████▎| 35329/37962 [81:48:13<6:07:14,  8.37s/it]
 93%|█████████▎| 35330/37962 [81:48:21<6:05:06,  8.32s/it]
                                                          
{'loss': 1.116, 'grad_norm': 16.147218704223633, 'learning_rate': 2.5106000213748206e-07, 'epoch': 0.93}

 93%|█████████▎| 35330/37962 [81:48:21<6:05:06,  8.32s/it]
 93%|█████████▎| 35331/37962 [81:48:29<6:05:37,  8.34s/it]
 93%|█████████▎| 35332/37962 [81:48:38<6:06:26,  8.36s/it]
 93%|█████████▎| 35333/37962 [81:48:46<6:06:34,  8.37s/it]
 93%|█████████▎| 35334/37962 [81:48:54<6:06:04,  8.36s/it]
 93%|█████████▎| 35335/37962 [81:49:03<6:03:20,  8.30s/it]
 93%|█████████▎| 35336/37962 [81:49:10<5:58:03,  8.18s/it]
 93%|█████████▎| 35337/37962 [81:49:19<5:59:38,  8.22s/it]
 93%|█████████▎| 35338/37962 [81:49:27<5:59:05,  8.21s/it]
 93%|█████████▎| 35339/37962 [81:49:35<5:59:05,  8.21s/it]
 93%|█████████▎| 35340/37962 [81:49:43<5:55:07,  8.13s/it]
                                                          
{'loss': 1.1687, 'grad_norm': 6.154407024383545, 'learning_rate': 2.4916382267182957e-07, 'epoch': 0.93}

 93%|█████████▎| 35340/37962 [81:49:43<5:55:07,  8.13s/it]
 93%|█████████▎| 35341/37962 [81:49:51<5:56:17,  8.16s/it]
 93%|█████████▎| 35342/37962 [81:50:00<5:58:56,  8.22s/it]
 93%|█████████▎| 35343/37962 [81:50:08<5:55:02,  8.13s/it]
 93%|█████████▎| 35344/37962 [81:50:16<5:55:52,  8.16s/it]
 93%|█████████▎| 35345/37962 [81:50:24<5:58:43,  8.22s/it]
 93%|█████████▎| 35346/37962 [81:50:33<6:00:06,  8.26s/it]
 93%|█████████▎| 35347/37962 [81:50:41<6:02:47,  8.32s/it]
 93%|█████████▎| 35348/37962 [81:50:49<5:57:47,  8.21s/it]
 93%|█████████▎| 35349/37962 [81:50:57<5:53:56,  8.13s/it]
 93%|█████████▎| 35350/37962 [81:51:05<5:56:53,  8.20s/it]
                                                          
{'loss': 1.073, 'grad_norm': 9.446101188659668, 'learning_rate': 2.472747406709097e-07, 'epoch': 0.93}

 93%|█████████▎| 35350/37962 [81:51:05<5:56:53,  8.20s/it]
 93%|█████████▎| 35351/37962 [81:51:14<6:00:34,  8.29s/it]
 93%|█████████▎| 35352/37962 [81:51:22<5:56:02,  8.18s/it]
 93%|█████████▎| 35353/37962 [81:51:30<5:58:14,  8.24s/it]
 93%|█████████▎| 35354/37962 [81:51:38<5:59:18,  8.27s/it]
 93%|█████████▎| 35355/37962 [81:51:47<5:57:55,  8.24s/it]
 93%|█████████▎| 35356/37962 [81:51:55<5:57:34,  8.23s/it]
 93%|█████████▎| 35357/37962 [81:52:03<5:59:06,  8.27s/it]
 93%|█████████▎| 35358/37962 [81:52:11<6:00:00,  8.30s/it]
 93%|█████████▎| 35359/37962 [81:52:20<5:57:44,  8.25s/it]
 93%|█████████▎| 35360/37962 [81:52:28<5:58:28,  8.27s/it]
                                                          
{'loss': 1.2464, 'grad_norm': 10.34638500213623, 'learning_rate': 2.4539275750975035e-07, 'epoch': 0.93}

 93%|█████████▎| 35360/37962 [81:52:28<5:58:28,  8.27s/it]
 93%|█████████▎| 35361/37962 [81:52:36<5:59:59,  8.30s/it]
 93%|█████████▎| 35362/37962 [81:52:45<5:58:39,  8.28s/it]
 93%|█████████▎| 35363/37962 [81:52:53<6:01:21,  8.34s/it]
 93%|█████████▎| 35364/37962 [81:53:01<5:59:58,  8.31s/it]
 93%|█████████▎| 35365/37962 [81:53:09<5:57:53,  8.27s/it]
 93%|█████████▎| 35366/37962 [81:53:18<5:58:55,  8.30s/it]
 93%|█████████▎| 35367/37962 [81:53:26<5:58:52,  8.30s/it]
 93%|█████████▎| 35368/37962 [81:53:34<5:58:31,  8.29s/it]
 93%|█████████▎| 35369/37962 [81:53:43<6:00:15,  8.34s/it]
 93%|█████████▎| 35370/37962 [81:53:51<5:59:54,  8.33s/it]
                                                          
{'loss': 1.1657, 'grad_norm': 9.2090482711792, 'learning_rate': 2.4351787455821675e-07, 'epoch': 0.93}

 93%|█████████▎| 35370/37962 [81:53:51<5:59:54,  8.33s/it]
 93%|█████████▎| 35371/37962 [81:54:00<6:00:05,  8.34s/it]
 93%|█████████▎| 35372/37962 [81:54:08<6:00:38,  8.35s/it]
 93%|█████████▎| 35373/37962 [81:54:16<6:00:21,  8.35s/it]
 93%|█████████▎| 35374/37962 [81:54:24<5:57:22,  8.29s/it]
 93%|█████████▎| 35375/37962 [81:54:32<5:49:49,  8.11s/it]
 93%|█████████▎| 35376/37962 [81:54:41<5:54:14,  8.22s/it]
 93%|█████████▎| 35377/37962 [81:54:48<5:49:14,  8.11s/it]
 93%|█████████▎| 35378/37962 [81:54:57<5:49:28,  8.11s/it]
 93%|█████████▎| 35379/37962 [81:55:05<5:51:53,  8.17s/it]
 93%|█████████▎| 35380/37962 [81:55:13<5:53:55,  8.22s/it]
                                                          
{'loss': 1.1746, 'grad_norm': 13.727375984191895, 'learning_rate': 2.4165009318100286e-07, 'epoch': 0.93}

 93%|█████████▎| 35380/37962 [81:55:13<5:53:55,  8.22s/it]
 93%|█████████▎| 35381/37962 [81:55:21<5:53:34,  8.22s/it]
 93%|█████████▎| 35382/37962 [81:55:29<5:51:35,  8.18s/it]
 93%|█████████▎| 35383/37962 [81:55:38<5:55:33,  8.27s/it]
 93%|█████████▎| 35384/37962 [81:55:46<5:56:27,  8.30s/it]
 93%|█████████▎| 35385/37962 [81:55:55<5:56:01,  8.29s/it]
 93%|█████████▎| 35386/37962 [81:56:03<5:55:32,  8.28s/it]
 93%|█████████▎| 35387/37962 [81:56:11<5:57:03,  8.32s/it]
 93%|█████████▎| 35388/37962 [81:56:19<5:50:57,  8.18s/it]
 93%|█████████▎| 35389/37962 [81:56:27<5:51:48,  8.20s/it]
 93%|█████████▎| 35390/37962 [81:56:35<5:49:07,  8.14s/it]
                                                          
{'loss': 1.2238, 'grad_norm': 15.226557731628418, 'learning_rate': 2.397894147376345e-07, 'epoch': 0.93}

 93%|█████████▎| 35390/37962 [81:56:35<5:49:07,  8.14s/it]
 93%|█████████▎| 35391/37962 [81:56:43<5:46:12,  8.08s/it]
 93%|█████████▎| 35392/37962 [81:56:52<5:51:45,  8.21s/it]
 93%|█████████▎| 35393/37962 [81:57:00<5:48:39,  8.14s/it]
 93%|█████████▎| 35394/37962 [81:57:08<5:44:55,  8.06s/it]
 93%|█████████▎| 35395/37962 [81:57:16<5:46:37,  8.10s/it]
 93%|█████████▎| 35396/37962 [81:57:24<5:45:51,  8.09s/it]
 93%|█████████▎| 35397/37962 [81:57:32<5:42:30,  8.01s/it]
 93%|█████████▎| 35398/37962 [81:57:40<5:44:49,  8.07s/it]
 93%|█████████▎| 35399/37962 [81:57:48<5:46:07,  8.10s/it]
 93%|█████████▎| 35400/37962 [81:57:56<5:47:52,  8.15s/it]
                                                          
{'loss': 1.1269, 'grad_norm': 6.827721118927002, 'learning_rate': 2.379358405824661e-07, 'epoch': 0.93}

 93%|█████████▎| 35400/37962 [81:57:56<5:47:52,  8.15s/it]
 93%|█████████▎| 35401/37962 [81:58:05<5:49:29,  8.19s/it]
 93%|█████████▎| 35402/37962 [81:58:13<5:49:37,  8.19s/it]
 93%|█████████▎| 35403/37962 [81:58:21<5:49:12,  8.19s/it]
 93%|█████████▎| 35404/37962 [81:58:29<5:48:39,  8.18s/it]
 93%|█████████▎| 35405/37962 [81:58:37<5:47:24,  8.15s/it]
 93%|█████████▎| 35406/37962 [81:58:46<5:50:29,  8.23s/it]
 93%|█████████▎| 35407/37962 [81:58:54<5:50:26,  8.23s/it]
 93%|█████████▎| 35408/37962 [81:59:02<5:53:22,  8.30s/it]
 93%|█████████▎| 35409/37962 [81:59:11<5:52:54,  8.29s/it]
 93%|█████████▎| 35410/37962 [81:59:19<5:52:22,  8.28s/it]
                                                          
{'loss': 1.1731, 'grad_norm': 17.900949478149414, 'learning_rate': 2.3608937206468395e-07, 'epoch': 0.93}

 93%|█████████▎| 35410/37962 [81:59:19<5:52:22,  8.28s/it]
 93%|█████████▎| 35411/37962 [81:59:27<5:51:59,  8.28s/it]
 93%|█████████▎| 35412/37962 [81:59:36<5:51:56,  8.28s/it]
 93%|█████████▎| 35413/37962 [81:59:44<5:51:33,  8.28s/it]
 93%|█████████▎| 35414/37962 [81:59:52<5:49:38,  8.23s/it]
 93%|█████████▎| 35415/37962 [82:00:00<5:48:47,  8.22s/it]
 93%|█████████▎| 35416/37962 [82:00:08<5:48:22,  8.21s/it]
 93%|█████████▎| 35417/37962 [82:00:16<5:46:14,  8.16s/it]
 93%|█████████▎| 35418/37962 [82:00:25<5:50:03,  8.26s/it]
 93%|█████████▎| 35419/37962 [82:00:33<5:50:42,  8.27s/it]
 93%|█████████▎| 35420/37962 [82:00:41<5:50:32,  8.27s/it]
                                                          
{'loss': 1.1744, 'grad_norm': 11.064220428466797, 'learning_rate': 2.3425001052829853e-07, 'epoch': 0.93}

 93%|█████████▎| 35420/37962 [82:00:41<5:50:32,  8.27s/it]
 93%|█████████▎| 35421/37962 [82:00:50<5:48:54,  8.24s/it]
 93%|█████████▎| 35422/37962 [82:00:58<5:48:58,  8.24s/it]
 93%|█████████▎| 35423/37962 [82:01:06<5:50:52,  8.29s/it]
 93%|█████████▎| 35424/37962 [82:01:14<5:49:22,  8.26s/it]
 93%|█████████▎| 35425/37962 [82:01:23<5:50:22,  8.29s/it]
 93%|█████████▎| 35426/37962 [82:01:31<5:46:13,  8.19s/it]
 93%|█████████▎| 35427/37962 [82:01:39<5:46:30,  8.20s/it]
 93%|█████████▎| 35428/37962 [82:01:47<5:46:53,  8.21s/it]
 93%|█████████▎| 35429/37962 [82:01:55<5:46:11,  8.20s/it]
 93%|█████████▎| 35430/37962 [82:02:03<5:40:58,  8.08s/it]
                                                          
{'loss': 1.0594, 'grad_norm': 5.221346378326416, 'learning_rate': 2.324177573121522e-07, 'epoch': 0.93}

 93%|█████████▎| 35430/37962 [82:02:03<5:40:58,  8.08s/it]
 93%|█████████▎| 35431/37962 [82:02:12<5:46:06,  8.20s/it]
 93%|█████████▎| 35432/37962 [82:02:20<5:45:54,  8.20s/it]
 93%|█████████▎| 35433/37962 [82:02:28<5:48:44,  8.27s/it]
 93%|█████████▎| 35434/37962 [82:02:37<5:50:05,  8.31s/it]
 93%|█████████▎| 35435/37962 [82:02:45<5:50:45,  8.33s/it]
 93%|█████████▎| 35436/37962 [82:02:53<5:48:48,  8.29s/it]
 93%|█████████▎| 35437/37962 [82:03:01<5:47:52,  8.27s/it]
 93%|█████████▎| 35438/37962 [82:03:10<5:50:09,  8.32s/it]
 93%|█████████▎| 35439/37962 [82:03:18<5:51:14,  8.35s/it]
 93%|█████████▎| 35440/37962 [82:03:27<5:50:37,  8.34s/it]
                                                          
{'loss': 1.0955, 'grad_norm': 9.296138763427734, 'learning_rate': 2.3059261374990705e-07, 'epoch': 0.93}

 93%|█████████▎| 35440/37962 [82:03:27<5:50:37,  8.34s/it]
 93%|█████████▎| 35441/37962 [82:03:35<5:52:05,  8.38s/it]
 93%|█████████▎| 35442/37962 [82:03:43<5:49:39,  8.33s/it]
 93%|█████████▎| 35443/37962 [82:03:51<5:46:30,  8.25s/it]
 93%|█████████▎| 35444/37962 [82:03:59<5:44:14,  8.20s/it]
 93%|█████████▎| 35445/37962 [82:04:08<5:45:17,  8.23s/it]
 93%|█████████▎| 35446/37962 [82:04:16<5:44:54,  8.23s/it]
 93%|█████████▎| 35447/37962 [82:04:24<5:45:04,  8.23s/it]
 93%|█████████▎| 35448/37962 [82:04:33<5:49:11,  8.33s/it]
 93%|█████████▎| 35449/37962 [82:04:41<5:45:15,  8.24s/it]
 93%|█████████▎| 35450/37962 [82:04:49<5:44:53,  8.24s/it]
                                                          
{'loss': 1.1081, 'grad_norm': 18.757396697998047, 'learning_rate': 2.2877458117005592e-07, 'epoch': 0.93}

 93%|█████████▎| 35450/37962 [82:04:49<5:44:53,  8.24s/it]
 93%|█████████▎| 35451/37962 [82:04:57<5:43:43,  8.21s/it]
 93%|█████████▎| 35452/37962 [82:05:06<5:45:22,  8.26s/it]
 93%|█████████▎| 35453/37962 [82:05:13<5:39:51,  8.13s/it]
 93%|█████████▎| 35454/37962 [82:05:22<5:40:38,  8.15s/it]
 93%|█████████▎| 35455/37962 [82:05:30<5:42:43,  8.20s/it]
 93%|█████████▎| 35456/37962 [82:05:38<5:43:52,  8.23s/it]
 93%|█████████▎| 35457/37962 [82:05:46<5:42:58,  8.21s/it]
 93%|█████████▎| 35458/37962 [82:05:55<5:41:13,  8.18s/it]
 93%|█████████▎| 35459/37962 [82:06:03<5:40:33,  8.16s/it]
 93%|█████████▎| 35460/37962 [82:06:11<5:42:02,  8.20s/it]
                                                          
{'loss': 1.096, 'grad_norm': 13.723665237426758, 'learning_rate': 2.269636608959125e-07, 'epoch': 0.93}

 93%|█████████▎| 35460/37962 [82:06:11<5:42:02,  8.20s/it]
 93%|█████████▎| 35461/37962 [82:06:19<5:44:08,  8.26s/it]
 93%|█████████▎| 35462/37962 [82:06:28<5:44:01,  8.26s/it]
 93%|█████████▎| 35463/37962 [82:06:36<5:44:48,  8.28s/it]
 93%|█████████▎| 35464/37962 [82:06:44<5:43:54,  8.26s/it]
 93%|█████████▎| 35465/37962 [82:06:52<5:43:48,  8.26s/it]
 93%|█████████▎| 35466/37962 [82:07:01<5:43:12,  8.25s/it]
 93%|█████████▎| 35467/37962 [82:07:08<5:38:28,  8.14s/it]
 93%|█████████▎| 35468/37962 [82:07:16<5:34:52,  8.06s/it]
 93%|█████████▎| 35469/37962 [82:07:25<5:39:03,  8.16s/it]
 93%|█████████▎| 35470/37962 [82:07:33<5:35:09,  8.07s/it]
                                                          
{'loss': 1.2109, 'grad_norm': 13.280776977539062, 'learning_rate': 2.251598542456135e-07, 'epoch': 0.93}

 93%|█████████▎| 35470/37962 [82:07:33<5:35:09,  8.07s/it]
 93%|█████████▎| 35471/37962 [82:07:41<5:33:04,  8.02s/it]
 93%|█████████▎| 35472/37962 [82:07:49<5:37:45,  8.14s/it]
 93%|█████████▎| 35473/37962 [82:07:57<5:40:31,  8.21s/it]
 93%|█████████▎| 35474/37962 [82:08:06<5:42:52,  8.27s/it]
 93%|█████████▎| 35475/37962 [82:08:14<5:46:34,  8.36s/it]
 93%|█████████▎| 35476/37962 [82:08:22<5:41:57,  8.25s/it]
 93%|█████████▎| 35477/37962 [82:08:31<5:42:46,  8.28s/it]
 93%|█████████▎| 35478/37962 [82:08:39<5:41:37,  8.25s/it]
 93%|█████████▎| 35479/37962 [82:08:47<5:40:44,  8.23s/it]
 93%|█████████▎| 35480/37962 [82:08:55<5:40:39,  8.23s/it]
                                                          
{'loss': 1.0302, 'grad_norm': 10.779294967651367, 'learning_rate': 2.2336316253211866e-07, 'epoch': 0.93}

 93%|█████████▎| 35480/37962 [82:08:55<5:40:39,  8.23s/it]
 93%|█████████▎| 35481/37962 [82:09:03<5:40:28,  8.23s/it]
 93%|█████████▎| 35482/37962 [82:09:12<5:41:43,  8.27s/it]
 93%|█████████▎| 35483/37962 [82:09:20<5:39:44,  8.22s/it]
 93%|█████████▎| 35484/37962 [82:09:28<5:38:08,  8.19s/it]
 93%|█████████▎| 35485/37962 [82:09:36<5:36:15,  8.15s/it]
 93%|█████████▎| 35486/37962 [82:09:44<5:38:16,  8.20s/it]
 93%|█████████▎| 35487/37962 [82:09:53<5:39:15,  8.22s/it]
 93%|█████████▎| 35488/37962 [82:10:01<5:36:19,  8.16s/it]
 93%|█████████▎| 35489/37962 [82:10:09<5:37:52,  8.20s/it]
 93%|█████████▎| 35490/37962 [82:10:17<5:39:28,  8.24s/it]
                                                          
{'loss': 1.0797, 'grad_norm': 10.112801551818848, 'learning_rate': 2.2157358706320963e-07, 'epoch': 0.93}

 93%|█████████▎| 35490/37962 [82:10:17<5:39:28,  8.24s/it]
 93%|█████████▎| 35491/37962 [82:10:26<5:40:33,  8.27s/it]
 93%|█████████▎| 35492/37962 [82:10:34<5:42:14,  8.31s/it]
 93%|█████████▎| 35493/37962 [82:10:42<5:36:07,  8.17s/it]
 93%|█████████▎| 35494/37962 [82:10:50<5:37:25,  8.20s/it]
 94%|█████████▎| 35495/37962 [82:10:59<5:43:49,  8.36s/it]
 94%|█████████▎| 35496/37962 [82:11:07<5:42:15,  8.33s/it]
 94%|█████████▎| 35497/37962 [82:11:15<5:40:38,  8.29s/it]
 94%|█████████▎| 35498/37962 [82:11:24<5:42:37,  8.34s/it]
 94%|█████████▎| 35499/37962 [82:11:32<5:45:35,  8.42s/it]
 94%|█████████▎| 35500/37962 [82:11:41<5:43:11,  8.36s/it]
                                                          
{'loss': 1.174, 'grad_norm': 9.152740478515625, 'learning_rate': 2.197911291414867e-07, 'epoch': 0.94}

 94%|█████████▎| 35500/37962 [82:11:41<5:43:11,  8.36s/it]
 94%|█████████▎| 35501/37962 [82:11:49<5:40:39,  8.31s/it]
 94%|█████████▎| 35502/37962 [82:11:57<5:36:39,  8.21s/it]
 94%|█████████▎| 35503/37962 [82:12:05<5:39:24,  8.28s/it]
 94%|█████████▎| 35504/37962 [82:12:14<5:38:36,  8.27s/it]
 94%|█████████▎| 35505/37962 [82:12:22<5:39:44,  8.30s/it]
 94%|█████████▎| 35506/37962 [82:12:30<5:38:38,  8.27s/it]
 94%|█████████▎| 35507/37962 [82:12:39<5:40:47,  8.33s/it]
 94%|█████████▎| 35508/37962 [82:12:47<5:39:49,  8.31s/it]
 94%|█████████▎| 35509/37962 [82:12:55<5:38:48,  8.29s/it]
 94%|█████████▎| 35510/37962 [82:13:04<5:42:09,  8.37s/it]
                                                          
{'loss': 1.1716, 'grad_norm': 9.863764762878418, 'learning_rate': 2.180157900643709e-07, 'epoch': 0.94}

 94%|█████████▎| 35510/37962 [82:13:04<5:42:09,  8.37s/it]
 94%|█████████▎| 35511/37962 [82:13:12<5:43:19,  8.40s/it]
 94%|█████████▎| 35512/37962 [82:13:20<5:41:53,  8.37s/it]
 94%|█████████▎| 35513/37962 [82:13:29<5:39:29,  8.32s/it]
 94%|█████████▎| 35514/37962 [82:13:37<5:38:21,  8.29s/it]
 94%|█████████▎| 35515/37962 [82:13:45<5:39:48,  8.33s/it]
 94%|█████████▎| 35516/37962 [82:13:54<5:39:50,  8.34s/it]
 94%|█████████▎| 35517/37962 [82:14:02<5:42:27,  8.40s/it]
 94%|█████████▎| 35518/37962 [82:14:10<5:40:45,  8.37s/it]
 94%|█████████▎| 35519/37962 [82:14:19<5:38:37,  8.32s/it]
 94%|█████████▎| 35520/37962 [82:14:27<5:38:58,  8.33s/it]
                                                          
{'loss': 1.1851, 'grad_norm': 6.8824543952941895, 'learning_rate': 2.1624757112409967e-07, 'epoch': 0.94}

 94%|█████████▎| 35520/37962 [82:14:27<5:38:58,  8.33s/it]
 94%|█████████▎| 35521/37962 [82:14:35<5:38:17,  8.32s/it]
 94%|█████████▎| 35522/37962 [82:14:44<5:38:17,  8.32s/it]
 94%|█████████▎| 35523/37962 [82:14:52<5:38:44,  8.33s/it]
 94%|█████████▎| 35524/37962 [82:15:00<5:39:43,  8.36s/it]
 94%|█████████▎| 35525/37962 [82:15:09<5:37:45,  8.32s/it]
 94%|█████████▎| 35526/37962 [82:15:16<5:31:17,  8.16s/it]
 94%|█████████▎| 35527/37962 [82:15:24<5:28:26,  8.09s/it]
 94%|█████████▎| 35528/37962 [82:15:32<5:27:27,  8.07s/it]
 94%|█████████▎| 35529/37962 [82:15:40<5:27:10,  8.07s/it]
 94%|█████████▎| 35530/37962 [82:15:49<5:29:42,  8.13s/it]
                                                          
{'loss': 1.0704, 'grad_norm': 9.15234661102295, 'learning_rate': 2.1448647360773013e-07, 'epoch': 0.94}

 94%|█████████▎| 35530/37962 [82:15:49<5:29:42,  8.13s/it]
 94%|█████████▎| 35531/37962 [82:15:57<5:30:24,  8.15s/it]
 94%|█████████▎| 35532/37962 [82:16:13<7:06:17, 10.53s/it]
 94%|█████████▎| 35533/37962 [82:16:21<6:37:02,  9.81s/it]
 94%|█████████▎| 35534/37962 [82:16:29<6:15:14,  9.27s/it]
 94%|█████████▎| 35535/37962 [82:16:37<6:03:48,  8.99s/it]
 94%|█████████▎| 35536/37962 [82:16:46<5:56:14,  8.81s/it]
 94%|█████████▎| 35537/37962 [82:16:54<5:46:19,  8.57s/it]
 94%|█████████▎| 35538/37962 [82:17:02<5:35:55,  8.32s/it]
 94%|█████████▎| 35539/37962 [82:17:10<5:34:35,  8.29s/it]
 94%|█████████▎| 35540/37962 [82:17:18<5:35:26,  8.31s/it]
                                                          
{'loss': 1.1245, 'grad_norm': 11.470961570739746, 'learning_rate': 2.1273249879713242e-07, 'epoch': 0.94}

 94%|█████████▎| 35540/37962 [82:17:18<5:35:26,  8.31s/it]
 94%|█████████▎| 35541/37962 [82:17:26<5:34:35,  8.29s/it]
 94%|█████████▎| 35542/37962 [82:17:35<5:32:12,  8.24s/it]
 94%|█████████▎| 35543/37962 [82:17:43<5:34:01,  8.29s/it]
 94%|█████████▎| 35544/37962 [82:17:51<5:33:42,  8.28s/it]
 94%|█████████▎| 35545/37962 [82:17:59<5:32:07,  8.24s/it]
 94%|█████████▎| 35546/37962 [82:18:08<5:30:44,  8.21s/it]
 94%|█████████▎| 35547/37962 [82:18:16<5:28:27,  8.16s/it]
 94%|█████████▎| 35548/37962 [82:18:24<5:29:24,  8.19s/it]
 94%|█████████▎| 35549/37962 [82:18:32<5:33:16,  8.29s/it]
 94%|█████████▎| 35550/37962 [82:18:40<5:29:45,  8.20s/it]
                                                          
{'loss': 1.075, 'grad_norm': 5.655580043792725, 'learning_rate': 2.1098564796899867e-07, 'epoch': 0.94}

 94%|█████████▎| 35550/37962 [82:18:40<5:29:45,  8.20s/it]
 94%|█████████▎| 35551/37962 [82:18:49<5:30:03,  8.21s/it]
 94%|█████████▎| 35552/37962 [82:18:57<5:31:39,  8.26s/it]
 94%|█████████▎| 35553/37962 [82:19:05<5:34:41,  8.34s/it]
 94%|█████████▎| 35554/37962 [82:19:14<5:32:36,  8.29s/it]
 94%|█████████▎| 35555/37962 [82:19:22<5:33:30,  8.31s/it]
 94%|█████████▎| 35556/37962 [82:19:30<5:29:25,  8.22s/it]
 94%|█████████▎| 35557/37962 [82:19:38<5:31:51,  8.28s/it]
 94%|█████████▎| 35558/37962 [82:19:46<5:26:10,  8.14s/it]
 94%|█████████▎| 35559/37962 [82:19:54<5:26:40,  8.16s/it]
 94%|█████████▎| 35560/37962 [82:20:03<5:28:20,  8.20s/it]
                                                          
{'loss': 1.1982, 'grad_norm': 10.25605583190918, 'learning_rate': 2.0924592239482734e-07, 'epoch': 0.94}

 94%|█████████▎| 35560/37962 [82:20:03<5:28:20,  8.20s/it]
 94%|█████████▎| 35561/37962 [82:20:11<5:26:14,  8.15s/it]
 94%|█████████▎| 35562/37962 [82:20:19<5:28:37,  8.22s/it]
 94%|█████████▎| 35563/37962 [82:20:28<5:30:41,  8.27s/it]
 94%|█████████▎| 35564/37962 [82:20:36<5:32:16,  8.31s/it]
 94%|█████████▎| 35565/37962 [82:20:44<5:32:21,  8.32s/it]
 94%|█████████▎| 35566/37962 [82:20:52<5:29:42,  8.26s/it]
 94%|█████████▎| 35567/37962 [82:21:01<5:30:49,  8.29s/it]
 94%|█████████▎| 35568/37962 [82:21:09<5:33:43,  8.36s/it]
 94%|█████████▎| 35569/37962 [82:21:17<5:31:09,  8.30s/it]
 94%|█████████▎| 35570/37962 [82:21:26<5:31:00,  8.30s/it]
                                                          
{'loss': 1.1473, 'grad_norm': 20.1669921875, 'learning_rate': 2.0751332334093544e-07, 'epoch': 0.94}

 94%|█████████▎| 35570/37962 [82:21:26<5:31:00,  8.30s/it]
 94%|█████████▎| 35571/37962 [82:21:34<5:29:52,  8.28s/it]
 94%|█████████▎| 35572/37962 [82:21:42<5:29:21,  8.27s/it]
 94%|█████████▎| 35573/37962 [82:21:50<5:27:08,  8.22s/it]
 94%|█████████▎| 35574/37962 [82:21:59<5:27:01,  8.22s/it]
 94%|█████████▎| 35575/37962 [82:22:07<5:26:14,  8.20s/it]
 94%|█████████▎| 35576/37962 [82:22:15<5:27:16,  8.23s/it]
 94%|█████████▎| 35577/37962 [82:22:23<5:25:08,  8.18s/it]
 94%|█████████▎| 35578/37962 [82:22:31<5:26:18,  8.21s/it]
 94%|█████████▎| 35579/37962 [82:22:40<5:25:53,  8.21s/it]
 94%|█████████▎| 35580/37962 [82:22:48<5:25:56,  8.21s/it]
                                                          
{'loss': 1.0911, 'grad_norm': 12.648122787475586, 'learning_rate': 2.0578785206845308e-07, 'epoch': 0.94}

 94%|█████████▎| 35580/37962 [82:22:48<5:25:56,  8.21s/it]
 94%|█████████▎| 35581/37962 [82:22:56<5:25:49,  8.21s/it]
 94%|█████████▎| 35582/37962 [82:23:04<5:29:26,  8.31s/it]
 94%|█████████▎| 35583/37962 [82:23:13<5:29:55,  8.32s/it]
 94%|█████████▎| 35584/37962 [82:23:21<5:28:43,  8.29s/it]
 94%|█████████▎| 35585/37962 [82:23:29<5:28:57,  8.30s/it]
 94%|█████████▎| 35586/37962 [82:23:38<5:27:35,  8.27s/it]
 94%|█████████▎| 35587/37962 [82:23:46<5:26:21,  8.24s/it]
 94%|█████████▎| 35588/37962 [82:23:54<5:26:29,  8.25s/it]
 94%|█████████▎| 35589/37962 [82:24:02<5:26:32,  8.26s/it]
 94%|█████████▍| 35590/37962 [82:24:11<5:26:47,  8.27s/it]
                                                          
{'loss': 1.1288, 'grad_norm': 10.0946683883667, 'learning_rate': 2.0406950983331897e-07, 'epoch': 0.94}

 94%|█████████▍| 35590/37962 [82:24:11<5:26:47,  8.27s/it]
 94%|█████████▍| 35591/37962 [82:24:19<5:27:26,  8.29s/it]
 94%|█████████▍| 35592/37962 [82:24:27<5:25:33,  8.24s/it]
 94%|█████████▍| 35593/37962 [82:24:35<5:25:19,  8.24s/it]
 94%|█████████▍| 35594/37962 [82:24:44<5:25:49,  8.26s/it]
 94%|█████████▍| 35595/37962 [82:24:52<5:26:00,  8.26s/it]
 94%|█████████▍| 35596/37962 [82:25:00<5:25:08,  8.25s/it]
 94%|█████████▍| 35597/37962 [82:25:09<5:28:11,  8.33s/it]
 94%|█████████▍| 35598/37962 [82:25:17<5:25:22,  8.26s/it]
 94%|█████████▍| 35599/37962 [82:25:25<5:25:16,  8.26s/it]
 94%|█████████▍| 35600/37962 [82:25:33<5:26:08,  8.28s/it]
                                                          
{'loss': 1.2308, 'grad_norm': 7.147735595703125, 'learning_rate': 2.0235829788628703e-07, 'epoch': 0.94}

 94%|█████████▍| 35600/37962 [82:25:33<5:26:08,  8.28s/it]
 94%|█████████▍| 35601/37962 [82:25:42<5:26:06,  8.29s/it]
 94%|█████████▍| 35602/37962 [82:25:50<5:25:18,  8.27s/it]
 94%|█████████▍| 35603/37962 [82:25:58<5:26:36,  8.31s/it]
 94%|█████████▍| 35604/37962 [82:26:07<5:26:35,  8.31s/it]
 94%|█████████▍| 35605/37962 [82:26:15<5:27:31,  8.34s/it]
 94%|█████████▍| 35606/37962 [82:26:23<5:27:05,  8.33s/it]
 94%|█████████▍| 35607/37962 [82:26:32<5:26:58,  8.33s/it]
 94%|█████████▍| 35608/37962 [82:26:40<5:26:27,  8.32s/it]
 94%|█████████▍| 35609/37962 [82:26:48<5:28:30,  8.38s/it]
 94%|█████████▍| 35610/37962 [82:26:57<5:28:06,  8.37s/it]
                                                          
{'loss': 1.1465, 'grad_norm': 11.543400764465332, 'learning_rate': 2.006542174729176e-07, 'epoch': 0.94}

 94%|█████████▍| 35610/37962 [82:26:57<5:28:06,  8.37s/it]
 94%|█████████▍| 35611/37962 [82:27:05<5:27:33,  8.36s/it]
 94%|█████████▍| 35612/37962 [82:27:13<5:20:44,  8.19s/it]
 94%|█████████▍| 35613/37962 [82:27:21<5:20:53,  8.20s/it]
 94%|█████████▍| 35614/37962 [82:27:29<5:20:47,  8.20s/it]
 94%|█████████▍| 35615/37962 [82:27:38<5:22:17,  8.24s/it]
 94%|█████████▍| 35616/37962 [82:27:46<5:22:45,  8.25s/it]
 94%|█████████▍| 35617/37962 [82:27:54<5:22:39,  8.26s/it]
 94%|█████████▍| 35618/37962 [82:28:02<5:23:00,  8.27s/it]
 94%|█████████▍| 35619/37962 [82:28:11<5:23:16,  8.28s/it]
 94%|█████████▍| 35620/37962 [82:28:19<5:18:46,  8.17s/it]
                                                          
{'loss': 1.0524, 'grad_norm': 9.378731727600098, 'learning_rate': 1.9895726983357955e-07, 'epoch': 0.94}

 94%|█████████▍| 35620/37962 [82:28:19<5:18:46,  8.17s/it]
 94%|█████████▍| 35621/37962 [82:28:27<5:21:10,  8.23s/it]
 94%|█████████▍| 35622/37962 [82:28:35<5:21:15,  8.24s/it]
 94%|█████████▍| 35623/37962 [82:28:43<5:17:02,  8.13s/it]
 94%|█████████▍| 35624/37962 [82:28:51<5:18:17,  8.17s/it]
 94%|█████████▍| 35625/37962 [82:29:00<5:19:04,  8.19s/it]
 94%|█████████▍| 35626/37962 [82:29:08<5:22:36,  8.29s/it]
 94%|█████████▍| 35627/37962 [82:29:17<5:23:40,  8.32s/it]
 94%|█████████▍| 35628/37962 [82:29:25<5:23:55,  8.33s/it]
 94%|█████████▍| 35629/37962 [82:29:33<5:20:36,  8.25s/it]
 94%|█████████▍| 35630/37962 [82:29:41<5:20:26,  8.24s/it]
                                                          
{'loss': 1.1366, 'grad_norm': 9.152973175048828, 'learning_rate': 1.9726745620345488e-07, 'epoch': 0.94}

 94%|█████████▍| 35630/37962 [82:29:41<5:20:26,  8.24s/it]
 94%|█████████▍| 35631/37962 [82:29:49<5:19:54,  8.23s/it]
 94%|█████████▍| 35632/37962 [82:29:58<5:19:21,  8.22s/it]
 94%|█████████▍| 35633/37962 [82:30:06<5:22:15,  8.30s/it]
 94%|█████████▍| 35634/37962 [82:30:14<5:22:51,  8.32s/it]
 94%|█████████▍| 35635/37962 [82:30:23<5:23:07,  8.33s/it]
 94%|█████████▍| 35636/37962 [82:30:31<5:22:56,  8.33s/it]
 94%|█████████▍| 35637/37962 [82:30:40<5:23:45,  8.35s/it]
 94%|█████████▍| 35638/37962 [82:30:48<5:22:10,  8.32s/it]
 94%|█████████▍| 35639/37962 [82:30:56<5:21:12,  8.30s/it]
 94%|█████████▍| 35640/37962 [82:31:04<5:20:36,  8.28s/it]
                                                          
{'loss': 1.1543, 'grad_norm': 9.166072845458984, 'learning_rate': 1.955847778125275e-07, 'epoch': 0.94}

 94%|█████████▍| 35640/37962 [82:31:04<5:20:36,  8.28s/it]
 94%|█████████▍| 35641/37962 [82:31:13<5:21:34,  8.31s/it]
 94%|█████████▍| 35642/37962 [82:31:21<5:21:14,  8.31s/it]
 94%|█████████▍| 35643/37962 [82:31:29<5:19:41,  8.27s/it]
 94%|█████████▍| 35644/37962 [82:31:37<5:19:41,  8.28s/it]
 94%|█████████▍| 35645/37962 [82:31:45<5:15:08,  8.16s/it]
 94%|█████████▍| 35646/37962 [82:31:53<5:14:39,  8.15s/it]
 94%|█████████▍| 35647/37962 [82:32:02<5:13:16,  8.12s/it]
 94%|█████████▍| 35648/37962 [82:32:10<5:13:55,  8.14s/it]
 94%|█████████▍| 35649/37962 [82:32:18<5:16:54,  8.22s/it]
 94%|█████████▍| 35650/37962 [82:32:26<5:18:14,  8.26s/it]
                                                          
{'loss': 1.1212, 'grad_norm': 7.036546230316162, 'learning_rate': 1.9390923588558986e-07, 'epoch': 0.94}

 94%|█████████▍| 35650/37962 [82:32:26<5:18:14,  8.26s/it]
 94%|█████████▍| 35651/37962 [82:32:35<5:17:48,  8.25s/it]
 94%|█████████▍| 35652/37962 [82:32:42<5:11:32,  8.09s/it]
 94%|█████████▍| 35653/37962 [82:32:51<5:13:18,  8.14s/it]
 94%|█████████▍| 35654/37962 [82:32:59<5:14:56,  8.19s/it]
 94%|█████████▍| 35655/37962 [82:33:07<5:15:51,  8.21s/it]
 94%|█████████▍| 35656/37962 [82:33:16<5:16:16,  8.23s/it]
 94%|█████████▍| 35657/37962 [82:33:23<5:12:08,  8.13s/it]
 94%|█████████▍| 35658/37962 [82:33:32<5:13:43,  8.17s/it]
 94%|█████████▍| 35659/37962 [82:33:40<5:14:05,  8.18s/it]
 94%|█████████▍| 35660/37962 [82:33:48<5:11:33,  8.12s/it]
                                                          
{'loss': 1.1766, 'grad_norm': 10.675925254821777, 'learning_rate': 1.9224083164224304e-07, 'epoch': 0.94}

 94%|█████████▍| 35660/37962 [82:33:48<5:11:33,  8.12s/it]
 94%|█████████▍| 35661/37962 [82:33:56<5:13:31,  8.18s/it]
 94%|█████████▍| 35662/37962 [82:34:04<5:14:40,  8.21s/it]
 94%|█████████▍| 35663/37962 [82:34:13<5:14:33,  8.21s/it]
 94%|█████████▍| 35664/37962 [82:34:21<5:12:52,  8.17s/it]
 94%|█████████▍| 35665/37962 [82:34:29<5:11:48,  8.14s/it]
 94%|█████████▍| 35666/37962 [82:34:37<5:13:50,  8.20s/it]
 94%|█████████▍| 35667/37962 [82:34:45<5:14:41,  8.23s/it]
 94%|█████████▍| 35668/37962 [82:34:54<5:15:39,  8.26s/it]
 94%|█████████▍| 35669/37962 [82:35:02<5:16:46,  8.29s/it]
 94%|█████████▍| 35670/37962 [82:35:11<5:18:00,  8.32s/it]
                                                          
{'loss': 1.0874, 'grad_norm': 10.443366050720215, 'learning_rate': 1.9057956629688456e-07, 'epoch': 0.94}

 94%|█████████▍| 35670/37962 [82:35:11<5:18:00,  8.32s/it]
 94%|█████████▍| 35671/37962 [82:35:19<5:18:29,  8.34s/it]
 94%|█████████▍| 35672/37962 [82:35:27<5:18:11,  8.34s/it]
 94%|█████████▍| 35673/37962 [82:35:35<5:14:19,  8.24s/it]
 94%|█████████▍| 35674/37962 [82:35:43<5:13:59,  8.23s/it]
 94%|█████████▍| 35675/37962 [82:35:52<5:14:14,  8.24s/it]
 94%|█████████▍| 35676/37962 [82:36:00<5:16:29,  8.31s/it]
 94%|█████████▍| 35677/37962 [82:36:08<5:13:18,  8.23s/it]
 94%|█████████▍| 35678/37962 [82:36:17<5:14:15,  8.26s/it]
 94%|█████████▍| 35679/37962 [82:36:25<5:16:23,  8.32s/it]
 94%|█████████▍| 35680/37962 [82:36:33<5:15:36,  8.30s/it]
                                                          
{'loss': 1.2272, 'grad_norm': 11.058358192443848, 'learning_rate': 1.8892544105872378e-07, 'epoch': 0.94}

 94%|█████████▍| 35680/37962 [82:36:33<5:15:36,  8.30s/it]
 94%|█████████▍| 35681/37962 [82:36:41<5:13:19,  8.24s/it]
 94%|█████████▍| 35682/37962 [82:36:49<5:10:12,  8.16s/it]
 94%|█████████▍| 35683/37962 [82:36:58<5:11:29,  8.20s/it]
 94%|█████████▍| 35684/37962 [82:37:06<5:12:19,  8.23s/it]
 94%|█████████▍| 35685/37962 [82:37:14<5:13:20,  8.26s/it]
 94%|█████████▍| 35686/37962 [82:37:22<5:11:58,  8.22s/it]
 94%|█████████▍| 35687/37962 [82:37:31<5:13:07,  8.26s/it]
 94%|█████████▍| 35688/37962 [82:37:39<5:16:19,  8.35s/it]
 94%|█████████▍| 35689/37962 [82:37:47<5:09:56,  8.18s/it]
 94%|█████████▍| 35690/37962 [82:37:55<5:10:33,  8.20s/it]
                                                          
{'loss': 1.1955, 'grad_norm': 11.761453628540039, 'learning_rate': 1.8727845713176985e-07, 'epoch': 0.94}

 94%|█████████▍| 35690/37962 [82:37:55<5:10:33,  8.20s/it]
 94%|█████████▍| 35691/37962 [82:38:04<5:11:43,  8.24s/it]
 94%|█████████▍| 35692/37962 [82:38:12<5:12:25,  8.26s/it]
 94%|█████████▍| 35693/37962 [82:38:21<5:15:40,  8.35s/it]
 94%|█████████▍| 35694/37962 [82:38:29<5:11:32,  8.24s/it]
 94%|█████████▍| 35695/37962 [82:38:37<5:10:38,  8.22s/it]
 94%|█████████▍| 35696/37962 [82:38:45<5:08:25,  8.17s/it]
 94%|█████████▍| 35697/37962 [82:38:53<5:10:15,  8.22s/it]
 94%|█████████▍| 35698/37962 [82:39:01<5:10:18,  8.22s/it]
 94%|█████████▍| 35699/37962 [82:39:10<5:10:34,  8.23s/it]
 94%|█████████▍| 35700/37962 [82:39:18<5:12:51,  8.30s/it]
                                                          
{'loss': 1.1088, 'grad_norm': 7.073545932769775, 'learning_rate': 1.8563861571483267e-07, 'epoch': 0.94}

 94%|█████████▍| 35700/37962 [82:39:18<5:12:51,  8.30s/it]
 94%|█████████▍| 35701/37962 [82:39:26<5:10:05,  8.23s/it]
 94%|█████████▍| 35702/37962 [82:39:34<5:11:03,  8.26s/it]
 94%|█████████▍| 35703/37962 [82:39:43<5:11:46,  8.28s/it]
 94%|█████████▍| 35704/37962 [82:39:51<5:10:31,  8.25s/it]
 94%|█████████▍| 35705/37962 [82:39:59<5:11:58,  8.29s/it]
 94%|█████████▍| 35706/37962 [82:40:07<5:08:18,  8.20s/it]
 94%|█████████▍| 35707/37962 [82:40:16<5:08:39,  8.21s/it]
 94%|█████████▍| 35708/37962 [82:40:24<5:06:51,  8.17s/it]
 94%|█████████▍| 35709/37962 [82:40:32<5:10:07,  8.26s/it]
 94%|█████████▍| 35710/37962 [82:40:40<5:09:44,  8.25s/it]
                                                          
{'loss': 1.2169, 'grad_norm': 11.718839645385742, 'learning_rate': 1.8400591800152522e-07, 'epoch': 0.94}

 94%|█████████▍| 35710/37962 [82:40:40<5:09:44,  8.25s/it]
 94%|█████████▍| 35711/37962 [82:40:49<5:09:49,  8.26s/it]
 94%|█████████▍| 35712/37962 [82:40:57<5:09:09,  8.24s/it]
 94%|█████████▍| 35713/37962 [82:41:05<5:10:02,  8.27s/it]
 94%|█████████▍| 35714/37962 [82:41:13<5:10:29,  8.29s/it]
 94%|█████████▍| 35715/37962 [82:41:22<5:11:42,  8.32s/it]
 94%|█████████▍| 35716/37962 [82:41:30<5:11:39,  8.33s/it]
 94%|█████████▍| 35717/37962 [82:41:39<5:14:26,  8.40s/it]
 94%|█████████▍| 35718/37962 [82:41:47<5:15:16,  8.43s/it]
 94%|█████████▍| 35719/37962 [82:41:56<5:14:03,  8.40s/it]
 94%|█████████▍| 35720/37962 [82:42:04<5:11:32,  8.34s/it]
                                                          
{'loss': 1.1526, 'grad_norm': 10.538962364196777, 'learning_rate': 1.8238036518026025e-07, 'epoch': 0.94}

 94%|█████████▍| 35720/37962 [82:42:04<5:11:32,  8.34s/it]
 94%|█████████▍| 35721/37962 [82:42:12<5:12:43,  8.37s/it]
 94%|█████████▍| 35722/37962 [82:42:20<5:10:51,  8.33s/it]
 94%|█████████▍| 35723/37962 [82:42:29<5:11:49,  8.36s/it]
 94%|█████████▍| 35724/37962 [82:42:37<5:10:10,  8.32s/it]
 94%|█████████▍| 35725/37962 [82:42:45<5:09:52,  8.31s/it]
 94%|█████████▍| 35726/37962 [82:42:54<5:11:51,  8.37s/it]
 94%|█████████▍| 35727/37962 [82:43:02<5:13:23,  8.41s/it]
 94%|█████████▍| 35728/37962 [82:43:11<5:11:25,  8.36s/it]
 94%|█████████▍| 35729/37962 [82:43:27<6:34:11, 10.59s/it]
 94%|█████████▍| 35730/37962 [82:43:35<6:08:50,  9.92s/it]
                                                          
{'loss': 1.1415, 'grad_norm': 9.23486042022705, 'learning_rate': 1.8076195843424793e-07, 'epoch': 0.94}

 94%|█████████▍| 35730/37962 [82:43:35<6:08:50,  9.92s/it]
 94%|█████████▍| 35731/37962 [82:43:43<5:49:49,  9.41s/it]
 94%|█████████▍| 35732/37962 [82:43:51<5:37:01,  9.07s/it]
 94%|█████████▍| 35733/37962 [82:43:59<5:24:23,  8.73s/it]
 94%|█████████▍| 35734/37962 [82:44:08<5:19:34,  8.61s/it]
 94%|█████████▍| 35735/37962 [82:44:16<5:17:57,  8.57s/it]
 94%|█████████▍| 35736/37962 [82:44:24<5:14:06,  8.47s/it]
 94%|█████████▍| 35737/37962 [82:44:33<5:12:53,  8.44s/it]
 94%|█████████▍| 35738/37962 [82:44:41<5:10:20,  8.37s/it]
 94%|█████████▍| 35739/37962 [82:44:49<5:10:46,  8.39s/it]
 94%|█████████▍| 35740/37962 [82:44:58<5:08:28,  8.33s/it]
                                                          
{'loss': 1.1478, 'grad_norm': 8.318564414978027, 'learning_rate': 1.7915069894149928e-07, 'epoch': 0.94}

 94%|█████████▍| 35740/37962 [82:44:58<5:08:28,  8.33s/it]
 94%|█████████▍| 35741/37962 [82:45:05<5:00:49,  8.13s/it]
 94%|█████████▍| 35742/37962 [82:45:13<5:02:18,  8.17s/it]
 94%|█████████▍| 35743/37962 [82:45:22<5:05:40,  8.27s/it]
 94%|█████████▍| 35744/37962 [82:45:30<5:04:45,  8.24s/it]
 94%|█████████▍| 35745/37962 [82:45:38<5:04:13,  8.23s/it]
 94%|█████████▍| 35746/37962 [82:45:47<5:06:31,  8.30s/it]
 94%|█████████▍| 35747/37962 [82:45:55<5:06:10,  8.29s/it]
 94%|█████████▍| 35748/37962 [82:46:03<5:06:04,  8.29s/it]
 94%|█████████▍| 35749/37962 [82:46:12<5:05:03,  8.27s/it]
 94%|█████████▍| 35750/37962 [82:46:20<5:06:22,  8.31s/it]
                                                          
{'loss': 1.1524, 'grad_norm': 7.397567272186279, 'learning_rate': 1.775465878748228e-07, 'epoch': 0.94}

 94%|█████████▍| 35750/37962 [82:46:20<5:06:22,  8.31s/it]
 94%|█████████▍| 35751/37962 [82:46:28<5:04:52,  8.27s/it]
 94%|█████████▍| 35752/37962 [82:46:36<5:03:53,  8.25s/it]
 94%|█████████▍| 35753/37962 [82:46:45<5:04:13,  8.26s/it]
 94%|█████████▍| 35754/37962 [82:46:53<5:07:49,  8.36s/it]
 94%|█████████▍| 35755/37962 [82:47:02<5:07:33,  8.36s/it]
 94%|█████████▍| 35756/37962 [82:47:10<5:09:29,  8.42s/it]
 94%|█████████▍| 35757/37962 [82:47:19<5:08:37,  8.40s/it]
 94%|█████████▍| 35758/37962 [82:47:27<5:08:56,  8.41s/it]
 94%|█████████▍| 35759/37962 [82:47:35<5:08:37,  8.41s/it]
 94%|█████████▍| 35760/37962 [82:47:43<5:05:05,  8.31s/it]
                                                          
{'loss': 1.172, 'grad_norm': 7.76917839050293, 'learning_rate': 1.7594962640182235e-07, 'epoch': 0.94}

 94%|█████████▍| 35760/37962 [82:47:43<5:05:05,  8.31s/it]
 94%|█████████▍| 35761/37962 [82:47:51<4:59:11,  8.16s/it]
 94%|█████████▍| 35762/37962 [82:47:59<4:59:50,  8.18s/it]
 94%|█████████▍| 35763/37962 [82:48:08<5:01:53,  8.24s/it]
 94%|█████████▍| 35764/37962 [82:48:16<5:01:50,  8.24s/it]
 94%|█████████▍| 35765/37962 [82:48:24<5:00:28,  8.21s/it]
 94%|█████████▍| 35766/37962 [82:48:32<4:59:59,  8.20s/it]
 94%|█████████▍| 35767/37962 [82:48:41<5:02:34,  8.27s/it]
 94%|█████████▍| 35768/37962 [82:48:49<5:01:52,  8.26s/it]
 94%|█████████▍| 35769/37962 [82:48:57<5:02:05,  8.27s/it]
 94%|█████████▍| 35770/37962 [82:49:06<5:02:25,  8.28s/it]
                                                          
{'loss': 1.1116, 'grad_norm': 9.835089683532715, 'learning_rate': 1.743598156848991e-07, 'epoch': 0.94}

 94%|█████████▍| 35770/37962 [82:49:06<5:02:25,  8.28s/it]
 94%|█████████▍| 35771/37962 [82:49:14<5:03:11,  8.30s/it]
 94%|█████████▍| 35772/37962 [82:49:22<5:00:11,  8.22s/it]
 94%|█████████▍| 35773/37962 [82:49:30<5:01:15,  8.26s/it]
 94%|█████████▍| 35774/37962 [82:49:39<5:01:48,  8.28s/it]
 94%|█████████▍| 35775/37962 [82:49:47<5:02:08,  8.29s/it]
 94%|█████████▍| 35776/37962 [82:49:55<5:00:17,  8.24s/it]
 94%|█████████▍| 35777/37962 [82:50:03<5:01:10,  8.27s/it]
 94%|█████████▍| 35778/37962 [82:50:12<5:01:59,  8.30s/it]
 94%|█████████▍| 35779/37962 [82:50:20<5:02:04,  8.30s/it]
 94%|█████████▍| 35780/37962 [82:50:29<5:05:23,  8.40s/it]
                                                          
{'loss': 1.12, 'grad_norm': 11.257411003112793, 'learning_rate': 1.7277715688124863e-07, 'epoch': 0.94}

 94%|█████████▍| 35780/37962 [82:50:29<5:05:23,  8.40s/it]
 94%|█████████▍| 35781/37962 [82:50:37<5:03:55,  8.36s/it]
 94%|█████████▍| 35782/37962 [82:50:45<5:02:31,  8.33s/it]
 94%|█████████▍| 35783/37962 [82:50:54<5:01:15,  8.30s/it]
 94%|█████████▍| 35784/37962 [82:51:02<5:01:42,  8.31s/it]
 94%|█████████▍| 35785/37962 [82:51:10<5:01:52,  8.32s/it]
 94%|█████████▍| 35786/37962 [82:51:18<4:59:29,  8.26s/it]
 94%|█████████▍| 35787/37962 [82:51:26<4:56:31,  8.18s/it]
 94%|█████████▍| 35788/37962 [82:51:35<4:57:56,  8.22s/it]
 94%|█████████▍| 35789/37962 [82:51:43<4:59:23,  8.27s/it]
 94%|█████████▍| 35790/37962 [82:51:51<4:59:02,  8.26s/it]
                                                          
{'loss': 1.0766, 'grad_norm': 5.076475620269775, 'learning_rate': 1.7120165114285936e-07, 'epoch': 0.94}

 94%|█████████▍| 35790/37962 [82:51:51<4:59:02,  8.26s/it]
 94%|█████████▍| 35791/37962 [82:51:59<4:57:22,  8.22s/it]
 94%|█████████▍| 35792/37962 [82:52:08<5:01:28,  8.34s/it]
 94%|█████████▍| 35793/37962 [82:52:16<5:01:05,  8.33s/it]
 94%|█████████▍| 35794/37962 [82:52:24<4:59:09,  8.28s/it]
 94%|█████████▍| 35795/37962 [82:52:33<4:59:30,  8.29s/it]
 94%|█████████▍| 35796/37962 [82:52:41<4:57:10,  8.23s/it]
 94%|█████████▍| 35797/37962 [82:52:49<4:58:35,  8.28s/it]
 94%|█████████▍| 35798/37962 [82:52:57<4:56:00,  8.21s/it]
 94%|█████████▍| 35799/37962 [82:53:06<4:59:03,  8.30s/it]
 94%|█████████▍| 35800/37962 [82:53:14<5:00:12,  8.33s/it]
                                                          
{'loss': 1.138, 'grad_norm': 6.8095245361328125, 'learning_rate': 1.6963329961651732e-07, 'epoch': 0.94}

 94%|█████████▍| 35800/37962 [82:53:14<5:00:12,  8.33s/it]
 94%|█████████▍| 35801/37962 [82:53:23<5:00:31,  8.34s/it]
 94%|█████████▍| 35802/37962 [82:53:31<4:56:11,  8.23s/it]
 94%|█████████▍| 35803/37962 [82:53:39<4:57:04,  8.26s/it]
 94%|█████████▍| 35804/37962 [82:53:47<4:57:39,  8.28s/it]
 94%|█████████▍| 35805/37962 [82:53:55<4:54:42,  8.20s/it]
 94%|█████████▍| 35806/37962 [82:54:03<4:55:03,  8.21s/it]
 94%|█████████▍| 35807/37962 [82:54:12<4:56:26,  8.25s/it]
 94%|█████████▍| 35808/37962 [82:54:20<4:56:33,  8.26s/it]
 94%|█████████▍| 35809/37962 [82:54:28<4:57:12,  8.28s/it]
 94%|█████████▍| 35810/37962 [82:54:37<4:57:23,  8.29s/it]
                                                          
{'loss': 1.1102, 'grad_norm': 15.803105354309082, 'learning_rate': 1.680721034437971e-07, 'epoch': 0.94}

 94%|█████████▍| 35810/37962 [82:54:37<4:57:23,  8.29s/it]
 94%|█████████▍| 35811/37962 [82:54:45<4:54:51,  8.22s/it]
 94%|█████████▍| 35812/37962 [82:54:53<4:58:45,  8.34s/it]
 94%|█████████▍| 35813/37962 [82:55:02<4:58:08,  8.32s/it]
 94%|█████████▍| 35814/37962 [82:55:10<4:57:48,  8.32s/it]
 94%|█████████▍| 35815/37962 [82:55:18<4:58:01,  8.33s/it]
 94%|█████████▍| 35816/37962 [82:55:27<4:57:13,  8.31s/it]
 94%|█████████▍| 35817/37962 [82:55:35<4:57:10,  8.31s/it]
 94%|█████████▍| 35818/37962 [82:55:43<4:56:51,  8.31s/it]
 94%|█████████▍| 35819/37962 [82:55:52<4:56:27,  8.30s/it]
 94%|█████████▍| 35820/37962 [82:56:00<4:55:20,  8.27s/it]
                                                          
{'loss': 1.2321, 'grad_norm': 11.27157211303711, 'learning_rate': 1.6651806376106638e-07, 'epoch': 0.94}

 94%|█████████▍| 35820/37962 [82:56:00<4:55:20,  8.27s/it]
 94%|█████████▍| 35821/37962 [82:56:08<4:57:32,  8.34s/it]
 94%|█████████▍| 35822/37962 [82:56:16<4:55:39,  8.29s/it]
 94%|█████████▍| 35823/37962 [82:56:25<4:54:36,  8.26s/it]
 94%|█████████▍| 35824/37962 [82:56:33<4:55:08,  8.28s/it]
 94%|█████████▍| 35825/37962 [82:56:41<4:55:43,  8.30s/it]
 94%|█████████▍| 35826/37962 [82:56:49<4:51:06,  8.18s/it]
 94%|█████████▍| 35827/37962 [82:56:57<4:51:21,  8.19s/it]
 94%|█████████▍| 35828/37962 [82:57:06<4:51:16,  8.19s/it]
 94%|█████████▍| 35829/37962 [82:57:13<4:47:22,  8.08s/it]
 94%|█████████▍| 35830/37962 [82:57:21<4:47:10,  8.08s/it]
                                                          
{'loss': 1.1638, 'grad_norm': 10.862615585327148, 'learning_rate': 1.6497118169948367e-07, 'epoch': 0.94}

 94%|█████████▍| 35830/37962 [82:57:21<4:47:10,  8.08s/it]
 94%|█████████▍| 35831/37962 [82:57:30<4:49:20,  8.15s/it]
 94%|█████████▍| 35832/37962 [82:57:38<4:50:02,  8.17s/it]
 94%|█████████▍| 35833/37962 [82:57:46<4:50:37,  8.19s/it]
 94%|█████████▍| 35834/37962 [82:57:54<4:50:54,  8.20s/it]
 94%|█████████▍| 35835/37962 [82:58:03<4:51:19,  8.22s/it]
 94%|█████████▍| 35836/37962 [82:58:11<4:52:27,  8.25s/it]
 94%|█████████▍| 35837/37962 [82:58:19<4:52:55,  8.27s/it]
 94%|█████████▍| 35838/37962 [82:58:28<4:54:02,  8.31s/it]
 94%|█████████▍| 35839/37962 [82:58:36<4:51:26,  8.24s/it]
 94%|█████████▍| 35840/37962 [82:58:44<4:52:17,  8.26s/it]
                                                          
{'loss': 1.1762, 'grad_norm': 7.417665004730225, 'learning_rate': 1.6343145838499609e-07, 'epoch': 0.94}

 94%|█████████▍| 35840/37962 [82:58:44<4:52:17,  8.26s/it]
 94%|█████████▍| 35841/37962 [82:58:53<4:53:17,  8.30s/it]
 94%|█████████▍| 35842/37962 [82:59:01<4:50:08,  8.21s/it]
 94%|█████████▍| 35843/37962 [82:59:09<4:50:40,  8.23s/it]
 94%|█████████▍| 35844/37962 [82:59:17<4:50:33,  8.23s/it]
 94%|█████████▍| 35845/37962 [82:59:25<4:50:51,  8.24s/it]
 94%|█████████▍| 35846/37962 [82:59:34<4:53:05,  8.31s/it]
 94%|█████████▍| 35847/37962 [82:59:42<4:53:35,  8.33s/it]
 94%|█████████▍| 35848/37962 [82:59:50<4:49:53,  8.23s/it]
 94%|█████████▍| 35849/37962 [82:59:58<4:49:17,  8.21s/it]
 94%|█████████▍| 35850/37962 [83:00:07<4:51:23,  8.28s/it]
                                                          
{'loss': 1.1525, 'grad_norm': 9.022090911865234, 'learning_rate': 1.6189889493834376e-07, 'epoch': 0.94}

 94%|█████████▍| 35850/37962 [83:00:07<4:51:23,  8.28s/it]
 94%|█████████▍| 35851/37962 [83:00:15<4:54:55,  8.38s/it]
 94%|█████████▍| 35852/37962 [83:00:23<4:51:49,  8.30s/it]
 94%|█████████▍| 35853/37962 [83:00:32<4:52:08,  8.31s/it]
 94%|█████████▍| 35854/37962 [83:00:40<4:51:06,  8.29s/it]
 94%|█████████▍| 35855/37962 [83:00:48<4:51:14,  8.29s/it]
 94%|█████████▍| 35856/37962 [83:00:57<4:50:07,  8.27s/it]
 94%|█████████▍| 35857/37962 [83:01:05<4:51:06,  8.30s/it]
 94%|█████████▍| 35858/37962 [83:01:13<4:50:58,  8.30s/it]
 94%|█████████▍| 35859/37962 [83:01:21<4:48:54,  8.24s/it]
 94%|█████████▍| 35860/37962 [83:01:30<4:48:57,  8.25s/it]
                                                          
{'loss': 1.2028, 'grad_norm': 8.349259376525879, 'learning_rate': 1.6037349247505328e-07, 'epoch': 0.94}

 94%|█████████▍| 35860/37962 [83:01:30<4:48:57,  8.25s/it]
 94%|█████████▍| 35861/37962 [83:01:38<4:51:03,  8.31s/it]
 94%|█████████▍| 35862/37962 [83:01:46<4:50:01,  8.29s/it]
 94%|█████████▍| 35863/37962 [83:01:54<4:48:43,  8.25s/it]
 94%|█████████▍| 35864/37962 [83:02:03<4:47:56,  8.23s/it]
 94%|█████████▍| 35865/37962 [83:02:11<4:49:59,  8.30s/it]
 94%|█████████▍| 35866/37962 [83:02:19<4:49:42,  8.29s/it]
 94%|█████████▍| 35867/37962 [83:02:28<4:50:55,  8.33s/it]
 94%|█████████▍| 35868/37962 [83:02:36<4:52:16,  8.37s/it]
 94%|█████████▍| 35869/37962 [83:02:45<4:50:22,  8.32s/it]
 94%|█████████▍| 35870/37962 [83:02:53<4:50:52,  8.34s/it]
                                                          
{'loss': 1.2098, 'grad_norm': 16.508569717407227, 'learning_rate': 1.5885525210543762e-07, 'epoch': 0.94}

 94%|█████████▍| 35870/37962 [83:02:53<4:50:52,  8.34s/it]
 94%|█████████▍| 35871/37962 [83:03:01<4:49:25,  8.30s/it]
 94%|█████████▍| 35872/37962 [83:03:09<4:48:51,  8.29s/it]
 94%|█████████▍| 35873/37962 [83:03:18<4:47:46,  8.27s/it]
 94%|█████████▍| 35874/37962 [83:03:26<4:48:14,  8.28s/it]
 95%|█████████▍| 35875/37962 [83:03:34<4:46:19,  8.23s/it]
 95%|█████████▍| 35876/37962 [83:03:42<4:48:41,  8.30s/it]
 95%|█████████▍| 35877/37962 [83:03:50<4:43:38,  8.16s/it]
 95%|█████████▍| 35878/37962 [83:03:59<4:44:55,  8.20s/it]
 95%|█████████▍| 35879/37962 [83:04:07<4:44:26,  8.19s/it]
 95%|█████████▍| 35880/37962 [83:04:15<4:45:49,  8.24s/it]
                                                          
{'loss': 1.0736, 'grad_norm': 11.990571022033691, 'learning_rate': 1.573441749345972e-07, 'epoch': 0.95}

 95%|█████████▍| 35880/37962 [83:04:15<4:45:49,  8.24s/it]
 95%|█████████▍| 35881/37962 [83:04:23<4:45:49,  8.24s/it]
 95%|█████████▍| 35882/37962 [83:04:32<4:47:36,  8.30s/it]
 95%|█████████▍| 35883/37962 [83:04:40<4:48:40,  8.33s/it]
 95%|█████████▍| 35884/37962 [83:04:48<4:47:56,  8.31s/it]
 95%|█████████▍| 35885/37962 [83:04:57<4:49:44,  8.37s/it]
 95%|█████████▍| 35886/37962 [83:05:05<4:50:31,  8.40s/it]
 95%|█████████▍| 35887/37962 [83:05:14<4:49:47,  8.38s/it]
 95%|█████████▍| 35888/37962 [83:05:22<4:45:29,  8.26s/it]
 95%|█████████▍| 35889/37962 [83:05:30<4:42:04,  8.16s/it]
 95%|█████████▍| 35890/37962 [83:05:38<4:43:17,  8.20s/it]
                                                          
{'loss': 1.1627, 'grad_norm': 12.153646469116211, 'learning_rate': 1.5584026206241997e-07, 'epoch': 0.95}

 95%|█████████▍| 35890/37962 [83:05:38<4:43:17,  8.20s/it]
 95%|█████████▍| 35891/37962 [83:05:47<4:46:14,  8.29s/it]
 95%|█████████▍| 35892/37962 [83:05:54<4:41:58,  8.17s/it]
 95%|█████████▍| 35893/37962 [83:06:03<4:45:29,  8.28s/it]
 95%|█████████▍| 35894/37962 [83:06:11<4:44:58,  8.27s/it]
 95%|█████████▍| 35895/37962 [83:06:19<4:39:50,  8.12s/it]
 95%|█████████▍| 35896/37962 [83:06:27<4:41:23,  8.17s/it]
 95%|█████████▍| 35897/37962 [83:06:35<4:41:11,  8.17s/it]
 95%|█████████▍| 35898/37962 [83:06:44<4:40:51,  8.16s/it]
 95%|█████████▍| 35899/37962 [83:06:51<4:38:24,  8.10s/it]
 95%|█████████▍| 35900/37962 [83:07:00<4:40:00,  8.15s/it]
                                                          
{'loss': 1.1352, 'grad_norm': 19.36845588684082, 'learning_rate': 1.5434351458357588e-07, 'epoch': 0.95}

 95%|█████████▍| 35900/37962 [83:07:00<4:40:00,  8.15s/it]
 95%|█████████▍| 35901/37962 [83:07:08<4:39:01,  8.12s/it]
 95%|█████████▍| 35902/37962 [83:07:16<4:41:00,  8.18s/it]
 95%|█████████▍| 35903/37962 [83:07:24<4:41:34,  8.21s/it]
 95%|█████████▍| 35904/37962 [83:07:33<4:44:03,  8.28s/it]
 95%|█████████▍| 35905/37962 [83:07:41<4:43:52,  8.28s/it]
 95%|█████████▍| 35906/37962 [83:07:49<4:42:29,  8.24s/it]
 95%|█████████▍| 35907/37962 [83:07:57<4:41:35,  8.22s/it]
 95%|█████████▍| 35908/37962 [83:08:06<4:41:32,  8.22s/it]
 95%|█████████▍| 35909/37962 [83:08:14<4:38:30,  8.14s/it]
 95%|█████████▍| 35910/37962 [83:08:22<4:41:08,  8.22s/it]
                                                          
{'loss': 1.1546, 'grad_norm': 11.167912483215332, 'learning_rate': 1.5285393358752566e-07, 'epoch': 0.95}

 95%|█████████▍| 35910/37962 [83:08:22<4:41:08,  8.22s/it]
 95%|█████████▍| 35911/37962 [83:08:30<4:42:21,  8.26s/it]
 95%|█████████▍| 35912/37962 [83:08:39<4:43:55,  8.31s/it]
 95%|█████████▍| 35913/37962 [83:08:47<4:41:59,  8.26s/it]
 95%|█████████▍| 35914/37962 [83:08:55<4:42:55,  8.29s/it]
 95%|█████████▍| 35915/37962 [83:09:04<4:42:13,  8.27s/it]
 95%|█████████▍| 35916/37962 [83:09:12<4:41:39,  8.26s/it]
 95%|█████████▍| 35917/37962 [83:09:20<4:37:45,  8.15s/it]
 95%|█████████▍| 35918/37962 [83:09:28<4:38:25,  8.17s/it]
 95%|█████████▍| 35919/37962 [83:09:36<4:41:47,  8.28s/it]
 95%|█████████▍| 35920/37962 [83:09:45<4:40:56,  8.26s/it]
                                                          
{'loss': 1.1473, 'grad_norm': 5.993119239807129, 'learning_rate': 1.5137152015850643e-07, 'epoch': 0.95}

 95%|█████████▍| 35920/37962 [83:09:45<4:40:56,  8.26s/it]
 95%|█████████▍| 35921/37962 [83:09:53<4:40:21,  8.24s/it]
 95%|█████████▍| 35922/37962 [83:10:01<4:40:16,  8.24s/it]
 95%|█████████▍| 35923/37962 [83:10:09<4:38:45,  8.20s/it]
 95%|█████████▍| 35924/37962 [83:10:18<4:39:49,  8.24s/it]
 95%|█████████▍| 35925/37962 [83:10:26<4:39:13,  8.22s/it]
 95%|█████████▍| 35926/37962 [83:10:34<4:42:32,  8.33s/it]
 95%|█████████▍| 35927/37962 [83:10:43<4:42:30,  8.33s/it]
 95%|█████████▍| 35928/37962 [83:10:51<4:44:29,  8.39s/it]
 95%|█████████▍| 35929/37962 [83:10:59<4:43:11,  8.36s/it]
 95%|█████████▍| 35930/37962 [83:11:08<4:43:35,  8.37s/it]
                                                          
{'loss': 1.0646, 'grad_norm': 9.531357765197754, 'learning_rate': 1.4989627537554063e-07, 'epoch': 0.95}

 95%|█████████▍| 35930/37962 [83:11:08<4:43:35,  8.37s/it]
 95%|█████████▍| 35931/37962 [83:11:16<4:41:00,  8.30s/it]
 95%|█████████▍| 35932/37962 [83:11:24<4:40:13,  8.28s/it]
 95%|█████████▍| 35933/37962 [83:11:33<4:42:03,  8.34s/it]
 95%|█████████▍| 35934/37962 [83:11:41<4:40:04,  8.29s/it]
 95%|█████████▍| 35935/37962 [83:11:49<4:41:22,  8.33s/it]
 95%|█████████▍| 35936/37962 [83:11:58<4:40:54,  8.32s/it]
 95%|█████████▍| 35937/37962 [83:12:06<4:39:40,  8.29s/it]
 95%|█████████▍| 35938/37962 [83:12:14<4:40:22,  8.31s/it]
 95%|█████████▍| 35939/37962 [83:12:22<4:36:16,  8.19s/it]
 95%|█████████▍| 35940/37962 [83:12:30<4:37:04,  8.22s/it]
                                                          
{'loss': 1.1948, 'grad_norm': 21.36967658996582, 'learning_rate': 1.4842820031243599e-07, 'epoch': 0.95}

 95%|█████████▍| 35940/37962 [83:12:30<4:37:04,  8.22s/it]
 95%|█████████▍| 35941/37962 [83:12:38<4:34:09,  8.14s/it]
 95%|█████████▍| 35942/37962 [83:12:46<4:31:21,  8.06s/it]
 95%|█████████▍| 35943/37962 [83:12:54<4:30:08,  8.03s/it]
 95%|█████████▍| 35944/37962 [83:13:03<4:33:29,  8.13s/it]
 95%|█████████▍| 35945/37962 [83:13:10<4:31:21,  8.07s/it]
 95%|█████████▍| 35946/37962 [83:13:19<4:31:36,  8.08s/it]
 95%|█████████▍| 35947/37962 [83:13:27<4:33:13,  8.14s/it]
 95%|█████████▍| 35948/37962 [83:13:35<4:37:10,  8.26s/it]
 95%|█████████▍| 35949/37962 [83:13:44<4:37:01,  8.26s/it]
 95%|█████████▍| 35950/37962 [83:13:52<4:35:17,  8.21s/it]
                                                          
{'loss': 1.0856, 'grad_norm': 15.26762580871582, 'learning_rate': 1.469672960377766e-07, 'epoch': 0.95}

 95%|█████████▍| 35950/37962 [83:13:52<4:35:17,  8.21s/it]
 95%|█████████▍| 35951/37962 [83:14:00<4:35:41,  8.23s/it]
 95%|█████████▍| 35952/37962 [83:14:08<4:35:36,  8.23s/it]
 95%|█████████▍| 35953/37962 [83:14:16<4:35:43,  8.23s/it]
 95%|█████████▍| 35954/37962 [83:14:25<4:34:58,  8.22s/it]
 95%|█████████▍| 35955/37962 [83:14:33<4:38:06,  8.31s/it]
 95%|█████████▍| 35956/37962 [83:14:41<4:34:28,  8.21s/it]
 95%|█████████▍| 35957/37962 [83:14:50<4:35:57,  8.26s/it]
 95%|█████████▍| 35958/37962 [83:14:58<4:34:52,  8.23s/it]
 95%|█████████▍| 35959/37962 [83:15:06<4:34:29,  8.22s/it]
 95%|█████████▍| 35960/37962 [83:15:14<4:36:22,  8.28s/it]
                                                          
{'loss': 1.1623, 'grad_norm': 11.183869361877441, 'learning_rate': 1.4551356361493074e-07, 'epoch': 0.95}

 95%|█████████▍| 35960/37962 [83:15:14<4:36:22,  8.28s/it]
 95%|█████████▍| 35961/37962 [83:15:23<4:36:24,  8.29s/it]
 95%|█████████▍| 35962/37962 [83:15:31<4:33:49,  8.21s/it]
 95%|█████████▍| 35963/37962 [83:15:39<4:32:33,  8.18s/it]
 95%|█████████▍| 35964/37962 [83:15:47<4:32:55,  8.20s/it]
 95%|█████████▍| 35965/37962 [83:15:55<4:33:13,  8.21s/it]
 95%|█████████▍| 35966/37962 [83:16:04<4:35:22,  8.28s/it]
 95%|█████████▍| 35967/37962 [83:16:12<4:31:25,  8.16s/it]
 95%|█████████▍| 35968/37962 [83:16:20<4:32:28,  8.20s/it]
 95%|█████████▍| 35969/37962 [83:16:28<4:31:21,  8.17s/it]
 95%|█████████▍| 35970/37962 [83:16:36<4:29:53,  8.13s/it]
                                                          
{'loss': 1.0568, 'grad_norm': 9.264765739440918, 'learning_rate': 1.4406700410204532e-07, 'epoch': 0.95}

 95%|█████████▍| 35970/37962 [83:16:36<4:29:53,  8.13s/it]
 95%|█████████▍| 35971/37962 [83:16:44<4:29:49,  8.13s/it]
 95%|█████████▍| 35972/37962 [83:16:52<4:30:55,  8.17s/it]
 95%|█████████▍| 35973/37962 [83:17:01<4:32:23,  8.22s/it]
 95%|█████████▍| 35974/37962 [83:17:09<4:35:18,  8.31s/it]
 95%|█████████▍| 35975/37962 [83:17:17<4:34:06,  8.28s/it]
 95%|█████████▍| 35976/37962 [83:17:26<4:32:37,  8.24s/it]
 95%|█████████▍| 35977/37962 [83:17:34<4:34:07,  8.29s/it]
 95%|█████████▍| 35978/37962 [83:17:42<4:31:30,  8.21s/it]
 95%|█████████▍| 35979/37962 [83:17:50<4:28:38,  8.13s/it]
 95%|█████████▍| 35980/37962 [83:17:58<4:30:00,  8.17s/it]
                                                          
{'loss': 1.0574, 'grad_norm': 10.81629467010498, 'learning_rate': 1.4262761855204477e-07, 'epoch': 0.95}

 95%|█████████▍| 35980/37962 [83:17:58<4:30:00,  8.17s/it]
 95%|█████████▍| 35981/37962 [83:18:07<4:32:34,  8.26s/it]
 95%|█████████▍| 35982/37962 [83:18:15<4:30:23,  8.19s/it]
 95%|█████████▍| 35983/37962 [83:18:23<4:26:41,  8.09s/it]
 95%|█████████▍| 35984/37962 [83:18:31<4:25:56,  8.07s/it]
 95%|█████████▍| 35985/37962 [83:18:39<4:25:35,  8.06s/it]
 95%|█████████▍| 35986/37962 [83:18:47<4:28:02,  8.14s/it]
 95%|█████████▍| 35987/37962 [83:18:55<4:26:42,  8.10s/it]
 95%|█████████▍| 35988/37962 [83:19:03<4:28:45,  8.17s/it]
 95%|█████████▍| 35989/37962 [83:19:11<4:28:43,  8.17s/it]
 95%|█████████▍| 35990/37962 [83:19:20<4:29:01,  8.19s/it]
                                                          
{'loss': 1.159, 'grad_norm': 10.056268692016602, 'learning_rate': 1.411954080126321e-07, 'epoch': 0.95}

 95%|█████████▍| 35990/37962 [83:19:20<4:29:01,  8.19s/it]
 95%|█████████▍| 35991/37962 [83:19:28<4:31:32,  8.27s/it]
 95%|█████████▍| 35992/37962 [83:19:36<4:30:43,  8.25s/it]
 95%|█████████▍| 35993/37962 [83:19:45<4:30:31,  8.24s/it]
 95%|█████████▍| 35994/37962 [83:19:53<4:32:06,  8.30s/it]
 95%|█████████▍| 35995/37962 [83:20:01<4:33:53,  8.35s/it]
 95%|█████████▍| 35996/37962 [83:20:10<4:32:26,  8.31s/it]
 95%|█████████▍| 35997/37962 [83:20:26<5:48:10, 10.63s/it]
 95%|█████████▍| 35998/37962 [83:20:34<5:25:44,  9.95s/it]
 95%|█████████▍| 35999/37962 [83:20:42<5:08:56,  9.44s/it]
 95%|█████████▍| 36000/37962 [83:20:51<4:57:39,  9.10s/it]
                                                          
{'loss': 1.2299, 'grad_norm': 12.725666046142578, 'learning_rate': 1.3977037352629118e-07, 'epoch': 0.95}

 95%|█████████▍| 36000/37962 [83:20:51<4:57:39,  9.10s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 95%|█████████▍| 36001/37962 [83:21:42<11:49:33, 21.71s/it]
 95%|█████████▍| 36002/37962 [83:21:50<9:39:42, 17.75s/it] 
 95%|█████████▍| 36003/37962 [83:21:59<8:08:44, 14.97s/it]
 95%|█████████▍| 36004/37962 [83:22:07<7:02:42, 12.95s/it]
 95%|█████████▍| 36005/37962 [83:22:15<6:16:00, 11.53s/it]
 95%|█████████▍| 36006/37962 [83:22:23<5:43:25, 10.53s/it]
 95%|█████████▍| 36007/37962 [83:22:32<5:22:18,  9.89s/it]
 95%|█████████▍| 36008/37962 [83:22:40<5:06:29,  9.41s/it]
 95%|█████████▍| 36009/37962 [83:22:48<4:54:57,  9.06s/it]
 95%|█████████▍| 36010/37962 [83:22:57<4:47:51,  8.85s/it]
                                                          
{'loss': 1.1731, 'grad_norm': 10.038346290588379, 'learning_rate': 1.38352516130279e-07, 'epoch': 0.95}

 95%|█████████▍| 36010/37962 [83:22:57<4:47:51,  8.85s/it]
 95%|█████████▍| 36011/37962 [83:23:05<4:43:22,  8.71s/it]
 95%|█████████▍| 36012/37962 [83:23:13<4:32:32,  8.39s/it]
 95%|█████████▍| 36013/37962 [83:23:21<4:30:10,  8.32s/it]
 95%|█████████▍| 36014/37962 [83:23:29<4:29:07,  8.29s/it]
 95%|█████████▍| 36015/37962 [83:23:38<4:30:01,  8.32s/it]
 95%|█████████▍| 36016/37962 [83:23:46<4:30:53,  8.35s/it]
 95%|█████████▍| 36017/37962 [83:23:54<4:29:00,  8.30s/it]
 95%|█████████▍| 36018/37962 [83:24:02<4:29:01,  8.30s/it]
 95%|█████████▍| 36019/37962 [83:24:11<4:31:01,  8.37s/it]
 95%|█████████▍| 36020/37962 [83:24:19<4:29:44,  8.33s/it]
                                                          
{'loss': 1.1327, 'grad_norm': 9.279366493225098, 'learning_rate': 1.3694183685662776e-07, 'epoch': 0.95}

 95%|█████████▍| 36020/37962 [83:24:19<4:29:44,  8.33s/it]
 95%|█████████▍| 36021/37962 [83:24:27<4:28:32,  8.30s/it]
 95%|█████████▍| 36022/37962 [83:24:36<4:28:04,  8.29s/it]
 95%|█████████▍| 36023/37962 [83:24:44<4:28:39,  8.31s/it]
 95%|█████████▍| 36024/37962 [83:24:52<4:27:48,  8.29s/it]
 95%|█████████▍| 36025/37962 [83:25:01<4:28:53,  8.33s/it]
 95%|█████████▍| 36026/37962 [83:25:09<4:30:07,  8.37s/it]
 95%|█████████▍| 36027/37962 [83:25:17<4:28:02,  8.31s/it]
 95%|█████████▍| 36028/37962 [83:25:26<4:28:40,  8.34s/it]
 95%|█████████▍| 36029/37962 [83:25:34<4:28:16,  8.33s/it]
 95%|█████████▍| 36030/37962 [83:25:42<4:26:11,  8.27s/it]
                                                          
{'loss': 1.0968, 'grad_norm': 8.229722023010254, 'learning_rate': 1.3553833673214945e-07, 'epoch': 0.95}

 95%|█████████▍| 36030/37962 [83:25:42<4:26:11,  8.27s/it]
 95%|█████████▍| 36031/37962 [83:25:50<4:26:03,  8.27s/it]
 95%|█████████▍| 36032/37962 [83:25:59<4:26:13,  8.28s/it]
 95%|█████████▍| 36033/37962 [83:26:07<4:27:45,  8.33s/it]
 95%|█████████▍| 36034/37962 [83:26:15<4:26:33,  8.30s/it]
 95%|█████████▍| 36035/37962 [83:26:24<4:27:08,  8.32s/it]
 95%|█████████▍| 36036/37962 [83:26:32<4:26:50,  8.31s/it]
 95%|█████████▍| 36037/37962 [83:26:40<4:25:37,  8.28s/it]
 95%|█████████▍| 36038/37962 [83:26:48<4:24:38,  8.25s/it]
 95%|█████████▍| 36039/37962 [83:26:57<4:23:53,  8.23s/it]
 95%|█████████▍| 36040/37962 [83:27:05<4:25:54,  8.30s/it]
                                                          
{'loss': 1.164, 'grad_norm': 9.047844886779785, 'learning_rate': 1.3414201677842353e-07, 'epoch': 0.95}

 95%|█████████▍| 36040/37962 [83:27:05<4:25:54,  8.30s/it]
 95%|█████████▍| 36041/37962 [83:27:13<4:25:23,  8.29s/it]
 95%|█████████▍| 36042/37962 [83:27:21<4:22:56,  8.22s/it]
 95%|█████████▍| 36043/37962 [83:27:30<4:22:41,  8.21s/it]
 95%|█████████▍| 36044/37962 [83:27:38<4:24:15,  8.27s/it]
 95%|█████████▍| 36045/37962 [83:27:46<4:22:37,  8.22s/it]
 95%|█████████▍| 36046/37962 [83:27:54<4:23:10,  8.24s/it]
 95%|█████████▍| 36047/37962 [83:28:03<4:22:47,  8.23s/it]
 95%|█████████▍| 36048/37962 [83:28:11<4:22:53,  8.24s/it]
 95%|█████████▍| 36049/37962 [83:28:19<4:21:46,  8.21s/it]
 95%|█████████▍| 36050/37962 [83:28:27<4:22:54,  8.25s/it]
                                                          
{'loss': 1.1583, 'grad_norm': 20.195104598999023, 'learning_rate': 1.3275287801181037e-07, 'epoch': 0.95}

 95%|█████████▍| 36050/37962 [83:28:27<4:22:54,  8.25s/it]
 95%|█████████▍| 36051/37962 [83:28:36<4:24:33,  8.31s/it]
 95%|█████████▍| 36052/37962 [83:28:44<4:25:39,  8.35s/it]
 95%|█████████▍| 36053/37962 [83:28:53<4:24:55,  8.33s/it]
 95%|█████████▍| 36054/37962 [83:29:01<4:24:27,  8.32s/it]
 95%|█████████▍| 36055/37962 [83:29:09<4:25:23,  8.35s/it]
 95%|█████████▍| 36056/37962 [83:29:17<4:21:21,  8.23s/it]
 95%|█████████▍| 36057/37962 [83:29:26<4:22:35,  8.27s/it]
 95%|█████████▍| 36058/37962 [83:29:34<4:23:09,  8.29s/it]
 95%|█████████▍| 36059/37962 [83:29:42<4:21:33,  8.25s/it]
 95%|█████████▍| 36060/37962 [83:29:50<4:19:53,  8.20s/it]
                                                          
{'loss': 1.011, 'grad_norm': 7.857487201690674, 'learning_rate': 1.3137092144343887e-07, 'epoch': 0.95}

 95%|█████████▍| 36060/37962 [83:29:50<4:19:53,  8.20s/it]
 95%|█████████▍| 36061/37962 [83:29:59<4:21:45,  8.26s/it]
 95%|█████████▍| 36062/37962 [83:30:07<4:22:13,  8.28s/it]
 95%|█████████▍| 36063/37962 [83:30:15<4:19:36,  8.20s/it]
 95%|█████████▌| 36064/37962 [83:30:23<4:19:57,  8.22s/it]
 95%|█████████▌| 36065/37962 [83:30:32<4:21:32,  8.27s/it]
 95%|█████████▌| 36066/37962 [83:30:40<4:21:54,  8.29s/it]
 95%|█████████▌| 36067/37962 [83:30:48<4:21:04,  8.27s/it]
 95%|█████████▌| 36068/37962 [83:30:56<4:20:18,  8.25s/it]
 95%|█████████▌| 36069/37962 [83:31:05<4:21:27,  8.29s/it]
 95%|█████████▌| 36070/37962 [83:31:13<4:18:01,  8.18s/it]
                                                          
{'loss': 1.2103, 'grad_norm': 17.092037200927734, 'learning_rate': 1.2999614807921225e-07, 'epoch': 0.95}

 95%|█████████▌| 36070/37962 [83:31:13<4:18:01,  8.18s/it]
 95%|█████████▌| 36071/37962 [83:31:21<4:18:17,  8.20s/it]
 95%|█████████▌| 36072/37962 [83:31:29<4:18:45,  8.21s/it]
 95%|█████████▌| 36073/37962 [83:31:37<4:19:20,  8.24s/it]
 95%|█████████▌| 36074/37962 [83:31:46<4:20:56,  8.29s/it]
 95%|█████████▌| 36075/37962 [83:31:54<4:19:08,  8.24s/it]
 95%|█████████▌| 36076/37962 [83:32:02<4:19:04,  8.24s/it]
 95%|█████████▌| 36077/37962 [83:32:11<4:19:41,  8.27s/it]
 95%|█████████▌| 36078/37962 [83:32:19<4:19:44,  8.27s/it]
 95%|█████████▌| 36079/37962 [83:32:27<4:18:33,  8.24s/it]
 95%|█████████▌| 36080/37962 [83:32:35<4:19:17,  8.27s/it]
                                                          
{'loss': 1.0598, 'grad_norm': 9.902064323425293, 'learning_rate': 1.286285589198022e-07, 'epoch': 0.95}

 95%|█████████▌| 36080/37962 [83:32:35<4:19:17,  8.27s/it]
 95%|█████████▌| 36081/37962 [83:32:43<4:16:14,  8.17s/it]
 95%|█████████▌| 36082/37962 [83:32:52<4:17:06,  8.21s/it]
 95%|█████████▌| 36083/37962 [83:33:00<4:18:28,  8.25s/it]
 95%|█████████▌| 36084/37962 [83:33:08<4:19:00,  8.27s/it]
 95%|█████████▌| 36085/37962 [83:33:17<4:20:13,  8.32s/it]
 95%|█████████▌| 36086/37962 [83:33:25<4:18:20,  8.26s/it]
 95%|█████████▌| 36087/37962 [83:33:33<4:18:53,  8.28s/it]
 95%|█████████▌| 36088/37962 [83:33:41<4:17:55,  8.26s/it]
 95%|█████████▌| 36089/37962 [83:33:50<4:17:19,  8.24s/it]
 95%|█████████▌| 36090/37962 [83:33:58<4:17:09,  8.24s/it]
                                                          
{'loss': 1.132, 'grad_norm': 16.6181583404541, 'learning_rate': 1.272681549606547e-07, 'epoch': 0.95}

 95%|█████████▌| 36090/37962 [83:33:58<4:17:09,  8.24s/it]
 95%|█████████▌| 36091/37962 [83:34:06<4:17:56,  8.27s/it]
 95%|█████████▌| 36092/37962 [83:34:14<4:16:46,  8.24s/it]
 95%|█████████▌| 36093/37962 [83:34:22<4:09:30,  8.01s/it]
 95%|█████████▌| 36094/37962 [83:34:30<4:11:32,  8.08s/it]
 95%|█████████▌| 36095/37962 [83:34:38<4:14:19,  8.17s/it]
 95%|█████████▌| 36096/37962 [83:34:47<4:14:28,  8.18s/it]
 95%|█████████▌| 36097/37962 [83:34:55<4:13:15,  8.15s/it]
 95%|█████████▌| 36098/37962 [83:35:03<4:14:19,  8.19s/it]
 95%|█████████▌| 36099/37962 [83:35:11<4:16:51,  8.27s/it]
 95%|█████████▌| 36100/37962 [83:35:20<4:18:13,  8.32s/it]
                                                          
{'loss': 1.1785, 'grad_norm': 9.175812721252441, 'learning_rate': 1.2591493719198323e-07, 'epoch': 0.95}

 95%|█████████▌| 36100/37962 [83:35:20<4:18:13,  8.32s/it]
 95%|█████████▌| 36101/37962 [83:35:28<4:18:08,  8.32s/it]
 95%|█████████▌| 36102/37962 [83:35:36<4:17:50,  8.32s/it]
 95%|█████████▌| 36103/37962 [83:35:45<4:17:22,  8.31s/it]
 95%|█████████▌| 36104/37962 [83:35:53<4:16:15,  8.28s/it]
 95%|█████████▌| 36105/37962 [83:36:01<4:15:37,  8.26s/it]
 95%|█████████▌| 36106/37962 [83:36:09<4:14:45,  8.24s/it]
 95%|█████████▌| 36107/37962 [83:36:18<4:15:16,  8.26s/it]
 95%|█████████▌| 36108/37962 [83:36:26<4:15:43,  8.28s/it]
 95%|█████████▌| 36109/37962 [83:36:34<4:15:18,  8.27s/it]
 95%|█████████▌| 36110/37962 [83:36:42<4:15:05,  8.26s/it]
                                                          
{'loss': 1.148, 'grad_norm': 8.035114288330078, 'learning_rate': 1.245689065987732e-07, 'epoch': 0.95}

 95%|█████████▌| 36110/37962 [83:36:42<4:15:05,  8.26s/it]
 95%|█████████▌| 36111/37962 [83:36:50<4:12:44,  8.19s/it]
 95%|█████████▌| 36112/37962 [83:36:59<4:14:04,  8.24s/it]
 95%|█████████▌| 36113/37962 [83:37:07<4:12:31,  8.19s/it]
 95%|█████████▌| 36114/37962 [83:37:15<4:10:51,  8.14s/it]
 95%|█████████▌| 36115/37962 [83:37:23<4:10:34,  8.14s/it]
 95%|█████████▌| 36116/37962 [83:37:31<4:12:14,  8.20s/it]
 95%|█████████▌| 36117/37962 [83:37:40<4:12:19,  8.21s/it]
 95%|█████████▌| 36118/37962 [83:37:48<4:14:05,  8.27s/it]
 95%|█████████▌| 36119/37962 [83:37:56<4:11:38,  8.19s/it]
 95%|█████████▌| 36120/37962 [83:38:04<4:11:48,  8.20s/it]
                                                          
{'loss': 1.0255, 'grad_norm': 13.891192436218262, 'learning_rate': 1.2323006416077532e-07, 'epoch': 0.95}

 95%|█████████▌| 36120/37962 [83:38:04<4:11:48,  8.20s/it]
 95%|█████████▌| 36121/37962 [83:38:13<4:13:07,  8.25s/it]
 95%|█████████▌| 36122/37962 [83:38:21<4:14:48,  8.31s/it]
 95%|█████████▌| 36123/37962 [83:38:29<4:15:01,  8.32s/it]
 95%|█████████▌| 36124/37962 [83:38:38<4:16:28,  8.37s/it]
 95%|█████████▌| 36125/37962 [83:38:46<4:17:21,  8.41s/it]
 95%|█████████▌| 36126/37962 [83:38:55<4:16:35,  8.39s/it]
 95%|█████████▌| 36127/37962 [83:39:03<4:16:19,  8.38s/it]
 95%|█████████▌| 36128/37962 [83:39:11<4:15:48,  8.37s/it]
 95%|█████████▌| 36129/37962 [83:39:20<4:13:46,  8.31s/it]
 95%|█████████▌| 36130/37962 [83:39:28<4:13:45,  8.31s/it]
                                                          
{'loss': 1.181, 'grad_norm': 8.833988189697266, 'learning_rate': 1.218984108525112e-07, 'epoch': 0.95}

 95%|█████████▌| 36130/37962 [83:39:28<4:13:45,  8.31s/it]
 95%|█████████▌| 36131/37962 [83:39:36<4:14:58,  8.36s/it]
 95%|█████████▌| 36132/37962 [83:39:44<4:11:34,  8.25s/it]
 95%|█████████▌| 36133/37962 [83:39:53<4:12:43,  8.29s/it]
 95%|█████████▌| 36134/37962 [83:40:01<4:09:04,  8.18s/it]
 95%|█████████▌| 36135/37962 [83:40:09<4:09:21,  8.19s/it]
 95%|█████████▌| 36136/37962 [83:40:17<4:10:23,  8.23s/it]
 95%|█████████▌| 36137/37962 [83:40:26<4:10:55,  8.25s/it]
 95%|█████████▌| 36138/37962 [83:40:33<4:06:24,  8.11s/it]
 95%|█████████▌| 36139/37962 [83:40:42<4:07:42,  8.15s/it]
 95%|█████████▌| 36140/37962 [83:40:50<4:09:01,  8.20s/it]
                                                          
{'loss': 1.238, 'grad_norm': 8.799610137939453, 'learning_rate': 1.2057394764326657e-07, 'epoch': 0.95}

 95%|█████████▌| 36140/37962 [83:40:50<4:09:01,  8.20s/it]
 95%|█████████▌| 36141/37962 [83:40:58<4:09:15,  8.21s/it]
 95%|█████████▌| 36142/37962 [83:41:06<4:08:02,  8.18s/it]
 95%|█████████▌| 36143/37962 [83:41:14<4:08:07,  8.18s/it]
 95%|█████████▌| 36144/37962 [83:41:23<4:08:35,  8.20s/it]
 95%|█████████▌| 36145/37962 [83:41:31<4:09:38,  8.24s/it]
 95%|█████████▌| 36146/37962 [83:41:39<4:10:11,  8.27s/it]
 95%|█████████▌| 36147/37962 [83:41:47<4:06:01,  8.13s/it]
 95%|█████████▌| 36148/37962 [83:41:55<4:05:32,  8.12s/it]
 95%|█████████▌| 36149/37962 [83:42:03<4:02:43,  8.03s/it]
 95%|█████████▌| 36150/37962 [83:42:11<4:05:04,  8.12s/it]
                                                          
{'loss': 1.1547, 'grad_norm': 7.695852756500244, 'learning_rate': 1.1925667549709585e-07, 'epoch': 0.95}

 95%|█████████▌| 36150/37962 [83:42:11<4:05:04,  8.12s/it]
 95%|█████████▌| 36151/37962 [83:42:20<4:05:44,  8.14s/it]
 95%|█████████▌| 36152/37962 [83:42:28<4:06:25,  8.17s/it]
 95%|█████████▌| 36153/37962 [83:42:36<4:05:23,  8.14s/it]
 95%|█████████▌| 36154/37962 [83:42:44<4:08:03,  8.23s/it]
 95%|█████████▌| 36155/37962 [83:42:53<4:11:24,  8.35s/it]
 95%|█████████▌| 36156/37962 [83:43:01<4:09:50,  8.30s/it]
 95%|█████████▌| 36157/37962 [83:43:09<4:06:24,  8.19s/it]
 95%|█████████▌| 36158/37962 [83:43:17<4:04:03,  8.12s/it]
 95%|█████████▌| 36159/37962 [83:43:26<4:08:46,  8.28s/it]
 95%|█████████▌| 36160/37962 [83:43:34<4:06:19,  8.20s/it]
                                                          
{'loss': 1.111, 'grad_norm': 9.022981643676758, 'learning_rate': 1.1794659537281982e-07, 'epoch': 0.95}

 95%|█████████▌| 36160/37962 [83:43:34<4:06:19,  8.20s/it]
 95%|█████████▌| 36161/37962 [83:43:42<4:07:49,  8.26s/it]
 95%|█████████▌| 36162/37962 [83:43:50<4:05:08,  8.17s/it]
 95%|█████████▌| 36163/37962 [83:43:58<4:05:45,  8.20s/it]
 95%|█████████▌| 36164/37962 [83:44:07<4:07:02,  8.24s/it]
 95%|█████████▌| 36165/37962 [83:44:15<4:08:06,  8.28s/it]
 95%|█████████▌| 36166/37962 [83:44:23<4:07:19,  8.26s/it]
 95%|█████████▌| 36167/37962 [83:44:31<4:05:37,  8.21s/it]
 95%|█████████▌| 36168/37962 [83:44:40<4:06:10,  8.23s/it]
 95%|█████████▌| 36169/37962 [83:44:48<4:07:06,  8.27s/it]
 95%|█████████▌| 36170/37962 [83:44:56<4:07:08,  8.27s/it]
                                                          
{'loss': 1.0575, 'grad_norm': 5.551951885223389, 'learning_rate': 1.1664370822402127e-07, 'epoch': 0.95}

 95%|█████████▌| 36170/37962 [83:44:56<4:07:08,  8.27s/it]
 95%|█████████▌| 36171/37962 [83:45:04<4:06:12,  8.25s/it]
 95%|█████████▌| 36172/37962 [83:45:13<4:07:08,  8.28s/it]
 95%|█████████▌| 36173/37962 [83:45:21<4:06:20,  8.26s/it]
 95%|█████████▌| 36174/37962 [83:45:29<4:06:30,  8.27s/it]
 95%|█████████▌| 36175/37962 [83:45:38<4:07:16,  8.30s/it]
 95%|█████████▌| 36176/37962 [83:45:46<4:07:47,  8.32s/it]
 95%|█████████▌| 36177/37962 [83:45:54<4:07:20,  8.31s/it]
 95%|█████████▌| 36178/37962 [83:46:03<4:07:31,  8.32s/it]
 95%|█████████▌| 36179/37962 [83:46:11<4:07:20,  8.32s/it]
 95%|█████████▌| 36180/37962 [83:46:19<4:07:59,  8.35s/it]
                                                          
{'loss': 1.1397, 'grad_norm': 9.394152641296387, 'learning_rate': 1.1534801499905157e-07, 'epoch': 0.95}

 95%|█████████▌| 36180/37962 [83:46:19<4:07:59,  8.35s/it]
 95%|█████████▌| 36181/37962 [83:46:28<4:05:20,  8.27s/it]
 95%|█████████▌| 36182/37962 [83:46:36<4:06:03,  8.29s/it]
 95%|█████████▌| 36183/37962 [83:46:44<4:06:51,  8.33s/it]
 95%|█████████▌| 36184/37962 [83:46:53<4:06:23,  8.31s/it]
 95%|█████████▌| 36185/37962 [83:47:01<4:04:14,  8.25s/it]
 95%|█████████▌| 36186/37962 [83:47:09<4:04:11,  8.25s/it]
 95%|█████████▌| 36187/37962 [83:47:17<4:02:32,  8.20s/it]
 95%|█████████▌| 36188/37962 [83:47:25<4:03:08,  8.22s/it]
 95%|█████████▌| 36189/37962 [83:47:34<4:03:41,  8.25s/it]
 95%|█████████▌| 36190/37962 [83:47:42<4:03:17,  8.24s/it]
                                                          
{'loss': 1.2463, 'grad_norm': 8.241857528686523, 'learning_rate': 1.1405951664101967e-07, 'epoch': 0.95}

 95%|█████████▌| 36190/37962 [83:47:42<4:03:17,  8.24s/it]
 95%|█████████▌| 36191/37962 [83:47:50<4:03:13,  8.24s/it]
 95%|█████████▌| 36192/37962 [83:47:58<4:03:22,  8.25s/it]
 95%|█████████▌| 36193/37962 [83:48:07<4:02:53,  8.24s/it]
 95%|█████████▌| 36194/37962 [83:48:15<4:04:22,  8.29s/it]
 95%|█████████▌| 36195/37962 [83:48:23<4:06:13,  8.36s/it]
 95%|█████████▌| 36196/37962 [83:48:31<4:01:51,  8.22s/it]
 95%|█████████▌| 36197/37962 [83:48:40<4:03:11,  8.27s/it]
 95%|█████████▌| 36198/37962 [83:48:48<4:02:39,  8.25s/it]
 95%|█████████▌| 36199/37962 [83:48:56<4:02:41,  8.26s/it]
 95%|█████████▌| 36200/37962 [83:49:05<4:03:42,  8.30s/it]
                                                          
{'loss': 1.0424, 'grad_norm': 10.305855751037598, 'learning_rate': 1.1277821408780532e-07, 'epoch': 0.95}

 95%|█████████▌| 36200/37962 [83:49:05<4:03:42,  8.30s/it]
 95%|█████████▌| 36201/37962 [83:49:13<4:03:40,  8.30s/it]
 95%|█████████▌| 36202/37962 [83:49:21<4:02:33,  8.27s/it]
 95%|█████████▌| 36203/37962 [83:49:29<4:02:18,  8.27s/it]
 95%|█████████▌| 36204/37962 [83:49:38<4:02:00,  8.26s/it]
 95%|█████████▌| 36205/37962 [83:49:46<4:02:48,  8.29s/it]
 95%|█████████▌| 36206/37962 [83:49:54<4:03:27,  8.32s/it]
 95%|█████████▌| 36207/37962 [83:50:03<4:02:20,  8.29s/it]
 95%|█████████▌| 36208/37962 [83:50:11<4:01:20,  8.26s/it]
 95%|█████████▌| 36209/37962 [83:50:19<4:01:45,  8.27s/it]
 95%|█████████▌| 36210/37962 [83:50:27<3:58:39,  8.17s/it]
                                                          
{'loss': 1.2198, 'grad_norm': 7.444577693939209, 'learning_rate': 1.1150410827204361e-07, 'epoch': 0.95}

 95%|█████████▌| 36210/37962 [83:50:27<3:58:39,  8.17s/it]
 95%|█████████▌| 36211/37962 [83:50:35<3:57:22,  8.13s/it]
 95%|█████████▌| 36212/37962 [83:50:43<3:58:39,  8.18s/it]
 95%|█████████▌| 36213/37962 [83:50:52<4:01:22,  8.28s/it]
 95%|█████████▌| 36214/37962 [83:51:00<4:01:58,  8.31s/it]
 95%|█████████▌| 36215/37962 [83:51:09<4:03:36,  8.37s/it]
 95%|█████████▌| 36216/37962 [83:51:17<4:03:46,  8.38s/it]
 95%|█████████▌| 36217/37962 [83:51:25<4:03:00,  8.36s/it]
 95%|█████████▌| 36218/37962 [83:51:33<3:59:14,  8.23s/it]
 95%|█████████▌| 36219/37962 [83:51:42<4:00:14,  8.27s/it]
 95%|█████████▌| 36220/37962 [83:51:50<3:59:32,  8.25s/it]
                                                          
{'loss': 1.1199, 'grad_norm': 10.578062057495117, 'learning_rate': 1.1023720012113493e-07, 'epoch': 0.95}

 95%|█████████▌| 36220/37962 [83:51:50<3:59:32,  8.25s/it]
 95%|█████████▌| 36221/37962 [83:51:58<3:59:31,  8.25s/it]
 95%|█████████▌| 36222/37962 [83:52:07<4:00:54,  8.31s/it]
 95%|█████████▌| 36223/37962 [83:52:15<3:59:40,  8.27s/it]
 95%|█████████▌| 36224/37962 [83:52:23<3:59:28,  8.27s/it]
 95%|█████████▌| 36225/37962 [83:52:31<3:59:28,  8.27s/it]
 95%|█████████▌| 36226/37962 [83:52:40<4:00:44,  8.32s/it]
 95%|█████████▌| 36227/37962 [83:52:48<4:01:04,  8.34s/it]
 95%|█████████▌| 36228/37962 [83:52:56<3:57:21,  8.21s/it]
 95%|█████████▌| 36229/37962 [83:53:04<3:56:43,  8.20s/it]
 95%|█████████▌| 36230/37962 [83:53:13<3:57:34,  8.23s/it]
                                                          
{'loss': 1.2524, 'grad_norm': 8.359997749328613, 'learning_rate': 1.0897749055724049e-07, 'epoch': 0.95}

 95%|█████████▌| 36230/37962 [83:53:13<3:57:34,  8.23s/it]
 95%|█████████▌| 36231/37962 [83:53:21<3:56:29,  8.20s/it]
 95%|█████████▌| 36232/37962 [83:53:29<3:56:16,  8.19s/it]
 95%|█████████▌| 36233/37962 [83:53:37<3:56:36,  8.21s/it]
 95%|█████████▌| 36234/37962 [83:53:45<3:57:25,  8.24s/it]
 95%|█████████▌| 36235/37962 [83:53:54<3:55:58,  8.20s/it]
 95%|█████████▌| 36236/37962 [83:54:02<3:56:30,  8.22s/it]
 95%|█████████▌| 36237/37962 [83:54:10<3:56:54,  8.24s/it]
 95%|█████████▌| 36238/37962 [83:54:18<3:56:47,  8.24s/it]
 95%|█████████▌| 36239/37962 [83:54:27<3:56:27,  8.23s/it]
 95%|█████████▌| 36240/37962 [83:54:35<3:55:13,  8.20s/it]
                                                          
{'loss': 1.1977, 'grad_norm': 7.529768466949463, 'learning_rate': 1.0772498049728019e-07, 'epoch': 0.95}

 95%|█████████▌| 36240/37962 [83:54:35<3:55:13,  8.20s/it]
 95%|█████████▌| 36241/37962 [83:54:43<3:56:19,  8.24s/it]
 95%|█████████▌| 36242/37962 [83:54:51<3:55:49,  8.23s/it]
 95%|█████████▌| 36243/37962 [83:55:00<3:56:51,  8.27s/it]
 95%|█████████▌| 36244/37962 [83:55:08<3:56:25,  8.26s/it]
 95%|█████████▌| 36245/37962 [83:55:16<3:53:36,  8.16s/it]
 95%|█████████▌| 36246/37962 [83:55:24<3:53:39,  8.17s/it]
 95%|█████████▌| 36247/37962 [83:55:32<3:55:40,  8.25s/it]
 95%|█████████▌| 36248/37962 [83:55:41<3:55:42,  8.25s/it]
 95%|█████████▌| 36249/37962 [83:55:49<3:55:19,  8.24s/it]
 95%|█████████▌| 36250/37962 [83:55:57<3:53:59,  8.20s/it]
                                                          
{'loss': 1.1383, 'grad_norm': 8.476798057556152, 'learning_rate': 1.0647967085293476e-07, 'epoch': 0.95}

 95%|█████████▌| 36250/37962 [83:55:57<3:53:59,  8.20s/it]
 95%|█████████▌| 36251/37962 [83:56:05<3:51:45,  8.13s/it]
 95%|█████████▌| 36252/37962 [83:56:13<3:51:36,  8.13s/it]
 95%|█████████▌| 36253/37962 [83:56:21<3:53:46,  8.21s/it]
 96%|█████████▌| 36254/37962 [83:56:29<3:52:00,  8.15s/it]
 96%|█████████▌| 36255/37962 [83:56:38<3:53:39,  8.21s/it]
 96%|█████████▌| 36256/37962 [83:56:46<3:56:25,  8.32s/it]
 96%|█████████▌| 36257/37962 [83:56:55<3:55:34,  8.29s/it]
 96%|█████████▌| 36258/37962 [83:57:02<3:52:11,  8.18s/it]
 96%|█████████▌| 36259/37962 [83:57:11<3:50:44,  8.13s/it]
 96%|█████████▌| 36260/37962 [83:57:19<3:52:23,  8.19s/it]
                                                          
{'loss': 1.1233, 'grad_norm': 7.85594367980957, 'learning_rate': 1.0524156253064355e-07, 'epoch': 0.96}

 96%|█████████▌| 36260/37962 [83:57:19<3:52:23,  8.19s/it]
 96%|█████████▌| 36261/37962 [83:57:27<3:52:26,  8.20s/it]
 96%|█████████▌| 36262/37962 [83:57:35<3:52:42,  8.21s/it]
 96%|█████████▌| 36263/37962 [83:57:44<3:52:59,  8.23s/it]
 96%|█████████▌| 36264/37962 [83:57:52<3:55:16,  8.31s/it]
 96%|█████████▌| 36265/37962 [83:58:01<3:56:39,  8.37s/it]
 96%|█████████▌| 36266/37962 [83:58:09<3:56:31,  8.37s/it]
 96%|█████████▌| 36267/37962 [83:58:17<3:56:09,  8.36s/it]
 96%|█████████▌| 36268/37962 [83:58:26<3:58:55,  8.46s/it]
 96%|█████████▌| 36269/37962 [83:58:34<3:58:09,  8.44s/it]
 96%|█████████▌| 36270/37962 [83:58:43<3:57:08,  8.41s/it]
                                                          
{'loss': 1.1588, 'grad_norm': 12.828485488891602, 'learning_rate': 1.0401065643160236e-07, 'epoch': 0.96}

 96%|█████████▌| 36270/37962 [83:58:43<3:57:08,  8.41s/it]
 96%|█████████▌| 36271/37962 [83:58:51<3:57:08,  8.41s/it]
 96%|█████████▌| 36272/37962 [83:58:59<3:55:40,  8.37s/it]
 96%|█████████▌| 36273/37962 [83:59:08<3:55:27,  8.36s/it]
 96%|█████████▌| 36274/37962 [83:59:16<3:53:39,  8.31s/it]
 96%|█████████▌| 36275/37962 [83:59:24<3:50:26,  8.20s/it]
 96%|█████████▌| 36276/37962 [83:59:32<3:49:47,  8.18s/it]
 96%|█████████▌| 36277/37962 [83:59:40<3:46:46,  8.07s/it]
 96%|█████████▌| 36278/37962 [83:59:56<4:55:30, 10.53s/it]
 96%|█████████▌| 36279/37962 [84:00:04<4:35:24,  9.82s/it]
 96%|█████████▌| 36280/37962 [84:00:13<4:25:55,  9.49s/it]
                                                          
{'loss': 1.2898, 'grad_norm': 19.179515838623047, 'learning_rate': 1.0278695345177003e-07, 'epoch': 0.96}

 96%|█████████▌| 36280/37962 [84:00:13<4:25:55,  9.49s/it]
 96%|█████████▌| 36281/37962 [84:00:21<4:12:30,  9.01s/it]
 96%|█████████▌| 36282/37962 [84:00:29<4:05:43,  8.78s/it]
 96%|█████████▌| 36283/37962 [84:00:37<4:01:46,  8.64s/it]
 96%|█████████▌| 36284/37962 [84:00:46<3:58:25,  8.53s/it]
 96%|█████████▌| 36285/37962 [84:00:54<3:56:35,  8.46s/it]
 96%|█████████▌| 36286/37962 [84:01:03<3:57:23,  8.50s/it]
 96%|█████████▌| 36287/37962 [84:01:11<3:55:23,  8.43s/it]
 96%|█████████▌| 36288/37962 [84:01:19<3:53:58,  8.39s/it]
 96%|█████████▌| 36289/37962 [84:01:27<3:53:20,  8.37s/it]
 96%|█████████▌| 36290/37962 [84:01:36<3:50:56,  8.29s/it]
                                                          
{'loss': 1.1095, 'grad_norm': 9.386930465698242, 'learning_rate': 1.0157045448185631e-07, 'epoch': 0.96}

 96%|█████████▌| 36290/37962 [84:01:36<3:50:56,  8.29s/it]
 96%|█████████▌| 36291/37962 [84:01:44<3:49:52,  8.25s/it]
 96%|█████████▌| 36292/37962 [84:01:52<3:49:46,  8.26s/it]
 96%|█████████▌| 36293/37962 [84:02:00<3:48:02,  8.20s/it]
 96%|█████████▌| 36294/37962 [84:02:08<3:49:36,  8.26s/it]
 96%|█████████▌| 36295/37962 [84:02:17<3:49:42,  8.27s/it]
 96%|█████████▌| 36296/37962 [84:02:25<3:47:07,  8.18s/it]
 96%|█████████▌| 36297/37962 [84:02:33<3:49:33,  8.27s/it]
 96%|█████████▌| 36298/37962 [84:02:42<3:49:58,  8.29s/it]
 96%|█████████▌| 36299/37962 [84:02:50<3:49:35,  8.28s/it]
 96%|█████████▌| 36300/37962 [84:02:58<3:48:51,  8.26s/it]
                                                          
{'loss': 1.0846, 'grad_norm': 6.440732479095459, 'learning_rate': 1.0036116040733069e-07, 'epoch': 0.96}

 96%|█████████▌| 36300/37962 [84:02:58<3:48:51,  8.26s/it]
 96%|█████████▌| 36301/37962 [84:03:07<3:51:32,  8.36s/it]
 96%|█████████▌| 36302/37962 [84:03:15<3:47:38,  8.23s/it]
 96%|█████████▌| 36303/37962 [84:03:23<3:48:24,  8.26s/it]
 96%|█████████▌| 36304/37962 [84:03:31<3:48:12,  8.26s/it]
 96%|█████████▌| 36305/37962 [84:03:39<3:48:25,  8.27s/it]
 96%|█████████▌| 36306/37962 [84:03:48<3:49:00,  8.30s/it]
 96%|█████████▌| 36307/37962 [84:03:56<3:48:36,  8.29s/it]
 96%|█████████▌| 36308/37962 [84:04:04<3:48:44,  8.30s/it]
 96%|█████████▌| 36309/37962 [84:04:13<3:48:16,  8.29s/it]
 96%|█████████▌| 36310/37962 [84:04:21<3:49:45,  8.34s/it]
                                                          
{'loss': 1.0687, 'grad_norm': 8.739423751831055, 'learning_rate': 9.91590721084168e-08, 'epoch': 0.96}

 96%|█████████▌| 36310/37962 [84:04:21<3:49:45,  8.34s/it]
 96%|█████████▌| 36311/37962 [84:04:29<3:48:02,  8.29s/it]
 96%|█████████▌| 36312/37962 [84:04:37<3:44:14,  8.15s/it]
 96%|█████████▌| 36313/37962 [84:04:45<3:44:49,  8.18s/it]
 96%|█████████▌| 36314/37962 [84:04:53<3:44:13,  8.16s/it]
 96%|█████████▌| 36315/37962 [84:05:02<3:45:56,  8.23s/it]
 96%|█████████▌| 36316/37962 [84:05:10<3:45:01,  8.20s/it]
 96%|█████████▌| 36317/37962 [84:05:18<3:46:12,  8.25s/it]
 96%|█████████▌| 36318/37962 [84:05:27<3:46:53,  8.28s/it]
 96%|█████████▌| 36319/37962 [84:05:35<3:46:41,  8.28s/it]
 96%|█████████▌| 36320/37962 [84:05:43<3:45:21,  8.23s/it]
                                                          
{'loss': 1.1312, 'grad_norm': 9.1159029006958, 'learning_rate': 9.796419046009254e-08, 'epoch': 0.96}

 96%|█████████▌| 36320/37962 [84:05:43<3:45:21,  8.23s/it]
 96%|█████████▌| 36321/37962 [84:05:51<3:44:01,  8.19s/it]
 96%|█████████▌| 36322/37962 [84:06:00<3:45:47,  8.26s/it]
 96%|█████████▌| 36323/37962 [84:06:08<3:47:05,  8.31s/it]
 96%|█████████▌| 36324/37962 [84:06:16<3:46:48,  8.31s/it]
 96%|█████████▌| 36325/37962 [84:06:25<3:47:19,  8.33s/it]
 96%|█████████▌| 36326/37962 [84:06:33<3:44:37,  8.24s/it]
 96%|█████████▌| 36327/37962 [84:06:41<3:45:07,  8.26s/it]
 96%|█████████▌| 36328/37962 [84:06:49<3:45:19,  8.27s/it]
 96%|█████████▌| 36329/37962 [84:06:57<3:43:28,  8.21s/it]
 96%|█████████▌| 36330/37962 [84:07:06<3:42:46,  8.19s/it]
                                                          
{'loss': 1.121, 'grad_norm': 6.196286201477051, 'learning_rate': 9.677651633209439e-08, 'epoch': 0.96}

 96%|█████████▌| 36330/37962 [84:07:06<3:42:46,  8.19s/it]
 96%|█████████▌| 36331/37962 [84:07:14<3:42:06,  8.17s/it]
 96%|█████████▌| 36332/37962 [84:07:22<3:41:51,  8.17s/it]
 96%|█████████▌| 36333/37962 [84:07:30<3:42:11,  8.18s/it]
 96%|█████████▌| 36334/37962 [84:07:38<3:40:25,  8.12s/it]
 96%|█████████▌| 36335/37962 [84:07:46<3:41:23,  8.16s/it]
 96%|█████████▌| 36336/37962 [84:07:55<3:41:54,  8.19s/it]
 96%|█████████▌| 36337/37962 [84:08:03<3:42:17,  8.21s/it]
 96%|█████████▌| 36338/37962 [84:08:11<3:42:56,  8.24s/it]
 96%|█████████▌| 36339/37962 [84:08:19<3:41:15,  8.18s/it]
 96%|█████████▌| 36340/37962 [84:08:27<3:41:47,  8.20s/it]
                                                          
{'loss': 1.0968, 'grad_norm': 8.255705833435059, 'learning_rate': 9.559605058890864e-08, 'epoch': 0.96}

 96%|█████████▌| 36340/37962 [84:08:27<3:41:47,  8.20s/it]
 96%|█████████▌| 36341/37962 [84:08:35<3:40:10,  8.15s/it]
 96%|█████████▌| 36342/37962 [84:08:44<3:41:57,  8.22s/it]
 96%|█████████▌| 36343/37962 [84:08:52<3:41:45,  8.22s/it]
 96%|█████████▌| 36344/37962 [84:09:00<3:40:42,  8.18s/it]
 96%|█████████▌| 36345/37962 [84:09:08<3:40:08,  8.17s/it]
 96%|█████████▌| 36346/37962 [84:09:16<3:39:15,  8.14s/it]
 96%|█████████▌| 36347/37962 [84:09:25<3:41:01,  8.21s/it]
 96%|█████████▌| 36348/37962 [84:09:33<3:42:21,  8.27s/it]
 96%|█████████▌| 36349/37962 [84:09:41<3:41:53,  8.25s/it]
 96%|█████████▌| 36350/37962 [84:09:50<3:41:19,  8.24s/it]
                                                          
{'loss': 1.1311, 'grad_norm': 9.942719459533691, 'learning_rate': 9.442279408977462e-08, 'epoch': 0.96}

 96%|█████████▌| 36350/37962 [84:09:50<3:41:19,  8.24s/it]
 96%|█████████▌| 36351/37962 [84:09:58<3:41:02,  8.23s/it]
 96%|█████████▌| 36352/37962 [84:10:06<3:43:42,  8.34s/it]
 96%|█████████▌| 36353/37962 [84:10:15<3:42:19,  8.29s/it]
 96%|█████████▌| 36354/37962 [84:10:23<3:40:53,  8.24s/it]
 96%|█████████▌| 36355/37962 [84:10:31<3:40:15,  8.22s/it]
 96%|█████████▌| 36356/37962 [84:10:39<3:39:34,  8.20s/it]
 96%|█████████▌| 36357/37962 [84:10:47<3:35:23,  8.05s/it]
 96%|█████████▌| 36358/37962 [84:10:55<3:37:28,  8.13s/it]
 96%|█████████▌| 36359/37962 [84:11:03<3:37:24,  8.14s/it]
 96%|█████████▌| 36360/37962 [84:11:11<3:36:45,  8.12s/it]
                                                          
{'loss': 1.1126, 'grad_norm': 8.991029739379883, 'learning_rate': 9.325674768868697e-08, 'epoch': 0.96}

 96%|█████████▌| 36360/37962 [84:11:11<3:36:45,  8.12s/it]
 96%|█████████▌| 36361/37962 [84:11:19<3:37:21,  8.15s/it]
 96%|█████████▌| 36362/37962 [84:11:28<3:38:35,  8.20s/it]
 96%|█████████▌| 36363/37962 [84:11:36<3:39:04,  8.22s/it]
 96%|█████████▌| 36364/37962 [84:11:44<3:36:12,  8.12s/it]
 96%|█████████▌| 36365/37962 [84:11:52<3:38:17,  8.20s/it]
 96%|█████████▌| 36366/37962 [84:12:00<3:36:38,  8.14s/it]
 96%|█████████▌| 36367/37962 [84:12:09<3:37:41,  8.19s/it]
 96%|█████████▌| 36368/37962 [84:12:17<3:36:51,  8.16s/it]
 96%|█████████▌| 36369/37962 [84:12:25<3:38:32,  8.23s/it]
 96%|█████████▌| 36370/37962 [84:12:33<3:38:37,  8.24s/it]
                                                          
{'loss': 1.1126, 'grad_norm': 10.643099784851074, 'learning_rate': 9.209791223439013e-08, 'epoch': 0.96}

 96%|█████████▌| 36370/37962 [84:12:33<3:38:37,  8.24s/it]
 96%|█████████▌| 36371/37962 [84:12:42<3:39:15,  8.27s/it]
 96%|█████████▌| 36372/37962 [84:12:50<3:38:38,  8.25s/it]
 96%|█████████▌| 36373/37962 [84:12:58<3:40:26,  8.32s/it]
 96%|█████████▌| 36374/37962 [84:13:07<3:40:02,  8.31s/it]
 96%|█████████▌| 36375/37962 [84:13:22<4:38:09, 10.52s/it]
 96%|█████████▌| 36376/37962 [84:13:31<4:20:18,  9.85s/it]
 96%|█████████▌| 36377/37962 [84:13:39<4:08:41,  9.41s/it]
 96%|█████████▌| 36378/37962 [84:13:47<3:58:10,  9.02s/it]
 96%|█████████▌| 36379/37962 [84:13:56<3:52:44,  8.82s/it]
 96%|█████████▌| 36380/37962 [84:14:04<3:48:17,  8.66s/it]
                                                          
{'loss': 1.1884, 'grad_norm': 12.688947677612305, 'learning_rate': 9.094628857037935e-08, 'epoch': 0.96}

 96%|█████████▌| 36380/37962 [84:14:04<3:48:17,  8.66s/it]
 96%|█████████▌| 36381/37962 [84:14:12<3:45:06,  8.54s/it]
 96%|█████████▌| 36382/37962 [84:14:20<3:42:56,  8.47s/it]
 96%|█████████▌| 36383/37962 [84:14:29<3:40:21,  8.37s/it]
 96%|█████████▌| 36384/37962 [84:14:37<3:37:21,  8.26s/it]
 96%|█████████▌| 36385/37962 [84:14:45<3:37:19,  8.27s/it]
 96%|█████████▌| 36386/37962 [84:14:53<3:36:40,  8.25s/it]
 96%|█████████▌| 36387/37962 [84:15:01<3:35:03,  8.19s/it]
 96%|█████████▌| 36388/37962 [84:15:09<3:33:52,  8.15s/it]
 96%|█████████▌| 36389/37962 [84:15:18<3:36:09,  8.25s/it]
 96%|█████████▌| 36390/37962 [84:15:26<3:35:59,  8.24s/it]
                                                          
{'loss': 1.2314, 'grad_norm': 9.372642517089844, 'learning_rate': 8.98018775349041e-08, 'epoch': 0.96}

 96%|█████████▌| 36390/37962 [84:15:26<3:35:59,  8.24s/it]
 96%|█████████▌| 36391/37962 [84:15:34<3:35:55,  8.25s/it]
 96%|█████████▌| 36392/37962 [84:15:42<3:34:17,  8.19s/it]
 96%|█████████▌| 36393/37962 [84:15:50<3:34:37,  8.21s/it]
 96%|█████████▌| 36394/37962 [84:15:59<3:35:21,  8.24s/it]
 96%|█████████▌| 36395/37962 [84:16:07<3:35:36,  8.26s/it]
 96%|█████████▌| 36396/37962 [84:16:15<3:35:23,  8.25s/it]
 96%|█████████▌| 36397/37962 [84:16:23<3:32:42,  8.15s/it]
 96%|█████████▌| 36398/37962 [84:16:31<3:32:44,  8.16s/it]
 96%|█████████▌| 36399/37962 [84:16:40<3:33:40,  8.20s/it]
 96%|█████████▌| 36400/37962 [84:16:48<3:33:42,  8.21s/it]
                                                          
{'loss': 1.1349, 'grad_norm': 7.384982109069824, 'learning_rate': 8.866467996095918e-08, 'epoch': 0.96}

 96%|█████████▌| 36400/37962 [84:16:48<3:33:42,  8.21s/it]
 96%|█████████▌| 36401/37962 [84:16:56<3:34:44,  8.25s/it]
 96%|█████████▌| 36402/37962 [84:17:05<3:36:05,  8.31s/it]
 96%|█████████▌| 36403/37962 [84:17:13<3:35:45,  8.30s/it]
 96%|█████████▌| 36404/37962 [84:17:21<3:34:40,  8.27s/it]
 96%|█████████▌| 36405/37962 [84:17:29<3:34:20,  8.26s/it]
 96%|█████████▌| 36406/37962 [84:17:38<3:36:27,  8.35s/it]
 96%|█████████▌| 36407/37962 [84:17:46<3:34:37,  8.28s/it]
 96%|█████████▌| 36408/37962 [84:17:54<3:35:13,  8.31s/it]
 96%|█████████▌| 36409/37962 [84:18:02<3:32:41,  8.22s/it]
 96%|█████████▌| 36410/37962 [84:18:11<3:33:38,  8.26s/it]
                                                          
{'loss': 1.1281, 'grad_norm': 13.505167961120605, 'learning_rate': 8.753469667629133e-08, 'epoch': 0.96}

 96%|█████████▌| 36410/37962 [84:18:11<3:33:38,  8.26s/it]
 96%|█████████▌| 36411/37962 [84:18:19<3:33:17,  8.25s/it]
 96%|█████████▌| 36412/37962 [84:18:27<3:33:25,  8.26s/it]
 96%|█████████▌| 36413/37962 [84:18:36<3:34:02,  8.29s/it]
 96%|█████████▌| 36414/37962 [84:18:44<3:34:03,  8.30s/it]
 96%|█████████▌| 36415/37962 [84:18:52<3:34:17,  8.31s/it]
 96%|█████████▌| 36416/37962 [84:19:01<3:34:13,  8.31s/it]
 96%|█████████▌| 36417/37962 [84:19:09<3:32:35,  8.26s/it]
 96%|█████████▌| 36418/37962 [84:19:17<3:33:22,  8.29s/it]
 96%|█████████▌| 36419/37962 [84:19:26<3:33:46,  8.31s/it]
 96%|█████████▌| 36420/37962 [84:19:34<3:34:22,  8.34s/it]
                                                          
{'loss': 1.011, 'grad_norm': 9.017455101013184, 'learning_rate': 8.641192850339597e-08, 'epoch': 0.96}

 96%|█████████▌| 36420/37962 [84:19:34<3:34:22,  8.34s/it]
 96%|█████████▌| 36421/37962 [84:19:42<3:32:37,  8.28s/it]
 96%|█████████▌| 36422/37962 [84:19:50<3:32:02,  8.26s/it]
 96%|█████████▌| 36423/37962 [84:19:59<3:32:13,  8.27s/it]
 96%|█████████▌| 36424/37962 [84:20:07<3:32:04,  8.27s/it]
 96%|█████████▌| 36425/37962 [84:20:15<3:34:11,  8.36s/it]
 96%|█████████▌| 36426/37962 [84:20:24<3:32:43,  8.31s/it]
 96%|█████████▌| 36427/37962 [84:20:32<3:33:12,  8.33s/it]
 96%|█████████▌| 36428/37962 [84:20:40<3:30:00,  8.21s/it]
 96%|█████████▌| 36429/37962 [84:20:48<3:31:13,  8.27s/it]
 96%|█████████▌| 36430/37962 [84:20:57<3:30:55,  8.26s/it]
                                                          
{'loss': 1.1207, 'grad_norm': 11.889460563659668, 'learning_rate': 8.529637625951714e-08, 'epoch': 0.96}

 96%|█████████▌| 36430/37962 [84:20:57<3:30:55,  8.26s/it]
 96%|█████████▌| 36431/37962 [84:21:05<3:32:45,  8.34s/it]
 96%|█████████▌| 36432/37962 [84:21:13<3:32:17,  8.32s/it]
 96%|█████████▌| 36433/37962 [84:21:22<3:30:40,  8.27s/it]
 96%|█████████▌| 36434/37962 [84:21:30<3:31:32,  8.31s/it]
 96%|█████████▌| 36435/37962 [84:21:38<3:31:25,  8.31s/it]
 96%|█████████▌| 36436/37962 [84:21:47<3:31:02,  8.30s/it]
 96%|█████████▌| 36437/37962 [84:21:55<3:30:22,  8.28s/it]
 96%|█████████▌| 36438/37962 [84:22:03<3:30:27,  8.29s/it]
 96%|█████████▌| 36439/37962 [84:22:11<3:30:08,  8.28s/it]
 96%|█████████▌| 36440/37962 [84:22:20<3:29:50,  8.27s/it]
                                                          
{'loss': 1.1362, 'grad_norm': 9.88829517364502, 'learning_rate': 8.418804075664533e-08, 'epoch': 0.96}

 96%|█████████▌| 36440/37962 [84:22:20<3:29:50,  8.27s/it]
 96%|█████████▌| 36441/37962 [84:22:28<3:30:08,  8.29s/it]
 96%|█████████▌| 36442/37962 [84:22:36<3:30:19,  8.30s/it]
 96%|█████████▌| 36443/37962 [84:22:45<3:30:01,  8.30s/it]
 96%|█████████▌| 36444/37962 [84:22:53<3:31:12,  8.35s/it]
 96%|█████████▌| 36445/37962 [84:23:01<3:31:35,  8.37s/it]
 96%|█████████▌| 36446/37962 [84:23:09<3:28:13,  8.24s/it]
 96%|█████████▌| 36447/37962 [84:23:18<3:30:46,  8.35s/it]
 96%|█████████▌| 36448/37962 [84:23:26<3:30:31,  8.34s/it]
 96%|█████████▌| 36449/37962 [84:23:35<3:29:37,  8.31s/it]
 96%|█████████▌| 36450/37962 [84:23:43<3:29:04,  8.30s/it]
                                                          
{'loss': 1.2125, 'grad_norm': 8.951654434204102, 'learning_rate': 8.308692280151853e-08, 'epoch': 0.96}

 96%|█████████▌| 36450/37962 [84:23:43<3:29:04,  8.30s/it]
 96%|█████████▌| 36451/37962 [84:23:51<3:28:32,  8.28s/it]
 96%|█████████▌| 36452/37962 [84:24:00<3:30:06,  8.35s/it]
 96%|█████████▌| 36453/37962 [84:24:08<3:29:03,  8.31s/it]
 96%|█████████▌| 36454/37962 [84:24:16<3:28:30,  8.30s/it]
 96%|█████████▌| 36455/37962 [84:24:24<3:28:12,  8.29s/it]
 96%|█████████▌| 36456/37962 [84:24:33<3:28:04,  8.29s/it]
 96%|█████████▌| 36457/37962 [84:24:41<3:28:26,  8.31s/it]
 96%|█████████▌| 36458/37962 [84:24:50<3:30:33,  8.40s/it]
 96%|█████████▌| 36459/37962 [84:24:58<3:29:18,  8.36s/it]
 96%|█████████▌| 36460/37962 [84:25:06<3:29:39,  8.38s/it]
                                                          
{'loss': 1.1503, 'grad_norm': 9.401697158813477, 'learning_rate': 8.19930231956223e-08, 'epoch': 0.96}

 96%|█████████▌| 36460/37962 [84:25:06<3:29:39,  8.38s/it]
 96%|█████████▌| 36461/37962 [84:25:15<3:30:07,  8.40s/it]
 96%|█████████▌| 36462/37962 [84:25:23<3:29:06,  8.36s/it]
 96%|█████████▌| 36463/37962 [84:25:31<3:28:30,  8.35s/it]
 96%|█████████▌| 36464/37962 [84:25:39<3:27:28,  8.31s/it]
 96%|█████████▌| 36465/37962 [84:25:48<3:27:24,  8.31s/it]
 96%|█████████▌| 36466/37962 [84:25:56<3:26:23,  8.28s/it]
 96%|█████████▌| 36467/37962 [84:26:04<3:26:26,  8.29s/it]
 96%|█████████▌| 36468/37962 [84:26:13<3:25:48,  8.27s/it]
 96%|█████████▌| 36469/37962 [84:26:21<3:27:33,  8.34s/it]
 96%|█████████▌| 36470/37962 [84:26:29<3:24:41,  8.23s/it]
                                                          
{'loss': 1.2169, 'grad_norm': 12.372333526611328, 'learning_rate': 8.090634273518638e-08, 'epoch': 0.96}

 96%|█████████▌| 36470/37962 [84:26:29<3:24:41,  8.23s/it]
 96%|█████████▌| 36471/37962 [84:26:37<3:25:40,  8.28s/it]
 96%|█████████▌| 36472/37962 [84:26:46<3:27:16,  8.35s/it]
 96%|█████████▌| 36473/37962 [84:26:54<3:26:55,  8.34s/it]
 96%|█████████▌| 36474/37962 [84:27:03<3:26:26,  8.32s/it]
 96%|█████████▌| 36475/37962 [84:27:11<3:25:30,  8.29s/it]
 96%|█████████▌| 36476/37962 [84:27:19<3:25:39,  8.30s/it]
 96%|█████████▌| 36477/37962 [84:27:27<3:26:00,  8.32s/it]
 96%|█████████▌| 36478/37962 [84:27:36<3:27:13,  8.38s/it]
 96%|█████████▌| 36479/37962 [84:27:44<3:24:33,  8.28s/it]
 96%|█████████▌| 36480/37962 [84:27:52<3:24:56,  8.30s/it]
                                                          
{'loss': 1.1022, 'grad_norm': 9.986246109008789, 'learning_rate': 7.982688221118806e-08, 'epoch': 0.96}

 96%|█████████▌| 36480/37962 [84:27:52<3:24:56,  8.30s/it]
 96%|█████████▌| 36481/37962 [84:28:00<3:23:27,  8.24s/it]
 96%|█████████▌| 36482/37962 [84:28:09<3:24:05,  8.27s/it]
 96%|█████████▌| 36483/37962 [84:28:17<3:21:45,  8.18s/it]
 96%|█████████▌| 36484/37962 [84:28:25<3:22:17,  8.21s/it]
 96%|█████████▌| 36485/37962 [84:28:33<3:23:24,  8.26s/it]
 96%|█████████▌| 36486/37962 [84:28:42<3:23:32,  8.27s/it]
 96%|█████████▌| 36487/37962 [84:28:50<3:23:48,  8.29s/it]
 96%|█████████▌| 36488/37962 [84:28:58<3:23:15,  8.27s/it]
 96%|█████████▌| 36489/37962 [84:29:07<3:24:21,  8.32s/it]
 96%|█████████▌| 36490/37962 [84:29:15<3:24:15,  8.33s/it]
                                                          
{'loss': 1.1152, 'grad_norm': 9.671172142028809, 'learning_rate': 7.875464240934661e-08, 'epoch': 0.96}

 96%|█████████▌| 36490/37962 [84:29:15<3:24:15,  8.33s/it]
 96%|█████████▌| 36491/37962 [84:29:23<3:24:01,  8.32s/it]
 96%|█████████▌| 36492/37962 [84:29:31<3:20:20,  8.18s/it]
 96%|█████████▌| 36493/37962 [84:29:39<3:20:24,  8.19s/it]
 96%|█████████▌| 36494/37962 [84:29:48<3:21:13,  8.22s/it]
 96%|█████████▌| 36495/37962 [84:29:56<3:21:02,  8.22s/it]
 96%|█████████▌| 36496/37962 [84:30:04<3:22:14,  8.28s/it]
 96%|█████████▌| 36497/37962 [84:30:12<3:18:38,  8.14s/it]
 96%|█████████▌| 36498/37962 [84:30:20<3:17:38,  8.10s/it]
 96%|█████████▌| 36499/37962 [84:30:28<3:17:51,  8.11s/it]
 96%|█████████▌| 36500/37962 [84:30:37<3:18:25,  8.14s/it]
                                                          
{'loss': 1.0919, 'grad_norm': 7.804324626922607, 'learning_rate': 7.76896241101266e-08, 'epoch': 0.96}

 96%|█████████▌| 36500/37962 [84:30:37<3:18:25,  8.14s/it]
 96%|█████████▌| 36501/37962 [84:30:45<3:19:31,  8.19s/it]
 96%|█████████▌| 36502/37962 [84:30:53<3:20:11,  8.23s/it]
 96%|█████████▌| 36503/37962 [84:31:02<3:20:54,  8.26s/it]
 96%|█████████▌| 36504/37962 [84:31:10<3:19:05,  8.19s/it]
 96%|█████████▌| 36505/37962 [84:31:18<3:18:18,  8.17s/it]
 96%|█████████▌| 36506/37962 [84:31:26<3:18:23,  8.18s/it]
 96%|█████████▌| 36507/37962 [84:31:34<3:18:12,  8.17s/it]
 96%|█████████▌| 36508/37962 [84:31:42<3:17:44,  8.16s/it]
 96%|█████████▌| 36509/37962 [84:31:50<3:18:27,  8.20s/it]
 96%|█████████▌| 36510/37962 [84:31:59<3:18:04,  8.18s/it]
                                                          
{'loss': 1.0773, 'grad_norm': 7.977505683898926, 'learning_rate': 7.663182808873682e-08, 'epoch': 0.96}

 96%|█████████▌| 36510/37962 [84:31:59<3:18:04,  8.18s/it]
 96%|█████████▌| 36511/37962 [84:32:07<3:19:50,  8.26s/it]
 96%|█████████▌| 36512/37962 [84:32:15<3:19:29,  8.25s/it]
 96%|█████████▌| 36513/37962 [84:32:24<3:20:55,  8.32s/it]
 96%|█████████▌| 36514/37962 [84:32:32<3:18:36,  8.23s/it]
 96%|█████████▌| 36515/37962 [84:32:40<3:17:30,  8.19s/it]
 96%|█████████▌| 36516/37962 [84:32:48<3:18:07,  8.22s/it]
 96%|█████████▌| 36517/37962 [84:32:57<3:19:24,  8.28s/it]
 96%|█████████▌| 36518/37962 [84:33:05<3:20:01,  8.31s/it]
 96%|█████████▌| 36519/37962 [84:33:13<3:19:15,  8.29s/it]
 96%|█████████▌| 36520/37962 [84:33:21<3:19:05,  8.28s/it]
                                                          
{'loss': 1.1061, 'grad_norm': 7.611614227294922, 'learning_rate': 7.558125511512915e-08, 'epoch': 0.96}

 96%|█████████▌| 36520/37962 [84:33:21<3:19:05,  8.28s/it]
 96%|█████████▌| 36521/37962 [84:33:30<3:18:14,  8.25s/it]
 96%|█████████▌| 36522/37962 [84:33:38<3:18:18,  8.26s/it]
 96%|█████████▌| 36523/37962 [84:33:46<3:17:49,  8.25s/it]
 96%|█████████▌| 36524/37962 [84:33:54<3:18:27,  8.28s/it]
 96%|█████████▌| 36525/37962 [84:34:03<3:18:02,  8.27s/it]
 96%|█████████▌| 36526/37962 [84:34:11<3:16:24,  8.21s/it]
 96%|█████████▌| 36527/37962 [84:34:19<3:17:49,  8.27s/it]
 96%|█████████▌| 36528/37962 [84:34:28<3:18:54,  8.32s/it]
 96%|█████████▌| 36529/37962 [84:34:36<3:20:26,  8.39s/it]
 96%|█████████▌| 36530/37962 [84:34:44<3:17:08,  8.26s/it]
                                                          
{'loss': 1.0123, 'grad_norm': 11.49341869354248, 'learning_rate': 7.453790595399746e-08, 'epoch': 0.96}

 96%|█████████▌| 36530/37962 [84:34:44<3:17:08,  8.26s/it]
 96%|█████████▌| 36531/37962 [84:34:52<3:16:08,  8.22s/it]
 96%|█████████▌| 36532/37962 [84:35:00<3:14:52,  8.18s/it]
 96%|█████████▌| 36533/37962 [84:35:09<3:15:40,  8.22s/it]
 96%|█████████▌| 36534/37962 [84:35:17<3:16:27,  8.25s/it]
 96%|█████████▌| 36535/37962 [84:35:25<3:15:51,  8.24s/it]
 96%|█████████▌| 36536/37962 [84:35:34<3:16:46,  8.28s/it]
 96%|█████████▌| 36537/37962 [84:35:42<3:17:00,  8.30s/it]
 96%|█████████▌| 36538/37962 [84:35:50<3:16:59,  8.30s/it]
 96%|█████████▋| 36539/37962 [84:35:58<3:15:27,  8.24s/it]
 96%|█████████▋| 36540/37962 [84:36:15<4:12:42, 10.66s/it]
                                                          
{'loss': 1.034, 'grad_norm': 11.881976127624512, 'learning_rate': 7.350178136477648e-08, 'epoch': 0.96}

 96%|█████████▋| 36540/37962 [84:36:15<4:12:42, 10.66s/it]
 96%|█████████▋| 36541/37962 [84:36:23<3:54:51,  9.92s/it]
 96%|█████████▋| 36542/37962 [84:36:31<3:41:04,  9.34s/it]
 96%|█████████▋| 36543/37962 [84:36:39<3:32:54,  9.00s/it]
 96%|█████████▋| 36544/37962 [84:36:47<3:27:02,  8.76s/it]
 96%|█████████▋| 36545/37962 [84:36:55<3:22:04,  8.56s/it]
 96%|█████████▋| 36546/37962 [84:37:03<3:16:38,  8.33s/it]
 96%|█████████▋| 36547/37962 [84:37:11<3:16:06,  8.32s/it]
 96%|█████████▋| 36548/37962 [84:37:19<3:13:47,  8.22s/it]
 96%|█████████▋| 36549/37962 [84:37:28<3:14:31,  8.26s/it]
 96%|█████████▋| 36550/37962 [84:37:35<3:10:10,  8.08s/it]
                                                          
{'loss': 1.0123, 'grad_norm': 7.419308662414551, 'learning_rate': 7.247288210164516e-08, 'epoch': 0.96}

 96%|█████████▋| 36550/37962 [84:37:35<3:10:10,  8.08s/it]
 96%|█████████▋| 36551/37962 [84:37:43<3:09:47,  8.07s/it]
 96%|█████████▋| 36552/37962 [84:37:52<3:11:28,  8.15s/it]
 96%|█████████▋| 36553/37962 [84:38:00<3:12:28,  8.20s/it]
 96%|█████████▋| 36554/37962 [84:38:08<3:12:41,  8.21s/it]
 96%|█████████▋| 36555/37962 [84:38:17<3:12:56,  8.23s/it]
 96%|█████████▋| 36556/37962 [84:38:25<3:12:57,  8.23s/it]
 96%|█████████▋| 36557/37962 [84:38:33<3:14:13,  8.29s/it]
 96%|█████████▋| 36558/37962 [84:38:41<3:13:30,  8.27s/it]
 96%|█████████▋| 36559/37962 [84:38:50<3:14:36,  8.32s/it]
 96%|█████████▋| 36560/37962 [84:38:58<3:13:58,  8.30s/it]
                                                          
{'loss': 1.2354, 'grad_norm': 11.622288703918457, 'learning_rate': 7.145120891351886e-08, 'epoch': 0.96}

 96%|█████████▋| 36560/37962 [84:38:58<3:13:58,  8.30s/it]
 96%|█████████▋| 36561/37962 [84:39:07<3:14:18,  8.32s/it]
 96%|█████████▋| 36562/37962 [84:39:15<3:12:22,  8.24s/it]
 96%|█████████▋| 36563/37962 [84:39:23<3:12:09,  8.24s/it]
 96%|█████████▋| 36564/37962 [84:39:31<3:13:43,  8.31s/it]
 96%|█████████▋| 36565/37962 [84:39:40<3:13:37,  8.32s/it]
 96%|█████████▋| 36566/37962 [84:39:48<3:14:01,  8.34s/it]
 96%|█████████▋| 36567/37962 [84:39:56<3:11:29,  8.24s/it]
 96%|█████████▋| 36568/37962 [84:40:04<3:11:53,  8.26s/it]
 96%|█████████▋| 36569/37962 [84:40:13<3:11:37,  8.25s/it]
 96%|█████████▋| 36570/37962 [84:40:21<3:12:27,  8.30s/it]
                                                          
{'loss': 1.1524, 'grad_norm': 9.981803894042969, 'learning_rate': 7.04367625440583e-08, 'epoch': 0.96}

 96%|█████████▋| 36570/37962 [84:40:21<3:12:27,  8.30s/it]
 96%|█████████▋| 36571/37962 [84:40:29<3:12:27,  8.30s/it]
 96%|█████████▋| 36572/37962 [84:40:38<3:13:58,  8.37s/it]
 96%|█████████▋| 36573/37962 [84:40:46<3:11:44,  8.28s/it]
 96%|█████████▋| 36574/37962 [84:40:54<3:11:48,  8.29s/it]
 96%|█████████▋| 36575/37962 [84:41:02<3:08:23,  8.15s/it]
 96%|█████████▋| 36576/37962 [84:41:11<3:10:27,  8.24s/it]
 96%|█████████▋| 36577/37962 [84:41:19<3:10:40,  8.26s/it]
 96%|█████████▋| 36578/37962 [84:41:27<3:11:15,  8.29s/it]
 96%|█████████▋| 36579/37962 [84:41:36<3:12:04,  8.33s/it]
 96%|█████████▋| 36580/37962 [84:41:44<3:12:22,  8.35s/it]
                                                          
{'loss': 1.1494, 'grad_norm': 10.852524757385254, 'learning_rate': 6.942954373165944e-08, 'epoch': 0.96}

 96%|█████████▋| 36580/37962 [84:41:44<3:12:22,  8.35s/it]
 96%|█████████▋| 36581/37962 [84:41:52<3:10:25,  8.27s/it]
 96%|█████████▋| 36582/37962 [84:42:00<3:10:39,  8.29s/it]
 96%|█████████▋| 36583/37962 [84:42:09<3:11:53,  8.35s/it]
 96%|█████████▋| 36584/37962 [84:42:17<3:11:06,  8.32s/it]
 96%|█████████▋| 36585/37962 [84:42:25<3:09:43,  8.27s/it]
 96%|█████████▋| 36586/37962 [84:42:34<3:09:56,  8.28s/it]
 96%|█████████▋| 36587/37962 [84:42:42<3:09:52,  8.29s/it]
 96%|█████████▋| 36588/37962 [84:42:50<3:10:02,  8.30s/it]
 96%|█████████▋| 36589/37962 [84:42:59<3:10:34,  8.33s/it]
 96%|█████████▋| 36590/37962 [84:43:07<3:11:58,  8.40s/it]
                                                          
{'loss': 1.0491, 'grad_norm': 8.66505241394043, 'learning_rate': 6.842955320946143e-08, 'epoch': 0.96}

 96%|█████████▋| 36590/37962 [84:43:07<3:11:58,  8.40s/it]
 96%|█████████▋| 36591/37962 [84:43:15<3:10:47,  8.35s/it]
 96%|█████████▋| 36592/37962 [84:43:24<3:09:14,  8.29s/it]
 96%|█████████▋| 36593/37962 [84:43:32<3:07:15,  8.21s/it]
 96%|█████████▋| 36594/37962 [84:43:40<3:07:01,  8.20s/it]
 96%|█████████▋| 36595/37962 [84:43:48<3:07:19,  8.22s/it]
 96%|█████████▋| 36596/37962 [84:43:56<3:07:22,  8.23s/it]
 96%|█████████▋| 36597/37962 [84:44:04<3:06:24,  8.19s/it]
 96%|█████████▋| 36598/37962 [84:44:13<3:06:34,  8.21s/it]
 96%|█████████▋| 36599/37962 [84:44:21<3:07:26,  8.25s/it]
 96%|█████████▋| 36600/37962 [84:44:29<3:08:40,  8.31s/it]
                                                          
{'loss': 1.047, 'grad_norm': 12.68745231628418, 'learning_rate': 6.743679170533868e-08, 'epoch': 0.96}

 96%|█████████▋| 36600/37962 [84:44:29<3:08:40,  8.31s/it]
 96%|█████████▋| 36601/37962 [84:44:38<3:09:10,  8.34s/it]
 96%|█████████▋| 36602/37962 [84:44:46<3:09:00,  8.34s/it]
 96%|█████████▋| 36603/37962 [84:44:54<3:08:02,  8.30s/it]
 96%|█████████▋| 36604/37962 [84:45:03<3:06:51,  8.26s/it]
 96%|█████████▋| 36605/37962 [84:45:11<3:07:37,  8.30s/it]
 96%|█████████▋| 36606/37962 [84:45:19<3:07:37,  8.30s/it]
 96%|█████████▋| 36607/37962 [84:45:27<3:06:19,  8.25s/it]
 96%|█████████▋| 36608/37962 [84:45:36<3:08:09,  8.34s/it]
 96%|█████████▋| 36609/37962 [84:45:44<3:04:05,  8.16s/it]
 96%|█████████▋| 36610/37962 [84:45:52<3:03:15,  8.13s/it]
                                                          
{'loss': 1.2583, 'grad_norm': 13.881913185119629, 'learning_rate': 6.645125994190538e-08, 'epoch': 0.96}

 96%|█████████▋| 36610/37962 [84:45:52<3:03:15,  8.13s/it]
 96%|█████████▋| 36611/37962 [84:46:00<3:03:48,  8.16s/it]
 96%|█████████▋| 36612/37962 [84:46:08<3:04:50,  8.22s/it]
 96%|█████████▋| 36613/37962 [84:46:17<3:04:46,  8.22s/it]
 96%|█████████▋| 36614/37962 [84:46:25<3:04:11,  8.20s/it]
 96%|█████████▋| 36615/37962 [84:46:33<3:04:51,  8.23s/it]
 96%|█████████▋| 36616/37962 [84:46:41<3:02:52,  8.15s/it]
 96%|█████████▋| 36617/37962 [84:46:49<3:00:27,  8.05s/it]
 96%|█████████▋| 36618/37962 [84:46:57<3:01:45,  8.11s/it]
 96%|█████████▋| 36619/37962 [84:47:05<3:03:21,  8.19s/it]
 96%|█████████▋| 36620/37962 [84:47:14<3:05:12,  8.28s/it]
                                                          
{'loss': 1.3079, 'grad_norm': 9.526371955871582, 'learning_rate': 6.547295863651326e-08, 'epoch': 0.96}

 96%|█████████▋| 36620/37962 [84:47:14<3:05:12,  8.28s/it]
 96%|█████████▋| 36621/37962 [84:47:22<3:05:17,  8.29s/it]
 96%|█████████▋| 36622/37962 [84:47:31<3:05:33,  8.31s/it]
 96%|█████████▋| 36623/37962 [84:47:39<3:05:28,  8.31s/it]
 96%|█████████▋| 36624/37962 [84:47:47<3:06:06,  8.35s/it]
 96%|█████████▋| 36625/37962 [84:47:55<3:02:55,  8.21s/it]
 96%|█████████▋| 36626/37962 [84:48:03<3:02:19,  8.19s/it]
 96%|█████████▋| 36627/37962 [84:48:12<3:02:05,  8.18s/it]
 96%|█████████▋| 36628/37962 [84:48:20<3:02:12,  8.20s/it]
 96%|█████████▋| 36629/37962 [84:48:28<3:03:19,  8.25s/it]
 96%|█████████▋| 36630/37962 [84:48:37<3:04:34,  8.31s/it]
                                                          
{'loss': 1.1213, 'grad_norm': 6.9679975509643555, 'learning_rate': 6.450188850124938e-08, 'epoch': 0.96}

 96%|█████████▋| 36630/37962 [84:48:37<3:04:34,  8.31s/it]
 96%|█████████▋| 36631/37962 [84:48:45<3:03:44,  8.28s/it]
 96%|█████████▋| 36632/37962 [84:48:53<3:03:11,  8.26s/it]
 96%|█████████▋| 36633/37962 [84:49:01<3:03:19,  8.28s/it]
 97%|█████████▋| 36634/37962 [84:49:10<3:05:05,  8.36s/it]
 97%|█████████▋| 36635/37962 [84:49:18<3:03:50,  8.31s/it]
 97%|█████████▋| 36636/37962 [84:49:27<3:04:33,  8.35s/it]
 97%|█████████▋| 36637/37962 [84:49:35<3:04:26,  8.35s/it]
 97%|█████████▋| 36638/37962 [84:49:43<3:04:06,  8.34s/it]
 97%|█████████▋| 36639/37962 [84:49:52<3:04:27,  8.37s/it]
 97%|█████████▋| 36640/37962 [84:50:00<3:03:17,  8.32s/it]
                                                          
{'loss': 1.0787, 'grad_norm': 13.912981986999512, 'learning_rate': 6.353805024294168e-08, 'epoch': 0.97}

 97%|█████████▋| 36640/37962 [84:50:00<3:03:17,  8.32s/it]
 97%|█████████▋| 36641/37962 [84:50:08<3:01:29,  8.24s/it]
 97%|█████████▋| 36642/37962 [84:50:16<3:00:47,  8.22s/it]
 97%|█████████▋| 36643/37962 [84:50:24<3:00:59,  8.23s/it]
 97%|█████████▋| 36644/37962 [84:50:33<3:01:05,  8.24s/it]
 97%|█████████▋| 36645/37962 [84:50:40<2:58:13,  8.12s/it]
 97%|█████████▋| 36646/37962 [84:50:49<2:58:11,  8.12s/it]
 97%|█████████▋| 36647/37962 [84:50:56<2:56:12,  8.04s/it]
 97%|█████████▋| 36648/37962 [84:51:04<2:55:56,  8.03s/it]
 97%|█████████▋| 36649/37962 [84:51:13<2:57:15,  8.10s/it]
 97%|█████████▋| 36650/37962 [84:51:21<2:58:47,  8.18s/it]
                                                          
{'loss': 1.0768, 'grad_norm': 22.561283111572266, 'learning_rate': 6.258144456314896e-08, 'epoch': 0.97}

 97%|█████████▋| 36650/37962 [84:51:21<2:58:47,  8.18s/it]
 97%|█████████▋| 36651/37962 [84:51:29<2:59:09,  8.20s/it]
 97%|█████████▋| 36652/37962 [84:51:37<2:57:43,  8.14s/it]
 97%|█████████▋| 36653/37962 [84:51:45<2:55:00,  8.02s/it]
 97%|█████████▋| 36654/37962 [84:51:54<2:57:51,  8.16s/it]
 97%|█████████▋| 36655/37962 [84:52:02<2:58:55,  8.21s/it]
 97%|█████████▋| 36656/37962 [84:52:10<2:58:46,  8.21s/it]
 97%|█████████▋| 36657/37962 [84:52:18<2:59:08,  8.24s/it]
 97%|█████████▋| 36658/37962 [84:52:27<3:00:08,  8.29s/it]
 97%|█████████▋| 36659/37962 [84:52:35<3:01:25,  8.35s/it]
 97%|█████████▋| 36660/37962 [84:52:43<2:59:19,  8.26s/it]
                                                          
{'loss': 1.0532, 'grad_norm': 8.544466972351074, 'learning_rate': 6.163207215816869e-08, 'epoch': 0.97}

 97%|█████████▋| 36660/37962 [84:52:43<2:59:19,  8.26s/it]
 97%|█████████▋| 36661/37962 [84:52:51<2:57:25,  8.18s/it]
 97%|█████████▋| 36662/37962 [84:53:00<2:57:24,  8.19s/it]
 97%|█████████▋| 36663/37962 [84:53:08<2:59:08,  8.27s/it]
 97%|█████████▋| 36664/37962 [84:53:16<2:57:31,  8.21s/it]
 97%|█████████▋| 36665/37962 [84:53:24<2:55:29,  8.12s/it]
 97%|█████████▋| 36666/37962 [84:53:32<2:55:01,  8.10s/it]
 97%|█████████▋| 36667/37962 [84:53:40<2:56:09,  8.16s/it]
 97%|█████████▋| 36668/37962 [84:53:49<2:56:45,  8.20s/it]
 97%|█████████▋| 36669/37962 [84:53:57<2:57:05,  8.22s/it]
 97%|█████████▋| 36670/37962 [84:54:05<2:55:56,  8.17s/it]
                                                          
{'loss': 0.9796, 'grad_norm': 8.309013366699219, 'learning_rate': 6.068993371903254e-08, 'epoch': 0.97}

 97%|█████████▋| 36670/37962 [84:54:05<2:55:56,  8.17s/it]
 97%|█████████▋| 36671/37962 [84:54:13<2:56:23,  8.20s/it]
 97%|█████████▋| 36672/37962 [84:54:22<2:56:49,  8.22s/it]
 97%|█████████▋| 36673/37962 [84:54:30<2:55:33,  8.17s/it]
 97%|█████████▋| 36674/37962 [84:54:38<2:56:00,  8.20s/it]
 97%|█████████▋| 36675/37962 [84:54:46<2:56:19,  8.22s/it]
 97%|█████████▋| 36676/37962 [84:54:54<2:57:02,  8.26s/it]
 97%|█████████▋| 36677/37962 [84:55:03<2:55:42,  8.20s/it]
 97%|█████████▋| 36678/37962 [84:55:11<2:55:43,  8.21s/it]
 97%|█████████▋| 36679/37962 [84:55:19<2:55:32,  8.21s/it]
 97%|█████████▋| 36680/37962 [84:55:27<2:57:22,  8.30s/it]
                                                          
{'loss': 1.1832, 'grad_norm': 13.463048934936523, 'learning_rate': 5.97550299315075e-08, 'epoch': 0.97}

 97%|█████████▋| 36680/37962 [84:55:27<2:57:22,  8.30s/it]
 97%|█████████▋| 36681/37962 [84:55:35<2:54:57,  8.19s/it]
 97%|█████████▋| 36682/37962 [84:55:44<2:55:17,  8.22s/it]
 97%|█████████▋| 36683/37962 [84:55:52<2:55:47,  8.25s/it]
 97%|█████████▋| 36684/37962 [84:56:01<2:57:41,  8.34s/it]
 97%|█████████▋| 36685/37962 [84:56:09<2:57:23,  8.33s/it]
 97%|█████████▋| 36686/37962 [84:56:17<2:55:38,  8.26s/it]
 97%|█████████▋| 36687/37962 [84:56:25<2:55:24,  8.25s/it]
 97%|█████████▋| 36688/37962 [84:56:33<2:54:59,  8.24s/it]
 97%|█████████▋| 36689/37962 [84:56:42<2:54:47,  8.24s/it]
 97%|█████████▋| 36690/37962 [84:56:50<2:53:08,  8.17s/it]
                                                          
{'loss': 1.1185, 'grad_norm': 10.513821601867676, 'learning_rate': 5.882736147609369e-08, 'epoch': 0.97}

 97%|█████████▋| 36690/37962 [84:56:50<2:53:08,  8.17s/it]
 97%|█████████▋| 36691/37962 [84:56:58<2:51:53,  8.11s/it]
 97%|█████████▋| 36692/37962 [84:57:06<2:52:07,  8.13s/it]
 97%|█████████▋| 36693/37962 [84:57:14<2:52:40,  8.16s/it]
 97%|█████████▋| 36694/37962 [84:57:22<2:52:54,  8.18s/it]
 97%|█████████▋| 36695/37962 [84:57:31<2:53:56,  8.24s/it]
 97%|█████████▋| 36696/37962 [84:57:39<2:55:04,  8.30s/it]
 97%|█████████▋| 36697/37962 [84:57:48<2:56:37,  8.38s/it]
 97%|█████████▋| 36698/37962 [84:57:56<2:54:41,  8.29s/it]
 97%|█████████▋| 36699/37962 [84:58:04<2:54:51,  8.31s/it]
 97%|█████████▋| 36700/37962 [84:58:12<2:51:03,  8.13s/it]
                                                          
{'loss': 1.121, 'grad_norm': 12.073942184448242, 'learning_rate': 5.7906929028023196e-08, 'epoch': 0.97}

 97%|█████████▋| 36700/37962 [84:58:12<2:51:03,  8.13s/it]
 97%|█████████▋| 36701/37962 [84:58:20<2:52:20,  8.20s/it]
 97%|█████████▋| 36702/37962 [84:58:28<2:52:40,  8.22s/it]
 97%|█████████▋| 36703/37962 [84:58:37<2:52:05,  8.20s/it]
 97%|█████████▋| 36704/37962 [84:58:45<2:51:38,  8.19s/it]
 97%|█████████▋| 36705/37962 [84:58:53<2:53:27,  8.28s/it]
 97%|█████████▋| 36706/37962 [84:59:01<2:53:09,  8.27s/it]
 97%|█████████▋| 36707/37962 [84:59:10<2:53:23,  8.29s/it]
 97%|█████████▋| 36708/37962 [84:59:18<2:50:24,  8.15s/it]
 97%|█████████▋| 36709/37962 [84:59:26<2:51:04,  8.19s/it]
 97%|█████████▋| 36710/37962 [84:59:34<2:52:35,  8.27s/it]
                                                          
{'loss': 1.0284, 'grad_norm': 12.310080528259277, 'learning_rate': 5.699373325726454e-08, 'epoch': 0.97}

 97%|█████████▋| 36710/37962 [84:59:34<2:52:35,  8.27s/it]
 97%|█████████▋| 36711/37962 [84:59:43<2:52:31,  8.27s/it]
 97%|█████████▋| 36712/37962 [84:59:51<2:51:46,  8.24s/it]
 97%|█████████▋| 36713/37962 [84:59:59<2:50:26,  8.19s/it]
 97%|█████████▋| 36714/37962 [85:00:07<2:52:07,  8.28s/it]
 97%|█████████▋| 36715/37962 [85:00:16<2:52:23,  8.30s/it]
 97%|█████████▋| 36716/37962 [85:00:24<2:50:42,  8.22s/it]
 97%|█████████▋| 36717/37962 [85:00:32<2:52:17,  8.30s/it]
 97%|█████████▋| 36718/37962 [85:00:41<2:52:03,  8.30s/it]
 97%|█████████▋| 36719/37962 [85:00:49<2:51:52,  8.30s/it]
 97%|█████████▋| 36720/37962 [85:00:57<2:51:11,  8.27s/it]
                                                          
{'loss': 1.1958, 'grad_norm': 12.44055461883545, 'learning_rate': 5.6087774828517124e-08, 'epoch': 0.97}

 97%|█████████▋| 36720/37962 [85:00:57<2:51:11,  8.27s/it]
 97%|█████████▋| 36721/37962 [85:01:05<2:51:49,  8.31s/it]
 97%|█████████▋| 36722/37962 [85:01:14<2:52:14,  8.33s/it]
 97%|█████████▋| 36723/37962 [85:01:22<2:49:34,  8.21s/it]
 97%|█████████▋| 36724/37962 [85:01:30<2:48:45,  8.18s/it]
 97%|█████████▋| 36725/37962 [85:01:38<2:49:05,  8.20s/it]
 97%|█████████▋| 36726/37962 [85:01:46<2:49:24,  8.22s/it]
 97%|█████████▋| 36727/37962 [85:01:55<2:49:39,  8.24s/it]
 97%|█████████▋| 36728/37962 [85:02:03<2:50:20,  8.28s/it]
 97%|█████████▋| 36729/37962 [85:02:11<2:49:02,  8.23s/it]
 97%|█████████▋| 36730/37962 [85:02:19<2:49:36,  8.26s/it]
                                                          
{'loss': 1.0873, 'grad_norm': 10.837935447692871, 'learning_rate': 5.5189054401211253e-08, 'epoch': 0.97}

 97%|█████████▋| 36730/37962 [85:02:19<2:49:36,  8.26s/it]
 97%|█████████▋| 36731/37962 [85:02:28<2:49:26,  8.26s/it]
 97%|█████████▋| 36732/37962 [85:02:36<2:49:53,  8.29s/it]
 97%|█████████▋| 36733/37962 [85:02:45<2:50:59,  8.35s/it]
 97%|█████████▋| 36734/37962 [85:02:53<2:51:20,  8.37s/it]
 97%|█████████▋| 36735/37962 [85:03:01<2:51:03,  8.36s/it]
 97%|█████████▋| 36736/37962 [85:03:10<2:50:24,  8.34s/it]
 97%|█████████▋| 36737/37962 [85:03:18<2:49:16,  8.29s/it]
 97%|█████████▋| 36738/37962 [85:03:26<2:47:59,  8.23s/it]
 97%|█████████▋| 36739/37962 [85:03:34<2:49:17,  8.31s/it]
 97%|█████████▋| 36740/37962 [85:03:43<2:49:21,  8.32s/it]
                                                          
{'loss': 1.1674, 'grad_norm': 12.407529830932617, 'learning_rate': 5.429757262951141e-08, 'epoch': 0.97}

 97%|█████████▋| 36740/37962 [85:03:43<2:49:21,  8.32s/it]
 97%|█████████▋| 36741/37962 [85:03:51<2:48:45,  8.29s/it]
 97%|█████████▋| 36742/37962 [85:03:59<2:47:32,  8.24s/it]
 97%|█████████▋| 36743/37962 [85:04:08<2:48:25,  8.29s/it]
 97%|█████████▋| 36744/37962 [85:04:16<2:47:36,  8.26s/it]
 97%|█████████▋| 36745/37962 [85:04:24<2:47:16,  8.25s/it]
 97%|█████████▋| 36746/37962 [85:04:32<2:47:03,  8.24s/it]
 97%|█████████▋| 36747/37962 [85:04:41<2:48:19,  8.31s/it]
 97%|█████████▋| 36748/37962 [85:04:49<2:48:13,  8.31s/it]
 97%|█████████▋| 36749/37962 [85:04:57<2:47:29,  8.29s/it]
 97%|█████████▋| 36750/37962 [85:05:05<2:47:00,  8.27s/it]
                                                          
{'loss': 1.0961, 'grad_norm': 11.307591438293457, 'learning_rate': 5.3413330162309654e-08, 'epoch': 0.97}

 97%|█████████▋| 36750/37962 [85:05:05<2:47:00,  8.27s/it]
 97%|█████████▋| 36751/37962 [85:05:14<2:47:28,  8.30s/it]
 97%|█████████▋| 36752/37962 [85:05:22<2:46:54,  8.28s/it]
 97%|█████████▋| 36753/37962 [85:05:30<2:46:10,  8.25s/it]
 97%|█████████▋| 36754/37962 [85:05:38<2:45:57,  8.24s/it]
 97%|█████████▋| 36755/37962 [85:05:47<2:47:19,  8.32s/it]
 97%|█████████▋| 36756/37962 [85:05:55<2:46:31,  8.28s/it]
 97%|█████████▋| 36757/37962 [85:06:03<2:47:01,  8.32s/it]
 97%|█████████▋| 36758/37962 [85:06:12<2:47:03,  8.32s/it]
 97%|█████████▋| 36759/37962 [85:06:20<2:43:47,  8.17s/it]
 97%|█████████▋| 36760/37962 [85:06:28<2:43:10,  8.14s/it]
                                                          
{'loss': 1.0781, 'grad_norm': 11.309042930603027, 'learning_rate': 5.2536327643233356e-08, 'epoch': 0.97}

 97%|█████████▋| 36760/37962 [85:06:28<2:43:10,  8.14s/it]
 97%|█████████▋| 36761/37962 [85:06:36<2:43:03,  8.15s/it]
 97%|█████████▋| 36762/37962 [85:06:44<2:43:35,  8.18s/it]
 97%|█████████▋| 36763/37962 [85:06:52<2:44:11,  8.22s/it]
 97%|█████████▋| 36764/37962 [85:07:01<2:45:34,  8.29s/it]
 97%|█████████▋| 36765/37962 [85:07:09<2:47:08,  8.38s/it]
 97%|█████████▋| 36766/37962 [85:07:18<2:45:08,  8.28s/it]
 97%|█████████▋| 36767/37962 [85:07:26<2:45:17,  8.30s/it]
 97%|█████████▋| 36768/37962 [85:07:34<2:45:58,  8.34s/it]
 97%|█████████▋| 36769/37962 [85:07:42<2:44:43,  8.28s/it]
 97%|█████████▋| 36770/37962 [85:07:51<2:44:14,  8.27s/it]
                                                          
{'loss': 1.1103, 'grad_norm': 10.898759841918945, 'learning_rate': 5.166656571063633e-08, 'epoch': 0.97}

 97%|█████████▋| 36770/37962 [85:07:51<2:44:14,  8.27s/it]
 97%|█████████▋| 36771/37962 [85:07:59<2:43:49,  8.25s/it]
 97%|█████████▋| 36772/37962 [85:08:07<2:44:34,  8.30s/it]
 97%|█████████▋| 36773/37962 [85:08:16<2:43:53,  8.27s/it]
 97%|█████████▋| 36774/37962 [85:08:24<2:43:27,  8.26s/it]
 97%|█████████▋| 36775/37962 [85:08:32<2:41:53,  8.18s/it]
 97%|█████████▋| 36776/37962 [85:08:40<2:41:48,  8.19s/it]
 97%|█████████▋| 36777/37962 [85:08:48<2:42:58,  8.25s/it]
 97%|█████████▋| 36778/37962 [85:08:57<2:42:24,  8.23s/it]
 97%|█████████▋| 36779/37962 [85:09:05<2:42:53,  8.26s/it]
 97%|█████████▋| 36780/37962 [85:09:13<2:43:31,  8.30s/it]
                                                          
{'loss': 1.2051, 'grad_norm': 10.572040557861328, 'learning_rate': 5.080404499760216e-08, 'epoch': 0.97}

 97%|█████████▋| 36780/37962 [85:09:13<2:43:31,  8.30s/it]
 97%|█████████▋| 36781/37962 [85:09:22<2:44:13,  8.34s/it]
 97%|█████████▋| 36782/37962 [85:09:30<2:42:35,  8.27s/it]
 97%|█████████▋| 36783/37962 [85:09:38<2:42:30,  8.27s/it]
 97%|█████████▋| 36784/37962 [85:09:46<2:42:00,  8.25s/it]
 97%|█████████▋| 36785/37962 [85:09:54<2:41:29,  8.23s/it]
 97%|█████████▋| 36786/37962 [85:10:03<2:42:21,  8.28s/it]
 97%|█████████▋| 36787/37962 [85:10:11<2:41:46,  8.26s/it]
 97%|█████████▋| 36788/37962 [85:10:19<2:40:50,  8.22s/it]
 97%|█████████▋| 36789/37962 [85:10:27<2:41:00,  8.24s/it]
 97%|█████████▋| 36790/37962 [85:10:36<2:41:17,  8.26s/it]
                                                          
{'loss': 1.1375, 'grad_norm': 15.641359329223633, 'learning_rate': 4.9948766131945324e-08, 'epoch': 0.97}

 97%|█████████▋| 36790/37962 [85:10:36<2:41:17,  8.26s/it]
 97%|█████████▋| 36791/37962 [85:10:44<2:41:17,  8.26s/it]
 97%|█████████▋| 36792/37962 [85:10:52<2:41:39,  8.29s/it]
 97%|█████████▋| 36793/37962 [85:11:01<2:41:13,  8.28s/it]
 97%|█████████▋| 36794/37962 [85:11:09<2:41:12,  8.28s/it]
 97%|█████████▋| 36795/37962 [85:11:17<2:38:39,  8.16s/it]
 97%|█████████▋| 36796/37962 [85:11:25<2:38:48,  8.17s/it]
 97%|█████████▋| 36797/37962 [85:11:33<2:39:08,  8.20s/it]
 97%|█████████▋| 36798/37962 [85:11:42<2:40:01,  8.25s/it]
 97%|█████████▋| 36799/37962 [85:11:50<2:39:27,  8.23s/it]
 97%|█████████▋| 36800/37962 [85:11:58<2:39:15,  8.22s/it]
                                                          
{'loss': 1.0547, 'grad_norm': 10.344244956970215, 'learning_rate': 4.9100729736208944e-08, 'epoch': 0.97}

 97%|█████████▋| 36800/37962 [85:11:58<2:39:15,  8.22s/it]
 97%|█████████▋| 36801/37962 [85:12:06<2:39:39,  8.25s/it]
 97%|█████████▋| 36802/37962 [85:12:15<2:39:19,  8.24s/it]
 97%|█████████▋| 36803/37962 [85:12:23<2:39:32,  8.26s/it]
 97%|█████████▋| 36804/37962 [85:12:31<2:39:29,  8.26s/it]
 97%|█████████▋| 36805/37962 [85:12:40<2:39:55,  8.29s/it]
 97%|█████████▋| 36806/37962 [85:12:48<2:39:43,  8.29s/it]
 97%|█████████▋| 36807/37962 [85:12:56<2:39:21,  8.28s/it]
 97%|█████████▋| 36808/37962 [85:13:04<2:38:02,  8.22s/it]
 97%|█████████▋| 36809/37962 [85:13:12<2:38:00,  8.22s/it]
 97%|█████████▋| 36810/37962 [85:13:21<2:38:37,  8.26s/it]
                                                          
{'loss': 1.11, 'grad_norm': 7.862698078155518, 'learning_rate': 4.82599364276648e-08, 'epoch': 0.97}

 97%|█████████▋| 36810/37962 [85:13:21<2:38:37,  8.26s/it]
 97%|█████████▋| 36811/37962 [85:13:29<2:38:09,  8.24s/it]
 97%|█████████▋| 36812/37962 [85:13:37<2:37:22,  8.21s/it]
 97%|█████████▋| 36813/37962 [85:13:45<2:37:38,  8.23s/it]
 97%|█████████▋| 36814/37962 [85:13:54<2:38:52,  8.30s/it]
 97%|█████████▋| 36815/37962 [85:14:02<2:38:20,  8.28s/it]
 97%|█████████▋| 36816/37962 [85:14:10<2:36:56,  8.22s/it]
 97%|█████████▋| 36817/37962 [85:14:18<2:37:29,  8.25s/it]
 97%|█████████▋| 36818/37962 [85:14:27<2:38:11,  8.30s/it]
 97%|█████████▋| 36819/37962 [85:14:35<2:39:10,  8.36s/it]
 97%|█████████▋| 36820/37962 [85:14:44<2:40:08,  8.41s/it]
                                                          
{'loss': 1.0532, 'grad_norm': 9.584527969360352, 'learning_rate': 4.742638681831113e-08, 'epoch': 0.97}

 97%|█████████▋| 36820/37962 [85:14:44<2:40:08,  8.41s/it]
 97%|█████████▋| 36821/37962 [85:14:52<2:39:03,  8.36s/it]
 97%|█████████▋| 36822/37962 [85:15:00<2:37:42,  8.30s/it]
 97%|█████████▋| 36823/37962 [85:15:08<2:36:26,  8.24s/it]
 97%|█████████▋| 36824/37962 [85:15:16<2:35:16,  8.19s/it]
 97%|█████████▋| 36825/37962 [85:15:25<2:35:56,  8.23s/it]
 97%|█████████▋| 36826/37962 [85:15:33<2:37:20,  8.31s/it]
 97%|█████████▋| 36827/37962 [85:15:42<2:37:28,  8.32s/it]
 97%|█████████▋| 36828/37962 [85:15:50<2:36:38,  8.29s/it]
 97%|█████████▋| 36829/37962 [85:15:58<2:35:20,  8.23s/it]
 97%|█████████▋| 36830/37962 [85:16:06<2:36:14,  8.28s/it]
                                                          
{'loss': 1.1725, 'grad_norm': 11.668069839477539, 'learning_rate': 4.6600081514872606e-08, 'epoch': 0.97}

 97%|█████████▋| 36830/37962 [85:16:06<2:36:14,  8.28s/it]
 97%|█████████▋| 36831/37962 [85:16:15<2:35:39,  8.26s/it]
 97%|█████████▋| 36832/37962 [85:16:23<2:34:37,  8.21s/it]
 97%|█████████▋| 36833/37962 [85:16:31<2:34:35,  8.22s/it]
 97%|█████████▋| 36834/37962 [85:16:39<2:35:57,  8.30s/it]
 97%|█████████▋| 36835/37962 [85:16:48<2:35:10,  8.26s/it]
 97%|█████████▋| 36836/37962 [85:16:56<2:35:16,  8.27s/it]
 97%|█████████▋| 36837/37962 [85:17:04<2:34:56,  8.26s/it]
 97%|█████████▋| 36838/37962 [85:17:12<2:34:32,  8.25s/it]
 97%|█████████▋| 36839/37962 [85:17:20<2:34:18,  8.24s/it]
 97%|█████████▋| 36840/37962 [85:17:29<2:34:13,  8.25s/it]
                                                          
{'loss': 1.2774, 'grad_norm': 18.925735473632812, 'learning_rate': 4.578102111880367e-08, 'epoch': 0.97}

 97%|█████████▋| 36840/37962 [85:17:29<2:34:13,  8.25s/it]
 97%|█████████▋| 36841/37962 [85:17:37<2:32:51,  8.18s/it]
 97%|█████████▋| 36842/37962 [85:17:45<2:30:29,  8.06s/it]
 97%|█████████▋| 36843/37962 [85:17:53<2:31:11,  8.11s/it]
 97%|█████████▋| 36844/37962 [85:18:01<2:30:24,  8.07s/it]
 97%|█████████▋| 36845/37962 [85:18:09<2:31:40,  8.15s/it]
 97%|█████████▋| 36846/37962 [85:18:17<2:32:52,  8.22s/it]
 97%|█████████▋| 36847/37962 [85:18:25<2:31:34,  8.16s/it]
 97%|█████████▋| 36848/37962 [85:18:34<2:31:28,  8.16s/it]
 97%|█████████▋| 36849/37962 [85:18:42<2:32:12,  8.21s/it]
 97%|█████████▋| 36850/37962 [85:18:50<2:31:52,  8.19s/it]
                                                          
{'loss': 1.3305, 'grad_norm': 8.283143997192383, 'learning_rate': 4.49692062262852e-08, 'epoch': 0.97}

 97%|█████████▋| 36850/37962 [85:18:50<2:31:52,  8.19s/it]
 97%|█████████▋| 36851/37962 [85:18:58<2:31:11,  8.16s/it]
 97%|█████████▋| 36852/37962 [85:19:07<2:32:21,  8.24s/it]
 97%|█████████▋| 36853/37962 [85:19:15<2:34:51,  8.38s/it]
 97%|█████████▋| 36854/37962 [85:19:24<2:34:28,  8.37s/it]
 97%|█████████▋| 36855/37962 [85:19:32<2:31:58,  8.24s/it]
 97%|█████████▋| 36856/37962 [85:19:40<2:32:03,  8.25s/it]
 97%|█████████▋| 36857/37962 [85:19:48<2:32:31,  8.28s/it]
 97%|█████████▋| 36858/37962 [85:19:57<2:32:20,  8.28s/it]
 97%|█████████▋| 36859/37962 [85:20:05<2:32:40,  8.31s/it]
 97%|█████████▋| 36860/37962 [85:20:13<2:31:45,  8.26s/it]
                                                          
{'loss': 1.1376, 'grad_norm': 11.803452491760254, 'learning_rate': 4.416463742822341e-08, 'epoch': 0.97}

 97%|█████████▋| 36860/37962 [85:20:13<2:31:45,  8.26s/it]
 97%|█████████▋| 36861/37962 [85:20:21<2:32:14,  8.30s/it]
 97%|█████████▋| 36862/37962 [85:20:29<2:30:46,  8.22s/it]
 97%|█████████▋| 36863/37962 [85:20:38<2:31:23,  8.26s/it]
 97%|█████████▋| 36864/37962 [85:20:46<2:28:56,  8.14s/it]
 97%|█████████▋| 36865/37962 [85:20:54<2:29:49,  8.19s/it]
 97%|█████████▋| 36866/37962 [85:21:02<2:30:04,  8.22s/it]
 97%|█████████▋| 36867/37962 [85:21:10<2:28:23,  8.13s/it]
 97%|█████████▋| 36868/37962 [85:21:18<2:26:59,  8.06s/it]
 97%|█████████▋| 36869/37962 [85:21:27<2:29:17,  8.20s/it]
 97%|█████████▋| 36870/37962 [85:21:35<2:30:17,  8.26s/it]
                                                          
{'loss': 1.0768, 'grad_norm': 11.225531578063965, 'learning_rate': 4.3367315310247626e-08, 'epoch': 0.97}

 97%|█████████▋| 36870/37962 [85:21:35<2:30:17,  8.26s/it]
 97%|█████████▋| 36871/37962 [85:21:43<2:30:30,  8.28s/it]
 97%|█████████▋| 36872/37962 [85:21:52<2:29:55,  8.25s/it]
 97%|█████████▋| 36873/37962 [85:22:00<2:30:08,  8.27s/it]
 97%|█████████▋| 36874/37962 [85:22:08<2:31:06,  8.33s/it]
 97%|█████████▋| 36875/37962 [85:22:17<2:30:46,  8.32s/it]
 97%|█████████▋| 36876/37962 [85:22:25<2:28:32,  8.21s/it]
 97%|█████████▋| 36877/37962 [85:22:33<2:28:25,  8.21s/it]
 97%|█████████▋| 36878/37962 [85:22:41<2:28:57,  8.25s/it]
 97%|█████████▋| 36879/37962 [85:22:49<2:26:52,  8.14s/it]
 97%|█████████▋| 36880/37962 [85:22:57<2:26:36,  8.13s/it]
                                                          
{'loss': 0.9991, 'grad_norm': 18.636228561401367, 'learning_rate': 4.257724045271694e-08, 'epoch': 0.97}

 97%|█████████▋| 36880/37962 [85:22:57<2:26:36,  8.13s/it]
 97%|█████████▋| 36881/37962 [85:23:05<2:26:46,  8.15s/it]
 97%|█████████▋| 36882/37962 [85:23:14<2:28:09,  8.23s/it]
 97%|█████████▋| 36883/37962 [85:23:22<2:26:14,  8.13s/it]
 97%|█████████▋| 36884/37962 [85:23:30<2:25:58,  8.13s/it]
 97%|█████████▋| 36885/37962 [85:23:38<2:26:59,  8.19s/it]
 97%|█████████▋| 36886/37962 [85:23:46<2:24:27,  8.05s/it]
 97%|█████████▋| 36887/37962 [85:23:54<2:23:35,  8.01s/it]
 97%|█████████▋| 36888/37962 [85:24:02<2:24:46,  8.09s/it]
 97%|█████████▋| 36889/37962 [85:24:10<2:25:21,  8.13s/it]
 97%|█████████▋| 36890/37962 [85:24:18<2:24:56,  8.11s/it]
                                                          
{'loss': 1.2337, 'grad_norm': 15.59659481048584, 'learning_rate': 4.1794413430711336e-08, 'epoch': 0.97}

 97%|█████████▋| 36890/37962 [85:24:18<2:24:56,  8.11s/it]
 97%|█████████▋| 36891/37962 [85:24:26<2:24:41,  8.11s/it]
 97%|█████████▋| 36892/37962 [85:24:35<2:25:41,  8.17s/it]
 97%|█████████▋| 36893/37962 [85:24:43<2:25:13,  8.15s/it]
 97%|█████████▋| 36894/37962 [85:24:51<2:24:33,  8.12s/it]
 97%|█████████▋| 36895/37962 [85:24:59<2:24:58,  8.15s/it]
 97%|█████████▋| 36896/37962 [85:25:07<2:25:31,  8.19s/it]
 97%|█████████▋| 36897/37962 [85:25:16<2:25:27,  8.19s/it]
 97%|█████████▋| 36898/37962 [85:25:24<2:25:25,  8.20s/it]
 97%|█████████▋| 36899/37962 [85:25:32<2:26:14,  8.25s/it]
 97%|█████████▋| 36900/37962 [85:25:40<2:26:04,  8.25s/it]
                                                          
{'loss': 1.1413, 'grad_norm': 7.88625955581665, 'learning_rate': 4.101883481403946e-08, 'epoch': 0.97}

 97%|█████████▋| 36900/37962 [85:25:40<2:26:04,  8.25s/it]
 97%|█████████▋| 36901/37962 [85:25:49<2:26:52,  8.31s/it]
 97%|█████████▋| 36902/37962 [85:25:57<2:26:25,  8.29s/it]
 97%|█████████▋| 36903/37962 [85:26:05<2:26:45,  8.32s/it]
 97%|█████████▋| 36904/37962 [85:26:14<2:25:11,  8.23s/it]
 97%|█████████▋| 36905/37962 [85:26:22<2:24:29,  8.20s/it]
 97%|█████████▋| 36906/37962 [85:26:30<2:24:52,  8.23s/it]
 97%|█████████▋| 36907/37962 [85:26:38<2:23:53,  8.18s/it]
 97%|█████████▋| 36908/37962 [85:26:46<2:24:14,  8.21s/it]
 97%|█████████▋| 36909/37962 [85:26:55<2:24:22,  8.23s/it]
 97%|█████████▋| 36910/37962 [85:27:03<2:25:05,  8.27s/it]
                                                          
{'loss': 1.1504, 'grad_norm': 8.369081497192383, 'learning_rate': 4.0250505167229727e-08, 'epoch': 0.97}

 97%|█████████▋| 36910/37962 [85:27:03<2:25:05,  8.27s/it]
 97%|█████████▋| 36911/37962 [85:27:12<2:26:35,  8.37s/it]
 97%|█████████▋| 36912/37962 [85:27:20<2:26:28,  8.37s/it]
 97%|█████████▋| 36913/37962 [85:27:28<2:24:56,  8.29s/it]
 97%|█████████▋| 36914/37962 [85:27:37<2:25:56,  8.36s/it]
 97%|█████████▋| 36915/37962 [85:27:44<2:23:08,  8.20s/it]
 97%|█████████▋| 36916/37962 [85:27:53<2:23:41,  8.24s/it]
 97%|█████████▋| 36917/37962 [85:28:01<2:22:21,  8.17s/it]
 97%|█████████▋| 36918/37962 [85:28:09<2:24:05,  8.28s/it]
 97%|█████████▋| 36919/37962 [85:28:18<2:24:08,  8.29s/it]
 97%|█████████▋| 36920/37962 [85:28:25<2:22:06,  8.18s/it]
                                                          
{'loss': 1.1788, 'grad_norm': 25.276561737060547, 'learning_rate': 3.94894250495359e-08, 'epoch': 0.97}

 97%|█████████▋| 36920/37962 [85:28:25<2:22:06,  8.18s/it]
 97%|█████████▋| 36921/37962 [85:28:34<2:22:57,  8.24s/it]
 97%|█████████▋| 36922/37962 [85:28:42<2:20:59,  8.13s/it]
 97%|█████████▋| 36923/37962 [85:28:50<2:22:02,  8.20s/it]
 97%|█████████▋| 36924/37962 [85:28:58<2:22:23,  8.23s/it]
 97%|█████████▋| 36925/37962 [85:29:07<2:24:33,  8.36s/it]
 97%|█████████▋| 36926/37962 [85:29:15<2:23:33,  8.31s/it]
 97%|█████████▋| 36927/37962 [85:29:24<2:23:16,  8.31s/it]
 97%|█████████▋| 36928/37962 [85:29:32<2:23:18,  8.32s/it]
 97%|█████████▋| 36929/37962 [85:29:40<2:22:29,  8.28s/it]
 97%|█████████▋| 36930/37962 [85:29:48<2:22:15,  8.27s/it]
                                                          
{'loss': 1.2007, 'grad_norm': 9.80546760559082, 'learning_rate': 3.8735595014935954e-08, 'epoch': 0.97}

 97%|█████████▋| 36930/37962 [85:29:48<2:22:15,  8.27s/it]
 97%|█████████▋| 36931/37962 [85:29:57<2:22:31,  8.29s/it]
 97%|█████████▋| 36932/37962 [85:30:05<2:20:47,  8.20s/it]
 97%|█████████▋| 36933/37962 [85:30:13<2:21:32,  8.25s/it]
 97%|█████████▋| 36934/37962 [85:30:21<2:21:41,  8.27s/it]
 97%|█████████▋| 36935/37962 [85:30:29<2:20:47,  8.23s/it]
 97%|█████████▋| 36936/37962 [85:30:38<2:21:39,  8.28s/it]
 97%|█████████▋| 36937/37962 [85:30:46<2:21:28,  8.28s/it]
 97%|█████████▋| 36938/37962 [85:30:54<2:20:57,  8.26s/it]
 97%|█████████▋| 36939/37962 [85:31:03<2:21:32,  8.30s/it]
 97%|█████████▋| 36940/37962 [85:31:11<2:21:32,  8.31s/it]
                                                          
{'loss': 1.2547, 'grad_norm': 9.60522174835205, 'learning_rate': 3.798901561212875e-08, 'epoch': 0.97}

 97%|█████████▋| 36940/37962 [85:31:11<2:21:32,  8.31s/it]
 97%|█████████▋| 36941/37962 [85:31:19<2:21:28,  8.31s/it]
 97%|█████████▋| 36942/37962 [85:31:28<2:21:56,  8.35s/it]
 97%|█████████▋| 36943/37962 [85:31:36<2:19:32,  8.22s/it]
 97%|█████████▋| 36944/37962 [85:31:44<2:19:57,  8.25s/it]
 97%|█████████▋| 36945/37962 [85:31:52<2:20:18,  8.28s/it]
 97%|█████████▋| 36946/37962 [85:32:01<2:20:03,  8.27s/it]
 97%|█████████▋| 36947/37962 [85:32:09<2:20:30,  8.31s/it]
 97%|█████████▋| 36948/37962 [85:32:17<2:20:16,  8.30s/it]
 97%|█████████▋| 36949/37962 [85:32:26<2:20:11,  8.30s/it]
 97%|█████████▋| 36950/37962 [85:32:34<2:20:20,  8.32s/it]
                                                          
{'loss': 1.1535, 'grad_norm': 7.3806586265563965, 'learning_rate': 3.724968738453738e-08, 'epoch': 0.97}

 97%|█████████▋| 36950/37962 [85:32:34<2:20:20,  8.32s/it]
 97%|█████████▋| 36951/37962 [85:32:42<2:19:39,  8.29s/it]
 97%|█████████▋| 36952/37962 [85:32:51<2:20:18,  8.34s/it]
 97%|█████████▋| 36953/37962 [85:32:59<2:20:46,  8.37s/it]
 97%|█████████▋| 36954/37962 [85:33:08<2:20:43,  8.38s/it]
 97%|█████████▋| 36955/37962 [85:33:16<2:20:25,  8.37s/it]
 97%|█████████▋| 36956/37962 [85:33:24<2:18:17,  8.25s/it]
 97%|█████████▋| 36957/37962 [85:33:32<2:18:32,  8.27s/it]
 97%|█████████▋| 36958/37962 [85:33:40<2:17:42,  8.23s/it]
 97%|█████████▋| 36959/37962 [85:33:49<2:17:36,  8.23s/it]
 97%|█████████▋| 36960/37962 [85:33:57<2:17:42,  8.25s/it]
                                                          
{'loss': 1.0804, 'grad_norm': 13.008478164672852, 'learning_rate': 3.651761087030581e-08, 'epoch': 0.97}

 97%|█████████▋| 36960/37962 [85:33:57<2:17:42,  8.25s/it]
 97%|█████████▋| 36961/37962 [85:34:05<2:16:01,  8.15s/it]
 97%|█████████▋| 36962/37962 [85:34:13<2:16:19,  8.18s/it]
 97%|█████████▋| 36963/37962 [85:34:21<2:16:24,  8.19s/it]
 97%|█████████▋| 36964/37962 [85:34:30<2:16:52,  8.23s/it]
 97%|█████████▋| 36965/37962 [85:34:38<2:17:38,  8.28s/it]
 97%|█████████▋| 36966/37962 [85:34:54<2:57:16, 10.68s/it]
 97%|█████████▋| 36967/37962 [85:35:03<2:45:15,  9.96s/it]
 97%|█████████▋| 36968/37962 [85:35:11<2:37:04,  9.48s/it]
 97%|█████████▋| 36969/37962 [85:35:19<2:31:28,  9.15s/it]
 97%|█████████▋| 36970/37962 [85:35:28<2:27:08,  8.90s/it]
                                                          
{'loss': 0.885, 'grad_norm': 6.863667964935303, 'learning_rate': 3.579278660230001e-08, 'epoch': 0.97}

 97%|█████████▋| 36970/37962 [85:35:28<2:27:08,  8.90s/it]
 97%|█████████▋| 36971/37962 [85:35:36<2:25:06,  8.79s/it]
 97%|█████████▋| 36972/37962 [85:35:44<2:21:03,  8.55s/it]
 97%|█████████▋| 36973/37962 [85:35:52<2:19:26,  8.46s/it]
 97%|█████████▋| 36974/37962 [85:36:01<2:18:54,  8.44s/it]
 97%|█████████▋| 36975/37962 [85:36:09<2:18:39,  8.43s/it]
 97%|█████████▋| 36976/37962 [85:36:17<2:16:05,  8.28s/it]
 97%|█████████▋| 36977/37962 [85:36:25<2:15:13,  8.24s/it]
 97%|█████████▋| 36978/37962 [85:36:34<2:16:10,  8.30s/it]
 97%|█████████▋| 36979/37962 [85:36:42<2:14:06,  8.19s/it]
 97%|█████████▋| 36980/37962 [85:36:50<2:14:08,  8.20s/it]
                                                          
{'loss': 1.1473, 'grad_norm': 11.556983947753906, 'learning_rate': 3.5075215108106855e-08, 'epoch': 0.97}

 97%|█████████▋| 36980/37962 [85:36:50<2:14:08,  8.20s/it]
 97%|█████████▋| 36981/37962 [85:36:58<2:12:56,  8.13s/it]
 97%|█████████▋| 36982/37962 [85:37:06<2:13:33,  8.18s/it]
 97%|█████████▋| 36983/37962 [85:37:14<2:13:53,  8.21s/it]
 97%|█████████▋| 36984/37962 [85:37:23<2:13:59,  8.22s/it]
 97%|█████████▋| 36985/37962 [85:37:31<2:15:05,  8.30s/it]
 97%|█████████▋| 36986/37962 [85:37:47<2:54:19, 10.72s/it]
 97%|█████████▋| 36987/37962 [85:37:55<2:40:49,  9.90s/it]
 97%|█████████▋| 36988/37962 [85:38:04<2:33:02,  9.43s/it]
 97%|█████████▋| 36989/37962 [85:38:12<2:27:16,  9.08s/it]
 97%|█████████▋| 36990/37962 [85:38:20<2:23:29,  8.86s/it]
                                                          
{'loss': 1.1134, 'grad_norm': 13.247139930725098, 'learning_rate': 3.4364896910034084e-08, 'epoch': 0.97}

 97%|█████████▋| 36990/37962 [85:38:20<2:23:29,  8.86s/it]
 97%|█████████▋| 36991/37962 [85:38:29<2:20:39,  8.69s/it]
 97%|█████████▋| 36992/37962 [85:38:36<2:15:35,  8.39s/it]
 97%|█████████▋| 36993/37962 [85:38:45<2:14:41,  8.34s/it]
 97%|█████████▋| 36994/37962 [85:38:53<2:14:57,  8.36s/it]
 97%|█████████▋| 36995/37962 [85:39:01<2:14:52,  8.37s/it]
 97%|█████████▋| 36996/37962 [85:39:10<2:14:47,  8.37s/it]
 97%|█████████▋| 36997/37962 [85:39:18<2:14:55,  8.39s/it]
 97%|█████████▋| 36998/37962 [85:39:26<2:14:12,  8.35s/it]
 97%|█████████▋| 36999/37962 [85:39:35<2:13:48,  8.34s/it]
 97%|█████████▋| 37000/37962 [85:39:43<2:13:05,  8.30s/it]
                                                          
{'loss': 1.1546, 'grad_norm': 9.239654541015625, 'learning_rate': 3.366183252510924e-08, 'epoch': 0.97}

 97%|█████████▋| 37000/37962 [85:39:43<2:13:05,  8.30s/it]/home/dungpq6/Project/JanusVLN_SpatialStack/src/qwen_vl/model/modeling_qwen2_5_vl.py:2620: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/anaconda3/envs/janusvln/lib/python3.10/site-packages/torch/utils/checkpoint.py:86: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

 97%|█████████▋| 37001/37962 [85:40:33<5:31:49, 20.72s/it]
 97%|█████████▋| 37002/37962 [85:40:41<4:31:34, 16.97s/it]
 97%|█████████▋| 37003/37962 [85:40:49<3:47:52, 14.26s/it]
 97%|█████████▋| 37004/37962 [85:40:57<3:17:09, 12.35s/it]
 97%|█████████▋| 37005/37962 [85:41:05<2:57:55, 11.15s/it]
 97%|█████████▋| 37006/37962 [85:41:14<2:45:03, 10.36s/it]
 97%|█████████▋| 37007/37962 [85:41:22<2:34:11,  9.69s/it]
 97%|█████████▋| 37008/37962 [85:41:30<2:27:23,  9.27s/it]
 97%|█████████▋| 37009/37962 [85:41:38<2:22:12,  8.95s/it]
 97%|█████████▋| 37010/37962 [85:41:46<2:18:24,  8.72s/it]
                                                          
{'loss': 1.025, 'grad_norm': 9.277846336364746, 'learning_rate': 3.296602246508296e-08, 'epoch': 0.97}

 97%|█████████▋| 37010/37962 [85:41:46<2:18:24,  8.72s/it]
 97%|█████████▋| 37011/37962 [85:41:55<2:15:45,  8.56s/it]
 97%|█████████▋| 37012/37962 [85:42:03<2:15:13,  8.54s/it]
 98%|█████████▊| 37013/37962 [85:42:11<2:13:28,  8.44s/it]
 98%|█████████▊| 37014/37962 [85:42:19<2:11:34,  8.33s/it]
 98%|█████████▊| 37015/37962 [85:42:28<2:10:49,  8.29s/it]
 98%|█████████▊| 37016/37962 [85:42:36<2:11:24,  8.33s/it]
 98%|█████████▊| 37017/37962 [85:42:45<2:12:12,  8.39s/it]
 98%|█████████▊| 37018/37962 [85:42:53<2:11:35,  8.36s/it]
 98%|█████████▊| 37019/37962 [85:43:01<2:11:34,  8.37s/it]
 98%|█████████▊| 37020/37962 [85:43:09<2:10:11,  8.29s/it]
                                                          
{'loss': 1.2091, 'grad_norm': 10.646523475646973, 'learning_rate': 3.227746723642122e-08, 'epoch': 0.98}

 98%|█████████▊| 37020/37962 [85:43:09<2:10:11,  8.29s/it]
 98%|█████████▊| 37021/37962 [85:43:18<2:10:34,  8.33s/it]
 98%|█████████▊| 37022/37962 [85:43:26<2:10:19,  8.32s/it]
 98%|█████████▊| 37023/37962 [85:43:34<2:10:21,  8.33s/it]
 98%|█████████▊| 37024/37962 [85:43:43<2:09:58,  8.31s/it]
 98%|█████████▊| 37025/37962 [85:43:51<2:09:23,  8.29s/it]
 98%|█████████▊| 37026/37962 [85:43:59<2:08:34,  8.24s/it]
 98%|█████████▊| 37027/37962 [85:44:07<2:07:35,  8.19s/it]
 98%|█████████▊| 37028/37962 [85:44:16<2:08:56,  8.28s/it]
 98%|█████████▊| 37029/37962 [85:44:24<2:09:54,  8.35s/it]
 98%|█████████▊| 37030/37962 [85:44:32<2:08:46,  8.29s/it]
                                                          
{'loss': 0.9881, 'grad_norm': 10.912693977355957, 'learning_rate': 3.15961673403109e-08, 'epoch': 0.98}

 98%|█████████▊| 37030/37962 [85:44:32<2:08:46,  8.29s/it]
 98%|█████████▊| 37031/37962 [85:44:40<2:08:12,  8.26s/it]
 98%|█████████▊| 37032/37962 [85:44:49<2:09:29,  8.35s/it]
 98%|█████████▊| 37033/37962 [85:44:57<2:08:48,  8.32s/it]
 98%|█████████▊| 37034/37962 [85:45:05<2:06:36,  8.19s/it]
 98%|█████████▊| 37035/37962 [85:45:13<2:05:25,  8.12s/it]
 98%|█████████▊| 37036/37962 [85:45:21<2:06:04,  8.17s/it]
 98%|█████████▊| 37037/37962 [85:45:30<2:06:25,  8.20s/it]
 98%|█████████▊| 37038/37962 [85:45:38<2:07:05,  8.25s/it]
 98%|█████████▊| 37039/37962 [85:45:46<2:06:49,  8.24s/it]
 98%|█████████▊| 37040/37962 [85:45:54<2:06:43,  8.25s/it]
                                                          
{'loss': 1.041, 'grad_norm': 10.229850769042969, 'learning_rate': 3.0922123272660865e-08, 'epoch': 0.98}

 98%|█████████▊| 37040/37962 [85:45:54<2:06:43,  8.25s/it]
 98%|█████████▊| 37041/37962 [85:46:03<2:07:25,  8.30s/it]
 98%|█████████▊| 37042/37962 [85:46:11<2:07:06,  8.29s/it]
 98%|█████████▊| 37043/37962 [85:46:19<2:07:08,  8.30s/it]
 98%|█████████▊| 37044/37962 [85:46:28<2:07:01,  8.30s/it]
 98%|█████████▊| 37045/37962 [85:46:36<2:07:44,  8.36s/it]
 98%|█████████▊| 37046/37962 [85:46:45<2:07:25,  8.35s/it]
 98%|█████████▊| 37047/37962 [85:46:53<2:06:42,  8.31s/it]
 98%|█████████▊| 37048/37962 [85:47:01<2:05:09,  8.22s/it]
 98%|█████████▊| 37049/37962 [85:47:09<2:06:24,  8.31s/it]
 98%|█████████▊| 37050/37962 [85:47:18<2:06:57,  8.35s/it]
                                                          
{'loss': 1.2437, 'grad_norm': 16.380971908569336, 'learning_rate': 3.025533552409421e-08, 'epoch': 0.98}

 98%|█████████▊| 37050/37962 [85:47:18<2:06:57,  8.35s/it]
 98%|█████████▊| 37051/37962 [85:47:26<2:06:13,  8.31s/it]
 98%|█████████▊| 37052/37962 [85:47:34<2:05:57,  8.31s/it]
 98%|█████████▊| 37053/37962 [85:47:43<2:07:28,  8.41s/it]
 98%|█████████▊| 37054/37962 [85:47:51<2:07:17,  8.41s/it]
 98%|█████████▊| 37055/37962 [85:47:59<2:05:48,  8.32s/it]
 98%|█████████▊| 37056/37962 [85:48:08<2:05:33,  8.32s/it]
 98%|█████████▊| 37057/37962 [85:48:16<2:05:18,  8.31s/it]
 98%|█████████▊| 37058/37962 [85:48:24<2:05:03,  8.30s/it]
 98%|█████████▊| 37059/37962 [85:48:33<2:04:45,  8.29s/it]
 98%|█████████▊| 37060/37962 [85:48:41<2:04:20,  8.27s/it]
                                                          
{'loss': 1.2084, 'grad_norm': 10.567758560180664, 'learning_rate': 2.9595804579956034e-08, 'epoch': 0.98}

 98%|█████████▊| 37060/37962 [85:48:41<2:04:20,  8.27s/it]
 98%|█████████▊| 37061/37962 [85:48:49<2:03:57,  8.25s/it]
 98%|█████████▊| 37062/37962 [85:48:57<2:04:00,  8.27s/it]
 98%|█████████▊| 37063/37962 [85:49:06<2:03:46,  8.26s/it]
 98%|█████████▊| 37064/37962 [85:49:14<2:03:38,  8.26s/it]
 98%|█████████▊| 37065/37962 [85:49:22<2:04:03,  8.30s/it]
 98%|█████████▊| 37066/37962 [85:49:31<2:04:40,  8.35s/it]
 98%|█████████▊| 37067/37962 [85:49:39<2:03:49,  8.30s/it]
 98%|█████████▊| 37068/37962 [85:49:47<2:03:50,  8.31s/it]
 98%|█████████▊| 37069/37962 [85:49:56<2:03:24,  8.29s/it]
 98%|█████████▊| 37070/37962 [85:50:04<2:03:56,  8.34s/it]
                                                          
{'loss': 1.1905, 'grad_norm': 10.075678825378418, 'learning_rate': 2.894353092030566e-08, 'epoch': 0.98}

 98%|█████████▊| 37070/37962 [85:50:04<2:03:56,  8.34s/it]
 98%|█████████▊| 37071/37962 [85:50:12<2:03:06,  8.29s/it]
 98%|█████████▊| 37072/37962 [85:50:20<2:01:20,  8.18s/it]
 98%|█████████▊| 37073/37962 [85:50:28<2:00:51,  8.16s/it]
 98%|█████████▊| 37074/37962 [85:50:36<2:01:33,  8.21s/it]
 98%|█████████▊| 37075/37962 [85:50:45<2:02:26,  8.28s/it]
 98%|█████████▊| 37076/37962 [85:50:53<2:01:59,  8.26s/it]
 98%|█████████▊| 37077/37962 [85:51:01<2:00:07,  8.14s/it]
 98%|█████████▊| 37078/37962 [85:51:09<2:00:34,  8.18s/it]
 98%|█████████▊| 37079/37962 [85:51:18<2:01:38,  8.27s/it]
 98%|█████████▊| 37080/37962 [85:51:26<2:01:25,  8.26s/it]
                                                          
{'loss': 1.1378, 'grad_norm': 10.458196640014648, 'learning_rate': 2.82985150199222e-08, 'epoch': 0.98}

 98%|█████████▊| 37080/37962 [85:51:26<2:01:25,  8.26s/it]
 98%|█████████▊| 37081/37962 [85:51:34<2:01:21,  8.27s/it]
 98%|█████████▊| 37082/37962 [85:51:43<2:01:30,  8.28s/it]
 98%|█████████▊| 37083/37962 [85:51:51<2:01:15,  8.28s/it]
 98%|█████████▊| 37084/37962 [85:51:59<2:01:09,  8.28s/it]
 98%|█████████▊| 37085/37962 [85:52:07<2:00:41,  8.26s/it]
 98%|█████████▊| 37086/37962 [85:52:16<2:02:09,  8.37s/it]
 98%|█████████▊| 37087/37962 [85:52:24<2:01:28,  8.33s/it]
 98%|█████████▊| 37088/37962 [85:52:33<2:01:33,  8.35s/it]
 98%|█████████▊| 37089/37962 [85:52:41<2:00:55,  8.31s/it]
 98%|█████████▊| 37090/37962 [85:52:49<2:00:29,  8.29s/it]
                                                          
{'loss': 1.0725, 'grad_norm': 8.47217082977295, 'learning_rate': 2.7660757348301203e-08, 'epoch': 0.98}

 98%|█████████▊| 37090/37962 [85:52:49<2:00:29,  8.29s/it]
 98%|█████████▊| 37091/37962 [85:52:57<2:00:48,  8.32s/it]
 98%|█████████▊| 37092/37962 [85:53:06<2:00:27,  8.31s/it]
 98%|█████████▊| 37093/37962 [85:53:14<1:59:50,  8.27s/it]
 98%|█████████▊| 37094/37962 [85:53:22<2:00:48,  8.35s/it]
 98%|█████████▊| 37095/37962 [85:53:31<2:01:13,  8.39s/it]
 98%|█████████▊| 37096/37962 [85:53:39<2:01:34,  8.42s/it]
 98%|█████████▊| 37097/37962 [85:53:48<2:00:21,  8.35s/it]
 98%|█████████▊| 37098/37962 [85:53:56<1:59:41,  8.31s/it]
 98%|█████████▊| 37099/37962 [85:54:04<2:00:26,  8.37s/it]
 98%|█████████▊| 37100/37962 [85:54:13<1:59:55,  8.35s/it]
                                                          
{'loss': 1.0842, 'grad_norm': 13.67198657989502, 'learning_rate': 2.7030258369656892e-08, 'epoch': 0.98}

 98%|█████████▊| 37100/37962 [85:54:13<1:59:55,  8.35s/it]
 98%|█████████▊| 37101/37962 [85:54:21<1:59:31,  8.33s/it]
 98%|█████████▊| 37102/37962 [85:54:29<1:59:01,  8.30s/it]
 98%|█████████▊| 37103/37962 [85:54:37<1:58:51,  8.30s/it]
 98%|█████████▊| 37104/37962 [85:54:45<1:56:53,  8.17s/it]
 98%|█████████▊| 37105/37962 [85:54:53<1:55:42,  8.10s/it]
 98%|█████████▊| 37106/37962 [85:55:01<1:55:42,  8.11s/it]
 98%|█████████▊| 37107/37962 [85:55:10<1:56:07,  8.15s/it]
 98%|█████████▊| 37108/37962 [85:55:18<1:57:03,  8.22s/it]
 98%|█████████▊| 37109/37962 [85:55:26<1:57:23,  8.26s/it]
 98%|█████████▊| 37110/37962 [85:55:35<1:57:04,  8.25s/it]
                                                          
{'loss': 1.1352, 'grad_norm': 12.729284286499023, 'learning_rate': 2.6407018542917717e-08, 'epoch': 0.98}

 98%|█████████▊| 37110/37962 [85:55:35<1:57:04,  8.25s/it]
 98%|█████████▊| 37111/37962 [85:55:43<1:57:42,  8.30s/it]
 98%|█████████▊| 37112/37962 [85:55:51<1:58:05,  8.34s/it]
 98%|█████████▊| 37113/37962 [85:55:59<1:55:38,  8.17s/it]
 98%|█████████▊| 37114/37962 [85:56:08<1:55:54,  8.20s/it]
 98%|█████████▊| 37115/37962 [85:56:16<1:55:47,  8.20s/it]
 98%|█████████▊| 37116/37962 [85:56:24<1:56:14,  8.24s/it]
 98%|█████████▊| 37117/37962 [85:56:32<1:55:20,  8.19s/it]
 98%|█████████▊| 37118/37962 [85:56:40<1:55:07,  8.18s/it]
 98%|█████████▊| 37119/37962 [85:56:49<1:55:09,  8.20s/it]
 98%|█████████▊| 37120/37962 [85:56:57<1:55:55,  8.26s/it]
                                                          
{'loss': 1.2346, 'grad_norm': 6.472448825836182, 'learning_rate': 2.579103832172858e-08, 'epoch': 0.98}

 98%|█████████▊| 37120/37962 [85:56:57<1:55:55,  8.26s/it]
 98%|█████████▊| 37121/37962 [85:57:05<1:56:09,  8.29s/it]
 98%|█████████▊| 37122/37962 [85:57:14<1:56:23,  8.31s/it]
 98%|█████████▊| 37123/37962 [85:57:22<1:56:11,  8.31s/it]
 98%|█████████▊| 37124/37962 [85:57:30<1:56:27,  8.34s/it]
 98%|█████████▊| 37125/37962 [85:57:39<1:56:16,  8.33s/it]
 98%|█████████▊| 37126/37962 [85:57:47<1:55:46,  8.31s/it]
 98%|█████████▊| 37127/37962 [85:57:55<1:55:15,  8.28s/it]
 98%|█████████▊| 37128/37962 [85:58:03<1:55:10,  8.29s/it]
 98%|█████████▊| 37129/37962 [85:58:12<1:55:07,  8.29s/it]
 98%|█████████▊| 37130/37962 [85:58:20<1:55:00,  8.29s/it]
                                                          
{'loss': 1.1004, 'grad_norm': 10.462120056152344, 'learning_rate': 2.5182318154451934e-08, 'epoch': 0.98}

 98%|█████████▊| 37130/37962 [85:58:20<1:55:00,  8.29s/it]
 98%|█████████▊| 37131/37962 [85:58:28<1:54:36,  8.28s/it]
 98%|█████████▊| 37132/37962 [85:58:36<1:53:44,  8.22s/it]
 98%|█████████▊| 37133/37962 [85:58:45<1:54:57,  8.32s/it]
 98%|█████████▊| 37134/37962 [85:58:53<1:53:38,  8.23s/it]
 98%|█████████▊| 37135/37962 [85:59:01<1:53:45,  8.25s/it]
 98%|█████████▊| 37136/37962 [85:59:10<1:53:34,  8.25s/it]
 98%|█████████▊| 37137/37962 [85:59:18<1:52:42,  8.20s/it]
 98%|█████████▊| 37138/37962 [85:59:26<1:52:06,  8.16s/it]
 98%|█████████▊| 37139/37962 [85:59:34<1:53:25,  8.27s/it]
 98%|█████████▊| 37140/37962 [85:59:42<1:52:58,  8.25s/it]
                                                          
{'loss': 1.0859, 'grad_norm': 9.162874221801758, 'learning_rate': 2.4580858484163362e-08, 'epoch': 0.98}

 98%|█████████▊| 37140/37962 [85:59:42<1:52:58,  8.25s/it]
 98%|█████████▊| 37141/37962 [85:59:51<1:53:20,  8.28s/it]
 98%|█████████▊| 37142/37962 [85:59:59<1:52:51,  8.26s/it]
 98%|█████████▊| 37143/37962 [86:00:07<1:52:36,  8.25s/it]
 98%|█████████▊| 37144/37962 [86:00:15<1:51:48,  8.20s/it]
 98%|█████████▊| 37145/37962 [86:00:24<1:52:06,  8.23s/it]
 98%|█████████▊| 37146/37962 [86:00:32<1:51:55,  8.23s/it]
 98%|█████████▊| 37147/37962 [86:00:40<1:51:40,  8.22s/it]
 98%|█████████▊| 37148/37962 [86:00:48<1:50:35,  8.15s/it]
 98%|█████████▊| 37149/37962 [86:00:56<1:50:25,  8.15s/it]
 98%|█████████▊| 37150/37962 [86:01:04<1:50:56,  8.20s/it]
                                                          
{'loss': 1.2228, 'grad_norm': 27.292343139648438, 'learning_rate': 2.3986659748654883e-08, 'epoch': 0.98}

 98%|█████████▊| 37150/37962 [86:01:04<1:50:56,  8.20s/it]
 98%|█████████▊| 37151/37962 [86:01:13<1:50:31,  8.18s/it]
 98%|█████████▊| 37152/37962 [86:01:21<1:50:49,  8.21s/it]
 98%|█████████▊| 37153/37962 [86:01:29<1:50:54,  8.23s/it]
 98%|█████████▊| 37154/37962 [86:01:37<1:51:00,  8.24s/it]
 98%|█████████▊| 37155/37962 [86:01:46<1:51:12,  8.27s/it]
 98%|█████████▊| 37156/37962 [86:01:54<1:50:43,  8.24s/it]
 98%|█████████▊| 37157/37962 [86:02:02<1:50:26,  8.23s/it]
 98%|█████████▊| 37158/37962 [86:02:11<1:50:55,  8.28s/it]
 98%|█████████▊| 37159/37962 [86:02:19<1:50:16,  8.24s/it]
 98%|█████████▊| 37160/37962 [86:02:27<1:50:08,  8.24s/it]
                                                          
{'loss': 1.1921, 'grad_norm': 5.657218933105469, 'learning_rate': 2.3399722380434974e-08, 'epoch': 0.98}

 98%|█████████▊| 37160/37962 [86:02:27<1:50:08,  8.24s/it]
 98%|█████████▊| 37161/37962 [86:02:35<1:51:11,  8.33s/it]
 98%|█████████▊| 37162/37962 [86:02:44<1:51:51,  8.39s/it]
 98%|█████████▊| 37163/37962 [86:02:52<1:51:51,  8.40s/it]
 98%|█████████▊| 37164/37962 [86:03:01<1:50:41,  8.32s/it]
 98%|█████████▊| 37165/37962 [86:03:09<1:50:05,  8.29s/it]
 98%|█████████▊| 37166/37962 [86:03:17<1:50:39,  8.34s/it]
 98%|█████████▊| 37167/37962 [86:03:25<1:49:50,  8.29s/it]
 98%|█████████▊| 37168/37962 [86:03:34<1:49:44,  8.29s/it]
 98%|█████████▊| 37169/37962 [86:03:42<1:49:37,  8.29s/it]
 98%|█████████▊| 37170/37962 [86:03:50<1:48:59,  8.26s/it]
                                                          
{'loss': 1.1352, 'grad_norm': 9.713508605957031, 'learning_rate': 2.2820046806725227e-08, 'epoch': 0.98}

 98%|█████████▊| 37170/37962 [86:03:50<1:48:59,  8.26s/it]
 98%|█████████▊| 37171/37962 [86:03:58<1:49:09,  8.28s/it]
 98%|█████████▊| 37172/37962 [86:04:07<1:48:02,  8.21s/it]
 98%|█████████▊| 37173/37962 [86:04:15<1:47:53,  8.20s/it]
 98%|█████████▊| 37174/37962 [86:04:23<1:47:27,  8.18s/it]
 98%|█████████▊| 37175/37962 [86:04:31<1:47:46,  8.22s/it]
 98%|█████████▊| 37176/37962 [86:04:39<1:46:21,  8.12s/it]
 98%|█████████▊| 37177/37962 [86:04:47<1:46:39,  8.15s/it]
 98%|█████████▊| 37178/37962 [86:04:56<1:47:06,  8.20s/it]
 98%|█████████▊| 37179/37962 [86:05:04<1:46:28,  8.16s/it]
 98%|█████████▊| 37180/37962 [86:05:12<1:46:50,  8.20s/it]
                                                          
{'loss': 1.1455, 'grad_norm': 25.21521759033203, 'learning_rate': 2.2247633449459237e-08, 'epoch': 0.98}

 98%|█████████▊| 37180/37962 [86:05:12<1:46:50,  8.20s/it]
 98%|█████████▊| 37181/37962 [86:05:20<1:46:44,  8.20s/it]
 98%|█████████▊| 37182/37962 [86:05:28<1:47:05,  8.24s/it]
 98%|█████████▊| 37183/37962 [86:05:36<1:46:11,  8.18s/it]
 98%|█████████▊| 37184/37962 [86:05:45<1:46:40,  8.23s/it]
 98%|█████████▊| 37185/37962 [86:05:53<1:46:46,  8.24s/it]
 98%|█████████▊| 37186/37962 [86:06:01<1:46:57,  8.27s/it]
 98%|█████████▊| 37187/37962 [86:06:10<1:47:05,  8.29s/it]
 98%|█████████▊| 37188/37962 [86:06:18<1:46:58,  8.29s/it]
 98%|█████████▊| 37189/37962 [86:06:26<1:46:41,  8.28s/it]
 98%|█████████▊| 37190/37962 [86:06:34<1:45:29,  8.20s/it]
                                                          
{'loss': 1.1601, 'grad_norm': 9.405596733093262, 'learning_rate': 2.168248272528928e-08, 'epoch': 0.98}

 98%|█████████▊| 37190/37962 [86:06:34<1:45:29,  8.20s/it]
 98%|█████████▊| 37191/37962 [86:06:43<1:45:23,  8.20s/it]
 98%|█████████▊| 37192/37962 [86:06:51<1:45:19,  8.21s/it]
 98%|█████████▊| 37193/37962 [86:06:59<1:45:25,  8.23s/it]
 98%|█████████▊| 37194/37962 [86:07:07<1:44:38,  8.17s/it]
 98%|█████████▊| 37195/37962 [86:07:15<1:44:51,  8.20s/it]
 98%|█████████▊| 37196/37962 [86:07:24<1:45:04,  8.23s/it]
 98%|█████████▊| 37197/37962 [86:07:32<1:43:50,  8.14s/it]
 98%|█████████▊| 37198/37962 [86:07:40<1:43:58,  8.17s/it]
 98%|█████████▊| 37199/37962 [86:07:48<1:45:15,  8.28s/it]
 98%|█████████▊| 37200/37962 [86:07:57<1:44:40,  8.24s/it]
                                                          
{'loss': 1.1553, 'grad_norm': 10.610394477844238, 'learning_rate': 2.11245950455774e-08, 'epoch': 0.98}

 98%|█████████▊| 37200/37962 [86:07:57<1:44:40,  8.24s/it]
 98%|█████████▊| 37201/37962 [86:08:05<1:45:09,  8.29s/it]
 98%|█████████▊| 37202/37962 [86:08:13<1:44:30,  8.25s/it]
 98%|█████████▊| 37203/37962 [86:08:21<1:44:09,  8.23s/it]
 98%|█████████▊| 37204/37962 [86:08:29<1:43:58,  8.23s/it]
 98%|█████████▊| 37205/37962 [86:08:38<1:44:02,  8.25s/it]
 98%|█████████▊| 37206/37962 [86:08:46<1:43:23,  8.21s/it]
 98%|█████████▊| 37207/37962 [86:08:54<1:42:55,  8.18s/it]
 98%|█████████▊| 37208/37962 [86:09:02<1:43:38,  8.25s/it]
 98%|█████████▊| 37209/37962 [86:09:11<1:44:44,  8.35s/it]
 98%|█████████▊| 37210/37962 [86:09:19<1:44:43,  8.36s/it]
                                                          
{'loss': 1.145, 'grad_norm': 8.835819244384766, 'learning_rate': 2.057397081639989e-08, 'epoch': 0.98}

 98%|█████████▊| 37210/37962 [86:09:19<1:44:43,  8.36s/it]
 98%|█████████▊| 37211/37962 [86:09:28<1:44:58,  8.39s/it]
 98%|█████████▊| 37212/37962 [86:09:36<1:43:44,  8.30s/it]
 98%|█████████▊| 37213/37962 [86:09:45<1:44:47,  8.39s/it]
 98%|█████████▊| 37214/37962 [86:09:53<1:44:06,  8.35s/it]
 98%|█████████▊| 37215/37962 [86:10:01<1:44:01,  8.36s/it]
 98%|█████████▊| 37216/37962 [86:10:09<1:43:10,  8.30s/it]
 98%|█████████▊| 37217/37962 [86:10:17<1:42:25,  8.25s/it]
 98%|█████████▊| 37218/37962 [86:10:26<1:42:54,  8.30s/it]
 98%|█████████▊| 37219/37962 [86:10:34<1:43:19,  8.34s/it]
 98%|█████████▊| 37220/37962 [86:10:43<1:42:41,  8.30s/it]
                                                          
{'loss': 1.0828, 'grad_norm': 29.819286346435547, 'learning_rate': 2.003061043854726e-08, 'epoch': 0.98}

 98%|█████████▊| 37220/37962 [86:10:43<1:42:41,  8.30s/it]
 98%|█████████▊| 37221/37962 [86:10:51<1:42:07,  8.27s/it]
 98%|█████████▊| 37222/37962 [86:10:59<1:41:54,  8.26s/it]
 98%|█████████▊| 37223/37962 [86:11:07<1:41:39,  8.25s/it]
 98%|█████████▊| 37224/37962 [86:11:15<1:39:36,  8.10s/it]
 98%|█████████▊| 37225/37962 [86:11:23<1:40:12,  8.16s/it]
 98%|█████████▊| 37226/37962 [86:11:32<1:40:43,  8.21s/it]
 98%|█████████▊| 37227/37962 [86:11:40<1:40:58,  8.24s/it]
 98%|█████████▊| 37228/37962 [86:11:48<1:41:14,  8.28s/it]
 98%|█████████▊| 37229/37962 [86:11:57<1:41:26,  8.30s/it]
 98%|█████████▊| 37230/37962 [86:12:05<1:41:24,  8.31s/it]
                                                          
{'loss': 1.0038, 'grad_norm': 8.96506404876709, 'learning_rate': 1.9494514307523137e-08, 'epoch': 0.98}

 98%|█████████▊| 37230/37962 [86:12:05<1:41:24,  8.31s/it]
 98%|█████████▊| 37231/37962 [86:12:13<1:39:21,  8.16s/it]
 98%|█████████▊| 37232/37962 [86:12:20<1:37:47,  8.04s/it]
 98%|█████████▊| 37233/37962 [86:12:29<1:38:37,  8.12s/it]
 98%|█████████▊| 37234/37962 [86:12:37<1:39:44,  8.22s/it]
 98%|█████████▊| 37235/37962 [86:12:45<1:39:23,  8.20s/it]
 98%|█████████▊| 37236/37962 [86:12:54<1:38:59,  8.18s/it]
 98%|█████████▊| 37237/37962 [86:13:02<1:39:48,  8.26s/it]
 98%|█████████▊| 37238/37962 [86:13:10<1:39:46,  8.27s/it]
 98%|█████████▊| 37239/37962 [86:13:19<1:39:28,  8.25s/it]
 98%|█████████▊| 37240/37962 [86:13:27<1:38:54,  8.22s/it]
                                                          
{'loss': 1.0476, 'grad_norm': 9.260284423828125, 'learning_rate': 1.896568281354094e-08, 'epoch': 0.98}

 98%|█████████▊| 37240/37962 [86:13:27<1:38:54,  8.22s/it]
 98%|█████████▊| 37241/37962 [86:13:35<1:39:39,  8.29s/it]
 98%|█████████▊| 37242/37962 [86:13:43<1:39:45,  8.31s/it]
 98%|█████████▊| 37243/37962 [86:13:52<1:39:32,  8.31s/it]
 98%|█████████▊| 37244/37962 [86:14:00<1:38:50,  8.26s/it]
 98%|█████████▊| 37245/37962 [86:14:08<1:38:34,  8.25s/it]
 98%|█████████▊| 37246/37962 [86:14:16<1:38:06,  8.22s/it]
 98%|█████████▊| 37247/37962 [86:14:25<1:38:29,  8.27s/it]
 98%|█████████▊| 37248/37962 [86:14:33<1:37:50,  8.22s/it]
 98%|█████████▊| 37249/37962 [86:14:41<1:38:39,  8.30s/it]
 98%|█████████▊| 37250/37962 [86:14:50<1:38:46,  8.32s/it]
                                                          
{'loss': 1.0549, 'grad_norm': 9.093425750732422, 'learning_rate': 1.844411634152943e-08, 'epoch': 0.98}

 98%|█████████▊| 37250/37962 [86:14:50<1:38:46,  8.32s/it]
 98%|█████████▊| 37251/37962 [86:14:58<1:39:04,  8.36s/it]
 98%|█████████▊| 37252/37962 [86:15:06<1:37:26,  8.24s/it]
 98%|█████████▊| 37253/37962 [86:15:14<1:37:07,  8.22s/it]
 98%|█████████▊| 37254/37962 [86:15:22<1:35:51,  8.12s/it]
 98%|█████████▊| 37255/37962 [86:15:31<1:36:43,  8.21s/it]
 98%|█████████▊| 37256/37962 [86:15:39<1:37:27,  8.28s/it]
 98%|█████████▊| 37257/37962 [86:15:47<1:37:18,  8.28s/it]
 98%|█████████▊| 37258/37962 [86:15:56<1:37:05,  8.27s/it]
 98%|█████████▊| 37259/37962 [86:16:04<1:36:42,  8.25s/it]
 98%|█████████▊| 37260/37962 [86:16:12<1:36:35,  8.26s/it]
                                                          
{'loss': 1.1138, 'grad_norm': 16.045421600341797, 'learning_rate': 1.7929815271126028e-08, 'epoch': 0.98}

 98%|█████████▊| 37260/37962 [86:16:12<1:36:35,  8.26s/it]
 98%|█████████▊| 37261/37962 [86:16:20<1:37:06,  8.31s/it]
 98%|█████████▊| 37262/37962 [86:16:29<1:36:54,  8.31s/it]
 98%|█████████▊| 37263/37962 [86:16:37<1:36:57,  8.32s/it]
 98%|█████████▊| 37264/37962 [86:16:46<1:37:42,  8.40s/it]
 98%|█████████▊| 37265/37962 [86:16:54<1:36:57,  8.35s/it]
 98%|█████████▊| 37266/37962 [86:17:02<1:36:34,  8.32s/it]
 98%|█████████▊| 37267/37962 [86:17:10<1:36:30,  8.33s/it]
 98%|█████████▊| 37268/37962 [86:17:19<1:36:30,  8.34s/it]
 98%|█████████▊| 37269/37962 [86:17:27<1:35:51,  8.30s/it]
 98%|█████████▊| 37270/37962 [86:17:35<1:35:50,  8.31s/it]
                                                          
{'loss': 1.2042, 'grad_norm': 7.838864326477051, 'learning_rate': 1.7422779976682403e-08, 'epoch': 0.98}

 98%|█████████▊| 37270/37962 [86:17:35<1:35:50,  8.31s/it]
 98%|█████████▊| 37271/37962 [86:17:44<1:35:36,  8.30s/it]
 98%|█████████▊| 37272/37962 [86:17:52<1:35:06,  8.27s/it]
 98%|█████████▊| 37273/37962 [86:18:00<1:34:28,  8.23s/it]
 98%|█████████▊| 37274/37962 [86:18:08<1:35:03,  8.29s/it]
 98%|█████████▊| 37275/37962 [86:18:17<1:35:05,  8.30s/it]
 98%|█████████▊| 37276/37962 [86:18:25<1:34:56,  8.30s/it]
 98%|█████████▊| 37277/37962 [86:18:33<1:34:21,  8.26s/it]
 98%|█████████▊| 37278/37962 [86:18:42<1:34:10,  8.26s/it]
 98%|█████████▊| 37279/37962 [86:18:50<1:34:05,  8.27s/it]
 98%|█████████▊| 37280/37962 [86:18:58<1:34:02,  8.27s/it]
                                                          
{'loss': 1.1571, 'grad_norm': 11.226157188415527, 'learning_rate': 1.692301082726222e-08, 'epoch': 0.98}

 98%|█████████▊| 37280/37962 [86:18:58<1:34:02,  8.27s/it]
 98%|█████████▊| 37281/37962 [86:19:07<1:34:47,  8.35s/it]
 98%|█████████▊| 37282/37962 [86:19:15<1:34:44,  8.36s/it]
 98%|█████████▊| 37283/37962 [86:19:23<1:33:10,  8.23s/it]
 98%|█████████▊| 37284/37962 [86:19:31<1:32:23,  8.18s/it]
 98%|█████████▊| 37285/37962 [86:19:39<1:33:04,  8.25s/it]
 98%|█████████▊| 37286/37962 [86:19:48<1:32:52,  8.24s/it]
 98%|█████████▊| 37287/37962 [86:19:56<1:32:34,  8.23s/it]
 98%|█████████▊| 37288/37962 [86:20:04<1:31:17,  8.13s/it]
 98%|█████████▊| 37289/37962 [86:20:12<1:32:18,  8.23s/it]
 98%|█████████▊| 37290/37962 [86:20:20<1:32:13,  8.23s/it]
                                                          
{'loss': 1.1458, 'grad_norm': 12.788179397583008, 'learning_rate': 1.6430508186636717e-08, 'epoch': 0.98}

 98%|█████████▊| 37290/37962 [86:20:20<1:32:13,  8.23s/it]
 98%|█████████▊| 37291/37962 [86:20:29<1:32:22,  8.26s/it]
 98%|█████████▊| 37292/37962 [86:20:37<1:31:48,  8.22s/it]
 98%|█████████▊| 37293/37962 [86:20:45<1:31:04,  8.17s/it]
 98%|█████████▊| 37294/37962 [86:20:53<1:31:08,  8.19s/it]
 98%|█████████▊| 37295/37962 [86:21:09<1:57:18, 10.55s/it]
 98%|█████████▊| 37296/37962 [86:21:17<1:49:21,  9.85s/it]
 98%|█████████▊| 37297/37962 [86:21:25<1:42:53,  9.28s/it]
 98%|█████████▊| 37298/37962 [86:21:34<1:39:22,  8.98s/it]
 98%|█████████▊| 37299/37962 [86:21:42<1:36:44,  8.76s/it]
 98%|█████████▊| 37300/37962 [86:21:50<1:34:41,  8.58s/it]
                                                          
{'loss': 1.1564, 'grad_norm': 11.170735359191895, 'learning_rate': 1.594527241329136e-08, 'epoch': 0.98}

 98%|█████████▊| 37300/37962 [86:21:50<1:34:41,  8.58s/it]
 98%|█████████▊| 37301/37962 [86:21:58<1:33:39,  8.50s/it]
 98%|█████████▊| 37302/37962 [86:22:06<1:31:49,  8.35s/it]
 98%|█████████▊| 37303/37962 [86:22:15<1:31:04,  8.29s/it]
 98%|█████████▊| 37304/37962 [86:22:23<1:30:32,  8.26s/it]
 98%|█████████▊| 37305/37962 [86:22:31<1:30:30,  8.27s/it]
 98%|█████████▊| 37306/37962 [86:22:39<1:29:39,  8.20s/it]
 98%|█████████▊| 37307/37962 [86:22:47<1:29:53,  8.23s/it]
 98%|█████████▊| 37308/37962 [86:22:56<1:30:13,  8.28s/it]
 98%|█████████▊| 37309/37962 [86:23:04<1:29:23,  8.21s/it]
 98%|█████████▊| 37310/37962 [86:23:12<1:29:31,  8.24s/it]
                                                          
{'loss': 1.0764, 'grad_norm': 14.343019485473633, 'learning_rate': 1.546730386041917e-08, 'epoch': 0.98}

 98%|█████████▊| 37310/37962 [86:23:12<1:29:31,  8.24s/it]
 98%|█████████▊| 37311/37962 [86:23:20<1:28:43,  8.18s/it]
 98%|█████████▊| 37312/37962 [86:23:28<1:27:49,  8.11s/it]
 98%|█████████▊| 37313/37962 [86:23:36<1:28:17,  8.16s/it]
 98%|█████████▊| 37314/37962 [86:23:45<1:28:39,  8.21s/it]
 98%|█████████▊| 37315/37962 [86:23:53<1:28:54,  8.25s/it]
 98%|█████████▊| 37316/37962 [86:24:01<1:28:11,  8.19s/it]
 98%|█████████▊| 37317/37962 [86:24:10<1:28:48,  8.26s/it]
 98%|█████████▊| 37318/37962 [86:24:18<1:28:23,  8.24s/it]
 98%|█████████▊| 37319/37962 [86:24:26<1:28:08,  8.22s/it]
 98%|█████████▊| 37320/37962 [86:24:34<1:28:06,  8.23s/it]
                                                          
{'loss': 1.0768, 'grad_norm': 9.570619583129883, 'learning_rate': 1.4996602875927412e-08, 'epoch': 0.98}

 98%|█████████▊| 37320/37962 [86:24:34<1:28:06,  8.23s/it]
 98%|█████████▊| 37321/37962 [86:24:42<1:26:55,  8.14s/it]
 98%|█████████▊| 37322/37962 [86:24:50<1:27:22,  8.19s/it]
 98%|█████████▊| 37323/37962 [86:24:59<1:27:36,  8.23s/it]
 98%|█████████▊| 37324/37962 [86:25:07<1:27:41,  8.25s/it]
 98%|█████████▊| 37325/37962 [86:25:15<1:27:23,  8.23s/it]
 98%|█████████▊| 37326/37962 [86:25:24<1:27:54,  8.29s/it]
 98%|█████████▊| 37327/37962 [86:25:32<1:27:50,  8.30s/it]
 98%|█████████▊| 37328/37962 [86:25:40<1:27:24,  8.27s/it]
 98%|█████████▊| 37329/37962 [86:25:48<1:27:06,  8.26s/it]
 98%|█████████▊| 37330/37962 [86:25:57<1:26:54,  8.25s/it]
                                                          
{'loss': 1.0995, 'grad_norm': 11.917278289794922, 'learning_rate': 1.4533169802429803e-08, 'epoch': 0.98}

 98%|█████████▊| 37330/37962 [86:25:57<1:26:54,  8.25s/it]
 98%|█████████▊| 37331/37962 [86:26:05<1:27:36,  8.33s/it]
 98%|█████████▊| 37332/37962 [86:26:13<1:26:27,  8.23s/it]
 98%|█████████▊| 37333/37962 [86:26:21<1:25:55,  8.20s/it]
 98%|█████████▊| 37334/37962 [86:26:29<1:25:57,  8.21s/it]
 98%|█████████▊| 37335/37962 [86:26:38<1:26:24,  8.27s/it]
 98%|█████████▊| 37336/37962 [86:26:46<1:26:00,  8.24s/it]
 98%|█████████▊| 37337/37962 [86:26:54<1:25:44,  8.23s/it]
 98%|█████████▊| 37338/37962 [86:27:03<1:25:47,  8.25s/it]
 98%|█████████▊| 37339/37962 [86:27:11<1:26:00,  8.28s/it]
 98%|█████████▊| 37340/37962 [86:27:19<1:25:38,  8.26s/it]
                                                          
{'loss': 1.0946, 'grad_norm': 12.944307327270508, 'learning_rate': 1.4077004977250952e-08, 'epoch': 0.98}

 98%|█████████▊| 37340/37962 [86:27:19<1:25:38,  8.26s/it]
 98%|█████████▊| 37341/37962 [86:27:27<1:25:32,  8.27s/it]
 98%|█████████▊| 37342/37962 [86:27:35<1:24:36,  8.19s/it]
 98%|█████████▊| 37343/37962 [86:27:44<1:25:48,  8.32s/it]
 98%|█████████▊| 37344/37962 [86:27:52<1:25:19,  8.28s/it]
 98%|█████████▊| 37345/37962 [86:28:01<1:25:20,  8.30s/it]
 98%|█████████▊| 37346/37962 [86:28:09<1:24:28,  8.23s/it]
 98%|█████████▊| 37347/37962 [86:28:17<1:24:58,  8.29s/it]
 98%|█████████▊| 37348/37962 [86:28:25<1:24:48,  8.29s/it]
 98%|█████████▊| 37349/37962 [86:28:34<1:25:24,  8.36s/it]
 98%|█████████▊| 37350/37962 [86:28:42<1:24:53,  8.32s/it]
                                                          
{'loss': 1.1488, 'grad_norm': 8.70278263092041, 'learning_rate': 1.3628108732426371e-08, 'epoch': 0.98}

 98%|█████████▊| 37350/37962 [86:28:42<1:24:53,  8.32s/it]
 98%|█████████▊| 37351/37962 [86:28:51<1:25:11,  8.37s/it]
 98%|█████████▊| 37352/37962 [86:28:59<1:24:09,  8.28s/it]
 98%|█████████▊| 37353/37962 [86:29:07<1:23:15,  8.20s/it]
 98%|█████████▊| 37354/37962 [86:29:15<1:23:03,  8.20s/it]
 98%|█████████▊| 37355/37962 [86:29:23<1:23:03,  8.21s/it]
 98%|█████████▊| 37356/37962 [86:29:32<1:23:43,  8.29s/it]
 98%|█████████▊| 37357/37962 [86:29:40<1:23:33,  8.29s/it]
 98%|█████████▊| 37358/37962 [86:29:48<1:22:18,  8.18s/it]
 98%|█████████▊| 37359/37962 [86:29:56<1:21:09,  8.08s/it]
 98%|█████████▊| 37360/37962 [86:30:04<1:21:45,  8.15s/it]
                                                          
{'loss': 1.1978, 'grad_norm': 11.318720817565918, 'learning_rate': 1.3186481394699135e-08, 'epoch': 0.98}

 98%|█████████▊| 37360/37962 [86:30:04<1:21:45,  8.15s/it]
 98%|█████████▊| 37361/37962 [86:30:12<1:21:54,  8.18s/it]
 98%|█████████▊| 37362/37962 [86:30:20<1:21:52,  8.19s/it]
 98%|█████████▊| 37363/37962 [86:30:29<1:22:09,  8.23s/it]
 98%|█████████▊| 37364/37962 [86:30:37<1:22:42,  8.30s/it]
 98%|█████████▊| 37365/37962 [86:30:45<1:22:21,  8.28s/it]
 98%|█████████▊| 37366/37962 [86:30:54<1:22:38,  8.32s/it]
 98%|█████████▊| 37367/37962 [86:31:02<1:21:45,  8.25s/it]
 98%|█████████▊| 37368/37962 [86:31:10<1:22:20,  8.32s/it]
 98%|█████████▊| 37369/37962 [86:31:19<1:22:29,  8.35s/it]
 98%|█████████▊| 37370/37962 [86:31:27<1:21:49,  8.29s/it]
                                                          
{'loss': 1.0356, 'grad_norm': 8.020891189575195, 'learning_rate': 1.2752123285522111e-08, 'epoch': 0.98}

 98%|█████████▊| 37370/37962 [86:31:27<1:21:49,  8.29s/it]
 98%|█████████▊| 37371/37962 [86:31:35<1:21:58,  8.32s/it]
 98%|█████████▊| 37372/37962 [86:31:43<1:21:16,  8.26s/it]
 98%|█████████▊| 37373/37962 [86:31:52<1:21:38,  8.32s/it]
 98%|█████████▊| 37374/37962 [86:32:00<1:21:35,  8.33s/it]
 98%|█████████▊| 37375/37962 [86:32:09<1:21:26,  8.32s/it]
 98%|█████████▊| 37376/37962 [86:32:17<1:21:06,  8.30s/it]
 98%|█████████▊| 37377/37962 [86:32:25<1:21:12,  8.33s/it]
 98%|█████████▊| 37378/37962 [86:32:34<1:20:59,  8.32s/it]
 98%|█████████▊| 37379/37962 [86:32:42<1:20:48,  8.32s/it]
 98%|█████████▊| 37380/37962 [86:32:50<1:20:24,  8.29s/it]
                                                          
{'loss': 1.1697, 'grad_norm': 6.4645538330078125, 'learning_rate': 1.232503472105795e-08, 'epoch': 0.98}

 98%|█████████▊| 37380/37962 [86:32:50<1:20:24,  8.29s/it]
 98%|█████████▊| 37381/37962 [86:32:58<1:19:48,  8.24s/it]
 98%|█████████▊| 37382/37962 [86:33:06<1:19:43,  8.25s/it]
 98%|█████████▊| 37383/37962 [86:33:15<1:19:35,  8.25s/it]
 98%|█████████▊| 37384/37962 [86:33:23<1:19:40,  8.27s/it]
 98%|█████████▊| 37385/37962 [86:33:31<1:20:07,  8.33s/it]
 98%|█████████▊| 37386/37962 [86:33:39<1:18:43,  8.20s/it]
 98%|█████████▊| 37387/37962 [86:33:48<1:19:05,  8.25s/it]
 98%|█████████▊| 37388/37962 [86:33:56<1:19:04,  8.27s/it]
 98%|█████████▊| 37389/37962 [86:34:04<1:19:22,  8.31s/it]
 98%|█████████▊| 37390/37962 [86:34:12<1:17:50,  8.17s/it]
                                                          
{'loss': 1.2071, 'grad_norm': 15.600767135620117, 'learning_rate': 1.1905216012176869e-08, 'epoch': 0.98}

 98%|█████████▊| 37390/37962 [86:34:12<1:17:50,  8.17s/it]
 98%|█████████▊| 37391/37962 [86:34:21<1:18:12,  8.22s/it]
 98%|█████████▊| 37392/37962 [86:34:28<1:16:02,  8.00s/it]
 99%|█████████▊| 37393/37962 [86:34:36<1:16:46,  8.10s/it]
 99%|█████████▊| 37394/37962 [86:34:45<1:17:01,  8.14s/it]
 99%|█████████▊| 37395/37962 [86:34:53<1:17:13,  8.17s/it]
 99%|█████████▊| 37396/37962 [86:35:01<1:17:45,  8.24s/it]
 99%|█████████▊| 37397/37962 [86:35:10<1:18:00,  8.28s/it]
 99%|█████████▊| 37398/37962 [86:35:18<1:18:00,  8.30s/it]
 99%|█████████▊| 37399/37962 [86:35:26<1:16:40,  8.17s/it]
 99%|█████████▊| 37400/37962 [86:35:34<1:16:50,  8.20s/it]
                                                          
{'loss': 1.1402, 'grad_norm': 14.369098663330078, 'learning_rate': 1.1492667464456652e-08, 'epoch': 0.99}

 99%|█████████▊| 37400/37962 [86:35:34<1:16:50,  8.20s/it]
 99%|█████████▊| 37401/37962 [86:35:43<1:17:09,  8.25s/it]
 99%|█████████▊| 37402/37962 [86:35:51<1:17:12,  8.27s/it]
 99%|█████████▊| 37403/37962 [86:35:59<1:16:25,  8.20s/it]
 99%|█████████▊| 37404/37962 [86:36:07<1:15:36,  8.13s/it]
 99%|█████████▊| 37405/37962 [86:36:15<1:15:24,  8.12s/it]
 99%|█████████▊| 37406/37962 [86:36:23<1:16:13,  8.23s/it]
 99%|█████████▊| 37407/37962 [86:36:32<1:15:52,  8.20s/it]
 99%|█████████▊| 37408/37962 [86:36:40<1:16:16,  8.26s/it]
 99%|█████████▊| 37409/37962 [86:36:48<1:16:07,  8.26s/it]
 99%|█████████▊| 37410/37962 [86:36:57<1:16:10,  8.28s/it]
                                                          
{'loss': 1.0541, 'grad_norm': 12.011136054992676, 'learning_rate': 1.1087389378184877e-08, 'epoch': 0.99}

 99%|█████████▊| 37410/37962 [86:36:57<1:16:10,  8.28s/it]
 99%|█████████▊| 37411/37962 [86:37:05<1:16:11,  8.30s/it]
 99%|█████████▊| 37412/37962 [86:37:13<1:15:14,  8.21s/it]
 99%|█████████▊| 37413/37962 [86:37:21<1:15:15,  8.23s/it]
 99%|█████████▊| 37414/37962 [86:37:29<1:15:12,  8.23s/it]
 99%|█████████▊| 37415/37962 [86:37:38<1:15:13,  8.25s/it]
 99%|█████████▊| 37416/37962 [86:37:46<1:15:10,  8.26s/it]
 99%|█████████▊| 37417/37962 [86:37:54<1:14:52,  8.24s/it]
 99%|█████████▊| 37418/37962 [86:38:02<1:14:38,  8.23s/it]
 99%|█████████▊| 37419/37962 [86:38:11<1:14:31,  8.24s/it]
 99%|█████████▊| 37420/37962 [86:38:18<1:13:04,  8.09s/it]
                                                          
{'loss': 1.1509, 'grad_norm': 11.413780212402344, 'learning_rate': 1.0689382048355567e-08, 'epoch': 0.99}

 99%|█████████▊| 37420/37962 [86:38:18<1:13:04,  8.09s/it]
 99%|█████████▊| 37421/37962 [86:38:34<1:34:00, 10.43s/it]
 99%|█████████▊| 37422/37962 [86:38:43<1:28:49,  9.87s/it]
 99%|█████████▊| 37423/37962 [86:38:51<1:24:49,  9.44s/it]
 99%|█████████▊| 37424/37962 [86:39:00<1:21:19,  9.07s/it]
 99%|█████████▊| 37425/37962 [86:39:08<1:19:19,  8.86s/it]
 99%|█████████▊| 37426/37962 [86:39:16<1:17:38,  8.69s/it]
 99%|█████████▊| 37427/37962 [86:39:24<1:15:28,  8.46s/it]
 99%|█████████▊| 37428/37962 [86:39:32<1:14:07,  8.33s/it]
 99%|█████████▊| 37429/37962 [86:39:40<1:13:25,  8.27s/it]
 99%|█████████▊| 37430/37962 [86:39:49<1:13:12,  8.26s/it]
                                                          
{'loss': 1.0617, 'grad_norm': 13.161310195922852, 'learning_rate': 1.0298645764672543e-08, 'epoch': 0.99}

 99%|█████████▊| 37430/37962 [86:39:49<1:13:12,  8.26s/it]
 99%|█████████▊| 37431/37962 [86:39:57<1:13:15,  8.28s/it]
 99%|█████████▊| 37432/37962 [86:40:05<1:13:10,  8.28s/it]
 99%|█████████▊| 37433/37962 [86:40:13<1:11:58,  8.16s/it]
 99%|█████████▊| 37434/37962 [86:40:21<1:12:19,  8.22s/it]
 99%|█████████▊| 37435/37962 [86:40:30<1:13:04,  8.32s/it]
 99%|█████████▊| 37436/37962 [86:40:38<1:12:46,  8.30s/it]
 99%|█████████▊| 37437/37962 [86:40:46<1:12:26,  8.28s/it]
 99%|█████████▊| 37438/37962 [86:40:55<1:12:00,  8.24s/it]
 99%|█████████▊| 37439/37962 [86:41:03<1:12:13,  8.29s/it]
 99%|█████████▊| 37440/37962 [86:41:11<1:12:04,  8.29s/it]
                                                          
{'loss': 1.1029, 'grad_norm': 14.529175758361816, 'learning_rate': 9.915180811546076e-09, 'epoch': 0.99}

 99%|█████████▊| 37440/37962 [86:41:11<1:12:04,  8.29s/it]
 99%|█████████▊| 37441/37962 [86:41:19<1:11:46,  8.27s/it]
 99%|█████████▊| 37442/37962 [86:41:28<1:11:33,  8.26s/it]
 99%|█████████▊| 37443/37962 [86:41:36<1:11:02,  8.21s/it]
 99%|█████████▊| 37444/37962 [86:41:44<1:11:34,  8.29s/it]
 99%|█████████▊| 37445/37962 [86:41:53<1:11:20,  8.28s/it]
 99%|█████████▊| 37446/37962 [86:42:01<1:11:18,  8.29s/it]
 99%|█████████▊| 37447/37962 [86:42:09<1:10:36,  8.23s/it]
 99%|█████████▊| 37448/37962 [86:42:17<1:10:45,  8.26s/it]
 99%|█████████▊| 37449/37962 [86:42:26<1:10:38,  8.26s/it]
 99%|█████████▊| 37450/37962 [86:42:34<1:10:39,  8.28s/it]
                                                          
{'loss': 1.0878, 'grad_norm': 8.117080688476562, 'learning_rate': 9.538987468092897e-09, 'epoch': 0.99}

 99%|█████████▊| 37450/37962 [86:42:34<1:10:39,  8.28s/it]
 99%|█████████▊| 37451/37962 [86:42:42<1:10:39,  8.30s/it]
 99%|█████████▊| 37452/37962 [86:42:50<1:10:30,  8.29s/it]
 99%|█████████▊| 37453/37962 [86:42:59<1:10:39,  8.33s/it]
 99%|█████████▊| 37454/37962 [86:43:07<1:10:35,  8.34s/it]
 99%|█████████▊| 37455/37962 [86:43:15<1:09:33,  8.23s/it]
 99%|█████████▊| 37456/37962 [86:43:24<1:10:13,  8.33s/it]
 99%|█████████▊| 37457/37962 [86:43:32<1:09:59,  8.32s/it]
 99%|█████████▊| 37458/37962 [86:43:40<1:09:23,  8.26s/it]
 99%|█████████▊| 37459/37962 [86:43:48<1:09:10,  8.25s/it]
 99%|█████████▊| 37460/37962 [86:43:57<1:09:10,  8.27s/it]
                                                          
{'loss': 1.1666, 'grad_norm': 10.85071086883545, 'learning_rate': 9.170066008137302e-09, 'epoch': 0.99}

 99%|█████████▊| 37460/37962 [86:43:57<1:09:10,  8.27s/it]
 99%|█████████▊| 37461/37962 [86:44:05<1:09:17,  8.30s/it]
 99%|█████████▊| 37462/37962 [86:44:13<1:08:49,  8.26s/it]
 99%|█████████▊| 37463/37962 [86:44:21<1:08:14,  8.21s/it]
 99%|█████████▊| 37464/37962 [86:44:30<1:08:16,  8.23s/it]
 99%|█████████▊| 37465/37962 [86:44:38<1:08:33,  8.28s/it]
 99%|█████████▊| 37466/37962 [86:44:46<1:08:28,  8.28s/it]
 99%|█████████▊| 37467/37962 [86:44:54<1:08:04,  8.25s/it]
 99%|█████████▊| 37468/37962 [86:45:03<1:07:54,  8.25s/it]
 99%|█████████▊| 37469/37962 [86:45:11<1:07:51,  8.26s/it]
 99%|█████████▊| 37470/37962 [86:45:19<1:07:33,  8.24s/it]
                                                          
{'loss': 1.2474, 'grad_norm': 7.760594367980957, 'learning_rate': 8.808416700211153e-09, 'epoch': 0.99}

 99%|█████████▊| 37470/37962 [86:45:19<1:07:33,  8.24s/it]
 99%|█████████▊| 37471/37962 [86:45:27<1:07:00,  8.19s/it]
 99%|█████████▊| 37472/37962 [86:45:36<1:07:11,  8.23s/it]
 99%|█████████▊| 37473/37962 [86:45:44<1:07:01,  8.22s/it]
 99%|█████████▊| 37474/37962 [86:45:52<1:06:50,  8.22s/it]
 99%|█████████▊| 37475/37962 [86:46:00<1:06:45,  8.23s/it]
 99%|█████████▊| 37476/37962 [86:46:09<1:06:44,  8.24s/it]
 99%|█████████▊| 37477/37962 [86:46:17<1:06:48,  8.26s/it]
 99%|█████████▊| 37478/37962 [86:46:25<1:06:02,  8.19s/it]
 99%|█████████▊| 37479/37962 [86:46:33<1:06:06,  8.21s/it]
 99%|█████████▊| 37480/37962 [86:46:41<1:05:17,  8.13s/it]
                                                          
{'loss': 1.0193, 'grad_norm': 10.442521095275879, 'learning_rate': 8.454039807553881e-09, 'epoch': 0.99}

 99%|█████████▊| 37480/37962 [86:46:41<1:05:17,  8.13s/it]
 99%|█████████▊| 37481/37962 [86:46:49<1:05:43,  8.20s/it]
 99%|█████████▊| 37482/37962 [86:46:58<1:05:35,  8.20s/it]
 99%|█████████▊| 37483/37962 [86:47:06<1:05:39,  8.22s/it]
 99%|█████████▊| 37484/37962 [86:47:14<1:05:28,  8.22s/it]
 99%|█████████▊| 37485/37962 [86:47:22<1:05:24,  8.23s/it]
 99%|█████████▊| 37486/37962 [86:47:30<1:05:00,  8.19s/it]
 99%|█████████▊| 37487/37962 [86:47:39<1:05:26,  8.27s/it]
 99%|█████████▉| 37488/37962 [86:47:47<1:05:00,  8.23s/it]
 99%|█████████▉| 37489/37962 [86:47:55<1:04:54,  8.23s/it]
 99%|█████████▉| 37490/37962 [86:48:04<1:04:45,  8.23s/it]
                                                          
{'loss': 1.0304, 'grad_norm': 13.61961841583252, 'learning_rate': 8.106935588109155e-09, 'epoch': 0.99}

 99%|█████████▉| 37490/37962 [86:48:04<1:04:45,  8.23s/it]
 99%|█████████▉| 37491/37962 [86:48:12<1:04:34,  8.23s/it]
 99%|█████████▉| 37492/37962 [86:48:20<1:03:37,  8.12s/it]
 99%|█████████▉| 37493/37962 [86:48:28<1:03:44,  8.16s/it]
 99%|█████████▉| 37494/37962 [86:48:36<1:03:43,  8.17s/it]
 99%|█████████▉| 37495/37962 [86:48:44<1:03:31,  8.16s/it]
 99%|█████████▉| 37496/37962 [86:48:52<1:03:37,  8.19s/it]
 99%|█████████▉| 37497/37962 [86:49:01<1:03:48,  8.23s/it]
 99%|█████████▉| 37498/37962 [86:49:09<1:03:59,  8.28s/it]
 99%|█████████▉| 37499/37962 [86:49:17<1:03:59,  8.29s/it]
 99%|█████████▉| 37500/37962 [86:49:25<1:02:56,  8.17s/it]
                                                          
{'loss': 1.1811, 'grad_norm': 9.076234817504883, 'learning_rate': 7.767104294528205e-09, 'epoch': 0.99}

 99%|█████████▉| 37500/37962 [86:49:25<1:02:56,  8.17s/it]
 99%|█████████▉| 37501/37962 [86:49:34<1:02:56,  8.19s/it]
 99%|█████████▉| 37502/37962 [86:49:42<1:02:13,  8.12s/it]
 99%|█████████▉| 37503/37962 [86:49:50<1:02:56,  8.23s/it]
 99%|█████████▉| 37504/37962 [86:49:58<1:02:55,  8.24s/it]
 99%|█████████▉| 37505/37962 [86:50:07<1:03:11,  8.30s/it]
 99%|█████████▉| 37506/37962 [86:50:15<1:02:50,  8.27s/it]
 99%|█████████▉| 37507/37962 [86:50:23<1:02:55,  8.30s/it]
 99%|█████████▉| 37508/37962 [86:50:31<1:02:20,  8.24s/it]
 99%|█████████▉| 37509/37962 [86:50:40<1:02:16,  8.25s/it]
 99%|█████████▉| 37510/37962 [86:50:48<1:02:43,  8.33s/it]
                                                          
{'loss': 1.1379, 'grad_norm': 11.785021781921387, 'learning_rate': 7.434546174168722e-09, 'epoch': 0.99}

 99%|█████████▉| 37510/37962 [86:50:48<1:02:43,  8.33s/it]
 99%|█████████▉| 37511/37962 [86:50:57<1:02:30,  8.32s/it]
 99%|█████████▉| 37512/37962 [86:51:05<1:02:17,  8.31s/it]
 99%|█████████▉| 37513/37962 [86:51:13<1:01:41,  8.24s/it]
 99%|█████████▉| 37514/37962 [86:51:21<1:01:04,  8.18s/it]
 99%|█████████▉| 37515/37962 [86:51:29<1:01:07,  8.20s/it]
 99%|█████████▉| 37516/37962 [86:51:37<1:00:42,  8.17s/it]
 99%|█████████▉| 37517/37962 [86:51:45<1:00:40,  8.18s/it]
 99%|█████████▉| 37518/37962 [86:51:54<1:00:39,  8.20s/it]
 99%|█████████▉| 37519/37962 [86:52:02<1:01:07,  8.28s/it]
 99%|█████████▉| 37520/37962 [86:52:10<1:00:57,  8.27s/it]
                                                          
{'loss': 1.2178, 'grad_norm': 9.68836498260498, 'learning_rate': 7.109261469093742e-09, 'epoch': 0.99}

 99%|█████████▉| 37520/37962 [86:52:10<1:00:57,  8.27s/it]
 99%|█████████▉| 37521/37962 [86:52:19<1:00:51,  8.28s/it]
 99%|█████████▉| 37522/37962 [86:52:27<1:01:00,  8.32s/it]
 99%|█████████▉| 37523/37962 [86:52:36<1:01:24,  8.39s/it]
 99%|█████████▉| 37524/37962 [86:52:44<1:00:50,  8.33s/it]
 99%|█████████▉| 37525/37962 [86:52:52<1:00:37,  8.32s/it]
 99%|█████████▉| 37526/37962 [86:53:00<1:00:10,  8.28s/it]
 99%|█████████▉| 37527/37962 [86:53:09<1:00:06,  8.29s/it]
 99%|█████████▉| 37528/37962 [86:53:17<59:56,  8.29s/it]  
 99%|█████████▉| 37529/37962 [86:53:25<59:51,  8.29s/it]
 99%|█████████▉| 37530/37962 [86:53:34<59:33,  8.27s/it]
                                                        
{'loss': 1.2114, 'grad_norm': 12.241327285766602, 'learning_rate': 6.7912504160716484e-09, 'epoch': 0.99}

 99%|█████████▉| 37530/37962 [86:53:34<59:33,  8.27s/it]
 99%|█████████▉| 37531/37962 [86:53:42<59:52,  8.33s/it]
 99%|█████████▉| 37532/37962 [86:53:50<59:33,  8.31s/it]
 99%|█████████▉| 37533/37962 [86:53:58<59:09,  8.27s/it]
 99%|█████████▉| 37534/37962 [86:54:07<59:17,  8.31s/it]
 99%|█████████▉| 37535/37962 [86:54:15<59:07,  8.31s/it]
 99%|█████████▉| 37536/37962 [86:54:23<57:47,  8.14s/it]
 99%|█████████▉| 37537/37962 [86:54:31<58:38,  8.28s/it]
 99%|█████████▉| 37538/37962 [86:54:40<58:37,  8.30s/it]
 99%|█████████▉| 37539/37962 [86:54:48<58:23,  8.28s/it]
 99%|█████████▉| 37540/37962 [86:54:56<57:27,  8.17s/it]
                                                        
{'loss': 1.1169, 'grad_norm': 7.527163505554199, 'learning_rate': 6.48051324657839e-09, 'epoch': 0.99}

 99%|█████████▉| 37540/37962 [86:54:56<57:27,  8.17s/it]
 99%|█████████▉| 37541/37962 [86:55:04<57:25,  8.19s/it]
 99%|█████████▉| 37542/37962 [86:55:13<57:45,  8.25s/it]
 99%|█████████▉| 37543/37962 [86:55:21<57:18,  8.21s/it]
 99%|█████████▉| 37544/37962 [86:55:29<57:16,  8.22s/it]
 99%|█████████▉| 37545/37962 [86:55:37<56:59,  8.20s/it]
 99%|█████████▉| 37546/37962 [86:55:45<57:03,  8.23s/it]
 99%|█████████▉| 37547/37962 [86:55:54<57:22,  8.29s/it]
 99%|█████████▉| 37548/37962 [86:56:02<57:23,  8.32s/it]
 99%|█████████▉| 37549/37962 [86:56:10<56:17,  8.18s/it]
 99%|█████████▉| 37550/37962 [86:56:18<56:09,  8.18s/it]
                                                        
{'loss': 1.0317, 'grad_norm': 8.736496925354004, 'learning_rate': 6.177050186793043e-09, 'epoch': 0.99}

 99%|█████████▉| 37550/37962 [86:56:18<56:09,  8.18s/it]
 99%|█████████▉| 37551/37962 [86:56:26<55:50,  8.15s/it]
 99%|█████████▉| 37552/37962 [86:56:34<55:25,  8.11s/it]
 99%|█████████▉| 37553/37962 [86:56:43<55:23,  8.13s/it]
 99%|█████████▉| 37554/37962 [86:56:51<55:16,  8.13s/it]
 99%|█████████▉| 37555/37962 [86:56:59<55:26,  8.17s/it]
 99%|█████████▉| 37556/37962 [86:57:07<55:30,  8.20s/it]
 99%|█████████▉| 37557/37962 [86:57:15<55:31,  8.23s/it]
 99%|█████████▉| 37558/37962 [86:57:24<55:32,  8.25s/it]
 99%|█████████▉| 37559/37962 [86:57:32<55:27,  8.26s/it]
 99%|█████████▉| 37560/37962 [86:57:40<55:30,  8.29s/it]
                                                        
{'loss': 1.1827, 'grad_norm': 13.77798080444336, 'learning_rate': 5.880861457602249e-09, 'epoch': 0.99}

 99%|█████████▉| 37560/37962 [86:57:40<55:30,  8.29s/it]
 99%|█████████▉| 37561/37962 [86:57:48<54:17,  8.12s/it]
 99%|█████████▉| 37562/37962 [86:57:57<54:35,  8.19s/it]
 99%|█████████▉| 37563/37962 [86:58:05<54:40,  8.22s/it]
 99%|█████████▉| 37564/37962 [86:58:13<54:29,  8.21s/it]
 99%|█████████▉| 37565/37962 [86:58:21<54:22,  8.22s/it]
 99%|█████████▉| 37566/37962 [86:58:29<54:20,  8.23s/it]
 99%|█████████▉| 37567/37962 [86:58:38<54:41,  8.31s/it]
 99%|█████████▉| 37568/37962 [86:58:46<54:25,  8.29s/it]
 99%|█████████▉| 37569/37962 [86:58:54<54:10,  8.27s/it]
 99%|█████████▉| 37570/37962 [86:59:03<54:07,  8.28s/it]
                                                        
{'loss': 1.0985, 'grad_norm': 11.816718101501465, 'learning_rate': 5.5919472745968875e-09, 'epoch': 0.99}

 99%|█████████▉| 37570/37962 [86:59:03<54:07,  8.28s/it]
 99%|█████████▉| 37571/37962 [86:59:11<54:06,  8.30s/it]
 99%|█████████▉| 37572/37962 [86:59:20<54:08,  8.33s/it]
 99%|█████████▉| 37573/37962 [86:59:28<53:57,  8.32s/it]
 99%|█████████▉| 37574/37962 [86:59:36<53:35,  8.29s/it]
 99%|█████████▉| 37575/37962 [86:59:45<53:52,  8.35s/it]
 99%|█████████▉| 37576/37962 [86:59:53<53:25,  8.30s/it]
 99%|█████████▉| 37577/37962 [87:00:01<53:23,  8.32s/it]
 99%|█████████▉| 37578/37962 [87:00:10<53:29,  8.36s/it]
 99%|█████████▉| 37579/37962 [87:00:17<52:14,  8.18s/it]
 99%|█████████▉| 37580/37962 [87:00:26<52:19,  8.22s/it]
                                                        
{'loss': 1.0616, 'grad_norm': 15.230541229248047, 'learning_rate': 5.310307848070962e-09, 'epoch': 0.99}

 99%|█████████▉| 37580/37962 [87:00:26<52:19,  8.22s/it]
 99%|█████████▉| 37581/37962 [87:00:34<52:21,  8.25s/it]
 99%|█████████▉| 37582/37962 [87:00:42<52:31,  8.29s/it]
 99%|█████████▉| 37583/37962 [87:00:51<52:30,  8.31s/it]
 99%|█████████▉| 37584/37962 [87:00:59<52:16,  8.30s/it]
 99%|█████████▉| 37585/37962 [87:01:07<52:13,  8.31s/it]
 99%|█████████▉| 37586/37962 [87:01:15<51:24,  8.20s/it]
 99%|█████████▉| 37587/37962 [87:01:23<50:36,  8.10s/it]
 99%|█████████▉| 37588/37962 [87:01:31<50:53,  8.16s/it]
 99%|█████████▉| 37589/37962 [87:01:40<50:47,  8.17s/it]
 99%|█████████▉| 37590/37962 [87:01:48<50:56,  8.22s/it]
                                                        
{'loss': 1.0327, 'grad_norm': 8.248568534851074, 'learning_rate': 5.035943383026043e-09, 'epoch': 0.99}

 99%|█████████▉| 37590/37962 [87:01:48<50:56,  8.22s/it]
 99%|█████████▉| 37591/37962 [87:01:56<51:00,  8.25s/it]
 99%|█████████▉| 37592/37962 [87:02:04<50:49,  8.24s/it]
 99%|█████████▉| 37593/37962 [87:02:12<50:15,  8.17s/it]
 99%|█████████▉| 37594/37962 [87:02:21<50:31,  8.24s/it]
 99%|█████████▉| 37595/37962 [87:02:29<50:02,  8.18s/it]
 99%|█████████▉| 37596/37962 [87:02:37<50:11,  8.23s/it]
 99%|█████████▉| 37597/37962 [87:02:46<50:11,  8.25s/it]
 99%|█████████▉| 37598/37962 [87:02:54<50:12,  8.28s/it]
 99%|█████████▉| 37599/37962 [87:03:02<50:25,  8.33s/it]
 99%|█████████▉| 37600/37962 [87:03:11<50:15,  8.33s/it]
                                                        
{'loss': 1.1544, 'grad_norm': 8.132597923278809, 'learning_rate': 4.7688540791668295e-09, 'epoch': 0.99}

 99%|█████████▉| 37600/37962 [87:03:11<50:15,  8.33s/it]
 99%|█████████▉| 37601/37962 [87:03:19<49:44,  8.27s/it]
 99%|█████████▉| 37602/37962 [87:03:27<49:40,  8.28s/it]
 99%|█████████▉| 37603/37962 [87:03:35<49:29,  8.27s/it]
 99%|█████████▉| 37604/37962 [87:03:44<49:36,  8.31s/it]
 99%|█████████▉| 37605/37962 [87:03:52<49:32,  8.33s/it]
 99%|█████████▉| 37606/37962 [87:04:00<49:01,  8.26s/it]
 99%|█████████▉| 37607/37962 [87:04:08<48:30,  8.20s/it]
 99%|█████████▉| 37608/37962 [87:04:17<48:27,  8.21s/it]
 99%|█████████▉| 37609/37962 [87:04:25<48:01,  8.16s/it]
 99%|█████████▉| 37610/37962 [87:04:33<48:11,  8.22s/it]
                                                        
{'loss': 1.114, 'grad_norm': 14.392091751098633, 'learning_rate': 4.509040130903364e-09, 'epoch': 0.99}

 99%|█████████▉| 37610/37962 [87:04:33<48:11,  8.22s/it]
 99%|█████████▉| 37611/37962 [87:04:41<48:17,  8.25s/it]
 99%|█████████▉| 37612/37962 [87:04:49<48:05,  8.24s/it]
 99%|█████████▉| 37613/37962 [87:04:58<47:56,  8.24s/it]
 99%|█████████▉| 37614/37962 [87:05:06<47:58,  8.27s/it]
 99%|█████████▉| 37615/37962 [87:05:14<47:07,  8.15s/it]
 99%|█████████▉| 37616/37962 [87:05:22<47:19,  8.21s/it]
 99%|█████████▉| 37617/37962 [87:05:30<47:02,  8.18s/it]
 99%|█████████▉| 37618/37962 [87:05:39<47:16,  8.25s/it]
 99%|█████████▉| 37619/37962 [87:05:47<47:32,  8.32s/it]
 99%|█████████▉| 37620/37962 [87:05:56<47:37,  8.36s/it]
                                                        
{'loss': 1.1063, 'grad_norm': 8.423932075500488, 'learning_rate': 4.256501727349927e-09, 'epoch': 0.99}

 99%|█████████▉| 37620/37962 [87:05:56<47:37,  8.36s/it]
 99%|█████████▉| 37621/37962 [87:06:04<47:29,  8.36s/it]
 99%|█████████▉| 37622/37962 [87:06:12<47:15,  8.34s/it]
 99%|█████████▉| 37623/37962 [87:06:21<46:58,  8.31s/it]
 99%|█████████▉| 37624/37962 [87:06:29<46:37,  8.28s/it]
 99%|█████████▉| 37625/37962 [87:06:37<46:11,  8.22s/it]
 99%|█████████▉| 37626/37962 [87:06:45<46:02,  8.22s/it]
 99%|█████████▉| 37627/37962 [87:06:53<46:03,  8.25s/it]
 99%|█████████▉| 37628/37962 [87:07:02<45:55,  8.25s/it]
 99%|█████████▉| 37629/37962 [87:07:10<45:57,  8.28s/it]
 99%|█████████▉| 37630/37962 [87:07:18<45:44,  8.27s/it]
                                                        
{'loss': 1.0748, 'grad_norm': 8.729024887084961, 'learning_rate': 4.011239052323923e-09, 'epoch': 0.99}

 99%|█████████▉| 37630/37962 [87:07:18<45:44,  8.27s/it]
 99%|█████████▉| 37631/37962 [87:07:27<45:35,  8.27s/it]
 99%|█████████▉| 37632/37962 [87:07:35<45:27,  8.26s/it]
 99%|█████████▉| 37633/37962 [87:07:43<44:48,  8.17s/it]
 99%|█████████▉| 37634/37962 [87:07:51<44:50,  8.20s/it]
 99%|█████████▉| 37635/37962 [87:07:59<44:25,  8.15s/it]
 99%|█████████▉| 37636/37962 [87:08:07<44:29,  8.19s/it]
 99%|█████████▉| 37637/37962 [87:08:16<44:23,  8.20s/it]
 99%|█████████▉| 37638/37962 [87:08:24<44:16,  8.20s/it]
 99%|█████████▉| 37639/37962 [87:08:32<44:19,  8.23s/it]
 99%|█████████▉| 37640/37962 [87:08:40<44:20,  8.26s/it]
                                                        
{'loss': 1.1656, 'grad_norm': 17.652915954589844, 'learning_rate': 3.773252284348105e-09, 'epoch': 0.99}

 99%|█████████▉| 37640/37962 [87:08:40<44:20,  8.26s/it]
 99%|█████████▉| 37641/37962 [87:08:48<43:50,  8.19s/it]
 99%|█████████▉| 37642/37962 [87:08:57<43:54,  8.23s/it]
 99%|█████████▉| 37643/37962 [87:09:05<43:46,  8.23s/it]
 99%|█████████▉| 37644/37962 [87:09:14<44:05,  8.32s/it]
 99%|█████████▉| 37645/37962 [87:09:21<43:24,  8.22s/it]
 99%|█████████▉| 37646/37962 [87:09:30<43:30,  8.26s/it]
 99%|█████████▉| 37647/37962 [87:09:38<43:32,  8.29s/it]
 99%|█████████▉| 37648/37962 [87:09:46<43:04,  8.23s/it]
 99%|█████████▉| 37649/37962 [87:09:55<43:03,  8.25s/it]
 99%|█████████▉| 37650/37962 [87:10:03<42:21,  8.15s/it]
                                                        
{'loss': 1.1486, 'grad_norm': 20.125713348388672, 'learning_rate': 3.5425415966505726e-09, 'epoch': 0.99}

 99%|█████████▉| 37650/37962 [87:10:03<42:21,  8.15s/it]
 99%|█████████▉| 37651/37962 [87:10:11<42:40,  8.23s/it]
 99%|█████████▉| 37652/37962 [87:10:19<42:45,  8.28s/it]
 99%|█████████▉| 37653/37962 [87:10:28<42:33,  8.26s/it]
 99%|█████████▉| 37654/37962 [87:10:36<42:45,  8.33s/it]
 99%|█████████▉| 37655/37962 [87:10:44<42:40,  8.34s/it]
 99%|█████████▉| 37656/37962 [87:10:53<42:24,  8.32s/it]
 99%|█████████▉| 37657/37962 [87:11:01<42:03,  8.28s/it]
 99%|█████████▉| 37658/37962 [87:11:09<41:54,  8.27s/it]
 99%|█████████▉| 37659/37962 [87:11:17<41:44,  8.27s/it]
 99%|█████████▉| 37660/37962 [87:11:26<41:36,  8.27s/it]
                                                        
{'loss': 1.0857, 'grad_norm': 12.343775749206543, 'learning_rate': 3.319107157159218e-09, 'epoch': 0.99}

 99%|█████████▉| 37660/37962 [87:11:26<41:36,  8.27s/it]
 99%|█████████▉| 37661/37962 [87:11:34<41:18,  8.23s/it]
 99%|█████████▉| 37662/37962 [87:11:42<40:26,  8.09s/it]
 99%|█████████▉| 37663/37962 [87:11:50<40:38,  8.15s/it]
 99%|█████████▉| 37664/37962 [87:11:58<40:08,  8.08s/it]
 99%|█████████▉| 37665/37962 [87:12:06<39:47,  8.04s/it]
 99%|█████████▉| 37666/37962 [87:12:14<39:51,  8.08s/it]
 99%|█████████▉| 37667/37962 [87:12:22<40:03,  8.15s/it]
 99%|█████████▉| 37668/37962 [87:12:30<40:03,  8.17s/it]
 99%|█████████▉| 37669/37962 [87:12:39<40:22,  8.27s/it]
 99%|█████████▉| 37670/37962 [87:12:47<40:14,  8.27s/it]
                                                        
{'loss': 1.1936, 'grad_norm': 10.12013053894043, 'learning_rate': 3.1029491285083923e-09, 'epoch': 0.99}

 99%|█████████▉| 37670/37962 [87:12:47<40:14,  8.27s/it]
 99%|█████████▉| 37671/37962 [87:12:55<39:50,  8.21s/it]
 99%|█████████▉| 37672/37962 [87:13:04<40:02,  8.29s/it]
 99%|█████████▉| 37673/37962 [87:13:12<39:47,  8.26s/it]
 99%|█████████▉| 37674/37962 [87:13:20<39:38,  8.26s/it]
 99%|█████████▉| 37675/37962 [87:13:28<39:32,  8.27s/it]
 99%|█████████▉| 37676/37962 [87:13:37<39:50,  8.36s/it]
 99%|█████████▉| 37677/37962 [87:13:45<39:27,  8.31s/it]
 99%|█████████▉| 37678/37962 [87:13:53<39:17,  8.30s/it]
 99%|█████████▉| 37679/37962 [87:14:02<39:16,  8.33s/it]
 99%|█████████▉| 37680/37962 [87:14:10<38:45,  8.25s/it]
                                                        
{'loss': 1.1639, 'grad_norm': 9.91645336151123, 'learning_rate': 2.8940676680377922e-09, 'epoch': 0.99}

 99%|█████████▉| 37680/37962 [87:14:10<38:45,  8.25s/it]
 99%|█████████▉| 37681/37962 [87:14:18<38:39,  8.25s/it]
 99%|█████████▉| 37682/37962 [87:14:26<38:23,  8.23s/it]
 99%|█████████▉| 37683/37962 [87:14:35<38:18,  8.24s/it]
 99%|█████████▉| 37684/37962 [87:14:43<37:50,  8.17s/it]
 99%|█████████▉| 37685/37962 [87:14:51<37:55,  8.21s/it]
 99%|█████████▉| 37686/37962 [87:14:59<37:44,  8.20s/it]
 99%|█████████▉| 37687/37962 [87:15:07<37:42,  8.23s/it]
 99%|█████████▉| 37688/37962 [87:15:16<37:42,  8.26s/it]
 99%|█████████▉| 37689/37962 [87:15:24<37:31,  8.25s/it]
 99%|█████████▉| 37690/37962 [87:15:32<37:24,  8.25s/it]
                                                        
{'loss': 1.011, 'grad_norm': 9.207098007202148, 'learning_rate': 2.6924629277869098e-09, 'epoch': 0.99}

 99%|█████████▉| 37690/37962 [87:15:32<37:24,  8.25s/it]
 99%|█████████▉| 37691/37962 [87:15:40<36:55,  8.17s/it]
 99%|█████████▉| 37692/37962 [87:15:48<36:52,  8.19s/it]
 99%|█████████▉| 37693/37962 [87:15:57<36:43,  8.19s/it]
 99%|█████████▉| 37694/37962 [87:16:05<36:26,  8.16s/it]
 99%|█████████▉| 37695/37962 [87:16:13<36:32,  8.21s/it]
 99%|█████████▉| 37696/37962 [87:16:21<36:28,  8.23s/it]
 99%|█████████▉| 37697/37962 [87:16:30<36:25,  8.25s/it]
 99%|█████████▉| 37698/37962 [87:16:38<36:13,  8.23s/it]
 99%|█████████▉| 37699/37962 [87:16:46<36:04,  8.23s/it]
 99%|█████████▉| 37700/37962 [87:16:54<35:58,  8.24s/it]
                                                        
{'loss': 1.2361, 'grad_norm': 13.363540649414062, 'learning_rate': 2.4981350545005833e-09, 'epoch': 0.99}

 99%|█████████▉| 37700/37962 [87:16:54<35:58,  8.24s/it]
 99%|█████████▉| 37701/37962 [87:17:02<35:16,  8.11s/it]
 99%|█████████▉| 37702/37962 [87:17:10<35:15,  8.14s/it]
 99%|█████████▉| 37703/37962 [87:17:19<35:13,  8.16s/it]
 99%|█████████▉| 37704/37962 [87:17:26<34:46,  8.09s/it]
 99%|█████████▉| 37705/37962 [87:17:35<34:50,  8.13s/it]
 99%|█████████▉| 37706/37962 [87:17:43<34:38,  8.12s/it]
 99%|█████████▉| 37707/37962 [87:17:51<34:36,  8.14s/it]
 99%|█████████▉| 37708/37962 [87:17:59<34:31,  8.15s/it]
 99%|█████████▉| 37709/37962 [87:18:07<34:13,  8.12s/it]
 99%|█████████▉| 37710/37962 [87:18:16<34:31,  8.22s/it]
                                                        
{'loss': 1.1561, 'grad_norm': 12.32223892211914, 'learning_rate': 2.311084189626778e-09, 'epoch': 0.99}

 99%|█████████▉| 37710/37962 [87:18:16<34:31,  8.22s/it]
 99%|█████████▉| 37711/37962 [87:18:24<34:32,  8.26s/it]
 99%|█████████▉| 37712/37962 [87:18:32<34:11,  8.21s/it]
 99%|█████████▉| 37713/37962 [87:18:40<34:04,  8.21s/it]
 99%|█████████▉| 37714/37962 [87:18:49<33:59,  8.22s/it]
 99%|█████████▉| 37715/37962 [87:18:57<33:30,  8.14s/it]
 99%|█████████▉| 37716/37962 [87:19:05<33:41,  8.22s/it]
 99%|█████████▉| 37717/37962 [87:19:13<33:40,  8.25s/it]
 99%|█████████▉| 37718/37962 [87:19:22<33:47,  8.31s/it]
 99%|█████████▉| 37719/37962 [87:19:30<33:40,  8.31s/it]
 99%|█████████▉| 37720/37962 [87:19:38<33:42,  8.36s/it]
                                                        
{'loss': 1.247, 'grad_norm': 14.073246955871582, 'learning_rate': 2.1313104693176934e-09, 'epoch': 0.99}

 99%|█████████▉| 37720/37962 [87:19:38<33:42,  8.36s/it]
 99%|█████████▉| 37721/37962 [87:19:47<33:26,  8.33s/it]
 99%|█████████▉| 37722/37962 [87:19:55<33:08,  8.28s/it]
 99%|█████████▉| 37723/37962 [87:20:03<32:56,  8.27s/it]
 99%|█████████▉| 37724/37962 [87:20:11<32:44,  8.26s/it]
 99%|█████████▉| 37725/37962 [87:20:20<32:45,  8.29s/it]
 99%|█████████▉| 37726/37962 [87:20:28<32:35,  8.28s/it]
 99%|█████████▉| 37727/37962 [87:20:36<32:25,  8.28s/it]
 99%|█████████▉| 37728/37962 [87:20:45<32:26,  8.32s/it]
 99%|█████████▉| 37729/37962 [87:20:53<32:20,  8.33s/it]
 99%|█████████▉| 37730/37962 [87:21:01<32:09,  8.31s/it]
                                                        
{'loss': 0.9954, 'grad_norm': 11.04294204711914, 'learning_rate': 1.958814024425326e-09, 'epoch': 0.99}

 99%|█████████▉| 37730/37962 [87:21:01<32:09,  8.31s/it]
 99%|█████████▉| 37731/37962 [87:21:10<32:10,  8.36s/it]
 99%|█████████▉| 37732/37962 [87:21:18<31:52,  8.32s/it]
 99%|█████████▉| 37733/37962 [87:21:26<31:34,  8.27s/it]
 99%|█████████▉| 37734/37962 [87:21:35<31:34,  8.31s/it]
 99%|█████████▉| 37735/37962 [87:21:43<31:24,  8.30s/it]
 99%|█████████▉| 37736/37962 [87:21:51<30:53,  8.20s/it]
 99%|█████████▉| 37737/37962 [87:21:59<30:53,  8.24s/it]
 99%|█████████▉| 37738/37962 [87:22:07<30:51,  8.26s/it]
 99%|█████████▉| 37739/37962 [87:22:16<30:40,  8.26s/it]
 99%|█████████▉| 37740/37962 [87:22:24<30:39,  8.28s/it]
                                                        
{'loss': 1.1036, 'grad_norm': 8.011744499206543, 'learning_rate': 1.793594980509239e-09, 'epoch': 0.99}

 99%|█████████▉| 37740/37962 [87:22:24<30:39,  8.28s/it]
 99%|█████████▉| 37741/37962 [87:22:32<30:34,  8.30s/it]
 99%|█████████▉| 37742/37962 [87:22:41<30:28,  8.31s/it]
 99%|█████████▉| 37743/37962 [87:22:49<30:27,  8.35s/it]
 99%|█████████▉| 37744/37962 [87:22:58<30:23,  8.36s/it]
 99%|█████████▉| 37745/37962 [87:23:06<30:11,  8.35s/it]
 99%|█████████▉| 37746/37962 [87:23:14<29:55,  8.31s/it]
 99%|█████████▉| 37747/37962 [87:23:22<29:42,  8.29s/it]
 99%|█████████▉| 37748/37962 [87:23:31<29:39,  8.31s/it]
 99%|█████████▉| 37749/37962 [87:23:39<29:04,  8.19s/it]
 99%|█████████▉| 37750/37962 [87:23:47<28:52,  8.17s/it]
                                                        
{'loss': 1.1784, 'grad_norm': 274.66278076171875, 'learning_rate': 1.6356534578276795e-09, 'epoch': 0.99}

 99%|█████████▉| 37750/37962 [87:23:47<28:52,  8.17s/it]
 99%|█████████▉| 37751/37962 [87:23:55<28:48,  8.19s/it]
 99%|█████████▉| 37752/37962 [87:24:03<28:52,  8.25s/it]
 99%|█████████▉| 37753/37962 [87:24:12<28:46,  8.26s/it]
 99%|█████████▉| 37754/37962 [87:24:20<28:51,  8.32s/it]
 99%|█████████▉| 37755/37962 [87:24:28<28:40,  8.31s/it]
 99%|█████████▉| 37756/37962 [87:24:37<28:29,  8.30s/it]
 99%|█████████▉| 37757/37962 [87:24:45<28:17,  8.28s/it]
 99%|█████████▉| 37758/37962 [87:24:53<27:50,  8.19s/it]
 99%|█████████▉| 37759/37962 [87:25:01<27:37,  8.16s/it]
 99%|█████████▉| 37760/37962 [87:25:09<27:33,  8.18s/it]
                                                        
{'loss': 1.1025, 'grad_norm': 14.736515045166016, 'learning_rate': 1.4849895713431316e-09, 'epoch': 0.99}

 99%|█████████▉| 37760/37962 [87:25:09<27:33,  8.18s/it]
 99%|█████████▉| 37761/37962 [87:25:17<27:28,  8.20s/it]
 99%|█████████▉| 37762/37962 [87:25:26<27:22,  8.21s/it]
 99%|█████████▉| 37763/37962 [87:25:34<27:09,  8.19s/it]
 99%|█████████▉| 37764/37962 [87:25:42<27:04,  8.21s/it]
 99%|█████████▉| 37765/37962 [87:25:50<27:01,  8.23s/it]
 99%|█████████▉| 37766/37962 [87:25:59<26:51,  8.22s/it]
 99%|█████████▉| 37767/37962 [87:26:07<26:43,  8.22s/it]
 99%|█████████▉| 37768/37962 [87:26:15<26:20,  8.15s/it]
 99%|█████████▉| 37769/37962 [87:26:23<26:24,  8.21s/it]
 99%|█████████▉| 37770/37962 [87:26:31<26:06,  8.16s/it]
                                                        
{'loss': 1.2254, 'grad_norm': 10.031840324401855, 'learning_rate': 1.341603430723426e-09, 'epoch': 0.99}

 99%|█████████▉| 37770/37962 [87:26:31<26:06,  8.16s/it]
 99%|█████████▉| 37771/37962 [87:26:39<26:08,  8.21s/it]
 99%|█████████▉| 37772/37962 [87:26:48<25:59,  8.21s/it]
100%|█████████▉| 37773/37962 [87:26:56<25:48,  8.19s/it]
100%|█████████▉| 37774/37962 [87:27:04<25:46,  8.23s/it]
100%|█████████▉| 37775/37962 [87:27:12<25:38,  8.23s/it]
100%|█████████▉| 37776/37962 [87:27:21<25:44,  8.30s/it]
100%|█████████▉| 37777/37962 [87:27:29<25:43,  8.34s/it]
100%|█████████▉| 37778/37962 [87:27:38<25:34,  8.34s/it]
100%|█████████▉| 37779/37962 [87:27:46<25:18,  8.30s/it]
100%|█████████▉| 37780/37962 [87:27:54<25:06,  8.28s/it]
                                                        
{'loss': 1.1696, 'grad_norm': 18.174930572509766, 'learning_rate': 1.205495140336188e-09, 'epoch': 1.0}

100%|█████████▉| 37780/37962 [87:27:54<25:06,  8.28s/it]
100%|█████████▉| 37781/37962 [87:28:02<24:58,  8.28s/it]
100%|█████████▉| 37782/37962 [87:28:11<24:50,  8.28s/it]
100%|█████████▉| 37783/37962 [87:28:19<24:42,  8.28s/it]
100%|█████████▉| 37784/37962 [87:28:27<24:43,  8.33s/it]
100%|█████████▉| 37785/37962 [87:28:35<24:13,  8.21s/it]
100%|█████████▉| 37786/37962 [87:28:44<24:14,  8.26s/it]
100%|█████████▉| 37787/37962 [87:28:52<24:10,  8.29s/it]
100%|█████████▉| 37788/37962 [87:29:00<24:03,  8.29s/it]
100%|█████████▉| 37789/37962 [87:29:09<23:52,  8.28s/it]
100%|█████████▉| 37790/37962 [87:29:17<23:40,  8.26s/it]
                                                        
{'loss': 1.1105, 'grad_norm': 14.761902809143066, 'learning_rate': 1.076664799249949e-09, 'epoch': 1.0}

100%|█████████▉| 37790/37962 [87:29:17<23:40,  8.26s/it]
100%|█████████▉| 37791/37962 [87:29:25<23:30,  8.25s/it]
100%|█████████▉| 37792/37962 [87:29:33<23:20,  8.24s/it]
100%|█████████▉| 37793/37962 [87:29:41<23:10,  8.23s/it]
100%|█████████▉| 37794/37962 [87:29:50<23:05,  8.25s/it]
100%|█████████▉| 37795/37962 [87:29:58<22:55,  8.24s/it]
100%|█████████▉| 37796/37962 [87:30:06<22:38,  8.18s/it]
100%|█████████▉| 37797/37962 [87:30:14<22:39,  8.24s/it]
100%|█████████▉| 37798/37962 [87:30:22<22:17,  8.16s/it]
100%|█████████▉| 37799/37962 [87:30:31<22:28,  8.27s/it]
100%|█████████▉| 37800/37962 [87:30:39<21:52,  8.10s/it]
                                                        
{'loss': 1.124, 'grad_norm': 12.710417747497559, 'learning_rate': 9.551125012419172e-10, 'epoch': 1.0}

100%|█████████▉| 37800/37962 [87:30:39<21:52,  8.10s/it]
100%|█████████▉| 37801/37962 [87:30:47<21:48,  8.13s/it]
100%|█████████▉| 37802/37962 [87:30:55<21:41,  8.13s/it]
100%|█████████▉| 37803/37962 [87:31:03<21:45,  8.21s/it]
100%|█████████▉| 37804/37962 [87:31:12<21:52,  8.31s/it]
100%|█████████▉| 37805/37962 [87:31:20<21:56,  8.39s/it]
100%|█████████▉| 37806/37962 [87:31:29<21:45,  8.37s/it]
100%|█████████▉| 37807/37962 [87:31:37<21:38,  8.38s/it]
100%|█████████▉| 37808/37962 [87:31:45<21:08,  8.24s/it]
100%|█████████▉| 37809/37962 [87:31:53<21:05,  8.27s/it]
100%|█████████▉| 37810/37962 [87:32:02<21:00,  8.29s/it]
                                                        
{'loss': 1.2128, 'grad_norm': 10.125197410583496, 'learning_rate': 8.408383347846549e-10, 'epoch': 1.0}

100%|█████████▉| 37810/37962 [87:32:02<21:00,  8.29s/it]
100%|█████████▉| 37811/37962 [87:32:18<26:58, 10.72s/it]
100%|█████████▉| 37812/37962 [87:32:26<24:46,  9.91s/it]
100%|█████████▉| 37813/37962 [87:32:35<23:32,  9.48s/it]
100%|█████████▉| 37814/37962 [87:32:43<22:31,  9.13s/it]
100%|█████████▉| 37815/37962 [87:32:51<21:41,  8.85s/it]
100%|█████████▉| 37816/37962 [87:32:59<20:59,  8.63s/it]
100%|█████████▉| 37817/37962 [87:33:08<20:35,  8.52s/it]
100%|█████████▉| 37818/37962 [87:33:16<20:15,  8.44s/it]
100%|█████████▉| 37819/37962 [87:33:24<19:50,  8.33s/it]
100%|█████████▉| 37820/37962 [87:33:32<19:39,  8.31s/it]
                                                        
{'loss': 1.1741, 'grad_norm': 10.500262260437012, 'learning_rate': 7.338423830582919e-10, 'epoch': 1.0}

100%|█████████▉| 37820/37962 [87:33:32<19:39,  8.31s/it]
100%|█████████▉| 37821/37962 [87:33:40<19:17,  8.21s/it]
100%|█████████▉| 37822/37962 [87:33:49<19:18,  8.27s/it]
100%|█████████▉| 37823/37962 [87:33:56<18:56,  8.17s/it]
100%|█████████▉| 37824/37962 [87:34:05<18:57,  8.24s/it]
100%|█████████▉| 37825/37962 [87:34:13<18:46,  8.22s/it]
100%|█████████▉| 37826/37962 [87:34:21<18:41,  8.25s/it]
100%|█████████▉| 37827/37962 [87:34:30<18:40,  8.30s/it]
100%|█████████▉| 37828/37962 [87:34:38<18:30,  8.29s/it]
100%|█████████▉| 37829/37962 [87:34:46<18:26,  8.32s/it]
100%|█████████▉| 37830/37962 [87:34:55<18:12,  8.27s/it]
                                                        
{'loss': 1.1737, 'grad_norm': 14.972089767456055, 'learning_rate': 6.341247239416426e-10, 'epoch': 1.0}

100%|█████████▉| 37830/37962 [87:34:55<18:12,  8.27s/it]
100%|█████████▉| 37831/37962 [87:35:03<18:11,  8.33s/it]
100%|█████████▉| 37832/37962 [87:35:11<17:59,  8.31s/it]
100%|█████████▉| 37833/37962 [87:35:19<17:36,  8.19s/it]
100%|█████████▉| 37834/37962 [87:35:27<17:28,  8.19s/it]
100%|█████████▉| 37835/37962 [87:35:36<17:28,  8.26s/it]
100%|█████████▉| 37836/37962 [87:35:44<17:07,  8.15s/it]
100%|█████████▉| 37837/37962 [87:35:52<17:00,  8.16s/it]
100%|█████████▉| 37838/37962 [87:36:00<17:01,  8.24s/it]
100%|█████████▉| 37839/37962 [87:36:09<16:52,  8.24s/it]
100%|█████████▉| 37840/37962 [87:36:17<16:51,  8.29s/it]
                                                        
{'loss': 1.2571, 'grad_norm': 10.579835891723633, 'learning_rate': 5.416854300199781e-10, 'epoch': 1.0}

100%|█████████▉| 37840/37962 [87:36:17<16:51,  8.29s/it]
100%|█████████▉| 37841/37962 [87:36:25<16:27,  8.16s/it]
100%|█████████▉| 37842/37962 [87:36:33<16:21,  8.18s/it]
100%|█████████▉| 37843/37962 [87:36:41<16:19,  8.23s/it]
100%|█████████▉| 37844/37962 [87:36:49<16:06,  8.19s/it]
100%|█████████▉| 37845/37962 [87:36:58<15:59,  8.20s/it]
100%|█████████▉| 37846/37962 [87:37:06<15:58,  8.26s/it]
100%|█████████▉| 37847/37962 [87:37:14<15:52,  8.28s/it]
100%|█████████▉| 37848/37962 [87:37:23<15:43,  8.28s/it]
100%|█████████▉| 37849/37962 [87:37:31<15:37,  8.30s/it]
100%|█████████▉| 37850/37962 [87:37:39<15:28,  8.29s/it]
                                                        
{'loss': 1.0952, 'grad_norm': 11.301043510437012, 'learning_rate': 4.5652456857503414e-10, 'epoch': 1.0}

100%|█████████▉| 37850/37962 [87:37:39<15:28,  8.29s/it]
100%|█████████▉| 37851/37962 [87:37:48<15:20,  8.30s/it]
100%|█████████▉| 37852/37962 [87:37:56<15:09,  8.27s/it]
100%|█████████▉| 37853/37962 [87:38:04<15:01,  8.27s/it]
100%|█████████▉| 37854/37962 [87:38:12<14:54,  8.28s/it]
100%|█████████▉| 37855/37962 [87:38:21<14:46,  8.28s/it]
100%|█████████▉| 37856/37962 [87:38:29<14:35,  8.26s/it]
100%|█████████▉| 37857/37962 [87:38:37<14:33,  8.32s/it]
100%|█████████▉| 37858/37962 [87:38:45<14:15,  8.23s/it]
100%|█████████▉| 37859/37962 [87:38:53<13:55,  8.11s/it]
100%|█████████▉| 37860/37962 [87:39:02<13:54,  8.19s/it]
                                                        
{'loss': 1.0867, 'grad_norm': 8.719541549682617, 'learning_rate': 3.7864220159722354e-10, 'epoch': 1.0}

100%|█████████▉| 37860/37962 [87:39:02<13:54,  8.19s/it]
100%|█████████▉| 37861/37962 [87:39:10<13:48,  8.20s/it]
100%|█████████▉| 37862/37962 [87:39:18<13:39,  8.20s/it]
100%|█████████▉| 37863/37962 [87:39:26<13:32,  8.20s/it]
100%|█████████▉| 37864/37962 [87:39:35<13:26,  8.23s/it]
100%|█████████▉| 37865/37962 [87:39:43<13:15,  8.20s/it]
100%|█████████▉| 37866/37962 [87:39:51<13:06,  8.19s/it]
100%|█████████▉| 37867/37962 [87:39:59<13:00,  8.22s/it]
100%|█████████▉| 37868/37962 [87:40:07<12:52,  8.22s/it]
100%|█████████▉| 37869/37962 [87:40:16<12:48,  8.26s/it]
100%|█████████▉| 37870/37962 [87:40:24<12:42,  8.29s/it]
                                                        
{'loss': 1.1613, 'grad_norm': 11.111146926879883, 'learning_rate': 3.080383857734237e-10, 'epoch': 1.0}

100%|█████████▉| 37870/37962 [87:40:24<12:42,  8.29s/it]
100%|█████████▉| 37871/37962 [87:40:32<12:38,  8.33s/it]
100%|█████████▉| 37872/37962 [87:40:41<12:28,  8.32s/it]
100%|█████████▉| 37873/37962 [87:40:49<12:18,  8.30s/it]
100%|█████████▉| 37874/37962 [87:40:57<12:14,  8.35s/it]
100%|█████████▉| 37875/37962 [87:41:06<12:02,  8.30s/it]
100%|█████████▉| 37876/37962 [87:41:14<11:49,  8.25s/it]
100%|█████████▉| 37877/37962 [87:41:22<11:40,  8.24s/it]
100%|█████████▉| 37878/37962 [87:41:30<11:25,  8.16s/it]
100%|█████████▉| 37879/37962 [87:41:38<11:21,  8.21s/it]
100%|█████████▉| 37880/37962 [87:41:46<11:07,  8.14s/it]
                                                        
{'loss': 1.2788, 'grad_norm': 8.674304962158203, 'learning_rate': 2.4471317249585846e-10, 'epoch': 1.0}

100%|█████████▉| 37880/37962 [87:41:46<11:07,  8.14s/it]
100%|█████████▉| 37881/37962 [87:41:55<11:08,  8.25s/it]
100%|█████████▉| 37882/37962 [87:42:03<11:07,  8.34s/it]
100%|█████████▉| 37883/37962 [87:42:12<11:02,  8.39s/it]
100%|█████████▉| 37884/37962 [87:42:20<10:51,  8.35s/it]
100%|█████████▉| 37885/37962 [87:42:28<10:39,  8.31s/it]
100%|█████████▉| 37886/37962 [87:42:37<10:35,  8.36s/it]
100%|█████████▉| 37887/37962 [87:42:45<10:23,  8.31s/it]
100%|█████████▉| 37888/37962 [87:42:53<10:15,  8.32s/it]
100%|█████████▉| 37889/37962 [87:43:01<09:57,  8.19s/it]
100%|█████████▉| 37890/37962 [87:43:10<09:56,  8.29s/it]
                                                        
{'loss': 1.1277, 'grad_norm': 9.656017303466797, 'learning_rate': 1.8866660785654689e-10, 'epoch': 1.0}

100%|█████████▉| 37890/37962 [87:43:10<09:56,  8.29s/it]
100%|█████████▉| 37891/37962 [87:43:18<09:49,  8.30s/it]
100%|█████████▉| 37892/37962 [87:43:26<09:38,  8.26s/it]
100%|█████████▉| 37893/37962 [87:43:35<09:30,  8.26s/it]
100%|█████████▉| 37894/37962 [87:43:43<09:22,  8.26s/it]
100%|█████████▉| 37895/37962 [87:43:51<09:18,  8.33s/it]
100%|█████████▉| 37896/37962 [87:44:00<09:08,  8.31s/it]
100%|█████████▉| 37897/37962 [87:44:08<08:54,  8.22s/it]
100%|█████████▉| 37898/37962 [87:44:16<08:48,  8.26s/it]
100%|█████████▉| 37899/37962 [87:44:24<08:42,  8.30s/it]
100%|█████████▉| 37900/37962 [87:44:33<08:35,  8.32s/it]
                                                        
{'loss': 1.1206, 'grad_norm': 7.973507404327393, 'learning_rate': 1.3989873265285446e-10, 'epoch': 1.0}

100%|█████████▉| 37900/37962 [87:44:33<08:35,  8.32s/it]
100%|█████████▉| 37901/37962 [87:44:41<08:28,  8.34s/it]
100%|█████████▉| 37902/37962 [87:44:49<08:20,  8.34s/it]
100%|█████████▉| 37903/37962 [87:44:58<08:10,  8.31s/it]
100%|█████████▉| 37904/37962 [87:45:06<08:01,  8.30s/it]
100%|█████████▉| 37905/37962 [87:45:14<07:49,  8.24s/it]
100%|█████████▉| 37906/37962 [87:45:22<07:37,  8.16s/it]
100%|█████████▉| 37907/37962 [87:45:30<07:33,  8.24s/it]
100%|█████████▉| 37908/37962 [87:45:38<07:22,  8.19s/it]
100%|█████████▉| 37909/37962 [87:45:47<07:15,  8.22s/it]
100%|█████████▉| 37910/37962 [87:45:55<07:08,  8.24s/it]
                                                        
{'loss': 1.0563, 'grad_norm': 8.449443817138672, 'learning_rate': 9.840958238194198e-11, 'epoch': 1.0}

100%|█████████▉| 37910/37962 [87:45:55<07:08,  8.24s/it]
100%|█████████▉| 37911/37962 [87:46:03<07:01,  8.27s/it]
100%|█████████▉| 37912/37962 [87:46:11<06:49,  8.20s/it]
100%|█████████▉| 37913/37962 [87:46:20<06:42,  8.22s/it]
100%|█████████▉| 37914/37962 [87:46:28<06:34,  8.21s/it]
100%|█████████▉| 37915/37962 [87:46:36<06:24,  8.18s/it]
100%|█████████▉| 37916/37962 [87:46:44<06:19,  8.26s/it]
100%|█████████▉| 37917/37962 [87:46:53<06:11,  8.26s/it]
100%|█████████▉| 37918/37962 [87:47:01<06:00,  8.20s/it]
100%|█████████▉| 37919/37962 [87:47:09<05:55,  8.28s/it]
100%|█████████▉| 37920/37962 [87:47:18<05:49,  8.31s/it]
                                                        
{'loss': 1.0823, 'grad_norm': 8.268136024475098, 'learning_rate': 6.419918724187569e-11, 'epoch': 1.0}

100%|█████████▉| 37920/37962 [87:47:18<05:49,  8.31s/it]
100%|█████████▉| 37921/37962 [87:47:26<05:37,  8.22s/it]
100%|█████████▉| 37922/37962 [87:47:34<05:30,  8.27s/it]
100%|█████████▉| 37923/37962 [87:47:42<05:22,  8.27s/it]
100%|█████████▉| 37924/37962 [87:47:51<05:16,  8.33s/it]
100%|█████████▉| 37925/37962 [87:47:59<05:07,  8.30s/it]
100%|█████████▉| 37926/37962 [87:48:07<04:58,  8.29s/it]
100%|█████████▉| 37927/37962 [87:48:15<04:49,  8.27s/it]
100%|█████████▉| 37928/37962 [87:48:24<04:40,  8.26s/it]
100%|█████████▉| 37929/37962 [87:48:32<04:36,  8.36s/it]
100%|█████████▉| 37930/37962 [87:48:41<04:26,  8.32s/it]
                                                        
{'loss': 1.1064, 'grad_norm': 8.095474243164062, 'learning_rate': 3.726757213384779e-11, 'epoch': 1.0}

100%|█████████▉| 37930/37962 [87:48:41<04:26,  8.32s/it]
100%|█████████▉| 37931/37962 [87:48:48<04:12,  8.15s/it]
100%|█████████▉| 37932/37962 [87:48:57<04:06,  8.21s/it]
100%|█████████▉| 37933/37962 [87:49:05<04:01,  8.32s/it]
100%|█████████▉| 37934/37962 [87:49:13<03:52,  8.32s/it]
100%|█████████▉| 37935/37962 [87:49:22<03:42,  8.23s/it]
100%|█████████▉| 37936/37962 [87:49:30<03:34,  8.25s/it]
100%|█████████▉| 37937/37962 [87:49:38<03:27,  8.29s/it]
100%|█████████▉| 37938/37962 [87:49:46<03:18,  8.27s/it]
100%|█████████▉| 37939/37962 [87:49:55<03:10,  8.30s/it]
100%|█████████▉| 37940/37962 [87:50:03<03:02,  8.31s/it]
                                                        
{'loss': 1.101, 'grad_norm': 18.85165786743164, 'learning_rate': 1.7614756661066267e-11, 'epoch': 1.0}

100%|█████████▉| 37940/37962 [87:50:03<03:02,  8.31s/it]
100%|█████████▉| 37941/37962 [87:50:11<02:53,  8.24s/it]
100%|█████████▉| 37942/37962 [87:50:19<02:44,  8.23s/it]
100%|█████████▉| 37943/37962 [87:50:28<02:38,  8.33s/it]
100%|█████████▉| 37944/37962 [87:50:37<02:32,  8.46s/it]
100%|█████████▉| 37945/37962 [87:50:45<02:23,  8.45s/it]
100%|█████████▉| 37946/37962 [87:50:53<02:13,  8.37s/it]
100%|█████████▉| 37947/37962 [87:51:02<02:05,  8.39s/it]
100%|█████████▉| 37948/37962 [87:51:10<01:56,  8.30s/it]
100%|█████████▉| 37949/37962 [87:51:18<01:48,  8.31s/it]
100%|█████████▉| 37950/37962 [87:51:26<01:39,  8.29s/it]
                                                        
{'loss': 1.188, 'grad_norm': 9.734877586364746, 'learning_rate': 5.240755128754771e-12, 'epoch': 1.0}

100%|█████████▉| 37950/37962 [87:51:26<01:39,  8.29s/it]
100%|█████████▉| 37951/37962 [87:51:35<01:30,  8.26s/it]
100%|█████████▉| 37952/37962 [87:51:43<01:23,  8.34s/it]
100%|█████████▉| 37953/37962 [87:51:51<01:14,  8.31s/it]
100%|█████████▉| 37954/37962 [87:52:00<01:06,  8.36s/it]
100%|█████████▉| 37955/37962 [87:52:08<00:58,  8.41s/it]
100%|█████████▉| 37956/37962 [87:52:17<00:50,  8.35s/it]
100%|█████████▉| 37957/37962 [87:52:25<00:41,  8.34s/it]
100%|█████████▉| 37958/37962 [87:52:33<00:33,  8.26s/it]
100%|█████████▉| 37959/37962 [87:52:41<00:24,  8.26s/it]
100%|█████████▉| 37960/37962 [87:52:50<00:16,  8.25s/it]
                                                        
{'loss': 1.1125, 'grad_norm': 9.190556526184082, 'learning_rate': 1.455765441527035e-13, 'epoch': 1.0}

100%|█████████▉| 37960/37962 [87:52:50<00:16,  8.25s/it]
100%|█████████▉| 37961/37962 [87:52:57<00:08,  8.14s/it]
100%|██████████| 37962/37962 [87:53:05<00:00,  8.09s/it]
                                                        
{'train_runtime': 316428.2007, 'train_samples_per_second': 7.678, 'train_steps_per_second': 0.12, 'train_loss': 1.3406300857599565, 'epoch': 1.0}

100%|██████████| 37962/37962 [87:53:48<00:00,  8.09s/it]
100%|██████████| 37962/37962 [87:53:48<00:00,  8.34s/it]
[rank0]:[W614 22:04:53.009219751 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())