W0624 23:17:56.656000 3086581 site-packages/torch/distributed/run.py:766] 
W0624 23:17:56.656000 3086581 site-packages/torch/distributed/run.py:766] *****************************************
W0624 23:17:56.656000 3086581 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
W0624 23:17:56.656000 3086581 site-packages/torch/distributed/run.py:766] *****************************************
[INFO] Running in WANDB offline mode
[INFO] Running in WANDB offline mode
[INFO] Running in WANDB offline mode
[INFO] Running in WANDB offline mode
Trainer._get_train_sampler replaced with custom implementation.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[2026-06-24 23:18:03,725] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
df: /home/dungpq6/.triton/autotune: No such file or directory
Trainer._get_train_sampler replaced with custom implementation.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[2026-06-24 23:18:04,045] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Trainer._get_train_sampler replaced with custom implementation.
[2026-06-24 23:18:04,086] [INFO] [comm.py:658:init_distributed] cdb=None
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
Trainer._get_train_sampler replaced with custom implementation.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[2026-06-24 23:18:04,136] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2026-06-24 23:18:04,159] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2026-06-24 23:18:04,349] [INFO] [comm.py:658:init_distributed] cdb=None
[2026-06-24 23:18:04,500] [INFO] [comm.py:658:init_distributed] cdb=None
[2026-06-24 23:18:04,507] [INFO] [comm.py:658:init_distributed] cdb=None
[2026-06-24 23:18:04,507] [INFO] [comm.py:689:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl

Loading weights:   0%|          | 0/723 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 723/723 [00:00<00:00, 12251.56it/s]

Loading weights:   0%|          | 0/723 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 723/723 [00:00<00:00, 13284.28it/s]

Loading weights:   0%|          | 0/723 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 723/723 [00:00<00:00, 12074.82it/s]

Loading weights:   0%|          | 0/723 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 723/723 [00:00<00:00, 11164.39it/s]
[VGGTEncoder] eval KV-cache window: start=8 recent=48 (total=56 frames)
[FeatureFusion] geometry fusion delta scaled by 0.2
[VGGTEncoder] eval KV-cache window: start=8 recent=48 (total=56 frames)
[FeatureFusion] geometry fusion delta scaled by 0.2
[VGGTEncoder] eval KV-cache window: start=8 recent=48 (total=56 frames)
[FeatureFusion] geometry fusion delta scaled by 0.2
[VGGTEncoder] eval KV-cache window: start=8 recent=48 (total=56 frames)
[FeatureFusion] geometry fusion delta scaled by 0.2
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Loading weights from local directory
Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "dtype": "bfloat16",
  "feature_fusion_method": "deepstack_language_add",
  "fusion_num_layers": 1,
  "geometry_encoder_layers": [
    11,
    17,
    23
  ],
  "geometry_encoder_path": "/mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B",
  "geometry_encoder_streaming": true,
  "geometry_encoder_type": "vggt",
  "geometry_fusion_layers": [
    0,
    1,
    2
  ],
  "geometry_fusion_scale": 0.2,
  "geometry_merger_type": "mlp",
  "image_token_id": 248056,
  "include_camera_token": false,
  "model_type": "qwen3_5",
  "pos_encoding_type": "none",
  "reference_frame": "first",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 2560,
    "initializer_range": 0.02,
    "intermediate_size": 9216,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": true,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": true,
  "transformers_version": "5.3.0",
  "use_cache": false,
  "use_geometry_encoder": true,
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 24,
    "dtype": "bfloat16",
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1024,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4096,
    "model_type": "qwen3_5",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 2560,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_language_fusion_layers": null,
  "vision_start_token_id": 248053
}

Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': '3d', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "dtype": "bfloat16",
  "feature_fusion_method": "deepstack_language_add",
  "fusion_num_layers": 1,
  "geometry_encoder_layers": [
    11,
    17,
    23
  ],
  "geometry_encoder_path": "/mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B",
  "geometry_encoder_streaming": true,
  "geometry_encoder_type": "vggt",
  "geometry_fusion_layers": [
    0,
    1,
    2
  ],
  "geometry_fusion_scale": 0.2,
  "geometry_merger_type": "mlp",
  "image_token_id": 248056,
  "include_camera_token": false,
  "model_type": "qwen3_5",
  "pos_encoding_type": "none",
  "reference_frame": "first",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 2560,
    "initializer_range": 0.02,
    "intermediate_size": 9216,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": true,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": true,
  "transformers_version": "5.3.0",
  "use_cache": false,
  "use_geometry_encoder": true,
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 24,
    "dtype": "bfloat16",
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1024,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4096,
    "model_type": "qwen3_5",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 2560,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_language_fusion_layers": null,
  "vision_start_token_id": 248053
}

Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': '3d', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "dtype": "bfloat16",
  "feature_fusion_method": "deepstack_language_add",
  "fusion_num_layers": 1,
  "geometry_encoder_layers": [
    11,
    17,
    23
  ],
  "geometry_encoder_path": "/mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B",
  "geometry_encoder_streaming": true,
  "geometry_encoder_type": "vggt",
  "geometry_fusion_layers": [
    0,
    1,
    2
  ],
  "geometry_fusion_scale": 0.2,
  "geometry_merger_type": "mlp",
  "image_token_id": 248056,
  "include_camera_token": false,
  "model_type": "qwen3_5",
  "pos_encoding_type": "none",
  "reference_frame": "first",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 2560,
    "initializer_range": 0.02,
    "intermediate_size": 9216,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": true,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": true,
  "transformers_version": "5.3.0",
  "use_cache": false,
  "use_geometry_encoder": true,
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 24,
    "dtype": "bfloat16",
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1024,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4096,
    "model_type": "qwen3_5",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 2560,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_language_fusion_layers": null,
  "vision_start_token_id": 248053
}

Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': '3d', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Vision Module - Attention Blocks:
Trainable Block Indices: None
Non-Trainable Block Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]
Merger Module Trainable: True
LLM Module - Embed Tokens Trainable: True
LLM Module - Trainable Layer Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
LLM Module - Non-Trainable Layer Indices: None
Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "dtype": "bfloat16",
  "feature_fusion_method": "deepstack_language_add",
  "fusion_num_layers": 1,
  "geometry_encoder_layers": [
    11,
    17,
    23
  ],
  "geometry_encoder_path": "/mnt/data/vmo-ai-task/dungpq6/model-checkpoint/VGGT-1B",
  "geometry_encoder_streaming": true,
  "geometry_encoder_type": "vggt",
  "geometry_fusion_layers": [
    0,
    1,
    2
  ],
  "geometry_fusion_scale": 0.2,
  "geometry_merger_type": "mlp",
  "image_token_id": 248056,
  "include_camera_token": false,
  "model_type": "qwen3_5",
  "pos_encoding_type": "none",
  "reference_frame": "first",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 2560,
    "initializer_range": 0.02,
    "intermediate_size": 9216,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": true,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": true,
  "transformers_version": "5.3.0",
  "use_cache": false,
  "use_geometry_encoder": true,
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 24,
    "dtype": "bfloat16",
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1024,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4096,
    "model_type": "qwen3_5",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 2560,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_language_fusion_layers": null,
  "vision_start_token_id": 248053
}

Loading datasets: [{'annotation_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN/train_r2r_rxr.json', 'data_path': '/mnt/data/vmo-ai-task/anhdh35/JanusVLN', 'tag': '3d', 'sampling_rate': 1.0, 'dataset_name': 'train_r2r_rxr'}]
Total training samples: 631244
Formatting inputs...Skip in lazy mode
Total training samples: 631244
Total training samples: 631244
Formatting inputs...Skip in lazy mode
Formatting inputs...Skip in lazy mode
Total training samples: 631244
Formatting inputs...Skip in lazy mode
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
NCCL version 2.26.2+cuda12.2

  0%|          | 0/9864 [00:00<?, ?it/s]/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 41%|████      | 4001/9864 [02:31<03:41, 26.42it/s]
 41%|████      | 4001/9864 [02:44<03:41, 26.42it/s]
 41%|████      | 4002/9864 [02:46<04:13, 23.10it/s]/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 41%|████      | 4003/9864 [03:01<04:57, 19.72it/s]
 41%|████      | 4004/9864 [03:16<06:00, 16.26it/s]
 41%|████      | 4005/9864 [03:30<07:29, 13.03it/s]
 41%|████      | 4006/9864 [03:46<09:41, 10.08it/s]
 41%|████      | 4007/9864 [04:04<13:26,  7.26it/s]
 41%|████      | 4008/9864 [04:22<18:44,  5.21it/s]
 41%|████      | 4009/9864 [04:37<24:51,  3.92it/s]
 41%|████      | 4010/9864 [04:51<32:56,  2.96it/s]
                                                   
{'loss': '0.1611', 'grad_norm': '1.468', 'learning_rate': '1.344e-05', 'epoch': '0.4066'}

 41%|████      | 4010/9864 [04:51<32:56,  2.96it/s]
 41%|████      | 4011/9864 [05:06<45:35,  2.14it/s]
 41%|████      | 4012/9864 [05:21<1:02:25,  1.56it/s]
 41%|████      | 4013/9864 [05:35<1:24:45,  1.15it/s]
 41%|████      | 4014/9864 [05:52<2:01:07,  1.24s/it]
 41%|████      | 4015/9864 [06:07<2:47:28,  1.72s/it]
 41%|████      | 4016/9864 [06:24<3:50:38,  2.37s/it]
 41%|████      | 4017/9864 [06:40<5:13:11,  3.21s/it]
 41%|████      | 4018/9864 [06:56<6:47:54,  4.19s/it]
 41%|████      | 4019/9864 [07:11<8:36:23,  5.30s/it]
 41%|████      | 4020/9864 [07:26<10:33:44,  6.51s/it]
                                                      
{'loss': '0.1364', 'grad_norm': '2.044', 'learning_rate': '1.341e-05', 'epoch': '0.4076'}

 41%|████      | 4020/9864 [07:26<10:33:44,  6.51s/it]
 41%|████      | 4021/9864 [07:42<12:58:52,  8.00s/it]
 41%|████      | 4022/9864 [07:57<15:04:40,  9.29s/it]
 41%|████      | 4023/9864 [08:13<17:25:28, 10.74s/it]
 41%|████      | 4024/9864 [08:32<20:11:50, 12.45s/it]
 41%|████      | 4025/9864 [08:47<21:16:39, 13.12s/it]
 41%|████      | 4026/9864 [09:02<22:05:35, 13.62s/it]
 41%|████      | 4027/9864 [09:17<22:31:50, 13.90s/it]
 41%|████      | 4028/9864 [09:31<22:54:56, 14.14s/it]
 41%|████      | 4029/9864 [09:48<23:57:16, 14.78s/it]
 41%|████      | 4030/9864 [10:03<23:56:49, 14.78s/it]
                                                      
{'loss': '0.1556', 'grad_norm': '2.803', 'learning_rate': '1.338e-05', 'epoch': '0.4086'}

 41%|████      | 4030/9864 [10:03<23:56:49, 14.78s/it]
 41%|████      | 4031/9864 [10:20<25:19:49, 15.63s/it]
 41%|████      | 4032/9864 [10:35<24:52:27, 15.35s/it]
 41%|████      | 4033/9864 [10:51<25:22:43, 15.67s/it]
 41%|████      | 4034/9864 [11:06<24:41:39, 15.25s/it]
 41%|████      | 4035/9864 [11:21<24:31:15, 15.14s/it]
 41%|████      | 4036/9864 [11:36<24:29:27, 15.13s/it]
 41%|████      | 4037/9864 [11:52<25:17:34, 15.63s/it]
 41%|████      | 4038/9864 [12:09<25:38:45, 15.85s/it]
 41%|████      | 4039/9864 [12:25<26:04:09, 16.11s/it]
 41%|████      | 4040/9864 [12:40<25:31:37, 15.78s/it]
                                                      
{'loss': '0.1364', 'grad_norm': '2.103', 'learning_rate': '1.335e-05', 'epoch': '0.4096'}

 41%|████      | 4040/9864 [12:41<25:31:37, 15.78s/it]
 41%|████      | 4041/9864 [12:57<25:42:09, 15.89s/it]
 41%|████      | 4042/9864 [13:12<25:13:35, 15.60s/it]
 41%|████      | 4043/9864 [13:26<24:42:19, 15.28s/it]
 41%|████      | 4044/9864 [13:40<24:13:59, 14.99s/it]
 41%|████      | 4045/9864 [13:58<25:41:46, 15.90s/it]
 41%|████      | 4046/9864 [14:13<25:00:26, 15.47s/it]
 41%|████      | 4047/9864 [14:29<25:19:45, 15.68s/it]
 41%|████      | 4048/9864 [14:44<24:49:51, 15.37s/it]
 41%|████      | 4049/9864 [15:01<25:38:45, 15.88s/it]
 41%|████      | 4050/9864 [15:16<25:10:48, 15.59s/it]
                                                      
{'loss': '0.1527', 'grad_norm': '4.257', 'learning_rate': '1.332e-05', 'epoch': '0.4106'}

 41%|████      | 4050/9864 [15:16<25:10:48, 15.59s/it]
 41%|████      | 4051/9864 [15:31<24:59:44, 15.48s/it]
 41%|████      | 4052/9864 [15:48<25:41:29, 15.91s/it]
 41%|████      | 4053/9864 [16:05<26:13:17, 16.24s/it]
 41%|████      | 4054/9864 [16:21<26:04:07, 16.15s/it]
 41%|████      | 4055/9864 [16:36<25:24:02, 15.74s/it]
 41%|████      | 4056/9864 [16:52<25:43:49, 15.95s/it]
 41%|████      | 4057/9864 [17:07<25:02:22, 15.52s/it]
 41%|████      | 4058/9864 [17:21<24:31:27, 15.21s/it]
 41%|████      | 4059/9864 [17:36<24:17:45, 15.07s/it]
 41%|████      | 4060/9864 [17:52<25:03:00, 15.54s/it]
                                                      
{'loss': '0.164', 'grad_norm': '2.401', 'learning_rate': '1.329e-05', 'epoch': '0.4116'}

 41%|████      | 4060/9864 [17:52<25:03:00, 15.54s/it]
 41%|████      | 4061/9864 [18:09<25:34:28, 15.87s/it]
 41%|████      | 4062/9864 [18:25<25:44:20, 15.97s/it]
 41%|████      | 4063/9864 [18:40<24:56:20, 15.48s/it]
 41%|████      | 4064/9864 [18:55<24:51:09, 15.43s/it]
 41%|████      | 4065/9864 [19:11<25:22:06, 15.75s/it]
 41%|████      | 4066/9864 [19:26<24:45:34, 15.37s/it]
 41%|████      | 4067/9864 [19:42<25:10:45, 15.64s/it]
 41%|████      | 4068/9864 [19:57<24:44:14, 15.36s/it]
 41%|████▏     | 4069/9864 [20:13<25:07:18, 15.61s/it]
 41%|████▏     | 4070/9864 [20:29<25:26:26, 15.81s/it]
                                                      
{'loss': '0.1489', 'grad_norm': '4.808', 'learning_rate': '1.326e-05', 'epoch': '0.4126'}

 41%|████▏     | 4070/9864 [20:29<25:26:26, 15.81s/it]
 41%|████▏     | 4071/9864 [20:44<25:08:42, 15.63s/it]
 41%|████▏     | 4072/9864 [20:59<24:49:01, 15.42s/it]
 41%|████▏     | 4073/9864 [21:14<24:29:20, 15.22s/it]
 41%|████▏     | 4074/9864 [21:30<24:57:28, 15.52s/it]
 41%|████▏     | 4075/9864 [21:47<25:27:49, 15.84s/it]
 41%|████▏     | 4076/9864 [22:04<25:50:52, 16.08s/it]
 41%|████▏     | 4077/9864 [22:18<25:02:43, 15.58s/it]
 41%|████▏     | 4078/9864 [22:35<25:48:21, 16.06s/it]
 41%|████▏     | 4079/9864 [22:50<25:18:10, 15.75s/it]
 41%|████▏     | 4080/9864 [23:04<24:33:52, 15.29s/it]
                                                      
{'loss': '0.1613', 'grad_norm': '2.954', 'learning_rate': '1.323e-05', 'epoch': '0.4137'}

 41%|████▏     | 4080/9864 [23:04<24:33:52, 15.29s/it]
 41%|████▏     | 4081/9864 [23:21<25:10:47, 15.67s/it]
 41%|████▏     | 4082/9864 [23:36<24:59:38, 15.56s/it]
 41%|████▏     | 4083/9864 [23:51<24:34:30, 15.30s/it]
 41%|████▏     | 4084/9864 [24:08<25:12:45, 15.70s/it]
 41%|████▏     | 4085/9864 [24:22<24:43:38, 15.40s/it]
 41%|████▏     | 4086/9864 [24:38<24:58:44, 15.56s/it]
 41%|████▏     | 4087/9864 [24:54<24:51:54, 15.50s/it]
 41%|████▏     | 4088/9864 [25:09<24:42:45, 15.40s/it]
 41%|████▏     | 4089/9864 [25:25<25:08:35, 15.67s/it]
 41%|████▏     | 4090/9864 [25:42<25:32:53, 15.93s/it]
                                                      
{'loss': '0.1465', 'grad_norm': '2.305', 'learning_rate': '1.32e-05', 'epoch': '0.4147'}

 41%|████▏     | 4090/9864 [25:42<25:32:53, 15.93s/it]
 41%|████▏     | 4091/9864 [25:57<25:03:18, 15.62s/it]
 41%|████▏     | 4092/9864 [26:11<24:33:18, 15.31s/it]
 41%|████▏     | 4093/9864 [26:27<24:49:09, 15.48s/it]
 42%|████▏     | 4094/9864 [26:44<25:22:33, 15.83s/it]
 42%|████▏     | 4095/9864 [26:59<24:52:46, 15.53s/it]
 42%|████▏     | 4096/9864 [27:15<25:12:02, 15.73s/it]
 42%|████▏     | 4097/9864 [27:31<25:29:29, 15.91s/it]
 42%|████▏     | 4098/9864 [27:46<24:52:02, 15.53s/it]
 42%|████▏     | 4099/9864 [28:00<24:23:20, 15.23s/it]
 42%|████▏     | 4100/9864 [28:17<25:04:18, 15.66s/it]
                                                      
{'loss': '0.1322', 'grad_norm': '2.537', 'learning_rate': '1.317e-05', 'epoch': '0.4157'}

 42%|████▏     | 4100/9864 [28:17<25:04:18, 15.66s/it]
 42%|████▏     | 4101/9864 [28:32<24:36:18, 15.37s/it]
 42%|████▏     | 4102/9864 [28:48<25:07:28, 15.70s/it]
 42%|████▏     | 4103/9864 [29:05<25:45:53, 16.10s/it]
 42%|████▏     | 4104/9864 [29:20<25:05:03, 15.68s/it]
 42%|████▏     | 4105/9864 [29:36<25:12:48, 15.76s/it]
 42%|████▏     | 4106/9864 [29:50<24:41:44, 15.44s/it]
 42%|████▏     | 4107/9864 [30:05<24:20:37, 15.22s/it]
 42%|████▏     | 4108/9864 [30:21<24:38:18, 15.41s/it]
 42%|████▏     | 4109/9864 [30:35<24:07:35, 15.09s/it]
 42%|████▏     | 4110/9864 [30:51<24:35:34, 15.39s/it]
                                                      
{'loss': '0.1535', 'grad_norm': '1.642', 'learning_rate': '1.313e-05', 'epoch': '0.4167'}

 42%|████▏     | 4110/9864 [30:51<24:35:34, 15.39s/it]
 42%|████▏     | 4111/9864 [31:07<24:50:02, 15.54s/it]
 42%|████▏     | 4112/9864 [31:24<25:16:59, 15.82s/it]
 42%|████▏     | 4113/9864 [31:38<24:41:17, 15.45s/it]
 42%|████▏     | 4114/9864 [31:53<24:27:03, 15.31s/it]
 42%|████▏     | 4115/9864 [32:10<25:01:20, 15.67s/it]
 42%|████▏     | 4116/9864 [32:24<24:30:56, 15.35s/it]
 42%|████▏     | 4117/9864 [32:39<24:03:08, 15.07s/it]
 42%|████▏     | 4118/9864 [32:55<24:23:00, 15.28s/it]
 42%|████▏     | 4119/9864 [33:11<24:59:42, 15.66s/it]
 42%|████▏     | 4120/9864 [33:28<25:30:35, 15.99s/it]
                                                      
{'loss': '0.143', 'grad_norm': '5.122', 'learning_rate': '1.31e-05', 'epoch': '0.4177'}

 42%|████▏     | 4120/9864 [33:28<25:30:35, 15.99s/it]
 42%|████▏     | 4121/9864 [33:42<24:48:07, 15.55s/it]
 42%|████▏     | 4122/9864 [33:57<24:19:38, 15.25s/it]
 42%|████▏     | 4123/9864 [34:13<24:53:58, 15.61s/it]
 42%|████▏     | 4124/9864 [34:28<24:33:06, 15.40s/it]
 42%|████▏     | 4125/9864 [34:45<24:55:16, 15.63s/it]
 42%|████▏     | 4126/9864 [35:01<25:20:50, 15.90s/it]
 42%|████▏     | 4127/9864 [35:16<24:52:12, 15.61s/it]
 42%|████▏     | 4128/9864 [35:32<25:11:06, 15.81s/it]
 42%|████▏     | 4129/9864 [35:47<24:42:45, 15.51s/it]
 42%|████▏     | 4130/9864 [36:04<25:13:51, 15.84s/it]
                                                      
{'loss': '0.1363', 'grad_norm': '1.953', 'learning_rate': '1.307e-05', 'epoch': '0.4187'}

 42%|████▏     | 4130/9864 [36:04<25:13:51, 15.84s/it]
 42%|████▏     | 4131/9864 [36:19<24:44:37, 15.54s/it]
 42%|████▏     | 4132/9864 [36:34<24:32:16, 15.41s/it]
 42%|████▏     | 4133/9864 [36:52<26:07:11, 16.41s/it]
 42%|████▏     | 4134/9864 [37:07<25:16:21, 15.88s/it]
 42%|████▏     | 4135/9864 [37:22<24:45:30, 15.56s/it]
 42%|████▏     | 4136/9864 [37:38<25:03:36, 15.75s/it]
 42%|████▏     | 4137/9864 [37:52<24:24:35, 15.34s/it]
 42%|████▏     | 4138/9864 [38:07<24:04:05, 15.13s/it]
 42%|████▏     | 4139/9864 [38:23<24:22:31, 15.33s/it]
 42%|████▏     | 4140/9864 [38:39<24:52:35, 15.65s/it]
                                                      
{'loss': '0.132', 'grad_norm': '3.632', 'learning_rate': '1.304e-05', 'epoch': '0.4197'}

 42%|████▏     | 4140/9864 [38:39<24:52:35, 15.65s/it]
 42%|████▏     | 4141/9864 [38:56<25:23:45, 15.98s/it]
 42%|████▏     | 4142/9864 [39:11<24:42:06, 15.54s/it]
 42%|████▏     | 4143/9864 [39:25<24:09:03, 15.20s/it]
 42%|████▏     | 4144/9864 [39:40<24:16:32, 15.28s/it]
 42%|████▏     | 4145/9864 [39:55<23:58:19, 15.09s/it]
 42%|████▏     | 4146/9864 [40:10<23:42:08, 14.92s/it]
 42%|████▏     | 4147/9864 [40:25<24:09:17, 15.21s/it]
 42%|████▏     | 4148/9864 [40:42<24:40:16, 15.54s/it]
 42%|████▏     | 4149/9864 [40:58<25:10:10, 15.85s/it]
 42%|████▏     | 4150/9864 [41:13<24:41:34, 15.56s/it]
                                                      
{'loss': '0.1451', 'grad_norm': '2.188', 'learning_rate': '1.301e-05', 'epoch': '0.4208'}

 42%|████▏     | 4150/9864 [41:13<24:41:34, 15.56s/it]
 42%|████▏     | 4151/9864 [41:30<25:17:10, 15.93s/it]
 42%|████▏     | 4152/9864 [41:45<24:50:20, 15.65s/it]
 42%|████▏     | 4153/9864 [41:59<24:13:00, 15.27s/it]
 42%|████▏     | 4154/9864 [42:14<24:03:46, 15.17s/it]
 42%|████▏     | 4155/9864 [42:31<24:47:41, 15.64s/it]
 42%|████▏     | 4156/9864 [42:47<25:05:55, 15.83s/it]
 42%|████▏     | 4157/9864 [43:04<25:28:53, 16.07s/it]
 42%|████▏     | 4158/9864 [43:20<25:36:19, 16.15s/it]
 42%|████▏     | 4159/9864 [43:35<24:45:50, 15.63s/it]
 42%|████▏     | 4160/9864 [43:50<24:26:08, 15.42s/it]
                                                      
{'loss': '0.1472', 'grad_norm': '1.775', 'learning_rate': '1.298e-05', 'epoch': '0.4218'}

 42%|████▏     | 4160/9864 [43:50<24:26:08, 15.42s/it]
 42%|████▏     | 4161/9864 [44:04<23:52:57, 15.08s/it]
 42%|████▏     | 4162/9864 [44:19<23:41:37, 14.96s/it]
 42%|████▏     | 4163/9864 [44:35<24:16:47, 15.33s/it]
 42%|████▏     | 4164/9864 [44:51<24:46:17, 15.65s/it]
 42%|████▏     | 4165/9864 [45:08<25:18:28, 15.99s/it]
 42%|████▏     | 4166/9864 [45:25<25:35:22, 16.17s/it]
 42%|████▏     | 4167/9864 [45:39<24:53:22, 15.73s/it]
 42%|████▏     | 4168/9864 [45:54<24:33:11, 15.52s/it]
 42%|████▏     | 4169/9864 [46:09<24:10:48, 15.29s/it]
 42%|████▏     | 4170/9864 [46:23<23:46:36, 15.03s/it]
                                                      
{'loss': '0.133', 'grad_norm': '2.661', 'learning_rate': '1.295e-05', 'epoch': '0.4228'}

 42%|████▏     | 4170/9864 [46:23<23:46:36, 15.03s/it]
 42%|████▏     | 4171/9864 [46:41<25:00:17, 15.81s/it]
 42%|████▏     | 4172/9864 [46:56<24:24:19, 15.44s/it]
 42%|████▏     | 4173/9864 [47:12<24:54:18, 15.75s/it]
 42%|████▏     | 4174/9864 [47:29<25:21:20, 16.04s/it]
 42%|████▏     | 4175/9864 [47:43<24:28:30, 15.49s/it]
 42%|████▏     | 4176/9864 [47:58<24:15:03, 15.35s/it]
 42%|████▏     | 4177/9864 [48:13<24:07:31, 15.27s/it]
 42%|████▏     | 4178/9864 [48:30<24:46:31, 15.69s/it]
 42%|████▏     | 4179/9864 [48:46<24:58:04, 15.81s/it]
 42%|████▏     | 4180/9864 [49:01<24:35:30, 15.58s/it]
                                                      
{'loss': '0.1421', 'grad_norm': '3.359', 'learning_rate': '1.292e-05', 'epoch': '0.4238'}

 42%|████▏     | 4180/9864 [49:01<24:35:30, 15.58s/it]
 42%|████▏     | 4181/9864 [49:18<25:04:38, 15.89s/it]
 42%|████▏     | 4182/9864 [49:34<25:13:59, 15.99s/it]
 42%|████▏     | 4183/9864 [49:49<24:46:25, 15.70s/it]
 42%|████▏     | 4184/9864 [50:04<24:20:41, 15.43s/it]
 42%|████▏     | 4185/9864 [50:19<24:12:37, 15.35s/it]
 42%|████▏     | 4186/9864 [50:35<24:30:47, 15.54s/it]
 42%|████▏     | 4187/9864 [50:51<24:49:31, 15.74s/it]
 42%|████▏     | 4188/9864 [51:06<24:23:54, 15.47s/it]
 42%|████▏     | 4189/9864 [51:24<25:40:34, 16.29s/it]
 42%|████▏     | 4190/9864 [51:39<24:50:35, 15.76s/it]
                                                      
{'loss': '0.1442', 'grad_norm': '1.347', 'learning_rate': '1.288e-05', 'epoch': '0.4248'}

 42%|████▏     | 4190/9864 [51:39<24:50:35, 15.76s/it]
 42%|████▏     | 4191/9864 [51:53<24:20:28, 15.45s/it]
 42%|████▏     | 4192/9864 [52:08<23:57:28, 15.21s/it]
 43%|████▎     | 4193/9864 [52:24<24:27:09, 15.52s/it]
 43%|████▎     | 4194/9864 [52:41<24:59:04, 15.86s/it]
 43%|████▎     | 4195/9864 [52:56<24:37:31, 15.64s/it]
 43%|████▎     | 4196/9864 [53:11<24:22:13, 15.48s/it]
 43%|████▎     | 4197/9864 [53:28<24:49:53, 15.77s/it]
 43%|████▎     | 4198/9864 [53:44<24:59:58, 15.88s/it]
 43%|████▎     | 4199/9864 [53:58<24:26:47, 15.54s/it]
 43%|████▎     | 4200/9864 [54:15<24:46:21, 15.75s/it]
                                                      
{'loss': '0.1458', 'grad_norm': '2.733', 'learning_rate': '1.285e-05', 'epoch': '0.4258'}

 43%|████▎     | 4200/9864 [54:15<24:46:21, 15.75s/it]
 43%|████▎     | 4201/9864 [54:29<24:14:43, 15.41s/it]
 43%|████▎     | 4202/9864 [54:46<24:54:34, 15.84s/it]
 43%|████▎     | 4203/9864 [55:01<24:22:37, 15.50s/it]
 43%|████▎     | 4204/9864 [55:18<24:58:36, 15.89s/it]
 43%|████▎     | 4205/9864 [55:34<25:24:53, 16.17s/it]
 43%|████▎     | 4206/9864 [55:50<24:56:34, 15.87s/it]
 43%|████▎     | 4207/9864 [56:04<24:14:06, 15.42s/it]
 43%|████▎     | 4208/9864 [56:19<24:06:30, 15.34s/it]
 43%|████▎     | 4209/9864 [56:35<24:12:41, 15.41s/it]
 43%|████▎     | 4210/9864 [56:51<24:35:46, 15.66s/it]
                                                      
{'loss': '0.1553', 'grad_norm': '2.24', 'learning_rate': '1.282e-05', 'epoch': '0.4268'}

 43%|████▎     | 4210/9864 [56:51<24:35:46, 15.66s/it]
 43%|████▎     | 4211/9864 [57:06<24:20:11, 15.50s/it]
 43%|████▎     | 4212/9864 [57:24<25:32:51, 16.27s/it]
 43%|████▎     | 4213/9864 [57:39<24:53:49, 15.86s/it]
 43%|████▎     | 4214/9864 [57:54<24:23:51, 15.55s/it]
 43%|████▎     | 4215/9864 [58:09<24:03:45, 15.33s/it]
 43%|████▎     | 4216/9864 [58:23<23:32:51, 15.01s/it]
 43%|████▎     | 4217/9864 [58:41<24:53:57, 15.87s/it]
 43%|████▎     | 4218/9864 [58:56<24:28:36, 15.61s/it]
 43%|████▎     | 4219/9864 [59:11<24:06:58, 15.38s/it]
 43%|████▎     | 4220/9864 [59:27<24:36:19, 15.69s/it]
                                                      
{'loss': '0.1503', 'grad_norm': '1.622', 'learning_rate': '1.279e-05', 'epoch': '0.4279'}

 43%|████▎     | 4220/9864 [59:27<24:36:19, 15.69s/it]
 43%|████▎     | 4221/9864 [59:44<25:03:27, 15.99s/it]
 43%|████▎     | 4222/9864 [59:59<24:49:04, 15.84s/it]
 43%|████▎     | 4223/9864 [1:00:14<24:18:36, 15.51s/it]
 43%|████▎     | 4224/9864 [1:00:29<23:58:57, 15.31s/it]
 43%|████▎     | 4225/9864 [1:00:47<25:07:16, 16.04s/it]
 43%|████▎     | 4226/9864 [1:01:02<24:39:13, 15.74s/it]
 43%|████▎     | 4227/9864 [1:01:16<24:11:50, 15.45s/it]
 43%|████▎     | 4228/9864 [1:01:31<23:56:46, 15.30s/it]
 43%|████▎     | 4229/9864 [1:01:49<25:11:29, 16.09s/it]
 43%|████▎     | 4230/9864 [1:02:04<24:21:48, 15.57s/it]
                                                        
{'loss': '0.1427', 'grad_norm': '6.641', 'learning_rate': '1.276e-05', 'epoch': '0.4289'}

 43%|████▎     | 4230/9864 [1:02:04<24:21:48, 15.57s/it]
 43%|████▎     | 4231/9864 [1:02:18<24:01:32, 15.35s/it]
 43%|████▎     | 4232/9864 [1:02:33<23:41:32, 15.14s/it]
 43%|████▎     | 4233/9864 [1:02:51<25:00:46, 15.99s/it]
 43%|████▎     | 4234/9864 [1:03:06<24:18:52, 15.55s/it]
 43%|████▎     | 4235/9864 [1:03:20<23:55:24, 15.30s/it]
 43%|████▎     | 4236/9864 [1:03:36<24:16:13, 15.52s/it]
 43%|████▎     | 4237/9864 [1:03:53<24:53:53, 15.93s/it]
 43%|████▎     | 4238/9864 [1:04:08<24:17:07, 15.54s/it]
 43%|████▎     | 4239/9864 [1:04:23<23:54:03, 15.30s/it]
 43%|████▎     | 4240/9864 [1:04:39<24:29:05, 15.67s/it]
                                                        
{'loss': '0.1502', 'grad_norm': '4.114', 'learning_rate': '1.273e-05', 'epoch': '0.4299'}

 43%|████▎     | 4240/9864 [1:04:39<24:29:05, 15.67s/it]
 43%|████▎     | 4241/9864 [1:04:56<25:03:26, 16.04s/it]
 43%|████▎     | 4242/9864 [1:05:11<24:32:24, 15.71s/it]
 43%|████▎     | 4243/9864 [1:05:26<24:18:00, 15.56s/it]
 43%|████▎     | 4244/9864 [1:05:45<25:35:52, 16.40s/it]
 43%|████▎     | 4245/9864 [1:05:59<24:49:34, 15.91s/it]
 43%|████▎     | 4246/9864 [1:06:14<24:22:20, 15.62s/it]
 43%|████▎     | 4247/9864 [1:06:31<24:47:27, 15.89s/it]
 43%|████▎     | 4248/9864 [1:06:46<24:22:40, 15.63s/it]
 43%|████▎     | 4249/9864 [1:07:02<24:32:54, 15.74s/it]
 43%|████▎     | 4250/9864 [1:07:16<23:53:47, 15.32s/it]
                                                        
{'loss': '0.1235', 'grad_norm': '2.307', 'learning_rate': '1.269e-05', 'epoch': '0.4309'}

 43%|████▎     | 4250/9864 [1:07:16<23:53:47, 15.32s/it]
 43%|████▎     | 4251/9864 [1:07:34<25:05:21, 16.09s/it]
 43%|████▎     | 4252/9864 [1:07:49<24:25:08, 15.66s/it]
 43%|████▎     | 4253/9864 [1:08:04<24:02:36, 15.43s/it]
 43%|████▎     | 4254/9864 [1:08:18<23:42:19, 15.21s/it]
 43%|████▎     | 4255/9864 [1:08:35<24:13:56, 15.55s/it]
 43%|████▎     | 4256/9864 [1:08:51<24:31:21, 15.74s/it]
 43%|████▎     | 4257/9864 [1:09:06<24:11:10, 15.53s/it]
 43%|████▎     | 4258/9864 [1:09:20<23:44:06, 15.24s/it]
 43%|████▎     | 4259/9864 [1:09:37<24:13:34, 15.56s/it]
 43%|████▎     | 4260/9864 [1:09:53<24:38:12, 15.83s/it]
                                                        
{'loss': '0.1452', 'grad_norm': '2.294', 'learning_rate': '1.266e-05', 'epoch': '0.4319'}

 43%|████▎     | 4260/9864 [1:09:53<24:38:12, 15.83s/it]
 43%|████▎     | 4261/9864 [1:10:08<24:16:15, 15.59s/it]
 43%|████▎     | 4262/9864 [1:10:22<23:28:46, 15.09s/it]
 43%|████▎     | 4263/9864 [1:10:38<23:57:58, 15.40s/it]
 43%|████▎     | 4264/9864 [1:10:55<24:25:41, 15.70s/it]
 43%|████▎     | 4265/9864 [1:11:10<24:01:05, 15.44s/it]
 43%|████▎     | 4266/9864 [1:11:24<23:21:33, 15.02s/it]
 43%|████▎     | 4267/9864 [1:11:40<23:50:15, 15.33s/it]
 43%|████▎     | 4268/9864 [1:11:54<23:33:06, 15.15s/it]
 43%|████▎     | 4269/9864 [1:12:11<24:04:47, 15.49s/it]
 43%|████▎     | 4270/9864 [1:12:25<23:38:24, 15.21s/it]
                                                        
{'loss': '0.1438', 'grad_norm': '2.335', 'learning_rate': '1.263e-05', 'epoch': '0.4329'}

 43%|████▎     | 4270/9864 [1:12:25<23:38:24, 15.21s/it]
 43%|████▎     | 4271/9864 [1:12:42<24:12:28, 15.58s/it]
 43%|████▎     | 4272/9864 [1:12:56<23:46:32, 15.31s/it]
 43%|████▎     | 4273/9864 [1:13:13<24:22:06, 15.69s/it]
 43%|████▎     | 4274/9864 [1:13:29<24:41:33, 15.90s/it]
 43%|████▎     | 4275/9864 [1:13:44<24:07:05, 15.54s/it]
 43%|████▎     | 4276/9864 [1:14:00<24:24:42, 15.73s/it]
 43%|████▎     | 4277/9864 [1:14:15<23:51:10, 15.37s/it]
 43%|████▎     | 4278/9864 [1:14:28<23:02:19, 14.85s/it]
 43%|████▎     | 4279/9864 [1:14:44<23:35:51, 15.21s/it]
 43%|████▎     | 4280/9864 [1:15:01<24:06:05, 15.54s/it]
                                                        
{'loss': '0.154', 'grad_norm': '3.197', 'learning_rate': '1.26e-05', 'epoch': '0.4339'}

 43%|████▎     | 4280/9864 [1:15:01<24:06:05, 15.54s/it]
 43%|████▎     | 4281/9864 [1:15:15<23:43:21, 15.30s/it]
 43%|████▎     | 4282/9864 [1:15:32<24:07:45, 15.56s/it]
 43%|████▎     | 4283/9864 [1:15:46<23:29:55, 15.16s/it]
 43%|████▎     | 4284/9864 [1:16:02<24:02:04, 15.51s/it]
 43%|████▎     | 4285/9864 [1:16:17<23:41:05, 15.28s/it]
 43%|████▎     | 4286/9864 [1:16:32<23:31:29, 15.18s/it]
 43%|████▎     | 4287/9864 [1:16:46<23:13:31, 14.99s/it]
 43%|████▎     | 4288/9864 [1:17:05<25:01:19, 16.15s/it]
 43%|████▎     | 4289/9864 [1:17:21<24:38:18, 15.91s/it]
 43%|████▎     | 4290/9864 [1:17:37<24:52:00, 16.06s/it]
                                                        
{'loss': '0.1419', 'grad_norm': '2.329', 'learning_rate': '1.257e-05', 'epoch': '0.435'}

 43%|████▎     | 4290/9864 [1:17:37<24:52:00, 16.06s/it]
 44%|████▎     | 4291/9864 [1:17:53<24:57:33, 16.12s/it]
 44%|████▎     | 4292/9864 [1:18:08<24:22:36, 15.75s/it]
 44%|████▎     | 4293/9864 [1:18:22<23:42:25, 15.32s/it]
 44%|████▎     | 4294/9864 [1:18:37<23:32:37, 15.22s/it]
 44%|████▎     | 4295/9864 [1:18:54<24:15:52, 15.69s/it]
 44%|████▎     | 4296/9864 [1:19:11<24:40:01, 15.95s/it]
 44%|████▎     | 4297/9864 [1:19:27<24:54:32, 16.11s/it]
 44%|████▎     | 4298/9864 [1:19:42<24:02:32, 15.55s/it]
 44%|████▎     | 4299/9864 [1:19:58<24:14:58, 15.69s/it]
 44%|████▎     | 4300/9864 [1:20:13<23:56:44, 15.49s/it]
                                                        
{'loss': '0.1428', 'grad_norm': '4.201', 'learning_rate': '1.254e-05', 'epoch': '0.436'}

 44%|████▎     | 4300/9864 [1:20:13<23:56:44, 15.49s/it]
 44%|████▎     | 4301/9864 [1:20:28<23:43:21, 15.35s/it]
 44%|████▎     | 4302/9864 [1:20:42<23:27:28, 15.18s/it]
 44%|████▎     | 4303/9864 [1:20:58<23:26:08, 15.17s/it]
 44%|████▎     | 4304/9864 [1:21:15<24:37:36, 15.95s/it]
 44%|████▎     | 4305/9864 [1:21:30<24:06:01, 15.61s/it]
 44%|████▎     | 4306/9864 [1:21:46<24:22:56, 15.79s/it]
 44%|████▎     | 4307/9864 [1:22:03<24:36:25, 15.94s/it]
 44%|████▎     | 4308/9864 [1:22:18<24:08:46, 15.65s/it]
 44%|████▎     | 4309/9864 [1:22:33<23:51:28, 15.46s/it]
 44%|████▎     | 4310/9864 [1:22:48<23:41:36, 15.36s/it]
                                                        
{'loss': '0.1548', 'grad_norm': '2.288', 'learning_rate': '1.25e-05', 'epoch': '0.437'}

 44%|████▎     | 4310/9864 [1:22:48<23:41:36, 15.36s/it]
 44%|████▎     | 4311/9864 [1:23:02<23:15:31, 15.08s/it]
 44%|████▎     | 4312/9864 [1:23:19<23:55:22, 15.51s/it]
 44%|████▎     | 4313/9864 [1:23:37<25:09:12, 16.31s/it]
 44%|████▎     | 4314/9864 [1:23:52<24:25:37, 15.84s/it]
 44%|████▎     | 4315/9864 [1:24:08<24:50:37, 16.12s/it]
 44%|████▍     | 4316/9864 [1:24:23<24:01:58, 15.59s/it]
 44%|████▍     | 4317/9864 [1:24:37<23:35:47, 15.31s/it]
 44%|████▍     | 4318/9864 [1:24:52<23:20:01, 15.15s/it]
 44%|████▍     | 4319/9864 [1:25:08<23:41:32, 15.38s/it]
 44%|████▍     | 4320/9864 [1:25:25<24:12:53, 15.72s/it]
                                                        
{'loss': '0.151', 'grad_norm': '3.285', 'learning_rate': '1.247e-05', 'epoch': '0.438'}

 44%|████▍     | 4320/9864 [1:25:25<24:12:53, 15.72s/it]
 44%|████▍     | 4321/9864 [1:25:41<24:18:09, 15.78s/it]
 44%|████▍     | 4322/9864 [1:25:55<23:48:03, 15.46s/it]
 44%|████▍     | 4323/9864 [1:26:12<24:23:41, 15.85s/it]
 44%|████▍     | 4324/9864 [1:26:27<23:47:19, 15.46s/it]
 44%|████▍     | 4325/9864 [1:26:41<23:10:23, 15.06s/it]
 44%|████▍     | 4326/9864 [1:26:55<22:50:56, 14.85s/it]
 44%|████▍     | 4327/9864 [1:27:13<24:14:45, 15.76s/it]
 44%|████▍     | 4328/9864 [1:27:29<24:22:29, 15.85s/it]
 44%|████▍     | 4329/9864 [1:27:44<23:48:41, 15.49s/it]
 44%|████▍     | 4330/9864 [1:28:00<24:11:31, 15.74s/it]
                                                        
{'loss': '0.1471', 'grad_norm': '3.945', 'learning_rate': '1.244e-05', 'epoch': '0.439'}

 44%|████▍     | 4330/9864 [1:28:00<24:11:31, 15.74s/it]
 44%|████▍     | 4331/9864 [1:28:15<23:41:45, 15.42s/it]
 44%|████▍     | 4332/9864 [1:28:30<23:30:29, 15.30s/it]
 44%|████▍     | 4333/9864 [1:28:44<23:09:33, 15.07s/it]
 44%|████▍     | 4334/9864 [1:29:01<23:47:27, 15.49s/it]
 44%|████▍     | 4335/9864 [1:29:17<24:21:16, 15.86s/it]
 44%|████▍     | 4336/9864 [1:29:34<24:32:27, 15.98s/it]
 44%|████▍     | 4337/9864 [1:29:48<23:42:28, 15.44s/it]
 44%|████▍     | 4338/9864 [1:30:04<24:00:35, 15.64s/it]
 44%|████▍     | 4339/9864 [1:30:19<23:32:58, 15.34s/it]
 44%|████▍     | 4340/9864 [1:30:33<23:15:08, 15.15s/it]
                                                        
{'loss': '0.1375', 'grad_norm': '2.549', 'learning_rate': '1.241e-05', 'epoch': '0.44'}

 44%|████▍     | 4340/9864 [1:30:33<23:15:08, 15.15s/it]
 44%|████▍     | 4341/9864 [1:30:50<24:01:38, 15.66s/it]
 44%|████▍     | 4342/9864 [1:31:05<23:42:09, 15.45s/it]
 44%|████▍     | 4343/9864 [1:31:22<24:19:19, 15.86s/it]
 44%|████▍     | 4344/9864 [1:31:39<24:44:36, 16.14s/it]
 44%|████▍     | 4345/9864 [1:31:55<24:56:05, 16.26s/it]
 44%|████▍     | 4346/9864 [1:32:10<24:02:35, 15.69s/it]
 44%|████▍     | 4347/9864 [1:32:24<23:37:04, 15.41s/it]
 44%|████▍     | 4348/9864 [1:32:39<23:12:17, 15.14s/it]
 44%|████▍     | 4349/9864 [1:32:55<23:50:56, 15.57s/it]
 44%|████▍     | 4350/9864 [1:33:12<24:15:58, 15.84s/it]
                                                        
{'loss': '0.1591', 'grad_norm': '4.73', 'learning_rate': '1.238e-05', 'epoch': '0.441'}

 44%|████▍     | 4350/9864 [1:33:12<24:15:58, 15.84s/it]
 44%|████▍     | 4351/9864 [1:33:26<23:30:22, 15.35s/it]
 44%|████▍     | 4352/9864 [1:33:43<24:02:03, 15.70s/it]
 44%|████▍     | 4353/9864 [1:33:59<24:14:27, 15.84s/it]
 44%|████▍     | 4354/9864 [1:34:13<23:38:43, 15.45s/it]
 44%|████▍     | 4355/9864 [1:34:28<23:07:13, 15.11s/it]
 44%|████▍     | 4356/9864 [1:34:43<23:12:15, 15.17s/it]
 44%|████▍     | 4357/9864 [1:34:58<22:57:50, 15.01s/it]
 44%|████▍     | 4358/9864 [1:35:16<24:24:15, 15.96s/it]
 44%|████▍     | 4359/9864 [1:35:30<23:33:31, 15.41s/it]
 44%|████▍     | 4360/9864 [1:35:48<24:50:01, 16.24s/it]
                                                        
{'loss': '0.1381', 'grad_norm': '4.858', 'learning_rate': '1.235e-05', 'epoch': '0.442'}

 44%|████▍     | 4360/9864 [1:35:48<24:50:01, 16.24s/it]
 44%|████▍     | 4361/9864 [1:36:03<24:02:28, 15.73s/it]
 44%|████▍     | 4362/9864 [1:36:17<23:31:41, 15.39s/it]
 44%|████▍     | 4363/9864 [1:36:32<23:17:00, 15.24s/it]
 44%|████▍     | 4364/9864 [1:36:48<23:21:59, 15.29s/it]
 44%|████▍     | 4365/9864 [1:37:04<23:46:00, 15.56s/it]
 44%|████▍     | 4366/9864 [1:37:20<24:01:53, 15.74s/it]
 44%|████▍     | 4367/9864 [1:37:39<25:22:41, 16.62s/it]
 44%|████▍     | 4368/9864 [1:37:53<24:37:17, 16.13s/it]
 44%|████▍     | 4369/9864 [1:38:08<23:55:19, 15.67s/it]
 44%|████▍     | 4370/9864 [1:38:23<23:41:04, 15.52s/it]
                                                        
{'loss': '0.1463', 'grad_norm': '2.643', 'learning_rate': '1.231e-05', 'epoch': '0.4431'}

 44%|████▍     | 4370/9864 [1:38:23<23:41:04, 15.52s/it]
 44%|████▍     | 4371/9864 [1:38:38<23:24:56, 15.35s/it]
 44%|████▍     | 4372/9864 [1:38:53<23:12:31, 15.21s/it]
 44%|████▍     | 4373/9864 [1:39:09<23:39:10, 15.51s/it]
 44%|████▍     | 4374/9864 [1:39:26<24:01:23, 15.75s/it]
 44%|████▍     | 4375/9864 [1:39:42<24:16:28, 15.92s/it]
 44%|████▍     | 4376/9864 [1:39:58<24:31:41, 16.09s/it]
 44%|████▍     | 4377/9864 [1:40:13<24:01:15, 15.76s/it]
 44%|████▍     | 4378/9864 [1:40:29<23:50:59, 15.65s/it]
 44%|████▍     | 4379/9864 [1:40:44<23:29:53, 15.42s/it]
 44%|████▍     | 4380/9864 [1:41:00<23:52:09, 15.67s/it]
                                                        
{'loss': '0.1372', 'grad_norm': '2.593', 'learning_rate': '1.228e-05', 'epoch': '0.4441'}

 44%|████▍     | 4380/9864 [1:41:00<23:52:09, 15.67s/it]
 44%|████▍     | 4381/9864 [1:41:15<23:33:23, 15.47s/it]
 44%|████▍     | 4382/9864 [1:41:31<23:54:51, 15.70s/it]
 44%|████▍     | 4383/9864 [1:41:48<24:17:01, 15.95s/it]
 44%|████▍     | 4384/9864 [1:42:04<24:36:05, 16.16s/it]
 44%|████▍     | 4385/9864 [1:42:19<24:04:33, 15.82s/it]
 44%|████▍     | 4386/9864 [1:42:35<23:46:09, 15.62s/it]
 44%|████▍     | 4387/9864 [1:42:49<23:10:08, 15.23s/it]
 44%|████▍     | 4388/9864 [1:43:05<23:42:17, 15.58s/it]
 44%|████▍     | 4389/9864 [1:43:21<23:33:32, 15.49s/it]
 45%|████▍     | 4390/9864 [1:43:37<24:06:25, 15.85s/it]
                                                        
{'loss': '0.1318', 'grad_norm': '4.503', 'learning_rate': '1.225e-05', 'epoch': '0.4451'}

 45%|████▍     | 4390/9864 [1:43:37<24:06:25, 15.85s/it]
 45%|████▍     | 4391/9864 [1:43:54<24:39:12, 16.22s/it]
 45%|████▍     | 4392/9864 [1:44:09<24:08:48, 15.89s/it]
 45%|████▍     | 4393/9864 [1:44:25<24:11:57, 15.92s/it]
 45%|████▍     | 4394/9864 [1:44:40<23:26:48, 15.43s/it]
 45%|████▍     | 4395/9864 [1:44:55<23:09:38, 15.25s/it]
 45%|████▍     | 4396/9864 [1:45:11<23:41:13, 15.59s/it]
 45%|████▍     | 4397/9864 [1:45:25<23:04:55, 15.20s/it]
 45%|████▍     | 4398/9864 [1:45:42<23:40:45, 15.60s/it]
 45%|████▍     | 4399/9864 [1:45:58<23:47:21, 15.67s/it]
 45%|████▍     | 4400/9864 [1:46:12<23:11:51, 15.28s/it]
                                                        
{'loss': '0.1417', 'grad_norm': '2.905', 'learning_rate': '1.222e-05', 'epoch': '0.4461'}

 45%|████▍     | 4400/9864 [1:46:12<23:11:51, 15.28s/it]
 45%|████▍     | 4401/9864 [1:46:29<23:47:12, 15.67s/it]
 45%|████▍     | 4402/9864 [1:46:43<23:17:10, 15.35s/it]
 45%|████▍     | 4403/9864 [1:46:59<23:43:12, 15.64s/it]
 45%|████▍     | 4404/9864 [1:47:15<23:30:44, 15.50s/it]
 45%|████▍     | 4405/9864 [1:47:30<23:28:15, 15.48s/it]
 45%|████▍     | 4406/9864 [1:47:47<24:02:53, 15.86s/it]
 45%|████▍     | 4407/9864 [1:48:03<24:22:03, 16.08s/it]
 45%|████▍     | 4408/9864 [1:48:20<24:45:30, 16.34s/it]
 45%|████▍     | 4409/9864 [1:48:35<24:01:52, 15.86s/it]
 45%|████▍     | 4410/9864 [1:48:50<23:25:58, 15.47s/it]
                                                        
{'loss': '0.1554', 'grad_norm': '5.275', 'learning_rate': '1.219e-05', 'epoch': '0.4471'}

 45%|████▍     | 4410/9864 [1:48:50<23:25:58, 15.47s/it]
 45%|████▍     | 4411/9864 [1:49:06<23:42:02, 15.65s/it]
 45%|████▍     | 4412/9864 [1:49:21<23:28:11, 15.50s/it]
 45%|████▍     | 4413/9864 [1:49:36<23:15:21, 15.36s/it]
 45%|████▍     | 4414/9864 [1:49:54<24:36:08, 16.25s/it]
 45%|████▍     | 4415/9864 [1:50:11<25:01:46, 16.54s/it]
 45%|████▍     | 4416/9864 [1:50:26<24:13:33, 16.01s/it]
 45%|████▍     | 4417/9864 [1:50:41<23:47:16, 15.72s/it]
 45%|████▍     | 4418/9864 [1:50:56<23:30:32, 15.54s/it]
 45%|████▍     | 4419/9864 [1:51:13<24:04:29, 15.92s/it]
 45%|████▍     | 4420/9864 [1:51:28<23:47:47, 15.74s/it]
                                                        
{'loss': '0.145', 'grad_norm': '3.259', 'learning_rate': '1.215e-05', 'epoch': '0.4481'}

 45%|████▍     | 4420/9864 [1:51:28<23:47:47, 15.74s/it]
 45%|████▍     | 4421/9864 [1:51:45<24:04:40, 15.93s/it]
 45%|████▍     | 4422/9864 [1:52:02<24:26:30, 16.17s/it]
 45%|████▍     | 4423/9864 [1:52:16<23:44:35, 15.71s/it]
 45%|████▍     | 4424/9864 [1:52:33<23:59:37, 15.88s/it]
 45%|████▍     | 4425/9864 [1:52:47<23:27:31, 15.53s/it]
 45%|████▍     | 4426/9864 [1:53:02<23:07:21, 15.31s/it]
 45%|████▍     | 4427/9864 [1:53:18<23:35:42, 15.62s/it]
 45%|████▍     | 4428/9864 [1:53:33<23:01:17, 15.25s/it]
 45%|████▍     | 4429/9864 [1:53:51<24:29:56, 16.23s/it]
 45%|████▍     | 4430/9864 [1:54:06<23:51:56, 15.81s/it]
                                                        
{'loss': '0.1541', 'grad_norm': '4.118', 'learning_rate': '1.212e-05', 'epoch': '0.4491'}

 45%|████▍     | 4430/9864 [1:54:06<23:51:56, 15.81s/it]
 45%|████▍     | 4431/9864 [1:54:23<24:09:57, 16.01s/it]
 45%|████▍     | 4432/9864 [1:54:38<23:45:52, 15.75s/it]
 45%|████▍     | 4433/9864 [1:54:52<23:18:48, 15.45s/it]
 45%|████▍     | 4434/9864 [1:55:07<22:52:31, 15.17s/it]
 45%|████▍     | 4435/9864 [1:55:22<22:42:35, 15.06s/it]
 45%|████▍     | 4436/9864 [1:55:40<23:58:30, 15.90s/it]
 45%|████▍     | 4437/9864 [1:55:56<24:17:48, 16.12s/it]
 45%|████▍     | 4438/9864 [1:56:11<23:46:28, 15.77s/it]
 45%|████▌     | 4439/9864 [1:56:28<24:03:22, 15.96s/it]
 45%|████▌     | 4440/9864 [1:56:42<23:08:31, 15.36s/it]
                                                        
{'loss': '0.1443', 'grad_norm': '3.1', 'learning_rate': '1.209e-05', 'epoch': '0.4502'}

 45%|████▌     | 4440/9864 [1:56:42<23:08:31, 15.36s/it]
 45%|████▌     | 4441/9864 [1:56:56<22:48:57, 15.15s/it]
 45%|████▌     | 4442/9864 [1:57:11<22:40:10, 15.05s/it]
 45%|████▌     | 4443/9864 [1:57:28<23:30:39, 15.61s/it]
 45%|████▌     | 4444/9864 [1:57:45<23:57:42, 15.92s/it]
 45%|████▌     | 4445/9864 [1:58:01<24:12:37, 16.08s/it]
 45%|████▌     | 4446/9864 [1:58:16<23:40:11, 15.73s/it]
 45%|████▌     | 4447/9864 [1:58:32<23:49:53, 15.84s/it]
 45%|████▌     | 4448/9864 [1:58:47<23:24:51, 15.56s/it]
 45%|████▌     | 4449/9864 [1:59:02<23:02:04, 15.31s/it]
 45%|████▌     | 4450/9864 [1:59:15<22:17:37, 14.82s/it]
                                                        
{'loss': '0.1395', 'grad_norm': '5.161', 'learning_rate': '1.206e-05', 'epoch': '0.4512'}

 45%|████▌     | 4450/9864 [1:59:15<22:17:37, 14.82s/it]
 45%|████▌     | 4451/9864 [1:59:32<23:00:58, 15.31s/it]
 45%|████▌     | 4452/9864 [1:59:49<23:44:05, 15.79s/it]
 45%|████▌     | 4453/9864 [2:00:05<23:59:47, 15.97s/it]
 45%|████▌     | 4454/9864 [2:00:20<23:17:49, 15.50s/it]
 45%|████▌     | 4455/9864 [2:00:36<23:47:44, 15.84s/it]
 45%|████▌     | 4456/9864 [2:00:51<23:06:25, 15.38s/it]
 45%|████▌     | 4457/9864 [2:01:05<22:52:53, 15.23s/it]
 45%|████▌     | 4458/9864 [2:01:20<22:41:13, 15.11s/it]
 45%|████▌     | 4459/9864 [2:01:36<23:10:26, 15.44s/it]
 45%|████▌     | 4460/9864 [2:01:55<24:29:02, 16.31s/it]
                                                        
{'loss': '0.1496', 'grad_norm': '3.866', 'learning_rate': '1.202e-05', 'epoch': '0.4522'}

 45%|████▌     | 4460/9864 [2:01:55<24:29:02, 16.31s/it]
 45%|████▌     | 4461/9864 [2:02:10<23:51:26, 15.90s/it]
 45%|████▌     | 4462/9864 [2:02:25<23:35:55, 15.73s/it]
 45%|████▌     | 4463/9864 [2:02:40<23:10:23, 15.45s/it]
 45%|████▌     | 4464/9864 [2:02:56<23:32:24, 15.69s/it]
 45%|████▌     | 4465/9864 [2:03:11<23:04:45, 15.39s/it]
 45%|████▌     | 4466/9864 [2:03:28<23:54:00, 15.94s/it]
 45%|████▌     | 4467/9864 [2:03:43<23:22:07, 15.59s/it]
 45%|████▌     | 4468/9864 [2:03:59<23:51:06, 15.91s/it]
 45%|████▌     | 4469/9864 [2:04:16<24:06:52, 16.09s/it]
 45%|████▌     | 4470/9864 [2:04:30<23:22:06, 15.60s/it]
                                                        
{'loss': '0.1431', 'grad_norm': '2.098', 'learning_rate': '1.199e-05', 'epoch': '0.4532'}

 45%|████▌     | 4470/9864 [2:04:30<23:22:06, 15.60s/it]
 45%|████▌     | 4471/9864 [2:04:45<23:00:31, 15.36s/it]
 45%|████▌     | 4472/9864 [2:05:00<22:42:50, 15.17s/it]
 45%|████▌     | 4473/9864 [2:05:18<23:59:44, 16.02s/it]
 45%|████▌     | 4474/9864 [2:05:32<23:19:13, 15.58s/it]
 45%|████▌     | 4475/9864 [2:05:49<23:40:29, 15.82s/it]
 45%|████▌     | 4476/9864 [2:06:05<23:54:18, 15.97s/it]
 45%|████▌     | 4477/9864 [2:06:20<23:17:26, 15.56s/it]
 45%|████▌     | 4478/9864 [2:06:34<22:51:33, 15.28s/it]
 45%|████▌     | 4479/9864 [2:06:50<22:51:31, 15.28s/it]
 45%|████▌     | 4480/9864 [2:07:06<23:29:24, 15.71s/it]
                                                        
{'loss': '0.1422', 'grad_norm': '2.899', 'learning_rate': '1.196e-05', 'epoch': '0.4542'}

 45%|████▌     | 4480/9864 [2:07:06<23:29:24, 15.71s/it]
 45%|████▌     | 4481/9864 [2:07:23<23:49:27, 15.93s/it]
 45%|████▌     | 4482/9864 [2:07:38<23:19:50, 15.61s/it]
 45%|████▌     | 4483/9864 [2:07:54<23:52:21, 15.97s/it]
 45%|████▌     | 4484/9864 [2:08:10<23:50:19, 15.95s/it]
 45%|████▌     | 4485/9864 [2:08:26<23:29:56, 15.73s/it]
 45%|████▌     | 4486/9864 [2:08:40<22:49:13, 15.28s/it]
 45%|████▌     | 4487/9864 [2:08:57<23:34:44, 15.79s/it]
 45%|████▌     | 4488/9864 [2:09:12<23:10:15, 15.52s/it]
 46%|████▌     | 4489/9864 [2:09:28<23:44:39, 15.90s/it]
 46%|████▌     | 4490/9864 [2:09:43<23:06:32, 15.48s/it]
                                                        
{'loss': '0.135', 'grad_norm': '4.284', 'learning_rate': '1.193e-05', 'epoch': '0.4552'}

 46%|████▌     | 4490/9864 [2:09:43<23:06:32, 15.48s/it]
 46%|████▌     | 4491/9864 [2:10:01<24:14:58, 16.25s/it]
 46%|████▌     | 4492/9864 [2:10:16<23:38:51, 15.85s/it]
 46%|████▌     | 4493/9864 [2:10:31<23:12:18, 15.55s/it]
 46%|████▌     | 4494/9864 [2:10:46<22:54:53, 15.36s/it]
 46%|████▌     | 4495/9864 [2:11:02<23:29:33, 15.75s/it]
 46%|████▌     | 4496/9864 [2:11:17<23:08:56, 15.52s/it]
 46%|████▌     | 4497/9864 [2:11:34<23:26:07, 15.72s/it]
 46%|████▌     | 4498/9864 [2:11:48<22:59:42, 15.43s/it]
 46%|████▌     | 4499/9864 [2:12:05<23:23:00, 15.69s/it]
 46%|████▌     | 4500/9864 [2:12:21<23:28:56, 15.76s/it]
                                                        
{'loss': '0.1485', 'grad_norm': '2.839', 'learning_rate': '1.19e-05', 'epoch': '0.4562'}

 46%|████▌     | 4500/9864 [2:12:21<23:28:56, 15.76s/it]
 46%|████▌     | 4501/9864 [2:12:35<22:59:14, 15.43s/it]
 46%|████▌     | 4502/9864 [2:12:50<22:46:20, 15.29s/it]
 46%|████▌     | 4503/9864 [2:13:07<23:16:47, 15.63s/it]
 46%|████▌     | 4504/9864 [2:13:23<23:48:28, 15.99s/it]
 46%|████▌     | 4505/9864 [2:13:38<23:15:13, 15.62s/it]
 46%|████▌     | 4506/9864 [2:13:53<22:47:19, 15.31s/it]
 46%|████▌     | 4507/9864 [2:14:12<24:19:38, 16.35s/it]
 46%|████▌     | 4508/9864 [2:14:27<23:43:51, 15.95s/it]
 46%|████▌     | 4509/9864 [2:14:41<23:12:47, 15.61s/it]
 46%|████▌     | 4510/9864 [2:14:56<22:35:15, 15.19s/it]
                                                        
{'loss': '0.1437', 'grad_norm': '4.274', 'learning_rate': '1.186e-05', 'epoch': '0.4573'}

 46%|████▌     | 4510/9864 [2:14:56<22:35:15, 15.19s/it]
 46%|████▌     | 4511/9864 [2:15:12<23:17:22, 15.66s/it]
 46%|████▌     | 4512/9864 [2:15:27<22:51:32, 15.38s/it]
 46%|████▌     | 4513/9864 [2:15:44<23:28:35, 15.79s/it]
 46%|████▌     | 4514/9864 [2:16:01<23:55:52, 16.10s/it]
 46%|████▌     | 4515/9864 [2:16:17<23:57:19, 16.12s/it]
 46%|████▌     | 4516/9864 [2:16:32<23:19:09, 15.70s/it]
 46%|████▌     | 4517/9864 [2:16:47<23:02:58, 15.52s/it]
 46%|████▌     | 4518/9864 [2:17:03<23:36:39, 15.90s/it]
 46%|████▌     | 4519/9864 [2:17:18<23:11:26, 15.62s/it]
 46%|████▌     | 4520/9864 [2:17:33<22:51:21, 15.40s/it]
                                                        
{'loss': '0.1451', 'grad_norm': '2.357', 'learning_rate': '1.183e-05', 'epoch': '0.4583'}

 46%|████▌     | 4520/9864 [2:17:33<22:51:21, 15.40s/it]
 46%|████▌     | 4521/9864 [2:17:52<24:16:09, 16.35s/it]
 46%|████▌     | 4522/9864 [2:18:07<23:36:02, 15.90s/it]
 46%|████▌     | 4523/9864 [2:18:24<24:00:23, 16.18s/it]
 46%|████▌     | 4524/9864 [2:18:38<23:03:28, 15.54s/it]
 46%|████▌     | 4525/9864 [2:18:52<22:30:34, 15.18s/it]
 46%|████▌     | 4526/9864 [2:19:08<23:05:03, 15.57s/it]
 46%|████▌     | 4527/9864 [2:19:24<22:58:24, 15.50s/it]
 46%|████▌     | 4528/9864 [2:19:38<22:35:59, 15.25s/it]
 46%|████▌     | 4529/9864 [2:19:56<23:50:26, 16.09s/it]
 46%|████▌     | 4530/9864 [2:20:13<24:06:58, 16.28s/it]
                                                        
{'loss': '0.1335', 'grad_norm': '1.877', 'learning_rate': '1.18e-05', 'epoch': '0.4593'}

 46%|████▌     | 4530/9864 [2:20:13<24:06:58, 16.28s/it]
 46%|████▌     | 4531/9864 [2:20:28<23:35:10, 15.92s/it]
 46%|████▌     | 4532/9864 [2:20:43<23:01:00, 15.54s/it]
 46%|████▌     | 4533/9864 [2:20:57<22:25:33, 15.14s/it]
 46%|████▌     | 4534/9864 [2:21:13<22:55:54, 15.49s/it]
 46%|████▌     | 4535/9864 [2:21:29<22:54:26, 15.48s/it]
 46%|████▌     | 4536/9864 [2:21:46<23:27:35, 15.85s/it]
 46%|████▌     | 4537/9864 [2:22:01<23:26:22, 15.84s/it]
 46%|████▌     | 4538/9864 [2:22:17<23:31:19, 15.90s/it]
 46%|████▌     | 4539/9864 [2:22:32<22:52:04, 15.46s/it]
 46%|████▌     | 4540/9864 [2:22:47<22:35:09, 15.27s/it]
                                                        
{'loss': '0.1417', 'grad_norm': '2.667', 'learning_rate': '1.177e-05', 'epoch': '0.4603'}

 46%|████▌     | 4540/9864 [2:22:47<22:35:09, 15.27s/it]
 46%|████▌     | 4541/9864 [2:23:02<22:28:47, 15.20s/it]
 46%|████▌     | 4542/9864 [2:23:19<23:16:00, 15.74s/it]
 46%|████▌     | 4543/9864 [2:23:34<22:51:47, 15.47s/it]
 46%|████▌     | 4544/9864 [2:23:52<24:14:04, 16.40s/it]
 46%|████▌     | 4545/9864 [2:24:08<24:08:16, 16.34s/it]
 46%|████▌     | 4546/9864 [2:24:24<23:43:48, 16.06s/it]
 46%|████▌     | 4547/9864 [2:24:38<23:07:14, 15.65s/it]
 46%|████▌     | 4548/9864 [2:24:53<22:43:54, 15.39s/it]
 46%|████▌     | 4549/9864 [2:25:08<22:38:49, 15.34s/it]
 46%|████▌     | 4550/9864 [2:25:26<23:26:10, 15.88s/it]
                                                        
{'loss': '0.1523', 'grad_norm': '4.204', 'learning_rate': '1.173e-05', 'epoch': '0.4613'}

 46%|████▌     | 4550/9864 [2:25:26<23:26:10, 15.88s/it]
 46%|████▌     | 4551/9864 [2:25:41<23:06:15, 15.66s/it]
 46%|████▌     | 4552/9864 [2:25:59<24:15:04, 16.44s/it]
 46%|████▌     | 4553/9864 [2:26:16<24:22:21, 16.52s/it]
 46%|████▌     | 4554/9864 [2:26:31<23:42:26, 16.07s/it]
 46%|████▌     | 4555/9864 [2:26:45<23:01:55, 15.62s/it]
 46%|████▌     | 4556/9864 [2:27:00<22:33:46, 15.30s/it]
 46%|████▌     | 4557/9864 [2:27:14<22:09:42, 15.03s/it]
 46%|████▌     | 4558/9864 [2:27:31<22:50:41, 15.50s/it]
 46%|████▌     | 4559/9864 [2:27:46<22:37:30, 15.35s/it]
 46%|████▌     | 4560/9864 [2:28:03<23:31:22, 15.97s/it]
                                                        
{'loss': '0.1653', 'grad_norm': '1.802', 'learning_rate': '1.17e-05', 'epoch': '0.4623'}

 46%|████▌     | 4560/9864 [2:28:03<23:31:22, 15.97s/it]
 46%|████▌     | 4561/9864 [2:28:20<23:50:49, 16.19s/it]
 46%|████▌     | 4562/9864 [2:28:35<23:10:01, 15.73s/it]
 46%|████▋     | 4563/9864 [2:28:49<22:46:23, 15.47s/it]
 46%|████▋     | 4564/9864 [2:29:04<22:28:31, 15.27s/it]
 46%|████▋     | 4565/9864 [2:29:19<22:12:15, 15.08s/it]
 46%|████▋     | 4566/9864 [2:29:34<22:16:42, 15.14s/it]
 46%|████▋     | 4567/9864 [2:29:53<23:50:59, 16.21s/it]
 46%|████▋     | 4568/9864 [2:30:10<24:03:16, 16.35s/it]
 46%|████▋     | 4569/9864 [2:30:25<23:31:54, 16.00s/it]
 46%|████▋     | 4570/9864 [2:30:41<23:28:32, 15.96s/it]
                                                        
{'loss': '0.1467', 'grad_norm': '3.809', 'learning_rate': '1.167e-05', 'epoch': '0.4633'}

 46%|████▋     | 4570/9864 [2:30:41<23:28:32, 15.96s/it]
 46%|████▋     | 4571/9864 [2:30:55<22:49:48, 15.53s/it]
 46%|████▋     | 4572/9864 [2:31:10<22:30:33, 15.31s/it]
 46%|████▋     | 4573/9864 [2:31:25<22:12:04, 15.11s/it]
 46%|████▋     | 4574/9864 [2:31:42<23:24:20, 15.93s/it]
 46%|████▋     | 4575/9864 [2:31:59<23:38:19, 16.09s/it]
 46%|████▋     | 4576/9864 [2:32:13<22:56:29, 15.62s/it]
 46%|████▋     | 4577/9864 [2:32:28<22:37:39, 15.41s/it]
 46%|████▋     | 4578/9864 [2:32:44<22:47:14, 15.52s/it]
 46%|████▋     | 4579/9864 [2:32:59<22:25:22, 15.27s/it]
 46%|████▋     | 4580/9864 [2:33:13<22:07:18, 15.07s/it]
                                                        
{'loss': '0.1391', 'grad_norm': '3.592', 'learning_rate': '1.164e-05', 'epoch': '0.4644'}

 46%|████▋     | 4580/9864 [2:33:13<22:07:18, 15.07s/it]
 46%|████▋     | 4581/9864 [2:33:28<21:59:43, 14.99s/it]
 46%|████▋     | 4582/9864 [2:33:46<23:19:46, 15.90s/it]
 46%|████▋     | 4583/9864 [2:34:03<23:43:02, 16.17s/it]
 46%|████▋     | 4584/9864 [2:34:18<23:15:53, 15.86s/it]
 46%|████▋     | 4585/9864 [2:34:34<23:22:54, 15.95s/it]
 46%|████▋     | 4586/9864 [2:34:49<22:42:05, 15.48s/it]
 47%|████▋     | 4587/9864 [2:35:03<22:07:11, 15.09s/it]
 47%|████▋     | 4588/9864 [2:35:18<21:55:53, 14.96s/it]
 47%|████▋     | 4589/9864 [2:35:32<21:51:03, 14.91s/it]
 47%|████▋     | 4590/9864 [2:35:50<23:05:07, 15.76s/it]
                                                        
{'loss': '0.1617', 'grad_norm': '4.58', 'learning_rate': '1.161e-05', 'epoch': '0.4654'}

 47%|████▋     | 4590/9864 [2:35:50<23:05:07, 15.76s/it]
 47%|████▋     | 4591/9864 [2:36:07<23:27:07, 16.01s/it]
 47%|████▋     | 4592/9864 [2:36:21<22:48:02, 15.57s/it]
 47%|████▋     | 4593/9864 [2:36:37<23:04:17, 15.76s/it]
 47%|████▋     | 4594/9864 [2:36:52<22:45:00, 15.54s/it]
 47%|████▋     | 4595/9864 [2:37:07<22:26:46, 15.34s/it]
 47%|████▋     | 4596/9864 [2:37:22<22:05:26, 15.10s/it]
 47%|████▋     | 4597/9864 [2:37:40<23:36:57, 16.14s/it]
 47%|████▋     | 4598/9864 [2:37:57<23:55:20, 16.35s/it]
 47%|████▋     | 4599/9864 [2:38:12<23:06:42, 15.80s/it]
 47%|████▋     | 4600/9864 [2:38:28<23:18:42, 15.94s/it]
                                                        
{'loss': '0.1447', 'grad_norm': '1.859', 'learning_rate': '1.157e-05', 'epoch': '0.4664'}

 47%|████▋     | 4600/9864 [2:38:28<23:18:42, 15.94s/it]
 47%|████▋     | 4601/9864 [2:38:43<22:50:18, 15.62s/it]
 47%|████▋     | 4602/9864 [2:38:58<22:37:55, 15.48s/it]
 47%|████▋     | 4603/9864 [2:39:13<22:15:29, 15.23s/it]
 47%|████▋     | 4604/9864 [2:39:30<23:08:02, 15.83s/it]
 47%|████▋     | 4605/9864 [2:39:48<24:15:48, 16.61s/it]
 47%|████▋     | 4606/9864 [2:40:04<23:41:32, 16.22s/it]
 47%|████▋     | 4607/9864 [2:40:19<23:05:14, 15.81s/it]
 47%|████▋     | 4608/9864 [2:40:35<23:33:44, 16.14s/it]
 47%|████▋     | 4609/9864 [2:40:50<22:57:44, 15.73s/it]
 47%|████▋     | 4610/9864 [2:41:05<22:31:27, 15.43s/it]
                                                        
{'loss': '0.1376', 'grad_norm': '3.127', 'learning_rate': '1.154e-05', 'epoch': '0.4674'}

 47%|████▋     | 4610/9864 [2:41:05<22:31:27, 15.43s/it]
 47%|████▋     | 4611/9864 [2:41:19<22:02:48, 15.11s/it]
 47%|████▋     | 4612/9864 [2:41:35<22:19:16, 15.30s/it]
 47%|████▋     | 4613/9864 [2:41:53<23:36:53, 16.19s/it]
 47%|████▋     | 4614/9864 [2:42:08<22:57:36, 15.74s/it]
 47%|████▋     | 4615/9864 [2:42:22<22:19:36, 15.31s/it]
 47%|████▋     | 4616/9864 [2:42:38<22:38:57, 15.54s/it]
 47%|████▋     | 4617/9864 [2:42:53<22:22:27, 15.35s/it]
 47%|████▋     | 4618/9864 [2:43:08<22:05:52, 15.16s/it]
 47%|████▋     | 4619/9864 [2:43:25<22:44:37, 15.61s/it]
 47%|████▋     | 4620/9864 [2:43:41<22:52:22, 15.70s/it]
                                                        
{'loss': '0.1331', 'grad_norm': '2.085', 'learning_rate': '1.151e-05', 'epoch': '0.4684'}

 47%|████▋     | 4620/9864 [2:43:41<22:52:22, 15.70s/it]
 47%|████▋     | 4621/9864 [2:43:57<22:58:00, 15.77s/it]
 47%|████▋     | 4622/9864 [2:44:12<22:54:31, 15.73s/it]
 47%|████▋     | 4623/9864 [2:44:27<22:27:18, 15.42s/it]
 47%|████▋     | 4624/9864 [2:44:44<23:01:47, 15.82s/it]
 47%|████▋     | 4625/9864 [2:44:58<22:25:51, 15.41s/it]
 47%|████▋     | 4626/9864 [2:45:15<23:08:57, 15.91s/it]
 47%|████▋     | 4627/9864 [2:45:30<22:52:30, 15.72s/it]
 47%|████▋     | 4628/9864 [2:45:47<23:01:12, 15.83s/it]
 47%|████▋     | 4629/9864 [2:46:03<23:19:36, 16.04s/it]
 47%|████▋     | 4630/9864 [2:46:18<22:48:50, 15.69s/it]
                                                        
{'loss': '0.145', 'grad_norm': '1.654', 'learning_rate': '1.148e-05', 'epoch': '0.4694'}

 47%|████▋     | 4630/9864 [2:46:18<22:48:50, 15.69s/it]
 47%|████▋     | 4631/9864 [2:46:33<22:26:43, 15.44s/it]
 47%|████▋     | 4632/9864 [2:46:49<22:58:34, 15.81s/it]
 47%|████▋     | 4633/9864 [2:47:05<22:40:00, 15.60s/it]
 47%|████▋     | 4634/9864 [2:47:20<22:27:10, 15.46s/it]
 47%|████▋     | 4635/9864 [2:47:36<23:00:44, 15.84s/it]
 47%|████▋     | 4636/9864 [2:47:53<23:12:01, 15.98s/it]
 47%|████▋     | 4637/9864 [2:48:09<23:24:24, 16.12s/it]
 47%|████▋     | 4638/9864 [2:48:24<22:39:09, 15.60s/it]
 47%|████▋     | 4639/9864 [2:48:39<22:22:56, 15.42s/it]
 47%|████▋     | 4640/9864 [2:48:56<23:11:12, 15.98s/it]
                                                        
{'loss': '0.1289', 'grad_norm': '2.005', 'learning_rate': '1.144e-05', 'epoch': '0.4704'}

 47%|████▋     | 4640/9864 [2:48:56<23:11:12, 15.98s/it]
 47%|████▋     | 4641/9864 [2:49:11<22:41:33, 15.64s/it]
 47%|████▋     | 4642/9864 [2:49:27<23:09:50, 15.97s/it]
 47%|████▋     | 4643/9864 [2:49:44<23:19:10, 16.08s/it]
 47%|████▋     | 4644/9864 [2:49:59<22:53:08, 15.78s/it]
 47%|████▋     | 4645/9864 [2:50:15<23:09:01, 15.97s/it]
 47%|████▋     | 4646/9864 [2:50:30<22:45:02, 15.70s/it]
 47%|████▋     | 4647/9864 [2:50:45<22:23:15, 15.45s/it]
 47%|████▋     | 4648/9864 [2:51:02<22:51:47, 15.78s/it]
 47%|████▋     | 4649/9864 [2:51:17<22:26:30, 15.49s/it]
 47%|████▋     | 4650/9864 [2:51:35<23:40:04, 16.34s/it]
                                                        
{'loss': '0.141', 'grad_norm': '3.284', 'learning_rate': '1.141e-05', 'epoch': '0.4715'}

 47%|████▋     | 4650/9864 [2:51:35<23:40:04, 16.34s/it]
 47%|████▋     | 4651/9864 [2:51:50<22:59:41, 15.88s/it]
 47%|████▋     | 4652/9864 [2:52:05<22:31:15, 15.56s/it]
 47%|████▋     | 4653/9864 [2:52:21<22:50:08, 15.78s/it]
 47%|████▋     | 4654/9864 [2:52:36<22:28:15, 15.53s/it]
 47%|████▋     | 4655/9864 [2:52:51<22:09:25, 15.31s/it]
 47%|████▋     | 4656/9864 [2:53:05<21:54:58, 15.15s/it]
 47%|████▋     | 4657/9864 [2:53:24<23:17:09, 16.10s/it]
 47%|████▋     | 4658/9864 [2:53:39<22:48:37, 15.77s/it]
 47%|████▋     | 4659/9864 [2:53:55<23:07:37, 16.00s/it]
 47%|████▋     | 4660/9864 [2:54:10<22:30:14, 15.57s/it]
                                                        
{'loss': '0.1403', 'grad_norm': '4.995', 'learning_rate': '1.138e-05', 'epoch': '0.4725'}

 47%|████▋     | 4660/9864 [2:54:10<22:30:14, 15.57s/it]
 47%|████▋     | 4661/9864 [2:54:26<22:46:39, 15.76s/it]
 47%|████▋     | 4662/9864 [2:54:41<22:33:06, 15.61s/it]
 47%|████▋     | 4663/9864 [2:54:56<22:07:15, 15.31s/it]
 47%|████▋     | 4664/9864 [2:55:14<23:28:19, 16.25s/it]
 47%|████▋     | 4665/9864 [2:55:29<22:56:33, 15.89s/it]
 47%|████▋     | 4666/9864 [2:55:44<22:26:08, 15.54s/it]
 47%|████▋     | 4667/9864 [2:55:59<22:11:23, 15.37s/it]
 47%|████▋     | 4668/9864 [2:56:14<22:13:25, 15.40s/it]
 47%|████▋     | 4669/9864 [2:56:31<22:47:35, 15.80s/it]
 47%|████▋     | 4670/9864 [2:56:46<22:30:18, 15.60s/it]
                                                        
{'loss': '0.135', 'grad_norm': '2.646', 'learning_rate': '1.135e-05', 'epoch': '0.4735'}

 47%|████▋     | 4670/9864 [2:56:46<22:30:18, 15.60s/it]
 47%|████▋     | 4671/9864 [2:57:01<22:14:59, 15.42s/it]
 47%|████▋     | 4672/9864 [2:57:19<23:13:43, 16.11s/it]
 47%|████▋     | 4673/9864 [2:57:34<22:39:38, 15.72s/it]
 47%|████▋     | 4674/9864 [2:57:49<22:13:07, 15.41s/it]
 47%|████▋     | 4675/9864 [2:58:05<22:28:52, 15.60s/it]
 47%|████▋     | 4676/9864 [2:58:19<22:02:20, 15.29s/it]
 47%|████▋     | 4677/9864 [2:58:36<22:43:32, 15.77s/it]
 47%|████▋     | 4678/9864 [2:58:51<22:25:28, 15.57s/it]
 47%|████▋     | 4679/9864 [2:59:07<22:34:47, 15.68s/it]
 47%|████▋     | 4680/9864 [2:59:24<22:57:43, 15.95s/it]
                                                        
{'loss': '0.1365', 'grad_norm': '3.018', 'learning_rate': '1.131e-05', 'epoch': '0.4745'}

 47%|████▋     | 4680/9864 [2:59:24<22:57:43, 15.95s/it]
 47%|████▋     | 4681/9864 [2:59:38<22:27:13, 15.60s/it]
 47%|████▋     | 4682/9864 [2:59:55<22:54:06, 15.91s/it]
 47%|████▋     | 4683/9864 [3:00:10<22:16:30, 15.48s/it]
 47%|████▋     | 4684/9864 [3:00:26<22:41:15, 15.77s/it]
 47%|████▋     | 4685/9864 [3:00:41<22:12:18, 15.44s/it]
 48%|████▊     | 4686/9864 [3:00:55<21:52:36, 15.21s/it]
 48%|████▊     | 4687/9864 [3:01:12<22:22:04, 15.55s/it]
 48%|████▊     | 4688/9864 [3:01:28<22:41:29, 15.78s/it]
 48%|████▊     | 4689/9864 [3:01:43<22:21:55, 15.56s/it]
 48%|████▊     | 4690/9864 [3:02:00<23:04:33, 16.06s/it]
                                                        
{'loss': '0.1443', 'grad_norm': '2.348', 'learning_rate': '1.128e-05', 'epoch': '0.4755'}

 48%|████▊     | 4690/9864 [3:02:00<23:04:33, 16.06s/it]
 48%|████▊     | 4691/9864 [3:02:17<23:24:18, 16.29s/it]
 48%|████▊     | 4692/9864 [3:02:32<22:36:00, 15.73s/it]
 48%|████▊     | 4693/9864 [3:02:46<22:14:11, 15.48s/it]
 48%|████▊     | 4694/9864 [3:03:01<21:52:53, 15.24s/it]
 48%|████▊     | 4695/9864 [3:03:18<22:29:49, 15.67s/it]
 48%|████▊     | 4696/9864 [3:03:34<22:53:39, 15.95s/it]
 48%|████▊     | 4697/9864 [3:03:51<23:09:37, 16.14s/it]
 48%|████▊     | 4698/9864 [3:04:06<22:37:30, 15.77s/it]
 48%|████▊     | 4699/9864 [3:04:21<22:14:22, 15.50s/it]
 48%|████▊     | 4700/9864 [3:04:37<22:39:01, 15.79s/it]
                                                        
{'loss': '0.1392', 'grad_norm': '2.358', 'learning_rate': '1.125e-05', 'epoch': '0.4765'}

 48%|████▊     | 4700/9864 [3:04:37<22:39:01, 15.79s/it]
 48%|████▊     | 4701/9864 [3:04:52<22:04:21, 15.39s/it]
 48%|████▊     | 4702/9864 [3:05:08<22:32:28, 15.72s/it]
 48%|████▊     | 4703/9864 [3:05:23<21:59:06, 15.34s/it]
 48%|████▊     | 4704/9864 [3:05:39<22:38:39, 15.80s/it]
 48%|████▊     | 4705/9864 [3:05:55<22:24:42, 15.64s/it]
 48%|████▊     | 4706/9864 [3:06:09<21:54:07, 15.29s/it]
 48%|████▊     | 4707/9864 [3:06:26<22:30:14, 15.71s/it]
 48%|████▊     | 4708/9864 [3:06:41<22:11:33, 15.50s/it]
 48%|████▊     | 4709/9864 [3:06:56<22:06:14, 15.44s/it]
 48%|████▊     | 4710/9864 [3:07:12<22:03:44, 15.41s/it]
                                                        
{'loss': '0.143', 'grad_norm': '2.723', 'learning_rate': '1.122e-05', 'epoch': '0.4775'}

 48%|████▊     | 4710/9864 [3:07:12<22:03:44, 15.41s/it]
 48%|████▊     | 4711/9864 [3:07:28<22:20:29, 15.61s/it]
 48%|████▊     | 4712/9864 [3:07:44<22:53:09, 15.99s/it]
 48%|████▊     | 4713/9864 [3:08:01<23:14:43, 16.25s/it]
 48%|████▊     | 4714/9864 [3:08:16<22:46:36, 15.92s/it]
 48%|████▊     | 4715/9864 [3:08:31<22:16:21, 15.57s/it]
 48%|████▊     | 4716/9864 [3:08:48<22:41:17, 15.87s/it]
 48%|████▊     | 4717/9864 [3:09:03<22:20:57, 15.63s/it]
 48%|████▊     | 4718/9864 [3:09:17<21:51:55, 15.30s/it]
 48%|████▊     | 4719/9864 [3:09:33<22:04:29, 15.45s/it]
 48%|████▊     | 4720/9864 [3:09:48<21:49:36, 15.28s/it]
                                                        
{'loss': '0.1295', 'grad_norm': '2.714', 'learning_rate': '1.118e-05', 'epoch': '0.4785'}

 48%|████▊     | 4720/9864 [3:09:48<21:49:36, 15.28s/it]
 48%|████▊     | 4721/9864 [3:10:05<22:27:10, 15.72s/it]
 48%|████▊     | 4722/9864 [3:10:20<22:04:39, 15.46s/it]
 48%|████▊     | 4723/9864 [3:10:35<21:50:51, 15.30s/it]
 48%|████▊     | 4724/9864 [3:10:51<22:24:10, 15.69s/it]
 48%|████▊     | 4725/9864 [3:11:06<21:59:39, 15.41s/it]
 48%|████▊     | 4726/9864 [3:11:22<22:19:04, 15.64s/it]
 48%|████▊     | 4727/9864 [3:11:37<22:06:48, 15.50s/it]
 48%|████▊     | 4728/9864 [3:11:55<23:12:20, 16.27s/it]
 48%|████▊     | 4729/9864 [3:12:10<22:32:49, 15.81s/it]
 48%|████▊     | 4730/9864 [3:12:24<21:55:11, 15.37s/it]
                                                        
{'loss': '0.1464', 'grad_norm': '2.407', 'learning_rate': '1.115e-05', 'epoch': '0.4796'}

 48%|████▊     | 4730/9864 [3:12:24<21:55:11, 15.37s/it]
 48%|████▊     | 4731/9864 [3:12:41<22:26:06, 15.73s/it]
 48%|████▊     | 4732/9864 [3:12:56<22:17:53, 15.64s/it]
 48%|████▊     | 4733/9864 [3:13:11<21:59:55, 15.43s/it]
 48%|████▊     | 4734/9864 [3:13:28<22:22:03, 15.70s/it]
 48%|████▊     | 4735/9864 [3:13:43<22:00:00, 15.44s/it]
 48%|████▊     | 4736/9864 [3:13:59<22:32:16, 15.82s/it]
 48%|████▊     | 4737/9864 [3:14:16<23:03:05, 16.19s/it]
 48%|████▊     | 4738/9864 [3:14:31<22:34:16, 15.85s/it]
 48%|████▊     | 4739/9864 [3:14:46<22:14:53, 15.63s/it]
 48%|████▊     | 4740/9864 [3:15:03<22:39:07, 15.91s/it]
                                                        
{'loss': '0.1407', 'grad_norm': '1.807', 'learning_rate': '1.112e-05', 'epoch': '0.4806'}

 48%|████▊     | 4740/9864 [3:15:03<22:39:07, 15.91s/it]
 48%|████▊     | 4741/9864 [3:15:19<22:50:24, 16.05s/it]
 48%|████▊     | 4742/9864 [3:15:34<22:05:27, 15.53s/it]
 48%|████▊     | 4743/9864 [3:15:50<22:33:16, 15.86s/it]
 48%|████▊     | 4744/9864 [3:16:07<22:50:36, 16.06s/it]
 48%|████▊     | 4745/9864 [3:16:22<22:17:28, 15.68s/it]
 48%|████▊     | 4746/9864 [3:16:37<22:00:01, 15.48s/it]
 48%|████▊     | 4747/9864 [3:16:53<22:29:40, 15.83s/it]
 48%|████▊     | 4748/9864 [3:17:08<22:00:21, 15.48s/it]
 48%|████▊     | 4749/9864 [3:17:24<22:24:45, 15.77s/it]
 48%|████▊     | 4750/9864 [3:17:39<21:44:13, 15.30s/it]
                                                        
{'loss': '0.1379', 'grad_norm': '3.043', 'learning_rate': '1.108e-05', 'epoch': '0.4816'}

 48%|████▊     | 4750/9864 [3:17:39<21:44:13, 15.30s/it]
 48%|████▊     | 4751/9864 [3:17:55<22:13:24, 15.65s/it]
 48%|████▊     | 4752/9864 [3:18:09<21:37:43, 15.23s/it]
 48%|████▊     | 4753/9864 [3:18:25<21:53:33, 15.42s/it]
 48%|████▊     | 4754/9864 [3:18:40<21:30:47, 15.16s/it]
 48%|████▊     | 4755/9864 [3:18:56<21:59:35, 15.50s/it]
 48%|████▊     | 4756/9864 [3:19:10<21:27:17, 15.12s/it]
 48%|████▊     | 4757/9864 [3:19:27<21:55:17, 15.45s/it]
 48%|████▊     | 4758/9864 [3:19:43<22:29:15, 15.85s/it]
 48%|████▊     | 4759/9864 [3:19:58<21:56:43, 15.48s/it]
 48%|████▊     | 4760/9864 [3:20:14<22:22:27, 15.78s/it]
                                                        
{'loss': '0.1274', 'grad_norm': '2.433', 'learning_rate': '1.105e-05', 'epoch': '0.4826'}

 48%|████▊     | 4760/9864 [3:20:15<22:22:27, 15.78s/it]
 48%|████▊     | 4761/9864 [3:20:28<21:36:22, 15.24s/it]
 48%|████▊     | 4762/9864 [3:20:44<21:32:09, 15.20s/it]
 48%|████▊     | 4763/9864 [3:21:00<21:58:44, 15.51s/it]
 48%|████▊     | 4764/9864 [3:21:15<21:39:57, 15.29s/it]
 48%|████▊     | 4765/9864 [3:21:31<22:17:46, 15.74s/it]
 48%|████▊     | 4766/9864 [3:21:48<22:31:27, 15.91s/it]
 48%|████▊     | 4767/9864 [3:22:03<22:05:26, 15.60s/it]
 48%|████▊     | 4768/9864 [3:22:18<22:14:55, 15.72s/it]
 48%|████▊     | 4769/9864 [3:22:34<21:57:43, 15.52s/it]
 48%|████▊     | 4770/9864 [3:22:49<21:43:22, 15.35s/it]
                                                        
{'loss': '0.1208', 'grad_norm': '5.932', 'learning_rate': '1.102e-05', 'epoch': '0.4836'}

 48%|████▊     | 4770/9864 [3:22:49<21:43:22, 15.35s/it]
 48%|████▊     | 4771/9864 [3:23:05<22:21:08, 15.80s/it]
 48%|████▊     | 4772/9864 [3:23:20<21:59:45, 15.55s/it]
 48%|████▊     | 4773/9864 [3:23:38<23:06:03, 16.34s/it]
 48%|████▊     | 4774/9864 [3:23:53<22:21:45, 15.82s/it]
 48%|████▊     | 4775/9864 [3:24:10<22:42:20, 16.06s/it]
 48%|████▊     | 4776/9864 [3:24:24<22:07:06, 15.65s/it]
 48%|████▊     | 4777/9864 [3:24:39<21:44:20, 15.38s/it]
 48%|████▊     | 4778/9864 [3:24:56<22:22:55, 15.84s/it]
 48%|████▊     | 4779/9864 [3:25:11<21:51:03, 15.47s/it]
 48%|████▊     | 4780/9864 [3:25:26<21:36:40, 15.30s/it]
                                                        
{'loss': '0.1483', 'grad_norm': '1.756', 'learning_rate': '1.099e-05', 'epoch': '0.4846'}

 48%|████▊     | 4780/9864 [3:25:26<21:36:40, 15.30s/it]
 48%|████▊     | 4781/9864 [3:25:40<21:23:59, 15.16s/it]
 48%|████▊     | 4782/9864 [3:26:00<23:15:54, 16.48s/it]
 48%|████▊     | 4783/9864 [3:26:15<22:35:57, 16.01s/it]
 48%|████▊     | 4784/9864 [3:26:29<21:57:19, 15.56s/it]
 49%|████▊     | 4785/9864 [3:26:45<21:50:33, 15.48s/it]
 49%|████▊     | 4786/9864 [3:27:00<21:32:38, 15.27s/it]
 49%|████▊     | 4787/9864 [3:27:15<21:34:25, 15.30s/it]
 49%|████▊     | 4788/9864 [3:27:30<21:28:10, 15.23s/it]
 49%|████▊     | 4789/9864 [3:27:45<21:23:22, 15.17s/it]
 49%|████▊     | 4790/9864 [3:28:05<23:17:20, 16.52s/it]
                                                        
{'loss': '0.1309', 'grad_norm': '2.951', 'learning_rate': '1.095e-05', 'epoch': '0.4856'}

 49%|████▊     | 4790/9864 [3:28:05<23:17:20, 16.52s/it]
 49%|████▊     | 4791/9864 [3:28:19<22:29:57, 15.97s/it]
 49%|████▊     | 4792/9864 [3:28:34<21:49:51, 15.50s/it]
 49%|████▊     | 4793/9864 [3:28:50<22:10:10, 15.74s/it]
 49%|████▊     | 4794/9864 [3:29:05<21:51:15, 15.52s/it]
 49%|████▊     | 4795/9864 [3:29:19<21:10:29, 15.04s/it]
 49%|████▊     | 4796/9864 [3:29:34<21:11:45, 15.06s/it]
 49%|████▊     | 4797/9864 [3:29:51<21:58:40, 15.61s/it]
 49%|████▊     | 4798/9864 [3:30:09<23:05:22, 16.41s/it]
 49%|████▊     | 4799/9864 [3:30:24<22:17:42, 15.85s/it]
 49%|████▊     | 4800/9864 [3:30:40<22:26:22, 15.95s/it]
                                                        
{'loss': '0.1464', 'grad_norm': '6.319', 'learning_rate': '1.092e-05', 'epoch': '0.4867'}

 49%|████▊     | 4800/9864 [3:30:40<22:26:22, 15.95s/it]
 49%|████▊     | 4801/9864 [3:30:55<21:59:02, 15.63s/it]
 49%|████▊     | 4802/9864 [3:31:09<21:29:51, 15.29s/it]
 49%|████▊     | 4803/9864 [3:31:24<21:09:04, 15.05s/it]
 49%|████▊     | 4804/9864 [3:31:38<20:52:00, 14.85s/it]
 49%|████▊     | 4805/9864 [3:31:56<22:06:54, 15.74s/it]
 49%|████▊     | 4806/9864 [3:32:12<22:24:37, 15.95s/it]
 49%|████▊     | 4807/9864 [3:32:27<21:53:18, 15.58s/it]
 49%|████▊     | 4808/9864 [3:32:43<22:10:11, 15.79s/it]
 49%|████▉     | 4809/9864 [3:32:58<21:44:42, 15.49s/it]
 49%|████▉     | 4810/9864 [3:33:13<21:15:45, 15.15s/it]
                                                        
{'loss': '0.1565', 'grad_norm': '2.77', 'learning_rate': '1.089e-05', 'epoch': '0.4877'}

 49%|████▉     | 4810/9864 [3:33:13<21:15:45, 15.15s/it]
 49%|████▉     | 4811/9864 [3:33:28<21:12:54, 15.11s/it]
 49%|████▉     | 4812/9864 [3:33:42<20:58:13, 14.94s/it]
 49%|████▉     | 4813/9864 [3:34:02<23:10:33, 16.52s/it]
 49%|████▉     | 4814/9864 [3:34:17<22:28:49, 16.03s/it]
 49%|████▉     | 4815/9864 [3:34:34<22:40:06, 16.16s/it]
 49%|████▉     | 4816/9864 [3:34:48<21:54:48, 15.63s/it]
 49%|████▉     | 4817/9864 [3:35:03<21:30:50, 15.35s/it]
 49%|████▉     | 4818/9864 [3:35:17<21:07:13, 15.07s/it]
 49%|████▉     | 4819/9864 [3:35:31<20:42:07, 14.77s/it]
 49%|████▉     | 4820/9864 [3:35:48<21:26:33, 15.30s/it]
                                                        
{'loss': '0.1499', 'grad_norm': '4.685', 'learning_rate': '1.086e-05', 'epoch': '0.4887'}

 49%|████▉     | 4820/9864 [3:35:48<21:26:33, 15.30s/it]
 49%|████▉     | 4821/9864 [3:36:04<21:56:42, 15.67s/it]
 49%|████▉     | 4822/9864 [3:36:21<22:16:46, 15.91s/it]
 49%|████▉     | 4823/9864 [3:36:37<22:29:50, 16.07s/it]
 49%|████▉     | 4824/9864 [3:36:52<21:55:31, 15.66s/it]
 49%|████▉     | 4825/9864 [3:37:07<21:40:33, 15.49s/it]
 49%|████▉     | 4826/9864 [3:37:22<21:22:46, 15.28s/it]
 49%|████▉     | 4827/9864 [3:37:37<21:17:44, 15.22s/it]
 49%|████▉     | 4828/9864 [3:37:55<22:18:03, 15.94s/it]
 49%|████▉     | 4829/9864 [3:38:11<22:22:47, 16.00s/it]
 49%|████▉     | 4830/9864 [3:38:25<21:50:54, 15.62s/it]
                                                        
{'loss': '0.1269', 'grad_norm': '2.094', 'learning_rate': '1.082e-05', 'epoch': '0.4897'}

 49%|████▉     | 4830/9864 [3:38:25<21:50:54, 15.62s/it]
 49%|████▉     | 4831/9864 [3:38:40<21:28:57, 15.37s/it]
 49%|████▉     | 4832/9864 [3:38:57<22:00:49, 15.75s/it]
 49%|████▉     | 4833/9864 [3:39:12<21:41:58, 15.53s/it]
 49%|████▉     | 4834/9864 [3:39:27<21:26:19, 15.34s/it]
 49%|████▉     | 4835/9864 [3:39:41<21:10:38, 15.16s/it]
 49%|████▉     | 4836/9864 [3:39:58<21:53:25, 15.67s/it]
 49%|████▉     | 4837/9864 [3:40:15<22:17:44, 15.97s/it]
 49%|████▉     | 4838/9864 [3:40:31<22:28:30, 16.10s/it]
 49%|████▉     | 4839/9864 [3:40:46<21:56:12, 15.72s/it]
 49%|████▉     | 4840/9864 [3:41:03<22:24:21, 16.06s/it]
                                                        
{'loss': '0.1376', 'grad_norm': '3.195', 'learning_rate': '1.079e-05', 'epoch': '0.4907'}

 49%|████▉     | 4840/9864 [3:41:03<22:24:21, 16.06s/it]
 49%|████▉     | 4841/9864 [3:41:18<22:08:17, 15.87s/it]
 49%|████▉     | 4842/9864 [3:41:33<21:44:25, 15.58s/it]
 49%|████▉     | 4843/9864 [3:41:50<22:19:04, 16.00s/it]
 49%|████▉     | 4844/9864 [3:42:07<22:28:16, 16.11s/it]
 49%|████▉     | 4845/9864 [3:42:23<22:41:55, 16.28s/it]
 49%|████▉     | 4846/9864 [3:42:38<22:01:37, 15.80s/it]
 49%|████▉     | 4847/9864 [3:42:53<21:41:08, 15.56s/it]
 49%|████▉     | 4848/9864 [3:43:08<21:36:08, 15.50s/it]
 49%|████▉     | 4849/9864 [3:43:26<22:17:41, 16.00s/it]
 49%|████▉     | 4850/9864 [3:43:41<22:02:30, 15.83s/it]
                                                        
{'loss': '0.1474', 'grad_norm': '3.481', 'learning_rate': '1.076e-05', 'epoch': '0.4917'}

 49%|████▉     | 4850/9864 [3:43:41<22:02:30, 15.83s/it]
 49%|████▉     | 4851/9864 [3:43:58<22:18:57, 16.03s/it]
 49%|████▉     | 4852/9864 [3:44:14<22:20:04, 16.04s/it]
 49%|████▉     | 4853/9864 [3:44:31<22:43:41, 16.33s/it]
 49%|████▉     | 4854/9864 [3:44:45<21:58:06, 15.79s/it]
 49%|████▉     | 4855/9864 [3:45:00<21:38:40, 15.56s/it]
 49%|████▉     | 4856/9864 [3:45:15<21:20:47, 15.34s/it]
 49%|████▉     | 4857/9864 [3:45:32<21:49:00, 15.69s/it]
 49%|████▉     | 4858/9864 [3:45:47<21:33:05, 15.50s/it]
 49%|████▉     | 4859/9864 [3:46:01<21:16:07, 15.30s/it]
 49%|████▉     | 4860/9864 [3:46:20<22:36:14, 16.26s/it]
                                                        
{'loss': '0.1332', 'grad_norm': '5.691', 'learning_rate': '1.073e-05', 'epoch': '0.4927'}

 49%|████▉     | 4860/9864 [3:46:20<22:36:14, 16.26s/it]
 49%|████▉     | 4861/9864 [3:46:37<22:48:07, 16.41s/it]
 49%|████▉     | 4862/9864 [3:46:51<22:04:21, 15.89s/it]
 49%|████▉     | 4863/9864 [3:47:06<21:43:37, 15.64s/it]
 49%|████▉     | 4864/9864 [3:47:23<22:06:21, 15.92s/it]
 49%|████▉     | 4865/9864 [3:47:38<21:47:45, 15.70s/it]
 49%|████▉     | 4866/9864 [3:47:53<21:30:50, 15.50s/it]
 49%|████▉     | 4867/9864 [3:48:08<21:05:36, 15.20s/it]
 49%|████▉     | 4868/9864 [3:48:26<22:13:41, 16.02s/it]
 49%|████▉     | 4869/9864 [3:48:42<22:29:55, 16.22s/it]
 49%|████▉     | 4870/9864 [3:48:57<21:51:10, 15.75s/it]
                                                        
{'loss': '0.1328', 'grad_norm': '4.705', 'learning_rate': '1.069e-05', 'epoch': '0.4938'}

 49%|████▉     | 4870/9864 [3:48:57<21:51:10, 15.75s/it]
 49%|████▉     | 4871/9864 [3:49:14<22:21:48, 16.12s/it]
 49%|████▉     | 4872/9864 [3:49:29<21:43:18, 15.66s/it]
 49%|████▉     | 4873/9864 [3:49:43<21:15:28, 15.33s/it]
 49%|████▉     | 4874/9864 [3:49:58<20:54:32, 15.08s/it]
 49%|████▉     | 4875/9864 [3:50:13<20:50:45, 15.04s/it]
 49%|████▉     | 4876/9864 [3:50:29<21:31:32, 15.54s/it]
 49%|████▉     | 4877/9864 [3:50:46<21:52:03, 15.79s/it]
 49%|████▉     | 4878/9864 [3:51:01<21:40:44, 15.65s/it]
 49%|████▉     | 4879/9864 [3:51:17<22:01:18, 15.90s/it]
 49%|████▉     | 4880/9864 [3:51:32<21:24:49, 15.47s/it]
                                                        
{'loss': '0.1397', 'grad_norm': '2.835', 'learning_rate': '1.066e-05', 'epoch': '0.4948'}

 49%|████▉     | 4880/9864 [3:51:32<21:24:49, 15.47s/it]
 49%|████▉     | 4881/9864 [3:51:47<21:17:38, 15.38s/it]
 49%|████▉     | 4882/9864 [3:52:02<21:12:42, 15.33s/it]
 50%|████▉     | 4883/9864 [3:52:17<20:53:28, 15.10s/it]
 50%|████▉     | 4884/9864 [3:52:34<21:56:12, 15.86s/it]
 50%|████▉     | 4885/9864 [3:52:49<21:21:27, 15.44s/it]
 50%|████▉     | 4886/9864 [3:53:06<21:57:31, 15.88s/it]
 50%|████▉     | 4887/9864 [3:53:20<21:11:29, 15.33s/it]
 50%|████▉     | 4888/9864 [3:53:35<21:00:41, 15.20s/it]
 50%|████▉     | 4889/9864 [3:53:49<20:46:08, 15.03s/it]
 50%|████▉     | 4890/9864 [3:54:04<20:39:14, 14.95s/it]
                                                        
{'loss': '0.1378', 'grad_norm': '6.671', 'learning_rate': '1.063e-05', 'epoch': '0.4958'}

 50%|████▉     | 4890/9864 [3:54:04<20:39:14, 14.95s/it]
 50%|████▉     | 4891/9864 [3:54:19<20:31:55, 14.86s/it]
 50%|████▉     | 4892/9864 [3:54:37<21:56:19, 15.88s/it]
 50%|████▉     | 4893/9864 [3:54:53<21:45:44, 15.76s/it]
 50%|████▉     | 4894/9864 [3:55:10<22:36:57, 16.38s/it]
 50%|████▉     | 4895/9864 [3:55:26<22:06:42, 16.02s/it]
 50%|████▉     | 4896/9864 [3:55:40<21:19:50, 15.46s/it]
 50%|████▉     | 4897/9864 [3:55:55<21:15:56, 15.41s/it]
 50%|████▉     | 4898/9864 [3:56:10<21:05:44, 15.29s/it]
 50%|████▉     | 4899/9864 [3:56:25<20:47:25, 15.07s/it]
 50%|████▉     | 4900/9864 [3:56:41<21:18:10, 15.45s/it]
                                                        
{'loss': '0.1346', 'grad_norm': '3.172', 'learning_rate': '1.059e-05', 'epoch': '0.4968'}

 50%|████▉     | 4900/9864 [3:56:41<21:18:10, 15.45s/it]
 50%|████▉     | 4901/9864 [3:57:01<22:59:51, 16.68s/it]
 50%|████▉     | 4902/9864 [3:57:15<21:59:49, 15.96s/it]
 50%|████▉     | 4903/9864 [3:57:29<21:11:23, 15.38s/it]
 50%|████▉     | 4904/9864 [3:57:44<21:05:21, 15.31s/it]
 50%|████▉     | 4905/9864 [3:57:59<21:08:12, 15.34s/it]
 50%|████▉     | 4906/9864 [3:58:14<20:54:57, 15.19s/it]
 50%|████▉     | 4907/9864 [3:58:29<20:43:46, 15.05s/it]
 50%|████▉     | 4908/9864 [3:58:46<21:24:02, 15.55s/it]
 50%|████▉     | 4909/9864 [3:59:06<23:15:34, 16.90s/it]
 50%|████▉     | 4910/9864 [3:59:21<22:29:15, 16.34s/it]
                                                        
{'loss': '0.1479', 'grad_norm': '2.607', 'learning_rate': '1.056e-05', 'epoch': '0.4978'}

 50%|████▉     | 4910/9864 [3:59:21<22:29:15, 16.34s/it]
 50%|████▉     | 4911/9864 [3:59:36<22:04:27, 16.04s/it]
 50%|████▉     | 4912/9864 [3:59:50<21:15:56, 15.46s/it]
 50%|████▉     | 4913/9864 [4:00:05<20:58:32, 15.25s/it]
 50%|████▉     | 4914/9864 [4:00:19<20:36:40, 14.99s/it]
 50%|████▉     | 4915/9864 [4:00:34<20:26:48, 14.87s/it]
 50%|████▉     | 4916/9864 [4:00:50<20:59:02, 15.27s/it]
 50%|████▉     | 4917/9864 [4:01:08<22:13:14, 16.17s/it]
 50%|████▉     | 4918/9864 [4:01:25<22:23:02, 16.29s/it]
 50%|████▉     | 4919/9864 [4:01:40<21:44:49, 15.83s/it]
 50%|████▉     | 4920/9864 [4:01:55<21:26:14, 15.61s/it]
                                                        
{'loss': '0.1403', 'grad_norm': '2.58', 'learning_rate': '1.053e-05', 'epoch': '0.4988'}

 50%|████▉     | 4920/9864 [4:01:55<21:26:14, 15.61s/it]
 50%|████▉     | 4921/9864 [4:02:09<20:56:48, 15.26s/it]
 50%|████▉     | 4922/9864 [4:02:24<20:46:48, 15.14s/it]
 50%|████▉     | 4923/9864 [4:02:41<21:26:16, 15.62s/it]
 50%|████▉     | 4924/9864 [4:02:56<21:08:09, 15.40s/it]
 50%|████▉     | 4925/9864 [4:03:14<22:30:22, 16.40s/it]
 50%|████▉     | 4926/9864 [4:03:31<22:42:27, 16.55s/it]
 50%|████▉     | 4927/9864 [4:03:46<22:02:26, 16.07s/it]
 50%|████▉     | 4928/9864 [4:04:01<21:26:59, 15.64s/it]
 50%|████▉     | 4929/9864 [4:04:16<21:02:06, 15.34s/it]
 50%|████▉     | 4930/9864 [4:04:30<20:44:51, 15.14s/it]
                                                        
{'loss': '0.1314', 'grad_norm': '1.981', 'learning_rate': '1.05e-05', 'epoch': '0.4998'}

 50%|████▉     | 4930/9864 [4:04:30<20:44:51, 15.14s/it]
 50%|████▉     | 4931/9864 [4:04:45<20:31:22, 14.98s/it]
 50%|█████     | 4932/9864 [4:05:03<21:55:39, 16.01s/it]
 50%|█████     | 4933/9864 [4:05:19<21:54:50, 16.00s/it]
 50%|█████     | 4934/9864 [4:05:34<21:13:45, 15.50s/it]
 50%|█████     | 4935/9864 [4:05:50<21:37:39, 15.80s/it]
 50%|█████     | 4936/9864 [4:06:05<21:08:12, 15.44s/it]
 50%|█████     | 4937/9864 [4:06:19<20:46:35, 15.18s/it]
 50%|█████     | 4938/9864 [4:06:34<20:31:06, 15.00s/it]
 50%|█████     | 4939/9864 [4:06:51<21:14:13, 15.52s/it]
 50%|█████     | 4940/9864 [4:07:09<22:23:19, 16.37s/it]
                                                        
{'loss': '0.1258', 'grad_norm': '3.28', 'learning_rate': '1.046e-05', 'epoch': '0.5009'}

 50%|█████     | 4940/9864 [4:07:09<22:23:19, 16.37s/it]
 50%|█████     | 4941/9864 [4:07:24<21:45:12, 15.91s/it]
 50%|█████     | 4942/9864 [4:07:39<21:21:16, 15.62s/it]
 50%|█████     | 4943/9864 [4:07:55<21:38:01, 15.83s/it]
 50%|█████     | 4944/9864 [4:08:10<21:20:18, 15.61s/it]
 50%|█████     | 4945/9864 [4:08:25<20:59:38, 15.36s/it]
 50%|█████     | 4946/9864 [4:08:40<20:48:19, 15.23s/it]
 50%|█████     | 4947/9864 [4:08:56<21:11:16, 15.51s/it]
 50%|█████     | 4948/9864 [4:09:14<22:16:51, 16.32s/it]
 50%|█████     | 4949/9864 [4:09:29<21:45:29, 15.94s/it]
 50%|█████     | 4950/9864 [4:09:46<21:56:52, 16.08s/it]
                                                        
{'loss': '0.1268', 'grad_norm': '2.101', 'learning_rate': '1.043e-05', 'epoch': '0.5019'}

 50%|█████     | 4950/9864 [4:09:46<21:56:52, 16.08s/it]
 50%|█████     | 4951/9864 [4:10:01<21:31:38, 15.77s/it]
 50%|█████     | 4952/9864 [4:10:15<21:00:02, 15.39s/it]
 50%|█████     | 4953/9864 [4:10:30<20:48:49, 15.26s/it]
 50%|█████     | 4954/9864 [4:10:45<20:38:26, 15.13s/it]
 50%|█████     | 4955/9864 [4:11:03<21:43:10, 15.93s/it]
 50%|█████     | 4956/9864 [4:11:17<21:11:43, 15.55s/it]
 50%|█████     | 4957/9864 [4:11:34<21:40:10, 15.90s/it]
 50%|█████     | 4958/9864 [4:11:48<20:57:26, 15.38s/it]
 50%|█████     | 4959/9864 [4:12:03<20:44:35, 15.22s/it]
 50%|█████     | 4960/9864 [4:12:18<20:31:53, 15.07s/it]
                                                        
{'loss': '0.1312', 'grad_norm': '2.716', 'learning_rate': '1.04e-05', 'epoch': '0.5029'}

 50%|█████     | 4960/9864 [4:12:18<20:31:53, 15.07s/it]
 50%|█████     | 4961/9864 [4:12:33<20:20:58, 14.94s/it]
 50%|█████     | 4962/9864 [4:12:49<21:00:47, 15.43s/it]
 50%|█████     | 4963/9864 [4:13:08<22:19:57, 16.40s/it]
 50%|█████     | 4964/9864 [4:13:23<21:47:09, 16.01s/it]
 50%|█████     | 4965/9864 [4:13:39<21:56:04, 16.12s/it]
 50%|█████     | 4966/9864 [4:13:54<21:13:50, 15.60s/it]
 50%|█████     | 4967/9864 [4:14:09<20:55:59, 15.39s/it]
 50%|█████     | 4968/9864 [4:14:23<20:44:12, 15.25s/it]
 50%|█████     | 4969/9864 [4:14:38<20:35:54, 15.15s/it]
 50%|█████     | 4970/9864 [4:14:54<20:56:30, 15.40s/it]
                                                        
{'loss': '0.1389', 'grad_norm': '2.787', 'learning_rate': '1.036e-05', 'epoch': '0.5039'}

 50%|█████     | 4970/9864 [4:14:54<20:56:30, 15.40s/it]
 50%|█████     | 4971/9864 [4:15:11<21:13:51, 15.62s/it]
 50%|█████     | 4972/9864 [4:15:25<20:53:01, 15.37s/it]
 50%|█████     | 4973/9864 [4:15:42<21:28:51, 15.81s/it]
 50%|█████     | 4974/9864 [4:15:57<21:04:15, 15.51s/it]
 50%|█████     | 4975/9864 [4:16:12<20:40:59, 15.23s/it]
 50%|█████     | 4976/9864 [4:16:26<20:22:41, 15.01s/it]
 50%|█████     | 4977/9864 [4:16:41<20:12:38, 14.89s/it]
 50%|█████     | 4978/9864 [4:16:57<20:44:33, 15.28s/it]
 50%|█████     | 4979/9864 [4:17:13<21:14:46, 15.66s/it]
 50%|█████     | 4980/9864 [4:17:28<20:49:18, 15.35s/it]
                                                        
{'loss': '0.1442', 'grad_norm': '3.207', 'learning_rate': '1.033e-05', 'epoch': '0.5049'}

 50%|█████     | 4980/9864 [4:17:28<20:49:18, 15.35s/it]
 50%|█████     | 4981/9864 [4:17:45<21:24:47, 15.79s/it]
 51%|█████     | 4982/9864 [4:18:00<20:59:09, 15.48s/it]
 51%|█████     | 4983/9864 [4:18:14<20:35:11, 15.18s/it]
 51%|█████     | 4984/9864 [4:18:29<20:23:52, 15.05s/it]
 51%|█████     | 4985/9864 [4:18:44<20:18:53, 14.99s/it]
 51%|█████     | 4986/9864 [4:19:02<21:39:37, 15.99s/it]
 51%|█████     | 4987/9864 [4:19:18<21:48:51, 16.10s/it]
 51%|█████     | 4988/9864 [4:19:33<21:07:06, 15.59s/it]
 51%|█████     | 4989/9864 [4:19:50<21:38:10, 15.98s/it]
 51%|█████     | 4990/9864 [4:20:04<20:58:35, 15.49s/it]
                                                        
{'loss': '0.1345', 'grad_norm': '1.769', 'learning_rate': '1.03e-05', 'epoch': '0.5059'}

 51%|█████     | 4990/9864 [4:20:04<20:58:35, 15.49s/it]
 51%|█████     | 4991/9864 [4:20:19<20:40:08, 15.27s/it]
 51%|█████     | 4992/9864 [4:20:33<20:24:11, 15.08s/it]
 51%|█████     | 4993/9864 [4:20:47<19:57:50, 14.75s/it]
 51%|█████     | 4994/9864 [4:21:04<20:43:40, 15.32s/it]
 51%|█████     | 4995/9864 [4:21:22<21:54:43, 16.20s/it]
 51%|█████     | 4996/9864 [4:21:37<21:07:18, 15.62s/it]
 51%|█████     | 4997/9864 [4:21:53<21:19:23, 15.77s/it]
 51%|█████     | 4998/9864 [4:22:08<21:08:05, 15.64s/it]
 51%|█████     | 4999/9864 [4:22:23<20:53:15, 15.46s/it]
 51%|█████     | 5000/9864 [4:22:38<20:40:04, 15.30s/it]
                                                        
{'loss': '0.1487', 'grad_norm': '4.101', 'learning_rate': '1.027e-05', 'epoch': '0.5069'}

 51%|█████     | 5000/9864 [4:22:38<20:40:04, 15.30s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:13<00:00, 13.60s/it]
Writing model shards: 100%|██████████| 1/1 [00:13<00:00, 13.60s/it]
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 51%|█████     | 5001/9864 [4:24:14<53:18:08, 39.46s/it]
 51%|█████     | 5002/9864 [4:24:31<44:25:26, 32.89s/it]
 51%|█████     | 5003/9864 [4:24:48<37:42:27, 27.93s/it]
 51%|█████     | 5004/9864 [4:25:02<32:16:10, 23.90s/it]
 51%|█████     | 5005/9864 [4:25:16<28:14:03, 20.92s/it]
 51%|█████     | 5006/9864 [4:25:30<25:31:45, 18.92s/it]
 51%|█████     | 5007/9864 [4:25:45<23:38:08, 17.52s/it]
 51%|█████     | 5008/9864 [4:25:59<22:19:34, 16.55s/it]
 51%|█████     | 5009/9864 [4:26:17<22:57:28, 17.02s/it]
 51%|█████     | 5010/9864 [4:26:31<21:43:05, 16.11s/it]
                                                        
{'loss': '0.1301', 'grad_norm': '1.697', 'learning_rate': '1.023e-05', 'epoch': '0.5079'}

 51%|█████     | 5010/9864 [4:26:31<21:43:05, 16.11s/it]
 51%|█████     | 5011/9864 [4:26:47<21:43:16, 16.11s/it]
 51%|█████     | 5012/9864 [4:27:04<21:54:14, 16.25s/it]
 51%|█████     | 5013/9864 [4:27:18<21:04:49, 15.64s/it]
 51%|█████     | 5014/9864 [4:27:33<20:48:47, 15.45s/it]
 51%|█████     | 5015/9864 [4:27:47<20:22:56, 15.13s/it]
 51%|█████     | 5016/9864 [4:28:05<21:33:18, 16.01s/it]
 51%|█████     | 5017/9864 [4:28:20<21:08:22, 15.70s/it]
 51%|█████     | 5018/9864 [4:28:35<20:38:16, 15.33s/it]
 51%|█████     | 5019/9864 [4:28:53<21:40:26, 16.10s/it]
 51%|█████     | 5020/9864 [4:29:07<20:56:21, 15.56s/it]
                                                        
{'loss': '0.1338', 'grad_norm': '5.254', 'learning_rate': '1.02e-05', 'epoch': '0.509'}

 51%|█████     | 5020/9864 [4:29:07<20:56:21, 15.56s/it]
 51%|█████     | 5021/9864 [4:29:21<20:20:21, 15.12s/it]
 51%|█████     | 5022/9864 [4:29:35<19:56:13, 14.82s/it]
 51%|█████     | 5023/9864 [4:29:49<19:33:50, 14.55s/it]
 51%|█████     | 5024/9864 [4:30:06<20:22:56, 15.16s/it]
 51%|█████     | 5025/9864 [4:30:22<20:50:39, 15.51s/it]
 51%|█████     | 5026/9864 [4:30:38<21:01:46, 15.65s/it]
 51%|█████     | 5027/9864 [4:30:54<21:05:34, 15.70s/it]
 51%|█████     | 5028/9864 [4:31:08<20:35:29, 15.33s/it]
 51%|█████     | 5029/9864 [4:31:23<20:14:29, 15.07s/it]
 51%|█████     | 5030/9864 [4:31:37<20:03:15, 14.93s/it]
                                                        
{'loss': '0.1335', 'grad_norm': '3.25', 'learning_rate': '1.017e-05', 'epoch': '0.51'}

 51%|█████     | 5030/9864 [4:31:37<20:03:15, 14.93s/it]
 51%|█████     | 5031/9864 [4:31:52<19:55:22, 14.84s/it]
 51%|█████     | 5032/9864 [4:32:08<20:25:41, 15.22s/it]
 51%|█████     | 5033/9864 [4:32:24<20:44:07, 15.45s/it]
 51%|█████     | 5034/9864 [4:32:40<20:59:20, 15.64s/it]
 51%|█████     | 5035/9864 [4:32:57<21:25:07, 15.97s/it]
 51%|█████     | 5036/9864 [4:33:11<20:39:34, 15.40s/it]
 51%|█████     | 5037/9864 [4:33:26<20:19:38, 15.16s/it]
 51%|█████     | 5038/9864 [4:33:40<20:00:58, 14.93s/it]
 51%|█████     | 5039/9864 [4:33:56<20:36:18, 15.37s/it]
 51%|█████     | 5040/9864 [4:34:11<20:11:09, 15.06s/it]
                                                        
{'loss': '0.1376', 'grad_norm': '3.888', 'learning_rate': '1.013e-05', 'epoch': '0.511'}

 51%|█████     | 5040/9864 [4:34:11<20:11:09, 15.06s/it]
 51%|█████     | 5041/9864 [4:34:25<19:44:47, 14.74s/it]
 51%|█████     | 5042/9864 [4:34:43<21:02:24, 15.71s/it]
 51%|█████     | 5043/9864 [4:34:59<21:08:38, 15.79s/it]
 51%|█████     | 5044/9864 [4:35:13<20:26:10, 15.26s/it]
 51%|█████     | 5045/9864 [4:35:27<20:10:51, 15.08s/it]
 51%|█████     | 5046/9864 [4:35:42<19:52:52, 14.86s/it]
 51%|█████     | 5047/9864 [4:35:57<20:14:16, 15.12s/it]
 51%|█████     | 5048/9864 [4:36:12<19:49:59, 14.83s/it]
 51%|█████     | 5049/9864 [4:36:27<20:14:18, 15.13s/it]
 51%|█████     | 5050/9864 [4:36:44<20:39:30, 15.45s/it]
                                                        
{'loss': '0.1366', 'grad_norm': '2.866', 'learning_rate': '1.01e-05', 'epoch': '0.512'}

 51%|█████     | 5050/9864 [4:36:44<20:39:30, 15.45s/it]
 51%|█████     | 5051/9864 [4:37:00<21:05:45, 15.78s/it]
 51%|█████     | 5052/9864 [4:37:14<20:27:00, 15.30s/it]
 51%|█████     | 5053/9864 [4:37:29<20:06:49, 15.05s/it]
 51%|█████     | 5054/9864 [4:37:43<19:46:37, 14.80s/it]
 51%|█████     | 5055/9864 [4:37:58<20:01:12, 14.99s/it]
 51%|█████▏    | 5056/9864 [4:38:13<19:40:01, 14.73s/it]
 51%|█████▏    | 5057/9864 [4:38:29<20:16:44, 15.19s/it]
 51%|█████▏    | 5058/9864 [4:38:44<20:18:12, 15.21s/it]
 51%|█████▏    | 5059/9864 [4:39:00<20:36:56, 15.45s/it]
 51%|█████▏    | 5060/9864 [4:39:14<20:05:27, 15.06s/it]
                                                        
{'loss': '0.1239', 'grad_norm': '4.738', 'learning_rate': '1.007e-05', 'epoch': '0.513'}

 51%|█████▏    | 5060/9864 [4:39:14<20:05:27, 15.06s/it]
 51%|█████▏    | 5061/9864 [4:39:29<19:46:19, 14.82s/it]
 51%|█████▏    | 5062/9864 [4:39:44<20:12:42, 15.15s/it]
 51%|█████▏    | 5063/9864 [4:39:59<19:51:34, 14.89s/it]
 51%|█████▏    | 5064/9864 [4:40:15<20:24:47, 15.31s/it]
 51%|█████▏    | 5065/9864 [4:40:30<20:09:15, 15.12s/it]
 51%|█████▏    | 5066/9864 [4:40:46<20:31:24, 15.40s/it]
 51%|█████▏    | 5067/9864 [4:41:01<20:36:51, 15.47s/it]
 51%|█████▏    | 5068/9864 [4:41:16<20:12:29, 15.17s/it]
 51%|█████▏    | 5069/9864 [4:41:32<20:39:30, 15.51s/it]
 51%|█████▏    | 5070/9864 [4:41:46<20:08:38, 15.13s/it]
                                                        
{'loss': '0.1382', 'grad_norm': '3.899', 'learning_rate': '1.004e-05', 'epoch': '0.514'}

 51%|█████▏    | 5070/9864 [4:41:46<20:08:38, 15.13s/it]
 51%|█████▏    | 5071/9864 [4:42:01<19:58:52, 15.01s/it]
 51%|█████▏    | 5072/9864 [4:42:15<19:42:51, 14.81s/it]
 51%|█████▏    | 5073/9864 [4:42:34<21:15:11, 15.97s/it]
 51%|█████▏    | 5074/9864 [4:42:50<21:18:51, 16.02s/it]
 51%|█████▏    | 5075/9864 [4:43:05<20:50:41, 15.67s/it]
 51%|█████▏    | 5076/9864 [4:43:22<21:07:40, 15.89s/it]
 51%|█████▏    | 5077/9864 [4:43:36<20:38:41, 15.53s/it]
 51%|█████▏    | 5078/9864 [4:43:51<20:13:33, 15.21s/it]
 51%|█████▏    | 5079/9864 [4:44:05<19:51:56, 14.95s/it]
 52%|█████▏    | 5080/9864 [4:44:23<20:53:18, 15.72s/it]
                                                        
{'loss': '0.1351', 'grad_norm': '4.999', 'learning_rate': '1e-05', 'epoch': '0.515'}

 52%|█████▏    | 5080/9864 [4:44:23<20:53:18, 15.72s/it]
 52%|█████▏    | 5081/9864 [4:44:37<20:29:31, 15.42s/it]
 52%|█████▏    | 5082/9864 [4:44:54<20:53:05, 15.72s/it]
 52%|█████▏    | 5083/9864 [4:45:09<20:35:35, 15.51s/it]
 52%|█████▏    | 5084/9864 [4:45:25<20:58:57, 15.80s/it]
 52%|█████▏    | 5085/9864 [4:45:40<20:27:04, 15.41s/it]
 52%|█████▏    | 5086/9864 [4:45:53<19:46:38, 14.90s/it]
 52%|█████▏    | 5087/9864 [4:46:09<20:11:14, 15.21s/it]
 52%|█████▏    | 5088/9864 [4:46:26<20:42:30, 15.61s/it]
 52%|█████▏    | 5089/9864 [4:46:42<20:55:11, 15.77s/it]
 52%|█████▏    | 5090/9864 [4:46:57<20:24:21, 15.39s/it]
                                                        
{'loss': '0.1312', 'grad_norm': '3.43', 'learning_rate': '9.97e-06', 'epoch': '0.5161'}

 52%|█████▏    | 5090/9864 [4:46:57<20:24:21, 15.39s/it]
 52%|█████▏    | 5091/9864 [4:47:12<20:28:46, 15.45s/it]
 52%|█████▏    | 5092/9864 [4:47:27<20:15:15, 15.28s/it]
 52%|█████▏    | 5093/9864 [4:47:41<19:51:36, 14.99s/it]
 52%|█████▏    | 5094/9864 [4:47:55<19:29:48, 14.71s/it]
 52%|█████▏    | 5095/9864 [4:48:11<19:58:37, 15.08s/it]
 52%|█████▏    | 5096/9864 [4:48:27<20:12:42, 15.26s/it]
 52%|█████▏    | 5097/9864 [4:48:42<20:16:39, 15.31s/it]
 52%|█████▏    | 5098/9864 [4:48:57<19:50:49, 14.99s/it]
 52%|█████▏    | 5099/9864 [4:49:12<20:03:02, 15.15s/it]
 52%|█████▏    | 5100/9864 [4:49:27<19:57:35, 15.08s/it]
                                                        
{'loss': '0.1288', 'grad_norm': '2.916', 'learning_rate': '9.938e-06', 'epoch': '0.5171'}

 52%|█████▏    | 5100/9864 [4:49:27<19:57:35, 15.08s/it]
 52%|█████▏    | 5101/9864 [4:49:42<19:45:19, 14.93s/it]
 52%|█████▏    | 5102/9864 [4:49:58<20:11:08, 15.26s/it]
 52%|█████▏    | 5103/9864 [4:50:14<20:32:09, 15.53s/it]
 52%|█████▏    | 5104/9864 [4:50:28<20:07:55, 15.23s/it]
 52%|█████▏    | 5105/9864 [4:50:45<20:36:59, 15.60s/it]
 52%|█████▏    | 5106/9864 [4:51:01<20:39:29, 15.63s/it]
 52%|█████▏    | 5107/9864 [4:51:15<20:19:36, 15.38s/it]
 52%|█████▏    | 5108/9864 [4:51:30<19:53:09, 15.05s/it]
 52%|█████▏    | 5109/9864 [4:51:44<19:31:08, 14.78s/it]
 52%|█████▏    | 5110/9864 [4:52:00<20:02:04, 15.17s/it]
                                                        
{'loss': '0.1416', 'grad_norm': '3.306', 'learning_rate': '9.905e-06', 'epoch': '0.5181'}

 52%|█████▏    | 5110/9864 [4:52:00<20:02:04, 15.17s/it]
 52%|█████▏    | 5111/9864 [4:52:16<20:21:28, 15.42s/it]
 52%|█████▏    | 5112/9864 [4:52:31<20:05:57, 15.23s/it]
 52%|█████▏    | 5113/9864 [4:52:49<21:08:06, 16.01s/it]
 52%|█████▏    | 5114/9864 [4:53:03<20:34:43, 15.60s/it]
 52%|█████▏    | 5115/9864 [4:53:18<20:05:45, 15.23s/it]
 52%|█████▏    | 5116/9864 [4:53:32<19:47:37, 15.01s/it]
 52%|█████▏    | 5117/9864 [4:53:48<20:06:45, 15.25s/it]
 52%|█████▏    | 5118/9864 [4:54:02<19:47:44, 15.02s/it]
 52%|█████▏    | 5119/9864 [4:54:17<19:39:37, 14.92s/it]
 52%|█████▏    | 5120/9864 [4:54:34<20:22:40, 15.46s/it]
                                                        
{'loss': '0.1358', 'grad_norm': '2.168', 'learning_rate': '9.872e-06', 'epoch': '0.5191'}

 52%|█████▏    | 5120/9864 [4:54:34<20:22:40, 15.46s/it]
 52%|█████▏    | 5121/9864 [4:54:49<20:27:25, 15.53s/it]
 52%|█████▏    | 5122/9864 [4:55:05<20:36:04, 15.64s/it]
 52%|█████▏    | 5123/9864 [4:55:20<20:06:31, 15.27s/it]
 52%|█████▏    | 5124/9864 [4:55:34<19:46:33, 15.02s/it]
 52%|█████▏    | 5125/9864 [4:55:50<20:01:34, 15.21s/it]
 52%|█████▏    | 5126/9864 [4:56:05<19:51:33, 15.09s/it]
 52%|█████▏    | 5127/9864 [4:56:21<20:27:57, 15.55s/it]
 52%|█████▏    | 5128/9864 [4:56:36<19:58:09, 15.18s/it]
 52%|█████▏    | 5129/9864 [4:56:52<20:19:07, 15.45s/it]
 52%|█████▏    | 5130/9864 [4:57:08<20:32:31, 15.62s/it]
                                                        
{'loss': '0.1342', 'grad_norm': '2.534', 'learning_rate': '9.839e-06', 'epoch': '0.5201'}

 52%|█████▏    | 5130/9864 [4:57:08<20:32:31, 15.62s/it]
 52%|█████▏    | 5131/9864 [4:57:22<19:53:05, 15.12s/it]
 52%|█████▏    | 5132/9864 [4:57:36<19:43:31, 15.01s/it]
 52%|█████▏    | 5133/9864 [4:57:52<19:59:10, 15.21s/it]
 52%|█████▏    | 5134/9864 [4:58:07<19:45:00, 15.03s/it]
 52%|█████▏    | 5135/9864 [4:58:23<20:16:42, 15.44s/it]
 52%|█████▏    | 5136/9864 [4:58:39<20:34:22, 15.66s/it]
 52%|█████▏    | 5137/9864 [4:58:54<20:09:11, 15.35s/it]
 52%|█████▏    | 5138/9864 [4:59:10<20:29:29, 15.61s/it]
 52%|█████▏    | 5139/9864 [4:59:25<20:03:37, 15.28s/it]
 52%|█████▏    | 5140/9864 [4:59:39<19:41:08, 15.00s/it]
                                                        
{'loss': '0.141', 'grad_norm': '3.615', 'learning_rate': '9.806e-06', 'epoch': '0.5211'}

 52%|█████▏    | 5140/9864 [4:59:39<19:41:08, 15.00s/it]
 52%|█████▏    | 5141/9864 [4:59:53<19:17:42, 14.71s/it]
 52%|█████▏    | 5142/9864 [5:00:09<19:38:31, 14.97s/it]
 52%|█████▏    | 5143/9864 [5:00:25<20:08:55, 15.36s/it]
 52%|█████▏    | 5144/9864 [5:00:41<20:32:51, 15.67s/it]
 52%|█████▏    | 5145/9864 [5:00:56<20:05:38, 15.33s/it]
 52%|█████▏    | 5146/9864 [5:01:12<20:20:10, 15.52s/it]
 52%|█████▏    | 5147/9864 [5:01:26<19:53:29, 15.18s/it]
 52%|█████▏    | 5148/9864 [5:01:40<19:32:14, 14.91s/it]
 52%|█████▏    | 5149/9864 [5:01:55<19:26:48, 14.85s/it]
 52%|█████▏    | 5150/9864 [5:02:11<19:52:16, 15.18s/it]
                                                        
{'loss': '0.128', 'grad_norm': '3.572', 'learning_rate': '9.773e-06', 'epoch': '0.5221'}

 52%|█████▏    | 5150/9864 [5:02:11<19:52:16, 15.18s/it]
 52%|█████▏    | 5151/9864 [5:02:27<20:15:35, 15.48s/it]
 52%|█████▏    | 5152/9864 [5:02:41<19:45:31, 15.10s/it]
 52%|█████▏    | 5153/9864 [5:02:58<20:10:43, 15.42s/it]
 52%|█████▏    | 5154/9864 [5:03:14<20:40:34, 15.80s/it]
 52%|█████▏    | 5155/9864 [5:03:28<20:00:44, 15.30s/it]
 52%|█████▏    | 5156/9864 [5:03:43<19:35:06, 14.98s/it]
 52%|█████▏    | 5157/9864 [5:03:57<19:18:05, 14.76s/it]
 52%|█████▏    | 5158/9864 [5:04:13<19:48:10, 15.15s/it]
 52%|█████▏    | 5159/9864 [5:04:29<20:20:33, 15.56s/it]
 52%|█████▏    | 5160/9864 [5:04:44<20:04:54, 15.37s/it]
                                                        
{'loss': '0.1423', 'grad_norm': '4.082', 'learning_rate': '9.741e-06', 'epoch': '0.5232'}

 52%|█████▏    | 5160/9864 [5:04:44<20:04:54, 15.37s/it]
 52%|█████▏    | 5161/9864 [5:05:00<20:18:18, 15.54s/it]
 52%|█████▏    | 5162/9864 [5:05:16<20:28:59, 15.68s/it]
 52%|█████▏    | 5163/9864 [5:05:31<19:55:13, 15.26s/it]
 52%|█████▏    | 5164/9864 [5:05:45<19:24:59, 14.87s/it]
 52%|█████▏    | 5165/9864 [5:06:00<19:45:02, 15.13s/it]
 52%|█████▏    | 5166/9864 [5:06:16<20:07:19, 15.42s/it]
 52%|█████▏    | 5167/9864 [5:06:30<19:34:17, 15.00s/it]
 52%|█████▏    | 5168/9864 [5:06:44<19:11:00, 14.71s/it]
 52%|█████▏    | 5169/9864 [5:07:01<19:49:41, 15.20s/it]
 52%|█████▏    | 5170/9864 [5:07:15<19:19:29, 14.82s/it]
                                                        
{'loss': '0.1389', 'grad_norm': '3.219', 'learning_rate': '9.708e-06', 'epoch': '0.5242'}

 52%|█████▏    | 5170/9864 [5:07:15<19:19:29, 14.82s/it]
 52%|█████▏    | 5171/9864 [5:07:31<19:56:36, 15.30s/it]
 52%|█████▏    | 5172/9864 [5:07:45<19:22:44, 14.87s/it]
 52%|█████▏    | 5173/9864 [5:08:01<19:43:32, 15.14s/it]
 52%|█████▏    | 5174/9864 [5:08:17<20:05:18, 15.42s/it]
 52%|█████▏    | 5175/9864 [5:08:31<19:40:01, 15.10s/it]
 52%|█████▏    | 5176/9864 [5:08:45<19:19:53, 14.85s/it]
 52%|█████▏    | 5177/9864 [5:09:01<19:46:28, 15.19s/it]
 52%|█████▏    | 5178/9864 [5:09:18<20:06:55, 15.45s/it]
 53%|█████▎    | 5179/9864 [5:09:32<19:38:57, 15.10s/it]
 53%|█████▎    | 5180/9864 [5:09:48<20:13:48, 15.55s/it]
                                                        
{'loss': '0.1337', 'grad_norm': '2.953', 'learning_rate': '9.675e-06', 'epoch': '0.5252'}

 53%|█████▎    | 5180/9864 [5:09:48<20:13:48, 15.55s/it]
 53%|█████▎    | 5181/9864 [5:10:03<19:55:02, 15.31s/it]
 53%|█████▎    | 5182/9864 [5:10:19<20:13:13, 15.55s/it]
 53%|█████▎    | 5183/9864 [5:10:33<19:42:50, 15.16s/it]
 53%|█████▎    | 5184/9864 [5:10:48<19:27:45, 14.97s/it]
 53%|█████▎    | 5185/9864 [5:11:02<19:11:40, 14.77s/it]
 53%|█████▎    | 5186/9864 [5:11:18<19:40:49, 15.15s/it]
 53%|█████▎    | 5187/9864 [5:11:37<20:56:05, 16.11s/it]
 53%|█████▎    | 5188/9864 [5:11:51<20:16:12, 15.61s/it]
 53%|█████▎    | 5189/9864 [5:12:07<20:30:04, 15.79s/it]
 53%|█████▎    | 5190/9864 [5:12:22<20:08:48, 15.52s/it]
                                                        
{'loss': '0.1412', 'grad_norm': '2.579', 'learning_rate': '9.642e-06', 'epoch': '0.5262'}

 53%|█████▎    | 5190/9864 [5:12:22<20:08:48, 15.52s/it]
 53%|█████▎    | 5191/9864 [5:12:37<19:44:54, 15.21s/it]
 53%|█████▎    | 5192/9864 [5:12:51<19:20:24, 14.90s/it]
 53%|█████▎    | 5193/9864 [5:13:08<20:00:46, 15.42s/it]
 53%|█████▎    | 5194/9864 [5:13:24<20:13:06, 15.59s/it]
 53%|█████▎    | 5195/9864 [5:13:40<20:30:07, 15.81s/it]
 53%|█████▎    | 5196/9864 [5:13:54<19:58:53, 15.41s/it]
 53%|█████▎    | 5197/9864 [5:14:10<20:13:41, 15.60s/it]
 53%|█████▎    | 5198/9864 [5:14:25<19:54:11, 15.36s/it]
 53%|█████▎    | 5199/9864 [5:14:39<19:14:28, 14.85s/it]
 53%|█████▎    | 5200/9864 [5:14:53<19:03:53, 14.72s/it]
                                                        
{'loss': '0.1483', 'grad_norm': '2.047', 'learning_rate': '9.609e-06', 'epoch': '0.5272'}

 53%|█████▎    | 5200/9864 [5:14:53<19:03:53, 14.72s/it]
 53%|█████▎    | 5201/9864 [5:15:08<18:56:47, 14.63s/it]
 53%|█████▎    | 5202/9864 [5:15:25<19:59:31, 15.44s/it]
 53%|█████▎    | 5203/9864 [5:15:39<19:30:04, 15.06s/it]
 53%|█████▎    | 5204/9864 [5:15:55<19:47:43, 15.29s/it]
 53%|█████▎    | 5205/9864 [5:16:12<20:25:56, 15.79s/it]
 53%|█████▎    | 5206/9864 [5:16:26<19:48:14, 15.31s/it]
 53%|█████▎    | 5207/9864 [5:16:41<19:26:36, 15.03s/it]
 53%|█████▎    | 5208/9864 [5:16:55<19:20:10, 14.95s/it]
 53%|█████▎    | 5209/9864 [5:17:11<19:46:07, 15.29s/it]
 53%|█████▎    | 5210/9864 [5:17:27<19:59:33, 15.46s/it]
                                                        
{'loss': '0.1281', 'grad_norm': '1.631', 'learning_rate': '9.577e-06', 'epoch': '0.5282'}

 53%|█████▎    | 5210/9864 [5:17:27<19:59:33, 15.46s/it]
 53%|█████▎    | 5211/9864 [5:17:42<19:34:16, 15.14s/it]
 53%|█████▎    | 5212/9864 [5:17:57<19:41:23, 15.24s/it]
 53%|█████▎    | 5213/9864 [5:18:11<19:13:35, 14.88s/it]
 53%|█████▎    | 5214/9864 [5:18:27<19:47:46, 15.33s/it]
 53%|█████▎    | 5215/9864 [5:18:42<19:18:51, 14.96s/it]
 53%|█████▎    | 5216/9864 [5:18:56<19:08:43, 14.83s/it]
 53%|█████▎    | 5217/9864 [5:19:12<19:34:47, 15.17s/it]
 53%|█████▎    | 5218/9864 [5:19:28<19:52:03, 15.39s/it]
 53%|█████▎    | 5219/9864 [5:19:42<19:09:53, 14.85s/it]
 53%|█████▎    | 5220/9864 [5:19:58<19:36:21, 15.20s/it]
                                                        
{'loss': '0.1352', 'grad_norm': '2.34', 'learning_rate': '9.544e-06', 'epoch': '0.5292'}

 53%|█████▎    | 5220/9864 [5:19:58<19:36:21, 15.20s/it]
 53%|█████▎    | 5221/9864 [5:20:13<19:51:37, 15.40s/it]
 53%|█████▎    | 5222/9864 [5:20:28<19:25:43, 15.07s/it]
 53%|█████▎    | 5223/9864 [5:20:43<19:21:28, 15.02s/it]
 53%|█████▎    | 5224/9864 [5:20:59<19:51:13, 15.40s/it]
 53%|█████▎    | 5225/9864 [5:21:13<19:22:58, 15.04s/it]
 53%|█████▎    | 5226/9864 [5:21:29<19:50:06, 15.40s/it]
 53%|█████▎    | 5227/9864 [5:21:44<19:33:22, 15.18s/it]
 53%|█████▎    | 5228/9864 [5:22:01<20:05:16, 15.60s/it]
 53%|█████▎    | 5229/9864 [5:22:16<20:07:38, 15.63s/it]
 53%|█████▎    | 5230/9864 [5:22:31<19:42:44, 15.31s/it]
                                                        
{'loss': '0.1285', 'grad_norm': '2.787', 'learning_rate': '9.511e-06', 'epoch': '0.5303'}

 53%|█████▎    | 5230/9864 [5:22:31<19:42:44, 15.31s/it]
 53%|█████▎    | 5231/9864 [5:22:44<18:59:49, 14.76s/it]
 53%|█████▎    | 5232/9864 [5:23:01<19:36:04, 15.23s/it]
 53%|█████▎    | 5233/9864 [5:23:17<19:59:48, 15.54s/it]
 53%|█████▎    | 5234/9864 [5:23:31<19:34:53, 15.23s/it]
 53%|█████▎    | 5235/9864 [5:23:46<19:10:54, 14.92s/it]
 53%|█████▎    | 5236/9864 [5:24:02<19:44:20, 15.35s/it]
 53%|█████▎    | 5237/9864 [5:24:18<20:08:23, 15.67s/it]
 53%|█████▎    | 5238/9864 [5:24:33<19:38:45, 15.29s/it]
 53%|█████▎    | 5239/9864 [5:24:48<19:27:40, 15.15s/it]
 53%|█████▎    | 5240/9864 [5:25:04<20:00:56, 15.58s/it]
                                                        
{'loss': '0.1473', 'grad_norm': '2.618', 'learning_rate': '9.478e-06', 'epoch': '0.5313'}

 53%|█████▎    | 5240/9864 [5:25:04<20:00:56, 15.58s/it]
 53%|█████▎    | 5241/9864 [5:25:20<20:04:48, 15.64s/it]
 53%|█████▎    | 5242/9864 [5:25:34<19:33:52, 15.24s/it]
 53%|█████▎    | 5243/9864 [5:25:48<19:04:57, 14.87s/it]
 53%|█████▎    | 5244/9864 [5:26:05<19:42:11, 15.35s/it]
 53%|█████▎    | 5245/9864 [5:26:20<19:46:19, 15.41s/it]
 53%|█████▎    | 5246/9864 [5:26:35<19:21:30, 15.09s/it]
 53%|█████▎    | 5247/9864 [5:26:50<19:16:48, 15.03s/it]
 53%|█████▎    | 5248/9864 [5:27:07<20:20:09, 15.86s/it]
 53%|█████▎    | 5249/9864 [5:27:22<19:48:50, 15.46s/it]
 53%|█████▎    | 5250/9864 [5:27:37<19:29:03, 15.20s/it]
                                                        
{'loss': '0.1299', 'grad_norm': '2.031', 'learning_rate': '9.445e-06', 'epoch': '0.5323'}

 53%|█████▎    | 5250/9864 [5:27:37<19:29:03, 15.20s/it]
 53%|█████▎    | 5251/9864 [5:27:53<19:49:54, 15.48s/it]
 53%|█████▎    | 5252/9864 [5:28:07<19:20:56, 15.10s/it]
 53%|█████▎    | 5253/9864 [5:28:22<19:21:59, 15.12s/it]
 53%|█████▎    | 5254/9864 [5:28:36<19:05:15, 14.91s/it]
 53%|█████▎    | 5255/9864 [5:28:52<19:30:31, 15.24s/it]
 53%|█████▎    | 5256/9864 [5:29:09<19:51:30, 15.51s/it]
 53%|█████▎    | 5257/9864 [5:29:23<19:16:19, 15.06s/it]
 53%|█████▎    | 5258/9864 [5:29:37<18:57:43, 14.82s/it]
 53%|█████▎    | 5259/9864 [5:29:51<18:33:13, 14.50s/it]
 53%|█████▎    | 5260/9864 [5:30:09<20:02:38, 15.67s/it]
                                                        
{'loss': '0.1413', 'grad_norm': '2.811', 'learning_rate': '9.413e-06', 'epoch': '0.5333'}

 53%|█████▎    | 5260/9864 [5:30:09<20:02:38, 15.67s/it]
 53%|█████▎    | 5261/9864 [5:30:23<19:20:35, 15.13s/it]
 53%|█████▎    | 5262/9864 [5:30:38<19:10:49, 15.00s/it]
 53%|█████▎    | 5263/9864 [5:30:55<20:12:49, 15.82s/it]
 53%|█████▎    | 5264/9864 [5:31:09<19:32:12, 15.29s/it]
 53%|█████▎    | 5265/9864 [5:31:23<19:03:35, 14.92s/it]
 53%|█████▎    | 5266/9864 [5:31:37<18:40:32, 14.62s/it]
 53%|█████▎    | 5267/9864 [5:31:55<19:59:15, 15.65s/it]
 53%|█████▎    | 5268/9864 [5:32:10<19:36:26, 15.36s/it]
 53%|█████▎    | 5269/9864 [5:32:24<19:07:36, 14.99s/it]
 53%|█████▎    | 5270/9864 [5:32:39<18:53:38, 14.81s/it]
                                                        
{'loss': '0.1412', 'grad_norm': '2.703', 'learning_rate': '9.38e-06', 'epoch': '0.5343'}

 53%|█████▎    | 5270/9864 [5:32:39<18:53:38, 14.81s/it]
 53%|█████▎    | 5271/9864 [5:32:55<19:26:02, 15.23s/it]
 53%|█████▎    | 5272/9864 [5:33:09<19:04:04, 14.95s/it]
 53%|█████▎    | 5273/9864 [5:33:23<18:41:34, 14.66s/it]
 53%|█████▎    | 5274/9864 [5:33:37<18:24:10, 14.43s/it]
 53%|█████▎    | 5275/9864 [5:33:53<19:08:43, 15.02s/it]
 53%|█████▎    | 5276/9864 [5:34:09<19:27:33, 15.27s/it]
 53%|█████▎    | 5277/9864 [5:34:24<19:06:03, 14.99s/it]
 54%|█████▎    | 5278/9864 [5:34:38<18:58:36, 14.90s/it]
 54%|█████▎    | 5279/9864 [5:34:55<19:31:41, 15.33s/it]
 54%|█████▎    | 5280/9864 [5:35:11<19:55:56, 15.65s/it]
                                                        
{'loss': '0.1362', 'grad_norm': '2.368', 'learning_rate': '9.347e-06', 'epoch': '0.5353'}

 54%|█████▎    | 5280/9864 [5:35:11<19:55:56, 15.65s/it]
 54%|█████▎    | 5281/9864 [5:35:26<19:29:25, 15.31s/it]
 54%|█████▎    | 5282/9864 [5:35:42<19:59:27, 15.71s/it]
 54%|█████▎    | 5283/9864 [5:35:57<19:29:13, 15.31s/it]
 54%|█████▎    | 5284/9864 [5:36:12<19:27:44, 15.30s/it]
 54%|█████▎    | 5285/9864 [5:36:26<19:07:12, 15.03s/it]
 54%|█████▎    | 5286/9864 [5:36:42<19:33:09, 15.38s/it]
 54%|█████▎    | 5287/9864 [5:36:59<19:57:32, 15.70s/it]
 54%|█████▎    | 5288/9864 [5:37:13<19:24:33, 15.27s/it]
 54%|█████▎    | 5289/9864 [5:37:27<18:52:34, 14.85s/it]
 54%|█████▎    | 5290/9864 [5:37:43<19:23:56, 15.27s/it]
                                                        
{'loss': '0.1366', 'grad_norm': '2.869', 'learning_rate': '9.314e-06', 'epoch': '0.5363'}

 54%|█████▎    | 5290/9864 [5:37:43<19:23:56, 15.27s/it]
 54%|█████▎    | 5291/9864 [5:37:58<19:07:43, 15.06s/it]
 54%|█████▎    | 5292/9864 [5:38:14<19:36:09, 15.44s/it]
 54%|█████▎    | 5293/9864 [5:38:30<19:51:45, 15.64s/it]
 54%|█████▎    | 5294/9864 [5:38:45<19:32:12, 15.39s/it]
 54%|█████▎    | 5295/9864 [5:39:01<19:54:38, 15.69s/it]
 54%|█████▎    | 5296/9864 [5:39:16<19:27:19, 15.33s/it]
 54%|█████▎    | 5297/9864 [5:39:32<19:51:40, 15.66s/it]
 54%|█████▎    | 5298/9864 [5:39:47<19:20:09, 15.25s/it]
 54%|█████▎    | 5299/9864 [5:40:01<19:06:35, 15.07s/it]
 54%|█████▎    | 5300/9864 [5:40:17<19:31:16, 15.40s/it]
                                                        
{'loss': '0.1338', 'grad_norm': '2.051', 'learning_rate': '9.282e-06', 'epoch': '0.5374'}

 54%|█████▎    | 5300/9864 [5:40:17<19:31:16, 15.40s/it]
 54%|█████▎    | 5301/9864 [5:40:34<20:02:11, 15.81s/it]
 54%|█████▍    | 5302/9864 [5:40:50<19:59:30, 15.78s/it]
 54%|█████▍    | 5303/9864 [5:41:04<19:28:46, 15.38s/it]
 54%|█████▍    | 5304/9864 [5:41:19<19:05:59, 15.08s/it]
 54%|█████▍    | 5305/9864 [5:41:35<19:23:15, 15.31s/it]
 54%|█████▍    | 5306/9864 [5:41:49<18:57:31, 14.97s/it]
 54%|█████▍    | 5307/9864 [5:42:05<19:33:00, 15.44s/it]
 54%|█████▍    | 5308/9864 [5:42:19<19:01:07, 15.03s/it]
 54%|█████▍    | 5309/9864 [5:42:36<19:29:50, 15.41s/it]
 54%|█████▍    | 5310/9864 [5:42:52<19:53:12, 15.72s/it]
                                                        
{'loss': '0.1307', 'grad_norm': '2.891', 'learning_rate': '9.249e-06', 'epoch': '0.5384'}

 54%|█████▍    | 5310/9864 [5:42:52<19:53:12, 15.72s/it]
 54%|█████▍    | 5311/9864 [5:43:06<19:14:09, 15.21s/it]
 54%|█████▍    | 5312/9864 [5:43:21<19:05:15, 15.10s/it]
 54%|█████▍    | 5313/9864 [5:43:37<19:36:37, 15.51s/it]
 54%|█████▍    | 5314/9864 [5:43:52<19:10:04, 15.17s/it]
 54%|█████▍    | 5315/9864 [5:44:08<19:31:00, 15.45s/it]
 54%|█████▍    | 5316/9864 [5:44:22<19:01:57, 15.07s/it]
 54%|█████▍    | 5317/9864 [5:44:40<20:10:45, 15.98s/it]
 54%|█████▍    | 5318/9864 [5:44:55<19:34:14, 15.50s/it]
 54%|█████▍    | 5319/9864 [5:45:09<19:19:12, 15.30s/it]
 54%|█████▍    | 5320/9864 [5:45:26<19:42:31, 15.61s/it]
                                                        
{'loss': '0.138', 'grad_norm': '5.661', 'learning_rate': '9.216e-06', 'epoch': '0.5394'}

 54%|█████▍    | 5320/9864 [5:45:26<19:42:31, 15.61s/it]
 54%|█████▍    | 5321/9864 [5:45:40<19:21:58, 15.35s/it]
 54%|█████▍    | 5322/9864 [5:45:56<19:30:25, 15.46s/it]
 54%|█████▍    | 5323/9864 [5:46:11<19:06:28, 15.15s/it]
 54%|█████▍    | 5324/9864 [5:46:25<18:46:14, 14.88s/it]
 54%|█████▍    | 5325/9864 [5:46:41<19:22:23, 15.37s/it]
 54%|█████▍    | 5326/9864 [5:46:58<19:55:45, 15.81s/it]
 54%|█████▍    | 5327/9864 [5:47:12<19:14:10, 15.26s/it]
 54%|█████▍    | 5328/9864 [5:47:29<19:41:07, 15.62s/it]
 54%|█████▍    | 5329/9864 [5:47:43<19:09:11, 15.20s/it]
 54%|█████▍    | 5330/9864 [5:47:59<19:21:54, 15.38s/it]
                                                        
{'loss': '0.1336', 'grad_norm': '2.423', 'learning_rate': '9.183e-06', 'epoch': '0.5404'}

 54%|█████▍    | 5330/9864 [5:47:59<19:21:54, 15.38s/it]
 54%|█████▍    | 5331/9864 [5:48:13<19:05:36, 15.16s/it]
 54%|█████▍    | 5332/9864 [5:48:28<18:54:34, 15.02s/it]
 54%|█████▍    | 5333/9864 [5:48:45<19:47:44, 15.73s/it]
 54%|█████▍    | 5334/9864 [5:49:00<19:16:24, 15.32s/it]
 54%|█████▍    | 5335/9864 [5:49:14<18:54:28, 15.03s/it]
 54%|█████▍    | 5336/9864 [5:49:31<19:40:06, 15.64s/it]
 54%|█████▍    | 5337/9864 [5:49:45<19:09:05, 15.23s/it]
 54%|█████▍    | 5338/9864 [5:50:02<19:33:40, 15.56s/it]
 54%|█████▍    | 5339/9864 [5:50:16<19:12:27, 15.28s/it]
 54%|█████▍    | 5340/9864 [5:50:33<19:30:42, 15.53s/it]
                                                        
{'loss': '0.1423', 'grad_norm': '2.45', 'learning_rate': '9.151e-06', 'epoch': '0.5414'}

 54%|█████▍    | 5340/9864 [5:50:33<19:30:42, 15.53s/it]
 54%|█████▍    | 5341/9864 [5:50:47<18:58:42, 15.11s/it]
 54%|█████▍    | 5342/9864 [5:51:03<19:17:12, 15.35s/it]
 54%|█████▍    | 5343/9864 [5:51:19<19:37:24, 15.63s/it]
 54%|█████▍    | 5344/9864 [5:51:33<19:05:47, 15.21s/it]
 54%|█████▍    | 5345/9864 [5:51:48<18:57:32, 15.10s/it]
 54%|█████▍    | 5346/9864 [5:52:04<19:16:58, 15.36s/it]
 54%|█████▍    | 5347/9864 [5:52:20<19:39:52, 15.67s/it]
 54%|█████▍    | 5348/9864 [5:52:35<19:10:25, 15.28s/it]
 54%|█████▍    | 5349/9864 [5:52:49<18:55:03, 15.08s/it]
 54%|█████▍    | 5350/9864 [5:53:05<19:18:13, 15.40s/it]
                                                        
{'loss': '0.1281', 'grad_norm': '3.747', 'learning_rate': '9.118e-06', 'epoch': '0.5424'}

 54%|█████▍    | 5350/9864 [5:53:05<19:18:13, 15.40s/it]
 54%|█████▍    | 5351/9864 [5:53:22<19:42:09, 15.72s/it]
 54%|█████▍    | 5352/9864 [5:53:37<19:20:36, 15.43s/it]
 54%|█████▍    | 5353/9864 [5:53:53<19:33:29, 15.61s/it]
 54%|█████▍    | 5354/9864 [5:54:07<19:01:50, 15.19s/it]
 54%|█████▍    | 5355/9864 [5:54:21<18:45:04, 14.97s/it]
 54%|█████▍    | 5356/9864 [5:54:37<18:58:32, 15.15s/it]
 54%|█████▍    | 5357/9864 [5:54:52<18:46:46, 15.00s/it]
 54%|█████▍    | 5358/9864 [5:55:08<19:22:12, 15.48s/it]
 54%|█████▍    | 5359/9864 [5:55:25<19:43:55, 15.77s/it]
 54%|█████▍    | 5360/9864 [5:55:41<19:59:51, 15.98s/it]
                                                        
{'loss': '0.122', 'grad_norm': '2.97', 'learning_rate': '9.085e-06', 'epoch': '0.5434'}

 54%|█████▍    | 5360/9864 [5:55:41<19:59:51, 15.98s/it]
 54%|█████▍    | 5361/9864 [5:55:56<19:30:59, 15.60s/it]
 54%|█████▍    | 5362/9864 [5:56:10<19:00:54, 15.21s/it]
 54%|█████▍    | 5363/9864 [5:56:26<19:15:48, 15.41s/it]
 54%|█████▍    | 5364/9864 [5:56:40<18:50:33, 15.07s/it]
 54%|█████▍    | 5365/9864 [5:56:57<19:20:28, 15.48s/it]
 54%|█████▍    | 5366/9864 [5:57:11<18:44:16, 15.00s/it]
 54%|█████▍    | 5367/9864 [5:57:25<18:33:08, 14.85s/it]
 54%|█████▍    | 5368/9864 [5:57:43<19:40:46, 15.76s/it]
 54%|█████▍    | 5369/9864 [5:57:57<19:02:16, 15.25s/it]
 54%|█████▍    | 5370/9864 [5:58:11<18:34:11, 14.88s/it]
                                                        
{'loss': '0.1279', 'grad_norm': '3.943', 'learning_rate': '9.053e-06', 'epoch': '0.5444'}

 54%|█████▍    | 5370/9864 [5:58:11<18:34:11, 14.88s/it]
 54%|█████▍    | 5371/9864 [5:58:27<19:08:02, 15.33s/it]
 54%|█████▍    | 5372/9864 [5:58:43<19:06:14, 15.31s/it]
 54%|█████▍    | 5373/9864 [5:58:59<19:22:17, 15.53s/it]
 54%|█████▍    | 5374/9864 [5:59:13<18:43:52, 15.02s/it]
 54%|█████▍    | 5375/9864 [5:59:26<18:19:07, 14.69s/it]
 55%|█████▍    | 5376/9864 [5:59:42<18:45:12, 15.04s/it]
 55%|█████▍    | 5377/9864 [5:59:57<18:27:17, 14.81s/it]
 55%|█████▍    | 5378/9864 [6:00:12<18:47:24, 15.08s/it]
 55%|█████▍    | 5379/9864 [6:00:27<18:40:19, 14.99s/it]
 55%|█████▍    | 5380/9864 [6:00:43<19:03:55, 15.31s/it]
                                                        
{'loss': '0.1322', 'grad_norm': '2.007', 'learning_rate': '9.02e-06', 'epoch': '0.5455'}

 55%|█████▍    | 5380/9864 [6:00:43<19:03:55, 15.31s/it]
 55%|█████▍    | 5381/9864 [6:00:58<18:43:13, 15.03s/it]
 55%|█████▍    | 5382/9864 [6:01:12<18:25:38, 14.80s/it]
 55%|█████▍    | 5383/9864 [6:01:28<19:00:25, 15.27s/it]
 55%|█████▍    | 5384/9864 [6:01:45<19:25:09, 15.60s/it]
 55%|█████▍    | 5385/9864 [6:01:59<18:58:45, 15.25s/it]
 55%|█████▍    | 5386/9864 [6:02:13<18:37:03, 14.97s/it]
 55%|█████▍    | 5387/9864 [6:02:30<19:13:41, 15.46s/it]
 55%|█████▍    | 5388/9864 [6:02:46<19:17:50, 15.52s/it]
 55%|█████▍    | 5389/9864 [6:03:00<18:54:54, 15.22s/it]
 55%|█████▍    | 5390/9864 [6:03:15<18:43:31, 15.07s/it]
                                                        
{'loss': '0.1303', 'grad_norm': '3.876', 'learning_rate': '8.987e-06', 'epoch': '0.5465'}

 55%|█████▍    | 5390/9864 [6:03:15<18:43:31, 15.07s/it]
 55%|█████▍    | 5391/9864 [6:03:33<19:43:58, 15.88s/it]
 55%|█████▍    | 5392/9864 [6:03:46<18:59:13, 15.28s/it]
 55%|█████▍    | 5393/9864 [6:04:01<18:37:46, 15.00s/it]
 55%|█████▍    | 5394/9864 [6:04:15<18:25:44, 14.84s/it]
 55%|█████▍    | 5395/9864 [6:04:31<18:54:55, 15.24s/it]
 55%|█████▍    | 5396/9864 [6:04:47<19:10:38, 15.45s/it]
 55%|█████▍    | 5397/9864 [6:05:02<18:43:53, 15.10s/it]
 55%|█████▍    | 5398/9864 [6:05:17<18:58:13, 15.29s/it]
 55%|█████▍    | 5399/9864 [6:05:34<19:17:04, 15.55s/it]
 55%|█████▍    | 5400/9864 [6:05:48<18:51:12, 15.20s/it]
                                                        
{'loss': '0.1222', 'grad_norm': '3.676', 'learning_rate': '8.954e-06', 'epoch': '0.5475'}

 55%|█████▍    | 5400/9864 [6:05:48<18:51:12, 15.20s/it]
 55%|█████▍    | 5401/9864 [6:06:03<18:40:24, 15.06s/it]
 55%|█████▍    | 5402/9864 [6:06:17<18:23:51, 14.84s/it]
 55%|█████▍    | 5403/9864 [6:06:33<18:49:46, 15.20s/it]
 55%|█████▍    | 5404/9864 [6:06:48<18:35:30, 15.01s/it]
 55%|█████▍    | 5405/9864 [6:07:05<19:24:47, 15.67s/it]
 55%|█████▍    | 5406/9864 [6:07:19<18:54:59, 15.28s/it]
 55%|█████▍    | 5407/9864 [6:07:35<18:58:17, 15.32s/it]
 55%|█████▍    | 5408/9864 [6:07:49<18:28:38, 14.93s/it]
 55%|█████▍    | 5409/9864 [6:08:03<18:12:08, 14.71s/it]
 55%|█████▍    | 5410/9864 [6:08:19<18:42:33, 15.12s/it]
                                                        
{'loss': '0.1271', 'grad_norm': '3.554', 'learning_rate': '8.922e-06', 'epoch': '0.5485'}

 55%|█████▍    | 5410/9864 [6:08:19<18:42:33, 15.12s/it]
 55%|█████▍    | 5411/9864 [6:08:33<18:12:02, 14.71s/it]
 55%|█████▍    | 5412/9864 [6:08:48<18:36:21, 15.05s/it]
 55%|█████▍    | 5413/9864 [6:09:05<19:05:48, 15.45s/it]
 55%|█████▍    | 5414/9864 [6:09:21<19:17:45, 15.61s/it]
 55%|█████▍    | 5415/9864 [6:09:35<18:46:45, 15.20s/it]
 55%|█████▍    | 5416/9864 [6:09:49<18:29:15, 14.96s/it]
 55%|█████▍    | 5417/9864 [6:10:04<18:24:31, 14.90s/it]
 55%|█████▍    | 5418/9864 [6:10:20<18:42:38, 15.15s/it]
 55%|█████▍    | 5419/9864 [6:10:35<18:29:11, 14.97s/it]
 55%|█████▍    | 5420/9864 [6:10:53<19:43:56, 15.98s/it]
                                                        
{'loss': '0.1348', 'grad_norm': '2.37', 'learning_rate': '8.889e-06', 'epoch': '0.5495'}

 55%|█████▍    | 5420/9864 [6:10:53<19:43:56, 15.98s/it]
 55%|█████▍    | 5421/9864 [6:11:07<19:10:23, 15.54s/it]
 55%|█████▍    | 5422/9864 [6:11:24<19:30:57, 15.82s/it]
 55%|█████▍    | 5423/9864 [6:11:39<19:09:24, 15.53s/it]
 55%|█████▍    | 5424/9864 [6:11:53<18:48:25, 15.25s/it]
 55%|█████▍    | 5425/9864 [6:12:08<18:32:49, 15.04s/it]
 55%|█████▌    | 5426/9864 [6:12:24<18:58:36, 15.39s/it]
 55%|█████▌    | 5427/9864 [6:12:38<18:30:54, 15.02s/it]
 55%|█████▌    | 5428/9864 [6:12:56<19:41:54, 15.99s/it]
 55%|█████▌    | 5429/9864 [6:13:11<19:16:44, 15.65s/it]
 55%|█████▌    | 5430/9864 [6:13:26<18:58:43, 15.41s/it]
                                                        
{'loss': '0.1284', 'grad_norm': '4.768', 'learning_rate': '8.857e-06', 'epoch': '0.5505'}

 55%|█████▌    | 5430/9864 [6:13:26<18:58:43, 15.41s/it]
 55%|█████▌    | 5431/9864 [6:13:42<19:13:40, 15.61s/it]
 55%|█████▌    | 5432/9864 [6:13:56<18:37:12, 15.12s/it]
 55%|█████▌    | 5433/9864 [6:14:12<18:49:09, 15.29s/it]
 55%|█████▌    | 5434/9864 [6:14:26<18:17:09, 14.86s/it]
 55%|█████▌    | 5435/9864 [6:14:41<18:15:59, 14.85s/it]
 55%|█████▌    | 5436/9864 [6:14:59<19:40:40, 16.00s/it]
 55%|█████▌    | 5437/9864 [6:15:13<18:51:23, 15.33s/it]
 55%|█████▌    | 5438/9864 [6:15:29<18:56:15, 15.40s/it]
 55%|█████▌    | 5439/9864 [6:15:43<18:23:47, 14.97s/it]
 55%|█████▌    | 5440/9864 [6:15:57<18:13:34, 14.83s/it]
                                                        
{'loss': '0.1475', 'grad_norm': '2.672', 'learning_rate': '8.824e-06', 'epoch': '0.5515'}

 55%|█████▌    | 5440/9864 [6:15:57<18:13:34, 14.83s/it]
 55%|█████▌    | 5441/9864 [6:16:13<18:46:48, 15.29s/it]
 55%|█████▌    | 5442/9864 [6:16:28<18:29:28, 15.05s/it]
 55%|█████▌    | 5443/9864 [6:16:42<18:10:06, 14.79s/it]
 55%|█████▌    | 5444/9864 [6:17:00<19:13:46, 15.66s/it]
 55%|█████▌    | 5445/9864 [6:17:14<18:48:08, 15.32s/it]
 55%|█████▌    | 5446/9864 [6:17:28<18:12:14, 14.83s/it]
 55%|█████▌    | 5447/9864 [6:17:44<18:38:07, 15.19s/it]
 55%|█████▌    | 5448/9864 [6:18:00<18:56:15, 15.44s/it]
 55%|█████▌    | 5449/9864 [6:18:14<18:26:40, 15.04s/it]
 55%|█████▌    | 5450/9864 [6:18:29<18:16:47, 14.91s/it]
                                                        
{'loss': '0.126', 'grad_norm': '2.962', 'learning_rate': '8.791e-06', 'epoch': '0.5526'}

 55%|█████▌    | 5450/9864 [6:18:29<18:16:47, 14.91s/it]
 55%|█████▌    | 5451/9864 [6:18:45<18:37:10, 15.19s/it]
 55%|█████▌    | 5452/9864 [6:19:00<18:40:52, 15.24s/it]
 55%|█████▌    | 5453/9864 [6:19:14<18:17:01, 14.92s/it]
 55%|█████▌    | 5454/9864 [6:19:29<18:06:59, 14.79s/it]
 55%|█████▌    | 5455/9864 [6:19:44<18:27:36, 15.07s/it]
 55%|█████▌    | 5456/9864 [6:20:00<18:43:53, 15.30s/it]
 55%|█████▌    | 5457/9864 [6:20:15<18:24:57, 15.04s/it]
 55%|█████▌    | 5458/9864 [6:20:31<18:52:30, 15.42s/it]
 55%|█████▌    | 5459/9864 [6:20:47<19:14:01, 15.72s/it]
 55%|█████▌    | 5460/9864 [6:21:02<18:48:41, 15.38s/it]
                                                        
{'loss': '0.1318', 'grad_norm': '3.087', 'learning_rate': '8.759e-06', 'epoch': '0.5536'}

 55%|█████▌    | 5460/9864 [6:21:02<18:48:41, 15.38s/it]
 55%|█████▌    | 5461/9864 [6:21:17<18:31:49, 15.15s/it]
 55%|█████▌    | 5462/9864 [6:21:32<18:28:50, 15.11s/it]
 55%|█████▌    | 5463/9864 [6:21:50<19:43:12, 16.13s/it]
 55%|█████▌    | 5464/9864 [6:22:05<19:08:11, 15.66s/it]
 55%|█████▌    | 5465/9864 [6:22:19<18:34:58, 15.21s/it]
 55%|█████▌    | 5466/9864 [6:22:35<18:48:07, 15.39s/it]
 55%|█████▌    | 5467/9864 [6:22:51<19:14:18, 15.75s/it]
 55%|█████▌    | 5468/9864 [6:23:06<18:43:49, 15.34s/it]
 55%|█████▌    | 5469/9864 [6:23:20<18:19:36, 15.01s/it]
 55%|█████▌    | 5470/9864 [6:23:36<18:46:38, 15.38s/it]
                                                        
{'loss': '0.1421', 'grad_norm': '3.846', 'learning_rate': '8.726e-06', 'epoch': '0.5546'}

 55%|█████▌    | 5470/9864 [6:23:36<18:46:38, 15.38s/it]
 55%|█████▌    | 5471/9864 [6:23:51<18:24:49, 15.09s/it]
 55%|█████▌    | 5472/9864 [6:24:06<18:39:04, 15.29s/it]
 55%|█████▌    | 5473/9864 [6:24:21<18:17:25, 15.00s/it]
 55%|█████▌    | 5474/9864 [6:24:37<18:45:05, 15.38s/it]
 56%|█████▌    | 5475/9864 [6:24:53<18:51:18, 15.47s/it]
 56%|█████▌    | 5476/9864 [6:25:07<18:28:50, 15.16s/it]
 56%|█████▌    | 5477/9864 [6:25:21<18:05:14, 14.84s/it]
 56%|█████▌    | 5478/9864 [6:25:37<18:22:40, 15.08s/it]
 56%|█████▌    | 5479/9864 [6:25:51<17:56:26, 14.73s/it]
 56%|█████▌    | 5480/9864 [6:26:06<18:17:25, 15.02s/it]
                                                        
{'loss': '0.1294', 'grad_norm': '2.242', 'learning_rate': '8.694e-06', 'epoch': '0.5556'}

 56%|█████▌    | 5480/9864 [6:26:06<18:17:25, 15.02s/it]
 56%|█████▌    | 5481/9864 [6:26:21<18:00:25, 14.79s/it]
 56%|█████▌    | 5482/9864 [6:26:38<19:07:34, 15.71s/it]
 56%|█████▌    | 5483/9864 [6:26:53<18:40:26, 15.35s/it]
 56%|█████▌    | 5484/9864 [6:27:07<18:20:51, 15.08s/it]
 56%|█████▌    | 5485/9864 [6:27:21<17:53:34, 14.71s/it]
 56%|█████▌    | 5486/9864 [6:27:38<18:29:30, 15.21s/it]
 56%|█████▌    | 5487/9864 [6:27:53<18:40:57, 15.37s/it]
 56%|█████▌    | 5488/9864 [6:28:08<18:16:38, 15.04s/it]
 56%|█████▌    | 5489/9864 [6:28:24<18:47:35, 15.46s/it]
 56%|█████▌    | 5490/9864 [6:28:40<18:59:50, 15.64s/it]
                                                        
{'loss': '0.1343', 'grad_norm': '2.732', 'learning_rate': '8.661e-06', 'epoch': '0.5566'}

 56%|█████▌    | 5490/9864 [6:28:40<18:59:50, 15.64s/it]
 56%|█████▌    | 5491/9864 [6:28:55<18:34:58, 15.30s/it]
 56%|█████▌    | 5492/9864 [6:29:09<18:12:04, 14.99s/it]
 56%|█████▌    | 5493/9864 [6:29:24<18:08:50, 14.95s/it]
 56%|█████▌    | 5494/9864 [6:29:40<18:39:01, 15.36s/it]
 56%|█████▌    | 5495/9864 [6:29:56<19:01:21, 15.67s/it]
 56%|█████▌    | 5496/9864 [6:30:11<18:33:14, 15.29s/it]
 56%|█████▌    | 5497/9864 [6:30:28<19:21:04, 15.95s/it]
 56%|█████▌    | 5498/9864 [6:30:43<18:51:49, 15.55s/it]
 56%|█████▌    | 5499/9864 [6:30:58<18:32:55, 15.30s/it]
 56%|█████▌    | 5500/9864 [6:31:12<18:05:25, 14.92s/it]
                                                        
{'loss': '0.1387', 'grad_norm': '3.674', 'learning_rate': '8.629e-06', 'epoch': '0.5576'}

 56%|█████▌    | 5500/9864 [6:31:12<18:05:25, 14.92s/it]
 56%|█████▌    | 5501/9864 [6:31:26<17:51:01, 14.73s/it]
 56%|█████▌    | 5502/9864 [6:31:42<18:11:08, 15.01s/it]
 56%|█████▌    | 5503/9864 [6:31:58<18:33:49, 15.32s/it]
 56%|█████▌    | 5504/9864 [6:32:14<18:46:05, 15.50s/it]
 56%|█████▌    | 5505/9864 [6:32:30<19:00:30, 15.70s/it]
 56%|█████▌    | 5506/9864 [6:32:44<18:26:24, 15.23s/it]
 56%|█████▌    | 5507/9864 [6:32:58<18:00:21, 14.88s/it]
 56%|█████▌    | 5508/9864 [6:33:12<17:51:16, 14.76s/it]
 56%|█████▌    | 5509/9864 [6:33:27<17:39:55, 14.60s/it]
 56%|█████▌    | 5510/9864 [6:33:43<18:16:01, 15.10s/it]
                                                        
{'loss': '0.1417', 'grad_norm': '3.138', 'learning_rate': '8.596e-06', 'epoch': '0.5586'}

 56%|█████▌    | 5510/9864 [6:33:43<18:16:01, 15.10s/it]
 56%|█████▌    | 5511/9864 [6:34:01<19:27:43, 16.10s/it]
 56%|█████▌    | 5512/9864 [6:34:16<18:53:06, 15.62s/it]
 56%|█████▌    | 5513/9864 [6:34:31<18:34:20, 15.37s/it]
 56%|█████▌    | 5514/9864 [6:34:47<18:48:49, 15.57s/it]
 56%|█████▌    | 5515/9864 [6:35:01<18:17:01, 15.13s/it]
 56%|█████▌    | 5516/9864 [6:35:15<17:51:31, 14.79s/it]
 56%|█████▌    | 5517/9864 [6:35:29<17:33:50, 14.55s/it]
 56%|█████▌    | 5518/9864 [6:35:47<18:46:13, 15.55s/it]
 56%|█████▌    | 5519/9864 [6:36:03<19:00:15, 15.75s/it]
 56%|█████▌    | 5520/9864 [6:36:17<18:34:08, 15.39s/it]
                                                        
{'loss': '0.1318', 'grad_norm': '3.446', 'learning_rate': '8.564e-06', 'epoch': '0.5597'}

 56%|█████▌    | 5520/9864 [6:36:17<18:34:08, 15.39s/it]
 56%|█████▌    | 5521/9864 [6:36:32<18:12:38, 15.10s/it]
 56%|█████▌    | 5522/9864 [6:36:48<18:28:18, 15.32s/it]
 56%|█████▌    | 5523/9864 [6:37:02<18:12:44, 15.10s/it]
 56%|█████▌    | 5524/9864 [6:37:17<17:58:59, 14.92s/it]
 56%|█████▌    | 5525/9864 [6:37:33<18:24:43, 15.28s/it]
 56%|█████▌    | 5526/9864 [6:37:49<18:35:26, 15.43s/it]
 56%|█████▌    | 5527/9864 [6:38:05<18:53:18, 15.68s/it]
 56%|█████▌    | 5528/9864 [6:38:19<18:08:09, 15.06s/it]
 56%|█████▌    | 5529/9864 [6:38:33<17:48:18, 14.79s/it]
 56%|█████▌    | 5530/9864 [6:38:49<18:29:59, 15.37s/it]
                                                        
{'loss': '0.1418', 'grad_norm': '2.801', 'learning_rate': '8.531e-06', 'epoch': '0.5607'}

 56%|█████▌    | 5530/9864 [6:38:49<18:29:59, 15.37s/it]
 56%|█████▌    | 5531/9864 [6:39:04<18:16:50, 15.19s/it]
 56%|█████▌    | 5532/9864 [6:39:19<17:58:28, 14.94s/it]
 56%|█████▌    | 5533/9864 [6:39:33<17:40:24, 14.69s/it]
 56%|█████▌    | 5534/9864 [6:39:52<19:28:32, 16.19s/it]
 56%|█████▌    | 5535/9864 [6:40:07<18:59:27, 15.79s/it]
 56%|█████▌    | 5536/9864 [6:40:22<18:26:51, 15.34s/it]
 56%|█████▌    | 5537/9864 [6:40:35<17:49:13, 14.83s/it]
 56%|█████▌    | 5538/9864 [6:40:51<18:11:33, 15.14s/it]
 56%|█████▌    | 5539/9864 [6:41:06<17:59:24, 14.97s/it]
 56%|█████▌    | 5540/9864 [6:41:20<17:48:45, 14.83s/it]
                                                        
{'loss': '0.1119', 'grad_norm': '2.117', 'learning_rate': '8.499e-06', 'epoch': '0.5617'}

 56%|█████▌    | 5540/9864 [6:41:20<17:48:45, 14.83s/it]
 56%|█████▌    | 5541/9864 [6:41:35<17:42:46, 14.75s/it]
 56%|█████▌    | 5542/9864 [6:41:53<19:07:27, 15.93s/it]
 56%|█████▌    | 5543/9864 [6:42:09<19:10:19, 15.97s/it]
 56%|█████▌    | 5544/9864 [6:42:24<18:33:50, 15.47s/it]
 56%|█████▌    | 5545/9864 [6:42:40<18:48:31, 15.68s/it]
 56%|█████▌    | 5546/9864 [6:42:54<18:19:10, 15.27s/it]
 56%|█████▌    | 5547/9864 [6:43:09<18:03:17, 15.06s/it]
 56%|█████▌    | 5548/9864 [6:43:23<17:45:28, 14.81s/it]
 56%|█████▋    | 5549/9864 [6:43:37<17:31:14, 14.62s/it]
 56%|█████▋    | 5550/9864 [6:43:57<19:21:48, 16.16s/it]
                                                        
{'loss': '0.1327', 'grad_norm': '2.879', 'learning_rate': '8.466e-06', 'epoch': '0.5627'}

 56%|█████▋    | 5550/9864 [6:43:57<19:21:48, 16.16s/it]
 56%|█████▋    | 5551/9864 [6:44:11<18:36:37, 15.53s/it]
 56%|█████▋    | 5552/9864 [6:44:25<18:09:46, 15.16s/it]
 56%|█████▋    | 5553/9864 [6:44:42<18:33:49, 15.50s/it]
 56%|█████▋    | 5554/9864 [6:44:56<18:13:36, 15.22s/it]
 56%|█████▋    | 5555/9864 [6:45:10<17:50:58, 14.91s/it]
 56%|█████▋    | 5556/9864 [6:45:25<17:41:51, 14.79s/it]
 56%|█████▋    | 5557/9864 [6:45:41<18:07:43, 15.15s/it]
 56%|█████▋    | 5558/9864 [6:45:57<18:36:44, 15.56s/it]
 56%|█████▋    | 5559/9864 [6:46:13<18:41:03, 15.62s/it]
 56%|█████▋    | 5560/9864 [6:46:27<18:13:45, 15.25s/it]
                                                        
{'loss': '0.1113', 'grad_norm': '4.202', 'learning_rate': '8.434e-06', 'epoch': '0.5637'}

 56%|█████▋    | 5560/9864 [6:46:27<18:13:45, 15.25s/it]
 56%|█████▋    | 5561/9864 [6:46:43<18:25:51, 15.42s/it]
 56%|█████▋    | 5562/9864 [6:46:58<18:15:51, 15.28s/it]
 56%|█████▋    | 5563/9864 [6:47:13<18:03:55, 15.12s/it]
 56%|█████▋    | 5564/9864 [6:47:27<17:38:06, 14.76s/it]
 56%|█████▋    | 5565/9864 [6:47:41<17:25:27, 14.59s/it]
 56%|█████▋    | 5566/9864 [6:47:57<17:57:46, 15.05s/it]
 56%|█████▋    | 5567/9864 [6:48:14<18:27:30, 15.46s/it]
 56%|█████▋    | 5568/9864 [6:48:32<19:29:03, 16.33s/it]
 56%|█████▋    | 5569/9864 [6:48:46<18:40:48, 15.66s/it]
 56%|█████▋    | 5570/9864 [6:49:00<18:09:57, 15.23s/it]
                                                        
{'loss': '0.1261', 'grad_norm': '3.166', 'learning_rate': '8.401e-06', 'epoch': '0.5647'}

 56%|█████▋    | 5570/9864 [6:49:00<18:09:57, 15.23s/it]
 56%|█████▋    | 5571/9864 [6:49:15<17:53:16, 15.00s/it]
 56%|█████▋    | 5572/9864 [6:49:29<17:42:36, 14.85s/it]
 56%|█████▋    | 5573/9864 [6:49:45<17:56:38, 15.05s/it]
 57%|█████▋    | 5574/9864 [6:49:59<17:44:02, 14.88s/it]
 57%|█████▋    | 5575/9864 [6:50:15<18:07:45, 15.22s/it]
 57%|█████▋    | 5576/9864 [6:50:34<19:11:57, 16.12s/it]
 57%|█████▋    | 5577/9864 [6:50:48<18:28:13, 15.51s/it]
 57%|█████▋    | 5578/9864 [6:51:02<18:07:50, 15.23s/it]
 57%|█████▋    | 5579/9864 [6:51:15<17:20:37, 14.57s/it]
 57%|█████▋    | 5580/9864 [6:51:32<18:07:48, 15.24s/it]
                                                        
{'loss': '0.1171', 'grad_norm': '3.067', 'learning_rate': '8.369e-06', 'epoch': '0.5657'}

 57%|█████▋    | 5580/9864 [6:51:32<18:07:48, 15.24s/it]
 57%|█████▋    | 5581/9864 [6:51:47<17:56:32, 15.08s/it]
 57%|█████▋    | 5582/9864 [6:52:01<17:39:53, 14.85s/it]
 57%|█████▋    | 5583/9864 [6:52:19<18:52:26, 15.87s/it]
 57%|█████▋    | 5584/9864 [6:52:36<19:08:15, 16.10s/it]
 57%|█████▋    | 5585/9864 [6:52:50<18:21:24, 15.44s/it]
 57%|█████▋    | 5586/9864 [6:53:04<17:59:01, 15.13s/it]
 57%|█████▋    | 5587/9864 [6:53:19<17:41:54, 14.90s/it]
 57%|█████▋    | 5588/9864 [6:53:35<18:13:09, 15.34s/it]
 57%|█████▋    | 5589/9864 [6:53:49<17:45:36, 14.96s/it]
 57%|█████▋    | 5590/9864 [6:54:04<17:37:04, 14.84s/it]
                                                        
{'loss': '0.1238', 'grad_norm': '2.934', 'learning_rate': '8.336e-06', 'epoch': '0.5668'}

 57%|█████▋    | 5590/9864 [6:54:04<17:37:04, 14.84s/it]
 57%|█████▋    | 5591/9864 [6:54:20<18:01:20, 15.18s/it]
 57%|█████▋    | 5592/9864 [6:54:34<17:44:53, 14.96s/it]
 57%|█████▋    | 5593/9864 [6:54:50<18:02:51, 15.21s/it]
 57%|█████▋    | 5594/9864 [6:55:04<17:41:29, 14.92s/it]
 57%|█████▋    | 5595/9864 [6:55:20<18:03:04, 15.22s/it]
 57%|█████▋    | 5596/9864 [6:55:35<17:49:30, 15.04s/it]
 57%|█████▋    | 5597/9864 [6:55:49<17:32:35, 14.80s/it]
 57%|█████▋    | 5598/9864 [6:56:04<17:32:39, 14.81s/it]
 57%|█████▋    | 5599/9864 [6:56:21<18:19:51, 15.47s/it]
 57%|█████▋    | 5600/9864 [6:56:37<18:32:37, 15.66s/it]
                                                        
{'loss': '0.1332', 'grad_norm': '2.443', 'learning_rate': '8.304e-06', 'epoch': '0.5678'}

 57%|█████▋    | 5600/9864 [6:56:37<18:32:37, 15.66s/it]
 57%|█████▋    | 5601/9864 [6:56:52<18:18:36, 15.46s/it]
 57%|█████▋    | 5602/9864 [6:57:06<17:49:16, 15.05s/it]
 57%|█████▋    | 5603/9864 [6:57:22<18:14:27, 15.41s/it]
 57%|█████▋    | 5604/9864 [6:57:37<17:54:07, 15.13s/it]
 57%|█████▋    | 5605/9864 [6:57:51<17:40:55, 14.95s/it]
 57%|█████▋    | 5606/9864 [6:58:06<17:37:32, 14.90s/it]
 57%|█████▋    | 5607/9864 [6:58:24<18:48:16, 15.90s/it]
 57%|█████▋    | 5608/9864 [6:58:40<18:48:51, 15.91s/it]
 57%|█████▋    | 5609/9864 [6:58:54<18:09:59, 15.37s/it]
 57%|█████▋    | 5610/9864 [6:59:08<17:36:48, 14.91s/it]
                                                        
{'loss': '0.1303', 'grad_norm': '2.817', 'learning_rate': '8.272e-06', 'epoch': '0.5688'}

 57%|█████▋    | 5610/9864 [6:59:08<17:36:48, 14.91s/it]
 57%|█████▋    | 5611/9864 [6:59:24<18:05:15, 15.31s/it]
 57%|█████▋    | 5612/9864 [6:59:38<17:41:13, 14.98s/it]
 57%|█████▋    | 5613/9864 [6:59:53<17:34:49, 14.89s/it]
 57%|█████▋    | 5614/9864 [7:00:09<18:02:37, 15.28s/it]
 57%|█████▋    | 5615/9864 [7:00:26<18:26:47, 15.63s/it]
 57%|█████▋    | 5616/9864 [7:00:43<18:52:56, 16.00s/it]
 57%|█████▋    | 5617/9864 [7:00:57<18:20:05, 15.54s/it]
 57%|█████▋    | 5618/9864 [7:01:13<18:28:39, 15.67s/it]
 57%|█████▋    | 5619/9864 [7:01:27<17:58:47, 15.25s/it]
 57%|█████▋    | 5620/9864 [7:01:42<17:37:33, 14.95s/it]
                                                        
{'loss': '0.1202', 'grad_norm': '2.108', 'learning_rate': '8.239e-06', 'epoch': '0.5698'}

 57%|█████▋    | 5620/9864 [7:01:42<17:37:33, 14.95s/it]
 57%|█████▋    | 5621/9864 [7:01:56<17:16:50, 14.66s/it]
 57%|█████▋    | 5622/9864 [7:02:13<18:12:23, 15.45s/it]
 57%|█████▋    | 5623/9864 [7:02:27<17:44:01, 15.05s/it]
 57%|█████▋    | 5624/9864 [7:02:43<18:07:40, 15.39s/it]
 57%|█████▋    | 5625/9864 [7:02:57<17:41:32, 15.03s/it]
 57%|█████▋    | 5626/9864 [7:03:14<18:13:24, 15.48s/it]
 57%|█████▋    | 5627/9864 [7:03:29<17:56:04, 15.24s/it]
 57%|█████▋    | 5628/9864 [7:03:43<17:40:50, 15.03s/it]
 57%|█████▋    | 5629/9864 [7:03:59<17:58:36, 15.28s/it]
 57%|█████▋    | 5630/9864 [7:04:15<18:21:07, 15.60s/it]
                                                        
{'loss': '0.1317', 'grad_norm': '4.541', 'learning_rate': '8.207e-06', 'epoch': '0.5708'}

 57%|█████▋    | 5630/9864 [7:04:15<18:21:07, 15.60s/it]
 57%|█████▋    | 5631/9864 [7:04:32<18:33:59, 15.79s/it]
 57%|█████▋    | 5632/9864 [7:04:46<17:54:46, 15.24s/it]
 57%|█████▋    | 5633/9864 [7:05:00<17:43:05, 15.08s/it]
 57%|█████▋    | 5634/9864 [7:05:16<17:59:20, 15.31s/it]
 57%|█████▋    | 5635/9864 [7:05:31<17:47:39, 15.15s/it]
 57%|█████▋    | 5636/9864 [7:05:45<17:16:06, 14.70s/it]
 57%|█████▋    | 5637/9864 [7:06:01<17:44:38, 15.11s/it]
 57%|█████▋    | 5638/9864 [7:06:15<17:29:30, 14.90s/it]
 57%|█████▋    | 5639/9864 [7:06:31<18:01:56, 15.36s/it]
 57%|█████▋    | 5640/9864 [7:06:46<17:50:09, 15.20s/it]
                                                        
{'loss': '0.1269', 'grad_norm': '4.65', 'learning_rate': '8.175e-06', 'epoch': '0.5718'}

 57%|█████▋    | 5640/9864 [7:06:46<17:50:09, 15.20s/it]
 57%|█████▋    | 5641/9864 [7:07:01<17:46:58, 15.16s/it]
 57%|█████▋    | 5642/9864 [7:07:18<18:12:43, 15.53s/it]
 57%|█████▋    | 5643/9864 [7:07:33<17:57:04, 15.31s/it]
 57%|█████▋    | 5644/9864 [7:07:49<18:16:01, 15.58s/it]
 57%|█████▋    | 5645/9864 [7:08:05<18:29:17, 15.78s/it]
 57%|█████▋    | 5646/9864 [7:08:20<18:04:58, 15.43s/it]
 57%|█████▋    | 5647/9864 [7:08:35<18:10:35, 15.52s/it]
 57%|█████▋    | 5648/9864 [7:08:50<17:42:24, 15.12s/it]
 57%|█████▋    | 5649/9864 [7:09:06<18:00:25, 15.38s/it]
 57%|█████▋    | 5650/9864 [7:09:20<17:38:42, 15.07s/it]
                                                        
{'loss': '0.1202', 'grad_norm': '3.185', 'learning_rate': '8.143e-06', 'epoch': '0.5728'}

 57%|█████▋    | 5650/9864 [7:09:20<17:38:42, 15.07s/it]
 57%|█████▋    | 5651/9864 [7:09:33<17:05:28, 14.60s/it]
 57%|█████▋    | 5652/9864 [7:09:51<18:06:19, 15.47s/it]
 57%|█████▋    | 5653/9864 [7:10:05<17:41:10, 15.12s/it]
 57%|█████▋    | 5654/9864 [7:10:20<17:31:28, 14.99s/it]
 57%|█████▋    | 5655/9864 [7:10:36<17:56:37, 15.35s/it]
 57%|█████▋    | 5656/9864 [7:10:50<17:27:11, 14.93s/it]
 57%|█████▋    | 5657/9864 [7:11:04<17:08:52, 14.67s/it]
 57%|█████▋    | 5658/9864 [7:11:20<17:42:02, 15.15s/it]
 57%|█████▋    | 5659/9864 [7:11:34<17:18:43, 14.82s/it]
 57%|█████▋    | 5660/9864 [7:11:50<17:41:00, 15.14s/it]
                                                        
{'loss': '0.1169', 'grad_norm': '4.239', 'learning_rate': '8.11e-06', 'epoch': '0.5739'}

 57%|█████▋    | 5660/9864 [7:11:50<17:41:00, 15.14s/it]
 57%|█████▋    | 5661/9864 [7:12:06<17:51:27, 15.30s/it]
 57%|█████▋    | 5662/9864 [7:12:22<18:06:46, 15.52s/it]
 57%|█████▋    | 5663/9864 [7:12:36<17:40:29, 15.15s/it]
 57%|█████▋    | 5664/9864 [7:12:50<17:13:33, 14.77s/it]
 57%|█████▋    | 5665/9864 [7:13:06<17:38:58, 15.13s/it]
 57%|█████▋    | 5666/9864 [7:13:21<17:23:57, 14.92s/it]
 57%|█████▋    | 5667/9864 [7:13:36<17:35:08, 15.08s/it]
 57%|█████▋    | 5668/9864 [7:13:52<17:55:09, 15.37s/it]
 57%|█████▋    | 5669/9864 [7:14:06<17:19:23, 14.87s/it]
 57%|█████▋    | 5670/9864 [7:14:21<17:32:11, 15.05s/it]
                                                        
{'loss': '0.1045', 'grad_norm': '3.154', 'learning_rate': '8.078e-06', 'epoch': '0.5749'}

 57%|█████▋    | 5670/9864 [7:14:21<17:32:11, 15.05s/it]
 57%|█████▋    | 5671/9864 [7:14:35<17:10:46, 14.75s/it]
 58%|█████▊    | 5672/9864 [7:14:51<17:32:40, 15.07s/it]
 58%|█████▊    | 5673/9864 [7:15:06<17:20:52, 14.90s/it]
 58%|█████▊    | 5674/9864 [7:15:22<17:50:48, 15.33s/it]
 58%|█████▊    | 5675/9864 [7:15:37<17:53:59, 15.38s/it]
 58%|█████▊    | 5676/9864 [7:15:52<17:38:27, 15.16s/it]
 58%|█████▊    | 5677/9864 [7:16:07<17:30:21, 15.05s/it]
 58%|█████▊    | 5678/9864 [7:16:23<17:53:14, 15.38s/it]
 58%|█████▊    | 5679/9864 [7:16:37<17:29:17, 15.04s/it]
 58%|█████▊    | 5680/9864 [7:16:54<17:58:55, 15.47s/it]
                                                        
{'loss': '0.1209', 'grad_norm': '2.594', 'learning_rate': '8.046e-06', 'epoch': '0.5759'}

 58%|█████▊    | 5680/9864 [7:16:54<17:58:55, 15.47s/it]
 58%|█████▊    | 5681/9864 [7:17:08<17:39:47, 15.20s/it]
 58%|█████▊    | 5682/9864 [7:17:24<17:48:15, 15.33s/it]
 58%|█████▊    | 5683/9864 [7:17:40<18:02:14, 15.53s/it]
 58%|█████▊    | 5684/9864 [7:17:55<17:51:00, 15.37s/it]
 58%|█████▊    | 5685/9864 [7:18:10<17:33:42, 15.13s/it]
 58%|█████▊    | 5686/9864 [7:18:25<17:44:27, 15.29s/it]
 58%|█████▊    | 5687/9864 [7:18:40<17:33:05, 15.13s/it]
 58%|█████▊    | 5688/9864 [7:18:56<17:53:37, 15.43s/it]
 58%|█████▊    | 5689/9864 [7:19:10<17:28:31, 15.07s/it]
 58%|█████▊    | 5690/9864 [7:19:25<17:18:43, 14.93s/it]
                                                        
{'loss': '0.1271', 'grad_norm': '2.451', 'learning_rate': '8.014e-06', 'epoch': '0.5769'}

 58%|█████▊    | 5690/9864 [7:19:25<17:18:43, 14.93s/it]
 58%|█████▊    | 5691/9864 [7:19:43<18:22:42, 15.85s/it]
 58%|█████▊    | 5692/9864 [7:19:57<17:55:20, 15.47s/it]
 58%|█████▊    | 5693/9864 [7:20:11<17:23:57, 15.02s/it]
 58%|█████▊    | 5694/9864 [7:20:28<17:53:17, 15.44s/it]
 58%|█████▊    | 5695/9864 [7:20:42<17:30:58, 15.13s/it]
 58%|█████▊    | 5696/9864 [7:20:58<17:47:20, 15.36s/it]
 58%|█████▊    | 5697/9864 [7:21:12<17:21:18, 14.99s/it]
 58%|█████▊    | 5698/9864 [7:21:30<18:19:40, 15.84s/it]
 58%|█████▊    | 5699/9864 [7:21:44<17:47:50, 15.38s/it]
 58%|█████▊    | 5700/9864 [7:21:59<17:28:09, 15.10s/it]
                                                        
{'loss': '0.1376', 'grad_norm': '2.689', 'learning_rate': '7.982e-06', 'epoch': '0.5779'}

 58%|█████▊    | 5700/9864 [7:21:59<17:28:09, 15.10s/it]
 58%|█████▊    | 5701/9864 [7:22:13<17:03:13, 14.75s/it]
 58%|█████▊    | 5702/9864 [7:22:29<17:30:39, 15.15s/it]
 58%|█████▊    | 5703/9864 [7:22:44<17:21:26, 15.02s/it]
 58%|█████▊    | 5704/9864 [7:22:59<17:37:55, 15.26s/it]
 58%|█████▊    | 5705/9864 [7:23:13<17:11:11, 14.88s/it]
 58%|█████▊    | 5706/9864 [7:23:29<17:31:50, 15.18s/it]
 58%|█████▊    | 5707/9864 [7:23:45<17:45:09, 15.37s/it]
 58%|█████▊    | 5708/9864 [7:23:59<17:24:08, 15.07s/it]
 58%|█████▊    | 5709/9864 [7:24:13<17:00:59, 14.74s/it]
 58%|█████▊    | 5710/9864 [7:24:28<16:51:27, 14.61s/it]
                                                        
{'loss': '0.1212', 'grad_norm': '3.029', 'learning_rate': '7.949e-06', 'epoch': '0.5789'}

 58%|█████▊    | 5710/9864 [7:24:28<16:51:27, 14.61s/it]
 58%|█████▊    | 5711/9864 [7:24:44<17:18:35, 15.00s/it]
 58%|█████▊    | 5712/9864 [7:25:00<17:52:51, 15.50s/it]
 58%|█████▊    | 5713/9864 [7:25:15<17:33:18, 15.22s/it]
 58%|█████▊    | 5714/9864 [7:25:31<17:48:56, 15.45s/it]
 58%|█████▊    | 5715/9864 [7:25:45<17:28:09, 15.16s/it]
 58%|█████▊    | 5716/9864 [7:26:01<17:47:09, 15.44s/it]
 58%|█████▊    | 5717/9864 [7:26:16<17:24:32, 15.11s/it]
 58%|█████▊    | 5718/9864 [7:26:32<17:42:30, 15.38s/it]
 58%|█████▊    | 5719/9864 [7:26:48<17:51:03, 15.50s/it]
 58%|█████▊    | 5720/9864 [7:27:02<17:27:38, 15.17s/it]
                                                        
{'loss': '0.1316', 'grad_norm': '3.53', 'learning_rate': '7.917e-06', 'epoch': '0.5799'}

 58%|█████▊    | 5720/9864 [7:27:02<17:27:38, 15.17s/it]
 58%|█████▊    | 5721/9864 [7:27:16<17:02:25, 14.81s/it]
 58%|█████▊    | 5722/9864 [7:27:32<17:25:15, 15.14s/it]
 58%|█████▊    | 5723/9864 [7:27:47<17:15:19, 15.00s/it]
 58%|█████▊    | 5724/9864 [7:28:03<17:48:18, 15.48s/it]
 58%|█████▊    | 5725/9864 [7:28:17<17:23:27, 15.13s/it]
 58%|█████▊    | 5726/9864 [7:28:32<17:02:58, 14.83s/it]
 58%|█████▊    | 5727/9864 [7:28:49<17:59:13, 15.65s/it]
 58%|█████▊    | 5728/9864 [7:29:04<17:39:34, 15.37s/it]
 58%|█████▊    | 5729/9864 [7:29:20<18:00:09, 15.67s/it]
 58%|█████▊    | 5730/9864 [7:29:34<17:27:46, 15.21s/it]
                                                        
{'loss': '0.1239', 'grad_norm': '2.474', 'learning_rate': '7.885e-06', 'epoch': '0.5809'}

 58%|█████▊    | 5730/9864 [7:29:34<17:27:46, 15.21s/it]
 58%|█████▊    | 5731/9864 [7:29:49<17:07:21, 14.91s/it]
 58%|█████▊    | 5732/9864 [7:30:04<17:25:50, 15.19s/it]
 58%|█████▊    | 5733/9864 [7:30:19<17:14:37, 15.03s/it]
 58%|█████▊    | 5734/9864 [7:30:34<17:06:17, 14.91s/it]
 58%|█████▊    | 5735/9864 [7:30:50<17:38:52, 15.39s/it]
 58%|█████▊    | 5736/9864 [7:31:06<17:52:12, 15.58s/it]
 58%|█████▊    | 5737/9864 [7:31:22<17:57:21, 15.66s/it]
 58%|█████▊    | 5738/9864 [7:31:36<17:30:01, 15.27s/it]
 58%|█████▊    | 5739/9864 [7:31:51<17:06:44, 14.93s/it]
 58%|█████▊    | 5740/9864 [7:32:07<17:26:23, 15.22s/it]
                                                        
{'loss': '0.1259', 'grad_norm': '2.332', 'learning_rate': '7.853e-06', 'epoch': '0.582'}

 58%|█████▊    | 5740/9864 [7:32:07<17:26:23, 15.22s/it]
 58%|█████▊    | 5741/9864 [7:32:21<17:06:24, 14.94s/it]
 58%|█████▊    | 5742/9864 [7:32:35<16:49:40, 14.70s/it]
 58%|█████▊    | 5743/9864 [7:32:49<16:44:10, 14.62s/it]
 58%|█████▊    | 5744/9864 [7:33:07<17:52:52, 15.62s/it]
 58%|█████▊    | 5745/9864 [7:33:23<18:01:57, 15.76s/it]
 58%|█████▊    | 5746/9864 [7:33:38<17:29:10, 15.29s/it]
 58%|█████▊    | 5747/9864 [7:33:52<17:01:48, 14.89s/it]
 58%|█████▊    | 5748/9864 [7:34:08<17:27:30, 15.27s/it]
 58%|█████▊    | 5749/9864 [7:34:22<16:57:22, 14.83s/it]
 58%|█████▊    | 5750/9864 [7:34:35<16:33:17, 14.49s/it]
                                                        
{'loss': '1.099', 'grad_norm': '3.454', 'learning_rate': '7.821e-06', 'epoch': '0.583'}

 58%|█████▊    | 5750/9864 [7:34:35<16:33:17, 14.49s/it]
 58%|█████▊    | 5751/9864 [7:34:49<16:25:44, 14.38s/it]
 58%|█████▊    | 5752/9864 [7:35:09<18:16:26, 16.00s/it]
 58%|█████▊    | 5753/9864 [7:35:24<17:47:29, 15.58s/it]
 58%|█████▊    | 5754/9864 [7:35:38<17:23:44, 15.24s/it]
 58%|█████▊    | 5755/9864 [7:35:52<17:04:00, 14.95s/it]
 58%|█████▊    | 5756/9864 [7:36:08<17:25:31, 15.27s/it]
 58%|█████▊    | 5757/9864 [7:36:23<17:06:02, 14.99s/it]
 58%|█████▊    | 5758/9864 [7:36:37<16:47:56, 14.73s/it]
 58%|█████▊    | 5759/9864 [7:36:53<17:11:18, 15.07s/it]
 58%|█████▊    | 5760/9864 [7:37:09<17:39:37, 15.49s/it]
                                                        
{'loss': '0.1496', 'grad_norm': '4.984', 'learning_rate': '7.789e-06', 'epoch': '0.584'}

 58%|█████▊    | 5760/9864 [7:37:09<17:39:37, 15.49s/it]
 58%|█████▊    | 5761/9864 [7:37:25<17:53:48, 15.70s/it]
 58%|█████▊    | 5762/9864 [7:37:40<17:22:36, 15.25s/it]
 58%|█████▊    | 5763/9864 [7:37:54<17:02:25, 14.96s/it]
 58%|█████▊    | 5764/9864 [7:38:08<16:43:03, 14.68s/it]
 58%|█████▊    | 5765/9864 [7:38:24<17:11:39, 15.10s/it]
 58%|█████▊    | 5766/9864 [7:38:38<16:53:34, 14.84s/it]
 58%|█████▊    | 5767/9864 [7:38:54<17:21:13, 15.25s/it]
 58%|█████▊    | 5768/9864 [7:39:11<17:44:35, 15.59s/it]
 58%|█████▊    | 5769/9864 [7:39:27<17:49:32, 15.67s/it]
 58%|█████▊    | 5770/9864 [7:39:41<17:17:57, 15.21s/it]
                                                        
{'loss': '0.1357', 'grad_norm': '3.48', 'learning_rate': '7.757e-06', 'epoch': '0.585'}

 58%|█████▊    | 5770/9864 [7:39:41<17:17:57, 15.21s/it]
 59%|█████▊    | 5771/9864 [7:39:55<17:01:55, 14.98s/it]
 59%|█████▊    | 5772/9864 [7:40:11<17:21:29, 15.27s/it]
 59%|█████▊    | 5773/9864 [7:40:26<17:04:31, 15.03s/it]
 59%|█████▊    | 5774/9864 [7:40:40<16:49:01, 14.80s/it]
 59%|█████▊    | 5775/9864 [7:40:55<16:43:56, 14.73s/it]
 59%|█████▊    | 5776/9864 [7:41:14<18:29:49, 16.29s/it]
 59%|█████▊    | 5777/9864 [7:41:29<17:48:30, 15.69s/it]
 59%|█████▊    | 5778/9864 [7:41:43<17:26:09, 15.36s/it]
 59%|█████▊    | 5779/9864 [7:42:00<17:44:05, 15.63s/it]
 59%|█████▊    | 5780/9864 [7:42:14<17:12:40, 15.17s/it]
                                                        
{'loss': '0.1416', 'grad_norm': '4.454', 'learning_rate': '7.725e-06', 'epoch': '0.586'}

 59%|█████▊    | 5780/9864 [7:42:14<17:12:40, 15.17s/it]
 59%|█████▊    | 5781/9864 [7:42:28<16:46:18, 14.79s/it]
 59%|█████▊    | 5782/9864 [7:42:42<16:41:00, 14.71s/it]
 59%|█████▊    | 5783/9864 [7:42:59<17:15:40, 15.23s/it]
 59%|█████▊    | 5784/9864 [7:43:17<18:10:59, 16.04s/it]
 59%|█████▊    | 5785/9864 [7:43:31<17:39:14, 15.58s/it]
 59%|█████▊    | 5786/9864 [7:43:45<17:15:29, 15.24s/it]
 59%|█████▊    | 5787/9864 [7:44:02<17:41:33, 15.62s/it]
 59%|█████▊    | 5788/9864 [7:44:16<17:17:17, 15.27s/it]
 59%|█████▊    | 5789/9864 [7:44:31<16:55:29, 14.95s/it]
 59%|█████▊    | 5790/9864 [7:44:45<16:45:59, 14.82s/it]
                                                        
{'loss': '0.1629', 'grad_norm': '3.261', 'learning_rate': '7.693e-06', 'epoch': '0.587'}

 59%|█████▊    | 5790/9864 [7:44:45<16:45:59, 14.82s/it]
 59%|█████▊    | 5791/9864 [7:45:03<17:46:28, 15.71s/it]
 59%|█████▊    | 5792/9864 [7:45:19<17:49:46, 15.76s/it]
 59%|█████▊    | 5793/9864 [7:45:34<17:35:20, 15.55s/it]
 59%|█████▊    | 5794/9864 [7:45:48<17:04:51, 15.11s/it]
 59%|█████▊    | 5795/9864 [7:46:02<16:36:45, 14.70s/it]
 59%|█████▉    | 5796/9864 [7:46:17<16:58:24, 15.02s/it]
 59%|█████▉    | 5797/9864 [7:46:32<16:47:53, 14.87s/it]
 59%|█████▉    | 5798/9864 [7:46:48<17:07:54, 15.17s/it]
 59%|█████▉    | 5799/9864 [7:47:05<17:44:02, 15.71s/it]
 59%|█████▉    | 5800/9864 [7:47:21<17:51:29, 15.82s/it]
                                                        
{'loss': '0.1277', 'grad_norm': '5.377', 'learning_rate': '7.661e-06', 'epoch': '0.588'}

 59%|█████▉    | 5800/9864 [7:47:21<17:51:29, 15.82s/it]
 59%|█████▉    | 5801/9864 [7:47:36<17:29:05, 15.49s/it]
 59%|█████▉    | 5802/9864 [7:47:50<17:00:14, 15.07s/it]
 59%|█████▉    | 5803/9864 [7:48:04<16:46:32, 14.87s/it]
 59%|█████▉    | 5804/9864 [7:48:20<17:11:54, 15.25s/it]
 59%|█████▉    | 5805/9864 [7:48:35<16:52:44, 14.97s/it]
 59%|█████▉    | 5806/9864 [7:48:51<17:30:36, 15.53s/it]
 59%|█████▉    | 5807/9864 [7:49:07<17:37:26, 15.64s/it]
 59%|█████▉    | 5808/9864 [7:49:23<17:47:03, 15.78s/it]
 59%|█████▉    | 5809/9864 [7:49:38<17:21:31, 15.41s/it]
 59%|█████▉    | 5810/9864 [7:49:52<17:02:28, 15.13s/it]
                                                        
{'loss': '0.1381', 'grad_norm': '2.754', 'learning_rate': '7.629e-06', 'epoch': '0.5891'}

 59%|█████▉    | 5810/9864 [7:49:52<17:02:28, 15.13s/it]
 59%|█████▉    | 5811/9864 [7:50:07<16:40:19, 14.81s/it]
 59%|█████▉    | 5812/9864 [7:50:23<17:07:57, 15.22s/it]
 59%|█████▉    | 5813/9864 [7:50:36<16:36:30, 14.76s/it]
 59%|█████▉    | 5814/9864 [7:50:52<17:01:30, 15.13s/it]
 59%|█████▉    | 5815/9864 [7:51:09<17:21:41, 15.44s/it]
 59%|█████▉    | 5816/9864 [7:51:23<17:10:33, 15.27s/it]
 59%|█████▉    | 5817/9864 [7:51:38<16:54:56, 15.05s/it]
 59%|█████▉    | 5818/9864 [7:51:53<16:47:47, 14.95s/it]
 59%|█████▉    | 5819/9864 [7:52:09<17:07:18, 15.24s/it]
 59%|█████▉    | 5820/9864 [7:52:23<16:49:10, 14.97s/it]
                                                        
{'loss': '0.1375', 'grad_norm': '3.483', 'learning_rate': '7.597e-06', 'epoch': '0.5901'}

 59%|█████▉    | 5820/9864 [7:52:23<16:49:10, 14.97s/it]
 59%|█████▉    | 5821/9864 [7:52:39<17:16:31, 15.38s/it]
 59%|█████▉    | 5822/9864 [7:52:57<18:08:46, 16.16s/it]
 59%|█████▉    | 5823/9864 [7:53:12<17:32:18, 15.62s/it]
 59%|█████▉    | 5824/9864 [7:53:26<17:07:40, 15.26s/it]
 59%|█████▉    | 5825/9864 [7:53:40<16:49:48, 15.00s/it]
 59%|█████▉    | 5826/9864 [7:53:54<16:30:40, 14.72s/it]
 59%|█████▉    | 5827/9864 [7:54:11<16:58:28, 15.14s/it]
 59%|█████▉    | 5828/9864 [7:54:26<17:12:40, 15.35s/it]
 59%|█████▉    | 5829/9864 [7:54:41<16:58:48, 15.15s/it]
 59%|█████▉    | 5830/9864 [7:54:59<17:56:48, 16.02s/it]
                                                        
{'loss': '0.1229', 'grad_norm': '3.318', 'learning_rate': '7.565e-06', 'epoch': '0.5911'}

 59%|█████▉    | 5830/9864 [7:54:59<17:56:48, 16.02s/it]
 59%|█████▉    | 5831/9864 [7:55:13<17:16:08, 15.41s/it]
 59%|█████▉    | 5832/9864 [7:55:27<16:45:56, 14.97s/it]
 59%|█████▉    | 5833/9864 [7:55:42<16:36:49, 14.84s/it]
 59%|█████▉    | 5834/9864 [7:55:55<16:14:57, 14.52s/it]
 59%|█████▉    | 5835/9864 [7:56:11<16:41:58, 14.92s/it]
 59%|█████▉    | 5836/9864 [7:56:27<16:53:00, 15.09s/it]
 59%|█████▉    | 5837/9864 [7:56:43<17:16:40, 15.45s/it]
 59%|█████▉    | 5838/9864 [7:56:59<17:19:18, 15.49s/it]
 59%|█████▉    | 5839/9864 [7:57:13<16:50:53, 15.07s/it]
 59%|█████▉    | 5840/9864 [7:57:27<16:31:39, 14.79s/it]
                                                        
{'loss': '0.1378', 'grad_norm': '3.433', 'learning_rate': '7.534e-06', 'epoch': '0.5921'}

 59%|█████▉    | 5840/9864 [7:57:27<16:31:39, 14.79s/it]
 59%|█████▉    | 5841/9864 [7:57:41<16:26:43, 14.72s/it]
 59%|█████▉    | 5842/9864 [7:57:56<16:23:09, 14.67s/it]
 59%|█████▉    | 5843/9864 [7:58:12<16:54:09, 15.13s/it]
 59%|█████▉    | 5844/9864 [7:58:30<17:42:17, 15.86s/it]
 59%|█████▉    | 5845/9864 [7:58:44<17:11:10, 15.39s/it]
 59%|█████▉    | 5846/9864 [7:59:00<17:19:08, 15.52s/it]
 59%|█████▉    | 5847/9864 [7:59:14<16:59:05, 15.22s/it]
 59%|█████▉    | 5848/9864 [7:59:29<16:44:43, 15.01s/it]
 59%|█████▉    | 5849/9864 [7:59:43<16:26:18, 14.74s/it]
 59%|█████▉    | 5850/9864 [7:59:59<16:56:51, 15.20s/it]
                                                        
{'loss': '0.1322', 'grad_norm': '2.885', 'learning_rate': '7.502e-06', 'epoch': '0.5931'}

 59%|█████▉    | 5850/9864 [7:59:59<16:56:51, 15.20s/it]
 59%|█████▉    | 5851/9864 [8:00:14<16:39:52, 14.95s/it]
 59%|█████▉    | 5852/9864 [8:00:30<17:06:12, 15.35s/it]
 59%|█████▉    | 5853/9864 [8:00:46<17:26:57, 15.66s/it]
 59%|█████▉    | 5854/9864 [8:01:01<17:06:38, 15.36s/it]
 59%|█████▉    | 5855/9864 [8:01:16<16:53:12, 15.16s/it]
 59%|█████▉    | 5856/9864 [8:01:30<16:41:47, 15.00s/it]
 59%|█████▉    | 5857/9864 [8:01:46<16:55:01, 15.20s/it]
 59%|█████▉    | 5858/9864 [8:02:01<16:42:35, 15.02s/it]
 59%|█████▉    | 5859/9864 [8:02:16<16:55:14, 15.21s/it]
 59%|█████▉    | 5860/9864 [8:02:32<17:17:00, 15.54s/it]
                                                        
{'loss': '0.1355', 'grad_norm': '3.989', 'learning_rate': '7.47e-06', 'epoch': '0.5941'}

 59%|█████▉    | 5860/9864 [8:02:32<17:17:00, 15.54s/it]
 59%|█████▉    | 5861/9864 [8:02:48<17:25:09, 15.67s/it]
 59%|█████▉    | 5862/9864 [8:03:03<16:59:19, 15.28s/it]
 59%|█████▉    | 5863/9864 [8:03:17<16:39:44, 14.99s/it]
 59%|█████▉    | 5864/9864 [8:03:32<16:29:01, 14.84s/it]
 59%|█████▉    | 5865/9864 [8:03:47<16:41:17, 15.02s/it]
 59%|█████▉    | 5866/9864 [8:04:01<16:20:36, 14.72s/it]
 59%|█████▉    | 5867/9864 [8:04:17<16:45:40, 15.10s/it]
 59%|█████▉    | 5868/9864 [8:04:34<17:19:30, 15.61s/it]
 59%|█████▉    | 5869/9864 [8:04:50<17:20:25, 15.63s/it]
 60%|█████▉    | 5870/9864 [8:05:04<16:55:50, 15.26s/it]
                                                        
{'loss': '0.1322', 'grad_norm': '2.489', 'learning_rate': '7.438e-06', 'epoch': '0.5951'}

 60%|█████▉    | 5870/9864 [8:05:04<16:55:50, 15.26s/it]
 60%|█████▉    | 5871/9864 [8:05:18<16:35:04, 14.95s/it]
 60%|█████▉    | 5872/9864 [8:05:34<16:51:14, 15.20s/it]
 60%|█████▉    | 5873/9864 [8:05:49<16:46:19, 15.13s/it]
 60%|█████▉    | 5874/9864 [8:06:03<16:32:04, 14.92s/it]
 60%|█████▉    | 5875/9864 [8:06:18<16:26:58, 14.85s/it]
 60%|█████▉    | 5876/9864 [8:06:37<17:56:28, 16.20s/it]
 60%|█████▉    | 5877/9864 [8:06:51<17:10:42, 15.51s/it]
 60%|█████▉    | 5878/9864 [8:07:06<16:51:26, 15.22s/it]
 60%|█████▉    | 5879/9864 [8:07:22<17:08:31, 15.49s/it]
 60%|█████▉    | 5880/9864 [8:07:37<16:54:19, 15.28s/it]
                                                        
{'loss': '0.1239', 'grad_norm': '3.305', 'learning_rate': '7.407e-06', 'epoch': '0.5962'}

 60%|█████▉    | 5880/9864 [8:07:37<16:54:19, 15.28s/it]
 60%|█████▉    | 5881/9864 [8:07:51<16:25:02, 14.84s/it]
 60%|█████▉    | 5882/9864 [8:08:05<16:16:21, 14.71s/it]
 60%|█████▉    | 5883/9864 [8:08:21<16:42:38, 15.11s/it]
 60%|█████▉    | 5884/9864 [8:08:39<17:36:30, 15.93s/it]
 60%|█████▉    | 5885/9864 [8:08:53<17:00:55, 15.39s/it]
 60%|█████▉    | 5886/9864 [8:09:07<16:41:33, 15.11s/it]
 60%|█████▉    | 5887/9864 [8:09:23<16:58:10, 15.36s/it]
 60%|█████▉    | 5888/9864 [8:09:38<16:34:05, 15.00s/it]
 60%|█████▉    | 5889/9864 [8:09:52<16:27:32, 14.91s/it]
 60%|█████▉    | 5890/9864 [8:10:09<16:55:55, 15.34s/it]
                                                        
{'loss': '0.1257', 'grad_norm': '3.373', 'learning_rate': '7.375e-06', 'epoch': '0.5972'}

 60%|█████▉    | 5890/9864 [8:10:09<16:55:55, 15.34s/it]
 60%|█████▉    | 5891/9864 [8:10:23<16:35:12, 15.03s/it]
 60%|█████▉    | 5892/9864 [8:10:39<16:47:12, 15.21s/it]
 60%|█████▉    | 5893/9864 [8:10:56<17:23:57, 15.77s/it]
 60%|█████▉    | 5894/9864 [8:11:12<17:36:39, 15.97s/it]
 60%|█████▉    | 5895/9864 [8:11:26<17:03:29, 15.47s/it]
 60%|█████▉    | 5896/9864 [8:11:41<16:39:53, 15.12s/it]
 60%|█████▉    | 5897/9864 [8:11:55<16:19:19, 14.81s/it]
 60%|█████▉    | 5898/9864 [8:12:09<16:03:12, 14.57s/it]
 60%|█████▉    | 5899/9864 [8:12:25<16:29:25, 14.97s/it]
 60%|█████▉    | 5900/9864 [8:12:40<16:44:54, 15.21s/it]
                                                        
{'loss': '0.1386', 'grad_norm': '3.419', 'learning_rate': '7.343e-06', 'epoch': '0.5982'}

 60%|█████▉    | 5900/9864 [8:12:40<16:44:54, 15.21s/it]
 60%|█████▉    | 5901/9864 [8:12:56<17:01:16, 15.46s/it]
 60%|█████▉    | 5902/9864 [8:13:13<17:15:04, 15.68s/it]
 60%|█████▉    | 5903/9864 [8:13:27<16:48:31, 15.28s/it]
 60%|█████▉    | 5904/9864 [8:13:41<16:32:04, 15.03s/it]
 60%|█████▉    | 5905/9864 [8:13:56<16:21:44, 14.88s/it]
 60%|█████▉    | 5906/9864 [8:14:12<16:49:46, 15.31s/it]
 60%|█████▉    | 5907/9864 [8:14:27<16:30:31, 15.02s/it]
 60%|█████▉    | 5908/9864 [8:14:41<16:16:51, 14.82s/it]
 60%|█████▉    | 5909/9864 [8:14:57<16:44:31, 15.24s/it]
 60%|█████▉    | 5910/9864 [8:15:15<17:33:21, 15.98s/it]
                                                        
{'loss': '0.1317', 'grad_norm': '4.045', 'learning_rate': '7.312e-06', 'epoch': '0.5992'}

 60%|█████▉    | 5910/9864 [8:15:15<17:33:21, 15.98s/it]
 60%|█████▉    | 5911/9864 [8:15:29<16:56:17, 15.43s/it]
 60%|█████▉    | 5912/9864 [8:15:43<16:20:21, 14.88s/it]
 60%|█████▉    | 5913/9864 [8:15:57<16:08:25, 14.71s/it]
 60%|█████▉    | 5914/9864 [8:16:13<16:27:59, 15.01s/it]
 60%|█████▉    | 5915/9864 [8:16:27<16:11:10, 14.76s/it]
 60%|█████▉    | 5916/9864 [8:16:42<16:10:26, 14.75s/it]
 60%|█████▉    | 5917/9864 [8:17:00<17:19:03, 15.80s/it]
 60%|█████▉    | 5918/9864 [8:17:14<16:42:07, 15.24s/it]
 60%|██████    | 5919/9864 [8:17:30<16:52:54, 15.41s/it]
 60%|██████    | 5920/9864 [8:17:43<16:24:03, 14.97s/it]
                                                        
{'loss': '0.1348', 'grad_norm': '2.99', 'learning_rate': '7.28e-06', 'epoch': '0.6002'}

 60%|██████    | 5920/9864 [8:17:43<16:24:03, 14.97s/it]
 60%|██████    | 5921/9864 [8:17:58<16:12:40, 14.80s/it]
 60%|██████    | 5922/9864 [8:18:14<16:35:12, 15.15s/it]
 60%|██████    | 5923/9864 [8:18:29<16:29:07, 15.06s/it]
 60%|██████    | 5924/9864 [8:18:45<16:45:57, 15.32s/it]
 60%|██████    | 5925/9864 [8:19:01<17:09:53, 15.69s/it]
 60%|██████    | 5926/9864 [8:19:15<16:36:05, 15.18s/it]
 60%|██████    | 5927/9864 [8:19:32<16:59:19, 15.53s/it]
 60%|██████    | 5928/9864 [8:19:45<16:27:39, 15.06s/it]
 60%|██████    | 5929/9864 [8:20:00<16:23:16, 14.99s/it]
 60%|██████    | 5930/9864 [8:20:17<16:51:45, 15.43s/it]
                                                        
{'loss': '0.1319', 'grad_norm': '3.43', 'learning_rate': '7.248e-06', 'epoch': '0.6012'}

 60%|██████    | 5930/9864 [8:20:17<16:51:45, 15.43s/it]
 60%|██████    | 5931/9864 [8:20:33<16:59:53, 15.56s/it]
 60%|██████    | 5932/9864 [8:20:49<17:06:22, 15.66s/it]
 60%|██████    | 5933/9864 [8:21:03<16:45:29, 15.35s/it]
 60%|██████    | 5934/9864 [8:21:17<16:15:05, 14.89s/it]
 60%|██████    | 5935/9864 [8:21:32<16:24:07, 15.03s/it]
 60%|██████    | 5936/9864 [8:21:46<16:01:35, 14.69s/it]
 60%|██████    | 5937/9864 [8:22:02<16:33:10, 15.17s/it]
 60%|██████    | 5938/9864 [8:22:17<16:13:26, 14.88s/it]
 60%|██████    | 5939/9864 [8:22:33<16:41:05, 15.30s/it]
 60%|██████    | 5940/9864 [8:22:49<17:00:43, 15.61s/it]
                                                        
{'loss': '0.109', 'grad_norm': '2.67', 'learning_rate': '7.217e-06', 'epoch': '0.6022'}

 60%|██████    | 5940/9864 [8:22:49<17:00:43, 15.61s/it]
 60%|██████    | 5941/9864 [8:23:03<16:29:56, 15.14s/it]
 60%|██████    | 5942/9864 [8:23:18<16:17:47, 14.96s/it]
 60%|██████    | 5943/9864 [8:23:34<16:33:27, 15.20s/it]
 60%|██████    | 5944/9864 [8:23:47<16:04:47, 14.77s/it]
 60%|██████    | 5945/9864 [8:24:03<16:21:34, 15.03s/it]
 60%|██████    | 5946/9864 [8:24:17<16:06:13, 14.80s/it]
 60%|██████    | 5947/9864 [8:24:33<16:30:28, 15.17s/it]
 60%|██████    | 5948/9864 [8:24:50<16:55:53, 15.57s/it]
 60%|██████    | 5949/9864 [8:25:04<16:34:31, 15.24s/it]
 60%|██████    | 5950/9864 [8:25:19<16:21:22, 15.04s/it]
                                                        
{'loss': '0.1317', 'grad_norm': '3.478', 'learning_rate': '7.185e-06', 'epoch': '0.6033'}

 60%|██████    | 5950/9864 [8:25:19<16:21:22, 15.04s/it]
 60%|██████    | 5951/9864 [8:25:35<16:36:33, 15.28s/it]
 60%|██████    | 5952/9864 [8:25:50<16:45:56, 15.43s/it]
 60%|██████    | 5953/9864 [8:26:04<16:16:29, 14.98s/it]
 60%|██████    | 5954/9864 [8:26:18<15:57:23, 14.69s/it]
 60%|██████    | 5955/9864 [8:26:32<15:40:14, 14.43s/it]
 60%|██████    | 5956/9864 [8:26:49<16:22:46, 15.09s/it]
 60%|██████    | 5957/9864 [8:27:02<15:51:50, 14.62s/it]
 60%|██████    | 5958/9864 [8:27:19<16:25:57, 15.15s/it]
 60%|██████    | 5959/9864 [8:27:33<16:11:17, 14.92s/it]
 60%|██████    | 5960/9864 [8:27:49<16:35:51, 15.31s/it]
                                                        
{'loss': '0.1041', 'grad_norm': '3.238', 'learning_rate': '7.154e-06', 'epoch': '0.6043'}

 60%|██████    | 5960/9864 [8:27:49<16:35:51, 15.31s/it]
 60%|██████    | 5961/9864 [8:28:04<16:15:45, 15.00s/it]
 60%|██████    | 5962/9864 [8:28:18<16:08:47, 14.90s/it]
 60%|██████    | 5963/9864 [8:28:35<16:38:51, 15.36s/it]
 60%|██████    | 5964/9864 [8:28:51<16:51:40, 15.56s/it]
 60%|██████    | 5965/9864 [8:29:05<16:30:15, 15.24s/it]
 60%|██████    | 5966/9864 [8:29:22<16:56:55, 15.65s/it]
 60%|██████    | 5967/9864 [8:29:38<17:13:39, 15.91s/it]
 61%|██████    | 5968/9864 [8:29:52<16:35:51, 15.34s/it]
 61%|██████    | 5969/9864 [8:30:06<16:10:08, 14.94s/it]
 61%|██████    | 5970/9864 [8:30:21<16:03:37, 14.85s/it]
                                                        
{'loss': '0.1213', 'grad_norm': '3.234', 'learning_rate': '7.122e-06', 'epoch': '0.6053'}

 61%|██████    | 5970/9864 [8:30:21<16:03:37, 14.85s/it]
 61%|██████    | 5971/9864 [8:30:39<17:07:30, 15.84s/it]
 61%|██████    | 5972/9864 [8:30:54<16:39:00, 15.40s/it]
 61%|██████    | 5973/9864 [8:31:08<16:24:39, 15.18s/it]
 61%|██████    | 5974/9864 [8:31:24<16:39:10, 15.41s/it]
 61%|██████    | 5975/9864 [8:31:40<16:52:29, 15.62s/it]
 61%|██████    | 5976/9864 [8:31:55<16:41:47, 15.46s/it]
 61%|██████    | 5977/9864 [8:32:10<16:15:27, 15.06s/it]
 61%|██████    | 5978/9864 [8:32:23<15:51:27, 14.69s/it]
 61%|██████    | 5979/9864 [8:32:41<16:49:44, 15.59s/it]
 61%|██████    | 5980/9864 [8:32:55<16:22:42, 15.18s/it]
                                                        
{'loss': '0.1162', 'grad_norm': '3.341', 'learning_rate': '7.091e-06', 'epoch': '0.6063'}

 61%|██████    | 5980/9864 [8:32:55<16:22:42, 15.18s/it]
 61%|██████    | 5981/9864 [8:33:10<16:08:37, 14.97s/it]
 61%|██████    | 5982/9864 [8:33:28<17:13:49, 15.98s/it]
 61%|██████    | 5983/9864 [8:33:42<16:42:45, 15.50s/it]
 61%|██████    | 5984/9864 [8:33:57<16:18:16, 15.13s/it]
 61%|██████    | 5985/9864 [8:34:11<16:04:04, 14.91s/it]
 61%|██████    | 5986/9864 [8:34:28<16:35:04, 15.40s/it]
 61%|██████    | 5987/9864 [8:34:44<16:45:13, 15.56s/it]
 61%|██████    | 5988/9864 [8:34:58<16:18:24, 15.15s/it]
 61%|██████    | 5989/9864 [8:35:14<16:42:52, 15.53s/it]
 61%|██████    | 5990/9864 [8:35:29<16:21:18, 15.20s/it]
                                                        
{'loss': '0.1201', 'grad_norm': '3.696', 'learning_rate': '7.059e-06', 'epoch': '0.6073'}

 61%|██████    | 5990/9864 [8:35:29<16:21:18, 15.20s/it]
 61%|██████    | 5991/9864 [8:35:45<16:35:49, 15.43s/it]
 61%|██████    | 5992/9864 [8:35:59<16:20:53, 15.20s/it]
 61%|██████    | 5993/9864 [8:36:14<16:03:26, 14.93s/it]
 61%|██████    | 5994/9864 [8:36:30<16:32:24, 15.39s/it]
 61%|██████    | 5995/9864 [8:36:46<16:41:48, 15.54s/it]
 61%|██████    | 5996/9864 [8:37:00<16:18:10, 15.17s/it]
 61%|██████    | 5997/9864 [8:37:14<15:49:10, 14.73s/it]
 61%|██████    | 5998/9864 [8:37:32<16:50:17, 15.68s/it]
 61%|██████    | 5999/9864 [8:37:46<16:29:30, 15.36s/it]
 61%|██████    | 6000/9864 [8:38:01<16:12:32, 15.10s/it]
                                                        
{'loss': '0.111', 'grad_norm': '3.493', 'learning_rate': '7.028e-06', 'epoch': '0.6083'}

 61%|██████    | 6000/9864 [8:38:01<16:12:32, 15.10s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:09<00:00,  9.52s/it]
Writing model shards: 100%|██████████| 1/1 [00:09<00:00,  9.52s/it]
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 61%|██████    | 6001/9864 [8:39:16<35:35:57, 33.18s/it]
 61%|██████    | 6002/9864 [8:39:33<30:17:06, 28.23s/it]
 61%|██████    | 6003/9864 [8:39:47<25:42:55, 23.98s/it]
 61%|██████    | 6004/9864 [8:40:01<22:30:12, 20.99s/it]
 61%|██████    | 6005/9864 [8:40:17<20:59:25, 19.58s/it]
 61%|██████    | 6006/9864 [8:40:33<19:50:44, 18.52s/it]
 61%|██████    | 6007/9864 [8:40:48<18:34:58, 17.34s/it]
 61%|██████    | 6008/9864 [8:41:04<18:17:50, 17.08s/it]
 61%|██████    | 6009/9864 [8:41:21<18:04:06, 16.87s/it]
 61%|██████    | 6010/9864 [8:41:35<17:10:08, 16.04s/it]
                                                        
{'loss': '0.1143', 'grad_norm': '4.116', 'learning_rate': '6.997e-06', 'epoch': '0.6093'}

 61%|██████    | 6010/9864 [8:41:35<17:10:08, 16.04s/it]
 61%|██████    | 6011/9864 [8:41:49<16:35:02, 15.49s/it]
 61%|██████    | 6012/9864 [8:42:04<16:19:39, 15.26s/it]
 61%|██████    | 6013/9864 [8:42:20<16:35:31, 15.51s/it]
 61%|██████    | 6014/9864 [8:42:36<16:37:13, 15.54s/it]
 61%|██████    | 6015/9864 [8:42:50<16:19:37, 15.27s/it]
 61%|██████    | 6016/9864 [8:43:08<17:04:36, 15.98s/it]
 61%|██████    | 6017/9864 [8:43:22<16:21:42, 15.31s/it]
 61%|██████    | 6018/9864 [8:43:35<15:52:34, 14.86s/it]
 61%|██████    | 6019/9864 [8:43:49<15:32:26, 14.55s/it]
 61%|██████    | 6020/9864 [8:44:05<16:01:48, 15.01s/it]
                                                        
{'loss': '0.1032', 'grad_norm': '3.261', 'learning_rate': '6.965e-06', 'epoch': '0.6104'}

 61%|██████    | 6020/9864 [8:44:05<16:01:48, 15.01s/it]
 61%|██████    | 6021/9864 [8:44:22<16:24:56, 15.38s/it]
 61%|██████    | 6022/9864 [8:44:36<16:13:41, 15.21s/it]
 61%|██████    | 6023/9864 [8:44:50<15:51:32, 14.86s/it]
 61%|██████    | 6024/9864 [8:45:04<15:35:15, 14.61s/it]
 61%|██████    | 6025/9864 [8:45:22<16:40:29, 15.64s/it]
 61%|██████    | 6026/9864 [8:45:37<16:18:42, 15.30s/it]
 61%|██████    | 6027/9864 [8:45:51<15:51:54, 14.89s/it]
 61%|██████    | 6028/9864 [8:46:05<15:34:27, 14.62s/it]
 61%|██████    | 6029/9864 [8:46:22<16:30:56, 15.50s/it]
 61%|██████    | 6030/9864 [8:46:37<16:16:38, 15.28s/it]
                                                        
{'loss': '0.1031', 'grad_norm': '3.685', 'learning_rate': '6.934e-06', 'epoch': '0.6114'}

 61%|██████    | 6030/9864 [8:46:37<16:16:38, 15.28s/it]
 61%|██████    | 6031/9864 [8:46:51<15:55:15, 14.95s/it]
 61%|██████    | 6032/9864 [8:47:09<16:47:20, 15.77s/it]
 61%|██████    | 6033/9864 [8:47:24<16:21:39, 15.37s/it]
 61%|██████    | 6034/9864 [8:47:38<16:07:42, 15.16s/it]
 61%|██████    | 6035/9864 [8:47:53<15:54:24, 14.96s/it]
 61%|██████    | 6036/9864 [8:48:09<16:15:38, 15.29s/it]
 61%|██████    | 6037/9864 [8:48:25<16:24:20, 15.43s/it]
 61%|██████    | 6038/9864 [8:48:39<16:03:51, 15.12s/it]
 61%|██████    | 6039/9864 [8:48:53<15:42:07, 14.78s/it]
 61%|██████    | 6040/9864 [8:49:09<16:06:04, 15.16s/it]
                                                        
{'loss': '0.1155', 'grad_norm': '3.816', 'learning_rate': '6.903e-06', 'epoch': '0.6124'}

 61%|██████    | 6040/9864 [8:49:09<16:06:04, 15.16s/it]
 61%|██████    | 6041/9864 [8:49:25<16:23:15, 15.43s/it]
 61%|██████▏   | 6042/9864 [8:49:39<15:58:27, 15.05s/it]
 61%|██████▏   | 6043/9864 [8:49:55<16:12:03, 15.26s/it]
 61%|██████▏   | 6044/9864 [8:50:09<15:52:39, 14.96s/it]
 61%|██████▏   | 6045/9864 [8:50:24<15:40:55, 14.78s/it]
 61%|██████▏   | 6046/9864 [8:50:40<16:04:24, 15.16s/it]
 61%|██████▏   | 6047/9864 [8:50:54<15:43:18, 14.83s/it]
 61%|██████▏   | 6048/9864 [8:51:11<16:28:09, 15.54s/it]
 61%|██████▏   | 6049/9864 [8:51:25<16:09:10, 15.24s/it]
 61%|██████▏   | 6050/9864 [8:51:40<15:48:57, 14.93s/it]
                                                        
{'loss': '0.1296', 'grad_norm': '3.086', 'learning_rate': '6.872e-06', 'epoch': '0.6134'}

 61%|██████▏   | 6050/9864 [8:51:40<15:48:57, 14.93s/it]
 61%|██████▏   | 6051/9864 [8:51:53<15:28:43, 14.61s/it]
 61%|██████▏   | 6052/9864 [8:52:10<16:04:54, 15.19s/it]
 61%|██████▏   | 6053/9864 [8:52:26<16:14:32, 15.34s/it]
 61%|██████▏   | 6054/9864 [8:52:40<15:52:29, 15.00s/it]
 61%|██████▏   | 6055/9864 [8:52:54<15:36:46, 14.76s/it]
 61%|██████▏   | 6056/9864 [8:53:10<15:55:38, 15.06s/it]
 61%|██████▏   | 6057/9864 [8:53:25<15:58:58, 15.11s/it]
 61%|██████▏   | 6058/9864 [8:53:40<15:48:46, 14.96s/it]
 61%|██████▏   | 6059/9864 [8:53:57<16:24:26, 15.52s/it]
 61%|██████▏   | 6060/9864 [8:54:12<16:20:42, 15.47s/it]
                                                        
{'loss': '0.1188', 'grad_norm': '3.089', 'learning_rate': '6.841e-06', 'epoch': '0.6144'}

 61%|██████▏   | 6060/9864 [8:54:12<16:20:42, 15.47s/it]
 61%|██████▏   | 6061/9864 [8:54:26<15:54:22, 15.06s/it]
 61%|██████▏   | 6062/9864 [8:54:41<15:48:22, 14.97s/it]
 61%|██████▏   | 6063/9864 [8:54:56<15:45:44, 14.93s/it]
 61%|██████▏   | 6064/9864 [8:55:12<16:05:11, 15.24s/it]
 61%|██████▏   | 6065/9864 [8:55:28<16:23:06, 15.53s/it]
 61%|██████▏   | 6066/9864 [8:55:42<15:57:01, 15.12s/it]
 62%|██████▏   | 6067/9864 [8:55:58<16:24:33, 15.56s/it]
 62%|██████▏   | 6068/9864 [8:56:14<16:32:19, 15.68s/it]
 62%|██████▏   | 6069/9864 [8:56:29<16:10:04, 15.34s/it]
 62%|██████▏   | 6070/9864 [8:56:43<15:53:04, 15.07s/it]
                                                        
{'loss': '0.1208', 'grad_norm': '3.492', 'learning_rate': '6.809e-06', 'epoch': '0.6154'}

 62%|██████▏   | 6070/9864 [8:56:43<15:53:04, 15.07s/it]
 62%|██████▏   | 6071/9864 [8:56:58<15:36:25, 14.81s/it]
 62%|██████▏   | 6072/9864 [8:57:15<16:27:45, 15.63s/it]
 62%|██████▏   | 6073/9864 [8:57:30<16:03:51, 15.25s/it]
 62%|██████▏   | 6074/9864 [8:57:44<15:44:50, 14.96s/it]
 62%|██████▏   | 6075/9864 [8:58:00<16:06:19, 15.30s/it]
 62%|██████▏   | 6076/9864 [8:58:15<16:10:08, 15.37s/it]
 62%|██████▏   | 6077/9864 [8:58:30<15:51:25, 15.07s/it]
 62%|██████▏   | 6078/9864 [8:58:44<15:30:34, 14.75s/it]
 62%|██████▏   | 6079/9864 [8:59:02<16:29:02, 15.68s/it]
 62%|██████▏   | 6080/9864 [8:59:16<16:05:46, 15.31s/it]
                                                        
{'loss': '0.1073', 'grad_norm': '2.923', 'learning_rate': '6.778e-06', 'epoch': '0.6164'}

 62%|██████▏   | 6080/9864 [8:59:16<16:05:46, 15.31s/it]
 62%|██████▏   | 6081/9864 [8:59:31<15:47:33, 15.03s/it]
 62%|██████▏   | 6082/9864 [8:59:47<16:08:08, 15.36s/it]
 62%|██████▏   | 6083/9864 [9:00:02<15:58:49, 15.22s/it]
 62%|██████▏   | 6084/9864 [9:00:16<15:40:31, 14.93s/it]
 62%|██████▏   | 6085/9864 [9:00:31<15:55:01, 15.16s/it]
 62%|██████▏   | 6086/9864 [9:00:46<15:43:03, 14.98s/it]
 62%|██████▏   | 6087/9864 [9:01:04<16:33:14, 15.78s/it]
 62%|██████▏   | 6088/9864 [9:01:18<16:10:59, 15.43s/it]
 62%|██████▏   | 6089/9864 [9:01:32<15:46:17, 15.04s/it]
 62%|██████▏   | 6090/9864 [9:01:49<16:08:48, 15.40s/it]
                                                        
{'loss': '0.1103', 'grad_norm': '3.442', 'learning_rate': '6.747e-06', 'epoch': '0.6174'}

 62%|██████▏   | 6090/9864 [9:01:49<16:08:48, 15.40s/it]
 62%|██████▏   | 6091/9864 [9:02:03<15:49:59, 15.11s/it]
 62%|██████▏   | 6092/9864 [9:02:17<15:34:43, 14.87s/it]
 62%|██████▏   | 6093/9864 [9:02:34<15:59:34, 15.27s/it]
 62%|██████▏   | 6094/9864 [9:02:52<16:56:18, 16.17s/it]
 62%|██████▏   | 6095/9864 [9:03:06<16:25:44, 15.69s/it]
 62%|██████▏   | 6096/9864 [9:03:21<16:05:08, 15.37s/it]
 62%|██████▏   | 6097/9864 [9:03:36<15:49:36, 15.13s/it]
 62%|██████▏   | 6098/9864 [9:03:50<15:34:10, 14.88s/it]
 62%|██████▏   | 6099/9864 [9:04:06<15:51:58, 15.17s/it]
 62%|██████▏   | 6100/9864 [9:04:22<16:11:16, 15.48s/it]
                                                        
{'loss': '0.125', 'grad_norm': '5.378', 'learning_rate': '6.716e-06', 'epoch': '0.6185'}

 62%|██████▏   | 6100/9864 [9:04:22<16:11:16, 15.48s/it]
 62%|██████▏   | 6101/9864 [9:04:36<15:48:48, 15.13s/it]
 62%|██████▏   | 6102/9864 [9:04:55<16:47:10, 16.06s/it]
 62%|██████▏   | 6103/9864 [9:05:09<16:13:04, 15.52s/it]
 62%|██████▏   | 6104/9864 [9:05:23<15:49:01, 15.14s/it]
 62%|██████▏   | 6105/9864 [9:05:38<15:46:24, 15.11s/it]
 62%|██████▏   | 6106/9864 [9:05:53<15:36:53, 14.96s/it]
 62%|██████▏   | 6107/9864 [9:06:08<15:51:10, 15.19s/it]
 62%|██████▏   | 6108/9864 [9:06:24<16:04:10, 15.40s/it]
 62%|██████▏   | 6109/9864 [9:06:39<15:55:18, 15.26s/it]
 62%|██████▏   | 6110/9864 [9:06:58<16:59:11, 16.29s/it]
                                                        
{'loss': '0.1245', 'grad_norm': '3.264', 'learning_rate': '6.685e-06', 'epoch': '0.6195'}

 62%|██████▏   | 6110/9864 [9:06:58<16:59:11, 16.29s/it]
 62%|██████▏   | 6111/9864 [9:07:12<16:23:58, 15.73s/it]
 62%|██████▏   | 6112/9864 [9:07:27<15:58:52, 15.33s/it]
 62%|██████▏   | 6113/9864 [9:07:41<15:44:30, 15.11s/it]
 62%|██████▏   | 6114/9864 [9:07:55<15:25:08, 14.80s/it]
 62%|██████▏   | 6115/9864 [9:08:12<15:55:07, 15.29s/it]
 62%|██████▏   | 6116/9864 [9:08:28<16:04:50, 15.45s/it]
 62%|██████▏   | 6117/9864 [9:08:42<15:34:06, 14.96s/it]
 62%|██████▏   | 6118/9864 [9:08:59<16:24:09, 15.76s/it]
 62%|██████▏   | 6119/9864 [9:09:14<16:03:50, 15.44s/it]
 62%|██████▏   | 6120/9864 [9:09:28<15:44:33, 15.14s/it]
                                                        
{'loss': '0.1326', 'grad_norm': '4.388', 'learning_rate': '6.654e-06', 'epoch': '0.6205'}

 62%|██████▏   | 6120/9864 [9:09:28<15:44:33, 15.14s/it]
 62%|██████▏   | 6121/9864 [9:09:43<15:28:01, 14.88s/it]
 62%|██████▏   | 6122/9864 [9:09:57<15:19:36, 14.75s/it]
 62%|██████▏   | 6123/9864 [9:10:14<16:02:25, 15.44s/it]
 62%|██████▏   | 6124/9864 [9:10:29<15:44:50, 15.16s/it]
 62%|██████▏   | 6125/9864 [9:10:45<16:05:18, 15.49s/it]
 62%|██████▏   | 6126/9864 [9:11:01<16:25:47, 15.82s/it]
 62%|██████▏   | 6127/9864 [9:11:16<15:54:24, 15.32s/it]
 62%|██████▏   | 6128/9864 [9:11:29<15:27:35, 14.90s/it]
 62%|██████▏   | 6129/9864 [9:11:44<15:14:55, 14.70s/it]
 62%|██████▏   | 6130/9864 [9:12:00<15:36:05, 15.04s/it]
                                                        
{'loss': '0.1285', 'grad_norm': '3.303', 'learning_rate': '6.623e-06', 'epoch': '0.6215'}

 62%|██████▏   | 6130/9864 [9:12:00<15:36:05, 15.04s/it]
 62%|██████▏   | 6131/9864 [9:12:14<15:19:47, 14.78s/it]
 62%|██████▏   | 6132/9864 [9:12:31<16:06:43, 15.54s/it]
 62%|██████▏   | 6133/9864 [9:12:45<15:44:29, 15.19s/it]
 62%|██████▏   | 6134/9864 [9:13:02<16:09:06, 15.59s/it]
 62%|██████▏   | 6135/9864 [9:13:16<15:42:33, 15.17s/it]
 62%|██████▏   | 6136/9864 [9:13:31<15:30:13, 14.97s/it]
 62%|██████▏   | 6137/9864 [9:13:44<15:07:30, 14.61s/it]
 62%|██████▏   | 6138/9864 [9:14:00<15:33:59, 15.04s/it]
 62%|██████▏   | 6139/9864 [9:14:17<15:53:27, 15.36s/it]
 62%|██████▏   | 6140/9864 [9:14:33<16:09:46, 15.62s/it]
                                                        
{'loss': '0.1262', 'grad_norm': '4.729', 'learning_rate': '6.592e-06', 'epoch': '0.6225'}

 62%|██████▏   | 6140/9864 [9:14:33<16:09:46, 15.62s/it]
 62%|██████▏   | 6141/9864 [9:14:49<16:16:27, 15.74s/it]
 62%|██████▏   | 6142/9864 [9:15:03<15:52:35, 15.36s/it]
 62%|██████▏   | 6143/9864 [9:15:17<15:27:34, 14.96s/it]
 62%|██████▏   | 6144/9864 [9:15:32<15:22:41, 14.88s/it]
 62%|██████▏   | 6145/9864 [9:15:48<15:51:49, 15.36s/it]
 62%|██████▏   | 6146/9864 [9:16:03<15:34:49, 15.09s/it]
 62%|██████▏   | 6147/9864 [9:16:19<15:55:29, 15.42s/it]
 62%|██████▏   | 6148/9864 [9:16:35<16:03:08, 15.55s/it]
 62%|██████▏   | 6149/9864 [9:16:51<16:03:21, 15.56s/it]
 62%|██████▏   | 6150/9864 [9:17:04<15:30:28, 15.03s/it]
                                                        
{'loss': '0.1073', 'grad_norm': '4.616', 'learning_rate': '6.562e-06', 'epoch': '0.6235'}

 62%|██████▏   | 6150/9864 [9:17:04<15:30:28, 15.03s/it]
 62%|██████▏   | 6151/9864 [9:17:19<15:19:40, 14.86s/it]
 62%|██████▏   | 6152/9864 [9:17:33<14:59:05, 14.53s/it]
 62%|██████▏   | 6153/9864 [9:17:49<15:26:45, 14.98s/it]
 62%|██████▏   | 6154/9864 [9:18:03<15:12:39, 14.76s/it]
 62%|██████▏   | 6155/9864 [9:18:20<16:02:22, 15.57s/it]
 62%|██████▏   | 6156/9864 [9:18:35<15:38:57, 15.19s/it]
 62%|██████▏   | 6157/9864 [9:18:51<15:59:32, 15.53s/it]
 62%|██████▏   | 6158/9864 [9:19:05<15:37:08, 15.17s/it]
 62%|██████▏   | 6159/9864 [9:19:20<15:26:34, 15.01s/it]
 62%|██████▏   | 6160/9864 [9:19:35<15:22:14, 14.94s/it]
                                                        
{'loss': '0.1244', 'grad_norm': '5.626', 'learning_rate': '6.531e-06', 'epoch': '0.6245'}

 62%|██████▏   | 6160/9864 [9:19:35<15:22:14, 14.94s/it]
 62%|██████▏   | 6161/9864 [9:19:49<15:16:09, 14.84s/it]
 62%|██████▏   | 6162/9864 [9:20:07<16:11:14, 15.74s/it]
 62%|██████▏   | 6163/9864 [9:20:24<16:27:53, 16.02s/it]
 62%|██████▏   | 6164/9864 [9:20:38<16:00:49, 15.58s/it]
 62%|██████▎   | 6165/9864 [9:20:54<16:07:09, 15.69s/it]
 63%|██████▎   | 6166/9864 [9:21:09<15:40:12, 15.25s/it]
 63%|██████▎   | 6167/9864 [9:21:23<15:20:57, 14.95s/it]
 63%|██████▎   | 6168/9864 [9:21:37<15:12:37, 14.82s/it]
 63%|██████▎   | 6169/9864 [9:21:52<15:05:40, 14.71s/it]
 63%|██████▎   | 6170/9864 [9:22:11<16:35:23, 16.17s/it]
                                                        
{'loss': '0.1194', 'grad_norm': '3.501', 'learning_rate': '6.5e-06', 'epoch': '0.6256'}

 63%|██████▎   | 6170/9864 [9:22:11<16:35:23, 16.17s/it]
 63%|██████▎   | 6171/9864 [9:22:26<16:01:00, 15.61s/it]
 63%|██████▎   | 6172/9864 [9:22:40<15:45:49, 15.37s/it]
 63%|██████▎   | 6173/9864 [9:22:57<16:00:01, 15.61s/it]
 63%|██████▎   | 6174/9864 [9:23:11<15:36:49, 15.23s/it]
 63%|██████▎   | 6175/9864 [9:23:25<15:13:43, 14.86s/it]
 63%|██████▎   | 6176/9864 [9:23:39<14:54:14, 14.55s/it]
 63%|██████▎   | 6177/9864 [9:23:55<15:20:07, 14.97s/it]
 63%|██████▎   | 6178/9864 [9:24:12<16:04:29, 15.70s/it]
 63%|██████▎   | 6179/9864 [9:24:26<15:35:19, 15.23s/it]
 63%|██████▎   | 6180/9864 [9:24:41<15:22:05, 15.02s/it]
                                                        
{'loss': '0.1067', 'grad_norm': '3.225', 'learning_rate': '6.469e-06', 'epoch': '0.6266'}

 63%|██████▎   | 6180/9864 [9:24:41<15:22:05, 15.02s/it]
 63%|██████▎   | 6181/9864 [9:24:57<15:42:17, 15.35s/it]
 63%|██████▎   | 6182/9864 [9:25:11<15:21:15, 15.01s/it]
 63%|██████▎   | 6183/9864 [9:25:25<15:03:18, 14.72s/it]
 63%|██████▎   | 6184/9864 [9:25:40<15:03:06, 14.72s/it]
 63%|██████▎   | 6185/9864 [9:25:56<15:31:00, 15.18s/it]
 63%|██████▎   | 6186/9864 [9:26:14<16:19:13, 15.97s/it]
 63%|██████▎   | 6187/9864 [9:26:28<15:51:32, 15.53s/it]
 63%|██████▎   | 6188/9864 [9:26:43<15:32:32, 15.22s/it]
 63%|██████▎   | 6189/9864 [9:26:59<15:51:11, 15.53s/it]
 63%|██████▎   | 6190/9864 [9:27:13<15:17:15, 14.98s/it]
                                                        
{'loss': '0.1274', 'grad_norm': '3.298', 'learning_rate': '6.439e-06', 'epoch': '0.6276'}

 63%|██████▎   | 6190/9864 [9:27:13<15:17:15, 14.98s/it]
 63%|██████▎   | 6191/9864 [9:27:28<15:18:29, 15.00s/it]
 63%|██████▎   | 6192/9864 [9:27:42<15:08:29, 14.84s/it]
 63%|██████▎   | 6193/9864 [9:27:58<15:23:43, 15.10s/it]
 63%|██████▎   | 6194/9864 [9:28:16<16:11:27, 15.88s/it]
 63%|██████▎   | 6195/9864 [9:28:30<15:44:05, 15.44s/it]
 63%|██████▎   | 6196/9864 [9:28:44<15:08:02, 14.85s/it]
 63%|██████▎   | 6197/9864 [9:28:59<15:16:00, 14.99s/it]
 63%|██████▎   | 6198/9864 [9:29:13<15:02:24, 14.77s/it]
 63%|██████▎   | 6199/9864 [9:29:27<14:48:58, 14.55s/it]
 63%|██████▎   | 6200/9864 [9:29:42<14:45:23, 14.50s/it]
                                                        
{'loss': '0.1086', 'grad_norm': '4.011', 'learning_rate': '6.408e-06', 'epoch': '0.6286'}

 63%|██████▎   | 6200/9864 [9:29:42<14:45:23, 14.50s/it]
 63%|██████▎   | 6201/9864 [9:29:57<15:06:23, 14.85s/it]
 63%|██████▎   | 6202/9864 [9:30:15<16:02:32, 15.77s/it]
 63%|██████▎   | 6203/9864 [9:30:30<15:41:24, 15.43s/it]
 63%|██████▎   | 6204/9864 [9:30:46<16:00:32, 15.75s/it]
 63%|██████▎   | 6205/9864 [9:31:01<15:32:54, 15.30s/it]
 63%|██████▎   | 6206/9864 [9:31:15<15:18:37, 15.07s/it]
 63%|██████▎   | 6207/9864 [9:31:29<15:01:03, 14.78s/it]
 63%|██████▎   | 6208/9864 [9:31:44<14:52:19, 14.64s/it]
 63%|██████▎   | 6209/9864 [9:32:00<15:22:22, 15.14s/it]
 63%|██████▎   | 6210/9864 [9:32:18<16:13:58, 15.99s/it]
                                                        
{'loss': '0.1179', 'grad_norm': '2.985', 'learning_rate': '6.377e-06', 'epoch': '0.6296'}

 63%|██████▎   | 6210/9864 [9:32:18<16:13:58, 15.99s/it]
 63%|██████▎   | 6211/9864 [9:32:32<15:44:18, 15.51s/it]
 63%|██████▎   | 6212/9864 [9:32:48<15:55:27, 15.70s/it]
 63%|██████▎   | 6213/9864 [9:33:03<15:26:43, 15.23s/it]
 63%|██████▎   | 6214/9864 [9:33:17<15:13:56, 15.02s/it]
 63%|██████▎   | 6215/9864 [9:33:32<15:19:12, 15.11s/it]
 63%|██████▎   | 6216/9864 [9:33:47<15:02:59, 14.85s/it]
 63%|██████▎   | 6217/9864 [9:34:03<15:26:50, 15.25s/it]
 63%|██████▎   | 6218/9864 [9:34:19<15:43:26, 15.53s/it]
 63%|██████▎   | 6219/9864 [9:34:35<15:58:43, 15.78s/it]
 63%|██████▎   | 6220/9864 [9:34:51<16:00:10, 15.81s/it]
                                                        
{'loss': '0.1224', 'grad_norm': '3.89', 'learning_rate': '6.347e-06', 'epoch': '0.6306'}

 63%|██████▎   | 6220/9864 [9:34:51<16:00:10, 15.81s/it]
 63%|██████▎   | 6221/9864 [9:35:05<15:26:26, 15.26s/it]
 63%|██████▎   | 6222/9864 [9:35:19<15:02:46, 14.87s/it]
 63%|██████▎   | 6223/9864 [9:35:33<14:50:09, 14.67s/it]
 63%|██████▎   | 6224/9864 [9:35:48<14:47:14, 14.62s/it]
 63%|██████▎   | 6225/9864 [9:36:04<15:14:47, 15.08s/it]
 63%|██████▎   | 6226/9864 [9:36:20<15:30:13, 15.34s/it]
 63%|██████▎   | 6227/9864 [9:36:36<15:44:16, 15.58s/it]
 63%|██████▎   | 6228/9864 [9:36:52<15:47:43, 15.64s/it]
 63%|██████▎   | 6229/9864 [9:37:06<15:25:32, 15.28s/it]
 63%|██████▎   | 6230/9864 [9:37:21<15:06:06, 14.96s/it]
                                                        
{'loss': '0.1108', 'grad_norm': '4.455', 'learning_rate': '6.316e-06', 'epoch': '0.6316'}

 63%|██████▎   | 6230/9864 [9:37:21<15:06:06, 14.96s/it]
 63%|██████▎   | 6231/9864 [9:37:35<14:57:59, 14.83s/it]
 63%|██████▎   | 6232/9864 [9:37:49<14:43:31, 14.60s/it]
 63%|██████▎   | 6233/9864 [9:38:03<14:34:17, 14.45s/it]
 63%|██████▎   | 6234/9864 [9:38:20<15:17:37, 15.17s/it]
 63%|██████▎   | 6235/9864 [9:38:38<16:07:25, 15.99s/it]
 63%|██████▎   | 6236/9864 [9:38:54<16:11:54, 16.07s/it]
 63%|██████▎   | 6237/9864 [9:39:09<15:43:18, 15.60s/it]
 63%|██████▎   | 6238/9864 [9:39:23<15:14:36, 15.13s/it]
 63%|██████▎   | 6239/9864 [9:39:37<15:03:59, 14.96s/it]
 63%|██████▎   | 6240/9864 [9:39:52<14:47:56, 14.70s/it]
                                                        
{'loss': '0.1189', 'grad_norm': '3.594', 'learning_rate': '6.286e-06', 'epoch': '0.6327'}

 63%|██████▎   | 6240/9864 [9:39:52<14:47:56, 14.70s/it]
 63%|██████▎   | 6241/9864 [9:40:06<14:36:57, 14.52s/it]
 63%|██████▎   | 6242/9864 [9:40:22<15:09:51, 15.07s/it]
 63%|██████▎   | 6243/9864 [9:40:42<16:35:04, 16.49s/it]
 63%|██████▎   | 6244/9864 [9:40:56<16:00:37, 15.92s/it]
 63%|██████▎   | 6245/9864 [9:41:11<15:32:06, 15.45s/it]
 63%|██████▎   | 6246/9864 [9:41:25<15:19:10, 15.24s/it]
 63%|██████▎   | 6247/9864 [9:41:40<15:07:04, 15.05s/it]
 63%|██████▎   | 6248/9864 [9:41:54<14:48:49, 14.75s/it]
 63%|██████▎   | 6249/9864 [9:42:08<14:35:56, 14.54s/it]
 63%|██████▎   | 6250/9864 [9:42:27<15:50:26, 15.78s/it]
                                                        
{'loss': '0.109', 'grad_norm': '3.838', 'learning_rate': '6.255e-06', 'epoch': '0.6337'}

 63%|██████▎   | 6250/9864 [9:42:27<15:50:26, 15.78s/it]
 63%|██████▎   | 6251/9864 [9:42:45<16:30:46, 16.45s/it]
 63%|██████▎   | 6252/9864 [9:42:59<15:49:00, 15.76s/it]
 63%|██████▎   | 6253/9864 [9:43:13<15:17:21, 15.24s/it]
 63%|██████▎   | 6254/9864 [9:43:27<14:58:03, 14.93s/it]
 63%|██████▎   | 6255/9864 [9:43:41<14:42:50, 14.68s/it]
 63%|██████▎   | 6256/9864 [9:43:56<14:42:56, 14.68s/it]
 63%|██████▎   | 6257/9864 [9:44:12<15:09:12, 15.12s/it]
 63%|██████▎   | 6258/9864 [9:44:30<16:03:56, 16.04s/it]
 63%|██████▎   | 6259/9864 [9:44:47<16:06:10, 16.08s/it]
 63%|██████▎   | 6260/9864 [9:45:01<15:36:58, 15.60s/it]
                                                        
{'loss': '0.1078', 'grad_norm': '4.202', 'learning_rate': '6.225e-06', 'epoch': '0.6347'}

 63%|██████▎   | 6260/9864 [9:45:01<15:36:58, 15.60s/it]
 63%|██████▎   | 6261/9864 [9:45:16<15:20:32, 15.33s/it]
 63%|██████▎   | 6262/9864 [9:45:30<15:04:09, 15.06s/it]
 63%|██████▎   | 6263/9864 [9:45:44<14:46:49, 14.78s/it]
 64%|██████▎   | 6264/9864 [9:45:58<14:35:49, 14.60s/it]
 64%|██████▎   | 6265/9864 [9:46:14<15:00:55, 15.02s/it]
 64%|██████▎   | 6266/9864 [9:46:33<16:00:28, 16.02s/it]
 64%|██████▎   | 6267/9864 [9:46:49<16:03:12, 16.07s/it]
 64%|██████▎   | 6268/9864 [9:47:04<15:39:37, 15.68s/it]
 64%|██████▎   | 6269/9864 [9:47:19<15:22:58, 15.40s/it]
 64%|██████▎   | 6270/9864 [9:47:33<15:04:09, 15.09s/it]
                                                        
{'loss': '0.1146', 'grad_norm': '4.629', 'learning_rate': '6.194e-06', 'epoch': '0.6357'}

 64%|██████▎   | 6270/9864 [9:47:33<15:04:09, 15.09s/it]
 64%|██████▎   | 6271/9864 [9:47:47<14:52:33, 14.91s/it]
 64%|██████▎   | 6272/9864 [9:48:01<14:33:49, 14.60s/it]
 64%|██████▎   | 6273/9864 [9:48:17<15:01:29, 15.06s/it]
 64%|██████▎   | 6274/9864 [9:48:36<15:58:20, 16.02s/it]
 64%|██████▎   | 6275/9864 [9:48:52<15:57:09, 16.00s/it]
 64%|██████▎   | 6276/9864 [9:49:06<15:27:19, 15.51s/it]
 64%|██████▎   | 6277/9864 [9:49:20<15:09:55, 15.22s/it]
 64%|██████▎   | 6278/9864 [9:49:35<14:56:52, 15.01s/it]
 64%|██████▎   | 6279/9864 [9:49:49<14:46:44, 14.84s/it]
 64%|██████▎   | 6280/9864 [9:50:04<14:37:21, 14.69s/it]
                                                        
{'loss': '0.1086', 'grad_norm': '4.741', 'learning_rate': '6.164e-06', 'epoch': '0.6367'}

 64%|██████▎   | 6280/9864 [9:50:04<14:37:21, 14.69s/it]
 64%|██████▎   | 6281/9864 [9:50:22<15:32:57, 15.62s/it]
 64%|██████▎   | 6282/9864 [9:50:37<15:28:51, 15.56s/it]
 64%|██████▎   | 6283/9864 [9:50:53<15:30:26, 15.59s/it]
 64%|██████▎   | 6284/9864 [9:51:07<15:09:57, 15.25s/it]
 64%|██████▎   | 6285/9864 [9:51:21<14:47:36, 14.88s/it]
 64%|██████▎   | 6286/9864 [9:51:35<14:31:35, 14.62s/it]
 64%|██████▎   | 6287/9864 [9:51:49<14:25:57, 14.53s/it]
 64%|██████▎   | 6288/9864 [9:52:04<14:23:46, 14.49s/it]
 64%|██████▍   | 6289/9864 [9:52:18<14:19:54, 14.43s/it]
 64%|██████▍   | 6290/9864 [9:52:37<15:30:25, 15.62s/it]
                                                        
{'loss': '0.1256', 'grad_norm': '3.987', 'learning_rate': '6.134e-06', 'epoch': '0.6377'}

 64%|██████▍   | 6290/9864 [9:52:37<15:30:25, 15.62s/it]
 64%|██████▍   | 6291/9864 [9:52:55<16:15:45, 16.39s/it]
 64%|██████▍   | 6292/9864 [9:53:09<15:43:56, 15.86s/it]
 64%|██████▍   | 6293/9864 [9:53:24<15:23:39, 15.52s/it]
 64%|██████▍   | 6294/9864 [9:53:39<15:11:03, 15.31s/it]
 64%|██████▍   | 6295/9864 [9:53:54<15:01:03, 15.15s/it]
 64%|██████▍   | 6296/9864 [9:54:08<14:39:52, 14.80s/it]
 64%|██████▍   | 6297/9864 [9:54:24<15:06:28, 15.25s/it]
 64%|██████▍   | 6298/9864 [9:54:40<15:21:24, 15.50s/it]
 64%|██████▍   | 6299/9864 [9:54:59<16:19:34, 16.49s/it]
 64%|██████▍   | 6300/9864 [9:55:13<15:44:39, 15.90s/it]
                                                        
{'loss': '0.1118', 'grad_norm': '4.578', 'learning_rate': '6.104e-06', 'epoch': '0.6387'}

 64%|██████▍   | 6300/9864 [9:55:13<15:44:39, 15.90s/it]
 64%|██████▍   | 6301/9864 [9:55:28<15:24:39, 15.57s/it]
 64%|██████▍   | 6302/9864 [9:55:43<15:12:04, 15.36s/it]
 64%|██████▍   | 6303/9864 [9:55:58<15:01:35, 15.19s/it]
 64%|██████▍   | 6304/9864 [9:56:14<15:24:03, 15.57s/it]
 64%|██████▍   | 6305/9864 [9:56:30<15:34:39, 15.76s/it]
 64%|██████▍   | 6306/9864 [9:56:44<15:00:42, 15.19s/it]
 64%|██████▍   | 6307/9864 [9:57:02<15:37:15, 15.81s/it]
 64%|██████▍   | 6308/9864 [9:57:16<15:04:59, 15.27s/it]
 64%|██████▍   | 6309/9864 [9:57:30<14:47:44, 14.98s/it]
 64%|██████▍   | 6310/9864 [9:57:45<14:40:19, 14.86s/it]
                                                        
{'loss': '0.1187', 'grad_norm': '3.945', 'learning_rate': '6.073e-06', 'epoch': '0.6398'}

 64%|██████▍   | 6310/9864 [9:57:45<14:40:19, 14.86s/it]
 64%|██████▍   | 6311/9864 [9:57:58<14:22:23, 14.56s/it]
 64%|██████▍   | 6312/9864 [9:58:13<14:15:21, 14.45s/it]
 64%|██████▍   | 6313/9864 [9:58:31<15:28:24, 15.69s/it]
 64%|██████▍   | 6314/9864 [9:58:47<15:32:45, 15.76s/it]
 64%|██████▍   | 6315/9864 [9:59:04<15:44:47, 15.97s/it]
 64%|██████▍   | 6316/9864 [9:59:18<15:23:36, 15.62s/it]
 64%|██████▍   | 6317/9864 [9:59:33<15:03:34, 15.28s/it]
 64%|██████▍   | 6318/9864 [9:59:47<14:37:59, 14.86s/it]
 64%|██████▍   | 6319/9864 [10:00:01<14:23:33, 14.62s/it]
 64%|██████▍   | 6320/9864 [10:00:15<14:23:53, 14.63s/it]
                                                         
{'loss': '0.107', 'grad_norm': '3.44', 'learning_rate': '6.043e-06', 'epoch': '0.6408'}

 64%|██████▍   | 6320/9864 [10:00:15<14:23:53, 14.63s/it]
 64%|██████▍   | 6321/9864 [10:00:32<14:59:48, 15.24s/it]
 64%|██████▍   | 6322/9864 [10:00:49<15:37:17, 15.88s/it]
 64%|██████▍   | 6323/9864 [10:01:05<15:35:20, 15.85s/it]
 64%|██████▍   | 6324/9864 [10:01:20<15:10:47, 15.44s/it]
 64%|██████▍   | 6325/9864 [10:01:34<14:50:10, 15.09s/it]
 64%|██████▍   | 6326/9864 [10:01:48<14:38:46, 14.90s/it]
 64%|██████▍   | 6327/9864 [10:02:03<14:32:21, 14.80s/it]
 64%|██████▍   | 6328/9864 [10:02:17<14:18:47, 14.57s/it]
 64%|██████▍   | 6329/9864 [10:02:33<14:44:58, 15.02s/it]
 64%|██████▍   | 6330/9864 [10:02:51<15:40:49, 15.97s/it]
                                                         
{'loss': '0.1064', 'grad_norm': '5.456', 'learning_rate': '6.013e-06', 'epoch': '0.6418'}

 64%|██████▍   | 6330/9864 [10:02:51<15:40:49, 15.97s/it]
 64%|██████▍   | 6331/9864 [10:03:07<15:40:15, 15.97s/it]
 64%|██████▍   | 6332/9864 [10:03:22<15:15:38, 15.55s/it]
 64%|██████▍   | 6333/9864 [10:03:36<14:59:05, 15.28s/it]
 64%|██████▍   | 6334/9864 [10:03:50<14:36:47, 14.90s/it]
 64%|██████▍   | 6335/9864 [10:04:05<14:25:30, 14.72s/it]
 64%|██████▍   | 6336/9864 [10:04:21<14:49:16, 15.12s/it]
 64%|██████▍   | 6337/9864 [10:04:37<15:15:22, 15.57s/it]
 64%|██████▍   | 6338/9864 [10:04:53<15:21:15, 15.68s/it]
 64%|██████▍   | 6339/9864 [10:05:10<15:40:06, 16.00s/it]
 64%|██████▍   | 6340/9864 [10:05:25<15:12:49, 15.54s/it]
                                                         
{'loss': '0.1148', 'grad_norm': '4.297', 'learning_rate': '5.983e-06', 'epoch': '0.6428'}

 64%|██████▍   | 6340/9864 [10:05:25<15:12:49, 15.54s/it]
 64%|██████▍   | 6341/9864 [10:05:39<14:49:19, 15.15s/it]
 64%|██████▍   | 6342/9864 [10:05:53<14:32:35, 14.87s/it]
 64%|██████▍   | 6343/9864 [10:06:08<14:30:22, 14.83s/it]
 64%|██████▍   | 6344/9864 [10:06:24<14:55:29, 15.26s/it]
 64%|██████▍   | 6345/9864 [10:06:40<15:10:07, 15.52s/it]
 64%|██████▍   | 6346/9864 [10:06:58<15:43:33, 16.09s/it]
 64%|██████▍   | 6347/9864 [10:07:12<15:11:12, 15.55s/it]
 64%|██████▍   | 6348/9864 [10:07:25<14:32:41, 14.89s/it]
 64%|██████▍   | 6349/9864 [10:07:40<14:21:19, 14.70s/it]
 64%|██████▍   | 6350/9864 [10:07:54<14:22:58, 14.73s/it]
                                                         
{'loss': '0.1022', 'grad_norm': '3.432', 'learning_rate': '5.953e-06', 'epoch': '0.6438'}

 64%|██████▍   | 6350/9864 [10:07:54<14:22:58, 14.73s/it]
 64%|██████▍   | 6351/9864 [10:08:11<14:58:19, 15.34s/it]
 64%|██████▍   | 6352/9864 [10:08:25<14:26:25, 14.80s/it]
 64%|██████▍   | 6353/9864 [10:08:42<15:14:47, 15.63s/it]
 64%|██████▍   | 6354/9864 [10:08:56<14:49:39, 15.21s/it]
 64%|██████▍   | 6355/9864 [10:09:13<15:10:41, 15.57s/it]
 64%|██████▍   | 6356/9864 [10:09:27<14:51:51, 15.25s/it]
 64%|██████▍   | 6357/9864 [10:09:41<14:26:03, 14.82s/it]
 64%|██████▍   | 6358/9864 [10:09:56<14:18:44, 14.70s/it]
 64%|██████▍   | 6359/9864 [10:10:11<14:36:32, 15.00s/it]
 64%|██████▍   | 6360/9864 [10:10:26<14:31:51, 14.93s/it]
                                                         
{'loss': '0.1168', 'grad_norm': '5.928', 'learning_rate': '5.923e-06', 'epoch': '0.6448'}

 64%|██████▍   | 6360/9864 [10:10:26<14:31:51, 14.93s/it]
 64%|██████▍   | 6361/9864 [10:10:44<15:22:53, 15.81s/it]
 64%|██████▍   | 6362/9864 [10:11:00<15:22:41, 15.81s/it]
 65%|██████▍   | 6363/9864 [10:11:14<15:00:55, 15.44s/it]
 65%|██████▍   | 6364/9864 [10:11:29<14:39:09, 15.07s/it]
 65%|██████▍   | 6365/9864 [10:11:43<14:30:31, 14.93s/it]
 65%|██████▍   | 6366/9864 [10:11:59<14:53:42, 15.33s/it]
 65%|██████▍   | 6367/9864 [10:12:13<14:25:47, 14.85s/it]
 65%|██████▍   | 6368/9864 [10:12:27<14:14:46, 14.67s/it]
 65%|██████▍   | 6369/9864 [10:12:44<14:42:35, 15.15s/it]
 65%|██████▍   | 6370/9864 [10:13:00<14:58:39, 15.43s/it]
                                                         
{'loss': '0.09569', 'grad_norm': '2.92', 'learning_rate': '5.893e-06', 'epoch': '0.6458'}

 65%|██████▍   | 6370/9864 [10:13:00<14:58:39, 15.43s/it]
 65%|██████▍   | 6371/9864 [10:13:14<14:41:29, 15.14s/it]
 65%|██████▍   | 6372/9864 [10:13:28<14:22:00, 14.81s/it]
 65%|██████▍   | 6373/9864 [10:13:42<14:11:15, 14.63s/it]
 65%|██████▍   | 6374/9864 [10:13:59<14:42:09, 15.17s/it]
 65%|██████▍   | 6375/9864 [10:14:13<14:31:54, 14.99s/it]
 65%|██████▍   | 6376/9864 [10:14:30<14:51:46, 15.34s/it]
 65%|██████▍   | 6377/9864 [10:14:45<14:58:46, 15.46s/it]
 65%|██████▍   | 6378/9864 [10:15:01<15:06:29, 15.60s/it]
 65%|██████▍   | 6379/9864 [10:15:16<14:45:57, 15.25s/it]
 65%|██████▍   | 6380/9864 [10:15:30<14:34:07, 15.05s/it]
                                                         
{'loss': '0.114', 'grad_norm': '3.56', 'learning_rate': '5.863e-06', 'epoch': '0.6468'}

 65%|██████▍   | 6380/9864 [10:15:30<14:34:07, 15.05s/it]
 65%|██████▍   | 6381/9864 [10:15:45<14:24:21, 14.89s/it]
 65%|██████▍   | 6382/9864 [10:16:01<14:43:05, 15.22s/it]
 65%|██████▍   | 6383/9864 [10:16:15<14:21:14, 14.84s/it]
 65%|██████▍   | 6384/9864 [10:16:31<14:48:30, 15.32s/it]
 65%|██████▍   | 6385/9864 [10:16:49<15:37:13, 16.16s/it]
 65%|██████▍   | 6386/9864 [10:17:03<15:00:26, 15.53s/it]
 65%|██████▍   | 6387/9864 [10:17:17<14:29:01, 15.00s/it]
 65%|██████▍   | 6388/9864 [10:17:31<14:16:58, 14.79s/it]
 65%|██████▍   | 6389/9864 [10:17:46<14:10:17, 14.68s/it]
 65%|██████▍   | 6390/9864 [10:18:02<14:30:12, 15.03s/it]
                                                         
{'loss': '0.1004', 'grad_norm': '3.625', 'learning_rate': '5.833e-06', 'epoch': '0.6479'}

 65%|██████▍   | 6390/9864 [10:18:02<14:30:12, 15.03s/it]
 65%|██████▍   | 6391/9864 [10:18:17<14:40:51, 15.22s/it]
 65%|██████▍   | 6392/9864 [10:18:34<15:13:01, 15.78s/it]
 65%|██████▍   | 6393/9864 [10:18:49<14:49:33, 15.38s/it]
 65%|██████▍   | 6394/9864 [10:19:05<15:03:41, 15.63s/it]
 65%|██████▍   | 6395/9864 [10:19:19<14:34:53, 15.13s/it]
 65%|██████▍   | 6396/9864 [10:19:34<14:24:27, 14.96s/it]
 65%|██████▍   | 6397/9864 [10:19:49<14:39:00, 15.21s/it]
 65%|██████▍   | 6398/9864 [10:20:03<14:18:45, 14.87s/it]
 65%|██████▍   | 6399/9864 [10:20:21<14:56:12, 15.52s/it]
 65%|██████▍   | 6400/9864 [10:20:35<14:31:09, 15.09s/it]
                                                         
{'loss': '0.1128', 'grad_norm': '4.669', 'learning_rate': '5.803e-06', 'epoch': '0.6489'}

 65%|██████▍   | 6400/9864 [10:20:35<14:31:09, 15.09s/it]
 65%|██████▍   | 6401/9864 [10:20:50<14:44:33, 15.33s/it]
 65%|██████▍   | 6402/9864 [10:21:05<14:28:21, 15.05s/it]
 65%|██████▍   | 6403/9864 [10:21:19<14:17:20, 14.86s/it]
 65%|██████▍   | 6404/9864 [10:21:34<14:06:57, 14.69s/it]
 65%|██████▍   | 6405/9864 [10:21:50<14:28:30, 15.07s/it]
 65%|██████▍   | 6406/9864 [10:22:04<14:16:29, 14.86s/it]
 65%|██████▍   | 6407/9864 [10:22:22<15:07:08, 15.74s/it]
 65%|██████▍   | 6408/9864 [10:22:38<15:16:24, 15.91s/it]
 65%|██████▍   | 6409/9864 [10:22:52<14:47:43, 15.42s/it]
 65%|██████▍   | 6410/9864 [10:23:06<14:23:13, 15.00s/it]
                                                         
{'loss': '0.09384', 'grad_norm': '3.389', 'learning_rate': '5.773e-06', 'epoch': '0.6499'}

 65%|██████▍   | 6410/9864 [10:23:06<14:23:13, 15.00s/it]
 65%|██████▍   | 6411/9864 [10:23:21<14:09:53, 14.77s/it]
 65%|██████▌   | 6412/9864 [10:23:35<14:01:59, 14.63s/it]
 65%|██████▌   | 6413/9864 [10:23:49<13:55:40, 14.53s/it]
 65%|██████▌   | 6414/9864 [10:24:06<14:29:40, 15.12s/it]
 65%|██████▌   | 6415/9864 [10:24:24<15:17:03, 15.95s/it]
 65%|██████▌   | 6416/9864 [10:24:39<15:13:46, 15.90s/it]
 65%|██████▌   | 6417/9864 [10:24:54<14:48:17, 15.46s/it]
 65%|██████▌   | 6418/9864 [10:25:09<14:39:06, 15.31s/it]
 65%|██████▌   | 6419/9864 [10:25:23<14:18:00, 14.94s/it]
 65%|██████▌   | 6420/9864 [10:25:37<14:08:10, 14.78s/it]
                                                         
{'loss': '0.1203', 'grad_norm': '3.744', 'learning_rate': '5.744e-06', 'epoch': '0.6509'}

 65%|██████▌   | 6420/9864 [10:25:37<14:08:10, 14.78s/it]
 65%|██████▌   | 6421/9864 [10:25:51<13:57:36, 14.60s/it]
 65%|██████▌   | 6422/9864 [10:26:09<14:56:36, 15.63s/it]
 65%|██████▌   | 6423/9864 [10:26:24<14:42:09, 15.38s/it]
 65%|██████▌   | 6424/9864 [10:26:42<15:27:07, 16.17s/it]
 65%|██████▌   | 6425/9864 [10:26:57<15:00:05, 15.70s/it]
 65%|██████▌   | 6426/9864 [10:27:11<14:35:57, 15.29s/it]
 65%|██████▌   | 6427/9864 [10:27:25<14:16:53, 14.96s/it]
 65%|██████▌   | 6428/9864 [10:27:40<14:12:06, 14.88s/it]
 65%|██████▌   | 6429/9864 [10:27:56<14:29:51, 15.19s/it]
 65%|██████▌   | 6430/9864 [10:28:12<14:41:23, 15.40s/it]
                                                         
{'loss': '0.08772', 'grad_norm': '3.416', 'learning_rate': '5.714e-06', 'epoch': '0.6519'}

 65%|██████▌   | 6430/9864 [10:28:12<14:41:23, 15.40s/it]
 65%|██████▌   | 6431/9864 [10:28:26<14:13:46, 14.92s/it]
 65%|██████▌   | 6432/9864 [10:28:42<14:41:21, 15.41s/it]
 65%|██████▌   | 6433/9864 [10:28:58<14:48:54, 15.54s/it]
 65%|██████▌   | 6434/9864 [10:29:12<14:25:50, 15.15s/it]
 65%|██████▌   | 6435/9864 [10:29:27<14:12:42, 14.92s/it]
 65%|██████▌   | 6436/9864 [10:29:41<14:05:08, 14.79s/it]
 65%|██████▌   | 6437/9864 [10:29:57<14:26:57, 15.18s/it]
 65%|██████▌   | 6438/9864 [10:30:14<14:45:43, 15.51s/it]
 65%|██████▌   | 6439/9864 [10:30:28<14:31:24, 15.27s/it]
 65%|██████▌   | 6440/9864 [10:30:43<14:19:53, 15.07s/it]
                                                         
{'loss': '0.1103', 'grad_norm': '3.836', 'learning_rate': '5.684e-06', 'epoch': '0.6529'}

 65%|██████▌   | 6440/9864 [10:30:43<14:19:53, 15.07s/it]
 65%|██████▌   | 6441/9864 [10:31:01<15:17:54, 16.09s/it]
 65%|██████▌   | 6442/9864 [10:31:16<14:49:07, 15.59s/it]
 65%|██████▌   | 6443/9864 [10:31:30<14:31:46, 15.29s/it]
 65%|██████▌   | 6444/9864 [10:31:46<14:46:29, 15.55s/it]
 65%|██████▌   | 6445/9864 [10:32:01<14:21:17, 15.11s/it]
 65%|██████▌   | 6446/9864 [10:32:16<14:34:38, 15.35s/it]
 65%|██████▌   | 6447/9864 [10:32:31<14:25:03, 15.19s/it]
 65%|██████▌   | 6448/9864 [10:32:49<15:12:11, 16.02s/it]
 65%|██████▌   | 6449/9864 [10:33:04<14:46:55, 15.58s/it]
 65%|██████▌   | 6450/9864 [10:33:18<14:28:55, 15.27s/it]
                                                         
{'loss': '0.09616', 'grad_norm': '3.326', 'learning_rate': '5.655e-06', 'epoch': '0.6539'}

 65%|██████▌   | 6450/9864 [10:33:18<14:28:55, 15.27s/it]
 65%|██████▌   | 6451/9864 [10:33:33<14:10:15, 14.95s/it]
 65%|██████▌   | 6452/9864 [10:33:48<14:24:32, 15.20s/it]
 65%|██████▌   | 6453/9864 [10:34:03<14:07:16, 14.90s/it]
 65%|██████▌   | 6454/9864 [10:34:19<14:32:48, 15.36s/it]
 65%|██████▌   | 6455/9864 [10:34:34<14:23:35, 15.20s/it]
 65%|██████▌   | 6456/9864 [10:34:51<15:05:42, 15.95s/it]
 65%|██████▌   | 6457/9864 [10:35:06<14:36:20, 15.43s/it]
 65%|██████▌   | 6458/9864 [10:35:20<14:09:59, 14.97s/it]
 65%|██████▌   | 6459/9864 [10:35:34<14:06:39, 14.92s/it]
 65%|██████▌   | 6460/9864 [10:35:51<14:29:53, 15.33s/it]
                                                         
{'loss': '0.1046', 'grad_norm': '3.062', 'learning_rate': '5.625e-06', 'epoch': '0.655'}

 65%|██████▌   | 6460/9864 [10:35:51<14:29:53, 15.33s/it]
 66%|██████▌   | 6461/9864 [10:36:05<14:11:11, 15.01s/it]
 66%|██████▌   | 6462/9864 [10:36:21<14:33:52, 15.41s/it]
 66%|██████▌   | 6463/9864 [10:36:36<14:14:14, 15.07s/it]
 66%|██████▌   | 6464/9864 [10:36:52<14:38:32, 15.50s/it]
 66%|██████▌   | 6465/9864 [10:37:08<14:50:21, 15.72s/it]
 66%|██████▌   | 6466/9864 [10:37:23<14:27:41, 15.32s/it]
 66%|██████▌   | 6467/9864 [10:37:37<14:13:09, 15.07s/it]
 66%|██████▌   | 6468/9864 [10:37:53<14:29:37, 15.36s/it]
 66%|██████▌   | 6469/9864 [10:38:10<14:44:48, 15.64s/it]
 66%|██████▌   | 6470/9864 [10:38:23<14:14:24, 15.10s/it]
                                                         
{'loss': '0.1103', 'grad_norm': '4.323', 'learning_rate': '5.596e-06', 'epoch': '0.656'}

 66%|██████▌   | 6470/9864 [10:38:23<14:14:24, 15.10s/it]
 66%|██████▌   | 6471/9864 [10:38:37<13:55:20, 14.77s/it]
 66%|██████▌   | 6472/9864 [10:38:54<14:29:15, 15.38s/it]
 66%|██████▌   | 6473/9864 [10:39:11<14:47:46, 15.71s/it]
 66%|██████▌   | 6474/9864 [10:39:25<14:27:58, 15.36s/it]
 66%|██████▌   | 6475/9864 [10:39:40<14:09:40, 15.04s/it]
 66%|██████▌   | 6476/9864 [10:39:57<14:55:00, 15.85s/it]
 66%|██████▌   | 6477/9864 [10:40:12<14:30:50, 15.43s/it]
 66%|██████▌   | 6478/9864 [10:40:26<14:14:24, 15.14s/it]
 66%|██████▌   | 6479/9864 [10:40:41<14:01:57, 14.92s/it]
 66%|██████▌   | 6480/9864 [10:40:57<14:21:02, 15.27s/it]
                                                         
{'loss': '0.1035', 'grad_norm': '3.264', 'learning_rate': '5.566e-06', 'epoch': '0.657'}

 66%|██████▌   | 6480/9864 [10:40:57<14:21:02, 15.27s/it]
 66%|██████▌   | 6481/9864 [10:41:12<14:22:55, 15.30s/it]
 66%|██████▌   | 6482/9864 [10:41:26<14:01:35, 14.93s/it]
 66%|██████▌   | 6483/9864 [10:41:43<14:27:46, 15.40s/it]
 66%|██████▌   | 6484/9864 [10:41:59<14:46:44, 15.74s/it]
 66%|██████▌   | 6485/9864 [10:42:13<14:22:18, 15.31s/it]
 66%|██████▌   | 6486/9864 [10:42:28<14:07:14, 15.05s/it]
 66%|██████▌   | 6487/9864 [10:42:42<13:51:03, 14.77s/it]
 66%|██████▌   | 6488/9864 [10:42:58<14:13:31, 15.17s/it]
 66%|██████▌   | 6489/9864 [10:43:12<13:59:49, 14.93s/it]
 66%|██████▌   | 6490/9864 [10:43:28<14:17:40, 15.25s/it]
                                                         
{'loss': '0.1036', 'grad_norm': '4.836', 'learning_rate': '5.537e-06', 'epoch': '0.658'}

 66%|██████▌   | 6490/9864 [10:43:28<14:17:40, 15.25s/it]
 66%|██████▌   | 6491/9864 [10:43:44<14:28:36, 15.45s/it]
 66%|██████▌   | 6492/9864 [10:44:00<14:37:43, 15.62s/it]
 66%|██████▌   | 6493/9864 [10:44:15<14:16:40, 15.25s/it]
 66%|██████▌   | 6494/9864 [10:44:29<14:03:49, 15.02s/it]
 66%|██████▌   | 6495/9864 [10:44:44<13:58:48, 14.94s/it]
 66%|██████▌   | 6496/9864 [10:45:01<14:27:51, 15.46s/it]
 66%|██████▌   | 6497/9864 [10:45:15<14:10:22, 15.15s/it]
 66%|██████▌   | 6498/9864 [10:45:33<14:49:46, 15.86s/it]
 66%|██████▌   | 6499/9864 [10:45:47<14:24:43, 15.42s/it]
 66%|██████▌   | 6500/9864 [10:46:03<14:33:34, 15.58s/it]
                                                         
{'loss': '0.09976', 'grad_norm': '3.842', 'learning_rate': '5.508e-06', 'epoch': '0.659'}

 66%|██████▌   | 6500/9864 [10:46:03<14:33:34, 15.58s/it]
 66%|██████▌   | 6501/9864 [10:46:17<14:09:16, 15.15s/it]
 66%|██████▌   | 6502/9864 [10:46:31<13:50:17, 14.82s/it]
 66%|██████▌   | 6503/9864 [10:46:46<13:43:10, 14.70s/it]
 66%|██████▌   | 6504/9864 [10:47:02<14:05:34, 15.10s/it]
 66%|██████▌   | 6505/9864 [10:47:16<13:46:46, 14.77s/it]
 66%|██████▌   | 6506/9864 [10:47:34<14:47:20, 15.85s/it]
 66%|██████▌   | 6507/9864 [10:47:48<14:22:31, 15.42s/it]
 66%|██████▌   | 6508/9864 [10:48:04<14:33:03, 15.61s/it]
 66%|██████▌   | 6509/9864 [10:48:19<14:09:34, 15.19s/it]
 66%|██████▌   | 6510/9864 [10:48:33<13:57:15, 14.98s/it]
                                                         
{'loss': '0.09607', 'grad_norm': '3.679', 'learning_rate': '5.478e-06', 'epoch': '0.66'}

 66%|██████▌   | 6510/9864 [10:48:33<13:57:15, 14.98s/it]
 66%|██████▌   | 6511/9864 [10:48:49<14:17:38, 15.35s/it]
 66%|██████▌   | 6512/9864 [10:49:04<14:01:51, 15.07s/it]
 66%|██████▌   | 6513/9864 [10:49:19<14:07:36, 15.18s/it]
 66%|██████▌   | 6514/9864 [10:49:36<14:27:11, 15.53s/it]
 66%|██████▌   | 6515/9864 [10:49:50<14:02:36, 15.10s/it]
 66%|██████▌   | 6516/9864 [10:50:06<14:18:25, 15.38s/it]
 66%|██████▌   | 6517/9864 [10:50:20<14:04:47, 15.14s/it]
 66%|██████▌   | 6518/9864 [10:50:35<13:54:39, 14.97s/it]
 66%|██████▌   | 6519/9864 [10:50:51<14:09:54, 15.25s/it]
 66%|██████▌   | 6520/9864 [10:51:05<13:59:53, 15.07s/it]
                                                         
{'loss': '0.09724', 'grad_norm': '4.034', 'learning_rate': '5.449e-06', 'epoch': '0.661'}

 66%|██████▌   | 6520/9864 [10:51:05<13:59:53, 15.07s/it]
 66%|██████▌   | 6521/9864 [10:51:22<14:21:56, 15.47s/it]
 66%|██████▌   | 6522/9864 [10:51:38<14:33:33, 15.68s/it]
 66%|██████▌   | 6523/9864 [10:51:54<14:41:00, 15.82s/it]
 66%|██████▌   | 6524/9864 [10:52:08<14:15:21, 15.37s/it]
 66%|██████▌   | 6525/9864 [10:52:23<13:55:48, 15.02s/it]
 66%|██████▌   | 6526/9864 [10:52:37<13:41:59, 14.78s/it]
 66%|██████▌   | 6527/9864 [10:52:53<14:04:05, 15.18s/it]
 66%|██████▌   | 6528/9864 [10:53:07<13:52:58, 14.98s/it]
 66%|██████▌   | 6529/9864 [10:53:24<14:11:42, 15.32s/it]
 66%|██████▌   | 6530/9864 [10:53:39<14:18:54, 15.46s/it]
                                                         
{'loss': '0.099', 'grad_norm': '3.677', 'learning_rate': '5.42e-06', 'epoch': '0.6621'}

 66%|██████▌   | 6530/9864 [10:53:39<14:18:54, 15.46s/it]
 66%|██████▌   | 6531/9864 [10:53:56<14:32:46, 15.71s/it]
 66%|██████▌   | 6532/9864 [10:54:10<14:01:53, 15.16s/it]
 66%|██████▌   | 6533/9864 [10:54:24<13:52:13, 14.99s/it]
 66%|██████▌   | 6534/9864 [10:54:40<14:01:55, 15.17s/it]
 66%|██████▋   | 6535/9864 [10:54:54<13:46:45, 14.90s/it]
 66%|██████▋   | 6536/9864 [10:55:08<13:25:06, 14.52s/it]
 66%|██████▋   | 6537/9864 [10:55:23<13:43:30, 14.85s/it]
 66%|██████▋   | 6538/9864 [10:55:39<14:02:03, 15.19s/it]
 66%|██████▋   | 6539/9864 [10:55:55<14:17:30, 15.47s/it]
 66%|██████▋   | 6540/9864 [10:56:10<14:01:30, 15.19s/it]
                                                         
{'loss': '0.1144', 'grad_norm': '4.971', 'learning_rate': '5.391e-06', 'epoch': '0.6631'}

 66%|██████▋   | 6540/9864 [10:56:10<14:01:30, 15.19s/it]
 66%|██████▋   | 6541/9864 [10:56:24<13:46:18, 14.92s/it]
 66%|██████▋   | 6542/9864 [10:56:40<13:58:56, 15.15s/it]
 66%|██████▋   | 6543/9864 [10:56:54<13:45:53, 14.92s/it]
 66%|██████▋   | 6544/9864 [10:57:09<13:39:40, 14.81s/it]
 66%|██████▋   | 6545/9864 [10:57:26<14:26:17, 15.66s/it]
 66%|██████▋   | 6546/9864 [10:57:41<13:59:06, 15.17s/it]
 66%|██████▋   | 6547/9864 [10:57:57<14:16:42, 15.50s/it]
 66%|██████▋   | 6548/9864 [10:58:11<13:55:58, 15.13s/it]
 66%|██████▋   | 6549/9864 [10:58:26<13:47:00, 14.97s/it]
 66%|██████▋   | 6550/9864 [10:58:42<14:03:37, 15.27s/it]
                                                         
{'loss': '0.1091', 'grad_norm': '3.434', 'learning_rate': '5.361e-06', 'epoch': '0.6641'}

 66%|██████▋   | 6550/9864 [10:58:42<14:03:37, 15.27s/it]
 66%|██████▋   | 6551/9864 [10:58:56<13:49:42, 15.03s/it]
 66%|██████▋   | 6552/9864 [10:59:12<14:09:06, 15.38s/it]
 66%|██████▋   | 6553/9864 [10:59:27<13:55:29, 15.14s/it]
 66%|██████▋   | 6554/9864 [10:59:43<14:05:00, 15.32s/it]
 66%|██████▋   | 6555/9864 [10:59:59<14:16:01, 15.52s/it]
 66%|██████▋   | 6556/9864 [11:00:13<13:59:40, 15.23s/it]
 66%|██████▋   | 6557/9864 [11:00:29<14:16:29, 15.54s/it]
 66%|██████▋   | 6558/9864 [11:00:44<13:54:04, 15.14s/it]
 66%|██████▋   | 6559/9864 [11:00:58<13:36:39, 14.83s/it]
 67%|██████▋   | 6560/9864 [11:01:12<13:35:30, 14.81s/it]
                                                         
{'loss': '0.0944', 'grad_norm': '3.446', 'learning_rate': '5.332e-06', 'epoch': '0.6651'}

 67%|██████▋   | 6560/9864 [11:01:12<13:35:30, 14.81s/it]
 67%|██████▋   | 6561/9864 [11:01:30<14:24:27, 15.70s/it]
 67%|██████▋   | 6562/9864 [11:01:46<14:29:43, 15.80s/it]
 67%|██████▋   | 6563/9864 [11:02:00<14:02:30, 15.31s/it]
 67%|██████▋   | 6564/9864 [11:02:15<13:44:02, 14.98s/it]
 67%|██████▋   | 6565/9864 [11:02:31<13:58:11, 15.24s/it]
 67%|██████▋   | 6566/9864 [11:02:45<13:43:47, 14.99s/it]
 67%|██████▋   | 6567/9864 [11:02:59<13:35:29, 14.84s/it]
 67%|██████▋   | 6568/9864 [11:03:16<13:57:22, 15.24s/it]
 67%|██████▋   | 6569/9864 [11:03:30<13:44:00, 15.00s/it]
 67%|██████▋   | 6570/9864 [11:03:47<14:21:33, 15.69s/it]
                                                         
{'loss': '0.1073', 'grad_norm': '4.83', 'learning_rate': '5.303e-06', 'epoch': '0.6661'}

 67%|██████▋   | 6570/9864 [11:03:47<14:21:33, 15.69s/it]
 67%|██████▋   | 6571/9864 [11:04:02<13:57:21, 15.26s/it]
 67%|██████▋   | 6572/9864 [11:04:18<14:13:04, 15.55s/it]
 67%|██████▋   | 6573/9864 [11:04:32<13:49:23, 15.12s/it]
 67%|██████▋   | 6574/9864 [11:04:46<13:35:45, 14.88s/it]
 67%|██████▋   | 6575/9864 [11:05:03<13:59:07, 15.31s/it]
 67%|██████▋   | 6576/9864 [11:05:17<13:41:05, 14.98s/it]
 67%|██████▋   | 6577/9864 [11:05:33<13:58:55, 15.31s/it]
 67%|██████▋   | 6578/9864 [11:05:49<14:12:43, 15.57s/it]
 67%|██████▋   | 6579/9864 [11:06:04<14:01:35, 15.37s/it]
 67%|██████▋   | 6580/9864 [11:06:18<13:46:39, 15.10s/it]
                                                         
{'loss': '0.1127', 'grad_norm': '5.505', 'learning_rate': '5.274e-06', 'epoch': '0.6671'}

 67%|██████▋   | 6580/9864 [11:06:18<13:46:39, 15.10s/it]
 67%|██████▋   | 6581/9864 [11:06:35<14:03:57, 15.42s/it]
 67%|██████▋   | 6582/9864 [11:06:51<14:16:49, 15.66s/it]
 67%|██████▋   | 6583/9864 [11:07:05<13:48:38, 15.15s/it]
 67%|██████▋   | 6584/9864 [11:07:19<13:35:36, 14.92s/it]
 67%|██████▋   | 6585/9864 [11:07:35<13:50:03, 15.19s/it]
 67%|██████▋   | 6586/9864 [11:07:52<14:12:46, 15.61s/it]
 67%|██████▋   | 6587/9864 [11:08:06<13:57:28, 15.33s/it]
 67%|██████▋   | 6588/9864 [11:08:21<13:43:51, 15.09s/it]
 67%|██████▋   | 6589/9864 [11:08:38<14:26:12, 15.87s/it]
 67%|██████▋   | 6590/9864 [11:08:53<13:58:16, 15.36s/it]
                                                         
{'loss': '0.09201', 'grad_norm': '4.587', 'learning_rate': '5.246e-06', 'epoch': '0.6681'}

 67%|██████▋   | 6590/9864 [11:08:53<13:58:16, 15.36s/it]
 67%|██████▋   | 6591/9864 [11:09:07<13:44:25, 15.11s/it]
 67%|██████▋   | 6592/9864 [11:09:23<14:00:57, 15.42s/it]
 67%|██████▋   | 6593/9864 [11:09:38<13:46:25, 15.16s/it]
 67%|██████▋   | 6594/9864 [11:09:53<13:44:10, 15.12s/it]
 67%|██████▋   | 6595/9864 [11:10:08<13:50:43, 15.25s/it]
 67%|██████▋   | 6596/9864 [11:10:23<13:38:56, 15.04s/it]
 67%|██████▋   | 6597/9864 [11:10:38<13:45:32, 15.16s/it]
 67%|██████▋   | 6598/9864 [11:10:54<13:58:07, 15.40s/it]
 67%|██████▋   | 6599/9864 [11:11:10<14:06:22, 15.55s/it]
 67%|██████▋   | 6600/9864 [11:11:25<13:45:50, 15.18s/it]
                                                         
{'loss': '0.09638', 'grad_norm': '4.633', 'learning_rate': '5.217e-06', 'epoch': '0.6692'}

 67%|██████▋   | 6600/9864 [11:11:25<13:45:50, 15.18s/it]
 67%|██████▋   | 6601/9864 [11:11:39<13:34:12, 14.97s/it]
 67%|██████▋   | 6602/9864 [11:11:53<13:22:21, 14.76s/it]
 67%|██████▋   | 6603/9864 [11:12:09<13:38:01, 15.05s/it]
 67%|██████▋   | 6604/9864 [11:12:24<13:30:44, 14.92s/it]
 67%|██████▋   | 6605/9864 [11:12:42<14:19:23, 15.82s/it]
 67%|██████▋   | 6606/9864 [11:12:56<14:02:07, 15.51s/it]
 67%|██████▋   | 6607/9864 [11:13:12<14:08:15, 15.63s/it]
 67%|██████▋   | 6608/9864 [11:13:27<13:46:22, 15.23s/it]
 67%|██████▋   | 6609/9864 [11:13:41<13:31:52, 14.97s/it]
 67%|██████▋   | 6610/9864 [11:13:58<13:57:42, 15.45s/it]
                                                         
{'loss': '0.09333', 'grad_norm': '3.151', 'learning_rate': '5.188e-06', 'epoch': '0.6702'}

 67%|██████▋   | 6610/9864 [11:13:58<13:57:42, 15.45s/it]
 67%|██████▋   | 6611/9864 [11:14:12<13:42:19, 15.17s/it]
 67%|██████▋   | 6612/9864 [11:14:28<14:00:07, 15.50s/it]
 67%|██████▋   | 6613/9864 [11:14:44<14:09:20, 15.68s/it]
 67%|██████▋   | 6614/9864 [11:14:59<13:47:38, 15.28s/it]
 67%|██████▋   | 6615/9864 [11:15:15<13:59:26, 15.50s/it]
 67%|██████▋   | 6616/9864 [11:15:29<13:43:19, 15.21s/it]
 67%|██████▋   | 6617/9864 [11:15:44<13:28:35, 14.94s/it]
 67%|██████▋   | 6618/9864 [11:16:00<13:44:36, 15.24s/it]
 67%|██████▋   | 6619/9864 [11:16:14<13:38:45, 15.14s/it]
 67%|██████▋   | 6620/9864 [11:16:30<13:48:55, 15.33s/it]
                                                         
{'loss': '0.1031', 'grad_norm': '4.098', 'learning_rate': '5.159e-06', 'epoch': '0.6712'}

 67%|██████▋   | 6620/9864 [11:16:30<13:48:55, 15.33s/it]
 67%|██████▋   | 6621/9864 [11:16:46<13:54:37, 15.44s/it]
 67%|██████▋   | 6622/9864 [11:17:01<13:55:31, 15.46s/it]
 67%|██████▋   | 6623/9864 [11:17:16<13:38:26, 15.15s/it]
 67%|██████▋   | 6624/9864 [11:17:30<13:26:43, 14.94s/it]
 67%|██████▋   | 6625/9864 [11:17:46<13:39:55, 15.19s/it]
 67%|██████▋   | 6626/9864 [11:18:01<13:29:13, 14.99s/it]
 67%|██████▋   | 6627/9864 [11:18:15<13:24:09, 14.91s/it]
 67%|██████▋   | 6628/9864 [11:18:31<13:43:51, 15.28s/it]
 67%|██████▋   | 6629/9864 [11:18:47<13:53:18, 15.46s/it]
 67%|██████▋   | 6630/9864 [11:19:02<13:38:26, 15.18s/it]
                                                         
{'loss': '0.09959', 'grad_norm': '4.038', 'learning_rate': '5.13e-06', 'epoch': '0.6722'}

 67%|██████▋   | 6630/9864 [11:19:02<13:38:26, 15.18s/it]
 67%|██████▋   | 6631/9864 [11:19:18<13:59:24, 15.58s/it]
 67%|██████▋   | 6632/9864 [11:19:33<13:39:34, 15.21s/it]
 67%|██████▋   | 6633/9864 [11:19:47<13:28:47, 15.02s/it]
 67%|██████▋   | 6634/9864 [11:20:04<13:48:08, 15.38s/it]
 67%|██████▋   | 6635/9864 [11:20:20<14:00:21, 15.62s/it]
 67%|██████▋   | 6636/9864 [11:20:34<13:42:35, 15.29s/it]
 67%|██████▋   | 6637/9864 [11:20:50<13:53:27, 15.50s/it]
 67%|██████▋   | 6638/9864 [11:21:06<14:03:04, 15.68s/it]
 67%|██████▋   | 6639/9864 [11:21:21<13:38:58, 15.24s/it]
 67%|██████▋   | 6640/9864 [11:21:35<13:26:44, 15.01s/it]
                                                         
{'loss': '0.11', 'grad_norm': '4.174', 'learning_rate': '5.102e-06', 'epoch': '0.6732'}

 67%|██████▋   | 6640/9864 [11:21:35<13:26:44, 15.01s/it]
 67%|██████▋   | 6641/9864 [11:21:51<13:49:33, 15.44s/it]
 67%|██████▋   | 6642/9864 [11:22:06<13:37:35, 15.23s/it]
 67%|██████▋   | 6643/9864 [11:22:23<13:59:46, 15.64s/it]
 67%|██████▋   | 6644/9864 [11:22:37<13:40:45, 15.29s/it]
 67%|██████▋   | 6645/9864 [11:22:53<13:54:59, 15.56s/it]
 67%|██████▋   | 6646/9864 [11:23:07<13:28:24, 15.07s/it]
 67%|██████▋   | 6647/9864 [11:23:24<13:46:07, 15.41s/it]
 67%|██████▋   | 6648/9864 [11:23:38<13:35:07, 15.21s/it]
 67%|██████▋   | 6649/9864 [11:23:55<13:52:08, 15.53s/it]
 67%|██████▋   | 6650/9864 [11:24:11<14:00:36, 15.69s/it]
                                                         
{'loss': '0.1077', 'grad_norm': '3.555', 'learning_rate': '5.073e-06', 'epoch': '0.6742'}

 67%|██████▋   | 6650/9864 [11:24:11<14:00:36, 15.69s/it]
 67%|██████▋   | 6651/9864 [11:24:24<13:29:56, 15.12s/it]
 67%|██████▋   | 6652/9864 [11:24:39<13:19:44, 14.94s/it]
 67%|██████▋   | 6653/9864 [11:24:55<13:43:22, 15.39s/it]
 67%|██████▋   | 6654/9864 [11:25:10<13:29:02, 15.12s/it]
 67%|██████▋   | 6655/9864 [11:25:24<13:10:16, 14.78s/it]
 67%|██████▋   | 6656/9864 [11:25:42<14:04:41, 15.80s/it]
 67%|██████▋   | 6657/9864 [11:25:56<13:37:28, 15.29s/it]
 67%|██████▋   | 6658/9864 [11:26:11<13:30:38, 15.17s/it]
 68%|██████▊   | 6659/9864 [11:26:25<13:15:31, 14.89s/it]
 68%|██████▊   | 6660/9864 [11:26:39<13:02:23, 14.65s/it]
                                                         
{'loss': '0.1014', 'grad_norm': '4.617', 'learning_rate': '5.045e-06', 'epoch': '0.6752'}

 68%|██████▊   | 6660/9864 [11:26:39<13:02:23, 14.65s/it]
 68%|██████▊   | 6661/9864 [11:26:55<13:18:14, 14.95s/it]
 68%|██████▊   | 6662/9864 [11:27:09<13:06:08, 14.73s/it]
 68%|██████▊   | 6663/9864 [11:27:27<13:54:12, 15.64s/it]
 68%|██████▊   | 6664/9864 [11:27:41<13:25:51, 15.11s/it]
 68%|██████▊   | 6665/9864 [11:27:55<13:09:33, 14.81s/it]
 68%|██████▊   | 6666/9864 [11:28:10<13:18:35, 14.98s/it]
 68%|██████▊   | 6667/9864 [11:28:24<12:58:22, 14.61s/it]
 68%|██████▊   | 6668/9864 [11:28:39<13:08:02, 14.79s/it]
 68%|██████▊   | 6669/9864 [11:28:53<12:53:48, 14.53s/it]
 68%|██████▊   | 6670/9864 [11:29:08<12:54:55, 14.56s/it]
                                                         
{'loss': '0.09254', 'grad_norm': '3.749', 'learning_rate': '5.016e-06', 'epoch': '0.6763'}

 68%|██████▊   | 6670/9864 [11:29:08<12:54:55, 14.56s/it]
 68%|██████▊   | 6671/9864 [11:29:26<13:43:00, 15.47s/it]
 68%|██████▊   | 6672/9864 [11:29:40<13:27:46, 15.18s/it]
 68%|██████▊   | 6673/9864 [11:29:54<13:15:15, 14.95s/it]
 68%|██████▊   | 6674/9864 [11:30:11<13:34:53, 15.33s/it]
 68%|██████▊   | 6675/9864 [11:30:25<13:12:31, 14.91s/it]
 68%|██████▊   | 6676/9864 [11:30:39<12:59:07, 14.66s/it]
 68%|██████▊   | 6677/9864 [11:30:54<13:14:16, 14.95s/it]
 68%|██████▊   | 6678/9864 [11:31:12<13:56:32, 15.75s/it]
 68%|██████▊   | 6679/9864 [11:31:26<13:26:25, 15.19s/it]
 68%|██████▊   | 6680/9864 [11:31:40<13:03:41, 14.77s/it]
                                                         
{'loss': '0.1024', 'grad_norm': '3.958', 'learning_rate': '4.988e-06', 'epoch': '0.6773'}

 68%|██████▊   | 6680/9864 [11:31:40<13:03:41, 14.77s/it]
 68%|██████▊   | 6681/9864 [11:31:54<12:52:54, 14.57s/it]
 68%|██████▊   | 6682/9864 [11:32:09<13:06:46, 14.84s/it]
 68%|██████▊   | 6683/9864 [11:32:24<13:00:31, 14.72s/it]
 68%|██████▊   | 6684/9864 [11:32:39<13:10:08, 14.91s/it]
 68%|██████▊   | 6685/9864 [11:32:54<13:18:32, 15.07s/it]
 68%|██████▊   | 6686/9864 [11:33:10<13:22:40, 15.15s/it]
 68%|██████▊   | 6687/9864 [11:33:24<13:01:06, 14.75s/it]
 68%|██████▊   | 6688/9864 [11:33:37<12:41:30, 14.39s/it]
 68%|██████▊   | 6689/9864 [11:33:51<12:32:30, 14.22s/it]
 68%|██████▊   | 6690/9864 [11:34:05<12:30:02, 14.18s/it]
                                                         
{'loss': '0.1048', 'grad_norm': '5.221', 'learning_rate': '4.959e-06', 'epoch': '0.6783'}

 68%|██████▊   | 6690/9864 [11:34:05<12:30:02, 14.18s/it]
 68%|██████▊   | 6691/9864 [11:34:22<13:15:33, 15.04s/it]
 68%|██████▊   | 6692/9864 [11:34:38<13:28:49, 15.30s/it]
 68%|██████▊   | 6693/9864 [11:34:52<13:13:25, 15.01s/it]
 68%|██████▊   | 6694/9864 [11:35:08<13:29:39, 15.32s/it]
 68%|██████▊   | 6695/9864 [11:35:23<13:10:51, 14.97s/it]
 68%|██████▊   | 6696/9864 [11:35:37<12:58:18, 14.74s/it]
 68%|██████▊   | 6697/9864 [11:35:51<12:52:29, 14.64s/it]
 68%|██████▊   | 6698/9864 [11:36:07<13:10:03, 14.97s/it]
 68%|██████▊   | 6699/9864 [11:36:23<13:29:54, 15.35s/it]
 68%|██████▊   | 6700/9864 [11:36:39<13:40:57, 15.57s/it]
                                                         
{'loss': '0.08443', 'grad_norm': '3.268', 'learning_rate': '4.931e-06', 'epoch': '0.6793'}

 68%|██████▊   | 6700/9864 [11:36:39<13:40:57, 15.57s/it]
 68%|██████▊   | 6701/9864 [11:36:53<13:16:31, 15.11s/it]
 68%|██████▊   | 6702/9864 [11:37:09<13:23:27, 15.25s/it]
 68%|██████▊   | 6703/9864 [11:37:23<13:04:43, 14.90s/it]
 68%|██████▊   | 6704/9864 [11:37:37<12:57:55, 14.77s/it]
 68%|██████▊   | 6705/9864 [11:37:52<12:53:40, 14.69s/it]
 68%|██████▊   | 6706/9864 [11:38:10<13:41:56, 15.62s/it]
 68%|██████▊   | 6707/9864 [11:38:24<13:18:44, 15.18s/it]
 68%|██████▊   | 6708/9864 [11:38:40<13:27:54, 15.36s/it]
 68%|██████▊   | 6709/9864 [11:38:53<13:00:29, 14.84s/it]
 68%|██████▊   | 6710/9864 [11:39:08<13:00:12, 14.84s/it]
                                                         
{'loss': '0.105', 'grad_norm': '6.459', 'learning_rate': '4.903e-06', 'epoch': '0.6803'}

 68%|██████▊   | 6710/9864 [11:39:08<13:00:12, 14.84s/it]
 68%|██████▊   | 6711/9864 [11:39:22<12:49:37, 14.65s/it]
 68%|██████▊   | 6712/9864 [11:39:36<12:36:47, 14.41s/it]
 68%|██████▊   | 6713/9864 [11:39:52<12:58:55, 14.83s/it]
 68%|██████▊   | 6714/9864 [11:40:07<13:05:03, 14.95s/it]
 68%|██████▊   | 6715/9864 [11:40:21<12:50:41, 14.68s/it]
 68%|██████▊   | 6716/9864 [11:40:36<12:44:57, 14.58s/it]
 68%|██████▊   | 6717/9864 [11:40:53<13:23:24, 15.32s/it]
 68%|██████▊   | 6718/9864 [11:41:07<13:09:00, 15.05s/it]
 68%|██████▊   | 6719/9864 [11:41:21<12:53:54, 14.76s/it]
 68%|██████▊   | 6720/9864 [11:41:36<12:49:20, 14.68s/it]
                                                         
{'loss': '0.09669', 'grad_norm': '3.99', 'learning_rate': '4.874e-06', 'epoch': '0.6813'}

 68%|██████▊   | 6720/9864 [11:41:36<12:49:20, 14.68s/it]
 68%|██████▊   | 6721/9864 [11:41:51<13:02:05, 14.93s/it]
 68%|██████▊   | 6722/9864 [11:42:05<12:46:36, 14.64s/it]
 68%|██████▊   | 6723/9864 [11:42:21<13:10:25, 15.10s/it]
 68%|██████▊   | 6724/9864 [11:42:37<13:16:24, 15.22s/it]
 68%|██████▊   | 6725/9864 [11:42:52<13:24:46, 15.38s/it]
 68%|██████▊   | 6726/9864 [11:43:07<13:08:13, 15.07s/it]
 68%|██████▊   | 6727/9864 [11:43:21<12:51:35, 14.76s/it]
 68%|██████▊   | 6728/9864 [11:43:37<13:06:09, 15.04s/it]
 68%|██████▊   | 6729/9864 [11:43:51<12:50:56, 14.75s/it]
 68%|██████▊   | 6730/9864 [11:44:05<12:40:34, 14.56s/it]
                                                         
{'loss': '0.09891', 'grad_norm': '5.492', 'learning_rate': '4.846e-06', 'epoch': '0.6823'}

 68%|██████▊   | 6730/9864 [11:44:05<12:40:34, 14.56s/it]
 68%|██████▊   | 6731/9864 [11:44:21<13:08:32, 15.10s/it]
 68%|██████▊   | 6732/9864 [11:44:39<13:56:21, 16.02s/it]
 68%|██████▊   | 6733/9864 [11:44:53<13:22:12, 15.37s/it]
 68%|██████▊   | 6734/9864 [11:45:07<12:51:57, 14.80s/it]
 68%|██████▊   | 6735/9864 [11:45:23<13:09:54, 15.15s/it]
 68%|██████▊   | 6736/9864 [11:45:37<12:55:22, 14.87s/it]
 68%|██████▊   | 6737/9864 [11:45:51<12:43:21, 14.65s/it]
 68%|██████▊   | 6738/9864 [11:46:05<12:35:28, 14.50s/it]
 68%|██████▊   | 6739/9864 [11:46:23<13:27:32, 15.50s/it]
 68%|██████▊   | 6740/9864 [11:46:38<13:23:07, 15.43s/it]
                                                         
{'loss': '0.1072', 'grad_norm': '5.023', 'learning_rate': '4.818e-06', 'epoch': '0.6833'}

 68%|██████▊   | 6740/9864 [11:46:38<13:23:07, 15.43s/it]
 68%|██████▊   | 6741/9864 [11:46:53<13:10:13, 15.18s/it]
 68%|██████▊   | 6742/9864 [11:47:06<12:43:29, 14.67s/it]
 68%|██████▊   | 6743/9864 [11:47:21<12:41:12, 14.63s/it]
 68%|██████▊   | 6744/9864 [11:47:36<12:53:09, 14.87s/it]
 68%|██████▊   | 6745/9864 [11:47:50<12:42:09, 14.66s/it]
 68%|██████▊   | 6746/9864 [11:48:05<12:38:27, 14.60s/it]
 68%|██████▊   | 6747/9864 [11:48:21<12:59:15, 15.00s/it]
 68%|██████▊   | 6748/9864 [11:48:39<13:46:06, 15.91s/it]
 68%|██████▊   | 6749/9864 [11:48:53<13:19:08, 15.39s/it]
 68%|██████▊   | 6750/9864 [11:49:07<12:55:39, 14.95s/it]
                                                         
{'loss': '0.08944', 'grad_norm': '3.311', 'learning_rate': '4.79e-06', 'epoch': '0.6844'}

 68%|██████▊   | 6750/9864 [11:49:07<12:55:39, 14.95s/it]
 68%|██████▊   | 6751/9864 [11:49:22<13:03:42, 15.11s/it]
 68%|██████▊   | 6752/9864 [11:49:37<12:52:11, 14.89s/it]
 68%|██████▊   | 6753/9864 [11:49:51<12:34:39, 14.55s/it]
 68%|██████▊   | 6754/9864 [11:50:04<12:24:49, 14.37s/it]
 68%|██████▊   | 6755/9864 [11:50:20<12:42:48, 14.72s/it]
 68%|██████▊   | 6756/9864 [11:50:36<13:03:45, 15.13s/it]
 69%|██████▊   | 6757/9864 [11:50:50<12:39:02, 14.66s/it]
 69%|██████▊   | 6758/9864 [11:51:06<12:58:39, 15.04s/it]
 69%|██████▊   | 6759/9864 [11:51:20<12:43:14, 14.75s/it]
 69%|██████▊   | 6760/9864 [11:51:34<12:30:16, 14.50s/it]
                                                         
{'loss': '0.1015', 'grad_norm': '5.209', 'learning_rate': '4.762e-06', 'epoch': '0.6854'}

 69%|██████▊   | 6760/9864 [11:51:34<12:30:16, 14.50s/it]
 69%|██████▊   | 6761/9864 [11:51:47<12:18:58, 14.29s/it]
 69%|██████▊   | 6762/9864 [11:52:03<12:38:25, 14.67s/it]
 69%|██████▊   | 6763/9864 [11:52:20<13:22:21, 15.52s/it]
 69%|██████▊   | 6764/9864 [11:52:35<13:00:08, 15.10s/it]
 69%|██████▊   | 6765/9864 [11:52:48<12:41:42, 14.75s/it]
 69%|██████▊   | 6766/9864 [11:53:04<12:59:30, 15.10s/it]
 69%|██████▊   | 6767/9864 [11:53:19<12:44:06, 14.80s/it]
 69%|██████▊   | 6768/9864 [11:53:33<12:32:07, 14.58s/it]
 69%|██████▊   | 6769/9864 [11:53:48<12:46:28, 14.86s/it]
 69%|██████▊   | 6770/9864 [11:54:02<12:33:02, 14.60s/it]
                                                         
{'loss': '0.09357', 'grad_norm': '3.957', 'learning_rate': '4.734e-06', 'epoch': '0.6864'}

 69%|██████▊   | 6770/9864 [11:54:02<12:33:02, 14.60s/it]
 69%|██████▊   | 6771/9864 [11:54:18<12:47:33, 14.89s/it]
 69%|██████▊   | 6772/9864 [11:54:34<13:03:22, 15.20s/it]
 69%|██████▊   | 6773/9864 [11:54:49<13:09:25, 15.32s/it]
 69%|██████▊   | 6774/9864 [11:55:03<12:48:24, 14.92s/it]
 69%|██████▊   | 6775/9864 [11:55:17<12:33:17, 14.63s/it]
 69%|██████▊   | 6776/9864 [11:55:33<12:54:36, 15.05s/it]
 69%|██████▊   | 6777/9864 [11:55:47<12:41:44, 14.81s/it]
 69%|██████▊   | 6778/9864 [11:56:02<12:33:12, 14.64s/it]
 69%|██████▊   | 6779/9864 [11:56:19<13:17:41, 15.51s/it]
 69%|██████▊   | 6780/9864 [11:56:34<13:00:29, 15.18s/it]
                                                         
{'loss': '0.1043', 'grad_norm': '4.336', 'learning_rate': '4.706e-06', 'epoch': '0.6874'}

 69%|██████▊   | 6780/9864 [11:56:34<13:00:29, 15.18s/it]
 69%|██████▊   | 6781/9864 [11:56:49<13:07:44, 15.33s/it]
 69%|██████▉   | 6782/9864 [11:57:03<12:42:55, 14.85s/it]
 69%|██████▉   | 6783/9864 [11:57:18<12:37:40, 14.76s/it]
 69%|██████▉   | 6784/9864 [11:57:33<12:50:02, 15.00s/it]
 69%|██████▉   | 6785/9864 [11:57:47<12:35:18, 14.72s/it]
 69%|██████▉   | 6786/9864 [11:58:03<12:48:42, 14.98s/it]
 69%|██████▉   | 6787/9864 [11:58:18<12:57:27, 15.16s/it]
 69%|██████▉   | 6788/9864 [11:58:34<12:58:10, 15.18s/it]
 69%|██████▉   | 6789/9864 [11:58:48<12:45:14, 14.93s/it]
 69%|██████▉   | 6790/9864 [11:59:02<12:29:42, 14.63s/it]
                                                         
{'loss': '0.0953', 'grad_norm': '4.146', 'learning_rate': '4.678e-06', 'epoch': '0.6884'}

 69%|██████▉   | 6790/9864 [11:59:02<12:29:42, 14.63s/it]
 69%|██████▉   | 6791/9864 [11:59:18<12:49:56, 15.03s/it]
 69%|██████▉   | 6792/9864 [11:59:32<12:33:22, 14.71s/it]
 69%|██████▉   | 6793/9864 [11:59:46<12:18:11, 14.42s/it]
 69%|██████▉   | 6794/9864 [12:00:00<12:13:09, 14.33s/it]
 69%|██████▉   | 6795/9864 [12:00:16<12:40:19, 14.86s/it]
 69%|██████▉   | 6796/9864 [12:00:30<12:23:30, 14.54s/it]
 69%|██████▉   | 6797/9864 [12:00:45<12:40:13, 14.87s/it]
 69%|██████▉   | 6798/9864 [12:01:01<12:49:10, 15.05s/it]
 69%|██████▉   | 6799/9864 [12:01:15<12:39:40, 14.87s/it]
 69%|██████▉   | 6800/9864 [12:01:29<12:21:04, 14.51s/it]
                                                         
{'loss': '0.09067', 'grad_norm': '4.154', 'learning_rate': '4.651e-06', 'epoch': '0.6894'}

 69%|██████▉   | 6800/9864 [12:01:29<12:21:04, 14.51s/it]
 69%|██████▉   | 6801/9864 [12:01:42<12:06:14, 14.23s/it]
 69%|██████▉   | 6802/9864 [12:01:59<12:50:19, 15.09s/it]
 69%|██████▉   | 6803/9864 [12:02:13<12:23:40, 14.58s/it]
 69%|██████▉   | 6804/9864 [12:02:28<12:37:05, 14.84s/it]
 69%|██████▉   | 6805/9864 [12:02:43<12:41:10, 14.93s/it]
 69%|██████▉   | 6806/9864 [12:02:58<12:29:05, 14.70s/it]
 69%|██████▉   | 6807/9864 [12:03:11<12:11:37, 14.36s/it]
 69%|██████▉   | 6808/9864 [12:03:25<12:09:02, 14.31s/it]
 69%|██████▉   | 6809/9864 [12:03:40<12:09:06, 14.32s/it]
 69%|██████▉   | 6810/9864 [12:03:55<12:30:18, 14.74s/it]
                                                         
{'loss': '0.1061', 'grad_norm': '4.579', 'learning_rate': '4.623e-06', 'epoch': '0.6904'}

 69%|██████▉   | 6810/9864 [12:03:55<12:30:18, 14.74s/it]
 69%|██████▉   | 6811/9864 [12:04:11<12:46:57, 15.07s/it]
 69%|██████▉   | 6812/9864 [12:04:27<13:01:07, 15.36s/it]
 69%|██████▉   | 6813/9864 [12:04:43<13:09:12, 15.52s/it]
 69%|██████▉   | 6814/9864 [12:04:58<12:50:24, 15.16s/it]
 69%|██████▉   | 6815/9864 [12:05:11<12:30:00, 14.76s/it]
 69%|██████▉   | 6816/9864 [12:05:26<12:24:25, 14.65s/it]
 69%|██████▉   | 6817/9864 [12:05:40<12:19:45, 14.57s/it]
 69%|██████▉   | 6818/9864 [12:05:55<12:30:53, 14.79s/it]
 69%|██████▉   | 6819/9864 [12:06:09<12:18:52, 14.56s/it]
 69%|██████▉   | 6820/9864 [12:06:25<12:33:28, 14.85s/it]
                                                         
{'loss': '0.0933', 'grad_norm': '3.698', 'learning_rate': '4.595e-06', 'epoch': '0.6915'}

 69%|██████▉   | 6820/9864 [12:06:25<12:33:28, 14.85s/it]
 69%|██████▉   | 6821/9864 [12:06:41<12:45:02, 15.08s/it]
 69%|██████▉   | 6822/9864 [12:06:54<12:26:17, 14.72s/it]
 69%|██████▉   | 6823/9864 [12:07:09<12:16:23, 14.53s/it]
 69%|██████▉   | 6824/9864 [12:07:22<12:05:28, 14.32s/it]
 69%|██████▉   | 6825/9864 [12:07:37<12:03:29, 14.28s/it]
 69%|██████▉   | 6826/9864 [12:07:50<11:55:30, 14.13s/it]
 69%|██████▉   | 6827/9864 [12:08:06<12:19:14, 14.60s/it]
 69%|██████▉   | 6828/9864 [12:08:24<13:16:40, 15.74s/it]
 69%|██████▉   | 6829/9864 [12:08:39<12:50:52, 15.24s/it]
 69%|██████▉   | 6830/9864 [12:08:53<12:32:37, 14.88s/it]
                                                         
{'loss': '0.08779', 'grad_norm': '4.099', 'learning_rate': '4.568e-06', 'epoch': '0.6925'}

 69%|██████▉   | 6830/9864 [12:08:53<12:32:37, 14.88s/it]
 69%|██████▉   | 6831/9864 [12:09:07<12:25:00, 14.74s/it]
 69%|██████▉   | 6832/9864 [12:09:22<12:21:38, 14.68s/it]
 69%|██████▉   | 6833/9864 [12:09:36<12:16:12, 14.57s/it]
 69%|██████▉   | 6834/9864 [12:09:52<12:34:57, 14.95s/it]
 69%|██████▉   | 6835/9864 [12:10:09<13:15:01, 15.75s/it]
 69%|██████▉   | 6836/9864 [12:10:23<12:46:26, 15.19s/it]
 69%|██████▉   | 6837/9864 [12:10:39<12:55:10, 15.37s/it]
 69%|██████▉   | 6838/9864 [12:10:53<12:35:26, 14.98s/it]
 69%|██████▉   | 6839/9864 [12:11:07<12:14:39, 14.57s/it]
 69%|██████▉   | 6840/9864 [12:11:21<12:05:52, 14.40s/it]
                                                         
{'loss': '0.1018', 'grad_norm': '3.882', 'learning_rate': '4.54e-06', 'epoch': '0.6935'}

 69%|██████▉   | 6840/9864 [12:11:21<12:05:52, 14.40s/it]
 69%|██████▉   | 6841/9864 [12:11:35<12:08:57, 14.47s/it]
 69%|██████▉   | 6842/9864 [12:11:51<12:33:28, 14.96s/it]
 69%|██████▉   | 6843/9864 [12:12:07<12:49:16, 15.28s/it]
 69%|██████▉   | 6844/9864 [12:12:23<12:57:01, 15.44s/it]
 69%|██████▉   | 6845/9864 [12:12:39<13:06:14, 15.63s/it]
 69%|██████▉   | 6846/9864 [12:12:53<12:36:11, 15.03s/it]
 69%|██████▉   | 6847/9864 [12:13:07<12:18:23, 14.68s/it]
 69%|██████▉   | 6848/9864 [12:13:20<12:02:10, 14.37s/it]
 69%|██████▉   | 6849/9864 [12:13:34<11:50:11, 14.13s/it]
 69%|██████▉   | 6850/9864 [12:13:51<12:25:25, 14.84s/it]
                                                         
{'loss': '0.09006', 'grad_norm': '4.616', 'learning_rate': '4.513e-06', 'epoch': '0.6945'}

 69%|██████▉   | 6850/9864 [12:13:51<12:25:25, 14.84s/it]
 69%|██████▉   | 6851/9864 [12:14:08<13:04:24, 15.62s/it]
 69%|██████▉   | 6852/9864 [12:14:24<13:06:22, 15.66s/it]
 69%|██████▉   | 6853/9864 [12:14:38<12:42:34, 15.20s/it]
 69%|██████▉   | 6854/9864 [12:14:52<12:19:43, 14.75s/it]
 69%|██████▉   | 6855/9864 [12:15:05<12:04:45, 14.45s/it]
 70%|██████▉   | 6856/9864 [12:15:20<12:01:02, 14.38s/it]
 70%|██████▉   | 6857/9864 [12:15:36<12:26:00, 14.89s/it]
 70%|██████▉   | 6858/9864 [12:15:52<12:43:55, 15.25s/it]
 70%|██████▉   | 6859/9864 [12:16:08<12:54:59, 15.47s/it]
 70%|██████▉   | 6860/9864 [12:16:23<12:57:21, 15.53s/it]
                                                         
{'loss': '0.08373', 'grad_norm': '5.057', 'learning_rate': '4.485e-06', 'epoch': '0.6955'}

 70%|██████▉   | 6860/9864 [12:16:23<12:57:21, 15.53s/it]
 70%|██████▉   | 6861/9864 [12:16:37<12:36:11, 15.11s/it]
 70%|██████▉   | 6862/9864 [12:16:51<12:17:03, 14.73s/it]
 70%|██████▉   | 6863/9864 [12:17:06<12:09:27, 14.58s/it]
 70%|██████▉   | 6864/9864 [12:17:20<12:03:06, 14.46s/it]
 70%|██████▉   | 6865/9864 [12:17:35<12:21:21, 14.83s/it]
 70%|██████▉   | 6866/9864 [12:17:53<13:01:46, 15.65s/it]
 70%|██████▉   | 6867/9864 [12:18:07<12:42:35, 15.27s/it]
 70%|██████▉   | 6868/9864 [12:18:23<12:52:50, 15.48s/it]
 70%|██████▉   | 6869/9864 [12:18:38<12:34:44, 15.12s/it]
 70%|██████▉   | 6870/9864 [12:18:52<12:23:06, 14.89s/it]
                                                         
{'loss': '0.1021', 'grad_norm': '6.035', 'learning_rate': '4.458e-06', 'epoch': '0.6965'}

 70%|██████▉   | 6870/9864 [12:18:52<12:23:06, 14.89s/it]
 70%|██████▉   | 6871/9864 [12:19:06<12:13:41, 14.71s/it]
 70%|██████▉   | 6872/9864 [12:19:20<12:03:03, 14.50s/it]
 70%|██████▉   | 6873/9864 [12:19:36<12:21:49, 14.88s/it]
 70%|██████▉   | 6874/9864 [12:19:52<12:39:11, 15.23s/it]
 70%|██████▉   | 6875/9864 [12:20:08<12:48:42, 15.43s/it]
 70%|██████▉   | 6876/9864 [12:20:24<12:54:55, 15.56s/it]
 70%|██████▉   | 6877/9864 [12:20:38<12:33:35, 15.14s/it]
 70%|██████▉   | 6878/9864 [12:20:52<12:13:21, 14.74s/it]
 70%|██████▉   | 6879/9864 [12:21:06<12:00:21, 14.48s/it]
 70%|██████▉   | 6880/9864 [12:21:20<11:54:23, 14.36s/it]
                                                         
{'loss': '0.08928', 'grad_norm': '4.635', 'learning_rate': '4.431e-06', 'epoch': '0.6975'}

 70%|██████▉   | 6880/9864 [12:21:20<11:54:23, 14.36s/it]
 70%|██████▉   | 6881/9864 [12:21:36<12:15:52, 14.80s/it]
 70%|██████▉   | 6882/9864 [12:21:51<12:28:26, 15.06s/it]
 70%|██████▉   | 6883/9864 [12:22:07<12:38:39, 15.27s/it]
 70%|██████▉   | 6884/9864 [12:22:23<12:47:06, 15.45s/it]
 70%|██████▉   | 6885/9864 [12:22:37<12:30:18, 15.11s/it]
 70%|██████▉   | 6886/9864 [12:22:52<12:22:45, 14.96s/it]
 70%|██████▉   | 6887/9864 [12:23:06<12:05:37, 14.62s/it]
 70%|██████▉   | 6888/9864 [12:23:20<11:57:10, 14.46s/it]
 70%|██████▉   | 6889/9864 [12:23:37<12:38:22, 15.29s/it]
 70%|██████▉   | 6890/9864 [12:23:53<12:46:19, 15.46s/it]
                                                         
{'loss': '0.09784', 'grad_norm': '6.203', 'learning_rate': '4.403e-06', 'epoch': '0.6986'}

 70%|██████▉   | 6890/9864 [12:23:53<12:46:19, 15.46s/it]
 70%|██████▉   | 6891/9864 [12:24:07<12:21:33, 14.97s/it]
 70%|██████▉   | 6892/9864 [12:24:23<12:35:19, 15.25s/it]
 70%|██████▉   | 6893/9864 [12:24:37<12:16:28, 14.87s/it]
 70%|██████▉   | 6894/9864 [12:24:51<12:04:19, 14.63s/it]
 70%|██████▉   | 6895/9864 [12:25:05<11:58:14, 14.51s/it]
 70%|██████▉   | 6896/9864 [12:25:19<11:54:19, 14.44s/it]
 70%|██████▉   | 6897/9864 [12:25:36<12:35:55, 15.29s/it]
 70%|██████▉   | 6898/9864 [12:25:52<12:42:43, 15.43s/it]
 70%|██████▉   | 6899/9864 [12:26:07<12:27:56, 15.14s/it]
 70%|██████▉   | 6900/9864 [12:26:22<12:34:23, 15.27s/it]
                                                         
{'loss': '0.08964', 'grad_norm': '4.434', 'learning_rate': '4.376e-06', 'epoch': '0.6996'}

 70%|██████▉   | 6900/9864 [12:26:22<12:34:23, 15.27s/it]
 70%|██████▉   | 6901/9864 [12:26:36<12:11:11, 14.81s/it]
 70%|██████▉   | 6902/9864 [12:26:50<12:01:01, 14.61s/it]
 70%|██████▉   | 6903/9864 [12:27:04<11:49:36, 14.38s/it]
 70%|██████▉   | 6904/9864 [12:27:18<11:46:22, 14.32s/it]
 70%|███████   | 6905/9864 [12:27:34<12:12:10, 14.85s/it]
 70%|███████   | 6906/9864 [12:27:51<12:41:26, 15.45s/it]
 70%|███████   | 6907/9864 [12:28:05<12:23:04, 15.08s/it]
 70%|███████   | 6908/9864 [12:28:21<12:33:39, 15.30s/it]
 70%|███████   | 6909/9864 [12:28:35<12:18:07, 14.99s/it]
 70%|███████   | 6910/9864 [12:28:49<11:56:56, 14.56s/it]
                                                         
{'loss': '0.09178', 'grad_norm': '4.76', 'learning_rate': '4.349e-06', 'epoch': '0.7006'}

 70%|███████   | 6910/9864 [12:28:49<11:56:56, 14.56s/it]
 70%|███████   | 6911/9864 [12:29:03<11:52:33, 14.48s/it]
 70%|███████   | 6912/9864 [12:29:17<11:44:21, 14.32s/it]
 70%|███████   | 6913/9864 [12:29:35<12:30:50, 15.27s/it]
 70%|███████   | 6914/9864 [12:29:50<12:40:16, 15.46s/it]
 70%|███████   | 6915/9864 [12:30:04<12:16:12, 14.98s/it]
 70%|███████   | 6916/9864 [12:30:20<12:24:48, 15.16s/it]
 70%|███████   | 6917/9864 [12:30:34<12:07:24, 14.81s/it]
 70%|███████   | 6918/9864 [12:30:48<11:56:42, 14.60s/it]
 70%|███████   | 6919/9864 [12:31:03<11:59:40, 14.66s/it]
 70%|███████   | 6920/9864 [12:31:19<12:18:13, 15.05s/it]
                                                         
{'loss': '0.08823', 'grad_norm': '4.899', 'learning_rate': '4.322e-06', 'epoch': '0.7016'}

 70%|███████   | 6920/9864 [12:31:19<12:18:13, 15.05s/it]
 70%|███████   | 6921/9864 [12:31:35<12:35:42, 15.41s/it]
 70%|███████   | 6922/9864 [12:31:50<12:36:22, 15.43s/it]
 70%|███████   | 6923/9864 [12:32:05<12:16:58, 15.04s/it]
 70%|███████   | 6924/9864 [12:32:20<12:26:51, 15.24s/it]
 70%|███████   | 6925/9864 [12:32:34<12:07:14, 14.85s/it]
 70%|███████   | 6926/9864 [12:32:48<11:52:47, 14.56s/it]
 70%|███████   | 6927/9864 [12:33:04<12:06:01, 14.83s/it]
 70%|███████   | 6928/9864 [12:33:18<11:55:33, 14.62s/it]
 70%|███████   | 6929/9864 [12:33:34<12:14:04, 15.01s/it]
 70%|███████   | 6930/9864 [12:33:50<12:27:09, 15.28s/it]
                                                         
{'loss': '0.09153', 'grad_norm': '4.018', 'learning_rate': '4.295e-06', 'epoch': '0.7026'}

 70%|███████   | 6930/9864 [12:33:50<12:27:09, 15.28s/it]
 70%|███████   | 6931/9864 [12:34:04<12:14:20, 15.02s/it]
 70%|███████   | 6932/9864 [12:34:18<12:06:19, 14.86s/it]
 70%|███████   | 6933/9864 [12:34:34<12:15:58, 15.07s/it]
 70%|███████   | 6934/9864 [12:34:49<12:20:27, 15.16s/it]
 70%|███████   | 6935/9864 [12:35:04<12:05:29, 14.86s/it]
 70%|███████   | 6936/9864 [12:35:19<12:19:05, 15.15s/it]
 70%|███████   | 6937/9864 [12:35:35<12:25:51, 15.29s/it]
 70%|███████   | 6938/9864 [12:35:49<12:07:43, 14.92s/it]
 70%|███████   | 6939/9864 [12:36:03<11:51:02, 14.59s/it]
 70%|███████   | 6940/9864 [12:36:19<12:06:59, 14.92s/it]
                                                         
{'loss': '0.08837', 'grad_norm': '5.045', 'learning_rate': '4.268e-06', 'epoch': '0.7036'}

 70%|███████   | 6940/9864 [12:36:19<12:06:59, 14.92s/it]
 70%|███████   | 6941/9864 [12:36:33<11:53:17, 14.64s/it]
 70%|███████   | 6942/9864 [12:36:49<12:17:15, 15.14s/it]
 70%|███████   | 6943/9864 [12:37:03<12:02:32, 14.84s/it]
 70%|███████   | 6944/9864 [12:37:20<12:40:24, 15.62s/it]
 70%|███████   | 6945/9864 [12:37:34<12:08:11, 14.97s/it]
 70%|███████   | 6946/9864 [12:37:48<11:50:59, 14.62s/it]
 70%|███████   | 6947/9864 [12:38:01<11:36:39, 14.33s/it]
 70%|███████   | 6948/9864 [12:38:17<11:55:26, 14.72s/it]
 70%|███████   | 6949/9864 [12:38:31<11:49:07, 14.60s/it]
 70%|███████   | 6950/9864 [12:38:47<12:00:40, 14.84s/it]
                                                         
{'loss': '0.09176', 'grad_norm': '4.508', 'learning_rate': '4.241e-06', 'epoch': '0.7046'}

 70%|███████   | 6950/9864 [12:38:47<12:00:40, 14.84s/it]
 70%|███████   | 6951/9864 [12:39:05<12:44:17, 15.74s/it]
 70%|███████   | 6952/9864 [12:39:18<12:15:04, 15.15s/it]
 70%|███████   | 6953/9864 [12:39:33<12:05:40, 14.96s/it]
 70%|███████   | 6954/9864 [12:39:47<12:00:54, 14.86s/it]
 71%|███████   | 6955/9864 [12:40:01<11:44:53, 14.54s/it]
 71%|███████   | 6956/9864 [12:40:17<12:07:51, 15.02s/it]
 71%|███████   | 6957/9864 [12:40:31<11:54:34, 14.75s/it]
 71%|███████   | 6958/9864 [12:40:47<12:07:10, 15.01s/it]
 71%|███████   | 6959/9864 [12:41:04<12:41:16, 15.72s/it]
 71%|███████   | 6960/9864 [12:41:19<12:20:14, 15.29s/it]
                                                         
{'loss': '0.09094', 'grad_norm': '4.99', 'learning_rate': '4.214e-06', 'epoch': '0.7057'}

 71%|███████   | 6960/9864 [12:41:19<12:20:14, 15.29s/it]
 71%|███████   | 6961/9864 [12:41:33<11:59:29, 14.87s/it]
 71%|███████   | 6962/9864 [12:41:47<11:49:48, 14.68s/it]
 71%|███████   | 6963/9864 [12:42:01<11:40:16, 14.48s/it]
 71%|███████   | 6964/9864 [12:42:17<12:03:55, 14.98s/it]
 71%|███████   | 6965/9864 [12:42:31<11:52:45, 14.75s/it]
 71%|███████   | 6966/9864 [12:42:47<12:09:55, 15.11s/it]
 71%|███████   | 6967/9864 [12:43:04<12:40:38, 15.75s/it]
 71%|███████   | 6968/9864 [12:43:19<12:18:12, 15.29s/it]
 71%|███████   | 6969/9864 [12:43:32<11:56:22, 14.85s/it]
 71%|███████   | 6970/9864 [12:43:46<11:43:46, 14.59s/it]
                                                         
{'loss': '0.09258', 'grad_norm': '4.947', 'learning_rate': '4.188e-06', 'epoch': '0.7067'}

 71%|███████   | 6970/9864 [12:43:46<11:43:46, 14.59s/it]
 71%|███████   | 6971/9864 [12:44:02<12:00:10, 14.94s/it]
 71%|███████   | 6972/9864 [12:44:16<11:42:46, 14.58s/it]
 71%|███████   | 6973/9864 [12:44:30<11:38:55, 14.51s/it]
 71%|███████   | 6974/9864 [12:44:46<12:00:49, 14.97s/it]
 71%|███████   | 6975/9864 [12:45:04<12:39:38, 15.78s/it]
 71%|███████   | 6976/9864 [12:45:18<12:16:57, 15.31s/it]
 71%|███████   | 6977/9864 [12:45:32<11:54:32, 14.85s/it]
 71%|███████   | 6978/9864 [12:45:46<11:44:29, 14.65s/it]
 71%|███████   | 6979/9864 [12:46:02<11:54:32, 14.86s/it]
 71%|███████   | 6980/9864 [12:46:16<11:44:07, 14.65s/it]
                                                         
{'loss': '0.1074', 'grad_norm': '4.399', 'learning_rate': '4.161e-06', 'epoch': '0.7077'}

 71%|███████   | 6980/9864 [12:46:16<11:44:07, 14.65s/it]
 71%|███████   | 6981/9864 [12:46:30<11:34:11, 14.45s/it]
 71%|███████   | 6982/9864 [12:46:45<11:50:29, 14.79s/it]
 71%|███████   | 6983/9864 [12:47:01<12:05:48, 15.12s/it]
 71%|███████   | 6984/9864 [12:47:17<12:13:19, 15.28s/it]
 71%|███████   | 6985/9864 [12:47:31<12:01:06, 15.03s/it]
 71%|███████   | 6986/9864 [12:47:46<11:52:12, 14.85s/it]
 71%|███████   | 6987/9864 [12:48:01<12:04:25, 15.11s/it]
 71%|███████   | 6988/9864 [12:48:16<11:50:40, 14.83s/it]
 71%|███████   | 6989/9864 [12:48:31<12:02:13, 15.07s/it]
 71%|███████   | 6990/9864 [12:48:45<11:48:56, 14.80s/it]
                                                         
{'loss': '0.08526', 'grad_norm': '3.217', 'learning_rate': '4.134e-06', 'epoch': '0.7087'}

 71%|███████   | 6990/9864 [12:48:45<11:48:56, 14.80s/it]
 71%|███████   | 6991/9864 [12:49:01<12:04:45, 15.14s/it]
 71%|███████   | 6992/9864 [12:49:17<12:17:10, 15.40s/it]
 71%|███████   | 6993/9864 [12:49:32<12:02:07, 15.09s/it]
 71%|███████   | 6994/9864 [12:49:48<12:14:14, 15.35s/it]
 71%|███████   | 6995/9864 [12:50:02<11:57:39, 15.01s/it]
 71%|███████   | 6996/9864 [12:50:16<11:48:59, 14.83s/it]
 71%|███████   | 6997/9864 [12:50:32<12:05:15, 15.18s/it]
 71%|███████   | 6998/9864 [12:50:46<11:46:04, 14.78s/it]
 71%|███████   | 6999/9864 [12:51:02<12:02:44, 15.14s/it]
 71%|███████   | 7000/9864 [12:51:17<11:58:37, 15.05s/it]
                                                         
{'loss': '0.09613', 'grad_norm': '4.897', 'learning_rate': '4.108e-06', 'epoch': '0.7097'}

 71%|███████   | 7000/9864 [12:51:17<11:58:37, 15.05s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:10<00:00, 10.06s/it]
Writing model shards: 100%|██████████| 1/1 [00:10<00:00, 10.06s/it]
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 71%|███████   | 7001/9864 [12:52:32<26:10:27, 32.91s/it]
 71%|███████   | 7002/9864 [12:52:46<21:43:10, 27.32s/it]
 71%|███████   | 7003/9864 [12:53:00<18:32:52, 23.34s/it]
 71%|███████   | 7004/9864 [12:53:13<16:12:07, 20.39s/it]
 71%|███████   | 7005/9864 [12:53:29<15:02:35, 18.94s/it]
 71%|███████   | 7006/9864 [12:53:43<13:57:49, 17.59s/it]
 71%|███████   | 7007/9864 [12:54:00<13:38:06, 17.18s/it]
 71%|███████   | 7008/9864 [12:54:16<13:21:38, 16.84s/it]
 71%|███████   | 7009/9864 [12:54:30<12:42:13, 16.02s/it]
 71%|███████   | 7010/9864 [12:54:46<12:43:34, 16.05s/it]
                                                         
{'loss': '0.08253', 'grad_norm': '4.323', 'learning_rate': '4.081e-06', 'epoch': '0.7107'}

 71%|███████   | 7010/9864 [12:54:46<12:43:34, 16.05s/it]
 71%|███████   | 7011/9864 [12:55:00<12:15:22, 15.47s/it]
 71%|███████   | 7012/9864 [12:55:14<11:55:56, 15.06s/it]
 71%|███████   | 7013/9864 [12:55:29<11:59:10, 15.14s/it]
 71%|███████   | 7014/9864 [12:55:44<11:53:40, 15.02s/it]
 71%|███████   | 7015/9864 [12:56:00<12:02:18, 15.21s/it]
 71%|███████   | 7016/9864 [12:56:16<12:11:49, 15.42s/it]
 71%|███████   | 7017/9864 [12:56:30<11:53:20, 15.03s/it]
 71%|███████   | 7018/9864 [12:56:46<12:07:52, 15.35s/it]
 71%|███████   | 7019/9864 [12:57:00<11:46:51, 14.91s/it]
 71%|███████   | 7020/9864 [12:57:16<12:02:34, 15.24s/it]
                                                         
{'loss': '0.09129', 'grad_norm': '4.653', 'learning_rate': '4.055e-06', 'epoch': '0.7117'}

 71%|███████   | 7020/9864 [12:57:16<12:02:34, 15.24s/it]
 71%|███████   | 7021/9864 [12:57:30<11:47:56, 14.94s/it]
 71%|███████   | 7022/9864 [12:57:44<11:39:37, 14.77s/it]
 71%|███████   | 7023/9864 [12:58:02<12:14:42, 15.52s/it]
 71%|███████   | 7024/9864 [12:58:15<11:46:24, 14.92s/it]
 71%|███████   | 7025/9864 [12:58:31<11:53:03, 15.07s/it]
 71%|███████   | 7026/9864 [12:58:45<11:36:30, 14.73s/it]
 71%|███████   | 7027/9864 [12:58:59<11:26:16, 14.51s/it]
 71%|███████   | 7028/9864 [12:59:15<11:49:16, 15.01s/it]
 71%|███████▏  | 7029/9864 [12:59:29<11:37:38, 14.76s/it]
 71%|███████▏  | 7030/9864 [12:59:45<11:57:02, 15.18s/it]
                                                         
{'loss': '0.0869', 'grad_norm': '5.489', 'learning_rate': '4.029e-06', 'epoch': '0.7128'}

 71%|███████▏  | 7030/9864 [12:59:45<11:57:02, 15.18s/it]
 71%|███████▏  | 7031/9864 [12:59:59<11:41:39, 14.86s/it]
 71%|███████▏  | 7032/9864 [13:00:17<12:18:52, 15.65s/it]
 71%|███████▏  | 7033/9864 [13:00:31<11:58:38, 15.23s/it]
 71%|███████▏  | 7034/9864 [13:00:45<11:41:29, 14.87s/it]
 71%|███████▏  | 7035/9864 [13:00:59<11:25:47, 14.54s/it]
 71%|███████▏  | 7036/9864 [13:01:15<11:46:26, 14.99s/it]
 71%|███████▏  | 7037/9864 [13:01:28<11:27:11, 14.58s/it]
 71%|███████▏  | 7038/9864 [13:01:44<11:45:33, 14.98s/it]
 71%|███████▏  | 7039/9864 [13:02:00<11:55:19, 15.19s/it]
 71%|███████▏  | 7040/9864 [13:02:16<12:07:17, 15.45s/it]
                                                         
{'loss': '0.101', 'grad_norm': '5.702', 'learning_rate': '4.002e-06', 'epoch': '0.7138'}

 71%|███████▏  | 7040/9864 [13:02:16<12:07:17, 15.45s/it]
 71%|███████▏  | 7041/9864 [13:02:30<11:41:54, 14.92s/it]
 71%|███████▏  | 7042/9864 [13:02:44<11:27:06, 14.61s/it]
 71%|███████▏  | 7043/9864 [13:02:58<11:20:45, 14.48s/it]
 71%|███████▏  | 7044/9864 [13:03:14<11:43:16, 14.96s/it]
 71%|███████▏  | 7045/9864 [13:03:28<11:35:27, 14.80s/it]
 71%|███████▏  | 7046/9864 [13:03:44<11:44:10, 14.99s/it]
 71%|███████▏  | 7047/9864 [13:03:59<11:48:51, 15.10s/it]
 71%|███████▏  | 7048/9864 [13:04:15<11:53:43, 15.21s/it]
 71%|███████▏  | 7049/9864 [13:04:29<11:37:57, 14.88s/it]
 71%|███████▏  | 7050/9864 [13:04:43<11:27:29, 14.66s/it]
                                                         
{'loss': '0.09978', 'grad_norm': '4.338', 'learning_rate': '3.976e-06', 'epoch': '0.7148'}

 71%|███████▏  | 7050/9864 [13:04:43<11:27:29, 14.66s/it]
 71%|███████▏  | 7051/9864 [13:04:56<11:11:32, 14.32s/it]
 71%|███████▏  | 7052/9864 [13:05:12<11:32:05, 14.77s/it]
 72%|███████▏  | 7053/9864 [13:05:26<11:20:58, 14.54s/it]
 72%|███████▏  | 7054/9864 [13:05:42<11:37:53, 14.90s/it]
 72%|███████▏  | 7055/9864 [13:05:59<12:10:11, 15.60s/it]
 72%|███████▏  | 7056/9864 [13:06:13<11:49:20, 15.16s/it]
 72%|███████▏  | 7057/9864 [13:06:28<11:38:15, 14.93s/it]
 72%|███████▏  | 7058/9864 [13:06:42<11:30:01, 14.75s/it]
 72%|███████▏  | 7059/9864 [13:06:56<11:20:15, 14.55s/it]
 72%|███████▏  | 7060/9864 [13:07:12<11:43:16, 15.05s/it]
                                                         
{'loss': '0.08631', 'grad_norm': '5.831', 'learning_rate': '3.95e-06', 'epoch': '0.7158'}

 72%|███████▏  | 7060/9864 [13:07:12<11:43:16, 15.05s/it]
 72%|███████▏  | 7061/9864 [13:07:27<11:32:12, 14.82s/it]
 72%|███████▏  | 7062/9864 [13:07:44<12:13:36, 15.71s/it]
 72%|███████▏  | 7063/9864 [13:08:00<12:12:08, 15.68s/it]
 72%|███████▏  | 7064/9864 [13:08:14<11:55:11, 15.33s/it]
 72%|███████▏  | 7065/9864 [13:08:29<11:40:36, 15.02s/it]
 72%|███████▏  | 7066/9864 [13:08:43<11:31:15, 14.82s/it]
 72%|███████▏  | 7067/9864 [13:08:59<11:42:28, 15.07s/it]
 72%|███████▏  | 7068/9864 [13:09:13<11:33:29, 14.88s/it]
 72%|███████▏  | 7069/9864 [13:09:27<11:19:40, 14.59s/it]
 72%|███████▏  | 7070/9864 [13:09:45<12:03:26, 15.54s/it]
                                                         
{'loss': '0.09365', 'grad_norm': '5.2', 'learning_rate': '3.924e-06', 'epoch': '0.7168'}

 72%|███████▏  | 7070/9864 [13:09:45<12:03:26, 15.54s/it]
 72%|███████▏  | 7071/9864 [13:10:01<12:05:29, 15.59s/it]
 72%|███████▏  | 7072/9864 [13:10:15<11:41:58, 15.09s/it]
 72%|███████▏  | 7073/9864 [13:10:29<11:30:35, 14.85s/it]
 72%|███████▏  | 7074/9864 [13:10:44<11:40:37, 15.07s/it]
 72%|███████▏  | 7075/9864 [13:10:59<11:31:02, 14.87s/it]
 72%|███████▏  | 7076/9864 [13:11:13<11:23:12, 14.70s/it]
 72%|███████▏  | 7077/9864 [13:11:29<11:43:32, 15.15s/it]
 72%|███████▏  | 7078/9864 [13:11:45<11:51:29, 15.32s/it]
 72%|███████▏  | 7079/9864 [13:12:00<11:51:28, 15.33s/it]
 72%|███████▏  | 7080/9864 [13:12:15<11:39:19, 15.07s/it]
                                                         
{'loss': '0.1023', 'grad_norm': '5.152', 'learning_rate': '3.898e-06', 'epoch': '0.7178'}

 72%|███████▏  | 7080/9864 [13:12:15<11:39:19, 15.07s/it]
 72%|███████▏  | 7081/9864 [13:12:29<11:26:26, 14.80s/it]
 72%|███████▏  | 7082/9864 [13:12:45<11:39:00, 15.08s/it]
 72%|███████▏  | 7083/9864 [13:12:59<11:30:57, 14.91s/it]
 72%|███████▏  | 7084/9864 [13:13:13<11:21:31, 14.71s/it]
 72%|███████▏  | 7085/9864 [13:13:30<11:42:36, 15.17s/it]
 72%|███████▏  | 7086/9864 [13:13:46<11:53:24, 15.41s/it]
 72%|███████▏  | 7087/9864 [13:14:00<11:37:06, 15.06s/it]
 72%|███████▏  | 7088/9864 [13:14:14<11:18:27, 14.66s/it]
 72%|███████▏  | 7089/9864 [13:14:28<11:11:25, 14.52s/it]
 72%|███████▏  | 7090/9864 [13:14:44<11:34:12, 15.02s/it]
                                                         
{'loss': '0.08385', 'grad_norm': '5.26', 'learning_rate': '3.872e-06', 'epoch': '0.7188'}

 72%|███████▏  | 7090/9864 [13:14:44<11:34:12, 15.02s/it]
 72%|███████▏  | 7091/9864 [13:14:58<11:23:48, 14.80s/it]
 72%|███████▏  | 7092/9864 [13:15:13<11:18:56, 14.70s/it]
 72%|███████▏  | 7093/9864 [13:15:29<11:39:33, 15.15s/it]
 72%|███████▏  | 7094/9864 [13:15:46<12:07:30, 15.76s/it]
 72%|███████▏  | 7095/9864 [13:16:01<11:49:56, 15.38s/it]
 72%|███████▏  | 7096/9864 [13:16:15<11:33:35, 15.03s/it]
 72%|███████▏  | 7097/9864 [13:16:28<11:12:26, 14.58s/it]
 72%|███████▏  | 7098/9864 [13:16:44<11:30:40, 14.98s/it]
 72%|███████▏  | 7099/9864 [13:16:58<11:16:38, 14.68s/it]
 72%|███████▏  | 7100/9864 [13:17:14<11:26:11, 14.90s/it]
                                                         
{'loss': '0.1036', 'grad_norm': '3.699', 'learning_rate': '3.846e-06', 'epoch': '0.7198'}

 72%|███████▏  | 7100/9864 [13:17:14<11:26:11, 14.90s/it]
 72%|███████▏  | 7101/9864 [13:17:27<11:09:17, 14.53s/it]
 72%|███████▏  | 7102/9864 [13:17:45<11:48:07, 15.38s/it]
 72%|███████▏  | 7103/9864 [13:17:59<11:34:26, 15.09s/it]
 72%|███████▏  | 7104/9864 [13:18:13<11:13:58, 14.65s/it]
 72%|███████▏  | 7105/9864 [13:18:28<11:25:21, 14.90s/it]
 72%|███████▏  | 7106/9864 [13:18:42<11:09:41, 14.57s/it]
 72%|███████▏  | 7107/9864 [13:18:56<11:04:25, 14.46s/it]
 72%|███████▏  | 7108/9864 [13:19:12<11:20:05, 14.81s/it]
 72%|███████▏  | 7109/9864 [13:19:28<11:31:07, 15.05s/it]
 72%|███████▏  | 7110/9864 [13:19:43<11:40:52, 15.27s/it]
                                                         
{'loss': '0.1027', 'grad_norm': '3.357', 'learning_rate': '3.82e-06', 'epoch': '0.7209'}

 72%|███████▏  | 7110/9864 [13:19:43<11:40:52, 15.27s/it]
 72%|███████▏  | 7111/9864 [13:19:58<11:30:37, 15.05s/it]
 72%|███████▏  | 7112/9864 [13:20:12<11:22:33, 14.88s/it]
 72%|███████▏  | 7113/9864 [13:20:28<11:35:22, 15.17s/it]
 72%|███████▏  | 7114/9864 [13:20:42<11:16:29, 14.76s/it]
 72%|███████▏  | 7115/9864 [13:20:56<11:10:13, 14.63s/it]
 72%|███████▏  | 7116/9864 [13:21:12<11:28:53, 15.04s/it]
 72%|███████▏  | 7117/9864 [13:21:26<11:13:11, 14.70s/it]
 72%|███████▏  | 7118/9864 [13:21:43<11:47:55, 15.47s/it]
 72%|███████▏  | 7119/9864 [13:21:57<11:22:07, 14.91s/it]
 72%|███████▏  | 7120/9864 [13:22:13<11:38:31, 15.27s/it]
                                                         
{'loss': '0.09129', 'grad_norm': '3.791', 'learning_rate': '3.794e-06', 'epoch': '0.7219'}

 72%|███████▏  | 7120/9864 [13:22:13<11:38:31, 15.27s/it]
 72%|███████▏  | 7121/9864 [13:22:28<11:25:37, 15.00s/it]
 72%|███████▏  | 7122/9864 [13:22:42<11:16:37, 14.81s/it]
 72%|███████▏  | 7123/9864 [13:22:56<11:05:39, 14.57s/it]
 72%|███████▏  | 7124/9864 [13:23:12<11:22:38, 14.95s/it]
 72%|███████▏  | 7125/9864 [13:23:28<11:33:58, 15.20s/it]
 72%|███████▏  | 7126/9864 [13:23:42<11:22:11, 14.95s/it]
 72%|███████▏  | 7127/9864 [13:23:56<11:12:20, 14.74s/it]
 72%|███████▏  | 7128/9864 [13:24:11<11:19:38, 14.90s/it]
 72%|███████▏  | 7129/9864 [13:24:26<11:12:55, 14.76s/it]
 72%|███████▏  | 7130/9864 [13:24:40<11:04:53, 14.59s/it]
                                                         
{'loss': '0.07868', 'grad_norm': '3.572', 'learning_rate': '3.768e-06', 'epoch': '0.7229'}

 72%|███████▏  | 7130/9864 [13:24:40<11:04:53, 14.59s/it]
 72%|███████▏  | 7131/9864 [13:24:55<11:03:39, 14.57s/it]
 72%|███████▏  | 7132/9864 [13:25:11<11:23:18, 15.01s/it]
 72%|███████▏  | 7133/9864 [13:25:28<11:57:31, 15.76s/it]
 72%|███████▏  | 7134/9864 [13:25:42<11:35:52, 15.29s/it]
 72%|███████▏  | 7135/9864 [13:25:57<11:21:00, 14.97s/it]
 72%|███████▏  | 7136/9864 [13:26:12<11:32:19, 15.23s/it]
 72%|███████▏  | 7137/9864 [13:26:27<11:19:03, 14.94s/it]
 72%|███████▏  | 7138/9864 [13:26:41<11:06:03, 14.66s/it]
 72%|███████▏  | 7139/9864 [13:26:57<11:23:13, 15.04s/it]
 72%|███████▏  | 7140/9864 [13:27:13<11:37:00, 15.35s/it]
                                                         
{'loss': '0.1009', 'grad_norm': '4.866', 'learning_rate': '3.743e-06', 'epoch': '0.7239'}

 72%|███████▏  | 7140/9864 [13:27:13<11:37:00, 15.35s/it]
 72%|███████▏  | 7141/9864 [13:27:28<11:35:59, 15.34s/it]
 72%|███████▏  | 7142/9864 [13:27:42<11:19:29, 14.98s/it]
 72%|███████▏  | 7143/9864 [13:27:56<11:07:46, 14.73s/it]
 72%|███████▏  | 7144/9864 [13:28:12<11:27:19, 15.16s/it]
 72%|███████▏  | 7145/9864 [13:28:27<11:14:59, 14.90s/it]
 72%|███████▏  | 7146/9864 [13:28:43<11:29:38, 15.22s/it]
 72%|███████▏  | 7147/9864 [13:28:57<11:14:37, 14.90s/it]
 72%|███████▏  | 7148/9864 [13:29:14<11:50:45, 15.70s/it]
 72%|███████▏  | 7149/9864 [13:29:29<11:31:04, 15.27s/it]
 72%|███████▏  | 7150/9864 [13:29:43<11:11:30, 14.85s/it]
                                                         
{'loss': '0.09332', 'grad_norm': '4.763', 'learning_rate': '3.717e-06', 'epoch': '0.7249'}

 72%|███████▏  | 7150/9864 [13:29:43<11:11:30, 14.85s/it]
 72%|███████▏  | 7151/9864 [13:29:58<11:23:58, 15.13s/it]
 73%|███████▎  | 7152/9864 [13:30:12<11:08:03, 14.78s/it]
 73%|███████▎  | 7153/9864 [13:30:26<10:53:39, 14.47s/it]
 73%|███████▎  | 7154/9864 [13:30:42<11:10:19, 14.84s/it]
 73%|███████▎  | 7155/9864 [13:30:58<11:24:44, 15.17s/it]
 73%|███████▎  | 7156/9864 [13:31:14<11:36:01, 15.42s/it]
 73%|███████▎  | 7157/9864 [13:31:28<11:16:45, 15.00s/it]
 73%|███████▎  | 7158/9864 [13:31:42<11:04:11, 14.73s/it]
 73%|███████▎  | 7159/9864 [13:31:57<11:15:54, 14.99s/it]
 73%|███████▎  | 7160/9864 [13:32:11<11:02:29, 14.70s/it]
                                                         
{'loss': '0.08249', 'grad_norm': '2.614', 'learning_rate': '3.692e-06', 'epoch': '0.7259'}

 73%|███████▎  | 7160/9864 [13:32:11<11:02:29, 14.70s/it]
 73%|███████▎  | 7161/9864 [13:32:27<11:20:26, 15.10s/it]
 73%|███████▎  | 7162/9864 [13:32:42<11:06:58, 14.81s/it]
 73%|███████▎  | 7163/9864 [13:32:58<11:27:01, 15.26s/it]
 73%|███████▎  | 7164/9864 [13:33:14<11:33:09, 15.40s/it]
 73%|███████▎  | 7165/9864 [13:33:28<11:13:15, 14.97s/it]
 73%|███████▎  | 7166/9864 [13:33:43<11:21:13, 15.15s/it]
 73%|███████▎  | 7167/9864 [13:33:58<11:13:21, 14.98s/it]
 73%|███████▎  | 7168/9864 [13:34:12<11:00:14, 14.69s/it]
 73%|███████▎  | 7169/9864 [13:34:27<11:13:22, 14.99s/it]
 73%|███████▎  | 7170/9864 [13:34:42<11:04:38, 14.80s/it]
                                                         
{'loss': '0.08766', 'grad_norm': '4.925', 'learning_rate': '3.666e-06', 'epoch': '0.7269'}

 73%|███████▎  | 7170/9864 [13:34:42<11:04:38, 14.80s/it]
 73%|███████▎  | 7171/9864 [13:34:55<10:48:54, 14.46s/it]
 73%|███████▎  | 7172/9864 [13:35:13<11:26:44, 15.31s/it]
 73%|███████▎  | 7173/9864 [13:35:27<11:14:23, 15.04s/it]
 73%|███████▎  | 7174/9864 [13:35:43<11:22:09, 15.22s/it]
 73%|███████▎  | 7175/9864 [13:35:57<11:11:08, 14.98s/it]
 73%|███████▎  | 7176/9864 [13:36:12<11:04:05, 14.82s/it]
 73%|███████▎  | 7177/9864 [13:36:27<11:16:43, 15.11s/it]
 73%|███████▎  | 7178/9864 [13:36:42<11:03:06, 14.81s/it]
 73%|███████▎  | 7179/9864 [13:36:57<11:08:13, 14.93s/it]
 73%|███████▎  | 7180/9864 [13:37:13<11:20:49, 15.22s/it]
                                                         
{'loss': '0.09576', 'grad_norm': '5.87', 'learning_rate': '3.641e-06', 'epoch': '0.728'}

 73%|███████▎  | 7180/9864 [13:37:13<11:20:49, 15.22s/it]
 73%|███████▎  | 7181/9864 [13:37:27<11:04:08, 14.85s/it]
 73%|███████▎  | 7182/9864 [13:37:43<11:18:30, 15.18s/it]
 73%|███████▎  | 7183/9864 [13:37:56<10:58:31, 14.74s/it]
 73%|███████▎  | 7184/9864 [13:38:11<10:50:47, 14.57s/it]
 73%|███████▎  | 7185/9864 [13:38:26<11:02:58, 14.85s/it]
 73%|███████▎  | 7186/9864 [13:38:42<11:21:54, 15.28s/it]
 73%|███████▎  | 7187/9864 [13:38:56<11:02:04, 14.84s/it]
 73%|███████▎  | 7188/9864 [13:39:11<11:08:46, 14.99s/it]
 73%|███████▎  | 7189/9864 [13:39:25<10:54:37, 14.68s/it]
 73%|███████▎  | 7190/9864 [13:39:41<11:05:25, 14.93s/it]
                                                         
{'loss': '0.08826', 'grad_norm': '4.794', 'learning_rate': '3.616e-06', 'epoch': '0.729'}

 73%|███████▎  | 7190/9864 [13:39:41<11:05:25, 14.93s/it]
 73%|███████▎  | 7191/9864 [13:39:55<10:51:00, 14.61s/it]
 73%|███████▎  | 7192/9864 [13:40:11<11:11:15, 15.07s/it]
 73%|███████▎  | 7193/9864 [13:40:26<11:05:26, 14.95s/it]
 73%|███████▎  | 7194/9864 [13:40:41<11:17:03, 15.21s/it]
 73%|███████▎  | 7195/9864 [13:40:56<11:07:02, 15.00s/it]
 73%|███████▎  | 7196/9864 [13:41:10<10:54:21, 14.72s/it]
 73%|███████▎  | 7197/9864 [13:41:26<11:11:16, 15.10s/it]
 73%|███████▎  | 7198/9864 [13:41:42<11:25:10, 15.42s/it]
 73%|███████▎  | 7199/9864 [13:41:56<11:07:32, 15.03s/it]
 73%|███████▎  | 7200/9864 [13:42:12<11:15:17, 15.21s/it]
                                                         
{'loss': '0.08045', 'grad_norm': '2.408', 'learning_rate': '3.59e-06', 'epoch': '0.73'}

 73%|███████▎  | 7200/9864 [13:42:12<11:15:17, 15.21s/it]
 73%|███████▎  | 7201/9864 [13:42:26<11:04:53, 14.98s/it]
 73%|███████▎  | 7202/9864 [13:42:40<10:46:19, 14.57s/it]
 73%|███████▎  | 7203/9864 [13:42:56<11:08:24, 15.07s/it]
 73%|███████▎  | 7204/9864 [13:43:11<11:02:12, 14.94s/it]
 73%|███████▎  | 7205/9864 [13:43:27<11:15:04, 15.23s/it]
 73%|███████▎  | 7206/9864 [13:43:43<11:26:20, 15.49s/it]
 73%|███████▎  | 7207/9864 [13:43:57<11:09:21, 15.12s/it]
 73%|███████▎  | 7208/9864 [13:44:13<11:22:45, 15.42s/it]
 73%|███████▎  | 7209/9864 [13:44:27<11:03:38, 15.00s/it]
 73%|███████▎  | 7210/9864 [13:44:41<10:50:18, 14.70s/it]
                                                         
{'loss': '0.0752', 'grad_norm': '5.708', 'learning_rate': '3.565e-06', 'epoch': '0.731'}

 73%|███████▎  | 7210/9864 [13:44:41<10:50:18, 14.70s/it]
 73%|███████▎  | 7211/9864 [13:44:57<11:10:00, 15.15s/it]
 73%|███████▎  | 7212/9864 [13:45:12<10:58:29, 14.90s/it]
 73%|███████▎  | 7213/9864 [13:45:27<11:07:03, 15.10s/it]
 73%|███████▎  | 7214/9864 [13:45:41<10:52:33, 14.78s/it]
 73%|███████▎  | 7215/9864 [13:45:57<11:01:09, 14.98s/it]
 73%|███████▎  | 7216/9864 [13:46:13<11:12:00, 15.23s/it]
 73%|███████▎  | 7217/9864 [13:46:27<10:58:53, 14.94s/it]
 73%|███████▎  | 7218/9864 [13:46:40<10:41:02, 14.54s/it]
 73%|███████▎  | 7219/9864 [13:46:56<10:56:17, 14.89s/it]
 73%|███████▎  | 7220/9864 [13:47:10<10:44:52, 14.63s/it]
                                                         
{'loss': '0.07066', 'grad_norm': '4.25', 'learning_rate': '3.54e-06', 'epoch': '0.732'}

 73%|███████▎  | 7220/9864 [13:47:10<10:44:52, 14.63s/it]
 73%|███████▎  | 7221/9864 [13:47:24<10:37:24, 14.47s/it]
 73%|███████▎  | 7222/9864 [13:47:40<10:51:21, 14.79s/it]
 73%|███████▎  | 7223/9864 [13:47:55<10:59:06, 14.97s/it]
 73%|███████▎  | 7224/9864 [13:48:11<11:07:21, 15.17s/it]
 73%|███████▎  | 7225/9864 [13:48:24<10:45:50, 14.68s/it]
 73%|███████▎  | 7226/9864 [13:48:38<10:36:49, 14.48s/it]
 73%|███████▎  | 7227/9864 [13:48:54<10:51:27, 14.82s/it]
 73%|███████▎  | 7228/9864 [13:49:08<10:42:39, 14.63s/it]
 73%|███████▎  | 7229/9864 [13:49:24<11:03:34, 15.11s/it]
 73%|███████▎  | 7230/9864 [13:49:40<11:14:00, 15.35s/it]
                                                         
{'loss': '0.09', 'grad_norm': '6.374', 'learning_rate': '3.515e-06', 'epoch': '0.733'}

 73%|███████▎  | 7230/9864 [13:49:40<11:14:00, 15.35s/it]
 73%|███████▎  | 7231/9864 [13:49:56<11:21:59, 15.54s/it]
 73%|███████▎  | 7232/9864 [13:50:10<10:58:14, 15.01s/it]
 73%|███████▎  | 7233/9864 [13:50:24<10:46:09, 14.74s/it]
 73%|███████▎  | 7234/9864 [13:50:40<10:57:03, 14.99s/it]
 73%|███████▎  | 7235/9864 [13:50:53<10:40:00, 14.61s/it]
 73%|███████▎  | 7236/9864 [13:51:07<10:29:28, 14.37s/it]
 73%|███████▎  | 7237/9864 [13:51:23<10:49:20, 14.83s/it]
 73%|███████▎  | 7238/9864 [13:51:39<11:04:37, 15.19s/it]
 73%|███████▎  | 7239/9864 [13:51:55<11:10:40, 15.33s/it]
 73%|███████▎  | 7240/9864 [13:52:09<10:55:23, 14.99s/it]
                                                         
{'loss': '0.08212', 'grad_norm': '3.961', 'learning_rate': '3.49e-06', 'epoch': '0.734'}

 73%|███████▎  | 7240/9864 [13:52:09<10:55:23, 14.99s/it]
 73%|███████▎  | 7241/9864 [13:52:23<10:40:30, 14.65s/it]
 73%|███████▎  | 7242/9864 [13:52:39<10:57:02, 15.04s/it]
 73%|███████▎  | 7243/9864 [13:52:53<10:40:19, 14.66s/it]
 73%|███████▎  | 7244/9864 [13:53:07<10:36:56, 14.59s/it]
 73%|███████▎  | 7245/9864 [13:53:22<10:45:47, 14.79s/it]
 73%|███████▎  | 7246/9864 [13:53:39<11:05:35, 15.25s/it]
 73%|███████▎  | 7247/9864 [13:53:54<11:12:17, 15.41s/it]
 73%|███████▎  | 7248/9864 [13:54:09<10:57:08, 15.07s/it]
 73%|███████▎  | 7249/9864 [13:54:23<10:42:13, 14.74s/it]
 73%|███████▎  | 7250/9864 [13:54:39<10:57:43, 15.10s/it]
                                                         
{'loss': '0.08605', 'grad_norm': '4.625', 'learning_rate': '3.465e-06', 'epoch': '0.7351'}

 73%|███████▎  | 7250/9864 [13:54:39<10:57:43, 15.10s/it]
 74%|███████▎  | 7251/9864 [13:54:52<10:38:39, 14.67s/it]
 74%|███████▎  | 7252/9864 [13:55:09<10:58:57, 15.14s/it]
 74%|███████▎  | 7253/9864 [13:55:23<10:48:30, 14.90s/it]
 74%|███████▎  | 7254/9864 [13:55:38<10:44:29, 14.82s/it]
 74%|███████▎  | 7255/9864 [13:55:53<10:55:42, 15.08s/it]
 74%|███████▎  | 7256/9864 [13:56:07<10:39:10, 14.70s/it]
 74%|███████▎  | 7257/9864 [13:56:23<10:51:19, 14.99s/it]
 74%|███████▎  | 7258/9864 [13:56:37<10:43:42, 14.82s/it]
 74%|███████▎  | 7259/9864 [13:56:51<10:35:18, 14.63s/it]
 74%|███████▎  | 7260/9864 [13:57:07<10:46:52, 14.90s/it]
                                                         
{'loss': '0.0779', 'grad_norm': '4.988', 'learning_rate': '3.44e-06', 'epoch': '0.7361'}

 74%|███████▎  | 7260/9864 [13:57:07<10:46:52, 14.90s/it]
 74%|███████▎  | 7261/9864 [13:57:21<10:37:56, 14.70s/it]
 74%|███████▎  | 7262/9864 [13:57:35<10:33:12, 14.60s/it]
 74%|███████▎  | 7263/9864 [13:57:53<11:08:35, 15.42s/it]
 74%|███████▎  | 7264/9864 [13:58:07<10:49:46, 14.99s/it]
 74%|███████▎  | 7265/9864 [13:58:22<10:55:04, 15.12s/it]
 74%|███████▎  | 7266/9864 [13:58:36<10:38:42, 14.75s/it]
 74%|███████▎  | 7267/9864 [13:58:50<10:29:47, 14.55s/it]
 74%|███████▎  | 7268/9864 [13:59:04<10:25:43, 14.46s/it]
 74%|███████▎  | 7269/9864 [13:59:20<10:39:54, 14.80s/it]
 74%|███████▎  | 7270/9864 [13:59:36<10:52:48, 15.10s/it]
                                                         
{'loss': '0.07557', 'grad_norm': '2.873', 'learning_rate': '3.416e-06', 'epoch': '0.7371'}

 74%|███████▎  | 7270/9864 [13:59:36<10:52:48, 15.10s/it]
 74%|███████▎  | 7271/9864 [13:59:51<10:55:16, 15.16s/it]
 74%|███████▎  | 7272/9864 [14:00:06<10:57:25, 15.22s/it]
 74%|███████▎  | 7273/9864 [14:00:21<10:45:34, 14.95s/it]
 74%|███████▎  | 7274/9864 [14:00:35<10:34:51, 14.71s/it]
 74%|███████▍  | 7275/9864 [14:00:49<10:27:38, 14.55s/it]
 74%|███████▍  | 7276/9864 [14:01:05<10:39:47, 14.83s/it]
 74%|███████▍  | 7277/9864 [14:01:20<10:50:34, 15.09s/it]
 74%|███████▍  | 7278/9864 [14:01:36<10:59:22, 15.30s/it]
 74%|███████▍  | 7279/9864 [14:01:50<10:42:00, 14.90s/it]
 74%|███████▍  | 7280/9864 [14:02:04<10:32:45, 14.69s/it]
                                                         
{'loss': '0.0816', 'grad_norm': '5.644', 'learning_rate': '3.391e-06', 'epoch': '0.7381'}

 74%|███████▍  | 7280/9864 [14:02:04<10:32:45, 14.69s/it]
 74%|███████▍  | 7281/9864 [14:02:20<10:40:21, 14.87s/it]
 74%|███████▍  | 7282/9864 [14:02:33<10:27:15, 14.58s/it]
 74%|███████▍  | 7283/9864 [14:02:47<10:18:27, 14.38s/it]
 74%|███████▍  | 7284/9864 [14:03:02<10:19:17, 14.40s/it]
 74%|███████▍  | 7285/9864 [14:03:19<10:57:56, 15.31s/it]
 74%|███████▍  | 7286/9864 [14:03:35<11:08:56, 15.57s/it]
 74%|███████▍  | 7287/9864 [14:03:50<10:52:50, 15.20s/it]
 74%|███████▍  | 7288/9864 [14:04:05<10:51:47, 15.18s/it]
 74%|███████▍  | 7289/9864 [14:04:19<10:39:08, 14.89s/it]
 74%|███████▍  | 7290/9864 [14:04:33<10:25:21, 14.58s/it]
                                                         
{'loss': '0.07051', 'grad_norm': '4.323', 'learning_rate': '3.366e-06', 'epoch': '0.7391'}

 74%|███████▍  | 7290/9864 [14:04:33<10:25:21, 14.58s/it]
 74%|███████▍  | 7291/9864 [14:04:47<10:13:58, 14.32s/it]
 74%|███████▍  | 7292/9864 [14:05:02<10:32:29, 14.75s/it]
 74%|███████▍  | 7293/9864 [14:05:18<10:43:03, 15.01s/it]
 74%|███████▍  | 7294/9864 [14:05:34<11:00:23, 15.42s/it]
 74%|███████▍  | 7295/9864 [14:05:49<10:44:04, 15.04s/it]
 74%|███████▍  | 7296/9864 [14:06:03<10:30:50, 14.74s/it]
 74%|███████▍  | 7297/9864 [14:06:18<10:42:47, 15.02s/it]
 74%|███████▍  | 7298/9864 [14:06:33<10:33:30, 14.81s/it]
 74%|███████▍  | 7299/9864 [14:06:47<10:29:09, 14.72s/it]
 74%|███████▍  | 7300/9864 [14:07:04<10:53:35, 15.29s/it]
                                                         
{'loss': '0.08702', 'grad_norm': '4.073', 'learning_rate': '3.342e-06', 'epoch': '0.7401'}

 74%|███████▍  | 7300/9864 [14:07:04<10:53:35, 15.29s/it]
 74%|███████▍  | 7301/9864 [14:07:21<11:14:43, 15.80s/it]
 74%|███████▍  | 7302/9864 [14:07:35<10:53:32, 15.31s/it]
 74%|███████▍  | 7303/9864 [14:07:48<10:30:39, 14.78s/it]
 74%|███████▍  | 7304/9864 [14:08:03<10:23:59, 14.62s/it]
 74%|███████▍  | 7305/9864 [14:08:18<10:37:53, 14.96s/it]
 74%|███████▍  | 7306/9864 [14:08:32<10:26:10, 14.69s/it]
 74%|███████▍  | 7307/9864 [14:08:48<10:43:01, 15.09s/it]
 74%|███████▍  | 7308/9864 [14:09:04<10:50:07, 15.26s/it]
 74%|███████▍  | 7309/9864 [14:09:20<10:56:22, 15.41s/it]
 74%|███████▍  | 7310/9864 [14:09:35<10:48:13, 15.23s/it]
                                                         
{'loss': '0.09816', 'grad_norm': '4.8', 'learning_rate': '3.317e-06', 'epoch': '0.7411'}

 74%|███████▍  | 7310/9864 [14:09:35<10:48:13, 15.23s/it]
 74%|███████▍  | 7311/9864 [14:09:49<10:38:13, 15.00s/it]
 74%|███████▍  | 7312/9864 [14:10:03<10:28:22, 14.77s/it]
 74%|███████▍  | 7313/9864 [14:10:19<10:38:08, 15.01s/it]
 74%|███████▍  | 7314/9864 [14:10:33<10:24:53, 14.70s/it]
 74%|███████▍  | 7315/9864 [14:10:49<10:45:26, 15.19s/it]
 74%|███████▍  | 7316/9864 [14:11:06<10:58:20, 15.50s/it]
 74%|███████▍  | 7317/9864 [14:11:20<10:40:51, 15.10s/it]
 74%|███████▍  | 7318/9864 [14:11:35<10:44:51, 15.20s/it]
 74%|███████▍  | 7319/9864 [14:11:49<10:28:50, 14.83s/it]
 74%|███████▍  | 7320/9864 [14:12:04<10:28:24, 14.82s/it]
                                                         
{'loss': '0.08921', 'grad_norm': '5.891', 'learning_rate': '3.293e-06', 'epoch': '0.7422'}

 74%|███████▍  | 7320/9864 [14:12:04<10:28:24, 14.82s/it]
 74%|███████▍  | 7321/9864 [14:12:18<10:22:54, 14.70s/it]
 74%|███████▍  | 7322/9864 [14:12:36<10:56:44, 15.50s/it]
 74%|███████▍  | 7323/9864 [14:12:49<10:33:09, 14.95s/it]
 74%|███████▍  | 7324/9864 [14:13:05<10:47:00, 15.28s/it]
 74%|███████▍  | 7325/9864 [14:13:21<10:54:40, 15.47s/it]
 74%|███████▍  | 7326/9864 [14:13:35<10:32:25, 14.95s/it]
 74%|███████▍  | 7327/9864 [14:13:49<10:21:00, 14.69s/it]
 74%|███████▍  | 7328/9864 [14:14:03<10:10:46, 14.45s/it]
 74%|███████▍  | 7329/9864 [14:14:19<10:31:01, 14.94s/it]
 74%|███████▍  | 7330/9864 [14:14:35<10:43:30, 15.24s/it]
                                                         
{'loss': '0.09198', 'grad_norm': '4.181', 'learning_rate': '3.269e-06', 'epoch': '0.7432'}

 74%|███████▍  | 7330/9864 [14:14:35<10:43:30, 15.24s/it]
 74%|███████▍  | 7331/9864 [14:14:49<10:33:34, 15.01s/it]
 74%|███████▍  | 7332/9864 [14:15:04<10:26:19, 14.84s/it]
 74%|███████▍  | 7333/9864 [14:15:22<11:05:24, 15.77s/it]
 74%|███████▍  | 7334/9864 [14:15:36<10:46:19, 15.33s/it]
 74%|███████▍  | 7335/9864 [14:15:50<10:30:48, 14.97s/it]
 74%|███████▍  | 7336/9864 [14:16:06<10:37:19, 15.13s/it]
 74%|███████▍  | 7337/9864 [14:16:20<10:28:23, 14.92s/it]
 74%|███████▍  | 7338/9864 [14:16:34<10:16:55, 14.65s/it]
 74%|███████▍  | 7339/9864 [14:16:50<10:27:05, 14.90s/it]
 74%|███████▍  | 7340/9864 [14:17:07<10:57:33, 15.63s/it]
                                                         
{'loss': '0.0834', 'grad_norm': '4.359', 'learning_rate': '3.244e-06', 'epoch': '0.7442'}

 74%|███████▍  | 7340/9864 [14:17:07<10:57:33, 15.63s/it]
 74%|███████▍  | 7341/9864 [14:17:21<10:34:55, 15.10s/it]
 74%|███████▍  | 7342/9864 [14:17:35<10:21:50, 14.79s/it]
 74%|███████▍  | 7343/9864 [14:17:49<10:12:55, 14.59s/it]
 74%|███████▍  | 7344/9864 [14:18:05<10:24:32, 14.87s/it]
 74%|███████▍  | 7345/9864 [14:18:19<10:17:11, 14.70s/it]
 74%|███████▍  | 7346/9864 [14:18:33<10:13:06, 14.61s/it]
 74%|███████▍  | 7347/9864 [14:18:49<10:27:31, 14.96s/it]
 74%|███████▍  | 7348/9864 [14:19:07<11:00:54, 15.76s/it]
 75%|███████▍  | 7349/9864 [14:19:21<10:37:50, 15.22s/it]
 75%|███████▍  | 7350/9864 [14:19:35<10:26:34, 14.95s/it]
                                                         
{'loss': '0.09223', 'grad_norm': '3.317', 'learning_rate': '3.22e-06', 'epoch': '0.7452'}

 75%|███████▍  | 7350/9864 [14:19:35<10:26:34, 14.95s/it]
 75%|███████▍  | 7351/9864 [14:19:50<10:31:45, 15.08s/it]
 75%|███████▍  | 7352/9864 [14:20:04<10:17:38, 14.75s/it]
 75%|███████▍  | 7353/9864 [14:20:19<10:11:04, 14.60s/it]
 75%|███████▍  | 7354/9864 [14:20:33<10:08:30, 14.55s/it]
 75%|███████▍  | 7355/9864 [14:20:49<10:30:22, 15.07s/it]
 75%|███████▍  | 7356/9864 [14:21:07<11:03:54, 15.88s/it]
 75%|███████▍  | 7357/9864 [14:21:22<10:44:55, 15.44s/it]
 75%|███████▍  | 7358/9864 [14:21:36<10:30:08, 15.09s/it]
 75%|███████▍  | 7359/9864 [14:21:51<10:36:23, 15.24s/it]
 75%|███████▍  | 7360/9864 [14:22:05<10:21:11, 14.88s/it]
                                                         
{'loss': '0.08368', 'grad_norm': '4.375', 'learning_rate': '3.196e-06', 'epoch': '0.7462'}

 75%|███████▍  | 7360/9864 [14:22:06<10:21:11, 14.88s/it]
 75%|███████▍  | 7361/9864 [14:22:19<10:04:49, 14.50s/it]
 75%|███████▍  | 7362/9864 [14:22:33<9:53:46, 14.24s/it] 
 75%|███████▍  | 7363/9864 [14:22:52<10:53:30, 15.68s/it]
 75%|███████▍  | 7364/9864 [14:23:06<10:31:04, 15.15s/it]
 75%|███████▍  | 7365/9864 [14:23:20<10:20:43, 14.90s/it]
 75%|███████▍  | 7366/9864 [14:23:36<10:32:56, 15.20s/it]
 75%|███████▍  | 7367/9864 [14:23:49<10:10:18, 14.66s/it]
 75%|███████▍  | 7368/9864 [14:24:03<10:01:43, 14.46s/it]
 75%|███████▍  | 7369/9864 [14:24:17<9:57:01, 14.36s/it] 
 75%|███████▍  | 7370/9864 [14:24:32<9:54:20, 14.30s/it]
                                                        
{'loss': '0.08763', 'grad_norm': '6.108', 'learning_rate': '3.172e-06', 'epoch': '0.7472'}

 75%|███████▍  | 7370/9864 [14:24:32<9:54:20, 14.30s/it]
 75%|███████▍  | 7371/9864 [14:24:51<10:55:26, 15.77s/it]
 75%|███████▍  | 7372/9864 [14:25:05<10:36:23, 15.32s/it]
 75%|███████▍  | 7373/9864 [14:25:19<10:19:03, 14.91s/it]
 75%|███████▍  | 7374/9864 [14:25:35<10:29:49, 15.18s/it]
 75%|███████▍  | 7375/9864 [14:25:49<10:17:17, 14.88s/it]
 75%|███████▍  | 7376/9864 [14:26:03<10:08:51, 14.68s/it]
 75%|███████▍  | 7377/9864 [14:26:17<10:02:56, 14.55s/it]
 75%|███████▍  | 7378/9864 [14:26:33<10:17:45, 14.91s/it]
 75%|███████▍  | 7379/9864 [14:26:47<10:02:43, 14.55s/it]
 75%|███████▍  | 7380/9864 [14:27:03<10:16:53, 14.90s/it]
                                                         
{'loss': '0.08507', 'grad_norm': '5.643', 'learning_rate': '3.148e-06', 'epoch': '0.7482'}

 75%|███████▍  | 7380/9864 [14:27:03<10:16:53, 14.90s/it]
 75%|███████▍  | 7381/9864 [14:27:17<10:07:44, 14.69s/it]
 75%|███████▍  | 7382/9864 [14:27:31<9:59:37, 14.50s/it] 
 75%|███████▍  | 7383/9864 [14:27:47<10:15:53, 14.89s/it]
 75%|███████▍  | 7384/9864 [14:28:01<10:06:27, 14.67s/it]
 75%|███████▍  | 7385/9864 [14:28:15<9:54:18, 14.38s/it] 
 75%|███████▍  | 7386/9864 [14:28:32<10:28:42, 15.22s/it]
 75%|███████▍  | 7387/9864 [14:28:46<10:19:53, 15.02s/it]
 75%|███████▍  | 7388/9864 [14:29:02<10:27:57, 15.22s/it]
 75%|███████▍  | 7389/9864 [14:29:16<10:14:59, 14.91s/it]
 75%|███████▍  | 7390/9864 [14:29:30<9:59:15, 14.53s/it] 
                                                        
{'loss': '0.0869', 'grad_norm': '3.416', 'learning_rate': '3.124e-06', 'epoch': '0.7493'}

 75%|███████▍  | 7390/9864 [14:29:30<9:59:15, 14.53s/it]
 75%|███████▍  | 7391/9864 [14:29:45<10:11:57, 14.85s/it]
 75%|███████▍  | 7392/9864 [14:30:00<10:08:34, 14.77s/it]
 75%|███████▍  | 7393/9864 [14:30:14<10:01:56, 14.62s/it]
 75%|███████▍  | 7394/9864 [14:30:31<10:31:13, 15.33s/it]
 75%|███████▍  | 7395/9864 [14:30:47<10:36:22, 15.46s/it]
 75%|███████▍  | 7396/9864 [14:31:01<10:21:22, 15.11s/it]
 75%|███████▍  | 7397/9864 [14:31:15<10:07:43, 14.78s/it]
 75%|███████▌  | 7398/9864 [14:31:29<9:56:40, 14.52s/it] 
 75%|███████▌  | 7399/9864 [14:31:44<9:54:34, 14.47s/it]
 75%|███████▌  | 7400/9864 [14:32:00<10:14:36, 14.97s/it]
                                                         
{'loss': '0.08318', 'grad_norm': '4.085', 'learning_rate': '3.1e-06', 'epoch': '0.7503'}

 75%|███████▌  | 7400/9864 [14:32:00<10:14:36, 14.97s/it]
 75%|███████▌  | 7401/9864 [14:32:15<10:22:35, 15.17s/it]
 75%|███████▌  | 7402/9864 [14:32:31<10:29:55, 15.35s/it]
 75%|███████▌  | 7403/9864 [14:32:45<10:09:31, 14.86s/it]
 75%|███████▌  | 7404/9864 [14:33:01<10:21:26, 15.16s/it]
 75%|███████▌  | 7405/9864 [14:33:14<10:03:32, 14.73s/it]
 75%|███████▌  | 7406/9864 [14:33:29<9:56:19, 14.56s/it] 
 75%|███████▌  | 7407/9864 [14:33:42<9:48:22, 14.37s/it]
 75%|███████▌  | 7408/9864 [14:33:58<10:03:20, 14.74s/it]
 75%|███████▌  | 7409/9864 [14:34:14<10:16:59, 15.08s/it]
 75%|███████▌  | 7410/9864 [14:34:28<10:09:42, 14.91s/it]
                                                         
{'loss': '0.08337', 'grad_norm': '4.595', 'learning_rate': '3.077e-06', 'epoch': '0.7513'}

 75%|███████▌  | 7410/9864 [14:34:28<10:09:42, 14.91s/it]
 75%|███████▌  | 7411/9864 [14:34:46<10:40:47, 15.67s/it]
 75%|███████▌  | 7412/9864 [14:35:00<10:17:18, 15.11s/it]
 75%|███████▌  | 7413/9864 [14:35:14<10:10:48, 14.95s/it]
 75%|███████▌  | 7414/9864 [14:35:28<9:57:46, 14.64s/it] 
 75%|███████▌  | 7415/9864 [14:35:43<10:04:58, 14.82s/it]
 75%|███████▌  | 7416/9864 [14:35:58<10:00:06, 14.71s/it]
 75%|███████▌  | 7417/9864 [14:36:14<10:12:56, 15.03s/it]
 75%|███████▌  | 7418/9864 [14:36:28<9:58:20, 14.68s/it] 
 75%|███████▌  | 7419/9864 [14:36:43<10:10:14, 14.98s/it]
 75%|███████▌  | 7420/9864 [14:36:59<10:16:31, 15.14s/it]
                                                         
{'loss': '0.0812', 'grad_norm': '5.552', 'learning_rate': '3.053e-06', 'epoch': '0.7523'}

 75%|███████▌  | 7420/9864 [14:36:59<10:16:31, 15.14s/it]
 75%|███████▌  | 7421/9864 [14:37:13<10:03:24, 14.82s/it]
 75%|███████▌  | 7422/9864 [14:37:26<9:48:32, 14.46s/it] 
 75%|███████▌  | 7423/9864 [14:37:42<9:57:29, 14.69s/it]
 75%|███████▌  | 7424/9864 [14:37:56<9:50:15, 14.51s/it]
 75%|███████▌  | 7425/9864 [14:38:12<10:07:22, 14.94s/it]
 75%|███████▌  | 7426/9864 [14:38:26<10:03:39, 14.86s/it]
 75%|███████▌  | 7427/9864 [14:38:42<10:12:09, 15.07s/it]
 75%|███████▌  | 7428/9864 [14:38:58<10:22:35, 15.33s/it]
 75%|███████▌  | 7429/9864 [14:39:12<10:07:32, 14.97s/it]
 75%|███████▌  | 7430/9864 [14:39:26<9:56:12, 14.70s/it] 
                                                        
{'loss': '0.07257', 'grad_norm': '5.46', 'learning_rate': '3.029e-06', 'epoch': '0.7533'}

 75%|███████▌  | 7430/9864 [14:39:26<9:56:12, 14.70s/it]
 75%|███████▌  | 7431/9864 [14:39:42<10:10:20, 15.05s/it]
 75%|███████▌  | 7432/9864 [14:39:58<10:17:09, 15.23s/it]
 75%|███████▌  | 7433/9864 [14:40:11<9:57:26, 14.75s/it] 
 75%|███████▌  | 7434/9864 [14:40:25<9:50:18, 14.58s/it]
 75%|███████▌  | 7435/9864 [14:40:42<10:20:10, 15.32s/it]
 75%|███████▌  | 7436/9864 [14:40:57<10:07:20, 15.01s/it]
 75%|███████▌  | 7437/9864 [14:41:11<9:53:01, 14.66s/it] 
 75%|███████▌  | 7438/9864 [14:41:25<9:55:06, 14.72s/it]
 75%|███████▌  | 7439/9864 [14:41:41<10:04:03, 14.95s/it]
 75%|███████▌  | 7440/9864 [14:41:57<10:18:00, 15.30s/it]
                                                         
{'loss': '0.08089', 'grad_norm': '7.037', 'learning_rate': '3.006e-06', 'epoch': '0.7543'}

 75%|███████▌  | 7440/9864 [14:41:57<10:18:00, 15.30s/it]
 75%|███████▌  | 7441/9864 [14:42:11<10:04:39, 14.97s/it]
 75%|███████▌  | 7442/9864 [14:42:27<10:15:16, 15.24s/it]
 75%|███████▌  | 7443/9864 [14:42:43<10:23:00, 15.44s/it]
 75%|███████▌  | 7444/9864 [14:42:57<10:05:35, 15.01s/it]
 75%|███████▌  | 7445/9864 [14:43:11<9:54:37, 14.75s/it] 
 75%|███████▌  | 7446/9864 [14:43:25<9:48:57, 14.61s/it]
 75%|███████▌  | 7447/9864 [14:43:41<9:54:37, 14.76s/it]
 76%|███████▌  | 7448/9864 [14:43:56<10:02:08, 14.95s/it]
 76%|███████▌  | 7449/9864 [14:44:10<9:51:59, 14.71s/it] 
 76%|███████▌  | 7450/9864 [14:44:26<10:06:02, 15.06s/it]
                                                         
{'loss': '0.08303', 'grad_norm': '5.444', 'learning_rate': '2.983e-06', 'epoch': '0.7553'}

 76%|███████▌  | 7450/9864 [14:44:26<10:06:02, 15.06s/it]
 76%|███████▌  | 7451/9864 [14:44:42<10:19:03, 15.39s/it]
 76%|███████▌  | 7452/9864 [14:44:57<10:07:16, 15.11s/it]
 76%|███████▌  | 7453/9864 [14:45:10<9:52:37, 14.75s/it] 
 76%|███████▌  | 7454/9864 [14:45:25<9:47:19, 14.62s/it]
 76%|███████▌  | 7455/9864 [14:45:40<9:59:42, 14.94s/it]
 76%|███████▌  | 7456/9864 [14:45:54<9:48:06, 14.65s/it]
 76%|███████▌  | 7457/9864 [14:46:12<10:22:49, 15.53s/it]
 76%|███████▌  | 7458/9864 [14:46:28<10:26:03, 15.61s/it]
 76%|███████▌  | 7459/9864 [14:46:42<10:06:07, 15.12s/it]
 76%|███████▌  | 7460/9864 [14:46:56<9:48:46, 14.69s/it] 
                                                        
{'loss': '0.07957', 'grad_norm': '3.662', 'learning_rate': '2.959e-06', 'epoch': '0.7563'}

 76%|███████▌  | 7460/9864 [14:46:56<9:48:46, 14.69s/it]
 76%|███████▌  | 7461/9864 [14:47:10<9:40:27, 14.49s/it]
 76%|███████▌  | 7462/9864 [14:47:24<9:36:31, 14.40s/it]
 76%|███████▌  | 7463/9864 [14:47:38<9:31:05, 14.27s/it]
 76%|███████▌  | 7464/9864 [14:47:57<10:28:02, 15.70s/it]
 76%|███████▌  | 7465/9864 [14:48:11<10:07:00, 15.18s/it]
 76%|███████▌  | 7466/9864 [14:48:26<10:08:56, 15.24s/it]
 76%|███████▌  | 7467/9864 [14:48:40<9:50:02, 14.77s/it] 
 76%|███████▌  | 7468/9864 [14:48:54<9:38:15, 14.48s/it]
 76%|███████▌  | 7469/9864 [14:49:08<9:33:55, 14.38s/it]
 76%|███████▌  | 7470/9864 [14:49:22<9:30:30, 14.30s/it]
                                                        
{'loss': '0.08586', 'grad_norm': '4.816', 'learning_rate': '2.936e-06', 'epoch': '0.7574'}

 76%|███████▌  | 7470/9864 [14:49:22<9:30:30, 14.30s/it]
 76%|███████▌  | 7471/9864 [14:49:36<9:28:47, 14.26s/it]
 76%|███████▌  | 7472/9864 [14:49:53<10:04:18, 15.16s/it]
 76%|███████▌  | 7473/9864 [14:50:10<10:17:28, 15.50s/it]
 76%|███████▌  | 7474/9864 [14:50:25<10:18:50, 15.54s/it]
 76%|███████▌  | 7475/9864 [14:50:39<10:00:40, 15.09s/it]
 76%|███████▌  | 7476/9864 [14:50:54<9:54:45, 14.94s/it] 
 76%|███████▌  | 7477/9864 [14:51:08<9:47:27, 14.77s/it]
 76%|███████▌  | 7478/9864 [14:51:22<9:34:43, 14.45s/it]
 76%|███████▌  | 7479/9864 [14:51:36<9:31:46, 14.38s/it]
 76%|███████▌  | 7480/9864 [14:51:53<10:05:33, 15.24s/it]
                                                         
{'loss': '0.07124', 'grad_norm': '5.818', 'learning_rate': '2.913e-06', 'epoch': '0.7584'}

 76%|███████▌  | 7480/9864 [14:51:53<10:05:33, 15.24s/it]
 76%|███████▌  | 7481/9864 [14:52:10<10:27:52, 15.81s/it]
 76%|███████▌  | 7482/9864 [14:52:25<10:09:10, 15.34s/it]
 76%|███████▌  | 7483/9864 [14:52:39<9:52:16, 14.92s/it] 
 76%|███████▌  | 7484/9864 [14:52:53<9:42:05, 14.67s/it]
 76%|███████▌  | 7485/9864 [14:53:07<9:33:05, 14.45s/it]
 76%|███████▌  | 7486/9864 [14:53:21<9:30:50, 14.40s/it]
 76%|███████▌  | 7487/9864 [14:53:37<9:49:55, 14.89s/it]
 76%|███████▌  | 7488/9864 [14:53:53<9:56:34, 15.07s/it]
 76%|███████▌  | 7489/9864 [14:54:09<10:07:33, 15.35s/it]
 76%|███████▌  | 7490/9864 [14:54:24<10:14:06, 15.52s/it]
                                                         
{'loss': '0.09136', 'grad_norm': '3.58', 'learning_rate': '2.89e-06', 'epoch': '0.7594'}

 76%|███████▌  | 7490/9864 [14:54:24<10:14:06, 15.52s/it]
 76%|███████▌  | 7491/9864 [14:54:38<9:54:11, 15.02s/it] 
 76%|███████▌  | 7492/9864 [14:54:53<9:46:18, 14.83s/it]
 76%|███████▌  | 7493/9864 [14:55:07<9:37:22, 14.61s/it]
 76%|███████▌  | 7494/9864 [14:55:21<9:29:32, 14.42s/it]
 76%|███████▌  | 7495/9864 [14:55:36<9:44:02, 14.79s/it]
 76%|███████▌  | 7496/9864 [14:55:52<9:49:01, 14.92s/it]
 76%|███████▌  | 7497/9864 [14:56:07<9:53:40, 15.05s/it]
 76%|███████▌  | 7498/9864 [14:56:23<10:05:56, 15.37s/it]
 76%|███████▌  | 7499/9864 [14:56:37<9:51:46, 15.01s/it] 
 76%|███████▌  | 7500/9864 [14:56:51<9:38:04, 14.67s/it]
                                                        
{'loss': '0.07238', 'grad_norm': '4.344', 'learning_rate': '2.867e-06', 'epoch': '0.7604'}

 76%|███████▌  | 7500/9864 [14:56:51<9:38:04, 14.67s/it]
 76%|███████▌  | 7501/9864 [14:57:06<9:34:18, 14.58s/it]
 76%|███████▌  | 7502/9864 [14:57:20<9:33:34, 14.57s/it]
 76%|███████▌  | 7503/9864 [14:57:38<10:16:09, 15.66s/it]
 76%|███████▌  | 7504/9864 [14:57:52<9:55:09, 15.13s/it] 
 76%|███████▌  | 7505/9864 [14:58:08<10:01:44, 15.31s/it]
 76%|███████▌  | 7506/9864 [14:58:24<10:06:07, 15.42s/it]
 76%|███████▌  | 7507/9864 [14:58:38<9:53:24, 15.11s/it] 
 76%|███████▌  | 7508/9864 [14:58:52<9:35:55, 14.67s/it]
 76%|███████▌  | 7509/9864 [14:59:05<9:25:53, 14.42s/it]
 76%|███████▌  | 7510/9864 [14:59:19<9:16:05, 14.17s/it]
                                                        
{'loss': '0.07974', 'grad_norm': '4.333', 'learning_rate': '2.844e-06', 'epoch': '0.7614'}

 76%|███████▌  | 7510/9864 [14:59:19<9:16:05, 14.17s/it]
 76%|███████▌  | 7511/9864 [14:59:35<9:36:13, 14.69s/it]
 76%|███████▌  | 7512/9864 [14:59:50<9:44:31, 14.91s/it]
 76%|███████▌  | 7513/9864 [15:00:06<9:55:56, 15.21s/it]
 76%|███████▌  | 7514/9864 [15:00:20<9:39:57, 14.81s/it]
 76%|███████▌  | 7515/9864 [15:00:36<9:49:25, 15.06s/it]
 76%|███████▌  | 7516/9864 [15:00:49<9:28:54, 14.54s/it]
 76%|███████▌  | 7517/9864 [15:01:03<9:19:31, 14.30s/it]
 76%|███████▌  | 7518/9864 [15:01:18<9:33:16, 14.66s/it]
 76%|███████▌  | 7519/9864 [15:01:34<9:45:20, 14.98s/it]
 76%|███████▌  | 7520/9864 [15:01:49<9:39:09, 14.83s/it]
                                                        
{'loss': '0.09238', 'grad_norm': '4.428', 'learning_rate': '2.821e-06', 'epoch': '0.7624'}

 76%|███████▌  | 7520/9864 [15:01:49<9:39:09, 14.83s/it]
 76%|███████▌  | 7521/9864 [15:02:04<9:49:02, 15.08s/it]
 76%|███████▋  | 7522/9864 [15:02:20<10:00:35, 15.39s/it]
 76%|███████▋  | 7523/9864 [15:02:35<9:47:45, 15.06s/it] 
 76%|███████▋  | 7524/9864 [15:02:49<9:35:28, 14.76s/it]
 76%|███████▋  | 7525/9864 [15:03:03<9:26:47, 14.54s/it]
 76%|███████▋  | 7526/9864 [15:03:18<9:39:14, 14.87s/it]
 76%|███████▋  | 7527/9864 [15:03:32<9:30:16, 14.64s/it]
 76%|███████▋  | 7528/9864 [15:03:48<9:44:28, 15.01s/it]
 76%|███████▋  | 7529/9864 [15:04:06<10:14:06, 15.78s/it]
 76%|███████▋  | 7530/9864 [15:04:20<9:50:20, 15.18s/it] 
                                                        
{'loss': '0.07969', 'grad_norm': '5.558', 'learning_rate': '2.798e-06', 'epoch': '0.7634'}

 76%|███████▋  | 7530/9864 [15:04:20<9:50:20, 15.18s/it]
 76%|███████▋  | 7531/9864 [15:04:34<9:41:35, 14.96s/it]
 76%|███████▋  | 7532/9864 [15:04:48<9:31:57, 14.72s/it]
 76%|███████▋  | 7533/9864 [15:05:04<9:45:58, 15.08s/it]
 76%|███████▋  | 7534/9864 [15:05:18<9:34:49, 14.80s/it]
 76%|███████▋  | 7535/9864 [15:05:34<9:40:56, 14.97s/it]
 76%|███████▋  | 7536/9864 [15:05:48<9:35:50, 14.84s/it]
 76%|███████▋  | 7537/9864 [15:06:06<10:06:00, 15.63s/it]
 76%|███████▋  | 7538/9864 [15:06:20<9:48:20, 15.18s/it] 
 76%|███████▋  | 7539/9864 [15:06:34<9:32:17, 14.77s/it]
 76%|███████▋  | 7540/9864 [15:06:48<9:22:06, 14.51s/it]
                                                        
{'loss': '0.09184', 'grad_norm': '3.568', 'learning_rate': '2.775e-06', 'epoch': '0.7645'}

 76%|███████▋  | 7540/9864 [15:06:48<9:22:06, 14.51s/it]
 76%|███████▋  | 7541/9864 [15:07:02<9:15:24, 14.35s/it]
 76%|███████▋  | 7542/9864 [15:07:17<9:29:07, 14.71s/it]
 76%|███████▋  | 7543/9864 [15:07:33<9:41:30, 15.03s/it]
 76%|███████▋  | 7544/9864 [15:07:50<10:09:44, 15.77s/it]
 76%|███████▋  | 7545/9864 [15:08:05<9:54:55, 15.39s/it] 
 77%|███████▋  | 7546/9864 [15:08:19<9:44:54, 15.14s/it]
 77%|███████▋  | 7547/9864 [15:08:34<9:36:29, 14.93s/it]
 77%|███████▋  | 7548/9864 [15:08:48<9:28:29, 14.73s/it]
 77%|███████▋  | 7549/9864 [15:09:04<9:42:30, 15.10s/it]
 77%|███████▋  | 7550/9864 [15:09:20<9:52:19, 15.36s/it]
                                                        
{'loss': '0.08196', 'grad_norm': '4.754', 'learning_rate': '2.752e-06', 'epoch': '0.7655'}

 77%|███████▋  | 7550/9864 [15:09:20<9:52:19, 15.36s/it]
 77%|███████▋  | 7551/9864 [15:09:34<9:37:18, 14.98s/it]
 77%|███████▋  | 7552/9864 [15:09:49<9:40:19, 15.06s/it]
 77%|███████▋  | 7553/9864 [15:10:05<9:49:18, 15.30s/it]
 77%|███████▋  | 7554/9864 [15:10:19<9:34:58, 14.93s/it]
 77%|███████▋  | 7555/9864 [15:10:33<9:24:31, 14.67s/it]
 77%|███████▋  | 7556/9864 [15:10:47<9:16:19, 14.46s/it]
 77%|███████▋  | 7557/9864 [15:11:02<9:22:29, 14.63s/it]
 77%|███████▋  | 7558/9864 [15:11:17<9:16:47, 14.49s/it]
 77%|███████▋  | 7559/9864 [15:11:32<9:29:47, 14.83s/it]
 77%|███████▋  | 7560/9864 [15:11:46<9:23:10, 14.67s/it]
                                                        
{'loss': '0.07895', 'grad_norm': '4.962', 'learning_rate': '2.73e-06', 'epoch': '0.7665'}

 77%|███████▋  | 7560/9864 [15:11:46<9:23:10, 14.67s/it]
 77%|███████▋  | 7561/9864 [15:12:04<9:57:51, 15.58s/it]
 77%|███████▋  | 7562/9864 [15:12:18<9:41:33, 15.16s/it]
 77%|███████▋  | 7563/9864 [15:12:32<9:28:19, 14.82s/it]
 77%|███████▋  | 7564/9864 [15:12:48<9:40:04, 15.13s/it]
 77%|███████▋  | 7565/9864 [15:13:02<9:29:54, 14.87s/it]
 77%|███████▋  | 7566/9864 [15:13:16<9:15:48, 14.51s/it]
 77%|███████▋  | 7567/9864 [15:13:32<9:32:44, 14.96s/it]
 77%|███████▋  | 7568/9864 [15:13:49<9:58:01, 15.63s/it]
 77%|███████▋  | 7569/9864 [15:14:04<9:41:39, 15.21s/it]
 77%|███████▋  | 7570/9864 [15:14:18<9:29:04, 14.88s/it]
                                                        
{'loss': '0.08254', 'grad_norm': '6.113', 'learning_rate': '2.707e-06', 'epoch': '0.7675'}

 77%|███████▋  | 7570/9864 [15:14:18<9:29:04, 14.88s/it]
 77%|███████▋  | 7571/9864 [15:14:32<9:21:05, 14.68s/it]
 77%|███████▋  | 7572/9864 [15:14:47<9:30:26, 14.93s/it]
 77%|███████▋  | 7573/9864 [15:15:02<9:24:43, 14.79s/it]
 77%|███████▋  | 7574/9864 [15:15:17<9:30:57, 14.96s/it]
 77%|███████▋  | 7575/9864 [15:15:33<9:41:42, 15.25s/it]
 77%|███████▋  | 7576/9864 [15:15:49<9:50:31, 15.49s/it]
 77%|███████▋  | 7577/9864 [15:16:04<9:37:22, 15.15s/it]
 77%|███████▋  | 7578/9864 [15:16:18<9:25:45, 14.85s/it]
 77%|███████▋  | 7579/9864 [15:16:33<9:33:43, 15.06s/it]
 77%|███████▋  | 7580/9864 [15:16:47<9:22:18, 14.77s/it]
                                                        
{'loss': '0.08028', 'grad_norm': '6.126', 'learning_rate': '2.685e-06', 'epoch': '0.7685'}

 77%|███████▋  | 7580/9864 [15:16:47<9:22:18, 14.77s/it]
 77%|███████▋  | 7581/9864 [15:17:03<9:33:43, 15.08s/it]
 77%|███████▋  | 7582/9864 [15:17:17<9:19:07, 14.70s/it]
 77%|███████▋  | 7583/9864 [15:17:33<9:31:27, 15.03s/it]
 77%|███████▋  | 7584/9864 [15:17:49<9:39:07, 15.24s/it]
 77%|███████▋  | 7585/9864 [15:18:03<9:27:33, 14.94s/it]
 77%|███████▋  | 7586/9864 [15:18:17<9:19:38, 14.74s/it]
 77%|███████▋  | 7587/9864 [15:18:32<9:22:44, 14.83s/it]
 77%|███████▋  | 7588/9864 [15:18:46<9:13:10, 14.58s/it]
 77%|███████▋  | 7589/9864 [15:19:00<9:08:03, 14.45s/it]
 77%|███████▋  | 7590/9864 [15:19:17<9:38:35, 15.27s/it]
                                                        
{'loss': '0.07317', 'grad_norm': '2.968', 'learning_rate': '2.663e-06', 'epoch': '0.7695'}

 77%|███████▋  | 7590/9864 [15:19:17<9:38:35, 15.27s/it]
 77%|███████▋  | 7591/9864 [15:19:32<9:28:53, 15.02s/it]
 77%|███████▋  | 7592/9864 [15:19:48<9:38:16, 15.27s/it]
 77%|███████▋  | 7593/9864 [15:20:02<9:23:14, 14.88s/it]
 77%|███████▋  | 7594/9864 [15:20:17<9:33:50, 15.17s/it]
 77%|███████▋  | 7595/9864 [15:20:31<9:20:08, 14.81s/it]
 77%|███████▋  | 7596/9864 [15:20:45<9:07:31, 14.48s/it]
 77%|███████▋  | 7597/9864 [15:21:01<9:24:52, 14.95s/it]
 77%|███████▋  | 7598/9864 [15:21:17<9:33:44, 15.19s/it]
 77%|███████▋  | 7599/9864 [15:21:31<9:24:20, 14.95s/it]
 77%|███████▋  | 7600/9864 [15:21:47<9:32:52, 15.18s/it]
                                                        
{'loss': '0.07653', 'grad_norm': '4.807', 'learning_rate': '2.64e-06', 'epoch': '0.7705'}

 77%|███████▋  | 7600/9864 [15:21:47<9:32:52, 15.18s/it]
 77%|███████▋  | 7601/9864 [15:22:01<9:23:10, 14.93s/it]
 77%|███████▋  | 7602/9864 [15:22:17<9:26:10, 15.02s/it]
 77%|███████▋  | 7603/9864 [15:22:31<9:15:39, 14.75s/it]
 77%|███████▋  | 7604/9864 [15:22:45<9:08:44, 14.57s/it]
 77%|███████▋  | 7605/9864 [15:23:01<9:21:54, 14.92s/it]
 77%|███████▋  | 7606/9864 [15:23:16<9:27:10, 15.07s/it]
 77%|███████▋  | 7607/9864 [15:23:30<9:11:11, 14.65s/it]
 77%|███████▋  | 7608/9864 [15:23:45<9:20:22, 14.90s/it]
 77%|███████▋  | 7609/9864 [15:23:59<9:11:08, 14.66s/it]
 77%|███████▋  | 7610/9864 [15:24:15<9:24:07, 15.02s/it]
                                                        
{'loss': '0.08696', 'grad_norm': '4.879', 'learning_rate': '2.618e-06', 'epoch': '0.7716'}

 77%|███████▋  | 7610/9864 [15:24:15<9:24:07, 15.02s/it]
 77%|███████▋  | 7611/9864 [15:24:29<9:06:59, 14.57s/it]
 77%|███████▋  | 7612/9864 [15:24:43<9:01:17, 14.42s/it]
 77%|███████▋  | 7613/9864 [15:24:58<9:12:01, 14.71s/it]
 77%|███████▋  | 7614/9864 [15:25:13<9:14:46, 14.79s/it]
 77%|███████▋  | 7615/9864 [15:25:27<9:07:02, 14.59s/it]
 77%|███████▋  | 7616/9864 [15:25:43<9:16:04, 14.84s/it]
 77%|███████▋  | 7617/9864 [15:25:57<9:09:18, 14.67s/it]
 77%|███████▋  | 7618/9864 [15:26:11<8:59:05, 14.40s/it]
 77%|███████▋  | 7619/9864 [15:26:27<9:14:45, 14.83s/it]
 77%|███████▋  | 7620/9864 [15:26:41<9:06:04, 14.60s/it]
                                                        
{'loss': '0.07639', 'grad_norm': '5.289', 'learning_rate': '2.596e-06', 'epoch': '0.7726'}

 77%|███████▋  | 7620/9864 [15:26:41<9:06:04, 14.60s/it]
 77%|███████▋  | 7621/9864 [15:26:58<9:35:32, 15.40s/it]
 77%|███████▋  | 7622/9864 [15:27:12<9:19:32, 14.97s/it]
 77%|███████▋  | 7623/9864 [15:27:26<9:10:04, 14.73s/it]
 77%|███████▋  | 7624/9864 [15:27:42<9:23:09, 15.08s/it]
 77%|███████▋  | 7625/9864 [15:27:56<9:09:46, 14.73s/it]
 77%|███████▋  | 7626/9864 [15:28:10<9:01:16, 14.51s/it]
 77%|███████▋  | 7627/9864 [15:28:26<9:19:50, 15.02s/it]
 77%|███████▋  | 7628/9864 [15:28:40<9:11:36, 14.80s/it]
 77%|███████▋  | 7629/9864 [15:28:58<9:46:57, 15.76s/it]
 77%|███████▋  | 7630/9864 [15:29:12<9:26:46, 15.22s/it]
                                                        
{'loss': '0.07723', 'grad_norm': '7.031', 'learning_rate': '2.574e-06', 'epoch': '0.7736'}

 77%|███████▋  | 7630/9864 [15:29:12<9:26:46, 15.22s/it]
 77%|███████▋  | 7631/9864 [15:29:27<9:18:54, 15.02s/it]
 77%|███████▋  | 7632/9864 [15:29:42<9:23:14, 15.14s/it]
 77%|███████▋  | 7633/9864 [15:29:56<9:11:40, 14.84s/it]
 77%|███████▋  | 7634/9864 [15:30:10<8:58:17, 14.48s/it]
 77%|███████▋  | 7635/9864 [15:30:26<9:11:03, 14.83s/it]
 77%|███████▋  | 7636/9864 [15:30:40<8:59:40, 14.53s/it]
 77%|███████▋  | 7637/9864 [15:30:57<9:32:07, 15.41s/it]
 77%|███████▋  | 7638/9864 [15:31:11<9:20:11, 15.10s/it]
 77%|███████▋  | 7639/9864 [15:31:28<9:33:00, 15.45s/it]
 77%|███████▋  | 7640/9864 [15:31:42<9:14:54, 14.97s/it]
                                                        
{'loss': '0.08365', 'grad_norm': '4.906', 'learning_rate': '2.552e-06', 'epoch': '0.7746'}

 77%|███████▋  | 7640/9864 [15:31:42<9:14:54, 14.97s/it]
 77%|███████▋  | 7641/9864 [15:31:55<8:59:21, 14.56s/it]
 77%|███████▋  | 7642/9864 [15:32:09<8:55:48, 14.47s/it]
 77%|███████▋  | 7643/9864 [15:32:25<9:09:09, 14.84s/it]
 77%|███████▋  | 7644/9864 [15:32:41<9:21:10, 15.17s/it]
 78%|███████▊  | 7645/9864 [15:32:57<9:29:37, 15.40s/it]
 78%|███████▊  | 7646/9864 [15:33:13<9:31:06, 15.45s/it]
 78%|███████▊  | 7647/9864 [15:33:27<9:17:53, 15.10s/it]
 78%|███████▊  | 7648/9864 [15:33:41<9:07:36, 14.83s/it]
 78%|███████▊  | 7649/9864 [15:33:54<8:52:42, 14.43s/it]
 78%|███████▊  | 7650/9864 [15:34:09<8:50:53, 14.39s/it]
                                                        
{'loss': '0.08129', 'grad_norm': '4.851', 'learning_rate': '2.53e-06', 'epoch': '0.7756'}

 78%|███████▊  | 7650/9864 [15:34:09<8:50:53, 14.39s/it]
 78%|███████▊  | 7651/9864 [15:34:24<9:03:31, 14.74s/it]
 78%|███████▊  | 7652/9864 [15:34:42<9:31:01, 15.49s/it]
 78%|███████▊  | 7653/9864 [15:34:58<9:41:02, 15.77s/it]
 78%|███████▊  | 7654/9864 [15:35:12<9:25:44, 15.36s/it]
 78%|███████▊  | 7655/9864 [15:35:27<9:13:11, 15.03s/it]
 78%|███████▊  | 7656/9864 [15:35:41<9:02:19, 14.74s/it]
 78%|███████▊  | 7657/9864 [15:35:55<8:55:49, 14.57s/it]
 78%|███████▊  | 7658/9864 [15:36:09<8:46:21, 14.32s/it]
 78%|███████▊  | 7659/9864 [15:36:24<9:02:27, 14.76s/it]
 78%|███████▊  | 7660/9864 [15:36:42<9:33:28, 15.61s/it]
                                                        
{'loss': '0.084', 'grad_norm': '5.003', 'learning_rate': '2.508e-06', 'epoch': '0.7766'}

 78%|███████▊  | 7660/9864 [15:36:42<9:33:28, 15.61s/it]
 78%|███████▊  | 7661/9864 [15:36:58<9:35:24, 15.67s/it]
 78%|███████▊  | 7662/9864 [15:37:12<9:21:00, 15.29s/it]
 78%|███████▊  | 7663/9864 [15:37:27<9:10:32, 15.01s/it]
 78%|███████▊  | 7664/9864 [15:37:41<9:00:00, 14.73s/it]
 78%|███████▊  | 7665/9864 [15:37:55<8:55:18, 14.61s/it]
 78%|███████▊  | 7666/9864 [15:38:09<8:47:15, 14.39s/it]
 78%|███████▊  | 7667/9864 [15:38:26<9:19:31, 15.28s/it]
 78%|███████▊  | 7668/9864 [15:38:42<9:26:31, 15.48s/it]
 78%|███████▊  | 7669/9864 [15:38:58<9:33:15, 15.67s/it]
 78%|███████▊  | 7670/9864 [15:39:13<9:17:55, 15.26s/it]
                                                        
{'loss': '0.06915', 'grad_norm': '6.222', 'learning_rate': '2.487e-06', 'epoch': '0.7776'}

 78%|███████▊  | 7670/9864 [15:39:13<9:17:55, 15.26s/it]
 78%|███████▊  | 7671/9864 [15:39:27<9:10:23, 15.06s/it]
 78%|███████▊  | 7672/9864 [15:39:41<8:59:43, 14.77s/it]
 78%|███████▊  | 7673/9864 [15:39:55<8:52:11, 14.57s/it]
 78%|███████▊  | 7674/9864 [15:40:10<8:48:35, 14.48s/it]
 78%|███████▊  | 7675/9864 [15:40:29<9:39:59, 15.90s/it]
 78%|███████▊  | 7676/9864 [15:40:45<9:43:07, 15.99s/it]
 78%|███████▊  | 7677/9864 [15:40:59<9:21:52, 15.42s/it]
 78%|███████▊  | 7678/9864 [15:41:13<9:08:22, 15.05s/it]
 78%|███████▊  | 7679/9864 [15:41:27<8:57:52, 14.77s/it]
 78%|███████▊  | 7680/9864 [15:41:41<8:46:37, 14.47s/it]
                                                        
{'loss': '0.06959', 'grad_norm': '5.283', 'learning_rate': '2.465e-06', 'epoch': '0.7787'}

 78%|███████▊  | 7680/9864 [15:41:41<8:46:37, 14.47s/it]
 78%|███████▊  | 7681/9864 [15:41:55<8:35:10, 14.16s/it]
 78%|███████▊  | 7682/9864 [15:42:11<8:54:07, 14.69s/it]
 78%|███████▊  | 7683/9864 [15:42:25<8:50:12, 14.59s/it]
 78%|███████▊  | 7684/9864 [15:42:45<9:45:11, 16.11s/it]
 78%|███████▊  | 7685/9864 [15:42:59<9:23:05, 15.51s/it]
 78%|███████▊  | 7686/9864 [15:43:13<9:06:25, 15.05s/it]
 78%|███████▊  | 7687/9864 [15:43:27<8:55:07, 14.75s/it]
 78%|███████▊  | 7688/9864 [15:43:41<8:45:50, 14.50s/it]
 78%|███████▊  | 7689/9864 [15:43:55<8:41:58, 14.40s/it]
 78%|███████▊  | 7690/9864 [15:44:09<8:34:27, 14.20s/it]
                                                        
{'loss': '0.07732', 'grad_norm': '3.865', 'learning_rate': '2.443e-06', 'epoch': '0.7797'}

 78%|███████▊  | 7690/9864 [15:44:09<8:34:27, 14.20s/it]
 78%|███████▊  | 7691/9864 [15:44:26<9:04:38, 15.04s/it]
 78%|███████▊  | 7692/9864 [15:44:41<9:12:25, 15.26s/it]
 78%|███████▊  | 7693/9864 [15:44:57<9:20:43, 15.50s/it]
 78%|███████▊  | 7694/9864 [15:45:11<9:03:35, 15.03s/it]
 78%|███████▊  | 7695/9864 [15:45:26<8:56:14, 14.83s/it]
 78%|███████▊  | 7696/9864 [15:45:40<8:46:13, 14.56s/it]
 78%|███████▊  | 7697/9864 [15:45:54<8:40:04, 14.40s/it]
 78%|███████▊  | 7698/9864 [15:46:11<9:15:17, 15.38s/it]
 78%|███████▊  | 7699/9864 [15:46:27<9:23:34, 15.62s/it]
 78%|███████▊  | 7700/9864 [15:46:43<9:23:42, 15.63s/it]
                                                        
{'loss': '0.0785', 'grad_norm': '5.275', 'learning_rate': '2.422e-06', 'epoch': '0.7807'}

 78%|███████▊  | 7700/9864 [15:46:43<9:23:42, 15.63s/it]
 78%|███████▊  | 7701/9864 [15:46:57<9:09:34, 15.24s/it]
 78%|███████▊  | 7702/9864 [15:47:12<9:01:27, 15.03s/it]
 78%|███████▊  | 7703/9864 [15:47:27<8:56:54, 14.91s/it]
 78%|███████▊  | 7704/9864 [15:47:41<8:45:52, 14.61s/it]
 78%|███████▊  | 7705/9864 [15:47:56<8:58:42, 14.97s/it]
 78%|███████▊  | 7706/9864 [15:48:12<9:08:36, 15.25s/it]
 78%|███████▊  | 7707/9864 [15:48:28<9:13:29, 15.40s/it]
 78%|███████▊  | 7708/9864 [15:48:44<9:16:57, 15.50s/it]
 78%|███████▊  | 7709/9864 [15:48:57<8:53:38, 14.86s/it]
 78%|███████▊  | 7710/9864 [15:49:11<8:48:05, 14.71s/it]
                                                        
{'loss': '0.07443', 'grad_norm': '5.66', 'learning_rate': '2.401e-06', 'epoch': '0.7817'}

 78%|███████▊  | 7710/9864 [15:49:11<8:48:05, 14.71s/it]
 78%|███████▊  | 7711/9864 [15:49:26<8:42:52, 14.57s/it]
 78%|███████▊  | 7712/9864 [15:49:40<8:35:25, 14.37s/it]
 78%|███████▊  | 7713/9864 [15:49:56<8:53:23, 14.88s/it]
 78%|███████▊  | 7714/9864 [15:50:12<9:11:39, 15.40s/it]
 78%|███████▊  | 7715/9864 [15:50:28<9:16:57, 15.55s/it]
 78%|███████▊  | 7716/9864 [15:50:42<8:59:04, 15.06s/it]
 78%|███████▊  | 7717/9864 [15:50:56<8:47:59, 14.76s/it]
 78%|███████▊  | 7718/9864 [15:51:10<8:39:42, 14.53s/it]
 78%|███████▊  | 7719/9864 [15:51:24<8:34:04, 14.38s/it]
 78%|███████▊  | 7720/9864 [15:51:38<8:23:39, 14.10s/it]
                                                        
{'loss': '0.07647', 'grad_norm': '6.313', 'learning_rate': '2.379e-06', 'epoch': '0.7827'}

 78%|███████▊  | 7720/9864 [15:51:38<8:23:39, 14.10s/it]
 78%|███████▊  | 7721/9864 [15:51:55<9:00:26, 15.13s/it]
 78%|███████▊  | 7722/9864 [15:52:12<9:22:09, 15.75s/it]
 78%|███████▊  | 7723/9864 [15:52:26<9:02:26, 15.20s/it]
 78%|███████▊  | 7724/9864 [15:52:41<8:52:10, 14.92s/it]
 78%|███████▊  | 7725/9864 [15:52:54<8:39:48, 14.58s/it]
 78%|███████▊  | 7726/9864 [15:53:08<8:35:21, 14.46s/it]
 78%|███████▊  | 7727/9864 [15:53:22<8:29:05, 14.29s/it]
 78%|███████▊  | 7728/9864 [15:53:36<8:20:22, 14.06s/it]
 78%|███████▊  | 7729/9864 [15:53:52<8:39:55, 14.61s/it]
 78%|███████▊  | 7730/9864 [15:54:11<9:27:35, 15.96s/it]
                                                        
{'loss': '0.06195', 'grad_norm': '6.048', 'learning_rate': '2.358e-06', 'epoch': '0.7837'}

 78%|███████▊  | 7730/9864 [15:54:11<9:27:35, 15.96s/it]
 78%|███████▊  | 7731/9864 [15:54:25<9:03:29, 15.29s/it]
 78%|███████▊  | 7732/9864 [15:54:39<8:50:10, 14.92s/it]
 78%|███████▊  | 7733/9864 [15:54:53<8:42:03, 14.70s/it]
 78%|███████▊  | 7734/9864 [15:55:07<8:37:21, 14.57s/it]
 78%|███████▊  | 7735/9864 [15:55:21<8:28:54, 14.34s/it]
 78%|███████▊  | 7736/9864 [15:55:35<8:27:59, 14.32s/it]
 78%|███████▊  | 7737/9864 [15:55:52<8:57:09, 15.15s/it]
 78%|███████▊  | 7738/9864 [15:56:08<9:02:18, 15.31s/it]
 78%|███████▊  | 7739/9864 [15:56:23<9:01:44, 15.30s/it]
 78%|███████▊  | 7740/9864 [15:56:37<8:49:05, 14.95s/it]
                                                        
{'loss': '0.07526', 'grad_norm': '4.98', 'learning_rate': '2.337e-06', 'epoch': '0.7847'}

 78%|███████▊  | 7740/9864 [15:56:37<8:49:05, 14.95s/it]
 78%|███████▊  | 7741/9864 [15:56:52<8:42:03, 14.75s/it]
 78%|███████▊  | 7742/9864 [15:57:06<8:34:20, 14.54s/it]
 78%|███████▊  | 7743/9864 [15:57:20<8:27:24, 14.35s/it]
 79%|███████▊  | 7744/9864 [15:57:37<9:00:01, 15.28s/it]
 79%|███████▊  | 7745/9864 [15:57:51<8:48:44, 14.97s/it]
 79%|███████▊  | 7746/9864 [15:58:07<8:56:59, 15.21s/it]
 79%|███████▊  | 7747/9864 [15:58:21<8:44:49, 14.87s/it]
 79%|███████▊  | 7748/9864 [15:58:37<8:53:29, 15.13s/it]
 79%|███████▊  | 7749/9864 [15:58:51<8:42:55, 14.83s/it]
 79%|███████▊  | 7750/9864 [15:59:05<8:37:28, 14.69s/it]
                                                        
{'loss': '0.07419', 'grad_norm': '5.158', 'learning_rate': '2.316e-06', 'epoch': '0.7858'}

 79%|███████▊  | 7750/9864 [15:59:05<8:37:28, 14.69s/it]
 79%|███████▊  | 7751/9864 [15:59:21<8:46:12, 14.94s/it]
 79%|███████▊  | 7752/9864 [15:59:36<8:49:56, 15.06s/it]
 79%|███████▊  | 7753/9864 [15:59:50<8:37:20, 14.70s/it]
 79%|███████▊  | 7754/9864 [16:00:06<8:46:39, 14.98s/it]
 79%|███████▊  | 7755/9864 [16:00:22<8:57:08, 15.28s/it]
 79%|███████▊  | 7756/9864 [16:00:36<8:44:02, 14.92s/it]
 79%|███████▊  | 7757/9864 [16:00:50<8:38:04, 14.75s/it]
 79%|███████▊  | 7758/9864 [16:01:06<8:45:49, 14.98s/it]
 79%|███████▊  | 7759/9864 [16:01:20<8:39:52, 14.82s/it]
 79%|███████▊  | 7760/9864 [16:01:35<8:44:54, 14.97s/it]
                                                        
{'loss': '0.07334', 'grad_norm': '4.97', 'learning_rate': '2.295e-06', 'epoch': '0.7868'}

 79%|███████▊  | 7760/9864 [16:01:35<8:44:54, 14.97s/it]
 79%|███████▊  | 7761/9864 [16:01:52<8:57:07, 15.32s/it]
 79%|███████▊  | 7762/9864 [16:02:06<8:42:34, 14.92s/it]
 79%|███████▊  | 7763/9864 [16:02:20<8:36:22, 14.75s/it]
 79%|███████▊  | 7764/9864 [16:02:36<8:45:23, 15.01s/it]
 79%|███████▊  | 7765/9864 [16:02:50<8:36:01, 14.75s/it]
 79%|███████▊  | 7766/9864 [16:03:04<8:28:27, 14.54s/it]
 79%|███████▊  | 7767/9864 [16:03:20<8:41:12, 14.91s/it]
 79%|███████▉  | 7768/9864 [16:03:35<8:51:25, 15.21s/it]
 79%|███████▉  | 7769/9864 [16:03:52<9:00:41, 15.49s/it]
 79%|███████▉  | 7770/9864 [16:04:06<8:45:17, 15.05s/it]
                                                        
{'loss': '0.07598', 'grad_norm': '7.445', 'learning_rate': '2.274e-06', 'epoch': '0.7878'}

 79%|███████▉  | 7770/9864 [16:04:06<8:45:17, 15.05s/it]
 79%|███████▉  | 7771/9864 [16:04:21<8:52:54, 15.28s/it]
 79%|███████▉  | 7772/9864 [16:04:36<8:41:01, 14.94s/it]
 79%|███████▉  | 7773/9864 [16:04:50<8:33:24, 14.73s/it]
 79%|███████▉  | 7774/9864 [16:05:04<8:26:48, 14.55s/it]
 79%|███████▉  | 7775/9864 [16:05:20<8:41:53, 14.99s/it]
 79%|███████▉  | 7776/9864 [16:05:36<8:56:18, 15.41s/it]
 79%|███████▉  | 7777/9864 [16:05:52<9:00:58, 15.55s/it]
 79%|███████▉  | 7778/9864 [16:06:06<8:43:51, 15.07s/it]
 79%|███████▉  | 7779/9864 [16:06:22<8:47:44, 15.19s/it]
 79%|███████▉  | 7780/9864 [16:06:36<8:34:45, 14.82s/it]
                                                        
{'loss': '0.07964', 'grad_norm': '4.6', 'learning_rate': '2.253e-06', 'epoch': '0.7888'}

 79%|███████▉  | 7780/9864 [16:06:36<8:34:45, 14.82s/it]
 79%|███████▉  | 7781/9864 [16:06:50<8:30:51, 14.72s/it]
 79%|███████▉  | 7782/9864 [16:07:04<8:27:10, 14.62s/it]
 79%|███████▉  | 7783/9864 [16:07:20<8:36:50, 14.90s/it]
 79%|███████▉  | 7784/9864 [16:07:37<9:01:35, 15.62s/it]
 79%|███████▉  | 7785/9864 [16:07:52<8:47:48, 15.23s/it]
 79%|███████▉  | 7786/9864 [16:08:07<8:53:27, 15.40s/it]
 79%|███████▉  | 7787/9864 [16:08:22<8:43:39, 15.13s/it]
 79%|███████▉  | 7788/9864 [16:08:36<8:31:41, 14.79s/it]
 79%|███████▉  | 7789/9864 [16:08:50<8:26:22, 14.64s/it]
 79%|███████▉  | 7790/9864 [16:09:06<8:39:59, 15.04s/it]
                                                        
{'loss': '0.07438', 'grad_norm': '4.203', 'learning_rate': '2.233e-06', 'epoch': '0.7898'}

 79%|███████▉  | 7790/9864 [16:09:06<8:39:59, 15.04s/it]
 79%|███████▉  | 7791/9864 [16:09:22<8:48:07, 15.29s/it]
 79%|███████▉  | 7792/9864 [16:09:37<8:46:04, 15.23s/it]
 79%|███████▉  | 7793/9864 [16:09:51<8:35:45, 14.94s/it]
 79%|███████▉  | 7794/9864 [16:10:07<8:42:55, 15.16s/it]
 79%|███████▉  | 7795/9864 [16:10:21<8:33:50, 14.90s/it]
 79%|███████▉  | 7796/9864 [16:10:35<8:22:51, 14.59s/it]
 79%|███████▉  | 7797/9864 [16:10:49<8:14:27, 14.35s/it]
 79%|███████▉  | 7798/9864 [16:11:05<8:25:34, 14.68s/it]
 79%|███████▉  | 7799/9864 [16:11:22<8:54:41, 15.54s/it]
 79%|███████▉  | 7800/9864 [16:11:36<8:37:56, 15.06s/it]
                                                        
{'loss': '0.07117', 'grad_norm': '5.3', 'learning_rate': '2.212e-06', 'epoch': '0.7908'}

 79%|███████▉  | 7800/9864 [16:11:36<8:37:56, 15.06s/it]
 79%|███████▉  | 7801/9864 [16:11:50<8:22:38, 14.62s/it]
 79%|███████▉  | 7802/9864 [16:12:05<8:33:25, 14.94s/it]
 79%|███████▉  | 7803/9864 [16:12:19<8:21:22, 14.60s/it]
 79%|███████▉  | 7804/9864 [16:12:33<8:16:36, 14.46s/it]
 79%|███████▉  | 7805/9864 [16:12:47<8:10:49, 14.30s/it]
 79%|███████▉  | 7806/9864 [16:13:04<8:40:52, 15.19s/it]
 79%|███████▉  | 7807/9864 [16:13:20<8:47:32, 15.39s/it]
 79%|███████▉  | 7808/9864 [16:13:34<8:34:59, 15.03s/it]
 79%|███████▉  | 7809/9864 [16:13:48<8:23:01, 14.69s/it]
 79%|███████▉  | 7810/9864 [16:14:04<8:37:20, 15.11s/it]
                                                        
{'loss': '0.1026', 'grad_norm': '5.754', 'learning_rate': '2.191e-06', 'epoch': '0.7918'}

 79%|███████▉  | 7810/9864 [16:14:04<8:37:20, 15.11s/it]
 79%|███████▉  | 7811/9864 [16:14:19<8:28:12, 14.85s/it]
 79%|███████▉  | 7812/9864 [16:14:33<8:21:05, 14.65s/it]
 79%|███████▉  | 7813/9864 [16:14:49<8:33:14, 15.01s/it]
 79%|███████▉  | 7814/9864 [16:15:04<8:40:26, 15.23s/it]
 79%|███████▉  | 7815/9864 [16:15:20<8:47:22, 15.44s/it]
 79%|███████▉  | 7816/9864 [16:15:35<8:36:17, 15.13s/it]
 79%|███████▉  | 7817/9864 [16:15:49<8:28:11, 14.90s/it]
 79%|███████▉  | 7818/9864 [16:16:05<8:34:55, 15.10s/it]
 79%|███████▉  | 7819/9864 [16:16:19<8:28:39, 14.92s/it]
 79%|███████▉  | 7820/9864 [16:16:33<8:18:06, 14.62s/it]
                                                        
{'loss': '0.08306', 'grad_norm': '3.986', 'learning_rate': '2.171e-06', 'epoch': '0.7928'}

 79%|███████▉  | 7820/9864 [16:16:33<8:18:06, 14.62s/it]
 79%|███████▉  | 7821/9864 [16:16:48<8:22:39, 14.76s/it]
 79%|███████▉  | 7822/9864 [16:17:04<8:35:15, 15.14s/it]
 79%|███████▉  | 7823/9864 [16:17:20<8:45:01, 15.43s/it]
 79%|███████▉  | 7824/9864 [16:17:34<8:29:52, 15.00s/it]
 79%|███████▉  | 7825/9864 [16:17:48<8:17:21, 14.64s/it]
 79%|███████▉  | 7826/9864 [16:18:04<8:27:49, 14.95s/it]
 79%|███████▉  | 7827/9864 [16:18:18<8:18:51, 14.69s/it]
 79%|███████▉  | 7828/9864 [16:18:32<8:11:22, 14.48s/it]
 79%|███████▉  | 7829/9864 [16:18:48<8:25:51, 14.91s/it]
 79%|███████▉  | 7830/9864 [16:19:03<8:32:27, 15.12s/it]
                                                        
{'loss': '0.07623', 'grad_norm': '4.79', 'learning_rate': '2.15e-06', 'epoch': '0.7939'}

 79%|███████▉  | 7830/9864 [16:19:03<8:32:27, 15.12s/it]
 79%|███████▉  | 7831/9864 [16:19:18<8:23:25, 14.86s/it]
 79%|███████▉  | 7832/9864 [16:19:34<8:34:54, 15.20s/it]
 79%|███████▉  | 7833/9864 [16:19:49<8:39:26, 15.35s/it]
 79%|███████▉  | 7834/9864 [16:20:03<8:26:03, 14.96s/it]
 79%|███████▉  | 7835/9864 [16:20:17<8:12:45, 14.57s/it]
 79%|███████▉  | 7836/9864 [16:20:31<8:06:03, 14.38s/it]
 79%|███████▉  | 7837/9864 [16:20:47<8:19:04, 14.77s/it]
 79%|███████▉  | 7838/9864 [16:21:01<8:11:22, 14.55s/it]
 79%|███████▉  | 7839/9864 [16:21:17<8:23:22, 14.91s/it]
 79%|███████▉  | 7840/9864 [16:21:31<8:18:00, 14.76s/it]
                                                        
{'loss': '0.0715', 'grad_norm': '4.478', 'learning_rate': '2.13e-06', 'epoch': '0.7949'}

 79%|███████▉  | 7840/9864 [16:21:31<8:18:00, 14.76s/it]
 79%|███████▉  | 7841/9864 [16:21:47<8:26:39, 15.03s/it]
 80%|███████▉  | 7842/9864 [16:22:01<8:19:42, 14.83s/it]
 80%|███████▉  | 7843/9864 [16:22:16<8:17:39, 14.77s/it]
 80%|███████▉  | 7844/9864 [16:22:32<8:29:09, 15.12s/it]
 80%|███████▉  | 7845/9864 [16:22:46<8:20:25, 14.87s/it]
 80%|███████▉  | 7846/9864 [16:23:01<8:27:49, 15.10s/it]
 80%|███████▉  | 7847/9864 [16:23:16<8:21:24, 14.92s/it]
 80%|███████▉  | 7848/9864 [16:23:32<8:35:04, 15.33s/it]
 80%|███████▉  | 7849/9864 [16:23:48<8:40:33, 15.50s/it]
 80%|███████▉  | 7850/9864 [16:24:02<8:22:55, 14.98s/it]
                                                        
{'loss': '0.06599', 'grad_norm': '5.395', 'learning_rate': '2.11e-06', 'epoch': '0.7959'}

 80%|███████▉  | 7850/9864 [16:24:02<8:22:55, 14.98s/it]
 80%|███████▉  | 7851/9864 [16:24:16<8:13:30, 14.71s/it]
 80%|███████▉  | 7852/9864 [16:24:32<8:24:50, 15.05s/it]
 80%|███████▉  | 7853/9864 [16:24:46<8:15:06, 14.77s/it]
 80%|███████▉  | 7854/9864 [16:25:02<8:25:55, 15.10s/it]
 80%|███████▉  | 7855/9864 [16:25:17<8:28:58, 15.20s/it]
 80%|███████▉  | 7856/9864 [16:25:32<8:22:23, 15.01s/it]
 80%|███████▉  | 7857/9864 [16:25:47<8:28:37, 15.21s/it]
 80%|███████▉  | 7858/9864 [16:26:02<8:19:32, 14.94s/it]
 80%|███████▉  | 7859/9864 [16:26:16<8:08:11, 14.61s/it]
 80%|███████▉  | 7860/9864 [16:26:30<8:04:17, 14.50s/it]
                                                        
{'loss': '0.06782', 'grad_norm': '5.496', 'learning_rate': '2.09e-06', 'epoch': '0.7969'}

 80%|███████▉  | 7860/9864 [16:26:30<8:04:17, 14.50s/it]
 80%|███████▉  | 7861/9864 [16:26:45<8:07:49, 14.61s/it]
 80%|███████▉  | 7862/9864 [16:27:02<8:33:35, 15.39s/it]
 80%|███████▉  | 7863/9864 [16:27:16<8:17:48, 14.93s/it]
 80%|███████▉  | 7864/9864 [16:27:32<8:31:11, 15.34s/it]
 80%|███████▉  | 7865/9864 [16:27:46<8:17:14, 14.92s/it]
 80%|███████▉  | 7866/9864 [16:28:00<8:07:30, 14.64s/it]
 80%|███████▉  | 7867/9864 [16:28:14<8:00:49, 14.45s/it]
 80%|███████▉  | 7868/9864 [16:28:28<7:59:56, 14.43s/it]
 80%|███████▉  | 7869/9864 [16:28:44<8:06:36, 14.63s/it]
 80%|███████▉  | 7870/9864 [16:28:59<8:11:43, 14.80s/it]
                                                        
{'loss': '0.07787', 'grad_norm': '5.224', 'learning_rate': '2.07e-06', 'epoch': '0.7979'}

 80%|███████▉  | 7870/9864 [16:28:59<8:11:43, 14.80s/it]
 80%|███████▉  | 7871/9864 [16:29:15<8:25:52, 15.23s/it]
 80%|███████▉  | 7872/9864 [16:29:30<8:25:15, 15.22s/it]
 80%|███████▉  | 7873/9864 [16:29:44<8:07:41, 14.70s/it]
 80%|███████▉  | 7874/9864 [16:29:58<8:03:30, 14.58s/it]
 80%|███████▉  | 7875/9864 [16:30:12<7:58:50, 14.44s/it]
 80%|███████▉  | 7876/9864 [16:30:27<7:59:12, 14.46s/it]
 80%|███████▉  | 7877/9864 [16:30:41<7:54:56, 14.34s/it]
 80%|███████▉  | 7878/9864 [16:31:00<8:42:19, 15.78s/it]
 80%|███████▉  | 7879/9864 [16:31:14<8:21:49, 15.17s/it]
 80%|███████▉  | 7880/9864 [16:31:29<8:28:45, 15.39s/it]
                                                        
{'loss': '0.07508', 'grad_norm': '7.244', 'learning_rate': '2.05e-06', 'epoch': '0.7989'}

 80%|███████▉  | 7880/9864 [16:31:29<8:28:45, 15.39s/it]
 80%|███████▉  | 7881/9864 [16:31:44<8:18:08, 15.07s/it]
 80%|███████▉  | 7882/9864 [16:31:58<8:09:56, 14.83s/it]
 80%|███████▉  | 7883/9864 [16:32:12<7:59:02, 14.51s/it]
 80%|███████▉  | 7884/9864 [16:32:26<7:55:29, 14.41s/it]
 80%|███████▉  | 7885/9864 [16:32:41<8:03:08, 14.65s/it]
 80%|███████▉  | 7886/9864 [16:32:58<8:29:24, 15.45s/it]
 80%|███████▉  | 7887/9864 [16:33:13<8:21:36, 15.22s/it]
 80%|███████▉  | 7888/9864 [16:33:29<8:27:47, 15.42s/it]
 80%|███████▉  | 7889/9864 [16:33:43<8:16:02, 15.07s/it]
 80%|███████▉  | 7890/9864 [16:33:58<8:09:50, 14.89s/it]
                                                        
{'loss': '0.08447', 'grad_norm': '5.071', 'learning_rate': '2.03e-06', 'epoch': '0.7999'}

 80%|███████▉  | 7890/9864 [16:33:58<8:09:50, 14.89s/it]
 80%|███████▉  | 7891/9864 [16:34:12<8:02:37, 14.68s/it]
 80%|████████  | 7892/9864 [16:34:26<7:54:25, 14.43s/it]
 80%|████████  | 7893/9864 [16:34:42<8:11:42, 14.97s/it]
 80%|████████  | 7894/9864 [16:34:58<8:21:27, 15.27s/it]
 80%|████████  | 7895/9864 [16:35:14<8:28:17, 15.49s/it]
 80%|████████  | 7896/9864 [16:35:28<8:14:29, 15.08s/it]
 80%|████████  | 7897/9864 [16:35:43<8:15:40, 15.12s/it]
 80%|████████  | 7898/9864 [16:35:57<8:03:40, 14.76s/it]
 80%|████████  | 7899/9864 [16:36:12<8:01:13, 14.69s/it]
 80%|████████  | 7900/9864 [16:36:26<7:53:00, 14.45s/it]
                                                        
{'loss': '0.08021', 'grad_norm': '4.66', 'learning_rate': '2.01e-06', 'epoch': '0.801'}

 80%|████████  | 7900/9864 [16:36:26<7:53:00, 14.45s/it]
 80%|████████  | 7901/9864 [16:36:43<8:23:43, 15.40s/it]
 80%|████████  | 7902/9864 [16:36:59<8:21:43, 15.34s/it]
 80%|████████  | 7903/9864 [16:37:13<8:13:35, 15.10s/it]
 80%|████████  | 7904/9864 [16:37:29<8:20:39, 15.33s/it]
 80%|████████  | 7905/9864 [16:37:43<8:12:33, 15.09s/it]
 80%|████████  | 7906/9864 [16:37:58<8:05:23, 14.87s/it]
 80%|████████  | 7907/9864 [16:38:12<7:59:02, 14.69s/it]
 80%|████████  | 7908/9864 [16:38:26<7:47:48, 14.35s/it]
 80%|████████  | 7909/9864 [16:38:43<8:19:34, 15.33s/it]
 80%|████████  | 7910/9864 [16:38:57<8:06:45, 14.95s/it]
                                                        
{'loss': '0.07037', 'grad_norm': '4.884', 'learning_rate': '1.99e-06', 'epoch': '0.802'}

 80%|████████  | 7910/9864 [16:38:57<8:06:45, 14.95s/it]
 80%|████████  | 7911/9864 [16:39:11<7:57:58, 14.68s/it]
 80%|████████  | 7912/9864 [16:39:27<8:06:27, 14.95s/it]
 80%|████████  | 7913/9864 [16:39:41<7:59:25, 14.74s/it]
 80%|████████  | 7914/9864 [16:39:55<7:54:18, 14.59s/it]
 80%|████████  | 7915/9864 [16:40:10<7:49:04, 14.44s/it]
 80%|████████  | 7916/9864 [16:40:27<8:14:54, 15.24s/it]
 80%|████████  | 7917/9864 [16:40:42<8:20:22, 15.42s/it]
 80%|████████  | 7918/9864 [16:40:56<8:06:40, 15.01s/it]
 80%|████████  | 7919/9864 [16:41:11<8:00:28, 14.82s/it]
 80%|████████  | 7920/9864 [16:41:27<8:09:14, 15.10s/it]
                                                        
{'loss': '0.0718', 'grad_norm': '3.986', 'learning_rate': '1.971e-06', 'epoch': '0.803'}

 80%|████████  | 7920/9864 [16:41:27<8:09:14, 15.10s/it]
 80%|████████  | 7921/9864 [16:41:41<8:00:51, 14.85s/it]
 80%|████████  | 7922/9864 [16:41:55<7:57:01, 14.74s/it]
 80%|████████  | 7923/9864 [16:42:10<8:00:03, 14.84s/it]
 80%|████████  | 7924/9864 [16:42:26<8:09:22, 15.14s/it]
 80%|████████  | 7925/9864 [16:42:42<8:19:30, 15.46s/it]
 80%|████████  | 7926/9864 [16:42:57<8:05:40, 15.04s/it]
 80%|████████  | 7927/9864 [16:43:11<7:56:24, 14.76s/it]
 80%|████████  | 7928/9864 [16:43:25<7:53:30, 14.67s/it]
 80%|████████  | 7929/9864 [16:43:41<8:00:27, 14.90s/it]
 80%|████████  | 7930/9864 [16:43:54<7:49:27, 14.56s/it]
                                                        
{'loss': '0.06167', 'grad_norm': '4.655', 'learning_rate': '1.951e-06', 'epoch': '0.804'}

 80%|████████  | 7930/9864 [16:43:54<7:49:27, 14.56s/it]
 80%|████████  | 7931/9864 [16:44:10<8:04:27, 15.04s/it]
 80%|████████  | 7932/9864 [16:44:28<8:24:54, 15.68s/it]
 80%|████████  | 7933/9864 [16:44:42<8:10:28, 15.24s/it]
 80%|████████  | 7934/9864 [16:44:56<7:57:34, 14.85s/it]
 80%|████████  | 7935/9864 [16:45:10<7:49:47, 14.61s/it]
 80%|████████  | 7936/9864 [16:45:24<7:47:55, 14.56s/it]
 80%|████████  | 7937/9864 [16:45:40<7:54:25, 14.77s/it]
 80%|████████  | 7938/9864 [16:45:53<7:45:52, 14.51s/it]
 80%|████████  | 7939/9864 [16:46:10<8:00:28, 14.98s/it]
 80%|████████  | 7940/9864 [16:46:27<8:23:45, 15.71s/it]
                                                        
{'loss': '0.06046', 'grad_norm': '7.06', 'learning_rate': '1.932e-06', 'epoch': '0.805'}

 80%|████████  | 7940/9864 [16:46:27<8:23:45, 15.71s/it]
 81%|████████  | 7941/9864 [16:46:41<8:10:29, 15.30s/it]
 81%|████████  | 7942/9864 [16:46:56<8:01:02, 15.02s/it]
 81%|████████  | 7943/9864 [16:47:10<7:52:51, 14.77s/it]
 81%|████████  | 7944/9864 [16:47:24<7:47:44, 14.62s/it]
 81%|████████  | 7945/9864 [16:47:40<8:01:56, 15.07s/it]
 81%|████████  | 7946/9864 [16:47:54<7:52:25, 14.78s/it]
 81%|████████  | 7947/9864 [16:48:11<8:07:50, 15.27s/it]
 81%|████████  | 7948/9864 [16:48:26<8:10:27, 15.36s/it]
 81%|████████  | 7949/9864 [16:48:40<7:58:13, 14.98s/it]
 81%|████████  | 7950/9864 [16:48:54<7:46:38, 14.63s/it]
                                                        
{'loss': '0.07649', 'grad_norm': '5.559', 'learning_rate': '1.913e-06', 'epoch': '0.806'}

 81%|████████  | 7950/9864 [16:48:54<7:46:38, 14.63s/it]
 81%|████████  | 7951/9864 [16:49:09<7:44:00, 14.55s/it]
 81%|████████  | 7952/9864 [16:49:23<7:42:22, 14.51s/it]
 81%|████████  | 7953/9864 [16:49:39<7:56:42, 14.97s/it]
 81%|████████  | 7954/9864 [16:49:57<8:20:44, 15.73s/it]
 81%|████████  | 7955/9864 [16:50:10<8:02:44, 15.17s/it]
 81%|████████  | 7956/9864 [16:50:26<8:04:15, 15.23s/it]
 81%|████████  | 7957/9864 [16:50:40<7:52:48, 14.88s/it]
 81%|████████  | 7958/9864 [16:50:54<7:43:27, 14.59s/it]
 81%|████████  | 7959/9864 [16:51:07<7:34:21, 14.31s/it]
 81%|████████  | 7960/9864 [16:51:22<7:35:52, 14.37s/it]
                                                        
{'loss': '0.07123', 'grad_norm': '4.825', 'learning_rate': '1.893e-06', 'epoch': '0.807'}

 81%|████████  | 7960/9864 [16:51:22<7:35:52, 14.37s/it]
 81%|████████  | 7961/9864 [16:51:38<7:47:20, 14.73s/it]
 81%|████████  | 7962/9864 [16:51:55<8:08:52, 15.42s/it]
 81%|████████  | 7963/9864 [16:52:09<7:58:12, 15.09s/it]
 81%|████████  | 7964/9864 [16:52:25<8:05:47, 15.34s/it]
 81%|████████  | 7965/9864 [16:52:39<7:51:32, 14.90s/it]
 81%|████████  | 7966/9864 [16:52:53<7:44:10, 14.67s/it]
 81%|████████  | 7967/9864 [16:53:07<7:40:55, 14.58s/it]
 81%|████████  | 7968/9864 [16:53:23<7:51:29, 14.92s/it]
 81%|████████  | 7969/9864 [16:53:39<8:03:35, 15.31s/it]
 81%|████████  | 7970/9864 [16:53:54<8:02:01, 15.27s/it]
                                                        
{'loss': '0.07119', 'grad_norm': '5.746', 'learning_rate': '1.874e-06', 'epoch': '0.8081'}

 81%|████████  | 7970/9864 [16:53:54<8:02:01, 15.27s/it]
 81%|████████  | 7971/9864 [16:54:09<7:52:36, 14.98s/it]
 81%|████████  | 7972/9864 [16:54:25<8:02:07, 15.29s/it]
 81%|████████  | 7973/9864 [16:54:39<7:50:44, 14.94s/it]
 81%|████████  | 7974/9864 [16:54:52<7:39:18, 14.58s/it]
 81%|████████  | 7975/9864 [16:55:07<7:36:45, 14.51s/it]
 81%|████████  | 7976/9864 [16:55:23<7:51:23, 14.98s/it]
 81%|████████  | 7977/9864 [16:55:39<7:57:20, 15.18s/it]
 81%|████████  | 7978/9864 [16:55:55<8:04:54, 15.43s/it]
 81%|████████  | 7979/9864 [16:56:09<7:51:28, 15.01s/it]
 81%|████████  | 7980/9864 [16:56:24<7:59:15, 15.26s/it]
                                                        
{'loss': '0.07407', 'grad_norm': '6.285', 'learning_rate': '1.855e-06', 'epoch': '0.8091'}

 81%|████████  | 7980/9864 [16:56:24<7:59:15, 15.26s/it]
 81%|████████  | 7981/9864 [16:56:38<7:44:31, 14.80s/it]
 81%|████████  | 7982/9864 [16:56:52<7:38:51, 14.63s/it]
 81%|████████  | 7983/9864 [16:57:08<7:50:36, 15.01s/it]
 81%|████████  | 7984/9864 [16:57:22<7:38:19, 14.63s/it]
 81%|████████  | 7985/9864 [16:57:39<7:57:49, 15.26s/it]
 81%|████████  | 7986/9864 [16:57:52<7:43:04, 14.79s/it]
 81%|████████  | 7987/9864 [16:58:09<7:56:06, 15.22s/it]
 81%|████████  | 7988/9864 [16:58:23<7:46:56, 14.93s/it]
 81%|████████  | 7989/9864 [16:58:37<7:35:43, 14.58s/it]
 81%|████████  | 7990/9864 [16:58:51<7:33:50, 14.53s/it]
                                                        
{'loss': '0.06125', 'grad_norm': '3.771', 'learning_rate': '1.836e-06', 'epoch': '0.8101'}

 81%|████████  | 7990/9864 [16:58:51<7:33:50, 14.53s/it]
 81%|████████  | 7991/9864 [16:59:05<7:27:18, 14.33s/it]
 81%|████████  | 7992/9864 [16:59:24<8:13:29, 15.82s/it]
 81%|████████  | 7993/9864 [16:59:38<7:57:59, 15.33s/it]
 81%|████████  | 7994/9864 [16:59:53<7:46:35, 14.97s/it]
 81%|████████  | 7995/9864 [17:00:08<7:50:05, 15.09s/it]
 81%|████████  | 7996/9864 [17:00:22<7:42:13, 14.85s/it]
 81%|████████  | 7997/9864 [17:00:36<7:35:06, 14.63s/it]
 81%|████████  | 7998/9864 [17:00:51<7:35:09, 14.64s/it]
 81%|████████  | 7999/9864 [17:01:07<7:49:31, 15.11s/it]
 81%|████████  | 8000/9864 [17:01:24<8:07:20, 15.69s/it]
                                                        
{'loss': '0.07748', 'grad_norm': '3.462', 'learning_rate': '1.817e-06', 'epoch': '0.8111'}

 81%|████████  | 8000/9864 [17:01:24<8:07:20, 15.69s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.73s/it]
Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.73s/it]
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 81%|████████  | 8001/9864 [17:02:44<18:02:28, 34.86s/it]
 81%|████████  | 8002/9864 [17:02:58<14:45:45, 28.54s/it]
 81%|████████  | 8003/9864 [17:03:13<12:44:39, 24.65s/it]
 81%|████████  | 8004/9864 [17:03:27<11:04:35, 21.44s/it]
 81%|████████  | 8005/9864 [17:03:41<9:54:37, 19.19s/it] 
 81%|████████  | 8006/9864 [17:03:57<9:22:09, 18.15s/it]
 81%|████████  | 8007/9864 [17:04:12<8:56:35, 17.34s/it]
 81%|████████  | 8008/9864 [17:04:28<8:40:31, 16.83s/it]
 81%|████████  | 8009/9864 [17:04:43<8:19:38, 16.16s/it]
 81%|████████  | 8010/9864 [17:04:56<7:58:09, 15.47s/it]
                                                        
{'loss': '0.07639', 'grad_norm': '5.06', 'learning_rate': '1.798e-06', 'epoch': '0.8121'}

 81%|████████  | 8010/9864 [17:04:56<7:58:09, 15.47s/it]
 81%|████████  | 8011/9864 [17:05:12<8:03:20, 15.65s/it]
 81%|████████  | 8012/9864 [17:05:27<7:51:50, 15.29s/it]
 81%|████████  | 8013/9864 [17:05:41<7:37:32, 14.83s/it]
 81%|████████  | 8014/9864 [17:05:54<7:27:20, 14.51s/it]
 81%|████████▏ | 8015/9864 [17:06:12<7:57:38, 15.50s/it]
 81%|████████▏ | 8016/9864 [17:06:28<8:01:10, 15.62s/it]
 81%|████████▏ | 8017/9864 [17:06:42<7:48:46, 15.23s/it]
 81%|████████▏ | 8018/9864 [17:06:56<7:32:40, 14.71s/it]
 81%|████████▏ | 8019/9864 [17:07:12<7:47:24, 15.20s/it]
 81%|████████▏ | 8020/9864 [17:07:27<7:39:08, 14.94s/it]
                                                        
{'loss': '0.08007', 'grad_norm': '4.897', 'learning_rate': '1.779e-06', 'epoch': '0.8131'}

 81%|████████▏ | 8020/9864 [17:07:27<7:39:08, 14.94s/it]
 81%|████████▏ | 8021/9864 [17:07:41<7:30:23, 14.66s/it]
 81%|████████▏ | 8022/9864 [17:07:55<7:28:06, 14.60s/it]
 81%|████████▏ | 8023/9864 [17:08:09<7:24:46, 14.50s/it]
 81%|████████▏ | 8024/9864 [17:08:28<8:05:59, 15.85s/it]
 81%|████████▏ | 8025/9864 [17:08:43<7:51:00, 15.37s/it]
 81%|████████▏ | 8026/9864 [17:08:56<7:35:59, 14.89s/it]
 81%|████████▏ | 8027/9864 [17:09:12<7:42:28, 15.11s/it]
 81%|████████▏ | 8028/9864 [17:09:26<7:32:44, 14.80s/it]
 81%|████████▏ | 8029/9864 [17:09:40<7:28:06, 14.65s/it]
 81%|████████▏ | 8030/9864 [17:09:55<7:24:59, 14.56s/it]
                                                        
{'loss': '0.06125', 'grad_norm': '3.405', 'learning_rate': '1.761e-06', 'epoch': '0.8141'}

 81%|████████▏ | 8030/9864 [17:09:55<7:24:59, 14.56s/it]
 81%|████████▏ | 8031/9864 [17:10:13<7:55:21, 15.56s/it]
 81%|████████▏ | 8032/9864 [17:10:29<8:01:27, 15.77s/it]
 81%|████████▏ | 8033/9864 [17:10:43<7:48:27, 15.35s/it]
 81%|████████▏ | 8034/9864 [17:10:59<7:53:56, 15.54s/it]
 81%|████████▏ | 8035/9864 [17:11:13<7:38:12, 15.03s/it]
 81%|████████▏ | 8036/9864 [17:11:27<7:25:27, 14.62s/it]
 81%|████████▏ | 8037/9864 [17:11:41<7:21:00, 14.48s/it]
 81%|████████▏ | 8038/9864 [17:11:55<7:18:55, 14.42s/it]
 81%|████████▏ | 8039/9864 [17:12:11<7:32:16, 14.87s/it]
 82%|████████▏ | 8040/9864 [17:12:27<7:41:22, 15.18s/it]
                                                        
{'loss': '0.07911', 'grad_norm': '3.926', 'learning_rate': '1.742e-06', 'epoch': '0.8152'}

 82%|████████▏ | 8040/9864 [17:12:27<7:41:22, 15.18s/it]
 82%|████████▏ | 8041/9864 [17:12:41<7:30:51, 14.84s/it]
 82%|████████▏ | 8042/9864 [17:12:57<7:40:52, 15.18s/it]
 82%|████████▏ | 8043/9864 [17:13:11<7:30:39, 14.85s/it]
 82%|████████▏ | 8044/9864 [17:13:25<7:25:36, 14.69s/it]
 82%|████████▏ | 8045/9864 [17:13:39<7:17:47, 14.44s/it]
 82%|████████▏ | 8046/9864 [17:13:54<7:17:51, 14.45s/it]
 82%|████████▏ | 8047/9864 [17:14:08<7:17:19, 14.44s/it]
 82%|████████▏ | 8048/9864 [17:14:26<7:43:55, 15.33s/it]
 82%|████████▏ | 8049/9864 [17:14:39<7:29:05, 14.85s/it]
 82%|████████▏ | 8050/9864 [17:14:55<7:38:00, 15.15s/it]
                                                        
{'loss': '0.06684', 'grad_norm': '5.647', 'learning_rate': '1.724e-06', 'epoch': '0.8162'}

 82%|████████▏ | 8050/9864 [17:14:55<7:38:00, 15.15s/it]
 82%|████████▏ | 8051/9864 [17:15:09<7:28:30, 14.84s/it]
 82%|████████▏ | 8052/9864 [17:15:23<7:22:13, 14.64s/it]
 82%|████████▏ | 8053/9864 [17:15:38<7:20:41, 14.60s/it]
 82%|████████▏ | 8054/9864 [17:15:51<7:10:50, 14.28s/it]
 82%|████████▏ | 8055/9864 [17:16:07<7:20:44, 14.62s/it]
 82%|████████▏ | 8056/9864 [17:16:24<7:45:52, 15.46s/it]
 82%|████████▏ | 8057/9864 [17:16:39<7:35:45, 15.13s/it]
 82%|████████▏ | 8058/9864 [17:16:55<7:43:50, 15.41s/it]
 82%|████████▏ | 8059/9864 [17:17:09<7:34:34, 15.11s/it]
 82%|████████▏ | 8060/9864 [17:17:23<7:23:24, 14.75s/it]
                                                        
{'loss': '0.05887', 'grad_norm': '2.807', 'learning_rate': '1.705e-06', 'epoch': '0.8172'}

 82%|████████▏ | 8060/9864 [17:17:23<7:23:24, 14.75s/it]
 82%|████████▏ | 8061/9864 [17:17:37<7:14:42, 14.47s/it]
 82%|████████▏ | 8062/9864 [17:17:50<7:07:02, 14.22s/it]
 82%|████████▏ | 8063/9864 [17:18:06<7:16:57, 14.56s/it]
 82%|████████▏ | 8064/9864 [17:18:22<7:29:46, 14.99s/it]
 82%|████████▏ | 8065/9864 [17:18:38<7:38:53, 15.30s/it]
 82%|████████▏ | 8066/9864 [17:18:54<7:42:15, 15.43s/it]
 82%|████████▏ | 8067/9864 [17:19:08<7:35:37, 15.21s/it]
 82%|████████▏ | 8068/9864 [17:19:23<7:28:15, 14.98s/it]
 82%|████████▏ | 8069/9864 [17:19:37<7:19:29, 14.69s/it]
 82%|████████▏ | 8070/9864 [17:19:51<7:13:46, 14.51s/it]
                                                        
{'loss': '0.06619', 'grad_norm': '4.809', 'learning_rate': '1.687e-06', 'epoch': '0.8182'}

 82%|████████▏ | 8070/9864 [17:19:51<7:13:46, 14.51s/it]
 82%|████████▏ | 8071/9864 [17:20:07<7:25:54, 14.92s/it]
 82%|████████▏ | 8072/9864 [17:20:21<7:16:08, 14.60s/it]
 82%|████████▏ | 8073/9864 [17:20:37<7:28:09, 15.01s/it]
 82%|████████▏ | 8074/9864 [17:20:51<7:21:14, 14.79s/it]
 82%|████████▏ | 8075/9864 [17:21:07<7:30:29, 15.11s/it]
 82%|████████▏ | 8076/9864 [17:21:21<7:22:10, 14.84s/it]
 82%|████████▏ | 8077/9864 [17:21:35<7:14:24, 14.59s/it]
 82%|████████▏ | 8078/9864 [17:21:48<7:02:31, 14.19s/it]
 82%|████████▏ | 8079/9864 [17:22:06<7:32:30, 15.21s/it]
 82%|████████▏ | 8080/9864 [17:22:20<7:21:20, 14.84s/it]
                                                        
{'loss': '0.06632', 'grad_norm': '4.907', 'learning_rate': '1.669e-06', 'epoch': '0.8192'}

 82%|████████▏ | 8080/9864 [17:22:20<7:21:20, 14.84s/it]
 82%|████████▏ | 8081/9864 [17:22:36<7:30:25, 15.16s/it]
 82%|████████▏ | 8082/9864 [17:22:49<7:18:13, 14.76s/it]
 82%|████████▏ | 8083/9864 [17:23:05<7:24:30, 14.98s/it]
 82%|████████▏ | 8084/9864 [17:23:19<7:13:17, 14.61s/it]
 82%|████████▏ | 8085/9864 [17:23:33<7:07:00, 14.40s/it]
 82%|████████▏ | 8086/9864 [17:23:47<7:04:14, 14.32s/it]
 82%|████████▏ | 8087/9864 [17:24:04<7:32:21, 15.27s/it]
 82%|████████▏ | 8088/9864 [17:24:19<7:25:35, 15.05s/it]
 82%|████████▏ | 8089/9864 [17:24:35<7:32:51, 15.31s/it]
 82%|████████▏ | 8090/9864 [17:24:49<7:23:57, 15.02s/it]
                                                        
{'loss': '0.08035', 'grad_norm': '5.804', 'learning_rate': '1.651e-06', 'epoch': '0.8202'}

 82%|████████▏ | 8090/9864 [17:24:49<7:23:57, 15.02s/it]
 82%|████████▏ | 8091/9864 [17:25:04<7:26:38, 15.12s/it]
 82%|████████▏ | 8092/9864 [17:25:18<7:17:57, 14.83s/it]
 82%|████████▏ | 8093/9864 [17:25:33<7:12:07, 14.64s/it]
 82%|████████▏ | 8094/9864 [17:25:50<7:33:11, 15.36s/it]
 82%|████████▏ | 8095/9864 [17:26:04<7:21:32, 14.98s/it]
 82%|████████▏ | 8096/9864 [17:26:18<7:13:00, 14.69s/it]
 82%|████████▏ | 8097/9864 [17:26:34<7:23:31, 15.06s/it]
 82%|████████▏ | 8098/9864 [17:26:50<7:32:58, 15.39s/it]
 82%|████████▏ | 8099/9864 [17:27:04<7:17:17, 14.87s/it]
 82%|████████▏ | 8100/9864 [17:27:17<7:06:25, 14.50s/it]
                                                        
{'loss': '0.06365', 'grad_norm': '6.948', 'learning_rate': '1.633e-06', 'epoch': '0.8212'}

 82%|████████▏ | 8100/9864 [17:27:17<7:06:25, 14.50s/it]
 82%|████████▏ | 8101/9864 [17:27:31<7:01:43, 14.35s/it]
 82%|████████▏ | 8102/9864 [17:27:49<7:32:15, 15.40s/it]
 82%|████████▏ | 8103/9864 [17:28:03<7:16:41, 14.88s/it]
 82%|████████▏ | 8104/9864 [17:28:18<7:21:13, 15.04s/it]
 82%|████████▏ | 8105/9864 [17:28:32<7:11:37, 14.72s/it]
 82%|████████▏ | 8106/9864 [17:28:46<7:02:46, 14.43s/it]
 82%|████████▏ | 8107/9864 [17:29:02<7:14:38, 14.84s/it]
 82%|████████▏ | 8108/9864 [17:29:16<7:07:17, 14.60s/it]
 82%|████████▏ | 8109/9864 [17:29:33<7:34:08, 15.53s/it]
 82%|████████▏ | 8110/9864 [17:29:47<7:19:29, 15.03s/it]
                                                        
{'loss': '0.06698', 'grad_norm': '5.836', 'learning_rate': '1.615e-06', 'epoch': '0.8222'}

 82%|████████▏ | 8110/9864 [17:29:47<7:19:29, 15.03s/it]
 82%|████████▏ | 8111/9864 [17:30:02<7:12:27, 14.80s/it]
 82%|████████▏ | 8112/9864 [17:30:17<7:17:44, 14.99s/it]
 82%|████████▏ | 8113/9864 [17:30:31<7:13:07, 14.84s/it]
 82%|████████▏ | 8114/9864 [17:30:45<7:04:25, 14.55s/it]
 82%|████████▏ | 8115/9864 [17:31:01<7:13:07, 14.86s/it]
 82%|████████▏ | 8116/9864 [17:31:15<7:04:35, 14.57s/it]
 82%|████████▏ | 8117/9864 [17:31:30<7:08:37, 14.72s/it]
 82%|████████▏ | 8118/9864 [17:31:46<7:18:09, 15.06s/it]
 82%|████████▏ | 8119/9864 [17:32:00<7:08:10, 14.72s/it]
 82%|████████▏ | 8120/9864 [17:32:15<7:15:06, 14.97s/it]
                                                        
{'loss': '0.07368', 'grad_norm': '5.066', 'learning_rate': '1.597e-06', 'epoch': '0.8233'}

 82%|████████▏ | 8120/9864 [17:32:15<7:15:06, 14.97s/it]
 82%|████████▏ | 8121/9864 [17:32:29<7:07:47, 14.73s/it]
 82%|████████▏ | 8122/9864 [17:32:43<6:59:09, 14.44s/it]
 82%|████████▏ | 8123/9864 [17:32:59<7:09:54, 14.82s/it]
 82%|████████▏ | 8124/9864 [17:33:13<7:06:55, 14.72s/it]
 82%|████████▏ | 8125/9864 [17:33:29<7:15:50, 15.04s/it]
 82%|████████▏ | 8126/9864 [17:33:45<7:19:07, 15.16s/it]
 82%|████████▏ | 8127/9864 [17:33:59<7:09:20, 14.83s/it]
 82%|████████▏ | 8128/9864 [17:34:14<7:15:32, 15.05s/it]
 82%|████████▏ | 8129/9864 [17:34:28<7:06:50, 14.76s/it]
 82%|████████▏ | 8130/9864 [17:34:42<6:59:42, 14.52s/it]
                                                        
{'loss': '0.06382', 'grad_norm': '3.894', 'learning_rate': '1.579e-06', 'epoch': '0.8243'}

 82%|████████▏ | 8130/9864 [17:34:42<6:59:42, 14.52s/it]
 82%|████████▏ | 8131/9864 [17:34:57<7:05:11, 14.72s/it]
 82%|████████▏ | 8132/9864 [17:35:12<6:59:36, 14.54s/it]
 82%|████████▏ | 8133/9864 [17:35:28<7:12:35, 14.99s/it]
 82%|████████▏ | 8134/9864 [17:35:43<7:16:48, 15.15s/it]
 82%|████████▏ | 8135/9864 [17:35:57<7:08:17, 14.86s/it]
 82%|████████▏ | 8136/9864 [17:36:13<7:11:25, 14.98s/it]
 82%|████████▏ | 8137/9864 [17:36:27<7:06:58, 14.83s/it]
 83%|████████▎ | 8138/9864 [17:36:43<7:13:50, 15.08s/it]
 83%|████████▎ | 8139/9864 [17:36:57<7:04:34, 14.77s/it]
 83%|████████▎ | 8140/9864 [17:37:12<7:12:57, 15.07s/it]
                                                        
{'loss': '0.0561', 'grad_norm': '6.331', 'learning_rate': '1.562e-06', 'epoch': '0.8253'}

 83%|████████▎ | 8140/9864 [17:37:12<7:12:57, 15.07s/it]
 83%|████████▎ | 8141/9864 [17:37:29<7:21:06, 15.36s/it]
 83%|████████▎ | 8142/9864 [17:37:43<7:11:29, 15.03s/it]
 83%|████████▎ | 8143/9864 [17:37:57<7:04:49, 14.81s/it]
 83%|████████▎ | 8144/9864 [17:38:11<6:57:02, 14.55s/it]
 83%|████████▎ | 8145/9864 [17:38:27<7:06:42, 14.89s/it]
 83%|████████▎ | 8146/9864 [17:38:40<6:53:54, 14.46s/it]
 83%|████████▎ | 8147/9864 [17:38:55<7:01:04, 14.71s/it]
 83%|████████▎ | 8148/9864 [17:39:12<7:12:35, 15.13s/it]
 83%|████████▎ | 8149/9864 [17:39:27<7:13:46, 15.18s/it]
 83%|████████▎ | 8150/9864 [17:39:41<7:05:58, 14.91s/it]
                                                        
{'loss': '0.05659', 'grad_norm': '4.903', 'learning_rate': '1.544e-06', 'epoch': '0.8263'}

 83%|████████▎ | 8150/9864 [17:39:41<7:05:58, 14.91s/it]
 83%|████████▎ | 8151/9864 [17:39:55<6:57:39, 14.63s/it]
 83%|████████▎ | 8152/9864 [17:40:09<6:52:18, 14.45s/it]
 83%|████████▎ | 8153/9864 [17:40:25<7:00:48, 14.76s/it]
 83%|████████▎ | 8154/9864 [17:40:40<7:09:37, 15.07s/it]
 83%|████████▎ | 8155/9864 [17:40:55<7:01:50, 14.81s/it]
 83%|████████▎ | 8156/9864 [17:41:12<7:26:38, 15.69s/it]
 83%|████████▎ | 8157/9864 [17:41:26<7:11:36, 15.17s/it]
 83%|████████▎ | 8158/9864 [17:41:40<6:59:26, 14.75s/it]
 83%|████████▎ | 8159/9864 [17:41:54<6:53:44, 14.56s/it]
 83%|████████▎ | 8160/9864 [17:42:10<7:05:53, 15.00s/it]
                                                        
{'loss': '0.06624', 'grad_norm': '5.774', 'learning_rate': '1.527e-06', 'epoch': '0.8273'}

 83%|████████▎ | 8160/9864 [17:42:10<7:05:53, 15.00s/it]
 83%|████████▎ | 8161/9864 [17:42:23<6:50:45, 14.47s/it]
 83%|████████▎ | 8162/9864 [17:42:39<6:59:54, 14.80s/it]
 83%|████████▎ | 8163/9864 [17:42:55<7:09:42, 15.16s/it]
 83%|████████▎ | 8164/9864 [17:43:10<7:08:24, 15.12s/it]
 83%|████████▎ | 8165/9864 [17:43:24<6:57:08, 14.73s/it]
 83%|████████▎ | 8166/9864 [17:43:38<6:55:22, 14.68s/it]
 83%|████████▎ | 8167/9864 [17:43:53<6:51:19, 14.54s/it]
 83%|████████▎ | 8168/9864 [17:44:08<7:00:25, 14.87s/it]
 83%|████████▎ | 8169/9864 [17:44:23<6:55:10, 14.70s/it]
 83%|████████▎ | 8170/9864 [17:44:41<7:22:42, 15.68s/it]
                                                        
{'loss': '0.06918', 'grad_norm': '5.249', 'learning_rate': '1.509e-06', 'epoch': '0.8283'}

 83%|████████▎ | 8170/9864 [17:44:41<7:22:42, 15.68s/it]
 83%|████████▎ | 8171/9864 [17:44:54<7:06:40, 15.12s/it]
 83%|████████▎ | 8172/9864 [17:45:10<7:12:55, 15.35s/it]
 83%|████████▎ | 8173/9864 [17:45:25<7:04:46, 15.07s/it]
 83%|████████▎ | 8174/9864 [17:45:39<6:54:57, 14.73s/it]
 83%|████████▎ | 8175/9864 [17:45:53<6:52:06, 14.64s/it]
 83%|████████▎ | 8176/9864 [17:46:09<7:00:52, 14.96s/it]
 83%|████████▎ | 8177/9864 [17:46:23<6:54:18, 14.74s/it]
 83%|████████▎ | 8178/9864 [17:46:40<7:16:13, 15.52s/it]
 83%|████████▎ | 8179/9864 [17:46:54<7:03:38, 15.09s/it]
 83%|████████▎ | 8180/9864 [17:47:10<7:10:10, 15.33s/it]
                                                        
{'loss': '0.06062', 'grad_norm': '5.534', 'learning_rate': '1.492e-06', 'epoch': '0.8293'}

 83%|████████▎ | 8180/9864 [17:47:10<7:10:10, 15.33s/it]
 83%|████████▎ | 8181/9864 [17:47:24<6:58:52, 14.93s/it]
 83%|████████▎ | 8182/9864 [17:47:39<6:52:53, 14.73s/it]
 83%|████████▎ | 8183/9864 [17:47:52<6:44:32, 14.44s/it]
 83%|████████▎ | 8184/9864 [17:48:07<6:49:53, 14.64s/it]
 83%|████████▎ | 8185/9864 [17:48:23<6:58:05, 14.94s/it]
 83%|████████▎ | 8186/9864 [17:48:39<7:05:12, 15.20s/it]
 83%|████████▎ | 8187/9864 [17:48:56<7:16:52, 15.63s/it]
 83%|████████▎ | 8188/9864 [17:49:10<7:08:07, 15.33s/it]
 83%|████████▎ | 8189/9864 [17:49:24<6:57:33, 14.96s/it]
 83%|████████▎ | 8190/9864 [17:49:39<6:51:28, 14.75s/it]
                                                        
{'loss': '0.06845', 'grad_norm': '4.736', 'learning_rate': '1.475e-06', 'epoch': '0.8304'}

 83%|████████▎ | 8190/9864 [17:49:39<6:51:28, 14.75s/it]
 83%|████████▎ | 8191/9864 [17:49:53<6:47:57, 14.63s/it]
 83%|████████▎ | 8192/9864 [17:50:09<7:00:20, 15.08s/it]
 83%|████████▎ | 8193/9864 [17:50:27<7:21:03, 15.84s/it]
 83%|████████▎ | 8194/9864 [17:50:43<7:23:20, 15.93s/it]
 83%|████████▎ | 8195/9864 [17:50:57<7:06:09, 15.32s/it]
 83%|████████▎ | 8196/9864 [17:51:11<6:55:18, 14.94s/it]
 83%|████████▎ | 8197/9864 [17:51:25<6:46:57, 14.65s/it]
 83%|████████▎ | 8198/9864 [17:51:38<6:38:43, 14.36s/it]
 83%|████████▎ | 8199/9864 [17:51:52<6:35:59, 14.27s/it]
 83%|████████▎ | 8200/9864 [17:52:08<6:45:26, 14.62s/it]
                                                        
{'loss': '0.06965', 'grad_norm': '5.34', 'learning_rate': '1.458e-06', 'epoch': '0.8314'}

 83%|████████▎ | 8200/9864 [17:52:08<6:45:26, 14.62s/it]
 83%|████████▎ | 8201/9864 [17:52:24<7:00:09, 15.16s/it]
 83%|████████▎ | 8202/9864 [17:52:42<7:19:31, 15.87s/it]
 83%|████████▎ | 8203/9864 [17:52:56<7:04:30, 15.33s/it]
 83%|████████▎ | 8204/9864 [17:53:10<6:52:28, 14.91s/it]
 83%|████████▎ | 8205/9864 [17:53:24<6:43:43, 14.60s/it]
 83%|████████▎ | 8206/9864 [17:53:38<6:39:45, 14.47s/it]
 83%|████████▎ | 8207/9864 [17:53:53<6:48:33, 14.79s/it]
 83%|████████▎ | 8208/9864 [17:54:09<6:54:17, 15.01s/it]
 83%|████████▎ | 8209/9864 [17:54:25<6:59:49, 15.22s/it]
 83%|████████▎ | 8210/9864 [17:54:38<6:47:24, 14.78s/it]
                                                        
{'loss': '0.06783', 'grad_norm': '6.106', 'learning_rate': '1.44e-06', 'epoch': '0.8324'}

 83%|████████▎ | 8210/9864 [17:54:38<6:47:24, 14.78s/it]
 83%|████████▎ | 8211/9864 [17:54:55<6:58:48, 15.20s/it]
 83%|████████▎ | 8212/9864 [17:55:09<6:49:49, 14.88s/it]
 83%|████████▎ | 8213/9864 [17:55:23<6:41:31, 14.59s/it]
 83%|████████▎ | 8214/9864 [17:55:37<6:36:53, 14.43s/it]
 83%|████████▎ | 8215/9864 [17:55:53<6:48:53, 14.88s/it]
 83%|████████▎ | 8216/9864 [17:56:07<6:41:44, 14.63s/it]
 83%|████████▎ | 8217/9864 [17:56:23<6:52:49, 15.04s/it]
 83%|████████▎ | 8218/9864 [17:56:37<6:43:28, 14.71s/it]
 83%|████████▎ | 8219/9864 [17:56:52<6:51:30, 15.01s/it]
 83%|████████▎ | 8220/9864 [17:57:06<6:44:07, 14.75s/it]
                                                        
{'loss': '0.06444', 'grad_norm': '5.916', 'learning_rate': '1.424e-06', 'epoch': '0.8334'}

 83%|████████▎ | 8220/9864 [17:57:06<6:44:07, 14.75s/it]
 83%|████████▎ | 8221/9864 [17:57:21<6:41:32, 14.66s/it]
 83%|████████▎ | 8222/9864 [17:57:38<7:03:23, 15.47s/it]
 83%|████████▎ | 8223/9864 [17:57:52<6:50:20, 15.00s/it]
 83%|████████▎ | 8224/9864 [17:58:06<6:40:36, 14.66s/it]
 83%|████████▎ | 8225/9864 [17:58:22<6:48:33, 14.96s/it]
 83%|████████▎ | 8226/9864 [17:58:38<6:59:52, 15.38s/it]
 83%|████████▎ | 8227/9864 [17:58:52<6:47:27, 14.93s/it]
 83%|████████▎ | 8228/9864 [17:59:06<6:38:03, 14.60s/it]
 83%|████████▎ | 8229/9864 [17:59:20<6:36:46, 14.56s/it]
 83%|████████▎ | 8230/9864 [17:59:36<6:44:56, 14.87s/it]
                                                        
{'loss': '0.05679', 'grad_norm': '4.796', 'learning_rate': '1.407e-06', 'epoch': '0.8344'}

 83%|████████▎ | 8230/9864 [17:59:36<6:44:56, 14.87s/it]
 83%|████████▎ | 8231/9864 [17:59:51<6:49:34, 15.05s/it]
 83%|████████▎ | 8232/9864 [18:00:05<6:40:08, 14.71s/it]
 83%|████████▎ | 8233/9864 [18:00:20<6:44:20, 14.87s/it]
 83%|████████▎ | 8234/9864 [18:00:36<6:52:17, 15.18s/it]
 83%|████████▎ | 8235/9864 [18:00:51<6:44:06, 14.88s/it]
 83%|████████▎ | 8236/9864 [18:01:05<6:37:32, 14.65s/it]
 84%|████████▎ | 8237/9864 [18:01:19<6:33:20, 14.51s/it]
 84%|████████▎ | 8238/9864 [18:01:32<6:25:24, 14.22s/it]
 84%|████████▎ | 8239/9864 [18:01:48<6:39:14, 14.74s/it]
 84%|████████▎ | 8240/9864 [18:02:04<6:46:30, 15.02s/it]
                                                        
{'loss': '0.07079', 'grad_norm': '4.127', 'learning_rate': '1.39e-06', 'epoch': '0.8354'}

 84%|████████▎ | 8240/9864 [18:02:04<6:46:30, 15.02s/it]
 84%|████████▎ | 8241/9864 [18:02:20<6:56:50, 15.41s/it]
 84%|████████▎ | 8242/9864 [18:02:37<7:03:02, 15.65s/it]
 84%|████████▎ | 8243/9864 [18:02:50<6:49:06, 15.14s/it]
 84%|████████▎ | 8244/9864 [18:03:05<6:41:04, 14.85s/it]
 84%|████████▎ | 8245/9864 [18:03:19<6:38:15, 14.76s/it]
 84%|████████▎ | 8246/9864 [18:03:35<6:46:10, 15.06s/it]
 84%|████████▎ | 8247/9864 [18:03:50<6:44:38, 15.01s/it]
 84%|████████▎ | 8248/9864 [18:04:04<6:38:55, 14.81s/it]
 84%|████████▎ | 8249/9864 [18:04:18<6:34:32, 14.66s/it]
 84%|████████▎ | 8250/9864 [18:04:35<6:51:20, 15.29s/it]
                                                        
{'loss': '0.07084', 'grad_norm': '6.089', 'learning_rate': '1.373e-06', 'epoch': '0.8364'}

 84%|████████▎ | 8250/9864 [18:04:35<6:51:20, 15.29s/it]
 84%|████████▎ | 8251/9864 [18:04:49<6:40:53, 14.91s/it]
 84%|████████▎ | 8252/9864 [18:05:03<6:31:46, 14.58s/it]
 84%|████████▎ | 8253/9864 [18:05:17<6:29:28, 14.51s/it]
 84%|████████▎ | 8254/9864 [18:05:32<6:30:36, 14.56s/it]
 84%|████████▎ | 8255/9864 [18:05:48<6:41:28, 14.97s/it]
 84%|████████▎ | 8256/9864 [18:06:03<6:45:02, 15.11s/it]
 84%|████████▎ | 8257/9864 [18:06:17<6:35:00, 14.75s/it]
 84%|████████▎ | 8258/9864 [18:06:35<6:56:09, 15.55s/it]
 84%|████████▎ | 8259/9864 [18:06:49<6:42:54, 15.06s/it]
 84%|████████▎ | 8260/9864 [18:07:03<6:36:24, 14.83s/it]
                                                        
{'loss': '0.06628', 'grad_norm': '5.172', 'learning_rate': '1.357e-06', 'epoch': '0.8375'}

 84%|████████▎ | 8260/9864 [18:07:03<6:36:24, 14.83s/it]
 84%|████████▎ | 8261/9864 [18:07:17<6:28:41, 14.55s/it]
 84%|████████▍ | 8262/9864 [18:07:31<6:26:30, 14.48s/it]
 84%|████████▍ | 8263/9864 [18:07:49<6:52:16, 15.45s/it]
 84%|████████▍ | 8264/9864 [18:08:03<6:40:06, 15.00s/it]
 84%|████████▍ | 8265/9864 [18:08:19<6:46:55, 15.27s/it]
 84%|████████▍ | 8266/9864 [18:08:34<6:49:49, 15.39s/it]
 84%|████████▍ | 8267/9864 [18:08:48<6:37:23, 14.93s/it]
 84%|████████▍ | 8268/9864 [18:09:02<6:29:50, 14.66s/it]
 84%|████████▍ | 8269/9864 [18:09:17<6:27:51, 14.59s/it]
 84%|████████▍ | 8270/9864 [18:09:33<6:40:06, 15.06s/it]
                                                        
{'loss': '0.06669', 'grad_norm': '6.134', 'learning_rate': '1.34e-06', 'epoch': '0.8385'}

 84%|████████▍ | 8270/9864 [18:09:33<6:40:06, 15.06s/it]
 84%|████████▍ | 8271/9864 [18:09:49<6:44:38, 15.24s/it]
 84%|████████▍ | 8272/9864 [18:10:02<6:32:46, 14.80s/it]
 84%|████████▍ | 8273/9864 [18:10:17<6:28:38, 14.66s/it]
 84%|████████▍ | 8274/9864 [18:10:32<6:36:09, 14.95s/it]
 84%|████████▍ | 8275/9864 [18:10:46<6:26:47, 14.61s/it]
 84%|████████▍ | 8276/9864 [18:11:00<6:24:22, 14.52s/it]
 84%|████████▍ | 8277/9864 [18:11:16<6:34:45, 14.92s/it]
 84%|████████▍ | 8278/9864 [18:11:33<6:45:11, 15.33s/it]
 84%|████████▍ | 8279/9864 [18:11:47<6:36:00, 14.99s/it]
 84%|████████▍ | 8280/9864 [18:12:01<6:32:13, 14.86s/it]
                                                        
{'loss': '0.07224', 'grad_norm': '5.314', 'learning_rate': '1.324e-06', 'epoch': '0.8395'}

 84%|████████▍ | 8280/9864 [18:12:01<6:32:13, 14.86s/it]
 84%|████████▍ | 8281/9864 [18:12:17<6:40:55, 15.20s/it]
 84%|████████▍ | 8282/9864 [18:12:33<6:44:55, 15.36s/it]
 84%|████████▍ | 8283/9864 [18:12:47<6:30:23, 14.82s/it]
 84%|████████▍ | 8284/9864 [18:13:02<6:37:13, 15.08s/it]
 84%|████████▍ | 8285/9864 [18:13:18<6:39:28, 15.18s/it]
 84%|████████▍ | 8286/9864 [18:13:32<6:29:16, 14.80s/it]
 84%|████████▍ | 8287/9864 [18:13:46<6:22:25, 14.55s/it]
 84%|████████▍ | 8288/9864 [18:14:00<6:20:15, 14.48s/it]
 84%|████████▍ | 8289/9864 [18:14:16<6:31:48, 14.93s/it]
 84%|████████▍ | 8290/9864 [18:14:31<6:35:04, 15.06s/it]
                                                        
{'loss': '0.05999', 'grad_norm': '4.083', 'learning_rate': '1.308e-06', 'epoch': '0.8405'}

 84%|████████▍ | 8290/9864 [18:14:31<6:35:04, 15.06s/it]
 84%|████████▍ | 8291/9864 [18:14:45<6:24:41, 14.67s/it]
 84%|████████▍ | 8292/9864 [18:14:59<6:22:07, 14.59s/it]
 84%|████████▍ | 8293/9864 [18:15:17<6:48:17, 15.59s/it]
 84%|████████▍ | 8294/9864 [18:15:31<6:34:29, 15.08s/it]
 84%|████████▍ | 8295/9864 [18:15:45<6:24:48, 14.72s/it]
 84%|████████▍ | 8296/9864 [18:15:59<6:19:57, 14.54s/it]
 84%|████████▍ | 8297/9864 [18:16:16<6:40:25, 15.33s/it]
 84%|████████▍ | 8298/9864 [18:16:30<6:29:53, 14.94s/it]
 84%|████████▍ | 8299/9864 [18:16:45<6:26:04, 14.80s/it]
 84%|████████▍ | 8300/9864 [18:17:00<6:30:45, 14.99s/it]
                                                        
{'loss': '0.06053', 'grad_norm': '4.552', 'learning_rate': '1.291e-06', 'epoch': '0.8415'}

 84%|████████▍ | 8300/9864 [18:17:00<6:30:45, 14.99s/it]
 84%|████████▍ | 8301/9864 [18:17:16<6:38:46, 15.31s/it]
 84%|████████▍ | 8302/9864 [18:17:31<6:30:22, 15.00s/it]
 84%|████████▍ | 8303/9864 [18:17:45<6:21:58, 14.68s/it]
 84%|████████▍ | 8304/9864 [18:17:58<6:14:28, 14.40s/it]
 84%|████████▍ | 8305/9864 [18:18:16<6:42:39, 15.50s/it]
 84%|████████▍ | 8306/9864 [18:18:30<6:30:21, 15.03s/it]
 84%|████████▍ | 8307/9864 [18:18:44<6:19:19, 14.62s/it]
 84%|████████▍ | 8308/9864 [18:19:02<6:42:07, 15.51s/it]
 84%|████████▍ | 8309/9864 [18:19:16<6:32:14, 15.13s/it]
 84%|████████▍ | 8310/9864 [18:19:30<6:25:04, 14.87s/it]
                                                        
{'loss': '0.05869', 'grad_norm': '5.004', 'learning_rate': '1.275e-06', 'epoch': '0.8425'}

 84%|████████▍ | 8310/9864 [18:19:30<6:25:04, 14.87s/it]
 84%|████████▍ | 8311/9864 [18:19:44<6:20:22, 14.70s/it]
 84%|████████▍ | 8312/9864 [18:19:58<6:15:16, 14.51s/it]
 84%|████████▍ | 8313/9864 [18:20:16<6:39:01, 15.44s/it]
 84%|████████▍ | 8314/9864 [18:20:30<6:28:35, 15.04s/it]
 84%|████████▍ | 8315/9864 [18:20:44<6:21:43, 14.79s/it]
 84%|████████▍ | 8316/9864 [18:21:00<6:24:20, 14.90s/it]
 84%|████████▍ | 8317/9864 [18:21:15<6:29:49, 15.12s/it]
 84%|████████▍ | 8318/9864 [18:21:29<6:23:01, 14.87s/it]
 84%|████████▍ | 8319/9864 [18:21:44<6:17:14, 14.65s/it]
 84%|████████▍ | 8320/9864 [18:22:00<6:32:37, 15.26s/it]
                                                        
{'loss': '0.06305', 'grad_norm': '5.631', 'learning_rate': '1.259e-06', 'epoch': '0.8435'}

 84%|████████▍ | 8320/9864 [18:22:00<6:32:37, 15.26s/it]
 84%|████████▍ | 8321/9864 [18:22:16<6:35:58, 15.40s/it]
 84%|████████▍ | 8322/9864 [18:22:30<6:25:30, 15.00s/it]
 84%|████████▍ | 8323/9864 [18:22:44<6:20:25, 14.81s/it]
 84%|████████▍ | 8324/9864 [18:23:00<6:24:01, 14.96s/it]
 84%|████████▍ | 8325/9864 [18:23:15<6:29:01, 15.17s/it]
 84%|████████▍ | 8326/9864 [18:23:29<6:19:39, 14.81s/it]
 84%|████████▍ | 8327/9864 [18:23:44<6:14:30, 14.62s/it]
 84%|████████▍ | 8328/9864 [18:24:01<6:37:35, 15.53s/it]
 84%|████████▍ | 8329/9864 [18:24:15<6:27:12, 15.14s/it]
 84%|████████▍ | 8330/9864 [18:24:29<6:15:01, 14.67s/it]
                                                        
{'loss': '0.06001', 'grad_norm': '5.371', 'learning_rate': '1.243e-06', 'epoch': '0.8446'}

 84%|████████▍ | 8330/9864 [18:24:29<6:15:01, 14.67s/it]
 84%|████████▍ | 8331/9864 [18:24:43<6:11:34, 14.54s/it]
 84%|████████▍ | 8332/9864 [18:25:01<6:34:31, 15.45s/it]
 84%|████████▍ | 8333/9864 [18:25:15<6:26:21, 15.14s/it]
 84%|████████▍ | 8334/9864 [18:25:29<6:15:07, 14.71s/it]
 84%|████████▍ | 8335/9864 [18:25:43<6:11:43, 14.59s/it]
 85%|████████▍ | 8336/9864 [18:25:58<6:10:26, 14.55s/it]
 85%|████████▍ | 8337/9864 [18:26:15<6:30:36, 15.35s/it]
 85%|████████▍ | 8338/9864 [18:26:29<6:19:24, 14.92s/it]
 85%|████████▍ | 8339/9864 [18:26:45<6:31:07, 15.39s/it]
 85%|████████▍ | 8340/9864 [18:27:01<6:35:26, 15.57s/it]
                                                        
{'loss': '0.05882', 'grad_norm': '4.232', 'learning_rate': '1.228e-06', 'epoch': '0.8456'}

 85%|████████▍ | 8340/9864 [18:27:01<6:35:26, 15.57s/it]
 85%|████████▍ | 8341/9864 [18:27:15<6:23:46, 15.12s/it]
 85%|████████▍ | 8342/9864 [18:27:29<6:15:25, 14.80s/it]
 85%|████████▍ | 8343/9864 [18:27:44<6:11:02, 14.64s/it]
 85%|████████▍ | 8344/9864 [18:28:01<6:33:25, 15.53s/it]
 85%|████████▍ | 8345/9864 [18:28:15<6:21:34, 15.07s/it]
 85%|████████▍ | 8346/9864 [18:28:32<6:30:42, 15.44s/it]
 85%|████████▍ | 8347/9864 [18:28:46<6:21:16, 15.08s/it]
 85%|████████▍ | 8348/9864 [18:29:02<6:26:14, 15.29s/it]
 85%|████████▍ | 8349/9864 [18:29:16<6:17:04, 14.93s/it]
 85%|████████▍ | 8350/9864 [18:29:30<6:11:04, 14.71s/it]
                                                        
{'loss': '0.05754', 'grad_norm': '4.872', 'learning_rate': '1.212e-06', 'epoch': '0.8466'}

 85%|████████▍ | 8350/9864 [18:29:30<6:11:04, 14.71s/it]
 85%|████████▍ | 8351/9864 [18:29:44<6:08:38, 14.62s/it]
 85%|████████▍ | 8352/9864 [18:29:59<6:09:31, 14.66s/it]
 85%|████████▍ | 8353/9864 [18:30:13<6:05:53, 14.53s/it]
 85%|████████▍ | 8354/9864 [18:30:29<6:18:04, 15.02s/it]
 85%|████████▍ | 8355/9864 [18:30:43<6:09:56, 14.71s/it]
 85%|████████▍ | 8356/9864 [18:30:59<6:16:42, 14.99s/it]
 85%|████████▍ | 8357/9864 [18:31:13<6:11:09, 14.78s/it]
 85%|████████▍ | 8358/9864 [18:31:28<6:08:06, 14.67s/it]
 85%|████████▍ | 8359/9864 [18:31:42<6:02:08, 14.44s/it]
 85%|████████▍ | 8360/9864 [18:31:57<6:09:17, 14.73s/it]
                                                        
{'loss': '0.06099', 'grad_norm': '4.837', 'learning_rate': '1.196e-06', 'epoch': '0.8476'}

 85%|████████▍ | 8360/9864 [18:31:57<6:09:17, 14.73s/it]
 85%|████████▍ | 8361/9864 [18:32:15<6:32:26, 15.67s/it]
 85%|████████▍ | 8362/9864 [18:32:29<6:20:41, 15.21s/it]
 85%|████████▍ | 8363/9864 [18:32:44<6:19:12, 15.16s/it]
 85%|████████▍ | 8364/9864 [18:32:58<6:12:07, 14.89s/it]
 85%|████████▍ | 8365/9864 [18:33:13<6:06:44, 14.68s/it]
 85%|████████▍ | 8366/9864 [18:33:26<5:59:29, 14.40s/it]
 85%|████████▍ | 8367/9864 [18:33:40<5:55:33, 14.25s/it]
 85%|████████▍ | 8368/9864 [18:33:55<6:02:54, 14.55s/it]
 85%|████████▍ | 8369/9864 [18:34:14<6:29:11, 15.62s/it]
 85%|████████▍ | 8370/9864 [18:34:28<6:19:39, 15.25s/it]
                                                        
{'loss': '0.06338', 'grad_norm': '5.562', 'learning_rate': '1.181e-06', 'epoch': '0.8486'}

 85%|████████▍ | 8370/9864 [18:34:28<6:19:39, 15.25s/it]
 85%|████████▍ | 8371/9864 [18:34:42<6:08:36, 14.81s/it]
 85%|████████▍ | 8372/9864 [18:34:58<6:15:29, 15.10s/it]
 85%|████████▍ | 8373/9864 [18:35:12<6:11:40, 14.96s/it]
 85%|████████▍ | 8374/9864 [18:35:26<6:05:10, 14.70s/it]
 85%|████████▍ | 8375/9864 [18:35:40<6:01:18, 14.56s/it]
 85%|████████▍ | 8376/9864 [18:35:57<6:12:14, 15.01s/it]
 85%|████████▍ | 8377/9864 [18:36:14<6:28:06, 15.66s/it]
 85%|████████▍ | 8378/9864 [18:36:28<6:17:14, 15.23s/it]
 85%|████████▍ | 8379/9864 [18:36:42<6:09:23, 14.92s/it]
 85%|████████▍ | 8380/9864 [18:36:58<6:16:35, 15.23s/it]
                                                        
{'loss': '0.0644', 'grad_norm': '5.493', 'learning_rate': '1.165e-06', 'epoch': '0.8496'}

 85%|████████▍ | 8380/9864 [18:36:58<6:16:35, 15.23s/it]
 85%|████████▍ | 8381/9864 [18:37:12<6:10:03, 14.97s/it]
 85%|████████▍ | 8382/9864 [18:37:27<6:04:48, 14.77s/it]
 85%|████████▍ | 8383/9864 [18:37:41<5:57:17, 14.48s/it]
 85%|████████▍ | 8384/9864 [18:37:58<6:22:12, 15.49s/it]
 85%|████████▌ | 8385/9864 [18:38:14<6:25:32, 15.64s/it]
 85%|████████▌ | 8386/9864 [18:38:29<6:16:35, 15.29s/it]
 85%|████████▌ | 8387/9864 [18:38:45<6:20:03, 15.44s/it]
 85%|████████▌ | 8388/9864 [18:38:59<6:13:48, 15.20s/it]
 85%|████████▌ | 8389/9864 [18:39:13<6:05:19, 14.86s/it]
 85%|████████▌ | 8390/9864 [18:39:27<5:57:15, 14.54s/it]
                                                        
{'loss': '0.06822', 'grad_norm': '4.427', 'learning_rate': '1.15e-06', 'epoch': '0.8506'}

 85%|████████▌ | 8390/9864 [18:39:27<5:57:15, 14.54s/it]
 85%|████████▌ | 8391/9864 [18:39:44<6:17:22, 15.37s/it]
 85%|████████▌ | 8392/9864 [18:40:00<6:16:20, 15.34s/it]
 85%|████████▌ | 8393/9864 [18:40:14<6:08:06, 15.01s/it]
 85%|████████▌ | 8394/9864 [18:40:29<6:10:25, 15.12s/it]
 85%|████████▌ | 8395/9864 [18:40:43<6:00:29, 14.72s/it]
 85%|████████▌ | 8396/9864 [18:40:57<5:56:50, 14.59s/it]
 85%|████████▌ | 8397/9864 [18:41:11<5:51:10, 14.36s/it]
 85%|████████▌ | 8398/9864 [18:41:25<5:47:43, 14.23s/it]
 85%|████████▌ | 8399/9864 [18:41:41<5:58:55, 14.70s/it]
 85%|████████▌ | 8400/9864 [18:41:59<6:19:58, 15.57s/it]
                                                        
{'loss': '0.07203', 'grad_norm': '5.109', 'learning_rate': '1.135e-06', 'epoch': '0.8517'}

 85%|████████▌ | 8400/9864 [18:41:59<6:19:58, 15.57s/it]
 85%|████████▌ | 8401/9864 [18:42:13<6:09:30, 15.15s/it]
 85%|████████▌ | 8402/9864 [18:42:27<6:00:47, 14.81s/it]
 85%|████████▌ | 8403/9864 [18:42:42<6:04:21, 14.96s/it]
 85%|████████▌ | 8404/9864 [18:42:57<6:00:14, 14.80s/it]
 85%|████████▌ | 8405/9864 [18:43:10<5:52:26, 14.49s/it]
 85%|████████▌ | 8406/9864 [18:43:26<5:59:57, 14.81s/it]
 85%|████████▌ | 8407/9864 [18:43:40<5:56:40, 14.69s/it]
 85%|████████▌ | 8408/9864 [18:43:58<6:17:12, 15.54s/it]
 85%|████████▌ | 8409/9864 [18:44:12<6:05:30, 15.07s/it]
 85%|████████▌ | 8410/9864 [18:44:28<6:12:10, 15.36s/it]
                                                        
{'loss': '0.06249', 'grad_norm': '5.043', 'learning_rate': '1.12e-06', 'epoch': '0.8527'}

 85%|████████▌ | 8410/9864 [18:44:28<6:12:10, 15.36s/it]
 85%|████████▌ | 8411/9864 [18:44:42<6:04:20, 15.05s/it]
 85%|████████▌ | 8412/9864 [18:44:56<5:56:58, 14.75s/it]
 85%|████████▌ | 8413/9864 [18:45:10<5:51:19, 14.53s/it]
 85%|████████▌ | 8414/9864 [18:45:26<6:01:09, 14.94s/it]
 85%|████████▌ | 8415/9864 [18:45:40<5:52:47, 14.61s/it]
 85%|████████▌ | 8416/9864 [18:45:56<6:01:13, 14.97s/it]
 85%|████████▌ | 8417/9864 [18:46:12<6:09:35, 15.33s/it]
 85%|████████▌ | 8418/9864 [18:46:28<6:13:18, 15.49s/it]
 85%|████████▌ | 8419/9864 [18:46:42<6:04:57, 15.15s/it]
 85%|████████▌ | 8420/9864 [18:46:56<5:57:29, 14.85s/it]
                                                        
{'loss': '0.06925', 'grad_norm': '4.293', 'learning_rate': '1.105e-06', 'epoch': '0.8537'}

 85%|████████▌ | 8420/9864 [18:46:56<5:57:29, 14.85s/it]
 85%|████████▌ | 8421/9864 [18:47:11<5:53:15, 14.69s/it]
 85%|████████▌ | 8422/9864 [18:47:27<6:02:13, 15.07s/it]
 85%|████████▌ | 8423/9864 [18:47:41<5:56:03, 14.83s/it]
 85%|████████▌ | 8424/9864 [18:47:56<6:00:32, 15.02s/it]
 85%|████████▌ | 8425/9864 [18:48:12<6:04:40, 15.21s/it]
 85%|████████▌ | 8426/9864 [18:48:26<5:58:10, 14.94s/it]
 85%|████████▌ | 8427/9864 [18:48:42<6:02:54, 15.15s/it]
 85%|████████▌ | 8428/9864 [18:48:56<5:54:07, 14.80s/it]
 85%|████████▌ | 8429/9864 [18:49:10<5:49:58, 14.63s/it]
 85%|████████▌ | 8430/9864 [18:49:27<6:03:40, 15.22s/it]
                                                        
{'loss': '0.06687', 'grad_norm': '4.787', 'learning_rate': '1.09e-06', 'epoch': '0.8547'}

 85%|████████▌ | 8430/9864 [18:49:27<6:03:40, 15.22s/it]
 85%|████████▌ | 8431/9864 [18:49:41<5:56:40, 14.93s/it]
 85%|████████▌ | 8432/9864 [18:49:57<6:04:52, 15.29s/it]
 85%|████████▌ | 8433/9864 [18:50:13<6:08:56, 15.47s/it]
 86%|████████▌ | 8434/9864 [18:50:27<5:56:34, 14.96s/it]
 86%|████████▌ | 8435/9864 [18:50:43<6:03:22, 15.26s/it]
 86%|████████▌ | 8436/9864 [18:50:57<5:52:57, 14.83s/it]
 86%|████████▌ | 8437/9864 [18:51:11<5:49:33, 14.70s/it]
 86%|████████▌ | 8438/9864 [18:51:26<5:54:17, 14.91s/it]
 86%|████████▌ | 8439/9864 [18:51:41<5:50:08, 14.74s/it]
 86%|████████▌ | 8440/9864 [18:51:56<5:53:49, 14.91s/it]
                                                        
{'loss': '0.0541', 'grad_norm': '4.433', 'learning_rate': '1.075e-06', 'epoch': '0.8557'}

 86%|████████▌ | 8440/9864 [18:51:56<5:53:49, 14.91s/it]
 86%|████████▌ | 8441/9864 [18:52:12<5:59:31, 15.16s/it]
 86%|████████▌ | 8442/9864 [18:52:28<6:03:56, 15.36s/it]
 86%|████████▌ | 8443/9864 [18:52:41<5:51:09, 14.83s/it]
 86%|████████▌ | 8444/9864 [18:52:56<5:48:51, 14.74s/it]
 86%|████████▌ | 8445/9864 [18:53:09<5:39:22, 14.35s/it]
 86%|████████▌ | 8446/9864 [18:53:25<5:48:18, 14.74s/it]
 86%|████████▌ | 8447/9864 [18:53:41<5:57:07, 15.12s/it]
 86%|████████▌ | 8448/9864 [18:53:54<5:46:59, 14.70s/it]
 86%|████████▌ | 8449/9864 [18:54:10<5:52:43, 14.96s/it]
 86%|████████▌ | 8450/9864 [18:54:24<5:48:48, 14.80s/it]
                                                        
{'loss': '0.05871', 'grad_norm': '5.595', 'learning_rate': '1.06e-06', 'epoch': '0.8567'}

 86%|████████▌ | 8450/9864 [18:54:24<5:48:48, 14.80s/it]
 86%|████████▌ | 8451/9864 [18:54:40<5:54:15, 15.04s/it]
 86%|████████▌ | 8452/9864 [18:54:54<5:44:05, 14.62s/it]
 86%|████████▌ | 8453/9864 [18:55:08<5:42:31, 14.57s/it]
 86%|████████▌ | 8454/9864 [18:55:22<5:38:56, 14.42s/it]
 86%|████████▌ | 8455/9864 [18:55:40<5:58:52, 15.28s/it]
 86%|████████▌ | 8456/9864 [18:55:55<5:59:36, 15.32s/it]
 86%|████████▌ | 8457/9864 [18:56:09<5:52:56, 15.05s/it]
 86%|████████▌ | 8458/9864 [18:56:25<5:56:06, 15.20s/it]
 86%|████████▌ | 8459/9864 [18:56:39<5:46:59, 14.82s/it]
 86%|████████▌ | 8460/9864 [18:56:53<5:40:21, 14.54s/it]
                                                        
{'loss': '0.06812', 'grad_norm': '5.374', 'learning_rate': '1.045e-06', 'epoch': '0.8577'}

 86%|████████▌ | 8460/9864 [18:56:53<5:40:21, 14.54s/it]
 86%|████████▌ | 8461/9864 [18:57:07<5:38:40, 14.48s/it]
 86%|████████▌ | 8462/9864 [18:57:22<5:44:38, 14.75s/it]
 86%|████████▌ | 8463/9864 [18:57:38<5:52:46, 15.11s/it]
 86%|████████▌ | 8464/9864 [18:57:53<5:46:11, 14.84s/it]
 86%|████████▌ | 8465/9864 [18:58:10<6:03:06, 15.57s/it]
 86%|████████▌ | 8466/9864 [18:58:25<5:57:41, 15.35s/it]
 86%|████████▌ | 8467/9864 [18:58:39<5:50:43, 15.06s/it]
 86%|████████▌ | 8468/9864 [18:58:54<5:46:48, 14.91s/it]
 86%|████████▌ | 8469/9864 [18:59:09<5:51:09, 15.10s/it]
 86%|████████▌ | 8470/9864 [18:59:23<5:42:43, 14.75s/it]
                                                        
{'loss': '0.05624', 'grad_norm': '5.383', 'learning_rate': '1.031e-06', 'epoch': '0.8587'}

 86%|████████▌ | 8470/9864 [18:59:23<5:42:43, 14.75s/it]
 86%|████████▌ | 8471/9864 [18:59:38<5:44:45, 14.85s/it]
 86%|████████▌ | 8472/9864 [18:59:54<5:49:18, 15.06s/it]
 86%|████████▌ | 8473/9864 [19:00:09<5:52:19, 15.20s/it]
 86%|████████▌ | 8474/9864 [19:00:23<5:44:12, 14.86s/it]
 86%|████████▌ | 8475/9864 [19:00:37<5:37:42, 14.59s/it]
 86%|████████▌ | 8476/9864 [19:00:52<5:35:34, 14.51s/it]
 86%|████████▌ | 8477/9864 [19:01:05<5:30:23, 14.29s/it]
 86%|████████▌ | 8478/9864 [19:01:21<5:39:07, 14.68s/it]
 86%|████████▌ | 8479/9864 [19:01:37<5:44:55, 14.94s/it]
 86%|████████▌ | 8480/9864 [19:01:53<5:51:56, 15.26s/it]
                                                        
{'loss': '0.05917', 'grad_norm': '5.301', 'learning_rate': '1.016e-06', 'epoch': '0.8598'}

 86%|████████▌ | 8480/9864 [19:01:53<5:51:56, 15.26s/it]
 86%|████████▌ | 8481/9864 [19:02:09<5:57:11, 15.50s/it]
 86%|████████▌ | 8482/9864 [19:02:23<5:46:40, 15.05s/it]
 86%|████████▌ | 8483/9864 [19:02:38<5:45:19, 15.00s/it]
 86%|████████▌ | 8484/9864 [19:02:51<5:37:40, 14.68s/it]
 86%|████████▌ | 8485/9864 [19:03:06<5:33:03, 14.49s/it]
 86%|████████▌ | 8486/9864 [19:03:21<5:42:10, 14.90s/it]
 86%|████████▌ | 8487/9864 [19:03:37<5:48:25, 15.18s/it]
 86%|████████▌ | 8488/9864 [19:03:54<6:00:56, 15.74s/it]
 86%|████████▌ | 8489/9864 [19:04:08<5:47:43, 15.17s/it]
 86%|████████▌ | 8490/9864 [19:04:22<5:38:33, 14.78s/it]
                                                        
{'loss': '0.05534', 'grad_norm': '4.748', 'learning_rate': '1.002e-06', 'epoch': '0.8608'}

 86%|████████▌ | 8490/9864 [19:04:22<5:38:33, 14.78s/it]
 86%|████████▌ | 8491/9864 [19:04:36<5:35:25, 14.66s/it]
 86%|████████▌ | 8492/9864 [19:04:50<5:31:25, 14.49s/it]
 86%|████████▌ | 8493/9864 [19:05:04<5:27:45, 14.34s/it]
 86%|████████▌ | 8494/9864 [19:05:22<5:48:03, 15.24s/it]
 86%|████████▌ | 8495/9864 [19:05:38<5:53:18, 15.48s/it]
 86%|████████▌ | 8496/9864 [19:05:53<5:54:01, 15.53s/it]
 86%|████████▌ | 8497/9864 [19:06:08<5:44:41, 15.13s/it]
 86%|████████▌ | 8498/9864 [19:06:22<5:36:04, 14.76s/it]
 86%|████████▌ | 8499/9864 [19:06:35<5:30:08, 14.51s/it]
 86%|████████▌ | 8500/9864 [19:06:50<5:28:38, 14.46s/it]
                                                        
{'loss': '0.06058', 'grad_norm': '4.768', 'learning_rate': '9.877e-07', 'epoch': '0.8618'}

 86%|████████▌ | 8500/9864 [19:06:50<5:28:38, 14.46s/it]
 86%|████████▌ | 8501/9864 [19:07:05<5:30:03, 14.53s/it]
 86%|████████▌ | 8502/9864 [19:07:22<5:51:37, 15.49s/it]
 86%|████████▌ | 8503/9864 [19:07:37<5:44:24, 15.18s/it]
 86%|████████▌ | 8504/9864 [19:07:54<5:58:56, 15.84s/it]
 86%|████████▌ | 8505/9864 [19:08:08<5:48:28, 15.39s/it]
 86%|████████▌ | 8506/9864 [19:08:23<5:40:18, 15.04s/it]
 86%|████████▌ | 8507/9864 [19:08:36<5:30:11, 14.60s/it]
 86%|████████▋ | 8508/9864 [19:08:51<5:28:10, 14.52s/it]
 86%|████████▋ | 8509/9864 [19:09:06<5:36:31, 14.90s/it]
 86%|████████▋ | 8510/9864 [19:09:22<5:41:25, 15.13s/it]
                                                        
{'loss': '0.05747', 'grad_norm': '5.425', 'learning_rate': '9.735e-07', 'epoch': '0.8628'}

 86%|████████▋ | 8510/9864 [19:09:22<5:41:25, 15.13s/it]
 86%|████████▋ | 8511/9864 [19:09:37<5:41:19, 15.14s/it]
 86%|████████▋ | 8512/9864 [19:09:53<5:45:58, 15.35s/it]
 86%|████████▋ | 8513/9864 [19:10:07<5:35:03, 14.88s/it]
 86%|████████▋ | 8514/9864 [19:10:21<5:30:52, 14.71s/it]
 86%|████████▋ | 8515/9864 [19:10:35<5:26:24, 14.52s/it]
 86%|████████▋ | 8516/9864 [19:10:49<5:20:57, 14.29s/it]
 86%|████████▋ | 8517/9864 [19:11:07<5:43:57, 15.32s/it]
 86%|████████▋ | 8518/9864 [19:11:21<5:34:22, 14.91s/it]
 86%|████████▋ | 8519/9864 [19:11:38<5:50:16, 15.63s/it]
 86%|████████▋ | 8520/9864 [19:11:52<5:40:37, 15.21s/it]
                                                        
{'loss': '0.0563', 'grad_norm': '6.851', 'learning_rate': '9.594e-07', 'epoch': '0.8638'}

 86%|████████▋ | 8520/9864 [19:11:52<5:40:37, 15.21s/it]
 86%|████████▋ | 8521/9864 [19:12:06<5:29:57, 14.74s/it]
 86%|████████▋ | 8522/9864 [19:12:20<5:26:36, 14.60s/it]
 86%|████████▋ | 8523/9864 [19:12:34<5:20:54, 14.36s/it]
 86%|████████▋ | 8524/9864 [19:12:50<5:30:31, 14.80s/it]
 86%|████████▋ | 8525/9864 [19:13:05<5:34:53, 15.01s/it]
 86%|████████▋ | 8526/9864 [19:13:19<5:27:19, 14.68s/it]
 86%|████████▋ | 8527/9864 [19:13:37<5:47:06, 15.58s/it]
 86%|████████▋ | 8528/9864 [19:13:51<5:36:41, 15.12s/it]
 86%|████████▋ | 8529/9864 [19:14:05<5:27:44, 14.73s/it]
 86%|████████▋ | 8530/9864 [19:14:19<5:25:26, 14.64s/it]
                                                        
{'loss': '0.05006', 'grad_norm': '4.568', 'learning_rate': '9.454e-07', 'epoch': '0.8648'}

 86%|████████▋ | 8530/9864 [19:14:19<5:25:26, 14.64s/it]
 86%|████████▋ | 8531/9864 [19:14:33<5:23:41, 14.57s/it]
 86%|████████▋ | 8532/9864 [19:14:49<5:27:46, 14.76s/it]
 87%|████████▋ | 8533/9864 [19:15:04<5:31:40, 14.95s/it]
 87%|████████▋ | 8534/9864 [19:15:20<5:35:14, 15.12s/it]
 87%|████████▋ | 8535/9864 [19:15:33<5:26:19, 14.73s/it]
 87%|████████▋ | 8536/9864 [19:15:49<5:31:30, 14.98s/it]
 87%|████████▋ | 8537/9864 [19:16:03<5:25:18, 14.71s/it]
 87%|████████▋ | 8538/9864 [19:16:17<5:18:50, 14.43s/it]
 87%|████████▋ | 8539/9864 [19:16:31<5:16:27, 14.33s/it]
 87%|████████▋ | 8540/9864 [19:16:47<5:26:46, 14.81s/it]
                                                        
{'loss': '0.05622', 'grad_norm': '3.714', 'learning_rate': '9.315e-07', 'epoch': '0.8658'}

 87%|████████▋ | 8540/9864 [19:16:47<5:26:46, 14.81s/it]
 87%|████████▋ | 8541/9864 [19:17:01<5:25:13, 14.75s/it]
 87%|████████▋ | 8542/9864 [19:17:19<5:43:07, 15.57s/it]
 87%|████████▋ | 8543/9864 [19:17:33<5:31:53, 15.07s/it]
 87%|████████▋ | 8544/9864 [19:17:49<5:36:12, 15.28s/it]
 87%|████████▋ | 8545/9864 [19:18:02<5:25:56, 14.83s/it]
 87%|████████▋ | 8546/9864 [19:18:16<5:20:35, 14.59s/it]
 87%|████████▋ | 8547/9864 [19:18:30<5:16:00, 14.40s/it]
 87%|████████▋ | 8548/9864 [19:18:47<5:28:41, 14.99s/it]
 87%|████████▋ | 8549/9864 [19:19:03<5:34:57, 15.28s/it]
 87%|████████▋ | 8550/9864 [19:19:18<5:37:38, 15.42s/it]
                                                        
{'loss': '0.05997', 'grad_norm': '4.956', 'learning_rate': '9.177e-07', 'epoch': '0.8669'}

 87%|████████▋ | 8550/9864 [19:19:18<5:37:38, 15.42s/it]
 87%|████████▋ | 8551/9864 [19:19:34<5:40:59, 15.58s/it]
 87%|████████▋ | 8552/9864 [19:19:49<5:32:43, 15.22s/it]
 87%|████████▋ | 8553/9864 [19:20:02<5:20:14, 14.66s/it]
 87%|████████▋ | 8554/9864 [19:20:16<5:14:52, 14.42s/it]
 87%|████████▋ | 8555/9864 [19:20:30<5:13:43, 14.38s/it]
 87%|████████▋ | 8556/9864 [19:20:45<5:14:30, 14.43s/it]
 87%|████████▋ | 8557/9864 [19:21:03<5:38:02, 15.52s/it]
 87%|████████▋ | 8558/9864 [19:21:19<5:41:36, 15.69s/it]
 87%|████████▋ | 8559/9864 [19:21:35<5:40:27, 15.65s/it]
 87%|████████▋ | 8560/9864 [19:21:49<5:32:16, 15.29s/it]
                                                        
{'loss': '0.05849', 'grad_norm': '5.198', 'learning_rate': '9.041e-07', 'epoch': '0.8679'}

 87%|████████▋ | 8560/9864 [19:21:49<5:32:16, 15.29s/it]
 87%|████████▋ | 8561/9864 [19:22:03<5:25:07, 14.97s/it]
 87%|████████▋ | 8562/9864 [19:22:17<5:20:02, 14.75s/it]
 87%|████████▋ | 8563/9864 [19:22:32<5:16:17, 14.59s/it]
 87%|████████▋ | 8564/9864 [19:22:46<5:11:42, 14.39s/it]
 87%|████████▋ | 8565/9864 [19:23:05<5:41:23, 15.77s/it]
 87%|████████▋ | 8566/9864 [19:23:19<5:32:12, 15.36s/it]
 87%|████████▋ | 8567/9864 [19:23:35<5:36:20, 15.56s/it]
 87%|████████▋ | 8568/9864 [19:23:49<5:23:55, 15.00s/it]
 87%|████████▋ | 8569/9864 [19:24:03<5:17:04, 14.69s/it]
 87%|████████▋ | 8570/9864 [19:24:17<5:14:19, 14.57s/it]
                                                        
{'loss': '0.05471', 'grad_norm': '6.39', 'learning_rate': '8.905e-07', 'epoch': '0.8689'}

 87%|████████▋ | 8570/9864 [19:24:17<5:14:19, 14.57s/it]
 87%|████████▋ | 8571/9864 [19:24:31<5:10:02, 14.39s/it]
 87%|████████▋ | 8572/9864 [19:24:45<5:05:23, 14.18s/it]
 87%|████████▋ | 8573/9864 [19:25:04<5:36:08, 15.62s/it]
 87%|████████▋ | 8574/9864 [19:25:18<5:28:40, 15.29s/it]
 87%|████████▋ | 8575/9864 [19:25:34<5:33:48, 15.54s/it]
 87%|████████▋ | 8576/9864 [19:25:48<5:25:22, 15.16s/it]
 87%|████████▋ | 8577/9864 [19:26:03<5:19:40, 14.90s/it]
 87%|████████▋ | 8578/9864 [19:26:16<5:10:12, 14.47s/it]
 87%|████████▋ | 8579/9864 [19:26:30<5:05:53, 14.28s/it]
 87%|████████▋ | 8580/9864 [19:26:44<5:05:33, 14.28s/it]
                                                        
{'loss': '0.06319', 'grad_norm': '4.148', 'learning_rate': '8.77e-07', 'epoch': '0.8699'}

 87%|████████▋ | 8580/9864 [19:26:44<5:05:33, 14.28s/it]
 87%|████████▋ | 8581/9864 [19:27:04<5:37:02, 15.76s/it]
 87%|████████▋ | 8582/9864 [19:27:18<5:28:11, 15.36s/it]
 87%|████████▋ | 8583/9864 [19:27:34<5:29:14, 15.42s/it]
 87%|████████▋ | 8584/9864 [19:27:48<5:22:22, 15.11s/it]
 87%|████████▋ | 8585/9864 [19:28:03<5:18:43, 14.95s/it]
 87%|████████▋ | 8586/9864 [19:28:17<5:13:11, 14.70s/it]
 87%|████████▋ | 8587/9864 [19:28:31<5:09:46, 14.55s/it]
 87%|████████▋ | 8588/9864 [19:28:47<5:16:37, 14.89s/it]
 87%|████████▋ | 8589/9864 [19:29:04<5:31:39, 15.61s/it]
 87%|████████▋ | 8590/9864 [19:29:18<5:23:49, 15.25s/it]
                                                        
{'loss': '0.06396', 'grad_norm': '4.235', 'learning_rate': '8.636e-07', 'epoch': '0.8709'}

 87%|████████▋ | 8590/9864 [19:29:18<5:23:49, 15.25s/it]
 87%|████████▋ | 8591/9864 [19:29:34<5:28:06, 15.46s/it]
 87%|████████▋ | 8592/9864 [19:29:48<5:19:12, 15.06s/it]
 87%|████████▋ | 8593/9864 [19:30:03<5:13:53, 14.82s/it]
 87%|████████▋ | 8594/9864 [19:30:17<5:09:08, 14.61s/it]
 87%|████████▋ | 8595/9864 [19:30:33<5:18:37, 15.07s/it]
 87%|████████▋ | 8596/9864 [19:30:51<5:35:39, 15.88s/it]
 87%|████████▋ | 8597/9864 [19:31:04<5:19:55, 15.15s/it]
 87%|████████▋ | 8598/9864 [19:31:19<5:20:56, 15.21s/it]
 87%|████████▋ | 8599/9864 [19:31:33<5:11:07, 14.76s/it]
 87%|████████▋ | 8600/9864 [19:31:48<5:09:38, 14.70s/it]
                                                        
{'loss': '0.05134', 'grad_norm': '5.738', 'learning_rate': '8.503e-07', 'epoch': '0.8719'}

 87%|████████▋ | 8600/9864 [19:31:48<5:09:38, 14.70s/it]
 87%|████████▋ | 8601/9864 [19:32:01<5:01:06, 14.30s/it]
 87%|████████▋ | 8602/9864 [19:32:15<5:00:30, 14.29s/it]
 87%|████████▋ | 8603/9864 [19:32:31<5:06:43, 14.59s/it]
 87%|████████▋ | 8604/9864 [19:32:48<5:23:13, 15.39s/it]
 87%|████████▋ | 8605/9864 [19:33:02<5:13:10, 14.92s/it]
 87%|████████▋ | 8606/9864 [19:33:17<5:18:12, 15.18s/it]
 87%|████████▋ | 8607/9864 [19:33:32<5:11:00, 14.85s/it]
 87%|████████▋ | 8608/9864 [19:33:45<5:04:53, 14.57s/it]
 87%|████████▋ | 8609/9864 [19:33:59<5:00:45, 14.38s/it]
 87%|████████▋ | 8610/9864 [19:34:14<4:59:03, 14.31s/it]
                                                        
{'loss': '0.05958', 'grad_norm': '4.634', 'learning_rate': '8.371e-07', 'epoch': '0.8729'}

 87%|████████▋ | 8610/9864 [19:34:14<4:59:03, 14.31s/it]
 87%|████████▋ | 8611/9864 [19:34:29<5:08:28, 14.77s/it]
 87%|████████▋ | 8612/9864 [19:34:47<5:24:22, 15.54s/it]
 87%|████████▋ | 8613/9864 [19:35:01<5:16:31, 15.18s/it]
 87%|████████▋ | 8614/9864 [19:35:17<5:18:37, 15.29s/it]
 87%|████████▋ | 8615/9864 [19:35:30<5:06:43, 14.73s/it]
 87%|████████▋ | 8616/9864 [19:35:44<5:03:17, 14.58s/it]
 87%|████████▋ | 8617/9864 [19:35:59<5:02:52, 14.57s/it]
 87%|████████▋ | 8618/9864 [19:36:14<5:09:02, 14.88s/it]
 87%|████████▋ | 8619/9864 [19:36:30<5:15:07, 15.19s/it]
 87%|████████▋ | 8620/9864 [19:36:46<5:17:45, 15.33s/it]
                                                        
{'loss': '0.05099', 'grad_norm': '4.109', 'learning_rate': '8.24e-07', 'epoch': '0.874'}

 87%|████████▋ | 8620/9864 [19:36:46<5:17:45, 15.33s/it]
 87%|████████▋ | 8621/9864 [19:37:00<5:10:06, 14.97s/it]
 87%|████████▋ | 8622/9864 [19:37:16<5:13:21, 15.14s/it]
 87%|████████▋ | 8623/9864 [19:37:30<5:07:31, 14.87s/it]
 87%|████████▋ | 8624/9864 [19:37:44<5:02:58, 14.66s/it]
 87%|████████▋ | 8625/9864 [19:37:58<4:59:47, 14.52s/it]
 87%|████████▋ | 8626/9864 [19:38:15<5:15:47, 15.31s/it]
 87%|████████▋ | 8627/9864 [19:38:31<5:18:45, 15.46s/it]
 87%|████████▋ | 8628/9864 [19:38:45<5:06:47, 14.89s/it]
 87%|████████▋ | 8629/9864 [19:39:00<5:11:39, 15.14s/it]
 87%|████████▋ | 8630/9864 [19:39:14<5:00:08, 14.59s/it]
                                                        
{'loss': '0.04828', 'grad_norm': '2.837', 'learning_rate': '8.11e-07', 'epoch': '0.875'}

 87%|████████▋ | 8630/9864 [19:39:14<5:00:08, 14.59s/it]
 88%|████████▊ | 8631/9864 [19:39:27<4:53:47, 14.30s/it]
 88%|████████▊ | 8632/9864 [19:39:42<4:53:45, 14.31s/it]
 88%|████████▊ | 8633/9864 [19:39:56<4:50:25, 14.16s/it]
 88%|████████▊ | 8634/9864 [19:40:13<5:10:33, 15.15s/it]
 88%|████████▊ | 8635/9864 [19:40:27<5:05:09, 14.90s/it]
 88%|████████▊ | 8636/9864 [19:40:43<5:11:34, 15.22s/it]
 88%|████████▊ | 8637/9864 [19:40:59<5:14:51, 15.40s/it]
 88%|████████▊ | 8638/9864 [19:41:14<5:09:04, 15.13s/it]
 88%|████████▊ | 8639/9864 [19:41:28<5:02:53, 14.84s/it]
 88%|████████▊ | 8640/9864 [19:41:42<5:00:15, 14.72s/it]
                                                        
{'loss': '0.06534', 'grad_norm': '5.33', 'learning_rate': '7.981e-07', 'epoch': '0.876'}

 88%|████████▊ | 8640/9864 [19:41:42<5:00:15, 14.72s/it]
 88%|████████▊ | 8641/9864 [19:41:58<5:07:47, 15.10s/it]
 88%|████████▊ | 8642/9864 [19:42:14<5:10:30, 15.25s/it]
 88%|████████▊ | 8643/9864 [19:42:30<5:14:50, 15.47s/it]
 88%|████████▊ | 8644/9864 [19:42:44<5:05:56, 15.05s/it]
 88%|████████▊ | 8645/9864 [19:42:59<5:09:10, 15.22s/it]
 88%|████████▊ | 8646/9864 [19:43:13<5:01:44, 14.86s/it]
 88%|████████▊ | 8647/9864 [19:43:28<4:59:26, 14.76s/it]
 88%|████████▊ | 8648/9864 [19:43:43<4:57:59, 14.70s/it]
 88%|████████▊ | 8649/9864 [19:43:58<5:03:43, 15.00s/it]
 88%|████████▊ | 8650/9864 [19:44:14<5:05:12, 15.08s/it]
                                                        
{'loss': '0.04662', 'grad_norm': '4.082', 'learning_rate': '7.853e-07', 'epoch': '0.877'}

 88%|████████▊ | 8650/9864 [19:44:14<5:05:12, 15.08s/it]
 88%|████████▊ | 8651/9864 [19:44:29<5:09:48, 15.32s/it]
 88%|████████▊ | 8652/9864 [19:44:44<5:04:05, 15.05s/it]
 88%|████████▊ | 8653/9864 [19:45:00<5:08:32, 15.29s/it]
 88%|████████▊ | 8654/9864 [19:45:14<5:01:27, 14.95s/it]
 88%|████████▊ | 8655/9864 [19:45:28<4:54:42, 14.63s/it]
 88%|████████▊ | 8656/9864 [19:45:41<4:46:50, 14.25s/it]
 88%|████████▊ | 8657/9864 [19:45:57<4:53:43, 14.60s/it]
 88%|████████▊ | 8658/9864 [19:46:14<5:08:06, 15.33s/it]
 88%|████████▊ | 8659/9864 [19:46:27<4:58:09, 14.85s/it]
 88%|████████▊ | 8660/9864 [19:46:42<4:56:13, 14.76s/it]
                                                        
{'loss': '0.04076', 'grad_norm': '5.013', 'learning_rate': '7.726e-07', 'epoch': '0.878'}

 88%|████████▊ | 8660/9864 [19:46:42<4:56:13, 14.76s/it]
 88%|████████▊ | 8661/9864 [19:46:58<5:03:28, 15.14s/it]
 88%|████████▊ | 8662/9864 [19:47:12<4:55:32, 14.75s/it]
 88%|████████▊ | 8663/9864 [19:47:26<4:51:08, 14.55s/it]
 88%|████████▊ | 8664/9864 [19:47:40<4:51:53, 14.59s/it]
 88%|████████▊ | 8665/9864 [19:47:56<4:58:02, 14.91s/it]
 88%|████████▊ | 8666/9864 [19:48:13<5:10:56, 15.57s/it]
 88%|████████▊ | 8667/9864 [19:48:27<5:02:17, 15.15s/it]
 88%|████████▊ | 8668/9864 [19:48:43<5:06:27, 15.37s/it]
 88%|████████▊ | 8669/9864 [19:48:57<4:57:03, 14.92s/it]
 88%|████████▊ | 8670/9864 [19:49:11<4:51:05, 14.63s/it]
                                                        
{'loss': '0.0519', 'grad_norm': '5.025', 'learning_rate': '7.599e-07', 'epoch': '0.879'}

 88%|████████▊ | 8670/9864 [19:49:11<4:51:05, 14.63s/it]
 88%|████████▊ | 8671/9864 [19:49:25<4:48:50, 14.53s/it]
 88%|████████▊ | 8672/9864 [19:49:40<4:48:26, 14.52s/it]
 88%|████████▊ | 8673/9864 [19:49:56<4:57:09, 14.97s/it]
 88%|████████▊ | 8674/9864 [19:50:13<5:10:57, 15.68s/it]
 88%|████████▊ | 8675/9864 [19:50:27<5:02:03, 15.24s/it]
 88%|████████▊ | 8676/9864 [19:50:43<5:05:10, 15.41s/it]
 88%|████████▊ | 8677/9864 [19:50:57<4:57:46, 15.05s/it]
 88%|████████▊ | 8678/9864 [19:51:11<4:49:20, 14.64s/it]
 88%|████████▊ | 8679/9864 [19:51:25<4:47:16, 14.55s/it]
 88%|████████▊ | 8680/9864 [19:51:39<4:43:21, 14.36s/it]
                                                        
{'loss': '0.05954', 'grad_norm': '5.248', 'learning_rate': '7.474e-07', 'epoch': '0.88'}

 88%|████████▊ | 8680/9864 [19:51:39<4:43:21, 14.36s/it]
 88%|████████▊ | 8681/9864 [19:51:55<4:50:30, 14.73s/it]
 88%|████████▊ | 8682/9864 [19:52:11<4:58:00, 15.13s/it]
 88%|████████▊ | 8683/9864 [19:52:26<4:59:25, 15.21s/it]
 88%|████████▊ | 8684/9864 [19:52:42<5:02:39, 15.39s/it]
 88%|████████▊ | 8685/9864 [19:52:56<4:54:35, 14.99s/it]
 88%|████████▊ | 8686/9864 [19:53:11<4:49:48, 14.76s/it]
 88%|████████▊ | 8687/9864 [19:53:25<4:47:38, 14.66s/it]
 88%|████████▊ | 8688/9864 [19:53:39<4:42:24, 14.41s/it]
 88%|████████▊ | 8689/9864 [19:53:55<4:51:19, 14.88s/it]
 88%|████████▊ | 8690/9864 [19:54:10<4:55:06, 15.08s/it]
                                                        
{'loss': '0.05026', 'grad_norm': '4.877', 'learning_rate': '7.35e-07', 'epoch': '0.8811'}

 88%|████████▊ | 8690/9864 [19:54:10<4:55:06, 15.08s/it]
 88%|████████▊ | 8691/9864 [19:54:25<4:49:24, 14.80s/it]
 88%|████████▊ | 8692/9864 [19:54:41<4:56:55, 15.20s/it]
 88%|████████▊ | 8693/9864 [19:54:57<5:01:19, 15.44s/it]
 88%|████████▊ | 8694/9864 [19:55:11<4:53:04, 15.03s/it]
 88%|████████▊ | 8695/9864 [19:55:25<4:49:03, 14.84s/it]
 88%|████████▊ | 8696/9864 [19:55:39<4:44:47, 14.63s/it]
 88%|████████▊ | 8697/9864 [19:55:55<4:51:45, 15.00s/it]
 88%|████████▊ | 8698/9864 [19:56:11<4:55:14, 15.19s/it]
 88%|████████▊ | 8699/9864 [19:56:26<4:56:59, 15.30s/it]
 88%|████████▊ | 8700/9864 [19:56:42<5:01:28, 15.54s/it]
                                                        
{'loss': '0.06329', 'grad_norm': '4.793', 'learning_rate': '7.227e-07', 'epoch': '0.8821'}

 88%|████████▊ | 8700/9864 [19:56:42<5:01:28, 15.54s/it]
 88%|████████▊ | 8701/9864 [19:56:56<4:51:15, 15.03s/it]
 88%|████████▊ | 8702/9864 [19:57:10<4:44:56, 14.71s/it]
 88%|████████▊ | 8703/9864 [19:57:24<4:40:42, 14.51s/it]
 88%|████████▊ | 8704/9864 [19:57:40<4:45:01, 14.74s/it]
 88%|████████▊ | 8705/9864 [19:57:53<4:39:57, 14.49s/it]
 88%|████████▊ | 8706/9864 [19:58:09<4:48:17, 14.94s/it]
 88%|████████▊ | 8707/9864 [19:58:27<5:05:20, 15.83s/it]
 88%|████████▊ | 8708/9864 [19:58:41<4:53:29, 15.23s/it]
 88%|████████▊ | 8709/9864 [19:58:55<4:47:09, 14.92s/it]
 88%|████████▊ | 8710/9864 [19:59:09<4:41:49, 14.65s/it]
                                                        
{'loss': '0.04886', 'grad_norm': '4.169', 'learning_rate': '7.105e-07', 'epoch': '0.8831'}

 88%|████████▊ | 8710/9864 [19:59:09<4:41:49, 14.65s/it]
 88%|████████▊ | 8711/9864 [19:59:25<4:48:31, 15.01s/it]
 88%|████████▊ | 8712/9864 [19:59:39<4:43:42, 14.78s/it]
 88%|████████▊ | 8713/9864 [19:59:53<4:38:18, 14.51s/it]
 88%|████████▊ | 8714/9864 [20:00:11<4:54:16, 15.35s/it]
 88%|████████▊ | 8715/9864 [20:00:27<4:57:28, 15.53s/it]
 88%|████████▊ | 8716/9864 [20:00:41<4:50:10, 15.17s/it]
 88%|████████▊ | 8717/9864 [20:00:55<4:42:33, 14.78s/it]
 88%|████████▊ | 8718/9864 [20:01:09<4:40:01, 14.66s/it]
 88%|████████▊ | 8719/9864 [20:01:24<4:38:13, 14.58s/it]
 88%|████████▊ | 8720/9864 [20:01:39<4:44:26, 14.92s/it]
                                                        
{'loss': '0.05452', 'grad_norm': '5.084', 'learning_rate': '6.984e-07', 'epoch': '0.8841'}

 88%|████████▊ | 8720/9864 [20:01:39<4:44:26, 14.92s/it]
 88%|████████▊ | 8721/9864 [20:01:53<4:37:56, 14.59s/it]
 88%|████████▊ | 8722/9864 [20:02:11<4:54:45, 15.49s/it]
 88%|████████▊ | 8723/9864 [20:02:27<4:57:12, 15.63s/it]
 88%|████████▊ | 8724/9864 [20:02:41<4:50:04, 15.27s/it]
 88%|████████▊ | 8725/9864 [20:02:55<4:39:54, 14.75s/it]
 88%|████████▊ | 8726/9864 [20:03:09<4:35:07, 14.51s/it]
 88%|████████▊ | 8727/9864 [20:03:23<4:36:05, 14.57s/it]
 88%|████████▊ | 8728/9864 [20:03:40<4:45:54, 15.10s/it]
 88%|████████▊ | 8729/9864 [20:03:54<4:42:20, 14.93s/it]
 89%|████████▊ | 8730/9864 [20:04:11<4:55:30, 15.64s/it]
                                                        
{'loss': '0.03686', 'grad_norm': '2.738', 'learning_rate': '6.864e-07', 'epoch': '0.8851'}

 89%|████████▊ | 8730/9864 [20:04:11<4:55:30, 15.64s/it]
 89%|████████▊ | 8731/9864 [20:04:27<4:55:55, 15.67s/it]
 89%|████████▊ | 8732/9864 [20:04:41<4:44:47, 15.09s/it]
 89%|████████▊ | 8733/9864 [20:04:55<4:36:43, 14.68s/it]
 89%|████████▊ | 8734/9864 [20:05:09<4:36:06, 14.66s/it]
 89%|████████▊ | 8735/9864 [20:05:25<4:40:50, 14.93s/it]
 89%|████████▊ | 8736/9864 [20:05:39<4:37:05, 14.74s/it]
 89%|████████▊ | 8737/9864 [20:05:55<4:40:48, 14.95s/it]
 89%|████████▊ | 8738/9864 [20:06:12<4:56:35, 15.80s/it]
 89%|████████▊ | 8739/9864 [20:06:26<4:46:33, 15.28s/it]
 89%|████████▊ | 8740/9864 [20:06:40<4:37:45, 14.83s/it]
                                                        
{'loss': '0.06011', 'grad_norm': '4.288', 'learning_rate': '6.745e-07', 'epoch': '0.8861'}

 89%|████████▊ | 8740/9864 [20:06:40<4:37:45, 14.83s/it]
 89%|████████▊ | 8741/9864 [20:06:54<4:31:49, 14.52s/it]
 89%|████████▊ | 8742/9864 [20:07:09<4:36:17, 14.78s/it]
 89%|████████▊ | 8743/9864 [20:07:23<4:29:57, 14.45s/it]
 89%|████████▊ | 8744/9864 [20:07:37<4:27:44, 14.34s/it]
 89%|████████▊ | 8745/9864 [20:07:53<4:34:41, 14.73s/it]
 89%|████████▊ | 8746/9864 [20:08:10<4:50:00, 15.56s/it]
 89%|████████▊ | 8747/9864 [20:08:25<4:42:51, 15.19s/it]
 89%|████████▊ | 8748/9864 [20:08:38<4:34:21, 14.75s/it]
 89%|████████▊ | 8749/9864 [20:08:54<4:39:58, 15.07s/it]
 89%|████████▊ | 8750/9864 [20:09:09<4:36:04, 14.87s/it]
                                                        
{'loss': '0.05699', 'grad_norm': '4.687', 'learning_rate': '6.627e-07', 'epoch': '0.8871'}

 89%|████████▊ | 8750/9864 [20:09:09<4:36:04, 14.87s/it]
 89%|████████▊ | 8751/9864 [20:09:23<4:33:26, 14.74s/it]
 89%|████████▊ | 8752/9864 [20:09:39<4:40:00, 15.11s/it]
 89%|████████▊ | 8753/9864 [20:09:53<4:33:27, 14.77s/it]
 89%|████████▊ | 8754/9864 [20:10:11<4:49:06, 15.63s/it]
 89%|████████▉ | 8755/9864 [20:10:25<4:43:13, 15.32s/it]
 89%|████████▉ | 8756/9864 [20:10:39<4:36:15, 14.96s/it]
 89%|████████▉ | 8757/9864 [20:10:55<4:38:39, 15.10s/it]
 89%|████████▉ | 8758/9864 [20:11:09<4:34:05, 14.87s/it]
 89%|████████▉ | 8759/9864 [20:11:24<4:33:38, 14.86s/it]
 89%|████████▉ | 8760/9864 [20:11:38<4:29:10, 14.63s/it]
                                                        
{'loss': '0.06948', 'grad_norm': '4.747', 'learning_rate': '6.51e-07', 'epoch': '0.8882'}

 89%|████████▉ | 8760/9864 [20:11:38<4:29:10, 14.63s/it]
 89%|████████▉ | 8761/9864 [20:11:54<4:35:55, 15.01s/it]
 89%|████████▉ | 8762/9864 [20:12:08<4:33:35, 14.90s/it]
 89%|████████▉ | 8763/9864 [20:12:26<4:49:00, 15.75s/it]
 89%|████████▉ | 8764/9864 [20:12:40<4:40:29, 15.30s/it]
 89%|████████▉ | 8765/9864 [20:12:56<4:41:59, 15.40s/it]
 89%|████████▉ | 8766/9864 [20:13:10<4:36:07, 15.09s/it]
 89%|████████▉ | 8767/9864 [20:13:25<4:33:21, 14.95s/it]
 89%|████████▉ | 8768/9864 [20:13:39<4:26:21, 14.58s/it]
 89%|████████▉ | 8769/9864 [20:13:52<4:20:20, 14.27s/it]
 89%|████████▉ | 8770/9864 [20:14:08<4:28:48, 14.74s/it]
                                                        
{'loss': '0.05169', 'grad_norm': '5.476', 'learning_rate': '6.394e-07', 'epoch': '0.8892'}

 89%|████████▉ | 8770/9864 [20:14:08<4:28:48, 14.74s/it]
 89%|████████▉ | 8771/9864 [20:14:26<4:43:18, 15.55s/it]
 89%|████████▉ | 8772/9864 [20:14:40<4:35:00, 15.11s/it]
 89%|████████▉ | 8773/9864 [20:14:56<4:40:29, 15.43s/it]
 89%|████████▉ | 8774/9864 [20:15:10<4:33:44, 15.07s/it]
 89%|████████▉ | 8775/9864 [20:15:25<4:30:19, 14.89s/it]
 89%|████████▉ | 8776/9864 [20:15:38<4:24:21, 14.58s/it]
 89%|████████▉ | 8777/9864 [20:15:54<4:28:56, 14.85s/it]
 89%|████████▉ | 8778/9864 [20:16:09<4:32:32, 15.06s/it]
 89%|████████▉ | 8779/9864 [20:16:25<4:34:50, 15.20s/it]
 89%|████████▉ | 8780/9864 [20:16:39<4:27:49, 14.82s/it]
                                                        
{'loss': '0.05507', 'grad_norm': '5.096', 'learning_rate': '6.279e-07', 'epoch': '0.8902'}

 89%|████████▉ | 8780/9864 [20:16:39<4:27:49, 14.82s/it]
 89%|████████▉ | 8781/9864 [20:16:55<4:31:37, 15.05s/it]
 89%|████████▉ | 8782/9864 [20:17:09<4:28:31, 14.89s/it]
 89%|████████▉ | 8783/9864 [20:17:23<4:21:23, 14.51s/it]
 89%|████████▉ | 8784/9864 [20:17:36<4:17:11, 14.29s/it]
 89%|████████▉ | 8785/9864 [20:17:54<4:33:50, 15.23s/it]
 89%|████████▉ | 8786/9864 [20:18:10<4:37:24, 15.44s/it]
 89%|████████▉ | 8787/9864 [20:18:24<4:30:59, 15.10s/it]
 89%|████████▉ | 8788/9864 [20:18:39<4:31:31, 15.14s/it]
 89%|████████▉ | 8789/9864 [20:18:54<4:27:28, 14.93s/it]
 89%|████████▉ | 8790/9864 [20:19:08<4:23:46, 14.74s/it]
                                                        
{'loss': '0.06055', 'grad_norm': '6.557', 'learning_rate': '6.165e-07', 'epoch': '0.8912'}

 89%|████████▉ | 8790/9864 [20:19:08<4:23:46, 14.74s/it]
 89%|████████▉ | 8791/9864 [20:19:23<4:22:33, 14.68s/it]
 89%|████████▉ | 8792/9864 [20:19:36<4:17:55, 14.44s/it]
 89%|████████▉ | 8793/9864 [20:19:52<4:24:10, 14.80s/it]
 89%|████████▉ | 8794/9864 [20:20:10<4:38:04, 15.59s/it]
 89%|████████▉ | 8795/9864 [20:20:24<4:30:45, 15.20s/it]
 89%|████████▉ | 8796/9864 [20:20:40<4:33:47, 15.38s/it]
 89%|████████▉ | 8797/9864 [20:20:54<4:27:56, 15.07s/it]
 89%|████████▉ | 8798/9864 [20:21:08<4:24:14, 14.87s/it]
 89%|████████▉ | 8799/9864 [20:21:22<4:18:04, 14.54s/it]
 89%|████████▉ | 8800/9864 [20:21:38<4:23:45, 14.87s/it]
                                                        
{'loss': '0.0612', 'grad_norm': '5.828', 'learning_rate': '6.052e-07', 'epoch': '0.8922'}

 89%|████████▉ | 8800/9864 [20:21:38<4:23:45, 14.87s/it]
 89%|████████▉ | 8801/9864 [20:21:54<4:28:21, 15.15s/it]
 89%|████████▉ | 8802/9864 [20:22:09<4:29:29, 15.23s/it]
 89%|████████▉ | 8803/9864 [20:22:23<4:25:10, 15.00s/it]
 89%|████████▉ | 8804/9864 [20:22:40<4:30:47, 15.33s/it]
 89%|████████▉ | 8805/9864 [20:22:54<4:24:00, 14.96s/it]
 89%|████████▉ | 8806/9864 [20:23:08<4:19:11, 14.70s/it]
 89%|████████▉ | 8807/9864 [20:23:22<4:15:59, 14.53s/it]
 89%|████████▉ | 8808/9864 [20:23:36<4:12:36, 14.35s/it]
 89%|████████▉ | 8809/9864 [20:23:54<4:31:58, 15.47s/it]
 89%|████████▉ | 8810/9864 [20:24:10<4:33:37, 15.58s/it]
                                                        
{'loss': '0.05737', 'grad_norm': '5.459', 'learning_rate': '5.94e-07', 'epoch': '0.8932'}

 89%|████████▉ | 8810/9864 [20:24:10<4:33:37, 15.58s/it]
 89%|████████▉ | 8811/9864 [20:24:26<4:34:53, 15.66s/it]
 89%|████████▉ | 8812/9864 [20:24:40<4:27:03, 15.23s/it]
 89%|████████▉ | 8813/9864 [20:24:54<4:20:59, 14.90s/it]
 89%|████████▉ | 8814/9864 [20:25:08<4:17:37, 14.72s/it]
 89%|████████▉ | 8815/9864 [20:25:22<4:10:21, 14.32s/it]
 89%|████████▉ | 8816/9864 [20:25:36<4:10:05, 14.32s/it]
 89%|████████▉ | 8817/9864 [20:25:54<4:29:17, 15.43s/it]
 89%|████████▉ | 8818/9864 [20:26:10<4:31:04, 15.55s/it]
 89%|████████▉ | 8819/9864 [20:26:26<4:32:03, 15.62s/it]
 89%|████████▉ | 8820/9864 [20:26:39<4:22:07, 15.06s/it]
                                                        
{'loss': '0.06413', 'grad_norm': '4.078', 'learning_rate': '5.829e-07', 'epoch': '0.8942'}

 89%|████████▉ | 8820/9864 [20:26:39<4:22:07, 15.06s/it]
 89%|████████▉ | 8821/9864 [20:26:54<4:17:10, 14.79s/it]
 89%|████████▉ | 8822/9864 [20:27:07<4:11:21, 14.47s/it]
 89%|████████▉ | 8823/9864 [20:27:21<4:09:56, 14.41s/it]
 89%|████████▉ | 8824/9864 [20:27:38<4:18:43, 14.93s/it]
 89%|████████▉ | 8825/9864 [20:27:54<4:24:25, 15.27s/it]
 89%|████████▉ | 8826/9864 [20:28:11<4:36:27, 15.98s/it]
 89%|████████▉ | 8827/9864 [20:28:26<4:26:51, 15.44s/it]
 89%|████████▉ | 8828/9864 [20:28:39<4:18:12, 14.95s/it]
 90%|████████▉ | 8829/9864 [20:28:54<4:14:05, 14.73s/it]
 90%|████████▉ | 8830/9864 [20:29:07<4:08:05, 14.40s/it]
                                                        
{'loss': '0.05413', 'grad_norm': '5.587', 'learning_rate': '5.719e-07', 'epoch': '0.8952'}

 90%|████████▉ | 8830/9864 [20:29:07<4:08:05, 14.40s/it]
 90%|████████▉ | 8831/9864 [20:29:21<4:07:15, 14.36s/it]
 90%|████████▉ | 8832/9864 [20:29:37<4:14:18, 14.79s/it]
 90%|████████▉ | 8833/9864 [20:29:53<4:20:46, 15.18s/it]
 90%|████████▉ | 8834/9864 [20:30:11<4:34:14, 15.98s/it]
 90%|████████▉ | 8835/9864 [20:30:25<4:23:07, 15.34s/it]
 90%|████████▉ | 8836/9864 [20:30:39<4:14:43, 14.87s/it]
 90%|████████▉ | 8837/9864 [20:30:53<4:10:47, 14.65s/it]
 90%|████████▉ | 8838/9864 [20:31:07<4:08:03, 14.51s/it]
 90%|████████▉ | 8839/9864 [20:31:21<4:06:43, 14.44s/it]
 90%|████████▉ | 8840/9864 [20:31:39<4:21:17, 15.31s/it]
                                                        
{'loss': '0.05244', 'grad_norm': '6.953', 'learning_rate': '5.61e-07', 'epoch': '0.8963'}

 90%|████████▉ | 8840/9864 [20:31:39<4:21:17, 15.31s/it]
 90%|████████▉ | 8841/9864 [20:31:54<4:22:13, 15.38s/it]
 90%|████████▉ | 8842/9864 [20:32:11<4:26:25, 15.64s/it]
 90%|████████▉ | 8843/9864 [20:32:25<4:18:41, 15.20s/it]
 90%|████████▉ | 8844/9864 [20:32:39<4:13:46, 14.93s/it]
 90%|████████▉ | 8845/9864 [20:32:53<4:10:47, 14.77s/it]
 90%|████████▉ | 8846/9864 [20:33:07<4:06:37, 14.54s/it]
 90%|████████▉ | 8847/9864 [20:33:21<4:00:54, 14.21s/it]
 90%|████████▉ | 8848/9864 [20:33:38<4:16:04, 15.12s/it]
 90%|████████▉ | 8849/9864 [20:33:54<4:18:20, 15.27s/it]
 90%|████████▉ | 8850/9864 [20:34:10<4:21:10, 15.45s/it]
                                                        
{'loss': '0.0589', 'grad_norm': '5.318', 'learning_rate': '5.502e-07', 'epoch': '0.8973'}

 90%|████████▉ | 8850/9864 [20:34:10<4:21:10, 15.45s/it]
 90%|████████▉ | 8851/9864 [20:34:24<4:13:23, 15.01s/it]
 90%|████████▉ | 8852/9864 [20:34:38<4:10:32, 14.85s/it]
 90%|████████▉ | 8853/9864 [20:34:52<4:06:30, 14.63s/it]
 90%|████████▉ | 8854/9864 [20:35:06<4:01:55, 14.37s/it]
 90%|████████▉ | 8855/9864 [20:35:23<4:17:56, 15.34s/it]
 90%|████████▉ | 8856/9864 [20:35:38<4:12:36, 15.04s/it]
 90%|████████▉ | 8857/9864 [20:35:55<4:24:31, 15.76s/it]
 90%|████████▉ | 8858/9864 [20:36:10<4:16:46, 15.31s/it]
 90%|████████▉ | 8859/9864 [20:36:24<4:09:51, 14.92s/it]
 90%|████████▉ | 8860/9864 [20:36:37<4:03:56, 14.58s/it]
                                                        
{'loss': '0.04218', 'grad_norm': '5.98', 'learning_rate': '5.395e-07', 'epoch': '0.8983'}

 90%|████████▉ | 8860/9864 [20:36:37<4:03:56, 14.58s/it]
 90%|████████▉ | 8861/9864 [20:36:52<4:01:47, 14.46s/it]
 90%|████████▉ | 8862/9864 [20:37:07<4:05:27, 14.70s/it]
 90%|████████▉ | 8863/9864 [20:37:22<4:08:43, 14.91s/it]
 90%|████████▉ | 8864/9864 [20:37:36<4:04:55, 14.70s/it]
 90%|████████▉ | 8865/9864 [20:37:51<4:02:20, 14.56s/it]
 90%|████████▉ | 8866/9864 [20:38:08<4:18:38, 15.55s/it]
 90%|████████▉ | 8867/9864 [20:38:23<4:11:18, 15.12s/it]
 90%|████████▉ | 8868/9864 [20:38:37<4:08:21, 14.96s/it]
 90%|████████▉ | 8869/9864 [20:38:51<4:03:10, 14.66s/it]
 90%|████████▉ | 8870/9864 [20:39:09<4:17:28, 15.54s/it]
                                                        
{'loss': '0.05614', 'grad_norm': '4.232', 'learning_rate': '5.289e-07', 'epoch': '0.8993'}

 90%|████████▉ | 8870/9864 [20:39:09<4:17:28, 15.54s/it]
 90%|████████▉ | 8871/9864 [20:39:22<4:07:51, 14.98s/it]
 90%|████████▉ | 8872/9864 [20:39:37<4:03:54, 14.75s/it]
 90%|████████▉ | 8873/9864 [20:39:52<4:09:00, 15.08s/it]
 90%|████████▉ | 8874/9864 [20:40:09<4:14:08, 15.40s/it]
 90%|████████▉ | 8875/9864 [20:40:23<4:10:13, 15.18s/it]
 90%|████████▉ | 8876/9864 [20:40:38<4:05:38, 14.92s/it]
 90%|████████▉ | 8877/9864 [20:40:52<4:01:59, 14.71s/it]
 90%|█████████ | 8878/9864 [20:41:08<4:07:29, 15.06s/it]
 90%|█████████ | 8879/9864 [20:41:24<4:12:22, 15.37s/it]
 90%|█████████ | 8880/9864 [20:41:38<4:07:26, 15.09s/it]
                                                        
{'loss': '0.05542', 'grad_norm': '4.216', 'learning_rate': '5.185e-07', 'epoch': '0.9003'}

 90%|█████████ | 8880/9864 [20:41:38<4:07:26, 15.09s/it]
 90%|█████████ | 8881/9864 [20:41:54<4:10:42, 15.30s/it]
 90%|█████████ | 8882/9864 [20:42:09<4:10:55, 15.33s/it]
 90%|█████████ | 8883/9864 [20:42:24<4:07:18, 15.13s/it]
 90%|█████████ | 8884/9864 [20:42:38<4:01:42, 14.80s/it]
 90%|█████████ | 8885/9864 [20:42:52<3:58:05, 14.59s/it]
 90%|█████████ | 8886/9864 [20:43:09<4:09:32, 15.31s/it]
 90%|█████████ | 8887/9864 [20:43:24<4:04:37, 15.02s/it]
 90%|█████████ | 8888/9864 [20:43:39<4:04:25, 15.03s/it]
 90%|█████████ | 8889/9864 [20:43:54<4:04:31, 15.05s/it]
 90%|█████████ | 8890/9864 [20:44:07<3:55:57, 14.54s/it]
                                                        
{'loss': '0.05283', 'grad_norm': '5.941', 'learning_rate': '5.081e-07', 'epoch': '0.9013'}

 90%|█████████ | 8890/9864 [20:44:07<3:55:57, 14.54s/it]
 90%|█████████ | 8891/9864 [20:44:21<3:53:21, 14.39s/it]
 90%|█████████ | 8892/9864 [20:44:35<3:50:55, 14.25s/it]
 90%|█████████ | 8893/9864 [20:44:49<3:48:32, 14.12s/it]
 90%|█████████ | 8894/9864 [20:45:06<4:04:34, 15.13s/it]
 90%|█████████ | 8895/9864 [20:45:21<4:00:26, 14.89s/it]
 90%|█████████ | 8896/9864 [20:45:36<4:04:01, 15.13s/it]
 90%|█████████ | 8897/9864 [20:45:53<4:09:59, 15.51s/it]
 90%|█████████ | 8898/9864 [20:46:07<4:05:26, 15.24s/it]
 90%|█████████ | 8899/9864 [20:46:22<4:00:38, 14.96s/it]
 90%|█████████ | 8900/9864 [20:46:36<3:55:36, 14.66s/it]
                                                        
{'loss': '0.0496', 'grad_norm': '4.573', 'learning_rate': '4.978e-07', 'epoch': '0.9023'}

 90%|█████████ | 8900/9864 [20:46:36<3:55:36, 14.66s/it]
 90%|█████████ | 8901/9864 [20:46:50<3:53:12, 14.53s/it]
 90%|█████████ | 8902/9864 [20:47:07<4:07:52, 15.46s/it]
 90%|█████████ | 8903/9864 [20:47:22<4:02:21, 15.13s/it]
 90%|█████████ | 8904/9864 [20:47:35<3:54:24, 14.65s/it]
 90%|█████████ | 8905/9864 [20:47:52<4:04:31, 15.30s/it]
 90%|█████████ | 8906/9864 [20:48:06<3:58:49, 14.96s/it]
 90%|█████████ | 8907/9864 [20:48:20<3:54:19, 14.69s/it]
 90%|█████████ | 8908/9864 [20:48:34<3:50:00, 14.44s/it]
 90%|█████████ | 8909/9864 [20:48:50<3:56:52, 14.88s/it]
 90%|█████████ | 8910/9864 [20:49:06<3:59:40, 15.07s/it]
                                                        
{'loss': '0.05596', 'grad_norm': '4.319', 'learning_rate': '4.876e-07', 'epoch': '0.9034'}

 90%|█████████ | 8910/9864 [20:49:06<3:59:40, 15.07s/it]
 90%|█████████ | 8911/9864 [20:49:20<3:55:38, 14.84s/it]
 90%|█████████ | 8912/9864 [20:49:34<3:53:50, 14.74s/it]
 90%|█████████ | 8913/9864 [20:49:52<4:05:18, 15.48s/it]
 90%|█████████ | 8914/9864 [20:50:06<3:57:37, 15.01s/it]
 90%|█████████ | 8915/9864 [20:50:20<3:53:33, 14.77s/it]
 90%|█████████ | 8916/9864 [20:50:36<3:58:12, 15.08s/it]
 90%|█████████ | 8917/9864 [20:50:50<3:52:59, 14.76s/it]
 90%|█████████ | 8918/9864 [20:51:06<3:59:05, 15.16s/it]
 90%|█████████ | 8919/9864 [20:51:20<3:52:32, 14.76s/it]
 90%|█████████ | 8920/9864 [20:51:34<3:50:19, 14.64s/it]
                                                        
{'loss': '0.05048', 'grad_norm': '6.506', 'learning_rate': '4.775e-07', 'epoch': '0.9044'}

 90%|█████████ | 8920/9864 [20:51:34<3:50:19, 14.64s/it]
 90%|█████████ | 8921/9864 [20:51:51<4:00:49, 15.32s/it]
 90%|█████████ | 8922/9864 [20:52:05<3:54:37, 14.94s/it]
 90%|█████████ | 8923/9864 [20:52:19<3:49:15, 14.62s/it]
 90%|█████████ | 8924/9864 [20:52:35<3:55:06, 15.01s/it]
 90%|█████████ | 8925/9864 [20:52:51<4:00:40, 15.38s/it]
 90%|█████████ | 8926/9864 [20:53:05<3:54:19, 14.99s/it]
 91%|█████████ | 8927/9864 [20:53:18<3:46:07, 14.48s/it]
 91%|█████████ | 8928/9864 [20:53:32<3:44:11, 14.37s/it]
 91%|█████████ | 8929/9864 [20:53:50<3:58:28, 15.30s/it]
 91%|█████████ | 8930/9864 [20:54:04<3:52:28, 14.93s/it]
                                                        
{'loss': '0.05222', 'grad_norm': '5.05', 'learning_rate': '4.676e-07', 'epoch': '0.9054'}

 91%|█████████ | 8930/9864 [20:54:04<3:52:28, 14.93s/it]
 91%|█████████ | 8931/9864 [20:54:19<3:55:00, 15.11s/it]
 91%|█████████ | 8932/9864 [20:54:34<3:50:59, 14.87s/it]
 91%|█████████ | 8933/9864 [20:54:48<3:46:09, 14.58s/it]
 91%|█████████ | 8934/9864 [20:55:03<3:51:03, 14.91s/it]
 91%|█████████ | 8935/9864 [20:55:18<3:47:22, 14.69s/it]
 91%|█████████ | 8936/9864 [20:55:32<3:44:28, 14.51s/it]
 91%|█████████ | 8937/9864 [20:55:49<3:58:27, 15.43s/it]
 91%|█████████ | 8938/9864 [20:56:03<3:49:51, 14.89s/it]
 91%|█████████ | 8939/9864 [20:56:19<3:53:27, 15.14s/it]
 91%|█████████ | 8940/9864 [20:56:33<3:48:36, 14.85s/it]
                                                        
{'loss': '0.06404', 'grad_norm': '5.852', 'learning_rate': '4.577e-07', 'epoch': '0.9064'}

 91%|█████████ | 8940/9864 [20:56:33<3:48:36, 14.85s/it]
 91%|█████████ | 8941/9864 [20:56:47<3:44:54, 14.62s/it]
 91%|█████████ | 8942/9864 [20:57:03<3:51:35, 15.07s/it]
 91%|█████████ | 8943/9864 [20:57:17<3:45:56, 14.72s/it]
 91%|█████████ | 8944/9864 [20:57:31<3:41:49, 14.47s/it]
 91%|█████████ | 8945/9864 [20:57:48<3:56:47, 15.46s/it]
 91%|█████████ | 8946/9864 [20:58:03<3:51:36, 15.14s/it]
 91%|█████████ | 8947/9864 [20:58:19<3:54:59, 15.38s/it]
 91%|█████████ | 8948/9864 [20:58:33<3:48:58, 15.00s/it]
 91%|█████████ | 8949/9864 [20:58:47<3:43:56, 14.69s/it]
 91%|█████████ | 8950/9864 [20:59:02<3:47:31, 14.94s/it]
                                                        
{'loss': '0.04773', 'grad_norm': '4.371', 'learning_rate': '4.479e-07', 'epoch': '0.9074'}

 91%|█████████ | 8950/9864 [20:59:02<3:47:31, 14.94s/it]
 91%|█████████ | 8951/9864 [20:59:16<3:41:35, 14.56s/it]
 91%|█████████ | 8952/9864 [20:59:30<3:38:44, 14.39s/it]
 91%|█████████ | 8953/9864 [20:59:48<3:54:52, 15.47s/it]
 91%|█████████ | 8954/9864 [21:00:02<3:48:46, 15.08s/it]
 91%|█████████ | 8955/9864 [21:00:17<3:48:17, 15.07s/it]
 91%|█████████ | 8956/9864 [21:00:31<3:41:50, 14.66s/it]
 91%|█████████ | 8957/9864 [21:00:47<3:47:29, 15.05s/it]
 91%|█████████ | 8958/9864 [21:01:01<3:44:49, 14.89s/it]
 91%|█████████ | 8959/9864 [21:01:16<3:41:21, 14.68s/it]
 91%|█████████ | 8960/9864 [21:01:30<3:37:45, 14.45s/it]
                                                        
{'loss': '0.05141', 'grad_norm': '5.457', 'learning_rate': '4.383e-07', 'epoch': '0.9084'}

 91%|█████████ | 8960/9864 [21:01:30<3:37:45, 14.45s/it]
 91%|█████████ | 8961/9864 [21:01:45<3:43:26, 14.85s/it]
 91%|█████████ | 8962/9864 [21:02:03<3:54:37, 15.61s/it]
 91%|█████████ | 8963/9864 [21:02:17<3:47:21, 15.14s/it]
 91%|█████████ | 8964/9864 [21:02:31<3:43:24, 14.89s/it]
 91%|█████████ | 8965/9864 [21:02:45<3:40:38, 14.73s/it]
 91%|█████████ | 8966/9864 [21:03:01<3:43:13, 14.92s/it]
 91%|█████████ | 8967/9864 [21:03:15<3:40:32, 14.75s/it]
 91%|█████████ | 8968/9864 [21:03:31<3:45:00, 15.07s/it]
 91%|█████████ | 8969/9864 [21:03:49<3:56:35, 15.86s/it]
 91%|█████████ | 8970/9864 [21:04:03<3:48:43, 15.35s/it]
                                                        
{'loss': '0.05346', 'grad_norm': '5.231', 'learning_rate': '4.287e-07', 'epoch': '0.9094'}

 91%|█████████ | 8970/9864 [21:04:03<3:48:43, 15.35s/it]
 91%|█████████ | 8971/9864 [21:04:16<3:40:10, 14.79s/it]
 91%|█████████ | 8972/9864 [21:04:30<3:35:09, 14.47s/it]
 91%|█████████ | 8973/9864 [21:04:44<3:32:59, 14.34s/it]
 91%|█████████ | 8974/9864 [21:05:00<3:38:22, 14.72s/it]
 91%|█████████ | 8975/9864 [21:05:14<3:35:39, 14.56s/it]
 91%|█████████ | 8976/9864 [21:05:30<3:41:51, 14.99s/it]
 91%|█████████ | 8977/9864 [21:05:45<3:43:26, 15.11s/it]
 91%|█████████ | 8978/9864 [21:06:01<3:46:13, 15.32s/it]
 91%|█████████ | 8979/9864 [21:06:15<3:40:44, 14.96s/it]
 91%|█████████ | 8980/9864 [21:06:29<3:34:33, 14.56s/it]
                                                        
{'loss': '0.04801', 'grad_norm': '3.85', 'learning_rate': '4.192e-07', 'epoch': '0.9105'}

 91%|█████████ | 8980/9864 [21:06:29<3:34:33, 14.56s/it]
 91%|█████████ | 8981/9864 [21:06:45<3:40:19, 14.97s/it]
 91%|█████████ | 8982/9864 [21:06:59<3:36:02, 14.70s/it]
 91%|█████████ | 8983/9864 [21:07:13<3:33:15, 14.52s/it]
 91%|█████████ | 8984/9864 [21:07:28<3:36:33, 14.77s/it]
 91%|█████████ | 8985/9864 [21:07:44<3:42:34, 15.19s/it]
 91%|█████████ | 8986/9864 [21:07:59<3:39:05, 14.97s/it]
 91%|█████████ | 8987/9864 [21:08:15<3:44:05, 15.33s/it]
 91%|█████████ | 8988/9864 [21:08:29<3:38:13, 14.95s/it]
 91%|█████████ | 8989/9864 [21:08:45<3:39:58, 15.08s/it]
 91%|█████████ | 8990/9864 [21:08:59<3:35:32, 14.80s/it]
                                                        
{'loss': '0.05495', 'grad_norm': '5.911', 'learning_rate': '4.099e-07', 'epoch': '0.9115'}

 91%|█████████ | 8990/9864 [21:08:59<3:35:32, 14.80s/it]
 91%|█████████ | 8991/9864 [21:09:14<3:39:29, 15.09s/it]
 91%|█████████ | 8992/9864 [21:09:28<3:33:55, 14.72s/it]
 91%|█████████ | 8993/9864 [21:09:43<3:35:48, 14.87s/it]
 91%|█████████ | 8994/9864 [21:09:58<3:33:45, 14.74s/it]
 91%|█████████ | 8995/9864 [21:10:13<3:36:47, 14.97s/it]
 91%|█████████ | 8996/9864 [21:10:27<3:32:23, 14.68s/it]
 91%|█████████ | 8997/9864 [21:10:41<3:28:46, 14.45s/it]
 91%|█████████ | 8998/9864 [21:10:57<3:32:22, 14.71s/it]
 91%|█████████ | 8999/9864 [21:11:11<3:29:26, 14.53s/it]
 91%|█████████ | 9000/9864 [21:11:27<3:35:53, 14.99s/it]
                                                        
{'loss': '0.05654', 'grad_norm': '5.809', 'learning_rate': '4.006e-07', 'epoch': '0.9125'}

 91%|█████████ | 9000/9864 [21:11:27<3:35:53, 14.99s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.63s/it]
Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.63s/it]
/home/dungpq6/Project/SpatialStack/src/qwen_vl/model/geometry_encoders/vggt_encoder.py:285: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(dtype=dtype):

 91%|█████████▏| 9001/9864 [21:12:45<8:07:05, 33.86s/it]
 91%|█████████▏| 9002/9864 [21:13:01<6:49:52, 28.53s/it]
 91%|█████████▏| 9003/9864 [21:13:15<5:45:47, 24.10s/it]
 91%|█████████▏| 9004/9864 [21:13:28<5:00:33, 20.97s/it]
 91%|█████████▏| 9005/9864 [21:13:44<4:38:03, 19.42s/it]
 91%|█████████▏| 9006/9864 [21:13:58<4:14:04, 17.77s/it]
 91%|█████████▏| 9007/9864 [21:14:12<3:58:23, 16.69s/it]
 91%|█████████▏| 9008/9864 [21:14:28<3:55:57, 16.54s/it]
 91%|█████████▏| 9009/9864 [21:14:44<3:53:50, 16.41s/it]
 91%|█████████▏| 9010/9864 [21:15:00<3:50:38, 16.20s/it]
                                                        
{'loss': '0.04401', 'grad_norm': '4.659', 'learning_rate': '3.915e-07', 'epoch': '0.9135'}

 91%|█████████▏| 9010/9864 [21:15:00<3:50:38, 16.20s/it]
 91%|█████████▏| 9011/9864 [21:15:14<3:41:38, 15.59s/it]
 91%|█████████▏| 9012/9864 [21:15:29<3:35:39, 15.19s/it]
 91%|█████████▏| 9013/9864 [21:15:43<3:31:45, 14.93s/it]
 91%|█████████▏| 9014/9864 [21:15:59<3:34:40, 15.15s/it]
 91%|█████████▏| 9015/9864 [21:16:12<3:27:31, 14.67s/it]
 91%|█████████▏| 9016/9864 [21:16:28<3:33:31, 15.11s/it]
 91%|█████████▏| 9017/9864 [21:16:44<3:35:22, 15.26s/it]
 91%|█████████▏| 9018/9864 [21:17:00<3:38:25, 15.49s/it]
 91%|█████████▏| 9019/9864 [21:17:14<3:31:37, 15.03s/it]
 91%|█████████▏| 9020/9864 [21:17:28<3:28:38, 14.83s/it]
                                                        
{'loss': '0.06076', 'grad_norm': '6.447', 'learning_rate': '3.824e-07', 'epoch': '0.9145'}

 91%|█████████▏| 9020/9864 [21:17:28<3:28:38, 14.83s/it]
 91%|█████████▏| 9021/9864 [21:17:44<3:33:06, 15.17s/it]
 91%|█████████▏| 9022/9864 [21:17:58<3:28:50, 14.88s/it]
 91%|█████████▏| 9023/9864 [21:18:12<3:24:58, 14.62s/it]
 91%|█████████▏| 9024/9864 [21:18:28<3:28:30, 14.89s/it]
 91%|█████████▏| 9025/9864 [21:18:44<3:32:02, 15.16s/it]
 92%|█████████▏| 9026/9864 [21:18:59<3:34:08, 15.33s/it]
 92%|█████████▏| 9027/9864 [21:19:14<3:29:57, 15.05s/it]
 92%|█████████▏| 9028/9864 [21:19:28<3:25:40, 14.76s/it]
 92%|█████████▏| 9029/9864 [21:19:44<3:29:11, 15.03s/it]
 92%|█████████▏| 9030/9864 [21:19:58<3:25:04, 14.75s/it]
                                                        
{'loss': '0.04889', 'grad_norm': '5.631', 'learning_rate': '3.735e-07', 'epoch': '0.9155'}

 92%|█████████▏| 9030/9864 [21:19:58<3:25:04, 14.75s/it]
 92%|█████████▏| 9031/9864 [21:20:14<3:31:36, 15.24s/it]
 92%|█████████▏| 9032/9864 [21:20:28<3:26:23, 14.88s/it]
 92%|█████████▏| 9033/9864 [21:20:46<3:37:17, 15.69s/it]
 92%|█████████▏| 9034/9864 [21:21:00<3:30:10, 15.19s/it]
 92%|█████████▏| 9035/9864 [21:21:13<3:23:53, 14.76s/it]
 92%|█████████▏| 9036/9864 [21:21:28<3:21:03, 14.57s/it]
 92%|█████████▏| 9037/9864 [21:21:43<3:24:37, 14.85s/it]
 92%|█████████▏| 9038/9864 [21:21:59<3:27:10, 15.05s/it]
 92%|█████████▏| 9039/9864 [21:22:13<3:22:18, 14.71s/it]
 92%|█████████▏| 9040/9864 [21:22:30<3:32:13, 15.45s/it]
                                                        
{'loss': '0.05445', 'grad_norm': '3.404', 'learning_rate': '3.647e-07', 'epoch': '0.9165'}

 92%|█████████▏| 9040/9864 [21:22:30<3:32:13, 15.45s/it]
 92%|█████████▏| 9041/9864 [21:22:44<3:25:58, 15.02s/it]
 92%|█████████▏| 9042/9864 [21:22:57<3:20:23, 14.63s/it]
 92%|█████████▏| 9043/9864 [21:23:12<3:19:06, 14.55s/it]
 92%|█████████▏| 9044/9864 [21:23:26<3:17:57, 14.48s/it]
 92%|█████████▏| 9045/9864 [21:23:42<3:23:18, 14.89s/it]
 92%|█████████▏| 9046/9864 [21:23:58<3:27:52, 15.25s/it]
 92%|█████████▏| 9047/9864 [21:24:12<3:23:49, 14.97s/it]
 92%|█████████▏| 9048/9864 [21:24:30<3:34:51, 15.80s/it]
 92%|█████████▏| 9049/9864 [21:24:45<3:29:44, 15.44s/it]
 92%|█████████▏| 9050/9864 [21:24:59<3:23:21, 14.99s/it]
                                                        
{'loss': '0.05159', 'grad_norm': '5.929', 'learning_rate': '3.559e-07', 'epoch': '0.9176'}

 92%|█████████▏| 9050/9864 [21:24:59<3:23:21, 14.99s/it]
 92%|█████████▏| 9051/9864 [21:25:12<3:17:52, 14.60s/it]
 92%|█████████▏| 9052/9864 [21:25:26<3:15:16, 14.43s/it]
 92%|█████████▏| 9053/9864 [21:25:42<3:19:58, 14.79s/it]
 92%|█████████▏| 9054/9864 [21:25:58<3:23:54, 15.10s/it]
 92%|█████████▏| 9055/9864 [21:26:12<3:20:28, 14.87s/it]
 92%|█████████▏| 9056/9864 [21:26:28<3:24:22, 15.18s/it]
 92%|█████████▏| 9057/9864 [21:26:45<3:29:33, 15.58s/it]
 92%|█████████▏| 9058/9864 [21:26:59<3:25:02, 15.26s/it]
 92%|█████████▏| 9059/9864 [21:27:13<3:17:41, 14.73s/it]
 92%|█████████▏| 9060/9864 [21:27:28<3:21:58, 15.07s/it]
                                                        
{'loss': '0.04747', 'grad_norm': '4.313', 'learning_rate': '3.473e-07', 'epoch': '0.9186'}

 92%|█████████▏| 9060/9864 [21:27:28<3:21:58, 15.07s/it]
 92%|█████████▏| 9061/9864 [21:27:42<3:17:30, 14.76s/it]
 92%|█████████▏| 9062/9864 [21:27:59<3:22:27, 15.15s/it]
 92%|█████████▏| 9063/9864 [21:28:13<3:17:41, 14.81s/it]
 92%|█████████▏| 9064/9864 [21:28:30<3:27:16, 15.55s/it]
 92%|█████████▏| 9065/9864 [21:28:44<3:20:28, 15.05s/it]
 92%|█████████▏| 9066/9864 [21:28:58<3:16:31, 14.78s/it]
 92%|█████████▏| 9067/9864 [21:29:12<3:12:42, 14.51s/it]
 92%|█████████▏| 9068/9864 [21:29:28<3:17:52, 14.91s/it]
 92%|█████████▏| 9069/9864 [21:29:43<3:19:35, 15.06s/it]
 92%|█████████▏| 9070/9864 [21:29:57<3:16:18, 14.83s/it]
                                                        
{'loss': '0.05801', 'grad_norm': '4.172', 'learning_rate': '3.388e-07', 'epoch': '0.9196'}

 92%|█████████▏| 9070/9864 [21:29:57<3:16:18, 14.83s/it]
 92%|█████████▏| 9071/9864 [21:30:11<3:12:53, 14.59s/it]
 92%|█████████▏| 9072/9864 [21:30:27<3:18:05, 15.01s/it]
 92%|█████████▏| 9073/9864 [21:30:43<3:21:38, 15.30s/it]
 92%|█████████▏| 9074/9864 [21:30:57<3:15:51, 14.88s/it]
 92%|█████████▏| 9075/9864 [21:31:12<3:13:31, 14.72s/it]
 92%|█████████▏| 9076/9864 [21:31:27<3:17:24, 15.03s/it]
 92%|█████████▏| 9077/9864 [21:31:43<3:20:04, 15.25s/it]
 92%|█████████▏| 9078/9864 [21:31:57<3:14:43, 14.86s/it]
 92%|█████████▏| 9079/9864 [21:32:11<3:11:09, 14.61s/it]
 92%|█████████▏| 9080/9864 [21:32:27<3:16:13, 15.02s/it]
                                                        
{'loss': '0.04725', 'grad_norm': '3.254', 'learning_rate': '3.303e-07', 'epoch': '0.9206'}

 92%|█████████▏| 9080/9864 [21:32:27<3:16:13, 15.02s/it]
 92%|█████████▏| 9081/9864 [21:32:43<3:18:41, 15.23s/it]
 92%|█████████▏| 9082/9864 [21:32:57<3:13:51, 14.87s/it]
 92%|█████████▏| 9083/9864 [21:33:11<3:11:18, 14.70s/it]
 92%|█████████▏| 9084/9864 [21:33:27<3:14:53, 14.99s/it]
 92%|█████████▏| 9085/9864 [21:33:41<3:11:25, 14.74s/it]
 92%|█████████▏| 9086/9864 [21:33:57<3:16:25, 15.15s/it]
 92%|█████████▏| 9087/9864 [21:34:12<3:17:30, 15.25s/it]
 92%|█████████▏| 9088/9864 [21:34:26<3:12:12, 14.86s/it]
 92%|█████████▏| 9089/9864 [21:34:43<3:17:01, 15.25s/it]
 92%|█████████▏| 9090/9864 [21:34:57<3:13:02, 14.96s/it]
                                                        
{'loss': '0.04978', 'grad_norm': '5.638', 'learning_rate': '3.22e-07', 'epoch': '0.9216'}

 92%|█████████▏| 9090/9864 [21:34:57<3:13:02, 14.96s/it]
 92%|█████████▏| 9091/9864 [21:35:11<3:09:46, 14.73s/it]
 92%|█████████▏| 9092/9864 [21:35:27<3:14:39, 15.13s/it]
 92%|█████████▏| 9093/9864 [21:35:43<3:15:33, 15.22s/it]
 92%|█████████▏| 9094/9864 [21:35:57<3:10:30, 14.85s/it]
 92%|█████████▏| 9095/9864 [21:36:12<3:13:45, 15.12s/it]
 92%|█████████▏| 9096/9864 [21:36:26<3:09:51, 14.83s/it]
 92%|█████████▏| 9097/9864 [21:36:42<3:11:36, 14.99s/it]
 92%|█████████▏| 9098/9864 [21:36:56<3:08:59, 14.80s/it]
 92%|█████████▏| 9099/9864 [21:37:10<3:04:42, 14.49s/it]
 92%|█████████▏| 9100/9864 [21:37:28<3:17:42, 15.53s/it]
                                                        
{'loss': '0.05607', 'grad_norm': '7.482', 'learning_rate': '3.138e-07', 'epoch': '0.9226'}

 92%|█████████▏| 9100/9864 [21:37:28<3:17:42, 15.53s/it]
 92%|█████████▏| 9101/9864 [21:37:42<3:10:10, 14.96s/it]
 92%|█████████▏| 9102/9864 [21:37:56<3:08:20, 14.83s/it]
 92%|█████████▏| 9103/9864 [21:38:10<3:06:24, 14.70s/it]
 92%|█████████▏| 9104/9864 [21:38:26<3:09:11, 14.94s/it]
 92%|█████████▏| 9105/9864 [21:38:40<3:05:09, 14.64s/it]
 92%|█████████▏| 9106/9864 [21:38:56<3:08:44, 14.94s/it]
 92%|█████████▏| 9107/9864 [21:39:13<3:17:02, 15.62s/it]
 92%|█████████▏| 9108/9864 [21:39:26<3:08:10, 14.93s/it]
 92%|█████████▏| 9109/9864 [21:39:40<3:04:37, 14.67s/it]
 92%|█████████▏| 9110/9864 [21:39:55<3:04:09, 14.65s/it]
                                                        
{'loss': '0.04034', 'grad_norm': '4.154', 'learning_rate': '3.057e-07', 'epoch': '0.9236'}

 92%|█████████▏| 9110/9864 [21:39:55<3:04:09, 14.65s/it]
 92%|█████████▏| 9111/9864 [21:40:10<3:08:00, 14.98s/it]
 92%|█████████▏| 9112/9864 [21:40:25<3:04:27, 14.72s/it]
 92%|█████████▏| 9113/9864 [21:40:39<3:02:24, 14.57s/it]
 92%|█████████▏| 9114/9864 [21:40:54<3:06:12, 14.90s/it]
 92%|█████████▏| 9115/9864 [21:41:10<3:08:47, 15.12s/it]
 92%|█████████▏| 9116/9864 [21:41:26<3:11:23, 15.35s/it]
 92%|█████████▏| 9117/9864 [21:41:40<3:06:10, 14.95s/it]
 92%|█████████▏| 9118/9864 [21:41:54<3:01:51, 14.63s/it]
 92%|█████████▏| 9119/9864 [21:42:08<3:00:17, 14.52s/it]
 92%|█████████▏| 9120/9864 [21:42:24<3:04:46, 14.90s/it]
                                                        
{'loss': '0.04679', 'grad_norm': '3.861', 'learning_rate': '2.977e-07', 'epoch': '0.9247'}

 92%|█████████▏| 9120/9864 [21:42:24<3:04:46, 14.90s/it]
 92%|█████████▏| 9121/9864 [21:42:38<3:00:47, 14.60s/it]
 92%|█████████▏| 9122/9864 [21:42:54<3:04:56, 14.95s/it]
 92%|█████████▏| 9123/9864 [21:43:11<3:14:51, 15.78s/it]
 92%|█████████▏| 9124/9864 [21:43:25<3:08:30, 15.28s/it]
 93%|█████████▎| 9125/9864 [21:43:40<3:04:33, 14.99s/it]
 93%|█████████▎| 9126/9864 [21:43:54<3:01:25, 14.75s/it]
 93%|█████████▎| 9127/9864 [21:44:10<3:05:18, 15.09s/it]
 93%|█████████▎| 9128/9864 [21:44:24<3:02:22, 14.87s/it]
 93%|█████████▎| 9129/9864 [21:44:40<3:04:49, 15.09s/it]
 93%|█████████▎| 9130/9864 [21:44:53<2:59:07, 14.64s/it]
                                                        
{'loss': '0.04509', 'grad_norm': '3.25', 'learning_rate': '2.898e-07', 'epoch': '0.9257'}

 93%|█████████▎| 9130/9864 [21:44:53<2:59:07, 14.64s/it]
 93%|█████████▎| 9131/9864 [21:45:10<3:06:22, 15.26s/it]
 93%|█████████▎| 9132/9864 [21:45:26<3:09:14, 15.51s/it]
 93%|█████████▎| 9133/9864 [21:45:40<3:03:54, 15.09s/it]
 93%|█████████▎| 9134/9864 [21:45:55<3:00:24, 14.83s/it]
 93%|█████████▎| 9135/9864 [21:46:09<2:57:23, 14.60s/it]
 93%|█████████▎| 9136/9864 [21:46:26<3:07:44, 15.47s/it]
 93%|█████████▎| 9137/9864 [21:46:41<3:03:59, 15.18s/it]
 93%|█████████▎| 9138/9864 [21:46:54<2:57:35, 14.68s/it]
 93%|█████████▎| 9139/9864 [21:47:11<3:06:47, 15.46s/it]
 93%|█████████▎| 9140/9864 [21:47:25<3:01:28, 15.04s/it]
                                                        
{'loss': '0.04466', 'grad_norm': '5.511', 'learning_rate': '2.82e-07', 'epoch': '0.9267'}

 93%|█████████▎| 9140/9864 [21:47:25<3:01:28, 15.04s/it]
 93%|█████████▎| 9141/9864 [21:47:40<2:58:09, 14.79s/it]
 93%|█████████▎| 9142/9864 [21:47:54<2:55:01, 14.55s/it]
 93%|█████████▎| 9143/9864 [21:48:09<2:57:22, 14.76s/it]
 93%|█████████▎| 9144/9864 [21:48:25<3:02:42, 15.23s/it]
 93%|█████████▎| 9145/9864 [21:48:40<3:00:12, 15.04s/it]
 93%|█████████▎| 9146/9864 [21:48:55<3:01:37, 15.18s/it]
 93%|█████████▎| 9147/9864 [21:49:11<3:02:56, 15.31s/it]
 93%|█████████▎| 9148/9864 [21:49:25<2:57:55, 14.91s/it]
 93%|█████████▎| 9149/9864 [21:49:39<2:54:19, 14.63s/it]
 93%|█████████▎| 9150/9864 [21:49:53<2:51:08, 14.38s/it]
                                                        
{'loss': '0.0456', 'grad_norm': '4.275', 'learning_rate': '2.743e-07', 'epoch': '0.9277'}

 93%|█████████▎| 9150/9864 [21:49:53<2:51:08, 14.38s/it]
 93%|█████████▎| 9151/9864 [21:50:07<2:50:41, 14.36s/it]
 93%|█████████▎| 9152/9864 [21:50:24<3:01:20, 15.28s/it]
 93%|█████████▎| 9153/9864 [21:50:39<2:57:27, 14.98s/it]
 93%|█████████▎| 9154/9864 [21:50:55<3:00:54, 15.29s/it]
 93%|█████████▎| 9155/9864 [21:51:11<3:03:25, 15.52s/it]
 93%|█████████▎| 9156/9864 [21:51:25<2:59:01, 15.17s/it]
 93%|█████████▎| 9157/9864 [21:51:39<2:52:49, 14.67s/it]
 93%|█████████▎| 9158/9864 [21:51:53<2:50:29, 14.49s/it]
 93%|█████████▎| 9159/9864 [21:52:07<2:48:18, 14.32s/it]
 93%|█████████▎| 9160/9864 [21:52:24<2:59:01, 15.26s/it]
                                                        
{'loss': '0.04157', 'grad_norm': '4.98', 'learning_rate': '2.667e-07', 'epoch': '0.9287'}

 93%|█████████▎| 9160/9864 [21:52:24<2:59:01, 15.26s/it]
 93%|█████████▎| 9161/9864 [21:52:38<2:52:27, 14.72s/it]
 93%|█████████▎| 9162/9864 [21:52:51<2:49:22, 14.48s/it]
 93%|█████████▎| 9163/9864 [21:53:09<2:59:05, 15.33s/it]
 93%|█████████▎| 9164/9864 [21:53:23<2:55:30, 15.04s/it]
 93%|█████████▎| 9165/9864 [21:53:37<2:52:37, 14.82s/it]
 93%|█████████▎| 9166/9864 [21:53:52<2:50:11, 14.63s/it]
 93%|█████████▎| 9167/9864 [21:54:07<2:54:00, 14.98s/it]
 93%|█████████▎| 9168/9864 [21:54:23<2:56:56, 15.25s/it]
 93%|█████████▎| 9169/9864 [21:54:37<2:52:32, 14.90s/it]
 93%|█████████▎| 9170/9864 [21:54:53<2:54:34, 15.09s/it]
                                                        
{'loss': '0.05791', 'grad_norm': '5.259', 'learning_rate': '2.592e-07', 'epoch': '0.9297'}

 93%|█████████▎| 9170/9864 [21:54:53<2:54:34, 15.09s/it]
 93%|█████████▎| 9171/9864 [21:55:08<2:55:07, 15.16s/it]
 93%|█████████▎| 9172/9864 [21:55:22<2:51:06, 14.84s/it]
 93%|█████████▎| 9173/9864 [21:55:37<2:49:17, 14.70s/it]
 93%|█████████▎| 9174/9864 [21:55:50<2:45:43, 14.41s/it]
 93%|█████████▎| 9175/9864 [21:56:08<2:55:56, 15.32s/it]
 93%|█████████▎| 9176/9864 [21:56:22<2:52:29, 15.04s/it]
 93%|█████████▎| 9177/9864 [21:56:36<2:48:55, 14.75s/it]
 93%|█████████▎| 9178/9864 [21:56:54<2:59:12, 15.67s/it]
 93%|█████████▎| 9179/9864 [21:57:08<2:54:01, 15.24s/it]
 93%|█████████▎| 9180/9864 [21:57:23<2:50:05, 14.92s/it]
                                                        
{'loss': '0.04295', 'grad_norm': '7.149', 'learning_rate': '2.519e-07', 'epoch': '0.9307'}

 93%|█████████▎| 9180/9864 [21:57:23<2:50:05, 14.92s/it]
 93%|█████████▎| 9181/9864 [21:57:37<2:48:45, 14.82s/it]
 93%|█████████▎| 9182/9864 [21:57:53<2:52:57, 15.22s/it]
 93%|█████████▎| 9183/9864 [21:58:09<2:53:42, 15.30s/it]
 93%|█████████▎| 9184/9864 [21:58:23<2:50:05, 15.01s/it]
 93%|█████████▎| 9185/9864 [21:58:39<2:53:38, 15.34s/it]
 93%|█████████▎| 9186/9864 [21:58:54<2:51:31, 15.18s/it]
 93%|█████████▎| 9187/9864 [21:59:08<2:47:11, 14.82s/it]
 93%|█████████▎| 9188/9864 [21:59:22<2:44:27, 14.60s/it]
 93%|█████████▎| 9189/9864 [21:59:38<2:47:18, 14.87s/it]
 93%|█████████▎| 9190/9864 [21:59:53<2:49:53, 15.12s/it]
                                                        
{'loss': '0.04125', 'grad_norm': '3.685', 'learning_rate': '2.446e-07', 'epoch': '0.9317'}

 93%|█████████▎| 9190/9864 [21:59:53<2:49:53, 15.12s/it]
 93%|█████████▎| 9191/9864 [22:00:07<2:45:57, 14.80s/it]
 93%|█████████▎| 9192/9864 [22:00:23<2:49:54, 15.17s/it]
 93%|█████████▎| 9193/9864 [22:00:39<2:52:23, 15.42s/it]
 93%|█████████▎| 9194/9864 [22:00:53<2:46:09, 14.88s/it]
 93%|█████████▎| 9195/9864 [22:01:07<2:44:17, 14.73s/it]
 93%|█████████▎| 9196/9864 [22:01:22<2:42:10, 14.57s/it]
 93%|█████████▎| 9197/9864 [22:01:37<2:44:41, 14.82s/it]
 93%|█████████▎| 9198/9864 [22:01:53<2:48:23, 15.17s/it]
 93%|█████████▎| 9199/9864 [22:02:07<2:45:08, 14.90s/it]
 93%|█████████▎| 9200/9864 [22:02:23<2:48:02, 15.18s/it]
                                                        
{'loss': '0.05577', 'grad_norm': '6.549', 'learning_rate': '2.374e-07', 'epoch': '0.9328'}

 93%|█████████▎| 9200/9864 [22:02:23<2:48:02, 15.18s/it]
 93%|█████████▎| 9201/9864 [22:02:40<2:52:04, 15.57s/it]
 93%|█████████▎| 9202/9864 [22:02:54<2:47:48, 15.21s/it]
 93%|█████████▎| 9203/9864 [22:03:08<2:44:05, 14.89s/it]
 93%|█████████▎| 9204/9864 [22:03:22<2:41:04, 14.64s/it]
 93%|█████████▎| 9205/9864 [22:03:38<2:44:28, 14.97s/it]
 93%|█████████▎| 9206/9864 [22:03:54<2:46:35, 15.19s/it]
 93%|█████████▎| 9207/9864 [22:04:09<2:47:52, 15.33s/it]
 93%|█████████▎| 9208/9864 [22:04:24<2:44:21, 15.03s/it]
 93%|█████████▎| 9209/9864 [22:04:39<2:45:58, 15.20s/it]
 93%|█████████▎| 9210/9864 [22:04:53<2:42:03, 14.87s/it]
                                                        
{'loss': '0.03894', 'grad_norm': '3.03', 'learning_rate': '2.304e-07', 'epoch': '0.9338'}

 93%|█████████▎| 9210/9864 [22:04:53<2:42:03, 14.87s/it]
 93%|█████████▎| 9211/9864 [22:05:07<2:38:47, 14.59s/it]
 93%|█████████▎| 9212/9864 [22:05:21<2:35:59, 14.36s/it]
 93%|█████████▎| 9213/9864 [22:05:37<2:40:00, 14.75s/it]
 93%|█████████▎| 9214/9864 [22:05:53<2:43:59, 15.14s/it]
 93%|█████████▎| 9215/9864 [22:06:09<2:46:43, 15.41s/it]
 93%|█████████▎| 9216/9864 [22:06:22<2:40:37, 14.87s/it]
 93%|█████████▎| 9217/9864 [22:06:38<2:42:52, 15.11s/it]
 93%|█████████▎| 9218/9864 [22:06:52<2:38:23, 14.71s/it]
 93%|█████████▎| 9219/9864 [22:07:06<2:36:03, 14.52s/it]
 93%|█████████▎| 9220/9864 [22:07:20<2:35:08, 14.45s/it]
                                                        
{'loss': '0.04275', 'grad_norm': '6.036', 'learning_rate': '2.234e-07', 'epoch': '0.9348'}

 93%|█████████▎| 9220/9864 [22:07:20<2:35:08, 14.45s/it]
 93%|█████████▎| 9221/9864 [22:07:37<2:43:05, 15.22s/it]
 93%|█████████▎| 9222/9864 [22:07:52<2:41:09, 15.06s/it]
 94%|█████████▎| 9223/9864 [22:08:06<2:38:30, 14.84s/it]
 94%|█████████▎| 9224/9864 [22:08:22<2:40:14, 15.02s/it]
 94%|█████████▎| 9225/9864 [22:08:37<2:42:22, 15.25s/it]
 94%|█████████▎| 9226/9864 [22:08:51<2:37:24, 14.80s/it]
 94%|█████████▎| 9227/9864 [22:09:05<2:33:57, 14.50s/it]
 94%|█████████▎| 9228/9864 [22:09:19<2:33:00, 14.43s/it]
 94%|█████████▎| 9229/9864 [22:09:36<2:41:15, 15.24s/it]
 94%|█████████▎| 9230/9864 [22:09:50<2:36:25, 14.80s/it]
                                                        
{'loss': '0.04206', 'grad_norm': '7.52', 'learning_rate': '2.166e-07', 'epoch': '0.9358'}

 94%|█████████▎| 9230/9864 [22:09:50<2:36:25, 14.80s/it]
 94%|█████████▎| 9231/9864 [22:10:04<2:33:22, 14.54s/it]
 94%|█████████▎| 9232/9864 [22:10:20<2:37:03, 14.91s/it]
 94%|█████████▎| 9233/9864 [22:10:36<2:40:13, 15.23s/it]
 94%|█████████▎| 9234/9864 [22:10:50<2:36:53, 14.94s/it]
 94%|█████████▎| 9235/9864 [22:11:04<2:32:51, 14.58s/it]
 94%|█████████▎| 9236/9864 [22:11:20<2:37:07, 15.01s/it]
 94%|█████████▎| 9237/9864 [22:11:34<2:34:18, 14.77s/it]
 94%|█████████▎| 9238/9864 [22:11:50<2:36:12, 14.97s/it]
 94%|█████████▎| 9239/9864 [22:12:04<2:33:18, 14.72s/it]
 94%|█████████▎| 9240/9864 [22:12:20<2:36:39, 15.06s/it]
                                                        
{'loss': '0.04406', 'grad_norm': '5.112', 'learning_rate': '2.098e-07', 'epoch': '0.9368'}

 94%|█████████▎| 9240/9864 [22:12:20<2:36:39, 15.06s/it]
 94%|█████████▎| 9241/9864 [22:12:35<2:38:00, 15.22s/it]
 94%|█████████▎| 9242/9864 [22:12:49<2:32:32, 14.72s/it]
 94%|█████████▎| 9243/9864 [22:13:04<2:35:42, 15.04s/it]
 94%|█████████▎| 9244/9864 [22:13:19<2:32:20, 14.74s/it]
 94%|█████████▎| 9245/9864 [22:13:34<2:35:51, 15.11s/it]
 94%|█████████▎| 9246/9864 [22:13:48<2:32:15, 14.78s/it]
 94%|█████████▎| 9247/9864 [22:14:03<2:29:54, 14.58s/it]
 94%|█████████▍| 9248/9864 [22:14:17<2:30:02, 14.62s/it]
 94%|█████████▍| 9249/9864 [22:14:35<2:38:00, 15.42s/it]
 94%|█████████▍| 9250/9864 [22:14:49<2:33:41, 15.02s/it]
                                                        
{'loss': '0.04169', 'grad_norm': '4.112', 'learning_rate': '2.032e-07', 'epoch': '0.9378'}

 94%|█████████▍| 9250/9864 [22:14:49<2:33:41, 15.02s/it]
 94%|█████████▍| 9251/9864 [22:15:03<2:32:19, 14.91s/it]
 94%|█████████▍| 9252/9864 [22:15:20<2:36:02, 15.30s/it]
 94%|█████████▍| 9253/9864 [22:15:35<2:36:40, 15.39s/it]
 94%|█████████▍| 9254/9864 [22:15:49<2:31:17, 14.88s/it]
 94%|█████████▍| 9255/9864 [22:16:03<2:28:12, 14.60s/it]
 94%|█████████▍| 9256/9864 [22:16:19<2:32:20, 15.03s/it]
 94%|█████████▍| 9257/9864 [22:16:34<2:32:43, 15.10s/it]
 94%|█████████▍| 9258/9864 [22:16:47<2:26:56, 14.55s/it]
 94%|█████████▍| 9259/9864 [22:17:03<2:30:33, 14.93s/it]
 94%|█████████▍| 9260/9864 [22:17:18<2:28:32, 14.76s/it]
                                                        
{'loss': '0.04417', 'grad_norm': '3.005', 'learning_rate': '1.967e-07', 'epoch': '0.9388'}

 94%|█████████▍| 9260/9864 [22:17:18<2:28:32, 14.76s/it]
 94%|█████████▍| 9261/9864 [22:17:33<2:31:47, 15.10s/it]
 94%|█████████▍| 9262/9864 [22:17:47<2:27:32, 14.71s/it]
 94%|█████████▍| 9263/9864 [22:18:02<2:26:33, 14.63s/it]
 94%|█████████▍| 9264/9864 [22:18:16<2:24:43, 14.47s/it]
 94%|█████████▍| 9265/9864 [22:18:34<2:34:26, 15.47s/it]
 94%|█████████▍| 9266/9864 [22:18:48<2:29:37, 15.01s/it]
 94%|█████████▍| 9267/9864 [22:19:03<2:30:58, 15.17s/it]
 94%|█████████▍| 9268/9864 [22:19:19<2:32:59, 15.40s/it]
 94%|█████████▍| 9269/9864 [22:19:34<2:30:22, 15.16s/it]
 94%|█████████▍| 9270/9864 [22:19:48<2:26:47, 14.83s/it]
                                                        
{'loss': '0.04947', 'grad_norm': '3.966', 'learning_rate': '1.902e-07', 'epoch': '0.9399'}

 94%|█████████▍| 9270/9864 [22:19:48<2:26:47, 14.83s/it]
 94%|█████████▍| 9271/9864 [22:20:02<2:25:20, 14.71s/it]
 94%|█████████▍| 9272/9864 [22:20:20<2:34:08, 15.62s/it]
 94%|█████████▍| 9273/9864 [22:20:34<2:29:22, 15.17s/it]
 94%|█████████▍| 9274/9864 [22:20:49<2:29:28, 15.20s/it]
 94%|█████████▍| 9275/9864 [22:21:03<2:24:36, 14.73s/it]
 94%|█████████▍| 9276/9864 [22:21:18<2:26:57, 15.00s/it]
 94%|█████████▍| 9277/9864 [22:21:32<2:23:37, 14.68s/it]
 94%|█████████▍| 9278/9864 [22:21:47<2:22:22, 14.58s/it]
 94%|█████████▍| 9279/9864 [22:22:01<2:21:09, 14.48s/it]
 94%|█████████▍| 9280/9864 [22:22:18<2:29:10, 15.33s/it]
                                                        
{'loss': '0.04448', 'grad_norm': '4.791', 'learning_rate': '1.839e-07', 'epoch': '0.9409'}

 94%|█████████▍| 9280/9864 [22:22:18<2:29:10, 15.33s/it]
 94%|█████████▍| 9281/9864 [22:22:33<2:25:58, 15.02s/it]
 94%|█████████▍| 9282/9864 [22:22:48<2:27:13, 15.18s/it]
 94%|█████████▍| 9283/9864 [22:23:04<2:29:37, 15.45s/it]
 94%|█████████▍| 9284/9864 [22:23:18<2:25:02, 15.00s/it]
 94%|█████████▍| 9285/9864 [22:23:32<2:20:46, 14.59s/it]
 94%|█████████▍| 9286/9864 [22:23:46<2:19:42, 14.50s/it]
 94%|█████████▍| 9287/9864 [22:24:02<2:22:42, 14.84s/it]
 94%|█████████▍| 9288/9864 [22:24:17<2:24:57, 15.10s/it]
 94%|█████████▍| 9289/9864 [22:24:32<2:22:06, 14.83s/it]
 94%|█████████▍| 9290/9864 [22:24:47<2:24:09, 15.07s/it]
                                                        
{'loss': '0.04987', 'grad_norm': '7.459', 'learning_rate': '1.777e-07', 'epoch': '0.9419'}

 94%|█████████▍| 9290/9864 [22:24:47<2:24:09, 15.07s/it]
 94%|█████████▍| 9291/9864 [22:25:01<2:21:07, 14.78s/it]
 94%|█████████▍| 9292/9864 [22:25:17<2:23:58, 15.10s/it]
 94%|█████████▍| 9293/9864 [22:25:31<2:19:46, 14.69s/it]
 94%|█████████▍| 9294/9864 [22:25:45<2:18:33, 14.58s/it]
 94%|█████████▍| 9295/9864 [22:26:03<2:27:24, 15.54s/it]
 94%|█████████▍| 9296/9864 [22:26:18<2:24:03, 15.22s/it]
 94%|█████████▍| 9297/9864 [22:26:32<2:20:17, 14.85s/it]
 94%|█████████▍| 9298/9864 [22:26:47<2:22:06, 15.07s/it]
 94%|█████████▍| 9299/9864 [22:27:01<2:18:38, 14.72s/it]
 94%|█████████▍| 9300/9864 [22:27:17<2:21:12, 15.02s/it]
                                                        
{'loss': '0.04031', 'grad_norm': '4.107', 'learning_rate': '1.716e-07', 'epoch': '0.9429'}

 94%|█████████▍| 9300/9864 [22:27:17<2:21:12, 15.02s/it]
 94%|█████████▍| 9301/9864 [22:27:31<2:19:01, 14.82s/it]
 94%|█████████▍| 9302/9864 [22:27:47<2:22:27, 15.21s/it]
 94%|█████████▍| 9303/9864 [22:28:03<2:23:41, 15.37s/it]
 94%|█████████▍| 9304/9864 [22:28:17<2:20:51, 15.09s/it]
 94%|█████████▍| 9305/9864 [22:28:32<2:18:16, 14.84s/it]
 94%|█████████▍| 9306/9864 [22:28:47<2:20:03, 15.06s/it]
 94%|█████████▍| 9307/9864 [22:29:03<2:22:22, 15.34s/it]
 94%|█████████▍| 9308/9864 [22:29:17<2:18:35, 14.96s/it]
 94%|█████████▍| 9309/9864 [22:29:31<2:15:26, 14.64s/it]
 94%|█████████▍| 9310/9864 [22:29:49<2:23:41, 15.56s/it]
                                                        
{'loss': '0.04647', 'grad_norm': '6.182', 'learning_rate': '1.656e-07', 'epoch': '0.9439'}

 94%|█████████▍| 9310/9864 [22:29:49<2:23:41, 15.56s/it]
 94%|█████████▍| 9311/9864 [22:30:03<2:18:34, 15.04s/it]
 94%|█████████▍| 9312/9864 [22:30:17<2:16:43, 14.86s/it]
 94%|█████████▍| 9313/9864 [22:30:32<2:15:58, 14.81s/it]
 94%|█████████▍| 9314/9864 [22:30:48<2:18:33, 15.11s/it]
 94%|█████████▍| 9315/9864 [22:31:03<2:20:08, 15.32s/it]
 94%|█████████▍| 9316/9864 [22:31:18<2:17:10, 15.02s/it]
 94%|█████████▍| 9317/9864 [22:31:34<2:20:05, 15.37s/it]
 94%|█████████▍| 9318/9864 [22:31:49<2:19:46, 15.36s/it]
 94%|█████████▍| 9319/9864 [22:32:04<2:16:49, 15.06s/it]
 94%|█████████▍| 9320/9864 [22:32:17<2:12:56, 14.66s/it]
                                                        
{'loss': '0.04714', 'grad_norm': '5.014', 'learning_rate': '1.597e-07', 'epoch': '0.9449'}

 94%|█████████▍| 9320/9864 [22:32:17<2:12:56, 14.66s/it]
 94%|█████████▍| 9321/9864 [22:32:32<2:11:48, 14.56s/it]
 95%|█████████▍| 9322/9864 [22:32:48<2:15:18, 14.98s/it]
 95%|█████████▍| 9323/9864 [22:33:04<2:17:42, 15.27s/it]
 95%|█████████▍| 9324/9864 [22:33:18<2:15:34, 15.06s/it]
 95%|█████████▍| 9325/9864 [22:33:36<2:21:56, 15.80s/it]
 95%|█████████▍| 9326/9864 [22:33:50<2:17:55, 15.38s/it]
 95%|█████████▍| 9327/9864 [22:34:04<2:13:03, 14.87s/it]
 95%|█████████▍| 9328/9864 [22:34:18<2:11:15, 14.69s/it]
 95%|█████████▍| 9329/9864 [22:34:34<2:15:30, 15.20s/it]
 95%|█████████▍| 9330/9864 [22:34:50<2:17:15, 15.42s/it]
                                                        
{'loss': '0.04368', 'grad_norm': '8.612', 'learning_rate': '1.539e-07', 'epoch': '0.9459'}

 95%|█████████▍| 9330/9864 [22:34:50<2:17:15, 15.42s/it]
 95%|█████████▍| 9331/9864 [22:35:04<2:13:25, 15.02s/it]
 95%|█████████▍| 9332/9864 [22:35:20<2:15:10, 15.24s/it]
 95%|█████████▍| 9333/9864 [22:35:36<2:17:19, 15.52s/it]
 95%|█████████▍| 9334/9864 [22:35:51<2:14:28, 15.22s/it]
 95%|█████████▍| 9335/9864 [22:36:05<2:11:06, 14.87s/it]
 95%|█████████▍| 9336/9864 [22:36:19<2:07:54, 14.54s/it]
 95%|█████████▍| 9337/9864 [22:36:34<2:10:48, 14.89s/it]
 95%|█████████▍| 9338/9864 [22:36:50<2:12:28, 15.11s/it]
 95%|█████████▍| 9339/9864 [22:37:04<2:09:17, 14.78s/it]
 95%|█████████▍| 9340/9864 [22:37:21<2:14:24, 15.39s/it]
                                                        
{'loss': '0.04683', 'grad_norm': '4.235', 'learning_rate': '1.482e-07', 'epoch': '0.947'}

 95%|█████████▍| 9340/9864 [22:37:21<2:14:24, 15.39s/it]
 95%|█████████▍| 9341/9864 [22:37:35<2:09:44, 14.88s/it]
 95%|█████████▍| 9342/9864 [22:37:48<2:06:54, 14.59s/it]
 95%|█████████▍| 9343/9864 [22:38:02<2:05:03, 14.40s/it]
 95%|█████████▍| 9344/9864 [22:38:17<2:05:03, 14.43s/it]
 95%|█████████▍| 9345/9864 [22:38:31<2:03:16, 14.25s/it]
 95%|█████████▍| 9346/9864 [22:38:47<2:08:42, 14.91s/it]
 95%|█████████▍| 9347/9864 [22:39:03<2:10:45, 15.18s/it]
 95%|█████████▍| 9348/9864 [22:39:18<2:11:00, 15.23s/it]
 95%|█████████▍| 9349/9864 [22:39:35<2:13:18, 15.53s/it]
 95%|█████████▍| 9350/9864 [22:39:49<2:09:17, 15.09s/it]
                                                        
{'loss': '0.04275', 'grad_norm': '3.309', 'learning_rate': '1.426e-07', 'epoch': '0.948'}

 95%|█████████▍| 9350/9864 [22:39:49<2:09:17, 15.09s/it]
 95%|█████████▍| 9351/9864 [22:40:03<2:06:31, 14.80s/it]
 95%|█████████▍| 9352/9864 [22:40:17<2:04:53, 14.64s/it]
 95%|█████████▍| 9353/9864 [22:40:33<2:08:28, 15.09s/it]
 95%|█████████▍| 9354/9864 [22:40:47<2:05:42, 14.79s/it]
 95%|█████████▍| 9355/9864 [22:41:04<2:11:12, 15.47s/it]
 95%|█████████▍| 9356/9864 [22:41:18<2:07:26, 15.05s/it]
 95%|█████████▍| 9357/9864 [22:41:34<2:09:45, 15.36s/it]
 95%|█████████▍| 9358/9864 [22:41:49<2:06:12, 14.96s/it]
 95%|█████████▍| 9359/9864 [22:42:03<2:04:49, 14.83s/it]
 95%|█████████▍| 9360/9864 [22:42:17<2:02:01, 14.53s/it]
                                                        
{'loss': '0.0501', 'grad_norm': '4.77', 'learning_rate': '1.372e-07', 'epoch': '0.949'}

 95%|█████████▍| 9360/9864 [22:42:17<2:02:01, 14.53s/it]
 95%|█████████▍| 9361/9864 [22:42:33<2:05:04, 14.92s/it]
 95%|█████████▍| 9362/9864 [22:42:50<2:11:52, 15.76s/it]
 95%|█████████▍| 9363/9864 [22:43:05<2:08:35, 15.40s/it]
 95%|█████████▍| 9364/9864 [22:43:19<2:04:32, 14.94s/it]
 95%|█████████▍| 9365/9864 [22:43:35<2:07:36, 15.34s/it]
 95%|█████████▍| 9366/9864 [22:43:49<2:04:00, 14.94s/it]
 95%|█████████▍| 9367/9864 [22:44:03<2:01:22, 14.65s/it]
 95%|█████████▍| 9368/9864 [22:44:17<1:59:41, 14.48s/it]
 95%|█████████▍| 9369/9864 [22:44:33<2:03:39, 14.99s/it]
 95%|█████████▍| 9370/9864 [22:44:49<2:05:13, 15.21s/it]
                                                        
{'loss': '0.04858', 'grad_norm': '5.2', 'learning_rate': '1.318e-07', 'epoch': '0.95'}

 95%|█████████▍| 9370/9864 [22:44:49<2:05:13, 15.21s/it]
 95%|█████████▌| 9371/9864 [22:45:05<2:06:44, 15.43s/it]
 95%|█████████▌| 9372/9864 [22:45:21<2:07:35, 15.56s/it]
 95%|█████████▌| 9373/9864 [22:45:35<2:03:03, 15.04s/it]
 95%|█████████▌| 9374/9864 [22:45:49<2:00:11, 14.72s/it]
 95%|█████████▌| 9375/9864 [22:46:03<1:58:21, 14.52s/it]
 95%|█████████▌| 9376/9864 [22:46:16<1:55:49, 14.24s/it]
 95%|█████████▌| 9377/9864 [22:46:32<1:58:55, 14.65s/it]
 95%|█████████▌| 9378/9864 [22:46:47<2:00:19, 14.85s/it]
 95%|█████████▌| 9379/9864 [22:47:04<2:03:36, 15.29s/it]
 95%|█████████▌| 9380/9864 [22:47:20<2:05:52, 15.61s/it]
                                                        
{'loss': '0.049', 'grad_norm': '5.61', 'learning_rate': '1.265e-07', 'epoch': '0.951'}

 95%|█████████▌| 9380/9864 [22:47:20<2:05:52, 15.61s/it]
 95%|█████████▌| 9381/9864 [22:47:34<2:02:09, 15.18s/it]
 95%|█████████▌| 9382/9864 [22:47:48<1:58:59, 14.81s/it]
 95%|█████████▌| 9383/9864 [22:48:01<1:55:16, 14.38s/it]
 95%|█████████▌| 9384/9864 [22:48:15<1:53:37, 14.20s/it]
 95%|█████████▌| 9385/9864 [22:48:32<2:00:00, 15.03s/it]
 95%|█████████▌| 9386/9864 [22:48:46<1:57:36, 14.76s/it]
 95%|█████████▌| 9387/9864 [22:49:02<2:00:01, 15.10s/it]
 95%|█████████▌| 9388/9864 [22:49:18<2:01:09, 15.27s/it]
 95%|█████████▌| 9389/9864 [22:49:32<1:58:58, 15.03s/it]
 95%|█████████▌| 9390/9864 [22:49:46<1:55:47, 14.66s/it]
                                                        
{'loss': '0.05369', 'grad_norm': '7.868', 'learning_rate': '1.214e-07', 'epoch': '0.952'}

 95%|█████████▌| 9390/9864 [22:49:46<1:55:47, 14.66s/it]
 95%|█████████▌| 9391/9864 [22:50:01<1:54:58, 14.58s/it]
 95%|█████████▌| 9392/9864 [22:50:15<1:53:52, 14.48s/it]
 95%|█████████▌| 9393/9864 [22:50:30<1:56:22, 14.83s/it]
 95%|█████████▌| 9394/9864 [22:50:46<1:58:25, 15.12s/it]
 95%|█████████▌| 9395/9864 [22:51:00<1:55:34, 14.79s/it]
 95%|█████████▌| 9396/9864 [22:51:18<2:01:52, 15.62s/it]
 95%|█████████▌| 9397/9864 [22:51:32<1:58:50, 15.27s/it]
 95%|█████████▌| 9398/9864 [22:51:46<1:55:33, 14.88s/it]
 95%|█████████▌| 9399/9864 [22:52:00<1:53:09, 14.60s/it]
 95%|█████████▌| 9400/9864 [22:52:16<1:55:07, 14.89s/it]
                                                        
{'loss': '0.04792', 'grad_norm': '5.899', 'learning_rate': '1.163e-07', 'epoch': '0.953'}

 95%|█████████▌| 9400/9864 [22:52:16<1:55:07, 14.89s/it]
 95%|█████████▌| 9401/9864 [22:52:32<1:57:29, 15.22s/it]
 95%|█████████▌| 9402/9864 [22:52:46<1:54:16, 14.84s/it]
 95%|█████████▌| 9403/9864 [22:52:59<1:51:17, 14.48s/it]
 95%|█████████▌| 9404/9864 [22:53:17<1:57:41, 15.35s/it]
 95%|█████████▌| 9405/9864 [22:53:31<1:54:49, 15.01s/it]
 95%|█████████▌| 9406/9864 [22:53:45<1:51:28, 14.60s/it]
 95%|█████████▌| 9407/9864 [22:53:58<1:49:14, 14.34s/it]
 95%|█████████▌| 9408/9864 [22:54:14<1:52:28, 14.80s/it]
 95%|█████████▌| 9409/9864 [22:54:30<1:54:58, 15.16s/it]
 95%|█████████▌| 9410/9864 [22:54:44<1:51:38, 14.75s/it]
                                                        
{'loss': '0.04427', 'grad_norm': '5.627', 'learning_rate': '1.114e-07', 'epoch': '0.9541'}

 95%|█████████▌| 9410/9864 [22:54:44<1:51:38, 14.75s/it]
 95%|█████████▌| 9411/9864 [22:54:59<1:53:00, 14.97s/it]
 95%|█████████▌| 9412/9864 [22:55:15<1:53:31, 15.07s/it]
 95%|█████████▌| 9413/9864 [22:55:29<1:50:55, 14.76s/it]
 95%|█████████▌| 9414/9864 [22:55:43<1:48:36, 14.48s/it]
 95%|█████████▌| 9415/9864 [22:55:58<1:50:42, 14.79s/it]
 95%|█████████▌| 9416/9864 [22:56:12<1:49:10, 14.62s/it]
 95%|█████████▌| 9417/9864 [22:56:28<1:51:22, 14.95s/it]
 95%|█████████▌| 9418/9864 [22:56:42<1:49:06, 14.68s/it]
 95%|█████████▌| 9419/9864 [22:57:00<1:55:29, 15.57s/it]
 95%|█████████▌| 9420/9864 [22:57:14<1:52:33, 15.21s/it]
                                                        
{'loss': '0.04608', 'grad_norm': '4.748', 'learning_rate': '1.066e-07', 'epoch': '0.9551'}

 95%|█████████▌| 9420/9864 [22:57:14<1:52:33, 15.21s/it]
 96%|█████████▌| 9421/9864 [22:57:28<1:50:07, 14.92s/it]
 96%|█████████▌| 9422/9864 [22:57:43<1:48:43, 14.76s/it]
 96%|█████████▌| 9423/9864 [22:57:56<1:46:04, 14.43s/it]
 96%|█████████▌| 9424/9864 [22:58:13<1:51:10, 15.16s/it]
 96%|█████████▌| 9425/9864 [22:58:28<1:49:05, 14.91s/it]
 96%|█████████▌| 9426/9864 [22:58:43<1:50:40, 15.16s/it]
 96%|█████████▌| 9427/9864 [22:58:58<1:48:16, 14.87s/it]
 96%|█████████▌| 9428/9864 [22:59:12<1:46:18, 14.63s/it]
 96%|█████████▌| 9429/9864 [22:59:26<1:44:54, 14.47s/it]
 96%|█████████▌| 9430/9864 [22:59:40<1:43:36, 14.32s/it]
                                                        
{'loss': '0.04247', 'grad_norm': '2.814', 'learning_rate': '1.018e-07', 'epoch': '0.9561'}

 96%|█████████▌| 9430/9864 [22:59:40<1:43:36, 14.32s/it]
 96%|█████████▌| 9431/9864 [22:59:54<1:42:31, 14.21s/it]
 96%|█████████▌| 9432/9864 [23:00:11<1:49:35, 15.22s/it]
 96%|█████████▌| 9433/9864 [23:00:25<1:46:18, 14.80s/it]
 96%|█████████▌| 9434/9864 [23:00:39<1:43:08, 14.39s/it]
 96%|█████████▌| 9435/9864 [23:00:55<1:47:43, 15.07s/it]
 96%|█████████▌| 9436/9864 [23:01:09<1:44:40, 14.67s/it]
 96%|█████████▌| 9437/9864 [23:01:23<1:42:52, 14.46s/it]
 96%|█████████▌| 9438/9864 [23:01:37<1:42:53, 14.49s/it]
 96%|█████████▌| 9439/9864 [23:01:53<1:44:02, 14.69s/it]
 96%|█████████▌| 9440/9864 [23:02:09<1:46:33, 15.08s/it]
                                                        
{'loss': '0.03447', 'grad_norm': '5.391', 'learning_rate': '9.721e-08', 'epoch': '0.9571'}

 96%|█████████▌| 9440/9864 [23:02:09<1:46:33, 15.08s/it]
 96%|█████████▌| 9441/9864 [23:02:23<1:44:58, 14.89s/it]
 96%|█████████▌| 9442/9864 [23:02:39<1:46:03, 15.08s/it]
 96%|█████████▌| 9443/9864 [23:02:55<1:47:55, 15.38s/it]
 96%|█████████▌| 9444/9864 [23:03:09<1:45:28, 15.07s/it]
 96%|█████████▌| 9445/9864 [23:03:23<1:42:44, 14.71s/it]
 96%|█████████▌| 9446/9864 [23:03:37<1:42:10, 14.67s/it]
 96%|█████████▌| 9447/9864 [23:03:51<1:40:45, 14.50s/it]
 96%|█████████▌| 9448/9864 [23:04:10<1:47:53, 15.56s/it]
 96%|█████████▌| 9449/9864 [23:04:23<1:43:56, 15.03s/it]
 96%|█████████▌| 9450/9864 [23:04:41<1:49:45, 15.91s/it]
                                                        
{'loss': '0.04657', 'grad_norm': '5.978', 'learning_rate': '9.269e-08', 'epoch': '0.9581'}

 96%|█████████▌| 9450/9864 [23:04:41<1:49:45, 15.91s/it]
 96%|█████████▌| 9451/9864 [23:04:55<1:45:38, 15.35s/it]
 96%|█████████▌| 9452/9864 [23:05:10<1:43:29, 15.07s/it]
 96%|█████████▌| 9453/9864 [23:05:24<1:41:47, 14.86s/it]
 96%|█████████▌| 9454/9864 [23:05:38<1:40:05, 14.65s/it]
 96%|█████████▌| 9455/9864 [23:05:56<1:45:13, 15.44s/it]
 96%|█████████▌| 9456/9864 [23:06:10<1:42:40, 15.10s/it]
 96%|█████████▌| 9457/9864 [23:06:24<1:41:11, 14.92s/it]
 96%|█████████▌| 9458/9864 [23:06:42<1:45:58, 15.66s/it]
 96%|█████████▌| 9459/9864 [23:06:57<1:43:53, 15.39s/it]
 96%|█████████▌| 9460/9864 [23:07:11<1:41:26, 15.07s/it]
                                                        
{'loss': '0.04153', 'grad_norm': '3.917', 'learning_rate': '8.829e-08', 'epoch': '0.9591'}

 96%|█████████▌| 9460/9864 [23:07:11<1:41:26, 15.07s/it]
 96%|█████████▌| 9461/9864 [23:07:25<1:38:24, 14.65s/it]
 96%|█████████▌| 9462/9864 [23:07:40<1:40:33, 15.01s/it]
 96%|█████████▌| 9463/9864 [23:07:56<1:42:05, 15.27s/it]
 96%|█████████▌| 9464/9864 [23:08:10<1:39:10, 14.88s/it]
 96%|█████████▌| 9465/9864 [23:08:26<1:40:02, 15.04s/it]
 96%|█████████▌| 9466/9864 [23:08:41<1:41:18, 15.27s/it]
 96%|█████████▌| 9467/9864 [23:08:56<1:39:35, 15.05s/it]
 96%|█████████▌| 9468/9864 [23:09:11<1:38:22, 14.91s/it]
 96%|█████████▌| 9469/9864 [23:09:25<1:36:44, 14.69s/it]
 96%|█████████▌| 9470/9864 [23:09:41<1:39:17, 15.12s/it]
                                                        
{'loss': '0.03169', 'grad_norm': '5.915', 'learning_rate': '8.399e-08', 'epoch': '0.9601'}

 96%|█████████▌| 9470/9864 [23:09:41<1:39:17, 15.12s/it]
 96%|█████████▌| 9471/9864 [23:09:55<1:37:10, 14.84s/it]
 96%|█████████▌| 9472/9864 [23:10:11<1:39:04, 15.17s/it]
 96%|█████████▌| 9473/9864 [23:10:27<1:40:15, 15.38s/it]
 96%|█████████▌| 9474/9864 [23:10:42<1:40:30, 15.46s/it]
 96%|█████████▌| 9475/9864 [23:10:57<1:37:28, 15.03s/it]
 96%|█████████▌| 9476/9864 [23:11:11<1:35:29, 14.77s/it]
 96%|█████████▌| 9477/9864 [23:11:27<1:37:45, 15.16s/it]
 96%|█████████▌| 9478/9864 [23:11:41<1:35:18, 14.82s/it]
 96%|█████████▌| 9479/9864 [23:11:55<1:33:44, 14.61s/it]
 96%|█████████▌| 9480/9864 [23:12:11<1:36:48, 15.13s/it]
                                                        
{'loss': '0.05209', 'grad_norm': '7.72', 'learning_rate': '7.979e-08', 'epoch': '0.9611'}

 96%|█████████▌| 9480/9864 [23:12:11<1:36:48, 15.13s/it]
 96%|█████████▌| 9481/9864 [23:12:27<1:37:14, 15.23s/it]
 96%|█████████▌| 9482/9864 [23:12:42<1:37:28, 15.31s/it]
 96%|█████████▌| 9483/9864 [23:12:56<1:34:37, 14.90s/it]
 96%|█████████▌| 9484/9864 [23:13:12<1:35:15, 15.04s/it]
 96%|█████████▌| 9485/9864 [23:13:26<1:34:16, 14.92s/it]
 96%|█████████▌| 9486/9864 [23:13:41<1:33:20, 14.82s/it]
 96%|█████████▌| 9487/9864 [23:13:55<1:31:35, 14.58s/it]
 96%|█████████▌| 9488/9864 [23:14:08<1:29:39, 14.31s/it]
 96%|█████████▌| 9489/9864 [23:14:26<1:35:13, 15.23s/it]
 96%|█████████▌| 9490/9864 [23:14:40<1:32:04, 14.77s/it]
                                                        
{'loss': '0.04312', 'grad_norm': '3.163', 'learning_rate': '7.571e-08', 'epoch': '0.9622'}

 96%|█████████▌| 9490/9864 [23:14:40<1:32:04, 14.77s/it]
 96%|█████████▌| 9491/9864 [23:14:54<1:31:14, 14.68s/it]
 96%|█████████▌| 9492/9864 [23:15:10<1:32:56, 14.99s/it]
 96%|█████████▌| 9493/9864 [23:15:24<1:30:37, 14.66s/it]
 96%|█████████▌| 9494/9864 [23:15:37<1:28:59, 14.43s/it]
 96%|█████████▋| 9495/9864 [23:15:52<1:28:53, 14.45s/it]
 96%|█████████▋| 9496/9864 [23:16:10<1:34:26, 15.40s/it]
 96%|█████████▋| 9497/9864 [23:16:25<1:34:12, 15.40s/it]
 96%|█████████▋| 9498/9864 [23:16:39<1:31:38, 15.02s/it]
 96%|█████████▋| 9499/9864 [23:16:55<1:32:34, 15.22s/it]
 96%|█████████▋| 9500/9864 [23:17:09<1:29:58, 14.83s/it]
                                                        
{'loss': '0.04467', 'grad_norm': '6.108', 'learning_rate': '7.173e-08', 'epoch': '0.9632'}

 96%|█████████▋| 9500/9864 [23:17:09<1:29:58, 14.83s/it]
 96%|█████████▋| 9501/9864 [23:17:23<1:28:00, 14.55s/it]
 96%|█████████▋| 9502/9864 [23:17:37<1:26:53, 14.40s/it]
 96%|█████████▋| 9503/9864 [23:17:51<1:26:02, 14.30s/it]
 96%|█████████▋| 9504/9864 [23:18:06<1:27:23, 14.56s/it]
 96%|█████████▋| 9505/9864 [23:18:23<1:32:03, 15.39s/it]
 96%|█████████▋| 9506/9864 [23:18:38<1:30:22, 15.15s/it]
 96%|█████████▋| 9507/9864 [23:18:54<1:31:25, 15.36s/it]
 96%|█████████▋| 9508/9864 [23:19:08<1:29:52, 15.15s/it]
 96%|█████████▋| 9509/9864 [23:19:23<1:28:28, 14.95s/it]
 96%|█████████▋| 9510/9864 [23:19:37<1:26:24, 14.65s/it]
                                                        
{'loss': '0.04648', 'grad_norm': '9.469', 'learning_rate': '6.786e-08', 'epoch': '0.9642'}

 96%|█████████▋| 9510/9864 [23:19:37<1:26:24, 14.65s/it]
 96%|█████████▋| 9511/9864 [23:19:51<1:25:11, 14.48s/it]
 96%|█████████▋| 9512/9864 [23:20:10<1:32:57, 15.85s/it]
 96%|█████████▋| 9513/9864 [23:20:24<1:28:58, 15.21s/it]
 96%|█████████▋| 9514/9864 [23:20:40<1:30:09, 15.46s/it]
 96%|█████████▋| 9515/9864 [23:20:53<1:26:12, 14.82s/it]
 96%|█████████▋| 9516/9864 [23:21:07<1:23:48, 14.45s/it]
 96%|█████████▋| 9517/9864 [23:21:21<1:23:34, 14.45s/it]
 96%|█████████▋| 9518/9864 [23:21:35<1:22:49, 14.36s/it]
 97%|█████████▋| 9519/9864 [23:21:51<1:25:40, 14.90s/it]
 97%|█████████▋| 9520/9864 [23:22:09<1:30:15, 15.74s/it]
                                                        
{'loss': '0.04828', 'grad_norm': '4.243', 'learning_rate': '6.409e-08', 'epoch': '0.9652'}

 97%|█████████▋| 9520/9864 [23:22:09<1:30:15, 15.74s/it]
 97%|█████████▋| 9521/9864 [23:22:23<1:26:26, 15.12s/it]
 97%|█████████▋| 9522/9864 [23:22:38<1:27:19, 15.32s/it]
 97%|█████████▋| 9523/9864 [23:22:52<1:24:38, 14.89s/it]
 97%|█████████▋| 9524/9864 [23:23:06<1:22:36, 14.58s/it]
 97%|█████████▋| 9525/9864 [23:23:21<1:21:51, 14.49s/it]
 97%|█████████▋| 9526/9864 [23:23:35<1:21:15, 14.43s/it]
 97%|█████████▋| 9527/9864 [23:23:51<1:23:50, 14.93s/it]
 97%|█████████▋| 9528/9864 [23:24:09<1:28:18, 15.77s/it]
 97%|█████████▋| 9529/9864 [23:24:23<1:25:24, 15.30s/it]
 97%|█████████▋| 9530/9864 [23:24:39<1:26:21, 15.51s/it]
                                                        
{'loss': '0.05291', 'grad_norm': '6.257', 'learning_rate': '6.043e-08', 'epoch': '0.9662'}

 97%|█████████▋| 9530/9864 [23:24:39<1:26:21, 15.51s/it]
 97%|█████████▋| 9531/9864 [23:24:54<1:24:41, 15.26s/it]
 97%|█████████▋| 9532/9864 [23:25:07<1:22:08, 14.84s/it]
 97%|█████████▋| 9533/9864 [23:25:22<1:21:31, 14.78s/it]
 97%|█████████▋| 9534/9864 [23:25:36<1:20:42, 14.67s/it]
 97%|█████████▋| 9535/9864 [23:25:54<1:24:32, 15.42s/it]
 97%|█████████▋| 9536/9864 [23:26:09<1:24:35, 15.47s/it]
 97%|█████████▋| 9537/9864 [23:26:23<1:21:23, 14.93s/it]
 97%|█████████▋| 9538/9864 [23:26:39<1:22:58, 15.27s/it]
 97%|█████████▋| 9539/9864 [23:26:53<1:20:53, 14.93s/it]
 97%|█████████▋| 9540/9864 [23:27:08<1:19:51, 14.79s/it]
                                                        
{'loss': '0.05061', 'grad_norm': '4.109', 'learning_rate': '5.688e-08', 'epoch': '0.9672'}

 97%|█████████▋| 9540/9864 [23:27:08<1:19:51, 14.79s/it]
 97%|█████████▋| 9541/9864 [23:27:22<1:18:26, 14.57s/it]
 97%|█████████▋| 9542/9864 [23:27:36<1:17:44, 14.49s/it]
 97%|█████████▋| 9543/9864 [23:27:54<1:22:42, 15.46s/it]
 97%|█████████▋| 9544/9864 [23:28:09<1:23:09, 15.59s/it]
 97%|█████████▋| 9545/9864 [23:28:24<1:20:32, 15.15s/it]
 97%|█████████▋| 9546/9864 [23:28:39<1:20:55, 15.27s/it]
 97%|█████████▋| 9547/9864 [23:28:53<1:19:07, 14.97s/it]
 97%|█████████▋| 9548/9864 [23:29:08<1:17:54, 14.79s/it]
 97%|█████████▋| 9549/9864 [23:29:22<1:16:30, 14.57s/it]
 97%|█████████▋| 9550/9864 [23:29:36<1:15:24, 14.41s/it]
                                                        
{'loss': '0.04518', 'grad_norm': '3.647', 'learning_rate': '5.344e-08', 'epoch': '0.9682'}

 97%|█████████▋| 9550/9864 [23:29:36<1:15:24, 14.41s/it]
 97%|█████████▋| 9551/9864 [23:29:53<1:20:03, 15.35s/it]
 97%|█████████▋| 9552/9864 [23:30:08<1:17:57, 14.99s/it]
 97%|█████████▋| 9553/9864 [23:30:23<1:18:27, 15.14s/it]
 97%|█████████▋| 9554/9864 [23:30:36<1:15:16, 14.57s/it]
 97%|█████████▋| 9555/9864 [23:30:50<1:14:10, 14.40s/it]
 97%|█████████▋| 9556/9864 [23:31:04<1:13:05, 14.24s/it]
 97%|█████████▋| 9557/9864 [23:31:18<1:12:29, 14.17s/it]
 97%|█████████▋| 9558/9864 [23:31:34<1:14:27, 14.60s/it]
 97%|█████████▋| 9559/9864 [23:31:51<1:18:53, 15.52s/it]
 97%|█████████▋| 9560/9864 [23:32:06<1:16:53, 15.17s/it]
                                                        
{'loss': '0.05474', 'grad_norm': '4.137', 'learning_rate': '5.01e-08', 'epoch': '0.9693'}

 97%|█████████▋| 9560/9864 [23:32:06<1:16:53, 15.17s/it]
 97%|█████████▋| 9561/9864 [23:32:21<1:17:10, 15.28s/it]
 97%|█████████▋| 9562/9864 [23:32:35<1:15:03, 14.91s/it]
 97%|█████████▋| 9563/9864 [23:32:49<1:13:09, 14.58s/it]
 97%|█████████▋| 9564/9864 [23:33:03<1:11:56, 14.39s/it]
 97%|█████████▋| 9565/9864 [23:33:18<1:11:52, 14.42s/it]
 97%|█████████▋| 9566/9864 [23:33:35<1:16:09, 15.33s/it]
 97%|█████████▋| 9567/9864 [23:33:51<1:16:17, 15.41s/it]
 97%|█████████▋| 9568/9864 [23:34:06<1:16:19, 15.47s/it]
 97%|█████████▋| 9569/9864 [23:34:20<1:13:57, 15.04s/it]
 97%|█████████▋| 9570/9864 [23:34:34<1:11:45, 14.64s/it]
                                                        
{'loss': '0.04286', 'grad_norm': '3.138', 'learning_rate': '4.687e-08', 'epoch': '0.9703'}

 97%|█████████▋| 9570/9864 [23:34:34<1:11:45, 14.64s/it]
 97%|█████████▋| 9571/9864 [23:34:48<1:10:31, 14.44s/it]
 97%|█████████▋| 9572/9864 [23:35:02<1:08:55, 14.16s/it]
 97%|█████████▋| 9573/9864 [23:35:17<1:10:56, 14.63s/it]
 97%|█████████▋| 9574/9864 [23:35:34<1:13:12, 15.15s/it]
 97%|█████████▋| 9575/9864 [23:35:49<1:13:45, 15.31s/it]
 97%|█████████▋| 9576/9864 [23:36:05<1:14:21, 15.49s/it]
 97%|█████████▋| 9577/9864 [23:36:19<1:11:53, 15.03s/it]
 97%|█████████▋| 9578/9864 [23:36:33<1:10:08, 14.71s/it]
 97%|█████████▋| 9579/9864 [23:36:47<1:08:01, 14.32s/it]
 97%|█████████▋| 9580/9864 [23:37:01<1:07:29, 14.26s/it]
                                                        
{'loss': '0.04447', 'grad_norm': '5.041', 'learning_rate': '4.375e-08', 'epoch': '0.9713'}

 97%|█████████▋| 9580/9864 [23:37:01<1:07:29, 14.26s/it]
 97%|█████████▋| 9581/9864 [23:37:18<1:11:28, 15.15s/it]
 97%|█████████▋| 9582/9864 [23:37:34<1:12:46, 15.48s/it]
 97%|█████████▋| 9583/9864 [23:37:48<1:09:45, 14.90s/it]
 97%|█████████▋| 9584/9864 [23:38:03<1:10:32, 15.12s/it]
 97%|█████████▋| 9585/9864 [23:38:18<1:08:58, 14.83s/it]
 97%|█████████▋| 9586/9864 [23:38:31<1:07:24, 14.55s/it]
 97%|█████████▋| 9587/9864 [23:38:46<1:06:36, 14.43s/it]
 97%|█████████▋| 9588/9864 [23:39:00<1:06:29, 14.45s/it]
 97%|█████████▋| 9589/9864 [23:39:17<1:09:54, 15.25s/it]
 97%|█████████▋| 9590/9864 [23:39:33<1:10:20, 15.40s/it]
                                                        
{'loss': '0.04641', 'grad_norm': '5.651', 'learning_rate': '4.074e-08', 'epoch': '0.9723'}

 97%|█████████▋| 9590/9864 [23:39:33<1:10:20, 15.40s/it]
 97%|█████████▋| 9591/9864 [23:39:49<1:10:28, 15.49s/it]
 97%|█████████▋| 9592/9864 [23:40:03<1:08:06, 15.02s/it]
 97%|█████████▋| 9593/9864 [23:40:16<1:06:18, 14.68s/it]
 97%|█████████▋| 9594/9864 [23:40:30<1:04:49, 14.40s/it]
 97%|█████████▋| 9595/9864 [23:40:45<1:04:25, 14.37s/it]
 97%|█████████▋| 9596/9864 [23:41:01<1:06:42, 14.94s/it]
 97%|█████████▋| 9597/9864 [23:41:16<1:07:19, 15.13s/it]
 97%|█████████▋| 9598/9864 [23:41:32<1:08:10, 15.38s/it]
 97%|█████████▋| 9599/9864 [23:41:48<1:08:31, 15.52s/it]
 97%|█████████▋| 9600/9864 [23:42:02<1:06:16, 15.06s/it]
                                                        
{'loss': '0.0405', 'grad_norm': '5.804', 'learning_rate': '3.783e-08', 'epoch': '0.9733'}

 97%|█████████▋| 9600/9864 [23:42:02<1:06:16, 15.06s/it]
 97%|█████████▋| 9601/9864 [23:42:16<1:04:27, 14.70s/it]
 97%|█████████▋| 9602/9864 [23:42:31<1:03:56, 14.64s/it]
 97%|█████████▋| 9603/9864 [23:42:45<1:02:56, 14.47s/it]
 97%|█████████▋| 9604/9864 [23:43:00<1:04:31, 14.89s/it]
 97%|█████████▋| 9605/9864 [23:43:16<1:05:39, 15.21s/it]
 97%|█████████▋| 9606/9864 [23:43:32<1:05:54, 15.33s/it]
 97%|█████████▋| 9607/9864 [23:43:48<1:06:42, 15.57s/it]
 97%|█████████▋| 9608/9864 [23:44:02<1:04:49, 15.19s/it]
 97%|█████████▋| 9609/9864 [23:44:16<1:02:47, 14.77s/it]
 97%|█████████▋| 9610/9864 [23:44:30<1:01:26, 14.51s/it]
                                                        
{'loss': '0.03786', 'grad_norm': '3.992', 'learning_rate': '3.503e-08', 'epoch': '0.9743'}

 97%|█████████▋| 9610/9864 [23:44:30<1:01:26, 14.51s/it]
 97%|█████████▋| 9611/9864 [23:44:44<1:00:42, 14.40s/it]
 97%|█████████▋| 9612/9864 [23:45:00<1:02:32, 14.89s/it]
 97%|█████████▋| 9613/9864 [23:45:18<1:05:48, 15.73s/it]
 97%|█████████▋| 9614/9864 [23:45:34<1:05:41, 15.77s/it]
 97%|█████████▋| 9615/9864 [23:45:48<1:02:52, 15.15s/it]
 97%|█████████▋| 9616/9864 [23:46:02<1:01:41, 14.93s/it]
 97%|█████████▋| 9617/9864 [23:46:16<1:00:13, 14.63s/it]
 98%|█████████▊| 9618/9864 [23:46:30<59:29, 14.51s/it]  
 98%|█████████▊| 9619/9864 [23:46:45<59:05, 14.47s/it]
 98%|█████████▊| 9620/9864 [23:47:04<1:04:30, 15.86s/it]
                                                        
{'loss': '0.0528', 'grad_norm': '4.755', 'learning_rate': '3.234e-08', 'epoch': '0.9753'}

 98%|█████████▊| 9620/9864 [23:47:04<1:04:30, 15.86s/it]
 98%|█████████▊| 9621/9864 [23:47:18<1:02:42, 15.48s/it]
 98%|█████████▊| 9622/9864 [23:47:33<1:01:24, 15.23s/it]
 98%|█████████▊| 9623/9864 [23:47:49<1:01:47, 15.38s/it]
 98%|█████████▊| 9624/9864 [23:48:03<1:00:22, 15.10s/it]
 98%|█████████▊| 9625/9864 [23:48:17<58:58, 14.80s/it]  
 98%|█████████▊| 9626/9864 [23:48:31<58:01, 14.63s/it]
 98%|█████████▊| 9627/9864 [23:48:47<59:07, 14.97s/it]
 98%|█████████▊| 9628/9864 [23:49:03<59:57, 15.24s/it]
 98%|█████████▊| 9629/9864 [23:49:18<59:57, 15.31s/it]
 98%|█████████▊| 9630/9864 [23:49:34<59:50, 15.35s/it]
                                                      
{'loss': '0.05201', 'grad_norm': '4.798', 'learning_rate': '2.975e-08', 'epoch': '0.9764'}

 98%|█████████▊| 9630/9864 [23:49:34<59:50, 15.35s/it]
 98%|█████████▊| 9631/9864 [23:49:48<57:56, 14.92s/it]
 98%|█████████▊| 9632/9864 [23:50:02<56:57, 14.73s/it]
 98%|█████████▊| 9633/9864 [23:50:16<56:14, 14.61s/it]
 98%|█████████▊| 9634/9864 [23:50:31<55:24, 14.46s/it]
 98%|█████████▊| 9635/9864 [23:50:46<56:35, 14.83s/it]
 98%|█████████▊| 9636/9864 [23:51:02<57:17, 15.08s/it]
 98%|█████████▊| 9637/9864 [23:51:18<57:43, 15.26s/it]
 98%|█████████▊| 9638/9864 [23:51:33<58:02, 15.41s/it]
 98%|█████████▊| 9639/9864 [23:51:48<56:40, 15.11s/it]
 98%|█████████▊| 9640/9864 [23:52:02<55:36, 14.89s/it]
                                                      
{'loss': '0.03951', 'grad_norm': '5.146', 'learning_rate': '2.728e-08', 'epoch': '0.9774'}

 98%|█████████▊| 9640/9864 [23:52:02<55:36, 14.89s/it]
 98%|█████████▊| 9641/9864 [23:52:16<53:57, 14.52s/it]
 98%|█████████▊| 9642/9864 [23:52:30<53:23, 14.43s/it]
 98%|█████████▊| 9643/9864 [23:52:44<52:37, 14.29s/it]
 98%|█████████▊| 9644/9864 [23:53:01<55:42, 15.19s/it]
 98%|█████████▊| 9645/9864 [23:53:17<56:13, 15.40s/it]
 98%|█████████▊| 9646/9864 [23:53:33<56:19, 15.50s/it]
 98%|█████████▊| 9647/9864 [23:53:47<54:41, 15.12s/it]
 98%|█████████▊| 9648/9864 [23:54:01<52:56, 14.70s/it]
 98%|█████████▊| 9649/9864 [23:54:14<51:12, 14.29s/it]
 98%|█████████▊| 9650/9864 [23:54:28<50:40, 14.21s/it]
                                                      
{'loss': '0.05282', 'grad_norm': '5.343', 'learning_rate': '2.491e-08', 'epoch': '0.9784'}

 98%|█████████▊| 9650/9864 [23:54:28<50:40, 14.21s/it]
 98%|█████████▊| 9651/9864 [23:54:42<50:19, 14.18s/it]
 98%|█████████▊| 9652/9864 [23:55:00<53:44, 15.21s/it]
 98%|█████████▊| 9653/9864 [23:55:17<55:37, 15.82s/it]
 98%|█████████▊| 9654/9864 [23:55:32<54:00, 15.43s/it]
 98%|█████████▊| 9655/9864 [23:55:46<52:10, 14.98s/it]
 98%|█████████▊| 9656/9864 [23:55:59<50:41, 14.62s/it]
 98%|█████████▊| 9657/9864 [23:56:14<50:15, 14.57s/it]
 98%|█████████▊| 9658/9864 [23:56:28<49:56, 14.55s/it]
 98%|█████████▊| 9659/9864 [23:56:44<50:45, 14.86s/it]
 98%|█████████▊| 9660/9864 [23:56:58<49:36, 14.59s/it]
                                                      
{'loss': '0.04476', 'grad_norm': '6.542', 'learning_rate': '2.264e-08', 'epoch': '0.9794'}

 98%|█████████▊| 9660/9864 [23:56:58<49:36, 14.59s/it]
 98%|█████████▊| 9661/9864 [23:57:17<54:23, 16.07s/it]
 98%|█████████▊| 9662/9864 [23:57:31<52:03, 15.46s/it]
 98%|█████████▊| 9663/9864 [23:57:45<50:13, 14.99s/it]
 98%|█████████▊| 9664/9864 [23:58:00<49:28, 14.84s/it]
 98%|█████████▊| 9665/9864 [23:58:14<48:17, 14.56s/it]
 98%|█████████▊| 9666/9864 [23:58:28<47:25, 14.37s/it]
 98%|█████████▊| 9667/9864 [23:58:44<48:37, 14.81s/it]
 98%|█████████▊| 9668/9864 [23:59:01<50:58, 15.61s/it]
 98%|█████████▊| 9669/9864 [23:59:15<49:27, 15.22s/it]
 98%|█████████▊| 9670/9864 [23:59:31<49:38, 15.35s/it]
                                                      
{'loss': '0.03445', 'grad_norm': '6.386', 'learning_rate': '2.049e-08', 'epoch': '0.9804'}

 98%|█████████▊| 9670/9864 [23:59:31<49:38, 15.35s/it]
 98%|█████████▊| 9671/9864 [23:59:45<47:41, 14.83s/it]
 98%|█████████▊| 9672/9864 [23:59:59<46:54, 14.66s/it]
 98%|█████████▊| 9673/9864 [24:00:12<45:40, 14.35s/it]
 98%|█████████▊| 9674/9864 [24:00:27<45:16, 14.30s/it]
 98%|█████████▊| 9675/9864 [24:00:43<46:33, 14.78s/it]
 98%|█████████▊| 9676/9864 [24:01:00<48:42, 15.55s/it]
 98%|█████████▊| 9677/9864 [24:01:16<48:50, 15.67s/it]
 98%|█████████▊| 9678/9864 [24:01:30<47:13, 15.24s/it]
 98%|█████████▊| 9679/9864 [24:01:44<45:35, 14.79s/it]
 98%|█████████▊| 9680/9864 [24:01:58<44:26, 14.49s/it]
                                                      
{'loss': '0.04816', 'grad_norm': '3.294', 'learning_rate': '1.844e-08', 'epoch': '0.9814'}

 98%|█████████▊| 9680/9864 [24:01:58<44:26, 14.49s/it]
 98%|█████████▊| 9681/9864 [24:02:12<43:58, 14.42s/it]
 98%|█████████▊| 9682/9864 [24:02:28<45:28, 14.99s/it]
 98%|█████████▊| 9683/9864 [24:02:44<45:44, 15.17s/it]
 98%|█████████▊| 9684/9864 [24:03:00<46:09, 15.38s/it]
 98%|█████████▊| 9685/9864 [24:03:14<44:52, 15.04s/it]
 98%|█████████▊| 9686/9864 [24:03:29<44:35, 15.03s/it]
 98%|█████████▊| 9687/9864 [24:03:43<43:41, 14.81s/it]
 98%|█████████▊| 9688/9864 [24:03:57<42:59, 14.65s/it]
 98%|█████████▊| 9689/9864 [24:04:11<42:07, 14.44s/it]
 98%|█████████▊| 9690/9864 [24:04:27<43:13, 14.90s/it]
                                                      
{'loss': '0.05207', 'grad_norm': '4.658', 'learning_rate': '1.65e-08', 'epoch': '0.9824'}

 98%|█████████▊| 9690/9864 [24:04:27<43:13, 14.90s/it]
 98%|█████████▊| 9691/9864 [24:04:43<43:47, 15.19s/it]
 98%|█████████▊| 9692/9864 [24:04:59<43:57, 15.34s/it]
 98%|█████████▊| 9693/9864 [24:05:12<41:30, 14.57s/it]
 98%|█████████▊| 9694/9864 [24:05:26<41:00, 14.48s/it]
 98%|█████████▊| 9695/9864 [24:05:43<42:37, 15.13s/it]
 98%|█████████▊| 9696/9864 [24:05:57<41:27, 14.81s/it]
 98%|█████████▊| 9697/9864 [24:06:13<42:25, 15.24s/it]
 98%|█████████▊| 9698/9864 [24:06:29<42:43, 15.44s/it]
 98%|█████████▊| 9699/9864 [24:06:45<42:38, 15.50s/it]
 98%|█████████▊| 9700/9864 [24:06:58<41:03, 15.02s/it]
                                                      
{'loss': '0.03676', 'grad_norm': '3.499', 'learning_rate': '1.467e-08', 'epoch': '0.9835'}

 98%|█████████▊| 9700/9864 [24:06:58<41:03, 15.02s/it]
 98%|█████████▊| 9701/9864 [24:07:12<39:50, 14.66s/it]
 98%|█████████▊| 9702/9864 [24:07:28<40:35, 15.03s/it]
 98%|█████████▊| 9703/9864 [24:07:42<39:38, 14.77s/it]
 98%|█████████▊| 9704/9864 [24:07:57<39:11, 14.70s/it]
 98%|█████████▊| 9705/9864 [24:08:13<40:16, 15.20s/it]
 98%|█████████▊| 9706/9864 [24:08:29<40:24, 15.35s/it]
 98%|█████████▊| 9707/9864 [24:08:45<40:28, 15.47s/it]
 98%|█████████▊| 9708/9864 [24:08:59<39:12, 15.08s/it]
 98%|█████████▊| 9709/9864 [24:09:13<38:29, 14.90s/it]
 98%|█████████▊| 9710/9864 [24:09:29<38:44, 15.10s/it]
                                                      
{'loss': '0.04602', 'grad_norm': '6.24', 'learning_rate': '1.295e-08', 'epoch': '0.9845'}

 98%|█████████▊| 9710/9864 [24:09:29<38:44, 15.10s/it]
 98%|█████████▊| 9711/9864 [24:09:43<37:27, 14.69s/it]
 98%|█████████▊| 9712/9864 [24:09:57<36:43, 14.50s/it]
 98%|█████████▊| 9713/9864 [24:10:14<38:57, 15.48s/it]
 98%|█████████▊| 9714/9864 [24:10:28<37:34, 15.03s/it]
 98%|█████████▊| 9715/9864 [24:10:43<36:53, 14.86s/it]
 98%|█████████▊| 9716/9864 [24:10:59<37:20, 15.14s/it]
 99%|█████████▊| 9717/9864 [24:11:13<36:32, 14.91s/it]
 99%|█████████▊| 9718/9864 [24:11:29<37:03, 15.23s/it]
 99%|█████████▊| 9719/9864 [24:11:43<36:06, 14.94s/it]
 99%|█████████▊| 9720/9864 [24:11:58<35:34, 14.82s/it]
                                                      
{'loss': '0.04166', 'grad_norm': '6.69', 'learning_rate': '1.133e-08', 'epoch': '0.9855'}

 99%|█████████▊| 9720/9864 [24:11:58<35:34, 14.82s/it]
 99%|█████████▊| 9721/9864 [24:12:16<37:37, 15.79s/it]
 99%|█████████▊| 9722/9864 [24:12:30<36:22, 15.37s/it]
 99%|█████████▊| 9723/9864 [24:12:46<36:31, 15.55s/it]
 99%|█████████▊| 9724/9864 [24:13:01<35:27, 15.20s/it]
 99%|█████████▊| 9725/9864 [24:13:14<34:12, 14.77s/it]
 99%|█████████▊| 9726/9864 [24:13:30<34:17, 14.91s/it]
 99%|█████████▊| 9727/9864 [24:13:43<33:18, 14.59s/it]
 99%|█████████▊| 9728/9864 [24:13:57<32:36, 14.38s/it]
 99%|█████████▊| 9729/9864 [24:14:15<34:39, 15.40s/it]
 99%|█████████▊| 9730/9864 [24:14:29<33:15, 14.89s/it]
                                                      
{'loss': '0.04572', 'grad_norm': '4.058', 'learning_rate': '9.823e-09', 'epoch': '0.9865'}

 99%|█████████▊| 9730/9864 [24:14:29<33:15, 14.89s/it]
 99%|█████████▊| 9731/9864 [24:14:43<32:26, 14.63s/it]
 99%|█████████▊| 9732/9864 [24:14:59<32:53, 14.95s/it]
 99%|█████████▊| 9733/9864 [24:15:14<33:10, 15.20s/it]
 99%|█████████▊| 9734/9864 [24:15:29<32:21, 14.94s/it]
 99%|█████████▊| 9735/9864 [24:15:43<31:30, 14.66s/it]
 99%|█████████▊| 9736/9864 [24:15:58<31:57, 14.98s/it]
 99%|█████████▊| 9737/9864 [24:16:14<32:18, 15.26s/it]
 99%|█████████▊| 9738/9864 [24:16:29<31:25, 14.96s/it]
 99%|█████████▊| 9739/9864 [24:16:44<31:29, 15.12s/it]
 99%|█████████▊| 9740/9864 [24:17:00<31:52, 15.43s/it]
                                                      
{'loss': '0.04494', 'grad_norm': '5.839', 'learning_rate': '8.421e-09', 'epoch': '0.9875'}

 99%|█████████▊| 9740/9864 [24:17:00<31:52, 15.43s/it]
 99%|█████████▉| 9741/9864 [24:17:14<30:49, 15.04s/it]
 99%|█████████▉| 9742/9864 [24:17:29<30:05, 14.80s/it]
 99%|█████████▉| 9743/9864 [24:17:43<29:34, 14.66s/it]
 99%|█████████▉| 9744/9864 [24:17:57<29:08, 14.57s/it]
 99%|█████████▉| 9745/9864 [24:18:15<30:47, 15.53s/it]
 99%|█████████▉| 9746/9864 [24:18:31<30:32, 15.53s/it]
 99%|█████████▉| 9747/9864 [24:18:45<29:29, 15.12s/it]
 99%|█████████▉| 9748/9864 [24:19:00<29:34, 15.30s/it]
 99%|█████████▉| 9749/9864 [24:19:15<28:38, 14.94s/it]
 99%|█████████▉| 9750/9864 [24:19:29<27:50, 14.65s/it]
                                                      
{'loss': '0.04616', 'grad_norm': '6.877', 'learning_rate': '7.128e-09', 'epoch': '0.9885'}

 99%|█████████▉| 9750/9864 [24:19:29<27:50, 14.65s/it]
 99%|█████████▉| 9751/9864 [24:19:42<26:57, 14.32s/it]
 99%|█████████▉| 9752/9864 [24:19:57<27:01, 14.48s/it]
 99%|█████████▉| 9753/9864 [24:20:13<27:41, 14.97s/it]
 99%|█████████▉| 9754/9864 [24:20:29<27:43, 15.12s/it]
 99%|█████████▉| 9755/9864 [24:20:44<27:55, 15.37s/it]
 99%|█████████▉| 9756/9864 [24:20:59<27:14, 15.13s/it]
 99%|█████████▉| 9757/9864 [24:21:14<26:39, 14.94s/it]
 99%|█████████▉| 9758/9864 [24:21:28<26:07, 14.79s/it]
 99%|█████████▉| 9759/9864 [24:21:42<25:38, 14.66s/it]
 99%|█████████▉| 9760/9864 [24:22:00<26:55, 15.53s/it]
                                                      
{'loss': '0.04719', 'grad_norm': '5.688', 'learning_rate': '5.942e-09', 'epoch': '0.9895'}

 99%|█████████▉| 9760/9864 [24:22:00<26:55, 15.53s/it]
 99%|█████████▉| 9761/9864 [24:22:16<27:05, 15.78s/it]
 99%|█████████▉| 9762/9864 [24:22:30<25:58, 15.28s/it]
 99%|█████████▉| 9763/9864 [24:22:46<25:59, 15.44s/it]
 99%|█████████▉| 9764/9864 [24:23:00<25:06, 15.07s/it]
 99%|█████████▉| 9765/9864 [24:23:14<24:23, 14.79s/it]
 99%|█████████▉| 9766/9864 [24:23:28<23:43, 14.53s/it]
 99%|█████████▉| 9767/9864 [24:23:42<23:14, 14.38s/it]
 99%|█████████▉| 9768/9864 [24:23:58<23:39, 14.79s/it]
 99%|█████████▉| 9769/9864 [24:24:14<23:57, 15.13s/it]
 99%|█████████▉| 9770/9864 [24:24:30<24:04, 15.36s/it]
                                                      
{'loss': '0.04205', 'grad_norm': '5.305', 'learning_rate': '4.865e-09', 'epoch': '0.9906'}

 99%|█████████▉| 9770/9864 [24:24:30<24:04, 15.36s/it]
 99%|█████████▉| 9771/9864 [24:24:46<24:01, 15.50s/it]
 99%|█████████▉| 9772/9864 [24:25:00<23:07, 15.08s/it]
 99%|█████████▉| 9773/9864 [24:25:14<22:28, 14.82s/it]
 99%|█████████▉| 9774/9864 [24:25:28<21:59, 14.66s/it]
 99%|█████████▉| 9775/9864 [24:25:45<22:23, 15.09s/it]
 99%|█████████▉| 9776/9864 [24:25:59<21:42, 14.81s/it]
 99%|█████████▉| 9777/9864 [24:26:15<22:09, 15.28s/it]
 99%|█████████▉| 9778/9864 [24:26:30<21:50, 15.24s/it]
 99%|█████████▉| 9779/9864 [24:26:45<21:35, 15.24s/it]
 99%|█████████▉| 9780/9864 [24:27:00<20:51, 14.90s/it]
                                                      
{'loss': '0.04283', 'grad_norm': '5.892', 'learning_rate': '3.894e-09', 'epoch': '0.9916'}

 99%|█████████▉| 9780/9864 [24:27:00<20:51, 14.90s/it]
 99%|█████████▉| 9781/9864 [24:27:14<20:12, 14.61s/it]
 99%|█████████▉| 9782/9864 [24:27:29<20:21, 14.90s/it]
 99%|█████████▉| 9783/9864 [24:27:43<19:54, 14.74s/it]
 99%|█████████▉| 9784/9864 [24:27:58<19:23, 14.54s/it]
 99%|█████████▉| 9785/9864 [24:28:13<19:28, 14.79s/it]
 99%|█████████▉| 9786/9864 [24:28:30<20:10, 15.52s/it]
 99%|█████████▉| 9787/9864 [24:28:45<19:29, 15.19s/it]
 99%|█████████▉| 9788/9864 [24:28:58<18:45, 14.82s/it]
 99%|█████████▉| 9789/9864 [24:29:13<18:19, 14.66s/it]
 99%|█████████▉| 9790/9864 [24:29:29<18:34, 15.07s/it]
                                                      
{'loss': '0.0478', 'grad_norm': '5.883', 'learning_rate': '3.032e-09', 'epoch': '0.9926'}

 99%|█████████▉| 9790/9864 [24:29:29<18:34, 15.07s/it]
 99%|█████████▉| 9791/9864 [24:29:43<18:02, 14.84s/it]
 99%|█████████▉| 9792/9864 [24:29:59<18:11, 15.17s/it]
 99%|█████████▉| 9793/9864 [24:30:15<18:07, 15.32s/it]
 99%|█████████▉| 9794/9864 [24:30:31<18:04, 15.50s/it]
 99%|█████████▉| 9795/9864 [24:30:44<17:09, 14.92s/it]
 99%|█████████▉| 9796/9864 [24:30:59<16:48, 14.83s/it]
 99%|█████████▉| 9797/9864 [24:31:15<16:52, 15.11s/it]
 99%|█████████▉| 9798/9864 [24:31:29<16:22, 14.89s/it]
 99%|█████████▉| 9799/9864 [24:31:43<15:56, 14.71s/it]
 99%|█████████▉| 9800/9864 [24:31:59<16:03, 15.06s/it]
                                                      
{'loss': '0.03659', 'grad_norm': '5.366', 'learning_rate': '2.277e-09', 'epoch': '0.9936'}

 99%|█████████▉| 9800/9864 [24:31:59<16:03, 15.06s/it]
 99%|█████████▉| 9801/9864 [24:32:15<16:01, 15.27s/it]
 99%|█████████▉| 9802/9864 [24:32:31<16:00, 15.49s/it]
 99%|█████████▉| 9803/9864 [24:32:45<15:15, 15.00s/it]
 99%|█████████▉| 9804/9864 [24:33:00<15:08, 15.14s/it]
 99%|█████████▉| 9805/9864 [24:33:14<14:34, 14.83s/it]
 99%|█████████▉| 9806/9864 [24:33:29<14:09, 14.65s/it]
 99%|█████████▉| 9807/9864 [24:33:43<13:48, 14.53s/it]
 99%|█████████▉| 9808/9864 [24:33:59<13:53, 14.89s/it]
 99%|█████████▉| 9809/9864 [24:34:14<13:49, 15.07s/it]
 99%|█████████▉| 9810/9864 [24:34:30<13:50, 15.38s/it]
                                                      
{'loss': '0.03835', 'grad_norm': '3.033', 'learning_rate': '1.631e-09', 'epoch': '0.9946'}

 99%|█████████▉| 9810/9864 [24:34:30<13:50, 15.38s/it]
 99%|█████████▉| 9811/9864 [24:34:44<13:09, 14.89s/it]
 99%|█████████▉| 9812/9864 [24:35:00<13:07, 15.15s/it]
 99%|█████████▉| 9813/9864 [24:35:13<12:28, 14.67s/it]
 99%|█████████▉| 9814/9864 [24:35:27<12:06, 14.53s/it]
100%|█████████▉| 9815/9864 [24:35:43<12:09, 14.89s/it]
100%|█████████▉| 9816/9864 [24:35:57<11:43, 14.65s/it]
100%|█████████▉| 9817/9864 [24:36:13<11:40, 14.90s/it]
100%|█████████▉| 9818/9864 [24:36:29<11:42, 15.28s/it]
100%|█████████▉| 9819/9864 [24:36:44<11:25, 15.23s/it]
100%|█████████▉| 9820/9864 [24:36:58<10:57, 14.93s/it]
                                                      
{'loss': '0.04609', 'grad_norm': '4.685', 'learning_rate': '1.092e-09', 'epoch': '0.9956'}

100%|█████████▉| 9820/9864 [24:36:58<10:57, 14.93s/it]
100%|█████████▉| 9821/9864 [24:37:12<10:32, 14.71s/it]
100%|█████████▉| 9822/9864 [24:37:26<10:10, 14.53s/it]
100%|█████████▉| 9823/9864 [24:37:40<09:45, 14.27s/it]
100%|█████████▉| 9824/9864 [24:37:56<09:52, 14.81s/it]
100%|█████████▉| 9825/9864 [24:38:12<09:53, 15.21s/it]
100%|█████████▉| 9826/9864 [24:38:26<09:23, 14.83s/it]
100%|█████████▉| 9827/9864 [24:38:42<09:21, 15.18s/it]
100%|█████████▉| 9828/9864 [24:38:57<08:58, 14.96s/it]
100%|█████████▉| 9829/9864 [24:39:11<08:36, 14.74s/it]
100%|█████████▉| 9830/9864 [24:39:25<08:17, 14.64s/it]
                                                      
{'loss': '0.04817', 'grad_norm': '7.374', 'learning_rate': '6.603e-10', 'epoch': '0.9966'}

100%|█████████▉| 9830/9864 [24:39:25<08:17, 14.64s/it]
100%|█████████▉| 9831/9864 [24:39:40<08:00, 14.55s/it]
100%|█████████▉| 9832/9864 [24:39:57<08:09, 15.30s/it]
100%|█████████▉| 9833/9864 [24:40:11<07:40, 14.86s/it]
100%|█████████▉| 9834/9864 [24:40:26<07:33, 15.11s/it]
100%|█████████▉| 9835/9864 [24:40:42<07:22, 15.26s/it]
100%|█████████▉| 9836/9864 [24:40:56<06:57, 14.92s/it]
100%|█████████▉| 9837/9864 [24:41:10<06:31, 14.51s/it]
100%|█████████▉| 9838/9864 [24:41:24<06:16, 14.49s/it]
100%|█████████▉| 9839/9864 [24:41:38<05:59, 14.40s/it]
100%|█████████▉| 9840/9864 [24:41:54<05:54, 14.77s/it]
                                                      
{'loss': '0.04552', 'grad_norm': '3.59', 'learning_rate': '3.369e-10', 'epoch': '0.9976'}

100%|█████████▉| 9840/9864 [24:41:54<05:54, 14.77s/it]
100%|█████████▉| 9841/9864 [24:42:10<05:47, 15.10s/it]
100%|█████████▉| 9842/9864 [24:42:24<05:24, 14.75s/it]
100%|█████████▉| 9843/9864 [24:42:39<05:15, 15.03s/it]
100%|█████████▉| 9844/9864 [24:42:55<05:03, 15.19s/it]
100%|█████████▉| 9845/9864 [24:43:09<04:44, 15.00s/it]
100%|█████████▉| 9846/9864 [24:43:24<04:26, 14.80s/it]
100%|█████████▉| 9847/9864 [24:43:40<04:16, 15.08s/it]
100%|█████████▉| 9848/9864 [24:43:53<03:54, 14.66s/it]
100%|█████████▉| 9849/9864 [24:44:09<03:45, 15.05s/it]
100%|█████████▉| 9850/9864 [24:44:23<03:26, 14.73s/it]
                                                      
{'loss': '0.04002', 'grad_norm': '5.001', 'learning_rate': '1.213e-10', 'epoch': '0.9987'}

100%|█████████▉| 9850/9864 [24:44:23<03:26, 14.73s/it]
100%|█████████▉| 9851/9864 [24:44:37<03:09, 14.57s/it]
100%|█████████▉| 9852/9864 [24:44:55<03:05, 15.47s/it]
100%|█████████▉| 9853/9864 [24:45:09<02:46, 15.11s/it]
100%|█████████▉| 9854/9864 [24:45:24<02:28, 14.89s/it]
100%|█████████▉| 9855/9864 [24:45:38<02:12, 14.68s/it]
100%|█████████▉| 9856/9864 [24:45:53<01:59, 14.96s/it]
100%|█████████▉| 9857/9864 [24:46:09<01:46, 15.18s/it]
100%|█████████▉| 9858/9864 [24:46:23<01:29, 14.94s/it]
100%|█████████▉| 9859/9864 [24:46:39<01:15, 15.14s/it]
100%|█████████▉| 9860/9864 [24:46:53<00:58, 14.68s/it]
                                                      
{'loss': '0.04129', 'grad_norm': '3.748', 'learning_rate': '1.348e-11', 'epoch': '0.9997'}

100%|█████████▉| 9860/9864 [24:46:53<00:58, 14.68s/it]
100%|█████████▉| 9861/9864 [24:47:08<00:44, 14.94s/it]
100%|█████████▉| 9862/9864 [24:47:22<00:29, 14.60s/it]
100%|█████████▉| 9863/9864 [24:47:35<00:14, 14.17s/it]
100%|██████████| 9864/9864 [24:47:39<00:00, 11.04s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.84s/it]
Writing model shards: 100%|██████████| 1/1 [00:08<00:00,  8.84s/it]

                                                      
{'train_runtime': '8.931e+04', 'train_samples_per_second': '7.068', 'train_steps_per_second': '0.11', 'train_loss': '0.0577', 'epoch': '1'}

100%|██████████| 9864/9864 [24:48:32<00:00, 11.04s/it]
100%|██████████| 9864/9864 [24:48:32<00:00,  9.05s/it]

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:09<00:00,  9.27s/it]
Writing model shards: 100%|██████████| 1/1 [00:09<00:00,  9.27s/it]
[rank0]:[W626 00:08:28.410267542 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())