[INFO|2026-05-14 01:13:19] llamafactory.launcher:144 >> Initializing 8 distributed tasks at: 127.0.0.1:45499
W0514 01:13:24.035000 3057693 site-packages/torch/distributed/run.py:774] 
W0514 01:13:24.035000 3057693 site-packages/torch/distributed/run.py:774] *****************************************
W0514 01:13:24.035000 3057693 site-packages/torch/distributed/run.py:774] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
W0514 01:13:24.035000 3057693 site-packages/torch/distributed/run.py:774] *****************************************
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[W514 01:14:30.442700089 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442704929 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442709679 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442710409 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442710709 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442711029 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442713449 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W514 01:14:30.442749408 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[INFO|2026-05-14 01:14:33] llamafactory.hparams.parser:507 >> Process rank: 6, world size: 8, device: cuda:6, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:33] llamafactory.hparams.parser:507 >> Process rank: 1, world size: 8, device: cuda:1, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:33] llamafactory.hparams.parser:507 >> Process rank: 3, world size: 8, device: cuda:3, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:33] llamafactory.hparams.parser:507 >> Process rank: 0, world size: 8, device: cuda:0, distributed training: True, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:769] 2026-05-14 01:14:34,021 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/config.json
[INFO|2026-05-14 01:14:34] llamafactory.hparams.parser:507 >> Process rank: 5, world size: 8, device: cuda:5, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:34] llamafactory.hparams.parser:507 >> Process rank: 4, world size: 8, device: cuda:4, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:34] llamafactory.hparams.parser:507 >> Process rank: 7, world size: 8, device: cuda:7, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-05-14 01:14:34] llamafactory.hparams.parser:507 >> Process rank: 2, world size: 8, device: cuda:2, distributed training: True, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:847] 2026-05-14 01:14:34,060 >> Model config Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "image_token_id": 248056,
  "model_type": "qwen3_5",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 4096,
    "initializer_range": 0.02,
    "intermediate_size": 12288,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": false,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.6.0",
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 27,
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1152,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4304,
    "model_type": "qwen3_5_vision",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 4096,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_start_token_id": 248053
}

[INFO|image_processing_base.py:342] 2026-05-14 01:14:34,779 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/preprocessor_config.json
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[INFO|processing_utils.py:1122] 2026-05-14 01:14:34,833 >> loading configuration file None
[INFO|image_processing_base.py:342] 2026-05-14 01:14:34,842 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/preprocessor_config.json
[WARNING|logging.py:340] 2026-05-14 01:14:34,842 >> The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
[INFO|image_processing_base.py:342] 2026-05-14 01:14:34,892 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/preprocessor_config.json
[INFO|image_processing_base.py:385] 2026-05-14 01:14:34,892 >> Image processor Qwen2VLImageProcessor {
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_rescale": true,
  "do_resize": true,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_processor_type": "Qwen2VLImageProcessor",
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "merge_size": 2,
  "patch_size": 16,
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "longest_edge": 16777216,
    "shortest_edge": 65536
  },
  "temporal_patch_size": 2
}

[INFO|configuration_utils.py:769] 2026-05-14 01:14:34,895 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/config.json
[INFO|configuration_utils.py:847] 2026-05-14 01:14:34,901 >> Model config Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "image_token_id": 248056,
  "model_type": "qwen3_5",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 4096,
    "initializer_range": 0.02,
    "intermediate_size": 12288,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": false,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.6.0",
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 27,
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1152,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4304,
    "model_type": "qwen3_5_vision",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 4096,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_start_token_id": 248053
}

[INFO|video_processing_utils.py:684] 2026-05-14 01:14:35,403 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/video_preprocessor_config.json
[INFO|video_processing_utils.py:684] 2026-05-14 01:14:35,407 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/video_preprocessor_config.json
[INFO|video_processing_utils.py:727] 2026-05-14 01:14:35,407 >> Video processor Qwen3VLVideoProcessor {
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_rescale": true,
  "do_resize": true,
  "do_sample_frames": true,
  "fps": 2,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "max_frames": 768,
  "merge_size": 2,
  "min_frames": 4,
  "patch_size": 16,
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "return_metadata": false,
  "size": {
    "longest_edge": 234881024,
    "shortest_edge": 4096
  },
  "temporal_patch_size": 2,
  "video_processor_type": "Qwen3VLVideoProcessor"
}

[INFO|processing_utils.py:1199] 2026-05-14 01:14:35,408 >> Processor Qwen3VLProcessor:
- image_processor: Qwen2VLImageProcessor {
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_rescale": true,
  "do_resize": true,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_processor_type": "Qwen2VLImageProcessor",
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "merge_size": 2,
  "patch_size": 16,
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "longest_edge": 16777216,
    "shortest_edge": 65536
  },
  "temporal_patch_size": 2
}

- tokenizer: Qwen2Tokenizer(name_or_path='/jizhicfs/wongzhenhao/models/Qwen3.5-9B-base', vocab_size=248044, model_max_length=262144, padding_side='right', truncation_side='right', special_tokens={'eos_token': '<|endoftext|>', 'pad_token': '<|endoftext|>', 'audio_bos_token': '<|audio_start|>', 'audio_eos_token': '<|audio_end|>', 'audio_token': '<|audio_pad|>', 'image_token': '<|image_pad|>', 'video_token': '<|video_pad|>', 'vision_bos_token': '<|vision_start|>', 'vision_eos_token': '<|vision_end|>'}, added_tokens_decoder={
	248044: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248045: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248046: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248047: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248048: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248049: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248050: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248051: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248052: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248053: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248054: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248055: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248056: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248057: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248058: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248059: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248060: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248061: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248062: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248063: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248064: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248065: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248066: AddedToken("<tool_response>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248067: AddedToken("</tool_response>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248068: AddedToken("<think>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248069: AddedToken("</think>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
	248070: AddedToken("<|audio_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248071: AddedToken("<|audio_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248072: AddedToken("<tts_pad>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248073: AddedToken("<tts_text_bos>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248074: AddedToken("<tts_text_eod>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248075: AddedToken("<tts_text_bos_single>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	248076: AddedToken("<|audio_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
})
- video_processor: Qwen3VLVideoProcessor {
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_rescale": true,
  "do_resize": true,
  "do_sample_frames": true,
  "fps": 2,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "max_frames": 768,
  "merge_size": 2,
  "min_frames": 4,
  "patch_size": 16,
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "return_metadata": false,
  "size": {
    "longest_edge": 234881024,
    "shortest_edge": 4096
  },
  "temporal_patch_size": 2,
  "video_processor_type": "Qwen3VLVideoProcessor"
}


{
  "image_processor": {
    "do_convert_rgb": true,
    "do_normalize": true,
    "do_rescale": true,
    "do_resize": true,
    "image_mean": [
      0.5,
      0.5,
      0.5
    ],
    "image_processor_type": "Qwen2VLImageProcessor",
    "image_std": [
      0.5,
      0.5,
      0.5
    ],
    "merge_size": 2,
    "patch_size": 16,
    "resample": 3,
    "rescale_factor": 0.00392156862745098,
    "size": {
      "longest_edge": 16777216,
      "shortest_edge": 65536
    },
    "temporal_patch_size": 2
  },
  "processor_class": "Qwen3VLProcessor",
  "video_processor": {
    "do_convert_rgb": true,
    "do_normalize": true,
    "do_rescale": true,
    "do_resize": true,
    "do_sample_frames": true,
    "fps": 2,
    "image_mean": [
      0.5,
      0.5,
      0.5
    ],
    "image_std": [
      0.5,
      0.5,
      0.5
    ],
    "max_frames": 768,
    "merge_size": 2,
    "min_frames": 4,
    "patch_size": 16,
    "resample": 3,
    "rescale_factor": 0.00392156862745098,
    "return_metadata": false,
    "size": {
      "longest_edge": 234881024,
      "shortest_edge": 4096
    },
    "temporal_patch_size": 2,
    "video_processor_type": "Qwen3VLVideoProcessor"
  }
}

[INFO|2026-05-14 01:14:35] llamafactory.data.template:144 >> Replace eos token: <|im_end|>.
[INFO|2026-05-14 01:14:35] llamafactory.data.loader:144 >> Loading dataset /jizhicfs/wongzhenhao/omniscience/dataset/Omniscience-VQA-0513-llamafactory/omniscience_vqa.json...
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 60000 examples [00:01, 42679.29 examples/s]
Generating train split: 60000 examples [00:01, 41062.34 examples/s]

Converting format of dataset (num_proc=16):   0%|          | 0/60000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16):   1%|          | 311/60000 [00:00<01:51, 536.43 examples/s]
Converting format of dataset (num_proc=16):  15%|█▍        | 8938/60000 [00:00<00:02, 17300.66 examples/s]
Converting format of dataset (num_proc=16):  30%|██▉       | 17849/60000 [00:00<00:01, 32910.27 examples/s]
Converting format of dataset (num_proc=16):  46%|████▌     | 27581/60000 [00:00<00:00, 48021.85 examples/s]
Converting format of dataset (num_proc=16):  62%|██████▏   | 36928/60000 [00:00<00:00, 59402.73 examples/s]
Converting format of dataset (num_proc=16):  76%|███████▋  | 45763/60000 [00:01<00:00, 66468.05 examples/s]
Converting format of dataset (num_proc=16):  91%|█████████ | 54482/60000 [00:01<00:00, 66316.19 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 60000/60000 [00:01<00:00, 39916.09 examples/s]

Running tokenizer on dataset (num_proc=16):   0%|          | 0/60000 [00:00<?, ? examples/s]
Running tokenizer on dataset (num_proc=16):   2%|▏         | 1000/60000 [00:39<39:14, 25.06 examples/s]
Running tokenizer on dataset (num_proc=16):   3%|▎         | 2000/60000 [00:41<16:42, 57.83 examples/s]
Running tokenizer on dataset (num_proc=16):   5%|▌         | 3000/60000 [00:42<09:18, 102.05 examples/s]
Running tokenizer on dataset (num_proc=16):   7%|▋         | 4000/60000 [00:46<07:03, 132.23 examples/s]
Running tokenizer on dataset (num_proc=16):   8%|▊         | 5000/60000 [00:48<05:07, 178.92 examples/s]
Running tokenizer on dataset (num_proc=16):   8%|▊         | 5000/60000 [01:04<05:07, 178.92 examples/s]
Running tokenizer on dataset (num_proc=16):  10%|█         | 6000/60000 [01:19<12:45, 70.57 examples/s] 
Running tokenizer on dataset (num_proc=16):  13%|█▎        | 8000/60000 [01:19<06:17, 137.62 examples/s]
Running tokenizer on dataset (num_proc=16):  15%|█▌        | 9000/60000 [01:26<06:06, 139.01 examples/s]
Running tokenizer on dataset (num_proc=16):  17%|█▋        | 10000/60000 [01:27<04:27, 186.77 examples/s]
Running tokenizer on dataset (num_proc=16):  18%|█▊        | 11000/60000 [01:29<03:45, 217.55 examples/s]
Running tokenizer on dataset (num_proc=16):  20%|██        | 12000/60000 [01:31<02:59, 267.94 examples/s]
Running tokenizer on dataset (num_proc=16):  22%|██▏       | 13000/60000 [01:35<03:08, 249.64 examples/s]
Running tokenizer on dataset (num_proc=16):  23%|██▎       | 14000/60000 [01:36<02:12, 348.20 examples/s]
Running tokenizer on dataset (num_proc=16):  25%|██▌       | 15000/60000 [01:37<01:44, 430.58 examples/s]
Running tokenizer on dataset (num_proc=16):  27%|██▋       | 16000/60000 [01:37<01:20, 544.51 examples/s]
Running tokenizer on dataset (num_proc=16):  28%|██▊       | 17000/60000 [01:39<01:14, 580.91 examples/s]
Running tokenizer on dataset (num_proc=16):  30%|███       | 18000/60000 [01:42<01:35, 440.62 examples/s]
Running tokenizer on dataset (num_proc=16):  32%|███▏      | 19000/60000 [01:48<02:10, 313.32 examples/s]
Running tokenizer on dataset (num_proc=16):  33%|███▎      | 20000/60000 [01:48<01:32, 431.92 examples/s]
Running tokenizer on dataset (num_proc=16):  35%|███▌      | 21000/60000 [01:55<02:21, 276.29 examples/s]
Running tokenizer on dataset (num_proc=16):  37%|███▋      | 22000/60000 [01:56<01:53, 334.19 examples/s]
Running tokenizer on dataset (num_proc=16):  38%|███▊      | 23000/60000 [01:57<01:32, 398.03 examples/s]
Running tokenizer on dataset (num_proc=16):  40%|████      | 24000/60000 [01:59<01:20, 448.38 examples/s]
Running tokenizer on dataset (num_proc=16):  42%|████▏     | 25000/60000 [02:02<01:27, 401.11 examples/s]
Running tokenizer on dataset (num_proc=16):  42%|████▏     | 25000/60000 [02:14<01:27, 401.11 examples/s]
Running tokenizer on dataset (num_proc=16):  43%|████▎     | 25750/60000 [02:25<05:14, 108.89 examples/s]
Running tokenizer on dataset (num_proc=16):  44%|████▍     | 26500/60000 [02:26<04:02, 138.09 examples/s]
Running tokenizer on dataset (num_proc=16):  45%|████▌     | 27250/60000 [02:28<03:23, 160.69 examples/s]
Running tokenizer on dataset (num_proc=16):  47%|████▋     | 28000/60000 [02:30<02:47, 191.05 examples/s]
Running tokenizer on dataset (num_proc=16):  48%|████▊     | 29000/60000 [02:32<02:04, 248.09 examples/s]
Running tokenizer on dataset (num_proc=16):  48%|████▊     | 29000/60000 [02:44<02:04, 248.09 examples/s]
Running tokenizer on dataset (num_proc=16):  50%|█████     | 30000/60000 [02:45<03:32, 141.09 examples/s]
Running tokenizer on dataset (num_proc=16):  52%|█████▏    | 31000/60000 [02:50<03:02, 158.94 examples/s]
Running tokenizer on dataset (num_proc=16):  53%|█████▎    | 32000/60000 [02:53<02:27, 189.53 examples/s]
Running tokenizer on dataset (num_proc=16):  55%|█████▌    | 33000/60000 [02:53<01:39, 270.74 examples/s]
Running tokenizer on dataset (num_proc=16):  57%|█████▋    | 34000/60000 [02:55<01:18, 332.96 examples/s]
Running tokenizer on dataset (num_proc=16):  60%|██████    | 36000/60000 [03:01<01:15, 319.19 examples/s]
Running tokenizer on dataset (num_proc=16):  62%|██████▏   | 37000/60000 [03:06<01:19, 290.83 examples/s]
Running tokenizer on dataset (num_proc=16):  63%|██████▎   | 38000/60000 [03:10<01:22, 268.26 examples/s]
Running tokenizer on dataset (num_proc=16):  65%|██████▌   | 39000/60000 [03:10<00:58, 361.02 examples/s]
Running tokenizer on dataset (num_proc=16):  67%|██████▋   | 40000/60000 [03:24<01:55, 173.58 examples/s]
Running tokenizer on dataset (num_proc=16):  68%|██████▊   | 40750/60000 [03:29<01:52, 170.36 examples/s]
Running tokenizer on dataset (num_proc=16):  68%|██████▊   | 40750/60000 [03:44<01:52, 170.36 examples/s]
Running tokenizer on dataset (num_proc=16):  70%|██████▉   | 41750/60000 [04:00<04:12, 72.32 examples/s] 
Running tokenizer on dataset (num_proc=16):  71%|███████▏  | 42750/60000 [04:04<03:05, 92.90 examples/s]
Running tokenizer on dataset (num_proc=16):  73%|███████▎  | 43750/60000 [04:07<02:17, 117.82 examples/s]
Running tokenizer on dataset (num_proc=16):  75%|███████▍  | 44750/60000 [04:09<01:39, 153.60 examples/s]
Running tokenizer on dataset (num_proc=16):  76%|███████▋  | 45750/60000 [04:11<01:12, 195.79 examples/s]
Running tokenizer on dataset (num_proc=16):  78%|███████▊  | 46750/60000 [04:16<01:05, 202.53 examples/s]
Running tokenizer on dataset (num_proc=16):  80%|███████▉  | 47750/60000 [04:21<01:00, 203.83 examples/s]
Running tokenizer on dataset (num_proc=16):  81%|████████▏ | 48750/60000 [04:29<01:08, 164.32 examples/s]
Running tokenizer on dataset (num_proc=16):  83%|████████▎ | 49750/60000 [04:30<00:44, 227.99 examples/s]
Running tokenizer on dataset (num_proc=16):  85%|████████▍ | 50750/60000 [04:37<00:47, 196.15 examples/s]
Running tokenizer on dataset (num_proc=16):  86%|████████▋ | 51750/60000 [04:43<00:46, 178.01 examples/s]
Running tokenizer on dataset (num_proc=16):  88%|████████▊ | 52500/60000 [04:53<00:56, 133.34 examples/s]
Running tokenizer on dataset (num_proc=16):  89%|████████▉ | 53250/60000 [04:56<00:43, 156.89 examples/s]
Running tokenizer on dataset (num_proc=16):  90%|█████████ | 54000/60000 [05:04<00:46, 130.37 examples/s]
Running tokenizer on dataset (num_proc=16):  91%|█████████▏| 54750/60000 [05:05<00:31, 168.66 examples/s]
Running tokenizer on dataset (num_proc=16):  92%|█████████▎| 55500/60000 [05:07<00:21, 205.06 examples/s]
Running tokenizer on dataset (num_proc=16):  94%|█████████▍| 56250/60000 [05:10<00:17, 212.68 examples/s]
Running tokenizer on dataset (num_proc=16):  95%|█████████▌| 57000/60000 [05:22<00:24, 123.63 examples/s]
Running tokenizer on dataset (num_proc=16):  96%|█████████▋| 57750/60000 [05:29<00:18, 121.37 examples/s]
Running tokenizer on dataset (num_proc=16):  98%|█████████▊| 58500/60000 [05:29<00:08, 170.42 examples/s]
Running tokenizer on dataset (num_proc=16):  99%|█████████▉| 59250/60000 [05:38<00:05, 129.30 examples/s]
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [05:46<00:00, 118.43 examples/s]
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [05:46<00:00, 173.17 examples/s]
training example:
input_ids:
[248045, 8678, 198, 27382, 1386, 279, 1328, 8851, 5721, 303, 279, 2099, 321, 1179, 4087, 279, 3296, 13, 248046, 198, 248045, 846, 198, 248053, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248056, 248054, 198, 22365, 3759, 369, 23098, 310, 279, 15725, 7772, 30, 248046, 198, 248045, 74455, 198, 89408, 25, 561, 2306, 3559, 279, 15725, 830, 34150, 15339, 5036, 59, 12564, 14717, 59, 1272, 90, 37, 2662, 73150, 16, 3307, 3, 318, 19038, 90765, 198, 27, 8944, 22849, 59, 12564, 14717, 59, 1272, 90, 37, 2662, 73150, 16, 3307, 3, 510, 8944, 29, 248046, 198]
inputs:
<|im_start|>system
Identify the key visual evidence in the image and then answer the question.<|im_end|>
<|im_start|>user
<|vision_start|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|vision_end|>
Which variable is mapped to the horizontal axis?<|im_end|>
<|im_start|>assistant
Evidence: The label below the horizontal x-axis reads "$\lambda_{\text{FEL\_1}}$ (nm)".
<answer>$\lambda_{\text{FEL\_1}}$</answer><|im_end|>

label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 89408, 25, 561, 2306, 3559, 279, 15725, 830, 34150, 15339, 5036, 59, 12564, 14717, 59, 1272, 90, 37, 2662, 73150, 16, 3307, 3, 318, 19038, 90765, 198, 27, 8944, 22849, 59, 12564, 14717, 59, 1272, 90, 37, 2662, 73150, 16, 3307, 3, 510, 8944, 29, 248046, 198]
labels:
Evidence: The label below the horizontal x-axis reads "$\lambda_{\text{FEL\_1}}$ (nm)".
<answer>$\lambda_{\text{FEL\_1}}$</answer><|im_end|>

[INFO|configuration_utils.py:769] 2026-05-14 01:20:28,119 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/config.json
[INFO|configuration_utils.py:847] 2026-05-14 01:20:28,127 >> Model config Qwen3_5Config {
  "architectures": [
    "Qwen3_5ForConditionalGeneration"
  ],
  "image_token_id": 248056,
  "model_type": "qwen3_5",
  "text_config": {
    "attention_bias": false,
    "attention_dropout": 0.0,
    "attn_output_gate": true,
    "bos_token_id": null,
    "dtype": "bfloat16",
    "eos_token_id": 248044,
    "full_attention_interval": 4,
    "head_dim": 256,
    "hidden_act": "silu",
    "hidden_size": 4096,
    "initializer_range": 0.02,
    "intermediate_size": 12288,
    "layer_types": [
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention",
      "linear_attention",
      "linear_attention",
      "linear_attention",
      "full_attention"
    ],
    "linear_conv_kernel_dim": 4,
    "linear_key_head_dim": 128,
    "linear_num_key_heads": 16,
    "linear_num_value_heads": 32,
    "linear_value_head_dim": 128,
    "mamba_ssm_dtype": "float32",
    "max_position_embeddings": 262144,
    "mlp_only_layers": [],
    "model_type": "qwen3_5_text",
    "mtp_num_hidden_layers": 1,
    "mtp_use_dedicated_embeddings": false,
    "num_attention_heads": 16,
    "num_hidden_layers": 32,
    "num_key_value_heads": 4,
    "pad_token_id": null,
    "partial_rotary_factor": 0.25,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "mrope_interleaved": true,
      "mrope_section": [
        11,
        11,
        10
      ],
      "partial_rotary_factor": 0.25,
      "rope_theta": 10000000,
      "rope_type": "default"
    },
    "tie_word_embeddings": false,
    "use_cache": true,
    "vocab_size": 248320
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.6.0",
  "video_token_id": 248057,
  "vision_config": {
    "deepstack_visual_indexes": [],
    "depth": 27,
    "hidden_act": "gelu_pytorch_tanh",
    "hidden_size": 1152,
    "in_channels": 3,
    "initializer_range": 0.02,
    "intermediate_size": 4304,
    "model_type": "qwen3_5_vision",
    "num_heads": 16,
    "num_position_embeddings": 2304,
    "out_hidden_size": 4096,
    "patch_size": 16,
    "spatial_merge_size": 2,
    "temporal_patch_size": 2
  },
  "vision_end_token_id": 248054,
  "vision_start_token_id": 248053
}

[INFO|2026-05-14 01:20:28] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "binding_blocks" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
[WARNING] Field "output" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "binding_blocks" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
[WARNING] Field "output" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "[WARNING] Field "ir.Type". Child types should not re-register inherited fields.
span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType[WARNING] Field "" duplicates an ancestor field in "ir.Typespan". Child types should not re-register inherited fields." in type "ir.FuncType
" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "[WARNING] Field "binding_blocks[WARNING] Field "binding_blocks" in type "binding_blocks[WARNING] Field "" in type "" in type "script.ir_builder.relax.FunctionFramescript.ir_builder.relax.FunctionFramescript.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "binding_blocksscript.ir_builder.relax.SeqExprFrame" duplicates an ancestor field in "". Child types should not re-register inherited fields." duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame" in type "
". Child types should not re-register inherited fields.script.ir_builder.relax.FunctionFramescript.ir_builder.relax.SeqExprFrame
" duplicates an ancestor field in "[WARNING] Field "". Child types should not re-register inherited fields.script.ir_builder.relax.SeqExprFrame[WARNING] Field "output
". Child types should not re-register inherited fields.output" in type "" in type "script.ir_builder.relax.FunctionFrame[WARNING] Field "
script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "" duplicates an ancestor field in "output[WARNING] Field "script.ir_builder.relax.SeqExprFramescript.ir_builder.relax.SeqExprFrame" in type "output". Child types should not re-register inherited fields." in type "". Child types should not re-register inherited fields.script.ir_builder.relax.FunctionFrame
script.ir_builder.relax.FunctionFrame
" duplicates an ancestor field in "" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFramescript.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.". Child types should not re-register inherited fields.

[WARNING] Field "binding_blocks" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
[WARNING] Field "output" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
[WARNING] Field "span" in type "ir.TupleType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.FuncType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "span" in type "ir.TensorMapType" duplicates an ancestor field in "ir.Type". Child types should not re-register inherited fields.
[WARNING] Field "binding_blocks" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
[WARNING] Field "output" in type "script.ir_builder.relax.FunctionFrame" duplicates an ancestor field in "script.ir_builder.relax.SeqExprFrame". Child types should not re-register inherited fields.
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TupleType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.FuncType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'span' in 'ir.TensorMapType' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'name' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'tag' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'attrs' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'shape' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'dtype' in 'relax.TEPlaceholderOp' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "[WARNING] Field "policy_typepolicy_type" in type "" in type "tl.GemmSPWarpPolicytl.GemmSPWarpPolicy" duplicates an ancestor field in "" duplicates an ancestor field in "tl.GemmWarpPolicytl.GemmWarpPolicy". Child types should not re-register inherited fields.". Child types should not re-register inherited fields.

[WARNING] Field "[WARNING] Field "m_warpm_warp" in type "" in type "tl.GemmSPWarpPolicytl.GemmSPWarpPolicy" duplicates an ancestor field in "" duplicates an ancestor field in "tl.GemmWarpPolicytl.GemmWarpPolicy". Child types should not re-register inherited fields.". Child types should not re-register inherited fields.

[WARNING] Field "n_warp[WARNING] Field "" in type "n_warptl.GemmSPWarpPolicy" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "policy_type" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "m_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
[WARNING] Field "n_warp" in type "tl.GemmSPWarpPolicy" duplicates an ancestor field in "tl.GemmWarpPolicy". Child types should not re-register inherited fields.
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'policy_type' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'm_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
/jizhicfs/wongzhenhao/miniconda/envs/llama_factory/lib/python3.11/site-packages/tvm_ffi/registry.py:85: UserWarning: Field 'n_warp' in 'tl.GemmSPWarpPolicy' duplicates an ancestor field. Child types should not re-register inherited fields.
  info = core._register_object_by_index(type_index, cls)
[INFO|2026-05-14 01:20:36] llamafactory.model.model_utils.liger_kernel:144 >> Liger kernel has been applied to the model.
[INFO|modeling_utils.py:748] 2026-05-14 01:20:37,927 >> loading weights file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/model.safetensors.index.json
[INFO|modeling_utils.py:834] 2026-05-14 01:20:40,256 >> Since the `dtype` attribute can't be found in model's config object, will use dtype=torch.bfloat16 as derived from model's weights
[INFO|modeling_utils.py:3656] 2026-05-14 01:20:40,256 >> Detected DeepSpeed ZeRO-3: activating zero.init() for this model
[INFO|configuration_utils.py:1027] 2026-05-14 01:20:40,276 >> Generate config GenerationConfig {
  "output_attentions": false,
  "output_hidden_states": false,
  "use_cache": false
}

[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
[INFO|utils.py:411] 2026-05-14 01:21:06,942 >> Generation config file not found, using a generation config created from the model config.
[INFO|configuration_utils.py:978] 2026-05-14 01:21:06,944 >> loading configuration file /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base/config.json
[INFO|configuration_utils.py:1027] 2026-05-14 01:21:06,944 >> Generate config GenerationConfig {}

[INFO|dynamic_module_utils.py:406] 2026-05-14 01:21:06,944 >> Could not locate the custom_generate/generate.py inside /jizhicfs/wongzhenhao/models/Qwen3.5-9B-base.
[INFO|2026-05-14 01:21:06] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
[INFO|2026-05-14 01:21:06] llamafactory.model.patcher:144 >> Patched Qwen3.5 decoder forward to support cu_seqlens input only patch when do training.
[INFO|2026-05-14 01:21:06] llamafactory.model.model_utils.attention:144 >> Using FlashAttention-2 for faster training and inference.
[INFO|2026-05-14 01:21:06] llamafactory.model.adapter:144 >> DeepSpeed ZeRO3 detected, remaining trainable params in float32.
[INFO|2026-05-14 01:21:06] llamafactory.model.adapter:144 >> Fine-tuning method: Full
[INFO|2026-05-14 01:21:06] llamafactory.model.model_utils.visual:144 >> Set vision model not trainable: ['visual.pos_embed', 'visual.patch_embed', 'visual.blocks'].
[INFO|2026-05-14 01:21:06] llamafactory.model.model_utils.visual:144 >> Set multi model projector not trainable: ['model.visual.merger'].
[INFO|2026-05-14 01:21:06] llamafactory.model.loader:144 >> trainable params: 8,953,803,264 || all params: 9,409,813,744 || trainable%: 95.1539
Detected kernel version 5.4.241, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
[WARNING|trainer_utils.py:1238] 2026-05-14 01:21:06,979 >> The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
Gradient accumulation steps mismatch: GradientAccumulationPlugin has 1, DeepSpeed config has 8. Using DeepSpeed's value.
Stage 3 initialize beginning
MA 2.19 GB         Max_MA 6.06 GB         CA 2.24 GB         Max_CA 6 GB 
CPU Virtual Memory:  used = 787.73 GB, percent = 34.8%
DeepSpeedZeRoOffload initialize [begin]
MA 2.19 GB         Max_MA 2.19 GB         CA 2.24 GB         Max_CA 2 GB 
CPU Virtual Memory:  used = 787.73 GB, percent = 34.8%
Parameter Offload - Persistent parameters statistics: param_count = 398, numel = 1469680
DeepSpeedZeRoOffload initialize [end]
MA 2.19 GB         Max_MA 2.19 GB         CA 2.24 GB         Max_CA 2 GB 
CPU Virtual Memory:  used = 787.73 GB, percent = 34.8%
Before creating fp16 partitions
MA 2.19 GB         Max_MA 2.19 GB         CA 2.24 GB         Max_CA 2 GB 
CPU Virtual Memory:  used = 787.73 GB, percent = 34.8%
After creating fp16 partitions: 2
MA 2.19 GB         Max_MA 2.19 GB         CA 2.23 GB         Max_CA 2 GB 
CPU Virtual Memory:  used = 787.75 GB, percent = 34.8%
Before creating fp32 partitions
MA 2.19 GB         Max_MA 2.19 GB         CA 2.23 GB         Max_CA 2 GB 
CPU Virtual Memory:  used = 787.75 GB, percent = 34.8%
After creating fp32 partitions
MA 6.36 GB         Max_MA 10.53 GB         CA 10.57 GB         Max_CA 11 GB 
CPU Virtual Memory:  used = 787.13 GB, percent = 34.7%
Before initializing optimizer states
MA 6.36 GB         Max_MA 6.36 GB         CA 10.57 GB         Max_CA 11 GB 
CPU Virtual Memory:  used = 787.17 GB, percent = 34.7%
After initializing optimizer states
MA 6.36 GB         Max_MA 10.53 GB         CA 10.57 GB         Max_CA 11 GB 
CPU Virtual Memory:  used = 787.45 GB, percent = 34.8%
After initializing ZeRO optimizer
MA 8.48 GB         Max_MA 12.27 GB         CA 12.46 GB         Max_CA 12 GB 
CPU Virtual Memory:  used = 788.95 GB, percent = 34.8%
[INFO|trainer.py:1469] 2026-05-14 01:21:14,473 >> ***** Running training *****
[INFO|trainer.py:1470] 2026-05-14 01:21:14,473 >>   Num examples = 60,000
[INFO|trainer.py:1471] 2026-05-14 01:21:14,473 >>   Num Epochs = 5
[INFO|trainer.py:1472] 2026-05-14 01:21:14,473 >>   Num update steps per epoch = 938
[INFO|trainer.py:1473] 2026-05-14 01:21:14,473 >>   Instantaneous batch size per device = 1
[INFO|trainer.py:1476] 2026-05-14 01:21:14,473 >>   Total train batch size (w. parallel, distributed & accumulation) = 64
[INFO|trainer.py:1477] 2026-05-14 01:21:14,473 >>   Gradient Accumulation steps = 8
[INFO|trainer.py:1478] 2026-05-14 01:21:14,473 >>   Total optimization steps = 4,690
[INFO|trainer.py:1479] 2026-05-14 01:21:14,475 >>   Number of trainable parameters = 8,953,803,264

  0%|          | 0/4690 [00:00<?, ?it/s][WARNING|logging.py:340] 2026-05-14 01:21:14,877 >> `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!
[transformers] `use_return_dict` is deprecated! Use `return_dict` instead!

  0%|          | 1/4690 [00:54<70:21:08, 54.01s/it]
  0%|          | 2/4690 [01:26<54:13:36, 41.64s/it]
  0%|          | 3/4690 [01:59<49:00:19, 37.64s/it]
  0%|          | 4/4690 [02:32<46:39:39, 35.85s/it]
  0%|          | 5/4690 [03:05<45:11:34, 34.73s/it]
  0%|          | 6/4690 [03:38<44:25:08, 34.14s/it]
  0%|          | 7/4690 [04:11<43:47:59, 33.67s/it]
  0%|          | 8/4690 [04:44<43:38:09, 33.55s/it]
  0%|          | 9/4690 [05:17<43:30:59, 33.47s/it]
  0%|          | 10/4690 [05:50<43:14:52, 33.27s/it]
                                                    
{'loss': '1.131', 'grad_norm': '14.72', 'learning_rate': '6.383e-07', 'epoch': '0.01067'}

  0%|          | 10/4690 [05:50<43:14:52, 33.27s/it]
  0%|          | 11/4690 [06:23<43:10:27, 33.22s/it]
  0%|          | 12/4690 [06:56<42:59:48, 33.09s/it]
  0%|          | 13/4690 [07:29<42:51:27, 32.99s/it]
  0%|          | 14/4690 [08:02<42:41:10, 32.86s/it]
  0%|          | 15/4690 [08:34<42:32:43, 32.76s/it]
  0%|          | 16/4690 [09:07<42:26:06, 32.68s/it]
  0%|          | 17/4690 [09:40<42:32:23, 32.77s/it]
  0%|          | 18/4690 [10:12<42:27:16, 32.71s/it]
  0%|          | 19/4690 [10:45<42:22:54, 32.66s/it]
  0%|          | 20/4690 [11:17<42:20:22, 32.64s/it]
                                                    
{'loss': '0.8277', 'grad_norm': '19.79', 'learning_rate': '1.348e-06', 'epoch': '0.02133'}

  0%|          | 20/4690 [11:17<42:20:22, 32.64s/it]
  0%|          | 21/4690 [11:50<42:19:18, 32.63s/it]
  0%|          | 22/4690 [12:23<42:18:37, 32.63s/it]
  0%|          | 23/4690 [12:55<42:16:59, 32.62s/it]
  1%|          | 24/4690 [13:28<42:22:41, 32.70s/it]
  1%|          | 25/4690 [14:01<42:28:33, 32.78s/it]
  1%|          | 26/4690 [14:34<42:24:33, 32.73s/it]
  1%|          | 27/4690 [15:06<42:20:58, 32.70s/it]
  1%|          | 28/4690 [15:39<42:23:18, 32.73s/it]
  1%|          | 29/4690 [16:12<42:20:23, 32.70s/it]
  1%|          | 30/4690 [16:44<42:21:28, 32.72s/it]
                                                    
{'loss': '0.5475', 'grad_norm': '4.009', 'learning_rate': '2.057e-06', 'epoch': '0.032'}

  1%|          | 30/4690 [16:44<42:21:28, 32.72s/it]
  1%|          | 31/4690 [17:17<42:18:27, 32.69s/it]
  1%|          | 32/4690 [17:50<42:24:26, 32.78s/it]
  1%|          | 33/4690 [18:23<42:18:22, 32.70s/it]
  1%|          | 34/4690 [18:55<42:21:34, 32.75s/it]
  1%|          | 35/4690 [19:29<42:31:20, 32.89s/it]
  1%|          | 36/4690 [20:01<42:21:37, 32.77s/it]
  1%|          | 37/4690 [20:34<42:17:56, 32.73s/it]
  1%|          | 38/4690 [21:07<42:28:15, 32.87s/it]
  1%|          | 39/4690 [21:40<42:23:25, 32.81s/it]
  1%|          | 40/4690 [22:13<42:25:36, 32.85s/it]
                                                    
{'loss': '0.497', 'grad_norm': '3.497', 'learning_rate': '2.766e-06', 'epoch': '0.04267'}

  1%|          | 40/4690 [22:13<42:25:36, 32.85s/it]
  1%|          | 41/4690 [22:46<42:35:22, 32.98s/it]
  1%|          | 42/4690 [23:18<42:26:17, 32.87s/it]
  1%|          | 43/4690 [23:51<42:19:40, 32.79s/it]
  1%|          | 44/4690 [24:24<42:18:10, 32.78s/it]
  1%|          | 45/4690 [24:56<42:15:46, 32.75s/it]
  1%|          | 46/4690 [25:29<42:20:30, 32.82s/it]
  1%|          | 47/4690 [26:02<42:20:30, 32.83s/it]
  1%|          | 48/4690 [26:35<42:18:16, 32.81s/it]
  1%|          | 49/4690 [27:08<42:28:34, 32.95s/it]
  1%|          | 50/4690 [27:41<42:21:34, 32.87s/it]
                                                    
{'loss': '0.4999', 'grad_norm': '3.497', 'learning_rate': '3.475e-06', 'epoch': '0.05333'}

  1%|          | 50/4690 [27:41<42:21:34, 32.87s/it]
  1%|          | 51/4690 [28:14<42:17:28, 32.82s/it]
  1%|          | 52/4690 [28:47<42:22:23, 32.89s/it]
  1%|          | 53/4690 [29:20<42:20:08, 32.87s/it]
  1%|          | 54/4690 [29:52<42:19:57, 32.87s/it]
  1%|          | 55/4690 [30:25<42:13:51, 32.80s/it]
  1%|          | 56/4690 [30:59<42:27:18, 32.98s/it]
  1%|          | 57/4690 [31:32<42:33:10, 33.07s/it]
  1%|          | 58/4690 [32:05<42:31:19, 33.05s/it]
  1%|▏         | 59/4690 [32:38<42:26:57, 33.00s/it]
  1%|▏         | 60/4690 [33:10<42:15:13, 32.85s/it]
                                                    
{'loss': '0.4771', 'grad_norm': '3.303', 'learning_rate': '4.184e-06', 'epoch': '0.064'}

  1%|▏         | 60/4690 [33:10<42:15:13, 32.85s/it]
  1%|▏         | 61/4690 [33:43<42:19:37, 32.92s/it]
  1%|▏         | 62/4690 [34:16<42:11:42, 32.82s/it]
  1%|▏         | 63/4690 [34:49<42:09:05, 32.80s/it]
  1%|▏         | 64/4690 [35:21<42:01:35, 32.71s/it]
  1%|▏         | 65/4690 [35:54<42:00:48, 32.70s/it]
  1%|▏         | 66/4690 [36:27<42:04:12, 32.75s/it]
  1%|▏         | 67/4690 [36:59<41:59:10, 32.70s/it]
  1%|▏         | 68/4690 [37:32<41:58:14, 32.69s/it]
  1%|▏         | 69/4690 [38:04<41:53:48, 32.64s/it]
  1%|▏         | 70/4690 [38:37<41:55:10, 32.66s/it]
                                                    
{'loss': '0.4837', 'grad_norm': '3.568', 'learning_rate': '4.894e-06', 'epoch': '0.07467'}

  1%|▏         | 70/4690 [38:37<41:55:10, 32.66s/it]
  2%|▏         | 71/4690 [39:10<41:54:01, 32.66s/it]
  2%|▏         | 72/4690 [39:42<41:46:03, 32.56s/it]
  2%|▏         | 73/4690 [40:15<41:56:17, 32.70s/it]
  2%|▏         | 74/4690 [40:48<41:53:01, 32.67s/it]
  2%|▏         | 75/4690 [41:20<41:48:32, 32.61s/it]
  2%|▏         | 76/4690 [41:53<42:01:20, 32.79s/it]
  2%|▏         | 77/4690 [42:26<41:59:07, 32.77s/it]
  2%|▏         | 78/4690 [43:00<42:14:12, 32.97s/it]
  2%|▏         | 79/4690 [43:32<42:12:07, 32.95s/it]
  2%|▏         | 80/4690 [44:05<42:11:40, 32.95s/it]
                                                    
{'loss': '0.4731', 'grad_norm': '3.229', 'learning_rate': '5.603e-06', 'epoch': '0.08533'}

  2%|▏         | 80/4690 [44:05<42:11:40, 32.95s/it]
  2%|▏         | 81/4690 [44:38<42:14:07, 32.99s/it]
  2%|▏         | 82/4690 [45:12<42:22:13, 33.10s/it]
  2%|▏         | 83/4690 [45:45<42:11:46, 32.97s/it]
  2%|▏         | 84/4690 [46:17<42:07:39, 32.93s/it]
  2%|▏         | 85/4690 [46:50<42:02:07, 32.86s/it]
  2%|▏         | 86/4690 [47:23<41:58:31, 32.82s/it]
  2%|▏         | 87/4690 [47:56<42:07:30, 32.95s/it]
  2%|▏         | 88/4690 [48:29<42:01:34, 32.88s/it]
  2%|▏         | 89/4690 [49:02<42:01:52, 32.89s/it]
  2%|▏         | 90/4690 [49:34<41:59:02, 32.86s/it]
                                                    
{'loss': '0.4791', 'grad_norm': '2.999', 'learning_rate': '6.312e-06', 'epoch': '0.096'}

  2%|▏         | 90/4690 [49:34<41:59:02, 32.86s/it]
  2%|▏         | 91/4690 [50:07<41:52:09, 32.77s/it]
  2%|▏         | 92/4690 [50:39<41:44:51, 32.69s/it]
  2%|▏         | 93/4690 [51:12<41:45:28, 32.70s/it]
  2%|▏         | 94/4690 [51:45<41:50:45, 32.78s/it]
  2%|▏         | 95/4690 [52:18<41:53:00, 32.81s/it]
  2%|▏         | 96/4690 [52:51<42:01:08, 32.93s/it]
  2%|▏         | 97/4690 [53:24<41:58:09, 32.90s/it]
  2%|▏         | 98/4690 [53:57<42:02:12, 32.96s/it]
  2%|▏         | 99/4690 [54:30<42:04:54, 33.00s/it]
  2%|▏         | 100/4690 [55:03<41:58:12, 32.92s/it]
                                                     
{'loss': '0.4781', 'grad_norm': '3.34', 'learning_rate': '7.021e-06', 'epoch': '0.1067'}

  2%|▏         | 100/4690 [55:03<41:58:12, 32.92s/it]
  2%|▏         | 101/4690 [55:36<41:55:14, 32.89s/it]
  2%|▏         | 102/4690 [56:09<41:59:28, 32.95s/it]
  2%|▏         | 103/4690 [56:42<41:54:18, 32.89s/it]
  2%|▏         | 104/4690 [57:14<41:47:10, 32.80s/it]
  2%|▏         | 105/4690 [57:47<41:47:35, 32.81s/it]
  2%|▏         | 106/4690 [58:20<41:49:09, 32.84s/it]
  2%|▏         | 107/4690 [58:53<41:48:19, 32.84s/it]
  2%|▏         | 108/4690 [59:26<41:43:28, 32.78s/it]
  2%|▏         | 109/4690 [59:58<41:42:16, 32.77s/it]
  2%|▏         | 110/4690 [1:00:31<41:50:22, 32.89s/it]
                                                       
{'loss': '0.4914', 'grad_norm': '2.705', 'learning_rate': '7.73e-06', 'epoch': '0.1173'}

  2%|▏         | 110/4690 [1:00:31<41:50:22, 32.89s/it]
  2%|▏         | 111/4690 [1:01:04<41:43:08, 32.80s/it]
  2%|▏         | 112/4690 [1:01:37<41:41:09, 32.78s/it]
  2%|▏         | 113/4690 [1:02:10<41:40:07, 32.77s/it]
  2%|▏         | 114/4690 [1:02:42<41:39:40, 32.78s/it]
  2%|▏         | 115/4690 [1:03:15<41:46:18, 32.87s/it]
  2%|▏         | 116/4690 [1:03:48<41:36:13, 32.74s/it]
  2%|▏         | 117/4690 [1:04:21<41:36:13, 32.75s/it]
  3%|▎         | 118/4690 [1:04:53<41:36:20, 32.76s/it]
  3%|▎         | 119/4690 [1:05:26<41:34:23, 32.74s/it]
  3%|▎         | 120/4690 [1:05:59<41:45:18, 32.89s/it]
                                                       
{'loss': '0.4856', 'grad_norm': '2.903', 'learning_rate': '8.44e-06', 'epoch': '0.128'}

  3%|▎         | 120/4690 [1:05:59<41:45:18, 32.89s/it]
  3%|▎         | 121/4690 [1:06:32<41:46:47, 32.92s/it]
  3%|▎         | 122/4690 [1:07:05<41:48:02, 32.94s/it]
  3%|▎         | 123/4690 [1:07:38<41:45:34, 32.92s/it]
  3%|▎         | 124/4690 [1:08:10<41:31:29, 32.74s/it]
  3%|▎         | 125/4690 [1:08:43<41:31:26, 32.75s/it]
  3%|▎         | 126/4690 [1:09:16<41:37:11, 32.83s/it]
  3%|▎         | 127/4690 [1:09:49<41:44:19, 32.93s/it]
  3%|▎         | 128/4690 [1:10:22<41:35:26, 32.82s/it]
  3%|▎         | 129/4690 [1:10:55<41:33:03, 32.80s/it]
  3%|▎         | 130/4690 [1:11:28<41:32:54, 32.80s/it]
                                                       
{'loss': '0.4959', 'grad_norm': '2.849', 'learning_rate': '9.149e-06', 'epoch': '0.1387'}

  3%|▎         | 130/4690 [1:11:28<41:32:54, 32.80s/it]
  3%|▎         | 131/4690 [1:12:01<41:41:26, 32.92s/it]
  3%|▎         | 132/4690 [1:12:33<41:27:34, 32.75s/it]
  3%|▎         | 133/4690 [1:13:06<41:38:37, 32.90s/it]
  3%|▎         | 134/4690 [1:13:39<41:37:49, 32.89s/it]
  3%|▎         | 135/4690 [1:14:12<41:32:09, 32.83s/it]
  3%|▎         | 136/4690 [1:14:44<41:24:21, 32.73s/it]
  3%|▎         | 137/4690 [1:15:17<41:21:25, 32.70s/it]
  3%|▎         | 138/4690 [1:15:50<41:22:14, 32.72s/it]
  3%|▎         | 139/4690 [1:16:23<41:32:58, 32.87s/it]
  3%|▎         | 140/4690 [1:16:56<41:29:28, 32.83s/it]
                                                       
{'loss': '0.4964', 'grad_norm': '3.19', 'learning_rate': '9.858e-06', 'epoch': '0.1493'}

  3%|▎         | 140/4690 [1:16:56<41:29:28, 32.83s/it]
  3%|▎         | 141/4690 [1:17:29<41:28:32, 32.82s/it]
  3%|▎         | 142/4690 [1:18:01<41:21:53, 32.74s/it]
  3%|▎         | 143/4690 [1:18:34<41:22:58, 32.76s/it]
  3%|▎         | 144/4690 [1:19:06<41:18:15, 32.71s/it]
  3%|▎         | 145/4690 [1:19:39<41:20:42, 32.75s/it]
  3%|▎         | 146/4690 [1:20:12<41:16:42, 32.70s/it]
  3%|▎         | 147/4690 [1:20:45<41:19:07, 32.74s/it]
  3%|▎         | 148/4690 [1:21:18<41:32:05, 32.92s/it]
  3%|▎         | 149/4690 [1:21:51<41:34:36, 32.96s/it]
  3%|▎         | 150/4690 [1:22:23<41:17:17, 32.74s/it]
                                                       
{'loss': '0.493', 'grad_norm': '3.18', 'learning_rate': '1e-05', 'epoch': '0.16'}

  3%|▎         | 150/4690 [1:22:23<41:17:17, 32.74s/it]
  3%|▎         | 151/4690 [1:22:56<41:18:50, 32.77s/it]
  3%|▎         | 152/4690 [1:23:29<41:15:50, 32.73s/it]
  3%|▎         | 153/4690 [1:24:02<41:15:24, 32.74s/it]
  3%|▎         | 154/4690 [1:24:34<41:14:08, 32.73s/it]
  3%|▎         | 155/4690 [1:25:07<41:13:55, 32.73s/it]
  3%|▎         | 156/4690 [1:25:40<41:16:20, 32.77s/it]
  3%|▎         | 157/4690 [1:26:12<41:10:34, 32.70s/it]
  3%|▎         | 158/4690 [1:26:45<41:06:14, 32.65s/it]
  3%|▎         | 159/4690 [1:27:18<41:09:16, 32.70s/it]
  3%|▎         | 160/4690 [1:27:50<41:07:24, 32.68s/it]
                                                       
{'loss': '0.4908', 'grad_norm': '2.59', 'learning_rate': '1e-05', 'epoch': '0.1707'}

  3%|▎         | 160/4690 [1:27:50<41:07:24, 32.68s/it]
  3%|▎         | 161/4690 [1:28:23<41:11:09, 32.74s/it]
  3%|▎         | 162/4690 [1:28:56<41:17:48, 32.83s/it]
  3%|▎         | 163/4690 [1:29:29<41:13:26, 32.78s/it]
  3%|▎         | 164/4690 [1:30:02<41:18:04, 32.85s/it]
  4%|▎         | 165/4690 [1:30:35<41:17:50, 32.86s/it]
  4%|▎         | 166/4690 [1:31:08<41:22:40, 32.93s/it]
  4%|▎         | 167/4690 [1:31:41<41:19:49, 32.90s/it]
  4%|▎         | 168/4690 [1:32:13<41:12:05, 32.80s/it]
  4%|▎         | 169/4690 [1:32:46<41:16:10, 32.86s/it]
  4%|▎         | 170/4690 [1:33:19<41:13:08, 32.83s/it]
                                                       
{'loss': '0.5124', 'grad_norm': '2.583', 'learning_rate': '9.999e-06', 'epoch': '0.1813'}

  4%|▎         | 170/4690 [1:33:19<41:13:08, 32.83s/it]
  4%|▎         | 171/4690 [1:33:52<41:17:43, 32.90s/it]
  4%|▎         | 172/4690 [1:34:25<41:18:41, 32.92s/it]
  4%|▎         | 173/4690 [1:34:58<41:14:27, 32.87s/it]
  4%|▎         | 174/4690 [1:35:31<41:09:59, 32.82s/it]
  4%|▎         | 175/4690 [1:36:03<41:09:14, 32.81s/it]
  4%|▍         | 176/4690 [1:36:36<41:01:11, 32.71s/it]
  4%|▍         | 177/4690 [1:37:09<41:04:35, 32.77s/it]
  4%|▍         | 178/4690 [1:37:42<41:08:09, 32.82s/it]
  4%|▍         | 179/4690 [1:38:14<41:03:45, 32.77s/it]
  4%|▍         | 180/4690 [1:38:47<41:07:14, 32.82s/it]
                                                       
{'loss': '0.509', 'grad_norm': '2.948', 'learning_rate': '9.998e-06', 'epoch': '0.192'}

  4%|▍         | 180/4690 [1:38:47<41:07:14, 32.82s/it]
  4%|▍         | 181/4690 [1:39:20<41:10:32, 32.87s/it]
  4%|▍         | 182/4690 [1:39:53<41:06:32, 32.83s/it]
  4%|▍         | 183/4690 [1:40:26<41:06:14, 32.83s/it]
  4%|▍         | 184/4690 [1:40:59<41:04:10, 32.81s/it]
  4%|▍         | 185/4690 [1:41:31<41:01:58, 32.79s/it]
  4%|▍         | 186/4690 [1:42:04<41:01:58, 32.80s/it]
  4%|▍         | 187/4690 [1:42:37<41:02:02, 32.81s/it]
  4%|▍         | 188/4690 [1:43:09<40:50:56, 32.66s/it]
  4%|▍         | 189/4690 [1:43:42<40:54:38, 32.72s/it]
  4%|▍         | 190/4690 [1:44:15<40:54:06, 32.72s/it]
                                                       
{'loss': '0.4933', 'grad_norm': '2.743', 'learning_rate': '9.997e-06', 'epoch': '0.2027'}

  4%|▍         | 190/4690 [1:44:15<40:54:06, 32.72s/it]
  4%|▍         | 191/4690 [1:44:48<40:55:13, 32.74s/it]
  4%|▍         | 192/4690 [1:45:20<40:49:40, 32.68s/it]
  4%|▍         | 193/4690 [1:45:53<40:56:56, 32.78s/it]
  4%|▍         | 194/4690 [1:46:26<41:04:15, 32.89s/it]
  4%|▍         | 195/4690 [1:46:59<41:03:45, 32.89s/it]
  4%|▍         | 196/4690 [1:47:32<41:06:35, 32.93s/it]
  4%|▍         | 197/4690 [1:48:05<41:00:17, 32.85s/it]
  4%|▍         | 198/4690 [1:48:38<40:59:35, 32.85s/it]
  4%|▍         | 199/4690 [1:49:11<40:56:26, 32.82s/it]
  4%|▍         | 200/4690 [1:49:44<41:01:59, 32.90s/it]
                                                       
{'loss': '0.4973', 'grad_norm': '2.766', 'learning_rate': '9.996e-06', 'epoch': '0.2133'}

  4%|▍         | 200/4690 [1:49:44<41:01:59, 32.90s/it]
  4%|▍         | 201/4690 [1:50:17<40:59:18, 32.87s/it]
  4%|▍         | 202/4690 [1:50:49<40:55:39, 32.83s/it]
  4%|▍         | 203/4690 [1:51:22<40:53:22, 32.81s/it]
  4%|▍         | 204/4690 [1:51:54<40:38:27, 32.61s/it]
  4%|▍         | 205/4690 [1:52:27<40:37:36, 32.61s/it]
  4%|▍         | 206/4690 [1:53:00<40:50:29, 32.79s/it]
  4%|▍         | 207/4690 [1:53:33<40:50:08, 32.79s/it]
  4%|▍         | 208/4690 [1:54:06<40:59:15, 32.92s/it]
  4%|▍         | 209/4690 [1:54:39<41:00:08, 32.94s/it]
  4%|▍         | 210/4690 [1:55:12<40:58:13, 32.92s/it]
                                                       
{'loss': '0.4917', 'grad_norm': '2.77', 'learning_rate': '9.994e-06', 'epoch': '0.224'}

  4%|▍         | 210/4690 [1:55:12<40:58:13, 32.92s/it]
  4%|▍         | 211/4690 [1:55:44<40:48:15, 32.80s/it]
  5%|▍         | 212/4690 [1:56:17<40:44:47, 32.76s/it]
  5%|▍         | 213/4690 [1:56:50<40:49:06, 32.82s/it]
  5%|▍         | 214/4690 [1:57:23<40:54:30, 32.90s/it]
  5%|▍         | 215/4690 [1:57:56<40:48:04, 32.82s/it]
  5%|▍         | 216/4690 [1:58:29<40:52:36, 32.89s/it]
  5%|▍         | 217/4690 [1:59:01<40:46:10, 32.81s/it]
  5%|▍         | 218/4690 [1:59:34<40:48:09, 32.85s/it]
  5%|▍         | 219/4690 [2:00:07<40:41:41, 32.77s/it]
  5%|▍         | 220/4690 [2:00:40<40:44:33, 32.81s/it]
                                                       
{'loss': '0.5013', 'grad_norm': '2.674', 'learning_rate': '9.993e-06', 'epoch': '0.2347'}

  5%|▍         | 220/4690 [2:00:40<40:44:33, 32.81s/it]
  5%|▍         | 221/4690 [2:01:13<40:45:46, 32.84s/it]
  5%|▍         | 222/4690 [2:01:46<40:49:16, 32.89s/it]
  5%|▍         | 223/4690 [2:02:18<40:44:15, 32.83s/it]
  5%|▍         | 224/4690 [2:02:51<40:42:54, 32.82s/it]
  5%|▍         | 225/4690 [2:03:24<40:45:03, 32.86s/it]
  5%|▍         | 226/4690 [2:03:57<40:36:30, 32.75s/it]
  5%|▍         | 227/4690 [2:04:29<40:32:06, 32.70s/it]
  5%|▍         | 228/4690 [2:05:02<40:34:57, 32.74s/it]
  5%|▍         | 229/4690 [2:05:35<40:39:17, 32.81s/it]
  5%|▍         | 230/4690 [2:06:08<40:37:39, 32.79s/it]
                                                       
{'loss': '0.498', 'grad_norm': '2.564', 'learning_rate': '9.991e-06', 'epoch': '0.2453'}

  5%|▍         | 230/4690 [2:06:08<40:37:39, 32.79s/it]
  5%|▍         | 231/4690 [2:06:41<40:41:45, 32.86s/it]
  5%|▍         | 232/4690 [2:07:14<40:48:50, 32.96s/it]
  5%|▍         | 233/4690 [2:07:47<40:46:01, 32.93s/it]
  5%|▍         | 234/4690 [2:08:20<40:42:55, 32.89s/it]
  5%|▌         | 235/4690 [2:08:53<40:50:54, 33.01s/it]
  5%|▌         | 236/4690 [2:09:26<40:43:05, 32.91s/it]
  5%|▌         | 237/4690 [2:09:58<40:40:13, 32.88s/it]
  5%|▌         | 238/4690 [2:10:31<40:35:57, 32.83s/it]
  5%|▌         | 239/4690 [2:11:04<40:40:42, 32.90s/it]
  5%|▌         | 240/4690 [2:11:37<40:38:03, 32.87s/it]
                                                       
{'loss': '0.5079', 'grad_norm': '2.503', 'learning_rate': '9.989e-06', 'epoch': '0.256'}

  5%|▌         | 240/4690 [2:11:37<40:38:03, 32.87s/it]
  5%|▌         | 241/4690 [2:12:10<40:38:14, 32.88s/it]
  5%|▌         | 242/4690 [2:12:43<40:43:32, 32.96s/it]
  5%|▌         | 243/4690 [2:13:16<40:47:54, 33.03s/it]
  5%|▌         | 244/4690 [2:13:49<40:47:31, 33.03s/it]
  5%|▌         | 245/4690 [2:14:22<40:40:46, 32.95s/it]
  5%|▌         | 246/4690 [2:14:55<40:36:21, 32.89s/it]
  5%|▌         | 247/4690 [2:15:28<40:34:50, 32.88s/it]
  5%|▌         | 248/4690 [2:16:00<40:27:54, 32.79s/it]
  5%|▌         | 249/4690 [2:16:33<40:19:37, 32.69s/it]
  5%|▌         | 250/4690 [2:17:06<40:22:24, 32.74s/it]
                                                       
{'loss': '0.5045', 'grad_norm': '3.017', 'learning_rate': '9.986e-06', 'epoch': '0.2667'}

  5%|▌         | 250/4690 [2:17:06<40:22:24, 32.74s/it]
  5%|▌         | 251/4690 [2:17:38<40:20:40, 32.72s/it]
  5%|▌         | 252/4690 [2:18:11<40:25:49, 32.80s/it]
  5%|▌         | 253/4690 [2:18:44<40:18:54, 32.71s/it]
  5%|▌         | 254/4690 [2:19:17<40:20:18, 32.74s/it]
  5%|▌         | 255/4690 [2:19:49<40:19:28, 32.73s/it]
  5%|▌         | 256/4690 [2:20:22<40:20:10, 32.75s/it]
  5%|▌         | 257/4690 [2:20:55<40:14:37, 32.68s/it]
  6%|▌         | 258/4690 [2:21:27<40:15:59, 32.71s/it]
  6%|▌         | 259/4690 [2:22:00<40:21:21, 32.79s/it]
  6%|▌         | 260/4690 [2:22:33<40:13:08, 32.68s/it]
                                                       
{'loss': '0.5154', 'grad_norm': '2.679', 'learning_rate': '9.983e-06', 'epoch': '0.2773'}

  6%|▌         | 260/4690 [2:22:33<40:13:08, 32.68s/it]
  6%|▌         | 261/4690 [2:23:06<40:22:26, 32.82s/it]
  6%|▌         | 262/4690 [2:23:39<40:23:02, 32.83s/it]
  6%|▌         | 263/4690 [2:24:12<40:26:04, 32.88s/it]
  6%|▌         | 264/4690 [2:24:44<40:12:18, 32.70s/it]
  6%|▌         | 265/4690 [2:25:17<40:17:16, 32.78s/it]
  6%|▌         | 266/4690 [2:25:50<40:16:41, 32.78s/it]
  6%|▌         | 267/4690 [2:26:22<40:06:24, 32.64s/it]
  6%|▌         | 268/4690 [2:26:55<40:04:27, 32.63s/it]
  6%|▌         | 269/4690 [2:27:27<40:01:22, 32.59s/it]
  6%|▌         | 270/4690 [2:28:00<40:08:54, 32.70s/it]
                                                       
{'loss': '0.4907', 'grad_norm': '2.447', 'learning_rate': '9.98e-06', 'epoch': '0.288'}

  6%|▌         | 270/4690 [2:28:00<40:08:54, 32.70s/it]
  6%|▌         | 271/4690 [2:28:33<40:12:53, 32.76s/it]
  6%|▌         | 272/4690 [2:29:06<40:18:52, 32.85s/it]
  6%|▌         | 273/4690 [2:29:39<40:24:41, 32.94s/it]
  6%|▌         | 274/4690 [2:30:12<40:22:08, 32.91s/it]
  6%|▌         | 275/4690 [2:30:45<40:17:05, 32.85s/it]
  6%|▌         | 276/4690 [2:31:18<40:14:27, 32.82s/it]
  6%|▌         | 277/4690 [2:31:51<40:20:42, 32.91s/it]
  6%|▌         | 278/4690 [2:32:23<40:15:31, 32.85s/it]
  6%|▌         | 279/4690 [2:32:56<40:12:26, 32.81s/it]
  6%|▌         | 280/4690 [2:33:29<40:19:35, 32.92s/it]
                                                       
{'loss': '0.4914', 'grad_norm': '2.46', 'learning_rate': '9.977e-06', 'epoch': '0.2987'}

  6%|▌         | 280/4690 [2:33:29<40:19:35, 32.92s/it]
  6%|▌         | 281/4690 [2:34:02<40:19:35, 32.93s/it]
  6%|▌         | 282/4690 [2:34:35<40:23:28, 32.99s/it]
  6%|▌         | 283/4690 [2:35:08<40:25:06, 33.02s/it]
  6%|▌         | 284/4690 [2:35:41<40:19:48, 32.95s/it]
  6%|▌         | 285/4690 [2:36:14<40:16:26, 32.91s/it]
  6%|▌         | 286/4690 [2:36:47<40:14:37, 32.90s/it]
  6%|▌         | 287/4690 [2:37:20<40:14:41, 32.91s/it]
  6%|▌         | 288/4690 [2:37:53<40:10:44, 32.86s/it]
  6%|▌         | 289/4690 [2:38:25<40:05:55, 32.80s/it]
  6%|▌         | 290/4690 [2:38:58<40:02:14, 32.76s/it]
                                                       
{'loss': '0.4986', 'grad_norm': '2.658', 'learning_rate': '9.974e-06', 'epoch': '0.3093'}

  6%|▌         | 290/4690 [2:38:58<40:02:14, 32.76s/it]
  6%|▌         | 291/4690 [2:39:30<39:57:24, 32.70s/it]
  6%|▌         | 292/4690 [2:40:03<39:56:14, 32.69s/it]
  6%|▌         | 293/4690 [2:40:36<39:59:35, 32.74s/it]
  6%|▋         | 294/4690 [2:41:09<39:58:20, 32.73s/it]
  6%|▋         | 295/4690 [2:41:42<40:03:30, 32.81s/it]
  6%|▋         | 296/4690 [2:42:15<40:04:52, 32.84s/it]
  6%|▋         | 297/4690 [2:42:47<40:04:57, 32.85s/it]
  6%|▋         | 298/4690 [2:43:21<40:09:43, 32.92s/it]
  6%|▋         | 299/4690 [2:43:54<40:09:49, 32.93s/it]
  6%|▋         | 300/4690 [2:44:27<40:11:22, 32.96s/it]
                                                       
{'loss': '0.4931', 'grad_norm': '2.465', 'learning_rate': '9.97e-06', 'epoch': '0.32'}

  6%|▋         | 300/4690 [2:44:27<40:11:22, 32.96s/it]
  6%|▋         | 301/4690 [2:45:00<40:19:33, 33.08s/it]
  6%|▋         | 302/4690 [2:45:33<40:09:50, 32.95s/it]
  6%|▋         | 303/4690 [2:46:06<40:10:31, 32.97s/it]
  6%|▋         | 304/4690 [2:46:38<40:02:49, 32.87s/it]
  7%|▋         | 305/4690 [2:47:11<40:06:10, 32.92s/it]
  7%|▋         | 306/4690 [2:47:44<40:07:27, 32.95s/it]
  7%|▋         | 307/4690 [2:48:17<40:00:13, 32.86s/it]
  7%|▋         | 308/4690 [2:48:50<40:05:42, 32.94s/it]
  7%|▋         | 309/4690 [2:49:23<40:07:10, 32.97s/it]
  7%|▋         | 310/4690 [2:49:56<40:02:14, 32.91s/it]
                                                       
{'loss': '0.5048', 'grad_norm': '3.149', 'learning_rate': '9.966e-06', 'epoch': '0.3307'}

  7%|▋         | 310/4690 [2:49:56<40:02:14, 32.91s/it]
  7%|▋         | 311/4690 [2:50:28<39:52:42, 32.78s/it]
  7%|▋         | 312/4690 [2:51:01<39:56:14, 32.84s/it]
  7%|▋         | 313/4690 [2:51:34<39:56:30, 32.85s/it]
  7%|▋         | 314/4690 [2:52:08<40:07:11, 33.01s/it]
  7%|▋         | 315/4690 [2:52:40<39:58:21, 32.89s/it]
  7%|▋         | 316/4690 [2:53:13<39:56:46, 32.88s/it]
  7%|▋         | 317/4690 [2:53:45<39:46:30, 32.74s/it]
  7%|▋         | 318/4690 [2:54:18<39:46:29, 32.75s/it]
  7%|▋         | 319/4690 [2:54:51<39:50:25, 32.81s/it]
  7%|▋         | 320/4690 [2:55:24<39:47:22, 32.78s/it]
                                                       
{'loss': '0.4958', 'grad_norm': '2.618', 'learning_rate': '9.962e-06', 'epoch': '0.3413'}

  7%|▋         | 320/4690 [2:55:24<39:47:22, 32.78s/it]
  7%|▋         | 321/4690 [2:55:56<39:38:24, 32.66s/it]
  7%|▋         | 322/4690 [2:56:29<39:37:11, 32.65s/it]
  7%|▋         | 323/4690 [2:57:02<39:44:04, 32.76s/it]
  7%|▋         | 324/4690 [2:57:35<39:46:02, 32.79s/it]
  7%|▋         | 325/4690 [2:58:08<39:53:03, 32.89s/it]
  7%|▋         | 326/4690 [2:58:41<39:49:53, 32.86s/it]
  7%|▋         | 327/4690 [2:59:14<39:50:46, 32.88s/it]
  7%|▋         | 328/4690 [2:59:46<39:49:29, 32.87s/it]
  7%|▋         | 329/4690 [3:00:20<39:56:11, 32.97s/it]
  7%|▋         | 330/4690 [3:00:52<39:48:33, 32.87s/it]
                                                       
{'loss': '0.4949', 'grad_norm': '2.616', 'learning_rate': '9.958e-06', 'epoch': '0.352'}

  7%|▋         | 330/4690 [3:00:52<39:48:33, 32.87s/it]
  7%|▋         | 331/4690 [3:01:25<39:49:07, 32.89s/it]
  7%|▋         | 332/4690 [3:01:58<39:41:49, 32.79s/it]
  7%|▋         | 333/4690 [3:02:31<39:46:05, 32.86s/it]
  7%|▋         | 334/4690 [3:03:04<39:47:31, 32.89s/it]
  7%|▋         | 335/4690 [3:03:37<39:48:10, 32.90s/it]
  7%|▋         | 336/4690 [3:04:10<39:46:15, 32.88s/it]
  7%|▋         | 337/4690 [3:04:42<39:42:13, 32.84s/it]
  7%|▋         | 338/4690 [3:05:16<39:51:08, 32.97s/it]
  7%|▋         | 339/4690 [3:05:48<39:49:32, 32.95s/it]
  7%|▋         | 340/4690 [3:06:21<39:39:39, 32.82s/it]
                                                       
{'loss': '0.4907', 'grad_norm': '2.633', 'learning_rate': '9.953e-06', 'epoch': '0.3627'}

  7%|▋         | 340/4690 [3:06:21<39:39:39, 32.82s/it]
  7%|▋         | 341/4690 [3:06:54<39:39:53, 32.83s/it]
  7%|▋         | 342/4690 [3:07:27<39:38:27, 32.82s/it]
  7%|▋         | 343/4690 [3:08:00<39:40:45, 32.86s/it]
  7%|▋         | 344/4690 [3:08:32<39:39:48, 32.86s/it]
  7%|▋         | 345/4690 [3:09:05<39:42:57, 32.91s/it]
  7%|▋         | 346/4690 [3:09:38<39:43:27, 32.92s/it]
  7%|▋         | 347/4690 [3:10:11<39:45:12, 32.95s/it]
  7%|▋         | 348/4690 [3:10:45<39:50:11, 33.03s/it]
  7%|▋         | 349/4690 [3:11:17<39:36:10, 32.84s/it]
  7%|▋         | 350/4690 [3:11:50<39:35:58, 32.85s/it]
                                                       
{'loss': '0.5094', 'grad_norm': '2.883', 'learning_rate': '9.949e-06', 'epoch': '0.3733'}

  7%|▋         | 350/4690 [3:11:50<39:35:58, 32.85s/it]
  7%|▋         | 351/4690 [3:12:23<39:31:08, 32.79s/it]
  8%|▊         | 352/4690 [3:12:55<39:32:10, 32.81s/it]
  8%|▊         | 353/4690 [3:13:28<39:28:40, 32.77s/it]
  8%|▊         | 354/4690 [3:14:00<39:16:46, 32.61s/it]
  8%|▊         | 355/4690 [3:14:33<39:18:33, 32.64s/it]
  8%|▊         | 356/4690 [3:15:06<39:20:47, 32.68s/it]
  8%|▊         | 357/4690 [3:15:39<39:26:03, 32.76s/it]
  8%|▊         | 358/4690 [3:16:11<39:23:33, 32.74s/it]
  8%|▊         | 359/4690 [3:16:44<39:17:47, 32.66s/it]
  8%|▊         | 360/4690 [3:17:17<39:17:17, 32.66s/it]
                                                       
{'loss': '0.482', 'grad_norm': '2.498', 'learning_rate': '9.943e-06', 'epoch': '0.384'}

  8%|▊         | 360/4690 [3:17:17<39:17:17, 32.66s/it]
  8%|▊         | 361/4690 [3:17:49<39:15:26, 32.65s/it]
  8%|▊         | 362/4690 [3:18:22<39:19:00, 32.70s/it]
  8%|▊         | 363/4690 [3:18:55<39:25:19, 32.80s/it]
  8%|▊         | 364/4690 [3:19:28<39:24:57, 32.80s/it]
  8%|▊         | 365/4690 [3:20:01<39:31:08, 32.89s/it]
  8%|▊         | 366/4690 [3:20:33<39:21:12, 32.76s/it]
  8%|▊         | 367/4690 [3:21:06<39:20:53, 32.77s/it]
  8%|▊         | 368/4690 [3:21:39<39:23:12, 32.81s/it]
  8%|▊         | 369/4690 [3:22:12<39:25:06, 32.84s/it]
  8%|▊         | 370/4690 [3:22:45<39:23:54, 32.83s/it]
                                                       
{'loss': '0.507', 'grad_norm': '2.346', 'learning_rate': '9.938e-06', 'epoch': '0.3947'}

  8%|▊         | 370/4690 [3:22:45<39:23:54, 32.83s/it]
  8%|▊         | 371/4690 [3:23:18<39:21:24, 32.81s/it]
  8%|▊         | 372/4690 [3:23:50<39:09:59, 32.65s/it]
  8%|▊         | 373/4690 [3:24:23<39:09:04, 32.65s/it]
  8%|▊         | 374/4690 [3:24:55<39:05:32, 32.61s/it]
  8%|▊         | 375/4690 [3:25:28<39:04:14, 32.60s/it]
  8%|▊         | 376/4690 [3:26:01<39:11:02, 32.70s/it]
  8%|▊         | 377/4690 [3:26:33<39:10:24, 32.70s/it]
  8%|▊         | 378/4690 [3:27:06<39:06:08, 32.65s/it]
  8%|▊         | 379/4690 [3:27:39<39:09:48, 32.70s/it]
  8%|▊         | 380/4690 [3:28:12<39:16:47, 32.81s/it]
                                                       
{'loss': '0.4871', 'grad_norm': '2.485', 'learning_rate': '9.933e-06', 'epoch': '0.4053'}

  8%|▊         | 380/4690 [3:28:12<39:16:47, 32.81s/it]
  8%|▊         | 381/4690 [3:28:44<39:09:12, 32.71s/it]
  8%|▊         | 382/4690 [3:29:17<39:13:45, 32.78s/it]
  8%|▊         | 383/4690 [3:29:50<39:10:38, 32.75s/it]
  8%|▊         | 384/4690 [3:30:23<39:12:12, 32.78s/it]
  8%|▊         | 385/4690 [3:30:55<39:08:12, 32.73s/it]
  8%|▊         | 386/4690 [3:31:28<39:12:55, 32.80s/it]
  8%|▊         | 387/4690 [3:32:01<39:03:18, 32.67s/it]
  8%|▊         | 388/4690 [3:32:33<39:02:24, 32.67s/it]
  8%|▊         | 389/4690 [3:33:06<39:01:05, 32.66s/it]
  8%|▊         | 390/4690 [3:33:39<39:11:46, 32.82s/it]
                                                       
{'loss': '0.4863', 'grad_norm': '2.391', 'learning_rate': '9.927e-06', 'epoch': '0.416'}

  8%|▊         | 390/4690 [3:33:39<39:11:46, 32.82s/it]
  8%|▊         | 391/4690 [3:34:12<39:12:50, 32.84s/it]
  8%|▊         | 392/4690 [3:34:45<39:15:55, 32.89s/it]
  8%|▊         | 393/4690 [3:35:18<39:11:55, 32.84s/it]
  8%|▊         | 394/4690 [3:35:50<39:10:49, 32.83s/it]
  8%|▊         | 395/4690 [3:36:23<39:09:39, 32.82s/it]
  8%|▊         | 396/4690 [3:36:56<39:10:00, 32.84s/it]
  8%|▊         | 397/4690 [3:37:29<39:11:48, 32.87s/it]
  8%|▊         | 398/4690 [3:38:02<39:07:01, 32.81s/it]
  9%|▊         | 399/4690 [3:38:35<39:15:14, 32.93s/it]
  9%|▊         | 400/4690 [3:39:08<39:08:32, 32.85s/it]
                                                       
{'loss': '0.4907', 'grad_norm': '2.653', 'learning_rate': '9.921e-06', 'epoch': '0.4267'}

  9%|▊         | 400/4690 [3:39:08<39:08:32, 32.85s/it]
  9%|▊         | 401/4690 [3:39:41<39:16:12, 32.96s/it]
  9%|▊         | 402/4690 [3:40:14<39:12:36, 32.92s/it]
  9%|▊         | 403/4690 [3:40:46<39:08:21, 32.87s/it]
  9%|▊         | 404/4690 [3:41:19<39:05:46, 32.84s/it]
  9%|▊         | 405/4690 [3:41:52<38:55:50, 32.71s/it]
  9%|▊         | 406/4690 [3:42:24<38:51:54, 32.66s/it]
  9%|▊         | 407/4690 [3:42:58<39:06:48, 32.88s/it]
  9%|▊         | 408/4690 [3:43:30<39:00:27, 32.79s/it]
  9%|▊         | 409/4690 [3:44:03<38:59:34, 32.79s/it]
  9%|▊         | 410/4690 [3:44:36<38:55:42, 32.74s/it]
                                                       
{'loss': '0.4913', 'grad_norm': '2.563', 'learning_rate': '9.915e-06', 'epoch': '0.4373'}

  9%|▊         | 410/4690 [3:44:36<38:55:42, 32.74s/it]
  9%|▉         | 411/4690 [3:45:08<38:44:51, 32.60s/it]
  9%|▉         | 412/4690 [3:45:41<38:51:15, 32.70s/it]
  9%|▉         | 413/4690 [3:46:14<38:55:40, 32.77s/it]
  9%|▉         | 414/4690 [3:46:47<39:05:15, 32.91s/it]
  9%|▉         | 415/4690 [3:47:20<38:59:54, 32.84s/it]
  9%|▉         | 416/4690 [3:47:52<38:49:50, 32.71s/it]
  9%|▉         | 417/4690 [3:48:24<38:42:52, 32.62s/it]
  9%|▉         | 418/4690 [3:48:57<38:39:58, 32.58s/it]
  9%|▉         | 419/4690 [3:49:30<38:40:17, 32.60s/it]
  9%|▉         | 420/4690 [3:50:02<38:46:29, 32.69s/it]
                                                       
{'loss': '0.4801', 'grad_norm': '2.468', 'learning_rate': '9.908e-06', 'epoch': '0.448'}

  9%|▉         | 420/4690 [3:50:02<38:46:29, 32.69s/it]
  9%|▉         | 421/4690 [3:50:35<38:51:57, 32.78s/it]
  9%|▉         | 422/4690 [3:51:08<38:55:38, 32.83s/it]
  9%|▉         | 423/4690 [3:51:41<38:55:31, 32.84s/it]
  9%|▉         | 424/4690 [3:52:14<38:50:11, 32.77s/it]
  9%|▉         | 425/4690 [3:52:47<38:52:45, 32.82s/it]
  9%|▉         | 426/4690 [3:53:20<38:54:31, 32.85s/it]
  9%|▉         | 427/4690 [3:53:52<38:50:05, 32.79s/it]
  9%|▉         | 428/4690 [3:54:25<38:52:03, 32.83s/it]
  9%|▉         | 429/4690 [3:54:58<38:54:08, 32.87s/it]
  9%|▉         | 430/4690 [3:55:31<38:59:32, 32.95s/it]
                                                       
{'loss': '0.4843', 'grad_norm': '2.447', 'learning_rate': '9.901e-06', 'epoch': '0.4587'}

  9%|▉         | 430/4690 [3:55:31<38:59:32, 32.95s/it]
  9%|▉         | 431/4690 [3:56:04<38:50:35, 32.83s/it]
  9%|▉         | 432/4690 [3:56:37<38:48:40, 32.81s/it]
  9%|▉         | 433/4690 [3:57:10<38:49:19, 32.83s/it]
  9%|▉         | 434/4690 [3:57:42<38:42:21, 32.74s/it]
  9%|▉         | 435/4690 [3:58:15<38:47:48, 32.82s/it]
  9%|▉         | 436/4690 [3:58:48<38:42:21, 32.76s/it]
  9%|▉         | 437/4690 [3:59:21<38:48:14, 32.85s/it]
  9%|▉         | 438/4690 [3:59:53<38:42:53, 32.78s/it]
  9%|▉         | 439/4690 [4:00:26<38:47:30, 32.85s/it]
  9%|▉         | 440/4690 [4:00:59<38:42:19, 32.79s/it]
                                                       
{'loss': '0.4898', 'grad_norm': '2.343', 'learning_rate': '9.894e-06', 'epoch': '0.4693'}

  9%|▉         | 440/4690 [4:00:59<38:42:19, 32.79s/it]
  9%|▉         | 441/4690 [4:01:32<38:44:14, 32.82s/it]
  9%|▉         | 442/4690 [4:02:05<38:45:22, 32.84s/it]
  9%|▉         | 443/4690 [4:02:37<38:39:22, 32.77s/it]
  9%|▉         | 444/4690 [4:03:10<38:37:11, 32.74s/it]
  9%|▉         | 445/4690 [4:03:43<38:42:34, 32.83s/it]
 10%|▉         | 446/4690 [4:04:16<38:43:41, 32.85s/it]
 10%|▉         | 447/4690 [4:04:49<38:38:09, 32.78s/it]
 10%|▉         | 448/4690 [4:05:21<38:29:43, 32.67s/it]
 10%|▉         | 449/4690 [4:05:54<38:25:55, 32.62s/it]
 10%|▉         | 450/4690 [4:06:27<38:31:18, 32.71s/it]
                                                       
{'loss': '0.4931', 'grad_norm': '2.524', 'learning_rate': '9.887e-06', 'epoch': '0.48'}

 10%|▉         | 450/4690 [4:06:27<38:31:18, 32.71s/it]
 10%|▉         | 451/4690 [4:06:59<38:20:37, 32.56s/it]
 10%|▉         | 452/4690 [4:07:32<38:40:46, 32.86s/it]
 10%|▉         | 453/4690 [4:08:05<38:46:30, 32.95s/it]
 10%|▉         | 454/4690 [4:08:38<38:45:04, 32.93s/it]
 10%|▉         | 455/4690 [4:09:11<38:39:40, 32.86s/it]
 10%|▉         | 456/4690 [4:09:44<38:31:22, 32.75s/it]
 10%|▉         | 457/4690 [4:10:16<38:29:10, 32.73s/it]
 10%|▉         | 458/4690 [4:10:49<38:27:04, 32.71s/it]
 10%|▉         | 459/4690 [4:11:22<38:30:56, 32.77s/it]
 10%|▉         | 460/4690 [4:11:54<38:26:01, 32.71s/it]
                                                       
{'loss': '0.4853', 'grad_norm': '2.591', 'learning_rate': '9.88e-06', 'epoch': '0.4907'}

 10%|▉         | 460/4690 [4:11:54<38:26:01, 32.71s/it]
 10%|▉         | 461/4690 [4:12:27<38:30:17, 32.78s/it]
 10%|▉         | 462/4690 [4:13:00<38:30:04, 32.78s/it]
 10%|▉         | 463/4690 [4:13:33<38:25:57, 32.73s/it]
 10%|▉         | 464/4690 [4:14:05<38:18:43, 32.64s/it]
 10%|▉         | 465/4690 [4:14:38<38:13:11, 32.57s/it]
 10%|▉         | 466/4690 [4:15:10<38:18:18, 32.65s/it]
 10%|▉         | 467/4690 [4:15:43<38:26:38, 32.77s/it]
 10%|▉         | 468/4690 [4:16:16<38:21:08, 32.70s/it]
 10%|█         | 469/4690 [4:16:49<38:20:00, 32.69s/it]
 10%|█         | 470/4690 [4:17:21<38:21:50, 32.73s/it]
                                                       
{'loss': '0.4795', 'grad_norm': '2.318', 'learning_rate': '9.872e-06', 'epoch': '0.5013'}

 10%|█         | 470/4690 [4:17:21<38:21:50, 32.73s/it]
 10%|█         | 471/4690 [4:17:54<38:15:59, 32.65s/it]
 10%|█         | 472/4690 [4:18:27<38:15:15, 32.65s/it]
 10%|█         | 473/4690 [4:19:00<38:21:08, 32.74s/it]
 10%|█         | 474/4690 [4:19:32<38:18:44, 32.71s/it]
 10%|█         | 475/4690 [4:20:05<38:25:11, 32.81s/it]
 10%|█         | 476/4690 [4:20:38<38:34:08, 32.95s/it]
 10%|█         | 477/4690 [4:21:11<38:27:04, 32.86s/it]
 10%|█         | 478/4690 [4:21:44<38:21:40, 32.79s/it]
 10%|█         | 479/4690 [4:22:16<38:12:07, 32.66s/it]
 10%|█         | 480/4690 [4:22:49<38:17:19, 32.74s/it]
                                                       
{'loss': '0.4953', 'grad_norm': '2.317', 'learning_rate': '9.864e-06', 'epoch': '0.512'}

 10%|█         | 480/4690 [4:22:49<38:17:19, 32.74s/it]
 10%|█         | 481/4690 [4:23:22<38:18:49, 32.77s/it]
 10%|█         | 482/4690 [4:23:55<38:17:57, 32.77s/it]
 10%|█         | 483/4690 [4:24:28<38:20:07, 32.80s/it]
 10%|█         | 484/4690 [4:25:00<38:22:07, 32.84s/it]
 10%|█         | 485/4690 [4:25:33<38:18:07, 32.79s/it]
 10%|█         | 486/4690 [4:26:06<38:13:34, 32.73s/it]
 10%|█         | 487/4690 [4:26:39<38:17:13, 32.79s/it]
 10%|█         | 488/4690 [4:27:11<38:15:37, 32.78s/it]
 10%|█         | 489/4690 [4:27:45<38:22:31, 32.89s/it]
 10%|█         | 490/4690 [4:28:17<38:20:22, 32.86s/it]
                                                       
{'loss': '0.4924', 'grad_norm': '2.508', 'learning_rate': '9.856e-06', 'epoch': '0.5227'}

 10%|█         | 490/4690 [4:28:17<38:20:22, 32.86s/it]
 10%|█         | 491/4690 [4:28:50<38:14:21, 32.78s/it]
 10%|█         | 492/4690 [4:29:23<38:22:33, 32.91s/it]
 11%|█         | 493/4690 [4:29:56<38:23:05, 32.92s/it]
 11%|█         | 494/4690 [4:30:29<38:14:53, 32.82s/it]
 11%|█         | 495/4690 [4:31:02<38:22:07, 32.93s/it]
 11%|█         | 496/4690 [4:31:34<38:11:41, 32.79s/it]
 11%|█         | 497/4690 [4:32:07<38:08:52, 32.75s/it]
 11%|█         | 498/4690 [4:32:40<38:13:55, 32.83s/it]
 11%|█         | 499/4690 [4:33:13<38:09:35, 32.78s/it]
 11%|█         | 500/4690 [4:33:45<38:04:33, 32.71s/it]
                                                       
{'loss': '0.4844', 'grad_norm': '2.415', 'learning_rate': '9.848e-06', 'epoch': '0.5333'}

 11%|█         | 500/4690 [4:33:45<38:04:33, 32.71s/it]
 11%|█         | 501/4690 [4:34:18<38:13:35, 32.85s/it]
 11%|█         | 502/4690 [4:34:51<38:07:00, 32.77s/it]
 11%|█         | 503/4690 [4:35:24<38:03:24, 32.72s/it]
 11%|█         | 504/4690 [4:35:57<38:11:59, 32.85s/it]
 11%|█         | 505/4690 [4:36:29<38:06:20, 32.78s/it]
 11%|█         | 506/4690 [4:37:02<38:05:40, 32.78s/it]
 11%|█         | 507/4690 [4:37:35<38:02:39, 32.74s/it]
 11%|█         | 508/4690 [4:38:07<37:57:25, 32.67s/it]
 11%|█         | 509/4690 [4:38:40<38:06:35, 32.81s/it]
 11%|█         | 510/4690 [4:39:13<38:10:24, 32.88s/it]
                                                       
{'loss': '0.4836', 'grad_norm': '2.417', 'learning_rate': '9.839e-06', 'epoch': '0.544'}

 11%|█         | 510/4690 [4:39:13<38:10:24, 32.88s/it]
 11%|█         | 511/4690 [4:39:46<38:05:28, 32.81s/it]
 11%|█         | 512/4690 [4:40:19<38:09:10, 32.87s/it]
 11%|█         | 513/4690 [4:40:52<38:07:29, 32.86s/it]
 11%|█         | 514/4690 [4:41:25<38:02:38, 32.80s/it]
 11%|█         | 515/4690 [4:41:57<37:58:49, 32.75s/it]
 11%|█         | 516/4690 [4:42:30<37:57:18, 32.74s/it]
 11%|█         | 517/4690 [4:43:03<37:55:52, 32.72s/it]
 11%|█         | 518/4690 [4:43:35<37:46:18, 32.59s/it]
 11%|█         | 519/4690 [4:44:08<37:46:28, 32.60s/it]
 11%|█         | 520/4690 [4:44:40<37:41:49, 32.54s/it]
                                                       
{'loss': '0.4881', 'grad_norm': '2.421', 'learning_rate': '9.831e-06', 'epoch': '0.5547'}

 11%|█         | 520/4690 [4:44:40<37:41:49, 32.54s/it]
 11%|█         | 521/4690 [4:45:13<37:43:19, 32.57s/it]
 11%|█         | 522/4690 [4:45:45<37:45:09, 32.61s/it]
 11%|█         | 523/4690 [4:46:18<37:38:22, 32.52s/it]
 11%|█         | 524/4690 [4:46:51<37:49:56, 32.69s/it]
 11%|█         | 525/4690 [4:47:23<37:40:39, 32.57s/it]
 11%|█         | 526/4690 [4:47:56<37:49:31, 32.70s/it]
 11%|█         | 527/4690 [4:48:29<37:52:14, 32.75s/it]
 11%|█▏        | 528/4690 [4:49:01<37:48:48, 32.71s/it]
 11%|█▏        | 529/4690 [4:49:34<37:50:58, 32.75s/it]
 11%|█▏        | 530/4690 [4:50:07<37:58:31, 32.86s/it]
                                                       
{'loss': '0.4937', 'grad_norm': '2.237', 'learning_rate': '9.822e-06', 'epoch': '0.5653'}

 11%|█▏        | 530/4690 [4:50:07<37:58:31, 32.86s/it]
 11%|█▏        | 531/4690 [4:50:41<38:09:27, 33.03s/it]
 11%|█▏        | 532/4690 [4:51:14<38:05:30, 32.98s/it]
 11%|█▏        | 533/4690 [4:51:47<38:07:50, 33.02s/it]
 11%|█▏        | 534/4690 [4:52:20<38:00:25, 32.92s/it]
 11%|█▏        | 535/4690 [4:52:52<37:58:29, 32.90s/it]
 11%|█▏        | 536/4690 [4:53:25<37:56:13, 32.88s/it]
 11%|█▏        | 537/4690 [4:53:58<37:54:28, 32.86s/it]
 11%|█▏        | 538/4690 [4:54:30<37:43:32, 32.71s/it]
 11%|█▏        | 539/4690 [4:55:04<37:56:46, 32.91s/it]
 12%|█▏        | 540/4690 [4:55:37<37:58:34, 32.94s/it]
                                                       
{'loss': '0.4851', 'grad_norm': '2.607', 'learning_rate': '9.812e-06', 'epoch': '0.576'}

 12%|█▏        | 540/4690 [4:55:37<37:58:34, 32.94s/it]
 12%|█▏        | 541/4690 [4:56:09<37:49:54, 32.83s/it]
 12%|█▏        | 542/4690 [4:56:42<37:46:07, 32.78s/it]
 12%|█▏        | 543/4690 [4:57:15<37:54:00, 32.90s/it]
 12%|█▏        | 544/4690 [4:57:48<37:44:38, 32.77s/it]
 12%|█▏        | 545/4690 [4:58:21<37:49:33, 32.85s/it]
 12%|█▏        | 546/4690 [4:58:53<37:45:59, 32.81s/it]
 12%|█▏        | 547/4690 [4:59:26<37:49:25, 32.87s/it]
 12%|█▏        | 548/4690 [4:59:59<37:42:12, 32.77s/it]
 12%|█▏        | 549/4690 [5:00:32<37:42:04, 32.78s/it]
 12%|█▏        | 550/4690 [5:01:04<37:40:24, 32.76s/it]
                                                       
{'loss': '0.4763', 'grad_norm': '2.506', 'learning_rate': '9.803e-06', 'epoch': '0.5867'}

 12%|█▏        | 550/4690 [5:01:04<37:40:24, 32.76s/it]
 12%|█▏        | 551/4690 [5:01:37<37:35:29, 32.70s/it]
 12%|█▏        | 552/4690 [5:02:09<37:24:11, 32.54s/it]
 12%|█▏        | 553/4690 [5:02:42<37:35:12, 32.71s/it]
 12%|█▏        | 554/4690 [5:03:15<37:44:02, 32.84s/it]
 12%|█▏        | 555/4690 [5:03:49<37:51:36, 32.96s/it]
 12%|█▏        | 556/4690 [5:04:21<37:44:24, 32.87s/it]
 12%|█▏        | 557/4690 [5:04:54<37:41:38, 32.83s/it]
 12%|█▏        | 558/4690 [5:05:27<37:38:48, 32.80s/it]
 12%|█▏        | 559/4690 [5:06:00<37:39:30, 32.82s/it]
 12%|█▏        | 560/4690 [5:06:32<37:33:34, 32.74s/it]
                                                       
{'loss': '0.4899', 'grad_norm': '2.454', 'learning_rate': '9.793e-06', 'epoch': '0.5973'}

 12%|█▏        | 560/4690 [5:06:32<37:33:34, 32.74s/it]
 12%|█▏        | 561/4690 [5:07:05<37:38:53, 32.82s/it]
 12%|█▏        | 562/4690 [5:07:38<37:37:15, 32.81s/it]
 12%|█▏        | 563/4690 [5:08:11<37:36:43, 32.81s/it]
 12%|█▏        | 564/4690 [5:08:44<37:38:02, 32.84s/it]
 12%|█▏        | 565/4690 [5:09:17<37:36:52, 32.83s/it]
 12%|█▏        | 566/4690 [5:09:50<37:48:59, 33.01s/it]
 12%|█▏        | 567/4690 [5:10:23<37:38:18, 32.86s/it]
 12%|█▏        | 568/4690 [5:10:55<37:37:00, 32.85s/it]
 12%|█▏        | 569/4690 [5:11:28<37:36:35, 32.86s/it]
 12%|█▏        | 570/4690 [5:12:01<37:33:07, 32.81s/it]
                                                       
{'loss': '0.4793', 'grad_norm': '2.336', 'learning_rate': '9.783e-06', 'epoch': '0.608'}

 12%|█▏        | 570/4690 [5:12:01<37:33:07, 32.81s/it]
 12%|█▏        | 571/4690 [5:12:34<37:34:01, 32.83s/it]
 12%|█▏        | 572/4690 [5:13:07<37:33:59, 32.84s/it]
 12%|█▏        | 573/4690 [5:13:40<37:34:47, 32.86s/it]
 12%|█▏        | 574/4690 [5:14:13<37:39:18, 32.93s/it]
 12%|█▏        | 575/4690 [5:14:46<37:37:53, 32.92s/it]
 12%|█▏        | 576/4690 [5:15:18<37:28:55, 32.80s/it]
 12%|█▏        | 577/4690 [5:15:51<37:30:06, 32.82s/it]
 12%|█▏        | 578/4690 [5:16:24<37:24:23, 32.75s/it]
 12%|█▏        | 579/4690 [5:16:56<37:19:33, 32.69s/it]
 12%|█▏        | 580/4690 [5:17:29<37:27:09, 32.81s/it]
                                                       
{'loss': '0.4934', 'grad_norm': '2.362', 'learning_rate': '9.773e-06', 'epoch': '0.6187'}

 12%|█▏        | 580/4690 [5:17:29<37:27:09, 32.81s/it]
 12%|█▏        | 581/4690 [5:18:02<37:26:04, 32.80s/it]
 12%|█▏        | 582/4690 [5:18:35<37:26:43, 32.81s/it]
 12%|█▏        | 583/4690 [5:19:07<37:18:29, 32.70s/it]
 12%|█▏        | 584/4690 [5:19:39<37:07:59, 32.56s/it]
 12%|█▏        | 585/4690 [5:20:12<37:13:36, 32.65s/it]
 12%|█▏        | 586/4690 [5:20:45<37:24:16, 32.81s/it]
 13%|█▎        | 587/4690 [5:21:18<37:25:43, 32.84s/it]
 13%|█▎        | 588/4690 [5:21:51<37:17:15, 32.72s/it]
 13%|█▎        | 589/4690 [5:22:23<37:11:06, 32.64s/it]
 13%|█▎        | 590/4690 [5:22:56<37:14:47, 32.70s/it]
                                                       
{'loss': '0.4741', 'grad_norm': '2.356', 'learning_rate': '9.763e-06', 'epoch': '0.6293'}

 13%|█▎        | 590/4690 [5:22:56<37:14:47, 32.70s/it]
 13%|█▎        | 591/4690 [5:23:29<37:25:33, 32.87s/it]
 13%|█▎        | 592/4690 [5:24:03<37:29:42, 32.94s/it]
 13%|█▎        | 593/4690 [5:24:35<37:27:55, 32.92s/it]
 13%|█▎        | 594/4690 [5:25:08<37:19:47, 32.81s/it]
 13%|█▎        | 595/4690 [5:25:41<37:17:19, 32.78s/it]
 13%|█▎        | 596/4690 [5:26:13<37:16:43, 32.78s/it]
 13%|█▎        | 597/4690 [5:26:46<37:10:35, 32.70s/it]
 13%|█▎        | 598/4690 [5:27:19<37:11:10, 32.72s/it]
 13%|█▎        | 599/4690 [5:27:52<37:14:10, 32.77s/it]
 13%|█▎        | 600/4690 [5:28:25<37:20:06, 32.86s/it]
                                                       
{'loss': '0.4836', 'grad_norm': '2.335', 'learning_rate': '9.752e-06', 'epoch': '0.64'}

 13%|█▎        | 600/4690 [5:28:25<37:20:06, 32.86s/it]
 13%|█▎        | 601/4690 [5:28:57<37:08:51, 32.71s/it]
 13%|█▎        | 602/4690 [5:29:30<37:10:24, 32.74s/it]
 13%|█▎        | 603/4690 [5:30:02<37:04:49, 32.66s/it]
 13%|█▎        | 604/4690 [5:30:35<37:00:29, 32.61s/it]
 13%|█▎        | 605/4690 [5:31:08<37:10:10, 32.76s/it]
 13%|█▎        | 606/4690 [5:31:41<37:07:23, 32.72s/it]
 13%|█▎        | 607/4690 [5:32:14<37:16:55, 32.87s/it]
 13%|█▎        | 608/4690 [5:32:46<37:07:52, 32.75s/it]
 13%|█▎        | 609/4690 [5:33:19<37:09:54, 32.78s/it]
 13%|█▎        | 610/4690 [5:33:52<37:09:43, 32.79s/it]
                                                       
{'loss': '0.4975', 'grad_norm': '2.312', 'learning_rate': '9.741e-06', 'epoch': '0.6507'}

 13%|█▎        | 610/4690 [5:33:52<37:09:43, 32.79s/it]
 13%|█▎        | 611/4690 [5:34:25<37:12:31, 32.84s/it]
 13%|█▎        | 612/4690 [5:34:57<37:07:02, 32.77s/it]
 13%|█▎        | 613/4690 [5:35:30<37:01:33, 32.69s/it]
 13%|█▎        | 614/4690 [5:36:03<37:12:00, 32.86s/it]
 13%|█▎        | 615/4690 [5:36:36<37:05:12, 32.76s/it]
 13%|█▎        | 616/4690 [5:37:08<37:04:09, 32.76s/it]
 13%|█▎        | 617/4690 [5:37:42<37:09:43, 32.85s/it]
 13%|█▎        | 618/4690 [5:38:14<37:03:06, 32.76s/it]
 13%|█▎        | 619/4690 [5:38:47<36:59:27, 32.71s/it]
 13%|█▎        | 620/4690 [5:39:19<36:59:08, 32.71s/it]
                                                       
{'loss': '0.4813', 'grad_norm': '2.412', 'learning_rate': '9.73e-06', 'epoch': '0.6613'}

 13%|█▎        | 620/4690 [5:39:19<36:59:08, 32.71s/it]
 13%|█▎        | 621/4690 [5:39:52<37:01:55, 32.76s/it]
 13%|█▎        | 622/4690 [5:40:25<37:02:35, 32.78s/it]
 13%|█▎        | 623/4690 [5:40:58<36:54:38, 32.67s/it]
 13%|█▎        | 624/4690 [5:41:31<37:01:11, 32.78s/it]
 13%|█▎        | 625/4690 [5:42:03<36:52:39, 32.66s/it]
 13%|█▎        | 626/4690 [5:42:36<36:59:51, 32.77s/it]
 13%|█▎        | 627/4690 [5:43:09<37:03:53, 32.84s/it]
 13%|█▎        | 628/4690 [5:43:42<37:10:58, 32.95s/it]
 13%|█▎        | 629/4690 [5:44:15<37:15:18, 33.03s/it]
 13%|█▎        | 630/4690 [5:44:48<37:06:39, 32.91s/it]
                                                       
{'loss': '0.4673', 'grad_norm': '2.701', 'learning_rate': '9.719e-06', 'epoch': '0.672'}

 13%|█▎        | 630/4690 [5:44:48<37:06:39, 32.91s/it]
 13%|█▎        | 631/4690 [5:45:21<37:13:40, 33.02s/it]
 13%|█▎        | 632/4690 [5:45:54<37:11:59, 33.00s/it]
 13%|█▎        | 633/4690 [5:46:27<37:11:20, 33.00s/it]
 14%|█▎        | 634/4690 [5:47:00<36:57:18, 32.80s/it]
 14%|█▎        | 635/4690 [5:47:33<37:01:14, 32.87s/it]
 14%|█▎        | 636/4690 [5:48:05<36:59:24, 32.85s/it]
 14%|█▎        | 637/4690 [5:48:39<37:06:17, 32.96s/it]
 14%|█▎        | 638/4690 [5:49:12<37:06:24, 32.97s/it]
 14%|█▎        | 639/4690 [5:49:44<37:03:34, 32.93s/it]
 14%|█▎        | 640/4690 [5:50:17<36:57:45, 32.86s/it]
                                                       
{'loss': '0.4806', 'grad_norm': '2.412', 'learning_rate': '9.707e-06', 'epoch': '0.6827'}

 14%|█▎        | 640/4690 [5:50:17<36:57:45, 32.86s/it]
 14%|█▎        | 641/4690 [5:50:50<37:00:30, 32.90s/it]
 14%|█▎        | 642/4690 [5:51:23<37:02:42, 32.95s/it]
 14%|█▎        | 643/4690 [5:51:56<36:56:38, 32.86s/it]
 14%|█▎        | 644/4690 [5:52:28<36:49:56, 32.77s/it]
 14%|█▍        | 645/4690 [5:53:02<36:59:41, 32.92s/it]
 14%|█▍        | 646/4690 [5:53:34<36:52:00, 32.82s/it]
 14%|█▍        | 647/4690 [5:54:07<36:50:22, 32.80s/it]
 14%|█▍        | 648/4690 [5:54:40<36:43:00, 32.70s/it]
 14%|█▍        | 649/4690 [5:55:13<36:52:03, 32.84s/it]
 14%|█▍        | 650/4690 [5:55:46<36:57:24, 32.93s/it]
                                                       
{'loss': '0.4903', 'grad_norm': '2.326', 'learning_rate': '9.695e-06', 'epoch': '0.6933'}

 14%|█▍        | 650/4690 [5:55:46<36:57:24, 32.93s/it]
 14%|█▍        | 651/4690 [5:56:18<36:49:00, 32.82s/it]
 14%|█▍        | 652/4690 [5:56:52<36:55:30, 32.92s/it]
 14%|█▍        | 653/4690 [5:57:25<37:03:01, 33.04s/it]
 14%|█▍        | 654/4690 [5:57:57<36:53:58, 32.91s/it]
 14%|█▍        | 655/4690 [5:58:30<36:47:46, 32.83s/it]
 14%|█▍        | 656/4690 [5:59:03<36:42:44, 32.76s/it]
 14%|█▍        | 657/4690 [5:59:36<36:49:10, 32.87s/it]
 14%|█▍        | 658/4690 [6:00:09<36:53:33, 32.94s/it]
 14%|█▍        | 659/4690 [6:00:42<36:47:30, 32.86s/it]
 14%|█▍        | 660/4690 [6:01:14<36:41:43, 32.78s/it]
                                                       
{'loss': '0.4805', 'grad_norm': '2.61', 'learning_rate': '9.683e-06', 'epoch': '0.704'}

 14%|█▍        | 660/4690 [6:01:14<36:41:43, 32.78s/it]
 14%|█▍        | 661/4690 [6:01:47<36:37:25, 32.72s/it]
 14%|█▍        | 662/4690 [6:02:20<36:42:46, 32.81s/it]
 14%|█▍        | 663/4690 [6:02:53<36:44:52, 32.85s/it]
 14%|█▍        | 664/4690 [6:03:26<36:49:32, 32.93s/it]
 14%|█▍        | 665/4690 [6:03:59<36:47:08, 32.90s/it]
 14%|█▍        | 666/4690 [6:04:31<36:41:04, 32.82s/it]
 14%|█▍        | 667/4690 [6:05:04<36:42:47, 32.85s/it]
 14%|█▍        | 668/4690 [6:05:37<36:40:01, 32.82s/it]
 14%|█▍        | 669/4690 [6:06:10<36:39:53, 32.83s/it]
 14%|█▍        | 670/4690 [6:06:43<36:40:42, 32.85s/it]
                                                       
{'loss': '0.4843', 'grad_norm': '2.384', 'learning_rate': '9.671e-06', 'epoch': '0.7147'}

 14%|█▍        | 670/4690 [6:06:43<36:40:42, 32.85s/it]
 14%|█▍        | 671/4690 [6:07:16<36:45:49, 32.93s/it]
 14%|█▍        | 672/4690 [6:07:49<36:46:49, 32.95s/it]
 14%|█▍        | 673/4690 [6:08:22<36:42:30, 32.90s/it]
 14%|█▍        | 674/4690 [6:08:55<36:47:25, 32.98s/it]
 14%|█▍        | 675/4690 [6:09:28<36:46:39, 32.98s/it]
 14%|█▍        | 676/4690 [6:10:00<36:39:44, 32.88s/it]
 14%|█▍        | 677/4690 [6:10:33<36:42:36, 32.93s/it]
 14%|█▍        | 678/4690 [6:11:06<36:40:05, 32.90s/it]
 14%|█▍        | 679/4690 [6:11:39<36:41:55, 32.94s/it]
 14%|█▍        | 680/4690 [6:12:12<36:42:14, 32.95s/it]
                                                       
{'loss': '0.487', 'grad_norm': '2.396', 'learning_rate': '9.659e-06', 'epoch': '0.7253'}

 14%|█▍        | 680/4690 [6:12:12<36:42:14, 32.95s/it]
 15%|█▍        | 681/4690 [6:12:45<36:35:05, 32.85s/it]
 15%|█▍        | 682/4690 [6:13:18<36:32:57, 32.83s/it]
 15%|█▍        | 683/4690 [6:13:50<36:29:10, 32.78s/it]
 15%|█▍        | 684/4690 [6:14:23<36:29:05, 32.79s/it]
 15%|█▍        | 685/4690 [6:14:56<36:31:48, 32.84s/it]
 15%|█▍        | 686/4690 [6:15:29<36:22:39, 32.71s/it]
 15%|█▍        | 687/4690 [6:16:01<36:15:19, 32.61s/it]
 15%|█▍        | 688/4690 [6:16:34<36:19:25, 32.67s/it]
 15%|█▍        | 689/4690 [6:17:06<36:19:59, 32.69s/it]
 15%|█▍        | 690/4690 [6:17:40<36:26:17, 32.79s/it]
                                                       
{'loss': '0.47', 'grad_norm': '2.188', 'learning_rate': '9.646e-06', 'epoch': '0.736'}

 15%|█▍        | 690/4690 [6:17:40<36:26:17, 32.79s/it]
 15%|█▍        | 691/4690 [6:18:13<36:33:58, 32.92s/it]
 15%|█▍        | 692/4690 [6:18:46<36:36:40, 32.97s/it]
 15%|█▍        | 693/4690 [6:19:19<36:34:22, 32.94s/it]
 15%|█▍        | 694/4690 [6:19:51<36:31:06, 32.90s/it]
 15%|█▍        | 695/4690 [6:20:24<36:23:34, 32.79s/it]
 15%|█▍        | 696/4690 [6:20:57<36:21:24, 32.77s/it]
 15%|█▍        | 697/4690 [6:21:30<36:22:23, 32.79s/it]
 15%|█▍        | 698/4690 [6:22:03<36:27:39, 32.88s/it]
 15%|█▍        | 699/4690 [6:22:36<36:30:46, 32.94s/it]
 15%|█▍        | 700/4690 [6:23:08<36:21:35, 32.81s/it]
                                                       
{'loss': '0.4903', 'grad_norm': '2.399', 'learning_rate': '9.633e-06', 'epoch': '0.7467'}

 15%|█▍        | 700/4690 [6:23:08<36:21:35, 32.81s/it]
 15%|█▍        | 701/4690 [6:23:41<36:19:13, 32.78s/it]
 15%|█▍        | 702/4690 [6:24:14<36:21:32, 32.82s/it]
 15%|█▍        | 703/4690 [6:24:47<36:19:37, 32.80s/it]
 15%|█▌        | 704/4690 [6:25:19<36:12:50, 32.71s/it]
 15%|█▌        | 705/4690 [6:25:52<36:11:48, 32.70s/it]
 15%|█▌        | 706/4690 [6:26:25<36:13:53, 32.74s/it]
 15%|█▌        | 707/4690 [6:26:57<36:06:01, 32.63s/it]
 15%|█▌        | 708/4690 [6:27:29<36:02:22, 32.58s/it]
 15%|█▌        | 709/4690 [6:28:02<36:07:33, 32.67s/it]
 15%|█▌        | 710/4690 [6:28:35<36:07:27, 32.68s/it]
                                                       
{'loss': '0.4668', 'grad_norm': '2.547', 'learning_rate': '9.62e-06', 'epoch': '0.7573'}

 15%|█▌        | 710/4690 [6:28:35<36:07:27, 32.68s/it]
 15%|█▌        | 711/4690 [6:29:08<36:09:43, 32.72s/it]
 15%|█▌        | 712/4690 [6:29:41<36:20:52, 32.89s/it]
 15%|█▌        | 713/4690 [6:30:14<36:17:08, 32.85s/it]
 15%|█▌        | 714/4690 [6:30:47<36:15:42, 32.83s/it]
 15%|█▌        | 715/4690 [6:31:20<36:19:33, 32.90s/it]
 15%|█▌        | 716/4690 [6:31:52<36:12:54, 32.81s/it]
 15%|█▌        | 717/4690 [6:32:25<36:16:30, 32.87s/it]
 15%|█▌        | 718/4690 [6:32:58<36:15:36, 32.86s/it]
 15%|█▌        | 719/4690 [6:33:31<36:11:57, 32.82s/it]
 15%|█▌        | 720/4690 [6:34:04<36:11:03, 32.81s/it]
                                                       
{'loss': '0.4839', 'grad_norm': '2.187', 'learning_rate': '9.607e-06', 'epoch': '0.768'}

 15%|█▌        | 720/4690 [6:34:04<36:11:03, 32.81s/it]
 15%|█▌        | 721/4690 [6:34:36<36:06:11, 32.75s/it]
 15%|█▌        | 722/4690 [6:35:09<36:03:44, 32.72s/it]
 15%|█▌        | 723/4690 [6:35:42<36:05:41, 32.76s/it]
 15%|█▌        | 724/4690 [6:36:14<36:02:45, 32.72s/it]
 15%|█▌        | 725/4690 [6:36:47<36:04:36, 32.76s/it]
 15%|█▌        | 726/4690 [6:37:20<36:01:52, 32.72s/it]
 16%|█▌        | 727/4690 [6:37:52<35:57:17, 32.66s/it]
 16%|█▌        | 728/4690 [6:38:26<36:05:20, 32.79s/it]
 16%|█▌        | 729/4690 [6:38:58<35:58:46, 32.70s/it]
 16%|█▌        | 730/4690 [6:39:31<36:00:01, 32.73s/it]
                                                       
{'loss': '0.4883', 'grad_norm': '2.283', 'learning_rate': '9.593e-06', 'epoch': '0.7787'}

 16%|█▌        | 730/4690 [6:39:31<36:00:01, 32.73s/it]
 16%|█▌        | 731/4690 [6:40:04<36:01:29, 32.76s/it]
 16%|█▌        | 732/4690 [6:40:36<35:54:43, 32.66s/it]
 16%|█▌        | 733/4690 [6:41:09<35:57:16, 32.71s/it]
 16%|█▌        | 734/4690 [6:41:41<35:53:21, 32.66s/it]
 16%|█▌        | 735/4690 [6:42:14<35:51:14, 32.64s/it]
 16%|█▌        | 736/4690 [6:42:47<36:01:00, 32.79s/it]
 16%|█▌        | 737/4690 [6:43:20<35:57:06, 32.74s/it]
 16%|█▌        | 738/4690 [6:43:52<35:49:13, 32.63s/it]
 16%|█▌        | 739/4690 [6:44:26<36:03:59, 32.86s/it]
 16%|█▌        | 740/4690 [6:44:58<35:59:51, 32.81s/it]
                                                       
{'loss': '0.4781', 'grad_norm': '2.29', 'learning_rate': '9.58e-06', 'epoch': '0.7893'}

 16%|█▌        | 740/4690 [6:44:58<35:59:51, 32.81s/it]
 16%|█▌        | 741/4690 [6:45:31<35:55:31, 32.75s/it]
 16%|█▌        | 742/4690 [6:46:04<35:53:48, 32.73s/it]
 16%|█▌        | 743/4690 [6:46:36<35:54:44, 32.76s/it]
 16%|█▌        | 744/4690 [6:47:09<35:50:35, 32.70s/it]
 16%|█▌        | 745/4690 [6:47:41<35:46:37, 32.65s/it]
 16%|█▌        | 746/4690 [6:48:14<35:43:25, 32.61s/it]
 16%|█▌        | 747/4690 [6:48:47<35:44:41, 32.64s/it]
 16%|█▌        | 748/4690 [6:49:19<35:41:27, 32.59s/it]
 16%|█▌        | 749/4690 [6:49:52<35:37:59, 32.55s/it]
 16%|█▌        | 750/4690 [6:50:24<35:38:42, 32.57s/it]
                                                       
{'loss': '0.4758', 'grad_norm': '2.287', 'learning_rate': '9.566e-06', 'epoch': '0.8'}

 16%|█▌        | 750/4690 [6:50:24<35:38:42, 32.57s/it]
 16%|█▌        | 751/4690 [6:50:57<35:43:59, 32.66s/it]
 16%|█▌        | 752/4690 [6:51:30<35:43:48, 32.66s/it]
 16%|█▌        | 753/4690 [6:52:03<35:45:55, 32.70s/it]
 16%|█▌        | 754/4690 [6:52:35<35:44:07, 32.68s/it]
 16%|█▌        | 755/4690 [6:53:08<35:39:56, 32.63s/it]
 16%|█▌        | 756/4690 [6:53:41<35:43:16, 32.69s/it]
 16%|█▌        | 757/4690 [6:54:13<35:46:06, 32.74s/it]
 16%|█▌        | 758/4690 [6:54:46<35:46:26, 32.75s/it]
 16%|█▌        | 759/4690 [6:55:19<35:51:10, 32.83s/it]
 16%|█▌        | 760/4690 [6:55:52<35:50:31, 32.83s/it]
                                                       
{'loss': '0.4841', 'grad_norm': '2.306', 'learning_rate': '9.551e-06', 'epoch': '0.8107'}

 16%|█▌        | 760/4690 [6:55:52<35:50:31, 32.83s/it]
 16%|█▌        | 761/4690 [6:56:25<35:57:55, 32.95s/it]
 16%|█▌        | 762/4690 [6:56:58<35:50:00, 32.84s/it]
 16%|█▋        | 763/4690 [6:57:31<35:52:55, 32.89s/it]
 16%|█▋        | 764/4690 [6:58:04<35:49:41, 32.85s/it]
 16%|█▋        | 765/4690 [6:58:36<35:47:14, 32.82s/it]
 16%|█▋        | 766/4690 [6:59:10<35:55:17, 32.96s/it]
 16%|█▋        | 767/4690 [6:59:42<35:50:35, 32.89s/it]
 16%|█▋        | 768/4690 [7:00:15<35:48:02, 32.86s/it]
 16%|█▋        | 769/4690 [7:00:48<35:46:03, 32.84s/it]
 16%|█▋        | 770/4690 [7:01:20<35:36:27, 32.70s/it]
                                                       
{'loss': '0.4821', 'grad_norm': '2.327', 'learning_rate': '9.537e-06', 'epoch': '0.8213'}

 16%|█▋        | 770/4690 [7:01:20<35:36:27, 32.70s/it]
 16%|█▋        | 771/4690 [7:01:53<35:36:36, 32.71s/it]
 16%|█▋        | 772/4690 [7:02:26<35:38:07, 32.74s/it]
 16%|█▋        | 773/4690 [7:02:59<35:50:12, 32.94s/it]
 17%|█▋        | 774/4690 [7:03:32<35:40:35, 32.80s/it]
 17%|█▋        | 775/4690 [7:04:05<35:44:09, 32.86s/it]
 17%|█▋        | 776/4690 [7:04:38<35:41:42, 32.83s/it]
 17%|█▋        | 777/4690 [7:05:10<35:43:09, 32.86s/it]
 17%|█▋        | 778/4690 [7:05:43<35:37:48, 32.79s/it]
 17%|█▋        | 779/4690 [7:06:16<35:41:26, 32.85s/it]
 17%|█▋        | 780/4690 [7:06:49<35:39:35, 32.83s/it]
                                                       
{'loss': '0.4839', 'grad_norm': '2.356', 'learning_rate': '9.522e-06', 'epoch': '0.832'}

 17%|█▋        | 780/4690 [7:06:49<35:39:35, 32.83s/it]
 17%|█▋        | 781/4690 [7:07:22<35:35:52, 32.78s/it]
 17%|█▋        | 782/4690 [7:07:54<35:34:54, 32.78s/it]
 17%|█▋        | 783/4690 [7:08:27<35:32:23, 32.75s/it]
 17%|█▋        | 784/4690 [7:09:00<35:34:59, 32.80s/it]
 17%|█▋        | 785/4690 [7:09:33<35:33:24, 32.78s/it]
 17%|█▋        | 786/4690 [7:10:06<35:34:27, 32.80s/it]
 17%|█▋        | 787/4690 [7:10:38<35:30:38, 32.75s/it]
 17%|█▋        | 788/4690 [7:11:11<35:33:07, 32.80s/it]
 17%|█▋        | 789/4690 [7:11:44<35:30:23, 32.77s/it]
 17%|█▋        | 790/4690 [7:12:17<35:33:41, 32.83s/it]
                                                       
{'loss': '0.4778', 'grad_norm': '2.283', 'learning_rate': '9.508e-06', 'epoch': '0.8427'}

 17%|█▋        | 790/4690 [7:12:17<35:33:41, 32.83s/it]
 17%|█▋        | 791/4690 [7:12:50<35:35:47, 32.87s/it]
 17%|█▋        | 792/4690 [7:13:23<35:38:08, 32.91s/it]
 17%|█▋        | 793/4690 [7:13:56<35:42:40, 32.99s/it]
 17%|█▋        | 794/4690 [7:14:29<35:41:31, 32.98s/it]
 17%|█▋        | 795/4690 [7:15:02<35:37:25, 32.93s/it]
 17%|█▋        | 796/4690 [7:15:35<35:41:31, 33.00s/it]
 17%|█▋        | 797/4690 [7:16:08<35:44:26, 33.05s/it]
 17%|█▋        | 798/4690 [7:16:41<35:44:46, 33.06s/it]
 17%|█▋        | 799/4690 [7:17:13<35:30:22, 32.85s/it]
 17%|█▋        | 800/4690 [7:17:46<35:30:55, 32.87s/it]
                                                       
{'loss': '0.4635', 'grad_norm': '1.959', 'learning_rate': '9.493e-06', 'epoch': '0.8533'}

 17%|█▋        | 800/4690 [7:17:46<35:30:55, 32.87s/it]
 17%|█▋        | 801/4690 [7:18:20<35:38:49, 33.00s/it]
 17%|█▋        | 802/4690 [7:18:52<35:30:23, 32.88s/it]
 17%|█▋        | 803/4690 [7:19:25<35:20:04, 32.73s/it]
 17%|█▋        | 804/4690 [7:19:57<35:21:21, 32.75s/it]
 17%|█▋        | 805/4690 [7:20:30<35:16:05, 32.68s/it]
 17%|█▋        | 806/4690 [7:21:03<35:15:25, 32.68s/it]
 17%|█▋        | 807/4690 [7:21:35<35:13:50, 32.66s/it]
 17%|█▋        | 808/4690 [7:22:08<35:11:29, 32.64s/it]
 17%|█▋        | 809/4690 [7:22:40<35:06:41, 32.57s/it]
 17%|█▋        | 810/4690 [7:23:13<35:15:04, 32.71s/it]
                                                       
{'loss': '0.4811', 'grad_norm': '2.428', 'learning_rate': '9.477e-06', 'epoch': '0.864'}

 17%|█▋        | 810/4690 [7:23:13<35:15:04, 32.71s/it]
 17%|█▋        | 811/4690 [7:23:46<35:16:10, 32.73s/it]
 17%|█▋        | 812/4690 [7:24:19<35:15:35, 32.73s/it]
 17%|█▋        | 813/4690 [7:24:52<35:16:20, 32.75s/it]
 17%|█▋        | 814/4690 [7:25:24<35:19:20, 32.81s/it]
 17%|█▋        | 815/4690 [7:25:58<35:23:42, 32.88s/it]
 17%|█▋        | 816/4690 [7:26:30<35:18:12, 32.81s/it]
 17%|█▋        | 817/4690 [7:27:03<35:16:23, 32.79s/it]
 17%|█▋        | 818/4690 [7:27:36<35:14:55, 32.77s/it]
 17%|█▋        | 819/4690 [7:28:09<35:19:48, 32.86s/it]
 17%|█▋        | 820/4690 [7:28:42<35:21:09, 32.89s/it]
                                                       
{'loss': '0.4723', 'grad_norm': '2.204', 'learning_rate': '9.462e-06', 'epoch': '0.8747'}

 17%|█▋        | 820/4690 [7:28:42<35:21:09, 32.89s/it]
 18%|█▊        | 821/4690 [7:29:14<35:16:07, 32.82s/it]
 18%|█▊        | 822/4690 [7:29:47<35:18:35, 32.86s/it]
 18%|█▊        | 823/4690 [7:30:20<35:14:13, 32.80s/it]
 18%|█▊        | 824/4690 [7:30:53<35:10:51, 32.76s/it]
 18%|█▊        | 825/4690 [7:31:25<35:09:58, 32.76s/it]
 18%|█▊        | 826/4690 [7:31:58<35:11:23, 32.79s/it]
 18%|█▊        | 827/4690 [7:32:31<35:07:26, 32.73s/it]
 18%|█▊        | 828/4690 [7:33:04<35:15:06, 32.86s/it]
 18%|█▊        | 829/4690 [7:33:36<35:06:26, 32.73s/it]
 18%|█▊        | 830/4690 [7:34:10<35:19:54, 32.95s/it]
                                                       
{'loss': '0.4747', 'grad_norm': '2.342', 'learning_rate': '9.446e-06', 'epoch': '0.8853'}

 18%|█▊        | 830/4690 [7:34:10<35:19:54, 32.95s/it]
 18%|█▊        | 831/4690 [7:34:43<35:17:58, 32.93s/it]
 18%|█▊        | 832/4690 [7:35:16<35:13:54, 32.88s/it]
 18%|█▊        | 833/4690 [7:35:48<35:06:22, 32.77s/it]
 18%|█▊        | 834/4690 [7:36:21<35:10:54, 32.85s/it]
 18%|█▊        | 835/4690 [7:36:54<35:06:00, 32.78s/it]
 18%|█▊        | 836/4690 [7:37:26<35:05:16, 32.78s/it]
 18%|█▊        | 837/4690 [7:37:59<34:59:50, 32.70s/it]
 18%|█▊        | 838/4690 [7:38:32<35:00:27, 32.72s/it]
 18%|█▊        | 839/4690 [7:39:05<35:01:21, 32.74s/it]
 18%|█▊        | 840/4690 [7:39:38<35:07:48, 32.85s/it]
                                                       
{'loss': '0.4756', 'grad_norm': '2.308', 'learning_rate': '9.43e-06', 'epoch': '0.896'}

 18%|█▊        | 840/4690 [7:39:38<35:07:48, 32.85s/it]
 18%|█▊        | 841/4690 [7:40:10<35:01:19, 32.76s/it]
 18%|█▊        | 842/4690 [7:40:43<35:01:26, 32.77s/it]
 18%|█▊        | 843/4690 [7:41:16<35:02:59, 32.80s/it]
 18%|█▊        | 844/4690 [7:41:49<35:00:39, 32.77s/it]
 18%|█▊        | 845/4690 [7:42:21<34:59:23, 32.76s/it]
 18%|█▊        | 846/4690 [7:42:54<35:03:46, 32.84s/it]
 18%|█▊        | 847/4690 [7:43:27<35:00:05, 32.79s/it]
 18%|█▊        | 848/4690 [7:43:59<34:50:04, 32.64s/it]
 18%|█▊        | 849/4690 [7:44:32<34:59:19, 32.79s/it]
 18%|█▊        | 850/4690 [7:45:05<35:02:44, 32.86s/it]
                                                       
{'loss': '0.4609', 'grad_norm': '2.469', 'learning_rate': '9.414e-06', 'epoch': '0.9067'}

 18%|█▊        | 850/4690 [7:45:05<35:02:44, 32.86s/it]
 18%|█▊        | 851/4690 [7:45:38<34:58:34, 32.80s/it]
 18%|█▊        | 852/4690 [7:46:11<34:54:20, 32.74s/it]
 18%|█▊        | 853/4690 [7:46:43<34:51:45, 32.71s/it]
 18%|█▊        | 854/4690 [7:47:16<34:52:42, 32.73s/it]
 18%|█▊        | 855/4690 [7:47:48<34:45:06, 32.62s/it]
 18%|█▊        | 856/4690 [7:48:21<34:51:42, 32.73s/it]
 18%|█▊        | 857/4690 [7:48:54<34:51:52, 32.75s/it]
 18%|█▊        | 858/4690 [7:49:27<34:56:05, 32.82s/it]
 18%|█▊        | 859/4690 [7:50:00<34:54:12, 32.80s/it]
 18%|█▊        | 860/4690 [7:50:32<34:48:06, 32.71s/it]
                                                       
{'loss': '0.4672', 'grad_norm': '2.253', 'learning_rate': '9.398e-06', 'epoch': '0.9173'}

 18%|█▊        | 860/4690 [7:50:32<34:48:06, 32.71s/it]
 18%|█▊        | 861/4690 [7:51:06<34:57:19, 32.86s/it]
 18%|█▊        | 862/4690 [7:51:38<34:49:46, 32.76s/it]
 18%|█▊        | 863/4690 [7:52:11<34:43:52, 32.67s/it]
 18%|█▊        | 864/4690 [7:52:43<34:45:53, 32.71s/it]
 18%|█▊        | 865/4690 [7:53:16<34:48:02, 32.75s/it]
 18%|█▊        | 866/4690 [7:53:50<34:55:27, 32.88s/it]
 18%|█▊        | 867/4690 [7:54:22<34:56:26, 32.90s/it]
 19%|█▊        | 868/4690 [7:54:55<34:48:33, 32.79s/it]
 19%|█▊        | 869/4690 [7:55:28<34:45:01, 32.74s/it]
 19%|█▊        | 870/4690 [7:56:00<34:44:44, 32.74s/it]
                                                       
{'loss': '0.4596', 'grad_norm': '2.673', 'learning_rate': '9.381e-06', 'epoch': '0.928'}

 19%|█▊        | 870/4690 [7:56:00<34:44:44, 32.74s/it]
 19%|█▊        | 871/4690 [7:56:34<34:56:53, 32.94s/it]
 19%|█▊        | 872/4690 [7:57:07<34:54:05, 32.91s/it]
 19%|█▊        | 873/4690 [7:57:39<34:52:56, 32.90s/it]
 19%|█▊        | 874/4690 [7:58:12<34:49:25, 32.85s/it]
 19%|█▊        | 875/4690 [7:58:45<34:45:24, 32.80s/it]
 19%|█▊        | 876/4690 [7:59:17<34:39:28, 32.71s/it]
 19%|█▊        | 877/4690 [7:59:50<34:38:56, 32.71s/it]
 19%|█▊        | 878/4690 [8:00:23<34:40:28, 32.75s/it]
 19%|█▊        | 879/4690 [8:00:56<34:40:58, 32.76s/it]
 19%|█▉        | 880/4690 [8:01:29<34:45:52, 32.85s/it]
                                                       
{'loss': '0.4709', 'grad_norm': '2.119', 'learning_rate': '9.365e-06', 'epoch': '0.9387'}

 19%|█▉        | 880/4690 [8:01:29<34:45:52, 32.85s/it]
 19%|█▉        | 881/4690 [8:02:02<34:51:27, 32.94s/it]
 19%|█▉        | 882/4690 [8:02:34<34:38:07, 32.74s/it]
 19%|█▉        | 883/4690 [8:03:07<34:41:23, 32.80s/it]
 19%|█▉        | 884/4690 [8:03:40<34:46:21, 32.89s/it]
 19%|█▉        | 885/4690 [8:04:13<34:49:03, 32.94s/it]
 19%|█▉        | 886/4690 [8:04:46<34:52:25, 33.00s/it]
 19%|█▉        | 887/4690 [8:05:19<34:39:23, 32.81s/it]
 19%|█▉        | 888/4690 [8:05:51<34:33:40, 32.73s/it]
 19%|█▉        | 889/4690 [8:06:24<34:29:36, 32.67s/it]
 19%|█▉        | 890/4690 [8:06:57<34:27:57, 32.65s/it]
                                                       
{'loss': '0.4814', 'grad_norm': '2.669', 'learning_rate': '9.348e-06', 'epoch': '0.9493'}

 19%|█▉        | 890/4690 [8:06:57<34:27:57, 32.65s/it]
 19%|█▉        | 891/4690 [8:07:29<34:29:39, 32.69s/it]
 19%|█▉        | 892/4690 [8:08:02<34:30:08, 32.70s/it]
 19%|█▉        | 893/4690 [8:08:35<34:33:09, 32.76s/it]
 19%|█▉        | 894/4690 [8:09:08<34:31:04, 32.74s/it]
 19%|█▉        | 895/4690 [8:09:41<34:39:16, 32.87s/it]
 19%|█▉        | 896/4690 [8:10:14<34:40:31, 32.90s/it]
 19%|█▉        | 897/4690 [8:10:46<34:36:36, 32.85s/it]
 19%|█▉        | 898/4690 [8:11:19<34:32:43, 32.80s/it]
 19%|█▉        | 899/4690 [8:11:52<34:30:01, 32.76s/it]
 19%|█▉        | 900/4690 [8:12:25<34:31:03, 32.79s/it]
                                                       
{'loss': '0.4699', 'grad_norm': '2.293', 'learning_rate': '9.33e-06', 'epoch': '0.96'}

 19%|█▉        | 900/4690 [8:12:25<34:31:03, 32.79s/it]
 19%|█▉        | 901/4690 [8:12:58<34:39:02, 32.92s/it]
 19%|█▉        | 902/4690 [8:13:31<34:34:02, 32.85s/it]
 19%|█▉        | 903/4690 [8:14:04<34:36:22, 32.90s/it]
 19%|█▉        | 904/4690 [8:14:36<34:27:37, 32.77s/it]
 19%|█▉        | 905/4690 [8:15:09<34:27:04, 32.77s/it]
 19%|█▉        | 906/4690 [8:15:42<34:26:25, 32.77s/it]
 19%|█▉        | 907/4690 [8:16:14<34:16:30, 32.62s/it]
 19%|█▉        | 908/4690 [8:16:46<34:13:46, 32.58s/it]
 19%|█▉        | 909/4690 [8:17:19<34:16:28, 32.63s/it]
 19%|█▉        | 910/4690 [8:17:52<34:22:32, 32.74s/it]
                                                       
{'loss': '0.4774', 'grad_norm': '2.384', 'learning_rate': '9.313e-06', 'epoch': '0.9707'}

 19%|█▉        | 910/4690 [8:17:52<34:22:32, 32.74s/it]
 19%|█▉        | 911/4690 [8:18:25<34:17:12, 32.66s/it]
 19%|█▉        | 912/4690 [8:18:57<34:13:29, 32.61s/it]
 19%|█▉        | 913/4690 [8:19:31<34:29:25, 32.87s/it]
 19%|█▉        | 914/4690 [8:20:03<34:23:58, 32.80s/it]
 20%|█▉        | 915/4690 [8:20:36<34:26:17, 32.84s/it]
 20%|█▉        | 916/4690 [8:21:09<34:20:00, 32.75s/it]
 20%|█▉        | 917/4690 [8:21:41<34:19:16, 32.75s/it]
 20%|█▉        | 918/4690 [8:22:14<34:16:40, 32.71s/it]
 20%|█▉        | 919/4690 [8:22:47<34:16:35, 32.72s/it]
 20%|█▉        | 920/4690 [8:23:20<34:17:42, 32.75s/it]
                                                       
{'loss': '0.4684', 'grad_norm': '2.28', 'learning_rate': '9.295e-06', 'epoch': '0.9813'}

 20%|█▉        | 920/4690 [8:23:20<34:17:42, 32.75s/it]
 20%|█▉        | 921/4690 [8:23:52<34:15:11, 32.72s/it]
 20%|█▉        | 922/4690 [8:24:25<34:18:47, 32.78s/it]
 20%|█▉        | 923/4690 [8:24:58<34:16:12, 32.75s/it]
 20%|█▉        | 924/4690 [8:25:31<34:17:14, 32.78s/it]
 20%|█▉        | 925/4690 [8:26:03<34:15:59, 32.76s/it]
 20%|█▉        | 926/4690 [8:26:36<34:19:19, 32.83s/it]
 20%|█▉        | 927/4690 [8:27:09<34:13:47, 32.75s/it]
 20%|█▉        | 928/4690 [8:27:42<34:17:04, 32.81s/it]
 20%|█▉        | 929/4690 [8:28:15<34:17:33, 32.82s/it]
 20%|█▉        | 930/4690 [8:28:48<34:20:47, 32.88s/it]
                                                       
{'loss': '0.4542', 'grad_norm': '2.391', 'learning_rate': '9.278e-06', 'epoch': '0.992'}

 20%|█▉        | 930/4690 [8:28:48<34:20:47, 32.88s/it]
 20%|█▉        | 931/4690 [8:29:21<34:27:20, 33.00s/it]
 20%|█▉        | 932/4690 [8:29:54<34:19:32, 32.88s/it]
 20%|█▉        | 933/4690 [8:30:26<34:15:19, 32.82s/it]
 20%|█▉        | 934/4690 [8:30:59<34:14:59, 32.83s/it]
 20%|█▉        | 935/4690 [8:31:32<34:16:18, 32.86s/it]
 20%|█▉        | 936/4690 [8:32:05<34:18:14, 32.90s/it]
 20%|█▉        | 937/4690 [8:32:38<34:20:46, 32.95s/it]
 20%|██        | 938/4690 [8:32:55<29:14:49, 28.06s/it][INFO|trainer.py:3815] 2026-05-14 09:54:18,882 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938
[INFO|configuration_utils.py:533] 2026-05-14 09:54:18,902 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/config.json
[INFO|configuration_utils.py:816] 2026-05-14 09:54:18,903 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/generation_config.json


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:15<00:00, 15.67s/it]
Writing model shards: 100%|██████████| 1/1 [00:15<00:00, 15.67s/it]
[INFO|modeling_utils.py:3471] 2026-05-14 09:54:34,626 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-14 09:54:34,629 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-14 09:54:34,631 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/tokenizer_config.json
[INFO|tokenization_utils_base.py:3278] 2026-05-14 09:55:09,166 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-14 09:55:09,168 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-14 09:55:09,319 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-938/processor_config.json

 20%|██        | 939/4690 [8:34:29<49:58:47, 47.97s/it]
 20%|██        | 940/4690 [8:35:02<45:08:37, 43.34s/it]
                                                       
{'loss': '0.4227', 'grad_norm': '1.753', 'learning_rate': '9.26e-06', 'epoch': '1.002'}

 20%|██        | 940/4690 [8:35:02<45:08:37, 43.34s/it]
 20%|██        | 941/4690 [8:35:35<41:48:39, 40.15s/it]
 20%|██        | 942/4690 [8:36:07<39:27:33, 37.90s/it]
 20%|██        | 943/4690 [8:36:40<37:49:16, 36.34s/it]
 20%|██        | 944/4690 [8:37:12<36:38:33, 35.21s/it]
 20%|██        | 945/4690 [8:37:45<35:50:30, 34.45s/it]
 20%|██        | 946/4690 [8:38:18<35:19:19, 33.96s/it]
 20%|██        | 947/4690 [8:38:51<34:53:43, 33.56s/it]
 20%|██        | 948/4690 [8:39:23<34:34:46, 33.27s/it]
 20%|██        | 949/4690 [8:39:56<34:26:00, 33.14s/it]
 20%|██        | 950/4690 [8:40:29<34:17:00, 33.00s/it]
                                                       
{'loss': '0.2571', 'grad_norm': '2.367', 'learning_rate': '9.242e-06', 'epoch': '1.013'}

 20%|██        | 950/4690 [8:40:29<34:17:00, 33.00s/it]
 20%|██        | 951/4690 [8:41:02<34:14:26, 32.97s/it]
 20%|██        | 952/4690 [8:41:35<34:15:55, 33.00s/it]
 20%|██        | 953/4690 [8:42:07<34:11:50, 32.94s/it]
 20%|██        | 954/4690 [8:42:40<34:03:39, 32.82s/it]
 20%|██        | 955/4690 [8:43:13<34:05:40, 32.86s/it]
 20%|██        | 956/4690 [8:43:46<34:02:33, 32.82s/it]
 20%|██        | 957/4690 [8:44:19<34:05:01, 32.87s/it]
 20%|██        | 958/4690 [8:44:51<33:59:30, 32.79s/it]
 20%|██        | 959/4690 [8:45:24<33:52:10, 32.68s/it]
 20%|██        | 960/4690 [8:45:56<33:53:38, 32.71s/it]
                                                       
{'loss': '0.2519', 'grad_norm': '2.169', 'learning_rate': '9.223e-06', 'epoch': '1.023'}

 20%|██        | 960/4690 [8:45:56<33:53:38, 32.71s/it]
 20%|██        | 961/4690 [8:46:29<33:52:38, 32.71s/it]
 21%|██        | 962/4690 [8:47:02<33:58:45, 32.81s/it]
 21%|██        | 963/4690 [8:47:35<33:55:47, 32.77s/it]
 21%|██        | 964/4690 [8:48:08<33:52:23, 32.73s/it]
 21%|██        | 965/4690 [8:48:40<33:55:56, 32.79s/it]
 21%|██        | 966/4690 [8:49:13<33:54:21, 32.78s/it]
 21%|██        | 967/4690 [8:49:46<33:53:18, 32.77s/it]
 21%|██        | 968/4690 [8:50:19<33:51:36, 32.75s/it]
 21%|██        | 969/4690 [8:50:52<33:57:22, 32.85s/it]
 21%|██        | 970/4690 [8:51:25<33:55:26, 32.83s/it]
                                                       
{'loss': '0.2573', 'grad_norm': '2.211', 'learning_rate': '9.205e-06', 'epoch': '1.034'}

 21%|██        | 970/4690 [8:51:25<33:55:26, 32.83s/it]
 21%|██        | 971/4690 [8:51:58<33:59:53, 32.91s/it]
 21%|██        | 972/4690 [8:52:30<33:54:05, 32.83s/it]
 21%|██        | 973/4690 [8:53:03<33:48:56, 32.75s/it]
 21%|██        | 974/4690 [8:53:36<33:48:14, 32.75s/it]
 21%|██        | 975/4690 [8:54:08<33:43:11, 32.68s/it]
 21%|██        | 976/4690 [8:54:41<33:41:14, 32.65s/it]
 21%|██        | 977/4690 [8:55:14<33:47:28, 32.76s/it]
 21%|██        | 978/4690 [8:55:47<33:54:09, 32.88s/it]
 21%|██        | 979/4690 [8:56:20<33:54:18, 32.89s/it]
 21%|██        | 980/4690 [8:56:53<33:53:02, 32.88s/it]
                                                       
{'loss': '0.2479', 'grad_norm': '2.36', 'learning_rate': '9.186e-06', 'epoch': '1.045'}

 21%|██        | 980/4690 [8:56:53<33:53:02, 32.88s/it]
 21%|██        | 981/4690 [8:57:26<33:55:41, 32.93s/it]
 21%|██        | 982/4690 [8:57:58<33:49:10, 32.83s/it]
 21%|██        | 983/4690 [8:58:31<33:51:50, 32.89s/it]
 21%|██        | 984/4690 [8:59:04<33:53:42, 32.93s/it]
 21%|██        | 985/4690 [8:59:37<33:50:40, 32.89s/it]
 21%|██        | 986/4690 [9:00:10<33:47:19, 32.84s/it]
 21%|██        | 987/4690 [9:00:43<33:49:34, 32.89s/it]
 21%|██        | 988/4690 [9:01:16<33:48:02, 32.87s/it]
 21%|██        | 989/4690 [9:01:48<33:45:15, 32.83s/it]
 21%|██        | 990/4690 [9:02:21<33:47:01, 32.87s/it]
                                                       
{'loss': '0.2469', 'grad_norm': '2.123', 'learning_rate': '9.167e-06', 'epoch': '1.055'}

 21%|██        | 990/4690 [9:02:21<33:47:01, 32.87s/it]
 21%|██        | 991/4690 [9:02:54<33:40:02, 32.77s/it]
 21%|██        | 992/4690 [9:03:27<33:39:12, 32.76s/it]
 21%|██        | 993/4690 [9:03:59<33:37:42, 32.75s/it]
 21%|██        | 994/4690 [9:04:32<33:41:38, 32.82s/it]
 21%|██        | 995/4690 [9:05:05<33:45:50, 32.90s/it]
 21%|██        | 996/4690 [9:05:38<33:33:54, 32.71s/it]
 21%|██▏       | 997/4690 [9:06:11<33:38:15, 32.79s/it]
 21%|██▏       | 998/4690 [9:06:43<33:35:22, 32.75s/it]
 21%|██▏       | 999/4690 [9:07:16<33:28:37, 32.65s/it]
 21%|██▏       | 1000/4690 [9:07:49<33:30:33, 32.69s/it]
                                                        
{'loss': '0.2449', 'grad_norm': '2.179', 'learning_rate': '9.148e-06', 'epoch': '1.066'}

 21%|██▏       | 1000/4690 [9:07:49<33:30:33, 32.69s/it]
 21%|██▏       | 1001/4690 [9:08:21<33:32:20, 32.73s/it]
 21%|██▏       | 1002/4690 [9:08:54<33:27:03, 32.65s/it]
 21%|██▏       | 1003/4690 [9:09:26<33:25:31, 32.64s/it]
 21%|██▏       | 1004/4690 [9:09:59<33:23:01, 32.60s/it]
 21%|██▏       | 1005/4690 [9:10:32<33:26:38, 32.67s/it]
 21%|██▏       | 1006/4690 [9:11:04<33:22:19, 32.61s/it]
 21%|██▏       | 1007/4690 [9:11:37<33:15:33, 32.51s/it]
 21%|██▏       | 1008/4690 [9:12:09<33:14:22, 32.50s/it]
 22%|██▏       | 1009/4690 [9:12:42<33:25:36, 32.69s/it]
 22%|██▏       | 1010/4690 [9:13:15<33:25:00, 32.69s/it]
                                                        
{'loss': '0.2364', 'grad_norm': '2.032', 'learning_rate': '9.128e-06', 'epoch': '1.077'}

 22%|██▏       | 1010/4690 [9:13:15<33:25:00, 32.69s/it]
 22%|██▏       | 1011/4690 [9:13:48<33:32:38, 32.82s/it]
 22%|██▏       | 1012/4690 [9:14:21<33:36:29, 32.90s/it]
 22%|██▏       | 1013/4690 [9:14:54<33:28:46, 32.78s/it]
 22%|██▏       | 1014/4690 [9:15:26<33:21:03, 32.66s/it]
 22%|██▏       | 1015/4690 [9:15:59<33:28:53, 32.80s/it]
 22%|██▏       | 1016/4690 [9:16:32<33:24:30, 32.74s/it]
 22%|██▏       | 1017/4690 [9:17:04<33:24:00, 32.74s/it]
 22%|██▏       | 1018/4690 [9:17:37<33:27:14, 32.80s/it]
 22%|██▏       | 1019/4690 [9:18:10<33:24:12, 32.76s/it]
 22%|██▏       | 1020/4690 [9:18:43<33:23:24, 32.75s/it]
                                                        
{'loss': '0.2441', 'grad_norm': '2.146', 'learning_rate': '9.109e-06', 'epoch': '1.087'}

 22%|██▏       | 1020/4690 [9:18:43<33:23:24, 32.75s/it]
 22%|██▏       | 1021/4690 [9:19:16<33:25:04, 32.79s/it]
 22%|██▏       | 1022/4690 [9:19:48<33:23:03, 32.77s/it]
 22%|██▏       | 1023/4690 [9:20:21<33:22:59, 32.77s/it]
 22%|██▏       | 1024/4690 [9:20:54<33:21:14, 32.75s/it]
 22%|██▏       | 1025/4690 [9:21:27<33:20:34, 32.75s/it]
 22%|██▏       | 1026/4690 [9:21:59<33:18:28, 32.73s/it]
 22%|██▏       | 1027/4690 [9:22:32<33:25:54, 32.86s/it]
 22%|██▏       | 1028/4690 [9:23:05<33:24:14, 32.84s/it]
 22%|██▏       | 1029/4690 [9:23:38<33:26:00, 32.88s/it]
 22%|██▏       | 1030/4690 [9:24:11<33:25:42, 32.88s/it]
                                                        
{'loss': '0.2529', 'grad_norm': '2.114', 'learning_rate': '9.089e-06', 'epoch': '1.098'}

 22%|██▏       | 1030/4690 [9:24:11<33:25:42, 32.88s/it]
 22%|██▏       | 1031/4690 [9:24:44<33:21:27, 32.82s/it]
 22%|██▏       | 1032/4690 [9:25:16<33:19:16, 32.79s/it]
 22%|██▏       | 1033/4690 [9:25:49<33:19:24, 32.80s/it]
 22%|██▏       | 1034/4690 [9:26:22<33:16:20, 32.76s/it]
 22%|██▏       | 1035/4690 [9:26:55<33:19:52, 32.83s/it]
 22%|██▏       | 1036/4690 [9:27:28<33:21:23, 32.86s/it]
 22%|██▏       | 1037/4690 [9:28:01<33:21:31, 32.87s/it]
 22%|██▏       | 1038/4690 [9:28:34<33:20:58, 32.87s/it]
 22%|██▏       | 1039/4690 [9:29:06<33:16:54, 32.82s/it]
 22%|██▏       | 1040/4690 [9:29:39<33:11:54, 32.74s/it]
                                                        
{'loss': '0.2406', 'grad_norm': '2.035', 'learning_rate': '9.069e-06', 'epoch': '1.109'}

 22%|██▏       | 1040/4690 [9:29:39<33:11:54, 32.74s/it]
 22%|██▏       | 1041/4690 [9:30:12<33:16:52, 32.83s/it]
 22%|██▏       | 1042/4690 [9:30:45<33:15:01, 32.81s/it]
 22%|██▏       | 1043/4690 [9:31:18<33:15:52, 32.84s/it]
 22%|██▏       | 1044/4690 [9:31:51<33:20:22, 32.92s/it]
 22%|██▏       | 1045/4690 [9:32:23<33:17:28, 32.88s/it]
 22%|██▏       | 1046/4690 [9:32:56<33:13:04, 32.82s/it]
 22%|██▏       | 1047/4690 [9:33:29<33:07:42, 32.74s/it]
 22%|██▏       | 1048/4690 [9:34:02<33:09:46, 32.78s/it]
 22%|██▏       | 1049/4690 [9:34:34<33:05:22, 32.72s/it]
 22%|██▏       | 1050/4690 [9:35:07<33:05:12, 32.72s/it]
                                                        
{'loss': '0.2438', 'grad_norm': '2.242', 'learning_rate': '9.049e-06', 'epoch': '1.119'}

 22%|██▏       | 1050/4690 [9:35:07<33:05:12, 32.72s/it]
 22%|██▏       | 1051/4690 [9:35:39<32:56:08, 32.58s/it]
 22%|██▏       | 1052/4690 [9:36:12<33:02:40, 32.70s/it]
 22%|██▏       | 1053/4690 [9:36:45<33:06:27, 32.77s/it]
 22%|██▏       | 1054/4690 [9:37:18<33:03:22, 32.73s/it]
 22%|██▏       | 1055/4690 [9:37:50<33:01:31, 32.71s/it]
 23%|██▎       | 1056/4690 [9:38:23<33:01:28, 32.72s/it]
 23%|██▎       | 1057/4690 [9:38:56<33:02:27, 32.74s/it]
 23%|██▎       | 1058/4690 [9:39:29<33:08:47, 32.85s/it]
 23%|██▎       | 1059/4690 [9:40:02<33:09:43, 32.88s/it]
 23%|██▎       | 1060/4690 [9:40:34<33:01:38, 32.75s/it]
                                                        
{'loss': '0.2497', 'grad_norm': '2.347', 'learning_rate': '9.028e-06', 'epoch': '1.13'}

 23%|██▎       | 1060/4690 [9:40:34<33:01:38, 32.75s/it]
 23%|██▎       | 1061/4690 [9:41:07<32:59:25, 32.73s/it]
 23%|██▎       | 1062/4690 [9:41:39<32:52:58, 32.63s/it]
 23%|██▎       | 1063/4690 [9:42:12<32:56:52, 32.70s/it]
 23%|██▎       | 1064/4690 [9:42:45<32:56:19, 32.70s/it]
 23%|██▎       | 1065/4690 [9:43:18<33:02:42, 32.82s/it]
 23%|██▎       | 1066/4690 [9:43:51<33:05:15, 32.87s/it]
 23%|██▎       | 1067/4690 [9:44:24<33:06:08, 32.89s/it]
 23%|██▎       | 1068/4690 [9:44:57<33:04:35, 32.88s/it]
 23%|██▎       | 1069/4690 [9:45:30<33:03:39, 32.87s/it]
 23%|██▎       | 1070/4690 [9:46:03<33:13:59, 33.05s/it]
                                                        
{'loss': '0.2375', 'grad_norm': '2.329', 'learning_rate': '9.008e-06', 'epoch': '1.141'}

 23%|██▎       | 1070/4690 [9:46:03<33:13:59, 33.05s/it]
 23%|██▎       | 1071/4690 [9:46:36<33:05:26, 32.92s/it]
 23%|██▎       | 1072/4690 [9:47:08<32:59:17, 32.82s/it]
 23%|██▎       | 1073/4690 [9:47:41<33:00:10, 32.85s/it]
 23%|██▎       | 1074/4690 [9:48:14<32:53:09, 32.74s/it]
 23%|██▎       | 1075/4690 [9:48:47<32:59:05, 32.85s/it]
 23%|██▎       | 1076/4690 [9:49:20<33:05:57, 32.97s/it]
 23%|██▎       | 1077/4690 [9:49:53<33:07:02, 33.00s/it]
 23%|██▎       | 1078/4690 [9:50:26<33:07:17, 33.01s/it]
 23%|██▎       | 1079/4690 [9:50:59<33:00:58, 32.92s/it]
 23%|██▎       | 1080/4690 [9:51:31<32:52:24, 32.78s/it]
                                                        
{'loss': '0.2472', 'grad_norm': '2.072', 'learning_rate': '8.987e-06', 'epoch': '1.151'}

 23%|██▎       | 1080/4690 [9:51:31<32:52:24, 32.78s/it]
 23%|██▎       | 1081/4690 [9:52:04<32:55:38, 32.85s/it]
 23%|██▎       | 1082/4690 [9:52:37<32:51:31, 32.79s/it]
 23%|██▎       | 1083/4690 [9:53:10<32:48:13, 32.74s/it]
 23%|██▎       | 1084/4690 [9:53:43<32:53:08, 32.83s/it]
 23%|██▎       | 1085/4690 [9:54:16<32:52:31, 32.83s/it]
 23%|██▎       | 1086/4690 [9:54:48<32:49:22, 32.79s/it]
 23%|██▎       | 1087/4690 [9:55:21<32:42:08, 32.68s/it]
 23%|██▎       | 1088/4690 [9:55:54<32:49:53, 32.81s/it]
 23%|██▎       | 1089/4690 [9:56:26<32:46:20, 32.76s/it]
 23%|██▎       | 1090/4690 [9:56:59<32:42:52, 32.71s/it]
                                                        
{'loss': '0.2498', 'grad_norm': '2.202', 'learning_rate': '8.966e-06', 'epoch': '1.162'}

 23%|██▎       | 1090/4690 [9:56:59<32:42:52, 32.71s/it]
 23%|██▎       | 1091/4690 [9:57:32<32:42:16, 32.71s/it]
 23%|██▎       | 1092/4690 [9:58:05<32:46:54, 32.80s/it]
 23%|██▎       | 1093/4690 [9:58:38<32:46:53, 32.81s/it]
 23%|██▎       | 1094/4690 [9:59:10<32:46:27, 32.81s/it]
 23%|██▎       | 1095/4690 [9:59:43<32:42:07, 32.75s/it]
 23%|██▎       | 1096/4690 [10:00:15<32:36:04, 32.66s/it]
 23%|██▎       | 1097/4690 [10:00:48<32:36:08, 32.67s/it]
 23%|██▎       | 1098/4690 [10:01:21<32:35:01, 32.66s/it]
 23%|██▎       | 1099/4690 [10:01:54<32:37:25, 32.71s/it]
 23%|██▎       | 1100/4690 [10:02:26<32:38:27, 32.73s/it]
                                                         
{'loss': '0.2457', 'grad_norm': '2.189', 'learning_rate': '8.945e-06', 'epoch': '1.173'}

 23%|██▎       | 1100/4690 [10:02:26<32:38:27, 32.73s/it]
 23%|██▎       | 1101/4690 [10:02:59<32:41:10, 32.79s/it]
 23%|██▎       | 1102/4690 [10:03:32<32:43:16, 32.83s/it]
 24%|██▎       | 1103/4690 [10:04:05<32:38:43, 32.76s/it]
 24%|██▎       | 1104/4690 [10:04:38<32:36:55, 32.74s/it]
 24%|██▎       | 1105/4690 [10:05:10<32:39:19, 32.79s/it]
 24%|██▎       | 1106/4690 [10:05:43<32:36:54, 32.76s/it]
 24%|██▎       | 1107/4690 [10:06:16<32:37:33, 32.78s/it]
 24%|██▎       | 1108/4690 [10:06:49<32:45:43, 32.93s/it]
 24%|██▎       | 1109/4690 [10:07:22<32:39:21, 32.83s/it]
 24%|██▎       | 1110/4690 [10:07:54<32:29:56, 32.68s/it]
                                                         
{'loss': '0.2587', 'grad_norm': '2.372', 'learning_rate': '8.924e-06', 'epoch': '1.183'}

 24%|██▎       | 1110/4690 [10:07:54<32:29:56, 32.68s/it]
 24%|██▎       | 1111/4690 [10:08:27<32:28:50, 32.67s/it]
 24%|██▎       | 1112/4690 [10:08:59<32:24:30, 32.61s/it]
 24%|██▎       | 1113/4690 [10:09:32<32:30:02, 32.71s/it]
 24%|██▍       | 1114/4690 [10:10:06<32:39:36, 32.88s/it]
 24%|██▍       | 1115/4690 [10:10:38<32:36:03, 32.83s/it]
 24%|██▍       | 1116/4690 [10:11:11<32:34:45, 32.82s/it]
 24%|██▍       | 1117/4690 [10:11:44<32:31:14, 32.77s/it]
 24%|██▍       | 1118/4690 [10:12:17<32:38:11, 32.89s/it]
 24%|██▍       | 1119/4690 [10:12:50<32:41:53, 32.96s/it]
 24%|██▍       | 1120/4690 [10:13:23<32:41:43, 32.97s/it]
                                                         
{'loss': '0.2605', 'grad_norm': '2.083', 'learning_rate': '8.902e-06', 'epoch': '1.194'}

 24%|██▍       | 1120/4690 [10:13:23<32:41:43, 32.97s/it]
 24%|██▍       | 1121/4690 [10:13:56<32:40:48, 32.96s/it]
 24%|██▍       | 1122/4690 [10:14:29<32:37:20, 32.91s/it]
 24%|██▍       | 1123/4690 [10:15:02<32:40:37, 32.98s/it]
 24%|██▍       | 1124/4690 [10:15:35<32:46:22, 33.09s/it]
 24%|██▍       | 1125/4690 [10:16:08<32:38:59, 32.97s/it]
 24%|██▍       | 1126/4690 [10:16:40<32:29:39, 32.82s/it]
 24%|██▍       | 1127/4690 [10:17:13<32:30:06, 32.84s/it]
 24%|██▍       | 1128/4690 [10:17:46<32:27:23, 32.80s/it]
 24%|██▍       | 1129/4690 [10:18:19<32:27:44, 32.82s/it]
 24%|██▍       | 1130/4690 [10:18:52<32:25:50, 32.80s/it]
                                                         
{'loss': '0.2488', 'grad_norm': '2.207', 'learning_rate': '8.881e-06', 'epoch': '1.205'}

 24%|██▍       | 1130/4690 [10:18:52<32:25:50, 32.80s/it]
 24%|██▍       | 1131/4690 [10:19:24<32:21:21, 32.73s/it]
 24%|██▍       | 1132/4690 [10:19:57<32:20:12, 32.72s/it]
 24%|██▍       | 1133/4690 [10:20:30<32:27:13, 32.85s/it]
 24%|██▍       | 1134/4690 [10:21:03<32:26:56, 32.85s/it]
 24%|██▍       | 1135/4690 [10:21:36<32:29:02, 32.90s/it]
 24%|██▍       | 1136/4690 [10:22:09<32:25:26, 32.84s/it]
 24%|██▍       | 1137/4690 [10:22:41<32:21:05, 32.78s/it]
 24%|██▍       | 1138/4690 [10:23:14<32:20:27, 32.78s/it]
 24%|██▍       | 1139/4690 [10:23:47<32:17:30, 32.74s/it]
 24%|██▍       | 1140/4690 [10:24:19<32:16:49, 32.73s/it]
                                                         
{'loss': '0.2448', 'grad_norm': '2.258', 'learning_rate': '8.859e-06', 'epoch': '1.215'}

 24%|██▍       | 1140/4690 [10:24:19<32:16:49, 32.73s/it]
 24%|██▍       | 1141/4690 [10:24:52<32:22:47, 32.85s/it]
 24%|██▍       | 1142/4690 [10:25:26<32:27:58, 32.94s/it]
 24%|██▍       | 1143/4690 [10:25:58<32:24:57, 32.90s/it]
 24%|██▍       | 1144/4690 [10:26:31<32:23:22, 32.88s/it]
 24%|██▍       | 1145/4690 [10:27:04<32:20:55, 32.85s/it]
 24%|██▍       | 1146/4690 [10:27:37<32:25:52, 32.94s/it]
 24%|██▍       | 1147/4690 [10:28:10<32:28:06, 32.99s/it]
 24%|██▍       | 1148/4690 [10:28:44<32:34:46, 33.11s/it]
 24%|██▍       | 1149/4690 [10:29:17<32:34:13, 33.11s/it]
 25%|██▍       | 1150/4690 [10:29:50<32:26:50, 33.00s/it]
                                                         
{'loss': '0.2489', 'grad_norm': '2.105', 'learning_rate': '8.837e-06', 'epoch': '1.226'}

 25%|██▍       | 1150/4690 [10:29:50<32:26:50, 33.00s/it]
 25%|██▍       | 1151/4690 [10:30:22<32:16:39, 32.83s/it]
 25%|██▍       | 1152/4690 [10:30:55<32:14:29, 32.81s/it]
 25%|██▍       | 1153/4690 [10:31:27<32:13:06, 32.79s/it]
 25%|██▍       | 1154/4690 [10:32:00<32:15:09, 32.84s/it]
 25%|██▍       | 1155/4690 [10:32:33<32:17:36, 32.89s/it]
 25%|██▍       | 1156/4690 [10:33:06<32:10:54, 32.78s/it]
 25%|██▍       | 1157/4690 [10:33:39<32:10:59, 32.79s/it]
 25%|██▍       | 1158/4690 [10:34:12<32:11:50, 32.82s/it]
 25%|██▍       | 1159/4690 [10:34:44<32:05:01, 32.71s/it]
 25%|██▍       | 1160/4690 [10:35:17<32:04:20, 32.71s/it]
                                                         
{'loss': '0.245', 'grad_norm': '2.339', 'learning_rate': '8.814e-06', 'epoch': '1.237'}

 25%|██▍       | 1160/4690 [10:35:17<32:04:20, 32.71s/it]
 25%|██▍       | 1161/4690 [10:35:50<32:06:25, 32.75s/it]
 25%|██▍       | 1162/4690 [10:36:22<32:05:58, 32.75s/it]
 25%|██▍       | 1163/4690 [10:36:55<32:07:25, 32.79s/it]
 25%|██▍       | 1164/4690 [10:37:28<32:09:22, 32.83s/it]
 25%|██▍       | 1165/4690 [10:38:01<32:10:35, 32.86s/it]
 25%|██▍       | 1166/4690 [10:38:34<32:14:00, 32.93s/it]
 25%|██▍       | 1167/4690 [10:39:07<32:03:09, 32.75s/it]
 25%|██▍       | 1168/4690 [10:39:40<32:06:42, 32.82s/it]
 25%|██▍       | 1169/4690 [10:40:12<32:04:21, 32.79s/it]
 25%|██▍       | 1170/4690 [10:40:45<32:09:04, 32.88s/it]
                                                         
{'loss': '0.2431', 'grad_norm': '2.033', 'learning_rate': '8.792e-06', 'epoch': '1.247'}

 25%|██▍       | 1170/4690 [10:40:45<32:09:04, 32.88s/it]
 25%|██▍       | 1171/4690 [10:41:18<32:11:38, 32.94s/it]
 25%|██▍       | 1172/4690 [10:41:51<32:12:36, 32.96s/it]
 25%|██▌       | 1173/4690 [10:42:24<32:11:40, 32.95s/it]
 25%|██▌       | 1174/4690 [10:42:57<32:08:57, 32.92s/it]
 25%|██▌       | 1175/4690 [10:43:30<32:06:26, 32.88s/it]
 25%|██▌       | 1176/4690 [10:44:03<32:01:45, 32.81s/it]
 25%|██▌       | 1177/4690 [10:44:36<32:03:02, 32.84s/it]
 25%|██▌       | 1178/4690 [10:45:09<32:09:07, 32.96s/it]
 25%|██▌       | 1179/4690 [10:45:41<32:02:33, 32.85s/it]
 25%|██▌       | 1180/4690 [10:46:14<32:04:02, 32.89s/it]
                                                         
{'loss': '0.2564', 'grad_norm': '3.183', 'learning_rate': '8.769e-06', 'epoch': '1.258'}

 25%|██▌       | 1180/4690 [10:46:14<32:04:02, 32.89s/it]
 25%|██▌       | 1181/4690 [10:46:48<32:09:07, 32.99s/it]
 25%|██▌       | 1182/4690 [10:47:20<32:04:19, 32.91s/it]
 25%|██▌       | 1183/4690 [10:47:53<32:01:26, 32.87s/it]
 25%|██▌       | 1184/4690 [10:48:26<31:51:52, 32.72s/it]
 25%|██▌       | 1185/4690 [10:48:58<31:42:49, 32.57s/it]
 25%|██▌       | 1186/4690 [10:49:31<31:51:09, 32.73s/it]
 25%|██▌       | 1187/4690 [10:50:04<31:53:37, 32.78s/it]
 25%|██▌       | 1188/4690 [10:50:36<31:50:30, 32.73s/it]
 25%|██▌       | 1189/4690 [10:51:09<31:56:30, 32.85s/it]
 25%|██▌       | 1190/4690 [10:51:42<31:58:28, 32.89s/it]
                                                         
{'loss': '0.2509', 'grad_norm': '2.281', 'learning_rate': '8.747e-06', 'epoch': '1.269'}

 25%|██▌       | 1190/4690 [10:51:42<31:58:28, 32.89s/it]
 25%|██▌       | 1191/4690 [10:52:15<31:58:10, 32.89s/it]
 25%|██▌       | 1192/4690 [10:52:48<31:59:31, 32.93s/it]
 25%|██▌       | 1193/4690 [10:53:21<31:54:46, 32.85s/it]
 25%|██▌       | 1194/4690 [10:53:54<31:52:35, 32.82s/it]
 25%|██▌       | 1195/4690 [10:54:27<31:54:38, 32.87s/it]
 26%|██▌       | 1196/4690 [10:55:00<31:55:56, 32.90s/it]
 26%|██▌       | 1197/4690 [10:55:32<31:50:35, 32.82s/it]
 26%|██▌       | 1198/4690 [10:56:05<31:53:42, 32.88s/it]
 26%|██▌       | 1199/4690 [10:56:38<31:43:06, 32.71s/it]
 26%|██▌       | 1200/4690 [10:57:10<31:42:01, 32.70s/it]
                                                         
{'loss': '0.2533', 'grad_norm': '2.193', 'learning_rate': '8.724e-06', 'epoch': '1.279'}

 26%|██▌       | 1200/4690 [10:57:10<31:42:01, 32.70s/it]
 26%|██▌       | 1201/4690 [10:57:43<31:42:46, 32.72s/it]
 26%|██▌       | 1202/4690 [10:58:16<31:38:32, 32.66s/it]
 26%|██▌       | 1203/4690 [10:58:48<31:35:57, 32.62s/it]
 26%|██▌       | 1204/4690 [10:59:21<31:34:56, 32.62s/it]
 26%|██▌       | 1205/4690 [10:59:53<31:29:35, 32.53s/it]
 26%|██▌       | 1206/4690 [11:00:26<31:32:18, 32.59s/it]
 26%|██▌       | 1207/4690 [11:00:59<31:45:09, 32.82s/it]
 26%|██▌       | 1208/4690 [11:01:32<31:44:03, 32.81s/it]
 26%|██▌       | 1209/4690 [11:02:04<31:35:58, 32.68s/it]
 26%|██▌       | 1210/4690 [11:02:37<31:36:48, 32.70s/it]
                                                         
{'loss': '0.2539', 'grad_norm': '2.134', 'learning_rate': '8.701e-06', 'epoch': '1.29'}

 26%|██▌       | 1210/4690 [11:02:37<31:36:48, 32.70s/it]
 26%|██▌       | 1211/4690 [11:03:10<31:35:51, 32.70s/it]
 26%|██▌       | 1212/4690 [11:03:43<31:39:37, 32.77s/it]
 26%|██▌       | 1213/4690 [11:04:16<31:41:25, 32.81s/it]
 26%|██▌       | 1214/4690 [11:04:49<31:42:42, 32.84s/it]
 26%|██▌       | 1215/4690 [11:05:21<31:39:32, 32.80s/it]
 26%|██▌       | 1216/4690 [11:05:54<31:37:06, 32.77s/it]
 26%|██▌       | 1217/4690 [11:06:27<31:39:13, 32.81s/it]
 26%|██▌       | 1218/4690 [11:07:00<31:37:41, 32.79s/it]
 26%|██▌       | 1219/4690 [11:07:32<31:31:25, 32.70s/it]
 26%|██▌       | 1220/4690 [11:08:05<31:25:55, 32.61s/it]
                                                         
{'loss': '0.2512', 'grad_norm': '2.24', 'learning_rate': '8.677e-06', 'epoch': '1.301'}

 26%|██▌       | 1220/4690 [11:08:05<31:25:55, 32.61s/it]
 26%|██▌       | 1221/4690 [11:08:37<31:24:57, 32.60s/it]
 26%|██▌       | 1222/4690 [11:09:10<31:28:26, 32.67s/it]
 26%|██▌       | 1223/4690 [11:09:43<31:28:48, 32.69s/it]
 26%|██▌       | 1224/4690 [11:10:16<31:35:53, 32.82s/it]
 26%|██▌       | 1225/4690 [11:10:48<31:32:16, 32.77s/it]
 26%|██▌       | 1226/4690 [11:11:22<31:37:43, 32.87s/it]
 26%|██▌       | 1227/4690 [11:11:54<31:33:48, 32.81s/it]
 26%|██▌       | 1228/4690 [11:12:27<31:34:57, 32.84s/it]
 26%|██▌       | 1229/4690 [11:13:00<31:33:04, 32.82s/it]
 26%|██▌       | 1230/4690 [11:13:32<31:25:51, 32.70s/it]
                                                         
{'loss': '0.2486', 'grad_norm': '2.4', 'learning_rate': '8.654e-06', 'epoch': '1.311'}

 26%|██▌       | 1230/4690 [11:13:32<31:25:51, 32.70s/it]
 26%|██▌       | 1231/4690 [11:14:05<31:26:22, 32.72s/it]
 26%|██▋       | 1232/4690 [11:14:38<31:22:23, 32.66s/it]
 26%|██▋       | 1233/4690 [11:15:11<31:26:55, 32.75s/it]
 26%|██▋       | 1234/4690 [11:15:43<31:27:25, 32.77s/it]
 26%|██▋       | 1235/4690 [11:16:16<31:27:28, 32.78s/it]
 26%|██▋       | 1236/4690 [11:16:49<31:35:36, 32.93s/it]
 26%|██▋       | 1237/4690 [11:17:23<31:37:49, 32.98s/it]
 26%|██▋       | 1238/4690 [11:17:56<31:38:16, 32.99s/it]
 26%|██▋       | 1239/4690 [11:18:28<31:31:16, 32.88s/it]
 26%|██▋       | 1240/4690 [11:19:01<31:27:30, 32.83s/it]
                                                         
{'loss': '0.2503', 'grad_norm': '2.074', 'learning_rate': '8.63e-06', 'epoch': '1.322'}

 26%|██▋       | 1240/4690 [11:19:01<31:27:30, 32.83s/it]
 26%|██▋       | 1241/4690 [11:19:34<31:30:15, 32.88s/it]
 26%|██▋       | 1242/4690 [11:20:06<31:20:41, 32.73s/it]
 27%|██▋       | 1243/4690 [11:20:39<31:24:45, 32.81s/it]
 27%|██▋       | 1244/4690 [11:21:13<31:36:46, 33.03s/it]
 27%|██▋       | 1245/4690 [11:21:46<31:33:26, 32.98s/it]
 27%|██▋       | 1246/4690 [11:22:18<31:26:21, 32.86s/it]
 27%|██▋       | 1247/4690 [11:22:51<31:21:52, 32.79s/it]
 27%|██▋       | 1248/4690 [11:23:24<31:21:32, 32.80s/it]
 27%|██▋       | 1249/4690 [11:23:56<31:19:09, 32.77s/it]
 27%|██▋       | 1250/4690 [11:24:29<31:18:40, 32.77s/it]
                                                         
{'loss': '0.2553', 'grad_norm': '2.314', 'learning_rate': '8.606e-06', 'epoch': '1.333'}

 27%|██▋       | 1250/4690 [11:24:29<31:18:40, 32.77s/it]
 27%|██▋       | 1251/4690 [11:25:02<31:19:01, 32.78s/it]
 27%|██▋       | 1252/4690 [11:25:35<31:17:10, 32.76s/it]
 27%|██▋       | 1253/4690 [11:26:08<31:21:05, 32.84s/it]
 27%|██▋       | 1254/4690 [11:26:41<31:26:30, 32.94s/it]
 27%|██▋       | 1255/4690 [11:27:14<31:26:26, 32.95s/it]
 27%|██▋       | 1256/4690 [11:27:46<31:18:21, 32.82s/it]
 27%|██▋       | 1257/4690 [11:28:20<31:23:58, 32.93s/it]
 27%|██▋       | 1258/4690 [11:28:53<31:25:26, 32.96s/it]
 27%|██▋       | 1259/4690 [11:29:25<31:19:43, 32.87s/it]
 27%|██▋       | 1260/4690 [11:29:58<31:19:37, 32.88s/it]
                                                         
{'loss': '0.2509', 'grad_norm': '2.266', 'learning_rate': '8.582e-06', 'epoch': '1.343'}

 27%|██▋       | 1260/4690 [11:29:58<31:19:37, 32.88s/it]
 27%|██▋       | 1261/4690 [11:30:31<31:21:34, 32.92s/it]
 27%|██▋       | 1262/4690 [11:31:04<31:16:03, 32.84s/it]
 27%|██▋       | 1263/4690 [11:31:37<31:20:03, 32.92s/it]
 27%|██▋       | 1264/4690 [11:32:10<31:17:37, 32.88s/it]
 27%|██▋       | 1265/4690 [11:32:42<31:11:35, 32.79s/it]
 27%|██▋       | 1266/4690 [11:33:15<31:12:49, 32.82s/it]
 27%|██▋       | 1267/4690 [11:33:48<31:12:32, 32.82s/it]
 27%|██▋       | 1268/4690 [11:34:21<31:06:51, 32.73s/it]
 27%|██▋       | 1269/4690 [11:34:53<31:09:18, 32.79s/it]
 27%|██▋       | 1270/4690 [11:35:26<31:12:33, 32.85s/it]
                                                         
{'loss': '0.2447', 'grad_norm': '2.166', 'learning_rate': '8.558e-06', 'epoch': '1.354'}

 27%|██▋       | 1270/4690 [11:35:26<31:12:33, 32.85s/it]
 27%|██▋       | 1271/4690 [11:35:59<31:09:43, 32.81s/it]
 27%|██▋       | 1272/4690 [11:36:32<31:00:40, 32.66s/it]
 27%|██▋       | 1273/4690 [11:37:05<31:10:40, 32.85s/it]
 27%|██▋       | 1274/4690 [11:37:38<31:10:29, 32.85s/it]
 27%|██▋       | 1275/4690 [11:38:11<31:20:16, 33.04s/it]
 27%|██▋       | 1276/4690 [11:38:44<31:16:44, 32.98s/it]
 27%|██▋       | 1277/4690 [11:39:17<31:11:34, 32.90s/it]
 27%|██▋       | 1278/4690 [11:39:49<31:03:19, 32.77s/it]
 27%|██▋       | 1279/4690 [11:40:22<31:09:49, 32.89s/it]
 27%|██▋       | 1280/4690 [11:40:55<31:07:04, 32.85s/it]
                                                         
{'loss': '0.2478', 'grad_norm': '2.106', 'learning_rate': '8.534e-06', 'epoch': '1.365'}

 27%|██▋       | 1280/4690 [11:40:55<31:07:04, 32.85s/it]
 27%|██▋       | 1281/4690 [11:41:28<31:02:35, 32.78s/it]
 27%|██▋       | 1282/4690 [11:42:00<31:00:43, 32.76s/it]
 27%|██▋       | 1283/4690 [11:42:33<30:57:04, 32.70s/it]
 27%|██▋       | 1284/4690 [11:43:06<30:56:42, 32.71s/it]
 27%|██▋       | 1285/4690 [11:43:39<30:59:21, 32.76s/it]
 27%|██▋       | 1286/4690 [11:44:12<31:02:52, 32.84s/it]
 27%|██▋       | 1287/4690 [11:44:44<31:00:47, 32.81s/it]
 27%|██▋       | 1288/4690 [11:45:17<30:55:19, 32.72s/it]
 27%|██▋       | 1289/4690 [11:45:49<30:52:56, 32.69s/it]
 28%|██▊       | 1290/4690 [11:46:22<30:54:10, 32.72s/it]
                                                         
{'loss': '0.2532', 'grad_norm': '2.274', 'learning_rate': '8.509e-06', 'epoch': '1.375'}

 28%|██▊       | 1290/4690 [11:46:22<30:54:10, 32.72s/it]
 28%|██▊       | 1291/4690 [11:46:55<30:55:12, 32.75s/it]
 28%|██▊       | 1292/4690 [11:47:28<31:02:18, 32.88s/it]
 28%|██▊       | 1293/4690 [11:48:01<31:04:08, 32.93s/it]
 28%|██▊       | 1294/4690 [11:48:34<30:58:42, 32.84s/it]
 28%|██▊       | 1295/4690 [11:49:07<31:00:15, 32.88s/it]
 28%|██▊       | 1296/4690 [11:49:40<30:57:13, 32.83s/it]
 28%|██▊       | 1297/4690 [11:50:13<30:59:22, 32.88s/it]
 28%|██▊       | 1298/4690 [11:50:45<30:56:12, 32.83s/it]
 28%|██▊       | 1299/4690 [11:51:18<30:55:14, 32.83s/it]
 28%|██▊       | 1300/4690 [11:51:51<30:52:34, 32.79s/it]
                                                         
{'loss': '0.2516', 'grad_norm': '2.095', 'learning_rate': '8.485e-06', 'epoch': '1.386'}

 28%|██▊       | 1300/4690 [11:51:51<30:52:34, 32.79s/it]
 28%|██▊       | 1301/4690 [11:52:24<30:56:38, 32.87s/it]
 28%|██▊       | 1302/4690 [11:52:56<30:50:20, 32.77s/it]
 28%|██▊       | 1303/4690 [11:53:29<30:53:14, 32.83s/it]
 28%|██▊       | 1304/4690 [11:54:02<30:54:23, 32.86s/it]
 28%|██▊       | 1305/4690 [11:54:35<30:57:32, 32.93s/it]
 28%|██▊       | 1306/4690 [11:55:08<30:53:52, 32.87s/it]
 28%|██▊       | 1307/4690 [11:55:41<30:51:21, 32.84s/it]
 28%|██▊       | 1308/4690 [11:56:14<30:46:32, 32.76s/it]
 28%|██▊       | 1309/4690 [11:56:46<30:46:35, 32.77s/it]
 28%|██▊       | 1310/4690 [11:57:19<30:45:32, 32.76s/it]
                                                         
{'loss': '0.2516', 'grad_norm': '2.344', 'learning_rate': '8.46e-06', 'epoch': '1.397'}

 28%|██▊       | 1310/4690 [11:57:19<30:45:32, 32.76s/it]
 28%|██▊       | 1311/4690 [11:57:52<30:43:38, 32.74s/it]
 28%|██▊       | 1312/4690 [11:58:25<30:49:06, 32.84s/it]
 28%|██▊       | 1313/4690 [11:58:58<30:51:46, 32.90s/it]
 28%|██▊       | 1314/4690 [11:59:30<30:44:51, 32.79s/it]
 28%|██▊       | 1315/4690 [12:00:03<30:48:26, 32.86s/it]
 28%|██▊       | 1316/4690 [12:00:36<30:45:33, 32.82s/it]
 28%|██▊       | 1317/4690 [12:01:09<30:49:45, 32.90s/it]
 28%|██▊       | 1318/4690 [12:01:42<30:45:33, 32.84s/it]
 28%|██▊       | 1319/4690 [12:02:15<30:45:18, 32.84s/it]
 28%|██▊       | 1320/4690 [12:02:47<30:39:25, 32.75s/it]
                                                         
{'loss': '0.247', 'grad_norm': '2.132', 'learning_rate': '8.435e-06', 'epoch': '1.407'}

 28%|██▊       | 1320/4690 [12:02:47<30:39:25, 32.75s/it]
 28%|██▊       | 1321/4690 [12:03:20<30:37:58, 32.73s/it]
 28%|██▊       | 1322/4690 [12:03:53<30:34:09, 32.68s/it]
 28%|██▊       | 1323/4690 [12:04:26<30:40:39, 32.80s/it]
 28%|██▊       | 1324/4690 [12:04:59<30:45:19, 32.89s/it]
 28%|██▊       | 1325/4690 [12:05:32<30:45:09, 32.90s/it]
 28%|██▊       | 1326/4690 [12:06:04<30:42:58, 32.87s/it]
 28%|██▊       | 1327/4690 [12:06:37<30:39:21, 32.82s/it]
 28%|██▊       | 1328/4690 [12:07:10<30:41:27, 32.86s/it]
 28%|██▊       | 1329/4690 [12:07:43<30:41:37, 32.88s/it]
 28%|██▊       | 1330/4690 [12:08:16<30:41:22, 32.88s/it]
                                                         
{'loss': '0.2557', 'grad_norm': '2.085', 'learning_rate': '8.409e-06', 'epoch': '1.418'}

 28%|██▊       | 1330/4690 [12:08:16<30:41:22, 32.88s/it]
 28%|██▊       | 1331/4690 [12:08:49<30:43:56, 32.94s/it]
 28%|██▊       | 1332/4690 [12:09:22<30:37:30, 32.83s/it]
 28%|██▊       | 1333/4690 [12:09:55<30:44:31, 32.97s/it]
 28%|██▊       | 1334/4690 [12:10:27<30:34:18, 32.79s/it]
 28%|██▊       | 1335/4690 [12:11:01<30:42:49, 32.96s/it]
 28%|██▊       | 1336/4690 [12:11:33<30:41:18, 32.94s/it]
 29%|██▊       | 1337/4690 [12:12:06<30:38:53, 32.91s/it]
 29%|██▊       | 1338/4690 [12:12:39<30:32:42, 32.81s/it]
 29%|██▊       | 1339/4690 [12:13:11<30:27:15, 32.72s/it]
 29%|██▊       | 1340/4690 [12:13:44<30:32:47, 32.83s/it]
                                                         
{'loss': '0.2548', 'grad_norm': '2.191', 'learning_rate': '8.384e-06', 'epoch': '1.429'}

 29%|██▊       | 1340/4690 [12:13:44<30:32:47, 32.83s/it]
 29%|██▊       | 1341/4690 [12:14:17<30:30:15, 32.79s/it]
 29%|██▊       | 1342/4690 [12:14:50<30:33:04, 32.85s/it]
 29%|██▊       | 1343/4690 [12:15:23<30:31:45, 32.84s/it]
 29%|██▊       | 1344/4690 [12:15:56<30:31:07, 32.84s/it]
 29%|██▊       | 1345/4690 [12:16:29<30:29:22, 32.81s/it]
 29%|██▊       | 1346/4690 [12:17:01<30:20:18, 32.66s/it]
 29%|██▊       | 1347/4690 [12:17:33<30:16:51, 32.61s/it]
 29%|██▊       | 1348/4690 [12:18:06<30:17:58, 32.64s/it]
 29%|██▉       | 1349/4690 [12:18:39<30:19:44, 32.68s/it]
 29%|██▉       | 1350/4690 [12:19:12<30:22:10, 32.73s/it]
                                                         
{'loss': '0.2456', 'grad_norm': '2.158', 'learning_rate': '8.359e-06', 'epoch': '1.439'}

 29%|██▉       | 1350/4690 [12:19:12<30:22:10, 32.73s/it]
 29%|██▉       | 1351/4690 [12:19:45<30:23:52, 32.77s/it]
 29%|██▉       | 1352/4690 [12:20:18<30:27:27, 32.85s/it]
 29%|██▉       | 1353/4690 [12:20:50<30:20:56, 32.74s/it]
 29%|██▉       | 1354/4690 [12:21:23<30:19:26, 32.72s/it]
 29%|██▉       | 1355/4690 [12:21:56<30:22:19, 32.79s/it]
 29%|██▉       | 1356/4690 [12:22:28<30:17:41, 32.71s/it]
 29%|██▉       | 1357/4690 [12:23:01<30:16:57, 32.71s/it]
 29%|██▉       | 1358/4690 [12:23:34<30:22:24, 32.82s/it]
 29%|██▉       | 1359/4690 [12:24:07<30:16:28, 32.72s/it]
 29%|██▉       | 1360/4690 [12:24:40<30:20:28, 32.80s/it]
                                                         
{'loss': '0.2538', 'grad_norm': '2.244', 'learning_rate': '8.333e-06', 'epoch': '1.45'}

 29%|██▉       | 1360/4690 [12:24:40<30:20:28, 32.80s/it]
 29%|██▉       | 1361/4690 [12:25:12<30:16:58, 32.75s/it]
 29%|██▉       | 1362/4690 [12:25:45<30:14:18, 32.71s/it]
 29%|██▉       | 1363/4690 [12:26:18<30:21:15, 32.85s/it]
 29%|██▉       | 1364/4690 [12:26:50<30:14:58, 32.74s/it]
 29%|██▉       | 1365/4690 [12:27:23<30:12:21, 32.70s/it]
 29%|██▉       | 1366/4690 [12:27:56<30:14:43, 32.76s/it]
 29%|██▉       | 1367/4690 [12:28:29<30:17:16, 32.81s/it]
 29%|██▉       | 1368/4690 [12:29:02<30:15:44, 32.79s/it]
 29%|██▉       | 1369/4690 [12:29:34<30:12:30, 32.75s/it]
 29%|██▉       | 1370/4690 [12:30:07<30:19:08, 32.88s/it]
                                                         
{'loss': '0.2458', 'grad_norm': '2.008', 'learning_rate': '8.307e-06', 'epoch': '1.461'}

 29%|██▉       | 1370/4690 [12:30:07<30:19:08, 32.88s/it]
 29%|██▉       | 1371/4690 [12:30:40<30:10:05, 32.72s/it]
 29%|██▉       | 1372/4690 [12:31:13<30:15:56, 32.84s/it]
 29%|██▉       | 1373/4690 [12:31:45<30:04:39, 32.64s/it]
 29%|██▉       | 1374/4690 [12:32:18<30:08:41, 32.73s/it]
 29%|██▉       | 1375/4690 [12:32:51<30:05:10, 32.67s/it]
 29%|██▉       | 1376/4690 [12:33:23<30:05:37, 32.69s/it]
 29%|██▉       | 1377/4690 [12:33:56<30:03:41, 32.67s/it]
 29%|██▉       | 1378/4690 [12:34:29<30:10:50, 32.81s/it]
 29%|██▉       | 1379/4690 [12:35:02<30:08:14, 32.77s/it]
 29%|██▉       | 1380/4690 [12:35:35<30:11:31, 32.84s/it]
                                                         
{'loss': '0.2524', 'grad_norm': '2.066', 'learning_rate': '8.281e-06', 'epoch': '1.471'}

 29%|██▉       | 1380/4690 [12:35:35<30:11:31, 32.84s/it]
 29%|██▉       | 1381/4690 [12:36:07<30:04:02, 32.71s/it]
 29%|██▉       | 1382/4690 [12:36:40<30:08:03, 32.79s/it]
 29%|██▉       | 1383/4690 [12:37:13<30:12:26, 32.88s/it]
 30%|██▉       | 1384/4690 [12:37:46<30:15:02, 32.94s/it]
 30%|██▉       | 1385/4690 [12:38:19<30:12:13, 32.90s/it]
 30%|██▉       | 1386/4690 [12:38:52<30:15:13, 32.96s/it]
 30%|██▉       | 1387/4690 [12:39:25<30:09:53, 32.88s/it]
 30%|██▉       | 1388/4690 [12:39:58<30:13:50, 32.96s/it]
 30%|██▉       | 1389/4690 [12:40:31<30:05:44, 32.82s/it]
 30%|██▉       | 1390/4690 [12:41:03<30:07:46, 32.87s/it]
                                                         
{'loss': '0.2531', 'grad_norm': '2.206', 'learning_rate': '8.255e-06', 'epoch': '1.482'}

 30%|██▉       | 1390/4690 [12:41:03<30:07:46, 32.87s/it]
 30%|██▉       | 1391/4690 [12:41:37<30:11:03, 32.94s/it]
 30%|██▉       | 1392/4690 [12:42:10<30:13:52, 33.00s/it]
 30%|██▉       | 1393/4690 [12:42:43<30:18:07, 33.09s/it]
 30%|██▉       | 1394/4690 [12:43:16<30:22:22, 33.17s/it]
 30%|██▉       | 1395/4690 [12:43:49<30:11:49, 32.99s/it]
 30%|██▉       | 1396/4690 [12:44:22<30:05:18, 32.88s/it]
 30%|██▉       | 1397/4690 [12:44:54<30:02:48, 32.85s/it]
 30%|██▉       | 1398/4690 [12:45:27<30:02:41, 32.86s/it]
 30%|██▉       | 1399/4690 [12:46:00<29:59:40, 32.81s/it]
 30%|██▉       | 1400/4690 [12:46:33<29:56:25, 32.76s/it]
                                                         
{'loss': '0.2567', 'grad_norm': '2.182', 'learning_rate': '8.229e-06', 'epoch': '1.493'}

 30%|██▉       | 1400/4690 [12:46:33<29:56:25, 32.76s/it]
 30%|██▉       | 1401/4690 [12:47:06<30:01:46, 32.87s/it]
 30%|██▉       | 1402/4690 [12:47:39<29:59:57, 32.85s/it]
 30%|██▉       | 1403/4690 [12:48:11<29:58:32, 32.83s/it]
 30%|██▉       | 1404/4690 [12:48:44<29:56:53, 32.81s/it]
 30%|██▉       | 1405/4690 [12:49:17<29:50:45, 32.71s/it]
 30%|██▉       | 1406/4690 [12:49:49<29:53:20, 32.76s/it]
 30%|███       | 1407/4690 [12:50:22<29:53:56, 32.79s/it]
 30%|███       | 1408/4690 [12:50:55<29:52:41, 32.77s/it]
 30%|███       | 1409/4690 [12:51:28<29:56:33, 32.85s/it]
 30%|███       | 1410/4690 [12:52:01<29:54:25, 32.82s/it]
                                                         
{'loss': '0.2528', 'grad_norm': '2.4', 'learning_rate': '8.202e-06', 'epoch': '1.503'}

 30%|███       | 1410/4690 [12:52:01<29:54:25, 32.82s/it]
 30%|███       | 1411/4690 [12:52:34<29:55:33, 32.86s/it]
 30%|███       | 1412/4690 [12:53:07<29:56:15, 32.88s/it]
 30%|███       | 1413/4690 [12:53:39<29:51:23, 32.80s/it]
 30%|███       | 1414/4690 [12:54:12<29:57:09, 32.91s/it]
 30%|███       | 1415/4690 [12:54:46<30:05:57, 33.09s/it]
 30%|███       | 1416/4690 [12:55:19<30:02:46, 33.04s/it]
 30%|███       | 1417/4690 [12:55:52<29:57:42, 32.96s/it]
 30%|███       | 1418/4690 [12:56:25<29:57:01, 32.95s/it]
 30%|███       | 1419/4690 [12:56:58<29:58:39, 32.99s/it]
 30%|███       | 1420/4690 [12:57:31<29:58:34, 33.00s/it]
                                                         
{'loss': '0.2554', 'grad_norm': '2.179', 'learning_rate': '8.176e-06', 'epoch': '1.514'}

 30%|███       | 1420/4690 [12:57:31<29:58:34, 33.00s/it]
 30%|███       | 1421/4690 [12:58:04<29:57:47, 33.00s/it]
 30%|███       | 1422/4690 [12:58:37<29:56:09, 32.98s/it]
 30%|███       | 1423/4690 [12:59:10<29:54:26, 32.96s/it]
 30%|███       | 1424/4690 [12:59:42<29:52:53, 32.94s/it]
 30%|███       | 1425/4690 [13:00:15<29:52:42, 32.94s/it]
 30%|███       | 1426/4690 [13:00:48<29:44:02, 32.80s/it]
 30%|███       | 1427/4690 [13:01:20<29:39:59, 32.73s/it]
 30%|███       | 1428/4690 [13:01:53<29:43:05, 32.80s/it]
 30%|███       | 1429/4690 [13:02:26<29:42:32, 32.80s/it]
 30%|███       | 1430/4690 [13:02:59<29:39:43, 32.76s/it]
                                                         
{'loss': '0.2615', 'grad_norm': '1.986', 'learning_rate': '8.149e-06', 'epoch': '1.525'}

 30%|███       | 1430/4690 [13:02:59<29:39:43, 32.76s/it]
 31%|███       | 1431/4690 [13:03:32<29:40:43, 32.78s/it]
 31%|███       | 1432/4690 [13:04:04<29:39:46, 32.78s/it]
 31%|███       | 1433/4690 [13:04:37<29:39:51, 32.79s/it]
 31%|███       | 1434/4690 [13:05:10<29:39:49, 32.80s/it]
 31%|███       | 1435/4690 [13:05:43<29:41:05, 32.83s/it]
 31%|███       | 1436/4690 [13:06:16<29:46:17, 32.94s/it]
 31%|███       | 1437/4690 [13:06:49<29:43:28, 32.90s/it]
 31%|███       | 1438/4690 [13:07:22<29:38:18, 32.81s/it]
 31%|███       | 1439/4690 [13:07:55<29:44:19, 32.93s/it]
 31%|███       | 1440/4690 [13:08:27<29:39:45, 32.86s/it]
                                                         
{'loss': '0.2604', 'grad_norm': '2.253', 'learning_rate': '8.122e-06', 'epoch': '1.535'}

 31%|███       | 1440/4690 [13:08:27<29:39:45, 32.86s/it]
 31%|███       | 1441/4690 [13:09:00<29:38:07, 32.84s/it]
 31%|███       | 1442/4690 [13:09:33<29:42:38, 32.93s/it]
 31%|███       | 1443/4690 [13:10:06<29:44:24, 32.97s/it]
 31%|███       | 1444/4690 [13:10:39<29:36:25, 32.84s/it]
 31%|███       | 1445/4690 [13:11:12<29:36:25, 32.85s/it]
 31%|███       | 1446/4690 [13:11:45<29:35:35, 32.84s/it]
 31%|███       | 1447/4690 [13:12:18<29:44:26, 33.01s/it]
 31%|███       | 1448/4690 [13:12:51<29:34:37, 32.84s/it]
 31%|███       | 1449/4690 [13:13:23<29:31:48, 32.80s/it]
 31%|███       | 1450/4690 [13:13:56<29:33:56, 32.85s/it]
                                                         
{'loss': '0.2556', 'grad_norm': '2.335', 'learning_rate': '8.095e-06', 'epoch': '1.546'}

 31%|███       | 1450/4690 [13:13:56<29:33:56, 32.85s/it]
 31%|███       | 1451/4690 [13:14:29<29:31:27, 32.81s/it]
 31%|███       | 1452/4690 [13:15:02<29:33:24, 32.86s/it]
 31%|███       | 1453/4690 [13:15:35<29:32:30, 32.85s/it]
 31%|███       | 1454/4690 [13:16:08<29:34:20, 32.90s/it]
 31%|███       | 1455/4690 [13:16:41<29:37:10, 32.96s/it]
 31%|███       | 1456/4690 [13:17:14<29:37:30, 32.98s/it]
 31%|███       | 1457/4690 [13:17:46<29:29:07, 32.83s/it]
 31%|███       | 1458/4690 [13:18:20<29:33:26, 32.92s/it]
 31%|███       | 1459/4690 [13:18:52<29:28:20, 32.84s/it]
 31%|███       | 1460/4690 [13:19:25<29:25:31, 32.80s/it]
                                                         
{'loss': '0.2591', 'grad_norm': '2.015', 'learning_rate': '8.068e-06', 'epoch': '1.557'}

 31%|███       | 1460/4690 [13:19:25<29:25:31, 32.80s/it]
 31%|███       | 1461/4690 [13:19:58<29:27:55, 32.85s/it]
 31%|███       | 1462/4690 [13:20:31<29:28:18, 32.87s/it]
 31%|███       | 1463/4690 [13:21:04<29:29:43, 32.90s/it]
 31%|███       | 1464/4690 [13:21:37<29:35:44, 33.03s/it]
 31%|███       | 1465/4690 [13:22:10<29:34:33, 33.02s/it]
 31%|███▏      | 1466/4690 [13:22:42<29:25:07, 32.85s/it]
 31%|███▏      | 1467/4690 [13:23:15<29:22:20, 32.81s/it]
 31%|███▏      | 1468/4690 [13:23:48<29:20:33, 32.78s/it]
 31%|███▏      | 1469/4690 [13:24:21<29:20:58, 32.80s/it]
 31%|███▏      | 1470/4690 [13:24:54<29:24:19, 32.88s/it]
                                                         
{'loss': '0.2582', 'grad_norm': '2.153', 'learning_rate': '8.04e-06', 'epoch': '1.567'}

 31%|███▏      | 1470/4690 [13:24:54<29:24:19, 32.88s/it]
 31%|███▏      | 1471/4690 [13:25:27<29:22:29, 32.85s/it]
 31%|███▏      | 1472/4690 [13:26:00<29:25:10, 32.91s/it]
 31%|███▏      | 1473/4690 [13:26:33<29:24:15, 32.90s/it]
 31%|███▏      | 1474/4690 [13:27:05<29:22:48, 32.89s/it]
 31%|███▏      | 1475/4690 [13:27:38<29:21:18, 32.87s/it]
 31%|███▏      | 1476/4690 [13:28:11<29:16:02, 32.78s/it]
 31%|███▏      | 1477/4690 [13:28:43<29:13:40, 32.75s/it]
 32%|███▏      | 1478/4690 [13:29:16<29:12:59, 32.75s/it]
 32%|███▏      | 1479/4690 [13:29:50<29:23:36, 32.95s/it]
 32%|███▏      | 1480/4690 [13:30:22<29:20:04, 32.90s/it]
                                                         
{'loss': '0.2611', 'grad_norm': '2.167', 'learning_rate': '8.013e-06', 'epoch': '1.578'}

 32%|███▏      | 1480/4690 [13:30:22<29:20:04, 32.90s/it]
 32%|███▏      | 1481/4690 [13:30:55<29:19:53, 32.91s/it]
 32%|███▏      | 1482/4690 [13:31:28<29:21:31, 32.95s/it]
 32%|███▏      | 1483/4690 [13:32:01<29:22:08, 32.97s/it]
 32%|███▏      | 1484/4690 [13:32:34<29:18:40, 32.91s/it]
 32%|███▏      | 1485/4690 [13:33:07<29:21:41, 32.98s/it]
 32%|███▏      | 1486/4690 [13:33:40<29:22:02, 33.00s/it]
 32%|███▏      | 1487/4690 [13:34:13<29:18:15, 32.94s/it]
 32%|███▏      | 1488/4690 [13:34:46<29:09:28, 32.78s/it]
 32%|███▏      | 1489/4690 [13:35:18<29:05:47, 32.72s/it]
 32%|███▏      | 1490/4690 [13:35:51<29:04:44, 32.71s/it]
                                                         
{'loss': '0.2534', 'grad_norm': '2.337', 'learning_rate': '7.985e-06', 'epoch': '1.589'}

 32%|███▏      | 1490/4690 [13:35:51<29:04:44, 32.71s/it]
 32%|███▏      | 1491/4690 [13:36:24<29:04:18, 32.72s/it]
 32%|███▏      | 1492/4690 [13:36:56<29:04:10, 32.72s/it]
 32%|███▏      | 1493/4690 [13:37:30<29:14:23, 32.93s/it]
 32%|███▏      | 1494/4690 [13:38:02<29:10:20, 32.86s/it]
 32%|███▏      | 1495/4690 [13:38:36<29:13:28, 32.93s/it]
 32%|███▏      | 1496/4690 [13:39:09<29:14:49, 32.96s/it]
 32%|███▏      | 1497/4690 [13:39:42<29:18:18, 33.04s/it]
 32%|███▏      | 1498/4690 [13:40:14<29:11:35, 32.92s/it]
 32%|███▏      | 1499/4690 [13:40:47<29:03:57, 32.79s/it]
 32%|███▏      | 1500/4690 [13:41:20<29:06:42, 32.85s/it]
                                                         
{'loss': '0.2609', 'grad_norm': '2.146', 'learning_rate': '7.958e-06', 'epoch': '1.599'}

 32%|███▏      | 1500/4690 [13:41:20<29:06:42, 32.85s/it]
 32%|███▏      | 1501/4690 [13:41:53<29:05:28, 32.84s/it]
 32%|███▏      | 1502/4690 [13:42:26<29:08:20, 32.90s/it]
 32%|███▏      | 1503/4690 [13:42:58<29:03:00, 32.81s/it]
 32%|███▏      | 1504/4690 [13:43:31<28:58:31, 32.74s/it]
 32%|███▏      | 1505/4690 [13:44:04<28:55:33, 32.69s/it]
 32%|███▏      | 1506/4690 [13:44:37<29:01:36, 32.82s/it]
 32%|███▏      | 1507/4690 [13:45:10<29:05:25, 32.90s/it]
 32%|███▏      | 1508/4690 [13:45:43<29:03:39, 32.88s/it]
 32%|███▏      | 1509/4690 [13:46:15<29:03:44, 32.89s/it]
 32%|███▏      | 1510/4690 [13:46:48<29:04:32, 32.92s/it]
                                                         
{'loss': '0.2538', 'grad_norm': '2.449', 'learning_rate': '7.93e-06', 'epoch': '1.61'}

 32%|███▏      | 1510/4690 [13:46:48<29:04:32, 32.92s/it]
 32%|███▏      | 1511/4690 [13:47:22<29:06:19, 32.96s/it]
 32%|███▏      | 1512/4690 [13:47:55<29:11:03, 33.06s/it]
 32%|███▏      | 1513/4690 [13:48:28<29:06:17, 32.98s/it]
 32%|███▏      | 1514/4690 [13:49:00<28:59:03, 32.85s/it]
 32%|███▏      | 1515/4690 [13:49:33<28:54:06, 32.77s/it]
 32%|███▏      | 1516/4690 [13:50:06<28:58:35, 32.87s/it]
 32%|███▏      | 1517/4690 [13:50:39<29:01:22, 32.93s/it]
 32%|███▏      | 1518/4690 [13:51:12<29:00:43, 32.93s/it]
 32%|███▏      | 1519/4690 [13:51:45<28:57:22, 32.87s/it]
 32%|███▏      | 1520/4690 [13:52:17<28:53:22, 32.81s/it]
                                                         
{'loss': '0.256', 'grad_norm': '2.023', 'learning_rate': '7.902e-06', 'epoch': '1.621'}

 32%|███▏      | 1520/4690 [13:52:17<28:53:22, 32.81s/it]
 32%|███▏      | 1521/4690 [13:52:50<28:56:38, 32.88s/it]
 32%|███▏      | 1522/4690 [13:53:23<28:56:22, 32.89s/it]
 32%|███▏      | 1523/4690 [13:53:56<28:57:15, 32.91s/it]
 32%|███▏      | 1524/4690 [13:54:29<28:54:08, 32.86s/it]
 33%|███▎      | 1525/4690 [13:55:02<29:01:18, 33.01s/it]
 33%|███▎      | 1526/4690 [13:55:35<28:55:14, 32.91s/it]
 33%|███▎      | 1527/4690 [13:56:08<28:51:30, 32.85s/it]
 33%|███▎      | 1528/4690 [13:56:40<28:48:28, 32.80s/it]
 33%|███▎      | 1529/4690 [13:57:13<28:45:30, 32.75s/it]
 33%|███▎      | 1530/4690 [13:57:46<28:56:52, 32.98s/it]
                                                         
{'loss': '0.2685', 'grad_norm': '2.312', 'learning_rate': '7.873e-06', 'epoch': '1.631'}

 33%|███▎      | 1530/4690 [13:57:46<28:56:52, 32.98s/it]
 33%|███▎      | 1531/4690 [13:58:19<28:48:38, 32.83s/it]
 33%|███▎      | 1532/4690 [13:58:52<28:47:13, 32.82s/it]
 33%|███▎      | 1533/4690 [13:59:25<28:49:01, 32.86s/it]
 33%|███▎      | 1534/4690 [13:59:58<28:51:09, 32.91s/it]
 33%|███▎      | 1535/4690 [14:00:30<28:44:29, 32.80s/it]
 33%|███▎      | 1536/4690 [14:01:03<28:41:24, 32.75s/it]
 33%|███▎      | 1537/4690 [14:01:36<28:39:56, 32.73s/it]
 33%|███▎      | 1538/4690 [14:02:08<28:35:48, 32.66s/it]
 33%|███▎      | 1539/4690 [14:02:41<28:42:28, 32.80s/it]
 33%|███▎      | 1540/4690 [14:03:14<28:42:29, 32.81s/it]
                                                         
{'loss': '0.2538', 'grad_norm': '2.351', 'learning_rate': '7.845e-06', 'epoch': '1.642'}

 33%|███▎      | 1540/4690 [14:03:14<28:42:29, 32.81s/it]
 33%|███▎      | 1541/4690 [14:03:47<28:44:03, 32.85s/it]
 33%|███▎      | 1542/4690 [14:04:19<28:38:02, 32.75s/it]
 33%|███▎      | 1543/4690 [14:04:52<28:41:05, 32.81s/it]
 33%|███▎      | 1544/4690 [14:05:25<28:42:40, 32.85s/it]
 33%|███▎      | 1545/4690 [14:05:58<28:38:14, 32.78s/it]
 33%|███▎      | 1546/4690 [14:06:31<28:34:12, 32.71s/it]
 33%|███▎      | 1547/4690 [14:07:03<28:36:51, 32.77s/it]
 33%|███▎      | 1548/4690 [14:07:36<28:36:29, 32.78s/it]
 33%|███▎      | 1549/4690 [14:08:09<28:38:30, 32.83s/it]
 33%|███▎      | 1550/4690 [14:08:42<28:33:04, 32.73s/it]
                                                         
{'loss': '0.2513', 'grad_norm': '2.232', 'learning_rate': '7.817e-06', 'epoch': '1.653'}

 33%|███▎      | 1550/4690 [14:08:42<28:33:04, 32.73s/it]
 33%|███▎      | 1551/4690 [14:09:15<28:41:02, 32.90s/it]
 33%|███▎      | 1552/4690 [14:09:48<28:34:30, 32.78s/it]
 33%|███▎      | 1553/4690 [14:10:20<28:33:51, 32.78s/it]
 33%|███▎      | 1554/4690 [14:10:53<28:32:22, 32.76s/it]
 33%|███▎      | 1555/4690 [14:11:26<28:36:32, 32.85s/it]
 33%|███▎      | 1556/4690 [14:12:00<28:48:29, 33.09s/it]
 33%|███▎      | 1557/4690 [14:12:33<28:46:45, 33.07s/it]
 33%|███▎      | 1558/4690 [14:13:06<28:45:17, 33.05s/it]
 33%|███▎      | 1559/4690 [14:13:39<28:41:35, 32.99s/it]
 33%|███▎      | 1560/4690 [14:14:11<28:38:45, 32.95s/it]
                                                         
{'loss': '0.2542', 'grad_norm': '2.328', 'learning_rate': '7.788e-06', 'epoch': '1.663'}

 33%|███▎      | 1560/4690 [14:14:11<28:38:45, 32.95s/it]
 33%|███▎      | 1561/4690 [14:14:44<28:38:25, 32.95s/it]
 33%|███▎      | 1562/4690 [14:15:18<28:42:35, 33.04s/it]
 33%|███▎      | 1563/4690 [14:15:51<28:39:21, 32.99s/it]
 33%|███▎      | 1564/4690 [14:16:24<28:43:07, 33.07s/it]
 33%|███▎      | 1565/4690 [14:16:57<28:36:59, 32.97s/it]
 33%|███▎      | 1566/4690 [14:17:30<28:40:25, 33.04s/it]
 33%|███▎      | 1567/4690 [14:18:03<28:37:50, 33.00s/it]
 33%|███▎      | 1568/4690 [14:18:35<28:30:45, 32.88s/it]
 33%|███▎      | 1569/4690 [14:19:08<28:30:14, 32.88s/it]
 33%|███▎      | 1570/4690 [14:19:41<28:27:28, 32.84s/it]
                                                         
{'loss': '0.2653', 'grad_norm': '2.106', 'learning_rate': '7.759e-06', 'epoch': '1.674'}

 33%|███▎      | 1570/4690 [14:19:41<28:27:28, 32.84s/it]
 33%|███▎      | 1571/4690 [14:20:14<28:23:53, 32.78s/it]
 34%|███▎      | 1572/4690 [14:20:46<28:21:04, 32.73s/it]
 34%|███▎      | 1573/4690 [14:21:19<28:18:29, 32.69s/it]
 34%|███▎      | 1574/4690 [14:21:52<28:20:45, 32.75s/it]
 34%|███▎      | 1575/4690 [14:22:25<28:22:48, 32.80s/it]
 34%|███▎      | 1576/4690 [14:22:57<28:17:45, 32.71s/it]
 34%|███▎      | 1577/4690 [14:23:30<28:19:57, 32.76s/it]
 34%|███▎      | 1578/4690 [14:24:03<28:21:34, 32.81s/it]
 34%|███▎      | 1579/4690 [14:24:36<28:21:07, 32.81s/it]
 34%|███▎      | 1580/4690 [14:25:09<28:23:05, 32.86s/it]
                                                         
{'loss': '0.2475', 'grad_norm': '2.091', 'learning_rate': '7.73e-06', 'epoch': '1.685'}

 34%|███▎      | 1580/4690 [14:25:09<28:23:05, 32.86s/it]
 34%|███▎      | 1581/4690 [14:25:41<28:20:56, 32.83s/it]
 34%|███▎      | 1582/4690 [14:26:14<28:18:05, 32.78s/it]
 34%|███▍      | 1583/4690 [14:26:47<28:18:15, 32.80s/it]
 34%|███▍      | 1584/4690 [14:27:20<28:18:15, 32.81s/it]
 34%|███▍      | 1585/4690 [14:27:53<28:19:33, 32.84s/it]
 34%|███▍      | 1586/4690 [14:28:25<28:18:44, 32.84s/it]
 34%|███▍      | 1587/4690 [14:28:59<28:21:36, 32.90s/it]
 34%|███▍      | 1588/4690 [14:29:31<28:15:25, 32.79s/it]
 34%|███▍      | 1589/4690 [14:30:04<28:22:48, 32.95s/it]
 34%|███▍      | 1590/4690 [14:30:37<28:16:02, 32.83s/it]
                                                         
{'loss': '0.267', 'grad_norm': '2.35', 'learning_rate': '7.701e-06', 'epoch': '1.695'}

 34%|███▍      | 1590/4690 [14:30:37<28:16:02, 32.83s/it]
 34%|███▍      | 1591/4690 [14:31:10<28:18:09, 32.88s/it]
 34%|███▍      | 1592/4690 [14:31:42<28:11:59, 32.77s/it]
 34%|███▍      | 1593/4690 [14:32:16<28:18:28, 32.91s/it]
 34%|███▍      | 1594/4690 [14:32:48<28:15:08, 32.85s/it]
 34%|███▍      | 1595/4690 [14:33:21<28:18:46, 32.93s/it]
 34%|███▍      | 1596/4690 [14:33:54<28:19:17, 32.95s/it]
 34%|███▍      | 1597/4690 [14:34:27<28:15:42, 32.89s/it]
 34%|███▍      | 1598/4690 [14:35:00<28:17:42, 32.94s/it]
 34%|███▍      | 1599/4690 [14:35:33<28:12:40, 32.86s/it]
 34%|███▍      | 1600/4690 [14:36:06<28:12:45, 32.87s/it]
                                                         
{'loss': '0.2564', 'grad_norm': '2.277', 'learning_rate': '7.672e-06', 'epoch': '1.706'}

 34%|███▍      | 1600/4690 [14:36:06<28:12:45, 32.87s/it]
 34%|███▍      | 1601/4690 [14:36:38<28:05:45, 32.74s/it]
 34%|███▍      | 1602/4690 [14:37:11<28:10:20, 32.84s/it]
 34%|███▍      | 1603/4690 [14:37:45<28:14:53, 32.94s/it]
 34%|███▍      | 1604/4690 [14:38:17<28:07:32, 32.81s/it]
 34%|███▍      | 1605/4690 [14:38:50<28:07:10, 32.81s/it]
 34%|███▍      | 1606/4690 [14:39:23<28:07:38, 32.83s/it]
 34%|███▍      | 1607/4690 [14:39:55<28:04:22, 32.78s/it]
 34%|███▍      | 1608/4690 [14:40:28<28:01:05, 32.73s/it]
 34%|███▍      | 1609/4690 [14:41:01<28:06:15, 32.84s/it]
 34%|███▍      | 1610/4690 [14:41:34<28:01:59, 32.77s/it]
                                                         
{'loss': '0.2491', 'grad_norm': '2.122', 'learning_rate': '7.643e-06', 'epoch': '1.717'}

 34%|███▍      | 1610/4690 [14:41:34<28:01:59, 32.77s/it]
 34%|███▍      | 1611/4690 [14:42:07<28:03:09, 32.80s/it]
 34%|███▍      | 1612/4690 [14:42:39<28:00:23, 32.76s/it]
 34%|███▍      | 1613/4690 [14:43:12<28:00:44, 32.77s/it]
 34%|███▍      | 1614/4690 [14:43:45<27:57:30, 32.72s/it]
 34%|███▍      | 1615/4690 [14:44:17<27:55:59, 32.70s/it]
 34%|███▍      | 1616/4690 [14:44:50<27:54:54, 32.69s/it]
 34%|███▍      | 1617/4690 [14:45:23<27:58:17, 32.77s/it]
 34%|███▍      | 1618/4690 [14:45:56<27:56:32, 32.74s/it]
 35%|███▍      | 1619/4690 [14:46:28<27:52:02, 32.67s/it]
 35%|███▍      | 1620/4690 [14:47:01<27:54:31, 32.73s/it]
                                                         
{'loss': '0.2473', 'grad_norm': '2.23', 'learning_rate': '7.614e-06', 'epoch': '1.727'}

 35%|███▍      | 1620/4690 [14:47:01<27:54:31, 32.73s/it]
 35%|███▍      | 1621/4690 [14:47:34<28:02:50, 32.90s/it]
 35%|███▍      | 1622/4690 [14:48:07<28:05:31, 32.96s/it]
 35%|███▍      | 1623/4690 [14:48:40<27:58:20, 32.83s/it]
 35%|███▍      | 1624/4690 [14:49:14<28:11:35, 33.10s/it]
 35%|███▍      | 1625/4690 [14:49:47<28:08:02, 33.04s/it]
 35%|███▍      | 1626/4690 [14:50:19<28:01:55, 32.94s/it]
 35%|███▍      | 1627/4690 [14:50:52<28:05:47, 33.02s/it]
 35%|███▍      | 1628/4690 [14:51:25<28:01:12, 32.94s/it]
 35%|███▍      | 1629/4690 [14:51:58<27:57:15, 32.88s/it]
 35%|███▍      | 1630/4690 [14:52:30<27:50:09, 32.75s/it]
                                                         
{'loss': '0.2562', 'grad_norm': '2.225', 'learning_rate': '7.584e-06', 'epoch': '1.738'}

 35%|███▍      | 1630/4690 [14:52:30<27:50:09, 32.75s/it]
 35%|███▍      | 1631/4690 [14:53:03<27:52:22, 32.80s/it]
 35%|███▍      | 1632/4690 [14:53:36<27:52:37, 32.82s/it]
 35%|███▍      | 1633/4690 [14:54:09<27:52:38, 32.83s/it]
 35%|███▍      | 1634/4690 [14:54:42<27:47:26, 32.74s/it]
 35%|███▍      | 1635/4690 [14:55:14<27:47:07, 32.74s/it]
 35%|███▍      | 1636/4690 [14:55:47<27:50:37, 32.82s/it]
 35%|███▍      | 1637/4690 [14:56:20<27:52:19, 32.87s/it]
 35%|███▍      | 1638/4690 [14:56:53<27:49:35, 32.82s/it]
 35%|███▍      | 1639/4690 [14:57:26<27:49:31, 32.83s/it]
 35%|███▍      | 1640/4690 [14:57:59<27:48:50, 32.83s/it]
                                                         
{'loss': '0.2535', 'grad_norm': '2.21', 'learning_rate': '7.555e-06', 'epoch': '1.749'}

 35%|███▍      | 1640/4690 [14:57:59<27:48:50, 32.83s/it]
 35%|███▍      | 1641/4690 [14:58:31<27:45:14, 32.77s/it]
 35%|███▌      | 1642/4690 [14:59:04<27:50:09, 32.88s/it]
 35%|███▌      | 1643/4690 [14:59:37<27:48:48, 32.86s/it]
 35%|███▌      | 1644/4690 [15:00:10<27:53:19, 32.96s/it]
 35%|███▌      | 1645/4690 [15:00:43<27:48:25, 32.88s/it]
 35%|███▌      | 1646/4690 [15:01:16<27:45:13, 32.82s/it]
 35%|███▌      | 1647/4690 [15:01:48<27:40:39, 32.74s/it]
 35%|███▌      | 1648/4690 [15:02:21<27:36:42, 32.68s/it]
 35%|███▌      | 1649/4690 [15:02:54<27:44:50, 32.85s/it]
 35%|███▌      | 1650/4690 [15:03:27<27:42:06, 32.80s/it]
                                                         
{'loss': '0.2549', 'grad_norm': '2.271', 'learning_rate': '7.525e-06', 'epoch': '1.759'}

 35%|███▌      | 1650/4690 [15:03:27<27:42:06, 32.80s/it]
 35%|███▌      | 1651/4690 [15:03:59<27:38:03, 32.74s/it]
 35%|███▌      | 1652/4690 [15:04:32<27:35:36, 32.70s/it]
 35%|███▌      | 1653/4690 [15:05:05<27:38:18, 32.76s/it]
 35%|███▌      | 1654/4690 [15:05:38<27:36:03, 32.73s/it]
 35%|███▌      | 1655/4690 [15:06:10<27:37:41, 32.77s/it]
 35%|███▌      | 1656/4690 [15:06:44<27:43:28, 32.90s/it]
 35%|███▌      | 1657/4690 [15:07:16<27:38:45, 32.81s/it]
 35%|███▌      | 1658/4690 [15:07:49<27:38:45, 32.83s/it]
 35%|███▌      | 1659/4690 [15:08:22<27:34:06, 32.74s/it]
 35%|███▌      | 1660/4690 [15:08:55<27:36:42, 32.81s/it]
                                                         
{'loss': '0.2451', 'grad_norm': '2.227', 'learning_rate': '7.495e-06', 'epoch': '1.77'}

 35%|███▌      | 1660/4690 [15:08:55<27:36:42, 32.81s/it]
 35%|███▌      | 1661/4690 [15:09:27<27:32:04, 32.72s/it]
 35%|███▌      | 1662/4690 [15:10:00<27:35:21, 32.80s/it]
 35%|███▌      | 1663/4690 [15:10:33<27:36:12, 32.83s/it]
 35%|███▌      | 1664/4690 [15:11:06<27:38:57, 32.89s/it]
 36%|███▌      | 1665/4690 [15:11:39<27:36:24, 32.85s/it]
 36%|███▌      | 1666/4690 [15:12:12<27:35:15, 32.84s/it]
 36%|███▌      | 1667/4690 [15:12:44<27:32:42, 32.80s/it]
 36%|███▌      | 1668/4690 [15:13:17<27:29:10, 32.74s/it]
 36%|███▌      | 1669/4690 [15:13:50<27:27:35, 32.72s/it]
 36%|███▌      | 1670/4690 [15:14:23<27:32:21, 32.83s/it]
                                                         
{'loss': '0.2522', 'grad_norm': '2.099', 'learning_rate': '7.465e-06', 'epoch': '1.781'}

 36%|███▌      | 1670/4690 [15:14:23<27:32:21, 32.83s/it]
 36%|███▌      | 1671/4690 [15:14:56<27:34:20, 32.88s/it]
 36%|███▌      | 1672/4690 [15:15:29<27:34:02, 32.88s/it]
 36%|███▌      | 1673/4690 [15:16:01<27:31:11, 32.84s/it]
 36%|███▌      | 1674/4690 [15:16:34<27:30:28, 32.83s/it]
 36%|███▌      | 1675/4690 [15:17:07<27:30:27, 32.84s/it]
 36%|███▌      | 1676/4690 [15:17:40<27:26:04, 32.77s/it]
 36%|███▌      | 1677/4690 [15:18:12<27:25:42, 32.77s/it]
 36%|███▌      | 1678/4690 [15:18:45<27:27:47, 32.82s/it]
 36%|███▌      | 1679/4690 [15:19:19<27:33:07, 32.94s/it]
 36%|███▌      | 1680/4690 [15:19:51<27:28:45, 32.87s/it]
                                                         
{'loss': '0.2602', 'grad_norm': '2.198', 'learning_rate': '7.435e-06', 'epoch': '1.791'}

 36%|███▌      | 1680/4690 [15:19:51<27:28:45, 32.87s/it]
 36%|███▌      | 1681/4690 [15:20:24<27:27:33, 32.85s/it]
 36%|███▌      | 1682/4690 [15:20:57<27:21:59, 32.75s/it]
 36%|███▌      | 1683/4690 [15:21:29<27:21:12, 32.75s/it]
 36%|███▌      | 1684/4690 [15:22:03<27:28:19, 32.90s/it]
 36%|███▌      | 1685/4690 [15:22:36<27:30:42, 32.96s/it]
 36%|███▌      | 1686/4690 [15:23:08<27:22:18, 32.80s/it]
 36%|███▌      | 1687/4690 [15:23:41<27:17:39, 32.72s/it]
 36%|███▌      | 1688/4690 [15:24:13<27:14:33, 32.67s/it]
 36%|███▌      | 1689/4690 [15:24:46<27:11:52, 32.63s/it]
 36%|███▌      | 1690/4690 [15:25:19<27:18:48, 32.78s/it]
                                                         
{'loss': '0.2631', 'grad_norm': '2.274', 'learning_rate': '7.405e-06', 'epoch': '1.802'}

 36%|███▌      | 1690/4690 [15:25:19<27:18:48, 32.78s/it]
 36%|███▌      | 1691/4690 [15:25:51<27:11:16, 32.64s/it]
 36%|███▌      | 1692/4690 [15:26:24<27:13:33, 32.69s/it]
 36%|███▌      | 1693/4690 [15:26:57<27:19:31, 32.82s/it]
 36%|███▌      | 1694/4690 [15:27:29<27:11:36, 32.68s/it]
 36%|███▌      | 1695/4690 [15:28:02<27:11:18, 32.68s/it]
 36%|███▌      | 1696/4690 [15:28:35<27:09:54, 32.66s/it]
 36%|███▌      | 1697/4690 [15:29:08<27:11:11, 32.70s/it]
 36%|███▌      | 1698/4690 [15:29:41<27:14:32, 32.78s/it]
 36%|███▌      | 1699/4690 [15:30:14<27:16:54, 32.84s/it]
 36%|███▌      | 1700/4690 [15:30:46<27:17:56, 32.87s/it]
                                                         
{'loss': '0.2546', 'grad_norm': '2.236', 'learning_rate': '7.374e-06', 'epoch': '1.813'}

 36%|███▌      | 1700/4690 [15:30:46<27:17:56, 32.87s/it]
 36%|███▋      | 1701/4690 [15:31:19<27:12:01, 32.76s/it]
 36%|███▋      | 1702/4690 [15:31:52<27:08:50, 32.71s/it]
 36%|███▋      | 1703/4690 [15:32:25<27:14:43, 32.84s/it]
 36%|███▋      | 1704/4690 [15:32:58<27:20:54, 32.97s/it]
 36%|███▋      | 1705/4690 [15:33:30<27:11:19, 32.79s/it]
 36%|███▋      | 1706/4690 [15:34:03<27:11:06, 32.80s/it]
 36%|███▋      | 1707/4690 [15:34:36<27:07:20, 32.73s/it]
 36%|███▋      | 1708/4690 [15:35:08<27:00:32, 32.61s/it]
 36%|███▋      | 1709/4690 [15:35:41<27:02:36, 32.66s/it]
 36%|███▋      | 1710/4690 [15:36:14<27:06:17, 32.74s/it]
                                                         
{'loss': '0.2503', 'grad_norm': '2.186', 'learning_rate': '7.344e-06', 'epoch': '1.823'}

 36%|███▋      | 1710/4690 [15:36:14<27:06:17, 32.74s/it]
 36%|███▋      | 1711/4690 [15:36:47<27:13:04, 32.89s/it]
 37%|███▋      | 1712/4690 [15:37:20<27:12:12, 32.89s/it]
 37%|███▋      | 1713/4690 [15:37:53<27:13:05, 32.91s/it]
 37%|███▋      | 1714/4690 [15:38:26<27:13:26, 32.93s/it]
 37%|███▋      | 1715/4690 [15:38:58<27:04:40, 32.77s/it]
 37%|███▋      | 1716/4690 [15:39:31<27:04:49, 32.78s/it]
 37%|███▋      | 1717/4690 [15:40:04<26:59:56, 32.69s/it]
 37%|███▋      | 1718/4690 [15:40:37<27:06:56, 32.85s/it]
 37%|███▋      | 1719/4690 [15:41:09<27:04:26, 32.81s/it]
 37%|███▋      | 1720/4690 [15:41:43<27:07:57, 32.89s/it]
                                                         
{'loss': '0.2529', 'grad_norm': '2.209', 'learning_rate': '7.313e-06', 'epoch': '1.834'}

 37%|███▋      | 1720/4690 [15:41:43<27:07:57, 32.89s/it]
 37%|███▋      | 1721/4690 [15:42:15<27:01:41, 32.77s/it]
 37%|███▋      | 1722/4690 [15:42:48<27:03:28, 32.82s/it]
 37%|███▋      | 1723/4690 [15:43:21<27:01:51, 32.80s/it]
 37%|███▋      | 1724/4690 [15:43:53<26:56:31, 32.70s/it]
 37%|███▋      | 1725/4690 [15:44:26<27:00:30, 32.79s/it]
 37%|███▋      | 1726/4690 [15:44:59<26:57:39, 32.75s/it]
 37%|███▋      | 1727/4690 [15:45:31<26:50:32, 32.61s/it]
 37%|███▋      | 1728/4690 [15:46:04<26:53:03, 32.68s/it]
 37%|███▋      | 1729/4690 [15:46:37<26:59:46, 32.82s/it]
 37%|███▋      | 1730/4690 [15:47:10<26:54:38, 32.73s/it]
                                                         
{'loss': '0.2525', 'grad_norm': '2.162', 'learning_rate': '7.283e-06', 'epoch': '1.845'}

 37%|███▋      | 1730/4690 [15:47:10<26:54:38, 32.73s/it]
 37%|███▋      | 1731/4690 [15:47:43<27:00:09, 32.85s/it]
 37%|███▋      | 1732/4690 [15:48:15<26:51:25, 32.69s/it]
 37%|███▋      | 1733/4690 [15:48:48<26:57:16, 32.82s/it]
 37%|███▋      | 1734/4690 [15:49:21<26:58:19, 32.85s/it]
 37%|███▋      | 1735/4690 [15:49:54<27:00:19, 32.90s/it]
 37%|███▋      | 1736/4690 [15:50:27<26:58:05, 32.87s/it]
 37%|███▋      | 1737/4690 [15:51:00<26:58:40, 32.89s/it]
 37%|███▋      | 1738/4690 [15:51:32<26:51:55, 32.76s/it]
 37%|███▋      | 1739/4690 [15:52:05<26:54:39, 32.83s/it]
 37%|███▋      | 1740/4690 [15:52:38<26:50:33, 32.76s/it]
                                                         
{'loss': '0.2531', 'grad_norm': '2.205', 'learning_rate': '7.252e-06', 'epoch': '1.855'}

 37%|███▋      | 1740/4690 [15:52:38<26:50:33, 32.76s/it]
 37%|███▋      | 1741/4690 [15:53:11<26:53:42, 32.83s/it]
 37%|███▋      | 1742/4690 [15:53:44<26:53:29, 32.84s/it]
 37%|███▋      | 1743/4690 [15:54:17<26:57:27, 32.93s/it]
 37%|███▋      | 1744/4690 [15:54:49<26:50:37, 32.80s/it]
 37%|███▋      | 1745/4690 [15:55:22<26:49:01, 32.78s/it]
 37%|███▋      | 1746/4690 [15:55:55<26:46:43, 32.75s/it]
 37%|███▋      | 1747/4690 [15:56:28<26:48:00, 32.78s/it]
 37%|███▋      | 1748/4690 [15:57:01<26:50:11, 32.84s/it]
 37%|███▋      | 1749/4690 [15:57:33<26:48:48, 32.82s/it]
 37%|███▋      | 1750/4690 [15:58:06<26:46:10, 32.78s/it]
                                                         
{'loss': '0.2533', 'grad_norm': '2.674', 'learning_rate': '7.221e-06', 'epoch': '1.866'}

 37%|███▋      | 1750/4690 [15:58:06<26:46:10, 32.78s/it]
 37%|███▋      | 1751/4690 [15:58:39<26:49:17, 32.85s/it]
 37%|███▋      | 1752/4690 [15:59:12<26:52:46, 32.94s/it]
 37%|███▋      | 1753/4690 [15:59:45<26:49:56, 32.89s/it]
 37%|███▋      | 1754/4690 [16:00:18<26:49:53, 32.90s/it]
 37%|███▋      | 1755/4690 [16:00:51<26:51:58, 32.95s/it]
 37%|███▋      | 1756/4690 [16:01:24<26:53:17, 32.99s/it]
 37%|███▋      | 1757/4690 [16:01:57<26:49:48, 32.93s/it]
 37%|███▋      | 1758/4690 [16:02:29<26:42:14, 32.79s/it]
 38%|███▊      | 1759/4690 [16:03:02<26:40:28, 32.76s/it]
 38%|███▊      | 1760/4690 [16:03:35<26:37:48, 32.72s/it]
                                                         
{'loss': '0.25', 'grad_norm': '2.225', 'learning_rate': '7.19e-06', 'epoch': '1.877'}

 38%|███▊      | 1760/4690 [16:03:35<26:37:48, 32.72s/it]
 38%|███▊      | 1761/4690 [16:04:08<26:44:36, 32.87s/it]
 38%|███▊      | 1762/4690 [16:04:41<26:44:14, 32.87s/it]
 38%|███▊      | 1763/4690 [16:05:14<26:49:07, 32.99s/it]
 38%|███▊      | 1764/4690 [16:05:47<26:47:40, 32.97s/it]
 38%|███▊      | 1765/4690 [16:06:20<26:47:57, 32.98s/it]
 38%|███▊      | 1766/4690 [16:06:53<26:45:26, 32.94s/it]
 38%|███▊      | 1767/4690 [16:07:25<26:37:07, 32.78s/it]
 38%|███▊      | 1768/4690 [16:07:58<26:34:27, 32.74s/it]
 38%|███▊      | 1769/4690 [16:08:31<26:33:12, 32.73s/it]
 38%|███▊      | 1770/4690 [16:09:04<26:42:44, 32.93s/it]
                                                         
{'loss': '0.2571', 'grad_norm': '2.2', 'learning_rate': '7.159e-06', 'epoch': '1.887'}

 38%|███▊      | 1770/4690 [16:09:04<26:42:44, 32.93s/it]
 38%|███▊      | 1771/4690 [16:09:36<26:35:31, 32.80s/it]
 38%|███▊      | 1772/4690 [16:10:09<26:36:15, 32.82s/it]
 38%|███▊      | 1773/4690 [16:10:42<26:34:55, 32.81s/it]
 38%|███▊      | 1774/4690 [16:11:15<26:29:09, 32.70s/it]
 38%|███▊      | 1775/4690 [16:11:47<26:30:06, 32.73s/it]
 38%|███▊      | 1776/4690 [16:12:20<26:24:45, 32.63s/it]
 38%|███▊      | 1777/4690 [16:12:53<26:27:30, 32.70s/it]
 38%|███▊      | 1778/4690 [16:13:25<26:27:35, 32.71s/it]
 38%|███▊      | 1779/4690 [16:13:58<26:23:31, 32.64s/it]
 38%|███▊      | 1780/4690 [16:14:30<26:20:22, 32.59s/it]
                                                         
{'loss': '0.2471', 'grad_norm': '2.66', 'learning_rate': '7.128e-06', 'epoch': '1.898'}

 38%|███▊      | 1780/4690 [16:14:30<26:20:22, 32.59s/it]
 38%|███▊      | 1781/4690 [16:15:03<26:26:02, 32.71s/it]
 38%|███▊      | 1782/4690 [16:15:36<26:23:45, 32.68s/it]
 38%|███▊      | 1783/4690 [16:16:09<26:26:34, 32.75s/it]
 38%|███▊      | 1784/4690 [16:16:42<26:26:51, 32.76s/it]
 38%|███▊      | 1785/4690 [16:17:14<26:25:22, 32.74s/it]
 38%|███▊      | 1786/4690 [16:17:47<26:27:14, 32.79s/it]
 38%|███▊      | 1787/4690 [16:18:20<26:22:46, 32.71s/it]
 38%|███▊      | 1788/4690 [16:18:52<26:15:27, 32.57s/it]
 38%|███▊      | 1789/4690 [16:19:25<26:22:11, 32.72s/it]
 38%|███▊      | 1790/4690 [16:19:58<26:26:00, 32.81s/it]
                                                         
{'loss': '0.2447', 'grad_norm': '2.098', 'learning_rate': '7.096e-06', 'epoch': '1.909'}

 38%|███▊      | 1790/4690 [16:19:58<26:26:00, 32.81s/it]
 38%|███▊      | 1791/4690 [16:20:31<26:21:10, 32.73s/it]
 38%|███▊      | 1792/4690 [16:21:04<26:24:06, 32.80s/it]
 38%|███▊      | 1793/4690 [16:21:36<26:24:34, 32.82s/it]
 38%|███▊      | 1794/4690 [16:22:09<26:25:00, 32.84s/it]
 38%|███▊      | 1795/4690 [16:22:42<26:24:46, 32.85s/it]
 38%|███▊      | 1796/4690 [16:23:15<26:24:31, 32.85s/it]
 38%|███▊      | 1797/4690 [16:23:48<26:21:08, 32.79s/it]
 38%|███▊      | 1798/4690 [16:24:20<26:18:45, 32.75s/it]
 38%|███▊      | 1799/4690 [16:24:53<26:16:26, 32.72s/it]
 38%|███▊      | 1800/4690 [16:25:26<26:18:52, 32.78s/it]
                                                         
{'loss': '0.2609', 'grad_norm': '2.047', 'learning_rate': '7.065e-06', 'epoch': '1.919'}

 38%|███▊      | 1800/4690 [16:25:26<26:18:52, 32.78s/it]
 38%|███▊      | 1801/4690 [16:25:59<26:18:09, 32.78s/it]
 38%|███▊      | 1802/4690 [16:26:32<26:22:51, 32.88s/it]
 38%|███▊      | 1803/4690 [16:27:05<26:22:13, 32.88s/it]
 38%|███▊      | 1804/4690 [16:27:37<26:19:26, 32.84s/it]
 38%|███▊      | 1805/4690 [16:28:10<26:12:29, 32.70s/it]
 39%|███▊      | 1806/4690 [16:28:42<26:07:48, 32.62s/it]
 39%|███▊      | 1807/4690 [16:29:15<26:10:15, 32.68s/it]
 39%|███▊      | 1808/4690 [16:29:48<26:07:23, 32.63s/it]
 39%|███▊      | 1809/4690 [16:30:20<26:04:07, 32.57s/it]
 39%|███▊      | 1810/4690 [16:30:53<26:11:34, 32.74s/it]
                                                         
{'loss': '0.2498', 'grad_norm': '2.157', 'learning_rate': '7.034e-06', 'epoch': '1.93'}

 39%|███▊      | 1810/4690 [16:30:53<26:11:34, 32.74s/it]
 39%|███▊      | 1811/4690 [16:31:26<26:12:25, 32.77s/it]
 39%|███▊      | 1812/4690 [16:31:59<26:13:46, 32.81s/it]
 39%|███▊      | 1813/4690 [16:32:32<26:12:03, 32.79s/it]
 39%|███▊      | 1814/4690 [16:33:04<26:12:36, 32.81s/it]
 39%|███▊      | 1815/4690 [16:33:37<26:13:36, 32.84s/it]
 39%|███▊      | 1816/4690 [16:34:10<26:07:10, 32.72s/it]
 39%|███▊      | 1817/4690 [16:34:43<26:08:01, 32.75s/it]
 39%|███▉      | 1818/4690 [16:35:15<26:08:05, 32.76s/it]
 39%|███▉      | 1819/4690 [16:35:48<26:09:06, 32.79s/it]
 39%|███▉      | 1820/4690 [16:36:21<26:03:03, 32.68s/it]
                                                         
{'loss': '0.2451', 'grad_norm': '2.106', 'learning_rate': '7.002e-06', 'epoch': '1.941'}

 39%|███▉      | 1820/4690 [16:36:21<26:03:03, 32.68s/it]
 39%|███▉      | 1821/4690 [16:36:54<26:08:51, 32.81s/it]
 39%|███▉      | 1822/4690 [16:37:27<26:13:53, 32.93s/it]
 39%|███▉      | 1823/4690 [16:38:00<26:10:41, 32.87s/it]
 39%|███▉      | 1824/4690 [16:38:32<26:07:45, 32.82s/it]
 39%|███▉      | 1825/4690 [16:39:05<26:09:20, 32.87s/it]
 39%|███▉      | 1826/4690 [16:39:38<26:10:08, 32.89s/it]
 39%|███▉      | 1827/4690 [16:40:11<26:10:35, 32.91s/it]
 39%|███▉      | 1828/4690 [16:40:44<26:10:26, 32.92s/it]
 39%|███▉      | 1829/4690 [16:41:17<26:02:28, 32.77s/it]
 39%|███▉      | 1830/4690 [16:41:50<26:02:58, 32.79s/it]
                                                         
{'loss': '0.256', 'grad_norm': '2.285', 'learning_rate': '6.97e-06', 'epoch': '1.951'}

 39%|███▉      | 1830/4690 [16:41:50<26:02:58, 32.79s/it]
 39%|███▉      | 1831/4690 [16:42:22<25:57:44, 32.69s/it]
 39%|███▉      | 1832/4690 [16:42:55<25:59:40, 32.74s/it]
 39%|███▉      | 1833/4690 [16:43:28<25:57:30, 32.71s/it]
 39%|███▉      | 1834/4690 [16:44:00<25:59:23, 32.76s/it]
 39%|███▉      | 1835/4690 [16:44:33<25:55:57, 32.70s/it]
 39%|███▉      | 1836/4690 [16:45:06<25:55:43, 32.71s/it]
 39%|███▉      | 1837/4690 [16:45:39<25:57:02, 32.75s/it]
 39%|███▉      | 1838/4690 [16:46:11<25:55:34, 32.73s/it]
 39%|███▉      | 1839/4690 [16:46:44<25:52:57, 32.68s/it]
 39%|███▉      | 1840/4690 [16:47:16<25:49:52, 32.63s/it]
                                                         
{'loss': '0.2484', 'grad_norm': '2.158', 'learning_rate': '6.939e-06', 'epoch': '1.962'}

 39%|███▉      | 1840/4690 [16:47:16<25:49:52, 32.63s/it]
 39%|███▉      | 1841/4690 [16:47:49<25:53:22, 32.71s/it]
 39%|███▉      | 1842/4690 [16:48:22<25:48:46, 32.63s/it]
 39%|███▉      | 1843/4690 [16:48:54<25:49:19, 32.65s/it]
 39%|███▉      | 1844/4690 [16:49:27<25:50:28, 32.69s/it]
 39%|███▉      | 1845/4690 [16:50:00<25:51:42, 32.73s/it]
 39%|███▉      | 1846/4690 [16:50:33<25:52:26, 32.75s/it]
 39%|███▉      | 1847/4690 [16:51:06<25:57:14, 32.86s/it]
 39%|███▉      | 1848/4690 [16:51:38<25:49:04, 32.70s/it]
 39%|███▉      | 1849/4690 [16:52:11<25:52:14, 32.78s/it]
 39%|███▉      | 1850/4690 [16:52:44<25:54:20, 32.84s/it]
                                                         
{'loss': '0.249', 'grad_norm': '2.059', 'learning_rate': '6.907e-06', 'epoch': '1.973'}

 39%|███▉      | 1850/4690 [16:52:44<25:54:20, 32.84s/it]
 39%|███▉      | 1851/4690 [16:53:17<25:55:59, 32.88s/it]
 39%|███▉      | 1852/4690 [16:53:50<25:51:39, 32.80s/it]
 40%|███▉      | 1853/4690 [16:54:22<25:47:00, 32.72s/it]
 40%|███▉      | 1854/4690 [16:54:55<25:47:54, 32.75s/it]
 40%|███▉      | 1855/4690 [16:55:28<25:48:17, 32.77s/it]
 40%|███▉      | 1856/4690 [16:56:00<25:44:14, 32.69s/it]
 40%|███▉      | 1857/4690 [16:56:33<25:45:32, 32.73s/it]
 40%|███▉      | 1858/4690 [16:57:06<25:45:25, 32.74s/it]
 40%|███▉      | 1859/4690 [16:57:39<25:47:33, 32.80s/it]
 40%|███▉      | 1860/4690 [16:58:12<25:44:54, 32.75s/it]
                                                         
{'loss': '0.2493', 'grad_norm': '2.105', 'learning_rate': '6.875e-06', 'epoch': '1.983'}

 40%|███▉      | 1860/4690 [16:58:12<25:44:54, 32.75s/it]
 40%|███▉      | 1861/4690 [16:58:44<25:39:30, 32.65s/it]
 40%|███▉      | 1862/4690 [16:59:17<25:45:15, 32.78s/it]
 40%|███▉      | 1863/4690 [16:59:50<25:42:50, 32.75s/it]
 40%|███▉      | 1864/4690 [17:00:22<25:42:25, 32.75s/it]
 40%|███▉      | 1865/4690 [17:00:56<25:53:25, 32.99s/it]
 40%|███▉      | 1866/4690 [17:01:29<25:54:42, 33.03s/it]
 40%|███▉      | 1867/4690 [17:02:02<25:47:45, 32.90s/it]
 40%|███▉      | 1868/4690 [17:02:34<25:44:04, 32.83s/it]
 40%|███▉      | 1869/4690 [17:03:07<25:40:11, 32.76s/it]
 40%|███▉      | 1870/4690 [17:03:40<25:39:05, 32.75s/it]
                                                         
{'loss': '0.2481', 'grad_norm': '2.188', 'learning_rate': '6.843e-06', 'epoch': '1.994'}

 40%|███▉      | 1870/4690 [17:03:40<25:39:05, 32.75s/it]
 40%|███▉      | 1871/4690 [17:04:13<25:39:52, 32.78s/it]
 40%|███▉      | 1872/4690 [17:04:45<25:36:44, 32.72s/it]
 40%|███▉      | 1873/4690 [17:05:18<25:37:48, 32.75s/it]
 40%|███▉      | 1874/4690 [17:05:51<25:34:34, 32.70s/it]
 40%|███▉      | 1875/4690 [17:06:24<25:41:39, 32.86s/it]
 40%|████      | 1876/4690 [17:06:41<21:54:12, 28.02s/it][INFO|trainer.py:3815] 2026-05-14 18:28:04,123 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876
[INFO|configuration_utils.py:533] 2026-05-14 18:28:04,140 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/config.json
[INFO|configuration_utils.py:816] 2026-05-14 18:28:04,142 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/generation_config.json


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:22<00:00, 22.69s/it]
Writing model shards: 100%|██████████| 1/1 [00:22<00:00, 22.69s/it]
[INFO|modeling_utils.py:3471] 2026-05-14 18:28:26,883 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-14 18:28:26,892 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-14 18:28:26,893 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/tokenizer_config.json
[INFO|tokenization_utils_base.py:3278] 2026-05-14 18:29:01,633 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-14 18:29:01,636 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-14 18:29:02,267 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-1876/processor_config.json

 40%|████      | 1877/4690 [17:08:21<38:48:00, 49.66s/it]
 40%|████      | 1878/4690 [17:08:53<34:47:10, 44.53s/it]
 40%|████      | 1879/4690 [17:09:26<32:00:38, 41.00s/it]
 40%|████      | 1880/4690 [17:09:59<30:01:41, 38.47s/it]
                                                         
{'loss': '0.1886', 'grad_norm': '1.578', 'learning_rate': '6.81e-06', 'epoch': '2.004'}

 40%|████      | 1880/4690 [17:09:59<30:01:41, 38.47s/it]
 40%|████      | 1881/4690 [17:10:31<28:42:32, 36.79s/it]
 40%|████      | 1882/4690 [17:11:04<27:42:33, 35.52s/it]
 40%|████      | 1883/4690 [17:11:37<27:01:09, 34.65s/it]
 40%|████      | 1884/4690 [17:12:09<26:31:45, 34.04s/it]
 40%|████      | 1885/4690 [17:12:42<26:16:29, 33.72s/it]
 40%|████      | 1886/4690 [17:13:15<26:01:04, 33.40s/it]
 40%|████      | 1887/4690 [17:13:48<25:53:31, 33.25s/it]
 40%|████      | 1888/4690 [17:14:20<25:44:56, 33.08s/it]
 40%|████      | 1889/4690 [17:14:53<25:41:10, 33.01s/it]
 40%|████      | 1890/4690 [17:15:26<25:35:38, 32.91s/it]
                                                         
{'loss': '0.1015', 'grad_norm': '2.9', 'learning_rate': '6.778e-06', 'epoch': '2.015'}

 40%|████      | 1890/4690 [17:15:26<25:35:38, 32.91s/it]
 40%|████      | 1891/4690 [17:15:59<25:35:01, 32.91s/it]
 40%|████      | 1892/4690 [17:16:32<25:33:16, 32.88s/it]
 40%|████      | 1893/4690 [17:17:05<25:33:44, 32.90s/it]
 40%|████      | 1894/4690 [17:17:38<25:34:19, 32.93s/it]
 40%|████      | 1895/4690 [17:18:10<25:31:53, 32.88s/it]
 40%|████      | 1896/4690 [17:18:43<25:29:28, 32.84s/it]
 40%|████      | 1897/4690 [17:19:16<25:26:06, 32.78s/it]
 40%|████      | 1898/4690 [17:19:48<25:21:18, 32.69s/it]
 40%|████      | 1899/4690 [17:20:21<25:18:54, 32.65s/it]
 41%|████      | 1900/4690 [17:20:54<25:21:37, 32.72s/it]
                                                         
{'loss': '0.1003', 'grad_norm': '1.68', 'learning_rate': '6.746e-06', 'epoch': '2.026'}

 41%|████      | 1900/4690 [17:20:54<25:21:37, 32.72s/it]
 41%|████      | 1901/4690 [17:21:26<25:18:58, 32.68s/it]
 41%|████      | 1902/4690 [17:21:59<25:20:08, 32.71s/it]
 41%|████      | 1903/4690 [17:22:32<25:21:51, 32.76s/it]
 41%|████      | 1904/4690 [17:23:05<25:22:35, 32.79s/it]
 41%|████      | 1905/4690 [17:23:38<25:26:38, 32.89s/it]
 41%|████      | 1906/4690 [17:24:11<25:22:05, 32.80s/it]
 41%|████      | 1907/4690 [17:24:43<25:19:24, 32.76s/it]
 41%|████      | 1908/4690 [17:25:16<25:18:57, 32.76s/it]
 41%|████      | 1909/4690 [17:25:49<25:18:46, 32.77s/it]
 41%|████      | 1910/4690 [17:26:22<25:18:17, 32.77s/it]
                                                         
{'loss': '0.1003', 'grad_norm': '1.933', 'learning_rate': '6.714e-06', 'epoch': '2.036'}

 41%|████      | 1910/4690 [17:26:22<25:18:17, 32.77s/it]
 41%|████      | 1911/4690 [17:26:54<25:20:02, 32.82s/it]
 41%|████      | 1912/4690 [17:27:27<25:20:28, 32.84s/it]
 41%|████      | 1913/4690 [17:28:00<25:16:06, 32.76s/it]
 41%|████      | 1914/4690 [17:28:33<25:16:20, 32.77s/it]
 41%|████      | 1915/4690 [17:29:05<25:12:34, 32.70s/it]
 41%|████      | 1916/4690 [17:29:38<25:17:50, 32.83s/it]
 41%|████      | 1917/4690 [17:30:11<25:15:38, 32.79s/it]
 41%|████      | 1918/4690 [17:30:44<25:14:14, 32.78s/it]
 41%|████      | 1919/4690 [17:31:17<25:17:38, 32.86s/it]
 41%|████      | 1920/4690 [17:31:50<25:20:58, 32.95s/it]
                                                         
{'loss': '0.1011', 'grad_norm': '1.727', 'learning_rate': '6.681e-06', 'epoch': '2.047'}

 41%|████      | 1920/4690 [17:31:50<25:20:58, 32.95s/it]
 41%|████      | 1921/4690 [17:32:23<25:17:49, 32.89s/it]
 41%|████      | 1922/4690 [17:32:55<25:13:53, 32.82s/it]
 41%|████      | 1923/4690 [17:33:28<25:08:27, 32.71s/it]
 41%|████      | 1924/4690 [17:34:01<25:07:25, 32.70s/it]
 41%|████      | 1925/4690 [17:34:34<25:13:24, 32.84s/it]
 41%|████      | 1926/4690 [17:35:06<25:06:54, 32.71s/it]
 41%|████      | 1927/4690 [17:35:39<25:13:04, 32.86s/it]
 41%|████      | 1928/4690 [17:36:12<25:11:47, 32.84s/it]
 41%|████      | 1929/4690 [17:36:45<25:15:31, 32.93s/it]
 41%|████      | 1930/4690 [17:37:18<25:10:16, 32.83s/it]
                                                         
{'loss': '0.09544', 'grad_norm': '2.194', 'learning_rate': '6.648e-06', 'epoch': '2.058'}

 41%|████      | 1930/4690 [17:37:18<25:10:16, 32.83s/it]
 41%|████      | 1931/4690 [17:37:51<25:07:59, 32.79s/it]
 41%|████      | 1932/4690 [17:38:23<25:04:23, 32.73s/it]
 41%|████      | 1933/4690 [17:38:56<25:03:54, 32.73s/it]
 41%|████      | 1934/4690 [17:39:29<25:01:24, 32.69s/it]
 41%|████▏     | 1935/4690 [17:40:01<24:57:35, 32.62s/it]
 41%|████▏     | 1936/4690 [17:40:34<24:58:36, 32.65s/it]
 41%|████▏     | 1937/4690 [17:41:06<24:52:47, 32.53s/it]
 41%|████▏     | 1938/4690 [17:41:39<24:57:16, 32.64s/it]
 41%|████▏     | 1939/4690 [17:42:12<24:57:40, 32.66s/it]
 41%|████▏     | 1940/4690 [17:42:44<24:58:58, 32.70s/it]
                                                         
{'loss': '0.09482', 'grad_norm': '1.832', 'learning_rate': '6.616e-06', 'epoch': '2.068'}

 41%|████▏     | 1940/4690 [17:42:44<24:58:58, 32.70s/it]
 41%|████▏     | 1941/4690 [17:43:17<25:02:33, 32.80s/it]
 41%|████▏     | 1942/4690 [17:43:50<25:02:08, 32.80s/it]
 41%|████▏     | 1943/4690 [17:44:23<24:58:59, 32.74s/it]
 41%|████▏     | 1944/4690 [17:44:56<25:01:22, 32.80s/it]
 41%|████▏     | 1945/4690 [17:45:29<25:07:06, 32.94s/it]
 41%|████▏     | 1946/4690 [17:46:02<25:06:14, 32.94s/it]
 42%|████▏     | 1947/4690 [17:46:35<25:04:48, 32.92s/it]
 42%|████▏     | 1948/4690 [17:47:08<25:06:58, 32.98s/it]
 42%|████▏     | 1949/4690 [17:47:41<25:06:05, 32.97s/it]
 42%|████▏     | 1950/4690 [17:48:14<25:00:51, 32.87s/it]
                                                         
{'loss': '0.1022', 'grad_norm': '1.874', 'learning_rate': '6.583e-06', 'epoch': '2.079'}

 42%|████▏     | 1950/4690 [17:48:14<25:00:51, 32.87s/it]
 42%|████▏     | 1951/4690 [17:48:46<24:52:23, 32.69s/it]
 42%|████▏     | 1952/4690 [17:49:19<24:57:20, 32.81s/it]
 42%|████▏     | 1953/4690 [17:49:52<24:57:31, 32.83s/it]
 42%|████▏     | 1954/4690 [17:50:25<24:57:56, 32.85s/it]
 42%|████▏     | 1955/4690 [17:50:58<25:01:50, 32.95s/it]
 42%|████▏     | 1956/4690 [17:51:31<25:01:57, 32.96s/it]
 42%|████▏     | 1957/4690 [17:52:03<24:56:57, 32.86s/it]
 42%|████▏     | 1958/4690 [17:52:36<24:55:05, 32.84s/it]
 42%|████▏     | 1959/4690 [17:53:09<24:54:05, 32.83s/it]
 42%|████▏     | 1960/4690 [17:53:42<24:51:51, 32.79s/it]
                                                         
{'loss': '0.105', 'grad_norm': '1.897', 'learning_rate': '6.55e-06', 'epoch': '2.09'}

 42%|████▏     | 1960/4690 [17:53:42<24:51:51, 32.79s/it]
 42%|████▏     | 1961/4690 [17:54:15<24:51:12, 32.79s/it]
 42%|████▏     | 1962/4690 [17:54:47<24:51:17, 32.80s/it]
 42%|████▏     | 1963/4690 [17:55:20<24:49:16, 32.77s/it]
 42%|████▏     | 1964/4690 [17:55:53<24:49:46, 32.79s/it]
 42%|████▏     | 1965/4690 [17:56:25<24:46:20, 32.73s/it]
 42%|████▏     | 1966/4690 [17:56:58<24:40:29, 32.61s/it]
 42%|████▏     | 1967/4690 [17:57:30<24:39:27, 32.60s/it]
 42%|████▏     | 1968/4690 [17:58:03<24:42:10, 32.67s/it]
 42%|████▏     | 1969/4690 [17:58:36<24:41:22, 32.67s/it]
 42%|████▏     | 1970/4690 [17:59:08<24:40:15, 32.65s/it]
                                                         
{'loss': '0.09149', 'grad_norm': '1.979', 'learning_rate': '6.517e-06', 'epoch': '2.1'}

 42%|████▏     | 1970/4690 [17:59:08<24:40:15, 32.65s/it]
 42%|████▏     | 1971/4690 [17:59:41<24:38:33, 32.63s/it]
 42%|████▏     | 1972/4690 [18:00:14<24:40:09, 32.67s/it]
 42%|████▏     | 1973/4690 [18:00:46<24:35:34, 32.59s/it]
 42%|████▏     | 1974/4690 [18:01:19<24:37:09, 32.63s/it]
 42%|████▏     | 1975/4690 [18:01:52<24:42:33, 32.76s/it]
 42%|████▏     | 1976/4690 [18:02:25<24:41:52, 32.76s/it]
 42%|████▏     | 1977/4690 [18:02:57<24:36:05, 32.64s/it]
 42%|████▏     | 1978/4690 [18:03:30<24:38:58, 32.72s/it]
 42%|████▏     | 1979/4690 [18:04:03<24:41:24, 32.79s/it]
 42%|████▏     | 1980/4690 [18:04:36<24:40:28, 32.78s/it]
                                                         
{'loss': '0.1017', 'grad_norm': '1.895', 'learning_rate': '6.485e-06', 'epoch': '2.111'}

 42%|████▏     | 1980/4690 [18:04:36<24:40:28, 32.78s/it]
 42%|████▏     | 1981/4690 [18:05:09<24:42:39, 32.84s/it]
 42%|████▏     | 1982/4690 [18:05:41<24:41:03, 32.82s/it]
 42%|████▏     | 1983/4690 [18:06:15<24:45:25, 32.92s/it]
 42%|████▏     | 1984/4690 [18:06:47<24:39:10, 32.80s/it]
 42%|████▏     | 1985/4690 [18:07:20<24:42:05, 32.87s/it]
 42%|████▏     | 1986/4690 [18:07:53<24:37:36, 32.79s/it]
 42%|████▏     | 1987/4690 [18:08:26<24:38:01, 32.81s/it]
 42%|████▏     | 1988/4690 [18:08:59<24:38:05, 32.82s/it]
 42%|████▏     | 1989/4690 [18:09:31<24:39:34, 32.87s/it]
 42%|████▏     | 1990/4690 [18:10:04<24:35:44, 32.79s/it]
                                                         
{'loss': '0.1012', 'grad_norm': '2.267', 'learning_rate': '6.452e-06', 'epoch': '2.122'}

 42%|████▏     | 1990/4690 [18:10:04<24:35:44, 32.79s/it]
 42%|████▏     | 1991/4690 [18:10:37<24:36:09, 32.82s/it]
 42%|████▏     | 1992/4690 [18:11:10<24:38:43, 32.89s/it]
 42%|████▏     | 1993/4690 [18:11:43<24:34:20, 32.80s/it]
 43%|████▎     | 1994/4690 [18:12:16<24:43:09, 33.01s/it]
 43%|████▎     | 1995/4690 [18:12:50<24:49:58, 33.17s/it]
 43%|████▎     | 1996/4690 [18:13:23<24:44:54, 33.07s/it]
 43%|████▎     | 1997/4690 [18:13:55<24:38:47, 32.95s/it]
 43%|████▎     | 1998/4690 [18:14:28<24:35:19, 32.88s/it]
 43%|████▎     | 1999/4690 [18:15:01<24:31:27, 32.81s/it]
 43%|████▎     | 2000/4690 [18:15:33<24:26:16, 32.71s/it]
                                                         
{'loss': '0.09814', 'grad_norm': '1.769', 'learning_rate': '6.418e-06', 'epoch': '2.132'}

 43%|████▎     | 2000/4690 [18:15:33<24:26:16, 32.71s/it]
 43%|████▎     | 2001/4690 [18:16:06<24:25:59, 32.71s/it]
 43%|████▎     | 2002/4690 [18:16:38<24:25:53, 32.72s/it]
 43%|████▎     | 2003/4690 [18:17:11<24:27:22, 32.77s/it]
 43%|████▎     | 2004/4690 [18:17:44<24:26:33, 32.76s/it]
 43%|████▎     | 2005/4690 [18:18:17<24:27:38, 32.80s/it]
 43%|████▎     | 2006/4690 [18:18:50<24:30:42, 32.88s/it]
 43%|████▎     | 2007/4690 [18:19:23<24:34:20, 32.97s/it]
 43%|████▎     | 2008/4690 [18:19:57<24:39:09, 33.09s/it]
 43%|████▎     | 2009/4690 [18:20:30<24:37:52, 33.07s/it]
 43%|████▎     | 2010/4690 [18:21:03<24:37:32, 33.08s/it]
                                                         
{'loss': '0.1007', 'grad_norm': '2.059', 'learning_rate': '6.385e-06', 'epoch': '2.143'}

 43%|████▎     | 2010/4690 [18:21:03<24:37:32, 33.08s/it]
 43%|████▎     | 2011/4690 [18:21:36<24:34:35, 33.03s/it]
 43%|████▎     | 2012/4690 [18:22:08<24:27:27, 32.88s/it]
 43%|████▎     | 2013/4690 [18:22:41<24:22:38, 32.78s/it]
 43%|████▎     | 2014/4690 [18:23:13<24:22:08, 32.78s/it]
 43%|████▎     | 2015/4690 [18:23:46<24:23:26, 32.82s/it]
 43%|████▎     | 2016/4690 [18:24:19<24:22:45, 32.82s/it]
 43%|████▎     | 2017/4690 [18:24:52<24:24:24, 32.87s/it]
 43%|████▎     | 2018/4690 [18:25:25<24:16:24, 32.70s/it]
 43%|████▎     | 2019/4690 [18:25:57<24:13:18, 32.65s/it]
 43%|████▎     | 2020/4690 [18:26:30<24:14:49, 32.69s/it]
                                                         
{'loss': '0.09863', 'grad_norm': '1.904', 'learning_rate': '6.352e-06', 'epoch': '2.154'}

 43%|████▎     | 2020/4690 [18:26:30<24:14:49, 32.69s/it]
 43%|████▎     | 2021/4690 [18:27:03<24:22:42, 32.88s/it]
 43%|████▎     | 2022/4690 [18:27:37<24:30:18, 33.07s/it]
 43%|████▎     | 2023/4690 [18:28:09<24:24:38, 32.95s/it]
 43%|████▎     | 2024/4690 [18:28:42<24:22:48, 32.92s/it]
 43%|████▎     | 2025/4690 [18:29:15<24:21:01, 32.89s/it]
 43%|████▎     | 2026/4690 [18:29:48<24:21:32, 32.92s/it]
 43%|████▎     | 2027/4690 [18:30:21<24:20:39, 32.91s/it]
 43%|████▎     | 2028/4690 [18:30:54<24:17:31, 32.85s/it]
 43%|████▎     | 2029/4690 [18:31:26<24:15:46, 32.82s/it]
 43%|████▎     | 2030/4690 [18:31:59<24:16:29, 32.85s/it]
                                                         
{'loss': '0.1041', 'grad_norm': '2.08', 'learning_rate': '6.319e-06', 'epoch': '2.164'}

 43%|████▎     | 2030/4690 [18:31:59<24:16:29, 32.85s/it]
 43%|████▎     | 2031/4690 [18:32:32<24:13:00, 32.79s/it]
 43%|████▎     | 2032/4690 [18:33:05<24:13:54, 32.82s/it]
 43%|████▎     | 2033/4690 [18:33:38<24:15:14, 32.86s/it]
 43%|████▎     | 2034/4690 [18:34:10<24:11:18, 32.79s/it]
 43%|████▎     | 2035/4690 [18:34:43<24:06:03, 32.68s/it]
 43%|████▎     | 2036/4690 [18:35:16<24:06:06, 32.69s/it]
 43%|████▎     | 2037/4690 [18:35:49<24:09:04, 32.77s/it]
 43%|████▎     | 2038/4690 [18:36:21<24:04:42, 32.69s/it]
 43%|████▎     | 2039/4690 [18:36:54<24:04:07, 32.68s/it]
 43%|████▎     | 2040/4690 [18:37:26<24:02:10, 32.65s/it]
                                                         
{'loss': '0.1035', 'grad_norm': '1.971', 'learning_rate': '6.285e-06', 'epoch': '2.175'}

 43%|████▎     | 2040/4690 [18:37:26<24:02:10, 32.65s/it]
 44%|████▎     | 2041/4690 [18:37:59<24:02:39, 32.68s/it]
 44%|████▎     | 2042/4690 [18:38:32<24:06:46, 32.78s/it]
 44%|████▎     | 2043/4690 [18:39:05<24:02:40, 32.70s/it]
 44%|████▎     | 2044/4690 [18:39:37<24:05:46, 32.78s/it]
 44%|████▎     | 2045/4690 [18:40:10<24:03:12, 32.74s/it]
 44%|████▎     | 2046/4690 [18:40:43<24:05:08, 32.79s/it]
 44%|████▎     | 2047/4690 [18:41:16<24:07:13, 32.85s/it]
 44%|████▎     | 2048/4690 [18:41:49<24:04:20, 32.80s/it]
 44%|████▎     | 2049/4690 [18:42:21<24:02:52, 32.78s/it]
 44%|████▎     | 2050/4690 [18:42:54<24:02:50, 32.79s/it]
                                                         
{'loss': '0.09948', 'grad_norm': '1.837', 'learning_rate': '6.252e-06', 'epoch': '2.186'}

 44%|████▎     | 2050/4690 [18:42:54<24:02:50, 32.79s/it]
 44%|████▎     | 2051/4690 [18:43:27<24:07:14, 32.90s/it]
 44%|████▍     | 2052/4690 [18:44:00<24:02:49, 32.82s/it]
 44%|████▍     | 2053/4690 [18:44:33<23:59:55, 32.76s/it]
 44%|████▍     | 2054/4690 [18:45:06<24:04:11, 32.87s/it]
 44%|████▍     | 2055/4690 [18:45:39<24:04:35, 32.89s/it]
 44%|████▍     | 2056/4690 [18:46:11<24:01:42, 32.84s/it]
 44%|████▍     | 2057/4690 [18:46:44<23:58:07, 32.77s/it]
 44%|████▍     | 2058/4690 [18:47:17<23:57:37, 32.77s/it]
 44%|████▍     | 2059/4690 [18:47:50<23:59:30, 32.83s/it]
 44%|████▍     | 2060/4690 [18:48:23<24:02:58, 32.92s/it]
                                                         
{'loss': '0.09795', 'grad_norm': '1.64', 'learning_rate': '6.219e-06', 'epoch': '2.196'}

 44%|████▍     | 2060/4690 [18:48:23<24:02:58, 32.92s/it]
 44%|████▍     | 2061/4690 [18:48:56<23:59:12, 32.85s/it]
 44%|████▍     | 2062/4690 [18:49:29<23:59:42, 32.87s/it]
 44%|████▍     | 2063/4690 [18:50:01<23:56:55, 32.82s/it]
 44%|████▍     | 2064/4690 [18:50:34<23:57:59, 32.86s/it]
 44%|████▍     | 2065/4690 [18:51:07<23:53:23, 32.76s/it]
 44%|████▍     | 2066/4690 [18:51:39<23:52:23, 32.75s/it]
 44%|████▍     | 2067/4690 [18:52:12<23:52:39, 32.77s/it]
 44%|████▍     | 2068/4690 [18:52:45<23:54:37, 32.83s/it]
 44%|████▍     | 2069/4690 [18:53:18<23:57:22, 32.90s/it]
 44%|████▍     | 2070/4690 [18:53:51<23:55:41, 32.88s/it]
                                                         
{'loss': '0.1025', 'grad_norm': '1.688', 'learning_rate': '6.185e-06', 'epoch': '2.207'}

 44%|████▍     | 2070/4690 [18:53:51<23:55:41, 32.88s/it]
 44%|████▍     | 2071/4690 [18:54:24<23:55:50, 32.89s/it]
 44%|████▍     | 2072/4690 [18:54:57<23:53:46, 32.86s/it]
 44%|████▍     | 2073/4690 [18:55:30<23:52:53, 32.85s/it]
 44%|████▍     | 2074/4690 [18:56:03<23:53:20, 32.87s/it]
 44%|████▍     | 2075/4690 [18:56:35<23:45:07, 32.70s/it]
 44%|████▍     | 2076/4690 [18:57:07<23:42:13, 32.64s/it]
 44%|████▍     | 2077/4690 [18:57:40<23:39:29, 32.59s/it]
 44%|████▍     | 2078/4690 [18:58:13<23:40:33, 32.63s/it]
 44%|████▍     | 2079/4690 [18:58:45<23:41:11, 32.66s/it]
 44%|████▍     | 2080/4690 [18:59:18<23:41:04, 32.67s/it]
                                                         
{'loss': '0.1006', 'grad_norm': '1.951', 'learning_rate': '6.152e-06', 'epoch': '2.218'}

 44%|████▍     | 2080/4690 [18:59:18<23:41:04, 32.67s/it]
 44%|████▍     | 2081/4690 [18:59:51<23:43:55, 32.75s/it]
 44%|████▍     | 2082/4690 [19:00:24<23:45:19, 32.79s/it]
 44%|████▍     | 2083/4690 [19:00:56<23:42:24, 32.74s/it]
 44%|████▍     | 2084/4690 [19:01:29<23:41:30, 32.73s/it]
 44%|████▍     | 2085/4690 [19:02:02<23:43:05, 32.78s/it]
 44%|████▍     | 2086/4690 [19:02:35<23:38:56, 32.69s/it]
 44%|████▍     | 2087/4690 [19:03:07<23:37:23, 32.67s/it]
 45%|████▍     | 2088/4690 [19:03:40<23:40:15, 32.75s/it]
 45%|████▍     | 2089/4690 [19:04:13<23:38:53, 32.73s/it]
 45%|████▍     | 2090/4690 [19:04:46<23:42:08, 32.82s/it]
                                                         
{'loss': '0.1001', 'grad_norm': '1.94', 'learning_rate': '6.118e-06', 'epoch': '2.228'}

 45%|████▍     | 2090/4690 [19:04:46<23:42:08, 32.82s/it]
 45%|████▍     | 2091/4690 [19:05:19<23:42:53, 32.85s/it]
 45%|████▍     | 2092/4690 [19:05:51<23:38:28, 32.76s/it]
 45%|████▍     | 2093/4690 [19:06:24<23:36:13, 32.72s/it]
 45%|████▍     | 2094/4690 [19:06:57<23:39:38, 32.81s/it]
 45%|████▍     | 2095/4690 [19:07:30<23:37:27, 32.77s/it]
 45%|████▍     | 2096/4690 [19:08:03<23:42:40, 32.91s/it]
 45%|████▍     | 2097/4690 [19:08:36<23:43:37, 32.94s/it]
 45%|████▍     | 2098/4690 [19:09:09<23:44:28, 32.97s/it]
 45%|████▍     | 2099/4690 [19:09:42<23:42:42, 32.95s/it]
 45%|████▍     | 2100/4690 [19:10:15<23:39:19, 32.88s/it]
                                                         
{'loss': '0.09955', 'grad_norm': '1.849', 'learning_rate': '6.084e-06', 'epoch': '2.239'}

 45%|████▍     | 2100/4690 [19:10:15<23:39:19, 32.88s/it]
 45%|████▍     | 2101/4690 [19:10:47<23:35:05, 32.79s/it]
 45%|████▍     | 2102/4690 [19:11:20<23:37:06, 32.85s/it]
 45%|████▍     | 2103/4690 [19:11:53<23:39:52, 32.93s/it]
 45%|████▍     | 2104/4690 [19:12:26<23:41:31, 32.98s/it]
 45%|████▍     | 2105/4690 [19:12:59<23:40:01, 32.96s/it]
 45%|████▍     | 2106/4690 [19:13:32<23:33:42, 32.83s/it]
 45%|████▍     | 2107/4690 [19:14:05<23:34:47, 32.86s/it]
 45%|████▍     | 2108/4690 [19:14:37<23:30:21, 32.77s/it]
 45%|████▍     | 2109/4690 [19:15:11<23:38:28, 32.97s/it]
 45%|████▍     | 2110/4690 [19:15:43<23:33:44, 32.88s/it]
                                                         
{'loss': '0.09735', 'grad_norm': '2.008', 'learning_rate': '6.05e-06', 'epoch': '2.25'}

 45%|████▍     | 2110/4690 [19:15:43<23:33:44, 32.88s/it]
 45%|████▌     | 2111/4690 [19:16:16<23:32:36, 32.86s/it]
 45%|████▌     | 2112/4690 [19:16:49<23:28:33, 32.78s/it]
 45%|████▌     | 2113/4690 [19:17:22<23:31:21, 32.86s/it]
 45%|████▌     | 2114/4690 [19:17:55<23:28:19, 32.80s/it]
 45%|████▌     | 2115/4690 [19:18:27<23:26:04, 32.76s/it]
 45%|████▌     | 2116/4690 [19:19:00<23:23:38, 32.72s/it]
 45%|████▌     | 2117/4690 [19:19:32<23:20:42, 32.66s/it]
 45%|████▌     | 2118/4690 [19:20:05<23:21:08, 32.69s/it]
 45%|████▌     | 2119/4690 [19:20:38<23:24:23, 32.77s/it]
 45%|████▌     | 2120/4690 [19:21:11<23:22:31, 32.74s/it]
                                                         
{'loss': '0.1026', 'grad_norm': '1.792', 'learning_rate': '6.017e-06', 'epoch': '2.26'}

 45%|████▌     | 2120/4690 [19:21:11<23:22:31, 32.74s/it]
 45%|████▌     | 2121/4690 [19:21:43<23:17:52, 32.65s/it]
 45%|████▌     | 2122/4690 [19:22:16<23:19:32, 32.70s/it]
 45%|████▌     | 2123/4690 [19:22:48<23:15:38, 32.62s/it]
 45%|████▌     | 2124/4690 [19:23:21<23:12:26, 32.56s/it]
 45%|████▌     | 2125/4690 [19:23:54<23:19:22, 32.73s/it]
 45%|████▌     | 2126/4690 [19:24:27<23:20:50, 32.78s/it]
 45%|████▌     | 2127/4690 [19:25:00<23:20:11, 32.78s/it]
 45%|████▌     | 2128/4690 [19:25:33<23:22:09, 32.84s/it]
 45%|████▌     | 2129/4690 [19:26:05<23:16:34, 32.72s/it]
 45%|████▌     | 2130/4690 [19:26:38<23:15:28, 32.71s/it]
                                                         
{'loss': '0.09902', 'grad_norm': '1.894', 'learning_rate': '5.983e-06', 'epoch': '2.271'}

 45%|████▌     | 2130/4690 [19:26:38<23:15:28, 32.71s/it]
 45%|████▌     | 2131/4690 [19:27:11<23:21:05, 32.85s/it]
 45%|████▌     | 2132/4690 [19:27:44<23:20:19, 32.85s/it]
 45%|████▌     | 2133/4690 [19:28:17<23:26:54, 33.01s/it]
 46%|████▌     | 2134/4690 [19:28:50<23:24:12, 32.96s/it]
 46%|████▌     | 2135/4690 [19:29:22<23:17:03, 32.81s/it]
 46%|████▌     | 2136/4690 [19:29:55<23:11:44, 32.70s/it]
 46%|████▌     | 2137/4690 [19:30:28<23:13:00, 32.74s/it]
 46%|████▌     | 2138/4690 [19:31:00<23:11:35, 32.72s/it]
 46%|████▌     | 2139/4690 [19:31:33<23:13:36, 32.78s/it]
 46%|████▌     | 2140/4690 [19:32:06<23:12:57, 32.78s/it]
                                                         
{'loss': '0.104', 'grad_norm': '2.054', 'learning_rate': '5.949e-06', 'epoch': '2.282'}

 46%|████▌     | 2140/4690 [19:32:06<23:12:57, 32.78s/it]
 46%|████▌     | 2141/4690 [19:32:39<23:10:56, 32.74s/it]
 46%|████▌     | 2142/4690 [19:33:12<23:14:38, 32.84s/it]
 46%|████▌     | 2143/4690 [19:33:45<23:12:30, 32.80s/it]
 46%|████▌     | 2144/4690 [19:34:17<23:09:47, 32.75s/it]
 46%|████▌     | 2145/4690 [19:34:50<23:07:32, 32.71s/it]
 46%|████▌     | 2146/4690 [19:35:23<23:08:34, 32.75s/it]
 46%|████▌     | 2147/4690 [19:35:56<23:11:03, 32.82s/it]
 46%|████▌     | 2148/4690 [19:36:28<23:04:45, 32.69s/it]
 46%|████▌     | 2149/4690 [19:37:01<23:07:37, 32.77s/it]
 46%|████▌     | 2150/4690 [19:37:34<23:07:37, 32.78s/it]
                                                         
{'loss': '0.1014', 'grad_norm': '2.021', 'learning_rate': '5.915e-06', 'epoch': '2.292'}

 46%|████▌     | 2150/4690 [19:37:34<23:07:37, 32.78s/it]
 46%|████▌     | 2151/4690 [19:38:06<23:03:31, 32.69s/it]
 46%|████▌     | 2152/4690 [19:38:39<23:05:07, 32.75s/it]
 46%|████▌     | 2153/4690 [19:39:12<23:01:37, 32.68s/it]
 46%|████▌     | 2154/4690 [19:39:44<23:00:44, 32.67s/it]
 46%|████▌     | 2155/4690 [19:40:17<23:00:36, 32.68s/it]
 46%|████▌     | 2156/4690 [19:40:50<23:01:27, 32.71s/it]
 46%|████▌     | 2157/4690 [19:41:22<23:01:06, 32.71s/it]
 46%|████▌     | 2158/4690 [19:41:55<23:00:21, 32.71s/it]
 46%|████▌     | 2159/4690 [19:42:28<23:01:22, 32.75s/it]
 46%|████▌     | 2160/4690 [19:43:01<23:01:50, 32.77s/it]
                                                         
{'loss': '0.1012', 'grad_norm': '2.044', 'learning_rate': '5.881e-06', 'epoch': '2.303'}

 46%|████▌     | 2160/4690 [19:43:01<23:01:50, 32.77s/it]
 46%|████▌     | 2161/4690 [19:43:34<23:01:22, 32.77s/it]
 46%|████▌     | 2162/4690 [19:44:07<23:05:56, 32.89s/it]
 46%|████▌     | 2163/4690 [19:44:40<23:08:13, 32.96s/it]
 46%|████▌     | 2164/4690 [19:45:13<23:04:08, 32.88s/it]
 46%|████▌     | 2165/4690 [19:45:45<23:03:32, 32.88s/it]
 46%|████▌     | 2166/4690 [19:46:18<22:59:05, 32.78s/it]
 46%|████▌     | 2167/4690 [19:46:52<23:10:02, 33.06s/it]
 46%|████▌     | 2168/4690 [19:47:24<23:06:12, 32.98s/it]
 46%|████▌     | 2169/4690 [19:47:58<23:07:32, 33.02s/it]
 46%|████▋     | 2170/4690 [19:48:30<23:01:26, 32.89s/it]
                                                         
{'loss': '0.1024', 'grad_norm': '1.884', 'learning_rate': '5.847e-06', 'epoch': '2.314'}

 46%|████▋     | 2170/4690 [19:48:30<23:01:26, 32.89s/it]
 46%|████▋     | 2171/4690 [19:49:03<22:59:20, 32.85s/it]
 46%|████▋     | 2172/4690 [19:49:36<22:58:07, 32.84s/it]
 46%|████▋     | 2173/4690 [19:50:09<23:00:31, 32.91s/it]
 46%|████▋     | 2174/4690 [19:50:42<22:57:12, 32.84s/it]
 46%|████▋     | 2175/4690 [19:51:15<22:59:32, 32.91s/it]
 46%|████▋     | 2176/4690 [19:51:47<22:50:55, 32.72s/it]
 46%|████▋     | 2177/4690 [19:52:20<22:52:50, 32.78s/it]
 46%|████▋     | 2178/4690 [19:52:52<22:50:23, 32.73s/it]
 46%|████▋     | 2179/4690 [19:53:25<22:49:57, 32.73s/it]
 46%|████▋     | 2180/4690 [19:53:58<22:52:11, 32.80s/it]
                                                         
{'loss': '0.1021', 'grad_norm': '2.139', 'learning_rate': '5.813e-06', 'epoch': '2.324'}

 46%|████▋     | 2180/4690 [19:53:58<22:52:11, 32.80s/it]
 47%|████▋     | 2181/4690 [19:54:31<22:51:34, 32.80s/it]
 47%|████▋     | 2182/4690 [19:55:04<22:56:59, 32.94s/it]
 47%|████▋     | 2183/4690 [19:55:37<22:52:43, 32.85s/it]
 47%|████▋     | 2184/4690 [19:56:10<22:51:41, 32.84s/it]
 47%|████▋     | 2185/4690 [19:56:42<22:48:15, 32.77s/it]
 47%|████▋     | 2186/4690 [19:57:15<22:47:43, 32.77s/it]
 47%|████▋     | 2187/4690 [19:57:48<22:46:21, 32.75s/it]
 47%|████▋     | 2188/4690 [19:58:21<22:47:49, 32.80s/it]
 47%|████▋     | 2189/4690 [19:58:53<22:45:54, 32.77s/it]
 47%|████▋     | 2190/4690 [19:59:26<22:43:41, 32.73s/it]
                                                         
{'loss': '0.1034', 'grad_norm': '1.829', 'learning_rate': '5.779e-06', 'epoch': '2.335'}

 47%|████▋     | 2190/4690 [19:59:26<22:43:41, 32.73s/it]
 47%|████▋     | 2191/4690 [19:59:59<22:40:36, 32.67s/it]
 47%|████▋     | 2192/4690 [20:00:31<22:42:20, 32.72s/it]
 47%|████▋     | 2193/4690 [20:01:04<22:44:42, 32.79s/it]
 47%|████▋     | 2194/4690 [20:01:37<22:44:33, 32.80s/it]
 47%|████▋     | 2195/4690 [20:02:10<22:44:11, 32.81s/it]
 47%|████▋     | 2196/4690 [20:02:43<22:40:38, 32.73s/it]
 47%|████▋     | 2197/4690 [20:03:15<22:41:41, 32.77s/it]
 47%|████▋     | 2198/4690 [20:03:48<22:41:29, 32.78s/it]
 47%|████▋     | 2199/4690 [20:04:21<22:37:28, 32.70s/it]
 47%|████▋     | 2200/4690 [20:04:54<22:39:40, 32.76s/it]
                                                         
{'loss': '0.09914', 'grad_norm': '1.962', 'learning_rate': '5.745e-06', 'epoch': '2.346'}

 47%|████▋     | 2200/4690 [20:04:54<22:39:40, 32.76s/it]
 47%|████▋     | 2201/4690 [20:05:27<22:43:35, 32.87s/it]
 47%|████▋     | 2202/4690 [20:06:00<22:43:10, 32.87s/it]
 47%|████▋     | 2203/4690 [20:06:33<22:43:59, 32.91s/it]
 47%|████▋     | 2204/4690 [20:07:06<22:46:10, 32.97s/it]
 47%|████▋     | 2205/4690 [20:07:39<22:46:23, 32.99s/it]
 47%|████▋     | 2206/4690 [20:08:12<22:45:56, 32.99s/it]
 47%|████▋     | 2207/4690 [20:08:45<22:44:23, 32.97s/it]
 47%|████▋     | 2208/4690 [20:09:17<22:40:40, 32.89s/it]
 47%|████▋     | 2209/4690 [20:09:50<22:40:38, 32.91s/it]
 47%|████▋     | 2210/4690 [20:10:24<22:45:29, 33.04s/it]
                                                         
{'loss': '0.1008', 'grad_norm': '2.021', 'learning_rate': '5.711e-06', 'epoch': '2.356'}

 47%|████▋     | 2210/4690 [20:10:24<22:45:29, 33.04s/it]
 47%|████▋     | 2211/4690 [20:10:57<22:44:53, 33.03s/it]
 47%|████▋     | 2212/4690 [20:11:29<22:40:13, 32.94s/it]
 47%|████▋     | 2213/4690 [20:12:02<22:40:19, 32.95s/it]
 47%|████▋     | 2214/4690 [20:12:35<22:38:27, 32.92s/it]
 47%|████▋     | 2215/4690 [20:13:08<22:37:00, 32.90s/it]
 47%|████▋     | 2216/4690 [20:13:41<22:36:42, 32.90s/it]
 47%|████▋     | 2217/4690 [20:14:14<22:37:04, 32.93s/it]
 47%|████▋     | 2218/4690 [20:14:47<22:35:10, 32.89s/it]
 47%|████▋     | 2219/4690 [20:15:20<22:34:56, 32.90s/it]
 47%|████▋     | 2220/4690 [20:15:53<22:33:32, 32.88s/it]
                                                         
{'loss': '0.09781', 'grad_norm': '2.171', 'learning_rate': '5.676e-06', 'epoch': '2.367'}

 47%|████▋     | 2220/4690 [20:15:53<22:33:32, 32.88s/it]
 47%|████▋     | 2221/4690 [20:16:25<22:30:10, 32.81s/it]
 47%|████▋     | 2222/4690 [20:16:58<22:30:19, 32.83s/it]
 47%|████▋     | 2223/4690 [20:17:31<22:30:03, 32.83s/it]
 47%|████▋     | 2224/4690 [20:18:04<22:28:24, 32.81s/it]
 47%|████▋     | 2225/4690 [20:18:36<22:26:27, 32.77s/it]
 47%|████▋     | 2226/4690 [20:19:09<22:26:44, 32.79s/it]
 47%|████▋     | 2227/4690 [20:19:42<22:30:57, 32.91s/it]
 48%|████▊     | 2228/4690 [20:20:15<22:30:46, 32.92s/it]
 48%|████▊     | 2229/4690 [20:20:48<22:32:29, 32.97s/it]
 48%|████▊     | 2230/4690 [20:21:21<22:30:06, 32.93s/it]
                                                         
{'loss': '0.0984', 'grad_norm': '1.961', 'learning_rate': '5.642e-06', 'epoch': '2.378'}

 48%|████▊     | 2230/4690 [20:21:21<22:30:06, 32.93s/it]
 48%|████▊     | 2231/4690 [20:21:54<22:22:29, 32.76s/it]
 48%|████▊     | 2232/4690 [20:22:26<22:22:40, 32.77s/it]
 48%|████▊     | 2233/4690 [20:22:59<22:22:36, 32.79s/it]
 48%|████▊     | 2234/4690 [20:23:32<22:24:39, 32.85s/it]
 48%|████▊     | 2235/4690 [20:24:05<22:25:50, 32.89s/it]
 48%|████▊     | 2236/4690 [20:24:38<22:23:34, 32.85s/it]
 48%|████▊     | 2237/4690 [20:25:11<22:22:47, 32.84s/it]
 48%|████▊     | 2238/4690 [20:25:44<22:24:39, 32.90s/it]
 48%|████▊     | 2239/4690 [20:26:17<22:21:37, 32.84s/it]
 48%|████▊     | 2240/4690 [20:26:49<22:20:05, 32.82s/it]
                                                         
{'loss': '0.1018', 'grad_norm': '2.084', 'learning_rate': '5.608e-06', 'epoch': '2.388'}

 48%|████▊     | 2240/4690 [20:26:49<22:20:05, 32.82s/it]
 48%|████▊     | 2241/4690 [20:27:22<22:16:56, 32.75s/it]
 48%|████▊     | 2242/4690 [20:27:55<22:20:01, 32.84s/it]
 48%|████▊     | 2243/4690 [20:28:28<22:18:24, 32.82s/it]
 48%|████▊     | 2244/4690 [20:29:00<22:16:10, 32.78s/it]
 48%|████▊     | 2245/4690 [20:29:33<22:13:47, 32.73s/it]
 48%|████▊     | 2246/4690 [20:30:05<22:09:21, 32.64s/it]
 48%|████▊     | 2247/4690 [20:30:38<22:10:45, 32.68s/it]
 48%|████▊     | 2248/4690 [20:31:11<22:14:12, 32.78s/it]
 48%|████▊     | 2249/4690 [20:31:44<22:11:51, 32.74s/it]
 48%|████▊     | 2250/4690 [20:32:16<22:09:04, 32.68s/it]
                                                         
{'loss': '0.09855', 'grad_norm': '1.803', 'learning_rate': '5.574e-06', 'epoch': '2.399'}

 48%|████▊     | 2250/4690 [20:32:16<22:09:04, 32.68s/it]
 48%|████▊     | 2251/4690 [20:32:50<22:14:40, 32.83s/it]
 48%|████▊     | 2252/4690 [20:33:22<22:13:15, 32.81s/it]
 48%|████▊     | 2253/4690 [20:33:55<22:06:05, 32.65s/it]
 48%|████▊     | 2254/4690 [20:34:27<22:06:23, 32.67s/it]
 48%|████▊     | 2255/4690 [20:35:00<22:03:04, 32.60s/it]
 48%|████▊     | 2256/4690 [20:35:33<22:07:28, 32.72s/it]
 48%|████▊     | 2257/4690 [20:36:06<22:07:31, 32.74s/it]
 48%|████▊     | 2258/4690 [20:36:38<22:03:50, 32.66s/it]
 48%|████▊     | 2259/4690 [20:37:11<22:04:12, 32.68s/it]
 48%|████▊     | 2260/4690 [20:37:44<22:06:21, 32.75s/it]
                                                         
{'loss': '0.1019', 'grad_norm': '1.96', 'learning_rate': '5.539e-06', 'epoch': '2.41'}

 48%|████▊     | 2260/4690 [20:37:44<22:06:21, 32.75s/it]
 48%|████▊     | 2261/4690 [20:38:16<22:04:22, 32.71s/it]
 48%|████▊     | 2262/4690 [20:38:49<22:04:38, 32.73s/it]
 48%|████▊     | 2263/4690 [20:39:22<22:02:17, 32.69s/it]
 48%|████▊     | 2264/4690 [20:39:54<22:00:01, 32.65s/it]
 48%|████▊     | 2265/4690 [20:40:27<22:03:18, 32.74s/it]
 48%|████▊     | 2266/4690 [20:41:00<22:05:25, 32.81s/it]
 48%|████▊     | 2267/4690 [20:41:33<22:05:18, 32.82s/it]
 48%|████▊     | 2268/4690 [20:42:06<22:05:55, 32.85s/it]
 48%|████▊     | 2269/4690 [20:42:39<22:06:39, 32.88s/it]
 48%|████▊     | 2270/4690 [20:43:12<22:04:42, 32.84s/it]
                                                         
{'loss': '0.1009', 'grad_norm': '2.084', 'learning_rate': '5.505e-06', 'epoch': '2.42'}

 48%|████▊     | 2270/4690 [20:43:12<22:04:42, 32.84s/it]
 48%|████▊     | 2271/4690 [20:43:44<22:02:08, 32.79s/it]
 48%|████▊     | 2272/4690 [20:44:17<21:58:31, 32.72s/it]
 48%|████▊     | 2273/4690 [20:44:50<21:59:04, 32.74s/it]
 48%|████▊     | 2274/4690 [20:45:22<21:57:25, 32.72s/it]
 49%|████▊     | 2275/4690 [20:45:55<21:56:32, 32.71s/it]
 49%|████▊     | 2276/4690 [20:46:27<21:50:24, 32.57s/it]
 49%|████▊     | 2277/4690 [20:47:00<21:54:04, 32.67s/it]
 49%|████▊     | 2278/4690 [20:47:33<21:52:18, 32.64s/it]
 49%|████▊     | 2279/4690 [20:48:06<21:59:06, 32.83s/it]
 49%|████▊     | 2280/4690 [20:48:39<21:58:19, 32.82s/it]
                                                         
{'loss': '0.09183', 'grad_norm': '1.969', 'learning_rate': '5.471e-06', 'epoch': '2.431'}

 49%|████▊     | 2280/4690 [20:48:39<21:58:19, 32.82s/it]
 49%|████▊     | 2281/4690 [20:49:12<21:58:17, 32.83s/it]
 49%|████▊     | 2282/4690 [20:49:44<21:56:22, 32.80s/it]
 49%|████▊     | 2283/4690 [20:50:17<21:55:09, 32.78s/it]
 49%|████▊     | 2284/4690 [20:50:50<21:56:38, 32.83s/it]
 49%|████▊     | 2285/4690 [20:51:23<21:56:06, 32.83s/it]
 49%|████▊     | 2286/4690 [20:51:56<21:56:09, 32.85s/it]
 49%|████▉     | 2287/4690 [20:52:29<22:01:24, 32.99s/it]
 49%|████▉     | 2288/4690 [20:53:02<22:04:23, 33.08s/it]
 49%|████▉     | 2289/4690 [20:53:35<22:00:26, 33.00s/it]
 49%|████▉     | 2290/4690 [20:54:08<21:57:26, 32.94s/it]
                                                         
{'loss': '0.1046', 'grad_norm': '1.965', 'learning_rate': '5.436e-06', 'epoch': '2.442'}

 49%|████▉     | 2290/4690 [20:54:08<21:57:26, 32.94s/it]
 49%|████▉     | 2291/4690 [20:54:41<21:54:01, 32.86s/it]
 49%|████▉     | 2292/4690 [20:55:14<21:53:35, 32.87s/it]
 49%|████▉     | 2293/4690 [20:55:46<21:50:49, 32.81s/it]
 49%|████▉     | 2294/4690 [20:56:19<21:48:28, 32.77s/it]
 49%|████▉     | 2295/4690 [20:56:52<21:48:24, 32.78s/it]
 49%|████▉     | 2296/4690 [20:57:25<21:50:23, 32.84s/it]
 49%|████▉     | 2297/4690 [20:57:58<21:50:29, 32.86s/it]
 49%|████▉     | 2298/4690 [20:58:30<21:47:34, 32.80s/it]
 49%|████▉     | 2299/4690 [20:59:03<21:48:22, 32.83s/it]
 49%|████▉     | 2300/4690 [20:59:36<21:46:46, 32.81s/it]
                                                         
{'loss': '0.09628', 'grad_norm': '1.909', 'learning_rate': '5.402e-06', 'epoch': '2.452'}

 49%|████▉     | 2300/4690 [20:59:36<21:46:46, 32.81s/it]
 49%|████▉     | 2301/4690 [21:00:08<21:43:14, 32.73s/it]
 49%|████▉     | 2302/4690 [21:00:41<21:42:12, 32.72s/it]
 49%|████▉     | 2303/4690 [21:01:14<21:45:02, 32.80s/it]
 49%|████▉     | 2304/4690 [21:01:47<21:44:21, 32.80s/it]
 49%|████▉     | 2305/4690 [21:02:20<21:45:42, 32.85s/it]
 49%|████▉     | 2306/4690 [21:02:53<21:46:56, 32.89s/it]
 49%|████▉     | 2307/4690 [21:03:26<21:47:24, 32.92s/it]
 49%|████▉     | 2308/4690 [21:03:59<21:50:01, 33.00s/it]
 49%|████▉     | 2309/4690 [21:04:32<21:53:39, 33.10s/it]
 49%|████▉     | 2310/4690 [21:05:06<21:52:32, 33.09s/it]
                                                         
{'loss': '0.1012', 'grad_norm': '1.856', 'learning_rate': '5.367e-06', 'epoch': '2.463'}

 49%|████▉     | 2310/4690 [21:05:06<21:52:32, 33.09s/it]
 49%|████▉     | 2311/4690 [21:05:38<21:49:01, 33.01s/it]
 49%|████▉     | 2312/4690 [21:06:11<21:45:28, 32.94s/it]
 49%|████▉     | 2313/4690 [21:06:43<21:37:37, 32.75s/it]
 49%|████▉     | 2314/4690 [21:07:16<21:39:08, 32.81s/it]
 49%|████▉     | 2315/4690 [21:07:49<21:39:44, 32.84s/it]
 49%|████▉     | 2316/4690 [21:08:22<21:39:00, 32.83s/it]
 49%|████▉     | 2317/4690 [21:08:55<21:35:13, 32.75s/it]
 49%|████▉     | 2318/4690 [21:09:27<21:30:39, 32.65s/it]
 49%|████▉     | 2319/4690 [21:10:00<21:31:10, 32.67s/it]
 49%|████▉     | 2320/4690 [21:10:33<21:31:12, 32.69s/it]
                                                         
{'loss': '0.09822', 'grad_norm': '1.98', 'learning_rate': '5.333e-06', 'epoch': '2.474'}

 49%|████▉     | 2320/4690 [21:10:33<21:31:12, 32.69s/it]
 49%|████▉     | 2321/4690 [21:11:05<21:31:19, 32.71s/it]
 50%|████▉     | 2322/4690 [21:11:38<21:27:44, 32.63s/it]
 50%|████▉     | 2323/4690 [21:12:11<21:36:54, 32.87s/it]
 50%|████▉     | 2324/4690 [21:12:44<21:36:46, 32.89s/it]
 50%|████▉     | 2325/4690 [21:13:17<21:32:27, 32.79s/it]
 50%|████▉     | 2326/4690 [21:13:50<21:33:40, 32.83s/it]
 50%|████▉     | 2327/4690 [21:14:22<21:28:02, 32.71s/it]
 50%|████▉     | 2328/4690 [21:14:55<21:26:07, 32.67s/it]
 50%|████▉     | 2329/4690 [21:15:28<21:28:56, 32.76s/it]
 50%|████▉     | 2330/4690 [21:16:00<21:28:49, 32.77s/it]
                                                         
{'loss': '0.1027', 'grad_norm': '1.98', 'learning_rate': '5.299e-06', 'epoch': '2.484'}

 50%|████▉     | 2330/4690 [21:16:00<21:28:49, 32.77s/it]
 50%|████▉     | 2331/4690 [21:16:34<21:35:07, 32.94s/it]
 50%|████▉     | 2332/4690 [21:17:07<21:36:05, 32.98s/it]
 50%|████▉     | 2333/4690 [21:17:39<21:33:06, 32.92s/it]
 50%|████▉     | 2334/4690 [21:18:12<21:32:46, 32.92s/it]
 50%|████▉     | 2335/4690 [21:18:45<21:32:23, 32.93s/it]
 50%|████▉     | 2336/4690 [21:19:18<21:32:02, 32.93s/it]
 50%|████▉     | 2337/4690 [21:19:51<21:32:27, 32.96s/it]
 50%|████▉     | 2338/4690 [21:20:24<21:27:07, 32.83s/it]
 50%|████▉     | 2339/4690 [21:20:56<21:23:14, 32.75s/it]
 50%|████▉     | 2340/4690 [21:21:29<21:23:38, 32.77s/it]
                                                         
{'loss': '0.09921', 'grad_norm': '1.869', 'learning_rate': '5.264e-06', 'epoch': '2.495'}

 50%|████▉     | 2340/4690 [21:21:29<21:23:38, 32.77s/it]
 50%|████▉     | 2341/4690 [21:22:03<21:30:17, 32.96s/it]
 50%|████▉     | 2342/4690 [21:22:36<21:29:51, 32.96s/it]
 50%|████▉     | 2343/4690 [21:23:09<21:28:53, 32.95s/it]
 50%|████▉     | 2344/4690 [21:23:41<21:21:45, 32.78s/it]
 50%|█████     | 2345/4690 [21:24:14<21:21:32, 32.79s/it]
 50%|█████     | 2346/4690 [21:24:47<21:26:36, 32.93s/it]
 50%|█████     | 2347/4690 [21:25:19<21:20:20, 32.79s/it]
 50%|█████     | 2348/4690 [21:25:52<21:15:57, 32.69s/it]
 50%|█████     | 2349/4690 [21:26:24<21:12:52, 32.62s/it]
 50%|█████     | 2350/4690 [21:26:57<21:13:17, 32.65s/it]
                                                         
{'loss': '0.09889', 'grad_norm': '2.202', 'learning_rate': '5.23e-06', 'epoch': '2.506'}

 50%|█████     | 2350/4690 [21:26:57<21:13:17, 32.65s/it]
 50%|█████     | 2351/4690 [21:27:30<21:15:47, 32.73s/it]
 50%|█████     | 2352/4690 [21:28:03<21:14:18, 32.70s/it]
 50%|█████     | 2353/4690 [21:28:35<21:15:31, 32.75s/it]
 50%|█████     | 2354/4690 [21:29:08<21:16:15, 32.78s/it]
 50%|█████     | 2355/4690 [21:29:42<21:20:27, 32.90s/it]
 50%|█████     | 2356/4690 [21:30:14<21:16:39, 32.82s/it]
 50%|█████     | 2357/4690 [21:30:47<21:13:05, 32.74s/it]
 50%|█████     | 2358/4690 [21:31:19<21:12:25, 32.74s/it]
 50%|█████     | 2359/4690 [21:31:53<21:17:10, 32.87s/it]
 50%|█████     | 2360/4690 [21:32:26<21:22:54, 33.04s/it]
                                                         
{'loss': '0.09742', 'grad_norm': '2.012', 'learning_rate': '5.195e-06', 'epoch': '2.516'}

 50%|█████     | 2360/4690 [21:32:26<21:22:54, 33.04s/it]
 50%|█████     | 2361/4690 [21:32:59<21:16:18, 32.88s/it]
 50%|█████     | 2362/4690 [21:33:31<21:14:44, 32.85s/it]
 50%|█████     | 2363/4690 [21:34:04<21:10:30, 32.76s/it]
 50%|█████     | 2364/4690 [21:34:37<21:09:09, 32.74s/it]
 50%|█████     | 2365/4690 [21:35:09<21:06:24, 32.68s/it]
 50%|█████     | 2366/4690 [21:35:42<21:06:13, 32.69s/it]
 50%|█████     | 2367/4690 [21:36:15<21:07:27, 32.74s/it]
 50%|█████     | 2368/4690 [21:36:48<21:08:58, 32.79s/it]
 51%|█████     | 2369/4690 [21:37:20<21:07:01, 32.75s/it]
 51%|█████     | 2370/4690 [21:37:53<21:05:06, 32.72s/it]
                                                         
{'loss': '0.09721', 'grad_norm': '2.03', 'learning_rate': '5.161e-06', 'epoch': '2.527'}

 51%|█████     | 2370/4690 [21:37:53<21:05:06, 32.72s/it]
 51%|█████     | 2371/4690 [21:38:25<21:02:34, 32.67s/it]
 51%|█████     | 2372/4690 [21:38:58<21:04:12, 32.72s/it]
 51%|█████     | 2373/4690 [21:39:31<21:01:03, 32.66s/it]
 51%|█████     | 2374/4690 [21:40:04<21:03:26, 32.73s/it]
 51%|█████     | 2375/4690 [21:40:37<21:09:04, 32.89s/it]
 51%|█████     | 2376/4690 [21:41:10<21:05:50, 32.82s/it]
 51%|█████     | 2377/4690 [21:41:42<20:59:33, 32.67s/it]
 51%|█████     | 2378/4690 [21:42:14<20:54:16, 32.55s/it]
 51%|█████     | 2379/4690 [21:42:47<20:56:38, 32.63s/it]
 51%|█████     | 2380/4690 [21:43:20<21:02:30, 32.79s/it]
                                                         
{'loss': '0.09673', 'grad_norm': '1.952', 'learning_rate': '5.126e-06', 'epoch': '2.538'}

 51%|█████     | 2380/4690 [21:43:20<21:02:30, 32.79s/it]
 51%|█████     | 2381/4690 [21:43:53<21:01:48, 32.79s/it]
 51%|█████     | 2382/4690 [21:44:26<21:01:03, 32.78s/it]
 51%|█████     | 2383/4690 [21:44:58<20:59:32, 32.76s/it]
 51%|█████     | 2384/4690 [21:45:31<20:56:52, 32.70s/it]
 51%|█████     | 2385/4690 [21:46:04<20:57:50, 32.74s/it]
 51%|█████     | 2386/4690 [21:46:37<20:59:26, 32.80s/it]
 51%|█████     | 2387/4690 [21:47:09<20:56:07, 32.73s/it]
 51%|█████     | 2388/4690 [21:47:42<20:54:27, 32.70s/it]
 51%|█████     | 2389/4690 [21:48:15<20:56:24, 32.76s/it]
 51%|█████     | 2390/4690 [21:48:47<20:52:04, 32.66s/it]
                                                         
{'loss': '0.09959', 'grad_norm': '2.078', 'learning_rate': '5.092e-06', 'epoch': '2.548'}

 51%|█████     | 2390/4690 [21:48:47<20:52:04, 32.66s/it]
 51%|█████     | 2391/4690 [21:49:20<20:55:46, 32.77s/it]
 51%|█████     | 2392/4690 [21:49:53<20:54:21, 32.75s/it]
 51%|█████     | 2393/4690 [21:50:26<20:50:09, 32.66s/it]
 51%|█████     | 2394/4690 [21:50:58<20:50:22, 32.68s/it]
 51%|█████     | 2395/4690 [21:51:31<20:50:14, 32.69s/it]
 51%|█████     | 2396/4690 [21:52:04<20:56:32, 32.87s/it]
 51%|█████     | 2397/4690 [21:52:37<20:49:40, 32.70s/it]
 51%|█████     | 2398/4690 [21:53:10<20:53:25, 32.81s/it]
 51%|█████     | 2399/4690 [21:53:43<20:56:37, 32.91s/it]
 51%|█████     | 2400/4690 [21:54:16<20:54:23, 32.87s/it]
                                                         
{'loss': '0.0975', 'grad_norm': '1.926', 'learning_rate': '5.057e-06', 'epoch': '2.559'}

 51%|█████     | 2400/4690 [21:54:16<20:54:23, 32.87s/it]
 51%|█████     | 2401/4690 [21:54:49<21:00:05, 33.03s/it]
 51%|█████     | 2402/4690 [21:55:22<20:56:39, 32.95s/it]
 51%|█████     | 2403/4690 [21:55:54<20:51:43, 32.84s/it]
 51%|█████▏    | 2404/4690 [21:56:27<20:46:31, 32.72s/it]
 51%|█████▏    | 2405/4690 [21:56:59<20:43:19, 32.65s/it]
 51%|█████▏    | 2406/4690 [21:57:32<20:39:44, 32.57s/it]
 51%|█████▏    | 2407/4690 [21:58:04<20:39:15, 32.57s/it]
 51%|█████▏    | 2408/4690 [21:58:37<20:43:04, 32.68s/it]
 51%|█████▏    | 2409/4690 [21:59:10<20:45:05, 32.75s/it]
 51%|█████▏    | 2410/4690 [21:59:43<20:48:55, 32.87s/it]
                                                         
{'loss': '0.1002', 'grad_norm': '2.074', 'learning_rate': '5.022e-06', 'epoch': '2.57'}

 51%|█████▏    | 2410/4690 [21:59:43<20:48:55, 32.87s/it]
 51%|█████▏    | 2411/4690 [22:00:16<20:51:03, 32.94s/it]
 51%|█████▏    | 2412/4690 [22:00:49<20:49:13, 32.90s/it]
 51%|█████▏    | 2413/4690 [22:01:22<20:52:32, 33.00s/it]
 51%|█████▏    | 2414/4690 [22:01:55<20:48:40, 32.92s/it]
 51%|█████▏    | 2415/4690 [22:02:28<20:44:11, 32.81s/it]
 52%|█████▏    | 2416/4690 [22:03:00<20:40:55, 32.74s/it]
 52%|█████▏    | 2417/4690 [22:03:33<20:39:19, 32.71s/it]
 52%|█████▏    | 2418/4690 [22:04:06<20:40:17, 32.75s/it]
 52%|█████▏    | 2419/4690 [22:04:38<20:38:26, 32.72s/it]
 52%|█████▏    | 2420/4690 [22:05:11<20:35:19, 32.65s/it]
                                                         
{'loss': '0.09448', 'grad_norm': '1.85', 'learning_rate': '4.988e-06', 'epoch': '2.58'}

 52%|█████▏    | 2420/4690 [22:05:11<20:35:19, 32.65s/it]
 52%|█████▏    | 2421/4690 [22:05:44<20:38:33, 32.75s/it]
 52%|█████▏    | 2422/4690 [22:06:17<20:38:59, 32.78s/it]
 52%|█████▏    | 2423/4690 [22:06:49<20:38:55, 32.79s/it]
 52%|█████▏    | 2424/4690 [22:07:22<20:37:26, 32.77s/it]
 52%|█████▏    | 2425/4690 [22:07:55<20:41:58, 32.90s/it]
 52%|█████▏    | 2426/4690 [22:08:28<20:38:49, 32.83s/it]
 52%|█████▏    | 2427/4690 [22:09:01<20:40:19, 32.89s/it]
 52%|█████▏    | 2428/4690 [22:09:34<20:38:07, 32.84s/it]
 52%|█████▏    | 2429/4690 [22:10:07<20:38:53, 32.88s/it]
 52%|█████▏    | 2430/4690 [22:10:40<20:38:01, 32.87s/it]
                                                         
{'loss': '0.09817', 'grad_norm': '2.022', 'learning_rate': '4.953e-06', 'epoch': '2.591'}

 52%|█████▏    | 2430/4690 [22:10:40<20:38:01, 32.87s/it]
 52%|█████▏    | 2431/4690 [22:11:12<20:37:30, 32.87s/it]
 52%|█████▏    | 2432/4690 [22:11:45<20:32:25, 32.75s/it]
 52%|█████▏    | 2433/4690 [22:12:18<20:32:27, 32.76s/it]
 52%|█████▏    | 2434/4690 [22:12:50<20:29:54, 32.71s/it]
 52%|█████▏    | 2435/4690 [22:13:23<20:28:57, 32.70s/it]
 52%|█████▏    | 2436/4690 [22:13:56<20:29:46, 32.74s/it]
 52%|█████▏    | 2437/4690 [22:14:29<20:33:18, 32.84s/it]
 52%|█████▏    | 2438/4690 [22:15:02<20:34:33, 32.89s/it]
 52%|█████▏    | 2439/4690 [22:15:35<20:31:25, 32.82s/it]
 52%|█████▏    | 2440/4690 [22:16:07<20:26:09, 32.70s/it]
                                                         
{'loss': '0.0954', 'grad_norm': '1.956', 'learning_rate': '4.919e-06', 'epoch': '2.602'}

 52%|█████▏    | 2440/4690 [22:16:07<20:26:09, 32.70s/it]
 52%|█████▏    | 2441/4690 [22:16:40<20:30:57, 32.84s/it]
 52%|█████▏    | 2442/4690 [22:17:13<20:33:21, 32.92s/it]
 52%|█████▏    | 2443/4690 [22:17:46<20:26:22, 32.75s/it]
 52%|█████▏    | 2444/4690 [22:18:19<20:29:02, 32.83s/it]
 52%|█████▏    | 2445/4690 [22:18:51<20:25:30, 32.75s/it]
 52%|█████▏    | 2446/4690 [22:19:24<20:25:42, 32.77s/it]
 52%|█████▏    | 2447/4690 [22:19:57<20:24:17, 32.75s/it]
 52%|█████▏    | 2448/4690 [22:20:29<20:21:43, 32.70s/it]
 52%|█████▏    | 2449/4690 [22:21:02<20:25:49, 32.82s/it]
 52%|█████▏    | 2450/4690 [22:21:35<20:23:38, 32.78s/it]
                                                         
{'loss': '0.09845', 'grad_norm': '2.167', 'learning_rate': '4.884e-06', 'epoch': '2.612'}

 52%|█████▏    | 2450/4690 [22:21:35<20:23:38, 32.78s/it]
 52%|█████▏    | 2451/4690 [22:22:08<20:22:21, 32.76s/it]
 52%|█████▏    | 2452/4690 [22:22:41<20:25:39, 32.86s/it]
 52%|█████▏    | 2453/4690 [22:23:14<20:25:05, 32.86s/it]
 52%|█████▏    | 2454/4690 [22:23:47<20:24:26, 32.86s/it]
 52%|█████▏    | 2455/4690 [22:24:19<20:20:51, 32.77s/it]
 52%|█████▏    | 2456/4690 [22:24:52<20:18:44, 32.73s/it]
 52%|█████▏    | 2457/4690 [22:25:24<20:15:35, 32.66s/it]
 52%|█████▏    | 2458/4690 [22:25:57<20:18:32, 32.76s/it]
 52%|█████▏    | 2459/4690 [22:26:30<20:15:18, 32.68s/it]
 52%|█████▏    | 2460/4690 [22:27:03<20:15:42, 32.71s/it]
                                                         
{'loss': '0.093', 'grad_norm': '1.812', 'learning_rate': '4.85e-06', 'epoch': '2.623'}

 52%|█████▏    | 2460/4690 [22:27:03<20:15:42, 32.71s/it]
 52%|█████▏    | 2461/4690 [22:27:35<20:14:00, 32.68s/it]
 52%|█████▏    | 2462/4690 [22:28:08<20:16:41, 32.77s/it]
 53%|█████▎    | 2463/4690 [22:28:41<20:19:22, 32.85s/it]
 53%|█████▎    | 2464/4690 [22:29:14<20:16:14, 32.78s/it]
 53%|█████▎    | 2465/4690 [22:29:47<20:15:42, 32.78s/it]
 53%|█████▎    | 2466/4690 [22:30:20<20:18:27, 32.87s/it]
 53%|█████▎    | 2467/4690 [22:30:52<20:12:09, 32.72s/it]
 53%|█████▎    | 2468/4690 [22:31:25<20:11:49, 32.72s/it]
 53%|█████▎    | 2469/4690 [22:31:58<20:13:25, 32.78s/it]
 53%|█████▎    | 2470/4690 [22:32:30<20:13:00, 32.78s/it]
                                                         
{'loss': '0.09923', 'grad_norm': '2.252', 'learning_rate': '4.815e-06', 'epoch': '2.634'}

 53%|█████▎    | 2470/4690 [22:32:30<20:13:00, 32.78s/it]
 53%|█████▎    | 2471/4690 [22:33:04<20:15:32, 32.87s/it]
 53%|█████▎    | 2472/4690 [22:33:36<20:15:14, 32.87s/it]
 53%|█████▎    | 2473/4690 [22:34:10<20:17:13, 32.94s/it]
 53%|█████▎    | 2474/4690 [22:34:43<20:21:18, 33.07s/it]
 53%|█████▎    | 2475/4690 [22:35:16<20:18:47, 33.01s/it]
 53%|█████▎    | 2476/4690 [22:35:48<20:13:34, 32.89s/it]
 53%|█████▎    | 2477/4690 [22:36:21<20:08:23, 32.76s/it]
 53%|█████▎    | 2478/4690 [22:36:54<20:08:51, 32.79s/it]
 53%|█████▎    | 2479/4690 [22:37:27<20:08:55, 32.81s/it]
 53%|█████▎    | 2480/4690 [22:38:00<20:11:32, 32.89s/it]
                                                         
{'loss': '0.09963', 'grad_norm': '1.656', 'learning_rate': '4.781e-06', 'epoch': '2.644'}

 53%|█████▎    | 2480/4690 [22:38:00<20:11:32, 32.89s/it]
 53%|█████▎    | 2481/4690 [22:38:32<20:07:55, 32.81s/it]
 53%|█████▎    | 2482/4690 [22:39:05<20:04:25, 32.73s/it]
 53%|█████▎    | 2483/4690 [22:39:38<20:06:46, 32.81s/it]
 53%|█████▎    | 2484/4690 [22:40:11<20:05:55, 32.80s/it]
 53%|█████▎    | 2485/4690 [22:40:43<20:06:11, 32.82s/it]
 53%|█████▎    | 2486/4690 [22:41:16<20:04:09, 32.78s/it]
 53%|█████▎    | 2487/4690 [22:41:49<20:08:39, 32.92s/it]
 53%|█████▎    | 2488/4690 [22:42:22<20:06:28, 32.87s/it]
 53%|█████▎    | 2489/4690 [22:42:55<20:03:39, 32.81s/it]
 53%|█████▎    | 2490/4690 [22:43:28<20:02:17, 32.79s/it]
                                                         
{'loss': '0.09291', 'grad_norm': '1.954', 'learning_rate': '4.746e-06', 'epoch': '2.655'}

 53%|█████▎    | 2490/4690 [22:43:28<20:02:17, 32.79s/it]
 53%|█████▎    | 2491/4690 [22:44:00<20:01:51, 32.79s/it]
 53%|█████▎    | 2492/4690 [22:44:33<19:58:54, 32.73s/it]
 53%|█████▎    | 2493/4690 [22:45:06<20:00:44, 32.79s/it]
 53%|█████▎    | 2494/4690 [22:45:39<19:59:03, 32.76s/it]
 53%|█████▎    | 2495/4690 [22:46:12<20:03:42, 32.90s/it]
 53%|█████▎    | 2496/4690 [22:46:44<19:58:38, 32.78s/it]
 53%|█████▎    | 2497/4690 [22:47:17<19:57:49, 32.77s/it]
 53%|█████▎    | 2498/4690 [22:47:50<19:56:25, 32.75s/it]
 53%|█████▎    | 2499/4690 [22:48:22<19:55:53, 32.75s/it]
 53%|█████▎    | 2500/4690 [22:48:56<19:59:02, 32.85s/it]
                                                         
{'loss': '0.1002', 'grad_norm': '1.843', 'learning_rate': '4.712e-06', 'epoch': '2.666'}

 53%|█████▎    | 2500/4690 [22:48:56<19:59:02, 32.85s/it]
 53%|█████▎    | 2501/4690 [22:49:29<20:01:12, 32.92s/it]
 53%|█████▎    | 2502/4690 [22:50:01<19:57:20, 32.83s/it]
 53%|█████▎    | 2503/4690 [22:50:34<19:57:18, 32.85s/it]
 53%|█████▎    | 2504/4690 [22:51:07<19:59:30, 32.92s/it]
 53%|█████▎    | 2505/4690 [22:51:40<19:55:21, 32.82s/it]
 53%|█████▎    | 2506/4690 [22:52:12<19:52:32, 32.76s/it]
 53%|█████▎    | 2507/4690 [22:52:45<19:53:49, 32.81s/it]
 53%|█████▎    | 2508/4690 [22:53:18<19:53:56, 32.83s/it]
 53%|█████▎    | 2509/4690 [22:53:51<19:57:19, 32.94s/it]
 54%|█████▎    | 2510/4690 [22:54:24<19:55:41, 32.91s/it]
                                                         
{'loss': '0.102', 'grad_norm': '1.986', 'learning_rate': '4.677e-06', 'epoch': '2.676'}

 54%|█████▎    | 2510/4690 [22:54:24<19:55:41, 32.91s/it]
 54%|█████▎    | 2511/4690 [22:54:57<19:56:10, 32.94s/it]
 54%|█████▎    | 2512/4690 [22:55:30<19:52:12, 32.84s/it]
 54%|█████▎    | 2513/4690 [22:56:03<19:49:27, 32.78s/it]
 54%|█████▎    | 2514/4690 [22:56:35<19:45:36, 32.69s/it]
 54%|█████▎    | 2515/4690 [22:57:08<19:46:39, 32.74s/it]
 54%|█████▎    | 2516/4690 [22:57:40<19:42:09, 32.63s/it]
 54%|█████▎    | 2517/4690 [22:58:13<19:42:56, 32.66s/it]
 54%|█████▎    | 2518/4690 [22:58:46<19:43:39, 32.70s/it]
 54%|█████▎    | 2519/4690 [22:59:18<19:42:51, 32.69s/it]
 54%|█████▎    | 2520/4690 [22:59:51<19:42:22, 32.69s/it]
                                                         
{'loss': '0.0986', 'grad_norm': '1.758', 'learning_rate': '4.643e-06', 'epoch': '2.687'}

 54%|█████▎    | 2520/4690 [22:59:51<19:42:22, 32.69s/it]
 54%|█████▍    | 2521/4690 [23:00:24<19:42:10, 32.70s/it]
 54%|█████▍    | 2522/4690 [23:00:56<19:40:34, 32.67s/it]
 54%|█████▍    | 2523/4690 [23:01:29<19:43:14, 32.76s/it]
 54%|█████▍    | 2524/4690 [23:02:02<19:42:40, 32.76s/it]
 54%|█████▍    | 2525/4690 [23:02:35<19:40:05, 32.70s/it]
 54%|█████▍    | 2526/4690 [23:03:07<19:35:59, 32.61s/it]
 54%|█████▍    | 2527/4690 [23:03:40<19:37:52, 32.67s/it]
 54%|█████▍    | 2528/4690 [23:04:13<19:35:50, 32.63s/it]
 54%|█████▍    | 2529/4690 [23:04:46<19:40:50, 32.79s/it]
 54%|█████▍    | 2530/4690 [23:05:18<19:36:05, 32.67s/it]
                                                         
{'loss': '0.09404', 'grad_norm': '2.331', 'learning_rate': '4.608e-06', 'epoch': '2.698'}

 54%|█████▍    | 2530/4690 [23:05:18<19:36:05, 32.67s/it]
 54%|█████▍    | 2531/4690 [23:05:51<19:39:35, 32.78s/it]
 54%|█████▍    | 2532/4690 [23:06:24<19:35:48, 32.69s/it]
 54%|█████▍    | 2533/4690 [23:06:56<19:34:04, 32.66s/it]
 54%|█████▍    | 2534/4690 [23:07:29<19:36:31, 32.74s/it]
 54%|█████▍    | 2535/4690 [23:08:02<19:39:28, 32.84s/it]
 54%|█████▍    | 2536/4690 [23:08:35<19:41:17, 32.90s/it]
 54%|█████▍    | 2537/4690 [23:09:08<19:43:52, 32.99s/it]
 54%|█████▍    | 2538/4690 [23:09:41<19:35:09, 32.76s/it]
 54%|█████▍    | 2539/4690 [23:10:14<19:36:19, 32.81s/it]
 54%|█████▍    | 2540/4690 [23:10:46<19:32:29, 32.72s/it]
                                                         
{'loss': '0.09787', 'grad_norm': '1.806', 'learning_rate': '4.574e-06', 'epoch': '2.708'}

 54%|█████▍    | 2540/4690 [23:10:46<19:32:29, 32.72s/it]
 54%|█████▍    | 2541/4690 [23:11:19<19:34:08, 32.78s/it]
 54%|█████▍    | 2542/4690 [23:11:52<19:32:04, 32.74s/it]
 54%|█████▍    | 2543/4690 [23:12:25<19:34:45, 32.83s/it]
 54%|█████▍    | 2544/4690 [23:12:58<19:34:48, 32.85s/it]
 54%|█████▍    | 2545/4690 [23:13:31<19:36:26, 32.91s/it]
 54%|█████▍    | 2546/4690 [23:14:04<19:37:23, 32.95s/it]
 54%|█████▍    | 2547/4690 [23:14:36<19:31:43, 32.81s/it]
 54%|█████▍    | 2548/4690 [23:15:09<19:33:02, 32.86s/it]
 54%|█████▍    | 2549/4690 [23:15:42<19:30:50, 32.81s/it]
 54%|█████▍    | 2550/4690 [23:16:15<19:30:40, 32.82s/it]
                                                         
{'loss': '0.09959', 'grad_norm': '1.782', 'learning_rate': '4.54e-06', 'epoch': '2.719'}

 54%|█████▍    | 2550/4690 [23:16:15<19:30:40, 32.82s/it]
 54%|█████▍    | 2551/4690 [23:16:47<19:26:06, 32.71s/it]
 54%|█████▍    | 2552/4690 [23:17:21<19:33:04, 32.92s/it]
 54%|█████▍    | 2553/4690 [23:17:53<19:30:40, 32.87s/it]
 54%|█████▍    | 2554/4690 [23:18:26<19:29:46, 32.86s/it]
 54%|█████▍    | 2555/4690 [23:18:59<19:29:53, 32.88s/it]
 54%|█████▍    | 2556/4690 [23:19:32<19:26:48, 32.81s/it]
 55%|█████▍    | 2557/4690 [23:20:04<19:23:33, 32.73s/it]
 55%|█████▍    | 2558/4690 [23:20:37<19:22:56, 32.73s/it]
 55%|█████▍    | 2559/4690 [23:21:10<19:22:59, 32.75s/it]
 55%|█████▍    | 2560/4690 [23:21:43<19:25:57, 32.84s/it]
                                                         
{'loss': '0.09491', 'grad_norm': '2.123', 'learning_rate': '4.505e-06', 'epoch': '2.73'}

 55%|█████▍    | 2560/4690 [23:21:43<19:25:57, 32.84s/it]
 55%|█████▍    | 2561/4690 [23:22:16<19:29:48, 32.97s/it]
 55%|█████▍    | 2562/4690 [23:22:49<19:28:46, 32.95s/it]
 55%|█████▍    | 2563/4690 [23:23:22<19:24:14, 32.84s/it]
 55%|█████▍    | 2564/4690 [23:23:54<19:20:02, 32.74s/it]
 55%|█████▍    | 2565/4690 [23:24:27<19:23:04, 32.84s/it]
 55%|█████▍    | 2566/4690 [23:25:00<19:20:58, 32.80s/it]
 55%|█████▍    | 2567/4690 [23:25:32<19:18:37, 32.74s/it]
 55%|█████▍    | 2568/4690 [23:26:05<19:19:46, 32.79s/it]
 55%|█████▍    | 2569/4690 [23:26:38<19:19:54, 32.81s/it]
 55%|█████▍    | 2570/4690 [23:27:11<19:19:57, 32.83s/it]
                                                         
{'loss': '0.09844', 'grad_norm': '2.063', 'learning_rate': '4.471e-06', 'epoch': '2.74'}

 55%|█████▍    | 2570/4690 [23:27:11<19:19:57, 32.83s/it]
 55%|█████▍    | 2571/4690 [23:27:44<19:15:07, 32.71s/it]
 55%|█████▍    | 2572/4690 [23:28:16<19:14:04, 32.69s/it]
 55%|█████▍    | 2573/4690 [23:28:49<19:11:51, 32.65s/it]
 55%|█████▍    | 2574/4690 [23:29:21<19:11:51, 32.66s/it]
 55%|█████▍    | 2575/4690 [23:29:54<19:11:31, 32.67s/it]
 55%|█████▍    | 2576/4690 [23:30:27<19:12:15, 32.70s/it]
 55%|█████▍    | 2577/4690 [23:31:00<19:11:13, 32.69s/it]
 55%|█████▍    | 2578/4690 [23:31:32<19:12:21, 32.74s/it]
 55%|█████▍    | 2579/4690 [23:32:05<19:14:53, 32.82s/it]
 55%|█████▌    | 2580/4690 [23:32:38<19:13:22, 32.80s/it]
                                                         
{'loss': '0.09282', 'grad_norm': '2.07', 'learning_rate': '4.437e-06', 'epoch': '2.751'}

 55%|█████▌    | 2580/4690 [23:32:38<19:13:22, 32.80s/it]
 55%|█████▌    | 2581/4690 [23:33:11<19:13:32, 32.82s/it]
 55%|█████▌    | 2582/4690 [23:33:44<19:12:15, 32.80s/it]
 55%|█████▌    | 2583/4690 [23:34:17<19:11:28, 32.79s/it]
 55%|█████▌    | 2584/4690 [23:34:50<19:15:14, 32.91s/it]
 55%|█████▌    | 2585/4690 [23:35:23<19:14:16, 32.90s/it]
 55%|█████▌    | 2586/4690 [23:35:56<19:15:25, 32.95s/it]
 55%|█████▌    | 2587/4690 [23:36:28<19:08:55, 32.78s/it]
 55%|█████▌    | 2588/4690 [23:37:01<19:08:10, 32.77s/it]
 55%|█████▌    | 2589/4690 [23:37:33<19:05:45, 32.72s/it]
 55%|█████▌    | 2590/4690 [23:38:06<19:05:49, 32.74s/it]
                                                         
{'loss': '0.09413', 'grad_norm': '1.911', 'learning_rate': '4.402e-06', 'epoch': '2.762'}

 55%|█████▌    | 2590/4690 [23:38:06<19:05:49, 32.74s/it]
 55%|█████▌    | 2591/4690 [23:38:39<19:06:17, 32.77s/it]
 55%|█████▌    | 2592/4690 [23:39:12<19:06:29, 32.79s/it]
 55%|█████▌    | 2593/4690 [23:39:45<19:05:59, 32.79s/it]
 55%|█████▌    | 2594/4690 [23:40:17<19:01:44, 32.68s/it]
 55%|█████▌    | 2595/4690 [23:40:50<19:03:09, 32.74s/it]
 55%|█████▌    | 2596/4690 [23:41:23<19:07:22, 32.88s/it]
 55%|█████▌    | 2597/4690 [23:41:56<19:07:32, 32.90s/it]
 55%|█████▌    | 2598/4690 [23:42:29<19:10:42, 33.00s/it]
 55%|█████▌    | 2599/4690 [23:43:03<19:12:15, 33.06s/it]
 55%|█████▌    | 2600/4690 [23:43:35<19:06:44, 32.92s/it]
                                                         
{'loss': '0.09814', 'grad_norm': '1.971', 'learning_rate': '4.368e-06', 'epoch': '2.772'}

 55%|█████▌    | 2600/4690 [23:43:35<19:06:44, 32.92s/it]
 55%|█████▌    | 2601/4690 [23:44:08<19:03:04, 32.83s/it]
 55%|█████▌    | 2602/4690 [23:44:41<19:06:20, 32.94s/it]
 56%|█████▌    | 2603/4690 [23:45:14<19:05:16, 32.93s/it]
 56%|█████▌    | 2604/4690 [23:45:47<19:05:02, 32.93s/it]
 56%|█████▌    | 2605/4690 [23:46:20<19:02:46, 32.89s/it]
 56%|█████▌    | 2606/4690 [23:46:52<18:58:45, 32.79s/it]
 56%|█████▌    | 2607/4690 [23:47:25<19:00:21, 32.85s/it]
 56%|█████▌    | 2608/4690 [23:47:58<18:55:04, 32.71s/it]
 56%|█████▌    | 2609/4690 [23:48:30<18:56:31, 32.77s/it]
 56%|█████▌    | 2610/4690 [23:49:03<18:57:22, 32.81s/it]
                                                         
{'loss': '0.09452', 'grad_norm': '2.042', 'learning_rate': '4.334e-06', 'epoch': '2.783'}

 56%|█████▌    | 2610/4690 [23:49:03<18:57:22, 32.81s/it]
 56%|█████▌    | 2611/4690 [23:49:36<18:59:24, 32.88s/it]
 56%|█████▌    | 2612/4690 [23:50:09<18:55:49, 32.80s/it]
 56%|█████▌    | 2613/4690 [23:50:41<18:51:23, 32.68s/it]
 56%|█████▌    | 2614/4690 [23:51:14<18:53:01, 32.75s/it]
 56%|█████▌    | 2615/4690 [23:51:47<18:55:36, 32.84s/it]
 56%|█████▌    | 2616/4690 [23:52:20<18:56:40, 32.88s/it]
 56%|█████▌    | 2617/4690 [23:52:53<18:57:04, 32.91s/it]
 56%|█████▌    | 2618/4690 [23:53:26<18:55:45, 32.89s/it]
 56%|█████▌    | 2619/4690 [23:53:59<18:56:07, 32.92s/it]
 56%|█████▌    | 2620/4690 [23:54:32<18:57:17, 32.96s/it]
                                                         
{'loss': '0.102', 'grad_norm': '1.93', 'learning_rate': '4.3e-06', 'epoch': '2.794'}

 56%|█████▌    | 2620/4690 [23:54:32<18:57:17, 32.96s/it]
 56%|█████▌    | 2621/4690 [23:55:05<18:52:46, 32.85s/it]
 56%|█████▌    | 2622/4690 [23:55:37<18:49:03, 32.76s/it]
 56%|█████▌    | 2623/4690 [23:56:10<18:46:50, 32.71s/it]
 56%|█████▌    | 2624/4690 [23:56:43<18:46:20, 32.71s/it]
 56%|█████▌    | 2625/4690 [23:57:16<18:48:35, 32.79s/it]
 56%|█████▌    | 2626/4690 [23:57:48<18:45:22, 32.71s/it]
 56%|█████▌    | 2627/4690 [23:58:21<18:42:06, 32.64s/it]
 56%|█████▌    | 2628/4690 [23:58:54<18:47:35, 32.81s/it]
 56%|█████▌    | 2629/4690 [23:59:27<18:47:45, 32.83s/it]
 56%|█████▌    | 2630/4690 [24:00:00<18:47:57, 32.85s/it]
                                                         
{'loss': '0.09719', 'grad_norm': '2.108', 'learning_rate': '4.265e-06', 'epoch': '2.804'}

 56%|█████▌    | 2630/4690 [24:00:00<18:47:57, 32.85s/it]
 56%|█████▌    | 2631/4690 [24:00:32<18:47:27, 32.85s/it]
 56%|█████▌    | 2632/4690 [24:01:06<18:49:41, 32.94s/it]
 56%|█████▌    | 2633/4690 [24:01:38<18:47:03, 32.87s/it]
 56%|█████▌    | 2634/4690 [24:02:11<18:44:01, 32.80s/it]
 56%|█████▌    | 2635/4690 [24:02:44<18:41:00, 32.73s/it]
 56%|█████▌    | 2636/4690 [24:03:16<18:39:28, 32.70s/it]
 56%|█████▌    | 2637/4690 [24:03:49<18:37:46, 32.67s/it]
 56%|█████▌    | 2638/4690 [24:04:21<18:34:08, 32.58s/it]
 56%|█████▋    | 2639/4690 [24:04:54<18:35:50, 32.64s/it]
 56%|█████▋    | 2640/4690 [24:05:27<18:35:45, 32.66s/it]
                                                         
{'loss': '0.09019', 'grad_norm': '1.9', 'learning_rate': '4.231e-06', 'epoch': '2.815'}

 56%|█████▋    | 2640/4690 [24:05:27<18:35:45, 32.66s/it]
 56%|█████▋    | 2641/4690 [24:05:59<18:37:06, 32.71s/it]
 56%|█████▋    | 2642/4690 [24:06:32<18:37:39, 32.74s/it]
 56%|█████▋    | 2643/4690 [24:07:05<18:37:22, 32.75s/it]
 56%|█████▋    | 2644/4690 [24:07:38<18:34:47, 32.69s/it]
 56%|█████▋    | 2645/4690 [24:08:10<18:34:40, 32.70s/it]
 56%|█████▋    | 2646/4690 [24:08:43<18:37:44, 32.81s/it]
 56%|█████▋    | 2647/4690 [24:09:16<18:35:30, 32.76s/it]
 56%|█████▋    | 2648/4690 [24:09:49<18:35:41, 32.78s/it]
 56%|█████▋    | 2649/4690 [24:10:22<18:35:09, 32.78s/it]
 57%|█████▋    | 2650/4690 [24:10:54<18:34:43, 32.79s/it]
                                                         
{'loss': '0.09156', 'grad_norm': '2.005', 'learning_rate': '4.197e-06', 'epoch': '2.826'}

 57%|█████▋    | 2650/4690 [24:10:54<18:34:43, 32.79s/it]
 57%|█████▋    | 2651/4690 [24:11:27<18:33:04, 32.75s/it]
 57%|█████▋    | 2652/4690 [24:12:00<18:32:57, 32.77s/it]
 57%|█████▋    | 2653/4690 [24:12:33<18:35:11, 32.85s/it]
 57%|█████▋    | 2654/4690 [24:13:05<18:31:25, 32.75s/it]
 57%|█████▋    | 2655/4690 [24:13:39<18:33:54, 32.84s/it]
 57%|█████▋    | 2656/4690 [24:14:12<18:35:22, 32.90s/it]
 57%|█████▋    | 2657/4690 [24:14:44<18:31:10, 32.79s/it]
 57%|█████▋    | 2658/4690 [24:15:17<18:31:23, 32.82s/it]
 57%|█████▋    | 2659/4690 [24:15:50<18:34:12, 32.92s/it]
 57%|█████▋    | 2660/4690 [24:16:23<18:31:18, 32.85s/it]
                                                         
{'loss': '0.09814', 'grad_norm': '1.943', 'learning_rate': '4.163e-06', 'epoch': '2.836'}

 57%|█████▋    | 2660/4690 [24:16:23<18:31:18, 32.85s/it]
 57%|█████▋    | 2661/4690 [24:16:55<18:27:18, 32.74s/it]
 57%|█████▋    | 2662/4690 [24:17:28<18:29:36, 32.83s/it]
 57%|█████▋    | 2663/4690 [24:18:01<18:31:37, 32.90s/it]
 57%|█████▋    | 2664/4690 [24:18:34<18:28:43, 32.83s/it]
 57%|█████▋    | 2665/4690 [24:19:07<18:29:03, 32.86s/it]
 57%|█████▋    | 2666/4690 [24:19:40<18:28:49, 32.87s/it]
 57%|█████▋    | 2667/4690 [24:20:13<18:30:07, 32.92s/it]
 57%|█████▋    | 2668/4690 [24:20:46<18:25:53, 32.82s/it]
 57%|█████▋    | 2669/4690 [24:21:19<18:30:16, 32.96s/it]
 57%|█████▋    | 2670/4690 [24:21:52<18:29:09, 32.95s/it]
                                                         
{'loss': '0.0996', 'grad_norm': '1.973', 'learning_rate': '4.129e-06', 'epoch': '2.847'}

 57%|█████▋    | 2670/4690 [24:21:52<18:29:09, 32.95s/it]
 57%|█████▋    | 2671/4690 [24:22:24<18:23:53, 32.80s/it]
 57%|█████▋    | 2672/4690 [24:22:57<18:25:20, 32.86s/it]
 57%|█████▋    | 2673/4690 [24:23:30<18:24:29, 32.86s/it]
 57%|█████▋    | 2674/4690 [24:24:03<18:23:09, 32.83s/it]
 57%|█████▋    | 2675/4690 [24:24:35<18:20:39, 32.77s/it]
 57%|█████▋    | 2676/4690 [24:25:09<18:23:08, 32.86s/it]
 57%|█████▋    | 2677/4690 [24:25:41<18:19:53, 32.78s/it]
 57%|█████▋    | 2678/4690 [24:26:14<18:17:19, 32.72s/it]
 57%|█████▋    | 2679/4690 [24:26:46<18:17:13, 32.74s/it]
 57%|█████▋    | 2680/4690 [24:27:19<18:16:31, 32.73s/it]
                                                         
{'loss': '0.101', 'grad_norm': '2.152', 'learning_rate': '4.095e-06', 'epoch': '2.858'}

 57%|█████▋    | 2680/4690 [24:27:19<18:16:31, 32.73s/it]
 57%|█████▋    | 2681/4690 [24:27:52<18:16:08, 32.74s/it]
 57%|█████▋    | 2682/4690 [24:28:25<18:16:22, 32.76s/it]
 57%|█████▋    | 2683/4690 [24:28:58<18:17:24, 32.81s/it]
 57%|█████▋    | 2684/4690 [24:29:30<18:14:31, 32.74s/it]
 57%|█████▋    | 2685/4690 [24:30:03<18:10:28, 32.63s/it]
 57%|█████▋    | 2686/4690 [24:30:36<18:12:51, 32.72s/it]
 57%|█████▋    | 2687/4690 [24:31:08<18:11:00, 32.68s/it]
 57%|█████▋    | 2688/4690 [24:31:41<18:11:21, 32.71s/it]
 57%|█████▋    | 2689/4690 [24:32:14<18:12:58, 32.77s/it]
 57%|█████▋    | 2690/4690 [24:32:47<18:11:32, 32.75s/it]
                                                         
{'loss': '0.09429', 'grad_norm': '2.106', 'learning_rate': '4.061e-06', 'epoch': '2.868'}

 57%|█████▋    | 2690/4690 [24:32:47<18:11:32, 32.75s/it]
 57%|█████▋    | 2691/4690 [24:33:19<18:12:35, 32.79s/it]
 57%|█████▋    | 2692/4690 [24:33:52<18:09:20, 32.71s/it]
 57%|█████▋    | 2693/4690 [24:34:24<18:06:25, 32.64s/it]
 57%|█████▋    | 2694/4690 [24:34:58<18:10:19, 32.78s/it]
 57%|█████▋    | 2695/4690 [24:35:30<18:10:22, 32.79s/it]
 57%|█████▋    | 2696/4690 [24:36:03<18:11:57, 32.86s/it]
 58%|█████▊    | 2697/4690 [24:36:36<18:10:36, 32.83s/it]
 58%|█████▊    | 2698/4690 [24:37:09<18:12:12, 32.90s/it]
 58%|█████▊    | 2699/4690 [24:37:42<18:11:23, 32.89s/it]
 58%|█████▊    | 2700/4690 [24:38:15<18:08:56, 32.83s/it]
                                                         
{'loss': '0.09709', 'grad_norm': '1.731', 'learning_rate': '4.027e-06', 'epoch': '2.879'}

 58%|█████▊    | 2700/4690 [24:38:15<18:08:56, 32.83s/it]
 58%|█████▊    | 2701/4690 [24:38:48<18:08:59, 32.85s/it]
 58%|█████▊    | 2702/4690 [24:39:20<18:05:45, 32.77s/it]
 58%|█████▊    | 2703/4690 [24:39:53<18:04:48, 32.76s/it]
 58%|█████▊    | 2704/4690 [24:40:26<18:08:16, 32.88s/it]
 58%|█████▊    | 2705/4690 [24:40:59<18:11:18, 32.99s/it]
 58%|█████▊    | 2706/4690 [24:41:32<18:09:31, 32.95s/it]
 58%|█████▊    | 2707/4690 [24:42:05<18:06:07, 32.86s/it]
 58%|█████▊    | 2708/4690 [24:42:38<18:06:46, 32.90s/it]
 58%|█████▊    | 2709/4690 [24:43:11<18:05:51, 32.89s/it]
 58%|█████▊    | 2710/4690 [24:43:44<18:05:48, 32.90s/it]
                                                         
{'loss': '0.09912', 'grad_norm': '1.809', 'learning_rate': '3.993e-06', 'epoch': '2.89'}

 58%|█████▊    | 2710/4690 [24:43:44<18:05:48, 32.90s/it]
 58%|█████▊    | 2711/4690 [24:44:17<18:06:20, 32.94s/it]
 58%|█████▊    | 2712/4690 [24:44:49<18:02:33, 32.84s/it]
 58%|█████▊    | 2713/4690 [24:45:22<18:00:41, 32.80s/it]
 58%|█████▊    | 2714/4690 [24:45:54<17:56:01, 32.67s/it]
 58%|█████▊    | 2715/4690 [24:46:27<17:55:43, 32.68s/it]
 58%|█████▊    | 2716/4690 [24:47:00<17:52:38, 32.60s/it]
 58%|█████▊    | 2717/4690 [24:47:32<17:54:22, 32.67s/it]
 58%|█████▊    | 2718/4690 [24:48:05<17:54:29, 32.69s/it]
 58%|█████▊    | 2719/4690 [24:48:38<17:56:30, 32.77s/it]
 58%|█████▊    | 2720/4690 [24:49:11<17:56:51, 32.80s/it]
                                                         
{'loss': '0.09605', 'grad_norm': '1.972', 'learning_rate': '3.96e-06', 'epoch': '2.9'}

 58%|█████▊    | 2720/4690 [24:49:11<17:56:51, 32.80s/it]
 58%|█████▊    | 2721/4690 [24:49:44<17:56:49, 32.81s/it]
 58%|█████▊    | 2722/4690 [24:50:17<18:00:07, 32.93s/it]
 58%|█████▊    | 2723/4690 [24:50:50<18:02:15, 33.01s/it]
 58%|█████▊    | 2724/4690 [24:51:23<17:59:39, 32.95s/it]
 58%|█████▊    | 2725/4690 [24:51:56<17:57:12, 32.89s/it]
 58%|█████▊    | 2726/4690 [24:52:28<17:53:46, 32.80s/it]
 58%|█████▊    | 2727/4690 [24:53:01<17:49:30, 32.69s/it]
 58%|█████▊    | 2728/4690 [24:53:33<17:47:51, 32.66s/it]
 58%|█████▊    | 2729/4690 [24:54:06<17:43:57, 32.55s/it]
 58%|█████▊    | 2730/4690 [24:54:38<17:45:19, 32.61s/it]
                                                         
{'loss': '0.09273', 'grad_norm': '1.888', 'learning_rate': '3.926e-06', 'epoch': '2.911'}

 58%|█████▊    | 2730/4690 [24:54:38<17:45:19, 32.61s/it]
 58%|█████▊    | 2731/4690 [24:55:11<17:46:57, 32.68s/it]
 58%|█████▊    | 2732/4690 [24:55:44<17:44:06, 32.61s/it]
 58%|█████▊    | 2733/4690 [24:56:16<17:43:33, 32.61s/it]
 58%|█████▊    | 2734/4690 [24:56:49<17:48:14, 32.77s/it]
 58%|█████▊    | 2735/4690 [24:57:22<17:48:14, 32.79s/it]
 58%|█████▊    | 2736/4690 [24:57:55<17:47:44, 32.79s/it]
 58%|█████▊    | 2737/4690 [24:58:28<17:45:19, 32.73s/it]
 58%|█████▊    | 2738/4690 [24:59:00<17:46:01, 32.77s/it]
 58%|█████▊    | 2739/4690 [24:59:33<17:39:52, 32.59s/it]
 58%|█████▊    | 2740/4690 [25:00:06<17:42:01, 32.68s/it]
                                                         
{'loss': '0.0927', 'grad_norm': '2.03', 'learning_rate': '3.892e-06', 'epoch': '2.922'}

 58%|█████▊    | 2740/4690 [25:00:06<17:42:01, 32.68s/it]
 58%|█████▊    | 2741/4690 [25:00:38<17:42:34, 32.71s/it]
 58%|█████▊    | 2742/4690 [25:01:12<17:47:28, 32.88s/it]
 58%|█████▊    | 2743/4690 [25:01:44<17:43:15, 32.77s/it]
 59%|█████▊    | 2744/4690 [25:02:17<17:44:16, 32.81s/it]
 59%|█████▊    | 2745/4690 [25:02:50<17:40:43, 32.72s/it]
 59%|█████▊    | 2746/4690 [25:03:22<17:40:17, 32.73s/it]
 59%|█████▊    | 2747/4690 [25:03:55<17:41:36, 32.78s/it]
 59%|█████▊    | 2748/4690 [25:04:28<17:42:07, 32.82s/it]
 59%|█████▊    | 2749/4690 [25:05:01<17:42:06, 32.83s/it]
 59%|█████▊    | 2750/4690 [25:05:34<17:42:34, 32.86s/it]
                                                         
{'loss': '0.09962', 'grad_norm': '2.08', 'learning_rate': '3.859e-06', 'epoch': '2.932'}

 59%|█████▊    | 2750/4690 [25:05:34<17:42:34, 32.86s/it]
 59%|█████▊    | 2751/4690 [25:06:06<17:38:37, 32.76s/it]
 59%|█████▊    | 2752/4690 [25:06:39<17:38:27, 32.77s/it]
 59%|█████▊    | 2753/4690 [25:07:12<17:40:21, 32.85s/it]
 59%|█████▊    | 2754/4690 [25:07:45<17:40:08, 32.86s/it]
 59%|█████▊    | 2755/4690 [25:08:18<17:38:13, 32.81s/it]
 59%|█████▉    | 2756/4690 [25:08:50<17:33:52, 32.70s/it]
 59%|█████▉    | 2757/4690 [25:09:23<17:32:26, 32.67s/it]
 59%|█████▉    | 2758/4690 [25:09:56<17:32:36, 32.69s/it]
 59%|█████▉    | 2759/4690 [25:10:28<17:30:48, 32.65s/it]
 59%|█████▉    | 2760/4690 [25:11:01<17:31:26, 32.69s/it]
                                                         
{'loss': '0.0934', 'grad_norm': '2.138', 'learning_rate': '3.825e-06', 'epoch': '2.943'}

 59%|█████▉    | 2760/4690 [25:11:01<17:31:26, 32.69s/it]
 59%|█████▉    | 2761/4690 [25:11:33<17:28:59, 32.63s/it]
 59%|█████▉    | 2762/4690 [25:12:06<17:29:53, 32.67s/it]
 59%|█████▉    | 2763/4690 [25:12:39<17:29:07, 32.67s/it]
 59%|█████▉    | 2764/4690 [25:13:12<17:32:14, 32.78s/it]
 59%|█████▉    | 2765/4690 [25:13:45<17:30:41, 32.75s/it]
 59%|█████▉    | 2766/4690 [25:14:17<17:27:58, 32.68s/it]
 59%|█████▉    | 2767/4690 [25:14:50<17:27:37, 32.69s/it]
 59%|█████▉    | 2768/4690 [25:15:23<17:29:37, 32.77s/it]
 59%|█████▉    | 2769/4690 [25:15:55<17:28:18, 32.74s/it]
 59%|█████▉    | 2770/4690 [25:16:28<17:27:31, 32.74s/it]
                                                         
{'loss': '0.0997', 'grad_norm': '1.873', 'learning_rate': '3.791e-06', 'epoch': '2.954'}

 59%|█████▉    | 2770/4690 [25:16:28<17:27:31, 32.74s/it]
 59%|█████▉    | 2771/4690 [25:17:00<17:21:37, 32.57s/it]
 59%|█████▉    | 2772/4690 [25:17:33<17:24:18, 32.67s/it]
 59%|█████▉    | 2773/4690 [25:18:06<17:22:46, 32.64s/it]
 59%|█████▉    | 2774/4690 [25:18:38<17:20:13, 32.57s/it]
 59%|█████▉    | 2775/4690 [25:19:11<17:20:03, 32.59s/it]
 59%|█████▉    | 2776/4690 [25:19:44<17:21:55, 32.66s/it]
 59%|█████▉    | 2777/4690 [25:20:17<17:26:22, 32.82s/it]
 59%|█████▉    | 2778/4690 [25:20:50<17:26:23, 32.84s/it]
 59%|█████▉    | 2779/4690 [25:21:22<17:25:09, 32.81s/it]
 59%|█████▉    | 2780/4690 [25:21:55<17:23:51, 32.79s/it]
                                                         
{'loss': '0.08675', 'grad_norm': '1.827', 'learning_rate': '3.758e-06', 'epoch': '2.964'}

 59%|█████▉    | 2780/4690 [25:21:55<17:23:51, 32.79s/it]
 59%|█████▉    | 2781/4690 [25:22:28<17:24:19, 32.82s/it]
 59%|█████▉    | 2782/4690 [25:23:01<17:26:06, 32.90s/it]
 59%|█████▉    | 2783/4690 [25:23:34<17:25:20, 32.89s/it]
 59%|█████▉    | 2784/4690 [25:24:07<17:23:25, 32.85s/it]
 59%|█████▉    | 2785/4690 [25:24:40<17:23:49, 32.88s/it]
 59%|█████▉    | 2786/4690 [25:25:12<17:20:29, 32.79s/it]
 59%|█████▉    | 2787/4690 [25:25:45<17:21:21, 32.83s/it]
 59%|█████▉    | 2788/4690 [25:26:18<17:17:06, 32.72s/it]
 59%|█████▉    | 2789/4690 [25:26:50<17:16:42, 32.72s/it]
 59%|█████▉    | 2790/4690 [25:27:23<17:14:13, 32.66s/it]
                                                         
{'loss': '0.09663', 'grad_norm': '2.095', 'learning_rate': '3.725e-06', 'epoch': '2.975'}

 59%|█████▉    | 2790/4690 [25:27:23<17:14:13, 32.66s/it]
 60%|█████▉    | 2791/4690 [25:27:56<17:18:13, 32.80s/it]
 60%|█████▉    | 2792/4690 [25:28:29<17:19:09, 32.85s/it]
 60%|█████▉    | 2793/4690 [25:29:02<17:17:17, 32.81s/it]
 60%|█████▉    | 2794/4690 [25:29:34<17:12:54, 32.69s/it]
 60%|█████▉    | 2795/4690 [25:30:07<17:16:58, 32.83s/it]
 60%|█████▉    | 2796/4690 [25:30:40<17:15:36, 32.81s/it]
 60%|█████▉    | 2797/4690 [25:31:13<17:15:02, 32.81s/it]
 60%|█████▉    | 2798/4690 [25:31:45<17:11:16, 32.70s/it]
 60%|█████▉    | 2799/4690 [25:32:18<17:12:02, 32.75s/it]
 60%|█████▉    | 2800/4690 [25:32:51<17:11:43, 32.75s/it]
                                                         
{'loss': '0.09207', 'grad_norm': '1.957', 'learning_rate': '3.691e-06', 'epoch': '2.986'}

 60%|█████▉    | 2800/4690 [25:32:51<17:11:43, 32.75s/it]
 60%|█████▉    | 2801/4690 [25:33:24<17:11:13, 32.75s/it]
 60%|█████▉    | 2802/4690 [25:33:56<17:09:27, 32.72s/it]
 60%|█████▉    | 2803/4690 [25:34:29<17:07:45, 32.68s/it]
 60%|█████▉    | 2804/4690 [25:35:01<17:04:47, 32.60s/it]
 60%|█████▉    | 2805/4690 [25:35:34<17:04:20, 32.60s/it]
 60%|█████▉    | 2806/4690 [25:36:07<17:05:46, 32.67s/it]
 60%|█████▉    | 2807/4690 [25:36:40<17:06:01, 32.69s/it]
 60%|█████▉    | 2808/4690 [25:37:13<17:10:42, 32.86s/it]
 60%|█████▉    | 2809/4690 [25:37:45<17:06:28, 32.74s/it]
 60%|█████▉    | 2810/4690 [25:38:18<17:06:45, 32.77s/it]
                                                         
{'loss': '0.09618', 'grad_norm': '2.084', 'learning_rate': '3.658e-06', 'epoch': '2.996'}

 60%|█████▉    | 2810/4690 [25:38:18<17:06:45, 32.77s/it]
 60%|█████▉    | 2811/4690 [25:38:51<17:04:25, 32.71s/it]
 60%|█████▉    | 2812/4690 [25:39:24<17:10:48, 32.93s/it]
 60%|█████▉    | 2813/4690 [25:39:57<17:12:18, 33.00s/it]
 60%|██████    | 2814/4690 [25:40:14<14:38:35, 28.10s/it][INFO|trainer.py:3815] 2026-05-15 03:01:37,524 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814
[INFO|configuration_utils.py:533] 2026-05-15 03:01:37,546 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/config.json
[INFO|configuration_utils.py:816] 2026-05-15 03:01:37,547 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/generation_config.json


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:15<00:00, 15.73s/it]
Writing model shards: 100%|██████████| 1/1 [00:15<00:00, 15.73s/it]
[INFO|modeling_utils.py:3471] 2026-05-15 03:01:53,324 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-15 03:01:53,327 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 03:01:53,329 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/tokenizer_config.json
[INFO|tokenization_utils_base.py:3278] 2026-05-15 03:02:26,652 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 03:02:26,654 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-15 03:02:26,798 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-2814/processor_config.json

 60%|██████    | 2815/4690 [25:41:45<24:28:52, 47.00s/it]
 60%|██████    | 2816/4690 [25:42:18<22:16:33, 42.79s/it]
 60%|██████    | 2817/4690 [25:42:51<20:43:43, 39.84s/it]
 60%|██████    | 2818/4690 [25:43:24<19:35:30, 37.68s/it]
 60%|██████    | 2819/4690 [25:43:56<18:49:28, 36.22s/it]
 60%|██████    | 2820/4690 [25:44:29<18:14:15, 35.11s/it]
                                                         
{'loss': '0.05845', 'grad_norm': '1.225', 'learning_rate': '3.625e-06', 'epoch': '3.006'}

 60%|██████    | 2820/4690 [25:44:29<18:14:15, 35.11s/it]
 60%|██████    | 2821/4690 [25:45:02<17:55:36, 34.53s/it]
 60%|██████    | 2822/4690 [25:45:35<17:39:23, 34.03s/it]
 60%|██████    | 2823/4690 [25:46:08<17:26:39, 33.64s/it]
 60%|██████    | 2824/4690 [25:46:40<17:17:05, 33.35s/it]
 60%|██████    | 2825/4690 [25:47:14<17:15:56, 33.33s/it]
 60%|██████    | 2826/4690 [25:47:46<17:08:58, 33.12s/it]
 60%|██████    | 2827/4690 [25:48:19<17:04:14, 32.99s/it]
 60%|██████    | 2828/4690 [25:48:52<17:01:23, 32.91s/it]
 60%|██████    | 2829/4690 [25:49:25<17:02:39, 32.97s/it]
 60%|██████    | 2830/4690 [25:49:57<16:59:26, 32.89s/it]
                                                         
{'loss': '0.03125', 'grad_norm': '1.312', 'learning_rate': '3.591e-06', 'epoch': '3.017'}

 60%|██████    | 2830/4690 [25:49:58<16:59:26, 32.89s/it]
 60%|██████    | 2831/4690 [25:50:30<16:57:00, 32.82s/it]
 60%|██████    | 2832/4690 [25:51:03<16:54:04, 32.75s/it]
 60%|██████    | 2833/4690 [25:51:35<16:52:40, 32.72s/it]
 60%|██████    | 2834/4690 [25:52:09<16:55:39, 32.83s/it]
 60%|██████    | 2835/4690 [25:52:41<16:49:52, 32.66s/it]
 60%|██████    | 2836/4690 [25:53:14<16:50:38, 32.71s/it]
 60%|██████    | 2837/4690 [25:53:47<16:54:28, 32.85s/it]
 61%|██████    | 2838/4690 [25:54:20<16:55:18, 32.89s/it]
 61%|██████    | 2839/4690 [25:54:53<16:54:40, 32.89s/it]
 61%|██████    | 2840/4690 [25:55:25<16:51:38, 32.81s/it]
                                                         
{'loss': '0.03089', 'grad_norm': '1.254', 'learning_rate': '3.558e-06', 'epoch': '3.028'}

 61%|██████    | 2840/4690 [25:55:25<16:51:38, 32.81s/it]
 61%|██████    | 2841/4690 [25:55:58<16:50:01, 32.78s/it]
 61%|██████    | 2842/4690 [25:56:31<16:48:13, 32.73s/it]
 61%|██████    | 2843/4690 [25:57:04<16:50:56, 32.84s/it]
 61%|██████    | 2844/4690 [25:57:36<16:48:40, 32.78s/it]
 61%|██████    | 2845/4690 [25:58:10<16:51:58, 32.91s/it]
 61%|██████    | 2846/4690 [25:58:43<16:52:21, 32.94s/it]
 61%|██████    | 2847/4690 [25:59:15<16:50:04, 32.88s/it]
 61%|██████    | 2848/4690 [25:59:48<16:46:07, 32.77s/it]
 61%|██████    | 2849/4690 [26:00:21<16:47:35, 32.84s/it]
 61%|██████    | 2850/4690 [26:00:53<16:42:45, 32.70s/it]
                                                         
{'loss': '0.03219', 'grad_norm': '1.478', 'learning_rate': '3.525e-06', 'epoch': '3.038'}

 61%|██████    | 2850/4690 [26:00:53<16:42:45, 32.70s/it]
 61%|██████    | 2851/4690 [26:01:26<16:42:55, 32.72s/it]
 61%|██████    | 2852/4690 [26:01:59<16:42:07, 32.71s/it]
 61%|██████    | 2853/4690 [26:02:32<16:42:50, 32.75s/it]
 61%|██████    | 2854/4690 [26:03:04<16:41:55, 32.74s/it]
 61%|██████    | 2855/4690 [26:03:37<16:38:54, 32.66s/it]
 61%|██████    | 2856/4690 [26:04:10<16:41:57, 32.78s/it]
 61%|██████    | 2857/4690 [26:04:43<16:44:24, 32.88s/it]
 61%|██████    | 2858/4690 [26:05:16<16:45:03, 32.92s/it]
 61%|██████    | 2859/4690 [26:05:48<16:40:29, 32.79s/it]
 61%|██████    | 2860/4690 [26:06:21<16:41:26, 32.83s/it]
                                                         
{'loss': '0.03183', 'grad_norm': '1.411', 'learning_rate': '3.492e-06', 'epoch': '3.049'}

 61%|██████    | 2860/4690 [26:06:21<16:41:26, 32.83s/it]
 61%|██████    | 2861/4690 [26:06:54<16:38:03, 32.74s/it]
 61%|██████    | 2862/4690 [26:07:26<16:36:56, 32.72s/it]
 61%|██████    | 2863/4690 [26:07:59<16:37:29, 32.76s/it]
 61%|██████    | 2864/4690 [26:08:32<16:36:28, 32.74s/it]
 61%|██████    | 2865/4690 [26:09:05<16:38:16, 32.82s/it]
 61%|██████    | 2866/4690 [26:09:38<16:38:16, 32.84s/it]
 61%|██████    | 2867/4690 [26:10:11<16:36:37, 32.80s/it]
 61%|██████    | 2868/4690 [26:10:44<16:37:30, 32.85s/it]
 61%|██████    | 2869/4690 [26:11:16<16:37:23, 32.86s/it]
 61%|██████    | 2870/4690 [26:11:49<16:34:52, 32.80s/it]
                                                         
{'loss': '0.03236', 'grad_norm': '1.341', 'learning_rate': '3.46e-06', 'epoch': '3.06'}

 61%|██████    | 2870/4690 [26:11:49<16:34:52, 32.80s/it]
 61%|██████    | 2871/4690 [26:12:22<16:31:15, 32.70s/it]
 61%|██████    | 2872/4690 [26:12:54<16:29:38, 32.66s/it]
 61%|██████▏   | 2873/4690 [26:13:27<16:27:15, 32.60s/it]
 61%|██████▏   | 2874/4690 [26:13:59<16:29:03, 32.68s/it]
 61%|██████▏   | 2875/4690 [26:14:32<16:28:46, 32.69s/it]
 61%|██████▏   | 2876/4690 [26:15:05<16:27:22, 32.66s/it]
 61%|██████▏   | 2877/4690 [26:15:37<16:24:46, 32.59s/it]
 61%|██████▏   | 2878/4690 [26:16:10<16:27:31, 32.70s/it]
 61%|██████▏   | 2879/4690 [26:16:43<16:26:21, 32.68s/it]
 61%|██████▏   | 2880/4690 [26:17:16<16:27:03, 32.72s/it]
                                                         
{'loss': '0.03173', 'grad_norm': '1.492', 'learning_rate': '3.427e-06', 'epoch': '3.07'}

 61%|██████▏   | 2880/4690 [26:17:16<16:27:03, 32.72s/it]
 61%|██████▏   | 2881/4690 [26:17:48<16:25:38, 32.69s/it]
 61%|██████▏   | 2882/4690 [26:18:21<16:28:29, 32.80s/it]
 61%|██████▏   | 2883/4690 [26:18:55<16:32:15, 32.95s/it]
 61%|██████▏   | 2884/4690 [26:19:27<16:27:38, 32.81s/it]
 62%|██████▏   | 2885/4690 [26:20:00<16:27:33, 32.83s/it]
 62%|██████▏   | 2886/4690 [26:20:33<16:26:20, 32.81s/it]
 62%|██████▏   | 2887/4690 [26:21:05<16:23:00, 32.71s/it]
 62%|██████▏   | 2888/4690 [26:21:38<16:23:33, 32.75s/it]
 62%|██████▏   | 2889/4690 [26:22:11<16:26:35, 32.87s/it]
 62%|██████▏   | 2890/4690 [26:22:44<16:26:24, 32.88s/it]
                                                         
{'loss': '0.03145', 'grad_norm': '1.089', 'learning_rate': '3.394e-06', 'epoch': '3.081'}

 62%|██████▏   | 2890/4690 [26:22:44<16:26:24, 32.88s/it]
 62%|██████▏   | 2891/4690 [26:23:17<16:27:41, 32.94s/it]
 62%|██████▏   | 2892/4690 [26:23:50<16:25:16, 32.88s/it]
 62%|██████▏   | 2893/4690 [26:24:22<16:20:10, 32.73s/it]
 62%|██████▏   | 2894/4690 [26:24:55<16:23:33, 32.86s/it]
 62%|██████▏   | 2895/4690 [26:25:28<16:23:54, 32.89s/it]
 62%|██████▏   | 2896/4690 [26:26:01<16:22:18, 32.85s/it]
 62%|██████▏   | 2897/4690 [26:26:34<16:17:39, 32.72s/it]
 62%|██████▏   | 2898/4690 [26:27:06<16:18:53, 32.78s/it]
 62%|██████▏   | 2899/4690 [26:27:39<16:15:27, 32.68s/it]
 62%|██████▏   | 2900/4690 [26:28:12<16:15:35, 32.70s/it]
                                                         
{'loss': '0.03028', 'grad_norm': '1.863', 'learning_rate': '3.361e-06', 'epoch': '3.092'}

 62%|██████▏   | 2900/4690 [26:28:12<16:15:35, 32.70s/it]
 62%|██████▏   | 2901/4690 [26:28:45<16:21:16, 32.91s/it]
 62%|██████▏   | 2902/4690 [26:29:18<16:21:11, 32.93s/it]
 62%|██████▏   | 2903/4690 [26:29:51<16:17:37, 32.82s/it]
 62%|██████▏   | 2904/4690 [26:30:23<16:14:08, 32.73s/it]
 62%|██████▏   | 2905/4690 [26:30:56<16:13:01, 32.71s/it]
 62%|██████▏   | 2906/4690 [26:31:29<16:13:07, 32.73s/it]
 62%|██████▏   | 2907/4690 [26:32:01<16:12:18, 32.72s/it]
 62%|██████▏   | 2908/4690 [26:32:34<16:11:07, 32.70s/it]
 62%|██████▏   | 2909/4690 [26:33:07<16:11:08, 32.72s/it]
 62%|██████▏   | 2910/4690 [26:33:40<16:11:43, 32.75s/it]
                                                         
{'loss': '0.03216', 'grad_norm': '1.505', 'learning_rate': '3.329e-06', 'epoch': '3.102'}

 62%|██████▏   | 2910/4690 [26:33:40<16:11:43, 32.75s/it]
 62%|██████▏   | 2911/4690 [26:34:13<16:15:21, 32.90s/it]
 62%|██████▏   | 2912/4690 [26:34:46<16:15:00, 32.90s/it]
 62%|██████▏   | 2913/4690 [26:35:18<16:11:56, 32.82s/it]
 62%|██████▏   | 2914/4690 [26:35:51<16:12:12, 32.85s/it]
 62%|██████▏   | 2915/4690 [26:36:24<16:12:25, 32.87s/it]
 62%|██████▏   | 2916/4690 [26:36:57<16:09:15, 32.78s/it]
 62%|██████▏   | 2917/4690 [26:37:29<16:04:21, 32.63s/it]
 62%|██████▏   | 2918/4690 [26:38:02<16:05:09, 32.68s/it]
 62%|██████▏   | 2919/4690 [26:38:35<16:07:13, 32.77s/it]
 62%|██████▏   | 2920/4690 [26:39:08<16:06:52, 32.78s/it]
                                                         
{'loss': '0.0329', 'grad_norm': '1.538', 'learning_rate': '3.296e-06', 'epoch': '3.113'}

 62%|██████▏   | 2920/4690 [26:39:08<16:06:52, 32.78s/it]
 62%|██████▏   | 2921/4690 [26:39:40<16:03:58, 32.70s/it]
 62%|██████▏   | 2922/4690 [26:40:13<16:03:31, 32.70s/it]
 62%|██████▏   | 2923/4690 [26:40:45<16:02:25, 32.68s/it]
 62%|██████▏   | 2924/4690 [26:41:18<16:02:19, 32.69s/it]
 62%|██████▏   | 2925/4690 [26:41:51<16:03:55, 32.77s/it]
 62%|██████▏   | 2926/4690 [26:42:24<16:03:03, 32.76s/it]
 62%|██████▏   | 2927/4690 [26:42:57<16:06:04, 32.88s/it]
 62%|██████▏   | 2928/4690 [26:43:30<16:06:58, 32.93s/it]
 62%|██████▏   | 2929/4690 [26:44:03<16:07:42, 32.97s/it]
 62%|██████▏   | 2930/4690 [26:44:36<16:04:51, 32.89s/it]
                                                         
{'loss': '0.03341', 'grad_norm': '1.341', 'learning_rate': '3.264e-06', 'epoch': '3.124'}

 62%|██████▏   | 2930/4690 [26:44:36<16:04:51, 32.89s/it]
 62%|██████▏   | 2931/4690 [26:45:08<15:59:41, 32.74s/it]
 63%|██████▎   | 2932/4690 [26:45:41<16:03:56, 32.90s/it]
 63%|██████▎   | 2933/4690 [26:46:14<16:02:21, 32.86s/it]
 63%|██████▎   | 2934/4690 [26:46:47<15:59:35, 32.79s/it]
 63%|██████▎   | 2935/4690 [26:47:20<16:00:41, 32.84s/it]
 63%|██████▎   | 2936/4690 [26:47:53<16:01:05, 32.88s/it]
 63%|██████▎   | 2937/4690 [26:48:25<15:56:03, 32.72s/it]
 63%|██████▎   | 2938/4690 [26:48:58<15:56:09, 32.74s/it]
 63%|██████▎   | 2939/4690 [26:49:30<15:51:15, 32.60s/it]
 63%|██████▎   | 2940/4690 [26:50:03<15:52:31, 32.66s/it]
                                                         
{'loss': '0.03164', 'grad_norm': '1.762', 'learning_rate': '3.231e-06', 'epoch': '3.134'}

 63%|██████▎   | 2940/4690 [26:50:03<15:52:31, 32.66s/it]
 63%|██████▎   | 2941/4690 [26:50:36<15:52:28, 32.68s/it]
 63%|██████▎   | 2942/4690 [26:51:08<15:53:09, 32.72s/it]
 63%|██████▎   | 2943/4690 [26:51:41<15:54:12, 32.77s/it]
 63%|██████▎   | 2944/4690 [26:52:14<15:49:41, 32.64s/it]
 63%|██████▎   | 2945/4690 [26:52:46<15:49:53, 32.66s/it]
 63%|██████▎   | 2946/4690 [26:53:19<15:46:44, 32.57s/it]
 63%|██████▎   | 2947/4690 [26:53:52<15:48:47, 32.66s/it]
 63%|██████▎   | 2948/4690 [26:54:24<15:46:48, 32.61s/it]
 63%|██████▎   | 2949/4690 [26:54:57<15:47:48, 32.66s/it]
 63%|██████▎   | 2950/4690 [26:55:30<15:50:36, 32.78s/it]
                                                         
{'loss': '0.03094', 'grad_norm': '1.223', 'learning_rate': '3.199e-06', 'epoch': '3.145'}

 63%|██████▎   | 2950/4690 [26:55:30<15:50:36, 32.78s/it]
 63%|██████▎   | 2951/4690 [26:56:02<15:45:45, 32.63s/it]
 63%|██████▎   | 2952/4690 [26:56:35<15:45:34, 32.64s/it]
 63%|██████▎   | 2953/4690 [26:57:07<15:43:30, 32.59s/it]
 63%|██████▎   | 2954/4690 [26:57:40<15:45:53, 32.69s/it]
 63%|██████▎   | 2955/4690 [26:58:13<15:45:25, 32.70s/it]
 63%|██████▎   | 2956/4690 [26:58:46<15:46:50, 32.76s/it]
 63%|██████▎   | 2957/4690 [26:59:19<15:48:46, 32.85s/it]
 63%|██████▎   | 2958/4690 [26:59:52<15:45:47, 32.76s/it]
 63%|██████▎   | 2959/4690 [27:00:24<15:40:32, 32.60s/it]
 63%|██████▎   | 2960/4690 [27:00:57<15:41:11, 32.64s/it]
                                                         
{'loss': '0.03121', 'grad_norm': '1.382', 'learning_rate': '3.167e-06', 'epoch': '3.156'}

 63%|██████▎   | 2960/4690 [27:00:57<15:41:11, 32.64s/it]
 63%|██████▎   | 2961/4690 [27:01:29<15:41:32, 32.67s/it]
 63%|██████▎   | 2962/4690 [27:02:02<15:41:50, 32.70s/it]
 63%|██████▎   | 2963/4690 [27:02:35<15:41:02, 32.69s/it]
 63%|██████▎   | 2964/4690 [27:03:08<15:43:06, 32.78s/it]
 63%|██████▎   | 2965/4690 [27:03:40<15:41:59, 32.77s/it]
 63%|██████▎   | 2966/4690 [27:04:14<15:45:34, 32.91s/it]
 63%|██████▎   | 2967/4690 [27:04:46<15:43:33, 32.86s/it]
 63%|██████▎   | 2968/4690 [27:05:20<15:44:52, 32.92s/it]
 63%|██████▎   | 2969/4690 [27:05:52<15:43:13, 32.88s/it]
 63%|██████▎   | 2970/4690 [27:06:25<15:42:05, 32.86s/it]
                                                         
{'loss': '0.03076', 'grad_norm': '1.464', 'learning_rate': '3.135e-06', 'epoch': '3.166'}

 63%|██████▎   | 2970/4690 [27:06:25<15:42:05, 32.86s/it]
 63%|██████▎   | 2971/4690 [27:06:58<15:44:03, 32.95s/it]
 63%|██████▎   | 2972/4690 [27:07:31<15:39:35, 32.81s/it]
 63%|██████▎   | 2973/4690 [27:08:04<15:40:45, 32.87s/it]
 63%|██████▎   | 2974/4690 [27:08:37<15:40:15, 32.88s/it]
 63%|██████▎   | 2975/4690 [27:09:09<15:38:13, 32.82s/it]
 63%|██████▎   | 2976/4690 [27:09:42<15:37:05, 32.80s/it]
 63%|██████▎   | 2977/4690 [27:10:15<15:33:49, 32.71s/it]
 63%|██████▎   | 2978/4690 [27:10:48<15:35:39, 32.79s/it]
 64%|██████▎   | 2979/4690 [27:11:20<15:35:51, 32.82s/it]
 64%|██████▎   | 2980/4690 [27:11:53<15:34:55, 32.80s/it]
                                                         
{'loss': '0.02986', 'grad_norm': '1.19', 'learning_rate': '3.103e-06', 'epoch': '3.177'}

 64%|██████▎   | 2980/4690 [27:11:53<15:34:55, 32.80s/it]
 64%|██████▎   | 2981/4690 [27:12:26<15:33:38, 32.78s/it]
 64%|██████▎   | 2982/4690 [27:12:59<15:35:01, 32.85s/it]
 64%|██████▎   | 2983/4690 [27:13:32<15:38:05, 32.97s/it]
 64%|██████▎   | 2984/4690 [27:14:05<15:35:31, 32.90s/it]
 64%|██████▎   | 2985/4690 [27:14:38<15:33:34, 32.85s/it]
 64%|██████▎   | 2986/4690 [27:15:10<15:30:51, 32.78s/it]
 64%|██████▎   | 2987/4690 [27:15:43<15:32:54, 32.87s/it]
 64%|██████▎   | 2988/4690 [27:16:16<15:31:19, 32.83s/it]
 64%|██████▎   | 2989/4690 [27:16:49<15:29:23, 32.78s/it]
 64%|██████▍   | 2990/4690 [27:17:22<15:29:10, 32.79s/it]
                                                         
{'loss': '0.03403', 'grad_norm': '1.442', 'learning_rate': '3.071e-06', 'epoch': '3.188'}

 64%|██████▍   | 2990/4690 [27:17:22<15:29:10, 32.79s/it]
 64%|██████▍   | 2991/4690 [27:17:54<15:26:28, 32.72s/it]
 64%|██████▍   | 2992/4690 [27:18:27<15:23:14, 32.62s/it]
 64%|██████▍   | 2993/4690 [27:18:59<15:23:54, 32.67s/it]
 64%|██████▍   | 2994/4690 [27:19:32<15:23:38, 32.68s/it]
 64%|██████▍   | 2995/4690 [27:20:04<15:20:35, 32.59s/it]
 64%|██████▍   | 2996/4690 [27:20:37<15:22:32, 32.68s/it]
 64%|██████▍   | 2997/4690 [27:21:11<15:26:26, 32.83s/it]
 64%|██████▍   | 2998/4690 [27:21:44<15:28:48, 32.94s/it]
 64%|██████▍   | 2999/4690 [27:22:16<15:25:05, 32.82s/it]
 64%|██████▍   | 3000/4690 [27:22:49<15:27:42, 32.94s/it]
                                                         
{'loss': '0.03084', 'grad_norm': '1.206', 'learning_rate': '3.039e-06', 'epoch': '3.198'}

 64%|██████▍   | 3000/4690 [27:22:49<15:27:42, 32.94s/it]
 64%|██████▍   | 3001/4690 [27:23:22<15:26:47, 32.92s/it]
 64%|██████▍   | 3002/4690 [27:23:55<15:23:56, 32.84s/it]
 64%|██████▍   | 3003/4690 [27:24:28<15:22:53, 32.82s/it]
 64%|██████▍   | 3004/4690 [27:25:01<15:22:46, 32.84s/it]
 64%|██████▍   | 3005/4690 [27:25:33<15:21:07, 32.80s/it]
 64%|██████▍   | 3006/4690 [27:26:06<15:21:02, 32.82s/it]
 64%|██████▍   | 3007/4690 [27:26:39<15:21:49, 32.86s/it]
 64%|██████▍   | 3008/4690 [27:27:12<15:18:37, 32.77s/it]
 64%|██████▍   | 3009/4690 [27:27:45<15:18:31, 32.79s/it]
 64%|██████▍   | 3010/4690 [27:28:17<15:15:38, 32.70s/it]
                                                         
{'loss': '0.03053', 'grad_norm': '1.24', 'learning_rate': '3.008e-06', 'epoch': '3.209'}

 64%|██████▍   | 3010/4690 [27:28:17<15:15:38, 32.70s/it]
 64%|██████▍   | 3011/4690 [27:28:50<15:18:52, 32.84s/it]
 64%|██████▍   | 3012/4690 [27:29:23<15:16:51, 32.78s/it]
 64%|██████▍   | 3013/4690 [27:29:55<15:13:21, 32.68s/it]
 64%|██████▍   | 3014/4690 [27:30:28<15:14:45, 32.75s/it]
 64%|██████▍   | 3015/4690 [27:31:01<15:14:55, 32.77s/it]
 64%|██████▍   | 3016/4690 [27:31:34<15:13:47, 32.75s/it]
 64%|██████▍   | 3017/4690 [27:32:06<15:10:19, 32.65s/it]
 64%|██████▍   | 3018/4690 [27:32:39<15:11:59, 32.73s/it]
 64%|██████▍   | 3019/4690 [27:33:12<15:10:46, 32.70s/it]
 64%|██████▍   | 3020/4690 [27:33:45<15:13:13, 32.81s/it]
                                                         
{'loss': '0.0315', 'grad_norm': '1.468', 'learning_rate': '2.976e-06', 'epoch': '3.22'}

 64%|██████▍   | 3020/4690 [27:33:45<15:13:13, 32.81s/it]
 64%|██████▍   | 3021/4690 [27:34:18<15:14:48, 32.89s/it]
 64%|██████▍   | 3022/4690 [27:34:51<15:14:13, 32.89s/it]
 64%|██████▍   | 3023/4690 [27:35:23<15:09:31, 32.74s/it]
 64%|██████▍   | 3024/4690 [27:35:56<15:08:34, 32.72s/it]
 64%|██████▍   | 3025/4690 [27:36:29<15:10:27, 32.81s/it]
 65%|██████▍   | 3026/4690 [27:37:01<15:06:31, 32.69s/it]
 65%|██████▍   | 3027/4690 [27:37:34<15:04:20, 32.63s/it]
 65%|██████▍   | 3028/4690 [27:38:07<15:08:06, 32.78s/it]
 65%|██████▍   | 3029/4690 [27:38:40<15:08:23, 32.81s/it]
 65%|██████▍   | 3030/4690 [27:39:13<15:11:07, 32.93s/it]
                                                         
{'loss': '0.03123', 'grad_norm': '1.744', 'learning_rate': '2.944e-06', 'epoch': '3.23'}

 65%|██████▍   | 3030/4690 [27:39:13<15:11:07, 32.93s/it]
 65%|██████▍   | 3031/4690 [27:39:45<15:05:15, 32.74s/it]
 65%|██████▍   | 3032/4690 [27:40:18<15:05:55, 32.78s/it]
 65%|██████▍   | 3033/4690 [27:40:51<15:02:23, 32.68s/it]
 65%|██████▍   | 3034/4690 [27:41:23<15:03:00, 32.72s/it]
 65%|██████▍   | 3035/4690 [27:41:56<15:01:22, 32.68s/it]
 65%|██████▍   | 3036/4690 [27:42:29<15:00:32, 32.67s/it]
 65%|██████▍   | 3037/4690 [27:43:01<15:00:44, 32.69s/it]
 65%|██████▍   | 3038/4690 [27:43:34<15:01:56, 32.76s/it]
 65%|██████▍   | 3039/4690 [27:44:07<15:01:15, 32.75s/it]
 65%|██████▍   | 3040/4690 [27:44:40<15:02:21, 32.81s/it]
                                                         
{'loss': '0.03027', 'grad_norm': '1.451', 'learning_rate': '2.913e-06', 'epoch': '3.241'}

 65%|██████▍   | 3040/4690 [27:44:40<15:02:21, 32.81s/it]
 65%|██████▍   | 3041/4690 [27:45:13<15:04:22, 32.91s/it]
 65%|██████▍   | 3042/4690 [27:45:46<15:05:05, 32.95s/it]
 65%|██████▍   | 3043/4690 [27:46:19<15:04:03, 32.93s/it]
 65%|██████▍   | 3044/4690 [27:46:52<15:02:10, 32.89s/it]
 65%|██████▍   | 3045/4690 [27:47:25<15:02:14, 32.91s/it]
 65%|██████▍   | 3046/4690 [27:47:58<15:00:31, 32.87s/it]
 65%|██████▍   | 3047/4690 [27:48:30<14:56:58, 32.76s/it]
 65%|██████▍   | 3048/4690 [27:49:03<14:58:28, 32.83s/it]
 65%|██████▌   | 3049/4690 [27:49:36<14:57:48, 32.83s/it]
 65%|██████▌   | 3050/4690 [27:50:09<14:57:55, 32.85s/it]
                                                         
{'loss': '0.03289', 'grad_norm': '1.901', 'learning_rate': '2.882e-06', 'epoch': '3.252'}

 65%|██████▌   | 3050/4690 [27:50:09<14:57:55, 32.85s/it]
 65%|██████▌   | 3051/4690 [27:50:42<14:58:58, 32.91s/it]
 65%|██████▌   | 3052/4690 [27:51:15<14:57:55, 32.89s/it]
 65%|██████▌   | 3053/4690 [27:51:48<14:59:14, 32.96s/it]
 65%|██████▌   | 3054/4690 [27:52:20<14:56:49, 32.89s/it]
 65%|██████▌   | 3055/4690 [27:52:53<14:56:00, 32.88s/it]
 65%|██████▌   | 3056/4690 [27:53:26<14:53:48, 32.82s/it]
 65%|██████▌   | 3057/4690 [27:53:59<14:51:10, 32.74s/it]
 65%|██████▌   | 3058/4690 [27:54:32<14:53:54, 32.86s/it]
 65%|██████▌   | 3059/4690 [27:55:04<14:50:09, 32.75s/it]
 65%|██████▌   | 3060/4690 [27:55:37<14:47:03, 32.65s/it]
                                                         
{'loss': '0.03234', 'grad_norm': '1.24', 'learning_rate': '2.85e-06', 'epoch': '3.262'}

 65%|██████▌   | 3060/4690 [27:55:37<14:47:03, 32.65s/it]
 65%|██████▌   | 3061/4690 [27:56:10<14:52:22, 32.87s/it]
 65%|██████▌   | 3062/4690 [27:56:43<14:50:42, 32.83s/it]
 65%|██████▌   | 3063/4690 [27:57:16<14:49:30, 32.80s/it]
 65%|██████▌   | 3064/4690 [27:57:48<14:49:00, 32.80s/it]
 65%|██████▌   | 3065/4690 [27:58:21<14:47:46, 32.78s/it]
 65%|██████▌   | 3066/4690 [27:58:54<14:47:01, 32.77s/it]
 65%|██████▌   | 3067/4690 [27:59:26<14:44:11, 32.69s/it]
 65%|██████▌   | 3068/4690 [27:59:59<14:45:22, 32.75s/it]
 65%|██████▌   | 3069/4690 [28:00:32<14:44:43, 32.75s/it]
 65%|██████▌   | 3070/4690 [28:01:05<14:43:40, 32.73s/it]
                                                         
{'loss': '0.02995', 'grad_norm': '1.306', 'learning_rate': '2.819e-06', 'epoch': '3.273'}

 65%|██████▌   | 3070/4690 [28:01:05<14:43:40, 32.73s/it]
 65%|██████▌   | 3071/4690 [28:01:37<14:40:53, 32.65s/it]
 66%|██████▌   | 3072/4690 [28:02:10<14:39:50, 32.63s/it]
 66%|██████▌   | 3073/4690 [28:02:43<14:45:50, 32.87s/it]
 66%|██████▌   | 3074/4690 [28:03:16<14:46:38, 32.92s/it]
 66%|██████▌   | 3075/4690 [28:03:49<14:46:13, 32.92s/it]
 66%|██████▌   | 3076/4690 [28:04:22<14:45:14, 32.91s/it]
 66%|██████▌   | 3077/4690 [28:04:55<14:44:03, 32.88s/it]
 66%|██████▌   | 3078/4690 [28:05:27<14:40:28, 32.77s/it]
 66%|██████▌   | 3079/4690 [28:06:00<14:40:23, 32.79s/it]
 66%|██████▌   | 3080/4690 [28:06:33<14:37:37, 32.71s/it]
                                                         
{'loss': '0.03026', 'grad_norm': '1.381', 'learning_rate': '2.788e-06', 'epoch': '3.284'}

 66%|██████▌   | 3080/4690 [28:06:33<14:37:37, 32.71s/it]
 66%|██████▌   | 3081/4690 [28:07:05<14:38:23, 32.76s/it]
 66%|██████▌   | 3082/4690 [28:07:38<14:38:05, 32.76s/it]
 66%|██████▌   | 3083/4690 [28:08:11<14:36:29, 32.73s/it]
 66%|██████▌   | 3084/4690 [28:08:44<14:39:16, 32.85s/it]
 66%|██████▌   | 3085/4690 [28:09:17<14:38:13, 32.83s/it]
 66%|██████▌   | 3086/4690 [28:09:50<14:37:39, 32.83s/it]
 66%|██████▌   | 3087/4690 [28:10:22<14:36:33, 32.81s/it]
 66%|██████▌   | 3088/4690 [28:10:55<14:33:17, 32.71s/it]
 66%|██████▌   | 3089/4690 [28:11:27<14:30:37, 32.63s/it]
 66%|██████▌   | 3090/4690 [28:12:00<14:30:48, 32.66s/it]
                                                         
{'loss': '0.03273', 'grad_norm': '1.47', 'learning_rate': '2.757e-06', 'epoch': '3.294'}

 66%|██████▌   | 3090/4690 [28:12:00<14:30:48, 32.66s/it]
 66%|██████▌   | 3091/4690 [28:12:33<14:30:44, 32.67s/it]
 66%|██████▌   | 3092/4690 [28:13:06<14:31:54, 32.74s/it]
 66%|██████▌   | 3093/4690 [28:13:39<14:33:55, 32.83s/it]
 66%|██████▌   | 3094/4690 [28:14:12<14:35:34, 32.92s/it]
 66%|██████▌   | 3095/4690 [28:14:44<14:31:17, 32.78s/it]
 66%|██████▌   | 3096/4690 [28:15:17<14:31:07, 32.79s/it]
 66%|██████▌   | 3097/4690 [28:15:50<14:29:47, 32.76s/it]
 66%|██████▌   | 3098/4690 [28:16:23<14:32:46, 32.89s/it]
 66%|██████▌   | 3099/4690 [28:16:56<14:33:08, 32.93s/it]
 66%|██████▌   | 3100/4690 [28:17:29<14:32:36, 32.93s/it]
                                                         
{'loss': '0.03322', 'grad_norm': '1.565', 'learning_rate': '2.727e-06', 'epoch': '3.305'}

 66%|██████▌   | 3100/4690 [28:17:29<14:32:36, 32.93s/it]
 66%|██████▌   | 3101/4690 [28:18:01<14:28:22, 32.79s/it]
 66%|██████▌   | 3102/4690 [28:18:35<14:30:37, 32.90s/it]
 66%|██████▌   | 3103/4690 [28:19:07<14:30:33, 32.91s/it]
 66%|██████▌   | 3104/4690 [28:19:40<14:28:52, 32.87s/it]
 66%|██████▌   | 3105/4690 [28:20:13<14:26:03, 32.78s/it]
 66%|██████▌   | 3106/4690 [28:20:45<14:24:33, 32.75s/it]
 66%|██████▌   | 3107/4690 [28:21:19<14:27:37, 32.89s/it]
 66%|██████▋   | 3108/4690 [28:21:52<14:26:48, 32.88s/it]
 66%|██████▋   | 3109/4690 [28:22:24<14:22:05, 32.72s/it]
 66%|██████▋   | 3110/4690 [28:22:56<14:20:05, 32.66s/it]
                                                         
{'loss': '0.03185', 'grad_norm': '1.943', 'learning_rate': '2.696e-06', 'epoch': '3.316'}

 66%|██████▋   | 3110/4690 [28:22:56<14:20:05, 32.66s/it]
 66%|██████▋   | 3111/4690 [28:23:29<14:19:27, 32.66s/it]
 66%|██████▋   | 3112/4690 [28:24:01<14:15:13, 32.52s/it]
 66%|██████▋   | 3113/4690 [28:24:34<14:17:03, 32.61s/it]
 66%|██████▋   | 3114/4690 [28:25:07<14:18:41, 32.69s/it]
 66%|██████▋   | 3115/4690 [28:25:40<14:19:15, 32.73s/it]
 66%|██████▋   | 3116/4690 [28:26:12<14:17:18, 32.68s/it]
 66%|██████▋   | 3117/4690 [28:26:45<14:15:56, 32.65s/it]
 66%|██████▋   | 3118/4690 [28:27:18<14:19:43, 32.81s/it]
 67%|██████▋   | 3119/4690 [28:27:51<14:16:57, 32.73s/it]
 67%|██████▋   | 3120/4690 [28:28:24<14:17:49, 32.78s/it]
                                                         
{'loss': '0.02858', 'grad_norm': '1.337', 'learning_rate': '2.665e-06', 'epoch': '3.326'}

 67%|██████▋   | 3120/4690 [28:28:24<14:17:49, 32.78s/it]
 67%|██████▋   | 3121/4690 [28:28:56<14:14:53, 32.69s/it]
 67%|██████▋   | 3122/4690 [28:29:29<14:13:25, 32.66s/it]
 67%|██████▋   | 3123/4690 [28:30:01<14:12:13, 32.63s/it]
 67%|██████▋   | 3124/4690 [28:30:34<14:13:11, 32.69s/it]
 67%|██████▋   | 3125/4690 [28:31:07<14:11:30, 32.65s/it]
 67%|██████▋   | 3126/4690 [28:31:39<14:13:10, 32.73s/it]
 67%|██████▋   | 3127/4690 [28:32:12<14:11:49, 32.70s/it]
 67%|██████▋   | 3128/4690 [28:32:44<14:08:21, 32.59s/it]
 67%|██████▋   | 3129/4690 [28:33:17<14:09:56, 32.67s/it]
 67%|██████▋   | 3130/4690 [28:33:50<14:13:10, 32.81s/it]
                                                         
{'loss': '0.03382', 'grad_norm': '1.27', 'learning_rate': '2.635e-06', 'epoch': '3.337'}

 67%|██████▋   | 3130/4690 [28:33:50<14:13:10, 32.81s/it]
 67%|██████▋   | 3131/4690 [28:34:23<14:12:09, 32.80s/it]
 67%|██████▋   | 3132/4690 [28:34:56<14:11:46, 32.80s/it]
 67%|██████▋   | 3133/4690 [28:35:29<14:15:48, 32.98s/it]
 67%|██████▋   | 3134/4690 [28:36:03<14:16:29, 33.03s/it]
 67%|██████▋   | 3135/4690 [28:36:35<14:14:59, 32.99s/it]
 67%|██████▋   | 3136/4690 [28:37:08<14:13:26, 32.95s/it]
 67%|██████▋   | 3137/4690 [28:37:41<14:11:41, 32.90s/it]
 67%|██████▋   | 3138/4690 [28:38:14<14:13:40, 33.00s/it]
 67%|██████▋   | 3139/4690 [28:38:47<14:10:21, 32.90s/it]
 67%|██████▋   | 3140/4690 [28:39:20<14:10:01, 32.90s/it]
                                                         
{'loss': '0.03076', 'grad_norm': '1.287', 'learning_rate': '2.604e-06', 'epoch': '3.348'}

 67%|██████▋   | 3140/4690 [28:39:20<14:10:01, 32.90s/it]
 67%|██████▋   | 3141/4690 [28:39:53<14:08:39, 32.87s/it]
 67%|██████▋   | 3142/4690 [28:40:26<14:07:41, 32.86s/it]
 67%|██████▋   | 3143/4690 [28:40:58<14:05:25, 32.79s/it]
 67%|██████▋   | 3144/4690 [28:41:31<14:03:41, 32.74s/it]
 67%|██████▋   | 3145/4690 [28:42:04<14:03:48, 32.77s/it]
 67%|██████▋   | 3146/4690 [28:42:36<14:02:25, 32.74s/it]
 67%|██████▋   | 3147/4690 [28:43:09<14:01:13, 32.71s/it]
 67%|██████▋   | 3148/4690 [28:43:41<13:59:06, 32.65s/it]
 67%|██████▋   | 3149/4690 [28:44:14<13:59:34, 32.69s/it]
 67%|██████▋   | 3150/4690 [28:44:47<14:00:16, 32.74s/it]
                                                         
{'loss': '0.03197', 'grad_norm': '1.486', 'learning_rate': '2.574e-06', 'epoch': '3.358'}

 67%|██████▋   | 3150/4690 [28:44:47<14:00:16, 32.74s/it]
 67%|██████▋   | 3151/4690 [28:45:19<13:56:56, 32.63s/it]
 67%|██████▋   | 3152/4690 [28:45:52<13:58:50, 32.72s/it]
 67%|██████▋   | 3153/4690 [28:46:26<14:02:11, 32.88s/it]
 67%|██████▋   | 3154/4690 [28:46:59<14:03:41, 32.96s/it]
 67%|██████▋   | 3155/4690 [28:47:32<14:03:36, 32.97s/it]
 67%|██████▋   | 3156/4690 [28:48:05<14:02:07, 32.94s/it]
 67%|██████▋   | 3157/4690 [28:48:38<14:01:54, 32.95s/it]
 67%|██████▋   | 3158/4690 [28:49:10<13:58:13, 32.83s/it]
 67%|██████▋   | 3159/4690 [28:49:43<13:53:58, 32.68s/it]
 67%|██████▋   | 3160/4690 [28:50:15<13:54:23, 32.72s/it]
                                                         
{'loss': '0.03141', 'grad_norm': '1.384', 'learning_rate': '2.544e-06', 'epoch': '3.369'}

 67%|██████▋   | 3160/4690 [28:50:15<13:54:23, 32.72s/it]
 67%|██████▋   | 3161/4690 [28:50:48<13:55:51, 32.80s/it]
 67%|██████▋   | 3162/4690 [28:51:21<13:55:22, 32.80s/it]
 67%|██████▋   | 3163/4690 [28:51:54<13:52:18, 32.70s/it]
 67%|██████▋   | 3164/4690 [28:52:26<13:50:38, 32.66s/it]
 67%|██████▋   | 3165/4690 [28:52:58<13:47:32, 32.56s/it]
 68%|██████▊   | 3166/4690 [28:53:31<13:47:13, 32.57s/it]
 68%|██████▊   | 3167/4690 [28:54:04<13:46:56, 32.58s/it]
 68%|██████▊   | 3168/4690 [28:54:36<13:47:02, 32.60s/it]
 68%|██████▊   | 3169/4690 [28:55:09<13:45:04, 32.55s/it]
 68%|██████▊   | 3170/4690 [28:55:42<13:46:55, 32.64s/it]
                                                         
{'loss': '0.03003', 'grad_norm': '1.571', 'learning_rate': '2.514e-06', 'epoch': '3.38'}

 68%|██████▊   | 3170/4690 [28:55:42<13:46:55, 32.64s/it]
 68%|██████▊   | 3171/4690 [28:56:14<13:46:27, 32.64s/it]
 68%|██████▊   | 3172/4690 [28:56:47<13:47:30, 32.71s/it]
 68%|██████▊   | 3173/4690 [28:57:20<13:51:36, 32.89s/it]
 68%|██████▊   | 3174/4690 [28:57:53<13:50:10, 32.86s/it]
 68%|██████▊   | 3175/4690 [28:58:26<13:50:02, 32.87s/it]
 68%|██████▊   | 3176/4690 [28:58:59<13:48:48, 32.85s/it]
 68%|██████▊   | 3177/4690 [28:59:32<13:46:48, 32.79s/it]
 68%|██████▊   | 3178/4690 [29:00:04<13:45:05, 32.74s/it]
 68%|██████▊   | 3179/4690 [29:00:37<13:41:58, 32.64s/it]
 68%|██████▊   | 3180/4690 [29:01:09<13:41:38, 32.65s/it]
                                                         
{'loss': '0.02732', 'grad_norm': '1.28', 'learning_rate': '2.484e-06', 'epoch': '3.39'}

 68%|██████▊   | 3180/4690 [29:01:09<13:41:38, 32.65s/it]
 68%|██████▊   | 3181/4690 [29:01:42<13:44:52, 32.80s/it]
 68%|██████▊   | 3182/4690 [29:02:15<13:43:19, 32.76s/it]
 68%|██████▊   | 3183/4690 [29:02:48<13:45:58, 32.89s/it]
 68%|██████▊   | 3184/4690 [29:03:21<13:45:14, 32.88s/it]
 68%|██████▊   | 3185/4690 [29:03:54<13:41:28, 32.75s/it]
 68%|██████▊   | 3186/4690 [29:04:27<13:43:59, 32.87s/it]
 68%|██████▊   | 3187/4690 [29:04:59<13:40:09, 32.74s/it]
 68%|██████▊   | 3188/4690 [29:05:32<13:40:52, 32.79s/it]
 68%|██████▊   | 3189/4690 [29:06:05<13:40:21, 32.79s/it]
 68%|██████▊   | 3190/4690 [29:06:38<13:41:28, 32.86s/it]
                                                         
{'loss': '0.03051', 'grad_norm': '1.338', 'learning_rate': '2.454e-06', 'epoch': '3.401'}

 68%|██████▊   | 3190/4690 [29:06:38<13:41:28, 32.86s/it]
 68%|██████▊   | 3191/4690 [29:07:11<13:41:20, 32.88s/it]
 68%|██████▊   | 3192/4690 [29:07:44<13:42:19, 32.94s/it]
 68%|██████▊   | 3193/4690 [29:08:16<13:39:18, 32.84s/it]
 68%|██████▊   | 3194/4690 [29:08:49<13:39:07, 32.85s/it]
 68%|██████▊   | 3195/4690 [29:09:22<13:38:21, 32.84s/it]
 68%|██████▊   | 3196/4690 [29:09:55<13:38:12, 32.86s/it]
 68%|██████▊   | 3197/4690 [29:10:28<13:36:09, 32.80s/it]
 68%|██████▊   | 3198/4690 [29:11:00<13:32:39, 32.68s/it]
 68%|██████▊   | 3199/4690 [29:11:33<13:33:31, 32.74s/it]
 68%|██████▊   | 3200/4690 [29:12:06<13:35:40, 32.85s/it]
                                                         
{'loss': '0.0328', 'grad_norm': '1.582', 'learning_rate': '2.425e-06', 'epoch': '3.412'}

 68%|██████▊   | 3200/4690 [29:12:06<13:35:40, 32.85s/it]
 68%|██████▊   | 3201/4690 [29:12:39<13:35:14, 32.85s/it]
 68%|██████▊   | 3202/4690 [29:13:12<13:32:16, 32.75s/it]
 68%|██████▊   | 3203/4690 [29:13:44<13:28:31, 32.62s/it]
 68%|██████▊   | 3204/4690 [29:14:16<13:26:43, 32.57s/it]
 68%|██████▊   | 3205/4690 [29:14:49<13:29:40, 32.71s/it]
 68%|██████▊   | 3206/4690 [29:15:22<13:28:08, 32.67s/it]
 68%|██████▊   | 3207/4690 [29:15:55<13:28:33, 32.71s/it]
 68%|██████▊   | 3208/4690 [29:16:27<13:26:15, 32.64s/it]
 68%|██████▊   | 3209/4690 [29:17:00<13:24:14, 32.58s/it]
 68%|██████▊   | 3210/4690 [29:17:32<13:22:38, 32.54s/it]
                                                         
{'loss': '0.03172', 'grad_norm': '1.086', 'learning_rate': '2.395e-06', 'epoch': '3.422'}

 68%|██████▊   | 3210/4690 [29:17:32<13:22:38, 32.54s/it]
 68%|██████▊   | 3211/4690 [29:18:04<13:20:33, 32.48s/it]
 68%|██████▊   | 3212/4690 [29:18:37<13:21:41, 32.54s/it]
 69%|██████▊   | 3213/4690 [29:19:10<13:21:29, 32.56s/it]
 69%|██████▊   | 3214/4690 [29:19:43<13:25:17, 32.74s/it]
 69%|██████▊   | 3215/4690 [29:20:15<13:23:22, 32.68s/it]
 69%|██████▊   | 3216/4690 [29:20:48<13:25:03, 32.77s/it]
 69%|██████▊   | 3217/4690 [29:21:21<13:22:40, 32.70s/it]
 69%|██████▊   | 3218/4690 [29:21:54<13:24:51, 32.81s/it]
 69%|██████▊   | 3219/4690 [29:22:27<13:27:27, 32.93s/it]
 69%|██████▊   | 3220/4690 [29:23:00<13:23:32, 32.80s/it]
                                                         
{'loss': '0.03087', 'grad_norm': '1.3', 'learning_rate': '2.366e-06', 'epoch': '3.433'}

 69%|██████▊   | 3220/4690 [29:23:00<13:23:32, 32.80s/it]
 69%|██████▊   | 3221/4690 [29:23:33<13:23:28, 32.82s/it]
 69%|██████▊   | 3222/4690 [29:24:06<13:25:41, 32.93s/it]
 69%|██████▊   | 3223/4690 [29:24:39<13:26:37, 32.99s/it]
 69%|██████▊   | 3224/4690 [29:25:12<13:27:41, 33.06s/it]
 69%|██████▉   | 3225/4690 [29:25:45<13:25:48, 33.00s/it]
 69%|██████▉   | 3226/4690 [29:26:18<13:23:28, 32.93s/it]
 69%|██████▉   | 3227/4690 [29:26:50<13:19:23, 32.78s/it]
 69%|██████▉   | 3228/4690 [29:27:23<13:22:00, 32.91s/it]
 69%|██████▉   | 3229/4690 [29:27:56<13:22:10, 32.94s/it]
 69%|██████▉   | 3230/4690 [29:28:29<13:20:17, 32.89s/it]
                                                         
{'loss': '0.03161', 'grad_norm': '1.487', 'learning_rate': '2.336e-06', 'epoch': '3.444'}

 69%|██████▉   | 3230/4690 [29:28:29<13:20:17, 32.89s/it]
 69%|██████▉   | 3231/4690 [29:29:02<13:20:00, 32.90s/it]
 69%|██████▉   | 3232/4690 [29:29:35<13:17:57, 32.84s/it]
 69%|██████▉   | 3233/4690 [29:30:07<13:15:42, 32.77s/it]
 69%|██████▉   | 3234/4690 [29:30:40<13:16:41, 32.83s/it]
 69%|██████▉   | 3235/4690 [29:31:13<13:17:28, 32.89s/it]
 69%|██████▉   | 3236/4690 [29:31:47<13:19:18, 32.98s/it]
 69%|██████▉   | 3237/4690 [29:32:20<13:18:40, 32.98s/it]
 69%|██████▉   | 3238/4690 [29:32:52<13:17:12, 32.94s/it]
 69%|██████▉   | 3239/4690 [29:33:25<13:15:58, 32.91s/it]
 69%|██████▉   | 3240/4690 [29:33:58<13:14:31, 32.88s/it]
                                                         
{'loss': '0.03233', 'grad_norm': '1.221', 'learning_rate': '2.307e-06', 'epoch': '3.454'}

 69%|██████▉   | 3240/4690 [29:33:58<13:14:31, 32.88s/it]
 69%|██████▉   | 3241/4690 [29:34:31<13:12:37, 32.82s/it]
 69%|██████▉   | 3242/4690 [29:35:04<13:13:36, 32.88s/it]
 69%|██████▉   | 3243/4690 [29:35:37<13:16:30, 33.03s/it]
 69%|██████▉   | 3244/4690 [29:36:10<13:15:25, 33.01s/it]
 69%|██████▉   | 3245/4690 [29:36:43<13:15:21, 33.03s/it]
 69%|██████▉   | 3246/4690 [29:37:16<13:13:41, 32.98s/it]
 69%|██████▉   | 3247/4690 [29:37:49<13:12:18, 32.94s/it]
 69%|██████▉   | 3248/4690 [29:38:22<13:13:04, 33.00s/it]
 69%|██████▉   | 3249/4690 [29:38:54<13:08:28, 32.83s/it]
 69%|██████▉   | 3250/4690 [29:39:28<13:11:35, 32.98s/it]
                                                         
{'loss': '0.03106', 'grad_norm': '1.189', 'learning_rate': '2.278e-06', 'epoch': '3.465'}

 69%|██████▉   | 3250/4690 [29:39:28<13:11:35, 32.98s/it]
 69%|██████▉   | 3251/4690 [29:40:01<13:10:12, 32.95s/it]
 69%|██████▉   | 3252/4690 [29:40:33<13:07:02, 32.84s/it]
 69%|██████▉   | 3253/4690 [29:41:06<13:06:59, 32.86s/it]
 69%|██████▉   | 3254/4690 [29:41:39<13:04:22, 32.77s/it]
 69%|██████▉   | 3255/4690 [29:42:12<13:04:50, 32.82s/it]
 69%|██████▉   | 3256/4690 [29:42:44<13:03:19, 32.77s/it]
 69%|██████▉   | 3257/4690 [29:43:17<13:03:52, 32.82s/it]
 69%|██████▉   | 3258/4690 [29:43:50<13:02:48, 32.80s/it]
 69%|██████▉   | 3259/4690 [29:44:23<13:04:51, 32.91s/it]
 70%|██████▉   | 3260/4690 [29:44:56<13:01:27, 32.79s/it]
                                                         
{'loss': '0.03037', 'grad_norm': '1.247', 'learning_rate': '2.249e-06', 'epoch': '3.476'}

 70%|██████▉   | 3260/4690 [29:44:56<13:01:27, 32.79s/it]
 70%|██████▉   | 3261/4690 [29:45:28<12:58:48, 32.70s/it]
 70%|██████▉   | 3262/4690 [29:46:01<12:58:41, 32.72s/it]
 70%|██████▉   | 3263/4690 [29:46:33<12:57:01, 32.67s/it]
 70%|██████▉   | 3264/4690 [29:47:06<12:57:19, 32.71s/it]
 70%|██████▉   | 3265/4690 [29:47:39<12:58:52, 32.79s/it]
 70%|██████▉   | 3266/4690 [29:48:12<12:59:33, 32.85s/it]
 70%|██████▉   | 3267/4690 [29:48:45<12:56:21, 32.73s/it]
 70%|██████▉   | 3268/4690 [29:49:18<12:57:41, 32.81s/it]
 70%|██████▉   | 3269/4690 [29:49:50<12:54:31, 32.70s/it]
 70%|██████▉   | 3270/4690 [29:50:23<12:54:35, 32.73s/it]
                                                         
{'loss': '0.02966', 'grad_norm': '1.292', 'learning_rate': '2.221e-06', 'epoch': '3.486'}

 70%|██████▉   | 3270/4690 [29:50:23<12:54:35, 32.73s/it]
 70%|██████▉   | 3271/4690 [29:50:56<12:54:28, 32.75s/it]
 70%|██████▉   | 3272/4690 [29:51:28<12:52:58, 32.71s/it]
 70%|██████▉   | 3273/4690 [29:52:01<12:50:42, 32.63s/it]
 70%|██████▉   | 3274/4690 [29:52:34<12:50:56, 32.67s/it]
 70%|██████▉   | 3275/4690 [29:53:06<12:51:36, 32.72s/it]
 70%|██████▉   | 3276/4690 [29:53:39<12:48:28, 32.61s/it]
 70%|██████▉   | 3277/4690 [29:54:12<12:49:52, 32.69s/it]
 70%|██████▉   | 3278/4690 [29:54:44<12:48:28, 32.65s/it]
 70%|██████▉   | 3279/4690 [29:55:17<12:48:42, 32.69s/it]
 70%|██████▉   | 3280/4690 [29:55:50<12:51:18, 32.82s/it]
                                                         
{'loss': '0.02784', 'grad_norm': '1.471', 'learning_rate': '2.192e-06', 'epoch': '3.497'}

 70%|██████▉   | 3280/4690 [29:55:50<12:51:18, 32.82s/it]
 70%|██████▉   | 3281/4690 [29:56:23<12:49:45, 32.78s/it]
 70%|██████▉   | 3282/4690 [29:56:55<12:46:10, 32.65s/it]
 70%|███████   | 3283/4690 [29:57:28<12:47:47, 32.74s/it]
 70%|███████   | 3284/4690 [29:58:01<12:46:23, 32.71s/it]
 70%|███████   | 3285/4690 [29:58:34<12:48:22, 32.81s/it]
 70%|███████   | 3286/4690 [29:59:07<12:47:50, 32.81s/it]
 70%|███████   | 3287/4690 [29:59:40<12:50:50, 32.97s/it]
 70%|███████   | 3288/4690 [30:00:13<12:49:59, 32.95s/it]
 70%|███████   | 3289/4690 [30:00:46<12:48:54, 32.93s/it]
 70%|███████   | 3290/4690 [30:01:19<12:48:35, 32.94s/it]
                                                         
{'loss': '0.03171', 'grad_norm': '1.836', 'learning_rate': '2.163e-06', 'epoch': '3.508'}

 70%|███████   | 3290/4690 [30:01:19<12:48:35, 32.94s/it]
 70%|███████   | 3291/4690 [30:01:52<12:49:33, 33.00s/it]
 70%|███████   | 3292/4690 [30:02:25<12:47:54, 32.96s/it]
 70%|███████   | 3293/4690 [30:02:57<12:45:43, 32.89s/it]
 70%|███████   | 3294/4690 [30:03:30<12:42:55, 32.79s/it]
 70%|███████   | 3295/4690 [30:04:03<12:41:06, 32.74s/it]
 70%|███████   | 3296/4690 [30:04:35<12:40:23, 32.73s/it]
 70%|███████   | 3297/4690 [30:05:08<12:41:37, 32.81s/it]
 70%|███████   | 3298/4690 [30:05:41<12:41:37, 32.83s/it]
 70%|███████   | 3299/4690 [30:06:14<12:42:49, 32.90s/it]
 70%|███████   | 3300/4690 [30:06:47<12:42:04, 32.90s/it]
                                                         
{'loss': '0.02956', 'grad_norm': '1.404', 'learning_rate': '2.135e-06', 'epoch': '3.518'}

 70%|███████   | 3300/4690 [30:06:47<12:42:04, 32.90s/it]
 70%|███████   | 3301/4690 [30:07:20<12:41:18, 32.89s/it]
 70%|███████   | 3302/4690 [30:07:53<12:40:33, 32.88s/it]
 70%|███████   | 3303/4690 [30:08:25<12:37:51, 32.78s/it]
 70%|███████   | 3304/4690 [30:08:58<12:39:26, 32.88s/it]
 70%|███████   | 3305/4690 [30:09:31<12:35:27, 32.73s/it]
 70%|███████   | 3306/4690 [30:10:04<12:36:43, 32.81s/it]
 71%|███████   | 3307/4690 [30:10:36<12:34:22, 32.73s/it]
 71%|███████   | 3308/4690 [30:11:09<12:36:05, 32.83s/it]
 71%|███████   | 3309/4690 [30:11:42<12:35:19, 32.82s/it]
 71%|███████   | 3310/4690 [30:12:15<12:35:12, 32.84s/it]
                                                         
{'loss': '0.0303', 'grad_norm': '1.289', 'learning_rate': '2.107e-06', 'epoch': '3.529'}

 71%|███████   | 3310/4690 [30:12:15<12:35:12, 32.84s/it]
 71%|███████   | 3311/4690 [30:12:48<12:35:05, 32.85s/it]
 71%|███████   | 3312/4690 [30:13:21<12:34:58, 32.87s/it]
 71%|███████   | 3313/4690 [30:13:54<12:33:04, 32.81s/it]
 71%|███████   | 3314/4690 [30:14:26<12:32:53, 32.83s/it]
 71%|███████   | 3315/4690 [30:14:59<12:33:31, 32.88s/it]
 71%|███████   | 3316/4690 [30:15:32<12:32:19, 32.85s/it]
 71%|███████   | 3317/4690 [30:16:05<12:31:51, 32.86s/it]
 71%|███████   | 3318/4690 [30:16:38<12:31:14, 32.85s/it]
 71%|███████   | 3319/4690 [30:17:11<12:30:55, 32.86s/it]
 71%|███████   | 3320/4690 [30:17:43<12:26:38, 32.70s/it]
                                                         
{'loss': '0.03073', 'grad_norm': '1.316', 'learning_rate': '2.079e-06', 'epoch': '3.54'}

 71%|███████   | 3320/4690 [30:17:43<12:26:38, 32.70s/it]
 71%|███████   | 3321/4690 [30:18:16<12:28:36, 32.81s/it]
 71%|███████   | 3322/4690 [30:18:49<12:24:30, 32.65s/it]
 71%|███████   | 3323/4690 [30:19:21<12:24:28, 32.68s/it]
 71%|███████   | 3324/4690 [30:19:54<12:26:57, 32.81s/it]
 71%|███████   | 3325/4690 [30:20:27<12:26:26, 32.81s/it]
 71%|███████   | 3326/4690 [30:21:00<12:25:55, 32.81s/it]
 71%|███████   | 3327/4690 [30:21:33<12:23:18, 32.72s/it]
 71%|███████   | 3328/4690 [30:22:05<12:22:55, 32.73s/it]
 71%|███████   | 3329/4690 [30:22:38<12:22:45, 32.74s/it]
 71%|███████   | 3330/4690 [30:23:11<12:22:06, 32.74s/it]
                                                         
{'loss': '0.03001', 'grad_norm': '1.493', 'learning_rate': '2.051e-06', 'epoch': '3.55'}

 71%|███████   | 3330/4690 [30:23:11<12:22:06, 32.74s/it]
 71%|███████   | 3331/4690 [30:23:43<12:20:40, 32.70s/it]
 71%|███████   | 3332/4690 [30:24:16<12:19:54, 32.69s/it]
 71%|███████   | 3333/4690 [30:24:49<12:20:21, 32.74s/it]
 71%|███████   | 3334/4690 [30:25:22<12:22:43, 32.86s/it]
 71%|███████   | 3335/4690 [30:25:55<12:23:52, 32.94s/it]
 71%|███████   | 3336/4690 [30:26:28<12:23:08, 32.93s/it]
 71%|███████   | 3337/4690 [30:27:01<12:23:12, 32.96s/it]
 71%|███████   | 3338/4690 [30:27:34<12:22:25, 32.95s/it]
 71%|███████   | 3339/4690 [30:28:07<12:23:54, 33.04s/it]
 71%|███████   | 3340/4690 [30:28:40<12:21:37, 32.96s/it]
                                                         
{'loss': '0.03015', 'grad_norm': '1.564', 'learning_rate': '2.023e-06', 'epoch': '3.561'}

 71%|███████   | 3340/4690 [30:28:40<12:21:37, 32.96s/it]
 71%|███████   | 3341/4690 [30:29:13<12:20:40, 32.94s/it]
 71%|███████▏  | 3342/4690 [30:29:46<12:19:31, 32.92s/it]
 71%|███████▏  | 3343/4690 [30:30:19<12:20:11, 32.97s/it]
 71%|███████▏  | 3344/4690 [30:30:51<12:16:35, 32.83s/it]
 71%|███████▏  | 3345/4690 [30:31:24<12:13:48, 32.74s/it]
 71%|███████▏  | 3346/4690 [30:31:57<12:15:48, 32.85s/it]
 71%|███████▏  | 3347/4690 [30:32:30<12:16:38, 32.91s/it]
 71%|███████▏  | 3348/4690 [30:33:03<12:16:07, 32.91s/it]
 71%|███████▏  | 3349/4690 [30:33:36<12:17:04, 32.98s/it]
 71%|███████▏  | 3350/4690 [30:34:09<12:13:24, 32.84s/it]
                                                         
{'loss': '0.02931', 'grad_norm': '1.259', 'learning_rate': '1.995e-06', 'epoch': '3.572'}

 71%|███████▏  | 3350/4690 [30:34:09<12:13:24, 32.84s/it]
 71%|███████▏  | 3351/4690 [30:34:42<12:15:43, 32.97s/it]
 71%|███████▏  | 3352/4690 [30:35:15<12:15:03, 32.96s/it]
 71%|███████▏  | 3353/4690 [30:35:48<12:15:28, 33.01s/it]
 72%|███████▏  | 3354/4690 [30:36:21<12:13:28, 32.94s/it]
 72%|███████▏  | 3355/4690 [30:36:54<12:15:45, 33.07s/it]
 72%|███████▏  | 3356/4690 [30:37:27<12:12:27, 32.94s/it]
 72%|███████▏  | 3357/4690 [30:38:00<12:11:21, 32.92s/it]
 72%|███████▏  | 3358/4690 [30:38:33<12:10:18, 32.90s/it]
 72%|███████▏  | 3359/4690 [30:39:06<12:10:49, 32.94s/it]
 72%|███████▏  | 3360/4690 [30:39:39<12:10:44, 32.97s/it]
                                                         
{'loss': '0.02973', 'grad_norm': '1.056', 'learning_rate': '1.968e-06', 'epoch': '3.582'}

 72%|███████▏  | 3360/4690 [30:39:39<12:10:44, 32.97s/it]
 72%|███████▏  | 3361/4690 [30:40:11<12:09:39, 32.94s/it]
 72%|███████▏  | 3362/4690 [30:40:44<12:08:00, 32.89s/it]
 72%|███████▏  | 3363/4690 [30:41:17<12:06:48, 32.86s/it]
 72%|███████▏  | 3364/4690 [30:41:50<12:05:50, 32.84s/it]
 72%|███████▏  | 3365/4690 [30:42:23<12:05:42, 32.86s/it]
 72%|███████▏  | 3366/4690 [30:42:56<12:04:34, 32.84s/it]
 72%|███████▏  | 3367/4690 [30:43:28<12:03:33, 32.81s/it]
 72%|███████▏  | 3368/4690 [30:44:01<12:03:28, 32.84s/it]
 72%|███████▏  | 3369/4690 [30:44:34<12:01:13, 32.76s/it]
 72%|███████▏  | 3370/4690 [30:45:07<12:03:25, 32.88s/it]
                                                         
{'loss': '0.02883', 'grad_norm': '1.415', 'learning_rate': '1.94e-06', 'epoch': '3.593'}

 72%|███████▏  | 3370/4690 [30:45:07<12:03:25, 32.88s/it]
 72%|███████▏  | 3371/4690 [30:45:40<12:03:01, 32.89s/it]
 72%|███████▏  | 3372/4690 [30:46:13<12:04:04, 32.96s/it]
 72%|███████▏  | 3373/4690 [30:46:46<12:01:54, 32.89s/it]
 72%|███████▏  | 3374/4690 [30:47:19<12:01:05, 32.88s/it]
 72%|███████▏  | 3375/4690 [30:47:51<11:59:36, 32.83s/it]
 72%|███████▏  | 3376/4690 [30:48:24<11:58:20, 32.80s/it]
 72%|███████▏  | 3377/4690 [30:48:57<11:59:01, 32.86s/it]
 72%|███████▏  | 3378/4690 [30:49:30<11:58:38, 32.86s/it]
 72%|███████▏  | 3379/4690 [30:50:03<11:58:43, 32.89s/it]
 72%|███████▏  | 3380/4690 [30:50:36<12:00:59, 33.02s/it]
                                                         
{'loss': '0.031', 'grad_norm': '1.469', 'learning_rate': '1.913e-06', 'epoch': '3.604'}

 72%|███████▏  | 3380/4690 [30:50:36<12:00:59, 33.02s/it]
 72%|███████▏  | 3381/4690 [30:51:09<12:00:34, 33.03s/it]
 72%|███████▏  | 3382/4690 [30:51:42<11:56:49, 32.88s/it]
 72%|███████▏  | 3383/4690 [30:52:15<11:55:41, 32.85s/it]
 72%|███████▏  | 3384/4690 [30:52:48<11:56:06, 32.90s/it]
 72%|███████▏  | 3385/4690 [30:53:20<11:56:05, 32.92s/it]
 72%|███████▏  | 3386/4690 [30:53:54<11:56:13, 32.95s/it]
 72%|███████▏  | 3387/4690 [30:54:26<11:53:51, 32.87s/it]
 72%|███████▏  | 3388/4690 [30:54:59<11:54:10, 32.91s/it]
 72%|███████▏  | 3389/4690 [30:55:32<11:53:53, 32.92s/it]
 72%|███████▏  | 3390/4690 [30:56:05<11:52:36, 32.89s/it]
                                                         
{'loss': '0.03084', 'grad_norm': '1.356', 'learning_rate': '1.886e-06', 'epoch': '3.614'}

 72%|███████▏  | 3390/4690 [30:56:05<11:52:36, 32.89s/it]
 72%|███████▏  | 3391/4690 [30:56:38<11:52:46, 32.92s/it]
 72%|███████▏  | 3392/4690 [30:57:11<11:51:17, 32.88s/it]
 72%|███████▏  | 3393/4690 [30:57:44<11:52:20, 32.95s/it]
 72%|███████▏  | 3394/4690 [30:58:17<11:52:25, 32.98s/it]
 72%|███████▏  | 3395/4690 [30:58:50<11:51:04, 32.95s/it]
 72%|███████▏  | 3396/4690 [30:59:22<11:47:53, 32.82s/it]
 72%|███████▏  | 3397/4690 [30:59:55<11:46:48, 32.80s/it]
 72%|███████▏  | 3398/4690 [31:00:28<11:46:19, 32.80s/it]
 72%|███████▏  | 3399/4690 [31:01:01<11:46:49, 32.85s/it]
 72%|███████▏  | 3400/4690 [31:01:34<11:47:53, 32.92s/it]
                                                         
{'loss': '0.03042', 'grad_norm': '1.189', 'learning_rate': '1.859e-06', 'epoch': '3.625'}

 72%|███████▏  | 3400/4690 [31:01:34<11:47:53, 32.92s/it]
 73%|███████▎  | 3401/4690 [31:02:06<11:44:56, 32.81s/it]
 73%|███████▎  | 3402/4690 [31:02:39<11:44:16, 32.81s/it]
 73%|███████▎  | 3403/4690 [31:03:12<11:45:07, 32.87s/it]
 73%|███████▎  | 3404/4690 [31:03:45<11:44:38, 32.88s/it]
 73%|███████▎  | 3405/4690 [31:04:18<11:44:00, 32.87s/it]
 73%|███████▎  | 3406/4690 [31:04:50<11:40:47, 32.75s/it]
 73%|███████▎  | 3407/4690 [31:05:23<11:41:07, 32.79s/it]
 73%|███████▎  | 3408/4690 [31:05:56<11:40:52, 32.80s/it]
 73%|███████▎  | 3409/4690 [31:06:29<11:40:12, 32.80s/it]
 73%|███████▎  | 3410/4690 [31:07:02<11:38:48, 32.76s/it]
                                                         
{'loss': '0.02937', 'grad_norm': '1.506', 'learning_rate': '1.832e-06', 'epoch': '3.636'}

 73%|███████▎  | 3410/4690 [31:07:02<11:38:48, 32.76s/it]
 73%|███████▎  | 3411/4690 [31:07:34<11:37:55, 32.74s/it]
 73%|███████▎  | 3412/4690 [31:08:07<11:37:46, 32.76s/it]
 73%|███████▎  | 3413/4690 [31:08:40<11:38:19, 32.81s/it]
 73%|███████▎  | 3414/4690 [31:09:13<11:38:11, 32.83s/it]
 73%|███████▎  | 3415/4690 [31:09:46<11:38:40, 32.88s/it]
 73%|███████▎  | 3416/4690 [31:10:18<11:33:52, 32.68s/it]
 73%|███████▎  | 3417/4690 [31:10:51<11:36:01, 32.81s/it]
 73%|███████▎  | 3418/4690 [31:11:24<11:33:59, 32.74s/it]
 73%|███████▎  | 3419/4690 [31:11:57<11:36:37, 32.89s/it]
 73%|███████▎  | 3420/4690 [31:12:29<11:31:59, 32.69s/it]
                                                         
{'loss': '0.03032', 'grad_norm': '1.331', 'learning_rate': '1.806e-06', 'epoch': '3.646'}

 73%|███████▎  | 3420/4690 [31:12:29<11:31:59, 32.69s/it]
 73%|███████▎  | 3421/4690 [31:13:02<11:30:20, 32.64s/it]
 73%|███████▎  | 3422/4690 [31:13:35<11:31:06, 32.70s/it]
 73%|███████▎  | 3423/4690 [31:14:08<11:31:23, 32.74s/it]
 73%|███████▎  | 3424/4690 [31:14:40<11:29:33, 32.68s/it]
 73%|███████▎  | 3425/4690 [31:15:13<11:30:05, 32.73s/it]
 73%|███████▎  | 3426/4690 [31:15:46<11:30:59, 32.80s/it]
 73%|███████▎  | 3427/4690 [31:16:19<11:29:29, 32.75s/it]
 73%|███████▎  | 3428/4690 [31:16:51<11:29:11, 32.77s/it]
 73%|███████▎  | 3429/4690 [31:17:24<11:28:10, 32.74s/it]
 73%|███████▎  | 3430/4690 [31:17:57<11:29:16, 32.82s/it]
                                                         
{'loss': '0.03077', 'grad_norm': '1.7', 'learning_rate': '1.779e-06', 'epoch': '3.657'}

 73%|███████▎  | 3430/4690 [31:17:57<11:29:16, 32.82s/it]
 73%|███████▎  | 3431/4690 [31:18:30<11:27:42, 32.77s/it]
 73%|███████▎  | 3432/4690 [31:19:02<11:27:18, 32.78s/it]
 73%|███████▎  | 3433/4690 [31:19:35<11:24:29, 32.67s/it]
 73%|███████▎  | 3434/4690 [31:20:07<11:23:15, 32.64s/it]
 73%|███████▎  | 3435/4690 [31:20:40<11:25:00, 32.75s/it]
 73%|███████▎  | 3436/4690 [31:21:13<11:26:05, 32.83s/it]
 73%|███████▎  | 3437/4690 [31:21:46<11:24:51, 32.79s/it]
 73%|███████▎  | 3438/4690 [31:22:19<11:21:22, 32.65s/it]
 73%|███████▎  | 3439/4690 [31:22:52<11:24:44, 32.84s/it]
 73%|███████▎  | 3440/4690 [31:23:25<11:23:45, 32.82s/it]
                                                         
{'loss': '0.03309', 'grad_norm': '1.294', 'learning_rate': '1.753e-06', 'epoch': '3.668'}

 73%|███████▎  | 3440/4690 [31:23:25<11:23:45, 32.82s/it]
 73%|███████▎  | 3441/4690 [31:23:57<11:22:22, 32.78s/it]
 73%|███████▎  | 3442/4690 [31:24:30<11:18:49, 32.64s/it]
 73%|███████▎  | 3443/4690 [31:25:03<11:21:40, 32.80s/it]
 73%|███████▎  | 3444/4690 [31:25:36<11:22:12, 32.85s/it]
 73%|███████▎  | 3445/4690 [31:26:08<11:19:24, 32.74s/it]
 73%|███████▎  | 3446/4690 [31:26:41<11:18:04, 32.70s/it]
 73%|███████▎  | 3447/4690 [31:27:14<11:19:09, 32.78s/it]
 74%|███████▎  | 3448/4690 [31:27:47<11:18:36, 32.78s/it]
 74%|███████▎  | 3449/4690 [31:28:19<11:16:30, 32.71s/it]
 74%|███████▎  | 3450/4690 [31:28:52<11:14:10, 32.62s/it]
                                                         
{'loss': '0.02782', 'grad_norm': '1.445', 'learning_rate': '1.727e-06', 'epoch': '3.678'}

 74%|███████▎  | 3450/4690 [31:28:52<11:14:10, 32.62s/it]
 74%|███████▎  | 3451/4690 [31:29:24<11:15:43, 32.72s/it]
 74%|███████▎  | 3452/4690 [31:29:57<11:16:12, 32.77s/it]
 74%|███████▎  | 3453/4690 [31:30:31<11:18:08, 32.89s/it]
 74%|███████▎  | 3454/4690 [31:31:04<11:18:25, 32.93s/it]
 74%|███████▎  | 3455/4690 [31:31:36<11:16:20, 32.86s/it]
 74%|███████▎  | 3456/4690 [31:32:09<11:15:18, 32.84s/it]
 74%|███████▎  | 3457/4690 [31:32:42<11:12:58, 32.75s/it]
 74%|███████▎  | 3458/4690 [31:33:14<11:12:33, 32.75s/it]
 74%|███████▍  | 3459/4690 [31:33:47<11:11:35, 32.73s/it]
 74%|███████▍  | 3460/4690 [31:34:20<11:11:58, 32.78s/it]
                                                         
{'loss': '0.03135', 'grad_norm': '1.541', 'learning_rate': '1.701e-06', 'epoch': '3.689'}

 74%|███████▍  | 3460/4690 [31:34:20<11:11:58, 32.78s/it]
 74%|███████▍  | 3461/4690 [31:34:53<11:10:19, 32.73s/it]
 74%|███████▍  | 3462/4690 [31:35:26<11:13:22, 32.90s/it]
 74%|███████▍  | 3463/4690 [31:35:59<11:12:22, 32.88s/it]
 74%|███████▍  | 3464/4690 [31:36:31<11:10:53, 32.83s/it]
 74%|███████▍  | 3465/4690 [31:37:04<11:11:44, 32.90s/it]
 74%|███████▍  | 3466/4690 [31:37:37<11:11:33, 32.92s/it]
 74%|███████▍  | 3467/4690 [31:38:10<11:11:14, 32.93s/it]
 74%|███████▍  | 3468/4690 [31:38:43<11:08:29, 32.82s/it]
 74%|███████▍  | 3469/4690 [31:39:16<11:11:40, 33.01s/it]
 74%|███████▍  | 3470/4690 [31:39:49<11:10:40, 32.98s/it]
                                                         
{'loss': '0.03063', 'grad_norm': '1.193', 'learning_rate': '1.675e-06', 'epoch': '3.7'}

 74%|███████▍  | 3470/4690 [31:39:49<11:10:40, 32.98s/it]
 74%|███████▍  | 3471/4690 [31:40:22<11:08:02, 32.88s/it]
 74%|███████▍  | 3472/4690 [31:40:55<11:08:33, 32.93s/it]
 74%|███████▍  | 3473/4690 [31:41:28<11:06:20, 32.85s/it]
 74%|███████▍  | 3474/4690 [31:42:01<11:08:18, 32.98s/it]
 74%|███████▍  | 3475/4690 [31:42:34<11:09:34, 33.07s/it]
 74%|███████▍  | 3476/4690 [31:43:07<11:07:45, 33.00s/it]
 74%|███████▍  | 3477/4690 [31:43:40<11:06:50, 32.98s/it]
 74%|███████▍  | 3478/4690 [31:44:13<11:08:39, 33.10s/it]
 74%|███████▍  | 3479/4690 [31:44:46<11:05:06, 32.95s/it]
 74%|███████▍  | 3480/4690 [31:45:18<11:01:50, 32.82s/it]
                                                         
{'loss': '0.02883', 'grad_norm': '1.23', 'learning_rate': '1.649e-06', 'epoch': '3.71'}

 74%|███████▍  | 3480/4690 [31:45:18<11:01:50, 32.82s/it]
 74%|███████▍  | 3481/4690 [31:45:51<11:00:38, 32.79s/it]
 74%|███████▍  | 3482/4690 [31:46:24<10:57:45, 32.67s/it]
 74%|███████▍  | 3483/4690 [31:46:57<10:58:51, 32.75s/it]
 74%|███████▍  | 3484/4690 [31:47:29<10:58:17, 32.75s/it]
 74%|███████▍  | 3485/4690 [31:48:02<10:58:08, 32.77s/it]
 74%|███████▍  | 3486/4690 [31:48:35<10:58:43, 32.83s/it]
 74%|███████▍  | 3487/4690 [31:49:08<10:58:18, 32.83s/it]
 74%|███████▍  | 3488/4690 [31:49:41<10:58:11, 32.86s/it]
 74%|███████▍  | 3489/4690 [31:50:14<10:59:34, 32.95s/it]
 74%|███████▍  | 3490/4690 [31:50:47<10:58:56, 32.95s/it]
                                                         
{'loss': '0.02828', 'grad_norm': '1.21', 'learning_rate': '1.623e-06', 'epoch': '3.721'}

 74%|███████▍  | 3490/4690 [31:50:47<10:58:56, 32.95s/it]
 74%|███████▍  | 3491/4690 [31:51:19<10:55:23, 32.80s/it]
 74%|███████▍  | 3492/4690 [31:51:52<10:52:49, 32.70s/it]
 74%|███████▍  | 3493/4690 [31:52:25<10:54:15, 32.79s/it]
 74%|███████▍  | 3494/4690 [31:52:58<10:55:01, 32.86s/it]
 75%|███████▍  | 3495/4690 [31:53:31<10:54:02, 32.84s/it]
 75%|███████▍  | 3496/4690 [31:54:03<10:50:40, 32.70s/it]
 75%|███████▍  | 3497/4690 [31:54:36<10:49:56, 32.69s/it]
 75%|███████▍  | 3498/4690 [31:55:08<10:49:42, 32.70s/it]
 75%|███████▍  | 3499/4690 [31:55:41<10:49:01, 32.70s/it]
 75%|███████▍  | 3500/4690 [31:56:14<10:50:08, 32.78s/it]
                                                         
{'loss': '0.03226', 'grad_norm': '1.361', 'learning_rate': '1.598e-06', 'epoch': '3.732'}

 75%|███████▍  | 3500/4690 [31:56:14<10:50:08, 32.78s/it]
 75%|███████▍  | 3501/4690 [31:56:47<10:48:28, 32.72s/it]
 75%|███████▍  | 3502/4690 [31:57:19<10:46:46, 32.67s/it]
 75%|███████▍  | 3503/4690 [31:57:52<10:46:55, 32.70s/it]
 75%|███████▍  | 3504/4690 [31:58:25<10:46:22, 32.70s/it]
 75%|███████▍  | 3505/4690 [31:58:58<10:46:38, 32.74s/it]
 75%|███████▍  | 3506/4690 [31:59:30<10:46:53, 32.78s/it]
 75%|███████▍  | 3507/4690 [32:00:03<10:47:31, 32.84s/it]
 75%|███████▍  | 3508/4690 [32:00:37<10:49:09, 32.95s/it]
 75%|███████▍  | 3509/4690 [32:01:09<10:48:05, 32.93s/it]
 75%|███████▍  | 3510/4690 [32:01:42<10:45:07, 32.80s/it]
                                                         
{'loss': '0.02841', 'grad_norm': '1.257', 'learning_rate': '1.573e-06', 'epoch': '3.742'}

 75%|███████▍  | 3510/4690 [32:01:42<10:45:07, 32.80s/it]
 75%|███████▍  | 3511/4690 [32:02:15<10:43:15, 32.74s/it]
 75%|███████▍  | 3512/4690 [32:02:47<10:42:52, 32.74s/it]
 75%|███████▍  | 3513/4690 [32:03:20<10:43:35, 32.81s/it]
 75%|███████▍  | 3514/4690 [32:03:53<10:42:54, 32.80s/it]
 75%|███████▍  | 3515/4690 [32:04:26<10:41:41, 32.77s/it]
 75%|███████▍  | 3516/4690 [32:04:58<10:40:53, 32.75s/it]
 75%|███████▍  | 3517/4690 [32:05:31<10:41:54, 32.83s/it]
 75%|███████▌  | 3518/4690 [32:06:05<10:42:39, 32.90s/it]
 75%|███████▌  | 3519/4690 [32:06:37<10:40:43, 32.83s/it]
 75%|███████▌  | 3520/4690 [32:07:10<10:39:34, 32.80s/it]
                                                         
{'loss': '0.02861', 'grad_norm': '1.129', 'learning_rate': '1.548e-06', 'epoch': '3.753'}

 75%|███████▌  | 3520/4690 [32:07:10<10:39:34, 32.80s/it]
 75%|███████▌  | 3521/4690 [32:07:43<10:38:19, 32.76s/it]
 75%|███████▌  | 3522/4690 [32:08:15<10:37:39, 32.76s/it]
 75%|███████▌  | 3523/4690 [32:08:48<10:37:02, 32.75s/it]
 75%|███████▌  | 3524/4690 [32:09:21<10:35:58, 32.73s/it]
 75%|███████▌  | 3525/4690 [32:09:54<10:36:49, 32.80s/it]
 75%|███████▌  | 3526/4690 [32:10:27<10:37:04, 32.84s/it]
 75%|███████▌  | 3527/4690 [32:11:00<10:37:50, 32.91s/it]
 75%|███████▌  | 3528/4690 [32:11:32<10:36:03, 32.84s/it]
 75%|███████▌  | 3529/4690 [32:12:06<10:36:56, 32.92s/it]
 75%|███████▌  | 3530/4690 [32:12:38<10:35:06, 32.85s/it]
                                                         
{'loss': '0.03038', 'grad_norm': '1.362', 'learning_rate': '1.523e-06', 'epoch': '3.764'}

 75%|███████▌  | 3530/4690 [32:12:38<10:35:06, 32.85s/it]
 75%|███████▌  | 3531/4690 [32:13:11<10:35:45, 32.91s/it]
 75%|███████▌  | 3532/4690 [32:13:44<10:34:29, 32.88s/it]
 75%|███████▌  | 3533/4690 [32:14:17<10:35:25, 32.95s/it]
 75%|███████▌  | 3534/4690 [32:14:50<10:34:33, 32.94s/it]
 75%|███████▌  | 3535/4690 [32:15:23<10:36:09, 33.05s/it]
 75%|███████▌  | 3536/4690 [32:15:56<10:33:11, 32.92s/it]
 75%|███████▌  | 3537/4690 [32:16:29<10:30:16, 32.80s/it]
 75%|███████▌  | 3538/4690 [32:17:01<10:28:45, 32.75s/it]
 75%|███████▌  | 3539/4690 [32:17:34<10:28:16, 32.75s/it]
 75%|███████▌  | 3540/4690 [32:18:07<10:28:53, 32.81s/it]
                                                         
{'loss': '0.0283', 'grad_norm': '1.363', 'learning_rate': '1.498e-06', 'epoch': '3.774'}

 75%|███████▌  | 3540/4690 [32:18:07<10:28:53, 32.81s/it]
 76%|███████▌  | 3541/4690 [32:18:40<10:29:25, 32.87s/it]
 76%|███████▌  | 3542/4690 [32:19:13<10:27:37, 32.80s/it]
 76%|███████▌  | 3543/4690 [32:19:45<10:26:57, 32.80s/it]
 76%|███████▌  | 3544/4690 [32:20:18<10:26:52, 32.82s/it]
 76%|███████▌  | 3545/4690 [32:20:51<10:23:53, 32.69s/it]
 76%|███████▌  | 3546/4690 [32:21:23<10:24:08, 32.73s/it]
 76%|███████▌  | 3547/4690 [32:21:56<10:23:28, 32.73s/it]
 76%|███████▌  | 3548/4690 [32:22:29<10:24:19, 32.80s/it]
 76%|███████▌  | 3549/4690 [32:23:02<10:23:50, 32.80s/it]
 76%|███████▌  | 3550/4690 [32:23:35<10:22:33, 32.77s/it]
                                                         
{'loss': '0.02884', 'grad_norm': '1.574', 'learning_rate': '1.474e-06', 'epoch': '3.785'}

 76%|███████▌  | 3550/4690 [32:23:35<10:22:33, 32.77s/it]
 76%|███████▌  | 3551/4690 [32:24:07<10:22:48, 32.81s/it]
 76%|███████▌  | 3552/4690 [32:24:40<10:21:27, 32.77s/it]
 76%|███████▌  | 3553/4690 [32:25:13<10:21:39, 32.81s/it]
 76%|███████▌  | 3554/4690 [32:25:45<10:18:59, 32.69s/it]
 76%|███████▌  | 3555/4690 [32:26:18<10:16:56, 32.61s/it]
 76%|███████▌  | 3556/4690 [32:26:51<10:18:19, 32.72s/it]
 76%|███████▌  | 3557/4690 [32:27:23<10:17:12, 32.69s/it]
 76%|███████▌  | 3558/4690 [32:27:56<10:16:50, 32.70s/it]
 76%|███████▌  | 3559/4690 [32:28:29<10:16:24, 32.70s/it]
 76%|███████▌  | 3560/4690 [32:29:01<10:14:40, 32.64s/it]
                                                         
{'loss': '0.02712', 'grad_norm': '1.603', 'learning_rate': '1.449e-06', 'epoch': '3.796'}

 76%|███████▌  | 3560/4690 [32:29:01<10:14:40, 32.64s/it]
 76%|███████▌  | 3561/4690 [32:29:34<10:16:41, 32.77s/it]
 76%|███████▌  | 3562/4690 [32:30:07<10:16:48, 32.81s/it]
 76%|███████▌  | 3563/4690 [32:30:41<10:18:05, 32.91s/it]
 76%|███████▌  | 3564/4690 [32:31:13<10:16:51, 32.87s/it]
 76%|███████▌  | 3565/4690 [32:31:46<10:17:28, 32.93s/it]
 76%|███████▌  | 3566/4690 [32:32:19<10:16:14, 32.90s/it]
 76%|███████▌  | 3567/4690 [32:32:52<10:15:13, 32.87s/it]
 76%|███████▌  | 3568/4690 [32:33:24<10:11:16, 32.69s/it]
 76%|███████▌  | 3569/4690 [32:33:57<10:11:56, 32.75s/it]
 76%|███████▌  | 3570/4690 [32:34:30<10:11:14, 32.75s/it]
                                                         
{'loss': '0.02797', 'grad_norm': '1.479', 'learning_rate': '1.425e-06', 'epoch': '3.806'}

 76%|███████▌  | 3570/4690 [32:34:30<10:11:14, 32.75s/it]
 76%|███████▌  | 3571/4690 [32:35:03<10:10:11, 32.72s/it]
 76%|███████▌  | 3572/4690 [32:35:35<10:09:48, 32.73s/it]
 76%|███████▌  | 3573/4690 [32:36:08<10:10:14, 32.78s/it]
 76%|███████▌  | 3574/4690 [32:36:41<10:10:53, 32.84s/it]
 76%|███████▌  | 3575/4690 [32:37:14<10:08:33, 32.75s/it]
 76%|███████▌  | 3576/4690 [32:37:46<10:07:33, 32.72s/it]
 76%|███████▋  | 3577/4690 [32:38:19<10:06:07, 32.68s/it]
 76%|███████▋  | 3578/4690 [32:38:52<10:06:15, 32.71s/it]
 76%|███████▋  | 3579/4690 [32:39:25<10:07:40, 32.82s/it]
 76%|███████▋  | 3580/4690 [32:39:57<10:05:09, 32.71s/it]
                                                         
{'loss': '0.02881', 'grad_norm': '1.21', 'learning_rate': '1.401e-06', 'epoch': '3.817'}

 76%|███████▋  | 3580/4690 [32:39:57<10:05:09, 32.71s/it]
 76%|███████▋  | 3581/4690 [32:40:30<10:07:09, 32.85s/it]
 76%|███████▋  | 3582/4690 [32:41:04<10:09:33, 33.01s/it]
 76%|███████▋  | 3583/4690 [32:41:36<10:05:31, 32.82s/it]
 76%|███████▋  | 3584/4690 [32:42:09<10:07:07, 32.94s/it]
 76%|███████▋  | 3585/4690 [32:42:42<10:06:05, 32.91s/it]
 76%|███████▋  | 3586/4690 [32:43:15<10:06:11, 32.94s/it]
 76%|███████▋  | 3587/4690 [32:43:48<10:04:53, 32.90s/it]
 77%|███████▋  | 3588/4690 [32:44:21<10:04:45, 32.93s/it]
 77%|███████▋  | 3589/4690 [32:44:54<10:02:45, 32.85s/it]
 77%|███████▋  | 3590/4690 [32:45:26<10:00:21, 32.75s/it]
                                                         
{'loss': '0.02742', 'grad_norm': '1.128', 'learning_rate': '1.377e-06', 'epoch': '3.828'}

 77%|███████▋  | 3590/4690 [32:45:26<10:00:21, 32.75s/it]
 77%|███████▋  | 3591/4690 [32:45:59<10:02:02, 32.87s/it]
 77%|███████▋  | 3592/4690 [32:46:32<10:00:51, 32.83s/it]
 77%|███████▋  | 3593/4690 [32:47:05<9:58:39, 32.74s/it] 
 77%|███████▋  | 3594/4690 [32:47:37<9:57:35, 32.72s/it]
 77%|███████▋  | 3595/4690 [32:48:10<9:57:56, 32.76s/it]
 77%|███████▋  | 3596/4690 [32:48:43<9:57:07, 32.75s/it]
 77%|███████▋  | 3597/4690 [32:49:16<9:58:31, 32.86s/it]
 77%|███████▋  | 3598/4690 [32:49:49<9:58:38, 32.89s/it]
 77%|███████▋  | 3599/4690 [32:50:22<9:57:39, 32.87s/it]
 77%|███████▋  | 3600/4690 [32:50:54<9:54:39, 32.73s/it]
                                                        
{'loss': '0.03004', 'grad_norm': '1.267', 'learning_rate': '1.353e-06', 'epoch': '3.838'}

 77%|███████▋  | 3600/4690 [32:50:54<9:54:39, 32.73s/it]
 77%|███████▋  | 3601/4690 [32:51:27<9:53:29, 32.70s/it]
 77%|███████▋  | 3602/4690 [32:52:00<9:53:56, 32.75s/it]
 77%|███████▋  | 3603/4690 [32:52:33<9:53:27, 32.76s/it]
 77%|███████▋  | 3604/4690 [32:53:05<9:53:19, 32.78s/it]
 77%|███████▋  | 3605/4690 [32:53:38<9:53:22, 32.81s/it]
 77%|███████▋  | 3606/4690 [32:54:11<9:53:45, 32.87s/it]
 77%|███████▋  | 3607/4690 [32:54:45<9:56:22, 33.04s/it]
 77%|███████▋  | 3608/4690 [32:55:17<9:54:39, 32.98s/it]
 77%|███████▋  | 3609/4690 [32:55:51<9:56:14, 33.09s/it]
 77%|███████▋  | 3610/4690 [32:56:24<9:55:05, 33.06s/it]
                                                        
{'loss': '0.03', 'grad_norm': '1.335', 'learning_rate': '1.33e-06', 'epoch': '3.849'}

 77%|███████▋  | 3610/4690 [32:56:24<9:55:05, 33.06s/it]
 77%|███████▋  | 3611/4690 [32:56:57<9:52:49, 32.97s/it]
 77%|███████▋  | 3612/4690 [32:57:30<9:53:05, 33.01s/it]
 77%|███████▋  | 3613/4690 [32:58:02<9:48:59, 32.81s/it]
 77%|███████▋  | 3614/4690 [32:58:35<9:49:21, 32.86s/it]
 77%|███████▋  | 3615/4690 [32:59:08<9:49:14, 32.89s/it]
 77%|███████▋  | 3616/4690 [32:59:41<9:47:06, 32.80s/it]
 77%|███████▋  | 3617/4690 [33:00:13<9:45:14, 32.73s/it]
 77%|███████▋  | 3618/4690 [33:00:46<9:43:32, 32.66s/it]
 77%|███████▋  | 3619/4690 [33:01:18<9:43:08, 32.67s/it]
 77%|███████▋  | 3620/4690 [33:01:51<9:41:45, 32.62s/it]
                                                        
{'loss': '0.02627', 'grad_norm': '1.161', 'learning_rate': '1.306e-06', 'epoch': '3.86'}

 77%|███████▋  | 3620/4690 [33:01:51<9:41:45, 32.62s/it]
 77%|███████▋  | 3621/4690 [33:02:24<9:41:39, 32.65s/it]
 77%|███████▋  | 3622/4690 [33:02:56<9:42:18, 32.71s/it]
 77%|███████▋  | 3623/4690 [33:03:29<9:42:17, 32.74s/it]
 77%|███████▋  | 3624/4690 [33:04:02<9:41:39, 32.74s/it]
 77%|███████▋  | 3625/4690 [33:04:35<9:41:58, 32.79s/it]
 77%|███████▋  | 3626/4690 [33:05:08<9:42:44, 32.86s/it]
 77%|███████▋  | 3627/4690 [33:05:41<9:43:50, 32.95s/it]
 77%|███████▋  | 3628/4690 [33:06:14<9:42:16, 32.90s/it]
 77%|███████▋  | 3629/4690 [33:06:47<9:40:41, 32.84s/it]
 77%|███████▋  | 3630/4690 [33:07:19<9:39:15, 32.79s/it]
                                                        
{'loss': '0.02866', 'grad_norm': '1.462', 'learning_rate': '1.283e-06', 'epoch': '3.87'}

 77%|███████▋  | 3630/4690 [33:07:19<9:39:15, 32.79s/it]
 77%|███████▋  | 3631/4690 [33:07:52<9:37:02, 32.69s/it]
 77%|███████▋  | 3632/4690 [33:08:25<9:37:40, 32.76s/it]
 77%|███████▋  | 3633/4690 [33:08:57<9:37:05, 32.76s/it]
 77%|███████▋  | 3634/4690 [33:09:30<9:36:15, 32.74s/it]
 78%|███████▊  | 3635/4690 [33:10:03<9:35:48, 32.75s/it]
 78%|███████▊  | 3636/4690 [33:10:35<9:34:57, 32.73s/it]
 78%|███████▊  | 3637/4690 [33:11:08<9:35:06, 32.77s/it]
 78%|███████▊  | 3638/4690 [33:11:42<9:36:32, 32.88s/it]
 78%|███████▊  | 3639/4690 [33:12:14<9:35:06, 32.83s/it]
 78%|███████▊  | 3640/4690 [33:12:47<9:36:11, 32.93s/it]
                                                        
{'loss': '0.02781', 'grad_norm': '1.465', 'learning_rate': '1.26e-06', 'epoch': '3.881'}

 78%|███████▊  | 3640/4690 [33:12:47<9:36:11, 32.93s/it]
 78%|███████▊  | 3641/4690 [33:13:21<9:36:49, 32.99s/it]
 78%|███████▊  | 3642/4690 [33:13:53<9:35:24, 32.94s/it]
 78%|███████▊  | 3643/4690 [33:14:26<9:35:36, 32.99s/it]
 78%|███████▊  | 3644/4690 [33:14:59<9:33:55, 32.92s/it]
 78%|███████▊  | 3645/4690 [33:15:32<9:32:19, 32.86s/it]
 78%|███████▊  | 3646/4690 [33:16:04<9:29:46, 32.75s/it]
 78%|███████▊  | 3647/4690 [33:16:37<9:31:00, 32.85s/it]
 78%|███████▊  | 3648/4690 [33:17:10<9:30:28, 32.85s/it]
 78%|███████▊  | 3649/4690 [33:17:43<9:27:55, 32.73s/it]
 78%|███████▊  | 3650/4690 [33:18:16<9:27:15, 32.73s/it]
                                                        
{'loss': '0.02682', 'grad_norm': '1.45', 'learning_rate': '1.237e-06', 'epoch': '3.892'}

 78%|███████▊  | 3650/4690 [33:18:16<9:27:15, 32.73s/it]
 78%|███████▊  | 3651/4690 [33:18:48<9:26:26, 32.71s/it]
 78%|███████▊  | 3652/4690 [33:19:21<9:26:03, 32.72s/it]
 78%|███████▊  | 3653/4690 [33:19:54<9:25:32, 32.72s/it]
 78%|███████▊  | 3654/4690 [33:20:27<9:26:13, 32.79s/it]
 78%|███████▊  | 3655/4690 [33:21:00<9:26:24, 32.84s/it]
 78%|███████▊  | 3656/4690 [33:21:32<9:24:24, 32.75s/it]
 78%|███████▊  | 3657/4690 [33:22:05<9:25:56, 32.87s/it]
 78%|███████▊  | 3658/4690 [33:22:38<9:25:54, 32.90s/it]
 78%|███████▊  | 3659/4690 [33:23:11<9:23:43, 32.81s/it]
 78%|███████▊  | 3660/4690 [33:23:43<9:20:39, 32.66s/it]
                                                        
{'loss': '0.03143', 'grad_norm': '1.639', 'learning_rate': '1.215e-06', 'epoch': '3.902'}

 78%|███████▊  | 3660/4690 [33:23:43<9:20:39, 32.66s/it]
 78%|███████▊  | 3661/4690 [33:24:16<9:19:53, 32.65s/it]
 78%|███████▊  | 3662/4690 [33:24:48<9:19:08, 32.63s/it]
 78%|███████▊  | 3663/4690 [33:25:22<9:21:39, 32.81s/it]
 78%|███████▊  | 3664/4690 [33:25:54<9:19:52, 32.74s/it]
 78%|███████▊  | 3665/4690 [33:26:27<9:18:52, 32.72s/it]
 78%|███████▊  | 3666/4690 [33:27:00<9:20:40, 32.85s/it]
 78%|███████▊  | 3667/4690 [33:27:33<9:20:09, 32.85s/it]
 78%|███████▊  | 3668/4690 [33:28:06<9:20:40, 32.92s/it]
 78%|███████▊  | 3669/4690 [33:28:39<9:19:21, 32.87s/it]
 78%|███████▊  | 3670/4690 [33:29:12<9:20:21, 32.96s/it]
                                                        
{'loss': '0.02976', 'grad_norm': '1.573', 'learning_rate': '1.192e-06', 'epoch': '3.913'}

 78%|███████▊  | 3670/4690 [33:29:12<9:20:21, 32.96s/it]
 78%|███████▊  | 3671/4690 [33:29:44<9:17:13, 32.81s/it]
 78%|███████▊  | 3672/4690 [33:30:17<9:17:26, 32.86s/it]
 78%|███████▊  | 3673/4690 [33:30:50<9:17:41, 32.90s/it]
 78%|███████▊  | 3674/4690 [33:31:23<9:17:00, 32.89s/it]
 78%|███████▊  | 3675/4690 [33:31:56<9:16:22, 32.89s/it]
 78%|███████▊  | 3676/4690 [33:32:29<9:13:50, 32.77s/it]
 78%|███████▊  | 3677/4690 [33:33:01<9:13:50, 32.80s/it]
 78%|███████▊  | 3678/4690 [33:33:34<9:13:35, 32.82s/it]
 78%|███████▊  | 3679/4690 [33:34:07<9:13:11, 32.83s/it]
 78%|███████▊  | 3680/4690 [33:34:40<9:12:07, 32.80s/it]
                                                        
{'loss': '0.02919', 'grad_norm': '1.198', 'learning_rate': '1.17e-06', 'epoch': '3.924'}

 78%|███████▊  | 3680/4690 [33:34:40<9:12:07, 32.80s/it]
 78%|███████▊  | 3681/4690 [33:35:13<9:11:34, 32.80s/it]
 79%|███████▊  | 3682/4690 [33:35:45<9:10:42, 32.78s/it]
 79%|███████▊  | 3683/4690 [33:36:18<9:11:54, 32.88s/it]
 79%|███████▊  | 3684/4690 [33:36:51<9:11:39, 32.90s/it]
 79%|███████▊  | 3685/4690 [33:37:24<9:11:10, 32.91s/it]
 79%|███████▊  | 3686/4690 [33:37:57<9:08:47, 32.80s/it]
 79%|███████▊  | 3687/4690 [33:38:30<9:09:08, 32.85s/it]
 79%|███████▊  | 3688/4690 [33:39:03<9:08:28, 32.84s/it]
 79%|███████▊  | 3689/4690 [33:39:36<9:07:49, 32.84s/it]
 79%|███████▊  | 3690/4690 [33:40:08<9:06:12, 32.77s/it]
                                                        
{'loss': '0.02641', 'grad_norm': '1.16', 'learning_rate': '1.148e-06', 'epoch': '3.934'}

 79%|███████▊  | 3690/4690 [33:40:08<9:06:12, 32.77s/it]
 79%|███████▊  | 3691/4690 [33:40:41<9:05:02, 32.73s/it]
 79%|███████▊  | 3692/4690 [33:41:14<9:05:00, 32.77s/it]
 79%|███████▊  | 3693/4690 [33:41:46<9:03:32, 32.71s/it]
 79%|███████▉  | 3694/4690 [33:42:19<9:03:59, 32.77s/it]
 79%|███████▉  | 3695/4690 [33:42:52<9:01:50, 32.67s/it]
 79%|███████▉  | 3696/4690 [33:43:24<9:02:17, 32.73s/it]
 79%|███████▉  | 3697/4690 [33:43:57<9:01:33, 32.72s/it]
 79%|███████▉  | 3698/4690 [33:44:30<8:59:48, 32.65s/it]
 79%|███████▉  | 3699/4690 [33:45:03<9:02:43, 32.86s/it]
 79%|███████▉  | 3700/4690 [33:45:36<9:03:05, 32.91s/it]
                                                        
{'loss': '0.02736', 'grad_norm': '1.109', 'learning_rate': '1.126e-06', 'epoch': '3.945'}

 79%|███████▉  | 3700/4690 [33:45:36<9:03:05, 32.91s/it]
 79%|███████▉  | 3701/4690 [33:46:09<9:01:57, 32.88s/it]
 79%|███████▉  | 3702/4690 [33:46:42<9:01:09, 32.86s/it]
 79%|███████▉  | 3703/4690 [33:47:14<9:00:09, 32.84s/it]
 79%|███████▉  | 3704/4690 [33:47:47<9:00:18, 32.88s/it]
 79%|███████▉  | 3705/4690 [33:48:20<9:00:28, 32.92s/it]
 79%|███████▉  | 3706/4690 [33:48:53<8:58:14, 32.82s/it]
 79%|███████▉  | 3707/4690 [33:49:26<8:57:16, 32.79s/it]
 79%|███████▉  | 3708/4690 [33:49:58<8:56:37, 32.79s/it]
 79%|███████▉  | 3709/4690 [33:50:31<8:56:54, 32.84s/it]
 79%|███████▉  | 3710/4690 [33:51:04<8:56:34, 32.85s/it]
                                                        
{'loss': '0.02601', 'grad_norm': '1.326', 'learning_rate': '1.104e-06', 'epoch': '3.956'}

 79%|███████▉  | 3710/4690 [33:51:04<8:56:34, 32.85s/it]
 79%|███████▉  | 3711/4690 [33:51:37<8:56:30, 32.88s/it]
 79%|███████▉  | 3712/4690 [33:52:10<8:55:09, 32.83s/it]
 79%|███████▉  | 3713/4690 [33:52:43<8:54:24, 32.82s/it]
 79%|███████▉  | 3714/4690 [33:53:16<8:55:41, 32.93s/it]
 79%|███████▉  | 3715/4690 [33:53:49<8:55:07, 32.93s/it]
 79%|███████▉  | 3716/4690 [33:54:22<8:53:43, 32.88s/it]
 79%|███████▉  | 3717/4690 [33:54:54<8:50:49, 32.73s/it]
 79%|███████▉  | 3718/4690 [33:55:27<8:49:26, 32.68s/it]
 79%|███████▉  | 3719/4690 [33:56:00<8:50:52, 32.80s/it]
 79%|███████▉  | 3720/4690 [33:56:32<8:49:37, 32.76s/it]
                                                        
{'loss': '0.03044', 'grad_norm': '1.185', 'learning_rate': '1.083e-06', 'epoch': '3.966'}

 79%|███████▉  | 3720/4690 [33:56:32<8:49:37, 32.76s/it]
 79%|███████▉  | 3721/4690 [33:57:06<8:52:41, 32.98s/it]
 79%|███████▉  | 3722/4690 [33:57:39<8:51:17, 32.93s/it]
 79%|███████▉  | 3723/4690 [33:58:12<8:50:24, 32.91s/it]
 79%|███████▉  | 3724/4690 [33:58:44<8:48:22, 32.82s/it]
 79%|███████▉  | 3725/4690 [33:59:17<8:48:27, 32.86s/it]
 79%|███████▉  | 3726/4690 [33:59:50<8:48:09, 32.87s/it]
 79%|███████▉  | 3727/4690 [34:00:23<8:46:48, 32.82s/it]
 79%|███████▉  | 3728/4690 [34:00:56<8:48:47, 32.98s/it]
 80%|███████▉  | 3729/4690 [34:01:29<8:46:07, 32.85s/it]
 80%|███████▉  | 3730/4690 [34:02:02<8:45:52, 32.87s/it]
                                                        
{'loss': '0.027', 'grad_norm': '1.51', 'learning_rate': '1.061e-06', 'epoch': '3.977'}

 80%|███████▉  | 3730/4690 [34:02:02<8:45:52, 32.87s/it]
 80%|███████▉  | 3731/4690 [34:02:34<8:44:24, 32.81s/it]
 80%|███████▉  | 3732/4690 [34:03:07<8:44:15, 32.83s/it]
 80%|███████▉  | 3733/4690 [34:03:40<8:42:46, 32.78s/it]
 80%|███████▉  | 3734/4690 [34:04:12<8:41:45, 32.75s/it]
 80%|███████▉  | 3735/4690 [34:04:45<8:40:37, 32.71s/it]
 80%|███████▉  | 3736/4690 [34:05:18<8:41:49, 32.82s/it]
 80%|███████▉  | 3737/4690 [34:05:51<8:41:40, 32.84s/it]
 80%|███████▉  | 3738/4690 [34:06:24<8:43:37, 33.00s/it]
 80%|███████▉  | 3739/4690 [34:06:57<8:43:36, 33.04s/it]
 80%|███████▉  | 3740/4690 [34:07:30<8:41:58, 32.97s/it]
                                                        
{'loss': '0.02994', 'grad_norm': '1.462', 'learning_rate': '1.04e-06', 'epoch': '3.988'}

 80%|███████▉  | 3740/4690 [34:07:30<8:41:58, 32.97s/it]
 80%|███████▉  | 3741/4690 [34:08:03<8:41:56, 33.00s/it]
 80%|███████▉  | 3742/4690 [34:08:37<8:42:42, 33.08s/it]
 80%|███████▉  | 3743/4690 [34:09:09<8:39:17, 32.90s/it]
 80%|███████▉  | 3744/4690 [34:09:42<8:38:05, 32.86s/it]
 80%|███████▉  | 3745/4690 [34:10:14<8:36:18, 32.78s/it]
 80%|███████▉  | 3746/4690 [34:10:47<8:34:50, 32.72s/it]
 80%|███████▉  | 3747/4690 [34:11:20<8:35:42, 32.81s/it]
 80%|███████▉  | 3748/4690 [34:11:53<8:33:59, 32.74s/it]
 80%|███████▉  | 3749/4690 [34:12:25<8:33:10, 32.72s/it]
 80%|███████▉  | 3750/4690 [34:12:58<8:32:27, 32.71s/it]
                                                        
{'loss': '0.02728', 'grad_norm': '1.144', 'learning_rate': '1.019e-06', 'epoch': '3.998'}

 80%|███████▉  | 3750/4690 [34:12:58<8:32:27, 32.71s/it]
 80%|███████▉  | 3751/4690 [34:13:31<8:31:02, 32.65s/it]
 80%|████████  | 3752/4690 [34:13:47<7:16:35, 27.93s/it][INFO|trainer.py:3815] 2026-05-15 11:35:11,161 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752
[INFO|configuration_utils.py:533] 2026-05-15 11:35:11,183 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/config.json
[INFO|configuration_utils.py:816] 2026-05-15 11:35:11,184 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/generation_config.json


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:16<00:00, 16.62s/it]
Writing model shards: 100%|██████████| 1/1 [00:16<00:00, 16.62s/it]
[INFO|modeling_utils.py:3471] 2026-05-15 11:35:27,856 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-15 11:35:27,865 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 11:35:27,866 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/tokenizer_config.json
[INFO|tokenization_utils_base.py:3278] 2026-05-15 11:35:59,682 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 11:35:59,683 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-15 11:35:59,808 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-3752/processor_config.json

 80%|████████  | 3753/4690 [34:15:18<12:11:04, 46.81s/it]
 80%|████████  | 3754/4690 [34:15:51<11:04:41, 42.61s/it]
 80%|████████  | 3755/4690 [34:16:24<10:17:58, 39.66s/it]
 80%|████████  | 3756/4690 [34:16:57<9:45:42, 37.63s/it] 
 80%|████████  | 3757/4690 [34:17:29<9:21:35, 36.12s/it]
 80%|████████  | 3758/4690 [34:18:02<9:05:50, 35.14s/it]
 80%|████████  | 3759/4690 [34:18:35<8:54:26, 34.44s/it]
 80%|████████  | 3760/4690 [34:19:08<8:45:54, 33.93s/it]
                                                        
{'loss': '0.0131', 'grad_norm': '0.5474', 'learning_rate': '9.984e-07', 'epoch': '4.009'}

 80%|████████  | 3760/4690 [34:19:08<8:45:54, 33.93s/it]
 80%|████████  | 3761/4690 [34:19:41<8:41:00, 33.65s/it]
 80%|████████  | 3762/4690 [34:20:13<8:34:41, 33.28s/it]
 80%|████████  | 3763/4690 [34:20:46<8:33:05, 33.21s/it]
 80%|████████  | 3764/4690 [34:21:19<8:30:56, 33.11s/it]
 80%|████████  | 3765/4690 [34:21:52<8:28:22, 32.98s/it]
 80%|████████  | 3766/4690 [34:22:24<8:26:24, 32.88s/it]
 80%|████████  | 3767/4690 [34:22:57<8:25:38, 32.87s/it]
 80%|████████  | 3768/4690 [34:23:30<8:24:41, 32.84s/it]
 80%|████████  | 3769/4690 [34:24:03<8:24:41, 32.88s/it]
 80%|████████  | 3770/4690 [34:24:36<8:23:28, 32.84s/it]
                                                        
{'loss': '0.008076', 'grad_norm': '0.3779', 'learning_rate': '9.778e-07', 'epoch': '4.019'}

 80%|████████  | 3770/4690 [34:24:36<8:23:28, 32.84s/it]
 80%|████████  | 3771/4690 [34:25:09<8:23:42, 32.89s/it]
 80%|████████  | 3772/4690 [34:25:42<8:23:13, 32.89s/it]
 80%|████████  | 3773/4690 [34:26:15<8:23:04, 32.92s/it]
 80%|████████  | 3774/4690 [34:26:47<8:20:03, 32.75s/it]
 80%|████████  | 3775/4690 [34:27:20<8:19:43, 32.77s/it]
 81%|████████  | 3776/4690 [34:27:52<8:18:44, 32.74s/it]
 81%|████████  | 3777/4690 [34:28:26<8:19:28, 32.82s/it]
 81%|████████  | 3778/4690 [34:28:58<8:18:08, 32.77s/it]
 81%|████████  | 3779/4690 [34:29:31<8:18:40, 32.84s/it]
 81%|████████  | 3780/4690 [34:30:04<8:17:19, 32.79s/it]
                                                        
{'loss': '0.008184', 'grad_norm': '0.555', 'learning_rate': '9.574e-07', 'epoch': '4.03'}

 81%|████████  | 3780/4690 [34:30:04<8:17:19, 32.79s/it]
 81%|████████  | 3781/4690 [34:30:37<8:16:19, 32.76s/it]
 81%|████████  | 3782/4690 [34:31:09<8:14:46, 32.69s/it]
 81%|████████  | 3783/4690 [34:31:41<8:12:59, 32.61s/it]
 81%|████████  | 3784/4690 [34:32:14<8:13:58, 32.71s/it]
 81%|████████  | 3785/4690 [34:32:47<8:12:44, 32.67s/it]
 81%|████████  | 3786/4690 [34:33:20<8:13:38, 32.76s/it]
 81%|████████  | 3787/4690 [34:33:53<8:14:22, 32.85s/it]
 81%|████████  | 3788/4690 [34:34:26<8:12:51, 32.78s/it]
 81%|████████  | 3789/4690 [34:34:58<8:10:35, 32.67s/it]
 81%|████████  | 3790/4690 [34:35:31<8:09:09, 32.61s/it]
                                                        
{'loss': '0.007112', 'grad_norm': '0.4951', 'learning_rate': '9.371e-07', 'epoch': '4.041'}

 81%|████████  | 3790/4690 [34:35:31<8:09:09, 32.61s/it]
 81%|████████  | 3791/4690 [34:36:04<8:10:15, 32.72s/it]
 81%|████████  | 3792/4690 [34:36:36<8:08:45, 32.66s/it]
 81%|████████  | 3793/4690 [34:37:09<8:07:58, 32.64s/it]
 81%|████████  | 3794/4690 [34:37:42<8:09:11, 32.76s/it]
 81%|████████  | 3795/4690 [34:38:15<8:10:11, 32.86s/it]
 81%|████████  | 3796/4690 [34:38:47<8:07:40, 32.73s/it]
 81%|████████  | 3797/4690 [34:39:20<8:07:07, 32.73s/it]
 81%|████████  | 3798/4690 [34:39:53<8:06:29, 32.72s/it]
 81%|████████  | 3799/4690 [34:40:25<8:05:19, 32.68s/it]
 81%|████████  | 3800/4690 [34:40:58<8:03:17, 32.58s/it]
                                                        
{'loss': '0.007163', 'grad_norm': '0.4599', 'learning_rate': '9.171e-07', 'epoch': '4.051'}

 81%|████████  | 3800/4690 [34:40:58<8:03:17, 32.58s/it]
 81%|████████  | 3801/4690 [34:41:30<8:03:24, 32.63s/it]
 81%|████████  | 3802/4690 [34:42:03<8:02:27, 32.60s/it]
 81%|████████  | 3803/4690 [34:42:36<8:02:54, 32.67s/it]
 81%|████████  | 3804/4690 [34:43:08<8:02:45, 32.69s/it]
 81%|████████  | 3805/4690 [34:43:41<8:02:53, 32.74s/it]
 81%|████████  | 3806/4690 [34:44:14<8:02:11, 32.73s/it]
 81%|████████  | 3807/4690 [34:44:47<8:01:48, 32.74s/it]
 81%|████████  | 3808/4690 [34:45:19<8:01:07, 32.73s/it]
 81%|████████  | 3809/4690 [34:45:52<7:59:52, 32.68s/it]
 81%|████████  | 3810/4690 [34:46:25<7:59:01, 32.66s/it]
                                                        
{'loss': '0.006506', 'grad_norm': '0.5192', 'learning_rate': '8.973e-07', 'epoch': '4.062'}

 81%|████████  | 3810/4690 [34:46:25<7:59:01, 32.66s/it]
 81%|████████▏ | 3811/4690 [34:46:57<7:58:37, 32.67s/it]
 81%|████████▏ | 3812/4690 [34:47:31<8:00:48, 32.86s/it]
 81%|████████▏ | 3813/4690 [34:48:04<8:01:28, 32.94s/it]
 81%|████████▏ | 3814/4690 [34:48:37<8:00:49, 32.93s/it]
 81%|████████▏ | 3815/4690 [34:49:09<7:59:16, 32.86s/it]
 81%|████████▏ | 3816/4690 [34:49:42<7:56:53, 32.74s/it]
 81%|████████▏ | 3817/4690 [34:50:14<7:56:11, 32.73s/it]
 81%|████████▏ | 3818/4690 [34:50:47<7:55:16, 32.70s/it]
 81%|████████▏ | 3819/4690 [34:51:20<7:55:13, 32.74s/it]
 81%|████████▏ | 3820/4690 [34:51:53<7:55:31, 32.80s/it]
                                                        
{'loss': '0.006933', 'grad_norm': '0.8438', 'learning_rate': '8.776e-07', 'epoch': '4.073'}

 81%|████████▏ | 3820/4690 [34:51:53<7:55:31, 32.80s/it]
 81%|████████▏ | 3821/4690 [34:52:26<7:54:40, 32.77s/it]
 81%|████████▏ | 3822/4690 [34:52:58<7:53:14, 32.71s/it]
 82%|████████▏ | 3823/4690 [34:53:31<7:51:22, 32.62s/it]
 82%|████████▏ | 3824/4690 [34:54:04<7:52:38, 32.75s/it]
 82%|████████▏ | 3825/4690 [34:54:36<7:52:28, 32.77s/it]
 82%|████████▏ | 3826/4690 [34:55:09<7:52:00, 32.78s/it]
 82%|████████▏ | 3827/4690 [34:55:42<7:51:36, 32.79s/it]
 82%|████████▏ | 3828/4690 [34:56:15<7:51:00, 32.78s/it]
 82%|████████▏ | 3829/4690 [34:56:48<7:52:17, 32.91s/it]
 82%|████████▏ | 3830/4690 [34:57:21<7:50:02, 32.79s/it]
                                                        
{'loss': '0.006969', 'grad_norm': '0.7045', 'learning_rate': '8.582e-07', 'epoch': '4.083'}

 82%|████████▏ | 3830/4690 [34:57:21<7:50:02, 32.79s/it]
 82%|████████▏ | 3831/4690 [34:57:54<7:51:47, 32.95s/it]
 82%|████████▏ | 3832/4690 [34:58:27<7:51:03, 32.94s/it]
 82%|████████▏ | 3833/4690 [34:58:59<7:49:13, 32.85s/it]
 82%|████████▏ | 3834/4690 [34:59:33<7:51:21, 33.04s/it]
 82%|████████▏ | 3835/4690 [35:00:06<7:49:35, 32.95s/it]
 82%|████████▏ | 3836/4690 [35:00:39<7:49:53, 33.01s/it]
 82%|████████▏ | 3837/4690 [35:01:11<7:47:38, 32.89s/it]
 82%|████████▏ | 3838/4690 [35:01:44<7:47:39, 32.93s/it]
 82%|████████▏ | 3839/4690 [35:02:17<7:44:57, 32.78s/it]
 82%|████████▏ | 3840/4690 [35:02:50<7:46:52, 32.96s/it]
                                                        
{'loss': '0.008399', 'grad_norm': '0.6902', 'learning_rate': '8.389e-07', 'epoch': '4.094'}

 82%|████████▏ | 3840/4690 [35:02:50<7:46:52, 32.96s/it]
 82%|████████▏ | 3841/4690 [35:03:23<7:47:12, 33.02s/it]
 82%|████████▏ | 3842/4690 [35:03:56<7:46:14, 32.99s/it]
 82%|████████▏ | 3843/4690 [35:04:29<7:46:21, 33.04s/it]
 82%|████████▏ | 3844/4690 [35:05:02<7:44:36, 32.95s/it]
 82%|████████▏ | 3845/4690 [35:05:35<7:44:23, 32.97s/it]
 82%|████████▏ | 3846/4690 [35:06:08<7:43:27, 32.95s/it]
 82%|████████▏ | 3847/4690 [35:06:41<7:43:53, 33.02s/it]
 82%|████████▏ | 3848/4690 [35:07:14<7:42:00, 32.92s/it]
 82%|████████▏ | 3849/4690 [35:07:47<7:40:46, 32.87s/it]
 82%|████████▏ | 3850/4690 [35:08:20<7:42:15, 33.02s/it]
                                                        
{'loss': '0.007258', 'grad_norm': '0.6016', 'learning_rate': '8.199e-07', 'epoch': '4.105'}

 82%|████████▏ | 3850/4690 [35:08:20<7:42:15, 33.02s/it]
 82%|████████▏ | 3851/4690 [35:08:53<7:40:09, 32.91s/it]
 82%|████████▏ | 3852/4690 [35:09:25<7:38:40, 32.84s/it]
 82%|████████▏ | 3853/4690 [35:09:59<7:39:08, 32.91s/it]
 82%|████████▏ | 3854/4690 [35:10:31<7:38:24, 32.90s/it]
 82%|████████▏ | 3855/4690 [35:11:04<7:37:38, 32.88s/it]
 82%|████████▏ | 3856/4690 [35:11:37<7:35:52, 32.80s/it]
 82%|████████▏ | 3857/4690 [35:12:10<7:35:11, 32.79s/it]
 82%|████████▏ | 3858/4690 [35:12:42<7:34:15, 32.76s/it]
 82%|████████▏ | 3859/4690 [35:13:15<7:34:40, 32.83s/it]
 82%|████████▏ | 3860/4690 [35:13:48<7:33:41, 32.80s/it]
                                                        
{'loss': '0.007289', 'grad_norm': '0.6161', 'learning_rate': '8.01e-07', 'epoch': '4.115'}

 82%|████████▏ | 3860/4690 [35:13:48<7:33:41, 32.80s/it]
 82%|████████▏ | 3861/4690 [35:14:21<7:33:09, 32.80s/it]
 82%|████████▏ | 3862/4690 [35:14:53<7:31:28, 32.72s/it]
 82%|████████▏ | 3863/4690 [35:15:26<7:32:26, 32.83s/it]
 82%|████████▏ | 3864/4690 [35:15:59<7:31:08, 32.77s/it]
 82%|████████▏ | 3865/4690 [35:16:32<7:32:01, 32.87s/it]
 82%|████████▏ | 3866/4690 [35:17:05<7:31:07, 32.85s/it]
 82%|████████▏ | 3867/4690 [35:17:37<7:28:44, 32.72s/it]
 82%|████████▏ | 3868/4690 [35:18:10<7:27:05, 32.63s/it]
 82%|████████▏ | 3869/4690 [35:18:42<7:26:03, 32.60s/it]
 83%|████████▎ | 3870/4690 [35:19:15<7:25:51, 32.62s/it]
                                                        
{'loss': '0.007842', 'grad_norm': '0.7333', 'learning_rate': '7.824e-07', 'epoch': '4.126'}

 83%|████████▎ | 3870/4690 [35:19:15<7:25:51, 32.62s/it]
 83%|████████▎ | 3871/4690 [35:19:48<7:24:59, 32.60s/it]
 83%|████████▎ | 3872/4690 [35:20:20<7:24:40, 32.62s/it]
 83%|████████▎ | 3873/4690 [35:20:53<7:25:17, 32.70s/it]
 83%|████████▎ | 3874/4690 [35:21:26<7:25:22, 32.75s/it]
 83%|████████▎ | 3875/4690 [35:21:58<7:23:45, 32.67s/it]
 83%|████████▎ | 3876/4690 [35:22:31<7:22:33, 32.62s/it]
 83%|████████▎ | 3877/4690 [35:23:04<7:21:39, 32.59s/it]
 83%|████████▎ | 3878/4690 [35:23:37<7:22:53, 32.73s/it]
 83%|████████▎ | 3879/4690 [35:24:10<7:23:40, 32.82s/it]
 83%|████████▎ | 3880/4690 [35:24:43<7:24:22, 32.92s/it]
                                                        
{'loss': '0.00715', 'grad_norm': '0.6135', 'learning_rate': '7.64e-07', 'epoch': '4.137'}

 83%|████████▎ | 3880/4690 [35:24:43<7:24:22, 32.92s/it]
 83%|████████▎ | 3881/4690 [35:25:16<7:23:20, 32.88s/it]
 83%|████████▎ | 3882/4690 [35:25:48<7:22:47, 32.88s/it]
 83%|████████▎ | 3883/4690 [35:26:21<7:20:40, 32.76s/it]
 83%|████████▎ | 3884/4690 [35:26:54<7:20:48, 32.81s/it]
 83%|████████▎ | 3885/4690 [35:27:27<7:20:49, 32.86s/it]
 83%|████████▎ | 3886/4690 [35:28:00<7:21:14, 32.93s/it]
 83%|████████▎ | 3887/4690 [35:28:33<7:20:22, 32.90s/it]
 83%|████████▎ | 3888/4690 [35:29:06<7:20:24, 32.95s/it]
 83%|████████▎ | 3889/4690 [35:29:39<7:20:50, 33.02s/it]
 83%|████████▎ | 3890/4690 [35:30:12<7:20:21, 33.03s/it]
                                                        
{'loss': '0.008679', 'grad_norm': '0.8728', 'learning_rate': '7.457e-07', 'epoch': '4.147'}

 83%|████████▎ | 3890/4690 [35:30:12<7:20:21, 33.03s/it]
 83%|████████▎ | 3891/4690 [35:30:45<7:17:50, 32.88s/it]
 83%|████████▎ | 3892/4690 [35:31:18<7:17:50, 32.92s/it]
 83%|████████▎ | 3893/4690 [35:31:50<7:15:58, 32.82s/it]
 83%|████████▎ | 3894/4690 [35:32:23<7:15:32, 32.83s/it]
 83%|████████▎ | 3895/4690 [35:32:56<7:14:55, 32.82s/it]
 83%|████████▎ | 3896/4690 [35:33:29<7:14:05, 32.80s/it]
 83%|████████▎ | 3897/4690 [35:34:02<7:14:15, 32.86s/it]
 83%|████████▎ | 3898/4690 [35:34:34<7:11:37, 32.70s/it]
 83%|████████▎ | 3899/4690 [35:35:07<7:12:29, 32.81s/it]
 83%|████████▎ | 3900/4690 [35:35:40<7:11:27, 32.77s/it]
                                                        
{'loss': '0.007436', 'grad_norm': '0.7204', 'learning_rate': '7.277e-07', 'epoch': '4.158'}

 83%|████████▎ | 3900/4690 [35:35:40<7:11:27, 32.77s/it]
 83%|████████▎ | 3901/4690 [35:36:12<7:10:31, 32.74s/it]
 83%|████████▎ | 3902/4690 [35:36:45<7:11:08, 32.83s/it]
 83%|████████▎ | 3903/4690 [35:37:18<7:11:37, 32.91s/it]
 83%|████████▎ | 3904/4690 [35:37:51<7:09:28, 32.78s/it]
 83%|████████▎ | 3905/4690 [35:38:24<7:08:30, 32.75s/it]
 83%|████████▎ | 3906/4690 [35:38:56<7:07:46, 32.74s/it]
 83%|████████▎ | 3907/4690 [35:39:29<7:07:42, 32.77s/it]
 83%|████████▎ | 3908/4690 [35:40:02<7:06:06, 32.69s/it]
 83%|████████▎ | 3909/4690 [35:40:34<7:05:39, 32.70s/it]
 83%|████████▎ | 3910/4690 [35:41:07<7:05:19, 32.72s/it]
                                                        
{'loss': '0.007057', 'grad_norm': '0.8087', 'learning_rate': '7.098e-07', 'epoch': '4.169'}

 83%|████████▎ | 3910/4690 [35:41:07<7:05:19, 32.72s/it]
 83%|████████▎ | 3911/4690 [35:41:40<7:06:15, 32.83s/it]
 83%|████████▎ | 3912/4690 [35:42:13<7:05:02, 32.78s/it]
 83%|████████▎ | 3913/4690 [35:42:46<7:04:02, 32.74s/it]
 83%|████████▎ | 3914/4690 [35:43:18<7:02:09, 32.64s/it]
 83%|████████▎ | 3915/4690 [35:43:50<7:01:03, 32.60s/it]
 83%|████████▎ | 3916/4690 [35:44:23<7:01:25, 32.67s/it]
 84%|████████▎ | 3917/4690 [35:44:56<7:01:37, 32.73s/it]
 84%|████████▎ | 3918/4690 [35:45:29<7:02:32, 32.84s/it]
 84%|████████▎ | 3919/4690 [35:46:02<7:00:11, 32.70s/it]
 84%|████████▎ | 3920/4690 [35:46:35<7:01:45, 32.86s/it]
                                                        
{'loss': '0.007321', 'grad_norm': '4.543', 'learning_rate': '6.922e-07', 'epoch': '4.179'}

 84%|████████▎ | 3920/4690 [35:46:35<7:01:45, 32.86s/it]
 84%|████████▎ | 3921/4690 [35:47:08<7:00:18, 32.79s/it]
 84%|████████▎ | 3922/4690 [35:47:40<7:00:04, 32.82s/it]
 84%|████████▎ | 3923/4690 [35:48:13<6:59:56, 32.85s/it]
 84%|████████▎ | 3924/4690 [35:48:46<6:58:20, 32.77s/it]
 84%|████████▎ | 3925/4690 [35:49:19<6:57:21, 32.73s/it]
 84%|████████▎ | 3926/4690 [35:49:51<6:56:50, 32.74s/it]
 84%|████████▎ | 3927/4690 [35:50:24<6:55:51, 32.70s/it]
 84%|████████▍ | 3928/4690 [35:50:57<6:55:38, 32.73s/it]
 84%|████████▍ | 3929/4690 [35:51:30<6:55:53, 32.79s/it]
 84%|████████▍ | 3930/4690 [35:52:03<6:55:55, 32.84s/it]
                                                        
{'loss': '0.007425', 'grad_norm': '0.6915', 'learning_rate': '6.748e-07', 'epoch': '4.19'}

 84%|████████▍ | 3930/4690 [35:52:03<6:55:55, 32.84s/it]
 84%|████████▍ | 3931/4690 [35:52:35<6:55:05, 32.81s/it]
 84%|████████▍ | 3932/4690 [35:53:08<6:54:20, 32.80s/it]
 84%|████████▍ | 3933/4690 [35:53:41<6:54:14, 32.83s/it]
 84%|████████▍ | 3934/4690 [35:54:14<6:54:07, 32.87s/it]
 84%|████████▍ | 3935/4690 [35:54:47<6:54:56, 32.98s/it]
 84%|████████▍ | 3936/4690 [35:55:20<6:53:01, 32.87s/it]
 84%|████████▍ | 3937/4690 [35:55:53<6:51:52, 32.82s/it]
 84%|████████▍ | 3938/4690 [35:56:25<6:50:40, 32.77s/it]
 84%|████████▍ | 3939/4690 [35:56:58<6:49:53, 32.75s/it]
 84%|████████▍ | 3940/4690 [35:57:31<6:49:19, 32.75s/it]
                                                        
{'loss': '0.007283', 'grad_norm': '0.8994', 'learning_rate': '6.576e-07', 'epoch': '4.201'}

 84%|████████▍ | 3940/4690 [35:57:31<6:49:19, 32.75s/it]
 84%|████████▍ | 3941/4690 [35:58:03<6:48:17, 32.71s/it]
 84%|████████▍ | 3942/4690 [35:58:36<6:48:43, 32.78s/it]
 84%|████████▍ | 3943/4690 [35:59:09<6:48:57, 32.85s/it]
 84%|████████▍ | 3944/4690 [35:59:42<6:47:52, 32.81s/it]
 84%|████████▍ | 3945/4690 [36:00:15<6:47:18, 32.80s/it]
 84%|████████▍ | 3946/4690 [36:00:47<6:46:16, 32.76s/it]
 84%|████████▍ | 3947/4690 [36:01:21<6:47:44, 32.93s/it]
 84%|████████▍ | 3948/4690 [36:01:53<6:46:28, 32.87s/it]
 84%|████████▍ | 3949/4690 [36:02:26<6:45:49, 32.86s/it]
 84%|████████▍ | 3950/4690 [36:02:59<6:44:05, 32.76s/it]
                                                        
{'loss': '0.0063', 'grad_norm': '0.9054', 'learning_rate': '6.405e-07', 'epoch': '4.211'}

 84%|████████▍ | 3950/4690 [36:02:59<6:44:05, 32.76s/it]
 84%|████████▍ | 3951/4690 [36:03:32<6:44:17, 32.82s/it]
 84%|████████▍ | 3952/4690 [36:04:04<6:43:25, 32.80s/it]
 84%|████████▍ | 3953/4690 [36:04:37<6:42:54, 32.80s/it]
 84%|████████▍ | 3954/4690 [36:05:11<6:44:31, 32.98s/it]
 84%|████████▍ | 3955/4690 [36:05:44<6:43:28, 32.94s/it]
 84%|████████▍ | 3956/4690 [36:06:17<6:43:37, 32.99s/it]
 84%|████████▍ | 3957/4690 [36:06:49<6:41:56, 32.90s/it]
 84%|████████▍ | 3958/4690 [36:07:22<6:41:46, 32.93s/it]
 84%|████████▍ | 3959/4690 [36:07:55<6:40:36, 32.88s/it]
 84%|████████▍ | 3960/4690 [36:08:28<6:38:52, 32.78s/it]
                                                        
{'loss': '0.006837', 'grad_norm': '0.6747', 'learning_rate': '6.237e-07', 'epoch': '4.222'}

 84%|████████▍ | 3960/4690 [36:08:28<6:38:52, 32.78s/it]
 84%|████████▍ | 3961/4690 [36:09:00<6:38:23, 32.79s/it]
 84%|████████▍ | 3962/4690 [36:09:33<6:36:17, 32.66s/it]
 84%|████████▍ | 3963/4690 [36:10:05<6:35:42, 32.66s/it]
 85%|████████▍ | 3964/4690 [36:10:38<6:34:15, 32.58s/it]
 85%|████████▍ | 3965/4690 [36:11:11<6:34:13, 32.63s/it]
 85%|████████▍ | 3966/4690 [36:11:43<6:32:14, 32.51s/it]
 85%|████████▍ | 3967/4690 [36:12:16<6:32:51, 32.60s/it]
 85%|████████▍ | 3968/4690 [36:12:48<6:32:20, 32.60s/it]
 85%|████████▍ | 3969/4690 [36:13:21<6:33:48, 32.77s/it]
 85%|████████▍ | 3970/4690 [36:13:54<6:32:31, 32.71s/it]
                                                        
{'loss': '0.006357', 'grad_norm': '0.893', 'learning_rate': '6.071e-07', 'epoch': '4.233'}

 85%|████████▍ | 3970/4690 [36:13:54<6:32:31, 32.71s/it]
 85%|████████▍ | 3971/4690 [36:14:27<6:32:47, 32.78s/it]
 85%|████████▍ | 3972/4690 [36:14:59<6:31:05, 32.68s/it]
 85%|████████▍ | 3973/4690 [36:15:32<6:32:00, 32.80s/it]
 85%|████████▍ | 3974/4690 [36:16:05<6:31:14, 32.79s/it]
 85%|████████▍ | 3975/4690 [36:16:38<6:30:36, 32.78s/it]
 85%|████████▍ | 3976/4690 [36:17:11<6:30:42, 32.83s/it]
 85%|████████▍ | 3977/4690 [36:17:43<6:28:47, 32.72s/it]
 85%|████████▍ | 3978/4690 [36:18:16<6:28:07, 32.71s/it]
 85%|████████▍ | 3979/4690 [36:18:49<6:28:49, 32.81s/it]
 85%|████████▍ | 3980/4690 [36:19:22<6:27:00, 32.71s/it]
                                                        
{'loss': '0.007808', 'grad_norm': '1.182', 'learning_rate': '5.907e-07', 'epoch': '4.243'}

 85%|████████▍ | 3980/4690 [36:19:22<6:27:00, 32.71s/it]
 85%|████████▍ | 3981/4690 [36:19:54<6:26:52, 32.74s/it]
 85%|████████▍ | 3982/4690 [36:20:27<6:26:58, 32.79s/it]
 85%|████████▍ | 3983/4690 [36:21:00<6:26:34, 32.81s/it]
 85%|████████▍ | 3984/4690 [36:21:33<6:25:20, 32.75s/it]
 85%|████████▍ | 3985/4690 [36:22:06<6:25:10, 32.78s/it]
 85%|████████▍ | 3986/4690 [36:22:39<6:25:22, 32.84s/it]
 85%|████████▌ | 3987/4690 [36:23:11<6:24:29, 32.82s/it]
 85%|████████▌ | 3988/4690 [36:23:44<6:23:12, 32.75s/it]
 85%|████████▌ | 3989/4690 [36:24:17<6:24:09, 32.88s/it]
 85%|████████▌ | 3990/4690 [36:24:50<6:22:53, 32.82s/it]
                                                        
{'loss': '0.00755', 'grad_norm': '0.6026', 'learning_rate': '5.746e-07', 'epoch': '4.254'}

 85%|████████▌ | 3990/4690 [36:24:50<6:22:53, 32.82s/it]
 85%|████████▌ | 3991/4690 [36:25:23<6:22:09, 32.80s/it]
 85%|████████▌ | 3992/4690 [36:25:55<6:21:25, 32.79s/it]
 85%|████████▌ | 3993/4690 [36:26:28<6:20:36, 32.76s/it]
 85%|████████▌ | 3994/4690 [36:27:00<6:18:52, 32.66s/it]
 85%|████████▌ | 3995/4690 [36:27:33<6:19:19, 32.75s/it]
 85%|████████▌ | 3996/4690 [36:28:06<6:18:34, 32.73s/it]
 85%|████████▌ | 3997/4690 [36:28:39<6:18:18, 32.75s/it]
 85%|████████▌ | 3998/4690 [36:29:12<6:17:44, 32.75s/it]
 85%|████████▌ | 3999/4690 [36:29:45<6:17:26, 32.77s/it]
 85%|████████▌ | 4000/4690 [36:30:17<6:16:39, 32.75s/it]
                                                        
{'loss': '0.006817', 'grad_norm': '0.6821', 'learning_rate': '5.586e-07', 'epoch': '4.265'}

 85%|████████▌ | 4000/4690 [36:30:17<6:16:39, 32.75s/it]
 85%|████████▌ | 4001/4690 [36:30:50<6:16:26, 32.78s/it]
 85%|████████▌ | 4002/4690 [36:31:23<6:14:59, 32.70s/it]
 85%|████████▌ | 4003/4690 [36:31:55<6:14:53, 32.74s/it]
 85%|████████▌ | 4004/4690 [36:32:28<6:14:55, 32.79s/it]
 85%|████████▌ | 4005/4690 [36:33:01<6:15:08, 32.86s/it]
 85%|████████▌ | 4006/4690 [36:33:34<6:14:46, 32.88s/it]
 85%|████████▌ | 4007/4690 [36:34:07<6:14:15, 32.88s/it]
 85%|████████▌ | 4008/4690 [36:34:40<6:13:27, 32.86s/it]
 85%|████████▌ | 4009/4690 [36:35:13<6:13:08, 32.88s/it]
 86%|████████▌ | 4010/4690 [36:35:45<6:11:21, 32.77s/it]
                                                        
{'loss': '0.0068', 'grad_norm': '0.5271', 'learning_rate': '5.429e-07', 'epoch': '4.275'}

 86%|████████▌ | 4010/4690 [36:35:45<6:11:21, 32.77s/it]
 86%|████████▌ | 4011/4690 [36:36:19<6:12:01, 32.87s/it]
 86%|████████▌ | 4012/4690 [36:36:52<6:11:59, 32.92s/it]
 86%|████████▌ | 4013/4690 [36:37:24<6:11:08, 32.89s/it]
 86%|████████▌ | 4014/4690 [36:37:57<6:10:23, 32.88s/it]
 86%|████████▌ | 4015/4690 [36:38:29<6:07:53, 32.70s/it]
 86%|████████▌ | 4016/4690 [36:39:02<6:07:09, 32.69s/it]
 86%|████████▌ | 4017/4690 [36:39:35<6:07:11, 32.74s/it]
 86%|████████▌ | 4018/4690 [36:40:09<6:09:20, 32.98s/it]
 86%|████████▌ | 4019/4690 [36:40:41<6:06:43, 32.79s/it]
 86%|████████▌ | 4020/4690 [36:41:13<6:05:22, 32.72s/it]
                                                        
{'loss': '0.007622', 'grad_norm': '0.6283', 'learning_rate': '5.273e-07', 'epoch': '4.286'}

 86%|████████▌ | 4020/4690 [36:41:13<6:05:22, 32.72s/it]
 86%|████████▌ | 4021/4690 [36:41:46<6:05:31, 32.78s/it]
 86%|████████▌ | 4022/4690 [36:42:20<6:06:17, 32.90s/it]
 86%|████████▌ | 4023/4690 [36:42:52<6:05:19, 32.86s/it]
 86%|████████▌ | 4024/4690 [36:43:25<6:04:52, 32.87s/it]
 86%|████████▌ | 4025/4690 [36:43:58<6:04:25, 32.88s/it]
 86%|████████▌ | 4026/4690 [36:44:31<6:04:04, 32.90s/it]
 86%|████████▌ | 4027/4690 [36:45:04<6:03:58, 32.94s/it]
 86%|████████▌ | 4028/4690 [36:45:37<6:03:53, 32.98s/it]
 86%|████████▌ | 4029/4690 [36:46:10<6:01:44, 32.84s/it]
 86%|████████▌ | 4030/4690 [36:46:42<6:00:16, 32.75s/it]
                                                        
{'loss': '0.006069', 'grad_norm': '0.6499', 'learning_rate': '5.12e-07', 'epoch': '4.297'}

 86%|████████▌ | 4030/4690 [36:46:42<6:00:16, 32.75s/it]
 86%|████████▌ | 4031/4690 [36:47:15<6:00:13, 32.80s/it]
 86%|████████▌ | 4032/4690 [36:47:48<5:59:18, 32.76s/it]
 86%|████████▌ | 4033/4690 [36:48:20<5:57:41, 32.67s/it]
 86%|████████▌ | 4034/4690 [36:48:53<5:57:42, 32.72s/it]
 86%|████████▌ | 4035/4690 [36:49:26<5:58:31, 32.84s/it]
 86%|████████▌ | 4036/4690 [36:49:59<5:57:27, 32.79s/it]
 86%|████████▌ | 4037/4690 [36:50:32<5:57:04, 32.81s/it]
 86%|████████▌ | 4038/4690 [36:51:05<5:56:43, 32.83s/it]
 86%|████████▌ | 4039/4690 [36:51:37<5:55:23, 32.76s/it]
 86%|████████▌ | 4040/4690 [36:52:10<5:55:08, 32.78s/it]
                                                        
{'loss': '0.008014', 'grad_norm': '0.6925', 'learning_rate': '4.969e-07', 'epoch': '4.307'}

 86%|████████▌ | 4040/4690 [36:52:10<5:55:08, 32.78s/it]
 86%|████████▌ | 4041/4690 [36:52:43<5:55:35, 32.88s/it]
 86%|████████▌ | 4042/4690 [36:53:16<5:56:22, 33.00s/it]
 86%|████████▌ | 4043/4690 [36:53:49<5:54:47, 32.90s/it]
 86%|████████▌ | 4044/4690 [36:54:22<5:54:23, 32.92s/it]
 86%|████████▌ | 4045/4690 [36:54:55<5:53:34, 32.89s/it]
 86%|████████▋ | 4046/4690 [36:55:27<5:51:43, 32.77s/it]
 86%|████████▋ | 4047/4690 [36:56:00<5:51:01, 32.76s/it]
 86%|████████▋ | 4048/4690 [36:56:33<5:50:55, 32.80s/it]
 86%|████████▋ | 4049/4690 [36:57:06<5:50:08, 32.77s/it]
 86%|████████▋ | 4050/4690 [36:57:39<5:50:09, 32.83s/it]
                                                        
{'loss': '0.007642', 'grad_norm': '0.6828', 'learning_rate': '4.82e-07', 'epoch': '4.318'}

 86%|████████▋ | 4050/4690 [36:57:39<5:50:09, 32.83s/it]
 86%|████████▋ | 4051/4690 [36:58:11<5:49:14, 32.79s/it]
 86%|████████▋ | 4052/4690 [36:58:44<5:49:07, 32.83s/it]
 86%|████████▋ | 4053/4690 [36:59:17<5:48:37, 32.84s/it]
 86%|████████▋ | 4054/4690 [36:59:50<5:48:41, 32.90s/it]
 86%|████████▋ | 4055/4690 [37:00:23<5:47:43, 32.86s/it]
 86%|████████▋ | 4056/4690 [37:00:56<5:47:50, 32.92s/it]
 87%|████████▋ | 4057/4690 [37:01:29<5:47:36, 32.95s/it]
 87%|████████▋ | 4058/4690 [37:02:01<5:45:26, 32.79s/it]
 87%|████████▋ | 4059/4690 [37:02:34<5:44:57, 32.80s/it]
 87%|████████▋ | 4060/4690 [37:03:08<5:45:52, 32.94s/it]
                                                        
{'loss': '0.007198', 'grad_norm': '0.8624', 'learning_rate': '4.673e-07', 'epoch': '4.329'}

 87%|████████▋ | 4060/4690 [37:03:08<5:45:52, 32.94s/it]
 87%|████████▋ | 4061/4690 [37:03:40<5:45:11, 32.93s/it]
 87%|████████▋ | 4062/4690 [37:04:13<5:44:47, 32.94s/it]
 87%|████████▋ | 4063/4690 [37:04:46<5:44:01, 32.92s/it]
 87%|████████▋ | 4064/4690 [37:05:19<5:42:39, 32.84s/it]
 87%|████████▋ | 4065/4690 [37:05:52<5:41:17, 32.76s/it]
 87%|████████▋ | 4066/4690 [37:06:25<5:42:10, 32.90s/it]
 87%|████████▋ | 4067/4690 [37:06:58<5:41:44, 32.91s/it]
 87%|████████▋ | 4068/4690 [37:07:31<5:43:17, 33.12s/it]
 87%|████████▋ | 4069/4690 [37:08:04<5:42:29, 33.09s/it]
 87%|████████▋ | 4070/4690 [37:08:37<5:40:22, 32.94s/it]
                                                        
{'loss': '0.006997', 'grad_norm': '0.5632', 'learning_rate': '4.528e-07', 'epoch': '4.339'}

 87%|████████▋ | 4070/4690 [37:08:37<5:40:22, 32.94s/it]
 87%|████████▋ | 4071/4690 [37:09:10<5:39:38, 32.92s/it]
 87%|████████▋ | 4072/4690 [37:09:43<5:39:02, 32.92s/it]
 87%|████████▋ | 4073/4690 [37:10:15<5:38:06, 32.88s/it]
 87%|████████▋ | 4074/4690 [37:10:49<5:38:08, 32.94s/it]
 87%|████████▋ | 4075/4690 [37:11:21<5:36:39, 32.84s/it]
 87%|████████▋ | 4076/4690 [37:11:54<5:36:52, 32.92s/it]
 87%|████████▋ | 4077/4690 [37:12:27<5:35:42, 32.86s/it]
 87%|████████▋ | 4078/4690 [37:13:00<5:34:44, 32.82s/it]
 87%|████████▋ | 4079/4690 [37:13:33<5:34:08, 32.81s/it]
 87%|████████▋ | 4080/4690 [37:14:05<5:33:49, 32.83s/it]
                                                        
{'loss': '0.006266', 'grad_norm': '0.7464', 'learning_rate': '4.386e-07', 'epoch': '4.35'}

 87%|████████▋ | 4080/4690 [37:14:05<5:33:49, 32.83s/it]
 87%|████████▋ | 4081/4690 [37:14:38<5:33:34, 32.86s/it]
 87%|████████▋ | 4082/4690 [37:15:12<5:34:03, 32.97s/it]
 87%|████████▋ | 4083/4690 [37:15:44<5:32:30, 32.87s/it]
 87%|████████▋ | 4084/4690 [37:16:17<5:31:41, 32.84s/it]
 87%|████████▋ | 4085/4690 [37:16:50<5:30:16, 32.75s/it]
 87%|████████▋ | 4086/4690 [37:17:22<5:29:57, 32.78s/it]
 87%|████████▋ | 4087/4690 [37:17:55<5:29:03, 32.74s/it]
 87%|████████▋ | 4088/4690 [37:18:28<5:28:43, 32.76s/it]
 87%|████████▋ | 4089/4690 [37:19:00<5:27:17, 32.67s/it]
 87%|████████▋ | 4090/4690 [37:19:33<5:27:45, 32.78s/it]
                                                        
{'loss': '0.006089', 'grad_norm': '0.611', 'learning_rate': '4.245e-07', 'epoch': '4.361'}

 87%|████████▋ | 4090/4690 [37:19:33<5:27:45, 32.78s/it]
 87%|████████▋ | 4091/4690 [37:20:07<5:28:37, 32.92s/it]
 87%|████████▋ | 4092/4690 [37:20:40<5:29:03, 33.02s/it]
 87%|████████▋ | 4093/4690 [37:21:13<5:28:21, 33.00s/it]
 87%|████████▋ | 4094/4690 [37:21:45<5:26:26, 32.86s/it]
 87%|████████▋ | 4095/4690 [37:22:18<5:26:13, 32.90s/it]
 87%|████████▋ | 4096/4690 [37:22:51<5:25:12, 32.85s/it]
 87%|████████▋ | 4097/4690 [37:23:24<5:25:26, 32.93s/it]
 87%|████████▋ | 4098/4690 [37:23:57<5:24:17, 32.87s/it]
 87%|████████▋ | 4099/4690 [37:24:30<5:23:50, 32.88s/it]
 87%|████████▋ | 4100/4690 [37:25:03<5:23:41, 32.92s/it]
                                                        
{'loss': '0.007628', 'grad_norm': '0.7258', 'learning_rate': '4.107e-07', 'epoch': '4.371'}

 87%|████████▋ | 4100/4690 [37:25:03<5:23:41, 32.92s/it]
 87%|████████▋ | 4101/4690 [37:25:36<5:23:44, 32.98s/it]
 87%|████████▋ | 4102/4690 [37:26:09<5:22:35, 32.92s/it]
 87%|████████▋ | 4103/4690 [37:26:41<5:21:33, 32.87s/it]
 88%|████████▊ | 4104/4690 [37:27:14<5:19:55, 32.76s/it]
 88%|████████▊ | 4105/4690 [37:27:47<5:20:29, 32.87s/it]
 88%|████████▊ | 4106/4690 [37:28:20<5:20:02, 32.88s/it]
 88%|████████▊ | 4107/4690 [37:28:53<5:18:40, 32.80s/it]
 88%|████████▊ | 4108/4690 [37:29:25<5:17:49, 32.76s/it]
 88%|████████▊ | 4109/4690 [37:29:58<5:17:20, 32.77s/it]
 88%|████████▊ | 4110/4690 [37:30:31<5:17:12, 32.81s/it]
                                                        
{'loss': '0.007633', 'grad_norm': '0.8309', 'learning_rate': '3.971e-07', 'epoch': '4.382'}

 88%|████████▊ | 4110/4690 [37:30:31<5:17:12, 32.81s/it]
 88%|████████▊ | 4111/4690 [37:31:04<5:15:57, 32.74s/it]
 88%|████████▊ | 4112/4690 [37:31:36<5:15:19, 32.73s/it]
 88%|████████▊ | 4113/4690 [37:32:09<5:16:17, 32.89s/it]
 88%|████████▊ | 4114/4690 [37:32:42<5:16:05, 32.93s/it]
 88%|████████▊ | 4115/4690 [37:33:15<5:15:09, 32.89s/it]
 88%|████████▊ | 4116/4690 [37:33:48<5:14:34, 32.88s/it]
 88%|████████▊ | 4117/4690 [37:34:21<5:13:34, 32.84s/it]
 88%|████████▊ | 4118/4690 [37:34:54<5:12:28, 32.78s/it]
 88%|████████▊ | 4119/4690 [37:35:26<5:11:43, 32.76s/it]
 88%|████████▊ | 4120/4690 [37:35:59<5:10:14, 32.66s/it]
                                                        
{'loss': '0.008131', 'grad_norm': '1.03', 'learning_rate': '3.837e-07', 'epoch': '4.393'}

 88%|████████▊ | 4120/4690 [37:35:59<5:10:14, 32.66s/it]
 88%|████████▊ | 4121/4690 [37:36:32<5:10:20, 32.72s/it]
 88%|████████▊ | 4122/4690 [37:37:04<5:09:11, 32.66s/it]
 88%|████████▊ | 4123/4690 [37:37:37<5:08:27, 32.64s/it]
 88%|████████▊ | 4124/4690 [37:38:09<5:08:28, 32.70s/it]
 88%|████████▊ | 4125/4690 [37:38:42<5:08:07, 32.72s/it]
 88%|████████▊ | 4126/4690 [37:39:15<5:08:10, 32.79s/it]
 88%|████████▊ | 4127/4690 [37:39:48<5:07:40, 32.79s/it]
 88%|████████▊ | 4128/4690 [37:40:21<5:06:31, 32.73s/it]
 88%|████████▊ | 4129/4690 [37:40:53<5:06:04, 32.74s/it]
 88%|████████▊ | 4130/4690 [37:41:26<5:05:03, 32.69s/it]
                                                        
{'loss': '0.0068', 'grad_norm': '1.058', 'learning_rate': '3.706e-07', 'epoch': '4.403'}

 88%|████████▊ | 4130/4690 [37:41:26<5:05:03, 32.69s/it]
 88%|████████▊ | 4131/4690 [37:41:59<5:04:34, 32.69s/it]
 88%|████████▊ | 4132/4690 [37:42:31<5:04:10, 32.71s/it]
 88%|████████▊ | 4133/4690 [37:43:04<5:04:09, 32.76s/it]
 88%|████████▊ | 4134/4690 [37:43:37<5:03:52, 32.79s/it]
 88%|████████▊ | 4135/4690 [37:44:10<5:03:02, 32.76s/it]
 88%|████████▊ | 4136/4690 [37:44:43<5:02:44, 32.79s/it]
 88%|████████▊ | 4137/4690 [37:45:15<5:01:29, 32.71s/it]
 88%|████████▊ | 4138/4690 [37:45:48<5:00:31, 32.67s/it]
 88%|████████▊ | 4139/4690 [37:46:20<4:59:39, 32.63s/it]
 88%|████████▊ | 4140/4690 [37:46:53<5:00:22, 32.77s/it]
                                                        
{'loss': '0.006961', 'grad_norm': '0.7438', 'learning_rate': '3.577e-07', 'epoch': '4.414'}

 88%|████████▊ | 4140/4690 [37:46:53<5:00:22, 32.77s/it]
 88%|████████▊ | 4141/4690 [37:47:26<4:59:43, 32.76s/it]
 88%|████████▊ | 4142/4690 [37:47:59<5:00:13, 32.87s/it]
 88%|████████▊ | 4143/4690 [37:48:32<4:59:57, 32.90s/it]
 88%|████████▊ | 4144/4690 [37:49:05<5:00:07, 32.98s/it]
 88%|████████▊ | 4145/4690 [37:49:38<4:59:20, 32.96s/it]
 88%|████████▊ | 4146/4690 [37:50:11<4:59:10, 33.00s/it]
 88%|████████▊ | 4147/4690 [37:50:44<4:58:05, 32.94s/it]
 88%|████████▊ | 4148/4690 [37:51:17<4:56:47, 32.85s/it]
 88%|████████▊ | 4149/4690 [37:51:50<4:56:40, 32.90s/it]
 88%|████████▊ | 4150/4690 [37:52:23<4:55:31, 32.84s/it]
                                                        
{'loss': '0.00724', 'grad_norm': '0.9615', 'learning_rate': '3.449e-07', 'epoch': '4.425'}

 88%|████████▊ | 4150/4690 [37:52:23<4:55:31, 32.84s/it]
 89%|████████▊ | 4151/4690 [37:52:55<4:54:44, 32.81s/it]
 89%|████████▊ | 4152/4690 [37:53:28<4:54:17, 32.82s/it]
 89%|████████▊ | 4153/4690 [37:54:01<4:53:47, 32.83s/it]
 89%|████████▊ | 4154/4690 [37:54:34<4:54:22, 32.95s/it]
 89%|████████▊ | 4155/4690 [37:55:07<4:53:19, 32.90s/it]
 89%|████████▊ | 4156/4690 [37:55:40<4:53:27, 32.97s/it]
 89%|████████▊ | 4157/4690 [37:56:13<4:52:09, 32.89s/it]
 89%|████████▊ | 4158/4690 [37:56:46<4:51:42, 32.90s/it]
 89%|████████▊ | 4159/4690 [37:57:19<4:51:50, 32.98s/it]
 89%|████████▊ | 4160/4690 [37:57:52<4:50:21, 32.87s/it]
                                                        
{'loss': '0.007647', 'grad_norm': '0.397', 'learning_rate': '3.324e-07', 'epoch': '4.435'}

 89%|████████▊ | 4160/4690 [37:57:52<4:50:21, 32.87s/it]
 89%|████████▊ | 4161/4690 [37:58:24<4:49:42, 32.86s/it]
 89%|████████▊ | 4162/4690 [37:58:57<4:49:02, 32.85s/it]
 89%|████████▉ | 4163/4690 [37:59:30<4:48:21, 32.83s/it]
 89%|████████▉ | 4164/4690 [38:00:03<4:47:42, 32.82s/it]
 89%|████████▉ | 4165/4690 [38:00:35<4:46:05, 32.70s/it]
 89%|████████▉ | 4166/4690 [38:01:08<4:45:21, 32.68s/it]
 89%|████████▉ | 4167/4690 [38:01:41<4:45:40, 32.77s/it]
 89%|████████▉ | 4168/4690 [38:02:13<4:44:46, 32.73s/it]
 89%|████████▉ | 4169/4690 [38:02:46<4:44:40, 32.78s/it]
 89%|████████▉ | 4170/4690 [38:03:19<4:43:45, 32.74s/it]
                                                        
{'loss': '0.007255', 'grad_norm': '0.7504', 'learning_rate': '3.202e-07', 'epoch': '4.446'}

 89%|████████▉ | 4170/4690 [38:03:19<4:43:45, 32.74s/it]
 89%|████████▉ | 4171/4690 [38:03:52<4:43:37, 32.79s/it]
 89%|████████▉ | 4172/4690 [38:04:25<4:43:29, 32.84s/it]
 89%|████████▉ | 4173/4690 [38:04:58<4:43:47, 32.94s/it]
 89%|████████▉ | 4174/4690 [38:05:31<4:43:17, 32.94s/it]
 89%|████████▉ | 4175/4690 [38:06:04<4:43:33, 33.04s/it]
 89%|████████▉ | 4176/4690 [38:06:37<4:42:38, 32.99s/it]
 89%|████████▉ | 4177/4690 [38:07:10<4:42:09, 33.00s/it]
 89%|████████▉ | 4178/4690 [38:07:43<4:40:57, 32.93s/it]
 89%|████████▉ | 4179/4690 [38:08:16<4:39:58, 32.87s/it]
 89%|████████▉ | 4180/4690 [38:08:49<4:40:07, 32.96s/it]
                                                        
{'loss': '0.00657', 'grad_norm': '0.7222', 'learning_rate': '3.081e-07', 'epoch': '4.457'}

 89%|████████▉ | 4180/4690 [38:08:49<4:40:07, 32.96s/it]
 89%|████████▉ | 4181/4690 [38:09:22<4:40:01, 33.01s/it]
 89%|████████▉ | 4182/4690 [38:09:55<4:39:18, 32.99s/it]
 89%|████████▉ | 4183/4690 [38:10:28<4:39:16, 33.05s/it]
 89%|████████▉ | 4184/4690 [38:11:01<4:37:36, 32.92s/it]
 89%|████████▉ | 4185/4690 [38:11:33<4:36:08, 32.81s/it]
 89%|████████▉ | 4186/4690 [38:12:06<4:35:03, 32.75s/it]
 89%|████████▉ | 4187/4690 [38:12:39<4:34:50, 32.79s/it]
 89%|████████▉ | 4188/4690 [38:13:12<4:34:32, 32.81s/it]
 89%|████████▉ | 4189/4690 [38:13:45<4:34:32, 32.88s/it]
 89%|████████▉ | 4190/4690 [38:14:17<4:33:46, 32.85s/it]
                                                        
{'loss': '0.006313', 'grad_norm': '0.5658', 'learning_rate': '2.963e-07', 'epoch': '4.467'}

 89%|████████▉ | 4190/4690 [38:14:17<4:33:46, 32.85s/it]
 89%|████████▉ | 4191/4690 [38:14:50<4:33:23, 32.87s/it]
 89%|████████▉ | 4192/4690 [38:15:23<4:32:47, 32.87s/it]
 89%|████████▉ | 4193/4690 [38:15:56<4:32:00, 32.84s/it]
 89%|████████▉ | 4194/4690 [38:16:29<4:32:33, 32.97s/it]
 89%|████████▉ | 4195/4690 [38:17:02<4:32:10, 32.99s/it]
 89%|████████▉ | 4196/4690 [38:17:35<4:30:37, 32.87s/it]
 89%|████████▉ | 4197/4690 [38:18:08<4:29:46, 32.83s/it]
 90%|████████▉ | 4198/4690 [38:18:40<4:28:45, 32.78s/it]
 90%|████████▉ | 4199/4690 [38:19:13<4:28:58, 32.87s/it]
 90%|████████▉ | 4200/4690 [38:19:46<4:27:48, 32.79s/it]
                                                        
{'loss': '0.00792', 'grad_norm': '0.6631', 'learning_rate': '2.847e-07', 'epoch': '4.478'}

 90%|████████▉ | 4200/4690 [38:19:46<4:27:48, 32.79s/it]
 90%|████████▉ | 4201/4690 [38:20:19<4:27:05, 32.77s/it]
 90%|████████▉ | 4202/4690 [38:20:51<4:25:47, 32.68s/it]
 90%|████████▉ | 4203/4690 [38:21:24<4:26:50, 32.88s/it]
 90%|████████▉ | 4204/4690 [38:21:57<4:25:28, 32.77s/it]
 90%|████████▉ | 4205/4690 [38:22:30<4:25:17, 32.82s/it]
 90%|████████▉ | 4206/4690 [38:23:03<4:25:20, 32.89s/it]
 90%|████████▉ | 4207/4690 [38:23:36<4:24:35, 32.87s/it]
 90%|████████▉ | 4208/4690 [38:24:09<4:24:50, 32.97s/it]
 90%|████████▉ | 4209/4690 [38:24:42<4:24:23, 32.98s/it]
 90%|████████▉ | 4210/4690 [38:25:15<4:23:21, 32.92s/it]
                                                        
{'loss': '0.007127', 'grad_norm': '0.5966', 'learning_rate': '2.733e-07', 'epoch': '4.489'}

 90%|████████▉ | 4210/4690 [38:25:15<4:23:21, 32.92s/it]
 90%|████████▉ | 4211/4690 [38:25:48<4:23:33, 33.01s/it]
 90%|████████▉ | 4212/4690 [38:26:21<4:22:31, 32.95s/it]
 90%|████████▉ | 4213/4690 [38:26:53<4:20:49, 32.81s/it]
 90%|████████▉ | 4214/4690 [38:27:26<4:20:25, 32.83s/it]
 90%|████████▉ | 4215/4690 [38:27:59<4:19:58, 32.84s/it]
 90%|████████▉ | 4216/4690 [38:28:32<4:19:50, 32.89s/it]
 90%|████████▉ | 4217/4690 [38:29:05<4:19:05, 32.87s/it]
 90%|████████▉ | 4218/4690 [38:29:38<4:18:36, 32.87s/it]
 90%|████████▉ | 4219/4690 [38:30:10<4:17:18, 32.78s/it]
 90%|████████▉ | 4220/4690 [38:30:43<4:16:31, 32.75s/it]
                                                        
{'loss': '0.007394', 'grad_norm': '0.6656', 'learning_rate': '2.622e-07', 'epoch': '4.499'}

 90%|████████▉ | 4220/4690 [38:30:43<4:16:31, 32.75s/it]
 90%|█████████ | 4221/4690 [38:31:16<4:15:56, 32.74s/it]
 90%|█████████ | 4222/4690 [38:31:48<4:15:34, 32.76s/it]
 90%|█████████ | 4223/4690 [38:32:21<4:15:04, 32.77s/it]
 90%|█████████ | 4224/4690 [38:32:54<4:14:07, 32.72s/it]
 90%|█████████ | 4225/4690 [38:33:26<4:13:08, 32.66s/it]
 90%|█████████ | 4226/4690 [38:34:00<4:13:55, 32.83s/it]
 90%|█████████ | 4227/4690 [38:34:32<4:13:19, 32.83s/it]
 90%|█████████ | 4228/4690 [38:35:06<4:13:24, 32.91s/it]
 90%|█████████ | 4229/4690 [38:35:38<4:12:38, 32.88s/it]
 90%|█████████ | 4230/4690 [38:36:11<4:11:59, 32.87s/it]
                                                        
{'loss': '0.006955', 'grad_norm': '1.018', 'learning_rate': '2.513e-07', 'epoch': '4.51'}

 90%|█████████ | 4230/4690 [38:36:11<4:11:59, 32.87s/it]
 90%|█████████ | 4231/4690 [38:36:45<4:13:04, 33.08s/it]
 90%|█████████ | 4232/4690 [38:37:17<4:11:23, 32.93s/it]
 90%|█████████ | 4233/4690 [38:37:50<4:10:28, 32.88s/it]
 90%|█████████ | 4234/4690 [38:38:23<4:09:03, 32.77s/it]
 90%|█████████ | 4235/4690 [38:38:55<4:08:22, 32.75s/it]
 90%|█████████ | 4236/4690 [38:39:28<4:08:02, 32.78s/it]
 90%|█████████ | 4237/4690 [38:40:01<4:06:52, 32.70s/it]
 90%|█████████ | 4238/4690 [38:40:33<4:06:01, 32.66s/it]
 90%|█████████ | 4239/4690 [38:41:06<4:05:13, 32.62s/it]
 90%|█████████ | 4240/4690 [38:41:39<4:05:12, 32.69s/it]
                                                        
{'loss': '0.006717', 'grad_norm': '0.8349', 'learning_rate': '2.406e-07', 'epoch': '4.521'}

 90%|█████████ | 4240/4690 [38:41:39<4:05:12, 32.69s/it]
 90%|█████████ | 4241/4690 [38:42:12<4:05:35, 32.82s/it]
 90%|█████████ | 4242/4690 [38:42:44<4:04:23, 32.73s/it]
 90%|█████████ | 4243/4690 [38:43:17<4:03:56, 32.74s/it]
 90%|█████████ | 4244/4690 [38:43:50<4:03:38, 32.78s/it]
 91%|█████████ | 4245/4690 [38:44:23<4:02:46, 32.73s/it]
 91%|█████████ | 4246/4690 [38:44:55<4:02:32, 32.78s/it]
 91%|█████████ | 4247/4690 [38:45:28<4:02:20, 32.82s/it]
 91%|█████████ | 4248/4690 [38:46:01<4:02:00, 32.85s/it]
 91%|█████████ | 4249/4690 [38:46:34<4:00:45, 32.76s/it]
 91%|█████████ | 4250/4690 [38:47:07<4:01:08, 32.88s/it]
                                                        
{'loss': '0.007117', 'grad_norm': '0.6241', 'learning_rate': '2.301e-07', 'epoch': '4.531'}

 91%|█████████ | 4250/4690 [38:47:07<4:01:08, 32.88s/it]
 91%|█████████ | 4251/4690 [38:47:40<4:00:46, 32.91s/it]
 91%|█████████ | 4252/4690 [38:48:12<3:59:10, 32.76s/it]
 91%|█████████ | 4253/4690 [38:48:45<3:58:24, 32.73s/it]
 91%|█████████ | 4254/4690 [38:49:18<3:58:18, 32.79s/it]
 91%|█████████ | 4255/4690 [38:49:51<3:58:13, 32.86s/it]
 91%|█████████ | 4256/4690 [38:50:24<3:57:53, 32.89s/it]
 91%|█████████ | 4257/4690 [38:50:57<3:56:59, 32.84s/it]
 91%|█████████ | 4258/4690 [38:51:29<3:56:18, 32.82s/it]
 91%|█████████ | 4259/4690 [38:52:02<3:55:51, 32.83s/it]
 91%|█████████ | 4260/4690 [38:52:35<3:54:27, 32.72s/it]
                                                        
{'loss': '0.006584', 'grad_norm': '0.9569', 'learning_rate': '2.199e-07', 'epoch': '4.542'}

 91%|█████████ | 4260/4690 [38:52:35<3:54:27, 32.72s/it]
 91%|█████████ | 4261/4690 [38:53:08<3:54:46, 32.84s/it]
 91%|█████████ | 4262/4690 [38:53:41<3:54:09, 32.83s/it]
 91%|█████████ | 4263/4690 [38:54:14<3:55:04, 33.03s/it]
 91%|█████████ | 4264/4690 [38:54:47<3:53:34, 32.90s/it]
 91%|█████████ | 4265/4690 [38:55:20<3:53:08, 32.91s/it]
 91%|█████████ | 4266/4690 [38:55:53<3:52:47, 32.94s/it]
 91%|█████████ | 4267/4690 [38:56:25<3:51:39, 32.86s/it]
 91%|█████████ | 4268/4690 [38:56:58<3:50:54, 32.83s/it]
 91%|█████████ | 4269/4690 [38:57:31<3:50:33, 32.86s/it]
 91%|█████████ | 4270/4690 [38:58:04<3:50:17, 32.90s/it]
                                                        
{'loss': '0.007813', 'grad_norm': '0.5662', 'learning_rate': '2.099e-07', 'epoch': '4.553'}

 91%|█████████ | 4270/4690 [38:58:04<3:50:17, 32.90s/it]
 91%|█████████ | 4271/4690 [38:58:37<3:50:20, 32.98s/it]
 91%|█████████ | 4272/4690 [38:59:10<3:49:10, 32.90s/it]
 91%|█████████ | 4273/4690 [38:59:43<3:48:55, 32.94s/it]
 91%|█████████ | 4274/4690 [39:00:16<3:48:14, 32.92s/it]
 91%|█████████ | 4275/4690 [39:00:49<3:47:47, 32.93s/it]
 91%|█████████ | 4276/4690 [39:01:22<3:47:38, 32.99s/it]
 91%|█████████ | 4277/4690 [39:01:55<3:46:08, 32.85s/it]
 91%|█████████ | 4278/4690 [39:02:27<3:45:49, 32.89s/it]
 91%|█████████ | 4279/4690 [39:03:00<3:44:59, 32.85s/it]
 91%|█████████▏| 4280/4690 [39:03:33<3:44:08, 32.80s/it]
                                                        
{'loss': '0.007558', 'grad_norm': '0.8473', 'learning_rate': '2.001e-07', 'epoch': '4.563'}

 91%|█████████▏| 4280/4690 [39:03:33<3:44:08, 32.80s/it]
 91%|█████████▏| 4281/4690 [39:04:06<3:43:52, 32.84s/it]
 91%|█████████▏| 4282/4690 [39:04:39<3:43:11, 32.82s/it]
 91%|█████████▏| 4283/4690 [39:05:12<3:44:02, 33.03s/it]
 91%|█████████▏| 4284/4690 [39:05:45<3:42:15, 32.85s/it]
 91%|█████████▏| 4285/4690 [39:06:18<3:41:53, 32.87s/it]
 91%|█████████▏| 4286/4690 [39:06:50<3:41:29, 32.89s/it]
 91%|█████████▏| 4287/4690 [39:07:23<3:40:20, 32.80s/it]
 91%|█████████▏| 4288/4690 [39:07:56<3:39:33, 32.77s/it]
 91%|█████████▏| 4289/4690 [39:08:29<3:38:59, 32.77s/it]
 91%|█████████▏| 4290/4690 [39:09:01<3:37:46, 32.67s/it]
                                                        
{'loss': '0.006191', 'grad_norm': '0.5758', 'learning_rate': '1.905e-07', 'epoch': '4.574'}

 91%|█████████▏| 4290/4690 [39:09:01<3:37:46, 32.67s/it]
 91%|█████████▏| 4291/4690 [39:09:33<3:36:38, 32.58s/it]
 92%|█████████▏| 4292/4690 [39:10:06<3:36:57, 32.71s/it]
 92%|█████████▏| 4293/4690 [39:10:39<3:36:44, 32.76s/it]
 92%|█████████▏| 4294/4690 [39:11:12<3:35:46, 32.69s/it]
 92%|█████████▏| 4295/4690 [39:11:45<3:36:08, 32.83s/it]
 92%|█████████▏| 4296/4690 [39:12:18<3:35:15, 32.78s/it]
 92%|█████████▏| 4297/4690 [39:12:50<3:34:04, 32.68s/it]
 92%|█████████▏| 4298/4690 [39:13:23<3:33:51, 32.73s/it]
 92%|█████████▏| 4299/4690 [39:13:56<3:35:00, 32.99s/it]
 92%|█████████▏| 4300/4690 [39:14:30<3:34:33, 33.01s/it]
                                                        
{'loss': '0.007253', 'grad_norm': '0.7662', 'learning_rate': '1.812e-07', 'epoch': '4.585'}

 92%|█████████▏| 4300/4690 [39:14:30<3:34:33, 33.01s/it]
 92%|█████████▏| 4301/4690 [39:15:02<3:33:16, 32.90s/it]
 92%|█████████▏| 4302/4690 [39:15:35<3:32:38, 32.88s/it]
 92%|█████████▏| 4303/4690 [39:16:08<3:32:08, 32.89s/it]
 92%|█████████▏| 4304/4690 [39:16:40<3:30:50, 32.77s/it]
 92%|█████████▏| 4305/4690 [39:17:13<3:30:40, 32.83s/it]
 92%|█████████▏| 4306/4690 [39:17:46<3:29:45, 32.77s/it]
 92%|█████████▏| 4307/4690 [39:18:19<3:30:04, 32.91s/it]
 92%|█████████▏| 4308/4690 [39:18:52<3:30:01, 32.99s/it]
 92%|█████████▏| 4309/4690 [39:19:25<3:28:47, 32.88s/it]
 92%|█████████▏| 4310/4690 [39:19:58<3:28:18, 32.89s/it]
                                                        
{'loss': '0.007866', 'grad_norm': '1.104', 'learning_rate': '1.721e-07', 'epoch': '4.595'}

 92%|█████████▏| 4310/4690 [39:19:58<3:28:18, 32.89s/it]
 92%|█████████▏| 4311/4690 [39:20:31<3:27:26, 32.84s/it]
 92%|█████████▏| 4312/4690 [39:21:03<3:26:26, 32.77s/it]
 92%|█████████▏| 4313/4690 [39:21:36<3:25:36, 32.72s/it]
 92%|█████████▏| 4314/4690 [39:22:09<3:26:03, 32.88s/it]
 92%|█████████▏| 4315/4690 [39:22:42<3:25:15, 32.84s/it]
 92%|█████████▏| 4316/4690 [39:23:15<3:24:24, 32.79s/it]
 92%|█████████▏| 4317/4690 [39:23:47<3:23:54, 32.80s/it]
 92%|█████████▏| 4318/4690 [39:24:20<3:22:53, 32.72s/it]
 92%|█████████▏| 4319/4690 [39:24:53<3:23:50, 32.97s/it]
 92%|█████████▏| 4320/4690 [39:25:26<3:23:22, 32.98s/it]
                                                        
{'loss': '0.007532', 'grad_norm': '0.7364', 'learning_rate': '1.632e-07', 'epoch': '4.606'}

 92%|█████████▏| 4320/4690 [39:25:26<3:23:22, 32.98s/it]
 92%|█████████▏| 4321/4690 [39:26:00<3:23:18, 33.06s/it]
 92%|█████████▏| 4322/4690 [39:26:32<3:22:08, 32.96s/it]
 92%|█████████▏| 4323/4690 [39:27:05<3:20:54, 32.85s/it]
 92%|█████████▏| 4324/4690 [39:27:38<3:20:22, 32.85s/it]
 92%|█████████▏| 4325/4690 [39:28:10<3:19:22, 32.77s/it]
 92%|█████████▏| 4326/4690 [39:28:43<3:18:32, 32.73s/it]
 92%|█████████▏| 4327/4690 [39:29:16<3:17:51, 32.70s/it]
 92%|█████████▏| 4328/4690 [39:29:49<3:18:00, 32.82s/it]
 92%|█████████▏| 4329/4690 [39:30:22<3:17:59, 32.91s/it]
 92%|█████████▏| 4330/4690 [39:30:54<3:16:45, 32.79s/it]
                                                        
{'loss': '0.006224', 'grad_norm': '0.5404', 'learning_rate': '1.546e-07', 'epoch': '4.617'}

 92%|█████████▏| 4330/4690 [39:30:54<3:16:45, 32.79s/it]
 92%|█████████▏| 4331/4690 [39:31:27<3:16:18, 32.81s/it]
 92%|█████████▏| 4332/4690 [39:32:00<3:15:31, 32.77s/it]
 92%|█████████▏| 4333/4690 [39:32:33<3:15:05, 32.79s/it]
 92%|█████████▏| 4334/4690 [39:33:05<3:14:11, 32.73s/it]
 92%|█████████▏| 4335/4690 [39:33:38<3:13:54, 32.77s/it]
 92%|█████████▏| 4336/4690 [39:34:11<3:13:58, 32.88s/it]
 92%|█████████▏| 4337/4690 [39:34:44<3:13:43, 32.93s/it]
 92%|█████████▏| 4338/4690 [39:35:18<3:13:36, 33.00s/it]
 93%|█████████▎| 4339/4690 [39:35:50<3:12:23, 32.89s/it]
 93%|█████████▎| 4340/4690 [39:36:23<3:11:36, 32.85s/it]
                                                        
{'loss': '0.006037', 'grad_norm': '1.105', 'learning_rate': '1.462e-07', 'epoch': '4.627'}

 93%|█████████▎| 4340/4690 [39:36:23<3:11:36, 32.85s/it]
 93%|█████████▎| 4341/4690 [39:36:56<3:11:29, 32.92s/it]
 93%|█████████▎| 4342/4690 [39:37:29<3:10:30, 32.85s/it]
 93%|█████████▎| 4343/4690 [39:38:02<3:10:20, 32.91s/it]
 93%|█████████▎| 4344/4690 [39:38:34<3:08:59, 32.77s/it]
 93%|█████████▎| 4345/4690 [39:39:07<3:08:27, 32.78s/it]
 93%|█████████▎| 4346/4690 [39:39:40<3:08:08, 32.81s/it]
 93%|█████████▎| 4347/4690 [39:40:12<3:07:04, 32.72s/it]
 93%|█████████▎| 4348/4690 [39:40:45<3:06:17, 32.68s/it]
 93%|█████████▎| 4349/4690 [39:41:18<3:05:41, 32.67s/it]
 93%|█████████▎| 4350/4690 [39:41:51<3:06:57, 32.99s/it]
                                                        
{'loss': '0.00619', 'grad_norm': '0.707', 'learning_rate': '1.38e-07', 'epoch': '4.638'}

 93%|█████████▎| 4350/4690 [39:41:51<3:06:57, 32.99s/it]
 93%|█████████▎| 4351/4690 [39:42:24<3:06:14, 32.96s/it]
 93%|█████████▎| 4352/4690 [39:42:57<3:05:34, 32.94s/it]
 93%|█████████▎| 4353/4690 [39:43:30<3:04:07, 32.78s/it]
 93%|█████████▎| 4354/4690 [39:44:02<3:02:42, 32.63s/it]
 93%|█████████▎| 4355/4690 [39:44:35<3:02:27, 32.68s/it]
 93%|█████████▎| 4356/4690 [39:45:08<3:02:05, 32.71s/it]
 93%|█████████▎| 4357/4690 [39:45:40<3:01:53, 32.77s/it]
 93%|█████████▎| 4358/4690 [39:46:13<3:01:05, 32.73s/it]
 93%|█████████▎| 4359/4690 [39:46:46<3:00:10, 32.66s/it]
 93%|█████████▎| 4360/4690 [39:47:19<3:00:08, 32.75s/it]
                                                        
{'loss': '0.006188', 'grad_norm': '1.156', 'learning_rate': '1.301e-07', 'epoch': '4.649'}

 93%|█████████▎| 4360/4690 [39:47:19<3:00:08, 32.75s/it]
 93%|█████████▎| 4361/4690 [39:47:51<2:59:48, 32.79s/it]
 93%|█████████▎| 4362/4690 [39:48:24<2:59:39, 32.86s/it]
 93%|█████████▎| 4363/4690 [39:48:57<2:58:18, 32.72s/it]
 93%|█████████▎| 4364/4690 [39:49:30<2:57:53, 32.74s/it]
 93%|█████████▎| 4365/4690 [39:50:02<2:57:10, 32.71s/it]
 93%|█████████▎| 4366/4690 [39:50:35<2:56:47, 32.74s/it]
 93%|█████████▎| 4367/4690 [39:51:08<2:55:48, 32.66s/it]
 93%|█████████▎| 4368/4690 [39:51:40<2:55:09, 32.64s/it]
 93%|█████████▎| 4369/4690 [39:52:13<2:54:37, 32.64s/it]
 93%|█████████▎| 4370/4690 [39:52:45<2:54:04, 32.64s/it]
                                                        
{'loss': '0.006015', 'grad_norm': '0.5815', 'learning_rate': '1.224e-07', 'epoch': '4.659'}

 93%|█████████▎| 4370/4690 [39:52:45<2:54:04, 32.64s/it]
 93%|█████████▎| 4371/4690 [39:53:18<2:54:14, 32.77s/it]
 93%|█████████▎| 4372/4690 [39:53:51<2:53:57, 32.82s/it]
 93%|█████████▎| 4373/4690 [39:54:24<2:53:18, 32.80s/it]
 93%|█████████▎| 4374/4690 [39:54:57<2:52:21, 32.73s/it]
 93%|█████████▎| 4375/4690 [39:55:30<2:52:23, 32.84s/it]
 93%|█████████▎| 4376/4690 [39:56:03<2:51:53, 32.85s/it]
 93%|█████████▎| 4377/4690 [39:56:35<2:51:10, 32.81s/it]
 93%|█████████▎| 4378/4690 [39:57:08<2:50:22, 32.77s/it]
 93%|█████████▎| 4379/4690 [39:57:41<2:50:03, 32.81s/it]
 93%|█████████▎| 4380/4690 [39:58:14<2:49:37, 32.83s/it]
                                                        
{'loss': '0.00707', 'grad_norm': '0.7578', 'learning_rate': '1.149e-07', 'epoch': '4.67'}

 93%|█████████▎| 4380/4690 [39:58:14<2:49:37, 32.83s/it]
 93%|█████████▎| 4381/4690 [39:58:47<2:48:56, 32.81s/it]
 93%|█████████▎| 4382/4690 [39:59:20<2:48:53, 32.90s/it]
 93%|█████████▎| 4383/4690 [39:59:52<2:47:56, 32.82s/it]
 93%|█████████▎| 4384/4690 [40:00:25<2:47:30, 32.84s/it]
 93%|█████████▎| 4385/4690 [40:00:58<2:47:19, 32.92s/it]
 94%|█████████▎| 4386/4690 [40:01:31<2:46:30, 32.86s/it]
 94%|█████████▎| 4387/4690 [40:02:04<2:45:42, 32.81s/it]
 94%|█████████▎| 4388/4690 [40:02:36<2:44:42, 32.72s/it]
 94%|█████████▎| 4389/4690 [40:03:09<2:44:12, 32.73s/it]
 94%|█████████▎| 4390/4690 [40:03:42<2:43:49, 32.76s/it]
                                                        
{'loss': '0.006456', 'grad_norm': '1.324', 'learning_rate': '1.076e-07', 'epoch': '4.681'}

 94%|█████████▎| 4390/4690 [40:03:42<2:43:49, 32.76s/it]
 94%|█████████▎| 4391/4690 [40:04:14<2:42:47, 32.67s/it]
 94%|█████████▎| 4392/4690 [40:04:47<2:42:54, 32.80s/it]
 94%|█████████▎| 4393/4690 [40:05:20<2:41:40, 32.66s/it]
 94%|█████████▎| 4394/4690 [40:05:52<2:40:48, 32.60s/it]
 94%|█████████▎| 4395/4690 [40:06:25<2:40:42, 32.69s/it]
 94%|█████████▎| 4396/4690 [40:06:58<2:40:04, 32.67s/it]
 94%|█████████▍| 4397/4690 [40:07:30<2:39:10, 32.60s/it]
 94%|█████████▍| 4398/4690 [40:08:03<2:39:15, 32.72s/it]
 94%|█████████▍| 4399/4690 [40:08:36<2:38:23, 32.66s/it]
 94%|█████████▍| 4400/4690 [40:09:09<2:38:19, 32.76s/it]
                                                        
{'loss': '0.005081', 'grad_norm': '0.5423', 'learning_rate': '1.006e-07', 'epoch': '4.691'}

 94%|█████████▍| 4400/4690 [40:09:09<2:38:19, 32.76s/it]
 94%|█████████▍| 4401/4690 [40:09:41<2:37:23, 32.68s/it]
 94%|█████████▍| 4402/4690 [40:10:14<2:37:09, 32.74s/it]
 94%|█████████▍| 4403/4690 [40:10:47<2:37:03, 32.84s/it]
 94%|█████████▍| 4404/4690 [40:11:20<2:36:16, 32.78s/it]
 94%|█████████▍| 4405/4690 [40:11:53<2:35:51, 32.81s/it]
 94%|█████████▍| 4406/4690 [40:12:26<2:35:34, 32.87s/it]
 94%|█████████▍| 4407/4690 [40:12:59<2:35:16, 32.92s/it]
 94%|█████████▍| 4408/4690 [40:13:31<2:34:15, 32.82s/it]
 94%|█████████▍| 4409/4690 [40:14:04<2:33:50, 32.85s/it]
 94%|█████████▍| 4410/4690 [40:14:37<2:33:11, 32.83s/it]
                                                        
{'loss': '0.006654', 'grad_norm': '0.6622', 'learning_rate': '9.385e-08', 'epoch': '4.702'}

 94%|█████████▍| 4410/4690 [40:14:37<2:33:11, 32.83s/it]
 94%|█████████▍| 4411/4690 [40:15:10<2:33:02, 32.91s/it]
 94%|█████████▍| 4412/4690 [40:15:43<2:32:14, 32.86s/it]
 94%|█████████▍| 4413/4690 [40:16:15<2:31:08, 32.74s/it]
 94%|█████████▍| 4414/4690 [40:16:48<2:30:45, 32.78s/it]
 94%|█████████▍| 4415/4690 [40:17:21<2:30:00, 32.73s/it]
 94%|█████████▍| 4416/4690 [40:17:54<2:30:19, 32.92s/it]
 94%|█████████▍| 4417/4690 [40:18:27<2:29:25, 32.84s/it]
 94%|█████████▍| 4418/4690 [40:19:00<2:29:03, 32.88s/it]
 94%|█████████▍| 4419/4690 [40:19:33<2:28:55, 32.97s/it]
 94%|█████████▍| 4420/4690 [40:20:06<2:28:04, 32.91s/it]
                                                        
{'loss': '0.006648', 'grad_norm': '0.4621', 'learning_rate': '8.731e-08', 'epoch': '4.713'}

 94%|█████████▍| 4420/4690 [40:20:06<2:28:04, 32.91s/it]
 94%|█████████▍| 4421/4690 [40:20:39<2:27:33, 32.91s/it]
 94%|█████████▍| 4422/4690 [40:21:12<2:27:02, 32.92s/it]
 94%|█████████▍| 4423/4690 [40:21:44<2:26:07, 32.84s/it]
 94%|█████████▍| 4424/4690 [40:22:17<2:25:36, 32.84s/it]
 94%|█████████▍| 4425/4690 [40:22:50<2:24:53, 32.81s/it]
 94%|█████████▍| 4426/4690 [40:23:23<2:24:51, 32.92s/it]
 94%|█████████▍| 4427/4690 [40:23:56<2:24:12, 32.90s/it]
 94%|█████████▍| 4428/4690 [40:24:29<2:23:23, 32.84s/it]
 94%|█████████▍| 4429/4690 [40:25:01<2:22:33, 32.77s/it]
 94%|█████████▍| 4430/4690 [40:25:34<2:22:33, 32.90s/it]
                                                        
{'loss': '0.006399', 'grad_norm': '0.5993', 'learning_rate': '8.101e-08', 'epoch': '4.723'}

 94%|█████████▍| 4430/4690 [40:25:34<2:22:33, 32.90s/it]
 94%|█████████▍| 4431/4690 [40:26:07<2:21:58, 32.89s/it]
 94%|█████████▍| 4432/4690 [40:26:40<2:21:31, 32.91s/it]
 95%|█████████▍| 4433/4690 [40:27:13<2:21:29, 33.03s/it]
 95%|█████████▍| 4434/4690 [40:27:46<2:20:35, 32.95s/it]
 95%|█████████▍| 4435/4690 [40:28:19<2:19:58, 32.93s/it]
 95%|█████████▍| 4436/4690 [40:28:52<2:19:06, 32.86s/it]
 95%|█████████▍| 4437/4690 [40:29:24<2:18:08, 32.76s/it]
 95%|█████████▍| 4438/4690 [40:29:57<2:17:35, 32.76s/it]
 95%|█████████▍| 4439/4690 [40:30:30<2:16:55, 32.73s/it]
 95%|█████████▍| 4440/4690 [40:31:02<2:16:17, 32.71s/it]
                                                        
{'loss': '0.006977', 'grad_norm': '0.4254', 'learning_rate': '7.493e-08', 'epoch': '4.734'}

 95%|█████████▍| 4440/4690 [40:31:02<2:16:17, 32.71s/it]
 95%|█████████▍| 4441/4690 [40:31:35<2:15:46, 32.72s/it]
 95%|█████████▍| 4442/4690 [40:32:08<2:14:45, 32.60s/it]
 95%|█████████▍| 4443/4690 [40:32:40<2:14:24, 32.65s/it]
 95%|█████████▍| 4444/4690 [40:33:13<2:14:14, 32.74s/it]
 95%|█████████▍| 4445/4690 [40:33:46<2:13:44, 32.75s/it]
 95%|█████████▍| 4446/4690 [40:34:19<2:13:15, 32.77s/it]
 95%|█████████▍| 4447/4690 [40:34:52<2:12:48, 32.79s/it]
 95%|█████████▍| 4448/4690 [40:35:24<2:12:01, 32.73s/it]
 95%|█████████▍| 4449/4690 [40:35:57<2:11:47, 32.81s/it]
 95%|█████████▍| 4450/4690 [40:36:30<2:11:24, 32.85s/it]
                                                        
{'loss': '0.006485', 'grad_norm': '0.6048', 'learning_rate': '6.909e-08', 'epoch': '4.745'}

 95%|█████████▍| 4450/4690 [40:36:30<2:11:24, 32.85s/it]
 95%|█████████▍| 4451/4690 [40:37:03<2:10:43, 32.82s/it]
 95%|█████████▍| 4452/4690 [40:37:36<2:10:05, 32.80s/it]
 95%|█████████▍| 4453/4690 [40:38:08<2:09:28, 32.78s/it]
 95%|█████████▍| 4454/4690 [40:38:41<2:08:39, 32.71s/it]
 95%|█████████▍| 4455/4690 [40:39:14<2:08:05, 32.70s/it]
 95%|█████████▌| 4456/4690 [40:39:47<2:08:32, 32.96s/it]
 95%|█████████▌| 4457/4690 [40:40:20<2:07:55, 32.94s/it]
 95%|█████████▌| 4458/4690 [40:40:53<2:07:02, 32.86s/it]
 95%|█████████▌| 4459/4690 [40:41:26<2:06:47, 32.93s/it]
 95%|█████████▌| 4460/4690 [40:41:59<2:06:11, 32.92s/it]
                                                        
{'loss': '0.007485', 'grad_norm': '0.6629', 'learning_rate': '6.349e-08', 'epoch': '4.755'}

 95%|█████████▌| 4460/4690 [40:41:59<2:06:11, 32.92s/it]
 95%|█████████▌| 4461/4690 [40:42:32<2:06:04, 33.03s/it]
 95%|█████████▌| 4462/4690 [40:43:05<2:05:33, 33.04s/it]
 95%|█████████▌| 4463/4690 [40:43:38<2:04:47, 32.99s/it]
 95%|█████████▌| 4464/4690 [40:44:11<2:03:55, 32.90s/it]
 95%|█████████▌| 4465/4690 [40:44:44<2:03:51, 33.03s/it]
 95%|█████████▌| 4466/4690 [40:45:17<2:03:28, 33.07s/it]
 95%|█████████▌| 4467/4690 [40:45:50<2:02:44, 33.03s/it]
 95%|█████████▌| 4468/4690 [40:46:23<2:01:33, 32.85s/it]
 95%|█████████▌| 4469/4690 [40:46:56<2:01:15, 32.92s/it]
 95%|█████████▌| 4470/4690 [40:47:29<2:01:30, 33.14s/it]
                                                        
{'loss': '0.006779', 'grad_norm': '0.9288', 'learning_rate': '5.812e-08', 'epoch': '4.766'}

 95%|█████████▌| 4470/4690 [40:47:29<2:01:30, 33.14s/it]
 95%|█████████▌| 4471/4690 [40:48:02<2:00:31, 33.02s/it]
 95%|█████████▌| 4472/4690 [40:48:35<1:59:41, 32.94s/it]
 95%|█████████▌| 4473/4690 [40:49:08<1:59:19, 32.99s/it]
 95%|█████████▌| 4474/4690 [40:49:41<1:58:36, 32.95s/it]
 95%|█████████▌| 4475/4690 [40:50:13<1:57:41, 32.84s/it]
 95%|█████████▌| 4476/4690 [40:50:46<1:57:15, 32.87s/it]
 95%|█████████▌| 4477/4690 [40:51:19<1:56:32, 32.83s/it]
 95%|█████████▌| 4478/4690 [40:51:52<1:55:43, 32.75s/it]
 96%|█████████▌| 4479/4690 [40:52:24<1:55:01, 32.71s/it]
 96%|█████████▌| 4480/4690 [40:52:57<1:54:25, 32.69s/it]
                                                        
{'loss': '0.005671', 'grad_norm': '0.482', 'learning_rate': '5.299e-08', 'epoch': '4.777'}

 96%|█████████▌| 4480/4690 [40:52:57<1:54:25, 32.69s/it]
 96%|█████████▌| 4481/4690 [40:53:30<1:53:59, 32.72s/it]
 96%|█████████▌| 4482/4690 [40:54:02<1:53:33, 32.76s/it]
 96%|█████████▌| 4483/4690 [40:54:35<1:53:14, 32.82s/it]
 96%|█████████▌| 4484/4690 [40:55:08<1:52:51, 32.87s/it]
 96%|█████████▌| 4485/4690 [40:55:41<1:51:37, 32.67s/it]
 96%|█████████▌| 4486/4690 [40:56:13<1:51:11, 32.70s/it]
 96%|█████████▌| 4487/4690 [40:56:46<1:50:33, 32.68s/it]
 96%|█████████▌| 4488/4690 [40:57:19<1:50:05, 32.70s/it]
 96%|█████████▌| 4489/4690 [40:57:52<1:49:52, 32.80s/it]
 96%|█████████▌| 4490/4690 [40:58:24<1:49:10, 32.75s/it]
                                                        
{'loss': '0.006701', 'grad_norm': '0.5393', 'learning_rate': '4.81e-08', 'epoch': '4.787'}

 96%|█████████▌| 4490/4690 [40:58:24<1:49:10, 32.75s/it]
 96%|█████████▌| 4491/4690 [40:58:58<1:49:00, 32.87s/it]
 96%|█████████▌| 4492/4690 [40:59:30<1:48:05, 32.76s/it]
 96%|█████████▌| 4493/4690 [41:00:03<1:47:21, 32.70s/it]
 96%|█████████▌| 4494/4690 [41:00:36<1:47:00, 32.76s/it]
 96%|█████████▌| 4495/4690 [41:01:08<1:46:26, 32.75s/it]
 96%|█████████▌| 4496/4690 [41:01:41<1:45:50, 32.73s/it]
 96%|█████████▌| 4497/4690 [41:02:14<1:45:26, 32.78s/it]
 96%|█████████▌| 4498/4690 [41:02:47<1:44:55, 32.79s/it]
 96%|█████████▌| 4499/4690 [41:03:19<1:44:22, 32.79s/it]
 96%|█████████▌| 4500/4690 [41:03:52<1:43:39, 32.73s/it]
                                                        
{'loss': '0.006184', 'grad_norm': '0.4592', 'learning_rate': '4.344e-08', 'epoch': '4.798'}

 96%|█████████▌| 4500/4690 [41:03:52<1:43:39, 32.73s/it]
 96%|█████████▌| 4501/4690 [41:04:25<1:43:42, 32.92s/it]
 96%|█████████▌| 4502/4690 [41:04:58<1:43:13, 32.95s/it]
 96%|█████████▌| 4503/4690 [41:05:31<1:42:33, 32.91s/it]
 96%|█████████▌| 4504/4690 [41:06:04<1:41:56, 32.88s/it]
 96%|█████████▌| 4505/4690 [41:06:37<1:41:11, 32.82s/it]
 96%|█████████▌| 4506/4690 [41:07:09<1:40:26, 32.75s/it]
 96%|█████████▌| 4507/4690 [41:07:42<1:40:13, 32.86s/it]
 96%|█████████▌| 4508/4690 [41:08:15<1:39:42, 32.87s/it]
 96%|█████████▌| 4509/4690 [41:08:48<1:39:10, 32.87s/it]
 96%|█████████▌| 4510/4690 [41:09:21<1:38:46, 32.92s/it]
                                                        
{'loss': '0.006321', 'grad_norm': '0.7338', 'learning_rate': '3.901e-08', 'epoch': '4.809'}

 96%|█████████▌| 4510/4690 [41:09:21<1:38:46, 32.92s/it]
 96%|█████████▌| 4511/4690 [41:09:55<1:38:34, 33.04s/it]
 96%|█████████▌| 4512/4690 [41:10:27<1:37:36, 32.90s/it]
 96%|█████████▌| 4513/4690 [41:11:00<1:36:48, 32.81s/it]
 96%|█████████▌| 4514/4690 [41:11:33<1:36:40, 32.96s/it]
 96%|█████████▋| 4515/4690 [41:12:06<1:36:04, 32.94s/it]
 96%|█████████▋| 4516/4690 [41:12:39<1:35:13, 32.84s/it]
 96%|█████████▋| 4517/4690 [41:13:12<1:34:50, 32.89s/it]
 96%|█████████▋| 4518/4690 [41:13:45<1:34:26, 32.95s/it]
 96%|█████████▋| 4519/4690 [41:14:17<1:33:40, 32.87s/it]
 96%|█████████▋| 4520/4690 [41:14:50<1:33:03, 32.84s/it]
                                                        
{'loss': '0.007506', 'grad_norm': '0.8495', 'learning_rate': '3.483e-08', 'epoch': '4.819'}

 96%|█████████▋| 4520/4690 [41:14:50<1:33:03, 32.84s/it]
 96%|█████████▋| 4521/4690 [41:15:23<1:32:29, 32.84s/it]
 96%|█████████▋| 4522/4690 [41:15:56<1:31:55, 32.83s/it]
 96%|█████████▋| 4523/4690 [41:16:29<1:31:37, 32.92s/it]
 96%|█████████▋| 4524/4690 [41:17:01<1:30:48, 32.82s/it]
 96%|█████████▋| 4525/4690 [41:17:34<1:29:57, 32.71s/it]
 97%|█████████▋| 4526/4690 [41:18:07<1:29:44, 32.84s/it]
 97%|█████████▋| 4527/4690 [41:18:40<1:29:15, 32.85s/it]
 97%|█████████▋| 4528/4690 [41:19:13<1:28:44, 32.87s/it]
 97%|█████████▋| 4529/4690 [41:19:45<1:27:54, 32.76s/it]
 97%|█████████▋| 4530/4690 [41:20:18<1:27:34, 32.84s/it]
                                                        
{'loss': '0.006157', 'grad_norm': '0.608', 'learning_rate': '3.088e-08', 'epoch': '4.83'}

 97%|█████████▋| 4530/4690 [41:20:18<1:27:34, 32.84s/it]
 97%|█████████▋| 4531/4690 [41:20:51<1:27:06, 32.87s/it]
 97%|█████████▋| 4532/4690 [41:21:24<1:26:21, 32.80s/it]
 97%|█████████▋| 4533/4690 [41:21:57<1:25:45, 32.77s/it]
 97%|█████████▋| 4534/4690 [41:22:29<1:25:08, 32.75s/it]
 97%|█████████▋| 4535/4690 [41:23:02<1:24:28, 32.70s/it]
 97%|█████████▋| 4536/4690 [41:23:34<1:23:48, 32.65s/it]
 97%|█████████▋| 4537/4690 [41:24:07<1:23:22, 32.69s/it]
 97%|█████████▋| 4538/4690 [41:24:40<1:23:00, 32.77s/it]
 97%|█████████▋| 4539/4690 [41:25:13<1:22:32, 32.79s/it]
 97%|█████████▋| 4540/4690 [41:25:46<1:21:54, 32.77s/it]
                                                        
{'loss': '0.006017', 'grad_norm': '0.5259', 'learning_rate': '2.716e-08', 'epoch': '4.841'}

 97%|█████████▋| 4540/4690 [41:25:46<1:21:54, 32.77s/it]
 97%|█████████▋| 4541/4690 [41:26:19<1:21:27, 32.80s/it]
 97%|█████████▋| 4542/4690 [41:26:51<1:20:44, 32.73s/it]
 97%|█████████▋| 4543/4690 [41:27:25<1:20:50, 32.99s/it]
 97%|█████████▋| 4544/4690 [41:27:57<1:19:50, 32.81s/it]
 97%|█████████▋| 4545/4690 [41:28:30<1:19:10, 32.76s/it]
 97%|█████████▋| 4546/4690 [41:29:03<1:18:53, 32.87s/it]
 97%|█████████▋| 4547/4690 [41:29:36<1:18:26, 32.91s/it]
 97%|█████████▋| 4548/4690 [41:30:09<1:17:36, 32.79s/it]
 97%|█████████▋| 4549/4690 [41:30:42<1:17:23, 32.94s/it]
 97%|█████████▋| 4550/4690 [41:31:14<1:16:32, 32.80s/it]
                                                        
{'loss': '0.006343', 'grad_norm': '0.4766', 'learning_rate': '2.369e-08', 'epoch': '4.851'}

 97%|█████████▋| 4550/4690 [41:31:14<1:16:32, 32.80s/it]
 97%|█████████▋| 4551/4690 [41:31:47<1:15:42, 32.68s/it]
 97%|█████████▋| 4552/4690 [41:32:19<1:15:08, 32.67s/it]
 97%|█████████▋| 4553/4690 [41:32:52<1:14:44, 32.73s/it]
 97%|█████████▋| 4554/4690 [41:33:25<1:14:14, 32.76s/it]
 97%|█████████▋| 4555/4690 [41:33:58<1:13:49, 32.81s/it]
 97%|█████████▋| 4556/4690 [41:34:31<1:13:19, 32.83s/it]
 97%|█████████▋| 4557/4690 [41:35:04<1:13:03, 32.96s/it]
 97%|█████████▋| 4558/4690 [41:35:37<1:12:28, 32.94s/it]
 97%|█████████▋| 4559/4690 [41:36:10<1:11:51, 32.91s/it]
 97%|█████████▋| 4560/4690 [41:36:43<1:11:37, 33.06s/it]
                                                        
{'loss': '0.006308', 'grad_norm': '0.6367', 'learning_rate': '2.045e-08', 'epoch': '4.862'}

 97%|█████████▋| 4560/4690 [41:36:43<1:11:37, 33.06s/it]
 97%|█████████▋| 4561/4690 [41:37:16<1:10:59, 33.02s/it]
 97%|█████████▋| 4562/4690 [41:37:49<1:10:07, 32.87s/it]
 97%|█████████▋| 4563/4690 [41:38:22<1:09:45, 32.96s/it]
 97%|█████████▋| 4564/4690 [41:38:55<1:09:10, 32.94s/it]
 97%|█████████▋| 4565/4690 [41:39:27<1:08:16, 32.77s/it]
 97%|█████████▋| 4566/4690 [41:40:00<1:07:40, 32.74s/it]
 97%|█████████▋| 4567/4690 [41:40:33<1:07:14, 32.80s/it]
 97%|█████████▋| 4568/4690 [41:41:06<1:06:41, 32.80s/it]
 97%|█████████▋| 4569/4690 [41:41:38<1:06:12, 32.83s/it]
 97%|█████████▋| 4570/4690 [41:42:11<1:05:40, 32.84s/it]
                                                        
{'loss': '0.007026', 'grad_norm': '0.4913', 'learning_rate': '1.745e-08', 'epoch': '4.873'}

 97%|█████████▋| 4570/4690 [41:42:11<1:05:40, 32.84s/it]
 97%|█████████▋| 4571/4690 [41:42:44<1:05:04, 32.81s/it]
 97%|█████████▋| 4572/4690 [41:43:17<1:04:40, 32.89s/it]
 98%|█████████▊| 4573/4690 [41:43:50<1:04:15, 32.96s/it]
 98%|█████████▊| 4574/4690 [41:44:24<1:03:55, 33.06s/it]
 98%|█████████▊| 4575/4690 [41:44:56<1:03:07, 32.93s/it]
 98%|█████████▊| 4576/4690 [41:45:29<1:02:29, 32.89s/it]
 98%|█████████▊| 4577/4690 [41:46:02<1:02:02, 32.94s/it]
 98%|█████████▊| 4578/4690 [41:46:35<1:01:33, 32.98s/it]
 98%|█████████▊| 4579/4690 [41:47:08<1:01:00, 32.98s/it]
 98%|█████████▊| 4580/4690 [41:47:41<1:00:24, 32.95s/it]
                                                        
{'loss': '0.007313', 'grad_norm': '0.6579', 'learning_rate': '1.468e-08', 'epoch': '4.883'}

 98%|█████████▊| 4580/4690 [41:47:41<1:00:24, 32.95s/it]
 98%|█████████▊| 4581/4690 [41:48:14<59:57, 33.00s/it]  
 98%|█████████▊| 4582/4690 [41:48:47<59:21, 32.97s/it]
 98%|█████████▊| 4583/4690 [41:49:20<58:40, 32.90s/it]
 98%|█████████▊| 4584/4690 [41:49:53<58:04, 32.87s/it]
 98%|█████████▊| 4585/4690 [41:50:25<57:17, 32.74s/it]
 98%|█████████▊| 4586/4690 [41:50:58<56:48, 32.77s/it]
 98%|█████████▊| 4587/4690 [41:51:31<56:17, 32.79s/it]
 98%|█████████▊| 4588/4690 [41:52:03<55:37, 32.72s/it]
 98%|█████████▊| 4589/4690 [41:52:37<55:25, 32.92s/it]
 98%|█████████▊| 4590/4690 [41:53:09<54:51, 32.92s/it]
                                                      
{'loss': '0.005821', 'grad_norm': '0.773', 'learning_rate': '1.216e-08', 'epoch': '4.894'}

 98%|█████████▊| 4590/4690 [41:53:10<54:51, 32.92s/it]
 98%|█████████▊| 4591/4690 [41:53:42<54:15, 32.89s/it]
 98%|█████████▊| 4592/4690 [41:54:15<53:33, 32.79s/it]
 98%|█████████▊| 4593/4690 [41:54:48<52:56, 32.75s/it]
 98%|█████████▊| 4594/4690 [41:55:21<52:31, 32.83s/it]
 98%|█████████▊| 4595/4690 [41:55:54<52:03, 32.88s/it]
 98%|█████████▊| 4596/4690 [41:56:26<51:23, 32.80s/it]
 98%|█████████▊| 4597/4690 [41:56:59<51:03, 32.95s/it]
 98%|█████████▊| 4598/4690 [41:57:32<50:22, 32.86s/it]
 98%|█████████▊| 4599/4690 [41:58:05<49:56, 32.93s/it]
 98%|█████████▊| 4600/4690 [41:58:38<49:18, 32.88s/it]
                                                      
{'loss': '0.006997', 'grad_norm': '0.5032', 'learning_rate': '9.871e-09', 'epoch': '4.905'}

 98%|█████████▊| 4600/4690 [41:58:38<49:18, 32.88s/it]
 98%|█████████▊| 4601/4690 [41:59:11<48:40, 32.82s/it]
 98%|█████████▊| 4602/4690 [41:59:43<48:08, 32.82s/it]
 98%|█████████▊| 4603/4690 [42:00:16<47:31, 32.77s/it]
 98%|█████████▊| 4604/4690 [42:00:49<47:02, 32.82s/it]
 98%|█████████▊| 4605/4690 [42:01:22<46:24, 32.76s/it]
 98%|█████████▊| 4606/4690 [42:01:55<45:56, 32.81s/it]
 98%|█████████▊| 4607/4690 [42:02:27<45:15, 32.71s/it]
 98%|█████████▊| 4608/4690 [42:03:00<44:40, 32.69s/it]
 98%|█████████▊| 4609/4690 [42:03:33<44:20, 32.85s/it]
 98%|█████████▊| 4610/4690 [42:04:06<43:45, 32.82s/it]
                                                      
{'loss': '0.005547', 'grad_norm': '0.4265', 'learning_rate': '7.821e-09', 'epoch': '4.915'}

 98%|█████████▊| 4610/4690 [42:04:06<43:45, 32.82s/it]
 98%|█████████▊| 4611/4690 [42:04:38<43:08, 32.76s/it]
 98%|█████████▊| 4612/4690 [42:05:11<42:38, 32.80s/it]
 98%|█████████▊| 4613/4690 [42:05:44<42:14, 32.92s/it]
 98%|█████████▊| 4614/4690 [42:06:17<41:38, 32.87s/it]
 98%|█████████▊| 4615/4690 [42:06:50<41:09, 32.92s/it]
 98%|█████████▊| 4616/4690 [42:07:23<40:34, 32.90s/it]
 98%|█████████▊| 4617/4690 [42:07:56<40:00, 32.88s/it]
 98%|█████████▊| 4618/4690 [42:08:29<39:25, 32.86s/it]
 98%|█████████▊| 4619/4690 [42:09:02<38:54, 32.87s/it]
 99%|█████████▊| 4620/4690 [42:09:35<38:22, 32.89s/it]
                                                      
{'loss': '0.005631', 'grad_norm': '0.6343', 'learning_rate': '6.009e-09', 'epoch': '4.926'}

 99%|█████████▊| 4620/4690 [42:09:35<38:22, 32.89s/it]
 99%|█████████▊| 4621/4690 [42:10:07<37:42, 32.80s/it]
 99%|█████████▊| 4622/4690 [42:10:40<37:16, 32.89s/it]
 99%|█████████▊| 4623/4690 [42:11:13<36:44, 32.90s/it]
 99%|█████████▊| 4624/4690 [42:11:46<36:12, 32.92s/it]
 99%|█████████▊| 4625/4690 [42:12:19<35:40, 32.94s/it]
 99%|█████████▊| 4626/4690 [42:12:52<35:00, 32.82s/it]
 99%|█████████▊| 4627/4690 [42:13:25<34:29, 32.85s/it]
 99%|█████████▊| 4628/4690 [42:13:57<33:51, 32.77s/it]
 99%|█████████▊| 4629/4690 [42:14:30<33:23, 32.84s/it]
 99%|█████████▊| 4630/4690 [42:15:03<32:50, 32.84s/it]
                                                      
{'loss': '0.005499', 'grad_norm': '0.7853', 'learning_rate': '4.436e-09', 'epoch': '4.937'}

 99%|█████████▊| 4630/4690 [42:15:03<32:50, 32.84s/it]
 99%|█████████▊| 4631/4690 [42:15:36<32:15, 32.80s/it]
 99%|█████████▉| 4632/4690 [42:16:08<31:41, 32.79s/it]
 99%|█████████▉| 4633/4690 [42:16:41<31:09, 32.79s/it]
 99%|█████████▉| 4634/4690 [42:17:14<30:31, 32.70s/it]
 99%|█████████▉| 4635/4690 [42:17:46<29:55, 32.64s/it]
 99%|█████████▉| 4636/4690 [42:18:19<29:26, 32.71s/it]
 99%|█████████▉| 4637/4690 [42:18:52<28:59, 32.82s/it]
 99%|█████████▉| 4638/4690 [42:19:25<28:25, 32.79s/it]
 99%|█████████▉| 4639/4690 [42:19:58<27:59, 32.92s/it]
 99%|█████████▉| 4640/4690 [42:20:31<27:17, 32.75s/it]
                                                      
{'loss': '0.00691', 'grad_norm': '0.7542', 'learning_rate': '3.101e-09', 'epoch': '4.947'}

 99%|█████████▉| 4640/4690 [42:20:31<27:17, 32.75s/it]
 99%|█████████▉| 4641/4690 [42:21:03<26:42, 32.70s/it]
 99%|█████████▉| 4642/4690 [42:21:36<26:15, 32.82s/it]
 99%|█████████▉| 4643/4690 [42:22:09<25:44, 32.87s/it]
 99%|█████████▉| 4644/4690 [42:22:42<25:06, 32.75s/it]
 99%|█████████▉| 4645/4690 [42:23:15<24:36, 32.82s/it]
 99%|█████████▉| 4646/4690 [42:23:47<24:03, 32.80s/it]
 99%|█████████▉| 4647/4690 [42:24:20<23:33, 32.88s/it]
 99%|█████████▉| 4648/4690 [42:24:53<23:01, 32.88s/it]
 99%|█████████▉| 4649/4690 [42:25:26<22:30, 32.94s/it]
 99%|█████████▉| 4650/4690 [42:25:59<21:53, 32.84s/it]
                                                      
{'loss': '0.006925', 'grad_norm': '0.405', 'learning_rate': '2.004e-09', 'epoch': '4.958'}

 99%|█████████▉| 4650/4690 [42:25:59<21:53, 32.84s/it]
 99%|█████████▉| 4651/4690 [42:26:32<21:21, 32.87s/it]
 99%|█████████▉| 4652/4690 [42:27:05<20:48, 32.86s/it]
 99%|█████████▉| 4653/4690 [42:27:38<20:21, 33.01s/it]
 99%|█████████▉| 4654/4690 [42:28:11<19:44, 32.90s/it]
 99%|█████████▉| 4655/4690 [42:28:44<19:12, 32.94s/it]
 99%|█████████▉| 4656/4690 [42:29:17<18:41, 32.99s/it]
 99%|█████████▉| 4657/4690 [42:29:50<18:07, 32.97s/it]
 99%|█████████▉| 4658/4690 [42:30:23<17:37, 33.05s/it]
 99%|█████████▉| 4659/4690 [42:30:56<17:01, 32.97s/it]
 99%|█████████▉| 4660/4690 [42:31:29<16:27, 32.93s/it]
                                                      
{'loss': '0.006201', 'grad_norm': '0.653', 'learning_rate': '1.146e-09', 'epoch': '4.969'}

 99%|█████████▉| 4660/4690 [42:31:29<16:27, 32.93s/it]
 99%|█████████▉| 4661/4690 [42:32:01<15:53, 32.89s/it]
 99%|█████████▉| 4662/4690 [42:32:34<15:21, 32.90s/it]
 99%|█████████▉| 4663/4690 [42:33:07<14:44, 32.74s/it]
 99%|█████████▉| 4664/4690 [42:33:40<14:13, 32.81s/it]
 99%|█████████▉| 4665/4690 [42:34:13<13:43, 32.92s/it]
 99%|█████████▉| 4666/4690 [42:34:46<13:10, 32.93s/it]
100%|█████████▉| 4667/4690 [42:35:19<12:37, 32.91s/it]
100%|█████████▉| 4668/4690 [42:35:51<12:01, 32.78s/it]
100%|█████████▉| 4669/4690 [42:36:24<11:28, 32.79s/it]
100%|█████████▉| 4670/4690 [42:36:57<10:57, 32.86s/it]
                                                      
{'loss': '0.006494', 'grad_norm': '0.5574', 'learning_rate': '5.258e-10', 'epoch': '4.979'}

100%|█████████▉| 4670/4690 [42:36:57<10:57, 32.86s/it]
100%|█████████▉| 4671/4690 [42:37:30<10:23, 32.80s/it]
100%|█████████▉| 4672/4690 [42:38:02<09:49, 32.77s/it]
100%|█████████▉| 4673/4690 [42:38:36<09:19, 32.93s/it]
100%|█████████▉| 4674/4690 [42:39:08<08:45, 32.83s/it]
100%|█████████▉| 4675/4690 [42:39:41<08:12, 32.85s/it]
100%|█████████▉| 4676/4690 [42:40:14<07:39, 32.82s/it]
100%|█████████▉| 4677/4690 [42:40:47<07:06, 32.80s/it]
100%|█████████▉| 4678/4690 [42:41:19<06:33, 32.77s/it]
100%|█████████▉| 4679/4690 [42:41:52<06:00, 32.73s/it]
100%|█████████▉| 4680/4690 [42:42:25<05:28, 32.86s/it]
                                                      
{'loss': '0.006495', 'grad_norm': '0.44', 'learning_rate': '1.443e-10', 'epoch': '4.99'}

100%|█████████▉| 4680/4690 [42:42:25<05:28, 32.86s/it]
100%|█████████▉| 4681/4690 [42:42:58<04:55, 32.85s/it]
100%|█████████▉| 4682/4690 [42:43:31<04:23, 32.98s/it]
100%|█████████▉| 4683/4690 [42:44:04<03:50, 32.99s/it]
100%|█████████▉| 4684/4690 [42:44:37<03:17, 32.91s/it]
100%|█████████▉| 4685/4690 [42:45:10<02:44, 32.82s/it]
100%|█████████▉| 4686/4690 [42:45:42<02:11, 32.78s/it]
100%|█████████▉| 4687/4690 [42:46:15<01:38, 32.85s/it]
100%|█████████▉| 4688/4690 [42:46:48<01:05, 32.91s/it]
100%|█████████▉| 4689/4690 [42:47:21<00:32, 32.82s/it]
100%|██████████| 4690/4690 [42:47:38<00:00, 28.10s/it]
                                                      
{'loss': '0.007599', 'grad_norm': '1.364', 'learning_rate': '1.192e-12', 'epoch': '5'}

100%|██████████| 4690/4690 [42:47:38<00:00, 28.10s/it][INFO|trainer.py:3815] 2026-05-15 20:09:04,413 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690
[INFO|configuration_utils.py:533] 2026-05-15 20:09:04,437 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/config.json
[INFO|configuration_utils.py:816] 2026-05-15 20:09:04,439 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/generation_config.json


Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]

Writing model shards: 100%|██████████| 1/1 [00:25<00:00, 25.12s/it]
Writing model shards: 100%|██████████| 1/1 [00:25<00:00, 25.13s/it]
[INFO|modeling_utils.py:3471] 2026-05-15 20:09:29,614 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-15 20:09:29,751 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 20:09:29,761 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/tokenizer_config.json
[INFO|tokenization_utils_base.py:3278] 2026-05-15 20:10:05,722 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 20:10:05,724 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-15 20:10:05,877 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/checkpoint-4690/processor_config.json
[INFO|trainer.py:1823] 2026-05-15 20:10:05,878 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)



                                                      
{'train_runtime': '1.541e+05', 'train_samples_per_second': '1.946', 'train_steps_per_second': '0.03', 'train_loss': '0.1769', 'epoch': '5'}

100%|██████████| 4690/4690 [42:48:51<00:00, 28.10s/it]
100%|██████████| 4690/4690 [42:48:51<00:00, 32.86s/it]
[INFO|tokenization_utils_base.py:3278] 2026-05-15 20:10:05,885 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 20:10:05,887 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/tokenizer_config.json
[INFO|processing_utils.py:899] 2026-05-15 20:10:06,033 >> processor saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/processor_config.json
[INFO|trainer.py:3815] 2026-05-15 20:10:15,603 >> Saving model checkpoint to /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full
[INFO|configuration_utils.py:533] 2026-05-15 20:10:15,622 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/config.json
[INFO|configuration_utils.py:816] 2026-05-15 20:10:15,624 >> Configuration saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/generation_config.json

Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:17<00:00, 17.88s/it]
Writing model shards: 100%|██████████| 1/1 [00:17<00:00, 17.88s/it]
[INFO|modeling_utils.py:3471] 2026-05-15 20:10:33,560 >> Model weights saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/model.safetensors
[INFO|tokenization_utils_base.py:3278] 2026-05-15 20:10:33,564 >> chat template saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/chat_template.jinja
[INFO|tokenization_utils_base.py:2111] 2026-05-15 20:10:33,566 >> tokenizer config file saved in /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/tokenizer_config.json
***** train metrics *****
  epoch                    =                5.0
  total_flos               =       1149465466GF
  train_loss               =             0.1769
  train_runtime            = 1 day, 18:48:51.40
  train_samples_per_second =              1.946
  train_steps_per_second   =               0.03
Figure saved at: /jizhicfs/wongzhenhao/omniscience/ckpt/qwen3-9b/omniscience_vqa_5epoch_full/training_loss.png
[WARNING|2026-05-15 20:10:34] llamafactory.extras.ploting:149 >> No metric eval_loss to plot.
[WARNING|2026-05-15 20:10:34] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
[INFO|modelcard.py:263] 2026-05-15 20:10:34,977 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}