[2025-02-23 00:59:59,602] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,622] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,625] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,625] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,625] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,625] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-02-23 00:59:59,625] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
INFO 02-23 01:00:06 __init__.py:190] Automatically detected platform cuda.
[2025-02-23 01:00:16,656] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:683:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None
[2025-02-23 01:00:16,657] [INFO] [comm.py:652:init_distributed] cdb=None

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 4508 examples [00:00, 8645.06 examples/s]
Generating train split: 4508 examples [00:00, 8507.05 examples/s]

Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:   0%|          | 0/4508 [00:00<?, ? examples/s]
Map:  22%|██▏       | 1000/4508 [00:00<00:00, 9406.61 examples/s]
Map:  22%|██▏       | 1000/4508 [00:00<00:00, 9380.44 examples/s]
Map:  22%|██▏       | 1000/4508 [00:00<00:00, 9364.21 examples/s]
Map:  23%|██▎       | 1041/4508 [00:00<00:00, 10249.89 examples/s]
Map:  25%|██▍       | 1111/4508 [00:00<00:00, 10902.45 examples/s]
Map:  28%|██▊       | 1252/4508 [00:00<00:00, 12320.88 examples/s]
Map:  28%|██▊       | 1248/4508 [00:00<00:00, 12261.72 examples/s]
Map:  46%|████▌     | 2074/4508 [00:00<00:00, 10164.74 examples/s]
Map:  45%|████▌     | 2029/4508 [00:00<00:00, 9892.76 examples/s]
Map:  45%|████▌     | 2032/4508 [00:00<00:00, 9913.24 examples/s]
Map:  64%|██████▍   | 2891/4508 [00:00<00:00, 11656.40 examples/s]
Map:  64%|██████▍   | 2882/4508 [00:00<00:00, 11509.07 examples/s]
Map:  65%|██████▍   | 2925/4508 [00:00<00:00, 14491.18 examples/s]
Map:  65%|██████▍   | 2922/4508 [00:00<00:00, 14306.04 examples/s]
Map:  70%|███████   | 3156/4508 [00:00<00:00, 10459.25 examples/s]
Map:  78%|███████▊  | 3504/4508 [00:00<00:00, 12072.67 examples/s]
Map:  77%|███████▋  | 3467/4508 [00:00<00:00, 11907.10 examples/s]
Map:  92%|█████████▏| 4140/4508 [00:00<00:00, 11979.17 examples/s]
Map:  92%|█████████▏| 4131/4508 [00:00<00:00, 11889.33 examples/s]
Map: 100%|██████████| 4508/4508 [00:00<00:00, 13705.45 examples/s]
Map: 100%|██████████| 4508/4508 [00:00<00:00, 11792.64 examples/s]

Map: 100%|██████████| 4508/4508 [00:00<00:00, 11565.51 examples/s]

Map: 100%|██████████| 4508/4508 [00:00<00:00, 11558.29 examples/s]

Map: 100%|██████████| 4508/4508 [00:00<00:00, 13624.15 examples/s]

Map: 100%|██████████| 4508/4508 [00:00<00:00, 11784.18 examples/s]

Map: 100%|██████████| 4508/4508 [00:00<00:00, 13448.96 examples/s]
Map: 100%|██████████| 4508/4508 [00:00<00:00, 13398.53 examples/s]
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:18,561] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.

Map: 100%|██████████| 4508/4508 [00:00<00:00, 11409.27 examples/s]
Map: 100%|██████████| 4508/4508 [00:00<00:00, 10892.61 examples/s]
[2025-02-23 01:00:18,574] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)`
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3091171 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3091171 [0] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3091171 [0] NCCL INFO cudaDriverVersion 12040
NCCL version 2.21.5+cuda12.4
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3091173 [2] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3091173 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3091172 [1] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3091172 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3091173 [2] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3091172 [1] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3091174 [3] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3091177 [6] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3091174 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3091177 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3091176 [5] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3091175 [4] NCCL INFO cudaDriverVersion 12040
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3091176 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3091175 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3091174 [3] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3091177 [6] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3091175 [4] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3091176 [5] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO P2P plugin IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0>
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO ncclCommInitRank comm 0x5581e0f4d030 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO ncclCommInitRank comm 0x5648d0c85f00 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO ncclCommInitRank comm 0x563494c2ee80 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO ncclCommInitRank comm 0x5599e0457cc0 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO ncclCommInitRank comm 0x55a0cf858190 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO ncclCommInitRank comm 0x55b4b2767d50 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO ncclCommInitRank comm 0x563a13b70840 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 commId 0x9cbaf5ff7ec583da - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO NVLS multicast support is not available on dev 6
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Setting affinity for GPU 3 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO NVLS multicast support is not available on dev 3
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Setting affinity for GPU 2 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO NVLS multicast support is not available on dev 2
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO NVLS multicast support is not available on dev 5
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0.
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO NVLS multicast support is not available on dev 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO NVLS multicast support is not available on dev 1
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO NVLS multicast support is not available on dev 4
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO comm 0x563a13b70840 rank 2 nRanks 7 nNodes 1 localRanks 7 localRank 2 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO comm 0x563494c2ee80 rank 3 nRanks 7 nNodes 1 localRanks 7 localRank 3 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO comm 0x55a0cf858190 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO comm 0x5599e0457cc0 rank 0 nRanks 7 nNodes 1 localRanks 7 localRank 0 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO comm 0x55b4b2767d50 rank 6 nRanks 7 nNodes 1 localRanks 7 localRank 6 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 00/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 01/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 02/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 03/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 04/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 05/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 06/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 07/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 08/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 09/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO comm 0x5648d0c85f00 rank 4 nRanks 7 nNodes 1 localRanks 7 localRank 4 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 10/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 11/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 12/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 13/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 14/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO comm 0x5581e0f4d030 rank 5 nRanks 7 nNodes 1 localRanks 7 localRank 5 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 15/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Trees [0] -1/-1/-1->6->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3093900 [2] NCCL INFO ncclCommInitRank comm 0x563a13b70840 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3093897 [3] NCCL INFO ncclCommInitRank comm 0x563494c2ee80 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3093898 [4] NCCL INFO ncclCommInitRank comm 0x5648d0c85f00 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3093899 [5] NCCL INFO ncclCommInitRank comm 0x5581e0f4d030 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3093896 [1] NCCL INFO ncclCommInitRank comm 0x55a0cf858190 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead.
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3093901 [6] NCCL INFO ncclCommInitRank comm 0x55b4b2767d50 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3093895 [0] NCCL INFO ncclCommInitRank comm 0x5599e0457cc0 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 commId 0x9cbaf5ff7ec583da - Init COMPLETE
[2025-02-23 01:00:20,753] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 730, num_elems = 8.29B

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.78s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.80s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.78s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.79s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.79s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.91s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:06<00:20,  6.89s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.43s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.43s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.43s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.44s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.46s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.45s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:10,  5.43s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.69s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.69s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.69s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.70s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.70s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.72s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:17<00:05,  5.76s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.68s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.48s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.72s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.49s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.71s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.49s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.71s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.49s/it]
[2025-02-23 01:00:38,765] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7

Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.71s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.49s/it]
[2025-02-23 01:00:38,770] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:38,771] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:38,772] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:38,775] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7

Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  3.73s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:17<00:00,  4.49s/it]
[2025-02-23 01:00:38,788] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7

Loading checkpoint shards: 100%|██████████| 4/4 [00:19<00:00,  4.24s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:19<00:00,  4.85s/it]
[2025-02-23 01:00:40,213] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s][2025-02-23 01:00:40,517] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 1460, num_elems = 16.58B

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.32s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.32s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.32s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.31s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.33s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:07,  2.36s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:02<00:06,  2.25s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.29s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.29s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.30s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.30s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.31s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.32s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:04<00:04,  2.25s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:06<00:02,  2.22s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.82s/it]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.

Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.73s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:07<00:00,  1.92s/it]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
[2025-02-23 01:00:48,995] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:48,996] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:48,998] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:48,998] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:49,000] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:49,004] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
[2025-02-23 01:00:49,155] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed info: version=0.16.3, git-hash=unknown, git-branch=unknown
[2025-02-23 01:00:49,155] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7
[2025-02-23 01:00:49,170] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
[2025-02-23 01:00:49,173] [INFO] [logging.py:128:log_dist] [Rank 0] Creating ZeRO Offload
[2025-02-23 01:00:49,364] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [begin]
[2025-02-23 01:00:49,365] [INFO] [utils.py:782:see_memory_usage] MA 4.43 GB         Max_MA 7.33 GB         CA 7.56 GB         Max_CA 8 GB 
[2025-02-23 01:00:49,365] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 522.06 GB, percent = 51.8%
Parameter Offload: Total persistent parameters: 877056 in 401 params
[2025-02-23 01:00:49,580] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [end]
[2025-02-23 01:00:49,580] [INFO] [utils.py:782:see_memory_usage] MA 4.43 GB         Max_MA 4.43 GB         CA 7.56 GB         Max_CA 8 GB 
[2025-02-23 01:00:49,581] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 522.07 GB, percent = 51.8%
[2025-02-23 01:00:49,582] [INFO] [config.py:999:print] DeepSpeedEngine configuration:
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   activation_checkpointing_config  {
    "partition_activations": false, 
    "contiguous_memory_optimization": false, 
    "cpu_checkpointing": false, 
    "number_checkpoints": null, 
    "synchronize_checkpoint_boundary": false, 
    "profile": false
}
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   amp_enabled .................. False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   amp_params ................... False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   autotuning_config ............ {
    "enabled": false, 
    "start_step": null, 
    "end_step": null, 
    "metric_path": null, 
    "arg_mappings": null, 
    "metric": "throughput", 
    "model_info": null, 
    "results_dir": "autotuning_results", 
    "exps_dir": "autotuning_exps", 
    "overwrite": true, 
    "fast": true, 
    "start_profile_step": 3, 
    "end_profile_step": 5, 
    "tuner_type": "gridsearch", 
    "tuner_early_stopping": 5, 
    "tuner_num_trials": 50, 
    "model_info_path": null, 
    "mp_size": 1, 
    "max_train_batch_size": null, 
    "min_train_batch_size": 1, 
    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
    "min_train_micro_batch_size_per_gpu": 1, 
    "num_tuning_micro_batch_sizes": 3
}
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   bfloat16_enabled ............. True
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   bfloat16_immediate_grad_update  False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   checkpoint_parallel_write_pipeline  False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   checkpoint_tag_validation_enabled  True
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   checkpoint_tag_validation_fail  False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x7f62fec229e0>
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   communication_data_type ...... None
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   curriculum_enabled_legacy .... False
[2025-02-23 01:00:49,583] [INFO] [config.py:1003:print]   curriculum_params_legacy ..... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   data_efficiency_enabled ...... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   dataloader_drop_last ......... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   disable_allgather ............ False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   dump_state ................... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   dynamic_loss_scale_args ...... None
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_enabled ........... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_gas_boundary_resolution  1
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_layer_name ........ bert.encoder.layer
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_layer_num ......... 0
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_max_iter .......... 100
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_stability ......... 1e-06
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_tol ............... 0.01
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   eigenvalue_verbose ........... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   elasticity_enabled ........... False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   flops_profiler_config ........ {
    "enabled": false, 
    "recompute_fwd_factor": 0.0, 
    "profile_step": 1, 
    "module_depth": -1, 
    "top_modules": 1, 
    "detailed": true, 
    "output_file": null
}
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   fp16_auto_cast ............... None
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   fp16_enabled ................. False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   fp16_master_weights_and_gradients  False
[2025-02-23 01:00:49,584] [INFO] [config.py:1003:print]   global_rank .................. 0
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   grad_accum_dtype ............. None
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   gradient_accumulation_steps .. 2
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   gradient_clipping ............ 1.0
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   gradient_predivide_factor .... 1.0
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   graph_harvesting ............. False
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   initial_dynamic_scale ........ 1
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   load_universal_checkpoint .... False
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   loss_scale ................... 1.0
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   memory_breakdown ............. False
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   mics_hierarchial_params_gather  False
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   mics_shard_size .............. -1
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   nebula_config ................ {
    "enabled": false, 
    "persistent_storage_path": null, 
    "persistent_time_interval": 100, 
    "num_of_version_in_retention": 2, 
    "enable_nebula_load": true, 
    "load_path": null
}
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   optimizer_legacy_fusion ...... False
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   optimizer_name ............... None
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   optimizer_params ............. None
[2025-02-23 01:00:49,585] [INFO] [config.py:1003:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   pld_enabled .................. False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   pld_params ................... False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   prescale_gradients ........... False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   scheduler_name ............... None
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   scheduler_params ............. None
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   seq_parallel_communication_data_type  torch.float32
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   sparse_attention ............. None
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   sparse_gradients_enabled ..... False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   steps_per_print .............. inf
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   timers_config ................ enabled=True synchronized=True
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   train_batch_size ............. 14
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   train_micro_batch_size_per_gpu  1
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   use_data_before_expert_parallel_  False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   use_node_local_storage ....... False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   wall_clock_breakdown ......... False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   weight_quantization_config ... None
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   world_size ................... 7
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   zero_allow_untested_optimizer  False
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100000000, max_in_cpu=1000000000, pin_memory=True) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=True module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True
[2025-02-23 01:00:49,586] [INFO] [config.py:1003:print]   zero_enabled ................. True
[2025-02-23 01:00:49,587] [INFO] [config.py:1003:print]   zero_force_ds_cpu_optimizer .. True
[2025-02-23 01:00:49,587] [INFO] [config.py:1003:print]   zero_optimization_stage ...... 3
[2025-02-23 01:00:49,587] [INFO] [config.py:989:print_user_config]   json = {
    "fp16": {
        "enabled": false, 
        "loss_scale": 0, 
        "loss_scale_window": 1000, 
        "initial_scale_power": 16, 
        "hysteresis": 2, 
        "min_loss_scale": 1
    }, 
    "bf16": {
        "enabled": true
    }, 
    "zero_optimization": {
        "stage": 3, 
        "offload_optimizer": {
            "device": "none", 
            "pin_memory": true
        }, 
        "offload_param": {
            "device": "none", 
            "pin_memory": true
        }, 
        "overlap_comm": true, 
        "contiguous_gradients": true, 
        "sub_group_size": 1.000000e+09, 
        "reduce_bucket_size": "auto", 
        "stage3_prefetch_bucket_size": "auto", 
        "stage3_param_persistence_threshold": "auto", 
        "stage3_max_live_parameters": 1.000000e+09, 
        "stage3_max_reuse_distance": 1.000000e+09, 
        "stage3_gather_16bit_weights_on_model_save": true
    }, 
    "gradient_accumulation_steps": 2, 
    "gradient_clipping": 1.0, 
    "steps_per_print": inf, 
    "train_batch_size": 14, 
    "train_micro_batch_size_per_gpu": 1, 
    "wall_clock_breakdown": false, 
    "zero_optimization.reduce_bucket_size": 1.284506e+07, 
    "zero_optimization.stage3_param_persistence_threshold": 3.584000e+04, 
    "zero_optimization.stage3_prefetch_bucket_size": 1.156055e+07
}
INFO 02-23 01:01:30 config.py:542] This model supports multiple tasks: {'generate', 'embed', 'reward', 'score', 'classify'}. Defaulting to 'generate'.
WARNING 02-23 01:01:30 arg_utils.py:1079] --enable-prefix-caching is currently not supported for multimodal models in v0 and has been disabled.
INFO 02-23 01:01:30 llm_engine.py:234] Initializing a V0 LLM engine (v0.7.2) with config: model='/home/vlm/workspace/r1_checkpoints/qwen2vl_7b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400', speculative_config=None, tokenizer='/home/vlm/workspace/r1_checkpoints/qwen2vl_7b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto,  device_config=cuda:7, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=/home/vlm/workspace/r1_checkpoints/qwen2vl_7b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":256}, use_cached_outputs=False, 
INFO 02-23 01:01:31 cuda.py:230] Using Flash Attention backend.
INFO 02-23 01:01:32 model_runner.py:1110] Starting to load model /home/vlm/workspace/r1_checkpoints/qwen2vl_7b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400...
INFO 02-23 01:01:32 config.py:2992] cudagraph sizes specified by model runner [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256] is overridden by config [256, 128, 2, 1, 4, 136, 8, 144, 16, 152, 24, 160, 32, 168, 40, 176, 48, 184, 56, 192, 64, 200, 72, 208, 80, 216, 88, 120, 224, 96, 232, 104, 240, 112, 248]

Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  25% Completed | 1/4 [00:01<00:05,  1.69s/it]

Loading safetensors checkpoint shards:  50% Completed | 2/4 [00:02<00:02,  1.08s/it]

Loading safetensors checkpoint shards:  75% Completed | 3/4 [00:06<00:02,  2.66s/it]

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:11<00:00,  3.42s/it]

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:11<00:00,  2.87s/it]

INFO 02-23 01:01:44 model_runner.py:1115] Loading model weights took 0.0000 GB
WARNING 02-23 01:01:46 model_runner.py:1288] Computed max_num_seqs (min(256, 32768 // 49152)) to be less than 1. Setting it to the minimum value of 1.
It looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.
Token indices sequence length is longer than the specified maximum sequence length for this model (49152 > 4096). Running this sequence through the model will result in indexing errors
WARNING 02-23 01:01:51 profiling.py:187] The context length (32768) of the model is too short to hold the multi-modal embeddings in the worst case (49152 tokens in total, out of which {'image': 32768, 'video': 16384} are reserved for multi-modal embeddings). This may cause certain multi-modal inputs to fail during inference, even when the input text is short. To avoid this, you should increase `max_model_len`, reduce `max_num_seqs`, and/or reduce `mm_counts`.
INFO 02-23 01:01:54 worker.py:267] Memory profiling takes 9.82 seconds
INFO 02-23 01:01:54 worker.py:267] the current vLLM instance can use total_gpu_memory (79.32GiB) x gpu_memory_utilization (0.70) = 55.53GiB
INFO 02-23 01:01:54 worker.py:267] model weights take 0.00GiB; non_torch_memory takes 0.00GiB; PyTorch activation peak memory takes 0.00GiB; the rest of the memory reserved for KV Cache is 55.53GiB.
INFO 02-23 01:01:54 executor_base.py:110] # CUDA blocks: 64982, # CPU blocks: 4681
INFO 02-23 01:01:54 executor_base.py:115] Maximum concurrency for 32768 tokens per request: 31.73x
INFO 02-23 01:01:57 model_runner.py:1434] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI. If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.

Capturing CUDA graph shapes:   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graph shapes:   3%|▎         | 1/35 [00:00<00:17,  1.92it/s]
Capturing CUDA graph shapes:   6%|▌         | 2/35 [00:01<00:16,  1.99it/s]
Capturing CUDA graph shapes:   9%|▊         | 3/35 [00:01<00:15,  2.03it/s]
Capturing CUDA graph shapes:  11%|█▏        | 4/35 [00:01<00:15,  2.04it/s]
Capturing CUDA graph shapes:  14%|█▍        | 5/35 [00:02<00:14,  2.05it/s]
Capturing CUDA graph shapes:  17%|█▋        | 6/35 [00:02<00:14,  2.04it/s]
Capturing CUDA graph shapes:  20%|██        | 7/35 [00:03<00:13,  2.03it/s]
Capturing CUDA graph shapes:  23%|██▎       | 8/35 [00:03<00:13,  2.03it/s]
Capturing CUDA graph shapes:  26%|██▌       | 9/35 [00:04<00:12,  2.01it/s]
Capturing CUDA graph shapes:  29%|██▊       | 10/35 [00:04<00:12,  2.02it/s]
Capturing CUDA graph shapes:  31%|███▏      | 11/35 [00:05<00:11,  2.04it/s]
Capturing CUDA graph shapes:  34%|███▍      | 12/35 [00:05<00:11,  2.04it/s]
Capturing CUDA graph shapes:  37%|███▋      | 13/35 [00:06<00:10,  2.05it/s]
Capturing CUDA graph shapes:  40%|████      | 14/35 [00:06<00:10,  2.07it/s]
Capturing CUDA graph shapes:  43%|████▎     | 15/35 [00:07<00:09,  2.07it/s]
Capturing CUDA graph shapes:  46%|████▌     | 16/35 [00:07<00:09,  2.08it/s]
Capturing CUDA graph shapes:  49%|████▊     | 17/35 [00:08<00:08,  2.09it/s]
Capturing CUDA graph shapes:  51%|█████▏    | 18/35 [00:08<00:08,  2.08it/s]
Capturing CUDA graph shapes:  54%|█████▍    | 19/35 [00:09<00:07,  2.04it/s]
Capturing CUDA graph shapes:  57%|█████▋    | 20/35 [00:09<00:07,  2.06it/s]
Capturing CUDA graph shapes:  60%|██████    | 21/35 [00:10<00:06,  2.08it/s]
Capturing CUDA graph shapes:  63%|██████▎   | 22/35 [00:10<00:06,  2.09it/s]
Capturing CUDA graph shapes:  66%|██████▌   | 23/35 [00:11<00:05,  2.10it/s]
Capturing CUDA graph shapes:  69%|██████▊   | 24/35 [00:11<00:05,  2.11it/s]
Capturing CUDA graph shapes:  71%|███████▏  | 25/35 [00:12<00:04,  2.11it/s]
Capturing CUDA graph shapes:  74%|███████▍  | 26/35 [00:12<00:04,  2.11it/s]
Capturing CUDA graph shapes:  77%|███████▋  | 27/35 [00:13<00:03,  2.11it/s]
Capturing CUDA graph shapes:  80%|████████  | 28/35 [00:13<00:03,  2.08it/s]
Capturing CUDA graph shapes:  83%|████████▎ | 29/35 [00:14<00:02,  2.09it/s]
Capturing CUDA graph shapes:  86%|████████▌ | 30/35 [00:14<00:02,  2.07it/s]
Capturing CUDA graph shapes:  89%|████████▊ | 31/35 [00:15<00:01,  2.07it/s]
Capturing CUDA graph shapes:  91%|█████████▏| 32/35 [00:15<00:01,  2.08it/s]
Capturing CUDA graph shapes:  94%|█████████▍| 33/35 [00:15<00:00,  2.06it/s]
Capturing CUDA graph shapes:  97%|█████████▋| 34/35 [00:16<00:00,  2.09it/s]
Capturing CUDA graph shapes: 100%|██████████| 35/35 [00:16<00:00,  2.10it/s]
Capturing CUDA graph shapes: 100%|██████████| 35/35 [00:16<00:00,  2.07it/s]
INFO 02-23 01:02:14 model_runner.py:1562] Graph capturing finished in 17 secs, took 0.00 GiB
INFO 02-23 01:02:14 llm_engine.py:431] init engine (profile, create kv cache, warmup model) took 29.59 seconds
Parameter Offload: Total persistent parameters: 877056 in 401 params
wandb: Currently logged in as: tanhuajie264 (tanhuajie264-peking-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.
wandb: - Waiting for wandb.init()...
wandb: \ Waiting for wandb.init()...
wandb: Tracking run with wandb version 0.19.5
wandb: Run data is saved locally in /home/vlm/workspace/vision-open-r1-spatial/wandb/run-20250223_010222-bg4bagha
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run R1-Resume-COT-VLLM-Correct-Qwen2-VL-7B-GRPO-GEOQA-4k5-2025-02-23-00-58-58
wandb: ⭐️ View project at https://wandb.ai/tanhuajie264-peking-university/vison-open-r1
wandb: 🚀 View run at https://wandb.ai/tanhuajie264-peking-university/vison-open-r1/runs/bg4bagha

  0%|          | 0/1610 [00:00<?, ?it/s]p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Using non-device net plugin version 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Using network IBext_v8
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO bootstrapSplit: comm 0x7fb65006fd20 parent 0x563494c2ee80 rank 3 nranks 7 color -1326228412 key 3 prev 2 next 4 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO bootstrapSplit: comm 0x7feca8070b10 parent 0x5581e0f4d030 rank 5 nranks 7 color -1326228412 key 5 prev 4 next 6 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO bootstrapSplit: comm 0x7f70a40720d0 parent 0x55a0cf858190 rank 1 nranks 7 color -1326228412 key 1 prev 0 next 2 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO bootstrapSplit: comm 0x7f410c070c10 parent 0x5599e0457cc0 rank 0 nranks 7 color -1326228412 key 0 prev 6 next 1 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO ncclCommSplit comm 0x7fb65006fd20 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 parent 0x563494c2ee80 color -1326228412 key 3 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO ncclCommSplit comm 0x7feca8070b10 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 parent 0x5581e0f4d030 color -1326228412 key 5 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO ncclCommSplit comm 0x7f70a40720d0 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 parent 0x55a0cf858190 color -1326228412 key 1 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO bootstrapSplit: comm 0x7f7184070e50 parent 0x5648d0c85f00 rank 4 nranks 7 color -1326228412 key 4 prev 3 next 5 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO ncclCommSplit comm 0x7f410c070c10 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 parent 0x5599e0457cc0 color -1326228412 key 0 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO bootstrapSplit: comm 0x7f4b34070e40 parent 0x55b4b2767d50 rank 6 nranks 7 color -1326228412 key 6 prev 5 next 0 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO ncclCommSplit comm 0x7f4b34070e40 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 parent 0x55b4b2767d50 color -1326228412 key 6 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO ncclCommSplit comm 0x7f7184070e50 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 parent 0x5648d0c85f00 color -1326228412 key 4 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO bootstrapSplit: comm 0x7f0594070af0 parent 0x563a13b70840 rank 2 nranks 7 color -1326228412 key 2 prev 1 next 3 - DONE
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO ncclCommSplit comm 0x7f0594070af0 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 parent 0x563a13b70840 color -1326228412 key 2 commId 0x8c0cc1f60de0c6ad - Init START
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Setting affinity for GPU 3 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO NVLS multicast support is not available on dev 3
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO NVLS multicast support is not available on dev 6
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO NVLS multicast support is not available on dev 5
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO NVLS multicast support is not available on dev 1
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,00000000,ffffffff,00000000
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO NVLS multicast support is not available on dev 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO NVLS multicast support is not available on dev 4
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Setting affinity for GPU 2 to ffffffff,00000000,ffffffff
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO NVLS multicast support is not available on dev 2
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO comm 0x7f0594070af0 rank 2 nRanks 7 nNodes 1 localRanks 7 localRank 2 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO comm 0x7f70a40720d0 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO comm 0x7f410c070c10 rank 0 nRanks 7 nNodes 1 localRanks 7 localRank 0 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO comm 0x7f4b34070e40 rank 6 nRanks 7 nNodes 1 localRanks 7 localRank 6 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO comm 0x7f7184070e50 rank 4 nRanks 7 nNodes 1 localRanks 7 localRank 4 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO comm 0x7feca8070b10 rank 5 nRanks 7 nNodes 1 localRanks 7 localRank 5 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 00/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 01/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 02/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Trees [0] -1/-1/-1->6->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 03/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 04/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 05/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 06/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 07/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 08/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 09/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO comm 0x7fb65006fd20 rank 3 nRanks 7 nNodes 1 localRanks 7 localRank 3 MNNVL 0
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 10/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 11/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 12/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 13/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 14/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 15/16 :    0   1   2   3   4   5   6
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO P2P Chunksize set to 524288
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Connected all rings
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO Connected all trees
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
p-phy-ctyun-gz-a800-node-prod-200-104:3091174:3102466 [3] NCCL INFO ncclCommSplit comm 0x7fb65006fd20 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 parent 0x563494c2ee80 color -1326228412 key 3 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091176:3102470 [5] NCCL INFO ncclCommSplit comm 0x7feca8070b10 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 parent 0x5581e0f4d030 color -1326228412 key 5 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091172:3102468 [1] NCCL INFO ncclCommSplit comm 0x7f70a40720d0 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 parent 0x55a0cf858190 color -1326228412 key 1 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091175:3102469 [4] NCCL INFO ncclCommSplit comm 0x7f7184070e50 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 parent 0x5648d0c85f00 color -1326228412 key 4 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091177:3102467 [6] NCCL INFO ncclCommSplit comm 0x7f4b34070e40 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 parent 0x55b4b2767d50 color -1326228412 key 6 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091173:3102472 [2] NCCL INFO ncclCommSplit comm 0x7f0594070af0 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 parent 0x563a13b70840 color -1326228412 key 2 commId 0x8c0cc1f60de0c6ad - Init COMPLETE
p-phy-ctyun-gz-a800-node-prod-200-104:3091171:3102471 [0] NCCL INFO ncclCommSplit comm 0x7f410c070c10 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 parent 0x5599e0457cc0 color -1326228412 key 0 commId 0x8c0cc1f60de0c6ad - Init COMPLETE

  0%|          | 1/1610 [00:25<11:36:30, 25.97s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.9913871805096857, 'learning_rate': 9.993788819875776e-07, 'completion_length': 192.6428680419922, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4285715222358704, 'reward_std': 0.32489626109600067, 'kl': 0.0, 'epoch': 0.0}

  0%|          | 1/1610 [00:25<11:36:30, 25.97s/it]
  0%|          | 2/1610 [00:43<9:25:33, 21.10s/it] 
                                                  
{'loss': 0.0, 'grad_norm': 0.9287893629019393, 'learning_rate': 9.987577639751552e-07, 'completion_length': 162.6964340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.26657507568597794, 'kl': 3.546476364135742e-06, 'epoch': 0.01}

  0%|          | 2/1610 [00:43<9:25:33, 21.10s/it]
  0%|          | 3/1610 [01:01<8:43:08, 19.53s/it]
                                                  
{'loss': -0.0, 'grad_norm': 1.1354526450625364, 'learning_rate': 9.981366459627329e-07, 'completion_length': 170.50000762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.3193712383508682, 'kl': -2.7418136596679688e-06, 'epoch': 0.01}

  0%|          | 3/1610 [01:01<8:43:08, 19.53s/it]
  0%|          | 4/1610 [01:14<7:40:44, 17.21s/it]
                                                  
{'loss': 0.0, 'grad_norm': 1.6558136214770627, 'learning_rate': 9.975155279503105e-07, 'completion_length': 132.6785774230957, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3324785977602005, 'kl': 2.421438694000244e-05, 'epoch': 0.01}

  0%|          | 4/1610 [01:14<7:40:44, 17.21s/it]
  0%|          | 5/1610 [01:31<7:30:46, 16.85s/it]
                                                  
{'loss': 0.0, 'grad_norm': 0.6025594465970825, 'learning_rate': 9.968944099378881e-07, 'completion_length': 166.3571548461914, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4464285969734192, 'reward_std': 0.23086076974868774, 'kl': 4.690885543823242e-05, 'epoch': 0.02}

  0%|          | 5/1610 [01:31<7:30:46, 16.85s/it]
  0%|          | 6/1610 [01:51<7:58:07, 17.89s/it]
                                                  
{'loss': 0.0, 'grad_norm': 0.5959916363028317, 'learning_rate': 9.962732919254658e-07, 'completion_length': 195.0178680419922, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4464285969734192, 'reward_std': 0.14838216453790665, 'kl': 0.00012087821960449219, 'epoch': 0.02}

  0%|          | 6/1610 [01:51<7:58:07, 17.89s/it]
  0%|          | 7/1610 [02:12<8:27:22, 18.99s/it]
                                                  
{'loss': 0.0, 'grad_norm': 2.504273613987083, 'learning_rate': 9.956521739130434e-07, 'completion_length': 213.6071548461914, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.4642857909202576, 'reward_std': 0.3963248133659363, 'kl': 2.3663043975830078e-05, 'epoch': 0.02}

  0%|          | 7/1610 [02:12<8:27:22, 18.99s/it]
  0%|          | 8/1610 [02:30<8:17:24, 18.63s/it]
                                                  
{'loss': 0.0, 'grad_norm': 0.9736440792618575, 'learning_rate': 9.95031055900621e-07, 'completion_length': 161.6607208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5892857909202576, 'reward_std': 0.30228933691978455, 'kl': 0.00012886524200439453, 'epoch': 0.02}

  0%|          | 8/1610 [02:30<8:17:24, 18.63s/it]
  1%|          | 9/1610 [02:43<7:33:25, 16.99s/it]
                                                  
{'loss': 0.0, 'grad_norm': 0.8374004200174586, 'learning_rate': 9.944099378881986e-07, 'completion_length': 128.17857360839844, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.15943220257759094, 'kl': 0.00017002224922180176, 'epoch': 0.03}

  1%|          | 9/1610 [02:43<7:33:25, 16.99s/it]
  1%|          | 10/1610 [03:03<7:58:46, 17.95s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.6512111831842913, 'learning_rate': 9.937888198757763e-07, 'completion_length': 180.1607208251953, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178571939468384, 'reward_std': 0.36266788840293884, 'kl': 7.082149386405945e-05, 'epoch': 0.03}

  1%|          | 10/1610 [03:03<7:58:46, 17.95s/it]
  1%|          | 11/1610 [03:21<7:58:32, 17.96s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.6256748188620059, 'learning_rate': 9.93167701863354e-07, 'completion_length': 175.75000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178571939468384, 'reward_std': 0.20670334994792938, 'kl': 0.00013685226440429688, 'epoch': 0.03}

  1%|          | 11/1610 [03:21<7:58:32, 17.96s/it]
  1%|          | 12/1610 [03:40<8:08:49, 18.35s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.8142131489106341, 'learning_rate': 9.925465838509315e-07, 'completion_length': 159.64286041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.25552502274513245, 'kl': 0.0001468658447265625, 'epoch': 0.04}

  1%|          | 12/1610 [03:40<8:08:49, 18.35s/it]
  1%|          | 13/1610 [04:00<8:16:07, 18.64s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.0587120869054272, 'learning_rate': 9.919254658385092e-07, 'completion_length': 175.46429443359375, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.410714328289032, 'reward_std': 0.3435286581516266, 'kl': 0.00019216537475585938, 'epoch': 0.04}

  1%|          | 13/1610 [04:00<8:16:07, 18.64s/it]
  1%|          | 14/1610 [04:18<8:11:32, 18.48s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.9976788497248201, 'learning_rate': 9.91304347826087e-07, 'completion_length': 175.0357208251953, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3750000596046448, 'reward_std': 0.21981074661016464, 'kl': 0.00020873546600341797, 'epoch': 0.04}

  1%|          | 14/1610 [04:18<8:11:32, 18.48s/it]
  1%|          | 15/1610 [04:36<8:12:26, 18.52s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.1143749284345166, 'learning_rate': 9.906832298136647e-07, 'completion_length': 171.55358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5535715222358704, 'reward_std': 0.2610500492155552, 'kl': 0.00022268295288085938, 'epoch': 0.05}

  1%|          | 15/1610 [04:36<8:12:26, 18.52s/it]
  1%|          | 16/1610 [04:53<7:56:51, 17.95s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.3548374609979388, 'learning_rate': 9.900621118012423e-07, 'completion_length': 139.39286041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.365151971578598, 'kl': 0.00038814544677734375, 'epoch': 0.05}

  1%|          | 16/1610 [04:53<7:56:51, 17.95s/it]
  1%|          | 17/1610 [05:11<7:53:43, 17.84s/it]
                                                   
{'loss': 0.0, 'grad_norm': 2.605362586783903, 'learning_rate': 9.8944099378882e-07, 'completion_length': 173.5357208251953, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3750000596046448, 'reward_std': 0.26353414356708527, 'kl': 0.00031948089599609375, 'epoch': 0.05}

  1%|          | 17/1610 [05:11<7:53:43, 17.84s/it]
  1%|          | 18/1610 [05:29<8:00:24, 18.11s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.9931606839141948, 'learning_rate': 9.888198757763976e-07, 'completion_length': 161.51786041259766, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5357143878936768, 'reward_std': 0.3465195968747139, 'kl': 0.00042057037353515625, 'epoch': 0.06}

  1%|          | 18/1610 [05:29<8:00:24, 18.11s/it]
  1%|          | 19/1610 [05:51<8:27:22, 19.13s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.71945874978547, 'learning_rate': 9.881987577639752e-07, 'completion_length': 182.33929443359375, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4285715222358704, 'reward_std': 0.4315322935581207, 'kl': 0.00028896331787109375, 'epoch': 0.06}

  1%|          | 19/1610 [05:51<8:27:22, 19.13s/it]
  1%|          | 20/1610 [06:12<8:40:48, 19.65s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.9448249689240844, 'learning_rate': 9.875776397515528e-07, 'completion_length': 219.3571548461914, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.910714328289032, 'reward': 1.321428656578064, 'reward_std': 0.38775889575481415, 'kl': 0.0005464553833007812, 'epoch': 0.06}

  1%|          | 20/1610 [06:12<8:40:48, 19.65s/it]
  1%|▏         | 21/1610 [06:29<8:24:26, 19.05s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.5001180524562032, 'learning_rate': 9.869565217391304e-07, 'completion_length': 180.3928680419922, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.410714328289032, 'reward_std': 0.30228933691978455, 'kl': 0.0006694793701171875, 'epoch': 0.07}

  1%|▏         | 21/1610 [06:29<8:24:26, 19.05s/it]
  1%|▏         | 22/1610 [06:48<8:21:45, 18.96s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.0270815993251574, 'learning_rate': 9.86335403726708e-07, 'completion_length': 195.12501525878906, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.0007076263427734375, 'epoch': 0.07}

  1%|▏         | 22/1610 [06:48<8:21:45, 18.96s/it]
  1%|▏         | 23/1610 [07:04<7:56:10, 18.00s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.8875090098297056, 'learning_rate': 9.857142857142857e-07, 'completion_length': 155.3214340209961, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.11266788095235825, 'kl': 0.0004911422729492188, 'epoch': 0.07}

  1%|▏         | 23/1610 [07:04<7:56:10, 18.00s/it]
  1%|▏         | 24/1610 [07:24<8:12:36, 18.64s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.1203451135308142, 'learning_rate': 9.850931677018633e-07, 'completion_length': 224.83929443359375, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.3681929036974907, 'kl': 0.001049041748046875, 'epoch': 0.07}

  1%|▏         | 24/1610 [07:24<8:12:36, 18.64s/it]
  2%|▏         | 25/1610 [07:41<7:58:01, 18.10s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.9779284686596399, 'learning_rate': 9.84472049689441e-07, 'completion_length': 163.92857360839844, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7142858505249023, 'reward_std': 0.35962697863578796, 'kl': 0.0009632110595703125, 'epoch': 0.08}

  2%|▏         | 25/1610 [07:41<7:58:01, 18.10s/it]
  2%|▏         | 26/1610 [07:57<7:43:52, 17.57s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.3095165488553673, 'learning_rate': 9.838509316770186e-07, 'completion_length': 140.58929443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.30228933691978455, 'kl': 0.0005893707275390625, 'epoch': 0.08}

  2%|▏         | 26/1610 [07:57<7:43:52, 17.57s/it]
  2%|▏         | 27/1610 [08:17<8:03:07, 18.31s/it]
                                                   
{'loss': 0.0, 'grad_norm': 0.9364589413785328, 'learning_rate': 9.832298136645962e-07, 'completion_length': 169.71429443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.20670335739850998, 'kl': 0.0006847381591796875, 'epoch': 0.08}

  2%|▏         | 27/1610 [08:17<8:03:07, 18.31s/it]
  2%|▏         | 28/1610 [08:37<8:14:56, 18.77s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.1746765227409603, 'learning_rate': 9.826086956521739e-07, 'completion_length': 160.8214340209961, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.446428656578064, 'reward_std': 0.31333939731121063, 'kl': 0.000751495361328125, 'epoch': 0.09}

  2%|▏         | 28/1610 [08:37<8:14:56, 18.77s/it]
  2%|▏         | 29/1610 [08:55<8:09:38, 18.58s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.0078263413716302, 'learning_rate': 9.819875776397515e-07, 'completion_length': 164.92858123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.535714328289032, 'reward_std': 0.24241764098405838, 'kl': 0.0010986328125, 'epoch': 0.09}

  2%|▏         | 29/1610 [08:55<8:09:38, 18.58s/it]
  2%|▏         | 30/1610 [09:16<8:28:17, 19.30s/it]
                                                   
{'loss': 0.0, 'grad_norm': 6.119601915632987, 'learning_rate': 9.813664596273291e-07, 'completion_length': 203.1071548461914, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.7142857909202576, 'reward_std': 0.3208717554807663, 'kl': 0.0006313323974609375, 'epoch': 0.09}

  2%|▏         | 30/1610 [09:16<8:28:17, 19.30s/it]
  2%|▏         | 31/1610 [09:30<7:44:57, 17.67s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.3224151183173594, 'learning_rate': 9.807453416149068e-07, 'completion_length': 136.8928680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.23638580739498138, 'kl': 0.0010967254638671875, 'epoch': 0.1}

  2%|▏         | 31/1610 [09:30<7:44:57, 17.67s/it]
  2%|▏         | 32/1610 [09:50<8:02:50, 18.36s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.1561450713127879, 'learning_rate': 9.801242236024844e-07, 'completion_length': 170.71428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.571428656578064, 'reward_std': 0.29068250954151154, 'kl': 0.000789642333984375, 'epoch': 0.1}

  2%|▏         | 32/1610 [09:50<8:02:50, 18.36s/it]
  2%|▏         | 33/1610 [10:10<8:17:42, 18.94s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.8925039751809666, 'learning_rate': 9.79503105590062e-07, 'completion_length': 190.50000762939453, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.3750000596046448, 'reward_std': 0.2610500529408455, 'kl': 0.00150299072265625, 'epoch': 0.1}

  2%|▏         | 33/1610 [10:10<8:17:42, 18.94s/it]
  2%|▏         | 34/1610 [10:26<7:54:12, 18.05s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.4961268374882197, 'learning_rate': 9.788819875776397e-07, 'completion_length': 132.21429061889648, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.349560484290123, 'kl': 0.001430511474609375, 'epoch': 0.11}

  2%|▏         | 34/1610 [10:26<7:54:12, 18.05s/it]
  2%|▏         | 35/1610 [10:44<7:48:41, 17.85s/it]
                                                   
{'loss': 0.0, 'grad_norm': 1.2283692867629143, 'learning_rate': 9.782608695652173e-07, 'completion_length': 180.2678680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5535715222358704, 'reward_std': 0.33496273308992386, 'kl': 0.0011959075927734375, 'epoch': 0.11}

  2%|▏         | 35/1610 [10:44<7:48:41, 17.85s/it]
  2%|▏         | 36/1610 [11:02<7:48:50, 17.87s/it]
                                                   
{'loss': 0.0, 'grad_norm': 2.8393413244498333, 'learning_rate': 9.77639751552795e-07, 'completion_length': 194.4464340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.4647124856710434, 'kl': 0.0011119842529296875, 'epoch': 0.11}

  2%|▏         | 36/1610 [11:02<7:48:50, 17.87s/it]
  2%|▏         | 37/1610 [11:22<8:05:36, 18.52s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.192826032533785, 'learning_rate': 9.770186335403726e-07, 'completion_length': 162.71428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6785715222358704, 'reward_std': 0.39838217198848724, 'kl': 0.0012664794921875, 'epoch': 0.11}

  2%|▏         | 37/1610 [11:22<8:05:36, 18.52s/it]
  2%|▏         | 38/1610 [11:42<8:19:31, 19.07s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.1356645328985158, 'learning_rate': 9.763975155279502e-07, 'completion_length': 180.2321548461914, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.4821429252624512, 'reward_std': 0.2760745882987976, 'kl': 0.00157928466796875, 'epoch': 0.12}

  2%|▏         | 38/1610 [11:42<8:19:31, 19.07s/it]
  2%|▏         | 39/1610 [12:02<8:24:46, 19.28s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.0564308476868531, 'learning_rate': 9.757763975155278e-07, 'completion_length': 168.96429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5357143878936768, 'reward_std': 0.26657506078481674, 'kl': 0.0014171600341796875, 'epoch': 0.12}

  2%|▏         | 39/1610 [12:02<8:24:46, 19.28s/it]
  2%|▏         | 40/1610 [12:16<7:40:44, 17.61s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.3504330502168447, 'learning_rate': 9.751552795031055e-07, 'completion_length': 128.6428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981074661016464, 'kl': 0.00135040283203125, 'epoch': 0.12}

  2%|▏         | 40/1610 [12:16<7:40:44, 17.61s/it]
  3%|▎         | 41/1610 [12:31<7:21:40, 16.89s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.485710222397684, 'learning_rate': 9.745341614906833e-07, 'completion_length': 156.21428680419922, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.1428571529686451, 'kl': 0.0017242431640625, 'epoch': 0.13}

  3%|▎         | 41/1610 [12:31<7:21:40, 16.89s/it]
  3%|▎         | 42/1610 [12:47<7:14:14, 16.62s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 3.0347439273785697, 'learning_rate': 9.73913043478261e-07, 'completion_length': 167.2678680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.23086079210042953, 'kl': 0.001674652099609375, 'epoch': 0.13}

  3%|▎         | 42/1610 [12:47<7:14:14, 16.62s/it]
  3%|▎         | 43/1610 [12:59<6:43:03, 15.43s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.290743500037464, 'learning_rate': 9.732919254658386e-07, 'completion_length': 136.53571701049805, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.25552501529455185, 'kl': 0.001689910888671875, 'epoch': 0.13}

  3%|▎         | 43/1610 [12:59<6:43:03, 15.43s/it]
  3%|▎         | 44/1610 [13:17<7:00:17, 16.10s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.764922217635598, 'learning_rate': 9.726708074534162e-07, 'completion_length': 179.30358123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.21981073170900345, 'kl': 0.001674652099609375, 'epoch': 0.14}

  3%|▎         | 44/1610 [13:17<7:00:17, 16.10s/it]
  3%|▎         | 45/1610 [13:37<7:27:54, 17.17s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.407236541689567, 'learning_rate': 9.720496894409938e-07, 'completion_length': 191.98214721679688, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.29123931378126144, 'kl': 0.001857757568359375, 'epoch': 0.14}

  3%|▎         | 45/1610 [13:37<7:27:54, 17.17s/it]
  3%|▎         | 46/1610 [13:57<7:54:27, 18.20s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.9880947404375586, 'learning_rate': 9.714285714285715e-07, 'completion_length': 169.0357208251953, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4821429252624512, 'reward_std': 0.21981073915958405, 'kl': 0.001537322998046875, 'epoch': 0.14}

  3%|▎         | 46/1610 [13:57<7:54:27, 18.20s/it]
  3%|▎         | 47/1610 [14:15<7:51:27, 18.10s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.0132210027994701, 'learning_rate': 9.708074534161491e-07, 'completion_length': 161.1607208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.2253357619047165, 'kl': 0.002002716064453125, 'epoch': 0.15}

  3%|▎         | 47/1610 [14:15<7:51:27, 18.10s/it]
  3%|▎         | 48/1610 [14:30<7:27:28, 17.19s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.7626021901133967, 'learning_rate': 9.701863354037265e-07, 'completion_length': 159.35714721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357544541359, 'kl': 0.00156402587890625, 'epoch': 0.15}

  3%|▎         | 48/1610 [14:30<7:27:28, 17.19s/it]
  3%|▎         | 49/1610 [14:49<7:36:52, 17.56s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.8534394615786124, 'learning_rate': 9.695652173913042e-07, 'completion_length': 157.33929443359375, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.446428656578064, 'reward_std': 0.2826733738183975, 'kl': 0.00223541259765625, 'epoch': 0.15}

  3%|▎         | 49/1610 [14:49<7:36:52, 17.56s/it]
  3%|▎         | 50/1610 [15:06<7:36:09, 17.54s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.491981388561349, 'learning_rate': 9.68944099378882e-07, 'completion_length': 161.51786041259766, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.2610500529408455, 'kl': 0.00196075439453125, 'epoch': 0.16}

  3%|▎         | 50/1610 [15:06<7:36:09, 17.54s/it]
  3%|▎         | 51/1610 [15:23<7:31:58, 17.39s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.6988588635267982, 'learning_rate': 9.683229813664596e-07, 'completion_length': 165.98214721679688, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5178571939468384, 'reward_std': 0.26353414356708527, 'kl': 0.001674652099609375, 'epoch': 0.16}

  3%|▎         | 51/1610 [15:23<7:31:58, 17.39s/it]
  3%|▎         | 52/1610 [15:40<7:25:57, 17.17s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.44438141020105715, 'learning_rate': 9.677018633540373e-07, 'completion_length': 164.33929443359375, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1181928962469101, 'kl': 0.001739501953125, 'epoch': 0.16}

  3%|▎         | 52/1610 [15:40<7:25:57, 17.17s/it]
  3%|▎         | 53/1610 [15:57<7:25:06, 17.15s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.439694351641339, 'learning_rate': 9.67080745341615e-07, 'completion_length': 174.7857208251953, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.22229483723640442, 'kl': 0.00193023681640625, 'epoch': 0.16}

  3%|▎         | 53/1610 [15:57<7:25:06, 17.15s/it]
  3%|▎         | 54/1610 [16:15<7:28:34, 17.30s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.7807699323299682, 'learning_rate': 9.664596273291925e-07, 'completion_length': 167.30358123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.25552503019571304, 'kl': 0.00235748291015625, 'epoch': 0.17}

  3%|▎         | 54/1610 [16:15<7:28:34, 17.30s/it]
  3%|▎         | 55/1610 [16:31<7:19:28, 16.96s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.102946732997162, 'learning_rate': 9.658385093167702e-07, 'completion_length': 157.50000762939453, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2967643216252327, 'kl': 0.001590728759765625, 'epoch': 0.17}

  3%|▎         | 55/1610 [16:31<7:19:28, 16.96s/it]
  3%|▎         | 56/1610 [16:52<7:51:29, 18.20s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.2418462169568973, 'learning_rate': 9.652173913043478e-07, 'completion_length': 178.2857208251953, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.410714328289032, 'reward_std': 0.36266788840293884, 'kl': 0.001689910888671875, 'epoch': 0.17}

  3%|▎         | 56/1610 [16:52<7:51:29, 18.20s/it]
  4%|▎         | 57/1610 [17:06<7:19:32, 16.98s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.8655260325954797, 'learning_rate': 9.645962732919254e-07, 'completion_length': 122.53572082519531, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.0015869140625, 'epoch': 0.18}

  4%|▎         | 57/1610 [17:06<7:19:32, 16.98s/it]
  4%|▎         | 58/1610 [17:27<7:53:03, 18.29s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 3.59942522672637, 'learning_rate': 9.63975155279503e-07, 'completion_length': 158.1607208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5357143878936768, 'reward_std': 0.313846230506897, 'kl': 0.00218963623046875, 'epoch': 0.18}

  4%|▎         | 58/1610 [17:27<7:53:03, 18.29s/it]
  4%|▎         | 59/1610 [17:40<7:06:37, 16.50s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 9.39154763660421, 'learning_rate': 9.633540372670807e-07, 'completion_length': 119.85715103149414, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2721000984311104, 'kl': 0.002437591552734375, 'epoch': 0.18}

  4%|▎         | 59/1610 [17:40<7:06:37, 16.50s/it]
  4%|▎         | 60/1610 [17:59<7:24:44, 17.22s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.407829767227674, 'learning_rate': 9.627329192546583e-07, 'completion_length': 142.10714721679688, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5357143878936768, 'reward_std': 0.32695358991622925, 'kl': 0.00151824951171875, 'epoch': 0.19}

  4%|▎         | 60/1610 [17:59<7:24:44, 17.22s/it]
  4%|▍         | 61/1610 [18:13<6:59:49, 16.26s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.792485310993058, 'learning_rate': 9.62111801242236e-07, 'completion_length': 128.0714340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.19514649361371994, 'kl': 0.001811981201171875, 'epoch': 0.19}

  4%|▍         | 61/1610 [18:13<6:59:49, 16.26s/it]
  4%|▍         | 62/1610 [18:29<6:58:57, 16.24s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.129637004694952, 'learning_rate': 9.614906832298136e-07, 'completion_length': 147.8214340209961, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.0019378662109375, 'epoch': 0.19}

  4%|▍         | 62/1610 [18:29<6:58:57, 16.24s/it]
  4%|▍         | 63/1610 [18:44<6:47:06, 15.79s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.78472860458145, 'learning_rate': 9.608695652173912e-07, 'completion_length': 144.5357208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.00225067138671875, 'epoch': 0.2}

  4%|▍         | 63/1610 [18:44<6:47:06, 15.79s/it]
  4%|▍         | 64/1610 [19:00<6:51:47, 15.98s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.9836067195276224, 'learning_rate': 9.602484472049689e-07, 'completion_length': 162.1428680419922, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.26657506078481674, 'kl': 0.002719879150390625, 'epoch': 0.2}

  4%|▍         | 64/1610 [19:00<6:51:47, 15.98s/it]
  4%|▍         | 65/1610 [19:13<6:32:24, 15.24s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.9466968994736978, 'learning_rate': 9.596273291925465e-07, 'completion_length': 132.85715103149414, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25552502274513245, 'kl': 0.0028839111328125, 'epoch': 0.2}

  4%|▍         | 65/1610 [19:13<6:32:24, 15.24s/it]
  4%|▍         | 66/1610 [19:34<7:15:30, 16.92s/it]
                                                   
{'loss': 0.0002, 'grad_norm': 2.7184819872515216, 'learning_rate': 9.590062111801241e-07, 'completion_length': 200.50000762939453, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 0.8750000298023224, 'reward': 1.3928572535514832, 'reward_std': 0.3379536382853985, 'kl': 0.0050048828125, 'epoch': 0.2}

  4%|▍         | 66/1610 [19:34<7:15:30, 16.92s/it]
  4%|▍         | 67/1610 [19:55<7:42:49, 18.00s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 8.954990607694489, 'learning_rate': 9.583850931677018e-07, 'completion_length': 174.00000762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.2967643290758133, 'kl': 0.002101898193359375, 'epoch': 0.21}

  4%|▍         | 67/1610 [19:55<7:42:49, 18.00s/it]
  4%|▍         | 68/1610 [20:14<7:48:08, 18.22s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.8867680624615146, 'learning_rate': 9.577639751552796e-07, 'completion_length': 160.7678680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.3294376879930496, 'kl': 0.0030059814453125, 'epoch': 0.21}

  4%|▍         | 68/1610 [20:14<7:48:08, 18.22s/it]
  4%|▍         | 69/1610 [20:33<7:54:09, 18.46s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.0326135796296527, 'learning_rate': 9.571428571428572e-07, 'completion_length': 171.08929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.2937234118580818, 'kl': 0.00238800048828125, 'epoch': 0.21}

  4%|▍         | 69/1610 [20:33<7:54:09, 18.46s/it]
  4%|▍         | 70/1610 [20:50<7:48:18, 18.25s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.3555649804981207, 'learning_rate': 9.565217391304349e-07, 'completion_length': 162.0178680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5535715222358704, 'reward_std': 0.2610500454902649, 'kl': 0.0017852783203125, 'epoch': 0.22}

  4%|▍         | 70/1610 [20:50<7:48:18, 18.25s/it]
  4%|▍         | 71/1610 [21:09<7:50:38, 18.35s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.014987618486424, 'learning_rate': 9.559006211180125e-07, 'completion_length': 177.73214721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5714285969734192, 'reward_std': 0.25552502647042274, 'kl': 0.00223541259765625, 'epoch': 0.22}

  4%|▍         | 71/1610 [21:09<7:50:38, 18.35s/it]
  4%|▍         | 72/1610 [21:25<7:31:51, 17.63s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.1388598239748378, 'learning_rate': 9.5527950310559e-07, 'completion_length': 127.39286422729492, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857313156128, 'reward_std': 0.33800363540649414, 'kl': 0.001934051513671875, 'epoch': 0.22}

  4%|▍         | 72/1610 [21:25<7:31:51, 17.63s/it]
  5%|▍         | 73/1610 [21:38<6:56:51, 16.27s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.6087544645768136, 'learning_rate': 9.546583850931676e-07, 'completion_length': 141.41072463989258, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838216826319695, 'kl': 0.002655029296875, 'epoch': 0.23}

  5%|▍         | 73/1610 [21:38<6:56:51, 16.27s/it]
  5%|▍         | 74/1610 [21:50<6:25:50, 15.07s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.3720240863299873, 'learning_rate': 9.540372670807452e-07, 'completion_length': 121.5535774230957, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.00218963623046875, 'epoch': 0.23}

  5%|▍         | 74/1610 [21:50<6:25:50, 15.07s/it]
  5%|▍         | 75/1610 [22:04<6:17:19, 14.75s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.0276477523446659, 'learning_rate': 9.534161490683229e-07, 'completion_length': 139.85715103149414, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409644439816475, 'kl': 0.0022430419921875, 'epoch': 0.23}

  5%|▍         | 75/1610 [22:04<6:17:19, 14.75s/it]
  5%|▍         | 76/1610 [22:25<7:04:42, 16.61s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.4007521972875852, 'learning_rate': 9.527950310559006e-07, 'completion_length': 198.46429443359375, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5000000596046448, 'reward_std': 0.32695358991622925, 'kl': 0.003204345703125, 'epoch': 0.24}

  5%|▍         | 76/1610 [22:25<7:04:42, 16.61s/it]
  5%|▍         | 77/1610 [22:39<6:46:05, 15.89s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.363109630787747, 'learning_rate': 9.521739130434783e-07, 'completion_length': 134.35715103149414, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.00226593017578125, 'epoch': 0.24}

  5%|▍         | 77/1610 [22:39<6:46:05, 15.89s/it]
  5%|▍         | 78/1610 [22:57<6:56:26, 16.31s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.7277135090919103, 'learning_rate': 9.515527950310559e-07, 'completion_length': 168.80358123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.2142857313156128, 'kl': 0.0015106201171875, 'epoch': 0.24}

  5%|▍         | 78/1610 [22:57<6:56:26, 16.31s/it]
  5%|▍         | 79/1610 [23:11<6:40:32, 15.70s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.71208350470824, 'learning_rate': 9.509316770186336e-07, 'completion_length': 147.6964340209961, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.002288818359375, 'epoch': 0.25}

  5%|▍         | 79/1610 [23:11<6:40:32, 15.70s/it]
  5%|▍         | 80/1610 [23:27<6:43:50, 15.84s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.7631552202116891, 'learning_rate': 9.503105590062112e-07, 'completion_length': 179.75000762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.21981074661016464, 'kl': 0.00258636474609375, 'epoch': 0.25}

  5%|▍         | 80/1610 [23:27<6:43:50, 15.84s/it]
  5%|▌         | 81/1610 [23:44<6:50:30, 16.11s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.6540738658898306, 'learning_rate': 9.496894409937888e-07, 'completion_length': 163.87500762939453, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.24241765588521957, 'kl': 0.00254058837890625, 'epoch': 0.25}

  5%|▌         | 81/1610 [23:44<6:50:30, 16.11s/it]
  5%|▌         | 82/1610 [24:02<7:06:00, 16.73s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.3630448336843438, 'learning_rate': 9.490683229813665e-07, 'completion_length': 160.5714340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.696428656578064, 'reward_std': 0.1181928999722004, 'kl': 0.002166748046875, 'epoch': 0.25}

  5%|▌         | 82/1610 [24:02<7:06:00, 16.73s/it]
  5%|▌         | 83/1610 [24:21<7:19:09, 17.26s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.36439554753773595, 'learning_rate': 9.48447204968944e-07, 'completion_length': 168.58929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.00310516357421875, 'epoch': 0.26}

  5%|▌         | 83/1610 [24:21<7:19:09, 17.26s/it]
  5%|▌         | 84/1610 [24:34<6:46:54, 16.00s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.150186635693032, 'learning_rate': 9.478260869565216e-07, 'completion_length': 130.83929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.23086078464984894, 'kl': 0.001735687255859375, 'epoch': 0.26}

  5%|▌         | 84/1610 [24:34<6:46:54, 16.00s/it]
  5%|▌         | 85/1610 [24:51<6:57:58, 16.45s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.8857099672906691, 'learning_rate': 9.472049689440993e-07, 'completion_length': 153.55357360839844, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.21124479919672012, 'kl': 0.0023040771484375, 'epoch': 0.26}

  5%|▌         | 85/1610 [24:51<6:57:58, 16.45s/it]
  5%|▌         | 86/1610 [25:06<6:42:12, 15.83s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.8279797183882052, 'learning_rate': 9.46583850931677e-07, 'completion_length': 133.35714721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514648616313934, 'kl': 0.00229644775390625, 'epoch': 0.27}

  5%|▌         | 86/1610 [25:06<6:42:12, 15.83s/it]
  5%|▌         | 87/1610 [25:24<7:00:28, 16.57s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.7627361638191886, 'learning_rate': 9.459627329192546e-07, 'completion_length': 187.91072845458984, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.4642857909202576, 'reward_std': 0.3078143820166588, 'kl': 0.0031585693359375, 'epoch': 0.27}

  5%|▌         | 87/1610 [25:24<7:00:28, 16.57s/it]
  5%|▌         | 88/1610 [25:40<6:55:20, 16.37s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.26110367960803, 'learning_rate': 9.453416149068323e-07, 'completion_length': 142.4107208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.002471923828125, 'epoch': 0.27}

  5%|▌         | 88/1610 [25:40<6:55:20, 16.37s/it]
  6%|▌         | 89/1610 [25:57<7:01:59, 16.65s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.563584492926257, 'learning_rate': 9.447204968944099e-07, 'completion_length': 142.58929443359375, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.3392857909202576, 'reward_std': 0.29123931378126144, 'kl': 0.00315093994140625, 'epoch': 0.28}

  6%|▌         | 89/1610 [25:57<7:01:59, 16.65s/it]
  6%|▌         | 90/1610 [26:18<7:34:34, 17.94s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.8099800860620351, 'learning_rate': 9.440993788819875e-07, 'completion_length': 205.82144165039062, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.321428656578064, 'reward_std': 0.22781985998153687, 'kl': 0.00255584716796875, 'epoch': 0.28}

  6%|▌         | 90/1610 [26:18<7:34:34, 17.94s/it]
  6%|▌         | 91/1610 [26:37<7:39:25, 18.15s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.0133162860997715, 'learning_rate': 9.434782608695652e-07, 'completion_length': 175.50000762939453, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3928571939468384, 'reward_std': 0.33800365030765533, 'kl': 0.00223541259765625, 'epoch': 0.28}

  6%|▌         | 91/1610 [26:37<7:39:25, 18.15s/it]
  6%|▌         | 92/1610 [26:54<7:36:00, 18.02s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 2.207064712081355, 'learning_rate': 9.428571428571428e-07, 'completion_length': 165.62500762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5535715222358704, 'reward_std': 0.26353414356708527, 'kl': 0.00290679931640625, 'epoch': 0.29}

  6%|▌         | 92/1610 [26:54<7:36:00, 18.02s/it]
  6%|▌         | 93/1610 [27:14<7:51:10, 18.64s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.2931180046356248, 'learning_rate': 9.422360248447204e-07, 'completion_length': 159.60715103149414, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.535714328289032, 'reward_std': 0.4033503830432892, 'kl': 0.00348663330078125, 'epoch': 0.29}

  6%|▌         | 93/1610 [27:14<7:51:10, 18.64s/it]
  6%|▌         | 94/1610 [27:30<7:27:06, 17.70s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.8830950060012719, 'learning_rate': 9.41614906832298e-07, 'completion_length': 135.80358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981073915958405, 'kl': 0.00331878662109375, 'epoch': 0.29}

  6%|▌         | 94/1610 [27:30<7:27:06, 17.70s/it]
  6%|▌         | 95/1610 [27:48<7:27:03, 17.71s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.4095115402599572, 'learning_rate': 9.409937888198758e-07, 'completion_length': 158.33929443359375, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.3661355525255203, 'kl': 0.0034027099609375, 'epoch': 0.3}

  6%|▌         | 95/1610 [27:48<7:27:03, 17.71s/it]
  6%|▌         | 96/1610 [28:05<7:21:44, 17.51s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 0.5683180439570286, 'learning_rate': 9.403726708074534e-07, 'completion_length': 165.8214340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6250000596046448, 'reward_std': 0.23086076974868774, 'kl': 0.0033111572265625, 'epoch': 0.3}

  6%|▌         | 96/1610 [28:05<7:21:44, 17.51s/it]
  6%|▌         | 97/1610 [28:25<7:41:48, 18.31s/it]
                                                   
{'loss': 0.0001, 'grad_norm': 1.3599162599525438, 'learning_rate': 9.39751552795031e-07, 'completion_length': 226.50001525878906, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6071429252624512, 'reward_std': 0.2363857962191105, 'kl': 0.00360107421875, 'epoch': 0.3}

  6%|▌         | 97/1610 [28:25<7:41:48, 18.31s/it]
  6%|▌         | 98/1610 [28:44<7:45:48, 18.48s/it]
                                                   
{'loss': 0.0002, 'grad_norm': 1.1883135263547397, 'learning_rate': 9.391304347826087e-07, 'completion_length': 185.0714340209961, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3928571939468384, 'reward_std': 0.3963248133659363, 'kl': 0.0037689208984375, 'epoch': 0.3}

  6%|▌         | 98/1610 [28:44<7:45:48, 18.48s/it]
  6%|▌         | 99/1610 [28:57<7:02:19, 16.77s/it]
                                                   
{'loss': 0.0002, 'grad_norm': 1.1186231716621229, 'learning_rate': 9.385093167701863e-07, 'completion_length': 114.30357360839844, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2721000984311104, 'kl': 0.003753662109375, 'epoch': 0.31}

  6%|▌         | 99/1610 [28:57<7:02:19, 16.77s/it]
  6%|▌         | 100/1610 [29:18<7:34:58, 18.08s/it]
                                                    
{'loss': 0.0001, 'grad_norm': 1.1466972148170007, 'learning_rate': 9.37888198757764e-07, 'completion_length': 194.05358123779297, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5000000596046448, 'reward_std': 0.2967643141746521, 'kl': 0.00318145751953125, 'epoch': 0.31}

  6%|▌         | 100/1610 [29:18<7:34:58, 18.08s/it]
  6%|▋         | 101/1610 [33:00<33:16:22, 79.38s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.6349925313961503, 'learning_rate': 9.372670807453416e-07, 'completion_length': 171.05358123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.14838216826319695, 'kl': 0.0028533935546875, 'epoch': 0.31}

  6%|▋         | 101/1610 [33:00<33:16:22, 79.38s/it]
  6%|▋         | 102/1610 [33:18<25:34:48, 61.07s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.5260117323220221, 'learning_rate': 9.366459627329192e-07, 'completion_length': 166.50000762939453, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1071428656578064, 'kl': 0.00305938720703125, 'epoch': 0.32}

  6%|▋         | 102/1610 [33:18<25:34:48, 61.07s/it]
  6%|▋         | 103/1610 [33:36<20:09:50, 48.17s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.365087539460231, 'learning_rate': 9.360248447204968e-07, 'completion_length': 144.01786422729492, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.15943220630288124, 'kl': 0.00252532958984375, 'epoch': 0.32}

  6%|▋         | 103/1610 [33:36<20:09:50, 48.17s/it]
  6%|▋         | 104/1610 [33:58<16:46:05, 40.08s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.3941802087404997, 'learning_rate': 9.354037267080745e-07, 'completion_length': 156.60714721679688, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.2967643439769745, 'kl': 0.0032806396484375, 'epoch': 0.32}

  6%|▋         | 104/1610 [33:58<16:46:05, 40.08s/it]
  7%|▋         | 105/1610 [34:24<15:04:21, 36.05s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 2.190076141519707, 'learning_rate': 9.347826086956522e-07, 'completion_length': 170.00000762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.2967643290758133, 'kl': 0.00400543212890625, 'epoch': 0.33}

  7%|▋         | 105/1610 [34:24<15:04:21, 36.05s/it]
  7%|▋         | 106/1610 [34:45<13:04:54, 31.31s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.6371144939155062, 'learning_rate': 9.341614906832299e-07, 'completion_length': 137.80358123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.26657508313655853, 'kl': 0.00278472900390625, 'epoch': 0.33}

  7%|▋         | 106/1610 [34:45<13:04:54, 31.31s/it]
  7%|▋         | 107/1610 [35:16<13:02:26, 31.24s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.977543327777888, 'learning_rate': 9.335403726708074e-07, 'completion_length': 169.51786041259766, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5357143878936768, 'reward_std': 0.29924844950437546, 'kl': 0.00345611572265625, 'epoch': 0.33}

  7%|▋         | 107/1610 [35:16<13:02:26, 31.24s/it]
  7%|▋         | 108/1610 [35:54<13:56:07, 33.40s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.9784390654238404, 'learning_rate': 9.32919254658385e-07, 'completion_length': 183.7321548461914, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.2580091394484043, 'kl': 0.00287628173828125, 'epoch': 0.34}

  7%|▋         | 108/1610 [35:54<13:56:07, 33.40s/it]
  7%|▋         | 109/1610 [37:43<23:24:31, 56.14s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.7554084618306902, 'learning_rate': 9.322981366459626e-07, 'completion_length': 150.89286041259766, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.1539071872830391, 'kl': 0.003448486328125, 'epoch': 0.34}

  7%|▋         | 109/1610 [37:43<23:24:31, 56.14s/it]
  7%|▋         | 110/1610 [39:29<29:32:12, 70.89s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.3233747364125457, 'learning_rate': 9.316770186335403e-07, 'completion_length': 162.0714340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.45619654655456543, 'kl': 0.00396728515625, 'epoch': 0.34}

  7%|▋         | 110/1610 [39:29<29:32:12, 70.89s/it]
  7%|▋         | 111/1610 [40:43<29:55:16, 71.86s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 0.7559654776881304, 'learning_rate': 9.310559006211179e-07, 'completion_length': 143.23215103149414, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071835577488, 'kl': 0.00434112548828125, 'epoch': 0.34}

  7%|▋         | 111/1610 [40:43<29:55:16, 71.86s/it]
  7%|▋         | 112/1610 [42:34<34:49:18, 83.68s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.7047354919294013, 'learning_rate': 9.304347826086955e-07, 'completion_length': 191.12500762939453, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2253357470035553, 'kl': 0.00302886962890625, 'epoch': 0.35}

  7%|▋         | 112/1610 [42:34<34:49:18, 83.68s/it]
  7%|▋         | 113/1610 [44:04<35:37:22, 85.67s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 0.8859876176812524, 'learning_rate': 9.298136645962732e-07, 'completion_length': 209.7321548461914, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5535714626312256, 'reward_std': 0.39782536029815674, 'kl': 0.00399017333984375, 'epoch': 0.35}

  7%|▋         | 113/1610 [44:04<35:37:22, 85.67s/it]
  7%|▋         | 114/1610 [45:09<33:01:52, 79.49s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.3429892138532624, 'learning_rate': 9.291925465838509e-07, 'completion_length': 167.5178680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.26657508313655853, 'kl': 0.0030975341796875, 'epoch': 0.35}

  7%|▋         | 114/1610 [45:09<33:01:52, 79.49s/it]
  7%|▋         | 115/1610 [46:41<34:29:25, 83.05s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.7764617777312725, 'learning_rate': 9.285714285714285e-07, 'completion_length': 173.96429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357656300068, 'kl': 0.0035552978515625, 'epoch': 0.36}

  7%|▋         | 115/1610 [46:41<34:29:25, 83.05s/it]
  7%|▋         | 116/1610 [48:16<36:02:58, 86.87s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 0.5317396651783775, 'learning_rate': 9.279503105590062e-07, 'completion_length': 183.96429443359375, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7321429252624512, 'reward_std': 0.19813742116093636, 'kl': 0.003662109375, 'epoch': 0.36}

  7%|▋         | 116/1610 [48:17<36:02:58, 86.87s/it]
  7%|▋         | 117/1610 [49:28<34:05:13, 82.19s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 2.217783266326365, 'learning_rate': 9.273291925465838e-07, 'completion_length': 131.71428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.30228933691978455, 'kl': 0.00274658203125, 'epoch': 0.36}

  7%|▋         | 117/1610 [49:28<34:05:13, 82.19s/it]
  7%|▋         | 118/1610 [50:39<32:42:46, 78.93s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.9359933905572135, 'learning_rate': 9.267080745341614e-07, 'completion_length': 134.6785774230957, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.535714328289032, 'reward_std': 0.20117833465337753, 'kl': 0.00301361083984375, 'epoch': 0.37}

  7%|▋         | 118/1610 [50:39<32:42:46, 78.93s/it]
  7%|▋         | 119/1610 [51:49<31:35:10, 76.26s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.2725484336380037, 'learning_rate': 9.260869565217391e-07, 'completion_length': 153.26786041259766, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.3435286581516266, 'kl': 0.0040740966796875, 'epoch': 0.37}

  7%|▋         | 119/1610 [51:49<31:35:10, 76.26s/it]
  7%|▋         | 120/1610 [53:37<35:26:46, 85.64s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 2.064515268229422, 'learning_rate': 9.254658385093167e-07, 'completion_length': 155.23215103149414, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5535715222358704, 'reward_std': 0.3495605140924454, 'kl': 0.0055999755859375, 'epoch': 0.37}

  7%|▋         | 120/1610 [53:37<35:26:46, 85.64s/it]
  8%|▊         | 121/1610 [54:12<29:12:43, 70.63s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.4524349300285162, 'learning_rate': 9.248447204968943e-07, 'completion_length': 192.8571548461914, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.5357143878936768, 'reward_std': 0.3248962536454201, 'kl': 0.00443267822265625, 'epoch': 0.38}

  8%|▊         | 121/1610 [54:12<29:12:43, 70.63s/it]
  8%|▊         | 122/1610 [54:28<22:23:24, 54.17s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.675562344157013, 'learning_rate': 9.24223602484472e-07, 'completion_length': 125.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2142857238650322, 'kl': 0.00390625, 'epoch': 0.38}

  8%|▊         | 122/1610 [54:28<22:23:24, 54.17s/it]
  8%|▊         | 123/1610 [54:43<17:32:05, 42.45s/it]
                                                     
{'loss': 0.0001, 'grad_norm': 1.5523159588318467, 'learning_rate': 9.236024844720497e-07, 'completion_length': 141.10714721679688, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.00325775146484375, 'epoch': 0.38}

  8%|▊         | 123/1610 [54:43<17:32:05, 42.45s/it]
  8%|▊         | 124/1610 [55:01<14:31:48, 35.20s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 0.8176928865899492, 'learning_rate': 9.229813664596273e-07, 'completion_length': 170.7857208251953, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.571428656578064, 'reward_std': 0.23638580739498138, 'kl': 0.00389862060546875, 'epoch': 0.39}

  8%|▊         | 124/1610 [55:01<14:31:48, 35.20s/it]
  8%|▊         | 125/1610 [55:15<11:48:08, 28.61s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.3022724852070962, 'learning_rate': 9.22360248447205e-07, 'completion_length': 135.75000762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.4149572253227234, 'kl': 0.00382232666015625, 'epoch': 0.39}

  8%|▊         | 125/1610 [55:15<11:48:08, 28.61s/it]
  8%|▊         | 126/1610 [55:30<10:06:46, 24.53s/it]
                                                     
{'loss': 0.0002, 'grad_norm': 1.8874844476382755, 'learning_rate': 9.217391304347826e-07, 'completion_length': 138.1607208251953, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.4149572402238846, 'kl': 0.004058837890625, 'epoch': 0.39}

  8%|▊         | 126/1610 [55:30<10:06:46, 24.53s/it]
  8%|▊         | 127/1610 [55:49<9:30:55, 23.10s/it] 
                                                    
{'loss': 0.0002, 'grad_norm': 1.1163721941179752, 'learning_rate': 9.211180124223602e-07, 'completion_length': 202.7857208251953, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.410714328289032, 'reward_std': 0.30228936672210693, 'kl': 0.0045318603515625, 'epoch': 0.39}

  8%|▊         | 127/1610 [55:49<9:30:55, 23.10s/it]
  8%|▊         | 128/1610 [56:03<8:19:09, 20.21s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.3176819413403666, 'learning_rate': 9.204968944099379e-07, 'completion_length': 127.58928680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.2500000074505806, 'kl': 0.0038909912109375, 'epoch': 0.4}

  8%|▊         | 128/1610 [56:03<8:19:09, 20.21s/it]
  8%|▊         | 129/1610 [56:22<8:07:14, 19.74s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 0.6374585816240141, 'learning_rate': 9.198757763975155e-07, 'completion_length': 169.23214721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.00384521484375, 'epoch': 0.4}

  8%|▊         | 129/1610 [56:22<8:07:14, 19.74s/it]
  8%|▊         | 130/1610 [56:41<8:06:30, 19.72s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 0.8565921359486017, 'learning_rate': 9.19254658385093e-07, 'completion_length': 176.3214340209961, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.2580091208219528, 'kl': 0.00479888916015625, 'epoch': 0.4}

  8%|▊         | 130/1610 [56:41<8:06:30, 19.72s/it]
  8%|▊         | 131/1610 [56:58<7:46:14, 18.91s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.3162502041335888, 'learning_rate': 9.186335403726707e-07, 'completion_length': 140.12500762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2967643216252327, 'kl': 0.0041046142578125, 'epoch': 0.41}

  8%|▊         | 131/1610 [56:58<7:46:14, 18.91s/it]
  8%|▊         | 132/1610 [57:15<7:30:20, 18.28s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 0.8916731767030114, 'learning_rate': 9.180124223602484e-07, 'completion_length': 160.1607208251953, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.2253357470035553, 'kl': 0.0037841796875, 'epoch': 0.41}

  8%|▊         | 132/1610 [57:15<7:30:20, 18.28s/it]
  8%|▊         | 133/1610 [57:32<7:22:58, 18.00s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.1149515349835337, 'learning_rate': 9.17391304347826e-07, 'completion_length': 166.1964340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25552501529455185, 'kl': 0.0052337646484375, 'epoch': 0.41}

  8%|▊         | 133/1610 [57:32<7:22:58, 18.00s/it]
  8%|▊         | 134/1610 [57:48<7:07:21, 17.37s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 0.9098453410702299, 'learning_rate': 9.167701863354037e-07, 'completion_length': 163.5357208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714402794838, 'kl': 0.00439453125, 'epoch': 0.42}

  8%|▊         | 134/1610 [57:48<7:07:21, 17.37s/it]
  8%|▊         | 135/1610 [58:08<7:21:04, 17.94s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 2.0428814679997016, 'learning_rate': 9.161490683229813e-07, 'completion_length': 152.4464340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.29123930633068085, 'kl': 0.005401611328125, 'epoch': 0.42}

  8%|▊         | 135/1610 [58:08<7:21:04, 17.94s/it]
  8%|▊         | 136/1610 [58:24<7:10:50, 17.54s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.6141062448440855, 'learning_rate': 9.155279503105589e-07, 'completion_length': 171.96429443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2967643439769745, 'kl': 0.0053863525390625, 'epoch': 0.42}

  8%|▊         | 136/1610 [58:24<7:10:50, 17.54s/it]
  9%|▊         | 137/1610 [58:41<7:08:44, 17.46s/it]
                                                    
{'loss': 0.0001, 'grad_norm': 2.463527774459715, 'learning_rate': 9.149068322981366e-07, 'completion_length': 167.30357360839844, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.23086077719926834, 'kl': 0.00350189208984375, 'epoch': 0.43}

  9%|▊         | 137/1610 [58:41<7:08:44, 17.46s/it]
  9%|▊         | 138/1610 [58:57<6:56:27, 16.98s/it]
                                                    
{'loss': 0.0001, 'grad_norm': 1.408385545351653, 'learning_rate': 9.142857142857142e-07, 'completion_length': 168.14286041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.3550855219364166, 'kl': 0.00356292724609375, 'epoch': 0.43}

  9%|▊         | 138/1610 [58:57<6:56:27, 16.98s/it]
  9%|▊         | 139/1610 [59:15<6:59:19, 17.10s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 0.9972800665956963, 'learning_rate': 9.136645962732918e-07, 'completion_length': 154.1428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25552502274513245, 'kl': 0.00421905517578125, 'epoch': 0.43}

  9%|▊         | 139/1610 [59:15<6:59:19, 17.10s/it]
  9%|▊         | 140/1610 [59:29<6:40:43, 16.36s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.2754437126885732, 'learning_rate': 9.130434782608695e-07, 'completion_length': 145.3928680419922, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.30228935927152634, 'kl': 0.0043487548828125, 'epoch': 0.43}

  9%|▊         | 140/1610 [59:29<6:40:43, 16.36s/it]
  9%|▉         | 141/1610 [59:49<7:02:41, 17.26s/it]
                                                    
{'loss': 0.0002, 'grad_norm': 1.2016439457896007, 'learning_rate': 9.124223602484472e-07, 'completion_length': 172.5178680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071798324585, 'kl': 0.0045166015625, 'epoch': 0.44}

  9%|▉         | 141/1610 [59:49<7:02:41, 17.26s/it]
  9%|▉         | 142/1610 [1:00:02<6:32:33, 16.04s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.1263950709389763, 'learning_rate': 9.118012422360248e-07, 'completion_length': 122.46429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.0042724609375, 'epoch': 0.44}

  9%|▉         | 142/1610 [1:00:02<6:32:33, 16.04s/it]
  9%|▉         | 143/1610 [1:00:17<6:28:04, 15.87s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.4270582244772767, 'learning_rate': 9.111801242236025e-07, 'completion_length': 149.53572463989258, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.25552503019571304, 'kl': 0.0050811767578125, 'epoch': 0.44}

  9%|▉         | 143/1610 [1:00:17<6:28:04, 15.87s/it]
  9%|▉         | 144/1610 [1:00:32<6:18:29, 15.49s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.5052697349942228, 'learning_rate': 9.105590062111801e-07, 'completion_length': 142.28572463989258, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.30228936672210693, 'kl': 0.0049285888671875, 'epoch': 0.45}

  9%|▉         | 144/1610 [1:00:32<6:18:29, 15.49s/it]
  9%|▉         | 145/1610 [1:00:44<5:51:48, 14.41s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.7933787461192083, 'learning_rate': 9.099378881987577e-07, 'completion_length': 126.64286422729492, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216826319695, 'kl': 0.0048065185546875, 'epoch': 0.45}

  9%|▉         | 145/1610 [1:00:44<5:51:48, 14.41s/it]
  9%|▉         | 146/1610 [1:01:00<6:07:29, 15.06s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.730239003845002, 'learning_rate': 9.093167701863354e-07, 'completion_length': 172.98214721679688, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.23638580739498138, 'kl': 0.0068206787109375, 'epoch': 0.45}

  9%|▉         | 146/1610 [1:01:00<6:07:29, 15.06s/it]
  9%|▉         | 147/1610 [1:01:17<6:18:02, 15.50s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8714951965102284, 'learning_rate': 9.08695652173913e-07, 'completion_length': 154.30358123779297, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.1539071835577488, 'kl': 0.004058837890625, 'epoch': 0.46}

  9%|▉         | 147/1610 [1:01:17<6:18:02, 15.50s/it]
  9%|▉         | 148/1610 [1:01:29<5:51:15, 14.42s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.7940331580379969, 'learning_rate': 9.080745341614906e-07, 'completion_length': 104.21429061889648, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.00386810302734375, 'epoch': 0.46}

  9%|▉         | 148/1610 [1:01:29<5:51:15, 14.42s/it]
  9%|▉         | 149/1610 [1:02:54<14:29:15, 35.70s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 1.4252409046070424, 'learning_rate': 9.074534161490683e-07, 'completion_length': 135.73214721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.26657506823539734, 'kl': 0.006256103515625, 'epoch': 0.46}

  9%|▉         | 149/1610 [1:02:54<14:29:15, 35.70s/it]
  9%|▉         | 150/1610 [1:05:48<31:18:02, 77.18s/it]
                                                       
{'loss': 0.0002, 'grad_norm': 11.452472805244504, 'learning_rate': 9.06832298136646e-07, 'completion_length': 149.23214721679688, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.3078143745660782, 'kl': 0.0052490234375, 'epoch': 0.47}

  9%|▉         | 150/1610 [1:05:48<31:18:02, 77.18s/it]
  9%|▉         | 151/1610 [1:08:10<39:08:51, 96.59s/it]
                                                       
{'loss': 0.0001, 'grad_norm': 0.7566630876200011, 'learning_rate': 9.062111801242236e-07, 'completion_length': 157.82143783569336, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.2610500380396843, 'kl': 0.003265380859375, 'epoch': 0.47}

  9%|▉         | 151/1610 [1:08:10<39:08:51, 96.59s/it]
  9%|▉         | 152/1610 [1:10:48<46:34:06, 114.98s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 1.471344114414134, 'learning_rate': 9.055900621118013e-07, 'completion_length': 188.00001525878906, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.2967643290758133, 'kl': 0.0045928955078125, 'epoch': 0.47}

  9%|▉         | 152/1610 [1:10:48<46:34:06, 114.98s/it]
 10%|▉         | 153/1610 [1:11:05<34:37:59, 85.57s/it] 
                                                       
{'loss': 0.0002, 'grad_norm': 1.4829453617481654, 'learning_rate': 9.049689440993789e-07, 'completion_length': 170.17858123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500001192092896, 'reward_std': 0.2253357619047165, 'kl': 0.00537109375, 'epoch': 0.48}

 10%|▉         | 153/1610 [1:11:05<34:37:59, 85.57s/it]
 10%|▉         | 154/1610 [1:12:17<33:01:36, 81.66s/it]
                                                       
{'loss': 0.0001, 'grad_norm': 1.2593458353680342, 'learning_rate': 9.043478260869564e-07, 'completion_length': 150.25000762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5535715222358704, 'reward_std': 0.25248411670327187, 'kl': 0.00356292724609375, 'epoch': 0.48}

 10%|▉         | 154/1610 [1:12:18<33:01:36, 81.66s/it]
 10%|▉         | 155/1610 [1:15:01<42:59:33, 106.37s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 0.9699075418882815, 'learning_rate': 9.037267080745341e-07, 'completion_length': 158.7857208251953, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.2142857238650322, 'kl': 0.00473785400390625, 'epoch': 0.48}

 10%|▉         | 155/1610 [1:15:01<42:59:33, 106.37s/it]
 10%|▉         | 156/1610 [1:15:22<32:33:19, 80.60s/it] 
                                                       
{'loss': 0.0002, 'grad_norm': 1.495374737238485, 'learning_rate': 9.031055900621117e-07, 'completion_length': 152.75000762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.25552502274513245, 'kl': 0.00592041015625, 'epoch': 0.48}

 10%|▉         | 156/1610 [1:15:22<32:33:19, 80.60s/it]
 10%|▉         | 157/1610 [1:18:15<43:45:36, 108.42s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 1.0370168774316852, 'learning_rate': 9.024844720496893e-07, 'completion_length': 121.66072463989258, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.0043487548828125, 'epoch': 0.49}

 10%|▉         | 157/1610 [1:18:15<43:45:36, 108.42s/it]
 10%|▉         | 158/1610 [1:20:39<48:03:18, 119.14s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 0.9222625293875729, 'learning_rate': 9.01863354037267e-07, 'completion_length': 181.37500762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25552501529455185, 'kl': 0.005340576171875, 'epoch': 0.49}

 10%|▉         | 158/1610 [1:20:40<48:03:18, 119.14s/it]
 10%|▉         | 159/1610 [1:22:56<50:02:27, 124.15s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 1.6081175704706514, 'learning_rate': 9.012422360248447e-07, 'completion_length': 140.0178680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.18409645557403564, 'kl': 0.0055389404296875, 'epoch': 0.49}

 10%|▉         | 159/1610 [1:22:56<50:02:27, 124.15s/it]
 10%|▉         | 160/1610 [1:25:29<53:35:42, 133.06s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 0.8569288866601684, 'learning_rate': 9.006211180124223e-07, 'completion_length': 159.05357360839844, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357619047165, 'kl': 0.0048065185546875, 'epoch': 0.5}

 10%|▉         | 160/1610 [1:25:29<53:35:42, 133.06s/it]
 10%|█         | 161/1610 [1:26:01<41:22:45, 102.81s/it]
                                                        
{'loss': 0.0002, 'grad_norm': 1.157340463192231, 'learning_rate': 9e-07, 'completion_length': 149.125, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.21981074661016464, 'kl': 0.005584716796875, 'epoch': 0.5}

 10%|█         | 161/1610 [1:26:01<41:22:45, 102.81s/it]
 10%|█         | 162/1610 [1:26:16<30:44:07, 76.41s/it] 
                                                       
{'loss': 0.0002, 'grad_norm': 2.655069020269298, 'learning_rate': 8.993788819875776e-07, 'completion_length': 161.25000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.0043182373046875, 'epoch': 0.5}

 10%|█         | 162/1610 [1:26:16<30:44:07, 76.41s/it]
 10%|█         | 163/1610 [1:26:30<23:08:52, 57.59s/it]
                                                       
{'loss': 0.0002, 'grad_norm': 0.7474239631754882, 'learning_rate': 8.987577639751552e-07, 'completion_length': 147.58929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838217198848724, 'kl': 0.004425048828125, 'epoch': 0.51}

 10%|█         | 163/1610 [1:26:30<23:08:52, 57.59s/it]
 10%|█         | 164/1610 [1:26:47<18:18:02, 45.56s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 1.3129315497523586, 'learning_rate': 8.981366459627329e-07, 'completion_length': 128.5357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.0069427490234375, 'epoch': 0.51}

 10%|█         | 164/1610 [1:26:47<18:18:02, 45.56s/it]
 10%|█         | 165/1610 [1:27:03<14:40:27, 36.56s/it]
                                                       
{'loss': 0.0002, 'grad_norm': 0.5898750222467962, 'learning_rate': 8.975155279503105e-07, 'completion_length': 166.92857360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.0040435791015625, 'epoch': 0.51}

 10%|█         | 165/1610 [1:27:03<14:40:27, 36.56s/it]
 10%|█         | 166/1610 [1:27:22<12:36:39, 31.44s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 1.2016358702243257, 'learning_rate': 8.968944099378881e-07, 'completion_length': 172.75000762939453, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.3078143820166588, 'kl': 0.0067291259765625, 'epoch': 0.52}

 10%|█         | 166/1610 [1:27:22<12:36:39, 31.44s/it]
 10%|█         | 167/1610 [1:27:41<11:04:12, 27.62s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.8748140695939824, 'learning_rate': 8.962732919254658e-07, 'completion_length': 152.5178680419922, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7857143878936768, 'reward_std': 0.28365693986415863, 'kl': 0.00628662109375, 'epoch': 0.52}

 10%|█         | 167/1610 [1:27:41<11:04:12, 27.62s/it]
 10%|█         | 168/1610 [1:27:57<9:37:07, 24.01s/it] 
                                                      
{'loss': 0.0002, 'grad_norm': 0.8709524026953059, 'learning_rate': 8.956521739130435e-07, 'completion_length': 146.3928680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.0060577392578125, 'epoch': 0.52}

 10%|█         | 168/1610 [1:27:57<9:37:07, 24.01s/it]
 10%|█         | 169/1610 [1:28:11<8:28:46, 21.18s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8313355538444024, 'learning_rate': 8.950310559006211e-07, 'completion_length': 150.1607208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2610500454902649, 'kl': 0.0054168701171875, 'epoch': 0.52}

 10%|█         | 169/1610 [1:28:11<8:28:46, 21.18s/it]
 11%|█         | 170/1610 [1:28:26<7:40:08, 19.17s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.752903687785537, 'learning_rate': 8.944099378881988e-07, 'completion_length': 123.6964340209961, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.0044708251953125, 'epoch': 0.53}

 11%|█         | 170/1610 [1:28:26<7:40:08, 19.17s/it]
 11%|█         | 171/1610 [1:28:41<7:14:57, 18.14s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 4.488361394057115, 'learning_rate': 8.937888198757764e-07, 'completion_length': 143.33929443359375, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.37371790409088135, 'kl': 0.005706787109375, 'epoch': 0.53}

 11%|█         | 171/1610 [1:28:41<7:14:57, 18.14s/it]
 11%|█         | 172/1610 [1:28:57<6:55:22, 17.33s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.7824490043939485, 'learning_rate': 8.93167701863354e-07, 'completion_length': 150.1964340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.23086077719926834, 'kl': 0.0047454833984375, 'epoch': 0.53}

 11%|█         | 172/1610 [1:28:57<6:55:22, 17.33s/it]
 11%|█         | 173/1610 [1:29:14<6:55:48, 17.36s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.2923647473834328, 'learning_rate': 8.925465838509317e-07, 'completion_length': 132.25000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.26353414356708527, 'kl': 0.006866455078125, 'epoch': 0.54}

 11%|█         | 173/1610 [1:29:14<6:55:48, 17.36s/it]
 11%|█         | 174/1610 [1:29:31<6:48:45, 17.08s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.3370372170000226, 'learning_rate': 8.919254658385093e-07, 'completion_length': 143.0178680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.25552501529455185, 'kl': 0.0058746337890625, 'epoch': 0.54}

 11%|█         | 174/1610 [1:29:31<6:48:45, 17.08s/it]
 11%|█         | 175/1610 [1:29:45<6:27:53, 16.22s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.0237255461506507, 'learning_rate': 8.913043478260869e-07, 'completion_length': 159.37500762939453, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.2363857924938202, 'kl': 0.0063018798828125, 'epoch': 0.54}

 11%|█         | 175/1610 [1:29:45<6:27:53, 16.22s/it]
 11%|█         | 176/1610 [1:30:03<6:42:35, 16.84s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.6154418094438104, 'learning_rate': 8.906832298136646e-07, 'completion_length': 151.01786041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1428571529686451, 'kl': 0.0055999755859375, 'epoch': 0.55}

 11%|█         | 176/1610 [1:30:03<6:42:35, 16.84s/it]
 11%|█         | 177/1610 [1:30:20<6:39:21, 16.72s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.2919509856390514, 'learning_rate': 8.900621118012423e-07, 'completion_length': 175.08928680419922, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.3188644349575043, 'kl': 0.00653076171875, 'epoch': 0.55}

 11%|█         | 177/1610 [1:30:20<6:39:21, 16.72s/it]
 11%|█         | 178/1610 [1:30:35<6:29:34, 16.32s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.9750529035181053, 'learning_rate': 8.894409937888198e-07, 'completion_length': 154.87500762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.26657507568597794, 'kl': 0.0059356689453125, 'epoch': 0.55}

 11%|█         | 178/1610 [1:30:35<6:29:34, 16.32s/it]
 11%|█         | 179/1610 [1:30:54<6:44:39, 16.97s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8768942971589356, 'learning_rate': 8.888198757763975e-07, 'completion_length': 155.1964340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142858505249023, 'reward_std': 0.25552501529455185, 'kl': 0.0088653564453125, 'epoch': 0.56}

 11%|█         | 179/1610 [1:30:54<6:44:39, 16.97s/it]
 11%|█         | 180/1610 [1:31:11<6:45:28, 17.01s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9131067556753617, 'learning_rate': 8.881987577639751e-07, 'completion_length': 171.8571548461914, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.17553051561117172, 'kl': 0.009063720703125, 'epoch': 0.56}

 11%|█         | 180/1610 [1:31:11<6:45:28, 17.01s/it]
 11%|█         | 181/1610 [1:31:25<6:26:54, 16.25s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.5317746383491884, 'learning_rate': 8.875776397515527e-07, 'completion_length': 153.94644165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.0052337646484375, 'epoch': 0.56}

 11%|█         | 181/1610 [1:31:25<6:26:54, 16.25s/it]
 11%|█▏        | 182/1610 [1:31:43<6:35:38, 16.62s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.492393264733909, 'learning_rate': 8.869565217391303e-07, 'completion_length': 173.0357208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5892857909202576, 'reward_std': 0.37371791899204254, 'kl': 0.0054473876953125, 'epoch': 0.57}

 11%|█▏        | 182/1610 [1:31:43<6:35:38, 16.62s/it]
 11%|█▏        | 183/1610 [1:31:59<6:33:14, 16.53s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.5422811290113992, 'learning_rate': 8.86335403726708e-07, 'completion_length': 167.53572845458984, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.1071428619325161, 'kl': 0.00518798828125, 'epoch': 0.57}

 11%|█▏        | 183/1610 [1:31:59<6:33:14, 16.53s/it]
 11%|█▏        | 184/1610 [1:32:17<6:45:36, 17.07s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.0384897598903482, 'learning_rate': 8.857142857142856e-07, 'completion_length': 166.48214721679688, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.11266788095235825, 'kl': 0.006011962890625, 'epoch': 0.57}

 11%|█▏        | 184/1610 [1:32:17<6:45:36, 17.07s/it]
 11%|█▏        | 185/1610 [1:32:31<6:23:22, 16.14s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.5894726634037153, 'learning_rate': 8.850931677018632e-07, 'completion_length': 153.4107208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.00543212890625, 'epoch': 0.57}

 11%|█▏        | 185/1610 [1:32:31<6:23:22, 16.14s/it]
 12%|█▏        | 186/1610 [1:32:45<6:03:11, 15.30s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.4824825485728162, 'learning_rate': 8.84472049689441e-07, 'completion_length': 134.05358123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.0824786126613617, 'kl': 0.00640869140625, 'epoch': 0.58}

 12%|█▏        | 186/1610 [1:32:45<6:03:11, 15.30s/it]
 12%|█▏        | 187/1610 [1:33:02<6:15:51, 15.85s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.43780563964345054, 'learning_rate': 8.838509316770186e-07, 'completion_length': 154.1607208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.07695358991622925, 'kl': 0.0052490234375, 'epoch': 0.58}

 12%|█▏        | 187/1610 [1:33:02<6:15:51, 15.85s/it]
 12%|█▏        | 188/1610 [1:33:20<6:33:32, 16.60s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.1167197734601564, 'learning_rate': 8.832298136645962e-07, 'completion_length': 178.32144165039062, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.24695909023284912, 'kl': 0.00579833984375, 'epoch': 0.58}

 12%|█▏        | 188/1610 [1:33:20<6:33:32, 16.60s/it]
 12%|█▏        | 189/1610 [1:33:36<6:26:45, 16.33s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.097859173862402, 'learning_rate': 8.826086956521739e-07, 'completion_length': 161.21429443359375, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409644439816475, 'kl': 0.00628662109375, 'epoch': 0.59}

 12%|█▏        | 189/1610 [1:33:36<6:26:45, 16.33s/it]
 12%|█▏        | 190/1610 [1:33:51<6:20:05, 16.06s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 3.7568711155468204, 'learning_rate': 8.819875776397515e-07, 'completion_length': 166.08928680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.23638580739498138, 'kl': 0.005767822265625, 'epoch': 0.59}

 12%|█▏        | 190/1610 [1:33:51<6:20:05, 16.06s/it]
 12%|█▏        | 191/1610 [1:34:10<6:38:33, 16.85s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.6453071844895886, 'learning_rate': 8.813664596273291e-07, 'completion_length': 163.2321548461914, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.0059051513671875, 'epoch': 0.59}

 12%|█▏        | 191/1610 [1:34:10<6:38:33, 16.85s/it]
 12%|█▏        | 192/1610 [1:34:28<6:48:49, 17.30s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.6108678184424081, 'learning_rate': 8.807453416149068e-07, 'completion_length': 160.9107208251953, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3571428656578064, 'kl': 0.0055694580078125, 'epoch': 0.6}

 12%|█▏        | 192/1610 [1:34:28<6:48:49, 17.30s/it]
 12%|█▏        | 193/1610 [1:34:44<6:33:56, 16.68s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.2515822546581903, 'learning_rate': 8.801242236024844e-07, 'completion_length': 150.4107208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.26657506823539734, 'kl': 0.006317138671875, 'epoch': 0.6}

 12%|█▏        | 193/1610 [1:34:44<6:33:56, 16.68s/it]
 12%|█▏        | 194/1610 [1:34:57<6:11:29, 15.74s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.5030897399038436, 'learning_rate': 8.79503105590062e-07, 'completion_length': 119.96429061889648, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.005096435546875, 'epoch': 0.6}

 12%|█▏        | 194/1610 [1:34:57<6:11:29, 15.74s/it]
 12%|█▏        | 195/1610 [1:35:15<6:27:53, 16.45s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8702341288903694, 'learning_rate': 8.788819875776398e-07, 'completion_length': 196.73214721679688, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.14838216453790665, 'kl': 0.0070648193359375, 'epoch': 0.61}

 12%|█▏        | 195/1610 [1:35:15<6:27:53, 16.45s/it]
 12%|█▏        | 196/1610 [1:35:30<6:18:00, 16.04s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8461561643641895, 'learning_rate': 8.782608695652174e-07, 'completion_length': 164.44644165039062, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.29123931378126144, 'kl': 0.0070953369140625, 'epoch': 0.61}

 12%|█▏        | 196/1610 [1:35:30<6:18:00, 16.04s/it]
 12%|█▏        | 197/1610 [1:35:44<6:03:15, 15.43s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.6102114113117648, 'learning_rate': 8.77639751552795e-07, 'completion_length': 133.83929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.07695358991622925, 'kl': 0.0063934326171875, 'epoch': 0.61}

 12%|█▏        | 197/1610 [1:35:44<6:03:15, 15.43s/it]
 12%|█▏        | 198/1610 [1:36:02<6:17:11, 16.03s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8002560888683549, 'learning_rate': 8.770186335403727e-07, 'completion_length': 162.01786041259766, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.21981073915958405, 'kl': 0.007049560546875, 'epoch': 0.61}

 12%|█▏        | 198/1610 [1:36:02<6:17:11, 16.03s/it]
 12%|█▏        | 199/1610 [1:36:19<6:23:06, 16.29s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 2.165903304409091, 'learning_rate': 8.763975155279503e-07, 'completion_length': 167.5178680419922, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4642857909202576, 'reward_std': 0.32695358991622925, 'kl': 0.0068817138671875, 'epoch': 0.62}

 12%|█▏        | 199/1610 [1:36:19<6:23:06, 16.29s/it]
 12%|█▏        | 200/1610 [1:36:36<6:30:27, 16.61s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.55647663129588, 'learning_rate': 8.757763975155279e-07, 'completion_length': 163.96428680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.19514648616313934, 'kl': 0.00726318359375, 'epoch': 0.62}

 12%|█▏        | 200/1610 [1:36:36<6:30:27, 16.61s/it]
 12%|█▏        | 201/1610 [1:39:34<25:24:05, 64.90s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.7758378205011851, 'learning_rate': 8.751552795031055e-07, 'completion_length': 162.1607208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.571428656578064, 'reward_std': 0.26657506823539734, 'kl': 0.0070343017578125, 'epoch': 0.62}

 12%|█▏        | 201/1610 [1:39:34<25:24:05, 64.90s/it]
 13%|█▎        | 202/1610 [1:39:46<19:16:38, 49.29s/it]
                                                       
{'loss': 0.0002, 'grad_norm': 1.2270395001156176, 'learning_rate': 8.745341614906831e-07, 'completion_length': 136.64286041259766, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.07695358991622925, 'kl': 0.005950927734375, 'epoch': 0.63}

 13%|█▎        | 202/1610 [1:39:46<19:16:38, 49.29s/it]
 13%|█▎        | 203/1610 [1:40:04<15:33:35, 39.81s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.7499011542180827, 'learning_rate': 8.739130434782607e-07, 'completion_length': 158.3214340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2253357470035553, 'kl': 0.0072784423828125, 'epoch': 0.63}

 13%|█▎        | 203/1610 [1:40:04<15:33:35, 39.81s/it]
 13%|█▎        | 204/1610 [1:40:21<12:54:18, 33.04s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.9927031241417711, 'learning_rate': 8.732919254658385e-07, 'completion_length': 168.0714340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2967643439769745, 'kl': 0.007843017578125, 'epoch': 0.63}

 13%|█▎        | 204/1610 [1:40:21<12:54:18, 33.04s/it]
 13%|█▎        | 205/1610 [1:40:40<11:10:48, 28.65s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 3.626676598025751, 'learning_rate': 8.726708074534161e-07, 'completion_length': 183.2321548461914, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.33800365030765533, 'kl': 0.0074310302734375, 'epoch': 0.64}

 13%|█▎        | 205/1610 [1:40:40<11:10:48, 28.65s/it]
 13%|█▎        | 206/1610 [1:40:54<9:28:08, 24.28s/it] 
                                                      
{'loss': 0.0002, 'grad_norm': 2.573401179615728, 'learning_rate': 8.720496894409937e-07, 'completion_length': 133.87500381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.0055389404296875, 'epoch': 0.64}

 13%|█▎        | 206/1610 [1:40:54<9:28:08, 24.28s/it]
 13%|█▎        | 207/1610 [1:41:09<8:27:21, 21.70s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 2.826205395444688, 'learning_rate': 8.714285714285714e-07, 'completion_length': 141.44643783569336, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535715222358704, 'reward_std': 0.2610500380396843, 'kl': 0.005401611328125, 'epoch': 0.64}

 13%|█▎        | 207/1610 [1:41:09<8:27:21, 21.70s/it]
 13%|█▎        | 208/1610 [1:41:28<8:06:10, 20.81s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8862961827361796, 'learning_rate': 8.70807453416149e-07, 'completion_length': 187.6964340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.25248410552740097, 'kl': 0.009735107421875, 'epoch': 0.65}

 13%|█▎        | 208/1610 [1:41:28<8:06:10, 20.81s/it]
 13%|█▎        | 209/1610 [1:41:47<7:49:39, 20.11s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.4053663503316258, 'learning_rate': 8.701863354037266e-07, 'completion_length': 192.00000762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357470035553, 'kl': 0.0059661865234375, 'epoch': 0.65}

 13%|█▎        | 209/1610 [1:41:47<7:49:39, 20.11s/it]
 13%|█▎        | 210/1610 [1:42:05<7:33:15, 19.43s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9429737780609919, 'learning_rate': 8.695652173913043e-07, 'completion_length': 141.87500381469727, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.14534124732017517, 'kl': 0.007232666015625, 'epoch': 0.65}

 13%|█▎        | 210/1610 [1:42:05<7:33:15, 19.43s/it]
 13%|█▎        | 211/1610 [1:42:22<7:19:13, 18.84s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8509489731020787, 'learning_rate': 8.689440993788819e-07, 'completion_length': 128.3035774230957, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.14838216453790665, 'kl': 0.005157470703125, 'epoch': 0.66}

 13%|█▎        | 211/1610 [1:42:22<7:19:13, 18.84s/it]
 13%|█▎        | 212/1610 [1:42:37<6:50:22, 17.61s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 2.2198608100943304, 'learning_rate': 8.683229813664595e-07, 'completion_length': 134.1964340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2142857238650322, 'kl': 0.0065155029296875, 'epoch': 0.66}

 13%|█▎        | 212/1610 [1:42:37<6:50:22, 17.61s/it]
 13%|█▎        | 213/1610 [1:42:51<6:25:31, 16.56s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.4722613678068574, 'learning_rate': 8.677018633540373e-07, 'completion_length': 145.5357208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266787722706795, 'kl': 0.00634765625, 'epoch': 0.66}

 13%|█▎        | 213/1610 [1:42:51<6:25:31, 16.56s/it]
 13%|█▎        | 214/1610 [1:43:08<6:27:58, 16.68s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8339652698850958, 'learning_rate': 8.670807453416149e-07, 'completion_length': 160.55358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.006927490234375, 'epoch': 0.66}

 13%|█▎        | 214/1610 [1:43:08<6:27:58, 16.68s/it]
 13%|█▎        | 215/1610 [1:43:21<6:04:44, 15.69s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.6929203094034522, 'learning_rate': 8.664596273291925e-07, 'completion_length': 146.87500762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.0070343017578125, 'epoch': 0.67}

 13%|█▎        | 215/1610 [1:43:21<6:04:44, 15.69s/it]
 13%|█▎        | 216/1610 [1:43:34<5:41:08, 14.68s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.7416809297539255, 'learning_rate': 8.658385093167702e-07, 'completion_length': 123.26786422729492, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2721000909805298, 'kl': 0.0063018798828125, 'epoch': 0.67}

 13%|█▎        | 216/1610 [1:43:34<5:41:08, 14.68s/it]
 13%|█▎        | 217/1610 [1:43:46<5:23:33, 13.94s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.1764664716544808, 'learning_rate': 8.652173913043478e-07, 'completion_length': 123.55358123779297, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.1539071872830391, 'kl': 0.0052337646484375, 'epoch': 0.67}

 13%|█▎        | 217/1610 [1:43:46<5:23:33, 13.94s/it]
 14%|█▎        | 218/1610 [1:44:01<5:35:59, 14.48s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8783683528635019, 'learning_rate': 8.645962732919254e-07, 'completion_length': 187.21429443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.14838216453790665, 'kl': 0.006805419921875, 'epoch': 0.68}

 14%|█▎        | 218/1610 [1:44:01<5:35:59, 14.48s/it]
 14%|█▎        | 219/1610 [1:44:19<5:55:59, 15.36s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8243020547288964, 'learning_rate': 8.639751552795031e-07, 'completion_length': 164.87500381469727, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.00518798828125, 'epoch': 0.68}

 14%|█▎        | 219/1610 [1:44:19<5:55:59, 15.36s/it]
 14%|█▎        | 220/1610 [1:44:36<6:11:01, 16.02s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2269257387136678, 'learning_rate': 8.633540372670807e-07, 'completion_length': 170.2857208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.3681928962469101, 'kl': 0.008880615234375, 'epoch': 0.68}

 14%|█▎        | 220/1610 [1:44:36<6:11:01, 16.02s/it]
 14%|█▎        | 221/1610 [1:44:55<6:25:40, 16.66s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.5950094027560497, 'learning_rate': 8.627329192546583e-07, 'completion_length': 176.37500762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.2253357544541359, 'kl': 0.0047760009765625, 'epoch': 0.69}

 14%|█▎        | 221/1610 [1:44:55<6:25:40, 16.66s/it]
 14%|█▍        | 222/1610 [1:45:09<6:11:25, 16.06s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 6.202006156361131, 'learning_rate': 8.621118012422361e-07, 'completion_length': 140.75000381469727, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216453790665, 'kl': 0.00560760498046875, 'epoch': 0.69}

 14%|█▍        | 222/1610 [1:45:09<6:11:25, 16.06s/it]
 14%|█▍        | 223/1610 [1:45:21<5:44:33, 14.91s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 2.3403389088751743, 'learning_rate': 8.614906832298137e-07, 'completion_length': 131.8928680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073915958405, 'kl': 0.0057373046875, 'epoch': 0.69}

 14%|█▍        | 223/1610 [1:45:21<5:44:33, 14.91s/it]
 14%|█▍        | 224/1610 [1:45:39<6:00:07, 15.59s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.0559502234178886, 'learning_rate': 8.608695652173913e-07, 'completion_length': 167.55358123779297, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.25552501529455185, 'kl': 0.0070953369140625, 'epoch': 0.7}

 14%|█▍        | 224/1610 [1:45:39<6:00:07, 15.59s/it]
 14%|█▍        | 225/1610 [1:45:54<5:55:49, 15.41s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.6356071954463898, 'learning_rate': 8.60248447204969e-07, 'completion_length': 138.41071701049805, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1428571529686451, 'kl': 0.0043487548828125, 'epoch': 0.7}

 14%|█▍        | 225/1610 [1:45:54<5:55:49, 15.41s/it]
 14%|█▍        | 226/1610 [1:46:10<6:01:46, 15.68s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8164582746164868, 'learning_rate': 8.596273291925465e-07, 'completion_length': 166.8214340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.29123930633068085, 'kl': 0.006683349609375, 'epoch': 0.7}

 14%|█▍        | 226/1610 [1:46:10<6:01:46, 15.68s/it]
 14%|█▍        | 227/1610 [1:46:28<6:18:43, 16.43s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 3.3183224479020987, 'learning_rate': 8.590062111801241e-07, 'completion_length': 207.62501525878906, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2142857238650322, 'kl': 0.0071563720703125, 'epoch': 0.7}

 14%|█▍        | 227/1610 [1:46:28<6:18:43, 16.43s/it]
 14%|█▍        | 228/1610 [1:46:44<6:13:48, 16.23s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2350310579668753, 'learning_rate': 8.583850931677018e-07, 'completion_length': 144.05357360839844, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981073170900345, 'kl': 0.00885009765625, 'epoch': 0.71}

 14%|█▍        | 228/1610 [1:46:44<6:13:48, 16.23s/it]
 14%|█▍        | 229/1610 [1:46:59<6:02:53, 15.77s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.5516570411396327, 'learning_rate': 8.577639751552794e-07, 'completion_length': 140.7678680419922, 'rewards/accuracy_reward': 0.696428582072258, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.07695358991622925, 'kl': 0.00640869140625, 'epoch': 0.71}

 14%|█▍        | 229/1610 [1:46:59<6:02:53, 15.77s/it]
 14%|█▍        | 230/1610 [1:47:11<5:42:03, 14.87s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.8099354432647952, 'learning_rate': 8.57142857142857e-07, 'completion_length': 136.89286422729492, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2721000760793686, 'kl': 0.0075531005859375, 'epoch': 0.71}

 14%|█▍        | 230/1610 [1:47:11<5:42:03, 14.87s/it]
 14%|█▍        | 231/1610 [1:47:30<6:07:17, 15.98s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.1689214244446986, 'learning_rate': 8.565217391304348e-07, 'completion_length': 172.35714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.21676981449127197, 'kl': 0.005645751953125, 'epoch': 0.72}

 14%|█▍        | 231/1610 [1:47:30<6:07:17, 15.98s/it]
 14%|█▍        | 232/1610 [1:47:45<6:00:44, 15.71s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.43193690550930436, 'learning_rate': 8.559006211180124e-07, 'completion_length': 139.26786422729492, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.005584716796875, 'epoch': 0.72}

 14%|█▍        | 232/1610 [1:47:45<6:00:44, 15.71s/it]
 14%|█▍        | 233/1610 [1:47:59<5:48:24, 15.18s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.6909987201746606, 'learning_rate': 8.5527950310559e-07, 'completion_length': 131.9285774230957, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1539071798324585, 'kl': 0.0069580078125, 'epoch': 0.72}

 14%|█▍        | 233/1610 [1:47:59<5:48:24, 15.18s/it]
 15%|█▍        | 234/1610 [1:48:13<5:41:07, 14.87s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.881266883990745, 'learning_rate': 8.546583850931677e-07, 'completion_length': 158.37500762939453, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1896214634180069, 'kl': 0.0082855224609375, 'epoch': 0.73}

 15%|█▍        | 234/1610 [1:48:13<5:41:07, 14.87s/it]
 15%|█▍        | 235/1610 [1:48:29<5:49:32, 15.25s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8782096548388313, 'learning_rate': 8.540372670807453e-07, 'completion_length': 137.12500762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2142857238650322, 'kl': 0.00677490234375, 'epoch': 0.73}

 15%|█▍        | 235/1610 [1:48:29<5:49:32, 15.25s/it]
 15%|█▍        | 236/1610 [1:48:48<6:16:36, 16.45s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7780461000738152, 'learning_rate': 8.534161490683229e-07, 'completion_length': 167.25000762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.2253357544541359, 'kl': 0.0066680908203125, 'epoch': 0.73}

 15%|█▍        | 236/1610 [1:48:48<6:16:36, 16.45s/it]
 15%|█▍        | 237/1610 [1:49:06<6:23:50, 16.77s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.325725778823914, 'learning_rate': 8.527950310559006e-07, 'completion_length': 162.30358123779297, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.11266787722706795, 'kl': 0.0087738037109375, 'epoch': 0.74}

 15%|█▍        | 237/1610 [1:49:06<6:23:50, 16.77s/it]
 15%|█▍        | 238/1610 [1:49:21<6:12:45, 16.30s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.432600280185961, 'learning_rate': 8.521739130434782e-07, 'completion_length': 150.75000762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1181928962469101, 'kl': 0.0064239501953125, 'epoch': 0.74}

 15%|█▍        | 238/1610 [1:49:21<6:12:45, 16.30s/it]
 15%|█▍        | 239/1610 [1:49:37<6:06:00, 16.02s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8097667875862216, 'learning_rate': 8.515527950310558e-07, 'completion_length': 176.7321548461914, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.0064697265625, 'epoch': 0.74}

 15%|█▍        | 239/1610 [1:49:37<6:06:00, 16.02s/it]
 15%|█▍        | 240/1610 [1:49:52<6:03:25, 15.92s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 3.6209724954111637, 'learning_rate': 8.509316770186336e-07, 'completion_length': 140.7857208251953, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.27260690927505493, 'kl': 0.0065765380859375, 'epoch': 0.75}

 15%|█▍        | 240/1610 [1:49:52<6:03:25, 15.92s/it]
 15%|█▍        | 241/1610 [1:50:11<6:19:29, 16.63s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.9878618598466304, 'learning_rate': 8.503105590062112e-07, 'completion_length': 172.75000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.26657508313655853, 'kl': 0.0070953369140625, 'epoch': 0.75}

 15%|█▍        | 241/1610 [1:50:11<6:19:29, 16.63s/it]
 15%|█▌        | 242/1610 [1:50:25<6:03:50, 15.96s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.6815299170723055, 'learning_rate': 8.496894409937888e-07, 'completion_length': 148.73214721679688, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.23086079210042953, 'kl': 0.0057373046875, 'epoch': 0.75}

 15%|█▌        | 242/1610 [1:50:25<6:03:50, 15.96s/it]
 15%|█▌        | 243/1610 [1:50:42<6:11:49, 16.32s/it]
                                                      
{'loss': 0.0018, 'grad_norm': 1.7519469079467531, 'learning_rate': 8.490683229813665e-07, 'completion_length': 162.4821548461914, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178572535514832, 'reward_std': 0.2610500305891037, 'kl': 0.04595947265625, 'epoch': 0.75}

 15%|█▌        | 243/1610 [1:50:42<6:11:49, 16.32s/it]
 15%|█▌        | 244/1610 [1:51:03<6:39:57, 17.57s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.8439489553257467, 'learning_rate': 8.484472049689441e-07, 'completion_length': 148.98214721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5000000596046448, 'reward_std': 0.34403547644615173, 'kl': 0.006561279296875, 'epoch': 0.76}

 15%|█▌        | 244/1610 [1:51:03<6:39:57, 17.57s/it]
 15%|█▌        | 245/1610 [1:51:17<6:20:19, 16.72s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.9669761102818065, 'learning_rate': 8.478260869565217e-07, 'completion_length': 167.41072845458984, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.2610500380396843, 'kl': 0.006011962890625, 'epoch': 0.76}

 15%|█▌        | 245/1610 [1:51:17<6:20:19, 16.72s/it]
 15%|█▌        | 246/1610 [1:51:34<6:18:37, 16.65s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.5401435291349019, 'learning_rate': 8.472049689440994e-07, 'completion_length': 155.12500762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.00400543212890625, 'epoch': 0.76}

 15%|█▌        | 246/1610 [1:51:34<6:18:37, 16.65s/it]
 15%|█▌        | 247/1610 [1:51:51<6:20:57, 16.77s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.0172642107690077, 'learning_rate': 8.46583850931677e-07, 'completion_length': 170.96429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981074661016464, 'kl': 0.0052947998046875, 'epoch': 0.77}

 15%|█▌        | 247/1610 [1:51:51<6:20:57, 16.77s/it]
 15%|█▌        | 248/1610 [1:52:05<6:00:59, 15.90s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.0873724383892356, 'learning_rate': 8.459627329192546e-07, 'completion_length': 153.33929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.26657505333423615, 'kl': 0.0072479248046875, 'epoch': 0.77}

 15%|█▌        | 248/1610 [1:52:05<6:00:59, 15.90s/it]
 15%|█▌        | 249/1610 [1:52:21<6:00:52, 15.91s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9142065258810711, 'learning_rate': 8.453416149068324e-07, 'completion_length': 173.96429443359375, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2967643141746521, 'kl': 0.008880615234375, 'epoch': 0.77}

 15%|█▌        | 249/1610 [1:52:21<6:00:52, 15.91s/it]
 16%|█▌        | 250/1610 [1:52:36<5:53:45, 15.61s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8767854388882079, 'learning_rate': 8.447204968944099e-07, 'completion_length': 139.4285774230957, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178571939468384, 'reward_std': 0.2610500380396843, 'kl': 0.0049896240234375, 'epoch': 0.78}

 16%|█▌        | 250/1610 [1:52:36<5:53:45, 15.61s/it]
 16%|█▌        | 251/1610 [1:52:51<5:51:01, 15.50s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 2.0325559056445357, 'learning_rate': 8.440993788819875e-07, 'completion_length': 143.48214721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1539071798324585, 'kl': 0.00579833984375, 'epoch': 0.78}

 16%|█▌        | 251/1610 [1:52:51<5:51:01, 15.50s/it]
 16%|█▌        | 252/1610 [1:53:10<6:14:01, 16.53s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.6127740586625066, 'learning_rate': 8.434782608695652e-07, 'completion_length': 172.51786041259766, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.007537841796875, 'epoch': 0.78}

 16%|█▌        | 252/1610 [1:53:10<6:14:01, 16.53s/it]
 16%|█▌        | 253/1610 [1:53:25<6:06:26, 16.20s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.806349987505162, 'learning_rate': 8.428571428571428e-07, 'completion_length': 139.98215103149414, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714365541935, 'kl': 0.00388336181640625, 'epoch': 0.79}

 16%|█▌        | 253/1610 [1:53:25<6:06:26, 16.20s/it]
 16%|█▌        | 254/1610 [1:53:42<6:10:37, 16.40s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.5885267487010364, 'learning_rate': 8.422360248447204e-07, 'completion_length': 142.83929443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1428571529686451, 'kl': 0.006317138671875, 'epoch': 0.79}

 16%|█▌        | 254/1610 [1:53:42<6:10:37, 16.40s/it]
 16%|█▌        | 255/1610 [1:53:57<5:57:33, 15.83s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.763567556388913, 'learning_rate': 8.416149068322981e-07, 'completion_length': 150.35714721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428656578064, 'kl': 0.00677490234375, 'epoch': 0.79}

 16%|█▌        | 255/1610 [1:53:57<5:57:33, 15.83s/it]
 16%|█▌        | 256/1610 [1:54:12<5:56:20, 15.79s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.2844609902827286, 'learning_rate': 8.409937888198757e-07, 'completion_length': 152.50000762939453, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.25552502274513245, 'kl': 0.0061492919921875, 'epoch': 0.8}

 16%|█▌        | 256/1610 [1:54:12<5:56:20, 15.79s/it]
 16%|█▌        | 257/1610 [1:54:29<6:01:22, 16.03s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 3.614509331331854, 'learning_rate': 8.403726708074533e-07, 'completion_length': 170.42857360839844, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.21981073170900345, 'kl': 0.006134033203125, 'epoch': 0.8}

 16%|█▌        | 257/1610 [1:54:29<6:01:22, 16.03s/it]
 16%|█▌        | 258/1610 [1:54:47<6:13:09, 16.56s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.7928998361498643, 'learning_rate': 8.397515527950311e-07, 'completion_length': 148.1428680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.30228933691978455, 'kl': 0.0070953369140625, 'epoch': 0.8}

 16%|█▌        | 258/1610 [1:54:47<6:13:09, 16.56s/it]
 16%|█▌        | 259/1610 [1:55:05<6:25:22, 17.11s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.916874930686291, 'learning_rate': 8.391304347826087e-07, 'completion_length': 193.50000762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.2524840980768204, 'kl': 0.0069580078125, 'epoch': 0.8}

 16%|█▌        | 259/1610 [1:55:05<6:25:22, 17.11s/it]
 16%|█▌        | 260/1610 [1:55:23<6:28:32, 17.27s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 2.542717096578457, 'learning_rate': 8.385093167701863e-07, 'completion_length': 209.2321548461914, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.14838216826319695, 'kl': 0.0074615478515625, 'epoch': 0.81}

 16%|█▌        | 260/1610 [1:55:23<6:28:32, 17.27s/it]
 16%|█▌        | 261/1610 [1:55:38<6:18:37, 16.84s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9905657636204794, 'learning_rate': 8.37888198757764e-07, 'completion_length': 174.46429443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2500000074505806, 'kl': 0.0076751708984375, 'epoch': 0.81}

 16%|█▌        | 261/1610 [1:55:38<6:18:37, 16.84s/it]
 16%|█▋        | 262/1610 [1:55:53<5:59:48, 16.02s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.8827079004427368, 'learning_rate': 8.372670807453416e-07, 'completion_length': 140.03571701049805, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1071428619325161, 'kl': 0.0058746337890625, 'epoch': 0.81}

 16%|█▋        | 262/1610 [1:55:53<5:59:48, 16.02s/it]
 16%|█▋        | 263/1610 [1:56:06<5:43:30, 15.30s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.14323568441883, 'learning_rate': 8.366459627329192e-07, 'completion_length': 127.12500762939453, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.2253357619047165, 'kl': 0.0067901611328125, 'epoch': 0.82}

 16%|█▋        | 263/1610 [1:56:06<5:43:30, 15.30s/it]
 16%|█▋        | 264/1610 [1:56:22<5:45:20, 15.39s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.5265707394969117, 'learning_rate': 8.360248447204969e-07, 'completion_length': 158.01786041259766, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.007293701171875, 'epoch': 0.82}

 16%|█▋        | 264/1610 [1:56:22<5:45:20, 15.39s/it]
 16%|█▋        | 265/1610 [1:56:38<5:47:00, 15.48s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.7089716392876364, 'learning_rate': 8.354037267080745e-07, 'completion_length': 152.51786422729492, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.0062408447265625, 'epoch': 0.82}

 16%|█▋        | 265/1610 [1:56:38<5:47:00, 15.48s/it]
 17%|█▋        | 266/1610 [1:56:55<6:01:00, 16.12s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.8151191681831105, 'learning_rate': 8.347826086956521e-07, 'completion_length': 169.4107208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1785714402794838, 'kl': 0.0066375732421875, 'epoch': 0.83}

 17%|█▋        | 266/1610 [1:56:55<6:01:00, 16.12s/it]
 17%|█▋        | 267/1610 [1:57:11<6:02:33, 16.20s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.610330431749205, 'learning_rate': 8.341614906832299e-07, 'completion_length': 167.25000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.15943220257759094, 'kl': 0.007110595703125, 'epoch': 0.83}

 17%|█▋        | 267/1610 [1:57:11<6:02:33, 16.20s/it]
 17%|█▋        | 268/1610 [1:57:27<6:00:36, 16.12s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.203927447896029, 'learning_rate': 8.335403726708075e-07, 'completion_length': 133.00000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7678572535514832, 'reward_std': 0.1896214708685875, 'kl': 0.0054168701171875, 'epoch': 0.83}

 17%|█▋        | 268/1610 [1:57:27<6:00:36, 16.12s/it]
 17%|█▋        | 269/1610 [1:57:43<5:54:03, 15.84s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.9116957184197545, 'learning_rate': 8.329192546583851e-07, 'completion_length': 143.50000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.19514648616313934, 'kl': 0.005615234375, 'epoch': 0.84}

 17%|█▋        | 269/1610 [1:57:43<5:54:03, 15.84s/it]
 17%|█▋        | 270/1610 [1:57:58<5:47:28, 15.56s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.7643141220771091, 'learning_rate': 8.322981366459628e-07, 'completion_length': 149.71429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.07695358991622925, 'kl': 0.0060272216796875, 'epoch': 0.84}

 17%|█▋        | 270/1610 [1:57:58<5:47:28, 15.56s/it]
 17%|█▋        | 271/1610 [1:58:14<5:50:44, 15.72s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.271527438928348, 'learning_rate': 8.316770186335404e-07, 'completion_length': 165.96429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214708685875, 'kl': 0.005157470703125, 'epoch': 0.84}

 17%|█▋        | 271/1610 [1:58:14<5:50:44, 15.72s/it]
 17%|█▋        | 272/1610 [1:58:30<5:55:46, 15.95s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.8584297038770599, 'learning_rate': 8.31055900621118e-07, 'completion_length': 140.64286041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.23086076974868774, 'kl': 0.0070648193359375, 'epoch': 0.84}

 17%|█▋        | 272/1610 [1:58:30<5:55:46, 15.95s/it]
 17%|█▋        | 273/1610 [1:58:46<5:52:09, 15.80s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7282416425477006, 'learning_rate': 8.304347826086955e-07, 'completion_length': 156.6607208251953, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.0063018798828125, 'epoch': 0.85}

 17%|█▋        | 273/1610 [1:58:46<5:52:09, 15.80s/it]
 17%|█▋        | 274/1610 [1:59:00<5:45:59, 15.54s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.0467979890591192, 'learning_rate': 8.298136645962732e-07, 'completion_length': 134.4285774230957, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.005767822265625, 'epoch': 0.85}

 17%|█▋        | 274/1610 [1:59:00<5:45:59, 15.54s/it]
 17%|█▋        | 275/1610 [1:59:15<5:38:13, 15.20s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.6530596106173663, 'learning_rate': 8.291925465838508e-07, 'completion_length': 136.25000762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.25552501529455185, 'kl': 0.0055084228515625, 'epoch': 0.85}

 17%|█▋        | 275/1610 [1:59:15<5:38:13, 15.20s/it]
 17%|█▋        | 276/1610 [1:59:29<5:30:02, 14.84s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8462327130431142, 'learning_rate': 8.285714285714285e-07, 'completion_length': 138.9107208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.23086077719926834, 'kl': 0.00677490234375, 'epoch': 0.86}

 17%|█▋        | 276/1610 [1:59:29<5:30:02, 14.84s/it]
 17%|█▋        | 277/1610 [1:59:45<5:41:07, 15.35s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.871005673104152, 'learning_rate': 8.279503105590062e-07, 'completion_length': 138.6071548461914, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.32695358991622925, 'kl': 0.00653076171875, 'epoch': 0.86}

 17%|█▋        | 277/1610 [1:59:45<5:41:07, 15.35s/it]
 17%|█▋        | 278/1610 [2:00:03<5:54:29, 15.97s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.5279651612890627, 'learning_rate': 8.273291925465838e-07, 'completion_length': 207.08929443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.18409644067287445, 'kl': 0.008087158203125, 'epoch': 0.86}

 17%|█▋        | 278/1610 [2:00:03<5:54:29, 15.97s/it]
 17%|█▋        | 279/1610 [2:00:19<5:52:31, 15.89s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.0183906883554452, 'learning_rate': 8.267080745341614e-07, 'completion_length': 165.14286041259766, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.26657506823539734, 'kl': 0.006103515625, 'epoch': 0.87}

 17%|█▋        | 279/1610 [2:00:19<5:52:31, 15.89s/it]
 17%|█▋        | 280/1610 [2:00:35<5:54:49, 16.01s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.4949825593394867, 'learning_rate': 8.260869565217391e-07, 'completion_length': 166.03572845458984, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142858505249023, 'reward_std': 0.17553051561117172, 'kl': 0.005889892578125, 'epoch': 0.87}

 17%|█▋        | 280/1610 [2:00:35<5:54:49, 16.01s/it]
 17%|█▋        | 281/1610 [2:00:55<6:24:05, 17.34s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.974071385952314, 'learning_rate': 8.254658385093167e-07, 'completion_length': 188.05358123779297, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4464285969734192, 'reward_std': 0.29123931378126144, 'kl': 0.0069427490234375, 'epoch': 0.87}

 17%|█▋        | 281/1610 [2:00:55<6:24:05, 17.34s/it]
 18%|█▊        | 282/1610 [2:01:12<6:18:05, 17.08s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8527605121865527, 'learning_rate': 8.248447204968943e-07, 'completion_length': 162.50000762939453, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.006866455078125, 'epoch': 0.88}

 18%|█▊        | 282/1610 [2:01:12<6:18:05, 17.08s/it]
 18%|█▊        | 283/1610 [2:01:27<6:07:28, 16.62s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 2.150601285352237, 'learning_rate': 8.24223602484472e-07, 'completion_length': 141.9107208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25552502274513245, 'kl': 0.0077972412109375, 'epoch': 0.88}

 18%|█▊        | 283/1610 [2:01:27<6:07:28, 16.62s/it]
 18%|█▊        | 284/1610 [2:01:43<6:00:02, 16.29s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9879888454150236, 'learning_rate': 8.236024844720496e-07, 'completion_length': 144.55358123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.0073394775390625, 'epoch': 0.88}

 18%|█▊        | 284/1610 [2:01:43<6:00:02, 16.29s/it]
 18%|█▊        | 285/1610 [2:01:59<5:58:25, 16.23s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.0441067597239597, 'learning_rate': 8.229813664596273e-07, 'completion_length': 169.80358123779297, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.26657506078481674, 'kl': 0.0073699951171875, 'epoch': 0.89}

 18%|█▊        | 285/1610 [2:01:59<5:58:25, 16.23s/it]
 18%|█▊        | 286/1610 [2:02:14<5:50:55, 15.90s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 4.565614946166378, 'learning_rate': 8.22360248447205e-07, 'completion_length': 156.48214721679688, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.14838216826319695, 'kl': 0.0111236572265625, 'epoch': 0.89}

 18%|█▊        | 286/1610 [2:02:14<5:50:55, 15.90s/it]
 18%|█▊        | 287/1610 [2:02:30<5:54:02, 16.06s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.0333462387995973, 'learning_rate': 8.217391304347826e-07, 'completion_length': 165.85714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214708685875, 'kl': 0.0078582763671875, 'epoch': 0.89}

 18%|█▊        | 287/1610 [2:02:30<5:54:02, 16.06s/it]
 18%|█▊        | 288/1610 [2:02:47<5:59:38, 16.32s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.404322569665495, 'learning_rate': 8.211180124223602e-07, 'completion_length': 183.0357208251953, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.25552502274513245, 'kl': 0.008514404296875, 'epoch': 0.89}

 18%|█▊        | 288/1610 [2:02:47<5:59:38, 16.32s/it]
 18%|█▊        | 289/1610 [2:03:02<5:45:26, 15.69s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.1809637768787935, 'learning_rate': 8.204968944099379e-07, 'completion_length': 171.5714340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.30228936672210693, 'kl': 0.0075836181640625, 'epoch': 0.9}

 18%|█▊        | 289/1610 [2:03:02<5:45:26, 15.69s/it]
 18%|█▊        | 290/1610 [2:03:17<5:45:16, 15.69s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.4086532369054539, 'learning_rate': 8.198757763975155e-07, 'completion_length': 165.92857360839844, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.3078143820166588, 'kl': 0.007080078125, 'epoch': 0.9}

 18%|█▊        | 290/1610 [2:03:17<5:45:16, 15.69s/it]
 18%|█▊        | 291/1610 [2:03:32<5:35:21, 15.26s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7192156307894616, 'learning_rate': 8.192546583850931e-07, 'completion_length': 153.71428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714328289032, 'kl': 0.006744384765625, 'epoch': 0.9}

 18%|█▊        | 291/1610 [2:03:32<5:35:21, 15.26s/it]
 18%|█▊        | 292/1610 [2:03:45<5:24:03, 14.75s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9087057615961025, 'learning_rate': 8.186335403726708e-07, 'completion_length': 152.64286041259766, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357619047165, 'kl': 0.0079345703125, 'epoch': 0.91}

 18%|█▊        | 292/1610 [2:03:45<5:24:03, 14.75s/it]
 18%|█▊        | 293/1610 [2:04:00<5:26:11, 14.86s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8886031284835493, 'learning_rate': 8.180124223602484e-07, 'completion_length': 151.71428680419922, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.15943220257759094, 'kl': 0.0070648193359375, 'epoch': 0.91}

 18%|█▊        | 293/1610 [2:04:00<5:26:11, 14.86s/it]
 18%|█▊        | 294/1610 [2:04:16<5:32:52, 15.18s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 1.086212243174825, 'learning_rate': 8.173913043478261e-07, 'completion_length': 147.0357208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.0050048828125, 'epoch': 0.91}

 18%|█▊        | 294/1610 [2:04:16<5:32:52, 15.18s/it]
 18%|█▊        | 295/1610 [2:04:33<5:46:21, 15.80s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.7559535863338827, 'learning_rate': 8.167701863354038e-07, 'completion_length': 183.87500762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2610500454902649, 'kl': 0.006866455078125, 'epoch': 0.92}

 18%|█▊        | 295/1610 [2:04:33<5:46:21, 15.80s/it]
 18%|█▊        | 296/1610 [2:04:50<5:53:18, 16.13s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9372172379009277, 'learning_rate': 8.161490683229814e-07, 'completion_length': 169.39286041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.007354736328125, 'epoch': 0.92}

 18%|█▊        | 296/1610 [2:04:50<5:53:18, 16.13s/it]
 18%|█▊        | 297/1610 [2:05:09<6:07:38, 16.80s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7881650695598189, 'learning_rate': 8.155279503105589e-07, 'completion_length': 184.00000762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266788095235825, 'kl': 0.0076904296875, 'epoch': 0.92}

 18%|█▊        | 297/1610 [2:05:09<6:07:38, 16.80s/it]
 19%|█▊        | 298/1610 [2:05:23<5:50:14, 16.02s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.2289740537076808, 'learning_rate': 8.149068322981366e-07, 'completion_length': 155.1428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.007293701171875, 'epoch': 0.93}

 19%|█▊        | 298/1610 [2:05:23<5:50:14, 16.02s/it]
 19%|█▊        | 299/1610 [2:05:43<6:14:55, 17.16s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9917335632126938, 'learning_rate': 8.142857142857142e-07, 'completion_length': 202.71429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.30228935927152634, 'kl': 0.0073394775390625, 'epoch': 0.93}

 19%|█▊        | 299/1610 [2:05:43<6:14:55, 17.16s/it]
 19%|█▊        | 300/1610 [2:05:59<6:11:23, 17.01s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.171432417474164, 'learning_rate': 8.136645962732918e-07, 'completion_length': 164.50000762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5357143878936768, 'reward_std': 0.2967643216252327, 'kl': 0.0076904296875, 'epoch': 0.93}

 19%|█▊        | 300/1610 [2:05:59<6:11:23, 17.01s/it]
 19%|█▊        | 301/1610 [2:10:04<31:03:13, 85.40s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.8423867923115669, 'learning_rate': 8.130434782608695e-07, 'completion_length': 132.5357208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1539071798324585, 'kl': 0.007781982421875, 'epoch': 0.93}

 19%|█▊        | 301/1610 [2:10:04<31:03:13, 85.40s/it]
 19%|█▉        | 302/1610 [2:10:22<23:36:04, 64.96s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 1.1463758605169656, 'learning_rate': 8.124223602484471e-07, 'completion_length': 172.1964340209961, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3188644051551819, 'kl': 0.00897216796875, 'epoch': 0.94}

 19%|█▉        | 302/1610 [2:10:22<23:36:04, 64.96s/it]
 19%|█▉        | 303/1610 [2:10:40<18:29:14, 50.92s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.5371537900627268, 'learning_rate': 8.118012422360247e-07, 'completion_length': 193.96429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.14534124732017517, 'kl': 0.00787353515625, 'epoch': 0.94}

 19%|█▉        | 303/1610 [2:10:40<18:29:14, 50.92s/it]
 19%|█▉        | 304/1610 [2:10:56<14:39:04, 40.39s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.6751893856377422, 'learning_rate': 8.111801242236025e-07, 'completion_length': 160.12500762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.0357142873108387, 'kl': 0.00799560546875, 'epoch': 0.94}

 19%|█▉        | 304/1610 [2:10:56<14:39:04, 40.39s/it]
 19%|█▉        | 305/1610 [2:11:13<12:08:08, 33.48s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 1.0810155756922895, 'learning_rate': 8.105590062111801e-07, 'completion_length': 165.1607208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23086076974868774, 'kl': 0.00732421875, 'epoch': 0.95}

 19%|█▉        | 305/1610 [2:11:13<12:08:08, 33.48s/it]
 19%|█▉        | 306/1610 [2:11:28<10:04:25, 27.81s/it]
                                                       
{'loss': 0.0003, 'grad_norm': 0.2765809751169129, 'learning_rate': 8.099378881987577e-07, 'completion_length': 149.60715103149414, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.04123930633068085, 'kl': 0.0063934326171875, 'epoch': 0.95}

 19%|█▉        | 306/1610 [2:11:28<10:04:25, 27.81s/it]
 19%|█▉        | 307/1610 [2:11:44<8:51:01, 24.45s/it] 
                                                      
{'loss': 0.0003, 'grad_norm': 0.9366165811620077, 'learning_rate': 8.093167701863354e-07, 'completion_length': 167.8571548461914, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.24241763353347778, 'kl': 0.0066680908203125, 'epoch': 0.95}

 19%|█▉        | 307/1610 [2:11:44<8:51:01, 24.45s/it]
 19%|█▉        | 308/1610 [2:12:00<7:53:44, 21.83s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.6929700161915497, 'learning_rate': 8.08695652173913e-07, 'completion_length': 158.62500762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.15943220257759094, 'kl': 0.0059967041015625, 'epoch': 0.96}

 19%|█▉        | 308/1610 [2:12:00<7:53:44, 21.83s/it]
 19%|█▉        | 309/1610 [2:12:14<7:00:11, 19.38s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.9581296729587641, 'learning_rate': 8.080745341614906e-07, 'completion_length': 144.80358123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.19514648616313934, 'kl': 0.0073699951171875, 'epoch': 0.96}

 19%|█▉        | 309/1610 [2:12:14<7:00:11, 19.38s/it]
 19%|█▉        | 310/1610 [2:12:29<6:33:55, 18.18s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2945950830850743, 'learning_rate': 8.074534161490683e-07, 'completion_length': 162.12500762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2142857238650322, 'kl': 0.0087890625, 'epoch': 0.96}

 19%|█▉        | 310/1610 [2:12:29<6:33:55, 18.18s/it]
 19%|█▉        | 311/1610 [2:12:43<6:03:53, 16.81s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.716324890219927, 'learning_rate': 8.068322981366459e-07, 'completion_length': 145.14286041259766, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.19514648616313934, 'kl': 0.0074462890625, 'epoch': 0.97}

 19%|█▉        | 311/1610 [2:12:43<6:03:53, 16.81s/it]
 19%|█▉        | 312/1610 [2:13:03<6:28:35, 17.96s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 3.653945541361069, 'learning_rate': 8.062111801242235e-07, 'completion_length': 221.42858123779297, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5000000596046448, 'reward_std': 0.3681929111480713, 'kl': 0.0098876953125, 'epoch': 0.97}

 19%|█▉        | 312/1610 [2:13:03<6:28:35, 17.96s/it]
 19%|█▉        | 313/1610 [2:13:20<6:22:18, 17.69s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9583863972495321, 'learning_rate': 8.055900621118013e-07, 'completion_length': 158.3214340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.26657506078481674, 'kl': 0.010406494140625, 'epoch': 0.97}

 19%|█▉        | 313/1610 [2:13:20<6:22:18, 17.69s/it]
 20%|█▉        | 314/1610 [2:13:35<6:05:09, 16.91s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0030775923923392, 'learning_rate': 8.049689440993789e-07, 'completion_length': 158.0357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.008880615234375, 'epoch': 0.98}

 20%|█▉        | 314/1610 [2:13:35<6:05:09, 16.91s/it]
 20%|█▉        | 315/1610 [2:13:52<6:06:33, 16.98s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 2.5570317932406237, 'learning_rate': 8.043478260869565e-07, 'completion_length': 163.71429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.33800363540649414, 'kl': 0.009246826171875, 'epoch': 0.98}

 20%|█▉        | 315/1610 [2:13:52<6:06:33, 16.98s/it]
 20%|█▉        | 316/1610 [2:14:07<5:52:43, 16.35s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7267971574613259, 'learning_rate': 8.037267080745342e-07, 'completion_length': 162.9464340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.14838216826319695, 'kl': 0.010009765625, 'epoch': 0.98}

 20%|█▉        | 316/1610 [2:14:07<5:52:43, 16.35s/it]
 20%|█▉        | 317/1610 [2:14:26<6:07:50, 17.07s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.978127916034366, 'learning_rate': 8.031055900621118e-07, 'completion_length': 190.55358123779297, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6250001192092896, 'reward_std': 0.2500000149011612, 'kl': 0.009307861328125, 'epoch': 0.98}

 20%|█▉        | 317/1610 [2:14:26<6:07:50, 17.07s/it]
 20%|█▉        | 318/1610 [2:14:43<6:09:30, 17.16s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0054841835365393, 'learning_rate': 8.024844720496894e-07, 'completion_length': 155.33929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357619047165, 'kl': 0.00933837890625, 'epoch': 0.99}

 20%|█▉        | 318/1610 [2:14:43<6:09:30, 17.16s/it]
 20%|█▉        | 319/1610 [2:14:58<5:54:58, 16.50s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8596110850099483, 'learning_rate': 8.018633540372671e-07, 'completion_length': 137.8214340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.19514650478959084, 'kl': 0.008453369140625, 'epoch': 0.99}

 20%|█▉        | 319/1610 [2:14:58<5:54:58, 16.50s/it]
 20%|█▉        | 320/1610 [2:15:15<5:55:40, 16.54s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.6498195328154643, 'learning_rate': 8.012422360248446e-07, 'completion_length': 148.80358123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.23086077719926834, 'kl': 0.0077972412109375, 'epoch': 0.99}

 20%|█▉        | 320/1610 [2:15:15<5:55:40, 16.54s/it]
 20%|█▉        | 321/1610 [2:15:34<6:08:15, 17.14s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.6302462519929175, 'learning_rate': 8.006211180124222e-07, 'completion_length': 181.87500762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357619047165, 'kl': 0.00830078125, 'epoch': 1.0}

 20%|█▉        | 321/1610 [2:15:34<6:08:15, 17.14s/it]
 20%|██        | 322/1610 [2:15:50<6:03:41, 16.94s/it]
                                                      
{'loss': 0.0002, 'grad_norm': 0.6660372447257961, 'learning_rate': 8e-07, 'completion_length': 151.10714721679688, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.00579833984375, 'epoch': 1.0}

 20%|██        | 322/1610 [2:15:50<6:03:41, 16.94s/it]
 20%|██        | 323/1610 [2:16:08<6:06:38, 17.09s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.441905887820762, 'learning_rate': 7.993788819875776e-07, 'completion_length': 166.01786041259766, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.007476806640625, 'epoch': 1.0}

 20%|██        | 323/1610 [2:16:08<6:06:38, 17.09s/it]
 20%|██        | 324/1610 [2:16:25<6:10:59, 17.31s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.6054995971933042, 'learning_rate': 7.987577639751552e-07, 'completion_length': 187.30357360839844, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.01031494140625, 'epoch': 1.01}

 20%|██        | 324/1610 [2:16:25<6:10:59, 17.31s/it]
 20%|██        | 325/1610 [2:16:44<6:21:49, 17.83s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.2689959155433237, 'learning_rate': 7.981366459627329e-07, 'completion_length': 172.17858123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428619325161, 'kl': 0.008544921875, 'epoch': 1.01}

 20%|██        | 325/1610 [2:16:44<6:21:49, 17.83s/it]
 20%|██        | 326/1610 [2:17:01<6:14:16, 17.49s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7014872629831966, 'learning_rate': 7.975155279503105e-07, 'completion_length': 169.82144165039062, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.1428571529686451, 'kl': 0.009307861328125, 'epoch': 1.01}

 20%|██        | 326/1610 [2:17:01<6:14:16, 17.49s/it]
 20%|██        | 327/1610 [2:17:17<6:04:25, 17.04s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.6022486587114432, 'learning_rate': 7.968944099378881e-07, 'completion_length': 151.92858123779297, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0714285746216774, 'kl': 0.007232666015625, 'epoch': 1.02}

 20%|██        | 327/1610 [2:17:17<6:04:25, 17.04s/it]
 20%|██        | 328/1610 [2:17:32<5:52:33, 16.50s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7734560676683051, 'learning_rate': 7.962732919254658e-07, 'completion_length': 177.62500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1539071798324585, 'kl': 0.007781982421875, 'epoch': 1.02}

 20%|██        | 328/1610 [2:17:32<5:52:33, 16.50s/it]
 20%|██        | 329/1610 [2:17:52<6:12:42, 17.46s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9425785071601653, 'learning_rate': 7.956521739130434e-07, 'completion_length': 193.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.607142984867096, 'reward_std': 0.25552503019571304, 'kl': 0.0096435546875, 'epoch': 1.02}

 20%|██        | 329/1610 [2:17:52<6:12:42, 17.46s/it]
 20%|██        | 330/1610 [2:18:10<6:15:30, 17.60s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7254732390297693, 'learning_rate': 7.95031055900621e-07, 'completion_length': 196.3928680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2610500380396843, 'kl': 0.007110595703125, 'epoch': 1.02}

 20%|██        | 330/1610 [2:18:10<6:15:30, 17.60s/it]
 21%|██        | 331/1610 [2:18:26<6:04:29, 17.10s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7776346092834138, 'learning_rate': 7.944099378881988e-07, 'completion_length': 158.67858123779297, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1539071872830391, 'kl': 0.009429931640625, 'epoch': 1.03}

 21%|██        | 331/1610 [2:18:26<6:04:29, 17.10s/it]
 21%|██        | 332/1610 [2:18:44<6:12:25, 17.48s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0055819366957226, 'learning_rate': 7.937888198757764e-07, 'completion_length': 171.48214721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.3435286581516266, 'kl': 0.00982666015625, 'epoch': 1.03}

 21%|██        | 332/1610 [2:18:44<6:12:25, 17.48s/it]
 21%|██        | 333/1610 [2:19:00<6:00:50, 16.95s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.8328317152842958, 'learning_rate': 7.93167701863354e-07, 'completion_length': 152.3571548461914, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.008575439453125, 'epoch': 1.03}

 21%|██        | 333/1610 [2:19:00<6:00:50, 16.95s/it]
 21%|██        | 334/1610 [2:19:17<6:00:30, 16.95s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.684087900097813, 'learning_rate': 7.925465838509317e-07, 'completion_length': 176.6607208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2253357656300068, 'kl': 0.0066070556640625, 'epoch': 1.04}

 21%|██        | 334/1610 [2:19:17<6:00:30, 16.95s/it]
 21%|██        | 335/1610 [2:19:31<5:42:03, 16.10s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0684553807841564, 'learning_rate': 7.919254658385093e-07, 'completion_length': 139.9464340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266787722706795, 'kl': 0.0103759765625, 'epoch': 1.04}

 21%|██        | 335/1610 [2:19:31<5:42:03, 16.10s/it]
 21%|██        | 336/1610 [2:19:45<5:30:47, 15.58s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7696364426451506, 'learning_rate': 7.913043478260869e-07, 'completion_length': 137.05358123779297, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.1539071872830391, 'kl': 0.009674072265625, 'epoch': 1.04}

 21%|██        | 336/1610 [2:19:45<5:30:47, 15.58s/it]
 21%|██        | 337/1610 [2:20:01<5:33:10, 15.70s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.898750747013499, 'learning_rate': 7.906832298136646e-07, 'completion_length': 153.0714340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.00726318359375, 'epoch': 1.05}

 21%|██        | 337/1610 [2:20:01<5:33:10, 15.70s/it]
 21%|██        | 338/1610 [2:20:21<5:55:55, 16.79s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7645983665277204, 'learning_rate': 7.900621118012422e-07, 'completion_length': 205.71429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.3078143745660782, 'kl': 0.009979248046875, 'epoch': 1.05}

 21%|██        | 338/1610 [2:20:21<5:55:55, 16.79s/it]
 21%|██        | 339/1610 [2:20:37<5:55:38, 16.79s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.4128175714548554, 'learning_rate': 7.894409937888198e-07, 'completion_length': 190.21429443359375, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.19514649361371994, 'kl': 0.01025390625, 'epoch': 1.05}

 21%|██        | 339/1610 [2:20:37<5:55:38, 16.79s/it]
 21%|██        | 340/1610 [2:20:56<6:04:03, 17.20s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.366066827492104, 'learning_rate': 7.888198757763976e-07, 'completion_length': 160.39286041259766, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.3193712458014488, 'kl': 0.013824462890625, 'epoch': 1.06}

 21%|██        | 340/1610 [2:20:56<6:04:03, 17.20s/it]
 21%|██        | 341/1610 [2:21:14<6:12:11, 17.60s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.5623055183600476, 'learning_rate': 7.881987577639752e-07, 'completion_length': 167.2678680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.2500000186264515, 'kl': 0.010406494140625, 'epoch': 1.06}

 21%|██        | 341/1610 [2:21:14<6:12:11, 17.60s/it]
 21%|██        | 342/1610 [2:21:29<5:54:27, 16.77s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8694356136835888, 'learning_rate': 7.875776397515528e-07, 'completion_length': 171.8928680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657506078481674, 'kl': 0.01239013671875, 'epoch': 1.06}

 21%|██        | 342/1610 [2:21:29<5:54:27, 16.77s/it]
 21%|██▏       | 343/1610 [2:21:47<6:01:33, 17.12s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.096144313551489, 'learning_rate': 7.869565217391305e-07, 'completion_length': 166.01786041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.23086078464984894, 'kl': 0.0080413818359375, 'epoch': 1.07}

 21%|██▏       | 343/1610 [2:21:47<6:01:33, 17.12s/it]
 21%|██▏       | 344/1610 [2:22:06<6:10:40, 17.57s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 2.0570352467614708, 'learning_rate': 7.86335403726708e-07, 'completion_length': 158.1428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.2142857238650322, 'kl': 0.008087158203125, 'epoch': 1.07}

 21%|██▏       | 344/1610 [2:22:06<6:10:40, 17.57s/it]
 21%|██▏       | 345/1610 [2:22:22<6:03:09, 17.23s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7374117251057398, 'learning_rate': 7.857142857142856e-07, 'completion_length': 161.5178680419922, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428619325161, 'kl': 0.00970458984375, 'epoch': 1.07}

 21%|██▏       | 345/1610 [2:22:22<6:03:09, 17.23s/it]
 21%|██▏       | 346/1610 [2:22:38<5:54:48, 16.84s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2269582741872789, 'learning_rate': 7.850931677018633e-07, 'completion_length': 171.83929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.1896214708685875, 'kl': 0.014801025390625, 'epoch': 1.07}

 21%|██▏       | 346/1610 [2:22:38<5:54:48, 16.84s/it]
 22%|██▏       | 347/1610 [2:22:52<5:35:54, 15.96s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.043270366612942, 'learning_rate': 7.844720496894409e-07, 'completion_length': 143.42858123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.29123931378126144, 'kl': 0.01104736328125, 'epoch': 1.08}

 22%|██▏       | 347/1610 [2:22:52<5:35:54, 15.96s/it]
 22%|██▏       | 348/1610 [2:23:11<5:54:32, 16.86s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7199131735067835, 'learning_rate': 7.838509316770185e-07, 'completion_length': 176.33929443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.25552501529455185, 'kl': 0.0071258544921875, 'epoch': 1.08}

 22%|██▏       | 348/1610 [2:23:11<5:54:32, 16.86s/it]
 22%|██▏       | 349/1610 [2:23:29<6:02:22, 17.24s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.982640417745443, 'learning_rate': 7.832298136645963e-07, 'completion_length': 174.42858123779297, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.26657507568597794, 'kl': 0.01104736328125, 'epoch': 1.08}

 22%|██▏       | 349/1610 [2:23:29<6:02:22, 17.24s/it]
 22%|██▏       | 350/1610 [2:23:48<6:11:36, 17.70s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 6.941865271596261, 'learning_rate': 7.826086956521739e-07, 'completion_length': 186.62500762939453, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.3078143671154976, 'kl': 0.012786865234375, 'epoch': 1.09}

 22%|██▏       | 350/1610 [2:23:48<6:11:36, 17.70s/it]
 22%|██▏       | 351/1610 [2:24:06<6:14:18, 17.84s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7414862689373847, 'learning_rate': 7.819875776397515e-07, 'completion_length': 187.83929443359375, 'rewards/accuracy_reward': 0.7321428656578064, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.23086078837513924, 'kl': 0.009124755859375, 'epoch': 1.09}

 22%|██▏       | 351/1610 [2:24:06<6:14:18, 17.84s/it]
 22%|██▏       | 352/1610 [2:24:22<6:02:35, 17.29s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.5072923221311331, 'learning_rate': 7.813664596273292e-07, 'completion_length': 185.1428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.14838215708732605, 'kl': 0.010162353515625, 'epoch': 1.09}

 22%|██▏       | 352/1610 [2:24:22<6:02:35, 17.29s/it]
 22%|██▏       | 353/1610 [2:24:41<6:14:57, 17.90s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7883613230044295, 'learning_rate': 7.807453416149068e-07, 'completion_length': 215.12500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357544541359, 'kl': 0.012542724609375, 'epoch': 1.1}

 22%|██▏       | 353/1610 [2:24:41<6:14:57, 17.90s/it]
 22%|██▏       | 354/1610 [2:24:57<6:04:42, 17.42s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.544905543501432, 'learning_rate': 7.801242236024844e-07, 'completion_length': 147.9821548461914, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.14534124732017517, 'kl': 0.014251708984375, 'epoch': 1.1}

 22%|██▏       | 354/1610 [2:24:58<6:04:42, 17.42s/it]
 22%|██▏       | 355/1610 [2:25:15<6:06:27, 17.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.3245859545524983, 'learning_rate': 7.79503105590062e-07, 'completion_length': 164.9821548461914, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.30228936672210693, 'kl': 0.012359619140625, 'epoch': 1.1}

 22%|██▏       | 355/1610 [2:25:15<6:06:27, 17.52s/it]
 22%|██▏       | 356/1610 [2:25:33<6:10:26, 17.72s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7482798482592642, 'learning_rate': 7.788819875776397e-07, 'completion_length': 169.1964340209961, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.446428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.0101318359375, 'epoch': 1.11}

 22%|██▏       | 356/1610 [2:25:33<6:10:26, 17.72s/it]
 22%|██▏       | 357/1610 [2:25:48<5:49:54, 16.76s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6860401167633384, 'learning_rate': 7.782608695652173e-07, 'completion_length': 158.8214340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1539071872830391, 'kl': 0.012908935546875, 'epoch': 1.11}

 22%|██▏       | 357/1610 [2:25:48<5:49:54, 16.76s/it]
 22%|██▏       | 358/1610 [2:26:07<6:03:30, 17.42s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7775180861921305, 'learning_rate': 7.776397515527951e-07, 'completion_length': 189.5714340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.696428656578064, 'reward_std': 0.13981622830033302, 'kl': 0.009979248046875, 'epoch': 1.11}

 22%|██▏       | 358/1610 [2:26:07<6:03:30, 17.42s/it]
 22%|██▏       | 359/1610 [2:26:22<5:48:02, 16.69s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.4945240708350227, 'learning_rate': 7.770186335403727e-07, 'completion_length': 168.58929443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2253357656300068, 'kl': 0.008636474609375, 'epoch': 1.11}

 22%|██▏       | 359/1610 [2:26:22<5:48:02, 16.69s/it]
 22%|██▏       | 360/1610 [2:26:34<5:19:53, 15.35s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.3379988629278643, 'learning_rate': 7.763975155279503e-07, 'completion_length': 138.89286422729492, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.0118408203125, 'epoch': 1.12}

 22%|██▏       | 360/1610 [2:26:34<5:19:53, 15.35s/it]
 22%|██▏       | 361/1610 [2:26:52<5:37:22, 16.21s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8814050169794712, 'learning_rate': 7.75776397515528e-07, 'completion_length': 148.98214721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.1896214671432972, 'kl': 0.011932373046875, 'epoch': 1.12}

 22%|██▏       | 361/1610 [2:26:52<5:37:22, 16.21s/it]
 22%|██▏       | 362/1610 [2:27:08<5:30:39, 15.90s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.146252849774281, 'learning_rate': 7.751552795031056e-07, 'completion_length': 155.76786041259766, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.15943220257759094, 'kl': 0.0146484375, 'epoch': 1.12}

 22%|██▏       | 362/1610 [2:27:08<5:30:39, 15.90s/it]
 23%|██▎       | 363/1610 [2:27:22<5:23:07, 15.55s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9822606741259431, 'learning_rate': 7.745341614906832e-07, 'completion_length': 156.58929443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.26657505333423615, 'kl': 0.01190185546875, 'epoch': 1.13}

 23%|██▎       | 363/1610 [2:27:22<5:23:07, 15.55s/it]
 23%|██▎       | 364/1610 [2:27:37<5:16:06, 15.22s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.28508584883367194, 'learning_rate': 7.739130434782608e-07, 'completion_length': 162.50000762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.0357142873108387, 'kl': 0.007965087890625, 'epoch': 1.13}

 23%|██▎       | 364/1610 [2:27:37<5:16:06, 15.22s/it]
 23%|██▎       | 365/1610 [2:27:54<5:26:45, 15.75s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.010232839778838, 'learning_rate': 7.732919254658385e-07, 'completion_length': 180.39286041259766, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.2610500305891037, 'kl': 0.01434326171875, 'epoch': 1.13}

 23%|██▎       | 365/1610 [2:27:54<5:26:45, 15.75s/it]
 23%|██▎       | 366/1610 [2:28:10<5:27:08, 15.78s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0891291913131964, 'learning_rate': 7.726708074534161e-07, 'completion_length': 174.2678680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2610500380396843, 'kl': 0.009796142578125, 'epoch': 1.14}

 23%|██▎       | 366/1610 [2:28:10<5:27:08, 15.78s/it]
 23%|██▎       | 367/1610 [2:28:23<5:13:53, 15.15s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6335696090650407, 'learning_rate': 7.720496894409939e-07, 'completion_length': 147.3214340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409644439816475, 'kl': 0.009918212890625, 'epoch': 1.14}

 23%|██▎       | 367/1610 [2:28:23<5:13:53, 15.15s/it]
 23%|██▎       | 368/1610 [2:28:40<5:23:07, 15.61s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.582346630573416, 'learning_rate': 7.714285714285714e-07, 'completion_length': 196.08929443359375, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571529686451, 'kl': 0.009979248046875, 'epoch': 1.14}

 23%|██▎       | 368/1610 [2:28:40<5:23:07, 15.61s/it]
 23%|██▎       | 369/1610 [2:29:00<5:51:51, 17.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7856757822386664, 'learning_rate': 7.70807453416149e-07, 'completion_length': 203.67858123779297, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.21676981449127197, 'kl': 0.01171875, 'epoch': 1.15}

 23%|██▎       | 369/1610 [2:29:00<5:51:51, 17.01s/it]
 23%|██▎       | 370/1610 [2:29:16<5:45:36, 16.72s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6429803999275125, 'learning_rate': 7.701863354037266e-07, 'completion_length': 154.1607208251953, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2610500454902649, 'kl': 0.010009765625, 'epoch': 1.15}

 23%|██▎       | 370/1610 [2:29:16<5:45:36, 16.72s/it]
 23%|██▎       | 371/1610 [2:29:33<5:43:24, 16.63s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8139291882872771, 'learning_rate': 7.695652173913043e-07, 'completion_length': 155.875, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.2142857238650322, 'kl': 0.011566162109375, 'epoch': 1.15}

 23%|██▎       | 371/1610 [2:29:33<5:43:24, 16.63s/it]
 23%|██▎       | 372/1610 [2:29:48<5:37:55, 16.38s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.766785328494099, 'learning_rate': 7.689440993788819e-07, 'completion_length': 166.5178680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2142857313156128, 'kl': 0.010162353515625, 'epoch': 1.16}

 23%|██▎       | 372/1610 [2:29:48<5:37:55, 16.38s/it]
 23%|██▎       | 373/1610 [2:30:08<5:55:27, 17.24s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.8564522821533258, 'learning_rate': 7.683229813664595e-07, 'completion_length': 179.46429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535715222358704, 'reward_std': 0.23689261823892593, 'kl': 0.011016845703125, 'epoch': 1.16}

 23%|██▎       | 373/1610 [2:30:08<5:55:27, 17.24s/it]
 23%|██▎       | 374/1610 [2:30:27<6:07:57, 17.86s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1525962816943673, 'learning_rate': 7.677018633540372e-07, 'completion_length': 205.55358123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216826319695, 'kl': 0.012115478515625, 'epoch': 1.16}

 23%|██▎       | 374/1610 [2:30:27<6:07:57, 17.86s/it]
 23%|██▎       | 375/1610 [2:30:46<6:12:27, 18.09s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6445795154653666, 'learning_rate': 7.670807453416148e-07, 'completion_length': 205.4464340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.07695358991622925, 'kl': 0.009002685546875, 'epoch': 1.16}

 23%|██▎       | 375/1610 [2:30:46<6:12:27, 18.09s/it]
 23%|██▎       | 376/1610 [2:31:02<5:59:12, 17.47s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.7753864563256543, 'learning_rate': 7.664596273291925e-07, 'completion_length': 159.1428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.009674072265625, 'epoch': 1.17}

 23%|██▎       | 376/1610 [2:31:02<5:59:12, 17.47s/it]
 23%|██▎       | 377/1610 [2:31:17<5:46:23, 16.86s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 3.3254086756955696, 'learning_rate': 7.658385093167702e-07, 'completion_length': 163.62500762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981074661016464, 'kl': 0.0086517333984375, 'epoch': 1.17}

 23%|██▎       | 377/1610 [2:31:17<5:46:23, 16.86s/it]
 23%|██▎       | 378/1610 [2:31:34<5:46:40, 16.88s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8747504958839437, 'learning_rate': 7.652173913043478e-07, 'completion_length': 194.05358123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.30228935927152634, 'kl': 0.010223388671875, 'epoch': 1.17}

 23%|██▎       | 378/1610 [2:31:34<5:46:40, 16.88s/it]
 24%|██▎       | 379/1610 [2:31:53<5:57:23, 17.42s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.3042458697377341, 'learning_rate': 7.645962732919254e-07, 'completion_length': 189.3571548461914, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.1071428656578064, 'kl': 0.009246826171875, 'epoch': 1.18}

 24%|██▎       | 379/1610 [2:31:53<5:57:23, 17.42s/it]
 24%|██▎       | 380/1610 [2:32:10<5:57:18, 17.43s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7205497791869955, 'learning_rate': 7.639751552795031e-07, 'completion_length': 170.75000762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1428571529686451, 'kl': 0.012481689453125, 'epoch': 1.18}

 24%|██▎       | 380/1610 [2:32:10<5:57:18, 17.43s/it]
 24%|██▎       | 381/1610 [2:32:25<5:43:42, 16.78s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1878020316661524, 'learning_rate': 7.633540372670807e-07, 'completion_length': 169.83929443359375, 'rewards/accuracy_reward': 0.3928571529686451, 'rewards/format_reward': 1.0, 'reward': 1.3928572535514832, 'reward_std': 0.1428571492433548, 'kl': 0.016326904296875, 'epoch': 1.18}

 24%|██▎       | 381/1610 [2:32:25<5:43:42, 16.78s/it]
 24%|██▎       | 382/1610 [2:32:43<5:50:44, 17.14s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7419092832991042, 'learning_rate': 7.627329192546583e-07, 'completion_length': 176.08929443359375, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.010894775390625, 'epoch': 1.19}

 24%|██▎       | 382/1610 [2:32:43<5:50:44, 17.14s/it]
 24%|██▍       | 383/1610 [2:33:01<5:54:13, 17.32s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7306820943368826, 'learning_rate': 7.62111801242236e-07, 'completion_length': 178.67858123779297, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.0120849609375, 'epoch': 1.19}

 24%|██▍       | 383/1610 [2:33:01<5:54:13, 17.32s/it]
 24%|██▍       | 384/1610 [2:33:18<5:49:35, 17.11s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6871436571457419, 'learning_rate': 7.614906832298136e-07, 'completion_length': 165.33929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500000596046448, 'reward_std': 0.24695909023284912, 'kl': 0.009857177734375, 'epoch': 1.19}

 24%|██▍       | 384/1610 [2:33:18<5:49:35, 17.11s/it]
 24%|██▍       | 385/1610 [2:33:36<5:55:54, 17.43s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.5979292903324074, 'learning_rate': 7.608695652173913e-07, 'completion_length': 180.92858123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.011199951171875, 'epoch': 1.2}

 24%|██▍       | 385/1610 [2:33:36<5:55:54, 17.43s/it]
 24%|██▍       | 386/1610 [2:33:53<5:55:52, 17.45s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7334516003469496, 'learning_rate': 7.60248447204969e-07, 'completion_length': 178.6428680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.012908935546875, 'epoch': 1.2}

 24%|██▍       | 386/1610 [2:33:53<5:55:52, 17.45s/it]
 24%|██▍       | 387/1610 [2:34:12<6:02:35, 17.79s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0854590786108955, 'learning_rate': 7.596273291925466e-07, 'completion_length': 170.5178680419922, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.2967643216252327, 'kl': 0.01043701171875, 'epoch': 1.2}

 24%|██▍       | 387/1610 [2:34:12<6:02:35, 17.79s/it]
 24%|██▍       | 388/1610 [2:34:24<5:28:10, 16.11s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.510176937773149, 'learning_rate': 7.590062111801242e-07, 'completion_length': 144.50000381469727, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.01177978515625, 'epoch': 1.2}

 24%|██▍       | 388/1610 [2:34:24<5:28:10, 16.11s/it]
 24%|██▍       | 389/1610 [2:34:39<5:21:15, 15.79s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2270227275069123, 'learning_rate': 7.583850931677019e-07, 'completion_length': 160.71429443359375, 'rewards/accuracy_reward': 0.4642857387661934, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.25552503019571304, 'kl': 0.0107421875, 'epoch': 1.21}

 24%|██▍       | 389/1610 [2:34:39<5:21:15, 15.79s/it]
 24%|██▍       | 390/1610 [2:34:51<4:54:28, 14.48s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9728814805008528, 'learning_rate': 7.577639751552795e-07, 'completion_length': 129.66071701049805, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.011260986328125, 'epoch': 1.21}

 24%|██▍       | 390/1610 [2:34:51<4:54:28, 14.48s/it]
 24%|██▍       | 391/1610 [2:35:06<4:57:55, 14.66s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 2.7579518815312025, 'learning_rate': 7.57142857142857e-07, 'completion_length': 149.5357208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.26657505333423615, 'kl': 0.0103759765625, 'epoch': 1.21}

 24%|██▍       | 391/1610 [2:35:06<4:57:55, 14.66s/it]
 24%|██▍       | 392/1610 [2:35:25<5:23:42, 15.95s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.260923315519923, 'learning_rate': 7.565217391304347e-07, 'completion_length': 183.55357360839844, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838216826319695, 'kl': 0.011932373046875, 'epoch': 1.22}

 24%|██▍       | 392/1610 [2:35:25<5:23:42, 15.95s/it]
 24%|██▍       | 393/1610 [2:35:39<5:14:26, 15.50s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7661433884095068, 'learning_rate': 7.559006211180123e-07, 'completion_length': 150.9464340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.23086076974868774, 'kl': 0.011322021484375, 'epoch': 1.22}

 24%|██▍       | 393/1610 [2:35:39<5:14:26, 15.50s/it]
 24%|██▍       | 394/1610 [2:35:54<5:09:57, 15.29s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.4667136778056367, 'learning_rate': 7.5527950310559e-07, 'completion_length': 155.83929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1539071798324585, 'kl': 0.009490966796875, 'epoch': 1.22}

 24%|██▍       | 394/1610 [2:35:54<5:09:57, 15.29s/it]
 25%|██▍       | 395/1610 [2:36:11<5:19:58, 15.80s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 3.1203406867605668, 'learning_rate': 7.546583850931677e-07, 'completion_length': 185.37500762939453, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.21981073170900345, 'kl': 0.0093994140625, 'epoch': 1.23}

 25%|██▍       | 395/1610 [2:36:11<5:19:58, 15.80s/it]
 25%|██▍       | 396/1610 [2:36:27<5:20:21, 15.83s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.3945673406554986, 'learning_rate': 7.540372670807453e-07, 'completion_length': 158.53572845458984, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.307814359664917, 'kl': 0.0106201171875, 'epoch': 1.23}

 25%|██▍       | 396/1610 [2:36:27<5:20:21, 15.83s/it]
 25%|██▍       | 397/1610 [2:36:45<5:31:41, 16.41s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7200959422690038, 'learning_rate': 7.534161490683229e-07, 'completion_length': 185.30358123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981073170900345, 'kl': 0.0089111328125, 'epoch': 1.23}

 25%|██▍       | 397/1610 [2:36:45<5:31:41, 16.41s/it]
 25%|██▍       | 398/1610 [2:37:03<5:41:46, 16.92s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 2.0724362232842233, 'learning_rate': 7.527950310559006e-07, 'completion_length': 148.5714340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.009765625, 'epoch': 1.24}

 25%|██▍       | 398/1610 [2:37:03<5:41:46, 16.92s/it]
 25%|██▍       | 399/1610 [2:37:20<5:42:41, 16.98s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.4599152149028754, 'learning_rate': 7.521739130434782e-07, 'completion_length': 184.19644165039062, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.011932373046875, 'epoch': 1.24}

 25%|██▍       | 399/1610 [2:37:20<5:42:41, 16.98s/it]
 25%|██▍       | 400/1610 [2:37:39<5:54:00, 17.55s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7485540199488304, 'learning_rate': 7.515527950310558e-07, 'completion_length': 191.07144165039062, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357470035553, 'kl': 0.01483154296875, 'epoch': 1.24}

 25%|██▍       | 400/1610 [2:37:39<5:54:00, 17.55s/it]
 25%|██▍       | 401/1610 [2:41:20<26:24:55, 78.66s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.7590059791154383, 'learning_rate': 7.509316770186335e-07, 'completion_length': 161.7857208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1785714328289032, 'kl': 0.01251220703125, 'epoch': 1.25}

 25%|██▍       | 401/1610 [2:41:20<26:24:55, 78.66s/it]
 25%|██▍       | 402/1610 [2:41:34<19:51:52, 59.20s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.5995134144529783, 'learning_rate': 7.503105590062111e-07, 'completion_length': 122.05357360839844, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428656578064, 'kl': 0.011138916015625, 'epoch': 1.25}

 25%|██▍       | 402/1610 [2:41:34<19:51:52, 59.20s/it]
 25%|██▌       | 403/1610 [2:41:49<15:26:09, 46.04s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.0839297366557037, 'learning_rate': 7.496894409937888e-07, 'completion_length': 170.4107208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357470035553, 'kl': 0.011962890625, 'epoch': 1.25}

 25%|██▌       | 403/1610 [2:41:49<15:26:09, 46.04s/it]
 25%|██▌       | 404/1610 [2:42:03<12:11:34, 36.40s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.4801221921156385, 'learning_rate': 7.490683229813665e-07, 'completion_length': 145.00000762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266787722706795, 'kl': 0.0098876953125, 'epoch': 1.25}

 25%|██▌       | 404/1610 [2:42:03<12:11:34, 36.40s/it]
 25%|██▌       | 405/1610 [2:42:20<10:13:39, 30.56s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 2.007257744231057, 'learning_rate': 7.484472049689441e-07, 'completion_length': 185.44644165039062, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.2363857924938202, 'kl': 0.012603759765625, 'epoch': 1.26}

 25%|██▌       | 405/1610 [2:42:20<10:13:39, 30.56s/it]
 25%|██▌       | 406/1610 [2:42:40<9:09:20, 27.38s/it] 
                                                      
{'loss': 0.0003, 'grad_norm': 1.0000566230569112, 'learning_rate': 7.478260869565217e-07, 'completion_length': 213.58929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7142857909202576, 'reward_std': 0.2881983816623688, 'kl': 0.008514404296875, 'epoch': 1.26}

 25%|██▌       | 406/1610 [2:42:40<9:09:20, 27.38s/it]
 25%|██▌       | 407/1610 [2:42:52<7:40:07, 22.95s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.5941574215590801, 'learning_rate': 7.472049689440994e-07, 'completion_length': 141.0714340209961, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.0082855224609375, 'epoch': 1.26}

 25%|██▌       | 407/1610 [2:42:52<7:40:07, 22.95s/it]
 25%|██▌       | 408/1610 [2:43:11<7:10:22, 21.48s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.330821964607959, 'learning_rate': 7.46583850931677e-07, 'completion_length': 163.07144165039062, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.0357142873108387, 'kl': 0.01116943359375, 'epoch': 1.27}

 25%|██▌       | 408/1610 [2:43:11<7:10:22, 21.48s/it]
 25%|██▌       | 409/1610 [2:43:25<6:27:50, 19.38s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0574206897034122, 'learning_rate': 7.459627329192546e-07, 'completion_length': 154.62500762939453, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.25552501529455185, 'kl': 0.010528564453125, 'epoch': 1.27}

 25%|██▌       | 409/1610 [2:43:25<6:27:50, 19.38s/it]
 25%|██▌       | 410/1610 [2:43:43<6:20:12, 19.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5396348373635381, 'learning_rate': 7.453416149068323e-07, 'completion_length': 174.71429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785714626312256, 'reward_std': 0.14534124732017517, 'kl': 0.011383056640625, 'epoch': 1.27}

 25%|██▌       | 410/1610 [2:43:43<6:20:12, 19.01s/it]
 26%|██▌       | 411/1610 [2:43:59<6:01:19, 18.08s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9313590687107354, 'learning_rate': 7.447204968944099e-07, 'completion_length': 152.3750114440918, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2253357619047165, 'kl': 0.0115966796875, 'epoch': 1.28}

 26%|██▌       | 411/1610 [2:43:59<6:01:19, 18.08s/it]
 26%|██▌       | 412/1610 [2:44:13<5:35:14, 16.79s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2645507497764807, 'learning_rate': 7.440993788819876e-07, 'completion_length': 149.62500762939453, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.29123931378126144, 'kl': 0.0142822265625, 'epoch': 1.28}

 26%|██▌       | 412/1610 [2:44:13<5:35:14, 16.79s/it]
 26%|██▌       | 413/1610 [2:44:26<5:15:09, 15.80s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2519212569603957, 'learning_rate': 7.434782608695653e-07, 'completion_length': 144.35714721679688, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.19514649361371994, 'kl': 0.010467529296875, 'epoch': 1.28}

 26%|██▌       | 413/1610 [2:44:26<5:15:09, 15.80s/it]
 26%|██▌       | 414/1610 [2:44:44<5:27:27, 16.43s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7765054586149542, 'learning_rate': 7.428571428571429e-07, 'completion_length': 166.48214721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2006715089082718, 'kl': 0.01519775390625, 'epoch': 1.29}

 26%|██▌       | 414/1610 [2:44:44<5:27:27, 16.43s/it]
 26%|██▌       | 415/1610 [2:45:01<5:28:03, 16.47s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8773486036683271, 'learning_rate': 7.422360248447204e-07, 'completion_length': 164.26786041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7678571939468384, 'reward_std': 0.29123931005597115, 'kl': 0.0098876953125, 'epoch': 1.29}

 26%|██▌       | 415/1610 [2:45:01<5:28:03, 16.47s/it]
 26%|██▌       | 416/1610 [2:45:16<5:18:54, 16.03s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.828936310232208, 'learning_rate': 7.416149068322981e-07, 'completion_length': 134.73215103149414, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.25552502274513245, 'kl': 0.0095977783203125, 'epoch': 1.29}

 26%|██▌       | 416/1610 [2:45:16<5:18:54, 16.03s/it]
 26%|██▌       | 417/1610 [2:45:34<5:31:07, 16.65s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2119858540920359, 'learning_rate': 7.409937888198757e-07, 'completion_length': 161.76786041259766, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5178571939468384, 'reward_std': 0.2826733738183975, 'kl': 0.012786865234375, 'epoch': 1.3}

 26%|██▌       | 417/1610 [2:45:34<5:31:07, 16.65s/it]
 26%|██▌       | 418/1610 [2:45:48<5:13:00, 15.76s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0752503368578286, 'learning_rate': 7.403726708074533e-07, 'completion_length': 134.08929061889648, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214671432972, 'kl': 0.01031494140625, 'epoch': 1.3}

 26%|██▌       | 418/1610 [2:45:48<5:13:00, 15.76s/it]
 26%|██▌       | 419/1610 [2:46:06<5:30:49, 16.67s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5829532752638823, 'learning_rate': 7.39751552795031e-07, 'completion_length': 201.1071548461914, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1785714365541935, 'kl': 0.01361083984375, 'epoch': 1.3}

 26%|██▌       | 419/1610 [2:46:06<5:30:49, 16.67s/it]
 26%|██▌       | 420/1610 [2:46:25<5:44:22, 17.36s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.689320232184276, 'learning_rate': 7.391304347826086e-07, 'completion_length': 174.7321548461914, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.015838623046875, 'epoch': 1.3}

 26%|██▌       | 420/1610 [2:46:25<5:44:22, 17.36s/it]
 26%|██▌       | 421/1610 [2:46:41<5:36:12, 16.97s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6697808628637423, 'learning_rate': 7.385093167701863e-07, 'completion_length': 149.30357360839844, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838217198848724, 'kl': 0.010101318359375, 'epoch': 1.31}

 26%|██▌       | 421/1610 [2:46:41<5:36:12, 16.97s/it]
 26%|██▌       | 422/1610 [2:46:58<5:35:05, 16.92s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 2.367195384524125, 'learning_rate': 7.37888198757764e-07, 'completion_length': 163.01786041259766, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.010986328125, 'epoch': 1.31}

 26%|██▌       | 422/1610 [2:46:58<5:35:05, 16.92s/it]
 26%|██▋       | 423/1610 [2:47:13<5:20:08, 16.18s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.6167445002297919, 'learning_rate': 7.372670807453416e-07, 'completion_length': 165.08929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214783191681, 'kl': 0.00927734375, 'epoch': 1.31}

 26%|██▋       | 423/1610 [2:47:13<5:20:08, 16.18s/it]
 26%|██▋       | 424/1610 [2:47:29<5:22:13, 16.30s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.4578665500827954, 'learning_rate': 7.366459627329192e-07, 'completion_length': 151.96428680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.2610500529408455, 'kl': 0.0107421875, 'epoch': 1.32}

 26%|██▋       | 424/1610 [2:47:29<5:22:13, 16.30s/it]
 26%|██▋       | 425/1610 [2:47:50<5:48:10, 17.63s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.791425099506591, 'learning_rate': 7.360248447204969e-07, 'completion_length': 196.33929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.3404877483844757, 'kl': 0.01080322265625, 'epoch': 1.32}

 26%|██▋       | 425/1610 [2:47:50<5:48:10, 17.63s/it]
 26%|██▋       | 426/1610 [2:48:06<5:36:24, 17.05s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0543339007713644, 'learning_rate': 7.354037267080745e-07, 'completion_length': 159.35714721679688, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1785714365541935, 'kl': 0.012298583984375, 'epoch': 1.32}

 26%|██▋       | 426/1610 [2:48:06<5:36:24, 17.05s/it]
 27%|██▋       | 427/1610 [2:48:26<5:54:06, 17.96s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.8105674913372036, 'learning_rate': 7.347826086956521e-07, 'completion_length': 193.44644165039062, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4464285969734192, 'reward_std': 0.37371791899204254, 'kl': 0.013946533203125, 'epoch': 1.33}

 27%|██▋       | 427/1610 [2:48:26<5:54:06, 17.96s/it]
 27%|██▋       | 428/1610 [2:48:42<5:45:14, 17.53s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.5101988610561807, 'learning_rate': 7.341614906832298e-07, 'completion_length': 167.51786041259766, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.01031494140625, 'epoch': 1.33}

 27%|██▋       | 428/1610 [2:48:42<5:45:14, 17.53s/it]
 27%|██▋       | 429/1610 [2:48:58<5:35:24, 17.04s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.4116729797664058, 'learning_rate': 7.335403726708074e-07, 'completion_length': 157.23214721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.01263427734375, 'epoch': 1.33}

 27%|██▋       | 429/1610 [2:48:58<5:35:24, 17.04s/it]
 27%|██▋       | 430/1610 [2:49:18<5:49:13, 17.76s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.3933268924548763, 'learning_rate': 7.329192546583851e-07, 'completion_length': 164.01786041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657505333423615, 'kl': 0.011871337890625, 'epoch': 1.34}

 27%|██▋       | 430/1610 [2:49:18<5:49:13, 17.76s/it]
 27%|██▋       | 431/1610 [2:49:35<5:45:51, 17.60s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8362767758469605, 'learning_rate': 7.322981366459628e-07, 'completion_length': 165.48214721679688, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214634180069, 'kl': 0.01263427734375, 'epoch': 1.34}

 27%|██▋       | 431/1610 [2:49:35<5:45:51, 17.60s/it]
 27%|██▋       | 432/1610 [2:49:51<5:35:58, 17.11s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.312868742292956, 'learning_rate': 7.316770186335404e-07, 'completion_length': 144.35715103149414, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.821428656578064, 'reward_std': 0.2580091208219528, 'kl': 0.00927734375, 'epoch': 1.34}

 27%|██▋       | 432/1610 [2:49:51<5:35:58, 17.11s/it]
 27%|██▋       | 433/1610 [2:50:08<5:38:26, 17.25s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0115081371142896, 'learning_rate': 7.31055900621118e-07, 'completion_length': 177.78572845458984, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.01239013671875, 'epoch': 1.34}

 27%|██▋       | 433/1610 [2:50:08<5:38:26, 17.25s/it]
 27%|██▋       | 434/1610 [2:50:27<5:45:20, 17.62s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.4339645665031884, 'learning_rate': 7.304347826086957e-07, 'completion_length': 177.4464340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.23086077719926834, 'kl': 0.00994873046875, 'epoch': 1.35}

 27%|██▋       | 434/1610 [2:50:27<5:45:20, 17.62s/it]
 27%|██▋       | 435/1610 [2:50:44<5:44:43, 17.60s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2564201873064997, 'learning_rate': 7.298136645962733e-07, 'completion_length': 185.6607208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.011138916015625, 'epoch': 1.35}

 27%|██▋       | 435/1610 [2:50:44<5:44:43, 17.60s/it]
 27%|██▋       | 436/1610 [2:50:58<5:18:48, 16.29s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 0.7843001996334007, 'learning_rate': 7.291925465838509e-07, 'completion_length': 162.5178680419922, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.19514649361371994, 'kl': 0.00860595703125, 'epoch': 1.35}

 27%|██▋       | 436/1610 [2:50:58<5:18:48, 16.29s/it]
 27%|██▋       | 437/1610 [2:51:16<5:31:44, 16.97s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9254928848593261, 'learning_rate': 7.285714285714286e-07, 'completion_length': 179.83929443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.29123930633068085, 'kl': 0.0146484375, 'epoch': 1.36}

 27%|██▋       | 437/1610 [2:51:16<5:31:44, 16.97s/it]
 27%|██▋       | 438/1610 [2:51:31<5:17:57, 16.28s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9632071259006911, 'learning_rate': 7.279503105590061e-07, 'completion_length': 131.3214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.26657505333423615, 'kl': 0.013275146484375, 'epoch': 1.36}

 27%|██▋       | 438/1610 [2:51:31<5:17:57, 16.28s/it]
 27%|██▋       | 439/1610 [2:51:48<5:23:43, 16.59s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0073020332413762, 'learning_rate': 7.273291925465838e-07, 'completion_length': 181.37500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.32695358991622925, 'kl': 0.011566162109375, 'epoch': 1.36}

 27%|██▋       | 439/1610 [2:51:48<5:23:43, 16.59s/it]
 27%|██▋       | 440/1610 [2:52:08<5:44:19, 17.66s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9884455038126712, 'learning_rate': 7.267080745341615e-07, 'completion_length': 198.55358123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2967643216252327, 'kl': 0.012359619140625, 'epoch': 1.37}

 27%|██▋       | 440/1610 [2:52:08<5:44:19, 17.66s/it]
 27%|██▋       | 441/1610 [2:52:26<5:46:46, 17.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2791787887020072, 'learning_rate': 7.260869565217391e-07, 'completion_length': 168.17858123779297, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.19514649361371994, 'kl': 0.01373291015625, 'epoch': 1.37}

 27%|██▋       | 441/1610 [2:52:26<5:46:46, 17.80s/it]
 27%|██▋       | 442/1610 [2:52:43<5:36:23, 17.28s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4131501834518345, 'learning_rate': 7.254658385093167e-07, 'completion_length': 163.6607208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.31333939731121063, 'kl': 0.013946533203125, 'epoch': 1.37}

 27%|██▋       | 442/1610 [2:52:43<5:36:23, 17.28s/it]
 28%|██▊       | 443/1610 [2:53:02<5:48:45, 17.93s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6617083388842668, 'learning_rate': 7.248447204968943e-07, 'completion_length': 210.80358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.2857142984867096, 'kl': 0.011627197265625, 'epoch': 1.38}

 28%|██▊       | 443/1610 [2:53:02<5:48:45, 17.93s/it]
 28%|██▊       | 444/1610 [2:53:18<5:38:27, 17.42s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0471883049936226, 'learning_rate': 7.24223602484472e-07, 'completion_length': 139.17858123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981073170900345, 'kl': 0.012054443359375, 'epoch': 1.38}

 28%|██▊       | 444/1610 [2:53:18<5:38:27, 17.42s/it]
 28%|██▊       | 445/1610 [2:53:33<5:23:24, 16.66s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0801104364829375, 'learning_rate': 7.236024844720496e-07, 'completion_length': 173.4107208251953, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.33800365030765533, 'kl': 0.01629638671875, 'epoch': 1.38}

 28%|██▊       | 445/1610 [2:53:33<5:23:24, 16.66s/it]
 28%|██▊       | 446/1610 [2:53:50<5:23:02, 16.65s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9289853239273006, 'learning_rate': 7.229813664596272e-07, 'completion_length': 155.12500762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.21981074661016464, 'kl': 0.0118408203125, 'epoch': 1.39}

 28%|██▊       | 446/1610 [2:53:50<5:23:02, 16.65s/it]
 28%|██▊       | 447/1610 [2:54:08<5:29:26, 17.00s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2340071450237817, 'learning_rate': 7.223602484472049e-07, 'completion_length': 182.01786041259766, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.0714285746216774, 'kl': 0.0115966796875, 'epoch': 1.39}

 28%|██▊       | 447/1610 [2:54:08<5:29:26, 17.00s/it]
 28%|██▊       | 448/1610 [2:54:25<5:33:58, 17.24s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.2908326741292651, 'learning_rate': 7.217391304347826e-07, 'completion_length': 181.12500762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.1071428656578064, 'kl': 0.01116943359375, 'epoch': 1.39}

 28%|██▊       | 448/1610 [2:54:25<5:33:58, 17.24s/it]
 28%|██▊       | 449/1610 [2:54:45<5:50:06, 18.09s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0136066260182233, 'learning_rate': 7.211180124223603e-07, 'completion_length': 162.6607208251953, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.732142984867096, 'reward_std': 0.31937122344970703, 'kl': 0.013214111328125, 'epoch': 1.39}

 28%|██▊       | 449/1610 [2:54:45<5:50:06, 18.09s/it]
 28%|██▊       | 450/1610 [2:55:01<5:36:05, 17.38s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7931824247414122, 'learning_rate': 7.204968944099379e-07, 'completion_length': 155.32144165039062, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1071428656578064, 'kl': 0.012847900390625, 'epoch': 1.4}

 28%|██▊       | 450/1610 [2:55:01<5:36:05, 17.38s/it]
 28%|██▊       | 451/1610 [2:55:17<5:26:30, 16.90s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.2371718164513521, 'learning_rate': 7.198757763975155e-07, 'completion_length': 151.60714721679688, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.0357142873108387, 'kl': 0.010650634765625, 'epoch': 1.4}

 28%|██▊       | 451/1610 [2:55:17<5:26:30, 16.90s/it]
 28%|██▊       | 452/1610 [2:55:31<5:09:09, 16.02s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0162448638252988, 'learning_rate': 7.192546583850931e-07, 'completion_length': 144.42857360839844, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2253357470035553, 'kl': 0.012237548828125, 'epoch': 1.4}

 28%|██▊       | 452/1610 [2:55:31<5:09:09, 16.02s/it]
 28%|██▊       | 453/1610 [2:55:45<4:58:08, 15.46s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.054455011059388325, 'learning_rate': 7.186335403726708e-07, 'completion_length': 136.55358123779297, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.0, 'kl': 0.009429931640625, 'epoch': 1.41}

 28%|██▊       | 453/1610 [2:55:45<4:58:08, 15.46s/it]
 28%|██▊       | 454/1610 [2:56:02<5:06:50, 15.93s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 7.236891432705753, 'learning_rate': 7.180124223602484e-07, 'completion_length': 160.76786041259766, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.21981073170900345, 'kl': 0.0130615234375, 'epoch': 1.41}

 28%|██▊       | 454/1610 [2:56:02<5:06:50, 15.93s/it]
 28%|██▊       | 455/1610 [2:56:15<4:48:29, 14.99s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.5648165180243538, 'learning_rate': 7.17391304347826e-07, 'completion_length': 137.1964340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.23086079210042953, 'kl': 0.01031494140625, 'epoch': 1.41}

 28%|██▊       | 455/1610 [2:56:15<4:48:29, 14.99s/it]
 28%|██▊       | 456/1610 [2:56:30<4:50:59, 15.13s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.3655401348660274, 'learning_rate': 7.167701863354037e-07, 'completion_length': 156.75000762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0714285746216774, 'kl': 0.00982666015625, 'epoch': 1.42}

 28%|██▊       | 456/1610 [2:56:30<4:50:59, 15.13s/it]
 28%|██▊       | 457/1610 [2:56:48<5:08:10, 16.04s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.924765622804583, 'learning_rate': 7.161490683229814e-07, 'completion_length': 192.66072845458984, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.11266788095235825, 'kl': 0.011871337890625, 'epoch': 1.42}

 28%|██▊       | 457/1610 [2:56:49<5:08:10, 16.04s/it]
 28%|██▊       | 458/1610 [2:57:05<5:10:54, 16.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.6095455536461813, 'learning_rate': 7.15527950310559e-07, 'completion_length': 181.57144165039062, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500000596046448, 'reward_std': 0.25552502274513245, 'kl': 0.014190673828125, 'epoch': 1.42}

 28%|██▊       | 458/1610 [2:57:05<5:10:54, 16.19s/it]
 29%|██▊       | 459/1610 [2:57:22<5:13:34, 16.35s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.5262383277602842, 'learning_rate': 7.149068322981367e-07, 'completion_length': 162.80358123779297, 'rewards/accuracy_reward': 0.8035714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1071428656578064, 'kl': 0.01153564453125, 'epoch': 1.43}

 29%|██▊       | 459/1610 [2:57:22<5:13:34, 16.35s/it]
 29%|██▊       | 460/1610 [2:57:39<5:15:54, 16.48s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.1527113619703537, 'learning_rate': 7.142857142857143e-07, 'completion_length': 187.98214721679688, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1539071872830391, 'kl': 0.011566162109375, 'epoch': 1.43}

 29%|██▊       | 460/1610 [2:57:39<5:15:54, 16.48s/it]
 29%|██▊       | 461/1610 [2:57:57<5:28:47, 17.17s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.2801945619708005, 'learning_rate': 7.136645962732919e-07, 'completion_length': 177.98214721679688, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.0097503662109375, 'epoch': 1.43}

 29%|██▊       | 461/1610 [2:57:57<5:28:47, 17.17s/it]
 29%|██▊       | 462/1610 [2:58:11<5:11:15, 16.27s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.6316802689473247, 'learning_rate': 7.130434782608695e-07, 'completion_length': 144.21429443359375, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428656578064, 'kl': 0.00836181640625, 'epoch': 1.43}

 29%|██▊       | 462/1610 [2:58:11<5:11:15, 16.27s/it]
 29%|██▉       | 463/1610 [2:58:31<5:28:19, 17.17s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8099622756639822, 'learning_rate': 7.124223602484471e-07, 'completion_length': 193.55358123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5178572535514832, 'reward_std': 0.1896214708685875, 'kl': 0.010162353515625, 'epoch': 1.44}

 29%|██▉       | 463/1610 [2:58:31<5:28:19, 17.17s/it]
 29%|██▉       | 464/1610 [2:58:49<5:32:46, 17.42s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.1128993346403822, 'learning_rate': 7.118012422360247e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6428571939468384, 'reward_std': 0.1539071798324585, 'kl': 0.009552001953125, 'epoch': 1.44}

 29%|██▉       | 464/1610 [2:58:49<5:32:46, 17.42s/it]
 29%|██▉       | 465/1610 [2:59:07<5:35:47, 17.60s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1682323498241727, 'learning_rate': 7.111801242236024e-07, 'completion_length': 175.62500762939453, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.011993408203125, 'epoch': 1.44}

 29%|██▉       | 465/1610 [2:59:07<5:35:47, 17.60s/it]
 29%|██▉       | 466/1610 [2:59:24<5:33:58, 17.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.7320468253119408, 'learning_rate': 7.105590062111801e-07, 'completion_length': 182.25000762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.32695358991622925, 'kl': 0.012847900390625, 'epoch': 1.45}

 29%|██▉       | 466/1610 [2:59:24<5:33:58, 17.52s/it]
 29%|██▉       | 467/1610 [2:59:42<5:34:01, 17.53s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.5181534947988455, 'learning_rate': 7.099378881987577e-07, 'completion_length': 170.17858123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.29123931378126144, 'kl': 0.012481689453125, 'epoch': 1.45}

 29%|██▉       | 467/1610 [2:59:42<5:34:01, 17.53s/it]
 29%|██▉       | 468/1610 [2:59:56<5:16:39, 16.64s/it]
                                                      
{'loss': 0.0003, 'grad_norm': 1.07949342377661, 'learning_rate': 7.093167701863354e-07, 'completion_length': 156.1607208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.008697509765625, 'epoch': 1.45}

 29%|██▉       | 468/1610 [2:59:56<5:16:39, 16.64s/it]
 29%|██▉       | 469/1610 [3:00:14<5:21:52, 16.93s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 3.840921124009164, 'learning_rate': 7.08695652173913e-07, 'completion_length': 157.05357360839844, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.011383056640625, 'epoch': 1.46}

 29%|██▉       | 469/1610 [3:00:14<5:21:52, 16.93s/it]
 29%|██▉       | 470/1610 [3:00:29<5:11:39, 16.40s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8343760796603785, 'learning_rate': 7.080745341614906e-07, 'completion_length': 167.01786422729492, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.12371791899204254, 'kl': 0.011322021484375, 'epoch': 1.46}

 29%|██▉       | 470/1610 [3:00:29<5:11:39, 16.40s/it]
 29%|██▉       | 471/1610 [3:00:46<5:13:02, 16.49s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8507313058675119, 'learning_rate': 7.074534161490683e-07, 'completion_length': 161.5357208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.23086077719926834, 'kl': 0.0146484375, 'epoch': 1.46}

 29%|██▉       | 471/1610 [3:00:46<5:13:02, 16.49s/it]
 29%|██▉       | 472/1610 [3:00:59<4:52:32, 15.42s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8796834202035237, 'learning_rate': 7.068322981366459e-07, 'completion_length': 122.1785774230957, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.01202392578125, 'epoch': 1.47}

 29%|██▉       | 472/1610 [3:00:59<4:52:32, 15.42s/it]
 29%|██▉       | 473/1610 [3:01:15<4:59:07, 15.78s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.883931813451176, 'learning_rate': 7.062111801242235e-07, 'completion_length': 165.78572463989258, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2253357470035553, 'kl': 0.015533447265625, 'epoch': 1.47}

 29%|██▉       | 473/1610 [3:01:15<4:59:07, 15.78s/it]
 29%|██▉       | 474/1610 [3:01:29<4:47:21, 15.18s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6612284182835233, 'learning_rate': 7.055900621118012e-07, 'completion_length': 145.25, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.01141357421875, 'epoch': 1.47}

 29%|██▉       | 474/1610 [3:01:29<4:47:21, 15.18s/it]
 30%|██▉       | 475/1610 [3:01:47<5:02:46, 16.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.3559533246906, 'learning_rate': 7.049689440993789e-07, 'completion_length': 184.58929443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.33800363540649414, 'kl': 0.01300048828125, 'epoch': 1.48}

 30%|██▉       | 475/1610 [3:01:47<5:02:46, 16.01s/it]
 30%|██▉       | 476/1610 [3:02:04<5:11:04, 16.46s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 2.3855920741557113, 'learning_rate': 7.043478260869565e-07, 'completion_length': 182.9107208251953, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.01934814453125, 'epoch': 1.48}

 30%|██▉       | 476/1610 [3:02:04<5:11:04, 16.46s/it]
 30%|██▉       | 477/1610 [3:02:23<5:22:27, 17.08s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1356604885240829, 'learning_rate': 7.037267080745342e-07, 'completion_length': 157.5357208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.29123931378126144, 'kl': 0.01251220703125, 'epoch': 1.48}

 30%|██▉       | 477/1610 [3:02:23<5:22:27, 17.08s/it]
 30%|██▉       | 478/1610 [3:02:40<5:18:57, 16.91s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.6726366955158692, 'learning_rate': 7.031055900621118e-07, 'completion_length': 164.62500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.009918212890625, 'epoch': 1.48}

 30%|██▉       | 478/1610 [3:02:40<5:18:57, 16.91s/it]
 30%|██▉       | 479/1610 [3:02:59<5:30:46, 17.55s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9273750580095537, 'learning_rate': 7.024844720496894e-07, 'completion_length': 185.2678680419922, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.26657505333423615, 'kl': 0.0142822265625, 'epoch': 1.49}

 30%|██▉       | 479/1610 [3:02:59<5:30:46, 17.55s/it]
 30%|██▉       | 480/1610 [3:03:14<5:17:32, 16.86s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.3155265150043707, 'learning_rate': 7.018633540372671e-07, 'completion_length': 143.33929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.19514648616313934, 'kl': 0.0096435546875, 'epoch': 1.49}

 30%|██▉       | 480/1610 [3:03:14<5:17:32, 16.86s/it]
 30%|██▉       | 481/1610 [3:03:31<5:19:54, 17.00s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9777527337656984, 'learning_rate': 7.012422360248447e-07, 'completion_length': 164.0714340209961, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.2967643216252327, 'kl': 0.012420654296875, 'epoch': 1.49}

 30%|██▉       | 481/1610 [3:03:31<5:19:54, 17.00s/it]
 30%|██▉       | 482/1610 [3:03:49<5:25:45, 17.33s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.4801173300253645, 'learning_rate': 7.006211180124223e-07, 'completion_length': 189.69644165039062, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1181928962469101, 'kl': 0.014617919921875, 'epoch': 1.5}

 30%|██▉       | 482/1610 [3:03:49<5:25:45, 17.33s/it]
 30%|███       | 483/1610 [3:04:04<5:09:07, 16.46s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.5975933295174256, 'learning_rate': 7e-07, 'completion_length': 145.64286041259766, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.15943221747875214, 'kl': 0.010650634765625, 'epoch': 1.5}

 30%|███       | 483/1610 [3:04:04<5:09:07, 16.46s/it]
 30%|███       | 484/1610 [3:04:21<5:11:21, 16.59s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.970319251675595, 'learning_rate': 6.993788819875777e-07, 'completion_length': 160.89286041259766, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428619325161, 'kl': 0.012115478515625, 'epoch': 1.5}

 30%|███       | 484/1610 [3:04:21<5:11:21, 16.59s/it]
 30%|███       | 485/1610 [3:04:42<5:35:42, 17.90s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2401062738583535, 'learning_rate': 6.987577639751553e-07, 'completion_length': 194.7321548461914, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.732142984867096, 'reward_std': 0.36266787350177765, 'kl': 0.01300048828125, 'epoch': 1.51}

 30%|███       | 485/1610 [3:04:42<5:35:42, 17.90s/it]
 30%|███       | 486/1610 [3:04:56<5:14:18, 16.78s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1478395626544546, 'learning_rate': 6.981366459627329e-07, 'completion_length': 145.5, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.2610500454902649, 'kl': 0.01129150390625, 'epoch': 1.51}

 30%|███       | 486/1610 [3:04:56<5:14:18, 16.78s/it]
 30%|███       | 487/1610 [3:05:15<5:27:04, 17.48s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.7878948896977296, 'learning_rate': 6.975155279503105e-07, 'completion_length': 177.10714721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178571939468384, 'reward_std': 0.1896214783191681, 'kl': 0.013702392578125, 'epoch': 1.51}

 30%|███       | 487/1610 [3:05:15<5:27:04, 17.48s/it]
 30%|███       | 488/1610 [3:05:35<5:40:50, 18.23s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7302265084819752, 'learning_rate': 6.968944099378881e-07, 'completion_length': 199.4464340209961, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.2610500305891037, 'kl': 0.011474609375, 'epoch': 1.52}

 30%|███       | 488/1610 [3:05:35<5:40:50, 18.23s/it]
 30%|███       | 489/1610 [3:05:52<5:33:09, 17.83s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.1489044787818465, 'learning_rate': 6.962732919254658e-07, 'completion_length': 157.3214340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23086076974868774, 'kl': 0.01422119140625, 'epoch': 1.52}

 30%|███       | 489/1610 [3:05:52<5:33:09, 17.83s/it]
 30%|███       | 490/1610 [3:06:10<5:36:09, 18.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6061892198449332, 'learning_rate': 6.956521739130434e-07, 'completion_length': 201.35714721679688, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.20117833465337753, 'kl': 0.012542724609375, 'epoch': 1.52}

 30%|███       | 490/1610 [3:06:10<5:36:09, 18.01s/it]
 30%|███       | 491/1610 [3:06:30<5:45:45, 18.54s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5869960777096556, 'learning_rate': 6.95031055900621e-07, 'completion_length': 174.30358123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.1539071872830391, 'kl': 0.011932373046875, 'epoch': 1.52}

 30%|███       | 491/1610 [3:06:30<5:45:45, 18.54s/it]
 31%|███       | 492/1610 [3:06:42<5:10:50, 16.68s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9115196667257238, 'learning_rate': 6.944099378881987e-07, 'completion_length': 127.35714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.23086077719926834, 'kl': 0.0157470703125, 'epoch': 1.53}

 31%|███       | 492/1610 [3:06:42<5:10:50, 16.68s/it]
 31%|███       | 493/1610 [3:07:01<5:20:44, 17.23s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 2.6168675557497214, 'learning_rate': 6.937888198757764e-07, 'completion_length': 162.85714721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.14838216826319695, 'kl': 0.015411376953125, 'epoch': 1.53}

 31%|███       | 493/1610 [3:07:01<5:20:44, 17.23s/it]
 31%|███       | 494/1610 [3:07:19<5:25:58, 17.53s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7758746813854572, 'learning_rate': 6.93167701863354e-07, 'completion_length': 177.12500762939453, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178572535514832, 'reward_std': 0.2610500380396843, 'kl': 0.01202392578125, 'epoch': 1.53}

 31%|███       | 494/1610 [3:07:19<5:25:58, 17.53s/it]
 31%|███       | 495/1610 [3:07:38<5:31:55, 17.86s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3457153015674168, 'learning_rate': 6.925465838509317e-07, 'completion_length': 179.7321548461914, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.013763427734375, 'epoch': 1.54}

 31%|███       | 495/1610 [3:07:38<5:31:55, 17.86s/it]
 31%|███       | 496/1610 [3:07:51<5:05:32, 16.46s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7301325043903074, 'learning_rate': 6.919254658385093e-07, 'completion_length': 145.75000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.012786865234375, 'epoch': 1.54}

 31%|███       | 496/1610 [3:07:51<5:05:32, 16.46s/it]
 31%|███       | 497/1610 [3:08:06<4:59:52, 16.17s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.5707516384689335, 'learning_rate': 6.913043478260869e-07, 'completion_length': 158.6607208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981074661016464, 'kl': 0.0120849609375, 'epoch': 1.54}

 31%|███       | 497/1610 [3:08:06<4:59:52, 16.17s/it]
 31%|███       | 498/1610 [3:08:20<4:45:26, 15.40s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2794095756579913, 'learning_rate': 6.906832298136646e-07, 'completion_length': 123.96429443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2967643216252327, 'kl': 0.01556396484375, 'epoch': 1.55}

 31%|███       | 498/1610 [3:08:20<4:45:26, 15.40s/it]
 31%|███       | 499/1610 [3:08:33<4:31:33, 14.67s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.8669073799528855, 'learning_rate': 6.900621118012422e-07, 'completion_length': 144.75000762939453, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.01092529296875, 'epoch': 1.55}

 31%|███       | 499/1610 [3:08:33<4:31:33, 14.67s/it]
 31%|███       | 500/1610 [3:08:47<4:30:41, 14.63s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.895371461802126, 'learning_rate': 6.894409937888198e-07, 'completion_length': 150.50000762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2610500305891037, 'kl': 0.012664794921875, 'epoch': 1.55}

 31%|███       | 500/1610 [3:08:47<4:30:41, 14.63s/it]
 31%|███       | 501/1610 [3:12:18<22:34:32, 73.28s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.5798383974783028, 'learning_rate': 6.888198757763975e-07, 'completion_length': 149.10714721679688, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.11266787722706795, 'kl': 0.011993408203125, 'epoch': 1.56}

 31%|███       | 501/1610 [3:12:18<22:34:32, 73.28s/it]
 31%|███       | 502/1610 [3:12:32<17:08:28, 55.69s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.2921584250512268, 'learning_rate': 6.881987577639752e-07, 'completion_length': 150.37500762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2721000984311104, 'kl': 0.015869140625, 'epoch': 1.56}

 31%|███       | 502/1610 [3:12:32<17:08:28, 55.69s/it]
 31%|███       | 503/1610 [3:12:47<13:19:23, 43.33s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.9867347494730931, 'learning_rate': 6.875776397515528e-07, 'completion_length': 162.76786041259766, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.24191081523895264, 'kl': 0.009979248046875, 'epoch': 1.56}

 31%|███       | 503/1610 [3:12:47<13:19:23, 43.33s/it]
 31%|███▏      | 504/1610 [3:13:05<11:00:03, 35.81s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.8842065462861125, 'learning_rate': 6.869565217391305e-07, 'completion_length': 160.7857208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7857143878936768, 'reward_std': 0.2142857238650322, 'kl': 0.012939453125, 'epoch': 1.57}

 31%|███▏      | 504/1610 [3:13:05<11:00:03, 35.81s/it]
 31%|███▏      | 505/1610 [3:13:20<9:03:54, 29.53s/it] 
                                                      
{'loss': 0.0005, 'grad_norm': 2.4665490550793985, 'learning_rate': 6.863354037267081e-07, 'completion_length': 147.33929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.30228933691978455, 'kl': 0.01251220703125, 'epoch': 1.57}

 31%|███▏      | 505/1610 [3:13:20<9:03:54, 29.53s/it]
 31%|███▏      | 506/1610 [3:13:37<7:56:35, 25.90s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7152444629366147, 'learning_rate': 6.857142857142857e-07, 'completion_length': 151.67858123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.11266787722706795, 'kl': 0.011077880859375, 'epoch': 1.57}

 31%|███▏      | 506/1610 [3:13:37<7:56:35, 25.90s/it]
 31%|███▏      | 507/1610 [3:13:51<6:50:27, 22.33s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7655636108702902, 'learning_rate': 6.850931677018634e-07, 'completion_length': 142.14286041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.013885498046875, 'epoch': 1.57}

 31%|███▏      | 507/1610 [3:13:51<6:50:27, 22.33s/it]
 32%|███▏      | 508/1610 [3:14:09<6:23:05, 20.86s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2059847094951845, 'learning_rate': 6.84472049689441e-07, 'completion_length': 178.98214721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.2857142984867096, 'kl': 0.016204833984375, 'epoch': 1.58}

 32%|███▏      | 508/1610 [3:14:09<6:23:05, 20.86s/it]
 32%|███▏      | 509/1610 [3:14:23<5:48:44, 19.00s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9537192879369951, 'learning_rate': 6.838509316770185e-07, 'completion_length': 154.50000381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2142857238650322, 'kl': 0.01666259765625, 'epoch': 1.58}

 32%|███▏      | 509/1610 [3:14:23<5:48:44, 19.00s/it]
 32%|███▏      | 510/1610 [3:14:41<5:43:01, 18.71s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.183743726967962, 'learning_rate': 6.832298136645962e-07, 'completion_length': 185.0714340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6250000596046448, 'reward_std': 0.33496272563934326, 'kl': 0.0125732421875, 'epoch': 1.58}

 32%|███▏      | 510/1610 [3:14:41<5:43:01, 18.71s/it]
 32%|███▏      | 511/1610 [3:14:58<5:29:58, 18.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9750396455932051, 'learning_rate': 6.826086956521738e-07, 'completion_length': 168.67858123779297, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071835577488, 'kl': 0.013214111328125, 'epoch': 1.59}

 32%|███▏      | 511/1610 [3:14:58<5:29:58, 18.01s/it]
 32%|███▏      | 512/1610 [3:15:17<5:34:42, 18.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8304847445403551, 'learning_rate': 6.819875776397515e-07, 'completion_length': 194.17858123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5357143878936768, 'reward_std': 0.25552501529455185, 'kl': 0.013824462890625, 'epoch': 1.59}

 32%|███▏      | 512/1610 [3:15:17<5:34:42, 18.29s/it]
 32%|███▏      | 513/1610 [3:15:32<5:20:14, 17.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7542334254750331, 'learning_rate': 6.813664596273292e-07, 'completion_length': 157.25, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1539071798324585, 'kl': 0.011871337890625, 'epoch': 1.59}

 32%|███▏      | 513/1610 [3:15:32<5:20:14, 17.52s/it]
 32%|███▏      | 514/1610 [3:15:49<5:15:40, 17.28s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9157337243883521, 'learning_rate': 6.807453416149068e-07, 'completion_length': 175.30358123779297, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981074661016464, 'kl': 0.011810302734375, 'epoch': 1.6}

 32%|███▏      | 514/1610 [3:15:49<5:15:40, 17.28s/it]
 32%|███▏      | 515/1610 [3:16:05<5:07:33, 16.85s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2011539616440674, 'learning_rate': 6.801242236024844e-07, 'completion_length': 157.58929443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.01397705078125, 'epoch': 1.6}

 32%|███▏      | 515/1610 [3:16:05<5:07:33, 16.85s/it]
 32%|███▏      | 516/1610 [3:16:21<5:02:44, 16.60s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9171439283096923, 'learning_rate': 6.795031055900621e-07, 'completion_length': 149.08929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.1539071835577488, 'kl': 0.015045166015625, 'epoch': 1.6}

 32%|███▏      | 516/1610 [3:16:21<5:02:44, 16.60s/it]
 32%|███▏      | 517/1610 [3:16:39<5:11:14, 17.09s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6134911092898779, 'learning_rate': 6.788819875776397e-07, 'completion_length': 171.83929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.17098907008767128, 'kl': 0.0118408203125, 'epoch': 1.61}

 32%|███▏      | 517/1610 [3:16:39<5:11:14, 17.09s/it]
 32%|███▏      | 518/1610 [3:16:54<4:55:52, 16.26s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7056692321371109, 'learning_rate': 6.782608695652173e-07, 'completion_length': 159.1428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.013092041015625, 'epoch': 1.61}

 32%|███▏      | 518/1610 [3:16:54<4:55:52, 16.26s/it]
 32%|███▏      | 519/1610 [3:17:12<5:06:34, 16.86s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8105070668986861, 'learning_rate': 6.77639751552795e-07, 'completion_length': 169.35714721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.696428656578064, 'reward_std': 0.2500000074505806, 'kl': 0.01226806640625, 'epoch': 1.61}

 32%|███▏      | 519/1610 [3:17:12<5:06:34, 16.86s/it]
 32%|███▏      | 520/1610 [3:17:29<5:10:02, 17.07s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6869531360479667, 'learning_rate': 6.770186335403726e-07, 'completion_length': 186.58929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.07695359364151955, 'kl': 0.012420654296875, 'epoch': 1.61}

 32%|███▏      | 520/1610 [3:17:29<5:10:02, 17.07s/it]
 32%|███▏      | 521/1610 [3:17:48<5:17:59, 17.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.8032169414770078, 'learning_rate': 6.763975155279503e-07, 'completion_length': 160.44644165039062, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2500000149011612, 'kl': 0.012969970703125, 'epoch': 1.62}

 32%|███▏      | 521/1610 [3:17:48<5:17:59, 17.52s/it]
 32%|███▏      | 522/1610 [3:18:02<4:59:39, 16.52s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4795760403434202, 'learning_rate': 6.75776397515528e-07, 'completion_length': 149.30358123779297, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.31333939731121063, 'kl': 0.014373779296875, 'epoch': 1.62}

 32%|███▏      | 522/1610 [3:18:02<4:59:39, 16.52s/it]
 32%|███▏      | 523/1610 [3:18:20<5:05:19, 16.85s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6596828410941304, 'learning_rate': 6.751552795031056e-07, 'completion_length': 172.69644165039062, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1071428619325161, 'kl': 0.01422119140625, 'epoch': 1.62}

 32%|███▏      | 523/1610 [3:18:20<5:05:19, 16.85s/it]
 33%|███▎      | 524/1610 [3:18:37<5:08:08, 17.02s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.4899536935074598, 'learning_rate': 6.745341614906832e-07, 'completion_length': 174.3214340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.1428571529686451, 'kl': 0.012786865234375, 'epoch': 1.63}

 33%|███▎      | 524/1610 [3:18:37<5:08:08, 17.02s/it]
 33%|███▎      | 525/1610 [3:18:53<5:03:48, 16.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8623741923954757, 'learning_rate': 6.739130434782609e-07, 'completion_length': 161.1964340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2721000909805298, 'kl': 0.013671875, 'epoch': 1.63}

 33%|███▎      | 525/1610 [3:18:53<5:03:48, 16.80s/it]
 33%|███▎      | 526/1610 [3:19:10<5:04:37, 16.86s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2907592595220307, 'learning_rate': 6.732919254658385e-07, 'completion_length': 159.8928680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.1539071835577488, 'kl': 0.012664794921875, 'epoch': 1.63}

 33%|███▎      | 526/1610 [3:19:10<5:04:37, 16.86s/it]
 33%|███▎      | 527/1610 [3:19:28<5:10:28, 17.20s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6480841155993843, 'learning_rate': 6.726708074534161e-07, 'completion_length': 166.73214721679688, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.17098907008767128, 'kl': 0.012176513671875, 'epoch': 1.64}

 33%|███▎      | 527/1610 [3:19:28<5:10:28, 17.20s/it]
 33%|███▎      | 528/1610 [3:19:44<5:00:02, 16.64s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 3.1271122424742566, 'learning_rate': 6.720496894409938e-07, 'completion_length': 141.01786041259766, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.18409645557403564, 'kl': 0.0120849609375, 'epoch': 1.64}

 33%|███▎      | 528/1610 [3:19:44<5:00:02, 16.64s/it]
 33%|███▎      | 529/1610 [3:20:01<5:03:35, 16.85s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0736307929385933, 'learning_rate': 6.714285714285714e-07, 'completion_length': 174.39286041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.32695358991622925, 'kl': 0.011871337890625, 'epoch': 1.64}

 33%|███▎      | 529/1610 [3:20:01<5:03:35, 16.85s/it]
 33%|███▎      | 530/1610 [3:20:19<5:09:54, 17.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.3595773084518278, 'learning_rate': 6.708074534161491e-07, 'completion_length': 216.4821548461914, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.013946533203125, 'epoch': 1.65}

 33%|███▎      | 530/1610 [3:20:19<5:09:54, 17.22s/it]
 33%|███▎      | 531/1610 [3:20:32<4:48:20, 16.03s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8645286678859204, 'learning_rate': 6.701863354037268e-07, 'completion_length': 126.51786422729492, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.016082763671875, 'epoch': 1.65}

 33%|███▎      | 531/1610 [3:20:32<4:48:20, 16.03s/it]
 33%|███▎      | 532/1610 [3:20:48<4:46:29, 15.95s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9033776234596376, 'learning_rate': 6.695652173913044e-07, 'completion_length': 181.76786041259766, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.3078143745660782, 'kl': 0.012176513671875, 'epoch': 1.65}

 33%|███▎      | 532/1610 [3:20:48<4:46:29, 15.95s/it]
 33%|███▎      | 533/1610 [3:21:08<5:08:11, 17.17s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9188754556063864, 'learning_rate': 6.689440993788819e-07, 'completion_length': 183.6071548461914, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.0123291015625, 'epoch': 1.66}

 33%|███▎      | 533/1610 [3:21:08<5:08:11, 17.17s/it]
 33%|███▎      | 534/1610 [3:21:25<5:05:28, 17.03s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 3.463683079905623, 'learning_rate': 6.683229813664595e-07, 'completion_length': 151.96429061889648, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.4974358528852463, 'kl': 0.016357421875, 'epoch': 1.66}

 33%|███▎      | 534/1610 [3:21:25<5:05:28, 17.03s/it]
 33%|███▎      | 535/1610 [3:21:41<4:58:31, 16.66s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6048489816765629, 'learning_rate': 6.677018633540372e-07, 'completion_length': 160.4821548461914, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1181928962469101, 'kl': 0.014739990234375, 'epoch': 1.66}

 33%|███▎      | 535/1610 [3:21:41<4:58:31, 16.66s/it]
 33%|███▎      | 536/1610 [3:21:56<4:52:04, 16.32s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.3368473211095317, 'learning_rate': 6.670807453416148e-07, 'completion_length': 171.4821548461914, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071835577488, 'kl': 0.011627197265625, 'epoch': 1.66}

 33%|███▎      | 536/1610 [3:21:56<4:52:04, 16.32s/it]
 33%|███▎      | 537/1610 [3:22:14<5:00:22, 16.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5371971479290647, 'learning_rate': 6.664596273291924e-07, 'completion_length': 184.12500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.607142984867096, 'reward_std': 0.11266788095235825, 'kl': 0.011749267578125, 'epoch': 1.67}

 33%|███▎      | 537/1610 [3:22:14<5:00:22, 16.80s/it]
 33%|███▎      | 538/1610 [3:22:30<4:55:07, 16.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1321035881272477, 'learning_rate': 6.658385093167701e-07, 'completion_length': 153.10714721679688, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1428571492433548, 'kl': 0.013397216796875, 'epoch': 1.67}

 33%|███▎      | 538/1610 [3:22:30<4:55:07, 16.52s/it]
 33%|███▎      | 539/1610 [3:22:46<4:53:03, 16.42s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.3944126161212331, 'learning_rate': 6.652173913043478e-07, 'completion_length': 161.08928680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.1428571492433548, 'kl': 0.01043701171875, 'epoch': 1.67}

 33%|███▎      | 539/1610 [3:22:46<4:53:03, 16.42s/it]
 34%|███▎      | 540/1610 [3:23:04<5:00:57, 16.88s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2771809602044553, 'learning_rate': 6.645962732919254e-07, 'completion_length': 165.82144165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.3324786201119423, 'kl': 0.0186767578125, 'epoch': 1.68}

 34%|███▎      | 540/1610 [3:23:04<5:00:57, 16.88s/it]
 34%|███▎      | 541/1610 [3:23:20<4:53:34, 16.48s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3052124737395572, 'learning_rate': 6.639751552795031e-07, 'completion_length': 167.0714340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.21981073915958405, 'kl': 0.0150146484375, 'epoch': 1.68}

 34%|███▎      | 541/1610 [3:23:20<4:53:34, 16.48s/it]
 34%|███▎      | 542/1610 [3:23:36<4:54:06, 16.52s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2704197213714048, 'learning_rate': 6.633540372670807e-07, 'completion_length': 185.33929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2610500380396843, 'kl': 0.0146484375, 'epoch': 1.68}

 34%|███▎      | 542/1610 [3:23:36<4:54:06, 16.52s/it]
 34%|███▎      | 543/1610 [3:23:50<4:38:46, 15.68s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.055220882675842, 'learning_rate': 6.627329192546583e-07, 'completion_length': 167.76786041259766, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.2253357470035553, 'kl': 0.015960693359375, 'epoch': 1.69}

 34%|███▎      | 543/1610 [3:23:50<4:38:46, 15.68s/it]
 34%|███▍      | 544/1610 [3:24:07<4:43:03, 15.93s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.7907865546399457, 'learning_rate': 6.62111801242236e-07, 'completion_length': 156.19644165039062, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.25552502274513245, 'kl': 0.010101318359375, 'epoch': 1.69}

 34%|███▍      | 544/1610 [3:24:07<4:43:03, 15.93s/it]
 34%|███▍      | 545/1610 [3:24:20<4:30:31, 15.24s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0083287396686007, 'learning_rate': 6.614906832298136e-07, 'completion_length': 135.48214721679688, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2253357470035553, 'kl': 0.01263427734375, 'epoch': 1.69}

 34%|███▍      | 545/1610 [3:24:20<4:30:31, 15.24s/it]
 34%|███▍      | 546/1610 [3:24:33<4:16:53, 14.49s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 2.5639484153828995, 'learning_rate': 6.608695652173912e-07, 'completion_length': 130.46429061889648, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.011077880859375, 'epoch': 1.7}

 34%|███▍      | 546/1610 [3:24:33<4:16:53, 14.49s/it]
 34%|███▍      | 547/1610 [3:24:49<4:25:04, 14.96s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1583899023233666, 'learning_rate': 6.602484472049689e-07, 'completion_length': 168.51786041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.23086078837513924, 'kl': 0.01568603515625, 'epoch': 1.7}

 34%|███▍      | 547/1610 [3:24:49<4:25:04, 14.96s/it]
 34%|███▍      | 548/1610 [3:25:05<4:29:48, 15.24s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.1180214488942517, 'learning_rate': 6.596273291925466e-07, 'completion_length': 161.60714721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.011627197265625, 'epoch': 1.7}

 34%|███▍      | 548/1610 [3:25:05<4:29:48, 15.24s/it]
 34%|███▍      | 549/1610 [3:25:20<4:26:56, 15.10s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.280615988586841, 'learning_rate': 6.590062111801242e-07, 'completion_length': 172.2321548461914, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.012939453125, 'epoch': 1.7}

 34%|███▍      | 549/1610 [3:25:20<4:26:56, 15.10s/it]
 34%|███▍      | 550/1610 [3:25:38<4:44:30, 16.10s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8503309901779349, 'learning_rate': 6.583850931677019e-07, 'completion_length': 188.2321548461914, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1539071872830391, 'kl': 0.01165771484375, 'epoch': 1.71}

 34%|███▍      | 550/1610 [3:25:38<4:44:30, 16.10s/it]
 34%|███▍      | 551/1610 [3:25:51<4:28:19, 15.20s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9767226380112988, 'learning_rate': 6.577639751552795e-07, 'completion_length': 139.6071548461914, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.26657505333423615, 'kl': 0.010101318359375, 'epoch': 1.71}

 34%|███▍      | 551/1610 [3:25:51<4:28:19, 15.20s/it]
 34%|███▍      | 552/1610 [3:26:08<4:38:32, 15.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8203127193938559, 'learning_rate': 6.571428571428571e-07, 'completion_length': 166.1964340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.23086077719926834, 'kl': 0.011260986328125, 'epoch': 1.71}

 34%|███▍      | 552/1610 [3:26:08<4:38:32, 15.80s/it]
 34%|███▍      | 553/1610 [3:26:24<4:36:37, 15.70s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.3998115523409989, 'learning_rate': 6.565217391304348e-07, 'completion_length': 136.05358123779297, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.19514648616313934, 'kl': 0.010711669921875, 'epoch': 1.72}

 34%|███▍      | 553/1610 [3:26:24<4:36:37, 15.70s/it]
 34%|███▍      | 554/1610 [3:26:40<4:39:22, 15.87s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.25867577700401356, 'learning_rate': 6.559006211180124e-07, 'completion_length': 166.08929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.0357142873108387, 'kl': 0.0105133056640625, 'epoch': 1.72}

 34%|███▍      | 554/1610 [3:26:40<4:39:22, 15.87s/it]
 34%|███▍      | 555/1610 [3:26:57<4:42:50, 16.09s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.8687037953910293, 'learning_rate': 6.5527950310559e-07, 'completion_length': 183.30357360839844, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.3792429566383362, 'kl': 0.016448974609375, 'epoch': 1.72}

 34%|███▍      | 555/1610 [3:26:57<4:42:50, 16.09s/it]
 35%|███▍      | 556/1610 [3:27:12<4:38:03, 15.83s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9977994921948506, 'learning_rate': 6.546583850931676e-07, 'completion_length': 160.53572845458984, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266788095235825, 'kl': 0.013336181640625, 'epoch': 1.73}

 35%|███▍      | 556/1610 [3:27:12<4:38:03, 15.83s/it]
 35%|███▍      | 557/1610 [3:27:25<4:22:32, 14.96s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9017961066439268, 'learning_rate': 6.540372670807453e-07, 'completion_length': 144.71429443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.14838216453790665, 'kl': 0.01287841796875, 'epoch': 1.73}

 35%|███▍      | 557/1610 [3:27:25<4:22:32, 14.96s/it]
 35%|███▍      | 558/1610 [3:27:39<4:16:48, 14.65s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7468764703406395, 'learning_rate': 6.534161490683229e-07, 'completion_length': 172.2857208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428619325161, 'kl': 0.01580810546875, 'epoch': 1.73}

 35%|███▍      | 558/1610 [3:27:39<4:16:48, 14.65s/it]
 35%|███▍      | 559/1610 [3:27:58<4:39:31, 15.96s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.48900314755038604, 'learning_rate': 6.527950310559006e-07, 'completion_length': 167.0714340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.11266788095235825, 'kl': 0.01104736328125, 'epoch': 1.74}

 35%|███▍      | 559/1610 [3:27:58<4:39:31, 15.96s/it]
 35%|███▍      | 560/1610 [3:28:15<4:47:30, 16.43s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.6792912893091176, 'learning_rate': 6.521739130434782e-07, 'completion_length': 163.21428680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.011932373046875, 'epoch': 1.74}

 35%|███▍      | 560/1610 [3:28:15<4:47:30, 16.43s/it]
 35%|███▍      | 561/1610 [3:28:34<4:58:18, 17.06s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3892227637981884, 'learning_rate': 6.515527950310558e-07, 'completion_length': 204.5714340209961, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.0147705078125, 'epoch': 1.74}

 35%|███▍      | 561/1610 [3:28:34<4:58:18, 17.06s/it]
 35%|███▍      | 562/1610 [3:28:50<4:50:55, 16.66s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3792591579211677, 'learning_rate': 6.509316770186335e-07, 'completion_length': 148.2857208251953, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216826319695, 'kl': 0.015380859375, 'epoch': 1.75}

 35%|███▍      | 562/1610 [3:28:50<4:50:55, 16.66s/it]
 35%|███▍      | 563/1610 [3:29:04<4:40:26, 16.07s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.949201492400636, 'learning_rate': 6.503105590062111e-07, 'completion_length': 160.58929443359375, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.011138916015625, 'epoch': 1.75}

 35%|███▍      | 563/1610 [3:29:04<4:40:26, 16.07s/it]
 35%|███▌      | 564/1610 [3:29:20<4:40:32, 16.09s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8869628792713992, 'learning_rate': 6.496894409937887e-07, 'completion_length': 159.33929443359375, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.012176513671875, 'epoch': 1.75}

 35%|███▌      | 564/1610 [3:29:20<4:40:32, 16.09s/it]
 35%|███▌      | 565/1610 [3:29:35<4:33:49, 15.72s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1592724197862232, 'learning_rate': 6.490683229813664e-07, 'completion_length': 153.0357208251953, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1428571492433548, 'kl': 0.011749267578125, 'epoch': 1.75}

 35%|███▌      | 565/1610 [3:29:35<4:33:49, 15.72s/it]
 35%|███▌      | 566/1610 [3:29:51<4:32:05, 15.64s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1423026133041845, 'learning_rate': 6.484472049689441e-07, 'completion_length': 145.8928680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.14838216826319695, 'kl': 0.013153076171875, 'epoch': 1.76}

 35%|███▌      | 566/1610 [3:29:51<4:32:05, 15.64s/it]
 35%|███▌      | 567/1610 [3:30:05<4:23:57, 15.18s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2060622262932374, 'learning_rate': 6.478260869565217e-07, 'completion_length': 144.82144165039062, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2500000149011612, 'kl': 0.0133056640625, 'epoch': 1.76}

 35%|███▌      | 567/1610 [3:30:05<4:23:57, 15.18s/it]
 35%|███▌      | 568/1610 [3:30:24<4:44:26, 16.38s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.165086045991153, 'learning_rate': 6.472049689440994e-07, 'completion_length': 179.25001525878906, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.011871337890625, 'epoch': 1.76}

 35%|███▌      | 568/1610 [3:30:24<4:44:26, 16.38s/it]
 35%|███▌      | 569/1610 [3:30:40<4:41:18, 16.21s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0369618646880034, 'learning_rate': 6.46583850931677e-07, 'completion_length': 135.76786422729492, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1785714365541935, 'kl': 0.016845703125, 'epoch': 1.77}

 35%|███▌      | 569/1610 [3:30:40<4:41:18, 16.21s/it]
 35%|███▌      | 570/1610 [3:30:54<4:29:48, 15.57s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0055384503408509, 'learning_rate': 6.459627329192546e-07, 'completion_length': 149.3928680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2721000760793686, 'kl': 0.01287841796875, 'epoch': 1.77}

 35%|███▌      | 570/1610 [3:30:54<4:29:48, 15.57s/it]
 35%|███▌      | 571/1610 [3:31:13<4:46:30, 16.54s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2292821600740595, 'learning_rate': 6.453416149068323e-07, 'completion_length': 166.26786041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428572535514832, 'reward_std': 0.29924842715263367, 'kl': 0.01409912109375, 'epoch': 1.77}

 35%|███▌      | 571/1610 [3:31:13<4:46:30, 16.54s/it]
 36%|███▌      | 572/1610 [3:31:31<4:56:25, 17.13s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9462132193018752, 'learning_rate': 6.447204968944099e-07, 'completion_length': 200.1428680419922, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1785714365541935, 'kl': 0.01861572265625, 'epoch': 1.78}

 36%|███▌      | 572/1610 [3:31:31<4:56:25, 17.13s/it]
 36%|███▌      | 573/1610 [3:31:48<4:53:21, 16.97s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4098704846021168, 'learning_rate': 6.440993788819875e-07, 'completion_length': 153.4107208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2142857313156128, 'kl': 0.01593017578125, 'epoch': 1.78}

 36%|███▌      | 573/1610 [3:31:48<4:53:21, 16.97s/it]
 36%|███▌      | 574/1610 [3:32:09<5:12:50, 18.12s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.433405859339452, 'learning_rate': 6.434782608695652e-07, 'completion_length': 224.96429443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.910714328289032, 'reward': 1.410714328289032, 'reward_std': 0.4476306140422821, 'kl': 0.0157470703125, 'epoch': 1.78}

 36%|███▌      | 574/1610 [3:32:09<5:12:50, 18.12s/it]
 36%|███▌      | 575/1610 [3:32:24<5:00:53, 17.44s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.22157338979754, 'learning_rate': 6.428571428571429e-07, 'completion_length': 162.25000762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1785714402794838, 'kl': 0.01678466796875, 'epoch': 1.79}

 36%|███▌      | 575/1610 [3:32:24<5:00:53, 17.44s/it]
 36%|███▌      | 576/1610 [3:32:40<4:52:32, 16.98s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.917819869884002, 'learning_rate': 6.422360248447205e-07, 'completion_length': 158.08929443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5357143878936768, 'reward_std': 0.21222837269306183, 'kl': 0.013916015625, 'epoch': 1.79}

 36%|███▌      | 576/1610 [3:32:40<4:52:32, 16.98s/it]
 36%|███▌      | 577/1610 [3:32:59<4:59:06, 17.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7870667543750951, 'learning_rate': 6.416149068322982e-07, 'completion_length': 193.9107208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.015228271484375, 'epoch': 1.79}

 36%|███▌      | 577/1610 [3:32:59<4:59:06, 17.37s/it]
 36%|███▌      | 578/1610 [3:33:14<4:46:15, 16.64s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2192048583182467, 'learning_rate': 6.409937888198758e-07, 'completion_length': 142.33929061889648, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.14838216453790665, 'kl': 0.012725830078125, 'epoch': 1.8}

 36%|███▌      | 578/1610 [3:33:14<4:46:15, 16.64s/it]
 36%|███▌      | 579/1610 [3:33:30<4:43:55, 16.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1004349882028395, 'learning_rate': 6.403726708074534e-07, 'completion_length': 154.12500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.25552501529455185, 'kl': 0.0115966796875, 'epoch': 1.8}

 36%|███▌      | 579/1610 [3:33:30<4:43:55, 16.52s/it]
 36%|███▌      | 580/1610 [3:33:48<4:51:31, 16.98s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.9091884187736131, 'learning_rate': 6.39751552795031e-07, 'completion_length': 174.62500762939453, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.2937234118580818, 'kl': 0.0106201171875, 'epoch': 1.8}

 36%|███▌      | 580/1610 [3:33:48<4:51:31, 16.98s/it]
 36%|███▌      | 581/1610 [3:34:01<4:31:53, 15.85s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.038416726174946, 'learning_rate': 6.391304347826086e-07, 'completion_length': 153.12500762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.2253357619047165, 'kl': 0.013092041015625, 'epoch': 1.8}

 36%|███▌      | 581/1610 [3:34:01<4:31:53, 15.85s/it]
 36%|███▌      | 582/1610 [3:34:17<4:33:45, 15.98s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9713101733309837, 'learning_rate': 6.385093167701862e-07, 'completion_length': 181.19644165039062, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357619047165, 'kl': 0.011688232421875, 'epoch': 1.81}

 36%|███▌      | 582/1610 [3:34:17<4:33:45, 15.98s/it]
 36%|███▌      | 583/1610 [3:34:31<4:22:41, 15.35s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.7255138604543971, 'learning_rate': 6.378881987577639e-07, 'completion_length': 157.12500762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.21981073170900345, 'kl': 0.014556884765625, 'epoch': 1.81}

 36%|███▌      | 583/1610 [3:34:31<4:22:41, 15.35s/it]
 36%|███▋      | 584/1610 [3:34:46<4:19:41, 15.19s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9394272971763572, 'learning_rate': 6.372670807453416e-07, 'completion_length': 156.85714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.07695358991622925, 'kl': 0.01318359375, 'epoch': 1.81}

 36%|███▋      | 584/1610 [3:34:46<4:19:41, 15.19s/it]
 36%|███▋      | 585/1610 [3:35:01<4:20:03, 15.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.5878584765017392, 'learning_rate': 6.366459627329192e-07, 'completion_length': 172.01786041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.2253357656300068, 'kl': 0.01470947265625, 'epoch': 1.82}

 36%|███▋      | 585/1610 [3:35:01<4:20:03, 15.22s/it]
 36%|███▋      | 586/1610 [3:35:13<3:59:56, 14.06s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 6.756200079752855, 'learning_rate': 6.360248447204969e-07, 'completion_length': 128.80357360839844, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.01409912109375, 'epoch': 1.82}

 36%|███▋      | 586/1610 [3:35:13<3:59:56, 14.06s/it]
 36%|███▋      | 587/1610 [3:35:29<4:10:23, 14.69s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5165535161232665, 'learning_rate': 6.354037267080745e-07, 'completion_length': 151.73214721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.0714285746216774, 'kl': 0.015899658203125, 'epoch': 1.82}

 36%|███▋      | 587/1610 [3:35:29<4:10:23, 14.69s/it]
 37%|███▋      | 588/1610 [3:35:44<4:13:23, 14.88s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7664445530982114, 'learning_rate': 6.347826086956521e-07, 'completion_length': 144.55357360839844, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266788095235825, 'kl': 0.015228271484375, 'epoch': 1.83}

 37%|███▋      | 588/1610 [3:35:44<4:13:23, 14.88s/it]
 37%|███▋      | 589/1610 [3:36:02<4:26:25, 15.66s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.4596872885947829, 'learning_rate': 6.341614906832298e-07, 'completion_length': 163.28572845458984, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266787722706795, 'kl': 0.01318359375, 'epoch': 1.83}

 37%|███▋      | 589/1610 [3:36:02<4:26:25, 15.66s/it]
 37%|███▋      | 590/1610 [3:36:18<4:31:03, 15.94s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7386504464402518, 'learning_rate': 6.335403726708074e-07, 'completion_length': 161.4107208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2363857924938202, 'kl': 0.013763427734375, 'epoch': 1.83}

 37%|███▋      | 590/1610 [3:36:18<4:31:03, 15.94s/it]
 37%|███▋      | 591/1610 [3:36:35<4:33:48, 16.12s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1251861935755716, 'learning_rate': 6.32919254658385e-07, 'completion_length': 152.42858123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981074661016464, 'kl': 0.015380859375, 'epoch': 1.84}

 37%|███▋      | 591/1610 [3:36:35<4:33:48, 16.12s/it]
 37%|███▋      | 592/1610 [3:36:51<4:33:57, 16.15s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5348116845856775, 'learning_rate': 6.322981366459627e-07, 'completion_length': 171.4464340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1181928962469101, 'kl': 0.0130615234375, 'epoch': 1.84}

 37%|███▋      | 592/1610 [3:36:51<4:33:57, 16.15s/it]
 37%|███▋      | 593/1610 [3:37:07<4:35:13, 16.24s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.327793658034332, 'learning_rate': 6.316770186335404e-07, 'completion_length': 159.94644165039062, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.015533447265625, 'epoch': 1.84}

 37%|███▋      | 593/1610 [3:37:07<4:35:13, 16.24s/it]
 37%|███▋      | 594/1610 [3:37:26<4:44:52, 16.82s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.3017896456102704, 'learning_rate': 6.31055900621118e-07, 'completion_length': 156.5357208251953, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.732142984867096, 'reward_std': 0.2937234044075012, 'kl': 0.0093994140625, 'epoch': 1.84}

 37%|███▋      | 594/1610 [3:37:26<4:44:52, 16.82s/it]
 37%|███▋      | 595/1610 [3:37:43<4:44:45, 16.83s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.47800676723324775, 'learning_rate': 6.304347826086957e-07, 'completion_length': 157.55358123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.07695358991622925, 'kl': 0.01153564453125, 'epoch': 1.85}

 37%|███▋      | 595/1610 [3:37:43<4:44:45, 16.83s/it]
 37%|███▋      | 596/1610 [3:37:56<4:26:55, 15.79s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9332753104155493, 'learning_rate': 6.298136645962733e-07, 'completion_length': 139.26786041259766, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.0147705078125, 'epoch': 1.85}

 37%|███▋      | 596/1610 [3:37:56<4:26:55, 15.79s/it]
 37%|███▋      | 597/1610 [3:38:11<4:20:56, 15.46s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5824328420188645, 'learning_rate': 6.291925465838509e-07, 'completion_length': 143.6964340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1181928962469101, 'kl': 0.01220703125, 'epoch': 1.85}

 37%|███▋      | 597/1610 [3:38:11<4:20:56, 15.46s/it]
 37%|███▋      | 598/1610 [3:38:24<4:08:28, 14.73s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1189700624461316, 'learning_rate': 6.285714285714286e-07, 'completion_length': 145.58929061889648, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25552502274513245, 'kl': 0.01263427734375, 'epoch': 1.86}

 37%|███▋      | 598/1610 [3:38:24<4:08:28, 14.73s/it]
 37%|███▋      | 599/1610 [3:38:37<4:00:44, 14.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.4820445649121837, 'learning_rate': 6.279503105590062e-07, 'completion_length': 134.75000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.07695359364151955, 'kl': 0.015289306640625, 'epoch': 1.86}

 37%|███▋      | 599/1610 [3:38:37<4:00:44, 14.29s/it]
 37%|███▋      | 600/1610 [3:38:52<4:04:48, 14.54s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3357798715187503, 'learning_rate': 6.273291925465838e-07, 'completion_length': 187.71429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2610500454902649, 'kl': 0.01617431640625, 'epoch': 1.86}

 37%|███▋      | 600/1610 [3:38:52<4:04:48, 14.54s/it]
 37%|███▋      | 601/1610 [3:43:10<24:31:01, 87.47s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.9716474570063147, 'learning_rate': 6.267080745341615e-07, 'completion_length': 176.46429443359375, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.0115966796875, 'epoch': 1.87}

 37%|███▋      | 601/1610 [3:43:10<24:31:01, 87.47s/it]
 37%|███▋      | 602/1610 [3:43:24<18:23:03, 65.66s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 2.326983648297545, 'learning_rate': 6.260869565217392e-07, 'completion_length': 156.51786422729492, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1181928962469101, 'kl': 0.013275146484375, 'epoch': 1.87}

 37%|███▋      | 602/1610 [3:43:24<18:23:03, 65.66s/it]
 37%|███▋      | 603/1610 [3:43:43<14:23:44, 51.46s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 3.055834616536448, 'learning_rate': 6.254658385093168e-07, 'completion_length': 208.42858123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.30228935927152634, 'kl': 0.015960693359375, 'epoch': 1.87}

 37%|███▋      | 603/1610 [3:43:43<14:23:44, 51.46s/it]
 38%|███▊      | 604/1610 [3:43:59<11:23:18, 40.75s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9114235044785814, 'learning_rate': 6.248447204968945e-07, 'completion_length': 179.4464340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6071429252624512, 'reward_std': 0.21676981449127197, 'kl': 0.01348876953125, 'epoch': 1.88}

 38%|███▊      | 604/1610 [3:43:59<11:23:18, 40.75s/it]
 38%|███▊      | 605/1610 [3:44:17<9:29:01, 33.97s/it] 
                                                      
{'loss': 0.0004, 'grad_norm': 0.6900629278247705, 'learning_rate': 6.24223602484472e-07, 'completion_length': 215.98214721679688, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.26657506078481674, 'kl': 0.00958251953125, 'epoch': 1.88}

 38%|███▊      | 605/1610 [3:44:17<9:29:01, 33.97s/it]
 38%|███▊      | 606/1610 [3:44:32<7:53:32, 28.30s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.2980827916453832, 'learning_rate': 6.236024844720496e-07, 'completion_length': 171.8214340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.21981074661016464, 'kl': 0.010101318359375, 'epoch': 1.88}

 38%|███▊      | 606/1610 [3:44:32<7:53:32, 28.30s/it]
 38%|███▊      | 607/1610 [3:44:49<6:58:42, 25.05s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.4288206065573494, 'learning_rate': 6.229813664596273e-07, 'completion_length': 168.25000762939453, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.26657506078481674, 'kl': 0.01898193359375, 'epoch': 1.89}

 38%|███▊      | 607/1610 [3:44:49<6:58:42, 25.05s/it]
 38%|███▊      | 608/1610 [3:45:06<6:15:52, 22.51s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 12.29036981796265, 'learning_rate': 6.223602484472049e-07, 'completion_length': 146.0714340209961, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.014892578125, 'epoch': 1.89}

 38%|███▊      | 608/1610 [3:45:06<6:15:52, 22.51s/it]
 38%|███▊      | 609/1610 [3:45:21<5:40:27, 20.41s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0475370042943706, 'learning_rate': 6.217391304347825e-07, 'completion_length': 182.32144165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.25552502274513245, 'kl': 0.016357421875, 'epoch': 1.89}

 38%|███▊      | 609/1610 [3:45:21<5:40:27, 20.41s/it]
 38%|███▊      | 610/1610 [3:45:40<5:30:49, 19.85s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.9469492752400688, 'learning_rate': 6.211180124223601e-07, 'completion_length': 195.9821548461914, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.012786865234375, 'epoch': 1.89}

 38%|███▊      | 610/1610 [3:45:40<5:30:49, 19.85s/it]
 38%|███▊      | 611/1610 [3:45:55<5:07:38, 18.48s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6655357291081702, 'learning_rate': 6.204968944099379e-07, 'completion_length': 145.37500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514649361371994, 'kl': 0.013031005859375, 'epoch': 1.9}

 38%|███▊      | 611/1610 [3:45:55<5:07:38, 18.48s/it]
 38%|███▊      | 612/1610 [3:46:10<4:50:03, 17.44s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.6905126572524585, 'learning_rate': 6.198757763975155e-07, 'completion_length': 154.7857208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.30228933691978455, 'kl': 0.01519775390625, 'epoch': 1.9}

 38%|███▊      | 612/1610 [3:46:10<4:50:03, 17.44s/it]
 38%|███▊      | 613/1610 [3:46:26<4:42:55, 17.03s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1968440987492113, 'learning_rate': 6.192546583850932e-07, 'completion_length': 156.55358123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714402794838, 'kl': 0.014678955078125, 'epoch': 1.9}

 38%|███▊      | 613/1610 [3:46:26<4:42:55, 17.03s/it]
 38%|███▊      | 614/1610 [3:46:39<4:20:29, 15.69s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.7038122354633196, 'learning_rate': 6.186335403726708e-07, 'completion_length': 138.80358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.012847900390625, 'epoch': 1.91}

 38%|███▊      | 614/1610 [3:46:39<4:20:29, 15.69s/it]
 38%|███▊      | 615/1610 [3:46:56<4:27:01, 16.10s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7773690549640706, 'learning_rate': 6.180124223602484e-07, 'completion_length': 197.28572845458984, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2721000984311104, 'kl': 0.01263427734375, 'epoch': 1.91}

 38%|███▊      | 615/1610 [3:46:56<4:27:01, 16.10s/it]
 38%|███▊      | 616/1610 [3:47:13<4:33:33, 16.51s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7109508020039308, 'learning_rate': 6.17391304347826e-07, 'completion_length': 177.46429443359375, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.1539071798324585, 'kl': 0.01239013671875, 'epoch': 1.91}

 38%|███▊      | 616/1610 [3:47:13<4:33:33, 16.51s/it]
 38%|███▊      | 617/1610 [3:47:28<4:25:39, 16.05s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8200208524075095, 'learning_rate': 6.167701863354037e-07, 'completion_length': 158.0714340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.01385498046875, 'epoch': 1.92}

 38%|███▊      | 617/1610 [3:47:28<4:25:39, 16.05s/it]
 38%|███▊      | 618/1610 [3:47:46<4:36:02, 16.70s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2019816446128382, 'learning_rate': 6.161490683229813e-07, 'completion_length': 162.00000762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357544541359, 'kl': 0.01654052734375, 'epoch': 1.92}

 38%|███▊      | 618/1610 [3:47:46<4:36:02, 16.70s/it]
 38%|███▊      | 619/1610 [3:48:04<4:38:26, 16.86s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0082739695245666, 'learning_rate': 6.15527950310559e-07, 'completion_length': 179.75000762939453, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2142857238650322, 'kl': 0.01568603515625, 'epoch': 1.92}

 38%|███▊      | 619/1610 [3:48:04<4:38:26, 16.86s/it]
 39%|███▊      | 620/1610 [3:48:20<4:33:09, 16.56s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9533476090033707, 'learning_rate': 6.149068322981367e-07, 'completion_length': 176.4464340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071798324585, 'kl': 0.0130615234375, 'epoch': 1.93}

 39%|███▊      | 620/1610 [3:48:20<4:33:09, 16.56s/it]
 39%|███▊      | 621/1610 [3:48:36<4:33:55, 16.62s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.5401045688095892, 'learning_rate': 6.142857142857143e-07, 'completion_length': 184.30358123779297, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1071428619325161, 'kl': 0.019439697265625, 'epoch': 1.93}

 39%|███▊      | 621/1610 [3:48:36<4:33:55, 16.62s/it]
 39%|███▊      | 622/1610 [3:48:51<4:23:54, 16.03s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.11898609440946, 'learning_rate': 6.13664596273292e-07, 'completion_length': 151.0535774230957, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1785714402794838, 'kl': 0.01080322265625, 'epoch': 1.93}

 39%|███▊      | 622/1610 [3:48:51<4:23:54, 16.03s/it]
 39%|███▊      | 623/1610 [3:49:06<4:21:05, 15.87s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4720968596953719, 'learning_rate': 6.130434782608696e-07, 'completion_length': 160.7857208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357619047165, 'kl': 0.01385498046875, 'epoch': 1.93}

 39%|███▊      | 623/1610 [3:49:06<4:21:05, 15.87s/it]
 39%|███▉      | 624/1610 [3:49:22<4:20:57, 15.88s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9816016096109075, 'learning_rate': 6.124223602484472e-07, 'completion_length': 178.7857208251953, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.14838217198848724, 'kl': 0.01300048828125, 'epoch': 1.94}

 39%|███▉      | 624/1610 [3:49:22<4:20:57, 15.88s/it]
 39%|███▉      | 625/1610 [3:49:43<4:42:36, 17.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9741194266290374, 'learning_rate': 6.118012422360248e-07, 'completion_length': 198.4821548461914, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8035715222358704, 'reward_std': 0.1896214671432972, 'kl': 0.0142822265625, 'epoch': 1.94}

 39%|███▉      | 625/1610 [3:49:43<4:42:36, 17.22s/it]
 39%|███▉      | 626/1610 [3:49:57<4:26:15, 16.23s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9973880800396221, 'learning_rate': 6.111801242236025e-07, 'completion_length': 143.9821548461914, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071835577488, 'kl': 0.0146484375, 'epoch': 1.94}

 39%|███▉      | 626/1610 [3:49:57<4:26:15, 16.23s/it]
 39%|███▉      | 627/1610 [3:50:12<4:21:52, 15.98s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8461707862648221, 'learning_rate': 6.105590062111801e-07, 'completion_length': 150.0357208251953, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.2500000074505806, 'kl': 0.013671875, 'epoch': 1.95}

 39%|███▉      | 627/1610 [3:50:12<4:21:52, 15.98s/it]
 39%|███▉      | 628/1610 [3:50:30<4:29:40, 16.48s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1803305686077257, 'learning_rate': 6.099378881987576e-07, 'completion_length': 188.96429443359375, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.26657505333423615, 'kl': 0.013946533203125, 'epoch': 1.95}

 39%|███▉      | 628/1610 [3:50:30<4:29:40, 16.48s/it]
 39%|███▉      | 629/1610 [3:50:49<4:41:29, 17.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6064914681880826, 'learning_rate': 6.093167701863354e-07, 'completion_length': 183.7321548461914, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.01483154296875, 'epoch': 1.95}

 39%|███▉      | 629/1610 [3:50:49<4:41:29, 17.22s/it]
 39%|███▉      | 630/1610 [3:51:01<4:17:09, 15.74s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3833357039520238, 'learning_rate': 6.08695652173913e-07, 'completion_length': 128.28571701049805, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.21981073170900345, 'kl': 0.014678955078125, 'epoch': 1.96}

 39%|███▉      | 630/1610 [3:51:01<4:17:09, 15.74s/it]
 39%|███▉      | 631/1610 [3:51:18<4:22:03, 16.06s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7635610471111571, 'learning_rate': 6.080745341614906e-07, 'completion_length': 174.8571548461914, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.01275634765625, 'epoch': 1.96}

 39%|███▉      | 631/1610 [3:51:18<4:22:03, 16.06s/it]
 39%|███▉      | 632/1610 [3:51:33<4:19:23, 15.91s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0953901812623974, 'learning_rate': 6.074534161490683e-07, 'completion_length': 184.92858123779297, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2967643290758133, 'kl': 0.013092041015625, 'epoch': 1.96}

 39%|███▉      | 632/1610 [3:51:33<4:19:23, 15.91s/it]
 39%|███▉      | 633/1610 [3:51:44<3:54:10, 14.38s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9967926064455107, 'learning_rate': 6.068322981366459e-07, 'completion_length': 117.08929061889648, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.1428571492433548, 'kl': 0.013824462890625, 'epoch': 1.97}

 39%|███▉      | 633/1610 [3:51:44<3:54:10, 14.38s/it]
 39%|███▉      | 634/1610 [3:51:59<3:58:44, 14.68s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3034525347250734, 'learning_rate': 6.062111801242235e-07, 'completion_length': 165.9464340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.14838217198848724, 'kl': 0.013885498046875, 'epoch': 1.97}

 39%|███▉      | 634/1610 [3:51:59<3:58:44, 14.68s/it]
 39%|███▉      | 635/1610 [3:52:15<4:00:40, 14.81s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.780531975619544, 'learning_rate': 6.055900621118012e-07, 'completion_length': 143.23214721679688, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1896214783191681, 'kl': 0.015106201171875, 'epoch': 1.97}

 39%|███▉      | 635/1610 [3:52:15<4:00:40, 14.81s/it]
 40%|███▉      | 636/1610 [3:52:28<3:55:19, 14.50s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5986720864199628, 'learning_rate': 6.049689440993788e-07, 'completion_length': 149.12500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.0714285746216774, 'kl': 0.011932373046875, 'epoch': 1.98}

 40%|███▉      | 636/1610 [3:52:28<3:55:19, 14.50s/it]
 40%|███▉      | 637/1610 [3:52:47<4:15:12, 15.74s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.6646453761894355, 'learning_rate': 6.043478260869564e-07, 'completion_length': 163.17857360839844, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.19514649361371994, 'kl': 0.01171875, 'epoch': 1.98}

 40%|███▉      | 637/1610 [3:52:47<4:15:12, 15.74s/it]
 40%|███▉      | 638/1610 [3:53:05<4:26:35, 16.46s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4098184229844535, 'learning_rate': 6.037267080745342e-07, 'completion_length': 151.87500762939453, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.26657505333423615, 'kl': 0.015472412109375, 'epoch': 1.98}

 40%|███▉      | 638/1610 [3:53:05<4:26:35, 16.46s/it]
 40%|███▉      | 639/1610 [3:53:21<4:23:39, 16.29s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.3712366965665152, 'learning_rate': 6.031055900621118e-07, 'completion_length': 159.12500762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.0714285746216774, 'kl': 0.0089111328125, 'epoch': 1.98}

 40%|███▉      | 639/1610 [3:53:21<4:23:39, 16.29s/it]
 40%|███▉      | 640/1610 [3:53:38<4:24:26, 16.36s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.985787140880161, 'learning_rate': 6.024844720496894e-07, 'completion_length': 163.2857208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2142857238650322, 'kl': 0.01800537109375, 'epoch': 1.99}

 40%|███▉      | 640/1610 [3:53:38<4:24:26, 16.36s/it]
 40%|███▉      | 641/1610 [3:53:58<4:43:18, 17.54s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3829814597578747, 'learning_rate': 6.018633540372671e-07, 'completion_length': 173.98214721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.535714328289032, 'reward_std': 0.25552502274513245, 'kl': 0.01617431640625, 'epoch': 1.99}

 40%|███▉      | 641/1610 [3:53:58<4:43:18, 17.54s/it]
 40%|███▉      | 642/1610 [3:54:18<4:54:35, 18.26s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8751203372214605, 'learning_rate': 6.012422360248447e-07, 'completion_length': 165.75000762939453, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4821429252624512, 'reward_std': 0.2610500454902649, 'kl': 0.0137939453125, 'epoch': 1.99}

 40%|███▉      | 642/1610 [3:54:18<4:54:35, 18.26s/it]
 40%|███▉      | 643/1610 [3:54:33<4:38:44, 17.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6279097857378633, 'learning_rate': 6.006211180124223e-07, 'completion_length': 155.5357208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.1428571492433548, 'kl': 0.015472412109375, 'epoch': 2.0}

 40%|███▉      | 643/1610 [3:54:33<4:38:44, 17.29s/it]
 40%|████      | 644/1610 [3:54:50<4:36:42, 17.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4173066939938168, 'learning_rate': 6e-07, 'completion_length': 173.69644165039062, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.2253357544541359, 'kl': 0.01458740234375, 'epoch': 2.0}

 40%|████      | 644/1610 [3:54:50<4:36:42, 17.19s/it]
 40%|████      | 645/1610 [3:55:07<4:36:17, 17.18s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4989470751267602, 'learning_rate': 5.993788819875776e-07, 'completion_length': 166.7857208251953, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.1539071798324585, 'kl': 0.015045166015625, 'epoch': 2.0}

 40%|████      | 645/1610 [3:55:07<4:36:17, 17.18s/it]
 40%|████      | 646/1610 [3:55:23<4:31:51, 16.92s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.1969268817928236, 'learning_rate': 5.987577639751552e-07, 'completion_length': 164.5357208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.0107879638671875, 'epoch': 2.01}

 40%|████      | 646/1610 [3:55:23<4:31:51, 16.92s/it]
 40%|████      | 647/1610 [3:55:43<4:43:24, 17.66s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9048871286478752, 'learning_rate': 5.98136645962733e-07, 'completion_length': 182.19644165039062, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.31333939731121063, 'kl': 0.0152587890625, 'epoch': 2.01}

 40%|████      | 647/1610 [3:55:43<4:43:24, 17.66s/it]
 40%|████      | 648/1610 [3:55:57<4:28:35, 16.75s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8961607943894677, 'learning_rate': 5.975155279503106e-07, 'completion_length': 141.9464340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.29123930633068085, 'kl': 0.01611328125, 'epoch': 2.01}

 40%|████      | 648/1610 [3:55:57<4:28:35, 16.75s/it]
 40%|████      | 649/1610 [3:56:14<4:27:50, 16.72s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7043532093508736, 'learning_rate': 5.968944099378882e-07, 'completion_length': 155.25000762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.11266788095235825, 'kl': 0.01568603515625, 'epoch': 2.02}

 40%|████      | 649/1610 [3:56:14<4:27:50, 16.72s/it]
 40%|████      | 650/1610 [3:56:30<4:22:32, 16.41s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7306175017329188, 'learning_rate': 5.962732919254659e-07, 'completion_length': 157.30358123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216826319695, 'kl': 0.0126953125, 'epoch': 2.02}

 40%|████      | 650/1610 [3:56:30<4:22:32, 16.41s/it]
 40%|████      | 651/1610 [3:56:47<4:26:39, 16.68s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.0897814783279298, 'learning_rate': 5.956521739130435e-07, 'completion_length': 174.96429443359375, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.446428656578064, 'reward_std': 0.24794266000390053, 'kl': 0.0189208984375, 'epoch': 2.02}

 40%|████      | 651/1610 [3:56:47<4:26:39, 16.68s/it]
 40%|████      | 652/1610 [3:57:03<4:23:46, 16.52s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8658165000959333, 'learning_rate': 5.95031055900621e-07, 'completion_length': 151.10714721679688, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714402794838, 'kl': 0.016845703125, 'epoch': 2.02}

 40%|████      | 652/1610 [3:57:03<4:23:46, 16.52s/it]
 41%|████      | 653/1610 [3:57:18<4:16:19, 16.07s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.811659086590266, 'learning_rate': 5.944099378881987e-07, 'completion_length': 155.6964340209961, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.012786865234375, 'epoch': 2.03}

 41%|████      | 653/1610 [3:57:18<4:16:19, 16.07s/it]
 41%|████      | 654/1610 [3:57:33<4:08:21, 15.59s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6083436580261986, 'learning_rate': 5.937888198757763e-07, 'completion_length': 149.21429443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266787722706795, 'kl': 0.012847900390625, 'epoch': 2.03}

 41%|████      | 654/1610 [3:57:33<4:08:21, 15.59s/it]
 41%|████      | 655/1610 [3:57:47<4:01:28, 15.17s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1105233666105456, 'learning_rate': 5.931677018633539e-07, 'completion_length': 161.7857208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1896214708685875, 'kl': 0.012115478515625, 'epoch': 2.03}

 41%|████      | 655/1610 [3:57:47<4:01:28, 15.17s/it]
 41%|████      | 656/1610 [3:58:00<3:54:26, 14.74s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6051837582769276, 'learning_rate': 5.925465838509317e-07, 'completion_length': 148.9107208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.0174560546875, 'epoch': 2.04}

 41%|████      | 656/1610 [3:58:00<3:54:26, 14.74s/it]
 41%|████      | 657/1610 [3:58:17<4:01:14, 15.19s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0588595088020696, 'learning_rate': 5.919254658385093e-07, 'completion_length': 167.23214721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.26353414356708527, 'kl': 0.0135498046875, 'epoch': 2.04}

 41%|████      | 657/1610 [3:58:17<4:01:14, 15.19s/it]
 41%|████      | 658/1610 [3:58:35<4:18:02, 16.26s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5769528467993551, 'learning_rate': 5.913043478260869e-07, 'completion_length': 204.9464340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.01409912109375, 'epoch': 2.04}

 41%|████      | 658/1610 [3:58:35<4:18:02, 16.26s/it]
 41%|████      | 659/1610 [3:58:52<4:16:49, 16.20s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 13.377639377252395, 'learning_rate': 5.906832298136646e-07, 'completion_length': 156.23214721679688, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409644439816475, 'kl': 0.013946533203125, 'epoch': 2.05}

 41%|████      | 659/1610 [3:58:52<4:16:49, 16.20s/it]
 41%|████      | 660/1610 [3:59:07<4:11:22, 15.88s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7054182332044848, 'learning_rate': 5.900621118012422e-07, 'completion_length': 167.17858123779297, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1896214708685875, 'kl': 0.015838623046875, 'epoch': 2.05}

 41%|████      | 660/1610 [3:59:07<4:11:22, 15.88s/it]
 41%|████      | 661/1610 [3:59:20<3:58:32, 15.08s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8731553719086994, 'learning_rate': 5.894409937888198e-07, 'completion_length': 139.55358123779297, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.01544189453125, 'epoch': 2.05}

 41%|████      | 661/1610 [3:59:20<3:58:32, 15.08s/it]
 41%|████      | 662/1610 [3:59:36<4:04:18, 15.46s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7451679175462991, 'learning_rate': 5.888198757763975e-07, 'completion_length': 154.08929443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216826319695, 'kl': 0.0150146484375, 'epoch': 2.06}

 41%|████      | 662/1610 [3:59:36<4:04:18, 15.46s/it]
 41%|████      | 663/1610 [3:59:52<4:03:24, 15.42s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.9179005414746126, 'learning_rate': 5.881987577639751e-07, 'completion_length': 163.73214721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2142857313156128, 'kl': 0.019927978515625, 'epoch': 2.06}

 41%|████      | 663/1610 [3:59:52<4:03:24, 15.42s/it]
 41%|████      | 664/1610 [4:00:09<4:11:37, 15.96s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.6947983445305828, 'learning_rate': 5.875776397515527e-07, 'completion_length': 165.00000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409644439816475, 'kl': 0.019134521484375, 'epoch': 2.06}

 41%|████      | 664/1610 [4:00:09<4:11:37, 15.96s/it]
 41%|████▏     | 665/1610 [4:00:25<4:14:53, 16.18s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.6468392232809986, 'learning_rate': 5.869565217391305e-07, 'completion_length': 157.98214721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.14838216453790665, 'kl': 0.011016845703125, 'epoch': 2.07}

 41%|████▏     | 665/1610 [4:00:25<4:14:53, 16.18s/it]
 41%|████▏     | 666/1610 [4:00:44<4:25:52, 16.90s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.361399254665687, 'learning_rate': 5.863354037267081e-07, 'completion_length': 194.62500762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357470035553, 'kl': 0.014739990234375, 'epoch': 2.07}

 41%|████▏     | 666/1610 [4:00:44<4:25:52, 16.90s/it]
 41%|████▏     | 667/1610 [4:00:59<4:17:24, 16.38s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4639323970228908, 'learning_rate': 5.857142857142857e-07, 'completion_length': 143.42858123779297, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.01556396484375, 'epoch': 2.07}

 41%|████▏     | 667/1610 [4:00:59<4:17:24, 16.38s/it]
 41%|████▏     | 668/1610 [4:01:14<4:09:57, 15.92s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.9526121426210108, 'learning_rate': 5.850931677018634e-07, 'completion_length': 171.5357208251953, 'rewards/accuracy_reward': 0.696428582072258, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.1896214783191681, 'kl': 0.0145263671875, 'epoch': 2.07}

 41%|████▏     | 668/1610 [4:01:14<4:09:57, 15.92s/it]
 42%|████▏     | 669/1610 [4:01:26<3:53:12, 14.87s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9896563760625721, 'learning_rate': 5.84472049689441e-07, 'completion_length': 124.01786422729492, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1539071835577488, 'kl': 0.015350341796875, 'epoch': 2.08}

 42%|████▏     | 669/1610 [4:01:26<3:53:12, 14.87s/it]
 42%|████▏     | 670/1610 [4:01:45<4:09:08, 15.90s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1301506952190445, 'learning_rate': 5.838509316770186e-07, 'completion_length': 194.4821548461914, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.11266788095235825, 'kl': 0.01409912109375, 'epoch': 2.08}

 42%|████▏     | 670/1610 [4:01:45<4:09:08, 15.90s/it]
 42%|████▏     | 671/1610 [4:02:01<4:08:33, 15.88s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7181616118637573, 'learning_rate': 5.832298136645963e-07, 'completion_length': 176.1607208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1428571529686451, 'kl': 0.013519287109375, 'epoch': 2.08}

 42%|████▏     | 671/1610 [4:02:01<4:08:33, 15.88s/it]
 42%|████▏     | 672/1610 [4:02:15<4:03:08, 15.55s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.607963235859198, 'learning_rate': 5.826086956521739e-07, 'completion_length': 158.35714721679688, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.013885498046875, 'epoch': 2.09}

 42%|████▏     | 672/1610 [4:02:15<4:03:08, 15.55s/it]
 42%|████▏     | 673/1610 [4:02:32<4:07:40, 15.86s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8010727400148536, 'learning_rate': 5.819875776397515e-07, 'completion_length': 158.50000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514648616313934, 'kl': 0.01361083984375, 'epoch': 2.09}

 42%|████▏     | 673/1610 [4:02:32<4:07:40, 15.86s/it]
 42%|████▏     | 674/1610 [4:02:49<4:11:32, 16.12s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 7.129163981807131, 'learning_rate': 5.813664596273293e-07, 'completion_length': 157.85714721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714365541935, 'kl': 0.0125732421875, 'epoch': 2.09}

 42%|████▏     | 674/1610 [4:02:49<4:11:32, 16.12s/it]
 42%|████▏     | 675/1610 [4:03:02<3:59:48, 15.39s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.613972908191067, 'learning_rate': 5.807453416149069e-07, 'completion_length': 147.67858123779297, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.1539071798324585, 'kl': 0.013275146484375, 'epoch': 2.1}

 42%|████▏     | 675/1610 [4:03:02<3:59:48, 15.39s/it]
 42%|████▏     | 676/1610 [4:03:19<4:05:43, 15.79s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8636066188068842, 'learning_rate': 5.801242236024844e-07, 'completion_length': 164.55357360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.015533447265625, 'epoch': 2.1}

 42%|████▏     | 676/1610 [4:03:19<4:05:43, 15.79s/it]
 42%|████▏     | 677/1610 [4:03:33<3:56:05, 15.18s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5375229074131949, 'learning_rate': 5.795031055900621e-07, 'completion_length': 152.69644165039062, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.14838216453790665, 'kl': 0.0177001953125, 'epoch': 2.1}

 42%|████▏     | 677/1610 [4:03:33<3:56:05, 15.18s/it]
 42%|████▏     | 678/1610 [4:03:45<3:40:47, 14.21s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.879756592689587, 'learning_rate': 5.788819875776397e-07, 'completion_length': 118.25000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1896214783191681, 'kl': 0.01177978515625, 'epoch': 2.11}

 42%|████▏     | 678/1610 [4:03:45<3:40:47, 14.21s/it]
 42%|████▏     | 679/1610 [4:03:59<3:39:14, 14.13s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9455979609235878, 'learning_rate': 5.782608695652173e-07, 'completion_length': 148.7678680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266788095235825, 'kl': 0.013458251953125, 'epoch': 2.11}

 42%|████▏     | 679/1610 [4:03:59<3:39:14, 14.13s/it]
 42%|████▏     | 680/1610 [4:04:19<4:06:11, 15.88s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6029300847473926, 'learning_rate': 5.77639751552795e-07, 'completion_length': 187.94644165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7142857909202576, 'reward_std': 0.12974976375699043, 'kl': 0.01678466796875, 'epoch': 2.11}

 42%|████▏     | 680/1610 [4:04:19<4:06:11, 15.88s/it]
 42%|████▏     | 681/1610 [4:04:36<4:12:54, 16.33s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8054909649143439, 'learning_rate': 5.770186335403726e-07, 'completion_length': 184.01786041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1181928962469101, 'kl': 0.014312744140625, 'epoch': 2.11}

 42%|████▏     | 681/1610 [4:04:36<4:12:54, 16.33s/it]
 42%|████▏     | 682/1610 [4:04:50<3:59:00, 15.45s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 2.1269903231172145, 'learning_rate': 5.763975155279502e-07, 'completion_length': 123.66072082519531, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.0714285746216774, 'kl': 0.015594482421875, 'epoch': 2.12}

 42%|████▏     | 682/1610 [4:04:50<3:59:00, 15.45s/it]
 42%|████▏     | 683/1610 [4:05:09<4:15:12, 16.52s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6021480309759493, 'learning_rate': 5.75776397515528e-07, 'completion_length': 174.1607208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.14838217198848724, 'kl': 0.0164794921875, 'epoch': 2.12}

 42%|████▏     | 683/1610 [4:05:09<4:15:12, 16.52s/it]
 42%|████▏     | 684/1610 [4:05:21<3:57:51, 15.41s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2774840529028402, 'learning_rate': 5.751552795031056e-07, 'completion_length': 146.7678680419922, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.013336181640625, 'epoch': 2.12}

 42%|████▏     | 684/1610 [4:05:21<3:57:51, 15.41s/it]
 43%|████▎     | 685/1610 [4:05:37<3:56:53, 15.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0746635374613032, 'learning_rate': 5.745341614906832e-07, 'completion_length': 141.46429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.3078143745660782, 'kl': 0.016082763671875, 'epoch': 2.13}

 43%|████▎     | 685/1610 [4:05:37<3:56:53, 15.37s/it]
 43%|████▎     | 686/1610 [4:05:52<3:56:58, 15.39s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9737860855468736, 'learning_rate': 5.739130434782609e-07, 'completion_length': 143.0357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.014892578125, 'epoch': 2.13}

 43%|████▎     | 686/1610 [4:05:52<3:56:58, 15.39s/it]
 43%|████▎     | 687/1610 [4:06:05<3:47:18, 14.78s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.8772312112133829, 'learning_rate': 5.732919254658385e-07, 'completion_length': 130.50000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.01837158203125, 'epoch': 2.13}

 43%|████▎     | 687/1610 [4:06:05<3:47:18, 14.78s/it]
 43%|████▎     | 688/1610 [4:06:24<4:05:42, 15.99s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.1965617534318045, 'learning_rate': 5.726708074534161e-07, 'completion_length': 176.69644165039062, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.0135498046875, 'epoch': 2.14}

 43%|████▎     | 688/1610 [4:06:24<4:05:42, 15.99s/it]
 43%|████▎     | 689/1610 [4:06:45<4:27:35, 17.43s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9875484841371233, 'learning_rate': 5.720496894409938e-07, 'completion_length': 209.2321548461914, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5178572535514832, 'reward_std': 0.37974977493286133, 'kl': 0.016571044921875, 'epoch': 2.14}

 43%|████▎     | 689/1610 [4:06:45<4:27:35, 17.43s/it]
 43%|████▎     | 690/1610 [4:06:59<4:10:28, 16.34s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 4.970028000239941, 'learning_rate': 5.714285714285714e-07, 'completion_length': 135.25000381469727, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1428571492433548, 'kl': 0.0157470703125, 'epoch': 2.14}

 43%|████▎     | 690/1610 [4:06:59<4:10:28, 16.34s/it]
 43%|████▎     | 691/1610 [4:07:15<4:09:14, 16.27s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.182434868985094, 'learning_rate': 5.70807453416149e-07, 'completion_length': 152.62500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.2253357656300068, 'kl': 0.01654052734375, 'epoch': 2.15}

 43%|████▎     | 691/1610 [4:07:15<4:09:14, 16.27s/it]
 43%|████▎     | 692/1610 [4:07:27<3:51:24, 15.13s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.984136294973241, 'learning_rate': 5.701863354037268e-07, 'completion_length': 129.10714721679688, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1539071798324585, 'kl': 0.010009765625, 'epoch': 2.15}

 43%|████▎     | 692/1610 [4:07:27<3:51:24, 15.13s/it]
 43%|████▎     | 693/1610 [4:07:47<4:09:33, 16.33s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.2963217471070911, 'learning_rate': 5.695652173913044e-07, 'completion_length': 196.75000762939453, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535715222358704, 'reward_std': 0.3435286581516266, 'kl': 0.02081298828125, 'epoch': 2.15}

 43%|████▎     | 693/1610 [4:07:47<4:09:33, 16.33s/it]
 43%|████▎     | 694/1610 [4:08:04<4:13:56, 16.63s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7537359792912512, 'learning_rate': 5.68944099378882e-07, 'completion_length': 168.3928680419922, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1428571492433548, 'kl': 0.01287841796875, 'epoch': 2.16}

 43%|████▎     | 694/1610 [4:08:04<4:13:56, 16.63s/it]
 43%|████▎     | 695/1610 [4:08:20<4:10:57, 16.46s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7015022494718097, 'learning_rate': 5.683229813664597e-07, 'completion_length': 168.33929443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.015472412109375, 'epoch': 2.16}

 43%|████▎     | 695/1610 [4:08:20<4:10:57, 16.46s/it]
 43%|████▎     | 696/1610 [4:08:35<4:05:13, 16.10s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6430827385078525, 'learning_rate': 5.677018633540373e-07, 'completion_length': 146.75000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.015411376953125, 'epoch': 2.16}

 43%|████▎     | 696/1610 [4:08:35<4:05:13, 16.10s/it]
 43%|████▎     | 697/1610 [4:08:53<4:14:47, 16.74s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4106900780328804, 'learning_rate': 5.670807453416149e-07, 'completion_length': 175.5357208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1896214671432972, 'kl': 0.014923095703125, 'epoch': 2.16}

 43%|████▎     | 697/1610 [4:08:53<4:14:47, 16.74s/it]
 43%|████▎     | 698/1610 [4:09:09<4:10:22, 16.47s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.6807752684178712, 'learning_rate': 5.664596273291926e-07, 'completion_length': 163.30357360839844, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.21981073170900345, 'kl': 0.010101318359375, 'epoch': 2.17}

 43%|████▎     | 698/1610 [4:09:09<4:10:22, 16.47s/it]
 43%|████▎     | 699/1610 [4:09:24<4:02:44, 15.99s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8352903487373752, 'learning_rate': 5.658385093167701e-07, 'completion_length': 145.83929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214708685875, 'kl': 0.015869140625, 'epoch': 2.17}

 43%|████▎     | 699/1610 [4:09:24<4:02:44, 15.99s/it]
 43%|████▎     | 700/1610 [4:09:37<3:48:58, 15.10s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1343398137065377, 'learning_rate': 5.652173913043477e-07, 'completion_length': 153.08929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657507568597794, 'kl': 0.0145263671875, 'epoch': 2.17}

 43%|████▎     | 700/1610 [4:09:37<3:48:58, 15.10s/it]
 44%|████▎     | 701/1610 [4:14:35<25:14:01, 99.94s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.08757367985498, 'learning_rate': 5.645962732919255e-07, 'completion_length': 161.58929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.3078143745660782, 'kl': 0.015472412109375, 'epoch': 2.18}

 44%|████▎     | 701/1610 [4:14:35<25:14:01, 99.94s/it]
 44%|████▎     | 702/1610 [4:14:55<19:08:04, 75.86s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6179614333524412, 'learning_rate': 5.639751552795031e-07, 'completion_length': 180.2321548461914, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.11266787722706795, 'kl': 0.011505126953125, 'epoch': 2.18}

 44%|████▎     | 702/1610 [4:14:55<19:08:04, 75.86s/it]
 44%|████▎     | 703/1610 [4:15:14<14:48:20, 58.77s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.4389678101056005, 'learning_rate': 5.633540372670807e-07, 'completion_length': 155.1607208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.1896214671432972, 'kl': 0.01617431640625, 'epoch': 2.18}

 44%|████▎     | 703/1610 [4:15:14<14:48:20, 58.77s/it]
 44%|████▎     | 704/1610 [4:15:29<11:29:54, 45.69s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8794872274340149, 'learning_rate': 5.627329192546583e-07, 'completion_length': 155.4107208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.014251708984375, 'epoch': 2.19}

 44%|████▎     | 704/1610 [4:15:29<11:29:54, 45.69s/it]
 44%|████▍     | 705/1610 [4:15:47<9:22:43, 37.31s/it] 
                                                      
{'loss': 0.0006, 'grad_norm': 0.8313236223810243, 'learning_rate': 5.62111801242236e-07, 'completion_length': 144.3214340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.01397705078125, 'epoch': 2.19}

 44%|████▍     | 705/1610 [4:15:47<9:22:43, 37.31s/it]
 44%|████▍     | 706/1610 [4:16:00<7:33:51, 30.12s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8782770414554223, 'learning_rate': 5.614906832298136e-07, 'completion_length': 129.8928680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2363857962191105, 'kl': 0.01300048828125, 'epoch': 2.19}

 44%|████▍     | 706/1610 [4:16:00<7:33:51, 30.12s/it]
 44%|████▍     | 707/1610 [4:16:14<6:19:40, 25.23s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.4642824485626027, 'learning_rate': 5.608695652173912e-07, 'completion_length': 145.23214721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.0357142873108387, 'kl': 0.016571044921875, 'epoch': 2.2}

 44%|████▍     | 707/1610 [4:16:14<6:19:40, 25.23s/it]
 44%|████▍     | 708/1610 [4:16:27<5:27:30, 21.79s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9275702707710993, 'learning_rate': 5.602484472049689e-07, 'completion_length': 140.5357208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086078464984894, 'kl': 0.014404296875, 'epoch': 2.2}

 44%|████▍     | 708/1610 [4:16:27<5:27:30, 21.79s/it]
 44%|████▍     | 709/1610 [4:16:45<5:07:46, 20.50s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0855699540290942, 'learning_rate': 5.596273291925465e-07, 'completion_length': 182.58929443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981073915958405, 'kl': 0.01434326171875, 'epoch': 2.2}

 44%|████▍     | 709/1610 [4:16:45<5:07:46, 20.50s/it]
 44%|████▍     | 710/1610 [4:17:00<4:42:41, 18.85s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1779963654788304, 'learning_rate': 5.590062111801241e-07, 'completion_length': 136.60714721679688, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25552502274513245, 'kl': 0.015838623046875, 'epoch': 2.2}

 44%|████▍     | 710/1610 [4:17:00<4:42:41, 18.85s/it]
 44%|████▍     | 711/1610 [4:17:17<4:32:40, 18.20s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7536909109931899, 'learning_rate': 5.583850931677019e-07, 'completion_length': 166.1607208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.017120361328125, 'epoch': 2.21}

 44%|████▍     | 711/1610 [4:17:17<4:32:40, 18.20s/it]
 44%|████▍     | 712/1610 [4:17:32<4:17:45, 17.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.734669399767813, 'learning_rate': 5.577639751552795e-07, 'completion_length': 152.4107208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1896214708685875, 'kl': 0.014007568359375, 'epoch': 2.21}

 44%|████▍     | 712/1610 [4:17:32<4:17:45, 17.22s/it]
 44%|████▍     | 713/1610 [4:17:46<4:03:44, 16.30s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1587758319325774, 'learning_rate': 5.571428571428571e-07, 'completion_length': 150.1428680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2610500454902649, 'kl': 0.01385498046875, 'epoch': 2.21}

 44%|████▍     | 713/1610 [4:17:46<4:03:44, 16.30s/it]
 44%|████▍     | 714/1610 [4:18:01<3:58:55, 16.00s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0996393053474294, 'learning_rate': 5.565217391304348e-07, 'completion_length': 137.4107208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.0179443359375, 'epoch': 2.22}

 44%|████▍     | 714/1610 [4:18:01<3:58:55, 16.00s/it]
 44%|████▍     | 715/1610 [4:18:17<3:57:43, 15.94s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7610803514818412, 'learning_rate': 5.559006211180124e-07, 'completion_length': 197.6428680419922, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.23086077719926834, 'kl': 0.01739501953125, 'epoch': 2.22}

 44%|████▍     | 715/1610 [4:18:17<3:57:43, 15.94s/it]
 44%|████▍     | 716/1610 [4:18:31<3:51:51, 15.56s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.5101606479561372, 'learning_rate': 5.5527950310559e-07, 'completion_length': 155.3571548461914, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.11266788095235825, 'kl': 0.0162353515625, 'epoch': 2.22}

 44%|████▍     | 716/1610 [4:18:31<3:51:51, 15.56s/it]
 45%|████▍     | 717/1610 [4:18:49<4:00:13, 16.14s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8588906090757725, 'learning_rate': 5.546583850931677e-07, 'completion_length': 176.9464340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.14838216453790665, 'kl': 0.01739501953125, 'epoch': 2.23}

 45%|████▍     | 717/1610 [4:18:49<4:00:13, 16.14s/it]
 45%|████▍     | 718/1610 [4:19:06<4:02:24, 16.31s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.947831421674742, 'learning_rate': 5.540372670807453e-07, 'completion_length': 151.64286041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981073915958405, 'kl': 0.019775390625, 'epoch': 2.23}

 45%|████▍     | 718/1610 [4:19:06<4:02:24, 16.31s/it]
 45%|████▍     | 719/1610 [4:19:22<4:03:50, 16.42s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7116755962937474, 'learning_rate': 5.534161490683229e-07, 'completion_length': 170.05358123779297, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.01654052734375, 'epoch': 2.23}

 45%|████▍     | 719/1610 [4:19:22<4:03:50, 16.42s/it]
 45%|████▍     | 720/1610 [4:19:38<3:59:22, 16.14s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2409055193275758, 'learning_rate': 5.527950310559007e-07, 'completion_length': 156.17857360839844, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.25552502274513245, 'kl': 0.01812744140625, 'epoch': 2.24}

 45%|████▍     | 720/1610 [4:19:38<3:59:22, 16.14s/it]
 45%|████▍     | 721/1610 [4:19:51<3:45:35, 15.23s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.4127139860258318, 'learning_rate': 5.521739130434783e-07, 'completion_length': 137.55357360839844, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.01361083984375, 'epoch': 2.24}

 45%|████▍     | 721/1610 [4:19:51<3:45:35, 15.23s/it]
 45%|████▍     | 722/1610 [4:20:10<4:02:45, 16.40s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.928008289188427, 'learning_rate': 5.515527950310559e-07, 'completion_length': 207.32144165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.016876220703125, 'epoch': 2.24}

 45%|████▍     | 722/1610 [4:20:10<4:02:45, 16.40s/it]
 45%|████▍     | 723/1610 [4:20:23<3:47:16, 15.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.993210586665821, 'learning_rate': 5.509316770186335e-07, 'completion_length': 138.42858123779297, 'rewards/accuracy_reward': 0.8035714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.07695358991622925, 'kl': 0.01531982421875, 'epoch': 2.25}

 45%|████▍     | 723/1610 [4:20:23<3:47:16, 15.37s/it]
 45%|████▍     | 724/1610 [4:20:36<3:34:34, 14.53s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.04533575273446149, 'learning_rate': 5.503105590062111e-07, 'completion_length': 124.28572082519531, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0, 'kl': 0.01300048828125, 'epoch': 2.25}

 45%|████▍     | 724/1610 [4:20:36<3:34:34, 14.53s/it]
 45%|████▌     | 725/1610 [4:20:53<3:45:03, 15.26s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1194672128430656, 'learning_rate': 5.496894409937887e-07, 'completion_length': 186.08929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2363857924938202, 'kl': 0.014984130859375, 'epoch': 2.25}

 45%|████▌     | 725/1610 [4:20:53<3:45:03, 15.26s/it]
 45%|████▌     | 726/1610 [4:21:06<3:35:26, 14.62s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.038998304149398, 'learning_rate': 5.490683229813664e-07, 'completion_length': 112.64286422729492, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.012664794921875, 'epoch': 2.25}

 45%|████▌     | 726/1610 [4:21:06<3:35:26, 14.62s/it]
 45%|████▌     | 727/1610 [4:21:21<3:39:01, 14.88s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.6721537849257566, 'learning_rate': 5.48447204968944e-07, 'completion_length': 149.98215103149414, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.01739501953125, 'epoch': 2.26}

 45%|████▌     | 727/1610 [4:21:21<3:39:01, 14.88s/it]
 45%|████▌     | 728/1610 [4:21:36<3:40:00, 14.97s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.953912368578846, 'learning_rate': 5.478260869565216e-07, 'completion_length': 158.80358123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357544541359, 'kl': 0.01470947265625, 'epoch': 2.26}

 45%|████▌     | 728/1610 [4:21:36<3:40:00, 14.97s/it]
 45%|████▌     | 729/1610 [4:21:55<3:54:10, 15.95s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.1511511602646855, 'learning_rate': 5.472049689440994e-07, 'completion_length': 144.37500762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8928571939468384, 'reward_std': 0.0714285746216774, 'kl': 0.011566162109375, 'epoch': 2.26}

 45%|████▌     | 729/1610 [4:21:55<3:54:10, 15.95s/it]
 45%|████▌     | 730/1610 [4:22:09<3:45:24, 15.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.234058713040672, 'learning_rate': 5.46583850931677e-07, 'completion_length': 157.5178680419922, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.2253357619047165, 'kl': 0.013916015625, 'epoch': 2.27}

 45%|████▌     | 730/1610 [4:22:09<3:45:24, 15.37s/it]
 45%|████▌     | 731/1610 [4:22:22<3:37:12, 14.83s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6873215418395742, 'learning_rate': 5.459627329192546e-07, 'completion_length': 132.3214340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1071428619325161, 'kl': 0.015411376953125, 'epoch': 2.27}

 45%|████▌     | 731/1610 [4:22:22<3:37:12, 14.83s/it]
 45%|████▌     | 732/1610 [4:22:37<3:37:46, 14.88s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5594257537512404, 'learning_rate': 5.453416149068323e-07, 'completion_length': 163.17858123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.017822265625, 'epoch': 2.27}

 45%|████▌     | 732/1610 [4:22:37<3:37:46, 14.88s/it]
 46%|████▌     | 733/1610 [4:22:53<3:41:55, 15.18s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.344636832454804, 'learning_rate': 5.447204968944099e-07, 'completion_length': 151.7857208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.016998291015625, 'epoch': 2.28}

 46%|████▌     | 733/1610 [4:22:53<3:41:55, 15.18s/it]
 46%|████▌     | 734/1610 [4:23:06<3:32:35, 14.56s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1261737779369176, 'learning_rate': 5.440993788819875e-07, 'completion_length': 138.00000381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.1428571492433548, 'kl': 0.014190673828125, 'epoch': 2.28}

 46%|████▌     | 734/1610 [4:23:06<3:32:35, 14.56s/it]
 46%|████▌     | 735/1610 [4:23:22<3:39:04, 15.02s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7503481417816388, 'learning_rate': 5.434782608695652e-07, 'completion_length': 168.87500762939453, 'rewards/accuracy_reward': 0.6071428805589676, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.571428656578064, 'reward_std': 0.2253357470035553, 'kl': 0.0140380859375, 'epoch': 2.28}

 46%|████▌     | 735/1610 [4:23:22<3:39:04, 15.02s/it]
 46%|████▌     | 736/1610 [4:23:38<3:41:15, 15.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.4262815372003903, 'learning_rate': 5.428571428571428e-07, 'completion_length': 169.4107208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1428571492433548, 'kl': 0.01470947265625, 'epoch': 2.29}

 46%|████▌     | 736/1610 [4:23:38<3:41:15, 15.19s/it]
 46%|████▌     | 737/1610 [4:23:50<3:28:30, 14.33s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.7364363447487576, 'learning_rate': 5.422360248447204e-07, 'completion_length': 108.4464340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.1428571529686451, 'kl': 0.016845703125, 'epoch': 2.29}

 46%|████▌     | 737/1610 [4:23:50<3:28:30, 14.33s/it]
 46%|████▌     | 738/1610 [4:24:03<3:19:49, 13.75s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9074004683318098, 'learning_rate': 5.416149068322982e-07, 'completion_length': 133.07143020629883, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838217198848724, 'kl': 0.01422119140625, 'epoch': 2.29}

 46%|████▌     | 738/1610 [4:24:03<3:19:49, 13.75s/it]
 46%|████▌     | 739/1610 [4:24:20<3:37:36, 14.99s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0862290833517552, 'learning_rate': 5.409937888198758e-07, 'completion_length': 166.9107208251953, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7142857909202576, 'reward_std': 0.18658055737614632, 'kl': 0.01763916015625, 'epoch': 2.3}

 46%|████▌     | 739/1610 [4:24:20<3:37:36, 14.99s/it]
 46%|████▌     | 740/1610 [4:24:35<3:33:22, 14.72s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 4.126494577220192, 'learning_rate': 5.403726708074534e-07, 'completion_length': 160.6607208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1896214634180069, 'kl': 0.01959228515625, 'epoch': 2.3}

 46%|████▌     | 740/1610 [4:24:35<3:33:22, 14.72s/it]
 46%|████▌     | 741/1610 [4:24:52<3:44:01, 15.47s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.647211679946913, 'learning_rate': 5.397515527950311e-07, 'completion_length': 163.05358123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.21676982194185257, 'kl': 0.01458740234375, 'epoch': 2.3}

 46%|████▌     | 741/1610 [4:24:52<3:44:01, 15.47s/it]
 46%|████▌     | 742/1610 [4:25:09<3:50:55, 15.96s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6698452042369838, 'learning_rate': 5.391304347826087e-07, 'completion_length': 176.98214721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.013397216796875, 'epoch': 2.3}

 46%|████▌     | 742/1610 [4:25:09<3:50:55, 15.96s/it]
 46%|████▌     | 743/1610 [4:25:25<3:52:29, 16.09s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.08812234984072, 'learning_rate': 5.385093167701863e-07, 'completion_length': 162.05358123779297, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981073915958405, 'kl': 0.01812744140625, 'epoch': 2.31}

 46%|████▌     | 743/1610 [4:25:25<3:52:29, 16.09s/it]
 46%|████▌     | 744/1610 [4:25:40<3:47:36, 15.77s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.984865881699009, 'learning_rate': 5.37888198757764e-07, 'completion_length': 149.33929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1896214783191681, 'kl': 0.015289306640625, 'epoch': 2.31}

 46%|████▌     | 744/1610 [4:25:40<3:47:36, 15.77s/it]
 46%|████▋     | 745/1610 [4:25:58<3:54:54, 16.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8249140042451132, 'learning_rate': 5.372670807453416e-07, 'completion_length': 190.0357208251953, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.21981073170900345, 'kl': 0.014801025390625, 'epoch': 2.31}

 46%|████▋     | 745/1610 [4:25:58<3:54:54, 16.29s/it]
 46%|████▋     | 746/1610 [4:26:10<3:36:26, 15.03s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.468851755125986, 'learning_rate': 5.366459627329191e-07, 'completion_length': 127.87500381469727, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.015350341796875, 'epoch': 2.32}

 46%|████▋     | 746/1610 [4:26:10<3:36:26, 15.03s/it]
 46%|████▋     | 747/1610 [4:26:24<3:33:29, 14.84s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7742096502930252, 'learning_rate': 5.360248447204969e-07, 'completion_length': 171.35714721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.0162353515625, 'epoch': 2.32}

 46%|████▋     | 747/1610 [4:26:24<3:33:29, 14.84s/it]
 46%|████▋     | 748/1610 [4:26:39<3:34:28, 14.93s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0493995246767356, 'learning_rate': 5.354037267080745e-07, 'completion_length': 130.46429061889648, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1428571529686451, 'kl': 0.01690673828125, 'epoch': 2.32}

 46%|████▋     | 748/1610 [4:26:39<3:34:28, 14.93s/it]
 47%|████▋     | 749/1610 [4:26:55<3:34:59, 14.98s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3593148952769603, 'learning_rate': 5.347826086956521e-07, 'completion_length': 166.85714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214634180069, 'kl': 0.014801025390625, 'epoch': 2.33}

 47%|████▋     | 749/1610 [4:26:55<3:34:59, 14.98s/it]
 47%|████▋     | 750/1610 [4:27:12<3:46:04, 15.77s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6157263378371652, 'learning_rate': 5.341614906832298e-07, 'completion_length': 197.3214340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428619325161, 'kl': 0.014739990234375, 'epoch': 2.33}

 47%|████▋     | 750/1610 [4:27:12<3:46:04, 15.77s/it]
 47%|████▋     | 751/1610 [4:27:26<3:38:59, 15.30s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2982536285131623, 'learning_rate': 5.335403726708074e-07, 'completion_length': 138.9107208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.25552501529455185, 'kl': 0.01434326171875, 'epoch': 2.33}

 47%|████▋     | 751/1610 [4:27:26<3:38:59, 15.30s/it]
 47%|████▋     | 752/1610 [4:27:42<3:39:47, 15.37s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.826858707029143, 'learning_rate': 5.32919254658385e-07, 'completion_length': 145.33929443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.11266787722706795, 'kl': 0.01715087890625, 'epoch': 2.34}

 47%|████▋     | 752/1610 [4:27:42<3:39:47, 15.37s/it]
 47%|████▋     | 753/1610 [4:27:54<3:25:24, 14.38s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.625060010463603, 'learning_rate': 5.322981366459627e-07, 'completion_length': 128.8035774230957, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.01568603515625, 'epoch': 2.34}

 47%|████▋     | 753/1610 [4:27:54<3:25:24, 14.38s/it]
 47%|████▋     | 754/1610 [4:28:12<3:41:49, 15.55s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8683262769383234, 'learning_rate': 5.316770186335403e-07, 'completion_length': 177.2678680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.2610500305891037, 'kl': 0.016845703125, 'epoch': 2.34}

 47%|████▋     | 754/1610 [4:28:12<3:41:49, 15.55s/it]
 47%|████▋     | 755/1610 [4:28:27<3:38:54, 15.36s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.6104291115877694, 'learning_rate': 5.310559006211179e-07, 'completion_length': 154.2857208251953, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8214285969734192, 'reward_std': 0.11266788095235825, 'kl': 0.01092529296875, 'epoch': 2.34}

 47%|████▋     | 755/1610 [4:28:27<3:38:54, 15.36s/it]
 47%|████▋     | 756/1610 [4:28:43<3:38:47, 15.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7601830711431776, 'learning_rate': 5.304347826086957e-07, 'completion_length': 168.50000762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428656578064, 'kl': 0.01531982421875, 'epoch': 2.35}

 47%|████▋     | 756/1610 [4:28:43<3:38:47, 15.37s/it]
 47%|████▋     | 757/1610 [4:28:59<3:42:34, 15.66s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0041992946034757, 'learning_rate': 5.298136645962733e-07, 'completion_length': 173.55357360839844, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.1896214634180069, 'kl': 0.01409912109375, 'epoch': 2.35}

 47%|████▋     | 757/1610 [4:28:59<3:42:34, 15.66s/it]
 47%|████▋     | 758/1610 [4:29:17<3:54:32, 16.52s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6595629235678612, 'learning_rate': 5.291925465838509e-07, 'completion_length': 174.00000762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.21124479919672012, 'kl': 0.01654052734375, 'epoch': 2.35}

 47%|████▋     | 758/1610 [4:29:17<3:54:32, 16.52s/it]
 47%|████▋     | 759/1610 [4:29:35<3:59:19, 16.87s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 2.0092497113322767, 'learning_rate': 5.285714285714286e-07, 'completion_length': 165.87500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6250000596046448, 'reward_std': 0.21981073915958405, 'kl': 0.0194091796875, 'epoch': 2.36}

 47%|████▋     | 759/1610 [4:29:35<3:59:19, 16.87s/it]
 47%|████▋     | 760/1610 [4:29:52<3:58:19, 16.82s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.7301031672167955, 'learning_rate': 5.279503105590062e-07, 'completion_length': 156.1428680419922, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981073915958405, 'kl': 0.01727294921875, 'epoch': 2.36}

 47%|████▋     | 760/1610 [4:29:52<3:58:19, 16.82s/it]
 47%|████▋     | 761/1610 [4:30:09<4:01:08, 17.04s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.6765026746885072, 'learning_rate': 5.273291925465838e-07, 'completion_length': 170.67858123779297, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.2006715089082718, 'kl': 0.0189208984375, 'epoch': 2.36}

 47%|████▋     | 761/1610 [4:30:09<4:01:08, 17.04s/it]
 47%|████▋     | 762/1610 [4:30:23<3:47:47, 16.12s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.6737141437810492, 'learning_rate': 5.267080745341615e-07, 'completion_length': 151.85714721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.11266788095235825, 'kl': 0.013519287109375, 'epoch': 2.37}

 47%|████▋     | 762/1610 [4:30:23<3:47:47, 16.12s/it]
 47%|████▋     | 763/1610 [4:30:40<3:51:33, 16.40s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1798828045104905, 'learning_rate': 5.260869565217391e-07, 'completion_length': 177.17858123779297, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.19514649361371994, 'kl': 0.01470947265625, 'epoch': 2.37}

 47%|████▋     | 763/1610 [4:30:40<3:51:33, 16.40s/it]
 47%|████▋     | 764/1610 [4:30:56<3:48:22, 16.20s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.116443321495808, 'learning_rate': 5.254658385093167e-07, 'completion_length': 166.55358123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571492433548, 'kl': 0.014068603515625, 'epoch': 2.37}

 47%|████▋     | 764/1610 [4:30:56<3:48:22, 16.20s/it]
 48%|████▊     | 765/1610 [4:31:15<3:58:08, 16.91s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7915724005232347, 'learning_rate': 5.248447204968945e-07, 'completion_length': 182.60714721679688, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.14838216826319695, 'kl': 0.0172119140625, 'epoch': 2.38}

 48%|████▊     | 765/1610 [4:31:15<3:58:08, 16.91s/it]
 48%|████▊     | 766/1610 [4:31:31<3:56:07, 16.79s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9662492936118366, 'learning_rate': 5.242236024844721e-07, 'completion_length': 138.4285774230957, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571529686451, 'kl': 0.013946533203125, 'epoch': 2.38}

 48%|████▊     | 766/1610 [4:31:31<3:56:07, 16.79s/it]
 48%|████▊     | 767/1610 [4:31:48<3:55:02, 16.73s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8384082279977728, 'learning_rate': 5.236024844720497e-07, 'completion_length': 174.75000762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1071428619325161, 'kl': 0.01666259765625, 'epoch': 2.38}

 48%|████▊     | 767/1610 [4:31:48<3:55:02, 16.73s/it]
 48%|████▊     | 768/1610 [4:32:05<3:56:03, 16.82s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.288288093227104, 'learning_rate': 5.229813664596274e-07, 'completion_length': 160.12500762939453, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.23086079210042953, 'kl': 0.01837158203125, 'epoch': 2.39}

 48%|████▊     | 768/1610 [4:32:05<3:56:03, 16.82s/it]
 48%|████▊     | 769/1610 [4:32:21<3:54:45, 16.75s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5247397084185934, 'learning_rate': 5.22360248447205e-07, 'completion_length': 183.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2610500529408455, 'kl': 0.0185546875, 'epoch': 2.39}

 48%|████▊     | 769/1610 [4:32:21<3:54:45, 16.75s/it]
 48%|████▊     | 770/1610 [4:32:33<3:34:47, 15.34s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7685315976778094, 'learning_rate': 5.217391304347825e-07, 'completion_length': 114.76786041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.01666259765625, 'epoch': 2.39}

 48%|████▊     | 770/1610 [4:32:33<3:34:47, 15.34s/it]
 48%|████▊     | 771/1610 [4:32:50<3:41:15, 15.82s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.553077241854566, 'learning_rate': 5.211180124223602e-07, 'completion_length': 159.5357208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1181928962469101, 'kl': 0.01702880859375, 'epoch': 2.39}

 48%|████▊     | 771/1610 [4:32:50<3:41:15, 15.82s/it]
 48%|████▊     | 772/1610 [4:33:06<3:40:24, 15.78s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.686293366920382, 'learning_rate': 5.204968944099378e-07, 'completion_length': 148.80357360839844, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.11266788095235825, 'kl': 0.012542724609375, 'epoch': 2.4}

 48%|████▊     | 772/1610 [4:33:06<3:40:24, 15.78s/it]
 48%|████▊     | 773/1610 [4:33:21<3:35:58, 15.48s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0628293773349151, 'learning_rate': 5.198757763975154e-07, 'completion_length': 137.89286422729492, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.14838217198848724, 'kl': 0.01263427734375, 'epoch': 2.4}

 48%|████▊     | 773/1610 [4:33:21<3:35:58, 15.48s/it]
 48%|████▊     | 774/1610 [4:33:35<3:28:30, 14.96s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9918146995863847, 'learning_rate': 5.192546583850932e-07, 'completion_length': 142.16071701049805, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409645557403564, 'kl': 0.01470947265625, 'epoch': 2.4}

 48%|████▊     | 774/1610 [4:33:35<3:28:30, 14.96s/it]
 48%|████▊     | 775/1610 [4:33:49<3:25:56, 14.80s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.466927582148124, 'learning_rate': 5.186335403726708e-07, 'completion_length': 147.00000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1071428619325161, 'kl': 0.0162353515625, 'epoch': 2.41}

 48%|████▊     | 775/1610 [4:33:49<3:25:56, 14.80s/it]
 48%|████▊     | 776/1610 [4:34:02<3:16:38, 14.15s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8084783215291796, 'learning_rate': 5.180124223602484e-07, 'completion_length': 137.67858123779297, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357656300068, 'kl': 0.0140380859375, 'epoch': 2.41}

 48%|████▊     | 776/1610 [4:34:02<3:16:38, 14.15s/it]
 48%|████▊     | 777/1610 [4:34:16<3:18:53, 14.33s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9479482371824367, 'learning_rate': 5.173913043478261e-07, 'completion_length': 145.8571548461914, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.18409645557403564, 'kl': 0.01568603515625, 'epoch': 2.41}

 48%|████▊     | 777/1610 [4:34:16<3:18:53, 14.33s/it]
 48%|████▊     | 778/1610 [4:34:32<3:23:50, 14.70s/it]
                                                      
{'loss': 0.0011, 'grad_norm': 4.135627255975852, 'learning_rate': 5.167701863354037e-07, 'completion_length': 170.30358123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428656578064, 'kl': 0.0269775390625, 'epoch': 2.42}

 48%|████▊     | 778/1610 [4:34:32<3:23:50, 14.70s/it]
 48%|████▊     | 779/1610 [4:34:50<3:37:18, 15.69s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5876406664984898, 'learning_rate': 5.161490683229813e-07, 'completion_length': 185.30358123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428572535514832, 'reward_std': 0.2142857238650322, 'kl': 0.0135498046875, 'epoch': 2.42}

 48%|████▊     | 779/1610 [4:34:50<3:37:18, 15.69s/it]
 48%|████▊     | 780/1610 [4:35:07<3:41:44, 16.03s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.4049685430310144, 'learning_rate': 5.15527950310559e-07, 'completion_length': 155.62500762939453, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.02117919921875, 'epoch': 2.42}

 48%|████▊     | 780/1610 [4:35:07<3:41:44, 16.03s/it]
 49%|████▊     | 781/1610 [4:35:25<3:52:04, 16.80s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8714375004356231, 'learning_rate': 5.149068322981366e-07, 'completion_length': 157.12500762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142858505249023, 'reward_std': 0.25552503019571304, 'kl': 0.01702880859375, 'epoch': 2.43}

 49%|████▊     | 781/1610 [4:35:25<3:52:04, 16.80s/it]
 49%|████▊     | 782/1610 [4:35:42<3:50:00, 16.67s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5450556737522321, 'learning_rate': 5.142857142857142e-07, 'completion_length': 149.6964340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071798324585, 'kl': 0.014892578125, 'epoch': 2.43}

 49%|████▊     | 782/1610 [4:35:42<3:50:00, 16.67s/it]
 49%|████▊     | 783/1610 [4:35:59<3:52:30, 16.87s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5767762192918992, 'learning_rate': 5.13664596273292e-07, 'completion_length': 165.21429443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.014129638671875, 'epoch': 2.43}

 49%|████▊     | 783/1610 [4:35:59<3:52:30, 16.87s/it]
 49%|████▊     | 784/1610 [4:36:12<3:36:48, 15.75s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.811958336598224, 'learning_rate': 5.130434782608696e-07, 'completion_length': 134.92857360839844, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.014068603515625, 'epoch': 2.43}

 49%|████▊     | 784/1610 [4:36:12<3:36:48, 15.75s/it]
 49%|████▉     | 785/1610 [4:36:26<3:29:12, 15.22s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.20703656979703502, 'learning_rate': 5.124223602484472e-07, 'completion_length': 150.9107208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.0357142873108387, 'kl': 0.012420654296875, 'epoch': 2.44}

 49%|████▉     | 785/1610 [4:36:26<3:29:12, 15.22s/it]
 49%|████▉     | 786/1610 [4:36:41<3:27:21, 15.10s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7877676869432492, 'learning_rate': 5.118012422360249e-07, 'completion_length': 168.1964340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1785714402794838, 'kl': 0.014312744140625, 'epoch': 2.44}

 49%|████▉     | 786/1610 [4:36:41<3:27:21, 15.10s/it]
 49%|████▉     | 787/1610 [4:36:56<3:27:47, 15.15s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.5008351154825017, 'learning_rate': 5.111801242236025e-07, 'completion_length': 161.5714340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.0167236328125, 'epoch': 2.44}

 49%|████▉     | 787/1610 [4:36:56<3:27:47, 15.15s/it]
 49%|████▉     | 788/1610 [4:37:12<3:30:56, 15.40s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8236849175149594, 'learning_rate': 5.105590062111801e-07, 'completion_length': 171.9107208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.15943221747875214, 'kl': 0.0164794921875, 'epoch': 2.45}

 49%|████▉     | 788/1610 [4:37:12<3:30:56, 15.40s/it]
 49%|████▉     | 789/1610 [4:37:24<3:15:54, 14.32s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8146488378723308, 'learning_rate': 5.099378881987578e-07, 'completion_length': 116.17857360839844, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.11266787722706795, 'kl': 0.0126953125, 'epoch': 2.45}

 49%|████▉     | 789/1610 [4:37:24<3:15:54, 14.32s/it]
 49%|████▉     | 790/1610 [4:37:43<3:36:03, 15.81s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9702991250974562, 'learning_rate': 5.093167701863354e-07, 'completion_length': 170.96428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.607142984867096, 'reward_std': 0.2967643216252327, 'kl': 0.01861572265625, 'epoch': 2.45}

 49%|████▉     | 790/1610 [4:37:43<3:36:03, 15.81s/it]
 49%|████▉     | 791/1610 [4:37:58<3:30:40, 15.43s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6636613825245353, 'learning_rate': 5.08695652173913e-07, 'completion_length': 147.6964340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.07695358991622925, 'kl': 0.0185546875, 'epoch': 2.46}

 49%|████▉     | 791/1610 [4:37:58<3:30:40, 15.43s/it]
 49%|████▉     | 792/1610 [4:38:14<3:31:50, 15.54s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.396148960859739, 'learning_rate': 5.080745341614908e-07, 'completion_length': 151.50000762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.014556884765625, 'epoch': 2.46}

 49%|████▉     | 792/1610 [4:38:14<3:31:50, 15.54s/it]
 49%|████▉     | 793/1610 [4:38:31<3:38:20, 16.03s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7450531025407817, 'learning_rate': 5.074534161490684e-07, 'completion_length': 174.9464340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.20117833465337753, 'kl': 0.015960693359375, 'epoch': 2.46}

 49%|████▉     | 793/1610 [4:38:31<3:38:20, 16.03s/it]
 49%|████▉     | 794/1610 [4:38:46<3:33:52, 15.73s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5187136232811158, 'learning_rate': 5.068322981366459e-07, 'completion_length': 144.0, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.01776123046875, 'epoch': 2.47}

 49%|████▉     | 794/1610 [4:38:46<3:33:52, 15.73s/it]
 49%|████▉     | 795/1610 [4:39:00<3:26:17, 15.19s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 2.1679730355768956, 'learning_rate': 5.062111801242235e-07, 'completion_length': 164.9464340209961, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.343528650701046, 'kl': 0.02154541015625, 'epoch': 2.47}

 49%|████▉     | 795/1610 [4:39:00<3:26:17, 15.19s/it]
 49%|████▉     | 796/1610 [4:39:17<3:34:56, 15.84s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9736685184452513, 'learning_rate': 5.055900621118012e-07, 'completion_length': 154.6607208251953, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.014862060546875, 'epoch': 2.47}

 49%|████▉     | 796/1610 [4:39:17<3:34:56, 15.84s/it]
 50%|████▉     | 797/1610 [4:39:30<3:23:32, 15.02s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1794256994612553, 'learning_rate': 5.049689440993788e-07, 'completion_length': 122.21429061889648, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1428571492433548, 'kl': 0.01556396484375, 'epoch': 2.48}

 50%|████▉     | 797/1610 [4:39:30<3:23:32, 15.02s/it]
 50%|████▉     | 798/1610 [4:39:47<3:31:32, 15.63s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.7695030538895817, 'learning_rate': 5.043478260869564e-07, 'completion_length': 188.42858123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214783191681, 'kl': 0.01666259765625, 'epoch': 2.48}

 50%|████▉     | 798/1610 [4:39:47<3:31:32, 15.63s/it]
 50%|████▉     | 799/1610 [4:40:02<3:26:45, 15.30s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8827634049534849, 'learning_rate': 5.037267080745341e-07, 'completion_length': 169.9821548461914, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2142857238650322, 'kl': 0.014923095703125, 'epoch': 2.48}

 50%|████▉     | 799/1610 [4:40:02<3:26:45, 15.30s/it]
 50%|████▉     | 800/1610 [4:40:20<3:36:38, 16.05s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6429844590452897, 'learning_rate': 5.031055900621117e-07, 'completion_length': 195.5178680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1539071872830391, 'kl': 0.01806640625, 'epoch': 2.48}

 50%|████▉     | 800/1610 [4:40:20<3:36:38, 16.05s/it]
 50%|████▉     | 801/1610 [4:45:12<22:12:16, 98.81s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9228104772710017, 'learning_rate': 5.024844720496894e-07, 'completion_length': 140.46429443359375, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714365541935, 'kl': 0.01611328125, 'epoch': 2.49}

 50%|████▉     | 801/1610 [4:45:12<22:12:16, 98.81s/it]
 50%|████▉     | 802/1610 [4:45:26<16:29:27, 73.48s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.3928960688122212, 'learning_rate': 5.018633540372671e-07, 'completion_length': 144.6607208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.25552502274513245, 'kl': 0.01904296875, 'epoch': 2.49}

 50%|████▉     | 802/1610 [4:45:26<16:29:27, 73.48s/it]
 50%|████▉     | 803/1610 [4:45:42<12:37:56, 56.35s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.222969046483427, 'learning_rate': 5.012422360248447e-07, 'completion_length': 163.51786041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266788095235825, 'kl': 0.0162353515625, 'epoch': 2.49}

 50%|████▉     | 803/1610 [4:45:42<12:37:56, 56.35s/it]
 50%|████▉     | 804/1610 [4:45:59<9:55:16, 44.31s/it] 
                                                      
{'loss': 0.0009, 'grad_norm': 1.1760365731856617, 'learning_rate': 5.006211180124223e-07, 'completion_length': 186.3571548461914, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.36266787350177765, 'kl': 0.021240234375, 'epoch': 2.5}

 50%|████▉     | 804/1610 [4:45:59<9:55:16, 44.31s/it]
 50%|█████     | 805/1610 [4:46:10<7:43:33, 34.55s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.9871266524793504, 'learning_rate': 5e-07, 'completion_length': 131.9285774230957, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.23086076974868774, 'kl': 0.02105712890625, 'epoch': 2.5}

 50%|█████     | 805/1610 [4:46:10<7:43:33, 34.55s/it]
 50%|█████     | 806/1610 [4:46:25<6:24:37, 28.70s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.4802572497472583, 'learning_rate': 4.993788819875776e-07, 'completion_length': 159.6428680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.016448974609375, 'epoch': 2.5}

 50%|█████     | 806/1610 [4:46:25<6:24:37, 28.70s/it]
 50%|█████     | 807/1610 [4:46:42<5:34:38, 25.00s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1620985118917813, 'learning_rate': 4.987577639751552e-07, 'completion_length': 144.5357208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2721000909805298, 'kl': 0.01715087890625, 'epoch': 2.51}

 50%|█████     | 807/1610 [4:46:42<5:34:38, 25.00s/it]
 50%|█████     | 808/1610 [4:46:56<4:49:06, 21.63s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.8057691640700402, 'learning_rate': 4.981366459627329e-07, 'completion_length': 147.62500762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.01434326171875, 'epoch': 2.51}

 50%|█████     | 808/1610 [4:46:56<4:49:06, 21.63s/it]
 50%|█████     | 809/1610 [4:47:09<4:14:44, 19.08s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8879480438175343, 'learning_rate': 4.975155279503105e-07, 'completion_length': 134.7857208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.14838216453790665, 'kl': 0.0145263671875, 'epoch': 2.51}

 50%|█████     | 809/1610 [4:47:09<4:14:44, 19.08s/it]
 50%|█████     | 810/1610 [4:47:25<4:03:52, 18.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0225841847813546, 'learning_rate': 4.968944099378881e-07, 'completion_length': 159.10714721679688, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.2967643216252327, 'kl': 0.01593017578125, 'epoch': 2.52}

 50%|█████     | 810/1610 [4:47:25<4:03:52, 18.29s/it]
 50%|█████     | 811/1610 [4:47:40<3:49:01, 17.20s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 4.578431194179083, 'learning_rate': 4.962732919254658e-07, 'completion_length': 144.76786041259766, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.014373779296875, 'epoch': 2.52}

 50%|█████     | 811/1610 [4:47:40<3:49:01, 17.20s/it]
 50%|█████     | 812/1610 [4:47:56<3:43:26, 16.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8185492392516399, 'learning_rate': 4.956521739130435e-07, 'completion_length': 147.0357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.012939453125, 'epoch': 2.52}

 50%|█████     | 812/1610 [4:47:56<3:43:26, 16.80s/it]
 50%|█████     | 813/1610 [4:48:12<3:42:26, 16.75s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8180422596292903, 'learning_rate': 4.950310559006211e-07, 'completion_length': 159.94644165039062, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8750000596046448, 'reward_std': 0.16546405106782913, 'kl': 0.014495849609375, 'epoch': 2.52}

 50%|█████     | 813/1610 [4:48:12<3:42:26, 16.75s/it]
 51%|█████     | 814/1610 [4:48:28<3:40:05, 16.59s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6246176272188191, 'learning_rate': 4.944099378881988e-07, 'completion_length': 192.57144165039062, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266788095235825, 'kl': 0.01751708984375, 'epoch': 2.53}

 51%|█████     | 814/1610 [4:48:28<3:40:05, 16.59s/it]
 51%|█████     | 815/1610 [4:48:43<3:30:27, 15.88s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7343374852428745, 'learning_rate': 4.937888198757764e-07, 'completion_length': 140.78572463989258, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.18409645557403564, 'kl': 0.0133056640625, 'epoch': 2.53}

 51%|█████     | 815/1610 [4:48:43<3:30:27, 15.88s/it]
 51%|█████     | 816/1610 [4:49:00<3:37:18, 16.42s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7067861376401419, 'learning_rate': 4.93167701863354e-07, 'completion_length': 136.51786041259766, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1181928999722004, 'kl': 0.01385498046875, 'epoch': 2.53}

 51%|█████     | 816/1610 [4:49:00<3:37:18, 16.42s/it]
 51%|█████     | 817/1610 [4:49:15<3:28:22, 15.77s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.4629959916683783, 'learning_rate': 4.925465838509317e-07, 'completion_length': 142.25000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0824786126613617, 'kl': 0.015472412109375, 'epoch': 2.54}

 51%|█████     | 817/1610 [4:49:15<3:28:22, 15.77s/it]
 51%|█████     | 818/1610 [4:49:27<3:16:02, 14.85s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.2593125470130238, 'learning_rate': 4.919254658385093e-07, 'completion_length': 131.08929061889648, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409645557403564, 'kl': 0.01849365234375, 'epoch': 2.54}

 51%|█████     | 818/1610 [4:49:27<3:16:02, 14.85s/it]
 51%|█████     | 819/1610 [4:49:43<3:20:38, 15.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8694651116838928, 'learning_rate': 4.913043478260869e-07, 'completion_length': 178.3928680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.01507568359375, 'epoch': 2.54}

 51%|█████     | 819/1610 [4:49:43<3:20:38, 15.22s/it]
 51%|█████     | 820/1610 [4:49:59<3:21:41, 15.32s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9442368023093669, 'learning_rate': 4.906832298136646e-07, 'completion_length': 150.9464340209961, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2253357619047165, 'kl': 0.013519287109375, 'epoch': 2.55}

 51%|█████     | 820/1610 [4:49:59<3:21:41, 15.32s/it]
 51%|█████     | 821/1610 [4:50:17<3:32:18, 16.14s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0874498350744966, 'learning_rate': 4.900621118012422e-07, 'completion_length': 164.1964340209961, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.1896214708685875, 'kl': 0.01837158203125, 'epoch': 2.55}

 51%|█████     | 821/1610 [4:50:17<3:32:18, 16.14s/it]
 51%|█████     | 822/1610 [4:50:33<3:32:10, 16.16s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1088255426190676, 'learning_rate': 4.894409937888198e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.15943220257759094, 'kl': 0.01519775390625, 'epoch': 2.55}

 51%|█████     | 822/1610 [4:50:33<3:32:10, 16.16s/it]
 51%|█████     | 823/1610 [4:50:46<3:19:13, 15.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.55786127915879, 'learning_rate': 4.888198757763975e-07, 'completion_length': 124.48214721679688, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.0357142873108387, 'kl': 0.014556884765625, 'epoch': 2.56}

 51%|█████     | 823/1610 [4:50:46<3:19:13, 15.19s/it]
 51%|█████     | 824/1610 [4:51:02<3:23:04, 15.50s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9620789474461489, 'learning_rate': 4.881987577639751e-07, 'completion_length': 193.1428680419922, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.25552502274513245, 'kl': 0.01708984375, 'epoch': 2.56}

 51%|█████     | 824/1610 [4:51:02<3:23:04, 15.50s/it]
 51%|█████     | 825/1610 [4:51:17<3:20:42, 15.34s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9425933637961031, 'learning_rate': 4.875776397515527e-07, 'completion_length': 150.23214721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.23086076974868774, 'kl': 0.014007568359375, 'epoch': 2.56}

 51%|█████     | 825/1610 [4:51:17<3:20:42, 15.34s/it]
 51%|█████▏    | 826/1610 [4:51:30<3:08:55, 14.46s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.7898677804768705, 'learning_rate': 4.869565217391305e-07, 'completion_length': 120.60714721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.18409645557403564, 'kl': 0.010345458984375, 'epoch': 2.57}

 51%|█████▏    | 826/1610 [4:51:30<3:08:55, 14.46s/it]
 51%|█████▏    | 827/1610 [4:51:47<3:18:26, 15.21s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.231461121621513, 'learning_rate': 4.863354037267081e-07, 'completion_length': 176.08928680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5714285969734192, 'reward_std': 0.2142857313156128, 'kl': 0.0164794921875, 'epoch': 2.57}

 51%|█████▏    | 827/1610 [4:51:47<3:18:26, 15.21s/it]
 51%|█████▏    | 828/1610 [4:52:01<3:13:09, 14.82s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1320726049838912, 'learning_rate': 4.857142857142857e-07, 'completion_length': 123.00000381469727, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.015472412109375, 'epoch': 2.57}

 51%|█████▏    | 828/1610 [4:52:01<3:13:09, 14.82s/it]
 51%|█████▏    | 829/1610 [4:52:16<3:15:24, 15.01s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1345632720225647, 'learning_rate': 4.850931677018633e-07, 'completion_length': 184.8214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266788095235825, 'kl': 0.01348876953125, 'epoch': 2.57}

 51%|█████▏    | 829/1610 [4:52:16<3:15:24, 15.01s/it]
 52%|█████▏    | 830/1610 [4:52:34<3:27:04, 15.93s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.7619590172677504, 'learning_rate': 4.84472049689441e-07, 'completion_length': 185.0178680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.25552502274513245, 'kl': 0.01934814453125, 'epoch': 2.58}

 52%|█████▏    | 830/1610 [4:52:34<3:27:04, 15.93s/it]
 52%|█████▏    | 831/1610 [4:52:51<3:28:57, 16.09s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.1383998007452256, 'learning_rate': 4.838509316770186e-07, 'completion_length': 154.85714721679688, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.29123931378126144, 'kl': 0.01885986328125, 'epoch': 2.58}

 52%|█████▏    | 831/1610 [4:52:51<3:28:57, 16.09s/it]
 52%|█████▏    | 832/1610 [4:53:04<3:18:07, 15.28s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.2913199990957116, 'learning_rate': 4.832298136645963e-07, 'completion_length': 148.7857208251953, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2363857999444008, 'kl': 0.01593017578125, 'epoch': 2.58}

 52%|█████▏    | 832/1610 [4:53:04<3:18:07, 15.28s/it]
 52%|█████▏    | 833/1610 [4:53:21<3:23:20, 15.70s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8713649741194979, 'learning_rate': 4.826086956521739e-07, 'completion_length': 162.4464340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357619047165, 'kl': 0.01611328125, 'epoch': 2.59}

 52%|█████▏    | 833/1610 [4:53:21<3:23:20, 15.70s/it]
 52%|█████▏    | 834/1610 [4:53:37<3:23:42, 15.75s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.4021658575763256, 'learning_rate': 4.819875776397515e-07, 'completion_length': 158.73214721679688, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2142857313156128, 'kl': 0.01678466796875, 'epoch': 2.59}

 52%|█████▏    | 834/1610 [4:53:37<3:23:42, 15.75s/it]
 52%|█████▏    | 835/1610 [4:53:55<3:35:23, 16.68s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.445600319769114, 'learning_rate': 4.813664596273292e-07, 'completion_length': 188.17858123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5357143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.016021728515625, 'epoch': 2.59}

 52%|█████▏    | 835/1610 [4:53:55<3:35:23, 16.68s/it]
 52%|█████▏    | 836/1610 [4:54:13<3:39:27, 17.01s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5658761355372525, 'learning_rate': 4.807453416149068e-07, 'completion_length': 161.7857208251953, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.1071428656578064, 'kl': 0.01611328125, 'epoch': 2.6}

 52%|█████▏    | 836/1610 [4:54:13<3:39:27, 17.01s/it]
 52%|█████▏    | 837/1610 [4:54:27<3:24:58, 15.91s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9431275733674342, 'learning_rate': 4.801242236024844e-07, 'completion_length': 140.00000381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.23638580739498138, 'kl': 0.0174560546875, 'epoch': 2.6}

 52%|█████▏    | 837/1610 [4:54:27<3:24:58, 15.91s/it]
 52%|█████▏    | 838/1610 [4:54:42<3:22:35, 15.75s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.8116740042634596, 'learning_rate': 4.795031055900621e-07, 'completion_length': 162.73214721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214783191681, 'kl': 0.016571044921875, 'epoch': 2.6}

 52%|█████▏    | 838/1610 [4:54:42<3:22:35, 15.75s/it]
 52%|█████▏    | 839/1610 [4:54:58<3:23:22, 15.83s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 3.143881174967097, 'learning_rate': 4.788819875776398e-07, 'completion_length': 142.1785774230957, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216453790665, 'kl': 0.01690673828125, 'epoch': 2.61}

 52%|█████▏    | 839/1610 [4:54:58<3:23:22, 15.83s/it]
 52%|█████▏    | 840/1610 [4:55:14<3:25:46, 16.03s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 2.008890572361954, 'learning_rate': 4.782608695652174e-07, 'completion_length': 146.71428680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3324786275625229, 'kl': 0.016204833984375, 'epoch': 2.61}

 52%|█████▏    | 840/1610 [4:55:14<3:25:46, 16.03s/it]
 52%|█████▏    | 841/1610 [4:55:29<3:18:59, 15.53s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8931931417649694, 'learning_rate': 4.77639751552795e-07, 'completion_length': 133.8214340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.016448974609375, 'epoch': 2.61}

 52%|█████▏    | 841/1610 [4:55:29<3:18:59, 15.53s/it]
 52%|█████▏    | 842/1610 [4:55:45<3:23:18, 15.88s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4158607210394147, 'learning_rate': 4.770186335403726e-07, 'completion_length': 149.9107208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.015655517578125, 'epoch': 2.61}

 52%|█████▏    | 842/1610 [4:55:45<3:23:18, 15.88s/it]
 52%|█████▏    | 843/1610 [4:56:03<3:29:35, 16.40s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2399219644770911, 'learning_rate': 4.763975155279503e-07, 'completion_length': 163.50000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.21981072798371315, 'kl': 0.017578125, 'epoch': 2.62}

 52%|█████▏    | 843/1610 [4:56:03<3:29:35, 16.40s/it]
 52%|█████▏    | 844/1610 [4:56:16<3:17:34, 15.48s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9082939500302075, 'learning_rate': 4.7577639751552796e-07, 'completion_length': 137.83929443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.018096923828125, 'epoch': 2.62}

 52%|█████▏    | 844/1610 [4:56:16<3:17:34, 15.48s/it]
 52%|█████▏    | 845/1610 [4:56:32<3:17:21, 15.48s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8473802229569015, 'learning_rate': 4.751552795031056e-07, 'completion_length': 134.75000381469727, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.01416015625, 'epoch': 2.62}

 52%|█████▏    | 845/1610 [4:56:32<3:17:21, 15.48s/it]
 53%|█████▎    | 846/1610 [4:56:46<3:12:17, 15.10s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1529778109039088, 'learning_rate': 4.7453416149068323e-07, 'completion_length': 155.05358123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23086076974868774, 'kl': 0.01776123046875, 'epoch': 2.63}

 53%|█████▎    | 846/1610 [4:56:46<3:12:17, 15.10s/it]
 53%|█████▎    | 847/1610 [4:57:01<3:11:02, 15.02s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9316944799004981, 'learning_rate': 4.739130434782608e-07, 'completion_length': 137.4107208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25552502274513245, 'kl': 0.018524169921875, 'epoch': 2.63}

 53%|█████▎    | 847/1610 [4:57:01<3:11:02, 15.02s/it]
 53%|█████▎    | 848/1610 [4:57:15<3:07:01, 14.73s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9254783019070314, 'learning_rate': 4.732919254658385e-07, 'completion_length': 139.3035774230957, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.1428571529686451, 'kl': 0.0137939453125, 'epoch': 2.63}

 53%|█████▎    | 848/1610 [4:57:15<3:07:01, 14.73s/it]
 53%|█████▎    | 849/1610 [4:57:28<3:00:20, 14.22s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7110006374957255, 'learning_rate': 4.7267080745341613e-07, 'completion_length': 128.96429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.014984130859375, 'epoch': 2.64}

 53%|█████▎    | 849/1610 [4:57:28<3:00:20, 14.22s/it]
 53%|█████▎    | 850/1610 [4:57:43<3:04:11, 14.54s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9020281517671845, 'learning_rate': 4.7204968944099376e-07, 'completion_length': 162.37500762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.01263427734375, 'epoch': 2.64}

 53%|█████▎    | 850/1610 [4:57:43<3:04:11, 14.54s/it]
 53%|█████▎    | 851/1610 [4:57:57<3:02:24, 14.42s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 2.076728053679118, 'learning_rate': 4.714285714285714e-07, 'completion_length': 123.62500381469727, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838217198848724, 'kl': 0.014678955078125, 'epoch': 2.64}

 53%|█████▎    | 851/1610 [4:57:57<3:02:24, 14.42s/it]
 53%|█████▎    | 852/1610 [4:58:13<3:05:41, 14.70s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.927493472016653, 'learning_rate': 4.70807453416149e-07, 'completion_length': 142.21429443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.15943220257759094, 'kl': 0.01702880859375, 'epoch': 2.65}

 53%|█████▎    | 852/1610 [4:58:13<3:05:41, 14.70s/it]
 53%|█████▎    | 853/1610 [4:58:25<2:54:31, 13.83s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.4596812461223219, 'learning_rate': 4.701863354037267e-07, 'completion_length': 125.12500381469727, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.26657506078481674, 'kl': 0.0191650390625, 'epoch': 2.65}

 53%|█████▎    | 853/1610 [4:58:25<2:54:31, 13.83s/it]
 53%|█████▎    | 854/1610 [4:58:40<3:00:26, 14.32s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.0563735713150202, 'learning_rate': 4.6956521739130434e-07, 'completion_length': 169.80357360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2610500305891037, 'kl': 0.02142333984375, 'epoch': 2.65}

 53%|█████▎    | 854/1610 [4:58:40<3:00:26, 14.32s/it]
 53%|█████▎    | 855/1610 [4:58:53<2:55:12, 13.92s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.124837273207026, 'learning_rate': 4.68944099378882e-07, 'completion_length': 131.62500381469727, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1896214708685875, 'kl': 0.017425537109375, 'epoch': 2.66}

 53%|█████▎    | 855/1610 [4:58:53<2:55:12, 13.92s/it]
 53%|█████▎    | 856/1610 [4:59:09<3:00:47, 14.39s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5485797250572801, 'learning_rate': 4.683229813664596e-07, 'completion_length': 142.80357360839844, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.015838623046875, 'epoch': 2.66}

 53%|█████▎    | 856/1610 [4:59:09<3:00:47, 14.39s/it]
 53%|█████▎    | 857/1610 [4:59:22<2:58:02, 14.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.0364832561849186, 'learning_rate': 4.6770186335403724e-07, 'completion_length': 149.55358123779297, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.0, 'kl': 0.015350341796875, 'epoch': 2.66}

 53%|█████▎    | 857/1610 [4:59:22<2:58:02, 14.19s/it]
 53%|█████▎    | 858/1610 [4:59:39<3:07:27, 14.96s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.5732994436592462, 'learning_rate': 4.670807453416149e-07, 'completion_length': 155.37500762939453, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.23086077719926834, 'kl': 0.01666259765625, 'epoch': 2.66}

 53%|█████▎    | 858/1610 [4:59:39<3:07:27, 14.96s/it]
 53%|█████▎    | 859/1610 [4:59:53<3:03:20, 14.65s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6227185936429936, 'learning_rate': 4.664596273291925e-07, 'completion_length': 159.83928680419922, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.011810302734375, 'epoch': 2.67}

 53%|█████▎    | 859/1610 [4:59:53<3:03:20, 14.65s/it]
 53%|█████▎    | 860/1610 [5:00:07<3:00:34, 14.45s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3497724780277909, 'learning_rate': 4.6583850931677014e-07, 'completion_length': 146.12500762939453, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2610500454902649, 'kl': 0.018218994140625, 'epoch': 2.67}

 53%|█████▎    | 860/1610 [5:00:07<3:00:34, 14.45s/it]
 53%|█████▎    | 861/1610 [5:00:22<3:04:05, 14.75s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 2.252983987014853, 'learning_rate': 4.6521739130434777e-07, 'completion_length': 149.6964340209961, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2142857238650322, 'kl': 0.01971435546875, 'epoch': 2.67}

 53%|█████▎    | 861/1610 [5:00:22<3:04:05, 14.75s/it]
 54%|█████▎    | 862/1610 [5:00:38<3:05:24, 14.87s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7398766820082541, 'learning_rate': 4.6459627329192546e-07, 'completion_length': 145.8035774230957, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.012786865234375, 'epoch': 2.68}

 54%|█████▎    | 862/1610 [5:00:38<3:05:24, 14.87s/it]
 54%|█████▎    | 863/1610 [5:00:56<3:19:16, 16.01s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9250959962021903, 'learning_rate': 4.639751552795031e-07, 'completion_length': 175.55358123779297, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1896214708685875, 'kl': 0.016021728515625, 'epoch': 2.68}

 54%|█████▎    | 863/1610 [5:00:56<3:19:16, 16.01s/it]
 54%|█████▎    | 864/1610 [5:01:12<3:19:48, 16.07s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1901531678875403, 'learning_rate': 4.633540372670807e-07, 'completion_length': 140.2857208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1428571529686451, 'kl': 0.01397705078125, 'epoch': 2.68}

 54%|█████▎    | 864/1610 [5:01:12<3:19:48, 16.07s/it]
 54%|█████▎    | 865/1610 [5:01:26<3:09:50, 15.29s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.4525577423954121, 'learning_rate': 4.6273291925465835e-07, 'completion_length': 123.60714721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.21981074661016464, 'kl': 0.015228271484375, 'epoch': 2.69}

 54%|█████▎    | 865/1610 [5:01:26<3:09:50, 15.29s/it]
 54%|█████▍    | 866/1610 [5:01:40<3:06:19, 15.03s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.2034121115145493, 'learning_rate': 4.62111801242236e-07, 'completion_length': 157.9464340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1785714402794838, 'kl': 0.0194091796875, 'epoch': 2.69}

 54%|█████▍    | 866/1610 [5:01:40<3:06:19, 15.03s/it]
 54%|█████▍    | 867/1610 [5:01:54<2:59:31, 14.50s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3005758241626422, 'learning_rate': 4.6149068322981367e-07, 'completion_length': 138.5714340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.01861572265625, 'epoch': 2.69}

 54%|█████▍    | 867/1610 [5:01:54<2:59:31, 14.50s/it]
 54%|█████▍    | 868/1610 [5:02:11<3:12:10, 15.54s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.47220682372108286, 'learning_rate': 4.608695652173913e-07, 'completion_length': 177.57144165039062, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.016876220703125, 'epoch': 2.7}

 54%|█████▍    | 868/1610 [5:02:12<3:12:10, 15.54s/it]
 54%|█████▍    | 869/1610 [5:02:26<3:09:10, 15.32s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.034482579694852, 'learning_rate': 4.6024844720496894e-07, 'completion_length': 152.1607208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214783191681, 'kl': 0.01959228515625, 'epoch': 2.7}

 54%|█████▍    | 869/1610 [5:02:26<3:09:10, 15.32s/it]
 54%|█████▍    | 870/1610 [5:02:41<3:06:56, 15.16s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.7852123969251323, 'learning_rate': 4.596273291925465e-07, 'completion_length': 141.37500762939453, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.29123930633068085, 'kl': 0.0206298828125, 'epoch': 2.7}

 54%|█████▍    | 870/1610 [5:02:41<3:06:56, 15.16s/it]
 54%|█████▍    | 871/1610 [5:02:53<2:54:04, 14.13s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.7896173236829263, 'learning_rate': 4.590062111801242e-07, 'completion_length': 128.73215103149414, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2967643141746521, 'kl': 0.01531982421875, 'epoch': 2.7}

 54%|█████▍    | 871/1610 [5:02:53<2:54:04, 14.13s/it]
 54%|█████▍    | 872/1610 [5:03:05<2:45:31, 13.46s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.087567864258304, 'learning_rate': 4.5838509316770183e-07, 'completion_length': 130.67858123779297, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.1071428619325161, 'kl': 0.014129638671875, 'epoch': 2.71}

 54%|█████▍    | 872/1610 [5:03:05<2:45:31, 13.46s/it]
 54%|█████▍    | 873/1610 [5:03:19<2:49:27, 13.80s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.102096545993781, 'learning_rate': 4.5776397515527947e-07, 'completion_length': 135.9821548461914, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.11266787722706795, 'kl': 0.01702880859375, 'epoch': 2.71}

 54%|█████▍    | 873/1610 [5:03:19<2:49:27, 13.80s/it]
 54%|█████▍    | 874/1610 [5:03:33<2:48:31, 13.74s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0869148467822958, 'learning_rate': 4.571428571428571e-07, 'completion_length': 135.46429443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428656578064, 'kl': 0.01739501953125, 'epoch': 2.71}

 54%|█████▍    | 874/1610 [5:03:33<2:48:31, 13.74s/it]
 54%|█████▍    | 875/1610 [5:03:46<2:45:42, 13.53s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.7975923947948436, 'learning_rate': 4.5652173913043473e-07, 'completion_length': 117.91072082519531, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214783191681, 'kl': 0.01947021484375, 'epoch': 2.72}

 54%|█████▍    | 875/1610 [5:03:46<2:45:42, 13.53s/it]
 54%|█████▍    | 876/1610 [5:04:03<2:59:09, 14.64s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6999210333816911, 'learning_rate': 4.559006211180124e-07, 'completion_length': 155.21429443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.821428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.01739501953125, 'epoch': 2.72}

 54%|█████▍    | 876/1610 [5:04:03<2:59:09, 14.64s/it]
 54%|█████▍    | 877/1610 [5:04:18<2:58:22, 14.60s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1645776042961948, 'learning_rate': 4.5527950310559005e-07, 'completion_length': 134.71429443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1896214634180069, 'kl': 0.015472412109375, 'epoch': 2.72}

 54%|█████▍    | 877/1610 [5:04:18<2:58:22, 14.60s/it]
 55%|█████▍    | 878/1610 [5:04:33<3:00:49, 14.82s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 3.0836277497778437, 'learning_rate': 4.546583850931677e-07, 'completion_length': 153.75000762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.2721000760793686, 'kl': 0.022216796875, 'epoch': 2.73}

 55%|█████▍    | 878/1610 [5:04:33<3:00:49, 14.82s/it]
 55%|█████▍    | 879/1610 [5:04:49<3:03:59, 15.10s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1830856585996947, 'learning_rate': 4.540372670807453e-07, 'completion_length': 157.3214340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.20067152380943298, 'kl': 0.016754150390625, 'epoch': 2.73}

 55%|█████▍    | 879/1610 [5:04:49<3:03:59, 15.10s/it]
 55%|█████▍    | 880/1610 [5:05:04<3:03:30, 15.08s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.185207960587262, 'learning_rate': 4.53416149068323e-07, 'completion_length': 154.64286422729492, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657505333423615, 'kl': 0.01544189453125, 'epoch': 2.73}

 55%|█████▍    | 880/1610 [5:05:04<3:03:30, 15.08s/it]
 55%|█████▍    | 881/1610 [5:05:23<3:16:43, 16.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7569845523958748, 'learning_rate': 4.5279503105590063e-07, 'completion_length': 199.75000762939453, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5535714626312256, 'reward_std': 0.2610500529408455, 'kl': 0.01483154296875, 'epoch': 2.74}

 55%|█████▍    | 881/1610 [5:05:23<3:16:43, 16.19s/it]
 55%|█████▍    | 882/1610 [5:05:38<3:15:13, 16.09s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2682506401004936, 'learning_rate': 4.521739130434782e-07, 'completion_length': 135.75000381469727, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1539071798324585, 'kl': 0.01837158203125, 'epoch': 2.74}

 55%|█████▍    | 882/1610 [5:05:38<3:15:13, 16.09s/it]
 55%|█████▍    | 883/1610 [5:05:55<3:15:10, 16.11s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6429800691129737, 'learning_rate': 4.5155279503105585e-07, 'completion_length': 151.42858123779297, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.015045166015625, 'epoch': 2.74}

 55%|█████▍    | 883/1610 [5:05:55<3:15:10, 16.11s/it]
 55%|█████▍    | 884/1610 [5:06:11<3:17:37, 16.33s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.8308405472770233, 'learning_rate': 4.509316770186335e-07, 'completion_length': 168.0357208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.0213623046875, 'epoch': 2.75}

 55%|█████▍    | 884/1610 [5:06:11<3:17:37, 16.33s/it]
 55%|█████▍    | 885/1610 [5:06:26<3:09:48, 15.71s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9153229160748867, 'learning_rate': 4.5031055900621116e-07, 'completion_length': 149.3571548461914, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.14838216826319695, 'kl': 0.013641357421875, 'epoch': 2.75}

 55%|█████▍    | 885/1610 [5:06:26<3:09:48, 15.71s/it]
 55%|█████▌    | 886/1610 [5:06:43<3:14:35, 16.13s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.7470135114701891, 'learning_rate': 4.496894409937888e-07, 'completion_length': 160.83928680419922, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.0198974609375, 'epoch': 2.75}

 55%|█████▌    | 886/1610 [5:06:43<3:14:35, 16.13s/it]
 55%|█████▌    | 887/1610 [5:07:00<3:17:47, 16.41s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9489450182519261, 'learning_rate': 4.4906832298136643e-07, 'completion_length': 155.21429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.18409644439816475, 'kl': 0.018798828125, 'epoch': 2.75}

 55%|█████▌    | 887/1610 [5:07:00<3:17:47, 16.41s/it]
 55%|█████▌    | 888/1610 [5:07:16<3:15:58, 16.29s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7023249247639582, 'learning_rate': 4.4844720496894406e-07, 'completion_length': 168.5714340209961, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.016876220703125, 'epoch': 2.76}

 55%|█████▌    | 888/1610 [5:07:16<3:15:58, 16.29s/it]
 55%|█████▌    | 889/1610 [5:07:32<3:14:56, 16.22s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.26384397206499, 'learning_rate': 4.4782608695652175e-07, 'completion_length': 142.12500762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0357142873108387, 'kl': 0.018310546875, 'epoch': 2.76}

 55%|█████▌    | 889/1610 [5:07:32<3:14:56, 16.22s/it]
 55%|█████▌    | 890/1610 [5:07:48<3:15:40, 16.31s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.1000916507310723, 'learning_rate': 4.472049689440994e-07, 'completion_length': 155.7678680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.1428571529686451, 'kl': 0.012664794921875, 'epoch': 2.76}

 55%|█████▌    | 890/1610 [5:07:48<3:15:40, 16.31s/it]
 55%|█████▌    | 891/1610 [5:08:02<3:06:58, 15.60s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6948739709791284, 'learning_rate': 4.46583850931677e-07, 'completion_length': 130.08929443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1181928962469101, 'kl': 0.012939453125, 'epoch': 2.77}

 55%|█████▌    | 891/1610 [5:08:02<3:06:58, 15.60s/it]
 55%|█████▌    | 892/1610 [5:08:20<3:13:10, 16.14s/it]
                                                      
{'loss': 0.001, 'grad_norm': 1.4795457931598759, 'learning_rate': 4.4596273291925464e-07, 'completion_length': 150.1607208251953, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.02392578125, 'epoch': 2.77}

 55%|█████▌    | 892/1610 [5:08:20<3:13:10, 16.14s/it]
 55%|█████▌    | 893/1610 [5:08:35<3:08:36, 15.78s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.49582226802972046, 'learning_rate': 4.453416149068323e-07, 'completion_length': 163.08929443359375, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071872830391, 'kl': 0.0133056640625, 'epoch': 2.77}

 55%|█████▌    | 893/1610 [5:08:35<3:08:36, 15.78s/it]
 56%|█████▌    | 894/1610 [5:08:49<3:01:32, 15.21s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 3.1969885552004538, 'learning_rate': 4.447204968944099e-07, 'completion_length': 150.21428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.19514648616313934, 'kl': 0.015899658203125, 'epoch': 2.78}

 56%|█████▌    | 894/1610 [5:08:49<3:01:32, 15.21s/it]
 56%|█████▌    | 895/1610 [5:09:00<2:46:51, 14.00s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 1.0359837453155414, 'learning_rate': 4.4409937888198754e-07, 'completion_length': 114.53572082519531, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071798324585, 'kl': 0.01007080078125, 'epoch': 2.78}

 56%|█████▌    | 895/1610 [5:09:00<2:46:51, 14.00s/it]
 56%|█████▌    | 896/1610 [5:09:17<2:56:58, 14.87s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6323269515544263, 'learning_rate': 4.434782608695652e-07, 'completion_length': 160.33929443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.07695358991622925, 'kl': 0.015380859375, 'epoch': 2.78}

 56%|█████▌    | 896/1610 [5:09:17<2:56:58, 14.87s/it]
 56%|█████▌    | 897/1610 [5:09:33<3:02:18, 15.34s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.9041397442401544, 'learning_rate': 4.428571428571428e-07, 'completion_length': 184.37500762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.23086077719926834, 'kl': 0.0191650390625, 'epoch': 2.79}

 56%|█████▌    | 897/1610 [5:09:33<3:02:18, 15.34s/it]
 56%|█████▌    | 898/1610 [5:09:45<2:50:20, 14.35s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0554595612847106, 'learning_rate': 4.422360248447205e-07, 'completion_length': 119.26786041259766, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1071428619325161, 'kl': 0.013275146484375, 'epoch': 2.79}

 56%|█████▌    | 898/1610 [5:09:45<2:50:20, 14.35s/it]
 56%|█████▌    | 899/1610 [5:10:01<2:54:52, 14.76s/it]
                                                      
{'loss': 0.0004, 'grad_norm': 0.3409230017242466, 'learning_rate': 4.416149068322981e-07, 'completion_length': 146.67857360839844, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.0357142873108387, 'kl': 0.010345458984375, 'epoch': 2.79}

 56%|█████▌    | 899/1610 [5:10:01<2:54:52, 14.76s/it]
 56%|█████▌    | 900/1610 [5:10:15<2:53:00, 14.62s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8581878253416519, 'learning_rate': 4.4099378881987576e-07, 'completion_length': 163.12500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.2142857238650322, 'kl': 0.01812744140625, 'epoch': 2.8}

 56%|█████▌    | 900/1610 [5:10:15<2:53:00, 14.62s/it]
 56%|█████▌    | 901/1610 [5:13:56<15:02:27, 76.37s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9599466713609006, 'learning_rate': 4.403726708074534e-07, 'completion_length': 187.2321548461914, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214708685875, 'kl': 0.0186767578125, 'epoch': 2.8}

 56%|█████▌    | 901/1610 [5:13:56<15:02:27, 76.37s/it]
 56%|█████▌    | 902/1610 [5:14:12<11:29:52, 58.46s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0632427614023097, 'learning_rate': 4.39751552795031e-07, 'completion_length': 176.08929443359375, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.014068603515625, 'epoch': 2.8}

 56%|█████▌    | 902/1610 [5:14:12<11:29:52, 58.46s/it]
 56%|█████▌    | 903/1610 [5:14:25<8:46:01, 44.64s/it] 
                                                      
{'loss': 0.0007, 'grad_norm': 0.6197012217984778, 'learning_rate': 4.391304347826087e-07, 'completion_length': 134.92857360839844, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.11266787722706795, 'kl': 0.01763916015625, 'epoch': 2.8}

 56%|█████▌    | 903/1610 [5:14:25<8:46:01, 44.64s/it]
 56%|█████▌    | 904/1610 [5:14:40<7:00:22, 35.73s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7116779697325033, 'learning_rate': 4.3850931677018634e-07, 'completion_length': 160.57144165039062, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.18409644439816475, 'kl': 0.0167236328125, 'epoch': 2.81}

 56%|█████▌    | 904/1610 [5:14:40<7:00:22, 35.73s/it]
 56%|█████▌    | 905/1610 [5:14:52<5:37:27, 28.72s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9230536233570078, 'learning_rate': 4.3788819875776397e-07, 'completion_length': 136.12500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.01434326171875, 'epoch': 2.81}

 56%|█████▌    | 905/1610 [5:14:52<5:37:27, 28.72s/it]
 56%|█████▋    | 906/1610 [5:15:08<4:53:12, 24.99s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 0.7331850344732503, 'learning_rate': 4.3726708074534155e-07, 'completion_length': 175.14286041259766, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1181928962469101, 'kl': 0.02191162109375, 'epoch': 2.81}

 56%|█████▋    | 906/1610 [5:15:08<4:53:12, 24.99s/it]
 56%|█████▋    | 907/1610 [5:15:23<4:16:17, 21.87s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8352356073433637, 'learning_rate': 4.3664596273291924e-07, 'completion_length': 142.7321548461914, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.21981074661016464, 'kl': 0.01824951171875, 'epoch': 2.82}

 56%|█████▋    | 907/1610 [5:15:23<4:16:17, 21.87s/it]
 56%|█████▋    | 908/1610 [5:15:36<3:45:02, 19.23s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.9386187931323373, 'learning_rate': 4.3602484472049687e-07, 'completion_length': 135.10714721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2967643141746521, 'kl': 0.01324462890625, 'epoch': 2.82}

 56%|█████▋    | 908/1610 [5:15:36<3:45:02, 19.23s/it]
 56%|█████▋    | 909/1610 [5:15:52<3:31:44, 18.12s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.332435518090154, 'learning_rate': 4.354037267080745e-07, 'completion_length': 167.33929443359375, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2967643290758133, 'kl': 0.021484375, 'epoch': 2.82}

 56%|█████▋    | 909/1610 [5:15:52<3:31:44, 18.12s/it]
 57%|█████▋    | 910/1610 [5:16:08<3:25:05, 17.58s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9617636924227324, 'learning_rate': 4.3478260869565214e-07, 'completion_length': 183.1964340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.25552502274513245, 'kl': 0.01727294921875, 'epoch': 2.83}

 57%|█████▋    | 910/1610 [5:16:08<3:25:05, 17.58s/it]
 57%|█████▋    | 911/1610 [5:16:23<3:16:13, 16.84s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8790139047448629, 'learning_rate': 4.3416149068322977e-07, 'completion_length': 149.4107208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1428571529686451, 'kl': 0.01580810546875, 'epoch': 2.83}

 57%|█████▋    | 911/1610 [5:16:23<3:16:13, 16.84s/it]
 57%|█████▋    | 912/1610 [5:16:37<3:06:33, 16.04s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.4594788683187419, 'learning_rate': 4.3354037267080745e-07, 'completion_length': 131.33929443359375, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0714285746216774, 'kl': 0.0162353515625, 'epoch': 2.83}

 57%|█████▋    | 912/1610 [5:16:37<3:06:33, 16.04s/it]
 57%|█████▋    | 913/1610 [5:16:50<2:55:46, 15.13s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9456224845839349, 'learning_rate': 4.329192546583851e-07, 'completion_length': 138.4464340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.01409912109375, 'epoch': 2.84}

 57%|█████▋    | 913/1610 [5:16:50<2:55:46, 15.13s/it]
 57%|█████▋    | 914/1610 [5:17:06<2:57:57, 15.34s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.823978936777993, 'learning_rate': 4.322981366459627e-07, 'completion_length': 157.75000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.14838216453790665, 'kl': 0.0147705078125, 'epoch': 2.84}

 57%|█████▋    | 914/1610 [5:17:06<2:57:57, 15.34s/it]
 57%|█████▋    | 915/1610 [5:17:21<2:56:10, 15.21s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.6287744155239872, 'learning_rate': 4.3167701863354035e-07, 'completion_length': 156.8214340209961, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.01739501953125, 'epoch': 2.84}

 57%|█████▋    | 915/1610 [5:17:21<2:56:10, 15.21s/it]
 57%|█████▋    | 916/1610 [5:17:34<2:47:39, 14.49s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.5754967154274657, 'learning_rate': 4.3105590062111804e-07, 'completion_length': 121.8214340209961, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.01239013671875, 'epoch': 2.84}

 57%|█████▋    | 916/1610 [5:17:34<2:47:39, 14.49s/it]
 57%|█████▋    | 917/1610 [5:17:48<2:46:23, 14.41s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8323841906506964, 'learning_rate': 4.3043478260869567e-07, 'completion_length': 158.42858123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357470035553, 'kl': 0.0159912109375, 'epoch': 2.85}

 57%|█████▋    | 917/1610 [5:17:48<2:46:23, 14.41s/it]
 57%|█████▋    | 918/1610 [5:18:02<2:44:55, 14.30s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.5327267157135666, 'learning_rate': 4.2981366459627325e-07, 'completion_length': 142.25000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.0714285746216774, 'kl': 0.0203857421875, 'epoch': 2.85}

 57%|█████▋    | 918/1610 [5:18:02<2:44:55, 14.30s/it]
 57%|█████▋    | 919/1610 [5:18:18<2:51:03, 14.85s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.5110167007556349, 'learning_rate': 4.291925465838509e-07, 'completion_length': 136.48215103149414, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.0172119140625, 'epoch': 2.85}

 57%|█████▋    | 919/1610 [5:18:18<2:51:03, 14.85s/it]
 57%|█████▋    | 920/1610 [5:18:34<2:55:20, 15.25s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.635815949641896, 'learning_rate': 4.285714285714285e-07, 'completion_length': 149.1964340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.23086077719926834, 'kl': 0.013885498046875, 'epoch': 2.86}

 57%|█████▋    | 920/1610 [5:18:34<2:55:20, 15.25s/it]
 57%|█████▋    | 921/1610 [5:18:47<2:47:28, 14.58s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.3463867072703768, 'learning_rate': 4.279503105590062e-07, 'completion_length': 132.4107208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.0152587890625, 'epoch': 2.86}

 57%|█████▋    | 921/1610 [5:18:47<2:47:28, 14.58s/it]
 57%|█████▋    | 922/1610 [5:19:01<2:45:35, 14.44s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.6021236241468815, 'learning_rate': 4.2732919254658383e-07, 'completion_length': 137.62500762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11266787722706795, 'kl': 0.013275146484375, 'epoch': 2.86}

 57%|█████▋    | 922/1610 [5:19:01<2:45:35, 14.44s/it]
 57%|█████▋    | 923/1610 [5:19:17<2:50:08, 14.86s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3558661973816029, 'learning_rate': 4.2670807453416146e-07, 'completion_length': 174.00000762939453, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2967643216252327, 'kl': 0.017333984375, 'epoch': 2.87}

 57%|█████▋    | 923/1610 [5:19:17<2:50:08, 14.86s/it]
 57%|█████▋    | 924/1610 [5:19:34<2:58:01, 15.57s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3049921324057039, 'learning_rate': 4.260869565217391e-07, 'completion_length': 183.8571548461914, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.0179443359375, 'epoch': 2.87}

 57%|█████▋    | 924/1610 [5:19:34<2:58:01, 15.57s/it]
 57%|█████▋    | 925/1610 [5:19:48<2:49:27, 14.84s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9239170708410824, 'learning_rate': 4.254658385093168e-07, 'completion_length': 127.91071701049805, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.19514648616313934, 'kl': 0.0174560546875, 'epoch': 2.87}

 57%|█████▋    | 925/1610 [5:19:48<2:49:27, 14.84s/it]
 58%|█████▊    | 926/1610 [5:20:00<2:41:57, 14.21s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.042871532326354, 'learning_rate': 4.248447204968944e-07, 'completion_length': 136.7678680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2721000984311104, 'kl': 0.01812744140625, 'epoch': 2.88}

 58%|█████▊    | 926/1610 [5:20:00<2:41:57, 14.21s/it]
 58%|█████▊    | 927/1610 [5:20:18<2:52:13, 15.13s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.9308893638060212, 'learning_rate': 4.2422360248447205e-07, 'completion_length': 163.33929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571529686451, 'kl': 0.016845703125, 'epoch': 2.88}

 58%|█████▊    | 927/1610 [5:20:18<2:52:13, 15.13s/it]
 58%|█████▊    | 928/1610 [5:20:34<2:57:13, 15.59s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6272011741803027, 'learning_rate': 4.236024844720497e-07, 'completion_length': 183.33929443359375, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.01409912109375, 'epoch': 2.88}

 58%|█████▊    | 928/1610 [5:20:34<2:57:13, 15.59s/it]
 58%|█████▊    | 929/1610 [5:20:49<2:53:19, 15.27s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6839864117750926, 'learning_rate': 4.229813664596273e-07, 'completion_length': 148.73214721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266788095235825, 'kl': 0.014862060546875, 'epoch': 2.89}

 58%|█████▊    | 929/1610 [5:20:49<2:53:19, 15.27s/it]
 58%|█████▊    | 930/1610 [5:21:02<2:47:40, 14.79s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.80300806433673, 'learning_rate': 4.2236024844720495e-07, 'completion_length': 138.83929443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1539071798324585, 'kl': 0.01715087890625, 'epoch': 2.89}

 58%|█████▊    | 930/1610 [5:21:02<2:47:40, 14.79s/it]
 58%|█████▊    | 931/1610 [5:21:19<2:53:35, 15.34s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.0554691524710471, 'learning_rate': 4.217391304347826e-07, 'completion_length': 158.26786041259766, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.25552503019571304, 'kl': 0.013519287109375, 'epoch': 2.89}

 58%|█████▊    | 931/1610 [5:21:19<2:53:35, 15.34s/it]
 58%|█████▊    | 932/1610 [5:21:35<2:53:44, 15.37s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.256794094501626, 'learning_rate': 4.211180124223602e-07, 'completion_length': 168.25000762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.0357142873108387, 'kl': 0.015899658203125, 'epoch': 2.89}

 58%|█████▊    | 932/1610 [5:21:35<2:53:44, 15.37s/it]
 58%|█████▊    | 933/1610 [5:21:49<2:51:15, 15.18s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6988286624666279, 'learning_rate': 4.2049689440993784e-07, 'completion_length': 139.51786041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.15943220257759094, 'kl': 0.015380859375, 'epoch': 2.9}

 58%|█████▊    | 933/1610 [5:21:49<2:51:15, 15.18s/it]
 58%|█████▊    | 934/1610 [5:22:03<2:45:11, 14.66s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7045105403004831, 'learning_rate': 4.1987577639751553e-07, 'completion_length': 140.3928680419922, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428619325161, 'kl': 0.017578125, 'epoch': 2.9}

 58%|█████▊    | 934/1610 [5:22:03<2:45:11, 14.66s/it]
 58%|█████▊    | 935/1610 [5:22:19<2:51:13, 15.22s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.165578401079098, 'learning_rate': 4.1925465838509316e-07, 'completion_length': 170.35714721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.30228933691978455, 'kl': 0.02093505859375, 'epoch': 2.9}

 58%|█████▊    | 935/1610 [5:22:19<2:51:13, 15.22s/it]
 58%|█████▊    | 936/1610 [5:22:34<2:48:11, 14.97s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.7479843402805864, 'learning_rate': 4.186335403726708e-07, 'completion_length': 140.08928680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3324785977602005, 'kl': 0.0174560546875, 'epoch': 2.91}

 58%|█████▊    | 936/1610 [5:22:34<2:48:11, 14.97s/it]
 58%|█████▊    | 937/1610 [5:22:51<2:55:38, 15.66s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7418041094214738, 'learning_rate': 4.180124223602484e-07, 'completion_length': 174.1428680419922, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.19514648616313934, 'kl': 0.017120361328125, 'epoch': 2.91}

 58%|█████▊    | 937/1610 [5:22:51<2:55:38, 15.66s/it]
 58%|█████▊    | 938/1610 [5:23:10<3:08:03, 16.79s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8040184738986691, 'learning_rate': 4.1739130434782606e-07, 'completion_length': 192.16072845458984, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.018310546875, 'epoch': 2.91}

 58%|█████▊    | 938/1610 [5:23:10<3:08:03, 16.79s/it]
 58%|█████▊    | 939/1610 [5:23:29<3:14:40, 17.41s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.0843628773819134, 'learning_rate': 4.1677018633540374e-07, 'completion_length': 153.64286422729492, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.2610500380396843, 'kl': 0.016387939453125, 'epoch': 2.92}

 58%|█████▊    | 939/1610 [5:23:29<3:14:40, 17.41s/it]
 58%|█████▊    | 940/1610 [5:23:47<3:15:42, 17.53s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3119080710504778, 'learning_rate': 4.161490683229814e-07, 'completion_length': 170.9464340209961, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.26657505333423615, 'kl': 0.016357421875, 'epoch': 2.92}

 58%|█████▊    | 940/1610 [5:23:47<3:15:42, 17.53s/it]
 58%|█████▊    | 941/1610 [5:24:03<3:09:25, 16.99s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5285424443801616, 'learning_rate': 4.15527950310559e-07, 'completion_length': 157.83929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2967643290758133, 'kl': 0.01837158203125, 'epoch': 2.92}

 58%|█████▊    | 941/1610 [5:24:03<3:09:25, 16.99s/it]
 59%|█████▊    | 942/1610 [5:24:19<3:06:47, 16.78s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5774918138371572, 'learning_rate': 4.149068322981366e-07, 'completion_length': 161.33929443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1785714402794838, 'kl': 0.01629638671875, 'epoch': 2.93}

 59%|█████▊    | 942/1610 [5:24:19<3:06:47, 16.78s/it]
 59%|█████▊    | 943/1610 [5:24:37<3:09:22, 17.03s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.3105575109988916, 'learning_rate': 4.142857142857143e-07, 'completion_length': 201.46429443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.1428571492433548, 'kl': 0.017791748046875, 'epoch': 2.93}

 59%|█████▊    | 943/1610 [5:24:37<3:09:22, 17.03s/it]
 59%|█████▊    | 944/1610 [5:24:55<3:11:48, 17.28s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.20680308495924, 'learning_rate': 4.136645962732919e-07, 'completion_length': 144.30357360839844, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.2142857238650322, 'kl': 0.017822265625, 'epoch': 2.93}

 59%|█████▊    | 944/1610 [5:24:55<3:11:48, 17.28s/it]
 59%|█████▊    | 945/1610 [5:25:12<3:11:34, 17.28s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.7452756840216204, 'learning_rate': 4.1304347826086954e-07, 'completion_length': 169.60714721679688, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7142857909202576, 'reward_std': 0.2253357470035553, 'kl': 0.017822265625, 'epoch': 2.93}

 59%|█████▊    | 945/1610 [5:25:12<3:11:34, 17.28s/it]
 59%|█████▉    | 946/1610 [5:25:30<3:14:19, 17.56s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7746131163632455, 'learning_rate': 4.1242236024844717e-07, 'completion_length': 167.46429443359375, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.24191083014011383, 'kl': 0.015289306640625, 'epoch': 2.94}

 59%|█████▉    | 946/1610 [5:25:30<3:14:19, 17.56s/it]
 59%|█████▉    | 947/1610 [5:25:44<3:03:09, 16.57s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5778222098077153, 'learning_rate': 4.118012422360248e-07, 'completion_length': 144.4107208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1181928999722004, 'kl': 0.016021728515625, 'epoch': 2.94}

 59%|█████▉    | 947/1610 [5:25:44<3:03:09, 16.57s/it]
 59%|█████▉    | 948/1610 [5:25:57<2:49:21, 15.35s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.1593222826962821, 'learning_rate': 4.111801242236025e-07, 'completion_length': 118.64286041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.23086077719926834, 'kl': 0.014434814453125, 'epoch': 2.94}

 59%|█████▉    | 948/1610 [5:25:57<2:49:21, 15.35s/it]
 59%|█████▉    | 949/1610 [5:26:15<2:57:18, 16.10s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.9216031531965516, 'learning_rate': 4.105590062111801e-07, 'completion_length': 180.1964340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5714285969734192, 'reward_std': 0.2967643141746521, 'kl': 0.0223388671875, 'epoch': 2.95}

 59%|█████▉    | 949/1610 [5:26:15<2:57:18, 16.10s/it]
 59%|█████▉    | 950/1610 [5:26:31<2:59:04, 16.28s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.4157046564959856, 'learning_rate': 4.0993788819875776e-07, 'completion_length': 167.25, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1785714402794838, 'kl': 0.020751953125, 'epoch': 2.95}

 59%|█████▉    | 950/1610 [5:26:31<2:59:04, 16.28s/it]
 59%|█████▉    | 951/1610 [5:26:48<3:00:09, 16.40s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.857726214609041, 'learning_rate': 4.093167701863354e-07, 'completion_length': 157.2857208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.02325439453125, 'epoch': 2.95}

 59%|█████▉    | 951/1610 [5:26:48<3:00:09, 16.40s/it]
 59%|█████▉    | 952/1610 [5:27:04<2:58:54, 16.31s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1321892437969776, 'learning_rate': 4.0869565217391307e-07, 'completion_length': 174.89286041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.01824951171875, 'epoch': 2.96}

 59%|█████▉    | 952/1610 [5:27:04<2:58:54, 16.31s/it]
 59%|█████▉    | 953/1610 [5:27:21<3:01:27, 16.57s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.7050150877048535, 'learning_rate': 4.080745341614907e-07, 'completion_length': 175.55358123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1649572253227234, 'kl': 0.02044677734375, 'epoch': 2.96}

 59%|█████▉    | 953/1610 [5:27:21<3:01:27, 16.57s/it]
 59%|█████▉    | 954/1610 [5:27:37<2:59:42, 16.44s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.6455067828761784, 'learning_rate': 4.074534161490683e-07, 'completion_length': 173.08929443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1181928962469101, 'kl': 0.015625, 'epoch': 2.96}

 59%|█████▉    | 954/1610 [5:27:37<2:59:42, 16.44s/it]
 59%|█████▉    | 955/1610 [5:27:56<3:07:25, 17.17s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.5060535759358857, 'learning_rate': 4.068322981366459e-07, 'completion_length': 163.42857360839844, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.01873779296875, 'epoch': 2.97}

 59%|█████▉    | 955/1610 [5:27:56<3:07:25, 17.17s/it]
 59%|█████▉    | 956/1610 [5:28:09<2:53:57, 15.96s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7978192481504499, 'learning_rate': 4.0621118012422355e-07, 'completion_length': 149.25000762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409644439816475, 'kl': 0.0147705078125, 'epoch': 2.97}

 59%|█████▉    | 956/1610 [5:28:09<2:53:57, 15.96s/it]
 59%|█████▉    | 957/1610 [5:28:24<2:48:22, 15.47s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9173253501358949, 'learning_rate': 4.0559006211180124e-07, 'completion_length': 156.96428680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071835577488, 'kl': 0.015045166015625, 'epoch': 2.97}

 59%|█████▉    | 957/1610 [5:28:24<2:48:22, 15.47s/it]
 60%|█████▉    | 958/1610 [5:28:42<2:55:53, 16.19s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.429741656468405, 'learning_rate': 4.0496894409937887e-07, 'completion_length': 152.92858123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.660714328289032, 'reward_std': 0.1896214708685875, 'kl': 0.014312744140625, 'epoch': 2.98}

 60%|█████▉    | 958/1610 [5:28:42<2:55:53, 16.19s/it]
 60%|█████▉    | 959/1610 [5:28:56<2:50:28, 15.71s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 2.1351412570093387, 'learning_rate': 4.043478260869565e-07, 'completion_length': 133.62500762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.21981074661016464, 'kl': 0.0145263671875, 'epoch': 2.98}

 60%|█████▉    | 959/1610 [5:28:56<2:50:28, 15.71s/it]
 60%|█████▉    | 960/1610 [5:29:14<2:57:05, 16.35s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7329664874120624, 'learning_rate': 4.0372670807453413e-07, 'completion_length': 133.3214340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500000596046448, 'reward_std': 0.2142857313156128, 'kl': 0.01275634765625, 'epoch': 2.98}

 60%|█████▉    | 960/1610 [5:29:14<2:57:05, 16.35s/it]
 60%|█████▉    | 961/1610 [5:29:27<2:46:11, 15.37s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 3.078203613034669, 'learning_rate': 4.0310559006211177e-07, 'completion_length': 144.26786041259766, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.33800362050533295, 'kl': 0.017333984375, 'epoch': 2.98}

 60%|█████▉    | 961/1610 [5:29:27<2:46:11, 15.37s/it]
 60%|█████▉    | 962/1610 [5:29:43<2:47:38, 15.52s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8556983162216504, 'learning_rate': 4.0248447204968945e-07, 'completion_length': 181.17858123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571529686451, 'kl': 0.013702392578125, 'epoch': 2.99}

 60%|█████▉    | 962/1610 [5:29:43<2:47:38, 15.52s/it]
 60%|█████▉    | 963/1610 [5:29:57<2:42:02, 15.03s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.06109857572216, 'learning_rate': 4.018633540372671e-07, 'completion_length': 136.9464340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.2831501364707947, 'kl': 0.013153076171875, 'epoch': 2.99}

 60%|█████▉    | 963/1610 [5:29:57<2:42:02, 15.03s/it]
 60%|█████▉    | 964/1610 [5:30:11<2:37:51, 14.66s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.8489903188254297, 'learning_rate': 4.012422360248447e-07, 'completion_length': 144.6428680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.01263427734375, 'epoch': 2.99}

 60%|█████▉    | 964/1610 [5:30:11<2:37:51, 14.66s/it]
 60%|█████▉    | 965/1610 [5:30:28<2:45:27, 15.39s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.1353207709959627, 'learning_rate': 4.006211180124223e-07, 'completion_length': 160.01786041259766, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4285714626312256, 'reward_std': 0.21676982194185257, 'kl': 0.017974853515625, 'epoch': 3.0}

 60%|█████▉    | 965/1610 [5:30:28<2:45:27, 15.39s/it]
 60%|██████    | 966/1610 [5:30:45<2:52:24, 16.06s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.0339005868799034, 'learning_rate': 4e-07, 'completion_length': 177.23214721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.26657507568597794, 'kl': 0.019989013671875, 'epoch': 3.0}

 60%|██████    | 966/1610 [5:30:45<2:52:24, 16.06s/it]
 60%|██████    | 967/1610 [5:31:06<3:06:20, 17.39s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.8091654758196509, 'learning_rate': 3.993788819875776e-07, 'completion_length': 175.0178680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5892857909202576, 'reward_std': 0.25248410552740097, 'kl': 0.015167236328125, 'epoch': 3.0}

 60%|██████    | 967/1610 [5:31:06<3:06:20, 17.39s/it]
 60%|██████    | 968/1610 [5:31:18<2:49:05, 15.80s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.7086334701063556, 'learning_rate': 3.9875776397515525e-07, 'completion_length': 120.58929061889648, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.07695359364151955, 'kl': 0.01287841796875, 'epoch': 3.01}

 60%|██████    | 968/1610 [5:31:18<2:49:05, 15.80s/it]
 60%|██████    | 969/1610 [5:31:34<2:50:36, 15.97s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.5166252129801514, 'learning_rate': 3.981366459627329e-07, 'completion_length': 166.75000762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.21981074661016464, 'kl': 0.017333984375, 'epoch': 3.01}

 60%|██████    | 969/1610 [5:31:34<2:50:36, 15.97s/it]
 60%|██████    | 970/1610 [5:31:48<2:44:24, 15.41s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.7912586337900276, 'learning_rate': 3.975155279503105e-07, 'completion_length': 149.0357208251953, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.18409644439816475, 'kl': 0.01617431640625, 'epoch': 3.01}

 60%|██████    | 970/1610 [5:31:48<2:44:24, 15.41s/it]
 60%|██████    | 971/1610 [5:32:04<2:43:43, 15.37s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.6813339258217455, 'learning_rate': 3.968944099378882e-07, 'completion_length': 151.71429443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1896214708685875, 'kl': 0.01171875, 'epoch': 3.02}

 60%|██████    | 971/1610 [5:32:04<2:43:43, 15.37s/it]
 60%|██████    | 972/1610 [5:32:20<2:46:22, 15.65s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.779358693867339, 'learning_rate': 3.9627329192546583e-07, 'completion_length': 161.3214340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.21981074661016464, 'kl': 0.01690673828125, 'epoch': 3.02}

 60%|██████    | 972/1610 [5:32:20<2:46:22, 15.65s/it]
 60%|██████    | 973/1610 [5:32:37<2:49:29, 15.96s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 0.4738511909542972, 'learning_rate': 3.9565217391304346e-07, 'completion_length': 158.42858123779297, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.013153076171875, 'epoch': 3.02}

 60%|██████    | 973/1610 [5:32:37<2:49:29, 15.96s/it]
 60%|██████    | 974/1610 [5:32:55<2:55:55, 16.60s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.80699797003875, 'learning_rate': 3.950310559006211e-07, 'completion_length': 174.08929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357470035553, 'kl': 0.01708984375, 'epoch': 3.02}

 60%|██████    | 974/1610 [5:32:55<2:55:55, 16.60s/it]
 61%|██████    | 975/1610 [5:33:10<2:50:03, 16.07s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 4.5479912470321775, 'learning_rate': 3.944099378881988e-07, 'completion_length': 145.3928680419922, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.23086076974868774, 'kl': 0.01251220703125, 'epoch': 3.03}

 61%|██████    | 975/1610 [5:33:10<2:50:03, 16.07s/it]
 61%|██████    | 976/1610 [5:33:27<2:54:54, 16.55s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.7526746132877622, 'learning_rate': 3.937888198757764e-07, 'completion_length': 163.16072463989258, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.01739501953125, 'epoch': 3.03}

 61%|██████    | 976/1610 [5:33:27<2:54:54, 16.55s/it]
 61%|██████    | 977/1610 [5:33:47<3:04:26, 17.48s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5570724966197337, 'learning_rate': 3.93167701863354e-07, 'completion_length': 167.21429443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.13527478277683258, 'kl': 0.0157470703125, 'epoch': 3.03}

 61%|██████    | 977/1610 [5:33:47<3:04:26, 17.48s/it]
 61%|██████    | 978/1610 [5:34:03<2:58:04, 16.91s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.7809770716301156, 'learning_rate': 3.925465838509316e-07, 'completion_length': 144.3928680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.02020263671875, 'epoch': 3.04}

 61%|██████    | 978/1610 [5:34:03<2:58:04, 16.91s/it]
 61%|██████    | 979/1610 [5:34:18<2:51:47, 16.34s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.9254579078890597, 'learning_rate': 3.9192546583850926e-07, 'completion_length': 148.30357360839844, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981073170900345, 'kl': 0.0142822265625, 'epoch': 3.04}

 61%|██████    | 979/1610 [5:34:18<2:51:47, 16.34s/it]
 61%|██████    | 980/1610 [5:34:32<2:44:24, 15.66s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 2.171866965291186, 'learning_rate': 3.9130434782608694e-07, 'completion_length': 133.7857208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.0135498046875, 'epoch': 3.04}

 61%|██████    | 980/1610 [5:34:32<2:44:24, 15.66s/it]
 61%|██████    | 981/1610 [5:34:45<2:35:37, 14.85s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 0.6650306498619888, 'learning_rate': 3.906832298136646e-07, 'completion_length': 141.64286041259766, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.0824786126613617, 'kl': 0.0189208984375, 'epoch': 3.05}

 61%|██████    | 981/1610 [5:34:45<2:35:37, 14.85s/it]
 61%|██████    | 982/1610 [5:35:02<2:43:03, 15.58s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.0669153903531368, 'learning_rate': 3.900621118012422e-07, 'completion_length': 163.42858123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357656300068, 'kl': 0.01580810546875, 'epoch': 3.05}

 61%|██████    | 982/1610 [5:35:02<2:43:03, 15.58s/it]
 61%|██████    | 983/1610 [5:35:17<2:41:58, 15.50s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.3347199019859488, 'learning_rate': 3.8944099378881984e-07, 'completion_length': 175.44644165039062, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.02227783203125, 'epoch': 3.05}

 61%|██████    | 983/1610 [5:35:17<2:41:58, 15.50s/it]
 61%|██████    | 984/1610 [5:35:33<2:41:23, 15.47s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.6106959747025917, 'learning_rate': 3.8881987577639753e-07, 'completion_length': 156.4107208251953, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.18409645557403564, 'kl': 0.015228271484375, 'epoch': 3.06}

 61%|██████    | 984/1610 [5:35:33<2:41:23, 15.47s/it]
 61%|██████    | 985/1610 [5:35:49<2:44:43, 15.81s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.0648177103456467, 'learning_rate': 3.8819875776397516e-07, 'completion_length': 174.23214721679688, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2610500380396843, 'kl': 0.02008056640625, 'epoch': 3.06}

 61%|██████    | 985/1610 [5:35:49<2:44:43, 15.81s/it]
 61%|██████    | 986/1610 [5:36:03<2:37:36, 15.16s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.3411620465984764, 'learning_rate': 3.875776397515528e-07, 'completion_length': 144.3214340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.21981073170900345, 'kl': 0.02001953125, 'epoch': 3.06}

 61%|██████    | 986/1610 [5:36:03<2:37:36, 15.16s/it]
 61%|██████▏   | 987/1610 [5:36:17<2:33:58, 14.83s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 3.2724154178449454, 'learning_rate': 3.869565217391304e-07, 'completion_length': 127.16072082519531, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.19514648616313934, 'kl': 0.0179443359375, 'epoch': 3.07}

 61%|██████▏   | 987/1610 [5:36:17<2:33:58, 14.83s/it]
 61%|██████▏   | 988/1610 [5:36:33<2:37:12, 15.16s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 0.8292178327825951, 'learning_rate': 3.8633540372670806e-07, 'completion_length': 166.6607208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086076974868774, 'kl': 0.01776123046875, 'epoch': 3.07}

 61%|██████▏   | 988/1610 [5:36:33<2:37:12, 15.16s/it]
 61%|██████▏   | 989/1610 [5:36:45<2:27:05, 14.21s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.2969109529754161, 'learning_rate': 3.857142857142857e-07, 'completion_length': 120.78571701049805, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2967643439769745, 'kl': 0.01348876953125, 'epoch': 3.07}

 61%|██████▏   | 989/1610 [5:36:45<2:27:05, 14.21s/it]
 61%|██████▏   | 990/1610 [5:37:02<2:36:29, 15.14s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 1.2262441275762321, 'learning_rate': 3.850931677018633e-07, 'completion_length': 155.33929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657506078481674, 'kl': 0.02020263671875, 'epoch': 3.07}

 61%|██████▏   | 990/1610 [5:37:02<2:36:29, 15.14s/it]
 62%|██████▏   | 991/1610 [5:37:17<2:34:13, 14.95s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 2.3460555522991355, 'learning_rate': 3.8447204968944095e-07, 'completion_length': 155.14286041259766, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.19514650478959084, 'kl': 0.017913818359375, 'epoch': 3.08}

 62%|██████▏   | 991/1610 [5:37:17<2:34:13, 14.95s/it]
 62%|██████▏   | 992/1610 [5:37:32<2:36:27, 15.19s/it]
                                                      
{'loss': 0.0005, 'grad_norm': 1.627732603853306, 'learning_rate': 3.838509316770186e-07, 'completion_length': 147.39286422729492, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214671432972, 'kl': 0.013153076171875, 'epoch': 3.08}

 62%|██████▏   | 992/1610 [5:37:32<2:36:27, 15.19s/it]
 62%|██████▏   | 993/1610 [5:37:49<2:39:53, 15.55s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.5513723696003046, 'learning_rate': 3.8322981366459627e-07, 'completion_length': 164.17858123779297, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.11266787722706795, 'kl': 0.01409912109375, 'epoch': 3.08}

 62%|██████▏   | 993/1610 [5:37:49<2:39:53, 15.55s/it]
 62%|██████▏   | 994/1610 [5:38:03<2:36:44, 15.27s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.2216404005058945, 'learning_rate': 3.826086956521739e-07, 'completion_length': 148.82143783569336, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.18409645557403564, 'kl': 0.01812744140625, 'epoch': 3.09}

 62%|██████▏   | 994/1610 [5:38:03<2:36:44, 15.27s/it]
 62%|██████▏   | 995/1610 [5:38:17<2:32:26, 14.87s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 1.670169410392187, 'learning_rate': 3.8198757763975154e-07, 'completion_length': 147.42857360839844, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.0142822265625, 'epoch': 3.09}

 62%|██████▏   | 995/1610 [5:38:17<2:32:26, 14.87s/it]
 62%|██████▏   | 996/1610 [5:38:31<2:27:27, 14.41s/it]
                                                      
{'loss': 0.0006, 'grad_norm': 0.7017445900590005, 'learning_rate': 3.8136645962732917e-07, 'completion_length': 121.3214340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.01531982421875, 'epoch': 3.09}

 62%|██████▏   | 996/1610 [5:38:31<2:27:27, 14.41s/it]
 62%|██████▏   | 997/1610 [5:38:45<2:27:38, 14.45s/it]
                                                      
{'loss': 0.0008, 'grad_norm': 2.4855038265249747, 'learning_rate': 3.807453416149068e-07, 'completion_length': 149.8928680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.18409645557403564, 'kl': 0.02001953125, 'epoch': 3.1}

 62%|██████▏   | 997/1610 [5:38:45<2:27:38, 14.45s/it]
 62%|██████▏   | 998/1610 [5:38:59<2:25:04, 14.22s/it]
                                                      
{'loss': 0.0009, 'grad_norm': 1.163276139265777, 'learning_rate': 3.801242236024845e-07, 'completion_length': 149.55358123779297, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.0238037109375, 'epoch': 3.1}

 62%|██████▏   | 998/1610 [5:38:59<2:25:04, 14.22s/it]
 62%|██████▏   | 999/1610 [5:39:13<2:24:35, 14.20s/it]
                                                      
{'loss': 0.0007, 'grad_norm': 1.4650315637971256, 'learning_rate': 3.795031055900621e-07, 'completion_length': 150.5178680419922, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.07695358991622925, 'kl': 0.0174560546875, 'epoch': 3.1}

 62%|██████▏   | 999/1610 [5:39:13<2:24:35, 14.20s/it]
 62%|██████▏   | 1000/1610 [5:39:32<2:37:22, 15.48s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1138956127585389, 'learning_rate': 3.7888198757763975e-07, 'completion_length': 178.73214721679688, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7321429252624512, 'reward_std': 0.22229483723640442, 'kl': 0.01995849609375, 'epoch': 3.11}

 62%|██████▏   | 1000/1610 [5:39:32<2:37:22, 15.48s/it]
 62%|██████▏   | 1001/1610 [5:43:24<13:37:57, 80.59s/it]
                                                        
{'loss': 0.0005, 'grad_norm': 1.669586927906383, 'learning_rate': 3.7826086956521733e-07, 'completion_length': 173.89286041259766, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695358991622925, 'kl': 0.013458251953125, 'epoch': 3.11}

 62%|██████▏   | 1001/1610 [5:43:24<13:37:57, 80.59s/it]
 62%|██████▏   | 1002/1610 [5:43:40<10:19:16, 61.11s/it]
                                                        
{'loss': 0.0006, 'grad_norm': 0.528407437009876, 'learning_rate': 3.77639751552795e-07, 'completion_length': 159.7857208251953, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.0824786126613617, 'kl': 0.01605224609375, 'epoch': 3.11}

 62%|██████▏   | 1002/1610 [5:43:40<10:19:16, 61.11s/it]
 62%|██████▏   | 1003/1610 [5:43:58<8:07:56, 48.23s/it] 
                                                       
{'loss': 0.0006, 'grad_norm': 8.860721312069147, 'learning_rate': 3.7701863354037265e-07, 'completion_length': 167.05358123779297, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.8035715222358704, 'reward_std': 0.2721000909805298, 'kl': 0.01470947265625, 'epoch': 3.11}

 62%|██████▏   | 1003/1610 [5:43:58<8:07:56, 48.23s/it]
 62%|██████▏   | 1004/1610 [5:44:12<6:24:41, 38.09s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 2.6549584072737855, 'learning_rate': 3.763975155279503e-07, 'completion_length': 155.80358123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2253357656300068, 'kl': 0.022216796875, 'epoch': 3.12}

 62%|██████▏   | 1004/1610 [5:44:12<6:24:41, 38.09s/it]
 62%|██████▏   | 1005/1610 [5:44:30<5:23:27, 32.08s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.205876121656299, 'learning_rate': 3.757763975155279e-07, 'completion_length': 148.3571548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.2253357470035553, 'kl': 0.01959228515625, 'epoch': 3.12}

 62%|██████▏   | 1005/1610 [5:44:30<5:23:27, 32.08s/it]
 62%|██████▏   | 1006/1610 [5:44:46<4:34:32, 27.27s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2369686693167659, 'learning_rate': 3.7515527950310555e-07, 'completion_length': 162.80358123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.01904296875, 'epoch': 3.12}

 62%|██████▏   | 1006/1610 [5:44:46<4:34:32, 27.27s/it]
 63%|██████▎   | 1007/1610 [5:45:01<3:56:54, 23.57s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.490486150171732, 'learning_rate': 3.7453416149068323e-07, 'completion_length': 140.46428680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.26657506823539734, 'kl': 0.0184326171875, 'epoch': 3.13}

 63%|██████▎   | 1007/1610 [5:45:01<3:56:54, 23.57s/it]
 63%|██████▎   | 1008/1610 [5:45:15<3:27:57, 20.73s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.7752349242567605, 'learning_rate': 3.7391304347826087e-07, 'completion_length': 151.75000762939453, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981072798371315, 'kl': 0.01898193359375, 'epoch': 3.13}

 63%|██████▎   | 1008/1610 [5:45:15<3:27:57, 20.73s/it]
 63%|██████▎   | 1009/1610 [5:45:30<3:08:12, 18.79s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0276370935254473, 'learning_rate': 3.732919254658385e-07, 'completion_length': 137.41071701049805, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981074661016464, 'kl': 0.01873779296875, 'epoch': 3.13}

 63%|██████▎   | 1009/1610 [5:45:30<3:08:12, 18.79s/it]
 63%|██████▎   | 1010/1610 [5:45:45<2:56:27, 17.65s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9912167642058306, 'learning_rate': 3.7267080745341613e-07, 'completion_length': 133.05357360839844, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428619325161, 'kl': 0.01361083984375, 'epoch': 3.14}

 63%|██████▎   | 1010/1610 [5:45:45<2:56:27, 17.65s/it]
 63%|██████▎   | 1011/1610 [5:45:58<2:43:40, 16.40s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.42026175564221074, 'learning_rate': 3.720496894409938e-07, 'completion_length': 147.50000762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.04123930633068085, 'kl': 0.0111083984375, 'epoch': 3.14}

 63%|██████▎   | 1011/1610 [5:45:58<2:43:40, 16.40s/it]
 63%|██████▎   | 1012/1610 [5:46:15<2:43:45, 16.43s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6842192249081783, 'learning_rate': 3.7142857142857145e-07, 'completion_length': 170.69644165039062, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.018280029296875, 'epoch': 3.14}

 63%|██████▎   | 1012/1610 [5:46:15<2:43:45, 16.43s/it]
 63%|██████▎   | 1013/1610 [5:46:30<2:39:38, 16.04s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6851129605201592, 'learning_rate': 3.7080745341614903e-07, 'completion_length': 160.8214340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.1428571492433548, 'kl': 0.01971435546875, 'epoch': 3.15}

 63%|██████▎   | 1013/1610 [5:46:30<2:39:38, 16.04s/it]
 63%|██████▎   | 1014/1610 [5:46:43<2:31:53, 15.29s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 2.8677718273264157, 'learning_rate': 3.7018633540372666e-07, 'completion_length': 136.4107208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357656300068, 'kl': 0.01959228515625, 'epoch': 3.15}

 63%|██████▎   | 1014/1610 [5:46:43<2:31:53, 15.29s/it]
 63%|██████▎   | 1015/1610 [5:47:01<2:37:12, 15.85s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.314801342536085, 'learning_rate': 3.695652173913043e-07, 'completion_length': 190.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3324786275625229, 'kl': 0.01849365234375, 'epoch': 3.15}

 63%|██████▎   | 1015/1610 [5:47:01<2:37:12, 15.85s/it]
 63%|██████▎   | 1016/1610 [5:47:18<2:41:58, 16.36s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.5725786795575454, 'learning_rate': 3.68944099378882e-07, 'completion_length': 176.33929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.12371791899204254, 'kl': 0.02264404296875, 'epoch': 3.16}

 63%|██████▎   | 1016/1610 [5:47:18<2:41:58, 16.36s/it]
 63%|██████▎   | 1017/1610 [5:47:33<2:38:19, 16.02s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7238227395901499, 'learning_rate': 3.683229813664596e-07, 'completion_length': 144.7857208251953, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071798324585, 'kl': 0.0159912109375, 'epoch': 3.16}

 63%|██████▎   | 1017/1610 [5:47:33<2:38:19, 16.02s/it]
 63%|██████▎   | 1018/1610 [5:47:49<2:37:08, 15.93s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0581466754849893, 'learning_rate': 3.6770186335403724e-07, 'completion_length': 163.71429443359375, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.25552503019571304, 'kl': 0.01458740234375, 'epoch': 3.16}

 63%|██████▎   | 1018/1610 [5:47:49<2:37:08, 15.93s/it]
 63%|██████▎   | 1019/1610 [5:48:06<2:40:14, 16.27s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0976469181297728, 'learning_rate': 3.670807453416149e-07, 'completion_length': 175.42858123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981074661016464, 'kl': 0.01739501953125, 'epoch': 3.16}

 63%|██████▎   | 1019/1610 [5:48:06<2:40:14, 16.27s/it]
 63%|██████▎   | 1020/1610 [5:48:25<2:49:05, 17.20s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.4324814311857842, 'learning_rate': 3.6645962732919256e-07, 'completion_length': 195.7678680419922, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1181928999722004, 'kl': 0.01336669921875, 'epoch': 3.17}

 63%|██████▎   | 1020/1610 [5:48:25<2:49:05, 17.20s/it]
 63%|██████▎   | 1021/1610 [5:48:41<2:45:00, 16.81s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.740942737444905, 'learning_rate': 3.658385093167702e-07, 'completion_length': 157.69644165039062, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216826319695, 'kl': 0.0184326171875, 'epoch': 3.17}

 63%|██████▎   | 1021/1610 [5:48:41<2:45:00, 16.81s/it]
 63%|██████▎   | 1022/1610 [5:49:00<2:49:41, 17.32s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.884601955856243, 'learning_rate': 3.6521739130434783e-07, 'completion_length': 144.3214340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.013763427734375, 'epoch': 3.17}

 63%|██████▎   | 1022/1610 [5:49:00<2:49:41, 17.32s/it]
 64%|██████▎   | 1023/1610 [5:49:19<2:54:56, 17.88s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5397615395355096, 'learning_rate': 3.6459627329192546e-07, 'completion_length': 180.8214340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.2142857238650322, 'kl': 0.018310546875, 'epoch': 3.18}

 64%|██████▎   | 1023/1610 [5:49:19<2:54:56, 17.88s/it]
 64%|██████▎   | 1024/1610 [5:49:32<2:41:21, 16.52s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0825621672884762, 'learning_rate': 3.6397515527950304e-07, 'completion_length': 150.1071548461914, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23086076974868774, 'kl': 0.02044677734375, 'epoch': 3.18}

 64%|██████▎   | 1024/1610 [5:49:32<2:41:21, 16.52s/it]
 64%|██████▎   | 1025/1610 [5:49:48<2:39:43, 16.38s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.114929547687851, 'learning_rate': 3.633540372670807e-07, 'completion_length': 185.58929443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1428571529686451, 'kl': 0.017913818359375, 'epoch': 3.18}

 64%|██████▎   | 1025/1610 [5:49:48<2:39:43, 16.38s/it]
 64%|██████▎   | 1026/1610 [5:50:02<2:30:42, 15.48s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9904192622911638, 'learning_rate': 3.6273291925465836e-07, 'completion_length': 135.10714721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.15943220257759094, 'kl': 0.01580810546875, 'epoch': 3.19}

 64%|██████▎   | 1026/1610 [5:50:02<2:30:42, 15.48s/it]
 64%|██████▍   | 1027/1610 [5:50:15<2:23:33, 14.77s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.0133940082002444, 'learning_rate': 3.62111801242236e-07, 'completion_length': 135.2857208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.02142333984375, 'epoch': 3.19}

 64%|██████▍   | 1027/1610 [5:50:15<2:23:33, 14.77s/it]
 64%|██████▍   | 1028/1610 [5:50:29<2:22:34, 14.70s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9105707644067788, 'learning_rate': 3.614906832298136e-07, 'completion_length': 143.64286041259766, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714402794838, 'kl': 0.01629638671875, 'epoch': 3.19}

 64%|██████▍   | 1028/1610 [5:50:29<2:22:34, 14.70s/it]
 64%|██████▍   | 1029/1610 [5:50:43<2:18:36, 14.31s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8011994370372003, 'learning_rate': 3.608695652173913e-07, 'completion_length': 125.00000762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.0147705078125, 'epoch': 3.2}

 64%|██████▍   | 1029/1610 [5:50:43<2:18:36, 14.31s/it]
 64%|██████▍   | 1030/1610 [5:50:56<2:15:17, 14.00s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.994698166917586, 'learning_rate': 3.6024844720496894e-07, 'completion_length': 133.2857208251953, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.32695360481739044, 'kl': 0.0152587890625, 'epoch': 3.2}

 64%|██████▍   | 1030/1610 [5:50:56<2:15:17, 14.00s/it]
 64%|██████▍   | 1031/1610 [5:51:13<2:23:02, 14.82s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.787279764321872, 'learning_rate': 3.596273291925466e-07, 'completion_length': 158.85714721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.20117833837866783, 'kl': 0.016265869140625, 'epoch': 3.2}

 64%|██████▍   | 1031/1610 [5:51:13<2:23:02, 14.82s/it]
 64%|██████▍   | 1032/1610 [5:51:29<2:26:19, 15.19s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7328056373126871, 'learning_rate': 3.590062111801242e-07, 'completion_length': 169.9464340209961, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266787722706795, 'kl': 0.01800537109375, 'epoch': 3.2}

 64%|██████▍   | 1032/1610 [5:51:29<2:26:19, 15.19s/it]
 64%|██████▍   | 1033/1610 [5:51:44<2:24:41, 15.05s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.550812386804144, 'learning_rate': 3.5838509316770184e-07, 'completion_length': 149.21429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357619047165, 'kl': 0.02099609375, 'epoch': 3.21}

 64%|██████▍   | 1033/1610 [5:51:44<2:24:41, 15.05s/it]
 64%|██████▍   | 1034/1610 [5:51:59<2:26:14, 15.23s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1171889118409752, 'learning_rate': 3.577639751552795e-07, 'completion_length': 149.1964340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1785714402794838, 'kl': 0.020751953125, 'epoch': 3.21}

 64%|██████▍   | 1034/1610 [5:51:59<2:26:14, 15.23s/it]
 64%|██████▍   | 1035/1610 [5:52:16<2:29:57, 15.65s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.4904613892957657, 'learning_rate': 3.5714285714285716e-07, 'completion_length': 164.46429443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428619325161, 'kl': 0.02215576171875, 'epoch': 3.21}

 64%|██████▍   | 1035/1610 [5:52:16<2:29:57, 15.65s/it]
 64%|██████▍   | 1036/1610 [5:52:33<2:33:25, 16.04s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.142300233347083, 'learning_rate': 3.5652173913043474e-07, 'completion_length': 154.30357360839844, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.30228933691978455, 'kl': 0.01953125, 'epoch': 3.22}

 64%|██████▍   | 1036/1610 [5:52:33<2:33:25, 16.04s/it]
 64%|██████▍   | 1037/1610 [5:52:48<2:29:43, 15.68s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0379739346194579, 'learning_rate': 3.5590062111801237e-07, 'completion_length': 149.00000762939453, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428656578064, 'kl': 0.01470947265625, 'epoch': 3.22}

 64%|██████▍   | 1037/1610 [5:52:48<2:29:43, 15.68s/it]
 64%|██████▍   | 1038/1610 [5:53:01<2:21:20, 14.83s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.7001906780431948, 'learning_rate': 3.5527950310559005e-07, 'completion_length': 132.05358123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981073170900345, 'kl': 0.0206298828125, 'epoch': 3.22}

 64%|██████▍   | 1038/1610 [5:53:01<2:21:20, 14.83s/it]
 65%|██████▍   | 1039/1610 [5:53:15<2:20:39, 14.78s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8348322501940616, 'learning_rate': 3.546583850931677e-07, 'completion_length': 145.60714721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.0164794921875, 'epoch': 3.23}

 65%|██████▍   | 1039/1610 [5:53:15<2:20:39, 14.78s/it]
 65%|██████▍   | 1040/1610 [5:53:30<2:19:52, 14.72s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.10428674302864938, 'learning_rate': 3.540372670807453e-07, 'completion_length': 164.37500762939453, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0, 'kl': 0.014068603515625, 'epoch': 3.23}

 65%|██████▍   | 1040/1610 [5:53:30<2:19:52, 14.72s/it]
 65%|██████▍   | 1041/1610 [5:53:44<2:19:19, 14.69s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2091480126967473, 'learning_rate': 3.5341614906832295e-07, 'completion_length': 158.7857208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2253357619047165, 'kl': 0.02117919921875, 'epoch': 3.23}

 65%|██████▍   | 1041/1610 [5:53:44<2:19:19, 14.69s/it]
 65%|██████▍   | 1042/1610 [5:54:03<2:30:41, 15.92s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6814867548251778, 'learning_rate': 3.527950310559006e-07, 'completion_length': 183.4821548461914, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266788095235825, 'kl': 0.019012451171875, 'epoch': 3.24}

 65%|██████▍   | 1042/1610 [5:54:03<2:30:41, 15.92s/it]
 65%|██████▍   | 1043/1610 [5:54:20<2:33:11, 16.21s/it]
                                                       
{'loss': 0.001, 'grad_norm': 0.8392095581599694, 'learning_rate': 3.5217391304347827e-07, 'completion_length': 188.42858123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1896214671432972, 'kl': 0.02398681640625, 'epoch': 3.24}

 65%|██████▍   | 1043/1610 [5:54:20<2:33:11, 16.21s/it]
 65%|██████▍   | 1044/1610 [5:54:33<2:23:47, 15.24s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.7583233789963778, 'learning_rate': 3.515527950310559e-07, 'completion_length': 146.9107208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.0213623046875, 'epoch': 3.24}

 65%|██████▍   | 1044/1610 [5:54:33<2:23:47, 15.24s/it]
 65%|██████▍   | 1045/1610 [5:54:50<2:27:44, 15.69s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0075972978443721, 'learning_rate': 3.5093167701863354e-07, 'completion_length': 147.33929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1539071872830391, 'kl': 0.015625, 'epoch': 3.25}

 65%|██████▍   | 1045/1610 [5:54:50<2:27:44, 15.69s/it]
 65%|██████▍   | 1046/1610 [5:55:07<2:31:09, 16.08s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.4582778745758267, 'learning_rate': 3.5031055900621117e-07, 'completion_length': 180.25000762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.19514648616313934, 'kl': 0.02197265625, 'epoch': 3.25}

 65%|██████▍   | 1046/1610 [5:55:07<2:31:09, 16.08s/it]
 65%|██████▌   | 1047/1610 [5:55:20<2:22:59, 15.24s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9032641017554165, 'learning_rate': 3.4968944099378885e-07, 'completion_length': 131.96429443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.27762509882450104, 'kl': 0.012847900390625, 'epoch': 3.25}

 65%|██████▌   | 1047/1610 [5:55:20<2:22:59, 15.24s/it]
 65%|██████▌   | 1048/1610 [5:55:34<2:19:50, 14.93s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6007604451345309, 'learning_rate': 3.4906832298136643e-07, 'completion_length': 153.64286041259766, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.1428571492433548, 'kl': 0.015380859375, 'epoch': 3.25}

 65%|██████▌   | 1048/1610 [5:55:34<2:19:50, 14.93s/it]
 65%|██████▌   | 1049/1610 [5:55:52<2:28:38, 15.90s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5465788856981929, 'learning_rate': 3.4844720496894407e-07, 'completion_length': 181.6428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.07695359364151955, 'kl': 0.01751708984375, 'epoch': 3.26}

 65%|██████▌   | 1049/1610 [5:55:52<2:28:38, 15.90s/it]
 65%|██████▌   | 1050/1610 [5:56:08<2:27:20, 15.79s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3137899219579434, 'learning_rate': 3.478260869565217e-07, 'completion_length': 142.07143783569336, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2142857238650322, 'kl': 0.0172119140625, 'epoch': 3.26}

 65%|██████▌   | 1050/1610 [5:56:08<2:27:20, 15.79s/it]
 65%|██████▌   | 1051/1610 [5:56:22<2:22:47, 15.33s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 4.300883513378532, 'learning_rate': 3.4720496894409933e-07, 'completion_length': 141.2857208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981074661016464, 'kl': 0.0181884765625, 'epoch': 3.26}

 65%|██████▌   | 1051/1610 [5:56:22<2:22:47, 15.33s/it]
 65%|██████▌   | 1052/1610 [5:56:37<2:19:55, 15.04s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7339629374364312, 'learning_rate': 3.46583850931677e-07, 'completion_length': 133.78571701049805, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1896214708685875, 'kl': 0.01861572265625, 'epoch': 3.27}

 65%|██████▌   | 1052/1610 [5:56:37<2:19:55, 15.04s/it]
 65%|██████▌   | 1053/1610 [5:56:49<2:11:21, 14.15s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9868433461093411, 'learning_rate': 3.4596273291925465e-07, 'completion_length': 133.46429061889648, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.19514648616313934, 'kl': 0.01629638671875, 'epoch': 3.27}

 65%|██████▌   | 1053/1610 [5:56:49<2:11:21, 14.15s/it]
 65%|██████▌   | 1054/1610 [5:57:05<2:17:18, 14.82s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.1214763214819643, 'learning_rate': 3.453416149068323e-07, 'completion_length': 152.5714340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1428571492433548, 'kl': 0.012939453125, 'epoch': 3.27}

 65%|██████▌   | 1054/1610 [5:57:05<2:17:18, 14.82s/it]
 66%|██████▌   | 1055/1610 [5:57:19<2:15:54, 14.69s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9169353214240702, 'learning_rate': 3.447204968944099e-07, 'completion_length': 146.33928680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981072798371315, 'kl': 0.015960693359375, 'epoch': 3.28}

 66%|██████▌   | 1055/1610 [5:57:19<2:15:54, 14.69s/it]
 66%|██████▌   | 1056/1610 [5:57:34<2:16:37, 14.80s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.4934250748409339, 'learning_rate': 3.440993788819876e-07, 'completion_length': 128.3214340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.07695358991622925, 'kl': 0.015350341796875, 'epoch': 3.28}

 66%|██████▌   | 1056/1610 [5:57:34<2:16:37, 14.80s/it]
 66%|██████▌   | 1057/1610 [5:57:51<2:22:21, 15.45s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.243765882764322, 'learning_rate': 3.4347826086956523e-07, 'completion_length': 182.08929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714402794838, 'kl': 0.01800537109375, 'epoch': 3.28}

 66%|██████▌   | 1057/1610 [5:57:51<2:22:21, 15.45s/it]
 66%|██████▌   | 1058/1610 [5:58:06<2:20:36, 15.28s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 3.8675976767249947, 'learning_rate': 3.4285714285714286e-07, 'completion_length': 161.07144165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071835577488, 'kl': 0.013092041015625, 'epoch': 3.29}

 66%|██████▌   | 1058/1610 [5:58:09<2:20:36, 15.28s/it]
 66%|██████▌   | 1059/1610 [5:58:20<2:17:00, 14.92s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2637277477043771, 'learning_rate': 3.422360248447205e-07, 'completion_length': 125.64286041259766, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1896214708685875, 'kl': 0.017822265625, 'epoch': 3.29}

 66%|██████▌   | 1059/1610 [5:58:20<2:17:00, 14.92s/it]
 66%|██████▌   | 1060/1610 [5:58:35<2:16:52, 14.93s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6605362630129894, 'learning_rate': 3.416149068322981e-07, 'completion_length': 154.76786041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.018951416015625, 'epoch': 3.29}

 66%|██████▌   | 1060/1610 [5:58:35<2:16:52, 14.93s/it]
 66%|██████▌   | 1061/1610 [5:58:50<2:16:22, 14.90s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.8522630740576267, 'learning_rate': 3.4099378881987576e-07, 'completion_length': 141.89286422729492, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.21981073170900345, 'kl': 0.01611328125, 'epoch': 3.3}

 66%|██████▌   | 1061/1610 [5:58:50<2:16:22, 14.90s/it]
 66%|██████▌   | 1062/1610 [5:59:07<2:20:59, 15.44s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7236258260899159, 'learning_rate': 3.403726708074534e-07, 'completion_length': 150.53572463989258, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.17651409283280373, 'kl': 0.016357421875, 'epoch': 3.3}

 66%|██████▌   | 1062/1610 [5:59:07<2:20:59, 15.44s/it]
 66%|██████▌   | 1063/1610 [5:59:23<2:23:35, 15.75s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2927175869413368, 'learning_rate': 3.3975155279503103e-07, 'completion_length': 174.7857208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.30228933691978455, 'kl': 0.017120361328125, 'epoch': 3.3}

 66%|██████▌   | 1063/1610 [5:59:23<2:23:35, 15.75s/it]
 66%|██████▌   | 1064/1610 [5:59:39<2:24:20, 15.86s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0744890441920465, 'learning_rate': 3.3913043478260866e-07, 'completion_length': 150.2857208251953, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.21981074661016464, 'kl': 0.015106201171875, 'epoch': 3.3}

 66%|██████▌   | 1064/1610 [5:59:40<2:24:20, 15.86s/it]
 66%|██████▌   | 1065/1610 [5:59:56<2:25:47, 16.05s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.2234859535819296, 'learning_rate': 3.385093167701863e-07, 'completion_length': 154.4464340209961, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.2500000149011612, 'kl': 0.02313232421875, 'epoch': 3.31}

 66%|██████▌   | 1065/1610 [5:59:56<2:25:47, 16.05s/it]
 66%|██████▌   | 1066/1610 [6:00:10<2:20:34, 15.51s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.7686651547569652, 'learning_rate': 3.37888198757764e-07, 'completion_length': 152.87500762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214671432972, 'kl': 0.02093505859375, 'epoch': 3.31}

 66%|██████▌   | 1066/1610 [6:00:10<2:20:34, 15.51s/it]
 66%|██████▋   | 1067/1610 [6:00:24<2:16:56, 15.13s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.6802851881977958, 'learning_rate': 3.372670807453416e-07, 'completion_length': 147.21428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1896214708685875, 'kl': 0.01837158203125, 'epoch': 3.31}

 66%|██████▋   | 1067/1610 [6:00:24<2:16:56, 15.13s/it]
 66%|██████▋   | 1068/1610 [6:00:41<2:20:39, 15.57s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.4715574894803282, 'learning_rate': 3.3664596273291924e-07, 'completion_length': 172.3214340209961, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.01397705078125, 'epoch': 3.32}

 66%|██████▋   | 1068/1610 [6:00:41<2:20:39, 15.57s/it]
 66%|██████▋   | 1069/1610 [6:00:57<2:21:42, 15.72s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6085057447860799, 'learning_rate': 3.360248447204969e-07, 'completion_length': 176.33929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266788095235825, 'kl': 0.014739990234375, 'epoch': 3.32}

 66%|██████▋   | 1069/1610 [6:00:57<2:21:42, 15.72s/it]
 66%|██████▋   | 1070/1610 [6:01:10<2:12:55, 14.77s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3106348307757072, 'learning_rate': 3.3540372670807456e-07, 'completion_length': 126.39286041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.25552503019571304, 'kl': 0.0186767578125, 'epoch': 3.32}

 66%|██████▋   | 1070/1610 [6:01:10<2:12:55, 14.77s/it]
 67%|██████▋   | 1071/1610 [6:01:24<2:11:19, 14.62s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9891071900491476, 'learning_rate': 3.347826086956522e-07, 'completion_length': 147.33929443359375, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.1428571529686451, 'kl': 0.018798828125, 'epoch': 3.33}

 67%|██████▋   | 1071/1610 [6:01:24<2:11:19, 14.62s/it]
 67%|██████▋   | 1072/1610 [6:01:39<2:13:14, 14.86s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9441510150054085, 'learning_rate': 3.3416149068322977e-07, 'completion_length': 160.53572845458984, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.013458251953125, 'epoch': 3.33}

 67%|██████▋   | 1072/1610 [6:01:39<2:13:14, 14.86s/it]
 67%|██████▋   | 1073/1610 [6:01:55<2:14:08, 14.99s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0241659923855098, 'learning_rate': 3.335403726708074e-07, 'completion_length': 162.48214721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.015106201171875, 'epoch': 3.33}

 67%|██████▋   | 1073/1610 [6:01:55<2:14:08, 14.99s/it]
 67%|██████▋   | 1074/1610 [6:02:09<2:13:03, 14.89s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.7906196309213378, 'learning_rate': 3.3291925465838504e-07, 'completion_length': 163.69644165039062, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.01519775390625, 'epoch': 3.34}

 67%|██████▋   | 1074/1610 [6:02:09<2:13:03, 14.89s/it]
 67%|██████▋   | 1075/1610 [6:02:24<2:11:57, 14.80s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6305608287538326, 'learning_rate': 3.322981366459627e-07, 'completion_length': 154.80357360839844, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071835577488, 'kl': 0.01397705078125, 'epoch': 3.34}

 67%|██████▋   | 1075/1610 [6:02:24<2:11:57, 14.80s/it]
 67%|██████▋   | 1076/1610 [6:02:38<2:08:33, 14.44s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.39337249701943816, 'learning_rate': 3.3167701863354036e-07, 'completion_length': 133.80357360839844, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.016876220703125, 'epoch': 3.34}

 67%|██████▋   | 1076/1610 [6:02:38<2:08:33, 14.44s/it]
 67%|██████▋   | 1077/1610 [6:02:53<2:11:26, 14.80s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6859270975939458, 'learning_rate': 3.31055900621118e-07, 'completion_length': 159.33929443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266788095235825, 'kl': 0.01690673828125, 'epoch': 3.34}

 67%|██████▋   | 1077/1610 [6:02:53<2:11:26, 14.80s/it]
 67%|██████▋   | 1078/1610 [6:03:06<2:05:22, 14.14s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.7540596510010213, 'learning_rate': 3.304347826086956e-07, 'completion_length': 134.85714721679688, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25552501529455185, 'kl': 0.01458740234375, 'epoch': 3.35}

 67%|██████▋   | 1078/1610 [6:03:06<2:05:22, 14.14s/it]
 67%|██████▋   | 1079/1610 [6:03:23<2:12:48, 15.01s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8748783565781679, 'learning_rate': 3.298136645962733e-07, 'completion_length': 169.10714721679688, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216826319695, 'kl': 0.0185546875, 'epoch': 3.35}

 67%|██████▋   | 1079/1610 [6:03:23<2:12:48, 15.01s/it]
 67%|██████▋   | 1080/1610 [6:03:36<2:08:51, 14.59s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6586415374769723, 'learning_rate': 3.2919254658385094e-07, 'completion_length': 146.14286041259766, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428656578064, 'kl': 0.015594482421875, 'epoch': 3.35}

 67%|██████▋   | 1080/1610 [6:03:36<2:08:51, 14.59s/it]
 67%|██████▋   | 1081/1610 [6:03:52<2:12:06, 14.98s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.095249970492347, 'learning_rate': 3.2857142857142857e-07, 'completion_length': 155.71429443359375, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.21981072798371315, 'kl': 0.02294921875, 'epoch': 3.36}

 67%|██████▋   | 1081/1610 [6:03:52<2:12:06, 14.98s/it]
 67%|██████▋   | 1082/1610 [6:04:09<2:15:55, 15.45s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7257758603254936, 'learning_rate': 3.279503105590062e-07, 'completion_length': 152.78571701049805, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.2253357619047165, 'kl': 0.01751708984375, 'epoch': 3.36}

 67%|██████▋   | 1082/1610 [6:04:09<2:15:55, 15.45s/it]
 67%|██████▋   | 1083/1610 [6:04:23<2:11:48, 15.01s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8812900321971548, 'learning_rate': 3.273291925465838e-07, 'completion_length': 139.5357208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.11266787722706795, 'kl': 0.01373291015625, 'epoch': 3.36}

 67%|██████▋   | 1083/1610 [6:04:23<2:11:48, 15.01s/it]
 67%|██████▋   | 1084/1610 [6:04:38<2:13:13, 15.20s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6833054061889978, 'learning_rate': 3.2670807453416147e-07, 'completion_length': 145.87500762939453, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.01727294921875, 'epoch': 3.37}

 67%|██████▋   | 1084/1610 [6:04:38<2:13:13, 15.20s/it]
 67%|██████▋   | 1085/1610 [6:04:52<2:08:15, 14.66s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.26972319989235133, 'learning_rate': 3.260869565217391e-07, 'completion_length': 144.4464340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.0357142873108387, 'kl': 0.014068603515625, 'epoch': 3.37}

 67%|██████▋   | 1085/1610 [6:04:52<2:08:15, 14.66s/it]
 67%|██████▋   | 1086/1610 [6:05:06<2:07:51, 14.64s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6925504860343065, 'learning_rate': 3.2546583850931673e-07, 'completion_length': 147.7857208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2006715089082718, 'kl': 0.0130615234375, 'epoch': 3.37}

 67%|██████▋   | 1086/1610 [6:05:06<2:07:51, 14.64s/it]
 68%|██████▊   | 1087/1610 [6:05:22<2:11:04, 15.04s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.4760304370924294, 'learning_rate': 3.2484472049689437e-07, 'completion_length': 153.00000762939453, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.0714285746216774, 'kl': 0.0102691650390625, 'epoch': 3.38}

 68%|██████▊   | 1087/1610 [6:05:22<2:11:04, 15.04s/it]
 68%|██████▊   | 1088/1610 [6:05:37<2:08:24, 14.76s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2709181506240266, 'learning_rate': 3.2422360248447205e-07, 'completion_length': 148.6071548461914, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2610500380396843, 'kl': 0.02020263671875, 'epoch': 3.38}

 68%|██████▊   | 1088/1610 [6:05:37<2:08:24, 14.76s/it]
 68%|██████▊   | 1089/1610 [6:05:51<2:07:59, 14.74s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.7709766577609296, 'learning_rate': 3.236024844720497e-07, 'completion_length': 141.0357208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.013458251953125, 'epoch': 3.38}

 68%|██████▊   | 1089/1610 [6:05:51<2:07:59, 14.74s/it]
 68%|██████▊   | 1090/1610 [6:06:06<2:08:13, 14.80s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9061766664967028, 'learning_rate': 3.229813664596273e-07, 'completion_length': 160.1607208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.23086077719926834, 'kl': 0.01904296875, 'epoch': 3.39}

 68%|██████▊   | 1090/1610 [6:06:06<2:08:13, 14.80s/it]
 68%|██████▊   | 1091/1610 [6:06:22<2:09:58, 15.03s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.078512327055951, 'learning_rate': 3.2236024844720495e-07, 'completion_length': 170.33929443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1896214708685875, 'kl': 0.0159912109375, 'epoch': 3.39}

 68%|██████▊   | 1091/1610 [6:06:22<2:09:58, 15.03s/it]
 68%|██████▊   | 1092/1610 [6:06:36<2:08:06, 14.84s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5077415148218481, 'learning_rate': 3.217391304347826e-07, 'completion_length': 156.46429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.11266788095235825, 'kl': 0.0174560546875, 'epoch': 3.39}

 68%|██████▊   | 1092/1610 [6:06:36<2:08:06, 14.84s/it]
 68%|██████▊   | 1093/1610 [6:06:51<2:07:54, 14.84s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.4646411622960314, 'learning_rate': 3.2111801242236027e-07, 'completion_length': 154.4107208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571492433548, 'kl': 0.02178955078125, 'epoch': 3.39}

 68%|██████▊   | 1093/1610 [6:06:51<2:07:54, 14.84s/it]
 68%|██████▊   | 1094/1610 [6:07:07<2:11:03, 15.24s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9889129621038867, 'learning_rate': 3.204968944099379e-07, 'completion_length': 150.37500762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.26657507196068764, 'kl': 0.020263671875, 'epoch': 3.4}

 68%|██████▊   | 1094/1610 [6:07:07<2:11:03, 15.24s/it]
 68%|██████▊   | 1095/1610 [6:07:25<2:17:55, 16.07s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9399969691524896, 'learning_rate': 3.198757763975155e-07, 'completion_length': 172.9107208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.2142857313156128, 'kl': 0.016143798828125, 'epoch': 3.4}

 68%|██████▊   | 1095/1610 [6:07:25<2:17:55, 16.07s/it]
 68%|██████▊   | 1096/1610 [6:07:41<2:17:50, 16.09s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8651287975567349, 'learning_rate': 3.192546583850931e-07, 'completion_length': 169.46428680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.14838216453790665, 'kl': 0.0208740234375, 'epoch': 3.4}

 68%|██████▊   | 1096/1610 [6:07:41<2:17:50, 16.09s/it]
 68%|██████▊   | 1097/1610 [6:07:56<2:14:53, 15.78s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.918821167019824, 'learning_rate': 3.186335403726708e-07, 'completion_length': 154.51786041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.2610500305891037, 'kl': 0.017791748046875, 'epoch': 3.41}

 68%|██████▊   | 1097/1610 [6:07:56<2:14:53, 15.78s/it]
 68%|██████▊   | 1098/1610 [6:08:09<2:06:52, 14.87s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.730173730872406, 'learning_rate': 3.1801242236024843e-07, 'completion_length': 132.58929443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571529686451, 'kl': 0.013153076171875, 'epoch': 3.41}

 68%|██████▊   | 1098/1610 [6:08:09<2:06:52, 14.87s/it]
 68%|██████▊   | 1099/1610 [6:08:26<2:11:30, 15.44s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 2.28098680328758, 'learning_rate': 3.1739130434782606e-07, 'completion_length': 148.85714721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.3078143820166588, 'kl': 0.015533447265625, 'epoch': 3.41}

 68%|██████▊   | 1099/1610 [6:08:26<2:11:30, 15.44s/it]
 68%|██████▊   | 1100/1610 [6:08:40<2:08:28, 15.12s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0410615514875303, 'learning_rate': 3.167701863354037e-07, 'completion_length': 144.125, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2253357470035553, 'kl': 0.018798828125, 'epoch': 3.42}

 68%|██████▊   | 1100/1610 [6:08:40<2:08:28, 15.12s/it]
 68%|██████▊   | 1101/1610 [6:13:26<13:35:50, 96.17s/it]
                                                        
{'loss': 0.0006, 'grad_norm': 0.7031431004832033, 'learning_rate': 3.1614906832298133e-07, 'completion_length': 173.8214340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.014129638671875, 'epoch': 3.42}

 68%|██████▊   | 1101/1610 [6:13:26<13:35:50, 96.17s/it]
 68%|██████▊   | 1102/1610 [6:13:42<10:10:41, 72.13s/it]
                                                        
{'loss': 0.0007, 'grad_norm': 0.7283138446328395, 'learning_rate': 3.15527950310559e-07, 'completion_length': 167.26786041259766, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.0184326171875, 'epoch': 3.42}

 68%|██████▊   | 1102/1610 [6:13:42<10:10:41, 72.13s/it]
 69%|██████▊   | 1103/1610 [6:13:59<7:51:23, 55.79s/it] 
                                                       
{'loss': 0.0006, 'grad_norm': 0.8475847126743883, 'learning_rate': 3.1490683229813665e-07, 'completion_length': 156.62500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514649361371994, 'kl': 0.01513671875, 'epoch': 3.43}

 69%|██████▊   | 1103/1610 [6:13:59<7:51:23, 55.79s/it]
 69%|██████▊   | 1104/1610 [6:14:16<6:13:00, 44.23s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.7596936670177368, 'learning_rate': 3.142857142857143e-07, 'completion_length': 176.73214721679688, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.013458251953125, 'epoch': 3.43}

 69%|██████▊   | 1104/1610 [6:14:16<6:13:00, 44.23s/it]
 69%|██████▊   | 1105/1610 [6:14:33<5:01:44, 35.85s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9647789956183358, 'learning_rate': 3.136645962732919e-07, 'completion_length': 177.1964340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409644439816475, 'kl': 0.015838623046875, 'epoch': 3.43}

 69%|██████▊   | 1105/1610 [6:14:33<5:01:44, 35.85s/it]
 69%|██████▊   | 1106/1610 [6:14:49<4:12:40, 30.08s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6922181353353477, 'learning_rate': 3.130434782608696e-07, 'completion_length': 165.33929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571492433548, 'kl': 0.012237548828125, 'epoch': 3.43}

 69%|██████▊   | 1106/1610 [6:14:49<4:12:40, 30.08s/it]
 69%|██████▉   | 1107/1610 [6:15:07<3:40:25, 26.29s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 2.5167353868415114, 'learning_rate': 3.1242236024844723e-07, 'completion_length': 178.55358123779297, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.23086078464984894, 'kl': 0.01776123046875, 'epoch': 3.44}

 69%|██████▉   | 1107/1610 [6:15:07<3:40:25, 26.29s/it]
 69%|██████▉   | 1108/1610 [6:15:27<3:24:33, 24.45s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7962418637052525, 'learning_rate': 3.118012422360248e-07, 'completion_length': 188.6071548461914, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.24241763353347778, 'kl': 0.0146484375, 'epoch': 3.44}

 69%|██████▉   | 1108/1610 [6:15:27<3:24:33, 24.45s/it]
 69%|██████▉   | 1109/1610 [6:15:44<3:04:58, 22.15s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8421260567597444, 'learning_rate': 3.1118012422360244e-07, 'completion_length': 161.9464340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357619047165, 'kl': 0.018798828125, 'epoch': 3.44}

 69%|██████▉   | 1109/1610 [6:15:44<3:04:58, 22.15s/it]
 69%|██████▉   | 1110/1610 [6:16:03<2:56:59, 21.24s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.5439592255656639, 'learning_rate': 3.105590062111801e-07, 'completion_length': 163.30358123779297, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500000596046448, 'reward_std': 0.1539071835577488, 'kl': 0.012908935546875, 'epoch': 3.45}

 69%|██████▉   | 1110/1610 [6:16:03<2:56:59, 21.24s/it]
 69%|██████▉   | 1111/1610 [6:16:21<2:47:53, 20.19s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9821629682195429, 'learning_rate': 3.0993788819875776e-07, 'completion_length': 149.60714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981072798371315, 'kl': 0.0166015625, 'epoch': 3.45}

 69%|██████▉   | 1111/1610 [6:16:21<2:47:53, 20.19s/it]
 69%|██████▉   | 1112/1610 [6:16:37<2:37:31, 18.98s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 3.6335942014957987, 'learning_rate': 3.093167701863354e-07, 'completion_length': 168.33929443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981074661016464, 'kl': 0.01806640625, 'epoch': 3.45}

 69%|██████▉   | 1112/1610 [6:16:37<2:37:31, 18.98s/it]
 69%|██████▉   | 1113/1610 [6:16:53<2:31:00, 18.23s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5964471848606717, 'learning_rate': 3.08695652173913e-07, 'completion_length': 153.17858123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428656578064, 'kl': 0.017059326171875, 'epoch': 3.46}

 69%|██████▉   | 1113/1610 [6:16:53<2:31:00, 18.23s/it]
 69%|██████▉   | 1114/1610 [6:17:13<2:33:22, 18.55s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9306038497595951, 'learning_rate': 3.0807453416149066e-07, 'completion_length': 180.2678680419922, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.14838216453790665, 'kl': 0.014373779296875, 'epoch': 3.46}

 69%|██████▉   | 1114/1610 [6:17:13<2:33:22, 18.55s/it]
 69%|██████▉   | 1115/1610 [6:17:29<2:26:37, 17.77s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8487166883958219, 'learning_rate': 3.0745341614906834e-07, 'completion_length': 150.01786041259766, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.0714285746216774, 'kl': 0.014404296875, 'epoch': 3.46}

 69%|██████▉   | 1115/1610 [6:17:29<2:26:37, 17.77s/it]
 69%|██████▉   | 1116/1610 [6:17:47<2:28:15, 18.01s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8894719317080902, 'learning_rate': 3.06832298136646e-07, 'completion_length': 169.4107208251953, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.018280029296875, 'epoch': 3.47}

 69%|██████▉   | 1116/1610 [6:17:47<2:28:15, 18.01s/it]
 69%|██████▉   | 1117/1610 [6:18:07<2:33:54, 18.73s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.1137036216231986, 'learning_rate': 3.062111801242236e-07, 'completion_length': 175.05357360839844, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.014251708984375, 'epoch': 3.47}

 69%|██████▉   | 1117/1610 [6:18:07<2:33:54, 18.73s/it]
 69%|██████▉   | 1118/1610 [6:18:26<2:33:49, 18.76s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.8857279992208468, 'learning_rate': 3.0559006211180124e-07, 'completion_length': 172.83929443359375, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.25552502274513245, 'kl': 0.02362060546875, 'epoch': 3.47}

 69%|██████▉   | 1118/1610 [6:18:26<2:33:49, 18.76s/it]
 70%|██████▉   | 1119/1610 [6:18:43<2:28:07, 18.10s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.002787606152119, 'learning_rate': 3.049689440993788e-07, 'completion_length': 134.6607208251953, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071798324585, 'kl': 0.015533447265625, 'epoch': 3.48}

 70%|██████▉   | 1119/1610 [6:18:43<2:28:07, 18.10s/it]
 70%|██████▉   | 1120/1610 [6:19:00<2:26:33, 17.95s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.689834440141825, 'learning_rate': 3.043478260869565e-07, 'completion_length': 152.4107208251953, 'rewards/accuracy_reward': 0.75, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266788095235825, 'kl': 0.01556396484375, 'epoch': 3.48}

 70%|██████▉   | 1120/1610 [6:19:00<2:26:33, 17.95s/it]
 70%|██████▉   | 1121/1610 [6:19:16<2:19:24, 17.11s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5523469801511884, 'learning_rate': 3.0372670807453414e-07, 'completion_length': 138.3928680419922, 'rewards/accuracy_reward': 0.6071428805589676, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266787722706795, 'kl': 0.016693115234375, 'epoch': 3.48}

 70%|██████▉   | 1121/1610 [6:19:16<2:19:24, 17.11s/it]
 70%|██████▉   | 1122/1610 [6:19:29<2:10:50, 16.09s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8192359348997025, 'learning_rate': 3.0310559006211177e-07, 'completion_length': 132.83929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928999722004, 'kl': 0.014892578125, 'epoch': 3.48}

 70%|██████▉   | 1122/1610 [6:19:29<2:10:50, 16.09s/it]
 70%|██████▉   | 1123/1610 [6:19:46<2:11:04, 16.15s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9890154177617543, 'learning_rate': 3.024844720496894e-07, 'completion_length': 186.3928680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.25552503019571304, 'kl': 0.01318359375, 'epoch': 3.49}

 70%|██████▉   | 1123/1610 [6:19:46<2:11:04, 16.15s/it]
 70%|██████▉   | 1124/1610 [6:19:59<2:04:58, 15.43s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7115274178128236, 'learning_rate': 3.018633540372671e-07, 'completion_length': 132.69643020629883, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.0146484375, 'epoch': 3.49}

 70%|██████▉   | 1124/1610 [6:19:59<2:04:58, 15.43s/it]
 70%|██████▉   | 1125/1610 [6:20:18<2:12:05, 16.34s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6663530380283761, 'learning_rate': 3.012422360248447e-07, 'completion_length': 187.4464340209961, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086079210042953, 'kl': 0.017578125, 'epoch': 3.49}

 70%|██████▉   | 1125/1610 [6:20:18<2:12:05, 16.34s/it]
 70%|██████▉   | 1126/1610 [6:20:35<2:13:10, 16.51s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.264581530733715, 'learning_rate': 3.0062111801242235e-07, 'completion_length': 163.71429443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.2610500454902649, 'kl': 0.02008056640625, 'epoch': 3.5}

 70%|██████▉   | 1126/1610 [6:20:35<2:13:10, 16.51s/it]
 70%|███████   | 1127/1610 [6:20:49<2:08:25, 15.95s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 5.895327999259108, 'learning_rate': 3e-07, 'completion_length': 141.62500762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.14838216453790665, 'kl': 0.011444091796875, 'epoch': 3.5}

 70%|███████   | 1127/1610 [6:20:49<2:08:25, 15.95s/it]
 70%|███████   | 1128/1610 [6:21:04<2:05:50, 15.67s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.3953098182282588, 'learning_rate': 2.993788819875776e-07, 'completion_length': 166.92858123779297, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.015960693359375, 'epoch': 3.5}

 70%|███████   | 1128/1610 [6:21:04<2:05:50, 15.67s/it]
 70%|███████   | 1129/1610 [6:21:21<2:07:31, 15.91s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.8702671382332248, 'learning_rate': 2.987577639751553e-07, 'completion_length': 169.26786041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3078143745660782, 'kl': 0.0196533203125, 'epoch': 3.51}

 70%|███████   | 1129/1610 [6:21:21<2:07:31, 15.91s/it]
 70%|███████   | 1130/1610 [6:21:36<2:06:26, 15.81s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.1282951985489853, 'learning_rate': 2.9813664596273294e-07, 'completion_length': 155.2857208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357619047165, 'kl': 0.02313232421875, 'epoch': 3.51}

 70%|███████   | 1130/1610 [6:21:36<2:06:26, 15.81s/it]
 70%|███████   | 1131/1610 [6:21:53<2:08:04, 16.04s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.739915607864709, 'learning_rate': 2.975155279503105e-07, 'completion_length': 147.17858123779297, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.014801025390625, 'epoch': 3.51}

 70%|███████   | 1131/1610 [6:21:53<2:08:04, 16.04s/it]
 70%|███████   | 1132/1610 [6:22:09<2:06:30, 15.88s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.157819038180308, 'learning_rate': 2.9689440993788815e-07, 'completion_length': 166.05358123779297, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.2253357619047165, 'kl': 0.018463134765625, 'epoch': 3.52}

 70%|███████   | 1132/1610 [6:22:09<2:06:30, 15.88s/it]
 70%|███████   | 1133/1610 [6:22:24<2:05:19, 15.76s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 2.300618860491481, 'learning_rate': 2.9627329192546583e-07, 'completion_length': 177.0178680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2967643290758133, 'kl': 0.018310546875, 'epoch': 3.52}

 70%|███████   | 1133/1610 [6:22:24<2:05:19, 15.76s/it]
 70%|███████   | 1134/1610 [6:22:42<2:09:56, 16.38s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.602898704493871, 'learning_rate': 2.9565217391304347e-07, 'completion_length': 157.73214721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.28365693986415863, 'kl': 0.01434326171875, 'epoch': 3.52}

 70%|███████   | 1134/1610 [6:22:42<2:09:56, 16.38s/it]
 70%|███████   | 1135/1610 [6:22:57<2:07:40, 16.13s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 4.998154884142832, 'learning_rate': 2.950310559006211e-07, 'completion_length': 152.6071548461914, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.1071428619325161, 'kl': 0.01593017578125, 'epoch': 3.52}

 70%|███████   | 1135/1610 [6:22:57<2:07:40, 16.13s/it]
 71%|███████   | 1136/1610 [6:23:14<2:07:57, 16.20s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9172049887023804, 'learning_rate': 2.9440993788819873e-07, 'completion_length': 166.0357208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981073170900345, 'kl': 0.0174560546875, 'epoch': 3.53}

 71%|███████   | 1136/1610 [6:23:14<2:07:57, 16.20s/it]
 71%|███████   | 1137/1610 [6:23:31<2:10:05, 16.50s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.4698795978337355, 'learning_rate': 2.9378881987577636e-07, 'completion_length': 156.1964340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.12371791899204254, 'kl': 0.013763427734375, 'epoch': 3.53}

 71%|███████   | 1137/1610 [6:23:31<2:10:05, 16.50s/it]
 71%|███████   | 1138/1610 [6:23:49<2:12:21, 16.83s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.7485861898066934, 'learning_rate': 2.9316770186335405e-07, 'completion_length': 165.33929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500000596046448, 'reward_std': 0.18409645557403564, 'kl': 0.0125732421875, 'epoch': 3.53}

 71%|███████   | 1138/1610 [6:23:49<2:12:21, 16.83s/it]
 71%|███████   | 1139/1610 [6:24:06<2:13:20, 16.99s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9840320683572892, 'learning_rate': 2.925465838509317e-07, 'completion_length': 177.62500762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.15943220630288124, 'kl': 0.014892578125, 'epoch': 3.54}

 71%|███████   | 1139/1610 [6:24:06<2:13:20, 16.99s/it]
 71%|███████   | 1140/1610 [6:24:21<2:07:57, 16.33s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6639900345372234, 'learning_rate': 2.919254658385093e-07, 'completion_length': 147.5357208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.014923095703125, 'epoch': 3.54}

 71%|███████   | 1140/1610 [6:24:21<2:07:57, 16.33s/it]
 71%|███████   | 1141/1610 [6:24:37<2:08:40, 16.46s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 2.560092977127457, 'learning_rate': 2.9130434782608695e-07, 'completion_length': 168.80358123779297, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981074661016464, 'kl': 0.0203857421875, 'epoch': 3.54}

 71%|███████   | 1141/1610 [6:24:37<2:08:40, 16.46s/it]
 71%|███████   | 1142/1610 [6:24:50<1:59:40, 15.34s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9986004365461308, 'learning_rate': 2.9068322981366463e-07, 'completion_length': 125.98214721679688, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.18409645557403564, 'kl': 0.016143798828125, 'epoch': 3.55}

 71%|███████   | 1142/1610 [6:24:50<1:59:40, 15.34s/it]
 71%|███████   | 1143/1610 [6:25:06<2:00:52, 15.53s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.3777093318526307, 'learning_rate': 2.900621118012422e-07, 'completion_length': 166.21429443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.07695359364151955, 'kl': 0.01190185546875, 'epoch': 3.55}

 71%|███████   | 1143/1610 [6:25:06<2:00:52, 15.53s/it]
 71%|███████   | 1144/1610 [6:25:20<1:55:33, 14.88s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 2.4343214841369036, 'learning_rate': 2.8944099378881985e-07, 'completion_length': 127.35715103149414, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.0714285746216774, 'kl': 0.015350341796875, 'epoch': 3.55}

 71%|███████   | 1144/1610 [6:25:20<1:55:33, 14.88s/it]
 71%|███████   | 1145/1610 [6:25:35<1:57:02, 15.10s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.4665078018259396, 'learning_rate': 2.888198757763975e-07, 'completion_length': 170.33929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216453790665, 'kl': 0.014923095703125, 'epoch': 3.56}

 71%|███████   | 1145/1610 [6:25:35<1:57:02, 15.10s/it]
 71%|███████   | 1146/1610 [6:25:50<1:56:09, 15.02s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.48386084249104516, 'learning_rate': 2.881987577639751e-07, 'completion_length': 149.37500762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.0357142873108387, 'kl': 0.0155029296875, 'epoch': 3.56}

 71%|███████   | 1146/1610 [6:25:50<1:56:09, 15.02s/it]
 71%|███████   | 1147/1610 [6:26:06<1:57:33, 15.23s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.743841724021297, 'learning_rate': 2.875776397515528e-07, 'completion_length': 149.12500762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.1428571529686451, 'kl': 0.01513671875, 'epoch': 3.56}

 71%|███████   | 1147/1610 [6:26:06<1:57:33, 15.23s/it]
 71%|███████▏  | 1148/1610 [6:26:20<1:56:15, 15.10s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0536168034752835, 'learning_rate': 2.8695652173913043e-07, 'completion_length': 150.50000762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.30228935927152634, 'kl': 0.02044677734375, 'epoch': 3.57}

 71%|███████▏  | 1148/1610 [6:26:20<1:56:15, 15.10s/it]
 71%|███████▏  | 1149/1610 [6:26:36<1:57:38, 15.31s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.3830063875983884, 'learning_rate': 2.8633540372670806e-07, 'completion_length': 148.33929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.01373291015625, 'epoch': 3.57}

 71%|███████▏  | 1149/1610 [6:26:36<1:57:38, 15.31s/it]
 71%|███████▏  | 1150/1610 [6:26:54<2:03:59, 16.17s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6979600085893118, 'learning_rate': 2.857142857142857e-07, 'completion_length': 175.7321548461914, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1539071872830391, 'kl': 0.01898193359375, 'epoch': 3.57}

 71%|███████▏  | 1150/1610 [6:26:54<2:03:59, 16.17s/it]
 71%|███████▏  | 1151/1610 [6:27:11<2:05:32, 16.41s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.49054879245695526, 'learning_rate': 2.850931677018634e-07, 'completion_length': 169.1071548461914, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.01654052734375, 'epoch': 3.57}

 71%|███████▏  | 1151/1610 [6:27:11<2:05:32, 16.41s/it]
 72%|███████▏  | 1152/1610 [6:27:28<2:05:58, 16.50s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 2.0003897428795403, 'learning_rate': 2.84472049689441e-07, 'completion_length': 144.35714721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.01751708984375, 'epoch': 3.58}

 72%|███████▏  | 1152/1610 [6:27:28<2:05:58, 16.50s/it]
 72%|███████▏  | 1153/1610 [6:27:40<1:54:24, 15.02s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0780067693572053, 'learning_rate': 2.8385093167701864e-07, 'completion_length': 122.39286041259766, 'rewards/accuracy_reward': 0.8035714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1181928962469101, 'kl': 0.02117919921875, 'epoch': 3.58}

 72%|███████▏  | 1153/1610 [6:27:40<1:54:24, 15.02s/it]
 72%|███████▏  | 1154/1610 [6:27:54<1:52:42, 14.83s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8674790553875968, 'learning_rate': 2.832298136645963e-07, 'completion_length': 140.92857360839844, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571492433548, 'kl': 0.0185546875, 'epoch': 3.58}

 72%|███████▏  | 1154/1610 [6:27:54<1:52:42, 14.83s/it]
 72%|███████▏  | 1155/1610 [6:28:12<1:58:35, 15.64s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.914543654427211, 'learning_rate': 2.8260869565217386e-07, 'completion_length': 181.46429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.013458251953125, 'epoch': 3.59}

 72%|███████▏  | 1155/1610 [6:28:12<1:58:35, 15.64s/it]
 72%|███████▏  | 1156/1610 [6:28:28<2:01:04, 16.00s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.4618639682444585, 'learning_rate': 2.8198757763975154e-07, 'completion_length': 186.6428680419922, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357470035553, 'kl': 0.01983642578125, 'epoch': 3.59}

 72%|███████▏  | 1156/1610 [6:28:28<2:01:04, 16.00s/it]
 72%|███████▏  | 1157/1610 [6:28:43<1:58:09, 15.65s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6603027074057657, 'learning_rate': 2.813664596273292e-07, 'completion_length': 168.32144165039062, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1896214708685875, 'kl': 0.01416015625, 'epoch': 3.59}

 72%|███████▏  | 1157/1610 [6:28:43<1:58:09, 15.65s/it]
 72%|███████▏  | 1158/1610 [6:28:58<1:55:41, 15.36s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0418462206604666, 'learning_rate': 2.807453416149068e-07, 'completion_length': 136.3928680419922, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.19514649361371994, 'kl': 0.017913818359375, 'epoch': 3.6}

 72%|███████▏  | 1158/1610 [6:28:58<1:55:41, 15.36s/it]
 72%|███████▏  | 1159/1610 [6:29:17<2:03:21, 16.41s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.8246693190939698, 'learning_rate': 2.8012422360248444e-07, 'completion_length': 204.80358123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5357143878936768, 'reward_std': 0.26657506078481674, 'kl': 0.021636962890625, 'epoch': 3.6}

 72%|███████▏  | 1159/1610 [6:29:17<2:03:21, 16.41s/it]
 72%|███████▏  | 1160/1610 [6:29:32<2:01:06, 16.15s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.1961102604599478, 'learning_rate': 2.7950310559006207e-07, 'completion_length': 155.5714340209961, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.017333984375, 'epoch': 3.6}

 72%|███████▏  | 1160/1610 [6:29:32<2:01:06, 16.15s/it]
 72%|███████▏  | 1161/1610 [6:29:50<2:04:50, 16.68s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 6.782498773195681, 'learning_rate': 2.7888198757763976e-07, 'completion_length': 196.5535888671875, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.21981074661016464, 'kl': 0.01837158203125, 'epoch': 3.61}

 72%|███████▏  | 1161/1610 [6:29:50<2:04:50, 16.68s/it]
 72%|███████▏  | 1162/1610 [6:30:06<2:02:57, 16.47s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9904750060844105, 'learning_rate': 2.782608695652174e-07, 'completion_length': 166.32144165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1428571529686451, 'kl': 0.02093505859375, 'epoch': 3.61}

 72%|███████▏  | 1162/1610 [6:30:06<2:02:57, 16.47s/it]
 72%|███████▏  | 1163/1610 [6:30:20<1:57:11, 15.73s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7228769262543113, 'learning_rate': 2.77639751552795e-07, 'completion_length': 140.69643783569336, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.016021728515625, 'epoch': 3.61}

 72%|███████▏  | 1163/1610 [6:30:20<1:57:11, 15.73s/it]
 72%|███████▏  | 1164/1610 [6:30:37<1:58:24, 15.93s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.635803826307291, 'learning_rate': 2.7701863354037266e-07, 'completion_length': 154.0714340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.946428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.0129852294921875, 'epoch': 3.61}

 72%|███████▏  | 1164/1610 [6:30:37<1:58:24, 15.93s/it]
 72%|███████▏  | 1165/1610 [6:30:50<1:51:52, 15.08s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2955939468809248, 'learning_rate': 2.7639751552795034e-07, 'completion_length': 130.33928680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266787722706795, 'kl': 0.01788330078125, 'epoch': 3.62}

 72%|███████▏  | 1165/1610 [6:30:50<1:51:52, 15.08s/it]
 72%|███████▏  | 1166/1610 [6:31:06<1:54:40, 15.50s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6480321300267018, 'learning_rate': 2.7577639751552797e-07, 'completion_length': 164.60714721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838217198848724, 'kl': 0.0113525390625, 'epoch': 3.62}

 72%|███████▏  | 1166/1610 [6:31:06<1:54:40, 15.50s/it]
 72%|███████▏  | 1167/1610 [6:31:22<1:54:50, 15.55s/it]
                                                       
{'loss': 0.001, 'grad_norm': 0.5109781744030122, 'learning_rate': 2.7515527950310555e-07, 'completion_length': 153.05357360839844, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1181928962469101, 'kl': 0.02435302734375, 'epoch': 3.62}

 72%|███████▏  | 1167/1610 [6:31:22<1:54:50, 15.55s/it]
 73%|███████▎  | 1168/1610 [6:31:38<1:55:42, 15.71s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5837903689484782, 'learning_rate': 2.745341614906832e-07, 'completion_length': 147.21428680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.015167236328125, 'epoch': 3.63}

 73%|███████▎  | 1168/1610 [6:31:38<1:55:42, 15.71s/it]
 73%|███████▎  | 1169/1610 [6:31:53<1:52:56, 15.37s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9547867436943055, 'learning_rate': 2.739130434782608e-07, 'completion_length': 137.3214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18409645557403564, 'kl': 0.017486572265625, 'epoch': 3.63}

 73%|███████▎  | 1169/1610 [6:31:53<1:52:56, 15.37s/it]
 73%|███████▎  | 1170/1610 [6:32:06<1:48:12, 14.76s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2336800692552197, 'learning_rate': 2.732919254658385e-07, 'completion_length': 139.6071548461914, 'rewards/accuracy_reward': 0.7321428656578064, 'rewards/format_reward': 1.0, 'reward': 1.7321428656578064, 'reward_std': 0.1181928962469101, 'kl': 0.016998291015625, 'epoch': 3.63}

 73%|███████▎  | 1170/1610 [6:32:06<1:48:12, 14.76s/it]
 73%|███████▎  | 1171/1610 [6:32:22<1:51:12, 15.20s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.5598921141315296, 'learning_rate': 2.7267080745341614e-07, 'completion_length': 169.30358123779297, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.0218505859375, 'epoch': 3.64}

 73%|███████▎  | 1171/1610 [6:32:22<1:51:12, 15.20s/it]
 73%|███████▎  | 1172/1610 [6:32:36<1:47:21, 14.71s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0607406389353742, 'learning_rate': 2.7204968944099377e-07, 'completion_length': 140.1785774230957, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1896214708685875, 'kl': 0.016326904296875, 'epoch': 3.64}

 73%|███████▎  | 1172/1610 [6:32:36<1:47:21, 14.71s/it]
 73%|███████▎  | 1173/1610 [6:32:54<1:54:19, 15.70s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6367893914622499, 'learning_rate': 2.714285714285714e-07, 'completion_length': 182.50000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.04123930633068085, 'kl': 0.015777587890625, 'epoch': 3.64}

 73%|███████▎  | 1173/1610 [6:32:54<1:54:19, 15.70s/it]
 73%|███████▎  | 1174/1610 [6:33:09<1:53:52, 15.67s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9496801842013571, 'learning_rate': 2.708074534161491e-07, 'completion_length': 156.5357208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838217198848724, 'kl': 0.01715087890625, 'epoch': 3.65}

 73%|███████▎  | 1174/1610 [6:33:09<1:53:52, 15.67s/it]
 73%|███████▎  | 1175/1610 [6:33:22<1:47:58, 14.89s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9597079864081159, 'learning_rate': 2.701863354037267e-07, 'completion_length': 128.12500381469727, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.15943220257759094, 'kl': 0.01190185546875, 'epoch': 3.65}

 73%|███████▎  | 1175/1610 [6:33:22<1:47:58, 14.89s/it]
 73%|███████▎  | 1176/1610 [6:33:40<1:52:49, 15.60s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3522740845284829, 'learning_rate': 2.6956521739130435e-07, 'completion_length': 150.5357208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2857142984867096, 'kl': 0.018280029296875, 'epoch': 3.65}

 73%|███████▎  | 1176/1610 [6:33:40<1:52:49, 15.60s/it]
 73%|███████▎  | 1177/1610 [6:33:56<1:54:38, 15.89s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8003800508692708, 'learning_rate': 2.68944099378882e-07, 'completion_length': 172.5357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073170900345, 'kl': 0.015228271484375, 'epoch': 3.66}

 73%|███████▎  | 1177/1610 [6:33:56<1:54:38, 15.89s/it]
 73%|███████▎  | 1178/1610 [6:34:10<1:50:14, 15.31s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9762656087727254, 'learning_rate': 2.6832298136645956e-07, 'completion_length': 128.89286422729492, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714402794838, 'kl': 0.011688232421875, 'epoch': 3.66}

 73%|███████▎  | 1178/1610 [6:34:10<1:50:14, 15.31s/it]
 73%|███████▎  | 1179/1610 [6:34:25<1:48:43, 15.13s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.4810825394676148, 'learning_rate': 2.6770186335403725e-07, 'completion_length': 154.23214721679688, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.011871337890625, 'epoch': 3.66}

 73%|███████▎  | 1179/1610 [6:34:25<1:48:43, 15.13s/it]
 73%|███████▎  | 1180/1610 [6:34:41<1:50:50, 15.47s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0806429786963252, 'learning_rate': 2.670807453416149e-07, 'completion_length': 173.17858123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981072798371315, 'kl': 0.015960693359375, 'epoch': 3.66}

 73%|███████▎  | 1180/1610 [6:34:41<1:50:50, 15.47s/it]
 73%|███████▎  | 1181/1610 [6:34:55<1:47:47, 15.07s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7574546088229831, 'learning_rate': 2.664596273291925e-07, 'completion_length': 149.55357360839844, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071872830391, 'kl': 0.0142822265625, 'epoch': 3.67}

 73%|███████▎  | 1181/1610 [6:34:55<1:47:47, 15.07s/it]
 73%|███████▎  | 1182/1610 [6:35:08<1:43:07, 14.46s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9421979483757782, 'learning_rate': 2.6583850931677015e-07, 'completion_length': 137.92857360839844, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.21981073915958405, 'kl': 0.01812744140625, 'epoch': 3.67}

 73%|███████▎  | 1182/1610 [6:35:08<1:43:07, 14.46s/it]
 73%|███████▎  | 1183/1610 [6:35:22<1:40:49, 14.17s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6622669594991447, 'learning_rate': 2.6521739130434783e-07, 'completion_length': 140.71429443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.07695359364151955, 'kl': 0.0155029296875, 'epoch': 3.67}

 73%|███████▎  | 1183/1610 [6:35:22<1:40:49, 14.17s/it]
 74%|███████▎  | 1184/1610 [6:35:39<1:46:18, 14.97s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8516926135415329, 'learning_rate': 2.6459627329192547e-07, 'completion_length': 179.3214340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2500000186264515, 'kl': 0.01715087890625, 'epoch': 3.68}

 74%|███████▎  | 1184/1610 [6:35:39<1:46:18, 14.97s/it]
 74%|███████▎  | 1185/1610 [6:35:55<1:49:36, 15.47s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0194434981542395, 'learning_rate': 2.639751552795031e-07, 'completion_length': 207.69644165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409644439816475, 'kl': 0.015289306640625, 'epoch': 3.68}

 74%|███████▎  | 1185/1610 [6:35:55<1:49:36, 15.47s/it]
 74%|███████▎  | 1186/1610 [6:36:10<1:48:34, 15.36s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.706502899776718, 'learning_rate': 2.6335403726708073e-07, 'completion_length': 169.8214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.18409645557403564, 'kl': 0.016082763671875, 'epoch': 3.68}

 74%|███████▎  | 1186/1610 [6:36:10<1:48:34, 15.36s/it]
 74%|███████▎  | 1187/1610 [6:36:25<1:45:41, 14.99s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0817840340474185, 'learning_rate': 2.6273291925465836e-07, 'completion_length': 134.42857360839844, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1785714402794838, 'kl': 0.019439697265625, 'epoch': 3.69}

 74%|███████▎  | 1187/1610 [6:36:25<1:45:41, 14.99s/it]
 74%|███████▍  | 1188/1610 [6:36:37<1:40:05, 14.23s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9117376655894049, 'learning_rate': 2.6211180124223605e-07, 'completion_length': 119.62500762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.01300048828125, 'epoch': 3.69}

 74%|███████▍  | 1188/1610 [6:36:37<1:40:05, 14.23s/it]
 74%|███████▍  | 1189/1610 [6:36:52<1:41:54, 14.52s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5597730679725795, 'learning_rate': 2.614906832298137e-07, 'completion_length': 144.26786041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.07695359364151955, 'kl': 0.014984130859375, 'epoch': 3.69}

 74%|███████▍  | 1189/1610 [6:36:52<1:41:54, 14.52s/it]
 74%|███████▍  | 1190/1610 [6:37:09<1:46:08, 15.16s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 4.01187935937413, 'learning_rate': 2.6086956521739126e-07, 'completion_length': 156.30357360839844, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7321429252624512, 'reward_std': 0.2589927092194557, 'kl': 0.015380859375, 'epoch': 3.7}

 74%|███████▍  | 1190/1610 [6:37:09<1:46:08, 15.16s/it]
 74%|███████▍  | 1191/1610 [6:37:21<1:39:33, 14.26s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.8240127250631493, 'learning_rate': 2.602484472049689e-07, 'completion_length': 118.3214340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357619047165, 'kl': 0.0169677734375, 'epoch': 3.7}

 74%|███████▍  | 1191/1610 [6:37:21<1:39:33, 14.26s/it]
 74%|███████▍  | 1192/1610 [6:37:37<1:43:45, 14.89s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.2269394600454695, 'learning_rate': 2.596273291925466e-07, 'completion_length': 159.08929443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3324786126613617, 'kl': 0.016082763671875, 'epoch': 3.7}

 74%|███████▍  | 1192/1610 [6:37:37<1:43:45, 14.89s/it]
 74%|███████▍  | 1193/1610 [6:37:52<1:43:14, 14.86s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0158869838001088, 'learning_rate': 2.590062111801242e-07, 'completion_length': 144.23215103149414, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2006715089082718, 'kl': 0.01910400390625, 'epoch': 3.7}

 74%|███████▍  | 1193/1610 [6:37:52<1:43:14, 14.86s/it]
 74%|███████▍  | 1194/1610 [6:38:09<1:47:21, 15.49s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6521051299756713, 'learning_rate': 2.5838509316770184e-07, 'completion_length': 161.05357360839844, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.14838216453790665, 'kl': 0.0194091796875, 'epoch': 3.71}

 74%|███████▍  | 1194/1610 [6:38:09<1:47:21, 15.49s/it]
 74%|███████▍  | 1195/1610 [6:38:25<1:48:28, 15.68s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0403211162851576, 'learning_rate': 2.577639751552795e-07, 'completion_length': 159.37500762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.18409645557403564, 'kl': 0.02056884765625, 'epoch': 3.71}

 74%|███████▍  | 1195/1610 [6:38:25<1:48:28, 15.68s/it]
 74%|███████▍  | 1196/1610 [6:38:41<1:47:48, 15.62s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8553174994698635, 'learning_rate': 2.571428571428571e-07, 'completion_length': 164.50000762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.2142857238650322, 'kl': 0.015380859375, 'epoch': 3.71}

 74%|███████▍  | 1196/1610 [6:38:41<1:47:48, 15.62s/it]
 74%|███████▍  | 1197/1610 [6:38:59<1:52:35, 16.36s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1075776558566506, 'learning_rate': 2.565217391304348e-07, 'completion_length': 157.8928680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.1539071872830391, 'kl': 0.02044677734375, 'epoch': 3.72}

 74%|███████▍  | 1197/1610 [6:38:59<1:52:35, 16.36s/it]
 74%|███████▍  | 1198/1610 [6:39:16<1:54:31, 16.68s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.768008198219923, 'learning_rate': 2.5590062111801243e-07, 'completion_length': 181.51786041259766, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1181928999722004, 'kl': 0.0133056640625, 'epoch': 3.72}

 74%|███████▍  | 1198/1610 [6:39:16<1:54:31, 16.68s/it]
 74%|███████▍  | 1199/1610 [6:39:33<1:54:15, 16.68s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8187821437237616, 'learning_rate': 2.5527950310559006e-07, 'completion_length': 168.08929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.0166015625, 'epoch': 3.72}

 74%|███████▍  | 1199/1610 [6:39:33<1:54:15, 16.68s/it]
 75%|███████▍  | 1200/1610 [6:39:48<1:49:55, 16.09s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.229595277731079, 'learning_rate': 2.546583850931677e-07, 'completion_length': 134.71428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357544541359, 'kl': 0.015777587890625, 'epoch': 3.73}

 75%|███████▍  | 1200/1610 [6:39:48<1:49:55, 16.09s/it]
 75%|███████▍  | 1201/1610 [6:44:06<10:05:42, 88.86s/it]
                                                        
{'loss': 0.0006, 'grad_norm': 0.5516697168379393, 'learning_rate': 2.540372670807454e-07, 'completion_length': 104.07143020629883, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.0714285746216774, 'kl': 0.0142364501953125, 'epoch': 3.73}

 75%|███████▍  | 1201/1610 [6:44:06<10:05:42, 88.86s/it]
 75%|███████▍  | 1202/1610 [6:44:24<7:38:25, 67.42s/it] 
                                                       
{'loss': 0.0006, 'grad_norm': 0.9028433897891649, 'learning_rate': 2.5341614906832296e-07, 'completion_length': 153.92857360839844, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981073170900345, 'kl': 0.015533447265625, 'epoch': 3.73}

 75%|███████▍  | 1202/1610 [6:44:24<7:38:25, 67.42s/it]
 75%|███████▍  | 1203/1610 [6:44:42<5:56:42, 52.59s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8701417769434465, 'learning_rate': 2.527950310559006e-07, 'completion_length': 186.7857208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2610500529408455, 'kl': 0.019012451171875, 'epoch': 3.74}

 75%|███████▍  | 1203/1610 [6:44:42<5:56:42, 52.59s/it]
 75%|███████▍  | 1204/1610 [6:44:55<4:36:08, 40.81s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.4584262964350003, 'learning_rate': 2.521739130434782e-07, 'completion_length': 122.82143020629883, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.21981073915958405, 'kl': 0.019256591796875, 'epoch': 3.74}

 75%|███████▍  | 1204/1610 [6:44:55<4:36:08, 40.81s/it]
 75%|███████▍  | 1205/1610 [6:45:14<3:50:28, 34.15s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.7560815955781164, 'learning_rate': 2.5155279503105585e-07, 'completion_length': 152.08929443359375, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500001192092896, 'reward_std': 0.1539071835577488, 'kl': 0.013336181640625, 'epoch': 3.74}

 75%|███████▍  | 1205/1610 [6:45:14<3:50:28, 34.15s/it]
 75%|███████▍  | 1206/1610 [6:45:30<3:14:17, 28.86s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 4.980807251703525, 'learning_rate': 2.5093167701863354e-07, 'completion_length': 143.62500762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086079210042953, 'kl': 0.019683837890625, 'epoch': 3.75}

 75%|███████▍  | 1206/1610 [6:45:30<3:14:17, 28.86s/it]
 75%|███████▍  | 1207/1610 [6:45:47<2:49:40, 25.26s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.275205195895851, 'learning_rate': 2.5031055900621117e-07, 'completion_length': 137.7857208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07695359364151955, 'kl': 0.0167236328125, 'epoch': 3.75}

 75%|███████▍  | 1207/1610 [6:45:47<2:49:40, 25.26s/it]
 75%|███████▌  | 1208/1610 [6:46:02<2:28:25, 22.15s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8575228735536291, 'learning_rate': 2.496894409937888e-07, 'completion_length': 198.51786041259766, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.2142857238650322, 'kl': 0.015472412109375, 'epoch': 3.75}

 75%|███████▌  | 1208/1610 [6:46:02<2:28:25, 22.15s/it]
 75%|███████▌  | 1209/1610 [6:46:19<2:18:24, 20.71s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7832457967394816, 'learning_rate': 2.4906832298136644e-07, 'completion_length': 157.1964340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.01788330078125, 'epoch': 3.75}

 75%|███████▌  | 1209/1610 [6:46:19<2:18:24, 20.71s/it]
 75%|███████▌  | 1210/1610 [6:46:34<2:05:44, 18.86s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7427890519904885, 'learning_rate': 2.4844720496894407e-07, 'completion_length': 155.26786041259766, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1896214634180069, 'kl': 0.0179443359375, 'epoch': 3.76}

 75%|███████▌  | 1210/1610 [6:46:34<2:05:44, 18.86s/it]
 75%|███████▌  | 1211/1610 [6:46:52<2:04:10, 18.67s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.1495422918473663, 'learning_rate': 2.4782608695652176e-07, 'completion_length': 166.07144165039062, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1785714402794838, 'kl': 0.018646240234375, 'epoch': 3.76}

 75%|███████▌  | 1211/1610 [6:46:52<2:04:10, 18.67s/it]
 75%|███████▌  | 1212/1610 [6:47:06<1:55:20, 17.39s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9090154711742249, 'learning_rate': 2.472049689440994e-07, 'completion_length': 139.08929443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.1539071835577488, 'kl': 0.012725830078125, 'epoch': 3.76}

 75%|███████▌  | 1212/1610 [6:47:06<1:55:20, 17.39s/it]
 75%|███████▌  | 1213/1610 [6:47:22<1:52:10, 16.95s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.41243939278585223, 'learning_rate': 2.46583850931677e-07, 'completion_length': 158.05358123779297, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.0714285746216774, 'kl': 0.014129638671875, 'epoch': 3.77}

 75%|███████▌  | 1213/1610 [6:47:22<1:52:10, 16.95s/it]
 75%|███████▌  | 1214/1610 [6:47:40<1:54:07, 17.29s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2187726962739163, 'learning_rate': 2.4596273291925465e-07, 'completion_length': 173.9464340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6250000596046448, 'reward_std': 0.1896214783191681, 'kl': 0.02069091796875, 'epoch': 3.77}

 75%|███████▌  | 1214/1610 [6:47:40<1:54:07, 17.29s/it]
 75%|███████▌  | 1215/1610 [6:47:55<1:48:23, 16.46s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6108426069524917, 'learning_rate': 2.453416149068323e-07, 'completion_length': 154.69644165039062, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.012420654296875, 'epoch': 3.77}

 75%|███████▌  | 1215/1610 [6:47:55<1:48:23, 16.46s/it]
 76%|███████▌  | 1216/1610 [6:48:11<1:47:21, 16.35s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.08440966073850975, 'learning_rate': 2.447204968944099e-07, 'completion_length': 158.6607208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0, 'kl': 0.014739990234375, 'epoch': 3.78}

 76%|███████▌  | 1216/1610 [6:48:11<1:47:21, 16.35s/it]
 76%|███████▌  | 1217/1610 [6:48:27<1:45:42, 16.14s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7735447564187616, 'learning_rate': 2.4409937888198755e-07, 'completion_length': 163.96429443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1896214671432972, 'kl': 0.018096923828125, 'epoch': 3.78}

 76%|███████▌  | 1217/1610 [6:48:27<1:45:42, 16.14s/it]
 76%|███████▌  | 1218/1610 [6:48:42<1:44:26, 15.99s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.408426592592231, 'learning_rate': 2.4347826086956524e-07, 'completion_length': 154.5714340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.0357142873108387, 'kl': 0.01361083984375, 'epoch': 3.78}

 76%|███████▌  | 1218/1610 [6:48:42<1:44:26, 15.99s/it]
 76%|███████▌  | 1219/1610 [6:48:56<1:38:54, 15.18s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0069645219927135, 'learning_rate': 2.4285714285714287e-07, 'completion_length': 139.4107208251953, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.2142857313156128, 'kl': 0.01483154296875, 'epoch': 3.79}

 76%|███████▌  | 1219/1610 [6:48:56<1:38:54, 15.18s/it]
 76%|███████▌  | 1220/1610 [6:49:12<1:40:21, 15.44s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9195256571127983, 'learning_rate': 2.422360248447205e-07, 'completion_length': 158.7857208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2253357470035553, 'kl': 0.018798828125, 'epoch': 3.79}

 76%|███████▌  | 1220/1610 [6:49:12<1:40:21, 15.44s/it]
 76%|███████▌  | 1221/1610 [6:49:27<1:40:10, 15.45s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.7261985010042284, 'learning_rate': 2.4161490683229813e-07, 'completion_length': 164.9464340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.23086076974868774, 'kl': 0.01483154296875, 'epoch': 3.79}

 76%|███████▌  | 1221/1610 [6:49:27<1:40:10, 15.45s/it]
 76%|███████▌  | 1222/1610 [6:49:41<1:36:49, 14.97s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9214723209722797, 'learning_rate': 2.4099378881987577e-07, 'completion_length': 137.1607208251953, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071872830391, 'kl': 0.01861572265625, 'epoch': 3.8}

 76%|███████▌  | 1222/1610 [6:49:41<1:36:49, 14.97s/it]
 76%|███████▌  | 1223/1610 [6:49:59<1:43:10, 16.00s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.8864560912606475, 'learning_rate': 2.403726708074534e-07, 'completion_length': 165.69644165039062, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.25552502274513245, 'kl': 0.01324462890625, 'epoch': 3.8}

 76%|███████▌  | 1223/1610 [6:49:59<1:43:10, 16.00s/it]
 76%|███████▌  | 1224/1610 [6:50:16<1:43:59, 16.16s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9921171874804319, 'learning_rate': 2.3975155279503103e-07, 'completion_length': 151.50000381469727, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.25552503019571304, 'kl': 0.015533447265625, 'epoch': 3.8}

 76%|███████▌  | 1224/1610 [6:50:16<1:43:59, 16.16s/it]
 76%|███████▌  | 1225/1610 [6:50:33<1:45:50, 16.49s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5162998040249898, 'learning_rate': 2.391304347826087e-07, 'completion_length': 147.73215103149414, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.821428656578064, 'reward_std': 0.1428571492433548, 'kl': 0.01483154296875, 'epoch': 3.8}

 76%|███████▌  | 1225/1610 [6:50:33<1:45:50, 16.49s/it]
 76%|███████▌  | 1226/1610 [6:50:47<1:39:41, 15.58s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.7531739567492878, 'learning_rate': 2.385093167701863e-07, 'completion_length': 125.00000381469727, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.1181928962469101, 'kl': 0.012969970703125, 'epoch': 3.81}

 76%|███████▌  | 1226/1610 [6:50:47<1:39:41, 15.58s/it]
 76%|███████▌  | 1227/1610 [6:51:03<1:40:33, 15.75s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.4995562821095997, 'learning_rate': 2.3788819875776398e-07, 'completion_length': 158.2500114440918, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8750000596046448, 'reward_std': 0.14838217198848724, 'kl': 0.014801025390625, 'epoch': 3.81}

 76%|███████▌  | 1227/1610 [6:51:03<1:40:33, 15.75s/it]
 76%|███████▋  | 1228/1610 [6:51:19<1:40:51, 15.84s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9677309326827986, 'learning_rate': 2.3726708074534161e-07, 'completion_length': 145.35715103149414, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071798324585, 'kl': 0.01739501953125, 'epoch': 3.81}

 76%|███████▋  | 1228/1610 [6:51:19<1:40:51, 15.84s/it]
 76%|███████▋  | 1229/1610 [6:51:35<1:41:49, 16.04s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6010295643825185, 'learning_rate': 2.3664596273291925e-07, 'completion_length': 161.9821548461914, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.018310546875, 'epoch': 3.82}

 76%|███████▋  | 1229/1610 [6:51:35<1:41:49, 16.04s/it]
 76%|███████▋  | 1230/1610 [6:51:49<1:37:44, 15.43s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.4624974162895714, 'learning_rate': 2.3602484472049688e-07, 'completion_length': 136.75000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0824786126613617, 'kl': 0.012725830078125, 'epoch': 3.82}

 76%|███████▋  | 1230/1610 [6:51:49<1:37:44, 15.43s/it]
 76%|███████▋  | 1231/1610 [6:52:02<1:32:16, 14.61s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5095880429301048, 'learning_rate': 2.354037267080745e-07, 'completion_length': 131.00000381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.0714285746216774, 'kl': 0.01556396484375, 'epoch': 3.82}

 76%|███████▋  | 1231/1610 [6:52:02<1:32:16, 14.61s/it]
 77%|███████▋  | 1232/1610 [6:52:17<1:32:48, 14.73s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9492463403720605, 'learning_rate': 2.3478260869565217e-07, 'completion_length': 156.375, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.23086076974868774, 'kl': 0.01806640625, 'epoch': 3.83}

 77%|███████▋  | 1232/1610 [6:52:17<1:32:48, 14.73s/it]
 77%|███████▋  | 1233/1610 [6:52:32<1:32:35, 14.74s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0742562676601388, 'learning_rate': 2.341614906832298e-07, 'completion_length': 167.3571548461914, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.2253357544541359, 'kl': 0.02117919921875, 'epoch': 3.83}

 77%|███████▋  | 1233/1610 [6:52:32<1:32:35, 14.74s/it]
 77%|███████▋  | 1234/1610 [6:52:49<1:36:26, 15.39s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8239108891302299, 'learning_rate': 2.3354037267080746e-07, 'completion_length': 185.1607208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.23086076974868774, 'kl': 0.01702880859375, 'epoch': 3.83}

 77%|███████▋  | 1234/1610 [6:52:49<1:36:26, 15.39s/it]
 77%|███████▋  | 1235/1610 [6:53:07<1:41:30, 16.24s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.7699171433679188, 'learning_rate': 2.3291925465838507e-07, 'completion_length': 175.4107208251953, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071872830391, 'kl': 0.01885986328125, 'epoch': 3.84}

 77%|███████▋  | 1235/1610 [6:53:07<1:41:30, 16.24s/it]
 77%|███████▋  | 1236/1610 [6:53:24<1:42:57, 16.52s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6389861604139758, 'learning_rate': 2.3229813664596273e-07, 'completion_length': 183.67858123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.018798828125, 'epoch': 3.84}

 77%|███████▋  | 1236/1610 [6:53:24<1:42:57, 16.52s/it]
 77%|███████▋  | 1237/1610 [6:53:40<1:41:54, 16.39s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6058852108663815, 'learning_rate': 2.3167701863354036e-07, 'completion_length': 154.8928680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.07695359364151955, 'kl': 0.015625, 'epoch': 3.84}

 77%|███████▋  | 1237/1610 [6:53:40<1:41:54, 16.39s/it]
 77%|███████▋  | 1238/1610 [6:53:52<1:33:55, 15.15s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 1.2721647137687748, 'learning_rate': 2.31055900621118e-07, 'completion_length': 124.57143783569336, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571492433548, 'kl': 0.011138916015625, 'epoch': 3.84}

 77%|███████▋  | 1238/1610 [6:53:52<1:33:55, 15.15s/it]
 77%|███████▋  | 1239/1610 [6:54:08<1:33:59, 15.20s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5320817812375276, 'learning_rate': 2.3043478260869565e-07, 'completion_length': 163.6607208251953, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.07695359364151955, 'kl': 0.014678955078125, 'epoch': 3.85}

 77%|███████▋  | 1239/1610 [6:54:08<1:33:59, 15.20s/it]
 77%|███████▋  | 1240/1610 [6:54:24<1:35:37, 15.51s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8056964270410985, 'learning_rate': 2.2981366459627326e-07, 'completion_length': 155.42858123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428619325161, 'kl': 0.014678955078125, 'epoch': 3.85}

 77%|███████▋  | 1240/1610 [6:54:24<1:35:37, 15.51s/it]
 77%|███████▋  | 1241/1610 [6:54:38<1:32:38, 15.06s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 1.059310057016069, 'learning_rate': 2.2919254658385092e-07, 'completion_length': 135.125, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.0224609375, 'epoch': 3.85}

 77%|███████▋  | 1241/1610 [6:54:38<1:32:38, 15.06s/it]
 77%|███████▋  | 1242/1610 [6:54:55<1:35:21, 15.55s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8182050794648559, 'learning_rate': 2.2857142857142855e-07, 'completion_length': 164.37500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.01800537109375, 'epoch': 3.86}

 77%|███████▋  | 1242/1610 [6:54:55<1:35:21, 15.55s/it]
 77%|███████▋  | 1243/1610 [6:55:12<1:38:44, 16.14s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.9309486649859802, 'learning_rate': 2.279503105590062e-07, 'completion_length': 179.2321548461914, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.18409645557403564, 'kl': 0.01629638671875, 'epoch': 3.86}

 77%|███████▋  | 1243/1610 [6:55:12<1:38:44, 16.14s/it]
 77%|███████▋  | 1244/1610 [6:55:28<1:38:21, 16.12s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5842308260321292, 'learning_rate': 2.2732919254658384e-07, 'completion_length': 156.94644165039062, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.01751708984375, 'epoch': 3.86}

 77%|███████▋  | 1244/1610 [6:55:28<1:38:21, 16.12s/it]
 77%|███████▋  | 1245/1610 [6:55:46<1:41:15, 16.64s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.0431297633102319, 'learning_rate': 2.267080745341615e-07, 'completion_length': 178.87500762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214783191681, 'kl': 0.012603759765625, 'epoch': 3.87}

 77%|███████▋  | 1245/1610 [6:55:46<1:41:15, 16.64s/it]
 77%|███████▋  | 1246/1610 [6:56:02<1:39:56, 16.47s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1941326017386678, 'learning_rate': 2.260869565217391e-07, 'completion_length': 143.55357360839844, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071798324585, 'kl': 0.01898193359375, 'epoch': 3.87}

 77%|███████▋  | 1246/1610 [6:56:02<1:39:56, 16.47s/it]
 77%|███████▋  | 1247/1610 [6:56:19<1:39:51, 16.51s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.1474779473854049, 'learning_rate': 2.2546583850931674e-07, 'completion_length': 167.39286041259766, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.26657506823539734, 'kl': 0.016204833984375, 'epoch': 3.87}

 77%|███████▋  | 1247/1610 [6:56:19<1:39:51, 16.51s/it]
 78%|███████▊  | 1248/1610 [6:56:37<1:42:15, 16.95s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.531736632747303, 'learning_rate': 2.248447204968944e-07, 'completion_length': 176.92857360839844, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838217198848724, 'kl': 0.015228271484375, 'epoch': 3.88}

 78%|███████▊  | 1248/1610 [6:56:37<1:42:15, 16.95s/it]
 78%|███████▊  | 1249/1610 [6:56:52<1:38:51, 16.43s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2215183828745289, 'learning_rate': 2.2422360248447203e-07, 'completion_length': 156.33928680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.11266788095235825, 'kl': 0.0184326171875, 'epoch': 3.88}

 78%|███████▊  | 1249/1610 [6:56:52<1:38:51, 16.43s/it]
 78%|███████▊  | 1250/1610 [6:57:08<1:38:02, 16.34s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.381412597165688, 'learning_rate': 2.236024844720497e-07, 'completion_length': 178.5357208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.2610500454902649, 'kl': 0.016326904296875, 'epoch': 3.88}

 78%|███████▊  | 1250/1610 [6:57:08<1:38:02, 16.34s/it]
 78%|███████▊  | 1251/1610 [6:57:22<1:33:55, 15.70s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.9230541272440482, 'learning_rate': 2.2298136645962732e-07, 'completion_length': 165.6607208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.15943220257759094, 'kl': 0.014739990234375, 'epoch': 3.89}

 78%|███████▊  | 1251/1610 [6:57:22<1:33:55, 15.70s/it]
 78%|███████▊  | 1252/1610 [6:57:38<1:33:47, 15.72s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7087416857680711, 'learning_rate': 2.2236024844720495e-07, 'completion_length': 154.8035774230957, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1181928962469101, 'kl': 0.016021728515625, 'epoch': 3.89}

 78%|███████▊  | 1252/1610 [6:57:38<1:33:47, 15.72s/it]
 78%|███████▊  | 1253/1610 [6:57:59<1:42:42, 17.26s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.113470971206561, 'learning_rate': 2.217391304347826e-07, 'completion_length': 202.23214721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.696428656578064, 'reward_std': 0.25248410552740097, 'kl': 0.016387939453125, 'epoch': 3.89}

 78%|███████▊  | 1253/1610 [6:57:59<1:42:42, 17.26s/it]
 78%|███████▊  | 1254/1610 [6:58:16<1:42:51, 17.34s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.3686742438855964, 'learning_rate': 2.2111801242236025e-07, 'completion_length': 162.51786041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.09403547644615173, 'kl': 0.02099609375, 'epoch': 3.89}

 78%|███████▊  | 1254/1610 [6:58:16<1:42:51, 17.34s/it]
 78%|███████▊  | 1255/1610 [6:58:31<1:37:08, 16.42s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5835378488102794, 'learning_rate': 2.2049689440993788e-07, 'completion_length': 147.96429061889648, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.013824462890625, 'epoch': 3.9}

 78%|███████▊  | 1255/1610 [6:58:31<1:37:08, 16.42s/it]
 78%|███████▊  | 1256/1610 [6:58:45<1:33:21, 15.82s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.9580839485226808, 'learning_rate': 2.198757763975155e-07, 'completion_length': 134.50000381469727, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.009613037109375, 'epoch': 3.9}

 78%|███████▊  | 1256/1610 [6:58:45<1:33:21, 15.82s/it]
 78%|███████▊  | 1257/1610 [6:59:01<1:33:31, 15.90s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.356104521463355, 'learning_rate': 2.1925465838509317e-07, 'completion_length': 154.9464340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.21981073170900345, 'kl': 0.014007568359375, 'epoch': 3.9}

 78%|███████▊  | 1257/1610 [6:59:01<1:33:31, 15.90s/it]
 78%|███████▊  | 1258/1610 [6:59:17<1:33:38, 15.96s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.3785785914397877, 'learning_rate': 2.1863354037267078e-07, 'completion_length': 159.9107208251953, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695358991622925, 'kl': 0.01141357421875, 'epoch': 3.91}

 78%|███████▊  | 1258/1610 [6:59:17<1:33:38, 15.96s/it]
 78%|███████▊  | 1259/1610 [6:59:33<1:33:25, 15.97s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.2009704123005103, 'learning_rate': 2.1801242236024844e-07, 'completion_length': 164.01786041259766, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214671432972, 'kl': 0.0185546875, 'epoch': 3.91}

 78%|███████▊  | 1259/1610 [6:59:33<1:33:25, 15.97s/it]
 78%|███████▊  | 1260/1610 [6:59:48<1:31:23, 15.67s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.6499120314458555, 'learning_rate': 2.1739130434782607e-07, 'completion_length': 150.9464340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1428571492433548, 'kl': 0.020111083984375, 'epoch': 3.91}

 78%|███████▊  | 1260/1610 [6:59:48<1:31:23, 15.67s/it]
 78%|███████▊  | 1261/1610 [7:00:03<1:29:33, 15.40s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.363217335366361, 'learning_rate': 2.1677018633540373e-07, 'completion_length': 143.69644165039062, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2253357619047165, 'kl': 0.02056884765625, 'epoch': 3.92}

 78%|███████▊  | 1261/1610 [7:00:03<1:29:33, 15.40s/it]
 78%|███████▊  | 1262/1610 [7:00:20<1:32:38, 15.97s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.28080240916971844, 'learning_rate': 2.1614906832298136e-07, 'completion_length': 157.8571548461914, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.07695359364151955, 'kl': 0.012969970703125, 'epoch': 3.92}

 78%|███████▊  | 1262/1610 [7:00:20<1:32:38, 15.97s/it]
 78%|███████▊  | 1263/1610 [7:00:36<1:32:08, 15.93s/it]
                                                       
{'loss': 0.001, 'grad_norm': 2.072670920364949, 'learning_rate': 2.1552795031055902e-07, 'completion_length': 161.625, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981074661016464, 'kl': 0.0247802734375, 'epoch': 3.92}

 78%|███████▊  | 1263/1610 [7:00:36<1:32:08, 15.93s/it]
 79%|███████▊  | 1264/1610 [7:00:52<1:32:27, 16.03s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2295992683008896, 'learning_rate': 2.1490683229813662e-07, 'completion_length': 157.67857360839844, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409644439816475, 'kl': 0.02008056640625, 'epoch': 3.93}

 79%|███████▊  | 1264/1610 [7:00:52<1:32:27, 16.03s/it]
 79%|███████▊  | 1265/1610 [7:01:12<1:37:41, 16.99s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.352770180539853, 'learning_rate': 2.1428571428571426e-07, 'completion_length': 185.21429443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.571428656578064, 'reward_std': 0.28365693986415863, 'kl': 0.01654052734375, 'epoch': 3.93}

 79%|███████▊  | 1265/1610 [7:01:12<1:37:41, 16.99s/it]
 79%|███████▊  | 1266/1610 [7:01:28<1:35:29, 16.65s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8820135498162741, 'learning_rate': 2.1366459627329192e-07, 'completion_length': 158.83929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.19514649361371994, 'kl': 0.01751708984375, 'epoch': 3.93}

 79%|███████▊  | 1266/1610 [7:01:28<1:35:29, 16.65s/it]
 79%|███████▊  | 1267/1610 [7:01:44<1:34:05, 16.46s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.204480856333399, 'learning_rate': 2.1304347826086955e-07, 'completion_length': 158.57143783569336, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428572535514832, 'reward_std': 0.2253357544541359, 'kl': 0.0201416015625, 'epoch': 3.93}

 79%|███████▊  | 1267/1610 [7:01:44<1:34:05, 16.46s/it]
 79%|███████▉  | 1268/1610 [7:02:01<1:34:56, 16.66s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8753722363488358, 'learning_rate': 2.124223602484472e-07, 'completion_length': 171.78572845458984, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086077719926834, 'kl': 0.02044677734375, 'epoch': 3.94}

 79%|███████▉  | 1268/1610 [7:02:01<1:34:56, 16.66s/it]
 79%|███████▉  | 1269/1610 [7:02:14<1:29:42, 15.78s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.079001720687131, 'learning_rate': 2.1180124223602484e-07, 'completion_length': 146.50000762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.30228935182094574, 'kl': 0.01513671875, 'epoch': 3.94}

 79%|███████▉  | 1269/1610 [7:02:14<1:29:42, 15.78s/it]
 79%|███████▉  | 1270/1610 [7:02:28<1:26:17, 15.23s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 2.5082222138044, 'learning_rate': 2.1118012422360247e-07, 'completion_length': 141.9821548461914, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.18409644067287445, 'kl': 0.01715087890625, 'epoch': 3.94}

 79%|███████▉  | 1270/1610 [7:02:28<1:26:17, 15.23s/it]
 79%|███████▉  | 1271/1610 [7:02:44<1:27:23, 15.47s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.6785590434209228, 'learning_rate': 2.105590062111801e-07, 'completion_length': 172.37500762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.014129638671875, 'epoch': 3.95}

 79%|███████▉  | 1271/1610 [7:02:44<1:27:23, 15.47s/it]
 79%|███████▉  | 1272/1610 [7:02:59<1:25:14, 15.13s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.580692410848952, 'learning_rate': 2.0993788819875776e-07, 'completion_length': 158.75000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.01531982421875, 'epoch': 3.95}

 79%|███████▉  | 1272/1610 [7:02:59<1:25:14, 15.13s/it]
 79%|███████▉  | 1273/1610 [7:03:18<1:31:23, 16.27s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0107620129605288, 'learning_rate': 2.093167701863354e-07, 'completion_length': 185.39286041259766, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178571939468384, 'reward_std': 0.29123931378126144, 'kl': 0.014404296875, 'epoch': 3.95}

 79%|███████▉  | 1273/1610 [7:03:18<1:31:23, 16.27s/it]
 79%|███████▉  | 1274/1610 [7:03:35<1:33:04, 16.62s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5614880750473055, 'learning_rate': 2.0869565217391303e-07, 'completion_length': 201.1607208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.01416015625, 'epoch': 3.96}

 79%|███████▉  | 1274/1610 [7:03:35<1:33:04, 16.62s/it]
 79%|███████▉  | 1275/1610 [7:03:50<1:30:43, 16.25s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.3374293554784078, 'learning_rate': 2.080745341614907e-07, 'completion_length': 153.30358123779297, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.015594482421875, 'epoch': 3.96}

 79%|███████▉  | 1275/1610 [7:03:50<1:30:43, 16.25s/it]
 79%|███████▉  | 1276/1610 [7:04:08<1:33:18, 16.76s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.1961628914786944, 'learning_rate': 2.074534161490683e-07, 'completion_length': 159.58929443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838217198848724, 'kl': 0.013336181640625, 'epoch': 3.96}

 79%|███████▉  | 1276/1610 [7:04:08<1:33:18, 16.76s/it]
 79%|███████▉  | 1277/1610 [7:04:23<1:29:49, 16.18s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8589416346603325, 'learning_rate': 2.0683229813664595e-07, 'completion_length': 141.7678680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071798324585, 'kl': 0.016845703125, 'epoch': 3.97}

 79%|███████▉  | 1277/1610 [7:04:23<1:29:49, 16.18s/it]
 79%|███████▉  | 1278/1610 [7:04:39<1:28:50, 16.06s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3648574594244363, 'learning_rate': 2.0621118012422359e-07, 'completion_length': 147.25000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25552502274513245, 'kl': 0.0167236328125, 'epoch': 3.97}

 79%|███████▉  | 1278/1610 [7:04:39<1:28:50, 16.06s/it]
 79%|███████▉  | 1279/1610 [7:04:54<1:26:51, 15.75s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9549406898063318, 'learning_rate': 2.0559006211180125e-07, 'completion_length': 142.03571701049805, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.18409645557403564, 'kl': 0.017822265625, 'epoch': 3.97}

 79%|███████▉  | 1279/1610 [7:04:54<1:26:51, 15.75s/it]
 80%|███████▉  | 1280/1610 [7:05:11<1:28:48, 16.15s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7046257131815391, 'learning_rate': 2.0496894409937888e-07, 'completion_length': 155.25000762939453, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.15943220257759094, 'kl': 0.0185546875, 'epoch': 3.98}

 80%|███████▉  | 1280/1610 [7:05:11<1:28:48, 16.15s/it]
 80%|███████▉  | 1281/1610 [7:05:25<1:24:56, 15.49s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.117047256385777, 'learning_rate': 2.0434782608695654e-07, 'completion_length': 147.5357208251953, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2363857924938202, 'kl': 0.01953125, 'epoch': 3.98}

 80%|███████▉  | 1281/1610 [7:05:25<1:24:56, 15.49s/it]
 80%|███████▉  | 1282/1610 [7:05:40<1:23:22, 15.25s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0043624556582527, 'learning_rate': 2.0372670807453414e-07, 'completion_length': 152.87500762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266787722706795, 'kl': 0.015777587890625, 'epoch': 3.98}

 80%|███████▉  | 1282/1610 [7:05:40<1:23:22, 15.25s/it]
 80%|███████▉  | 1283/1610 [7:05:57<1:26:17, 15.83s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.3233333795334754, 'learning_rate': 2.0310559006211178e-07, 'completion_length': 162.42858123779297, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.0714285746216774, 'kl': 0.013824462890625, 'epoch': 3.98}

 80%|███████▉  | 1283/1610 [7:05:57<1:26:17, 15.83s/it]
 80%|███████▉  | 1284/1610 [7:06:16<1:31:07, 16.77s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0941191318841934, 'learning_rate': 2.0248447204968943e-07, 'completion_length': 177.2678680419922, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.0177001953125, 'epoch': 3.99}

 80%|███████▉  | 1284/1610 [7:06:16<1:31:07, 16.77s/it]
 80%|███████▉  | 1285/1610 [7:06:34<1:33:22, 17.24s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.2654317812701956, 'learning_rate': 2.0186335403726707e-07, 'completion_length': 157.5714340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.21981073170900345, 'kl': 0.02099609375, 'epoch': 3.99}

 80%|███████▉  | 1285/1610 [7:06:34<1:33:22, 17.24s/it]
 80%|███████▉  | 1286/1610 [7:06:51<1:32:54, 17.21s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6567915613658599, 'learning_rate': 2.0124223602484473e-07, 'completion_length': 175.5178680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.1539071872830391, 'kl': 0.01373291015625, 'epoch': 3.99}

 80%|███████▉  | 1286/1610 [7:06:51<1:32:54, 17.21s/it]
 80%|███████▉  | 1287/1610 [7:07:05<1:27:17, 16.21s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 2.1076549670109883, 'learning_rate': 2.0062111801242236e-07, 'completion_length': 145.69643020629883, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1896214783191681, 'kl': 0.012176513671875, 'epoch': 4.0}

 80%|███████▉  | 1287/1610 [7:07:05<1:27:17, 16.21s/it]
 80%|████████  | 1288/1610 [7:07:19<1:22:34, 15.39s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.38732034117683733, 'learning_rate': 2e-07, 'completion_length': 129.12500381469727, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.0107421875, 'epoch': 4.0}

 80%|████████  | 1288/1610 [7:07:19<1:22:34, 15.39s/it]
 80%|████████  | 1289/1610 [7:07:34<1:21:52, 15.30s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9518420641905052, 'learning_rate': 1.9937888198757762e-07, 'completion_length': 155.80358123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.19514648616313934, 'kl': 0.0196533203125, 'epoch': 4.0}

 80%|████████  | 1289/1610 [7:07:34<1:21:52, 15.30s/it]
 80%|████████  | 1290/1610 [7:07:52<1:26:16, 16.18s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.4496202826441442, 'learning_rate': 1.9875776397515526e-07, 'completion_length': 144.33929443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0714285746216774, 'kl': 0.015380859375, 'epoch': 4.01}

 80%|████████  | 1290/1610 [7:07:52<1:26:16, 16.18s/it]
 80%|████████  | 1291/1610 [7:08:08<1:26:08, 16.20s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7468209173033051, 'learning_rate': 1.9813664596273292e-07, 'completion_length': 149.14286422729492, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0824786126613617, 'kl': 0.01666259765625, 'epoch': 4.01}

 80%|████████  | 1291/1610 [7:08:08<1:26:08, 16.20s/it]
 80%|████████  | 1292/1610 [7:08:25<1:26:36, 16.34s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.033946100531895, 'learning_rate': 1.9751552795031055e-07, 'completion_length': 148.26786041259766, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514650478959084, 'kl': 0.0201416015625, 'epoch': 4.01}

 80%|████████  | 1292/1610 [7:08:25<1:26:36, 16.34s/it]
 80%|████████  | 1293/1610 [7:08:40<1:24:29, 15.99s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5063239934418624, 'learning_rate': 1.968944099378882e-07, 'completion_length': 156.9107208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428619325161, 'kl': 0.01483154296875, 'epoch': 4.02}

 80%|████████  | 1293/1610 [7:08:40<1:24:29, 15.99s/it]
 80%|████████  | 1294/1610 [7:08:54<1:21:20, 15.44s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.6534567481804956, 'learning_rate': 1.962732919254658e-07, 'completion_length': 128.21429443359375, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.3324785977602005, 'kl': 0.0184326171875, 'epoch': 4.02}

 80%|████████  | 1294/1610 [7:08:54<1:21:20, 15.44s/it]
 80%|████████  | 1295/1610 [7:09:09<1:19:13, 15.09s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.1692270159420364, 'learning_rate': 1.9565217391304347e-07, 'completion_length': 148.3214340209961, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2253357619047165, 'kl': 0.016876220703125, 'epoch': 4.02}

 80%|████████  | 1295/1610 [7:09:09<1:19:13, 15.09s/it]
 80%|████████  | 1296/1610 [7:09:25<1:20:54, 15.46s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0711393272169982, 'learning_rate': 1.950310559006211e-07, 'completion_length': 175.30358123779297, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2857142984867096, 'kl': 0.01806640625, 'epoch': 4.02}

 80%|████████  | 1296/1610 [7:09:25<1:20:54, 15.46s/it]
 81%|████████  | 1297/1610 [7:09:44<1:25:50, 16.46s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7648449137338728, 'learning_rate': 1.9440993788819876e-07, 'completion_length': 185.51786041259766, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428619325161, 'kl': 0.01397705078125, 'epoch': 4.03}

 81%|████████  | 1297/1610 [7:09:44<1:25:50, 16.46s/it]
 81%|████████  | 1298/1610 [7:09:58<1:21:50, 15.74s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.3235545267082782, 'learning_rate': 1.937888198757764e-07, 'completion_length': 149.05357360839844, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357619047165, 'kl': 0.01617431640625, 'epoch': 4.03}

 81%|████████  | 1298/1610 [7:09:58<1:21:50, 15.74s/it]
 81%|████████  | 1299/1610 [7:10:15<1:23:06, 16.03s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5153972552445263, 'learning_rate': 1.9316770186335403e-07, 'completion_length': 160.75000762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0824786126613617, 'kl': 0.015106201171875, 'epoch': 4.03}

 81%|████████  | 1299/1610 [7:10:15<1:23:06, 16.03s/it]
 81%|████████  | 1300/1610 [7:10:32<1:24:28, 16.35s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 7.363179583914277, 'learning_rate': 1.9254658385093166e-07, 'completion_length': 181.6607208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838217198848724, 'kl': 0.01385498046875, 'epoch': 4.04}

 81%|████████  | 1300/1610 [7:10:32<1:24:28, 16.35s/it]
 81%|████████  | 1301/1610 [7:14:56<7:47:38, 90.81s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.645875210956192, 'learning_rate': 1.919254658385093e-07, 'completion_length': 126.75000762939453, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1071428656578064, 'kl': 0.0151214599609375, 'epoch': 4.04}

 81%|████████  | 1301/1610 [7:14:56<7:47:38, 90.81s/it]
 81%|████████  | 1302/1610 [7:15:11<5:49:48, 68.15s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.7453190224691705, 'learning_rate': 1.9130434782608695e-07, 'completion_length': 144.3214340209961, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1181928962469101, 'kl': 0.0211181640625, 'epoch': 4.04}

 81%|████████  | 1302/1610 [7:15:11<5:49:48, 68.15s/it]
 81%|████████  | 1303/1610 [7:15:31<4:34:08, 53.58s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5262146976028763, 'learning_rate': 1.9068322981366459e-07, 'completion_length': 166.33929443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.9285714626312256, 'reward_std': 0.1428571492433548, 'kl': 0.014617919921875, 'epoch': 4.05}

 81%|████████  | 1303/1610 [7:15:31<4:34:08, 53.58s/it]
 81%|████████  | 1304/1610 [7:15:43<3:29:35, 41.10s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 3.8632487428322597, 'learning_rate': 1.9006211180124224e-07, 'completion_length': 133.4107208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.19514650478959084, 'kl': 0.01995849609375, 'epoch': 4.05}

 81%|████████  | 1304/1610 [7:15:43<3:29:35, 41.10s/it]
 81%|████████  | 1305/1610 [7:15:59<2:50:21, 33.51s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.1688568561083037, 'learning_rate': 1.8944099378881988e-07, 'completion_length': 176.25000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2967643439769745, 'kl': 0.0167236328125, 'epoch': 4.05}

 81%|████████  | 1305/1610 [7:15:59<2:50:21, 33.51s/it]
 81%|████████  | 1306/1610 [7:16:15<2:23:09, 28.25s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7122957937251455, 'learning_rate': 1.888198757763975e-07, 'completion_length': 151.1964340209961, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.015380859375, 'epoch': 4.06}

 81%|████████  | 1306/1610 [7:16:15<2:23:09, 28.25s/it]
 81%|████████  | 1307/1610 [7:16:35<2:09:55, 25.73s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.509412590506953, 'learning_rate': 1.8819875776397514e-07, 'completion_length': 184.83929443359375, 'rewards/accuracy_reward': 0.7321428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7142857909202576, 'reward_std': 0.2253357544541359, 'kl': 0.020263671875, 'epoch': 4.06}

 81%|████████  | 1307/1610 [7:16:35<2:09:55, 25.73s/it]
 81%|████████  | 1308/1610 [7:16:47<1:49:57, 21.85s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.0232926017559163, 'learning_rate': 1.8757763975155277e-07, 'completion_length': 136.91071701049805, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.1649572253227234, 'kl': 0.013458251953125, 'epoch': 4.06}

 81%|████████  | 1308/1610 [7:16:47<1:49:57, 21.85s/it]
 81%|████████▏ | 1309/1610 [7:17:04<1:41:54, 20.31s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 1.6633192767455909, 'learning_rate': 1.8695652173913043e-07, 'completion_length': 153.9464340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0714285746216774, 'kl': 0.0100250244140625, 'epoch': 4.07}

 81%|████████▏ | 1309/1610 [7:17:04<1:41:54, 20.31s/it]
 81%|████████▏ | 1310/1610 [7:17:22<1:38:30, 19.70s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 3.320303131223371, 'learning_rate': 1.8633540372670807e-07, 'completion_length': 172.83928680419922, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1785714365541935, 'kl': 0.017578125, 'epoch': 4.07}

 81%|████████▏ | 1310/1610 [7:17:22<1:38:30, 19.70s/it]
 81%|████████▏ | 1311/1610 [7:17:39<1:33:35, 18.78s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7918423990392894, 'learning_rate': 1.8571428571428572e-07, 'completion_length': 186.89286041259766, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.01617431640625, 'epoch': 4.07}

 81%|████████▏ | 1311/1610 [7:17:39<1:33:35, 18.78s/it]
 81%|████████▏ | 1312/1610 [7:17:53<1:26:45, 17.47s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8891899916626588, 'learning_rate': 1.8509316770186333e-07, 'completion_length': 138.42857360839844, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.15943220257759094, 'kl': 0.015777587890625, 'epoch': 4.07}

 81%|████████▏ | 1312/1610 [7:17:53<1:26:45, 17.47s/it]
 82%|████████▏ | 1313/1610 [7:18:10<1:24:32, 17.08s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.187143248882045, 'learning_rate': 1.84472049689441e-07, 'completion_length': 193.96429443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.016204833984375, 'epoch': 4.08}

 82%|████████▏ | 1313/1610 [7:18:10<1:24:32, 17.08s/it]
 82%|████████▏ | 1314/1610 [7:18:26<1:22:34, 16.74s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.4136318718177617, 'learning_rate': 1.8385093167701862e-07, 'completion_length': 153.3571548461914, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0714285746216774, 'kl': 0.0133056640625, 'epoch': 4.08}

 82%|████████▏ | 1314/1610 [7:18:26<1:22:34, 16.74s/it]
 82%|████████▏ | 1315/1610 [7:18:41<1:20:40, 16.41s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.318187797085334, 'learning_rate': 1.8322981366459628e-07, 'completion_length': 152.62500762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.0140380859375, 'epoch': 4.08}

 82%|████████▏ | 1315/1610 [7:18:41<1:20:40, 16.41s/it]
 82%|████████▏ | 1316/1610 [7:18:59<1:21:49, 16.70s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.2868936540148537, 'learning_rate': 1.8260869565217391e-07, 'completion_length': 162.4464340209961, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.0125885009765625, 'epoch': 4.09}

 82%|████████▏ | 1316/1610 [7:18:59<1:21:49, 16.70s/it]
 82%|████████▏ | 1317/1610 [7:19:13<1:17:35, 15.89s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1504127637826695, 'learning_rate': 1.8198757763975152e-07, 'completion_length': 152.10714721679688, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.26657506823539734, 'kl': 0.01922607421875, 'epoch': 4.09}

 82%|████████▏ | 1317/1610 [7:19:13<1:17:35, 15.89s/it]
 82%|████████▏ | 1318/1610 [7:19:30<1:19:43, 16.38s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.007378449069922, 'learning_rate': 1.8136645962732918e-07, 'completion_length': 168.35714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2967643290758133, 'kl': 0.016204833984375, 'epoch': 4.09}

 82%|████████▏ | 1318/1610 [7:19:30<1:19:43, 16.38s/it]
 82%|████████▏ | 1319/1610 [7:19:49<1:23:23, 17.19s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6820931653893191, 'learning_rate': 1.807453416149068e-07, 'completion_length': 171.2857208251953, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.821428656578064, 'reward_std': 0.1428571529686451, 'kl': 0.0166015625, 'epoch': 4.1}

 82%|████████▏ | 1319/1610 [7:19:49<1:23:23, 17.19s/it]
 82%|████████▏ | 1320/1610 [7:20:03<1:17:53, 16.12s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.5130495217639953, 'learning_rate': 1.8012422360248447e-07, 'completion_length': 146.9464340209961, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1539071872830391, 'kl': 0.011322021484375, 'epoch': 4.1}

 82%|████████▏ | 1320/1610 [7:20:03<1:17:53, 16.12s/it]
 82%|████████▏ | 1321/1610 [7:20:20<1:19:37, 16.53s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0053316342844236, 'learning_rate': 1.795031055900621e-07, 'completion_length': 164.1607208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6250001192092896, 'reward_std': 0.21124479919672012, 'kl': 0.0159912109375, 'epoch': 4.1}

 82%|████████▏ | 1321/1610 [7:20:20<1:19:37, 16.53s/it]
 82%|████████▏ | 1322/1610 [7:20:35<1:17:01, 16.05s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.45830168507077307, 'learning_rate': 1.7888198757763976e-07, 'completion_length': 166.2678680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266787722706795, 'kl': 0.0123291015625, 'epoch': 4.11}

 82%|████████▏ | 1322/1610 [7:20:35<1:17:01, 16.05s/it]
 82%|████████▏ | 1323/1610 [7:20:48<1:12:24, 15.14s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.712538510089845, 'learning_rate': 1.7826086956521737e-07, 'completion_length': 129.62500762939453, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086076974868774, 'kl': 0.0162353515625, 'epoch': 4.11}

 82%|████████▏ | 1323/1610 [7:20:48<1:12:24, 15.14s/it]
 82%|████████▏ | 1324/1610 [7:21:04<1:12:34, 15.23s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.47401454160435513, 'learning_rate': 1.7763975155279503e-07, 'completion_length': 173.21429443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.1181928999722004, 'kl': 0.012908935546875, 'epoch': 4.11}

 82%|████████▏ | 1324/1610 [7:21:04<1:12:34, 15.23s/it]
 82%|████████▏ | 1325/1610 [7:21:20<1:13:19, 15.44s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7872011579545215, 'learning_rate': 1.7701863354037266e-07, 'completion_length': 153.33928680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1181928962469101, 'kl': 0.015533447265625, 'epoch': 4.11}

 82%|████████▏ | 1325/1610 [7:21:20<1:13:19, 15.44s/it]
 82%|████████▏ | 1326/1610 [7:21:32<1:09:23, 14.66s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9595367812327525, 'learning_rate': 1.763975155279503e-07, 'completion_length': 139.5357208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.26657505333423615, 'kl': 0.01715087890625, 'epoch': 4.12}

 82%|████████▏ | 1326/1610 [7:21:32<1:09:23, 14.66s/it]
 82%|████████▏ | 1327/1610 [7:21:49<1:11:45, 15.21s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6967394434772755, 'learning_rate': 1.7577639751552795e-07, 'completion_length': 154.87500762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.01275634765625, 'epoch': 4.12}

 82%|████████▏ | 1327/1610 [7:21:49<1:11:45, 15.21s/it]
 82%|████████▏ | 1328/1610 [7:22:08<1:16:33, 16.29s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.47180757531477624, 'learning_rate': 1.7515527950310558e-07, 'completion_length': 169.0357208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.0714285746216774, 'kl': 0.0169677734375, 'epoch': 4.12}

 82%|████████▏ | 1328/1610 [7:22:08<1:16:33, 16.29s/it]
 83%|████████▎ | 1329/1610 [7:22:21<1:11:47, 15.33s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7267853657533943, 'learning_rate': 1.7453416149068322e-07, 'completion_length': 139.62500381469727, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1428571492433548, 'kl': 0.0185546875, 'epoch': 4.13}

 83%|████████▎ | 1329/1610 [7:22:21<1:11:47, 15.33s/it]
 83%|████████▎ | 1330/1610 [7:22:40<1:17:01, 16.50s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0127549687270971, 'learning_rate': 1.7391304347826085e-07, 'completion_length': 186.67858123779297, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.821428656578064, 'reward_std': 0.24695908278226852, 'kl': 0.0159912109375, 'epoch': 4.13}

 83%|████████▎ | 1330/1610 [7:22:40<1:17:01, 16.50s/it]
 83%|████████▎ | 1331/1610 [7:22:58<1:18:30, 16.88s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6605549019168703, 'learning_rate': 1.732919254658385e-07, 'completion_length': 172.89286041259766, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7678572535514832, 'reward_std': 0.1785714402794838, 'kl': 0.01507568359375, 'epoch': 4.13}

 83%|████████▎ | 1331/1610 [7:22:58<1:18:30, 16.88s/it]
 83%|████████▎ | 1332/1610 [7:23:16<1:19:23, 17.14s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.0659988285910138, 'learning_rate': 1.7267080745341614e-07, 'completion_length': 189.17858123779297, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.14838216826319695, 'kl': 0.01995849609375, 'epoch': 4.14}

 83%|████████▎ | 1332/1610 [7:23:16<1:19:23, 17.14s/it]
 83%|████████▎ | 1333/1610 [7:23:32<1:17:47, 16.85s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.41877385105549647, 'learning_rate': 1.720496894409938e-07, 'completion_length': 155.00000762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.017822265625, 'epoch': 4.14}

 83%|████████▎ | 1333/1610 [7:23:32<1:17:47, 16.85s/it]
 83%|████████▎ | 1334/1610 [7:23:48<1:16:49, 16.70s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.9359862266991186, 'learning_rate': 1.7142857142857143e-07, 'completion_length': 170.17858123779297, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2363857924938202, 'kl': 0.013580322265625, 'epoch': 4.14}

 83%|████████▎ | 1334/1610 [7:23:48<1:16:49, 16.70s/it]
 83%|████████▎ | 1335/1610 [7:23:59<1:09:10, 15.09s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1421575934939026, 'learning_rate': 1.7080745341614904e-07, 'completion_length': 128.01786422729492, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.01953125, 'epoch': 4.15}

 83%|████████▎ | 1335/1610 [7:23:59<1:09:10, 15.09s/it]
 83%|████████▎ | 1336/1610 [7:24:15<1:09:23, 15.20s/it]
                                                       
{'loss': 0.0009, 'grad_norm': 0.8081099813396825, 'learning_rate': 1.701863354037267e-07, 'completion_length': 160.5357208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18409644067287445, 'kl': 0.0223388671875, 'epoch': 4.15}

 83%|████████▎ | 1336/1610 [7:24:15<1:09:23, 15.20s/it]
 83%|████████▎ | 1337/1610 [7:24:31<1:10:47, 15.56s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6804855832163638, 'learning_rate': 1.6956521739130433e-07, 'completion_length': 181.05358123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1785714365541935, 'kl': 0.018707275390625, 'epoch': 4.15}

 83%|████████▎ | 1337/1610 [7:24:31<1:10:47, 15.56s/it]
 83%|████████▎ | 1338/1610 [7:24:46<1:09:17, 15.29s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6876936611326935, 'learning_rate': 1.68944099378882e-07, 'completion_length': 168.53572463989258, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.017608642578125, 'epoch': 4.16}

 83%|████████▎ | 1338/1610 [7:24:46<1:09:17, 15.29s/it]
 83%|████████▎ | 1339/1610 [7:25:02<1:10:13, 15.55s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5598651755101602, 'learning_rate': 1.6832298136645962e-07, 'completion_length': 160.9821548461914, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.015838623046875, 'epoch': 4.16}

 83%|████████▎ | 1339/1610 [7:25:02<1:10:13, 15.55s/it]
 83%|████████▎ | 1340/1610 [7:25:15<1:05:46, 14.62s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.1558525166296565, 'learning_rate': 1.6770186335403728e-07, 'completion_length': 128.48215103149414, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1785714402794838, 'kl': 0.01922607421875, 'epoch': 4.16}

 83%|████████▎ | 1340/1610 [7:25:15<1:05:46, 14.62s/it]
 83%|████████▎ | 1341/1610 [7:25:30<1:06:03, 14.73s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.49708109638672154, 'learning_rate': 1.6708074534161489e-07, 'completion_length': 158.85714721679688, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.07695359364151955, 'kl': 0.017486572265625, 'epoch': 4.16}

 83%|████████▎ | 1341/1610 [7:25:30<1:06:03, 14.73s/it]
 83%|████████▎ | 1342/1610 [7:25:45<1:06:40, 14.93s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5702878359511547, 'learning_rate': 1.6645962732919252e-07, 'completion_length': 167.33929443359375, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.14838216453790665, 'kl': 0.016845703125, 'epoch': 4.17}

 83%|████████▎ | 1342/1610 [7:25:45<1:06:40, 14.93s/it]
 83%|████████▎ | 1343/1610 [7:26:00<1:06:59, 15.05s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.9363117143042201, 'learning_rate': 1.6583850931677018e-07, 'completion_length': 166.1964340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2610500380396843, 'kl': 0.02044677734375, 'epoch': 4.17}

 83%|████████▎ | 1343/1610 [7:26:00<1:06:59, 15.05s/it]
 83%|████████▎ | 1344/1610 [7:26:15<1:06:18, 14.96s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8747274923767134, 'learning_rate': 1.652173913043478e-07, 'completion_length': 139.67858123779297, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.013763427734375, 'epoch': 4.17}

 83%|████████▎ | 1344/1610 [7:26:15<1:06:18, 14.96s/it]
 84%|████████▎ | 1345/1610 [7:26:31<1:07:18, 15.24s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0821336379174058, 'learning_rate': 1.6459627329192547e-07, 'completion_length': 176.6071548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.19514648616313934, 'kl': 0.017333984375, 'epoch': 4.18}

 84%|████████▎ | 1345/1610 [7:26:31<1:07:18, 15.24s/it]
 84%|████████▎ | 1346/1610 [7:26:46<1:06:23, 15.09s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3536294165682181, 'learning_rate': 1.639751552795031e-07, 'completion_length': 152.1964340209961, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357470035553, 'kl': 0.01715087890625, 'epoch': 4.18}

 84%|████████▎ | 1346/1610 [7:26:46<1:06:23, 15.09s/it]
 84%|████████▎ | 1347/1610 [7:27:04<1:09:51, 15.94s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.6254537262904776, 'learning_rate': 1.6335403726708073e-07, 'completion_length': 146.92857360839844, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428619325161, 'kl': 0.011993408203125, 'epoch': 4.18}

 84%|████████▎ | 1347/1610 [7:27:04<1:09:51, 15.94s/it]
 84%|████████▎ | 1348/1610 [7:27:21<1:12:00, 16.49s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 2.7903816716901773, 'learning_rate': 1.6273291925465837e-07, 'completion_length': 172.6428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1785714402794838, 'kl': 0.014739990234375, 'epoch': 4.19}

 84%|████████▎ | 1348/1610 [7:27:21<1:12:00, 16.49s/it]
 84%|████████▍ | 1349/1610 [7:27:37<1:10:46, 16.27s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.5896282562607545, 'learning_rate': 1.6211180124223603e-07, 'completion_length': 167.10714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2006715089082718, 'kl': 0.01544189453125, 'epoch': 4.19}

 84%|████████▍ | 1349/1610 [7:27:37<1:10:46, 16.27s/it]
 84%|████████▍ | 1350/1610 [7:27:51<1:08:00, 15.69s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 1.0250260259997472, 'learning_rate': 1.6149068322981366e-07, 'completion_length': 135.8035774230957, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.01324462890625, 'epoch': 4.19}

 84%|████████▍ | 1350/1610 [7:27:51<1:08:00, 15.69s/it]
 84%|████████▍ | 1351/1610 [7:28:05<1:04:41, 14.99s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 0.5209083266868366, 'learning_rate': 1.608695652173913e-07, 'completion_length': 128.21429443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1181928999722004, 'kl': 0.00970458984375, 'epoch': 4.2}

 84%|████████▍ | 1351/1610 [7:28:05<1:04:41, 14.99s/it]
 84%|████████▍ | 1352/1610 [7:28:18<1:02:09, 14.45s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0515397104871647, 'learning_rate': 1.6024844720496895e-07, 'completion_length': 128.30357360839844, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.0181884765625, 'epoch': 4.2}

 84%|████████▍ | 1352/1610 [7:28:18<1:02:09, 14.45s/it]
 84%|████████▍ | 1353/1610 [7:28:34<1:04:02, 14.95s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6539220028961957, 'learning_rate': 1.5962732919254656e-07, 'completion_length': 185.9821548461914, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1181928999722004, 'kl': 0.01666259765625, 'epoch': 4.2}

 84%|████████▍ | 1353/1610 [7:28:34<1:04:02, 14.95s/it]
 84%|████████▍ | 1354/1610 [7:28:50<1:04:23, 15.09s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.8234877944546424, 'learning_rate': 1.5900621118012422e-07, 'completion_length': 148.2321548461914, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714365541935, 'kl': 0.0174560546875, 'epoch': 4.2}

 84%|████████▍ | 1354/1610 [7:28:50<1:04:23, 15.09s/it]
 84%|████████▍ | 1355/1610 [7:29:04<1:02:50, 14.79s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.349401353300587, 'learning_rate': 1.5838509316770185e-07, 'completion_length': 151.17857360839844, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1539071835577488, 'kl': 0.01953125, 'epoch': 4.21}

 84%|████████▍ | 1355/1610 [7:29:04<1:02:50, 14.79s/it]
 84%|████████▍ | 1356/1610 [7:29:21<1:05:17, 15.43s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 1.111677186386952, 'learning_rate': 1.577639751552795e-07, 'completion_length': 197.01786041259766, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23086076974868774, 'kl': 0.020751953125, 'epoch': 4.21}

 84%|████████▍ | 1356/1610 [7:29:21<1:05:17, 15.43s/it]
 84%|████████▍ | 1357/1610 [7:29:39<1:08:22, 16.21s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8688342552570357, 'learning_rate': 1.5714285714285714e-07, 'completion_length': 198.73214721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2253357544541359, 'kl': 0.014404296875, 'epoch': 4.21}

 84%|████████▍ | 1357/1610 [7:29:39<1:08:22, 16.21s/it]
 84%|████████▍ | 1358/1610 [7:29:51<1:03:13, 15.06s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.2063850036054704, 'learning_rate': 1.565217391304348e-07, 'completion_length': 125.55357360839844, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.2610500454902649, 'kl': 0.015380859375, 'epoch': 4.22}

 84%|████████▍ | 1358/1610 [7:29:51<1:03:13, 15.06s/it]
 84%|████████▍ | 1359/1610 [7:30:07<1:03:41, 15.22s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7473545646196743, 'learning_rate': 1.559006211180124e-07, 'completion_length': 183.69644165039062, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.016876220703125, 'epoch': 4.22}

 84%|████████▍ | 1359/1610 [7:30:07<1:03:41, 15.22s/it]
 84%|████████▍ | 1360/1610 [7:30:19<59:31, 14.29s/it]  
                                                     
{'loss': 0.0007, 'grad_norm': 0.9731284642260439, 'learning_rate': 1.5527950310559004e-07, 'completion_length': 118.50000381469727, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1896214671432972, 'kl': 0.017181396484375, 'epoch': 4.22}

 84%|████████▍ | 1360/1610 [7:30:19<59:31, 14.29s/it]
 85%|████████▍ | 1361/1610 [7:30:31<56:40, 13.66s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.061918172645857, 'learning_rate': 1.546583850931677e-07, 'completion_length': 124.12500762939453, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.0194091796875, 'epoch': 4.23}

 85%|████████▍ | 1361/1610 [7:30:31<56:40, 13.66s/it]
 85%|████████▍ | 1362/1610 [7:30:45<56:47, 13.74s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.8845318285610428, 'learning_rate': 1.5403726708074533e-07, 'completion_length': 145.33929443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3324785977602005, 'kl': 0.01715087890625, 'epoch': 4.23}

 85%|████████▍ | 1362/1610 [7:30:45<56:47, 13.74s/it]
 85%|████████▍ | 1363/1610 [7:31:02<1:00:37, 14.73s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.197053146329875, 'learning_rate': 1.53416149068323e-07, 'completion_length': 168.67858123779297, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571529686451, 'kl': 0.01702880859375, 'epoch': 4.23}

 85%|████████▍ | 1363/1610 [7:31:02<1:00:37, 14.73s/it]
 85%|████████▍ | 1364/1610 [7:31:15<58:29, 14.27s/it]  
                                                     
{'loss': 0.0006, 'grad_norm': 0.8347391872379555, 'learning_rate': 1.5279503105590062e-07, 'completion_length': 141.30358123779297, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.11266788095235825, 'kl': 0.015960693359375, 'epoch': 4.24}

 85%|████████▍ | 1364/1610 [7:31:15<58:29, 14.27s/it]
 85%|████████▍ | 1365/1610 [7:31:29<57:48, 14.16s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.6960702376561667, 'learning_rate': 1.5217391304347825e-07, 'completion_length': 150.17857360839844, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.013397216796875, 'epoch': 4.24}

 85%|████████▍ | 1365/1610 [7:31:29<57:48, 14.16s/it]
 85%|████████▍ | 1366/1610 [7:31:42<56:10, 13.82s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 2.3196837994342787, 'learning_rate': 1.5155279503105589e-07, 'completion_length': 142.48214721679688, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.01702880859375, 'epoch': 4.24}

 85%|████████▍ | 1366/1610 [7:31:42<56:10, 13.82s/it]
 85%|████████▍ | 1367/1610 [7:32:00<1:01:08, 15.10s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0177834655449185, 'learning_rate': 1.5093167701863354e-07, 'completion_length': 179.10714721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.307814359664917, 'kl': 0.0181884765625, 'epoch': 4.25}

 85%|████████▍ | 1367/1610 [7:32:00<1:01:08, 15.10s/it]
 85%|████████▍ | 1368/1610 [7:32:13<58:37, 14.53s/it]  
                                                     
{'loss': 0.0006, 'grad_norm': 0.9036129518219604, 'learning_rate': 1.5031055900621118e-07, 'completion_length': 135.62500381469727, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.014373779296875, 'epoch': 4.25}

 85%|████████▍ | 1368/1610 [7:32:13<58:37, 14.53s/it]
 85%|████████▌ | 1369/1610 [7:32:30<1:00:47, 15.13s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.3719867658894265, 'learning_rate': 1.496894409937888e-07, 'completion_length': 175.9107208251953, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981074661016464, 'kl': 0.01611328125, 'epoch': 4.25}

 85%|████████▌ | 1369/1610 [7:32:30<1:00:47, 15.13s/it]
 85%|████████▌ | 1370/1610 [7:32:45<1:01:17, 15.32s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 2.7901232166583205, 'learning_rate': 1.4906832298136647e-07, 'completion_length': 160.85714721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.1539071798324585, 'kl': 0.0201416015625, 'epoch': 4.25}

 85%|████████▌ | 1370/1610 [7:32:45<1:01:17, 15.32s/it]
 85%|████████▌ | 1371/1610 [7:33:03<1:03:44, 16.00s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8667213517548934, 'learning_rate': 1.4844720496894407e-07, 'completion_length': 176.17858123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.19514648616313934, 'kl': 0.018798828125, 'epoch': 4.26}

 85%|████████▌ | 1371/1610 [7:33:03<1:03:44, 16.00s/it]
 85%|████████▌ | 1372/1610 [7:33:20<1:04:06, 16.16s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9008373415358729, 'learning_rate': 1.4782608695652173e-07, 'completion_length': 149.1964340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1428571529686451, 'kl': 0.01806640625, 'epoch': 4.26}

 85%|████████▌ | 1372/1610 [7:33:20<1:04:06, 16.16s/it]
 85%|████████▌ | 1373/1610 [7:33:36<1:04:09, 16.24s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.3794738891415543, 'learning_rate': 1.4720496894409937e-07, 'completion_length': 160.64286041259766, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.0174560546875, 'epoch': 4.26}

 85%|████████▌ | 1373/1610 [7:33:36<1:04:09, 16.24s/it]
 85%|████████▌ | 1374/1610 [7:33:52<1:03:44, 16.20s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.6949821758085625, 'learning_rate': 1.4658385093167703e-07, 'completion_length': 139.83929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.016693115234375, 'epoch': 4.27}

 85%|████████▌ | 1374/1610 [7:33:52<1:03:44, 16.20s/it]
 85%|████████▌ | 1375/1610 [7:34:08<1:02:30, 15.96s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9438857304198522, 'learning_rate': 1.4596273291925466e-07, 'completion_length': 170.12500762939453, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21981074661016464, 'kl': 0.01397705078125, 'epoch': 4.27}

 85%|████████▌ | 1375/1610 [7:34:08<1:02:30, 15.96s/it]
 85%|████████▌ | 1376/1610 [7:34:21<59:48, 15.33s/it]  
                                                     
{'loss': 0.0006, 'grad_norm': 0.9115933642943408, 'learning_rate': 1.4534161490683232e-07, 'completion_length': 136.10714721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.015838623046875, 'epoch': 4.27}

 85%|████████▌ | 1376/1610 [7:34:21<59:48, 15.33s/it]
 86%|████████▌ | 1377/1610 [7:34:38<1:01:05, 15.73s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.509050530729081, 'learning_rate': 1.4472049689440992e-07, 'completion_length': 172.58929443359375, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.014312744140625, 'epoch': 4.28}

 86%|████████▌ | 1377/1610 [7:34:38<1:01:05, 15.73s/it]
 86%|████████▌ | 1378/1610 [7:34:53<59:38, 15.43s/it]  
                                                     
{'loss': 0.0006, 'grad_norm': 1.1882582434138032, 'learning_rate': 1.4409937888198756e-07, 'completion_length': 138.67858123779297, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2363857924938202, 'kl': 0.01556396484375, 'epoch': 4.28}

 86%|████████▌ | 1378/1610 [7:34:53<59:38, 15.43s/it]
 86%|████████▌ | 1379/1610 [7:35:09<1:00:04, 15.60s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 1.0352642116406654, 'learning_rate': 1.4347826086956521e-07, 'completion_length': 173.92857360839844, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216453790665, 'kl': 0.01739501953125, 'epoch': 4.28}

 86%|████████▌ | 1379/1610 [7:35:09<1:00:04, 15.60s/it]
 86%|████████▌ | 1380/1610 [7:35:24<59:51, 15.61s/it]  
                                                     
{'loss': 0.0007, 'grad_norm': 0.9070929425780564, 'learning_rate': 1.4285714285714285e-07, 'completion_length': 179.17858123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409644439816475, 'kl': 0.017364501953125, 'epoch': 4.29}

 86%|████████▌ | 1380/1610 [7:35:24<59:51, 15.61s/it]
 86%|████████▌ | 1381/1610 [7:35:40<59:04, 15.48s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.39651578529313414, 'learning_rate': 1.422360248447205e-07, 'completion_length': 144.37500762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.07695359364151955, 'kl': 0.01470947265625, 'epoch': 4.29}

 86%|████████▌ | 1381/1610 [7:35:40<59:04, 15.48s/it]
 86%|████████▌ | 1382/1610 [7:35:55<58:14, 15.33s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 2.2202048734876487, 'learning_rate': 1.4161490683229814e-07, 'completion_length': 166.62500762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214671432972, 'kl': 0.0155029296875, 'epoch': 4.29}

 86%|████████▌ | 1382/1610 [7:35:55<58:14, 15.33s/it]
 86%|████████▌ | 1383/1610 [7:36:09<56:51, 15.03s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 3.455048087233868, 'learning_rate': 1.4099378881987577e-07, 'completion_length': 155.96429443359375, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1181928962469101, 'kl': 0.014892578125, 'epoch': 4.3}

 86%|████████▌ | 1383/1610 [7:36:09<56:51, 15.03s/it]
 86%|████████▌ | 1384/1610 [7:36:26<58:45, 15.60s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9345119835570864, 'learning_rate': 1.403726708074534e-07, 'completion_length': 164.67858123779297, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.1539071798324585, 'kl': 0.01702880859375, 'epoch': 4.3}

 86%|████████▌ | 1384/1610 [7:36:26<58:45, 15.60s/it]
 86%|████████▌ | 1385/1610 [7:36:42<58:47, 15.68s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.5783500150892051, 'learning_rate': 1.3975155279503104e-07, 'completion_length': 166.69644165039062, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657505333423615, 'kl': 0.0186767578125, 'epoch': 4.3}

 86%|████████▌ | 1385/1610 [7:36:42<58:47, 15.68s/it]
 86%|████████▌ | 1386/1610 [7:36:57<58:10, 15.58s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 0.5358587668115999, 'learning_rate': 1.391304347826087e-07, 'completion_length': 140.5714340209961, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.0714285746216774, 'kl': 0.011199951171875, 'epoch': 4.3}

 86%|████████▌ | 1386/1610 [7:36:57<58:10, 15.58s/it]
 86%|████████▌ | 1387/1610 [7:37:12<57:24, 15.45s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.7894720649826791, 'learning_rate': 1.3850931677018633e-07, 'completion_length': 151.57144165039062, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1071428656578064, 'kl': 0.0150146484375, 'epoch': 4.31}

 86%|████████▌ | 1387/1610 [7:37:12<57:24, 15.45s/it]
 86%|████████▌ | 1388/1610 [7:37:32<1:01:57, 16.74s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8918828263082613, 'learning_rate': 1.3788819875776399e-07, 'completion_length': 184.76786041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.2142857313156128, 'kl': 0.01544189453125, 'epoch': 4.31}

 86%|████████▌ | 1388/1610 [7:37:32<1:01:57, 16.74s/it]
 86%|████████▋ | 1389/1610 [7:37:48<1:00:34, 16.45s/it]
                                                       
{'loss': 0.0008, 'grad_norm': 0.8436196465354038, 'learning_rate': 1.372670807453416e-07, 'completion_length': 160.4464340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1896214671432972, 'kl': 0.02056884765625, 'epoch': 4.31}

 86%|████████▋ | 1389/1610 [7:37:48<1:00:34, 16.45s/it]
 86%|████████▋ | 1390/1610 [7:38:05<1:01:38, 16.81s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.8150220503823511, 'learning_rate': 1.3664596273291925e-07, 'completion_length': 167.71429443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.21981073915958405, 'kl': 0.017425537109375, 'epoch': 4.32}

 86%|████████▋ | 1390/1610 [7:38:05<1:01:38, 16.81s/it]
 86%|████████▋ | 1391/1610 [7:38:19<57:53, 15.86s/it]  
                                                     
{'loss': 0.0007, 'grad_norm': 0.7676914687732972, 'learning_rate': 1.3602484472049688e-07, 'completion_length': 134.4821548461914, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.018096923828125, 'epoch': 4.32}

 86%|████████▋ | 1391/1610 [7:38:19<57:53, 15.86s/it]
 86%|████████▋ | 1392/1610 [7:38:35<57:25, 15.81s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8200377562949566, 'learning_rate': 1.3540372670807454e-07, 'completion_length': 166.82144165039062, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1896214783191681, 'kl': 0.017822265625, 'epoch': 4.32}

 86%|████████▋ | 1392/1610 [7:38:35<57:25, 15.81s/it]
 87%|████████▋ | 1393/1610 [7:38:49<55:13, 15.27s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6062835512017966, 'learning_rate': 1.3478260869565218e-07, 'completion_length': 143.01786041259766, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.016571044921875, 'epoch': 4.33}

 87%|████████▋ | 1393/1610 [7:38:49<55:13, 15.27s/it]
 87%|████████▋ | 1394/1610 [7:39:04<55:09, 15.32s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8499299999926015, 'learning_rate': 1.3416149068322978e-07, 'completion_length': 152.64286041259766, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.21981072798371315, 'kl': 0.014739990234375, 'epoch': 4.33}

 87%|████████▋ | 1394/1610 [7:39:04<55:09, 15.32s/it]
 87%|████████▋ | 1395/1610 [7:39:18<53:15, 14.86s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.484389998952048, 'learning_rate': 1.3354037267080744e-07, 'completion_length': 143.5357208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.30228935927152634, 'kl': 0.017608642578125, 'epoch': 4.33}

 87%|████████▋ | 1395/1610 [7:39:18<53:15, 14.86s/it]
 87%|████████▋ | 1396/1610 [7:39:33<53:22, 14.96s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6714737897589267, 'learning_rate': 1.3291925465838507e-07, 'completion_length': 176.42858123779297, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.1428571492433548, 'kl': 0.016265869140625, 'epoch': 4.34}

 87%|████████▋ | 1396/1610 [7:39:33<53:22, 14.96s/it]
 87%|████████▋ | 1397/1610 [7:39:49<54:11, 15.27s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 2.0866409396994405, 'learning_rate': 1.3229813664596273e-07, 'completion_length': 162.33929443359375, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1896214708685875, 'kl': 0.0203857421875, 'epoch': 4.34}

 87%|████████▋ | 1397/1610 [7:39:49<54:11, 15.27s/it]
 87%|████████▋ | 1398/1610 [7:40:07<56:52, 16.10s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.716290142229105, 'learning_rate': 1.3167701863354037e-07, 'completion_length': 192.2321548461914, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4464285969734192, 'reward_std': 0.30228933691978455, 'kl': 0.0205078125, 'epoch': 4.34}

 87%|████████▋ | 1398/1610 [7:40:07<56:52, 16.10s/it]
 87%|████████▋ | 1399/1610 [7:40:23<56:48, 16.15s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.31133865318150566, 'learning_rate': 1.3105590062111802e-07, 'completion_length': 137.08929443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.0357142873108387, 'kl': 0.016510009765625, 'epoch': 4.34}

 87%|████████▋ | 1399/1610 [7:40:23<56:48, 16.15s/it]
 87%|████████▋ | 1400/1610 [7:40:38<54:33, 15.59s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.7264169671050261, 'learning_rate': 1.3043478260869563e-07, 'completion_length': 144.2678680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428619325161, 'kl': 0.02081298828125, 'epoch': 4.35}

 87%|████████▋ | 1400/1610 [7:40:38<54:33, 15.59s/it]
 87%|████████▋ | 1401/1610 [7:44:13<4:23:14, 75.57s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.6000403277817072, 'learning_rate': 1.298136645962733e-07, 'completion_length': 164.7857208251953, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2721000909805298, 'kl': 0.015869140625, 'epoch': 4.35}

 87%|████████▋ | 1401/1610 [7:44:13<4:23:14, 75.57s/it]
 87%|████████▋ | 1402/1610 [7:44:29<3:19:30, 57.55s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.9479346258339979, 'learning_rate': 1.2919254658385092e-07, 'completion_length': 157.17857360839844, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.14838216453790665, 'kl': 0.016754150390625, 'epoch': 4.35}

 87%|████████▋ | 1402/1610 [7:44:29<3:19:30, 57.55s/it]
 87%|████████▋ | 1403/1610 [7:44:45<2:35:49, 45.16s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.5917699022988252, 'learning_rate': 1.2857142857142855e-07, 'completion_length': 176.58929443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.1539071798324585, 'kl': 0.01678466796875, 'epoch': 4.36}

 87%|████████▋ | 1403/1610 [7:44:45<2:35:49, 45.16s/it]
 87%|████████▋ | 1404/1610 [7:44:58<2:01:31, 35.40s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.3380116109650195, 'learning_rate': 1.2795031055900621e-07, 'completion_length': 119.75000381469727, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.0357142873108387, 'kl': 0.015655517578125, 'epoch': 4.36}

 87%|████████▋ | 1404/1610 [7:44:58<2:01:31, 35.40s/it]
 87%|████████▋ | 1405/1610 [7:45:14<1:41:03, 29.58s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.6562837400335281, 'learning_rate': 1.2732919254658385e-07, 'completion_length': 156.32144165039062, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.014556884765625, 'epoch': 4.36}

 87%|████████▋ | 1405/1610 [7:45:14<1:41:03, 29.58s/it]
 87%|████████▋ | 1406/1610 [7:45:34<1:31:01, 26.77s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.9677716750251666, 'learning_rate': 1.2670807453416148e-07, 'completion_length': 169.3214340209961, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.6607143878936768, 'reward_std': 0.23086077719926834, 'kl': 0.01409912109375, 'epoch': 4.37}

 87%|████████▋ | 1406/1610 [7:45:34<1:31:01, 26.77s/it]
 87%|████████▋ | 1407/1610 [7:45:51<1:20:48, 23.89s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8412284478960265, 'learning_rate': 1.260869565217391e-07, 'completion_length': 167.6428680419922, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1181928962469101, 'kl': 0.015350341796875, 'epoch': 4.37}

 87%|████████▋ | 1407/1610 [7:45:51<1:20:48, 23.89s/it]
 87%|████████▋ | 1408/1610 [7:46:05<1:10:48, 21.03s/it]
                                                       
{'loss': 0.0005, 'grad_norm': 0.03697851544861726, 'learning_rate': 1.2546583850931677e-07, 'completion_length': 154.05358123779297, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.0, 'kl': 0.0133056640625, 'epoch': 4.37}

 87%|████████▋ | 1408/1610 [7:46:05<1:10:48, 21.03s/it]
 88%|████████▊ | 1409/1610 [7:46:20<1:03:32, 18.97s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.8252006950150014, 'learning_rate': 1.248447204968944e-07, 'completion_length': 142.62500762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.11266788095235825, 'kl': 0.01531982421875, 'epoch': 4.38}

 88%|████████▊ | 1409/1610 [7:46:20<1:03:32, 18.97s/it]
 88%|████████▊ | 1410/1610 [7:46:38<1:02:47, 18.84s/it]
                                                       
{'loss': 0.0004, 'grad_norm': 1.2762592287901424, 'learning_rate': 1.2422360248447204e-07, 'completion_length': 178.37500762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571529686451, 'kl': 0.010711669921875, 'epoch': 4.38}

 88%|████████▊ | 1410/1610 [7:46:38<1:02:47, 18.84s/it]
 88%|████████▊ | 1411/1610 [7:46:56<1:01:58, 18.69s/it]
                                                       
{'loss': 0.0007, 'grad_norm': 0.7423983548043552, 'learning_rate': 1.236024844720497e-07, 'completion_length': 170.9107208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11266787722706795, 'kl': 0.016510009765625, 'epoch': 4.38}

 88%|████████▊ | 1411/1610 [7:46:56<1:01:58, 18.69s/it]
 88%|████████▊ | 1412/1610 [7:47:13<59:07, 17.91s/it]  
                                                     
{'loss': 0.0006, 'grad_norm': 0.445512925081674, 'learning_rate': 1.2298136645962733e-07, 'completion_length': 142.75000762939453, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.0152587890625, 'epoch': 4.39}

 88%|████████▊ | 1412/1610 [7:47:13<59:07, 17.91s/it]
 88%|████████▊ | 1413/1610 [7:47:31<59:17, 18.06s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5788508638933226, 'learning_rate': 1.2236024844720496e-07, 'completion_length': 182.33929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1539071835577488, 'kl': 0.015869140625, 'epoch': 4.39}

 88%|████████▊ | 1413/1610 [7:47:31<59:17, 18.06s/it]
 88%|████████▊ | 1414/1610 [7:47:46<55:43, 17.06s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8572718255566415, 'learning_rate': 1.2173913043478262e-07, 'completion_length': 142.01786041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0357142873108387, 'kl': 0.01751708984375, 'epoch': 4.39}

 88%|████████▊ | 1414/1610 [7:47:46<55:43, 17.06s/it]
 88%|████████▊ | 1415/1610 [7:48:01<54:15, 16.70s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.6486269772374993, 'learning_rate': 1.2111801242236025e-07, 'completion_length': 179.0714340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.26657506823539734, 'kl': 0.02020263671875, 'epoch': 4.39}

 88%|████████▊ | 1415/1610 [7:48:01<54:15, 16.70s/it]
 88%|████████▊ | 1416/1610 [7:48:19<54:51, 16.97s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6179632191717508, 'learning_rate': 1.2049689440993788e-07, 'completion_length': 172.0357208251953, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.1071428619325161, 'kl': 0.0145263671875, 'epoch': 4.4}

 88%|████████▊ | 1416/1610 [7:48:19<54:51, 16.97s/it]
 88%|████████▊ | 1417/1610 [7:48:35<53:41, 16.69s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6509674343215177, 'learning_rate': 1.1987577639751552e-07, 'completion_length': 150.33929443359375, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.0714285746216774, 'kl': 0.01727294921875, 'epoch': 4.4}

 88%|████████▊ | 1417/1610 [7:48:35<53:41, 16.69s/it]
 88%|████████▊ | 1418/1610 [7:48:53<54:50, 17.14s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.2846141866936263, 'learning_rate': 1.1925465838509315e-07, 'completion_length': 183.2678680419922, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25552502274513245, 'kl': 0.01422119140625, 'epoch': 4.4}

 88%|████████▊ | 1418/1610 [7:48:53<54:50, 17.14s/it]
 88%|████████▊ | 1419/1610 [7:49:12<55:40, 17.49s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.1119308328527642, 'learning_rate': 1.1863354037267081e-07, 'completion_length': 156.3214340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.15943220630288124, 'kl': 0.01544189453125, 'epoch': 4.41}

 88%|████████▊ | 1419/1610 [7:49:12<55:40, 17.49s/it]
 88%|████████▊ | 1420/1610 [7:49:25<51:51, 16.37s/it]
                                                     
{'loss': 0.001, 'grad_norm': 1.1357440399934982, 'learning_rate': 1.1801242236024844e-07, 'completion_length': 146.1607208251953, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21981074661016464, 'kl': 0.0244140625, 'epoch': 4.41}

 88%|████████▊ | 1420/1610 [7:49:25<51:51, 16.37s/it]
 88%|████████▊ | 1421/1610 [7:49:40<49:58, 15.87s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6596513394087998, 'learning_rate': 1.1739130434782609e-07, 'completion_length': 151.2678680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.0714285746216774, 'kl': 0.01812744140625, 'epoch': 4.41}

 88%|████████▊ | 1421/1610 [7:49:40<49:58, 15.87s/it]
 88%|████████▊ | 1422/1610 [7:49:56<49:46, 15.89s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8270965903689225, 'learning_rate': 1.1677018633540373e-07, 'completion_length': 165.8928680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1181928962469101, 'kl': 0.014862060546875, 'epoch': 4.42}

 88%|████████▊ | 1422/1610 [7:49:56<49:46, 15.89s/it]
 88%|████████▊ | 1423/1610 [7:50:11<49:04, 15.75s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5688410221500653, 'learning_rate': 1.1614906832298136e-07, 'completion_length': 148.98214721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.014251708984375, 'epoch': 4.42}

 88%|████████▊ | 1423/1610 [7:50:11<49:04, 15.75s/it]
 88%|████████▊ | 1424/1610 [7:50:30<51:22, 16.57s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.92431531435234, 'learning_rate': 1.15527950310559e-07, 'completion_length': 164.50000762939453, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.01898193359375, 'epoch': 4.42}

 88%|████████▊ | 1424/1610 [7:50:30<51:22, 16.57s/it]
 89%|████████▊ | 1425/1610 [7:50:43<48:05, 15.60s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.4124767462401246, 'learning_rate': 1.1490683229813663e-07, 'completion_length': 130.41071701049805, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2142857313156128, 'kl': 0.01580810546875, 'epoch': 4.43}

 89%|████████▊ | 1425/1610 [7:50:43<48:05, 15.60s/it]
 89%|████████▊ | 1426/1610 [7:50:58<47:04, 15.35s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.4455034856169373, 'learning_rate': 1.1428571428571427e-07, 'completion_length': 141.64286422729492, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.26657506078481674, 'kl': 0.01953125, 'epoch': 4.43}

 89%|████████▊ | 1426/1610 [7:50:58<47:04, 15.35s/it]
 89%|████████▊ | 1427/1610 [7:51:12<45:32, 14.93s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 3.7595179072695206, 'learning_rate': 1.1366459627329192e-07, 'completion_length': 148.67857360839844, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.1539071798324585, 'kl': 0.015411376953125, 'epoch': 4.43}

 89%|████████▊ | 1427/1610 [7:51:12<45:32, 14.93s/it]
 89%|████████▊ | 1428/1610 [7:51:28<46:08, 15.21s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.4099748975943729, 'learning_rate': 1.1304347826086955e-07, 'completion_length': 146.39286041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428656578064, 'kl': 0.01446533203125, 'epoch': 4.43}

 89%|████████▊ | 1428/1610 [7:51:28<46:08, 15.21s/it]
 89%|████████▉ | 1429/1610 [7:51:44<46:53, 15.54s/it]
                                                     
{'loss': 0.001, 'grad_norm': 0.7433868417772754, 'learning_rate': 1.124223602484472e-07, 'completion_length': 168.58929443359375, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.02459716796875, 'epoch': 4.44}

 89%|████████▉ | 1429/1610 [7:51:44<46:53, 15.54s/it]
 89%|████████▉ | 1430/1610 [7:52:00<46:46, 15.59s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.010851674984859, 'learning_rate': 1.1180124223602484e-07, 'completion_length': 164.1071548461914, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535715222358704, 'reward_std': 0.2610500529408455, 'kl': 0.0174560546875, 'epoch': 4.44}

 89%|████████▉ | 1430/1610 [7:52:00<46:46, 15.59s/it]
 89%|████████▉ | 1431/1610 [7:52:16<47:11, 15.82s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.9389530492253573, 'learning_rate': 1.1118012422360248e-07, 'completion_length': 165.62500762939453, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1896214634180069, 'kl': 0.016693115234375, 'epoch': 4.44}

 89%|████████▉ | 1431/1610 [7:52:16<47:11, 15.82s/it]
 89%|████████▉ | 1432/1610 [7:52:33<48:04, 16.21s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.4486608376011154, 'learning_rate': 1.1055900621118012e-07, 'completion_length': 159.8571548461914, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2142857313156128, 'kl': 0.01873779296875, 'epoch': 4.45}

 89%|████████▉ | 1432/1610 [7:52:33<48:04, 16.21s/it]
 89%|████████▉ | 1433/1610 [7:52:51<49:22, 16.74s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8322911082757575, 'learning_rate': 1.0993788819875776e-07, 'completion_length': 155.14286041259766, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.14838217198848724, 'kl': 0.01690673828125, 'epoch': 4.45}

 89%|████████▉ | 1433/1610 [7:52:51<49:22, 16.74s/it]
 89%|████████▉ | 1434/1610 [7:53:09<49:55, 17.02s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8643633037727271, 'learning_rate': 1.0931677018633539e-07, 'completion_length': 171.83928680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.18409644439816475, 'kl': 0.0177001953125, 'epoch': 4.45}

 89%|████████▉ | 1434/1610 [7:53:09<49:55, 17.02s/it]
 89%|████████▉ | 1435/1610 [7:53:26<49:23, 16.94s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.7416129515909917, 'learning_rate': 1.0869565217391303e-07, 'completion_length': 160.5178680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.14838216826319695, 'kl': 0.0179443359375, 'epoch': 4.46}

 89%|████████▉ | 1435/1610 [7:53:26<49:23, 16.94s/it]
 89%|████████▉ | 1436/1610 [7:53:42<48:46, 16.82s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.5727932008228209, 'learning_rate': 1.0807453416149068e-07, 'completion_length': 169.60714721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.013153076171875, 'epoch': 4.46}

 89%|████████▉ | 1436/1610 [7:53:42<48:46, 16.82s/it]
 89%|████████▉ | 1437/1610 [7:53:57<46:23, 16.09s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.1361645610883249, 'learning_rate': 1.0745341614906831e-07, 'completion_length': 153.98214721679688, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.015228271484375, 'epoch': 4.46}

 89%|████████▉ | 1437/1610 [7:53:57<46:23, 16.09s/it]
 89%|████████▉ | 1438/1610 [7:54:12<45:44, 15.95s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.849676407731665, 'learning_rate': 1.0683229813664596e-07, 'completion_length': 146.5357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.14838216826319695, 'kl': 0.01593017578125, 'epoch': 4.47}

 89%|████████▉ | 1438/1610 [7:54:12<45:44, 15.95s/it]
 89%|████████▉ | 1439/1610 [7:54:25<42:59, 15.09s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.5556130810793487, 'learning_rate': 1.062111801242236e-07, 'completion_length': 131.85714721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.0211181640625, 'epoch': 4.47}

 89%|████████▉ | 1439/1610 [7:54:25<42:59, 15.09s/it]
 89%|████████▉ | 1440/1610 [7:54:41<42:50, 15.12s/it]
                                                     
{'loss': 0.0009, 'grad_norm': 1.1883000225477847, 'learning_rate': 1.0559006211180124e-07, 'completion_length': 144.73214721679688, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.3324786275625229, 'kl': 0.0234375, 'epoch': 4.47}

 89%|████████▉ | 1440/1610 [7:54:41<42:50, 15.12s/it]
 90%|████████▉ | 1441/1610 [7:54:57<43:52, 15.57s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.46626883265883384, 'learning_rate': 1.0496894409937888e-07, 'completion_length': 182.85714721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.0714285746216774, 'kl': 0.0126953125, 'epoch': 4.48}

 90%|████████▉ | 1441/1610 [7:54:57<43:52, 15.57s/it]
 90%|████████▉ | 1442/1610 [7:55:13<44:12, 15.79s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.185196746159043, 'learning_rate': 1.0434782608695651e-07, 'completion_length': 145.8571548461914, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.01519775390625, 'epoch': 4.48}

 90%|████████▉ | 1442/1610 [7:55:13<44:12, 15.79s/it]
 90%|████████▉ | 1443/1610 [7:55:29<43:26, 15.61s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.827041126646708, 'learning_rate': 1.0372670807453415e-07, 'completion_length': 160.96429061889648, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.0135498046875, 'epoch': 4.48}

 90%|████████▉ | 1443/1610 [7:55:29<43:26, 15.61s/it]
 90%|████████▉ | 1444/1610 [7:55:42<41:12, 14.90s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.4539164486525387, 'learning_rate': 1.0310559006211179e-07, 'completion_length': 149.89286041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.2610500380396843, 'kl': 0.016693115234375, 'epoch': 4.48}

 90%|████████▉ | 1444/1610 [7:55:42<41:12, 14.90s/it]
 90%|████████▉ | 1445/1610 [7:55:56<40:29, 14.72s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.3153378358570365, 'learning_rate': 1.0248447204968944e-07, 'completion_length': 134.19643783569336, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.0191650390625, 'epoch': 4.49}

 90%|████████▉ | 1445/1610 [7:55:56<40:29, 14.72s/it]
 90%|████████▉ | 1446/1610 [7:56:11<40:18, 14.75s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.8036916709672874, 'learning_rate': 1.0186335403726707e-07, 'completion_length': 157.33929443359375, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.02008056640625, 'epoch': 4.49}

 90%|████████▉ | 1446/1610 [7:56:11<40:18, 14.75s/it]
 90%|████████▉ | 1447/1610 [7:56:26<40:21, 14.86s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.504035971541114, 'learning_rate': 1.0124223602484472e-07, 'completion_length': 141.33929443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.017578125, 'epoch': 4.49}

 90%|████████▉ | 1447/1610 [7:56:26<40:21, 14.86s/it]
 90%|████████▉ | 1448/1610 [7:56:40<39:34, 14.66s/it]
                                                     
{'loss': 0.0009, 'grad_norm': 0.7124679255566938, 'learning_rate': 1.0062111801242236e-07, 'completion_length': 130.5357208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.19514648616313934, 'kl': 0.0235595703125, 'epoch': 4.5}

 90%|████████▉ | 1448/1610 [7:56:40<39:34, 14.66s/it]
 90%|█████████ | 1449/1610 [7:56:59<42:27, 15.82s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.592772547839188, 'learning_rate': 1e-07, 'completion_length': 151.23214721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.01519775390625, 'epoch': 4.5}

 90%|█████████ | 1449/1610 [7:56:59<42:27, 15.82s/it]
 90%|█████████ | 1450/1610 [7:57:16<43:18, 16.24s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.782299101244497, 'learning_rate': 9.937888198757763e-08, 'completion_length': 165.08929443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.07695358991622925, 'kl': 0.0146484375, 'epoch': 4.5}

 90%|█████████ | 1450/1610 [7:57:16<43:18, 16.24s/it]
 90%|█████████ | 1451/1610 [7:57:34<44:04, 16.63s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 4.8995264236097755, 'learning_rate': 9.875776397515527e-08, 'completion_length': 169.6428680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6250000596046448, 'reward_std': 0.2610500454902649, 'kl': 0.013519287109375, 'epoch': 4.51}

 90%|█████████ | 1451/1610 [7:57:34<44:04, 16.63s/it]
 90%|█████████ | 1452/1610 [7:57:47<41:31, 15.77s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.7873180272388043, 'learning_rate': 9.81366459627329e-08, 'completion_length': 168.1964340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.11266787722706795, 'kl': 0.014739990234375, 'epoch': 4.51}

 90%|█████████ | 1452/1610 [7:57:47<41:31, 15.77s/it]
 90%|█████████ | 1453/1610 [7:58:03<41:23, 15.82s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5908409286902172, 'learning_rate': 9.751552795031055e-08, 'completion_length': 172.6428680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.1539071835577488, 'kl': 0.01519775390625, 'epoch': 4.51}

 90%|█████████ | 1453/1610 [7:58:03<41:23, 15.82s/it]
 90%|█████████ | 1454/1610 [7:58:20<41:26, 15.94s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.7448901662147734, 'learning_rate': 9.68944099378882e-08, 'completion_length': 169.58929443359375, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.18409645557403564, 'kl': 0.01312255859375, 'epoch': 4.52}

 90%|█████████ | 1454/1610 [7:58:20<41:26, 15.94s/it]
 90%|█████████ | 1455/1610 [7:58:36<41:26, 16.04s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 2.13085211758399, 'learning_rate': 9.627329192546583e-08, 'completion_length': 168.96429443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.27762510627508163, 'kl': 0.01861572265625, 'epoch': 4.52}

 90%|█████████ | 1455/1610 [7:58:36<41:26, 16.04s/it]
 90%|█████████ | 1456/1610 [7:58:51<40:33, 15.80s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6207389883402753, 'learning_rate': 9.565217391304348e-08, 'completion_length': 158.58929443359375, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1181928999722004, 'kl': 0.015594482421875, 'epoch': 4.52}

 90%|█████████ | 1456/1610 [7:58:51<40:33, 15.80s/it]
 90%|█████████ | 1457/1610 [7:59:07<40:47, 16.00s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.741326308175309, 'learning_rate': 9.503105590062112e-08, 'completion_length': 149.8214340209961, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.18409644439816475, 'kl': 0.016693115234375, 'epoch': 4.52}

 90%|█████████ | 1457/1610 [7:59:08<40:47, 16.00s/it]
 91%|█████████ | 1458/1610 [7:59:21<38:30, 15.20s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.405694338251017, 'learning_rate': 9.440993788819875e-08, 'completion_length': 135.08928680419922, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1896214671432972, 'kl': 0.0157470703125, 'epoch': 4.53}

 91%|█████████ | 1458/1610 [7:59:21<38:30, 15.20s/it]
 91%|█████████ | 1459/1610 [7:59:37<39:09, 15.56s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.8867385910874842, 'learning_rate': 9.378881987577639e-08, 'completion_length': 148.46428680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.0194091796875, 'epoch': 4.53}

 91%|█████████ | 1459/1610 [7:59:37<39:09, 15.56s/it]
 91%|█████████ | 1460/1610 [7:59:49<36:03, 14.42s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6671203353382124, 'learning_rate': 9.316770186335403e-08, 'completion_length': 133.00000381469727, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.11266787722706795, 'kl': 0.01605224609375, 'epoch': 4.53}

 91%|█████████ | 1460/1610 [7:59:49<36:03, 14.42s/it]
 91%|█████████ | 1461/1610 [8:00:05<36:38, 14.76s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.7754682890593723, 'learning_rate': 9.254658385093167e-08, 'completion_length': 164.7857208251953, 'rewards/accuracy_reward': 0.6250000447034836, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.1071428656578064, 'kl': 0.014801025390625, 'epoch': 4.54}

 91%|█████████ | 1461/1610 [8:00:05<36:38, 14.76s/it]
 91%|█████████ | 1462/1610 [8:00:22<38:08, 15.46s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8011404004724518, 'learning_rate': 9.192546583850931e-08, 'completion_length': 178.96429443359375, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.01629638671875, 'epoch': 4.54}

 91%|█████████ | 1462/1610 [8:00:22<38:08, 15.46s/it]
 91%|█████████ | 1463/1610 [8:00:36<36:43, 14.99s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.36579813253311516, 'learning_rate': 9.130434782608696e-08, 'completion_length': 140.94643020629883, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0714285746216774, 'kl': 0.01153564453125, 'epoch': 4.54}

 91%|█████████ | 1463/1610 [8:00:36<36:43, 14.99s/it]
 91%|█████████ | 1464/1610 [8:00:52<37:30, 15.41s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.20744321884415964, 'learning_rate': 9.068322981366459e-08, 'completion_length': 153.62500762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.732142984867096, 'reward_std': 0.06838765740394592, 'kl': 0.015716552734375, 'epoch': 4.55}

 91%|█████████ | 1464/1610 [8:00:52<37:30, 15.41s/it]
 91%|█████████ | 1465/1610 [8:01:09<38:08, 15.79s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9086922966026261, 'learning_rate': 9.006211180124224e-08, 'completion_length': 180.3928680419922, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216826319695, 'kl': 0.018310546875, 'epoch': 4.55}

 91%|█████████ | 1465/1610 [8:01:09<38:08, 15.79s/it]
 91%|█████████ | 1466/1610 [8:01:25<38:13, 15.93s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8741131122961253, 'learning_rate': 8.944099378881988e-08, 'completion_length': 183.2857208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1896214671432972, 'kl': 0.015533447265625, 'epoch': 4.55}

 91%|█████████ | 1466/1610 [8:01:25<38:13, 15.93s/it]
 91%|█████████ | 1467/1610 [8:01:42<38:46, 16.27s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6920072306425328, 'learning_rate': 8.881987577639751e-08, 'completion_length': 163.9464340209961, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216453790665, 'kl': 0.017486572265625, 'epoch': 4.56}

 91%|█████████ | 1467/1610 [8:01:42<38:46, 16.27s/it]
 91%|█████████ | 1468/1610 [8:01:57<37:23, 15.80s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.3541177983865162, 'learning_rate': 8.819875776397515e-08, 'completion_length': 142.89286422729492, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714365541935, 'kl': 0.01385498046875, 'epoch': 4.56}

 91%|█████████ | 1468/1610 [8:01:57<37:23, 15.80s/it]
 91%|█████████ | 1469/1610 [8:02:11<35:54, 15.28s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.7341381907761457, 'learning_rate': 8.757763975155279e-08, 'completion_length': 163.5714340209961, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214634180069, 'kl': 0.02020263671875, 'epoch': 4.56}

 91%|█████████ | 1469/1610 [8:02:11<35:54, 15.28s/it]
 91%|█████████▏| 1470/1610 [8:02:27<36:12, 15.51s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.1917243616534385, 'learning_rate': 8.695652173913042e-08, 'completion_length': 134.01786422729492, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.21981074661016464, 'kl': 0.01239013671875, 'epoch': 4.57}

 91%|█████████▏| 1470/1610 [8:02:27<36:12, 15.51s/it]
 91%|█████████▏| 1471/1610 [8:02:42<36:03, 15.56s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 1.9280112865967538, 'learning_rate': 8.633540372670807e-08, 'completion_length': 169.0000114440918, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071835577488, 'kl': 0.0108795166015625, 'epoch': 4.57}

 91%|█████████▏| 1471/1610 [8:02:42<36:03, 15.56s/it]
 91%|█████████▏| 1472/1610 [8:02:58<36:02, 15.67s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.150556946224079, 'learning_rate': 8.571428571428572e-08, 'completion_length': 163.85714721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.18409645557403564, 'kl': 0.017425537109375, 'epoch': 4.57}

 91%|█████████▏| 1472/1610 [8:02:58<36:02, 15.67s/it]
 91%|█████████▏| 1473/1610 [8:03:12<34:37, 15.17s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8557428219127743, 'learning_rate': 8.509316770186335e-08, 'completion_length': 156.14286041259766, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.11266788095235825, 'kl': 0.016357421875, 'epoch': 4.57}

 91%|█████████▏| 1473/1610 [8:03:12<34:37, 15.17s/it]
 92%|█████████▏| 1474/1610 [8:03:27<33:52, 14.94s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.0836187549181189, 'learning_rate': 8.4472049689441e-08, 'completion_length': 153.5357208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26657505333423615, 'kl': 0.0184326171875, 'epoch': 4.58}

 92%|█████████▏| 1474/1610 [8:03:27<33:52, 14.94s/it]
 92%|█████████▏| 1475/1610 [8:03:43<34:23, 15.29s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.4963098288748061, 'learning_rate': 8.385093167701864e-08, 'completion_length': 154.37500762939453, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.01898193359375, 'epoch': 4.58}

 92%|█████████▏| 1475/1610 [8:03:43<34:23, 15.29s/it]
 92%|█████████▏| 1476/1610 [8:04:00<35:34, 15.93s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.760295829945192, 'learning_rate': 8.322981366459626e-08, 'completion_length': 159.50000762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.018402099609375, 'epoch': 4.58}

 92%|█████████▏| 1476/1610 [8:04:00<35:34, 15.93s/it]
 92%|█████████▏| 1477/1610 [8:04:16<35:09, 15.86s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.8053171203313922, 'learning_rate': 8.26086956521739e-08, 'completion_length': 152.62500762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.1071428619325161, 'kl': 0.01300048828125, 'epoch': 4.59}

 92%|█████████▏| 1477/1610 [8:04:16<35:09, 15.86s/it]
 92%|█████████▏| 1478/1610 [8:04:29<33:02, 15.02s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.431585683575755, 'learning_rate': 8.198757763975155e-08, 'completion_length': 136.33929061889648, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.307814359664917, 'kl': 0.017974853515625, 'epoch': 4.59}

 92%|█████████▏| 1478/1610 [8:04:29<33:02, 15.02s/it]
 92%|█████████▏| 1479/1610 [8:04:44<33:01, 15.12s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.806325477418681, 'learning_rate': 8.136645962732918e-08, 'completion_length': 149.08929061889648, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.1181928962469101, 'kl': 0.015869140625, 'epoch': 4.59}

 92%|█████████▏| 1479/1610 [8:04:44<33:01, 15.12s/it]
 92%|█████████▏| 1480/1610 [8:05:03<35:00, 16.16s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.33062287224296294, 'learning_rate': 8.074534161490683e-08, 'completion_length': 212.75000762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.04123930633068085, 'kl': 0.01287841796875, 'epoch': 4.6}

 92%|█████████▏| 1480/1610 [8:05:03<35:00, 16.16s/it]
 92%|█████████▏| 1481/1610 [8:05:18<34:09, 15.89s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.1072262501827386, 'learning_rate': 8.012422360248448e-08, 'completion_length': 145.0714340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1896214708685875, 'kl': 0.015167236328125, 'epoch': 4.6}

 92%|█████████▏| 1481/1610 [8:05:18<34:09, 15.89s/it]
 92%|█████████▏| 1482/1610 [8:05:30<31:32, 14.79s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.382159699250525, 'learning_rate': 7.950310559006211e-08, 'completion_length': 130.17857360839844, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.2253357656300068, 'kl': 0.018402099609375, 'epoch': 4.6}

 92%|█████████▏| 1482/1610 [8:05:30<31:32, 14.79s/it]
 92%|█████████▏| 1483/1610 [8:05:47<32:07, 15.18s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.1314059270003614, 'learning_rate': 7.888198757763975e-08, 'completion_length': 156.17858123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1896214708685875, 'kl': 0.013336181640625, 'epoch': 4.61}

 92%|█████████▏| 1483/1610 [8:05:47<32:07, 15.18s/it]
 92%|█████████▏| 1484/1610 [8:06:01<31:09, 14.84s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.839944455254871, 'learning_rate': 7.82608695652174e-08, 'completion_length': 150.12500762939453, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.21981074661016464, 'kl': 0.018463134765625, 'epoch': 4.61}

 92%|█████████▏| 1484/1610 [8:06:01<31:09, 14.84s/it]
 92%|█████████▏| 1485/1610 [8:06:20<33:47, 16.22s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8049658327079339, 'learning_rate': 7.763975155279502e-08, 'completion_length': 167.1607208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1896214708685875, 'kl': 0.014862060546875, 'epoch': 4.61}

 92%|█████████▏| 1485/1610 [8:06:20<33:47, 16.22s/it]
 92%|█████████▏| 1486/1610 [8:06:33<31:43, 15.35s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.36494129078866605, 'learning_rate': 7.701863354037266e-08, 'completion_length': 135.37500762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07695358991622925, 'kl': 0.013946533203125, 'epoch': 4.61}

 92%|█████████▏| 1486/1610 [8:06:33<31:43, 15.35s/it]
 92%|█████████▏| 1487/1610 [8:06:49<31:30, 15.37s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9590687502697364, 'learning_rate': 7.639751552795031e-08, 'completion_length': 134.87500762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.2142857238650322, 'kl': 0.016357421875, 'epoch': 4.62}

 92%|█████████▏| 1487/1610 [8:06:49<31:30, 15.37s/it]
 92%|█████████▏| 1488/1610 [8:07:07<33:13, 16.34s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8171487855687305, 'learning_rate': 7.577639751552794e-08, 'completion_length': 196.8928680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1896214708685875, 'kl': 0.015625, 'epoch': 4.62}

 92%|█████████▏| 1488/1610 [8:07:07<33:13, 16.34s/it]
 92%|█████████▏| 1489/1610 [8:07:22<32:11, 15.96s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9560933792652854, 'learning_rate': 7.515527950310559e-08, 'completion_length': 147.6428680419922, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.18409645557403564, 'kl': 0.01495361328125, 'epoch': 4.62}

 92%|█████████▏| 1489/1610 [8:07:22<32:11, 15.96s/it]
 93%|█████████▎| 1490/1610 [8:07:41<33:16, 16.64s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5080738490124603, 'learning_rate': 7.453416149068323e-08, 'completion_length': 180.16072845458984, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1539071872830391, 'kl': 0.014556884765625, 'epoch': 4.63}

 93%|█████████▎| 1490/1610 [8:07:41<33:16, 16.64s/it]
 93%|█████████▎| 1491/1610 [8:07:58<33:07, 16.70s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.694213495510824, 'learning_rate': 7.391304347826087e-08, 'completion_length': 175.87500762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.14838216453790665, 'kl': 0.0206298828125, 'epoch': 4.63}

 93%|█████████▎| 1491/1610 [8:07:58<33:07, 16.70s/it]
 93%|█████████▎| 1492/1610 [8:08:13<32:07, 16.33s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 2.029130821868115, 'learning_rate': 7.329192546583851e-08, 'completion_length': 149.71429443359375, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1785714402794838, 'kl': 0.01611328125, 'epoch': 4.63}

 93%|█████████▎| 1492/1610 [8:08:13<32:07, 16.33s/it]
 93%|█████████▎| 1493/1610 [8:08:26<29:39, 15.21s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 5.076219641150596, 'learning_rate': 7.267080745341616e-08, 'completion_length': 122.83929061889648, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1785714402794838, 'kl': 0.0162353515625, 'epoch': 4.64}

 93%|█████████▎| 1493/1610 [8:08:26<29:39, 15.21s/it]
 93%|█████████▎| 1494/1610 [8:08:43<30:25, 15.74s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8805380819656733, 'learning_rate': 7.204968944099378e-08, 'completion_length': 168.2857208251953, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.11266787722706795, 'kl': 0.018096923828125, 'epoch': 4.64}

 93%|█████████▎| 1494/1610 [8:08:43<30:25, 15.74s/it]
 93%|█████████▎| 1495/1610 [8:08:58<29:51, 15.58s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 2.4939268978130276, 'learning_rate': 7.142857142857142e-08, 'completion_length': 121.48214721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.01947021484375, 'epoch': 4.64}

 93%|█████████▎| 1495/1610 [8:08:58<29:51, 15.58s/it]
 93%|█████████▎| 1496/1610 [8:09:17<31:43, 16.70s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.7036191664470162, 'learning_rate': 7.080745341614907e-08, 'completion_length': 191.0178680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.696428656578064, 'reward_std': 0.23689262941479683, 'kl': 0.01483154296875, 'epoch': 4.65}

 93%|█████████▎| 1496/1610 [8:09:17<31:43, 16.70s/it]
 93%|█████████▎| 1497/1610 [8:09:35<32:05, 17.04s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8519302444514951, 'learning_rate': 7.01863354037267e-08, 'completion_length': 141.2678680419922, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.19514648616313934, 'kl': 0.01849365234375, 'epoch': 4.65}

 93%|█████████▎| 1497/1610 [8:09:35<32:05, 17.04s/it]
 93%|█████████▎| 1498/1610 [8:09:53<32:13, 17.26s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.0227446628635328, 'learning_rate': 6.956521739130435e-08, 'completion_length': 156.50000762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6607143878936768, 'reward_std': 0.22229483723640442, 'kl': 0.01739501953125, 'epoch': 4.65}

 93%|█████████▎| 1498/1610 [8:09:53<32:13, 17.26s/it]
 93%|█████████▎| 1499/1610 [8:10:08<30:37, 16.56s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.4356895049955481, 'learning_rate': 6.894409937888199e-08, 'completion_length': 154.46429443359375, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392857313156128, 'reward_std': 0.0357142873108387, 'kl': 0.0191650390625, 'epoch': 4.66}

 93%|█████████▎| 1499/1610 [8:10:08<30:37, 16.56s/it]
 93%|█████████▎| 1500/1610 [8:10:26<31:21, 17.11s/it]
                                                     
{'loss': 0.0009, 'grad_norm': 1.5619128200831192, 'learning_rate': 6.832298136645963e-08, 'completion_length': 167.125, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7500001192092896, 'reward_std': 0.2967643216252327, 'kl': 0.0213623046875, 'epoch': 4.66}

 93%|█████████▎| 1500/1610 [8:10:26<31:21, 17.11s/it]
 93%|█████████▎| 1501/1610 [8:15:52<3:19:40, 109.91s/it]
                                                        
{'loss': 0.0008, 'grad_norm': 1.196961036548965, 'learning_rate': 6.770186335403727e-08, 'completion_length': 170.75000762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21981073170900345, 'kl': 0.0189208984375, 'epoch': 4.66}

 93%|█████████▎| 1501/1610 [8:15:52<3:19:40, 109.91s/it]
 93%|█████████▎| 1502/1610 [8:16:07<2:26:18, 81.28s/it] 
                                                       
{'loss': 0.0008, 'grad_norm': 1.1579915735137412, 'learning_rate': 6.708074534161489e-08, 'completion_length': 157.12500762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2967643439769745, 'kl': 0.0189208984375, 'epoch': 4.66}

 93%|█████████▎| 1502/1610 [8:16:07<2:26:18, 81.28s/it]
 93%|█████████▎| 1503/1610 [8:16:22<1:49:28, 61.39s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.58388373927497, 'learning_rate': 6.645962732919254e-08, 'completion_length': 163.67857360839844, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1181928962469101, 'kl': 0.014923095703125, 'epoch': 4.67}

 93%|█████████▎| 1503/1610 [8:16:22<1:49:28, 61.39s/it]
 93%|█████████▎| 1504/1610 [8:16:39<1:24:56, 48.08s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 0.7205455066356821, 'learning_rate': 6.583850931677018e-08, 'completion_length': 173.83929443359375, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.2610500305891037, 'kl': 0.014984130859375, 'epoch': 4.67}

 93%|█████████▎| 1504/1610 [8:16:39<1:24:56, 48.08s/it]
 93%|█████████▎| 1505/1610 [8:16:52<1:05:55, 37.68s/it]
                                                       
{'loss': 0.0006, 'grad_norm': 1.0664937476287344, 'learning_rate': 6.521739130434782e-08, 'completion_length': 130.92857360839844, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1896214634180069, 'kl': 0.014556884765625, 'epoch': 4.67}

 93%|█████████▎| 1505/1610 [8:16:52<1:05:55, 37.68s/it]
 94%|█████████▎| 1506/1610 [8:17:09<54:19, 31.34s/it]  
                                                     
{'loss': 0.0008, 'grad_norm': 1.208211683591619, 'learning_rate': 6.459627329192546e-08, 'completion_length': 157.21428680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25552502274513245, 'kl': 0.0208740234375, 'epoch': 4.68}

 94%|█████████▎| 1506/1610 [8:17:09<54:19, 31.34s/it]
 94%|█████████▎| 1507/1610 [8:17:24<45:28, 26.49s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9790602430148136, 'learning_rate': 6.397515527950311e-08, 'completion_length': 150.0357208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2253357619047165, 'kl': 0.0150146484375, 'epoch': 4.68}

 94%|█████████▎| 1507/1610 [8:17:24<45:28, 26.49s/it]
 94%|█████████▎| 1508/1610 [8:17:43<41:06, 24.18s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.298871308960425, 'learning_rate': 6.335403726708074e-08, 'completion_length': 183.00000762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1539071872830391, 'kl': 0.01971435546875, 'epoch': 4.68}

 94%|█████████▎| 1508/1610 [8:17:43<41:06, 24.18s/it]
 94%|█████████▎| 1509/1610 [8:17:59<36:49, 21.88s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9985035935780867, 'learning_rate': 6.273291925465838e-08, 'completion_length': 162.73214721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.0824786126613617, 'kl': 0.01470947265625, 'epoch': 4.69}

 94%|█████████▎| 1509/1610 [8:17:59<36:49, 21.88s/it]
 94%|█████████▍| 1510/1610 [8:18:13<32:22, 19.43s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.7941524558090147, 'learning_rate': 6.211180124223602e-08, 'completion_length': 146.00000381469727, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.15943220257759094, 'kl': 0.019439697265625, 'epoch': 4.69}

 94%|█████████▍| 1510/1610 [8:18:13<32:22, 19.43s/it]
 94%|█████████▍| 1511/1610 [8:18:28<29:37, 17.95s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.7547639676745277, 'learning_rate': 6.149068322981366e-08, 'completion_length': 148.17857360839844, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1181928962469101, 'kl': 0.0155029296875, 'epoch': 4.69}

 94%|█████████▍| 1511/1610 [8:18:28<29:37, 17.95s/it]
 94%|█████████▍| 1512/1610 [8:18:47<30:06, 18.44s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6012561585054534, 'learning_rate': 6.086956521739131e-08, 'completion_length': 200.5714340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.5892857909202576, 'reward_std': 0.1071428619325161, 'kl': 0.014373779296875, 'epoch': 4.7}

 94%|█████████▍| 1512/1610 [8:18:47<30:06, 18.44s/it]
 94%|█████████▍| 1513/1610 [8:19:02<27:59, 17.31s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.1755891091687787, 'learning_rate': 6.024844720496894e-08, 'completion_length': 132.28571701049805, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.12371791899204254, 'kl': 0.014892578125, 'epoch': 4.7}

 94%|█████████▍| 1513/1610 [8:19:02<27:59, 17.31s/it]
 94%|█████████▍| 1514/1610 [8:19:18<27:05, 16.93s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8014095264508347, 'learning_rate': 5.962732919254657e-08, 'completion_length': 165.5714340209961, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.11266787722706795, 'kl': 0.016448974609375, 'epoch': 4.7}

 94%|█████████▍| 1514/1610 [8:19:18<27:05, 16.93s/it]
 94%|█████████▍| 1515/1610 [8:19:35<26:53, 16.99s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6068150530493213, 'learning_rate': 5.900621118012422e-08, 'completion_length': 176.3928680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571529686451, 'kl': 0.0166015625, 'epoch': 4.7}

 94%|█████████▍| 1515/1610 [8:19:35<26:53, 16.99s/it]
 94%|█████████▍| 1516/1610 [8:19:51<26:03, 16.63s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.7666915874504925, 'learning_rate': 5.8385093167701866e-08, 'completion_length': 141.23214721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.18409644439816475, 'kl': 0.0167236328125, 'epoch': 4.71}

 94%|█████████▍| 1516/1610 [8:19:51<26:03, 16.63s/it]
 94%|█████████▍| 1517/1610 [8:20:09<26:36, 17.17s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.5869083428493637, 'learning_rate': 5.77639751552795e-08, 'completion_length': 183.44644165039062, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5892857909202576, 'reward_std': 0.14838217198848724, 'kl': 0.012969970703125, 'epoch': 4.71}

 94%|█████████▍| 1517/1610 [8:20:09<26:36, 17.17s/it]
 94%|█████████▍| 1518/1610 [8:20:24<25:10, 16.42s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.3767293657498667, 'learning_rate': 5.714285714285714e-08, 'completion_length': 135.50000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.04123930633068085, 'kl': 0.011444091796875, 'epoch': 4.71}

 94%|█████████▍| 1518/1610 [8:20:24<25:10, 16.42s/it]
 94%|█████████▍| 1519/1610 [8:20:41<25:21, 16.71s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.1974343395149338, 'learning_rate': 5.6521739130434777e-08, 'completion_length': 173.9107208251953, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.18409645557403564, 'kl': 0.01776123046875, 'epoch': 4.72}

 94%|█████████▍| 1519/1610 [8:20:41<25:21, 16.71s/it]
 94%|█████████▍| 1520/1610 [8:21:00<25:46, 17.18s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.7150293907442248, 'learning_rate': 5.590062111801242e-08, 'completion_length': 199.6964340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.18409645557403564, 'kl': 0.01800537109375, 'epoch': 4.72}

 94%|█████████▍| 1520/1610 [8:21:00<25:46, 17.18s/it]
 94%|█████████▍| 1521/1610 [8:21:16<25:09, 16.96s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.2980383463974055, 'learning_rate': 5.527950310559006e-08, 'completion_length': 163.30358123779297, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.14838216826319695, 'kl': 0.01708984375, 'epoch': 4.72}

 94%|█████████▍| 1521/1610 [8:21:16<25:09, 16.96s/it]
 95%|█████████▍| 1522/1610 [8:21:30<23:22, 15.93s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6128257111464154, 'learning_rate': 5.4658385093167694e-08, 'completion_length': 134.07143783569336, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1071428619325161, 'kl': 0.017852783203125, 'epoch': 4.73}

 95%|█████████▍| 1522/1610 [8:21:30<23:22, 15.93s/it]
 95%|█████████▍| 1523/1610 [8:21:46<23:26, 16.17s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.4138415858763625, 'learning_rate': 5.403726708074534e-08, 'completion_length': 172.08929443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.07695359364151955, 'kl': 0.016143798828125, 'epoch': 4.73}

 95%|█████████▍| 1523/1610 [8:21:46<23:26, 16.17s/it]
 95%|█████████▍| 1524/1610 [8:22:01<22:27, 15.67s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.0749292645011737, 'learning_rate': 5.341614906832298e-08, 'completion_length': 151.8214340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.23086076974868774, 'kl': 0.0211181640625, 'epoch': 4.73}

 95%|█████████▍| 1524/1610 [8:22:01<22:27, 15.67s/it]
 95%|█████████▍| 1525/1610 [8:22:17<22:20, 15.77s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.2531150670207052, 'learning_rate': 5.279503105590062e-08, 'completion_length': 182.69644165039062, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2721000909805298, 'kl': 0.018524169921875, 'epoch': 4.74}

 95%|█████████▍| 1525/1610 [8:22:17<22:20, 15.77s/it]
 95%|█████████▍| 1526/1610 [8:22:32<21:49, 15.59s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.5773180282026125, 'learning_rate': 5.217391304347826e-08, 'completion_length': 152.7857208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2500000149011612, 'kl': 0.019317626953125, 'epoch': 4.74}

 95%|█████████▍| 1526/1610 [8:22:32<21:49, 15.59s/it]
 95%|█████████▍| 1527/1610 [8:22:47<21:14, 15.36s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9933473896238194, 'learning_rate': 5.1552795031055897e-08, 'completion_length': 143.17858123779297, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.15943220257759094, 'kl': 0.015289306640625, 'epoch': 4.74}

 95%|█████████▍| 1527/1610 [8:22:47<21:14, 15.36s/it]
 95%|█████████▍| 1528/1610 [8:23:00<20:12, 14.78s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.4102813420502307, 'learning_rate': 5.0931677018633536e-08, 'completion_length': 120.87500762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.26657505333423615, 'kl': 0.02008056640625, 'epoch': 4.75}

 95%|█████████▍| 1528/1610 [8:23:00<20:12, 14.78s/it]
 95%|█████████▍| 1529/1610 [8:23:17<20:58, 15.53s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.39022398423129395, 'learning_rate': 5.031055900621118e-08, 'completion_length': 171.89286041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8214285969734192, 'reward_std': 0.0714285746216774, 'kl': 0.0191650390625, 'epoch': 4.75}

 95%|█████████▍| 1529/1610 [8:23:17<20:58, 15.53s/it]
 95%|█████████▌| 1530/1610 [8:23:34<21:10, 15.88s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9376252780752055, 'learning_rate': 4.9689440993788814e-08, 'completion_length': 169.60715103149414, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6071429252624512, 'reward_std': 0.25552502274513245, 'kl': 0.01654052734375, 'epoch': 4.75}

 95%|█████████▌| 1530/1610 [8:23:34<21:10, 15.88s/it]
 95%|█████████▌| 1531/1610 [8:23:49<20:24, 15.51s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 2.6887220655535424, 'learning_rate': 4.906832298136645e-08, 'completion_length': 143.9464340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25552502274513245, 'kl': 0.01434326171875, 'epoch': 4.75}

 95%|█████████▌| 1531/1610 [8:23:49<20:24, 15.51s/it]
 95%|█████████▌| 1532/1610 [8:24:08<21:31, 16.56s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8067238624905239, 'learning_rate': 4.84472049689441e-08, 'completion_length': 182.6071548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571529686451, 'kl': 0.01580810546875, 'epoch': 4.76}

 95%|█████████▌| 1532/1610 [8:24:08<21:31, 16.56s/it]
 95%|█████████▌| 1533/1610 [8:24:20<19:41, 15.35s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6651581309328501, 'learning_rate': 4.782608695652174e-08, 'completion_length': 150.25000762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.07695359364151955, 'kl': 0.014007568359375, 'epoch': 4.76}

 95%|█████████▌| 1533/1610 [8:24:20<19:41, 15.35s/it]
 95%|█████████▌| 1534/1610 [8:24:36<19:35, 15.47s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.879083297756306, 'learning_rate': 4.720496894409938e-08, 'completion_length': 162.4107208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.14838216453790665, 'kl': 0.01702880859375, 'epoch': 4.76}

 95%|█████████▌| 1534/1610 [8:24:36<19:35, 15.47s/it]
 95%|█████████▌| 1535/1610 [8:24:47<17:32, 14.03s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.2353322435999445, 'learning_rate': 4.6583850931677016e-08, 'completion_length': 113.98215103149414, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.1071428619325161, 'kl': 0.01507568359375, 'epoch': 4.77}

 95%|█████████▌| 1535/1610 [8:24:47<17:32, 14.03s/it]
 95%|█████████▌| 1536/1610 [8:25:02<17:39, 14.32s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8280334423911095, 'learning_rate': 4.5962732919254656e-08, 'completion_length': 176.76786422729492, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.14838216453790665, 'kl': 0.016082763671875, 'epoch': 4.77}

 95%|█████████▌| 1536/1610 [8:25:02<17:39, 14.32s/it]
 95%|█████████▌| 1537/1610 [8:25:17<17:49, 14.65s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.446254378981087, 'learning_rate': 4.5341614906832295e-08, 'completion_length': 170.50000762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.11266788095235825, 'kl': 0.018798828125, 'epoch': 4.77}

 95%|█████████▌| 1537/1610 [8:25:17<17:49, 14.65s/it]
 96%|█████████▌| 1538/1610 [8:25:30<17:03, 14.21s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.4600447355634678, 'learning_rate': 4.472049689440994e-08, 'completion_length': 124.14286804199219, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2253357619047165, 'kl': 0.01739501953125, 'epoch': 4.78}

 96%|█████████▌| 1538/1610 [8:25:30<17:03, 14.21s/it]
 96%|█████████▌| 1539/1610 [8:25:44<16:27, 13.91s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.8331365361150601, 'learning_rate': 4.409937888198757e-08, 'completion_length': 133.0357208251953, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071835577488, 'kl': 0.01922607421875, 'epoch': 4.78}

 96%|█████████▌| 1539/1610 [8:25:44<16:27, 13.91s/it]
 96%|█████████▌| 1540/1610 [8:25:58<16:28, 14.12s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.302290562721513, 'learning_rate': 4.347826086956521e-08, 'completion_length': 157.1428680419922, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.18409645557403564, 'kl': 0.01495361328125, 'epoch': 4.78}

 96%|█████████▌| 1540/1610 [8:25:58<16:28, 14.12s/it]
 96%|█████████▌| 1541/1610 [8:26:12<16:14, 14.13s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.2512144636995042, 'learning_rate': 4.285714285714286e-08, 'completion_length': 139.92857360839844, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086077719926834, 'kl': 0.017181396484375, 'epoch': 4.79}

 96%|█████████▌| 1541/1610 [8:26:12<16:14, 14.13s/it]
 96%|█████████▌| 1542/1610 [8:26:26<15:56, 14.06s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 0.7848823371128435, 'learning_rate': 4.22360248447205e-08, 'completion_length': 142.3571548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.0714285746216774, 'kl': 0.011138916015625, 'epoch': 4.79}

 96%|█████████▌| 1542/1610 [8:26:26<15:56, 14.06s/it]
 96%|█████████▌| 1543/1610 [8:26:41<16:02, 14.36s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.6386899582967938, 'learning_rate': 4.161490683229813e-08, 'completion_length': 154.9464340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14838216826319695, 'kl': 0.019439697265625, 'epoch': 4.79}

 96%|█████████▌| 1543/1610 [8:26:41<16:02, 14.36s/it]
 96%|█████████▌| 1544/1610 [8:26:57<16:11, 14.72s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8363358542329146, 'learning_rate': 4.0993788819875776e-08, 'completion_length': 156.39286041259766, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2610500454902649, 'kl': 0.014984130859375, 'epoch': 4.8}

 96%|█████████▌| 1544/1610 [8:26:57<16:11, 14.72s/it]
 96%|█████████▌| 1545/1610 [8:27:11<15:48, 14.59s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.36061251805044425, 'learning_rate': 4.0372670807453415e-08, 'completion_length': 156.25000762939453, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8214285969734192, 'reward_std': 0.0714285746216774, 'kl': 0.0191650390625, 'epoch': 4.8}

 96%|█████████▌| 1545/1610 [8:27:11<15:48, 14.59s/it]
 96%|█████████▌| 1546/1610 [8:27:28<16:26, 15.41s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 2.7939063892286757, 'learning_rate': 3.9751552795031054e-08, 'completion_length': 188.71429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.32695358991622925, 'kl': 0.01605224609375, 'epoch': 4.8}

 96%|█████████▌| 1546/1610 [8:27:28<16:26, 15.41s/it]
 96%|█████████▌| 1547/1610 [8:27:44<16:19, 15.55s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.9833560411780446, 'learning_rate': 3.91304347826087e-08, 'completion_length': 129.21429443359375, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25552502274513245, 'kl': 0.0201416015625, 'epoch': 4.8}

 96%|█████████▌| 1547/1610 [8:27:44<16:19, 15.55s/it]
 96%|█████████▌| 1548/1610 [8:27:59<15:53, 15.38s/it]
                                                     
{'loss': 0.0011, 'grad_norm': 1.2893910789796919, 'learning_rate': 3.850931677018633e-08, 'completion_length': 167.00000762939453, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1896214708685875, 'kl': 0.02728271484375, 'epoch': 4.81}

 96%|█████████▌| 1548/1610 [8:27:59<15:53, 15.38s/it]
 96%|█████████▌| 1549/1610 [8:28:14<15:16, 15.03s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.0333074866820762, 'learning_rate': 3.788819875776397e-08, 'completion_length': 157.87500762939453, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.18409645557403564, 'kl': 0.0191650390625, 'epoch': 4.81}

 96%|█████████▌| 1549/1610 [8:28:14<15:16, 15.03s/it]
 96%|█████████▋| 1550/1610 [8:28:32<15:59, 15.99s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8966524640253294, 'learning_rate': 3.726708074534162e-08, 'completion_length': 216.08929443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.11266788095235825, 'kl': 0.016204833984375, 'epoch': 4.81}

 96%|█████████▋| 1550/1610 [8:28:32<15:59, 15.99s/it]
 96%|█████████▋| 1551/1610 [8:28:48<15:39, 15.92s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5579831808796769, 'learning_rate': 3.6645962732919256e-08, 'completion_length': 180.87500762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1785714402794838, 'kl': 0.0152587890625, 'epoch': 4.82}

 96%|█████████▋| 1551/1610 [8:28:48<15:39, 15.92s/it]
 96%|█████████▋| 1552/1610 [8:29:07<16:20, 16.91s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.8904481523038499, 'learning_rate': 3.602484472049689e-08, 'completion_length': 201.42858123779297, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.535714328289032, 'reward_std': 0.2142857238650322, 'kl': 0.01373291015625, 'epoch': 4.82}

 96%|█████████▋| 1552/1610 [8:29:07<16:20, 16.91s/it]
 96%|█████████▋| 1553/1610 [8:29:20<15:06, 15.90s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.7626787313042147, 'learning_rate': 3.5403726708074535e-08, 'completion_length': 140.64286041259766, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23086076974868774, 'kl': 0.013153076171875, 'epoch': 4.82}

 96%|█████████▋| 1553/1610 [8:29:20<15:06, 15.90s/it]
 97%|█████████▋| 1554/1610 [8:29:34<14:11, 15.20s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8957812779865121, 'learning_rate': 3.4782608695652174e-08, 'completion_length': 145.55357360839844, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1428571492433548, 'kl': 0.013916015625, 'epoch': 4.83}

 97%|█████████▋| 1554/1610 [8:29:34<14:11, 15.20s/it]
 97%|█████████▋| 1555/1610 [8:29:50<14:04, 15.35s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.2574195399893504, 'learning_rate': 3.416149068322981e-08, 'completion_length': 144.1964340209961, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2142857201397419, 'kl': 0.011871337890625, 'epoch': 4.83}

 97%|█████████▋| 1555/1610 [8:29:50<14:04, 15.35s/it]
 97%|█████████▋| 1556/1610 [8:30:06<13:58, 15.53s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.2642730614138344, 'learning_rate': 3.3540372670807445e-08, 'completion_length': 175.7857208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.01519775390625, 'epoch': 4.83}

 97%|█████████▋| 1556/1610 [8:30:06<13:58, 15.53s/it]
 97%|█████████▋| 1557/1610 [8:30:22<13:55, 15.77s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.7209414814323702, 'learning_rate': 3.291925465838509e-08, 'completion_length': 159.375, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1181928962469101, 'kl': 0.012481689453125, 'epoch': 4.84}

 97%|█████████▋| 1557/1610 [8:30:22<13:55, 15.77s/it]
 97%|█████████▋| 1558/1610 [8:30:34<12:41, 14.64s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.5006057466704847, 'learning_rate': 3.229813664596273e-08, 'completion_length': 124.87500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.0824786126613617, 'kl': 0.019927978515625, 'epoch': 4.84}

 97%|█████████▋| 1558/1610 [8:30:34<12:41, 14.64s/it]
 97%|█████████▋| 1559/1610 [8:30:50<12:48, 15.06s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5339768783733537, 'learning_rate': 3.167701863354037e-08, 'completion_length': 169.87500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.015350341796875, 'epoch': 4.84}

 97%|█████████▋| 1559/1610 [8:30:50<12:48, 15.06s/it]
 97%|█████████▋| 1560/1610 [8:31:04<12:17, 14.76s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.0626140005637377, 'learning_rate': 3.105590062111801e-08, 'completion_length': 148.48214721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.24191081523895264, 'kl': 0.013946533203125, 'epoch': 4.84}

 97%|█████████▋| 1560/1610 [8:31:04<12:17, 14.76s/it]
 97%|█████████▋| 1561/1610 [8:31:19<12:12, 14.96s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8468155881680134, 'learning_rate': 3.0434782608695655e-08, 'completion_length': 149.35714721679688, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.1428571529686451, 'kl': 0.01654052734375, 'epoch': 4.85}

 97%|█████████▋| 1561/1610 [8:31:19<12:12, 14.96s/it]
 97%|█████████▋| 1562/1610 [8:31:34<11:55, 14.90s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6564399323760632, 'learning_rate': 2.981366459627329e-08, 'completion_length': 136.33929443359375, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11266788095235825, 'kl': 0.0169677734375, 'epoch': 4.85}

 97%|█████████▋| 1562/1610 [8:31:34<11:55, 14.90s/it]
 97%|█████████▋| 1563/1610 [8:31:49<11:36, 14.83s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.479127687673639, 'learning_rate': 2.9192546583850933e-08, 'completion_length': 139.46428680419922, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.013427734375, 'epoch': 4.85}

 97%|█████████▋| 1563/1610 [8:31:49<11:36, 14.83s/it]
 97%|█████████▋| 1564/1610 [8:32:06<11:51, 15.47s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.7640605757447003, 'learning_rate': 2.857142857142857e-08, 'completion_length': 161.30358123779297, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1539071835577488, 'kl': 0.0194091796875, 'epoch': 4.86}

 97%|█████████▋| 1564/1610 [8:32:06<11:51, 15.47s/it]
 97%|█████████▋| 1565/1610 [8:32:19<11:06, 14.80s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6953268902857582, 'learning_rate': 2.795031055900621e-08, 'completion_length': 139.30357360839844, 'rewards/accuracy_reward': 0.6250000447034836, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.07695358991622925, 'kl': 0.014862060546875, 'epoch': 4.86}

 97%|█████████▋| 1565/1610 [8:32:19<11:06, 14.80s/it]
 97%|█████████▋| 1566/1610 [8:32:37<11:31, 15.72s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.266864490338537, 'learning_rate': 2.7329192546583847e-08, 'completion_length': 185.0357208251953, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1181928962469101, 'kl': 0.0140380859375, 'epoch': 4.86}

 97%|█████████▋| 1566/1610 [8:32:37<11:31, 15.72s/it]
 97%|█████████▋| 1567/1610 [8:32:52<11:13, 15.66s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8335616993309478, 'learning_rate': 2.670807453416149e-08, 'completion_length': 169.5714340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.0174560546875, 'epoch': 4.87}

 97%|█████████▋| 1567/1610 [8:32:52<11:13, 15.66s/it]
 97%|█████████▋| 1568/1610 [8:33:09<11:11, 16.00s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5443858778920044, 'learning_rate': 2.608695652173913e-08, 'completion_length': 167.6964340209961, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.1071428656578064, 'kl': 0.01385498046875, 'epoch': 4.87}

 97%|█████████▋| 1568/1610 [8:33:09<11:11, 16.00s/it]
 97%|█████████▋| 1569/1610 [8:33:24<10:46, 15.78s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.1276076275967855, 'learning_rate': 2.5465838509316768e-08, 'completion_length': 144.17857360839844, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.015228271484375, 'epoch': 4.87}

 97%|█████████▋| 1569/1610 [8:33:24<10:46, 15.78s/it]
 98%|█████████▊| 1570/1610 [8:33:39<10:12, 15.31s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.297624824909015, 'learning_rate': 2.4844720496894407e-08, 'completion_length': 129.12500762939453, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.21981074661016464, 'kl': 0.0166015625, 'epoch': 4.88}

 98%|█████████▊| 1570/1610 [8:33:39<10:12, 15.31s/it]
 98%|█████████▊| 1571/1610 [8:33:54<09:55, 15.27s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.0824443132466302, 'learning_rate': 2.422360248447205e-08, 'completion_length': 151.76786422729492, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18409644439816475, 'kl': 0.01580810546875, 'epoch': 4.88}

 98%|█████████▊| 1571/1610 [8:33:54<09:55, 15.27s/it]
 98%|█████████▊| 1572/1610 [8:34:10<09:46, 15.44s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 2.053436710015425, 'learning_rate': 2.360248447204969e-08, 'completion_length': 164.33929443359375, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1071428619325161, 'kl': 0.01849365234375, 'epoch': 4.88}

 98%|█████████▊| 1572/1610 [8:34:10<09:46, 15.44s/it]
 98%|█████████▊| 1573/1610 [8:34:25<09:32, 15.46s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6700442346867074, 'learning_rate': 2.2981366459627328e-08, 'completion_length': 136.12500762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.1181928962469101, 'kl': 0.0181884765625, 'epoch': 4.89}

 98%|█████████▊| 1573/1610 [8:34:25<09:32, 15.46s/it]
 98%|█████████▊| 1574/1610 [8:34:44<09:55, 16.54s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 0.5694414397563071, 'learning_rate': 2.236024844720497e-08, 'completion_length': 169.48214721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7678572535514832, 'reward_std': 0.14838217198848724, 'kl': 0.01983642578125, 'epoch': 4.89}

 98%|█████████▊| 1574/1610 [8:34:44<09:55, 16.54s/it]
 98%|█████████▊| 1575/1610 [8:35:01<09:41, 16.61s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 4.434094368203655, 'learning_rate': 2.1739130434782606e-08, 'completion_length': 149.1964340209961, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.25552503019571304, 'kl': 0.017333984375, 'epoch': 4.89}

 98%|█████████▊| 1575/1610 [8:35:01<09:41, 16.61s/it]
 98%|█████████▊| 1576/1610 [8:35:19<09:39, 17.04s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8543808734552094, 'learning_rate': 2.111801242236025e-08, 'completion_length': 156.0178680419922, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6785715222358704, 'reward_std': 0.21222837269306183, 'kl': 0.017578125, 'epoch': 4.89}

 98%|█████████▊| 1576/1610 [8:35:19<09:39, 17.04s/it]
 98%|█████████▊| 1577/1610 [8:35:38<09:43, 17.67s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 0.8158521420239411, 'learning_rate': 2.0496894409937888e-08, 'completion_length': 171.7321548461914, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.18409645557403564, 'kl': 0.010467529296875, 'epoch': 4.9}

 98%|█████████▊| 1577/1610 [8:35:38<09:43, 17.67s/it]
 98%|█████████▊| 1578/1610 [8:35:57<09:38, 18.07s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8071800928227274, 'learning_rate': 1.9875776397515527e-08, 'completion_length': 182.1607208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1071428619325161, 'kl': 0.017059326171875, 'epoch': 4.9}

 98%|█████████▊| 1578/1610 [8:35:57<09:38, 18.07s/it]
 98%|█████████▊| 1579/1610 [8:36:09<08:18, 16.09s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 0.6189246704164232, 'learning_rate': 1.9254658385093166e-08, 'completion_length': 122.71429443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.15943221747875214, 'kl': 0.009368896484375, 'epoch': 4.9}

 98%|█████████▊| 1579/1610 [8:36:09<08:18, 16.09s/it]
 98%|█████████▊| 1580/1610 [8:36:23<07:46, 15.54s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.034689099874073676, 'learning_rate': 1.863354037267081e-08, 'completion_length': 160.12500762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.0, 'kl': 0.012786865234375, 'epoch': 4.91}

 98%|█████████▊| 1580/1610 [8:36:23<07:46, 15.54s/it]
 98%|█████████▊| 1581/1610 [8:36:38<07:26, 15.40s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9217028771065089, 'learning_rate': 1.8012422360248444e-08, 'completion_length': 159.67858123779297, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678572535514832, 'reward_std': 0.1896214671432972, 'kl': 0.015380859375, 'epoch': 4.91}

 98%|█████████▊| 1581/1610 [8:36:38<07:26, 15.40s/it]
 98%|█████████▊| 1582/1610 [8:36:56<07:29, 16.06s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.4848713960386286, 'learning_rate': 1.7391304347826087e-08, 'completion_length': 190.58929443359375, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142858505249023, 'reward_std': 0.0824786126613617, 'kl': 0.013580322265625, 'epoch': 4.91}

 98%|█████████▊| 1582/1610 [8:36:56<07:29, 16.06s/it]
 98%|█████████▊| 1583/1610 [8:37:15<07:39, 17.03s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9080475295575422, 'learning_rate': 1.6770186335403723e-08, 'completion_length': 168.6964340209961, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.7857143878936768, 'reward_std': 0.1428571529686451, 'kl': 0.01806640625, 'epoch': 4.92}

 98%|█████████▊| 1583/1610 [8:37:15<07:39, 17.03s/it]
 98%|█████████▊| 1584/1610 [8:37:30<07:05, 16.36s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8495477581307714, 'learning_rate': 1.6149068322981365e-08, 'completion_length': 149.76786041259766, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.14838216826319695, 'kl': 0.0152587890625, 'epoch': 4.92}

 98%|█████████▊| 1584/1610 [8:37:30<07:05, 16.36s/it]
 98%|█████████▊| 1585/1610 [8:37:45<06:43, 16.13s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.5988727636186838, 'learning_rate': 1.5527950310559004e-08, 'completion_length': 146.26786422729492, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.11266788095235825, 'kl': 0.012786865234375, 'epoch': 4.92}

 98%|█████████▊| 1585/1610 [8:37:45<06:43, 16.13s/it]
 99%|█████████▊| 1586/1610 [8:37:59<06:09, 15.40s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8767576792922178, 'learning_rate': 1.4906832298136644e-08, 'completion_length': 143.39286041259766, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1428571492433548, 'kl': 0.015045166015625, 'epoch': 4.93}

 99%|█████████▊| 1586/1610 [8:37:59<06:09, 15.40s/it]
 99%|█████████▊| 1587/1610 [8:38:15<06:00, 15.66s/it]
                                                     
{'loss': 0.0004, 'grad_norm': 0.4398872752927834, 'learning_rate': 1.4285714285714284e-08, 'completion_length': 168.2857208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.0106201171875, 'epoch': 4.93}

 99%|█████████▊| 1587/1610 [8:38:15<06:00, 15.66s/it]
 99%|█████████▊| 1588/1610 [8:38:30<05:39, 15.42s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 1.6371324253922066, 'learning_rate': 1.3664596273291924e-08, 'completion_length': 146.3214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.18409644439816475, 'kl': 0.02056884765625, 'epoch': 4.93}

 99%|█████████▊| 1588/1610 [8:38:30<05:39, 15.42s/it]
 99%|█████████▊| 1589/1610 [8:38:43<05:10, 14.77s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.8629049106358612, 'learning_rate': 1.3043478260869564e-08, 'completion_length': 122.5714340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.1539071872830391, 'kl': 0.013153076171875, 'epoch': 4.93}

 99%|█████████▊| 1589/1610 [8:38:43<05:10, 14.77s/it]
 99%|█████████▉| 1590/1610 [8:38:57<04:50, 14.54s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.5580357180528401, 'learning_rate': 1.2422360248447204e-08, 'completion_length': 161.12500762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.0357142873108387, 'kl': 0.014251708984375, 'epoch': 4.94}

 99%|█████████▉| 1590/1610 [8:38:57<04:50, 14.54s/it]
 99%|█████████▉| 1591/1610 [8:39:13<04:43, 14.92s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.9532118858534471, 'learning_rate': 1.1801242236024844e-08, 'completion_length': 137.9107208251953, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.1896214708685875, 'kl': 0.01690673828125, 'epoch': 4.94}

 99%|█████████▉| 1591/1610 [8:39:13<04:43, 14.92s/it]
 99%|█████████▉| 1592/1610 [8:39:28<04:29, 14.98s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.743865061893522, 'learning_rate': 1.1180124223602485e-08, 'completion_length': 139.3928680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1539071872830391, 'kl': 0.01434326171875, 'epoch': 4.94}

 99%|█████████▉| 1592/1610 [8:39:28<04:29, 14.98s/it]
 99%|█████████▉| 1593/1610 [8:39:44<04:17, 15.15s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 0.6821945404307139, 'learning_rate': 1.0559006211180124e-08, 'completion_length': 163.60714721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.07695359364151955, 'kl': 0.01171875, 'epoch': 4.95}

 99%|█████████▉| 1593/1610 [8:39:44<04:17, 15.15s/it]
 99%|█████████▉| 1594/1610 [8:39:59<04:03, 15.23s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 1.2373532081371534, 'learning_rate': 9.937888198757763e-09, 'completion_length': 145.3214340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.18409645557403564, 'kl': 0.01763916015625, 'epoch': 4.95}

 99%|█████████▉| 1594/1610 [8:39:59<04:03, 15.23s/it]
 99%|█████████▉| 1595/1610 [8:40:13<03:41, 14.78s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.7992724975641801, 'learning_rate': 9.316770186335404e-09, 'completion_length': 140.37500381469727, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.14838216826319695, 'kl': 0.016693115234375, 'epoch': 4.95}

 99%|█████████▉| 1595/1610 [8:40:13<03:41, 14.78s/it]
 99%|█████████▉| 1596/1610 [8:40:28<03:26, 14.78s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.8106577419939849, 'learning_rate': 8.695652173913043e-09, 'completion_length': 157.9821548461914, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.21981073170900345, 'kl': 0.0159912109375, 'epoch': 4.96}

 99%|█████████▉| 1596/1610 [8:40:28<03:26, 14.78s/it]
 99%|█████████▉| 1597/1610 [8:40:41<03:04, 14.19s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 1.5400857375442363, 'learning_rate': 8.074534161490683e-09, 'completion_length': 131.21429443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.14838216453790665, 'kl': 0.014617919921875, 'epoch': 4.96}

 99%|█████████▉| 1597/1610 [8:40:41<03:04, 14.19s/it]
 99%|█████████▉| 1598/1610 [8:40:55<02:52, 14.39s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.9695351873250863, 'learning_rate': 7.453416149068322e-09, 'completion_length': 133.71429061889648, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1539071872830391, 'kl': 0.0150146484375, 'epoch': 4.96}

 99%|█████████▉| 1598/1610 [8:40:55<02:52, 14.39s/it]
 99%|█████████▉| 1599/1610 [8:41:13<02:47, 15.22s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6569076377165232, 'learning_rate': 6.832298136645962e-09, 'completion_length': 162.98214721679688, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.19514649361371994, 'kl': 0.01641845703125, 'epoch': 4.97}

 99%|█████████▉| 1599/1610 [8:41:13<02:47, 15.22s/it]
 99%|█████████▉| 1600/1610 [8:41:28<02:33, 15.31s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.6737404718940396, 'learning_rate': 6.211180124223602e-09, 'completion_length': 146.12500762939453, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.1428571529686451, 'kl': 0.0145263671875, 'epoch': 4.97}

 99%|█████████▉| 1600/1610 [8:41:28<02:33, 15.31s/it]
 99%|█████████▉| 1601/1610 [8:45:47<13:14, 88.29s/it]
                                                     
{'loss': 0.0009, 'grad_norm': 1.1350347802882819, 'learning_rate': 5.5900621118012426e-09, 'completion_length': 168.0714340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2967643290758133, 'kl': 0.02288818359375, 'epoch': 4.97}

 99%|█████████▉| 1601/1610 [8:45:47<13:14, 88.29s/it]
100%|█████████▉| 1602/1610 [8:46:03<08:52, 66.61s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.0127747709429364, 'learning_rate': 4.968944099378882e-09, 'completion_length': 150.64286041259766, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.1071428619325161, 'kl': 0.012115478515625, 'epoch': 4.98}

100%|█████████▉| 1602/1610 [8:46:03<08:52, 66.61s/it]
100%|█████████▉| 1603/1610 [8:46:18<05:59, 51.30s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.7421974739243686, 'learning_rate': 4.347826086956522e-09, 'completion_length': 151.71429443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.1785714402794838, 'kl': 0.01739501953125, 'epoch': 4.98}

100%|█████████▉| 1603/1610 [8:46:18<05:59, 51.30s/it]
100%|█████████▉| 1604/1610 [8:46:35<04:05, 40.92s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.6528715760221224, 'learning_rate': 3.726708074534161e-09, 'completion_length': 174.10714721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.07695359364151955, 'kl': 0.0169677734375, 'epoch': 4.98}

100%|█████████▉| 1604/1610 [8:46:35<04:05, 40.92s/it]
100%|█████████▉| 1605/1610 [8:46:51<02:47, 33.53s/it]
                                                     
{'loss': 0.0006, 'grad_norm': 0.874144082537712, 'learning_rate': 3.105590062111801e-09, 'completion_length': 187.30358123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.25552502274513245, 'kl': 0.014190673828125, 'epoch': 4.98}

100%|█████████▉| 1605/1610 [8:46:51<02:47, 33.53s/it]
100%|█████████▉| 1606/1610 [8:47:08<01:53, 28.45s/it]
                                                     
{'loss': 0.0008, 'grad_norm': 2.268886653926813, 'learning_rate': 2.484472049689441e-09, 'completion_length': 163.6964340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.21981073915958405, 'kl': 0.01922607421875, 'epoch': 4.99}

100%|█████████▉| 1606/1610 [8:47:08<01:53, 28.45s/it]
100%|█████████▉| 1607/1610 [8:47:23<01:13, 24.43s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.7550271277983768, 'learning_rate': 1.8633540372670804e-09, 'completion_length': 141.08929443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1785714365541935, 'kl': 0.0179443359375, 'epoch': 4.99}

100%|█████████▉| 1607/1610 [8:47:23<01:13, 24.43s/it]
100%|█████████▉| 1608/1610 [8:47:39<00:43, 21.85s/it]
                                                     
{'loss': 0.0005, 'grad_norm': 1.1112391783862485, 'learning_rate': 1.2422360248447204e-09, 'completion_length': 141.55357360839844, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.1896214708685875, 'kl': 0.013031005859375, 'epoch': 4.99}

100%|█████████▉| 1608/1610 [8:47:39<00:43, 21.85s/it]
100%|█████████▉| 1609/1610 [8:47:52<00:19, 19.38s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.8153771408451066, 'learning_rate': 6.211180124223602e-10, 'completion_length': 137.58929443359375, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.1896214708685875, 'kl': 0.01641845703125, 'epoch': 5.0}

100%|█████████▉| 1609/1610 [8:47:52<00:19, 19.38s/it]
100%|██████████| 1610/1610 [8:48:05<00:00, 17.50s/it]
                                                     
{'loss': 0.0007, 'grad_norm': 0.821200957466941, 'learning_rate': 0.0, 'completion_length': 143.125, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14838216826319695, 'kl': 0.017242431640625, 'epoch': 5.0}

100%|██████████| 1610/1610 [8:48:05<00:00, 17.50s/it]
                                                     
{'train_runtime': 31880.8409, 'train_samples_per_second': 0.707, 'train_steps_per_second': 0.051, 'train_loss': 0.0005302493254913979, 'epoch': 5.0}

100%|██████████| 1610/1610 [8:51:21<00:00, 17.50s/it]
100%|██████████| 1610/1610 [8:51:21<00:00, 19.80s/it]
wandb: 
wandb: 🚀 View run R1-Resume-COT-VLLM-Correct-Qwen2-VL-7B-GRPO-GEOQA-4k5-2025-02-23-00-58-58 at: https://wandb.ai/tanhuajie264-peking-university/vison-open-r1/runs/bg4bagha
wandb: Find logs at: wandb/run-20250223_010222-bg4bagha/logs