[2025-02-17 15:12:57,361] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,362] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,382] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,383] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,385] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,385] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-17 15:12:57,385] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. INFO 02-17 15:13:01 __init__.py:190] Automatically detected platform cuda. [2025-02-17 15:13:05,344] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,347] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,413] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,415] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,417] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,421] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:05,421] [INFO] [comm.py:683:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl [2025-02-17 15:13:05,426] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-17 15:13:07,452] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:07,453] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` p-phy-ctyun-gz-a800-node-prod-200-82:549927:549927 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:549927 [0] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549927:549927 [0] NCCL INFO cudaDriverVersion 12040 NCCL version 2.21.5+cuda12.4 p-phy-ctyun-gz-a800-node-prod-200-82:549933:549933 [6] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549933:549933 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549933:549933 [6] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Using network IBext_v8 [2025-02-17 15:13:09,092] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` [2025-02-17 15:13:09,105] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. [2025-02-17 15:13:09,112] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` p-phy-ctyun-gz-a800-node-prod-200-82:549929:549929 [2] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549929:549929 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549929:549929 [2] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` [2025-02-17 15:13:09,128] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. p-phy-ctyun-gz-a800-node-prod-200-82:549928:549928 [1] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549928:549928 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549928:549928 [1] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` [2025-02-17 15:13:09,143] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. p-phy-ctyun-gz-a800-node-prod-200-82:549931:549931 [4] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549931:549931 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549931:549931 [4] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` p-phy-ctyun-gz-a800-node-prod-200-82:549932:549932 [5] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549932:549932 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549930:549930 [3] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-82:549930:549930 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549932:549932 [5] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549930:549930 [3] NCCL INFO Bootstrap : Using bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.82<0> p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO ncclCommInitRank comm 0x55cfbeed0770 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 49000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO ncclCommInitRank comm 0x56322b138020 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 16000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO ncclCommInitRank comm 0x562924a5b660 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8a000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO ncclCommInitRank comm 0x56208dec9f20 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId c6000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO ncclCommInitRank comm 0x55d5b01507d0 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 4d000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO ncclCommInitRank comm 0x5592ec36b0b0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 8f000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO ncclCommInitRank comm 0x55e9b55fcc50 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 10000 commId 0xd160526dcbf6f44 - Init START p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO NVLS multicast support is not available on dev 0 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO NVLS multicast support is not available on dev 5 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO NVLS multicast support is not available on dev 6 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Setting affinity for GPU 3 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO NVLS multicast support is not available on dev 3 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO NVLS multicast support is not available on dev 4 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO NVLS multicast support is not available on dev 1 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Setting affinity for GPU 2 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO NVLS multicast support is not available on dev 2 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO comm 0x55d5b01507d0 rank 3 nRanks 7 nNodes 1 localRanks 7 localRank 3 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO comm 0x55cfbeed0770 rank 2 nRanks 7 nNodes 1 localRanks 7 localRank 2 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO comm 0x55e9b55fcc50 rank 0 nRanks 7 nNodes 1 localRanks 7 localRank 0 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO comm 0x56322b138020 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 00/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 01/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO comm 0x5592ec36b0b0 rank 5 nRanks 7 nNodes 1 localRanks 7 localRank 5 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 02/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 03/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 04/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 05/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO comm 0x56208dec9f20 rank 6 nRanks 7 nNodes 1 localRanks 7 localRank 6 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 06/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 07/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 08/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 09/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 10/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 11/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 12/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 13/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 14/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 15/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO comm 0x562924a5b660 rank 4 nRanks 7 nNodes 1 localRanks 7 localRank 4 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Trees [0] -1/-1/-1->6->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549931:551395 [4] NCCL INFO ncclCommInitRank comm 0x562924a5b660 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8a000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549927:551276 [0] NCCL INFO ncclCommInitRank comm 0x55e9b55fcc50 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 10000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549928:551391 [1] NCCL INFO ncclCommInitRank comm 0x56322b138020 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 16000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549930:551397 [3] NCCL INFO ncclCommInitRank comm 0x55d5b01507d0 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 4d000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549929:551393 [2] NCCL INFO ncclCommInitRank comm 0x55cfbeed0770 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 49000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-82:549932:551399 [5] NCCL INFO ncclCommInitRank comm 0x5592ec36b0b0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 8f000 commId 0xd160526dcbf6f44 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549933:551277 [6] NCCL INFO ncclCommInitRank comm 0x56208dec9f20 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId c6000 commId 0xd160526dcbf6f44 - Init COMPLETE [2025-02-17 15:13:11,079] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 730, num_elems = 2.44B [2025-02-17 15:13:17,126] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,126] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,126] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,126] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,126] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,130] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,364] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:17,637] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 1460, num_elems = 4.88B Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher. [2025-02-17 15:13:22,258] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed info: version=0.16.3, git-hash=unknown, git-branch=unknown [2025-02-17 15:13:22,258] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,258] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,259] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,259] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,259] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,260] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,261] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-17 15:13:22,275] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2025-02-17 15:13:22,278] [INFO] [logging.py:128:log_dist] [Rank 0] Creating ZeRO Offload [2025-02-17 15:13:22,543] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2025-02-17 15:13:22,544] [INFO] [utils.py:782:see_memory_usage] MA 1.19 GB Max_MA 2.49 GB CA 3.09 GB Max_CA 3 GB [2025-02-17 15:13:22,544] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 630.96 GB, percent = 62.7% Parameter Offload: Total persistent parameters: 686592 in 401 params [2025-02-17 15:13:22,768] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2025-02-17 15:13:22,768] [INFO] [utils.py:782:see_memory_usage] MA 1.19 GB Max_MA 1.19 GB CA 3.09 GB Max_CA 3 GB [2025-02-17 15:13:22,768] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 630.97 GB, percent = 62.7% [2025-02-17 15:13:22,770] [INFO] [config.py:999:print] DeepSpeedEngine configuration: [2025-02-17 15:13:22,770] [INFO] [config.py:1003:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2025-02-17 15:13:22,770] [INFO] [config.py:1003:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False} [2025-02-17 15:13:22,770] [INFO] [config.py:1003:print] amp_enabled .................. False [2025-02-17 15:13:22,770] [INFO] [config.py:1003:print] amp_params ................... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] bfloat16_enabled ............. True [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] bfloat16_immediate_grad_update False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] checkpoint_parallel_write_pipeline False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] checkpoint_tag_validation_enabled True [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] checkpoint_tag_validation_fail False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] comms_config ................. [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] communication_data_type ...... None [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] curriculum_enabled_legacy .... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] curriculum_params_legacy ..... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] data_efficiency_enabled ...... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] dataloader_drop_last ......... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] disable_allgather ............ False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] dump_state ................... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] dynamic_loss_scale_args ...... None [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] eigenvalue_enabled ........... False [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] eigenvalue_gas_boundary_resolution 1 [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] eigenvalue_layer_name ........ bert.encoder.layer [2025-02-17 15:13:22,771] [INFO] [config.py:1003:print] eigenvalue_layer_num ......... 0 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] eigenvalue_max_iter .......... 100 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] eigenvalue_stability ......... 1e-06 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] eigenvalue_tol ............... 0.01 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] eigenvalue_verbose ........... False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] elasticity_enabled ........... False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] flops_profiler_config ........ { "enabled": false, "recompute_fwd_factor": 0.0, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] fp16_auto_cast ............... None [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] fp16_enabled ................. False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] fp16_master_weights_and_gradients False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] global_rank .................. 0 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] grad_accum_dtype ............. None [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] gradient_accumulation_steps .. 2 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] gradient_clipping ............ 1.0 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] gradient_predivide_factor .... 1.0 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] graph_harvesting ............. False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] initial_dynamic_scale ........ 1 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] load_universal_checkpoint .... False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] loss_scale ................... 1.0 [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] memory_breakdown ............. False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] mics_hierarchial_params_gather False [2025-02-17 15:13:22,772] [INFO] [config.py:1003:print] mics_shard_size .............. -1 [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] optimizer_legacy_fusion ...... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] optimizer_name ............... None [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] optimizer_params ............. None [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True} [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] pld_enabled .................. False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] pld_params ................... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] prescale_gradients ........... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] scheduler_name ............... None [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] scheduler_params ............. None [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] seq_parallel_communication_data_type torch.float32 [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] sparse_attention ............. None [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] sparse_gradients_enabled ..... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] steps_per_print .............. inf [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] timers_config ................ enabled=True synchronized=True [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] train_batch_size ............. 14 [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] train_micro_batch_size_per_gpu 1 [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] use_data_before_expert_parallel_ False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] use_node_local_storage ....... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] wall_clock_breakdown ......... False [2025-02-17 15:13:22,773] [INFO] [config.py:1003:print] weight_quantization_config ... None [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] world_size ................... 7 [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] zero_allow_untested_optimizer False [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100000000, max_in_cpu=1000000000, pin_memory=True) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=True module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] zero_enabled ................. True [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] zero_force_ds_cpu_optimizer .. True [2025-02-17 15:13:22,774] [INFO] [config.py:1003:print] zero_optimization_stage ...... 3 [2025-02-17 15:13:22,774] [INFO] [config.py:989:print_user_config] json = { "fp16": { "enabled": false, "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1 }, "bf16": { "enabled": true }, "zero_optimization": { "stage": 3, "offload_optimizer": { "device": "none", "pin_memory": true }, "offload_param": { "device": "none", "pin_memory": true }, "overlap_comm": true, "contiguous_gradients": true, "sub_group_size": 1.000000e+09, "reduce_bucket_size": "auto", "stage3_prefetch_bucket_size": "auto", "stage3_param_persistence_threshold": "auto", "stage3_max_live_parameters": 1.000000e+09, "stage3_max_reuse_distance": 1.000000e+09, "stage3_gather_16bit_weights_on_model_save": true }, "gradient_accumulation_steps": 2, "gradient_clipping": 1.0, "steps_per_print": inf, "train_batch_size": 14, "train_micro_batch_size_per_gpu": 1, "wall_clock_breakdown": false, "zero_optimization.reduce_bucket_size": 2.359296e+06, "zero_optimization.stage3_param_persistence_threshold": 1.536000e+04, "zero_optimization.stage3_prefetch_bucket_size": 2.123366e+06 } INFO 02-17 15:13:38 config.py:542] This model supports multiple tasks: {'generate', 'reward', 'classify', 'score', 'embed'}. Defaulting to 'generate'. WARNING 02-17 15:13:38 arg_utils.py:1079] --enable-prefix-caching is currently not supported for multimodal models in v0 and has been disabled. INFO 02-17 15:13:38 llm_engine.py:234] Initializing a V0 LLM engine (v0.7.2) with config: model='/home/vlm/workspace/r1_checkpoints/qwen2_vl_2b_R1_finetune_by_clevr_math_correct_35k_cot_sft', speculative_config=None, tokenizer='/home/vlm/workspace/r1_checkpoints/qwen2_vl_2b_R1_finetune_by_clevr_math_correct_35k_cot_sft', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda:7, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=/home/vlm/workspace/r1_checkpoints/qwen2_vl_2b_R1_finetune_by_clevr_math_correct_35k_cot_sft, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":256}, use_cached_outputs=False, INFO 02-17 15:13:39 cuda.py:230] Using Flash Attention backend. INFO 02-17 15:13:40 model_runner.py:1110] Starting to load model /home/vlm/workspace/r1_checkpoints/qwen2_vl_2b_R1_finetune_by_clevr_math_correct_35k_cot_sft... INFO 02-17 15:13:40 config.py:2992] cudagraph sizes specified by model runner [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256] is overridden by config [256, 128, 2, 1, 4, 136, 8, 144, 16, 152, 24, 160, 32, 168, 40, 176, 48, 184, 56, 192, 64, 200, 72, 208, 80, 216, 88, 120, 224, 96, 232, 104, 240, 112, 248] Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00 4096). Running this sequence through the model will result in indexing errors WARNING 02-17 15:22:47 profiling.py:187] The context length (32768) of the model is too short to hold the multi-modal embeddings in the worst case (49152 tokens in total, out of which {'image': 16384, 'video': 32768} are reserved for multi-modal embeddings). This may cause certain multi-modal inputs to fail during inference, even when the input text is short. To avoid this, you should increase `max_model_len`, reduce `max_num_seqs`, and/or reduce `mm_counts`. INFO 02-17 15:22:51 worker.py:267] Memory profiling takes 547.65 seconds INFO 02-17 15:22:51 worker.py:267] the current vLLM instance can use total_gpu_memory (79.32GiB) x gpu_memory_utilization (0.70) = 55.53GiB INFO 02-17 15:22:51 worker.py:267] model weights take 0.00GiB; non_torch_memory takes 0.00GiB; PyTorch activation peak memory takes 0.00GiB; the rest of the memory reserved for KV Cache is 55.53GiB. INFO 02-17 15:22:51 executor_base.py:110] # CUDA blocks: 129965, # CPU blocks: 9362 INFO 02-17 15:22:51 executor_base.py:115] Maximum concurrency for 32768 tokens per request: 63.46x INFO 02-17 15:22:54 model_runner.py:1434] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI. If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage. Capturing CUDA graph shapes: 0%| | 0/35 [00:005->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO comm 0x7f48b80734d0 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO comm 0x7f64c4072560 rank 0 nRanks 7 nNodes 1 localRanks 7 localRank 0 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Trees [0] -1/-1/-1->6->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5 p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 00/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 01/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 02/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 03/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 04/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 05/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 06/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 07/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 08/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 09/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 10/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 11/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 12/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 13/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 14/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 15/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-82:549929:575433 [2] NCCL INFO ncclCommSplit comm 0x7fcff4073100 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 49000 parent 0x55cfbeed0770 color -1326228412 key 2 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549931:575432 [4] NCCL INFO ncclCommSplit comm 0x7f51e80732c0 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8a000 parent 0x562924a5b660 color -1326228412 key 4 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549930:575435 [3] NCCL INFO ncclCommSplit comm 0x7f5afc074050 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 4d000 parent 0x55d5b01507d0 color -1326228412 key 3 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549927:575437 [0] NCCL INFO ncclCommSplit comm 0x7f64c4072560 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 10000 parent 0x55e9b55fcc50 color -1326228412 key 0 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549933:575431 [6] NCCL INFO ncclCommSplit comm 0x7f418c0732c0 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId c6000 parent 0x56208dec9f20 color -1326228412 key 6 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549932:575436 [5] NCCL INFO ncclCommSplit comm 0x7f9bec072ec0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 8f000 parent 0x5592ec36b0b0 color -1326228412 key 5 commId 0x39587b66d188bc9e - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-82:549928:575434 [1] NCCL INFO ncclCommSplit comm 0x7f48b80734d0 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 16000 parent 0x56322b138020 color -1326228412 key 1 commId 0x39587b66d188bc9e - Init COMPLETE 0%| | 1/2500 [00:25<17:21:56, 25.02s/it] {'loss': 0.0, 'grad_norm': 0.6647307988652981, 'learning_rate': 9.996e-07, 'completion_length': 153.2678680419922, 'rewards/accuracy_reward': 0.8303571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8125000596046448, 'reward_std': 0.21009841561317444, 'kl': 0.0, 'epoch': 0.0} 0%| | 1/2500 [00:25<17:21:56, 25.02s/it] 0%| | 2/2500 [00:38<12:42:24, 18.31s/it] {'loss': -0.0, 'grad_norm': 0.34257768861990073, 'learning_rate': 9.992e-07, 'completion_length': 158.91964721679688, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382598757744, 'kl': -2.086162567138672e-07, 'epoch': 0.0} 0%| | 2/2500 [00:38<12:42:24, 18.31s/it] 0%| | 3/2500 [00:52<11:17:58, 16.29s/it] {'loss': -0.0, 'grad_norm': 0.577580589084214, 'learning_rate': 9.988e-07, 'completion_length': 160.16072845458984, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.22215460985898972, 'kl': -9.566545486450195e-06, 'epoch': 0.0} 0%| | 3/2500 [00:52<11:17:58, 16.29s/it] 0%| | 4/2500 [01:05<10:19:47, 14.90s/it] {'loss': -0.0, 'grad_norm': 0.3340575781772366, 'learning_rate': 9.983999999999998e-07, 'completion_length': 151.9732208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.1030978113412857, 'kl': -2.0265579223632812e-05, 'epoch': 0.0} 0%| | 4/2500 [01:05<10:19:47, 14.90s/it] 0%| | 5/2500 [01:18<9:48:38, 14.16s/it] {'loss': -0.0, 'grad_norm': 0.4526036702715222, 'learning_rate': 9.98e-07, 'completion_length': 152.92858123779297, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.128351628780365, 'kl': -4.5299530029296875e-06, 'epoch': 0.0} 0%| | 5/2500 [01:18<9:48:38, 14.16s/it] 0%| | 6/2500 [01:31<9:31:12, 13.74s/it] {'loss': 0.0, 'grad_norm': 0.706603063242287, 'learning_rate': 9.976e-07, 'completion_length': 158.56250762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.17495086789131165, 'kl': 4.295259714126587e-06, 'epoch': 0.0} 0%| | 6/2500 [01:31<9:31:12, 13.74s/it] 0%| | 7/2500 [01:43<9:13:49, 13.33s/it] {'loss': 0.0, 'grad_norm': 0.5561289291419418, 'learning_rate': 9.972e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.8928572535514832, 'reward_std': 0.13408026099205017, 'kl': 2.300739288330078e-05, 'epoch': 0.0} 0%| | 7/2500 [01:43<9:13:49, 13.33s/it] 0%| | 8/2500 [01:56<9:04:33, 13.11s/it] {'loss': 0.0, 'grad_norm': 1.5218898534325105, 'learning_rate': 9.968e-07, 'completion_length': 154.73214721679688, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.8392857909202576, 'reward_std': 0.25521962344646454, 'kl': 1.5020370483398438e-05, 'epoch': 0.0} 0%| | 8/2500 [01:56<9:04:33, 13.11s/it] 0%| | 9/2500 [02:08<8:51:10, 12.79s/it] {'loss': 0.0, 'grad_norm': 0.6127939784837472, 'learning_rate': 9.964e-07, 'completion_length': 134.35714721679688, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.1963018774986267, 'kl': 1.1101365089416504e-05, 'epoch': 0.0} 0%| | 9/2500 [02:08<8:51:10, 12.79s/it] 0%| | 10/2500 [02:21<8:59:47, 13.01s/it] {'loss': 0.0, 'grad_norm': 0.7674136446811566, 'learning_rate': 9.959999999999999e-07, 'completion_length': 164.4107208251953, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.2630308046936989, 'kl': 6.979703903198242e-05, 'epoch': 0.0} 0%| | 10/2500 [02:21<8:59:47, 13.01s/it] 0%| | 11/2500 [02:34<8:58:16, 12.98s/it] {'loss': 0.0, 'grad_norm': 0.3746286061240949, 'learning_rate': 9.956e-07, 'completion_length': 147.5446548461914, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10882644355297089, 'kl': 2.6732683181762695e-05, 'epoch': 0.0} 0%| | 11/2500 [02:34<8:58:16, 12.98s/it] 0%| | 12/2500 [02:47<8:54:32, 12.89s/it] {'loss': 0.0, 'grad_norm': 0.717156979951578, 'learning_rate': 9.952e-07, 'completion_length': 153.0803680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.17226044833660126, 'kl': 7.176399230957031e-05, 'epoch': 0.0} 0%| | 12/2500 [02:47<8:54:32, 12.89s/it] 1%| | 13/2500 [03:00<8:51:48, 12.83s/it] {'loss': 0.0, 'grad_norm': 0.5461862897857487, 'learning_rate': 9.948e-07, 'completion_length': 138.2589340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.16141663491725922, 'kl': 4.1961669921875e-05, 'epoch': 0.01} 1%| | 13/2500 [03:00<8:51:48, 12.83s/it] 1%| | 14/2500 [03:13<8:54:05, 12.89s/it] {'loss': 0.0, 'grad_norm': 1.148892894766526, 'learning_rate': 9.944e-07, 'completion_length': 150.9732208251953, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.8482143878936768, 'reward_std': 0.22364385426044464, 'kl': 0.00016689300537109375, 'epoch': 0.01} 1%| | 14/2500 [03:13<8:54:05, 12.89s/it] 1%| | 15/2500 [03:26<9:03:58, 13.13s/it] {'loss': 0.0, 'grad_norm': 0.6013269067962622, 'learning_rate': 9.94e-07, 'completion_length': 176.5089340209961, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.22875342518091202, 'kl': 0.00015592575073242188, 'epoch': 0.01} 1%| | 15/2500 [03:26<9:03:58, 13.13s/it] 1%| | 16/2500 [03:39<8:59:16, 13.03s/it] {'loss': 0.0, 'grad_norm': 0.6481185334199475, 'learning_rate': 9.936e-07, 'completion_length': 152.2589340209961, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.8571429252624512, 'reward_std': 0.2248450219631195, 'kl': 0.00014209747314453125, 'epoch': 0.01} 1%| | 16/2500 [03:39<8:59:16, 13.03s/it] 1%| | 17/2500 [03:53<9:06:04, 13.20s/it] {'loss': 0.0, 'grad_norm': 0.45405145277503506, 'learning_rate': 9.931999999999999e-07, 'completion_length': 158.9732208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.13346680253744125, 'kl': 0.00014495849609375, 'epoch': 0.01} 1%| | 17/2500 [03:53<9:06:04, 13.20s/it] 1%| | 18/2500 [04:05<8:59:20, 13.04s/it] {'loss': 0.0, 'grad_norm': 0.7191055036137959, 'learning_rate': 9.928e-07, 'completion_length': 143.63394165039062, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.2501044496893883, 'kl': 0.00013208389282226562, 'epoch': 0.01} 1%| | 18/2500 [04:05<8:59:20, 13.04s/it] 1%| | 19/2500 [04:18<8:55:20, 12.95s/it] {'loss': 0.0, 'grad_norm': 0.5237042194774787, 'learning_rate': 9.923999999999998e-07, 'completion_length': 139.93750762939453, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.08868780359625816, 'kl': 0.00020551681518554688, 'epoch': 0.01} 1%| | 19/2500 [04:18<8:55:20, 12.95s/it] 1%| | 20/2500 [04:31<8:54:06, 12.92s/it] {'loss': 0.0, 'grad_norm': 0.5814309550099412, 'learning_rate': 9.92e-07, 'completion_length': 160.46429443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.18397442996501923, 'kl': 0.00021028518676757812, 'epoch': 0.01} 1%| | 20/2500 [04:31<8:54:06, 12.92s/it] 1%| | 21/2500 [04:44<8:54:00, 12.92s/it] {'loss': 0.0, 'grad_norm': 0.5230602855849673, 'learning_rate': 9.916e-07, 'completion_length': 144.1875, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.15872061997652054, 'kl': 0.0001735687255859375, 'epoch': 0.01} 1%| | 21/2500 [04:44<8:54:00, 12.92s/it] 1%| | 22/2500 [04:58<9:09:52, 13.31s/it] {'loss': 0.0, 'grad_norm': 0.5999037734882178, 'learning_rate': 9.912e-07, 'completion_length': 161.46429443359375, 'rewards/accuracy_reward': 0.8482142984867096, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.1030978113412857, 'kl': 0.0002617835998535156, 'epoch': 0.01} 1%| | 22/2500 [04:58<9:09:52, 13.31s/it] 1%| | 23/2500 [05:11<9:06:15, 13.23s/it] {'loss': 0.0, 'grad_norm': 0.6778276757728413, 'learning_rate': 9.908e-07, 'completion_length': 156.3839340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.25279486179351807, 'kl': 0.0002460479736328125, 'epoch': 0.01} 1%| | 23/2500 [05:11<9:06:15, 13.23s/it] 1%| | 24/2500 [05:24<9:07:14, 13.26s/it] {'loss': 0.0, 'grad_norm': 0.5006583392220283, 'learning_rate': 9.903999999999999e-07, 'completion_length': 154.8839340209961, 'rewards/accuracy_reward': 0.8125000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8125000596046448, 'reward_std': 0.1379830539226532, 'kl': 0.00027370452880859375, 'epoch': 0.01} 1%| | 24/2500 [05:24<9:07:14, 13.26s/it] 1%| | 25/2500 [05:37<8:53:00, 12.92s/it] {'loss': 0.0, 'grad_norm': 0.7986930642180466, 'learning_rate': 9.9e-07, 'completion_length': 143.45536422729492, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.1827620416879654, 'kl': 0.0003223419189453125, 'epoch': 0.01} 1%| | 25/2500 [05:37<8:53:00, 12.92s/it] 1%| | 26/2500 [05:49<8:45:12, 12.74s/it] {'loss': 0.0, 'grad_norm': 0.4990853253052836, 'learning_rate': 9.896e-07, 'completion_length': 147.12500762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.1866704523563385, 'kl': 0.00023031234741210938, 'epoch': 0.01} 1%| | 26/2500 [05:49<8:45:12, 12.74s/it] 1%| | 27/2500 [06:02<8:53:26, 12.94s/it] {'loss': 0.0, 'grad_norm': 0.7192091483691633, 'learning_rate': 9.892e-07, 'completion_length': 166.16072845458984, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.17434298992156982, 'kl': 0.0002741813659667969, 'epoch': 0.01} 1%| | 27/2500 [06:02<8:53:26, 12.94s/it] 1%| | 28/2500 [06:15<8:53:32, 12.95s/it] {'loss': 0.0, 'grad_norm': 1.3803339667021606, 'learning_rate': 9.888e-07, 'completion_length': 154.68750762939453, 'rewards/accuracy_reward': 0.8303571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8303572535514832, 'reward_std': 0.23326967656612396, 'kl': 0.00029754638671875, 'epoch': 0.01} 1%| | 28/2500 [06:15<8:53:32, 12.95s/it] 1%| | 29/2500 [06:28<8:51:31, 12.91s/it] {'loss': 0.0, 'grad_norm': 0.7383876188617958, 'learning_rate': 9.884e-07, 'completion_length': 137.64286041259766, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10040178522467613, 'kl': 0.0002460479736328125, 'epoch': 0.01} 1%| | 29/2500 [06:28<8:51:31, 12.91s/it] 1%| | 30/2500 [06:41<8:57:36, 13.06s/it] {'loss': 0.0, 'grad_norm': 1.408580221805728, 'learning_rate': 9.88e-07, 'completion_length': 154.00000762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.20020468533039093, 'kl': 0.00039005279541015625, 'epoch': 0.01} 1%| | 30/2500 [06:42<8:57:36, 13.06s/it] 1%| | 31/2500 [06:54<8:53:00, 12.95s/it] {'loss': 0.0, 'grad_norm': 0.8226424250618992, 'learning_rate': 9.876e-07, 'completion_length': 140.61608123779297, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.1412779912352562, 'kl': 0.000392913818359375, 'epoch': 0.01} 1%| | 31/2500 [06:54<8:53:00, 12.95s/it] 1%|▏ | 32/2500 [07:07<8:55:41, 13.02s/it] {'loss': 0.0, 'grad_norm': 0.4301559212285345, 'learning_rate': 9.871999999999998e-07, 'completion_length': 145.83036041259766, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.10882644727826118, 'kl': 0.00038623809814453125, 'epoch': 0.01} 1%|▏ | 32/2500 [07:07<8:55:41, 13.02s/it] 1%|▏ | 33/2500 [07:20<8:53:31, 12.98s/it] {'loss': 0.0, 'grad_norm': 0.7540571765819915, 'learning_rate': 9.868e-07, 'completion_length': 158.92857360839844, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.22094223648309708, 'kl': 0.00047969818115234375, 'epoch': 0.01} 1%|▏ | 33/2500 [07:20<8:53:31, 12.98s/it] 1%|▏ | 34/2500 [07:33<8:52:29, 12.96s/it] {'loss': 0.0, 'grad_norm': 0.3771057624921979, 'learning_rate': 9.864e-07, 'completion_length': 144.15179443359375, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.10821298509836197, 'kl': 0.0005970001220703125, 'epoch': 0.01} 1%|▏ | 34/2500 [07:33<8:52:29, 12.96s/it] 1%|▏ | 35/2500 [07:47<9:01:18, 13.18s/it] {'loss': 0.0, 'grad_norm': 0.6259515372094137, 'learning_rate': 9.86e-07, 'completion_length': 166.2053680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.15993299335241318, 'kl': 0.0005044937133789062, 'epoch': 0.01} 1%|▏ | 35/2500 [07:47<9:01:18, 13.18s/it] 1%|▏ | 36/2500 [08:00<9:02:42, 13.22s/it] {'loss': 0.0, 'grad_norm': 1.222053768645087, 'learning_rate': 9.856e-07, 'completion_length': 157.08929443359375, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.22363825142383575, 'kl': 0.00043201446533203125, 'epoch': 0.01} 1%|▏ | 36/2500 [08:00<9:02:42, 13.22s/it] 1%|▏ | 37/2500 [08:14<9:10:03, 13.40s/it] {'loss': 0.0, 'grad_norm': 0.7517109995952175, 'learning_rate': 9.852e-07, 'completion_length': 146.3839340209961, 'rewards/accuracy_reward': 0.8125000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8125001192092896, 'reward_std': 0.16774418950080872, 'kl': 0.0005159378051757812, 'epoch': 0.01} 1%|▏ | 37/2500 [08:14<9:10:03, 13.40s/it] 2%|▏ | 38/2500 [08:26<8:55:37, 13.05s/it] {'loss': 0.0, 'grad_norm': 0.4939602234357431, 'learning_rate': 9.847999999999999e-07, 'completion_length': 148.3214340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1322600245475769, 'kl': 0.000530242919921875, 'epoch': 0.02} 2%|▏ | 38/2500 [08:26<8:55:37, 13.05s/it] 2%|▏ | 39/2500 [08:38<8:39:59, 12.68s/it] {'loss': 0.0, 'grad_norm': 0.9315856250505758, 'learning_rate': 9.844e-07, 'completion_length': 133.05358123779297, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.22754104435443878, 'kl': 0.0008563995361328125, 'epoch': 0.02} 2%|▏ | 39/2500 [08:38<8:39:59, 12.68s/it] 2%|▏ | 40/2500 [08:50<8:34:32, 12.55s/it] {'loss': 0.0, 'grad_norm': 0.6953009960251809, 'learning_rate': 9.84e-07, 'completion_length': 136.76786041259766, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.16531942784786224, 'kl': 0.0005817413330078125, 'epoch': 0.02} 2%|▏ | 40/2500 [08:50<8:34:32, 12.55s/it] 2%|▏ | 41/2500 [09:03<8:38:56, 12.66s/it] {'loss': 0.0, 'grad_norm': 0.5587237211140297, 'learning_rate': 9.836e-07, 'completion_length': 155.30357360839844, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.12956400960683823, 'kl': 0.0005970001220703125, 'epoch': 0.02} 2%|▏ | 41/2500 [09:03<8:38:56, 12.66s/it] 2%|▏ | 42/2500 [09:17<8:54:45, 13.05s/it] {'loss': 0.0, 'grad_norm': 0.4657675405329792, 'learning_rate': 9.832e-07, 'completion_length': 157.7857208251953, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.19690078496932983, 'kl': 0.0006866455078125, 'epoch': 0.02} 2%|▏ | 42/2500 [09:17<8:54:45, 13.05s/it] 2%|▏ | 43/2500 [09:32<9:13:44, 13.52s/it] {'loss': 0.0, 'grad_norm': 1.333477058671445, 'learning_rate': 9.828e-07, 'completion_length': 172.31250762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.2404673993587494, 'kl': 0.000835418701171875, 'epoch': 0.02} 2%|▏ | 43/2500 [09:32<9:13:44, 13.52s/it] 2%|▏ | 44/2500 [09:44<9:01:01, 13.22s/it] {'loss': 0.0, 'grad_norm': 0.7812059537842402, 'learning_rate': 9.824e-07, 'completion_length': 144.90179443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.1509094201028347, 'kl': 0.0007038116455078125, 'epoch': 0.02} 2%|▏ | 44/2500 [09:44<9:01:01, 13.22s/it] 2%|▏ | 45/2500 [09:56<8:48:21, 12.91s/it] {'loss': 0.0, 'grad_norm': 0.41881080089597666, 'learning_rate': 9.819999999999999e-07, 'completion_length': 133.56250762939453, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.09138382598757744, 'kl': 0.00047969818115234375, 'epoch': 0.02} 2%|▏ | 45/2500 [09:57<8:48:21, 12.91s/it] 2%|▏ | 46/2500 [10:08<8:36:17, 12.62s/it] {'loss': 0.0, 'grad_norm': 0.3287899411425449, 'learning_rate': 9.816e-07, 'completion_length': 141.3571548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0006561279296875, 'epoch': 0.02} 2%|▏ | 46/2500 [10:08<8:36:17, 12.62s/it] 2%|▏ | 47/2500 [10:23<8:58:12, 13.16s/it] {'loss': 0.0, 'grad_norm': 0.7600080191114982, 'learning_rate': 9.811999999999998e-07, 'completion_length': 159.26786041259766, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.20080357789993286, 'kl': 0.0007572174072265625, 'epoch': 0.02} 2%|▏ | 47/2500 [10:23<8:58:12, 13.16s/it] 2%|▏ | 48/2500 [10:35<8:44:43, 12.84s/it] {'loss': 0.0, 'grad_norm': 0.508742982667168, 'learning_rate': 9.808e-07, 'completion_length': 141.06250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.0964989960193634, 'kl': 0.0007572174072265625, 'epoch': 0.02} 2%|▏ | 48/2500 [10:35<8:44:43, 12.84s/it] 2%|▏ | 49/2500 [10:48<8:43:16, 12.81s/it] {'loss': 0.0, 'grad_norm': 1.150084968829166, 'learning_rate': 9.804e-07, 'completion_length': 145.5357208251953, 'rewards/accuracy_reward': 0.7767857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7767857909202576, 'reward_std': 0.18214858323335648, 'kl': 0.0009765625, 'epoch': 0.02} 2%|▏ | 49/2500 [10:48<8:43:16, 12.81s/it] 2%|▏ | 50/2500 [11:00<8:41:20, 12.77s/it] {'loss': 0.0, 'grad_norm': 0.5105990373279871, 'learning_rate': 9.8e-07, 'completion_length': 145.70536041259766, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.10882644355297089, 'kl': 0.00087738037109375, 'epoch': 0.02} 2%|▏ | 50/2500 [11:00<8:41:20, 12.77s/it] 2%|▏ | 51/2500 [11:13<8:35:35, 12.63s/it] {'loss': 0.0, 'grad_norm': 0.570126683881883, 'learning_rate': 9.796e-07, 'completion_length': 152.55358123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.11394161731004715, 'kl': 0.000835418701171875, 'epoch': 0.02} 2%|▏ | 51/2500 [11:13<8:35:35, 12.63s/it] 2%|▏ | 52/2500 [11:25<8:37:27, 12.68s/it] {'loss': 0.0, 'grad_norm': 1.058664562847347, 'learning_rate': 9.791999999999999e-07, 'completion_length': 154.6964340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.20801587402820587, 'kl': 0.0010395050048828125, 'epoch': 0.02} 2%|▏ | 52/2500 [11:25<8:37:27, 12.68s/it] 2%|▏ | 53/2500 [11:38<8:32:14, 12.56s/it] {'loss': 0.0, 'grad_norm': 0.45976066219678424, 'learning_rate': 9.788e-07, 'completion_length': 145.5982208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.1379830539226532, 'kl': 0.0008087158203125, 'epoch': 0.02} 2%|▏ | 53/2500 [11:38<8:32:14, 12.56s/it] 2%|▏ | 54/2500 [11:50<8:32:17, 12.57s/it] {'loss': 0.0, 'grad_norm': 0.21348601321661737, 'learning_rate': 9.784e-07, 'completion_length': 144.3928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.000827789306640625, 'epoch': 0.02} 2%|▏ | 54/2500 [11:50<8:32:17, 12.57s/it] 2%|▏ | 55/2500 [12:04<8:41:05, 12.79s/it] {'loss': 0.0, 'grad_norm': 0.5157423229424596, 'learning_rate': 9.78e-07, 'completion_length': 149.08929443359375, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.14579425752162933, 'kl': 0.0009708404541015625, 'epoch': 0.02} 2%|▏ | 55/2500 [12:04<8:41:05, 12.79s/it] 2%|▏ | 56/2500 [12:17<8:47:29, 12.95s/it] {'loss': 0.0, 'grad_norm': 0.29286740987317, 'learning_rate': 9.776e-07, 'completion_length': 152.41964721679688, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831881985068321, 'kl': 0.0010356903076171875, 'epoch': 0.02} 2%|▏ | 56/2500 [12:17<8:47:29, 12.95s/it] 2%|▏ | 57/2500 [12:30<8:50:15, 13.02s/it] {'loss': 0.0, 'grad_norm': 0.8441130971074334, 'learning_rate': 9.772e-07, 'completion_length': 153.56250762939453, 'rewards/accuracy_reward': 0.7767857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7767857909202576, 'reward_std': 0.23265621066093445, 'kl': 0.00103759765625, 'epoch': 0.02} 2%|▏ | 57/2500 [12:30<8:50:15, 13.02s/it] 2%|▏ | 58/2500 [12:44<8:55:11, 13.15s/it] {'loss': 0.0001, 'grad_norm': 0.8007046435148755, 'learning_rate': 9.768e-07, 'completion_length': 159.16964721679688, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.2540072202682495, 'kl': 0.001422882080078125, 'epoch': 0.02} 2%|▏ | 58/2500 [12:44<8:55:11, 13.15s/it] 2%|▏ | 59/2500 [12:57<8:55:10, 13.15s/it] {'loss': 0.0001, 'grad_norm': 1.4850141582382446, 'learning_rate': 9.764e-07, 'completion_length': 148.7857208251953, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.22363825142383575, 'kl': 0.001399993896484375, 'epoch': 0.02} 2%|▏ | 59/2500 [12:57<8:55:10, 13.15s/it] 2%|▏ | 60/2500 [13:10<8:51:23, 13.07s/it] {'loss': 0.0001, 'grad_norm': 1.1991319371032054, 'learning_rate': 9.759999999999998e-07, 'completion_length': 153.8839340209961, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.758928656578064, 'reward_std': 0.212715532630682, 'kl': 0.00133514404296875, 'epoch': 0.02} 2%|▏ | 60/2500 [13:10<8:51:23, 13.07s/it] 2%|▏ | 61/2500 [13:22<8:48:16, 13.00s/it] {'loss': 0.0001, 'grad_norm': 0.5882617508237178, 'learning_rate': 9.756e-07, 'completion_length': 147.33036041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.12444322556257248, 'kl': 0.0012969970703125, 'epoch': 0.02} 2%|▏ | 61/2500 [13:22<8:48:16, 13.00s/it] 2%|▏ | 62/2500 [13:36<8:51:29, 13.08s/it] {'loss': 0.0001, 'grad_norm': 1.0500899925029337, 'learning_rate': 9.752e-07, 'completion_length': 158.20536041259766, 'rewards/accuracy_reward': 0.830357164144516, 'rewards/format_reward': 1.0, 'reward': 1.8303571939468384, 'reward_std': 0.15872061252593994, 'kl': 0.00142669677734375, 'epoch': 0.02} 2%|▏ | 62/2500 [13:36<8:51:29, 13.08s/it] 3%|▎ | 63/2500 [13:49<8:53:39, 13.14s/it] {'loss': 0.0001, 'grad_norm': 0.6077216124383593, 'learning_rate': 9.748e-07, 'completion_length': 151.83036041259766, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.15360544621944427, 'kl': 0.001617431640625, 'epoch': 0.03} 3%|▎ | 63/2500 [13:49<8:53:39, 13.14s/it] 3%|▎ | 64/2500 [14:01<8:44:09, 12.91s/it] {'loss': 0.0, 'grad_norm': 0.8302718643390132, 'learning_rate': 9.744e-07, 'completion_length': 141.2589340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12956400960683823, 'kl': 0.00121307373046875, 'epoch': 0.03} 3%|▎ | 64/2500 [14:01<8:44:09, 12.91s/it] 3%|▎ | 65/2500 [14:14<8:38:05, 12.77s/it] {'loss': 0.0, 'grad_norm': 1.8905822342131584, 'learning_rate': 9.74e-07, 'completion_length': 146.87500762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.12444322556257248, 'kl': 0.001117706298828125, 'epoch': 0.03} 3%|▎ | 65/2500 [14:14<8:38:05, 12.77s/it] 3%|▎ | 66/2500 [14:27<8:44:08, 12.92s/it] {'loss': 0.0001, 'grad_norm': 1.3240708573922417, 'learning_rate': 9.735999999999999e-07, 'completion_length': 154.40179443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.001537322998046875, 'epoch': 0.03} 3%|▎ | 66/2500 [14:27<8:44:08, 12.92s/it] 3%|▎ | 67/2500 [14:40<8:43:39, 12.91s/it] {'loss': 0.0001, 'grad_norm': 0.5869490621302308, 'learning_rate': 9.731999999999998e-07, 'completion_length': 159.68750762939453, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.17226044833660126, 'kl': 0.001590728759765625, 'epoch': 0.03} 3%|▎ | 67/2500 [14:40<8:43:39, 12.91s/it] 3%|▎ | 68/2500 [14:53<8:38:26, 12.79s/it] {'loss': 0.0, 'grad_norm': 0.8840451661681501, 'learning_rate': 9.728e-07, 'completion_length': 147.1607208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.12444322928786278, 'kl': 0.001148223876953125, 'epoch': 0.03} 3%|▎ | 68/2500 [14:53<8:38:26, 12.79s/it] 3%|▎ | 69/2500 [15:05<8:34:22, 12.70s/it] {'loss': 0.0, 'grad_norm': 0.4598131902931717, 'learning_rate': 9.724e-07, 'completion_length': 139.79464721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0011157989501953125, 'epoch': 0.03} 3%|▎ | 69/2500 [15:05<8:34:22, 12.70s/it] 3%|▎ | 70/2500 [15:18<8:35:18, 12.72s/it] {'loss': 0.0001, 'grad_norm': 1.2571399724344317, 'learning_rate': 9.72e-07, 'completion_length': 150.0714340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.14579425752162933, 'kl': 0.00145721435546875, 'epoch': 0.03} 3%|▎ | 70/2500 [15:18<8:35:18, 12.72s/it] 3%|▎ | 71/2500 [15:31<8:45:02, 12.97s/it] {'loss': 0.0001, 'grad_norm': 0.8300265248012519, 'learning_rate': 9.716e-07, 'completion_length': 172.08929443359375, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.20141705125570297, 'kl': 0.0018463134765625, 'epoch': 0.03} 3%|▎ | 71/2500 [15:31<8:45:02, 12.97s/it] 3%|▎ | 72/2500 [15:46<8:59:43, 13.34s/it] {'loss': 0.0001, 'grad_norm': 0.9845280092317084, 'learning_rate': 9.712e-07, 'completion_length': 162.37500762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.23326969146728516, 'kl': 0.00173187255859375, 'epoch': 0.03} 3%|▎ | 72/2500 [15:46<8:59:43, 13.34s/it] 3%|▎ | 73/2500 [15:58<8:45:23, 12.99s/it] {'loss': 0.0, 'grad_norm': 0.21533930896647083, 'learning_rate': 9.707999999999999e-07, 'completion_length': 128.08036041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.001247406005859375, 'epoch': 0.03} 3%|▎ | 73/2500 [15:58<8:45:23, 12.99s/it] 3%|▎ | 74/2500 [16:10<8:36:02, 12.76s/it] {'loss': 0.0001, 'grad_norm': 0.39783637240950764, 'learning_rate': 9.704e-07, 'completion_length': 148.49107360839844, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.05050762742757797, 'kl': 0.001911163330078125, 'epoch': 0.03} 3%|▎ | 74/2500 [16:10<8:36:02, 12.76s/it] 3%|▎ | 75/2500 [16:24<8:46:40, 13.03s/it] {'loss': 0.0001, 'grad_norm': 1.4475535875099785, 'learning_rate': 9.7e-07, 'completion_length': 158.1607208251953, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1963018923997879, 'kl': 0.001674652099609375, 'epoch': 0.03} 3%|▎ | 75/2500 [16:24<8:46:40, 13.03s/it] 3%|▎ | 76/2500 [16:37<8:45:30, 13.01s/it] {'loss': 0.0001, 'grad_norm': 0.5032768639215688, 'learning_rate': 9.696e-07, 'completion_length': 164.60714721679688, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.11663764715194702, 'kl': 0.001689910888671875, 'epoch': 0.03} 3%|▎ | 76/2500 [16:37<8:45:30, 13.01s/it] 3%|▎ | 77/2500 [16:49<8:39:23, 12.86s/it] {'loss': 0.0001, 'grad_norm': 0.43956968496924875, 'learning_rate': 9.692e-07, 'completion_length': 139.62500762939453, 'rewards/accuracy_reward': 0.8125000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8125000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.001255035400390625, 'epoch': 0.03} 3%|▎ | 77/2500 [16:49<8:39:23, 12.86s/it] 3%|▎ | 78/2500 [17:02<8:36:39, 12.80s/it] {'loss': 0.0001, 'grad_norm': 1.2830107375665611, 'learning_rate': 9.688e-07, 'completion_length': 155.16964721679688, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.1283516250550747, 'kl': 0.001800537109375, 'epoch': 0.03} 3%|▎ | 78/2500 [17:02<8:36:39, 12.80s/it] 3%|▎ | 79/2500 [17:14<8:28:20, 12.60s/it] {'loss': 0.0001, 'grad_norm': 1.9065403162054433, 'learning_rate': 9.684e-07, 'completion_length': 134.46428680419922, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.20020467787981033, 'kl': 0.00170135498046875, 'epoch': 0.03} 3%|▎ | 79/2500 [17:14<8:28:20, 12.60s/it] 3%|▎ | 80/2500 [17:29<8:57:03, 13.32s/it] {'loss': 0.0001, 'grad_norm': 0.957550678322686, 'learning_rate': 9.679999999999999e-07, 'completion_length': 175.46429443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.22363823652267456, 'kl': 0.001922607421875, 'epoch': 0.03} 3%|▎ | 80/2500 [17:29<8:57:03, 13.32s/it] 3%|▎ | 81/2500 [17:41<8:45:23, 13.03s/it] {'loss': 0.0001, 'grad_norm': 0.4794736095802302, 'learning_rate': 9.676e-07, 'completion_length': 148.43750762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11663763970136642, 'kl': 0.00196075439453125, 'epoch': 0.03} 3%|▎ | 81/2500 [17:41<8:45:23, 13.03s/it] 3%|▎ | 82/2500 [17:54<8:46:13, 13.06s/it] {'loss': 0.0001, 'grad_norm': 0.8661722258625842, 'learning_rate': 9.671999999999998e-07, 'completion_length': 156.33036041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.14579425007104874, 'kl': 0.00186920166015625, 'epoch': 0.03} 3%|▎ | 82/2500 [17:54<8:46:13, 13.06s/it] 3%|▎ | 83/2500 [18:07<8:37:56, 12.86s/it] {'loss': 0.0001, 'grad_norm': 0.6489761741414566, 'learning_rate': 9.668e-07, 'completion_length': 145.5982208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.001834869384765625, 'epoch': 0.03} 3%|▎ | 83/2500 [18:07<8:37:56, 12.86s/it] 3%|▎ | 84/2500 [18:20<8:43:46, 13.01s/it] {'loss': 0.0001, 'grad_norm': 15.206879660688044, 'learning_rate': 9.664e-07, 'completion_length': 143.0089340209961, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.821428656578064, 'reward_std': 0.24558256566524506, 'kl': 0.00165557861328125, 'epoch': 0.03} 3%|▎ | 84/2500 [18:20<8:43:46, 13.01s/it] 3%|▎ | 85/2500 [18:33<8:43:17, 13.00s/it] {'loss': 0.0001, 'grad_norm': 0.5074205898813064, 'learning_rate': 9.66e-07, 'completion_length': 154.2946548461914, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12956400960683823, 'kl': 0.00177764892578125, 'epoch': 0.03} 3%|▎ | 85/2500 [18:33<8:43:17, 13.00s/it] 3%|▎ | 86/2500 [18:46<8:37:24, 12.86s/it] {'loss': 0.0001, 'grad_norm': 0.2818476600672667, 'learning_rate': 9.656e-07, 'completion_length': 151.49107360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00188446044921875, 'epoch': 0.03} 3%|▎ | 86/2500 [18:46<8:37:24, 12.86s/it] 3%|▎ | 87/2500 [18:58<8:34:01, 12.78s/it] {'loss': 0.0001, 'grad_norm': 1.3971912249398362, 'learning_rate': 9.651999999999999e-07, 'completion_length': 164.9464340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.0964989960193634, 'kl': 0.0023193359375, 'epoch': 0.03} 3%|▎ | 87/2500 [18:58<8:34:01, 12.78s/it] 4%|▎ | 88/2500 [19:11<8:33:27, 12.77s/it] {'loss': 0.0001, 'grad_norm': 0.5435370173084885, 'learning_rate': 9.647999999999999e-07, 'completion_length': 145.0, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.1704346016049385, 'kl': 0.00183868408203125, 'epoch': 0.04} 4%|▎ | 88/2500 [19:11<8:33:27, 12.77s/it] 4%|▎ | 89/2500 [19:24<8:37:41, 12.88s/it] {'loss': 0.0001, 'grad_norm': 0.8282158347844985, 'learning_rate': 9.644e-07, 'completion_length': 151.2232208251953, 'rewards/accuracy_reward': 0.830357164144516, 'rewards/format_reward': 1.0, 'reward': 1.8303572535514832, 'reward_std': 0.17104807496070862, 'kl': 0.0016021728515625, 'epoch': 0.04} 4%|▎ | 89/2500 [19:24<8:37:41, 12.88s/it] 4%|▎ | 90/2500 [19:39<9:04:23, 13.55s/it] {'loss': 0.0001, 'grad_norm': 0.39823137433306305, 'learning_rate': 9.64e-07, 'completion_length': 175.5357208251953, 'rewards/accuracy_reward': 0.7767857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7767857909202576, 'reward_std': 0.17554977536201477, 'kl': 0.00228118896484375, 'epoch': 0.04} 4%|▎ | 90/2500 [19:39<9:04:23, 13.55s/it] 4%|▎ | 91/2500 [19:52<8:55:59, 13.35s/it] {'loss': 0.0001, 'grad_norm': 0.6193641496490627, 'learning_rate': 9.636e-07, 'completion_length': 157.50000762939453, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1995968073606491, 'kl': 0.002056121826171875, 'epoch': 0.04} 4%|▎ | 91/2500 [19:52<8:55:59, 13.35s/it] 4%|▎ | 92/2500 [20:05<8:47:30, 13.14s/it] {'loss': 0.0001, 'grad_norm': 0.6432038124737164, 'learning_rate': 9.632e-07, 'completion_length': 158.90179443359375, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.17226044833660126, 'kl': 0.00238800048828125, 'epoch': 0.04} 4%|▎ | 92/2500 [20:05<8:47:30, 13.14s/it] 4%|▎ | 93/2500 [20:18<8:43:58, 13.06s/it] {'loss': 0.0001, 'grad_norm': 0.4945644178395041, 'learning_rate': 9.628e-07, 'completion_length': 163.92857360839844, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.14579425379633904, 'kl': 0.002185821533203125, 'epoch': 0.04} 4%|▎ | 93/2500 [20:18<8:43:58, 13.06s/it] 4%|▍ | 94/2500 [20:31<8:42:07, 13.02s/it] {'loss': 0.0001, 'grad_norm': 0.44699681569459987, 'learning_rate': 9.624e-07, 'completion_length': 153.05357360839844, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.10882644727826118, 'kl': 0.0021514892578125, 'epoch': 0.04} 4%|▍ | 94/2500 [20:31<8:42:07, 13.02s/it] 4%|▍ | 95/2500 [20:44<8:45:52, 13.12s/it] {'loss': 0.0001, 'grad_norm': 0.39010981737483724, 'learning_rate': 9.619999999999999e-07, 'completion_length': 169.86608123779297, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.12054043635725975, 'kl': 0.00231170654296875, 'epoch': 0.04} 4%|▍ | 95/2500 [20:44<8:45:52, 13.12s/it] 4%|▍ | 96/2500 [20:56<8:38:22, 12.94s/it] {'loss': 0.0001, 'grad_norm': 0.463811860769778, 'learning_rate': 9.616e-07, 'completion_length': 136.8839340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.1418914496898651, 'kl': 0.001735687255859375, 'epoch': 0.04} 4%|▍ | 96/2500 [20:56<8:38:22, 12.94s/it] 4%|▍ | 97/2500 [21:09<8:37:54, 12.93s/it] {'loss': 0.0001, 'grad_norm': 0.3186986936820928, 'learning_rate': 9.612e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.001895904541015625, 'epoch': 0.04} 4%|▍ | 97/2500 [21:09<8:37:54, 12.93s/it] 4%|▍ | 98/2500 [21:22<8:36:40, 12.91s/it] {'loss': 0.0001, 'grad_norm': 0.9065533559479095, 'learning_rate': 9.608e-07, 'completion_length': 154.96429443359375, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.18788282573223114, 'kl': 0.0025482177734375, 'epoch': 0.04} 4%|▍ | 98/2500 [21:22<8:36:40, 12.91s/it] 4%|▍ | 99/2500 [21:34<8:27:23, 12.68s/it] {'loss': 0.0001, 'grad_norm': 0.5440947995732379, 'learning_rate': 9.604e-07, 'completion_length': 141.33929443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.12175282090902328, 'kl': 0.00183868408203125, 'epoch': 0.04} 4%|▍ | 99/2500 [21:34<8:27:23, 12.68s/it] 4%|▍ | 100/2500 [21:46<8:19:09, 12.48s/it] {'loss': 0.0001, 'grad_norm': 0.33554188066671564, 'learning_rate': 9.6e-07, 'completion_length': 145.48214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00222015380859375, 'epoch': 0.04} 4%|▍ | 100/2500 [21:46<8:19:09, 12.48s/it] 4%|▍ | 101/2500 [23:25<25:28:38, 38.23s/it] {'loss': 0.0001, 'grad_norm': 0.6037280474515737, 'learning_rate': 9.595999999999999e-07, 'completion_length': 161.51786041259766, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.12565560638904572, 'kl': 0.002300262451171875, 'epoch': 0.04} 4%|▍ | 101/2500 [23:25<25:28:38, 38.23s/it] 4%|▍ | 102/2500 [23:48<22:32:12, 33.83s/it] {'loss': 0.0001, 'grad_norm': 0.5541879986439087, 'learning_rate': 9.592e-07, 'completion_length': 161.5714340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747543394565582, 'kl': 0.00228118896484375, 'epoch': 0.04} 4%|▍ | 102/2500 [23:48<22:32:12, 33.83s/it] 4%|▍ | 103/2500 [24:13<20:43:00, 31.11s/it] {'loss': 0.0001, 'grad_norm': 0.4129380894830744, 'learning_rate': 9.588e-07, 'completion_length': 157.26786041259766, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.06222161278128624, 'kl': 0.002033233642578125, 'epoch': 0.04} 4%|▍ | 103/2500 [24:13<20:43:00, 31.11s/it] 4%|▍ | 104/2500 [24:38<19:35:23, 29.43s/it] {'loss': 0.0001, 'grad_norm': 0.5047354012016491, 'learning_rate': 9.584e-07, 'completion_length': 153.23214721679688, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.1800716370344162, 'kl': 0.00292205810546875, 'epoch': 0.04} 4%|▍ | 104/2500 [24:38<19:35:23, 29.43s/it] 4%|▍ | 105/2500 [25:02<18:24:54, 27.68s/it] {'loss': 0.0001, 'grad_norm': 1.2591494537480952, 'learning_rate': 9.58e-07, 'completion_length': 141.77678680419922, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.12444322556257248, 'kl': 0.0026092529296875, 'epoch': 0.04} 4%|▍ | 105/2500 [25:02<18:24:54, 27.68s/it] 4%|▍ | 106/2500 [25:26<17:42:55, 26.64s/it] {'loss': 0.0001, 'grad_norm': 0.5961962133359808, 'learning_rate': 9.576e-07, 'completion_length': 153.51786041259766, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.16141663491725922, 'kl': 0.0032958984375, 'epoch': 0.04} 4%|▍ | 106/2500 [25:26<17:42:55, 26.64s/it] 4%|▍ | 107/2500 [25:53<17:41:34, 26.62s/it] {'loss': 0.0001, 'grad_norm': 0.42999432134811266, 'learning_rate': 9.572e-07, 'completion_length': 158.1428680419922, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.12444322928786278, 'kl': 0.00247955322265625, 'epoch': 0.04} 4%|▍ | 107/2500 [25:53<17:41:34, 26.62s/it] 4%|▍ | 108/2500 [26:18<17:17:50, 26.03s/it] {'loss': 0.0001, 'grad_norm': 1.0836180730829426, 'learning_rate': 9.567999999999999e-07, 'completion_length': 158.89286041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.15933407843112946, 'kl': 0.00273895263671875, 'epoch': 0.04} 4%|▍ | 108/2500 [26:18<17:17:50, 26.03s/it] 4%|▍ | 109/2500 [26:42<16:54:09, 25.45s/it] {'loss': 0.0001, 'grad_norm': 0.45114283795432775, 'learning_rate': 9.564e-07, 'completion_length': 156.2678680419922, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.13225442171096802, 'kl': 0.00223541259765625, 'epoch': 0.04} 4%|▍ | 109/2500 [26:42<16:54:09, 25.45s/it] 4%|▍ | 110/2500 [27:06<16:40:18, 25.11s/it] {'loss': 0.0001, 'grad_norm': 0.8942522546018334, 'learning_rate': 9.559999999999998e-07, 'completion_length': 151.55357360839844, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.20801585912704468, 'kl': 0.00330352783203125, 'epoch': 0.04} 4%|▍ | 110/2500 [27:06<16:40:18, 25.11s/it] 4%|▍ | 111/2500 [27:31<16:35:25, 25.00s/it] {'loss': 0.0001, 'grad_norm': 0.9945765813183604, 'learning_rate': 9.556e-07, 'completion_length': 160.93750762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.12686798349022865, 'kl': 0.00286865234375, 'epoch': 0.04} 4%|▍ | 111/2500 [27:31<16:35:25, 25.00s/it] 4%|▍ | 112/2500 [27:55<16:28:44, 24.84s/it] {'loss': 0.0001, 'grad_norm': 0.5346590342672585, 'learning_rate': 9.552e-07, 'completion_length': 158.6607208251953, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.16262340545654297, 'kl': 0.003570556640625, 'epoch': 0.04} 4%|▍ | 112/2500 [27:55<16:28:44, 24.84s/it] 5%|▍ | 113/2500 [28:19<16:11:09, 24.41s/it] {'loss': 0.0001, 'grad_norm': 0.6260550628219224, 'learning_rate': 9.548e-07, 'completion_length': 153.6964340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.785714328289032, 'reward_std': 0.20080359280109406, 'kl': 0.00295257568359375, 'epoch': 0.05} 5%|▍ | 113/2500 [28:19<16:11:09, 24.41s/it] 5%|▍ | 114/2500 [28:42<16:00:01, 24.14s/it] {'loss': 0.0001, 'grad_norm': 0.5182519487000018, 'learning_rate': 9.544e-07, 'completion_length': 144.4107208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.10882645100355148, 'kl': 0.002593994140625, 'epoch': 0.05} 5%|▍ | 114/2500 [28:42<16:00:01, 24.14s/it] 5%|▍ | 115/2500 [29:05<15:43:29, 23.74s/it] {'loss': 0.0001, 'grad_norm': 0.9226956962218277, 'learning_rate': 9.539999999999999e-07, 'completion_length': 145.36608123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.14579425007104874, 'kl': 0.0027008056640625, 'epoch': 0.05} 5%|▍ | 115/2500 [29:05<15:43:29, 23.74s/it] 5%|▍ | 116/2500 [29:28<15:37:21, 23.59s/it] {'loss': 0.0001, 'grad_norm': 0.45460215186769704, 'learning_rate': 9.536e-07, 'completion_length': 149.7857208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12444323301315308, 'kl': 0.0028839111328125, 'epoch': 0.05} 5%|▍ | 116/2500 [29:28<15:37:21, 23.59s/it] 5%|▍ | 117/2500 [29:52<15:41:26, 23.70s/it] {'loss': 0.0001, 'grad_norm': 0.6538421405410515, 'learning_rate': 9.532e-07, 'completion_length': 157.1071548461914, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12175280973315239, 'kl': 0.00311279296875, 'epoch': 0.05} 5%|▍ | 117/2500 [29:52<15:41:26, 23.70s/it] 5%|▍ | 118/2500 [30:16<15:42:45, 23.75s/it] {'loss': 0.0001, 'grad_norm': 0.27351499393341905, 'learning_rate': 9.527999999999999e-07, 'completion_length': 154.08036041259766, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0024871826171875, 'epoch': 0.05} 5%|▍ | 118/2500 [30:16<15:42:45, 23.75s/it] 5%|▍ | 119/2500 [30:40<15:49:20, 23.92s/it] {'loss': 0.0002, 'grad_norm': 0.6585473680294667, 'learning_rate': 9.524e-07, 'completion_length': 150.00000762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.14970265328884125, 'kl': 0.0037994384765625, 'epoch': 0.05} 5%|▍ | 119/2500 [30:40<15:49:20, 23.92s/it] 5%|▍ | 120/2500 [31:03<15:38:48, 23.67s/it] {'loss': 0.0002, 'grad_norm': 0.684087936634185, 'learning_rate': 9.52e-07, 'completion_length': 150.11607360839844, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12054044008255005, 'kl': 0.0043182373046875, 'epoch': 0.05} 5%|▍ | 120/2500 [31:03<15:38:48, 23.67s/it] 5%|▍ | 121/2500 [31:27<15:36:11, 23.61s/it] {'loss': 0.0001, 'grad_norm': 0.4502875067405339, 'learning_rate': 9.515999999999999e-07, 'completion_length': 157.3928680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11663764715194702, 'kl': 0.0025787353515625, 'epoch': 0.05} 5%|▍ | 121/2500 [31:27<15:36:11, 23.61s/it] 5%|▍ | 122/2500 [31:51<15:44:07, 23.82s/it] {'loss': 0.0001, 'grad_norm': 1.4511032785320606, 'learning_rate': 9.512e-07, 'completion_length': 159.7232208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.17433738335967064, 'kl': 0.00336456298828125, 'epoch': 0.05} 5%|▍ | 122/2500 [31:51<15:44:07, 23.82s/it] 5%|▍ | 123/2500 [32:15<15:41:32, 23.77s/it] {'loss': 0.0001, 'grad_norm': 0.5630947040848749, 'learning_rate': 9.508e-07, 'completion_length': 157.18750762939453, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12054043635725975, 'kl': 0.00251007080078125, 'epoch': 0.05} 5%|▍ | 123/2500 [32:15<15:41:32, 23.77s/it] 5%|▍ | 124/2500 [32:38<15:37:30, 23.67s/it] {'loss': 0.0002, 'grad_norm': 1.302135230828115, 'learning_rate': 9.503999999999999e-07, 'completion_length': 162.25000762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.1860513761639595, 'kl': 0.00377655029296875, 'epoch': 0.05} 5%|▍ | 124/2500 [32:38<15:37:30, 23.67s/it] 5%|▌ | 125/2500 [33:02<15:36:34, 23.66s/it] {'loss': 0.0001, 'grad_norm': 0.582622395100129, 'learning_rate': 9.499999999999999e-07, 'completion_length': 160.43750762939453, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.17164698243141174, 'kl': 0.00360107421875, 'epoch': 0.05} 5%|▌ | 125/2500 [33:02<15:36:34, 23.66s/it] 5%|▌ | 126/2500 [33:26<15:36:50, 23.68s/it] {'loss': 0.0001, 'grad_norm': 0.582082697191943, 'learning_rate': 9.496e-07, 'completion_length': 153.48214721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0033416748046875, 'epoch': 0.05} 5%|▌ | 126/2500 [33:26<15:36:50, 23.68s/it] 5%|▌ | 127/2500 [33:49<15:37:10, 23.70s/it] {'loss': 0.0001, 'grad_norm': 0.591519999533773, 'learning_rate': 9.492e-07, 'completion_length': 158.77679443359375, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.14700662344694138, 'kl': 0.003509521484375, 'epoch': 0.05} 5%|▌ | 127/2500 [33:49<15:37:10, 23.70s/it] 5%|▌ | 128/2500 [34:13<15:40:12, 23.78s/it] {'loss': 0.0002, 'grad_norm': 0.9921821649486067, 'learning_rate': 9.487999999999999e-07, 'completion_length': 165.2857208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.21765290200710297, 'kl': 0.0047149658203125, 'epoch': 0.05} 5%|▌ | 128/2500 [34:13<15:40:12, 23.78s/it] 5%|▌ | 129/2500 [34:36<15:28:47, 23.50s/it] {'loss': 0.0001, 'grad_norm': 0.7223967895238209, 'learning_rate': 9.484e-07, 'completion_length': 150.65179443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.1671452820301056, 'kl': 0.00370025634765625, 'epoch': 0.05} 5%|▌ | 129/2500 [34:36<15:28:47, 23.50s/it] 5%|▌ | 130/2500 [34:59<15:21:08, 23.32s/it] {'loss': 0.0001, 'grad_norm': 0.9124978279152401, 'learning_rate': 9.479999999999999e-07, 'completion_length': 139.33036041259766, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.10882645472884178, 'kl': 0.0029296875, 'epoch': 0.05} 5%|▌ | 130/2500 [34:59<15:21:08, 23.32s/it] 5%|▌ | 131/2500 [35:23<15:29:24, 23.54s/it] {'loss': 0.0001, 'grad_norm': 0.4872974225846307, 'learning_rate': 9.475999999999999e-07, 'completion_length': 163.81250762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.13225442171096802, 'kl': 0.00331878662109375, 'epoch': 0.05} 5%|▌ | 131/2500 [35:23<15:29:24, 23.54s/it] 5%|▌ | 132/2500 [35:46<15:19:42, 23.30s/it] {'loss': 0.0001, 'grad_norm': 1.1156852932903252, 'learning_rate': 9.472e-07, 'completion_length': 142.19644165039062, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.096499003469944, 'kl': 0.002590179443359375, 'epoch': 0.05} 5%|▌ | 132/2500 [35:46<15:19:42, 23.30s/it] 5%|▌ | 133/2500 [36:08<15:06:38, 22.98s/it] {'loss': 0.0001, 'grad_norm': 0.24661418898562104, 'learning_rate': 9.468e-07, 'completion_length': 134.00000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00212860107421875, 'epoch': 0.05} 5%|▌ | 133/2500 [36:08<15:06:38, 22.98s/it] 5%|▌ | 134/2500 [36:31<15:02:22, 22.88s/it] {'loss': 0.0001, 'grad_norm': 0.599709454309736, 'learning_rate': 9.464e-07, 'completion_length': 146.56250762939453, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.10821298509836197, 'kl': 0.00307464599609375, 'epoch': 0.05} 5%|▌ | 134/2500 [36:31<15:02:22, 22.88s/it] 5%|▌ | 135/2500 [36:54<15:05:26, 22.97s/it] {'loss': 0.0002, 'grad_norm': 0.488329602960946, 'learning_rate': 9.459999999999999e-07, 'completion_length': 168.32144165039062, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.00380706787109375, 'epoch': 0.05} 5%|▌ | 135/2500 [36:54<15:05:26, 22.97s/it] 5%|▌ | 136/2500 [37:17<15:03:43, 22.94s/it] {'loss': 0.0001, 'grad_norm': 0.6561850641264844, 'learning_rate': 9.456e-07, 'completion_length': 157.5089340209961, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.1731250137090683, 'kl': 0.00310516357421875, 'epoch': 0.05} 5%|▌ | 136/2500 [37:17<15:03:43, 22.94s/it] 5%|▌ | 137/2500 [37:39<14:59:39, 22.84s/it] {'loss': 0.0002, 'grad_norm': 0.5136459551335506, 'learning_rate': 9.452e-07, 'completion_length': 151.58929443359375, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.16531942784786224, 'kl': 0.00388336181640625, 'epoch': 0.05} 5%|▌ | 137/2500 [37:39<14:59:39, 22.84s/it] 6%|▌ | 138/2500 [38:02<14:59:24, 22.85s/it] {'loss': 0.0002, 'grad_norm': 0.6221890971465563, 'learning_rate': 9.447999999999999e-07, 'completion_length': 155.46428680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.08747543022036552, 'kl': 0.00494384765625, 'epoch': 0.06} 6%|▌ | 138/2500 [38:02<14:59:24, 22.85s/it] 6%|▌ | 139/2500 [38:25<15:01:12, 22.90s/it] {'loss': 0.0002, 'grad_norm': 0.7310109758403249, 'learning_rate': 9.444e-07, 'completion_length': 153.9464340209961, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.13736959546804428, 'kl': 0.0042572021484375, 'epoch': 0.06} 6%|▌ | 139/2500 [38:25<15:01:12, 22.90s/it] 6%|▌ | 140/2500 [38:49<15:05:29, 23.02s/it] {'loss': 0.0001, 'grad_norm': 1.2231146087961124, 'learning_rate': 9.439999999999999e-07, 'completion_length': 152.19644165039062, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.20801587402820587, 'kl': 0.00327301025390625, 'epoch': 0.06} 6%|▌ | 140/2500 [38:49<15:05:29, 23.02s/it] 6%|▌ | 141/2500 [39:11<15:04:10, 23.00s/it] {'loss': 0.0001, 'grad_norm': 0.6336269470776819, 'learning_rate': 9.436e-07, 'completion_length': 163.08036041259766, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.00347137451171875, 'epoch': 0.06} 6%|▌ | 141/2500 [39:12<15:04:10, 23.00s/it] 6%|▌ | 142/2500 [39:35<15:08:31, 23.12s/it] {'loss': 0.0002, 'grad_norm': 1.3895265516852333, 'learning_rate': 9.432e-07, 'completion_length': 170.6339340209961, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.17104806751012802, 'kl': 0.0041656494140625, 'epoch': 0.06} 6%|▌ | 142/2500 [39:35<15:08:31, 23.12s/it] 6%|▌ | 143/2500 [40:00<15:32:05, 23.73s/it] {'loss': 0.0001, 'grad_norm': 0.9501981968315752, 'learning_rate': 9.427999999999999e-07, 'completion_length': 153.18750762939453, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.12444322928786278, 'kl': 0.00323486328125, 'epoch': 0.06} 6%|▌ | 143/2500 [40:00<15:32:05, 23.73s/it] 6%|▌ | 144/2500 [40:23<15:28:27, 23.64s/it] {'loss': 0.0001, 'grad_norm': 0.4578650459353718, 'learning_rate': 9.424e-07, 'completion_length': 155.35714721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.12444883584976196, 'kl': 0.00323486328125, 'epoch': 0.06} 6%|▌ | 144/2500 [40:24<15:28:27, 23.64s/it] 6%|▌ | 145/2500 [40:46<15:18:52, 23.41s/it] {'loss': 0.0001, 'grad_norm': 0.3107392195894632, 'learning_rate': 9.419999999999999e-07, 'completion_length': 145.15179443359375, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06222161278128624, 'kl': 0.00260162353515625, 'epoch': 0.06} 6%|▌ | 145/2500 [40:46<15:18:52, 23.41s/it] 6%|▌ | 146/2500 [41:10<15:17:16, 23.38s/it] {'loss': 0.0001, 'grad_norm': 0.3133365295676395, 'learning_rate': 9.415999999999999e-07, 'completion_length': 156.20536041259766, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0032196044921875, 'epoch': 0.06} 6%|▌ | 146/2500 [41:10<15:17:16, 23.38s/it] 6%|▌ | 147/2500 [41:32<15:09:39, 23.20s/it] {'loss': 0.0001, 'grad_norm': 0.3869929320644671, 'learning_rate': 9.412e-07, 'completion_length': 151.60714721679688, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.00292205810546875, 'epoch': 0.06} 6%|▌ | 147/2500 [41:32<15:09:39, 23.20s/it] 6%|▌ | 148/2500 [41:56<15:10:13, 23.22s/it] {'loss': 0.0002, 'grad_norm': 0.8140824484038649, 'learning_rate': 9.408e-07, 'completion_length': 155.12500762939453, 'rewards/accuracy_reward': 0.7767857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7767857909202576, 'reward_std': 0.23265621066093445, 'kl': 0.0048675537109375, 'epoch': 0.06} 6%|▌ | 148/2500 [41:56<15:10:13, 23.22s/it] 6%|▌ | 149/2500 [42:19<15:09:14, 23.20s/it] {'loss': 0.0001, 'grad_norm': 0.6659699048020783, 'learning_rate': 9.403999999999999e-07, 'completion_length': 150.2053680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.157508235424757, 'kl': 0.002899169921875, 'epoch': 0.06} 6%|▌ | 149/2500 [42:19<15:09:14, 23.20s/it] 6%|▌ | 150/2500 [42:43<15:14:36, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.6011080589078811, 'learning_rate': 9.399999999999999e-07, 'completion_length': 160.30358123779297, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.15360544621944427, 'kl': 0.004180908203125, 'epoch': 0.06} 6%|▌ | 150/2500 [42:43<15:14:36, 23.35s/it] 6%|▌ | 151/2500 [43:06<15:12:55, 23.32s/it] {'loss': 0.0001, 'grad_norm': 0.29457128848443315, 'learning_rate': 9.396e-07, 'completion_length': 157.46429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00360870361328125, 'epoch': 0.06} 6%|▌ | 151/2500 [43:06<15:12:55, 23.32s/it] 6%|▌ | 152/2500 [43:29<15:10:13, 23.26s/it] {'loss': 0.0001, 'grad_norm': 0.6352190836840366, 'learning_rate': 9.391999999999999e-07, 'completion_length': 139.6607208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.13616281747817993, 'kl': 0.00286102294921875, 'epoch': 0.06} 6%|▌ | 152/2500 [43:29<15:10:13, 23.26s/it] 6%|▌ | 153/2500 [43:53<15:16:34, 23.43s/it] {'loss': 0.0002, 'grad_norm': 0.8184109010614502, 'learning_rate': 9.387999999999999e-07, 'completion_length': 155.15179443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.128351628780365, 'kl': 0.004913330078125, 'epoch': 0.06} 6%|▌ | 153/2500 [43:53<15:16:34, 23.43s/it] 6%|▌ | 154/2500 [44:16<15:17:36, 23.47s/it] {'loss': 0.0002, 'grad_norm': 0.48126731534095013, 'learning_rate': 9.384e-07, 'completion_length': 153.80357360839844, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12956400960683823, 'kl': 0.00382232666015625, 'epoch': 0.06} 6%|▌ | 154/2500 [44:16<15:17:36, 23.47s/it] 6%|▌ | 155/2500 [44:40<15:14:39, 23.40s/it] {'loss': 0.0002, 'grad_norm': 0.15842253334686832, 'learning_rate': 9.379999999999998e-07, 'completion_length': 138.36608123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00409698486328125, 'epoch': 0.06} 6%|▌ | 155/2500 [44:40<15:14:39, 23.40s/it] 6%|▌ | 156/2500 [45:03<15:10:51, 23.32s/it] {'loss': 0.0002, 'grad_norm': 0.32019848735924256, 'learning_rate': 9.375999999999999e-07, 'completion_length': 149.0357208251953, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06613001227378845, 'kl': 0.00380706787109375, 'epoch': 0.06} 6%|▌ | 156/2500 [45:03<15:10:51, 23.32s/it] 6%|▋ | 157/2500 [45:26<15:11:53, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.77906191766207, 'learning_rate': 9.372e-07, 'completion_length': 152.31250762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.18787720426917076, 'kl': 0.005096435546875, 'epoch': 0.06} 6%|▋ | 157/2500 [45:26<15:11:53, 23.35s/it] 6%|▋ | 158/2500 [45:50<15:15:33, 23.46s/it] {'loss': 0.0002, 'grad_norm': 0.4614817772717636, 'learning_rate': 9.368e-07, 'completion_length': 156.92858123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.00411224365234375, 'epoch': 0.06} 6%|▋ | 158/2500 [45:50<15:15:33, 23.46s/it] 6%|▋ | 159/2500 [46:13<15:12:31, 23.39s/it] {'loss': 0.0002, 'grad_norm': 0.5879133460108703, 'learning_rate': 9.363999999999999e-07, 'completion_length': 148.67857360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0044097900390625, 'epoch': 0.06} 6%|▋ | 159/2500 [46:13<15:12:31, 23.39s/it] 6%|▋ | 160/2500 [46:37<15:24:07, 23.70s/it] {'loss': 0.0001, 'grad_norm': 0.8786929794110504, 'learning_rate': 9.36e-07, 'completion_length': 162.35714721679688, 'rewards/accuracy_reward': 0.8125000596046448, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.8035715222358704, 'reward_std': 0.1995968148112297, 'kl': 0.0037078857421875, 'epoch': 0.06} 6%|▋ | 160/2500 [46:37<15:24:07, 23.70s/it] 6%|▋ | 161/2500 [47:01<15:25:33, 23.74s/it] {'loss': 0.0002, 'grad_norm': 0.659843629505592, 'learning_rate': 9.356e-07, 'completion_length': 159.4464340209961, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.1575082316994667, 'kl': 0.0045013427734375, 'epoch': 0.06} 6%|▋ | 161/2500 [47:01<15:25:33, 23.74s/it] 6%|▋ | 162/2500 [47:24<15:11:15, 23.39s/it] {'loss': 0.0001, 'grad_norm': 0.6929300604620995, 'learning_rate': 9.352e-07, 'completion_length': 142.5982208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0029754638671875, 'epoch': 0.06} 6%|▋ | 162/2500 [47:24<15:11:15, 23.39s/it] 7%|▋ | 163/2500 [47:50<15:39:04, 24.11s/it] {'loss': 0.0002, 'grad_norm': 0.7917840830110855, 'learning_rate': 9.347999999999999e-07, 'completion_length': 176.75894165039062, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.2696240097284317, 'kl': 0.005615234375, 'epoch': 0.07} 7%|▋ | 163/2500 [47:50<15:39:04, 24.11s/it] 7%|▋ | 164/2500 [48:13<15:26:42, 23.80s/it] {'loss': 0.0002, 'grad_norm': 0.7742918295391259, 'learning_rate': 9.344e-07, 'completion_length': 146.3125, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.17104807496070862, 'kl': 0.00406646728515625, 'epoch': 0.07} 7%|▋ | 164/2500 [48:13<15:26:42, 23.80s/it] 7%|▋ | 165/2500 [48:36<15:17:39, 23.58s/it] {'loss': 0.0001, 'grad_norm': 0.47363424465825804, 'learning_rate': 9.34e-07, 'completion_length': 143.0714340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0034637451171875, 'epoch': 0.07} 7%|▋ | 165/2500 [48:36<15:17:39, 23.58s/it] 7%|▋ | 166/2500 [48:59<15:13:29, 23.48s/it] {'loss': 0.0002, 'grad_norm': 1.5397357513460015, 'learning_rate': 9.335999999999999e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.1418914645910263, 'kl': 0.00421905517578125, 'epoch': 0.07} 7%|▋ | 166/2500 [48:59<15:13:29, 23.48s/it] 7%|▋ | 167/2500 [49:23<15:20:39, 23.68s/it] {'loss': 0.0002, 'grad_norm': 0.9041763395356388, 'learning_rate': 9.332e-07, 'completion_length': 150.83036041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.15481781959533691, 'kl': 0.00447845458984375, 'epoch': 0.07} 7%|▋ | 167/2500 [49:23<15:20:39, 23.68s/it] 7%|▋ | 168/2500 [49:47<15:18:07, 23.62s/it] {'loss': 0.0001, 'grad_norm': 0.7016954717669486, 'learning_rate': 9.327999999999999e-07, 'completion_length': 146.91964721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.003448486328125, 'epoch': 0.07} 7%|▋ | 168/2500 [49:47<15:18:07, 23.62s/it] 7%|▋ | 169/2500 [50:10<15:18:37, 23.65s/it] {'loss': 0.0001, 'grad_norm': 0.4863900449935007, 'learning_rate': 9.324e-07, 'completion_length': 143.37500762939453, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.10882644355297089, 'kl': 0.0031585693359375, 'epoch': 0.07} 7%|▋ | 169/2500 [50:10<15:18:37, 23.65s/it] 7%|▋ | 170/2500 [50:34<15:19:24, 23.68s/it] {'loss': 0.0001, 'grad_norm': 0.483086365238544, 'learning_rate': 9.32e-07, 'completion_length': 156.6428680419922, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.00345611572265625, 'epoch': 0.07} 7%|▋ | 170/2500 [50:34<15:19:24, 23.68s/it] 7%|▋ | 171/2500 [50:57<15:14:18, 23.55s/it] {'loss': 0.0001, 'grad_norm': 0.4783186933377896, 'learning_rate': 9.315999999999999e-07, 'completion_length': 155.13394165039062, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0036163330078125, 'epoch': 0.07} 7%|▋ | 171/2500 [50:57<15:14:18, 23.55s/it] 7%|▋ | 172/2500 [51:21<15:17:13, 23.64s/it] {'loss': 0.0002, 'grad_norm': 0.37584296921356275, 'learning_rate': 9.312e-07, 'completion_length': 148.5714340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.00457763671875, 'epoch': 0.07} 7%|▋ | 172/2500 [51:21<15:17:13, 23.64s/it] 7%|▋ | 173/2500 [51:44<15:11:35, 23.50s/it] {'loss': 0.0002, 'grad_norm': 0.7895230340275513, 'learning_rate': 9.307999999999999e-07, 'completion_length': 156.7857208251953, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.1827620416879654, 'kl': 0.0056304931640625, 'epoch': 0.07} 7%|▋ | 173/2500 [51:44<15:11:35, 23.50s/it] 7%|▋ | 174/2500 [52:08<15:12:15, 23.53s/it] {'loss': 0.0002, 'grad_norm': 0.46789636869894136, 'learning_rate': 9.303999999999999e-07, 'completion_length': 150.2589340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.004150390625, 'epoch': 0.07} 7%|▋ | 174/2500 [52:08<15:12:15, 23.53s/it] 7%|▋ | 175/2500 [52:32<15:13:20, 23.57s/it] {'loss': 0.0001, 'grad_norm': 0.4701373125784502, 'learning_rate': 9.3e-07, 'completion_length': 154.0714340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00327301025390625, 'epoch': 0.07} 7%|▋ | 175/2500 [52:32<15:13:20, 23.57s/it] 7%|▋ | 176/2500 [52:55<15:10:55, 23.52s/it] {'loss': 0.0001, 'grad_norm': 0.32508859649818816, 'learning_rate': 9.296e-07, 'completion_length': 141.55357360839844, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780732154846, 'kl': 0.00316619873046875, 'epoch': 0.07} 7%|▋ | 176/2500 [52:55<15:10:55, 23.52s/it] 7%|▋ | 177/2500 [53:19<15:09:25, 23.49s/it] {'loss': 0.0001, 'grad_norm': 0.25145742961282797, 'learning_rate': 9.292e-07, 'completion_length': 155.2321548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00333404541015625, 'epoch': 0.07} 7%|▋ | 177/2500 [53:19<15:09:25, 23.49s/it] 7%|▋ | 178/2500 [53:42<15:14:37, 23.63s/it] {'loss': 0.0002, 'grad_norm': 0.7473384976926035, 'learning_rate': 9.287999999999999e-07, 'completion_length': 164.37500762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.17885926365852356, 'kl': 0.006103515625, 'epoch': 0.07} 7%|▋ | 178/2500 [53:42<15:14:37, 23.63s/it] 7%|▋ | 179/2500 [54:06<15:14:14, 23.63s/it] {'loss': 0.0001, 'grad_norm': 1.1869978279729383, 'learning_rate': 9.284e-07, 'completion_length': 140.41964721679688, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.12956400215625763, 'kl': 0.00356292724609375, 'epoch': 0.07} 7%|▋ | 179/2500 [54:06<15:14:14, 23.63s/it] 7%|▋ | 180/2500 [54:30<15:15:00, 23.66s/it] {'loss': 0.0002, 'grad_norm': 0.9269387677599509, 'learning_rate': 9.28e-07, 'completion_length': 151.5982208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.1412779912352562, 'kl': 0.0039825439453125, 'epoch': 0.07} 7%|▋ | 180/2500 [54:30<15:15:00, 23.66s/it] 7%|▋ | 181/2500 [54:53<15:06:04, 23.44s/it] {'loss': 0.0001, 'grad_norm': 0.3584478195193104, 'learning_rate': 9.275999999999999e-07, 'completion_length': 144.04464721679688, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00348663330078125, 'epoch': 0.07} 7%|▋ | 181/2500 [54:53<15:06:04, 23.44s/it] 7%|▋ | 182/2500 [55:16<15:08:24, 23.51s/it] {'loss': 0.0001, 'grad_norm': 0.3549258431150824, 'learning_rate': 9.272e-07, 'completion_length': 147.52678680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.00331878662109375, 'epoch': 0.07} 7%|▋ | 182/2500 [55:16<15:08:24, 23.51s/it] 7%|▋ | 183/2500 [55:39<15:00:35, 23.32s/it] {'loss': 0.0001, 'grad_norm': 0.02055699515153395, 'learning_rate': 9.268e-07, 'completion_length': 152.33036041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0034942626953125, 'epoch': 0.07} 7%|▋ | 183/2500 [55:39<15:00:35, 23.32s/it] 7%|▋ | 184/2500 [56:03<15:03:20, 23.40s/it] {'loss': 0.0002, 'grad_norm': 0.9125970222152229, 'learning_rate': 9.263999999999999e-07, 'completion_length': 156.01786041259766, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.14006561040878296, 'kl': 0.0062103271484375, 'epoch': 0.07} 7%|▋ | 184/2500 [56:03<15:03:20, 23.40s/it] 7%|▋ | 185/2500 [56:26<14:57:45, 23.27s/it] {'loss': 0.0002, 'grad_norm': 0.5390994187369115, 'learning_rate': 9.26e-07, 'completion_length': 140.1607208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.07576143741607666, 'kl': 0.0042877197265625, 'epoch': 0.07} 7%|▋ | 185/2500 [56:26<14:57:45, 23.27s/it] 7%|▋ | 186/2500 [56:49<14:56:13, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.3369639242383857, 'learning_rate': 9.256e-07, 'completion_length': 154.8571548461914, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0052947998046875, 'epoch': 0.07} 7%|▋ | 186/2500 [56:49<14:56:13, 23.24s/it] 7%|▋ | 187/2500 [57:12<14:54:37, 23.21s/it] {'loss': 0.0002, 'grad_norm': 1.1904759368692195, 'learning_rate': 9.251999999999999e-07, 'completion_length': 154.6964340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.09528662264347076, 'kl': 0.00476837158203125, 'epoch': 0.07} 7%|▋ | 187/2500 [57:12<14:54:37, 23.21s/it] 8%|▊ | 188/2500 [57:35<14:53:58, 23.20s/it] {'loss': 0.0002, 'grad_norm': 0.6826910152498505, 'learning_rate': 9.247999999999999e-07, 'completion_length': 150.27679443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.16653180867433548, 'kl': 0.004913330078125, 'epoch': 0.08} 8%|▊ | 188/2500 [57:35<14:53:58, 23.20s/it] 8%|▊ | 189/2500 [57:58<14:45:05, 22.98s/it] {'loss': 0.0001, 'grad_norm': 0.4227688052848838, 'learning_rate': 9.244e-07, 'completion_length': 132.77679443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.0835726410150528, 'kl': 0.003631591796875, 'epoch': 0.08} 8%|▊ | 189/2500 [57:58<14:45:05, 22.98s/it] 8%|▊ | 190/2500 [58:22<15:00:14, 23.38s/it] {'loss': 0.0001, 'grad_norm': 0.593351395709011, 'learning_rate': 9.24e-07, 'completion_length': 138.0625, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.12444322928786278, 'kl': 0.0036468505859375, 'epoch': 0.08} 8%|▊ | 190/2500 [58:22<15:00:14, 23.38s/it] 8%|▊ | 191/2500 [58:46<15:01:16, 23.42s/it] {'loss': 0.0002, 'grad_norm': 0.5404134025528632, 'learning_rate': 9.235999999999999e-07, 'completion_length': 157.8214340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.10101525485515594, 'kl': 0.005767822265625, 'epoch': 0.08} 8%|▊ | 191/2500 [58:46<15:01:16, 23.42s/it] 8%|▊ | 192/2500 [59:10<15:07:52, 23.60s/it] {'loss': 0.0002, 'grad_norm': 0.7337665413235555, 'learning_rate': 9.232e-07, 'completion_length': 146.83036041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11663764715194702, 'kl': 0.0041961669921875, 'epoch': 0.08} 8%|▊ | 192/2500 [59:10<15:07:52, 23.60s/it] 8%|▊ | 193/2500 [59:34<15:17:58, 23.87s/it] {'loss': 0.0002, 'grad_norm': 1.5142243262524449, 'learning_rate': 9.227999999999999e-07, 'completion_length': 166.61607360839844, 'rewards/accuracy_reward': 0.7946428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7946429252624512, 'reward_std': 0.1412779912352562, 'kl': 0.005706787109375, 'epoch': 0.08} 8%|▊ | 193/2500 [59:34<15:17:58, 23.87s/it] 8%|▊ | 194/2500 [59:57<15:04:30, 23.53s/it] {'loss': 0.0002, 'grad_norm': 0.2842676501133988, 'learning_rate': 9.224e-07, 'completion_length': 146.83036041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004547119140625, 'epoch': 0.08} 8%|▊ | 194/2500 [59:57<15:04:30, 23.53s/it] 8%|▊ | 195/2500 [1:00:20<15:01:36, 23.47s/it] {'loss': 0.0002, 'grad_norm': 0.5927312973213402, 'learning_rate': 9.22e-07, 'completion_length': 149.8482208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00481414794921875, 'epoch': 0.08} 8%|▊ | 195/2500 [1:00:20<15:01:36, 23.47s/it] 8%|▊ | 196/2500 [1:00:44<15:02:33, 23.50s/it] {'loss': 0.0002, 'grad_norm': 0.29215128295232395, 'learning_rate': 9.215999999999999e-07, 'completion_length': 155.66964721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0046234130859375, 'epoch': 0.08} 8%|▊ | 196/2500 [1:00:44<15:02:33, 23.50s/it] 8%|▊ | 197/2500 [1:01:07<15:03:32, 23.54s/it] {'loss': 0.0002, 'grad_norm': 0.7924993374047111, 'learning_rate': 9.212e-07, 'completion_length': 167.65179443359375, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.24498365819454193, 'kl': 0.006103515625, 'epoch': 0.08} 8%|▊ | 197/2500 [1:01:07<15:03:32, 23.54s/it] 8%|▊ | 198/2500 [1:01:31<15:01:54, 23.51s/it] {'loss': 0.0002, 'grad_norm': 0.2574149816009823, 'learning_rate': 9.207999999999999e-07, 'completion_length': 158.56250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005401611328125, 'epoch': 0.08} 8%|▊ | 198/2500 [1:01:31<15:01:54, 23.51s/it] 8%|▊ | 199/2500 [1:01:54<14:53:46, 23.31s/it] {'loss': 0.0002, 'grad_norm': 0.26423982177030175, 'learning_rate': 9.203999999999999e-07, 'completion_length': 144.2232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004669189453125, 'epoch': 0.08} 8%|▊ | 199/2500 [1:01:54<14:53:46, 23.31s/it] 8%|▊ | 200/2500 [1:02:17<14:53:46, 23.32s/it] {'loss': 0.0003, 'grad_norm': 2.846179590325297, 'learning_rate': 9.2e-07, 'completion_length': 152.24107360839844, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.16653180867433548, 'kl': 0.00738525390625, 'epoch': 0.08} 8%|▊ | 200/2500 [1:02:17<14:53:46, 23.32s/it] 8%|▊ | 201/2500 [1:03:28<23:56:49, 37.50s/it] {'loss': 0.0002, 'grad_norm': 0.4386365585235521, 'learning_rate': 9.196e-07, 'completion_length': 148.16964721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004119873046875, 'epoch': 0.08} 8%|▊ | 201/2500 [1:03:28<23:56:49, 37.50s/it] 8%|▊ | 202/2500 [1:03:51<21:18:40, 33.39s/it] {'loss': 0.0004, 'grad_norm': 1.0973396384963576, 'learning_rate': 9.192e-07, 'completion_length': 171.81250762939453, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.723214328289032, 'reward_std': 0.22754104435443878, 'kl': 0.009033203125, 'epoch': 0.08} 8%|▊ | 202/2500 [1:03:51<21:18:40, 33.39s/it] 8%|▊ | 203/2500 [1:04:14<19:14:38, 30.16s/it] {'loss': 0.0002, 'grad_norm': 0.7186967361835666, 'learning_rate': 9.187999999999999e-07, 'completion_length': 151.375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.128351628780365, 'kl': 0.0054473876953125, 'epoch': 0.08} 8%|▊ | 203/2500 [1:04:14<19:14:38, 30.16s/it] 8%|▊ | 204/2500 [1:04:38<18:01:48, 28.27s/it] {'loss': 0.0002, 'grad_norm': 0.32638549334141287, 'learning_rate': 9.184e-07, 'completion_length': 153.36608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00533294677734375, 'epoch': 0.08} 8%|▊ | 204/2500 [1:04:38<18:01:48, 28.27s/it] 8%|▊ | 205/2500 [1:05:01<16:57:30, 26.60s/it] {'loss': 0.0003, 'grad_norm': 0.5218340122991897, 'learning_rate': 9.18e-07, 'completion_length': 153.50000762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00640869140625, 'epoch': 0.08} 8%|▊ | 205/2500 [1:05:01<16:57:30, 26.60s/it] 8%|▊ | 206/2500 [1:05:24<16:18:07, 25.58s/it] {'loss': 0.0002, 'grad_norm': 1.0726608270343927, 'learning_rate': 9.175999999999999e-07, 'completion_length': 152.05358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00435638427734375, 'epoch': 0.08} 8%|▊ | 206/2500 [1:05:24<16:18:07, 25.58s/it] 8%|▊ | 207/2500 [1:05:46<15:40:04, 24.60s/it] {'loss': 0.0002, 'grad_norm': 0.6642697641259713, 'learning_rate': 9.172e-07, 'completion_length': 149.40179443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.08868780732154846, 'kl': 0.00592041015625, 'epoch': 0.08} 8%|▊ | 207/2500 [1:05:46<15:40:04, 24.60s/it] 8%|▊ | 208/2500 [1:06:09<15:18:06, 24.03s/it] {'loss': 0.0002, 'grad_norm': 0.4755537403954103, 'learning_rate': 9.168e-07, 'completion_length': 148.27679443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0047454833984375, 'epoch': 0.08} 8%|▊ | 208/2500 [1:06:09<15:18:06, 24.03s/it] 8%|▊ | 209/2500 [1:06:32<15:03:33, 23.66s/it] {'loss': 0.0003, 'grad_norm': 0.7515964597964495, 'learning_rate': 9.163999999999999e-07, 'completion_length': 155.8839340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.14970265328884125, 'kl': 0.006866455078125, 'epoch': 0.08} 8%|▊ | 209/2500 [1:06:32<15:03:33, 23.66s/it] 8%|▊ | 210/2500 [1:06:55<15:00:17, 23.59s/it] {'loss': 0.0002, 'grad_norm': 0.20652511932577386, 'learning_rate': 9.16e-07, 'completion_length': 163.2589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0054168701171875, 'epoch': 0.08} 8%|▊ | 210/2500 [1:06:55<15:00:17, 23.59s/it] 8%|▊ | 211/2500 [1:07:19<14:59:17, 23.57s/it] {'loss': 0.0002, 'grad_norm': 0.9766021447107979, 'learning_rate': 9.156e-07, 'completion_length': 142.00000762939453, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.17164698243141174, 'kl': 0.0042572021484375, 'epoch': 0.08} 8%|▊ | 211/2500 [1:07:19<14:59:17, 23.57s/it] 8%|▊ | 212/2500 [1:07:45<15:35:52, 24.54s/it] {'loss': 0.0003, 'grad_norm': 0.2262060913214906, 'learning_rate': 9.151999999999999e-07, 'completion_length': 156.08929443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00640869140625, 'epoch': 0.08} 8%|▊ | 212/2500 [1:07:45<15:35:52, 24.54s/it] 9%|▊ | 213/2500 [1:08:08<15:18:15, 24.09s/it] {'loss': 0.0002, 'grad_norm': 0.41366862988088976, 'learning_rate': 9.147999999999999e-07, 'completion_length': 148.0982208251953, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.0964989960193634, 'kl': 0.0050506591796875, 'epoch': 0.09} 9%|▊ | 213/2500 [1:08:08<15:18:15, 24.09s/it] 9%|▊ | 214/2500 [1:08:31<15:04:20, 23.74s/it] {'loss': 0.0003, 'grad_norm': 0.6818730576416213, 'learning_rate': 9.144e-07, 'completion_length': 157.5714340209961, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.1575138419866562, 'kl': 0.0066070556640625, 'epoch': 0.09} 9%|▊ | 214/2500 [1:08:31<15:04:20, 23.74s/it] 9%|▊ | 215/2500 [1:08:55<14:57:19, 23.56s/it] {'loss': 0.0002, 'grad_norm': 0.49588531027089755, 'learning_rate': 9.14e-07, 'completion_length': 155.0446548461914, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.12444323301315308, 'kl': 0.0049591064453125, 'epoch': 0.09} 9%|▊ | 215/2500 [1:08:55<14:57:19, 23.56s/it] 9%|▊ | 216/2500 [1:09:19<15:02:35, 23.71s/it] {'loss': 0.0002, 'grad_norm': 0.43392056769219267, 'learning_rate': 9.135999999999999e-07, 'completion_length': 160.06250762939453, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.13858197629451752, 'kl': 0.0051422119140625, 'epoch': 0.09} 9%|▊ | 216/2500 [1:09:19<15:02:35, 23.71s/it] 9%|▊ | 217/2500 [1:09:42<15:00:33, 23.67s/it] {'loss': 0.0002, 'grad_norm': 0.24410189619452966, 'learning_rate': 9.132e-07, 'completion_length': 152.18750762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.005523681640625, 'epoch': 0.09} 9%|▊ | 217/2500 [1:09:42<15:00:33, 23.67s/it] 9%|▊ | 218/2500 [1:10:05<14:55:05, 23.53s/it] {'loss': 0.0002, 'grad_norm': 0.5958416465573698, 'learning_rate': 9.127999999999999e-07, 'completion_length': 158.08036041259766, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.0964989960193634, 'kl': 0.0056610107421875, 'epoch': 0.09} 9%|▊ | 218/2500 [1:10:05<14:55:05, 23.53s/it] 9%|▉ | 219/2500 [1:10:28<14:47:41, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.8021686995012733, 'learning_rate': 9.123999999999999e-07, 'completion_length': 147.0357208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.16444925218820572, 'kl': 0.0050811767578125, 'epoch': 0.09} 9%|▉ | 219/2500 [1:10:28<14:47:41, 23.35s/it] 9%|▉ | 220/2500 [1:10:54<15:14:16, 24.06s/it] {'loss': 0.0002, 'grad_norm': 0.32817189464418584, 'learning_rate': 9.12e-07, 'completion_length': 148.6607208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0054473876953125, 'epoch': 0.09} 9%|▉ | 220/2500 [1:10:54<15:14:16, 24.06s/it] 9%|▉ | 221/2500 [1:11:17<15:01:20, 23.73s/it] {'loss': 0.0002, 'grad_norm': 0.6105107880489737, 'learning_rate': 9.115999999999999e-07, 'completion_length': 147.04464721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0059356689453125, 'epoch': 0.09} 9%|▉ | 221/2500 [1:11:17<15:01:20, 23.73s/it] 9%|▉ | 222/2500 [1:11:41<15:00:34, 23.72s/it] {'loss': 0.0002, 'grad_norm': 1.1297047397046736, 'learning_rate': 9.112e-07, 'completion_length': 161.1875, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.15872061997652054, 'kl': 0.0050811767578125, 'epoch': 0.09} 9%|▉ | 222/2500 [1:11:41<15:00:34, 23.72s/it] 9%|▉ | 223/2500 [1:12:04<14:52:06, 23.51s/it] {'loss': 0.0002, 'grad_norm': 0.6332072882126736, 'learning_rate': 9.108e-07, 'completion_length': 146.7232208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.11663764342665672, 'kl': 0.0042266845703125, 'epoch': 0.09} 9%|▉ | 223/2500 [1:12:04<14:52:06, 23.51s/it] 9%|▉ | 224/2500 [1:12:26<14:42:54, 23.28s/it] {'loss': 0.0002, 'grad_norm': 0.663711290543811, 'learning_rate': 9.103999999999999e-07, 'completion_length': 145.21428680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.11272923648357391, 'kl': 0.0056610107421875, 'epoch': 0.09} 9%|▉ | 224/2500 [1:12:26<14:42:54, 23.28s/it] 9%|▉ | 225/2500 [1:12:50<14:41:01, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.35510613268751834, 'learning_rate': 9.1e-07, 'completion_length': 148.75000762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.006103515625, 'epoch': 0.09} 9%|▉ | 225/2500 [1:12:50<14:41:01, 23.24s/it] 9%|▉ | 226/2500 [1:13:13<14:43:40, 23.32s/it] {'loss': 0.0002, 'grad_norm': 0.6343567067640408, 'learning_rate': 9.095999999999999e-07, 'completion_length': 160.30357360839844, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.1379830539226532, 'kl': 0.0057373046875, 'epoch': 0.09} 9%|▉ | 226/2500 [1:13:13<14:43:40, 23.32s/it] 9%|▉ | 227/2500 [1:13:36<14:41:05, 23.26s/it] {'loss': 0.0003, 'grad_norm': 0.6783617422624002, 'learning_rate': 9.092e-07, 'completion_length': 165.4821548461914, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.12444323301315308, 'kl': 0.0068206787109375, 'epoch': 0.09} 9%|▉ | 227/2500 [1:13:36<14:41:05, 23.26s/it] 9%|▉ | 228/2500 [1:14:00<14:45:31, 23.39s/it] {'loss': 0.0002, 'grad_norm': 0.5986765413393283, 'learning_rate': 9.088e-07, 'completion_length': 163.8839340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.17495085299015045, 'kl': 0.00604248046875, 'epoch': 0.09} 9%|▉ | 228/2500 [1:14:00<14:45:31, 23.39s/it] 9%|▉ | 229/2500 [1:14:24<14:47:51, 23.46s/it] {'loss': 0.0003, 'grad_norm': 0.29371761191153, 'learning_rate': 9.084e-07, 'completion_length': 165.31250762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.00665283203125, 'epoch': 0.09} 9%|▉ | 229/2500 [1:14:24<14:47:51, 23.46s/it] 9%|▉ | 230/2500 [1:14:47<14:43:25, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.37441181995129097, 'learning_rate': 9.08e-07, 'completion_length': 154.17857360839844, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.06222161278128624, 'kl': 0.0059967041015625, 'epoch': 0.09} 9%|▉ | 230/2500 [1:14:47<14:43:25, 23.35s/it] 9%|▉ | 231/2500 [1:15:09<14:35:56, 23.16s/it] {'loss': 0.0003, 'grad_norm': 0.46405233076645946, 'learning_rate': 9.075999999999999e-07, 'completion_length': 138.73214721679688, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.10040178522467613, 'kl': 0.0062713623046875, 'epoch': 0.09} 9%|▉ | 231/2500 [1:15:09<14:35:56, 23.16s/it] 9%|▉ | 232/2500 [1:15:33<14:37:23, 23.21s/it] {'loss': 0.0003, 'grad_norm': 0.34952827187277125, 'learning_rate': 9.072e-07, 'completion_length': 164.5089340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.09528662264347076, 'kl': 0.008209228515625, 'epoch': 0.09} 9%|▉ | 232/2500 [1:15:33<14:37:23, 23.21s/it] 9%|▉ | 233/2500 [1:15:55<14:30:15, 23.03s/it] {'loss': 0.0003, 'grad_norm': 0.29044452378526436, 'learning_rate': 9.068e-07, 'completion_length': 151.83036041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0072174072265625, 'epoch': 0.09} 9%|▉ | 233/2500 [1:15:55<14:30:15, 23.03s/it] 9%|▉ | 234/2500 [1:16:18<14:22:00, 22.82s/it] {'loss': 0.0002, 'grad_norm': 0.5117853885083262, 'learning_rate': 9.063999999999999e-07, 'completion_length': 142.2857208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0048065185546875, 'epoch': 0.09} 9%|▉ | 234/2500 [1:16:18<14:22:00, 22.82s/it] 9%|▉ | 235/2500 [1:16:41<14:25:40, 22.93s/it] {'loss': 0.0004, 'grad_norm': 1.118291532684428, 'learning_rate': 9.06e-07, 'completion_length': 157.4732208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.18788281083106995, 'kl': 0.0091400146484375, 'epoch': 0.09} 9%|▉ | 235/2500 [1:16:41<14:25:40, 22.93s/it] 9%|▉ | 236/2500 [1:17:04<14:31:53, 23.11s/it] {'loss': 0.0003, 'grad_norm': 0.6856143728839675, 'learning_rate': 9.056e-07, 'completion_length': 156.5982208251953, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.16262341290712357, 'kl': 0.0066070556640625, 'epoch': 0.09} 9%|▉ | 236/2500 [1:17:04<14:31:53, 23.11s/it] 9%|▉ | 237/2500 [1:17:27<14:25:22, 22.94s/it] {'loss': 0.0002, 'grad_norm': 0.7653099688083497, 'learning_rate': 9.051999999999999e-07, 'completion_length': 141.67858123779297, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.15360544621944427, 'kl': 0.0046234130859375, 'epoch': 0.09} 9%|▉ | 237/2500 [1:17:27<14:25:22, 22.94s/it] 10%|▉ | 238/2500 [1:17:51<14:41:14, 23.38s/it] {'loss': 0.0003, 'grad_norm': 0.25041906115385115, 'learning_rate': 9.048e-07, 'completion_length': 155.5982208251953, 'rewards/accuracy_reward': 0.8035714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8035714626312256, 'reward_std': 0.033065006136894226, 'kl': 0.00626373291015625, 'epoch': 0.1} 10%|▉ | 238/2500 [1:17:51<14:41:14, 23.38s/it] 10%|▉ | 239/2500 [1:18:15<14:45:15, 23.49s/it] {'loss': 0.0002, 'grad_norm': 0.6197886274281362, 'learning_rate': 9.044e-07, 'completion_length': 148.40178680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.0060882568359375, 'epoch': 0.1} 10%|▉ | 239/2500 [1:18:15<14:45:15, 23.49s/it] 10%|▉ | 240/2500 [1:18:38<14:44:47, 23.49s/it] {'loss': 0.0003, 'grad_norm': 0.5001779059068496, 'learning_rate': 9.039999999999999e-07, 'completion_length': 146.30358123779297, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.10700060427188873, 'kl': 0.0069580078125, 'epoch': 0.1} 10%|▉ | 240/2500 [1:18:39<14:44:47, 23.49s/it] 10%|▉ | 241/2500 [1:19:01<14:34:20, 23.22s/it] {'loss': 0.0002, 'grad_norm': 0.4859649879408821, 'learning_rate': 9.035999999999999e-07, 'completion_length': 147.02679443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.0964989960193634, 'kl': 0.0057830810546875, 'epoch': 0.1} 10%|▉ | 241/2500 [1:19:01<14:34:20, 23.22s/it] 10%|▉ | 242/2500 [1:19:25<14:42:08, 23.44s/it] {'loss': 0.0003, 'grad_norm': 0.43255359402899723, 'learning_rate': 9.032e-07, 'completion_length': 155.86607360839844, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0067596435546875, 'epoch': 0.1} 10%|▉ | 242/2500 [1:19:25<14:42:08, 23.44s/it] 10%|▉ | 243/2500 [1:19:48<14:38:14, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.5468890162106045, 'learning_rate': 9.028e-07, 'completion_length': 148.42858123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.0046539306640625, 'epoch': 0.1} 10%|▉ | 243/2500 [1:19:48<14:38:14, 23.35s/it] 10%|▉ | 244/2500 [1:20:11<14:34:22, 23.25s/it] {'loss': 0.0003, 'grad_norm': 0.3884097631303477, 'learning_rate': 9.023999999999999e-07, 'completion_length': 155.55357360839844, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.09528662264347076, 'kl': 0.007537841796875, 'epoch': 0.1} 10%|▉ | 244/2500 [1:20:11<14:34:22, 23.25s/it] 10%|▉ | 245/2500 [1:20:35<14:44:00, 23.52s/it] {'loss': 0.0003, 'grad_norm': 0.5235585063484847, 'learning_rate': 9.02e-07, 'completion_length': 156.36608123779297, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.13858197629451752, 'kl': 0.006683349609375, 'epoch': 0.1} 10%|▉ | 245/2500 [1:20:35<14:44:00, 23.52s/it] 10%|▉ | 246/2500 [1:21:01<15:04:26, 24.08s/it] {'loss': 0.0003, 'grad_norm': 0.6735409036970644, 'learning_rate': 9.015999999999999e-07, 'completion_length': 160.51786041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.14700662344694138, 'kl': 0.0073699951171875, 'epoch': 0.1} 10%|▉ | 246/2500 [1:21:01<15:04:26, 24.08s/it] 10%|▉ | 247/2500 [1:21:24<14:58:19, 23.92s/it] {'loss': 0.0003, 'grad_norm': 0.45688560879696993, 'learning_rate': 9.011999999999999e-07, 'completion_length': 158.08929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10040178894996643, 'kl': 0.007720947265625, 'epoch': 0.1} 10%|▉ | 247/2500 [1:21:24<14:58:19, 23.92s/it] 10%|▉ | 248/2500 [1:21:49<15:03:51, 24.08s/it] {'loss': 0.0003, 'grad_norm': 0.3366918612108143, 'learning_rate': 9.008e-07, 'completion_length': 160.96429443359375, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.06343398988246918, 'kl': 0.0065765380859375, 'epoch': 0.1} 10%|▉ | 248/2500 [1:21:49<15:03:51, 24.08s/it] 10%|▉ | 249/2500 [1:22:12<14:55:16, 23.86s/it] {'loss': 0.0003, 'grad_norm': 0.4888538766402823, 'learning_rate': 9.004e-07, 'completion_length': 162.1964340209961, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.08747542649507523, 'kl': 0.007781982421875, 'epoch': 0.1} 10%|▉ | 249/2500 [1:22:12<14:55:16, 23.86s/it] 10%|█ | 250/2500 [1:22:35<14:44:39, 23.59s/it] {'loss': 0.0003, 'grad_norm': 1.0193153355820421, 'learning_rate': 9e-07, 'completion_length': 148.83929443359375, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.14006561040878296, 'kl': 0.0086212158203125, 'epoch': 0.1} 10%|█ | 250/2500 [1:22:35<14:44:39, 23.59s/it] 10%|█ | 251/2500 [1:23:17<18:11:05, 29.11s/it] {'loss': 0.0003, 'grad_norm': 0.9721178965489414, 'learning_rate': 8.995999999999999e-07, 'completion_length': 157.6428680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.1030978113412857, 'kl': 0.00653076171875, 'epoch': 0.1} 10%|█ | 251/2500 [1:23:17<18:11:05, 29.11s/it] 10%|█ | 252/2500 [1:23:41<17:08:50, 27.46s/it] {'loss': 0.0002, 'grad_norm': 0.455869959542903, 'learning_rate': 8.992e-07, 'completion_length': 136.43750762939453, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726372897625, 'kl': 0.0042266845703125, 'epoch': 0.1} 10%|█ | 252/2500 [1:23:41<17:08:50, 27.46s/it] 10%|█ | 253/2500 [1:24:04<16:23:27, 26.26s/it] {'loss': 0.0002, 'grad_norm': 0.6578300863396376, 'learning_rate': 8.988e-07, 'completion_length': 153.8839340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.005645751953125, 'epoch': 0.1} 10%|█ | 253/2500 [1:24:04<16:23:27, 26.26s/it] 10%|█ | 254/2500 [1:24:27<15:48:25, 25.34s/it] {'loss': 0.0002, 'grad_norm': 0.5441175484435328, 'learning_rate': 8.983999999999999e-07, 'completion_length': 145.89286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0056304931640625, 'epoch': 0.1} 10%|█ | 254/2500 [1:24:27<15:48:25, 25.34s/it] 10%|█ | 255/2500 [1:24:50<15:19:31, 24.58s/it] {'loss': 0.0002, 'grad_norm': 0.020419672051916894, 'learning_rate': 8.98e-07, 'completion_length': 141.5446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051116943359375, 'epoch': 0.1} 10%|█ | 255/2500 [1:24:50<15:19:31, 24.58s/it] 10%|█ | 256/2500 [1:25:14<15:08:42, 24.30s/it] {'loss': 0.0003, 'grad_norm': 0.29387139406799456, 'learning_rate': 8.975999999999999e-07, 'completion_length': 167.5357208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.007232666015625, 'epoch': 0.1} 10%|█ | 256/2500 [1:25:14<15:08:42, 24.30s/it] 10%|█ | 257/2500 [1:25:37<15:01:34, 24.12s/it] {'loss': 0.0004, 'grad_norm': 0.9397481470233779, 'learning_rate': 8.972e-07, 'completion_length': 169.9464340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.2158326655626297, 'kl': 0.0089569091796875, 'epoch': 0.1} 10%|█ | 257/2500 [1:25:37<15:01:34, 24.12s/it] 10%|█ | 258/2500 [1:26:01<14:54:34, 23.94s/it] {'loss': 0.0003, 'grad_norm': 0.698415791773649, 'learning_rate': 8.968e-07, 'completion_length': 163.84822845458984, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.14579425379633904, 'kl': 0.00720977783203125, 'epoch': 0.1} 10%|█ | 258/2500 [1:26:01<14:54:34, 23.94s/it] 10%|█ | 259/2500 [1:26:24<14:39:59, 23.56s/it] {'loss': 0.0002, 'grad_norm': 0.29397351234705826, 'learning_rate': 8.963999999999999e-07, 'completion_length': 135.1339340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.00382232666015625, 'epoch': 0.1} 10%|█ | 259/2500 [1:26:24<14:39:59, 23.56s/it] 10%|█ | 260/2500 [1:26:47<14:38:54, 23.54s/it] {'loss': 0.0003, 'grad_norm': 2.8367155046789674, 'learning_rate': 8.96e-07, 'completion_length': 158.60714721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11272924393415451, 'kl': 0.0066070556640625, 'epoch': 0.1} 10%|█ | 260/2500 [1:26:47<14:38:54, 23.54s/it] 10%|█ | 261/2500 [1:27:10<14:35:17, 23.46s/it] {'loss': 0.0003, 'grad_norm': 0.589412617738402, 'learning_rate': 8.955999999999999e-07, 'completion_length': 143.6607208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12054044008255005, 'kl': 0.006378173828125, 'epoch': 0.1} 10%|█ | 261/2500 [1:27:10<14:35:17, 23.46s/it] 10%|█ | 262/2500 [1:27:34<14:39:25, 23.58s/it] {'loss': 0.0002, 'grad_norm': 0.5850916504515579, 'learning_rate': 8.951999999999999e-07, 'completion_length': 155.1071548461914, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.09528662264347076, 'kl': 0.0056915283203125, 'epoch': 0.1} 10%|█ | 262/2500 [1:27:34<14:39:25, 23.58s/it] 11%|█ | 263/2500 [1:27:57<14:32:20, 23.40s/it] {'loss': 0.0002, 'grad_norm': 0.2924873484872255, 'learning_rate': 8.948e-07, 'completion_length': 142.26786041259766, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.07124518603086472, 'kl': 0.0055389404296875, 'epoch': 0.11} 11%|█ | 263/2500 [1:27:57<14:32:20, 23.40s/it] 11%|█ | 264/2500 [1:28:21<14:38:32, 23.57s/it] {'loss': 0.0002, 'grad_norm': 0.18191559728427267, 'learning_rate': 8.944e-07, 'completion_length': 147.42857360839844, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.033065006136894226, 'kl': 0.0049285888671875, 'epoch': 0.11} 11%|█ | 264/2500 [1:28:21<14:38:32, 23.57s/it] 11%|█ | 265/2500 [1:28:44<14:30:51, 23.38s/it] {'loss': 0.0002, 'grad_norm': 0.31081135186783654, 'learning_rate': 8.939999999999999e-07, 'completion_length': 149.5714340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.004669189453125, 'epoch': 0.11} 11%|█ | 265/2500 [1:28:44<14:30:51, 23.38s/it] 11%|█ | 266/2500 [1:29:07<14:28:11, 23.32s/it] {'loss': 0.0003, 'grad_norm': 0.3536935989168451, 'learning_rate': 8.935999999999999e-07, 'completion_length': 159.43750762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.006622314453125, 'epoch': 0.11} 11%|█ | 266/2500 [1:29:07<14:28:11, 23.32s/it] 11%|█ | 267/2500 [1:29:30<14:15:36, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.25638169128633115, 'learning_rate': 8.932e-07, 'completion_length': 142.81250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.006072998046875, 'epoch': 0.11} 11%|█ | 267/2500 [1:29:30<14:15:36, 22.99s/it] 11%|█ | 268/2500 [1:29:53<14:20:38, 23.14s/it] {'loss': 0.0002, 'grad_norm': 0.7036445892895252, 'learning_rate': 8.928e-07, 'completion_length': 161.56250762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.15872061252593994, 'kl': 0.005950927734375, 'epoch': 0.11} 11%|█ | 268/2500 [1:29:53<14:20:38, 23.14s/it] 11%|█ | 269/2500 [1:30:16<14:17:31, 23.06s/it] {'loss': 0.0002, 'grad_norm': 0.28740761691198813, 'learning_rate': 8.923999999999999e-07, 'completion_length': 152.6875, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831882357597351, 'kl': 0.0053558349609375, 'epoch': 0.11} 11%|█ | 269/2500 [1:30:16<14:17:31, 23.06s/it] 11%|█ | 270/2500 [1:30:39<14:22:29, 23.21s/it] {'loss': 0.0002, 'grad_norm': 1.0680631232212088, 'learning_rate': 8.92e-07, 'completion_length': 150.39286041259766, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.07576143741607666, 'kl': 0.0054168701171875, 'epoch': 0.11} 11%|█ | 270/2500 [1:30:39<14:22:29, 23.21s/it] 11%|█ | 271/2500 [1:31:02<14:19:54, 23.15s/it] {'loss': 0.0003, 'grad_norm': 0.8453374831727476, 'learning_rate': 8.915999999999999e-07, 'completion_length': 149.43750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.10040179640054703, 'kl': 0.0064849853515625, 'epoch': 0.11} 11%|█ | 271/2500 [1:31:02<14:19:54, 23.15s/it] 11%|█ | 272/2500 [1:31:26<14:27:13, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.8901091927310436, 'learning_rate': 8.911999999999999e-07, 'completion_length': 155.0714340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.14579425379633904, 'kl': 0.005096435546875, 'epoch': 0.11} 11%|█ | 272/2500 [1:31:26<14:27:13, 23.35s/it] 11%|█ | 273/2500 [1:31:50<14:30:12, 23.45s/it] {'loss': 0.0003, 'grad_norm': 0.6669982629866414, 'learning_rate': 8.908e-07, 'completion_length': 163.4732208251953, 'rewards/accuracy_reward': 0.8125000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8125000596046448, 'reward_std': 0.17554976791143417, 'kl': 0.006591796875, 'epoch': 0.11} 11%|█ | 273/2500 [1:31:50<14:30:12, 23.45s/it] 11%|█ | 274/2500 [1:32:13<14:29:22, 23.43s/it] {'loss': 0.0002, 'grad_norm': 0.5310678943417216, 'learning_rate': 8.904e-07, 'completion_length': 143.2589340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.13346679508686066, 'kl': 0.0058746337890625, 'epoch': 0.11} 11%|█ | 274/2500 [1:32:13<14:29:22, 23.43s/it] 11%|█ | 275/2500 [1:32:38<14:37:25, 23.66s/it] {'loss': 0.0003, 'grad_norm': 0.6910007103634559, 'learning_rate': 8.9e-07, 'completion_length': 162.27679443359375, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.13616281747817993, 'kl': 0.0067596435546875, 'epoch': 0.11} 11%|█ | 275/2500 [1:32:38<14:37:25, 23.66s/it] 11%|█ | 276/2500 [1:33:02<14:40:23, 23.75s/it] {'loss': 0.0002, 'grad_norm': 0.4003008610241271, 'learning_rate': 8.895999999999999e-07, 'completion_length': 161.84822845458984, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.005859375, 'epoch': 0.11} 11%|█ | 276/2500 [1:33:02<14:40:23, 23.75s/it] 11%|█ | 277/2500 [1:33:25<14:34:34, 23.61s/it] {'loss': 0.0002, 'grad_norm': 0.3926909924072811, 'learning_rate': 8.892e-07, 'completion_length': 157.0357208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0835726410150528, 'kl': 0.00524139404296875, 'epoch': 0.11} 11%|█ | 277/2500 [1:33:25<14:34:34, 23.61s/it] 11%|█ | 278/2500 [1:33:47<14:23:20, 23.31s/it] {'loss': 0.0002, 'grad_norm': 0.44951937369989897, 'learning_rate': 8.888e-07, 'completion_length': 133.9821548461914, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00423431396484375, 'epoch': 0.11} 11%|█ | 278/2500 [1:33:47<14:23:20, 23.31s/it] 11%|█ | 279/2500 [1:34:11<14:24:53, 23.36s/it] {'loss': 0.0003, 'grad_norm': 0.4878231466167303, 'learning_rate': 8.883999999999999e-07, 'completion_length': 156.1428680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.13615721464157104, 'kl': 0.00665283203125, 'epoch': 0.11} 11%|█ | 279/2500 [1:34:11<14:24:53, 23.36s/it] 11%|█ | 280/2500 [1:34:34<14:24:59, 23.38s/it] {'loss': 0.0002, 'grad_norm': 0.17243393115479264, 'learning_rate': 8.88e-07, 'completion_length': 153.1339340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005828857421875, 'epoch': 0.11} 11%|█ | 280/2500 [1:34:34<14:24:59, 23.38s/it] 11%|█ | 281/2500 [1:34:58<14:22:56, 23.33s/it] {'loss': 0.0002, 'grad_norm': 0.5544412174827985, 'learning_rate': 8.875999999999999e-07, 'completion_length': 140.7589340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.0049896240234375, 'epoch': 0.11} 11%|█ | 281/2500 [1:34:58<14:22:56, 23.33s/it] 11%|█▏ | 282/2500 [1:35:21<14:22:10, 23.32s/it] {'loss': 0.0003, 'grad_norm': 0.3704784109379296, 'learning_rate': 8.872e-07, 'completion_length': 156.75000762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0063934326171875, 'epoch': 0.11} 11%|█▏ | 282/2500 [1:35:21<14:22:10, 23.32s/it] 11%|█▏ | 283/2500 [1:35:44<14:24:38, 23.40s/it] {'loss': 0.0002, 'grad_norm': 0.3829176929876146, 'learning_rate': 8.868e-07, 'completion_length': 152.23214721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.00616455078125, 'epoch': 0.11} 11%|█▏ | 283/2500 [1:35:44<14:24:38, 23.40s/it] 11%|█▏ | 284/2500 [1:36:09<14:36:00, 23.72s/it] {'loss': 0.0003, 'grad_norm': 0.8307983348828484, 'learning_rate': 8.863999999999999e-07, 'completion_length': 151.0714340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12054044008255005, 'kl': 0.0067291259765625, 'epoch': 0.11} 11%|█▏ | 284/2500 [1:36:09<14:36:00, 23.72s/it] 11%|█▏ | 285/2500 [1:36:38<15:39:30, 25.45s/it] {'loss': 0.0003, 'grad_norm': 0.5649453062710253, 'learning_rate': 8.86e-07, 'completion_length': 158.3303680419922, 'rewards/accuracy_reward': 0.830357164144516, 'rewards/format_reward': 1.0, 'reward': 1.8303572535514832, 'reward_std': 0.1346791759133339, 'kl': 0.0081329345703125, 'epoch': 0.11} 11%|█▏ | 285/2500 [1:36:38<15:39:30, 25.45s/it] 11%|█▏ | 286/2500 [1:37:02<15:21:30, 24.97s/it] {'loss': 0.0003, 'grad_norm': 0.2760503145186213, 'learning_rate': 8.856e-07, 'completion_length': 154.36608123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.00830078125, 'epoch': 0.11} 11%|█▏ | 286/2500 [1:37:02<15:21:30, 24.97s/it] 11%|█▏ | 287/2500 [1:37:25<14:58:35, 24.36s/it] {'loss': 0.0003, 'grad_norm': 0.2553048946147629, 'learning_rate': 8.851999999999999e-07, 'completion_length': 146.89286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006683349609375, 'epoch': 0.11} 11%|█▏ | 287/2500 [1:37:25<14:58:35, 24.36s/it] 12%|█▏ | 288/2500 [1:37:50<14:58:21, 24.37s/it] {'loss': 0.0003, 'grad_norm': 0.537144825709583, 'learning_rate': 8.848e-07, 'completion_length': 160.04464721679688, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.09528662264347076, 'kl': 0.0068511962890625, 'epoch': 0.12} 12%|█▏ | 288/2500 [1:37:50<14:58:21, 24.37s/it] 12%|█▏ | 289/2500 [1:38:16<15:24:28, 25.09s/it] {'loss': 0.0003, 'grad_norm': 1.3466755344640993, 'learning_rate': 8.844e-07, 'completion_length': 157.63394165039062, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.10040178894996643, 'kl': 0.00665283203125, 'epoch': 0.12} 12%|█▏ | 289/2500 [1:38:16<15:24:28, 25.09s/it] 12%|█▏ | 290/2500 [1:38:40<15:11:47, 24.75s/it] {'loss': 0.0003, 'grad_norm': 0.31537736128796917, 'learning_rate': 8.839999999999999e-07, 'completion_length': 157.7589340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0065460205078125, 'epoch': 0.12} 12%|█▏ | 290/2500 [1:38:40<15:11:47, 24.75s/it] 12%|█▏ | 291/2500 [1:39:04<14:59:00, 24.42s/it] {'loss': 0.0002, 'grad_norm': 0.328559778798769, 'learning_rate': 8.836e-07, 'completion_length': 145.9464340209961, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.00455474853515625, 'epoch': 0.12} 12%|█▏ | 291/2500 [1:39:04<14:59:00, 24.42s/it] 12%|█▏ | 292/2500 [1:39:27<14:46:37, 24.09s/it] {'loss': 0.0002, 'grad_norm': 0.7240437490195898, 'learning_rate': 8.832e-07, 'completion_length': 154.75894165039062, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.128351628780365, 'kl': 0.0050201416015625, 'epoch': 0.12} 12%|█▏ | 292/2500 [1:39:27<14:46:37, 24.09s/it] 12%|█▏ | 293/2500 [1:39:58<16:01:43, 26.15s/it] {'loss': 0.0002, 'grad_norm': 0.029288503410604624, 'learning_rate': 8.827999999999999e-07, 'completion_length': 147.83036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.12} 12%|█▏ | 293/2500 [1:39:58<16:01:43, 26.15s/it] 12%|█▏ | 294/2500 [1:40:22<15:33:20, 25.39s/it] {'loss': 0.0003, 'grad_norm': 0.4265123353889468, 'learning_rate': 8.823999999999999e-07, 'completion_length': 156.3214340209961, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.09138382971286774, 'kl': 0.007080078125, 'epoch': 0.12} 12%|█▏ | 294/2500 [1:40:22<15:33:20, 25.39s/it] 12%|█▏ | 295/2500 [1:40:44<15:02:11, 24.55s/it] {'loss': 0.0003, 'grad_norm': 0.667789629548754, 'learning_rate': 8.82e-07, 'completion_length': 146.52678680419922, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.07003280520439148, 'kl': 0.00738525390625, 'epoch': 0.12} 12%|█▏ | 295/2500 [1:40:44<15:02:11, 24.55s/it] 12%|█▏ | 296/2500 [1:41:08<14:45:55, 24.12s/it] {'loss': 0.0001, 'grad_norm': 0.24579429773703668, 'learning_rate': 8.816000000000001e-07, 'completion_length': 140.9196548461914, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00368499755859375, 'epoch': 0.12} 12%|█▏ | 296/2500 [1:41:08<14:45:55, 24.12s/it] 12%|█▏ | 297/2500 [1:41:30<14:31:25, 23.73s/it] {'loss': 0.0003, 'grad_norm': 0.3467925370750569, 'learning_rate': 8.811999999999999e-07, 'completion_length': 156.21429443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0063323974609375, 'epoch': 0.12} 12%|█▏ | 297/2500 [1:41:30<14:31:25, 23.73s/it] 12%|█▏ | 298/2500 [1:41:54<14:26:01, 23.60s/it] {'loss': 0.0002, 'grad_norm': 1.3680496132703073, 'learning_rate': 8.808e-07, 'completion_length': 162.2232208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.13225442171096802, 'kl': 0.005767822265625, 'epoch': 0.12} 12%|█▏ | 298/2500 [1:41:54<14:26:01, 23.60s/it] 12%|█▏ | 299/2500 [1:42:17<14:25:14, 23.59s/it] {'loss': 0.0003, 'grad_norm': 0.3873590920979468, 'learning_rate': 8.804e-07, 'completion_length': 164.35714721679688, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.09528661891818047, 'kl': 0.006317138671875, 'epoch': 0.12} 12%|█▏ | 299/2500 [1:42:17<14:25:14, 23.59s/it] 12%|█▏ | 300/2500 [1:42:41<14:28:50, 23.70s/it] {'loss': 0.0002, 'grad_norm': 0.3213798650253609, 'learning_rate': 8.799999999999999e-07, 'completion_length': 159.14286041259766, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0052032470703125, 'epoch': 0.12} 12%|█▏ | 300/2500 [1:42:41<14:28:50, 23.70s/it] 12%|█▏ | 301/2500 [1:43:46<22:06:06, 36.18s/it] {'loss': 0.0002, 'grad_norm': 2.0326697741792072, 'learning_rate': 8.796e-07, 'completion_length': 161.9107208251953, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.13736958801746368, 'kl': 0.006195068359375, 'epoch': 0.12} 12%|█▏ | 301/2500 [1:43:46<22:06:06, 36.18s/it] 12%|█▏ | 302/2500 [1:44:09<19:39:23, 32.19s/it] {'loss': 0.0002, 'grad_norm': 1.1200882873150995, 'learning_rate': 8.792e-07, 'completion_length': 152.0089340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.10040178522467613, 'kl': 0.00616455078125, 'epoch': 0.12} 12%|█▏ | 302/2500 [1:44:09<19:39:23, 32.19s/it] 12%|█▏ | 303/2500 [1:44:33<18:02:59, 29.58s/it] {'loss': 0.0003, 'grad_norm': 0.6443883537840516, 'learning_rate': 8.788e-07, 'completion_length': 159.31250762939453, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.13979434967041016, 'kl': 0.00701904296875, 'epoch': 0.12} 12%|█▏ | 303/2500 [1:44:33<18:02:59, 29.58s/it] 12%|█▏ | 304/2500 [1:44:56<16:48:40, 27.56s/it] {'loss': 0.0002, 'grad_norm': 0.18551747634170399, 'learning_rate': 8.783999999999999e-07, 'completion_length': 146.11608123779297, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.005584716796875, 'epoch': 0.12} 12%|█▏ | 304/2500 [1:44:56<16:48:40, 27.56s/it] 12%|█▏ | 305/2500 [1:45:18<15:51:48, 26.02s/it] {'loss': 0.0002, 'grad_norm': 0.5415320096799152, 'learning_rate': 8.78e-07, 'completion_length': 145.00000762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11272924393415451, 'kl': 0.0053558349609375, 'epoch': 0.12} 12%|█▏ | 305/2500 [1:45:18<15:51:48, 26.02s/it] 12%|█▏ | 306/2500 [1:45:40<15:10:50, 24.91s/it] {'loss': 0.0002, 'grad_norm': 0.5137318835436104, 'learning_rate': 8.776e-07, 'completion_length': 144.3303680419922, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8214285969734192, 'reward_std': 0.0835726335644722, 'kl': 0.005584716796875, 'epoch': 0.12} 12%|█▏ | 306/2500 [1:45:40<15:10:50, 24.91s/it] 12%|█▏ | 307/2500 [1:46:03<14:49:12, 24.33s/it] {'loss': 0.0002, 'grad_norm': 0.534510739842596, 'learning_rate': 8.771999999999999e-07, 'completion_length': 150.08036041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.08747542649507523, 'kl': 0.0054931640625, 'epoch': 0.12} 12%|█▏ | 307/2500 [1:46:03<14:49:12, 24.33s/it] 12%|█▏ | 308/2500 [1:46:26<14:33:32, 23.91s/it] {'loss': 0.0002, 'grad_norm': 0.39652601566167506, 'learning_rate': 8.768e-07, 'completion_length': 158.0714340209961, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.1030978113412857, 'kl': 0.0055389404296875, 'epoch': 0.12} 12%|█▏ | 308/2500 [1:46:26<14:33:32, 23.91s/it] 12%|█▏ | 309/2500 [1:46:50<14:28:31, 23.78s/it] {'loss': 0.0002, 'grad_norm': 0.643324918690335, 'learning_rate': 8.763999999999999e-07, 'completion_length': 162.4732208251953, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.128351628780365, 'kl': 0.0056304931640625, 'epoch': 0.12} 12%|█▏ | 309/2500 [1:46:50<14:28:31, 23.78s/it] 12%|█▏ | 310/2500 [1:47:14<14:28:32, 23.80s/it] {'loss': 0.0002, 'grad_norm': 0.5303393972209495, 'learning_rate': 8.76e-07, 'completion_length': 157.51786041259766, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.12956400588154793, 'kl': 0.00586700439453125, 'epoch': 0.12} 12%|█▏ | 310/2500 [1:47:14<14:28:32, 23.80s/it] 12%|█▏ | 311/2500 [1:47:37<14:23:34, 23.67s/it] {'loss': 0.0003, 'grad_norm': 0.2770492099802402, 'learning_rate': 8.756e-07, 'completion_length': 155.80358123779297, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.006622314453125, 'epoch': 0.12} 12%|█▏ | 311/2500 [1:47:37<14:23:34, 23.67s/it] 12%|█▏ | 312/2500 [1:48:01<14:26:42, 23.77s/it] {'loss': 0.0002, 'grad_norm': 0.8220259309374518, 'learning_rate': 8.751999999999999e-07, 'completion_length': 167.0089340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.00567626953125, 'epoch': 0.12} 12%|█▏ | 312/2500 [1:48:01<14:26:42, 23.77s/it] 13%|█▎ | 313/2500 [1:48:24<14:13:03, 23.40s/it] {'loss': 0.0002, 'grad_norm': 0.3768401164194076, 'learning_rate': 8.748e-07, 'completion_length': 138.90179443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.005126953125, 'epoch': 0.13} 13%|█▎ | 313/2500 [1:48:24<14:13:03, 23.40s/it] 13%|█▎ | 314/2500 [1:48:48<14:18:38, 23.57s/it] {'loss': 0.0002, 'grad_norm': 0.36739779314345417, 'learning_rate': 8.743999999999999e-07, 'completion_length': 143.55358123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0054168701171875, 'epoch': 0.13} 13%|█▎ | 314/2500 [1:48:48<14:18:38, 23.57s/it] 13%|█▎ | 315/2500 [1:49:12<14:25:11, 23.76s/it] {'loss': 0.0002, 'grad_norm': 0.8894292503772113, 'learning_rate': 8.739999999999999e-07, 'completion_length': 153.62500762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.10821298509836197, 'kl': 0.00604248046875, 'epoch': 0.13} 13%|█▎ | 315/2500 [1:49:12<14:25:11, 23.76s/it] 13%|█▎ | 316/2500 [1:49:35<14:23:24, 23.72s/it] {'loss': 0.0003, 'grad_norm': 0.995613830868186, 'learning_rate': 8.736e-07, 'completion_length': 162.85714721679688, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.006500244140625, 'epoch': 0.13} 13%|█▎ | 316/2500 [1:49:35<14:23:24, 23.72s/it] 13%|█▎ | 317/2500 [1:49:58<14:16:05, 23.53s/it] {'loss': 0.0003, 'grad_norm': 0.029844617559615975, 'learning_rate': 8.732e-07, 'completion_length': 158.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0074462890625, 'epoch': 0.13} 13%|█▎ | 317/2500 [1:49:58<14:16:05, 23.53s/it] 13%|█▎ | 318/2500 [1:50:22<14:11:47, 23.42s/it] {'loss': 0.0002, 'grad_norm': 0.5717341712214439, 'learning_rate': 8.728e-07, 'completion_length': 143.17857360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0046539306640625, 'epoch': 0.13} 13%|█▎ | 318/2500 [1:50:22<14:11:47, 23.42s/it] 13%|█▎ | 319/2500 [1:50:45<14:15:15, 23.53s/it] {'loss': 0.0003, 'grad_norm': 0.9002738620880417, 'learning_rate': 8.723999999999999e-07, 'completion_length': 172.41964721679688, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.17885924875736237, 'kl': 0.0084381103515625, 'epoch': 0.13} 13%|█▎ | 319/2500 [1:50:45<14:15:15, 23.53s/it] 13%|█▎ | 320/2500 [1:51:09<14:12:05, 23.45s/it] {'loss': 0.0003, 'grad_norm': 0.4256728330827929, 'learning_rate': 8.72e-07, 'completion_length': 163.3482208251953, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.0071868896484375, 'epoch': 0.13} 13%|█▎ | 320/2500 [1:51:09<14:12:05, 23.45s/it] 13%|█▎ | 321/2500 [1:51:31<14:03:29, 23.23s/it] {'loss': 0.0002, 'grad_norm': 0.5022574028424076, 'learning_rate': 8.716e-07, 'completion_length': 150.1607208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.11663764342665672, 'kl': 0.0060272216796875, 'epoch': 0.13} 13%|█▎ | 321/2500 [1:51:31<14:03:29, 23.23s/it] 13%|█▎ | 322/2500 [1:51:55<14:05:28, 23.29s/it] {'loss': 0.0002, 'grad_norm': 0.3670201496471583, 'learning_rate': 8.711999999999999e-07, 'completion_length': 158.3928680419922, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382598757744, 'kl': 0.005645751953125, 'epoch': 0.13} 13%|█▎ | 322/2500 [1:51:55<14:05:28, 23.29s/it] 13%|█▎ | 323/2500 [1:52:18<13:59:07, 23.13s/it] {'loss': 0.0002, 'grad_norm': 0.265678424043783, 'learning_rate': 8.708e-07, 'completion_length': 144.58036041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00434112548828125, 'epoch': 0.13} 13%|█▎ | 323/2500 [1:52:18<13:59:07, 23.13s/it] 13%|█▎ | 324/2500 [1:52:41<14:04:59, 23.30s/it] {'loss': 0.0003, 'grad_norm': 0.6179413781313683, 'learning_rate': 8.704e-07, 'completion_length': 163.68750762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.13408026844263077, 'kl': 0.007537841796875, 'epoch': 0.13} 13%|█▎ | 324/2500 [1:52:41<14:04:59, 23.30s/it] 13%|█▎ | 325/2500 [1:53:04<13:58:01, 23.12s/it] {'loss': 0.0003, 'grad_norm': 1.0342397312768699, 'learning_rate': 8.699999999999999e-07, 'completion_length': 146.66964721679688, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.19178562611341476, 'kl': 0.0069122314453125, 'epoch': 0.13} 13%|█▎ | 325/2500 [1:53:04<13:58:01, 23.12s/it] 13%|█▎ | 326/2500 [1:53:27<13:58:21, 23.14s/it] {'loss': 0.0002, 'grad_norm': 0.40080303746213136, 'learning_rate': 8.696e-07, 'completion_length': 148.77679443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.005584716796875, 'epoch': 0.13} 13%|█▎ | 326/2500 [1:53:27<13:58:21, 23.14s/it] 13%|█▎ | 327/2500 [1:53:49<13:43:56, 22.75s/it] {'loss': 0.0001, 'grad_norm': 0.02179459627184672, 'learning_rate': 8.692e-07, 'completion_length': 136.9732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037384033203125, 'epoch': 0.13} 13%|█▎ | 327/2500 [1:53:49<13:43:56, 22.75s/it] 13%|█▎ | 328/2500 [1:54:12<13:43:57, 22.76s/it] {'loss': 0.0003, 'grad_norm': 0.8541853480820384, 'learning_rate': 8.687999999999999e-07, 'completion_length': 153.00000762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.15872061252593994, 'kl': 0.007232666015625, 'epoch': 0.13} 13%|█▎ | 328/2500 [1:54:12<13:43:57, 22.76s/it] 13%|█▎ | 329/2500 [1:54:34<13:43:01, 22.75s/it] {'loss': 0.0003, 'grad_norm': 0.46494837536502565, 'learning_rate': 8.683999999999999e-07, 'completion_length': 154.7857208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.007659912109375, 'epoch': 0.13} 13%|█▎ | 329/2500 [1:54:34<13:43:01, 22.75s/it] 13%|█▎ | 330/2500 [1:54:58<13:48:11, 22.90s/it] {'loss': 0.0003, 'grad_norm': 0.5324463896808379, 'learning_rate': 8.68e-07, 'completion_length': 157.81250762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.0070648193359375, 'epoch': 0.13} 13%|█▎ | 330/2500 [1:54:58<13:48:11, 22.90s/it] 13%|█▎ | 331/2500 [1:55:22<13:59:58, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.961031309650954, 'learning_rate': 8.676e-07, 'completion_length': 142.2857208251953, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.08868780359625816, 'kl': 0.00469970703125, 'epoch': 0.13} 13%|█▎ | 331/2500 [1:55:22<13:59:58, 23.24s/it] 13%|█▎ | 332/2500 [1:55:45<14:04:19, 23.37s/it] {'loss': 0.0003, 'grad_norm': 0.516866871935292, 'learning_rate': 8.671999999999999e-07, 'completion_length': 165.9464340209961, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.14700662344694138, 'kl': 0.007354736328125, 'epoch': 0.13} 13%|█▎ | 332/2500 [1:55:45<14:04:19, 23.37s/it] 13%|█▎ | 333/2500 [1:56:10<14:15:20, 23.68s/it] {'loss': 0.0003, 'grad_norm': 1.9070200676337339, 'learning_rate': 8.668e-07, 'completion_length': 173.8928680419922, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.1575138419866562, 'kl': 0.0068511962890625, 'epoch': 0.13} 13%|█▎ | 333/2500 [1:56:10<14:15:20, 23.68s/it] 13%|█▎ | 334/2500 [1:56:33<14:11:05, 23.58s/it] {'loss': 0.0002, 'grad_norm': 0.027503046075551905, 'learning_rate': 8.663999999999999e-07, 'completion_length': 160.0178680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.006011962890625, 'epoch': 0.13} 13%|█▎ | 334/2500 [1:56:33<14:11:05, 23.58s/it] 13%|█▎ | 335/2500 [1:57:00<14:48:42, 24.63s/it] {'loss': 0.0002, 'grad_norm': 0.1771123348312604, 'learning_rate': 8.659999999999999e-07, 'completion_length': 154.8482208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0057830810546875, 'epoch': 0.13} 13%|█▎ | 335/2500 [1:57:00<14:48:42, 24.63s/it] 13%|█▎ | 336/2500 [1:57:23<14:32:45, 24.20s/it] {'loss': 0.0002, 'grad_norm': 0.19189687436952502, 'learning_rate': 8.656e-07, 'completion_length': 140.51786041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0052032470703125, 'epoch': 0.13} 13%|█▎ | 336/2500 [1:57:23<14:32:45, 24.20s/it] 13%|█▎ | 337/2500 [1:57:47<14:27:56, 24.08s/it] {'loss': 0.0003, 'grad_norm': 0.7154499836942257, 'learning_rate': 8.651999999999999e-07, 'completion_length': 156.5357208251953, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.150909423828125, 'kl': 0.0073699951171875, 'epoch': 0.13} 13%|█▎ | 337/2500 [1:57:47<14:27:56, 24.08s/it] 14%|█▎ | 338/2500 [1:58:11<14:19:01, 23.84s/it] {'loss': 0.0003, 'grad_norm': 1.5104946231477154, 'learning_rate': 8.648e-07, 'completion_length': 166.0625, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.16323686391115189, 'kl': 0.00860595703125, 'epoch': 0.14} 14%|█▎ | 338/2500 [1:58:11<14:19:01, 23.84s/it] 14%|█▎ | 339/2500 [1:58:32<13:57:35, 23.26s/it] {'loss': 0.0002, 'grad_norm': 0.20202458267062165, 'learning_rate': 8.643999999999999e-07, 'completion_length': 130.93750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00394439697265625, 'epoch': 0.14} 14%|█▎ | 339/2500 [1:58:32<13:57:35, 23.26s/it] 14%|█▎ | 340/2500 [1:58:56<13:56:38, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.3975902162609452, 'learning_rate': 8.639999999999999e-07, 'completion_length': 154.58929443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0047607421875, 'epoch': 0.14} 14%|█▎ | 340/2500 [1:58:56<13:56:38, 23.24s/it] 14%|█▎ | 341/2500 [1:59:19<13:56:07, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.18762373481409034, 'learning_rate': 8.636e-07, 'completion_length': 162.96428680419922, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0061492919921875, 'epoch': 0.14} 14%|█▎ | 341/2500 [1:59:19<13:56:07, 23.24s/it] 14%|█▎ | 342/2500 [1:59:42<13:52:07, 23.14s/it] {'loss': 0.0003, 'grad_norm': 0.6182577721745269, 'learning_rate': 8.632e-07, 'completion_length': 158.0089340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.1509094163775444, 'kl': 0.006317138671875, 'epoch': 0.14} 14%|█▎ | 342/2500 [1:59:42<13:52:07, 23.14s/it] 14%|█▎ | 343/2500 [2:00:05<13:50:26, 23.10s/it] {'loss': 0.0003, 'grad_norm': 0.38903461331051625, 'learning_rate': 8.628e-07, 'completion_length': 154.16964721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.006805419921875, 'epoch': 0.14} 14%|█▎ | 343/2500 [2:00:05<13:50:26, 23.10s/it] 14%|█▍ | 344/2500 [2:00:28<13:46:20, 23.00s/it] {'loss': 0.0003, 'grad_norm': 2.2229812893480987, 'learning_rate': 8.624e-07, 'completion_length': 138.49108123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.0069580078125, 'epoch': 0.14} 14%|█▍ | 344/2500 [2:00:28<13:46:20, 23.00s/it] 14%|█▍ | 345/2500 [2:00:50<13:43:10, 22.92s/it] {'loss': 0.0003, 'grad_norm': 0.2950411021541653, 'learning_rate': 8.62e-07, 'completion_length': 144.8214340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.00665283203125, 'epoch': 0.14} 14%|█▍ | 345/2500 [2:00:50<13:43:10, 22.92s/it] 14%|█▍ | 346/2500 [2:01:13<13:44:05, 22.96s/it] {'loss': 0.0002, 'grad_norm': 0.29424181619416545, 'learning_rate': 8.616e-07, 'completion_length': 161.62500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00616455078125, 'epoch': 0.14} 14%|█▍ | 346/2500 [2:01:13<13:44:05, 22.96s/it] 14%|█▍ | 347/2500 [2:01:37<13:55:09, 23.27s/it] {'loss': 0.0002, 'grad_norm': 0.030554384855184927, 'learning_rate': 8.611999999999999e-07, 'completion_length': 146.6428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048828125, 'epoch': 0.14} 14%|█▍ | 347/2500 [2:01:37<13:55:09, 23.27s/it] 14%|█▍ | 348/2500 [2:02:00<13:51:45, 23.19s/it] {'loss': 0.0002, 'grad_norm': 0.28297904538050206, 'learning_rate': 8.608e-07, 'completion_length': 143.4732208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0058441162109375, 'epoch': 0.14} 14%|█▍ | 348/2500 [2:02:00<13:51:45, 23.19s/it] 14%|█▍ | 349/2500 [2:02:23<13:49:24, 23.14s/it] {'loss': 0.0003, 'grad_norm': 0.6566731393490205, 'learning_rate': 8.604000000000001e-07, 'completion_length': 150.7232208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747542649507523, 'kl': 0.0074920654296875, 'epoch': 0.14} 14%|█▍ | 349/2500 [2:02:23<13:49:24, 23.14s/it] 14%|█▍ | 350/2500 [2:02:48<14:00:25, 23.45s/it] {'loss': 0.0003, 'grad_norm': 2.4793371042414116, 'learning_rate': 8.599999999999999e-07, 'completion_length': 167.0178680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.12054044008255005, 'kl': 0.0083770751953125, 'epoch': 0.14} 14%|█▍ | 350/2500 [2:02:48<14:00:25, 23.45s/it] 14%|█▍ | 351/2500 [2:03:10<13:53:38, 23.28s/it] {'loss': 0.0002, 'grad_norm': 0.5584687816880416, 'learning_rate': 8.596e-07, 'completion_length': 147.5357208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00437164306640625, 'epoch': 0.14} 14%|█▍ | 351/2500 [2:03:10<13:53:38, 23.28s/it] 14%|█▍ | 352/2500 [2:03:33<13:51:29, 23.23s/it] {'loss': 0.0002, 'grad_norm': 0.7202202045829529, 'learning_rate': 8.592e-07, 'completion_length': 152.30358123779297, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.09528662264347076, 'kl': 0.0059967041015625, 'epoch': 0.14} 14%|█▍ | 352/2500 [2:03:33<13:51:29, 23.23s/it] 14%|█▍ | 353/2500 [2:03:58<14:00:37, 23.49s/it] {'loss': 0.0003, 'grad_norm': 0.478266537264378, 'learning_rate': 8.587999999999999e-07, 'completion_length': 153.9107208251953, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382971286774, 'kl': 0.0071563720703125, 'epoch': 0.14} 14%|█▍ | 353/2500 [2:03:58<14:00:37, 23.49s/it] 14%|█▍ | 354/2500 [2:04:20<13:52:33, 23.28s/it] {'loss': 0.0003, 'grad_norm': 1.0133820615119191, 'learning_rate': 8.584e-07, 'completion_length': 149.73214721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.0074615478515625, 'epoch': 0.14} 14%|█▍ | 354/2500 [2:04:20<13:52:33, 23.28s/it] 14%|█▍ | 355/2500 [2:04:43<13:41:55, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.24765352595330514, 'learning_rate': 8.58e-07, 'completion_length': 134.42857360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00452423095703125, 'epoch': 0.14} 14%|█▍ | 355/2500 [2:04:43<13:41:55, 22.99s/it] 14%|█▍ | 356/2500 [2:05:05<13:35:12, 22.81s/it] {'loss': 0.0003, 'grad_norm': 0.4431241417439708, 'learning_rate': 8.576e-07, 'completion_length': 144.85714721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0073699951171875, 'epoch': 0.14} 14%|█▍ | 356/2500 [2:05:05<13:35:12, 22.81s/it] 14%|█▍ | 357/2500 [2:05:28<13:40:34, 22.97s/it] {'loss': 0.0003, 'grad_norm': 0.31142686848639073, 'learning_rate': 8.571999999999999e-07, 'completion_length': 163.80358123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797151088715, 'kl': 0.00754547119140625, 'epoch': 0.14} 14%|█▍ | 357/2500 [2:05:28<13:40:34, 22.97s/it] 14%|█▍ | 358/2500 [2:05:52<13:46:15, 23.14s/it] {'loss': 0.0002, 'grad_norm': 0.4460632190108415, 'learning_rate': 8.568e-07, 'completion_length': 149.8839340209961, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06613001227378845, 'kl': 0.0057373046875, 'epoch': 0.14} 14%|█▍ | 358/2500 [2:05:52<13:46:15, 23.14s/it] 14%|█▍ | 359/2500 [2:06:16<13:51:29, 23.30s/it] {'loss': 0.0002, 'grad_norm': 0.5713356308403106, 'learning_rate': 8.564e-07, 'completion_length': 151.33036041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.096499003469944, 'kl': 0.004974365234375, 'epoch': 0.14} 14%|█▍ | 359/2500 [2:06:16<13:51:29, 23.30s/it] 14%|█▍ | 360/2500 [2:06:39<13:51:13, 23.31s/it] {'loss': 0.0002, 'grad_norm': 0.500734237787856, 'learning_rate': 8.559999999999999e-07, 'completion_length': 154.01786041259766, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.1502959430217743, 'kl': 0.006072998046875, 'epoch': 0.14} 14%|█▍ | 360/2500 [2:06:39<13:51:13, 23.31s/it] 14%|█▍ | 361/2500 [2:07:02<13:46:11, 23.18s/it] {'loss': 0.0002, 'grad_norm': 0.5903366835520157, 'learning_rate': 8.556e-07, 'completion_length': 155.2232208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.09528662264347076, 'kl': 0.0059967041015625, 'epoch': 0.14} 14%|█▍ | 361/2500 [2:07:02<13:46:11, 23.18s/it] 14%|█▍ | 362/2500 [2:07:25<13:45:31, 23.17s/it] {'loss': 0.0004, 'grad_norm': 0.8906056587008819, 'learning_rate': 8.551999999999999e-07, 'completion_length': 158.93750762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.17104807496070862, 'kl': 0.009307861328125, 'epoch': 0.14} 14%|█▍ | 362/2500 [2:07:25<13:45:31, 23.17s/it] 15%|█▍ | 363/2500 [2:07:48<13:38:44, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.4449467341399462, 'learning_rate': 8.548e-07, 'completion_length': 144.56250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00479888916015625, 'epoch': 0.15} 15%|█▍ | 363/2500 [2:07:48<13:38:44, 22.99s/it] 15%|█▍ | 364/2500 [2:08:10<13:31:52, 22.81s/it] {'loss': 0.0002, 'grad_norm': 0.40677892765670726, 'learning_rate': 8.544e-07, 'completion_length': 143.52678680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.0739355981349945, 'kl': 0.0047607421875, 'epoch': 0.15} 15%|█▍ | 364/2500 [2:08:10<13:31:52, 22.81s/it] 15%|█▍ | 365/2500 [2:08:33<13:30:23, 22.77s/it] {'loss': 0.0002, 'grad_norm': 0.5513149797121523, 'learning_rate': 8.539999999999999e-07, 'completion_length': 148.11607360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0053253173828125, 'epoch': 0.15} 15%|█▍ | 365/2500 [2:08:33<13:30:23, 22.77s/it] 15%|█▍ | 366/2500 [2:08:56<13:34:33, 22.90s/it] {'loss': 0.0003, 'grad_norm': 0.48044932647502264, 'learning_rate': 8.536e-07, 'completion_length': 155.5714340209961, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00640869140625, 'epoch': 0.15} 15%|█▍ | 366/2500 [2:08:56<13:34:33, 22.90s/it] 15%|█▍ | 367/2500 [2:09:19<13:32:10, 22.85s/it] {'loss': 0.0002, 'grad_norm': 0.524329291846398, 'learning_rate': 8.531999999999999e-07, 'completion_length': 151.16964721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.004486083984375, 'epoch': 0.15} 15%|█▍ | 367/2500 [2:09:19<13:32:10, 22.85s/it] 15%|█▍ | 368/2500 [2:09:42<13:35:27, 22.95s/it] {'loss': 0.0003, 'grad_norm': 0.9073801509366503, 'learning_rate': 8.528e-07, 'completion_length': 154.2589340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.18397441506385803, 'kl': 0.0063018798828125, 'epoch': 0.15} 15%|█▍ | 368/2500 [2:09:42<13:35:27, 22.95s/it] 15%|█▍ | 369/2500 [2:10:05<13:39:22, 23.07s/it] {'loss': 0.0003, 'grad_norm': 0.8913859055860988, 'learning_rate': 8.524e-07, 'completion_length': 160.4464340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.006622314453125, 'epoch': 0.15} 15%|█▍ | 369/2500 [2:10:05<13:39:22, 23.07s/it] 15%|█▍ | 370/2500 [2:10:29<13:48:29, 23.34s/it] {'loss': 0.0002, 'grad_norm': 0.2856032310833569, 'learning_rate': 8.52e-07, 'completion_length': 162.86608123779297, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831882357597351, 'kl': 0.0062408447265625, 'epoch': 0.15} 15%|█▍ | 370/2500 [2:10:29<13:48:29, 23.34s/it] 15%|█▍ | 371/2500 [2:10:52<13:41:41, 23.16s/it] {'loss': 0.0003, 'grad_norm': 0.9437426567507982, 'learning_rate': 8.516e-07, 'completion_length': 143.2232208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0064849853515625, 'epoch': 0.15} 15%|█▍ | 371/2500 [2:10:52<13:41:41, 23.16s/it] 15%|█▍ | 372/2500 [2:11:16<13:52:09, 23.46s/it] {'loss': 0.0002, 'grad_norm': 0.45272200205297125, 'learning_rate': 8.511999999999999e-07, 'completion_length': 153.8482208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0835726372897625, 'kl': 0.0055999755859375, 'epoch': 0.15} 15%|█▍ | 372/2500 [2:11:16<13:52:09, 23.46s/it] 15%|█▍ | 373/2500 [2:11:39<13:51:35, 23.46s/it] {'loss': 0.0003, 'grad_norm': 0.5750255809898536, 'learning_rate': 8.508e-07, 'completion_length': 155.64286041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.15360544621944427, 'kl': 0.0071258544921875, 'epoch': 0.15} 15%|█▍ | 373/2500 [2:11:39<13:51:35, 23.46s/it] 15%|█▍ | 374/2500 [2:12:02<13:44:00, 23.26s/it] {'loss': 0.0002, 'grad_norm': 0.39188496751297097, 'learning_rate': 8.504e-07, 'completion_length': 149.55358123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.005279541015625, 'epoch': 0.15} 15%|█▍ | 374/2500 [2:12:02<13:44:00, 23.26s/it] 15%|█▌ | 375/2500 [2:12:25<13:41:17, 23.19s/it] {'loss': 0.0002, 'grad_norm': 0.2552106022979255, 'learning_rate': 8.499999999999999e-07, 'completion_length': 155.0357208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0055084228515625, 'epoch': 0.15} 15%|█▌ | 375/2500 [2:12:25<13:41:17, 23.19s/it] 15%|█▌ | 376/2500 [2:12:49<13:42:30, 23.23s/it] {'loss': 0.0003, 'grad_norm': 0.4876700490427566, 'learning_rate': 8.496e-07, 'completion_length': 156.0178680419922, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.1030978113412857, 'kl': 0.0064697265625, 'epoch': 0.15} 15%|█▌ | 376/2500 [2:12:49<13:42:30, 23.23s/it] 15%|█▌ | 377/2500 [2:13:12<13:41:11, 23.21s/it] {'loss': 0.0003, 'grad_norm': 0.16104846428300215, 'learning_rate': 8.492e-07, 'completion_length': 153.5982208251953, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.03696779906749725, 'kl': 0.0066986083984375, 'epoch': 0.15} 15%|█▌ | 377/2500 [2:13:12<13:41:11, 23.21s/it] 15%|█▌ | 378/2500 [2:13:36<13:51:11, 23.50s/it] {'loss': 0.0002, 'grad_norm': 0.49381890514093274, 'learning_rate': 8.487999999999999e-07, 'completion_length': 152.71429443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882644355297089, 'kl': 0.0059661865234375, 'epoch': 0.15} 15%|█▌ | 378/2500 [2:13:36<13:51:11, 23.50s/it] 15%|█▌ | 379/2500 [2:13:59<13:47:57, 23.42s/it] {'loss': 0.0002, 'grad_norm': 0.4044208285892237, 'learning_rate': 8.484e-07, 'completion_length': 149.58036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0055389404296875, 'epoch': 0.15} 15%|█▌ | 379/2500 [2:13:59<13:47:57, 23.42s/it] 15%|█▌ | 380/2500 [2:14:22<13:41:14, 23.24s/it] {'loss': 0.0003, 'grad_norm': 1.0854024885285527, 'learning_rate': 8.48e-07, 'completion_length': 164.17857360839844, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.12054044008255005, 'kl': 0.00787353515625, 'epoch': 0.15} 15%|█▌ | 380/2500 [2:14:22<13:41:14, 23.24s/it] 15%|█▌ | 381/2500 [2:14:45<13:41:25, 23.26s/it] {'loss': 0.0003, 'grad_norm': 0.3275005090858387, 'learning_rate': 8.475999999999999e-07, 'completion_length': 147.5625, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.007232666015625, 'epoch': 0.15} 15%|█▌ | 381/2500 [2:14:45<13:41:25, 23.26s/it] 15%|█▌ | 382/2500 [2:15:08<13:39:00, 23.20s/it] {'loss': 0.0002, 'grad_norm': 0.8449725518018681, 'learning_rate': 8.471999999999999e-07, 'completion_length': 150.86608123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.00567626953125, 'epoch': 0.15} 15%|█▌ | 382/2500 [2:15:08<13:39:00, 23.20s/it] 15%|█▌ | 383/2500 [2:15:31<13:31:31, 23.00s/it] {'loss': 0.0002, 'grad_norm': 0.35740037819559595, 'learning_rate': 8.468e-07, 'completion_length': 150.42857360839844, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0061798095703125, 'epoch': 0.15} 15%|█▌ | 383/2500 [2:15:31<13:31:31, 23.00s/it] 15%|█▌ | 384/2500 [2:15:55<13:44:21, 23.37s/it] {'loss': 0.0002, 'grad_norm': 0.996268401376611, 'learning_rate': 8.464e-07, 'completion_length': 143.2857208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.08747542649507523, 'kl': 0.0049285888671875, 'epoch': 0.15} 15%|█▌ | 384/2500 [2:15:55<13:44:21, 23.37s/it] 15%|█▌ | 385/2500 [2:16:18<13:38:51, 23.23s/it] {'loss': 0.0002, 'grad_norm': 0.42703564119220583, 'learning_rate': 8.459999999999999e-07, 'completion_length': 149.43750762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941557407379, 'kl': 0.0054168701171875, 'epoch': 0.15} 15%|█▌ | 385/2500 [2:16:18<13:38:51, 23.23s/it] 15%|█▌ | 386/2500 [2:16:40<13:30:23, 23.00s/it] {'loss': 0.0002, 'grad_norm': 2.522923651687034, 'learning_rate': 8.456e-07, 'completion_length': 139.50000762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004547119140625, 'epoch': 0.15} 15%|█▌ | 386/2500 [2:16:40<13:30:23, 23.00s/it] 15%|█▌ | 387/2500 [2:17:05<13:43:33, 23.39s/it] {'loss': 0.0002, 'grad_norm': 0.49117705783536725, 'learning_rate': 8.451999999999999e-07, 'completion_length': 139.83036041259766, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.096499003469944, 'kl': 0.0039520263671875, 'epoch': 0.15} 15%|█▌ | 387/2500 [2:17:05<13:43:33, 23.39s/it] 16%|█▌ | 388/2500 [2:17:27<13:35:04, 23.16s/it] {'loss': 0.0002, 'grad_norm': 0.28824449544482367, 'learning_rate': 8.447999999999999e-07, 'completion_length': 141.1071548461914, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00463104248046875, 'epoch': 0.16} 16%|█▌ | 388/2500 [2:17:27<13:35:04, 23.16s/it] 16%|█▌ | 389/2500 [2:17:51<13:35:22, 23.18s/it] {'loss': 0.0002, 'grad_norm': 0.5365510764847855, 'learning_rate': 8.444e-07, 'completion_length': 155.23214721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006195068359375, 'epoch': 0.16} 16%|█▌ | 389/2500 [2:17:51<13:35:22, 23.18s/it] 16%|█▌ | 390/2500 [2:18:13<13:31:12, 23.07s/it] {'loss': 0.0002, 'grad_norm': 0.5193992293251337, 'learning_rate': 8.439999999999999e-07, 'completion_length': 142.42857360839844, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.004974365234375, 'epoch': 0.16} 16%|█▌ | 390/2500 [2:18:13<13:31:12, 23.07s/it] 16%|█▌ | 391/2500 [2:18:36<13:22:54, 22.84s/it] {'loss': 0.0002, 'grad_norm': 0.5032008013646401, 'learning_rate': 8.436e-07, 'completion_length': 144.375, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.005523681640625, 'epoch': 0.16} 16%|█▌ | 391/2500 [2:18:36<13:22:54, 22.84s/it] 16%|█▌ | 392/2500 [2:18:59<13:22:39, 22.85s/it] {'loss': 0.0002, 'grad_norm': 8.082024149779254, 'learning_rate': 8.431999999999999e-07, 'completion_length': 141.08928680419922, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.09528662264347076, 'kl': 0.00604248046875, 'epoch': 0.16} 16%|█▌ | 392/2500 [2:18:59<13:22:39, 22.85s/it] 16%|█▌ | 393/2500 [2:19:21<13:21:20, 22.82s/it] {'loss': 0.0002, 'grad_norm': 0.20680127897946926, 'learning_rate': 8.428e-07, 'completion_length': 145.3214340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0044708251953125, 'epoch': 0.16} 16%|█▌ | 393/2500 [2:19:21<13:21:20, 22.82s/it] 16%|█▌ | 394/2500 [2:19:44<13:19:57, 22.79s/it] {'loss': 0.0002, 'grad_norm': 0.4077925538719283, 'learning_rate': 8.424e-07, 'completion_length': 142.71429443359375, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0060272216796875, 'epoch': 0.16} 16%|█▌ | 394/2500 [2:19:44<13:19:57, 22.79s/it] 16%|█▌ | 395/2500 [2:20:07<13:18:07, 22.75s/it] {'loss': 0.0003, 'grad_norm': 0.46941721875552506, 'learning_rate': 8.419999999999999e-07, 'completion_length': 152.15179443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0066070556640625, 'epoch': 0.16} 16%|█▌ | 395/2500 [2:20:07<13:18:07, 22.75s/it] 16%|█▌ | 396/2500 [2:20:31<13:30:26, 23.11s/it] {'loss': 0.0003, 'grad_norm': 0.2391380957276617, 'learning_rate': 8.416e-07, 'completion_length': 145.8303680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0072021484375, 'epoch': 0.16} 16%|█▌ | 396/2500 [2:20:31<13:30:26, 23.11s/it] 16%|█▌ | 397/2500 [2:20:53<13:22:55, 22.91s/it] {'loss': 0.0004, 'grad_norm': 0.5086996824088768, 'learning_rate': 8.411999999999999e-07, 'completion_length': 147.9732208251953, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07576143741607666, 'kl': 0.008758544921875, 'epoch': 0.16} 16%|█▌ | 397/2500 [2:20:53<13:22:55, 22.91s/it] 16%|█▌ | 398/2500 [2:21:14<13:06:01, 22.44s/it] {'loss': 0.0001, 'grad_norm': 0.027390785618848022, 'learning_rate': 8.408e-07, 'completion_length': 127.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0034332275390625, 'epoch': 0.16} 16%|█▌ | 398/2500 [2:21:14<13:06:01, 22.44s/it] 16%|█▌ | 399/2500 [2:21:37<13:03:29, 22.37s/it] {'loss': 0.0002, 'grad_norm': 0.798481619052695, 'learning_rate': 8.404e-07, 'completion_length': 140.65179443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882645100355148, 'kl': 0.0059051513671875, 'epoch': 0.16} 16%|█▌ | 399/2500 [2:21:37<13:03:29, 22.37s/it] 16%|█▌ | 400/2500 [2:21:59<13:02:51, 22.37s/it] {'loss': 0.0003, 'grad_norm': 0.6002921841924583, 'learning_rate': 8.399999999999999e-07, 'completion_length': 151.2589340209961, 'rewards/accuracy_reward': 0.8125000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8125001192092896, 'reward_std': 0.21763834357261658, 'kl': 0.00628662109375, 'epoch': 0.16} 16%|█▌ | 400/2500 [2:21:59<13:02:51, 22.37s/it] 16%|█▌ | 401/2500 [2:23:12<21:51:25, 37.49s/it] {'loss': 0.0002, 'grad_norm': 0.431632340577804, 'learning_rate': 8.396e-07, 'completion_length': 153.58036041259766, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12175280973315239, 'kl': 0.005340576171875, 'epoch': 0.16} 16%|█▌ | 401/2500 [2:23:12<21:51:25, 37.49s/it] 16%|█▌ | 402/2500 [2:23:36<19:28:45, 33.42s/it] {'loss': 0.0003, 'grad_norm': 0.2724931852761459, 'learning_rate': 8.391999999999999e-07, 'completion_length': 147.60714721679688, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.025253813713788986, 'kl': 0.00738525390625, 'epoch': 0.16} 16%|█▌ | 402/2500 [2:23:36<19:28:45, 33.42s/it] 16%|█▌ | 403/2500 [2:23:59<17:43:46, 30.44s/it] {'loss': 0.0003, 'grad_norm': 0.39925659218179327, 'learning_rate': 8.387999999999999e-07, 'completion_length': 157.11607360839844, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.00775146484375, 'epoch': 0.16} 16%|█▌ | 403/2500 [2:23:59<17:43:46, 30.44s/it] 16%|█▌ | 404/2500 [2:24:22<16:26:27, 28.24s/it] {'loss': 0.0003, 'grad_norm': 1.5039808816344822, 'learning_rate': 8.384e-07, 'completion_length': 154.7946548461914, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.13225442171096802, 'kl': 0.006378173828125, 'epoch': 0.16} 16%|█▌ | 404/2500 [2:24:22<16:26:27, 28.24s/it] 16%|█▌ | 405/2500 [2:24:44<15:21:15, 26.38s/it] {'loss': 0.0002, 'grad_norm': 0.20734938268124714, 'learning_rate': 8.38e-07, 'completion_length': 140.1071548461914, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.005859375, 'epoch': 0.16} 16%|█▌ | 405/2500 [2:24:44<15:21:15, 26.38s/it] 16%|█▌ | 406/2500 [2:25:06<14:35:14, 25.08s/it] {'loss': 0.0003, 'grad_norm': 0.47745963701914135, 'learning_rate': 8.375999999999999e-07, 'completion_length': 133.00000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00640869140625, 'epoch': 0.16} 16%|█▌ | 406/2500 [2:25:06<14:35:14, 25.08s/it] 16%|█▋ | 407/2500 [2:25:29<14:08:51, 24.33s/it] {'loss': 0.0003, 'grad_norm': 0.9253798591701078, 'learning_rate': 8.372e-07, 'completion_length': 148.3482208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.12175281718373299, 'kl': 0.0064697265625, 'epoch': 0.16} 16%|█▋ | 407/2500 [2:25:29<14:08:51, 24.33s/it] 16%|█▋ | 408/2500 [2:25:51<13:47:37, 23.74s/it] {'loss': 0.0002, 'grad_norm': 0.3089180748032445, 'learning_rate': 8.368e-07, 'completion_length': 156.56250762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0053863525390625, 'epoch': 0.16} 16%|█▋ | 408/2500 [2:25:51<13:47:37, 23.74s/it] 16%|█▋ | 409/2500 [2:26:14<13:37:10, 23.45s/it] {'loss': 0.0004, 'grad_norm': 0.7058953186266308, 'learning_rate': 8.363999999999999e-07, 'completion_length': 159.6964340209961, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.12054043635725975, 'kl': 0.009918212890625, 'epoch': 0.16} 16%|█▋ | 409/2500 [2:26:14<13:37:10, 23.45s/it] 16%|█▋ | 410/2500 [2:26:37<13:26:46, 23.16s/it] {'loss': 0.0002, 'grad_norm': 0.3975375704225065, 'learning_rate': 8.359999999999999e-07, 'completion_length': 145.0714340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00499725341796875, 'epoch': 0.16} 16%|█▋ | 410/2500 [2:26:37<13:26:46, 23.16s/it] 16%|█▋ | 411/2500 [2:26:59<13:22:57, 23.06s/it] {'loss': 0.0002, 'grad_norm': 0.3734116103244139, 'learning_rate': 8.356e-07, 'completion_length': 154.5982208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.1030978113412857, 'kl': 0.0052490234375, 'epoch': 0.16} 16%|█▋ | 411/2500 [2:26:59<13:22:57, 23.06s/it] 16%|█▋ | 412/2500 [2:27:22<13:19:57, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.31152480334582977, 'learning_rate': 8.352000000000001e-07, 'completion_length': 155.1339340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.005706787109375, 'epoch': 0.16} 16%|█▋ | 412/2500 [2:27:22<13:19:57, 22.99s/it] 17%|█▋ | 413/2500 [2:27:45<13:12:59, 22.80s/it] {'loss': 0.0003, 'grad_norm': 0.42756364744330677, 'learning_rate': 8.347999999999999e-07, 'completion_length': 156.2857208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.0070648193359375, 'epoch': 0.17} 17%|█▋ | 413/2500 [2:27:45<13:12:59, 22.80s/it] 17%|█▋ | 414/2500 [2:28:08<13:14:09, 22.84s/it] {'loss': 0.0003, 'grad_norm': 0.8932596096168833, 'learning_rate': 8.344e-07, 'completion_length': 161.0982208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0073394775390625, 'epoch': 0.17} 17%|█▋ | 414/2500 [2:28:08<13:14:09, 22.84s/it] 17%|█▋ | 415/2500 [2:28:30<13:05:36, 22.61s/it] {'loss': 0.0002, 'grad_norm': 0.15627326838267236, 'learning_rate': 8.34e-07, 'completion_length': 138.02679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0049896240234375, 'epoch': 0.17} 17%|█▋ | 415/2500 [2:28:30<13:05:36, 22.61s/it] 17%|█▋ | 416/2500 [2:28:52<13:03:41, 22.56s/it] {'loss': 0.0002, 'grad_norm': 0.2530997592750351, 'learning_rate': 8.335999999999999e-07, 'completion_length': 150.61608123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0062255859375, 'epoch': 0.17} 17%|█▋ | 416/2500 [2:28:52<13:03:41, 22.56s/it] 17%|█▋ | 417/2500 [2:29:14<12:55:26, 22.34s/it] {'loss': 0.0002, 'grad_norm': 0.3343204038268746, 'learning_rate': 8.332e-07, 'completion_length': 137.8303680419922, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0062103271484375, 'epoch': 0.17} 17%|█▋ | 417/2500 [2:29:14<12:55:26, 22.34s/it] 17%|█▋ | 418/2500 [2:29:35<12:47:32, 22.12s/it] {'loss': 0.0002, 'grad_norm': 0.37760318962811573, 'learning_rate': 8.328e-07, 'completion_length': 142.55358123779297, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00521087646484375, 'epoch': 0.17} 17%|█▋ | 418/2500 [2:29:35<12:47:32, 22.12s/it] 17%|█▋ | 419/2500 [2:29:58<12:49:13, 22.18s/it] {'loss': 0.0003, 'grad_norm': 0.5445444422102713, 'learning_rate': 8.324e-07, 'completion_length': 144.77679443359375, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.0066070556640625, 'epoch': 0.17} 17%|█▋ | 419/2500 [2:29:58<12:49:13, 22.18s/it] 17%|█▋ | 420/2500 [2:30:20<12:48:20, 22.16s/it] {'loss': 0.0002, 'grad_norm': 0.4236475184750038, 'learning_rate': 8.319999999999999e-07, 'completion_length': 144.75894165039062, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.005340576171875, 'epoch': 0.17} 17%|█▋ | 420/2500 [2:30:20<12:48:20, 22.16s/it] 17%|█▋ | 421/2500 [2:30:42<12:45:22, 22.09s/it] {'loss': 0.0002, 'grad_norm': 0.9769818627224351, 'learning_rate': 8.316e-07, 'completion_length': 130.0982208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.00432586669921875, 'epoch': 0.17} 17%|█▋ | 421/2500 [2:30:42<12:45:22, 22.09s/it] 17%|█▋ | 422/2500 [2:31:04<12:48:32, 22.19s/it] {'loss': 0.0002, 'grad_norm': 0.7182895225269593, 'learning_rate': 8.312e-07, 'completion_length': 158.71429443359375, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.128351628780365, 'kl': 0.0055999755859375, 'epoch': 0.17} 17%|█▋ | 422/2500 [2:31:04<12:48:32, 22.19s/it] 17%|█▋ | 423/2500 [2:31:27<12:53:53, 22.36s/it] {'loss': 0.0003, 'grad_norm': 0.4333207747394023, 'learning_rate': 8.308e-07, 'completion_length': 148.61608123779297, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.0964989997446537, 'kl': 0.0081329345703125, 'epoch': 0.17} 17%|█▋ | 423/2500 [2:31:27<12:53:53, 22.36s/it] 17%|█▋ | 424/2500 [2:31:49<12:49:38, 22.24s/it] {'loss': 0.0002, 'grad_norm': 0.5515852436569484, 'learning_rate': 8.304e-07, 'completion_length': 141.64286041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0049285888671875, 'epoch': 0.17} 17%|█▋ | 424/2500 [2:31:49<12:49:38, 22.24s/it] 17%|█▋ | 425/2500 [2:32:11<12:42:01, 22.03s/it] {'loss': 0.0002, 'grad_norm': 0.02369061674074863, 'learning_rate': 8.299999999999999e-07, 'completion_length': 140.30358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004547119140625, 'epoch': 0.17} 17%|█▋ | 425/2500 [2:32:11<12:42:01, 22.03s/it] 17%|█▋ | 426/2500 [2:32:32<12:40:23, 22.00s/it] {'loss': 0.0002, 'grad_norm': 0.5033327768146049, 'learning_rate': 8.296e-07, 'completion_length': 140.91964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0058135986328125, 'epoch': 0.17} 17%|█▋ | 426/2500 [2:32:32<12:40:23, 22.00s/it] 17%|█▋ | 427/2500 [2:32:55<12:41:42, 22.05s/it] {'loss': 0.0002, 'grad_norm': 0.2583093080576051, 'learning_rate': 8.292e-07, 'completion_length': 148.98214721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.005340576171875, 'epoch': 0.17} 17%|█▋ | 427/2500 [2:32:55<12:41:42, 22.05s/it] 17%|█▋ | 428/2500 [2:33:17<12:41:34, 22.05s/it] {'loss': 0.0002, 'grad_norm': 0.8318581280581012, 'learning_rate': 8.287999999999999e-07, 'completion_length': 148.99108123779297, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.1283516250550747, 'kl': 0.0057525634765625, 'epoch': 0.17} 17%|█▋ | 428/2500 [2:33:17<12:41:34, 22.05s/it] 17%|█▋ | 429/2500 [2:33:39<12:46:58, 22.22s/it] {'loss': 0.0002, 'grad_norm': 1.0527237865055745, 'learning_rate': 8.284e-07, 'completion_length': 151.8303680419922, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.0739355981349945, 'kl': 0.005096435546875, 'epoch': 0.17} 17%|█▋ | 429/2500 [2:33:39<12:46:58, 22.22s/it] 17%|█▋ | 430/2500 [2:34:02<12:50:29, 22.33s/it] {'loss': 0.0002, 'grad_norm': 0.36665625799047186, 'learning_rate': 8.28e-07, 'completion_length': 148.14286041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.00379180908203125, 'epoch': 0.17} 17%|█▋ | 430/2500 [2:34:02<12:50:29, 22.33s/it] 17%|█▋ | 431/2500 [2:34:24<12:47:36, 22.26s/it] {'loss': 0.0003, 'grad_norm': 0.7655288388589886, 'learning_rate': 8.275999999999999e-07, 'completion_length': 148.02679443359375, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.11272924393415451, 'kl': 0.0078887939453125, 'epoch': 0.17} 17%|█▋ | 431/2500 [2:34:24<12:47:36, 22.26s/it] 17%|█▋ | 432/2500 [2:34:47<12:50:01, 22.34s/it] {'loss': 0.0002, 'grad_norm': 0.4001509931794338, 'learning_rate': 8.272e-07, 'completion_length': 147.99107360839844, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0739355981349945, 'kl': 0.005828857421875, 'epoch': 0.17} 17%|█▋ | 432/2500 [2:34:47<12:50:01, 22.34s/it] 17%|█▋ | 433/2500 [2:35:09<12:50:33, 22.37s/it] {'loss': 0.0003, 'grad_norm': 0.5198237693565251, 'learning_rate': 8.268e-07, 'completion_length': 143.58928680419922, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0835726372897625, 'kl': 0.0064849853515625, 'epoch': 0.17} 17%|█▋ | 433/2500 [2:35:09<12:50:33, 22.37s/it] 17%|█▋ | 434/2500 [2:35:31<12:50:15, 22.37s/it] {'loss': 0.0003, 'grad_norm': 2.6416971011555255, 'learning_rate': 8.263999999999999e-07, 'completion_length': 145.23214721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882645100355148, 'kl': 0.0072784423828125, 'epoch': 0.17} 17%|█▋ | 434/2500 [2:35:31<12:50:15, 22.37s/it] 17%|█▋ | 435/2500 [2:35:54<12:53:26, 22.47s/it] {'loss': 0.0003, 'grad_norm': 0.8010031239368415, 'learning_rate': 8.259999999999999e-07, 'completion_length': 152.3839340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.11663764715194702, 'kl': 0.00701904296875, 'epoch': 0.17} 17%|█▋ | 435/2500 [2:35:54<12:53:26, 22.47s/it] 17%|█▋ | 436/2500 [2:36:16<12:52:18, 22.45s/it] {'loss': 0.0002, 'grad_norm': 0.33741782977292517, 'learning_rate': 8.256e-07, 'completion_length': 156.99107360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005157470703125, 'epoch': 0.17} 17%|█▋ | 436/2500 [2:36:16<12:52:18, 22.45s/it] 17%|█▋ | 437/2500 [2:36:39<12:51:05, 22.43s/it] {'loss': 0.0003, 'grad_norm': 0.9191099786981198, 'learning_rate': 8.252000000000001e-07, 'completion_length': 152.1339340209961, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.17104807123541832, 'kl': 0.007110595703125, 'epoch': 0.17} 17%|█▋ | 437/2500 [2:36:39<12:51:05, 22.43s/it] 18%|█▊ | 438/2500 [2:37:01<12:46:49, 22.31s/it] {'loss': 0.0002, 'grad_norm': 0.2977376002048977, 'learning_rate': 8.247999999999999e-07, 'completion_length': 150.4107208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.00543212890625, 'epoch': 0.18} 18%|█▊ | 438/2500 [2:37:01<12:46:49, 22.31s/it] 18%|█▊ | 439/2500 [2:37:24<12:55:25, 22.57s/it] {'loss': 0.0003, 'grad_norm': 0.9276162135851366, 'learning_rate': 8.244e-07, 'completion_length': 151.3214340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0066680908203125, 'epoch': 0.18} 18%|█▊ | 439/2500 [2:37:24<12:55:25, 22.57s/it] 18%|█▊ | 440/2500 [2:37:46<12:50:50, 22.45s/it] {'loss': 0.0002, 'grad_norm': 1.605568494329358, 'learning_rate': 8.24e-07, 'completion_length': 148.98214721679688, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.13736959174275398, 'kl': 0.00579833984375, 'epoch': 0.18} 18%|█▊ | 440/2500 [2:37:46<12:50:50, 22.45s/it] 18%|█▊ | 441/2500 [2:38:09<12:52:41, 22.52s/it] {'loss': 0.0002, 'grad_norm': 0.8618771589559422, 'learning_rate': 8.235999999999999e-07, 'completion_length': 136.0357208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00445556640625, 'epoch': 0.18} 18%|█▊ | 441/2500 [2:38:09<12:52:41, 22.52s/it] 18%|█▊ | 442/2500 [2:38:31<12:52:50, 22.53s/it] {'loss': 0.0003, 'grad_norm': 0.7361872170209969, 'learning_rate': 8.232e-07, 'completion_length': 148.2589340209961, 'rewards/accuracy_reward': 0.7946428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7946429252624512, 'reward_std': 0.15872061252593994, 'kl': 0.00714111328125, 'epoch': 0.18} 18%|█▊ | 442/2500 [2:38:31<12:52:50, 22.53s/it] 18%|█▊ | 443/2500 [2:38:54<12:49:56, 22.46s/it] {'loss': 0.0003, 'grad_norm': 0.6970996283003275, 'learning_rate': 8.228e-07, 'completion_length': 138.4464340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0076141357421875, 'epoch': 0.18} 18%|█▊ | 443/2500 [2:38:54<12:49:56, 22.46s/it] 18%|█▊ | 444/2500 [2:39:17<12:58:24, 22.72s/it] {'loss': 0.0003, 'grad_norm': 0.6151129604908913, 'learning_rate': 8.224e-07, 'completion_length': 152.5357208251953, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.1322600245475769, 'kl': 0.006256103515625, 'epoch': 0.18} 18%|█▊ | 444/2500 [2:39:17<12:58:24, 22.72s/it] 18%|█▊ | 445/2500 [2:39:40<12:58:48, 22.74s/it] {'loss': 0.0003, 'grad_norm': 0.6732898419639102, 'learning_rate': 8.219999999999999e-07, 'completion_length': 157.8928680419922, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.14969705045223236, 'kl': 0.007568359375, 'epoch': 0.18} 18%|█▊ | 445/2500 [2:39:40<12:58:48, 22.74s/it] 18%|█▊ | 446/2500 [2:40:03<13:02:49, 22.87s/it] {'loss': 0.0002, 'grad_norm': 0.4448709637559031, 'learning_rate': 8.216e-07, 'completion_length': 141.46428680419922, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0052032470703125, 'epoch': 0.18} 18%|█▊ | 446/2500 [2:40:03<13:02:49, 22.87s/it] 18%|█▊ | 447/2500 [2:40:25<12:55:54, 22.68s/it] {'loss': 0.0003, 'grad_norm': 0.6313435950666504, 'learning_rate': 8.212e-07, 'completion_length': 141.4464340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.00750732421875, 'epoch': 0.18} 18%|█▊ | 447/2500 [2:40:25<12:55:54, 22.68s/it] 18%|█▊ | 448/2500 [2:40:48<12:54:23, 22.64s/it] {'loss': 0.0003, 'grad_norm': 0.36448762864439793, 'learning_rate': 8.207999999999999e-07, 'completion_length': 160.7589340209961, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0075531005859375, 'epoch': 0.18} 18%|█▊ | 448/2500 [2:40:48<12:54:23, 22.64s/it] 18%|█▊ | 449/2500 [2:41:11<12:57:14, 22.74s/it] {'loss': 0.0004, 'grad_norm': 0.646977912513286, 'learning_rate': 8.204e-07, 'completion_length': 153.10714721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.11272924765944481, 'kl': 0.0088348388671875, 'epoch': 0.18} 18%|█▊ | 449/2500 [2:41:11<12:57:14, 22.74s/it] 18%|█▊ | 450/2500 [2:41:33<12:56:49, 22.74s/it] {'loss': 0.0002, 'grad_norm': 0.020526915364792776, 'learning_rate': 8.199999999999999e-07, 'completion_length': 149.81250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0039825439453125, 'epoch': 0.18} 18%|█▊ | 450/2500 [2:41:33<12:56:49, 22.74s/it] 18%|█▊ | 451/2500 [2:41:56<12:52:52, 22.63s/it] {'loss': 0.0002, 'grad_norm': 0.5483249995902776, 'learning_rate': 8.196e-07, 'completion_length': 148.43750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831882357597351, 'kl': 0.00530242919921875, 'epoch': 0.18} 18%|█▊ | 451/2500 [2:41:56<12:52:52, 22.63s/it] 18%|█▊ | 452/2500 [2:42:18<12:49:11, 22.54s/it] {'loss': 0.0003, 'grad_norm': 0.3724520541878409, 'learning_rate': 8.192e-07, 'completion_length': 145.92857360839844, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831881985068321, 'kl': 0.0078887939453125, 'epoch': 0.18} 18%|█▊ | 452/2500 [2:42:18<12:49:11, 22.54s/it] 18%|█▊ | 453/2500 [2:42:42<12:57:06, 22.78s/it] {'loss': 0.0003, 'grad_norm': 0.28262094596542664, 'learning_rate': 8.187999999999999e-07, 'completion_length': 159.66964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064697265625, 'epoch': 0.18} 18%|█▊ | 453/2500 [2:42:42<12:57:06, 22.78s/it] 18%|█▊ | 454/2500 [2:43:04<12:57:38, 22.80s/it] {'loss': 0.0003, 'grad_norm': 0.5090782413187982, 'learning_rate': 8.184e-07, 'completion_length': 166.33036041259766, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.12626906484365463, 'kl': 0.0077056884765625, 'epoch': 0.18} 18%|█▊ | 454/2500 [2:43:04<12:57:38, 22.80s/it] 18%|█▊ | 455/2500 [2:43:27<13:00:17, 22.89s/it] {'loss': 0.0003, 'grad_norm': 0.6943246797880018, 'learning_rate': 8.179999999999999e-07, 'completion_length': 162.54464721679688, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0069427490234375, 'epoch': 0.18} 18%|█▊ | 455/2500 [2:43:28<13:00:17, 22.89s/it] 18%|█▊ | 456/2500 [2:43:50<12:54:01, 22.72s/it] {'loss': 0.0002, 'grad_norm': 0.3517574087423332, 'learning_rate': 8.175999999999999e-07, 'completion_length': 143.39286041259766, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0059051513671875, 'epoch': 0.18} 18%|█▊ | 456/2500 [2:43:50<12:54:01, 22.72s/it] 18%|█▊ | 457/2500 [2:44:12<12:52:26, 22.69s/it] {'loss': 0.0002, 'grad_norm': 0.03239444211521829, 'learning_rate': 8.172e-07, 'completion_length': 137.8839340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00567626953125, 'epoch': 0.18} 18%|█▊ | 457/2500 [2:44:12<12:52:26, 22.69s/it] 18%|█▊ | 458/2500 [2:44:36<12:58:42, 22.88s/it] {'loss': 0.0003, 'grad_norm': 0.3166099706395721, 'learning_rate': 8.168e-07, 'completion_length': 153.9375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0067291259765625, 'epoch': 0.18} 18%|█▊ | 458/2500 [2:44:36<12:58:42, 22.88s/it] 18%|█▊ | 459/2500 [2:45:00<13:14:22, 23.35s/it] {'loss': 0.0002, 'grad_norm': 0.12842186151510196, 'learning_rate': 8.163999999999999e-07, 'completion_length': 149.6964340209961, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.006195068359375, 'epoch': 0.18} 18%|█▊ | 459/2500 [2:45:00<13:14:22, 23.35s/it] 18%|█▊ | 460/2500 [2:45:23<13:06:51, 23.14s/it] {'loss': 0.0002, 'grad_norm': 0.36130610953657777, 'learning_rate': 8.159999999999999e-07, 'completion_length': 153.4107208251953, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.005828857421875, 'epoch': 0.18} 18%|█▊ | 460/2500 [2:45:23<13:06:51, 23.14s/it] 18%|█▊ | 461/2500 [2:45:45<12:58:18, 22.90s/it] {'loss': 0.0003, 'grad_norm': 0.49454736202340294, 'learning_rate': 8.156e-07, 'completion_length': 148.36607360839844, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0064239501953125, 'epoch': 0.18} 18%|█▊ | 461/2500 [2:45:45<12:58:18, 22.90s/it] 18%|█▊ | 462/2500 [2:46:08<12:57:17, 22.88s/it] {'loss': 0.0003, 'grad_norm': 0.9558597397138934, 'learning_rate': 8.152e-07, 'completion_length': 155.15178680419922, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.15872061252593994, 'kl': 0.0082244873046875, 'epoch': 0.18} 18%|█▊ | 462/2500 [2:46:08<12:57:17, 22.88s/it] 19%|█▊ | 463/2500 [2:46:31<13:01:09, 23.01s/it] {'loss': 0.0002, 'grad_norm': 0.05335501528353657, 'learning_rate': 8.147999999999999e-07, 'completion_length': 138.86607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00537109375, 'epoch': 0.19} 19%|█▊ | 463/2500 [2:46:31<13:01:09, 23.01s/it] 19%|█▊ | 464/2500 [2:46:55<13:05:40, 23.15s/it] {'loss': 0.0003, 'grad_norm': 0.39643351973921476, 'learning_rate': 8.144e-07, 'completion_length': 157.6339340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.00701904296875, 'epoch': 0.19} 19%|█▊ | 464/2500 [2:46:55<13:05:40, 23.15s/it] 19%|█▊ | 465/2500 [2:47:17<12:57:08, 22.91s/it] {'loss': 0.0003, 'grad_norm': 0.5598562364443079, 'learning_rate': 8.14e-07, 'completion_length': 143.04464721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0063018798828125, 'epoch': 0.19} 19%|█▊ | 465/2500 [2:47:17<12:57:08, 22.91s/it] 19%|█▊ | 466/2500 [2:47:39<12:50:13, 22.72s/it] {'loss': 0.0002, 'grad_norm': 0.23741959885403557, 'learning_rate': 8.135999999999999e-07, 'completion_length': 144.19644165039062, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0047607421875, 'epoch': 0.19} 19%|█▊ | 466/2500 [2:47:39<12:50:13, 22.72s/it] 19%|█▊ | 467/2500 [2:48:02<12:51:06, 22.76s/it] {'loss': 0.0003, 'grad_norm': 0.03941348518458606, 'learning_rate': 8.132e-07, 'completion_length': 145.52679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006500244140625, 'epoch': 0.19} 19%|█▊ | 467/2500 [2:48:02<12:51:06, 22.76s/it] 19%|█▊ | 468/2500 [2:48:25<12:47:07, 22.65s/it] {'loss': 0.0002, 'grad_norm': 0.6590193168522628, 'learning_rate': 8.128e-07, 'completion_length': 140.3214340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.08747542649507523, 'kl': 0.0044403076171875, 'epoch': 0.19} 19%|█▊ | 468/2500 [2:48:25<12:47:07, 22.65s/it] 19%|█▉ | 469/2500 [2:48:47<12:45:07, 22.60s/it] {'loss': 0.0003, 'grad_norm': 0.7748163684492769, 'learning_rate': 8.123999999999999e-07, 'completion_length': 147.2321548461914, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.14579425752162933, 'kl': 0.007904052734375, 'epoch': 0.19} 19%|█▉ | 469/2500 [2:48:47<12:45:07, 22.60s/it] 19%|█▉ | 470/2500 [2:49:11<12:58:49, 23.02s/it] {'loss': 0.0002, 'grad_norm': 0.6636420508649916, 'learning_rate': 8.12e-07, 'completion_length': 143.36608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0050048828125, 'epoch': 0.19} 19%|█▉ | 470/2500 [2:49:11<12:58:49, 23.02s/it] 19%|█▉ | 471/2500 [2:49:34<12:57:00, 22.98s/it] {'loss': 0.0003, 'grad_norm': 0.7645796372383267, 'learning_rate': 8.116e-07, 'completion_length': 154.75000762939453, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.12054044008255005, 'kl': 0.006866455078125, 'epoch': 0.19} 19%|█▉ | 471/2500 [2:49:34<12:57:00, 22.98s/it] 19%|█▉ | 472/2500 [2:50:19<16:39:05, 29.56s/it] {'loss': 0.0002, 'grad_norm': 0.645792369533007, 'learning_rate': 8.112e-07, 'completion_length': 146.7232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00537109375, 'epoch': 0.19} 19%|█▉ | 472/2500 [2:50:19<16:39:05, 29.56s/it] 19%|█▉ | 473/2500 [2:50:41<15:24:31, 27.37s/it] {'loss': 0.0002, 'grad_norm': 0.5251808331375951, 'learning_rate': 8.107999999999999e-07, 'completion_length': 142.625, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00507354736328125, 'epoch': 0.19} 19%|█▉ | 473/2500 [2:50:41<15:24:31, 27.37s/it] 19%|█▉ | 474/2500 [2:51:05<14:43:25, 26.16s/it] {'loss': 0.0002, 'grad_norm': 0.16138761030331114, 'learning_rate': 8.104e-07, 'completion_length': 160.86608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.19} 19%|█▉ | 474/2500 [2:51:05<14:43:25, 26.16s/it] 19%|█▉ | 475/2500 [2:51:27<14:08:52, 25.15s/it] {'loss': 0.0004, 'grad_norm': 0.821021173712693, 'learning_rate': 8.1e-07, 'completion_length': 158.86607360839844, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.16531942784786224, 'kl': 0.00982666015625, 'epoch': 0.19} 19%|█▉ | 475/2500 [2:51:27<14:08:52, 25.15s/it] 19%|█▉ | 476/2500 [2:51:50<13:40:26, 24.32s/it] {'loss': 0.0002, 'grad_norm': 0.3734531456434107, 'learning_rate': 8.095999999999999e-07, 'completion_length': 139.36607360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0046844482421875, 'epoch': 0.19} 19%|█▉ | 476/2500 [2:51:50<13:40:26, 24.32s/it] 19%|█▉ | 477/2500 [2:52:12<13:21:30, 23.77s/it] {'loss': 0.0002, 'grad_norm': 0.42191056049388936, 'learning_rate': 8.092e-07, 'completion_length': 146.1964340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0050506591796875, 'epoch': 0.19} 19%|█▉ | 477/2500 [2:52:12<13:21:30, 23.77s/it] 19%|█▉ | 478/2500 [2:52:34<13:04:52, 23.29s/it] {'loss': 0.0002, 'grad_norm': 0.7105578318735867, 'learning_rate': 8.087999999999999e-07, 'completion_length': 142.29464721679688, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989997446537, 'kl': 0.0052490234375, 'epoch': 0.19} 19%|█▉ | 478/2500 [2:52:34<13:04:52, 23.29s/it] 19%|█▉ | 479/2500 [2:52:57<12:58:09, 23.10s/it] {'loss': 0.0002, 'grad_norm': 0.6290142405173046, 'learning_rate': 8.084e-07, 'completion_length': 146.91964721679688, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.09528662264347076, 'kl': 0.0048065185546875, 'epoch': 0.19} 19%|█▉ | 479/2500 [2:52:57<12:58:09, 23.10s/it] 19%|█▉ | 480/2500 [2:53:21<13:01:09, 23.20s/it] {'loss': 0.0004, 'grad_norm': 0.6404632121785993, 'learning_rate': 8.08e-07, 'completion_length': 168.87500762939453, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.157508235424757, 'kl': 0.0106353759765625, 'epoch': 0.19} 19%|█▉ | 480/2500 [2:53:21<13:01:09, 23.20s/it] 19%|█▉ | 481/2500 [2:53:43<12:54:49, 23.03s/it] {'loss': 0.0004, 'grad_norm': 0.5706775163563731, 'learning_rate': 8.075999999999999e-07, 'completion_length': 155.99107360839844, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.12565559893846512, 'kl': 0.00897216796875, 'epoch': 0.19} 19%|█▉ | 481/2500 [2:53:43<12:54:49, 23.03s/it] 19%|█▉ | 482/2500 [2:54:07<13:00:30, 23.21s/it] {'loss': 0.0004, 'grad_norm': 0.9319380092657361, 'learning_rate': 8.072e-07, 'completion_length': 160.98214721679688, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.12444883584976196, 'kl': 0.01025390625, 'epoch': 0.19} 19%|█▉ | 482/2500 [2:54:07<13:00:30, 23.21s/it] 19%|█▉ | 483/2500 [2:54:29<12:46:13, 22.79s/it] {'loss': 0.0002, 'grad_norm': 0.7329510411622265, 'learning_rate': 8.067999999999999e-07, 'completion_length': 148.3214340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.0835726372897625, 'kl': 0.0053253173828125, 'epoch': 0.19} 19%|█▉ | 483/2500 [2:54:29<12:46:13, 22.79s/it] 19%|█▉ | 484/2500 [2:54:51<12:37:35, 22.55s/it] {'loss': 0.0003, 'grad_norm': 0.38341325286069144, 'learning_rate': 8.064e-07, 'completion_length': 142.17858123779297, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0739355981349945, 'kl': 0.0068817138671875, 'epoch': 0.19} 19%|█▉ | 484/2500 [2:54:51<12:37:35, 22.55s/it] 19%|█▉ | 485/2500 [2:55:13<12:34:49, 22.48s/it] {'loss': 0.0002, 'grad_norm': 0.8759757360705435, 'learning_rate': 8.06e-07, 'completion_length': 147.1607208251953, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07576143741607666, 'kl': 0.0059814453125, 'epoch': 0.19} 19%|█▉ | 485/2500 [2:55:13<12:34:49, 22.48s/it] 19%|█▉ | 486/2500 [2:55:35<12:32:08, 22.41s/it] {'loss': 0.0002, 'grad_norm': 0.431233735539414, 'learning_rate': 8.056e-07, 'completion_length': 147.30357360839844, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0060272216796875, 'epoch': 0.19} 19%|█▉ | 486/2500 [2:55:35<12:32:08, 22.41s/it] 19%|█▉ | 487/2500 [2:55:58<12:32:43, 22.44s/it] {'loss': 0.0003, 'grad_norm': 2.0986203797089265, 'learning_rate': 8.052e-07, 'completion_length': 150.64286041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.006591796875, 'epoch': 0.19} 19%|█▉ | 487/2500 [2:55:58<12:32:43, 22.44s/it] 20%|█▉ | 488/2500 [2:56:20<12:29:38, 22.36s/it] {'loss': 0.0003, 'grad_norm': 1.0512927936302603, 'learning_rate': 8.047999999999999e-07, 'completion_length': 156.7857208251953, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.17885924875736237, 'kl': 0.008087158203125, 'epoch': 0.2} 20%|█▉ | 488/2500 [2:56:20<12:29:38, 22.36s/it] 20%|█▉ | 489/2500 [2:56:42<12:32:13, 22.44s/it] {'loss': 0.0002, 'grad_norm': 0.458670116038439, 'learning_rate': 8.044e-07, 'completion_length': 152.95536041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0050506591796875, 'epoch': 0.2} 20%|█▉ | 489/2500 [2:56:42<12:32:13, 22.44s/it] 20%|█▉ | 490/2500 [2:57:04<12:27:37, 22.32s/it] {'loss': 0.0003, 'grad_norm': 0.6907071364340243, 'learning_rate': 8.04e-07, 'completion_length': 150.8482208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0739355981349945, 'kl': 0.0078582763671875, 'epoch': 0.2} 20%|█▉ | 490/2500 [2:57:04<12:27:37, 22.32s/it] 20%|█▉ | 491/2500 [2:57:27<12:25:00, 22.25s/it] {'loss': 0.0002, 'grad_norm': 0.40910941452081534, 'learning_rate': 8.035999999999999e-07, 'completion_length': 139.85714721679688, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.09528662264347076, 'kl': 0.005523681640625, 'epoch': 0.2} 20%|█▉ | 491/2500 [2:57:27<12:25:00, 22.25s/it] 20%|█▉ | 492/2500 [2:57:48<12:20:35, 22.13s/it] {'loss': 0.0003, 'grad_norm': 0.34633844660484325, 'learning_rate': 8.032e-07, 'completion_length': 138.51786041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00738525390625, 'epoch': 0.2} 20%|█▉ | 492/2500 [2:57:48<12:20:35, 22.13s/it] 20%|█▉ | 493/2500 [2:58:11<12:21:14, 22.16s/it] {'loss': 0.0002, 'grad_norm': 0.44187763859314594, 'learning_rate': 8.028e-07, 'completion_length': 148.86608123779297, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0048370361328125, 'epoch': 0.2} 20%|█▉ | 493/2500 [2:58:11<12:21:14, 22.16s/it] 20%|█▉ | 494/2500 [2:58:34<12:34:19, 22.56s/it] {'loss': 0.0004, 'grad_norm': 0.42816909773981404, 'learning_rate': 8.023999999999999e-07, 'completion_length': 160.83929443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.009552001953125, 'epoch': 0.2} 20%|█▉ | 494/2500 [2:58:34<12:34:19, 22.56s/it] 20%|█▉ | 495/2500 [2:59:04<13:49:22, 24.82s/it] {'loss': 0.0002, 'grad_norm': 0.640398832134076, 'learning_rate': 8.02e-07, 'completion_length': 156.77679443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.096499003469944, 'kl': 0.0058746337890625, 'epoch': 0.2} 20%|█▉ | 495/2500 [2:59:04<13:49:22, 24.82s/it] 20%|█▉ | 496/2500 [2:59:26<13:21:48, 24.01s/it] {'loss': 0.0003, 'grad_norm': 0.47373179053356984, 'learning_rate': 8.016e-07, 'completion_length': 149.01786041259766, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747543022036552, 'kl': 0.006683349609375, 'epoch': 0.2} 20%|█▉ | 496/2500 [2:59:26<13:21:48, 24.01s/it] 20%|█▉ | 497/2500 [2:59:49<13:07:22, 23.59s/it] {'loss': 0.0002, 'grad_norm': 1.2252257095220571, 'learning_rate': 8.012e-07, 'completion_length': 155.6607208251953, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941184878349, 'kl': 0.005584716796875, 'epoch': 0.2} 20%|█▉ | 497/2500 [2:59:49<13:07:22, 23.59s/it] 20%|█▉ | 498/2500 [3:00:11<12:54:57, 23.23s/it] {'loss': 0.0003, 'grad_norm': 0.28371944544613065, 'learning_rate': 8.007999999999999e-07, 'completion_length': 152.9107208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.033065006136894226, 'kl': 0.0084228515625, 'epoch': 0.2} 20%|█▉ | 498/2500 [3:00:11<12:54:57, 23.23s/it] 20%|█▉ | 499/2500 [3:00:34<12:50:27, 23.10s/it] {'loss': 0.0003, 'grad_norm': 1.7789731036433223, 'learning_rate': 8.004e-07, 'completion_length': 167.55358123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.07003280520439148, 'kl': 0.006683349609375, 'epoch': 0.2} 20%|█▉ | 499/2500 [3:00:34<12:50:27, 23.10s/it] 20%|██ | 500/2500 [3:00:57<12:46:42, 23.00s/it] {'loss': 0.0004, 'grad_norm': 1.2995140053816758, 'learning_rate': 8e-07, 'completion_length': 154.9732208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.16653180867433548, 'kl': 0.008819580078125, 'epoch': 0.2} 20%|██ | 500/2500 [3:00:57<12:46:42, 23.00s/it] 20%|██ | 501/2500 [3:02:37<25:39:09, 46.20s/it] {'loss': 0.0002, 'grad_norm': 0.2642246423615901, 'learning_rate': 7.995999999999999e-07, 'completion_length': 152.0089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059967041015625, 'epoch': 0.2} 20%|██ | 501/2500 [3:02:37<25:39:09, 46.20s/it] 20%|██ | 502/2500 [3:03:00<21:42:56, 39.13s/it] {'loss': 0.0002, 'grad_norm': 0.662210124320285, 'learning_rate': 7.992e-07, 'completion_length': 154.29464721679688, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.13225442171096802, 'kl': 0.0055084228515625, 'epoch': 0.2} 20%|██ | 502/2500 [3:03:00<21:42:56, 39.13s/it] 20%|██ | 503/2500 [3:03:22<18:56:41, 34.15s/it] {'loss': 0.0003, 'grad_norm': 1.0916083588503094, 'learning_rate': 7.987999999999999e-07, 'completion_length': 167.56250762939453, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.07003280520439148, 'kl': 0.0067901611328125, 'epoch': 0.2} 20%|██ | 503/2500 [3:03:22<18:56:41, 34.15s/it] 20%|██ | 504/2500 [3:03:45<16:58:11, 30.61s/it] {'loss': 0.0002, 'grad_norm': 0.2534532396004944, 'learning_rate': 7.984e-07, 'completion_length': 148.41964721679688, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.005218505859375, 'epoch': 0.2} 20%|██ | 504/2500 [3:03:45<16:58:11, 30.61s/it] 20%|██ | 505/2500 [3:04:07<15:39:29, 28.26s/it] {'loss': 0.0003, 'grad_norm': 0.032595315064220774, 'learning_rate': 7.98e-07, 'completion_length': 163.58929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006256103515625, 'epoch': 0.2} 20%|██ | 505/2500 [3:04:08<15:39:29, 28.26s/it] 20%|██ | 506/2500 [3:04:30<14:44:01, 26.60s/it] {'loss': 0.0002, 'grad_norm': 0.032921916251729647, 'learning_rate': 7.975999999999999e-07, 'completion_length': 147.68750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00567626953125, 'epoch': 0.2} 20%|██ | 506/2500 [3:04:30<14:44:01, 26.60s/it] 20%|██ | 507/2500 [3:04:53<14:01:42, 25.34s/it] {'loss': 0.0003, 'grad_norm': 0.7886032639620558, 'learning_rate': 7.972e-07, 'completion_length': 153.8214340209961, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726372897625, 'kl': 0.00738525390625, 'epoch': 0.2} 20%|██ | 507/2500 [3:04:53<14:01:42, 25.34s/it] 20%|██ | 508/2500 [3:05:15<13:31:31, 24.44s/it] {'loss': 0.0002, 'grad_norm': 0.2912844504425418, 'learning_rate': 7.967999999999999e-07, 'completion_length': 147.62500762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0055084228515625, 'epoch': 0.2} 20%|██ | 508/2500 [3:05:15<13:31:31, 24.44s/it] 20%|██ | 509/2500 [3:05:37<13:08:23, 23.76s/it] {'loss': 0.0002, 'grad_norm': 0.3212821351436517, 'learning_rate': 7.964e-07, 'completion_length': 135.4553680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005828857421875, 'epoch': 0.2} 20%|██ | 509/2500 [3:05:37<13:08:23, 23.76s/it] 20%|██ | 510/2500 [3:06:00<13:03:52, 23.63s/it] {'loss': 0.0003, 'grad_norm': 0.5352701697213514, 'learning_rate': 7.96e-07, 'completion_length': 152.2589340209961, 'rewards/accuracy_reward': 0.8303571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8303571939468384, 'reward_std': 0.07576143741607666, 'kl': 0.0064544677734375, 'epoch': 0.2} 20%|██ | 510/2500 [3:06:01<13:03:52, 23.63s/it] 20%|██ | 511/2500 [3:06:23<12:53:49, 23.34s/it] {'loss': 0.0003, 'grad_norm': 0.68760091779397, 'learning_rate': 7.956e-07, 'completion_length': 158.40178680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.112115778028965, 'kl': 0.0066070556640625, 'epoch': 0.2} 20%|██ | 511/2500 [3:06:23<12:53:49, 23.34s/it] 20%|██ | 512/2500 [3:06:46<12:51:58, 23.30s/it] {'loss': 0.0003, 'grad_norm': 0.562569456956819, 'learning_rate': 7.952e-07, 'completion_length': 163.9553680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12175281345844269, 'kl': 0.00750732421875, 'epoch': 0.2} 20%|██ | 512/2500 [3:06:46<12:51:58, 23.30s/it] 21%|██ | 513/2500 [3:07:09<12:46:55, 23.16s/it] {'loss': 0.0002, 'grad_norm': 0.44494700395595904, 'learning_rate': 7.947999999999999e-07, 'completion_length': 156.0178680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.1283516250550747, 'kl': 0.0061798095703125, 'epoch': 0.21} 21%|██ | 513/2500 [3:07:09<12:46:55, 23.16s/it] 21%|██ | 514/2500 [3:07:32<12:38:27, 22.91s/it] {'loss': 0.0003, 'grad_norm': 0.5712125621638898, 'learning_rate': 7.944e-07, 'completion_length': 155.4107208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0064544677734375, 'epoch': 0.21} 21%|██ | 514/2500 [3:07:32<12:38:27, 22.91s/it] 21%|██ | 515/2500 [3:07:54<12:34:53, 22.82s/it] {'loss': 0.0002, 'grad_norm': 0.1770233869141958, 'learning_rate': 7.94e-07, 'completion_length': 147.23214721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00531005859375, 'epoch': 0.21} 21%|██ | 515/2500 [3:07:54<12:34:53, 22.82s/it] 21%|██ | 516/2500 [3:08:17<12:38:03, 22.93s/it] {'loss': 0.0003, 'grad_norm': 0.37560691475835495, 'learning_rate': 7.935999999999999e-07, 'completion_length': 169.36607360839844, 'rewards/accuracy_reward': 0.8303571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8303571939468384, 'reward_std': 0.0964989997446537, 'kl': 0.0082855224609375, 'epoch': 0.21} 21%|██ | 516/2500 [3:08:17<12:38:03, 22.93s/it] 21%|██ | 517/2500 [3:08:40<12:38:04, 22.94s/it] {'loss': 0.0002, 'grad_norm': 0.23901684976933013, 'learning_rate': 7.932e-07, 'completion_length': 147.24107360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006103515625, 'epoch': 0.21} 21%|██ | 517/2500 [3:08:40<12:38:04, 22.94s/it] 21%|██ | 518/2500 [3:09:04<12:40:47, 23.03s/it] {'loss': 0.0002, 'grad_norm': 0.27847475769842667, 'learning_rate': 7.928e-07, 'completion_length': 146.3303680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.005828857421875, 'epoch': 0.21} 21%|██ | 518/2500 [3:09:04<12:40:47, 23.03s/it] 21%|██ | 519/2500 [3:09:26<12:38:51, 22.98s/it] {'loss': 0.0003, 'grad_norm': 0.3525532708448846, 'learning_rate': 7.923999999999999e-07, 'completion_length': 164.21429443359375, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0063629150390625, 'epoch': 0.21} 21%|██ | 519/2500 [3:09:26<12:38:51, 22.98s/it] 21%|██ | 520/2500 [3:09:51<12:51:51, 23.39s/it] {'loss': 0.0002, 'grad_norm': 0.6974731670962162, 'learning_rate': 7.92e-07, 'completion_length': 140.46429443359375, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.13225441798567772, 'kl': 0.006011962890625, 'epoch': 0.21} 21%|██ | 520/2500 [3:09:51<12:51:51, 23.39s/it] 21%|██ | 521/2500 [3:10:12<12:34:02, 22.86s/it] {'loss': 0.0004, 'grad_norm': 0.4315136532935768, 'learning_rate': 7.916e-07, 'completion_length': 170.62500762939453, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.11663763970136642, 'kl': 0.00885009765625, 'epoch': 0.21} 21%|██ | 521/2500 [3:10:12<12:34:02, 22.86s/it] 21%|██ | 522/2500 [3:10:34<12:18:13, 22.39s/it] {'loss': 0.0002, 'grad_norm': 0.026067554818845113, 'learning_rate': 7.911999999999999e-07, 'completion_length': 157.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005645751953125, 'epoch': 0.21} 21%|██ | 522/2500 [3:10:34<12:18:13, 22.39s/it] 21%|██ | 523/2500 [3:10:55<12:08:52, 22.12s/it] {'loss': 0.0002, 'grad_norm': 0.5080862284005568, 'learning_rate': 7.907999999999999e-07, 'completion_length': 161.71429443359375, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12565560638904572, 'kl': 0.006103515625, 'epoch': 0.21} 21%|██ | 523/2500 [3:10:55<12:08:52, 22.12s/it] 21%|██ | 524/2500 [3:11:17<12:05:44, 22.04s/it] {'loss': 0.0003, 'grad_norm': 0.8679676439002801, 'learning_rate': 7.904e-07, 'completion_length': 153.8482208251953, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382598757744, 'kl': 0.0072479248046875, 'epoch': 0.21} 21%|██ | 524/2500 [3:11:17<12:05:44, 22.04s/it] 21%|██ | 525/2500 [3:11:38<11:58:56, 21.84s/it] {'loss': 0.0002, 'grad_norm': 0.2887333842359188, 'learning_rate': 7.9e-07, 'completion_length': 162.02679443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.005767822265625, 'epoch': 0.21} 21%|██ | 525/2500 [3:11:38<11:58:56, 21.84s/it] 21%|██ | 526/2500 [3:12:01<12:02:22, 21.96s/it] {'loss': 0.0003, 'grad_norm': 0.30517234219128536, 'learning_rate': 7.895999999999999e-07, 'completion_length': 165.6607208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0077667236328125, 'epoch': 0.21} 21%|██ | 526/2500 [3:12:01<12:02:22, 21.96s/it] 21%|██ | 527/2500 [3:12:22<11:54:19, 21.72s/it] {'loss': 0.0002, 'grad_norm': 0.6636222994490275, 'learning_rate': 7.892e-07, 'completion_length': 148.25894165039062, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.08868780732154846, 'kl': 0.0059356689453125, 'epoch': 0.21} 21%|██ | 527/2500 [3:12:22<11:54:19, 21.72s/it] 21%|██ | 528/2500 [3:12:42<11:43:56, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.3361160539372713, 'learning_rate': 7.887999999999999e-07, 'completion_length': 150.56250762939453, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.0964989960193634, 'kl': 0.005645751953125, 'epoch': 0.21} 21%|██ | 528/2500 [3:12:42<11:43:56, 21.42s/it] 21%|██ | 529/2500 [3:13:04<11:45:56, 21.49s/it] {'loss': 0.0003, 'grad_norm': 0.6361929264362621, 'learning_rate': 7.883999999999999e-07, 'completion_length': 163.21429443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.07003280520439148, 'kl': 0.006805419921875, 'epoch': 0.21} 21%|██ | 529/2500 [3:13:04<11:45:56, 21.49s/it] 21%|██ | 530/2500 [3:13:26<11:46:22, 21.51s/it] {'loss': 0.0002, 'grad_norm': 0.8970068962737706, 'learning_rate': 7.88e-07, 'completion_length': 144.36608123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.09528662264347076, 'kl': 0.0058135986328125, 'epoch': 0.21} 21%|██ | 530/2500 [3:13:26<11:46:22, 21.51s/it] 21%|██ | 531/2500 [3:13:46<11:33:20, 21.13s/it] {'loss': 0.0001, 'grad_norm': 0.018802991737083725, 'learning_rate': 7.875999999999999e-07, 'completion_length': 134.51786422729492, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0037384033203125, 'epoch': 0.21} 21%|██ | 531/2500 [3:13:46<11:33:20, 21.13s/it] 21%|██▏ | 532/2500 [3:14:08<11:37:25, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.39227839978587076, 'learning_rate': 7.872e-07, 'completion_length': 154.73214721679688, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.09528662264347076, 'kl': 0.00531005859375, 'epoch': 0.21} 21%|██▏ | 532/2500 [3:14:08<11:37:25, 21.26s/it] 21%|██▏ | 533/2500 [3:14:29<11:37:01, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.4753965832074485, 'learning_rate': 7.868e-07, 'completion_length': 154.4107208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.00507354736328125, 'epoch': 0.21} 21%|██▏ | 533/2500 [3:14:29<11:37:01, 21.26s/it] 21%|██▏ | 534/2500 [3:14:49<11:26:53, 20.96s/it] {'loss': 0.0003, 'grad_norm': 0.8905157380134249, 'learning_rate': 7.864e-07, 'completion_length': 135.30357360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0062713623046875, 'epoch': 0.21} 21%|██▏ | 534/2500 [3:14:49<11:26:53, 20.96s/it] 21%|██▏ | 535/2500 [3:15:10<11:22:47, 20.85s/it] {'loss': 0.0003, 'grad_norm': 0.5168832227075021, 'learning_rate': 7.86e-07, 'completion_length': 152.12500762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.008392333984375, 'epoch': 0.21} 21%|██▏ | 535/2500 [3:15:10<11:22:47, 20.85s/it] 21%|██▏ | 536/2500 [3:15:31<11:26:08, 20.96s/it] {'loss': 0.0003, 'grad_norm': 0.9585888280905359, 'learning_rate': 7.855999999999999e-07, 'completion_length': 160.23214721679688, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382971286774, 'kl': 0.006744384765625, 'epoch': 0.21} 21%|██▏ | 536/2500 [3:15:31<11:26:08, 20.96s/it] 21%|██▏ | 537/2500 [3:15:52<11:27:28, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.46458528309650854, 'learning_rate': 7.852e-07, 'completion_length': 157.2232208251953, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10882645472884178, 'kl': 0.0061492919921875, 'epoch': 0.21} 21%|██▏ | 537/2500 [3:15:52<11:27:28, 21.01s/it] 22%|██▏ | 538/2500 [3:16:12<11:22:12, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.6006210791861504, 'learning_rate': 7.848e-07, 'completion_length': 147.8482208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.005706787109375, 'epoch': 0.22} 22%|██▏ | 538/2500 [3:16:12<11:22:12, 20.86s/it] 22%|██▏ | 539/2500 [3:16:33<11:17:17, 20.72s/it] {'loss': 0.0002, 'grad_norm': 0.3240850325765354, 'learning_rate': 7.844e-07, 'completion_length': 134.27679443359375, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.00457763671875, 'epoch': 0.22} 22%|██▏ | 539/2500 [3:16:33<11:17:17, 20.72s/it] 22%|██▏ | 540/2500 [3:16:53<11:15:22, 20.67s/it] {'loss': 0.0002, 'grad_norm': 0.5186580309613339, 'learning_rate': 7.84e-07, 'completion_length': 141.96428680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004913330078125, 'epoch': 0.22} 22%|██▏ | 540/2500 [3:16:53<11:15:22, 20.67s/it] 22%|██▏ | 541/2500 [3:17:16<11:29:15, 21.11s/it] {'loss': 0.0004, 'grad_norm': 0.761943647398269, 'learning_rate': 7.835999999999999e-07, 'completion_length': 153.4107208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.11663764342665672, 'kl': 0.008941650390625, 'epoch': 0.22} 22%|██▏ | 541/2500 [3:17:16<11:29:15, 21.11s/it] 22%|██▏ | 542/2500 [3:17:36<11:26:39, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.43496270107578394, 'learning_rate': 7.832e-07, 'completion_length': 143.0357208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.0049285888671875, 'epoch': 0.22} 22%|██▏ | 542/2500 [3:17:36<11:26:39, 21.04s/it] 22%|██▏ | 543/2500 [3:17:58<11:29:19, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.6673868026478961, 'learning_rate': 7.828e-07, 'completion_length': 145.49108123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.07003280520439148, 'kl': 0.004913330078125, 'epoch': 0.22} 22%|██▏ | 543/2500 [3:17:58<11:29:19, 21.13s/it] 22%|██▏ | 544/2500 [3:18:18<11:23:48, 20.98s/it] {'loss': 0.0003, 'grad_norm': 0.5048745973366343, 'learning_rate': 7.823999999999999e-07, 'completion_length': 142.98214721679688, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.007171630859375, 'epoch': 0.22} 22%|██▏ | 544/2500 [3:18:18<11:23:48, 20.98s/it] 22%|██▏ | 545/2500 [3:18:39<11:16:40, 20.77s/it] {'loss': 0.0003, 'grad_norm': 0.2812670683860003, 'learning_rate': 7.82e-07, 'completion_length': 136.90179443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.007568359375, 'epoch': 0.22} 22%|██▏ | 545/2500 [3:18:39<11:16:40, 20.77s/it] 22%|██▏ | 546/2500 [3:19:00<11:23:55, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.3499780172571131, 'learning_rate': 7.816e-07, 'completion_length': 162.4464340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.00689697265625, 'epoch': 0.22} 22%|██▏ | 546/2500 [3:19:00<11:23:55, 21.00s/it] 22%|██▏ | 547/2500 [3:19:22<11:26:44, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.8657806321024136, 'learning_rate': 7.811999999999999e-07, 'completion_length': 154.31250762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.10882645100355148, 'kl': 0.0067596435546875, 'epoch': 0.22} 22%|██▏ | 547/2500 [3:19:22<11:26:44, 21.10s/it] 22%|██▏ | 548/2500 [3:19:42<11:23:39, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.349774907502392, 'learning_rate': 7.808e-07, 'completion_length': 164.0357208251953, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.11272924393415451, 'kl': 0.0070343017578125, 'epoch': 0.22} 22%|██▏ | 548/2500 [3:19:42<11:23:39, 21.01s/it] 22%|██▏ | 549/2500 [3:20:03<11:23:23, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.21812882332825112, 'learning_rate': 7.804e-07, 'completion_length': 152.77679443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.033065006136894226, 'kl': 0.0074615478515625, 'epoch': 0.22} 22%|██▏ | 549/2500 [3:20:03<11:23:23, 21.02s/it] 22%|██▏ | 550/2500 [3:20:25<11:23:55, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.17344071393319985, 'learning_rate': 7.799999999999999e-07, 'completion_length': 149.41964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0055084228515625, 'epoch': 0.22} 22%|██▏ | 550/2500 [3:20:25<11:23:55, 21.04s/it] 22%|██▏ | 551/2500 [3:20:45<11:21:12, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.4369449888626939, 'learning_rate': 7.795999999999999e-07, 'completion_length': 159.8214340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.08747542649507523, 'kl': 0.0060882568359375, 'epoch': 0.22} 22%|██▏ | 551/2500 [3:20:45<11:21:12, 20.97s/it] 22%|██▏ | 552/2500 [3:21:06<11:22:45, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.15974018148329336, 'learning_rate': 7.792e-07, 'completion_length': 155.15179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00616455078125, 'epoch': 0.22} 22%|██▏ | 552/2500 [3:21:06<11:22:45, 21.03s/it] 22%|██▏ | 553/2500 [3:21:28<11:27:50, 21.20s/it] {'loss': 0.0002, 'grad_norm': 1.2520444483585933, 'learning_rate': 7.788000000000001e-07, 'completion_length': 157.10714721679688, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.12565560638904572, 'kl': 0.0062408447265625, 'epoch': 0.22} 22%|██▏ | 553/2500 [3:21:28<11:27:50, 21.20s/it] 22%|██▏ | 554/2500 [3:21:49<11:24:10, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.5081923292312228, 'learning_rate': 7.783999999999999e-07, 'completion_length': 163.95536041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.005706787109375, 'epoch': 0.22} 22%|██▏ | 554/2500 [3:21:49<11:24:10, 21.09s/it] 22%|██▏ | 555/2500 [3:22:11<11:36:01, 21.47s/it] {'loss': 0.0003, 'grad_norm': 0.556906675101479, 'learning_rate': 7.78e-07, 'completion_length': 168.10714721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11272924393415451, 'kl': 0.006622314453125, 'epoch': 0.22} 22%|██▏ | 555/2500 [3:22:11<11:36:01, 21.47s/it] 22%|██▏ | 556/2500 [3:22:33<11:35:44, 21.47s/it] {'loss': 0.0003, 'grad_norm': 0.6578868951391322, 'learning_rate': 7.776e-07, 'completion_length': 170.4107208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.14579424262046814, 'kl': 0.0069732666015625, 'epoch': 0.22} 22%|██▏ | 556/2500 [3:22:33<11:35:44, 21.47s/it] 22%|██▏ | 557/2500 [3:22:54<11:31:34, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.6272778658267543, 'learning_rate': 7.771999999999999e-07, 'completion_length': 161.9732208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726335644722, 'kl': 0.0065155029296875, 'epoch': 0.22} 22%|██▏ | 557/2500 [3:22:54<11:31:34, 21.36s/it] 22%|██▏ | 558/2500 [3:23:16<11:37:11, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.7653721981499187, 'learning_rate': 7.768e-07, 'completion_length': 157.6607208251953, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06613001227378845, 'kl': 0.0062408447265625, 'epoch': 0.22} 22%|██▏ | 558/2500 [3:23:16<11:37:11, 21.54s/it] 22%|██▏ | 559/2500 [3:23:37<11:30:49, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.28081731731335013, 'learning_rate': 7.764e-07, 'completion_length': 146.20536041259766, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0058746337890625, 'epoch': 0.22} 22%|██▏ | 559/2500 [3:23:37<11:30:49, 21.35s/it] 22%|██▏ | 560/2500 [3:23:57<11:23:22, 21.14s/it] {'loss': 0.0003, 'grad_norm': 0.3179843544154159, 'learning_rate': 7.76e-07, 'completion_length': 144.00000762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.006500244140625, 'epoch': 0.22} 22%|██▏ | 560/2500 [3:23:57<11:23:22, 21.14s/it] 22%|██▏ | 561/2500 [3:24:18<11:22:19, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.16327287225639256, 'learning_rate': 7.755999999999999e-07, 'completion_length': 142.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00467681884765625, 'epoch': 0.22} 22%|██▏ | 561/2500 [3:24:18<11:22:19, 21.11s/it] 22%|██▏ | 562/2500 [3:24:40<11:31:05, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.45426550516562403, 'learning_rate': 7.752e-07, 'completion_length': 153.61607360839844, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.10821297764778137, 'kl': 0.0085296630859375, 'epoch': 0.22} 22%|██▏ | 562/2500 [3:24:40<11:31:05, 21.40s/it] 23%|██▎ | 563/2500 [3:25:01<11:26:52, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.19264190777133355, 'learning_rate': 7.748e-07, 'completion_length': 144.08929443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00511932373046875, 'epoch': 0.23} 23%|██▎ | 563/2500 [3:25:01<11:26:52, 21.28s/it] 23%|██▎ | 564/2500 [3:25:22<11:22:58, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.5093663414439243, 'learning_rate': 7.743999999999999e-07, 'completion_length': 152.9464340209961, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.005523681640625, 'epoch': 0.23} 23%|██▎ | 564/2500 [3:25:22<11:22:58, 21.17s/it] 23%|██▎ | 565/2500 [3:25:43<11:21:42, 21.14s/it] {'loss': 0.0003, 'grad_norm': 0.5921934104989481, 'learning_rate': 7.74e-07, 'completion_length': 153.11607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007415771484375, 'epoch': 0.23} 23%|██▎ | 565/2500 [3:25:43<11:21:42, 21.14s/it] 23%|██▎ | 566/2500 [3:26:04<11:17:36, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.40853108672488647, 'learning_rate': 7.735999999999999e-07, 'completion_length': 154.00893783569336, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.00553131103515625, 'epoch': 0.23} 23%|██▎ | 566/2500 [3:26:04<11:17:36, 21.02s/it] 23%|██▎ | 567/2500 [3:26:26<11:20:20, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.44896834024298016, 'learning_rate': 7.732e-07, 'completion_length': 155.1071548461914, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.1030978113412857, 'kl': 0.005615234375, 'epoch': 0.23} 23%|██▎ | 567/2500 [3:26:26<11:20:20, 21.12s/it] 23%|██▎ | 568/2500 [3:26:47<11:27:41, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.6008224552228782, 'learning_rate': 7.728e-07, 'completion_length': 151.33036041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.006378173828125, 'epoch': 0.23} 23%|██▎ | 568/2500 [3:26:47<11:27:41, 21.36s/it] 23%|██▎ | 569/2500 [3:27:09<11:25:41, 21.31s/it] {'loss': 0.0003, 'grad_norm': 0.4235965231057802, 'learning_rate': 7.723999999999999e-07, 'completion_length': 159.58929443359375, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10040178522467613, 'kl': 0.00628662109375, 'epoch': 0.23} 23%|██▎ | 569/2500 [3:27:09<11:25:41, 21.31s/it] 23%|██▎ | 570/2500 [3:27:29<11:20:37, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.3424412000722935, 'learning_rate': 7.72e-07, 'completion_length': 152.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006805419921875, 'epoch': 0.23} 23%|██▎ | 570/2500 [3:27:29<11:20:37, 21.16s/it] 23%|██▎ | 571/2500 [3:27:52<11:31:12, 21.50s/it] {'loss': 0.0003, 'grad_norm': 0.5177193861449688, 'learning_rate': 7.716e-07, 'completion_length': 151.70536041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.0074005126953125, 'epoch': 0.23} 23%|██▎ | 571/2500 [3:27:52<11:31:12, 21.50s/it] 23%|██▎ | 572/2500 [3:28:13<11:23:41, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.03251324622573064, 'learning_rate': 7.711999999999999e-07, 'completion_length': 150.08036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00543212890625, 'epoch': 0.23} 23%|██▎ | 572/2500 [3:28:13<11:23:41, 21.28s/it] 23%|██▎ | 573/2500 [3:28:33<11:18:02, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.7699990722370081, 'learning_rate': 7.708e-07, 'completion_length': 155.10714721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0063629150390625, 'epoch': 0.23} 23%|██▎ | 573/2500 [3:28:33<11:18:02, 21.11s/it] 23%|██▎ | 574/2500 [3:28:54<11:13:35, 20.98s/it] {'loss': 0.0002, 'grad_norm': 1.0635117269508585, 'learning_rate': 7.704e-07, 'completion_length': 154.0178680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006195068359375, 'epoch': 0.23} 23%|██▎ | 574/2500 [3:28:54<11:13:35, 20.98s/it] 23%|██▎ | 575/2500 [3:29:15<11:14:03, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.4722131003317726, 'learning_rate': 7.699999999999999e-07, 'completion_length': 156.3482208251953, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392858505249023, 'reward_std': 0.10040178522467613, 'kl': 0.0086669921875, 'epoch': 0.23} 23%|██▎ | 575/2500 [3:29:15<11:14:03, 21.01s/it] 23%|██▎ | 576/2500 [3:29:36<11:13:24, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.26799062189845424, 'learning_rate': 7.695999999999999e-07, 'completion_length': 147.37500762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0054779052734375, 'epoch': 0.23} 23%|██▎ | 576/2500 [3:29:36<11:13:24, 21.00s/it] 23%|██▎ | 577/2500 [3:29:58<11:20:20, 21.23s/it] {'loss': 0.0003, 'grad_norm': 0.8650999646086904, 'learning_rate': 7.692e-07, 'completion_length': 141.83929443359375, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.03696779906749725, 'kl': 0.0072479248046875, 'epoch': 0.23} 23%|██▎ | 577/2500 [3:29:58<11:20:20, 21.23s/it] 23%|██▎ | 578/2500 [3:30:19<11:17:44, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.6375071890811438, 'learning_rate': 7.688000000000001e-07, 'completion_length': 144.71429443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.1418914571404457, 'kl': 0.0077667236328125, 'epoch': 0.23} 23%|██▎ | 578/2500 [3:30:19<11:17:44, 21.16s/it] 23%|██▎ | 579/2500 [3:30:40<11:14:15, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.3401276307089353, 'learning_rate': 7.683999999999999e-07, 'completion_length': 150.1339340209961, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0061798095703125, 'epoch': 0.23} 23%|██▎ | 579/2500 [3:30:40<11:14:15, 21.06s/it] 23%|██▎ | 580/2500 [3:31:01<11:14:51, 21.09s/it] {'loss': 0.0004, 'grad_norm': 1.3709263686923865, 'learning_rate': 7.68e-07, 'completion_length': 152.15178680419922, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875001192092896, 'reward_std': 0.1418914645910263, 'kl': 0.009368896484375, 'epoch': 0.23} 23%|██▎ | 580/2500 [3:31:01<11:14:51, 21.09s/it] 23%|██▎ | 581/2500 [3:31:22<11:16:26, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.6118361942839965, 'learning_rate': 7.676e-07, 'completion_length': 155.87500762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.005950927734375, 'epoch': 0.23} 23%|██▎ | 581/2500 [3:31:22<11:16:26, 21.15s/it] 23%|██▎ | 582/2500 [3:31:43<11:12:37, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.8305792507051093, 'learning_rate': 7.671999999999999e-07, 'completion_length': 145.24108123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0060577392578125, 'epoch': 0.23} 23%|██▎ | 582/2500 [3:31:43<11:12:37, 21.04s/it] 23%|██▎ | 583/2500 [3:32:04<11:11:36, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.619348801998581, 'learning_rate': 7.668e-07, 'completion_length': 160.31250762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0059051513671875, 'epoch': 0.23} 23%|██▎ | 583/2500 [3:32:04<11:11:36, 21.02s/it] 23%|██▎ | 584/2500 [3:32:25<11:09:01, 20.95s/it] {'loss': 0.0003, 'grad_norm': 0.5819724070349647, 'learning_rate': 7.664e-07, 'completion_length': 144.2946548461914, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.006256103515625, 'epoch': 0.23} 23%|██▎ | 584/2500 [3:32:25<11:09:01, 20.95s/it] 23%|██▎ | 585/2500 [3:32:45<11:08:26, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.7645633982974096, 'learning_rate': 7.66e-07, 'completion_length': 150.10714721679688, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.005859375, 'epoch': 0.23} 23%|██▎ | 585/2500 [3:32:45<11:08:26, 20.94s/it] 23%|██▎ | 586/2500 [3:33:07<11:13:40, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.6610525151474902, 'learning_rate': 7.655999999999999e-07, 'completion_length': 161.2589340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11663764342665672, 'kl': 0.00848388671875, 'epoch': 0.23} 23%|██▎ | 586/2500 [3:33:07<11:13:40, 21.12s/it] 23%|██▎ | 587/2500 [3:33:28<11:10:24, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.7777675865395401, 'learning_rate': 7.652e-07, 'completion_length': 157.46429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.12444323301315308, 'kl': 0.008544921875, 'epoch': 0.23} 23%|██▎ | 587/2500 [3:33:28<11:10:24, 21.03s/it] 24%|██▎ | 588/2500 [3:33:52<11:36:30, 21.86s/it] {'loss': 0.0002, 'grad_norm': 0.2439774684437507, 'learning_rate': 7.648e-07, 'completion_length': 155.2946548461914, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0059356689453125, 'epoch': 0.24} 24%|██▎ | 588/2500 [3:33:52<11:36:30, 21.86s/it] 24%|██▎ | 589/2500 [3:34:12<11:25:43, 21.53s/it] {'loss': 0.0002, 'grad_norm': 0.36420945251115433, 'learning_rate': 7.643999999999999e-07, 'completion_length': 152.4196548461914, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0049285888671875, 'epoch': 0.24} 24%|██▎ | 589/2500 [3:34:12<11:25:43, 21.53s/it] 24%|██▎ | 590/2500 [3:34:33<11:20:01, 21.36s/it] {'loss': 0.0002, 'grad_norm': 0.252142697996282, 'learning_rate': 7.64e-07, 'completion_length': 142.6071548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0053558349609375, 'epoch': 0.24} 24%|██▎ | 590/2500 [3:34:33<11:20:01, 21.36s/it] 24%|██▎ | 591/2500 [3:34:56<11:27:54, 21.62s/it] {'loss': 0.0003, 'grad_norm': 0.5326703015525096, 'learning_rate': 7.635999999999999e-07, 'completion_length': 166.96428680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.13615721464157104, 'kl': 0.0075836181640625, 'epoch': 0.24} 24%|██▎ | 591/2500 [3:34:56<11:27:54, 21.62s/it] 24%|██▎ | 592/2500 [3:35:16<11:19:59, 21.38s/it] {'loss': 0.0002, 'grad_norm': 4.436767852423316, 'learning_rate': 7.632e-07, 'completion_length': 147.2589340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.10040179640054703, 'kl': 0.0055389404296875, 'epoch': 0.24} 24%|██▎ | 592/2500 [3:35:16<11:19:59, 21.38s/it] 24%|██▎ | 593/2500 [3:35:42<12:03:28, 22.76s/it] {'loss': 0.0004, 'grad_norm': 0.8502287216294407, 'learning_rate': 7.628e-07, 'completion_length': 146.49108123779297, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.14006561040878296, 'kl': 0.009246826171875, 'epoch': 0.24} 24%|██▎ | 593/2500 [3:35:42<12:03:28, 22.76s/it] 24%|██▍ | 594/2500 [3:36:05<12:01:17, 22.71s/it] {'loss': 0.0002, 'grad_norm': 0.30712870722515734, 'learning_rate': 7.623999999999999e-07, 'completion_length': 149.74107360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0059661865234375, 'epoch': 0.24} 24%|██▍ | 594/2500 [3:36:05<12:01:17, 22.71s/it] 24%|██▍ | 595/2500 [3:36:26<11:49:48, 22.36s/it] {'loss': 0.0002, 'grad_norm': 0.5173559825184436, 'learning_rate': 7.62e-07, 'completion_length': 155.0714340209961, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.1575082391500473, 'kl': 0.0058135986328125, 'epoch': 0.24} 24%|██▍ | 595/2500 [3:36:26<11:49:48, 22.36s/it] 24%|██▍ | 596/2500 [3:36:51<12:09:42, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.21430561803202208, 'learning_rate': 7.616e-07, 'completion_length': 134.74107360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00421905517578125, 'epoch': 0.24} 24%|██▍ | 596/2500 [3:36:51<12:09:42, 22.99s/it] 24%|██▍ | 597/2500 [3:37:12<11:52:05, 22.45s/it] {'loss': 0.0003, 'grad_norm': 0.21163574349428116, 'learning_rate': 7.611999999999999e-07, 'completion_length': 155.61607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00628662109375, 'epoch': 0.24} 24%|██▍ | 597/2500 [3:37:12<11:52:05, 22.45s/it] 24%|██▍ | 598/2500 [3:37:33<11:37:17, 22.00s/it] {'loss': 0.0002, 'grad_norm': 0.5624100337338533, 'learning_rate': 7.608e-07, 'completion_length': 144.96428680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.0061798095703125, 'epoch': 0.24} 24%|██▍ | 598/2500 [3:37:33<11:37:17, 22.00s/it] 24%|██▍ | 599/2500 [3:37:54<11:29:43, 21.77s/it] {'loss': 0.0003, 'grad_norm': 1.0216942732459358, 'learning_rate': 7.604e-07, 'completion_length': 161.50000762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.06222161278128624, 'kl': 0.007904052734375, 'epoch': 0.24} 24%|██▍ | 599/2500 [3:37:54<11:29:43, 21.77s/it] 24%|██▍ | 600/2500 [3:38:16<11:25:28, 21.65s/it] {'loss': 0.0002, 'grad_norm': 0.24828304789195704, 'learning_rate': 7.599999999999999e-07, 'completion_length': 150.7857208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0061798095703125, 'epoch': 0.24} 24%|██▍ | 600/2500 [3:38:16<11:25:28, 21.65s/it] 24%|██▍ | 601/2500 [3:39:19<17:57:44, 34.05s/it] {'loss': 0.0002, 'grad_norm': 0.2684547946998944, 'learning_rate': 7.596e-07, 'completion_length': 159.98214721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0060882568359375, 'epoch': 0.24} 24%|██▍ | 601/2500 [3:39:19<17:57:44, 34.05s/it] 24%|██▍ | 602/2500 [3:39:40<15:53:51, 30.15s/it] {'loss': 0.0003, 'grad_norm': 0.9440913289763797, 'learning_rate': 7.592e-07, 'completion_length': 146.9107208251953, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.0964989960193634, 'kl': 0.008575439453125, 'epoch': 0.24} 24%|██▍ | 602/2500 [3:39:40<15:53:51, 30.15s/it] 24%|██▍ | 603/2500 [3:40:02<14:35:42, 27.70s/it] {'loss': 0.0003, 'grad_norm': 0.580851754320102, 'learning_rate': 7.588e-07, 'completion_length': 141.5714340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0064697265625, 'epoch': 0.24} 24%|██▍ | 603/2500 [3:40:02<14:35:42, 27.70s/it] 24%|██▍ | 604/2500 [3:40:22<13:28:33, 25.59s/it] {'loss': 0.0002, 'grad_norm': 0.13131621747614583, 'learning_rate': 7.583999999999999e-07, 'completion_length': 144.54464721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.005035400390625, 'epoch': 0.24} 24%|██▍ | 604/2500 [3:40:22<13:28:33, 25.59s/it] 24%|██▍ | 605/2500 [3:40:43<12:44:16, 24.20s/it] {'loss': 0.0003, 'grad_norm': 0.3140875426730459, 'learning_rate': 7.58e-07, 'completion_length': 142.35715103149414, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0080108642578125, 'epoch': 0.24} 24%|██▍ | 605/2500 [3:40:43<12:44:16, 24.20s/it] 24%|██▍ | 606/2500 [3:41:06<12:26:16, 23.64s/it] {'loss': 0.0004, 'grad_norm': 1.3702086704215537, 'learning_rate': 7.576000000000001e-07, 'completion_length': 174.90178680419922, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.12956400960683823, 'kl': 0.0090179443359375, 'epoch': 0.24} 24%|██▍ | 606/2500 [3:41:06<12:26:16, 23.64s/it] 24%|██▍ | 607/2500 [3:41:26<11:50:49, 22.53s/it] {'loss': 0.0001, 'grad_norm': 0.020444952705802894, 'learning_rate': 7.571999999999999e-07, 'completion_length': 126.56250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0034332275390625, 'epoch': 0.24} 24%|██▍ | 607/2500 [3:41:26<11:50:49, 22.53s/it] 24%|██▍ | 608/2500 [3:41:47<11:40:57, 22.23s/it] {'loss': 0.0002, 'grad_norm': 0.688080178494801, 'learning_rate': 7.568e-07, 'completion_length': 166.1339340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.11272924020886421, 'kl': 0.00616455078125, 'epoch': 0.24} 24%|██▍ | 608/2500 [3:41:47<11:40:57, 22.23s/it] 24%|██▍ | 609/2500 [3:42:08<11:28:46, 21.85s/it] {'loss': 0.0003, 'grad_norm': 0.40731937301452964, 'learning_rate': 7.564e-07, 'completion_length': 157.2857208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0074462890625, 'epoch': 0.24} 24%|██▍ | 609/2500 [3:42:08<11:28:46, 21.85s/it] 24%|██▍ | 610/2500 [3:42:29<11:20:21, 21.60s/it] {'loss': 0.0002, 'grad_norm': 0.29954611847959506, 'learning_rate': 7.559999999999999e-07, 'completion_length': 151.16964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0052490234375, 'epoch': 0.24} 24%|██▍ | 610/2500 [3:42:29<11:20:21, 21.60s/it] 24%|██▍ | 611/2500 [3:42:50<11:13:36, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.7610327837342181, 'learning_rate': 7.556e-07, 'completion_length': 138.10714721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0054473876953125, 'epoch': 0.24} 24%|██▍ | 611/2500 [3:42:50<11:13:36, 21.40s/it] 24%|██▍ | 612/2500 [3:43:11<11:12:17, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.6185584361880319, 'learning_rate': 7.552e-07, 'completion_length': 156.58929443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.09138382598757744, 'kl': 0.00628662109375, 'epoch': 0.24} 24%|██▍ | 612/2500 [3:43:11<11:12:17, 21.37s/it] 25%|██▍ | 613/2500 [3:43:32<11:07:31, 21.22s/it] {'loss': 0.0003, 'grad_norm': 1.0044064594477693, 'learning_rate': 7.548e-07, 'completion_length': 156.3214340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.0739355981349945, 'kl': 0.0068359375, 'epoch': 0.25} 25%|██▍ | 613/2500 [3:43:32<11:07:31, 21.22s/it] 25%|██▍ | 614/2500 [3:43:53<11:03:49, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.6226567801403318, 'learning_rate': 7.543999999999999e-07, 'completion_length': 158.0982208251953, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0057373046875, 'epoch': 0.25} 25%|██▍ | 614/2500 [3:43:53<11:03:49, 21.12s/it] 25%|██▍ | 615/2500 [3:44:15<11:07:29, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.46886022165267255, 'learning_rate': 7.54e-07, 'completion_length': 157.29464721679688, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.07576144114136696, 'kl': 0.006591796875, 'epoch': 0.25} 25%|██▍ | 615/2500 [3:44:15<11:07:29, 21.25s/it] 25%|██▍ | 616/2500 [3:44:36<11:08:39, 21.29s/it] {'loss': 0.0003, 'grad_norm': 0.7871233906914371, 'learning_rate': 7.536e-07, 'completion_length': 165.07144165039062, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392857313156128, 'reward_std': 0.12686799466609955, 'kl': 0.00762939453125, 'epoch': 0.25} 25%|██▍ | 616/2500 [3:44:36<11:08:39, 21.29s/it] 25%|██▍ | 617/2500 [3:44:57<11:08:19, 21.30s/it] {'loss': 0.0003, 'grad_norm': 0.9873993199828356, 'learning_rate': 7.531999999999999e-07, 'completion_length': 152.9464340209961, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.11663764342665672, 'kl': 0.0075836181640625, 'epoch': 0.25} 25%|██▍ | 617/2500 [3:44:57<11:08:19, 21.30s/it] 25%|██▍ | 618/2500 [3:45:19<11:11:59, 21.42s/it] {'loss': 0.0004, 'grad_norm': 0.9026998191245831, 'learning_rate': 7.528e-07, 'completion_length': 175.45536041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.16141663491725922, 'kl': 0.0105438232421875, 'epoch': 0.25} 25%|██▍ | 618/2500 [3:45:19<11:11:59, 21.42s/it] 25%|██▍ | 619/2500 [3:45:40<11:02:46, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.6013093827634596, 'learning_rate': 7.523999999999999e-07, 'completion_length': 148.2589340209961, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07576144114136696, 'kl': 0.005340576171875, 'epoch': 0.25} 25%|██▍ | 619/2500 [3:45:40<11:02:46, 21.14s/it] 25%|██▍ | 620/2500 [3:46:00<10:59:10, 21.04s/it] {'loss': 0.0003, 'grad_norm': 0.8311390331773292, 'learning_rate': 7.52e-07, 'completion_length': 151.8482208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.09528662264347076, 'kl': 0.0078277587890625, 'epoch': 0.25} 25%|██▍ | 620/2500 [3:46:00<10:59:10, 21.04s/it] 25%|██▍ | 621/2500 [3:46:22<11:01:29, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.757488643074752, 'learning_rate': 7.516e-07, 'completion_length': 168.60714721679688, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941557407379, 'kl': 0.0083770751953125, 'epoch': 0.25} 25%|██▍ | 621/2500 [3:46:22<11:01:29, 21.12s/it] 25%|██▍ | 622/2500 [3:46:42<10:56:02, 20.96s/it] {'loss': 0.0003, 'grad_norm': 0.6607304080493495, 'learning_rate': 7.511999999999999e-07, 'completion_length': 140.7857208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.0739355981349945, 'kl': 0.00665283203125, 'epoch': 0.25} 25%|██▍ | 622/2500 [3:46:42<10:56:02, 20.96s/it] 25%|██▍ | 623/2500 [3:47:03<10:53:46, 20.90s/it] {'loss': 0.0003, 'grad_norm': 0.7828049814156097, 'learning_rate': 7.508e-07, 'completion_length': 146.74108123779297, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.007171630859375, 'epoch': 0.25} 25%|██▍ | 623/2500 [3:47:03<10:53:46, 20.90s/it] 25%|██▍ | 624/2500 [3:47:25<11:00:17, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.6024230158899282, 'learning_rate': 7.503999999999999e-07, 'completion_length': 166.3303680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.007415771484375, 'epoch': 0.25} 25%|██▍ | 624/2500 [3:47:25<11:00:17, 21.12s/it] 25%|██▌ | 625/2500 [3:47:46<11:00:53, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.3672000757551286, 'learning_rate': 7.5e-07, 'completion_length': 145.89286041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.0063934326171875, 'epoch': 0.25} 25%|██▌ | 625/2500 [3:47:46<11:00:53, 21.15s/it] 25%|██▌ | 626/2500 [3:48:07<10:58:21, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.4721598389018358, 'learning_rate': 7.496e-07, 'completion_length': 148.55358123779297, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831881985068321, 'kl': 0.0052947998046875, 'epoch': 0.25} 25%|██▌ | 626/2500 [3:48:07<10:58:21, 21.08s/it] 25%|██▌ | 627/2500 [3:48:28<11:00:54, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.29932066890782766, 'learning_rate': 7.492e-07, 'completion_length': 155.5089340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0068359375, 'epoch': 0.25} 25%|██▌ | 627/2500 [3:48:28<11:00:54, 21.17s/it] 25%|██▌ | 628/2500 [3:48:49<11:02:01, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.22942121005289304, 'learning_rate': 7.488e-07, 'completion_length': 141.0982208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052642822265625, 'epoch': 0.25} 25%|██▌ | 628/2500 [3:48:50<11:02:01, 21.22s/it] 25%|██▌ | 629/2500 [3:49:11<11:03:42, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.7815890815690265, 'learning_rate': 7.483999999999999e-07, 'completion_length': 163.10714721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09919501841068268, 'kl': 0.008148193359375, 'epoch': 0.25} 25%|██▌ | 629/2500 [3:49:11<11:03:42, 21.28s/it] 25%|██▌ | 630/2500 [3:49:32<11:02:12, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.4994573940530891, 'learning_rate': 7.48e-07, 'completion_length': 142.0178680419922, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.005157470703125, 'epoch': 0.25} 25%|██▌ | 630/2500 [3:49:32<11:02:12, 21.25s/it] 25%|██▌ | 631/2500 [3:49:53<10:57:00, 21.09s/it] {'loss': 0.0002, 'grad_norm': 2.1607141730196804, 'learning_rate': 7.476e-07, 'completion_length': 151.3482208251953, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.14006561040878296, 'kl': 0.0061798095703125, 'epoch': 0.25} 25%|██▌ | 631/2500 [3:49:53<10:57:00, 21.09s/it] 25%|██▌ | 632/2500 [3:50:14<10:57:22, 21.11s/it] {'loss': 0.0003, 'grad_norm': 1.4079586152192383, 'learning_rate': 7.471999999999999e-07, 'completion_length': 158.1339340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831881985068321, 'kl': 0.0078125, 'epoch': 0.25} 25%|██▌ | 632/2500 [3:50:14<10:57:22, 21.11s/it] 25%|██▌ | 633/2500 [3:50:35<10:53:05, 20.99s/it] {'loss': 0.0003, 'grad_norm': 0.31830803357455484, 'learning_rate': 7.468e-07, 'completion_length': 153.04464721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0063018798828125, 'epoch': 0.25} 25%|██▌ | 633/2500 [3:50:35<10:53:05, 20.99s/it] 25%|██▌ | 634/2500 [3:50:55<10:49:33, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.05288962617732538, 'learning_rate': 7.464e-07, 'completion_length': 143.99108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00470733642578125, 'epoch': 0.25} 25%|██▌ | 634/2500 [3:50:55<10:49:33, 20.89s/it] 25%|██▌ | 635/2500 [3:51:17<10:59:30, 21.22s/it] {'loss': 0.0004, 'grad_norm': 0.5513897258149844, 'learning_rate': 7.459999999999999e-07, 'completion_length': 175.1339340209961, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.12054043635725975, 'kl': 0.010162353515625, 'epoch': 0.25} 25%|██▌ | 635/2500 [3:51:17<10:59:30, 21.22s/it] 25%|██▌ | 636/2500 [3:51:39<11:00:17, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.4465030071325127, 'learning_rate': 7.456e-07, 'completion_length': 143.6964340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831882357597351, 'kl': 0.0066375732421875, 'epoch': 0.25} 25%|██▌ | 636/2500 [3:51:39<11:00:17, 21.25s/it] 25%|██▌ | 637/2500 [3:52:03<11:29:45, 22.21s/it] {'loss': 0.0004, 'grad_norm': 0.7431004353150233, 'learning_rate': 7.452e-07, 'completion_length': 157.30357360839844, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.13737519830465317, 'kl': 0.0096588134765625, 'epoch': 0.25} 25%|██▌ | 637/2500 [3:52:03<11:29:45, 22.21s/it] 26%|██▌ | 638/2500 [3:52:26<11:33:27, 22.35s/it] {'loss': 0.0002, 'grad_norm': 0.3568535536642138, 'learning_rate': 7.447999999999999e-07, 'completion_length': 150.73214721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0053253173828125, 'epoch': 0.26} 26%|██▌ | 638/2500 [3:52:26<11:33:27, 22.35s/it] 26%|██▌ | 639/2500 [3:52:47<11:26:24, 22.13s/it] {'loss': 0.0004, 'grad_norm': 0.6269397647565927, 'learning_rate': 7.443999999999999e-07, 'completion_length': 162.77678680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.009765625, 'epoch': 0.26} 26%|██▌ | 639/2500 [3:52:47<11:26:24, 22.13s/it] 26%|██▌ | 640/2500 [3:53:09<11:24:20, 22.08s/it] {'loss': 0.0003, 'grad_norm': 0.5422210971564666, 'learning_rate': 7.44e-07, 'completion_length': 171.81250762939453, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.0084228515625, 'epoch': 0.26} 26%|██▌ | 640/2500 [3:53:09<11:24:20, 22.08s/it] 26%|██▌ | 641/2500 [3:53:30<11:11:08, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.48639756758153846, 'learning_rate': 7.436e-07, 'completion_length': 135.99108123779297, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0057525634765625, 'epoch': 0.26} 26%|██▌ | 641/2500 [3:53:30<11:11:08, 21.66s/it] 26%|██▌ | 642/2500 [3:53:51<11:07:40, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.02687321777432343, 'learning_rate': 7.431999999999999e-07, 'completion_length': 150.0178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053253173828125, 'epoch': 0.26} 26%|██▌ | 642/2500 [3:53:51<11:07:40, 21.56s/it] 26%|██▌ | 643/2500 [3:54:14<11:19:10, 21.94s/it] {'loss': 0.0002, 'grad_norm': 9.692244712395967, 'learning_rate': 7.428e-07, 'completion_length': 148.33929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.06613001227378845, 'kl': 0.004547119140625, 'epoch': 0.26} 26%|██▌ | 643/2500 [3:54:14<11:19:10, 21.94s/it] 26%|██▌ | 644/2500 [3:54:36<11:14:13, 21.80s/it] {'loss': 0.0002, 'grad_norm': 0.017303002107285716, 'learning_rate': 7.423999999999999e-07, 'completion_length': 150.7589340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00567626953125, 'epoch': 0.26} 26%|██▌ | 644/2500 [3:54:36<11:14:13, 21.80s/it] 26%|██▌ | 645/2500 [3:54:57<11:09:32, 21.66s/it] {'loss': 0.0003, 'grad_norm': 2.3617294898302683, 'learning_rate': 7.42e-07, 'completion_length': 148.5625, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.09138382598757744, 'kl': 0.0067291259765625, 'epoch': 0.26} 26%|██▌ | 645/2500 [3:54:57<11:09:32, 21.66s/it] 26%|██▌ | 646/2500 [3:55:18<11:02:20, 21.43s/it] {'loss': 0.0003, 'grad_norm': 1.1278155220926258, 'learning_rate': 7.416e-07, 'completion_length': 141.76786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00628662109375, 'epoch': 0.26} 26%|██▌ | 646/2500 [3:55:18<11:02:20, 21.43s/it] 26%|██▌ | 647/2500 [3:55:39<10:58:38, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.7355120684781622, 'learning_rate': 7.411999999999999e-07, 'completion_length': 152.7053680419922, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11663764715194702, 'kl': 0.00677490234375, 'epoch': 0.26} 26%|██▌ | 647/2500 [3:55:39<10:58:38, 21.33s/it] 26%|██▌ | 648/2500 [3:56:01<11:03:58, 21.51s/it] {'loss': 0.0004, 'grad_norm': 0.3569559726098545, 'learning_rate': 7.408e-07, 'completion_length': 164.27679443359375, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10821298509836197, 'kl': 0.009918212890625, 'epoch': 0.26} 26%|██▌ | 648/2500 [3:56:01<11:03:58, 21.51s/it] 26%|██▌ | 649/2500 [3:56:22<11:01:30, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.24718635895838412, 'learning_rate': 7.403999999999999e-07, 'completion_length': 151.26786041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0071258544921875, 'epoch': 0.26} 26%|██▌ | 649/2500 [3:56:22<11:01:30, 21.44s/it] 26%|██▌ | 650/2500 [3:56:44<11:00:17, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.7660904404612171, 'learning_rate': 7.4e-07, 'completion_length': 157.02679443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.006072998046875, 'epoch': 0.26} 26%|██▌ | 650/2500 [3:56:44<11:00:17, 21.41s/it] 26%|██▌ | 651/2500 [3:57:05<10:56:26, 21.30s/it] {'loss': 0.0003, 'grad_norm': 0.6284020138534423, 'learning_rate': 7.396e-07, 'completion_length': 145.17858123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09138382598757744, 'kl': 0.006439208984375, 'epoch': 0.26} 26%|██▌ | 651/2500 [3:57:05<10:56:26, 21.30s/it] 26%|██▌ | 652/2500 [3:57:26<11:00:16, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.24752549300143967, 'learning_rate': 7.392e-07, 'completion_length': 151.91964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004730224609375, 'epoch': 0.26} 26%|██▌ | 652/2500 [3:57:26<11:00:16, 21.44s/it] 26%|██▌ | 653/2500 [3:57:47<10:54:42, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.6914369614371625, 'learning_rate': 7.388e-07, 'completion_length': 150.58929443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007904052734375, 'epoch': 0.26} 26%|██▌ | 653/2500 [3:57:47<10:54:42, 21.27s/it] 26%|██▌ | 654/2500 [3:58:08<10:51:15, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.4034669968549359, 'learning_rate': 7.383999999999999e-07, 'completion_length': 152.93750762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007080078125, 'epoch': 0.26} 26%|██▌ | 654/2500 [3:58:08<10:51:15, 21.17s/it] 26%|██▌ | 655/2500 [3:58:30<10:55:09, 21.31s/it] {'loss': 0.0003, 'grad_norm': 0.3248148892692472, 'learning_rate': 7.38e-07, 'completion_length': 161.4553680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.006591796875, 'epoch': 0.26} 26%|██▌ | 655/2500 [3:58:30<10:55:09, 21.31s/it] 26%|██▌ | 656/2500 [3:58:51<10:50:01, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.15157012447265839, 'learning_rate': 7.376e-07, 'completion_length': 152.55358123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004730224609375, 'epoch': 0.26} 26%|██▌ | 656/2500 [3:58:51<10:50:01, 21.15s/it] 26%|██▋ | 657/2500 [3:59:12<10:54:57, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.468110683056691, 'learning_rate': 7.371999999999999e-07, 'completion_length': 165.125, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.09528662264347076, 'kl': 0.007659912109375, 'epoch': 0.26} 26%|██▋ | 657/2500 [3:59:12<10:54:57, 21.32s/it] 26%|██▋ | 658/2500 [3:59:34<10:58:50, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.38718585051511123, 'learning_rate': 7.368e-07, 'completion_length': 170.99107360839844, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.09528661891818047, 'kl': 0.0072784423828125, 'epoch': 0.26} 26%|██▋ | 658/2500 [3:59:34<10:58:50, 21.46s/it] 26%|██▋ | 659/2500 [3:59:56<10:58:44, 21.47s/it] {'loss': 0.0003, 'grad_norm': 0.5510036911585938, 'learning_rate': 7.364000000000001e-07, 'completion_length': 161.69644165039062, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.10101525112986565, 'kl': 0.0086517333984375, 'epoch': 0.26} 26%|██▋ | 659/2500 [3:59:56<10:58:44, 21.47s/it] 26%|██▋ | 660/2500 [4:00:18<11:02:44, 21.61s/it] {'loss': 0.0003, 'grad_norm': 0.5627370972854694, 'learning_rate': 7.359999999999999e-07, 'completion_length': 169.05357360839844, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726335644722, 'kl': 0.0074920654296875, 'epoch': 0.26} 26%|██▋ | 660/2500 [4:00:18<11:02:44, 21.61s/it] 26%|██▋ | 661/2500 [4:00:39<10:57:38, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.07493450818253414, 'learning_rate': 7.356e-07, 'completion_length': 143.24107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052032470703125, 'epoch': 0.26} 26%|██▋ | 661/2500 [4:00:39<10:57:38, 21.46s/it] 26%|██▋ | 662/2500 [4:00:59<10:51:52, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.2963801818102972, 'learning_rate': 7.352e-07, 'completion_length': 143.92858123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0048828125, 'epoch': 0.26} 26%|██▋ | 662/2500 [4:00:59<10:51:52, 21.28s/it] 27%|██▋ | 663/2500 [4:01:21<10:51:44, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.2932928759737595, 'learning_rate': 7.347999999999999e-07, 'completion_length': 144.95536041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.004791259765625, 'epoch': 0.27} 27%|██▋ | 663/2500 [4:01:21<10:51:44, 21.29s/it] 27%|██▋ | 664/2500 [4:01:43<11:00:49, 21.60s/it] {'loss': 0.0002, 'grad_norm': 0.36612453448744625, 'learning_rate': 7.344e-07, 'completion_length': 161.85714721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.005859375, 'epoch': 0.27} 27%|██▋ | 664/2500 [4:01:43<11:00:49, 21.60s/it] 27%|██▋ | 665/2500 [4:02:04<10:56:52, 21.48s/it] {'loss': 0.0002, 'grad_norm': 0.30268747790848, 'learning_rate': 7.34e-07, 'completion_length': 147.81250762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0062255859375, 'epoch': 0.27} 27%|██▋ | 665/2500 [4:02:04<10:56:52, 21.48s/it] 27%|██▋ | 666/2500 [4:02:25<10:50:41, 21.29s/it] {'loss': 0.0003, 'grad_norm': 0.323724136556201, 'learning_rate': 7.336e-07, 'completion_length': 145.7053680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0068359375, 'epoch': 0.27} 27%|██▋ | 666/2500 [4:02:25<10:50:41, 21.29s/it] 27%|██▋ | 667/2500 [4:02:47<10:51:13, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.41161664916949253, 'learning_rate': 7.331999999999999e-07, 'completion_length': 145.81250762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0069732666015625, 'epoch': 0.27} 27%|██▋ | 667/2500 [4:02:47<10:51:13, 21.32s/it] 27%|██▋ | 668/2500 [4:03:08<10:54:43, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.7239620315421716, 'learning_rate': 7.328e-07, 'completion_length': 160.6339340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797896146774, 'kl': 0.0067138671875, 'epoch': 0.27} 27%|██▋ | 668/2500 [4:03:08<10:54:43, 21.44s/it] 27%|██▋ | 669/2500 [4:03:30<10:53:49, 21.43s/it] {'loss': 0.0003, 'grad_norm': 0.7157283637201013, 'learning_rate': 7.324e-07, 'completion_length': 153.6339340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0076141357421875, 'epoch': 0.27} 27%|██▋ | 669/2500 [4:03:30<10:53:49, 21.43s/it] 27%|██▋ | 670/2500 [4:03:51<10:54:35, 21.46s/it] {'loss': 0.0004, 'grad_norm': 0.6962801382278191, 'learning_rate': 7.319999999999999e-07, 'completion_length': 145.56250762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.009246826171875, 'epoch': 0.27} 27%|██▋ | 670/2500 [4:03:51<10:54:35, 21.46s/it] 27%|██▋ | 671/2500 [4:04:14<11:09:25, 21.96s/it] {'loss': 0.0003, 'grad_norm': 0.033396592155944246, 'learning_rate': 7.316e-07, 'completion_length': 149.08929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064697265625, 'epoch': 0.27} 27%|██▋ | 671/2500 [4:04:14<11:09:25, 21.96s/it] 27%|██▋ | 672/2500 [4:04:36<11:02:57, 21.76s/it] {'loss': 0.0002, 'grad_norm': 0.16469632957722397, 'learning_rate': 7.311999999999999e-07, 'completion_length': 137.25000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.27} 27%|██▋ | 672/2500 [4:04:36<11:02:57, 21.76s/it] 27%|██▋ | 673/2500 [4:04:57<10:58:46, 21.63s/it] {'loss': 0.0003, 'grad_norm': 0.6955689064881515, 'learning_rate': 7.308e-07, 'completion_length': 148.50000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00836181640625, 'epoch': 0.27} 27%|██▋ | 673/2500 [4:04:57<10:58:46, 21.63s/it] 27%|██▋ | 674/2500 [4:05:18<10:53:44, 21.48s/it] {'loss': 0.0002, 'grad_norm': 0.028023132380295815, 'learning_rate': 7.304e-07, 'completion_length': 145.40178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052032470703125, 'epoch': 0.27} 27%|██▋ | 674/2500 [4:05:18<10:53:44, 21.48s/it] 27%|██▋ | 675/2500 [4:05:40<10:58:35, 21.65s/it] {'loss': 0.0003, 'grad_norm': 0.19675223249679638, 'learning_rate': 7.3e-07, 'completion_length': 165.08036041259766, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.007781982421875, 'epoch': 0.27} 27%|██▋ | 675/2500 [4:05:40<10:58:35, 21.65s/it] 27%|██▋ | 676/2500 [4:06:02<11:02:31, 21.79s/it] {'loss': 0.0003, 'grad_norm': 0.6334274439335659, 'learning_rate': 7.296e-07, 'completion_length': 157.8839340209961, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.12175281345844269, 'kl': 0.008697509765625, 'epoch': 0.27} 27%|██▋ | 676/2500 [4:06:02<11:02:31, 21.79s/it] 27%|██▋ | 677/2500 [4:06:24<10:58:47, 21.68s/it] {'loss': 0.0003, 'grad_norm': 0.7541824422477841, 'learning_rate': 7.291999999999999e-07, 'completion_length': 165.90179443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.0067901611328125, 'epoch': 0.27} 27%|██▋ | 677/2500 [4:06:24<10:58:47, 21.68s/it] 27%|██▋ | 678/2500 [4:06:45<10:53:56, 21.53s/it] {'loss': 0.0003, 'grad_norm': 2.274268602341627, 'learning_rate': 7.288e-07, 'completion_length': 150.6428680419922, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12444322928786278, 'kl': 0.00665283203125, 'epoch': 0.27} 27%|██▋ | 678/2500 [4:06:45<10:53:56, 21.53s/it] 27%|██▋ | 679/2500 [4:07:07<10:56:05, 21.62s/it] {'loss': 0.0002, 'grad_norm': 0.43171442486481226, 'learning_rate': 7.284e-07, 'completion_length': 155.35714721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882645100355148, 'kl': 0.006103515625, 'epoch': 0.27} 27%|██▋ | 679/2500 [4:07:07<10:56:05, 21.62s/it] 27%|██▋ | 680/2500 [4:07:28<10:54:39, 21.58s/it] {'loss': 0.0002, 'grad_norm': 0.9986093620500698, 'learning_rate': 7.28e-07, 'completion_length': 139.0714340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.1379830613732338, 'kl': 0.005157470703125, 'epoch': 0.27} 27%|██▋ | 680/2500 [4:07:28<10:54:39, 21.58s/it] 27%|██▋ | 681/2500 [4:07:50<10:53:11, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.38572983158124363, 'learning_rate': 7.276e-07, 'completion_length': 160.58036041259766, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0078887939453125, 'epoch': 0.27} 27%|██▋ | 681/2500 [4:07:50<10:53:11, 21.55s/it] 27%|██▋ | 682/2500 [4:08:10<10:44:42, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.022342293321450307, 'learning_rate': 7.271999999999999e-07, 'completion_length': 135.0357208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048828125, 'epoch': 0.27} 27%|██▋ | 682/2500 [4:08:10<10:44:42, 21.28s/it] 27%|██▋ | 683/2500 [4:08:31<10:39:24, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.017078033889299814, 'learning_rate': 7.268e-07, 'completion_length': 135.33929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0058746337890625, 'epoch': 0.27} 27%|██▋ | 683/2500 [4:08:31<10:39:24, 21.11s/it] 27%|██▋ | 684/2500 [4:08:52<10:35:20, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.2341118777524063, 'learning_rate': 7.264e-07, 'completion_length': 151.08036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005401611328125, 'epoch': 0.27} 27%|██▋ | 684/2500 [4:08:52<10:35:20, 20.99s/it] 27%|██▋ | 685/2500 [4:09:13<10:36:05, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.477914887314193, 'learning_rate': 7.259999999999999e-07, 'completion_length': 152.85714721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.006622314453125, 'epoch': 0.27} 27%|██▋ | 685/2500 [4:09:13<10:36:05, 21.03s/it] 27%|██▋ | 686/2500 [4:09:34<10:35:39, 21.03s/it] {'loss': 0.0004, 'grad_norm': 0.9601448201733163, 'learning_rate': 7.256e-07, 'completion_length': 149.25000762939453, 'rewards/accuracy_reward': 0.803571492433548, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.16262340545654297, 'kl': 0.009552001953125, 'epoch': 0.27} 27%|██▋ | 686/2500 [4:09:34<10:35:39, 21.03s/it] 27%|██▋ | 687/2500 [4:09:55<10:32:46, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.29329273794854543, 'learning_rate': 7.252e-07, 'completion_length': 138.06250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.0055084228515625, 'epoch': 0.27} 27%|██▋ | 687/2500 [4:09:55<10:32:46, 20.94s/it] 28%|██▊ | 688/2500 [4:10:16<10:39:50, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.0402236919783827, 'learning_rate': 7.247999999999999e-07, 'completion_length': 147.2232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055084228515625, 'epoch': 0.28} 28%|██▊ | 688/2500 [4:10:16<10:39:50, 21.19s/it] 28%|██▊ | 689/2500 [4:10:39<10:51:09, 21.57s/it] {'loss': 0.0004, 'grad_norm': 0.37618555536494225, 'learning_rate': 7.244e-07, 'completion_length': 164.74108123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.0091705322265625, 'epoch': 0.28} 28%|██▊ | 689/2500 [4:10:39<10:51:09, 21.57s/it] 28%|██▊ | 690/2500 [4:11:00<10:44:07, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.04231625225204514, 'learning_rate': 7.24e-07, 'completion_length': 146.25894165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006378173828125, 'epoch': 0.28} 28%|██▊ | 690/2500 [4:11:00<10:44:07, 21.35s/it] 28%|██▊ | 691/2500 [4:11:20<10:38:02, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.54739666241051, 'learning_rate': 7.235999999999999e-07, 'completion_length': 145.3482208251953, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00708770751953125, 'epoch': 0.28} 28%|██▊ | 691/2500 [4:11:20<10:38:02, 21.16s/it] 28%|██▊ | 692/2500 [4:11:42<10:39:40, 21.23s/it] {'loss': 0.0003, 'grad_norm': 0.10378998780645649, 'learning_rate': 7.231999999999999e-07, 'completion_length': 154.7857208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00762939453125, 'epoch': 0.28} 28%|██▊ | 692/2500 [4:11:42<10:39:40, 21.23s/it] 28%|██▊ | 693/2500 [4:12:03<10:39:10, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.30185733913308255, 'learning_rate': 7.228e-07, 'completion_length': 149.31250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.0084228515625, 'epoch': 0.28} 28%|██▊ | 693/2500 [4:12:03<10:39:10, 21.22s/it] 28%|██▊ | 694/2500 [4:12:24<10:38:36, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.6155271303339523, 'learning_rate': 7.224e-07, 'completion_length': 158.6964340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.10882645100355148, 'kl': 0.00689697265625, 'epoch': 0.28} 28%|██▊ | 694/2500 [4:12:24<10:38:36, 21.22s/it] 28%|██▊ | 695/2500 [4:12:45<10:37:11, 21.18s/it] {'loss': 0.0004, 'grad_norm': 0.8444371933681903, 'learning_rate': 7.219999999999999e-07, 'completion_length': 162.4107208251953, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.848214328289032, 'reward_std': 0.12565560638904572, 'kl': 0.009490966796875, 'epoch': 0.28} 28%|██▊ | 695/2500 [4:12:45<10:37:11, 21.18s/it] 28%|██▊ | 696/2500 [4:13:06<10:32:58, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.49329195599828246, 'learning_rate': 7.216e-07, 'completion_length': 142.0089340209961, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0064697265625, 'epoch': 0.28} 28%|██▊ | 696/2500 [4:13:06<10:32:58, 21.05s/it] 28%|██▊ | 697/2500 [4:13:27<10:29:07, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.193604729959396, 'learning_rate': 7.211999999999999e-07, 'completion_length': 142.43750762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00574493408203125, 'epoch': 0.28} 28%|██▊ | 697/2500 [4:13:27<10:29:07, 20.94s/it] 28%|██▊ | 698/2500 [4:13:48<10:30:56, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.44092605964260007, 'learning_rate': 7.207999999999999e-07, 'completion_length': 152.89286041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.128351628780365, 'kl': 0.0080413818359375, 'epoch': 0.28} 28%|██▊ | 698/2500 [4:13:48<10:30:56, 21.01s/it] 28%|██▊ | 699/2500 [4:14:10<10:37:49, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.6625526756280077, 'learning_rate': 7.204e-07, 'completion_length': 158.61608123779297, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0072784423828125, 'epoch': 0.28} 28%|██▊ | 699/2500 [4:14:10<10:37:49, 21.25s/it] 28%|██▊ | 700/2500 [4:14:30<10:32:01, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.02502333692025687, 'learning_rate': 7.2e-07, 'completion_length': 148.4107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052490234375, 'epoch': 0.28} 28%|██▊ | 700/2500 [4:14:30<10:32:01, 21.07s/it] 28%|██▊ | 701/2500 [4:15:41<17:54:17, 35.83s/it] {'loss': 0.0003, 'grad_norm': 0.30939373214143195, 'learning_rate': 7.196e-07, 'completion_length': 156.00000762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0067291259765625, 'epoch': 0.28} 28%|██▊ | 701/2500 [4:15:41<17:54:17, 35.83s/it] 28%|██▊ | 702/2500 [4:16:02<15:40:24, 31.38s/it] {'loss': 0.0002, 'grad_norm': 0.2330318771548674, 'learning_rate': 7.191999999999999e-07, 'completion_length': 150.3928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0047149658203125, 'epoch': 0.28} 28%|██▊ | 702/2500 [4:16:02<15:40:24, 31.38s/it] 28%|██▊ | 703/2500 [4:16:22<14:04:22, 28.19s/it] {'loss': 0.0002, 'grad_norm': 0.5195020404940878, 'learning_rate': 7.188e-07, 'completion_length': 150.21429443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726372897625, 'kl': 0.00543212890625, 'epoch': 0.28} 28%|██▊ | 703/2500 [4:16:22<14:04:22, 28.19s/it] 28%|██▊ | 704/2500 [4:16:43<12:56:25, 25.94s/it] {'loss': 0.0003, 'grad_norm': 0.6887472636914993, 'learning_rate': 7.184e-07, 'completion_length': 146.2321548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.0074310302734375, 'epoch': 0.28} 28%|██▊ | 704/2500 [4:16:43<12:56:25, 25.94s/it] 28%|██▊ | 705/2500 [4:17:04<12:09:43, 24.39s/it] {'loss': 0.0003, 'grad_norm': 0.2398229781375092, 'learning_rate': 7.179999999999999e-07, 'completion_length': 143.45536041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.006744384765625, 'epoch': 0.28} 28%|██▊ | 705/2500 [4:17:04<12:09:43, 24.39s/it] 28%|██▊ | 706/2500 [4:17:25<11:41:47, 23.47s/it] {'loss': 0.0003, 'grad_norm': 6.515457643717658, 'learning_rate': 7.176e-07, 'completion_length': 155.4553680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.19838443398475647, 'kl': 0.00811767578125, 'epoch': 0.28} 28%|██▊ | 706/2500 [4:17:25<11:41:47, 23.47s/it] 28%|██▊ | 707/2500 [4:17:46<11:16:52, 22.65s/it] {'loss': 0.0002, 'grad_norm': 1.257031577216771, 'learning_rate': 7.171999999999999e-07, 'completion_length': 141.39286041259766, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005157470703125, 'epoch': 0.28} 28%|██▊ | 707/2500 [4:17:46<11:16:52, 22.65s/it] 28%|██▊ | 708/2500 [4:18:07<11:04:34, 22.25s/it] {'loss': 0.0003, 'grad_norm': 0.8512559762786606, 'learning_rate': 7.168e-07, 'completion_length': 147.8482208251953, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.06343399360775948, 'kl': 0.0063629150390625, 'epoch': 0.28} 28%|██▊ | 708/2500 [4:18:07<11:04:34, 22.25s/it] 28%|██▊ | 709/2500 [4:18:28<10:54:32, 21.93s/it] {'loss': 0.0003, 'grad_norm': 0.5819308905632546, 'learning_rate': 7.164e-07, 'completion_length': 162.14286041259766, 'rewards/accuracy_reward': 0.7946428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7946429252624512, 'reward_std': 0.096499003469944, 'kl': 0.0083465576171875, 'epoch': 0.28} 28%|██▊ | 709/2500 [4:18:28<10:54:32, 21.93s/it] 28%|██▊ | 710/2500 [4:18:51<11:02:42, 22.21s/it] {'loss': 0.0003, 'grad_norm': 0.4136952917108122, 'learning_rate': 7.159999999999999e-07, 'completion_length': 158.49107360839844, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.05050762742757797, 'kl': 0.0081024169921875, 'epoch': 0.28} 28%|██▊ | 710/2500 [4:18:51<11:02:42, 22.21s/it] 28%|██▊ | 711/2500 [4:19:13<10:55:03, 21.97s/it] {'loss': 0.0003, 'grad_norm': 0.7750432317370342, 'learning_rate': 7.156e-07, 'completion_length': 154.49108123779297, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.17164697498083115, 'kl': 0.008209228515625, 'epoch': 0.28} 28%|██▊ | 711/2500 [4:19:13<10:55:03, 21.97s/it] 28%|██▊ | 712/2500 [4:19:33<10:43:44, 21.60s/it] {'loss': 0.0003, 'grad_norm': 0.2600575676595387, 'learning_rate': 7.151999999999999e-07, 'completion_length': 145.2589340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.007781982421875, 'epoch': 0.28} 28%|██▊ | 712/2500 [4:19:33<10:43:44, 21.60s/it] 29%|██▊ | 713/2500 [4:19:55<10:40:03, 21.49s/it] {'loss': 0.0003, 'grad_norm': 0.5438833784810692, 'learning_rate': 7.147999999999999e-07, 'completion_length': 138.9464340209961, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0074615478515625, 'epoch': 0.29} 29%|██▊ | 713/2500 [4:19:55<10:40:03, 21.49s/it] 29%|██▊ | 714/2500 [4:20:16<10:36:29, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.06486376827598336, 'learning_rate': 7.144e-07, 'completion_length': 160.8571548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0057220458984375, 'epoch': 0.29} 29%|██▊ | 714/2500 [4:20:16<10:36:29, 21.38s/it] 29%|██▊ | 715/2500 [4:20:37<10:32:43, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.19736631191844084, 'learning_rate': 7.14e-07, 'completion_length': 154.92857360839844, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0070343017578125, 'epoch': 0.29} 29%|██▊ | 715/2500 [4:20:37<10:32:43, 21.27s/it] 29%|██▊ | 716/2500 [4:20:59<10:41:54, 21.59s/it] {'loss': 0.0002, 'grad_norm': 0.3251624118686416, 'learning_rate': 7.135999999999999e-07, 'completion_length': 157.77678680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.06613001227378845, 'kl': 0.0062408447265625, 'epoch': 0.29} 29%|██▊ | 716/2500 [4:20:59<10:41:54, 21.59s/it] 29%|██▊ | 717/2500 [4:21:21<10:41:37, 21.59s/it] {'loss': 0.0003, 'grad_norm': 0.41836117839519804, 'learning_rate': 7.131999999999999e-07, 'completion_length': 169.7946548461914, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10882645100355148, 'kl': 0.0069427490234375, 'epoch': 0.29} 29%|██▊ | 717/2500 [4:21:21<10:41:37, 21.59s/it] 29%|██▊ | 718/2500 [4:21:42<10:35:46, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.20896351424723095, 'learning_rate': 7.128e-07, 'completion_length': 156.58036041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0052032470703125, 'epoch': 0.29} 29%|██▊ | 718/2500 [4:21:42<10:35:46, 21.41s/it] 29%|██▉ | 719/2500 [4:22:03<10:38:35, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.03394867173697287, 'learning_rate': 7.124e-07, 'completion_length': 151.1607208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0065155029296875, 'epoch': 0.29} 29%|██▉ | 719/2500 [4:22:03<10:38:35, 21.51s/it] 29%|██▉ | 720/2500 [4:22:25<10:42:46, 21.67s/it] {'loss': 0.0002, 'grad_norm': 0.01976831256781274, 'learning_rate': 7.119999999999999e-07, 'completion_length': 150.3571548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00421142578125, 'epoch': 0.29} 29%|██▉ | 720/2500 [4:22:25<10:42:46, 21.67s/it] 29%|██▉ | 721/2500 [4:22:47<10:41:39, 21.64s/it] {'loss': 0.0003, 'grad_norm': 0.20169827927332165, 'learning_rate': 7.116e-07, 'completion_length': 160.43750762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0074310302734375, 'epoch': 0.29} 29%|██▉ | 721/2500 [4:22:47<10:41:39, 21.64s/it] 29%|██▉ | 722/2500 [4:23:08<10:31:49, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.01966455616385285, 'learning_rate': 7.112000000000001e-07, 'completion_length': 141.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00506591796875, 'epoch': 0.29} 29%|██▉ | 722/2500 [4:23:08<10:31:49, 21.32s/it] 29%|██▉ | 723/2500 [4:23:29<10:28:49, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.5011318409649573, 'learning_rate': 7.107999999999999e-07, 'completion_length': 151.08929443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005706787109375, 'epoch': 0.29} 29%|██▉ | 723/2500 [4:23:29<10:28:49, 21.23s/it] 29%|██▉ | 724/2500 [4:23:50<10:29:39, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.3321861395401053, 'learning_rate': 7.104e-07, 'completion_length': 150.8928680419922, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.00634765625, 'epoch': 0.29} 29%|██▉ | 724/2500 [4:23:50<10:29:39, 21.27s/it] 29%|██▉ | 725/2500 [4:24:13<10:44:20, 21.78s/it] {'loss': 0.0004, 'grad_norm': 2.5721648961718095, 'learning_rate': 7.1e-07, 'completion_length': 163.2053680419922, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.17104807496070862, 'kl': 0.009002685546875, 'epoch': 0.29} 29%|██▉ | 725/2500 [4:24:13<10:44:20, 21.78s/it] 29%|██▉ | 726/2500 [4:24:34<10:34:37, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.19887664463822344, 'learning_rate': 7.096e-07, 'completion_length': 143.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.007171630859375, 'epoch': 0.29} 29%|██▉ | 726/2500 [4:24:34<10:34:37, 21.46s/it] 29%|██▉ | 727/2500 [4:24:55<10:30:26, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.5329439991451382, 'learning_rate': 7.092e-07, 'completion_length': 152.86607360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0071258544921875, 'epoch': 0.29} 29%|██▉ | 727/2500 [4:24:55<10:30:26, 21.33s/it] 29%|██▉ | 728/2500 [4:25:16<10:26:01, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.5289563499816281, 'learning_rate': 7.088e-07, 'completion_length': 154.1428680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0065460205078125, 'epoch': 0.29} 29%|██▉ | 728/2500 [4:25:16<10:26:01, 21.20s/it] 29%|██▉ | 729/2500 [4:25:36<10:20:09, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.3297430017867921, 'learning_rate': 7.084e-07, 'completion_length': 147.12500762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0062713623046875, 'epoch': 0.29} 29%|██▉ | 729/2500 [4:25:36<10:20:09, 21.01s/it] 29%|██▉ | 730/2500 [4:25:57<10:21:19, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.030257973954153337, 'learning_rate': 7.079999999999999e-07, 'completion_length': 151.7232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.007171630859375, 'epoch': 0.29} 29%|██▉ | 730/2500 [4:25:57<10:21:19, 21.06s/it] 29%|██▉ | 731/2500 [4:26:18<10:19:36, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.028816123334661426, 'learning_rate': 7.076e-07, 'completion_length': 150.16964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005462646484375, 'epoch': 0.29} 29%|██▉ | 731/2500 [4:26:18<10:19:36, 21.02s/it] 29%|██▉ | 732/2500 [4:26:38<10:10:35, 20.72s/it] {'loss': 0.0002, 'grad_norm': 0.5226681843809983, 'learning_rate': 7.072e-07, 'completion_length': 135.91964721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004608154296875, 'epoch': 0.29} 29%|██▉ | 732/2500 [4:26:38<10:10:35, 20.72s/it] 29%|██▉ | 733/2500 [4:26:59<10:13:42, 20.84s/it] {'loss': 0.0003, 'grad_norm': 0.5991902696097294, 'learning_rate': 7.068e-07, 'completion_length': 152.7946548461914, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941557407379, 'kl': 0.006591796875, 'epoch': 0.29} 29%|██▉ | 733/2500 [4:26:59<10:13:42, 20.84s/it] 29%|██▉ | 734/2500 [4:27:20<10:11:54, 20.79s/it] {'loss': 0.0002, 'grad_norm': 0.306008113878748, 'learning_rate': 7.064e-07, 'completion_length': 141.36607360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.29} 29%|██▉ | 734/2500 [4:27:20<10:11:54, 20.79s/it] 29%|██▉ | 735/2500 [4:27:41<10:16:22, 20.95s/it] {'loss': 0.0003, 'grad_norm': 0.15507956739875214, 'learning_rate': 7.059999999999999e-07, 'completion_length': 162.3839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0070343017578125, 'epoch': 0.29} 29%|██▉ | 735/2500 [4:27:41<10:16:22, 20.95s/it] 29%|██▉ | 736/2500 [4:28:03<10:17:56, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.38714837239780997, 'learning_rate': 7.056e-07, 'completion_length': 160.77678680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.00750732421875, 'epoch': 0.29} 29%|██▉ | 736/2500 [4:28:03<10:17:56, 21.02s/it] 29%|██▉ | 737/2500 [4:28:25<10:33:35, 21.56s/it] {'loss': 0.0003, 'grad_norm': 0.6797719001345915, 'learning_rate': 7.052e-07, 'completion_length': 161.96429443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006561279296875, 'epoch': 0.29} 29%|██▉ | 737/2500 [4:28:25<10:33:35, 21.56s/it] 30%|██▉ | 738/2500 [4:28:46<10:26:10, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.37831032326132713, 'learning_rate': 7.047999999999999e-07, 'completion_length': 152.74107360839844, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.07576144114136696, 'kl': 0.00811767578125, 'epoch': 0.3} 30%|██▉ | 738/2500 [4:28:46<10:26:10, 21.32s/it] 30%|██▉ | 739/2500 [4:29:08<10:26:06, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.023415271508407318, 'learning_rate': 7.044e-07, 'completion_length': 151.7857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005889892578125, 'epoch': 0.3} 30%|██▉ | 739/2500 [4:29:08<10:26:06, 21.33s/it] 30%|██▉ | 740/2500 [4:29:29<10:26:23, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.4386422560812514, 'learning_rate': 7.04e-07, 'completion_length': 153.12500762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.08747542649507523, 'kl': 0.0064697265625, 'epoch': 0.3} 30%|██▉ | 740/2500 [4:29:29<10:26:23, 21.35s/it] 30%|██▉ | 741/2500 [4:29:49<10:16:26, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.7676172256092377, 'learning_rate': 7.035999999999999e-07, 'completion_length': 145.14286041259766, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00567626953125, 'epoch': 0.3} 30%|██▉ | 741/2500 [4:29:49<10:16:26, 21.03s/it] 30%|██▉ | 742/2500 [4:30:10<10:13:22, 20.93s/it] {'loss': 0.0003, 'grad_norm': 0.22411878078209882, 'learning_rate': 7.032e-07, 'completion_length': 144.1339340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0084991455078125, 'epoch': 0.3} 30%|██▉ | 742/2500 [4:30:10<10:13:22, 20.93s/it] 30%|██▉ | 743/2500 [4:30:31<10:17:48, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.29698614270493895, 'learning_rate': 7.028e-07, 'completion_length': 163.83036041259766, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.00714111328125, 'epoch': 0.3} 30%|██▉ | 743/2500 [4:30:31<10:17:48, 21.10s/it] 30%|██▉ | 744/2500 [4:30:53<10:19:18, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.24010047565747453, 'learning_rate': 7.024e-07, 'completion_length': 155.00894165039062, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0078125, 'epoch': 0.3} 30%|██▉ | 744/2500 [4:30:53<10:19:18, 21.16s/it] 30%|██▉ | 745/2500 [4:31:13<10:15:04, 21.03s/it] {'loss': 0.0002, 'grad_norm': 1.1402710401514542, 'learning_rate': 7.019999999999999e-07, 'completion_length': 147.9732208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.006134033203125, 'epoch': 0.3} 30%|██▉ | 745/2500 [4:31:13<10:15:04, 21.03s/it] 30%|██▉ | 746/2500 [4:31:34<10:12:41, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.9933739567143457, 'learning_rate': 7.016e-07, 'completion_length': 149.68750762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.005828857421875, 'epoch': 0.3} 30%|██▉ | 746/2500 [4:31:34<10:12:41, 20.96s/it] 30%|██▉ | 747/2500 [4:31:54<10:05:23, 20.72s/it] {'loss': 0.0002, 'grad_norm': 0.985823480351829, 'learning_rate': 7.012000000000001e-07, 'completion_length': 134.05358123779297, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.08747542649507523, 'kl': 0.0046234130859375, 'epoch': 0.3} 30%|██▉ | 747/2500 [4:31:54<10:05:23, 20.72s/it] 30%|██▉ | 748/2500 [4:32:16<10:14:52, 21.06s/it] {'loss': 0.0004, 'grad_norm': 0.6736208191557504, 'learning_rate': 7.007999999999999e-07, 'completion_length': 162.1339340209961, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.11663764715194702, 'kl': 0.00933837890625, 'epoch': 0.3} 30%|██▉ | 748/2500 [4:32:16<10:14:52, 21.06s/it] 30%|██▉ | 749/2500 [4:32:38<10:20:58, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.6660064020885011, 'learning_rate': 7.004e-07, 'completion_length': 141.7232208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.06222161278128624, 'kl': 0.0052032470703125, 'epoch': 0.3} 30%|██▉ | 749/2500 [4:32:38<10:20:58, 21.28s/it] 30%|███ | 750/2500 [4:32:59<10:21:47, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.02028086425253503, 'learning_rate': 7e-07, 'completion_length': 142.56250762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0058135986328125, 'epoch': 0.3} 30%|███ | 750/2500 [4:32:59<10:21:47, 21.32s/it] 30%|███ | 751/2500 [4:33:21<10:19:48, 21.26s/it] {'loss': 0.0004, 'grad_norm': 1.3566834513217771, 'learning_rate': 6.995999999999999e-07, 'completion_length': 164.18750762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.10882645100355148, 'kl': 0.0093536376953125, 'epoch': 0.3} 30%|███ | 751/2500 [4:33:21<10:19:48, 21.26s/it] 30%|███ | 752/2500 [4:33:42<10:21:13, 21.32s/it] {'loss': 0.0004, 'grad_norm': 0.4817133317757271, 'learning_rate': 6.992e-07, 'completion_length': 176.80358123779297, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.13736960291862488, 'kl': 0.0094451904296875, 'epoch': 0.3} 30%|███ | 752/2500 [4:33:42<10:21:13, 21.32s/it] 30%|███ | 753/2500 [4:34:03<10:18:27, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.2809148698389823, 'learning_rate': 6.988e-07, 'completion_length': 157.00000762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.03696779906749725, 'kl': 0.0049285888671875, 'epoch': 0.3} 30%|███ | 753/2500 [4:34:03<10:18:27, 21.24s/it] 30%|███ | 754/2500 [4:34:24<10:14:23, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.51467052074507, 'learning_rate': 6.984e-07, 'completion_length': 152.65179443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.08868780359625816, 'kl': 0.0077362060546875, 'epoch': 0.3} 30%|███ | 754/2500 [4:34:24<10:14:23, 21.11s/it] 30%|███ | 755/2500 [4:34:45<10:16:22, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.37346194472558636, 'learning_rate': 6.979999999999999e-07, 'completion_length': 151.4107208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.007537841796875, 'epoch': 0.3} 30%|███ | 755/2500 [4:34:45<10:16:22, 21.19s/it] 30%|███ | 756/2500 [4:35:06<10:11:13, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.29730273188842404, 'learning_rate': 6.976e-07, 'completion_length': 143.52679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.007659912109375, 'epoch': 0.3} 30%|███ | 756/2500 [4:35:06<10:11:13, 21.03s/it] 30%|███ | 757/2500 [4:35:27<10:10:07, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.027230853861138878, 'learning_rate': 6.972e-07, 'completion_length': 141.91964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051727294921875, 'epoch': 0.3} 30%|███ | 757/2500 [4:35:27<10:10:07, 21.00s/it] 30%|███ | 758/2500 [4:35:48<10:09:20, 20.99s/it] {'loss': 0.0003, 'grad_norm': 1.1238107085747613, 'learning_rate': 6.967999999999999e-07, 'completion_length': 146.9464340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0066986083984375, 'epoch': 0.3} 30%|███ | 758/2500 [4:35:48<10:09:20, 20.99s/it] 30%|███ | 759/2500 [4:36:08<10:05:19, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.7665790211319309, 'learning_rate': 6.964e-07, 'completion_length': 147.20536041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12565560638904572, 'kl': 0.0052642822265625, 'epoch': 0.3} 30%|███ | 759/2500 [4:36:08<10:05:19, 20.86s/it] 30%|███ | 760/2500 [4:36:31<10:17:57, 21.31s/it] {'loss': 0.0003, 'grad_norm': 0.026824942439925697, 'learning_rate': 6.959999999999999e-07, 'completion_length': 156.90179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0065155029296875, 'epoch': 0.3} 30%|███ | 760/2500 [4:36:31<10:17:57, 21.31s/it] 30%|███ | 761/2500 [4:36:53<10:25:50, 21.59s/it] {'loss': 0.0004, 'grad_norm': 0.6349377987534818, 'learning_rate': 6.956e-07, 'completion_length': 167.8482208251953, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.07003280520439148, 'kl': 0.009002685546875, 'epoch': 0.3} 30%|███ | 761/2500 [4:36:53<10:25:50, 21.59s/it] 30%|███ | 762/2500 [4:37:14<10:17:29, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.581384968055428, 'learning_rate': 6.952e-07, 'completion_length': 151.5982208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0085906982421875, 'epoch': 0.3} 30%|███ | 762/2500 [4:37:14<10:17:29, 21.32s/it] 31%|███ | 763/2500 [4:37:34<10:09:50, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.2849391139918732, 'learning_rate': 6.947999999999999e-07, 'completion_length': 148.1696548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0061187744140625, 'epoch': 0.31} 31%|███ | 763/2500 [4:37:34<10:09:50, 21.07s/it] 31%|███ | 764/2500 [4:37:56<10:13:11, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.45519188331984267, 'learning_rate': 6.944e-07, 'completion_length': 163.55358123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.10040179640054703, 'kl': 0.007720947265625, 'epoch': 0.31} 31%|███ | 764/2500 [4:37:56<10:13:11, 21.19s/it] 31%|███ | 765/2500 [4:38:17<10:12:01, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.8509337561127087, 'learning_rate': 6.939999999999999e-07, 'completion_length': 159.5357208251953, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0056610107421875, 'epoch': 0.31} 31%|███ | 765/2500 [4:38:17<10:12:01, 21.16s/it] 31%|███ | 766/2500 [4:38:38<10:11:14, 21.15s/it] {'loss': 0.0004, 'grad_norm': 0.3023392216013139, 'learning_rate': 6.935999999999999e-07, 'completion_length': 170.71429443359375, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.009613037109375, 'epoch': 0.31} 31%|███ | 766/2500 [4:38:38<10:11:14, 21.15s/it] 31%|███ | 767/2500 [4:39:00<10:17:19, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.537403542612973, 'learning_rate': 6.932e-07, 'completion_length': 174.29464721679688, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.00830078125, 'epoch': 0.31} 31%|███ | 767/2500 [4:39:00<10:17:19, 21.37s/it] 31%|███ | 768/2500 [4:39:21<10:14:51, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.32424847947147606, 'learning_rate': 6.928e-07, 'completion_length': 151.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0062408447265625, 'epoch': 0.31} 31%|███ | 768/2500 [4:39:21<10:14:51, 21.30s/it] 31%|███ | 769/2500 [4:39:42<10:10:45, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.40013338188794106, 'learning_rate': 6.924e-07, 'completion_length': 152.36608123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00579833984375, 'epoch': 0.31} 31%|███ | 769/2500 [4:39:42<10:10:45, 21.17s/it] 31%|███ | 770/2500 [4:40:04<10:16:02, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.25121450149592683, 'learning_rate': 6.919999999999999e-07, 'completion_length': 173.68750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0076141357421875, 'epoch': 0.31} 31%|███ | 770/2500 [4:40:04<10:16:02, 21.37s/it] 31%|███ | 771/2500 [4:40:25<10:12:31, 21.26s/it] {'loss': 0.0003, 'grad_norm': 0.6147686955456194, 'learning_rate': 6.916e-07, 'completion_length': 156.37500762939453, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.0067138671875, 'epoch': 0.31} 31%|███ | 771/2500 [4:40:25<10:12:31, 21.26s/it] 31%|███ | 772/2500 [4:40:46<10:15:40, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.21050487829912054, 'learning_rate': 6.912e-07, 'completion_length': 155.74108123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006072998046875, 'epoch': 0.31} 31%|███ | 772/2500 [4:40:46<10:15:40, 21.38s/it] 31%|███ | 773/2500 [4:41:08<10:21:55, 21.61s/it] {'loss': 0.0002, 'grad_norm': 0.8341690564665739, 'learning_rate': 6.907999999999999e-07, 'completion_length': 161.33036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00611114501953125, 'epoch': 0.31} 31%|███ | 773/2500 [4:41:08<10:21:55, 21.61s/it] 31%|███ | 774/2500 [4:41:30<10:17:46, 21.48s/it] {'loss': 0.0003, 'grad_norm': 0.24153948636484082, 'learning_rate': 6.904e-07, 'completion_length': 164.06250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0069580078125, 'epoch': 0.31} 31%|███ | 774/2500 [4:41:30<10:17:46, 21.48s/it] 31%|███ | 775/2500 [4:41:51<10:13:11, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.21294536561699248, 'learning_rate': 6.9e-07, 'completion_length': 146.2678680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0050811767578125, 'epoch': 0.31} 31%|███ | 775/2500 [4:41:51<10:13:11, 21.33s/it] 31%|███ | 776/2500 [4:42:11<10:03:39, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.800007790054704, 'learning_rate': 6.895999999999999e-07, 'completion_length': 145.0446548461914, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.005584716796875, 'epoch': 0.31} 31%|███ | 776/2500 [4:42:11<10:03:39, 21.01s/it] 31%|███ | 777/2500 [4:42:32<10:03:13, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.29848322034620367, 'learning_rate': 6.892e-07, 'completion_length': 159.89286041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00616455078125, 'epoch': 0.31} 31%|███ | 777/2500 [4:42:32<10:03:13, 21.01s/it] 31%|███ | 778/2500 [4:42:53<10:04:27, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.8123958935519647, 'learning_rate': 6.888e-07, 'completion_length': 167.93750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.1379830539226532, 'kl': 0.0073089599609375, 'epoch': 0.31} 31%|███ | 778/2500 [4:42:53<10:04:27, 21.06s/it] 31%|███ | 779/2500 [4:43:14<10:01:59, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.6194166820221342, 'learning_rate': 6.883999999999999e-07, 'completion_length': 147.61608123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0056915283203125, 'epoch': 0.31} 31%|███ | 779/2500 [4:43:14<10:01:59, 20.99s/it] 31%|███ | 780/2500 [4:43:35<9:59:38, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.4197274223708725, 'learning_rate': 6.879999999999999e-07, 'completion_length': 153.0178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00555419921875, 'epoch': 0.31} 31%|███ | 780/2500 [4:43:35<9:59:38, 20.92s/it] 31%|███ | 781/2500 [4:43:56<10:05:34, 21.14s/it] {'loss': 0.0004, 'grad_norm': 0.5123278706029123, 'learning_rate': 6.876e-07, 'completion_length': 168.51786041259766, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06222161650657654, 'kl': 0.0087890625, 'epoch': 0.31} 31%|███ | 781/2500 [4:43:56<10:05:34, 21.14s/it] 31%|███▏ | 782/2500 [4:44:18<10:14:14, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.264844548725775, 'learning_rate': 6.872e-07, 'completion_length': 153.36607360839844, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.025253813713788986, 'kl': 0.0073089599609375, 'epoch': 0.31} 31%|███▏ | 782/2500 [4:44:18<10:14:14, 21.45s/it] 31%|███▏ | 783/2500 [4:44:40<10:14:14, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.3727309225442248, 'learning_rate': 6.867999999999999e-07, 'completion_length': 162.0357208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0059661865234375, 'epoch': 0.31} 31%|███▏ | 783/2500 [4:44:40<10:14:14, 21.46s/it] 31%|███▏ | 784/2500 [4:45:02<10:20:02, 21.68s/it] {'loss': 0.0002, 'grad_norm': 0.5554463234615679, 'learning_rate': 6.864e-07, 'completion_length': 144.21428680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0049285888671875, 'epoch': 0.31} 31%|███▏ | 784/2500 [4:45:02<10:20:02, 21.68s/it] 31%|███▏ | 785/2500 [4:45:23<10:17:08, 21.59s/it] {'loss': 0.0003, 'grad_norm': 0.30608414877754164, 'learning_rate': 6.86e-07, 'completion_length': 143.2232208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0079803466796875, 'epoch': 0.31} 31%|███▏ | 785/2500 [4:45:23<10:17:08, 21.59s/it] 31%|███▏ | 786/2500 [4:45:46<10:21:01, 21.74s/it] {'loss': 0.0002, 'grad_norm': 1.2648650280629012, 'learning_rate': 6.855999999999999e-07, 'completion_length': 159.4464340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0059051513671875, 'epoch': 0.31} 31%|███▏ | 786/2500 [4:45:46<10:21:01, 21.74s/it] 31%|███▏ | 787/2500 [4:46:07<10:16:07, 21.58s/it] {'loss': 0.0002, 'grad_norm': 0.5924729538387336, 'learning_rate': 6.852e-07, 'completion_length': 152.9107208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.005157470703125, 'epoch': 0.31} 31%|███▏ | 787/2500 [4:46:07<10:16:07, 21.58s/it] 32%|███▏ | 788/2500 [4:46:28<10:11:44, 21.44s/it] {'loss': 0.0002, 'grad_norm': 1.705750050573118, 'learning_rate': 6.847999999999999e-07, 'completion_length': 153.33036041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.08747542649507523, 'kl': 0.0059967041015625, 'epoch': 0.32} 32%|███▏ | 788/2500 [4:46:28<10:11:44, 21.44s/it] 32%|███▏ | 789/2500 [4:46:49<10:11:12, 21.43s/it] {'loss': 0.0003, 'grad_norm': 0.5499041501076446, 'learning_rate': 6.844e-07, 'completion_length': 150.92857360839844, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00640869140625, 'epoch': 0.32} 32%|███▏ | 789/2500 [4:46:49<10:11:12, 21.43s/it] 32%|███▏ | 790/2500 [4:47:10<10:03:00, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.022484873190690032, 'learning_rate': 6.84e-07, 'completion_length': 136.83929061889648, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005157470703125, 'epoch': 0.32} 32%|███▏ | 790/2500 [4:47:10<10:03:00, 21.16s/it] 32%|███▏ | 791/2500 [4:47:31<10:03:58, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.5564065609682223, 'learning_rate': 6.836e-07, 'completion_length': 143.62500762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.08747543022036552, 'kl': 0.00650787353515625, 'epoch': 0.32} 32%|███▏ | 791/2500 [4:47:31<10:03:58, 21.20s/it] 32%|███▏ | 792/2500 [4:47:52<10:01:55, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.3936711964781198, 'learning_rate': 6.832e-07, 'completion_length': 141.13394165039062, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0051422119140625, 'epoch': 0.32} 32%|███▏ | 792/2500 [4:47:52<10:01:55, 21.14s/it] 32%|███▏ | 793/2500 [4:48:15<10:16:06, 21.66s/it] {'loss': 0.0003, 'grad_norm': 0.5795545962565639, 'learning_rate': 6.827999999999999e-07, 'completion_length': 153.81250762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0064239501953125, 'epoch': 0.32} 32%|███▏ | 793/2500 [4:48:15<10:16:06, 21.66s/it] 32%|███▏ | 794/2500 [4:48:36<10:10:29, 21.47s/it] {'loss': 0.0004, 'grad_norm': 1.3117537678010873, 'learning_rate': 6.824e-07, 'completion_length': 164.2321548461914, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.12175281345844269, 'kl': 0.0090789794921875, 'epoch': 0.32} 32%|███▏ | 794/2500 [4:48:36<10:10:29, 21.47s/it] 32%|███▏ | 795/2500 [4:48:58<10:12:02, 21.54s/it] {'loss': 0.0003, 'grad_norm': 0.6515547637452069, 'learning_rate': 6.82e-07, 'completion_length': 152.9732208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.0070648193359375, 'epoch': 0.32} 32%|███▏ | 795/2500 [4:48:58<10:12:02, 21.54s/it] 32%|███▏ | 796/2500 [4:49:20<10:15:06, 21.66s/it] {'loss': 0.0003, 'grad_norm': 0.4517819976760438, 'learning_rate': 6.816e-07, 'completion_length': 153.83036041259766, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989997446537, 'kl': 0.00655364990234375, 'epoch': 0.32} 32%|███▏ | 796/2500 [4:49:20<10:15:06, 21.66s/it] 32%|███▏ | 797/2500 [4:49:41<10:10:33, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.6421913626154182, 'learning_rate': 6.812e-07, 'completion_length': 158.7857208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.008544921875, 'epoch': 0.32} 32%|███▏ | 797/2500 [4:49:41<10:10:33, 21.51s/it] 32%|███▏ | 798/2500 [4:50:02<10:11:16, 21.55s/it] {'loss': 0.0004, 'grad_norm': 0.40793885618822423, 'learning_rate': 6.807999999999999e-07, 'completion_length': 155.2857208251953, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.09918941557407379, 'kl': 0.0094146728515625, 'epoch': 0.32} 32%|███▏ | 798/2500 [4:50:02<10:11:16, 21.55s/it] 32%|███▏ | 799/2500 [4:50:25<10:17:22, 21.78s/it] {'loss': 0.0003, 'grad_norm': 0.26568219707522506, 'learning_rate': 6.804e-07, 'completion_length': 149.43750762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.008056640625, 'epoch': 0.32} 32%|███▏ | 799/2500 [4:50:25<10:17:22, 21.78s/it] 32%|███▏ | 800/2500 [4:50:46<10:10:59, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.3906434531592837, 'learning_rate': 6.800000000000001e-07, 'completion_length': 154.2857208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0047149658203125, 'epoch': 0.32} 32%|███▏ | 800/2500 [4:50:46<10:10:59, 21.56s/it] 32%|███▏ | 801/2500 [4:51:57<17:15:33, 36.57s/it] {'loss': 0.0003, 'grad_norm': 0.844362658488336, 'learning_rate': 6.795999999999999e-07, 'completion_length': 183.18750762939453, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.128351628780365, 'kl': 0.007659912109375, 'epoch': 0.32} 32%|███▏ | 801/2500 [4:51:57<17:15:33, 36.57s/it] 32%|███▏ | 802/2500 [4:52:19<15:07:43, 32.08s/it] {'loss': 0.0004, 'grad_norm': 0.40744133262282684, 'learning_rate': 6.792e-07, 'completion_length': 162.37500762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0739355981349945, 'kl': 0.010101318359375, 'epoch': 0.32} 32%|███▏ | 802/2500 [4:52:19<15:07:43, 32.08s/it] 32%|███▏ | 803/2500 [4:52:40<13:36:10, 28.86s/it] {'loss': 0.0002, 'grad_norm': 0.16180144047562986, 'learning_rate': 6.788e-07, 'completion_length': 145.55357360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0053253173828125, 'epoch': 0.32} 32%|███▏ | 803/2500 [4:52:40<13:36:10, 28.86s/it] 32%|███▏ | 804/2500 [4:53:01<12:27:46, 26.45s/it] {'loss': 0.0002, 'grad_norm': 0.22438304207524393, 'learning_rate': 6.783999999999999e-07, 'completion_length': 158.33036041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0061187744140625, 'epoch': 0.32} 32%|███▏ | 804/2500 [4:53:01<12:27:46, 26.45s/it] 32%|███▏ | 805/2500 [4:53:23<11:52:04, 25.21s/it] {'loss': 0.0002, 'grad_norm': 0.15515277449704798, 'learning_rate': 6.78e-07, 'completion_length': 152.8214340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00521087646484375, 'epoch': 0.32} 32%|███▏ | 805/2500 [4:53:23<11:52:04, 25.21s/it] 32%|███▏ | 806/2500 [4:53:44<11:10:20, 23.74s/it] {'loss': 0.0002, 'grad_norm': 0.6278329917827326, 'learning_rate': 6.776e-07, 'completion_length': 130.48214721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0046234130859375, 'epoch': 0.32} 32%|███▏ | 806/2500 [4:53:44<11:10:20, 23.74s/it] 32%|███▏ | 807/2500 [4:54:06<10:53:50, 23.17s/it] {'loss': 0.0002, 'grad_norm': 0.8188230808016912, 'learning_rate': 6.772e-07, 'completion_length': 152.58929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0062103271484375, 'epoch': 0.32} 32%|███▏ | 807/2500 [4:54:06<10:53:50, 23.17s/it] 32%|███▏ | 808/2500 [4:54:27<10:35:34, 22.54s/it] {'loss': 0.0002, 'grad_norm': 0.4711152690732179, 'learning_rate': 6.767999999999999e-07, 'completion_length': 152.80357360839844, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0055694580078125, 'epoch': 0.32} 32%|███▏ | 808/2500 [4:54:27<10:35:34, 22.54s/it] 32%|███▏ | 809/2500 [4:54:49<10:32:12, 22.43s/it] {'loss': 0.0002, 'grad_norm': 0.4654895329138225, 'learning_rate': 6.764e-07, 'completion_length': 160.7232208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.00616455078125, 'epoch': 0.32} 32%|███▏ | 809/2500 [4:54:49<10:32:12, 22.43s/it] 32%|███▏ | 810/2500 [4:55:10<10:23:23, 22.13s/it] {'loss': 0.0003, 'grad_norm': 1.0375102483661882, 'learning_rate': 6.76e-07, 'completion_length': 160.3482208251953, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11663764715194702, 'kl': 0.0082550048828125, 'epoch': 0.32} 32%|███▏ | 810/2500 [4:55:10<10:23:23, 22.13s/it] 32%|███▏ | 811/2500 [4:55:32<10:22:41, 22.12s/it] {'loss': 0.0004, 'grad_norm': 1.5491795250591014, 'learning_rate': 6.755999999999999e-07, 'completion_length': 165.65179443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.1575082391500473, 'kl': 0.009490966796875, 'epoch': 0.32} 32%|███▏ | 811/2500 [4:55:32<10:22:41, 22.12s/it] 32%|███▏ | 812/2500 [4:55:54<10:15:45, 21.89s/it] {'loss': 0.0003, 'grad_norm': 1.9684587680024663, 'learning_rate': 6.752e-07, 'completion_length': 146.9196548461914, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.12444883584976196, 'kl': 0.0067596435546875, 'epoch': 0.32} 32%|███▏ | 812/2500 [4:55:54<10:15:45, 21.89s/it] 33%|███▎ | 813/2500 [4:56:16<10:15:41, 21.90s/it] {'loss': 0.0003, 'grad_norm': 0.8849736438268841, 'learning_rate': 6.747999999999999e-07, 'completion_length': 159.75000762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.15872061252593994, 'kl': 0.006744384765625, 'epoch': 0.33} 33%|███▎ | 813/2500 [4:56:16<10:15:41, 21.90s/it] 33%|███▎ | 814/2500 [4:56:37<10:14:20, 21.86s/it] {'loss': 0.0003, 'grad_norm': 0.35555421685418503, 'learning_rate': 6.744e-07, 'completion_length': 153.2589340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.08747542649507523, 'kl': 0.0074005126953125, 'epoch': 0.33} 33%|███▎ | 814/2500 [4:56:37<10:14:20, 21.86s/it] 33%|███▎ | 815/2500 [4:56:59<10:10:36, 21.74s/it] {'loss': 0.0002, 'grad_norm': 0.02087584424809891, 'learning_rate': 6.74e-07, 'completion_length': 152.75000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0044403076171875, 'epoch': 0.33} 33%|███▎ | 815/2500 [4:56:59<10:10:36, 21.74s/it] 33%|███▎ | 816/2500 [4:57:20<10:07:52, 21.66s/it] {'loss': 0.0003, 'grad_norm': 0.038198678196864434, 'learning_rate': 6.736e-07, 'completion_length': 150.45536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064849853515625, 'epoch': 0.33} 33%|███▎ | 816/2500 [4:57:20<10:07:52, 21.66s/it] 33%|███▎ | 817/2500 [4:57:41<10:01:19, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.871252990702217, 'learning_rate': 6.732e-07, 'completion_length': 152.6696548461914, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.16922222077846527, 'kl': 0.00640869140625, 'epoch': 0.33} 33%|███▎ | 817/2500 [4:57:41<10:01:19, 21.44s/it] 33%|███▎ | 818/2500 [4:58:02<9:53:23, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.361868684441465, 'learning_rate': 6.727999999999999e-07, 'completion_length': 147.57144165039062, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0063629150390625, 'epoch': 0.33} 33%|███▎ | 818/2500 [4:58:02<9:53:23, 21.17s/it] 33%|███▎ | 819/2500 [4:58:22<9:44:34, 20.87s/it] {'loss': 0.0002, 'grad_norm': 3.169899653392437, 'learning_rate': 6.724e-07, 'completion_length': 144.06250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0053558349609375, 'epoch': 0.33} 33%|███▎ | 819/2500 [4:58:22<9:44:34, 20.87s/it] 33%|███▎ | 820/2500 [4:58:43<9:48:43, 21.03s/it] {'loss': 0.0004, 'grad_norm': 0.5097041523419108, 'learning_rate': 6.72e-07, 'completion_length': 160.08036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00927734375, 'epoch': 0.33} 33%|███▎ | 820/2500 [4:58:43<9:48:43, 21.03s/it] 33%|███▎ | 821/2500 [4:59:05<9:53:13, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.3317328220332406, 'learning_rate': 6.716e-07, 'completion_length': 146.9375, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.03696779906749725, 'kl': 0.0076751708984375, 'epoch': 0.33} 33%|███▎ | 821/2500 [4:59:05<9:53:13, 21.20s/it] 33%|███▎ | 822/2500 [4:59:26<9:49:48, 21.09s/it] {'loss': 0.0003, 'grad_norm': 0.2620863613622822, 'learning_rate': 6.712e-07, 'completion_length': 147.0089340209961, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0063323974609375, 'epoch': 0.33} 33%|███▎ | 822/2500 [4:59:26<9:49:48, 21.09s/it] 33%|███▎ | 823/2500 [4:59:47<9:49:29, 21.09s/it] {'loss': 0.0003, 'grad_norm': 0.733744092142812, 'learning_rate': 6.707999999999999e-07, 'completion_length': 157.3214340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0077667236328125, 'epoch': 0.33} 33%|███▎ | 823/2500 [4:59:47<9:49:29, 21.09s/it] 33%|███▎ | 824/2500 [5:00:08<9:51:40, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.8811240691551246, 'learning_rate': 6.704e-07, 'completion_length': 157.67858123779297, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.0059661865234375, 'epoch': 0.33} 33%|███▎ | 824/2500 [5:00:08<9:51:40, 21.18s/it] 33%|███▎ | 825/2500 [5:00:30<9:51:42, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.47966259318515064, 'learning_rate': 6.7e-07, 'completion_length': 148.7589340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.005401611328125, 'epoch': 0.33} 33%|███▎ | 825/2500 [5:00:30<9:51:42, 21.20s/it] 33%|███▎ | 826/2500 [5:00:50<9:45:55, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.4571482971994982, 'learning_rate': 6.695999999999999e-07, 'completion_length': 150.2857208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.007537841796875, 'epoch': 0.33} 33%|███▎ | 826/2500 [5:00:50<9:45:55, 21.00s/it] 33%|███▎ | 827/2500 [5:01:11<9:43:13, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.3439557994900676, 'learning_rate': 6.692e-07, 'completion_length': 151.5178680419922, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0050811767578125, 'epoch': 0.33} 33%|███▎ | 827/2500 [5:01:11<9:43:13, 20.92s/it] 33%|███▎ | 828/2500 [5:01:33<9:49:14, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.8760141223829101, 'learning_rate': 6.688e-07, 'completion_length': 167.87500762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.00860595703125, 'epoch': 0.33} 33%|███▎ | 828/2500 [5:01:33<9:49:14, 21.15s/it] 33%|███▎ | 829/2500 [5:01:54<9:48:26, 21.13s/it] {'loss': 0.0002, 'grad_norm': 3.1179831916607275, 'learning_rate': 6.683999999999999e-07, 'completion_length': 150.90178680419922, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00543212890625, 'epoch': 0.33} 33%|███▎ | 829/2500 [5:01:54<9:48:26, 21.13s/it] 33%|███▎ | 830/2500 [5:02:15<9:52:17, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.506557396432685, 'learning_rate': 6.68e-07, 'completion_length': 165.27678680419922, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.10700060427188873, 'kl': 0.0086517333984375, 'epoch': 0.33} 33%|███▎ | 830/2500 [5:02:15<9:52:17, 21.28s/it] 33%|███▎ | 831/2500 [5:02:37<9:52:04, 21.29s/it] {'loss': 0.0002, 'grad_norm': 1.1124074133715003, 'learning_rate': 6.676e-07, 'completion_length': 149.24107360839844, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0057220458984375, 'epoch': 0.33} 33%|███▎ | 831/2500 [5:02:37<9:52:04, 21.29s/it] 33%|███▎ | 832/2500 [5:02:59<9:59:11, 21.55s/it] {'loss': 0.0002, 'grad_norm': 0.1901885020938209, 'learning_rate': 6.671999999999999e-07, 'completion_length': 155.4553680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0051422119140625, 'epoch': 0.33} 33%|███▎ | 832/2500 [5:02:59<9:59:11, 21.55s/it] 33%|███▎ | 833/2500 [5:03:20<9:55:36, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.02514860800303005, 'learning_rate': 6.667999999999999e-07, 'completion_length': 152.56250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056304931640625, 'epoch': 0.33} 33%|███▎ | 833/2500 [5:03:20<9:55:36, 21.44s/it] 33%|███▎ | 834/2500 [5:03:41<9:54:02, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.17128858522460705, 'learning_rate': 6.664e-07, 'completion_length': 155.24108123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0063018798828125, 'epoch': 0.33} 33%|███▎ | 834/2500 [5:03:41<9:54:02, 21.39s/it] 33%|███▎ | 835/2500 [5:04:02<9:48:19, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.6993524279856125, 'learning_rate': 6.66e-07, 'completion_length': 147.65179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064697265625, 'epoch': 0.33} 33%|███▎ | 835/2500 [5:04:02<9:48:19, 21.20s/it] 33%|███▎ | 836/2500 [5:04:23<9:50:49, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.1936478895295517, 'learning_rate': 6.655999999999999e-07, 'completion_length': 159.21428680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005859375, 'epoch': 0.33} 33%|███▎ | 836/2500 [5:04:23<9:50:49, 21.30s/it] 33%|███▎ | 837/2500 [5:04:44<9:47:18, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.506382279458885, 'learning_rate': 6.652e-07, 'completion_length': 148.43750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048675537109375, 'epoch': 0.33} 33%|███▎ | 837/2500 [5:04:44<9:47:18, 21.19s/it] 34%|███▎ | 838/2500 [5:05:05<9:44:04, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.47204875694723497, 'learning_rate': 6.647999999999999e-07, 'completion_length': 149.50000762939453, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.11332815140485764, 'kl': 0.005706787109375, 'epoch': 0.34} 34%|███▎ | 838/2500 [5:05:05<9:44:04, 21.09s/it] 34%|███▎ | 839/2500 [5:05:27<9:46:56, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.3904349107181302, 'learning_rate': 6.643999999999999e-07, 'completion_length': 160.4732208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.00701904296875, 'epoch': 0.34} 34%|███▎ | 839/2500 [5:05:27<9:46:56, 21.20s/it] 34%|███▎ | 840/2500 [5:05:48<9:43:40, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.03391599118938812, 'learning_rate': 6.64e-07, 'completion_length': 152.67858123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0062103271484375, 'epoch': 0.34} 34%|███▎ | 840/2500 [5:05:48<9:43:40, 21.10s/it] 34%|███▎ | 841/2500 [5:06:08<9:41:47, 21.04s/it] {'loss': 0.0003, 'grad_norm': 0.31493326228936336, 'learning_rate': 6.636e-07, 'completion_length': 152.7232208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0078887939453125, 'epoch': 0.34} 34%|███▎ | 841/2500 [5:06:08<9:41:47, 21.04s/it] 34%|███▎ | 842/2500 [5:06:30<9:42:33, 21.08s/it] {'loss': 0.0004, 'grad_norm': 0.3837410763090757, 'learning_rate': 6.632e-07, 'completion_length': 162.7053680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.009002685546875, 'epoch': 0.34} 34%|███▎ | 842/2500 [5:06:30<9:42:33, 21.08s/it] 34%|███▎ | 843/2500 [5:06:51<9:43:00, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.7624480970902647, 'learning_rate': 6.627999999999999e-07, 'completion_length': 159.81250762939453, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.10821298509836197, 'kl': 0.00685882568359375, 'epoch': 0.34} 34%|███▎ | 843/2500 [5:06:51<9:43:00, 21.11s/it] 34%|███▍ | 844/2500 [5:07:13<9:48:51, 21.34s/it] {'loss': 0.0003, 'grad_norm': 0.7202872430862304, 'learning_rate': 6.624e-07, 'completion_length': 162.46429443359375, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10882644355297089, 'kl': 0.007293701171875, 'epoch': 0.34} 34%|███▍ | 844/2500 [5:07:13<9:48:51, 21.34s/it] 34%|███▍ | 845/2500 [5:07:34<9:46:56, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.5936813059981613, 'learning_rate': 6.62e-07, 'completion_length': 144.36608123779297, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.07576143741607666, 'kl': 0.00540924072265625, 'epoch': 0.34} 34%|███▍ | 845/2500 [5:07:34<9:46:56, 21.28s/it] 34%|███▍ | 846/2500 [5:07:57<9:58:23, 21.71s/it] {'loss': 0.0002, 'grad_norm': 0.30518620431325727, 'learning_rate': 6.615999999999999e-07, 'completion_length': 144.16964721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0055084228515625, 'epoch': 0.34} 34%|███▍ | 846/2500 [5:07:57<9:58:23, 21.71s/it] 34%|███▍ | 847/2500 [5:08:18<9:53:51, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.40417448720109717, 'learning_rate': 6.612e-07, 'completion_length': 152.2053680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.10431019216775894, 'kl': 0.0062255859375, 'epoch': 0.34} 34%|███▍ | 847/2500 [5:08:18<9:53:51, 21.56s/it] 34%|███▍ | 848/2500 [5:08:38<9:44:42, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.03660367289130353, 'learning_rate': 6.608e-07, 'completion_length': 158.17857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005767822265625, 'epoch': 0.34} 34%|███▍ | 848/2500 [5:08:38<9:44:42, 21.24s/it] 34%|███▍ | 849/2500 [5:08:59<9:43:32, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.5460992365796352, 'learning_rate': 6.604e-07, 'completion_length': 152.81250762939453, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0079345703125, 'epoch': 0.34} 34%|███▍ | 849/2500 [5:08:59<9:43:32, 21.21s/it] 34%|███▍ | 850/2500 [5:09:20<9:36:52, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.2833787314885143, 'learning_rate': 6.6e-07, 'completion_length': 136.24108123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0045013427734375, 'epoch': 0.34} 34%|███▍ | 850/2500 [5:09:20<9:36:52, 20.98s/it] 34%|███▍ | 851/2500 [5:09:40<9:31:42, 20.80s/it] {'loss': 0.0003, 'grad_norm': 1.0327608161031478, 'learning_rate': 6.595999999999999e-07, 'completion_length': 145.9107208251953, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12175280973315239, 'kl': 0.0079345703125, 'epoch': 0.34} 34%|███▍ | 851/2500 [5:09:40<9:31:42, 20.80s/it] 34%|███▍ | 852/2500 [5:10:02<9:35:39, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.708125173078164, 'learning_rate': 6.592e-07, 'completion_length': 158.55358123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.0052642822265625, 'epoch': 0.34} 34%|███▍ | 852/2500 [5:10:02<9:35:39, 20.96s/it] 34%|███▍ | 853/2500 [5:10:23<9:37:38, 21.04s/it] {'loss': 0.0003, 'grad_norm': 0.041884114692287304, 'learning_rate': 6.588e-07, 'completion_length': 149.4107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0068206787109375, 'epoch': 0.34} 34%|███▍ | 853/2500 [5:10:23<9:37:38, 21.04s/it] 34%|███▍ | 854/2500 [5:10:44<9:39:14, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.2974503007360024, 'learning_rate': 6.583999999999999e-07, 'completion_length': 163.9464340209961, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0072784423828125, 'epoch': 0.34} 34%|███▍ | 854/2500 [5:10:44<9:39:14, 21.11s/it] 34%|███▍ | 855/2500 [5:11:05<9:38:52, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.5972896961015575, 'learning_rate': 6.58e-07, 'completion_length': 162.16964721679688, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.10040178894996643, 'kl': 0.0086822509765625, 'epoch': 0.34} 34%|███▍ | 855/2500 [5:11:05<9:38:52, 21.11s/it] 34%|███▍ | 856/2500 [5:11:26<9:39:10, 21.14s/it] {'loss': 0.0001, 'grad_norm': 0.021500241977631663, 'learning_rate': 6.576e-07, 'completion_length': 140.95536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0035858154296875, 'epoch': 0.34} 34%|███▍ | 856/2500 [5:11:26<9:39:10, 21.14s/it] 34%|███▍ | 857/2500 [5:11:47<9:31:29, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.39291055561509436, 'learning_rate': 6.571999999999999e-07, 'completion_length': 135.04464721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0060272216796875, 'epoch': 0.34} 34%|███▍ | 857/2500 [5:11:47<9:31:29, 20.87s/it] 34%|███▍ | 858/2500 [5:12:08<9:32:16, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.31362632553215747, 'learning_rate': 6.568e-07, 'completion_length': 146.6339340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0060577392578125, 'epoch': 0.34} 34%|███▍ | 858/2500 [5:12:08<9:32:16, 20.91s/it] 34%|███▍ | 859/2500 [5:12:28<9:31:32, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.2983927549466528, 'learning_rate': 6.564e-07, 'completion_length': 154.3125, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00518798828125, 'epoch': 0.34} 34%|███▍ | 859/2500 [5:12:28<9:31:32, 20.90s/it] 34%|███▍ | 860/2500 [5:12:49<9:30:25, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.19502994863130654, 'learning_rate': 6.56e-07, 'completion_length': 152.31250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00537109375, 'epoch': 0.34} 34%|███▍ | 860/2500 [5:12:49<9:30:25, 20.87s/it] 34%|███▍ | 861/2500 [5:13:09<9:24:25, 20.66s/it] {'loss': 0.0002, 'grad_norm': 0.027798473432171733, 'learning_rate': 6.555999999999999e-07, 'completion_length': 145.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052337646484375, 'epoch': 0.34} 34%|███▍ | 861/2500 [5:13:09<9:24:25, 20.66s/it] 34%|███▍ | 862/2500 [5:13:31<9:27:26, 20.79s/it] {'loss': 0.0004, 'grad_norm': 0.5578521125955219, 'learning_rate': 6.552e-07, 'completion_length': 162.27679443359375, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747543022036552, 'kl': 0.0087890625, 'epoch': 0.34} 34%|███▍ | 862/2500 [5:13:31<9:27:26, 20.79s/it] 35%|███▍ | 863/2500 [5:13:51<9:28:01, 20.82s/it] {'loss': 0.0002, 'grad_norm': 0.5993276010566305, 'learning_rate': 6.548000000000001e-07, 'completion_length': 151.5178680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.0060272216796875, 'epoch': 0.35} 35%|███▍ | 863/2500 [5:13:51<9:28:01, 20.82s/it] 35%|███▍ | 864/2500 [5:14:12<9:25:47, 20.75s/it] {'loss': 0.0003, 'grad_norm': 0.8784934786174259, 'learning_rate': 6.543999999999999e-07, 'completion_length': 166.10714721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00634765625, 'epoch': 0.35} 35%|███▍ | 864/2500 [5:14:12<9:25:47, 20.75s/it] 35%|███▍ | 865/2500 [5:14:32<9:19:57, 20.55s/it] {'loss': 0.0003, 'grad_norm': 0.2511065613639487, 'learning_rate': 6.54e-07, 'completion_length': 134.0982208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00635528564453125, 'epoch': 0.35} 35%|███▍ | 865/2500 [5:14:32<9:19:57, 20.55s/it] 35%|███▍ | 866/2500 [5:14:52<9:15:26, 20.40s/it] {'loss': 0.0003, 'grad_norm': 0.024918372400967406, 'learning_rate': 6.536e-07, 'completion_length': 135.40179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006439208984375, 'epoch': 0.35} 35%|███▍ | 866/2500 [5:14:52<9:15:26, 20.40s/it] 35%|███▍ | 867/2500 [5:15:12<9:11:18, 20.26s/it] {'loss': 0.0002, 'grad_norm': 0.2325546144137674, 'learning_rate': 6.531999999999999e-07, 'completion_length': 135.92858123779297, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.03696779906749725, 'kl': 0.0053558349609375, 'epoch': 0.35} 35%|███▍ | 867/2500 [5:15:12<9:11:18, 20.26s/it] 35%|███▍ | 868/2500 [5:15:32<9:11:01, 20.26s/it] {'loss': 0.0002, 'grad_norm': 1.001938504930073, 'learning_rate': 6.528e-07, 'completion_length': 146.50000762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.00616455078125, 'epoch': 0.35} 35%|███▍ | 868/2500 [5:15:32<9:11:01, 20.26s/it] 35%|███▍ | 869/2500 [5:15:55<9:27:13, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.46020662718907906, 'learning_rate': 6.524e-07, 'completion_length': 137.5714340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0059967041015625, 'epoch': 0.35} 35%|███▍ | 869/2500 [5:15:55<9:27:13, 20.87s/it] 35%|███▍ | 870/2500 [5:16:17<9:37:50, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.4617121097693336, 'learning_rate': 6.52e-07, 'completion_length': 145.36607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.006622314453125, 'epoch': 0.35} 35%|███▍ | 870/2500 [5:16:17<9:37:50, 21.27s/it] 35%|███▍ | 871/2500 [5:16:40<9:56:58, 21.99s/it] {'loss': 0.0003, 'grad_norm': 0.3699015774482687, 'learning_rate': 6.515999999999999e-07, 'completion_length': 148.3303680419922, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.006500244140625, 'epoch': 0.35} 35%|███▍ | 871/2500 [5:16:40<9:56:58, 21.99s/it] 35%|███▍ | 872/2500 [5:17:01<9:44:57, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.6094364112840032, 'learning_rate': 6.512e-07, 'completion_length': 141.5178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0040283203125, 'epoch': 0.35} 35%|███▍ | 872/2500 [5:17:01<9:44:57, 21.56s/it] 35%|███▍ | 873/2500 [5:17:22<9:40:12, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.39808391682658467, 'learning_rate': 6.508e-07, 'completion_length': 148.05358123779297, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.0739355981349945, 'kl': 0.0080718994140625, 'epoch': 0.35} 35%|███▍ | 873/2500 [5:17:22<9:40:12, 21.40s/it] 35%|███▍ | 874/2500 [5:17:43<9:36:50, 21.29s/it] {'loss': 0.0003, 'grad_norm': 0.5763686691790253, 'learning_rate': 6.504e-07, 'completion_length': 148.18750762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.00762939453125, 'epoch': 0.35} 35%|███▍ | 874/2500 [5:17:43<9:36:50, 21.29s/it] 35%|███▌ | 875/2500 [5:18:04<9:37:07, 21.31s/it] {'loss': 0.0003, 'grad_norm': 0.35932347553420274, 'learning_rate': 6.5e-07, 'completion_length': 151.17858123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0071868896484375, 'epoch': 0.35} 35%|███▌ | 875/2500 [5:18:04<9:37:07, 21.31s/it] 35%|███▌ | 876/2500 [5:18:26<9:35:26, 21.26s/it] {'loss': 0.0003, 'grad_norm': 1.226937325226911, 'learning_rate': 6.495999999999999e-07, 'completion_length': 156.93750762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0071868896484375, 'epoch': 0.35} 35%|███▌ | 876/2500 [5:18:26<9:35:26, 21.26s/it] 35%|███▌ | 877/2500 [5:18:47<9:36:50, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.5379368694880182, 'learning_rate': 6.492e-07, 'completion_length': 172.6339340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.006866455078125, 'epoch': 0.35} 35%|███▌ | 877/2500 [5:18:47<9:36:50, 21.33s/it] 35%|███▌ | 878/2500 [5:19:09<9:41:49, 21.52s/it] {'loss': 0.0002, 'grad_norm': 0.5698013360736081, 'learning_rate': 6.488e-07, 'completion_length': 150.46429443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.005645751953125, 'epoch': 0.35} 35%|███▌ | 878/2500 [5:19:09<9:41:49, 21.52s/it] 35%|███▌ | 879/2500 [5:19:31<9:46:29, 21.71s/it] {'loss': 0.0003, 'grad_norm': 0.5466342577199386, 'learning_rate': 6.483999999999999e-07, 'completion_length': 168.0178680419922, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0076141357421875, 'epoch': 0.35} 35%|███▌ | 879/2500 [5:19:31<9:46:29, 21.71s/it] 35%|███▌ | 880/2500 [5:19:52<9:40:01, 21.48s/it] {'loss': 0.0002, 'grad_norm': 0.6607074545343505, 'learning_rate': 6.48e-07, 'completion_length': 150.85714721679688, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.14518077671527863, 'kl': 0.00524139404296875, 'epoch': 0.35} 35%|███▌ | 880/2500 [5:19:52<9:40:01, 21.48s/it] 35%|███▌ | 881/2500 [5:20:14<9:40:55, 21.53s/it] {'loss': 0.0003, 'grad_norm': 0.3709587939646114, 'learning_rate': 6.476e-07, 'completion_length': 165.8214340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006317138671875, 'epoch': 0.35} 35%|███▌ | 881/2500 [5:20:14<9:40:55, 21.53s/it] 35%|███▌ | 882/2500 [5:20:35<9:38:54, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.21102685978987346, 'learning_rate': 6.471999999999999e-07, 'completion_length': 145.05357360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00556182861328125, 'epoch': 0.35} 35%|███▌ | 882/2500 [5:20:35<9:38:54, 21.47s/it] 35%|███▌ | 883/2500 [5:20:57<9:39:56, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.58764515057753, 'learning_rate': 6.468e-07, 'completion_length': 164.87500762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007110595703125, 'epoch': 0.35} 35%|███▌ | 883/2500 [5:20:57<9:39:56, 21.52s/it] 35%|███▌ | 884/2500 [5:21:19<9:42:03, 21.61s/it] {'loss': 0.0003, 'grad_norm': 0.40438004178155446, 'learning_rate': 6.464e-07, 'completion_length': 161.92858123779297, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.06613001227378845, 'kl': 0.0069122314453125, 'epoch': 0.35} 35%|███▌ | 884/2500 [5:21:19<9:42:03, 21.61s/it] 35%|███▌ | 885/2500 [5:21:39<9:33:19, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.2981731749430391, 'learning_rate': 6.46e-07, 'completion_length': 131.8928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0043792724609375, 'epoch': 0.35} 35%|███▌ | 885/2500 [5:21:39<9:33:19, 21.30s/it] 35%|███▌ | 886/2500 [5:22:00<9:30:47, 21.22s/it] {'loss': 0.0005, 'grad_norm': 0.5874572719863328, 'learning_rate': 6.455999999999999e-07, 'completion_length': 155.4821548461914, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.13615721464157104, 'kl': 0.01275634765625, 'epoch': 0.35} 35%|███▌ | 886/2500 [5:22:00<9:30:47, 21.22s/it] 35%|███▌ | 887/2500 [5:22:22<9:33:08, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.629655915081114, 'learning_rate': 6.452e-07, 'completion_length': 143.90178680419922, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.12054043635725975, 'kl': 0.006011962890625, 'epoch': 0.35} 35%|███▌ | 887/2500 [5:22:22<9:33:08, 21.32s/it] 36%|███▌ | 888/2500 [5:22:43<9:29:41, 21.20s/it] {'loss': 0.0003, 'grad_norm': 2.3178074741278794, 'learning_rate': 6.448000000000001e-07, 'completion_length': 159.21429443359375, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.0835726372897625, 'kl': 0.0066986083984375, 'epoch': 0.36} 36%|███▌ | 888/2500 [5:22:43<9:29:41, 21.20s/it] 36%|███▌ | 889/2500 [5:23:04<9:26:21, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.4059105612601381, 'learning_rate': 6.443999999999999e-07, 'completion_length': 155.24108123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.07576143741607666, 'kl': 0.00604248046875, 'epoch': 0.36} 36%|███▌ | 889/2500 [5:23:04<9:26:21, 21.09s/it] 36%|███▌ | 890/2500 [5:23:24<9:20:03, 20.87s/it] {'loss': 0.0003, 'grad_norm': 0.5105918536882489, 'learning_rate': 6.44e-07, 'completion_length': 149.32144165039062, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0081329345703125, 'epoch': 0.36} 36%|███▌ | 890/2500 [5:23:24<9:20:03, 20.87s/it] 36%|███▌ | 891/2500 [5:23:45<9:18:57, 20.84s/it] {'loss': 0.0002, 'grad_norm': 0.5046478057943681, 'learning_rate': 6.436e-07, 'completion_length': 147.51786041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.00470733642578125, 'epoch': 0.36} 36%|███▌ | 891/2500 [5:23:45<9:18:57, 20.84s/it] 36%|███▌ | 892/2500 [5:24:05<9:18:25, 20.84s/it] {'loss': 0.0003, 'grad_norm': 0.28015229995273144, 'learning_rate': 6.431999999999999e-07, 'completion_length': 162.65179443359375, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0076141357421875, 'epoch': 0.36} 36%|███▌ | 892/2500 [5:24:05<9:18:25, 20.84s/it] 36%|███▌ | 893/2500 [5:24:26<9:13:47, 20.68s/it] {'loss': 0.0002, 'grad_norm': 0.7278978634760728, 'learning_rate': 6.428e-07, 'completion_length': 140.39286041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00498199462890625, 'epoch': 0.36} 36%|███▌ | 893/2500 [5:24:26<9:13:47, 20.68s/it] 36%|███▌ | 894/2500 [5:24:47<9:21:45, 20.99s/it] {'loss': 0.0003, 'grad_norm': 0.2499201132442381, 'learning_rate': 6.424e-07, 'completion_length': 166.64286041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0066986083984375, 'epoch': 0.36} 36%|███▌ | 894/2500 [5:24:47<9:21:45, 20.99s/it] 36%|███▌ | 895/2500 [5:25:09<9:25:30, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.43827745340953794, 'learning_rate': 6.42e-07, 'completion_length': 147.82144165039062, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0052642822265625, 'epoch': 0.36} 36%|███▌ | 895/2500 [5:25:09<9:25:30, 21.14s/it] 36%|███▌ | 896/2500 [5:25:30<9:26:47, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.023310078697785563, 'learning_rate': 6.415999999999999e-07, 'completion_length': 149.76786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005523681640625, 'epoch': 0.36} 36%|███▌ | 896/2500 [5:25:30<9:26:47, 21.20s/it] 36%|███▌ | 897/2500 [5:25:51<9:24:45, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.4540120759635597, 'learning_rate': 6.412e-07, 'completion_length': 165.56250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0056915283203125, 'epoch': 0.36} 36%|███▌ | 897/2500 [5:25:51<9:24:45, 21.14s/it] 36%|███▌ | 898/2500 [5:26:12<9:20:01, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.7544852633955629, 'learning_rate': 6.408e-07, 'completion_length': 142.61608123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0055694580078125, 'epoch': 0.36} 36%|███▌ | 898/2500 [5:26:12<9:20:01, 20.97s/it] 36%|███▌ | 899/2500 [5:26:33<9:19:49, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.467888183776142, 'learning_rate': 6.403999999999999e-07, 'completion_length': 149.4196548461914, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.00604248046875, 'epoch': 0.36} 36%|███▌ | 899/2500 [5:26:33<9:19:49, 20.98s/it] 36%|███▌ | 900/2500 [5:26:54<9:23:05, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.732889190984375, 'learning_rate': 6.4e-07, 'completion_length': 161.39286041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.12054044008255005, 'kl': 0.00689697265625, 'epoch': 0.36} 36%|███▌ | 900/2500 [5:26:54<9:23:05, 21.12s/it] 36%|███▌ | 901/2500 [5:27:45<13:16:49, 29.90s/it] {'loss': 0.0003, 'grad_norm': 0.35497752037594377, 'learning_rate': 6.395999999999999e-07, 'completion_length': 158.08929443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.0067901611328125, 'epoch': 0.36} 36%|███▌ | 901/2500 [5:27:45<13:16:49, 29.90s/it] 36%|███▌ | 902/2500 [5:27:55<10:42:06, 24.11s/it] {'loss': 0.0003, 'grad_norm': 0.5992661718396706, 'learning_rate': 6.392e-07, 'completion_length': 149.41964721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0063629150390625, 'epoch': 0.36} 36%|███▌ | 902/2500 [5:27:55<10:42:06, 24.11s/it] 36%|███▌ | 903/2500 [5:28:06<8:51:28, 19.97s/it] {'loss': 0.0002, 'grad_norm': 0.5675378075390644, 'learning_rate': 6.388e-07, 'completion_length': 144.06250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0038909912109375, 'epoch': 0.36} 36%|███▌ | 903/2500 [5:28:06<8:51:28, 19.97s/it] 36%|███▌ | 904/2500 [5:28:16<7:31:32, 16.98s/it] {'loss': 0.0002, 'grad_norm': 0.04917143246928002, 'learning_rate': 6.383999999999999e-07, 'completion_length': 144.33036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005615234375, 'epoch': 0.36} 36%|███▌ | 904/2500 [5:28:16<7:31:32, 16.98s/it] 36%|███▌ | 905/2500 [5:28:26<6:39:56, 15.04s/it] {'loss': 0.0003, 'grad_norm': 0.3196079429991478, 'learning_rate': 6.38e-07, 'completion_length': 160.82144165039062, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.007080078125, 'epoch': 0.36} 36%|███▌ | 905/2500 [5:28:26<6:39:56, 15.04s/it] 36%|███▌ | 906/2500 [5:28:37<6:05:25, 13.75s/it] {'loss': 0.0003, 'grad_norm': 18.179259508856763, 'learning_rate': 6.375999999999999e-07, 'completion_length': 147.23214721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.08747542649507523, 'kl': 0.008392333984375, 'epoch': 0.36} 36%|███▌ | 906/2500 [5:28:37<6:05:25, 13.75s/it] 36%|███▋ | 907/2500 [5:28:48<5:45:10, 13.00s/it] {'loss': 0.0003, 'grad_norm': 0.7889635218302227, 'learning_rate': 6.371999999999999e-07, 'completion_length': 158.11608123779297, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.09528662264347076, 'kl': 0.008331298828125, 'epoch': 0.36} 36%|███▋ | 907/2500 [5:28:48<5:45:10, 13.00s/it] 36%|███▋ | 908/2500 [5:28:58<5:23:41, 12.20s/it] {'loss': 0.0003, 'grad_norm': 0.7256820945006368, 'learning_rate': 6.368e-07, 'completion_length': 148.1607208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.10882644727826118, 'kl': 0.0076904296875, 'epoch': 0.36} 36%|███▋ | 908/2500 [5:28:58<5:23:41, 12.20s/it] 36%|███▋ | 909/2500 [5:29:09<5:12:28, 11.78s/it] {'loss': 0.0003, 'grad_norm': 0.5919623532822323, 'learning_rate': 6.364e-07, 'completion_length': 158.99107360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.07576143741607666, 'kl': 0.007659912109375, 'epoch': 0.36} 36%|███▋ | 909/2500 [5:29:09<5:12:28, 11.78s/it] 36%|███▋ | 910/2500 [5:29:20<5:04:16, 11.48s/it] {'loss': 0.0002, 'grad_norm': 0.26564900303304356, 'learning_rate': 6.36e-07, 'completion_length': 156.80358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0056304931640625, 'epoch': 0.36} 36%|███▋ | 910/2500 [5:29:20<5:04:16, 11.48s/it] 36%|███▋ | 911/2500 [5:29:31<4:59:06, 11.29s/it] {'loss': 0.0002, 'grad_norm': 0.18268535193920468, 'learning_rate': 6.356e-07, 'completion_length': 154.31250762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00556182861328125, 'epoch': 0.36} 36%|███▋ | 911/2500 [5:29:31<4:59:06, 11.29s/it] 36%|███▋ | 912/2500 [5:29:41<4:48:13, 10.89s/it] {'loss': 0.0002, 'grad_norm': 0.8137657936242009, 'learning_rate': 6.352e-07, 'completion_length': 148.67857360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0050048828125, 'epoch': 0.36} 36%|███▋ | 912/2500 [5:29:41<4:48:13, 10.89s/it] 37%|███▋ | 913/2500 [5:29:52<4:49:42, 10.95s/it] {'loss': 0.0003, 'grad_norm': 0.34343019533030805, 'learning_rate': 6.348e-07, 'completion_length': 156.00000762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006927490234375, 'epoch': 0.37} 37%|███▋ | 913/2500 [5:29:52<4:49:42, 10.95s/it] 37%|███▋ | 914/2500 [5:30:03<4:49:35, 10.96s/it] {'loss': 0.0003, 'grad_norm': 0.21552116516022723, 'learning_rate': 6.343999999999999e-07, 'completion_length': 163.5089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0063629150390625, 'epoch': 0.37} 37%|███▋ | 914/2500 [5:30:03<4:49:35, 10.96s/it] 37%|███▋ | 915/2500 [5:30:13<4:42:41, 10.70s/it] {'loss': 0.0003, 'grad_norm': 0.3404235069106375, 'learning_rate': 6.34e-07, 'completion_length': 153.00000762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0064239501953125, 'epoch': 0.37} 37%|███▋ | 915/2500 [5:30:13<4:42:41, 10.70s/it] 37%|███▋ | 916/2500 [5:30:24<4:45:52, 10.83s/it] {'loss': 0.0003, 'grad_norm': 0.252323014245711, 'learning_rate': 6.336000000000001e-07, 'completion_length': 159.7589340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831882357597351, 'kl': 0.0066680908203125, 'epoch': 0.37} 37%|███▋ | 916/2500 [5:30:24<4:45:52, 10.83s/it] 37%|███▋ | 917/2500 [5:30:35<4:43:45, 10.76s/it] {'loss': 0.0002, 'grad_norm': 0.044066040215571506, 'learning_rate': 6.331999999999999e-07, 'completion_length': 141.24107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.37} 37%|███▋ | 917/2500 [5:30:35<4:43:45, 10.76s/it] 37%|███▋ | 918/2500 [5:30:47<4:51:44, 11.06s/it] {'loss': 0.0002, 'grad_norm': 0.5094221991215572, 'learning_rate': 6.328e-07, 'completion_length': 161.40178680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.006195068359375, 'epoch': 0.37} 37%|███▋ | 918/2500 [5:30:47<4:51:44, 11.06s/it] 37%|███▋ | 919/2500 [5:30:57<4:46:14, 10.86s/it] {'loss': 0.0003, 'grad_norm': 0.5710094158785398, 'learning_rate': 6.324e-07, 'completion_length': 153.2678680419922, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0068817138671875, 'epoch': 0.37} 37%|███▋ | 919/2500 [5:30:57<4:46:14, 10.86s/it] 37%|███▋ | 920/2500 [5:31:07<4:42:05, 10.71s/it] {'loss': 0.0002, 'grad_norm': 1.0998898549377651, 'learning_rate': 6.319999999999999e-07, 'completion_length': 157.23214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052642822265625, 'epoch': 0.37} 37%|███▋ | 920/2500 [5:31:07<4:42:05, 10.71s/it] 37%|███▋ | 921/2500 [5:31:18<4:40:46, 10.67s/it] {'loss': 0.0003, 'grad_norm': 0.2547591766748892, 'learning_rate': 6.316e-07, 'completion_length': 159.58929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0071868896484375, 'epoch': 0.37} 37%|███▋ | 921/2500 [5:31:18<4:40:46, 10.67s/it] 37%|███▋ | 922/2500 [5:31:28<4:36:58, 10.53s/it] {'loss': 0.0003, 'grad_norm': 0.028736683750608196, 'learning_rate': 6.312e-07, 'completion_length': 144.62500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064849853515625, 'epoch': 0.37} 37%|███▋ | 922/2500 [5:31:28<4:36:58, 10.53s/it] 37%|███▋ | 923/2500 [5:31:41<4:55:12, 11.23s/it] {'loss': 0.0004, 'grad_norm': 1.216200761852753, 'learning_rate': 6.308e-07, 'completion_length': 150.54464721679688, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0739355981349945, 'kl': 0.0096435546875, 'epoch': 0.37} 37%|███▋ | 923/2500 [5:31:41<4:55:12, 11.23s/it] 37%|███▋ | 924/2500 [5:32:04<6:30:09, 14.85s/it] {'loss': 0.0003, 'grad_norm': 0.4164611269652584, 'learning_rate': 6.303999999999999e-07, 'completion_length': 162.00000762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.08747542649507523, 'kl': 0.0085906982421875, 'epoch': 0.37} 37%|███▋ | 924/2500 [5:32:04<6:30:09, 14.85s/it] 37%|███▋ | 925/2500 [5:32:25<7:16:44, 16.64s/it] {'loss': 0.0003, 'grad_norm': 0.587956437381121, 'learning_rate': 6.3e-07, 'completion_length': 146.06250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.006805419921875, 'epoch': 0.37} 37%|███▋ | 925/2500 [5:32:25<7:16:44, 16.64s/it] 37%|███▋ | 926/2500 [5:32:49<8:15:21, 18.88s/it] {'loss': 0.0003, 'grad_norm': 0.49829519521339033, 'learning_rate': 6.296e-07, 'completion_length': 161.40179443359375, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.00830078125, 'epoch': 0.37} 37%|███▋ | 926/2500 [5:32:49<8:15:21, 18.88s/it] 37%|███▋ | 927/2500 [5:33:10<8:30:40, 19.48s/it] {'loss': 0.0003, 'grad_norm': 0.29595057380971157, 'learning_rate': 6.291999999999999e-07, 'completion_length': 152.5178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0077972412109375, 'epoch': 0.37} 37%|███▋ | 927/2500 [5:33:10<8:30:40, 19.48s/it] 37%|███▋ | 928/2500 [5:33:31<8:43:45, 19.99s/it] {'loss': 0.0003, 'grad_norm': 0.4973817720929809, 'learning_rate': 6.288e-07, 'completion_length': 151.58929443359375, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941184878349, 'kl': 0.0074310302734375, 'epoch': 0.37} 37%|███▋ | 928/2500 [5:33:31<8:43:45, 19.99s/it] 37%|███▋ | 929/2500 [5:33:52<8:48:17, 20.18s/it] {'loss': 0.0002, 'grad_norm': 0.023210527507141254, 'learning_rate': 6.283999999999999e-07, 'completion_length': 139.9464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045166015625, 'epoch': 0.37} 37%|███▋ | 929/2500 [5:33:52<8:48:17, 20.18s/it] 37%|███▋ | 930/2500 [5:34:14<9:06:12, 20.87s/it] {'loss': 0.0003, 'grad_norm': 0.40879883922479665, 'learning_rate': 6.28e-07, 'completion_length': 163.92857360839844, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.033065006136894226, 'kl': 0.008087158203125, 'epoch': 0.37} 37%|███▋ | 930/2500 [5:34:14<9:06:12, 20.87s/it] 37%|███▋ | 931/2500 [5:34:36<9:10:14, 21.04s/it] {'loss': 0.0004, 'grad_norm': 2.0238755519371288, 'learning_rate': 6.276e-07, 'completion_length': 158.93750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.0089263916015625, 'epoch': 0.37} 37%|███▋ | 931/2500 [5:34:36<9:10:14, 21.04s/it] 37%|███▋ | 932/2500 [5:34:57<9:09:09, 21.01s/it] {'loss': 0.0003, 'grad_norm': 1.0018055238955375, 'learning_rate': 6.271999999999999e-07, 'completion_length': 141.2232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00677490234375, 'epoch': 0.37} 37%|███▋ | 932/2500 [5:34:57<9:09:09, 21.01s/it] 37%|███▋ | 933/2500 [5:35:18<9:12:02, 21.14s/it] {'loss': 0.0003, 'grad_norm': 0.29389728977729834, 'learning_rate': 6.268e-07, 'completion_length': 162.24108123779297, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.00677490234375, 'epoch': 0.37} 37%|███▋ | 933/2500 [5:35:18<9:12:02, 21.14s/it] 37%|███▋ | 934/2500 [5:35:40<9:16:07, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.6707684520965768, 'learning_rate': 6.263999999999999e-07, 'completion_length': 148.8482208251953, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10040178522467613, 'kl': 0.00543212890625, 'epoch': 0.37} 37%|███▋ | 934/2500 [5:35:40<9:16:07, 21.31s/it] 37%|███▋ | 935/2500 [5:36:01<9:15:18, 21.29s/it] {'loss': 0.0002, 'grad_norm': 1.161504011026789, 'learning_rate': 6.26e-07, 'completion_length': 162.1607208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.005859375, 'epoch': 0.37} 37%|███▋ | 935/2500 [5:36:01<9:15:18, 21.29s/it] 37%|███▋ | 936/2500 [5:36:22<9:14:20, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.7913322726844554, 'learning_rate': 6.256e-07, 'completion_length': 158.9553680419922, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.14969705045223236, 'kl': 0.0080413818359375, 'epoch': 0.37} 37%|███▋ | 936/2500 [5:36:22<9:14:20, 21.27s/it] 37%|███▋ | 937/2500 [5:36:45<9:21:32, 21.56s/it] {'loss': 0.0003, 'grad_norm': 0.21299169954337926, 'learning_rate': 6.252e-07, 'completion_length': 150.6696548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064849853515625, 'epoch': 0.37} 37%|███▋ | 937/2500 [5:36:45<9:21:32, 21.56s/it] 38%|███▊ | 938/2500 [5:37:05<9:14:21, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.7889742117738975, 'learning_rate': 6.248e-07, 'completion_length': 137.5357208251953, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004608154296875, 'epoch': 0.38} 38%|███▊ | 938/2500 [5:37:05<9:14:21, 21.29s/it] 38%|███▊ | 939/2500 [5:37:27<9:17:54, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.5920662840220432, 'learning_rate': 6.243999999999999e-07, 'completion_length': 160.58929443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.14579425752162933, 'kl': 0.0059051513671875, 'epoch': 0.38} 38%|███▊ | 939/2500 [5:37:27<9:17:54, 21.44s/it] 38%|███▊ | 940/2500 [5:37:50<9:32:25, 22.02s/it] {'loss': 0.0003, 'grad_norm': 0.6173142069636544, 'learning_rate': 6.24e-07, 'completion_length': 164.69644165039062, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12175281345844269, 'kl': 0.0064544677734375, 'epoch': 0.38} 38%|███▊ | 940/2500 [5:37:50<9:32:25, 22.02s/it] 38%|███▊ | 941/2500 [5:38:12<9:28:07, 21.86s/it] {'loss': 0.0003, 'grad_norm': 2.0276760131241915, 'learning_rate': 6.236e-07, 'completion_length': 156.3214340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.0065460205078125, 'epoch': 0.38} 38%|███▊ | 941/2500 [5:38:12<9:28:07, 21.86s/it] 38%|███▊ | 942/2500 [5:38:33<9:25:04, 21.76s/it] {'loss': 0.0002, 'grad_norm': 0.4712455372243932, 'learning_rate': 6.231999999999999e-07, 'completion_length': 150.90179443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0055999755859375, 'epoch': 0.38} 38%|███▊ | 942/2500 [5:38:33<9:25:04, 21.76s/it] 38%|███▊ | 943/2500 [5:38:55<9:25:02, 21.77s/it] {'loss': 0.0003, 'grad_norm': 0.39949341626011897, 'learning_rate': 6.228e-07, 'completion_length': 159.1339340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.11272924765944481, 'kl': 0.007568359375, 'epoch': 0.38} 38%|███▊ | 943/2500 [5:38:55<9:25:02, 21.77s/it] 38%|███▊ | 944/2500 [5:39:16<9:19:43, 21.58s/it] {'loss': 0.0003, 'grad_norm': 0.18190818400926143, 'learning_rate': 6.224e-07, 'completion_length': 156.2232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0063629150390625, 'epoch': 0.38} 38%|███▊ | 944/2500 [5:39:16<9:19:43, 21.58s/it] 38%|███▊ | 945/2500 [5:39:38<9:17:53, 21.53s/it] {'loss': 0.0002, 'grad_norm': 0.19053997648028728, 'learning_rate': 6.219999999999999e-07, 'completion_length': 140.27679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0051116943359375, 'epoch': 0.38} 38%|███▊ | 945/2500 [5:39:38<9:17:53, 21.53s/it] 38%|███▊ | 946/2500 [5:39:59<9:17:31, 21.53s/it] {'loss': 0.0002, 'grad_norm': 0.43457610352075426, 'learning_rate': 6.216e-07, 'completion_length': 146.3214340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0062255859375, 'epoch': 0.38} 38%|███▊ | 946/2500 [5:39:59<9:17:31, 21.53s/it] 38%|███▊ | 947/2500 [5:40:22<9:24:12, 21.80s/it] {'loss': 0.0003, 'grad_norm': 0.9796368674965262, 'learning_rate': 6.212e-07, 'completion_length': 169.2678680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.12054044008255005, 'kl': 0.0081787109375, 'epoch': 0.38} 38%|███▊ | 947/2500 [5:40:22<9:24:12, 21.80s/it] 38%|███▊ | 948/2500 [5:40:43<9:20:27, 21.67s/it] {'loss': 0.0003, 'grad_norm': 0.3060540313101837, 'learning_rate': 6.208e-07, 'completion_length': 161.83929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006805419921875, 'epoch': 0.38} 38%|███▊ | 948/2500 [5:40:43<9:20:27, 21.67s/it] 38%|███▊ | 949/2500 [5:41:04<9:12:11, 21.36s/it] {'loss': 0.0002, 'grad_norm': 3.0604513157785727, 'learning_rate': 6.203999999999999e-07, 'completion_length': 147.5357208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004852294921875, 'epoch': 0.38} 38%|███▊ | 949/2500 [5:41:04<9:12:11, 21.36s/it] 38%|███▊ | 950/2500 [5:41:25<9:13:12, 21.41s/it] {'loss': 0.0003, 'grad_norm': 0.06041894364877922, 'learning_rate': 6.2e-07, 'completion_length': 153.69644165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00701904296875, 'epoch': 0.38} 38%|███▊ | 950/2500 [5:41:25<9:13:12, 21.41s/it] 38%|███▊ | 951/2500 [5:41:46<9:06:42, 21.18s/it] {'loss': 0.0004, 'grad_norm': 1.0020614446801825, 'learning_rate': 6.196e-07, 'completion_length': 157.36608123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.0964989960193634, 'kl': 0.00921630859375, 'epoch': 0.38} 38%|███▊ | 951/2500 [5:41:46<9:06:42, 21.18s/it] 38%|███▊ | 952/2500 [5:42:07<9:02:32, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.41971632391612645, 'learning_rate': 6.191999999999999e-07, 'completion_length': 156.48214721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941557407379, 'kl': 0.007659912109375, 'epoch': 0.38} 38%|███▊ | 952/2500 [5:42:07<9:02:32, 21.03s/it] 38%|███▊ | 953/2500 [5:42:27<8:58:04, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.9578010883140455, 'learning_rate': 6.188e-07, 'completion_length': 142.3928680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06222161278128624, 'kl': 0.005828857421875, 'epoch': 0.38} 38%|███▊ | 953/2500 [5:42:27<8:58:04, 20.87s/it] 38%|███▊ | 954/2500 [5:42:49<9:07:13, 21.24s/it] {'loss': 0.0003, 'grad_norm': 1.2102151212172771, 'learning_rate': 6.183999999999999e-07, 'completion_length': 153.00894165039062, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.07576144114136696, 'kl': 0.006744384765625, 'epoch': 0.38} 38%|███▊ | 954/2500 [5:42:49<9:07:13, 21.24s/it] 38%|███▊ | 955/2500 [5:43:09<8:55:11, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.02112529040192053, 'learning_rate': 6.18e-07, 'completion_length': 125.74107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045318603515625, 'epoch': 0.38} 38%|███▊ | 955/2500 [5:43:09<8:55:11, 20.78s/it] 38%|███▊ | 956/2500 [5:43:30<8:59:00, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.4168028799486144, 'learning_rate': 6.176e-07, 'completion_length': 137.64286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0048370361328125, 'epoch': 0.38} 38%|███▊ | 956/2500 [5:43:30<8:59:00, 20.95s/it] 38%|███▊ | 957/2500 [5:43:52<9:04:47, 21.18s/it] {'loss': 0.0003, 'grad_norm': 0.42293326586642843, 'learning_rate': 6.172e-07, 'completion_length': 164.62500762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0075225830078125, 'epoch': 0.38} 38%|███▊ | 957/2500 [5:43:52<9:04:47, 21.18s/it] 38%|███▊ | 958/2500 [5:44:12<8:57:05, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.4012478184075003, 'learning_rate': 6.168e-07, 'completion_length': 149.01786041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0057220458984375, 'epoch': 0.38} 38%|███▊ | 958/2500 [5:44:12<8:57:05, 20.90s/it] 38%|███▊ | 959/2500 [5:44:33<8:59:24, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.32967147465484026, 'learning_rate': 6.163999999999999e-07, 'completion_length': 163.2857208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007232666015625, 'epoch': 0.38} 38%|███▊ | 959/2500 [5:44:33<8:59:24, 21.00s/it] 38%|███▊ | 960/2500 [5:44:54<8:57:45, 20.95s/it] {'loss': 0.0003, 'grad_norm': 0.48337695796890223, 'learning_rate': 6.16e-07, 'completion_length': 145.9464340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0086669921875, 'epoch': 0.38} 38%|███▊ | 960/2500 [5:44:54<8:57:45, 20.95s/it] 38%|███▊ | 961/2500 [5:45:15<8:52:41, 20.77s/it] {'loss': 0.0004, 'grad_norm': 1.0436614928123107, 'learning_rate': 6.156e-07, 'completion_length': 155.75000762939453, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.15360544621944427, 'kl': 0.01043701171875, 'epoch': 0.38} 38%|███▊ | 961/2500 [5:45:15<8:52:41, 20.77s/it] 38%|███▊ | 962/2500 [5:45:35<8:50:46, 20.71s/it] {'loss': 0.0003, 'grad_norm': 0.5573227200258182, 'learning_rate': 6.152e-07, 'completion_length': 140.6696548461914, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.00836181640625, 'epoch': 0.38} 38%|███▊ | 962/2500 [5:45:35<8:50:46, 20.71s/it] 39%|███▊ | 963/2500 [5:45:56<8:49:44, 20.68s/it] {'loss': 0.0003, 'grad_norm': 0.7848712280795002, 'learning_rate': 6.148e-07, 'completion_length': 145.76786041259766, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.09138382598757744, 'kl': 0.0066375732421875, 'epoch': 0.39} 39%|███▊ | 963/2500 [5:45:56<8:49:44, 20.68s/it] 39%|███▊ | 964/2500 [5:46:17<8:53:11, 20.83s/it] {'loss': 0.0003, 'grad_norm': 0.35681268517227877, 'learning_rate': 6.143999999999999e-07, 'completion_length': 167.2857208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.0077667236328125, 'epoch': 0.39} 39%|███▊ | 964/2500 [5:46:17<8:53:11, 20.83s/it] 39%|███▊ | 965/2500 [5:46:38<8:52:05, 20.80s/it] {'loss': 0.0003, 'grad_norm': 0.5786550955501314, 'learning_rate': 6.14e-07, 'completion_length': 154.33036041259766, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.00628662109375, 'epoch': 0.39} 39%|███▊ | 965/2500 [5:46:38<8:52:05, 20.80s/it] 39%|███▊ | 966/2500 [5:47:00<9:07:06, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.32272873661652657, 'learning_rate': 6.136e-07, 'completion_length': 151.4464340209961, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00787353515625, 'epoch': 0.39} 39%|███▊ | 966/2500 [5:47:00<9:07:06, 21.40s/it] 39%|███▊ | 967/2500 [5:47:22<9:07:51, 21.44s/it] {'loss': 0.0003, 'grad_norm': 1.2561707248505125, 'learning_rate': 6.131999999999999e-07, 'completion_length': 162.4464340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.00701904296875, 'epoch': 0.39} 39%|███▊ | 967/2500 [5:47:22<9:07:51, 21.44s/it] 39%|███▊ | 968/2500 [5:47:43<9:02:00, 21.23s/it] {'loss': 0.0003, 'grad_norm': 0.3846183876643441, 'learning_rate': 6.128e-07, 'completion_length': 154.29464721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00872802734375, 'epoch': 0.39} 39%|███▊ | 968/2500 [5:47:43<9:02:00, 21.23s/it] 39%|███▉ | 969/2500 [5:48:04<8:58:18, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.21453332475946438, 'learning_rate': 6.124000000000001e-07, 'completion_length': 149.77679443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00750732421875, 'epoch': 0.39} 39%|███▉ | 969/2500 [5:48:04<8:58:18, 21.10s/it] 39%|███▉ | 970/2500 [5:48:24<8:56:54, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.28400389917482133, 'learning_rate': 6.119999999999999e-07, 'completion_length': 162.2232208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00836181640625, 'epoch': 0.39} 39%|███▉ | 970/2500 [5:48:24<8:56:54, 21.06s/it] 39%|███▉ | 971/2500 [5:48:45<8:53:39, 20.94s/it] {'loss': 0.0003, 'grad_norm': 0.5966502668941783, 'learning_rate': 6.116e-07, 'completion_length': 150.35714721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.007659912109375, 'epoch': 0.39} 39%|███▉ | 971/2500 [5:48:45<8:53:39, 20.94s/it] 39%|███▉ | 972/2500 [5:49:05<8:48:17, 20.74s/it] {'loss': 0.0002, 'grad_norm': 0.4829605171702321, 'learning_rate': 6.112e-07, 'completion_length': 142.77678680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0050506591796875, 'epoch': 0.39} 39%|███▉ | 972/2500 [5:49:05<8:48:17, 20.74s/it] 39%|███▉ | 973/2500 [5:49:26<8:49:48, 20.82s/it] {'loss': 0.0003, 'grad_norm': 0.28623742106801775, 'learning_rate': 6.107999999999999e-07, 'completion_length': 149.23214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00665283203125, 'epoch': 0.39} 39%|███▉ | 973/2500 [5:49:26<8:49:48, 20.82s/it] 39%|███▉ | 974/2500 [5:49:47<8:49:23, 20.82s/it] {'loss': 0.0002, 'grad_norm': 0.5477872058854474, 'learning_rate': 6.104e-07, 'completion_length': 159.07144165039062, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0054931640625, 'epoch': 0.39} 39%|███▉ | 974/2500 [5:49:47<8:49:23, 20.82s/it] 39%|███▉ | 975/2500 [5:50:08<8:45:23, 20.67s/it] {'loss': 0.0002, 'grad_norm': 0.2396619251259025, 'learning_rate': 6.1e-07, 'completion_length': 143.33929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0051116943359375, 'epoch': 0.39} 39%|███▉ | 975/2500 [5:50:08<8:45:23, 20.67s/it] 39%|███▉ | 976/2500 [5:50:29<8:47:39, 20.77s/it] {'loss': 0.0002, 'grad_norm': 0.4698470187020569, 'learning_rate': 6.096e-07, 'completion_length': 151.2053680419922, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00559234619140625, 'epoch': 0.39} 39%|███▉ | 976/2500 [5:50:29<8:47:39, 20.77s/it] 39%|███▉ | 977/2500 [5:50:49<8:44:44, 20.67s/it] {'loss': 0.0003, 'grad_norm': 0.4244346030157636, 'learning_rate': 6.091999999999999e-07, 'completion_length': 140.0982208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.008514404296875, 'epoch': 0.39} 39%|███▉ | 977/2500 [5:50:49<8:44:44, 20.67s/it] 39%|███▉ | 978/2500 [5:51:10<8:47:37, 20.80s/it] {'loss': 0.0003, 'grad_norm': 0.028648297680590313, 'learning_rate': 6.088e-07, 'completion_length': 169.33929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0079193115234375, 'epoch': 0.39} 39%|███▉ | 978/2500 [5:51:10<8:47:37, 20.80s/it] 39%|███▉ | 979/2500 [5:51:31<8:48:41, 20.86s/it] {'loss': 0.0003, 'grad_norm': 0.6868470439744975, 'learning_rate': 6.084000000000001e-07, 'completion_length': 137.67858123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.006988525390625, 'epoch': 0.39} 39%|███▉ | 979/2500 [5:51:31<8:48:41, 20.86s/it] 39%|███▉ | 980/2500 [5:51:52<8:48:29, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.262886973980734, 'learning_rate': 6.079999999999999e-07, 'completion_length': 155.2232208251953, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0062408447265625, 'epoch': 0.39} 39%|███▉ | 980/2500 [5:51:52<8:48:29, 20.86s/it] 39%|███▉ | 981/2500 [5:52:12<8:42:34, 20.64s/it] {'loss': 0.0002, 'grad_norm': 0.025410007291150182, 'learning_rate': 6.076e-07, 'completion_length': 144.76786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00604248046875, 'epoch': 0.39} 39%|███▉ | 981/2500 [5:52:12<8:42:34, 20.64s/it] 39%|███▉ | 982/2500 [5:52:33<8:40:26, 20.57s/it] {'loss': 0.0003, 'grad_norm': 0.43391700201844313, 'learning_rate': 6.072e-07, 'completion_length': 144.875, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0065765380859375, 'epoch': 0.39} 39%|███▉ | 982/2500 [5:52:33<8:40:26, 20.57s/it] 39%|███▉ | 983/2500 [5:52:54<8:48:10, 20.89s/it] {'loss': 0.0003, 'grad_norm': 0.024352350177524885, 'learning_rate': 6.068e-07, 'completion_length': 146.58929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0067901611328125, 'epoch': 0.39} 39%|███▉ | 983/2500 [5:52:54<8:48:10, 20.89s/it] 39%|███▉ | 984/2500 [5:53:15<8:45:03, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.35576499514871646, 'learning_rate': 6.064e-07, 'completion_length': 152.89286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0059356689453125, 'epoch': 0.39} 39%|███▉ | 984/2500 [5:53:15<8:45:03, 20.78s/it] 39%|███▉ | 985/2500 [5:53:36<8:48:30, 20.93s/it] {'loss': 0.0003, 'grad_norm': 0.5134261711365273, 'learning_rate': 6.06e-07, 'completion_length': 156.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0073394775390625, 'epoch': 0.39} 39%|███▉ | 985/2500 [5:53:36<8:48:30, 20.93s/it] 39%|███▉ | 986/2500 [5:53:57<8:48:32, 20.95s/it] {'loss': 0.0003, 'grad_norm': 0.28332572558980923, 'learning_rate': 6.056e-07, 'completion_length': 159.95536041259766, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0063934326171875, 'epoch': 0.39} 39%|███▉ | 986/2500 [5:53:57<8:48:32, 20.95s/it] 39%|███▉ | 987/2500 [5:54:18<8:47:48, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.028922764402627338, 'learning_rate': 6.051999999999999e-07, 'completion_length': 154.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0061798095703125, 'epoch': 0.39} 39%|███▉ | 987/2500 [5:54:18<8:47:48, 20.93s/it] 40%|███▉ | 988/2500 [5:54:39<8:48:50, 20.99s/it] {'loss': 0.0003, 'grad_norm': 0.3012571028637129, 'learning_rate': 6.048e-07, 'completion_length': 162.0803680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0079193115234375, 'epoch': 0.4} 40%|███▉ | 988/2500 [5:54:39<8:48:50, 20.99s/it] 40%|███▉ | 989/2500 [5:55:00<8:48:49, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.6257223072383521, 'learning_rate': 6.044e-07, 'completion_length': 161.0446548461914, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.08747543022036552, 'kl': 0.0079803466796875, 'epoch': 0.4} 40%|███▉ | 989/2500 [5:55:00<8:48:49, 21.00s/it] 40%|███▉ | 990/2500 [5:55:21<8:49:17, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.029765872583380114, 'learning_rate': 6.04e-07, 'completion_length': 148.8571548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006195068359375, 'epoch': 0.4} 40%|███▉ | 990/2500 [5:55:21<8:49:17, 21.03s/it] 40%|███▉ | 991/2500 [5:55:44<9:05:26, 21.69s/it] {'loss': 0.0003, 'grad_norm': 0.7005414128090124, 'learning_rate': 6.036e-07, 'completion_length': 154.40178680419922, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0074005126953125, 'epoch': 0.4} 40%|███▉ | 991/2500 [5:55:44<9:05:26, 21.69s/it] 40%|███▉ | 992/2500 [5:56:05<8:56:34, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.6223682434696589, 'learning_rate': 6.031999999999999e-07, 'completion_length': 152.6964340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.0076446533203125, 'epoch': 0.4} 40%|███▉ | 992/2500 [5:56:05<8:56:34, 21.35s/it] 40%|███▉ | 993/2500 [5:56:26<8:57:16, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.2808796205552174, 'learning_rate': 6.028e-07, 'completion_length': 150.12500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0072021484375, 'epoch': 0.4} 40%|███▉ | 993/2500 [5:56:26<8:57:16, 21.39s/it] 40%|███▉ | 994/2500 [5:56:47<8:54:34, 21.30s/it] {'loss': 0.0003, 'grad_norm': 1.3174739340272321, 'learning_rate': 6.024e-07, 'completion_length': 165.4732208251953, 'rewards/accuracy_reward': 0.8303571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8303572535514832, 'reward_std': 0.15872061252593994, 'kl': 0.006988525390625, 'epoch': 0.4} 40%|███▉ | 994/2500 [5:56:47<8:54:34, 21.30s/it] 40%|███▉ | 995/2500 [5:57:09<8:53:58, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.41418407541867736, 'learning_rate': 6.019999999999999e-07, 'completion_length': 161.7053680419922, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.006103515625, 'epoch': 0.4} 40%|███▉ | 995/2500 [5:57:09<8:53:58, 21.29s/it] 40%|███▉ | 996/2500 [5:57:30<8:50:08, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.024156575317900872, 'learning_rate': 6.016e-07, 'completion_length': 146.4732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006072998046875, 'epoch': 0.4} 40%|███▉ | 996/2500 [5:57:30<8:50:08, 21.15s/it] 40%|███▉ | 997/2500 [5:57:50<8:47:21, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.03887934177336862, 'learning_rate': 6.012e-07, 'completion_length': 161.33929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0078582763671875, 'epoch': 0.4} 40%|███▉ | 997/2500 [5:57:50<8:47:21, 21.05s/it] 40%|███▉ | 998/2500 [5:58:11<8:45:16, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.28599108314627075, 'learning_rate': 6.007999999999999e-07, 'completion_length': 149.86608123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00485992431640625, 'epoch': 0.4} 40%|███▉ | 998/2500 [5:58:11<8:45:16, 20.98s/it] 40%|███▉ | 999/2500 [5:58:32<8:42:36, 20.89s/it] {'loss': 0.0003, 'grad_norm': 0.4792927135478493, 'learning_rate': 6.004e-07, 'completion_length': 155.0357208251953, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0078582763671875, 'epoch': 0.4} 40%|███▉ | 999/2500 [5:58:32<8:42:36, 20.89s/it] 40%|████ | 1000/2500 [5:58:54<8:48:52, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.36646107635198516, 'learning_rate': 6e-07, 'completion_length': 155.79464721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0086212158203125, 'epoch': 0.4} 40%|████ | 1000/2500 [5:58:54<8:48:52, 21.15s/it] 40%|████ | 1001/2500 [6:00:02<14:40:14, 35.23s/it] {'loss': 0.0004, 'grad_norm': 0.4054438790223726, 'learning_rate': 5.995999999999999e-07, 'completion_length': 160.91964721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.01092529296875, 'epoch': 0.4} 40%|████ | 1001/2500 [6:00:02<14:40:14, 35.23s/it] 40%|████ | 1002/2500 [6:00:22<12:51:25, 30.90s/it] {'loss': 0.0003, 'grad_norm': 0.47757208404240614, 'learning_rate': 5.991999999999999e-07, 'completion_length': 158.27679443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.007110595703125, 'epoch': 0.4} 40%|████ | 1002/2500 [6:00:22<12:51:25, 30.90s/it] 40%|████ | 1003/2500 [6:00:43<11:35:33, 27.88s/it] {'loss': 0.0003, 'grad_norm': 0.3660076279899779, 'learning_rate': 5.988e-07, 'completion_length': 149.00000762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0064697265625, 'epoch': 0.4} 40%|████ | 1003/2500 [6:00:43<11:35:33, 27.88s/it] 40%|████ | 1004/2500 [6:01:04<10:40:30, 25.69s/it] {'loss': 0.0003, 'grad_norm': 0.8231330472636211, 'learning_rate': 5.984000000000001e-07, 'completion_length': 158.92858123779297, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.08868780732154846, 'kl': 0.007415771484375, 'epoch': 0.4} 40%|████ | 1004/2500 [6:01:04<10:40:30, 25.69s/it] 40%|████ | 1005/2500 [6:01:25<10:06:22, 24.34s/it] {'loss': 0.0002, 'grad_norm': 0.2101569796478324, 'learning_rate': 5.979999999999999e-07, 'completion_length': 151.79464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00439453125, 'epoch': 0.4} 40%|████ | 1005/2500 [6:01:25<10:06:22, 24.34s/it] 40%|████ | 1006/2500 [6:01:46<9:42:05, 23.38s/it] {'loss': 0.0003, 'grad_norm': 0.32211892375008105, 'learning_rate': 5.976e-07, 'completion_length': 161.8928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0068206787109375, 'epoch': 0.4} 40%|████ | 1006/2500 [6:01:46<9:42:05, 23.38s/it] 40%|████ | 1007/2500 [6:02:07<9:20:03, 22.51s/it] {'loss': 0.0002, 'grad_norm': 0.025457958213903342, 'learning_rate': 5.972e-07, 'completion_length': 148.9196548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00616455078125, 'epoch': 0.4} 40%|████ | 1007/2500 [6:02:07<9:20:03, 22.51s/it] 40%|████ | 1008/2500 [6:02:28<9:08:54, 22.07s/it] {'loss': 0.0004, 'grad_norm': 0.45279496293548477, 'learning_rate': 5.967999999999999e-07, 'completion_length': 150.58929443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.08868780359625816, 'kl': 0.00927734375, 'epoch': 0.4} 40%|████ | 1008/2500 [6:02:28<9:08:54, 22.07s/it] 40%|████ | 1009/2500 [6:02:49<9:04:39, 21.92s/it] {'loss': 0.0003, 'grad_norm': 0.2659216027995951, 'learning_rate': 5.964e-07, 'completion_length': 164.4732208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.06222161278128624, 'kl': 0.00738525390625, 'epoch': 0.4} 40%|████ | 1009/2500 [6:02:49<9:04:39, 21.92s/it] 40%|████ | 1010/2500 [6:03:10<8:55:07, 21.55s/it] {'loss': 0.0002, 'grad_norm': 2.9879224220920575, 'learning_rate': 5.96e-07, 'completion_length': 141.1339340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0056915283203125, 'epoch': 0.4} 40%|████ | 1010/2500 [6:03:10<8:55:07, 21.55s/it] 40%|████ | 1011/2500 [6:03:30<8:45:31, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.528972026537916, 'learning_rate': 5.956e-07, 'completion_length': 146.4464340209961, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747543022036552, 'kl': 0.00516510009765625, 'epoch': 0.4} 40%|████ | 1011/2500 [6:03:30<8:45:31, 21.18s/it] 40%|████ | 1012/2500 [6:03:51<8:40:52, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.15393561734936062, 'learning_rate': 5.951999999999999e-07, 'completion_length': 156.83929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064544677734375, 'epoch': 0.4} 40%|████ | 1012/2500 [6:03:51<8:40:52, 21.00s/it] 41%|████ | 1013/2500 [6:04:12<8:40:02, 20.98s/it] {'loss': 0.0003, 'grad_norm': 0.2594418453989247, 'learning_rate': 5.948e-07, 'completion_length': 156.61607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006317138671875, 'epoch': 0.41} 41%|████ | 1013/2500 [6:04:12<8:40:02, 20.98s/it] 41%|████ | 1014/2500 [6:04:33<8:39:17, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.3848813544975081, 'learning_rate': 5.944e-07, 'completion_length': 158.5357208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.00567626953125, 'epoch': 0.41} 41%|████ | 1014/2500 [6:04:33<8:39:17, 20.97s/it] 41%|████ | 1015/2500 [6:04:55<8:52:04, 21.50s/it] {'loss': 0.0002, 'grad_norm': 0.3100299216289957, 'learning_rate': 5.939999999999999e-07, 'completion_length': 149.11607360839844, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0055694580078125, 'epoch': 0.41} 41%|████ | 1015/2500 [6:04:56<8:52:04, 21.50s/it] 41%|████ | 1016/2500 [6:05:16<8:46:54, 21.30s/it] {'loss': 0.0003, 'grad_norm': 0.3902182641802743, 'learning_rate': 5.936e-07, 'completion_length': 148.77679443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.07003280520439148, 'kl': 0.0070037841796875, 'epoch': 0.41} 41%|████ | 1016/2500 [6:05:16<8:46:54, 21.30s/it] 41%|████ | 1017/2500 [6:05:38<8:47:28, 21.34s/it] {'loss': 0.0003, 'grad_norm': 1.0961341514925023, 'learning_rate': 5.931999999999999e-07, 'completion_length': 170.91072845458984, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.11394162103533745, 'kl': 0.0084381103515625, 'epoch': 0.41} 41%|████ | 1017/2500 [6:05:38<8:47:28, 21.34s/it] 41%|████ | 1018/2500 [6:05:59<8:46:59, 21.34s/it] {'loss': 0.0003, 'grad_norm': 1.1197543766642795, 'learning_rate': 5.928e-07, 'completion_length': 159.40178680419922, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0075836181640625, 'epoch': 0.41} 41%|████ | 1018/2500 [6:05:59<8:46:59, 21.34s/it] 41%|████ | 1019/2500 [6:06:20<8:45:46, 21.30s/it] {'loss': 0.0003, 'grad_norm': 1.7919658512238612, 'learning_rate': 5.924e-07, 'completion_length': 168.42858123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0739355981349945, 'kl': 0.0074005126953125, 'epoch': 0.41} 41%|████ | 1019/2500 [6:06:20<8:45:46, 21.30s/it] 41%|████ | 1020/2500 [6:06:41<8:42:04, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.22948115008801753, 'learning_rate': 5.919999999999999e-07, 'completion_length': 163.75000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00811767578125, 'epoch': 0.41} 41%|████ | 1020/2500 [6:06:41<8:42:04, 21.17s/it] 41%|████ | 1021/2500 [6:07:02<8:36:42, 20.96s/it] {'loss': 0.0003, 'grad_norm': 1.315000323850719, 'learning_rate': 5.916e-07, 'completion_length': 144.9107208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0081634521484375, 'epoch': 0.41} 41%|████ | 1021/2500 [6:07:02<8:36:42, 20.96s/it] 41%|████ | 1022/2500 [6:07:23<8:38:40, 21.06s/it] {'loss': 0.0003, 'grad_norm': 1.916182223029056, 'learning_rate': 5.911999999999999e-07, 'completion_length': 155.1964340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.007537841796875, 'epoch': 0.41} 41%|████ | 1022/2500 [6:07:23<8:38:40, 21.06s/it] 41%|████ | 1023/2500 [6:07:44<8:37:14, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.015307330570940636, 'learning_rate': 5.907999999999999e-07, 'completion_length': 148.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00431060791015625, 'epoch': 0.41} 41%|████ | 1023/2500 [6:07:44<8:37:14, 21.01s/it] 41%|████ | 1024/2500 [6:08:05<8:40:55, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.32120595660047385, 'learning_rate': 5.904e-07, 'completion_length': 150.54464721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0061798095703125, 'epoch': 0.41} 41%|████ | 1024/2500 [6:08:05<8:40:55, 21.18s/it] 41%|████ | 1025/2500 [6:08:26<8:39:55, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.7843840060137622, 'learning_rate': 5.9e-07, 'completion_length': 160.7232208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.00836181640625, 'epoch': 0.41} 41%|████ | 1025/2500 [6:08:26<8:39:55, 21.15s/it] 41%|████ | 1026/2500 [6:08:48<8:38:55, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.1973462408077922, 'learning_rate': 5.896e-07, 'completion_length': 147.52679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00628662109375, 'epoch': 0.41} 41%|████ | 1026/2500 [6:08:48<8:38:55, 21.12s/it] 41%|████ | 1027/2500 [6:09:09<8:38:03, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.03466893059066821, 'learning_rate': 5.891999999999999e-07, 'completion_length': 140.79464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00634765625, 'epoch': 0.41} 41%|████ | 1027/2500 [6:09:09<8:38:03, 21.10s/it] 41%|████ | 1028/2500 [6:09:30<8:36:24, 21.05s/it] {'loss': 0.0004, 'grad_norm': 0.2911470346335143, 'learning_rate': 5.888e-07, 'completion_length': 148.30357360839844, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.033065006136894226, 'kl': 0.009063720703125, 'epoch': 0.41} 41%|████ | 1028/2500 [6:09:30<8:36:24, 21.05s/it] 41%|████ | 1029/2500 [6:09:50<8:32:50, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.8297205147426292, 'learning_rate': 5.884000000000001e-07, 'completion_length': 141.08929443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.0058441162109375, 'epoch': 0.41} 41%|████ | 1029/2500 [6:09:50<8:32:50, 20.92s/it] 41%|████ | 1030/2500 [6:10:11<8:31:19, 20.87s/it] {'loss': 0.0003, 'grad_norm': 0.30055345848380743, 'learning_rate': 5.879999999999999e-07, 'completion_length': 157.98214721679688, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.007659912109375, 'epoch': 0.41} 41%|████ | 1030/2500 [6:10:11<8:31:19, 20.87s/it] 41%|████ | 1031/2500 [6:10:32<8:31:14, 20.88s/it] {'loss': 0.0003, 'grad_norm': 0.386384706352307, 'learning_rate': 5.876e-07, 'completion_length': 165.7232208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.008026123046875, 'epoch': 0.41} 41%|████ | 1031/2500 [6:10:32<8:31:14, 20.88s/it] 41%|████▏ | 1032/2500 [6:10:53<8:32:37, 20.95s/it] {'loss': 0.0002, 'grad_norm': 1.0171470836659169, 'learning_rate': 5.872000000000001e-07, 'completion_length': 158.9107208251953, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0052337646484375, 'epoch': 0.41} 41%|████▏ | 1032/2500 [6:10:53<8:32:37, 20.95s/it] 41%|████▏ | 1033/2500 [6:11:15<8:39:44, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.042004772079512843, 'learning_rate': 5.867999999999999e-07, 'completion_length': 139.93750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004425048828125, 'epoch': 0.41} 41%|████▏ | 1033/2500 [6:11:15<8:39:44, 21.26s/it] 41%|████▏ | 1034/2500 [6:11:36<8:36:12, 21.13s/it] {'loss': 0.0003, 'grad_norm': 0.5650075303550712, 'learning_rate': 5.864e-07, 'completion_length': 159.25, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.007904052734375, 'epoch': 0.41} 41%|████▏ | 1034/2500 [6:11:36<8:36:12, 21.13s/it] 41%|████▏ | 1035/2500 [6:11:57<8:35:02, 21.09s/it] {'loss': 0.0005, 'grad_norm': 0.40159738701706926, 'learning_rate': 5.86e-07, 'completion_length': 156.11608123779297, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.05831882357597351, 'kl': 0.011932373046875, 'epoch': 0.41} 41%|████▏ | 1035/2500 [6:11:57<8:35:02, 21.09s/it] 41%|████▏ | 1036/2500 [6:12:18<8:34:17, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.4559549040964434, 'learning_rate': 5.856e-07, 'completion_length': 144.875, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0054779052734375, 'epoch': 0.41} 41%|████▏ | 1036/2500 [6:12:18<8:34:17, 21.08s/it] 41%|████▏ | 1037/2500 [6:12:40<8:40:36, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.6851531883742681, 'learning_rate': 5.852e-07, 'completion_length': 157.125, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.0835726335644722, 'kl': 0.007415771484375, 'epoch': 0.41} 41%|████▏ | 1037/2500 [6:12:40<8:40:36, 21.35s/it] 42%|████▏ | 1038/2500 [6:13:00<8:31:29, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.2730378911077361, 'learning_rate': 5.848e-07, 'completion_length': 139.29464721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0059967041015625, 'epoch': 0.42} 42%|████▏ | 1038/2500 [6:13:00<8:31:29, 20.99s/it] 42%|████▏ | 1039/2500 [6:13:21<8:35:10, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.21098132702530065, 'learning_rate': 5.844e-07, 'completion_length': 148.33929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064239501953125, 'epoch': 0.42} 42%|████▏ | 1039/2500 [6:13:21<8:35:10, 21.16s/it] 42%|████▏ | 1040/2500 [6:13:43<8:38:21, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.6148975036668503, 'learning_rate': 5.839999999999999e-07, 'completion_length': 151.08929443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00555419921875, 'epoch': 0.42} 42%|████▏ | 1040/2500 [6:13:43<8:38:21, 21.30s/it] 42%|████▏ | 1041/2500 [6:14:04<8:36:27, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.39491604544002273, 'learning_rate': 5.836e-07, 'completion_length': 152.86607360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0069427490234375, 'epoch': 0.42} 42%|████▏ | 1041/2500 [6:14:04<8:36:27, 21.24s/it] 42%|████▏ | 1042/2500 [6:14:25<8:34:22, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.4293586840346223, 'learning_rate': 5.832e-07, 'completion_length': 148.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005340576171875, 'epoch': 0.42} 42%|████▏ | 1042/2500 [6:14:25<8:34:22, 21.17s/it] 42%|████▏ | 1043/2500 [6:14:47<8:38:19, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.28588210344125786, 'learning_rate': 5.828e-07, 'completion_length': 165.6071548461914, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0065460205078125, 'epoch': 0.42} 42%|████▏ | 1043/2500 [6:14:47<8:38:19, 21.35s/it] 42%|████▏ | 1044/2500 [6:15:08<8:37:35, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.682316120188096, 'learning_rate': 5.824e-07, 'completion_length': 158.4821548461914, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06613001227378845, 'kl': 0.0071563720703125, 'epoch': 0.42} 42%|████▏ | 1044/2500 [6:15:08<8:37:35, 21.33s/it] 42%|████▏ | 1045/2500 [6:15:29<8:33:55, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.24855179737097882, 'learning_rate': 5.819999999999999e-07, 'completion_length': 146.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0061798095703125, 'epoch': 0.42} 42%|████▏ | 1045/2500 [6:15:29<8:33:55, 21.19s/it] 42%|████▏ | 1046/2500 [6:15:50<8:31:15, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.289608813825051, 'learning_rate': 5.816e-07, 'completion_length': 151.27679443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0078277587890625, 'epoch': 0.42} 42%|████▏ | 1046/2500 [6:15:50<8:31:15, 21.10s/it] 42%|████▏ | 1047/2500 [6:16:11<8:28:34, 21.00s/it] {'loss': 0.0002, 'grad_norm': 1.0195912263736013, 'learning_rate': 5.812e-07, 'completion_length': 139.96429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00433349609375, 'epoch': 0.42} 42%|████▏ | 1047/2500 [6:16:11<8:28:34, 21.00s/it] 42%|████▏ | 1048/2500 [6:16:31<8:23:56, 20.82s/it] {'loss': 0.0003, 'grad_norm': 0.1225927665010371, 'learning_rate': 5.807999999999999e-07, 'completion_length': 146.25000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0063629150390625, 'epoch': 0.42} 42%|████▏ | 1048/2500 [6:16:31<8:23:56, 20.82s/it] 42%|████▏ | 1049/2500 [6:16:52<8:25:52, 20.92s/it] {'loss': 0.0001, 'grad_norm': 0.014157906262585918, 'learning_rate': 5.804e-07, 'completion_length': 136.66964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00296783447265625, 'epoch': 0.42} 42%|████▏ | 1049/2500 [6:16:52<8:25:52, 20.92s/it] 42%|████▏ | 1050/2500 [6:17:13<8:23:01, 20.81s/it] {'loss': 0.0002, 'grad_norm': 0.44186320503834386, 'learning_rate': 5.8e-07, 'completion_length': 143.55358123779297, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0058135986328125, 'epoch': 0.42} 42%|████▏ | 1050/2500 [6:17:13<8:23:01, 20.81s/it] 42%|████▏ | 1051/2500 [6:17:33<8:17:21, 20.59s/it] {'loss': 0.0002, 'grad_norm': 0.4224122213628653, 'learning_rate': 5.796e-07, 'completion_length': 140.10714721679688, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0048675537109375, 'epoch': 0.42} 42%|████▏ | 1051/2500 [6:17:33<8:17:21, 20.59s/it] 42%|████▏ | 1052/2500 [6:17:54<8:23:07, 20.85s/it] {'loss': 0.0003, 'grad_norm': 4.979772482126655, 'learning_rate': 5.792e-07, 'completion_length': 152.80357360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0078887939453125, 'epoch': 0.42} 42%|████▏ | 1052/2500 [6:17:54<8:23:07, 20.85s/it] 42%|████▏ | 1053/2500 [6:18:15<8:24:24, 20.92s/it] {'loss': 0.0003, 'grad_norm': 1.4008009381036044, 'learning_rate': 5.788e-07, 'completion_length': 149.16964721679688, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.112115778028965, 'kl': 0.0080413818359375, 'epoch': 0.42} 42%|████▏ | 1053/2500 [6:18:15<8:24:24, 20.92s/it] 42%|████▏ | 1054/2500 [6:18:36<8:22:37, 20.86s/it] {'loss': 0.0005, 'grad_norm': 0.9361895248564873, 'learning_rate': 5.784e-07, 'completion_length': 150.82144165039062, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.06613001227378845, 'kl': 0.011810302734375, 'epoch': 0.42} 42%|████▏ | 1054/2500 [6:18:36<8:22:37, 20.86s/it] 42%|████▏ | 1055/2500 [6:18:57<8:23:03, 20.89s/it] {'loss': 0.0003, 'grad_norm': 0.2155987125084097, 'learning_rate': 5.779999999999999e-07, 'completion_length': 157.33929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.008392333984375, 'epoch': 0.42} 42%|████▏ | 1055/2500 [6:18:57<8:23:03, 20.89s/it] 42%|████▏ | 1056/2500 [6:19:18<8:22:11, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.19246806318927, 'learning_rate': 5.776e-07, 'completion_length': 145.49107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0047454833984375, 'epoch': 0.42} 42%|████▏ | 1056/2500 [6:19:18<8:22:11, 20.87s/it] 42%|████▏ | 1057/2500 [6:19:39<8:23:39, 20.94s/it] {'loss': 0.0004, 'grad_norm': 1.1984228305733775, 'learning_rate': 5.772000000000001e-07, 'completion_length': 163.0803680419922, 'rewards/accuracy_reward': 0.8482142984867096, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.14969704300165176, 'kl': 0.0096893310546875, 'epoch': 0.42} 42%|████▏ | 1057/2500 [6:19:39<8:23:39, 20.94s/it] 42%|████▏ | 1058/2500 [6:20:00<8:21:24, 20.86s/it] {'loss': 0.0004, 'grad_norm': 0.6551960322153831, 'learning_rate': 5.767999999999999e-07, 'completion_length': 159.8303680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.09528661891818047, 'kl': 0.00933837890625, 'epoch': 0.42} 42%|████▏ | 1058/2500 [6:20:00<8:21:24, 20.86s/it] 42%|████▏ | 1059/2500 [6:20:21<8:23:37, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.954552914715808, 'learning_rate': 5.764e-07, 'completion_length': 143.5446548461914, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00537109375, 'epoch': 0.42} 42%|████▏ | 1059/2500 [6:20:21<8:23:37, 20.97s/it] 42%|████▏ | 1060/2500 [6:20:42<8:26:57, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.02695968778458542, 'learning_rate': 5.76e-07, 'completion_length': 153.33929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.42} 42%|████▏ | 1060/2500 [6:20:42<8:26:57, 21.12s/it] 42%|████▏ | 1061/2500 [6:21:03<8:21:55, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.3417868045337401, 'learning_rate': 5.755999999999999e-07, 'completion_length': 139.39286041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00450897216796875, 'epoch': 0.42} 42%|████▏ | 1061/2500 [6:21:03<8:21:55, 20.93s/it] 42%|████▏ | 1062/2500 [6:21:24<8:21:10, 20.91s/it] {'loss': 0.0003, 'grad_norm': 0.5982092553164412, 'learning_rate': 5.752e-07, 'completion_length': 158.61608123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0077056884765625, 'epoch': 0.42} 42%|████▏ | 1062/2500 [6:21:24<8:21:10, 20.91s/it] 43%|████▎ | 1063/2500 [6:21:45<8:23:36, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.5796586277992887, 'learning_rate': 5.748e-07, 'completion_length': 151.9107208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0064239501953125, 'epoch': 0.43} 43%|████▎ | 1063/2500 [6:21:45<8:23:36, 21.03s/it] 43%|████▎ | 1064/2500 [6:22:06<8:23:09, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.022056083961732427, 'learning_rate': 5.744e-07, 'completion_length': 138.7857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004180908203125, 'epoch': 0.43} 43%|████▎ | 1064/2500 [6:22:06<8:23:09, 21.02s/it] 43%|████▎ | 1065/2500 [6:22:28<8:26:13, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.1672654123834784, 'learning_rate': 5.739999999999999e-07, 'completion_length': 155.2678680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048370361328125, 'epoch': 0.43} 43%|████▎ | 1065/2500 [6:22:28<8:26:13, 21.17s/it] 43%|████▎ | 1066/2500 [6:22:49<8:24:58, 21.13s/it] {'loss': 0.0003, 'grad_norm': 0.425763564200389, 'learning_rate': 5.736e-07, 'completion_length': 148.50000762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.006805419921875, 'epoch': 0.43} 43%|████▎ | 1066/2500 [6:22:49<8:24:58, 21.13s/it] 43%|████▎ | 1067/2500 [6:23:11<8:31:18, 21.41s/it] {'loss': 0.0003, 'grad_norm': 0.02133038323285329, 'learning_rate': 5.732e-07, 'completion_length': 153.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006561279296875, 'epoch': 0.43} 43%|████▎ | 1067/2500 [6:23:11<8:31:18, 21.41s/it] 43%|████▎ | 1068/2500 [6:23:32<8:28:10, 21.29s/it] {'loss': 0.0003, 'grad_norm': 0.5169803853557159, 'learning_rate': 5.727999999999999e-07, 'completion_length': 155.23214721679688, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.09918941557407379, 'kl': 0.0077362060546875, 'epoch': 0.43} 43%|████▎ | 1068/2500 [6:23:32<8:28:10, 21.29s/it] 43%|████▎ | 1069/2500 [6:23:53<8:27:34, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.5481223671281014, 'learning_rate': 5.724e-07, 'completion_length': 149.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0055999755859375, 'epoch': 0.43} 43%|████▎ | 1069/2500 [6:23:53<8:27:34, 21.28s/it] 43%|████▎ | 1070/2500 [6:24:14<8:26:43, 21.26s/it] {'loss': 0.0003, 'grad_norm': 0.803681942570524, 'learning_rate': 5.719999999999999e-07, 'completion_length': 154.8839340209961, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11272923648357391, 'kl': 0.0073699951171875, 'epoch': 0.43} 43%|████▎ | 1070/2500 [6:24:14<8:26:43, 21.26s/it] 43%|████▎ | 1071/2500 [6:24:35<8:24:08, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.20025565888191033, 'learning_rate': 5.716e-07, 'completion_length': 146.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006011962890625, 'epoch': 0.43} 43%|████▎ | 1071/2500 [6:24:35<8:24:08, 21.17s/it] 43%|████▎ | 1072/2500 [6:24:55<8:13:32, 20.74s/it] {'loss': 0.0003, 'grad_norm': 0.0290726156544533, 'learning_rate': 5.712e-07, 'completion_length': 134.17857360839844, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00634765625, 'epoch': 0.43} 43%|████▎ | 1072/2500 [6:24:55<8:13:32, 20.74s/it] 43%|████▎ | 1073/2500 [6:25:16<8:14:55, 20.81s/it] {'loss': 0.0003, 'grad_norm': 0.026592759956447543, 'learning_rate': 5.707999999999999e-07, 'completion_length': 161.90179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006805419921875, 'epoch': 0.43} 43%|████▎ | 1073/2500 [6:25:16<8:14:55, 20.81s/it] 43%|████▎ | 1074/2500 [6:25:37<8:16:26, 20.89s/it] {'loss': 0.0003, 'grad_norm': 0.028374820492482217, 'learning_rate': 5.704e-07, 'completion_length': 158.9464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.008575439453125, 'epoch': 0.43} 43%|████▎ | 1074/2500 [6:25:37<8:16:26, 20.89s/it] 43%|████▎ | 1075/2500 [6:25:58<8:16:29, 20.91s/it] {'loss': 0.0003, 'grad_norm': 0.5520135624631635, 'learning_rate': 5.699999999999999e-07, 'completion_length': 147.3482208251953, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0070037841796875, 'epoch': 0.43} 43%|████▎ | 1075/2500 [6:25:58<8:16:29, 20.91s/it] 43%|████▎ | 1076/2500 [6:26:19<8:18:38, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.018455259045218348, 'learning_rate': 5.696e-07, 'completion_length': 152.42857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049896240234375, 'epoch': 0.43} 43%|████▎ | 1076/2500 [6:26:19<8:18:38, 21.01s/it] 43%|████▎ | 1077/2500 [6:26:40<8:18:25, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.37324115309821226, 'learning_rate': 5.692e-07, 'completion_length': 151.85714721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0073394775390625, 'epoch': 0.43} 43%|████▎ | 1077/2500 [6:26:40<8:18:25, 21.02s/it] 43%|████▎ | 1078/2500 [6:27:02<8:21:12, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.9922088183951147, 'learning_rate': 5.688e-07, 'completion_length': 153.0982208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.05050762742757797, 'kl': 0.0063934326171875, 'epoch': 0.43} 43%|████▎ | 1078/2500 [6:27:02<8:21:12, 21.15s/it] 43%|████▎ | 1079/2500 [6:27:23<8:22:09, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.249999161350107, 'learning_rate': 5.684e-07, 'completion_length': 155.5089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0070037841796875, 'epoch': 0.43} 43%|████▎ | 1079/2500 [6:27:23<8:22:09, 21.20s/it] 43%|████▎ | 1080/2500 [6:27:44<8:21:06, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.5983786269065584, 'learning_rate': 5.679999999999999e-07, 'completion_length': 153.49107360839844, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.1094253659248352, 'kl': 0.0067596435546875, 'epoch': 0.43} 43%|████▎ | 1080/2500 [6:27:44<8:21:06, 21.17s/it] 43%|████▎ | 1081/2500 [6:28:06<8:22:43, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.02416928490021226, 'learning_rate': 5.676e-07, 'completion_length': 159.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00579833984375, 'epoch': 0.43} 43%|████▎ | 1081/2500 [6:28:06<8:22:43, 21.26s/it] 43%|████▎ | 1082/2500 [6:28:27<8:21:01, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.4764471583335711, 'learning_rate': 5.672e-07, 'completion_length': 171.13394165039062, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780732154846, 'kl': 0.006744384765625, 'epoch': 0.43} 43%|████▎ | 1082/2500 [6:28:27<8:21:01, 21.20s/it] 43%|████▎ | 1083/2500 [6:28:48<8:23:01, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.33784375405598904, 'learning_rate': 5.667999999999999e-07, 'completion_length': 159.1696548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005157470703125, 'epoch': 0.43} 43%|████▎ | 1083/2500 [6:28:48<8:23:01, 21.30s/it] 43%|████▎ | 1084/2500 [6:29:09<8:21:37, 21.26s/it] {'loss': 0.0004, 'grad_norm': 0.3836602117572205, 'learning_rate': 5.664e-07, 'completion_length': 157.8928680419922, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.010894775390625, 'epoch': 0.43} 43%|████▎ | 1084/2500 [6:29:09<8:21:37, 21.26s/it] 43%|████▎ | 1085/2500 [6:29:31<8:27:43, 21.53s/it] {'loss': 0.0004, 'grad_norm': 1.0496190409111257, 'learning_rate': 5.66e-07, 'completion_length': 158.24108123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797151088715, 'kl': 0.0088348388671875, 'epoch': 0.43} 43%|████▎ | 1085/2500 [6:29:31<8:27:43, 21.53s/it] 43%|████▎ | 1086/2500 [6:29:52<8:21:42, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.3903988591942718, 'learning_rate': 5.655999999999999e-07, 'completion_length': 147.4464340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0051727294921875, 'epoch': 0.43} 43%|████▎ | 1086/2500 [6:29:52<8:21:42, 21.29s/it] 43%|████▎ | 1087/2500 [6:30:14<8:24:51, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.23766803781736776, 'learning_rate': 5.652e-07, 'completion_length': 159.55358123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006805419921875, 'epoch': 0.43} 43%|████▎ | 1087/2500 [6:30:14<8:24:51, 21.44s/it] 44%|████▎ | 1088/2500 [6:30:35<8:24:33, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.02994723340361596, 'learning_rate': 5.648e-07, 'completion_length': 155.58929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.008636474609375, 'epoch': 0.44} 44%|████▎ | 1088/2500 [6:30:35<8:24:33, 21.44s/it] 44%|████▎ | 1089/2500 [6:30:57<8:23:14, 21.40s/it] {'loss': 0.0003, 'grad_norm': 1.5120708893809565, 'learning_rate': 5.643999999999999e-07, 'completion_length': 157.0446548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.00726318359375, 'epoch': 0.44} 44%|████▎ | 1089/2500 [6:30:57<8:23:14, 21.40s/it] 44%|████▎ | 1090/2500 [6:31:18<8:20:18, 21.29s/it] {'loss': 0.0003, 'grad_norm': 0.683376585449096, 'learning_rate': 5.639999999999999e-07, 'completion_length': 153.24108123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0077056884765625, 'epoch': 0.44} 44%|████▎ | 1090/2500 [6:31:18<8:20:18, 21.29s/it] 44%|████▎ | 1091/2500 [6:31:38<8:15:51, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.5116065335264535, 'learning_rate': 5.636e-07, 'completion_length': 149.42857360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0070037841796875, 'epoch': 0.44} 44%|████▎ | 1091/2500 [6:31:38<8:15:51, 21.12s/it] 44%|████▎ | 1092/2500 [6:32:00<8:19:03, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.40491591791284465, 'learning_rate': 5.632e-07, 'completion_length': 161.08929443359375, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.006195068359375, 'epoch': 0.44} 44%|████▎ | 1092/2500 [6:32:00<8:19:03, 21.27s/it] 44%|████▎ | 1093/2500 [6:32:22<8:21:13, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.22272446252857223, 'learning_rate': 5.627999999999999e-07, 'completion_length': 163.14286041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00514984130859375, 'epoch': 0.44} 44%|████▎ | 1093/2500 [6:32:22<8:21:13, 21.37s/it] 44%|████▍ | 1094/2500 [6:32:43<8:20:34, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.026164139043800022, 'learning_rate': 5.624e-07, 'completion_length': 150.0357208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0067138671875, 'epoch': 0.44} 44%|████▍ | 1094/2500 [6:32:43<8:20:34, 21.36s/it] 44%|████▍ | 1095/2500 [6:33:04<8:19:18, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.33491243677170807, 'learning_rate': 5.620000000000001e-07, 'completion_length': 158.79464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.44} 44%|████▍ | 1095/2500 [6:33:04<8:19:18, 21.32s/it] 44%|████▍ | 1096/2500 [6:33:25<8:12:25, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.019716697286766498, 'learning_rate': 5.615999999999999e-07, 'completion_length': 139.6785774230957, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054473876953125, 'epoch': 0.44} 44%|████▍ | 1096/2500 [6:33:25<8:12:25, 21.04s/it] 44%|████▍ | 1097/2500 [6:33:46<8:11:33, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.018602435986184534, 'learning_rate': 5.612e-07, 'completion_length': 152.33929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00482177734375, 'epoch': 0.44} 44%|████▍ | 1097/2500 [6:33:46<8:11:33, 21.02s/it] 44%|████▍ | 1098/2500 [6:34:09<8:24:20, 21.58s/it] {'loss': 0.0003, 'grad_norm': 0.7955476410963126, 'learning_rate': 5.608e-07, 'completion_length': 165.51786041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.0081787109375, 'epoch': 0.44} 44%|████▍ | 1098/2500 [6:34:09<8:24:20, 21.58s/it] 44%|████▍ | 1099/2500 [6:34:29<8:17:45, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.6096364732299157, 'learning_rate': 5.604e-07, 'completion_length': 151.30358123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0081787109375, 'epoch': 0.44} 44%|████▍ | 1099/2500 [6:34:29<8:17:45, 21.32s/it] 44%|████▍ | 1100/2500 [6:34:51<8:18:28, 21.36s/it] {'loss': 0.0002, 'grad_norm': 0.48624118392116156, 'learning_rate': 5.6e-07, 'completion_length': 149.76786041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0047454833984375, 'epoch': 0.44} 44%|████▍ | 1100/2500 [6:34:51<8:18:28, 21.36s/it] 44%|████▍ | 1101/2500 [6:36:09<14:55:02, 38.39s/it] {'loss': 0.0003, 'grad_norm': 0.6575748050994955, 'learning_rate': 5.596e-07, 'completion_length': 150.64286041259766, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.09138382598757744, 'kl': 0.0075836181640625, 'epoch': 0.44} 44%|████▍ | 1101/2500 [6:36:09<14:55:02, 38.39s/it] 44%|████▍ | 1102/2500 [6:36:30<12:56:42, 33.34s/it] {'loss': 0.0003, 'grad_norm': 0.2877543059269765, 'learning_rate': 5.592e-07, 'completion_length': 168.33928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.008209228515625, 'epoch': 0.44} 44%|████▍ | 1102/2500 [6:36:30<12:56:42, 33.34s/it] 44%|████▍ | 1103/2500 [6:36:52<11:36:12, 29.90s/it] {'loss': 0.0003, 'grad_norm': 1.0914910004400584, 'learning_rate': 5.588e-07, 'completion_length': 152.0089340209961, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.12175281718373299, 'kl': 0.0064697265625, 'epoch': 0.44} 44%|████▍ | 1103/2500 [6:36:52<11:36:12, 29.90s/it] 44%|████▍ | 1104/2500 [6:37:14<10:37:57, 27.42s/it] {'loss': 0.0002, 'grad_norm': 0.22393421454306733, 'learning_rate': 5.584e-07, 'completion_length': 157.08928680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0059967041015625, 'epoch': 0.44} 44%|████▍ | 1104/2500 [6:37:14<10:37:57, 27.42s/it] 44%|████▍ | 1105/2500 [6:37:35<9:50:50, 25.41s/it] {'loss': 0.0002, 'grad_norm': 0.6163345083809921, 'learning_rate': 5.58e-07, 'completion_length': 153.61608123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0055999755859375, 'epoch': 0.44} 44%|████▍ | 1105/2500 [6:37:35<9:50:50, 25.41s/it] 44%|████▍ | 1106/2500 [6:37:56<9:22:36, 24.22s/it] {'loss': 0.0003, 'grad_norm': 0.3131269395797215, 'learning_rate': 5.576e-07, 'completion_length': 160.5714340209961, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.007415771484375, 'epoch': 0.44} 44%|████▍ | 1106/2500 [6:37:56<9:22:36, 24.22s/it] 44%|████▍ | 1107/2500 [6:38:17<8:59:34, 23.24s/it] {'loss': 0.0002, 'grad_norm': 0.29913190152871516, 'learning_rate': 5.572e-07, 'completion_length': 130.5714340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.004119873046875, 'epoch': 0.44} 44%|████▍ | 1107/2500 [6:38:17<8:59:34, 23.24s/it] 44%|████▍ | 1108/2500 [6:38:38<8:44:00, 22.59s/it] {'loss': 0.0003, 'grad_norm': 0.034039416340307625, 'learning_rate': 5.567999999999999e-07, 'completion_length': 157.39286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0075225830078125, 'epoch': 0.44} 44%|████▍ | 1108/2500 [6:38:38<8:44:00, 22.59s/it] 44%|████▍ | 1109/2500 [6:38:59<8:34:30, 22.19s/it] {'loss': 0.0002, 'grad_norm': 0.024449948591442346, 'learning_rate': 5.564e-07, 'completion_length': 151.75000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005218505859375, 'epoch': 0.44} 44%|████▍ | 1109/2500 [6:38:59<8:34:30, 22.19s/it] 44%|████▍ | 1110/2500 [6:39:21<8:29:56, 22.01s/it] {'loss': 0.0002, 'grad_norm': 0.6962946705070407, 'learning_rate': 5.560000000000001e-07, 'completion_length': 160.40178680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.006134033203125, 'epoch': 0.44} 44%|████▍ | 1110/2500 [6:39:21<8:29:56, 22.01s/it] 44%|████▍ | 1111/2500 [6:39:51<9:25:43, 24.44s/it] {'loss': 0.0002, 'grad_norm': 0.8896220802002561, 'learning_rate': 5.555999999999999e-07, 'completion_length': 145.5714340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543394565582, 'kl': 0.005767822265625, 'epoch': 0.44} 44%|████▍ | 1111/2500 [6:39:51<9:25:43, 24.44s/it] 44%|████▍ | 1112/2500 [6:40:12<9:03:27, 23.49s/it] {'loss': 0.0002, 'grad_norm': 0.4470072823519991, 'learning_rate': 5.552e-07, 'completion_length': 137.3303680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00403594970703125, 'epoch': 0.44} 44%|████▍ | 1112/2500 [6:40:12<9:03:27, 23.49s/it] 45%|████▍ | 1113/2500 [6:40:33<8:43:47, 22.66s/it] {'loss': 0.0002, 'grad_norm': 0.04796247578259084, 'learning_rate': 5.548e-07, 'completion_length': 148.5178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.45} 45%|████▍ | 1113/2500 [6:40:33<8:43:47, 22.66s/it] 45%|████▍ | 1114/2500 [6:40:54<8:35:12, 22.30s/it] {'loss': 0.0002, 'grad_norm': 0.032396068817572156, 'learning_rate': 5.543999999999999e-07, 'completion_length': 158.58929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006134033203125, 'epoch': 0.45} 45%|████▍ | 1114/2500 [6:40:54<8:35:12, 22.30s/it] 45%|████▍ | 1115/2500 [6:41:15<8:22:05, 21.75s/it] {'loss': 0.0003, 'grad_norm': 0.5044071054012009, 'learning_rate': 5.54e-07, 'completion_length': 141.83036041259766, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0067596435546875, 'epoch': 0.45} 45%|████▍ | 1115/2500 [6:41:15<8:22:05, 21.75s/it] 45%|████▍ | 1116/2500 [6:41:36<8:19:51, 21.67s/it] {'loss': 0.0002, 'grad_norm': 0.30114053620016124, 'learning_rate': 5.536e-07, 'completion_length': 146.23214721679688, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0052490234375, 'epoch': 0.45} 45%|████▍ | 1116/2500 [6:41:36<8:19:51, 21.67s/it] 45%|████▍ | 1117/2500 [6:41:57<8:13:23, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.0178482972710728, 'learning_rate': 5.532e-07, 'completion_length': 138.3928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00428009033203125, 'epoch': 0.45} 45%|████▍ | 1117/2500 [6:41:57<8:13:23, 21.41s/it] 45%|████▍ | 1118/2500 [6:42:18<8:11:30, 21.34s/it] {'loss': 0.0002, 'grad_norm': 1.1269847980526464, 'learning_rate': 5.527999999999999e-07, 'completion_length': 154.18750762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0060577392578125, 'epoch': 0.45} 45%|████▍ | 1118/2500 [6:42:18<8:11:30, 21.34s/it] 45%|████▍ | 1119/2500 [6:42:39<8:06:33, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.49707254060867834, 'learning_rate': 5.524e-07, 'completion_length': 133.0982208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.00455474853515625, 'epoch': 0.45} 45%|████▍ | 1119/2500 [6:42:39<8:06:33, 21.14s/it] 45%|████▍ | 1120/2500 [6:43:02<8:15:15, 21.53s/it] {'loss': 0.0004, 'grad_norm': 0.4259273931152154, 'learning_rate': 5.520000000000001e-07, 'completion_length': 179.0357208251953, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.10821298509836197, 'kl': 0.01007080078125, 'epoch': 0.45} 45%|████▍ | 1120/2500 [6:43:02<8:15:15, 21.53s/it] 45%|████▍ | 1121/2500 [6:43:23<8:14:27, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.42943204851603606, 'learning_rate': 5.515999999999999e-07, 'completion_length': 153.0625, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.06343399360775948, 'kl': 0.00665283203125, 'epoch': 0.45} 45%|████▍ | 1121/2500 [6:43:23<8:14:27, 21.51s/it] 45%|████▍ | 1122/2500 [6:43:44<8:10:48, 21.37s/it] {'loss': 0.0002, 'grad_norm': 1.655107000293215, 'learning_rate': 5.512e-07, 'completion_length': 144.71429443359375, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.025253813713788986, 'kl': 0.0057220458984375, 'epoch': 0.45} 45%|████▍ | 1122/2500 [6:43:44<8:10:48, 21.37s/it] 45%|████▍ | 1123/2500 [6:44:05<8:09:03, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.9447184647956545, 'learning_rate': 5.508e-07, 'completion_length': 144.65179443359375, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.004974365234375, 'epoch': 0.45} 45%|████▍ | 1123/2500 [6:44:05<8:09:03, 21.31s/it] 45%|████▍ | 1124/2500 [6:44:27<8:11:04, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.030908843054603855, 'learning_rate': 5.504e-07, 'completion_length': 155.23214721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00592041015625, 'epoch': 0.45} 45%|████▍ | 1124/2500 [6:44:27<8:11:04, 21.41s/it] 45%|████▌ | 1125/2500 [6:44:48<8:07:22, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.2429804712507113, 'learning_rate': 5.5e-07, 'completion_length': 151.73214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052642822265625, 'epoch': 0.45} 45%|████▌ | 1125/2500 [6:44:48<8:07:22, 21.27s/it] 45%|████▌ | 1126/2500 [6:45:10<8:13:28, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.04453631918574614, 'learning_rate': 5.496e-07, 'completion_length': 160.2232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.007598876953125, 'epoch': 0.45} 45%|████▌ | 1126/2500 [6:45:10<8:13:28, 21.55s/it] 45%|████▌ | 1127/2500 [6:45:33<8:22:14, 21.95s/it] {'loss': 0.0003, 'grad_norm': 0.8957556189678438, 'learning_rate': 5.492e-07, 'completion_length': 178.75894165039062, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00811767578125, 'epoch': 0.45} 45%|████▌ | 1127/2500 [6:45:33<8:22:14, 21.95s/it] 45%|████▌ | 1128/2500 [6:45:56<8:30:32, 22.33s/it] {'loss': 0.0002, 'grad_norm': 0.028357996675441432, 'learning_rate': 5.487999999999999e-07, 'completion_length': 141.35714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0046539306640625, 'epoch': 0.45} 45%|████▌ | 1128/2500 [6:45:56<8:30:32, 22.33s/it] 45%|████▌ | 1129/2500 [6:46:17<8:23:18, 22.03s/it] {'loss': 0.0002, 'grad_norm': 0.020516870961264887, 'learning_rate': 5.484e-07, 'completion_length': 166.75000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00537109375, 'epoch': 0.45} 45%|████▌ | 1129/2500 [6:46:17<8:23:18, 22.03s/it] 45%|████▌ | 1130/2500 [6:46:38<8:11:27, 21.52s/it] {'loss': 0.0001, 'grad_norm': 0.01995563474430649, 'learning_rate': 5.48e-07, 'completion_length': 141.56250762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.003662109375, 'epoch': 0.45} 45%|████▌ | 1130/2500 [6:46:38<8:11:27, 21.52s/it] 45%|████▌ | 1131/2500 [6:46:59<8:12:34, 21.59s/it] {'loss': 0.0002, 'grad_norm': 0.5460120643270657, 'learning_rate': 5.476e-07, 'completion_length': 148.58036041259766, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0042266845703125, 'epoch': 0.45} 45%|████▌ | 1131/2500 [6:46:59<8:12:34, 21.59s/it] 45%|████▌ | 1132/2500 [6:47:20<8:06:03, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.255767827557853, 'learning_rate': 5.472e-07, 'completion_length': 149.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0049896240234375, 'epoch': 0.45} 45%|████▌ | 1132/2500 [6:47:20<8:06:03, 21.32s/it] 45%|████▌ | 1133/2500 [6:47:41<8:03:24, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.21551055091525062, 'learning_rate': 5.467999999999999e-07, 'completion_length': 153.83036041259766, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.008209228515625, 'epoch': 0.45} 45%|████▌ | 1133/2500 [6:47:41<8:03:24, 21.22s/it] 45%|████▌ | 1134/2500 [6:48:03<8:06:17, 21.36s/it] {'loss': 0.0002, 'grad_norm': 0.9124192199054127, 'learning_rate': 5.464e-07, 'completion_length': 152.23214721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.006134033203125, 'epoch': 0.45} 45%|████▌ | 1134/2500 [6:48:03<8:06:17, 21.36s/it] 45%|████▌ | 1135/2500 [6:48:24<8:03:36, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.34440668632735033, 'learning_rate': 5.46e-07, 'completion_length': 158.46429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005889892578125, 'epoch': 0.45} 45%|████▌ | 1135/2500 [6:48:24<8:03:36, 21.26s/it] 45%|████▌ | 1136/2500 [6:48:46<8:09:44, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.3978895624690664, 'learning_rate': 5.455999999999999e-07, 'completion_length': 159.8839340209961, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.00537109375, 'epoch': 0.45} 45%|████▌ | 1136/2500 [6:48:46<8:09:44, 21.54s/it] 45%|████▌ | 1137/2500 [6:49:08<8:09:25, 21.54s/it] {'loss': 0.0004, 'grad_norm': 0.5682023433619228, 'learning_rate': 5.452e-07, 'completion_length': 145.7053680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.08747543022036552, 'kl': 0.0107421875, 'epoch': 0.45} 45%|████▌ | 1137/2500 [6:49:08<8:09:25, 21.54s/it] 46%|████▌ | 1138/2500 [6:49:29<8:07:36, 21.48s/it] {'loss': 0.0003, 'grad_norm': 0.32132135591114935, 'learning_rate': 5.448e-07, 'completion_length': 155.1875, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0084686279296875, 'epoch': 0.46} 46%|████▌ | 1138/2500 [6:49:29<8:07:36, 21.48s/it] 46%|████▌ | 1139/2500 [6:49:50<8:04:23, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.42200488326834357, 'learning_rate': 5.443999999999999e-07, 'completion_length': 148.71429443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0050506591796875, 'epoch': 0.46} 46%|████▌ | 1139/2500 [6:49:50<8:04:23, 21.35s/it] 46%|████▌ | 1140/2500 [6:50:11<8:03:44, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.020106084922187923, 'learning_rate': 5.44e-07, 'completion_length': 145.4732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0038604736328125, 'epoch': 0.46} 46%|████▌ | 1140/2500 [6:50:11<8:03:44, 21.34s/it] 46%|████▌ | 1141/2500 [6:50:33<8:03:12, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.4292244583506022, 'learning_rate': 5.436e-07, 'completion_length': 140.48214721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.10101525485515594, 'kl': 0.0056915283203125, 'epoch': 0.46} 46%|████▌ | 1141/2500 [6:50:33<8:03:12, 21.33s/it] 46%|████▌ | 1142/2500 [6:50:54<8:00:17, 21.22s/it] {'loss': 0.0003, 'grad_norm': 1.5938057091062932, 'learning_rate': 5.431999999999999e-07, 'completion_length': 157.00000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00836181640625, 'epoch': 0.46} 46%|████▌ | 1142/2500 [6:50:54<8:00:17, 21.22s/it] 46%|████▌ | 1143/2500 [6:51:15<7:59:10, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.3159703829208529, 'learning_rate': 5.427999999999999e-07, 'completion_length': 140.7857208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0047149658203125, 'epoch': 0.46} 46%|████▌ | 1143/2500 [6:51:15<7:59:10, 21.19s/it] 46%|████▌ | 1144/2500 [6:51:37<8:02:59, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.5510429068937861, 'learning_rate': 5.424e-07, 'completion_length': 149.8928680419922, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989997446537, 'kl': 0.0065765380859375, 'epoch': 0.46} 46%|████▌ | 1144/2500 [6:51:37<8:02:59, 21.37s/it] 46%|████▌ | 1145/2500 [6:51:58<8:03:41, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.4773509211432059, 'learning_rate': 5.420000000000001e-07, 'completion_length': 153.61608123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0054473876953125, 'epoch': 0.46} 46%|████▌ | 1145/2500 [6:51:58<8:03:41, 21.42s/it] 46%|████▌ | 1146/2500 [6:52:19<8:02:14, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.3646999757272707, 'learning_rate': 5.415999999999999e-07, 'completion_length': 158.02679443359375, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831881985068321, 'kl': 0.00504302978515625, 'epoch': 0.46} 46%|████▌ | 1146/2500 [6:52:19<8:02:14, 21.37s/it] 46%|████▌ | 1147/2500 [6:52:40<7:57:01, 21.15s/it] {'loss': 0.0002, 'grad_norm': 2.5064804002258634, 'learning_rate': 5.412e-07, 'completion_length': 145.58929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0055999755859375, 'epoch': 0.46} 46%|████▌ | 1147/2500 [6:52:40<7:57:01, 21.15s/it] 46%|████▌ | 1148/2500 [6:53:02<8:00:20, 21.32s/it] {'loss': 0.0004, 'grad_norm': 0.7480422339431463, 'learning_rate': 5.408e-07, 'completion_length': 163.60714721679688, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831882357597351, 'kl': 0.0107421875, 'epoch': 0.46} 46%|████▌ | 1148/2500 [6:53:02<8:00:20, 21.32s/it] 46%|████▌ | 1149/2500 [6:53:25<8:11:38, 21.83s/it] {'loss': 0.0003, 'grad_norm': 0.3383640137256759, 'learning_rate': 5.403999999999999e-07, 'completion_length': 159.6964340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06222161278128624, 'kl': 0.006683349609375, 'epoch': 0.46} 46%|████▌ | 1149/2500 [6:53:25<8:11:38, 21.83s/it] 46%|████▌ | 1150/2500 [6:53:46<8:07:25, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.021608638429392066, 'learning_rate': 5.4e-07, 'completion_length': 144.26786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0044708251953125, 'epoch': 0.46} 46%|████▌ | 1150/2500 [6:53:46<8:07:25, 21.66s/it] 46%|████▌ | 1151/2500 [6:54:06<7:59:08, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.036612141122742324, 'learning_rate': 5.396e-07, 'completion_length': 148.58036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052337646484375, 'epoch': 0.46} 46%|████▌ | 1151/2500 [6:54:06<7:59:08, 21.31s/it] 46%|████▌ | 1152/2500 [6:54:28<8:00:45, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.020843545239912057, 'learning_rate': 5.392e-07, 'completion_length': 143.3482208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.46} 46%|████▌ | 1152/2500 [6:54:28<8:00:45, 21.40s/it] 46%|████▌ | 1153/2500 [6:54:50<8:01:45, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.16425160049395995, 'learning_rate': 5.387999999999999e-07, 'completion_length': 159.42858123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0056610107421875, 'epoch': 0.46} 46%|████▌ | 1153/2500 [6:54:50<8:01:45, 21.46s/it] 46%|████▌ | 1154/2500 [6:55:10<7:53:35, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.256124640581559, 'learning_rate': 5.384e-07, 'completion_length': 141.27679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0038909912109375, 'epoch': 0.46} 46%|████▌ | 1154/2500 [6:55:10<7:53:35, 21.11s/it] 46%|████▌ | 1155/2500 [6:55:31<7:51:33, 21.04s/it] {'loss': 0.0002, 'grad_norm': 1.1122119551689778, 'learning_rate': 5.38e-07, 'completion_length': 143.1339340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.005157470703125, 'epoch': 0.46} 46%|████▌ | 1155/2500 [6:55:31<7:51:33, 21.04s/it] 46%|████▌ | 1156/2500 [6:55:53<7:57:26, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.031095472501557413, 'learning_rate': 5.375999999999999e-07, 'completion_length': 157.68750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00521087646484375, 'epoch': 0.46} 46%|████▌ | 1156/2500 [6:55:53<7:57:26, 21.31s/it] 46%|████▋ | 1157/2500 [6:56:14<7:58:51, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.016055846812499365, 'learning_rate': 5.372e-07, 'completion_length': 151.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00513458251953125, 'epoch': 0.46} 46%|████▋ | 1157/2500 [6:56:14<7:58:51, 21.39s/it] 46%|████▋ | 1158/2500 [6:56:36<7:58:05, 21.38s/it] {'loss': 0.0003, 'grad_norm': 0.43838320688935256, 'learning_rate': 5.368e-07, 'completion_length': 157.61608123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.0073394775390625, 'epoch': 0.46} 46%|████▋ | 1158/2500 [6:56:36<7:58:05, 21.38s/it] 46%|████▋ | 1159/2500 [6:56:57<7:57:44, 21.38s/it] {'loss': 0.0002, 'grad_norm': 2.954788050491625, 'learning_rate': 5.364e-07, 'completion_length': 155.6964340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.005340576171875, 'epoch': 0.46} 46%|████▋ | 1159/2500 [6:56:57<7:57:44, 21.38s/it] 46%|████▋ | 1160/2500 [6:57:19<8:00:53, 21.53s/it] {'loss': 0.0002, 'grad_norm': 0.41610346549803984, 'learning_rate': 5.36e-07, 'completion_length': 158.0446548461914, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.006195068359375, 'epoch': 0.46} 46%|████▋ | 1160/2500 [6:57:19<8:00:53, 21.53s/it] 46%|████▋ | 1161/2500 [6:57:40<8:00:15, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.7465559029265304, 'learning_rate': 5.355999999999999e-07, 'completion_length': 159.98214721679688, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.10700060427188873, 'kl': 0.00787353515625, 'epoch': 0.46} 46%|████▋ | 1161/2500 [6:57:40<8:00:15, 21.52s/it] 46%|████▋ | 1162/2500 [6:58:01<7:52:57, 21.21s/it] {'loss': 0.0004, 'grad_norm': 0.3838376270420387, 'learning_rate': 5.352e-07, 'completion_length': 144.1696548461914, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.008941650390625, 'epoch': 0.46} 46%|████▋ | 1162/2500 [6:58:01<7:52:57, 21.21s/it] 47%|████▋ | 1163/2500 [6:58:22<7:52:52, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.4728854236186135, 'learning_rate': 5.348e-07, 'completion_length': 157.71429443359375, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.07124518603086472, 'kl': 0.0056915283203125, 'epoch': 0.47} 47%|████▋ | 1163/2500 [6:58:22<7:52:52, 21.22s/it] 47%|████▋ | 1164/2500 [6:58:44<7:53:46, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.3878066305800243, 'learning_rate': 5.343999999999999e-07, 'completion_length': 154.93750762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.00787353515625, 'epoch': 0.47} 47%|████▋ | 1164/2500 [6:58:44<7:53:46, 21.28s/it] 47%|████▋ | 1165/2500 [6:59:04<7:47:32, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.01785621130408659, 'learning_rate': 5.34e-07, 'completion_length': 137.89286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00385284423828125, 'epoch': 0.47} 47%|████▋ | 1165/2500 [6:59:04<7:47:32, 21.01s/it] 47%|████▋ | 1166/2500 [6:59:25<7:46:55, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.7900783268237564, 'learning_rate': 5.336e-07, 'completion_length': 138.0714340209961, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.08868780359625816, 'kl': 0.0078125, 'epoch': 0.47} 47%|████▋ | 1166/2500 [6:59:25<7:46:55, 21.00s/it] 47%|████▋ | 1167/2500 [6:59:47<7:53:28, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.021110997810084284, 'learning_rate': 5.331999999999999e-07, 'completion_length': 152.83036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00478363037109375, 'epoch': 0.47} 47%|████▋ | 1167/2500 [6:59:47<7:53:28, 21.31s/it] 47%|████▋ | 1168/2500 [7:00:08<7:51:40, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.7427882589829365, 'learning_rate': 5.328e-07, 'completion_length': 154.45536041259766, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.00567626953125, 'epoch': 0.47} 47%|████▋ | 1168/2500 [7:00:08<7:51:40, 21.25s/it] 47%|████▋ | 1169/2500 [7:00:29<7:51:18, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.019085136042695194, 'learning_rate': 5.324e-07, 'completion_length': 147.56250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0038299560546875, 'epoch': 0.47} 47%|████▋ | 1169/2500 [7:00:29<7:51:18, 21.25s/it] 47%|████▋ | 1170/2500 [7:00:51<7:51:01, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.536806571929774, 'learning_rate': 5.32e-07, 'completion_length': 157.35714721679688, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.00582122802734375, 'epoch': 0.47} 47%|████▋ | 1170/2500 [7:00:51<7:51:01, 21.25s/it] 47%|████▋ | 1171/2500 [7:01:12<7:48:51, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.15601875549652153, 'learning_rate': 5.315999999999999e-07, 'completion_length': 158.7946548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0040283203125, 'epoch': 0.47} 47%|████▋ | 1171/2500 [7:01:12<7:48:51, 21.17s/it] 47%|████▋ | 1172/2500 [7:01:33<7:48:09, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.3882896356932811, 'learning_rate': 5.312e-07, 'completion_length': 151.1696548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0063934326171875, 'epoch': 0.47} 47%|████▋ | 1172/2500 [7:01:33<7:48:09, 21.15s/it] 47%|████▋ | 1173/2500 [7:01:55<7:55:28, 21.50s/it] {'loss': 0.0004, 'grad_norm': 0.4152392988441237, 'learning_rate': 5.308000000000001e-07, 'completion_length': 155.4107208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.010894775390625, 'epoch': 0.47} 47%|████▋ | 1173/2500 [7:01:55<7:55:28, 21.50s/it] 47%|████▋ | 1174/2500 [7:02:16<7:52:15, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.030571027023889814, 'learning_rate': 5.303999999999999e-07, 'completion_length': 156.46428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055389404296875, 'epoch': 0.47} 47%|████▋ | 1174/2500 [7:02:16<7:52:15, 21.37s/it] 47%|████▋ | 1175/2500 [7:02:38<7:53:21, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.027045399448451524, 'learning_rate': 5.3e-07, 'completion_length': 146.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00505828857421875, 'epoch': 0.47} 47%|████▋ | 1175/2500 [7:02:38<7:53:21, 21.43s/it] 47%|████▋ | 1176/2500 [7:02:59<7:52:01, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.2131808024122989, 'learning_rate': 5.296e-07, 'completion_length': 154.40179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00447845458984375, 'epoch': 0.47} 47%|████▋ | 1176/2500 [7:02:59<7:52:01, 21.39s/it] 47%|████▋ | 1177/2500 [7:03:20<7:48:46, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.1722967822145577, 'learning_rate': 5.292e-07, 'completion_length': 146.91964721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0054473876953125, 'epoch': 0.47} 47%|████▋ | 1177/2500 [7:03:20<7:48:46, 21.26s/it] 47%|████▋ | 1178/2500 [7:03:41<7:44:30, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.507879724427143, 'learning_rate': 5.288e-07, 'completion_length': 152.04464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005828857421875, 'epoch': 0.47} 47%|████▋ | 1178/2500 [7:03:41<7:44:30, 21.08s/it] 47%|████▋ | 1179/2500 [7:04:02<7:45:17, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.3077132189777714, 'learning_rate': 5.284e-07, 'completion_length': 162.17858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0051727294921875, 'epoch': 0.47} 47%|████▋ | 1179/2500 [7:04:02<7:45:17, 21.13s/it] 47%|████▋ | 1180/2500 [7:04:24<7:53:22, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.4863289859251384, 'learning_rate': 5.28e-07, 'completion_length': 165.70536041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989960193634, 'kl': 0.006500244140625, 'epoch': 0.47} 47%|████▋ | 1180/2500 [7:04:24<7:53:22, 21.52s/it] 47%|████▋ | 1181/2500 [7:04:45<7:50:44, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.1125760584144534, 'learning_rate': 5.275999999999999e-07, 'completion_length': 153.52678680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.005706787109375, 'epoch': 0.47} 47%|████▋ | 1181/2500 [7:04:45<7:50:44, 21.41s/it] 47%|████▋ | 1182/2500 [7:05:06<7:45:29, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.06877194131267311, 'learning_rate': 5.272e-07, 'completion_length': 161.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0068206787109375, 'epoch': 0.47} 47%|████▋ | 1182/2500 [7:05:06<7:45:29, 21.19s/it] 47%|████▋ | 1183/2500 [7:05:27<7:42:32, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.2606140265611681, 'learning_rate': 5.268e-07, 'completion_length': 154.14286041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00579833984375, 'epoch': 0.47} 47%|████▋ | 1183/2500 [7:05:27<7:42:32, 21.07s/it] 47%|████▋ | 1184/2500 [7:05:48<7:44:57, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.02855585546501387, 'learning_rate': 5.264e-07, 'completion_length': 157.0357208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0072174072265625, 'epoch': 0.47} 47%|████▋ | 1184/2500 [7:05:48<7:44:57, 21.20s/it] 47%|████▋ | 1185/2500 [7:06:10<7:45:16, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.8421245289843852, 'learning_rate': 5.26e-07, 'completion_length': 144.40179443359375, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.05831882357597351, 'kl': 0.0062255859375, 'epoch': 0.47} 47%|████▋ | 1185/2500 [7:06:10<7:45:16, 21.23s/it] 47%|████▋ | 1186/2500 [7:06:30<7:42:16, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.29396502817169634, 'learning_rate': 5.255999999999999e-07, 'completion_length': 142.12500762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004791259765625, 'epoch': 0.47} 47%|████▋ | 1186/2500 [7:06:30<7:42:16, 21.11s/it] 47%|████▋ | 1187/2500 [7:06:52<7:43:13, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.3591003170053706, 'learning_rate': 5.252e-07, 'completion_length': 149.93750762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.0077362060546875, 'epoch': 0.47} 47%|████▋ | 1187/2500 [7:06:52<7:43:13, 21.17s/it] 48%|████▊ | 1188/2500 [7:07:14<7:47:54, 21.40s/it] {'loss': 0.0002, 'grad_norm': 1.0445596264205368, 'learning_rate': 5.248e-07, 'completion_length': 157.92858123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797151088715, 'kl': 0.004608154296875, 'epoch': 0.48} 48%|████▊ | 1188/2500 [7:07:14<7:47:54, 21.40s/it] 48%|████▊ | 1189/2500 [7:07:34<7:42:20, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.29067493540760553, 'learning_rate': 5.243999999999999e-07, 'completion_length': 152.83929443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00604248046875, 'epoch': 0.48} 48%|████▊ | 1189/2500 [7:07:34<7:42:20, 21.16s/it] 48%|████▊ | 1190/2500 [7:07:54<7:35:27, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.5344495126600501, 'learning_rate': 5.24e-07, 'completion_length': 130.9732208251953, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0039215087890625, 'epoch': 0.48} 48%|████▊ | 1190/2500 [7:07:54<7:35:27, 20.86s/it] 48%|████▊ | 1191/2500 [7:08:15<7:33:03, 20.77s/it] {'loss': 0.0002, 'grad_norm': 1.4466050665813657, 'learning_rate': 5.236e-07, 'completion_length': 155.93750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.1030978113412857, 'kl': 0.006256103515625, 'epoch': 0.48} 48%|████▊ | 1191/2500 [7:08:15<7:33:03, 20.77s/it] 48%|████▊ | 1192/2500 [7:08:36<7:31:43, 20.72s/it] {'loss': 0.0002, 'grad_norm': 0.023984539449666074, 'learning_rate': 5.232e-07, 'completion_length': 147.42857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049591064453125, 'epoch': 0.48} 48%|████▊ | 1192/2500 [7:08:36<7:31:43, 20.72s/it] 48%|████▊ | 1193/2500 [7:08:57<7:33:46, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.21524089077938635, 'learning_rate': 5.228e-07, 'completion_length': 159.08929443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0060577392578125, 'epoch': 0.48} 48%|████▊ | 1193/2500 [7:08:57<7:33:46, 20.83s/it] 48%|████▊ | 1194/2500 [7:09:18<7:33:27, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.25428307338063794, 'learning_rate': 5.224e-07, 'completion_length': 145.3303680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004730224609375, 'epoch': 0.48} 48%|████▊ | 1194/2500 [7:09:18<7:33:27, 20.83s/it] 48%|████▊ | 1195/2500 [7:09:39<7:34:26, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.5379010630322559, 'learning_rate': 5.22e-07, 'completion_length': 145.5357208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0041656494140625, 'epoch': 0.48} 48%|████▊ | 1195/2500 [7:09:39<7:34:26, 20.89s/it] 48%|████▊ | 1196/2500 [7:10:00<7:37:46, 21.06s/it] {'loss': 0.0002, 'grad_norm': 1.3551911687249494, 'learning_rate': 5.215999999999999e-07, 'completion_length': 162.56250762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.005950927734375, 'epoch': 0.48} 48%|████▊ | 1196/2500 [7:10:00<7:37:46, 21.06s/it] 48%|████▊ | 1197/2500 [7:10:21<7:38:32, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.027949404076907925, 'learning_rate': 5.212e-07, 'completion_length': 144.62500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006134033203125, 'epoch': 0.48} 48%|████▊ | 1197/2500 [7:10:21<7:38:32, 21.11s/it] 48%|████▊ | 1198/2500 [7:10:42<7:35:48, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.04808569353500826, 'learning_rate': 5.208000000000001e-07, 'completion_length': 148.25000762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0056304931640625, 'epoch': 0.48} 48%|████▊ | 1198/2500 [7:10:42<7:35:48, 21.00s/it] 48%|████▊ | 1199/2500 [7:11:08<8:08:40, 22.54s/it] {'loss': 0.0002, 'grad_norm': 0.5375890971844922, 'learning_rate': 5.203999999999999e-07, 'completion_length': 162.0178680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0057525634765625, 'epoch': 0.48} 48%|████▊ | 1199/2500 [7:11:08<8:08:40, 22.54s/it] 48%|████▊ | 1200/2500 [7:11:29<7:56:42, 22.00s/it] {'loss': 0.0003, 'grad_norm': 1.0089950580476854, 'learning_rate': 5.2e-07, 'completion_length': 148.65179443359375, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.08747543022036552, 'kl': 0.007110595703125, 'epoch': 0.48} 48%|████▊ | 1200/2500 [7:11:29<7:56:42, 22.00s/it] 48%|████▊ | 1201/2500 [7:12:39<13:10:18, 36.50s/it] {'loss': 0.0002, 'grad_norm': 0.6740310887756009, 'learning_rate': 5.196e-07, 'completion_length': 141.9464340209961, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.00576019287109375, 'epoch': 0.48} 48%|████▊ | 1201/2500 [7:12:39<13:10:18, 36.50s/it] 48%|████▊ | 1202/2500 [7:13:01<11:33:41, 32.07s/it] {'loss': 0.0002, 'grad_norm': 0.2164953911938709, 'learning_rate': 5.191999999999999e-07, 'completion_length': 155.2857208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0059051513671875, 'epoch': 0.48} 48%|████▊ | 1202/2500 [7:13:01<11:33:41, 32.07s/it] 48%|████▊ | 1203/2500 [7:13:22<10:21:37, 28.76s/it] {'loss': 0.0002, 'grad_norm': 0.49663057406437133, 'learning_rate': 5.188e-07, 'completion_length': 142.0089340209961, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.05050762742757797, 'kl': 0.006011962890625, 'epoch': 0.48} 48%|████▊ | 1203/2500 [7:13:22<10:21:37, 28.76s/it] 48%|████▊ | 1204/2500 [7:13:45<9:45:05, 27.09s/it] {'loss': 0.0002, 'grad_norm': 0.5801728038013867, 'learning_rate': 5.184e-07, 'completion_length': 151.5982208251953, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.005615234375, 'epoch': 0.48} 48%|████▊ | 1204/2500 [7:13:45<9:45:05, 27.09s/it] 48%|████▊ | 1205/2500 [7:14:07<9:09:56, 25.48s/it] {'loss': 0.0002, 'grad_norm': 0.024201907782742338, 'learning_rate': 5.18e-07, 'completion_length': 150.9464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005462646484375, 'epoch': 0.48} 48%|████▊ | 1205/2500 [7:14:07<9:09:56, 25.48s/it] 48%|████▊ | 1206/2500 [7:14:31<8:59:58, 25.04s/it] {'loss': 0.0002, 'grad_norm': 0.03572374973052974, 'learning_rate': 5.175999999999999e-07, 'completion_length': 148.91964721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.48} 48%|████▊ | 1206/2500 [7:14:31<8:59:58, 25.04s/it] 48%|████▊ | 1207/2500 [7:14:52<8:35:13, 23.91s/it] {'loss': 0.0002, 'grad_norm': 0.023764670347367078, 'learning_rate': 5.172e-07, 'completion_length': 143.9732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054473876953125, 'epoch': 0.48} 48%|████▊ | 1207/2500 [7:14:52<8:35:13, 23.91s/it] 48%|████▊ | 1208/2500 [7:15:14<8:19:04, 23.18s/it] {'loss': 0.0003, 'grad_norm': 0.641889942154897, 'learning_rate': 5.168e-07, 'completion_length': 158.94644165039062, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07576143741607666, 'kl': 0.0070037841796875, 'epoch': 0.48} 48%|████▊ | 1208/2500 [7:15:14<8:19:04, 23.18s/it] 48%|████▊ | 1209/2500 [7:15:36<8:13:56, 22.96s/it] {'loss': 0.0004, 'grad_norm': 0.32829589800308057, 'learning_rate': 5.163999999999999e-07, 'completion_length': 180.0357208251953, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.05831882357597351, 'kl': 0.009063720703125, 'epoch': 0.48} 48%|████▊ | 1209/2500 [7:15:36<8:13:56, 22.96s/it] 48%|████▊ | 1210/2500 [7:15:58<8:03:59, 22.51s/it] {'loss': 0.0003, 'grad_norm': 0.19393240740508366, 'learning_rate': 5.16e-07, 'completion_length': 149.92858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00653076171875, 'epoch': 0.48} 48%|████▊ | 1210/2500 [7:15:58<8:03:59, 22.51s/it] 48%|████▊ | 1211/2500 [7:16:19<7:58:59, 22.30s/it] {'loss': 0.0003, 'grad_norm': 0.2251273819207655, 'learning_rate': 5.155999999999999e-07, 'completion_length': 155.9107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.007537841796875, 'epoch': 0.48} 48%|████▊ | 1211/2500 [7:16:19<7:58:59, 22.30s/it] 48%|████▊ | 1212/2500 [7:16:40<7:47:31, 21.78s/it] {'loss': 0.0002, 'grad_norm': 0.020774228712864575, 'learning_rate': 5.152e-07, 'completion_length': 138.6607208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00390625, 'epoch': 0.48} 48%|████▊ | 1212/2500 [7:16:40<7:47:31, 21.78s/it] 49%|████▊ | 1213/2500 [7:17:01<7:43:35, 21.61s/it] {'loss': 0.0002, 'grad_norm': 0.022412676145504157, 'learning_rate': 5.148e-07, 'completion_length': 153.14286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0046539306640625, 'epoch': 0.49} 49%|████▊ | 1213/2500 [7:17:01<7:43:35, 21.61s/it] 49%|████▊ | 1214/2500 [7:17:23<7:41:57, 21.55s/it] {'loss': 0.0002, 'grad_norm': 0.4174576381337592, 'learning_rate': 5.143999999999999e-07, 'completion_length': 159.1696548461914, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.006103515625, 'epoch': 0.49} 49%|████▊ | 1214/2500 [7:17:23<7:41:57, 21.55s/it] 49%|████▊ | 1215/2500 [7:17:45<7:45:20, 21.73s/it] {'loss': 0.0003, 'grad_norm': 0.39895724429228324, 'learning_rate': 5.14e-07, 'completion_length': 170.21429443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0077972412109375, 'epoch': 0.49} 49%|████▊ | 1215/2500 [7:17:45<7:45:20, 21.73s/it] 49%|████▊ | 1216/2500 [7:18:06<7:40:58, 21.54s/it] {'loss': 0.0003, 'grad_norm': 0.671819238555372, 'learning_rate': 5.135999999999999e-07, 'completion_length': 138.70536041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00775146484375, 'epoch': 0.49} 49%|████▊ | 1216/2500 [7:18:06<7:40:58, 21.54s/it] 49%|████▊ | 1217/2500 [7:18:28<7:42:15, 21.62s/it] {'loss': 0.0003, 'grad_norm': 0.2441995307054917, 'learning_rate': 5.132e-07, 'completion_length': 166.03572845458984, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0072784423828125, 'epoch': 0.49} 49%|████▊ | 1217/2500 [7:18:28<7:42:15, 21.62s/it] 49%|████▊ | 1218/2500 [7:18:50<7:45:44, 21.80s/it] {'loss': 0.0002, 'grad_norm': 0.02623659446528421, 'learning_rate': 5.128e-07, 'completion_length': 148.83036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055694580078125, 'epoch': 0.49} 49%|████▊ | 1218/2500 [7:18:50<7:45:44, 21.80s/it] 49%|████▉ | 1219/2500 [7:19:11<7:39:19, 21.51s/it] {'loss': 0.0002, 'grad_norm': 0.3721015992693242, 'learning_rate': 5.124e-07, 'completion_length': 139.04464721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.004852294921875, 'epoch': 0.49} 49%|████▉ | 1219/2500 [7:19:11<7:39:19, 21.51s/it] 49%|████▉ | 1220/2500 [7:19:33<7:42:13, 21.67s/it] {'loss': 0.0002, 'grad_norm': 1.9699001145441772, 'learning_rate': 5.12e-07, 'completion_length': 157.3928680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10700060427188873, 'kl': 0.0056304931640625, 'epoch': 0.49} 49%|████▉ | 1220/2500 [7:19:33<7:42:13, 21.67s/it] 49%|████▉ | 1221/2500 [7:19:55<7:45:31, 21.84s/it] {'loss': 0.0003, 'grad_norm': 0.03805124424565089, 'learning_rate': 5.116e-07, 'completion_length': 143.64286041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.006317138671875, 'epoch': 0.49} 49%|████▉ | 1221/2500 [7:19:55<7:45:31, 21.84s/it] 49%|████▉ | 1222/2500 [7:20:15<7:36:47, 21.45s/it] {'loss': 0.0002, 'grad_norm': 0.02138814749356605, 'learning_rate': 5.112e-07, 'completion_length': 137.10714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004367828369140625, 'epoch': 0.49} 49%|████▉ | 1222/2500 [7:20:15<7:36:47, 21.45s/it] 49%|████▉ | 1223/2500 [7:20:37<7:37:46, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.03769882361425603, 'learning_rate': 5.108e-07, 'completion_length': 157.7232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0074005126953125, 'epoch': 0.49} 49%|████▉ | 1223/2500 [7:20:37<7:37:46, 21.51s/it] 49%|████▉ | 1224/2500 [7:21:00<7:43:46, 21.81s/it] {'loss': 0.0004, 'grad_norm': 0.34558883137451224, 'learning_rate': 5.103999999999999e-07, 'completion_length': 168.5178680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.009521484375, 'epoch': 0.49} 49%|████▉ | 1224/2500 [7:21:00<7:43:46, 21.81s/it] 49%|████▉ | 1225/2500 [7:21:21<7:42:16, 21.75s/it] {'loss': 0.0002, 'grad_norm': 0.2039160437192479, 'learning_rate': 5.1e-07, 'completion_length': 147.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00494384765625, 'epoch': 0.49} 49%|████▉ | 1225/2500 [7:21:21<7:42:16, 21.75s/it] 49%|████▉ | 1226/2500 [7:21:44<7:45:58, 21.95s/it] {'loss': 0.0003, 'grad_norm': 0.31713622582870327, 'learning_rate': 5.096000000000001e-07, 'completion_length': 170.3482208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0067596435546875, 'epoch': 0.49} 49%|████▉ | 1226/2500 [7:21:44<7:45:58, 21.95s/it] 49%|████▉ | 1227/2500 [7:22:05<7:40:04, 21.68s/it] {'loss': 0.0002, 'grad_norm': 1.4497985156930513, 'learning_rate': 5.091999999999999e-07, 'completion_length': 158.01786041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941557407379, 'kl': 0.0062255859375, 'epoch': 0.49} 49%|████▉ | 1227/2500 [7:22:05<7:40:04, 21.68s/it] 49%|████▉ | 1228/2500 [7:22:27<7:42:09, 21.80s/it] {'loss': 0.0002, 'grad_norm': 0.31962176552405264, 'learning_rate': 5.088e-07, 'completion_length': 160.25, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00612640380859375, 'epoch': 0.49} 49%|████▉ | 1228/2500 [7:22:27<7:42:09, 21.80s/it] 49%|████▉ | 1229/2500 [7:22:48<7:40:30, 21.74s/it] {'loss': 0.0002, 'grad_norm': 0.9919628912244209, 'learning_rate': 5.084e-07, 'completion_length': 151.75000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.0055999755859375, 'epoch': 0.49} 49%|████▉ | 1229/2500 [7:22:48<7:40:30, 21.74s/it] 49%|████▉ | 1230/2500 [7:23:10<7:39:11, 21.69s/it] {'loss': 0.0002, 'grad_norm': 0.03046697633124602, 'learning_rate': 5.079999999999999e-07, 'completion_length': 154.3839340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062103271484375, 'epoch': 0.49} 49%|████▉ | 1230/2500 [7:23:10<7:39:11, 21.69s/it] 49%|████▉ | 1231/2500 [7:23:32<7:37:54, 21.65s/it] {'loss': 0.0003, 'grad_norm': 0.1965027622728278, 'learning_rate': 5.076e-07, 'completion_length': 156.67858123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0081329345703125, 'epoch': 0.49} 49%|████▉ | 1231/2500 [7:23:32<7:37:54, 21.65s/it] 49%|████▉ | 1232/2500 [7:23:53<7:36:48, 21.62s/it] {'loss': 0.0003, 'grad_norm': 0.5532086344019352, 'learning_rate': 5.072e-07, 'completion_length': 150.3482208251953, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.008148193359375, 'epoch': 0.49} 49%|████▉ | 1232/2500 [7:23:53<7:36:48, 21.62s/it] 49%|████▉ | 1233/2500 [7:24:15<7:38:08, 21.70s/it] {'loss': 0.0003, 'grad_norm': 0.8989077155744088, 'learning_rate': 5.068e-07, 'completion_length': 156.61607360839844, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.1030978113412857, 'kl': 0.00738525390625, 'epoch': 0.49} 49%|████▉ | 1233/2500 [7:24:15<7:38:08, 21.70s/it] 49%|████▉ | 1234/2500 [7:24:38<7:47:50, 22.17s/it] {'loss': 0.0002, 'grad_norm': 0.25917579210029684, 'learning_rate': 5.063999999999999e-07, 'completion_length': 147.11608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0049896240234375, 'epoch': 0.49} 49%|████▉ | 1234/2500 [7:24:38<7:47:50, 22.17s/it] 49%|████▉ | 1235/2500 [7:25:00<7:42:35, 21.94s/it] {'loss': 0.0002, 'grad_norm': 0.019601508750086635, 'learning_rate': 5.06e-07, 'completion_length': 155.11608123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051116943359375, 'epoch': 0.49} 49%|████▉ | 1235/2500 [7:25:00<7:42:35, 21.94s/it] 49%|████▉ | 1236/2500 [7:25:20<7:32:20, 21.47s/it] {'loss': 0.0001, 'grad_norm': 0.3639637530052354, 'learning_rate': 5.056e-07, 'completion_length': 136.62500762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00353240966796875, 'epoch': 0.49} 49%|████▉ | 1236/2500 [7:25:20<7:32:20, 21.47s/it] 49%|████▉ | 1237/2500 [7:25:42<7:37:38, 21.74s/it] {'loss': 0.0002, 'grad_norm': 0.2523656091817526, 'learning_rate': 5.051999999999999e-07, 'completion_length': 158.50000762939453, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.006072998046875, 'epoch': 0.49} 49%|████▉ | 1237/2500 [7:25:42<7:37:38, 21.74s/it] 50%|████▉ | 1238/2500 [7:26:03<7:31:55, 21.49s/it] {'loss': 0.0002, 'grad_norm': 0.02085122090611174, 'learning_rate': 5.048e-07, 'completion_length': 142.24108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.5} 50%|████▉ | 1238/2500 [7:26:03<7:31:55, 21.49s/it] 50%|████▉ | 1239/2500 [7:26:25<7:30:24, 21.43s/it] {'loss': 0.0003, 'grad_norm': 0.24908226920510138, 'learning_rate': 5.043999999999999e-07, 'completion_length': 160.55358123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0081634521484375, 'epoch': 0.5} 50%|████▉ | 1239/2500 [7:26:25<7:30:24, 21.43s/it] 50%|████▉ | 1240/2500 [7:26:45<7:26:26, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.22736153894197694, 'learning_rate': 5.04e-07, 'completion_length': 149.43750762939453, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0056304931640625, 'epoch': 0.5} 50%|████▉ | 1240/2500 [7:26:45<7:26:26, 21.26s/it] 50%|████▉ | 1241/2500 [7:27:07<7:30:00, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.47477325605820325, 'learning_rate': 5.036e-07, 'completion_length': 178.9107208251953, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.09528662264347076, 'kl': 0.0084228515625, 'epoch': 0.5} 50%|████▉ | 1241/2500 [7:27:07<7:30:00, 21.45s/it] 50%|████▉ | 1242/2500 [7:27:30<7:36:39, 21.78s/it] {'loss': 0.0003, 'grad_norm': 1.2161337206387446, 'learning_rate': 5.032e-07, 'completion_length': 164.31250762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.13408026099205017, 'kl': 0.0074005126953125, 'epoch': 0.5} 50%|████▉ | 1242/2500 [7:27:30<7:36:39, 21.78s/it] 50%|████▉ | 1243/2500 [7:27:51<7:32:56, 21.62s/it] {'loss': 0.0003, 'grad_norm': 0.755400519066972, 'learning_rate': 5.028e-07, 'completion_length': 147.4732208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.00687408447265625, 'epoch': 0.5} 50%|████▉ | 1243/2500 [7:27:51<7:32:56, 21.62s/it] 50%|████▉ | 1244/2500 [7:28:12<7:29:59, 21.50s/it] {'loss': 0.0002, 'grad_norm': 1.0715983163305973, 'learning_rate': 5.023999999999999e-07, 'completion_length': 149.7232208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.11394161731004715, 'kl': 0.0053863525390625, 'epoch': 0.5} 50%|████▉ | 1244/2500 [7:28:12<7:29:59, 21.50s/it] 50%|████▉ | 1245/2500 [7:28:35<7:33:49, 21.70s/it] {'loss': 0.0003, 'grad_norm': 0.5454683016341575, 'learning_rate': 5.02e-07, 'completion_length': 153.25000762939453, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.096499003469944, 'kl': 0.007965087890625, 'epoch': 0.5} 50%|████▉ | 1245/2500 [7:28:35<7:33:49, 21.70s/it] 50%|████▉ | 1246/2500 [7:28:56<7:31:20, 21.59s/it] {'loss': 0.0002, 'grad_norm': 0.024125112915130576, 'learning_rate': 5.016e-07, 'completion_length': 158.5446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.5} 50%|████▉ | 1246/2500 [7:28:56<7:31:20, 21.59s/it] 50%|████▉ | 1247/2500 [7:29:17<7:30:21, 21.57s/it] {'loss': 0.0003, 'grad_norm': 0.6698704889289364, 'learning_rate': 5.012e-07, 'completion_length': 155.4464340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.09918941184878349, 'kl': 0.00860595703125, 'epoch': 0.5} 50%|████▉ | 1247/2500 [7:29:17<7:30:21, 21.57s/it] 50%|████▉ | 1248/2500 [7:29:40<7:33:51, 21.75s/it] {'loss': 0.0003, 'grad_norm': 0.024815824734824438, 'learning_rate': 5.008e-07, 'completion_length': 180.54464721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0082855224609375, 'epoch': 0.5} 50%|████▉ | 1248/2500 [7:29:40<7:33:51, 21.75s/it] 50%|████▉ | 1249/2500 [7:30:01<7:33:51, 21.77s/it] {'loss': 0.0003, 'grad_norm': 0.21860625844617856, 'learning_rate': 5.003999999999999e-07, 'completion_length': 159.4107208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0072174072265625, 'epoch': 0.5} 50%|████▉ | 1249/2500 [7:30:01<7:33:51, 21.77s/it] 50%|█████ | 1250/2500 [7:30:23<7:32:26, 21.72s/it] {'loss': 0.0003, 'grad_norm': 0.019679065645460094, 'learning_rate': 5e-07, 'completion_length': 155.8482208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064849853515625, 'epoch': 0.5} 50%|█████ | 1250/2500 [7:30:23<7:32:26, 21.72s/it] 50%|█████ | 1251/2500 [7:30:44<7:26:41, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.4211642745387498, 'learning_rate': 4.996e-07, 'completion_length': 149.4553680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0039215087890625, 'epoch': 0.5} 50%|█████ | 1251/2500 [7:30:44<7:26:41, 21.46s/it] 50%|█████ | 1252/2500 [7:31:06<7:29:24, 21.61s/it] {'loss': 0.0003, 'grad_norm': 0.43317178606524487, 'learning_rate': 4.991999999999999e-07, 'completion_length': 164.7053680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989960193634, 'kl': 0.007110595703125, 'epoch': 0.5} 50%|█████ | 1252/2500 [7:31:06<7:29:24, 21.61s/it] 50%|█████ | 1253/2500 [7:31:29<7:38:02, 22.04s/it] {'loss': 0.0001, 'grad_norm': 0.02574692004497497, 'learning_rate': 4.988e-07, 'completion_length': 151.11608123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00365447998046875, 'epoch': 0.5} 50%|█████ | 1253/2500 [7:31:29<7:38:02, 22.04s/it] 50%|█████ | 1254/2500 [7:31:50<7:32:14, 21.78s/it] {'loss': 0.0002, 'grad_norm': 0.18126861914023246, 'learning_rate': 4.984e-07, 'completion_length': 159.02679443359375, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.0052490234375, 'epoch': 0.5} 50%|█████ | 1254/2500 [7:31:50<7:32:14, 21.78s/it] 50%|█████ | 1255/2500 [7:32:11<7:24:25, 21.42s/it] {'loss': 0.0001, 'grad_norm': 0.44143264609341604, 'learning_rate': 4.979999999999999e-07, 'completion_length': 139.79464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0032958984375, 'epoch': 0.5} 50%|█████ | 1255/2500 [7:32:11<7:24:25, 21.42s/it] 50%|█████ | 1256/2500 [7:32:32<7:22:25, 21.34s/it] {'loss': 0.0003, 'grad_norm': 0.6574325201806035, 'learning_rate': 4.976e-07, 'completion_length': 142.49108123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.0062713623046875, 'epoch': 0.5} 50%|█████ | 1256/2500 [7:32:32<7:22:25, 21.34s/it] 50%|█████ | 1257/2500 [7:32:53<7:22:16, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.5246253666586268, 'learning_rate': 4.972e-07, 'completion_length': 139.50000762939453, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.10821298509836197, 'kl': 0.0064697265625, 'epoch': 0.5} 50%|█████ | 1257/2500 [7:32:53<7:22:16, 21.35s/it] 50%|█████ | 1258/2500 [7:33:14<7:20:08, 21.26s/it] {'loss': 0.0002, 'grad_norm': 1.0602471142173147, 'learning_rate': 4.968e-07, 'completion_length': 151.4553680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0050201416015625, 'epoch': 0.5} 50%|█████ | 1258/2500 [7:33:14<7:20:08, 21.26s/it] 50%|█████ | 1259/2500 [7:33:36<7:22:27, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.3987954566195798, 'learning_rate': 4.964e-07, 'completion_length': 158.08928680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.10700060427188873, 'kl': 0.00787353515625, 'epoch': 0.5} 50%|█████ | 1259/2500 [7:33:36<7:22:27, 21.39s/it] 50%|█████ | 1260/2500 [7:33:57<7:19:47, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.5198898392655573, 'learning_rate': 4.96e-07, 'completion_length': 151.06250762939453, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.0043792724609375, 'epoch': 0.5} 50%|█████ | 1260/2500 [7:33:57<7:19:47, 21.28s/it] 50%|█████ | 1261/2500 [7:34:18<7:18:21, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.026175036766923912, 'learning_rate': 4.956e-07, 'completion_length': 153.63394165039062, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00482940673828125, 'epoch': 0.5} 50%|█████ | 1261/2500 [7:34:18<7:18:21, 21.23s/it] 50%|█████ | 1262/2500 [7:34:40<7:20:11, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.40581276108203046, 'learning_rate': 4.951999999999999e-07, 'completion_length': 157.71428680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0050048828125, 'epoch': 0.5} 50%|█████ | 1262/2500 [7:34:40<7:20:11, 21.33s/it] 51%|█████ | 1263/2500 [7:35:02<7:24:10, 21.54s/it] {'loss': 0.0003, 'grad_norm': 1.8951746352382903, 'learning_rate': 4.948e-07, 'completion_length': 159.9732208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882645100355148, 'kl': 0.0076751708984375, 'epoch': 0.51} 51%|█████ | 1263/2500 [7:35:02<7:24:10, 21.54s/it] 51%|█████ | 1264/2500 [7:35:22<7:19:32, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.2577554247184793, 'learning_rate': 4.944e-07, 'completion_length': 145.5982208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0047607421875, 'epoch': 0.51} 51%|█████ | 1264/2500 [7:35:22<7:19:32, 21.34s/it] 51%|█████ | 1265/2500 [7:35:43<7:16:34, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.3368265117458143, 'learning_rate': 4.94e-07, 'completion_length': 148.8571548461914, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0062408447265625, 'epoch': 0.51} 51%|█████ | 1265/2500 [7:35:43<7:16:34, 21.21s/it] 51%|█████ | 1266/2500 [7:36:05<7:20:05, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.730343754536279, 'learning_rate': 4.935999999999999e-07, 'completion_length': 163.9732208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.007598876953125, 'epoch': 0.51} 51%|█████ | 1266/2500 [7:36:05<7:20:05, 21.40s/it] 51%|█████ | 1267/2500 [7:36:27<7:20:32, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.768190094817881, 'learning_rate': 4.932e-07, 'completion_length': 155.04464721679688, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.096499003469944, 'kl': 0.007781982421875, 'epoch': 0.51} 51%|█████ | 1267/2500 [7:36:27<7:20:32, 21.44s/it] 51%|█████ | 1268/2500 [7:36:47<7:14:54, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.024653666716588092, 'learning_rate': 4.928e-07, 'completion_length': 145.10714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00445556640625, 'epoch': 0.51} 51%|█████ | 1268/2500 [7:36:47<7:14:54, 21.18s/it] 51%|█████ | 1269/2500 [7:37:08<7:12:27, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.33326654141720413, 'learning_rate': 4.923999999999999e-07, 'completion_length': 147.31250762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00426483154296875, 'epoch': 0.51} 51%|█████ | 1269/2500 [7:37:08<7:12:27, 21.08s/it] 51%|█████ | 1270/2500 [7:37:29<7:10:54, 21.02s/it] {'loss': 0.0003, 'grad_norm': 1.5946341254546206, 'learning_rate': 4.92e-07, 'completion_length': 158.35714721679688, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.0080413818359375, 'epoch': 0.51} 51%|█████ | 1270/2500 [7:37:29<7:10:54, 21.02s/it] 51%|█████ | 1271/2500 [7:37:50<7:07:41, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.27934069359781943, 'learning_rate': 4.916e-07, 'completion_length': 143.55358123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006195068359375, 'epoch': 0.51} 51%|█████ | 1271/2500 [7:37:50<7:07:41, 20.88s/it] 51%|█████ | 1272/2500 [7:38:10<7:07:27, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.17684726292177655, 'learning_rate': 4.912e-07, 'completion_length': 157.30357360839844, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0055084228515625, 'epoch': 0.51} 51%|█████ | 1272/2500 [7:38:10<7:07:27, 20.89s/it] 51%|█████ | 1273/2500 [7:38:31<7:05:15, 20.79s/it] {'loss': 0.0002, 'grad_norm': 0.40855340545362173, 'learning_rate': 4.908e-07, 'completion_length': 142.5714340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00376129150390625, 'epoch': 0.51} 51%|█████ | 1273/2500 [7:38:31<7:05:15, 20.79s/it] 51%|█████ | 1274/2500 [7:38:52<7:08:05, 20.95s/it] {'loss': 0.0003, 'grad_norm': 0.36745287562845014, 'learning_rate': 4.904e-07, 'completion_length': 148.05357360839844, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726372897625, 'kl': 0.00787353515625, 'epoch': 0.51} 51%|█████ | 1274/2500 [7:38:52<7:08:05, 20.95s/it] 51%|█████ | 1275/2500 [7:39:13<7:07:34, 20.94s/it] {'loss': 0.0003, 'grad_norm': 0.3343969899197465, 'learning_rate': 4.9e-07, 'completion_length': 153.33929443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0065155029296875, 'epoch': 0.51} 51%|█████ | 1275/2500 [7:39:13<7:07:34, 20.94s/it] 51%|█████ | 1276/2500 [7:39:34<7:04:26, 20.81s/it] {'loss': 0.0002, 'grad_norm': 0.39723716240395995, 'learning_rate': 4.895999999999999e-07, 'completion_length': 149.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005279541015625, 'epoch': 0.51} 51%|█████ | 1276/2500 [7:39:34<7:04:26, 20.81s/it] 51%|█████ | 1277/2500 [7:39:55<7:04:46, 20.84s/it] {'loss': 0.0003, 'grad_norm': 0.24109579578486195, 'learning_rate': 4.892e-07, 'completion_length': 149.2321548461914, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0070953369140625, 'epoch': 0.51} 51%|█████ | 1277/2500 [7:39:55<7:04:46, 20.84s/it] 51%|█████ | 1278/2500 [7:40:16<7:05:37, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.8785006548629474, 'learning_rate': 4.888e-07, 'completion_length': 146.60714721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0045623779296875, 'epoch': 0.51} 51%|█████ | 1278/2500 [7:40:16<7:05:37, 20.90s/it] 51%|█████ | 1279/2500 [7:40:37<7:07:18, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.22541631065815232, 'learning_rate': 4.884e-07, 'completion_length': 156.2053680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0046539306640625, 'epoch': 0.51} 51%|█████ | 1279/2500 [7:40:37<7:07:18, 21.00s/it] 51%|█████ | 1280/2500 [7:40:57<7:02:54, 20.80s/it] {'loss': 0.0002, 'grad_norm': 0.024811379943205526, 'learning_rate': 4.879999999999999e-07, 'completion_length': 148.54464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00445556640625, 'epoch': 0.51} 51%|█████ | 1280/2500 [7:40:57<7:02:54, 20.80s/it] 51%|█████ | 1281/2500 [7:41:19<7:09:47, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.42021955902076574, 'learning_rate': 4.876e-07, 'completion_length': 163.91072845458984, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0835726372897625, 'kl': 0.0069732666015625, 'epoch': 0.51} 51%|█████ | 1281/2500 [7:41:19<7:09:47, 21.15s/it] 51%|█████▏ | 1282/2500 [7:41:40<7:05:50, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.3545331488797681, 'learning_rate': 4.872e-07, 'completion_length': 155.83929443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.09528661891818047, 'kl': 0.005889892578125, 'epoch': 0.51} 51%|█████▏ | 1282/2500 [7:41:40<7:05:50, 20.98s/it] 51%|█████▏ | 1283/2500 [7:42:01<7:07:13, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.3275585097010663, 'learning_rate': 4.867999999999999e-07, 'completion_length': 153.5446548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004974365234375, 'epoch': 0.51} 51%|█████▏ | 1283/2500 [7:42:01<7:07:13, 21.06s/it] 51%|█████▏ | 1284/2500 [7:42:22<7:06:05, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.03780473333033309, 'learning_rate': 4.864e-07, 'completion_length': 149.6339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00476837158203125, 'epoch': 0.51} 51%|█████▏ | 1284/2500 [7:42:22<7:06:05, 21.02s/it] 51%|█████▏ | 1285/2500 [7:42:42<6:59:00, 20.69s/it] {'loss': 0.0002, 'grad_norm': 0.38717832508516814, 'learning_rate': 4.86e-07, 'completion_length': 128.4732208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00405120849609375, 'epoch': 0.51} 51%|█████▏ | 1285/2500 [7:42:42<6:59:00, 20.69s/it] 51%|█████▏ | 1286/2500 [7:43:02<6:57:27, 20.63s/it] {'loss': 0.0003, 'grad_norm': 0.5871844935376738, 'learning_rate': 4.856e-07, 'completion_length': 147.39286041259766, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.10882645472884178, 'kl': 0.006744384765625, 'epoch': 0.51} 51%|█████▏ | 1286/2500 [7:43:02<6:57:27, 20.63s/it] 51%|█████▏ | 1287/2500 [7:43:23<6:59:16, 20.74s/it] {'loss': 0.0002, 'grad_norm': 0.028236208283933653, 'learning_rate': 4.852e-07, 'completion_length': 157.0089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00537109375, 'epoch': 0.51} 51%|█████▏ | 1287/2500 [7:43:23<6:59:16, 20.74s/it] 52%|█████▏ | 1288/2500 [7:43:45<7:05:41, 21.07s/it] {'loss': 0.0004, 'grad_norm': 1.2379985030413425, 'learning_rate': 4.848e-07, 'completion_length': 150.27679443359375, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726372897625, 'kl': 0.0095367431640625, 'epoch': 0.52} 52%|█████▏ | 1288/2500 [7:43:45<7:05:41, 21.07s/it] 52%|█████▏ | 1289/2500 [7:44:06<7:03:23, 20.98s/it] {'loss': 0.0003, 'grad_norm': 0.34869489523258174, 'learning_rate': 4.844e-07, 'completion_length': 151.96429443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.00726318359375, 'epoch': 0.52} 52%|█████▏ | 1289/2500 [7:44:06<7:03:23, 20.98s/it] 52%|█████▏ | 1290/2500 [7:44:27<7:03:33, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.5224467410260997, 'learning_rate': 4.839999999999999e-07, 'completion_length': 152.4553680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.125655610114336, 'kl': 0.00482177734375, 'epoch': 0.52} 52%|█████▏ | 1290/2500 [7:44:27<7:03:33, 21.00s/it] 52%|█████▏ | 1291/2500 [7:44:47<6:58:11, 20.75s/it] {'loss': 0.0002, 'grad_norm': 2.169808740134514, 'learning_rate': 4.835999999999999e-07, 'completion_length': 126.29464721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00444793701171875, 'epoch': 0.52} 52%|█████▏ | 1291/2500 [7:44:47<6:58:11, 20.75s/it] 52%|█████▏ | 1292/2500 [7:45:08<6:54:58, 20.61s/it] {'loss': 0.0003, 'grad_norm': 0.5640947707563961, 'learning_rate': 4.832e-07, 'completion_length': 155.0446548461914, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.007781982421875, 'epoch': 0.52} 52%|█████▏ | 1292/2500 [7:45:08<6:54:58, 20.61s/it] 52%|█████▏ | 1293/2500 [7:45:28<6:54:21, 20.60s/it] {'loss': 0.0002, 'grad_norm': 0.037497092896908425, 'learning_rate': 4.828e-07, 'completion_length': 140.6607208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0057830810546875, 'epoch': 0.52} 52%|█████▏ | 1293/2500 [7:45:28<6:54:21, 20.60s/it] 52%|█████▏ | 1294/2500 [7:45:50<6:59:31, 20.87s/it] {'loss': 0.0001, 'grad_norm': 0.015981830178636226, 'learning_rate': 4.823999999999999e-07, 'completion_length': 138.71429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003570556640625, 'epoch': 0.52} 52%|█████▏ | 1294/2500 [7:45:50<6:59:31, 20.87s/it] 52%|█████▏ | 1295/2500 [7:46:11<7:04:22, 21.13s/it] {'loss': 0.0003, 'grad_norm': 1.3377500206766397, 'learning_rate': 4.82e-07, 'completion_length': 146.20536041259766, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831882357597351, 'kl': 0.006591796875, 'epoch': 0.52} 52%|█████▏ | 1295/2500 [7:46:11<7:04:22, 21.13s/it] 52%|█████▏ | 1296/2500 [7:46:33<7:07:43, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.02953437786291307, 'learning_rate': 4.816e-07, 'completion_length': 153.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062255859375, 'epoch': 0.52} 52%|█████▏ | 1296/2500 [7:46:33<7:07:43, 21.31s/it] 52%|█████▏ | 1297/2500 [7:46:55<7:11:00, 21.50s/it] {'loss': 0.0001, 'grad_norm': 0.026757850845256514, 'learning_rate': 4.812e-07, 'completion_length': 144.2589340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0035858154296875, 'epoch': 0.52} 52%|█████▏ | 1297/2500 [7:46:55<7:11:00, 21.50s/it] 52%|█████▏ | 1298/2500 [7:47:16<7:08:05, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.7518747758317136, 'learning_rate': 4.808e-07, 'completion_length': 148.9464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.003997802734375, 'epoch': 0.52} 52%|█████▏ | 1298/2500 [7:47:16<7:08:05, 21.37s/it] 52%|█████▏ | 1299/2500 [7:47:37<7:06:57, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.9870200292627226, 'learning_rate': 4.804e-07, 'completion_length': 152.85714721679688, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0072479248046875, 'epoch': 0.52} 52%|█████▏ | 1299/2500 [7:47:37<7:06:57, 21.33s/it] 52%|█████▏ | 1300/2500 [7:47:58<7:05:05, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.02340126768007766, 'learning_rate': 4.8e-07, 'completion_length': 156.6964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062255859375, 'epoch': 0.52} 52%|█████▏ | 1300/2500 [7:47:58<7:05:05, 21.25s/it] 52%|█████▏ | 1301/2500 [7:48:56<10:40:58, 32.08s/it] {'loss': 0.0002, 'grad_norm': 0.02268726512023297, 'learning_rate': 4.796e-07, 'completion_length': 152.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005035400390625, 'epoch': 0.52} 52%|█████▏ | 1301/2500 [7:48:56<10:40:58, 32.08s/it] 52%|█████▏ | 1302/2500 [7:49:17<9:37:19, 28.91s/it] {'loss': 0.0003, 'grad_norm': 0.2847207174133619, 'learning_rate': 4.792e-07, 'completion_length': 152.75000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0063934326171875, 'epoch': 0.52} 52%|█████▏ | 1302/2500 [7:49:17<9:37:19, 28.91s/it] 52%|█████▏ | 1303/2500 [7:49:39<8:52:57, 26.71s/it] {'loss': 0.0002, 'grad_norm': 0.2664383668275155, 'learning_rate': 4.788e-07, 'completion_length': 146.7857208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00604248046875, 'epoch': 0.52} 52%|█████▏ | 1303/2500 [7:49:39<8:52:57, 26.71s/it] 52%|█████▏ | 1304/2500 [7:49:59<8:16:04, 24.89s/it] {'loss': 0.0002, 'grad_norm': 0.032434566204774876, 'learning_rate': 4.783999999999999e-07, 'completion_length': 136.17858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048828125, 'epoch': 0.52} 52%|█████▏ | 1304/2500 [7:49:59<8:16:04, 24.89s/it] 52%|█████▏ | 1305/2500 [7:50:21<7:54:25, 23.82s/it] {'loss': 0.0002, 'grad_norm': 0.2664084072840202, 'learning_rate': 4.779999999999999e-07, 'completion_length': 148.50000762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0056304931640625, 'epoch': 0.52} 52%|█████▏ | 1305/2500 [7:50:21<7:54:25, 23.82s/it] 52%|█████▏ | 1306/2500 [7:50:42<7:38:57, 23.06s/it] {'loss': 0.0002, 'grad_norm': 0.026776142778318822, 'learning_rate': 4.776e-07, 'completion_length': 145.21428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00453948974609375, 'epoch': 0.52} 52%|█████▏ | 1306/2500 [7:50:42<7:38:57, 23.06s/it] 52%|█████▏ | 1307/2500 [7:51:04<7:30:44, 22.67s/it] {'loss': 0.0002, 'grad_norm': 0.36125085028657267, 'learning_rate': 4.772e-07, 'completion_length': 149.625, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0045166015625, 'epoch': 0.52} 52%|█████▏ | 1307/2500 [7:51:04<7:30:44, 22.67s/it] 52%|█████▏ | 1308/2500 [7:51:25<7:21:39, 22.23s/it] {'loss': 0.0002, 'grad_norm': 0.976546921093333, 'learning_rate': 4.768e-07, 'completion_length': 137.10714721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941557407379, 'kl': 0.0045166015625, 'epoch': 0.52} 52%|█████▏ | 1308/2500 [7:51:25<7:21:39, 22.23s/it] 52%|█████▏ | 1309/2500 [7:51:47<7:16:57, 22.01s/it] {'loss': 0.0003, 'grad_norm': 0.4234905234974408, 'learning_rate': 4.7639999999999995e-07, 'completion_length': 151.49107360839844, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.08747543022036552, 'kl': 0.0072784423828125, 'epoch': 0.52} 52%|█████▏ | 1309/2500 [7:51:47<7:16:57, 22.01s/it] 52%|█████▏ | 1310/2500 [7:52:09<7:18:06, 22.09s/it] {'loss': 0.0002, 'grad_norm': 0.577208180716338, 'learning_rate': 4.76e-07, 'completion_length': 146.3303680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.52} 52%|█████▏ | 1310/2500 [7:52:09<7:18:06, 22.09s/it] 52%|█████▏ | 1311/2500 [7:52:33<7:27:02, 22.56s/it] {'loss': 0.0003, 'grad_norm': 0.718208126472397, 'learning_rate': 4.756e-07, 'completion_length': 158.17858123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0063629150390625, 'epoch': 0.52} 52%|█████▏ | 1311/2500 [7:52:33<7:27:02, 22.56s/it] 52%|█████▏ | 1312/2500 [7:52:54<7:17:31, 22.10s/it] {'loss': 0.0003, 'grad_norm': 0.47927232197380665, 'learning_rate': 4.7519999999999997e-07, 'completion_length': 140.9375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.006439208984375, 'epoch': 0.52} 52%|█████▏ | 1312/2500 [7:52:54<7:17:31, 22.10s/it] 53%|█████▎ | 1313/2500 [7:53:15<7:14:27, 21.96s/it] {'loss': 0.0002, 'grad_norm': 0.30847913652184084, 'learning_rate': 4.748e-07, 'completion_length': 160.30357360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005767822265625, 'epoch': 0.53} 53%|█████▎ | 1313/2500 [7:53:15<7:14:27, 21.96s/it] 53%|█████▎ | 1314/2500 [7:53:37<7:13:43, 21.94s/it] {'loss': 0.0003, 'grad_norm': 0.5430463263374603, 'learning_rate': 4.7439999999999996e-07, 'completion_length': 163.73214721679688, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0066070556640625, 'epoch': 0.53} 53%|█████▎ | 1314/2500 [7:53:37<7:13:43, 21.94s/it] 53%|█████▎ | 1315/2500 [7:53:59<7:10:25, 21.79s/it] {'loss': 0.0003, 'grad_norm': 0.43112357883358676, 'learning_rate': 4.7399999999999993e-07, 'completion_length': 164.0, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.11272924020886421, 'kl': 0.0072021484375, 'epoch': 0.53} 53%|█████▎ | 1315/2500 [7:53:59<7:10:25, 21.79s/it] 53%|█████▎ | 1316/2500 [7:54:20<7:07:26, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.5176044339544004, 'learning_rate': 4.736e-07, 'completion_length': 154.65179443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0059661865234375, 'epoch': 0.53} 53%|█████▎ | 1316/2500 [7:54:20<7:07:26, 21.66s/it] 53%|█████▎ | 1317/2500 [7:54:42<7:08:31, 21.73s/it] {'loss': 0.0003, 'grad_norm': 0.676256392128925, 'learning_rate': 4.732e-07, 'completion_length': 159.58929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.10040179640054703, 'kl': 0.00782012939453125, 'epoch': 0.53} 53%|█████▎ | 1317/2500 [7:54:42<7:08:31, 21.73s/it] 53%|█████▎ | 1318/2500 [7:55:03<7:06:26, 21.65s/it] {'loss': 0.0003, 'grad_norm': 0.7032412691040816, 'learning_rate': 4.728e-07, 'completion_length': 144.81250762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0068511962890625, 'epoch': 0.53} 53%|█████▎ | 1318/2500 [7:55:03<7:06:26, 21.65s/it] 53%|█████▎ | 1319/2500 [7:55:24<7:03:42, 21.53s/it] {'loss': 0.0003, 'grad_norm': 0.481549583750109, 'learning_rate': 4.7239999999999997e-07, 'completion_length': 147.4107208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.007354736328125, 'epoch': 0.53} 53%|█████▎ | 1319/2500 [7:55:24<7:03:42, 21.53s/it] 53%|█████▎ | 1320/2500 [7:55:47<7:06:35, 21.69s/it] {'loss': 0.0002, 'grad_norm': 0.17752180552510616, 'learning_rate': 4.7199999999999994e-07, 'completion_length': 147.99108123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006072998046875, 'epoch': 0.53} 53%|█████▎ | 1320/2500 [7:55:47<7:06:35, 21.69s/it] 53%|█████▎ | 1321/2500 [7:56:14<7:41:28, 23.49s/it] {'loss': 0.0003, 'grad_norm': 0.3148708981158463, 'learning_rate': 4.716e-07, 'completion_length': 143.64286041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.00665283203125, 'epoch': 0.53} 53%|█████▎ | 1321/2500 [7:56:14<7:41:28, 23.49s/it] 53%|█████▎ | 1322/2500 [7:56:35<7:25:50, 22.71s/it] {'loss': 0.0003, 'grad_norm': 0.34273828190295264, 'learning_rate': 4.712e-07, 'completion_length': 153.01786041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.006683349609375, 'epoch': 0.53} 53%|█████▎ | 1322/2500 [7:56:35<7:25:50, 22.71s/it] 53%|█████▎ | 1323/2500 [7:56:57<7:20:22, 22.45s/it] {'loss': 0.0002, 'grad_norm': 0.025797605011028287, 'learning_rate': 4.7079999999999995e-07, 'completion_length': 156.29464721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0055084228515625, 'epoch': 0.53} 53%|█████▎ | 1323/2500 [7:56:57<7:20:22, 22.45s/it] 53%|█████▎ | 1324/2500 [7:57:18<7:11:04, 21.99s/it] {'loss': 0.0002, 'grad_norm': 0.4803045909814044, 'learning_rate': 4.704e-07, 'completion_length': 139.74108123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543394565582, 'kl': 0.00411224365234375, 'epoch': 0.53} 53%|█████▎ | 1324/2500 [7:57:18<7:11:04, 21.99s/it] 53%|█████▎ | 1325/2500 [7:57:39<7:04:39, 21.68s/it] {'loss': 0.0003, 'grad_norm': 0.20750459843666466, 'learning_rate': 4.6999999999999995e-07, 'completion_length': 145.9464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00701904296875, 'epoch': 0.53} 53%|█████▎ | 1325/2500 [7:57:39<7:04:39, 21.68s/it] 53%|█████▎ | 1326/2500 [7:58:01<7:04:56, 21.72s/it] {'loss': 0.0002, 'grad_norm': 0.18009955347287152, 'learning_rate': 4.6959999999999997e-07, 'completion_length': 146.0982208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0052490234375, 'epoch': 0.53} 53%|█████▎ | 1326/2500 [7:58:01<7:04:56, 21.72s/it] 53%|█████▎ | 1327/2500 [7:58:23<7:05:41, 21.77s/it] {'loss': 0.0003, 'grad_norm': 0.8081266929121311, 'learning_rate': 4.692e-07, 'completion_length': 162.6339340209961, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06613001227378845, 'kl': 0.00634765625, 'epoch': 0.53} 53%|█████▎ | 1327/2500 [7:58:23<7:05:41, 21.77s/it] 53%|█████▎ | 1328/2500 [7:58:44<7:04:27, 21.73s/it] {'loss': 0.0003, 'grad_norm': 0.22752283432906673, 'learning_rate': 4.6879999999999996e-07, 'completion_length': 156.80358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0075531005859375, 'epoch': 0.53} 53%|█████▎ | 1328/2500 [7:58:44<7:04:27, 21.73s/it] 53%|█████▎ | 1329/2500 [7:59:05<6:59:20, 21.49s/it] {'loss': 0.0002, 'grad_norm': 0.22752673878461754, 'learning_rate': 4.684e-07, 'completion_length': 142.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00446319580078125, 'epoch': 0.53} 53%|█████▎ | 1329/2500 [7:59:05<6:59:20, 21.49s/it] 53%|█████▎ | 1330/2500 [7:59:27<7:00:15, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.33513436304240574, 'learning_rate': 4.68e-07, 'completion_length': 159.62500762939453, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0073699951171875, 'epoch': 0.53} 53%|█████▎ | 1330/2500 [7:59:27<7:00:15, 21.55s/it] 53%|█████▎ | 1331/2500 [7:59:49<7:05:29, 21.84s/it] {'loss': 0.0002, 'grad_norm': 0.5154521259940061, 'learning_rate': 4.676e-07, 'completion_length': 159.86608123779297, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0054931640625, 'epoch': 0.53} 53%|█████▎ | 1331/2500 [7:59:49<7:05:29, 21.84s/it] 53%|█████▎ | 1332/2500 [8:00:11<7:05:10, 21.84s/it] {'loss': 0.0002, 'grad_norm': 0.275827635096674, 'learning_rate': 4.672e-07, 'completion_length': 149.64286041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00539398193359375, 'epoch': 0.53} 53%|█████▎ | 1332/2500 [8:00:11<7:05:10, 21.84s/it] 53%|█████▎ | 1333/2500 [8:00:32<7:01:04, 21.65s/it] {'loss': 0.0003, 'grad_norm': 1.0829030586068338, 'learning_rate': 4.6679999999999997e-07, 'completion_length': 160.8303680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.0068511962890625, 'epoch': 0.53} 53%|█████▎ | 1333/2500 [8:00:32<7:01:04, 21.65s/it] 53%|█████▎ | 1334/2500 [8:00:53<6:56:43, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.27512983785580186, 'learning_rate': 4.6639999999999994e-07, 'completion_length': 147.9196548461914, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0060272216796875, 'epoch': 0.53} 53%|█████▎ | 1334/2500 [8:00:53<6:56:43, 21.44s/it] 53%|█████▎ | 1335/2500 [8:01:14<6:51:44, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.46456187183388126, 'learning_rate': 4.66e-07, 'completion_length': 146.12500762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.03696779906749725, 'kl': 0.0070037841796875, 'epoch': 0.53} 53%|█████▎ | 1335/2500 [8:01:14<6:51:44, 21.21s/it] 53%|█████▎ | 1336/2500 [8:01:35<6:49:35, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.017825257092724638, 'learning_rate': 4.656e-07, 'completion_length': 151.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049591064453125, 'epoch': 0.53} 53%|█████▎ | 1336/2500 [8:01:35<6:49:35, 21.11s/it] 53%|█████▎ | 1337/2500 [8:01:55<6:45:42, 20.93s/it] {'loss': 0.0003, 'grad_norm': 0.4974302248851713, 'learning_rate': 4.6519999999999996e-07, 'completion_length': 154.02678680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.0066986083984375, 'epoch': 0.53} 53%|█████▎ | 1337/2500 [8:01:55<6:45:42, 20.93s/it] 54%|█████▎ | 1338/2500 [8:02:17<6:46:45, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.3543083487586188, 'learning_rate': 4.648e-07, 'completion_length': 142.70536041259766, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.07003280520439148, 'kl': 0.00482177734375, 'epoch': 0.54} 54%|█████▎ | 1338/2500 [8:02:17<6:46:45, 21.00s/it] 54%|█████▎ | 1339/2500 [8:02:38<6:48:49, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.30219132406107146, 'learning_rate': 4.6439999999999995e-07, 'completion_length': 141.7589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0053863525390625, 'epoch': 0.54} 54%|█████▎ | 1339/2500 [8:02:38<6:48:49, 21.13s/it] 54%|█████▎ | 1340/2500 [8:02:59<6:50:50, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.01676550532408782, 'learning_rate': 4.64e-07, 'completion_length': 135.66964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00518798828125, 'epoch': 0.54} 54%|█████▎ | 1340/2500 [8:02:59<6:50:50, 21.25s/it] 54%|█████▎ | 1341/2500 [8:03:21<6:54:09, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.01708082446135516, 'learning_rate': 4.636e-07, 'completion_length': 154.5357208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.007476806640625, 'epoch': 0.54} 54%|█████▎ | 1341/2500 [8:03:21<6:54:09, 21.44s/it] 54%|█████▎ | 1342/2500 [8:03:42<6:49:34, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.6187718080403314, 'learning_rate': 4.6319999999999997e-07, 'completion_length': 148.1160774230957, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00600433349609375, 'epoch': 0.54} 54%|█████▎ | 1342/2500 [8:03:42<6:49:34, 21.22s/it] 54%|█████▎ | 1343/2500 [8:04:04<6:52:03, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.3964719630185974, 'learning_rate': 4.628e-07, 'completion_length': 140.06250762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0051422119140625, 'epoch': 0.54} 54%|█████▎ | 1343/2500 [8:04:04<6:52:03, 21.37s/it] 54%|█████▍ | 1344/2500 [8:04:25<6:50:03, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.5376952320426127, 'learning_rate': 4.6239999999999996e-07, 'completion_length': 157.43750762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0053253173828125, 'epoch': 0.54} 54%|█████▍ | 1344/2500 [8:04:25<6:50:03, 21.28s/it] 54%|█████▍ | 1345/2500 [8:04:46<6:49:23, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.028409976573802245, 'learning_rate': 4.62e-07, 'completion_length': 153.7678680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0071868896484375, 'epoch': 0.54} 54%|█████▍ | 1345/2500 [8:04:46<6:49:23, 21.27s/it] 54%|█████▍ | 1346/2500 [8:05:07<6:44:53, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.04379459002627243, 'learning_rate': 4.616e-07, 'completion_length': 138.70536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00441741943359375, 'epoch': 0.54} 54%|█████▍ | 1346/2500 [8:05:07<6:44:53, 21.05s/it] 54%|█████▍ | 1347/2500 [8:05:28<6:45:51, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.019417595268091288, 'learning_rate': 4.612e-07, 'completion_length': 161.96429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00612640380859375, 'epoch': 0.54} 54%|█████▍ | 1347/2500 [8:05:28<6:45:51, 21.12s/it] 54%|█████▍ | 1348/2500 [8:05:49<6:47:10, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.25259801605280097, 'learning_rate': 4.6079999999999994e-07, 'completion_length': 151.71428680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0044097900390625, 'epoch': 0.54} 54%|█████▍ | 1348/2500 [8:05:49<6:47:10, 21.21s/it] 54%|█████▍ | 1349/2500 [8:06:10<6:44:57, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.21182980902410437, 'learning_rate': 4.6039999999999997e-07, 'completion_length': 141.99108123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0057220458984375, 'epoch': 0.54} 54%|█████▍ | 1349/2500 [8:06:10<6:44:57, 21.11s/it] 54%|█████▍ | 1350/2500 [8:06:31<6:44:16, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.5558979333611921, 'learning_rate': 4.6e-07, 'completion_length': 151.58036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00516510009765625, 'epoch': 0.54} 54%|█████▍ | 1350/2500 [8:06:31<6:44:16, 21.09s/it] 54%|█████▍ | 1351/2500 [8:06:52<6:43:43, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.7768624718974724, 'learning_rate': 4.596e-07, 'completion_length': 146.12500762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0045318603515625, 'epoch': 0.54} 54%|█████▍ | 1351/2500 [8:06:52<6:43:43, 21.08s/it] 54%|█████▍ | 1352/2500 [8:07:14<6:44:51, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.584792148060583, 'learning_rate': 4.592e-07, 'completion_length': 153.27678680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0082855224609375, 'epoch': 0.54} 54%|█████▍ | 1352/2500 [8:07:14<6:44:51, 21.16s/it] 54%|█████▍ | 1353/2500 [8:07:37<6:54:54, 21.70s/it] {'loss': 0.0003, 'grad_norm': 0.4375480269813004, 'learning_rate': 4.5879999999999995e-07, 'completion_length': 160.10714721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941557407379, 'kl': 0.0064697265625, 'epoch': 0.54} 54%|█████▍ | 1353/2500 [8:07:37<6:54:54, 21.70s/it] 54%|█████▍ | 1354/2500 [8:07:58<6:50:33, 21.50s/it] {'loss': 0.0002, 'grad_norm': 0.4091893505753727, 'learning_rate': 4.584e-07, 'completion_length': 143.54464721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0044708251953125, 'epoch': 0.54} 54%|█████▍ | 1354/2500 [8:07:58<6:50:33, 21.50s/it] 54%|█████▍ | 1355/2500 [8:08:20<6:54:48, 21.74s/it] {'loss': 0.0003, 'grad_norm': 0.13676996505134084, 'learning_rate': 4.58e-07, 'completion_length': 152.25, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00701904296875, 'epoch': 0.54} 54%|█████▍ | 1355/2500 [8:08:20<6:54:48, 21.74s/it] 54%|█████▍ | 1356/2500 [8:08:45<7:12:37, 22.69s/it] {'loss': 0.0002, 'grad_norm': 0.015171064422385616, 'learning_rate': 4.5759999999999997e-07, 'completion_length': 146.25000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00392913818359375, 'epoch': 0.54} 54%|█████▍ | 1356/2500 [8:08:45<7:12:37, 22.69s/it] 54%|█████▍ | 1357/2500 [8:09:09<7:19:08, 23.05s/it] {'loss': 0.0003, 'grad_norm': 0.3124748395705717, 'learning_rate': 4.572e-07, 'completion_length': 149.39286041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0063934326171875, 'epoch': 0.54} 54%|█████▍ | 1357/2500 [8:09:09<7:19:08, 23.05s/it] 54%|█████▍ | 1358/2500 [8:09:31<7:15:04, 22.86s/it] {'loss': 0.0002, 'grad_norm': 0.5578063944290773, 'learning_rate': 4.5679999999999996e-07, 'completion_length': 142.10715103149414, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00464630126953125, 'epoch': 0.54} 54%|█████▍ | 1358/2500 [8:09:31<7:15:04, 22.86s/it] 54%|█████▍ | 1359/2500 [8:09:53<7:10:30, 22.64s/it] {'loss': 0.0002, 'grad_norm': 0.8799245705290826, 'learning_rate': 4.5639999999999993e-07, 'completion_length': 153.46429443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0056610107421875, 'epoch': 0.54} 54%|█████▍ | 1359/2500 [8:09:53<7:10:30, 22.64s/it] 54%|█████▍ | 1360/2500 [8:10:15<7:05:03, 22.37s/it] {'loss': 0.0002, 'grad_norm': 1.7042147359352335, 'learning_rate': 4.56e-07, 'completion_length': 144.0089340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0057830810546875, 'epoch': 0.54} 54%|█████▍ | 1360/2500 [8:10:15<7:05:03, 22.37s/it] 54%|█████▍ | 1361/2500 [8:10:39<7:14:14, 22.87s/it] {'loss': 0.0002, 'grad_norm': 0.36840089770477763, 'learning_rate': 4.556e-07, 'completion_length': 141.30358123779297, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06222161650657654, 'kl': 0.00612640380859375, 'epoch': 0.54} 54%|█████▍ | 1361/2500 [8:10:39<7:14:14, 22.87s/it] 54%|█████▍ | 1362/2500 [8:11:02<7:13:02, 22.83s/it] {'loss': 0.0003, 'grad_norm': 0.6138745850313825, 'learning_rate': 4.5519999999999995e-07, 'completion_length': 164.58036041259766, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.09138382598757744, 'kl': 0.0081787109375, 'epoch': 0.54} 54%|█████▍ | 1362/2500 [8:11:02<7:13:02, 22.83s/it] 55%|█████▍ | 1363/2500 [8:11:24<7:06:46, 22.52s/it] {'loss': 0.0003, 'grad_norm': 0.849054563231397, 'learning_rate': 4.5479999999999997e-07, 'completion_length': 156.23214721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0066986083984375, 'epoch': 0.55} 55%|█████▍ | 1363/2500 [8:11:24<7:06:46, 22.52s/it] 55%|█████▍ | 1364/2500 [8:11:46<7:06:09, 22.51s/it] {'loss': 0.0002, 'grad_norm': 0.5014338991485349, 'learning_rate': 4.544e-07, 'completion_length': 147.1607208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0056610107421875, 'epoch': 0.55} 55%|█████▍ | 1364/2500 [8:11:46<7:06:09, 22.51s/it] 55%|█████▍ | 1365/2500 [8:12:08<7:00:55, 22.25s/it] {'loss': 0.0002, 'grad_norm': 0.01804865605264475, 'learning_rate': 4.54e-07, 'completion_length': 150.17857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0061798095703125, 'epoch': 0.55} 55%|█████▍ | 1365/2500 [8:12:08<7:00:55, 22.25s/it] 55%|█████▍ | 1366/2500 [8:12:28<6:50:59, 21.75s/it] {'loss': 0.0002, 'grad_norm': 0.31769885875883885, 'learning_rate': 4.536e-07, 'completion_length': 138.5982208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.0046234130859375, 'epoch': 0.55} 55%|█████▍ | 1366/2500 [8:12:28<6:50:59, 21.75s/it] 55%|█████▍ | 1367/2500 [8:12:50<6:49:18, 21.68s/it] {'loss': 0.0002, 'grad_norm': 0.6302084224127918, 'learning_rate': 4.5319999999999996e-07, 'completion_length': 144.3214340209961, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.12054044008255005, 'kl': 0.005401611328125, 'epoch': 0.55} 55%|█████▍ | 1367/2500 [8:12:50<6:49:18, 21.68s/it] 55%|█████▍ | 1368/2500 [8:13:12<6:52:45, 21.88s/it] {'loss': 0.0003, 'grad_norm': 0.7249458400069976, 'learning_rate': 4.528e-07, 'completion_length': 166.02678680419922, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0074462890625, 'epoch': 0.55} 55%|█████▍ | 1368/2500 [8:13:12<6:52:45, 21.88s/it] 55%|█████▍ | 1369/2500 [8:13:33<6:47:50, 21.64s/it] {'loss': 0.0003, 'grad_norm': 0.19365657485078525, 'learning_rate': 4.524e-07, 'completion_length': 148.1339340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.006866455078125, 'epoch': 0.55} 55%|█████▍ | 1369/2500 [8:13:33<6:47:50, 21.64s/it] 55%|█████▍ | 1370/2500 [8:13:55<6:47:51, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.31652913164813057, 'learning_rate': 4.5199999999999997e-07, 'completion_length': 157.41964721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06613001227378845, 'kl': 0.0052642822265625, 'epoch': 0.55} 55%|█████▍ | 1370/2500 [8:13:55<6:47:51, 21.66s/it] 55%|█████▍ | 1371/2500 [8:14:16<6:43:43, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.034907300141554805, 'learning_rate': 4.516e-07, 'completion_length': 152.7589340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0072021484375, 'epoch': 0.55} 55%|█████▍ | 1371/2500 [8:14:16<6:43:43, 21.46s/it] 55%|█████▍ | 1372/2500 [8:14:36<6:37:41, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.028035639395802923, 'learning_rate': 4.5119999999999996e-07, 'completion_length': 135.85714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062103271484375, 'epoch': 0.55} 55%|█████▍ | 1372/2500 [8:14:36<6:37:41, 21.15s/it] 55%|█████▍ | 1373/2500 [8:14:58<6:41:07, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.48825400763872906, 'learning_rate': 4.5079999999999993e-07, 'completion_length': 159.01786041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726372897625, 'kl': 0.0066986083984375, 'epoch': 0.55} 55%|█████▍ | 1373/2500 [8:14:58<6:41:07, 21.36s/it] 55%|█████▍ | 1374/2500 [8:15:20<6:41:13, 21.38s/it] {'loss': 0.0003, 'grad_norm': 0.4120943633866375, 'learning_rate': 4.504e-07, 'completion_length': 161.92857360839844, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.007232666015625, 'epoch': 0.55} 55%|█████▍ | 1374/2500 [8:15:20<6:41:13, 21.38s/it] 55%|█████▌ | 1375/2500 [8:15:41<6:38:25, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.01963136382049017, 'learning_rate': 4.5e-07, 'completion_length': 145.36607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003753662109375, 'epoch': 0.55} 55%|█████▌ | 1375/2500 [8:15:41<6:38:25, 21.25s/it] 55%|█████▌ | 1376/2500 [8:16:01<6:33:48, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.34942078632957735, 'learning_rate': 4.496e-07, 'completion_length': 143.26786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0046539306640625, 'epoch': 0.55} 55%|█████▌ | 1376/2500 [8:16:01<6:33:48, 21.02s/it] 55%|█████▌ | 1377/2500 [8:16:22<6:32:09, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.7059175333805163, 'learning_rate': 4.4919999999999997e-07, 'completion_length': 146.4732208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048980712890625, 'epoch': 0.55} 55%|█████▌ | 1377/2500 [8:16:22<6:32:09, 20.95s/it] 55%|█████▌ | 1378/2500 [8:16:43<6:30:24, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.4497427532838779, 'learning_rate': 4.4879999999999994e-07, 'completion_length': 142.9732208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005096435546875, 'epoch': 0.55} 55%|█████▌ | 1378/2500 [8:16:43<6:30:24, 20.88s/it] 55%|█████▌ | 1379/2500 [8:17:04<6:31:07, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.9947037195663668, 'learning_rate': 4.484e-07, 'completion_length': 143.65179443359375, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.0060882568359375, 'epoch': 0.55} 55%|█████▌ | 1379/2500 [8:17:04<6:31:07, 20.93s/it] 55%|█████▌ | 1380/2500 [8:17:24<6:28:41, 20.82s/it] {'loss': 0.0003, 'grad_norm': 0.7725038713740989, 'learning_rate': 4.48e-07, 'completion_length': 140.5714340209961, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0072479248046875, 'epoch': 0.55} 55%|█████▌ | 1380/2500 [8:17:24<6:28:41, 20.82s/it] 55%|█████▌ | 1381/2500 [8:17:45<6:29:30, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.3861426689622674, 'learning_rate': 4.4759999999999996e-07, 'completion_length': 151.6696548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00399017333984375, 'epoch': 0.55} 55%|█████▌ | 1381/2500 [8:17:45<6:29:30, 20.89s/it] 55%|█████▌ | 1382/2500 [8:18:06<6:30:09, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.2564572776514721, 'learning_rate': 4.472e-07, 'completion_length': 149.625, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0050201416015625, 'epoch': 0.55} 55%|█████▌ | 1382/2500 [8:18:06<6:30:09, 20.94s/it] 55%|█████▌ | 1383/2500 [8:18:27<6:30:27, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.04019513676320458, 'learning_rate': 4.4679999999999995e-07, 'completion_length': 147.5714340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.55} 55%|█████▌ | 1383/2500 [8:18:27<6:30:27, 20.97s/it] 55%|█████▌ | 1384/2500 [8:18:48<6:29:42, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.034970305014176264, 'learning_rate': 4.464e-07, 'completion_length': 149.42858123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00592041015625, 'epoch': 0.55} 55%|█████▌ | 1384/2500 [8:18:48<6:29:42, 20.95s/it] 55%|█████▌ | 1385/2500 [8:19:09<6:29:45, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.7348791205431108, 'learning_rate': 4.46e-07, 'completion_length': 161.61608123779297, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00506591796875, 'epoch': 0.55} 55%|█████▌ | 1385/2500 [8:19:09<6:29:45, 20.97s/it] 55%|█████▌ | 1386/2500 [8:19:30<6:29:23, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.6096359391478028, 'learning_rate': 4.4559999999999997e-07, 'completion_length': 155.29464721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0061187744140625, 'epoch': 0.55} 55%|█████▌ | 1386/2500 [8:19:30<6:29:23, 20.97s/it] 55%|█████▌ | 1387/2500 [8:19:51<6:25:00, 20.76s/it] {'loss': 0.0002, 'grad_norm': 0.6558857189698305, 'learning_rate': 4.452e-07, 'completion_length': 145.00000762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.006134033203125, 'epoch': 0.55} 55%|█████▌ | 1387/2500 [8:19:51<6:25:00, 20.76s/it] 56%|█████▌ | 1388/2500 [8:20:12<6:29:10, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.5019782408705752, 'learning_rate': 4.4479999999999996e-07, 'completion_length': 157.2857208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0065460205078125, 'epoch': 0.56} 56%|█████▌ | 1388/2500 [8:20:12<6:29:10, 21.00s/it] 56%|█████▌ | 1389/2500 [8:20:33<6:29:00, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.6949352101253387, 'learning_rate': 4.444e-07, 'completion_length': 152.23214721679688, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0051422119140625, 'epoch': 0.56} 56%|█████▌ | 1389/2500 [8:20:33<6:29:00, 21.01s/it] 56%|█████▌ | 1390/2500 [8:20:54<6:26:40, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.01976215252440743, 'learning_rate': 4.44e-07, 'completion_length': 161.71428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.56} 56%|█████▌ | 1390/2500 [8:20:54<6:26:40, 20.90s/it] 56%|█████▌ | 1391/2500 [8:21:15<6:26:01, 20.89s/it] {'loss': 0.0001, 'grad_norm': 0.5283302227030542, 'learning_rate': 4.436e-07, 'completion_length': 143.42858123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00362396240234375, 'epoch': 0.56} 56%|█████▌ | 1391/2500 [8:21:15<6:26:01, 20.89s/it] 56%|█████▌ | 1392/2500 [8:21:36<6:28:58, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.03347329169401106, 'learning_rate': 4.4319999999999995e-07, 'completion_length': 153.00000762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.56} 56%|█████▌ | 1392/2500 [8:21:36<6:28:58, 21.06s/it] 56%|█████▌ | 1393/2500 [8:21:57<6:26:59, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.02194848319758044, 'learning_rate': 4.428e-07, 'completion_length': 146.73214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004364013671875, 'epoch': 0.56} 56%|█████▌ | 1393/2500 [8:21:57<6:26:59, 20.97s/it] 56%|█████▌ | 1394/2500 [8:22:18<6:29:03, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.698075096053777, 'learning_rate': 4.424e-07, 'completion_length': 169.69644165039062, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.10700060427188873, 'kl': 0.0067596435546875, 'epoch': 0.56} 56%|█████▌ | 1394/2500 [8:22:18<6:29:03, 21.11s/it] 56%|█████▌ | 1395/2500 [8:22:39<6:26:47, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.03339615176031803, 'learning_rate': 4.4199999999999996e-07, 'completion_length': 142.90179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003875732421875, 'epoch': 0.56} 56%|█████▌ | 1395/2500 [8:22:39<6:26:47, 21.00s/it] 56%|█████▌ | 1396/2500 [8:23:00<6:27:08, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.020080112883241505, 'learning_rate': 4.416e-07, 'completion_length': 146.68750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00482940673828125, 'epoch': 0.56} 56%|█████▌ | 1396/2500 [8:23:00<6:27:08, 21.04s/it] 56%|█████▌ | 1397/2500 [8:23:22<6:30:15, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.30114270141197513, 'learning_rate': 4.4119999999999995e-07, 'completion_length': 171.25000762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005706787109375, 'epoch': 0.56} 56%|█████▌ | 1397/2500 [8:23:22<6:30:15, 21.23s/it] 56%|█████▌ | 1398/2500 [8:23:43<6:29:08, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.7037097858523723, 'learning_rate': 4.4080000000000003e-07, 'completion_length': 148.5, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.10700060427188873, 'kl': 0.0078277587890625, 'epoch': 0.56} 56%|█████▌ | 1398/2500 [8:23:43<6:29:08, 21.19s/it] 56%|█████▌ | 1399/2500 [8:24:04<6:26:05, 21.04s/it] {'loss': 0.0003, 'grad_norm': 0.23129564780055994, 'learning_rate': 4.404e-07, 'completion_length': 146.7678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0066680908203125, 'epoch': 0.56} 56%|█████▌ | 1399/2500 [8:24:04<6:26:05, 21.04s/it] 56%|█████▌ | 1400/2500 [8:24:25<6:25:22, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.4737711611884783, 'learning_rate': 4.3999999999999997e-07, 'completion_length': 143.62500762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0048370361328125, 'epoch': 0.56} 56%|█████▌ | 1400/2500 [8:24:25<6:25:22, 21.02s/it] 56%|█████▌ | 1401/2500 [8:25:45<11:48:57, 38.71s/it] {'loss': 0.0002, 'grad_norm': 0.8904092468161376, 'learning_rate': 4.396e-07, 'completion_length': 140.52678680419922, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.006072998046875, 'epoch': 0.56} 56%|█████▌ | 1401/2500 [8:25:45<11:48:57, 38.71s/it] 56%|█████▌ | 1402/2500 [8:26:05<10:09:12, 33.29s/it] {'loss': 0.0004, 'grad_norm': 0.3875764809710827, 'learning_rate': 4.3919999999999996e-07, 'completion_length': 135.3571548461914, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.03696779906749725, 'kl': 0.0089111328125, 'epoch': 0.56} 56%|█████▌ | 1402/2500 [8:26:05<10:09:12, 33.29s/it] 56%|█████▌ | 1403/2500 [8:26:26<9:01:28, 29.62s/it] {'loss': 0.0001, 'grad_norm': 0.015996523157688926, 'learning_rate': 4.388e-07, 'completion_length': 141.83929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0028076171875, 'epoch': 0.56} 56%|█████▌ | 1403/2500 [8:26:26<9:01:28, 29.62s/it] 56%|█████▌ | 1404/2500 [8:26:48<8:15:44, 27.14s/it] {'loss': 0.0002, 'grad_norm': 0.21525141662781058, 'learning_rate': 4.384e-07, 'completion_length': 152.18750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00545501708984375, 'epoch': 0.56} 56%|█████▌ | 1404/2500 [8:26:48<8:15:44, 27.14s/it] 56%|█████▌ | 1405/2500 [8:27:09<7:43:11, 25.38s/it] {'loss': 0.0003, 'grad_norm': 0.6861751014328984, 'learning_rate': 4.38e-07, 'completion_length': 157.08929443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0064239501953125, 'epoch': 0.56} 56%|█████▌ | 1405/2500 [8:27:09<7:43:11, 25.38s/it] 56%|█████▌ | 1406/2500 [8:27:30<7:18:50, 24.07s/it] {'loss': 0.0002, 'grad_norm': 0.2831585312160102, 'learning_rate': 4.3759999999999995e-07, 'completion_length': 153.36608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0044403076171875, 'epoch': 0.56} 56%|█████▌ | 1406/2500 [8:27:30<7:18:50, 24.07s/it] 56%|█████▋ | 1407/2500 [8:27:50<6:58:50, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.7160015777008152, 'learning_rate': 4.3719999999999997e-07, 'completion_length': 155.43750762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.0047760009765625, 'epoch': 0.56} 56%|█████▋ | 1407/2500 [8:27:50<6:58:50, 22.99s/it] 56%|█████▋ | 1408/2500 [8:28:11<6:43:28, 22.17s/it] {'loss': 0.0003, 'grad_norm': 0.4501016207895249, 'learning_rate': 4.368e-07, 'completion_length': 146.1964340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00702667236328125, 'epoch': 0.56} 56%|█████▋ | 1408/2500 [8:28:11<6:43:28, 22.17s/it] 56%|█████▋ | 1409/2500 [8:28:31<6:32:53, 21.61s/it] {'loss': 0.0003, 'grad_norm': 0.4753366580252779, 'learning_rate': 4.364e-07, 'completion_length': 140.21429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0064544677734375, 'epoch': 0.56} 56%|█████▋ | 1409/2500 [8:28:31<6:32:53, 21.61s/it] 56%|█████▋ | 1410/2500 [8:28:52<6:30:11, 21.48s/it] {'loss': 0.0003, 'grad_norm': 0.24967111744033987, 'learning_rate': 4.36e-07, 'completion_length': 148.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0066375732421875, 'epoch': 0.56} 56%|█████▋ | 1410/2500 [8:28:52<6:30:11, 21.48s/it] 56%|█████▋ | 1411/2500 [8:29:13<6:24:59, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.23001130749918391, 'learning_rate': 4.3559999999999996e-07, 'completion_length': 158.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0077972412109375, 'epoch': 0.56} 56%|█████▋ | 1411/2500 [8:29:13<6:24:59, 21.21s/it] 56%|█████▋ | 1412/2500 [8:29:34<6:26:55, 21.34s/it] {'loss': 0.0003, 'grad_norm': 0.6720979834995279, 'learning_rate': 4.352e-07, 'completion_length': 155.48214721679688, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.096499003469944, 'kl': 0.00762939453125, 'epoch': 0.56} 56%|█████▋ | 1412/2500 [8:29:34<6:26:55, 21.34s/it] 57%|█████▋ | 1413/2500 [8:29:55<6:23:41, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.6438391543428847, 'learning_rate': 4.348e-07, 'completion_length': 161.9464340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.005767822265625, 'epoch': 0.57} 57%|█████▋ | 1413/2500 [8:29:55<6:23:41, 21.18s/it] 57%|█████▋ | 1414/2500 [8:30:17<6:24:43, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.5459999385278177, 'learning_rate': 4.3439999999999997e-07, 'completion_length': 137.32143020629883, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.0739355981349945, 'kl': 0.00585174560546875, 'epoch': 0.57} 57%|█████▋ | 1414/2500 [8:30:17<6:24:43, 21.26s/it] 57%|█████▋ | 1415/2500 [8:30:38<6:25:08, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.38082361537955645, 'learning_rate': 4.34e-07, 'completion_length': 155.25000762939453, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.033065006136894226, 'kl': 0.0060272216796875, 'epoch': 0.57} 57%|█████▋ | 1415/2500 [8:30:38<6:25:08, 21.30s/it] 57%|█████▋ | 1416/2500 [8:30:59<6:24:14, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.23885151378792077, 'learning_rate': 4.3359999999999997e-07, 'completion_length': 142.71429443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.004730224609375, 'epoch': 0.57} 57%|█████▋ | 1416/2500 [8:30:59<6:24:14, 21.27s/it] 57%|█████▋ | 1417/2500 [8:31:20<6:23:50, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.24288189225503298, 'learning_rate': 4.3319999999999994e-07, 'completion_length': 148.46429443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00567626953125, 'epoch': 0.57} 57%|█████▋ | 1417/2500 [8:31:20<6:23:50, 21.27s/it] 57%|█████▋ | 1418/2500 [8:31:41<6:21:17, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.2685604131625474, 'learning_rate': 4.328e-07, 'completion_length': 143.50894165039062, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004241943359375, 'epoch': 0.57} 57%|█████▋ | 1418/2500 [8:31:41<6:21:17, 21.14s/it] 57%|█████▋ | 1419/2500 [8:32:02<6:17:51, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.02384976775766039, 'learning_rate': 4.324e-07, 'completion_length': 131.66964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053558349609375, 'epoch': 0.57} 57%|█████▋ | 1419/2500 [8:32:02<6:17:51, 20.97s/it] 57%|█████▋ | 1420/2500 [8:32:23<6:16:05, 20.89s/it] {'loss': 0.0003, 'grad_norm': 0.3876437057158127, 'learning_rate': 4.3199999999999995e-07, 'completion_length': 146.9375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0063018798828125, 'epoch': 0.57} 57%|█████▋ | 1420/2500 [8:32:23<6:16:05, 20.89s/it] 57%|█████▋ | 1421/2500 [8:32:44<6:18:33, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.023686977116380372, 'learning_rate': 4.316e-07, 'completion_length': 153.08929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0067291259765625, 'epoch': 0.57} 57%|█████▋ | 1421/2500 [8:32:44<6:18:33, 21.05s/it] 57%|█████▋ | 1422/2500 [8:33:05<6:16:18, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.4110322077303467, 'learning_rate': 4.312e-07, 'completion_length': 145.80357360839844, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00403594970703125, 'epoch': 0.57} 57%|█████▋ | 1422/2500 [8:33:05<6:16:18, 20.94s/it] 57%|█████▋ | 1423/2500 [8:33:26<6:15:34, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.020195847311871143, 'learning_rate': 4.308e-07, 'completion_length': 152.60714721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00550079345703125, 'epoch': 0.57} 57%|█████▋ | 1423/2500 [8:33:26<6:15:34, 20.92s/it] 57%|█████▋ | 1424/2500 [8:33:47<6:19:00, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.35923874223676466, 'learning_rate': 4.304e-07, 'completion_length': 153.36608123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0060272216796875, 'epoch': 0.57} 57%|█████▋ | 1424/2500 [8:33:47<6:19:00, 21.13s/it] 57%|█████▋ | 1425/2500 [8:34:08<6:16:59, 21.04s/it] {'loss': 0.0001, 'grad_norm': 0.020919485997293985, 'learning_rate': 4.2999999999999996e-07, 'completion_length': 129.2857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00325775146484375, 'epoch': 0.57} 57%|█████▋ | 1425/2500 [8:34:08<6:16:59, 21.04s/it] 57%|█████▋ | 1426/2500 [8:34:30<6:20:07, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.36088334123949567, 'learning_rate': 4.296e-07, 'completion_length': 169.58929443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.033065006136894226, 'kl': 0.00762939453125, 'epoch': 0.57} 57%|█████▋ | 1426/2500 [8:34:30<6:20:07, 21.24s/it] 57%|█████▋ | 1427/2500 [8:34:50<6:16:37, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.022675063226733135, 'learning_rate': 4.292e-07, 'completion_length': 149.5803680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005584716796875, 'epoch': 0.57} 57%|█████▋ | 1427/2500 [8:34:50<6:16:37, 21.06s/it] 57%|█████▋ | 1428/2500 [8:35:12<6:20:30, 21.30s/it] {'loss': 0.0002, 'grad_norm': 1.079162780637973, 'learning_rate': 4.288e-07, 'completion_length': 160.01786041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0045166015625, 'epoch': 0.57} 57%|█████▋ | 1428/2500 [8:35:12<6:20:30, 21.30s/it] 57%|█████▋ | 1429/2500 [8:35:34<6:23:22, 21.48s/it] {'loss': 0.0002, 'grad_norm': 0.5108668184823815, 'learning_rate': 4.284e-07, 'completion_length': 146.8303680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0054931640625, 'epoch': 0.57} 57%|█████▋ | 1429/2500 [8:35:34<6:23:22, 21.48s/it] 57%|█████▋ | 1430/2500 [8:35:56<6:22:49, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.6536649207220712, 'learning_rate': 4.2799999999999997e-07, 'completion_length': 150.75000762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0739355981349945, 'kl': 0.0059051513671875, 'epoch': 0.57} 57%|█████▋ | 1430/2500 [8:35:56<6:22:49, 21.47s/it] 57%|█████▋ | 1431/2500 [8:36:16<6:18:07, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.3500989969173873, 'learning_rate': 4.2759999999999994e-07, 'completion_length': 145.65179443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0059967041015625, 'epoch': 0.57} 57%|█████▋ | 1431/2500 [8:36:16<6:18:07, 21.22s/it] 57%|█████▋ | 1432/2500 [8:36:37<6:17:10, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.03656564194291378, 'learning_rate': 4.272e-07, 'completion_length': 157.70536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054473876953125, 'epoch': 0.57} 57%|█████▋ | 1432/2500 [8:36:37<6:17:10, 21.19s/it] 57%|█████▋ | 1433/2500 [8:36:58<6:14:48, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.37579993448320437, 'learning_rate': 4.268e-07, 'completion_length': 148.125, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.006134033203125, 'epoch': 0.57} 57%|█████▋ | 1433/2500 [8:36:58<6:14:48, 21.08s/it] 57%|█████▋ | 1434/2500 [8:37:20<6:18:12, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.40917127353540517, 'learning_rate': 4.264e-07, 'completion_length': 150.81250762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0055694580078125, 'epoch': 0.57} 57%|█████▋ | 1434/2500 [8:37:20<6:18:12, 21.29s/it] 57%|█████▋ | 1435/2500 [8:37:41<6:17:57, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.2264507621331316, 'learning_rate': 4.26e-07, 'completion_length': 166.11608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00557708740234375, 'epoch': 0.57} 57%|█████▋ | 1435/2500 [8:37:41<6:17:57, 21.29s/it] 57%|█████▋ | 1436/2500 [8:38:03<6:20:02, 21.43s/it] {'loss': 0.0004, 'grad_norm': 0.5503649896633412, 'learning_rate': 4.2559999999999995e-07, 'completion_length': 153.85714721679688, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.09528662264347076, 'kl': 0.00982666015625, 'epoch': 0.57} 57%|█████▋ | 1436/2500 [8:38:03<6:20:02, 21.43s/it] 57%|█████▋ | 1437/2500 [8:38:24<6:18:07, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.8384453194494185, 'learning_rate': 4.252e-07, 'completion_length': 148.0803680419922, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.1030978113412857, 'kl': 0.004547119140625, 'epoch': 0.57} 57%|█████▋ | 1437/2500 [8:38:24<6:18:07, 21.34s/it] 58%|█████▊ | 1438/2500 [8:38:46<6:20:44, 21.51s/it] {'loss': 0.0002, 'grad_norm': 0.8015643069202321, 'learning_rate': 4.248e-07, 'completion_length': 154.4107208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0739355981349945, 'kl': 0.0047149658203125, 'epoch': 0.58} 58%|█████▊ | 1438/2500 [8:38:46<6:20:44, 21.51s/it] 58%|█████▊ | 1439/2500 [8:39:08<6:21:11, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.02027959091241966, 'learning_rate': 4.2439999999999996e-07, 'completion_length': 148.95536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00479888916015625, 'epoch': 0.58} 58%|█████▊ | 1439/2500 [8:39:08<6:21:11, 21.56s/it] 58%|█████▊ | 1440/2500 [8:39:30<6:23:40, 21.72s/it] {'loss': 0.0002, 'grad_norm': 0.2581974815663038, 'learning_rate': 4.24e-07, 'completion_length': 154.66964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059814453125, 'epoch': 0.58} 58%|█████▊ | 1440/2500 [8:39:30<6:23:40, 21.72s/it] 58%|█████▊ | 1441/2500 [8:39:51<6:20:44, 21.57s/it] {'loss': 0.0002, 'grad_norm': 0.028688556319820147, 'learning_rate': 4.2359999999999995e-07, 'completion_length': 147.46429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005645751953125, 'epoch': 0.58} 58%|█████▊ | 1441/2500 [8:39:51<6:20:44, 21.57s/it] 58%|█████▊ | 1442/2500 [8:40:13<6:20:11, 21.56s/it] {'loss': 0.0001, 'grad_norm': 1.501608173571478, 'learning_rate': 4.232e-07, 'completion_length': 144.8482208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00341796875, 'epoch': 0.58} 58%|█████▊ | 1442/2500 [8:40:13<6:20:11, 21.56s/it] 58%|█████▊ | 1443/2500 [8:40:33<6:16:23, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.8222114790421304, 'learning_rate': 4.228e-07, 'completion_length': 137.92857360839844, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.004425048828125, 'epoch': 0.58} 58%|█████▊ | 1443/2500 [8:40:33<6:16:23, 21.37s/it] 58%|█████▊ | 1444/2500 [8:40:54<6:13:13, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.4382216767827395, 'learning_rate': 4.2239999999999997e-07, 'completion_length': 144.42857360839844, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0054931640625, 'epoch': 0.58} 58%|█████▊ | 1444/2500 [8:40:54<6:13:13, 21.21s/it] 58%|█████▊ | 1445/2500 [8:41:15<6:10:20, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.2423960201085641, 'learning_rate': 4.2199999999999994e-07, 'completion_length': 147.0178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0074462890625, 'epoch': 0.58} 58%|█████▊ | 1445/2500 [8:41:15<6:10:20, 21.06s/it] 58%|█████▊ | 1446/2500 [8:41:36<6:11:12, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.3685360080234152, 'learning_rate': 4.2159999999999996e-07, 'completion_length': 155.68750762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0041351318359375, 'epoch': 0.58} 58%|█████▊ | 1446/2500 [8:41:36<6:11:12, 21.13s/it] 58%|█████▊ | 1447/2500 [8:41:57<6:10:23, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.057101123695623965, 'learning_rate': 4.212e-07, 'completion_length': 154.74108123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.005645751953125, 'epoch': 0.58} 58%|█████▊ | 1447/2500 [8:41:57<6:10:23, 21.10s/it] 58%|█████▊ | 1448/2500 [8:42:19<6:10:26, 21.13s/it] {'loss': 0.0003, 'grad_norm': 0.5610938829334359, 'learning_rate': 4.208e-07, 'completion_length': 155.87500762939453, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.1030978113412857, 'kl': 0.00775146484375, 'epoch': 0.58} 58%|█████▊ | 1448/2500 [8:42:19<6:10:26, 21.13s/it] 58%|█████▊ | 1449/2500 [8:42:39<6:08:54, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.25301419992947743, 'learning_rate': 4.204e-07, 'completion_length': 162.30357360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00640869140625, 'epoch': 0.58} 58%|█████▊ | 1449/2500 [8:42:39<6:08:54, 21.06s/it] 58%|█████▊ | 1450/2500 [8:43:01<6:10:47, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.3079637271553985, 'learning_rate': 4.1999999999999995e-07, 'completion_length': 157.9732208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.033065006136894226, 'kl': 0.0068511962890625, 'epoch': 0.58} 58%|█████▊ | 1450/2500 [8:43:01<6:10:47, 21.19s/it] 58%|█████▊ | 1451/2500 [8:43:22<6:12:22, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.2918150502681435, 'learning_rate': 4.1959999999999997e-07, 'completion_length': 143.7857208251953, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.004241943359375, 'epoch': 0.58} 58%|█████▊ | 1451/2500 [8:43:22<6:12:22, 21.30s/it] 58%|█████▊ | 1452/2500 [8:43:43<6:07:25, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.022148951599230488, 'learning_rate': 4.192e-07, 'completion_length': 145.7232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006011962890625, 'epoch': 0.58} 58%|█████▊ | 1452/2500 [8:43:43<6:07:25, 21.04s/it] 58%|█████▊ | 1453/2500 [8:44:04<6:09:58, 21.20s/it] {'loss': 0.0004, 'grad_norm': 0.4345701602748874, 'learning_rate': 4.1879999999999996e-07, 'completion_length': 161.33929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.009185791015625, 'epoch': 0.58} 58%|█████▊ | 1453/2500 [8:44:04<6:09:58, 21.20s/it] 58%|█████▊ | 1454/2500 [8:44:27<6:16:51, 21.62s/it] {'loss': 0.0003, 'grad_norm': 0.3302781911990405, 'learning_rate': 4.184e-07, 'completion_length': 165.5357208251953, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.03696779906749725, 'kl': 0.0081024169921875, 'epoch': 0.58} 58%|█████▊ | 1454/2500 [8:44:27<6:16:51, 21.62s/it] 58%|█████▊ | 1455/2500 [8:44:47<6:09:56, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.0878613912553331, 'learning_rate': 4.1799999999999996e-07, 'completion_length': 144.1607208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055389404296875, 'epoch': 0.58} 58%|█████▊ | 1455/2500 [8:44:47<6:09:56, 21.24s/it] 58%|█████▊ | 1456/2500 [8:45:08<6:07:05, 21.10s/it] {'loss': 0.0003, 'grad_norm': 1.0138381908663214, 'learning_rate': 4.1760000000000003e-07, 'completion_length': 152.56250762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.007476806640625, 'epoch': 0.58} 58%|█████▊ | 1456/2500 [8:45:08<6:07:05, 21.10s/it] 58%|█████▊ | 1457/2500 [8:45:29<6:05:00, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.2460756006593744, 'learning_rate': 4.172e-07, 'completion_length': 155.80357360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006317138671875, 'epoch': 0.58} 58%|█████▊ | 1457/2500 [8:45:29<6:05:00, 21.00s/it] 58%|█████▊ | 1458/2500 [8:45:49<6:01:42, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.020449476976190165, 'learning_rate': 4.1679999999999997e-07, 'completion_length': 138.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0046539306640625, 'epoch': 0.58} 58%|█████▊ | 1458/2500 [8:45:49<6:01:42, 20.83s/it] 58%|█████▊ | 1459/2500 [8:46:11<6:03:10, 20.93s/it] {'loss': 0.0004, 'grad_norm': 0.7715650473255785, 'learning_rate': 4.164e-07, 'completion_length': 158.74108123779297, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0108642578125, 'epoch': 0.58} 58%|█████▊ | 1459/2500 [8:46:11<6:03:10, 20.93s/it] 58%|█████▊ | 1460/2500 [8:46:31<6:02:04, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.22453456127038957, 'learning_rate': 4.1599999999999997e-07, 'completion_length': 147.05358123779297, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0047607421875, 'epoch': 0.58} 58%|█████▊ | 1460/2500 [8:46:31<6:02:04, 20.89s/it] 58%|█████▊ | 1461/2500 [8:46:53<6:03:17, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.3635458084332504, 'learning_rate': 4.156e-07, 'completion_length': 149.49107360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0051116943359375, 'epoch': 0.58} 58%|█████▊ | 1461/2500 [8:46:53<6:03:17, 20.98s/it] 58%|█████▊ | 1462/2500 [8:47:13<6:02:02, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.04183522561959329, 'learning_rate': 4.152e-07, 'completion_length': 146.12500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006134033203125, 'epoch': 0.58} 58%|█████▊ | 1462/2500 [8:47:13<6:02:02, 20.93s/it] 59%|█████▊ | 1463/2500 [8:47:34<5:59:58, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.0208668069303512, 'learning_rate': 4.148e-07, 'completion_length': 146.65178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047607421875, 'epoch': 0.59} 59%|█████▊ | 1463/2500 [8:47:34<5:59:58, 20.83s/it] 59%|█████▊ | 1464/2500 [8:47:55<6:00:01, 20.85s/it] {'loss': 0.0002, 'grad_norm': 0.43991026613038753, 'learning_rate': 4.1439999999999995e-07, 'completion_length': 144.73214721679688, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10161417350172997, 'kl': 0.006011962890625, 'epoch': 0.59} 59%|█████▊ | 1464/2500 [8:47:55<6:00:01, 20.85s/it] 59%|█████▊ | 1465/2500 [8:48:15<5:57:45, 20.74s/it] {'loss': 0.0002, 'grad_norm': 0.325063203106559, 'learning_rate': 4.14e-07, 'completion_length': 129.8035774230957, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.004241943359375, 'epoch': 0.59} 59%|█████▊ | 1465/2500 [8:48:15<5:57:45, 20.74s/it] 59%|█████▊ | 1466/2500 [8:48:36<5:57:05, 20.72s/it] {'loss': 0.0002, 'grad_norm': 0.017009663959341843, 'learning_rate': 4.136e-07, 'completion_length': 145.6964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045928955078125, 'epoch': 0.59} 59%|█████▊ | 1466/2500 [8:48:36<5:57:05, 20.72s/it] 59%|█████▊ | 1467/2500 [8:48:57<5:58:52, 20.84s/it] {'loss': 0.0001, 'grad_norm': 0.027330470714839405, 'learning_rate': 4.1319999999999997e-07, 'completion_length': 147.25, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003662109375, 'epoch': 0.59} 59%|█████▊ | 1467/2500 [8:48:57<5:58:52, 20.84s/it] 59%|█████▊ | 1468/2500 [8:49:19<6:01:23, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.25393144131254136, 'learning_rate': 4.128e-07, 'completion_length': 159.1428680419922, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0057373046875, 'epoch': 0.59} 59%|█████▊ | 1468/2500 [8:49:19<6:01:23, 21.01s/it] 59%|█████▉ | 1469/2500 [8:49:40<6:01:43, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.3862499060234043, 'learning_rate': 4.1239999999999996e-07, 'completion_length': 166.77679443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00467681884765625, 'epoch': 0.59} 59%|█████▉ | 1469/2500 [8:49:40<6:01:43, 21.05s/it] 59%|█████▉ | 1470/2500 [8:50:01<6:03:17, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.627721707107558, 'learning_rate': 4.12e-07, 'completion_length': 162.73214721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0069122314453125, 'epoch': 0.59} 59%|█████▉ | 1470/2500 [8:50:01<6:03:17, 21.16s/it] 59%|█████▉ | 1471/2500 [8:50:22<6:01:52, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.19007455194048678, 'learning_rate': 4.116e-07, 'completion_length': 146.55358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0037994384765625, 'epoch': 0.59} 59%|█████▉ | 1471/2500 [8:50:22<6:01:52, 21.10s/it] 59%|█████▉ | 1472/2500 [8:50:45<6:08:50, 21.53s/it] {'loss': 0.0003, 'grad_norm': 0.23243451419202824, 'learning_rate': 4.112e-07, 'completion_length': 155.83929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006378173828125, 'epoch': 0.59} 59%|█████▉ | 1472/2500 [8:50:45<6:08:50, 21.53s/it] 59%|█████▉ | 1473/2500 [8:51:06<6:05:39, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.5926632265865333, 'learning_rate': 4.108e-07, 'completion_length': 149.5714340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726410150528, 'kl': 0.0069122314453125, 'epoch': 0.59} 59%|█████▉ | 1473/2500 [8:51:06<6:05:39, 21.36s/it] 59%|█████▉ | 1474/2500 [8:51:26<6:02:09, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.28659398595780533, 'learning_rate': 4.1039999999999997e-07, 'completion_length': 146.36608123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.005401611328125, 'epoch': 0.59} 59%|█████▉ | 1474/2500 [8:51:26<6:02:09, 21.18s/it] 59%|█████▉ | 1475/2500 [8:51:47<5:59:58, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.6586356716648116, 'learning_rate': 4.0999999999999994e-07, 'completion_length': 153.12500762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0048370361328125, 'epoch': 0.59} 59%|█████▉ | 1475/2500 [8:51:47<5:59:58, 21.07s/it] 59%|█████▉ | 1476/2500 [8:52:08<6:00:08, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.40711018066544424, 'learning_rate': 4.096e-07, 'completion_length': 154.91964721679688, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.0964989997446537, 'kl': 0.0066680908203125, 'epoch': 0.59} 59%|█████▉ | 1476/2500 [8:52:08<6:00:08, 21.10s/it] 59%|█████▉ | 1477/2500 [8:52:30<6:00:33, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.3721501677948473, 'learning_rate': 4.092e-07, 'completion_length': 147.3482208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004364013671875, 'epoch': 0.59} 59%|█████▉ | 1477/2500 [8:52:30<6:00:33, 21.15s/it] 59%|█████▉ | 1478/2500 [8:52:52<6:06:52, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.1862699756553361, 'learning_rate': 4.0879999999999995e-07, 'completion_length': 157.99108123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00555419921875, 'epoch': 0.59} 59%|█████▉ | 1478/2500 [8:52:52<6:06:52, 21.54s/it] 59%|█████▉ | 1479/2500 [8:53:13<6:04:53, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.26896142354717467, 'learning_rate': 4.084e-07, 'completion_length': 162.80358123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0058746337890625, 'epoch': 0.59} 59%|█████▉ | 1479/2500 [8:53:13<6:04:53, 21.44s/it] 59%|█████▉ | 1480/2500 [8:53:34<6:01:44, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.022572996343999865, 'learning_rate': 4.0799999999999995e-07, 'completion_length': 151.12500762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0047149658203125, 'epoch': 0.59} 59%|█████▉ | 1480/2500 [8:53:34<6:01:44, 21.28s/it] 59%|█████▉ | 1481/2500 [8:53:55<5:59:46, 21.18s/it] {'loss': 0.0001, 'grad_norm': 0.22720095001511068, 'learning_rate': 4.076e-07, 'completion_length': 141.36608123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.003021240234375, 'epoch': 0.59} 59%|█████▉ | 1481/2500 [8:53:55<5:59:46, 21.18s/it] 59%|█████▉ | 1482/2500 [8:54:17<6:02:39, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.7620477419510682, 'learning_rate': 4.072e-07, 'completion_length': 164.11607360839844, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.13225442171096802, 'kl': 0.00677490234375, 'epoch': 0.59} 59%|█████▉ | 1482/2500 [8:54:17<6:02:39, 21.37s/it] 59%|█████▉ | 1483/2500 [8:54:38<5:59:30, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.025701140840604472, 'learning_rate': 4.0679999999999996e-07, 'completion_length': 146.4107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051727294921875, 'epoch': 0.59} 59%|█████▉ | 1483/2500 [8:54:38<5:59:30, 21.21s/it] 59%|█████▉ | 1484/2500 [8:54:59<5:57:07, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.17731586910776165, 'learning_rate': 4.064e-07, 'completion_length': 149.9107208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0061187744140625, 'epoch': 0.59} 59%|█████▉ | 1484/2500 [8:54:59<5:57:07, 21.09s/it] 59%|█████▉ | 1485/2500 [8:55:19<5:53:33, 20.90s/it] {'loss': 0.0003, 'grad_norm': 0.029680995816054788, 'learning_rate': 4.06e-07, 'completion_length': 138.49107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006866455078125, 'epoch': 0.59} 59%|█████▉ | 1485/2500 [8:55:19<5:53:33, 20.90s/it] 59%|█████▉ | 1486/2500 [8:55:40<5:53:37, 20.92s/it] {'loss': 0.0003, 'grad_norm': 0.03478338736010433, 'learning_rate': 4.056e-07, 'completion_length': 152.8839340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00640869140625, 'epoch': 0.59} 59%|█████▉ | 1486/2500 [8:55:40<5:53:37, 20.92s/it] 59%|█████▉ | 1487/2500 [8:56:01<5:51:18, 20.81s/it] {'loss': 0.0002, 'grad_norm': 0.4142408892575945, 'learning_rate': 4.052e-07, 'completion_length': 151.2589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005462646484375, 'epoch': 0.59} 59%|█████▉ | 1487/2500 [8:56:01<5:51:18, 20.81s/it] 60%|█████▉ | 1488/2500 [8:56:21<5:51:54, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.31718420701992145, 'learning_rate': 4.0479999999999997e-07, 'completion_length': 153.43750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.004791259765625, 'epoch': 0.6} 60%|█████▉ | 1488/2500 [8:56:22<5:51:54, 20.86s/it] 60%|█████▉ | 1489/2500 [8:56:44<5:58:24, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.033648703936297965, 'learning_rate': 4.0439999999999994e-07, 'completion_length': 155.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004638671875, 'epoch': 0.6} 60%|█████▉ | 1489/2500 [8:56:44<5:58:24, 21.27s/it] 60%|█████▉ | 1490/2500 [8:57:05<5:57:17, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.022326543092108023, 'learning_rate': 4.04e-07, 'completion_length': 151.30358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051116943359375, 'epoch': 0.6} 60%|█████▉ | 1490/2500 [8:57:05<5:57:17, 21.23s/it] 60%|█████▉ | 1491/2500 [8:57:25<5:53:42, 21.03s/it] {'loss': 0.0003, 'grad_norm': 1.3604045206159416, 'learning_rate': 4.036e-07, 'completion_length': 149.96429443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.006378173828125, 'epoch': 0.6} 60%|█████▉ | 1491/2500 [8:57:25<5:53:42, 21.03s/it] 60%|█████▉ | 1492/2500 [8:57:46<5:52:04, 20.96s/it] {'loss': 0.0003, 'grad_norm': 0.5400352544545494, 'learning_rate': 4.032e-07, 'completion_length': 153.75, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0066070556640625, 'epoch': 0.6} 60%|█████▉ | 1492/2500 [8:57:46<5:52:04, 20.96s/it] 60%|█████▉ | 1493/2500 [8:58:07<5:51:01, 20.92s/it] {'loss': 0.0003, 'grad_norm': 0.06587710019791894, 'learning_rate': 4.028e-07, 'completion_length': 143.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0069732666015625, 'epoch': 0.6} 60%|█████▉ | 1493/2500 [8:58:07<5:51:01, 20.92s/it] 60%|█████▉ | 1494/2500 [8:58:28<5:52:43, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.5574031862777472, 'learning_rate': 4.0239999999999995e-07, 'completion_length': 151.0714340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0060882568359375, 'epoch': 0.6} 60%|█████▉ | 1494/2500 [8:58:28<5:52:43, 21.04s/it] 60%|█████▉ | 1495/2500 [8:58:49<5:52:36, 21.05s/it] {'loss': 0.0001, 'grad_norm': 0.027599187389594073, 'learning_rate': 4.02e-07, 'completion_length': 143.71429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00341033935546875, 'epoch': 0.6} 60%|█████▉ | 1495/2500 [8:58:49<5:52:36, 21.05s/it] 60%|█████▉ | 1496/2500 [8:59:10<5:50:04, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.7389186417872863, 'learning_rate': 4.016e-07, 'completion_length': 142.0357208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.07576144114136696, 'kl': 0.00482177734375, 'epoch': 0.6} 60%|█████▉ | 1496/2500 [8:59:10<5:50:04, 20.92s/it] 60%|█████▉ | 1497/2500 [8:59:31<5:48:11, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.34841137846106884, 'learning_rate': 4.0119999999999997e-07, 'completion_length': 141.2946548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00399017333984375, 'epoch': 0.6} 60%|█████▉ | 1497/2500 [8:59:31<5:48:11, 20.83s/it] 60%|█████▉ | 1498/2500 [8:59:51<5:47:04, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.3378523381998944, 'learning_rate': 4.008e-07, 'completion_length': 157.24107360839844, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.006072998046875, 'epoch': 0.6} 60%|█████▉ | 1498/2500 [8:59:51<5:47:04, 20.78s/it] 60%|█████▉ | 1499/2500 [9:00:13<5:51:52, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.2954861669957325, 'learning_rate': 4.0039999999999996e-07, 'completion_length': 165.62500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0061798095703125, 'epoch': 0.6} 60%|█████▉ | 1499/2500 [9:00:13<5:51:52, 21.09s/it] 60%|██████ | 1500/2500 [9:00:34<5:51:52, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.5423208708060812, 'learning_rate': 4e-07, 'completion_length': 150.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00433349609375, 'epoch': 0.6} 60%|██████ | 1500/2500 [9:00:34<5:51:52, 21.11s/it] 60%|██████ | 1501/2500 [9:01:25<8:18:23, 29.93s/it] {'loss': 0.0002, 'grad_norm': 0.03064160854280917, 'learning_rate': 3.996e-07, 'completion_length': 141.81250762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00457763671875, 'epoch': 0.6} 60%|██████ | 1501/2500 [9:01:25<8:18:23, 29.93s/it] 60%|██████ | 1502/2500 [9:01:35<6:39:32, 24.02s/it] {'loss': 0.0002, 'grad_norm': 0.21762157046043049, 'learning_rate': 3.992e-07, 'completion_length': 154.15179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00537109375, 'epoch': 0.6} 60%|██████ | 1502/2500 [9:01:35<6:39:32, 24.02s/it] 60%|██████ | 1503/2500 [9:01:45<5:30:14, 19.87s/it] {'loss': 0.0003, 'grad_norm': 0.20563688776988295, 'learning_rate': 3.9879999999999994e-07, 'completion_length': 155.5089340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0085906982421875, 'epoch': 0.6} 60%|██████ | 1503/2500 [9:01:45<5:30:14, 19.87s/it] 60%|██████ | 1504/2500 [9:01:56<4:42:02, 16.99s/it] {'loss': 0.0002, 'grad_norm': 0.3186053550293273, 'learning_rate': 3.9839999999999997e-07, 'completion_length': 131.24108123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0052490234375, 'epoch': 0.6} 60%|██████ | 1504/2500 [9:01:56<4:42:02, 16.99s/it] 60%|██████ | 1505/2500 [9:02:06<4:11:03, 15.14s/it] {'loss': 0.0004, 'grad_norm': 0.23207951891303222, 'learning_rate': 3.98e-07, 'completion_length': 158.27679443359375, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.03696779906749725, 'kl': 0.0104217529296875, 'epoch': 0.6} 60%|██████ | 1505/2500 [9:02:06<4:11:03, 15.14s/it] 60%|██████ | 1506/2500 [9:02:17<3:47:43, 13.75s/it] {'loss': 0.0002, 'grad_norm': 0.4607959484739045, 'learning_rate': 3.976e-07, 'completion_length': 155.0357208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00579833984375, 'epoch': 0.6} 60%|██████ | 1506/2500 [9:02:17<3:47:43, 13.75s/it] 60%|██████ | 1507/2500 [9:02:27<3:30:39, 12.73s/it] {'loss': 0.0001, 'grad_norm': 0.015631522786508523, 'learning_rate': 3.972e-07, 'completion_length': 146.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003509521484375, 'epoch': 0.6} 60%|██████ | 1507/2500 [9:02:27<3:30:39, 12.73s/it] 60%|██████ | 1508/2500 [9:02:38<3:18:49, 12.03s/it] {'loss': 0.0003, 'grad_norm': 0.018941663581589198, 'learning_rate': 3.9679999999999995e-07, 'completion_length': 153.99107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006591796875, 'epoch': 0.6} 60%|██████ | 1508/2500 [9:02:38<3:18:49, 12.03s/it] 60%|██████ | 1509/2500 [9:02:48<3:09:52, 11.50s/it] {'loss': 0.0002, 'grad_norm': 0.873847479060256, 'learning_rate': 3.964e-07, 'completion_length': 152.3482208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00434112548828125, 'epoch': 0.6} 60%|██████ | 1509/2500 [9:02:48<3:09:52, 11.50s/it] 60%|██████ | 1510/2500 [9:02:58<3:03:44, 11.14s/it] {'loss': 0.0002, 'grad_norm': 0.39394097989354304, 'learning_rate': 3.96e-07, 'completion_length': 154.66964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005218505859375, 'epoch': 0.6} 60%|██████ | 1510/2500 [9:02:58<3:03:44, 11.14s/it] 60%|██████ | 1511/2500 [9:03:08<2:57:13, 10.75s/it] {'loss': 0.0001, 'grad_norm': 0.025252916060613504, 'learning_rate': 3.9559999999999997e-07, 'completion_length': 142.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00354766845703125, 'epoch': 0.6} 60%|██████ | 1511/2500 [9:03:08<2:57:13, 10.75s/it] 60%|██████ | 1512/2500 [9:03:18<2:55:09, 10.64s/it] {'loss': 0.0003, 'grad_norm': 0.026267737718916828, 'learning_rate': 3.952e-07, 'completion_length': 159.9732208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.007720947265625, 'epoch': 0.6} 60%|██████ | 1512/2500 [9:03:18<2:55:09, 10.64s/it] 61%|██████ | 1513/2500 [9:03:28<2:52:12, 10.47s/it] {'loss': 0.0002, 'grad_norm': 0.7883424162017931, 'learning_rate': 3.9479999999999996e-07, 'completion_length': 145.65179443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.0042572021484375, 'epoch': 0.61} 61%|██████ | 1513/2500 [9:03:28<2:52:12, 10.47s/it] 61%|██████ | 1514/2500 [9:03:39<2:52:20, 10.49s/it] {'loss': 0.0003, 'grad_norm': 0.5652764392516049, 'learning_rate': 3.9439999999999993e-07, 'completion_length': 153.4464340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11663764715194702, 'kl': 0.00775146484375, 'epoch': 0.61} 61%|██████ | 1514/2500 [9:03:39<2:52:20, 10.49s/it] 61%|██████ | 1515/2500 [9:03:49<2:48:11, 10.24s/it] {'loss': 0.0002, 'grad_norm': 0.31394730153789285, 'learning_rate': 3.94e-07, 'completion_length': 139.17858123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00446319580078125, 'epoch': 0.61} 61%|██████ | 1515/2500 [9:03:49<2:48:11, 10.24s/it] 61%|██████ | 1516/2500 [9:03:59<2:48:06, 10.25s/it] {'loss': 0.0002, 'grad_norm': 0.070416332394902, 'learning_rate': 3.936e-07, 'completion_length': 152.92857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004150390625, 'epoch': 0.61} 61%|██████ | 1516/2500 [9:03:59<2:48:06, 10.25s/it] 61%|██████ | 1517/2500 [9:04:10<2:50:00, 10.38s/it] {'loss': 0.0003, 'grad_norm': 0.39607171501033883, 'learning_rate': 3.932e-07, 'completion_length': 160.21429443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0071563720703125, 'epoch': 0.61} 61%|██████ | 1517/2500 [9:04:10<2:50:00, 10.38s/it] 61%|██████ | 1518/2500 [9:04:20<2:49:30, 10.36s/it] {'loss': 0.0002, 'grad_norm': 0.03549612989875487, 'learning_rate': 3.9279999999999997e-07, 'completion_length': 144.6964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003997802734375, 'epoch': 0.61} 61%|██████ | 1518/2500 [9:04:20<2:49:30, 10.36s/it] 61%|██████ | 1519/2500 [9:04:36<3:19:12, 12.18s/it] {'loss': 0.0002, 'grad_norm': 0.49756025228201556, 'learning_rate': 3.924e-07, 'completion_length': 148.30358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00513458251953125, 'epoch': 0.61} 61%|██████ | 1519/2500 [9:04:36<3:19:12, 12.18s/it] 61%|██████ | 1520/2500 [9:04:58<4:04:07, 14.95s/it] {'loss': 0.0002, 'grad_norm': 0.0179624542537447, 'learning_rate': 3.92e-07, 'completion_length': 151.2857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056610107421875, 'epoch': 0.61} 61%|██████ | 1520/2500 [9:04:58<4:04:07, 14.95s/it] 61%|██████ | 1521/2500 [9:05:19<4:35:45, 16.90s/it] {'loss': 0.0002, 'grad_norm': 0.6427645646697365, 'learning_rate': 3.916e-07, 'completion_length': 147.21429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.09528662264347076, 'kl': 0.00565338134765625, 'epoch': 0.61} 61%|██████ | 1521/2500 [9:05:19<4:35:45, 16.90s/it] 61%|██████ | 1522/2500 [9:05:40<4:52:59, 17.97s/it] {'loss': 0.0002, 'grad_norm': 0.34914307862848465, 'learning_rate': 3.9119999999999996e-07, 'completion_length': 147.06250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0059356689453125, 'epoch': 0.61} 61%|██████ | 1522/2500 [9:05:40<4:52:59, 17.97s/it] 61%|██████ | 1523/2500 [9:06:00<5:04:34, 18.70s/it] {'loss': 0.0001, 'grad_norm': 0.016852707355497862, 'learning_rate': 3.908e-07, 'completion_length': 139.17858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00345611572265625, 'epoch': 0.61} 61%|██████ | 1523/2500 [9:06:00<5:04:34, 18.70s/it] 61%|██████ | 1524/2500 [9:06:22<5:20:53, 19.73s/it] {'loss': 0.0003, 'grad_norm': 0.7558952002168196, 'learning_rate': 3.904e-07, 'completion_length': 161.23214721679688, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.11663764715194702, 'kl': 0.008392333984375, 'epoch': 0.61} 61%|██████ | 1524/2500 [9:06:22<5:20:53, 19.73s/it] 61%|██████ | 1525/2500 [9:06:43<5:25:48, 20.05s/it] {'loss': 0.0002, 'grad_norm': 0.2747429767345803, 'learning_rate': 3.8999999999999997e-07, 'completion_length': 146.0089340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.005126953125, 'epoch': 0.61} 61%|██████ | 1525/2500 [9:06:43<5:25:48, 20.05s/it] 61%|██████ | 1526/2500 [9:07:04<5:29:45, 20.31s/it] {'loss': 0.0002, 'grad_norm': 1.039053790221162, 'learning_rate': 3.896e-07, 'completion_length': 148.5357208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0046539306640625, 'epoch': 0.61} 61%|██████ | 1526/2500 [9:07:04<5:29:45, 20.31s/it] 61%|██████ | 1527/2500 [9:07:25<5:31:50, 20.46s/it] {'loss': 0.0003, 'grad_norm': 0.028915739452407604, 'learning_rate': 3.8919999999999996e-07, 'completion_length': 148.3303680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00640869140625, 'epoch': 0.61} 61%|██████ | 1527/2500 [9:07:25<5:31:50, 20.46s/it] 61%|██████ | 1528/2500 [9:07:46<5:34:16, 20.63s/it] {'loss': 0.0003, 'grad_norm': 0.7546706812097308, 'learning_rate': 3.888e-07, 'completion_length': 147.90179443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.00677490234375, 'epoch': 0.61} 61%|██████ | 1528/2500 [9:07:46<5:34:16, 20.63s/it] 61%|██████ | 1529/2500 [9:08:07<5:36:19, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.4192859849213428, 'learning_rate': 3.884e-07, 'completion_length': 147.63394165039062, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941557407379, 'kl': 0.00534820556640625, 'epoch': 0.61} 61%|██████ | 1529/2500 [9:08:07<5:36:19, 20.78s/it] 61%|██████ | 1530/2500 [9:08:27<5:33:49, 20.65s/it] {'loss': 0.0002, 'grad_norm': 0.3355300759703278, 'learning_rate': 3.88e-07, 'completion_length': 135.18750381469727, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.004241943359375, 'epoch': 0.61} 61%|██████ | 1530/2500 [9:08:27<5:33:49, 20.65s/it] 61%|██████ | 1531/2500 [9:08:49<5:36:49, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.21021501437919807, 'learning_rate': 3.876e-07, 'completion_length': 153.73214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0044097900390625, 'epoch': 0.61} 61%|██████ | 1531/2500 [9:08:49<5:36:49, 20.86s/it] 61%|██████▏ | 1532/2500 [9:09:09<5:34:09, 20.71s/it] {'loss': 0.0002, 'grad_norm': 0.6061203077337755, 'learning_rate': 3.8719999999999997e-07, 'completion_length': 140.91964721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.0964989960193634, 'kl': 0.00439453125, 'epoch': 0.61} 61%|██████▏ | 1532/2500 [9:09:09<5:34:09, 20.71s/it] 61%|██████▏ | 1533/2500 [9:09:31<5:40:41, 21.14s/it] {'loss': 0.0004, 'grad_norm': 1.3209775164070408, 'learning_rate': 3.8679999999999994e-07, 'completion_length': 166.21428680419922, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0095367431640625, 'epoch': 0.61} 61%|██████▏ | 1533/2500 [9:09:31<5:40:41, 21.14s/it] 61%|██████▏ | 1534/2500 [9:09:51<5:36:49, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.43845931084187123, 'learning_rate': 3.864e-07, 'completion_length': 138.7857208251953, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00411224365234375, 'epoch': 0.61} 61%|██████▏ | 1534/2500 [9:09:51<5:36:49, 20.92s/it] 61%|██████▏ | 1535/2500 [9:10:12<5:36:36, 20.93s/it] {'loss': 0.0003, 'grad_norm': 0.18808356676138233, 'learning_rate': 3.86e-07, 'completion_length': 154.36607360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.008453369140625, 'epoch': 0.61} 61%|██████▏ | 1535/2500 [9:10:12<5:36:36, 20.93s/it] 61%|██████▏ | 1536/2500 [9:10:33<5:33:51, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.044042927366519095, 'learning_rate': 3.8559999999999996e-07, 'completion_length': 148.71428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037994384765625, 'epoch': 0.61} 61%|██████▏ | 1536/2500 [9:10:33<5:33:51, 20.78s/it] 61%|██████▏ | 1537/2500 [9:10:54<5:36:44, 20.98s/it] {'loss': 0.0004, 'grad_norm': 0.3943722323371933, 'learning_rate': 3.852e-07, 'completion_length': 164.87500762939453, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.08747542649507523, 'kl': 0.00970458984375, 'epoch': 0.61} 61%|██████▏ | 1537/2500 [9:10:54<5:36:44, 20.98s/it] 62%|██████▏ | 1538/2500 [9:11:16<5:40:43, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.29915157111782, 'learning_rate': 3.8479999999999995e-07, 'completion_length': 158.6339340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0058441162109375, 'epoch': 0.62} 62%|██████▏ | 1538/2500 [9:11:16<5:40:43, 21.25s/it] 62%|██████▏ | 1539/2500 [9:11:37<5:37:04, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.029002915343836047, 'learning_rate': 3.8440000000000003e-07, 'completion_length': 142.80358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037994384765625, 'epoch': 0.62} 62%|██████▏ | 1539/2500 [9:11:37<5:37:04, 21.05s/it] 62%|██████▏ | 1540/2500 [9:11:58<5:37:31, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.26522286090636876, 'learning_rate': 3.84e-07, 'completion_length': 155.66964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004913330078125, 'epoch': 0.62} 62%|██████▏ | 1540/2500 [9:11:58<5:37:31, 21.10s/it] 62%|██████▏ | 1541/2500 [9:12:19<5:36:29, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.32365391089677775, 'learning_rate': 3.8359999999999997e-07, 'completion_length': 154.6339340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.0058746337890625, 'epoch': 0.62} 62%|██████▏ | 1541/2500 [9:12:19<5:36:29, 21.05s/it] 62%|██████▏ | 1542/2500 [9:12:40<5:35:45, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.0191006998945592, 'learning_rate': 3.832e-07, 'completion_length': 148.3928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00543212890625, 'epoch': 0.62} 62%|██████▏ | 1542/2500 [9:12:40<5:35:45, 21.03s/it] 62%|██████▏ | 1543/2500 [9:13:01<5:37:29, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.6905099990829837, 'learning_rate': 3.8279999999999996e-07, 'completion_length': 147.99107360839844, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0046844482421875, 'epoch': 0.62} 62%|██████▏ | 1543/2500 [9:13:01<5:37:29, 21.16s/it] 62%|██████▏ | 1544/2500 [9:13:23<5:37:34, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.24970106261186692, 'learning_rate': 3.824e-07, 'completion_length': 153.64286041259766, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0055999755859375, 'epoch': 0.62} 62%|██████▏ | 1544/2500 [9:13:23<5:37:34, 21.19s/it] 62%|██████▏ | 1545/2500 [9:13:45<5:41:19, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.20315460774378524, 'learning_rate': 3.82e-07, 'completion_length': 162.24107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006683349609375, 'epoch': 0.62} 62%|██████▏ | 1545/2500 [9:13:45<5:41:19, 21.44s/it] 62%|██████▏ | 1546/2500 [9:14:06<5:42:16, 21.53s/it] {'loss': 0.0003, 'grad_norm': 0.4877675466430061, 'learning_rate': 3.816e-07, 'completion_length': 158.54464721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0064849853515625, 'epoch': 0.62} 62%|██████▏ | 1546/2500 [9:14:06<5:42:16, 21.53s/it] 62%|██████▏ | 1547/2500 [9:14:28<5:40:54, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.02838152700651008, 'learning_rate': 3.8119999999999995e-07, 'completion_length': 147.3303680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005279541015625, 'epoch': 0.62} 62%|██████▏ | 1547/2500 [9:14:28<5:40:54, 21.46s/it] 62%|██████▏ | 1548/2500 [9:14:48<5:36:05, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.07592079537475112, 'learning_rate': 3.808e-07, 'completion_length': 132.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00405120849609375, 'epoch': 0.62} 62%|██████▏ | 1548/2500 [9:14:48<5:36:05, 21.18s/it] 62%|██████▏ | 1549/2500 [9:15:09<5:34:08, 21.08s/it] {'loss': 0.0003, 'grad_norm': 0.03107806849655602, 'learning_rate': 3.804e-07, 'completion_length': 142.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00701904296875, 'epoch': 0.62} 62%|██████▏ | 1549/2500 [9:15:09<5:34:08, 21.08s/it] 62%|██████▏ | 1550/2500 [9:15:29<5:30:40, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.02778294239020767, 'learning_rate': 3.7999999999999996e-07, 'completion_length': 126.37500381469727, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.003936767578125, 'epoch': 0.62} 62%|██████▏ | 1550/2500 [9:15:29<5:30:40, 20.88s/it] 62%|██████▏ | 1551/2500 [9:15:52<5:37:46, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.23989220081357052, 'learning_rate': 3.796e-07, 'completion_length': 180.18750762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00738525390625, 'epoch': 0.62} 62%|██████▏ | 1551/2500 [9:15:52<5:37:46, 21.36s/it] 62%|██████▏ | 1552/2500 [9:16:13<5:34:29, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.37112538292487496, 'learning_rate': 3.7919999999999995e-07, 'completion_length': 130.50000762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00579833984375, 'epoch': 0.62} 62%|██████▏ | 1552/2500 [9:16:13<5:34:29, 21.17s/it] 62%|██████▏ | 1553/2500 [9:16:34<5:34:46, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.2940140373522653, 'learning_rate': 3.7880000000000003e-07, 'completion_length': 149.16964721679688, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.03696779906749725, 'kl': 0.0072479248046875, 'epoch': 0.62} 62%|██████▏ | 1553/2500 [9:16:34<5:34:46, 21.21s/it] 62%|██████▏ | 1554/2500 [9:16:55<5:34:11, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.5177617809228618, 'learning_rate': 3.784e-07, 'completion_length': 151.2321548461914, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.005706787109375, 'epoch': 0.62} 62%|██████▏ | 1554/2500 [9:16:55<5:34:11, 21.20s/it] 62%|██████▏ | 1555/2500 [9:17:16<5:33:39, 21.18s/it] {'loss': 0.0003, 'grad_norm': 0.7778328117145124, 'learning_rate': 3.7799999999999997e-07, 'completion_length': 166.37500762939453, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0064697265625, 'epoch': 0.62} 62%|██████▏ | 1555/2500 [9:17:16<5:33:39, 21.18s/it] 62%|██████▏ | 1556/2500 [9:17:38<5:33:26, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.5203344861914398, 'learning_rate': 3.776e-07, 'completion_length': 157.38394165039062, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.0835726372897625, 'kl': 0.00775146484375, 'epoch': 0.62} 62%|██████▏ | 1556/2500 [9:17:38<5:33:26, 21.19s/it] 62%|██████▏ | 1557/2500 [9:17:59<5:33:19, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.22589464364122677, 'learning_rate': 3.7719999999999996e-07, 'completion_length': 153.43750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0066375732421875, 'epoch': 0.62} 62%|██████▏ | 1557/2500 [9:17:59<5:33:19, 21.21s/it] 62%|██████▏ | 1558/2500 [9:18:20<5:32:17, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.01580481228929162, 'learning_rate': 3.768e-07, 'completion_length': 145.90179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00569915771484375, 'epoch': 0.62} 62%|██████▏ | 1558/2500 [9:18:20<5:32:17, 21.17s/it] 62%|██████▏ | 1559/2500 [9:18:41<5:30:06, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.18833888345754565, 'learning_rate': 3.764e-07, 'completion_length': 148.01786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059661865234375, 'epoch': 0.62} 62%|██████▏ | 1559/2500 [9:18:41<5:30:06, 21.05s/it] 62%|██████▏ | 1560/2500 [9:19:02<5:29:35, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.0365555318190511, 'learning_rate': 3.76e-07, 'completion_length': 147.52678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0044708251953125, 'epoch': 0.62} 62%|██████▏ | 1560/2500 [9:19:02<5:29:35, 21.04s/it] 62%|██████▏ | 1561/2500 [9:19:23<5:30:13, 21.10s/it] {'loss': 0.0004, 'grad_norm': 0.456098480060657, 'learning_rate': 3.7559999999999995e-07, 'completion_length': 162.56250762939453, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00909423828125, 'epoch': 0.62} 62%|██████▏ | 1561/2500 [9:19:23<5:30:13, 21.10s/it] 62%|██████▏ | 1562/2500 [9:19:44<5:30:29, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.4656754539587553, 'learning_rate': 3.7519999999999997e-07, 'completion_length': 145.8214340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00431060791015625, 'epoch': 0.62} 62%|██████▏ | 1562/2500 [9:19:44<5:30:29, 21.14s/it] 63%|██████▎ | 1563/2500 [9:20:05<5:27:26, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.027376770532410277, 'learning_rate': 3.748e-07, 'completion_length': 138.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051422119140625, 'epoch': 0.63} 63%|██████▎ | 1563/2500 [9:20:05<5:27:26, 20.97s/it] 63%|██████▎ | 1564/2500 [9:20:26<5:28:39, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.030525114812712188, 'learning_rate': 3.744e-07, 'completion_length': 154.51786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005523681640625, 'epoch': 0.63} 63%|██████▎ | 1564/2500 [9:20:26<5:28:39, 21.07s/it] 63%|██████▎ | 1565/2500 [9:20:47<5:27:09, 20.99s/it] {'loss': 0.0001, 'grad_norm': 0.5272760186049704, 'learning_rate': 3.74e-07, 'completion_length': 127.47322082519531, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.003204345703125, 'epoch': 0.63} 63%|██████▎ | 1565/2500 [9:20:47<5:27:09, 20.99s/it] 63%|██████▎ | 1566/2500 [9:21:09<5:30:23, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.5737086256273952, 'learning_rate': 3.7359999999999996e-07, 'completion_length': 161.81250762939453, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06222161278128624, 'kl': 0.004852294921875, 'epoch': 0.63} 63%|██████▎ | 1566/2500 [9:21:09<5:30:23, 21.22s/it] 63%|██████▎ | 1567/2500 [9:21:29<5:28:18, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.021491505111843265, 'learning_rate': 3.732e-07, 'completion_length': 143.2857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00467681884765625, 'epoch': 0.63} 63%|██████▎ | 1567/2500 [9:21:29<5:28:18, 21.11s/it] 63%|██████▎ | 1568/2500 [9:21:50<5:25:27, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.4131548750973262, 'learning_rate': 3.728e-07, 'completion_length': 143.6875, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.07003280520439148, 'kl': 0.0042724609375, 'epoch': 0.63} 63%|██████▎ | 1568/2500 [9:21:50<5:25:27, 20.95s/it] 63%|██████▎ | 1569/2500 [9:22:11<5:24:47, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.21882091194584244, 'learning_rate': 3.7239999999999997e-07, 'completion_length': 154.05358123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.005157470703125, 'epoch': 0.63} 63%|██████▎ | 1569/2500 [9:22:11<5:24:47, 20.93s/it] 63%|██████▎ | 1570/2500 [9:22:32<5:24:25, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.2593089209359226, 'learning_rate': 3.72e-07, 'completion_length': 145.08928680419922, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.05831881985068321, 'kl': 0.005584716796875, 'epoch': 0.63} 63%|██████▎ | 1570/2500 [9:22:32<5:24:25, 20.93s/it] 63%|██████▎ | 1571/2500 [9:22:53<5:24:11, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.5918445368370454, 'learning_rate': 3.7159999999999997e-07, 'completion_length': 150.83928680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.00537109375, 'epoch': 0.63} 63%|██████▎ | 1571/2500 [9:22:53<5:24:11, 20.94s/it] 63%|██████▎ | 1572/2500 [9:23:14<5:27:23, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.8520649110729841, 'learning_rate': 3.7119999999999994e-07, 'completion_length': 159.5982208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0073394775390625, 'epoch': 0.63} 63%|██████▎ | 1572/2500 [9:23:14<5:27:23, 21.17s/it] 63%|██████▎ | 1573/2500 [9:23:37<5:31:33, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.3995894874637347, 'learning_rate': 3.708e-07, 'completion_length': 151.06250762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00701904296875, 'epoch': 0.63} 63%|██████▎ | 1573/2500 [9:23:37<5:31:33, 21.46s/it] 63%|██████▎ | 1574/2500 [9:23:59<5:35:06, 21.71s/it] {'loss': 0.0002, 'grad_norm': 0.4761043647660602, 'learning_rate': 3.704e-07, 'completion_length': 154.35714721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0056304931640625, 'epoch': 0.63} 63%|██████▎ | 1574/2500 [9:23:59<5:35:06, 21.71s/it] 63%|██████▎ | 1575/2500 [9:24:21<5:35:56, 21.79s/it] {'loss': 0.0003, 'grad_norm': 0.04240953585426081, 'learning_rate': 3.7e-07, 'completion_length': 153.83929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0070648193359375, 'epoch': 0.63} 63%|██████▎ | 1575/2500 [9:24:21<5:35:56, 21.79s/it] 63%|██████▎ | 1576/2500 [9:24:42<5:30:21, 21.45s/it] {'loss': 0.0004, 'grad_norm': 0.49971177980600295, 'learning_rate': 3.696e-07, 'completion_length': 138.23214721679688, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.0835726372897625, 'kl': 0.00885009765625, 'epoch': 0.63} 63%|██████▎ | 1576/2500 [9:24:42<5:30:21, 21.45s/it] 63%|██████▎ | 1577/2500 [9:25:02<5:26:53, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.02425075172359365, 'learning_rate': 3.6919999999999994e-07, 'completion_length': 149.9553680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00452423095703125, 'epoch': 0.63} 63%|██████▎ | 1577/2500 [9:25:02<5:26:53, 21.25s/it] 63%|██████▎ | 1578/2500 [9:25:23<5:24:42, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.37116859195573026, 'learning_rate': 3.688e-07, 'completion_length': 149.8303680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.0050048828125, 'epoch': 0.63} 63%|██████▎ | 1578/2500 [9:25:23<5:24:42, 21.13s/it] 63%|██████▎ | 1579/2500 [9:25:44<5:23:45, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.44440594131834243, 'learning_rate': 3.684e-07, 'completion_length': 146.71429443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.00450897216796875, 'epoch': 0.63} 63%|██████▎ | 1579/2500 [9:25:44<5:23:45, 21.09s/it] 63%|██████▎ | 1580/2500 [9:26:06<5:25:40, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.3067836015499024, 'learning_rate': 3.6799999999999996e-07, 'completion_length': 148.83036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0069732666015625, 'epoch': 0.63} 63%|██████▎ | 1580/2500 [9:26:06<5:25:40, 21.24s/it] 63%|██████▎ | 1581/2500 [9:26:29<5:34:04, 21.81s/it] {'loss': 0.0003, 'grad_norm': 0.02219704522806158, 'learning_rate': 3.676e-07, 'completion_length': 166.08929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0077056884765625, 'epoch': 0.63} 63%|██████▎ | 1581/2500 [9:26:29<5:34:04, 21.81s/it] 63%|██████▎ | 1582/2500 [9:26:50<5:31:27, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.35568580595350124, 'learning_rate': 3.672e-07, 'completion_length': 147.58036041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004058837890625, 'epoch': 0.63} 63%|██████▎ | 1582/2500 [9:26:50<5:31:27, 21.66s/it] 63%|██████▎ | 1583/2500 [9:27:12<5:33:17, 21.81s/it] {'loss': 0.0001, 'grad_norm': 0.2687039515137105, 'learning_rate': 3.668e-07, 'completion_length': 139.74108123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0037078857421875, 'epoch': 0.63} 63%|██████▎ | 1583/2500 [9:27:12<5:33:17, 21.81s/it] 63%|██████▎ | 1584/2500 [9:27:34<5:32:28, 21.78s/it] {'loss': 0.0003, 'grad_norm': 0.8741958126502526, 'learning_rate': 3.664e-07, 'completion_length': 158.8482208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.007781982421875, 'epoch': 0.63} 63%|██████▎ | 1584/2500 [9:27:34<5:32:28, 21.78s/it] 63%|██████▎ | 1585/2500 [9:27:56<5:30:48, 21.69s/it] {'loss': 0.0002, 'grad_norm': 0.42564563756265955, 'learning_rate': 3.6599999999999997e-07, 'completion_length': 164.61608123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005615234375, 'epoch': 0.63} 63%|██████▎ | 1585/2500 [9:27:56<5:30:48, 21.69s/it] 63%|██████▎ | 1586/2500 [9:28:17<5:30:16, 21.68s/it] {'loss': 0.0004, 'grad_norm': 0.3292826593434482, 'learning_rate': 3.6559999999999994e-07, 'completion_length': 162.8928680419922, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.00904083251953125, 'epoch': 0.63} 63%|██████▎ | 1586/2500 [9:28:17<5:30:16, 21.68s/it] 63%|██████▎ | 1587/2500 [9:28:39<5:31:19, 21.77s/it] {'loss': 0.0003, 'grad_norm': 0.30273355571419547, 'learning_rate': 3.652e-07, 'completion_length': 142.80357360839844, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.025253813713788986, 'kl': 0.0064697265625, 'epoch': 0.63} 63%|██████▎ | 1587/2500 [9:28:39<5:31:19, 21.77s/it] 64%|██████▎ | 1588/2500 [9:29:01<5:29:28, 21.68s/it] {'loss': 0.0003, 'grad_norm': 0.6496479401904687, 'learning_rate': 3.648e-07, 'completion_length': 154.2946548461914, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0739355981349945, 'kl': 0.006591796875, 'epoch': 0.64} 64%|██████▎ | 1588/2500 [9:29:01<5:29:28, 21.68s/it] 64%|██████▎ | 1589/2500 [9:29:22<5:28:02, 21.61s/it] {'loss': 0.0001, 'grad_norm': 0.08046771751219196, 'learning_rate': 3.644e-07, 'completion_length': 146.38394165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0035858154296875, 'epoch': 0.64} 64%|██████▎ | 1589/2500 [9:29:22<5:28:02, 21.61s/it] 64%|██████▎ | 1590/2500 [9:29:44<5:27:59, 21.63s/it] {'loss': 0.0002, 'grad_norm': 0.8939061821713649, 'learning_rate': 3.64e-07, 'completion_length': 145.6607208251953, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.09918941184878349, 'kl': 0.0050048828125, 'epoch': 0.64} 64%|██████▎ | 1590/2500 [9:29:44<5:27:59, 21.63s/it] 64%|██████▎ | 1591/2500 [9:30:05<5:23:44, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.26271180490457835, 'learning_rate': 3.6359999999999995e-07, 'completion_length': 143.65178680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0045623779296875, 'epoch': 0.64} 64%|██████▎ | 1591/2500 [9:30:05<5:23:44, 21.37s/it] 64%|██████▎ | 1592/2500 [9:30:25<5:21:20, 21.23s/it] {'loss': 0.0004, 'grad_norm': 0.5641353896698683, 'learning_rate': 3.632e-07, 'completion_length': 165.65179443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.009246826171875, 'epoch': 0.64} 64%|██████▎ | 1592/2500 [9:30:25<5:21:20, 21.23s/it] 64%|██████▎ | 1593/2500 [9:30:46<5:18:10, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.027961127368272814, 'learning_rate': 3.628e-07, 'completion_length': 142.71429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00628662109375, 'epoch': 0.64} 64%|██████▎ | 1593/2500 [9:30:46<5:18:10, 21.05s/it] 64%|██████▍ | 1594/2500 [9:31:07<5:17:26, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.02674536429441206, 'learning_rate': 3.6239999999999996e-07, 'completion_length': 149.4732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005096435546875, 'epoch': 0.64} 64%|██████▍ | 1594/2500 [9:31:07<5:17:26, 21.02s/it] 64%|██████▍ | 1595/2500 [9:31:28<5:17:05, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.5340516199288895, 'learning_rate': 3.62e-07, 'completion_length': 162.58929443359375, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0053253173828125, 'epoch': 0.64} 64%|██████▍ | 1595/2500 [9:31:28<5:17:05, 21.02s/it] 64%|██████▍ | 1596/2500 [9:31:49<5:14:56, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.321743333575442, 'learning_rate': 3.6159999999999996e-07, 'completion_length': 143.0357208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.0048828125, 'epoch': 0.64} 64%|██████▍ | 1596/2500 [9:31:49<5:14:56, 20.90s/it] 64%|██████▍ | 1597/2500 [9:32:10<5:16:43, 21.05s/it] {'loss': 0.0002, 'grad_norm': 1.894563220830217, 'learning_rate': 3.612e-07, 'completion_length': 147.00000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0057525634765625, 'epoch': 0.64} 64%|██████▍ | 1597/2500 [9:32:10<5:16:43, 21.05s/it] 64%|██████▍ | 1598/2500 [9:32:31<5:17:11, 21.10s/it] {'loss': 0.0001, 'grad_norm': 0.01671468904456065, 'learning_rate': 3.608e-07, 'completion_length': 141.18750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00323486328125, 'epoch': 0.64} 64%|██████▍ | 1598/2500 [9:32:31<5:17:11, 21.10s/it] 64%|██████▍ | 1599/2500 [9:32:53<5:20:28, 21.34s/it] {'loss': 0.0002, 'grad_norm': 1.172016987682847, 'learning_rate': 3.6039999999999997e-07, 'completion_length': 163.24108123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0058746337890625, 'epoch': 0.64} 64%|██████▍ | 1599/2500 [9:32:53<5:20:28, 21.34s/it] 64%|██████▍ | 1600/2500 [9:33:15<5:22:51, 21.52s/it] {'loss': 0.0002, 'grad_norm': 0.018779812048692467, 'learning_rate': 3.6e-07, 'completion_length': 161.4732208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00506591796875, 'epoch': 0.64} 64%|██████▍ | 1600/2500 [9:33:15<5:22:51, 21.52s/it] 64%|██████▍ | 1601/2500 [9:34:26<9:04:08, 36.32s/it] {'loss': 0.0002, 'grad_norm': 0.021450587565784062, 'learning_rate': 3.5959999999999996e-07, 'completion_length': 156.5357208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049285888671875, 'epoch': 0.64} 64%|██████▍ | 1601/2500 [9:34:26<9:04:08, 36.32s/it] 64%|██████▍ | 1602/2500 [9:34:47<7:54:21, 31.69s/it] {'loss': 0.0002, 'grad_norm': 0.33810783994409027, 'learning_rate': 3.592e-07, 'completion_length': 149.95536041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.07636035978794098, 'kl': 0.0043182373046875, 'epoch': 0.64} 64%|██████▍ | 1602/2500 [9:34:47<7:54:21, 31.69s/it] 64%|██████▍ | 1603/2500 [9:35:08<7:08:00, 28.63s/it] {'loss': 0.0002, 'grad_norm': 0.6273134917941174, 'learning_rate': 3.588e-07, 'completion_length': 161.83929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.06613001227378845, 'kl': 0.0061798095703125, 'epoch': 0.64} 64%|██████▍ | 1603/2500 [9:35:08<7:08:00, 28.63s/it] 64%|██████▍ | 1604/2500 [9:35:30<6:36:15, 26.53s/it] {'loss': 0.0002, 'grad_norm': 0.41438456155990405, 'learning_rate': 3.584e-07, 'completion_length': 163.15179443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0060272216796875, 'epoch': 0.64} 64%|██████▍ | 1604/2500 [9:35:30<6:36:15, 26.53s/it] 64%|██████▍ | 1605/2500 [9:35:52<6:16:55, 25.27s/it] {'loss': 0.0002, 'grad_norm': 0.022251700838533915, 'learning_rate': 3.5799999999999995e-07, 'completion_length': 154.46429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053863525390625, 'epoch': 0.64} 64%|██████▍ | 1605/2500 [9:35:52<6:16:55, 25.27s/it] 64%|██████▍ | 1606/2500 [9:36:14<5:58:33, 24.06s/it] {'loss': 0.0003, 'grad_norm': 0.04686026291355191, 'learning_rate': 3.5759999999999997e-07, 'completion_length': 158.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006927490234375, 'epoch': 0.64} 64%|██████▍ | 1606/2500 [9:36:14<5:58:33, 24.06s/it] 64%|██████▍ | 1607/2500 [9:36:35<5:44:18, 23.13s/it] {'loss': 0.0002, 'grad_norm': 1.6290670721883536, 'learning_rate': 3.572e-07, 'completion_length': 145.08036041259766, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.03818017989397049, 'kl': 0.00565338134765625, 'epoch': 0.64} 64%|██████▍ | 1607/2500 [9:36:35<5:44:18, 23.13s/it] 64%|██████▍ | 1608/2500 [9:36:56<5:38:23, 22.76s/it] {'loss': 0.0003, 'grad_norm': 0.49896895027466914, 'learning_rate': 3.5679999999999997e-07, 'completion_length': 157.4107208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.007568359375, 'epoch': 0.64} 64%|██████▍ | 1608/2500 [9:36:56<5:38:23, 22.76s/it] 64%|██████▍ | 1609/2500 [9:37:18<5:33:01, 22.43s/it] {'loss': 0.0002, 'grad_norm': 0.4759171428208411, 'learning_rate': 3.564e-07, 'completion_length': 152.82144165039062, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.0054168701171875, 'epoch': 0.64} 64%|██████▍ | 1609/2500 [9:37:18<5:33:01, 22.43s/it] 64%|██████▍ | 1610/2500 [9:37:39<5:27:07, 22.05s/it] {'loss': 0.0002, 'grad_norm': 0.2883931711274458, 'learning_rate': 3.5599999999999996e-07, 'completion_length': 152.58929443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.005523681640625, 'epoch': 0.64} 64%|██████▍ | 1610/2500 [9:37:39<5:27:07, 22.05s/it] 64%|██████▍ | 1611/2500 [9:38:01<5:24:16, 21.89s/it] {'loss': 0.0002, 'grad_norm': 0.3145368460039268, 'learning_rate': 3.5560000000000003e-07, 'completion_length': 148.00000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004913330078125, 'epoch': 0.64} 64%|██████▍ | 1611/2500 [9:38:01<5:24:16, 21.89s/it] 64%|██████▍ | 1612/2500 [9:38:22<5:19:59, 21.62s/it] {'loss': 0.0002, 'grad_norm': 0.19893440719016814, 'learning_rate': 3.552e-07, 'completion_length': 148.0803680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005340576171875, 'epoch': 0.64} 64%|██████▍ | 1612/2500 [9:38:22<5:19:59, 21.62s/it] 65%|██████▍ | 1613/2500 [9:38:43<5:20:01, 21.65s/it] {'loss': 0.0002, 'grad_norm': 0.20770058819582618, 'learning_rate': 3.548e-07, 'completion_length': 162.40178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00435638427734375, 'epoch': 0.65} 65%|██████▍ | 1613/2500 [9:38:43<5:20:01, 21.65s/it] 65%|██████▍ | 1614/2500 [9:39:05<5:18:51, 21.59s/it] {'loss': 0.0003, 'grad_norm': 0.04703242397363467, 'learning_rate': 3.544e-07, 'completion_length': 148.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0072784423828125, 'epoch': 0.65} 65%|██████▍ | 1614/2500 [9:39:05<5:18:51, 21.59s/it] 65%|██████▍ | 1615/2500 [9:39:26<5:16:35, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.018083796909749334, 'learning_rate': 3.5399999999999997e-07, 'completion_length': 150.81250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055084228515625, 'epoch': 0.65} 65%|██████▍ | 1615/2500 [9:39:26<5:16:35, 21.46s/it] 65%|██████▍ | 1616/2500 [9:39:48<5:16:13, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.020362564692188275, 'learning_rate': 3.536e-07, 'completion_length': 144.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006683349609375, 'epoch': 0.65} 65%|██████▍ | 1616/2500 [9:39:48<5:16:13, 21.46s/it] 65%|██████▍ | 1617/2500 [9:40:09<5:17:09, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.35784997808139196, 'learning_rate': 3.532e-07, 'completion_length': 166.52679443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.006988525390625, 'epoch': 0.65} 65%|██████▍ | 1617/2500 [9:40:09<5:17:09, 21.55s/it] 65%|██████▍ | 1618/2500 [9:40:31<5:17:35, 21.60s/it] {'loss': 0.0003, 'grad_norm': 0.3271428631338286, 'learning_rate': 3.528e-07, 'completion_length': 161.68750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0073394775390625, 'epoch': 0.65} 65%|██████▍ | 1618/2500 [9:40:31<5:17:35, 21.60s/it] 65%|██████▍ | 1619/2500 [9:40:53<5:18:20, 21.68s/it] {'loss': 0.0002, 'grad_norm': 0.5135636940843233, 'learning_rate': 3.5239999999999995e-07, 'completion_length': 164.99108123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.005462646484375, 'epoch': 0.65} 65%|██████▍ | 1619/2500 [9:40:53<5:18:20, 21.68s/it] 65%|██████▍ | 1620/2500 [9:41:15<5:17:53, 21.67s/it] {'loss': 0.0003, 'grad_norm': 0.2384426487948228, 'learning_rate': 3.52e-07, 'completion_length': 146.90179443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0065155029296875, 'epoch': 0.65} 65%|██████▍ | 1620/2500 [9:41:15<5:17:53, 21.67s/it] 65%|██████▍ | 1621/2500 [9:41:36<5:16:06, 21.58s/it] {'loss': 0.0002, 'grad_norm': 0.04070278858266602, 'learning_rate': 3.516e-07, 'completion_length': 147.52678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055999755859375, 'epoch': 0.65} 65%|██████▍ | 1621/2500 [9:41:36<5:16:06, 21.58s/it] 65%|██████▍ | 1622/2500 [9:41:57<5:12:02, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.18563248142409017, 'learning_rate': 3.512e-07, 'completion_length': 145.9107208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.005706787109375, 'epoch': 0.65} 65%|██████▍ | 1622/2500 [9:41:57<5:12:02, 21.32s/it] 65%|██████▍ | 1623/2500 [9:42:18<5:12:44, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.6847880805875877, 'learning_rate': 3.508e-07, 'completion_length': 148.71429443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.007171630859375, 'epoch': 0.65} 65%|██████▍ | 1623/2500 [9:42:18<5:12:44, 21.40s/it] 65%|██████▍ | 1624/2500 [9:42:40<5:13:09, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.02705016320834551, 'learning_rate': 3.5039999999999996e-07, 'completion_length': 150.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006500244140625, 'epoch': 0.65} 65%|██████▍ | 1624/2500 [9:42:40<5:13:09, 21.45s/it] 65%|██████▌ | 1625/2500 [9:43:02<5:15:07, 21.61s/it] {'loss': 0.0002, 'grad_norm': 0.015609978128323524, 'learning_rate': 3.5e-07, 'completion_length': 160.3839340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049591064453125, 'epoch': 0.65} 65%|██████▌ | 1625/2500 [9:43:02<5:15:07, 21.61s/it] 65%|██████▌ | 1626/2500 [9:43:24<5:16:25, 21.72s/it] {'loss': 0.0003, 'grad_norm': 0.4603947884135309, 'learning_rate': 3.496e-07, 'completion_length': 164.15179443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0081329345703125, 'epoch': 0.65} 65%|██████▌ | 1626/2500 [9:43:24<5:16:25, 21.72s/it] 65%|██████▌ | 1627/2500 [9:43:45<5:13:52, 21.57s/it] {'loss': 0.0003, 'grad_norm': 0.6106040097114364, 'learning_rate': 3.492e-07, 'completion_length': 155.2946548461914, 'rewards/accuracy_reward': 0.8660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.8660714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006561279296875, 'epoch': 0.65} 65%|██████▌ | 1627/2500 [9:43:45<5:13:52, 21.57s/it] 65%|██████▌ | 1628/2500 [9:44:06<5:13:02, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.018941910755481975, 'learning_rate': 3.488e-07, 'completion_length': 151.8214340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00518798828125, 'epoch': 0.65} 65%|██████▌ | 1628/2500 [9:44:06<5:13:02, 21.54s/it] 65%|██████▌ | 1629/2500 [9:44:28<5:12:35, 21.53s/it] {'loss': 0.0002, 'grad_norm': 0.02495910919707231, 'learning_rate': 3.4839999999999997e-07, 'completion_length': 156.7589340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005401611328125, 'epoch': 0.65} 65%|██████▌ | 1629/2500 [9:44:28<5:12:35, 21.53s/it] 65%|██████▌ | 1630/2500 [9:44:49<5:10:31, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.19627839550614826, 'learning_rate': 3.4799999999999994e-07, 'completion_length': 152.30358123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00518798828125, 'epoch': 0.65} 65%|██████▌ | 1630/2500 [9:44:49<5:10:31, 21.42s/it] 65%|██████▌ | 1631/2500 [9:45:10<5:07:55, 21.26s/it] {'loss': 0.0003, 'grad_norm': 0.4118682157038886, 'learning_rate': 3.476e-07, 'completion_length': 154.3482208251953, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726372897625, 'kl': 0.0075225830078125, 'epoch': 0.65} 65%|██████▌ | 1631/2500 [9:45:10<5:07:55, 21.26s/it] 65%|██████▌ | 1632/2500 [9:45:31<5:05:43, 21.13s/it] {'loss': 0.0003, 'grad_norm': 0.43761321299015, 'learning_rate': 3.472e-07, 'completion_length': 155.24107360839844, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0087432861328125, 'epoch': 0.65} 65%|██████▌ | 1632/2500 [9:45:31<5:05:43, 21.13s/it] 65%|██████▌ | 1633/2500 [9:45:52<5:04:33, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.24971377687758373, 'learning_rate': 3.4679999999999996e-07, 'completion_length': 143.11608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00466156005859375, 'epoch': 0.65} 65%|██████▌ | 1633/2500 [9:45:52<5:04:33, 21.08s/it] 65%|██████▌ | 1634/2500 [9:46:13<5:04:52, 21.12s/it] {'loss': 0.0004, 'grad_norm': 0.45345961315101013, 'learning_rate': 3.464e-07, 'completion_length': 153.02679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.009063720703125, 'epoch': 0.65} 65%|██████▌ | 1634/2500 [9:46:13<5:04:52, 21.12s/it] 65%|██████▌ | 1635/2500 [9:46:35<5:06:13, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.03173703341317845, 'learning_rate': 3.4599999999999995e-07, 'completion_length': 157.0714340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.65} 65%|██████▌ | 1635/2500 [9:46:35<5:06:13, 21.24s/it] 65%|██████▌ | 1636/2500 [9:46:55<5:02:49, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.07605472839681988, 'learning_rate': 3.456e-07, 'completion_length': 145.2678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.008697509765625, 'epoch': 0.65} 65%|██████▌ | 1636/2500 [9:46:55<5:02:49, 21.03s/it] 65%|██████▌ | 1637/2500 [9:47:16<5:02:25, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.5419097771482899, 'learning_rate': 3.452e-07, 'completion_length': 145.6339340209961, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941184878349, 'kl': 0.0054931640625, 'epoch': 0.65} 65%|██████▌ | 1637/2500 [9:47:16<5:02:25, 21.03s/it] 66%|██████▌ | 1638/2500 [9:47:38<5:06:14, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.20324990399951195, 'learning_rate': 3.4479999999999996e-07, 'completion_length': 163.10714721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00562286376953125, 'epoch': 0.66} 66%|██████▌ | 1638/2500 [9:47:38<5:06:14, 21.32s/it] 66%|██████▌ | 1639/2500 [9:47:59<5:05:23, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.4122402674838465, 'learning_rate': 3.444e-07, 'completion_length': 155.18750762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.005584716796875, 'epoch': 0.66} 66%|██████▌ | 1639/2500 [9:47:59<5:05:23, 21.28s/it] 66%|██████▌ | 1640/2500 [9:48:21<5:05:03, 21.28s/it] {'loss': 0.0004, 'grad_norm': 0.2416857750828973, 'learning_rate': 3.4399999999999996e-07, 'completion_length': 146.8928680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0090789794921875, 'epoch': 0.66} 66%|██████▌ | 1640/2500 [9:48:21<5:05:03, 21.28s/it] 66%|██████▌ | 1641/2500 [9:48:42<5:03:51, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.03039788770342184, 'learning_rate': 3.436e-07, 'completion_length': 144.75000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049591064453125, 'epoch': 0.66} 66%|██████▌ | 1641/2500 [9:48:42<5:03:51, 21.22s/it] 66%|██████▌ | 1642/2500 [9:49:03<5:03:37, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.45596842042682506, 'learning_rate': 3.432e-07, 'completion_length': 138.92858123779297, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.005035400390625, 'epoch': 0.66} 66%|██████▌ | 1642/2500 [9:49:03<5:03:37, 21.23s/it] 66%|██████▌ | 1643/2500 [9:49:25<5:04:59, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.04973545892077559, 'learning_rate': 3.4279999999999997e-07, 'completion_length': 162.3839340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0061187744140625, 'epoch': 0.66} 66%|██████▌ | 1643/2500 [9:49:25<5:04:59, 21.35s/it] 66%|██████▌ | 1644/2500 [9:49:46<5:04:20, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.3817181161408066, 'learning_rate': 3.4239999999999994e-07, 'completion_length': 149.15179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064849853515625, 'epoch': 0.66} 66%|██████▌ | 1644/2500 [9:49:46<5:04:20, 21.33s/it] 66%|██████▌ | 1645/2500 [9:50:07<5:02:42, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.06242736873483101, 'learning_rate': 3.42e-07, 'completion_length': 148.1428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00405120849609375, 'epoch': 0.66} 66%|██████▌ | 1645/2500 [9:50:07<5:02:42, 21.24s/it] 66%|██████▌ | 1646/2500 [9:50:27<4:59:24, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.5959874854717242, 'learning_rate': 3.416e-07, 'completion_length': 137.11607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00568389892578125, 'epoch': 0.66} 66%|██████▌ | 1646/2500 [9:50:27<4:59:24, 21.04s/it] 66%|██████▌ | 1647/2500 [9:50:48<4:58:45, 21.01s/it] {'loss': 0.0002, 'grad_norm': 1.0728623063993754, 'learning_rate': 3.412e-07, 'completion_length': 140.76786041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0050811767578125, 'epoch': 0.66} 66%|██████▌ | 1647/2500 [9:50:48<4:58:45, 21.01s/it] 66%|██████▌ | 1648/2500 [9:51:10<4:59:34, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.497812685566773, 'learning_rate': 3.408e-07, 'completion_length': 159.4732208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.0059356689453125, 'epoch': 0.66} 66%|██████▌ | 1648/2500 [9:51:10<4:59:34, 21.10s/it] 66%|██████▌ | 1649/2500 [9:51:30<4:57:32, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.38960064091271035, 'learning_rate': 3.4039999999999995e-07, 'completion_length': 138.35714721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0049285888671875, 'epoch': 0.66} 66%|██████▌ | 1649/2500 [9:51:30<4:57:32, 20.98s/it] 66%|██████▌ | 1650/2500 [9:51:52<4:58:50, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.23048203465654607, 'learning_rate': 3.4000000000000003e-07, 'completion_length': 157.6071548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00383758544921875, 'epoch': 0.66} 66%|██████▌ | 1650/2500 [9:51:52<4:58:50, 21.10s/it] 66%|██████▌ | 1651/2500 [9:52:13<4:57:37, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.6999462979354809, 'learning_rate': 3.396e-07, 'completion_length': 156.2589340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.005615234375, 'epoch': 0.66} 66%|██████▌ | 1651/2500 [9:52:13<4:57:37, 21.03s/it] 66%|██████▌ | 1652/2500 [9:52:33<4:56:11, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.24832746914375015, 'learning_rate': 3.3919999999999997e-07, 'completion_length': 139.26786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0039215087890625, 'epoch': 0.66} 66%|██████▌ | 1652/2500 [9:52:33<4:56:11, 20.96s/it] 66%|██████▌ | 1653/2500 [9:52:55<4:57:33, 21.08s/it] {'loss': 0.0001, 'grad_norm': 0.29712587762216547, 'learning_rate': 3.388e-07, 'completion_length': 141.0803680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00327301025390625, 'epoch': 0.66} 66%|██████▌ | 1653/2500 [9:52:55<4:57:33, 21.08s/it] 66%|██████▌ | 1654/2500 [9:53:16<4:58:25, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.03273422014119911, 'learning_rate': 3.3839999999999996e-07, 'completion_length': 153.60714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005523681640625, 'epoch': 0.66} 66%|██████▌ | 1654/2500 [9:53:16<4:58:25, 21.17s/it] 66%|██████▌ | 1655/2500 [9:53:38<5:00:24, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.2859221287382082, 'learning_rate': 3.38e-07, 'completion_length': 166.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0072021484375, 'epoch': 0.66} 66%|██████▌ | 1655/2500 [9:53:38<5:00:24, 21.33s/it] 66%|██████▌ | 1656/2500 [9:53:58<4:56:12, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.3810855460350583, 'learning_rate': 3.376e-07, 'completion_length': 132.7589340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00408172607421875, 'epoch': 0.66} 66%|██████▌ | 1656/2500 [9:53:58<4:56:12, 21.06s/it] 66%|██████▋ | 1657/2500 [9:54:20<4:58:03, 21.21s/it] {'loss': 0.0005, 'grad_norm': 0.8243858318394877, 'learning_rate': 3.372e-07, 'completion_length': 147.2232208251953, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.1121157705783844, 'kl': 0.011260986328125, 'epoch': 0.66} 66%|██████▋ | 1657/2500 [9:54:20<4:58:03, 21.21s/it] 66%|██████▋ | 1658/2500 [9:54:42<5:01:34, 21.49s/it] {'loss': 0.0003, 'grad_norm': 0.520850357272088, 'learning_rate': 3.368e-07, 'completion_length': 144.56250762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0066375732421875, 'epoch': 0.66} 66%|██████▋ | 1658/2500 [9:54:42<5:01:34, 21.49s/it] 66%|██████▋ | 1659/2500 [9:55:04<5:02:59, 21.62s/it] {'loss': 0.0001, 'grad_norm': 0.019126183781202667, 'learning_rate': 3.3639999999999997e-07, 'completion_length': 141.58928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032806396484375, 'epoch': 0.66} 66%|██████▋ | 1659/2500 [9:55:04<5:02:59, 21.62s/it] 66%|██████▋ | 1660/2500 [9:55:26<5:06:36, 21.90s/it] {'loss': 0.0002, 'grad_norm': 0.01950189226696958, 'learning_rate': 3.36e-07, 'completion_length': 154.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00490570068359375, 'epoch': 0.66} 66%|██████▋ | 1660/2500 [9:55:26<5:06:36, 21.90s/it] 66%|██████▋ | 1661/2500 [9:55:48<5:05:51, 21.87s/it] {'loss': 0.0004, 'grad_norm': 0.5362644125898761, 'learning_rate': 3.356e-07, 'completion_length': 166.5446548461914, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10882645100355148, 'kl': 0.010223388671875, 'epoch': 0.66} 66%|██████▋ | 1661/2500 [9:55:48<5:05:51, 21.87s/it] 66%|██████▋ | 1662/2500 [9:56:10<5:03:16, 21.71s/it] {'loss': 0.0002, 'grad_norm': 0.2332009537791132, 'learning_rate': 3.352e-07, 'completion_length': 148.30358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0051422119140625, 'epoch': 0.66} 66%|██████▋ | 1662/2500 [9:56:10<5:03:16, 21.71s/it] 67%|██████▋ | 1663/2500 [9:56:31<5:00:45, 21.56s/it] {'loss': 0.0003, 'grad_norm': 1.1110355610104166, 'learning_rate': 3.3479999999999995e-07, 'completion_length': 150.24108123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.1030978113412857, 'kl': 0.0069122314453125, 'epoch': 0.67} 67%|██████▋ | 1663/2500 [9:56:31<5:00:45, 21.56s/it] 67%|██████▋ | 1664/2500 [9:56:53<5:03:43, 21.80s/it] {'loss': 0.0002, 'grad_norm': 0.05000303768065213, 'learning_rate': 3.344e-07, 'completion_length': 158.62500762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0052032470703125, 'epoch': 0.67} 67%|██████▋ | 1664/2500 [9:56:53<5:03:43, 21.80s/it] 67%|██████▋ | 1665/2500 [9:57:14<5:00:23, 21.58s/it] {'loss': 0.0002, 'grad_norm': 0.9245403566788924, 'learning_rate': 3.34e-07, 'completion_length': 149.49108123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00450897216796875, 'epoch': 0.67} 67%|██████▋ | 1665/2500 [9:57:14<5:00:23, 21.58s/it] 67%|██████▋ | 1666/2500 [9:57:35<4:58:29, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.024981983992235775, 'learning_rate': 3.3359999999999997e-07, 'completion_length': 144.6696548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.67} 67%|██████▋ | 1666/2500 [9:57:35<4:58:29, 21.47s/it] 67%|██████▋ | 1667/2500 [9:57:56<4:55:10, 21.26s/it] {'loss': 0.0001, 'grad_norm': 0.014941543907644177, 'learning_rate': 3.332e-07, 'completion_length': 134.40179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00298309326171875, 'epoch': 0.67} 67%|██████▋ | 1667/2500 [9:57:56<4:55:10, 21.26s/it] 67%|██████▋ | 1668/2500 [9:58:19<4:59:07, 21.57s/it] {'loss': 0.0002, 'grad_norm': 0.876801387680781, 'learning_rate': 3.3279999999999996e-07, 'completion_length': 159.54464721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797151088715, 'kl': 0.00592041015625, 'epoch': 0.67} 67%|██████▋ | 1668/2500 [9:58:19<4:59:07, 21.57s/it] 67%|██████▋ | 1669/2500 [9:58:41<5:01:06, 21.74s/it] {'loss': 0.0002, 'grad_norm': 0.7286111510393697, 'learning_rate': 3.3239999999999993e-07, 'completion_length': 138.6071548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00439453125, 'epoch': 0.67} 67%|██████▋ | 1669/2500 [9:58:41<5:01:06, 21.74s/it] 67%|██████▋ | 1670/2500 [9:59:01<4:56:37, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.2690816130103397, 'learning_rate': 3.32e-07, 'completion_length': 152.9107208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005035400390625, 'epoch': 0.67} 67%|██████▋ | 1670/2500 [9:59:01<4:56:37, 21.44s/it] 67%|██████▋ | 1671/2500 [9:59:23<4:56:19, 21.45s/it] {'loss': 0.0004, 'grad_norm': 0.42695658669176556, 'learning_rate': 3.316e-07, 'completion_length': 140.7589340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.00919342041015625, 'epoch': 0.67} 67%|██████▋ | 1671/2500 [9:59:23<4:56:19, 21.45s/it] 67%|██████▋ | 1672/2500 [9:59:45<4:57:17, 21.54s/it] {'loss': 0.0001, 'grad_norm': 0.020024619891124824, 'learning_rate': 3.312e-07, 'completion_length': 143.92857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003692626953125, 'epoch': 0.67} 67%|██████▋ | 1672/2500 [9:59:45<4:57:17, 21.54s/it] 67%|██████▋ | 1673/2500 [10:00:06<4:58:12, 21.64s/it] {'loss': 0.0002, 'grad_norm': 0.5114159906108838, 'learning_rate': 3.3079999999999997e-07, 'completion_length': 159.46429443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.005462646484375, 'epoch': 0.67} 67%|██████▋ | 1673/2500 [10:00:06<4:58:12, 21.64s/it] 67%|██████▋ | 1674/2500 [10:00:27<4:55:04, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.14074483674800023, 'learning_rate': 3.304e-07, 'completion_length': 141.04464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00421142578125, 'epoch': 0.67} 67%|██████▋ | 1674/2500 [10:00:27<4:55:04, 21.43s/it] 67%|██████▋ | 1675/2500 [10:00:49<4:54:02, 21.38s/it] {'loss': 0.0001, 'grad_norm': 0.26777607852514945, 'learning_rate': 3.3e-07, 'completion_length': 135.8571548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.002899169921875, 'epoch': 0.67} 67%|██████▋ | 1675/2500 [10:00:49<4:54:02, 21.38s/it] 67%|██████▋ | 1676/2500 [10:01:10<4:53:22, 21.36s/it] {'loss': 0.0002, 'grad_norm': 0.02889732377957734, 'learning_rate': 3.296e-07, 'completion_length': 142.8303680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.67} 67%|██████▋ | 1676/2500 [10:01:10<4:53:22, 21.36s/it] 67%|██████▋ | 1677/2500 [10:01:31<4:50:42, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.03526269818135638, 'learning_rate': 3.2919999999999996e-07, 'completion_length': 147.90178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045318603515625, 'epoch': 0.67} 67%|██████▋ | 1677/2500 [10:01:31<4:50:42, 21.19s/it] 67%|██████▋ | 1678/2500 [10:01:53<4:52:28, 21.35s/it] {'loss': 0.0004, 'grad_norm': 0.20898715000070578, 'learning_rate': 3.288e-07, 'completion_length': 147.68750762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.009979248046875, 'epoch': 0.67} 67%|██████▋ | 1678/2500 [10:01:53<4:52:28, 21.35s/it] 67%|██████▋ | 1679/2500 [10:02:14<4:51:59, 21.34s/it] {'loss': 0.0003, 'grad_norm': 0.03211426363060676, 'learning_rate': 3.284e-07, 'completion_length': 160.4464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0077972412109375, 'epoch': 0.67} 67%|██████▋ | 1679/2500 [10:02:14<4:51:59, 21.34s/it] 67%|██████▋ | 1680/2500 [10:02:35<4:51:13, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.5119592899642763, 'learning_rate': 3.28e-07, 'completion_length': 139.9553680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0040435791015625, 'epoch': 0.67} 67%|██████▋ | 1680/2500 [10:02:35<4:51:13, 21.31s/it] 67%|██████▋ | 1681/2500 [10:02:56<4:48:30, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.25201296729731654, 'learning_rate': 3.276e-07, 'completion_length': 144.80357360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00543212890625, 'epoch': 0.67} 67%|██████▋ | 1681/2500 [10:02:56<4:48:30, 21.14s/it] 67%|██████▋ | 1682/2500 [10:03:18<4:50:40, 21.32s/it] {'loss': 0.0001, 'grad_norm': 0.017903963542566823, 'learning_rate': 3.2719999999999997e-07, 'completion_length': 149.52679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00334930419921875, 'epoch': 0.67} 67%|██████▋ | 1682/2500 [10:03:18<4:50:40, 21.32s/it] 67%|██████▋ | 1683/2500 [10:03:39<4:48:50, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.5854505060525049, 'learning_rate': 3.268e-07, 'completion_length': 146.36608123779297, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.09528662264347076, 'kl': 0.00643157958984375, 'epoch': 0.67} 67%|██████▋ | 1683/2500 [10:03:39<4:48:50, 21.21s/it] 67%|██████▋ | 1684/2500 [10:04:00<4:51:27, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.32253757686700535, 'learning_rate': 3.264e-07, 'completion_length': 154.1428680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00525665283203125, 'epoch': 0.67} 67%|██████▋ | 1684/2500 [10:04:01<4:51:27, 21.43s/it] 67%|██████▋ | 1685/2500 [10:04:24<4:58:08, 21.95s/it] {'loss': 0.0004, 'grad_norm': 0.24026566113746095, 'learning_rate': 3.26e-07, 'completion_length': 159.8839340209961, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.009613037109375, 'epoch': 0.67} 67%|██████▋ | 1685/2500 [10:04:24<4:58:08, 21.95s/it] 67%|██████▋ | 1686/2500 [10:04:45<4:56:01, 21.82s/it] {'loss': 0.0003, 'grad_norm': 0.2139254921242637, 'learning_rate': 3.256e-07, 'completion_length': 149.67858123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.007476806640625, 'epoch': 0.67} 67%|██████▋ | 1686/2500 [10:04:45<4:56:01, 21.82s/it] 67%|██████▋ | 1687/2500 [10:05:07<4:54:53, 21.76s/it] {'loss': 0.0002, 'grad_norm': 0.018460179339399946, 'learning_rate': 3.252e-07, 'completion_length': 153.17857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005035400390625, 'epoch': 0.67} 67%|██████▋ | 1687/2500 [10:05:07<4:54:53, 21.76s/it] 68%|██████▊ | 1688/2500 [10:05:28<4:53:20, 21.68s/it] {'loss': 0.0002, 'grad_norm': 0.38494862573822536, 'learning_rate': 3.2479999999999994e-07, 'completion_length': 149.9107208251953, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0053558349609375, 'epoch': 0.68} 68%|██████▊ | 1688/2500 [10:05:28<4:53:20, 21.68s/it] 68%|██████▊ | 1689/2500 [10:05:49<4:50:16, 21.48s/it] {'loss': 0.0002, 'grad_norm': 2.2749570898310636, 'learning_rate': 3.244e-07, 'completion_length': 146.8482208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.0739355981349945, 'kl': 0.0051422119140625, 'epoch': 0.68} 68%|██████▊ | 1689/2500 [10:05:49<4:50:16, 21.48s/it] 68%|██████▊ | 1690/2500 [10:06:11<4:50:57, 21.55s/it] {'loss': 0.0004, 'grad_norm': 0.25907289174675674, 'learning_rate': 3.24e-07, 'completion_length': 154.60714721679688, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.025253813713788986, 'kl': 0.0106201171875, 'epoch': 0.68} 68%|██████▊ | 1690/2500 [10:06:11<4:50:57, 21.55s/it] 68%|██████▊ | 1691/2500 [10:06:32<4:50:00, 21.51s/it] {'loss': 0.0002, 'grad_norm': 0.01717181222386224, 'learning_rate': 3.2359999999999996e-07, 'completion_length': 153.25000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045623779296875, 'epoch': 0.68} 68%|██████▊ | 1691/2500 [10:06:32<4:50:00, 21.51s/it] 68%|██████▊ | 1692/2500 [10:06:53<4:45:46, 21.22s/it] {'loss': 0.0001, 'grad_norm': 0.24085037606662302, 'learning_rate': 3.232e-07, 'completion_length': 141.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00370025634765625, 'epoch': 0.68} 68%|██████▊ | 1692/2500 [10:06:53<4:45:46, 21.22s/it] 68%|██████▊ | 1693/2500 [10:07:14<4:45:22, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.023702334770905432, 'learning_rate': 3.2279999999999995e-07, 'completion_length': 168.46428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00592041015625, 'epoch': 0.68} 68%|██████▊ | 1693/2500 [10:07:14<4:45:22, 21.22s/it] 68%|██████▊ | 1694/2500 [10:07:35<4:42:31, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.3737658072316598, 'learning_rate': 3.2240000000000003e-07, 'completion_length': 166.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006103515625, 'epoch': 0.68} 68%|██████▊ | 1694/2500 [10:07:35<4:42:31, 21.03s/it] 68%|██████▊ | 1695/2500 [10:07:55<4:40:33, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.6823485111459793, 'learning_rate': 3.22e-07, 'completion_length': 149.71429443359375, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004180908203125, 'epoch': 0.68} 68%|██████▊ | 1695/2500 [10:07:55<4:40:33, 20.91s/it] 68%|██████▊ | 1696/2500 [10:08:17<4:42:43, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.5995538242590356, 'learning_rate': 3.2159999999999997e-07, 'completion_length': 140.12500762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0739355981349945, 'kl': 0.005767822265625, 'epoch': 0.68} 68%|██████▊ | 1696/2500 [10:08:17<4:42:43, 21.10s/it] 68%|██████▊ | 1697/2500 [10:08:37<4:39:19, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.1599470446684991, 'learning_rate': 3.212e-07, 'completion_length': 143.9107208251953, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.03696779906749725, 'kl': 0.00461578369140625, 'epoch': 0.68} 68%|██████▊ | 1697/2500 [10:08:37<4:39:19, 20.87s/it] 68%|██████▊ | 1698/2500 [10:08:58<4:38:13, 20.81s/it] {'loss': 0.0002, 'grad_norm': 0.8245620328015786, 'learning_rate': 3.2079999999999996e-07, 'completion_length': 147.75894165039062, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00443267822265625, 'epoch': 0.68} 68%|██████▊ | 1698/2500 [10:08:58<4:38:13, 20.81s/it] 68%|██████▊ | 1699/2500 [10:09:20<4:42:04, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.45446598781680275, 'learning_rate': 3.204e-07, 'completion_length': 162.56250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0060577392578125, 'epoch': 0.68} 68%|██████▊ | 1699/2500 [10:09:20<4:42:04, 21.13s/it] 68%|██████▊ | 1700/2500 [10:09:41<4:40:10, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.292489758076449, 'learning_rate': 3.2e-07, 'completion_length': 144.90179443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00399017333984375, 'epoch': 0.68} 68%|██████▊ | 1700/2500 [10:09:41<4:40:10, 21.01s/it] 68%|██████▊ | 1701/2500 [10:10:50<7:55:05, 35.68s/it] {'loss': 0.0002, 'grad_norm': 0.020055459359571797, 'learning_rate': 3.196e-07, 'completion_length': 151.2857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0057373046875, 'epoch': 0.68} 68%|██████▊ | 1701/2500 [10:10:50<7:55:05, 35.68s/it] 68%|██████▊ | 1702/2500 [10:11:11<6:54:53, 31.19s/it] {'loss': 0.0002, 'grad_norm': 0.8171764037869187, 'learning_rate': 3.1919999999999995e-07, 'completion_length': 151.24108123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0058441162109375, 'epoch': 0.68} 68%|██████▊ | 1702/2500 [10:11:11<6:54:53, 31.19s/it] 68%|██████▊ | 1703/2500 [10:11:32<6:12:15, 28.02s/it] {'loss': 0.0002, 'grad_norm': 0.06981039798120595, 'learning_rate': 3.1879999999999997e-07, 'completion_length': 143.02679443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0044403076171875, 'epoch': 0.68} 68%|██████▊ | 1703/2500 [10:11:32<6:12:15, 28.02s/it] 68%|██████▊ | 1704/2500 [10:11:53<5:44:14, 25.95s/it] {'loss': 0.0003, 'grad_norm': 0.2861846876173314, 'learning_rate': 3.184e-07, 'completion_length': 168.24108123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0068359375, 'epoch': 0.68} 68%|██████▊ | 1704/2500 [10:11:53<5:44:14, 25.95s/it] 68%|██████▊ | 1705/2500 [10:12:14<5:25:25, 24.56s/it] {'loss': 0.0003, 'grad_norm': 0.27362561828383675, 'learning_rate': 3.18e-07, 'completion_length': 155.4553680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.007171630859375, 'epoch': 0.68} 68%|██████▊ | 1705/2500 [10:12:14<5:25:25, 24.56s/it] 68%|██████▊ | 1706/2500 [10:12:35<5:10:08, 23.44s/it] {'loss': 0.0002, 'grad_norm': 0.29771455944286496, 'learning_rate': 3.176e-07, 'completion_length': 144.8839340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0047454833984375, 'epoch': 0.68} 68%|██████▊ | 1706/2500 [10:12:35<5:10:08, 23.44s/it] 68%|██████▊ | 1707/2500 [10:12:56<4:59:44, 22.68s/it] {'loss': 0.0002, 'grad_norm': 0.3388606996228282, 'learning_rate': 3.1719999999999996e-07, 'completion_length': 149.46428680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004974365234375, 'epoch': 0.68} 68%|██████▊ | 1707/2500 [10:12:56<4:59:44, 22.68s/it] 68%|██████▊ | 1708/2500 [10:13:17<4:54:21, 22.30s/it] {'loss': 0.0003, 'grad_norm': 1.9311608818944677, 'learning_rate': 3.1680000000000003e-07, 'completion_length': 146.40178680419922, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.0835726335644722, 'kl': 0.0077667236328125, 'epoch': 0.68} 68%|██████▊ | 1708/2500 [10:13:17<4:54:21, 22.30s/it] 68%|██████▊ | 1709/2500 [10:13:38<4:48:19, 21.87s/it] {'loss': 0.0002, 'grad_norm': 0.19003776403239475, 'learning_rate': 3.164e-07, 'completion_length': 145.65179443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0052490234375, 'epoch': 0.68} 68%|██████▊ | 1709/2500 [10:13:38<4:48:19, 21.87s/it] 68%|██████▊ | 1710/2500 [10:13:59<4:43:50, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.40739401669086456, 'learning_rate': 3.1599999999999997e-07, 'completion_length': 146.05358123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00543975830078125, 'epoch': 0.68} 68%|██████▊ | 1710/2500 [10:13:59<4:43:50, 21.56s/it] 68%|██████▊ | 1711/2500 [10:14:20<4:41:15, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.47213408092298237, 'learning_rate': 3.156e-07, 'completion_length': 156.27679443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.00568389892578125, 'epoch': 0.68} 68%|██████▊ | 1711/2500 [10:14:20<4:41:15, 21.39s/it] 68%|██████▊ | 1712/2500 [10:14:41<4:40:10, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.02611375872013083, 'learning_rate': 3.1519999999999996e-07, 'completion_length': 158.0178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047760009765625, 'epoch': 0.68} 68%|██████▊ | 1712/2500 [10:14:41<4:40:10, 21.33s/it] 69%|██████▊ | 1713/2500 [10:15:02<4:38:41, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.22100717020555777, 'learning_rate': 3.148e-07, 'completion_length': 163.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0067138671875, 'epoch': 0.69} 69%|██████▊ | 1713/2500 [10:15:02<4:38:41, 21.25s/it] 69%|██████▊ | 1714/2500 [10:15:24<4:38:48, 21.28s/it] {'loss': 0.0004, 'grad_norm': 1.0806405450178105, 'learning_rate': 3.144e-07, 'completion_length': 169.7232208251953, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.11272924393415451, 'kl': 0.0105133056640625, 'epoch': 0.69} 69%|██████▊ | 1714/2500 [10:15:24<4:38:48, 21.28s/it] 69%|██████▊ | 1715/2500 [10:15:45<4:38:26, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.32319381071815784, 'learning_rate': 3.14e-07, 'completion_length': 164.35714721679688, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.857142984867096, 'reward_std': 0.05050762742757797, 'kl': 0.0068206787109375, 'epoch': 0.69} 69%|██████▊ | 1715/2500 [10:15:45<4:38:26, 21.28s/it] 69%|██████▊ | 1716/2500 [10:16:06<4:35:30, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.28255213521894135, 'learning_rate': 3.1359999999999995e-07, 'completion_length': 141.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00516510009765625, 'epoch': 0.69} 69%|██████▊ | 1716/2500 [10:16:06<4:35:30, 21.08s/it] 69%|██████▊ | 1717/2500 [10:16:26<4:34:28, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.49865925661790783, 'learning_rate': 3.1319999999999997e-07, 'completion_length': 143.62500762939453, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00567626953125, 'epoch': 0.69} 69%|██████▊ | 1717/2500 [10:16:26<4:34:28, 21.03s/it] 69%|██████▊ | 1718/2500 [10:16:47<4:33:57, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.2574689682237039, 'learning_rate': 3.128e-07, 'completion_length': 144.2232208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064544677734375, 'epoch': 0.69} 69%|██████▊ | 1718/2500 [10:16:47<4:33:57, 21.02s/it] 69%|██████▉ | 1719/2500 [10:17:08<4:33:17, 20.99s/it] {'loss': 0.0001, 'grad_norm': 0.22468286437668486, 'learning_rate': 3.124e-07, 'completion_length': 149.06250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00360870361328125, 'epoch': 0.69} 69%|██████▉ | 1719/2500 [10:17:08<4:33:17, 20.99s/it] 69%|██████▉ | 1720/2500 [10:17:30<4:35:24, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.9977530009820855, 'learning_rate': 3.12e-07, 'completion_length': 167.9107208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.096499003469944, 'kl': 0.006256103515625, 'epoch': 0.69} 69%|██████▉ | 1720/2500 [10:17:30<4:35:24, 21.19s/it] 69%|██████▉ | 1721/2500 [10:17:51<4:35:13, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.5869576674046714, 'learning_rate': 3.1159999999999996e-07, 'completion_length': 158.52679443359375, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.0964989997446537, 'kl': 0.0072021484375, 'epoch': 0.69} 69%|██████▉ | 1721/2500 [10:17:51<4:35:13, 21.20s/it] 69%|██████▉ | 1722/2500 [10:18:13<4:36:20, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.03557468189003328, 'learning_rate': 3.112e-07, 'completion_length': 164.55358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053863525390625, 'epoch': 0.69} 69%|██████▉ | 1722/2500 [10:18:13<4:36:20, 21.31s/it] 69%|██████▉ | 1723/2500 [10:18:34<4:37:00, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.02960397720733195, 'learning_rate': 3.108e-07, 'completion_length': 165.11607360839844, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0081329345703125, 'epoch': 0.69} 69%|██████▉ | 1723/2500 [10:18:34<4:37:00, 21.39s/it] 69%|██████▉ | 1724/2500 [10:18:56<4:38:18, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.027765750211481373, 'learning_rate': 3.104e-07, 'completion_length': 148.1964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00677490234375, 'epoch': 0.69} 69%|██████▉ | 1724/2500 [10:18:56<4:38:18, 21.52s/it] 69%|██████▉ | 1725/2500 [10:19:18<4:37:59, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.27498787837488436, 'learning_rate': 3.1e-07, 'completion_length': 158.43750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.007781982421875, 'epoch': 0.69} 69%|██████▉ | 1725/2500 [10:19:18<4:37:59, 21.52s/it] 69%|██████▉ | 1726/2500 [10:19:39<4:36:10, 21.41s/it] {'loss': 0.0003, 'grad_norm': 0.785079575489409, 'learning_rate': 3.0959999999999997e-07, 'completion_length': 155.55357360839844, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0064239501953125, 'epoch': 0.69} 69%|██████▉ | 1726/2500 [10:19:39<4:36:10, 21.41s/it] 69%|██████▉ | 1727/2500 [10:20:00<4:36:11, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.7430057424969868, 'learning_rate': 3.0919999999999994e-07, 'completion_length': 174.24108123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0062103271484375, 'epoch': 0.69} 69%|██████▉ | 1727/2500 [10:20:00<4:36:11, 21.44s/it] 69%|██████▉ | 1728/2500 [10:20:21<4:33:26, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.03172305618168314, 'learning_rate': 3.088e-07, 'completion_length': 150.0089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0057525634765625, 'epoch': 0.69} 69%|██████▉ | 1728/2500 [10:20:21<4:33:26, 21.25s/it] 69%|██████▉ | 1729/2500 [10:20:43<4:33:56, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.2723090665571047, 'learning_rate': 3.084e-07, 'completion_length': 173.81250762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.007080078125, 'epoch': 0.69} 69%|██████▉ | 1729/2500 [10:20:43<4:33:56, 21.32s/it] 69%|██████▉ | 1730/2500 [10:21:04<4:31:45, 21.18s/it] {'loss': 0.0002, 'grad_norm': 2.0455297482892743, 'learning_rate': 3.08e-07, 'completion_length': 150.25000762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.06343399360775948, 'kl': 0.0042572021484375, 'epoch': 0.69} 69%|██████▉ | 1730/2500 [10:21:04<4:31:45, 21.18s/it] 69%|██████▉ | 1731/2500 [10:21:24<4:29:26, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.20102612235823908, 'learning_rate': 3.076e-07, 'completion_length': 143.31250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0041351318359375, 'epoch': 0.69} 69%|██████▉ | 1731/2500 [10:21:24<4:29:26, 21.02s/it] 69%|██████▉ | 1732/2500 [10:21:46<4:32:35, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.18268450171321812, 'learning_rate': 3.0719999999999995e-07, 'completion_length': 151.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00478363037109375, 'epoch': 0.69} 69%|██████▉ | 1732/2500 [10:21:46<4:32:35, 21.30s/it] 69%|██████▉ | 1733/2500 [10:22:07<4:32:04, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.5953357085145127, 'learning_rate': 3.068e-07, 'completion_length': 161.77678680419922, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006317138671875, 'epoch': 0.69} 69%|██████▉ | 1733/2500 [10:22:07<4:32:04, 21.28s/it] 69%|██████▉ | 1734/2500 [10:22:29<4:31:17, 21.25s/it] {'loss': 0.0004, 'grad_norm': 0.4394025242938604, 'learning_rate': 3.064e-07, 'completion_length': 158.61607360839844, 'rewards/accuracy_reward': 0.7589286267757416, 'rewards/format_reward': 1.0, 'reward': 1.758928656578064, 'reward_std': 0.08747543394565582, 'kl': 0.0094757080078125, 'epoch': 0.69} 69%|██████▉ | 1734/2500 [10:22:29<4:31:17, 21.25s/it] 69%|██████▉ | 1735/2500 [10:22:49<4:29:04, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.8571031416661367, 'learning_rate': 3.0599999999999996e-07, 'completion_length': 149.4107208251953, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0056304931640625, 'epoch': 0.69} 69%|██████▉ | 1735/2500 [10:22:49<4:29:04, 21.10s/it] 69%|██████▉ | 1736/2500 [10:23:10<4:27:42, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.5622223401261836, 'learning_rate': 3.056e-07, 'completion_length': 152.51786041259766, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.05831881985068321, 'kl': 0.0077667236328125, 'epoch': 0.69} 69%|██████▉ | 1736/2500 [10:23:10<4:27:42, 21.02s/it] 69%|██████▉ | 1737/2500 [10:23:32<4:28:39, 21.13s/it] {'loss': 0.0003, 'grad_norm': 0.9077774529286156, 'learning_rate': 3.052e-07, 'completion_length': 164.11608123779297, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941557407379, 'kl': 0.006317138671875, 'epoch': 0.69} 69%|██████▉ | 1737/2500 [10:23:32<4:28:39, 21.13s/it] 70%|██████▉ | 1738/2500 [10:23:52<4:25:00, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.03551711880305326, 'learning_rate': 3.048e-07, 'completion_length': 142.68750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00389862060546875, 'epoch': 0.7} 70%|██████▉ | 1738/2500 [10:23:52<4:25:00, 20.87s/it] 70%|██████▉ | 1739/2500 [10:24:13<4:24:50, 20.88s/it] {'loss': 0.0003, 'grad_norm': 0.38526366529224443, 'learning_rate': 3.044e-07, 'completion_length': 158.1071548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0071563720703125, 'epoch': 0.7} 70%|██████▉ | 1739/2500 [10:24:13<4:24:50, 20.88s/it] 70%|██████▉ | 1740/2500 [10:24:34<4:26:45, 21.06s/it] {'loss': 0.0003, 'grad_norm': 0.4162601273039157, 'learning_rate': 3.0399999999999997e-07, 'completion_length': 151.56250762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.06222161650657654, 'kl': 0.00662994384765625, 'epoch': 0.7} 70%|██████▉ | 1740/2500 [10:24:34<4:26:45, 21.06s/it] 70%|██████▉ | 1741/2500 [10:24:55<4:25:59, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.19767297263774292, 'learning_rate': 3.036e-07, 'completion_length': 159.77679443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0066680908203125, 'epoch': 0.7} 70%|██████▉ | 1741/2500 [10:24:55<4:25:59, 21.03s/it] 70%|██████▉ | 1742/2500 [10:25:17<4:28:05, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.01998580267049362, 'learning_rate': 3.032e-07, 'completion_length': 162.8839340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.005889892578125, 'epoch': 0.7} 70%|██████▉ | 1742/2500 [10:25:17<4:28:05, 21.22s/it] 70%|██████▉ | 1743/2500 [10:25:38<4:27:56, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.24137058210294732, 'learning_rate': 3.028e-07, 'completion_length': 159.4196548461914, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0066375732421875, 'epoch': 0.7} 70%|██████▉ | 1743/2500 [10:25:38<4:27:56, 21.24s/it] 70%|██████▉ | 1744/2500 [10:26:00<4:28:34, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.48157835585628006, 'learning_rate': 3.024e-07, 'completion_length': 154.95536041259766, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0083160400390625, 'epoch': 0.7} 70%|██████▉ | 1744/2500 [10:26:00<4:28:34, 21.32s/it] 70%|██████▉ | 1745/2500 [10:26:21<4:29:29, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.02634698216943357, 'learning_rate': 3.02e-07, 'completion_length': 162.7321548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.7} 70%|██████▉ | 1745/2500 [10:26:21<4:29:29, 21.42s/it] 70%|██████▉ | 1746/2500 [10:26:42<4:28:04, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.4614013373756607, 'learning_rate': 3.0159999999999995e-07, 'completion_length': 149.25000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00472259521484375, 'epoch': 0.7} 70%|██████▉ | 1746/2500 [10:26:42<4:28:04, 21.33s/it] 70%|██████▉ | 1747/2500 [10:27:04<4:27:29, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.369495085137279, 'learning_rate': 3.012e-07, 'completion_length': 156.33036041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005859375, 'epoch': 0.7} 70%|██████▉ | 1747/2500 [10:27:04<4:27:29, 21.31s/it] 70%|██████▉ | 1748/2500 [10:27:24<4:23:51, 21.05s/it] {'loss': 0.0001, 'grad_norm': 0.014252188220326189, 'learning_rate': 3.008e-07, 'completion_length': 134.46429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0024871826171875, 'epoch': 0.7} 70%|██████▉ | 1748/2500 [10:27:24<4:23:51, 21.05s/it] 70%|██████▉ | 1749/2500 [10:27:44<4:19:50, 20.76s/it] {'loss': 0.0001, 'grad_norm': 0.2122696368622582, 'learning_rate': 3.0039999999999996e-07, 'completion_length': 134.37500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00321197509765625, 'epoch': 0.7} 70%|██████▉ | 1749/2500 [10:27:44<4:19:50, 20.76s/it] 70%|███████ | 1750/2500 [10:28:05<4:20:28, 20.84s/it] {'loss': 0.0002, 'grad_norm': 0.0189831674612148, 'learning_rate': 3e-07, 'completion_length': 138.15179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054168701171875, 'epoch': 0.7} 70%|███████ | 1750/2500 [10:28:05<4:20:28, 20.84s/it] 70%|███████ | 1751/2500 [10:28:27<4:23:57, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.02281466322768535, 'learning_rate': 2.9959999999999996e-07, 'completion_length': 148.3571548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00458526611328125, 'epoch': 0.7} 70%|███████ | 1751/2500 [10:28:27<4:23:57, 21.15s/it] 70%|███████ | 1752/2500 [10:28:48<4:24:29, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.027372555649719958, 'learning_rate': 2.9920000000000003e-07, 'completion_length': 148.65179443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0077056884765625, 'epoch': 0.7} 70%|███████ | 1752/2500 [10:28:48<4:24:29, 21.22s/it] 70%|███████ | 1753/2500 [10:29:10<4:25:20, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.04243372703175915, 'learning_rate': 2.988e-07, 'completion_length': 157.19644165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00438690185546875, 'epoch': 0.7} 70%|███████ | 1753/2500 [10:29:10<4:25:20, 21.31s/it] 70%|███████ | 1754/2500 [10:29:31<4:24:26, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.03245772582129142, 'learning_rate': 2.9839999999999997e-07, 'completion_length': 159.2232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006317138671875, 'epoch': 0.7} 70%|███████ | 1754/2500 [10:29:31<4:24:26, 21.27s/it] 70%|███████ | 1755/2500 [10:29:52<4:23:42, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.6189233795349338, 'learning_rate': 2.98e-07, 'completion_length': 150.99108123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.10431019216775894, 'kl': 0.00762939453125, 'epoch': 0.7} 70%|███████ | 1755/2500 [10:29:52<4:23:42, 21.24s/it] 70%|███████ | 1756/2500 [10:30:14<4:24:40, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.043412101706166654, 'learning_rate': 2.9759999999999996e-07, 'completion_length': 161.0357208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047607421875, 'epoch': 0.7} 70%|███████ | 1756/2500 [10:30:14<4:24:40, 21.34s/it] 70%|███████ | 1757/2500 [10:30:36<4:25:36, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.03919888541444859, 'learning_rate': 2.972e-07, 'completion_length': 170.00000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0065460205078125, 'epoch': 0.7} 70%|███████ | 1757/2500 [10:30:36<4:25:36, 21.45s/it] 70%|███████ | 1758/2500 [10:30:57<4:23:16, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.01903202044835848, 'learning_rate': 2.968e-07, 'completion_length': 136.36608123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00439453125, 'epoch': 0.7} 70%|███████ | 1758/2500 [10:30:57<4:23:16, 21.29s/it] 70%|███████ | 1759/2500 [10:31:18<4:24:03, 21.38s/it] {'loss': 0.0003, 'grad_norm': 0.6467409875907637, 'learning_rate': 2.964e-07, 'completion_length': 164.1071548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780732154846, 'kl': 0.0077056884765625, 'epoch': 0.7} 70%|███████ | 1759/2500 [10:31:18<4:24:03, 21.38s/it] 70%|███████ | 1760/2500 [10:31:39<4:21:09, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.028793092389146725, 'learning_rate': 2.9599999999999995e-07, 'completion_length': 149.86608123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00467681884765625, 'epoch': 0.7} 70%|███████ | 1760/2500 [10:31:39<4:21:09, 21.17s/it] 70%|███████ | 1761/2500 [10:32:00<4:20:34, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.023071531935638524, 'learning_rate': 2.9559999999999997e-07, 'completion_length': 159.37500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004974365234375, 'epoch': 0.7} 70%|███████ | 1761/2500 [10:32:00<4:20:34, 21.16s/it] 70%|███████ | 1762/2500 [10:32:21<4:20:21, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.4271543978296955, 'learning_rate': 2.952e-07, 'completion_length': 153.43750762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00669097900390625, 'epoch': 0.7} 70%|███████ | 1762/2500 [10:32:21<4:20:21, 21.17s/it] 71%|███████ | 1763/2500 [10:32:43<4:21:13, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.027339252279921736, 'learning_rate': 2.948e-07, 'completion_length': 157.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006072998046875, 'epoch': 0.71} 71%|███████ | 1763/2500 [10:32:43<4:21:13, 21.27s/it] 71%|███████ | 1764/2500 [10:33:04<4:22:32, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.5504728992718091, 'learning_rate': 2.944e-07, 'completion_length': 158.35714721679688, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.00592041015625, 'epoch': 0.71} 71%|███████ | 1764/2500 [10:33:04<4:22:32, 21.40s/it] 71%|███████ | 1765/2500 [10:33:26<4:21:32, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.017048197251770075, 'learning_rate': 2.9399999999999996e-07, 'completion_length': 142.3214340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0046539306640625, 'epoch': 0.71} 71%|███████ | 1765/2500 [10:33:26<4:21:32, 21.35s/it] 71%|███████ | 1766/2500 [10:33:47<4:20:13, 21.27s/it] {'loss': 0.0002, 'grad_norm': 1.3227426003287308, 'learning_rate': 2.9360000000000003e-07, 'completion_length': 151.50000762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0050811767578125, 'epoch': 0.71} 71%|███████ | 1766/2500 [10:33:47<4:20:13, 21.27s/it] 71%|███████ | 1767/2500 [10:34:08<4:19:17, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.334905005208822, 'learning_rate': 2.932e-07, 'completion_length': 155.98214721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0069122314453125, 'epoch': 0.71} 71%|███████ | 1767/2500 [10:34:08<4:19:17, 21.22s/it] 71%|███████ | 1768/2500 [10:34:30<4:24:01, 21.64s/it] {'loss': 0.0003, 'grad_norm': 0.0271487276793813, 'learning_rate': 2.928e-07, 'completion_length': 165.3839340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0067138671875, 'epoch': 0.71} 71%|███████ | 1768/2500 [10:34:30<4:24:01, 21.64s/it] 71%|███████ | 1769/2500 [10:34:52<4:21:50, 21.49s/it] {'loss': 0.0003, 'grad_norm': 1.7155439473882832, 'learning_rate': 2.924e-07, 'completion_length': 162.6607208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0072174072265625, 'epoch': 0.71} 71%|███████ | 1769/2500 [10:34:52<4:21:50, 21.49s/it] 71%|███████ | 1770/2500 [10:35:12<4:18:36, 21.26s/it] {'loss': 0.0001, 'grad_norm': 0.3301119540862098, 'learning_rate': 2.9199999999999997e-07, 'completion_length': 140.52679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00365447998046875, 'epoch': 0.71} 71%|███████ | 1770/2500 [10:35:12<4:18:36, 21.26s/it] 71%|███████ | 1771/2500 [10:35:32<4:14:32, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.5172551069257778, 'learning_rate': 2.916e-07, 'completion_length': 140.1607208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.00601959228515625, 'epoch': 0.71} 71%|███████ | 1771/2500 [10:35:32<4:14:32, 20.95s/it] 71%|███████ | 1772/2500 [10:35:54<4:14:42, 20.99s/it] {'loss': 0.0003, 'grad_norm': 0.2664785031129183, 'learning_rate': 2.912e-07, 'completion_length': 156.68750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006256103515625, 'epoch': 0.71} 71%|███████ | 1772/2500 [10:35:54<4:14:42, 20.99s/it] 71%|███████ | 1773/2500 [10:36:15<4:17:14, 21.23s/it] {'loss': 0.0003, 'grad_norm': 0.7741982183652036, 'learning_rate': 2.908e-07, 'completion_length': 156.33928680419922, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750001192092896, 'reward_std': 0.11663764342665672, 'kl': 0.006744384765625, 'epoch': 0.71} 71%|███████ | 1773/2500 [10:36:15<4:17:14, 21.23s/it] 71%|███████ | 1774/2500 [10:36:37<4:16:44, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.029731521273761097, 'learning_rate': 2.9039999999999995e-07, 'completion_length': 161.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.007232666015625, 'epoch': 0.71} 71%|███████ | 1774/2500 [10:36:37<4:16:44, 21.22s/it] 71%|███████ | 1775/2500 [10:36:58<4:17:28, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.022365108962928884, 'learning_rate': 2.9e-07, 'completion_length': 148.66964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005462646484375, 'epoch': 0.71} 71%|███████ | 1775/2500 [10:36:58<4:17:28, 21.31s/it] 71%|███████ | 1776/2500 [10:37:20<4:18:13, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.03962133963501358, 'learning_rate': 2.896e-07, 'completion_length': 149.16964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006011962890625, 'epoch': 0.71} 71%|███████ | 1776/2500 [10:37:20<4:18:13, 21.40s/it] 71%|███████ | 1777/2500 [10:37:41<4:18:19, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.19626466821294883, 'learning_rate': 2.892e-07, 'completion_length': 151.56250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00408935546875, 'epoch': 0.71} 71%|███████ | 1777/2500 [10:37:41<4:18:19, 21.44s/it] 71%|███████ | 1778/2500 [10:38:02<4:16:52, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.78741873072716, 'learning_rate': 2.888e-07, 'completion_length': 151.49107360839844, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.05050762742757797, 'kl': 0.006683349609375, 'epoch': 0.71} 71%|███████ | 1778/2500 [10:38:02<4:16:52, 21.35s/it] 71%|███████ | 1779/2500 [10:38:24<4:17:03, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.03820063104406845, 'learning_rate': 2.8839999999999996e-07, 'completion_length': 158.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005523681640625, 'epoch': 0.71} 71%|███████ | 1779/2500 [10:38:24<4:17:03, 21.39s/it] 71%|███████ | 1780/2500 [10:38:46<4:18:47, 21.57s/it] {'loss': 0.0002, 'grad_norm': 0.27643586736494913, 'learning_rate': 2.88e-07, 'completion_length': 147.65178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048675537109375, 'epoch': 0.71} 71%|███████ | 1780/2500 [10:38:46<4:18:47, 21.57s/it] 71%|███████ | 1781/2500 [10:39:07<4:18:11, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.560515606456157, 'learning_rate': 2.876e-07, 'completion_length': 161.1607208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.007476806640625, 'epoch': 0.71} 71%|███████ | 1781/2500 [10:39:07<4:18:11, 21.55s/it] 71%|███████▏ | 1782/2500 [10:39:29<4:17:18, 21.50s/it] {'loss': 0.0003, 'grad_norm': 0.8745571307477326, 'learning_rate': 2.872e-07, 'completion_length': 163.42857360839844, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.10101525112986565, 'kl': 0.0070648193359375, 'epoch': 0.71} 71%|███████▏ | 1782/2500 [10:39:29<4:17:18, 21.50s/it] 71%|███████▏ | 1783/2500 [10:39:50<4:16:15, 21.44s/it] {'loss': 0.0002, 'grad_norm': 1.6686167477098537, 'learning_rate': 2.868e-07, 'completion_length': 148.5714340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.0055694580078125, 'epoch': 0.71} 71%|███████▏ | 1783/2500 [10:39:50<4:16:15, 21.44s/it] 71%|███████▏ | 1784/2500 [10:40:13<4:20:09, 21.80s/it] {'loss': 0.0003, 'grad_norm': 0.463591911475302, 'learning_rate': 2.8639999999999997e-07, 'completion_length': 165.1339340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0064849853515625, 'epoch': 0.71} 71%|███████▏ | 1784/2500 [10:40:13<4:20:09, 21.80s/it] 71%|███████▏ | 1785/2500 [10:40:33<4:16:05, 21.49s/it] {'loss': 0.0002, 'grad_norm': 0.2865017881453717, 'learning_rate': 2.8599999999999994e-07, 'completion_length': 147.71429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0040130615234375, 'epoch': 0.71} 71%|███████▏ | 1785/2500 [10:40:33<4:16:05, 21.49s/it] 71%|███████▏ | 1786/2500 [10:40:54<4:14:15, 21.37s/it] {'loss': 0.0002, 'grad_norm': 0.03667541400909006, 'learning_rate': 2.856e-07, 'completion_length': 146.50000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054168701171875, 'epoch': 0.71} 71%|███████▏ | 1786/2500 [10:40:54<4:14:15, 21.37s/it] 71%|███████▏ | 1787/2500 [10:41:16<4:14:18, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.3365697723925446, 'learning_rate': 2.852e-07, 'completion_length': 155.54464721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.0072021484375, 'epoch': 0.71} 71%|███████▏ | 1787/2500 [10:41:16<4:14:18, 21.40s/it] 72%|███████▏ | 1788/2500 [10:41:37<4:14:20, 21.43s/it] {'loss': 0.0001, 'grad_norm': 0.015961644914427735, 'learning_rate': 2.848e-07, 'completion_length': 149.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00307464599609375, 'epoch': 0.72} 72%|███████▏ | 1788/2500 [10:41:37<4:14:20, 21.43s/it] 72%|███████▏ | 1789/2500 [10:41:58<4:11:27, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.5029389077093815, 'learning_rate': 2.844e-07, 'completion_length': 155.8214340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0048675537109375, 'epoch': 0.72} 72%|███████▏ | 1789/2500 [10:41:58<4:11:27, 21.22s/it] 72%|███████▏ | 1790/2500 [10:42:19<4:10:02, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.029555333762367023, 'learning_rate': 2.8399999999999995e-07, 'completion_length': 148.9464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0044097900390625, 'epoch': 0.72} 72%|███████▏ | 1790/2500 [10:42:19<4:10:02, 21.13s/it] 72%|███████▏ | 1791/2500 [10:42:40<4:10:14, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.026878061593122532, 'learning_rate': 2.836e-07, 'completion_length': 140.25000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0059356689453125, 'epoch': 0.72} 72%|███████▏ | 1791/2500 [10:42:40<4:10:14, 21.18s/it] 72%|███████▏ | 1792/2500 [10:43:02<4:12:02, 21.36s/it] {'loss': 0.0002, 'grad_norm': 0.02039150912679525, 'learning_rate': 2.832e-07, 'completion_length': 137.8571548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00445556640625, 'epoch': 0.72} 72%|███████▏ | 1792/2500 [10:43:02<4:12:02, 21.36s/it] 72%|███████▏ | 1793/2500 [10:43:23<4:10:41, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.03837396827022228, 'learning_rate': 2.8279999999999996e-07, 'completion_length': 152.12500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006103515625, 'epoch': 0.72} 72%|███████▏ | 1793/2500 [10:43:23<4:10:41, 21.28s/it] 72%|███████▏ | 1794/2500 [10:43:44<4:08:57, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.20477233583356771, 'learning_rate': 2.824e-07, 'completion_length': 145.3303680419922, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0056915283203125, 'epoch': 0.72} 72%|███████▏ | 1794/2500 [10:43:44<4:08:57, 21.16s/it] 72%|███████▏ | 1795/2500 [10:44:05<4:09:20, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.27917938728739317, 'learning_rate': 2.8199999999999996e-07, 'completion_length': 156.17858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00701904296875, 'epoch': 0.72} 72%|███████▏ | 1795/2500 [10:44:06<4:09:20, 21.22s/it] 72%|███████▏ | 1796/2500 [10:44:26<4:08:06, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.016552706891397418, 'learning_rate': 2.816e-07, 'completion_length': 154.4464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053253173828125, 'epoch': 0.72} 72%|███████▏ | 1796/2500 [10:44:26<4:08:06, 21.15s/it] 72%|███████▏ | 1797/2500 [10:44:48<4:08:58, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.5112707141078844, 'learning_rate': 2.812e-07, 'completion_length': 154.27679443359375, 'rewards/accuracy_reward': 0.9375000596046448, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831881985068321, 'kl': 0.0045166015625, 'epoch': 0.72} 72%|███████▏ | 1797/2500 [10:44:48<4:08:58, 21.25s/it] 72%|███████▏ | 1798/2500 [10:45:09<4:08:30, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.19019566119239237, 'learning_rate': 2.8079999999999997e-07, 'completion_length': 152.83036041259766, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.006103515625, 'epoch': 0.72} 72%|███████▏ | 1798/2500 [10:45:09<4:08:30, 21.24s/it] 72%|███████▏ | 1799/2500 [10:45:30<4:05:28, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.22867474804124524, 'learning_rate': 2.804e-07, 'completion_length': 150.81250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0051116943359375, 'epoch': 0.72} 72%|███████▏ | 1799/2500 [10:45:30<4:05:28, 21.01s/it] 72%|███████▏ | 1800/2500 [10:45:51<4:04:58, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.2371101166672375, 'learning_rate': 2.8e-07, 'completion_length': 149.3839340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0064849853515625, 'epoch': 0.72} 72%|███████▏ | 1800/2500 [10:45:51<4:04:58, 21.00s/it] 72%|███████▏ | 1801/2500 [10:47:02<6:59:47, 36.03s/it] {'loss': 0.0002, 'grad_norm': 0.024667934129115342, 'learning_rate': 2.796e-07, 'completion_length': 157.05357360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004180908203125, 'epoch': 0.72} 72%|███████▏ | 1801/2500 [10:47:02<6:59:47, 36.03s/it] 72%|███████▏ | 1802/2500 [10:47:22<6:05:14, 31.40s/it] {'loss': 0.0002, 'grad_norm': 0.5181149513374621, 'learning_rate': 2.792e-07, 'completion_length': 148.7589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004791259765625, 'epoch': 0.72} 72%|███████▏ | 1802/2500 [10:47:22<6:05:14, 31.40s/it] 72%|███████▏ | 1803/2500 [10:47:44<5:29:24, 28.36s/it] {'loss': 0.0002, 'grad_norm': 0.02541268802175369, 'learning_rate': 2.788e-07, 'completion_length': 155.45536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056915283203125, 'epoch': 0.72} 72%|███████▏ | 1803/2500 [10:47:44<5:29:24, 28.36s/it] 72%|███████▏ | 1804/2500 [10:48:04<5:02:48, 26.10s/it] {'loss': 0.0002, 'grad_norm': 0.4794316563296564, 'learning_rate': 2.7839999999999995e-07, 'completion_length': 144.12500762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0040435791015625, 'epoch': 0.72} 72%|███████▏ | 1804/2500 [10:48:04<5:02:48, 26.10s/it] 72%|███████▏ | 1805/2500 [10:48:25<4:42:47, 24.41s/it] {'loss': 0.0002, 'grad_norm': 0.026717878989977316, 'learning_rate': 2.7800000000000003e-07, 'completion_length': 139.63393783569336, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006072998046875, 'epoch': 0.72} 72%|███████▏ | 1805/2500 [10:48:25<4:42:47, 24.41s/it] 72%|███████▏ | 1806/2500 [10:48:48<4:36:20, 23.89s/it] {'loss': 0.0003, 'grad_norm': 0.017801485860083226, 'learning_rate': 2.776e-07, 'completion_length': 157.2053680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.006500244140625, 'epoch': 0.72} 72%|███████▏ | 1806/2500 [10:48:48<4:36:20, 23.89s/it] 72%|███████▏ | 1807/2500 [10:49:08<4:25:07, 22.96s/it] {'loss': 0.0002, 'grad_norm': 0.28170237847393176, 'learning_rate': 2.7719999999999997e-07, 'completion_length': 147.05357360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00537109375, 'epoch': 0.72} 72%|███████▏ | 1807/2500 [10:49:08<4:25:07, 22.96s/it] 72%|███████▏ | 1808/2500 [10:49:29<4:16:53, 22.27s/it] {'loss': 0.0003, 'grad_norm': 0.2951115099385935, 'learning_rate': 2.768e-07, 'completion_length': 155.68750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0062713623046875, 'epoch': 0.72} 72%|███████▏ | 1808/2500 [10:49:29<4:16:53, 22.27s/it] 72%|███████▏ | 1809/2500 [10:49:50<4:10:49, 21.78s/it] {'loss': 0.0001, 'grad_norm': 0.3814043257221876, 'learning_rate': 2.7639999999999996e-07, 'completion_length': 149.26786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00348663330078125, 'epoch': 0.72} 72%|███████▏ | 1809/2500 [10:49:50<4:10:49, 21.78s/it] 72%|███████▏ | 1810/2500 [10:50:11<4:08:25, 21.60s/it] {'loss': 0.0003, 'grad_norm': 0.38375527427471884, 'learning_rate': 2.7600000000000004e-07, 'completion_length': 161.42858123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0085601806640625, 'epoch': 0.72} 72%|███████▏ | 1810/2500 [10:50:11<4:08:25, 21.60s/it] 72%|███████▏ | 1811/2500 [10:50:31<4:04:32, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.022249386848464864, 'learning_rate': 2.756e-07, 'completion_length': 148.91964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005096435546875, 'epoch': 0.72} 72%|███████▏ | 1811/2500 [10:50:31<4:04:32, 21.30s/it] 72%|███████▏ | 1812/2500 [10:50:52<4:03:08, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.020584354697487157, 'learning_rate': 2.752e-07, 'completion_length': 162.25000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006103515625, 'epoch': 0.72} 72%|███████▏ | 1812/2500 [10:50:52<4:03:08, 21.20s/it] 73%|███████▎ | 1813/2500 [10:51:13<4:00:55, 21.04s/it] {'loss': 0.0001, 'grad_norm': 0.5639750721218268, 'learning_rate': 2.748e-07, 'completion_length': 147.3482208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00372314453125, 'epoch': 0.73} 73%|███████▎ | 1813/2500 [10:51:13<4:00:55, 21.04s/it] 73%|███████▎ | 1814/2500 [10:51:34<4:00:42, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.40267360790906054, 'learning_rate': 2.7439999999999997e-07, 'completion_length': 154.3839340209961, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0049285888671875, 'epoch': 0.73} 73%|███████▎ | 1814/2500 [10:51:34<4:00:42, 21.05s/it] 73%|███████▎ | 1815/2500 [10:51:55<3:58:17, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.3392639941100165, 'learning_rate': 2.74e-07, 'completion_length': 144.41964721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0057830810546875, 'epoch': 0.73} 73%|███████▎ | 1815/2500 [10:51:55<3:58:17, 20.87s/it] 73%|███████▎ | 1816/2500 [10:52:15<3:57:23, 20.82s/it] {'loss': 0.0002, 'grad_norm': 0.07678156864355949, 'learning_rate': 2.736e-07, 'completion_length': 164.74108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054931640625, 'epoch': 0.73} 73%|███████▎ | 1816/2500 [10:52:15<3:57:23, 20.82s/it] 73%|███████▎ | 1817/2500 [10:52:36<3:57:33, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.016947194234987965, 'learning_rate': 2.732e-07, 'completion_length': 144.55358123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00402069091796875, 'epoch': 0.73} 73%|███████▎ | 1817/2500 [10:52:36<3:57:33, 20.87s/it] 73%|███████▎ | 1818/2500 [10:52:58<3:59:32, 21.07s/it] {'loss': 0.0005, 'grad_norm': 0.9234439926845706, 'learning_rate': 2.7279999999999995e-07, 'completion_length': 173.79464721679688, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.128351628780365, 'kl': 0.01287841796875, 'epoch': 0.73} 73%|███████▎ | 1818/2500 [10:52:58<3:59:32, 21.07s/it] 73%|███████▎ | 1819/2500 [10:53:19<3:59:04, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.3298710957440674, 'learning_rate': 2.724e-07, 'completion_length': 149.14286041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.005706787109375, 'epoch': 0.73} 73%|███████▎ | 1819/2500 [10:53:19<3:59:04, 21.06s/it] 73%|███████▎ | 1820/2500 [10:53:39<3:55:25, 20.77s/it] {'loss': 0.0002, 'grad_norm': 0.15362872463460558, 'learning_rate': 2.72e-07, 'completion_length': 141.3482208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0045013427734375, 'epoch': 0.73} 73%|███████▎ | 1820/2500 [10:53:39<3:55:25, 20.77s/it] 73%|███████▎ | 1821/2500 [10:54:00<3:55:54, 20.85s/it] {'loss': 0.0003, 'grad_norm': 0.49990524428954064, 'learning_rate': 2.7159999999999997e-07, 'completion_length': 147.5625, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0072174072265625, 'epoch': 0.73} 73%|███████▎ | 1821/2500 [10:54:00<3:55:54, 20.85s/it] 73%|███████▎ | 1822/2500 [10:54:22<3:58:34, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.027538025961747217, 'learning_rate': 2.712e-07, 'completion_length': 169.79464721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0073699951171875, 'epoch': 0.73} 73%|███████▎ | 1822/2500 [10:54:22<3:58:34, 21.11s/it] 73%|███████▎ | 1823/2500 [10:54:43<3:59:21, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.027670860649779052, 'learning_rate': 2.7079999999999996e-07, 'completion_length': 154.0446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058746337890625, 'epoch': 0.73} 73%|███████▎ | 1823/2500 [10:54:43<3:59:21, 21.21s/it] 73%|███████▎ | 1824/2500 [10:55:04<3:57:16, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.022097230271821658, 'learning_rate': 2.704e-07, 'completion_length': 148.49108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056304931640625, 'epoch': 0.73} 73%|███████▎ | 1824/2500 [10:55:04<3:57:16, 21.06s/it] 73%|███████▎ | 1825/2500 [10:55:25<3:57:13, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.23363303569022192, 'learning_rate': 2.7e-07, 'completion_length': 150.93750762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.00423431396484375, 'epoch': 0.73} 73%|███████▎ | 1825/2500 [10:55:25<3:57:13, 21.09s/it] 73%|███████▎ | 1826/2500 [10:55:46<3:57:04, 21.10s/it] {'loss': 0.0001, 'grad_norm': 2.343577477191135, 'learning_rate': 2.696e-07, 'completion_length': 153.2589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.003173828125, 'epoch': 0.73} 73%|███████▎ | 1826/2500 [10:55:46<3:57:04, 21.10s/it] 73%|███████▎ | 1827/2500 [10:56:07<3:54:49, 20.93s/it] {'loss': 0.0001, 'grad_norm': 0.01612455647494035, 'learning_rate': 2.692e-07, 'completion_length': 142.44644165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.002960205078125, 'epoch': 0.73} 73%|███████▎ | 1827/2500 [10:56:07<3:54:49, 20.93s/it] 73%|███████▎ | 1828/2500 [10:56:28<3:55:18, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.3604670338233047, 'learning_rate': 2.6879999999999997e-07, 'completion_length': 144.4375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0057830810546875, 'epoch': 0.73} 73%|███████▎ | 1828/2500 [10:56:28<3:55:18, 21.01s/it] 73%|███████▎ | 1829/2500 [10:56:49<3:54:08, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.8211088847738724, 'learning_rate': 2.684e-07, 'completion_length': 141.5178680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.00440216064453125, 'epoch': 0.73} 73%|███████▎ | 1829/2500 [10:56:49<3:54:08, 20.94s/it] 73%|███████▎ | 1830/2500 [10:57:10<3:55:53, 21.12s/it] {'loss': 0.0004, 'grad_norm': 0.5227925588695935, 'learning_rate': 2.68e-07, 'completion_length': 158.2232208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.008880615234375, 'epoch': 0.73} 73%|███████▎ | 1830/2500 [10:57:10<3:55:53, 21.12s/it] 73%|███████▎ | 1831/2500 [10:57:32<3:56:11, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.3593362598470167, 'learning_rate': 2.676e-07, 'completion_length': 149.0089340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0060272216796875, 'epoch': 0.73} 73%|███████▎ | 1831/2500 [10:57:32<3:56:11, 21.18s/it] 73%|███████▎ | 1832/2500 [10:57:53<3:55:49, 21.18s/it] {'loss': 0.0001, 'grad_norm': 0.2174548800271743, 'learning_rate': 2.6719999999999996e-07, 'completion_length': 151.0089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00356292724609375, 'epoch': 0.73} 73%|███████▎ | 1832/2500 [10:57:53<3:55:49, 21.18s/it] 73%|███████▎ | 1833/2500 [10:58:15<3:57:40, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.01800572236042535, 'learning_rate': 2.668e-07, 'completion_length': 151.99108123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00458526611328125, 'epoch': 0.73} 73%|███████▎ | 1833/2500 [10:58:15<3:57:40, 21.38s/it] 73%|███████▎ | 1834/2500 [10:58:35<3:55:31, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.02317071249480396, 'learning_rate': 2.664e-07, 'completion_length': 146.17857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00555419921875, 'epoch': 0.73} 73%|███████▎ | 1834/2500 [10:58:35<3:55:31, 21.22s/it] 73%|███████▎ | 1835/2500 [10:58:56<3:52:05, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.02698648992856719, 'learning_rate': 2.66e-07, 'completion_length': 143.11608123779297, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0, 'kl': 0.005889892578125, 'epoch': 0.73} 73%|███████▎ | 1835/2500 [10:58:56<3:52:05, 20.94s/it] 73%|███████▎ | 1836/2500 [10:59:17<3:53:10, 21.07s/it] {'loss': 0.0003, 'grad_norm': 0.4740041096945309, 'learning_rate': 2.656e-07, 'completion_length': 163.25000762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.11663764342665672, 'kl': 0.0068206787109375, 'epoch': 0.73} 73%|███████▎ | 1836/2500 [10:59:17<3:53:10, 21.07s/it] 73%|███████▎ | 1837/2500 [10:59:38<3:51:01, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.2832762753943065, 'learning_rate': 2.6519999999999997e-07, 'completion_length': 147.68750762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0042572021484375, 'epoch': 0.73} 73%|███████▎ | 1837/2500 [10:59:38<3:51:01, 20.91s/it] 74%|███████▎ | 1838/2500 [10:59:59<3:53:04, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.6994595983690306, 'learning_rate': 2.648e-07, 'completion_length': 147.3839340209961, 'rewards/accuracy_reward': 0.8125000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8125000596046448, 'reward_std': 0.11332815885543823, 'kl': 0.007659912109375, 'epoch': 0.74} 74%|███████▎ | 1838/2500 [10:59:59<3:53:04, 21.12s/it] 74%|███████▎ | 1839/2500 [11:00:20<3:51:52, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.42133611978082575, 'learning_rate': 2.644e-07, 'completion_length': 153.92857360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0047454833984375, 'epoch': 0.74} 74%|███████▎ | 1839/2500 [11:00:20<3:51:52, 21.05s/it] 74%|███████▎ | 1840/2500 [11:00:41<3:52:17, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.6178725671545366, 'learning_rate': 2.64e-07, 'completion_length': 145.40179443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0048065185546875, 'epoch': 0.74} 74%|███████▎ | 1840/2500 [11:00:41<3:52:17, 21.12s/it] 74%|███████▎ | 1841/2500 [11:01:03<3:54:52, 21.38s/it] {'loss': 0.0003, 'grad_norm': 0.6191396504333098, 'learning_rate': 2.636e-07, 'completion_length': 169.77679443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0070343017578125, 'epoch': 0.74} 74%|███████▎ | 1841/2500 [11:01:03<3:54:52, 21.38s/it] 74%|███████▎ | 1842/2500 [11:01:24<3:53:21, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.4845093012552788, 'learning_rate': 2.632e-07, 'completion_length': 143.49107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00385284423828125, 'epoch': 0.74} 74%|███████▎ | 1842/2500 [11:01:24<3:53:21, 21.28s/it] 74%|███████▎ | 1843/2500 [11:01:46<3:52:52, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.4710338313942253, 'learning_rate': 2.6279999999999994e-07, 'completion_length': 149.7857208251953, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.06613001227378845, 'kl': 0.00738525390625, 'epoch': 0.74} 74%|███████▎ | 1843/2500 [11:01:46<3:52:52, 21.27s/it] 74%|███████▍ | 1844/2500 [11:02:07<3:51:26, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.015211343180017258, 'learning_rate': 2.624e-07, 'completion_length': 151.86607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005645751953125, 'epoch': 0.74} 74%|███████▍ | 1844/2500 [11:02:07<3:51:26, 21.17s/it] 74%|███████▍ | 1845/2500 [11:02:28<3:50:53, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.021051614716619222, 'learning_rate': 2.62e-07, 'completion_length': 145.7232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0042572021484375, 'epoch': 0.74} 74%|███████▍ | 1845/2500 [11:02:28<3:50:53, 21.15s/it] 74%|███████▍ | 1846/2500 [11:02:49<3:50:12, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.8121639891147178, 'learning_rate': 2.616e-07, 'completion_length': 160.52679443359375, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.0739355981349945, 'kl': 0.007537841796875, 'epoch': 0.74} 74%|███████▍ | 1846/2500 [11:02:49<3:50:12, 21.12s/it] 74%|███████▍ | 1847/2500 [11:03:10<3:50:38, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.605837599492668, 'learning_rate': 2.612e-07, 'completion_length': 148.1428680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0064849853515625, 'epoch': 0.74} 74%|███████▍ | 1847/2500 [11:03:10<3:50:38, 21.19s/it] 74%|███████▍ | 1848/2500 [11:03:31<3:48:14, 21.00s/it] {'loss': 0.0001, 'grad_norm': 0.22382133598040516, 'learning_rate': 2.6079999999999995e-07, 'completion_length': 131.97322463989258, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00286865234375, 'epoch': 0.74} 74%|███████▍ | 1848/2500 [11:03:31<3:48:14, 21.00s/it] 74%|███████▍ | 1849/2500 [11:03:52<3:49:48, 21.18s/it] {'loss': 0.0003, 'grad_norm': 0.6488527501407548, 'learning_rate': 2.6040000000000003e-07, 'completion_length': 154.31250762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00701904296875, 'epoch': 0.74} 74%|███████▍ | 1849/2500 [11:03:52<3:49:48, 21.18s/it] 74%|███████▍ | 1850/2500 [11:04:13<3:47:54, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.03757069651785723, 'learning_rate': 2.6e-07, 'completion_length': 146.27679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00421142578125, 'epoch': 0.74} 74%|███████▍ | 1850/2500 [11:04:13<3:47:54, 21.04s/it] 74%|███████▍ | 1851/2500 [11:04:34<3:48:44, 21.15s/it] {'loss': 0.0003, 'grad_norm': 1.384319693207922, 'learning_rate': 2.5959999999999997e-07, 'completion_length': 163.2053680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726335644722, 'kl': 0.0076141357421875, 'epoch': 0.74} 74%|███████▍ | 1851/2500 [11:04:34<3:48:44, 21.15s/it] 74%|███████▍ | 1852/2500 [11:04:56<3:51:10, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.5688886362757873, 'learning_rate': 2.592e-07, 'completion_length': 178.7232208251953, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.06613001227378845, 'kl': 0.0078582763671875, 'epoch': 0.74} 74%|███████▍ | 1852/2500 [11:04:56<3:51:10, 21.40s/it] 74%|███████▍ | 1853/2500 [11:05:18<3:51:15, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.43840110856813824, 'learning_rate': 2.5879999999999996e-07, 'completion_length': 158.75000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0068817138671875, 'epoch': 0.74} 74%|███████▍ | 1853/2500 [11:05:18<3:51:15, 21.45s/it] 74%|███████▍ | 1854/2500 [11:05:40<3:51:21, 21.49s/it] {'loss': 0.0002, 'grad_norm': 0.64404025186619, 'learning_rate': 2.584e-07, 'completion_length': 155.2232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004974365234375, 'epoch': 0.74} 74%|███████▍ | 1854/2500 [11:05:40<3:51:21, 21.49s/it] 74%|███████▍ | 1855/2500 [11:06:01<3:50:25, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.27632266540150235, 'learning_rate': 2.58e-07, 'completion_length': 161.1964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059967041015625, 'epoch': 0.74} 74%|███████▍ | 1855/2500 [11:06:01<3:50:25, 21.43s/it] 74%|███████▍ | 1856/2500 [11:06:22<3:49:47, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.7373474816161107, 'learning_rate': 2.576e-07, 'completion_length': 147.61607360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.0049285888671875, 'epoch': 0.74} 74%|███████▍ | 1856/2500 [11:06:22<3:49:47, 21.41s/it] 74%|███████▍ | 1857/2500 [11:06:43<3:47:37, 21.24s/it] {'loss': 0.0001, 'grad_norm': 0.02143080677047345, 'learning_rate': 2.5719999999999995e-07, 'completion_length': 148.62500762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0037384033203125, 'epoch': 0.74} 74%|███████▍ | 1857/2500 [11:06:43<3:47:37, 21.24s/it] 74%|███████▍ | 1858/2500 [11:07:04<3:45:45, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.027826948776222188, 'learning_rate': 2.5679999999999997e-07, 'completion_length': 147.8928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00392913818359375, 'epoch': 0.74} 74%|███████▍ | 1858/2500 [11:07:04<3:45:45, 21.10s/it] 74%|███████▍ | 1859/2500 [11:07:25<3:45:00, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.2475424770457851, 'learning_rate': 2.564e-07, 'completion_length': 141.5982208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.03696779906749725, 'kl': 0.006134033203125, 'epoch': 0.74} 74%|███████▍ | 1859/2500 [11:07:25<3:45:00, 21.06s/it] 74%|███████▍ | 1860/2500 [11:07:46<3:45:14, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.4909940826472103, 'learning_rate': 2.56e-07, 'completion_length': 153.25000762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.004608154296875, 'epoch': 0.74} 74%|███████▍ | 1860/2500 [11:07:46<3:45:14, 21.12s/it] 74%|███████▍ | 1861/2500 [11:08:06<3:42:26, 20.89s/it] {'loss': 0.0002, 'grad_norm': 0.21697750375506458, 'learning_rate': 2.556e-07, 'completion_length': 147.15179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00433349609375, 'epoch': 0.74} 74%|███████▍ | 1861/2500 [11:08:06<3:42:26, 20.89s/it] 74%|███████▍ | 1862/2500 [11:08:27<3:41:40, 20.85s/it] {'loss': 0.0002, 'grad_norm': 0.630322332260831, 'learning_rate': 2.5519999999999996e-07, 'completion_length': 148.11607360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0039520263671875, 'epoch': 0.74} 74%|███████▍ | 1862/2500 [11:08:27<3:41:40, 20.85s/it] 75%|███████▍ | 1863/2500 [11:08:48<3:42:06, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.2902081319875165, 'learning_rate': 2.5480000000000003e-07, 'completion_length': 151.28571701049805, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.005706787109375, 'epoch': 0.75} 75%|███████▍ | 1863/2500 [11:08:48<3:42:06, 20.92s/it] 75%|███████▍ | 1864/2500 [11:09:09<3:42:48, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.16773259177274263, 'learning_rate': 2.544e-07, 'completion_length': 163.05358123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0062408447265625, 'epoch': 0.75} 75%|███████▍ | 1864/2500 [11:09:09<3:42:48, 21.02s/it] 75%|███████▍ | 1865/2500 [11:09:31<3:44:03, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.9043041653938475, 'learning_rate': 2.5399999999999997e-07, 'completion_length': 147.12500762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.005157470703125, 'epoch': 0.75} 75%|███████▍ | 1865/2500 [11:09:31<3:44:03, 21.17s/it] 75%|███████▍ | 1866/2500 [11:09:52<3:43:42, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.6010354012292852, 'learning_rate': 2.536e-07, 'completion_length': 152.7946548461914, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.005340576171875, 'epoch': 0.75} 75%|███████▍ | 1866/2500 [11:09:52<3:43:42, 21.17s/it] 75%|███████▍ | 1867/2500 [11:10:13<3:41:52, 21.03s/it] {'loss': 0.0002, 'grad_norm': 6.25232568504442, 'learning_rate': 2.5319999999999996e-07, 'completion_length': 143.0178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00490570068359375, 'epoch': 0.75} 75%|███████▍ | 1867/2500 [11:10:13<3:41:52, 21.03s/it] 75%|███████▍ | 1868/2500 [11:10:34<3:42:11, 21.09s/it] {'loss': 0.0001, 'grad_norm': 0.023371375298165863, 'learning_rate': 2.528e-07, 'completion_length': 155.7678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0036773681640625, 'epoch': 0.75} 75%|███████▍ | 1868/2500 [11:10:34<3:42:11, 21.09s/it] 75%|███████▍ | 1869/2500 [11:10:55<3:42:18, 21.14s/it] {'loss': 0.0001, 'grad_norm': 0.023490142898703974, 'learning_rate': 2.524e-07, 'completion_length': 165.29464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00360870361328125, 'epoch': 0.75} 75%|███████▍ | 1869/2500 [11:10:55<3:42:18, 21.14s/it] 75%|███████▍ | 1870/2500 [11:11:16<3:41:46, 21.12s/it] {'loss': 0.0001, 'grad_norm': 0.01454958100948971, 'learning_rate': 2.52e-07, 'completion_length': 141.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00368499755859375, 'epoch': 0.75} 75%|███████▍ | 1870/2500 [11:11:16<3:41:46, 21.12s/it] 75%|███████▍ | 1871/2500 [11:11:37<3:41:14, 21.10s/it] {'loss': 0.0001, 'grad_norm': 0.2759295595683618, 'learning_rate': 2.516e-07, 'completion_length': 140.8482208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.003570556640625, 'epoch': 0.75} 75%|███████▍ | 1871/2500 [11:11:37<3:41:14, 21.10s/it] 75%|███████▍ | 1872/2500 [11:11:59<3:40:43, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.363442376105293, 'learning_rate': 2.5119999999999997e-07, 'completion_length': 154.5714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0049591064453125, 'epoch': 0.75} 75%|███████▍ | 1872/2500 [11:11:59<3:40:43, 21.09s/it] 75%|███████▍ | 1873/2500 [11:12:19<3:39:10, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.25543103235399284, 'learning_rate': 2.508e-07, 'completion_length': 136.56250381469727, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0045013427734375, 'epoch': 0.75} 75%|███████▍ | 1873/2500 [11:12:19<3:39:10, 20.97s/it] 75%|███████▍ | 1874/2500 [11:12:40<3:38:55, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.029931780112404247, 'learning_rate': 2.504e-07, 'completion_length': 143.17858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00461578369140625, 'epoch': 0.75} 75%|███████▍ | 1874/2500 [11:12:40<3:38:55, 20.98s/it] 75%|███████▌ | 1875/2500 [11:13:02<3:40:48, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.03029544581454734, 'learning_rate': 2.5e-07, 'completion_length': 164.4821548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056304931640625, 'epoch': 0.75} 75%|███████▌ | 1875/2500 [11:13:02<3:40:48, 21.20s/it] 75%|███████▌ | 1876/2500 [11:13:23<3:39:03, 21.06s/it] {'loss': 0.0002, 'grad_norm': 3.2283108316470774, 'learning_rate': 2.4959999999999996e-07, 'completion_length': 141.04464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004302978515625, 'epoch': 0.75} 75%|███████▌ | 1876/2500 [11:13:23<3:39:03, 21.06s/it] 75%|███████▌ | 1877/2500 [11:13:46<3:45:06, 21.68s/it] {'loss': 0.0003, 'grad_norm': 1.2124873837653043, 'learning_rate': 2.492e-07, 'completion_length': 166.8214340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.008148193359375, 'epoch': 0.75} 75%|███████▌ | 1877/2500 [11:13:46<3:45:06, 21.68s/it] 75%|███████▌ | 1878/2500 [11:14:08<3:44:46, 21.68s/it] {'loss': 0.0003, 'grad_norm': 1.4460334887835247, 'learning_rate': 2.488e-07, 'completion_length': 155.0982208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0074310302734375, 'epoch': 0.75} 75%|███████▌ | 1878/2500 [11:14:08<3:44:46, 21.68s/it] 75%|███████▌ | 1879/2500 [11:14:29<3:43:31, 21.60s/it] {'loss': 0.0003, 'grad_norm': 0.24984997497872585, 'learning_rate': 2.484e-07, 'completion_length': 157.8303680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.007049560546875, 'epoch': 0.75} 75%|███████▌ | 1879/2500 [11:14:29<3:43:31, 21.60s/it] 75%|███████▌ | 1880/2500 [11:14:50<3:41:29, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.727173670252553, 'learning_rate': 2.48e-07, 'completion_length': 158.4464340209961, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161417350172997, 'kl': 0.0058135986328125, 'epoch': 0.75} 75%|███████▌ | 1880/2500 [11:14:50<3:41:29, 21.43s/it] 75%|███████▌ | 1881/2500 [11:15:11<3:40:48, 21.40s/it] {'loss': 0.0001, 'grad_norm': 0.2680232479522894, 'learning_rate': 2.4759999999999997e-07, 'completion_length': 148.66964721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.003631591796875, 'epoch': 0.75} 75%|███████▌ | 1881/2500 [11:15:11<3:40:48, 21.40s/it] 75%|███████▌ | 1882/2500 [11:15:33<3:41:08, 21.47s/it] {'loss': 0.0003, 'grad_norm': 0.3952206991464004, 'learning_rate': 2.472e-07, 'completion_length': 160.77678680419922, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.06613001227378845, 'kl': 0.006805419921875, 'epoch': 0.75} 75%|███████▌ | 1882/2500 [11:15:33<3:41:08, 21.47s/it] 75%|███████▌ | 1883/2500 [11:15:55<3:43:13, 21.71s/it] {'loss': 0.0003, 'grad_norm': 0.790771989155939, 'learning_rate': 2.4679999999999996e-07, 'completion_length': 180.83036041259766, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.0082855224609375, 'epoch': 0.75} 75%|███████▌ | 1883/2500 [11:15:55<3:43:13, 21.71s/it] 75%|███████▌ | 1884/2500 [11:16:16<3:39:37, 21.39s/it] {'loss': 0.0001, 'grad_norm': 0.18678797757684798, 'learning_rate': 2.464e-07, 'completion_length': 143.4732208251953, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.03696779906749725, 'kl': 0.00336456298828125, 'epoch': 0.75} 75%|███████▌ | 1884/2500 [11:16:16<3:39:37, 21.39s/it] 75%|███████▌ | 1885/2500 [11:16:38<3:40:22, 21.50s/it] {'loss': 0.0003, 'grad_norm': 0.41166718917083095, 'learning_rate': 2.46e-07, 'completion_length': 170.24108123779297, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.07576144114136696, 'kl': 0.006744384765625, 'epoch': 0.75} 75%|███████▌ | 1885/2500 [11:16:38<3:40:22, 21.50s/it] 75%|███████▌ | 1886/2500 [11:17:00<3:41:18, 21.63s/it] {'loss': 0.0003, 'grad_norm': 0.19233308586431422, 'learning_rate': 2.456e-07, 'completion_length': 169.81250762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0081634521484375, 'epoch': 0.75} 75%|███████▌ | 1886/2500 [11:17:00<3:41:18, 21.63s/it] 75%|███████▌ | 1887/2500 [11:17:21<3:40:10, 21.55s/it] {'loss': 0.0002, 'grad_norm': 0.36022873742176803, 'learning_rate': 2.452e-07, 'completion_length': 149.0089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048370361328125, 'epoch': 0.75} 75%|███████▌ | 1887/2500 [11:17:21<3:40:10, 21.55s/it] 76%|███████▌ | 1888/2500 [11:17:42<3:38:28, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.03981904703360587, 'learning_rate': 2.4479999999999997e-07, 'completion_length': 142.20536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0059356689453125, 'epoch': 0.76} 76%|███████▌ | 1888/2500 [11:17:42<3:38:28, 21.42s/it] 76%|███████▌ | 1889/2500 [11:18:04<3:39:48, 21.58s/it] {'loss': 0.0002, 'grad_norm': 0.39896695396031767, 'learning_rate': 2.444e-07, 'completion_length': 168.4107208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00604248046875, 'epoch': 0.76} 76%|███████▌ | 1889/2500 [11:18:04<3:39:48, 21.58s/it] 76%|███████▌ | 1890/2500 [11:18:25<3:37:59, 21.44s/it] {'loss': 0.0001, 'grad_norm': 0.02898252930937648, 'learning_rate': 2.4399999999999996e-07, 'completion_length': 154.30357360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00354766845703125, 'epoch': 0.76} 76%|███████▌ | 1890/2500 [11:18:25<3:37:59, 21.44s/it] 76%|███████▌ | 1891/2500 [11:18:46<3:37:27, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.019174655598826838, 'learning_rate': 2.436e-07, 'completion_length': 156.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00457763671875, 'epoch': 0.76} 76%|███████▌ | 1891/2500 [11:18:46<3:37:27, 21.42s/it] 76%|███████▌ | 1892/2500 [11:19:08<3:36:44, 21.39s/it] {'loss': 0.0001, 'grad_norm': 0.1598195590537006, 'learning_rate': 2.432e-07, 'completion_length': 162.91964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0035858154296875, 'epoch': 0.76} 76%|███████▌ | 1892/2500 [11:19:08<3:36:44, 21.39s/it] 76%|███████▌ | 1893/2500 [11:19:29<3:36:31, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.5809590192458078, 'learning_rate': 2.428e-07, 'completion_length': 150.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005889892578125, 'epoch': 0.76} 76%|███████▌ | 1893/2500 [11:19:29<3:36:31, 21.40s/it] 76%|███████▌ | 1894/2500 [11:19:51<3:38:13, 21.61s/it] {'loss': 0.0004, 'grad_norm': 0.6940013097993074, 'learning_rate': 2.424e-07, 'completion_length': 160.87500762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0107879638671875, 'epoch': 0.76} 76%|███████▌ | 1894/2500 [11:19:51<3:38:13, 21.61s/it] 76%|███████▌ | 1895/2500 [11:20:12<3:36:24, 21.46s/it] {'loss': 0.0003, 'grad_norm': 0.022748255363528303, 'learning_rate': 2.4199999999999997e-07, 'completion_length': 157.6607208251953, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.0, 'kl': 0.0063629150390625, 'epoch': 0.76} 76%|███████▌ | 1895/2500 [11:20:12<3:36:24, 21.46s/it] 76%|███████▌ | 1896/2500 [11:20:33<3:34:24, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.36304398284642214, 'learning_rate': 2.416e-07, 'completion_length': 150.7321548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0037689208984375, 'epoch': 0.76} 76%|███████▌ | 1896/2500 [11:20:33<3:34:24, 21.30s/it] 76%|███████▌ | 1897/2500 [11:20:55<3:34:10, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.3131744952691785, 'learning_rate': 2.4119999999999996e-07, 'completion_length': 160.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0047149658203125, 'epoch': 0.76} 76%|███████▌ | 1897/2500 [11:20:55<3:34:10, 21.31s/it] 76%|███████▌ | 1898/2500 [11:21:16<3:34:27, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.21088193333832625, 'learning_rate': 2.408e-07, 'completion_length': 158.75000762939453, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00701904296875, 'epoch': 0.76} 76%|███████▌ | 1898/2500 [11:21:16<3:34:27, 21.37s/it] 76%|███████▌ | 1899/2500 [11:21:38<3:35:44, 21.54s/it] {'loss': 0.0003, 'grad_norm': 0.3626560435822112, 'learning_rate': 2.404e-07, 'completion_length': 158.5982208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.007476806640625, 'epoch': 0.76} 76%|███████▌ | 1899/2500 [11:21:38<3:35:44, 21.54s/it] 76%|███████▌ | 1900/2500 [11:22:00<3:37:16, 21.73s/it] {'loss': 0.0003, 'grad_norm': 0.39011162640210234, 'learning_rate': 2.4e-07, 'completion_length': 169.5357208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006622314453125, 'epoch': 0.76} 76%|███████▌ | 1900/2500 [11:22:00<3:37:16, 21.73s/it] 76%|███████▌ | 1901/2500 [11:23:13<6:10:52, 37.15s/it] {'loss': 0.0002, 'grad_norm': 0.14430953695537177, 'learning_rate': 2.396e-07, 'completion_length': 151.65179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00382232666015625, 'epoch': 0.76} 76%|███████▌ | 1901/2500 [11:23:13<6:10:52, 37.15s/it] 76%|███████▌ | 1902/2500 [11:23:34<5:21:18, 32.24s/it] {'loss': 0.0003, 'grad_norm': 1.2614164204545235, 'learning_rate': 2.3919999999999997e-07, 'completion_length': 157.91964721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.006988525390625, 'epoch': 0.76} 76%|███████▌ | 1902/2500 [11:23:34<5:21:18, 32.24s/it] 76%|███████▌ | 1903/2500 [11:23:55<4:46:47, 28.82s/it] {'loss': 0.0002, 'grad_norm': 0.032082762569871595, 'learning_rate': 2.388e-07, 'completion_length': 154.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004913330078125, 'epoch': 0.76} 76%|███████▌ | 1903/2500 [11:23:55<4:46:47, 28.82s/it] 76%|███████▌ | 1904/2500 [11:24:16<4:23:52, 26.56s/it] {'loss': 0.0003, 'grad_norm': 1.4255897219830196, 'learning_rate': 2.384e-07, 'completion_length': 159.3482208251953, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.05831882357597351, 'kl': 0.00848388671875, 'epoch': 0.76} 76%|███████▌ | 1904/2500 [11:24:16<4:23:52, 26.56s/it] 76%|███████▌ | 1905/2500 [11:24:37<4:05:25, 24.75s/it] {'loss': 0.0002, 'grad_norm': 0.022236504123732306, 'learning_rate': 2.38e-07, 'completion_length': 152.0446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005340576171875, 'epoch': 0.76} 76%|███████▌ | 1905/2500 [11:24:37<4:05:25, 24.75s/it] 76%|███████▌ | 1906/2500 [11:24:58<3:53:39, 23.60s/it] {'loss': 0.0003, 'grad_norm': 0.01896647638056423, 'learning_rate': 2.3759999999999998e-07, 'completion_length': 149.99108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0063629150390625, 'epoch': 0.76} 76%|███████▌ | 1906/2500 [11:24:58<3:53:39, 23.60s/it] 76%|███████▋ | 1907/2500 [11:25:19<3:47:15, 22.99s/it] {'loss': 0.0002, 'grad_norm': 0.6491492359785171, 'learning_rate': 2.3719999999999998e-07, 'completion_length': 165.87500762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.0835726335644722, 'kl': 0.0059967041015625, 'epoch': 0.76} 76%|███████▋ | 1907/2500 [11:25:19<3:47:15, 22.99s/it] 76%|███████▋ | 1908/2500 [11:25:40<3:40:53, 22.39s/it] {'loss': 0.0002, 'grad_norm': 0.2578479288253268, 'learning_rate': 2.368e-07, 'completion_length': 157.9107208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.005035400390625, 'epoch': 0.76} 76%|███████▋ | 1908/2500 [11:25:40<3:40:53, 22.39s/it] 76%|███████▋ | 1909/2500 [11:26:01<3:34:47, 21.81s/it] {'loss': 0.0001, 'grad_norm': 0.36537399032551726, 'learning_rate': 2.364e-07, 'completion_length': 142.0178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00359344482421875, 'epoch': 0.76} 76%|███████▋ | 1909/2500 [11:26:01<3:34:47, 21.81s/it] 76%|███████▋ | 1910/2500 [11:26:22<3:33:51, 21.75s/it] {'loss': 0.0002, 'grad_norm': 0.17297545259275585, 'learning_rate': 2.3599999999999997e-07, 'completion_length': 153.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006103515625, 'epoch': 0.76} 76%|███████▋ | 1910/2500 [11:26:22<3:33:51, 21.75s/it] 76%|███████▋ | 1911/2500 [11:26:44<3:31:46, 21.57s/it] {'loss': 0.0003, 'grad_norm': 0.04331123527412917, 'learning_rate': 2.356e-07, 'completion_length': 171.1607208251953, 'rewards/accuracy_reward': 0.8571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0, 'kl': 0.0069732666015625, 'epoch': 0.76} 76%|███████▋ | 1911/2500 [11:26:44<3:31:46, 21.57s/it] 76%|███████▋ | 1912/2500 [11:27:04<3:29:24, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.4278100212433933, 'learning_rate': 2.352e-07, 'completion_length': 154.5357208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.00833892822265625, 'epoch': 0.76} 76%|███████▋ | 1912/2500 [11:27:04<3:29:24, 21.37s/it] 77%|███████▋ | 1913/2500 [11:27:26<3:29:52, 21.45s/it] {'loss': 0.0003, 'grad_norm': 0.034458623985651325, 'learning_rate': 2.3479999999999998e-07, 'completion_length': 148.39286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0069122314453125, 'epoch': 0.77} 77%|███████▋ | 1913/2500 [11:27:26<3:29:52, 21.45s/it] 77%|███████▋ | 1914/2500 [11:27:47<3:27:55, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.04161695795267312, 'learning_rate': 2.3439999999999998e-07, 'completion_length': 153.11607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005950927734375, 'epoch': 0.77} 77%|███████▋ | 1914/2500 [11:27:47<3:27:55, 21.29s/it] 77%|███████▋ | 1915/2500 [11:28:07<3:24:22, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.018833693146901764, 'learning_rate': 2.34e-07, 'completion_length': 141.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00438690185546875, 'epoch': 0.77} 77%|███████▋ | 1915/2500 [11:28:07<3:24:22, 20.96s/it] 77%|███████▋ | 1916/2500 [11:28:29<3:26:15, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.22956405785907932, 'learning_rate': 2.336e-07, 'completion_length': 171.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006622314453125, 'epoch': 0.77} 77%|███████▋ | 1916/2500 [11:28:29<3:26:15, 21.19s/it] 77%|███████▋ | 1917/2500 [11:28:50<3:25:51, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.6063176780104912, 'learning_rate': 2.3319999999999997e-07, 'completion_length': 165.4107208251953, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.03696779906749725, 'kl': 0.0064239501953125, 'epoch': 0.77} 77%|███████▋ | 1917/2500 [11:28:50<3:25:51, 21.19s/it] 77%|███████▋ | 1918/2500 [11:29:11<3:25:39, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.2559339988255816, 'learning_rate': 2.328e-07, 'completion_length': 152.2946548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0045928955078125, 'epoch': 0.77} 77%|███████▋ | 1918/2500 [11:29:11<3:25:39, 21.20s/it] 77%|███████▋ | 1919/2500 [11:29:33<3:26:15, 21.30s/it] {'loss': 0.0003, 'grad_norm': 0.6513036750182948, 'learning_rate': 2.324e-07, 'completion_length': 167.96429443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0072174072265625, 'epoch': 0.77} 77%|███████▋ | 1919/2500 [11:29:33<3:26:15, 21.30s/it] 77%|███████▋ | 1920/2500 [11:29:54<3:24:21, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.31852058390809135, 'learning_rate': 2.32e-07, 'completion_length': 144.96429443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.004638671875, 'epoch': 0.77} 77%|███████▋ | 1920/2500 [11:29:54<3:24:21, 21.14s/it] 77%|███████▋ | 1921/2500 [11:30:15<3:25:02, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.4344164906877547, 'learning_rate': 2.3159999999999998e-07, 'completion_length': 171.1696548461914, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0065155029296875, 'epoch': 0.77} 77%|███████▋ | 1921/2500 [11:30:15<3:25:02, 21.25s/it] 77%|███████▋ | 1922/2500 [11:30:36<3:23:17, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.5248412129187825, 'learning_rate': 2.3119999999999998e-07, 'completion_length': 151.4732208251953, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0050048828125, 'epoch': 0.77} 77%|███████▋ | 1922/2500 [11:30:36<3:23:17, 21.10s/it] 77%|███████▋ | 1923/2500 [11:30:57<3:22:39, 21.07s/it] {'loss': 0.0003, 'grad_norm': 3.115227175217109, 'learning_rate': 2.308e-07, 'completion_length': 169.2053680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.09528662264347076, 'kl': 0.0077056884765625, 'epoch': 0.77} 77%|███████▋ | 1923/2500 [11:30:57<3:22:39, 21.07s/it] 77%|███████▋ | 1924/2500 [11:31:18<3:22:47, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.275335932232029, 'learning_rate': 2.3039999999999997e-07, 'completion_length': 171.52679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0058135986328125, 'epoch': 0.77} 77%|███████▋ | 1924/2500 [11:31:18<3:22:47, 21.12s/it] 77%|███████▋ | 1925/2500 [11:31:40<3:23:07, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.8251625461571256, 'learning_rate': 2.3e-07, 'completion_length': 145.40179443359375, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0055389404296875, 'epoch': 0.77} 77%|███████▋ | 1925/2500 [11:31:40<3:23:07, 21.20s/it] 77%|███████▋ | 1926/2500 [11:32:01<3:22:17, 21.14s/it] {'loss': 0.0003, 'grad_norm': 0.5776525466183515, 'learning_rate': 2.296e-07, 'completion_length': 151.0982208251953, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0075531005859375, 'epoch': 0.77} 77%|███████▋ | 1926/2500 [11:32:01<3:22:17, 21.14s/it] 77%|███████▋ | 1927/2500 [11:32:21<3:20:47, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.8466458012173754, 'learning_rate': 2.292e-07, 'completion_length': 140.3214340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00537109375, 'epoch': 0.77} 77%|███████▋ | 1927/2500 [11:32:21<3:20:47, 21.03s/it] 77%|███████▋ | 1928/2500 [11:32:43<3:21:44, 21.16s/it] {'loss': 0.0004, 'grad_norm': 0.4165926418937268, 'learning_rate': 2.2879999999999998e-07, 'completion_length': 160.8303680419922, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.009521484375, 'epoch': 0.77} 77%|███████▋ | 1928/2500 [11:32:43<3:21:44, 21.16s/it] 77%|███████▋ | 1929/2500 [11:33:04<3:20:46, 21.10s/it] {'loss': 0.0003, 'grad_norm': 0.2269073926293333, 'learning_rate': 2.2839999999999998e-07, 'completion_length': 153.2232208251953, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0066375732421875, 'epoch': 0.77} 77%|███████▋ | 1929/2500 [11:33:04<3:20:46, 21.10s/it] 77%|███████▋ | 1930/2500 [11:33:24<3:19:21, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.48090398424791303, 'learning_rate': 2.28e-07, 'completion_length': 161.5089340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00485992431640625, 'epoch': 0.77} 77%|███████▋ | 1930/2500 [11:33:24<3:19:21, 20.99s/it] 77%|███████▋ | 1931/2500 [11:33:46<3:19:36, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.7087876576593422, 'learning_rate': 2.2759999999999997e-07, 'completion_length': 158.70536041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0067901611328125, 'epoch': 0.77} 77%|███████▋ | 1931/2500 [11:33:46<3:19:36, 21.05s/it] 77%|███████▋ | 1932/2500 [11:34:07<3:19:59, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.01737828811412226, 'learning_rate': 2.272e-07, 'completion_length': 150.02678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004302978515625, 'epoch': 0.77} 77%|███████▋ | 1932/2500 [11:34:07<3:19:59, 21.13s/it] 77%|███████▋ | 1933/2500 [11:34:28<3:18:26, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.45662773257711203, 'learning_rate': 2.268e-07, 'completion_length': 134.9107208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00412750244140625, 'epoch': 0.77} 77%|███████▋ | 1933/2500 [11:34:28<3:18:26, 21.00s/it] 77%|███████▋ | 1934/2500 [11:34:49<3:19:45, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.3307824755066285, 'learning_rate': 2.264e-07, 'completion_length': 162.40178680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0047149658203125, 'epoch': 0.77} 77%|███████▋ | 1934/2500 [11:34:49<3:19:45, 21.18s/it] 77%|███████▋ | 1935/2500 [11:35:11<3:21:02, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.020473380415279113, 'learning_rate': 2.2599999999999999e-07, 'completion_length': 159.08929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004608154296875, 'epoch': 0.77} 77%|███████▋ | 1935/2500 [11:35:11<3:21:02, 21.35s/it] 77%|███████▋ | 1936/2500 [11:35:32<3:20:36, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.4651544597090534, 'learning_rate': 2.2559999999999998e-07, 'completion_length': 151.75000762939453, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0051422119140625, 'epoch': 0.77} 77%|███████▋ | 1936/2500 [11:35:32<3:20:36, 21.34s/it] 77%|███████▋ | 1937/2500 [11:35:54<3:20:27, 21.36s/it] {'loss': 0.0003, 'grad_norm': 0.5513367205761093, 'learning_rate': 2.252e-07, 'completion_length': 159.1428680419922, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.05831881985068321, 'kl': 0.00640869140625, 'epoch': 0.77} 77%|███████▋ | 1937/2500 [11:35:54<3:20:27, 21.36s/it] 78%|███████▊ | 1938/2500 [11:36:15<3:19:53, 21.34s/it] {'loss': 0.0004, 'grad_norm': 1.024406330358029, 'learning_rate': 2.248e-07, 'completion_length': 155.1428680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831881985068321, 'kl': 0.008819580078125, 'epoch': 0.78} 78%|███████▊ | 1938/2500 [11:36:15<3:19:53, 21.34s/it] 78%|███████▊ | 1939/2500 [11:36:37<3:20:32, 21.45s/it] {'loss': 0.0003, 'grad_norm': 5.047605698062099, 'learning_rate': 2.2439999999999997e-07, 'completion_length': 162.04464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00640869140625, 'epoch': 0.78} 78%|███████▊ | 1939/2500 [11:36:37<3:20:32, 21.45s/it] 78%|███████▊ | 1940/2500 [11:36:58<3:20:34, 21.49s/it] {'loss': 0.0002, 'grad_norm': 0.6016774561012727, 'learning_rate': 2.24e-07, 'completion_length': 150.55358123779297, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.05831882357597351, 'kl': 0.00518798828125, 'epoch': 0.78} 78%|███████▊ | 1940/2500 [11:36:58<3:20:34, 21.49s/it] 78%|███████▊ | 1941/2500 [11:37:19<3:18:09, 21.27s/it] {'loss': 0.0001, 'grad_norm': 0.03286774303801891, 'learning_rate': 2.236e-07, 'completion_length': 148.92858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0031890869140625, 'epoch': 0.78} 78%|███████▊ | 1941/2500 [11:37:19<3:18:09, 21.27s/it] 78%|███████▊ | 1942/2500 [11:37:40<3:16:14, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.7430996472604985, 'learning_rate': 2.232e-07, 'completion_length': 142.33036041259766, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.09918941557407379, 'kl': 0.0053863525390625, 'epoch': 0.78} 78%|███████▊ | 1942/2500 [11:37:40<3:16:14, 21.10s/it] 78%|███████▊ | 1943/2500 [11:38:01<3:15:04, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.2437963392162435, 'learning_rate': 2.2279999999999998e-07, 'completion_length': 142.7232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0039215087890625, 'epoch': 0.78} 78%|███████▊ | 1943/2500 [11:38:01<3:15:04, 21.01s/it] 78%|███████▊ | 1944/2500 [11:38:21<3:14:07, 20.95s/it] {'loss': 0.0001, 'grad_norm': 0.03230957991470122, 'learning_rate': 2.2239999999999998e-07, 'completion_length': 140.6964340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.002948760986328125, 'epoch': 0.78} 78%|███████▊ | 1944/2500 [11:38:21<3:14:07, 20.95s/it] 78%|███████▊ | 1945/2500 [11:38:43<3:14:54, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.29196213304212376, 'learning_rate': 2.22e-07, 'completion_length': 138.1339340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0037689208984375, 'epoch': 0.78} 78%|███████▊ | 1945/2500 [11:38:43<3:14:54, 21.07s/it] 78%|███████▊ | 1946/2500 [11:39:03<3:13:29, 20.96s/it] {'loss': 0.0002, 'grad_norm': 1.0628299786894926, 'learning_rate': 2.2159999999999997e-07, 'completion_length': 144.75000762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.13346679508686066, 'kl': 0.0056610107421875, 'epoch': 0.78} 78%|███████▊ | 1946/2500 [11:39:03<3:13:29, 20.96s/it] 78%|███████▊ | 1947/2500 [11:39:25<3:13:54, 21.04s/it] {'loss': 0.0003, 'grad_norm': 0.04387713779730086, 'learning_rate': 2.212e-07, 'completion_length': 153.39286041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0071868896484375, 'epoch': 0.78} 78%|███████▊ | 1947/2500 [11:39:25<3:13:54, 21.04s/it] 78%|███████▊ | 1948/2500 [11:39:46<3:14:00, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.5139986838301022, 'learning_rate': 2.208e-07, 'completion_length': 160.0357208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.07003280520439148, 'kl': 0.0061187744140625, 'epoch': 0.78} 78%|███████▊ | 1948/2500 [11:39:46<3:14:00, 21.09s/it] 78%|███████▊ | 1949/2500 [11:40:07<3:14:06, 21.14s/it] {'loss': 0.0001, 'grad_norm': 0.35756947674372713, 'learning_rate': 2.2040000000000001e-07, 'completion_length': 146.15178680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0034637451171875, 'epoch': 0.78} 78%|███████▊ | 1949/2500 [11:40:07<3:14:06, 21.14s/it] 78%|███████▊ | 1950/2500 [11:40:29<3:16:59, 21.49s/it] {'loss': 0.0003, 'grad_norm': 1.806480652397352, 'learning_rate': 2.1999999999999998e-07, 'completion_length': 163.2589340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.08868780359625816, 'kl': 0.0086212158203125, 'epoch': 0.78} 78%|███████▊ | 1950/2500 [11:40:29<3:16:59, 21.49s/it] 78%|███████▊ | 1951/2500 [11:40:50<3:14:44, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.01651248400571913, 'learning_rate': 2.1959999999999998e-07, 'completion_length': 147.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00429534912109375, 'epoch': 0.78} 78%|███████▊ | 1951/2500 [11:40:50<3:14:44, 21.28s/it] 78%|███████▊ | 1952/2500 [11:41:11<3:13:45, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.022929598512339923, 'learning_rate': 2.192e-07, 'completion_length': 149.8214340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006256103515625, 'epoch': 0.78} 78%|███████▊ | 1952/2500 [11:41:11<3:13:45, 21.21s/it] 78%|███████▊ | 1953/2500 [11:41:33<3:13:44, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.02106642772577689, 'learning_rate': 2.1879999999999997e-07, 'completion_length': 152.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005645751953125, 'epoch': 0.78} 78%|███████▊ | 1953/2500 [11:41:33<3:13:44, 21.25s/it] 78%|███████▊ | 1954/2500 [11:41:54<3:12:27, 21.15s/it] {'loss': 0.0003, 'grad_norm': 0.05491174219712237, 'learning_rate': 2.184e-07, 'completion_length': 142.64286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00641632080078125, 'epoch': 0.78} 78%|███████▊ | 1954/2500 [11:41:54<3:12:27, 21.15s/it] 78%|███████▊ | 1955/2500 [11:42:15<3:12:13, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.2705305903277663, 'learning_rate': 2.18e-07, 'completion_length': 161.86607360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0081024169921875, 'epoch': 0.78} 78%|███████▊ | 1955/2500 [11:42:15<3:12:13, 21.16s/it] 78%|███████▊ | 1956/2500 [11:42:36<3:10:57, 21.06s/it] {'loss': 0.0002, 'grad_norm': 1.491028610821134, 'learning_rate': 2.176e-07, 'completion_length': 148.18750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0739355981349945, 'kl': 0.00579833984375, 'epoch': 0.78} 78%|███████▊ | 1956/2500 [11:42:36<3:10:57, 21.06s/it] 78%|███████▊ | 1957/2500 [11:42:57<3:11:20, 21.14s/it] {'loss': 0.0002, 'grad_norm': 1.5856681379953268, 'learning_rate': 2.1719999999999999e-07, 'completion_length': 137.52679443359375, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.1030978113412857, 'kl': 0.0055999755859375, 'epoch': 0.78} 78%|███████▊ | 1957/2500 [11:42:57<3:11:20, 21.14s/it] 78%|███████▊ | 1958/2500 [11:43:18<3:10:48, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.7872568284890987, 'learning_rate': 2.1679999999999998e-07, 'completion_length': 146.8928680419922, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00485992431640625, 'epoch': 0.78} 78%|███████▊ | 1958/2500 [11:43:18<3:10:48, 21.12s/it] 78%|███████▊ | 1959/2500 [11:43:39<3:09:46, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.018167748524355473, 'learning_rate': 2.164e-07, 'completion_length': 158.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0045318603515625, 'epoch': 0.78} 78%|███████▊ | 1959/2500 [11:43:39<3:09:46, 21.05s/it] 78%|███████▊ | 1960/2500 [11:44:00<3:09:44, 21.08s/it] {'loss': 0.0003, 'grad_norm': 0.018094624632604986, 'learning_rate': 2.1599999999999998e-07, 'completion_length': 157.64286041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00689697265625, 'epoch': 0.78} 78%|███████▊ | 1960/2500 [11:44:00<3:09:44, 21.08s/it] 78%|███████▊ | 1961/2500 [11:44:21<3:10:10, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.2843747332189475, 'learning_rate': 2.156e-07, 'completion_length': 147.52679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00418853759765625, 'epoch': 0.78} 78%|███████▊ | 1961/2500 [11:44:21<3:10:10, 21.17s/it] 78%|███████▊ | 1962/2500 [11:44:43<3:10:03, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.03732721305138022, 'learning_rate': 2.152e-07, 'completion_length': 155.36608123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0063934326171875, 'epoch': 0.78} 78%|███████▊ | 1962/2500 [11:44:43<3:10:03, 21.20s/it] 79%|███████▊ | 1963/2500 [11:45:04<3:11:22, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.5029168717776697, 'learning_rate': 2.148e-07, 'completion_length': 151.62500762939453, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0041961669921875, 'epoch': 0.79} 79%|███████▊ | 1963/2500 [11:45:04<3:11:22, 21.38s/it] 79%|███████▊ | 1964/2500 [11:45:26<3:11:10, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.020503085266615047, 'learning_rate': 2.144e-07, 'completion_length': 146.9464340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0068511962890625, 'epoch': 0.79} 79%|███████▊ | 1964/2500 [11:45:26<3:11:10, 21.40s/it] 79%|███████▊ | 1965/2500 [11:45:48<3:11:55, 21.52s/it] {'loss': 0.0002, 'grad_norm': 0.2039021744633164, 'learning_rate': 2.1399999999999998e-07, 'completion_length': 155.65179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005462646484375, 'epoch': 0.79} 79%|███████▊ | 1965/2500 [11:45:48<3:11:55, 21.52s/it] 79%|███████▊ | 1966/2500 [11:46:09<3:11:22, 21.50s/it] {'loss': 0.0002, 'grad_norm': 0.015840342991365483, 'learning_rate': 2.136e-07, 'completion_length': 167.41964721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.005706787109375, 'epoch': 0.79} 79%|███████▊ | 1966/2500 [11:46:09<3:11:22, 21.50s/it] 79%|███████▊ | 1967/2500 [11:46:31<3:11:44, 21.58s/it] {'loss': 0.0003, 'grad_norm': 0.27549317966538256, 'learning_rate': 2.132e-07, 'completion_length': 168.89286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0069580078125, 'epoch': 0.79} 79%|███████▊ | 1967/2500 [11:46:31<3:11:44, 21.58s/it] 79%|███████▊ | 1968/2500 [11:46:52<3:10:10, 21.45s/it] {'loss': 0.0002, 'grad_norm': 0.351447744438347, 'learning_rate': 2.1279999999999997e-07, 'completion_length': 153.58929443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005218505859375, 'epoch': 0.79} 79%|███████▊ | 1968/2500 [11:46:52<3:10:10, 21.45s/it] 79%|███████▉ | 1969/2500 [11:47:14<3:10:15, 21.50s/it] {'loss': 0.0002, 'grad_norm': 0.560739580902671, 'learning_rate': 2.124e-07, 'completion_length': 152.3214340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00388336181640625, 'epoch': 0.79} 79%|███████▉ | 1969/2500 [11:47:14<3:10:15, 21.50s/it] 79%|███████▉ | 1970/2500 [11:47:36<3:11:28, 21.68s/it] {'loss': 0.0003, 'grad_norm': 1.1219387221438102, 'learning_rate': 2.12e-07, 'completion_length': 166.7946548461914, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.00628662109375, 'epoch': 0.79} 79%|███████▉ | 1970/2500 [11:47:36<3:11:28, 21.68s/it] 79%|███████▉ | 1971/2500 [11:47:57<3:10:50, 21.64s/it] {'loss': 0.0002, 'grad_norm': 0.19965532421795149, 'learning_rate': 2.116e-07, 'completion_length': 156.31250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00555419921875, 'epoch': 0.79} 79%|███████▉ | 1971/2500 [11:47:57<3:10:50, 21.64s/it] 79%|███████▉ | 1972/2500 [11:48:19<3:09:18, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.7938287148732796, 'learning_rate': 2.1119999999999999e-07, 'completion_length': 163.1339340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.05831881985068321, 'kl': 0.007537841796875, 'epoch': 0.79} 79%|███████▉ | 1972/2500 [11:48:19<3:09:18, 21.51s/it] 79%|███████▉ | 1973/2500 [11:48:42<3:12:59, 21.97s/it] {'loss': 0.0002, 'grad_norm': 0.8782738371631641, 'learning_rate': 2.1079999999999998e-07, 'completion_length': 152.40179443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.09528662264347076, 'kl': 0.0049285888671875, 'epoch': 0.79} 79%|███████▉ | 1973/2500 [11:48:42<3:12:59, 21.97s/it] 79%|███████▉ | 1974/2500 [11:49:02<3:08:13, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.23940091472329927, 'learning_rate': 2.104e-07, 'completion_length': 134.91964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004608154296875, 'epoch': 0.79} 79%|███████▉ | 1974/2500 [11:49:02<3:08:13, 21.47s/it] 79%|███████▉ | 1975/2500 [11:49:24<3:09:27, 21.65s/it] {'loss': 0.0003, 'grad_norm': 0.64788938871609, 'learning_rate': 2.0999999999999997e-07, 'completion_length': 173.8482208251953, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0079803466796875, 'epoch': 0.79} 79%|███████▉ | 1975/2500 [11:49:24<3:09:27, 21.65s/it] 79%|███████▉ | 1976/2500 [11:49:45<3:07:12, 21.44s/it] {'loss': 0.0001, 'grad_norm': 0.06327489401489743, 'learning_rate': 2.096e-07, 'completion_length': 139.1428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.002506256103515625, 'epoch': 0.79} 79%|███████▉ | 1976/2500 [11:49:45<3:07:12, 21.44s/it] 79%|███████▉ | 1977/2500 [11:50:06<3:05:39, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.02357415370887931, 'learning_rate': 2.092e-07, 'completion_length': 150.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0050811767578125, 'epoch': 0.79} 79%|███████▉ | 1977/2500 [11:50:06<3:05:39, 21.30s/it] 79%|███████▉ | 1978/2500 [11:50:28<3:06:15, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.01953852279925795, 'learning_rate': 2.0880000000000002e-07, 'completion_length': 152.2053680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0047760009765625, 'epoch': 0.79} 79%|███████▉ | 1978/2500 [11:50:28<3:06:15, 21.41s/it] 79%|███████▉ | 1979/2500 [11:50:49<3:06:02, 21.42s/it] {'loss': 0.0003, 'grad_norm': 0.21927630936290418, 'learning_rate': 2.0839999999999999e-07, 'completion_length': 162.37500762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0081787109375, 'epoch': 0.79} 79%|███████▉ | 1979/2500 [11:50:49<3:06:02, 21.42s/it] 79%|███████▉ | 1980/2500 [11:51:10<3:03:34, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.06391430910144691, 'learning_rate': 2.0799999999999998e-07, 'completion_length': 145.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00411224365234375, 'epoch': 0.79} 79%|███████▉ | 1980/2500 [11:51:10<3:03:34, 21.18s/it] 79%|███████▉ | 1981/2500 [11:51:31<3:02:49, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.019864044732399454, 'learning_rate': 2.076e-07, 'completion_length': 176.79464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00537872314453125, 'epoch': 0.79} 79%|███████▉ | 1981/2500 [11:51:31<3:02:49, 21.14s/it] 79%|███████▉ | 1982/2500 [11:51:52<3:02:52, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.02180020543695678, 'learning_rate': 2.0719999999999998e-07, 'completion_length': 153.27679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00489044189453125, 'epoch': 0.79} 79%|███████▉ | 1982/2500 [11:51:52<3:02:52, 21.18s/it] 79%|███████▉ | 1983/2500 [11:52:12<3:00:49, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.0239804358040367, 'learning_rate': 2.068e-07, 'completion_length': 142.80358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053558349609375, 'epoch': 0.79} 79%|███████▉ | 1983/2500 [11:52:12<3:00:49, 20.98s/it] 79%|███████▉ | 1984/2500 [11:52:33<2:59:04, 20.82s/it] {'loss': 0.0002, 'grad_norm': 0.3225199430372944, 'learning_rate': 2.064e-07, 'completion_length': 147.20536041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.00447845458984375, 'epoch': 0.79} 79%|███████▉ | 1984/2500 [11:52:33<2:59:04, 20.82s/it] 79%|███████▉ | 1985/2500 [11:52:55<3:02:41, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.4830272068686514, 'learning_rate': 2.06e-07, 'completion_length': 149.71429443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00537109375, 'epoch': 0.79} 79%|███████▉ | 1985/2500 [11:52:55<3:02:41, 21.29s/it] 79%|███████▉ | 1986/2500 [11:53:16<3:01:27, 21.18s/it] {'loss': 0.0003, 'grad_norm': 0.02885427568336711, 'learning_rate': 2.056e-07, 'completion_length': 154.31250762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0063934326171875, 'epoch': 0.79} 79%|███████▉ | 1986/2500 [11:53:16<3:01:27, 21.18s/it] 79%|███████▉ | 1987/2500 [11:53:37<2:59:19, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.02369632606226986, 'learning_rate': 2.0519999999999998e-07, 'completion_length': 146.02679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00377655029296875, 'epoch': 0.79} 79%|███████▉ | 1987/2500 [11:53:37<2:59:19, 20.97s/it] 80%|███████▉ | 1988/2500 [11:53:58<2:58:40, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.17608837357535395, 'learning_rate': 2.048e-07, 'completion_length': 155.6607208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052032470703125, 'epoch': 0.8} 80%|███████▉ | 1988/2500 [11:53:58<2:58:40, 20.94s/it] 80%|███████▉ | 1989/2500 [11:54:19<2:58:43, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.33657732823937875, 'learning_rate': 2.0439999999999998e-07, 'completion_length': 146.3839340209961, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.00597381591796875, 'epoch': 0.8} 80%|███████▉ | 1989/2500 [11:54:19<2:58:43, 20.98s/it] 80%|███████▉ | 1990/2500 [11:54:40<2:59:41, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.3388926831854403, 'learning_rate': 2.0399999999999997e-07, 'completion_length': 154.41964721679688, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.005767822265625, 'epoch': 0.8} 80%|███████▉ | 1990/2500 [11:54:40<2:59:41, 21.14s/it] 80%|███████▉ | 1991/2500 [11:55:00<2:56:22, 20.79s/it] {'loss': 0.0002, 'grad_norm': 0.19214108514282044, 'learning_rate': 2.036e-07, 'completion_length': 133.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00490570068359375, 'epoch': 0.8} 80%|███████▉ | 1991/2500 [11:55:00<2:56:22, 20.79s/it] 80%|███████▉ | 1992/2500 [11:55:21<2:56:29, 20.85s/it] {'loss': 0.0002, 'grad_norm': 0.024657716533777078, 'learning_rate': 2.032e-07, 'completion_length': 161.3482208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.8} 80%|███████▉ | 1992/2500 [11:55:21<2:56:29, 20.85s/it] 80%|███████▉ | 1993/2500 [11:55:43<2:59:26, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.021558555708361855, 'learning_rate': 2.028e-07, 'completion_length': 162.5982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0068511962890625, 'epoch': 0.8} 80%|███████▉ | 1993/2500 [11:55:43<2:59:26, 21.24s/it] 80%|███████▉ | 1994/2500 [11:56:04<2:57:49, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.020115425093359705, 'learning_rate': 2.0239999999999999e-07, 'completion_length': 149.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004486083984375, 'epoch': 0.8} 80%|███████▉ | 1994/2500 [11:56:04<2:57:49, 21.09s/it] 80%|███████▉ | 1995/2500 [11:56:25<2:57:32, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.01906670209693312, 'learning_rate': 2.02e-07, 'completion_length': 156.10714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004547119140625, 'epoch': 0.8} 80%|███████▉ | 1995/2500 [11:56:25<2:57:32, 21.09s/it] 80%|███████▉ | 1996/2500 [11:56:46<2:55:53, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.2342610933350528, 'learning_rate': 2.016e-07, 'completion_length': 142.55358123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00482177734375, 'epoch': 0.8} 80%|███████▉ | 1996/2500 [11:56:46<2:55:53, 20.94s/it] 80%|███████▉ | 1997/2500 [11:57:07<2:55:55, 20.99s/it] {'loss': 0.0001, 'grad_norm': 0.20247149793075367, 'learning_rate': 2.0119999999999998e-07, 'completion_length': 140.64286041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0030670166015625, 'epoch': 0.8} 80%|███████▉ | 1997/2500 [11:57:07<2:55:55, 20.99s/it] 80%|███████▉ | 1998/2500 [11:57:27<2:54:43, 20.88s/it] {'loss': 0.0001, 'grad_norm': 0.015171776634174176, 'learning_rate': 2.008e-07, 'completion_length': 151.48214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00351715087890625, 'epoch': 0.8} 80%|███████▉ | 1998/2500 [11:57:27<2:54:43, 20.88s/it] 80%|███████▉ | 1999/2500 [11:57:48<2:53:55, 20.83s/it] {'loss': 0.0003, 'grad_norm': 0.028205420567383725, 'learning_rate': 2.004e-07, 'completion_length': 159.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006744384765625, 'epoch': 0.8} 80%|███████▉ | 1999/2500 [11:57:48<2:53:55, 20.83s/it] 80%|████████ | 2000/2500 [11:58:09<2:53:40, 20.84s/it] {'loss': 0.0002, 'grad_norm': 0.8281902554002505, 'learning_rate': 2e-07, 'completion_length': 131.62500762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.06613001227378845, 'kl': 0.00463104248046875, 'epoch': 0.8} 80%|████████ | 2000/2500 [11:58:09<2:53:40, 20.84s/it] 80%|████████ | 2001/2500 [11:59:11<4:35:00, 33.07s/it] {'loss': 0.0002, 'grad_norm': 0.022371471865670838, 'learning_rate': 1.996e-07, 'completion_length': 148.35714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00588226318359375, 'epoch': 0.8} 80%|████████ | 2001/2500 [11:59:11<4:35:00, 33.07s/it] 80%|████████ | 2002/2500 [11:59:32<4:04:32, 29.46s/it] {'loss': 0.0002, 'grad_norm': 0.33739035891017, 'learning_rate': 1.9919999999999998e-07, 'completion_length': 148.89286041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0038604736328125, 'epoch': 0.8} 80%|████████ | 2002/2500 [11:59:32<4:04:32, 29.46s/it] 80%|████████ | 2003/2500 [11:59:52<3:41:01, 26.68s/it] {'loss': 0.0002, 'grad_norm': 0.030482462244684816, 'learning_rate': 1.988e-07, 'completion_length': 143.67858123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00499725341796875, 'epoch': 0.8} 80%|████████ | 2003/2500 [11:59:52<3:41:01, 26.68s/it] 80%|████████ | 2004/2500 [12:00:13<3:26:00, 24.92s/it] {'loss': 0.0002, 'grad_norm': 0.3081201351325494, 'learning_rate': 1.9839999999999998e-07, 'completion_length': 148.42858123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.005767822265625, 'epoch': 0.8} 80%|████████ | 2004/2500 [12:00:13<3:26:00, 24.92s/it] 80%|████████ | 2005/2500 [12:00:34<3:16:48, 23.86s/it] {'loss': 0.0002, 'grad_norm': 0.0305332169413106, 'learning_rate': 1.98e-07, 'completion_length': 156.5446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052642822265625, 'epoch': 0.8} 80%|████████ | 2005/2500 [12:00:34<3:16:48, 23.86s/it] 80%|████████ | 2006/2500 [12:00:55<3:08:46, 22.93s/it] {'loss': 0.0002, 'grad_norm': 0.2559078906677428, 'learning_rate': 1.976e-07, 'completion_length': 147.99108123779297, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0048065185546875, 'epoch': 0.8} 80%|████████ | 2006/2500 [12:00:55<3:08:46, 22.93s/it] 80%|████████ | 2007/2500 [12:01:16<3:04:59, 22.51s/it] {'loss': 0.0003, 'grad_norm': 0.2531467767253707, 'learning_rate': 1.9719999999999997e-07, 'completion_length': 167.3482208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00638580322265625, 'epoch': 0.8} 80%|████████ | 2007/2500 [12:01:16<3:04:59, 22.51s/it] 80%|████████ | 2008/2500 [12:01:36<2:58:20, 21.75s/it] {'loss': 0.0002, 'grad_norm': 0.8338098308491358, 'learning_rate': 1.968e-07, 'completion_length': 141.3482208251953, 'rewards/accuracy_reward': 0.8214286267757416, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.03818017989397049, 'kl': 0.0039825439453125, 'epoch': 0.8} 80%|████████ | 2008/2500 [12:01:36<2:58:20, 21.75s/it] 80%|████████ | 2009/2500 [12:01:57<2:56:06, 21.52s/it] {'loss': 0.0003, 'grad_norm': 1.6603149161502746, 'learning_rate': 1.9639999999999999e-07, 'completion_length': 159.7232208251953, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.10882645100355148, 'kl': 0.0073394775390625, 'epoch': 0.8} 80%|████████ | 2009/2500 [12:01:57<2:56:06, 21.52s/it] 80%|████████ | 2010/2500 [12:02:19<2:55:42, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.91302372270322, 'learning_rate': 1.96e-07, 'completion_length': 158.68750762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.0063018798828125, 'epoch': 0.8} 80%|████████ | 2010/2500 [12:02:19<2:55:42, 21.52s/it] 80%|████████ | 2011/2500 [12:02:40<2:54:30, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.49714505013440774, 'learning_rate': 1.9559999999999998e-07, 'completion_length': 152.26786041259766, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0055999755859375, 'epoch': 0.8} 80%|████████ | 2011/2500 [12:02:40<2:54:30, 21.41s/it] 80%|████████ | 2012/2500 [12:03:01<2:53:03, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.3930990607503492, 'learning_rate': 1.952e-07, 'completion_length': 159.8214340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00795745849609375, 'epoch': 0.8} 80%|████████ | 2012/2500 [12:03:01<2:53:03, 21.28s/it] 81%|████████ | 2013/2500 [12:03:22<2:51:44, 21.16s/it] {'loss': 0.0001, 'grad_norm': 0.313915485672867, 'learning_rate': 1.948e-07, 'completion_length': 154.00000762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00348663330078125, 'epoch': 0.81} 81%|████████ | 2013/2500 [12:03:22<2:51:44, 21.16s/it] 81%|████████ | 2014/2500 [12:03:42<2:49:39, 20.94s/it] {'loss': 0.0001, 'grad_norm': 0.4801453423068563, 'learning_rate': 1.944e-07, 'completion_length': 144.9464340209961, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.00372314453125, 'epoch': 0.81} 81%|████████ | 2014/2500 [12:03:42<2:49:39, 20.94s/it] 81%|████████ | 2015/2500 [12:04:03<2:48:58, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.2817018149629671, 'learning_rate': 1.94e-07, 'completion_length': 148.2678680419922, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0045928955078125, 'epoch': 0.81} 81%|████████ | 2015/2500 [12:04:03<2:48:58, 20.90s/it] 81%|████████ | 2016/2500 [12:04:24<2:47:52, 20.81s/it] {'loss': 0.0001, 'grad_norm': 0.03346019427447101, 'learning_rate': 1.9359999999999999e-07, 'completion_length': 144.74108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003631591796875, 'epoch': 0.81} 81%|████████ | 2016/2500 [12:04:24<2:47:52, 20.81s/it] 81%|████████ | 2017/2500 [12:04:45<2:47:59, 20.87s/it] {'loss': 0.0003, 'grad_norm': 0.9194871391402432, 'learning_rate': 1.932e-07, 'completion_length': 166.99108123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.0080718994140625, 'epoch': 0.81} 81%|████████ | 2017/2500 [12:04:45<2:47:59, 20.87s/it] 81%|████████ | 2018/2500 [12:05:06<2:47:53, 20.90s/it] {'loss': 0.0003, 'grad_norm': 0.34029298142396686, 'learning_rate': 1.9279999999999998e-07, 'completion_length': 163.6696548461914, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0069122314453125, 'epoch': 0.81} 81%|████████ | 2018/2500 [12:05:06<2:47:53, 20.90s/it] 81%|████████ | 2019/2500 [12:05:27<2:48:14, 20.99s/it] {'loss': 0.0003, 'grad_norm': 0.7067928679107841, 'learning_rate': 1.9239999999999998e-07, 'completion_length': 165.68750762939453, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.007598876953125, 'epoch': 0.81} 81%|████████ | 2019/2500 [12:05:27<2:48:14, 20.99s/it] 81%|████████ | 2020/2500 [12:05:48<2:48:14, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.0248262528449401, 'learning_rate': 1.92e-07, 'completion_length': 163.4821548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0057220458984375, 'epoch': 0.81} 81%|████████ | 2020/2500 [12:05:48<2:48:14, 21.03s/it] 81%|████████ | 2021/2500 [12:06:09<2:48:31, 21.11s/it] {'loss': 0.0001, 'grad_norm': 0.02064391246763755, 'learning_rate': 1.916e-07, 'completion_length': 151.40179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032501220703125, 'epoch': 0.81} 81%|████████ | 2021/2500 [12:06:09<2:48:31, 21.11s/it] 81%|████████ | 2022/2500 [12:06:30<2:48:23, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.017393285638096407, 'learning_rate': 1.912e-07, 'completion_length': 154.4464340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0058441162109375, 'epoch': 0.81} 81%|████████ | 2022/2500 [12:06:30<2:48:23, 21.14s/it] 81%|████████ | 2023/2500 [12:06:52<2:48:02, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.018256739191052793, 'learning_rate': 1.908e-07, 'completion_length': 153.30358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00457000732421875, 'epoch': 0.81} 81%|████████ | 2023/2500 [12:06:52<2:48:02, 21.14s/it] 81%|████████ | 2024/2500 [12:07:13<2:48:15, 21.21s/it] {'loss': 0.0003, 'grad_norm': 1.0464113893567202, 'learning_rate': 1.904e-07, 'completion_length': 153.70536041259766, 'rewards/accuracy_reward': 0.8660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.10882644355297089, 'kl': 0.00836181640625, 'epoch': 0.81} 81%|████████ | 2024/2500 [12:07:13<2:48:15, 21.21s/it] 81%|████████ | 2025/2500 [12:07:34<2:48:00, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.5870663550471085, 'learning_rate': 1.8999999999999998e-07, 'completion_length': 163.58929443359375, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0043792724609375, 'epoch': 0.81} 81%|████████ | 2025/2500 [12:07:34<2:48:00, 21.22s/it] 81%|████████ | 2026/2500 [12:07:56<2:48:38, 21.35s/it] {'loss': 0.0004, 'grad_norm': 0.038297982681083975, 'learning_rate': 1.8959999999999998e-07, 'completion_length': 160.71428680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.010101318359375, 'epoch': 0.81} 81%|████████ | 2026/2500 [12:07:56<2:48:38, 21.35s/it] 81%|████████ | 2027/2500 [12:08:16<2:46:20, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.019961289899559437, 'learning_rate': 1.892e-07, 'completion_length': 148.26786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0041046142578125, 'epoch': 0.81} 81%|████████ | 2027/2500 [12:08:16<2:46:20, 21.10s/it] 81%|████████ | 2028/2500 [12:08:38<2:47:12, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.015685742538125026, 'learning_rate': 1.888e-07, 'completion_length': 151.67858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0041351318359375, 'epoch': 0.81} 81%|████████ | 2028/2500 [12:08:38<2:47:12, 21.26s/it] 81%|████████ | 2029/2500 [12:08:59<2:45:20, 21.06s/it] {'loss': 0.0002, 'grad_norm': 1.065523407794595, 'learning_rate': 1.884e-07, 'completion_length': 135.51786041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004302978515625, 'epoch': 0.81} 81%|████████ | 2029/2500 [12:08:59<2:45:20, 21.06s/it] 81%|████████ | 2030/2500 [12:09:19<2:44:34, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.01673690258100063, 'learning_rate': 1.88e-07, 'completion_length': 146.30358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005035400390625, 'epoch': 0.81} 81%|████████ | 2030/2500 [12:09:19<2:44:34, 21.01s/it] 81%|████████ | 2031/2500 [12:09:40<2:43:58, 20.98s/it] {'loss': 0.0003, 'grad_norm': 0.3796691069865311, 'learning_rate': 1.8759999999999999e-07, 'completion_length': 139.80357360839844, 'rewards/accuracy_reward': 0.8125000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8125000596046448, 'reward_std': 0.08747542649507523, 'kl': 0.0063018798828125, 'epoch': 0.81} 81%|████████ | 2031/2500 [12:09:40<2:43:58, 20.98s/it] 81%|████████▏ | 2032/2500 [12:10:01<2:43:27, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.2979814855810502, 'learning_rate': 1.872e-07, 'completion_length': 147.6607208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.004974365234375, 'epoch': 0.81} 81%|████████▏ | 2032/2500 [12:10:01<2:43:27, 20.96s/it] 81%|████████▏ | 2033/2500 [12:10:22<2:42:56, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.018206500886950062, 'learning_rate': 1.8679999999999998e-07, 'completion_length': 148.0714340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049896240234375, 'epoch': 0.81} 81%|████████▏ | 2033/2500 [12:10:22<2:42:56, 20.94s/it] 81%|████████▏ | 2034/2500 [12:10:44<2:44:35, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.40012968108209257, 'learning_rate': 1.864e-07, 'completion_length': 158.9553680419922, 'rewards/accuracy_reward': 0.8928571939468384, 'rewards/format_reward': 1.0, 'reward': 1.8928572535514832, 'reward_std': 0.06222161278128624, 'kl': 0.0051727294921875, 'epoch': 0.81} 81%|████████▏ | 2034/2500 [12:10:44<2:44:35, 21.19s/it] 81%|████████▏ | 2035/2500 [12:11:05<2:43:51, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.3318705120328766, 'learning_rate': 1.86e-07, 'completion_length': 148.17858123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0048828125, 'epoch': 0.81} 81%|████████▏ | 2035/2500 [12:11:05<2:43:51, 21.14s/it] 81%|████████▏ | 2036/2500 [12:11:26<2:43:58, 21.20s/it] {'loss': 0.0001, 'grad_norm': 0.025250664522224516, 'learning_rate': 1.8559999999999997e-07, 'completion_length': 136.93750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0034942626953125, 'epoch': 0.81} 81%|████████▏ | 2036/2500 [12:11:26<2:43:58, 21.20s/it] 81%|████████▏ | 2037/2500 [12:11:48<2:43:37, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.04001730610041982, 'learning_rate': 1.852e-07, 'completion_length': 151.6428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00383758544921875, 'epoch': 0.81} 81%|████████▏ | 2037/2500 [12:11:48<2:43:37, 21.20s/it] 82%|████████▏ | 2038/2500 [12:12:08<2:42:33, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.38072633158080327, 'learning_rate': 1.848e-07, 'completion_length': 144.75000762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0053558349609375, 'epoch': 0.82} 82%|████████▏ | 2038/2500 [12:12:08<2:42:33, 21.11s/it] 82%|████████▏ | 2039/2500 [12:12:29<2:41:43, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.4909030518051189, 'learning_rate': 1.844e-07, 'completion_length': 156.05358123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.10882644355297089, 'kl': 0.0059814453125, 'epoch': 0.82} 82%|████████▏ | 2039/2500 [12:12:29<2:41:43, 21.05s/it] 82%|████████▏ | 2040/2500 [12:12:50<2:41:38, 21.08s/it] {'loss': 0.0003, 'grad_norm': 0.3510799065248996, 'learning_rate': 1.8399999999999998e-07, 'completion_length': 148.77679443359375, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00732421875, 'epoch': 0.82} 82%|████████▏ | 2040/2500 [12:12:50<2:41:38, 21.08s/it] 82%|████████▏ | 2041/2500 [12:13:12<2:41:12, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.2170957704227599, 'learning_rate': 1.836e-07, 'completion_length': 151.54464721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0054779052734375, 'epoch': 0.82} 82%|████████▏ | 2041/2500 [12:13:12<2:41:12, 21.07s/it] 82%|████████▏ | 2042/2500 [12:13:33<2:41:50, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.017079099244376655, 'learning_rate': 1.832e-07, 'completion_length': 154.9107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00525665283203125, 'epoch': 0.82} 82%|████████▏ | 2042/2500 [12:13:33<2:41:50, 21.20s/it] 82%|████████▏ | 2043/2500 [12:13:54<2:41:36, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.3198927425460645, 'learning_rate': 1.8279999999999997e-07, 'completion_length': 162.9107208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0065765380859375, 'epoch': 0.82} 82%|████████▏ | 2043/2500 [12:13:54<2:41:36, 21.22s/it] 82%|████████▏ | 2044/2500 [12:14:15<2:40:18, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.026577141976374452, 'learning_rate': 1.824e-07, 'completion_length': 148.83929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055999755859375, 'epoch': 0.82} 82%|████████▏ | 2044/2500 [12:14:15<2:40:18, 21.09s/it] 82%|████████▏ | 2045/2500 [12:14:37<2:41:42, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.4186444410057901, 'learning_rate': 1.82e-07, 'completion_length': 148.4107208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.005401611328125, 'epoch': 0.82} 82%|████████▏ | 2045/2500 [12:14:37<2:41:42, 21.32s/it] 82%|████████▏ | 2046/2500 [12:14:58<2:40:44, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.412118755728727, 'learning_rate': 1.816e-07, 'completion_length': 146.5803680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059814453125, 'epoch': 0.82} 82%|████████▏ | 2046/2500 [12:14:58<2:40:44, 21.24s/it] 82%|████████▏ | 2047/2500 [12:15:19<2:40:09, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.01576644657153484, 'learning_rate': 1.8119999999999998e-07, 'completion_length': 161.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0050048828125, 'epoch': 0.82} 82%|████████▏ | 2047/2500 [12:15:19<2:40:09, 21.21s/it] 82%|████████▏ | 2048/2500 [12:15:40<2:38:50, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.20668359465969155, 'learning_rate': 1.8079999999999998e-07, 'completion_length': 144.68750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00494384765625, 'epoch': 0.82} 82%|████████▏ | 2048/2500 [12:15:40<2:38:50, 21.08s/it] 82%|████████▏ | 2049/2500 [12:16:01<2:38:49, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.3808905024293788, 'learning_rate': 1.804e-07, 'completion_length': 158.8928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0051422119140625, 'epoch': 0.82} 82%|████████▏ | 2049/2500 [12:16:01<2:38:49, 21.13s/it] 82%|████████▏ | 2050/2500 [12:16:22<2:37:51, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.21875879101731885, 'learning_rate': 1.8e-07, 'completion_length': 145.4732208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0059967041015625, 'epoch': 0.82} 82%|████████▏ | 2050/2500 [12:16:22<2:37:51, 21.05s/it] 82%|████████▏ | 2051/2500 [12:16:43<2:37:08, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.024715660086914514, 'learning_rate': 1.796e-07, 'completion_length': 153.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0059661865234375, 'epoch': 0.82} 82%|████████▏ | 2051/2500 [12:16:43<2:37:08, 21.00s/it] 82%|████████▏ | 2052/2500 [12:17:03<2:35:48, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.019579464048710894, 'learning_rate': 1.792e-07, 'completion_length': 152.77679443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0057525634765625, 'epoch': 0.82} 82%|████████▏ | 2052/2500 [12:17:03<2:35:48, 20.87s/it] 82%|████████▏ | 2053/2500 [12:17:25<2:36:18, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.01713257553649797, 'learning_rate': 1.7879999999999999e-07, 'completion_length': 157.6964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055389404296875, 'epoch': 0.82} 82%|████████▏ | 2053/2500 [12:17:25<2:36:18, 20.98s/it] 82%|████████▏ | 2054/2500 [12:17:46<2:35:33, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.02550194137868142, 'learning_rate': 1.7839999999999998e-07, 'completion_length': 143.45536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00415802001953125, 'epoch': 0.82} 82%|████████▏ | 2054/2500 [12:17:46<2:35:33, 20.93s/it] 82%|████████▏ | 2055/2500 [12:18:07<2:36:07, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.298407192070569, 'learning_rate': 1.7799999999999998e-07, 'completion_length': 142.71429061889648, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004364013671875, 'epoch': 0.82} 82%|████████▏ | 2055/2500 [12:18:07<2:36:07, 21.05s/it] 82%|████████▏ | 2056/2500 [12:18:27<2:34:37, 20.90s/it] {'loss': 0.0001, 'grad_norm': 0.02448197454095598, 'learning_rate': 1.776e-07, 'completion_length': 148.33929443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00373077392578125, 'epoch': 0.82} 82%|████████▏ | 2056/2500 [12:18:27<2:34:37, 20.90s/it] 82%|████████▏ | 2057/2500 [12:18:48<2:34:11, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.04993447366509451, 'learning_rate': 1.772e-07, 'completion_length': 151.0089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00414276123046875, 'epoch': 0.82} 82%|████████▏ | 2057/2500 [12:18:48<2:34:11, 20.88s/it] 82%|████████▏ | 2058/2500 [12:19:11<2:39:00, 21.59s/it] {'loss': 0.0002, 'grad_norm': 0.019475384150384353, 'learning_rate': 1.768e-07, 'completion_length': 158.19644165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055999755859375, 'epoch': 0.82} 82%|████████▏ | 2058/2500 [12:19:11<2:39:00, 21.59s/it] 82%|████████▏ | 2059/2500 [12:19:32<2:37:11, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.36988282354461016, 'learning_rate': 1.764e-07, 'completion_length': 160.5089340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006439208984375, 'epoch': 0.82} 82%|████████▏ | 2059/2500 [12:19:32<2:37:11, 21.39s/it] 82%|████████▏ | 2060/2500 [12:19:53<2:36:00, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.015437975489952111, 'learning_rate': 1.76e-07, 'completion_length': 149.55358123779297, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00461578369140625, 'epoch': 0.82} 82%|████████▏ | 2060/2500 [12:19:53<2:36:00, 21.27s/it] 82%|████████▏ | 2061/2500 [12:20:14<2:34:47, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.25938344519129963, 'learning_rate': 1.756e-07, 'completion_length': 146.5357208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005340576171875, 'epoch': 0.82} 82%|████████▏ | 2061/2500 [12:20:14<2:34:47, 21.16s/it] 82%|████████▏ | 2062/2500 [12:20:35<2:33:38, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.020840946920855627, 'learning_rate': 1.7519999999999998e-07, 'completion_length': 150.18750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004608154296875, 'epoch': 0.82} 82%|████████▏ | 2062/2500 [12:20:35<2:33:38, 21.05s/it] 83%|████████▎ | 2063/2500 [12:20:56<2:32:36, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.03105488232612288, 'learning_rate': 1.748e-07, 'completion_length': 143.9821548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00426483154296875, 'epoch': 0.83} 83%|████████▎ | 2063/2500 [12:20:56<2:32:36, 20.95s/it] 83%|████████▎ | 2064/2500 [12:21:17<2:31:56, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.2734970512405316, 'learning_rate': 1.744e-07, 'completion_length': 154.7946548461914, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0046234130859375, 'epoch': 0.83} 83%|████████▎ | 2064/2500 [12:21:17<2:31:56, 20.91s/it] 83%|████████▎ | 2065/2500 [12:21:38<2:32:53, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.43735948982168266, 'learning_rate': 1.7399999999999997e-07, 'completion_length': 162.9196548461914, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.00494384765625, 'epoch': 0.83} 83%|████████▎ | 2065/2500 [12:21:38<2:32:53, 21.09s/it] 83%|████████▎ | 2066/2500 [12:21:59<2:32:23, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.5283157619787314, 'learning_rate': 1.736e-07, 'completion_length': 155.76786041259766, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0053253173828125, 'epoch': 0.83} 83%|████████▎ | 2066/2500 [12:21:59<2:32:23, 21.07s/it] 83%|████████▎ | 2067/2500 [12:22:20<2:30:55, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.028499966691857018, 'learning_rate': 1.732e-07, 'completion_length': 156.4553680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00443267822265625, 'epoch': 0.83} 83%|████████▎ | 2067/2500 [12:22:20<2:30:55, 20.91s/it] 83%|████████▎ | 2068/2500 [12:22:41<2:31:23, 21.03s/it] {'loss': 0.0003, 'grad_norm': 0.03382107463737766, 'learning_rate': 1.728e-07, 'completion_length': 156.7857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0072479248046875, 'epoch': 0.83} 83%|████████▎ | 2068/2500 [12:22:41<2:31:23, 21.03s/it] 83%|████████▎ | 2069/2500 [12:23:02<2:30:33, 20.96s/it] {'loss': 0.0001, 'grad_norm': 0.5616073299934758, 'learning_rate': 1.7239999999999998e-07, 'completion_length': 145.3303680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.00353240966796875, 'epoch': 0.83} 83%|████████▎ | 2069/2500 [12:23:02<2:30:33, 20.96s/it] 83%|████████▎ | 2070/2500 [12:23:23<2:30:58, 21.07s/it] {'loss': 0.0001, 'grad_norm': 0.014872239453729716, 'learning_rate': 1.7199999999999998e-07, 'completion_length': 148.3303680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00365447998046875, 'epoch': 0.83} 83%|████████▎ | 2070/2500 [12:23:23<2:30:58, 21.07s/it] 83%|████████▎ | 2071/2500 [12:23:44<2:29:36, 20.92s/it] {'loss': 0.0001, 'grad_norm': 0.5951686250461459, 'learning_rate': 1.716e-07, 'completion_length': 144.12500762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.07124518603086472, 'kl': 0.00354766845703125, 'epoch': 0.83} 83%|████████▎ | 2071/2500 [12:23:44<2:29:36, 20.92s/it] 83%|████████▎ | 2072/2500 [12:24:05<2:30:15, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.024281692472425898, 'learning_rate': 1.7119999999999997e-07, 'completion_length': 150.2678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052947998046875, 'epoch': 0.83} 83%|████████▎ | 2072/2500 [12:24:05<2:30:15, 21.07s/it] 83%|████████▎ | 2073/2500 [12:24:26<2:30:14, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.44708055260844687, 'learning_rate': 1.708e-07, 'completion_length': 151.1071548461914, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0062713623046875, 'epoch': 0.83} 83%|████████▎ | 2073/2500 [12:24:26<2:30:14, 21.11s/it] 83%|████████▎ | 2074/2500 [12:24:48<2:31:00, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.28861543341249807, 'learning_rate': 1.704e-07, 'completion_length': 156.81250762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064544677734375, 'epoch': 0.83} 83%|████████▎ | 2074/2500 [12:24:48<2:31:00, 21.27s/it] 83%|████████▎ | 2075/2500 [12:25:09<2:30:55, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.45907794925061024, 'learning_rate': 1.7000000000000001e-07, 'completion_length': 156.6964340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00555419921875, 'epoch': 0.83} 83%|████████▎ | 2075/2500 [12:25:09<2:30:55, 21.31s/it] 83%|████████▎ | 2076/2500 [12:25:30<2:29:19, 21.13s/it] {'loss': 0.0001, 'grad_norm': 0.020562662552511955, 'learning_rate': 1.6959999999999998e-07, 'completion_length': 145.4196548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00276947021484375, 'epoch': 0.83} 83%|████████▎ | 2076/2500 [12:25:30<2:29:19, 21.13s/it] 83%|████████▎ | 2077/2500 [12:25:51<2:27:39, 20.94s/it] {'loss': 0.0001, 'grad_norm': 0.015066352971944138, 'learning_rate': 1.6919999999999998e-07, 'completion_length': 148.91964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0034637451171875, 'epoch': 0.83} 83%|████████▎ | 2077/2500 [12:25:51<2:27:39, 20.94s/it] 83%|████████▎ | 2078/2500 [12:26:12<2:27:40, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.02126798204213158, 'learning_rate': 1.688e-07, 'completion_length': 150.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00411224365234375, 'epoch': 0.83} 83%|████████▎ | 2078/2500 [12:26:12<2:27:40, 21.00s/it] 83%|████████▎ | 2079/2500 [12:26:32<2:26:23, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.17352930862608146, 'learning_rate': 1.684e-07, 'completion_length': 150.24107360839844, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00525665283203125, 'epoch': 0.83} 83%|████████▎ | 2079/2500 [12:26:32<2:26:23, 20.86s/it] 83%|████████▎ | 2080/2500 [12:26:53<2:26:30, 20.93s/it] {'loss': 0.0002, 'grad_norm': 0.017424104376758058, 'learning_rate': 1.68e-07, 'completion_length': 160.7857208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003814697265625, 'epoch': 0.83} 83%|████████▎ | 2080/2500 [12:26:53<2:26:30, 20.93s/it] 83%|████████▎ | 2081/2500 [12:27:15<2:27:33, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.015691606723750653, 'learning_rate': 1.676e-07, 'completion_length': 172.7946548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00418853759765625, 'epoch': 0.83} 83%|████████▎ | 2081/2500 [12:27:15<2:27:33, 21.13s/it] 83%|████████▎ | 2082/2500 [12:27:36<2:26:44, 21.06s/it] {'loss': 0.0001, 'grad_norm': 0.016880224140042522, 'learning_rate': 1.672e-07, 'completion_length': 145.2053680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00360107421875, 'epoch': 0.83} 83%|████████▎ | 2082/2500 [12:27:36<2:26:44, 21.06s/it] 83%|████████▎ | 2083/2500 [12:27:57<2:26:27, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.015696427067660244, 'learning_rate': 1.6679999999999998e-07, 'completion_length': 136.41964721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0038299560546875, 'epoch': 0.83} 83%|████████▎ | 2083/2500 [12:27:57<2:26:27, 21.07s/it] 83%|████████▎ | 2084/2500 [12:28:18<2:26:35, 21.14s/it] {'loss': 0.0004, 'grad_norm': 0.7562918328347122, 'learning_rate': 1.6639999999999998e-07, 'completion_length': 156.00000762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.01019287109375, 'epoch': 0.83} 83%|████████▎ | 2084/2500 [12:28:18<2:26:35, 21.14s/it] 83%|████████▎ | 2085/2500 [12:28:39<2:25:17, 21.01s/it] {'loss': 0.0001, 'grad_norm': 0.023422447349719346, 'learning_rate': 1.66e-07, 'completion_length': 140.50000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003204345703125, 'epoch': 0.83} 83%|████████▎ | 2085/2500 [12:28:39<2:25:17, 21.01s/it] 83%|████████▎ | 2086/2500 [12:29:01<2:27:08, 21.33s/it] {'loss': 0.0002, 'grad_norm': 1.2789714988173564, 'learning_rate': 1.656e-07, 'completion_length': 156.93750762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0054779052734375, 'epoch': 0.83} 83%|████████▎ | 2086/2500 [12:29:01<2:27:08, 21.33s/it] 83%|████████▎ | 2087/2500 [12:29:22<2:26:28, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.2950731153627429, 'learning_rate': 1.652e-07, 'completion_length': 161.2678680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00545501708984375, 'epoch': 0.83} 83%|████████▎ | 2087/2500 [12:29:22<2:26:28, 21.28s/it] 84%|████████▎ | 2088/2500 [12:29:44<2:26:48, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.31152741138138396, 'learning_rate': 1.648e-07, 'completion_length': 151.1071548461914, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.006103515625, 'epoch': 0.84} 84%|████████▎ | 2088/2500 [12:29:44<2:26:48, 21.38s/it] 84%|████████▎ | 2089/2500 [12:30:05<2:25:20, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.3292361613278902, 'learning_rate': 1.644e-07, 'completion_length': 152.7589340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0053558349609375, 'epoch': 0.84} 84%|████████▎ | 2089/2500 [12:30:05<2:25:20, 21.22s/it] 84%|████████▎ | 2090/2500 [12:30:26<2:24:36, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.021512873776847248, 'learning_rate': 1.64e-07, 'completion_length': 149.08036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00458526611328125, 'epoch': 0.84} 84%|████████▎ | 2090/2500 [12:30:26<2:24:36, 21.16s/it] 84%|████████▎ | 2091/2500 [12:30:48<2:27:14, 21.60s/it] {'loss': 0.0003, 'grad_norm': 0.9158069021015146, 'learning_rate': 1.6359999999999998e-07, 'completion_length': 164.3571548461914, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.09919501841068268, 'kl': 0.00677490234375, 'epoch': 0.84} 84%|████████▎ | 2091/2500 [12:30:48<2:27:14, 21.60s/it] 84%|████████▎ | 2092/2500 [12:31:10<2:27:12, 21.65s/it] {'loss': 0.0002, 'grad_norm': 0.679252871079704, 'learning_rate': 1.632e-07, 'completion_length': 158.70536041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.07576143741607666, 'kl': 0.006011962890625, 'epoch': 0.84} 84%|████████▎ | 2092/2500 [12:31:10<2:27:12, 21.65s/it] 84%|████████▎ | 2093/2500 [12:31:32<2:27:10, 21.70s/it] {'loss': 0.0002, 'grad_norm': 0.3689571171740234, 'learning_rate': 1.628e-07, 'completion_length': 152.58036041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0050506591796875, 'epoch': 0.84} 84%|████████▎ | 2093/2500 [12:31:32<2:27:10, 21.70s/it] 84%|████████▍ | 2094/2500 [12:31:53<2:24:53, 21.41s/it] {'loss': 0.0002, 'grad_norm': 0.5750711504706532, 'learning_rate': 1.6239999999999997e-07, 'completion_length': 138.9821548461914, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831882357597351, 'kl': 0.00431060791015625, 'epoch': 0.84} 84%|████████▍ | 2094/2500 [12:31:53<2:24:53, 21.41s/it] 84%|████████▍ | 2095/2500 [12:32:15<2:26:26, 21.70s/it] {'loss': 0.0002, 'grad_norm': 0.41947933727819325, 'learning_rate': 1.62e-07, 'completion_length': 146.35714721679688, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0058135986328125, 'epoch': 0.84} 84%|████████▍ | 2095/2500 [12:32:15<2:26:26, 21.70s/it] 84%|████████▍ | 2096/2500 [12:32:38<2:28:31, 22.06s/it] {'loss': 0.0003, 'grad_norm': 0.20862828025824875, 'learning_rate': 1.616e-07, 'completion_length': 165.1428680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0063323974609375, 'epoch': 0.84} 84%|████████▍ | 2096/2500 [12:32:38<2:28:31, 22.06s/it] 84%|████████▍ | 2097/2500 [12:33:00<2:28:20, 22.09s/it] {'loss': 0.0003, 'grad_norm': 0.022368580517605367, 'learning_rate': 1.6120000000000001e-07, 'completion_length': 152.75000762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.007293701171875, 'epoch': 0.84} 84%|████████▍ | 2097/2500 [12:33:00<2:28:20, 22.09s/it] 84%|████████▍ | 2098/2500 [12:33:21<2:26:27, 21.86s/it] {'loss': 0.0002, 'grad_norm': 0.37075125635856, 'learning_rate': 1.6079999999999998e-07, 'completion_length': 150.35714721679688, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.07003280520439148, 'kl': 0.0053253173828125, 'epoch': 0.84} 84%|████████▍ | 2098/2500 [12:33:21<2:26:27, 21.86s/it] 84%|████████▍ | 2099/2500 [12:33:43<2:25:40, 21.80s/it] {'loss': 0.0003, 'grad_norm': 0.03640080235739907, 'learning_rate': 1.6039999999999998e-07, 'completion_length': 162.52679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00823974609375, 'epoch': 0.84} 84%|████████▍ | 2099/2500 [12:33:43<2:25:40, 21.80s/it] 84%|████████▍ | 2100/2500 [12:34:04<2:23:39, 21.55s/it] {'loss': 0.0003, 'grad_norm': 0.21973686070151835, 'learning_rate': 1.6e-07, 'completion_length': 149.02679443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0068359375, 'epoch': 0.84} 84%|████████▍ | 2100/2500 [12:34:04<2:23:39, 21.55s/it] 84%|████████▍ | 2101/2500 [12:35:14<4:00:04, 36.10s/it] {'loss': 0.0003, 'grad_norm': 0.33100421049291384, 'learning_rate': 1.5959999999999997e-07, 'completion_length': 155.0089340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.033065006136894226, 'kl': 0.00762939453125, 'epoch': 0.84} 84%|████████▍ | 2101/2500 [12:35:14<4:00:04, 36.10s/it] 84%|████████▍ | 2102/2500 [12:35:36<3:30:42, 31.76s/it] {'loss': 0.0003, 'grad_norm': 0.5792677447826832, 'learning_rate': 1.592e-07, 'completion_length': 165.73214721679688, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.05831881985068321, 'kl': 0.00787353515625, 'epoch': 0.84} 84%|████████▍ | 2102/2500 [12:35:36<3:30:42, 31.76s/it] 84%|████████▍ | 2103/2500 [12:35:57<3:09:25, 28.63s/it] {'loss': 0.0002, 'grad_norm': 0.025054474330003, 'learning_rate': 1.588e-07, 'completion_length': 156.8214340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00531005859375, 'epoch': 0.84} 84%|████████▍ | 2103/2500 [12:35:57<3:09:25, 28.63s/it] 84%|████████▍ | 2104/2500 [12:36:18<2:54:29, 26.44s/it] {'loss': 0.0003, 'grad_norm': 0.44079039041762946, 'learning_rate': 1.5840000000000002e-07, 'completion_length': 151.2232208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.006866455078125, 'epoch': 0.84} 84%|████████▍ | 2104/2500 [12:36:18<2:54:29, 26.44s/it] 84%|████████▍ | 2105/2500 [12:36:40<2:44:16, 24.95s/it] {'loss': 0.0002, 'grad_norm': 0.6422051322574663, 'learning_rate': 1.5799999999999999e-07, 'completion_length': 148.58928680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.004913330078125, 'epoch': 0.84} 84%|████████▍ | 2105/2500 [12:36:40<2:44:16, 24.95s/it] 84%|████████▍ | 2106/2500 [12:37:01<2:36:25, 23.82s/it] {'loss': 0.0002, 'grad_norm': 0.2319386660942285, 'learning_rate': 1.5759999999999998e-07, 'completion_length': 154.83929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00582122802734375, 'epoch': 0.84} 84%|████████▍ | 2106/2500 [12:37:01<2:36:25, 23.82s/it] 84%|████████▍ | 2107/2500 [12:37:23<2:32:19, 23.26s/it] {'loss': 0.0002, 'grad_norm': 0.44886154810247453, 'learning_rate': 1.572e-07, 'completion_length': 165.4732208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.006195068359375, 'epoch': 0.84} 84%|████████▍ | 2107/2500 [12:37:23<2:32:19, 23.26s/it] 84%|████████▍ | 2108/2500 [12:37:44<2:27:38, 22.60s/it] {'loss': 0.0001, 'grad_norm': 0.01338360335561512, 'learning_rate': 1.5679999999999997e-07, 'completion_length': 145.33928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0025787353515625, 'epoch': 0.84} 84%|████████▍ | 2108/2500 [12:37:44<2:27:38, 22.60s/it] 84%|████████▍ | 2109/2500 [12:38:05<2:24:31, 22.18s/it] {'loss': 0.0002, 'grad_norm': 1.0675522061593634, 'learning_rate': 1.564e-07, 'completion_length': 152.50000762939453, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.00417327880859375, 'epoch': 0.84} 84%|████████▍ | 2109/2500 [12:38:05<2:24:31, 22.18s/it] 84%|████████▍ | 2110/2500 [12:38:26<2:22:05, 21.86s/it] {'loss': 0.0002, 'grad_norm': 0.01885201537621694, 'learning_rate': 1.56e-07, 'completion_length': 155.99108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004730224609375, 'epoch': 0.84} 84%|████████▍ | 2110/2500 [12:38:26<2:22:05, 21.86s/it] 84%|████████▍ | 2111/2500 [12:38:47<2:19:09, 21.46s/it] {'loss': 0.0001, 'grad_norm': 0.011702688940679844, 'learning_rate': 1.556e-07, 'completion_length': 136.80358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0023345947265625, 'epoch': 0.84} 84%|████████▍ | 2111/2500 [12:38:47<2:19:09, 21.46s/it] 84%|████████▍ | 2112/2500 [12:39:08<2:17:45, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.22782251870301073, 'learning_rate': 1.552e-07, 'completion_length': 144.6964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0053253173828125, 'epoch': 0.84} 84%|████████▍ | 2112/2500 [12:39:08<2:17:45, 21.30s/it] 85%|████████▍ | 2113/2500 [12:39:29<2:17:14, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.2663120700260152, 'learning_rate': 1.5479999999999998e-07, 'completion_length': 162.4553680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005767822265625, 'epoch': 0.85} 85%|████████▍ | 2113/2500 [12:39:29<2:17:14, 21.28s/it] 85%|████████▍ | 2114/2500 [12:39:50<2:16:53, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.3508570925811249, 'learning_rate': 1.544e-07, 'completion_length': 167.70536041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0058441162109375, 'epoch': 0.85} 85%|████████▍ | 2114/2500 [12:39:50<2:16:53, 21.28s/it] 85%|████████▍ | 2115/2500 [12:40:11<2:15:54, 21.18s/it] {'loss': 0.0001, 'grad_norm': 0.2721318845461887, 'learning_rate': 1.54e-07, 'completion_length': 139.12500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00310516357421875, 'epoch': 0.85} 85%|████████▍ | 2115/2500 [12:40:11<2:15:54, 21.18s/it] 85%|████████▍ | 2116/2500 [12:40:33<2:15:53, 21.23s/it] {'loss': 0.0001, 'grad_norm': 0.025617928800218532, 'learning_rate': 1.5359999999999997e-07, 'completion_length': 146.02678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032501220703125, 'epoch': 0.85} 85%|████████▍ | 2116/2500 [12:40:33<2:15:53, 21.23s/it] 85%|████████▍ | 2117/2500 [12:40:54<2:16:06, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.598277147316783, 'learning_rate': 1.532e-07, 'completion_length': 143.1696548461914, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00439453125, 'epoch': 0.85} 85%|████████▍ | 2117/2500 [12:40:54<2:16:06, 21.32s/it] 85%|████████▍ | 2118/2500 [12:41:15<2:14:49, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.3175085090254208, 'learning_rate': 1.528e-07, 'completion_length': 145.86608123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0062103271484375, 'epoch': 0.85} 85%|████████▍ | 2118/2500 [12:41:15<2:14:49, 21.18s/it] 85%|████████▍ | 2119/2500 [12:41:37<2:15:28, 21.33s/it] {'loss': 0.0003, 'grad_norm': 0.6935450722612214, 'learning_rate': 1.524e-07, 'completion_length': 162.80358123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0065460205078125, 'epoch': 0.85} 85%|████████▍ | 2119/2500 [12:41:37<2:15:28, 21.33s/it] 85%|████████▍ | 2120/2500 [12:41:57<2:13:44, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.01537271733912188, 'learning_rate': 1.5199999999999998e-07, 'completion_length': 148.1785774230957, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0072174072265625, 'epoch': 0.85} 85%|████████▍ | 2120/2500 [12:41:57<2:13:44, 21.12s/it] 85%|████████▍ | 2121/2500 [12:42:18<2:12:03, 20.91s/it] {'loss': 0.0002, 'grad_norm': 0.9912971203891522, 'learning_rate': 1.516e-07, 'completion_length': 133.54464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00450897216796875, 'epoch': 0.85} 85%|████████▍ | 2121/2500 [12:42:18<2:12:03, 20.91s/it] 85%|████████▍ | 2122/2500 [12:42:39<2:12:10, 20.98s/it] {'loss': 0.0003, 'grad_norm': 0.5244246979181154, 'learning_rate': 1.512e-07, 'completion_length': 145.54464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0064239501953125, 'epoch': 0.85} 85%|████████▍ | 2122/2500 [12:42:39<2:12:10, 20.98s/it] 85%|████████▍ | 2123/2500 [12:43:00<2:12:07, 21.03s/it] {'loss': 0.0001, 'grad_norm': 0.6661495740876398, 'learning_rate': 1.5079999999999997e-07, 'completion_length': 148.9553680419922, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00348663330078125, 'epoch': 0.85} 85%|████████▍ | 2123/2500 [12:43:00<2:12:07, 21.03s/it] 85%|████████▍ | 2124/2500 [12:43:20<2:10:45, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.34638237801285776, 'learning_rate': 1.504e-07, 'completion_length': 143.5089340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00396728515625, 'epoch': 0.85} 85%|████████▍ | 2124/2500 [12:43:20<2:10:45, 20.87s/it] 85%|████████▌ | 2125/2500 [12:43:42<2:12:37, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.6697400910669821, 'learning_rate': 1.5e-07, 'completion_length': 163.0357208251953, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.12686798349022865, 'kl': 0.00615692138671875, 'epoch': 0.85} 85%|████████▌ | 2125/2500 [12:43:42<2:12:37, 21.22s/it] 85%|████████▌ | 2126/2500 [12:44:04<2:11:55, 21.16s/it] {'loss': 0.0001, 'grad_norm': 0.3362062287454062, 'learning_rate': 1.4960000000000002e-07, 'completion_length': 152.50894165039062, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00371551513671875, 'epoch': 0.85} 85%|████████▌ | 2126/2500 [12:44:04<2:11:55, 21.16s/it] 85%|████████▌ | 2127/2500 [12:44:25<2:11:38, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.03347791401364385, 'learning_rate': 1.4919999999999999e-07, 'completion_length': 154.76786041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0044708251953125, 'epoch': 0.85} 85%|████████▌ | 2127/2500 [12:44:25<2:11:38, 21.18s/it] 85%|████████▌ | 2128/2500 [12:44:46<2:11:23, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.18769998427881127, 'learning_rate': 1.4879999999999998e-07, 'completion_length': 152.4821548461914, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0054168701171875, 'epoch': 0.85} 85%|████████▌ | 2128/2500 [12:44:46<2:11:23, 21.19s/it] 85%|████████▌ | 2129/2500 [12:45:07<2:09:59, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.28539216855818206, 'learning_rate': 1.484e-07, 'completion_length': 127.16964721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0048675537109375, 'epoch': 0.85} 85%|████████▌ | 2129/2500 [12:45:07<2:09:59, 21.02s/it] 85%|████████▌ | 2130/2500 [12:45:28<2:10:44, 21.20s/it] {'loss': 0.0001, 'grad_norm': 0.037549660812459446, 'learning_rate': 1.4799999999999998e-07, 'completion_length': 144.75000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00341033935546875, 'epoch': 0.85} 85%|████████▌ | 2130/2500 [12:45:28<2:10:44, 21.20s/it] 85%|████████▌ | 2131/2500 [12:45:50<2:10:52, 21.28s/it] {'loss': 0.0001, 'grad_norm': 0.6138417791913523, 'learning_rate': 1.476e-07, 'completion_length': 149.9464340209961, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.00341033935546875, 'epoch': 0.85} 85%|████████▌ | 2131/2500 [12:45:50<2:10:52, 21.28s/it] 85%|████████▌ | 2132/2500 [12:46:10<2:08:37, 20.97s/it] {'loss': 0.0001, 'grad_norm': 0.3977116875453445, 'learning_rate': 1.472e-07, 'completion_length': 143.05357360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00374603271484375, 'epoch': 0.85} 85%|████████▌ | 2132/2500 [12:46:10<2:08:37, 20.97s/it] 85%|████████▌ | 2133/2500 [12:46:31<2:07:50, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.060738116478660814, 'learning_rate': 1.4680000000000002e-07, 'completion_length': 138.6160774230957, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00449371337890625, 'epoch': 0.85} 85%|████████▌ | 2133/2500 [12:46:31<2:07:50, 20.90s/it] 85%|████████▌ | 2134/2500 [12:46:52<2:07:38, 20.93s/it] {'loss': 0.0003, 'grad_norm': 0.4659857740768255, 'learning_rate': 1.464e-07, 'completion_length': 145.9464340209961, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0074310302734375, 'epoch': 0.85} 85%|████████▌ | 2134/2500 [12:46:52<2:07:38, 20.93s/it] 85%|████████▌ | 2135/2500 [12:47:12<2:06:45, 20.84s/it] {'loss': 0.0002, 'grad_norm': 0.016200802581956948, 'learning_rate': 1.4599999999999998e-07, 'completion_length': 149.80358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004058837890625, 'epoch': 0.85} 85%|████████▌ | 2135/2500 [12:47:12<2:06:45, 20.84s/it] 85%|████████▌ | 2136/2500 [12:47:33<2:07:06, 20.95s/it] {'loss': 0.0002, 'grad_norm': 0.01704005042033062, 'learning_rate': 1.456e-07, 'completion_length': 138.67858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004852294921875, 'epoch': 0.85} 85%|████████▌ | 2136/2500 [12:47:33<2:07:06, 20.95s/it] 85%|████████▌ | 2137/2500 [12:47:55<2:07:19, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.01624626168044759, 'learning_rate': 1.4519999999999998e-07, 'completion_length': 152.4107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004241943359375, 'epoch': 0.85} 85%|████████▌ | 2137/2500 [12:47:55<2:07:19, 21.05s/it] 86%|████████▌ | 2138/2500 [12:48:16<2:06:52, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.018954172847564704, 'learning_rate': 1.448e-07, 'completion_length': 149.61607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003814697265625, 'epoch': 0.86} 86%|████████▌ | 2138/2500 [12:48:16<2:06:52, 21.03s/it] 86%|████████▌ | 2139/2500 [12:48:37<2:06:40, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.9105813766861711, 'learning_rate': 1.444e-07, 'completion_length': 146.7053680419922, 'rewards/accuracy_reward': 0.8839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.8839285969734192, 'reward_std': 0.08747542649507523, 'kl': 0.0052337646484375, 'epoch': 0.86} 86%|████████▌ | 2139/2500 [12:48:37<2:06:40, 21.05s/it] 86%|████████▌ | 2140/2500 [12:48:59<2:07:30, 21.25s/it] {'loss': 0.0003, 'grad_norm': 1.0948161534329264, 'learning_rate': 1.44e-07, 'completion_length': 160.08036041259766, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.0835726372897625, 'kl': 0.0082244873046875, 'epoch': 0.86} 86%|████████▌ | 2140/2500 [12:48:59<2:07:30, 21.25s/it] 86%|████████▌ | 2141/2500 [12:49:19<2:06:20, 21.11s/it] {'loss': 0.0001, 'grad_norm': 0.015129478316364433, 'learning_rate': 1.436e-07, 'completion_length': 141.1607208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00327301025390625, 'epoch': 0.86} 86%|████████▌ | 2141/2500 [12:49:19<2:06:20, 21.11s/it] 86%|████████▌ | 2142/2500 [12:49:42<2:07:50, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.1926164819707069, 'learning_rate': 1.4319999999999999e-07, 'completion_length': 160.76786041259766, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0048065185546875, 'epoch': 0.86} 86%|████████▌ | 2142/2500 [12:49:42<2:07:50, 21.43s/it] 86%|████████▌ | 2143/2500 [12:50:03<2:06:54, 21.33s/it] {'loss': 0.0002, 'grad_norm': 0.015508541658761896, 'learning_rate': 1.428e-07, 'completion_length': 156.20536041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0049591064453125, 'epoch': 0.86} 86%|████████▌ | 2143/2500 [12:50:03<2:06:54, 21.33s/it] 86%|████████▌ | 2144/2500 [12:50:23<2:05:33, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.6181161797957468, 'learning_rate': 1.424e-07, 'completion_length': 151.10714721679688, 'rewards/accuracy_reward': 0.8571428954601288, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0835726372897625, 'kl': 0.00579833984375, 'epoch': 0.86} 86%|████████▌ | 2144/2500 [12:50:23<2:05:33, 21.16s/it] 86%|████████▌ | 2145/2500 [12:50:45<2:05:18, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.3611889097045201, 'learning_rate': 1.4199999999999997e-07, 'completion_length': 153.8303680419922, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.05831881985068321, 'kl': 0.00439453125, 'epoch': 0.86} 86%|████████▌ | 2145/2500 [12:50:45<2:05:18, 21.18s/it] 86%|████████▌ | 2146/2500 [12:51:06<2:04:40, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.19425378156212686, 'learning_rate': 1.416e-07, 'completion_length': 150.85714721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052032470703125, 'epoch': 0.86} 86%|████████▌ | 2146/2500 [12:51:06<2:04:40, 21.13s/it] 86%|████████▌ | 2147/2500 [12:51:27<2:04:00, 21.08s/it] {'loss': 0.0004, 'grad_norm': 0.7897054128639355, 'learning_rate': 1.412e-07, 'completion_length': 168.8214340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528661891818047, 'kl': 0.0095367431640625, 'epoch': 0.86} 86%|████████▌ | 2147/2500 [12:51:27<2:04:00, 21.08s/it] 86%|████████▌ | 2148/2500 [12:51:48<2:04:07, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.44293204084536353, 'learning_rate': 1.408e-07, 'completion_length': 157.3928680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.007293701171875, 'epoch': 0.86} 86%|████████▌ | 2148/2500 [12:51:48<2:04:07, 21.16s/it] 86%|████████▌ | 2149/2500 [12:52:09<2:03:14, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.022185195415339268, 'learning_rate': 1.4039999999999999e-07, 'completion_length': 145.11607360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055084228515625, 'epoch': 0.86} 86%|████████▌ | 2149/2500 [12:52:09<2:03:14, 21.07s/it] 86%|████████▌ | 2150/2500 [12:52:30<2:02:58, 21.08s/it] {'loss': 0.0003, 'grad_norm': 0.9461090347661261, 'learning_rate': 1.4e-07, 'completion_length': 154.16964721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0067596435546875, 'epoch': 0.86} 86%|████████▌ | 2150/2500 [12:52:30<2:02:58, 21.08s/it] 86%|████████▌ | 2151/2500 [12:52:51<2:03:15, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.265471115361197, 'learning_rate': 1.396e-07, 'completion_length': 169.0803680419922, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0075531005859375, 'epoch': 0.86} 86%|████████▌ | 2151/2500 [12:52:51<2:03:15, 21.19s/it] 86%|████████▌ | 2152/2500 [12:53:12<2:02:51, 21.18s/it] {'loss': 0.0001, 'grad_norm': 0.22935868204124152, 'learning_rate': 1.3919999999999998e-07, 'completion_length': 147.4464340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.003108978271484375, 'epoch': 0.86} 86%|████████▌ | 2152/2500 [12:53:13<2:02:51, 21.18s/it] 86%|████████▌ | 2153/2500 [12:53:34<2:03:15, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.4358642210730052, 'learning_rate': 1.388e-07, 'completion_length': 158.57144165039062, 'rewards/accuracy_reward': 0.8750000596046448, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.06222161650657654, 'kl': 0.0052337646484375, 'epoch': 0.86} 86%|████████▌ | 2153/2500 [12:53:34<2:03:15, 21.31s/it] 86%|████████▌ | 2154/2500 [12:53:55<2:02:10, 21.19s/it] {'loss': 0.0002, 'grad_norm': 0.4242713404895693, 'learning_rate': 1.384e-07, 'completion_length': 144.3571548461914, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.00528717041015625, 'epoch': 0.86} 86%|████████▌ | 2154/2500 [12:53:55<2:02:10, 21.19s/it] 86%|████████▌ | 2155/2500 [12:54:16<2:02:01, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.639216959076695, 'learning_rate': 1.3800000000000002e-07, 'completion_length': 156.41964721679688, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.003997802734375, 'epoch': 0.86} 86%|████████▌ | 2155/2500 [12:54:16<2:02:01, 21.22s/it] 86%|████████▌ | 2156/2500 [12:54:38<2:01:42, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.4042098063716415, 'learning_rate': 1.376e-07, 'completion_length': 151.00894165039062, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0049896240234375, 'epoch': 0.86} 86%|████████▌ | 2156/2500 [12:54:38<2:01:42, 21.23s/it] 86%|████████▋ | 2157/2500 [12:54:59<2:00:56, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.03296542160271475, 'learning_rate': 1.3719999999999998e-07, 'completion_length': 155.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0060272216796875, 'epoch': 0.86} 86%|████████▋ | 2157/2500 [12:54:59<2:00:56, 21.16s/it] 86%|████████▋ | 2158/2500 [12:55:24<2:08:03, 22.46s/it] {'loss': 0.0003, 'grad_norm': 0.6770083040115873, 'learning_rate': 1.368e-07, 'completion_length': 159.75000762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006683349609375, 'epoch': 0.86} 86%|████████▋ | 2158/2500 [12:55:24<2:08:03, 22.46s/it] 86%|████████▋ | 2159/2500 [12:55:45<2:04:45, 21.95s/it] {'loss': 0.0003, 'grad_norm': 0.3042410335863109, 'learning_rate': 1.3639999999999998e-07, 'completion_length': 151.32144165039062, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.025253813713788986, 'kl': 0.0081787109375, 'epoch': 0.86} 86%|████████▋ | 2159/2500 [12:55:45<2:04:45, 21.95s/it] 86%|████████▋ | 2160/2500 [12:56:06<2:03:17, 21.76s/it] {'loss': 0.0002, 'grad_norm': 8.509268551939511, 'learning_rate': 1.36e-07, 'completion_length': 156.31250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00466156005859375, 'epoch': 0.86} 86%|████████▋ | 2160/2500 [12:56:06<2:03:17, 21.76s/it] 86%|████████▋ | 2161/2500 [12:56:28<2:02:22, 21.66s/it] {'loss': 0.0002, 'grad_norm': 0.8013954752772611, 'learning_rate': 1.356e-07, 'completion_length': 161.42857360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00453948974609375, 'epoch': 0.86} 86%|████████▋ | 2161/2500 [12:56:28<2:02:22, 21.66s/it] 86%|████████▋ | 2162/2500 [12:56:48<2:00:01, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.032176695266253635, 'learning_rate': 1.352e-07, 'completion_length': 142.30358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00595855712890625, 'epoch': 0.86} 86%|████████▋ | 2162/2500 [12:56:48<2:00:01, 21.31s/it] 87%|████████▋ | 2163/2500 [12:57:10<2:00:47, 21.51s/it] {'loss': 0.0003, 'grad_norm': 0.4006095812079359, 'learning_rate': 1.348e-07, 'completion_length': 163.92858123779297, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.0064544677734375, 'epoch': 0.87} 87%|████████▋ | 2163/2500 [12:57:10<2:00:47, 21.51s/it] 87%|████████▋ | 2164/2500 [12:57:31<1:59:43, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.02125249117706926, 'learning_rate': 1.3439999999999999e-07, 'completion_length': 156.71429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00531005859375, 'epoch': 0.87} 87%|████████▋ | 2164/2500 [12:57:31<1:59:43, 21.38s/it] 87%|████████▋ | 2165/2500 [12:57:52<1:58:34, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.41815685849111245, 'learning_rate': 1.34e-07, 'completion_length': 144.86607360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00443267822265625, 'epoch': 0.87} 87%|████████▋ | 2165/2500 [12:57:52<1:58:34, 21.24s/it] 87%|████████▋ | 2166/2500 [12:58:14<1:59:53, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.32754972278858074, 'learning_rate': 1.3359999999999998e-07, 'completion_length': 149.83929443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00537872314453125, 'epoch': 0.87} 87%|████████▋ | 2166/2500 [12:58:14<1:59:53, 21.54s/it] 87%|████████▋ | 2167/2500 [12:58:36<1:59:27, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.020999036070349485, 'learning_rate': 1.332e-07, 'completion_length': 155.60714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062713623046875, 'epoch': 0.87} 87%|████████▋ | 2167/2500 [12:58:36<1:59:27, 21.52s/it] 87%|████████▋ | 2168/2500 [12:58:57<1:58:36, 21.43s/it] {'loss': 0.0001, 'grad_norm': 0.026476985532798696, 'learning_rate': 1.328e-07, 'completion_length': 152.3928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003204345703125, 'epoch': 0.87} 87%|████████▋ | 2168/2500 [12:58:57<1:58:36, 21.43s/it] 87%|████████▋ | 2169/2500 [12:59:18<1:58:14, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.1826101114813046, 'learning_rate': 1.324e-07, 'completion_length': 152.3214340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0057220458984375, 'epoch': 0.87} 87%|████████▋ | 2169/2500 [12:59:18<1:58:14, 21.43s/it] 87%|████████▋ | 2170/2500 [12:59:40<1:58:15, 21.50s/it] {'loss': 0.0003, 'grad_norm': 0.025191730463257116, 'learning_rate': 1.32e-07, 'completion_length': 160.2232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0069580078125, 'epoch': 0.87} 87%|████████▋ | 2170/2500 [12:59:40<1:58:15, 21.50s/it] 87%|████████▋ | 2171/2500 [13:00:01<1:57:26, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.02516929932851903, 'learning_rate': 1.316e-07, 'completion_length': 151.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0042266845703125, 'epoch': 0.87} 87%|████████▋ | 2171/2500 [13:00:01<1:57:26, 21.42s/it] 87%|████████▋ | 2172/2500 [13:00:22<1:56:23, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.6502971287716164, 'learning_rate': 1.312e-07, 'completion_length': 142.02679443359375, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.09138382971286774, 'kl': 0.0051422119140625, 'epoch': 0.87} 87%|████████▋ | 2172/2500 [13:00:22<1:56:23, 21.29s/it] 87%|████████▋ | 2173/2500 [13:00:43<1:55:57, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.03143918374645004, 'learning_rate': 1.308e-07, 'completion_length': 145.58928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0046539306640625, 'epoch': 0.87} 87%|████████▋ | 2173/2500 [13:00:44<1:55:57, 21.28s/it] 87%|████████▋ | 2174/2500 [13:01:05<1:56:06, 21.37s/it] {'loss': 0.0003, 'grad_norm': 0.5173528049844748, 'learning_rate': 1.3039999999999998e-07, 'completion_length': 170.99107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0074462890625, 'epoch': 0.87} 87%|████████▋ | 2174/2500 [13:01:05<1:56:06, 21.37s/it] 87%|████████▋ | 2175/2500 [13:01:27<1:56:08, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.504956037860597, 'learning_rate': 1.3e-07, 'completion_length': 154.14286041259766, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.004974365234375, 'epoch': 0.87} 87%|████████▋ | 2175/2500 [13:01:27<1:56:08, 21.44s/it] 87%|████████▋ | 2176/2500 [13:01:47<1:54:44, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.3787202022581646, 'learning_rate': 1.296e-07, 'completion_length': 144.73214721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.06613001227378845, 'kl': 0.00653076171875, 'epoch': 0.87} 87%|████████▋ | 2176/2500 [13:01:47<1:54:44, 21.25s/it] 87%|████████▋ | 2177/2500 [13:02:08<1:53:45, 21.13s/it] {'loss': 0.0001, 'grad_norm': 0.257384126742736, 'learning_rate': 1.292e-07, 'completion_length': 133.5089340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0036773681640625, 'epoch': 0.87} 87%|████████▋ | 2177/2500 [13:02:08<1:53:45, 21.13s/it] 87%|████████▋ | 2178/2500 [13:02:30<1:54:36, 21.35s/it] {'loss': 0.0003, 'grad_norm': 1.1991934111356606, 'learning_rate': 1.288e-07, 'completion_length': 155.46429443359375, 'rewards/accuracy_reward': 0.892857164144516, 'rewards/format_reward': 1.0, 'reward': 1.8928571939468384, 'reward_std': 0.08868780732154846, 'kl': 0.006500244140625, 'epoch': 0.87} 87%|████████▋ | 2178/2500 [13:02:30<1:54:36, 21.35s/it] 87%|████████▋ | 2179/2500 [13:02:51<1:53:35, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.2234821846608998, 'learning_rate': 1.2839999999999999e-07, 'completion_length': 154.26786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0042724609375, 'epoch': 0.87} 87%|████████▋ | 2179/2500 [13:02:51<1:53:35, 21.23s/it] 87%|████████▋ | 2180/2500 [13:03:12<1:53:17, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.015976741248026373, 'learning_rate': 1.28e-07, 'completion_length': 154.49107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00469970703125, 'epoch': 0.87} 87%|████████▋ | 2180/2500 [13:03:12<1:53:17, 21.24s/it] 87%|████████▋ | 2181/2500 [13:03:33<1:51:44, 21.02s/it] {'loss': 0.0001, 'grad_norm': 0.3420599456134932, 'learning_rate': 1.2759999999999998e-07, 'completion_length': 141.02679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0025787353515625, 'epoch': 0.87} 87%|████████▋ | 2181/2500 [13:03:33<1:51:44, 21.02s/it] 87%|████████▋ | 2182/2500 [13:03:54<1:51:41, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.2253379014355094, 'learning_rate': 1.272e-07, 'completion_length': 155.2857208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0060577392578125, 'epoch': 0.87} 87%|████████▋ | 2182/2500 [13:03:54<1:51:41, 21.07s/it] 87%|████████▋ | 2183/2500 [13:04:15<1:50:59, 21.01s/it] {'loss': 0.0001, 'grad_norm': 0.023281694515643804, 'learning_rate': 1.268e-07, 'completion_length': 154.16964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00371551513671875, 'epoch': 0.87} 87%|████████▋ | 2183/2500 [13:04:15<1:50:59, 21.01s/it] 87%|████████▋ | 2184/2500 [13:04:36<1:50:13, 20.93s/it] {'loss': 0.0001, 'grad_norm': 0.018555240657154522, 'learning_rate': 1.264e-07, 'completion_length': 140.55357360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00331878662109375, 'epoch': 0.87} 87%|████████▋ | 2184/2500 [13:04:36<1:50:13, 20.93s/it] 87%|████████▋ | 2185/2500 [13:04:57<1:50:14, 21.00s/it] {'loss': 0.0001, 'grad_norm': 0.015733962227644428, 'learning_rate': 1.26e-07, 'completion_length': 152.5625, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00362396240234375, 'epoch': 0.87} 87%|████████▋ | 2185/2500 [13:04:57<1:50:14, 21.00s/it] 87%|████████▋ | 2186/2500 [13:05:18<1:50:09, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.45192064037999957, 'learning_rate': 1.2559999999999999e-07, 'completion_length': 153.52679443359375, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.05831882357597351, 'kl': 0.0062103271484375, 'epoch': 0.87} 87%|████████▋ | 2186/2500 [13:05:18<1:50:09, 21.05s/it] 87%|████████▋ | 2187/2500 [13:05:39<1:49:55, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.2639544928304138, 'learning_rate': 1.252e-07, 'completion_length': 152.98214721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0043182373046875, 'epoch': 0.87} 87%|████████▋ | 2187/2500 [13:05:39<1:49:55, 21.07s/it] 88%|████████▊ | 2188/2500 [13:06:00<1:49:52, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.43432663123236365, 'learning_rate': 1.2479999999999998e-07, 'completion_length': 147.74107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00524139404296875, 'epoch': 0.88} 88%|████████▊ | 2188/2500 [13:06:00<1:49:52, 21.13s/it] 88%|████████▊ | 2189/2500 [13:06:22<1:50:12, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.025958802685958137, 'learning_rate': 1.244e-07, 'completion_length': 154.33928680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0051422119140625, 'epoch': 0.88} 88%|████████▊ | 2189/2500 [13:06:22<1:50:12, 21.26s/it] 88%|████████▊ | 2190/2500 [13:06:43<1:49:57, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.3975703778384294, 'learning_rate': 1.24e-07, 'completion_length': 151.26786041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00408935546875, 'epoch': 0.88} 88%|████████▊ | 2190/2500 [13:06:43<1:49:57, 21.28s/it] 88%|████████▊ | 2191/2500 [13:07:05<1:49:29, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.331286758392226, 'learning_rate': 1.236e-07, 'completion_length': 148.55358123779297, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00469207763671875, 'epoch': 0.88} 88%|████████▊ | 2191/2500 [13:07:05<1:49:29, 21.26s/it] 88%|████████▊ | 2192/2500 [13:07:26<1:49:16, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.07260574204621811, 'learning_rate': 1.232e-07, 'completion_length': 157.77678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00506591796875, 'epoch': 0.88} 88%|████████▊ | 2192/2500 [13:07:26<1:49:16, 21.29s/it] 88%|████████▊ | 2193/2500 [13:07:47<1:48:49, 21.27s/it] {'loss': 0.0001, 'grad_norm': 0.15647090013989431, 'learning_rate': 1.228e-07, 'completion_length': 141.30358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00371551513671875, 'epoch': 0.88} 88%|████████▊ | 2193/2500 [13:07:47<1:48:49, 21.27s/it] 88%|████████▊ | 2194/2500 [13:08:09<1:48:42, 21.32s/it] {'loss': 0.0003, 'grad_norm': 0.8989065780366845, 'learning_rate': 1.2239999999999998e-07, 'completion_length': 150.2857208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.06613001227378845, 'kl': 0.007293701171875, 'epoch': 0.88} 88%|████████▊ | 2194/2500 [13:08:09<1:48:42, 21.32s/it] 88%|████████▊ | 2195/2500 [13:08:30<1:48:21, 21.32s/it] {'loss': 0.0002, 'grad_norm': 0.01623604998109204, 'learning_rate': 1.2199999999999998e-07, 'completion_length': 158.99108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00406646728515625, 'epoch': 0.88} 88%|████████▊ | 2195/2500 [13:08:30<1:48:21, 21.32s/it] 88%|████████▊ | 2196/2500 [13:08:51<1:47:08, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.013695092589706599, 'learning_rate': 1.216e-07, 'completion_length': 146.65178680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0043792724609375, 'epoch': 0.88} 88%|████████▊ | 2196/2500 [13:08:51<1:47:08, 21.15s/it] 88%|████████▊ | 2197/2500 [13:09:12<1:47:04, 21.20s/it] {'loss': 0.0001, 'grad_norm': 0.49143945277602674, 'learning_rate': 1.212e-07, 'completion_length': 158.58036041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00360870361328125, 'epoch': 0.88} 88%|████████▊ | 2197/2500 [13:09:12<1:47:04, 21.20s/it] 88%|████████▊ | 2198/2500 [13:09:33<1:46:44, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.2330862134163916, 'learning_rate': 1.208e-07, 'completion_length': 144.5357208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00487518310546875, 'epoch': 0.88} 88%|████████▊ | 2198/2500 [13:09:33<1:46:44, 21.21s/it] 88%|████████▊ | 2199/2500 [13:09:55<1:46:52, 21.30s/it] {'loss': 0.0003, 'grad_norm': 0.4326511680888117, 'learning_rate': 1.204e-07, 'completion_length': 166.74107360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0063018798828125, 'epoch': 0.88} 88%|████████▊ | 2199/2500 [13:09:55<1:46:52, 21.30s/it] 88%|████████▊ | 2200/2500 [13:10:15<1:45:20, 21.07s/it] {'loss': 0.0002, 'grad_norm': 1.8480942423393412, 'learning_rate': 1.2e-07, 'completion_length': 142.67858123779297, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.05831882357597351, 'kl': 0.00555419921875, 'epoch': 0.88} 88%|████████▊ | 2200/2500 [13:10:15<1:45:20, 21.07s/it] 88%|████████▊ | 2201/2500 [13:11:36<3:13:42, 38.87s/it] {'loss': 0.0002, 'grad_norm': 0.6692749735335286, 'learning_rate': 1.1959999999999999e-07, 'completion_length': 145.93750762939453, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0047454833984375, 'epoch': 0.88} 88%|████████▊ | 2201/2500 [13:11:36<3:13:42, 38.87s/it] 88%|████████▊ | 2202/2500 [13:11:46<2:31:01, 30.41s/it] {'loss': 0.0002, 'grad_norm': 0.30861152586063745, 'learning_rate': 1.192e-07, 'completion_length': 144.99107360839844, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.006134033203125, 'epoch': 0.88} 88%|████████▊ | 2202/2500 [13:11:46<2:31:01, 30.41s/it] 88%|████████▊ | 2203/2500 [13:11:57<2:01:44, 24.59s/it] {'loss': 0.0002, 'grad_norm': 0.6787467769133496, 'learning_rate': 1.1879999999999999e-07, 'completion_length': 164.1607208251953, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.0056304931640625, 'epoch': 0.88} 88%|████████▊ | 2203/2500 [13:11:57<2:01:44, 24.59s/it] 88%|████████▊ | 2204/2500 [13:12:07<1:39:25, 20.16s/it] {'loss': 0.0002, 'grad_norm': 1.0451211668749916, 'learning_rate': 1.184e-07, 'completion_length': 146.0714340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0045928955078125, 'epoch': 0.88} 88%|████████▊ | 2204/2500 [13:12:07<1:39:25, 20.16s/it] 88%|████████▊ | 2205/2500 [13:12:18<1:25:00, 17.29s/it] {'loss': 0.0002, 'grad_norm': 0.5121938124437151, 'learning_rate': 1.1799999999999998e-07, 'completion_length': 146.1339340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375001192092896, 'reward_std': 0.07003280520439148, 'kl': 0.00519561767578125, 'epoch': 0.88} 88%|████████▊ | 2205/2500 [13:12:18<1:25:00, 17.29s/it] 88%|████████▊ | 2206/2500 [13:12:28<1:14:10, 15.14s/it] {'loss': 0.0002, 'grad_norm': 0.024166907275107705, 'learning_rate': 1.176e-07, 'completion_length': 148.96428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0059661865234375, 'epoch': 0.88} 88%|████████▊ | 2206/2500 [13:12:28<1:14:10, 15.14s/it] 88%|████████▊ | 2207/2500 [13:12:38<1:06:38, 13.65s/it] {'loss': 0.0002, 'grad_norm': 1.0850157944806855, 'learning_rate': 1.1719999999999999e-07, 'completion_length': 145.36608123779297, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.08747543394565582, 'kl': 0.00537109375, 'epoch': 0.88} 88%|████████▊ | 2207/2500 [13:12:38<1:06:38, 13.65s/it] 88%|████████▊ | 2208/2500 [13:12:49<1:01:57, 12.73s/it] {'loss': 0.0002, 'grad_norm': 0.24735911639581543, 'learning_rate': 1.168e-07, 'completion_length': 154.77678680419922, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00434112548828125, 'epoch': 0.88} 88%|████████▊ | 2208/2500 [13:12:49<1:01:57, 12.73s/it] 88%|████████▊ | 2209/2500 [13:12:59<57:56, 11.95s/it] {'loss': 0.0002, 'grad_norm': 0.2855285844934074, 'learning_rate': 1.164e-07, 'completion_length': 149.58036041259766, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0041351318359375, 'epoch': 0.88} 88%|████████▊ | 2209/2500 [13:12:59<57:56, 11.95s/it] 88%|████████▊ | 2210/2500 [13:13:09<55:41, 11.52s/it] {'loss': 0.0002, 'grad_norm': 1.0556680559117229, 'learning_rate': 1.16e-07, 'completion_length': 171.29464721679688, 'rewards/accuracy_reward': 0.9642857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9642857909202576, 'reward_std': 0.06222161278128624, 'kl': 0.0050201416015625, 'epoch': 0.88} 88%|████████▊ | 2210/2500 [13:13:09<55:41, 11.52s/it] 88%|████████▊ | 2211/2500 [13:13:19<53:28, 11.10s/it] {'loss': 0.0001, 'grad_norm': 0.022230287429461523, 'learning_rate': 1.1559999999999999e-07, 'completion_length': 144.25894165039062, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0035552978515625, 'epoch': 0.88} 88%|████████▊ | 2211/2500 [13:13:19<53:28, 11.10s/it] 88%|████████▊ | 2212/2500 [13:13:30<52:27, 10.93s/it] {'loss': 0.0002, 'grad_norm': 0.32041001058640944, 'learning_rate': 1.1519999999999999e-07, 'completion_length': 154.5357208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0057220458984375, 'epoch': 0.88} 88%|████████▊ | 2212/2500 [13:13:30<52:27, 10.93s/it] 89%|████████▊ | 2213/2500 [13:13:40<51:06, 10.68s/it] {'loss': 0.0002, 'grad_norm': 0.41273888166644135, 'learning_rate': 1.148e-07, 'completion_length': 145.9107208251953, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.005279541015625, 'epoch': 0.89} 89%|████████▊ | 2213/2500 [13:13:40<51:06, 10.68s/it] 89%|████████▊ | 2214/2500 [13:13:51<50:50, 10.67s/it] {'loss': 0.0002, 'grad_norm': 0.2034605099849378, 'learning_rate': 1.1439999999999999e-07, 'completion_length': 148.71428680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00531005859375, 'epoch': 0.89} 89%|████████▊ | 2214/2500 [13:13:51<50:50, 10.67s/it] 89%|████████▊ | 2215/2500 [13:14:02<51:03, 10.75s/it] {'loss': 0.0002, 'grad_norm': 0.033921866606582717, 'learning_rate': 1.14e-07, 'completion_length': 150.9464340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0039520263671875, 'epoch': 0.89} 89%|████████▊ | 2215/2500 [13:14:02<51:03, 10.75s/it] 89%|████████▊ | 2216/2500 [13:14:12<50:38, 10.70s/it] {'loss': 0.0003, 'grad_norm': 0.7167711252209148, 'learning_rate': 1.136e-07, 'completion_length': 169.65178680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.08747543022036552, 'kl': 0.0063934326171875, 'epoch': 0.89} 89%|████████▊ | 2216/2500 [13:14:12<50:38, 10.70s/it] 89%|████████▊ | 2217/2500 [13:14:23<50:12, 10.65s/it] {'loss': 0.0002, 'grad_norm': 0.19578620363231847, 'learning_rate': 1.132e-07, 'completion_length': 152.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.003814697265625, 'epoch': 0.89} 89%|████████▊ | 2217/2500 [13:14:23<50:12, 10.65s/it] 89%|████████▊ | 2218/2500 [13:14:33<50:08, 10.67s/it] {'loss': 0.0003, 'grad_norm': 0.28454763810731915, 'learning_rate': 1.1279999999999999e-07, 'completion_length': 146.3482208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00664520263671875, 'epoch': 0.89} 89%|████████▊ | 2218/2500 [13:14:33<50:08, 10.67s/it] 89%|████████▉ | 2219/2500 [13:14:44<49:29, 10.57s/it] {'loss': 0.0003, 'grad_norm': 0.5577801768695634, 'learning_rate': 1.124e-07, 'completion_length': 143.52678680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.007904052734375, 'epoch': 0.89} 89%|████████▉ | 2219/2500 [13:14:44<49:29, 10.57s/it] 89%|████████▉ | 2220/2500 [13:14:57<53:45, 11.52s/it] {'loss': 0.0002, 'grad_norm': 0.019148589064743284, 'learning_rate': 1.12e-07, 'completion_length': 135.5803680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037994384765625, 'epoch': 0.89} 89%|████████▉ | 2220/2500 [13:14:57<53:45, 11.52s/it] 89%|████████▉ | 2221/2500 [13:15:19<1:08:11, 14.66s/it] {'loss': 0.0005, 'grad_norm': 0.98288203894213, 'learning_rate': 1.116e-07, 'completion_length': 168.00000762939453, 'rewards/accuracy_reward': 0.8839286267757416, 'rewards/format_reward': 1.0, 'reward': 1.883928656578064, 'reward_std': 0.13225442171096802, 'kl': 0.0113525390625, 'epoch': 0.89} 89%|████████▉ | 2221/2500 [13:15:20<1:08:11, 14.66s/it] 89%|████████▉ | 2222/2500 [13:15:41<1:16:48, 16.58s/it] {'loss': 0.0002, 'grad_norm': 0.7464021316373439, 'learning_rate': 1.1119999999999999e-07, 'completion_length': 147.12500762939453, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.12565560638904572, 'kl': 0.00473785400390625, 'epoch': 0.89} 89%|████████▉ | 2222/2500 [13:15:41<1:16:48, 16.58s/it] 89%|████████▉ | 2223/2500 [13:16:01<1:21:56, 17.75s/it] {'loss': 0.0003, 'grad_norm': 0.02264258072217175, 'learning_rate': 1.1079999999999999e-07, 'completion_length': 144.30357360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006500244140625, 'epoch': 0.89} 89%|████████▉ | 2223/2500 [13:16:01<1:21:56, 17.75s/it] 89%|████████▉ | 2224/2500 [13:16:22<1:26:03, 18.71s/it] {'loss': 0.0002, 'grad_norm': 0.5016800151673803, 'learning_rate': 1.104e-07, 'completion_length': 160.21429443359375, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.06222161650657654, 'kl': 0.004150390625, 'epoch': 0.89} 89%|████████▉ | 2224/2500 [13:16:22<1:26:03, 18.71s/it] 89%|████████▉ | 2225/2500 [13:16:43<1:28:25, 19.29s/it] {'loss': 0.0002, 'grad_norm': 0.25780017173393677, 'learning_rate': 1.0999999999999999e-07, 'completion_length': 142.9196548461914, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00392913818359375, 'epoch': 0.89} 89%|████████▉ | 2225/2500 [13:16:43<1:28:25, 19.29s/it] 89%|████████▉ | 2226/2500 [13:17:03<1:29:33, 19.61s/it] {'loss': 0.0001, 'grad_norm': 0.42678978933488515, 'learning_rate': 1.096e-07, 'completion_length': 150.56250762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00301361083984375, 'epoch': 0.89} 89%|████████▉ | 2226/2500 [13:17:03<1:29:33, 19.61s/it] 89%|████████▉ | 2227/2500 [13:17:24<1:31:06, 20.02s/it] {'loss': 0.0002, 'grad_norm': 0.20268658669487952, 'learning_rate': 1.092e-07, 'completion_length': 143.42858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0058746337890625, 'epoch': 0.89} 89%|████████▉ | 2227/2500 [13:17:24<1:31:06, 20.02s/it] 89%|████████▉ | 2228/2500 [13:17:45<1:31:53, 20.27s/it] {'loss': 0.0002, 'grad_norm': 0.28360951364026743, 'learning_rate': 1.088e-07, 'completion_length': 147.46429443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005523681640625, 'epoch': 0.89} 89%|████████▉ | 2228/2500 [13:17:45<1:31:53, 20.27s/it] 89%|████████▉ | 2229/2500 [13:18:06<1:32:23, 20.45s/it] {'loss': 0.0003, 'grad_norm': 0.4632568885961896, 'learning_rate': 1.0839999999999999e-07, 'completion_length': 148.7857208251953, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00634765625, 'epoch': 0.89} 89%|████████▉ | 2229/2500 [13:18:06<1:32:23, 20.45s/it] 89%|████████▉ | 2230/2500 [13:18:27<1:33:33, 20.79s/it] {'loss': 0.0003, 'grad_norm': 0.2826507172993853, 'learning_rate': 1.0799999999999999e-07, 'completion_length': 160.12500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0072021484375, 'epoch': 0.89} 89%|████████▉ | 2230/2500 [13:18:27<1:33:33, 20.79s/it] 89%|████████▉ | 2231/2500 [13:18:48<1:33:37, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.024799962976885335, 'learning_rate': 1.076e-07, 'completion_length': 155.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005279541015625, 'epoch': 0.89} 89%|████████▉ | 2231/2500 [13:18:48<1:33:37, 20.88s/it] 89%|████████▉ | 2232/2500 [13:19:09<1:33:20, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.48470542598329863, 'learning_rate': 1.072e-07, 'completion_length': 156.74108123779297, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.0059356689453125, 'epoch': 0.89} 89%|████████▉ | 2232/2500 [13:19:09<1:33:20, 20.90s/it] 89%|████████▉ | 2233/2500 [13:19:31<1:33:42, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.22324107065472953, 'learning_rate': 1.068e-07, 'completion_length': 154.3928680419922, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0044403076171875, 'epoch': 0.89} 89%|████████▉ | 2233/2500 [13:19:31<1:33:42, 21.06s/it] 89%|████████▉ | 2234/2500 [13:19:52<1:33:26, 21.08s/it] {'loss': 0.0001, 'grad_norm': 0.014116963370442966, 'learning_rate': 1.0639999999999999e-07, 'completion_length': 151.83036041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00289154052734375, 'epoch': 0.89} 89%|████████▉ | 2234/2500 [13:19:52<1:33:26, 21.08s/it] 89%|████████▉ | 2235/2500 [13:20:13<1:33:16, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.42845761664190496, 'learning_rate': 1.06e-07, 'completion_length': 158.3214340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.096499003469944, 'kl': 0.0044708251953125, 'epoch': 0.89} 89%|████████▉ | 2235/2500 [13:20:13<1:33:16, 21.12s/it] 89%|████████▉ | 2236/2500 [13:20:34<1:32:27, 21.01s/it] {'loss': 0.0001, 'grad_norm': 0.5373806614797293, 'learning_rate': 1.0559999999999999e-07, 'completion_length': 152.85714721679688, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.00347900390625, 'epoch': 0.89} 89%|████████▉ | 2236/2500 [13:20:34<1:32:27, 21.01s/it] 89%|████████▉ | 2237/2500 [13:20:55<1:31:42, 20.92s/it] {'loss': 0.0002, 'grad_norm': 0.5441724852277054, 'learning_rate': 1.052e-07, 'completion_length': 144.0714340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0042877197265625, 'epoch': 0.89} 89%|████████▉ | 2237/2500 [13:20:55<1:31:42, 20.92s/it] 90%|████████▉ | 2238/2500 [13:21:15<1:30:51, 20.81s/it] {'loss': 0.0003, 'grad_norm': 0.04472276496569437, 'learning_rate': 1.048e-07, 'completion_length': 143.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00676727294921875, 'epoch': 0.9} 90%|████████▉ | 2238/2500 [13:21:15<1:30:51, 20.81s/it] 90%|████████▉ | 2239/2500 [13:21:36<1:30:17, 20.76s/it] {'loss': 0.0001, 'grad_norm': 0.04080676839672082, 'learning_rate': 1.0440000000000001e-07, 'completion_length': 142.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00335693359375, 'epoch': 0.9} 90%|████████▉ | 2239/2500 [13:21:36<1:30:17, 20.76s/it] 90%|████████▉ | 2240/2500 [13:21:57<1:31:02, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.5922843469858816, 'learning_rate': 1.0399999999999999e-07, 'completion_length': 164.24108123779297, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.08747543022036552, 'kl': 0.0063629150390625, 'epoch': 0.9} 90%|████████▉ | 2240/2500 [13:21:57<1:31:02, 21.01s/it] 90%|████████▉ | 2241/2500 [13:22:18<1:30:01, 20.86s/it] {'loss': 0.0001, 'grad_norm': 0.04141471465520914, 'learning_rate': 1.0359999999999999e-07, 'completion_length': 140.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003448486328125, 'epoch': 0.9} 90%|████████▉ | 2241/2500 [13:22:18<1:30:01, 20.86s/it] 90%|████████▉ | 2242/2500 [13:22:39<1:30:19, 21.01s/it] {'loss': 0.0003, 'grad_norm': 0.36984240617042236, 'learning_rate': 1.032e-07, 'completion_length': 154.1339340209961, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0086669921875, 'epoch': 0.9} 90%|████████▉ | 2242/2500 [13:22:39<1:30:19, 21.01s/it] 90%|████████▉ | 2243/2500 [13:23:00<1:29:31, 20.90s/it] {'loss': 0.0001, 'grad_norm': 0.04185024507869839, 'learning_rate': 1.028e-07, 'completion_length': 139.06250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.002960205078125, 'epoch': 0.9} 90%|████████▉ | 2243/2500 [13:23:00<1:29:31, 20.90s/it] 90%|████████▉ | 2244/2500 [13:23:20<1:28:37, 20.77s/it] {'loss': 0.0001, 'grad_norm': 0.18235851549184248, 'learning_rate': 1.024e-07, 'completion_length': 136.58929443359375, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.00321197509765625, 'epoch': 0.9} 90%|████████▉ | 2244/2500 [13:23:20<1:28:37, 20.77s/it] 90%|████████▉ | 2245/2500 [13:23:41<1:27:47, 20.66s/it] {'loss': 0.0002, 'grad_norm': 0.32876673262898315, 'learning_rate': 1.0199999999999999e-07, 'completion_length': 137.4732208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00583648681640625, 'epoch': 0.9} 90%|████████▉ | 2245/2500 [13:23:41<1:27:47, 20.66s/it] 90%|████████▉ | 2246/2500 [13:24:02<1:28:21, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.026549888653318345, 'learning_rate': 1.016e-07, 'completion_length': 152.6964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00494384765625, 'epoch': 0.9} 90%|████████▉ | 2246/2500 [13:24:02<1:28:21, 20.87s/it] 90%|████████▉ | 2247/2500 [13:24:24<1:29:36, 21.25s/it] {'loss': 0.0001, 'grad_norm': 0.049992507081693935, 'learning_rate': 1.0119999999999999e-07, 'completion_length': 150.67858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00353240966796875, 'epoch': 0.9} 90%|████████▉ | 2247/2500 [13:24:24<1:29:36, 21.25s/it] 90%|████████▉ | 2248/2500 [13:24:46<1:29:47, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.03721649637117605, 'learning_rate': 1.008e-07, 'completion_length': 162.42857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0061492919921875, 'epoch': 0.9} 90%|████████▉ | 2248/2500 [13:24:46<1:29:47, 21.38s/it] 90%|████████▉ | 2249/2500 [13:25:07<1:29:26, 21.38s/it] {'loss': 0.0002, 'grad_norm': 0.021758208658020868, 'learning_rate': 1.004e-07, 'completion_length': 142.02678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00439453125, 'epoch': 0.9} 90%|████████▉ | 2249/2500 [13:25:07<1:29:26, 21.38s/it] 90%|█████████ | 2250/2500 [13:25:28<1:28:39, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.2997385010573949, 'learning_rate': 1e-07, 'completion_length': 151.75000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0057220458984375, 'epoch': 0.9} 90%|█████████ | 2250/2500 [13:25:28<1:28:39, 21.28s/it] 90%|█████████ | 2251/2500 [13:25:51<1:29:36, 21.59s/it] {'loss': 0.0003, 'grad_norm': 0.5977574107060294, 'learning_rate': 9.959999999999999e-08, 'completion_length': 158.7232208251953, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0070953369140625, 'epoch': 0.9} 90%|█████████ | 2251/2500 [13:25:51<1:29:36, 21.59s/it] 90%|█████████ | 2252/2500 [13:26:12<1:28:34, 21.43s/it] {'loss': 0.0001, 'grad_norm': 0.44426880386316453, 'learning_rate': 9.919999999999999e-08, 'completion_length': 146.3482208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00293731689453125, 'epoch': 0.9} 90%|█████████ | 2252/2500 [13:26:12<1:28:34, 21.43s/it] 90%|█████████ | 2253/2500 [13:26:33<1:28:04, 21.40s/it] {'loss': 0.0001, 'grad_norm': 0.3003477854056322, 'learning_rate': 9.88e-08, 'completion_length': 151.10714721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00301361083984375, 'epoch': 0.9} 90%|█████████ | 2253/2500 [13:26:33<1:28:04, 21.40s/it] 90%|█████████ | 2254/2500 [13:26:54<1:26:48, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.7702089260576652, 'learning_rate': 9.84e-08, 'completion_length': 137.83036041259766, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00485992431640625, 'epoch': 0.9} 90%|█████████ | 2254/2500 [13:26:54<1:26:48, 21.17s/it] 90%|█████████ | 2255/2500 [13:27:14<1:25:56, 21.05s/it] {'loss': 0.0002, 'grad_norm': 0.330829179912342, 'learning_rate': 9.8e-08, 'completion_length': 153.50000762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00406646728515625, 'epoch': 0.9} 90%|█████████ | 2255/2500 [13:27:14<1:25:56, 21.05s/it] 90%|█████████ | 2256/2500 [13:27:35<1:25:22, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.029919018198117535, 'learning_rate': 9.76e-08, 'completion_length': 156.52679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00494384765625, 'epoch': 0.9} 90%|█████████ | 2256/2500 [13:27:35<1:25:22, 20.99s/it] 90%|█████████ | 2257/2500 [13:27:57<1:25:55, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.3015784530868849, 'learning_rate': 9.72e-08, 'completion_length': 157.57144165039062, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.008636474609375, 'epoch': 0.9} 90%|█████████ | 2257/2500 [13:27:57<1:25:55, 21.22s/it] 90%|█████████ | 2258/2500 [13:28:18<1:25:39, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.28256633989563407, 'learning_rate': 9.679999999999999e-08, 'completion_length': 161.1964340209961, 'rewards/accuracy_reward': 0.9017857611179352, 'rewards/format_reward': 1.0, 'reward': 1.9017857909202576, 'reward_std': 0.03696779906749725, 'kl': 0.004302978515625, 'epoch': 0.9} 90%|█████████ | 2258/2500 [13:28:18<1:25:39, 21.24s/it] 90%|█████████ | 2259/2500 [13:28:39<1:24:35, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.22109915285479365, 'learning_rate': 9.639999999999999e-08, 'completion_length': 148.71428680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00472259521484375, 'epoch': 0.9} 90%|█████████ | 2259/2500 [13:28:39<1:24:35, 21.06s/it] 90%|█████████ | 2260/2500 [13:28:59<1:23:36, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.027420363671956924, 'learning_rate': 9.6e-08, 'completion_length': 141.0714340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0040435791015625, 'epoch': 0.9} 90%|█████████ | 2260/2500 [13:28:59<1:23:36, 20.90s/it] 90%|█████████ | 2261/2500 [13:29:21<1:23:43, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.02317218587657598, 'learning_rate': 9.56e-08, 'completion_length': 172.77679443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0042877197265625, 'epoch': 0.9} 90%|█████████ | 2261/2500 [13:29:21<1:23:43, 21.02s/it] 90%|█████████ | 2262/2500 [13:29:42<1:23:45, 21.12s/it] {'loss': 0.0003, 'grad_norm': 0.419659862260625, 'learning_rate': 9.52e-08, 'completion_length': 174.79464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006378173828125, 'epoch': 0.9} 90%|█████████ | 2262/2500 [13:29:42<1:23:45, 21.12s/it] 91%|█████████ | 2263/2500 [13:30:03<1:23:38, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.4084086624968897, 'learning_rate': 9.479999999999999e-08, 'completion_length': 161.25894165039062, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0053863525390625, 'epoch': 0.91} 91%|█████████ | 2263/2500 [13:30:03<1:23:38, 21.17s/it] 91%|█████████ | 2264/2500 [13:30:25<1:23:47, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.01847747432742993, 'learning_rate': 9.44e-08, 'completion_length': 152.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0055389404296875, 'epoch': 0.91} 91%|█████████ | 2264/2500 [13:30:25<1:23:47, 21.30s/it] 91%|█████████ | 2265/2500 [13:30:46<1:23:17, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.9077294910917648, 'learning_rate': 9.4e-08, 'completion_length': 159.31250762939453, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.07124518603086472, 'kl': 0.00640869140625, 'epoch': 0.91} 91%|█████████ | 2265/2500 [13:30:46<1:23:17, 21.27s/it] 91%|█████████ | 2266/2500 [13:31:07<1:22:32, 21.17s/it] {'loss': 0.0003, 'grad_norm': 2.2416539240460613, 'learning_rate': 9.36e-08, 'completion_length': 161.5982208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0078582763671875, 'epoch': 0.91} 91%|█████████ | 2266/2500 [13:31:07<1:22:32, 21.17s/it] 91%|█████████ | 2267/2500 [13:31:29<1:22:34, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.20189984275940315, 'learning_rate': 9.32e-08, 'completion_length': 156.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004913330078125, 'epoch': 0.91} 91%|█████████ | 2267/2500 [13:31:29<1:22:34, 21.27s/it] 91%|█████████ | 2268/2500 [13:31:50<1:22:20, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.5106611749195662, 'learning_rate': 9.279999999999998e-08, 'completion_length': 162.83929443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0056915283203125, 'epoch': 0.91} 91%|█████████ | 2268/2500 [13:31:50<1:22:20, 21.30s/it] 91%|█████████ | 2269/2500 [13:32:10<1:20:59, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.05197884036844049, 'learning_rate': 9.24e-08, 'completion_length': 144.56250762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0053253173828125, 'epoch': 0.91} 91%|█████████ | 2269/2500 [13:32:10<1:20:59, 21.04s/it] 91%|█████████ | 2270/2500 [13:32:31<1:19:57, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.049584011871809906, 'learning_rate': 9.199999999999999e-08, 'completion_length': 155.2232208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037994384765625, 'epoch': 0.91} 91%|█████████ | 2270/2500 [13:32:31<1:19:57, 20.86s/it] 91%|█████████ | 2271/2500 [13:32:52<1:20:03, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.19954332690205, 'learning_rate': 9.16e-08, 'completion_length': 151.2589340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00433349609375, 'epoch': 0.91} 91%|█████████ | 2271/2500 [13:32:52<1:20:03, 20.98s/it] 91%|█████████ | 2272/2500 [13:33:13<1:19:55, 21.03s/it] {'loss': 0.0002, 'grad_norm': 0.016987208710503483, 'learning_rate': 9.12e-08, 'completion_length': 161.83036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00388336181640625, 'epoch': 0.91} 91%|█████████ | 2272/2500 [13:33:13<1:19:55, 21.03s/it] 91%|█████████ | 2273/2500 [13:33:34<1:19:43, 21.07s/it] {'loss': 0.0003, 'grad_norm': 0.41273766863122346, 'learning_rate': 9.08e-08, 'completion_length': 151.4375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0076904296875, 'epoch': 0.91} 91%|█████████ | 2273/2500 [13:33:34<1:19:43, 21.07s/it] 91%|█████████ | 2274/2500 [13:33:56<1:19:53, 21.21s/it] {'loss': 0.0002, 'grad_norm': 0.050511302930014816, 'learning_rate': 9.039999999999999e-08, 'completion_length': 159.71428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00531005859375, 'epoch': 0.91} 91%|█████████ | 2274/2500 [13:33:56<1:19:53, 21.21s/it] 91%|█████████ | 2275/2500 [13:34:18<1:20:01, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.2023055912515787, 'learning_rate': 9e-08, 'completion_length': 153.90179443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00409698486328125, 'epoch': 0.91} 91%|█████████ | 2275/2500 [13:34:18<1:20:01, 21.34s/it] 91%|█████████ | 2276/2500 [13:34:38<1:18:59, 21.16s/it] {'loss': 0.0003, 'grad_norm': 0.02496591700834005, 'learning_rate': 8.96e-08, 'completion_length': 151.35714721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.007354736328125, 'epoch': 0.91} 91%|█████████ | 2276/2500 [13:34:38<1:18:59, 21.16s/it] 91%|█████████ | 2277/2500 [13:34:58<1:17:26, 20.83s/it] {'loss': 0.0001, 'grad_norm': 0.8677790305533959, 'learning_rate': 8.919999999999999e-08, 'completion_length': 134.0, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00351715087890625, 'epoch': 0.91} 91%|█████████ | 2277/2500 [13:34:58<1:17:26, 20.83s/it] 91%|█████████ | 2278/2500 [13:35:19<1:16:54, 20.79s/it] {'loss': 0.0002, 'grad_norm': 0.8555368281684027, 'learning_rate': 8.88e-08, 'completion_length': 139.30358123779297, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.10161416977643967, 'kl': 0.0053253173828125, 'epoch': 0.91} 91%|█████████ | 2278/2500 [13:35:19<1:16:54, 20.79s/it] 91%|█████████ | 2279/2500 [13:35:40<1:16:50, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.9417134902148386, 'learning_rate': 8.84e-08, 'completion_length': 149.48214721679688, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.06222161650657654, 'kl': 0.006011962890625, 'epoch': 0.91} 91%|█████████ | 2279/2500 [13:35:40<1:16:50, 20.86s/it] 91%|█████████ | 2280/2500 [13:36:01<1:16:46, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.34180661636327164, 'learning_rate': 8.8e-08, 'completion_length': 155.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005645751953125, 'epoch': 0.91} 91%|█████████ | 2280/2500 [13:36:01<1:16:46, 20.94s/it] 91%|█████████ | 2281/2500 [13:36:22<1:16:07, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.02267315256932346, 'learning_rate': 8.759999999999999e-08, 'completion_length': 154.6428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0056915283203125, 'epoch': 0.91} 91%|█████████ | 2281/2500 [13:36:22<1:16:07, 20.86s/it] 91%|█████████▏| 2282/2500 [13:36:42<1:15:20, 20.74s/it] {'loss': 0.0002, 'grad_norm': 0.2743873069699144, 'learning_rate': 8.72e-08, 'completion_length': 148.5982208251953, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0046234130859375, 'epoch': 0.91} 91%|█████████▏| 2282/2500 [13:36:42<1:15:20, 20.74s/it] 91%|█████████▏| 2283/2500 [13:37:04<1:15:26, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.451240231354436, 'learning_rate': 8.68e-08, 'completion_length': 151.79464721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00463104248046875, 'epoch': 0.91} 91%|█████████▏| 2283/2500 [13:37:04<1:15:26, 20.86s/it] 91%|█████████▏| 2284/2500 [13:37:25<1:15:24, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.025663925994793675, 'learning_rate': 8.64e-08, 'completion_length': 163.91964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00458526611328125, 'epoch': 0.91} 91%|█████████▏| 2284/2500 [13:37:25<1:15:24, 20.94s/it] 91%|█████████▏| 2285/2500 [13:37:45<1:14:39, 20.83s/it] {'loss': 0.0002, 'grad_norm': 0.3716823810090477, 'learning_rate': 8.599999999999999e-08, 'completion_length': 151.74107360839844, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.00543212890625, 'epoch': 0.91} 91%|█████████▏| 2285/2500 [13:37:45<1:14:39, 20.83s/it] 91%|█████████▏| 2286/2500 [13:38:07<1:14:50, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.7252576957808182, 'learning_rate': 8.559999999999999e-08, 'completion_length': 161.7589340209961, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0052947998046875, 'epoch': 0.91} 91%|█████████▏| 2286/2500 [13:38:07<1:14:50, 20.98s/it] 91%|█████████▏| 2287/2500 [13:38:29<1:15:33, 21.28s/it] {'loss': 0.0002, 'grad_norm': 0.5726637117506579, 'learning_rate': 8.52e-08, 'completion_length': 171.33929443359375, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.033065006136894226, 'kl': 0.006195068359375, 'epoch': 0.91} 91%|█████████▏| 2287/2500 [13:38:29<1:15:33, 21.28s/it] 92%|█████████▏| 2288/2500 [13:38:49<1:14:37, 21.12s/it] {'loss': 0.0001, 'grad_norm': 0.025829290828351403, 'learning_rate': 8.479999999999999e-08, 'completion_length': 146.44644165039062, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003631591796875, 'epoch': 0.92} 92%|█████████▏| 2288/2500 [13:38:49<1:14:37, 21.12s/it] 92%|█████████▏| 2289/2500 [13:39:10<1:13:57, 21.03s/it] {'loss': 0.0002, 'grad_norm': 2.553630001317961, 'learning_rate': 8.44e-08, 'completion_length': 151.2678680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00461578369140625, 'epoch': 0.92} 92%|█████████▏| 2289/2500 [13:39:10<1:13:57, 21.03s/it] 92%|█████████▏| 2290/2500 [13:39:31<1:13:08, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.37855403379376595, 'learning_rate': 8.4e-08, 'completion_length': 144.2589340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0042572021484375, 'epoch': 0.92} 92%|█████████▏| 2290/2500 [13:39:31<1:13:08, 20.90s/it] 92%|█████████▏| 2291/2500 [13:39:52<1:13:18, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.018733633592346005, 'learning_rate': 8.36e-08, 'completion_length': 166.4553680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004302978515625, 'epoch': 0.92} 92%|█████████▏| 2291/2500 [13:39:52<1:13:18, 21.04s/it] 92%|█████████▏| 2292/2500 [13:40:14<1:13:51, 21.31s/it] {'loss': 0.0001, 'grad_norm': 0.03242603729343988, 'learning_rate': 8.319999999999999e-08, 'completion_length': 160.24108123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0031890869140625, 'epoch': 0.92} 92%|█████████▏| 2292/2500 [13:40:14<1:13:51, 21.31s/it] 92%|█████████▏| 2293/2500 [13:40:36<1:14:14, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.41426630892582994, 'learning_rate': 8.28e-08, 'completion_length': 164.95536041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00628662109375, 'epoch': 0.92} 92%|█████████▏| 2293/2500 [13:40:36<1:14:14, 21.52s/it] 92%|█████████▏| 2294/2500 [13:40:57<1:13:33, 21.43s/it] {'loss': 0.0003, 'grad_norm': 0.21913252371106232, 'learning_rate': 8.24e-08, 'completion_length': 149.2857208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0063018798828125, 'epoch': 0.92} 92%|█████████▏| 2294/2500 [13:40:57<1:13:33, 21.43s/it] 92%|█████████▏| 2295/2500 [13:41:18<1:12:39, 21.26s/it] {'loss': 0.0002, 'grad_norm': 0.043238842761392636, 'learning_rate': 8.2e-08, 'completion_length': 159.62500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00475311279296875, 'epoch': 0.92} 92%|█████████▏| 2295/2500 [13:41:18<1:12:39, 21.26s/it] 92%|█████████▏| 2296/2500 [13:41:39<1:11:51, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.4717994665523819, 'learning_rate': 8.16e-08, 'completion_length': 153.55357360839844, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00421142578125, 'epoch': 0.92} 92%|█████████▏| 2296/2500 [13:41:39<1:11:51, 21.14s/it] 92%|█████████▏| 2297/2500 [13:42:00<1:11:04, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.021274558464892266, 'learning_rate': 8.119999999999999e-08, 'completion_length': 157.39286041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00567626953125, 'epoch': 0.92} 92%|█████████▏| 2297/2500 [13:42:00<1:11:04, 21.01s/it] 92%|█████████▏| 2298/2500 [13:42:21<1:10:37, 20.98s/it] {'loss': 0.0002, 'grad_norm': 0.4807131680027173, 'learning_rate': 8.08e-08, 'completion_length': 155.4732208251953, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.00525665283203125, 'epoch': 0.92} 92%|█████████▏| 2298/2500 [13:42:21<1:10:37, 20.98s/it] 92%|█████████▏| 2299/2500 [13:42:42<1:10:32, 21.06s/it] {'loss': 0.0001, 'grad_norm': 0.014852065981588678, 'learning_rate': 8.039999999999999e-08, 'completion_length': 151.15179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00350189208984375, 'epoch': 0.92} 92%|█████████▏| 2299/2500 [13:42:42<1:10:32, 21.06s/it] 92%|█████████▏| 2300/2500 [13:43:03<1:10:17, 21.09s/it] {'loss': 0.0002, 'grad_norm': 1.1602166449322981, 'learning_rate': 8e-08, 'completion_length': 151.79464721679688, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0835726372897625, 'kl': 0.00531005859375, 'epoch': 0.92} 92%|█████████▏| 2300/2500 [13:43:03<1:10:17, 21.09s/it] 92%|█████████▏| 2301/2500 [13:44:11<1:56:38, 35.17s/it] {'loss': 0.0003, 'grad_norm': 0.3015204506426308, 'learning_rate': 7.96e-08, 'completion_length': 147.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0068359375, 'epoch': 0.92} 92%|█████████▏| 2301/2500 [13:44:11<1:56:38, 35.17s/it] 92%|█████████▏| 2302/2500 [13:44:32<1:41:43, 30.82s/it] {'loss': 0.0002, 'grad_norm': 0.01581280566185861, 'learning_rate': 7.920000000000001e-08, 'completion_length': 154.1339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0039825439453125, 'epoch': 0.92} 92%|█████████▏| 2302/2500 [13:44:32<1:41:43, 30.82s/it] 92%|█████████▏| 2303/2500 [13:44:52<1:30:58, 27.71s/it] {'loss': 0.0002, 'grad_norm': 0.020908230818534514, 'learning_rate': 7.879999999999999e-08, 'completion_length': 153.15179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00496673583984375, 'epoch': 0.92} 92%|█████████▏| 2303/2500 [13:44:52<1:30:58, 27.71s/it] 92%|█████████▏| 2304/2500 [13:45:13<1:23:53, 25.68s/it] {'loss': 0.0002, 'grad_norm': 0.18192392948523103, 'learning_rate': 7.839999999999999e-08, 'completion_length': 154.8214340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0045013427734375, 'epoch': 0.92} 92%|█████████▏| 2304/2500 [13:45:13<1:23:53, 25.68s/it] 92%|█████████▏| 2305/2500 [13:45:34<1:19:11, 24.37s/it] {'loss': 0.0002, 'grad_norm': 0.410209749802092, 'learning_rate': 7.8e-08, 'completion_length': 161.8303680419922, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.006134033203125, 'epoch': 0.92} 92%|█████████▏| 2305/2500 [13:45:34<1:19:11, 24.37s/it] 92%|█████████▏| 2306/2500 [13:45:55<1:15:10, 23.25s/it] {'loss': 0.0002, 'grad_norm': 0.1955311490096945, 'learning_rate': 7.76e-08, 'completion_length': 150.95536041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004058837890625, 'epoch': 0.92} 92%|█████████▏| 2306/2500 [13:45:55<1:15:10, 23.25s/it] 92%|█████████▏| 2307/2500 [13:46:16<1:12:51, 22.65s/it] {'loss': 0.0003, 'grad_norm': 1.2274621648675332, 'learning_rate': 7.72e-08, 'completion_length': 164.54464721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00667572021484375, 'epoch': 0.92} 92%|█████████▏| 2307/2500 [13:46:16<1:12:51, 22.65s/it] 92%|█████████▏| 2308/2500 [13:46:38<1:11:16, 22.27s/it] {'loss': 0.0002, 'grad_norm': 0.02983983043436107, 'learning_rate': 7.679999999999999e-08, 'completion_length': 143.4107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004150390625, 'epoch': 0.92} 92%|█████████▏| 2308/2500 [13:46:38<1:11:16, 22.27s/it] 92%|█████████▏| 2309/2500 [13:46:58<1:09:08, 21.72s/it] {'loss': 0.0002, 'grad_norm': 0.6033332370681115, 'learning_rate': 7.64e-08, 'completion_length': 141.66964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00437164306640625, 'epoch': 0.92} 92%|█████████▏| 2309/2500 [13:46:58<1:09:08, 21.72s/it] 92%|█████████▏| 2310/2500 [13:47:19<1:07:51, 21.43s/it] {'loss': 0.0003, 'grad_norm': 0.359586069598533, 'learning_rate': 7.599999999999999e-08, 'completion_length': 152.80358123779297, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0078887939453125, 'epoch': 0.92} 92%|█████████▏| 2310/2500 [13:47:19<1:07:51, 21.43s/it] 92%|█████████▏| 2311/2500 [13:47:39<1:06:34, 21.14s/it] {'loss': 0.0001, 'grad_norm': 0.018393021379264407, 'learning_rate': 7.56e-08, 'completion_length': 138.2678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003101348876953125, 'epoch': 0.92} 92%|█████████▏| 2311/2500 [13:47:39<1:06:34, 21.14s/it] 92%|█████████▏| 2312/2500 [13:48:01<1:07:02, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.3182147347742483, 'learning_rate': 7.52e-08, 'completion_length': 155.38394165039062, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0077056884765625, 'epoch': 0.92} 92%|█████████▏| 2312/2500 [13:48:01<1:07:02, 21.40s/it] 93%|█████████▎| 2313/2500 [13:48:22<1:06:17, 21.27s/it] {'loss': 0.0001, 'grad_norm': 0.3900060925879878, 'learning_rate': 7.480000000000001e-08, 'completion_length': 145.06250762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0031280517578125, 'epoch': 0.93} 93%|█████████▎| 2313/2500 [13:48:22<1:06:17, 21.27s/it] 93%|█████████▎| 2314/2500 [13:48:43<1:05:39, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.05731909665458035, 'learning_rate': 7.439999999999999e-08, 'completion_length': 151.93750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0042266845703125, 'epoch': 0.93} 93%|█████████▎| 2314/2500 [13:48:43<1:05:39, 21.18s/it] 93%|█████████▎| 2315/2500 [13:49:05<1:05:21, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.6881268017088819, 'learning_rate': 7.399999999999999e-08, 'completion_length': 153.2232208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0045623779296875, 'epoch': 0.93} 93%|█████████▎| 2315/2500 [13:49:05<1:05:21, 21.20s/it] 93%|█████████▎| 2316/2500 [13:49:25<1:04:28, 21.02s/it] {'loss': 0.0004, 'grad_norm': 0.2801112946295021, 'learning_rate': 7.36e-08, 'completion_length': 148.66964721679688, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.009033203125, 'epoch': 0.93} 93%|█████████▎| 2316/2500 [13:49:25<1:04:28, 21.02s/it] 93%|█████████▎| 2317/2500 [13:49:46<1:04:01, 20.99s/it] {'loss': 0.0002, 'grad_norm': 0.1554771336789151, 'learning_rate': 7.32e-08, 'completion_length': 162.16964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052947998046875, 'epoch': 0.93} 93%|█████████▎| 2317/2500 [13:49:46<1:04:01, 20.99s/it] 93%|█████████▎| 2318/2500 [13:50:09<1:05:39, 21.65s/it] {'loss': 0.0002, 'grad_norm': 0.23436421296609106, 'learning_rate': 7.28e-08, 'completion_length': 164.17858123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0062408447265625, 'epoch': 0.93} 93%|█████████▎| 2318/2500 [13:50:09<1:05:39, 21.65s/it] 93%|█████████▎| 2319/2500 [13:50:30<1:04:11, 21.28s/it] {'loss': 0.0001, 'grad_norm': 0.4383710324911139, 'learning_rate': 7.24e-08, 'completion_length': 138.02678680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00302886962890625, 'epoch': 0.93} 93%|█████████▎| 2319/2500 [13:50:30<1:04:11, 21.28s/it] 93%|█████████▎| 2320/2500 [13:50:51<1:03:48, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.03173671901252157, 'learning_rate': 7.2e-08, 'completion_length': 160.9464340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0071868896484375, 'epoch': 0.93} 93%|█████████▎| 2320/2500 [13:50:51<1:03:48, 21.27s/it] 93%|█████████▎| 2321/2500 [13:51:11<1:02:47, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.02049381392928424, 'learning_rate': 7.159999999999999e-08, 'completion_length': 146.35714721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005035400390625, 'epoch': 0.93} 93%|█████████▎| 2321/2500 [13:51:11<1:02:47, 21.04s/it] 93%|█████████▎| 2322/2500 [13:51:33<1:02:35, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.45007093285782207, 'learning_rate': 7.12e-08, 'completion_length': 155.18750762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797896146774, 'kl': 0.0056915283203125, 'epoch': 0.93} 93%|█████████▎| 2322/2500 [13:51:33<1:02:35, 21.10s/it] 93%|█████████▎| 2323/2500 [13:51:54<1:02:25, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.6861880581729957, 'learning_rate': 7.08e-08, 'completion_length': 161.3839340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.004150390625, 'epoch': 0.93} 93%|█████████▎| 2323/2500 [13:51:54<1:02:25, 21.16s/it] 93%|█████████▎| 2324/2500 [13:52:16<1:02:26, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.6440371068138631, 'learning_rate': 7.04e-08, 'completion_length': 156.11608123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0060272216796875, 'epoch': 0.93} 93%|█████████▎| 2324/2500 [13:52:16<1:02:26, 21.29s/it] 93%|█████████▎| 2325/2500 [13:52:36<1:01:31, 21.09s/it] {'loss': 0.0001, 'grad_norm': 0.01360559993881918, 'learning_rate': 7e-08, 'completion_length': 143.4821548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0035400390625, 'epoch': 0.93} 93%|█████████▎| 2325/2500 [13:52:36<1:01:31, 21.09s/it] 93%|█████████▎| 2326/2500 [13:52:57<1:01:01, 21.04s/it] {'loss': 0.0001, 'grad_norm': 0.015672622957953096, 'learning_rate': 6.959999999999999e-08, 'completion_length': 139.5446548461914, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003154754638671875, 'epoch': 0.93} 93%|█████████▎| 2326/2500 [13:52:57<1:01:01, 21.04s/it] 93%|█████████▎| 2327/2500 [13:53:18<1:00:41, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.47445271362002955, 'learning_rate': 6.92e-08, 'completion_length': 167.42858123779297, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.0071258544921875, 'epoch': 0.93} 93%|█████████▎| 2327/2500 [13:53:18<1:00:41, 21.05s/it] 93%|█████████▎| 2328/2500 [13:53:39<1:00:14, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.4657577335457096, 'learning_rate': 6.88e-08, 'completion_length': 161.2589340209961, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.004302978515625, 'epoch': 0.93} 93%|█████████▎| 2328/2500 [13:53:39<1:00:14, 21.01s/it] 93%|█████████▎| 2329/2500 [13:54:00<59:39, 20.93s/it] {'loss': 0.0001, 'grad_norm': 0.019129149492354135, 'learning_rate': 6.84e-08, 'completion_length': 148.0089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00345611572265625, 'epoch': 0.93} 93%|█████████▎| 2329/2500 [13:54:00<59:39, 20.93s/it] 93%|█████████▎| 2330/2500 [13:54:21<59:48, 21.11s/it] {'loss': 0.0003, 'grad_norm': 0.21359017132548383, 'learning_rate': 6.8e-08, 'completion_length': 157.7232208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.0068817138671875, 'epoch': 0.93} 93%|█████████▎| 2330/2500 [13:54:21<59:48, 21.11s/it] 93%|█████████▎| 2331/2500 [13:54:42<59:20, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.02193212582564428, 'learning_rate': 6.76e-08, 'completion_length': 152.2053680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00399017333984375, 'epoch': 0.93} 93%|█████████▎| 2331/2500 [13:54:42<59:20, 21.07s/it] 93%|█████████▎| 2332/2500 [13:55:04<59:09, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.38898467527111336, 'learning_rate': 6.719999999999999e-08, 'completion_length': 161.70536041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.06222161278128624, 'kl': 0.004852294921875, 'epoch': 0.93} 93%|█████████▎| 2332/2500 [13:55:04<59:09, 21.13s/it] 93%|█████████▎| 2333/2500 [13:55:25<58:59, 21.20s/it] {'loss': 0.0003, 'grad_norm': 0.05439201508663275, 'learning_rate': 6.679999999999999e-08, 'completion_length': 158.71428680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0070648193359375, 'epoch': 0.93} 93%|█████████▎| 2333/2500 [13:55:25<58:59, 21.20s/it] 93%|█████████▎| 2334/2500 [13:55:47<59:44, 21.59s/it] {'loss': 0.0003, 'grad_norm': 0.3812849388866582, 'learning_rate': 6.64e-08, 'completion_length': 159.04464721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.00653076171875, 'epoch': 0.93} 93%|█████████▎| 2334/2500 [13:55:47<59:44, 21.59s/it] 93%|█████████▎| 2335/2500 [13:56:09<59:16, 21.56s/it] {'loss': 0.0002, 'grad_norm': 0.30910522026099363, 'learning_rate': 6.6e-08, 'completion_length': 154.74108123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0052947998046875, 'epoch': 0.93} 93%|█████████▎| 2335/2500 [13:56:09<59:16, 21.56s/it] 93%|█████████▎| 2336/2500 [13:56:31<59:15, 21.68s/it] {'loss': 0.0003, 'grad_norm': 0.4477784366833237, 'learning_rate': 6.56e-08, 'completion_length': 161.1607208251953, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.05050762742757797, 'kl': 0.0072174072265625, 'epoch': 0.93} 93%|█████████▎| 2336/2500 [13:56:31<59:15, 21.68s/it] 93%|█████████▎| 2337/2500 [13:56:52<58:08, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.49225488121893024, 'learning_rate': 6.519999999999999e-08, 'completion_length': 152.21429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.0739355981349945, 'kl': 0.0050048828125, 'epoch': 0.93} 93%|█████████▎| 2337/2500 [13:56:52<58:08, 21.40s/it] 94%|█████████▎| 2338/2500 [13:57:12<57:13, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.029122635425727565, 'learning_rate': 6.48e-08, 'completion_length': 145.65179443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048065185546875, 'epoch': 0.94} 94%|█████████▎| 2338/2500 [13:57:12<57:13, 21.20s/it] 94%|█████████▎| 2339/2500 [13:57:34<56:51, 21.19s/it] {'loss': 0.0003, 'grad_norm': 0.365554065054777, 'learning_rate': 6.44e-08, 'completion_length': 167.68750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.0073394775390625, 'epoch': 0.94} 94%|█████████▎| 2339/2500 [13:57:34<56:51, 21.19s/it] 94%|█████████▎| 2340/2500 [13:57:55<56:29, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.4469994534068395, 'learning_rate': 6.4e-08, 'completion_length': 141.27679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00421905517578125, 'epoch': 0.94} 94%|█████████▎| 2340/2500 [13:57:55<56:29, 21.18s/it] 94%|█████████▎| 2341/2500 [13:58:16<56:11, 21.20s/it] {'loss': 0.0002, 'grad_norm': 1.0827854644197605, 'learning_rate': 6.36e-08, 'completion_length': 159.9732208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.004974365234375, 'epoch': 0.94} 94%|█████████▎| 2341/2500 [13:58:16<56:11, 21.20s/it] 94%|█████████▎| 2342/2500 [13:58:37<55:51, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.7747259207589026, 'learning_rate': 6.32e-08, 'completion_length': 162.58928680419922, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0064697265625, 'epoch': 0.94} 94%|█████████▎| 2342/2500 [13:58:37<55:51, 21.21s/it] 94%|█████████▎| 2343/2500 [13:58:58<55:06, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.4086625992693647, 'learning_rate': 6.279999999999999e-08, 'completion_length': 148.50000762939453, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.07124518603086472, 'kl': 0.0052947998046875, 'epoch': 0.94} 94%|█████████▎| 2343/2500 [13:58:58<55:06, 21.06s/it] 94%|█████████▍| 2344/2500 [13:59:19<55:04, 21.18s/it] {'loss': 0.0003, 'grad_norm': 0.019901029498116257, 'learning_rate': 6.239999999999999e-08, 'completion_length': 160.27678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00628662109375, 'epoch': 0.94} 94%|█████████▍| 2344/2500 [13:59:19<55:04, 21.18s/it] 94%|█████████▍| 2345/2500 [13:59:41<55:20, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.22977670977244985, 'learning_rate': 6.2e-08, 'completion_length': 165.06250762939453, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00543212890625, 'epoch': 0.94} 94%|█████████▍| 2345/2500 [13:59:41<55:20, 21.42s/it] 94%|█████████▍| 2346/2500 [14:00:02<54:27, 21.22s/it] {'loss': 0.0002, 'grad_norm': 0.018764104120957945, 'learning_rate': 6.16e-08, 'completion_length': 154.85714721679688, 'rewards/accuracy_reward': 0.8571429252624512, 'rewards/format_reward': 1.0, 'reward': 1.8571429252624512, 'reward_std': 0.0, 'kl': 0.00583648681640625, 'epoch': 0.94} 94%|█████████▍| 2346/2500 [14:00:02<54:27, 21.22s/it] 94%|█████████▍| 2347/2500 [14:00:24<54:26, 21.35s/it] {'loss': 0.0002, 'grad_norm': 0.02269898714221081, 'learning_rate': 6.119999999999999e-08, 'completion_length': 156.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0052642822265625, 'epoch': 0.94} 94%|█████████▍| 2347/2500 [14:00:24<54:26, 21.35s/it] 94%|█████████▍| 2348/2500 [14:00:45<53:46, 21.23s/it] {'loss': 0.0001, 'grad_norm': 0.017178107150874217, 'learning_rate': 6.08e-08, 'completion_length': 155.08036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032196044921875, 'epoch': 0.94} 94%|█████████▍| 2348/2500 [14:00:45<53:46, 21.23s/it] 94%|█████████▍| 2349/2500 [14:01:06<53:24, 21.22s/it] {'loss': 0.0003, 'grad_norm': 0.018285782943881043, 'learning_rate': 6.04e-08, 'completion_length': 157.48214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0063018798828125, 'epoch': 0.94} 94%|█████████▍| 2349/2500 [14:01:06<53:24, 21.22s/it] 94%|█████████▍| 2350/2500 [14:01:28<53:28, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.46837495988293715, 'learning_rate': 6e-08, 'completion_length': 172.49107360839844, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.0082550048828125, 'epoch': 0.94} 94%|█████████▍| 2350/2500 [14:01:28<53:28, 21.39s/it] 94%|█████████▍| 2351/2500 [14:01:49<53:11, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.1513776779026397, 'learning_rate': 5.96e-08, 'completion_length': 157.92858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005401611328125, 'epoch': 0.94} 94%|█████████▍| 2351/2500 [14:01:49<53:11, 21.42s/it] 94%|█████████▍| 2352/2500 [14:02:10<52:45, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.03855341997929401, 'learning_rate': 5.92e-08, 'completion_length': 164.1607208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00439453125, 'epoch': 0.94} 94%|█████████▍| 2352/2500 [14:02:10<52:45, 21.39s/it] 94%|█████████▍| 2353/2500 [14:02:31<51:56, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.3132123574795434, 'learning_rate': 5.88e-08, 'completion_length': 145.93750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005950927734375, 'epoch': 0.94} 94%|█████████▍| 2353/2500 [14:02:31<51:56, 21.20s/it] 94%|█████████▍| 2354/2500 [14:02:52<51:28, 21.15s/it] {'loss': 0.0002, 'grad_norm': 0.47379234148504557, 'learning_rate': 5.84e-08, 'completion_length': 158.3125, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.004791259765625, 'epoch': 0.94} 94%|█████████▍| 2354/2500 [14:02:52<51:28, 21.15s/it] 94%|█████████▍| 2355/2500 [14:03:13<50:45, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.4532189890025895, 'learning_rate': 5.8e-08, 'completion_length': 141.9732208251953, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0065765380859375, 'epoch': 0.94} 94%|█████████▍| 2355/2500 [14:03:13<50:45, 21.00s/it] 94%|█████████▍| 2356/2500 [14:03:34<50:33, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.28693182660142613, 'learning_rate': 5.759999999999999e-08, 'completion_length': 147.3482208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00392913818359375, 'epoch': 0.94} 94%|█████████▍| 2356/2500 [14:03:34<50:33, 21.06s/it] 94%|█████████▍| 2357/2500 [14:03:55<49:49, 20.90s/it] {'loss': 0.0002, 'grad_norm': 0.23348126554517992, 'learning_rate': 5.7199999999999996e-08, 'completion_length': 152.8303680419922, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00403594970703125, 'epoch': 0.94} 94%|█████████▍| 2357/2500 [14:03:55<49:49, 20.90s/it] 94%|█████████▍| 2358/2500 [14:04:15<49:23, 20.87s/it] {'loss': 0.0002, 'grad_norm': 0.02651433019395703, 'learning_rate': 5.68e-08, 'completion_length': 148.9107208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054168701171875, 'epoch': 0.94} 94%|█████████▍| 2358/2500 [14:04:15<49:23, 20.87s/it] 94%|█████████▍| 2359/2500 [14:04:36<49:00, 20.86s/it] {'loss': 0.0002, 'grad_norm': 0.028185599601512318, 'learning_rate': 5.6399999999999995e-08, 'completion_length': 146.0803680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00445556640625, 'epoch': 0.94} 94%|█████████▍| 2359/2500 [14:04:36<49:00, 20.86s/it] 94%|█████████▍| 2360/2500 [14:04:58<49:34, 21.24s/it] {'loss': 0.0003, 'grad_norm': 0.016124034803228802, 'learning_rate': 5.6e-08, 'completion_length': 163.0803680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0070343017578125, 'epoch': 0.94} 94%|█████████▍| 2360/2500 [14:04:58<49:34, 21.24s/it] 94%|█████████▍| 2361/2500 [14:05:19<48:46, 21.06s/it] {'loss': 0.0002, 'grad_norm': 0.11342909299022362, 'learning_rate': 5.5599999999999995e-08, 'completion_length': 146.79464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00382232666015625, 'epoch': 0.94} 94%|█████████▍| 2361/2500 [14:05:19<48:46, 21.06s/it] 94%|█████████▍| 2362/2500 [14:05:40<48:37, 21.14s/it] {'loss': 0.0003, 'grad_norm': 0.23050721878619065, 'learning_rate': 5.52e-08, 'completion_length': 164.62500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0082244873046875, 'epoch': 0.94} 94%|█████████▍| 2362/2500 [14:05:40<48:37, 21.14s/it] 95%|█████████▍| 2363/2500 [14:06:02<48:20, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.2618484918725073, 'learning_rate': 5.48e-08, 'completion_length': 158.35714721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.003814697265625, 'epoch': 0.95} 95%|█████████▍| 2363/2500 [14:06:02<48:20, 21.17s/it] 95%|█████████▍| 2364/2500 [14:06:23<48:05, 21.21s/it] {'loss': 0.0003, 'grad_norm': 0.23082608862032875, 'learning_rate': 5.44e-08, 'completion_length': 178.3839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.007080078125, 'epoch': 0.95} 95%|█████████▍| 2364/2500 [14:06:23<48:05, 21.21s/it] 95%|█████████▍| 2365/2500 [14:06:44<47:37, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.19916306813931392, 'learning_rate': 5.3999999999999994e-08, 'completion_length': 159.27679443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.005218505859375, 'epoch': 0.95} 95%|█████████▍| 2365/2500 [14:06:44<47:37, 21.17s/it] 95%|█████████▍| 2366/2500 [14:07:05<47:08, 21.11s/it] {'loss': 0.0002, 'grad_norm': 0.01613379531009966, 'learning_rate': 5.36e-08, 'completion_length': 146.8928680419922, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00457763671875, 'epoch': 0.95} 95%|█████████▍| 2366/2500 [14:07:05<47:08, 21.11s/it] 95%|█████████▍| 2367/2500 [14:07:26<46:39, 21.05s/it] {'loss': 0.0001, 'grad_norm': 0.2178023400796567, 'learning_rate': 5.319999999999999e-08, 'completion_length': 146.62500762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00368499755859375, 'epoch': 0.95} 95%|█████████▍| 2367/2500 [14:07:26<46:39, 21.05s/it] 95%|█████████▍| 2368/2500 [14:07:47<46:03, 20.93s/it] {'loss': 0.0001, 'grad_norm': 0.01981148486968921, 'learning_rate': 5.2799999999999996e-08, 'completion_length': 156.8839340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0034332275390625, 'epoch': 0.95} 95%|█████████▍| 2368/2500 [14:07:47<46:03, 20.93s/it] 95%|█████████▍| 2369/2500 [14:08:07<45:35, 20.88s/it] {'loss': 0.0003, 'grad_norm': 0.6808555350736408, 'learning_rate': 5.24e-08, 'completion_length': 150.43750762939453, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.006500244140625, 'epoch': 0.95} 95%|█████████▍| 2369/2500 [14:08:07<45:35, 20.88s/it] 95%|█████████▍| 2370/2500 [14:08:29<45:32, 21.02s/it] {'loss': 0.0003, 'grad_norm': 0.2732209621581238, 'learning_rate': 5.1999999999999996e-08, 'completion_length': 164.54464721679688, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.0067596435546875, 'epoch': 0.95} 95%|█████████▍| 2370/2500 [14:08:29<45:32, 21.02s/it] 95%|█████████▍| 2371/2500 [14:08:50<45:42, 21.26s/it] {'loss': 0.0003, 'grad_norm': 0.4295440150088229, 'learning_rate': 5.16e-08, 'completion_length': 159.10714721679688, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017858505249023, 'reward_std': 0.05831881985068321, 'kl': 0.0067291259765625, 'epoch': 0.95} 95%|█████████▍| 2371/2500 [14:08:50<45:42, 21.26s/it] 95%|█████████▍| 2372/2500 [14:09:11<44:50, 21.02s/it] {'loss': 0.0001, 'grad_norm': 0.2264080376903053, 'learning_rate': 5.12e-08, 'completion_length': 140.48214721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00359344482421875, 'epoch': 0.95} 95%|█████████▍| 2372/2500 [14:09:11<44:50, 21.02s/it] 95%|█████████▍| 2373/2500 [14:09:32<44:44, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.025175830492016334, 'learning_rate': 5.08e-08, 'completion_length': 151.00000762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00457000732421875, 'epoch': 0.95} 95%|█████████▍| 2373/2500 [14:09:32<44:44, 21.14s/it] 95%|█████████▍| 2374/2500 [14:09:53<44:17, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.76421636985433, 'learning_rate': 5.04e-08, 'completion_length': 153.93750762939453, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00377655029296875, 'epoch': 0.95} 95%|█████████▍| 2374/2500 [14:09:53<44:17, 21.09s/it] 95%|█████████▌| 2375/2500 [14:10:14<43:37, 20.94s/it] {'loss': 0.0002, 'grad_norm': 0.016674637200067295, 'learning_rate': 5e-08, 'completion_length': 153.5982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003997802734375, 'epoch': 0.95} 95%|█████████▌| 2375/2500 [14:10:14<43:37, 20.94s/it] 95%|█████████▌| 2376/2500 [14:10:35<43:26, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.28760032992202994, 'learning_rate': 4.9599999999999994e-08, 'completion_length': 155.15179443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.0041961669921875, 'epoch': 0.95} 95%|█████████▌| 2376/2500 [14:10:35<43:26, 21.02s/it] 95%|█████████▌| 2377/2500 [14:10:56<43:11, 21.07s/it] {'loss': 0.0003, 'grad_norm': 0.6232855730538611, 'learning_rate': 4.92e-08, 'completion_length': 145.96428680419922, 'rewards/accuracy_reward': 0.9017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9017857313156128, 'reward_std': 0.05831882357597351, 'kl': 0.007354736328125, 'epoch': 0.95} 95%|█████████▌| 2377/2500 [14:10:56<43:11, 21.07s/it] 95%|█████████▌| 2378/2500 [14:11:18<43:06, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.4697300305478976, 'learning_rate': 4.88e-08, 'completion_length': 158.74107360839844, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797896146774, 'kl': 0.0058441162109375, 'epoch': 0.95} 95%|█████████▌| 2378/2500 [14:11:18<43:06, 21.20s/it] 95%|█████████▌| 2379/2500 [14:11:39<42:53, 21.27s/it] {'loss': 0.0003, 'grad_norm': 0.6673389349737272, 'learning_rate': 4.8399999999999997e-08, 'completion_length': 158.68750762939453, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.07514797151088715, 'kl': 0.007110595703125, 'epoch': 0.95} 95%|█████████▌| 2379/2500 [14:11:39<42:53, 21.27s/it] 95%|█████████▌| 2380/2500 [14:12:00<42:29, 21.25s/it] {'loss': 0.0003, 'grad_norm': 0.37802605695013936, 'learning_rate': 4.8e-08, 'completion_length': 155.6071548461914, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.0075836181640625, 'epoch': 0.95} 95%|█████████▌| 2380/2500 [14:12:00<42:29, 21.25s/it] 95%|█████████▌| 2381/2500 [14:12:22<42:06, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.5752790666282602, 'learning_rate': 4.76e-08, 'completion_length': 148.96428680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.09138382971286774, 'kl': 0.00449371337890625, 'epoch': 0.95} 95%|█████████▌| 2381/2500 [14:12:22<42:06, 21.23s/it] 95%|█████████▌| 2382/2500 [14:12:43<41:38, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.04529778924494214, 'learning_rate': 4.72e-08, 'completion_length': 153.31250762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.003997802734375, 'epoch': 0.95} 95%|█████████▌| 2382/2500 [14:12:43<41:38, 21.18s/it] 95%|█████████▌| 2383/2500 [14:13:05<42:03, 21.57s/it] {'loss': 0.0004, 'grad_norm': 0.4076900648258715, 'learning_rate': 4.68e-08, 'completion_length': 164.50000762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.009063720703125, 'epoch': 0.95} 95%|█████████▌| 2383/2500 [14:13:05<42:03, 21.57s/it] 95%|█████████▌| 2384/2500 [14:13:26<41:21, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.6766738985217177, 'learning_rate': 4.639999999999999e-08, 'completion_length': 150.79464721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00411224365234375, 'epoch': 0.95} 95%|█████████▌| 2384/2500 [14:13:26<41:21, 21.39s/it] 95%|█████████▌| 2385/2500 [14:13:48<41:14, 21.52s/it] {'loss': 0.0003, 'grad_norm': 0.332424772679096, 'learning_rate': 4.5999999999999995e-08, 'completion_length': 155.81250762939453, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0072021484375, 'epoch': 0.95} 95%|█████████▌| 2385/2500 [14:13:48<41:14, 21.52s/it] 95%|█████████▌| 2386/2500 [14:14:09<40:43, 21.43s/it] {'loss': 0.0002, 'grad_norm': 0.5994225426731378, 'learning_rate': 4.56e-08, 'completion_length': 141.3482208251953, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.0047454833984375, 'epoch': 0.95} 95%|█████████▌| 2386/2500 [14:14:09<40:43, 21.43s/it] 95%|█████████▌| 2387/2500 [14:14:30<40:01, 21.25s/it] {'loss': 0.0002, 'grad_norm': 0.01686774847997247, 'learning_rate': 4.5199999999999994e-08, 'completion_length': 146.2678680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00449371337890625, 'epoch': 0.95} 95%|█████████▌| 2387/2500 [14:14:30<40:01, 21.25s/it] 96%|█████████▌| 2388/2500 [14:14:52<40:08, 21.50s/it] {'loss': 0.0003, 'grad_norm': 0.31695407884562504, 'learning_rate': 4.48e-08, 'completion_length': 176.14286041259766, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.0066070556640625, 'epoch': 0.96} 96%|█████████▌| 2388/2500 [14:14:52<40:08, 21.50s/it] 96%|█████████▌| 2389/2500 [14:15:13<39:34, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.21968542052468112, 'learning_rate': 4.44e-08, 'completion_length': 156.0714340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052032470703125, 'epoch': 0.96} 96%|█████████▌| 2389/2500 [14:15:13<39:34, 21.40s/it] 96%|█████████▌| 2390/2500 [14:15:34<38:57, 21.25s/it] {'loss': 0.0001, 'grad_norm': 0.02552052585815189, 'learning_rate': 4.4e-08, 'completion_length': 148.30357360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032196044921875, 'epoch': 0.96} 96%|█████████▌| 2390/2500 [14:15:34<38:57, 21.25s/it] 96%|█████████▌| 2391/2500 [14:15:55<38:31, 21.21s/it] {'loss': 0.0001, 'grad_norm': 0.042445438904293864, 'learning_rate': 4.36e-08, 'completion_length': 142.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032501220703125, 'epoch': 0.96} 96%|█████████▌| 2391/2500 [14:15:55<38:31, 21.21s/it] 96%|█████████▌| 2392/2500 [14:16:17<38:19, 21.29s/it] {'loss': 0.0002, 'grad_norm': 1.0652122695807407, 'learning_rate': 4.32e-08, 'completion_length': 156.83036041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.005462646484375, 'epoch': 0.96} 96%|█████████▌| 2392/2500 [14:16:17<38:19, 21.29s/it] 96%|█████████▌| 2393/2500 [14:16:38<38:03, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.03419434591898831, 'learning_rate': 4.279999999999999e-08, 'completion_length': 161.98214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005828857421875, 'epoch': 0.96} 96%|█████████▌| 2393/2500 [14:16:38<38:03, 21.34s/it] 96%|█████████▌| 2394/2500 [14:17:00<37:41, 21.34s/it] {'loss': 0.0002, 'grad_norm': 0.4193971583236635, 'learning_rate': 4.2399999999999996e-08, 'completion_length': 152.6428680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.005828857421875, 'epoch': 0.96} 96%|█████████▌| 2394/2500 [14:17:00<37:41, 21.34s/it] 96%|█████████▌| 2395/2500 [14:17:21<37:33, 21.46s/it] {'loss': 0.0002, 'grad_norm': 0.02257704641307597, 'learning_rate': 4.2e-08, 'completion_length': 160.87500762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006195068359375, 'epoch': 0.96} 96%|█████████▌| 2395/2500 [14:17:21<37:33, 21.46s/it] 96%|█████████▌| 2396/2500 [14:17:42<36:54, 21.29s/it] {'loss': 0.0002, 'grad_norm': 0.2245068537894557, 'learning_rate': 4.1599999999999995e-08, 'completion_length': 150.30358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0048675537109375, 'epoch': 0.96} 96%|█████████▌| 2396/2500 [14:17:42<36:54, 21.29s/it] 96%|█████████▌| 2397/2500 [14:18:04<36:59, 21.54s/it] {'loss': 0.0002, 'grad_norm': 0.03618295138270471, 'learning_rate': 4.12e-08, 'completion_length': 158.41964721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0054779052734375, 'epoch': 0.96} 96%|█████████▌| 2397/2500 [14:18:04<36:59, 21.54s/it] 96%|█████████▌| 2398/2500 [14:18:25<36:24, 21.42s/it] {'loss': 0.0002, 'grad_norm': 0.030291790164687468, 'learning_rate': 4.08e-08, 'completion_length': 153.21429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004547119140625, 'epoch': 0.96} 96%|█████████▌| 2398/2500 [14:18:25<36:24, 21.42s/it] 96%|█████████▌| 2399/2500 [14:18:46<35:38, 21.17s/it] {'loss': 0.0003, 'grad_norm': 0.2254186694595719, 'learning_rate': 4.04e-08, 'completion_length': 159.87500762939453, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.006866455078125, 'epoch': 0.96} 96%|█████████▌| 2399/2500 [14:18:46<35:38, 21.17s/it] 96%|█████████▌| 2400/2500 [14:19:08<35:40, 21.40s/it] {'loss': 0.0002, 'grad_norm': 0.032495840143271414, 'learning_rate': 4e-08, 'completion_length': 148.7232208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00403594970703125, 'epoch': 0.96} 96%|█████████▌| 2400/2500 [14:19:08<35:40, 21.40s/it] 96%|█████████▌| 2401/2500 [14:20:08<54:37, 33.10s/it] {'loss': 0.0002, 'grad_norm': 0.016903861931145566, 'learning_rate': 3.9600000000000004e-08, 'completion_length': 162.17857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00470733642578125, 'epoch': 0.96} 96%|█████████▌| 2401/2500 [14:20:08<54:37, 33.10s/it] 96%|█████████▌| 2402/2500 [14:20:19<42:48, 26.21s/it] {'loss': 0.0002, 'grad_norm': 0.262675645674279, 'learning_rate': 3.9199999999999994e-08, 'completion_length': 147.0357208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.004364013671875, 'epoch': 0.96} 96%|█████████▌| 2402/2500 [14:20:19<42:48, 26.21s/it] 96%|█████████▌| 2403/2500 [14:20:29<34:40, 21.45s/it] {'loss': 0.0002, 'grad_norm': 0.271473712793209, 'learning_rate': 3.88e-08, 'completion_length': 157.83036041259766, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006134033203125, 'epoch': 0.96} 96%|█████████▌| 2403/2500 [14:20:29<34:40, 21.45s/it] 96%|█████████▌| 2404/2500 [14:20:39<28:58, 18.11s/it] {'loss': 0.0002, 'grad_norm': 0.1773085909469583, 'learning_rate': 3.839999999999999e-08, 'completion_length': 155.67858123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0045013427734375, 'epoch': 0.96} 96%|█████████▌| 2404/2500 [14:20:39<28:58, 18.11s/it] 96%|█████████▌| 2405/2500 [14:20:51<25:28, 16.09s/it] {'loss': 0.0002, 'grad_norm': 0.6915189892882435, 'learning_rate': 3.7999999999999996e-08, 'completion_length': 164.70536041259766, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.10700060427188873, 'kl': 0.005950927734375, 'epoch': 0.96} 96%|█████████▌| 2405/2500 [14:20:51<25:28, 16.09s/it] 96%|█████████▌| 2406/2500 [14:21:01<22:24, 14.30s/it] {'loss': 0.0002, 'grad_norm': 0.02054882997645921, 'learning_rate': 3.76e-08, 'completion_length': 152.9732208251953, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00525665283203125, 'epoch': 0.96} 96%|█████████▌| 2406/2500 [14:21:01<22:24, 14.30s/it] 96%|█████████▋| 2407/2500 [14:21:12<20:40, 13.33s/it] {'loss': 0.0002, 'grad_norm': 0.1076366714238981, 'learning_rate': 3.7199999999999996e-08, 'completion_length': 165.3214340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.006134033203125, 'epoch': 0.96} 96%|█████████▋| 2407/2500 [14:21:12<20:40, 13.33s/it] 96%|█████████▋| 2408/2500 [14:21:22<18:47, 12.26s/it] {'loss': 0.0001, 'grad_norm': 0.29317087026709215, 'learning_rate': 3.68e-08, 'completion_length': 141.86607360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00292205810546875, 'epoch': 0.96} 96%|█████████▋| 2408/2500 [14:21:22<18:47, 12.26s/it] 96%|█████████▋| 2409/2500 [14:21:32<17:34, 11.59s/it] {'loss': 0.0002, 'grad_norm': 0.21680067963610639, 'learning_rate': 3.64e-08, 'completion_length': 148.4732208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0058135986328125, 'epoch': 0.96} 96%|█████████▋| 2409/2500 [14:21:32<17:34, 11.59s/it] 96%|█████████▋| 2410/2500 [14:21:42<17:02, 11.36s/it] {'loss': 0.0002, 'grad_norm': 0.02132138158908525, 'learning_rate': 3.6e-08, 'completion_length': 149.7053680419922, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00482177734375, 'epoch': 0.96} 96%|█████████▋| 2410/2500 [14:21:42<17:02, 11.36s/it] 96%|█████████▋| 2411/2500 [14:21:53<16:22, 11.03s/it] {'loss': 0.0002, 'grad_norm': 0.06168619753608493, 'learning_rate': 3.56e-08, 'completion_length': 161.7589340209961, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0059661865234375, 'epoch': 0.96} 96%|█████████▋| 2411/2500 [14:21:53<16:22, 11.03s/it] 96%|█████████▋| 2412/2500 [14:22:03<15:50, 10.80s/it] {'loss': 0.0003, 'grad_norm': 0.5888364959391911, 'learning_rate': 3.52e-08, 'completion_length': 151.8214340209961, 'rewards/accuracy_reward': 0.8750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.8750000596046448, 'reward_std': 0.0739355981349945, 'kl': 0.0065765380859375, 'epoch': 0.96} 96%|█████████▋| 2412/2500 [14:22:03<15:50, 10.80s/it] 97%|█████████▋| 2413/2500 [14:22:13<15:26, 10.65s/it] {'loss': 0.0003, 'grad_norm': 0.6154568932620706, 'learning_rate': 3.4799999999999994e-08, 'completion_length': 154.93750762939453, 'rewards/accuracy_reward': 0.928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.9285715222358704, 'reward_std': 0.12444322928786278, 'kl': 0.00653076171875, 'epoch': 0.97} 97%|█████████▋| 2413/2500 [14:22:13<15:26, 10.65s/it] 97%|█████████▋| 2414/2500 [14:22:24<15:12, 10.61s/it] {'loss': 0.0004, 'grad_norm': 0.7219639899882758, 'learning_rate': 3.44e-08, 'completion_length': 149.76786041259766, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726335644722, 'kl': 0.0088653564453125, 'epoch': 0.97} 97%|█████████▋| 2414/2500 [14:22:24<15:12, 10.61s/it] 97%|█████████▋| 2415/2500 [14:22:34<14:53, 10.51s/it] {'loss': 0.0002, 'grad_norm': 0.02183536447336196, 'learning_rate': 3.4e-08, 'completion_length': 154.79464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0042724609375, 'epoch': 0.97} 97%|█████████▋| 2415/2500 [14:22:34<14:53, 10.51s/it] 97%|█████████▋| 2416/2500 [14:22:44<14:34, 10.41s/it] {'loss': 0.0002, 'grad_norm': 0.7450576035238473, 'learning_rate': 3.3599999999999996e-08, 'completion_length': 145.8482208251953, 'rewards/accuracy_reward': 0.9553571939468384, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343399360775948, 'kl': 0.0058746337890625, 'epoch': 0.97} 97%|█████████▋| 2416/2500 [14:22:44<14:34, 10.41s/it] 97%|█████████▋| 2417/2500 [14:22:54<14:12, 10.27s/it] {'loss': 0.0002, 'grad_norm': 0.01904472713199284, 'learning_rate': 3.32e-08, 'completion_length': 153.04464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00400543212890625, 'epoch': 0.97} 97%|█████████▋| 2417/2500 [14:22:54<14:12, 10.27s/it] 97%|█████████▋| 2418/2500 [14:23:05<14:09, 10.36s/it] {'loss': 0.0002, 'grad_norm': 0.39647815349475346, 'learning_rate': 3.28e-08, 'completion_length': 155.5089340209961, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0045928955078125, 'epoch': 0.97} 97%|█████████▋| 2418/2500 [14:23:05<14:09, 10.36s/it] 97%|█████████▋| 2419/2500 [14:23:15<13:56, 10.32s/it] {'loss': 0.0001, 'grad_norm': 0.01979804336741467, 'learning_rate': 3.24e-08, 'completion_length': 145.21429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00279998779296875, 'epoch': 0.97} 97%|█████████▋| 2419/2500 [14:23:15<13:56, 10.32s/it] 97%|█████████▋| 2420/2500 [14:23:25<13:44, 10.31s/it] {'loss': 0.0003, 'grad_norm': 0.5125948355786821, 'learning_rate': 3.2e-08, 'completion_length': 154.20536041259766, 'rewards/accuracy_reward': 0.866071492433548, 'rewards/format_reward': 1.0, 'reward': 1.8660715222358704, 'reward_std': 0.025253813713788986, 'kl': 0.0081329345703125, 'epoch': 0.97} 97%|█████████▋| 2420/2500 [14:23:25<13:44, 10.31s/it] 97%|█████████▋| 2421/2500 [14:23:35<13:20, 10.13s/it] {'loss': 0.0001, 'grad_norm': 0.0167260314939699, 'learning_rate': 3.16e-08, 'completion_length': 146.67857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00325775146484375, 'epoch': 0.97} 97%|█████████▋| 2421/2500 [14:23:35<13:20, 10.13s/it] 97%|█████████▋| 2422/2500 [14:23:45<13:17, 10.22s/it] {'loss': 0.0001, 'grad_norm': 1.0659263493730666, 'learning_rate': 3.1199999999999995e-08, 'completion_length': 152.17858123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00353240966796875, 'epoch': 0.97} 97%|█████████▋| 2422/2500 [14:23:45<13:17, 10.22s/it] 97%|█████████▋| 2423/2500 [14:24:00<14:57, 11.65s/it] {'loss': 0.0002, 'grad_norm': 1.08180140317922, 'learning_rate': 3.08e-08, 'completion_length': 154.54464721679688, 'rewards/accuracy_reward': 0.9821429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00514984130859375, 'epoch': 0.97} 97%|█████████▋| 2423/2500 [14:24:00<14:57, 11.65s/it] 97%|█████████▋| 2424/2500 [14:24:22<18:44, 14.80s/it] {'loss': 0.0002, 'grad_norm': 0.42138468923903094, 'learning_rate': 3.04e-08, 'completion_length': 177.6964340209961, 'rewards/accuracy_reward': 0.9107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.910714328289032, 'reward_std': 0.033065006136894226, 'kl': 0.006103515625, 'epoch': 0.97} 97%|█████████▋| 2424/2500 [14:24:22<18:44, 14.80s/it] 97%|█████████▋| 2425/2500 [14:24:43<20:40, 16.53s/it] {'loss': 0.0002, 'grad_norm': 0.060246827114495874, 'learning_rate': 3e-08, 'completion_length': 160.26786041259766, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00537872314453125, 'epoch': 0.97} 97%|█████████▋| 2425/2500 [14:24:43<20:40, 16.53s/it] 97%|█████████▋| 2426/2500 [14:25:04<21:53, 17.75s/it] {'loss': 0.0002, 'grad_norm': 0.2714465585867484, 'learning_rate': 2.96e-08, 'completion_length': 153.33929443359375, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.03818017989397049, 'kl': 0.00449371337890625, 'epoch': 0.97} 97%|█████████▋| 2426/2500 [14:25:04<21:53, 17.75s/it] 97%|█████████▋| 2427/2500 [14:25:25<22:57, 18.87s/it] {'loss': 0.0003, 'grad_norm': 1.686125536008025, 'learning_rate': 2.92e-08, 'completion_length': 179.82144165039062, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.9464285969734192, 'reward_std': 0.033065006136894226, 'kl': 0.006591796875, 'epoch': 0.97} 97%|█████████▋| 2427/2500 [14:25:25<22:57, 18.87s/it] 97%|█████████▋| 2428/2500 [14:25:46<23:25, 19.52s/it] {'loss': 0.0002, 'grad_norm': 0.7050539225884589, 'learning_rate': 2.8799999999999996e-08, 'completion_length': 146.4464340209961, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0046234130859375, 'epoch': 0.97} 97%|█████████▋| 2428/2500 [14:25:46<23:25, 19.52s/it] 97%|█████████▋| 2429/2500 [14:26:07<23:26, 19.81s/it] {'loss': 0.0004, 'grad_norm': 0.3985784300608268, 'learning_rate': 2.84e-08, 'completion_length': 158.5357208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.05831882357597351, 'kl': 0.00885009765625, 'epoch': 0.97} 97%|█████████▋| 2429/2500 [14:26:07<23:26, 19.81s/it] 97%|█████████▋| 2430/2500 [14:26:28<23:29, 20.14s/it] {'loss': 0.0002, 'grad_norm': 0.02486031316899893, 'learning_rate': 2.8e-08, 'completion_length': 160.60714721679688, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0060272216796875, 'epoch': 0.97} 97%|█████████▋| 2430/2500 [14:26:28<23:29, 20.14s/it] 97%|█████████▋| 2431/2500 [14:26:48<23:21, 20.30s/it] {'loss': 0.0002, 'grad_norm': 0.2993870095932978, 'learning_rate': 2.76e-08, 'completion_length': 135.66964721679688, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00437164306640625, 'epoch': 0.97} 97%|█████████▋| 2431/2500 [14:26:48<23:21, 20.30s/it] 97%|█████████▋| 2432/2500 [14:27:09<23:04, 20.36s/it] {'loss': 0.0002, 'grad_norm': 0.4050725380992832, 'learning_rate': 2.72e-08, 'completion_length': 150.65179443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.005035400390625, 'epoch': 0.97} 97%|█████████▋| 2432/2500 [14:27:09<23:04, 20.36s/it] 97%|█████████▋| 2433/2500 [14:27:30<22:53, 20.50s/it] {'loss': 0.0002, 'grad_norm': 0.021205158704390614, 'learning_rate': 2.68e-08, 'completion_length': 151.48214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00380706787109375, 'epoch': 0.97} 97%|█████████▋| 2433/2500 [14:27:30<22:53, 20.50s/it] 97%|█████████▋| 2434/2500 [14:27:51<22:43, 20.65s/it] {'loss': 0.0003, 'grad_norm': 0.30020351123141914, 'learning_rate': 2.6399999999999998e-08, 'completion_length': 172.58928680419922, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.05050762742757797, 'kl': 0.00799560546875, 'epoch': 0.97} 97%|█████████▋| 2434/2500 [14:27:51<22:43, 20.65s/it] 97%|█████████▋| 2435/2500 [14:28:11<22:19, 20.61s/it] {'loss': 0.0002, 'grad_norm': 0.6763119074278878, 'learning_rate': 2.5999999999999998e-08, 'completion_length': 153.8214340209961, 'rewards/accuracy_reward': 0.9464286267757416, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.06222161278128624, 'kl': 0.0047760009765625, 'epoch': 0.97} 97%|█████████▋| 2435/2500 [14:28:11<22:19, 20.61s/it] 97%|█████████▋| 2436/2500 [14:28:32<21:57, 20.59s/it] {'loss': 0.0002, 'grad_norm': 0.014323333426579495, 'learning_rate': 2.56e-08, 'completion_length': 153.42858123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00555419921875, 'epoch': 0.97} 97%|█████████▋| 2436/2500 [14:28:32<21:57, 20.59s/it] 97%|█████████▋| 2437/2500 [14:28:53<21:48, 20.78s/it] {'loss': 0.0002, 'grad_norm': 0.015253913289468365, 'learning_rate': 2.52e-08, 'completion_length': 163.0982208251953, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00473785400390625, 'epoch': 0.97} 97%|█████████▋| 2437/2500 [14:28:53<21:48, 20.78s/it] 98%|█████████▊| 2438/2500 [14:29:13<21:20, 20.66s/it] {'loss': 0.0003, 'grad_norm': 0.18611194202556203, 'learning_rate': 2.4799999999999997e-08, 'completion_length': 158.80358123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0068817138671875, 'epoch': 0.98} 98%|█████████▊| 2438/2500 [14:29:13<21:20, 20.66s/it] 98%|█████████▊| 2439/2500 [14:29:34<21:04, 20.73s/it] {'loss': 0.0002, 'grad_norm': 0.20646982713266382, 'learning_rate': 2.44e-08, 'completion_length': 151.8928680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0053558349609375, 'epoch': 0.98} 98%|█████████▊| 2439/2500 [14:29:34<21:04, 20.73s/it] 98%|█████████▊| 2440/2500 [14:29:55<20:37, 20.63s/it] {'loss': 0.0002, 'grad_norm': 0.32439344651164875, 'learning_rate': 2.4e-08, 'completion_length': 149.33929443359375, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0060577392578125, 'epoch': 0.98} 98%|█████████▊| 2440/2500 [14:29:55<20:37, 20.63s/it] 98%|█████████▊| 2441/2500 [14:30:17<20:55, 21.27s/it] {'loss': 0.0003, 'grad_norm': 1.8825710600473378, 'learning_rate': 2.36e-08, 'completion_length': 166.2589340209961, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.07576144114136696, 'kl': 0.00714111328125, 'epoch': 0.98} 98%|█████████▊| 2441/2500 [14:30:17<20:55, 21.27s/it] 98%|█████████▊| 2442/2500 [14:30:39<20:35, 21.31s/it] {'loss': 0.0002, 'grad_norm': 0.4629393382092003, 'learning_rate': 2.3199999999999996e-08, 'completion_length': 153.0357208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.004791259765625, 'epoch': 0.98} 98%|█████████▊| 2442/2500 [14:30:39<20:35, 21.31s/it] 98%|█████████▊| 2443/2500 [14:31:00<20:22, 21.44s/it] {'loss': 0.0002, 'grad_norm': 0.23223127365777962, 'learning_rate': 2.28e-08, 'completion_length': 151.5089340209961, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00489044189453125, 'epoch': 0.98} 98%|█████████▊| 2443/2500 [14:31:00<20:22, 21.44s/it] 98%|█████████▊| 2444/2500 [14:31:22<20:00, 21.44s/it] {'loss': 0.0003, 'grad_norm': 0.3519487804820346, 'learning_rate': 2.24e-08, 'completion_length': 158.80358123779297, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642858505249023, 'reward_std': 0.06613001227378845, 'kl': 0.00762939453125, 'epoch': 0.98} 98%|█████████▊| 2444/2500 [14:31:22<20:00, 21.44s/it] 98%|█████████▊| 2445/2500 [14:31:43<19:34, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.3271043918987324, 'learning_rate': 2.2e-08, 'completion_length': 162.89286041259766, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0071868896484375, 'epoch': 0.98} 98%|█████████▊| 2445/2500 [14:31:43<19:34, 21.35s/it] 98%|█████████▊| 2446/2500 [14:32:04<19:00, 21.12s/it] {'loss': 0.0002, 'grad_norm': 0.6024001660338293, 'learning_rate': 2.16e-08, 'completion_length': 138.43750762939453, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00449371337890625, 'epoch': 0.98} 98%|█████████▊| 2446/2500 [14:32:04<19:00, 21.12s/it] 98%|█████████▊| 2447/2500 [14:32:24<18:29, 20.94s/it] {'loss': 0.0001, 'grad_norm': 0.021665868932683473, 'learning_rate': 2.1199999999999998e-08, 'completion_length': 130.33036041259766, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0032958984375, 'epoch': 0.98} 98%|█████████▊| 2447/2500 [14:32:24<18:29, 20.94s/it] 98%|█████████▊| 2448/2500 [14:32:45<18:15, 21.07s/it] {'loss': 0.0002, 'grad_norm': 0.5012671673472713, 'learning_rate': 2.0799999999999998e-08, 'completion_length': 153.58929443359375, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.06343398988246918, 'kl': 0.005859375, 'epoch': 0.98} 98%|█████████▊| 2448/2500 [14:32:45<18:15, 21.07s/it] 98%|█████████▊| 2449/2500 [14:33:07<17:55, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.4065482823196849, 'learning_rate': 2.04e-08, 'completion_length': 170.77679443359375, 'rewards/accuracy_reward': 0.973214328289032, 'rewards/format_reward': 1.0, 'reward': 1.9732143878936768, 'reward_std': 0.05831881985068321, 'kl': 0.006103515625, 'epoch': 0.98} 98%|█████████▊| 2449/2500 [14:33:07<17:55, 21.10s/it] 98%|█████████▊| 2450/2500 [14:33:28<17:42, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.049514383300183526, 'learning_rate': 2e-08, 'completion_length': 157.5089340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0062255859375, 'epoch': 0.98} 98%|█████████▊| 2450/2500 [14:33:28<17:42, 21.24s/it] 98%|█████████▊| 2451/2500 [14:33:49<17:10, 21.02s/it] {'loss': 0.0001, 'grad_norm': 0.015043211542960367, 'learning_rate': 1.9599999999999997e-08, 'completion_length': 148.24107360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00323486328125, 'epoch': 0.98} 98%|█████████▊| 2451/2500 [14:33:49<17:10, 21.02s/it] 98%|█████████▊| 2452/2500 [14:34:10<16:50, 21.05s/it] {'loss': 0.0003, 'grad_norm': 0.44517872534538055, 'learning_rate': 1.9199999999999997e-08, 'completion_length': 163.0357208251953, 'rewards/accuracy_reward': 0.9464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.946428656578064, 'reward_std': 0.09528662264347076, 'kl': 0.008392333984375, 'epoch': 0.98} 98%|█████████▊| 2452/2500 [14:34:10<16:50, 21.05s/it] 98%|█████████▊| 2453/2500 [14:34:31<16:24, 20.96s/it] {'loss': 0.0002, 'grad_norm': 0.016261468855027772, 'learning_rate': 1.88e-08, 'completion_length': 135.68750762939453, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.00382232666015625, 'epoch': 0.98} 98%|█████████▊| 2453/2500 [14:34:31<16:24, 20.96s/it] 98%|█████████▊| 2454/2500 [14:34:52<16:04, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.021919181480404035, 'learning_rate': 1.84e-08, 'completion_length': 148.43750762939453, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0050506591796875, 'epoch': 0.98} 98%|█████████▊| 2454/2500 [14:34:52<16:04, 20.97s/it] 98%|█████████▊| 2455/2500 [14:35:13<15:55, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.018052919180506288, 'learning_rate': 1.8e-08, 'completion_length': 153.1964340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.004852294921875, 'epoch': 0.98} 98%|█████████▊| 2455/2500 [14:35:13<15:55, 21.24s/it] 98%|█████████▊| 2456/2500 [14:35:34<15:25, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.0225997785691881, 'learning_rate': 1.76e-08, 'completion_length': 137.41965103149414, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0037841796875, 'epoch': 0.98} 98%|█████████▊| 2456/2500 [14:35:34<15:25, 21.04s/it] 98%|█████████▊| 2457/2500 [14:35:56<15:12, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.6387008668023377, 'learning_rate': 1.72e-08, 'completion_length': 169.36608123779297, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00485992431640625, 'epoch': 0.98} 98%|█████████▊| 2457/2500 [14:35:56<15:12, 21.23s/it] 98%|█████████▊| 2458/2500 [14:36:17<14:49, 21.18s/it] {'loss': 0.0002, 'grad_norm': 0.3252862645982042, 'learning_rate': 1.6799999999999998e-08, 'completion_length': 148.71428680419922, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0052947998046875, 'epoch': 0.98} 98%|█████████▊| 2458/2500 [14:36:17<14:49, 21.18s/it] 98%|█████████▊| 2459/2500 [14:36:38<14:31, 21.26s/it] {'loss': 0.0003, 'grad_norm': 0.218616251174804, 'learning_rate': 1.64e-08, 'completion_length': 171.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.006683349609375, 'epoch': 0.98} 98%|█████████▊| 2459/2500 [14:36:38<14:31, 21.26s/it] 98%|█████████▊| 2460/2500 [14:37:00<14:15, 21.39s/it] {'loss': 0.0003, 'grad_norm': 0.5636957485104218, 'learning_rate': 1.6e-08, 'completion_length': 166.30358123779297, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.03696779906749725, 'kl': 0.0063323974609375, 'epoch': 0.98} 98%|█████████▊| 2460/2500 [14:37:00<14:15, 21.39s/it] 98%|█████████▊| 2461/2500 [14:37:21<13:45, 21.16s/it] {'loss': 0.0002, 'grad_norm': 0.3179003886213921, 'learning_rate': 1.5599999999999997e-08, 'completion_length': 139.72321701049805, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.06222161650657654, 'kl': 0.00441741943359375, 'epoch': 0.98} 98%|█████████▊| 2461/2500 [14:37:21<13:45, 21.16s/it] 98%|█████████▊| 2462/2500 [14:37:41<13:16, 20.97s/it] {'loss': 0.0002, 'grad_norm': 0.029822535291044783, 'learning_rate': 1.52e-08, 'completion_length': 143.42857360839844, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00390625, 'epoch': 0.98} 98%|█████████▊| 2462/2500 [14:37:41<13:16, 20.97s/it] 99%|█████████▊| 2463/2500 [14:38:03<13:07, 21.28s/it] {'loss': 0.0001, 'grad_norm': 0.017551953770011158, 'learning_rate': 1.48e-08, 'completion_length': 155.80358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00368499755859375, 'epoch': 0.99} 99%|█████████▊| 2463/2500 [14:38:03<13:07, 21.28s/it] 99%|█████████▊| 2464/2500 [14:38:24<12:44, 21.23s/it] {'loss': 0.0002, 'grad_norm': 0.02770351927184016, 'learning_rate': 1.4399999999999998e-08, 'completion_length': 154.48214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00382232666015625, 'epoch': 0.99} 99%|█████████▊| 2464/2500 [14:38:24<12:44, 21.23s/it] 99%|█████████▊| 2465/2500 [14:38:46<12:24, 21.26s/it] {'loss': 0.0001, 'grad_norm': 0.22394552376335972, 'learning_rate': 1.4e-08, 'completion_length': 154.1964340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0034332275390625, 'epoch': 0.99} 99%|█████████▊| 2465/2500 [14:38:46<12:24, 21.26s/it] 99%|█████████▊| 2466/2500 [14:39:07<12:00, 21.20s/it] {'loss': 0.0002, 'grad_norm': 0.25907209434181655, 'learning_rate': 1.36e-08, 'completion_length': 159.52679443359375, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.919642984867096, 'reward_std': 0.05831881985068321, 'kl': 0.0047454833984375, 'epoch': 0.99} 99%|█████████▊| 2466/2500 [14:39:07<12:00, 21.20s/it] 99%|█████████▊| 2467/2500 [14:39:28<11:46, 21.40s/it] {'loss': 0.0003, 'grad_norm': 0.6097011589022879, 'learning_rate': 1.3199999999999999e-08, 'completion_length': 153.98214721679688, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07003280520439148, 'kl': 0.00738525390625, 'epoch': 0.99} 99%|█████████▊| 2467/2500 [14:39:28<11:46, 21.40s/it] 99%|█████████▊| 2468/2500 [14:39:50<11:22, 21.33s/it] {'loss': 0.0002, 'grad_norm': 1.2164503583014077, 'learning_rate': 1.28e-08, 'completion_length': 148.17857360839844, 'rewards/accuracy_reward': 0.9196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.07003280520439148, 'kl': 0.0048065185546875, 'epoch': 0.99} 99%|█████████▊| 2468/2500 [14:39:50<11:22, 21.33s/it] 99%|█████████▉| 2469/2500 [14:40:10<10:52, 21.04s/it] {'loss': 0.0002, 'grad_norm': 0.3793613216200063, 'learning_rate': 1.2399999999999999e-08, 'completion_length': 134.52679443359375, 'rewards/accuracy_reward': 0.9732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.973214328289032, 'reward_std': 0.05831882357597351, 'kl': 0.00592041015625, 'epoch': 0.99} 99%|█████████▉| 2469/2500 [14:40:10<10:52, 21.04s/it] 99%|█████████▉| 2470/2500 [14:40:31<10:30, 21.03s/it] {'loss': 0.0001, 'grad_norm': 0.18042237840287312, 'learning_rate': 1.2e-08, 'completion_length': 144.24108123779297, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0034332275390625, 'epoch': 0.99} 99%|█████████▉| 2470/2500 [14:40:31<10:30, 21.03s/it] 99%|█████████▉| 2471/2500 [14:40:52<10:09, 21.00s/it] {'loss': 0.0003, 'grad_norm': 0.025185813646754394, 'learning_rate': 1.1599999999999998e-08, 'completion_length': 141.47321701049805, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0064239501953125, 'epoch': 0.99} 99%|█████████▉| 2471/2500 [14:40:52<10:09, 21.00s/it] 99%|█████████▉| 2472/2500 [14:41:13<09:50, 21.08s/it] {'loss': 0.0002, 'grad_norm': 0.3339224051240862, 'learning_rate': 1.12e-08, 'completion_length': 152.2589340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00536346435546875, 'epoch': 0.99} 99%|█████████▉| 2472/2500 [14:41:13<09:50, 21.08s/it] 99%|█████████▉| 2473/2500 [14:41:34<09:27, 21.03s/it] {'loss': 0.0001, 'grad_norm': 0.02004195697461626, 'learning_rate': 1.08e-08, 'completion_length': 152.54464721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00360870361328125, 'epoch': 0.99} 99%|█████████▉| 2473/2500 [14:41:34<09:27, 21.03s/it] 99%|█████████▉| 2474/2500 [14:41:55<09:06, 21.02s/it] {'loss': 0.0002, 'grad_norm': 0.3196281305109746, 'learning_rate': 1.0399999999999999e-08, 'completion_length': 159.4107208251953, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.05050762742757797, 'kl': 0.00579833984375, 'epoch': 0.99} 99%|█████████▉| 2474/2500 [14:41:55<09:06, 21.02s/it] 99%|█████████▉| 2475/2500 [14:42:17<08:49, 21.17s/it] {'loss': 0.0002, 'grad_norm': 0.3818588824745006, 'learning_rate': 1e-08, 'completion_length': 158.71429443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0052337646484375, 'epoch': 0.99} 99%|█████████▉| 2475/2500 [14:42:17<08:49, 21.17s/it] 99%|█████████▉| 2476/2500 [14:42:37<08:22, 20.92s/it] {'loss': 0.0001, 'grad_norm': 0.20702189286039688, 'learning_rate': 9.599999999999998e-09, 'completion_length': 144.83929443359375, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0034332275390625, 'epoch': 0.99} 99%|█████████▉| 2476/2500 [14:42:37<08:22, 20.92s/it] 99%|█████████▉| 2477/2500 [14:42:59<08:09, 21.28s/it] {'loss': 0.0003, 'grad_norm': 0.4387299628207606, 'learning_rate': 9.2e-09, 'completion_length': 149.0, 'rewards/accuracy_reward': 0.848214328289032, 'rewards/format_reward': 1.0, 'reward': 1.8482143878936768, 'reward_std': 0.09918941557407379, 'kl': 0.0066070556640625, 'epoch': 0.99} 99%|█████████▉| 2477/2500 [14:42:59<08:09, 21.28s/it] 99%|█████████▉| 2478/2500 [14:43:20<07:48, 21.30s/it] {'loss': 0.0002, 'grad_norm': 0.2592901509165153, 'learning_rate': 8.8e-09, 'completion_length': 151.16964721679688, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.00555419921875, 'epoch': 0.99} 99%|█████████▉| 2478/2500 [14:43:20<07:48, 21.30s/it] 99%|█████████▉| 2479/2500 [14:43:42<07:29, 21.39s/it] {'loss': 0.0002, 'grad_norm': 0.02973696441435811, 'learning_rate': 8.399999999999999e-09, 'completion_length': 161.96429443359375, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.0048828125, 'epoch': 0.99} 99%|█████████▉| 2479/2500 [14:43:42<07:29, 21.39s/it] 99%|█████████▉| 2480/2500 [14:44:04<07:09, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.2634157830348159, 'learning_rate': 8e-09, 'completion_length': 173.4464340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0061187744140625, 'epoch': 0.99} 99%|█████████▉| 2480/2500 [14:44:04<07:09, 21.47s/it] 99%|█████████▉| 2481/2500 [14:44:24<06:44, 21.27s/it] {'loss': 0.0002, 'grad_norm': 0.02078130943191626, 'learning_rate': 7.6e-09, 'completion_length': 158.6339340209961, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00577545166015625, 'epoch': 0.99} 99%|█████████▉| 2481/2500 [14:44:24<06:44, 21.27s/it] 99%|█████████▉| 2482/2500 [14:44:45<06:20, 21.14s/it] {'loss': 0.0002, 'grad_norm': 0.21380482345412996, 'learning_rate': 7.199999999999999e-09, 'completion_length': 161.0089340209961, 'rewards/accuracy_reward': 0.9821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9821429252624512, 'reward_std': 0.033065006136894226, 'kl': 0.0047149658203125, 'epoch': 0.99} 99%|█████████▉| 2482/2500 [14:44:45<06:20, 21.14s/it] 99%|█████████▉| 2483/2500 [14:45:07<06:00, 21.18s/it] {'loss': 0.0002, 'grad_norm': 1.3203328573086726, 'learning_rate': 6.8e-09, 'completion_length': 152.74107360839844, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00391387939453125, 'epoch': 0.99} 99%|█████████▉| 2483/2500 [14:45:07<06:00, 21.18s/it] 99%|█████████▉| 2484/2500 [14:45:28<05:38, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.35373526702173164, 'learning_rate': 6.4e-09, 'completion_length': 146.70536041259766, 'rewards/accuracy_reward': 0.9196429252624512, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.004608154296875, 'epoch': 0.99} 99%|█████████▉| 2484/2500 [14:45:28<05:38, 21.13s/it] 99%|█████████▉| 2485/2500 [14:45:48<05:15, 21.01s/it] {'loss': 0.0002, 'grad_norm': 0.24704287994510132, 'learning_rate': 6e-09, 'completion_length': 147.67857360839844, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.025253813713788986, 'kl': 0.0059967041015625, 'epoch': 0.99} 99%|█████████▉| 2485/2500 [14:45:48<05:15, 21.01s/it] 99%|█████████▉| 2486/2500 [14:46:10<04:55, 21.13s/it] {'loss': 0.0002, 'grad_norm': 0.05411021254343343, 'learning_rate': 5.6e-09, 'completion_length': 150.05358123779297, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00432586669921875, 'epoch': 0.99} 99%|█████████▉| 2486/2500 [14:46:10<04:55, 21.13s/it] 99%|█████████▉| 2487/2500 [14:46:31<04:34, 21.09s/it] {'loss': 0.0002, 'grad_norm': 0.020217626290964748, 'learning_rate': 5.1999999999999994e-09, 'completion_length': 142.33929443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0054779052734375, 'epoch': 0.99} 99%|█████████▉| 2487/2500 [14:46:31<04:34, 21.09s/it] 100%|█████████▉| 2488/2500 [14:46:52<04:13, 21.14s/it] {'loss': 0.0001, 'grad_norm': 0.4557855943202116, 'learning_rate': 4.799999999999999e-09, 'completion_length': 156.5, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.0032196044921875, 'epoch': 1.0} 100%|█████████▉| 2488/2500 [14:46:52<04:13, 21.14s/it] 100%|█████████▉| 2489/2500 [14:47:14<03:54, 21.35s/it] {'loss': 0.0003, 'grad_norm': 0.5367890935728986, 'learning_rate': 4.4e-09, 'completion_length': 165.66964721679688, 'rewards/accuracy_reward': 0.9196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.9196429252624512, 'reward_std': 0.09918941557407379, 'kl': 0.0073699951171875, 'epoch': 1.0} 100%|█████████▉| 2489/2500 [14:47:14<03:54, 21.35s/it] 100%|█████████▉| 2490/2500 [14:47:36<03:34, 21.49s/it] {'loss': 0.0003, 'grad_norm': 0.8553974921633702, 'learning_rate': 4e-09, 'completion_length': 157.7678680419922, 'rewards/accuracy_reward': 0.9642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.9642857313156128, 'reward_std': 0.0835726335644722, 'kl': 0.0064239501953125, 'epoch': 1.0} 100%|█████████▉| 2490/2500 [14:47:36<03:34, 21.49s/it] 100%|█████████▉| 2491/2500 [14:47:57<03:13, 21.47s/it] {'loss': 0.0002, 'grad_norm': 0.5674288927068422, 'learning_rate': 3.5999999999999996e-09, 'completion_length': 141.8839340209961, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00508880615234375, 'epoch': 1.0} 100%|█████████▉| 2491/2500 [14:47:57<03:13, 21.47s/it] 100%|█████████▉| 2492/2500 [14:48:19<02:53, 21.69s/it] {'loss': 0.0002, 'grad_norm': 0.9511970542973733, 'learning_rate': 3.2e-09, 'completion_length': 167.2232208251953, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.07514797151088715, 'kl': 0.004425048828125, 'epoch': 1.0} 100%|█████████▉| 2492/2500 [14:48:19<02:53, 21.69s/it] 100%|█████████▉| 2493/2500 [14:48:42<02:34, 22.01s/it] {'loss': 0.0002, 'grad_norm': 0.01859793801972815, 'learning_rate': 2.8e-09, 'completion_length': 168.46429443359375, 'rewards/accuracy_reward': 0.9285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9285714626312256, 'reward_std': 0.0, 'kl': 0.0059356689453125, 'epoch': 1.0} 100%|█████████▉| 2493/2500 [14:48:42<02:34, 22.01s/it] 100%|█████████▉| 2494/2500 [14:49:03<02:09, 21.67s/it] {'loss': 0.0002, 'grad_norm': 0.5285593873217862, 'learning_rate': 2.3999999999999996e-09, 'completion_length': 152.8928680419922, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553572535514832, 'reward_std': 0.07003280520439148, 'kl': 0.00504302978515625, 'epoch': 1.0} 100%|█████████▉| 2494/2500 [14:49:03<02:09, 21.67s/it] 100%|█████████▉| 2495/2500 [14:49:24<01:47, 21.48s/it] {'loss': 0.0002, 'grad_norm': 0.02912029328164157, 'learning_rate': 2e-09, 'completion_length': 145.52679061889648, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.005401611328125, 'epoch': 1.0} 100%|█████████▉| 2495/2500 [14:49:24<01:47, 21.48s/it] 100%|█████████▉| 2496/2500 [14:49:45<01:24, 21.24s/it] {'loss': 0.0002, 'grad_norm': 0.9608085871915958, 'learning_rate': 1.6e-09, 'completion_length': 144.96428680419922, 'rewards/accuracy_reward': 0.9375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.9375000596046448, 'reward_std': 0.025253813713788986, 'kl': 0.0054168701171875, 'epoch': 1.0} 100%|█████████▉| 2496/2500 [14:49:45<01:24, 21.24s/it] 100%|█████████▉| 2497/2500 [14:50:05<01:02, 21.00s/it] {'loss': 0.0002, 'grad_norm': 0.18847001499430738, 'learning_rate': 1.1999999999999998e-09, 'completion_length': 135.6875, 'rewards/accuracy_reward': 0.9910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.9910714626312256, 'reward_std': 0.025253813713788986, 'kl': 0.00392913818359375, 'epoch': 1.0} 100%|█████████▉| 2497/2500 [14:50:05<01:02, 21.00s/it] 100%|█████████▉| 2498/2500 [14:50:26<00:42, 21.10s/it] {'loss': 0.0002, 'grad_norm': 0.9067047446427604, 'learning_rate': 8e-10, 'completion_length': 160.75000762939453, 'rewards/accuracy_reward': 0.910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.9107143878936768, 'reward_std': 0.0835726335644722, 'kl': 0.00518798828125, 'epoch': 1.0} 100%|█████████▉| 2498/2500 [14:50:26<00:42, 21.10s/it] 100%|█████████▉| 2499/2500 [14:50:47<00:20, 20.88s/it] {'loss': 0.0002, 'grad_norm': 0.21023482812676472, 'learning_rate': 4e-10, 'completion_length': 144.5982208251953, 'rewards/accuracy_reward': 0.955357164144516, 'rewards/format_reward': 1.0, 'reward': 1.9553571939468384, 'reward_std': 0.03696779906749725, 'kl': 0.00406646728515625, 'epoch': 1.0} 100%|█████████▉| 2499/2500 [14:50:47<00:20, 20.88s/it] 100%|██████████| 2500/2500 [14:51:07<00:00, 20.77s/it] {'loss': 0.0002, 'grad_norm': 0.05088790135303215, 'learning_rate': 0.0, 'completion_length': 139.23214721679688, 'rewards/accuracy_reward': 1.0, 'rewards/format_reward': 1.0, 'reward': 2.0, 'reward_std': 0.0, 'kl': 0.00582122802734375, 'epoch': 1.0} 100%|██████████| 2500/2500 [14:51:07<00:00, 20.77s/it] {'train_runtime': 53523.9391, 'train_samples_per_second': 0.654, 'train_steps_per_second': 0.047, 'train_loss': 0.0002267889650159117, 'epoch': 1.0} 100%|██████████| 2500/2500 [14:51:53<00:00, 20.77s/it] 100%|██████████| 2500/2500 [14:51:53<00:00, 21.41s/it] wandb: wandb: 🚀 View run R1-Resume-COT-VLLM-Correct-Qwen2-VL-2B-GRPO-ClevrMath-35k-2025-02-17-15-12-32 at: https://wandb.ai/tanhuajie264-peking-university/vison-open-r1/runs/9wwnyj03 wandb: Find logs at: wandb/run-20250217_152325-9wwnyj03/logs