[2025-02-23 13:20:28,803] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,803] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,804] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,805] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,805] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,805] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-02-23 13:20:28,805] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect) INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. INFO 02-23 13:20:35 __init__.py:190] Automatically detected platform cuda. [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:42,328] [INFO] [comm.py:683:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl [2025-02-23 13:20:42,328] [INFO] [comm.py:652:init_distributed] cdb=None [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:44,361] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen2VisionTransformerPretrainedModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `torch_dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", torch_dtype=torch.float16)` p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3740350 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3740350 [0] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3740350 [0] NCCL INFO cudaDriverVersion 12040 NCCL version 2.21.5+cuda12.4 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3740353 [3] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3740353 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3740352 [2] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3740352 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3740354 [4] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3740354 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3740351 [1] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3740351 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3740356 [6] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3740356 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3740355 [5] NCCL INFO cudaDriverVersion 12040 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3740355 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3740353 [3] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3740352 [2] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3740351 [1] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3740354 [4] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3740356 [6] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3740355 [5] NCCL INFO Bootstrap : Using bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO P2P plugin IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to bond0 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB/SHARP [1]mlx5_1:1/IB/SHARP [RO]; OOB bond0:10.9.200.104<0> p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Using non-device net plugin version 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Using network IBext_v8 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO ncclCommInitRank comm 0x5623f6dacf60 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO ncclCommInitRank comm 0x564164ec40d0 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO ncclCommInitRank comm 0x55af1f2cc9a0 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO ncclCommInitRank comm 0x556bf05699d0 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO ncclCommInitRank comm 0x5617f983cb00 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO ncclCommInitRank comm 0x55a0474b5520 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO ncclCommInitRank comm 0x558656ea6eb0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 commId 0x9bb06dee124e182c - Init START p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO NVLS multicast support is not available on dev 4 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Setting affinity for GPU 0 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO NVLS multicast support is not available on dev 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO NVLS multicast support is not available on dev 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,00000000,ffffffff,00000000 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO NVLS multicast support is not available on dev 5 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Setting affinity for GPU 3 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO NVLS multicast support is not available on dev 3 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Setting affinity for GPU 2 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO NVLS multicast support is not available on dev 2 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO NCCL_CUMEM_ENABLE set by environment to 0. p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Setting affinity for GPU 1 to ffffffff,00000000,ffffffff p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO NVLS multicast support is not available on dev 1 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO comm 0x5617f983cb00 rank 0 nRanks 7 nNodes 1 localRanks 7 localRank 0 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 00/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 01/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 02/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 03/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 04/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 05/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 06/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 07/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 08/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 09/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 10/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 11/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 12/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 13/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 14/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 15/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO comm 0x55a0474b5520 rank 6 nRanks 7 nNodes 1 localRanks 7 localRank 6 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO comm 0x556bf05699d0 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO comm 0x564164ec40d0 rank 2 nRanks 7 nNodes 1 localRanks 7 localRank 2 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO comm 0x5623f6dacf60 rank 4 nRanks 7 nNodes 1 localRanks 7 localRank 4 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO comm 0x558656ea6eb0 rank 5 nRanks 7 nNodes 1 localRanks 7 localRank 5 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Trees [0] -1/-1/-1->6->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO comm 0x55af1f2cc9a0 rank 3 nRanks 7 nNodes 1 localRanks 7 localRank 3 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3742167 [4] NCCL INFO ncclCommInitRank comm 0x5623f6dacf60 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3742149 [0] NCCL INFO ncclCommInitRank comm 0x5617f983cb00 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3742166 [3] NCCL INFO ncclCommInitRank comm 0x55af1f2cc9a0 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3742164 [1] NCCL INFO ncclCommInitRank comm 0x556bf05699d0 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3742163 [2] NCCL INFO ncclCommInitRank comm 0x564164ec40d0 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3742165 [6] NCCL INFO ncclCommInitRank comm 0x55a0474b5520 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 commId 0x9bb06dee124e182c - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2, using internal tuner instead. p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3742168 [5] NCCL INFO ncclCommInitRank comm 0x558656ea6eb0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 commId 0x9bb06dee124e182c - Init COMPLETE [2025-02-23 13:20:46,219] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 730, num_elems = 2.44B [2025-02-23 13:20:53,718] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,718] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,719] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,719] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,719] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,720] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:53,814] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:54,115] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 1460, num_elems = 4.88B Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.48, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`. Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher. [2025-02-23 13:20:57,299] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,301] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,301] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed info: version=0.16.3, git-hash=unknown, git-branch=unknown [2025-02-23 13:20:57,301] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,302] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,302] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,307] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,308] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 7 [2025-02-23 13:20:57,318] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2025-02-23 13:20:57,321] [INFO] [logging.py:128:log_dist] [Rank 0] Creating ZeRO Offload [2025-02-23 13:20:57,523] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2025-02-23 13:20:57,524] [INFO] [utils.py:782:see_memory_usage] MA 1.19 GB Max_MA 2.49 GB CA 3.09 GB Max_CA 3 GB [2025-02-23 13:20:57,524] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 493.13 GB, percent = 49.0% Parameter Offload: Total persistent parameters: 686592 in 401 params [2025-02-23 13:20:57,738] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2025-02-23 13:20:57,738] [INFO] [utils.py:782:see_memory_usage] MA 1.19 GB Max_MA 1.19 GB CA 3.09 GB Max_CA 3 GB [2025-02-23 13:20:57,739] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 493.13 GB, percent = 49.0% [2025-02-23 13:20:57,740] [INFO] [config.py:999:print] DeepSpeedEngine configuration: [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False} [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] amp_enabled .................. False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] amp_params ................... False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] bfloat16_enabled ............. True [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] bfloat16_immediate_grad_update False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] checkpoint_parallel_write_pipeline False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] checkpoint_tag_validation_enabled True [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] checkpoint_tag_validation_fail False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] comms_config ................. [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] communication_data_type ...... None [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] curriculum_enabled_legacy .... False [2025-02-23 13:20:57,741] [INFO] [config.py:1003:print] curriculum_params_legacy ..... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] data_efficiency_enabled ...... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] dataloader_drop_last ......... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] disable_allgather ............ False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] dump_state ................... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] dynamic_loss_scale_args ...... None [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_enabled ........... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_gas_boundary_resolution 1 [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_layer_name ........ bert.encoder.layer [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_layer_num ......... 0 [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_max_iter .......... 100 [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_stability ......... 1e-06 [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_tol ............... 0.01 [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] eigenvalue_verbose ........... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] elasticity_enabled ........... False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] flops_profiler_config ........ { "enabled": false, "recompute_fwd_factor": 0.0, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] fp16_auto_cast ............... None [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] fp16_enabled ................. False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] fp16_master_weights_and_gradients False [2025-02-23 13:20:57,742] [INFO] [config.py:1003:print] global_rank .................. 0 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] grad_accum_dtype ............. None [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] gradient_accumulation_steps .. 2 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] gradient_clipping ............ 1.0 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] gradient_predivide_factor .... 1.0 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] graph_harvesting ............. False [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] initial_dynamic_scale ........ 1 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] load_universal_checkpoint .... False [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] loss_scale ................... 1.0 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] memory_breakdown ............. False [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] mics_hierarchial_params_gather False [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] mics_shard_size .............. -1 [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] optimizer_legacy_fusion ...... False [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] optimizer_name ............... None [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] optimizer_params ............. None [2025-02-23 13:20:57,743] [INFO] [config.py:1003:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True} [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] pld_enabled .................. False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] pld_params ................... False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] prescale_gradients ........... False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] scheduler_name ............... None [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] scheduler_params ............. None [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] seq_parallel_communication_data_type torch.float32 [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] sparse_attention ............. None [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] sparse_gradients_enabled ..... False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] steps_per_print .............. inf [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] timers_config ................ enabled=True synchronized=True [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] train_batch_size ............. 14 [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] train_micro_batch_size_per_gpu 1 [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] use_data_before_expert_parallel_ False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] use_node_local_storage ....... False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] wall_clock_breakdown ......... False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] weight_quantization_config ... None [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] world_size ................... 7 [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] zero_allow_untested_optimizer False [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100000000, max_in_cpu=1000000000, pin_memory=True) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=True module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True [2025-02-23 13:20:57,744] [INFO] [config.py:1003:print] zero_enabled ................. True [2025-02-23 13:20:57,745] [INFO] [config.py:1003:print] zero_force_ds_cpu_optimizer .. True [2025-02-23 13:20:57,745] [INFO] [config.py:1003:print] zero_optimization_stage ...... 3 [2025-02-23 13:20:57,745] [INFO] [config.py:989:print_user_config] json = { "fp16": { "enabled": false, "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1 }, "bf16": { "enabled": true }, "zero_optimization": { "stage": 3, "offload_optimizer": { "device": "none", "pin_memory": true }, "offload_param": { "device": "none", "pin_memory": true }, "overlap_comm": true, "contiguous_gradients": true, "sub_group_size": 1.000000e+09, "reduce_bucket_size": "auto", "stage3_prefetch_bucket_size": "auto", "stage3_param_persistence_threshold": "auto", "stage3_max_live_parameters": 1.000000e+09, "stage3_max_reuse_distance": 1.000000e+09, "stage3_gather_16bit_weights_on_model_save": true }, "gradient_accumulation_steps": 2, "gradient_clipping": 1.0, "steps_per_print": inf, "train_batch_size": 14, "train_micro_batch_size_per_gpu": 1, "wall_clock_breakdown": false, "zero_optimization.reduce_bucket_size": 2.359296e+06, "zero_optimization.stage3_param_persistence_threshold": 1.536000e+04, "zero_optimization.stage3_prefetch_bucket_size": 2.123366e+06 } INFO 02-23 13:21:14 config.py:542] This model supports multiple tasks: {'generate', 'score', 'reward', 'embed', 'classify'}. Defaulting to 'generate'. WARNING 02-23 13:21:14 arg_utils.py:1079] --enable-prefix-caching is currently not supported for multimodal models in v0 and has been disabled. INFO 02-23 13:21:14 llm_engine.py:234] Initializing a V0 LLM engine (v0.7.2) with config: model='/home/vlm/workspace/r1_checkpoints/qwen2vl_2b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400', speculative_config=None, tokenizer='/home/vlm/workspace/r1_checkpoints/qwen2vl_2b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda:7, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=/home/vlm/workspace/r1_checkpoints/qwen2vl_2b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":256}, use_cached_outputs=False, INFO 02-23 13:21:16 cuda.py:230] Using Flash Attention backend. INFO 02-23 13:21:16 model_runner.py:1110] Starting to load model /home/vlm/workspace/r1_checkpoints/qwen2vl_2b_R1_finetune_by_geoqa_4k5_cot_sft_every_100/checkpoint-400... INFO 02-23 13:21:16 config.py:2992] cudagraph sizes specified by model runner [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256] is overridden by config [256, 128, 2, 1, 4, 136, 8, 144, 16, 152, 24, 160, 32, 168, 40, 176, 48, 184, 56, 192, 64, 200, 72, 208, 80, 216, 88, 120, 224, 96, 232, 104, 240, 112, 248] Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00 4096). Running this sequence through the model will result in indexing errors WARNING 02-23 13:21:25 profiling.py:187] The context length (32768) of the model is too short to hold the multi-modal embeddings in the worst case (49152 tokens in total, out of which {'image': 32768, 'video': 16384} are reserved for multi-modal embeddings). This may cause certain multi-modal inputs to fail during inference, even when the input text is short. To avoid this, you should increase `max_model_len`, reduce `max_num_seqs`, and/or reduce `mm_counts`. INFO 02-23 13:21:27 worker.py:267] Memory profiling takes 8.15 seconds INFO 02-23 13:21:27 worker.py:267] the current vLLM instance can use total_gpu_memory (79.32GiB) x gpu_memory_utilization (0.70) = 55.53GiB INFO 02-23 13:21:27 worker.py:267] model weights take 0.00GiB; non_torch_memory takes 0.00GiB; PyTorch activation peak memory takes 0.00GiB; the rest of the memory reserved for KV Cache is 55.53GiB. INFO 02-23 13:21:27 executor_base.py:110] # CUDA blocks: 129965, # CPU blocks: 9362 INFO 02-23 13:21:27 executor_base.py:115] Maximum concurrency for 32768 tokens per request: 63.46x INFO 02-23 13:21:30 model_runner.py:1434] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI. If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage. Capturing CUDA graph shapes: 0%| | 0/35 [00:006->5 [1] -1/-1/-1->6->5 [2] -1/-1/-1->6->5 [3] -1/-1/-1->6->5 [4] -1/-1/-1->6->5 [5] -1/-1/-1->6->5 [6] -1/-1/-1->6->5 [7] -1/-1/-1->6->5 [8] -1/-1/-1->6->5 [9] -1/-1/-1->6->5 [10] -1/-1/-1->6->5 [11] -1/-1/-1->6->5 [12] -1/-1/-1->6->5 [13] -1/-1/-1->6->5 [14] -1/-1/-1->6->5 [15] -1/-1/-1->6->5 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO comm 0x7f2a88073080 rank 2 nRanks 7 nNodes 1 localRanks 7 localRank 2 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 03/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 04/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 05/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 06/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 07/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 08/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 09/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 10/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO comm 0x7f98d4072a00 rank 1 nRanks 7 nNodes 1 localRanks 7 localRank 1 MNNVL 0 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 11/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 12/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 13/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 14/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 15/16 : 0 1 2 3 4 5 6 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO P2P Chunksize set to 524288 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 00/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 01/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 02/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 03/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 04/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 05/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 06/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 07/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 08/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 09/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 10/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 11/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 12/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 13/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 14/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 15/0 : 6[6] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Connected all rings p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/IPC/read p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO Connected all trees p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO threadThresholds 8/8/64 | 56/8/64 | 512 | 512 p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer p-phy-ctyun-gz-a800-node-prod-200-104:3740350:3746421 [0] NCCL INFO ncclCommSplit comm 0x7f8d78072f00 rank 0 nranks 7 cudaDev 0 nvmlDev 0 busId 27000 parent 0x5617f983cb00 color -1326228412 key 0 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740352:3746422 [2] NCCL INFO ncclCommSplit comm 0x7f2a88073080 rank 2 nranks 7 cudaDev 2 nvmlDev 2 busId 54000 parent 0x564164ec40d0 color -1326228412 key 2 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740353:3746423 [3] NCCL INFO ncclCommSplit comm 0x7fa9ec073dd0 rank 3 nranks 7 cudaDev 3 nvmlDev 3 busId 59000 parent 0x55af1f2cc9a0 color -1326228412 key 3 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740355:3746420 [5] NCCL INFO ncclCommSplit comm 0x7f98a80732d0 rank 5 nranks 7 cudaDev 5 nvmlDev 5 busId 92000 parent 0x558656ea6eb0 color -1326228412 key 5 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740356:3746424 [6] NCCL INFO ncclCommSplit comm 0x7f93e0072bc0 rank 6 nranks 7 cudaDev 6 nvmlDev 6 busId bf000 parent 0x55a0474b5520 color -1326228412 key 6 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740351:3746418 [1] NCCL INFO ncclCommSplit comm 0x7f98d4072a00 rank 1 nranks 7 cudaDev 1 nvmlDev 1 busId 2d000 parent 0x556bf05699d0 color -1326228412 key 1 commId 0x2de68209f23c5477 - Init COMPLETE p-phy-ctyun-gz-a800-node-prod-200-104:3740354:3746419 [4] NCCL INFO ncclCommSplit comm 0x7f3350073c30 rank 4 nranks 7 cudaDev 4 nvmlDev 4 busId 8d000 parent 0x5623f6dacf60 color -1326228412 key 4 commId 0x2de68209f23c5477 - Init COMPLETE 0%| | 1/1610 [00:24<10:59:25, 24.59s/it] {'loss': 0.0, 'grad_norm': 2.004362293099394, 'learning_rate': 9.993788819875776e-07, 'completion_length': 205.04464721679688, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 0.9375000298023224, 'reward': 1.3392857313156128, 'reward_std': 0.3593148738145828, 'kl': 0.0, 'epoch': 0.0} 0%| | 1/1610 [00:24<10:59:25, 24.59s/it] 0%| | 2/1610 [00:39<8:21:20, 18.71s/it] {'loss': 0.0, 'grad_norm': 1.236594992299391, 'learning_rate': 9.987577639751552e-07, 'completion_length': 166.3839340209961, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.4642857909202576, 'reward_std': 0.39257223904132843, 'kl': 0.00010848045349121094, 'epoch': 0.01} 0%| | 2/1610 [00:39<8:21:20, 18.71s/it] 0%| | 3/1610 [00:53<7:28:18, 16.74s/it] {'loss': 0.0, 'grad_norm': 2.631928527828467, 'learning_rate': 9.981366459627329e-07, 'completion_length': 177.93750762939453, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4285714626312256, 'reward_std': 0.4854046553373337, 'kl': 0.00010097026824951172, 'epoch': 0.01} 0%| | 3/1610 [00:53<7:28:18, 16.74s/it] 0%| | 4/1610 [01:07<6:55:00, 15.50s/it] {'loss': 0.0, 'grad_norm': 1.589126785483445, 'learning_rate': 9.975155279503105e-07, 'completion_length': 143.16964721679688, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3482143878936768, 'reward_std': 0.3823588639497757, 'kl': 0.0001964569091796875, 'epoch': 0.01} 0%| | 4/1610 [01:07<6:55:00, 15.50s/it] 0%| | 5/1610 [01:19<6:21:05, 14.25s/it] {'loss': 0.0, 'grad_norm': 1.2499505289258692, 'learning_rate': 9.968944099378881e-07, 'completion_length': 158.3303680419922, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660714626312256, 'reward_std': 0.4127334654331207, 'kl': 0.00010395050048828125, 'epoch': 0.02} 0%| | 5/1610 [01:19<6:21:05, 14.25s/it] 0%| | 6/1610 [01:33<6:21:10, 14.26s/it] {'loss': 0.0, 'grad_norm': 0.765579477515542, 'learning_rate': 9.962732919254658e-07, 'completion_length': 196.8571548461914, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.2767857909202576, 'reward_std': 0.21582705900073051, 'kl': 0.0005822181701660156, 'epoch': 0.02} 0%| | 6/1610 [01:33<6:21:10, 14.26s/it] 0%| | 7/1610 [01:48<6:24:22, 14.39s/it] {'loss': 0.0, 'grad_norm': 3.645157721082184, 'learning_rate': 9.956521739130434e-07, 'completion_length': 207.3928680419922, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.955357164144516, 'reward': 1.3303571939468384, 'reward_std': 0.4180952459573746, 'kl': 0.00018787384033203125, 'epoch': 0.02} 0%| | 7/1610 [01:48<6:24:22, 14.39s/it] 0%| | 8/1610 [02:00<6:08:26, 13.80s/it] {'loss': 0.0, 'grad_norm': 1.3293440168492976, 'learning_rate': 9.95031055900621e-07, 'completion_length': 148.0803680419922, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.22996581345796585, 'kl': 0.000232696533203125, 'epoch': 0.02} 0%| | 8/1610 [02:00<6:08:26, 13.80s/it] 1%| | 9/1610 [02:11<5:43:55, 12.89s/it] {'loss': 0.0, 'grad_norm': 1.6699734097433874, 'learning_rate': 9.944099378881986e-07, 'completion_length': 130.02679443359375, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.4109019935131073, 'kl': 0.0002837181091308594, 'epoch': 0.03} 1%| | 9/1610 [02:11<5:43:55, 12.89s/it] 1%| | 10/1610 [02:26<6:00:15, 13.51s/it] {'loss': 0.0, 'grad_norm': 1.1306826509290224, 'learning_rate': 9.937888198757763e-07, 'completion_length': 193.42858123779297, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.3839285969734192, 'reward_std': 0.34208156168460846, 'kl': 0.00012540817260742188, 'epoch': 0.03} 1%| | 10/1610 [02:26<6:00:15, 13.51s/it] 1%| | 11/1610 [02:40<6:07:22, 13.79s/it] {'loss': 0.0, 'grad_norm': 0.9290892858027889, 'learning_rate': 9.93167701863354e-07, 'completion_length': 200.9107208251953, 'rewards/accuracy_reward': 0.473214328289032, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.4196429252624512, 'reward_std': 0.31751881539821625, 'kl': 0.0002570152282714844, 'epoch': 0.03} 1%| | 11/1610 [02:40<6:07:22, 13.79s/it] 1%| | 12/1610 [02:54<6:03:10, 13.64s/it] {'loss': 0.0, 'grad_norm': 1.247998940654513, 'learning_rate': 9.925465838509315e-07, 'completion_length': 168.27679443359375, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.410714328289032, 'reward_std': 0.39011095464229584, 'kl': 0.00030517578125, 'epoch': 0.04} 1%| | 12/1610 [02:54<6:03:10, 13.64s/it] 1%| | 13/1610 [03:07<6:00:01, 13.53s/it] {'loss': 0.0, 'grad_norm': 1.6626408362027307, 'learning_rate': 9.919254658385092e-07, 'completion_length': 162.27679443359375, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.2678571939468384, 'reward_std': 0.33264249563217163, 'kl': 0.000286102294921875, 'epoch': 0.04} 1%| | 13/1610 [03:07<6:00:01, 13.53s/it] 1%| | 14/1610 [03:21<6:05:08, 13.73s/it] {'loss': 0.0, 'grad_norm': 1.3265699579889256, 'learning_rate': 9.91304347826087e-07, 'completion_length': 186.17858123779297, 'rewards/accuracy_reward': 0.3482143133878708, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3303571939468384, 'reward_std': 0.29857128858566284, 'kl': 0.0003604888916015625, 'epoch': 0.04} 1%| | 14/1610 [03:21<6:05:08, 13.73s/it] 1%| | 15/1610 [03:34<5:59:43, 13.53s/it] {'loss': 0.0, 'grad_norm': 1.156761088610291, 'learning_rate': 9.906832298136647e-07, 'completion_length': 158.2053680419922, 'rewards/accuracy_reward': 0.3482143133878708, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3303571939468384, 'reward_std': 0.22463567554950714, 'kl': 0.00033283233642578125, 'epoch': 0.05} 1%| | 15/1610 [03:34<5:59:43, 13.53s/it] 1%| | 16/1610 [03:48<5:59:18, 13.53s/it] {'loss': 0.0, 'grad_norm': 1.6218889652277912, 'learning_rate': 9.900621118012423e-07, 'completion_length': 152.64286041259766, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3750000596046448, 'reward_std': 0.41932664811611176, 'kl': 0.00029468536376953125, 'epoch': 0.05} 1%| | 16/1610 [03:48<5:59:18, 13.53s/it] 1%| | 17/1610 [04:01<5:54:56, 13.37s/it] {'loss': 0.0, 'grad_norm': 1.0642780069303064, 'learning_rate': 9.8944099378882e-07, 'completion_length': 171.6339340209961, 'rewards/accuracy_reward': 0.2321428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.2142857909202576, 'reward_std': 0.26181842386722565, 'kl': 0.00039005279541015625, 'epoch': 0.05} 1%| | 17/1610 [04:01<5:54:56, 13.37s/it] 1%| | 18/1610 [04:14<5:52:36, 13.29s/it] {'loss': 0.0, 'grad_norm': 1.560529685065414, 'learning_rate': 9.888198757763976e-07, 'completion_length': 145.62500762939453, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3571429252624512, 'reward_std': 0.3694324791431427, 'kl': 0.0007648468017578125, 'epoch': 0.06} 1%| | 18/1610 [04:14<5:52:36, 13.29s/it] 1%| | 19/1610 [04:29<6:06:10, 13.81s/it] {'loss': 0.0, 'grad_norm': 1.8839569357341275, 'learning_rate': 9.881987577639752e-07, 'completion_length': 165.4107208251953, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3392857909202576, 'reward_std': 0.41988103091716766, 'kl': 0.00035381317138671875, 'epoch': 0.06} 1%| | 19/1610 [04:29<6:06:10, 13.81s/it] 1%| | 20/1610 [04:43<6:12:08, 14.04s/it] {'loss': 0.0, 'grad_norm': 1.2890578462688902, 'learning_rate': 9.875776397515528e-07, 'completion_length': 217.8571548461914, 'rewards/accuracy_reward': 0.2946428805589676, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.2410714626312256, 'reward_std': 0.32715585827827454, 'kl': 0.00033855438232421875, 'epoch': 0.06} 1%| | 20/1610 [04:43<6:12:08, 14.04s/it] 1%|▏ | 21/1610 [04:58<6:18:18, 14.28s/it] {'loss': 0.0, 'grad_norm': 1.2534815084835456, 'learning_rate': 9.869565217391304e-07, 'completion_length': 180.67857360839844, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.3035714626312256, 'reward_std': 0.4112125486135483, 'kl': 0.000553131103515625, 'epoch': 0.07} 1%|▏ | 21/1610 [04:58<6:18:18, 14.28s/it] 1%|▏ | 22/1610 [05:13<6:19:43, 14.35s/it] {'loss': 0.0, 'grad_norm': 0.6622662426643416, 'learning_rate': 9.86335403726708e-07, 'completion_length': 199.35714721679688, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.348214328289032, 'reward_std': 0.18845915794372559, 'kl': 0.00048065185546875, 'epoch': 0.07} 1%|▏ | 22/1610 [05:13<6:19:43, 14.35s/it] 1%|▏ | 23/1610 [05:25<6:02:34, 13.71s/it] {'loss': 0.0, 'grad_norm': 1.3264885488910374, 'learning_rate': 9.857142857142857e-07, 'completion_length': 160.6964340209961, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.2858598530292511, 'kl': 0.000675201416015625, 'epoch': 0.07} 1%|▏ | 23/1610 [05:25<6:02:34, 13.71s/it] 1%|▏ | 24/1610 [05:40<6:09:05, 13.96s/it] {'loss': 0.0, 'grad_norm': 2.3731985613536573, 'learning_rate': 9.850931677018633e-07, 'completion_length': 234.90178680419922, 'rewards/accuracy_reward': 0.294642873108387, 'rewards/format_reward': 0.955357164144516, 'reward': 1.2500000596046448, 'reward_std': 0.3866512179374695, 'kl': 0.0008087158203125, 'epoch': 0.07} 1%|▏ | 24/1610 [05:40<6:09:05, 13.96s/it] 2%|▏ | 25/1610 [05:52<5:58:04, 13.55s/it] {'loss': 0.0, 'grad_norm': 1.3262129011923365, 'learning_rate': 9.84472049689441e-07, 'completion_length': 168.4821548461914, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.955357164144516, 'reward': 1.4642857313156128, 'reward_std': 0.3639267235994339, 'kl': 0.0008945465087890625, 'epoch': 0.08} 2%|▏ | 25/1610 [05:52<5:58:04, 13.55s/it] 2%|▏ | 26/1610 [06:03<5:35:59, 12.73s/it] {'loss': 0.0, 'grad_norm': 1.7561559547947319, 'learning_rate': 9.838509316770186e-07, 'completion_length': 140.15179443359375, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.294877827167511, 'kl': 0.000568389892578125, 'epoch': 0.08} 2%|▏ | 26/1610 [06:03<5:35:59, 12.73s/it] 2%|▏ | 27/1610 [06:17<5:46:59, 13.15s/it] {'loss': 0.0, 'grad_norm': 1.378347644599675, 'learning_rate': 9.832298136645962e-07, 'completion_length': 172.5714340209961, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 0.9553571939468384, 'reward': 1.4910715222358704, 'reward_std': 0.4221864640712738, 'kl': 0.0006542205810546875, 'epoch': 0.08} 2%|▏ | 27/1610 [06:17<5:46:59, 13.15s/it] 2%|▏ | 28/1610 [06:28<5:32:19, 12.60s/it] {'loss': 0.0, 'grad_norm': 3.1387635372291753, 'learning_rate': 9.826086956521739e-07, 'completion_length': 143.04464721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.25912240147590637, 'kl': 0.000621795654296875, 'epoch': 0.09} 2%|▏ | 28/1610 [06:28<5:32:19, 12.60s/it] 2%|▏ | 29/1610 [06:41<5:34:46, 12.71s/it] {'loss': 0.0, 'grad_norm': 2.149354795707953, 'learning_rate': 9.819875776397515e-07, 'completion_length': 154.00000762939453, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4375000596046448, 'reward_std': 0.3858461529016495, 'kl': 0.0011444091796875, 'epoch': 0.09} 2%|▏ | 29/1610 [06:41<5:34:46, 12.71s/it] 2%|▏ | 30/1610 [06:56<5:50:25, 13.31s/it] {'loss': 0.0, 'grad_norm': 1.1493492723738103, 'learning_rate': 9.813664596273291e-07, 'completion_length': 208.39286041259766, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.955357164144516, 'reward': 1.383928656578064, 'reward_std': 0.4101150333881378, 'kl': 0.0005092620849609375, 'epoch': 0.09} 2%|▏ | 30/1610 [06:56<5:50:25, 13.31s/it] 2%|▏ | 31/1610 [07:08<5:41:07, 12.96s/it] {'loss': 0.0, 'grad_norm': 2.4180446024709736, 'learning_rate': 9.807453416149068e-07, 'completion_length': 141.00000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.366128608584404, 'kl': 0.00074005126953125, 'epoch': 0.1} 2%|▏ | 31/1610 [07:08<5:41:07, 12.96s/it] 2%|▏ | 32/1610 [07:23<5:53:51, 13.45s/it] {'loss': 0.0, 'grad_norm': 2.0710653669453816, 'learning_rate': 9.801242236024844e-07, 'completion_length': 199.9553680419922, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.928571492433548, 'reward': 1.348214328289032, 'reward_std': 0.4564858078956604, 'kl': 0.000946044921875, 'epoch': 0.1} 2%|▏ | 32/1610 [07:23<5:53:51, 13.45s/it] 2%|▏ | 33/1610 [07:36<5:51:19, 13.37s/it] {'loss': 0.0, 'grad_norm': 1.1525960445841208, 'learning_rate': 9.79503105590062e-07, 'completion_length': 175.04464721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.30880722403526306, 'kl': 0.0011138916015625, 'epoch': 0.1} 2%|▏ | 33/1610 [07:36<5:51:19, 13.37s/it] 2%|▏ | 34/1610 [07:48<5:39:20, 12.92s/it] {'loss': 0.0, 'grad_norm': 2.0161058554441778, 'learning_rate': 9.788819875776397e-07, 'completion_length': 142.7589340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.4285714626312256, 'reward_std': 0.24025246500968933, 'kl': 0.0009899139404296875, 'epoch': 0.11} 2%|▏ | 34/1610 [07:48<5:39:20, 12.92s/it] 2%|▏ | 35/1610 [08:01<5:40:01, 12.95s/it] {'loss': 0.0001, 'grad_norm': 1.9267723602331153, 'learning_rate': 9.782608695652173e-07, 'completion_length': 165.4107208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553571939468384, 'reward_std': 0.33246469497680664, 'kl': 0.001255035400390625, 'epoch': 0.11} 2%|▏ | 35/1610 [08:01<5:40:01, 12.95s/it] 2%|▏ | 36/1610 [08:14<5:40:56, 13.00s/it] {'loss': 0.0, 'grad_norm': 1.563958362937296, 'learning_rate': 9.77639751552795e-07, 'completion_length': 182.40179443359375, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.3125000596046448, 'reward_std': 0.38274845480918884, 'kl': 0.0011272430419921875, 'epoch': 0.11} 2%|▏ | 36/1610 [08:14<5:40:56, 13.00s/it] 2%|▏ | 37/1610 [08:28<5:44:40, 13.15s/it] {'loss': 0.0, 'grad_norm': 1.9232885415416474, 'learning_rate': 9.770186335403726e-07, 'completion_length': 156.12500762939453, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4553571939468384, 'reward_std': 0.42461085319519043, 'kl': 0.0009288787841796875, 'epoch': 0.11} 2%|▏ | 37/1610 [08:28<5:44:40, 13.15s/it] 2%|▏ | 38/1610 [08:41<5:45:44, 13.20s/it] {'loss': 0.0, 'grad_norm': 1.2558662721289326, 'learning_rate': 9.763975155279502e-07, 'completion_length': 178.75000762939453, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4017857909202576, 'reward_std': 0.3864005208015442, 'kl': 0.00115203857421875, 'epoch': 0.12} 2%|▏ | 38/1610 [08:41<5:45:44, 13.20s/it] 2%|▏ | 39/1610 [08:55<5:52:09, 13.45s/it] {'loss': 0.0, 'grad_norm': 1.0937694202163886, 'learning_rate': 9.757763975155278e-07, 'completion_length': 185.56250762939453, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.3168664127588272, 'kl': 0.0010986328125, 'epoch': 0.12} 2%|▏ | 39/1610 [08:55<5:52:09, 13.45s/it] 2%|▏ | 40/1610 [09:06<5:31:37, 12.67s/it] {'loss': 0.0, 'grad_norm': 1.486641752456985, 'learning_rate': 9.751552795031055e-07, 'completion_length': 132.7232208251953, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3531966358423233, 'kl': 0.0010738372802734375, 'epoch': 0.12} 2%|▏ | 40/1610 [09:06<5:31:37, 12.67s/it] 3%|▎ | 41/1610 [09:19<5:36:00, 12.85s/it] {'loss': 0.0001, 'grad_norm': 3.201850787870938, 'learning_rate': 9.745341614906833e-07, 'completion_length': 164.1428680419922, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.3512975126504898, 'kl': 0.00131988525390625, 'epoch': 0.13} 3%|▎ | 41/1610 [09:19<5:36:00, 12.85s/it] 3%|▎ | 42/1610 [09:33<5:41:45, 13.08s/it] {'loss': 0.0, 'grad_norm': 1.2275967803417236, 'learning_rate': 9.73913043478261e-07, 'completion_length': 172.9464340209961, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.508928656578064, 'reward_std': 0.3492204546928406, 'kl': 0.0011615753173828125, 'epoch': 0.13} 3%|▎ | 42/1610 [09:33<5:41:45, 13.08s/it] 3%|▎ | 43/1610 [09:45<5:34:27, 12.81s/it] {'loss': 0.0001, 'grad_norm': 1.67240331731321, 'learning_rate': 9.732919254658386e-07, 'completion_length': 141.08929443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.3498482406139374, 'kl': 0.0013580322265625, 'epoch': 0.13} 3%|▎ | 43/1610 [09:45<5:34:27, 12.81s/it] 3%|▎ | 44/1610 [10:00<5:50:42, 13.44s/it] {'loss': 0.0001, 'grad_norm': 1.3914116912188856, 'learning_rate': 9.726708074534162e-07, 'completion_length': 190.21429443359375, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 0.955357164144516, 'reward': 1.3125000596046448, 'reward_std': 0.38068437576293945, 'kl': 0.001544952392578125, 'epoch': 0.14} 3%|▎ | 44/1610 [10:00<5:50:42, 13.44s/it] 3%|▎ | 45/1610 [10:14<5:58:16, 13.74s/it] {'loss': 0.0001, 'grad_norm': 0.9782378962831593, 'learning_rate': 9.720496894409938e-07, 'completion_length': 185.6964340209961, 'rewards/accuracy_reward': 0.321428582072258, 'rewards/format_reward': 0.973214328289032, 'reward': 1.2946429252624512, 'reward_std': 0.33240559697151184, 'kl': 0.0017547607421875, 'epoch': 0.14} 3%|▎ | 45/1610 [10:14<5:58:16, 13.74s/it] 3%|▎ | 46/1610 [10:28<5:56:40, 13.68s/it] {'loss': 0.0001, 'grad_norm': 2.256236549747633, 'learning_rate': 9.714285714285715e-07, 'completion_length': 173.5982208251953, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9464285969734192, 'reward': 1.3125000596046448, 'reward_std': 0.24229323118925095, 'kl': 0.001506805419921875, 'epoch': 0.14} 3%|▎ | 46/1610 [10:28<5:56:40, 13.68s/it] 3%|▎ | 47/1610 [10:42<5:59:42, 13.81s/it] {'loss': 0.0001, 'grad_norm': 1.7326039499195924, 'learning_rate': 9.708074534161491e-07, 'completion_length': 185.06250762939453, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.9375000298023224, 'reward': 1.3125000596046448, 'reward_std': 0.4309871792793274, 'kl': 0.001316070556640625, 'epoch': 0.15} 3%|▎ | 47/1610 [10:42<5:59:42, 13.81s/it] 3%|▎ | 48/1610 [10:56<6:01:30, 13.89s/it] {'loss': 0.0, 'grad_norm': 1.1808839364246033, 'learning_rate': 9.701863354037265e-07, 'completion_length': 177.20536041259766, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3750000596046448, 'reward_std': 0.2997836470603943, 'kl': 0.00124359130859375, 'epoch': 0.15} 3%|▎ | 48/1610 [10:56<6:01:30, 13.89s/it] 3%|▎ | 49/1610 [11:09<5:55:24, 13.66s/it] {'loss': 0.0001, 'grad_norm': 0.925794930762745, 'learning_rate': 9.695652173913042e-07, 'completion_length': 170.8303680419922, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3660715222358704, 'reward_std': 0.30590492486953735, 'kl': 0.0012760162353515625, 'epoch': 0.15} 3%|▎ | 49/1610 [11:09<5:55:24, 13.66s/it] 3%|▎ | 50/1610 [11:24<6:05:12, 14.05s/it] {'loss': 0.0001, 'grad_norm': 1.3128677917868734, 'learning_rate': 9.68944099378882e-07, 'completion_length': 197.7321548461914, 'rewards/accuracy_reward': 0.1785714402794838, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.1517857909202576, 'reward_std': 0.2894970029592514, 'kl': 0.001514434814453125, 'epoch': 0.16} 3%|▎ | 50/1610 [11:24<6:05:12, 14.05s/it] 3%|▎ | 51/1610 [11:38<6:05:30, 14.07s/it] {'loss': 0.0001, 'grad_norm': 1.5885193964962916, 'learning_rate': 9.683229813664596e-07, 'completion_length': 172.58929443359375, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 0.973214328289032, 'reward': 1.4285714626312256, 'reward_std': 0.373147189617157, 'kl': 0.001613616943359375, 'epoch': 0.16} 3%|▎ | 51/1610 [11:38<6:05:30, 14.07s/it] 3%|▎ | 52/1610 [11:51<5:56:23, 13.72s/it] {'loss': 0.0001, 'grad_norm': 4.5793071863062655, 'learning_rate': 9.677018633540373e-07, 'completion_length': 154.25000762939453, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.21703942120075226, 'kl': 0.00188446044921875, 'epoch': 0.16} 3%|▎ | 52/1610 [11:51<5:56:23, 13.72s/it] 3%|▎ | 53/1610 [12:04<5:50:31, 13.51s/it] {'loss': 0.0001, 'grad_norm': 1.5371339191604059, 'learning_rate': 9.67080745341615e-07, 'completion_length': 188.0982208251953, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.383928656578064, 'reward_std': 0.3186536133289337, 'kl': 0.001384735107421875, 'epoch': 0.16} 3%|▎ | 53/1610 [12:04<5:50:31, 13.51s/it] 3%|▎ | 54/1610 [12:16<5:35:36, 12.94s/it] {'loss': 0.0001, 'grad_norm': 1.1913203420940741, 'learning_rate': 9.664596273291925e-07, 'completion_length': 155.87500762939453, 'rewards/accuracy_reward': 0.3660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.3111136853694916, 'kl': 0.001697540283203125, 'epoch': 0.17} 3%|▎ | 54/1610 [12:16<5:35:36, 12.94s/it] 3%|▎ | 55/1610 [12:29<5:40:48, 13.15s/it] {'loss': 0.0001, 'grad_norm': 1.2247792019952088, 'learning_rate': 9.658385093167702e-07, 'completion_length': 179.0446548461914, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.402106910943985, 'kl': 0.00145721435546875, 'epoch': 0.17} 3%|▎ | 55/1610 [12:29<5:40:48, 13.15s/it] 3%|▎ | 56/1610 [12:44<5:50:47, 13.54s/it] {'loss': 0.0001, 'grad_norm': 1.376780841199642, 'learning_rate': 9.652173913043478e-07, 'completion_length': 170.0178680419922, 'rewards/accuracy_reward': 0.3035714477300644, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.2321429252624512, 'reward_std': 0.32213518023490906, 'kl': 0.001415252685546875, 'epoch': 0.17} 3%|▎ | 56/1610 [12:44<5:50:47, 13.54s/it] 4%|▎ | 57/1610 [12:55<5:34:33, 12.93s/it] {'loss': 0.0001, 'grad_norm': 5.418307331032984, 'learning_rate': 9.645962732919254e-07, 'completion_length': 134.72322463989258, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3571050316095352, 'kl': 0.0016937255859375, 'epoch': 0.18} 4%|▎ | 57/1610 [12:55<5:34:33, 12.93s/it] 4%|▎ | 58/1610 [13:08<5:32:17, 12.85s/it] {'loss': 0.0001, 'grad_norm': 2.662342435346138, 'learning_rate': 9.63975155279503e-07, 'completion_length': 153.16964721679688, 'rewards/accuracy_reward': 0.3035714477300644, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.285714328289032, 'reward_std': 0.3870585411787033, 'kl': 0.00206756591796875, 'epoch': 0.18} 4%|▎ | 58/1610 [13:08<5:32:17, 12.85s/it] 4%|▎ | 59/1610 [13:19<5:17:05, 12.27s/it] {'loss': 0.0001, 'grad_norm': 1.8368695481531347, 'learning_rate': 9.633540372670807e-07, 'completion_length': 131.50000762939453, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.35831743478775024, 'kl': 0.00243377685546875, 'epoch': 0.18} 4%|▎ | 59/1610 [13:19<5:17:05, 12.27s/it] 4%|▎ | 60/1610 [13:29<5:02:37, 11.71s/it] {'loss': 0.0001, 'grad_norm': 2.8984233274993887, 'learning_rate': 9.627329192546583e-07, 'completion_length': 124.94643020629883, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.34781017899513245, 'kl': 0.001468658447265625, 'epoch': 0.19} 4%|▎ | 60/1610 [13:29<5:02:37, 11.71s/it] 4%|▍ | 61/1610 [13:41<5:05:01, 11.82s/it] {'loss': 0.0001, 'grad_norm': 1.2321156809613814, 'learning_rate': 9.62111801242236e-07, 'completion_length': 145.73215103149414, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.4553572535514832, 'reward_std': 0.3396568149328232, 'kl': 0.00209808349609375, 'epoch': 0.19} 4%|▍ | 61/1610 [13:41<5:05:01, 11.82s/it] 4%|▍ | 62/1610 [13:55<5:23:34, 12.54s/it] {'loss': 0.0001, 'grad_norm': 1.839018801968502, 'learning_rate': 9.614906832298136e-07, 'completion_length': 169.07144165039062, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.5446429252624512, 'reward_std': 0.36271435022354126, 'kl': 0.001689910888671875, 'epoch': 0.19} 4%|▍ | 62/1610 [13:55<5:23:34, 12.54s/it] 4%|▍ | 63/1610 [14:08<5:26:19, 12.66s/it] {'loss': 0.0001, 'grad_norm': 1.1703546988039861, 'learning_rate': 9.608695652173912e-07, 'completion_length': 154.7321548461914, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.30446138978004456, 'kl': 0.00183868408203125, 'epoch': 0.2} 4%|▍ | 63/1610 [14:08<5:26:19, 12.66s/it] 4%|▍ | 64/1610 [14:22<5:29:48, 12.80s/it] {'loss': 0.0001, 'grad_norm': 1.967685500586031, 'learning_rate': 9.602484472049689e-07, 'completion_length': 151.52678680419922, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.3853914886713028, 'kl': 0.00255584716796875, 'epoch': 0.2} 4%|▍ | 64/1610 [14:22<5:29:48, 12.80s/it] 4%|▍ | 65/1610 [14:34<5:25:08, 12.63s/it] {'loss': 0.0001, 'grad_norm': 1.5055533822073504, 'learning_rate': 9.596273291925465e-07, 'completion_length': 140.91072463989258, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.4285715222358704, 'reward_std': 0.40343862771987915, 'kl': 0.00232696533203125, 'epoch': 0.2} 4%|▍ | 65/1610 [14:34<5:25:08, 12.63s/it] 4%|▍ | 66/1610 [14:48<5:37:50, 13.13s/it] {'loss': 0.0001, 'grad_norm': 2.513792277659124, 'learning_rate': 9.590062111801241e-07, 'completion_length': 183.43750762939453, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.973214328289032, 'reward': 1.3125000596046448, 'reward_std': 0.3492938429117203, 'kl': 0.0025787353515625, 'epoch': 0.2} 4%|▍ | 66/1610 [14:48<5:37:50, 13.13s/it] 4%|▍ | 67/1610 [15:02<5:47:47, 13.52s/it] {'loss': 0.0001, 'grad_norm': 1.3481226797239503, 'learning_rate': 9.583850931677018e-07, 'completion_length': 157.3839340209961, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.4196429252624512, 'reward_std': 0.3885670602321625, 'kl': 0.001430511474609375, 'epoch': 0.21} 4%|▍ | 67/1610 [15:02<5:47:47, 13.52s/it] 4%|▍ | 68/1610 [15:16<5:45:23, 13.44s/it] {'loss': 0.0001, 'grad_norm': 3.157834156001942, 'learning_rate': 9.577639751552796e-07, 'completion_length': 162.24108123779297, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.4439669996500015, 'kl': 0.0024261474609375, 'epoch': 0.21} 4%|▍ | 68/1610 [15:16<5:45:23, 13.44s/it] 4%|▍ | 69/1610 [15:30<5:48:35, 13.57s/it] {'loss': 0.0001, 'grad_norm': 1.8683904703025056, 'learning_rate': 9.571428571428572e-07, 'completion_length': 176.3482208251953, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.2728744149208069, 'kl': 0.002593994140625, 'epoch': 0.21} 4%|▍ | 69/1610 [15:30<5:48:35, 13.57s/it] 4%|▍ | 70/1610 [15:44<5:53:20, 13.77s/it] {'loss': 0.0001, 'grad_norm': 1.534342287852406, 'learning_rate': 9.565217391304349e-07, 'completion_length': 177.24108123779297, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.4553572535514832, 'reward_std': 0.34335024654865265, 'kl': 0.002292633056640625, 'epoch': 0.22} 4%|▍ | 70/1610 [15:44<5:53:20, 13.77s/it] 4%|▍ | 71/1610 [15:58<5:57:15, 13.93s/it] {'loss': 0.0001, 'grad_norm': 2.045748173624275, 'learning_rate': 9.559006211180125e-07, 'completion_length': 179.75000762939453, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3392857909202576, 'reward_std': 0.4055987596511841, 'kl': 0.00209808349609375, 'epoch': 0.22} 4%|▍ | 71/1610 [15:58<5:57:15, 13.93s/it] 4%|▍ | 72/1610 [16:10<5:44:03, 13.42s/it] {'loss': 0.0001, 'grad_norm': 2.87832460192737, 'learning_rate': 9.5527950310559e-07, 'completion_length': 134.5982208251953, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.348214328289032, 'reward_std': 0.3888985812664032, 'kl': 0.0017242431640625, 'epoch': 0.22} 4%|▍ | 72/1610 [16:10<5:44:03, 13.42s/it] 5%|▍ | 73/1610 [16:23<5:39:55, 13.27s/it] {'loss': 0.0001, 'grad_norm': 1.3837030647676982, 'learning_rate': 9.546583850931676e-07, 'completion_length': 152.0982208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553572535514832, 'reward_std': 0.31622885167598724, 'kl': 0.002532958984375, 'epoch': 0.23} 5%|▍ | 73/1610 [16:23<5:39:55, 13.27s/it] 5%|▍ | 74/1610 [16:36<5:36:24, 13.14s/it] {'loss': 0.0001, 'grad_norm': 2.954330959298883, 'learning_rate': 9.540372670807452e-07, 'completion_length': 134.40179443359375, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.23717807233333588, 'kl': 0.001796722412109375, 'epoch': 0.23} 5%|▍ | 74/1610 [16:36<5:36:24, 13.14s/it] 5%|▍ | 75/1610 [16:48<5:24:41, 12.69s/it] {'loss': 0.0001, 'grad_norm': 1.4305737740725961, 'learning_rate': 9.534161490683229e-07, 'completion_length': 141.7678680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.3103121370077133, 'kl': 0.00191497802734375, 'epoch': 0.23} 5%|▍ | 75/1610 [16:48<5:24:41, 12.69s/it] 5%|▍ | 76/1610 [17:02<5:33:17, 13.04s/it] {'loss': 0.0001, 'grad_norm': 1.1589003363120307, 'learning_rate': 9.527950310559006e-07, 'completion_length': 184.74108123779297, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3125000596046448, 'reward_std': 0.2552971988916397, 'kl': 0.00250244140625, 'epoch': 0.24} 5%|▍ | 76/1610 [17:02<5:33:17, 13.04s/it] 5%|▍ | 77/1610 [17:14<5:25:02, 12.72s/it] {'loss': 0.0001, 'grad_norm': 2.362384744935279, 'learning_rate': 9.521739130434783e-07, 'completion_length': 126.03572463989258, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.26450884342193604, 'kl': 0.00229644775390625, 'epoch': 0.24} 5%|▍ | 77/1610 [17:14<5:25:02, 12.72s/it] 5%|▍ | 78/1610 [17:26<5:20:41, 12.56s/it] {'loss': 0.0001, 'grad_norm': 1.2711234971387055, 'learning_rate': 9.515527950310559e-07, 'completion_length': 150.09821701049805, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3688190281391144, 'kl': 0.00202178955078125, 'epoch': 0.24} 5%|▍ | 78/1610 [17:26<5:20:41, 12.56s/it] 5%|▍ | 79/1610 [17:39<5:23:25, 12.67s/it] {'loss': 0.0001, 'grad_norm': 1.300403108563162, 'learning_rate': 9.509316770186336e-07, 'completion_length': 143.81250762939453, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196428656578064, 'reward_std': 0.33185121417045593, 'kl': 0.0023956298828125, 'epoch': 0.25} 5%|▍ | 79/1610 [17:39<5:23:25, 12.67s/it] 5%|▍ | 80/1610 [17:51<5:19:23, 12.53s/it] {'loss': 0.0001, 'grad_norm': 1.4008933822782939, 'learning_rate': 9.503105590062112e-07, 'completion_length': 173.85714721679688, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.3564859628677368, 'kl': 0.0025787353515625, 'epoch': 0.25} 5%|▍ | 80/1610 [17:51<5:19:23, 12.53s/it] 5%|▌ | 81/1610 [18:04<5:20:39, 12.58s/it] {'loss': 0.0001, 'grad_norm': 1.0743172698418386, 'learning_rate': 9.496894409937888e-07, 'completion_length': 169.02679061889648, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.3571428656578064, 'reward_std': 0.3416573107242584, 'kl': 0.001983642578125, 'epoch': 0.25} 5%|▌ | 81/1610 [18:04<5:20:39, 12.58s/it] 5%|▌ | 82/1610 [18:17<5:28:33, 12.90s/it] {'loss': 0.0001, 'grad_norm': 0.9374010155325527, 'learning_rate': 9.490683229813665e-07, 'completion_length': 172.12500762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.5892857909202576, 'reward_std': 0.3056681454181671, 'kl': 0.001712799072265625, 'epoch': 0.25} 5%|▌ | 82/1610 [18:17<5:28:33, 12.90s/it] 5%|▌ | 83/1610 [18:32<5:40:49, 13.39s/it] {'loss': 0.0001, 'grad_norm': 0.9456936423345788, 'learning_rate': 9.48447204968944e-07, 'completion_length': 181.15179443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.5535714626312256, 'reward_std': 0.3642582297325134, 'kl': 0.002655029296875, 'epoch': 0.26} 5%|▌ | 83/1610 [18:32<5:40:49, 13.39s/it] 5%|▌ | 84/1610 [18:44<5:27:44, 12.89s/it] {'loss': 0.0001, 'grad_norm': 1.8819553633444173, 'learning_rate': 9.478260869565216e-07, 'completion_length': 143.8214340209961, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.30390138924121857, 'kl': 0.00238037109375, 'epoch': 0.26} 5%|▌ | 84/1610 [18:44<5:27:44, 12.89s/it] 5%|▌ | 85/1610 [18:57<5:28:47, 12.94s/it] {'loss': 0.0001, 'grad_norm': 1.2945957340113257, 'learning_rate': 9.472049689440993e-07, 'completion_length': 162.43750762939453, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3839285969734192, 'reward_std': 0.3700314164161682, 'kl': 0.00272369384765625, 'epoch': 0.26} 5%|▌ | 85/1610 [18:57<5:28:47, 12.94s/it] 5%|▌ | 86/1610 [19:10<5:29:57, 12.99s/it] {'loss': 0.0001, 'grad_norm': 2.0480136645930043, 'learning_rate': 9.46583850931677e-07, 'completion_length': 138.40179443359375, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5982143878936768, 'reward_std': 0.3823588639497757, 'kl': 0.00237274169921875, 'epoch': 0.27} 5%|▌ | 86/1610 [19:10<5:29:57, 12.99s/it] 5%|▌ | 87/1610 [19:24<5:40:33, 13.42s/it] {'loss': 0.0001, 'grad_norm': 0.7353688081375601, 'learning_rate': 9.459627329192546e-07, 'completion_length': 189.86608123779297, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.973214328289032, 'reward': 1.3928572535514832, 'reward_std': 0.22082290053367615, 'kl': 0.002765655517578125, 'epoch': 0.27} 5%|▌ | 87/1610 [19:24<5:40:33, 13.42s/it] 5%|▌ | 88/1610 [19:38<5:42:57, 13.52s/it] {'loss': 0.0001, 'grad_norm': 1.3901489746007636, 'learning_rate': 9.453416149068323e-07, 'completion_length': 145.5089340209961, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.34269504249095917, 'kl': 0.001850128173828125, 'epoch': 0.27} 5%|▌ | 88/1610 [19:38<5:42:57, 13.52s/it] 6%|▌ | 89/1610 [19:50<5:29:46, 13.01s/it] {'loss': 0.0001, 'grad_norm': 1.258424049750692, 'learning_rate': 9.447204968944099e-07, 'completion_length': 142.92857360839844, 'rewards/accuracy_reward': 0.3125000223517418, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.2858654633164406, 'kl': 0.00266265869140625, 'epoch': 0.28} 6%|▌ | 89/1610 [19:50<5:29:46, 13.01s/it] 6%|▌ | 90/1610 [20:04<5:42:52, 13.53s/it] {'loss': 0.0001, 'grad_norm': 1.4698905544138299, 'learning_rate': 9.440993788819875e-07, 'completion_length': 196.39286041259766, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.2857143878936768, 'reward_std': 0.23775441944599152, 'kl': 0.0023956298828125, 'epoch': 0.28} 6%|▌ | 90/1610 [20:04<5:42:52, 13.53s/it] 6%|▌ | 91/1610 [20:19<5:53:37, 13.97s/it] {'loss': 0.0001, 'grad_norm': 1.2828365807993365, 'learning_rate': 9.434782608695652e-07, 'completion_length': 155.93750762939453, 'rewards/accuracy_reward': 0.2321428656578064, 'rewards/format_reward': 0.973214328289032, 'reward': 1.2053571939468384, 'reward_std': 0.27822330594062805, 'kl': 0.0023956298828125, 'epoch': 0.28} 6%|▌ | 91/1610 [20:19<5:53:37, 13.97s/it] 6%|▌ | 92/1610 [20:32<5:42:47, 13.55s/it] {'loss': 0.0001, 'grad_norm': 1.2958699335560022, 'learning_rate': 9.428571428571428e-07, 'completion_length': 172.2857208251953, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.4285715222358704, 'reward_std': 0.3026157021522522, 'kl': 0.002288818359375, 'epoch': 0.29} 6%|▌ | 92/1610 [20:32<5:42:47, 13.55s/it] 6%|▌ | 93/1610 [20:43<5:26:11, 12.90s/it] {'loss': 0.0001, 'grad_norm': 1.494514212909905, 'learning_rate': 9.422360248447204e-07, 'completion_length': 138.02679061889648, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3862616568803787, 'kl': 0.002471923828125, 'epoch': 0.29} 6%|▌ | 93/1610 [20:43<5:26:11, 12.90s/it] 6%|▌ | 94/1610 [20:57<5:31:12, 13.11s/it] {'loss': 0.0001, 'grad_norm': 2.808295611440555, 'learning_rate': 9.41614906832298e-07, 'completion_length': 147.40179443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.535714328289032, 'reward_std': 0.4055155813694, 'kl': 0.002349853515625, 'epoch': 0.29} 6%|▌ | 94/1610 [20:57<5:31:12, 13.11s/it] 6%|▌ | 95/1610 [21:11<5:35:25, 13.28s/it] {'loss': 0.0001, 'grad_norm': 2.2030003075062554, 'learning_rate': 9.409937888198758e-07, 'completion_length': 161.4464340209961, 'rewards/accuracy_reward': 0.2857143059372902, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.2678571939468384, 'reward_std': 0.4428874999284744, 'kl': 0.00344085693359375, 'epoch': 0.3} 6%|▌ | 95/1610 [21:11<5:35:25, 13.28s/it] 6%|▌ | 96/1610 [21:23<5:25:50, 12.91s/it] {'loss': 0.0001, 'grad_norm': 0.8485881456952379, 'learning_rate': 9.403726708074534e-07, 'completion_length': 160.85715103149414, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535714626312256, 'reward_std': 0.29257139563560486, 'kl': 0.0029754638671875, 'epoch': 0.3} 6%|▌ | 96/1610 [21:23<5:25:50, 12.91s/it] 6%|▌ | 97/1610 [21:38<5:40:17, 13.49s/it] {'loss': 0.0001, 'grad_norm': 1.5807423603775006, 'learning_rate': 9.39751552795031e-07, 'completion_length': 196.66964721679688, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.973214328289032, 'reward': 1.446428656578064, 'reward_std': 0.3915945887565613, 'kl': 0.002899169921875, 'epoch': 0.3} 6%|▌ | 97/1610 [21:38<5:40:17, 13.49s/it] 6%|▌ | 98/1610 [21:51<5:36:35, 13.36s/it] {'loss': 0.0001, 'grad_norm': 1.1078389487712625, 'learning_rate': 9.391304347826087e-07, 'completion_length': 188.33036041259766, 'rewards/accuracy_reward': 0.3125000223517418, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3035715222358704, 'reward_std': 0.3727274090051651, 'kl': 0.0033721923828125, 'epoch': 0.3} 6%|▌ | 98/1610 [21:51<5:36:35, 13.36s/it] 6%|▌ | 99/1610 [22:00<5:07:25, 12.21s/it] {'loss': 0.0001, 'grad_norm': 2.2701535529043, 'learning_rate': 9.385093167701863e-07, 'completion_length': 109.18750381469727, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.30330248177051544, 'kl': 0.00228118896484375, 'epoch': 0.31} 6%|▌ | 99/1610 [22:00<5:07:25, 12.21s/it] 6%|▌ | 100/1610 [22:14<5:22:14, 12.80s/it] {'loss': 0.0001, 'grad_norm': 1.3963903587842272, 'learning_rate': 9.37888198757764e-07, 'completion_length': 190.5357208251953, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4285715222358704, 'reward_std': 0.3974950462579727, 'kl': 0.00322723388671875, 'epoch': 0.31} 6%|▌ | 100/1610 [22:14<5:22:14, 12.80s/it] 6%|▋ | 101/1610 [23:25<12:37:20, 30.11s/it] {'loss': 0.0001, 'grad_norm': 1.3335959153808923, 'learning_rate': 9.372670807453416e-07, 'completion_length': 169.75894165039062, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.3913824260234833, 'kl': 0.00246429443359375, 'epoch': 0.31} 6%|▋ | 101/1610 [23:25<12:37:20, 30.11s/it] 6%|▋ | 102/1610 [23:41<10:48:36, 25.81s/it] {'loss': 0.0001, 'grad_norm': 2.772430356088683, 'learning_rate': 9.366459627329192e-07, 'completion_length': 172.71428680419922, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.25850892812013626, 'kl': 0.003570556640625, 'epoch': 0.32} 6%|▋ | 102/1610 [23:41<10:48:36, 25.81s/it] 6%|▋ | 103/1610 [23:56<9:30:37, 22.72s/it] {'loss': 0.0001, 'grad_norm': 1.1269599462599595, 'learning_rate': 9.360248447204968e-07, 'completion_length': 164.1964340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2314438372850418, 'kl': 0.0023956298828125, 'epoch': 0.32} 6%|▋ | 103/1610 [23:56<9:30:37, 22.72s/it] 6%|▋ | 104/1610 [24:12<8:36:56, 20.60s/it] {'loss': 0.0001, 'grad_norm': 1.1451671048720333, 'learning_rate': 9.354037267080745e-07, 'completion_length': 166.54464721679688, 'rewards/accuracy_reward': 0.39285717345774174, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.383928656578064, 'reward_std': 0.23838482797145844, 'kl': 0.00260162353515625, 'epoch': 0.32} 6%|▋ | 104/1610 [24:12<8:36:56, 20.60s/it] 7%|▋ | 105/1610 [24:29<8:08:20, 19.47s/it] {'loss': 0.0001, 'grad_norm': 1.363279119846215, 'learning_rate': 9.347826086956522e-07, 'completion_length': 173.80357360839844, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285714626312256, 'reward_std': 0.29999861121177673, 'kl': 0.00323486328125, 'epoch': 0.33} 7%|▋ | 105/1610 [24:29<8:08:20, 19.47s/it] 7%|▋ | 106/1610 [24:45<7:44:00, 18.51s/it] {'loss': 0.0001, 'grad_norm': 1.3299431847092924, 'learning_rate': 9.341614906832299e-07, 'completion_length': 147.1428680419922, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.973214328289032, 'reward': 1.4821429252624512, 'reward_std': 0.25950637459754944, 'kl': 0.0023193359375, 'epoch': 0.33} 7%|▋ | 106/1610 [24:45<7:44:00, 18.51s/it] 7%|▋ | 107/1610 [25:00<7:20:32, 17.59s/it] {'loss': 0.0001, 'grad_norm': 1.1343171467313333, 'learning_rate': 9.335403726708074e-07, 'completion_length': 158.75000762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2960902005434036, 'kl': 0.00305938720703125, 'epoch': 0.33} 7%|▋ | 107/1610 [25:00<7:20:32, 17.59s/it] 7%|▋ | 108/1610 [25:17<7:10:57, 17.22s/it] {'loss': 0.0001, 'grad_norm': 2.564924602257394, 'learning_rate': 9.32919254658385e-07, 'completion_length': 197.61607360839844, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.955357164144516, 'reward': 1.3482143878936768, 'reward_std': 0.38626593351364136, 'kl': 0.00270843505859375, 'epoch': 0.34} 7%|▋ | 108/1610 [25:17<7:10:57, 17.22s/it] 7%|▋ | 109/1610 [25:32<6:57:10, 16.68s/it] {'loss': 0.0001, 'grad_norm': 1.2129637894913623, 'learning_rate': 9.322981366459626e-07, 'completion_length': 158.04465103149414, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625001192092896, 'reward_std': 0.26941466331481934, 'kl': 0.00327301025390625, 'epoch': 0.34} 7%|▋ | 109/1610 [25:32<6:57:10, 16.68s/it] 7%|▋ | 110/1610 [25:48<6:51:36, 16.46s/it] {'loss': 0.0001, 'grad_norm': 0.9910941259321664, 'learning_rate': 9.316770186335403e-07, 'completion_length': 162.68750762939453, 'rewards/accuracy_reward': 0.2857143059372902, 'rewards/format_reward': 1.0, 'reward': 1.2857143878936768, 'reward_std': 0.237776979804039, 'kl': 0.00328826904296875, 'epoch': 0.34} 7%|▋ | 110/1610 [25:48<6:51:36, 16.46s/it] 7%|▋ | 111/1610 [26:05<6:54:08, 16.58s/it] {'loss': 0.0001, 'grad_norm': 1.6002124961570539, 'learning_rate': 9.310559006211179e-07, 'completion_length': 157.75000762939453, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4375000596046448, 'reward_std': 0.4088250696659088, 'kl': 0.00368499755859375, 'epoch': 0.34} 7%|▋ | 111/1610 [26:05<6:54:08, 16.58s/it] 7%|▋ | 112/1610 [26:21<6:52:45, 16.53s/it] {'loss': 0.0001, 'grad_norm': 1.0145817604618896, 'learning_rate': 9.304347826086955e-07, 'completion_length': 180.67858123779297, 'rewards/accuracy_reward': 0.3750000223517418, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.29361197352409363, 'kl': 0.00348663330078125, 'epoch': 0.35} 7%|▋ | 112/1610 [26:21<6:52:45, 16.53s/it] 7%|▋ | 113/1610 [26:38<6:52:36, 16.54s/it] {'loss': 0.0001, 'grad_norm': 0.9282105323224884, 'learning_rate': 9.298136645962732e-07, 'completion_length': 216.50000762939453, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.3660715222358704, 'reward_std': 0.42665766179561615, 'kl': 0.00372314453125, 'epoch': 0.35} 7%|▋ | 113/1610 [26:38<6:52:36, 16.54s/it] 7%|▋ | 114/1610 [26:54<6:51:41, 16.51s/it] {'loss': 0.0001, 'grad_norm': 1.6734902834402412, 'learning_rate': 9.291925465838509e-07, 'completion_length': 165.5982208251953, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.321428656578064, 'reward_std': 0.4149971604347229, 'kl': 0.00299072265625, 'epoch': 0.35} 7%|▋ | 114/1610 [26:54<6:51:41, 16.51s/it] 7%|▋ | 115/1610 [27:11<6:51:46, 16.53s/it] {'loss': 0.0001, 'grad_norm': 0.9784064162646492, 'learning_rate': 9.285714285714285e-07, 'completion_length': 190.3303680419922, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3750000596046448, 'reward_std': 0.35254426300525665, 'kl': 0.00351715087890625, 'epoch': 0.36} 7%|▋ | 115/1610 [27:11<6:51:46, 16.53s/it] 7%|▋ | 116/1610 [27:28<6:53:00, 16.59s/it] {'loss': 0.0001, 'grad_norm': 0.836913162124014, 'learning_rate': 9.279503105590062e-07, 'completion_length': 183.68750762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.5982143878936768, 'reward_std': 0.3187367618083954, 'kl': 0.0030517578125, 'epoch': 0.36} 7%|▋ | 116/1610 [27:28<6:53:00, 16.59s/it] 7%|▋ | 117/1610 [27:42<6:36:49, 15.95s/it] {'loss': 0.0001, 'grad_norm': 1.125816419194468, 'learning_rate': 9.273291925465838e-07, 'completion_length': 137.50000762939453, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.18787721544504166, 'kl': 0.003173828125, 'epoch': 0.36} 7%|▋ | 117/1610 [27:42<6:36:49, 15.95s/it] 7%|▋ | 118/1610 [27:56<6:24:31, 15.46s/it] {'loss': 0.0001, 'grad_norm': 2.4384737587751855, 'learning_rate': 9.267080745341614e-07, 'completion_length': 127.03572082519531, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.34923477470874786, 'kl': 0.003143310546875, 'epoch': 0.37} 7%|▋ | 118/1610 [27:56<6:24:31, 15.46s/it] 7%|▋ | 119/1610 [28:12<6:29:12, 15.66s/it] {'loss': 0.0001, 'grad_norm': 2.997737390087332, 'learning_rate': 9.260869565217391e-07, 'completion_length': 152.42858123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.39017005264759064, 'kl': 0.00374603271484375, 'epoch': 0.37} 7%|▋ | 119/1610 [28:12<6:29:12, 15.66s/it] 7%|▋ | 120/1610 [28:29<6:37:18, 16.00s/it] {'loss': 0.0002, 'grad_norm': 2.138146233861115, 'learning_rate': 9.254658385093167e-07, 'completion_length': 162.2589340209961, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3928571939468384, 'reward_std': 0.3554064780473709, 'kl': 0.004241943359375, 'epoch': 0.37} 7%|▋ | 120/1610 [28:29<6:37:18, 16.00s/it] 8%|▊ | 121/1610 [28:46<6:41:38, 16.18s/it] {'loss': 0.0002, 'grad_norm': 2.7853329915315905, 'learning_rate': 9.248447204968943e-07, 'completion_length': 184.21429443359375, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3928571939468384, 'reward_std': 0.3189248591661453, 'kl': 0.00379180908203125, 'epoch': 0.38} 8%|▊ | 121/1610 [28:46<6:41:38, 16.18s/it] 8%|▊ | 122/1610 [29:02<6:41:10, 16.18s/it] {'loss': 0.0001, 'grad_norm': 1.7915350662473761, 'learning_rate': 9.24223602484472e-07, 'completion_length': 133.48215103149414, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4642857909202576, 'reward_std': 0.4158078283071518, 'kl': 0.00360870361328125, 'epoch': 0.38} 8%|▊ | 122/1610 [29:02<6:41:10, 16.18s/it] 8%|▊ | 123/1610 [29:16<6:23:58, 15.49s/it] {'loss': 0.0001, 'grad_norm': 1.7814937795031165, 'learning_rate': 9.236024844720497e-07, 'completion_length': 134.50893783569336, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.297916054725647, 'kl': 0.00274658203125, 'epoch': 0.38} 8%|▊ | 123/1610 [29:16<6:23:58, 15.49s/it] 8%|▊ | 124/1610 [29:32<6:28:16, 15.68s/it] {'loss': 0.0001, 'grad_norm': 1.0040608770828452, 'learning_rate': 9.229813664596273e-07, 'completion_length': 173.6071548461914, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3214285969734192, 'reward_std': 0.2035771906375885, 'kl': 0.00347137451171875, 'epoch': 0.39} 8%|▊ | 124/1610 [29:32<6:28:16, 15.68s/it] 8%|▊ | 125/1610 [29:46<6:14:31, 15.13s/it] {'loss': 0.0001, 'grad_norm': 4.392850798533914, 'learning_rate': 9.22360248447205e-07, 'completion_length': 141.26786041259766, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.3492938429117203, 'kl': 0.00351715087890625, 'epoch': 0.39} 8%|▊ | 125/1610 [29:46<6:14:31, 15.13s/it] 8%|▊ | 126/1610 [30:01<6:13:04, 15.08s/it] {'loss': 0.0002, 'grad_norm': 2.3403222671280375, 'learning_rate': 9.217391304347826e-07, 'completion_length': 140.1696548461914, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.2857143878936768, 'reward_std': 0.38053302466869354, 'kl': 0.006103515625, 'epoch': 0.39} 8%|▊ | 126/1610 [30:01<6:13:04, 15.08s/it] 8%|▊ | 127/1610 [30:18<6:25:29, 15.60s/it] {'loss': 0.0001, 'grad_norm': 1.240050736823393, 'learning_rate': 9.211180124223602e-07, 'completion_length': 212.67858123779297, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.3303675800561905, 'kl': 0.00374603271484375, 'epoch': 0.39} 8%|▊ | 127/1610 [30:18<6:25:29, 15.60s/it] 8%|▊ | 128/1610 [30:32<6:17:29, 15.28s/it] {'loss': 0.0002, 'grad_norm': 1.548742107613559, 'learning_rate': 9.204968944099379e-07, 'completion_length': 133.36607360839844, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.29730819165706635, 'kl': 0.00376129150390625, 'epoch': 0.4} 8%|▊ | 128/1610 [30:32<6:17:29, 15.28s/it] 8%|▊ | 129/1610 [30:48<6:22:05, 15.48s/it] {'loss': 0.0001, 'grad_norm': 1.1479543709957367, 'learning_rate': 9.198757763975155e-07, 'completion_length': 164.10714721679688, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.40389329195022583, 'kl': 0.003631591796875, 'epoch': 0.4} 8%|▊ | 129/1610 [30:48<6:22:05, 15.48s/it] 8%|▊ | 130/1610 [31:03<6:20:42, 15.43s/it] {'loss': 0.0002, 'grad_norm': 2.5394469529585995, 'learning_rate': 9.19254658385093e-07, 'completion_length': 166.54464721679688, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.3045148700475693, 'kl': 0.0044708251953125, 'epoch': 0.4} 8%|▊ | 130/1610 [31:03<6:20:42, 15.43s/it] 8%|▊ | 131/1610 [31:18<6:16:02, 15.26s/it] {'loss': 0.0002, 'grad_norm': 1.5948945171940896, 'learning_rate': 9.186335403726707e-07, 'completion_length': 144.99108123779297, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.34868037700653076, 'kl': 0.00394439697265625, 'epoch': 0.41} 8%|▊ | 131/1610 [31:18<6:16:02, 15.26s/it] 8%|▊ | 132/1610 [31:34<6:17:05, 15.31s/it] {'loss': 0.0002, 'grad_norm': 1.4185143442586836, 'learning_rate': 9.180124223602484e-07, 'completion_length': 151.7857208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.41211436688899994, 'kl': 0.0040740966796875, 'epoch': 0.41} 8%|▊ | 132/1610 [31:34<6:17:05, 15.31s/it] 8%|▊ | 133/1610 [31:50<6:24:46, 15.63s/it] {'loss': 0.0001, 'grad_norm': 1.5263697032183883, 'learning_rate': 9.17391304347826e-07, 'completion_length': 158.85714721679688, 'rewards/accuracy_reward': 0.2946428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.285714328289032, 'reward_std': 0.421536460518837, 'kl': 0.0037078857421875, 'epoch': 0.41} 8%|▊ | 133/1610 [31:50<6:24:46, 15.63s/it] 8%|▊ | 134/1610 [32:04<6:14:50, 15.24s/it] {'loss': 0.0002, 'grad_norm': 1.478961194384616, 'learning_rate': 9.167701863354037e-07, 'completion_length': 163.7678680419922, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.2702430784702301, 'kl': 0.00439453125, 'epoch': 0.42} 8%|▊ | 134/1610 [32:04<6:14:50, 15.24s/it] 8%|▊ | 135/1610 [32:20<6:15:28, 15.27s/it] {'loss': 0.0002, 'grad_norm': 2.1620659668875986, 'learning_rate': 9.161490683229813e-07, 'completion_length': 153.5357208251953, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.30659179389476776, 'kl': 0.0048675537109375, 'epoch': 0.42} 8%|▊ | 135/1610 [32:20<6:15:28, 15.27s/it] 8%|▊ | 136/1610 [32:36<6:24:58, 15.67s/it] {'loss': 0.0002, 'grad_norm': 2.0520767974043665, 'learning_rate': 9.155279503105589e-07, 'completion_length': 166.3482208251953, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4017857909202576, 'reward_std': 0.3375798612833023, 'kl': 0.00518798828125, 'epoch': 0.42} 8%|▊ | 136/1610 [32:36<6:24:58, 15.67s/it] 9%|▊ | 137/1610 [32:52<6:27:02, 15.77s/it] {'loss': 0.0002, 'grad_norm': 1.257580993968175, 'learning_rate': 9.149068322981366e-07, 'completion_length': 161.58929443359375, 'rewards/accuracy_reward': 0.3660714626312256, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.2696296125650406, 'kl': 0.00399017333984375, 'epoch': 0.43} 9%|▊ | 137/1610 [32:52<6:27:02, 15.77s/it] 9%|▊ | 138/1610 [33:07<6:20:26, 15.51s/it] {'loss': 0.0001, 'grad_norm': 1.8722006365890376, 'learning_rate': 9.142857142857142e-07, 'completion_length': 156.10715103149414, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.348214328289032, 'reward_std': 0.35372215509414673, 'kl': 0.00307464599609375, 'epoch': 0.43} 9%|▊ | 138/1610 [33:07<6:20:26, 15.51s/it] 9%|▊ | 139/1610 [33:23<6:22:31, 15.60s/it] {'loss': 0.0002, 'grad_norm': 1.472380528890414, 'learning_rate': 9.136645962732918e-07, 'completion_length': 159.0, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5535714626312256, 'reward_std': 0.4515632688999176, 'kl': 0.0038604736328125, 'epoch': 0.43} 9%|▊ | 139/1610 [33:23<6:22:31, 15.60s/it] 9%|▊ | 140/1610 [33:39<6:22:53, 15.63s/it] {'loss': 0.0002, 'grad_norm': 1.01689095919514, 'learning_rate': 9.130434782608695e-07, 'completion_length': 150.17858123779297, 'rewards/accuracy_reward': 0.3750000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.3532022535800934, 'kl': 0.0042266845703125, 'epoch': 0.43} 9%|▊ | 140/1610 [33:39<6:22:53, 15.63s/it] 9%|▉ | 141/1610 [33:54<6:19:52, 15.52s/it] {'loss': 0.0002, 'grad_norm': 1.462401132692554, 'learning_rate': 9.124223602484472e-07, 'completion_length': 174.68750762939453, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.370031401515007, 'kl': 0.0040740966796875, 'epoch': 0.44} 9%|▉ | 141/1610 [33:54<6:19:52, 15.52s/it] 9%|▉ | 142/1610 [34:07<6:01:21, 14.77s/it] {'loss': 0.0002, 'grad_norm': 4.65315447103672, 'learning_rate': 9.118012422360248e-07, 'completion_length': 118.84821701049805, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.2960958033800125, 'kl': 0.0041351318359375, 'epoch': 0.44} 9%|▉ | 142/1610 [34:07<6:01:21, 14.77s/it] 9%|▉ | 143/1610 [34:23<6:08:13, 15.06s/it] {'loss': 0.0002, 'grad_norm': 2.1781271451165636, 'learning_rate': 9.111801242236025e-07, 'completion_length': 142.74108123779297, 'rewards/accuracy_reward': 0.6071428805589676, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.28828461468219757, 'kl': 0.0042572021484375, 'epoch': 0.44} 9%|▉ | 143/1610 [34:23<6:08:13, 15.06s/it] 9%|▉ | 144/1610 [34:36<5:56:30, 14.59s/it] {'loss': 0.0002, 'grad_norm': 1.5106388947985891, 'learning_rate': 9.105590062111801e-07, 'completion_length': 131.62500762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.3636982589960098, 'kl': 0.0039215087890625, 'epoch': 0.45} 9%|▉ | 144/1610 [34:36<5:56:30, 14.59s/it] 9%|▉ | 145/1610 [34:52<6:01:50, 14.82s/it] {'loss': 0.0002, 'grad_norm': 0.861960984829215, 'learning_rate': 9.099378881987577e-07, 'completion_length': 143.2232208251953, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.25770069658756256, 'kl': 0.0043182373046875, 'epoch': 0.45} 9%|▉ | 145/1610 [34:52<6:01:50, 14.82s/it] 9%|▉ | 146/1610 [35:08<6:15:11, 15.38s/it] {'loss': 0.0002, 'grad_norm': 1.4241260142264673, 'learning_rate': 9.093167701863354e-07, 'completion_length': 186.12500762939453, 'rewards/accuracy_reward': 0.2946428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.285714328289032, 'reward_std': 0.3174411952495575, 'kl': 0.005126953125, 'epoch': 0.45} 9%|▉ | 146/1610 [35:08<6:15:11, 15.38s/it] 9%|▉ | 147/1610 [35:23<6:08:47, 15.12s/it] {'loss': 0.0002, 'grad_norm': 1.158707478559955, 'learning_rate': 9.08695652173913e-07, 'completion_length': 152.3839340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2404673844575882, 'kl': 0.00445556640625, 'epoch': 0.46} 9%|▉ | 147/1610 [35:23<6:08:47, 15.12s/it] 9%|▉ | 148/1610 [35:35<5:46:47, 14.23s/it] {'loss': 0.0002, 'grad_norm': 1.2088836026200274, 'learning_rate': 9.080745341614906e-07, 'completion_length': 105.71429061889648, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2579156160354614, 'kl': 0.0038604736328125, 'epoch': 0.46} 9%|▉ | 148/1610 [35:35<5:46:47, 14.23s/it] 9%|▉ | 149/1610 [35:50<5:48:23, 14.31s/it] {'loss': 0.0002, 'grad_norm': 1.460034329712706, 'learning_rate': 9.074534161490683e-07, 'completion_length': 132.64286041259766, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.31865358352661133, 'kl': 0.00444793701171875, 'epoch': 0.46} 9%|▉ | 149/1610 [35:50<5:48:23, 14.31s/it] 9%|▉ | 150/1610 [36:04<5:52:02, 14.47s/it] {'loss': 0.0002, 'grad_norm': 3.290603262084739, 'learning_rate': 9.06832298136646e-07, 'completion_length': 137.5714340209961, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.3952852189540863, 'kl': 0.00377655029296875, 'epoch': 0.47} 9%|▉ | 150/1610 [36:04<5:52:02, 14.47s/it] 9%|▉ | 151/1610 [36:20<6:00:00, 14.81s/it] {'loss': 0.0002, 'grad_norm': 0.9979305841884388, 'learning_rate': 9.062111801242236e-07, 'completion_length': 156.0714340209961, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.3183113932609558, 'kl': 0.004058837890625, 'epoch': 0.47} 9%|▉ | 151/1610 [36:20<6:00:00, 14.81s/it] 9%|▉ | 152/1610 [36:37<6:19:11, 15.60s/it] {'loss': 0.0002, 'grad_norm': 2.1114666381424376, 'learning_rate': 9.055900621118013e-07, 'completion_length': 194.5357208251953, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.973214328289032, 'reward': 1.348214328289032, 'reward_std': 0.4334597736597061, 'kl': 0.00439453125, 'epoch': 0.47} 9%|▉ | 152/1610 [36:37<6:19:11, 15.60s/it] 10%|▉ | 153/1610 [36:54<6:23:37, 15.80s/it] {'loss': 0.0002, 'grad_norm': 1.645085793335784, 'learning_rate': 9.049689440993789e-07, 'completion_length': 173.71429443359375, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.4211323410272598, 'kl': 0.004913330078125, 'epoch': 0.48} 10%|▉ | 153/1610 [36:54<6:23:37, 15.80s/it] 10%|▉ | 154/1610 [37:08<6:09:36, 15.23s/it] {'loss': 0.0002, 'grad_norm': 2.213237617710594, 'learning_rate': 9.043478260869564e-07, 'completion_length': 143.4464340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.3940672427415848, 'kl': 0.004150390625, 'epoch': 0.48} 10%|▉ | 154/1610 [37:08<6:09:36, 15.23s/it] 10%|▉ | 155/1610 [37:25<6:21:54, 15.75s/it] {'loss': 0.0002, 'grad_norm': 1.1623477916047762, 'learning_rate': 9.037267080745341e-07, 'completion_length': 163.92858123779297, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4107143878936768, 'reward_std': 0.4106788784265518, 'kl': 0.004241943359375, 'epoch': 0.48} 10%|▉ | 155/1610 [37:25<6:21:54, 15.75s/it] 10%|▉ | 156/1610 [37:39<6:15:51, 15.51s/it] {'loss': 0.0002, 'grad_norm': 1.4214627676093514, 'learning_rate': 9.031055900621117e-07, 'completion_length': 153.80358123779297, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.301482230424881, 'kl': 0.0037994384765625, 'epoch': 0.48} 10%|▉ | 156/1610 [37:39<6:15:51, 15.51s/it] 10%|▉ | 157/1610 [37:53<5:59:05, 14.83s/it] {'loss': 0.0001, 'grad_norm': 2.2845864088813412, 'learning_rate': 9.024844720496893e-07, 'completion_length': 126.84822082519531, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.3252524212002754, 'kl': 0.00347137451171875, 'epoch': 0.49} 10%|▉ | 157/1610 [37:53<5:59:05, 14.83s/it] 10%|▉ | 158/1610 [38:08<6:03:09, 15.01s/it] {'loss': 0.0002, 'grad_norm': 1.056288835658631, 'learning_rate': 9.01863354037267e-07, 'completion_length': 183.5089340209961, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.33488383889198303, 'kl': 0.0045318603515625, 'epoch': 0.49} 10%|▉ | 158/1610 [38:08<6:03:09, 15.01s/it] 10%|▉ | 159/1610 [38:24<6:07:11, 15.18s/it] {'loss': 0.0002, 'grad_norm': 1.315592898956033, 'learning_rate': 9.012422360248447e-07, 'completion_length': 147.30357360839844, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.22504299134016037, 'kl': 0.0046234130859375, 'epoch': 0.49} 10%|▉ | 159/1610 [38:24<6:07:11, 15.18s/it] 10%|▉ | 160/1610 [38:41<6:23:41, 15.88s/it] {'loss': 0.0002, 'grad_norm': 1.2637960401869084, 'learning_rate': 9.006211180124223e-07, 'completion_length': 196.3928680419922, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 0.973214328289032, 'reward': 1.5178571939468384, 'reward_std': 0.4146730452775955, 'kl': 0.0042266845703125, 'epoch': 0.5} 10%|▉ | 160/1610 [38:41<6:23:41, 15.88s/it] 10%|█ | 161/1610 [38:58<6:28:02, 16.07s/it] {'loss': 0.0002, 'grad_norm': 1.5563466698866248, 'learning_rate': 9e-07, 'completion_length': 167.40179443359375, 'rewards/accuracy_reward': 0.3482142984867096, 'rewards/format_reward': 1.0, 'reward': 1.348214328289032, 'reward_std': 0.23265621066093445, 'kl': 0.004486083984375, 'epoch': 0.5} 10%|█ | 161/1610 [38:58<6:28:02, 16.07s/it] 10%|█ | 162/1610 [39:13<6:18:28, 15.68s/it] {'loss': 0.0002, 'grad_norm': 1.0295392109857815, 'learning_rate': 8.993788819875776e-07, 'completion_length': 162.75000762939453, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.2404673844575882, 'kl': 0.00409698486328125, 'epoch': 0.5} 10%|█ | 162/1610 [39:13<6:18:28, 15.68s/it] 10%|█ | 163/1610 [39:27<6:12:53, 15.46s/it] {'loss': 0.0002, 'grad_norm': 1.1449772170768302, 'learning_rate': 8.987577639751552e-07, 'completion_length': 149.62500762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5714285969734192, 'reward_std': 0.3166128247976303, 'kl': 0.0040435791015625, 'epoch': 0.51} 10%|█ | 163/1610 [39:27<6:12:53, 15.46s/it] 10%|█ | 164/1610 [39:41<6:01:45, 15.01s/it] {'loss': 0.0002, 'grad_norm': 4.234115979680448, 'learning_rate': 8.981366459627329e-07, 'completion_length': 128.19643783569336, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.325252428650856, 'kl': 0.0046844482421875, 'epoch': 0.51} 10%|█ | 164/1610 [39:41<6:01:45, 15.01s/it] 10%|█ | 165/1610 [39:57<6:02:01, 15.03s/it] {'loss': 0.0002, 'grad_norm': 1.276915755517863, 'learning_rate': 8.975155279503105e-07, 'completion_length': 175.47322845458984, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.32283324003219604, 'kl': 0.0047607421875, 'epoch': 0.51} 10%|█ | 165/1610 [39:57<6:02:01, 15.03s/it] 10%|█ | 166/1610 [40:13<6:11:53, 15.45s/it] {'loss': 0.0002, 'grad_norm': 1.8023330714204255, 'learning_rate': 8.968944099378881e-07, 'completion_length': 156.83929443359375, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.544642984867096, 'reward_std': 0.3097887188196182, 'kl': 0.0048065185546875, 'epoch': 0.52} 10%|█ | 166/1610 [40:13<6:11:53, 15.45s/it] 10%|█ | 167/1610 [40:28<6:06:06, 15.22s/it] {'loss': 0.0002, 'grad_norm': 3.034103780083778, 'learning_rate': 8.962732919254658e-07, 'completion_length': 166.54464721679688, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.2560841739177704, 'kl': 0.0047149658203125, 'epoch': 0.52} 10%|█ | 167/1610 [40:28<6:06:06, 15.22s/it] 10%|█ | 168/1610 [40:43<6:07:21, 15.29s/it] {'loss': 0.0002, 'grad_norm': 1.093800285206245, 'learning_rate': 8.956521739130435e-07, 'completion_length': 156.18750762939453, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.28707224130630493, 'kl': 0.004180908203125, 'epoch': 0.52} 10%|█ | 168/1610 [40:43<6:07:21, 15.29s/it] 10%|█ | 169/1610 [40:58<6:06:45, 15.27s/it] {'loss': 0.0002, 'grad_norm': 3.1359458412605834, 'learning_rate': 8.950310559006211e-07, 'completion_length': 169.68750762939453, 'rewards/accuracy_reward': 0.258928582072258, 'rewards/format_reward': 1.0, 'reward': 1.2589285969734192, 'reward_std': 0.27414026856422424, 'kl': 0.0047760009765625, 'epoch': 0.52} 10%|█ | 169/1610 [40:58<6:06:45, 15.27s/it] 11%|█ | 170/1610 [41:14<6:12:15, 15.51s/it] {'loss': 0.0002, 'grad_norm': 3.23881003390356, 'learning_rate': 8.944099378881988e-07, 'completion_length': 141.87500762939453, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.633928656578064, 'reward_std': 0.28707224130630493, 'kl': 0.00495147705078125, 'epoch': 0.53} 11%|█ | 170/1610 [41:14<6:12:15, 15.51s/it] 11%|█ | 171/1610 [41:29<6:04:13, 15.19s/it] {'loss': 0.0002, 'grad_norm': 1.7536385265878875, 'learning_rate': 8.937888198757764e-07, 'completion_length': 150.96429443359375, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.4018840342760086, 'kl': 0.0049285888671875, 'epoch': 0.53} 11%|█ | 171/1610 [41:29<6:04:13, 15.19s/it] 11%|█ | 172/1610 [41:44<6:04:47, 15.22s/it] {'loss': 0.0002, 'grad_norm': 1.150989272536174, 'learning_rate': 8.93167701863354e-07, 'completion_length': 163.75894165039062, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5803572535514832, 'reward_std': 0.3932444155216217, 'kl': 0.00405120849609375, 'epoch': 0.53} 11%|█ | 172/1610 [41:44<6:04:47, 15.22s/it] 11%|█ | 173/1610 [41:57<5:50:02, 14.62s/it] {'loss': 0.0002, 'grad_norm': 1.6256868019843012, 'learning_rate': 8.925465838509317e-07, 'completion_length': 126.66072463989258, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.3078097999095917, 'kl': 0.0053863525390625, 'epoch': 0.54} 11%|█ | 173/1610 [41:57<5:50:02, 14.62s/it] 11%|█ | 174/1610 [42:13<5:58:03, 14.96s/it] {'loss': 0.0002, 'grad_norm': 1.9803131901017617, 'learning_rate': 8.919254658385093e-07, 'completion_length': 141.41964721679688, 'rewards/accuracy_reward': 0.5982143133878708, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.26485104858875275, 'kl': 0.00400543212890625, 'epoch': 0.54} 11%|█ | 174/1610 [42:13<5:58:03, 14.96s/it] 11%|█ | 175/1610 [42:29<6:03:21, 15.19s/it] {'loss': 0.0002, 'grad_norm': 1.1176296775926977, 'learning_rate': 8.913043478260869e-07, 'completion_length': 177.8928680419922, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.2825559601187706, 'kl': 0.004974365234375, 'epoch': 0.54} 11%|█ | 175/1610 [42:29<6:03:21, 15.19s/it] 11%|█ | 176/1610 [42:46<6:14:46, 15.68s/it] {'loss': 0.0002, 'grad_norm': 1.6548013889543811, 'learning_rate': 8.906832298136646e-07, 'completion_length': 158.1696548461914, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.633928656578064, 'reward_std': 0.36241817474365234, 'kl': 0.004913330078125, 'epoch': 0.55} 11%|█ | 176/1610 [42:46<6:14:46, 15.68s/it] 11%|█ | 177/1610 [43:02<6:16:43, 15.77s/it] {'loss': 0.0002, 'grad_norm': 5.245023961215647, 'learning_rate': 8.900621118012423e-07, 'completion_length': 186.87500762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553571939468384, 'reward_std': 0.3754122406244278, 'kl': 0.0046234130859375, 'epoch': 0.55} 11%|█ | 177/1610 [43:02<6:16:43, 15.77s/it] 11%|█ | 178/1610 [43:18<6:19:38, 15.91s/it] {'loss': 0.0002, 'grad_norm': 1.0460617261632552, 'learning_rate': 8.894409937888198e-07, 'completion_length': 182.68750762939453, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.38357120752334595, 'kl': 0.004547119140625, 'epoch': 0.55} 11%|█ | 178/1610 [43:18<6:19:38, 15.91s/it] 11%|█ | 179/1610 [43:34<6:18:18, 15.86s/it] {'loss': 0.0002, 'grad_norm': 1.5115243284212099, 'learning_rate': 8.888198757763975e-07, 'completion_length': 169.9732208251953, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.34929387271404266, 'kl': 0.0055999755859375, 'epoch': 0.56} 11%|█ | 179/1610 [43:34<6:18:18, 15.86s/it] 11%|█ | 180/1610 [43:51<6:27:32, 16.26s/it] {'loss': 0.0002, 'grad_norm': 1.169477884765951, 'learning_rate': 8.881987577639751e-07, 'completion_length': 189.14286041259766, 'rewards/accuracy_reward': 0.2678571492433548, 'rewards/format_reward': 0.973214328289032, 'reward': 1.2410714626312256, 'reward_std': 0.25934018194675446, 'kl': 0.0056915283203125, 'epoch': 0.56} 11%|█ | 180/1610 [43:51<6:27:32, 16.26s/it] 11%|█ | 181/1610 [44:06<6:18:57, 15.91s/it] {'loss': 0.0002, 'grad_norm': 6.370472151761153, 'learning_rate': 8.875776397515527e-07, 'completion_length': 163.81250762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.508928656578064, 'reward_std': 0.19668584316968918, 'kl': 0.0049285888671875, 'epoch': 0.56} 11%|█ | 181/1610 [44:06<6:18:57, 15.91s/it] 11%|█▏ | 182/1610 [44:23<6:26:01, 16.22s/it] {'loss': 0.0002, 'grad_norm': 1.861454054192371, 'learning_rate': 8.869565217391303e-07, 'completion_length': 178.00000762939453, 'rewards/accuracy_reward': 0.3750000223517418, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3571429252624512, 'reward_std': 0.3882652819156647, 'kl': 0.0052642822265625, 'epoch': 0.57} 11%|█▏ | 182/1610 [44:23<6:26:01, 16.22s/it] 11%|█▏ | 183/1610 [44:40<6:29:04, 16.36s/it] {'loss': 0.0002, 'grad_norm': 1.1487223669252657, 'learning_rate': 8.86335403726708e-07, 'completion_length': 201.25894165039062, 'rewards/accuracy_reward': 0.3750000223517418, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660714626312256, 'reward_std': 0.27083639800548553, 'kl': 0.0050048828125, 'epoch': 0.57} 11%|█▏ | 183/1610 [44:40<6:29:04, 16.36s/it] 11%|█▏ | 184/1610 [44:54<6:17:53, 15.90s/it] {'loss': 0.0002, 'grad_norm': 1.9270340724772672, 'learning_rate': 8.857142857142856e-07, 'completion_length': 160.0089340209961, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.2858598530292511, 'kl': 0.0052337646484375, 'epoch': 0.57} 11%|█▏ | 184/1610 [44:54<6:17:53, 15.90s/it] 11%|█▏ | 185/1610 [45:10<6:13:39, 15.73s/it] {'loss': 0.0002, 'grad_norm': 2.304001314367613, 'learning_rate': 8.850931677018632e-07, 'completion_length': 170.40178680419922, 'rewards/accuracy_reward': 0.3660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.30659182369709015, 'kl': 0.0048065185546875, 'epoch': 0.57} 11%|█▏ | 185/1610 [45:10<6:13:39, 15.73s/it] 12%|█▏ | 186/1610 [45:25<6:10:04, 15.59s/it] {'loss': 0.0002, 'grad_norm': 0.7298148655443246, 'learning_rate': 8.84472049689441e-07, 'completion_length': 156.8482208251953, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.16531942784786224, 'kl': 0.0045928955078125, 'epoch': 0.58} 12%|█▏ | 186/1610 [45:25<6:10:04, 15.59s/it] 12%|█▏ | 187/1610 [45:41<6:09:41, 15.59s/it] {'loss': 0.0002, 'grad_norm': 0.7437522021665134, 'learning_rate': 8.838509316770186e-07, 'completion_length': 150.31250762939453, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.535714328289032, 'reward_std': 0.23244687914848328, 'kl': 0.0048828125, 'epoch': 0.58} 12%|█▏ | 187/1610 [45:41<6:09:41, 15.59s/it] 12%|█▏ | 188/1610 [45:56<6:11:22, 15.67s/it] {'loss': 0.0002, 'grad_norm': 1.5906087435685288, 'learning_rate': 8.832298136645962e-07, 'completion_length': 185.3482208251953, 'rewards/accuracy_reward': 0.3750000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.2987862229347229, 'kl': 0.004608154296875, 'epoch': 0.58} 12%|█▏ | 188/1610 [45:56<6:11:22, 15.67s/it] 12%|█▏ | 189/1610 [46:12<6:10:26, 15.64s/it] {'loss': 0.0002, 'grad_norm': 1.9455254247094356, 'learning_rate': 8.826086956521739e-07, 'completion_length': 160.40179443359375, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.20411306619644165, 'kl': 0.0051422119140625, 'epoch': 0.59} 12%|█▏ | 189/1610 [46:12<6:10:26, 15.64s/it] 12%|█▏ | 190/1610 [46:26<6:01:50, 15.29s/it] {'loss': 0.0002, 'grad_norm': 1.4870402473578586, 'learning_rate': 8.819875776397515e-07, 'completion_length': 163.36608123779297, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.38295778632164, 'kl': 0.0050811767578125, 'epoch': 0.59} 12%|█▏ | 190/1610 [46:26<6:01:50, 15.29s/it] 12%|█▏ | 191/1610 [46:42<6:00:32, 15.25s/it] {'loss': 0.0003, 'grad_norm': 1.2444264211369145, 'learning_rate': 8.813664596273291e-07, 'completion_length': 156.67858123779297, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.21972985565662384, 'kl': 0.0069732666015625, 'epoch': 0.59} 12%|█▏ | 191/1610 [46:42<6:00:32, 15.25s/it] 12%|█▏ | 192/1610 [46:58<6:07:21, 15.54s/it] {'loss': 0.0002, 'grad_norm': 2.6211248796185664, 'learning_rate': 8.807453416149068e-07, 'completion_length': 176.0178680419922, 'rewards/accuracy_reward': 0.321428582072258, 'rewards/format_reward': 1.0, 'reward': 1.3214285969734192, 'reward_std': 0.3847780227661133, 'kl': 0.004669189453125, 'epoch': 0.6} 12%|█▏ | 192/1610 [46:58<6:07:21, 15.54s/it] 12%|█▏ | 193/1610 [47:12<5:57:19, 15.13s/it] {'loss': 0.0002, 'grad_norm': 1.5578198825301421, 'learning_rate': 8.801242236024844e-07, 'completion_length': 154.4732208251953, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.2936766445636749, 'kl': 0.00469970703125, 'epoch': 0.6} 12%|█▏ | 193/1610 [47:12<5:57:19, 15.13s/it] 12%|█▏ | 194/1610 [47:26<5:48:52, 14.78s/it] {'loss': 0.0002, 'grad_norm': 4.5286013674529935, 'learning_rate': 8.79503105590062e-07, 'completion_length': 131.3482208251953, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.20349961519241333, 'kl': 0.004852294921875, 'epoch': 0.6} 12%|█▏ | 194/1610 [47:26<5:48:52, 14.78s/it] 12%|█▏ | 195/1610 [47:43<6:02:03, 15.35s/it] {'loss': 0.0002, 'grad_norm': 0.7386731663826127, 'learning_rate': 8.788819875776398e-07, 'completion_length': 198.81250762939453, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.15872060880064964, 'kl': 0.0049285888671875, 'epoch': 0.61} 12%|█▏ | 195/1610 [47:43<6:02:03, 15.35s/it] 12%|█▏ | 196/1610 [48:00<6:14:50, 15.91s/it] {'loss': 0.0002, 'grad_norm': 1.0036314427982327, 'learning_rate': 8.782608695652174e-07, 'completion_length': 194.49108123779297, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178572535514832, 'reward_std': 0.4059847891330719, 'kl': 0.0055999755859375, 'epoch': 0.61} 12%|█▏ | 196/1610 [48:00<6:14:50, 15.91s/it] 12%|█▏ | 197/1610 [48:16<6:13:21, 15.85s/it] {'loss': 0.0002, 'grad_norm': 1.4984254465216738, 'learning_rate': 8.77639751552795e-07, 'completion_length': 150.6964340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.3141373544931412, 'kl': 0.005462646484375, 'epoch': 0.61} 12%|█▏ | 197/1610 [48:16<6:13:21, 15.85s/it] 12%|█▏ | 198/1610 [48:31<6:12:02, 15.81s/it] {'loss': 0.0002, 'grad_norm': 1.0821663319157464, 'learning_rate': 8.770186335403727e-07, 'completion_length': 179.90179443359375, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.3303675726056099, 'kl': 0.00592041015625, 'epoch': 0.61} 12%|█▏ | 198/1610 [48:31<6:12:02, 15.81s/it] 12%|█▏ | 199/1610 [48:47<6:11:44, 15.81s/it] {'loss': 0.0002, 'grad_norm': 1.8407410938029243, 'learning_rate': 8.763975155279503e-07, 'completion_length': 167.8839340209961, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.32854174077510834, 'kl': 0.0062103271484375, 'epoch': 0.62} 12%|█▏ | 199/1610 [48:47<6:11:44, 15.81s/it] 12%|█▏ | 200/1610 [49:03<6:10:21, 15.76s/it] {'loss': 0.0002, 'grad_norm': 1.6680151388201088, 'learning_rate': 8.757763975155279e-07, 'completion_length': 167.10714721679688, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.1827620565891266, 'kl': 0.0057373046875, 'epoch': 0.62} 12%|█▏ | 200/1610 [49:03<6:10:21, 15.76s/it] 12%|█▏ | 201/1610 [50:07<11:52:52, 30.36s/it] {'loss': 0.0002, 'grad_norm': 2.307847124199349, 'learning_rate': 8.751552795031055e-07, 'completion_length': 166.0357208251953, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3952852040529251, 'kl': 0.0054931640625, 'epoch': 0.62} 12%|█▏ | 201/1610 [50:07<11:52:52, 30.36s/it] 13%|█▎ | 202/1610 [50:18<9:34:12, 24.47s/it] {'loss': 0.0002, 'grad_norm': 0.8751668385316291, 'learning_rate': 8.745341614906831e-07, 'completion_length': 130.00000381469727, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.15481781959533691, 'kl': 0.005706787109375, 'epoch': 0.63} 13%|█▎ | 202/1610 [50:18<9:34:12, 24.47s/it] 13%|█▎ | 203/1610 [50:30<8:09:08, 20.86s/it] {'loss': 0.0002, 'grad_norm': 1.7703513890027966, 'learning_rate': 8.739130434782607e-07, 'completion_length': 160.74107360839844, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.32645919919013977, 'kl': 0.00567626953125, 'epoch': 0.63} 13%|█▎ | 203/1610 [50:30<8:09:08, 20.86s/it] 13%|█▎ | 204/1610 [50:43<7:10:02, 18.35s/it] {'loss': 0.0002, 'grad_norm': 1.536802595953039, 'learning_rate': 8.732919254658385e-07, 'completion_length': 172.93750762939453, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.37454766035079956, 'kl': 0.005767822265625, 'epoch': 0.63} 13%|█▎ | 204/1610 [50:43<7:10:02, 18.35s/it] 13%|█▎ | 205/1610 [50:56<6:31:20, 16.71s/it] {'loss': 0.0003, 'grad_norm': 1.2256002832391573, 'learning_rate': 8.726708074534161e-07, 'completion_length': 192.96429443359375, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3375798612833023, 'kl': 0.006744384765625, 'epoch': 0.64} 13%|█▎ | 205/1610 [50:56<6:31:20, 16.71s/it] 13%|█▎ | 206/1610 [51:07<5:56:03, 15.22s/it] {'loss': 0.0002, 'grad_norm': 1.6603340954128138, 'learning_rate': 8.720496894409937e-07, 'completion_length': 138.3482208251953, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178572535514832, 'reward_std': 0.2753526419401169, 'kl': 0.0052490234375, 'epoch': 0.64} 13%|█▎ | 206/1610 [51:07<5:56:03, 15.22s/it] 13%|█▎ | 207/1610 [51:19<5:27:34, 14.01s/it] {'loss': 0.0002, 'grad_norm': 2.2166057530078342, 'learning_rate': 8.714285714285714e-07, 'completion_length': 139.10714721679688, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.2248506247997284, 'kl': 0.005279541015625, 'epoch': 0.64} 13%|█▎ | 207/1610 [51:19<5:27:34, 14.01s/it] 13%|█▎ | 208/1610 [51:33<5:27:38, 14.02s/it] {'loss': 0.0002, 'grad_norm': 1.091008978642836, 'learning_rate': 8.70807453416149e-07, 'completion_length': 193.02679443359375, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.446428656578064, 'reward_std': 0.352583184838295, 'kl': 0.0059356689453125, 'epoch': 0.65} 13%|█▎ | 208/1610 [51:33<5:27:38, 14.02s/it] 13%|█▎ | 209/1610 [51:49<5:45:20, 14.79s/it] {'loss': 0.0002, 'grad_norm': 2.4382078925097432, 'learning_rate': 8.701863354037266e-07, 'completion_length': 195.8839340209961, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.27804866433143616, 'kl': 0.0060577392578125, 'epoch': 0.65} 13%|█▎ | 209/1610 [51:49<5:45:20, 14.79s/it] 13%|█▎ | 210/1610 [52:04<5:48:08, 14.92s/it] {'loss': 0.0002, 'grad_norm': 2.7406928656624756, 'learning_rate': 8.695652173913043e-07, 'completion_length': 147.2232208251953, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.34208158403635025, 'kl': 0.0054473876953125, 'epoch': 0.65} 13%|█▎ | 210/1610 [52:04<5:48:08, 14.92s/it] 13%|█▎ | 211/1610 [52:21<5:56:18, 15.28s/it] {'loss': 0.0002, 'grad_norm': 1.4667284876839652, 'learning_rate': 8.689440993788819e-07, 'completion_length': 155.81250762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.446428656578064, 'reward_std': 0.31559839844703674, 'kl': 0.00506591796875, 'epoch': 0.66} 13%|█▎ | 211/1610 [52:21<5:56:18, 15.28s/it] 13%|█▎ | 212/1610 [52:36<5:54:50, 15.23s/it] {'loss': 0.0003, 'grad_norm': 1.3179367029705351, 'learning_rate': 8.683229813664595e-07, 'completion_length': 151.5982208251953, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.2540072202682495, 'kl': 0.0078887939453125, 'epoch': 0.66} 13%|█▎ | 212/1610 [52:36<5:54:50, 15.23s/it] 13%|█▎ | 213/1610 [52:51<5:57:07, 15.34s/it] {'loss': 0.0003, 'grad_norm': 1.0959387476738236, 'learning_rate': 8.677018633540373e-07, 'completion_length': 153.76786041259766, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.2287590280175209, 'kl': 0.0064544677734375, 'epoch': 0.66} 13%|█▎ | 213/1610 [52:51<5:57:07, 15.34s/it] 13%|█▎ | 214/1610 [53:07<6:01:22, 15.53s/it] {'loss': 0.0002, 'grad_norm': 1.3315135980400041, 'learning_rate': 8.670807453416149e-07, 'completion_length': 158.33929443359375, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.2708419933915138, 'kl': 0.0057220458984375, 'epoch': 0.66} 13%|█▎ | 214/1610 [53:07<6:01:22, 15.53s/it] 13%|█▎ | 215/1610 [53:23<6:00:05, 15.49s/it] {'loss': 0.0002, 'grad_norm': 1.4814253753450908, 'learning_rate': 8.664596273291925e-07, 'completion_length': 177.58929443359375, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3279428035020828, 'kl': 0.00592041015625, 'epoch': 0.67} 13%|█▎ | 215/1610 [53:23<6:00:05, 15.49s/it] 13%|█▎ | 216/1610 [53:37<5:51:02, 15.11s/it] {'loss': 0.0003, 'grad_norm': 3.8438644466391567, 'learning_rate': 8.658385093167702e-07, 'completion_length': 140.4464340209961, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.2857143878936768, 'reward_std': 0.29757384955883026, 'kl': 0.00689697265625, 'epoch': 0.67} 13%|█▎ | 216/1610 [53:37<5:51:02, 15.11s/it] 13%|█▎ | 217/1610 [53:50<5:39:12, 14.61s/it] {'loss': 0.0002, 'grad_norm': 1.4447039004355238, 'learning_rate': 8.652173913043478e-07, 'completion_length': 133.19643783569336, 'rewards/accuracy_reward': 0.3928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.36733539402484894, 'kl': 0.0042572021484375, 'epoch': 0.67} 13%|█▎ | 217/1610 [53:50<5:39:12, 14.61s/it] 14%|█▎ | 218/1610 [54:06<5:43:21, 14.80s/it] {'loss': 0.0003, 'grad_norm': 0.9348469712506865, 'learning_rate': 8.645962732919254e-07, 'completion_length': 179.61608123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.2663346976041794, 'kl': 0.0063934326171875, 'epoch': 0.68} 14%|█▎ | 218/1610 [54:06<5:43:21, 14.80s/it] 14%|█▎ | 219/1610 [54:20<5:41:38, 14.74s/it] {'loss': 0.0002, 'grad_norm': 1.2122889052943022, 'learning_rate': 8.639751552795031e-07, 'completion_length': 164.35714721679688, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.19690079241991043, 'kl': 0.0050201416015625, 'epoch': 0.68} 14%|█▎ | 219/1610 [54:20<5:41:38, 14.74s/it] 14%|█▎ | 220/1610 [54:37<5:53:14, 15.25s/it] {'loss': 0.0002, 'grad_norm': 1.4366549080909254, 'learning_rate': 8.633540372670807e-07, 'completion_length': 176.2589340209961, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.4148048013448715, 'kl': 0.0059356689453125, 'epoch': 0.68} 14%|█▎ | 220/1610 [54:37<5:53:14, 15.25s/it] 14%|█▎ | 221/1610 [54:53<5:59:37, 15.53s/it] {'loss': 0.0002, 'grad_norm': 1.1692523004260302, 'learning_rate': 8.627329192546583e-07, 'completion_length': 172.2857208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.3800467848777771, 'kl': 0.0046539306640625, 'epoch': 0.69} 14%|█▎ | 221/1610 [54:53<5:59:37, 15.53s/it] 14%|█▍ | 222/1610 [55:09<6:01:15, 15.62s/it] {'loss': 0.0002, 'grad_norm': 1.8101099138163774, 'learning_rate': 8.621118012422361e-07, 'completion_length': 153.89286041259766, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.40942396223545074, 'kl': 0.0056915283203125, 'epoch': 0.69} 14%|█▍ | 222/1610 [55:09<6:01:15, 15.62s/it] 14%|█▍ | 223/1610 [55:26<6:10:42, 16.04s/it] {'loss': 0.0002, 'grad_norm': 2.249325135662979, 'learning_rate': 8.614906832298137e-07, 'completion_length': 151.93750762939453, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.3077307790517807, 'kl': 0.005340576171875, 'epoch': 0.69} 14%|█▍ | 223/1610 [55:26<6:10:42, 16.04s/it][2025-02-23 14:17:41,327] [WARNING] [stage3.py:2134:step] 1 pytorch allocator cache flushes since last step. this happens when there is high memory pressure and is detrimental to performance. if this is happening frequently consider adjusting settings to reduce memory consumption. If you are unable to make the cache flushes go away consider adding get_accelerator().empty_cache() calls in your training loop to ensure that all ranks flush their caches at the same time 14%|█▍ | 224/1610 [55:42<6:09:51, 16.01s/it] {'loss': 0.0002, 'grad_norm': 1.8083314257310692, 'learning_rate': 8.608695652173913e-07, 'completion_length': 159.02679443359375, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.2819514572620392, 'kl': 0.006134033203125, 'epoch': 0.7} 14%|█▍ | 224/1610 [55:42<6:09:51, 16.01s/it] 14%|█▍ | 225/1610 [55:56<5:58:37, 15.54s/it] {'loss': 0.0002, 'grad_norm': 1.3447101280451912, 'learning_rate': 8.60248447204969e-07, 'completion_length': 146.8928680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5982143878936768, 'reward_std': 0.3014766424894333, 'kl': 0.0041351318359375, 'epoch': 0.7} 14%|█▍ | 225/1610 [55:56<5:58:37, 15.54s/it] 14%|█▍ | 226/1610 [56:12<6:02:26, 15.71s/it] {'loss': 0.0002, 'grad_norm': 1.1457136310454443, 'learning_rate': 8.596273291925465e-07, 'completion_length': 191.75894165039062, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.3685477375984192, 'kl': 0.00616455078125, 'epoch': 0.7} 14%|█▍ | 226/1610 [56:12<6:02:26, 15.71s/it] 14%|█▍ | 227/1610 [56:29<6:11:06, 16.10s/it] {'loss': 0.0002, 'grad_norm': 0.9641996645559258, 'learning_rate': 8.590062111801241e-07, 'completion_length': 215.08929443359375, 'rewards/accuracy_reward': 0.3035714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3035715222358704, 'reward_std': 0.24046741425991058, 'kl': 0.0062103271484375, 'epoch': 0.7} 14%|█▍ | 227/1610 [56:29<6:11:06, 16.10s/it] 14%|█▍ | 228/1610 [56:43<5:58:01, 15.54s/it] {'loss': 0.0002, 'grad_norm': 1.5558154849470722, 'learning_rate': 8.583850931677018e-07, 'completion_length': 135.42858123779297, 'rewards/accuracy_reward': 0.4464286118745804, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.29999858140945435, 'kl': 0.0051422119140625, 'epoch': 0.71} 14%|█▍ | 228/1610 [56:43<5:58:01, 15.54s/it] 14%|█▍ | 229/1610 [56:58<5:51:55, 15.29s/it] {'loss': 0.0002, 'grad_norm': 1.915878382541873, 'learning_rate': 8.577639751552794e-07, 'completion_length': 142.02679443359375, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.4016127735376358, 'kl': 0.0051422119140625, 'epoch': 0.71} 14%|█▍ | 229/1610 [56:58<5:51:55, 15.29s/it] 14%|█▍ | 230/1610 [57:14<5:57:56, 15.56s/it] {'loss': 0.0003, 'grad_norm': 2.5287108311605757, 'learning_rate': 8.57142857142857e-07, 'completion_length': 154.06250762939453, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.410714328289032, 'reward_std': 0.3870585113763809, 'kl': 0.0064239501953125, 'epoch': 0.71} 14%|█▍ | 230/1610 [57:14<5:57:56, 15.56s/it] 14%|█▍ | 231/1610 [57:30<6:00:10, 15.67s/it] {'loss': 0.0002, 'grad_norm': 2.0245283805818253, 'learning_rate': 8.565217391304348e-07, 'completion_length': 177.67858123779297, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.3839285969734192, 'reward_std': 0.36100783944129944, 'kl': 0.0050506591796875, 'epoch': 0.72} 14%|█▍ | 231/1610 [57:30<6:00:10, 15.67s/it] 14%|█▍ | 232/1610 [57:46<6:02:18, 15.78s/it] {'loss': 0.0002, 'grad_norm': 1.871910971101897, 'learning_rate': 8.559006211180124e-07, 'completion_length': 149.37500762939453, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.535714328289032, 'reward_std': 0.33636748790740967, 'kl': 0.004913330078125, 'epoch': 0.72} 14%|█▍ | 232/1610 [57:46<6:02:18, 15.78s/it] 14%|█▍ | 233/1610 [58:01<5:54:43, 15.46s/it] {'loss': 0.0002, 'grad_norm': 1.0760916662148863, 'learning_rate': 8.5527950310559e-07, 'completion_length': 138.94643783569336, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4464285969734192, 'reward_std': 0.24289214611053467, 'kl': 0.005279541015625, 'epoch': 0.72} 14%|█▍ | 233/1610 [58:01<5:54:43, 15.46s/it] 15%|█▍ | 234/1610 [58:16<5:48:36, 15.20s/it] {'loss': 0.0003, 'grad_norm': 2.1506968712092864, 'learning_rate': 8.546583850931677e-07, 'completion_length': 157.40179443359375, 'rewards/accuracy_reward': 0.3928571492433548, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3375742584466934, 'kl': 0.007537841796875, 'epoch': 0.73} 15%|█▍ | 234/1610 [58:16<5:48:36, 15.20s/it] 15%|█▍ | 235/1610 [58:30<5:45:53, 15.09s/it] {'loss': 0.0003, 'grad_norm': 1.5797852856734511, 'learning_rate': 8.540372670807453e-07, 'completion_length': 145.60715103149414, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.2987862229347229, 'kl': 0.006256103515625, 'epoch': 0.73} 15%|█▍ | 235/1610 [58:30<5:45:53, 15.09s/it] 15%|█▍ | 236/1610 [58:47<5:58:01, 15.63s/it] {'loss': 0.0003, 'grad_norm': 1.5039247776296405, 'learning_rate': 8.534161490683229e-07, 'completion_length': 168.45536041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910714626312256, 'reward_std': 0.4012260437011719, 'kl': 0.006378173828125, 'epoch': 0.73} 15%|█▍ | 236/1610 [58:47<5:58:01, 15.63s/it] 15%|█▍ | 237/1610 [59:03<5:59:49, 15.72s/it] {'loss': 0.0003, 'grad_norm': 1.2745506726881766, 'learning_rate': 8.527950310559006e-07, 'completion_length': 165.42857360839844, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3214285969734192, 'reward_std': 0.36156781017780304, 'kl': 0.0067901611328125, 'epoch': 0.74} 15%|█▍ | 237/1610 [59:03<5:59:49, 15.72s/it] 15%|█▍ | 238/1610 [59:20<6:04:10, 15.93s/it] {'loss': 0.0002, 'grad_norm': 2.6764002682301635, 'learning_rate': 8.521739130434782e-07, 'completion_length': 162.1607208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.508928656578064, 'reward_std': 0.22028984129428864, 'kl': 0.00604248046875, 'epoch': 0.74} 15%|█▍ | 238/1610 [59:20<6:04:10, 15.93s/it] 15%|█▍ | 239/1610 [59:35<6:00:16, 15.77s/it] {'loss': 0.0003, 'grad_norm': 2.3999709951755746, 'learning_rate': 8.515527950310558e-07, 'completion_length': 186.17858123779297, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.31291936337947845, 'kl': 0.0069122314453125, 'epoch': 0.74} 15%|█▍ | 239/1610 [59:35<6:00:16, 15.77s/it] 15%|█▍ | 240/1610 [59:48<5:41:41, 14.96s/it] {'loss': 0.0003, 'grad_norm': 1.6314268663360085, 'learning_rate': 8.509316770186336e-07, 'completion_length': 128.99108123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2948834300041199, 'kl': 0.006805419921875, 'epoch': 0.75} 15%|█▍ | 240/1610 [59:48<5:41:41, 14.96s/it] 15%|█▍ | 241/1610 [1:00:04<5:45:57, 15.16s/it] {'loss': 0.0003, 'grad_norm': 1.622291266387324, 'learning_rate': 8.503105590062112e-07, 'completion_length': 158.05357360839844, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3466034382581711, 'kl': 0.0067291259765625, 'epoch': 0.75} 15%|█▍ | 241/1610 [1:00:04<5:45:57, 15.16s/it] 15%|█▌ | 242/1610 [1:00:18<5:39:45, 14.90s/it] {'loss': 0.0003, 'grad_norm': 1.5241983347200114, 'learning_rate': 8.496894409937888e-07, 'completion_length': 135.23215103149414, 'rewards/accuracy_reward': 0.3303571492433548, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.2987862080335617, 'kl': 0.0067138671875, 'epoch': 0.75} 15%|█▌ | 242/1610 [1:00:18<5:39:45, 14.90s/it] 15%|█▌ | 243/1610 [1:00:33<5:40:54, 14.96s/it] {'loss': 0.0003, 'grad_norm': 1.9926066538308334, 'learning_rate': 8.490683229813665e-07, 'completion_length': 154.7053680419922, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.2987862080335617, 'kl': 0.007781982421875, 'epoch': 0.75} 15%|█▌ | 243/1610 [1:00:33<5:40:54, 14.96s/it] 15%|█▌ | 244/1610 [1:00:49<5:46:59, 15.24s/it] {'loss': 0.0003, 'grad_norm': 2.470035489967227, 'learning_rate': 8.484472049689441e-07, 'completion_length': 142.37500762939453, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.29999858140945435, 'kl': 0.006561279296875, 'epoch': 0.76} 15%|█▌ | 244/1610 [1:00:49<5:46:59, 15.24s/it] 15%|█▌ | 245/1610 [1:01:04<5:46:48, 15.24s/it] {'loss': 0.0003, 'grad_norm': 1.9599613090259331, 'learning_rate': 8.478260869565217e-07, 'completion_length': 154.04464721679688, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.30303123593330383, 'kl': 0.0065765380859375, 'epoch': 0.76} 15%|█▌ | 245/1610 [1:01:04<5:46:48, 15.24s/it] 15%|█▌ | 246/1610 [1:01:19<5:41:23, 15.02s/it] {'loss': 0.0002, 'grad_norm': 1.1746067877788222, 'learning_rate': 8.472049689440994e-07, 'completion_length': 135.16965103149414, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.27804866433143616, 'kl': 0.0045623779296875, 'epoch': 0.76} 15%|█▌ | 246/1610 [1:01:19<5:41:23, 15.02s/it] 15%|█▌ | 247/1610 [1:01:34<5:43:30, 15.12s/it] {'loss': 0.0002, 'grad_norm': 2.2859362357892876, 'learning_rate': 8.46583850931677e-07, 'completion_length': 153.6607208251953, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.21973545849323273, 'kl': 0.0061492919921875, 'epoch': 0.77} 15%|█▌ | 247/1610 [1:01:34<5:43:30, 15.12s/it] 15%|█▌ | 248/1610 [1:01:50<5:48:43, 15.36s/it] {'loss': 0.0003, 'grad_norm': 1.2399783119840204, 'learning_rate': 8.459627329192546e-07, 'completion_length': 160.55358123779297, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.28047342598438263, 'kl': 0.0076446533203125, 'epoch': 0.77} 15%|█▌ | 248/1610 [1:01:50<5:48:43, 15.36s/it] 15%|█▌ | 249/1610 [1:02:06<5:53:48, 15.60s/it] {'loss': 0.0003, 'grad_norm': 1.280469281055088, 'learning_rate': 8.453416149068324e-07, 'completion_length': 179.5357208251953, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.38026174902915955, 'kl': 0.0072784423828125, 'epoch': 0.77} 15%|█▌ | 249/1610 [1:02:06<5:53:48, 15.60s/it] 16%|█▌ | 250/1610 [1:02:20<5:41:20, 15.06s/it] {'loss': 0.0002, 'grad_norm': 0.9226283039757466, 'learning_rate': 8.447204968944099e-07, 'completion_length': 129.33036041259766, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2314438372850418, 'kl': 0.005096435546875, 'epoch': 0.78} 16%|█▌ | 250/1610 [1:02:20<5:41:20, 15.06s/it] 16%|█▌ | 251/1610 [1:02:35<5:42:02, 15.10s/it] {'loss': 0.0002, 'grad_norm': 1.3459132115263819, 'learning_rate': 8.440993788819875e-07, 'completion_length': 136.35714721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.34290722012519836, 'kl': 0.005340576171875, 'epoch': 0.78} 16%|█▌ | 251/1610 [1:02:35<5:42:02, 15.10s/it] 16%|█▌ | 252/1610 [1:02:50<5:42:14, 15.12s/it] {'loss': 0.0003, 'grad_norm': 1.9441011496625742, 'learning_rate': 8.434782608695652e-07, 'completion_length': 146.6964340209961, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910715222358704, 'reward_std': 0.43635378777980804, 'kl': 0.0072479248046875, 'epoch': 0.78} 16%|█▌ | 252/1610 [1:02:50<5:42:14, 15.12s/it] 16%|█▌ | 253/1610 [1:03:06<5:47:59, 15.39s/it] {'loss': 0.0002, 'grad_norm': 0.8787324866095247, 'learning_rate': 8.428571428571428e-07, 'completion_length': 139.1428680419922, 'rewards/accuracy_reward': 0.6517857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6428571939468384, 'reward_std': 0.21621103584766388, 'kl': 0.0047149658203125, 'epoch': 0.79} 16%|█▌ | 253/1610 [1:03:06<5:47:59, 15.39s/it] 16%|█▌ | 254/1610 [1:03:22<5:46:00, 15.31s/it] {'loss': 0.0002, 'grad_norm': 0.9415510568602923, 'learning_rate': 8.422360248447204e-07, 'completion_length': 137.29465103149414, 'rewards/accuracy_reward': 0.625, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.23386859893798828, 'kl': 0.0056610107421875, 'epoch': 0.79} 16%|█▌ | 254/1610 [1:03:22<5:46:00, 15.31s/it] 16%|█▌ | 255/1610 [1:03:36<5:42:08, 15.15s/it] {'loss': 0.0002, 'grad_norm': 1.671942436568068, 'learning_rate': 8.416149068322981e-07, 'completion_length': 146.08929443359375, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.375760018825531, 'kl': 0.006195068359375, 'epoch': 0.79} 16%|█▌ | 255/1610 [1:03:36<5:42:08, 15.15s/it] 16%|█▌ | 256/1610 [1:03:52<5:44:49, 15.28s/it] {'loss': 0.0003, 'grad_norm': 4.317788581950143, 'learning_rate': 8.409937888198757e-07, 'completion_length': 154.75894165039062, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3399990051984787, 'kl': 0.00665283203125, 'epoch': 0.8} 16%|█▌ | 256/1610 [1:03:52<5:44:49, 15.28s/it] 16%|█▌ | 257/1610 [1:04:08<5:49:55, 15.52s/it] {'loss': 0.0003, 'grad_norm': 2.328695866994175, 'learning_rate': 8.403726708074533e-07, 'completion_length': 156.75000762939453, 'rewards/accuracy_reward': 0.321428582072258, 'rewards/format_reward': 1.0, 'reward': 1.321428656578064, 'reward_std': 0.267547070980072, 'kl': 0.0064697265625, 'epoch': 0.8} 16%|█▌ | 257/1610 [1:04:08<5:49:55, 15.52s/it] 16%|█▌ | 258/1610 [1:04:23<5:49:13, 15.50s/it] {'loss': 0.0003, 'grad_norm': 1.6856340425199492, 'learning_rate': 8.397515527950311e-07, 'completion_length': 141.56250762939453, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.3445626348257065, 'kl': 0.0069122314453125, 'epoch': 0.8} 16%|█▌ | 258/1610 [1:04:23<5:49:13, 15.50s/it] 16%|█▌ | 259/1610 [1:04:38<5:43:48, 15.27s/it] {'loss': 0.0003, 'grad_norm': 2.6659583372579645, 'learning_rate': 8.391304347826087e-07, 'completion_length': 165.61608123779297, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.15150833874940872, 'kl': 0.0082244873046875, 'epoch': 0.8} 16%|█▌ | 259/1610 [1:04:38<5:43:48, 15.27s/it] 16%|█▌ | 260/1610 [1:04:56<5:59:16, 15.97s/it] {'loss': 0.0003, 'grad_norm': 1.559801750455078, 'learning_rate': 8.385093167701863e-07, 'completion_length': 207.7232208251953, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.410714328289032, 'reward_std': 0.3402758836746216, 'kl': 0.0070953369140625, 'epoch': 0.81} 16%|█▌ | 260/1610 [1:04:56<5:59:16, 15.97s/it] 16%|█▌ | 261/1610 [1:05:10<5:48:37, 15.51s/it] {'loss': 0.0003, 'grad_norm': 1.4283262495675837, 'learning_rate': 8.37888198757764e-07, 'completion_length': 159.5357208251953, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.25521960854530334, 'kl': 0.0064849853515625, 'epoch': 0.81} 16%|█▌ | 261/1610 [1:05:10<5:48:37, 15.51s/it] 16%|█▋ | 262/1610 [1:05:24<5:33:55, 14.86s/it] {'loss': 0.0003, 'grad_norm': 1.059966779275808, 'learning_rate': 8.372670807453416e-07, 'completion_length': 141.0089340209961, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.21703943610191345, 'kl': 0.0067138671875, 'epoch': 0.81} 16%|█▋ | 262/1610 [1:05:24<5:33:55, 14.86s/it] 16%|█▋ | 263/1610 [1:05:35<5:14:03, 13.99s/it] {'loss': 0.0003, 'grad_norm': 2.1828616616157652, 'learning_rate': 8.366459627329192e-07, 'completion_length': 113.67857360839844, 'rewards/accuracy_reward': 0.7053571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7053572535514832, 'reward_std': 0.3733352720737457, 'kl': 0.0073699951171875, 'epoch': 0.82} 16%|█▋ | 263/1610 [1:05:35<5:14:03, 13.99s/it] 16%|█▋ | 264/1610 [1:05:52<5:27:59, 14.62s/it] {'loss': 0.0003, 'grad_norm': 1.9256651463593166, 'learning_rate': 8.360248447204969e-07, 'completion_length': 159.36608123779297, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5982143878936768, 'reward_std': 0.38081611692905426, 'kl': 0.00860595703125, 'epoch': 0.82} 16%|█▋ | 264/1610 [1:05:52<5:27:59, 14.62s/it] 16%|█▋ | 265/1610 [1:06:06<5:27:21, 14.60s/it] {'loss': 0.0003, 'grad_norm': 1.5716123329592975, 'learning_rate': 8.354037267080745e-07, 'completion_length': 134.79465103149414, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857313156128, 'reward_std': 0.2987862229347229, 'kl': 0.006439208984375, 'epoch': 0.82} 16%|█▋ | 265/1610 [1:06:06<5:27:21, 14.60s/it] 17%|█▋ | 266/1610 [1:06:22<5:37:44, 15.08s/it] {'loss': 0.0003, 'grad_norm': 1.1527267426692676, 'learning_rate': 8.347826086956521e-07, 'completion_length': 175.78572845458984, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.32007813453674316, 'kl': 0.0067291259765625, 'epoch': 0.83} 17%|█▋ | 266/1610 [1:06:22<5:37:44, 15.08s/it] 17%|█▋ | 267/1610 [1:06:37<5:35:10, 14.97s/it] {'loss': 0.0003, 'grad_norm': 2.3526279331042663, 'learning_rate': 8.341614906832299e-07, 'completion_length': 160.5357208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3246535062789917, 'kl': 0.0072479248046875, 'epoch': 0.83} 17%|█▋ | 267/1610 [1:06:37<5:35:10, 14.97s/it] 17%|█▋ | 268/1610 [1:06:51<5:30:28, 14.78s/it] {'loss': 0.0002, 'grad_norm': 1.2804873065567073, 'learning_rate': 8.335403726708075e-07, 'completion_length': 126.45536804199219, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.21582704782485962, 'kl': 0.005340576171875, 'epoch': 0.83} 17%|█▋ | 268/1610 [1:06:51<5:30:28, 14.78s/it] 17%|█▋ | 269/1610 [1:07:07<5:37:50, 15.12s/it] {'loss': 0.0003, 'grad_norm': 2.7412555333540003, 'learning_rate': 8.329192546583851e-07, 'completion_length': 147.6339340209961, 'rewards/accuracy_reward': 0.4464286118745804, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.2767827957868576, 'kl': 0.0070648193359375, 'epoch': 0.84} 17%|█▋ | 269/1610 [1:07:07<5:37:50, 15.12s/it] 17%|█▋ | 270/1610 [1:07:23<5:42:35, 15.34s/it] {'loss': 0.0002, 'grad_norm': 1.2798251760034736, 'learning_rate': 8.322981366459628e-07, 'completion_length': 136.48214721679688, 'rewards/accuracy_reward': 0.6517857611179352, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.21582704782485962, 'kl': 0.006134033203125, 'epoch': 0.84} 17%|█▋ | 270/1610 [1:07:23<5:42:35, 15.34s/it] 17%|█▋ | 271/1610 [1:07:38<5:38:37, 15.17s/it] {'loss': 0.0002, 'grad_norm': 1.672751951987383, 'learning_rate': 8.316770186335404e-07, 'completion_length': 152.6696548461914, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3850492835044861, 'kl': 0.0058441162109375, 'epoch': 0.84} 17%|█▋ | 271/1610 [1:07:38<5:38:37, 15.17s/it] 17%|█▋ | 272/1610 [1:07:51<5:26:25, 14.64s/it] {'loss': 0.0003, 'grad_norm': 2.53028788496402, 'learning_rate': 8.31055900621118e-07, 'completion_length': 134.5982208251953, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.3318512439727783, 'kl': 0.0064697265625, 'epoch': 0.84} 17%|█▋ | 272/1610 [1:07:51<5:26:25, 14.64s/it] 17%|█▋ | 273/1610 [1:08:06<5:27:30, 14.70s/it] {'loss': 0.0003, 'grad_norm': 2.1704707917516743, 'learning_rate': 8.304347826086955e-07, 'completion_length': 140.7232208251953, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.32013723254203796, 'kl': 0.00738525390625, 'epoch': 0.85} 17%|█▋ | 273/1610 [1:08:06<5:27:30, 14.70s/it] 17%|█▋ | 274/1610 [1:08:18<5:11:39, 14.00s/it] {'loss': 0.0002, 'grad_norm': 0.9238304732387295, 'learning_rate': 8.298136645962732e-07, 'completion_length': 120.06250381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.669642984867096, 'reward_std': 0.19959121197462082, 'kl': 0.0052337646484375, 'epoch': 0.85} 17%|█▋ | 274/1610 [1:08:18<5:11:39, 14.00s/it] 17%|█▋ | 275/1610 [1:08:33<5:17:33, 14.27s/it] {'loss': 0.0002, 'grad_norm': 1.2686981117202536, 'learning_rate': 8.291925465838508e-07, 'completion_length': 131.50000762939453, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2726622074842453, 'kl': 0.0059356689453125, 'epoch': 0.85} 17%|█▋ | 275/1610 [1:08:33<5:17:33, 14.27s/it] 17%|█▋ | 276/1610 [1:08:48<5:17:20, 14.27s/it] {'loss': 0.0003, 'grad_norm': 0.9636514128030612, 'learning_rate': 8.285714285714285e-07, 'completion_length': 132.50894165039062, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.28828461468219757, 'kl': 0.006256103515625, 'epoch': 0.86} 17%|█▋ | 276/1610 [1:08:48<5:17:20, 14.27s/it] 17%|█▋ | 277/1610 [1:09:02<5:20:17, 14.42s/it] {'loss': 0.0003, 'grad_norm': 1.455458961003164, 'learning_rate': 8.279503105590062e-07, 'completion_length': 137.30357360839844, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.19447603821754456, 'kl': 0.006378173828125, 'epoch': 0.86} 17%|█▋ | 277/1610 [1:09:02<5:20:17, 14.42s/it] 17%|█▋ | 278/1610 [1:09:19<5:33:34, 15.03s/it] {'loss': 0.0003, 'grad_norm': 1.163377634806462, 'learning_rate': 8.273291925465838e-07, 'completion_length': 200.34822845458984, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3750000596046448, 'reward_std': 0.339447483420372, 'kl': 0.007843017578125, 'epoch': 0.86} 17%|█▋ | 278/1610 [1:09:19<5:33:34, 15.03s/it] 17%|█▋ | 279/1610 [1:09:35<5:41:17, 15.39s/it] {'loss': 0.0002, 'grad_norm': 1.4088959384853064, 'learning_rate': 8.267080745341614e-07, 'completion_length': 164.98214721679688, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.3441786915063858, 'kl': 0.006134033203125, 'epoch': 0.87} 17%|█▋ | 279/1610 [1:09:35<5:41:17, 15.39s/it] 17%|█▋ | 280/1610 [1:09:50<5:39:58, 15.34s/it] {'loss': 0.0002, 'grad_norm': 1.0029853464368075, 'learning_rate': 8.260869565217391e-07, 'completion_length': 160.3571548461914, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4107143878936768, 'reward_std': 0.27805428206920624, 'kl': 0.0054931640625, 'epoch': 0.87} 17%|█▋ | 280/1610 [1:09:50<5:39:58, 15.34s/it] 17%|█▋ | 281/1610 [1:10:06<5:41:02, 15.40s/it] {'loss': 0.0003, 'grad_norm': 0.7293697236695029, 'learning_rate': 8.254658385093167e-07, 'completion_length': 169.83929443359375, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.10882645100355148, 'kl': 0.0070648193359375, 'epoch': 0.87} 17%|█▋ | 281/1610 [1:10:06<5:41:02, 15.40s/it] 18%|█▊ | 282/1610 [1:10:22<5:48:22, 15.74s/it] {'loss': 0.0003, 'grad_norm': 0.7582455006108744, 'learning_rate': 8.248447204968943e-07, 'completion_length': 165.2053680419922, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.2546061500906944, 'kl': 0.0064544677734375, 'epoch': 0.88} 18%|█▊ | 282/1610 [1:10:22<5:48:22, 15.74s/it] 18%|█▊ | 283/1610 [1:10:36<5:37:04, 15.24s/it] {'loss': 0.0002, 'grad_norm': 1.828641640701901, 'learning_rate': 8.24223602484472e-07, 'completion_length': 133.7857208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.26423755288124084, 'kl': 0.005889892578125, 'epoch': 0.88} 18%|█▊ | 283/1610 [1:10:36<5:37:04, 15.24s/it] 18%|█▊ | 284/1610 [1:10:51<5:34:24, 15.13s/it] {'loss': 0.0003, 'grad_norm': 2.285308480976128, 'learning_rate': 8.236024844720496e-07, 'completion_length': 140.25000762939453, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3796684443950653, 'kl': 0.0073699951171875, 'epoch': 0.88} 18%|█▊ | 284/1610 [1:10:51<5:34:24, 15.13s/it] 18%|█▊ | 285/1610 [1:11:06<5:31:29, 15.01s/it] {'loss': 0.0003, 'grad_norm': 1.218943706392546, 'learning_rate': 8.229813664596273e-07, 'completion_length': 151.35714721679688, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.31986598670482635, 'kl': 0.0068206787109375, 'epoch': 0.89} 18%|█▊ | 285/1610 [1:11:06<5:31:29, 15.01s/it] 18%|█▊ | 286/1610 [1:11:22<5:36:11, 15.24s/it] {'loss': 0.0004, 'grad_norm': 1.4529127425332058, 'learning_rate': 8.22360248447205e-07, 'completion_length': 173.06250762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5625000596046448, 'reward_std': 0.36002394556999207, 'kl': 0.0099945068359375, 'epoch': 0.89} 18%|█▊ | 286/1610 [1:11:22<5:36:11, 15.24s/it] 18%|█▊ | 287/1610 [1:11:37<5:33:49, 15.14s/it] {'loss': 0.0003, 'grad_norm': 2.865974901325878, 'learning_rate': 8.217391304347826e-07, 'completion_length': 156.8839340209961, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.301482230424881, 'kl': 0.008697509765625, 'epoch': 0.89} 18%|█▊ | 287/1610 [1:11:37<5:33:49, 15.14s/it] 18%|█▊ | 288/1610 [1:11:53<5:41:41, 15.51s/it] {'loss': 0.0003, 'grad_norm': 2.879810910369227, 'learning_rate': 8.211180124223602e-07, 'completion_length': 184.37500762939453, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.3792473375797272, 'kl': 0.00823974609375, 'epoch': 0.89} 18%|█▊ | 288/1610 [1:11:53<5:41:41, 15.51s/it] 18%|█▊ | 289/1610 [1:12:08<5:37:24, 15.32s/it] {'loss': 0.0002, 'grad_norm': 1.403107757512005, 'learning_rate': 8.204968944099379e-07, 'completion_length': 163.2857208251953, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.33636748790740967, 'kl': 0.006103515625, 'epoch': 0.9} 18%|█▊ | 289/1610 [1:12:08<5:37:24, 15.32s/it] 18%|█▊ | 290/1610 [1:12:24<5:39:03, 15.41s/it] {'loss': 0.0002, 'grad_norm': 1.6301848183843008, 'learning_rate': 8.198757763975155e-07, 'completion_length': 173.07144165039062, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.348214328289032, 'reward_std': 0.30247966945171356, 'kl': 0.005279541015625, 'epoch': 0.9} 18%|█▊ | 290/1610 [1:12:24<5:39:03, 15.41s/it] 18%|█▊ | 291/1610 [1:12:36<5:21:53, 14.64s/it] {'loss': 0.0002, 'grad_norm': 1.1726275528102927, 'learning_rate': 8.192546583850931e-07, 'completion_length': 140.7232208251953, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.24889206886291504, 'kl': 0.00604248046875, 'epoch': 0.9} 18%|█▊ | 291/1610 [1:12:36<5:21:53, 14.64s/it] 18%|█▊ | 292/1610 [1:12:52<5:24:45, 14.78s/it] {'loss': 0.0003, 'grad_norm': 1.2067086387201011, 'learning_rate': 8.186335403726708e-07, 'completion_length': 159.16964721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3564915657043457, 'kl': 0.0075531005859375, 'epoch': 0.91} 18%|█▊ | 292/1610 [1:12:52<5:24:45, 14.78s/it] 18%|█▊ | 293/1610 [1:13:07<5:26:10, 14.86s/it] {'loss': 0.0003, 'grad_norm': 1.5835445332090279, 'learning_rate': 8.180124223602484e-07, 'completion_length': 151.7232208251953, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.31561537086963654, 'kl': 0.0063934326171875, 'epoch': 0.91} 18%|█▊ | 293/1610 [1:13:07<5:26:10, 14.86s/it] 18%|█▊ | 294/1610 [1:13:20<5:14:57, 14.36s/it] {'loss': 0.0002, 'grad_norm': 1.436267049119522, 'learning_rate': 8.173913043478261e-07, 'completion_length': 134.90179443359375, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.29730257391929626, 'kl': 0.004669189453125, 'epoch': 0.91} 18%|█▊ | 294/1610 [1:13:20<5:14:57, 14.36s/it] 18%|█▊ | 295/1610 [1:13:35<5:21:27, 14.67s/it] {'loss': 0.0003, 'grad_norm': 8.172845509654042, 'learning_rate': 8.167701863354038e-07, 'completion_length': 168.73214721679688, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.22094222903251648, 'kl': 0.00714111328125, 'epoch': 0.92} 18%|█▊ | 295/1610 [1:13:35<5:21:27, 14.67s/it] 18%|█▊ | 296/1610 [1:13:52<5:36:52, 15.38s/it] {'loss': 0.0003, 'grad_norm': 1.3764055471091226, 'learning_rate': 8.161490683229814e-07, 'completion_length': 165.92857360839844, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.973214328289032, 'reward': 1.3660714626312256, 'reward_std': 0.33106425404548645, 'kl': 0.007354736328125, 'epoch': 0.92} 18%|█▊ | 296/1610 [1:13:52<5:36:52, 15.38s/it] 18%|█▊ | 297/1610 [1:14:08<5:36:55, 15.40s/it] {'loss': 0.0003, 'grad_norm': 1.1399939007935496, 'learning_rate': 8.155279503105589e-07, 'completion_length': 164.96429443359375, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.21313664317131042, 'kl': 0.00634765625, 'epoch': 0.92} 18%|█▊ | 297/1610 [1:14:08<5:36:55, 15.40s/it] 19%|█▊ | 298/1610 [1:14:23<5:39:01, 15.50s/it] {'loss': 0.0003, 'grad_norm': 0.9090217595132158, 'learning_rate': 8.149068322981366e-07, 'completion_length': 156.1964340209961, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.27804867923259735, 'kl': 0.0062713623046875, 'epoch': 0.93} 19%|█▊ | 298/1610 [1:14:23<5:39:01, 15.50s/it] 19%|█▊ | 299/1610 [1:14:40<5:43:47, 15.73s/it] {'loss': 0.0003, 'grad_norm': 1.2267348458430258, 'learning_rate': 8.142857142857142e-07, 'completion_length': 185.23214721679688, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.18849068880081177, 'kl': 0.007110595703125, 'epoch': 0.93} 19%|█▊ | 299/1610 [1:14:40<5:43:47, 15.73s/it] 19%|█▊ | 300/1610 [1:14:54<5:34:43, 15.33s/it] {'loss': 0.0003, 'grad_norm': 1.1079115159214123, 'learning_rate': 8.136645962732918e-07, 'completion_length': 142.06250762939453, 'rewards/accuracy_reward': 0.6696428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6607143878936768, 'reward_std': 0.23266181349754333, 'kl': 0.0081787109375, 'epoch': 0.93} 19%|█▊ | 300/1610 [1:14:54<5:34:43, 15.33s/it] 19%|█▊ | 301/1610 [1:16:07<11:48:01, 32.45s/it] {'loss': 0.0003, 'grad_norm': 1.7510989194970525, 'learning_rate': 8.130434782608695e-07, 'completion_length': 127.29464721679688, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.22363825142383575, 'kl': 0.0080413818359375, 'epoch': 0.93} 19%|█▊ | 301/1610 [1:16:07<11:48:01, 32.45s/it] 19%|█▉ | 302/1610 [1:16:23<9:59:50, 27.52s/it] {'loss': 0.0003, 'grad_norm': 1.2766410203384544, 'learning_rate': 8.124223602484471e-07, 'completion_length': 172.55358123779297, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.285714328289032, 'reward_std': 0.30721089243888855, 'kl': 0.0077972412109375, 'epoch': 0.94} 19%|█▉ | 302/1610 [1:16:23<9:59:50, 27.52s/it] 19%|█▉ | 303/1610 [1:16:38<8:41:59, 23.96s/it] {'loss': 0.0003, 'grad_norm': 0.9237977998264938, 'learning_rate': 8.118012422360247e-07, 'completion_length': 171.74108123779297, 'rewards/accuracy_reward': 0.3660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.2891491949558258, 'kl': 0.007568359375, 'epoch': 0.94} 19%|█▉ | 303/1610 [1:16:38<8:41:59, 23.96s/it] 19%|█▉ | 304/1610 [1:16:53<7:38:51, 21.08s/it] {'loss': 0.0003, 'grad_norm': 0.9905504010739442, 'learning_rate': 8.111801242236025e-07, 'completion_length': 141.55358123779297, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.19569402188062668, 'kl': 0.007781982421875, 'epoch': 0.94} 19%|█▉ | 304/1610 [1:16:53<7:38:51, 21.08s/it] 19%|█▉ | 305/1610 [1:17:08<7:02:47, 19.44s/it] {'loss': 0.0003, 'grad_norm': 1.5095290015661738, 'learning_rate': 8.105590062111801e-07, 'completion_length': 145.3214340209961, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.37784259021282196, 'kl': 0.00653076171875, 'epoch': 0.95} 19%|█▉ | 305/1610 [1:17:08<7:02:47, 19.44s/it] 19%|█▉ | 306/1610 [1:17:23<6:31:09, 18.00s/it] {'loss': 0.0003, 'grad_norm': 1.260385379526116, 'learning_rate': 8.099378881987577e-07, 'completion_length': 141.16964721679688, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.20619001984596252, 'kl': 0.0079803466796875, 'epoch': 0.95} 19%|█▉ | 306/1610 [1:17:23<6:31:09, 18.00s/it] 19%|█▉ | 307/1610 [1:17:39<6:16:07, 17.32s/it] {'loss': 0.0002, 'grad_norm': 1.3250140406959066, 'learning_rate': 8.093167701863354e-07, 'completion_length': 159.33929443359375, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.362833708524704, 'kl': 0.0062255859375, 'epoch': 0.95} 19%|█▉ | 307/1610 [1:17:39<6:16:07, 17.32s/it] 19%|█▉ | 308/1610 [1:17:54<6:03:13, 16.74s/it] {'loss': 0.0003, 'grad_norm': 1.63518293768166, 'learning_rate': 8.08695652173913e-07, 'completion_length': 150.1875, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.348214328289032, 'reward_std': 0.27535825222730637, 'kl': 0.007598876953125, 'epoch': 0.96} 19%|█▉ | 308/1610 [1:17:54<6:03:13, 16.74s/it] 19%|█▉ | 309/1610 [1:18:09<5:51:33, 16.21s/it] {'loss': 0.0003, 'grad_norm': 1.315868644506619, 'learning_rate': 8.080745341614906e-07, 'completion_length': 137.5714340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2987806051969528, 'kl': 0.0068359375, 'epoch': 0.96} 19%|█▉ | 309/1610 [1:18:09<5:51:33, 16.21s/it] 19%|█▉ | 310/1610 [1:18:24<5:43:24, 15.85s/it] {'loss': 0.0003, 'grad_norm': 1.0519683752183357, 'learning_rate': 8.074534161490683e-07, 'completion_length': 156.93750762939453, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2985149472951889, 'kl': 0.0077362060546875, 'epoch': 0.96} 19%|█▉ | 310/1610 [1:18:24<5:43:24, 15.85s/it] 19%|█▉ | 311/1610 [1:18:37<5:26:35, 15.09s/it] {'loss': 0.0002, 'grad_norm': 2.19030811008431, 'learning_rate': 8.068322981366459e-07, 'completion_length': 138.6964340209961, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.23656460642814636, 'kl': 0.005706787109375, 'epoch': 0.97} 19%|█▉ | 311/1610 [1:18:37<5:26:35, 15.09s/it] 19%|█▉ | 312/1610 [1:18:54<5:37:19, 15.59s/it] {'loss': 0.0004, 'grad_norm': 1.1298812154719728, 'learning_rate': 8.062111801242235e-07, 'completion_length': 197.4107208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2813379764556885, 'kl': 0.00958251953125, 'epoch': 0.97} 19%|█▉ | 312/1610 [1:18:54<5:37:19, 15.59s/it] 19%|█▉ | 313/1610 [1:19:10<5:42:57, 15.87s/it] {'loss': 0.0003, 'grad_norm': 1.212632536738499, 'learning_rate': 8.055900621118013e-07, 'completion_length': 148.0982208251953, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.29367104172706604, 'kl': 0.0074920654296875, 'epoch': 0.97} 19%|█▉ | 313/1610 [1:19:10<5:42:57, 15.87s/it] 20%|█▉ | 314/1610 [1:19:25<5:33:45, 15.45s/it] {'loss': 0.0003, 'grad_norm': 1.2029074712439416, 'learning_rate': 8.049689440993789e-07, 'completion_length': 156.08929443359375, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23898375779390335, 'kl': 0.00701904296875, 'epoch': 0.98} 20%|█▉ | 314/1610 [1:19:25<5:33:45, 15.45s/it] 20%|█▉ | 315/1610 [1:19:41<5:35:21, 15.54s/it] {'loss': 0.0003, 'grad_norm': 1.5567115840986692, 'learning_rate': 8.043478260869565e-07, 'completion_length': 149.3660774230957, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.321428656578064, 'reward_std': 0.346597820520401, 'kl': 0.00738525390625, 'epoch': 0.98} 20%|█▉ | 315/1610 [1:19:41<5:35:21, 15.54s/it] 20%|█▉ | 316/1610 [1:19:56<5:35:51, 15.57s/it] {'loss': 0.0003, 'grad_norm': 1.5976414205685585, 'learning_rate': 8.037267080745342e-07, 'completion_length': 155.80358123779297, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.4139402061700821, 'kl': 0.007904052734375, 'epoch': 0.98} 20%|█▉ | 316/1610 [1:19:56<5:35:51, 15.57s/it] 20%|█▉ | 317/1610 [1:20:12<5:34:45, 15.53s/it] {'loss': 0.0003, 'grad_norm': 1.6359327903232823, 'learning_rate': 8.031055900621118e-07, 'completion_length': 163.9464340209961, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3571429252624512, 'reward_std': 0.36491061747074127, 'kl': 0.008270263671875, 'epoch': 0.98} 20%|█▉ | 317/1610 [1:20:12<5:34:45, 15.53s/it] 20%|█▉ | 318/1610 [1:20:28<5:39:45, 15.78s/it] {'loss': 0.0003, 'grad_norm': 1.2317595314743681, 'learning_rate': 8.024844720496894e-07, 'completion_length': 141.6428680419922, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.24102739989757538, 'kl': 0.007537841796875, 'epoch': 0.99} 20%|█▉ | 318/1610 [1:20:28<5:39:45, 15.78s/it] 20%|█▉ | 319/1610 [1:20:41<5:22:31, 14.99s/it] {'loss': 0.0003, 'grad_norm': 3.660616839777192, 'learning_rate': 8.018633540372671e-07, 'completion_length': 125.46429061889648, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.21434341371059418, 'kl': 0.008331298828125, 'epoch': 0.99} 20%|█▉ | 319/1610 [1:20:41<5:22:31, 14.99s/it] 20%|█▉ | 320/1610 [1:20:56<5:21:20, 14.95s/it] {'loss': 0.0003, 'grad_norm': 3.217132432412527, 'learning_rate': 8.012422360248446e-07, 'completion_length': 143.7589340209961, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.32916080951690674, 'kl': 0.007598876953125, 'epoch': 0.99} 20%|█▉ | 320/1610 [1:20:56<5:21:20, 14.95s/it] 20%|█▉ | 321/1610 [1:21:11<5:23:33, 15.06s/it] {'loss': 0.0003, 'grad_norm': 4.949131716248317, 'learning_rate': 8.006211180124222e-07, 'completion_length': 175.06250762939453, 'rewards/accuracy_reward': 0.5089286118745804, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.35831741988658905, 'kl': 0.0072479248046875, 'epoch': 1.0} 20%|█▉ | 321/1610 [1:21:11<5:23:33, 15.06s/it] 20%|██ | 322/1610 [1:21:27<5:23:18, 15.06s/it] {'loss': 0.0002, 'grad_norm': 11.192152814955607, 'learning_rate': 8e-07, 'completion_length': 141.0982208251953, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.2669392004609108, 'kl': 0.0062255859375, 'epoch': 1.0} 20%|██ | 322/1610 [1:21:27<5:23:18, 15.06s/it] 20%|██ | 323/1610 [1:21:42<5:23:16, 15.07s/it] {'loss': 0.0003, 'grad_norm': 0.9957043066066469, 'learning_rate': 7.993788819875776e-07, 'completion_length': 164.8928680419922, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.28768569231033325, 'kl': 0.0067901611328125, 'epoch': 1.0} 20%|██ | 323/1610 [1:21:42<5:23:16, 15.07s/it] 20%|██ | 324/1610 [1:21:57<5:26:47, 15.25s/it] {'loss': 0.0004, 'grad_norm': 1.601021102584194, 'learning_rate': 7.987577639751552e-07, 'completion_length': 180.42858123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.33570055663585663, 'kl': 0.00885009765625, 'epoch': 1.01} 20%|██ | 324/1610 [1:21:57<5:26:47, 15.25s/it] 20%|██ | 325/1610 [1:22:13<5:30:56, 15.45s/it] {'loss': 0.0003, 'grad_norm': 0.9935374363324602, 'learning_rate': 7.981366459627329e-07, 'completion_length': 175.64286041259766, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5892857909202576, 'reward_std': 0.24372340738773346, 'kl': 0.0070953369140625, 'epoch': 1.01} 20%|██ | 325/1610 [1:22:13<5:30:56, 15.45s/it] 20%|██ | 326/1610 [1:22:29<5:29:44, 15.41s/it] {'loss': 0.0003, 'grad_norm': 1.223500400199832, 'learning_rate': 7.975155279503105e-07, 'completion_length': 159.96429443359375, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.297916054725647, 'kl': 0.008514404296875, 'epoch': 1.01} 20%|██ | 326/1610 [1:22:29<5:29:44, 15.41s/it] 20%|██ | 327/1610 [1:22:44<5:29:06, 15.39s/it] {'loss': 0.0003, 'grad_norm': 1.5209777837272245, 'learning_rate': 7.968944099378881e-07, 'completion_length': 143.77679443359375, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.26814597845077515, 'kl': 0.007232666015625, 'epoch': 1.02} 20%|██ | 327/1610 [1:22:44<5:29:06, 15.39s/it] 20%|██ | 328/1610 [1:23:01<5:40:36, 15.94s/it] {'loss': 0.0003, 'grad_norm': 1.634148719448182, 'learning_rate': 7.962732919254658e-07, 'completion_length': 176.4821548461914, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.3558926433324814, 'kl': 0.0074310302734375, 'epoch': 1.02} 20%|██ | 328/1610 [1:23:01<5:40:36, 15.94s/it] 20%|██ | 329/1610 [1:23:18<5:47:56, 16.30s/it] {'loss': 0.0003, 'grad_norm': 1.6834714647729725, 'learning_rate': 7.956521739130434e-07, 'completion_length': 173.08929443359375, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.37393419444561005, 'kl': 0.0081787109375, 'epoch': 1.02} 20%|██ | 329/1610 [1:23:18<5:47:56, 16.30s/it] 20%|██ | 330/1610 [1:23:35<5:50:55, 16.45s/it] {'loss': 0.0003, 'grad_norm': 3.6163355281589036, 'learning_rate': 7.95031055900621e-07, 'completion_length': 182.66964721679688, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.3839285969734192, 'reward_std': 0.24167978018522263, 'kl': 0.0068206787109375, 'epoch': 1.02} 20%|██ | 330/1610 [1:23:35<5:50:55, 16.45s/it] 21%|██ | 331/1610 [1:23:49<5:37:10, 15.82s/it] {'loss': 0.0005, 'grad_norm': 1.9848957227281747, 'learning_rate': 7.944099378881988e-07, 'completion_length': 148.62500762939453, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2987918108701706, 'kl': 0.011932373046875, 'epoch': 1.03} 21%|██ | 331/1610 [1:23:49<5:37:10, 15.82s/it] 21%|██ | 332/1610 [1:24:04<5:31:51, 15.58s/it] {'loss': 0.0003, 'grad_norm': 1.375701647036999, 'learning_rate': 7.937888198757764e-07, 'completion_length': 159.1964340209961, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.359524205327034, 'kl': 0.0074462890625, 'epoch': 1.03} 21%|██ | 332/1610 [1:24:04<5:31:51, 15.58s/it] 21%|██ | 333/1610 [1:24:20<5:33:28, 15.67s/it] {'loss': 0.0003, 'grad_norm': 2.2251098338798734, 'learning_rate': 7.93167701863354e-07, 'completion_length': 152.6339340209961, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3117125779390335, 'kl': 0.008270263671875, 'epoch': 1.03} 21%|██ | 333/1610 [1:24:20<5:33:28, 15.67s/it] 21%|██ | 334/1610 [1:24:35<5:28:08, 15.43s/it] {'loss': 0.0003, 'grad_norm': 1.1386198986221334, 'learning_rate': 7.925465838509317e-07, 'completion_length': 168.1607208251953, 'rewards/accuracy_reward': 0.5178571492433548, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2708420008420944, 'kl': 0.00677490234375, 'epoch': 1.04} 21%|██ | 334/1610 [1:24:35<5:28:08, 15.43s/it] 21%|██ | 335/1610 [1:24:49<5:20:33, 15.08s/it] {'loss': 0.0003, 'grad_norm': 0.8298577738219276, 'learning_rate': 7.919254658385093e-07, 'completion_length': 137.6785774230957, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910715222358704, 'reward_std': 0.17616324126720428, 'kl': 0.00836181640625, 'epoch': 1.04} 21%|██ | 335/1610 [1:24:49<5:20:33, 15.08s/it] 21%|██ | 336/1610 [1:25:03<5:11:08, 14.65s/it] {'loss': 0.0003, 'grad_norm': 1.5427054476219921, 'learning_rate': 7.913043478260869e-07, 'completion_length': 128.64286422729492, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.3700370043516159, 'kl': 0.007904052734375, 'epoch': 1.04} 21%|██ | 336/1610 [1:25:03<5:11:08, 14.65s/it] 21%|██ | 337/1610 [1:25:18<5:12:00, 14.71s/it] {'loss': 0.0003, 'grad_norm': 1.1088597616895692, 'learning_rate': 7.906832298136646e-07, 'completion_length': 140.73214721679688, 'rewards/accuracy_reward': 0.5982143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.224850632250309, 'kl': 0.0067138671875, 'epoch': 1.05} 21%|██ | 337/1610 [1:25:18<5:12:00, 14.71s/it] 21%|██ | 338/1610 [1:25:34<5:17:25, 14.97s/it] {'loss': 0.0004, 'grad_norm': 1.147210355444577, 'learning_rate': 7.900621118012422e-07, 'completion_length': 186.41072845458984, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.3727218061685562, 'kl': 0.01031494140625, 'epoch': 1.05} 21%|██ | 338/1610 [1:25:34<5:17:25, 14.97s/it] 21%|██ | 339/1610 [1:25:49<5:18:51, 15.05s/it] {'loss': 0.0004, 'grad_norm': 1.2117788842821344, 'learning_rate': 7.894409937888198e-07, 'completion_length': 160.35714721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.32915520668029785, 'kl': 0.01025390625, 'epoch': 1.05} 21%|██ | 339/1610 [1:25:49<5:18:51, 15.05s/it] 21%|██ | 340/1610 [1:26:04<5:18:13, 15.03s/it] {'loss': 0.0003, 'grad_norm': 1.2978253738655556, 'learning_rate': 7.888198757763976e-07, 'completion_length': 139.91964721679688, 'rewards/accuracy_reward': 0.3303571492433548, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.2215557098388672, 'kl': 0.007537841796875, 'epoch': 1.06} 21%|██ | 340/1610 [1:26:04<5:18:13, 15.03s/it] 21%|██ | 341/1610 [1:26:19<5:18:22, 15.05s/it] {'loss': 0.0003, 'grad_norm': 1.1849965333657937, 'learning_rate': 7.881987577639752e-07, 'completion_length': 174.18750762939453, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.38357123732566833, 'kl': 0.008270263671875, 'epoch': 1.06} 21%|██ | 341/1610 [1:26:19<5:18:22, 15.05s/it] 21%|██ | 342/1610 [1:26:33<5:13:19, 14.83s/it] {'loss': 0.0004, 'grad_norm': 1.0315051277517837, 'learning_rate': 7.875776397515528e-07, 'completion_length': 160.83928680419922, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.16262900829315186, 'kl': 0.0087432861328125, 'epoch': 1.06} 21%|██ | 342/1610 [1:26:33<5:13:19, 14.83s/it] 21%|██▏ | 343/1610 [1:26:48<5:11:52, 14.77s/it] {'loss': 0.0003, 'grad_norm': 1.2229001992041897, 'learning_rate': 7.869565217391305e-07, 'completion_length': 161.4107208251953, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3117125928401947, 'kl': 0.0067901611328125, 'epoch': 1.07} 21%|██▏ | 343/1610 [1:26:48<5:11:52, 14.77s/it] 21%|██▏ | 344/1610 [1:27:02<5:09:59, 14.69s/it] {'loss': 0.0003, 'grad_norm': 1.002677644848842, 'learning_rate': 7.86335403726708e-07, 'completion_length': 148.7589340209961, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.17824578285217285, 'kl': 0.0069427490234375, 'epoch': 1.07} 21%|██▏ | 344/1610 [1:27:02<5:09:59, 14.69s/it] 21%|██▏ | 345/1610 [1:27:19<5:24:20, 15.38s/it] {'loss': 0.0003, 'grad_norm': 1.7128419987706625, 'learning_rate': 7.857142857142856e-07, 'completion_length': 164.02679443359375, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3750000596046448, 'reward_std': 0.3562348484992981, 'kl': 0.0081329345703125, 'epoch': 1.07} 21%|██▏ | 345/1610 [1:27:19<5:24:20, 15.38s/it] 21%|██▏ | 346/1610 [1:27:34<5:21:48, 15.28s/it] {'loss': 0.0004, 'grad_norm': 1.7650123146224523, 'learning_rate': 7.850931677018633e-07, 'completion_length': 161.0357208251953, 'rewards/accuracy_reward': 0.5625, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.4298282563686371, 'kl': 0.009735107421875, 'epoch': 1.07} 21%|██▏ | 346/1610 [1:27:34<5:21:48, 15.28s/it] 22%|██▏ | 347/1610 [1:27:48<5:13:55, 14.91s/it] {'loss': 0.0003, 'grad_norm': 1.947423230046198, 'learning_rate': 7.844720496894409e-07, 'completion_length': 146.4464340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3531966507434845, 'kl': 0.008148193359375, 'epoch': 1.08} 22%|██▏ | 347/1610 [1:27:48<5:13:55, 14.91s/it] 22%|██▏ | 348/1610 [1:28:04<5:21:00, 15.26s/it] {'loss': 0.0003, 'grad_norm': 1.225485937754779, 'learning_rate': 7.838509316770185e-07, 'completion_length': 164.46429443359375, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.3571050465106964, 'kl': 0.0071868896484375, 'epoch': 1.08} 22%|██▏ | 348/1610 [1:28:04<5:21:00, 15.26s/it] 22%|██▏ | 349/1610 [1:28:20<5:20:21, 15.24s/it] {'loss': 0.0003, 'grad_norm': 1.1384217502704304, 'learning_rate': 7.832298136645963e-07, 'completion_length': 163.76786041259766, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.34868597984313965, 'kl': 0.0081634521484375, 'epoch': 1.08} 22%|██▏ | 349/1610 [1:28:20<5:20:21, 15.24s/it] 22%|██▏ | 350/1610 [1:28:34<5:17:26, 15.12s/it] {'loss': 0.0004, 'grad_norm': 1.8482221256917226, 'learning_rate': 7.826086956521739e-07, 'completion_length': 159.4732208251953, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.24889206886291504, 'kl': 0.010223388671875, 'epoch': 1.09} 22%|██▏ | 350/1610 [1:28:34<5:17:26, 15.12s/it] 22%|██▏ | 351/1610 [1:28:50<5:19:38, 15.23s/it] {'loss': 0.0003, 'grad_norm': 1.320626419844421, 'learning_rate': 7.819875776397515e-07, 'completion_length': 176.0178680419922, 'rewards/accuracy_reward': 0.5089285895228386, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.2740867882966995, 'kl': 0.0079345703125, 'epoch': 1.09} 22%|██▏ | 351/1610 [1:28:50<5:19:38, 15.23s/it] 22%|██▏ | 352/1610 [1:29:06<5:23:56, 15.45s/it] {'loss': 0.0003, 'grad_norm': 1.0663790198886152, 'learning_rate': 7.813664596273292e-07, 'completion_length': 189.00000762939453, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4464285969734192, 'reward_std': 0.2784326523542404, 'kl': 0.0083770751953125, 'epoch': 1.09} 22%|██▏ | 352/1610 [1:29:06<5:23:56, 15.45s/it] 22%|██▏ | 353/1610 [1:29:24<5:38:15, 16.15s/it] {'loss': 0.0004, 'grad_norm': 1.0421353888277463, 'learning_rate': 7.807453416149068e-07, 'completion_length': 218.96430206298828, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9375000596046448, 'reward': 1.3214285969734192, 'reward_std': 0.4210788756608963, 'kl': 0.009552001953125, 'epoch': 1.1} 22%|██▏ | 353/1610 [1:29:24<5:38:15, 16.15s/it] 22%|██▏ | 354/1610 [1:29:38<5:28:45, 15.71s/it] {'loss': 0.0003, 'grad_norm': 1.000639516922082, 'learning_rate': 7.801242236024844e-07, 'completion_length': 147.0357208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.19057324528694153, 'kl': 0.007568359375, 'epoch': 1.1} 22%|██▏ | 354/1610 [1:29:38<5:28:45, 15.71s/it] 22%|██▏ | 355/1610 [1:29:54<5:28:04, 15.68s/it] {'loss': 0.0004, 'grad_norm': 1.2274580796795198, 'learning_rate': 7.79503105590062e-07, 'completion_length': 155.57144165039062, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.3369663953781128, 'kl': 0.009613037109375, 'epoch': 1.1} 22%|██▏ | 355/1610 [1:29:54<5:28:04, 15.68s/it] 22%|██▏ | 356/1610 [1:30:10<5:27:56, 15.69s/it] {'loss': 0.0003, 'grad_norm': 1.153886251974417, 'learning_rate': 7.788819875776397e-07, 'completion_length': 162.7589340209961, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3839285969734192, 'reward_std': 0.28707224130630493, 'kl': 0.0069427490234375, 'epoch': 1.11} 22%|██▏ | 356/1610 [1:30:10<5:27:56, 15.69s/it] 22%|██▏ | 357/1610 [1:30:25<5:25:39, 15.59s/it] {'loss': 0.0003, 'grad_norm': 2.17762068585578, 'learning_rate': 7.782608695652173e-07, 'completion_length': 171.06250762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.3780349791049957, 'kl': 0.008026123046875, 'epoch': 1.11} 22%|██▏ | 357/1610 [1:30:25<5:25:39, 15.59s/it] 22%|██▏ | 358/1610 [1:30:41<5:24:32, 15.55s/it] {'loss': 0.0003, 'grad_norm': 0.8557129381354912, 'learning_rate': 7.776397515527951e-07, 'completion_length': 154.20536041259766, 'rewards/accuracy_reward': 0.5625, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.22667087614536285, 'kl': 0.0074310302734375, 'epoch': 1.11} 22%|██▏ | 358/1610 [1:30:41<5:24:32, 15.55s/it] 22%|██▏ | 359/1610 [1:30:55<5:20:26, 15.37s/it] {'loss': 0.0003, 'grad_norm': 1.6127596936686714, 'learning_rate': 7.770186335403727e-07, 'completion_length': 162.80358123779297, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2501044422388077, 'kl': 0.0086669921875, 'epoch': 1.11} 22%|██▏ | 359/1610 [1:30:55<5:20:26, 15.37s/it] 22%|██▏ | 360/1610 [1:31:09<5:11:27, 14.95s/it] {'loss': 0.0003, 'grad_norm': 1.2936836609937061, 'learning_rate': 7.763975155279503e-07, 'completion_length': 141.91964721679688, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857313156128, 'reward_std': 0.27804866433143616, 'kl': 0.0078582763671875, 'epoch': 1.12} 22%|██▏ | 360/1610 [1:31:09<5:11:27, 14.95s/it] 22%|██▏ | 361/1610 [1:31:24<5:09:30, 14.87s/it] {'loss': 0.0004, 'grad_norm': 0.9786790501113058, 'learning_rate': 7.75776397515528e-07, 'completion_length': 148.92857360839844, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.1761632338166237, 'kl': 0.009490966796875, 'epoch': 1.12} 22%|██▏ | 361/1610 [1:31:24<5:09:30, 14.87s/it] 22%|██▏ | 362/1610 [1:31:38<5:04:26, 14.64s/it] {'loss': 0.0003, 'grad_norm': 1.4876562561743458, 'learning_rate': 7.751552795031056e-07, 'completion_length': 152.37500381469727, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.24619603902101517, 'kl': 0.0068817138671875, 'epoch': 1.12} 22%|██▏ | 362/1610 [1:31:38<5:04:26, 14.64s/it] 23%|██▎ | 363/1610 [1:31:53<5:03:17, 14.59s/it] {'loss': 0.0003, 'grad_norm': 2.026316589644258, 'learning_rate': 7.745341614906832e-07, 'completion_length': 137.0714340209961, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.30963005125522614, 'kl': 0.0077972412109375, 'epoch': 1.13} 23%|██▎ | 363/1610 [1:31:53<5:03:17, 14.59s/it] 23%|██▎ | 364/1610 [1:32:08<5:05:50, 14.73s/it] {'loss': 0.0004, 'grad_norm': 1.4676922983322973, 'learning_rate': 7.739130434782608e-07, 'completion_length': 160.50000762939453, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2726622223854065, 'kl': 0.009979248046875, 'epoch': 1.13} 23%|██▎ | 364/1610 [1:32:08<5:05:50, 14.73s/it] 23%|██▎ | 365/1610 [1:32:24<5:13:58, 15.13s/it] {'loss': 0.0004, 'grad_norm': 1.5498682498236696, 'learning_rate': 7.732919254658385e-07, 'completion_length': 154.33929443359375, 'rewards/accuracy_reward': 0.2678571492433548, 'rewards/format_reward': 1.0, 'reward': 1.2678571939468384, 'reward_std': 0.18276765942573547, 'kl': 0.0103759765625, 'epoch': 1.13} 23%|██▎ | 365/1610 [1:32:24<5:13:58, 15.13s/it] 23%|██▎ | 366/1610 [1:32:39<5:16:57, 15.29s/it] {'loss': 0.0003, 'grad_norm': 1.0337075736147152, 'learning_rate': 7.726708074534161e-07, 'completion_length': 168.46429443359375, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.2540072351694107, 'kl': 0.0086822509765625, 'epoch': 1.14} 23%|██▎ | 366/1610 [1:32:39<5:16:57, 15.29s/it] 23%|██▎ | 367/1610 [1:32:54<5:10:16, 14.98s/it] {'loss': 0.0003, 'grad_norm': 3.0715291751632896, 'learning_rate': 7.720496894409939e-07, 'completion_length': 140.8839340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.24949098378419876, 'kl': 0.0077362060546875, 'epoch': 1.14} 23%|██▎ | 367/1610 [1:32:54<5:10:16, 14.98s/it] 23%|██▎ | 368/1610 [1:33:09<5:11:21, 15.04s/it] {'loss': 0.0003, 'grad_norm': 0.973746694424955, 'learning_rate': 7.714285714285714e-07, 'completion_length': 187.2053680419922, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2897626608610153, 'kl': 0.0084991455078125, 'epoch': 1.14} 23%|██▎ | 368/1610 [1:33:09<5:11:21, 15.04s/it] 23%|██▎ | 369/1610 [1:33:25<5:19:42, 15.46s/it] {'loss': 0.0004, 'grad_norm': 1.4802915632146718, 'learning_rate': 7.70807453416149e-07, 'completion_length': 187.62500762939453, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.42620858550071716, 'kl': 0.011138916015625, 'epoch': 1.15} 23%|██▎ | 369/1610 [1:33:25<5:19:42, 15.46s/it] 23%|██▎ | 370/1610 [1:33:41<5:22:22, 15.60s/it] {'loss': 0.0004, 'grad_norm': 1.4978938368420527, 'learning_rate': 7.701863354037266e-07, 'completion_length': 153.68750762939453, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.3336714804172516, 'kl': 0.008819580078125, 'epoch': 1.15} 23%|██▎ | 370/1610 [1:33:41<5:22:22, 15.60s/it] 23%|██▎ | 371/1610 [1:33:56<5:17:09, 15.36s/it] {'loss': 0.0003, 'grad_norm': 1.0814805383160546, 'learning_rate': 7.695652173913043e-07, 'completion_length': 142.05357360839844, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3014766424894333, 'kl': 0.008087158203125, 'epoch': 1.15} 23%|██▎ | 371/1610 [1:33:56<5:17:09, 15.36s/it] 23%|██▎ | 372/1610 [1:34:11<5:15:25, 15.29s/it] {'loss': 0.0004, 'grad_norm': 1.6328765272863686, 'learning_rate': 7.689440993788819e-07, 'completion_length': 172.31250762939453, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.39649760723114014, 'kl': 0.0092010498046875, 'epoch': 1.16} 23%|██▎ | 372/1610 [1:34:11<5:15:25, 15.29s/it] 23%|██▎ | 373/1610 [1:34:27<5:17:32, 15.40s/it] {'loss': 0.0003, 'grad_norm': 1.7761683918099431, 'learning_rate': 7.683229813664595e-07, 'completion_length': 163.26786041259766, 'rewards/accuracy_reward': 0.473214328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857909202576, 'reward_std': 0.28437621891498566, 'kl': 0.00872802734375, 'epoch': 1.16} 23%|██▎ | 373/1610 [1:34:27<5:17:32, 15.40s/it] 23%|██▎ | 374/1610 [1:34:43<5:22:56, 15.68s/it] {'loss': 0.0005, 'grad_norm': 2.8310368619448063, 'learning_rate': 7.677018633540372e-07, 'completion_length': 190.64286041259766, 'rewards/accuracy_reward': 0.5625, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.42626766860485077, 'kl': 0.0113525390625, 'epoch': 1.16} 23%|██▎ | 374/1610 [1:34:43<5:22:56, 15.68s/it] 23%|██▎ | 375/1610 [1:35:01<5:34:05, 16.23s/it] {'loss': 0.0004, 'grad_norm': 1.4749359663560753, 'learning_rate': 7.670807453416148e-07, 'completion_length': 205.8571548461914, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3392857909202576, 'reward_std': 0.3899551033973694, 'kl': 0.01055908203125, 'epoch': 1.16} 23%|██▎ | 375/1610 [1:35:01<5:34:05, 16.23s/it] 23%|██▎ | 376/1610 [1:35:15<5:22:19, 15.67s/it] {'loss': 0.0003, 'grad_norm': 2.0277530829552313, 'learning_rate': 7.664596273291925e-07, 'completion_length': 148.61608123779297, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.26181280612945557, 'kl': 0.0080108642578125, 'epoch': 1.17} 23%|██▎ | 376/1610 [1:35:15<5:22:19, 15.67s/it] 23%|██▎ | 377/1610 [1:35:30<5:19:50, 15.56s/it] {'loss': 0.0003, 'grad_norm': 11.980882848512293, 'learning_rate': 7.658385093167702e-07, 'completion_length': 165.125, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857313156128, 'reward_std': 0.26181841641664505, 'kl': 0.00787353515625, 'epoch': 1.17} 23%|██▎ | 377/1610 [1:35:30<5:19:50, 15.56s/it] 23%|██▎ | 378/1610 [1:35:47<5:26:46, 15.91s/it] {'loss': 0.0004, 'grad_norm': 1.0347292715553675, 'learning_rate': 7.652173913043478e-07, 'completion_length': 183.8214340209961, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3303571939468384, 'reward_std': 0.3597419559955597, 'kl': 0.00994873046875, 'epoch': 1.17} 23%|██▎ | 378/1610 [1:35:47<5:26:46, 15.91s/it] 24%|██▎ | 379/1610 [1:36:04<5:30:46, 16.12s/it] {'loss': 0.0004, 'grad_norm': 3.1563778214961524, 'learning_rate': 7.645962732919254e-07, 'completion_length': 164.32144165039062, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.18787722289562225, 'kl': 0.009246826171875, 'epoch': 1.18} 24%|██▎ | 379/1610 [1:36:04<5:30:46, 16.12s/it] 24%|██▎ | 380/1610 [1:36:19<5:23:13, 15.77s/it] {'loss': 0.0004, 'grad_norm': 1.3127413055081167, 'learning_rate': 7.639751552795031e-07, 'completion_length': 168.2678680419922, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.27144986391067505, 'kl': 0.009185791015625, 'epoch': 1.18} 24%|██▎ | 380/1610 [1:36:19<5:23:13, 15.77s/it] 24%|██▎ | 381/1610 [1:36:34<5:18:54, 15.57s/it] {'loss': 0.0004, 'grad_norm': 1.2193789526177556, 'learning_rate': 7.633540372670807e-07, 'completion_length': 166.06250762939453, 'rewards/accuracy_reward': 0.258928582072258, 'rewards/format_reward': 1.0, 'reward': 1.2589285969734192, 'reward_std': 0.23925502598285675, 'kl': 0.00970458984375, 'epoch': 1.18} 24%|██▎ | 381/1610 [1:36:34<5:18:54, 15.57s/it] 24%|██▎ | 382/1610 [1:36:49<5:17:06, 15.49s/it] {'loss': 0.0004, 'grad_norm': 1.2721544442801238, 'learning_rate': 7.627329192546583e-07, 'completion_length': 177.49108123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.18788282573223114, 'kl': 0.009368896484375, 'epoch': 1.19} 24%|██▎ | 382/1610 [1:36:49<5:17:06, 15.49s/it] 24%|██▍ | 383/1610 [1:37:04<5:15:34, 15.43s/it] {'loss': 0.0004, 'grad_norm': 1.3823311374913363, 'learning_rate': 7.62111801242236e-07, 'completion_length': 157.3214340209961, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.26572123169898987, 'kl': 0.00970458984375, 'epoch': 1.19} 24%|██▍ | 383/1610 [1:37:04<5:15:34, 15.43s/it] 24%|██▍ | 384/1610 [1:37:19<5:09:39, 15.15s/it] {'loss': 0.0004, 'grad_norm': 1.8548283466570648, 'learning_rate': 7.614906832298136e-07, 'completion_length': 153.3660774230957, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.32282765209674835, 'kl': 0.009033203125, 'epoch': 1.19} 24%|██▍ | 384/1610 [1:37:19<5:09:39, 15.15s/it] 24%|██▍ | 385/1610 [1:37:35<5:13:46, 15.37s/it] {'loss': 0.0003, 'grad_norm': 0.7361930674070561, 'learning_rate': 7.608695652173913e-07, 'completion_length': 183.49108123779297, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2182518169283867, 'kl': 0.007965087890625, 'epoch': 1.2} 24%|██▍ | 385/1610 [1:37:35<5:13:46, 15.37s/it] 24%|██▍ | 386/1610 [1:37:50<5:15:08, 15.45s/it] {'loss': 0.0004, 'grad_norm': 1.26480080422688, 'learning_rate': 7.60248447204969e-07, 'completion_length': 166.02679443359375, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.410714328289032, 'reward_std': 0.35896705090999603, 'kl': 0.0091552734375, 'epoch': 1.2} 24%|██▍ | 386/1610 [1:37:50<5:15:08, 15.45s/it] 24%|██▍ | 387/1610 [1:38:06<5:15:02, 15.46s/it] {'loss': 0.0004, 'grad_norm': 1.474570419910363, 'learning_rate': 7.596273291925466e-07, 'completion_length': 170.85714721679688, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.3303731828927994, 'kl': 0.00958251953125, 'epoch': 1.2} 24%|██▍ | 387/1610 [1:38:06<5:15:02, 15.46s/it] 24%|██▍ | 388/1610 [1:38:20<5:05:38, 15.01s/it] {'loss': 0.0004, 'grad_norm': 1.1150058475278695, 'learning_rate': 7.590062111801242e-07, 'completion_length': 148.3214340209961, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.30721087753772736, 'kl': 0.009552001953125, 'epoch': 1.2} 24%|██▍ | 388/1610 [1:38:20<5:05:38, 15.01s/it] 24%|██▍ | 389/1610 [1:38:35<5:05:03, 14.99s/it] {'loss': 0.0004, 'grad_norm': 2.570018881452642, 'learning_rate': 7.583850931677019e-07, 'completion_length': 152.45536041259766, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.2960902005434036, 'kl': 0.009613037109375, 'epoch': 1.21} 24%|██▍ | 389/1610 [1:38:35<5:05:03, 14.99s/it] 24%|██▍ | 390/1610 [1:38:49<4:58:52, 14.70s/it] {'loss': 0.0004, 'grad_norm': 2.9087299131013076, 'learning_rate': 7.577639751552795e-07, 'completion_length': 134.08928680419922, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.25912241637706757, 'kl': 0.010101318359375, 'epoch': 1.21} 24%|██▍ | 390/1610 [1:38:49<4:58:52, 14.70s/it] 24%|██▍ | 391/1610 [1:39:04<5:02:30, 14.89s/it] {'loss': 0.0003, 'grad_norm': 1.5473153881348343, 'learning_rate': 7.57142857142857e-07, 'completion_length': 140.45536422729492, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.2540072277188301, 'kl': 0.0072784423828125, 'epoch': 1.21} 24%|██▍ | 391/1610 [1:39:04<5:02:30, 14.89s/it] 24%|██▍ | 392/1610 [1:39:20<5:10:43, 15.31s/it] {'loss': 0.0004, 'grad_norm': 1.6166548350203027, 'learning_rate': 7.565217391304347e-07, 'completion_length': 183.7678680419922, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857909202576, 'reward_std': 0.28828462213277817, 'kl': 0.0111083984375, 'epoch': 1.22} 24%|██▍ | 392/1610 [1:39:20<5:10:43, 15.31s/it] 24%|██▍ | 393/1610 [1:39:35<5:08:19, 15.20s/it] {'loss': 0.0004, 'grad_norm': 0.9389011112731034, 'learning_rate': 7.559006211180123e-07, 'completion_length': 155.54464721679688, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.26181842386722565, 'kl': 0.009552001953125, 'epoch': 1.22} 24%|██▍ | 393/1610 [1:39:35<5:08:19, 15.20s/it] 24%|██▍ | 394/1610 [1:39:51<5:13:10, 15.45s/it] {'loss': 0.0004, 'grad_norm': 1.4990574707869306, 'learning_rate': 7.5527950310559e-07, 'completion_length': 159.0089340209961, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.23265619575977325, 'kl': 0.009246826171875, 'epoch': 1.22} 24%|██▍ | 394/1610 [1:39:51<5:13:10, 15.45s/it] 25%|██▍ | 395/1610 [1:40:07<5:14:46, 15.54s/it] {'loss': 0.0004, 'grad_norm': 1.1416429163781658, 'learning_rate': 7.546583850931677e-07, 'completion_length': 185.5357208251953, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3279428333044052, 'kl': 0.0088958740234375, 'epoch': 1.23} 25%|██▍ | 395/1610 [1:40:07<5:14:46, 15.54s/it] 25%|██▍ | 396/1610 [1:40:23<5:13:53, 15.51s/it] {'loss': 0.0004, 'grad_norm': 2.5065851732694515, 'learning_rate': 7.540372670807453e-07, 'completion_length': 165.30357360839844, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.41211998462677, 'kl': 0.01080322265625, 'epoch': 1.23} 25%|██▍ | 396/1610 [1:40:23<5:13:53, 15.51s/it] 25%|██▍ | 397/1610 [1:40:38<5:15:57, 15.63s/it] {'loss': 0.0003, 'grad_norm': 1.2607440024499033, 'learning_rate': 7.534161490683229e-07, 'completion_length': 178.56250762939453, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.3219574838876724, 'kl': 0.0084228515625, 'epoch': 1.23} 25%|██▍ | 397/1610 [1:40:39<5:15:57, 15.63s/it] 25%|██▍ | 398/1610 [1:40:54<5:13:45, 15.53s/it] {'loss': 0.0004, 'grad_norm': 1.1625181587583024, 'learning_rate': 7.527950310559006e-07, 'completion_length': 147.39286041259766, 'rewards/accuracy_reward': 0.6250000149011612, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2735323905944824, 'kl': 0.00927734375, 'epoch': 1.24} 25%|██▍ | 398/1610 [1:40:54<5:13:45, 15.53s/it] 25%|██▍ | 399/1610 [1:41:11<5:23:45, 16.04s/it] {'loss': 0.0004, 'grad_norm': 1.1737140945262048, 'learning_rate': 7.521739130434782e-07, 'completion_length': 183.9196548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.973214328289032, 'reward': 1.5535714626312256, 'reward_std': 0.35298171639442444, 'kl': 0.008758544921875, 'epoch': 1.24} 25%|██▍ | 399/1610 [1:41:11<5:23:45, 16.04s/it] 25%|██▍ | 400/1610 [1:41:26<5:16:49, 15.71s/it] {'loss': 0.0004, 'grad_norm': 1.189644241168002, 'learning_rate': 7.515527950310558e-07, 'completion_length': 166.14286041259766, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.3267304301261902, 'kl': 0.009857177734375, 'epoch': 1.24} 25%|██▍ | 400/1610 [1:41:26<5:16:49, 15.71s/it] 25%|██▍ | 401/1610 [1:42:36<10:47:51, 32.15s/it] {'loss': 0.0004, 'grad_norm': 2.630964066092756, 'learning_rate': 7.509316770186335e-07, 'completion_length': 141.71429443359375, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.29097503423690796, 'kl': 0.010284423828125, 'epoch': 1.25} 25%|██▍ | 401/1610 [1:42:36<10:47:51, 32.15s/it] 25%|██▍ | 402/1610 [1:42:49<8:49:21, 26.29s/it] {'loss': 0.0003, 'grad_norm': 1.3025814394243582, 'learning_rate': 7.503105590062111e-07, 'completion_length': 120.41072082519531, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053572535514832, 'reward_std': 0.2657212167978287, 'kl': 0.00860595703125, 'epoch': 1.25} 25%|██▍ | 402/1610 [1:42:49<8:49:21, 26.29s/it] 25%|██▌ | 403/1610 [1:43:04<7:41:40, 22.95s/it] {'loss': 0.0004, 'grad_norm': 1.215659048678517, 'learning_rate': 7.496894409937888e-07, 'completion_length': 153.88393783569336, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.32915520668029785, 'kl': 0.0096435546875, 'epoch': 1.25} 25%|██▌ | 403/1610 [1:43:04<7:41:40, 22.95s/it] 25%|██▌ | 404/1610 [1:43:19<6:53:54, 20.59s/it] {'loss': 0.0003, 'grad_norm': 1.0675092696783204, 'learning_rate': 7.490683229813665e-07, 'completion_length': 137.9464340209961, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857313156128, 'reward_std': 0.24025246500968933, 'kl': 0.008026123046875, 'epoch': 1.25} 25%|██▌ | 404/1610 [1:43:19<6:53:54, 20.59s/it] 25%|██▌ | 405/1610 [1:43:34<6:18:42, 18.86s/it] {'loss': 0.0004, 'grad_norm': 1.6734249635484724, 'learning_rate': 7.484472049689441e-07, 'completion_length': 158.30358123779297, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3700314164161682, 'kl': 0.00909423828125, 'epoch': 1.26} 25%|██▌ | 405/1610 [1:43:34<6:18:42, 18.86s/it] 25%|██▌ | 406/1610 [1:43:51<6:04:30, 18.17s/it] {'loss': 0.0004, 'grad_norm': 0.9098962370318708, 'learning_rate': 7.478260869565217e-07, 'completion_length': 193.7589340209961, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553572535514832, 'reward_std': 0.25252359360456467, 'kl': 0.0096588134765625, 'epoch': 1.26} 25%|██▌ | 406/1610 [1:43:51<6:04:30, 18.17s/it] 25%|██▌ | 407/1610 [1:44:03<5:29:50, 16.45s/it] {'loss': 0.0003, 'grad_norm': 0.9865393162777908, 'learning_rate': 7.472049689440994e-07, 'completion_length': 138.45536041259766, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.22754104435443878, 'kl': 0.0074005126953125, 'epoch': 1.26} 25%|██▌ | 407/1610 [1:44:03<5:29:50, 16.45s/it] 25%|██▌ | 408/1610 [1:44:19<5:24:25, 16.19s/it] {'loss': 0.0004, 'grad_norm': 2.61888948694694, 'learning_rate': 7.46583850931677e-07, 'completion_length': 155.94644165039062, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.571428656578064, 'reward_std': 0.2669335901737213, 'kl': 0.0101776123046875, 'epoch': 1.27} 25%|██▌ | 408/1610 [1:44:19<5:24:25, 16.19s/it] 25%|██▌ | 409/1610 [1:44:33<5:12:53, 15.63s/it] {'loss': 0.0004, 'grad_norm': 1.8077885041530795, 'learning_rate': 7.459627329192546e-07, 'completion_length': 150.7500114440918, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.27474477887153625, 'kl': 0.0103759765625, 'epoch': 1.27} 25%|██▌ | 409/1610 [1:44:33<5:12:53, 15.63s/it] 25%|██▌ | 410/1610 [1:44:49<5:14:27, 15.72s/it] {'loss': 0.0003, 'grad_norm': 1.0755617607069436, 'learning_rate': 7.453416149068323e-07, 'completion_length': 171.55358123779297, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553571939468384, 'reward_std': 0.2579156458377838, 'kl': 0.0079803466796875, 'epoch': 1.27} 25%|██▌ | 410/1610 [1:44:49<5:14:27, 15.72s/it] 26%|██▌ | 411/1610 [1:45:04<5:10:47, 15.55s/it] {'loss': 0.0004, 'grad_norm': 1.870519677029246, 'learning_rate': 7.447204968944099e-07, 'completion_length': 162.52679443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.3655095398426056, 'kl': 0.009033203125, 'epoch': 1.28} 26%|██▌ | 411/1610 [1:45:04<5:10:47, 15.55s/it] 26%|██▌ | 412/1610 [1:45:19<5:06:22, 15.34s/it] {'loss': 0.0004, 'grad_norm': 3.6546328833913653, 'learning_rate': 7.440993788819876e-07, 'completion_length': 146.6607208251953, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.3214285969734192, 'reward_std': 0.3279484361410141, 'kl': 0.009918212890625, 'epoch': 1.28} 26%|██▌ | 412/1610 [1:45:19<5:06:22, 15.34s/it] 26%|██▌ | 413/1610 [1:45:34<5:01:11, 15.10s/it] {'loss': 0.0003, 'grad_norm': 2.488026180189813, 'learning_rate': 7.434782608695653e-07, 'completion_length': 142.90179443359375, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.41332675516605377, 'kl': 0.008544921875, 'epoch': 1.28} 26%|██▌ | 413/1610 [1:45:34<5:01:11, 15.10s/it] 26%|██▌ | 414/1610 [1:45:47<4:53:42, 14.73s/it] {'loss': 0.0004, 'grad_norm': 1.4085007087917119, 'learning_rate': 7.428571428571429e-07, 'completion_length': 148.80358123779297, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.35137641429901123, 'kl': 0.010040283203125, 'epoch': 1.29} 26%|██▌ | 414/1610 [1:45:47<4:53:42, 14.73s/it] 26%|██▌ | 415/1610 [1:46:02<4:50:27, 14.58s/it] {'loss': 0.0004, 'grad_norm': 1.7483055829056962, 'learning_rate': 7.422360248447204e-07, 'completion_length': 151.4732208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.36342698335647583, 'kl': 0.008880615234375, 'epoch': 1.29} 26%|██▌ | 415/1610 [1:46:02<4:50:27, 14.58s/it] 26%|██▌ | 416/1610 [1:46:17<4:55:43, 14.86s/it] {'loss': 0.0004, 'grad_norm': 1.2607622029412002, 'learning_rate': 7.416149068322981e-07, 'completion_length': 145.82144165039062, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2221546173095703, 'kl': 0.0089111328125, 'epoch': 1.29} 26%|██▌ | 416/1610 [1:46:17<4:55:43, 14.86s/it] 26%|██▌ | 417/1610 [1:46:32<4:55:50, 14.88s/it] {'loss': 0.0004, 'grad_norm': 2.2300625623827606, 'learning_rate': 7.409937888198757e-07, 'completion_length': 145.01786041259766, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.3057272583246231, 'kl': 0.0087890625, 'epoch': 1.3} 26%|██▌ | 417/1610 [1:46:32<4:55:50, 14.88s/it] 26%|██▌ | 418/1610 [1:46:46<4:50:48, 14.64s/it] {'loss': 0.0003, 'grad_norm': 1.2421688298186748, 'learning_rate': 7.403726708074533e-07, 'completion_length': 133.42858123779297, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.2987806349992752, 'kl': 0.0077667236328125, 'epoch': 1.3} 26%|██▌ | 418/1610 [1:46:46<4:50:48, 14.64s/it] 26%|██▌ | 419/1610 [1:47:03<5:04:14, 15.33s/it] {'loss': 0.0004, 'grad_norm': 1.1888943092940358, 'learning_rate': 7.39751552795031e-07, 'completion_length': 190.24108123779297, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.3392857909202576, 'reward_std': 0.35689009726047516, 'kl': 0.00994873046875, 'epoch': 1.3} 26%|██▌ | 419/1610 [1:47:03<5:04:14, 15.33s/it] 26%|██▌ | 420/1610 [1:47:18<5:00:19, 15.14s/it] {'loss': 0.0005, 'grad_norm': 1.1112661237689867, 'learning_rate': 7.391304347826086e-07, 'completion_length': 164.76786041259766, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.29097503423690796, 'kl': 0.011444091796875, 'epoch': 1.3} 26%|██▌ | 420/1610 [1:47:18<5:00:19, 15.14s/it] 26%|██▌ | 421/1610 [1:47:33<4:58:37, 15.07s/it] {'loss': 0.0003, 'grad_norm': 1.5131626727811514, 'learning_rate': 7.385093167701863e-07, 'completion_length': 157.4464340209961, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.30330249667167664, 'kl': 0.0079498291015625, 'epoch': 1.31} 26%|██▌ | 421/1610 [1:47:33<4:58:37, 15.07s/it] 26%|██▌ | 422/1610 [1:47:48<4:59:14, 15.11s/it] {'loss': 0.0004, 'grad_norm': 1.6504669370603529, 'learning_rate': 7.37888198757764e-07, 'completion_length': 158.77679443359375, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3213440179824829, 'kl': 0.011016845703125, 'epoch': 1.31} 26%|██▌ | 422/1610 [1:47:48<4:59:14, 15.11s/it] 26%|██▋ | 423/1610 [1:48:03<5:01:07, 15.22s/it] {'loss': 0.0004, 'grad_norm': 1.703201848398752, 'learning_rate': 7.372670807453416e-07, 'completion_length': 161.7589340209961, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.3084232658147812, 'kl': 0.00885009765625, 'epoch': 1.31} 26%|██▋ | 423/1610 [1:48:03<5:01:07, 15.22s/it] 26%|██▋ | 424/1610 [1:48:19<5:00:48, 15.22s/it] {'loss': 0.0004, 'grad_norm': 1.4425893568399153, 'learning_rate': 7.366459627329192e-07, 'completion_length': 156.4821548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3928660750389099, 'kl': 0.0108642578125, 'epoch': 1.32} 26%|██▋ | 424/1610 [1:48:19<5:00:48, 15.22s/it] 26%|██▋ | 425/1610 [1:48:35<5:08:46, 15.63s/it] {'loss': 0.0004, 'grad_norm': 1.7480697626599053, 'learning_rate': 7.360248447204969e-07, 'completion_length': 175.41964721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4821429252624512, 'reward_std': 0.3571050316095352, 'kl': 0.009063720703125, 'epoch': 1.32} 26%|██▋ | 425/1610 [1:48:35<5:08:46, 15.63s/it] 26%|██▋ | 426/1610 [1:48:52<5:12:31, 15.84s/it] {'loss': 0.0004, 'grad_norm': 1.300014872240272, 'learning_rate': 7.354037267080745e-07, 'completion_length': 156.4732208251953, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.3117125779390335, 'kl': 0.009002685546875, 'epoch': 1.32} 26%|██▋ | 426/1610 [1:48:52<5:12:31, 15.84s/it] 27%|██▋ | 427/1610 [1:49:07<5:10:30, 15.75s/it] {'loss': 0.0004, 'grad_norm': 1.145394213977329, 'learning_rate': 7.347826086956521e-07, 'completion_length': 162.5357208251953, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285714626312256, 'reward_std': 0.30659739673137665, 'kl': 0.011138916015625, 'epoch': 1.33} 27%|██▋ | 427/1610 [1:49:07<5:10:30, 15.75s/it] 27%|██▋ | 428/1610 [1:49:23<5:09:42, 15.72s/it] {'loss': 0.0004, 'grad_norm': 1.0863206063966624, 'learning_rate': 7.341614906832298e-07, 'completion_length': 175.5357208251953, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6160714626312256, 'reward_std': 0.35831741988658905, 'kl': 0.009521484375, 'epoch': 1.33} 27%|██▋ | 428/1610 [1:49:23<5:09:42, 15.72s/it] 27%|██▋ | 429/1610 [1:49:38<5:04:42, 15.48s/it] {'loss': 0.0004, 'grad_norm': 1.314717418717918, 'learning_rate': 7.335403726708074e-07, 'completion_length': 169.9553680419922, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.42403195798397064, 'kl': 0.01031494140625, 'epoch': 1.33} 27%|██▋ | 429/1610 [1:49:38<5:04:42, 15.48s/it] 27%|██▋ | 430/1610 [1:49:54<5:06:51, 15.60s/it] {'loss': 0.0004, 'grad_norm': 1.362648416409745, 'learning_rate': 7.329192546583851e-07, 'completion_length': 173.0982208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.29158851504325867, 'kl': 0.010162353515625, 'epoch': 1.34} 27%|██▋ | 430/1610 [1:49:54<5:06:51, 15.60s/it] 27%|██▋ | 431/1610 [1:50:08<5:01:49, 15.36s/it] {'loss': 0.0004, 'grad_norm': 2.0158646617332034, 'learning_rate': 7.322981366459628e-07, 'completion_length': 155.4553680419922, 'rewards/accuracy_reward': 0.5982143133878708, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2993997037410736, 'kl': 0.010101318359375, 'epoch': 1.34} 27%|██▋ | 431/1610 [1:50:08<5:01:49, 15.36s/it] 27%|██▋ | 432/1610 [1:50:24<5:01:59, 15.38s/it] {'loss': 0.0003, 'grad_norm': 1.255268997582583, 'learning_rate': 7.316770186335404e-07, 'completion_length': 145.2857208251953, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.27474478632211685, 'kl': 0.0078125, 'epoch': 1.34} 27%|██▋ | 432/1610 [1:50:24<5:01:59, 15.38s/it] 27%|██▋ | 433/1610 [1:50:39<4:59:54, 15.29s/it] {'loss': 0.0004, 'grad_norm': 1.1342998389156396, 'learning_rate': 7.31055900621118e-07, 'completion_length': 176.00000762939453, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.28047342598438263, 'kl': 0.0111083984375, 'epoch': 1.34} 27%|██▋ | 433/1610 [1:50:39<4:59:54, 15.29s/it] 27%|██▋ | 434/1610 [1:50:54<5:01:06, 15.36s/it] {'loss': 0.0003, 'grad_norm': 1.4636433247367429, 'learning_rate': 7.304347826086957e-07, 'completion_length': 168.39286041259766, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.35440899431705475, 'kl': 0.00860595703125, 'epoch': 1.35} 27%|██▋ | 434/1610 [1:50:54<5:01:06, 15.36s/it] 27%|██▋ | 435/1610 [1:51:09<4:58:21, 15.24s/it] {'loss': 0.0004, 'grad_norm': 1.3840745582053133, 'learning_rate': 7.298136645962733e-07, 'completion_length': 155.74108123779297, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.3375798612833023, 'kl': 0.00958251953125, 'epoch': 1.35} 27%|██▋ | 435/1610 [1:51:09<4:58:21, 15.24s/it] 27%|██▋ | 436/1610 [1:51:26<5:04:07, 15.54s/it] {'loss': 0.0003, 'grad_norm': 0.9574392314942125, 'learning_rate': 7.291925465838509e-07, 'completion_length': 176.3928680419922, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4553571939468384, 'reward_std': 0.3127100318670273, 'kl': 0.0076446533203125, 'epoch': 1.35} 27%|██▋ | 436/1610 [1:51:26<5:04:07, 15.54s/it] 27%|██▋ | 437/1610 [1:51:42<5:06:57, 15.70s/it] {'loss': 0.0005, 'grad_norm': 1.4193447064684845, 'learning_rate': 7.285714285714286e-07, 'completion_length': 169.85714721679688, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.29999297857284546, 'kl': 0.011444091796875, 'epoch': 1.36} 27%|██▋ | 437/1610 [1:51:42<5:06:57, 15.70s/it] 27%|██▋ | 438/1610 [1:51:57<5:03:20, 15.53s/it] {'loss': 0.0004, 'grad_norm': 1.8117802602101232, 'learning_rate': 7.279503105590061e-07, 'completion_length': 138.9464340209961, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6785715222358704, 'reward_std': 0.3351016193628311, 'kl': 0.009979248046875, 'epoch': 1.36} 27%|██▋ | 438/1610 [1:51:57<5:03:20, 15.53s/it] 27%|██▋ | 439/1610 [1:52:12<5:01:15, 15.44s/it] {'loss': 0.0004, 'grad_norm': 2.4475083102300186, 'learning_rate': 7.273291925465838e-07, 'completion_length': 167.50000762939453, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.3537510186433792, 'kl': 0.00970458984375, 'epoch': 1.36} 27%|██▋ | 439/1610 [1:52:12<5:01:15, 15.44s/it] 27%|██▋ | 440/1610 [1:52:29<5:07:47, 15.78s/it] {'loss': 0.0004, 'grad_norm': 1.7200928477736421, 'learning_rate': 7.267080745341615e-07, 'completion_length': 202.23214721679688, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375001192092896, 'reward_std': 0.37454207241535187, 'kl': 0.01116943359375, 'epoch': 1.37} 27%|██▋ | 440/1610 [1:52:29<5:07:47, 15.78s/it] 27%|██▋ | 441/1610 [1:52:43<5:02:18, 15.52s/it] {'loss': 0.0004, 'grad_norm': 1.2290640762222267, 'learning_rate': 7.260869565217391e-07, 'completion_length': 160.90179443359375, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.26181282103061676, 'kl': 0.00982666015625, 'epoch': 1.37} 27%|██▋ | 441/1610 [1:52:43<5:02:18, 15.52s/it] 27%|██▋ | 442/1610 [1:52:58<4:58:15, 15.32s/it] {'loss': 0.0005, 'grad_norm': 1.8952781152050548, 'learning_rate': 7.254658385093167e-07, 'completion_length': 152.9553680419922, 'rewards/accuracy_reward': 0.3750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.4057924300432205, 'kl': 0.011474609375, 'epoch': 1.37} 27%|██▋ | 442/1610 [1:52:58<4:58:15, 15.32s/it] 28%|██▊ | 443/1610 [1:53:16<5:09:40, 15.92s/it] {'loss': 0.0003, 'grad_norm': 1.4801376504989268, 'learning_rate': 7.248447204968943e-07, 'completion_length': 213.3303680419922, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9642857611179352, 'reward': 1.4821429252624512, 'reward_std': 0.3566839247941971, 'kl': 0.0081634521484375, 'epoch': 1.38} 28%|██▊ | 443/1610 [1:53:16<5:09:40, 15.92s/it] 28%|██▊ | 444/1610 [1:53:30<5:01:06, 15.49s/it] {'loss': 0.0004, 'grad_norm': 1.696997274479878, 'learning_rate': 7.24223602484472e-07, 'completion_length': 124.13393020629883, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.2696296125650406, 'kl': 0.010345458984375, 'epoch': 1.38} 28%|██▊ | 444/1610 [1:53:30<5:01:06, 15.49s/it] 28%|██▊ | 445/1610 [1:53:45<4:58:55, 15.40s/it] {'loss': 0.0005, 'grad_norm': 2.218591452784572, 'learning_rate': 7.236024844720496e-07, 'completion_length': 177.9821548461914, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.31623443961143494, 'kl': 0.012420654296875, 'epoch': 1.38} 28%|██▊ | 445/1610 [1:53:45<4:58:55, 15.40s/it] 28%|██▊ | 446/1610 [1:53:59<4:50:02, 14.95s/it] {'loss': 0.0003, 'grad_norm': 2.7589906648158915, 'learning_rate': 7.229813664596272e-07, 'completion_length': 136.63393783569336, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.22363825142383575, 'kl': 0.0078887939453125, 'epoch': 1.39} 28%|██▊ | 446/1610 [1:53:59<4:50:02, 14.95s/it] 28%|██▊ | 447/1610 [1:54:15<4:53:57, 15.17s/it] {'loss': 0.0005, 'grad_norm': 0.9973569494452941, 'learning_rate': 7.223602484472049e-07, 'completion_length': 168.54464721679688, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178572535514832, 'reward_std': 0.30963563919067383, 'kl': 0.0126953125, 'epoch': 1.39} 28%|██▊ | 447/1610 [1:54:15<4:53:57, 15.17s/it] 28%|██▊ | 448/1610 [1:54:31<4:58:26, 15.41s/it] {'loss': 0.0004, 'grad_norm': 5.743108542174556, 'learning_rate': 7.217391304347826e-07, 'completion_length': 168.91964721679688, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.33575962483882904, 'kl': 0.009735107421875, 'epoch': 1.39} 28%|██▊ | 448/1610 [1:54:31<4:58:26, 15.41s/it] 28%|██▊ | 449/1610 [1:54:45<4:53:09, 15.15s/it] {'loss': 0.0004, 'grad_norm': 0.9609850501084618, 'learning_rate': 7.211180124223603e-07, 'completion_length': 149.89286041259766, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.26181843876838684, 'kl': 0.010650634765625, 'epoch': 1.39} 28%|██▊ | 449/1610 [1:54:45<4:53:09, 15.15s/it] 28%|██▊ | 450/1610 [1:54:59<4:45:46, 14.78s/it] {'loss': 0.0004, 'grad_norm': 1.8620083255552302, 'learning_rate': 7.204968944099379e-07, 'completion_length': 140.8214340209961, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.22606299445033073, 'kl': 0.010284423828125, 'epoch': 1.4} 28%|██▊ | 450/1610 [1:54:59<4:45:46, 14.78s/it] 28%|██▊ | 451/1610 [1:55:14<4:45:30, 14.78s/it] {'loss': 0.0004, 'grad_norm': 1.1227361342439295, 'learning_rate': 7.198757763975155e-07, 'completion_length': 150.11607360839844, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.2158270627260208, 'kl': 0.00958251953125, 'epoch': 1.4} 28%|██▊ | 451/1610 [1:55:14<4:45:30, 14.78s/it] 28%|██▊ | 452/1610 [1:55:28<4:38:37, 14.44s/it] {'loss': 0.0004, 'grad_norm': 2.2965104074678906, 'learning_rate': 7.192546583850931e-07, 'completion_length': 138.49108123779297, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3396624028682709, 'kl': 0.009613037109375, 'epoch': 1.4} 28%|██▊ | 452/1610 [1:55:28<4:38:37, 14.44s/it] 28%|██▊ | 453/1610 [1:55:41<4:32:50, 14.15s/it] {'loss': 0.0003, 'grad_norm': 0.7527759311694654, 'learning_rate': 7.186335403726708e-07, 'completion_length': 136.5982208251953, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.14579425007104874, 'kl': 0.0084228515625, 'epoch': 1.41} 28%|██▊ | 453/1610 [1:55:41<4:32:50, 14.15s/it] 28%|██▊ | 454/1610 [1:55:58<4:48:22, 14.97s/it] {'loss': 0.0005, 'grad_norm': 1.9365951814741504, 'learning_rate': 7.180124223602484e-07, 'completion_length': 154.9732208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.7053571939468384, 'reward_std': 0.3155764490365982, 'kl': 0.011566162109375, 'epoch': 1.41} 28%|██▊ | 454/1610 [1:55:58<4:48:22, 14.97s/it] 28%|██▊ | 455/1610 [1:56:12<4:40:15, 14.56s/it] {'loss': 0.0003, 'grad_norm': 1.3287989665883284, 'learning_rate': 7.17391304347826e-07, 'completion_length': 142.2232208251953, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.30721089243888855, 'kl': 0.00848388671875, 'epoch': 1.41} 28%|██▊ | 455/1610 [1:56:12<4:40:15, 14.56s/it] 28%|██▊ | 456/1610 [1:56:27<4:44:31, 14.79s/it] {'loss': 0.0004, 'grad_norm': 0.8525610751684759, 'learning_rate': 7.167701863354037e-07, 'completion_length': 154.2857208251953, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.2708420008420944, 'kl': 0.010528564453125, 'epoch': 1.42} 28%|██▊ | 456/1610 [1:56:27<4:44:31, 14.79s/it] 28%|██▊ | 457/1610 [1:56:43<4:53:27, 15.27s/it] {'loss': 0.0004, 'grad_norm': 1.12529233399004, 'learning_rate': 7.161490683229814e-07, 'completion_length': 190.4821548461914, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553572535514832, 'reward_std': 0.2839607149362564, 'kl': 0.010650634765625, 'epoch': 1.42} 28%|██▊ | 457/1610 [1:56:43<4:53:27, 15.27s/it] 28%|██▊ | 458/1610 [1:56:59<4:54:54, 15.36s/it] {'loss': 0.0004, 'grad_norm': 1.4355071611031123, 'learning_rate': 7.15527950310559e-07, 'completion_length': 167.29464721679688, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285715222358704, 'reward_std': 0.2675470560789108, 'kl': 0.01043701171875, 'epoch': 1.42} 28%|██▊ | 458/1610 [1:56:59<4:54:54, 15.36s/it] 29%|██▊ | 459/1610 [1:57:14<4:54:52, 15.37s/it] {'loss': 0.0005, 'grad_norm': 1.4201369689706411, 'learning_rate': 7.149068322981367e-07, 'completion_length': 169.92858123779297, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.41211438179016113, 'kl': 0.012664794921875, 'epoch': 1.43} 29%|██▊ | 459/1610 [1:57:14<4:54:52, 15.37s/it] 29%|██▊ | 460/1610 [1:57:30<4:56:40, 15.48s/it] {'loss': 0.0005, 'grad_norm': 0.8638920680255581, 'learning_rate': 7.142857142857143e-07, 'completion_length': 174.6071548461914, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.20740239322185516, 'kl': 0.012786865234375, 'epoch': 1.43} 29%|██▊ | 460/1610 [1:57:30<4:56:40, 15.48s/it] 29%|██▊ | 461/1610 [1:57:46<4:59:30, 15.64s/it] {'loss': 0.0004, 'grad_norm': 1.8947024553320062, 'learning_rate': 7.136645962732919e-07, 'completion_length': 173.3928680419922, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.3889576643705368, 'kl': 0.0096588134765625, 'epoch': 1.43} 29%|██▊ | 461/1610 [1:57:46<4:59:30, 15.64s/it] 29%|██▊ | 462/1610 [1:58:01<4:54:54, 15.41s/it] {'loss': 0.0003, 'grad_norm': 1.9762057856604671, 'learning_rate': 7.130434782608695e-07, 'completion_length': 146.15179061889648, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.2948778420686722, 'kl': 0.008697509765625, 'epoch': 1.43} 29%|██▊ | 462/1610 [1:58:01<4:54:54, 15.41s/it] 29%|██▉ | 463/1610 [1:58:18<5:03:50, 15.89s/it] {'loss': 0.0004, 'grad_norm': 0.9717829224259511, 'learning_rate': 7.124223602484471e-07, 'completion_length': 212.05358123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9285714626312256, 'reward': 1.3392857909202576, 'reward_std': 0.33575402200222015, 'kl': 0.009674072265625, 'epoch': 1.44} 29%|██▉ | 463/1610 [1:58:18<5:03:50, 15.89s/it] 29%|██▉ | 464/1610 [1:58:31<4:49:26, 15.15s/it] {'loss': 0.0004, 'grad_norm': 2.0036643102889466, 'learning_rate': 7.118012422360247e-07, 'completion_length': 142.56250762939453, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.30659182369709015, 'kl': 0.00933837890625, 'epoch': 1.44} 29%|██▉ | 464/1610 [1:58:31<4:49:26, 15.15s/it] 29%|██▉ | 465/1610 [1:58:48<4:55:25, 15.48s/it] {'loss': 0.0005, 'grad_norm': 2.013939220994689, 'learning_rate': 7.111801242236024e-07, 'completion_length': 163.0357208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.28458838164806366, 'kl': 0.0120849609375, 'epoch': 1.44} 29%|██▉ | 465/1610 [1:58:48<4:55:25, 15.48s/it] 29%|██▉ | 466/1610 [1:59:03<4:53:47, 15.41s/it] {'loss': 0.0004, 'grad_norm': 1.647288626191533, 'learning_rate': 7.105590062111801e-07, 'completion_length': 173.33036041259766, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3240400552749634, 'kl': 0.009429931640625, 'epoch': 1.45} 29%|██▉ | 466/1610 [1:59:03<4:53:47, 15.41s/it] 29%|██▉ | 467/1610 [1:59:18<4:52:18, 15.34s/it] {'loss': 0.0005, 'grad_norm': 1.7738104924329248, 'learning_rate': 7.099378881987577e-07, 'completion_length': 164.06250762939453, 'rewards/accuracy_reward': 0.5625, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.34269505739212036, 'kl': 0.01202392578125, 'epoch': 1.45} 29%|██▉ | 467/1610 [1:59:18<4:52:18, 15.34s/it] 29%|██▉ | 468/1610 [1:59:33<4:51:27, 15.31s/it] {'loss': 0.0004, 'grad_norm': 0.8863538404347712, 'learning_rate': 7.093167701863354e-07, 'completion_length': 155.4464340209961, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.2152135968208313, 'kl': 0.009033203125, 'epoch': 1.45} 29%|██▉ | 468/1610 [1:59:33<4:51:27, 15.31s/it] 29%|██▉ | 469/1610 [1:59:49<4:55:09, 15.52s/it] {'loss': 0.0004, 'grad_norm': 5.685409642484068, 'learning_rate': 7.08695652173913e-07, 'completion_length': 171.6339340209961, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6339285969734192, 'reward_std': 0.327942818403244, 'kl': 0.01055908203125, 'epoch': 1.46} 29%|██▉ | 469/1610 [1:59:49<4:55:09, 15.52s/it] 29%|██▉ | 470/1610 [2:00:04<4:51:04, 15.32s/it] {'loss': 0.0004, 'grad_norm': 1.1997016123182596, 'learning_rate': 7.080745341614906e-07, 'completion_length': 151.83929443359375, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.30390140414237976, 'kl': 0.010284423828125, 'epoch': 1.46} 29%|██▉ | 470/1610 [2:00:04<4:51:04, 15.32s/it] 29%|██▉ | 471/1610 [2:00:19<4:47:09, 15.13s/it] {'loss': 0.0006, 'grad_norm': 1.6963572543017673, 'learning_rate': 7.074534161490683e-07, 'completion_length': 175.18750762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.4301704913377762, 'kl': 0.014495849609375, 'epoch': 1.46} 29%|██▉ | 471/1610 [2:00:19<4:47:09, 15.13s/it] 29%|██▉ | 472/1610 [2:00:32<4:36:34, 14.58s/it] {'loss': 0.0003, 'grad_norm': 1.659338347944293, 'learning_rate': 7.068322981366459e-07, 'completion_length': 121.16072082519531, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.19030197709798813, 'kl': 0.0086669921875, 'epoch': 1.47} 29%|██▉ | 472/1610 [2:00:32<4:36:34, 14.58s/it] 29%|██▉ | 473/1610 [2:00:48<4:41:20, 14.85s/it] {'loss': 0.0005, 'grad_norm': 1.3968933677592275, 'learning_rate': 7.062111801242235e-07, 'completion_length': 160.2232208251953, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.31622883677482605, 'kl': 0.0113525390625, 'epoch': 1.47} 29%|██▉ | 473/1610 [2:00:48<4:41:20, 14.85s/it] 29%|██▉ | 474/1610 [2:01:01<4:33:57, 14.47s/it] {'loss': 0.0004, 'grad_norm': 1.3325897190107046, 'learning_rate': 7.055900621118012e-07, 'completion_length': 147.66965103149414, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.31595757603645325, 'kl': 0.010101318359375, 'epoch': 1.47} 29%|██▉ | 474/1610 [2:01:01<4:33:57, 14.47s/it] 30%|██▉ | 475/1610 [2:01:16<4:36:45, 14.63s/it] {'loss': 0.0004, 'grad_norm': 1.2538071042971188, 'learning_rate': 7.049689440993789e-07, 'completion_length': 161.0714340209961, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3402702808380127, 'kl': 0.011199951171875, 'epoch': 1.48} 30%|██▉ | 475/1610 [2:01:16<4:36:45, 14.63s/it] 30%|██▉ | 476/1610 [2:01:33<4:48:30, 15.27s/it] {'loss': 0.0004, 'grad_norm': 1.5463353470153596, 'learning_rate': 7.043478260869565e-07, 'completion_length': 171.54464721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.33246469497680664, 'kl': 0.010986328125, 'epoch': 1.48} 30%|██▉ | 476/1610 [2:01:33<4:48:30, 15.27s/it] 30%|██▉ | 477/1610 [2:01:49<4:50:25, 15.38s/it] {'loss': 0.0005, 'grad_norm': 1.4303296015862799, 'learning_rate': 7.037267080745342e-07, 'completion_length': 152.48214721679688, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.3045148700475693, 'kl': 0.011566162109375, 'epoch': 1.48} 30%|██▉ | 477/1610 [2:01:49<4:50:25, 15.38s/it] 30%|██▉ | 478/1610 [2:02:04<4:48:37, 15.30s/it] {'loss': 0.0003, 'grad_norm': 5.108138384975141, 'learning_rate': 7.031055900621118e-07, 'completion_length': 160.89286041259766, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.327942818403244, 'kl': 0.0084686279296875, 'epoch': 1.48} 30%|██▉ | 478/1610 [2:02:04<4:48:37, 15.30s/it] 30%|██▉ | 479/1610 [2:02:19<4:49:05, 15.34s/it] {'loss': 0.0005, 'grad_norm': 0.9678803510428424, 'learning_rate': 7.024844720496894e-07, 'completion_length': 180.0982208251953, 'rewards/accuracy_reward': 0.5178571492433548, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.3724505603313446, 'kl': 0.01263427734375, 'epoch': 1.49} 30%|██▉ | 479/1610 [2:02:19<4:49:05, 15.34s/it] 30%|██▉ | 480/1610 [2:02:34<4:43:24, 15.05s/it] {'loss': 0.0004, 'grad_norm': 1.373277994522416, 'learning_rate': 7.018633540372671e-07, 'completion_length': 136.2232208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5089285969734192, 'reward_std': 0.34065985679626465, 'kl': 0.009033203125, 'epoch': 1.49} 30%|██▉ | 480/1610 [2:02:34<4:43:24, 15.05s/it] 30%|██▉ | 481/1610 [2:02:49<4:42:21, 15.01s/it] {'loss': 0.0004, 'grad_norm': 4.8911834984311024, 'learning_rate': 7.012422360248447e-07, 'completion_length': 161.92858123779297, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3336714655160904, 'kl': 0.010223388671875, 'epoch': 1.49} 30%|██▉ | 481/1610 [2:02:49<4:42:21, 15.01s/it] 30%|██▉ | 482/1610 [2:03:05<4:49:33, 15.40s/it] {'loss': 0.0005, 'grad_norm': 0.9204828653192065, 'learning_rate': 7.006211180124223e-07, 'completion_length': 189.22322845458984, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.22215460240840912, 'kl': 0.01141357421875, 'epoch': 1.5} 30%|██▉ | 482/1610 [2:03:05<4:49:33, 15.40s/it] 30%|███ | 483/1610 [2:03:19<4:43:30, 15.09s/it] {'loss': 0.0003, 'grad_norm': 1.1227981852429805, 'learning_rate': 7e-07, 'completion_length': 152.23214721679688, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2993997037410736, 'kl': 0.008636474609375, 'epoch': 1.5} 30%|███ | 483/1610 [2:03:19<4:43:30, 15.09s/it] 30%|███ | 484/1610 [2:03:36<4:50:17, 15.47s/it] {'loss': 0.0003, 'grad_norm': 1.0663292147795467, 'learning_rate': 6.993788819875777e-07, 'completion_length': 166.25894165039062, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.30659739673137665, 'kl': 0.008697509765625, 'epoch': 1.5} 30%|███ | 484/1610 [2:03:36<4:50:17, 15.47s/it] 30%|███ | 485/1610 [2:03:53<4:59:26, 15.97s/it] {'loss': 0.0005, 'grad_norm': 1.2757823202785512, 'learning_rate': 6.987577639751553e-07, 'completion_length': 179.3928680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.34148263931274414, 'kl': 0.0120849609375, 'epoch': 1.51} 30%|███ | 485/1610 [2:03:53<4:59:26, 15.97s/it] 30%|███ | 486/1610 [2:04:08<4:55:15, 15.76s/it] {'loss': 0.0004, 'grad_norm': 3.0423454083841706, 'learning_rate': 6.981366459627329e-07, 'completion_length': 156.62500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5535715222358704, 'reward_std': 0.38175100088119507, 'kl': 0.01068115234375, 'epoch': 1.51} 30%|███ | 486/1610 [2:04:08<4:55:15, 15.76s/it] 30%|███ | 487/1610 [2:04:24<4:54:33, 15.74s/it] {'loss': 0.0005, 'grad_norm': 1.7089453481165693, 'learning_rate': 6.975155279503105e-07, 'completion_length': 161.90179443359375, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.1890895962715149, 'kl': 0.0123291015625, 'epoch': 1.51} 30%|███ | 487/1610 [2:04:24<4:54:33, 15.74s/it] 30%|███ | 488/1610 [2:04:40<4:55:31, 15.80s/it] {'loss': 0.0004, 'grad_norm': 1.0236893769971085, 'learning_rate': 6.968944099378881e-07, 'completion_length': 187.05358123779297, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.22754104435443878, 'kl': 0.0111083984375, 'epoch': 1.52} 30%|███ | 488/1610 [2:04:40<4:55:31, 15.80s/it] 30%|███ | 489/1610 [2:04:55<4:53:13, 15.69s/it] {'loss': 0.0004, 'grad_norm': 1.7191351394822214, 'learning_rate': 6.962732919254658e-07, 'completion_length': 164.4821548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3026946187019348, 'kl': 0.010345458984375, 'epoch': 1.52} 30%|███ | 489/1610 [2:04:55<4:53:13, 15.69s/it] 30%|███ | 490/1610 [2:05:11<4:53:03, 15.70s/it] {'loss': 0.0005, 'grad_norm': 0.9940987586136681, 'learning_rate': 6.956521739130434e-07, 'completion_length': 194.90179443359375, 'rewards/accuracy_reward': 0.3392857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3303572535514832, 'reward_std': 0.3660840839147568, 'kl': 0.011322021484375, 'epoch': 1.52} 30%|███ | 490/1610 [2:05:11<4:53:03, 15.70s/it] 30%|███ | 491/1610 [2:05:26<4:52:33, 15.69s/it] {'loss': 0.0004, 'grad_norm': 0.7919015158359156, 'learning_rate': 6.95031055900621e-07, 'completion_length': 174.00894165039062, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.21972985565662384, 'kl': 0.009124755859375, 'epoch': 1.52} 30%|███ | 491/1610 [2:05:26<4:52:33, 15.69s/it] 31%|███ | 492/1610 [2:05:40<4:40:14, 15.04s/it] {'loss': 0.0004, 'grad_norm': 1.4215581930183132, 'learning_rate': 6.944099378881987e-07, 'completion_length': 137.03571701049805, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.3144029974937439, 'kl': 0.010162353515625, 'epoch': 1.53} 31%|███ | 492/1610 [2:05:40<4:40:14, 15.04s/it] 31%|███ | 493/1610 [2:05:54<4:37:03, 14.88s/it] {'loss': 0.0005, 'grad_norm': 1.3794380629145397, 'learning_rate': 6.937888198757764e-07, 'completion_length': 155.96429443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.23717808723449707, 'kl': 0.011810302734375, 'epoch': 1.53} 31%|███ | 493/1610 [2:05:54<4:37:03, 14.88s/it] 31%|███ | 494/1610 [2:06:10<4:40:41, 15.09s/it] {'loss': 0.0004, 'grad_norm': 1.1254400444274273, 'learning_rate': 6.93167701863354e-07, 'completion_length': 173.9732208251953, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3750000596046448, 'reward_std': 0.4010138660669327, 'kl': 0.01123046875, 'epoch': 1.53} 31%|███ | 494/1610 [2:06:10<4:40:41, 15.09s/it] 31%|███ | 495/1610 [2:06:26<4:44:07, 15.29s/it] {'loss': 0.0004, 'grad_norm': 1.0864541338277975, 'learning_rate': 6.925465838509317e-07, 'completion_length': 177.88394165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.33327578008174896, 'kl': 0.010589599609375, 'epoch': 1.54} 31%|███ | 495/1610 [2:06:26<4:44:07, 15.29s/it] 31%|███ | 496/1610 [2:06:41<4:44:04, 15.30s/it] {'loss': 0.0005, 'grad_norm': 1.9711500314335912, 'learning_rate': 6.919254658385093e-07, 'completion_length': 153.6607208251953, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2566976547241211, 'kl': 0.01202392578125, 'epoch': 1.54} 31%|███ | 496/1610 [2:06:41<4:44:04, 15.30s/it] 31%|███ | 497/1610 [2:06:57<4:47:45, 15.51s/it] {'loss': 0.0004, 'grad_norm': 1.7546967145695034, 'learning_rate': 6.913043478260869e-07, 'completion_length': 163.2053680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3628135025501251, 'kl': 0.009765625, 'epoch': 1.54} 31%|███ | 497/1610 [2:06:57<4:47:45, 15.51s/it] 31%|███ | 498/1610 [2:07:12<4:46:17, 15.45s/it] {'loss': 0.0004, 'grad_norm': 1.6881804395520985, 'learning_rate': 6.906832298136646e-07, 'completion_length': 132.2857208251953, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3750000596046448, 'reward_std': 0.3594707250595093, 'kl': 0.01043701171875, 'epoch': 1.55} 31%|███ | 498/1610 [2:07:12<4:46:17, 15.45s/it] 31%|███ | 499/1610 [2:07:27<4:39:16, 15.08s/it] {'loss': 0.0004, 'grad_norm': 1.1535130362617336, 'learning_rate': 6.900621118012422e-07, 'completion_length': 144.70536041259766, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.30632613599300385, 'kl': 0.010162353515625, 'epoch': 1.55} 31%|███ | 499/1610 [2:07:27<4:39:16, 15.08s/it] 31%|███ | 500/1610 [2:07:42<4:42:11, 15.25s/it] {'loss': 0.0004, 'grad_norm': 2.0640430595711114, 'learning_rate': 6.894409937888198e-07, 'completion_length': 158.9196548461914, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.2630307972431183, 'kl': 0.01123046875, 'epoch': 1.55} 31%|███ | 500/1610 [2:07:42<4:42:11, 15.25s/it] 31%|███ | 501/1610 [2:08:52<9:46:19, 31.72s/it] {'loss': 0.0004, 'grad_norm': 1.1055094043697622, 'learning_rate': 6.888198757763975e-07, 'completion_length': 152.8214340209961, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2344820350408554, 'kl': 0.01055908203125, 'epoch': 1.56} 31%|███ | 501/1610 [2:08:52<9:46:19, 31.72s/it] 31%|███ | 502/1610 [2:09:08<8:15:09, 26.81s/it] {'loss': 0.0004, 'grad_norm': 1.558720832992115, 'learning_rate': 6.881987577639752e-07, 'completion_length': 139.33929443359375, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2669336050748825, 'kl': 0.01123046875, 'epoch': 1.56} 31%|███ | 502/1610 [2:09:08<8:15:09, 26.81s/it] 31%|███ | 503/1610 [2:09:23<7:12:22, 23.44s/it] {'loss': 0.0004, 'grad_norm': 1.1952071869680587, 'learning_rate': 6.875776397515528e-07, 'completion_length': 167.8839340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.2759571820497513, 'kl': 0.0100555419921875, 'epoch': 1.56} 31%|███ | 503/1610 [2:09:23<7:12:22, 23.44s/it] 31%|███▏ | 504/1610 [2:09:39<6:26:40, 20.98s/it] {'loss': 0.0005, 'grad_norm': 0.9967671936884389, 'learning_rate': 6.869565217391305e-07, 'completion_length': 161.1339340209961, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.28437623381614685, 'kl': 0.0126953125, 'epoch': 1.57} 31%|███▏ | 504/1610 [2:09:39<6:26:40, 20.98s/it] 31%|███▏ | 505/1610 [2:09:54<5:54:39, 19.26s/it] {'loss': 0.0005, 'grad_norm': 1.5197447865968121, 'learning_rate': 6.863354037267081e-07, 'completion_length': 148.35714721679688, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.34148265421390533, 'kl': 0.012420654296875, 'epoch': 1.57} 31%|███▏ | 505/1610 [2:09:54<5:54:39, 19.26s/it] 31%|███▏ | 506/1610 [2:10:09<5:31:04, 17.99s/it] {'loss': 0.0005, 'grad_norm': 2.1719015159771864, 'learning_rate': 6.857142857142857e-07, 'completion_length': 147.02679443359375, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.33575402200222015, 'kl': 0.011962890625, 'epoch': 1.57} 31%|███▏ | 506/1610 [2:10:09<5:31:04, 17.99s/it] 31%|███▏ | 507/1610 [2:10:24<5:14:31, 17.11s/it] {'loss': 0.0005, 'grad_norm': 1.7362725642633872, 'learning_rate': 6.850931677018634e-07, 'completion_length': 144.10715103149414, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6785715222358704, 'reward_std': 0.27799517661333084, 'kl': 0.013580322265625, 'epoch': 1.57} 31%|███▏ | 507/1610 [2:10:24<5:14:31, 17.11s/it] 32%|███▏ | 508/1610 [2:10:39<5:02:23, 16.46s/it] {'loss': 0.0005, 'grad_norm': 0.848373379935529, 'learning_rate': 6.84472049689441e-07, 'completion_length': 174.36608123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.2540128380060196, 'kl': 0.012969970703125, 'epoch': 1.58} 32%|███▏ | 508/1610 [2:10:39<5:02:23, 16.46s/it] 32%|███▏ | 509/1610 [2:10:54<4:56:08, 16.14s/it] {'loss': 0.0005, 'grad_norm': 1.1377551083337798, 'learning_rate': 6.838509316770185e-07, 'completion_length': 170.9107208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.29305195808410645, 'kl': 0.01373291015625, 'epoch': 1.58} 32%|███▏ | 509/1610 [2:10:54<4:56:08, 16.14s/it] 32%|███▏ | 510/1610 [2:11:10<4:54:36, 16.07s/it] {'loss': 0.0005, 'grad_norm': 1.686357725981513, 'learning_rate': 6.832298136645962e-07, 'completion_length': 164.66964721679688, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.36552971601486206, 'kl': 0.012451171875, 'epoch': 1.58} 32%|███▏ | 510/1610 [2:11:10<4:54:36, 16.07s/it] 32%|███▏ | 511/1610 [2:11:27<4:58:05, 16.27s/it] {'loss': 0.0005, 'grad_norm': 2.3527209527546775, 'learning_rate': 6.826086956521738e-07, 'completion_length': 178.31250762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160715222358704, 'reward_std': 0.28195706009864807, 'kl': 0.01336669921875, 'epoch': 1.59} 32%|███▏ | 511/1610 [2:11:27<4:58:05, 16.27s/it] 32%|███▏ | 512/1610 [2:11:44<5:00:46, 16.44s/it] {'loss': 0.0005, 'grad_norm': 2.1783950108499304, 'learning_rate': 6.819875776397515e-07, 'completion_length': 184.9464340209961, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857313156128, 'reward_std': 0.1963018700480461, 'kl': 0.0120849609375, 'epoch': 1.59} 32%|███▏ | 512/1610 [2:11:44<5:00:46, 16.44s/it] 32%|███▏ | 513/1610 [2:12:01<5:02:29, 16.54s/it] {'loss': 0.0005, 'grad_norm': 1.2869863835842856, 'learning_rate': 6.813664596273292e-07, 'completion_length': 165.97322845458984, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.33845002949237823, 'kl': 0.011932373046875, 'epoch': 1.59} 32%|███▏ | 513/1610 [2:12:01<5:02:29, 16.54s/it] 32%|███▏ | 514/1610 [2:12:15<4:52:37, 16.02s/it] {'loss': 0.0005, 'grad_norm': 1.3509392181502669, 'learning_rate': 6.807453416149068e-07, 'completion_length': 175.32144165039062, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.3053906410932541, 'kl': 0.01202392578125, 'epoch': 1.6} 32%|███▏ | 514/1610 [2:12:15<4:52:37, 16.02s/it] 32%|███▏ | 515/1610 [2:12:31<4:49:37, 15.87s/it] {'loss': 0.0005, 'grad_norm': 0.9899182653823568, 'learning_rate': 6.801242236024844e-07, 'completion_length': 153.46428680419922, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.23386859148740768, 'kl': 0.01153564453125, 'epoch': 1.6} 32%|███▏ | 515/1610 [2:12:31<4:49:37, 15.87s/it] 32%|███▏ | 516/1610 [2:12:46<4:43:09, 15.53s/it] {'loss': 0.0005, 'grad_norm': 2.1647332525736998, 'learning_rate': 6.795031055900621e-07, 'completion_length': 147.7589340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160715222358704, 'reward_std': 0.3084232658147812, 'kl': 0.012969970703125, 'epoch': 1.6} 32%|███▏ | 516/1610 [2:12:46<4:43:09, 15.53s/it] 32%|███▏ | 517/1610 [2:13:01<4:41:43, 15.46s/it] {'loss': 0.0005, 'grad_norm': 1.2295569538890034, 'learning_rate': 6.788819875776397e-07, 'completion_length': 160.6696548461914, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2579156309366226, 'kl': 0.012298583984375, 'epoch': 1.61} 32%|███▏ | 517/1610 [2:13:01<4:41:43, 15.46s/it] 32%|███▏ | 518/1610 [2:13:17<4:43:25, 15.57s/it] {'loss': 0.0006, 'grad_norm': 1.3491067416475786, 'learning_rate': 6.782608695652173e-07, 'completion_length': 175.24107360839844, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3751465827226639, 'kl': 0.014373779296875, 'epoch': 1.61} 32%|███▏ | 518/1610 [2:13:17<4:43:25, 15.57s/it] 32%|███▏ | 519/1610 [2:13:32<4:42:39, 15.54s/it] {'loss': 0.0004, 'grad_norm': 1.2427308941549902, 'learning_rate': 6.77639751552795e-07, 'completion_length': 164.45536041259766, 'rewards/accuracy_reward': 0.4464286118745804, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.3090311288833618, 'kl': 0.011199951171875, 'epoch': 1.61} 32%|███▏ | 519/1610 [2:13:32<4:42:39, 15.54s/it] 32%|███▏ | 520/1610 [2:13:49<4:47:55, 15.85s/it] {'loss': 0.0005, 'grad_norm': 1.3450461261693032, 'learning_rate': 6.770186335403726e-07, 'completion_length': 202.65179443359375, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.3295391798019409, 'kl': 0.013336181640625, 'epoch': 1.61} 32%|███▏ | 520/1610 [2:13:49<4:47:55, 15.85s/it] 32%|███▏ | 521/1610 [2:14:05<4:49:51, 15.97s/it] {'loss': 0.0004, 'grad_norm': 4.102012522831084, 'learning_rate': 6.763975155279503e-07, 'completion_length': 158.9464340209961, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.3330579847097397, 'kl': 0.0107421875, 'epoch': 1.62} 32%|███▏ | 521/1610 [2:14:05<4:49:51, 15.97s/it] 32%|███▏ | 522/1610 [2:14:19<4:39:55, 15.44s/it] {'loss': 0.0004, 'grad_norm': 1.3616515581386834, 'learning_rate': 6.75776397515528e-07, 'completion_length': 150.36607360839844, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3156209737062454, 'kl': 0.0093994140625, 'epoch': 1.62} 32%|███▏ | 522/1610 [2:14:19<4:39:55, 15.44s/it] 32%|███▏ | 523/1610 [2:14:35<4:40:54, 15.51s/it] {'loss': 0.0005, 'grad_norm': 1.2138458482465548, 'learning_rate': 6.751552795031056e-07, 'completion_length': 176.24107360839844, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.3919757306575775, 'kl': 0.013092041015625, 'epoch': 1.62} 32%|███▏ | 523/1610 [2:14:35<4:40:54, 15.51s/it] 33%|███▎ | 524/1610 [2:14:51<4:41:25, 15.55s/it] {'loss': 0.0005, 'grad_norm': 1.69785508264712, 'learning_rate': 6.745341614906832e-07, 'completion_length': 176.7678680419922, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.27414587140083313, 'kl': 0.011993408203125, 'epoch': 1.63} 33%|███▎ | 524/1610 [2:14:51<4:41:25, 15.55s/it] 33%|███▎ | 525/1610 [2:15:07<4:44:54, 15.75s/it] {'loss': 0.0005, 'grad_norm': 0.973984660326546, 'learning_rate': 6.739130434782609e-07, 'completion_length': 179.38394165039062, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.27805427461862564, 'kl': 0.012786865234375, 'epoch': 1.63} 33%|███▎ | 525/1610 [2:15:07<4:44:54, 15.75s/it] 33%|███▎ | 526/1610 [2:15:23<4:48:38, 15.98s/it] {'loss': 0.0005, 'grad_norm': 1.411271460562836, 'learning_rate': 6.732919254658385e-07, 'completion_length': 161.56250762939453, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6696429252624512, 'reward_std': 0.35825833678245544, 'kl': 0.0118408203125, 'epoch': 1.63} 33%|███▎ | 526/1610 [2:15:23<4:48:38, 15.98s/it] 33%|███▎ | 527/1610 [2:15:39<4:45:10, 15.80s/it] {'loss': 0.0005, 'grad_norm': 1.1288254341902806, 'learning_rate': 6.726708074534161e-07, 'completion_length': 168.90179443359375, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.30061207711696625, 'kl': 0.012298583984375, 'epoch': 1.64} 33%|███▎ | 527/1610 [2:15:39<4:45:10, 15.80s/it] 33%|███▎ | 528/1610 [2:15:53<4:39:21, 15.49s/it] {'loss': 0.0005, 'grad_norm': 1.279245068121506, 'learning_rate': 6.720496894409938e-07, 'completion_length': 155.1607208251953, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.23717808723449707, 'kl': 0.01165771484375, 'epoch': 1.64} 33%|███▎ | 528/1610 [2:15:53<4:39:21, 15.49s/it] 33%|███▎ | 529/1610 [2:16:09<4:37:26, 15.40s/it] {'loss': 0.0004, 'grad_norm': 1.5921741975957286, 'learning_rate': 6.714285714285714e-07, 'completion_length': 156.49108123779297, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.3408747911453247, 'kl': 0.0096435546875, 'epoch': 1.64} 33%|███▎ | 529/1610 [2:16:09<4:37:26, 15.40s/it] 33%|███▎ | 530/1610 [2:16:24<4:37:59, 15.44s/it] {'loss': 0.0005, 'grad_norm': 0.8490243542153099, 'learning_rate': 6.708074534161491e-07, 'completion_length': 198.15179443359375, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2630307972431183, 'kl': 0.011627197265625, 'epoch': 1.65} 33%|███▎ | 530/1610 [2:16:24<4:37:59, 15.44s/it] 33%|███▎ | 531/1610 [2:16:37<4:25:12, 14.75s/it] {'loss': 0.0004, 'grad_norm': 1.6535052400172667, 'learning_rate': 6.701863354037268e-07, 'completion_length': 122.95536041259766, 'rewards/accuracy_reward': 0.473214328289032, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.3396568149328232, 'kl': 0.010009765625, 'epoch': 1.65} 33%|███▎ | 531/1610 [2:16:37<4:25:12, 14.75s/it] 33%|███▎ | 532/1610 [2:16:52<4:26:20, 14.82s/it] {'loss': 0.0004, 'grad_norm': 1.1105903073921384, 'learning_rate': 6.695652173913044e-07, 'completion_length': 168.62500762939453, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.31292495131492615, 'kl': 0.01104736328125, 'epoch': 1.65} 33%|███▎ | 532/1610 [2:16:52<4:26:20, 14.82s/it] 33%|███▎ | 533/1610 [2:17:09<4:36:31, 15.41s/it] {'loss': 0.0005, 'grad_norm': 1.4797813449562094, 'learning_rate': 6.689440993788819e-07, 'completion_length': 190.95536041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.25791002810001373, 'kl': 0.01251220703125, 'epoch': 1.66} 33%|███▎ | 533/1610 [2:17:09<4:36:31, 15.41s/it] 33%|███▎ | 534/1610 [2:17:23<4:27:57, 14.94s/it] {'loss': 0.0004, 'grad_norm': 2.56371910919142, 'learning_rate': 6.683229813664595e-07, 'completion_length': 136.80358123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.33575402200222015, 'kl': 0.01116943359375, 'epoch': 1.66} 33%|███▎ | 534/1610 [2:17:23<4:27:57, 14.94s/it] 33%|███▎ | 535/1610 [2:17:37<4:25:14, 14.80s/it] {'loss': 0.0005, 'grad_norm': 1.1538884039243615, 'learning_rate': 6.677018633540372e-07, 'completion_length': 153.46428680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.3144030123949051, 'kl': 0.01165771484375, 'epoch': 1.66} 33%|███▎ | 535/1610 [2:17:37<4:25:14, 14.80s/it] 33%|███▎ | 536/1610 [2:17:54<4:32:14, 15.21s/it] {'loss': 0.0005, 'grad_norm': 1.6746001540106918, 'learning_rate': 6.670807453416148e-07, 'completion_length': 165.9821548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.2811286598443985, 'kl': 0.01287841796875, 'epoch': 1.66} 33%|███▎ | 536/1610 [2:17:54<4:32:14, 15.21s/it] 33%|███▎ | 537/1610 [2:18:09<4:35:08, 15.39s/it] {'loss': 0.0005, 'grad_norm': 1.3934262796243069, 'learning_rate': 6.664596273291924e-07, 'completion_length': 189.99108123779297, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.2624262869358063, 'kl': 0.011688232421875, 'epoch': 1.67} 33%|███▎ | 537/1610 [2:18:09<4:35:08, 15.39s/it] 33%|███▎ | 538/1610 [2:18:25<4:34:06, 15.34s/it] {'loss': 0.0005, 'grad_norm': 1.9119699137086352, 'learning_rate': 6.658385093167701e-07, 'completion_length': 152.67858123779297, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.27414587140083313, 'kl': 0.011444091796875, 'epoch': 1.67} 33%|███▎ | 538/1610 [2:18:25<4:34:06, 15.34s/it] 33%|███▎ | 539/1610 [2:18:40<4:31:48, 15.23s/it] {'loss': 0.0005, 'grad_norm': 1.9773353205641333, 'learning_rate': 6.652173913043478e-07, 'completion_length': 162.00000762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.27144984900951385, 'kl': 0.013153076171875, 'epoch': 1.67} 33%|███▎ | 539/1610 [2:18:40<4:31:48, 15.23s/it] 34%|███▎ | 540/1610 [2:18:55<4:34:27, 15.39s/it] {'loss': 0.0006, 'grad_norm': 2.066168666293431, 'learning_rate': 6.645962732919254e-07, 'completion_length': 169.51786041259766, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.32915520668029785, 'kl': 0.013885498046875, 'epoch': 1.68} 34%|███▎ | 540/1610 [2:18:55<4:34:27, 15.39s/it] 34%|███▎ | 541/1610 [2:19:12<4:38:42, 15.64s/it] {'loss': 0.0005, 'grad_norm': 4.288590683203209, 'learning_rate': 6.639751552795031e-07, 'completion_length': 162.3928680419922, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.3141373544931412, 'kl': 0.0133056640625, 'epoch': 1.68} 34%|███▎ | 541/1610 [2:19:12<4:38:42, 15.64s/it] 34%|███▎ | 542/1610 [2:19:27<4:36:45, 15.55s/it] {'loss': 0.0005, 'grad_norm': 2.07085999121382, 'learning_rate': 6.633540372670807e-07, 'completion_length': 173.7232208251953, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.39341484010219574, 'kl': 0.013275146484375, 'epoch': 1.68} 34%|███▎ | 542/1610 [2:19:27<4:36:45, 15.55s/it] 34%|███▎ | 543/1610 [2:19:42<4:34:21, 15.43s/it] {'loss': 0.0005, 'grad_norm': 1.8815824587204193, 'learning_rate': 6.627329192546583e-07, 'completion_length': 153.9732208251953, 'rewards/accuracy_reward': 0.196428582072258, 'rewards/format_reward': 1.0, 'reward': 1.196428656578064, 'reward_std': 0.24350562691688538, 'kl': 0.012176513671875, 'epoch': 1.69} 34%|███▎ | 543/1610 [2:19:42<4:34:21, 15.43s/it] 34%|███▍ | 544/1610 [2:19:58<4:36:08, 15.54s/it] {'loss': 0.0004, 'grad_norm': 0.9910399773902235, 'learning_rate': 6.62111801242236e-07, 'completion_length': 134.18750762939453, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2741458863019943, 'kl': 0.01031494140625, 'epoch': 1.69} 34%|███▍ | 544/1610 [2:19:58<4:36:08, 15.54s/it] 34%|███▍ | 545/1610 [2:20:12<4:30:23, 15.23s/it] {'loss': 0.0004, 'grad_norm': 4.558464424882123, 'learning_rate': 6.614906832298136e-07, 'completion_length': 134.05357360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3129395395517349, 'kl': 0.010009765625, 'epoch': 1.69} 34%|███▍ | 545/1610 [2:20:12<4:30:23, 15.23s/it] 34%|███▍ | 546/1610 [2:20:25<4:15:51, 14.43s/it] {'loss': 0.0004, 'grad_norm': 2.1976682079776286, 'learning_rate': 6.608695652173912e-07, 'completion_length': 125.37500762939453, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.36073657870292664, 'kl': 0.01068115234375, 'epoch': 1.7} 34%|███▍ | 546/1610 [2:20:25<4:15:51, 14.43s/it] 34%|███▍ | 547/1610 [2:20:40<4:20:21, 14.70s/it] {'loss': 0.0005, 'grad_norm': 1.8305878110721943, 'learning_rate': 6.602484472049689e-07, 'completion_length': 167.4732208251953, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2741458863019943, 'kl': 0.013336181640625, 'epoch': 1.7} 34%|███▍ | 547/1610 [2:20:40<4:20:21, 14.70s/it] 34%|███▍ | 548/1610 [2:20:55<4:22:17, 14.82s/it] {'loss': 0.0005, 'grad_norm': 1.3713362299006102, 'learning_rate': 6.596273291925466e-07, 'completion_length': 161.60714721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.30659739673137665, 'kl': 0.012969970703125, 'epoch': 1.7} 34%|███▍ | 548/1610 [2:20:55<4:22:17, 14.82s/it] 34%|███▍ | 549/1610 [2:21:11<4:26:20, 15.06s/it] {'loss': 0.0005, 'grad_norm': 5.3474880660300474, 'learning_rate': 6.590062111801242e-07, 'completion_length': 182.0982208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.21251757442951202, 'kl': 0.013153076171875, 'epoch': 1.7} 34%|███▍ | 549/1610 [2:21:11<4:26:20, 15.06s/it] 34%|███▍ | 550/1610 [2:21:27<4:33:46, 15.50s/it] {'loss': 0.0005, 'grad_norm': 1.2902590858978704, 'learning_rate': 6.583850931677019e-07, 'completion_length': 171.9553680419922, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.23326969146728516, 'kl': 0.01214599609375, 'epoch': 1.71} 34%|███▍ | 550/1610 [2:21:27<4:33:46, 15.50s/it] 34%|███▍ | 551/1610 [2:21:42<4:26:58, 15.13s/it] {'loss': 0.0004, 'grad_norm': 1.1702482470614792, 'learning_rate': 6.577639751552795e-07, 'completion_length': 144.2053680419922, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.723214328289032, 'reward_std': 0.31353843212127686, 'kl': 0.010498046875, 'epoch': 1.71} 34%|███▍ | 551/1610 [2:21:42<4:26:58, 15.13s/it] 34%|███▍ | 552/1610 [2:21:57<4:28:04, 15.20s/it] {'loss': 0.0005, 'grad_norm': 2.9409417625070646, 'learning_rate': 6.571428571428571e-07, 'completion_length': 179.50001525878906, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.633928656578064, 'reward_std': 0.3597419559955597, 'kl': 0.01171875, 'epoch': 1.71} 34%|███▍ | 552/1610 [2:21:57<4:28:04, 15.20s/it] 34%|███▍ | 553/1610 [2:22:13<4:31:04, 15.39s/it] {'loss': 0.0003, 'grad_norm': 1.7851606718434754, 'learning_rate': 6.565217391304348e-07, 'completion_length': 123.14286422729492, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.2960902154445648, 'kl': 0.0078277587890625, 'epoch': 1.72} 34%|███▍ | 553/1610 [2:22:13<4:31:04, 15.39s/it] 34%|███▍ | 554/1610 [2:22:28<4:29:52, 15.33s/it] {'loss': 0.0005, 'grad_norm': 0.7865148198993528, 'learning_rate': 6.559006211180124e-07, 'completion_length': 169.56250762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.20349960774183273, 'kl': 0.0125732421875, 'epoch': 1.72} 34%|███▍ | 554/1610 [2:22:28<4:29:52, 15.33s/it] 34%|███▍ | 555/1610 [2:22:43<4:27:00, 15.19s/it] {'loss': 0.0005, 'grad_norm': 1.6475580444460747, 'learning_rate': 6.5527950310559e-07, 'completion_length': 174.67858123779297, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3318512290716171, 'kl': 0.01324462890625, 'epoch': 1.72} 34%|███▍ | 555/1610 [2:22:43<4:27:00, 15.19s/it] 35%|███▍ | 556/1610 [2:22:57<4:23:14, 14.99s/it] {'loss': 0.0004, 'grad_norm': 1.590281563330999, 'learning_rate': 6.546583850931676e-07, 'completion_length': 140.96429061889648, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.28047341853380203, 'kl': 0.010498046875, 'epoch': 1.73} 35%|███▍ | 556/1610 [2:22:57<4:23:14, 14.99s/it] 35%|███▍ | 557/1610 [2:23:11<4:17:41, 14.68s/it] {'loss': 0.0004, 'grad_norm': 2.0676286572372486, 'learning_rate': 6.540372670807453e-07, 'completion_length': 139.66965103149414, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.260606050491333, 'kl': 0.010101318359375, 'epoch': 1.73} 35%|███▍ | 557/1610 [2:23:11<4:17:41, 14.68s/it] 35%|███▍ | 558/1610 [2:23:25<4:13:57, 14.48s/it] {'loss': 0.0004, 'grad_norm': 3.252975128506961, 'learning_rate': 6.534161490683229e-07, 'completion_length': 160.50893783569336, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.36734098196029663, 'kl': 0.010986328125, 'epoch': 1.73} 35%|███▍ | 558/1610 [2:23:25<4:13:57, 14.48s/it] 35%|███▍ | 559/1610 [2:23:42<4:25:13, 15.14s/it] {'loss': 0.0005, 'grad_norm': 2.304989737281284, 'learning_rate': 6.527950310559006e-07, 'completion_length': 175.27679443359375, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.5267857909202576, 'reward_std': 0.3558926582336426, 'kl': 0.011932373046875, 'epoch': 1.74} 35%|███▍ | 559/1610 [2:23:42<4:25:13, 15.14s/it] 35%|███▍ | 560/1610 [2:23:57<4:23:24, 15.05s/it] {'loss': 0.0004, 'grad_norm': 5.69780643049361, 'learning_rate': 6.521739130434782e-07, 'completion_length': 147.5982208251953, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.3318512290716171, 'kl': 0.008758544921875, 'epoch': 1.74} 35%|███▍ | 560/1610 [2:23:57<4:23:24, 15.05s/it] 35%|███▍ | 561/1610 [2:24:13<4:30:11, 15.45s/it] {'loss': 0.0006, 'grad_norm': 1.4018578059431444, 'learning_rate': 6.515527950310558e-07, 'completion_length': 194.83929443359375, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3381787836551666, 'kl': 0.014434814453125, 'epoch': 1.74} 35%|███▍ | 561/1610 [2:24:13<4:30:11, 15.45s/it] 35%|███▍ | 562/1610 [2:24:28<4:27:35, 15.32s/it] {'loss': 0.0005, 'grad_norm': 1.0377090349964533, 'learning_rate': 6.509316770186335e-07, 'completion_length': 153.5357208251953, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.27804866433143616, 'kl': 0.01287841796875, 'epoch': 1.75} 35%|███▍ | 562/1610 [2:24:28<4:27:35, 15.32s/it] 35%|███▍ | 563/1610 [2:24:42<4:20:24, 14.92s/it] {'loss': 0.0005, 'grad_norm': 1.3666053991039848, 'learning_rate': 6.503105590062111e-07, 'completion_length': 143.93750762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.34208156168460846, 'kl': 0.011260986328125, 'epoch': 1.75} 35%|███▍ | 563/1610 [2:24:42<4:20:24, 14.92s/it] 35%|███▌ | 564/1610 [2:24:58<4:23:22, 15.11s/it] {'loss': 0.0004, 'grad_norm': 1.5245855057695938, 'learning_rate': 6.496894409937887e-07, 'completion_length': 143.60714721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2720487490296364, 'kl': 0.010101318359375, 'epoch': 1.75} 35%|███▌ | 564/1610 [2:24:58<4:23:22, 15.11s/it] 35%|███▌ | 565/1610 [2:25:13<4:21:33, 15.02s/it] {'loss': 0.0004, 'grad_norm': 1.8392716303099805, 'learning_rate': 6.490683229813664e-07, 'completion_length': 154.2946548461914, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.346597820520401, 'kl': 0.00982666015625, 'epoch': 1.75} 35%|███▌ | 565/1610 [2:25:13<4:21:33, 15.02s/it] 35%|███▌ | 566/1610 [2:25:26<4:14:15, 14.61s/it] {'loss': 0.0004, 'grad_norm': 1.4266101058343992, 'learning_rate': 6.484472049689441e-07, 'completion_length': 136.0446548461914, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.26814596354961395, 'kl': 0.00927734375, 'epoch': 1.76} 35%|███▌ | 566/1610 [2:25:26<4:14:15, 14.61s/it] 35%|███▌ | 567/1610 [2:25:42<4:21:13, 15.03s/it] {'loss': 0.0006, 'grad_norm': 2.16828276436315, 'learning_rate': 6.478260869565217e-07, 'completion_length': 159.6339340209961, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.26841723918914795, 'kl': 0.0140380859375, 'epoch': 1.76} 35%|███▌ | 567/1610 [2:25:42<4:21:13, 15.03s/it] 35%|███▌ | 568/1610 [2:26:00<4:35:16, 15.85s/it] {'loss': 0.0004, 'grad_norm': 1.5890183961922641, 'learning_rate': 6.472049689440994e-07, 'completion_length': 180.1964340209961, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3402702808380127, 'kl': 0.010650634765625, 'epoch': 1.76} 35%|███▌ | 568/1610 [2:26:00<4:35:16, 15.85s/it] 35%|███▌ | 569/1610 [2:26:14<4:23:06, 15.16s/it] {'loss': 0.0004, 'grad_norm': 1.217178173701061, 'learning_rate': 6.46583850931677e-07, 'completion_length': 128.38393020629883, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.19178561493754387, 'kl': 0.0108642578125, 'epoch': 1.77} 35%|███▌ | 569/1610 [2:26:14<4:23:06, 15.16s/it] 35%|███▌ | 570/1610 [2:26:29<4:25:39, 15.33s/it] {'loss': 0.0005, 'grad_norm': 1.1212398913027626, 'learning_rate': 6.459627329192546e-07, 'completion_length': 159.68750762939453, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.31292496621608734, 'kl': 0.012603759765625, 'epoch': 1.77} 35%|███▌ | 570/1610 [2:26:29<4:25:39, 15.33s/it] 35%|███▌ | 571/1610 [2:26:45<4:27:33, 15.45s/it] {'loss': 0.0005, 'grad_norm': 1.3231880518571906, 'learning_rate': 6.453416149068323e-07, 'completion_length': 163.15179443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178572535514832, 'reward_std': 0.31149764358997345, 'kl': 0.01141357421875, 'epoch': 1.77} 35%|███▌ | 571/1610 [2:26:45<4:27:33, 15.45s/it] 36%|███▌ | 572/1610 [2:27:03<4:37:11, 16.02s/it] {'loss': 0.0006, 'grad_norm': 1.8916642052610475, 'learning_rate': 6.447204968944099e-07, 'completion_length': 180.03572845458984, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.4123067408800125, 'kl': 0.014984130859375, 'epoch': 1.78} 36%|███▌ | 572/1610 [2:27:03<4:37:11, 16.02s/it] 36%|███▌ | 573/1610 [2:27:17<4:31:20, 15.70s/it] {'loss': 0.0005, 'grad_norm': 2.000046992953413, 'learning_rate': 6.440993788819875e-07, 'completion_length': 145.83929443359375, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.29366543889045715, 'kl': 0.012603759765625, 'epoch': 1.78} 36%|███▌ | 573/1610 [2:27:17<4:31:20, 15.70s/it] 36%|███▌ | 574/1610 [2:27:34<4:36:45, 16.03s/it] {'loss': 0.0005, 'grad_norm': 1.2524149561754672, 'learning_rate': 6.434782608695652e-07, 'completion_length': 212.4732208251953, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660714626312256, 'reward_std': 0.21434342116117477, 'kl': 0.013275146484375, 'epoch': 1.78} 36%|███▌ | 574/1610 [2:27:34<4:36:45, 16.03s/it] 36%|███▌ | 575/1610 [2:27:51<4:37:39, 16.10s/it] {'loss': 0.0005, 'grad_norm': 1.7678050107200465, 'learning_rate': 6.428571428571429e-07, 'completion_length': 154.79464721679688, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.22833791375160217, 'kl': 0.013336181640625, 'epoch': 1.79} 36%|███▌ | 575/1610 [2:27:51<4:37:39, 16.10s/it] 36%|███▌ | 576/1610 [2:28:06<4:35:30, 15.99s/it] {'loss': 0.0005, 'grad_norm': 1.158120566180329, 'learning_rate': 6.422360248447205e-07, 'completion_length': 154.2232208251953, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.25791002810001373, 'kl': 0.012603759765625, 'epoch': 1.79} 36%|███▌ | 576/1610 [2:28:06<4:35:30, 15.99s/it] 36%|███▌ | 577/1610 [2:28:23<4:41:25, 16.35s/it] {'loss': 0.0006, 'grad_norm': 2.9016135819875704, 'learning_rate': 6.416149068322982e-07, 'completion_length': 205.4553680419922, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910714626312256, 'reward_std': 0.35434994101524353, 'kl': 0.015411376953125, 'epoch': 1.79} 36%|███▌ | 577/1610 [2:28:23<4:41:25, 16.35s/it] 36%|███▌ | 578/1610 [2:28:38<4:30:50, 15.75s/it] {'loss': 0.0004, 'grad_norm': 1.50086964313529, 'learning_rate': 6.409937888198758e-07, 'completion_length': 146.97322463989258, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3111136853694916, 'kl': 0.010772705078125, 'epoch': 1.8} 36%|███▌ | 578/1610 [2:28:38<4:30:50, 15.75s/it] 36%|███▌ | 579/1610 [2:28:52<4:22:54, 15.30s/it] {'loss': 0.0004, 'grad_norm': 1.4007129284764641, 'learning_rate': 6.403726708074534e-07, 'completion_length': 148.4732208251953, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.31353843212127686, 'kl': 0.010528564453125, 'epoch': 1.8} 36%|███▌ | 579/1610 [2:28:52<4:22:54, 15.30s/it] 36%|███▌ | 580/1610 [2:29:09<4:29:14, 15.68s/it] {'loss': 0.0005, 'grad_norm': 1.2424646237655936, 'learning_rate': 6.39751552795031e-07, 'completion_length': 177.30357360839844, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.24350564181804657, 'kl': 0.0123291015625, 'epoch': 1.8} 36%|███▌ | 580/1610 [2:29:09<4:29:14, 15.68s/it] 36%|███▌ | 581/1610 [2:29:22<4:18:39, 15.08s/it] {'loss': 0.0005, 'grad_norm': 1.3355040091488704, 'learning_rate': 6.391304347826086e-07, 'completion_length': 146.1607208251953, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.321428656578064, 'reward_std': 0.2410808727145195, 'kl': 0.012939453125, 'epoch': 1.8} 36%|███▌ | 581/1610 [2:29:22<4:18:39, 15.08s/it] 36%|███▌ | 582/1610 [2:29:39<4:25:27, 15.49s/it] {'loss': 0.0005, 'grad_norm': 1.0006108247186778, 'learning_rate': 6.385093167701862e-07, 'completion_length': 182.73214721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910714626312256, 'reward_std': 0.2831638306379318, 'kl': 0.012542724609375, 'epoch': 1.81} 36%|███▌ | 582/1610 [2:29:39<4:25:27, 15.49s/it] 36%|███▌ | 583/1610 [2:29:54<4:26:01, 15.54s/it] {'loss': 0.0005, 'grad_norm': 1.2530353402107897, 'learning_rate': 6.378881987577639e-07, 'completion_length': 175.8303680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3201316297054291, 'kl': 0.01251220703125, 'epoch': 1.81} 36%|███▌ | 583/1610 [2:29:54<4:26:01, 15.54s/it] 36%|███▋ | 584/1610 [2:30:09<4:20:04, 15.21s/it] {'loss': 0.0004, 'grad_norm': 1.4678737828964665, 'learning_rate': 6.372670807453416e-07, 'completion_length': 139.75000762939453, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2960902079939842, 'kl': 0.011199951171875, 'epoch': 1.81} 36%|███▋ | 584/1610 [2:30:09<4:20:04, 15.21s/it] 36%|███▋ | 585/1610 [2:30:25<4:26:21, 15.59s/it] {'loss': 0.0006, 'grad_norm': 1.2197798420414758, 'learning_rate': 6.366459627329192e-07, 'completion_length': 173.61608123779297, 'rewards/accuracy_reward': 0.7053571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.3817453682422638, 'kl': 0.015045166015625, 'epoch': 1.82} 36%|███▋ | 585/1610 [2:30:25<4:26:21, 15.59s/it] 36%|███▋ | 586/1610 [2:30:41<4:28:51, 15.75s/it] {'loss': 0.0004, 'grad_norm': 1.2468285358340712, 'learning_rate': 6.360248447204969e-07, 'completion_length': 144.14286041259766, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4017857313156128, 'reward_std': 0.2254640907049179, 'kl': 0.010009765625, 'epoch': 1.82} 36%|███▋ | 586/1610 [2:30:41<4:28:51, 15.75s/it] 36%|███▋ | 587/1610 [2:30:57<4:27:18, 15.68s/it] {'loss': 0.0005, 'grad_norm': 3.1826949487705796, 'learning_rate': 6.354037267080745e-07, 'completion_length': 146.5089340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.17885926365852356, 'kl': 0.012939453125, 'epoch': 1.82} 36%|███▋ | 587/1610 [2:30:57<4:27:18, 15.68s/it] 37%|███▋ | 588/1610 [2:31:13<4:28:11, 15.75s/it] {'loss': 0.0005, 'grad_norm': 1.8748608378051805, 'learning_rate': 6.347826086956521e-07, 'completion_length': 146.99108123779297, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.27926105260849, 'kl': 0.0128173828125, 'epoch': 1.83} 37%|███▋ | 588/1610 [2:31:13<4:28:11, 15.75s/it] 37%|███▋ | 589/1610 [2:31:28<4:26:17, 15.65s/it] {'loss': 0.0006, 'grad_norm': 2.5885107835350705, 'learning_rate': 6.341614906832298e-07, 'completion_length': 166.5357208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.23057925701141357, 'kl': 0.014373779296875, 'epoch': 1.83} 37%|███▋ | 589/1610 [2:31:28<4:26:17, 15.65s/it] 37%|███▋ | 590/1610 [2:31:44<4:27:31, 15.74s/it] {'loss': 0.0005, 'grad_norm': 1.4239684516932327, 'learning_rate': 6.335403726708074e-07, 'completion_length': 157.6964340209961, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.3127100318670273, 'kl': 0.01348876953125, 'epoch': 1.83} 37%|███▋ | 590/1610 [2:31:44<4:27:31, 15.74s/it] 37%|███▋ | 591/1610 [2:32:00<4:28:48, 15.83s/it] {'loss': 0.0005, 'grad_norm': 1.6918829401631008, 'learning_rate': 6.32919254658385e-07, 'completion_length': 154.31250762939453, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.32282765209674835, 'kl': 0.01202392578125, 'epoch': 1.84} 37%|███▋ | 591/1610 [2:32:00<4:28:48, 15.83s/it] 37%|███▋ | 592/1610 [2:32:17<4:32:36, 16.07s/it] {'loss': 0.0005, 'grad_norm': 3.2559224483596196, 'learning_rate': 6.322981366459627e-07, 'completion_length': 167.52679443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2735324054956436, 'kl': 0.013214111328125, 'epoch': 1.84} 37%|███▋ | 592/1610 [2:32:17<4:32:36, 16.07s/it] 37%|███▋ | 593/1610 [2:32:33<4:33:12, 16.12s/it] {'loss': 0.0005, 'grad_norm': 1.726305316339455, 'learning_rate': 6.316770186335404e-07, 'completion_length': 155.50000762939453, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.39407283067703247, 'kl': 0.013031005859375, 'epoch': 1.84} 37%|███▋ | 593/1610 [2:32:33<4:33:12, 16.12s/it] 37%|███▋ | 594/1610 [2:32:49<4:29:32, 15.92s/it] {'loss': 0.0004, 'grad_norm': 1.5195781758312348, 'learning_rate': 6.31055900621118e-07, 'completion_length': 155.25000762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.598214328289032, 'reward_std': 0.31232044100761414, 'kl': 0.0093841552734375, 'epoch': 1.84} 37%|███▋ | 594/1610 [2:32:49<4:29:32, 15.92s/it] 37%|███▋ | 595/1610 [2:33:05<4:29:35, 15.94s/it] {'loss': 0.0005, 'grad_norm': 1.1793604800983128, 'learning_rate': 6.304347826086957e-07, 'completion_length': 178.02679443359375, 'rewards/accuracy_reward': 0.4464286118745804, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.4196429252624512, 'reward_std': 0.24889206886291504, 'kl': 0.0128173828125, 'epoch': 1.85} 37%|███▋ | 595/1610 [2:33:05<4:29:35, 15.94s/it] 37%|███▋ | 596/1610 [2:33:19<4:19:40, 15.37s/it] {'loss': 0.0004, 'grad_norm': 1.5044879747838864, 'learning_rate': 6.298136645962733e-07, 'completion_length': 137.73214721679688, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2915884852409363, 'kl': 0.011199951171875, 'epoch': 1.85} 37%|███▋ | 596/1610 [2:33:19<4:19:40, 15.37s/it] 37%|███▋ | 597/1610 [2:33:34<4:17:22, 15.24s/it] {'loss': 0.0006, 'grad_norm': 1.214551143566438, 'learning_rate': 6.291925465838509e-07, 'completion_length': 142.02679443359375, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6785714626312256, 'reward_std': 0.2566441595554352, 'kl': 0.014068603515625, 'epoch': 1.85} 37%|███▋ | 597/1610 [2:33:34<4:17:22, 15.24s/it] 37%|███▋ | 598/1610 [2:33:48<4:15:27, 15.15s/it] {'loss': 0.0005, 'grad_norm': 2.0760956062777183, 'learning_rate': 6.285714285714286e-07, 'completion_length': 155.63393783569336, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.2897626459598541, 'kl': 0.01177978515625, 'epoch': 1.86} 37%|███▋ | 598/1610 [2:33:48<4:15:27, 15.15s/it] 37%|███▋ | 599/1610 [2:34:02<4:04:41, 14.52s/it] {'loss': 0.0004, 'grad_norm': 1.5169989813072255, 'learning_rate': 6.279503105590062e-07, 'completion_length': 126.23214721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.260606050491333, 'kl': 0.010223388671875, 'epoch': 1.86} 37%|███▋ | 599/1610 [2:34:02<4:04:41, 14.52s/it] 37%|███▋ | 600/1610 [2:34:18<4:13:46, 15.08s/it] {'loss': 0.0006, 'grad_norm': 1.5421892787910199, 'learning_rate': 6.273291925465838e-07, 'completion_length': 186.36607360839844, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2858598679304123, 'kl': 0.0137939453125, 'epoch': 1.86} 37%|███▋ | 600/1610 [2:34:18<4:13:46, 15.08s/it] 37%|███▋ | 601/1610 [2:35:29<8:53:35, 31.73s/it] {'loss': 0.0004, 'grad_norm': 1.4760082991159975, 'learning_rate': 6.267080745341615e-07, 'completion_length': 156.2232208251953, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.7053572535514832, 'reward_std': 0.283960722386837, 'kl': 0.0107421875, 'epoch': 1.87} 37%|███▋ | 601/1610 [2:35:29<8:53:35, 31.73s/it] 37%|███▋ | 602/1610 [2:35:43<7:26:35, 26.58s/it] {'loss': 0.0005, 'grad_norm': 2.266505654123416, 'learning_rate': 6.260869565217392e-07, 'completion_length': 150.7589340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2696239948272705, 'kl': 0.012939453125, 'epoch': 1.87} 37%|███▋ | 602/1610 [2:35:43<7:26:35, 26.58s/it] 37%|███▋ | 603/1610 [2:36:00<6:39:45, 23.82s/it] {'loss': 0.0006, 'grad_norm': 1.848005013289075, 'learning_rate': 6.254658385093168e-07, 'completion_length': 197.6428680419922, 'rewards/accuracy_reward': 0.258928582072258, 'rewards/format_reward': 1.0, 'reward': 1.2589285969734192, 'reward_std': 0.30963000655174255, 'kl': 0.015960693359375, 'epoch': 1.87} 37%|███▋ | 603/1610 [2:36:00<6:39:45, 23.82s/it] 38%|███▊ | 604/1610 [2:36:17<6:01:04, 21.54s/it] {'loss': 0.0005, 'grad_norm': 1.4602351642618785, 'learning_rate': 6.248447204968945e-07, 'completion_length': 169.6071548461914, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.4464285969734192, 'reward_std': 0.3447185158729553, 'kl': 0.01165771484375, 'epoch': 1.88} 38%|███▊ | 604/1610 [2:36:17<6:01:04, 21.54s/it] 38%|███▊ | 605/1610 [2:36:33<5:32:13, 19.83s/it] {'loss': 0.0005, 'grad_norm': 1.2527371964087535, 'learning_rate': 6.24223602484472e-07, 'completion_length': 201.49107360839844, 'rewards/accuracy_reward': 0.3482143133878708, 'rewards/format_reward': 1.0, 'reward': 1.3482143878936768, 'reward_std': 0.3342759907245636, 'kl': 0.0118408203125, 'epoch': 1.88} 38%|███▊ | 605/1610 [2:36:33<5:32:13, 19.83s/it] 38%|███▊ | 606/1610 [2:36:47<5:05:54, 18.28s/it] {'loss': 0.0004, 'grad_norm': 1.1137718060219948, 'learning_rate': 6.236024844720496e-07, 'completion_length': 162.41964721679688, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.24619604647159576, 'kl': 0.009429931640625, 'epoch': 1.88} 38%|███▊ | 606/1610 [2:36:47<5:05:54, 18.28s/it] 38%|███▊ | 607/1610 [2:37:01<4:45:33, 17.08s/it] {'loss': 0.0004, 'grad_norm': 1.4748968801788591, 'learning_rate': 6.229813664596273e-07, 'completion_length': 143.71429443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.33245909214019775, 'kl': 0.01019287109375, 'epoch': 1.89} 38%|███▊ | 607/1610 [2:37:01<4:45:33, 17.08s/it] 38%|███▊ | 608/1610 [2:37:16<4:30:58, 16.23s/it] {'loss': 0.0006, 'grad_norm': 4.027567642234065, 'learning_rate': 6.223602484472049e-07, 'completion_length': 142.3482208251953, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.19057884812355042, 'kl': 0.014129638671875, 'epoch': 1.89} 38%|███▊ | 608/1610 [2:37:16<4:30:58, 16.23s/it] 38%|███▊ | 609/1610 [2:37:31<4:23:37, 15.80s/it] {'loss': 0.0006, 'grad_norm': 1.9181315811452502, 'learning_rate': 6.217391304347825e-07, 'completion_length': 176.24108123779297, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.36491623520851135, 'kl': 0.014312744140625, 'epoch': 1.89} 38%|███▊ | 609/1610 [2:37:31<4:23:37, 15.80s/it] 38%|███▊ | 610/1610 [2:37:46<4:20:04, 15.60s/it] {'loss': 0.0006, 'grad_norm': 1.023818006410792, 'learning_rate': 6.211180124223601e-07, 'completion_length': 171.0089340209961, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.2086147740483284, 'kl': 0.01470947265625, 'epoch': 1.89} 38%|███▊ | 610/1610 [2:37:46<4:20:04, 15.60s/it] 38%|███▊ | 611/1610 [2:38:01<4:20:18, 15.63s/it] {'loss': 0.0005, 'grad_norm': 1.0035994036605504, 'learning_rate': 6.204968944099379e-07, 'completion_length': 162.6607208251953, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857909202576, 'reward_std': 0.24025245010852814, 'kl': 0.0130615234375, 'epoch': 1.9} 38%|███▊ | 611/1610 [2:38:01<4:20:18, 15.63s/it] 38%|███▊ | 612/1610 [2:38:18<4:22:52, 15.80s/it] {'loss': 0.0006, 'grad_norm': 1.1942860476442938, 'learning_rate': 6.198757763975155e-07, 'completion_length': 156.98214721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910715222358704, 'reward_std': 0.28893424570560455, 'kl': 0.01385498046875, 'epoch': 1.9} 38%|███▊ | 612/1610 [2:38:18<4:22:52, 15.80s/it] 38%|███▊ | 613/1610 [2:38:35<4:30:27, 16.28s/it] {'loss': 0.0005, 'grad_norm': 1.6563704288351155, 'learning_rate': 6.192546583850932e-07, 'completion_length': 158.6964340209961, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.29488344490528107, 'kl': 0.0123291015625, 'epoch': 1.9} 38%|███▊ | 613/1610 [2:38:35<4:30:27, 16.28s/it] 38%|███▊ | 614/1610 [2:38:51<4:29:31, 16.24s/it] {'loss': 0.0005, 'grad_norm': 1.2951932299597577, 'learning_rate': 6.186335403726708e-07, 'completion_length': 157.91964721679688, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000001192092896, 'reward_std': 0.3032089099287987, 'kl': 0.012298583984375, 'epoch': 1.91} 38%|███▊ | 614/1610 [2:38:51<4:29:31, 16.24s/it] 38%|███▊ | 615/1610 [2:39:08<4:31:19, 16.36s/it] {'loss': 0.0005, 'grad_norm': 0.8181032072816005, 'learning_rate': 6.180124223602484e-07, 'completion_length': 192.21429443359375, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.2784382402896881, 'kl': 0.012603759765625, 'epoch': 1.91} 38%|███▊ | 615/1610 [2:39:08<4:31:19, 16.36s/it] 38%|███▊ | 616/1610 [2:39:23<4:28:01, 16.18s/it] {'loss': 0.0005, 'grad_norm': 4.538806438553366, 'learning_rate': 6.17391304347826e-07, 'completion_length': 162.20536041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.17226042971014977, 'kl': 0.01361083984375, 'epoch': 1.91} 38%|███▊ | 616/1610 [2:39:23<4:28:01, 16.18s/it] 38%|███▊ | 617/1610 [2:39:38<4:17:19, 15.55s/it] {'loss': 0.0005, 'grad_norm': 3.2757806503181857, 'learning_rate': 6.167701863354037e-07, 'completion_length': 140.0803680419922, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.3012109845876694, 'kl': 0.011810302734375, 'epoch': 1.92} 38%|███▊ | 617/1610 [2:39:38<4:17:19, 15.55s/it] 38%|███▊ | 618/1610 [2:39:55<4:25:30, 16.06s/it] {'loss': 0.0005, 'grad_norm': 3.7416045764903454, 'learning_rate': 6.161490683229813e-07, 'completion_length': 151.5178680419922, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.3661285936832428, 'kl': 0.011474609375, 'epoch': 1.92} 38%|███▊ | 618/1610 [2:39:55<4:25:30, 16.06s/it] 38%|███▊ | 619/1610 [2:40:11<4:24:00, 15.98s/it] {'loss': 0.0005, 'grad_norm': 1.2583663048662028, 'learning_rate': 6.15527950310559e-07, 'completion_length': 171.29464721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3117181807756424, 'kl': 0.0133056640625, 'epoch': 1.92} 38%|███▊ | 619/1610 [2:40:11<4:24:00, 15.98s/it] 39%|███▊ | 620/1610 [2:40:26<4:19:15, 15.71s/it] {'loss': 0.0005, 'grad_norm': 1.0789000970043672, 'learning_rate': 6.149068322981367e-07, 'completion_length': 170.58036041259766, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553572535514832, 'reward_std': 0.2921874076128006, 'kl': 0.012451171875, 'epoch': 1.93} 39%|███▊ | 620/1610 [2:40:26<4:19:15, 15.71s/it] 39%|███▊ | 621/1610 [2:40:41<4:18:08, 15.66s/it] {'loss': 0.0006, 'grad_norm': 1.8567724793785534, 'learning_rate': 6.142857142857143e-07, 'completion_length': 187.8214340209961, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3439074158668518, 'kl': 0.015655517578125, 'epoch': 1.93} 39%|███▊ | 621/1610 [2:40:41<4:18:08, 15.66s/it] 39%|███▊ | 622/1610 [2:40:58<4:21:29, 15.88s/it] {'loss': 0.0004, 'grad_norm': 1.4863726677032856, 'learning_rate': 6.13664596273292e-07, 'completion_length': 153.2857208251953, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.32013726234436035, 'kl': 0.010009765625, 'epoch': 1.93} 39%|███▊ | 622/1610 [2:40:58<4:21:29, 15.88s/it] 39%|███▊ | 623/1610 [2:41:14<4:23:09, 16.00s/it] {'loss': 0.0005, 'grad_norm': 2.9750697792022347, 'learning_rate': 6.130434782608696e-07, 'completion_length': 169.06250762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.3938579112291336, 'kl': 0.01239013671875, 'epoch': 1.93} 39%|███▊ | 623/1610 [2:41:14<4:23:09, 16.00s/it] 39%|███▉ | 624/1610 [2:41:30<4:22:44, 15.99s/it] {'loss': 0.0005, 'grad_norm': 2.8882188225949954, 'learning_rate': 6.124223602484472e-07, 'completion_length': 155.67858123779297, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2404673993587494, 'kl': 0.012237548828125, 'epoch': 1.94} 39%|███▉ | 624/1610 [2:41:30<4:22:44, 15.99s/it] 39%|███▉ | 625/1610 [2:41:48<4:32:39, 16.61s/it] {'loss': 0.0005, 'grad_norm': 1.100433937113955, 'learning_rate': 6.118012422360248e-07, 'completion_length': 181.7589340209961, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.5000000596046448, 'reward_std': 0.31678882241249084, 'kl': 0.01348876953125, 'epoch': 1.94} 39%|███▉ | 625/1610 [2:41:48<4:32:39, 16.61s/it] 39%|███▉ | 626/1610 [2:42:02<4:18:20, 15.75s/it] {'loss': 0.0004, 'grad_norm': 1.2368423411466902, 'learning_rate': 6.111801242236025e-07, 'completion_length': 128.2410774230957, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.3117125630378723, 'kl': 0.0111083984375, 'epoch': 1.94} 39%|███▉ | 626/1610 [2:42:02<4:18:20, 15.75s/it] 39%|███▉ | 627/1610 [2:42:16<4:11:52, 15.37s/it] {'loss': 0.0005, 'grad_norm': 1.5679267325244415, 'learning_rate': 6.105590062111801e-07, 'completion_length': 149.92858123779297, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.30363014340400696, 'kl': 0.0125732421875, 'epoch': 1.95} 39%|███▉ | 627/1610 [2:42:16<4:11:52, 15.37s/it] 39%|███▉ | 628/1610 [2:42:32<4:12:37, 15.44s/it] {'loss': 0.0006, 'grad_norm': 1.6159557828684146, 'learning_rate': 6.099378881987576e-07, 'completion_length': 163.01786041259766, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.2987862229347229, 'kl': 0.0147705078125, 'epoch': 1.95} 39%|███▉ | 628/1610 [2:42:32<4:12:37, 15.44s/it] 39%|███▉ | 629/1610 [2:42:48<4:16:01, 15.66s/it] {'loss': 0.0005, 'grad_norm': 1.0895285930608414, 'learning_rate': 6.093167701863354e-07, 'completion_length': 177.26786041259766, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.35831743478775024, 'kl': 0.013336181640625, 'epoch': 1.95} 39%|███▉ | 629/1610 [2:42:48<4:16:01, 15.66s/it] 39%|███▉ | 630/1610 [2:43:04<4:17:38, 15.77s/it] {'loss': 0.0004, 'grad_norm': 2.3572732493793205, 'learning_rate': 6.08695652173913e-07, 'completion_length': 129.4732208251953, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.660714328289032, 'reward_std': 0.2410808578133583, 'kl': 0.010711669921875, 'epoch': 1.96} 39%|███▉ | 630/1610 [2:43:04<4:17:38, 15.77s/it] 39%|███▉ | 631/1610 [2:43:20<4:19:33, 15.91s/it] {'loss': 0.0005, 'grad_norm': 3.4575520165963507, 'learning_rate': 6.080745341614906e-07, 'completion_length': 178.1339340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.26814597845077515, 'kl': 0.012969970703125, 'epoch': 1.96} 39%|███▉ | 631/1610 [2:43:20<4:19:33, 15.91s/it] 39%|███▉ | 632/1610 [2:43:38<4:26:59, 16.38s/it] {'loss': 0.0005, 'grad_norm': 0.892778629418144, 'learning_rate': 6.074534161490683e-07, 'completion_length': 197.58929443359375, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3928571939468384, 'reward_std': 0.27805426716804504, 'kl': 0.012054443359375, 'epoch': 1.96} 39%|███▉ | 632/1610 [2:43:38<4:26:59, 16.38s/it] 39%|███▉ | 633/1610 [2:43:51<4:12:55, 15.53s/it] {'loss': 0.0005, 'grad_norm': 1.389760083358324, 'learning_rate': 6.068322981366459e-07, 'completion_length': 133.9107208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.18666484206914902, 'kl': 0.01129150390625, 'epoch': 1.97} 39%|███▉ | 633/1610 [2:43:51<4:12:55, 15.53s/it] 39%|███▉ | 634/1610 [2:44:06<4:09:20, 15.33s/it] {'loss': 0.0006, 'grad_norm': 1.4260135078795597, 'learning_rate': 6.062111801242235e-07, 'completion_length': 162.98214721679688, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.2579156309366226, 'kl': 0.01416015625, 'epoch': 1.97} 39%|███▉ | 634/1610 [2:44:06<4:09:20, 15.33s/it] 39%|███▉ | 635/1610 [2:44:20<4:03:45, 15.00s/it] {'loss': 0.0005, 'grad_norm': 1.4316585069871346, 'learning_rate': 6.055900621118012e-07, 'completion_length': 129.51786041259766, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.31623445451259613, 'kl': 0.01361083984375, 'epoch': 1.97} 39%|███▉ | 635/1610 [2:44:20<4:03:45, 15.00s/it] 40%|███▉ | 636/1610 [2:44:37<4:09:21, 15.36s/it] {'loss': 0.0006, 'grad_norm': 1.005185054115174, 'learning_rate': 6.049689440993788e-07, 'completion_length': 157.5357208251953, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.571428656578064, 'reward_std': 0.2527948468923569, 'kl': 0.0142822265625, 'epoch': 1.98} 40%|███▉ | 636/1610 [2:44:37<4:09:21, 15.36s/it] 40%|███▉ | 637/1610 [2:44:52<4:08:11, 15.30s/it] {'loss': 0.0006, 'grad_norm': 1.2141399451421109, 'learning_rate': 6.043478260869564e-07, 'completion_length': 146.67857360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.244989275932312, 'kl': 0.0142822265625, 'epoch': 1.98} 40%|███▉ | 637/1610 [2:44:52<4:08:11, 15.30s/it] 40%|███▉ | 638/1610 [2:45:06<4:02:02, 14.94s/it] {'loss': 0.0004, 'grad_norm': 1.320490855657508, 'learning_rate': 6.037267080745342e-07, 'completion_length': 129.04464721679688, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.27535824477672577, 'kl': 0.010467529296875, 'epoch': 1.98} 40%|███▉ | 638/1610 [2:45:06<4:02:02, 14.94s/it] 40%|███▉ | 639/1610 [2:45:20<3:56:34, 14.62s/it] {'loss': 0.0004, 'grad_norm': 2.3524152331786152, 'learning_rate': 6.031055900621118e-07, 'completion_length': 155.5446548461914, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.28707222640514374, 'kl': 0.010894775390625, 'epoch': 1.98} 40%|███▉ | 639/1610 [2:45:20<3:56:34, 14.62s/it] 40%|███▉ | 640/1610 [2:45:35<3:58:57, 14.78s/it] {'loss': 0.0005, 'grad_norm': 2.6198342816576945, 'learning_rate': 6.024844720496894e-07, 'completion_length': 148.40178680419922, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.25791002810001373, 'kl': 0.013702392578125, 'epoch': 1.99} 40%|███▉ | 640/1610 [2:45:35<3:58:57, 14.78s/it] 40%|███▉ | 641/1610 [2:45:50<4:00:49, 14.91s/it] {'loss': 0.0006, 'grad_norm': 1.183423518862586, 'learning_rate': 6.018633540372671e-07, 'completion_length': 170.2232208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.272320032119751, 'kl': 0.014495849609375, 'epoch': 1.99} 40%|███▉ | 641/1610 [2:45:50<4:00:49, 14.91s/it] 40%|███▉ | 642/1610 [2:46:06<4:03:22, 15.09s/it] {'loss': 0.0005, 'grad_norm': 2.0865000743940936, 'learning_rate': 6.012422360248447e-07, 'completion_length': 150.89286041259766, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.321428656578064, 'reward_std': 0.24289216846227646, 'kl': 0.012420654296875, 'epoch': 1.99} 40%|███▉ | 642/1610 [2:46:06<4:03:22, 15.09s/it] 40%|███▉ | 643/1610 [2:46:20<3:59:13, 14.84s/it] {'loss': 0.0005, 'grad_norm': 1.363441790528963, 'learning_rate': 6.006211180124223e-07, 'completion_length': 164.32144165039062, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.2921874225139618, 'kl': 0.01251220703125, 'epoch': 2.0} 40%|███▉ | 643/1610 [2:46:20<3:59:13, 14.84s/it] 40%|████ | 644/1610 [2:46:34<3:56:14, 14.67s/it] {'loss': 0.0006, 'grad_norm': 1.7665182715283378, 'learning_rate': 6e-07, 'completion_length': 160.30358123779297, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3862616568803787, 'kl': 0.014495849609375, 'epoch': 2.0} 40%|████ | 644/1610 [2:46:34<3:56:14, 14.67s/it] 40%|████ | 645/1610 [2:46:48<3:54:27, 14.58s/it] {'loss': 0.0006, 'grad_norm': 1.3081283125216552, 'learning_rate': 5.993788819875776e-07, 'completion_length': 154.1964340209961, 'rewards/accuracy_reward': 0.267857164144516, 'rewards/format_reward': 1.0, 'reward': 1.2678571939468384, 'reward_std': 0.2507179230451584, 'kl': 0.014495849609375, 'epoch': 2.0} 40%|████ | 645/1610 [2:46:48<3:54:27, 14.58s/it] 40%|████ | 646/1610 [2:47:03<3:56:32, 14.72s/it] {'loss': 0.0004, 'grad_norm': 1.4015713158538878, 'learning_rate': 5.987577639751552e-07, 'completion_length': 155.26786041259766, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3571429252624512, 'reward_std': 0.33655090630054474, 'kl': 0.009765625, 'epoch': 2.01} 40%|████ | 646/1610 [2:47:03<3:56:32, 14.72s/it] 40%|████ | 647/1610 [2:47:20<4:05:00, 15.27s/it] {'loss': 0.0006, 'grad_norm': 0.9451504644249293, 'learning_rate': 5.98136645962733e-07, 'completion_length': 169.86608123779297, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.4375000596046448, 'reward_std': 0.296709269285202, 'kl': 0.015625, 'epoch': 2.01} 40%|████ | 647/1610 [2:47:20<4:05:00, 15.27s/it] 40%|████ | 648/1610 [2:47:35<4:02:08, 15.10s/it] {'loss': 0.0005, 'grad_norm': 1.073034980026268, 'learning_rate': 5.975155279503106e-07, 'completion_length': 136.0089340209961, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.272320032119751, 'kl': 0.013214111328125, 'epoch': 2.01} 40%|████ | 648/1610 [2:47:35<4:02:08, 15.10s/it] 40%|████ | 649/1610 [2:47:50<4:01:53, 15.10s/it] {'loss': 0.0006, 'grad_norm': 1.2037961809347035, 'learning_rate': 5.968944099378882e-07, 'completion_length': 155.8214340209961, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.3369663953781128, 'kl': 0.01385498046875, 'epoch': 2.02} 40%|████ | 649/1610 [2:47:50<4:01:53, 15.10s/it] 40%|████ | 650/1610 [2:48:06<4:05:02, 15.32s/it] {'loss': 0.0006, 'grad_norm': 1.4161470093509825, 'learning_rate': 5.962732919254659e-07, 'completion_length': 167.58928680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.24883297830820084, 'kl': 0.01385498046875, 'epoch': 2.02} 40%|████ | 650/1610 [2:48:06<4:05:02, 15.32s/it] 40%|████ | 651/1610 [2:48:20<4:00:05, 15.02s/it] {'loss': 0.0005, 'grad_norm': 1.2463604292096864, 'learning_rate': 5.956521739130435e-07, 'completion_length': 157.9464340209961, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.287077859044075, 'kl': 0.01361083984375, 'epoch': 2.02} 40%|████ | 651/1610 [2:48:20<4:00:05, 15.02s/it] 40%|████ | 652/1610 [2:48:39<4:20:05, 16.29s/it] {'loss': 0.0006, 'grad_norm': 1.606634440033501, 'learning_rate': 5.95031055900621e-07, 'completion_length': 161.85714721679688, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.38357125222682953, 'kl': 0.014434814453125, 'epoch': 2.02} 40%|████ | 652/1610 [2:48:39<4:20:05, 16.29s/it] 41%|████ | 653/1610 [2:48:54<4:12:53, 15.85s/it] {'loss': 0.0005, 'grad_norm': 3.862966534014086, 'learning_rate': 5.944099378881987e-07, 'completion_length': 154.54464721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.3279484361410141, 'kl': 0.012786865234375, 'epoch': 2.03} 41%|████ | 653/1610 [2:48:54<4:12:53, 15.85s/it] 41%|████ | 654/1610 [2:49:12<4:20:50, 16.37s/it] {'loss': 0.0005, 'grad_norm': 1.4568571593160593, 'learning_rate': 5.937888198757763e-07, 'completion_length': 159.80358123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.29757384955883026, 'kl': 0.01165771484375, 'epoch': 2.03} 41%|████ | 654/1610 [2:49:12<4:20:50, 16.37s/it] 41%|████ | 655/1610 [2:49:25<4:08:17, 15.60s/it] {'loss': 0.0005, 'grad_norm': 1.5440137354502645, 'learning_rate': 5.931677018633539e-07, 'completion_length': 152.8482208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2657212167978287, 'kl': 0.011260986328125, 'epoch': 2.03} 41%|████ | 655/1610 [2:49:25<4:08:17, 15.60s/it] 41%|████ | 656/1610 [2:49:39<4:00:35, 15.13s/it] {'loss': 0.0005, 'grad_norm': 0.894879601947155, 'learning_rate': 5.925465838509317e-07, 'completion_length': 139.43750381469727, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.22875340282917023, 'kl': 0.01226806640625, 'epoch': 2.04} 41%|████ | 656/1610 [2:49:40<4:00:35, 15.13s/it] 41%|████ | 657/1610 [2:49:54<3:56:33, 14.89s/it] {'loss': 0.0005, 'grad_norm': 1.2509278440948222, 'learning_rate': 5.919254658385093e-07, 'completion_length': 152.92858123779297, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.26572123169898987, 'kl': 0.01220703125, 'epoch': 2.04} 41%|████ | 657/1610 [2:49:54<3:56:33, 14.89s/it] 41%|████ | 658/1610 [2:50:10<4:03:31, 15.35s/it] {'loss': 0.0005, 'grad_norm': 0.8584640588002076, 'learning_rate': 5.913043478260869e-07, 'completion_length': 200.68750762939453, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.21192426979541779, 'kl': 0.01361083984375, 'epoch': 2.04} 41%|████ | 658/1610 [2:50:10<4:03:31, 15.35s/it] 41%|████ | 659/1610 [2:50:25<4:01:02, 15.21s/it] {'loss': 0.0006, 'grad_norm': 0.8856046346753205, 'learning_rate': 5.906832298136646e-07, 'completion_length': 154.0982208251953, 'rewards/accuracy_reward': 0.6339286118745804, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.15933407098054886, 'kl': 0.01416015625, 'epoch': 2.05} 41%|████ | 659/1610 [2:50:25<4:01:02, 15.21s/it] 41%|████ | 660/1610 [2:50:40<4:00:27, 15.19s/it] {'loss': 0.0006, 'grad_norm': 0.8178052736059561, 'learning_rate': 5.900621118012422e-07, 'completion_length': 177.3303680419922, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.17495086789131165, 'kl': 0.0140380859375, 'epoch': 2.05} 41%|████ | 660/1610 [2:50:40<4:00:27, 15.19s/it] 41%|████ | 661/1610 [2:50:55<3:55:58, 14.92s/it] {'loss': 0.0004, 'grad_norm': 1.7535695286876722, 'learning_rate': 5.894409937888198e-07, 'completion_length': 146.08036422729492, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2993996888399124, 'kl': 0.010101318359375, 'epoch': 2.05} 41%|████ | 661/1610 [2:50:55<3:55:58, 14.92s/it] 41%|████ | 662/1610 [2:51:09<3:54:20, 14.83s/it] {'loss': 0.0005, 'grad_norm': 2.7152734452752685, 'learning_rate': 5.888198757763975e-07, 'completion_length': 142.62500762939453, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.28438183665275574, 'kl': 0.01171875, 'epoch': 2.06} 41%|████ | 662/1610 [2:51:09<3:54:20, 14.83s/it] 41%|████ | 663/1610 [2:51:24<3:51:53, 14.69s/it] {'loss': 0.0006, 'grad_norm': 1.8849367451779795, 'learning_rate': 5.881987577639751e-07, 'completion_length': 145.77679443359375, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 1.0, 'reward': 1.544642984867096, 'reward_std': 0.2702430933713913, 'kl': 0.015899658203125, 'epoch': 2.06} 41%|████ | 663/1610 [2:51:24<3:51:53, 14.69s/it] 41%|████ | 664/1610 [2:51:39<3:53:37, 14.82s/it] {'loss': 0.0006, 'grad_norm': 2.4418474284186087, 'learning_rate': 5.875776397515527e-07, 'completion_length': 169.8214340209961, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.3850548714399338, 'kl': 0.013885498046875, 'epoch': 2.06} 41%|████ | 664/1610 [2:51:39<3:53:37, 14.82s/it] 41%|████▏ | 665/1610 [2:51:53<3:50:51, 14.66s/it] {'loss': 0.0005, 'grad_norm': 1.0803913532023555, 'learning_rate': 5.869565217391305e-07, 'completion_length': 153.33929443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.17104807496070862, 'kl': 0.01318359375, 'epoch': 2.07} 41%|████▏ | 665/1610 [2:51:53<3:50:51, 14.66s/it] 41%|████▏ | 666/1610 [2:52:10<4:03:29, 15.48s/it] {'loss': 0.0006, 'grad_norm': 1.5893727186247515, 'learning_rate': 5.863354037267081e-07, 'completion_length': 172.9732208251953, 'rewards/accuracy_reward': 0.5803571939468384, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5625000596046448, 'reward_std': 0.2247915416955948, 'kl': 0.014984130859375, 'epoch': 2.07} 41%|████▏ | 666/1610 [2:52:10<4:03:29, 15.48s/it] 41%|████▏ | 667/1610 [2:52:25<3:58:47, 15.19s/it] {'loss': 0.0005, 'grad_norm': 1.0064502272539861, 'learning_rate': 5.857142857142857e-07, 'completion_length': 141.0714340209961, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6607143878936768, 'reward_std': 0.27574223279953003, 'kl': 0.012939453125, 'epoch': 2.07} 41%|████▏ | 667/1610 [2:52:25<3:58:47, 15.19s/it] 41%|████▏ | 668/1610 [2:52:40<3:57:41, 15.14s/it] {'loss': 0.0006, 'grad_norm': 1.2562222790291204, 'learning_rate': 5.850931677018634e-07, 'completion_length': 170.7053680419922, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2630307972431183, 'kl': 0.01397705078125, 'epoch': 2.07} 41%|████▏ | 668/1610 [2:52:40<3:57:41, 15.14s/it] 42%|████▏ | 669/1610 [2:52:54<3:50:47, 14.72s/it] {'loss': 0.0004, 'grad_norm': 2.401832690401263, 'learning_rate': 5.84472049689441e-07, 'completion_length': 120.79464721679688, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.24046741425991058, 'kl': 0.010833740234375, 'epoch': 2.08} 42%|████▏ | 669/1610 [2:52:54<3:50:47, 14.72s/it] 42%|████▏ | 670/1610 [2:53:10<3:56:05, 15.07s/it] {'loss': 0.0005, 'grad_norm': 1.38473067964302, 'learning_rate': 5.838509316770186e-07, 'completion_length': 170.2946548461914, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.13346679881215096, 'kl': 0.01239013671875, 'epoch': 2.08} 42%|████▏ | 670/1610 [2:53:10<3:56:05, 15.07s/it] 42%|████▏ | 671/1610 [2:53:24<3:52:57, 14.89s/it] {'loss': 0.0005, 'grad_norm': 0.9352581143277646, 'learning_rate': 5.832298136645963e-07, 'completion_length': 157.71429443359375, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.25670325756073, 'kl': 0.01324462890625, 'epoch': 2.08} 42%|████▏ | 671/1610 [2:53:24<3:52:57, 14.89s/it] 42%|████▏ | 672/1610 [2:53:39<3:52:09, 14.85s/it] {'loss': 0.0005, 'grad_norm': 0.8774466346823231, 'learning_rate': 5.826086956521739e-07, 'completion_length': 167.52679443359375, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5892857909202576, 'reward_std': 0.17574774473905563, 'kl': 0.013458251953125, 'epoch': 2.09} 42%|████▏ | 672/1610 [2:53:39<3:52:09, 14.85s/it] 42%|████▏ | 673/1610 [2:53:54<3:54:51, 15.04s/it] {'loss': 0.0006, 'grad_norm': 1.0598787003559804, 'learning_rate': 5.819875776397515e-07, 'completion_length': 159.7589340209961, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.2723200172185898, 'kl': 0.0147705078125, 'epoch': 2.09} 42%|████▏ | 673/1610 [2:53:54<3:54:51, 15.04s/it] 42%|████▏ | 674/1610 [2:54:10<3:56:44, 15.18s/it] {'loss': 0.0005, 'grad_norm': 2.0993945863238515, 'learning_rate': 5.813664596273293e-07, 'completion_length': 157.51786041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.473214328289032, 'reward_std': 0.4570496082305908, 'kl': 0.0120849609375, 'epoch': 2.09} 42%|████▏ | 674/1610 [2:54:10<3:56:44, 15.18s/it] 42%|████▏ | 675/1610 [2:54:24<3:51:04, 14.83s/it] {'loss': 0.0005, 'grad_norm': 2.9039897562982997, 'learning_rate': 5.807453416149069e-07, 'completion_length': 141.52679443359375, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2786475867033005, 'kl': 0.01177978515625, 'epoch': 2.1} 42%|████▏ | 675/1610 [2:54:24<3:51:04, 14.83s/it] 42%|████▏ | 676/1610 [2:54:39<3:54:41, 15.08s/it] {'loss': 0.0006, 'grad_norm': 0.9539252094644092, 'learning_rate': 5.801242236024844e-07, 'completion_length': 161.7232208251953, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.21044062823057175, 'kl': 0.013946533203125, 'epoch': 2.1} 42%|████▏ | 676/1610 [2:54:39<3:54:41, 15.08s/it] 42%|████▏ | 677/1610 [2:54:54<3:52:00, 14.92s/it] {'loss': 0.0006, 'grad_norm': 1.236796313798509, 'learning_rate': 5.795031055900621e-07, 'completion_length': 153.79464721679688, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.12686797976493835, 'kl': 0.01611328125, 'epoch': 2.1} 42%|████▏ | 677/1610 [2:54:54<3:52:00, 14.92s/it] 42%|████▏ | 678/1610 [2:55:07<3:44:11, 14.43s/it] {'loss': 0.0004, 'grad_norm': 2.63424129914439, 'learning_rate': 5.788819875776397e-07, 'completion_length': 122.39286041259766, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.2248506247997284, 'kl': 0.01123046875, 'epoch': 2.11} 42%|████▏ | 678/1610 [2:55:07<3:44:11, 14.43s/it] 42%|████▏ | 679/1610 [2:55:22<3:43:17, 14.39s/it] {'loss': 0.0006, 'grad_norm': 4.607967238033141, 'learning_rate': 5.782608695652173e-07, 'completion_length': 152.39286041259766, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.38356563448905945, 'kl': 0.014862060546875, 'epoch': 2.11} 42%|████▏ | 679/1610 [2:55:22<3:43:17, 14.39s/it] 42%|████▏ | 680/1610 [2:55:38<3:51:12, 14.92s/it] {'loss': 0.0007, 'grad_norm': 1.2610734454041088, 'learning_rate': 5.77639751552795e-07, 'completion_length': 185.38394165039062, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6250000596046448, 'reward_std': 0.19057324528694153, 'kl': 0.01751708984375, 'epoch': 2.11} 42%|████▏ | 680/1610 [2:55:38<3:51:12, 14.92s/it] 42%|████▏ | 681/1610 [2:55:53<3:54:32, 15.15s/it] {'loss': 0.0007, 'grad_norm': 1.2374706184576851, 'learning_rate': 5.770186335403726e-07, 'completion_length': 176.37500762939453, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.21911637485027313, 'kl': 0.0167236328125, 'epoch': 2.11} 42%|████▏ | 681/1610 [2:55:53<3:54:32, 15.15s/it] 42%|████▏ | 682/1610 [2:56:07<3:48:27, 14.77s/it] {'loss': 0.0006, 'grad_norm': 7.665628316154019, 'learning_rate': 5.763975155279502e-07, 'completion_length': 125.1339340209961, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.3423472195863724, 'kl': 0.0140380859375, 'epoch': 2.12} 42%|████▏ | 682/1610 [2:56:07<3:48:27, 14.77s/it] 42%|████▏ | 683/1610 [2:56:23<3:50:30, 14.92s/it] {'loss': 0.0005, 'grad_norm': 1.172712329504125, 'learning_rate': 5.75776397515528e-07, 'completion_length': 168.48214721679688, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.22875341773033142, 'kl': 0.012115478515625, 'epoch': 2.12} 42%|████▏ | 683/1610 [2:56:23<3:50:30, 14.92s/it] 42%|████▏ | 684/1610 [2:56:36<3:43:33, 14.49s/it] {'loss': 0.0005, 'grad_norm': 1.1195710105586902, 'learning_rate': 5.751552795031056e-07, 'completion_length': 136.58036041259766, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2987862378358841, 'kl': 0.0128173828125, 'epoch': 2.12} 42%|████▏ | 684/1610 [2:56:36<3:43:33, 14.49s/it] 43%|████▎ | 685/1610 [2:56:50<3:41:55, 14.39s/it] {'loss': 0.0006, 'grad_norm': 2.540352317885401, 'learning_rate': 5.745341614906832e-07, 'completion_length': 140.58929061889648, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.38321781158447266, 'kl': 0.01385498046875, 'epoch': 2.13} 43%|████▎ | 685/1610 [2:56:50<3:41:55, 14.39s/it] 43%|████▎ | 686/1610 [2:57:05<3:42:26, 14.44s/it] {'loss': 0.0005, 'grad_norm': 2.5855368453280945, 'learning_rate': 5.739130434782609e-07, 'completion_length': 144.05357360839844, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.22094222903251648, 'kl': 0.013336181640625, 'epoch': 2.13} 43%|████▎ | 686/1610 [2:57:05<3:42:26, 14.44s/it] 43%|████▎ | 687/1610 [2:57:18<3:39:03, 14.24s/it] {'loss': 0.0006, 'grad_norm': 1.8805535430367752, 'learning_rate': 5.732919254658385e-07, 'completion_length': 132.11607360839844, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.24889205396175385, 'kl': 0.014801025390625, 'epoch': 2.13} 43%|████▎ | 687/1610 [2:57:18<3:39:03, 14.24s/it] 43%|████▎ | 688/1610 [2:57:34<3:44:50, 14.63s/it] {'loss': 0.0006, 'grad_norm': 10.959614058151267, 'learning_rate': 5.726708074534161e-07, 'completion_length': 168.12500762939453, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.3402702808380127, 'kl': 0.014007568359375, 'epoch': 2.14} 43%|████▎ | 688/1610 [2:57:34<3:44:50, 14.63s/it] 43%|████▎ | 689/1610 [2:57:50<3:50:46, 15.03s/it] {'loss': 0.0006, 'grad_norm': 1.0177915972075509, 'learning_rate': 5.720496894409938e-07, 'completion_length': 174.74108123779297, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.3642294853925705, 'kl': 0.013916015625, 'epoch': 2.14} 43%|████▎ | 689/1610 [2:57:50<3:50:46, 15.03s/it] 43%|████▎ | 690/1610 [2:58:06<3:56:06, 15.40s/it] {'loss': 0.0006, 'grad_norm': 1.5071335307812328, 'learning_rate': 5.714285714285714e-07, 'completion_length': 141.49108123779297, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.26181280612945557, 'kl': 0.014251708984375, 'epoch': 2.14} 43%|████▎ | 690/1610 [2:58:06<3:56:06, 15.40s/it] 43%|████▎ | 691/1610 [2:58:21<3:53:54, 15.27s/it] {'loss': 0.0006, 'grad_norm': 1.4757545002364585, 'learning_rate': 5.70807453416149e-07, 'completion_length': 154.93750762939453, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.3246389552950859, 'kl': 0.0140380859375, 'epoch': 2.15} 43%|████▎ | 691/1610 [2:58:21<3:53:54, 15.27s/it] 43%|████▎ | 692/1610 [2:58:35<3:47:23, 14.86s/it] {'loss': 0.0005, 'grad_norm': 0.8726881572246451, 'learning_rate': 5.701863354037268e-07, 'completion_length': 130.33036041259766, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053572535514832, 'reward_std': 0.19690079241991043, 'kl': 0.012054443359375, 'epoch': 2.15} 43%|████▎ | 692/1610 [2:58:35<3:47:23, 14.86s/it] 43%|████▎ | 693/1610 [2:58:51<3:51:49, 15.17s/it] {'loss': 0.0006, 'grad_norm': 2.931596992756351, 'learning_rate': 5.695652173913044e-07, 'completion_length': 180.08929443359375, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.37784260511398315, 'kl': 0.01531982421875, 'epoch': 2.15} 43%|████▎ | 693/1610 [2:58:51<3:51:49, 15.17s/it] 43%|████▎ | 694/1610 [2:59:06<3:52:45, 15.25s/it] {'loss': 0.0005, 'grad_norm': 1.1510436359860337, 'learning_rate': 5.68944099378882e-07, 'completion_length': 154.7053680419922, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.33897005021572113, 'kl': 0.01153564453125, 'epoch': 2.16} 43%|████▎ | 694/1610 [2:59:06<3:52:45, 15.25s/it] 43%|████▎ | 695/1610 [2:59:23<3:58:16, 15.62s/it] {'loss': 0.0006, 'grad_norm': 0.9652796289678076, 'learning_rate': 5.683229813664597e-07, 'completion_length': 171.4553680419922, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.16323687136173248, 'kl': 0.0142822265625, 'epoch': 2.16} 43%|████▎ | 695/1610 [2:59:23<3:58:16, 15.62s/it] 43%|████▎ | 696/1610 [2:59:38<3:53:22, 15.32s/it] {'loss': 0.0006, 'grad_norm': 1.5051827338972883, 'learning_rate': 5.677018633540373e-07, 'completion_length': 149.11608123779297, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3850548714399338, 'kl': 0.013824462890625, 'epoch': 2.16} 43%|████▎ | 696/1610 [2:59:38<3:53:22, 15.32s/it] 43%|████▎ | 697/1610 [2:59:53<3:52:11, 15.26s/it] {'loss': 0.0005, 'grad_norm': 1.9477493776099668, 'learning_rate': 5.670807453416149e-07, 'completion_length': 170.17857360839844, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 1.0, 'reward': 1.3303572535514832, 'reward_std': 0.34781019389629364, 'kl': 0.012359619140625, 'epoch': 2.16} 43%|████▎ | 697/1610 [2:59:53<3:52:11, 15.26s/it] 43%|████▎ | 698/1610 [3:00:09<3:57:45, 15.64s/it] {'loss': 0.0005, 'grad_norm': 1.087018266057134, 'learning_rate': 5.664596273291926e-07, 'completion_length': 184.9464340209961, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.3408691883087158, 'kl': 0.013427734375, 'epoch': 2.17} 43%|████▎ | 698/1610 [3:00:09<3:57:45, 15.64s/it] 43%|████▎ | 699/1610 [3:00:24<3:55:15, 15.49s/it] {'loss': 0.0005, 'grad_norm': 3.3003641672157276, 'learning_rate': 5.658385093167701e-07, 'completion_length': 147.43750762939453, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.31622885167598724, 'kl': 0.01275634765625, 'epoch': 2.17} 43%|████▎ | 699/1610 [3:00:24<3:55:15, 15.49s/it] 43%|████▎ | 700/1610 [3:00:40<3:54:38, 15.47s/it] {'loss': 0.0005, 'grad_norm': 1.5519315363153028, 'learning_rate': 5.652173913043477e-07, 'completion_length': 147.2857208251953, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.26181841641664505, 'kl': 0.013031005859375, 'epoch': 2.17} 43%|████▎ | 700/1610 [3:00:40<3:54:38, 15.47s/it] 44%|████▎ | 701/1610 [3:01:45<7:41:56, 30.49s/it] {'loss': 0.0006, 'grad_norm': 1.0663767717039567, 'learning_rate': 5.645962732919255e-07, 'completion_length': 162.0982208251953, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.2500988394021988, 'kl': 0.01513671875, 'epoch': 2.18} 44%|████▎ | 701/1610 [3:01:45<7:41:56, 30.49s/it] 44%|████▎ | 702/1610 [3:02:02<6:37:16, 26.25s/it] {'loss': 0.0005, 'grad_norm': 0.9007656830940208, 'learning_rate': 5.639751552795031e-07, 'completion_length': 159.18750762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.1956884115934372, 'kl': 0.01251220703125, 'epoch': 2.18} 44%|████▎ | 702/1610 [3:02:02<6:37:16, 26.25s/it] 44%|████▎ | 703/1610 [3:02:18<5:49:34, 23.12s/it] {'loss': 0.0006, 'grad_norm': 1.6956823219321915, 'learning_rate': 5.633540372670807e-07, 'completion_length': 148.68750762939453, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2467949539422989, 'kl': 0.01397705078125, 'epoch': 2.18} 44%|████▎ | 703/1610 [3:02:18<5:49:34, 23.12s/it] 44%|████▎ | 704/1610 [3:02:32<5:10:46, 20.58s/it] {'loss': 0.0006, 'grad_norm': 3.907104183823935, 'learning_rate': 5.627329192546583e-07, 'completion_length': 147.5535774230957, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.723214328289032, 'reward_std': 0.21009841561317444, 'kl': 0.01385498046875, 'epoch': 2.19} 44%|████▎ | 704/1610 [3:02:32<5:10:46, 20.58s/it] 44%|████▍ | 705/1610 [3:02:48<4:47:44, 19.08s/it] {'loss': 0.0006, 'grad_norm': 1.7434555961646354, 'learning_rate': 5.62111801242236e-07, 'completion_length': 139.33929443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.25670325756073, 'kl': 0.01397705078125, 'epoch': 2.19} 44%|████▍ | 705/1610 [3:02:48<4:47:44, 19.08s/it] 44%|████▍ | 706/1610 [3:03:02<4:27:35, 17.76s/it] {'loss': 0.0005, 'grad_norm': 1.9305929610708723, 'learning_rate': 5.614906832298136e-07, 'completion_length': 154.71428680419922, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.27535824477672577, 'kl': 0.013092041015625, 'epoch': 2.19} 44%|████▍ | 706/1610 [3:03:02<4:27:35, 17.76s/it] 44%|████▍ | 707/1610 [3:03:18<4:17:56, 17.14s/it] {'loss': 0.0005, 'grad_norm': 1.2829845270347158, 'learning_rate': 5.608695652173912e-07, 'completion_length': 152.93750762939453, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.32013724744319916, 'kl': 0.01177978515625, 'epoch': 2.2} 44%|████▍ | 707/1610 [3:03:18<4:17:56, 17.14s/it] 44%|████▍ | 708/1610 [3:03:33<4:08:59, 16.56s/it] {'loss': 0.0005, 'grad_norm': 1.6005294442894573, 'learning_rate': 5.602484472049689e-07, 'completion_length': 143.3571548461914, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2897682636976242, 'kl': 0.013092041015625, 'epoch': 2.2} 44%|████▍ | 708/1610 [3:03:33<4:08:59, 16.56s/it] 44%|████▍ | 709/1610 [3:03:50<4:07:34, 16.49s/it] {'loss': 0.0005, 'grad_norm': 5.472353943854624, 'learning_rate': 5.596273291925465e-07, 'completion_length': 176.0803680419922, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.2597358822822571, 'kl': 0.0126953125, 'epoch': 2.2} 44%|████▍ | 709/1610 [3:03:50<4:07:34, 16.49s/it] 44%|████▍ | 710/1610 [3:04:04<3:58:47, 15.92s/it] {'loss': 0.0005, 'grad_norm': 1.8372200989772338, 'learning_rate': 5.590062111801241e-07, 'completion_length': 144.06250762939453, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.30693960189819336, 'kl': 0.013580322265625, 'epoch': 2.2} 44%|████▍ | 710/1610 [3:04:04<3:58:47, 15.92s/it] 44%|████▍ | 711/1610 [3:04:20<3:58:22, 15.91s/it] {'loss': 0.0006, 'grad_norm': 1.2324800634419635, 'learning_rate': 5.583850931677019e-07, 'completion_length': 175.75000762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.36491064727306366, 'kl': 0.013885498046875, 'epoch': 2.21} 44%|████▍ | 711/1610 [3:04:20<3:58:22, 15.91s/it] 44%|████▍ | 712/1610 [3:04:35<3:52:01, 15.50s/it] {'loss': 0.0006, 'grad_norm': 0.9495015191158621, 'learning_rate': 5.577639751552795e-07, 'completion_length': 152.0982208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.20862038433551788, 'kl': 0.01544189453125, 'epoch': 2.21} 44%|████▍ | 712/1610 [3:04:35<3:52:01, 15.50s/it] 44%|████▍ | 713/1610 [3:04:50<3:50:39, 15.43s/it] {'loss': 0.0005, 'grad_norm': 1.1342420081842437, 'learning_rate': 5.571428571428571e-07, 'completion_length': 150.1339340209961, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4017857909202576, 'reward_std': 0.3303675800561905, 'kl': 0.012451171875, 'epoch': 2.21} 44%|████▍ | 713/1610 [3:04:50<3:50:39, 15.43s/it] 44%|████▍ | 714/1610 [3:05:04<3:43:45, 14.98s/it] {'loss': 0.0006, 'grad_norm': 3.2769113483786136, 'learning_rate': 5.565217391304348e-07, 'completion_length': 125.42858123779297, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.30842325091362, 'kl': 0.01422119140625, 'epoch': 2.22} 44%|████▍ | 714/1610 [3:05:04<3:43:45, 14.98s/it] 44%|████▍ | 715/1610 [3:05:18<3:40:44, 14.80s/it] {'loss': 0.0006, 'grad_norm': 0.978744834107409, 'learning_rate': 5.559006211180124e-07, 'completion_length': 183.7946548461914, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.29999858140945435, 'kl': 0.015594482421875, 'epoch': 2.22} 44%|████▍ | 715/1610 [3:05:18<3:40:44, 14.80s/it] 44%|████▍ | 716/1610 [3:05:33<3:40:39, 14.81s/it] {'loss': 0.0006, 'grad_norm': 1.213930834511399, 'learning_rate': 5.5527950310559e-07, 'completion_length': 167.8839340209961, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.35589267313480377, 'kl': 0.01580810546875, 'epoch': 2.22} 44%|████▍ | 716/1610 [3:05:33<3:40:39, 14.81s/it] 45%|████▍ | 717/1610 [3:05:49<3:45:13, 15.13s/it] {'loss': 0.0005, 'grad_norm': 1.5178204906823376, 'learning_rate': 5.546583850931677e-07, 'completion_length': 168.25000762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.3345416486263275, 'kl': 0.013153076171875, 'epoch': 2.23} 45%|████▍ | 717/1610 [3:05:49<3:45:13, 15.13s/it] 45%|████▍ | 718/1610 [3:06:05<3:49:11, 15.42s/it] {'loss': 0.0007, 'grad_norm': 1.2204292530688503, 'learning_rate': 5.540372670807453e-07, 'completion_length': 167.2678680419922, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.571428656578064, 'reward_std': 0.3952852040529251, 'kl': 0.016510009765625, 'epoch': 2.23} 45%|████▍ | 718/1610 [3:06:05<3:49:11, 15.42s/it] 45%|████▍ | 719/1610 [3:06:22<3:55:48, 15.88s/it] {'loss': 0.0007, 'grad_norm': 1.0537202195604043, 'learning_rate': 5.534161490683229e-07, 'completion_length': 155.2321548461914, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.23326967656612396, 'kl': 0.016448974609375, 'epoch': 2.23} 45%|████▍ | 719/1610 [3:06:22<3:55:48, 15.88s/it] 45%|████▍ | 720/1610 [3:06:37<3:52:47, 15.69s/it] {'loss': 0.0006, 'grad_norm': 1.4959729441499723, 'learning_rate': 5.527950310559007e-07, 'completion_length': 149.02679443359375, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.30632615089416504, 'kl': 0.01611328125, 'epoch': 2.24} 45%|████▍ | 720/1610 [3:06:37<3:52:47, 15.69s/it] 45%|████▍ | 721/1610 [3:06:52<3:47:27, 15.35s/it] {'loss': 0.0005, 'grad_norm': 0.9776346912056223, 'learning_rate': 5.521739130434783e-07, 'completion_length': 149.95536422729492, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.23365925252437592, 'kl': 0.0123291015625, 'epoch': 2.24} 45%|████▍ | 721/1610 [3:06:52<3:47:27, 15.35s/it] 45%|████▍ | 722/1610 [3:07:07<3:48:13, 15.42s/it] {'loss': 0.0006, 'grad_norm': 1.1610022617180014, 'learning_rate': 5.515527950310559e-07, 'completion_length': 186.94644165039062, 'rewards/accuracy_reward': 0.339285746216774, 'rewards/format_reward': 1.0, 'reward': 1.3392857909202576, 'reward_std': 0.3123260587453842, 'kl': 0.01458740234375, 'epoch': 2.24} 45%|████▍ | 722/1610 [3:07:07<3:48:13, 15.42s/it] 45%|████▍ | 723/1610 [3:07:21<3:41:22, 14.98s/it] {'loss': 0.0006, 'grad_norm': 1.3018634563665534, 'learning_rate': 5.509316770186335e-07, 'completion_length': 133.37500762939453, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.1827620565891266, 'kl': 0.014678955078125, 'epoch': 2.25} 45%|████▍ | 723/1610 [3:07:21<3:41:22, 14.98s/it] 45%|████▍ | 724/1610 [3:07:34<3:32:54, 14.42s/it] {'loss': 0.0005, 'grad_norm': 0.9869872563379272, 'learning_rate': 5.503105590062111e-07, 'completion_length': 122.7410774230957, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.19057324528694153, 'kl': 0.012481689453125, 'epoch': 2.25} 45%|████▍ | 724/1610 [3:07:34<3:32:54, 14.42s/it] 45%|████▌ | 725/1610 [3:07:54<3:53:43, 15.85s/it] {'loss': 0.0006, 'grad_norm': 1.217388323342781, 'learning_rate': 5.496894409937887e-07, 'completion_length': 172.0357208251953, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.40370987355709076, 'kl': 0.0140380859375, 'epoch': 2.25} 45%|████▌ | 725/1610 [3:07:54<3:53:43, 15.85s/it] 45%|████▌ | 726/1610 [3:08:06<3:36:21, 14.68s/it] {'loss': 0.0005, 'grad_norm': 1.21364808710967, 'learning_rate': 5.490683229813664e-07, 'completion_length': 114.25000381469727, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.21313104033470154, 'kl': 0.011566162109375, 'epoch': 2.25} 45%|████▌ | 726/1610 [3:08:06<3:36:21, 14.68s/it] 45%|████▌ | 727/1610 [3:08:19<3:28:20, 14.16s/it] {'loss': 0.0006, 'grad_norm': 0.9829914514702628, 'learning_rate': 5.48447204968944e-07, 'completion_length': 136.15178680419922, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.2020159587264061, 'kl': 0.0147705078125, 'epoch': 2.26} 45%|████▌ | 727/1610 [3:08:19<3:28:20, 14.16s/it] 45%|████▌ | 728/1610 [3:08:32<3:27:10, 14.09s/it] {'loss': 0.0006, 'grad_norm': 3.823646990872547, 'learning_rate': 5.478260869565216e-07, 'completion_length': 143.01786041259766, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.28707224130630493, 'kl': 0.015167236328125, 'epoch': 2.26} 45%|████▌ | 728/1610 [3:08:32<3:27:10, 14.09s/it] 45%|████▌ | 729/1610 [3:08:47<3:28:38, 14.21s/it] {'loss': 0.0004, 'grad_norm': 0.5320411835101285, 'learning_rate': 5.472049689440994e-07, 'completion_length': 132.56250762939453, 'rewards/accuracy_reward': 0.7767857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7767857909202576, 'reward_std': 0.05831881985068321, 'kl': 0.010650634765625, 'epoch': 2.26} 45%|████▌ | 729/1610 [3:08:47<3:28:38, 14.21s/it] 45%|████▌ | 730/1610 [3:09:01<3:27:52, 14.17s/it] {'loss': 0.0005, 'grad_norm': 1.8268057906779602, 'learning_rate': 5.46583850931677e-07, 'completion_length': 156.61608123779297, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928572535514832, 'reward_std': 0.2501044273376465, 'kl': 0.011871337890625, 'epoch': 2.27} 45%|████▌ | 730/1610 [3:09:01<3:27:52, 14.17s/it] 45%|████▌ | 731/1610 [3:09:15<3:27:12, 14.14s/it] {'loss': 0.0005, 'grad_norm': 1.5780989996985089, 'learning_rate': 5.459627329192546e-07, 'completion_length': 133.4464340209961, 'rewards/accuracy_reward': 0.7053571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.2540072426199913, 'kl': 0.013580322265625, 'epoch': 2.27} 45%|████▌ | 731/1610 [3:09:15<3:27:12, 14.14s/it] 45%|████▌ | 732/1610 [3:09:30<3:29:00, 14.28s/it] {'loss': 0.0006, 'grad_norm': 2.11421039406497, 'learning_rate': 5.453416149068323e-07, 'completion_length': 151.7232208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3571050465106964, 'kl': 0.015594482421875, 'epoch': 2.27} 45%|████▌ | 732/1610 [3:09:30<3:29:00, 14.28s/it] 46%|████▌ | 733/1610 [3:09:45<3:33:57, 14.64s/it] {'loss': 0.0006, 'grad_norm': 1.4136992771187094, 'learning_rate': 5.447204968944099e-07, 'completion_length': 147.70536041259766, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.31951816380023956, 'kl': 0.015380859375, 'epoch': 2.28} 46%|████▌ | 733/1610 [3:09:45<3:33:57, 14.64s/it] 46%|████▌ | 734/1610 [3:10:00<3:36:26, 14.82s/it] {'loss': 0.0006, 'grad_norm': 2.39948634266796, 'learning_rate': 5.440993788819875e-07, 'completion_length': 141.3482208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3439074158668518, 'kl': 0.01385498046875, 'epoch': 2.28} 46%|████▌ | 734/1610 [3:10:00<3:36:26, 14.82s/it] 46%|████▌ | 735/1610 [3:10:15<3:34:59, 14.74s/it] {'loss': 0.0006, 'grad_norm': 1.022003979244593, 'learning_rate': 5.434782608695652e-07, 'completion_length': 171.11608123779297, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285715222358704, 'reward_std': 0.20801587402820587, 'kl': 0.013946533203125, 'epoch': 2.28} 46%|████▌ | 735/1610 [3:10:15<3:34:59, 14.74s/it] 46%|████▌ | 736/1610 [3:10:29<3:32:11, 14.57s/it] {'loss': 0.0005, 'grad_norm': 1.1319416940071658, 'learning_rate': 5.428571428571428e-07, 'completion_length': 152.0714340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.34539106488227844, 'kl': 0.01214599609375, 'epoch': 2.29} 46%|████▌ | 736/1610 [3:10:29<3:32:11, 14.57s/it] 46%|████▌ | 737/1610 [3:10:42<3:25:32, 14.13s/it] {'loss': 0.0006, 'grad_norm': 1.8187766388464481, 'learning_rate': 5.422360248447204e-07, 'completion_length': 119.49107360839844, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.3045148551464081, 'kl': 0.0155029296875, 'epoch': 2.29} 46%|████▌ | 737/1610 [3:10:42<3:25:32, 14.13s/it] 46%|████▌ | 738/1610 [3:10:57<3:28:13, 14.33s/it] {'loss': 0.0006, 'grad_norm': 0.8102151888235771, 'learning_rate': 5.416149068322982e-07, 'completion_length': 132.75000381469727, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.26181842386722565, 'kl': 0.01422119140625, 'epoch': 2.29} 46%|████▌ | 738/1610 [3:10:57<3:28:13, 14.33s/it] 46%|████▌ | 739/1610 [3:11:13<3:33:14, 14.69s/it] {'loss': 0.0004, 'grad_norm': 1.281819866823299, 'learning_rate': 5.409937888198758e-07, 'completion_length': 146.8839340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.30659739673137665, 'kl': 0.011138916015625, 'epoch': 2.3} 46%|████▌ | 739/1610 [3:11:13<3:33:14, 14.69s/it] 46%|████▌ | 740/1610 [3:11:27<3:32:48, 14.68s/it] {'loss': 0.0006, 'grad_norm': 1.6159074987936533, 'learning_rate': 5.403726708074534e-07, 'completion_length': 151.79465103149414, 'rewards/accuracy_reward': 0.473214328289032, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.3345416486263275, 'kl': 0.014923095703125, 'epoch': 2.3} 46%|████▌ | 740/1610 [3:11:27<3:32:48, 14.68s/it] 46%|████▌ | 741/1610 [3:11:45<3:45:57, 15.60s/it] {'loss': 0.0005, 'grad_norm': 2.188524390881046, 'learning_rate': 5.397515527950311e-07, 'completion_length': 168.33036041259766, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4910714626312256, 'reward_std': 0.3472888916730881, 'kl': 0.01275634765625, 'epoch': 2.3} 46%|████▌ | 741/1610 [3:11:45<3:45:57, 15.60s/it] 46%|████▌ | 742/1610 [3:12:01<3:46:30, 15.66s/it] {'loss': 0.0006, 'grad_norm': 0.8192173365715685, 'learning_rate': 5.391304347826087e-07, 'completion_length': 172.5357208251953, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.21313663572072983, 'kl': 0.015533447265625, 'epoch': 2.3} 46%|████▌ | 742/1610 [3:12:01<3:46:30, 15.66s/it] 46%|████▌ | 743/1610 [3:12:16<3:46:20, 15.66s/it] {'loss': 0.0007, 'grad_norm': 1.3281802978479973, 'learning_rate': 5.385093167701863e-07, 'completion_length': 159.7321548461914, 'rewards/accuracy_reward': 0.571428582072258, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2792610377073288, 'kl': 0.016571044921875, 'epoch': 2.31} 46%|████▌ | 743/1610 [3:12:16<3:46:20, 15.66s/it] 46%|████▌ | 744/1610 [3:12:31<3:40:55, 15.31s/it] {'loss': 0.0006, 'grad_norm': 0.9633485253333949, 'learning_rate': 5.37888198757764e-07, 'completion_length': 143.79464721679688, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.19690079987049103, 'kl': 0.0159912109375, 'epoch': 2.31} 46%|████▌ | 744/1610 [3:12:31<3:40:55, 15.31s/it] 46%|████▋ | 745/1610 [3:12:48<3:47:39, 15.79s/it] {'loss': 0.0006, 'grad_norm': 1.622325044781913, 'learning_rate': 5.372670807453416e-07, 'completion_length': 184.4821548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.41267435252666473, 'kl': 0.0142822265625, 'epoch': 2.31} 46%|████▋ | 745/1610 [3:12:48<3:47:39, 15.79s/it] 46%|████▋ | 746/1610 [3:13:02<3:39:47, 15.26s/it] {'loss': 0.0005, 'grad_norm': 1.055312611625126, 'learning_rate': 5.366459627329191e-07, 'completion_length': 138.37500381469727, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.24167978763580322, 'kl': 0.01263427734375, 'epoch': 2.32} 46%|████▋ | 746/1610 [3:13:02<3:39:47, 15.26s/it] 46%|████▋ | 747/1610 [3:13:16<3:35:34, 14.99s/it] {'loss': 0.0006, 'grad_norm': 1.6772907474063852, 'learning_rate': 5.360248447204969e-07, 'completion_length': 152.8839340209961, 'rewards/accuracy_reward': 0.4642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.29999861121177673, 'kl': 0.014251708984375, 'epoch': 2.32} 46%|████▋ | 747/1610 [3:13:16<3:35:34, 14.99s/it] 46%|████▋ | 748/1610 [3:13:32<3:39:54, 15.31s/it] {'loss': 0.0006, 'grad_norm': 1.2337073696626348, 'learning_rate': 5.354037267080745e-07, 'completion_length': 147.58929443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2404673993587494, 'kl': 0.0140380859375, 'epoch': 2.32} 46%|████▋ | 748/1610 [3:13:32<3:39:54, 15.31s/it] 47%|████▋ | 749/1610 [3:13:47<3:36:00, 15.05s/it] {'loss': 0.0006, 'grad_norm': 1.512838283713109, 'learning_rate': 5.347826086956521e-07, 'completion_length': 168.25000762939453, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.24498365819454193, 'kl': 0.016021728515625, 'epoch': 2.33} 47%|████▋ | 749/1610 [3:13:47<3:36:00, 15.05s/it] 47%|████▋ | 750/1610 [3:14:03<3:42:21, 15.51s/it] {'loss': 0.0006, 'grad_norm': 1.3470021115033513, 'learning_rate': 5.341614906832298e-07, 'completion_length': 189.3303680419922, 'rewards/accuracy_reward': 0.4196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.2353578358888626, 'kl': 0.01544189453125, 'epoch': 2.33} 47%|████▋ | 750/1610 [3:14:03<3:42:21, 15.51s/it] 47%|████▋ | 751/1610 [3:14:17<3:34:51, 15.01s/it] {'loss': 0.0005, 'grad_norm': 1.523126433692569, 'learning_rate': 5.335403726708074e-07, 'completion_length': 135.2678680419922, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.30721089243888855, 'kl': 0.01300048828125, 'epoch': 2.33} 47%|████▋ | 751/1610 [3:14:17<3:34:51, 15.01s/it] 47%|████▋ | 752/1610 [3:14:32<3:35:38, 15.08s/it] {'loss': 0.0005, 'grad_norm': 0.4572393510773387, 'learning_rate': 5.32919254658385e-07, 'completion_length': 137.9464340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.08868780359625816, 'kl': 0.01336669921875, 'epoch': 2.34} 47%|████▋ | 752/1610 [3:14:32<3:35:38, 15.08s/it] 47%|████▋ | 753/1610 [3:14:46<3:27:52, 14.55s/it] {'loss': 0.0005, 'grad_norm': 4.458043952842065, 'learning_rate': 5.322981366459627e-07, 'completion_length': 127.20536422729492, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.23656461387872696, 'kl': 0.01300048828125, 'epoch': 2.34} 47%|████▋ | 753/1610 [3:14:46<3:27:52, 14.55s/it] 47%|████▋ | 754/1610 [3:15:00<3:28:28, 14.61s/it] {'loss': 0.0005, 'grad_norm': 1.3896943162846813, 'learning_rate': 5.316770186335403e-07, 'completion_length': 157.8482208251953, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2377769872546196, 'kl': 0.012969970703125, 'epoch': 2.34} 47%|████▋ | 754/1610 [3:15:00<3:28:28, 14.61s/it] 47%|████▋ | 755/1610 [3:15:15<3:29:03, 14.67s/it] {'loss': 0.0006, 'grad_norm': 2.4980522444332376, 'learning_rate': 5.310559006211179e-07, 'completion_length': 151.90179443359375, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2720487713813782, 'kl': 0.013763427734375, 'epoch': 2.34} 47%|████▋ | 755/1610 [3:15:15<3:29:03, 14.67s/it] 47%|████▋ | 756/1610 [3:15:33<3:41:00, 15.53s/it] {'loss': 0.0006, 'grad_norm': 1.667653146994088, 'learning_rate': 5.304347826086957e-07, 'completion_length': 155.64286041259766, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.3763589560985565, 'kl': 0.014434814453125, 'epoch': 2.35} 47%|████▋ | 756/1610 [3:15:33<3:41:00, 15.53s/it] 47%|████▋ | 757/1610 [3:15:49<3:41:59, 15.61s/it] {'loss': 0.0005, 'grad_norm': 1.212638535008819, 'learning_rate': 5.298136645962733e-07, 'completion_length': 162.4196548461914, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.1704346016049385, 'kl': 0.013031005859375, 'epoch': 2.35} 47%|████▋ | 757/1610 [3:15:49<3:41:59, 15.61s/it] 47%|████▋ | 758/1610 [3:16:04<3:39:19, 15.45s/it] {'loss': 0.0007, 'grad_norm': 0.8944679581883452, 'learning_rate': 5.291925465838509e-07, 'completion_length': 179.1607208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.1963018849492073, 'kl': 0.0177001953125, 'epoch': 2.35} 47%|████▋ | 758/1610 [3:16:04<3:39:19, 15.45s/it] 47%|████▋ | 759/1610 [3:16:18<3:36:21, 15.25s/it] {'loss': 0.0006, 'grad_norm': 1.4626570362681135, 'learning_rate': 5.285714285714286e-07, 'completion_length': 155.82143783569336, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625001192092896, 'reward_std': 0.26841163635253906, 'kl': 0.0155029296875, 'epoch': 2.36} 47%|████▋ | 759/1610 [3:16:18<3:36:21, 15.25s/it] 47%|████▋ | 760/1610 [3:16:32<3:29:43, 14.80s/it] {'loss': 0.0007, 'grad_norm': 1.7739662407368366, 'learning_rate': 5.279503105590062e-07, 'completion_length': 141.06250381469727, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.39711108803749084, 'kl': 0.018310546875, 'epoch': 2.36} 47%|████▋ | 760/1610 [3:16:32<3:29:43, 14.80s/it] 47%|████▋ | 761/1610 [3:16:47<3:28:08, 14.71s/it] {'loss': 0.0007, 'grad_norm': 1.3120941603673808, 'learning_rate': 5.273291925465838e-07, 'completion_length': 168.4107208251953, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.2921874225139618, 'kl': 0.017730712890625, 'epoch': 2.36} 47%|████▋ | 761/1610 [3:16:47<3:28:08, 14.71s/it] 47%|████▋ | 762/1610 [3:17:01<3:27:54, 14.71s/it] {'loss': 0.0006, 'grad_norm': 1.4804502326136508, 'learning_rate': 5.267080745341615e-07, 'completion_length': 142.5089340209961, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.33185122162103653, 'kl': 0.01519775390625, 'epoch': 2.37} 47%|████▋ | 762/1610 [3:17:01<3:27:54, 14.71s/it] 47%|████▋ | 763/1610 [3:17:18<3:34:42, 15.21s/it] {'loss': 0.0006, 'grad_norm': 2.661717383049036, 'learning_rate': 5.260869565217391e-07, 'completion_length': 163.46429443359375, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.28403399884700775, 'kl': 0.013946533203125, 'epoch': 2.37} 47%|████▋ | 763/1610 [3:17:18<3:34:42, 15.21s/it] 47%|████▋ | 764/1610 [3:17:34<3:38:14, 15.48s/it] {'loss': 0.0006, 'grad_norm': 1.5205574940409627, 'learning_rate': 5.254658385093167e-07, 'completion_length': 157.33929443359375, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857313156128, 'reward_std': 0.24229326844215393, 'kl': 0.013824462890625, 'epoch': 2.37} 47%|████▋ | 764/1610 [3:17:34<3:38:14, 15.48s/it] 48%|████▊ | 765/1610 [3:17:50<3:40:57, 15.69s/it] {'loss': 0.0007, 'grad_norm': 1.3936720899173296, 'learning_rate': 5.248447204968945e-07, 'completion_length': 176.8482208251953, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.598214328289032, 'reward_std': 0.32916080951690674, 'kl': 0.017852783203125, 'epoch': 2.38} 48%|████▊ | 765/1610 [3:17:50<3:40:57, 15.69s/it] 48%|████▊ | 766/1610 [3:18:04<3:34:14, 15.23s/it] {'loss': 0.0005, 'grad_norm': 1.2562093567513548, 'learning_rate': 5.242236024844721e-07, 'completion_length': 133.39286041259766, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2741458863019943, 'kl': 0.01226806640625, 'epoch': 2.38} 48%|████▊ | 766/1610 [3:18:04<3:34:14, 15.23s/it] 48%|████▊ | 767/1610 [3:18:20<3:34:21, 15.26s/it] {'loss': 0.0007, 'grad_norm': 1.2347103096353733, 'learning_rate': 5.236024844720497e-07, 'completion_length': 166.2678680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2669335901737213, 'kl': 0.016448974609375, 'epoch': 2.38} 48%|████▊ | 767/1610 [3:18:20<3:34:21, 15.26s/it] 48%|████▊ | 768/1610 [3:18:35<3:33:10, 15.19s/it] {'loss': 0.0006, 'grad_norm': 1.3119896743288937, 'learning_rate': 5.229813664596274e-07, 'completion_length': 150.96429443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.237776979804039, 'kl': 0.015838623046875, 'epoch': 2.39} 48%|████▊ | 768/1610 [3:18:35<3:33:10, 15.19s/it] 48%|████▊ | 769/1610 [3:18:49<3:30:18, 15.00s/it] {'loss': 0.0005, 'grad_norm': 1.2649645670130114, 'learning_rate': 5.22360248447205e-07, 'completion_length': 170.62500762939453, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.3214285969734192, 'reward_std': 0.3240400403738022, 'kl': 0.01373291015625, 'epoch': 2.39} 48%|████▊ | 769/1610 [3:18:49<3:30:18, 15.00s/it] 48%|████▊ | 770/1610 [3:19:02<3:20:28, 14.32s/it] {'loss': 0.0006, 'grad_norm': 2.6117355549929484, 'learning_rate': 5.217391304347825e-07, 'completion_length': 123.43750381469727, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.4178486317396164, 'kl': 0.015716552734375, 'epoch': 2.39} 48%|████▊ | 770/1610 [3:19:02<3:20:28, 14.32s/it] 48%|████▊ | 771/1610 [3:19:18<3:28:03, 14.88s/it] {'loss': 0.0005, 'grad_norm': 3.790650799644162, 'learning_rate': 5.211180124223602e-07, 'completion_length': 147.8482208251953, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160714626312256, 'reward_std': 0.2993997037410736, 'kl': 0.013397216796875, 'epoch': 2.39} 48%|████▊ | 771/1610 [3:19:18<3:28:03, 14.88s/it] 48%|████▊ | 772/1610 [3:19:32<3:23:46, 14.59s/it] {'loss': 0.0005, 'grad_norm': 0.7651970819786797, 'learning_rate': 5.204968944099378e-07, 'completion_length': 130.22321701049805, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.20532545447349548, 'kl': 0.011993408203125, 'epoch': 2.4} 48%|████▊ | 772/1610 [3:19:32<3:23:46, 14.59s/it] 48%|████▊ | 773/1610 [3:19:45<3:18:40, 14.24s/it] {'loss': 0.0006, 'grad_norm': 1.4950083718679923, 'learning_rate': 5.198757763975154e-07, 'completion_length': 137.0714340209961, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.19178560376167297, 'kl': 0.01409912109375, 'epoch': 2.4} 48%|████▊ | 773/1610 [3:19:45<3:18:40, 14.24s/it] 48%|████▊ | 774/1610 [3:20:00<3:20:28, 14.39s/it] {'loss': 0.0005, 'grad_norm': 1.3742595754827152, 'learning_rate': 5.192546583850932e-07, 'completion_length': 138.33929061889648, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.32586587965488434, 'kl': 0.01239013671875, 'epoch': 2.4} 48%|████▊ | 774/1610 [3:20:00<3:20:28, 14.39s/it] 48%|████▊ | 775/1610 [3:20:14<3:19:00, 14.30s/it] {'loss': 0.0006, 'grad_norm': 1.4775053397378173, 'learning_rate': 5.186335403726708e-07, 'completion_length': 140.12500381469727, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.25850892812013626, 'kl': 0.015594482421875, 'epoch': 2.41} 48%|████▊ | 775/1610 [3:20:14<3:19:00, 14.30s/it] 48%|████▊ | 776/1610 [3:20:28<3:16:19, 14.12s/it] {'loss': 0.0005, 'grad_norm': 0.9436600347689935, 'learning_rate': 5.180124223602484e-07, 'completion_length': 132.6339340209961, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.24229323863983154, 'kl': 0.013427734375, 'epoch': 2.41} 48%|████▊ | 776/1610 [3:20:28<3:16:19, 14.12s/it] 48%|████▊ | 777/1610 [3:20:42<3:16:35, 14.16s/it] {'loss': 0.0006, 'grad_norm': 1.445831321047582, 'learning_rate': 5.173913043478261e-07, 'completion_length': 141.68750762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.33219341933727264, 'kl': 0.016082763671875, 'epoch': 2.41} 48%|████▊ | 777/1610 [3:20:42<3:16:35, 14.16s/it] 48%|████▊ | 778/1610 [3:20:58<3:23:01, 14.64s/it] {'loss': 0.0007, 'grad_norm': 2.597048071363308, 'learning_rate': 5.167701863354037e-07, 'completion_length': 165.23214721679688, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.342695027589798, 'kl': 0.01788330078125, 'epoch': 2.42} 48%|████▊ | 778/1610 [3:20:58<3:23:01, 14.64s/it] 48%|████▊ | 779/1610 [3:21:13<3:26:21, 14.90s/it] {'loss': 0.0006, 'grad_norm': 3.9766920023382584, 'learning_rate': 5.161490683229813e-07, 'completion_length': 172.56250762939453, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.20020468533039093, 'kl': 0.014556884765625, 'epoch': 2.42} 48%|████▊ | 779/1610 [3:21:13<3:26:21, 14.90s/it] 48%|████▊ | 780/1610 [3:21:29<3:28:52, 15.10s/it] {'loss': 0.0006, 'grad_norm': 1.424320138948954, 'learning_rate': 5.15527950310559e-07, 'completion_length': 152.2232208251953, 'rewards/accuracy_reward': 0.6517857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6428571939468384, 'reward_std': 0.15152288228273392, 'kl': 0.0140380859375, 'epoch': 2.42} 48%|████▊ | 780/1610 [3:21:29<3:28:52, 15.10s/it] 49%|████▊ | 781/1610 [3:21:42<3:21:03, 14.55s/it] {'loss': 0.0006, 'grad_norm': 1.1358259059480995, 'learning_rate': 5.149068322981366e-07, 'completion_length': 140.3482208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.3084176629781723, 'kl': 0.0162353515625, 'epoch': 2.43} 49%|████▊ | 781/1610 [3:21:42<3:21:03, 14.55s/it] 49%|████▊ | 782/1610 [3:21:56<3:17:11, 14.29s/it] {'loss': 0.0005, 'grad_norm': 0.8402597493668426, 'learning_rate': 5.142857142857142e-07, 'completion_length': 137.04464721679688, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.20740239322185516, 'kl': 0.011932373046875, 'epoch': 2.43} 49%|████▊ | 782/1610 [3:21:56<3:17:11, 14.29s/it] 49%|████▊ | 783/1610 [3:22:10<3:16:58, 14.29s/it] {'loss': 0.0005, 'grad_norm': 1.1067539825458392, 'learning_rate': 5.13664596273292e-07, 'completion_length': 151.45536041259766, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.22754104435443878, 'kl': 0.012847900390625, 'epoch': 2.43} 49%|████▊ | 783/1610 [3:22:10<3:16:58, 14.29s/it] 49%|████▊ | 784/1610 [3:22:24<3:15:32, 14.20s/it] {'loss': 0.0006, 'grad_norm': 1.2940928482787326, 'learning_rate': 5.130434782608696e-07, 'completion_length': 137.1160774230957, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.27144986391067505, 'kl': 0.01446533203125, 'epoch': 2.43} 49%|████▊ | 784/1610 [3:22:24<3:15:32, 14.20s/it] 49%|████▉ | 785/1610 [3:22:38<3:14:40, 14.16s/it] {'loss': 0.0005, 'grad_norm': 1.349671211042013, 'learning_rate': 5.124223602484472e-07, 'completion_length': 140.0357208251953, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.28047342598438263, 'kl': 0.013092041015625, 'epoch': 2.44} 49%|████▉ | 785/1610 [3:22:38<3:14:40, 14.16s/it] 49%|████▉ | 786/1610 [3:22:54<3:20:36, 14.61s/it] {'loss': 0.0006, 'grad_norm': 1.126097636793234, 'learning_rate': 5.118012422360249e-07, 'completion_length': 163.88394165039062, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.410714328289032, 'reward_std': 0.3610134422779083, 'kl': 0.01458740234375, 'epoch': 2.44} 49%|████▉ | 786/1610 [3:22:54<3:20:36, 14.61s/it] 49%|████▉ | 787/1610 [3:23:09<3:22:57, 14.80s/it] {'loss': 0.0006, 'grad_norm': 1.599989477722117, 'learning_rate': 5.111801242236025e-07, 'completion_length': 157.12500762939453, 'rewards/accuracy_reward': 0.5982143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.2928008884191513, 'kl': 0.014312744140625, 'epoch': 2.44} 49%|████▉ | 787/1610 [3:23:09<3:22:57, 14.80s/it] 49%|████▉ | 788/1610 [3:23:25<3:28:31, 15.22s/it] {'loss': 0.0007, 'grad_norm': 1.4280087410474347, 'learning_rate': 5.105590062111801e-07, 'completion_length': 174.8571548461914, 'rewards/accuracy_reward': 0.4196428880095482, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.27414026856422424, 'kl': 0.016510009765625, 'epoch': 2.45} 49%|████▉ | 788/1610 [3:23:25<3:28:31, 15.22s/it] 49%|████▉ | 789/1610 [3:23:40<3:23:46, 14.89s/it] {'loss': 0.0004, 'grad_norm': 1.016958141706399, 'learning_rate': 5.099378881987578e-07, 'completion_length': 124.91072082519531, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.15872061252593994, 'kl': 0.010833740234375, 'epoch': 2.45} 49%|████▉ | 789/1610 [3:23:40<3:23:46, 14.89s/it] 49%|████▉ | 790/1610 [3:23:54<3:21:20, 14.73s/it] {'loss': 0.0006, 'grad_norm': 1.46389675623943, 'learning_rate': 5.093167701863354e-07, 'completion_length': 150.6964340209961, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.3396568149328232, 'kl': 0.014251708984375, 'epoch': 2.45} 49%|████▉ | 790/1610 [3:23:54<3:21:20, 14.73s/it] 49%|████▉ | 791/1610 [3:24:07<3:16:12, 14.37s/it] {'loss': 0.0006, 'grad_norm': 3.2995594491564875, 'learning_rate': 5.08695652173913e-07, 'completion_length': 138.58928680419922, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.2254640907049179, 'kl': 0.014556884765625, 'epoch': 2.46} 49%|████▉ | 791/1610 [3:24:07<3:16:12, 14.37s/it] 49%|████▉ | 792/1610 [3:24:23<3:20:57, 14.74s/it] {'loss': 0.0006, 'grad_norm': 1.3056472883291577, 'learning_rate': 5.080745341614908e-07, 'completion_length': 146.02679443359375, 'rewards/accuracy_reward': 0.3660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.28012561798095703, 'kl': 0.013824462890625, 'epoch': 2.46} 49%|████▉ | 792/1610 [3:24:23<3:20:57, 14.74s/it] 49%|████▉ | 793/1610 [3:24:39<3:25:02, 15.06s/it] {'loss': 0.0005, 'grad_norm': 1.3965412266520163, 'learning_rate': 5.074534161490684e-07, 'completion_length': 161.62500762939453, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.29488344490528107, 'kl': 0.01287841796875, 'epoch': 2.46} 49%|████▉ | 793/1610 [3:24:39<3:25:02, 15.06s/it] 49%|████▉ | 794/1610 [3:24:53<3:22:15, 14.87s/it] {'loss': 0.0006, 'grad_norm': 3.0611889206120435, 'learning_rate': 5.068322981366459e-07, 'completion_length': 137.8214340209961, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.33966243267059326, 'kl': 0.014892578125, 'epoch': 2.47} 49%|████▉ | 794/1610 [3:24:53<3:22:15, 14.87s/it] 49%|████▉ | 795/1610 [3:25:07<3:19:01, 14.65s/it] {'loss': 0.0006, 'grad_norm': 1.7312847998288885, 'learning_rate': 5.062111801242235e-07, 'completion_length': 158.4107208251953, 'rewards/accuracy_reward': 0.3035714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3035715222358704, 'reward_std': 0.24315781891345978, 'kl': 0.016021728515625, 'epoch': 2.47} 49%|████▉ | 795/1610 [3:25:07<3:19:01, 14.65s/it] 49%|████▉ | 796/1610 [3:25:23<3:21:48, 14.87s/it] {'loss': 0.0005, 'grad_norm': 2.1570721541148417, 'learning_rate': 5.055900621118012e-07, 'completion_length': 149.5982208251953, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.27144984900951385, 'kl': 0.011993408203125, 'epoch': 2.47} 49%|████▉ | 796/1610 [3:25:23<3:21:48, 14.87s/it] 50%|████▉ | 797/1610 [3:25:36<3:16:01, 14.47s/it] {'loss': 0.0006, 'grad_norm': 1.1232378991316239, 'learning_rate': 5.049689440993788e-07, 'completion_length': 130.12500762939453, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.283163845539093, 'kl': 0.01556396484375, 'epoch': 2.48} 50%|████▉ | 797/1610 [3:25:36<3:16:01, 14.47s/it] 50%|████▉ | 798/1610 [3:25:52<3:20:17, 14.80s/it] {'loss': 0.0008, 'grad_norm': 0.7949204636086173, 'learning_rate': 5.043478260869564e-07, 'completion_length': 198.73214721679688, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.2467949464917183, 'kl': 0.018798828125, 'epoch': 2.48} 50%|████▉ | 798/1610 [3:25:52<3:20:17, 14.80s/it] 50%|████▉ | 799/1610 [3:26:07<3:22:27, 14.98s/it] {'loss': 0.0007, 'grad_norm': 1.2084911322769318, 'learning_rate': 5.037267080745341e-07, 'completion_length': 162.24107360839844, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.2928008884191513, 'kl': 0.0174560546875, 'epoch': 2.48} 50%|████▉ | 799/1610 [3:26:07<3:22:27, 14.98s/it] 50%|████▉ | 800/1610 [3:26:24<3:27:49, 15.39s/it] {'loss': 0.0006, 'grad_norm': 1.1388403628862516, 'learning_rate': 5.031055900621117e-07, 'completion_length': 188.1696548461914, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.535714328289032, 'reward_std': 0.4361557811498642, 'kl': 0.01470947265625, 'epoch': 2.48} 50%|████▉ | 800/1610 [3:26:24<3:27:49, 15.39s/it] 50%|████▉ | 801/1610 [3:27:27<6:40:05, 29.67s/it] {'loss': 0.0004, 'grad_norm': 1.6971845363627154, 'learning_rate': 5.024844720496894e-07, 'completion_length': 135.26786422729492, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.27144983410835266, 'kl': 0.01092529296875, 'epoch': 2.49} 50%|████▉ | 801/1610 [3:27:27<6:40:05, 29.67s/it] 50%|████▉ | 802/1610 [3:27:42<5:42:54, 25.46s/it] {'loss': 0.0006, 'grad_norm': 3.2616644606739666, 'learning_rate': 5.018633540372671e-07, 'completion_length': 144.1607208251953, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3285619020462036, 'kl': 0.0142822265625, 'epoch': 2.49} 50%|████▉ | 802/1610 [3:27:42<5:42:54, 25.46s/it] 50%|████▉ | 803/1610 [3:27:58<5:01:17, 22.40s/it] {'loss': 0.0007, 'grad_norm': 1.0641003398386952, 'learning_rate': 5.012422360248447e-07, 'completion_length': 156.37500762939453, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.22605739533901215, 'kl': 0.01641845703125, 'epoch': 2.49} 50%|████▉ | 803/1610 [3:27:58<5:01:17, 22.40s/it] 50%|████▉ | 804/1610 [3:28:13<4:33:40, 20.37s/it] {'loss': 0.0009, 'grad_norm': 1.050567339606589, 'learning_rate': 5.006211180124223e-07, 'completion_length': 190.7857208251953, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.33184562623500824, 'kl': 0.021728515625, 'epoch': 2.5} 50%|████▉ | 804/1610 [3:28:13<4:33:40, 20.37s/it] 50%|█████ | 805/1610 [3:28:26<4:04:27, 18.22s/it] {'loss': 0.0005, 'grad_norm': 2.325114203549018, 'learning_rate': 5e-07, 'completion_length': 145.04464721679688, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.3207506984472275, 'kl': 0.013336181640625, 'epoch': 2.5} 50%|█████ | 805/1610 [3:28:26<4:04:27, 18.22s/it] 50%|█████ | 806/1610 [3:28:41<3:51:23, 17.27s/it] {'loss': 0.0007, 'grad_norm': 1.353838335058397, 'learning_rate': 4.993788819875776e-07, 'completion_length': 160.60714721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2837828993797302, 'kl': 0.01788330078125, 'epoch': 2.5} 50%|█████ | 806/1610 [3:28:41<3:51:23, 17.27s/it] 50%|█████ | 807/1610 [3:28:57<3:43:16, 16.68s/it] {'loss': 0.0006, 'grad_norm': 1.328162768172278, 'learning_rate': 4.987577639751552e-07, 'completion_length': 147.6339340209961, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3408748060464859, 'kl': 0.014190673828125, 'epoch': 2.51} 50%|█████ | 807/1610 [3:28:57<3:43:16, 16.68s/it] 50%|█████ | 808/1610 [3:29:10<3:29:30, 15.67s/it] {'loss': 0.0005, 'grad_norm': 1.541294979396806, 'learning_rate': 4.981366459627329e-07, 'completion_length': 149.99107360839844, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.20411306619644165, 'kl': 0.012542724609375, 'epoch': 2.51} 50%|█████ | 808/1610 [3:29:10<3:29:30, 15.67s/it] 50%|█████ | 809/1610 [3:29:25<3:25:44, 15.41s/it] {'loss': 0.0005, 'grad_norm': 1.1189436082289435, 'learning_rate': 4.975155279503105e-07, 'completion_length': 141.04464721679688, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.2753526568412781, 'kl': 0.012603759765625, 'epoch': 2.51} 50%|█████ | 809/1610 [3:29:25<3:25:44, 15.41s/it] 50%|█████ | 810/1610 [3:29:40<3:24:56, 15.37s/it] {'loss': 0.0005, 'grad_norm': 1.2057626142125233, 'learning_rate': 4.968944099378881e-07, 'completion_length': 152.5803680419922, 'rewards/accuracy_reward': 0.4196428954601288, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.3928571939468384, 'reward_std': 0.30268900096416473, 'kl': 0.011962890625, 'epoch': 2.52} 50%|█████ | 810/1610 [3:29:40<3:24:56, 15.37s/it] 50%|█████ | 811/1610 [3:29:54<3:19:15, 14.96s/it] {'loss': 0.0005, 'grad_norm': 1.8381517583767641, 'learning_rate': 4.962732919254658e-07, 'completion_length': 150.1607208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.23535221815109253, 'kl': 0.0135498046875, 'epoch': 2.52} 50%|█████ | 811/1610 [3:29:54<3:19:15, 14.96s/it] 50%|█████ | 812/1610 [3:30:09<3:19:51, 15.03s/it] {'loss': 0.0006, 'grad_norm': 0.6134209120049081, 'learning_rate': 4.956521739130435e-07, 'completion_length': 163.93750762939453, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.16653180122375488, 'kl': 0.01556396484375, 'epoch': 2.52} 50%|█████ | 812/1610 [3:30:09<3:19:51, 15.03s/it] 50%|█████ | 813/1610 [3:30:24<3:16:07, 14.77s/it] {'loss': 0.0006, 'grad_norm': 0.9465864014402405, 'learning_rate': 4.950310559006211e-07, 'completion_length': 158.6428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.22726976871490479, 'kl': 0.01617431640625, 'epoch': 2.52} 50%|█████ | 813/1610 [3:30:24<3:16:07, 14.77s/it] 51%|█████ | 814/1610 [3:30:41<3:26:06, 15.54s/it] {'loss': 0.0007, 'grad_norm': 0.6443020653265538, 'learning_rate': 4.944099378881988e-07, 'completion_length': 190.7321548461914, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.16444924101233482, 'kl': 0.0172119140625, 'epoch': 2.53} 51%|█████ | 814/1610 [3:30:41<3:26:06, 15.54s/it] 51%|█████ | 815/1610 [3:30:56<3:23:15, 15.34s/it] {'loss': 0.0006, 'grad_norm': 1.504996037752201, 'learning_rate': 4.937888198757764e-07, 'completion_length': 160.7232208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.33636748790740967, 'kl': 0.015350341796875, 'epoch': 2.53} 51%|█████ | 815/1610 [3:30:56<3:23:15, 15.34s/it] 51%|█████ | 816/1610 [3:31:10<3:19:34, 15.08s/it] {'loss': 0.0005, 'grad_norm': 1.3496643563421244, 'learning_rate': 4.93167701863354e-07, 'completion_length': 130.85715103149414, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.16922222077846527, 'kl': 0.0120849609375, 'epoch': 2.53} 51%|█████ | 816/1610 [3:31:10<3:19:34, 15.08s/it] 51%|█████ | 817/1610 [3:31:24<3:12:24, 14.56s/it] {'loss': 0.0005, 'grad_norm': 1.307475617336724, 'learning_rate': 4.925465838509317e-07, 'completion_length': 142.5714340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.17885926365852356, 'kl': 0.013275146484375, 'epoch': 2.54} 51%|█████ | 817/1610 [3:31:24<3:12:24, 14.56s/it] 51%|█████ | 818/1610 [3:31:39<3:14:18, 14.72s/it] {'loss': 0.0006, 'grad_norm': 1.7329992594396175, 'learning_rate': 4.919254658385093e-07, 'completion_length': 143.85714721679688, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.37784257531166077, 'kl': 0.01422119140625, 'epoch': 2.54} 51%|█████ | 818/1610 [3:31:39<3:14:18, 14.72s/it] 51%|█████ | 819/1610 [3:31:54<3:16:41, 14.92s/it] {'loss': 0.0005, 'grad_norm': 0.8646165516894818, 'learning_rate': 4.913043478260869e-07, 'completion_length': 164.7946548461914, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.1412779912352562, 'kl': 0.013397216796875, 'epoch': 2.54} 51%|█████ | 819/1610 [3:31:54<3:16:41, 14.92s/it] 51%|█████ | 820/1610 [3:32:11<3:23:09, 15.43s/it] {'loss': 0.0005, 'grad_norm': 1.4368653930426791, 'learning_rate': 4.906832298136646e-07, 'completion_length': 151.6607208251953, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.2645144462585449, 'kl': 0.011993408203125, 'epoch': 2.55} 51%|█████ | 820/1610 [3:32:11<3:23:09, 15.43s/it] 51%|█████ | 821/1610 [3:32:25<3:20:35, 15.25s/it] {'loss': 0.0006, 'grad_norm': 1.0386873747412375, 'learning_rate': 4.900621118012422e-07, 'completion_length': 162.8928680419922, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.2987862080335617, 'kl': 0.01458740234375, 'epoch': 2.55} 51%|█████ | 821/1610 [3:32:25<3:20:35, 15.25s/it] 51%|█████ | 822/1610 [3:32:41<3:22:56, 15.45s/it] {'loss': 0.0006, 'grad_norm': 1.2695175874190576, 'learning_rate': 4.894409937888198e-07, 'completion_length': 145.91964721679688, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2993997037410736, 'kl': 0.01397705078125, 'epoch': 2.55} 51%|█████ | 822/1610 [3:32:41<3:22:56, 15.45s/it] 51%|█████ | 823/1610 [3:32:54<3:13:15, 14.73s/it] {'loss': 0.0005, 'grad_norm': 1.0818795397474699, 'learning_rate': 4.888198757763975e-07, 'completion_length': 125.05357360839844, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.22094223648309708, 'kl': 0.0115966796875, 'epoch': 2.56} 51%|█████ | 823/1610 [3:32:54<3:13:15, 14.73s/it] 51%|█████ | 824/1610 [3:33:09<3:12:52, 14.72s/it] {'loss': 0.0006, 'grad_norm': 0.550196939003385, 'learning_rate': 4.881987577639751e-07, 'completion_length': 194.13394165039062, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.13346679508686066, 'kl': 0.01568603515625, 'epoch': 2.56} 51%|█████ | 824/1610 [3:33:09<3:12:52, 14.72s/it] 51%|█████ | 825/1610 [3:33:25<3:17:19, 15.08s/it] {'loss': 0.0006, 'grad_norm': 2.6047073302719177, 'learning_rate': 4.875776397515527e-07, 'completion_length': 155.49108123779297, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.40370987355709076, 'kl': 0.014312744140625, 'epoch': 2.56} 51%|█████ | 825/1610 [3:33:25<3:17:19, 15.08s/it] 51%|█████▏ | 826/1610 [3:33:38<3:10:07, 14.55s/it] {'loss': 0.0004, 'grad_norm': 3.034921722070382, 'learning_rate': 4.869565217391305e-07, 'completion_length': 128.43750762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2636386677622795, 'kl': 0.010986328125, 'epoch': 2.57} 51%|█████▏ | 826/1610 [3:33:38<3:10:07, 14.55s/it] 51%|█████▏ | 827/1610 [3:33:55<3:16:27, 15.05s/it] {'loss': 0.0006, 'grad_norm': 3.0566558246175406, 'learning_rate': 4.863354037267081e-07, 'completion_length': 165.27679443359375, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.535714328289032, 'reward_std': 0.2500453554093838, 'kl': 0.014190673828125, 'epoch': 2.57} 51%|█████▏ | 827/1610 [3:33:55<3:16:27, 15.05s/it] 51%|█████▏ | 828/1610 [3:34:08<3:09:06, 14.51s/it] {'loss': 0.0005, 'grad_norm': 1.4063107316803338, 'learning_rate': 4.857142857142857e-07, 'completion_length': 127.58929443359375, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.31622885167598724, 'kl': 0.01214599609375, 'epoch': 2.57} 51%|█████▏ | 828/1610 [3:34:08<3:09:06, 14.51s/it] 51%|█████▏ | 829/1610 [3:34:24<3:15:16, 15.00s/it] {'loss': 0.0005, 'grad_norm': 0.9700096260366678, 'learning_rate': 4.850931677018633e-07, 'completion_length': 178.23214721679688, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.29507580399513245, 'kl': 0.012664794921875, 'epoch': 2.57} 51%|█████▏ | 829/1610 [3:34:24<3:15:16, 15.00s/it] 52%|█████▏ | 830/1610 [3:34:39<3:13:25, 14.88s/it] {'loss': 0.0007, 'grad_norm': 1.6939896261783254, 'learning_rate': 4.84472049689441e-07, 'completion_length': 166.80358123779297, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.3751465827226639, 'kl': 0.01751708984375, 'epoch': 2.58} 52%|█████▏ | 830/1610 [3:34:39<3:13:25, 14.88s/it] 52%|█████▏ | 831/1610 [3:34:55<3:19:33, 15.37s/it] {'loss': 0.0007, 'grad_norm': 1.7285240894408491, 'learning_rate': 4.838509316770186e-07, 'completion_length': 165.56250762939453, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4732143878936768, 'reward_std': 0.4246540069580078, 'kl': 0.0164794921875, 'epoch': 2.58} 52%|█████▏ | 831/1610 [3:34:55<3:19:33, 15.37s/it] 52%|█████▏ | 832/1610 [3:35:09<3:11:48, 14.79s/it] {'loss': 0.0006, 'grad_norm': 1.8044562071024768, 'learning_rate': 4.832298136645963e-07, 'completion_length': 142.25000762939453, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.28195706009864807, 'kl': 0.014556884765625, 'epoch': 2.58} 52%|█████▏ | 832/1610 [3:35:09<3:11:48, 14.79s/it] 52%|█████▏ | 833/1610 [3:35:24<3:14:42, 15.04s/it] {'loss': 0.0005, 'grad_norm': 0.9731205640572538, 'learning_rate': 4.826086956521739e-07, 'completion_length': 164.45536041259766, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2513168156147003, 'kl': 0.013671875, 'epoch': 2.59} 52%|█████▏ | 833/1610 [3:35:24<3:14:42, 15.04s/it] 52%|█████▏ | 834/1610 [3:35:39<3:14:52, 15.07s/it] {'loss': 0.0005, 'grad_norm': 0.9301441868776786, 'learning_rate': 4.819875776397515e-07, 'completion_length': 148.91072463989258, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.18397443741559982, 'kl': 0.012298583984375, 'epoch': 2.59} 52%|█████▏ | 834/1610 [3:35:39<3:14:52, 15.07s/it] 52%|█████▏ | 835/1610 [3:35:56<3:20:27, 15.52s/it] {'loss': 0.0005, 'grad_norm': 1.1581581617193804, 'learning_rate': 4.813664596273292e-07, 'completion_length': 179.25000762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.2589130699634552, 'kl': 0.01263427734375, 'epoch': 2.59} 52%|█████▏ | 835/1610 [3:35:56<3:20:27, 15.52s/it] 52%|█████▏ | 836/1610 [3:36:12<3:21:38, 15.63s/it] {'loss': 0.0007, 'grad_norm': 1.6016676948081636, 'learning_rate': 4.807453416149068e-07, 'completion_length': 166.5803680419922, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.20080919563770294, 'kl': 0.01629638671875, 'epoch': 2.6} 52%|█████▏ | 836/1610 [3:36:12<3:21:38, 15.63s/it] 52%|█████▏ | 837/1610 [3:36:26<3:17:30, 15.33s/it] {'loss': 0.0006, 'grad_norm': 2.840984797873589, 'learning_rate': 4.801242236024844e-07, 'completion_length': 144.5714340209961, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3589252531528473, 'kl': 0.0145263671875, 'epoch': 2.6} 52%|█████▏ | 837/1610 [3:36:26<3:17:30, 15.33s/it] 52%|█████▏ | 838/1610 [3:36:42<3:16:46, 15.29s/it] {'loss': 0.0006, 'grad_norm': 9.418678443059143, 'learning_rate': 4.795031055900621e-07, 'completion_length': 166.12500762939453, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.2786475867033005, 'kl': 0.0142822265625, 'epoch': 2.6} 52%|█████▏ | 838/1610 [3:36:42<3:16:46, 15.29s/it] 52%|█████▏ | 839/1610 [3:36:57<3:16:34, 15.30s/it] {'loss': 0.0005, 'grad_norm': 1.228735165757545, 'learning_rate': 4.788819875776398e-07, 'completion_length': 164.3125114440918, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.5267857909202576, 'reward_std': 0.40263357758522034, 'kl': 0.0123291015625, 'epoch': 2.61} 52%|█████▏ | 839/1610 [3:36:57<3:16:34, 15.30s/it] 52%|█████▏ | 840/1610 [3:37:13<3:18:02, 15.43s/it] {'loss': 0.0006, 'grad_norm': 1.6849943415042512, 'learning_rate': 4.782608695652174e-07, 'completion_length': 144.5982208251953, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.33575960993766785, 'kl': 0.014617919921875, 'epoch': 2.61} 52%|█████▏ | 840/1610 [3:37:13<3:18:02, 15.43s/it] 52%|█████▏ | 841/1610 [3:37:27<3:12:48, 15.04s/it] {'loss': 0.0005, 'grad_norm': 1.2670172163533613, 'learning_rate': 4.77639751552795e-07, 'completion_length': 142.8214340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.31623443961143494, 'kl': 0.01251220703125, 'epoch': 2.61} 52%|█████▏ | 841/1610 [3:37:27<3:12:48, 15.04s/it] 52%|█████▏ | 842/1610 [3:37:42<3:13:28, 15.11s/it] {'loss': 0.0006, 'grad_norm': 1.1842302406049587, 'learning_rate': 4.770186335403726e-07, 'completion_length': 152.73214721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.22875343263149261, 'kl': 0.01580810546875, 'epoch': 2.61} 52%|█████▏ | 842/1610 [3:37:42<3:13:28, 15.11s/it] 52%|█████▏ | 843/1610 [3:37:58<3:17:08, 15.42s/it] {'loss': 0.0006, 'grad_norm': 1.3574934220947585, 'learning_rate': 4.763975155279503e-07, 'completion_length': 173.04464721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.5625000596046448, 'reward_std': 0.35137079656124115, 'kl': 0.01568603515625, 'epoch': 2.62} 52%|█████▏ | 843/1610 [3:37:58<3:17:08, 15.42s/it] 52%|█████▏ | 844/1610 [3:38:12<3:11:24, 14.99s/it] {'loss': 0.0006, 'grad_norm': 1.8348974924830677, 'learning_rate': 4.7577639751552796e-07, 'completion_length': 130.69643783569336, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.2663346901535988, 'kl': 0.01470947265625, 'epoch': 2.62} 52%|█████▏ | 844/1610 [3:38:12<3:11:24, 14.99s/it] 52%|█████▏ | 845/1610 [3:38:26<3:06:50, 14.65s/it] {'loss': 0.0006, 'grad_norm': 2.7144122356618263, 'learning_rate': 4.751552795031056e-07, 'completion_length': 125.37500381469727, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857313156128, 'reward_std': 0.3111136555671692, 'kl': 0.014251708984375, 'epoch': 2.62} 52%|█████▏ | 845/1610 [3:38:26<3:06:50, 14.65s/it] 53%|█████▎ | 846/1610 [3:38:40<3:03:30, 14.41s/it] {'loss': 0.0005, 'grad_norm': 1.5008751722436544, 'learning_rate': 4.7453416149068323e-07, 'completion_length': 148.2678680419922, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.22875341773033142, 'kl': 0.01312255859375, 'epoch': 2.63} 53%|█████▎ | 846/1610 [3:38:40<3:03:30, 14.41s/it] 53%|█████▎ | 847/1610 [3:38:56<3:08:11, 14.80s/it] {'loss': 0.0005, 'grad_norm': 1.3345378542008515, 'learning_rate': 4.739130434782608e-07, 'completion_length': 146.3928680419922, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.32283326983451843, 'kl': 0.0133056640625, 'epoch': 2.63} 53%|█████▎ | 847/1610 [3:38:56<3:08:11, 14.80s/it] 53%|█████▎ | 848/1610 [3:39:10<3:07:35, 14.77s/it] {'loss': 0.0005, 'grad_norm': 1.3287505199569076, 'learning_rate': 4.732919254658385e-07, 'completion_length': 136.1964340209961, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2377769872546196, 'kl': 0.01361083984375, 'epoch': 2.63} 53%|█████▎ | 848/1610 [3:39:10<3:07:35, 14.77s/it] 53%|█████▎ | 849/1610 [3:39:24<3:01:31, 14.31s/it] {'loss': 0.0004, 'grad_norm': 1.7262162001120172, 'learning_rate': 4.7267080745341613e-07, 'completion_length': 123.77679061889648, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.12175281345844269, 'kl': 0.01123046875, 'epoch': 2.64} 53%|█████▎ | 849/1610 [3:39:24<3:01:31, 14.31s/it] 53%|█████▎ | 850/1610 [3:39:40<3:08:51, 14.91s/it] {'loss': 0.0006, 'grad_norm': 0.810159271203386, 'learning_rate': 4.7204968944099376e-07, 'completion_length': 168.9464340209961, 'rewards/accuracy_reward': 0.5803571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.22754103690385818, 'kl': 0.013946533203125, 'epoch': 2.64} 53%|█████▎ | 850/1610 [3:39:40<3:08:51, 14.91s/it] 53%|█████▎ | 851/1610 [3:39:53<3:03:26, 14.50s/it] {'loss': 0.0005, 'grad_norm': 1.307072204436159, 'learning_rate': 4.714285714285714e-07, 'completion_length': 125.43750381469727, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2579156309366226, 'kl': 0.012054443359375, 'epoch': 2.64} 53%|█████▎ | 851/1610 [3:39:53<3:03:26, 14.50s/it] 53%|█████▎ | 852/1610 [3:40:08<3:05:01, 14.65s/it] {'loss': 0.0006, 'grad_norm': 1.989930437710155, 'learning_rate': 4.70807453416149e-07, 'completion_length': 138.58929443359375, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.24619606137275696, 'kl': 0.014556884765625, 'epoch': 2.65} 53%|█████▎ | 852/1610 [3:40:08<3:05:01, 14.65s/it] 53%|█████▎ | 853/1610 [3:40:21<2:58:28, 14.15s/it] {'loss': 0.0006, 'grad_norm': 1.806716990376321, 'learning_rate': 4.701863354037267e-07, 'completion_length': 130.06250381469727, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.27926105260849, 'kl': 0.0159912109375, 'epoch': 2.65} 53%|█████▎ | 853/1610 [3:40:21<2:58:28, 14.15s/it] 53%|█████▎ | 854/1610 [3:40:37<3:04:25, 14.64s/it] {'loss': 0.0006, 'grad_norm': 1.1930960485821667, 'learning_rate': 4.6956521739130434e-07, 'completion_length': 169.2053680419922, 'rewards/accuracy_reward': 0.3750000298023224, 'rewards/format_reward': 1.0, 'reward': 1.3750001192092896, 'reward_std': 0.3486747741699219, 'kl': 0.01409912109375, 'epoch': 2.65} 53%|█████▎ | 854/1610 [3:40:37<3:04:25, 14.64s/it] 53%|█████▎ | 855/1610 [3:40:51<3:02:36, 14.51s/it] {'loss': 0.0005, 'grad_norm': 3.023935223846936, 'learning_rate': 4.68944099378882e-07, 'completion_length': 134.58929061889648, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.36491063237190247, 'kl': 0.013092041015625, 'epoch': 2.66} 53%|█████▎ | 855/1610 [3:40:51<3:02:36, 14.51s/it] 53%|█████▎ | 856/1610 [3:41:05<2:59:55, 14.32s/it] {'loss': 0.0005, 'grad_norm': 1.3635595949076782, 'learning_rate': 4.683229813664596e-07, 'completion_length': 138.83036041259766, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2540072351694107, 'kl': 0.0133056640625, 'epoch': 2.66} 53%|█████▎ | 856/1610 [3:41:05<2:59:55, 14.32s/it] 53%|█████▎ | 857/1610 [3:41:19<2:58:13, 14.20s/it] {'loss': 0.0005, 'grad_norm': 0.5036732525019604, 'learning_rate': 4.6770186335403724e-07, 'completion_length': 151.96428680419922, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.14970265328884125, 'kl': 0.011810302734375, 'epoch': 2.66} 53%|█████▎ | 857/1610 [3:41:19<2:58:13, 14.20s/it] 53%|█████▎ | 858/1610 [3:41:34<3:01:48, 14.51s/it] {'loss': 0.0007, 'grad_norm': 1.9007107606458493, 'learning_rate': 4.670807453416149e-07, 'completion_length': 164.9732208251953, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.45447421073913574, 'kl': 0.01788330078125, 'epoch': 2.66} 53%|█████▎ | 858/1610 [3:41:34<3:01:48, 14.51s/it] 53%|█████▎ | 859/1610 [3:41:48<2:58:45, 14.28s/it] {'loss': 0.0005, 'grad_norm': 0.7977689582801205, 'learning_rate': 4.664596273291925e-07, 'completion_length': 155.73214721679688, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.22094222903251648, 'kl': 0.0126953125, 'epoch': 2.67} 53%|█████▎ | 859/1610 [3:41:48<2:58:45, 14.28s/it] 53%|█████▎ | 860/1610 [3:42:02<2:57:36, 14.21s/it] {'loss': 0.0006, 'grad_norm': 1.3043420123942817, 'learning_rate': 4.6583850931677014e-07, 'completion_length': 152.71428680419922, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.260606050491333, 'kl': 0.01385498046875, 'epoch': 2.67} 53%|█████▎ | 860/1610 [3:42:02<2:57:36, 14.21s/it] 53%|█████▎ | 861/1610 [3:42:18<3:03:06, 14.67s/it] {'loss': 0.0006, 'grad_norm': 1.3055927301511838, 'learning_rate': 4.6521739130434777e-07, 'completion_length': 144.0982208251953, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.2907657027244568, 'kl': 0.01434326171875, 'epoch': 2.67} 53%|█████▎ | 861/1610 [3:42:18<3:03:06, 14.67s/it] 54%|█████▎ | 862/1610 [3:42:34<3:06:56, 15.00s/it] {'loss': 0.0005, 'grad_norm': 1.4809720708384377, 'learning_rate': 4.6459627329192546e-07, 'completion_length': 142.37500762939453, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5892857909202576, 'reward_std': 0.34376318752765656, 'kl': 0.013580322265625, 'epoch': 2.68} 54%|█████▎ | 862/1610 [3:42:34<3:06:56, 15.00s/it] 54%|█████▎ | 863/1610 [3:42:50<3:11:21, 15.37s/it] {'loss': 0.0006, 'grad_norm': 0.7906778643334766, 'learning_rate': 4.639751552795031e-07, 'completion_length': 177.27679443359375, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.22133180499076843, 'kl': 0.0147705078125, 'epoch': 2.68} 54%|█████▎ | 863/1610 [3:42:50<3:11:21, 15.37s/it] 54%|█████▎ | 864/1610 [3:43:04<3:04:58, 14.88s/it] {'loss': 0.0005, 'grad_norm': 3.1059648366822667, 'learning_rate': 4.633540372670807e-07, 'completion_length': 133.57143783569336, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.12054043635725975, 'kl': 0.012420654296875, 'epoch': 2.68} 54%|█████▎ | 864/1610 [3:43:04<3:04:58, 14.88s/it] 54%|█████▎ | 865/1610 [3:43:18<3:03:16, 14.76s/it] {'loss': 0.0005, 'grad_norm': 1.2021918834427212, 'learning_rate': 4.6273291925465835e-07, 'completion_length': 125.23214721679688, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3240400403738022, 'kl': 0.012451171875, 'epoch': 2.69} 54%|█████▎ | 865/1610 [3:43:18<3:03:16, 14.76s/it] 54%|█████▍ | 866/1610 [3:43:32<2:59:24, 14.47s/it] {'loss': 0.0006, 'grad_norm': 1.8857656230402318, 'learning_rate': 4.62111801242236e-07, 'completion_length': 144.98215103149414, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.2591223865747452, 'kl': 0.01422119140625, 'epoch': 2.69} 54%|█████▍ | 866/1610 [3:43:32<2:59:24, 14.47s/it] 54%|█████▍ | 867/1610 [3:43:47<3:01:11, 14.63s/it] {'loss': 0.0006, 'grad_norm': 3.075844776857986, 'learning_rate': 4.6149068322981367e-07, 'completion_length': 151.05358123779297, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3324703127145767, 'kl': 0.01483154296875, 'epoch': 2.69} 54%|█████▍ | 867/1610 [3:43:47<3:01:11, 14.63s/it] 54%|█████▍ | 868/1610 [3:44:03<3:05:05, 14.97s/it] {'loss': 0.0008, 'grad_norm': 0.9722251922896522, 'learning_rate': 4.608695652173913e-07, 'completion_length': 170.0714340209961, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.17434299737215042, 'kl': 0.01910400390625, 'epoch': 2.7} 54%|█████▍ | 868/1610 [3:44:03<3:05:05, 14.97s/it] 54%|█████▍ | 869/1610 [3:44:18<3:06:49, 15.13s/it] {'loss': 0.0008, 'grad_norm': 2.0588763624500204, 'learning_rate': 4.6024844720496894e-07, 'completion_length': 157.6964340209961, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.2546207010746002, 'kl': 0.01922607421875, 'epoch': 2.7} 54%|█████▍ | 869/1610 [3:44:18<3:06:49, 15.13s/it] 54%|█████▍ | 870/1610 [3:44:32<3:00:02, 14.60s/it] {'loss': 0.0005, 'grad_norm': 1.398719261598101, 'learning_rate': 4.596273291925465e-07, 'completion_length': 128.52678680419922, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.24889205396175385, 'kl': 0.011444091796875, 'epoch': 2.7} 54%|█████▍ | 870/1610 [3:44:32<3:00:02, 14.60s/it] 54%|█████▍ | 871/1610 [3:44:44<2:52:36, 14.01s/it] {'loss': 0.0005, 'grad_norm': 1.4759360669902712, 'learning_rate': 4.590062111801242e-07, 'completion_length': 132.39286041259766, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.30330249667167664, 'kl': 0.013580322265625, 'epoch': 2.7} 54%|█████▍ | 871/1610 [3:44:44<2:52:36, 14.01s/it] 54%|█████▍ | 872/1610 [3:44:59<2:55:01, 14.23s/it] {'loss': 0.0006, 'grad_norm': 1.9355560136367338, 'learning_rate': 4.5838509316770183e-07, 'completion_length': 137.71428680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.30122554302215576, 'kl': 0.01446533203125, 'epoch': 2.71} 54%|█████▍ | 872/1610 [3:44:59<2:55:01, 14.23s/it] 54%|█████▍ | 873/1610 [3:45:14<2:56:51, 14.40s/it] {'loss': 0.0006, 'grad_norm': 2.0455044069340227, 'learning_rate': 4.5776397515527947e-07, 'completion_length': 139.17857360839844, 'rewards/accuracy_reward': 0.7232142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.3823588639497757, 'kl': 0.015380859375, 'epoch': 2.71} 54%|█████▍ | 873/1610 [3:45:14<2:56:51, 14.40s/it] 54%|█████▍ | 874/1610 [3:45:27<2:51:50, 14.01s/it] {'loss': 0.0006, 'grad_norm': 1.5670976394258562, 'learning_rate': 4.571428571428571e-07, 'completion_length': 133.57143020629883, 'rewards/accuracy_reward': 0.7678571939468384, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.24619604647159576, 'kl': 0.013946533203125, 'epoch': 2.71} 54%|█████▍ | 874/1610 [3:45:27<2:51:50, 14.01s/it] 54%|█████▍ | 875/1610 [3:45:42<2:56:30, 14.41s/it] {'loss': 0.0007, 'grad_norm': 1.4917718687821795, 'learning_rate': 4.5652173913043473e-07, 'completion_length': 126.0714340209961, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.26841723918914795, 'kl': 0.017059326171875, 'epoch': 2.72} 54%|█████▍ | 875/1610 [3:45:42<2:56:30, 14.41s/it] 54%|█████▍ | 876/1610 [3:45:57<2:56:16, 14.41s/it] {'loss': 0.0006, 'grad_norm': 1.021143736736134, 'learning_rate': 4.559006211180124e-07, 'completion_length': 155.68750762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2404673844575882, 'kl': 0.015655517578125, 'epoch': 2.72} 54%|█████▍ | 876/1610 [3:45:57<2:56:16, 14.41s/it] 54%|█████▍ | 877/1610 [3:46:11<2:55:42, 14.38s/it] {'loss': 0.0004, 'grad_norm': 2.243241702487263, 'learning_rate': 4.5527950310559005e-07, 'completion_length': 132.71429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3285619020462036, 'kl': 0.01007080078125, 'epoch': 2.72} 54%|█████▍ | 877/1610 [3:46:11<2:55:42, 14.38s/it] 55%|█████▍ | 878/1610 [3:46:25<2:55:33, 14.39s/it] {'loss': 0.0009, 'grad_norm': 1.575247270726541, 'learning_rate': 4.546583850931677e-07, 'completion_length': 156.30357360839844, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.4628932625055313, 'kl': 0.0225830078125, 'epoch': 2.73} 55%|█████▍ | 878/1610 [3:46:25<2:55:33, 14.39s/it] 55%|█████▍ | 879/1610 [3:46:41<2:58:52, 14.68s/it] {'loss': 0.0007, 'grad_norm': 0.8102972148616397, 'learning_rate': 4.540372670807453e-07, 'completion_length': 165.63394165039062, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.571428656578064, 'reward_std': 0.2527948468923569, 'kl': 0.017730712890625, 'epoch': 2.73} 55%|█████▍ | 879/1610 [3:46:41<2:58:52, 14.68s/it] 55%|█████▍ | 880/1610 [3:46:55<2:58:40, 14.69s/it] {'loss': 0.0006, 'grad_norm': 2.1190586156290148, 'learning_rate': 4.53416149068323e-07, 'completion_length': 157.77679061889648, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3571050316095352, 'kl': 0.014739990234375, 'epoch': 2.73} 55%|█████▍ | 880/1610 [3:46:55<2:58:40, 14.69s/it] 55%|█████▍ | 881/1610 [3:47:12<3:05:34, 15.27s/it] {'loss': 0.0008, 'grad_norm': 0.952798814370289, 'learning_rate': 4.5279503105590063e-07, 'completion_length': 196.06251525878906, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4107143878936768, 'reward_std': 0.2997836768627167, 'kl': 0.0196533203125, 'epoch': 2.74} 55%|█████▍ | 881/1610 [3:47:12<3:05:34, 15.27s/it] 55%|█████▍ | 882/1610 [3:47:27<3:03:24, 15.12s/it] {'loss': 0.0006, 'grad_norm': 3.746414190211265, 'learning_rate': 4.521739130434782e-07, 'completion_length': 138.08036041259766, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.16323687136173248, 'kl': 0.015228271484375, 'epoch': 2.74} 55%|█████▍ | 882/1610 [3:47:27<3:03:24, 15.12s/it] 55%|█████▍ | 883/1610 [3:47:42<3:04:41, 15.24s/it] {'loss': 0.0006, 'grad_norm': 0.9529301334418047, 'learning_rate': 4.5155279503105585e-07, 'completion_length': 151.49108123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.25791002810001373, 'kl': 0.01434326171875, 'epoch': 2.74} 55%|█████▍ | 883/1610 [3:47:42<3:04:41, 15.24s/it] 55%|█████▍ | 884/1610 [3:47:58<3:06:17, 15.40s/it] {'loss': 0.0007, 'grad_norm': 2.106568628933158, 'learning_rate': 4.509316770186335e-07, 'completion_length': 168.0803680419922, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375001192092896, 'reward_std': 0.3492938429117203, 'kl': 0.01824951171875, 'epoch': 2.75} 55%|█████▍ | 884/1610 [3:47:58<3:06:17, 15.40s/it] 55%|█████▍ | 885/1610 [3:48:12<3:01:41, 15.04s/it] {'loss': 0.0005, 'grad_norm': 4.182771998336851, 'learning_rate': 4.5031055900621116e-07, 'completion_length': 140.28571701049805, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.2975682318210602, 'kl': 0.011993408203125, 'epoch': 2.75} 55%|█████▍ | 885/1610 [3:48:12<3:01:41, 15.04s/it] 55%|█████▌ | 886/1610 [3:48:26<2:58:01, 14.75s/it] {'loss': 0.0006, 'grad_norm': 1.5590122141081615, 'learning_rate': 4.496894409937888e-07, 'completion_length': 153.58036041259766, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.3144030123949051, 'kl': 0.016021728515625, 'epoch': 2.75} 55%|█████▌ | 886/1610 [3:48:26<2:58:01, 14.75s/it] 55%|█████▌ | 887/1610 [3:48:42<3:01:03, 15.03s/it] {'loss': 0.0005, 'grad_norm': 1.1040089137290796, 'learning_rate': 4.4906832298136643e-07, 'completion_length': 141.16965103149414, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553571939468384, 'reward_std': 0.2254640907049179, 'kl': 0.013397216796875, 'epoch': 2.75} 55%|█████▌ | 887/1610 [3:48:42<3:01:03, 15.03s/it] 55%|█████▌ | 888/1610 [3:48:57<3:00:51, 15.03s/it] {'loss': 0.0008, 'grad_norm': 1.1678651603756658, 'learning_rate': 4.4844720496894406e-07, 'completion_length': 176.36608123779297, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.37845607101917267, 'kl': 0.01995849609375, 'epoch': 2.76} 55%|█████▌ | 888/1610 [3:48:57<3:00:51, 15.03s/it] 55%|█████▌ | 889/1610 [3:49:12<2:59:00, 14.90s/it] {'loss': 0.0006, 'grad_norm': 0.8531799632679841, 'learning_rate': 4.4782608695652175e-07, 'completion_length': 148.58036041259766, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.24889205396175385, 'kl': 0.01416015625, 'epoch': 2.76} 55%|█████▌ | 889/1610 [3:49:12<2:59:00, 14.90s/it] 55%|█████▌ | 890/1610 [3:49:27<2:59:27, 14.95s/it] {'loss': 0.0006, 'grad_norm': 3.8728420071429768, 'learning_rate': 4.472049689440994e-07, 'completion_length': 153.83036041259766, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.3207361698150635, 'kl': 0.013824462890625, 'epoch': 2.76} 55%|█████▌ | 890/1610 [3:49:27<2:59:27, 14.95s/it] 55%|█████▌ | 891/1610 [3:49:42<3:01:56, 15.18s/it] {'loss': 0.0005, 'grad_norm': 0.703188507093194, 'learning_rate': 4.46583850931677e-07, 'completion_length': 132.0178680419922, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.11394161731004715, 'kl': 0.0120849609375, 'epoch': 2.77} 55%|█████▌ | 891/1610 [3:49:42<3:01:56, 15.18s/it] 55%|█████▌ | 892/1610 [3:49:58<3:02:10, 15.22s/it] {'loss': 0.0006, 'grad_norm': 1.1758434290162918, 'learning_rate': 4.4596273291925464e-07, 'completion_length': 150.52678680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160715222358704, 'reward_std': 0.2086147665977478, 'kl': 0.015106201171875, 'epoch': 2.77} 55%|█████▌ | 892/1610 [3:49:58<3:02:10, 15.22s/it] 55%|█████▌ | 893/1610 [3:50:13<3:02:45, 15.29s/it] {'loss': 0.0005, 'grad_norm': 0.9665623074965144, 'learning_rate': 4.453416149068323e-07, 'completion_length': 168.81250762939453, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.3571050316095352, 'kl': 0.01324462890625, 'epoch': 2.77} 55%|█████▌ | 893/1610 [3:50:13<3:02:45, 15.29s/it] 56%|█████▌ | 894/1610 [3:50:28<2:59:30, 15.04s/it] {'loss': 0.0005, 'grad_norm': 2.25265615608245, 'learning_rate': 4.447204968944099e-07, 'completion_length': 144.24108123779297, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857313156128, 'reward_std': 0.2675470560789108, 'kl': 0.01275634765625, 'epoch': 2.78} 56%|█████▌ | 894/1610 [3:50:28<2:59:30, 15.04s/it] 56%|█████▌ | 895/1610 [3:50:40<2:49:38, 14.24s/it] {'loss': 0.0005, 'grad_norm': 0.8401303201365168, 'learning_rate': 4.4409937888198754e-07, 'completion_length': 113.0089340209961, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.09528662264347076, 'kl': 0.011260986328125, 'epoch': 2.78} 56%|█████▌ | 895/1610 [3:50:40<2:49:38, 14.24s/it] 56%|█████▌ | 896/1610 [3:50:56<2:53:49, 14.61s/it] {'loss': 0.0006, 'grad_norm': 1.5785264645819541, 'learning_rate': 4.434782608695652e-07, 'completion_length': 157.20536041259766, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.21972985565662384, 'kl': 0.0150146484375, 'epoch': 2.78} 56%|█████▌ | 896/1610 [3:50:56<2:53:49, 14.61s/it] 56%|█████▌ | 897/1610 [3:51:11<2:57:18, 14.92s/it] {'loss': 0.0007, 'grad_norm': 2.058616038354498, 'learning_rate': 4.428571428571428e-07, 'completion_length': 189.3839340209961, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4375000596046448, 'reward_std': 0.2987271398305893, 'kl': 0.017669677734375, 'epoch': 2.79} 56%|█████▌ | 897/1610 [3:51:11<2:57:18, 14.92s/it] 56%|█████▌ | 898/1610 [3:51:25<2:51:31, 14.45s/it] {'loss': 0.0005, 'grad_norm': 0.9360256611539622, 'learning_rate': 4.422360248447205e-07, 'completion_length': 122.66071701049805, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.17495645582675934, 'kl': 0.011566162109375, 'epoch': 2.79} 56%|█████▌ | 898/1610 [3:51:25<2:51:31, 14.45s/it] 56%|█████▌ | 899/1610 [3:51:38<2:47:02, 14.10s/it] {'loss': 0.0006, 'grad_norm': 1.3000387912800855, 'learning_rate': 4.416149068322981e-07, 'completion_length': 141.5357208251953, 'rewards/accuracy_reward': 0.6785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.21703943610191345, 'kl': 0.014312744140625, 'epoch': 2.79} 56%|█████▌ | 899/1610 [3:51:38<2:47:02, 14.10s/it] 56%|█████▌ | 900/1610 [3:51:53<2:49:52, 14.36s/it] {'loss': 0.0007, 'grad_norm': 2.1758782986831138, 'learning_rate': 4.4099378881987576e-07, 'completion_length': 159.51786041259766, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857313156128, 'reward_std': 0.4109020084142685, 'kl': 0.017791748046875, 'epoch': 2.8} 56%|█████▌ | 900/1610 [3:51:53<2:49:52, 14.36s/it] 56%|█████▌ | 901/1610 [3:52:58<5:50:31, 29.66s/it] {'loss': 0.0007, 'grad_norm': 1.1477364527922573, 'learning_rate': 4.403726708074534e-07, 'completion_length': 160.79464721679688, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.3189248740673065, 'kl': 0.01715087890625, 'epoch': 2.8} 56%|█████▌ | 901/1610 [3:52:58<5:50:31, 29.66s/it] 56%|█████▌ | 902/1610 [3:53:14<5:00:00, 25.42s/it] {'loss': 0.0006, 'grad_norm': 1.117757469024758, 'learning_rate': 4.39751552795031e-07, 'completion_length': 171.15179443359375, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2314438372850418, 'kl': 0.015289306640625, 'epoch': 2.8} 56%|█████▌ | 902/1610 [3:53:14<5:00:00, 25.42s/it] 56%|█████▌ | 903/1610 [3:53:27<4:15:48, 21.71s/it] {'loss': 0.0005, 'grad_norm': 1.8068756999972746, 'learning_rate': 4.391304347826087e-07, 'completion_length': 124.75000762939453, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.30963002145290375, 'kl': 0.01318359375, 'epoch': 2.8} 56%|█████▌ | 903/1610 [3:53:27<4:15:48, 21.71s/it] 56%|█████▌ | 904/1610 [3:53:40<3:46:40, 19.26s/it] {'loss': 0.0006, 'grad_norm': 1.6551605160142617, 'learning_rate': 4.3850931677018634e-07, 'completion_length': 158.61608123779297, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.31561537086963654, 'kl': 0.014556884765625, 'epoch': 2.81} 56%|█████▌ | 904/1610 [3:53:40<3:46:40, 19.26s/it] 56%|█████▌ | 905/1610 [3:53:54<3:26:22, 17.56s/it] {'loss': 0.0004, 'grad_norm': 1.131847317278341, 'learning_rate': 4.3788819875776397e-07, 'completion_length': 133.85715103149414, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3525831997394562, 'kl': 0.011199951171875, 'epoch': 2.81} 56%|█████▌ | 905/1610 [3:53:54<3:26:22, 17.56s/it] 56%|█████▋ | 906/1610 [3:54:08<3:15:02, 16.62s/it] {'loss': 0.0007, 'grad_norm': 1.1332664002982782, 'learning_rate': 4.3726708074534155e-07, 'completion_length': 173.83929443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3213496208190918, 'kl': 0.017547607421875, 'epoch': 2.81} 56%|█████▋ | 906/1610 [3:54:08<3:15:02, 16.62s/it] 56%|█████▋ | 907/1610 [3:54:22<3:06:06, 15.88s/it] {'loss': 0.0006, 'grad_norm': 1.6486062402619532, 'learning_rate': 4.3664596273291924e-07, 'completion_length': 151.90179443359375, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2888835370540619, 'kl': 0.014373779296875, 'epoch': 2.82} 56%|█████▋ | 907/1610 [3:54:22<3:06:06, 15.88s/it] 56%|█████▋ | 908/1610 [3:54:37<3:01:16, 15.49s/it] {'loss': 0.0005, 'grad_norm': 1.6704733551599658, 'learning_rate': 4.3602484472049687e-07, 'completion_length': 142.77679061889648, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857313156128, 'reward_std': 0.26450884342193604, 'kl': 0.012939453125, 'epoch': 2.82} 56%|█████▋ | 908/1610 [3:54:37<3:01:16, 15.49s/it] 56%|█████▋ | 909/1610 [3:54:53<3:01:37, 15.55s/it] {'loss': 0.0008, 'grad_norm': 1.222236112329164, 'learning_rate': 4.354037267080745e-07, 'completion_length': 164.05357360839844, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3839285969734192, 'reward_std': 0.35831741988658905, 'kl': 0.0198974609375, 'epoch': 2.82} 56%|█████▋ | 909/1610 [3:54:53<3:01:37, 15.55s/it] 57%|█████▋ | 910/1610 [3:55:09<3:02:47, 15.67s/it] {'loss': 0.0007, 'grad_norm': 0.7745595268768901, 'learning_rate': 4.3478260869565214e-07, 'completion_length': 171.5982208251953, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285715222358704, 'reward_std': 0.20961220562458038, 'kl': 0.01715087890625, 'epoch': 2.83} 57%|█████▋ | 910/1610 [3:55:09<3:02:47, 15.67s/it] 57%|█████▋ | 911/1610 [3:55:23<2:58:34, 15.33s/it] {'loss': 0.0005, 'grad_norm': 1.3101067040467715, 'learning_rate': 4.3416149068322977e-07, 'completion_length': 146.27679443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2410808652639389, 'kl': 0.012176513671875, 'epoch': 2.83} 57%|█████▋ | 911/1610 [3:55:23<2:58:34, 15.33s/it] 57%|█████▋ | 912/1610 [3:55:38<2:57:49, 15.29s/it] {'loss': 0.0006, 'grad_norm': 2.3760254557317335, 'learning_rate': 4.3354037267080745e-07, 'completion_length': 134.71429061889648, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.32729044556617737, 'kl': 0.015533447265625, 'epoch': 2.83} 57%|█████▋ | 912/1610 [3:55:38<2:57:49, 15.29s/it] 57%|█████▋ | 913/1610 [3:55:52<2:52:41, 14.87s/it] {'loss': 0.0006, 'grad_norm': 1.5228163754798525, 'learning_rate': 4.329192546583851e-07, 'completion_length': 130.77678680419922, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.29098062217235565, 'kl': 0.015625, 'epoch': 2.84} 57%|█████▋ | 913/1610 [3:55:52<2:52:41, 14.87s/it] 57%|█████▋ | 914/1610 [3:56:08<2:54:44, 15.06s/it] {'loss': 0.0006, 'grad_norm': 2.13978162421486, 'learning_rate': 4.322981366459627e-07, 'completion_length': 151.43750762939453, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910715222358704, 'reward_std': 0.2500988245010376, 'kl': 0.015228271484375, 'epoch': 2.84} 57%|█████▋ | 914/1610 [3:56:08<2:54:44, 15.06s/it] 57%|█████▋ | 915/1610 [3:56:22<2:52:56, 14.93s/it] {'loss': 0.0006, 'grad_norm': 1.1691166261670118, 'learning_rate': 4.3167701863354035e-07, 'completion_length': 165.5714340209961, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.33654530346393585, 'kl': 0.015777587890625, 'epoch': 2.84} 57%|█████▋ | 915/1610 [3:56:22<2:52:56, 14.93s/it] 57%|█████▋ | 916/1610 [3:56:36<2:47:27, 14.48s/it] {'loss': 0.0005, 'grad_norm': 1.731740474103119, 'learning_rate': 4.3105590062111804e-07, 'completion_length': 122.18750381469727, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.27414587140083313, 'kl': 0.01220703125, 'epoch': 2.84} 57%|█████▋ | 916/1610 [3:56:36<2:47:27, 14.48s/it] 57%|█████▋ | 917/1610 [3:56:51<2:49:22, 14.66s/it] {'loss': 0.0006, 'grad_norm': 0.8194296438500557, 'learning_rate': 4.3043478260869567e-07, 'completion_length': 155.06250762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.2702374905347824, 'kl': 0.014617919921875, 'epoch': 2.85} 57%|█████▋ | 917/1610 [3:56:51<2:49:22, 14.66s/it] 57%|█████▋ | 918/1610 [3:57:04<2:45:18, 14.33s/it] {'loss': 0.0005, 'grad_norm': 5.7819533673518, 'learning_rate': 4.2981366459627325e-07, 'completion_length': 129.48214721679688, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.24889206886291504, 'kl': 0.013336181640625, 'epoch': 2.85} 57%|█████▋ | 918/1610 [3:57:04<2:45:18, 14.33s/it] 57%|█████▋ | 919/1610 [3:57:20<2:50:00, 14.76s/it] {'loss': 0.0007, 'grad_norm': 1.6755337477912027, 'learning_rate': 4.291925465838509e-07, 'completion_length': 146.00000762939453, 'rewards/accuracy_reward': 0.366071455180645, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.2831638306379318, 'kl': 0.016357421875, 'epoch': 2.85} 57%|█████▋ | 919/1610 [3:57:20<2:50:00, 14.76s/it] 57%|█████▋ | 920/1610 [3:57:34<2:47:10, 14.54s/it] {'loss': 0.0006, 'grad_norm': 1.5218667920769033, 'learning_rate': 4.285714285714285e-07, 'completion_length': 147.42857360839844, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.30390138924121857, 'kl': 0.014495849609375, 'epoch': 2.86} 57%|█████▋ | 920/1610 [3:57:34<2:47:10, 14.54s/it] 57%|█████▋ | 921/1610 [3:57:48<2:45:37, 14.42s/it] {'loss': 0.0005, 'grad_norm': 1.455699505835357, 'learning_rate': 4.279503105590062e-07, 'completion_length': 136.65179443359375, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.1379830539226532, 'kl': 0.013214111328125, 'epoch': 2.86} 57%|█████▋ | 921/1610 [3:57:48<2:45:37, 14.42s/it] 57%|█████▋ | 922/1610 [3:58:02<2:41:51, 14.12s/it] {'loss': 0.0006, 'grad_norm': 1.3109837654986376, 'learning_rate': 4.2732919254658383e-07, 'completion_length': 139.51786041259766, 'rewards/accuracy_reward': 0.8214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.821428656578064, 'reward_std': 0.20801585912704468, 'kl': 0.015228271484375, 'epoch': 2.86} 57%|█████▋ | 922/1610 [3:58:02<2:41:51, 14.12s/it] 57%|█████▋ | 923/1610 [3:58:17<2:46:46, 14.56s/it] {'loss': 0.0006, 'grad_norm': 1.7245659548966439, 'learning_rate': 4.2670807453416146e-07, 'completion_length': 175.92858123779297, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.2657212167978287, 'kl': 0.01519775390625, 'epoch': 2.87} 57%|█████▋ | 923/1610 [3:58:17<2:46:46, 14.56s/it] 57%|█████▋ | 924/1610 [3:58:34<2:53:51, 15.21s/it] {'loss': 0.0006, 'grad_norm': 1.1308924728716805, 'learning_rate': 4.260869565217391e-07, 'completion_length': 183.6696548461914, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.3057272285223007, 'kl': 0.01495361328125, 'epoch': 2.87} 57%|█████▋ | 924/1610 [3:58:34<2:53:51, 15.21s/it] 57%|█████▋ | 925/1610 [3:58:49<2:52:14, 15.09s/it] {'loss': 0.0007, 'grad_norm': 1.5007596480601013, 'learning_rate': 4.254658385093168e-07, 'completion_length': 137.1339340209961, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.28707222640514374, 'kl': 0.016265869140625, 'epoch': 2.87} 57%|█████▋ | 925/1610 [3:58:49<2:52:14, 15.09s/it] 58%|█████▊ | 926/1610 [3:59:04<2:52:18, 15.11s/it] {'loss': 0.0006, 'grad_norm': 1.6791980980396535, 'learning_rate': 4.248447204968944e-07, 'completion_length': 155.33036041259766, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.26841723918914795, 'kl': 0.015960693359375, 'epoch': 2.88} 58%|█████▊ | 926/1610 [3:59:04<2:52:18, 15.11s/it] 58%|█████▊ | 927/1610 [3:59:19<2:52:29, 15.15s/it] {'loss': 0.0007, 'grad_norm': 3.076333265113054, 'learning_rate': 4.2422360248447205e-07, 'completion_length': 156.87500762939453, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.35440902411937714, 'kl': 0.01690673828125, 'epoch': 2.88} 58%|█████▊ | 927/1610 [3:59:19<2:52:29, 15.15s/it] 58%|█████▊ | 928/1610 [3:59:35<2:52:38, 15.19s/it] {'loss': 0.0007, 'grad_norm': 2.3982805969412215, 'learning_rate': 4.236024844720497e-07, 'completion_length': 180.81250762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.16141663491725922, 'kl': 0.018310546875, 'epoch': 2.88} 58%|█████▊ | 928/1610 [3:59:35<2:52:38, 15.19s/it] 58%|█████▊ | 929/1610 [3:59:50<2:53:09, 15.26s/it] {'loss': 0.0005, 'grad_norm': 1.1893989393863709, 'learning_rate': 4.229813664596273e-07, 'completion_length': 149.17857360839844, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.18397442996501923, 'kl': 0.01220703125, 'epoch': 2.89} 58%|█████▊ | 929/1610 [3:59:50<2:53:09, 15.26s/it] 58%|█████▊ | 930/1610 [4:00:03<2:46:39, 14.71s/it] {'loss': 0.0006, 'grad_norm': 1.4103359673424318, 'learning_rate': 4.2236024844720495e-07, 'completion_length': 130.12500762939453, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2696240097284317, 'kl': 0.0152587890625, 'epoch': 2.89} 58%|█████▊ | 930/1610 [4:00:03<2:46:39, 14.71s/it] 58%|█████▊ | 931/1610 [4:00:19<2:49:41, 14.99s/it] {'loss': 0.0005, 'grad_norm': 0.9729137835192722, 'learning_rate': 4.217391304347826e-07, 'completion_length': 155.62500762939453, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.22606300562620163, 'kl': 0.01336669921875, 'epoch': 2.89} 58%|█████▊ | 931/1610 [4:00:19<2:49:41, 14.99s/it] 58%|█████▊ | 932/1610 [4:00:35<2:52:27, 15.26s/it] {'loss': 0.0007, 'grad_norm': 1.2584470389323263, 'learning_rate': 4.211180124223602e-07, 'completion_length': 168.42858123779297, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.27083639800548553, 'kl': 0.01763916015625, 'epoch': 2.89} 58%|█████▊ | 932/1610 [4:00:35<2:52:27, 15.26s/it] 58%|█████▊ | 933/1610 [4:00:50<2:52:08, 15.26s/it] {'loss': 0.0007, 'grad_norm': 2.234029759113896, 'learning_rate': 4.2049689440993784e-07, 'completion_length': 146.65179443359375, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2467949464917183, 'kl': 0.017303466796875, 'epoch': 2.9} 58%|█████▊ | 933/1610 [4:00:50<2:52:08, 15.26s/it] 58%|█████▊ | 934/1610 [4:01:05<2:49:26, 15.04s/it] {'loss': 0.0007, 'grad_norm': 1.0556478925467445, 'learning_rate': 4.1987577639751553e-07, 'completion_length': 145.17858123779297, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3306388556957245, 'kl': 0.01806640625, 'epoch': 2.9} 58%|█████▊ | 934/1610 [4:01:05<2:49:26, 15.04s/it] 58%|█████▊ | 935/1610 [4:01:19<2:45:36, 14.72s/it] {'loss': 0.0007, 'grad_norm': 1.3081923931754158, 'learning_rate': 4.1925465838509316e-07, 'completion_length': 163.1339340209961, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.32013723254203796, 'kl': 0.0172119140625, 'epoch': 2.9} 58%|█████▊ | 935/1610 [4:01:19<2:45:36, 14.72s/it] 58%|█████▊ | 936/1610 [4:01:32<2:41:34, 14.38s/it] {'loss': 0.0005, 'grad_norm': 1.536877808606144, 'learning_rate': 4.186335403726708e-07, 'completion_length': 134.43750762939453, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.370031401515007, 'kl': 0.012237548828125, 'epoch': 2.91} 58%|█████▊ | 936/1610 [4:01:32<2:41:34, 14.38s/it] 58%|█████▊ | 937/1610 [4:01:48<2:45:13, 14.73s/it] {'loss': 0.0006, 'grad_norm': 2.1881285487939617, 'learning_rate': 4.180124223602484e-07, 'completion_length': 162.8571548461914, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3026890307664871, 'kl': 0.0162353515625, 'epoch': 2.91} 58%|█████▊ | 937/1610 [4:01:48<2:45:13, 14.73s/it] 58%|█████▊ | 938/1610 [4:02:04<2:48:49, 15.07s/it] {'loss': 0.0007, 'grad_norm': 0.8359310003718171, 'learning_rate': 4.1739130434782606e-07, 'completion_length': 189.38394165039062, 'rewards/accuracy_reward': 0.5803571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3186536133289337, 'kl': 0.016876220703125, 'epoch': 2.91} 58%|█████▊ | 938/1610 [4:02:04<2:48:49, 15.07s/it] 58%|█████▊ | 939/1610 [4:02:19<2:49:16, 15.14s/it] {'loss': 0.0006, 'grad_norm': 2.559254505447514, 'learning_rate': 4.1677018633540374e-07, 'completion_length': 150.2410774230957, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6250001192092896, 'reward_std': 0.23981501162052155, 'kl': 0.014617919921875, 'epoch': 2.92} 58%|█████▊ | 939/1610 [4:02:19<2:49:16, 15.14s/it] 58%|█████▊ | 940/1610 [4:02:35<2:51:45, 15.38s/it] {'loss': 0.0006, 'grad_norm': 1.3829850044835141, 'learning_rate': 4.161490683229814e-07, 'completion_length': 151.74108123779297, 'rewards/accuracy_reward': 0.625, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160715222358704, 'reward_std': 0.24883297085762024, 'kl': 0.015594482421875, 'epoch': 2.92} 58%|█████▊ | 940/1610 [4:02:35<2:51:45, 15.38s/it] 58%|█████▊ | 941/1610 [4:02:50<2:51:26, 15.38s/it] {'loss': 0.0006, 'grad_norm': 1.28009195586757, 'learning_rate': 4.15527950310559e-07, 'completion_length': 159.4107208251953, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.349907323718071, 'kl': 0.01556396484375, 'epoch': 2.92} 58%|█████▊ | 941/1610 [4:02:50<2:51:26, 15.38s/it] 59%|█████▊ | 942/1610 [4:03:07<2:53:47, 15.61s/it] {'loss': 0.0007, 'grad_norm': 2.5053096958553396, 'learning_rate': 4.149068322981366e-07, 'completion_length': 160.33929443359375, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6250000596046448, 'reward_std': 0.35668395459651947, 'kl': 0.01800537109375, 'epoch': 2.93} 59%|█████▊ | 942/1610 [4:03:07<2:53:47, 15.61s/it] 59%|█████▊ | 943/1610 [4:03:22<2:52:29, 15.52s/it] {'loss': 0.0006, 'grad_norm': 1.3365071928311894, 'learning_rate': 4.142857142857143e-07, 'completion_length': 181.50000762939453, 'rewards/accuracy_reward': 0.3482142984867096, 'rewards/format_reward': 1.0, 'reward': 1.348214328289032, 'reward_std': 0.24229325354099274, 'kl': 0.015167236328125, 'epoch': 2.93} 59%|█████▊ | 943/1610 [4:03:22<2:52:29, 15.52s/it] 59%|█████▊ | 944/1610 [4:03:37<2:52:34, 15.55s/it] {'loss': 0.0006, 'grad_norm': 1.5452046258161072, 'learning_rate': 4.136645962732919e-07, 'completion_length': 139.2232208251953, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.473214328289032, 'reward_std': 0.3394418805837631, 'kl': 0.0142822265625, 'epoch': 2.93} 59%|█████▊ | 944/1610 [4:03:37<2:52:34, 15.55s/it] 59%|█████▊ | 945/1610 [4:03:53<2:53:14, 15.63s/it] {'loss': 0.0006, 'grad_norm': 2.1615587627827173, 'learning_rate': 4.1304347826086954e-07, 'completion_length': 169.6785774230957, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4910715222358704, 'reward_std': 0.3441786766052246, 'kl': 0.015472412109375, 'epoch': 2.93} 59%|█████▊ | 945/1610 [4:03:53<2:53:14, 15.63s/it] 59%|█████▉ | 946/1610 [4:04:10<2:55:30, 15.86s/it] {'loss': 0.0007, 'grad_norm': 0.9796482434479623, 'learning_rate': 4.1242236024844717e-07, 'completion_length': 172.75894165039062, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.36760663986206055, 'kl': 0.018402099609375, 'epoch': 2.94} 59%|█████▉ | 946/1610 [4:04:10<2:55:30, 15.86s/it] 59%|█████▉ | 947/1610 [4:04:25<2:52:18, 15.59s/it] {'loss': 0.0006, 'grad_norm': 1.619604201356156, 'learning_rate': 4.118012422360248e-07, 'completion_length': 152.01786041259766, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.24619604647159576, 'kl': 0.0147705078125, 'epoch': 2.94} 59%|█████▉ | 947/1610 [4:04:25<2:52:18, 15.59s/it] 59%|█████▉ | 948/1610 [4:04:39<2:46:15, 15.07s/it] {'loss': 0.0006, 'grad_norm': 1.1008416763726876, 'learning_rate': 4.111801242236025e-07, 'completion_length': 122.16072082519531, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.24949097633361816, 'kl': 0.01416015625, 'epoch': 2.94} 59%|█████▉ | 948/1610 [4:04:39<2:46:15, 15.07s/it] 59%|█████▉ | 949/1610 [4:04:55<2:49:20, 15.37s/it] {'loss': 0.0007, 'grad_norm': 1.1490278553225683, 'learning_rate': 4.105590062111801e-07, 'completion_length': 176.36608123779297, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3751465678215027, 'kl': 0.01788330078125, 'epoch': 2.95} 59%|█████▉ | 949/1610 [4:04:55<2:49:20, 15.37s/it] 59%|█████▉ | 950/1610 [4:05:10<2:50:03, 15.46s/it] {'loss': 0.0007, 'grad_norm': 1.206831213052183, 'learning_rate': 4.0993788819875776e-07, 'completion_length': 180.17858123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.3183113932609558, 'kl': 0.01629638671875, 'epoch': 2.95} 59%|█████▉ | 950/1610 [4:05:10<2:50:03, 15.46s/it] 59%|█████▉ | 951/1610 [4:05:24<2:44:43, 15.00s/it] {'loss': 0.0007, 'grad_norm': 1.7421579202809265, 'learning_rate': 4.093167701863354e-07, 'completion_length': 145.93750762939453, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.24290670454502106, 'kl': 0.01739501953125, 'epoch': 2.95} 59%|█████▉ | 951/1610 [4:05:24<2:44:43, 15.00s/it] 59%|█████▉ | 952/1610 [4:05:39<2:45:28, 15.09s/it] {'loss': 0.0007, 'grad_norm': 0.8709232264657406, 'learning_rate': 4.0869565217391307e-07, 'completion_length': 185.8303680419922, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.26181842386722565, 'kl': 0.01812744140625, 'epoch': 2.96} 59%|█████▉ | 952/1610 [4:05:39<2:45:28, 15.09s/it] 59%|█████▉ | 953/1610 [4:05:54<2:42:19, 14.82s/it] {'loss': 0.0007, 'grad_norm': 1.665821735653527, 'learning_rate': 4.080745341614907e-07, 'completion_length': 149.08036041259766, 'rewards/accuracy_reward': 0.5178571492433548, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.2675470560789108, 'kl': 0.0181884765625, 'epoch': 2.96} 59%|█████▉ | 953/1610 [4:05:54<2:42:19, 14.82s/it] 59%|█████▉ | 954/1610 [4:06:10<2:46:56, 15.27s/it] {'loss': 0.0008, 'grad_norm': 1.0176098378992577, 'learning_rate': 4.074534161490683e-07, 'completion_length': 187.27679443359375, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5892857909202576, 'reward_std': 0.31886574625968933, 'kl': 0.02081298828125, 'epoch': 2.96} 59%|█████▉ | 954/1610 [4:06:10<2:46:56, 15.27s/it] 59%|█████▉ | 955/1610 [4:06:24<2:44:02, 15.03s/it] {'loss': 0.0007, 'grad_norm': 1.4814231418017014, 'learning_rate': 4.068322981366459e-07, 'completion_length': 155.37500762939453, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857313156128, 'reward_std': 0.2882790118455887, 'kl': 0.017333984375, 'epoch': 2.97} 59%|█████▉ | 955/1610 [4:06:24<2:44:02, 15.03s/it] 59%|█████▉ | 956/1610 [4:06:40<2:45:29, 15.18s/it] {'loss': 0.0006, 'grad_norm': 1.2039342439103484, 'learning_rate': 4.0621118012422355e-07, 'completion_length': 160.96429443359375, 'rewards/accuracy_reward': 0.5178571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2765706181526184, 'kl': 0.014495849609375, 'epoch': 2.97} 59%|█████▉ | 956/1610 [4:06:40<2:45:29, 15.18s/it] 59%|█████▉ | 957/1610 [4:06:55<2:43:30, 15.02s/it] {'loss': 0.0007, 'grad_norm': 3.256225799757635, 'learning_rate': 4.0559006211180124e-07, 'completion_length': 164.71429443359375, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.2663346976041794, 'kl': 0.0162353515625, 'epoch': 2.97} 59%|█████▉ | 957/1610 [4:06:55<2:43:30, 15.02s/it] 60%|█████▉ | 958/1610 [4:07:09<2:40:52, 14.80s/it] {'loss': 0.0006, 'grad_norm': 2.194951405641694, 'learning_rate': 4.0496894409937887e-07, 'completion_length': 148.68750762939453, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2735324203968048, 'kl': 0.015472412109375, 'epoch': 2.98} 60%|█████▉ | 958/1610 [4:07:09<2:40:52, 14.80s/it] 60%|█████▉ | 959/1610 [4:07:24<2:41:06, 14.85s/it] {'loss': 0.0006, 'grad_norm': 2.2513687330518386, 'learning_rate': 4.043478260869565e-07, 'completion_length': 145.5357208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2630307972431183, 'kl': 0.015838623046875, 'epoch': 2.98} 60%|█████▉ | 959/1610 [4:07:24<2:41:06, 14.85s/it] 60%|█████▉ | 960/1610 [4:07:37<2:35:11, 14.33s/it] {'loss': 0.0005, 'grad_norm': 1.2641349436781715, 'learning_rate': 4.0372670807453413e-07, 'completion_length': 116.28572082519531, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.660714328289032, 'reward_std': 0.25670325756073, 'kl': 0.01190185546875, 'epoch': 2.98} 60%|█████▉ | 960/1610 [4:07:37<2:35:11, 14.33s/it] 60%|█████▉ | 961/1610 [4:07:53<2:38:47, 14.68s/it] {'loss': 0.0007, 'grad_norm': 3.3626050254594078, 'learning_rate': 4.0310559006211177e-07, 'completion_length': 153.62500762939453, 'rewards/accuracy_reward': 0.330357164144516, 'rewards/format_reward': 1.0, 'reward': 1.3303571939468384, 'reward_std': 0.20411308109760284, 'kl': 0.016357421875, 'epoch': 2.98} 60%|█████▉ | 961/1610 [4:07:53<2:38:47, 14.68s/it] 60%|█████▉ | 962/1610 [4:08:09<2:43:26, 15.13s/it] {'loss': 0.0007, 'grad_norm': 0.6778268174023802, 'learning_rate': 4.0248447204968945e-07, 'completion_length': 180.9732208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.17433738708496094, 'kl': 0.0179443359375, 'epoch': 2.99} 60%|█████▉ | 962/1610 [4:08:09<2:43:26, 15.13s/it] 60%|█████▉ | 963/1610 [4:08:24<2:44:24, 15.25s/it] {'loss': 0.0007, 'grad_norm': 0.8702929660772242, 'learning_rate': 4.018633540372671e-07, 'completion_length': 159.60714721679688, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.2636386603116989, 'kl': 0.016876220703125, 'epoch': 2.99} 60%|█████▉ | 963/1610 [4:08:24<2:44:24, 15.25s/it] 60%|█████▉ | 964/1610 [4:08:37<2:37:34, 14.64s/it] {'loss': 0.0006, 'grad_norm': 0.9168905466547426, 'learning_rate': 4.012422360248447e-07, 'completion_length': 136.56250762939453, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875001192092896, 'reward_std': 0.1866704523563385, 'kl': 0.013885498046875, 'epoch': 2.99} 60%|█████▉ | 964/1610 [4:08:37<2:37:34, 14.64s/it] 60%|█████▉ | 965/1610 [4:08:51<2:35:03, 14.42s/it] {'loss': 0.0008, 'grad_norm': 1.621164960914565, 'learning_rate': 4.006211180124223e-07, 'completion_length': 157.8928680419922, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.3414882868528366, 'kl': 0.01898193359375, 'epoch': 3.0} 60%|█████▉ | 965/1610 [4:08:51<2:35:03, 14.42s/it] 60%|██████ | 966/1610 [4:09:08<2:40:48, 14.98s/it] {'loss': 0.0009, 'grad_norm': 1.5467238795431215, 'learning_rate': 4e-07, 'completion_length': 191.65179443359375, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.33245909214019775, 'kl': 0.0218505859375, 'epoch': 3.0} 60%|██████ | 966/1610 [4:09:08<2:40:48, 14.98s/it] 60%|██████ | 967/1610 [4:09:22<2:37:29, 14.70s/it] {'loss': 0.0006, 'grad_norm': 1.354090691122617, 'learning_rate': 3.993788819875776e-07, 'completion_length': 164.56250762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.3014766424894333, 'kl': 0.015838623046875, 'epoch': 3.0} 60%|██████ | 967/1610 [4:09:22<2:37:29, 14.70s/it] 60%|██████ | 968/1610 [4:09:34<2:30:51, 14.10s/it] {'loss': 0.0005, 'grad_norm': 1.7162114013341458, 'learning_rate': 3.9875776397515525e-07, 'completion_length': 114.4464340209961, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.24229323863983154, 'kl': 0.012481689453125, 'epoch': 3.01} 60%|██████ | 968/1610 [4:09:34<2:30:51, 14.10s/it] 60%|██████ | 969/1610 [4:09:50<2:35:25, 14.55s/it] {'loss': 0.0008, 'grad_norm': 1.1993072352407832, 'learning_rate': 3.981366459627329e-07, 'completion_length': 173.69644165039062, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.27474479377269745, 'kl': 0.01947021484375, 'epoch': 3.01} 60%|██████ | 969/1610 [4:09:50<2:35:25, 14.55s/it] 60%|██████ | 970/1610 [4:10:06<2:38:26, 14.85s/it] {'loss': 0.0006, 'grad_norm': 1.2930705046955684, 'learning_rate': 3.975155279503105e-07, 'completion_length': 147.33929443359375, 'rewards/accuracy_reward': 0.4107143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.20471197366714478, 'kl': 0.014404296875, 'epoch': 3.01} 60%|██████ | 970/1610 [4:10:06<2:38:26, 14.85s/it] 60%|██████ | 971/1610 [4:10:21<2:40:43, 15.09s/it] {'loss': 0.0007, 'grad_norm': 0.9482609012445224, 'learning_rate': 3.968944099378882e-07, 'completion_length': 166.30357360839844, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.23535223305225372, 'kl': 0.0169677734375, 'epoch': 3.02} 60%|██████ | 971/1610 [4:10:21<2:40:43, 15.09s/it] 60%|██████ | 972/1610 [4:10:37<2:41:45, 15.21s/it] {'loss': 0.0007, 'grad_norm': 1.1192614043745586, 'learning_rate': 3.9627329192546583e-07, 'completion_length': 165.59822845458984, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.3381787836551666, 'kl': 0.0167236328125, 'epoch': 3.02} 60%|██████ | 972/1610 [4:10:37<2:41:45, 15.21s/it] 60%|██████ | 973/1610 [4:10:52<2:40:50, 15.15s/it] {'loss': 0.0006, 'grad_norm': 1.5437309690535177, 'learning_rate': 3.9565217391304346e-07, 'completion_length': 160.04464721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2377769947052002, 'kl': 0.01531982421875, 'epoch': 3.02} 60%|██████ | 973/1610 [4:10:52<2:40:50, 15.15s/it] 60%|██████ | 974/1610 [4:11:07<2:40:24, 15.13s/it] {'loss': 0.0006, 'grad_norm': 0.9622325415030808, 'learning_rate': 3.950310559006211e-07, 'completion_length': 149.0714340209961, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.21972985565662384, 'kl': 0.014617919921875, 'epoch': 3.02} 60%|██████ | 974/1610 [4:11:07<2:40:24, 15.13s/it] 61%|██████ | 975/1610 [4:11:23<2:44:18, 15.53s/it] {'loss': 0.0006, 'grad_norm': 1.2337320652600785, 'learning_rate': 3.944099378881988e-07, 'completion_length': 154.3839340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.3051137775182724, 'kl': 0.013824462890625, 'epoch': 3.03} 61%|██████ | 975/1610 [4:11:23<2:44:18, 15.53s/it] 61%|██████ | 976/1610 [4:11:38<2:43:16, 15.45s/it] {'loss': 0.0007, 'grad_norm': 2.1249137530005626, 'learning_rate': 3.937888198757764e-07, 'completion_length': 157.3839340209961, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.2825503796339035, 'kl': 0.0164794921875, 'epoch': 3.03} 61%|██████ | 976/1610 [4:11:38<2:43:16, 15.45s/it] 61%|██████ | 977/1610 [4:11:55<2:46:50, 15.81s/it] {'loss': 0.0008, 'grad_norm': 1.0084274837687806, 'learning_rate': 3.93167701863354e-07, 'completion_length': 176.49108123779297, 'rewards/accuracy_reward': 0.5714286118745804, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625001192092896, 'reward_std': 0.2921874225139618, 'kl': 0.020751953125, 'epoch': 3.03} 61%|██████ | 977/1610 [4:11:55<2:46:50, 15.81s/it] 61%|██████ | 978/1610 [4:12:10<2:43:59, 15.57s/it] {'loss': 0.0006, 'grad_norm': 2.614785395702118, 'learning_rate': 3.925465838509316e-07, 'completion_length': 153.00000762939453, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3051137626171112, 'kl': 0.016143798828125, 'epoch': 3.04} 61%|██████ | 978/1610 [4:12:10<2:43:59, 15.57s/it] 61%|██████ | 979/1610 [4:12:25<2:42:29, 15.45s/it] {'loss': 0.0006, 'grad_norm': 1.920503538100698, 'learning_rate': 3.9192546583850926e-07, 'completion_length': 155.9107208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.3571050316095352, 'kl': 0.014129638671875, 'epoch': 3.04} 61%|██████ | 979/1610 [4:12:25<2:42:29, 15.45s/it] 61%|██████ | 980/1610 [4:12:39<2:35:29, 14.81s/it] {'loss': 0.0005, 'grad_norm': 1.7967558557251706, 'learning_rate': 3.9130434782608694e-07, 'completion_length': 127.85715103149414, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.32645919919013977, 'kl': 0.012115478515625, 'epoch': 3.04} 61%|██████ | 980/1610 [4:12:39<2:35:29, 14.81s/it] 61%|██████ | 981/1610 [4:12:54<2:38:31, 15.12s/it] {'loss': 0.0008, 'grad_norm': 1.4047534341626737, 'learning_rate': 3.906832298136646e-07, 'completion_length': 151.37500762939453, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.598214328289032, 'reward_std': 0.3252524062991142, 'kl': 0.019073486328125, 'epoch': 3.05} 61%|██████ | 981/1610 [4:12:54<2:38:31, 15.12s/it] 61%|██████ | 982/1610 [4:13:10<2:38:29, 15.14s/it] {'loss': 0.0005, 'grad_norm': 1.0650872518460566, 'learning_rate': 3.900621118012422e-07, 'completion_length': 164.17858123779297, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.23265621066093445, 'kl': 0.01361083984375, 'epoch': 3.05} 61%|██████ | 982/1610 [4:13:10<2:38:29, 15.14s/it] 61%|██████ | 983/1610 [4:13:25<2:39:53, 15.30s/it] {'loss': 0.0008, 'grad_norm': 2.278013137739439, 'learning_rate': 3.8944099378881984e-07, 'completion_length': 190.56250762939453, 'rewards/accuracy_reward': 0.2857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.285714328289032, 'reward_std': 0.23266181349754333, 'kl': 0.021240234375, 'epoch': 3.05} 61%|██████ | 983/1610 [4:13:25<2:39:53, 15.30s/it] 61%|██████ | 984/1610 [4:13:41<2:41:37, 15.49s/it] {'loss': 0.0007, 'grad_norm': 3.3411924300475304, 'learning_rate': 3.8881987577639753e-07, 'completion_length': 169.90179443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.3571050316095352, 'kl': 0.01727294921875, 'epoch': 3.06} 61%|██████ | 984/1610 [4:13:41<2:41:37, 15.49s/it] 61%|██████ | 985/1610 [4:13:58<2:45:09, 15.85s/it] {'loss': 0.0009, 'grad_norm': 1.3934299342158996, 'learning_rate': 3.8819875776397516e-07, 'completion_length': 180.74108123779297, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2741458863019943, 'kl': 0.023681640625, 'epoch': 3.06} 61%|██████ | 985/1610 [4:13:58<2:45:09, 15.85s/it] 61%|██████ | 986/1610 [4:14:14<2:44:17, 15.80s/it] {'loss': 0.0006, 'grad_norm': 1.2430114411144497, 'learning_rate': 3.875776397515528e-07, 'completion_length': 150.50000762939453, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.2847601920366287, 'kl': 0.0150146484375, 'epoch': 3.06} 61%|██████ | 986/1610 [4:14:14<2:44:17, 15.80s/it] 61%|██████▏ | 987/1610 [4:14:27<2:37:55, 15.21s/it] {'loss': 0.0008, 'grad_norm': 1.6917193367942398, 'learning_rate': 3.869565217391304e-07, 'completion_length': 137.06250762939453, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.35137639939785004, 'kl': 0.01910400390625, 'epoch': 3.07} 61%|██████▏ | 987/1610 [4:14:27<2:37:55, 15.21s/it] 61%|██████▏ | 988/1610 [4:14:43<2:38:03, 15.25s/it] {'loss': 0.0008, 'grad_norm': 0.9832135463175543, 'learning_rate': 3.8633540372670806e-07, 'completion_length': 181.5803680419922, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.22875341773033142, 'kl': 0.01959228515625, 'epoch': 3.07} 61%|██████▏ | 988/1610 [4:14:43<2:38:03, 15.25s/it] 61%|██████▏ | 989/1610 [4:14:56<2:30:17, 14.52s/it] {'loss': 0.0005, 'grad_norm': 1.5878104265237765, 'learning_rate': 3.857142857142857e-07, 'completion_length': 122.44643020629883, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.30721087753772736, 'kl': 0.01318359375, 'epoch': 3.07} 61%|██████▏ | 989/1610 [4:14:56<2:30:17, 14.52s/it] 61%|██████▏ | 990/1610 [4:15:11<2:33:04, 14.81s/it] {'loss': 0.0008, 'grad_norm': 1.5726995344866617, 'learning_rate': 3.850931677018633e-07, 'completion_length': 176.3303680419922, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.3186535984277725, 'kl': 0.020751953125, 'epoch': 3.07} 61%|██████▏ | 990/1610 [4:15:11<2:33:04, 14.81s/it] 62%|██████▏ | 991/1610 [4:15:26<2:33:32, 14.88s/it] {'loss': 0.0007, 'grad_norm': 1.4774637555139531, 'learning_rate': 3.8447204968944095e-07, 'completion_length': 165.77679443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1995968073606491, 'kl': 0.016693115234375, 'epoch': 3.08} 62%|██████▏ | 991/1610 [4:15:26<2:33:32, 14.88s/it] 62%|██████▏ | 992/1610 [4:15:41<2:31:57, 14.75s/it] {'loss': 0.0006, 'grad_norm': 1.9033859952941183, 'learning_rate': 3.838509316770186e-07, 'completion_length': 158.9107208251953, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.244989275932312, 'kl': 0.0150146484375, 'epoch': 3.08} 62%|██████▏ | 992/1610 [4:15:41<2:31:57, 14.75s/it] 62%|██████▏ | 993/1610 [4:15:56<2:32:47, 14.86s/it] {'loss': 0.0007, 'grad_norm': 1.5449652858889522, 'learning_rate': 3.8322981366459627e-07, 'completion_length': 153.30357360839844, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2831694409251213, 'kl': 0.01678466796875, 'epoch': 3.08} 62%|██████▏ | 993/1610 [4:15:56<2:32:47, 14.86s/it] 62%|██████▏ | 994/1610 [4:16:13<2:38:51, 15.47s/it] {'loss': 0.0007, 'grad_norm': 1.7659670652508237, 'learning_rate': 3.826086956521739e-07, 'completion_length': 167.67857360839844, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.3570459634065628, 'kl': 0.01751708984375, 'epoch': 3.09} 62%|██████▏ | 994/1610 [4:16:13<2:38:51, 15.47s/it] 62%|██████▏ | 995/1610 [4:16:29<2:41:46, 15.78s/it] {'loss': 0.0007, 'grad_norm': 1.077849968998467, 'learning_rate': 3.8198757763975154e-07, 'completion_length': 160.5714340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160714626312256, 'reward_std': 0.2467949539422989, 'kl': 0.0164794921875, 'epoch': 3.09} 62%|██████▏ | 995/1610 [4:16:29<2:41:46, 15.78s/it] 62%|██████▏ | 996/1610 [4:16:43<2:34:28, 15.10s/it] {'loss': 0.0005, 'grad_norm': 0.933193941172214, 'learning_rate': 3.8136645962732917e-07, 'completion_length': 126.20536041259766, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.15360543131828308, 'kl': 0.0113525390625, 'epoch': 3.09} 62%|██████▏ | 996/1610 [4:16:43<2:34:28, 15.10s/it] 62%|██████▏ | 997/1610 [4:16:57<2:31:35, 14.84s/it] {'loss': 0.0007, 'grad_norm': 1.1671241781704886, 'learning_rate': 3.807453416149068e-07, 'completion_length': 156.89286041259766, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.25280046463012695, 'kl': 0.01812744140625, 'epoch': 3.1} 62%|██████▏ | 997/1610 [4:16:57<2:31:35, 14.84s/it] 62%|██████▏ | 998/1610 [4:17:12<2:32:34, 14.96s/it] {'loss': 0.0007, 'grad_norm': 3.638197645220517, 'learning_rate': 3.801242236024845e-07, 'completion_length': 155.9732208251953, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.29339979588985443, 'kl': 0.016357421875, 'epoch': 3.1} 62%|██████▏ | 998/1610 [4:17:12<2:32:34, 14.96s/it] 62%|██████▏ | 999/1610 [4:17:28<2:33:57, 15.12s/it] {'loss': 0.0007, 'grad_norm': 1.8756973278683213, 'learning_rate': 3.795031055900621e-07, 'completion_length': 148.3214340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.25521960854530334, 'kl': 0.01678466796875, 'epoch': 3.1} 62%|██████▏ | 999/1610 [4:17:28<2:33:57, 15.12s/it] 62%|██████▏ | 1000/1610 [4:17:43<2:35:04, 15.25s/it] {'loss': 0.0008, 'grad_norm': 0.9484716726508298, 'learning_rate': 3.7888198757763975e-07, 'completion_length': 162.73214721679688, 'rewards/accuracy_reward': 0.6160714477300644, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.1866704449057579, 'kl': 0.02056884765625, 'epoch': 3.11} 62%|██████▏ | 1000/1610 [4:17:43<2:35:04, 15.25s/it] 62%|██████▏ | 1001/1610 [4:18:47<5:02:31, 29.81s/it] {'loss': 0.0007, 'grad_norm': 1.084037118241323, 'learning_rate': 3.7826086956521733e-07, 'completion_length': 174.23214721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.30061207711696625, 'kl': 0.01690673828125, 'epoch': 3.11} 62%|██████▏ | 1001/1610 [4:18:47<5:02:31, 29.81s/it] 62%|██████▏ | 1002/1610 [4:19:00<4:10:08, 24.68s/it] {'loss': 0.0006, 'grad_norm': 1.3125439590819863, 'learning_rate': 3.77639751552795e-07, 'completion_length': 153.1964340209961, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.2344820648431778, 'kl': 0.015655517578125, 'epoch': 3.11} 62%|██████▏ | 1002/1610 [4:19:00<4:10:08, 24.68s/it] 62%|██████▏ | 1003/1610 [4:19:12<3:31:59, 20.96s/it] {'loss': 0.0007, 'grad_norm': 1.0086507769679653, 'learning_rate': 3.7701863354037265e-07, 'completion_length': 159.9732208251953, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.32221420109272003, 'kl': 0.016876220703125, 'epoch': 3.11} 62%|██████▏ | 1003/1610 [4:19:12<3:31:59, 20.96s/it] 62%|██████▏ | 1004/1610 [4:19:24<3:04:31, 18.27s/it] {'loss': 0.0006, 'grad_norm': 1.1786699964270615, 'learning_rate': 3.763975155279503e-07, 'completion_length': 161.4107208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2500988245010376, 'kl': 0.01593017578125, 'epoch': 3.12} 62%|██████▏ | 1004/1610 [4:19:24<3:04:31, 18.27s/it] 62%|██████▏ | 1005/1610 [4:19:36<2:46:38, 16.53s/it] {'loss': 0.0007, 'grad_norm': 1.962117613194762, 'learning_rate': 3.757763975155279e-07, 'completion_length': 139.83036041259766, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.38747963309288025, 'kl': 0.018310546875, 'epoch': 3.12} 62%|██████▏ | 1005/1610 [4:19:36<2:46:38, 16.53s/it] 62%|██████▏ | 1006/1610 [4:19:48<2:33:02, 15.20s/it] {'loss': 0.0007, 'grad_norm': 1.2562877931029321, 'learning_rate': 3.7515527950310555e-07, 'completion_length': 160.6339340209961, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.2630251944065094, 'kl': 0.0185546875, 'epoch': 3.12} 62%|██████▏ | 1006/1610 [4:19:49<2:33:02, 15.20s/it] 63%|██████▎ | 1007/1610 [4:20:00<2:21:39, 14.09s/it] {'loss': 0.0006, 'grad_norm': 2.334317776561397, 'learning_rate': 3.7453416149068323e-07, 'completion_length': 127.57143783569336, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.633928656578064, 'reward_std': 0.22094222903251648, 'kl': 0.014129638671875, 'epoch': 3.13} 63%|██████▎ | 1007/1610 [4:20:00<2:21:39, 14.09s/it] 63%|██████▎ | 1008/1610 [4:20:12<2:15:02, 13.46s/it] {'loss': 0.0007, 'grad_norm': 1.1692123450592664, 'learning_rate': 3.7391304347826087e-07, 'completion_length': 162.7232208251953, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.34807586669921875, 'kl': 0.018341064453125, 'epoch': 3.13} 63%|██████▎ | 1008/1610 [4:20:12<2:15:02, 13.46s/it] 63%|██████▎ | 1009/1610 [4:20:22<2:04:21, 12.42s/it] {'loss': 0.0005, 'grad_norm': 1.3790620088463168, 'learning_rate': 3.732919254658385e-07, 'completion_length': 113.65179061889648, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410714626312256, 'reward_std': 0.2948778122663498, 'kl': 0.0128173828125, 'epoch': 3.13} 63%|██████▎ | 1009/1610 [4:20:22<2:04:21, 12.42s/it] 63%|██████▎ | 1010/1610 [4:20:34<2:02:02, 12.20s/it] {'loss': 0.0006, 'grad_norm': 1.1469836312626867, 'learning_rate': 3.7267080745341613e-07, 'completion_length': 145.88393783569336, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2443757951259613, 'kl': 0.01531982421875, 'epoch': 3.14} 63%|██████▎ | 1010/1610 [4:20:34<2:02:02, 12.20s/it] 63%|██████▎ | 1011/1610 [4:20:47<2:04:31, 12.47s/it] {'loss': 0.0006, 'grad_norm': 1.0153257968970197, 'learning_rate': 3.720496894409938e-07, 'completion_length': 146.76786041259766, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.22363264858722687, 'kl': 0.016204833984375, 'epoch': 3.14} 63%|██████▎ | 1011/1610 [4:20:47<2:04:31, 12.47s/it] 63%|██████▎ | 1012/1610 [4:21:02<2:12:32, 13.30s/it] {'loss': 0.0007, 'grad_norm': 1.338768032520896, 'learning_rate': 3.7142857142857145e-07, 'completion_length': 177.9464340209961, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.23265621066093445, 'kl': 0.01715087890625, 'epoch': 3.14} 63%|██████▎ | 1012/1610 [4:21:02<2:12:32, 13.30s/it] 63%|██████▎ | 1013/1610 [4:21:17<2:16:15, 13.69s/it] {'loss': 0.0007, 'grad_norm': 1.374109915170944, 'learning_rate': 3.7080745341614903e-07, 'completion_length': 161.5178680419922, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.20982715487480164, 'kl': 0.01641845703125, 'epoch': 3.15} 63%|██████▎ | 1013/1610 [4:21:17<2:16:15, 13.69s/it] 63%|██████▎ | 1014/1610 [4:21:30<2:16:00, 13.69s/it] {'loss': 0.0007, 'grad_norm': 1.6420236904767396, 'learning_rate': 3.7018633540372666e-07, 'completion_length': 134.3839340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3150164783000946, 'kl': 0.016510009765625, 'epoch': 3.15} 63%|██████▎ | 1014/1610 [4:21:30<2:16:00, 13.69s/it] 63%|██████▎ | 1015/1610 [4:21:47<2:25:47, 14.70s/it] {'loss': 0.0008, 'grad_norm': 1.4643665058206243, 'learning_rate': 3.695652173913043e-07, 'completion_length': 179.86607360839844, 'rewards/accuracy_reward': 0.571428582072258, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625001192092896, 'reward_std': 0.3441195785999298, 'kl': 0.018829345703125, 'epoch': 3.15} 63%|██████▎ | 1015/1610 [4:21:47<2:25:47, 14.70s/it] 63%|██████▎ | 1016/1610 [4:22:02<2:26:11, 14.77s/it] {'loss': 0.0008, 'grad_norm': 1.258935277300142, 'learning_rate': 3.68944099378882e-07, 'completion_length': 154.50894165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3279484361410141, 'kl': 0.02069091796875, 'epoch': 3.16} 63%|██████▎ | 1016/1610 [4:22:02<2:26:11, 14.77s/it] 63%|██████▎ | 1017/1610 [4:22:18<2:29:11, 15.10s/it] {'loss': 0.0006, 'grad_norm': 1.86109820010489, 'learning_rate': 3.683229813664596e-07, 'completion_length': 149.6875, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.2897091656923294, 'kl': 0.015045166015625, 'epoch': 3.16} 63%|██████▎ | 1017/1610 [4:22:18<2:29:11, 15.10s/it] 63%|██████▎ | 1018/1610 [4:22:34<2:30:55, 15.30s/it] {'loss': 0.0006, 'grad_norm': 1.0675264479292754, 'learning_rate': 3.6770186335403724e-07, 'completion_length': 159.7232208251953, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.2193143516778946, 'kl': 0.015625, 'epoch': 3.16} 63%|██████▎ | 1018/1610 [4:22:34<2:30:55, 15.30s/it] 63%|██████▎ | 1019/1610 [4:22:48<2:28:22, 15.06s/it] {'loss': 0.0008, 'grad_norm': 1.4074610066145445, 'learning_rate': 3.670807453416149e-07, 'completion_length': 155.75000762939453, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3402702808380127, 'kl': 0.01953125, 'epoch': 3.16} 63%|██████▎ | 1019/1610 [4:22:48<2:28:22, 15.06s/it] 63%|██████▎ | 1020/1610 [4:23:05<2:31:58, 15.46s/it] {'loss': 0.0007, 'grad_norm': 1.5062717809589472, 'learning_rate': 3.6645962732919256e-07, 'completion_length': 181.05358123779297, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.30060645937919617, 'kl': 0.01629638671875, 'epoch': 3.17} 63%|██████▎ | 1020/1610 [4:23:05<2:31:58, 15.46s/it] 63%|██████▎ | 1021/1610 [4:23:21<2:32:33, 15.54s/it] {'loss': 0.0007, 'grad_norm': 1.7224794882441639, 'learning_rate': 3.658385093167702e-07, 'completion_length': 158.91964721679688, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.2540072351694107, 'kl': 0.017791748046875, 'epoch': 3.17} 63%|██████▎ | 1021/1610 [4:23:21<2:32:33, 15.54s/it] 63%|██████▎ | 1022/1610 [4:23:36<2:31:05, 15.42s/it] {'loss': 0.0007, 'grad_norm': 1.5204983504774128, 'learning_rate': 3.6521739130434783e-07, 'completion_length': 146.3214340209961, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.33184562623500824, 'kl': 0.0164794921875, 'epoch': 3.17} 63%|██████▎ | 1022/1610 [4:23:36<2:31:05, 15.42s/it] 64%|██████▎ | 1023/1610 [4:23:52<2:32:25, 15.58s/it] {'loss': 0.0007, 'grad_norm': 1.6794075117197698, 'learning_rate': 3.6459627329192546e-07, 'completion_length': 165.77679443359375, 'rewards/accuracy_reward': 0.5803571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2825503647327423, 'kl': 0.016387939453125, 'epoch': 3.18} 64%|██████▎ | 1023/1610 [4:23:52<2:32:25, 15.58s/it] 64%|██████▎ | 1024/1610 [4:24:06<2:28:48, 15.24s/it] {'loss': 0.0006, 'grad_norm': 1.5384317459667642, 'learning_rate': 3.6397515527950304e-07, 'completion_length': 143.73214721679688, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.30061206221580505, 'kl': 0.014678955078125, 'epoch': 3.18} 64%|██████▎ | 1024/1610 [4:24:06<2:28:48, 15.24s/it] 64%|██████▎ | 1025/1610 [4:24:21<2:27:58, 15.18s/it] {'loss': 0.0008, 'grad_norm': 1.2506509719067191, 'learning_rate': 3.633540372670807e-07, 'completion_length': 166.7589340209961, 'rewards/accuracy_reward': 0.5178571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2816857844591141, 'kl': 0.019775390625, 'epoch': 3.18} 64%|██████▎ | 1025/1610 [4:24:21<2:27:58, 15.18s/it] 64%|██████▎ | 1026/1610 [4:24:36<2:25:56, 14.99s/it] {'loss': 0.0006, 'grad_norm': 2.379177898589147, 'learning_rate': 3.6273291925465836e-07, 'completion_length': 141.33929061889648, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.18397442996501923, 'kl': 0.014892578125, 'epoch': 3.19} 64%|██████▎ | 1026/1610 [4:24:36<2:25:56, 14.99s/it] 64%|██████▍ | 1027/1610 [4:24:49<2:21:50, 14.60s/it] {'loss': 0.0006, 'grad_norm': 1.8088681175473198, 'learning_rate': 3.62111801242236e-07, 'completion_length': 133.73214721679688, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.2897626608610153, 'kl': 0.014312744140625, 'epoch': 3.19} 64%|██████▍ | 1027/1610 [4:24:49<2:21:50, 14.60s/it] 64%|██████▍ | 1028/1610 [4:25:03<2:19:36, 14.39s/it] {'loss': 0.0006, 'grad_norm': 1.049640028843096, 'learning_rate': 3.614906832298136e-07, 'completion_length': 139.74107360839844, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.2696240097284317, 'kl': 0.014190673828125, 'epoch': 3.19} 64%|██████▍ | 1028/1610 [4:25:03<2:19:36, 14.39s/it] 64%|██████▍ | 1029/1610 [4:25:18<2:19:47, 14.44s/it] {'loss': 0.0005, 'grad_norm': 1.0156992329810286, 'learning_rate': 3.608695652173913e-07, 'completion_length': 131.1964340209961, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.20619001984596252, 'kl': 0.013214111328125, 'epoch': 3.2} 64%|██████▍ | 1029/1610 [4:25:18<2:19:47, 14.44s/it] 64%|██████▍ | 1030/1610 [4:25:31<2:15:12, 13.99s/it] {'loss': 0.0005, 'grad_norm': 2.879251628801568, 'learning_rate': 3.6024844720496894e-07, 'completion_length': 129.93750762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.42834463715553284, 'kl': 0.013458251953125, 'epoch': 3.2} 64%|██████▍ | 1030/1610 [4:25:31<2:15:12, 13.99s/it] 64%|██████▍ | 1031/1610 [4:25:46<2:17:47, 14.28s/it] {'loss': 0.0006, 'grad_norm': 0.7793353464200924, 'learning_rate': 3.596273291925466e-07, 'completion_length': 157.50000762939453, 'rewards/accuracy_reward': 0.6250000447034836, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.17495086044073105, 'kl': 0.0150146484375, 'epoch': 3.2} 64%|██████▍ | 1031/1610 [4:25:46<2:17:47, 14.28s/it] 64%|██████▍ | 1032/1610 [4:26:00<2:18:57, 14.43s/it] {'loss': 0.0008, 'grad_norm': 5.686612049210786, 'learning_rate': 3.590062111801242e-07, 'completion_length': 168.3303680419922, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2882845997810364, 'kl': 0.01885986328125, 'epoch': 3.2} 64%|██████▍ | 1032/1610 [4:26:00<2:18:57, 14.43s/it] 64%|██████▍ | 1033/1610 [4:26:16<2:20:30, 14.61s/it] {'loss': 0.0007, 'grad_norm': 0.9763090526377441, 'learning_rate': 3.5838509316770184e-07, 'completion_length': 149.30357360839844, 'rewards/accuracy_reward': 0.2946428805589676, 'rewards/format_reward': 1.0, 'reward': 1.2946429252624512, 'reward_std': 0.17554975673556328, 'kl': 0.01715087890625, 'epoch': 3.21} 64%|██████▍ | 1033/1610 [4:26:16<2:20:30, 14.61s/it] 64%|██████▍ | 1034/1610 [4:26:30<2:20:00, 14.58s/it] {'loss': 0.0006, 'grad_norm': 2.104900813108286, 'learning_rate': 3.577639751552795e-07, 'completion_length': 146.4553680419922, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.2540072202682495, 'kl': 0.0146484375, 'epoch': 3.21} 64%|██████▍ | 1034/1610 [4:26:30<2:20:00, 14.58s/it] 64%|██████▍ | 1035/1610 [4:26:45<2:21:50, 14.80s/it] {'loss': 0.0008, 'grad_norm': 2.3196601452294376, 'learning_rate': 3.5714285714285716e-07, 'completion_length': 161.00000762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.24498926103115082, 'kl': 0.02105712890625, 'epoch': 3.21} 64%|██████▍ | 1035/1610 [4:26:45<2:21:50, 14.80s/it] 64%|██████▍ | 1036/1610 [4:27:00<2:21:28, 14.79s/it] {'loss': 0.0008, 'grad_norm': 1.7648417884331131, 'learning_rate': 3.5652173913043474e-07, 'completion_length': 152.49108123779297, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.39318810403347015, 'kl': 0.0203857421875, 'epoch': 3.22} 64%|██████▍ | 1036/1610 [4:27:00<2:21:28, 14.79s/it] 64%|██████▍ | 1037/1610 [4:27:15<2:20:51, 14.75s/it] {'loss': 0.0004, 'grad_norm': 1.9663579927692727, 'learning_rate': 3.5590062111801237e-07, 'completion_length': 134.06250762939453, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.27804866433143616, 'kl': 0.010345458984375, 'epoch': 3.22} 64%|██████▍ | 1037/1610 [4:27:15<2:20:51, 14.75s/it] 64%|██████▍ | 1038/1610 [4:27:28<2:16:38, 14.33s/it] {'loss': 0.0006, 'grad_norm': 1.1676469021288072, 'learning_rate': 3.5527950310559005e-07, 'completion_length': 135.06250762939453, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.23925502598285675, 'kl': 0.015777587890625, 'epoch': 3.22} 64%|██████▍ | 1038/1610 [4:27:28<2:16:38, 14.33s/it] 65%|██████▍ | 1039/1610 [4:27:42<2:15:10, 14.20s/it] {'loss': 0.0006, 'grad_norm': 1.5467267374407194, 'learning_rate': 3.546583850931677e-07, 'completion_length': 142.5803680419922, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.23326967656612396, 'kl': 0.015106201171875, 'epoch': 3.23} 65%|██████▍ | 1039/1610 [4:27:42<2:15:10, 14.20s/it] 65%|██████▍ | 1040/1610 [4:27:57<2:16:40, 14.39s/it] {'loss': 0.0006, 'grad_norm': 0.9964266345626783, 'learning_rate': 3.540372670807453e-07, 'completion_length': 157.26786041259766, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2404673993587494, 'kl': 0.014923095703125, 'epoch': 3.23} 65%|██████▍ | 1040/1610 [4:27:57<2:16:40, 14.39s/it] 65%|██████▍ | 1041/1610 [4:28:11<2:16:58, 14.44s/it] {'loss': 0.0007, 'grad_norm': 1.9981908586493844, 'learning_rate': 3.5341614906832295e-07, 'completion_length': 153.20536041259766, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.35527920722961426, 'kl': 0.017486572265625, 'epoch': 3.23} 65%|██████▍ | 1041/1610 [4:28:11<2:16:58, 14.44s/it] 65%|██████▍ | 1042/1610 [4:28:28<2:22:31, 15.05s/it] {'loss': 0.0007, 'grad_norm': 0.8235374641801024, 'learning_rate': 3.527950310559006e-07, 'completion_length': 174.25000762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.26175932586193085, 'kl': 0.017333984375, 'epoch': 3.24} 65%|██████▍ | 1042/1610 [4:28:28<2:22:31, 15.05s/it] 65%|██████▍ | 1043/1610 [4:28:44<2:26:04, 15.46s/it] {'loss': 0.0011, 'grad_norm': 1.498832570055565, 'learning_rate': 3.5217391304347827e-07, 'completion_length': 191.0178680419922, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285715222358704, 'reward_std': 0.3067897707223892, 'kl': 0.02642822265625, 'epoch': 3.24} 65%|██████▍ | 1043/1610 [4:28:44<2:26:04, 15.46s/it] 65%|██████▍ | 1044/1610 [4:28:59<2:22:42, 15.13s/it] {'loss': 0.0007, 'grad_norm': 2.0485946446276246, 'learning_rate': 3.515527950310559e-07, 'completion_length': 155.82143783569336, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.16531942784786224, 'kl': 0.01849365234375, 'epoch': 3.24} 65%|██████▍ | 1044/1610 [4:28:59<2:22:42, 15.13s/it] 65%|██████▍ | 1045/1610 [4:29:14<2:22:42, 15.15s/it] {'loss': 0.0006, 'grad_norm': 2.0245065206165176, 'learning_rate': 3.5093167701863354e-07, 'completion_length': 144.91964721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.2663346976041794, 'kl': 0.014129638671875, 'epoch': 3.25} 65%|██████▍ | 1045/1610 [4:29:14<2:22:42, 15.15s/it] 65%|██████▍ | 1046/1610 [4:29:30<2:24:15, 15.35s/it] {'loss': 0.0009, 'grad_norm': 0.9805558152467561, 'learning_rate': 3.5031055900621117e-07, 'completion_length': 177.08929443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.25252360105514526, 'kl': 0.02252197265625, 'epoch': 3.25} 65%|██████▍ | 1046/1610 [4:29:30<2:24:15, 15.35s/it] 65%|██████▌ | 1047/1610 [4:29:44<2:19:51, 14.90s/it] {'loss': 0.0006, 'grad_norm': 1.1337155375910537, 'learning_rate': 3.4968944099378885e-07, 'completion_length': 128.7857208251953, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.24498926103115082, 'kl': 0.016021728515625, 'epoch': 3.25} 65%|██████▌ | 1047/1610 [4:29:44<2:19:51, 14.90s/it] 65%|██████▌ | 1048/1610 [4:29:59<2:20:14, 14.97s/it] {'loss': 0.0007, 'grad_norm': 1.3984992358112969, 'learning_rate': 3.4906832298136643e-07, 'completion_length': 157.50000762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803572535514832, 'reward_std': 0.36100783944129944, 'kl': 0.018310546875, 'epoch': 3.25} 65%|██████▌ | 1048/1610 [4:29:59<2:20:14, 14.97s/it] 65%|██████▌ | 1049/1610 [4:30:14<2:19:58, 14.97s/it] {'loss': 0.0008, 'grad_norm': 1.5311860907495025, 'learning_rate': 3.4844720496894407e-07, 'completion_length': 177.11608123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.32220859825611115, 'kl': 0.01898193359375, 'epoch': 3.26} 65%|██████▌ | 1049/1610 [4:30:14<2:19:58, 14.97s/it] 65%|██████▌ | 1050/1610 [4:30:28<2:17:40, 14.75s/it] {'loss': 0.0007, 'grad_norm': 1.5288613079716482, 'learning_rate': 3.478260869565217e-07, 'completion_length': 146.40178680419922, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.25670325756073, 'kl': 0.01806640625, 'epoch': 3.26} 65%|██████▌ | 1050/1610 [4:30:28<2:17:40, 14.75s/it] 65%|██████▌ | 1051/1610 [4:30:43<2:18:15, 14.84s/it] {'loss': 0.0007, 'grad_norm': 1.1313859815098473, 'learning_rate': 3.4720496894409933e-07, 'completion_length': 144.5714340209961, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.26181842386722565, 'kl': 0.017669677734375, 'epoch': 3.26} 65%|██████▌ | 1051/1610 [4:30:43<2:18:15, 14.84s/it] 65%|██████▌ | 1052/1610 [4:30:57<2:16:03, 14.63s/it] {'loss': 0.0006, 'grad_norm': 1.570680812069726, 'learning_rate': 3.46583850931677e-07, 'completion_length': 142.36607360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.32854174077510834, 'kl': 0.014068603515625, 'epoch': 3.27} 65%|██████▌ | 1052/1610 [4:30:57<2:16:03, 14.63s/it] 65%|██████▌ | 1053/1610 [4:31:11<2:13:46, 14.41s/it] {'loss': 0.0006, 'grad_norm': 1.245547286524566, 'learning_rate': 3.4596273291925465e-07, 'completion_length': 127.2589340209961, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2657212167978287, 'kl': 0.016204833984375, 'epoch': 3.27} 65%|██████▌ | 1053/1610 [4:31:11<2:13:46, 14.41s/it] 65%|██████▌ | 1054/1610 [4:31:25<2:12:14, 14.27s/it] {'loss': 0.0005, 'grad_norm': 1.313390490014859, 'learning_rate': 3.453416149068323e-07, 'completion_length': 146.55357360839844, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.27083639800548553, 'kl': 0.0123291015625, 'epoch': 3.27} 65%|██████▌ | 1054/1610 [4:31:25<2:12:14, 14.27s/it] 66%|██████▌ | 1055/1610 [4:31:38<2:09:45, 14.03s/it] {'loss': 0.0006, 'grad_norm': 1.077978381425977, 'learning_rate': 3.447204968944099e-07, 'completion_length': 144.50000762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.3369663804769516, 'kl': 0.015472412109375, 'epoch': 3.28} 66%|██████▌ | 1055/1610 [4:31:38<2:09:45, 14.03s/it] 66%|██████▌ | 1056/1610 [4:31:53<2:10:56, 14.18s/it] {'loss': 0.0007, 'grad_norm': 1.5600771729700738, 'learning_rate': 3.440993788819876e-07, 'completion_length': 152.81250762939453, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.2540072426199913, 'kl': 0.01812744140625, 'epoch': 3.28} 66%|██████▌ | 1056/1610 [4:31:53<2:10:56, 14.18s/it] 66%|██████▌ | 1057/1610 [4:32:08<2:12:11, 14.34s/it] {'loss': 0.0008, 'grad_norm': 0.8065506357328069, 'learning_rate': 3.4347826086956523e-07, 'completion_length': 167.9464340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2630251944065094, 'kl': 0.01922607421875, 'epoch': 3.28} 66%|██████▌ | 1057/1610 [4:32:08<2:12:11, 14.34s/it] 66%|██████▌ | 1058/1610 [4:32:23<2:15:17, 14.71s/it] {'loss': 0.0006, 'grad_norm': 1.2747225167718323, 'learning_rate': 3.4285714285714286e-07, 'completion_length': 158.80358123779297, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2531370371580124, 'kl': 0.014495849609375, 'epoch': 3.29} 66%|██████▌ | 1058/1610 [4:32:23<2:15:17, 14.71s/it] 66%|██████▌ | 1059/1610 [4:32:37<2:12:30, 14.43s/it] {'loss': 0.0007, 'grad_norm': 1.6064965633186103, 'learning_rate': 3.422360248447205e-07, 'completion_length': 127.8839340209961, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.2630307972431183, 'kl': 0.017181396484375, 'epoch': 3.29} 66%|██████▌ | 1059/1610 [4:32:37<2:12:30, 14.43s/it] 66%|██████▌ | 1060/1610 [4:32:52<2:14:32, 14.68s/it] {'loss': 0.0009, 'grad_norm': 1.480420337982088, 'learning_rate': 3.416149068322981e-07, 'completion_length': 158.58928680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.34148266911506653, 'kl': 0.02142333984375, 'epoch': 3.29} 66%|██████▌ | 1060/1610 [4:32:52<2:14:32, 14.68s/it] 66%|██████▌ | 1061/1610 [4:33:07<2:15:07, 14.77s/it] {'loss': 0.0007, 'grad_norm': 1.29962000627163, 'learning_rate': 3.4099378881987576e-07, 'completion_length': 142.73214721679688, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.20080919563770294, 'kl': 0.017059326171875, 'epoch': 3.3} 66%|██████▌ | 1061/1610 [4:33:07<2:15:07, 14.77s/it] 66%|██████▌ | 1062/1610 [4:33:22<2:14:49, 14.76s/it] {'loss': 0.0005, 'grad_norm': 0.718926778146047, 'learning_rate': 3.403726708074534e-07, 'completion_length': 152.05358123779297, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.17285936325788498, 'kl': 0.013458251953125, 'epoch': 3.3} 66%|██████▌ | 1062/1610 [4:33:22<2:14:49, 14.76s/it] 66%|██████▌ | 1063/1610 [4:33:38<2:17:53, 15.13s/it] {'loss': 0.0009, 'grad_norm': 1.2453516612349576, 'learning_rate': 3.3975155279503103e-07, 'completion_length': 192.1607208251953, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660715222358704, 'reward_std': 0.2987862229347229, 'kl': 0.022216796875, 'epoch': 3.3} 66%|██████▌ | 1063/1610 [4:33:38<2:17:53, 15.13s/it] 66%|██████▌ | 1064/1610 [4:33:53<2:16:43, 15.02s/it] {'loss': 0.0007, 'grad_norm': 1.3258572954155878, 'learning_rate': 3.3913043478260866e-07, 'completion_length': 159.18750762939453, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.30330248177051544, 'kl': 0.01763916015625, 'epoch': 3.3} 66%|██████▌ | 1064/1610 [4:33:53<2:16:43, 15.02s/it] 66%|██████▌ | 1065/1610 [4:34:08<2:16:08, 14.99s/it] {'loss': 0.0006, 'grad_norm': 1.2460739391412698, 'learning_rate': 3.385093167701863e-07, 'completion_length': 155.98214721679688, 'rewards/accuracy_reward': 0.267857164144516, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.2410715222358704, 'reward_std': 0.35440900921821594, 'kl': 0.015716552734375, 'epoch': 3.31} 66%|██████▌ | 1065/1610 [4:34:08<2:16:08, 14.99s/it] 66%|██████▌ | 1066/1610 [4:34:21<2:12:16, 14.59s/it] {'loss': 0.0006, 'grad_norm': 1.1176055497348025, 'learning_rate': 3.37888198757764e-07, 'completion_length': 141.12500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.30928783118724823, 'kl': 0.014556884765625, 'epoch': 3.31} 66%|██████▌ | 1066/1610 [4:34:21<2:12:16, 14.59s/it] 66%|██████▋ | 1067/1610 [4:34:35<2:09:29, 14.31s/it] {'loss': 0.0007, 'grad_norm': 2.2065630590804015, 'learning_rate': 3.372670807453416e-07, 'completion_length': 141.99107360839844, 'rewards/accuracy_reward': 0.4017857238650322, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.2696296274662018, 'kl': 0.017822265625, 'epoch': 3.31} 66%|██████▋ | 1067/1610 [4:34:35<2:09:29, 14.31s/it] 66%|██████▋ | 1068/1610 [4:34:50<2:11:45, 14.58s/it] {'loss': 0.0007, 'grad_norm': 0.7335862059428849, 'learning_rate': 3.3664596273291924e-07, 'completion_length': 159.0982208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.13615721464157104, 'kl': 0.016632080078125, 'epoch': 3.32} 66%|██████▋ | 1068/1610 [4:34:50<2:11:45, 14.58s/it] 66%|██████▋ | 1069/1610 [4:35:05<2:12:54, 14.74s/it] {'loss': 0.0007, 'grad_norm': 1.030872955494639, 'learning_rate': 3.360248447204969e-07, 'completion_length': 186.46429443359375, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.22754104062914848, 'kl': 0.01715087890625, 'epoch': 3.32} 66%|██████▋ | 1069/1610 [4:35:05<2:12:54, 14.74s/it] 66%|██████▋ | 1070/1610 [4:35:19<2:09:25, 14.38s/it] {'loss': 0.0006, 'grad_norm': 1.8234658928179115, 'learning_rate': 3.3540372670807456e-07, 'completion_length': 129.5357208251953, 'rewards/accuracy_reward': 0.3928571790456772, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.29097501933574677, 'kl': 0.014801025390625, 'epoch': 3.32} 66%|██████▋ | 1070/1610 [4:35:19<2:09:25, 14.38s/it] 67%|██████▋ | 1071/1610 [4:35:33<2:09:03, 14.37s/it] {'loss': 0.0008, 'grad_norm': 1.565899337038943, 'learning_rate': 3.347826086956522e-07, 'completion_length': 150.33929443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.4265187829732895, 'kl': 0.019927978515625, 'epoch': 3.33} 67%|██████▋ | 1071/1610 [4:35:33<2:09:03, 14.37s/it] 67%|██████▋ | 1072/1610 [4:35:50<2:16:12, 15.19s/it] {'loss': 0.001, 'grad_norm': 1.3539831454731925, 'learning_rate': 3.3416149068322977e-07, 'completion_length': 173.23214721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.3925947993993759, 'kl': 0.0247802734375, 'epoch': 3.33} 67%|██████▋ | 1072/1610 [4:35:50<2:16:12, 15.19s/it] 67%|██████▋ | 1073/1610 [4:36:05<2:14:59, 15.08s/it] {'loss': 0.0007, 'grad_norm': 1.5988117407081748, 'learning_rate': 3.335403726708074e-07, 'completion_length': 160.1339340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.37718459963798523, 'kl': 0.01654052734375, 'epoch': 3.33} 67%|██████▋ | 1073/1610 [4:36:05<2:14:59, 15.08s/it] 67%|██████▋ | 1074/1610 [4:36:20<2:15:38, 15.18s/it] {'loss': 0.0007, 'grad_norm': 1.6398709292843014, 'learning_rate': 3.3291925465838504e-07, 'completion_length': 151.10714721679688, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.25791002810001373, 'kl': 0.01702880859375, 'epoch': 3.34} 67%|██████▋ | 1074/1610 [4:36:20<2:15:38, 15.18s/it] 67%|██████▋ | 1075/1610 [4:36:36<2:15:45, 15.23s/it] {'loss': 0.0006, 'grad_norm': 1.591191751408611, 'learning_rate': 3.322981366459627e-07, 'completion_length': 166.2946548461914, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.294877827167511, 'kl': 0.015380859375, 'epoch': 3.34} 67%|██████▋ | 1075/1610 [4:36:36<2:15:45, 15.23s/it] 67%|██████▋ | 1076/1610 [4:36:50<2:12:22, 14.87s/it] {'loss': 0.0007, 'grad_norm': 1.33900162056978, 'learning_rate': 3.3167701863354036e-07, 'completion_length': 144.31250762939453, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.36882463097572327, 'kl': 0.016937255859375, 'epoch': 3.34} 67%|██████▋ | 1076/1610 [4:36:50<2:12:22, 14.87s/it] 67%|██████▋ | 1077/1610 [4:37:05<2:13:11, 14.99s/it] {'loss': 0.0008, 'grad_norm': 1.2578937921748417, 'learning_rate': 3.31055900621118e-07, 'completion_length': 157.66964721679688, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3459989130496979, 'kl': 0.01953125, 'epoch': 3.34} 67%|██████▋ | 1077/1610 [4:37:05<2:13:11, 14.99s/it] 67%|██████▋ | 1078/1610 [4:37:18<2:08:18, 14.47s/it] {'loss': 0.0005, 'grad_norm': 1.4154024646380745, 'learning_rate': 3.304347826086956e-07, 'completion_length': 137.04464721679688, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.23925502598285675, 'kl': 0.013580322265625, 'epoch': 3.35} 67%|██████▋ | 1078/1610 [4:37:18<2:08:18, 14.47s/it] 67%|██████▋ | 1079/1610 [4:37:34<2:10:59, 14.80s/it] {'loss': 0.0009, 'grad_norm': 1.383575744679071, 'learning_rate': 3.298136645962733e-07, 'completion_length': 164.0089340209961, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.3174412250518799, 'kl': 0.02252197265625, 'epoch': 3.35} 67%|██████▋ | 1079/1610 [4:37:34<2:10:59, 14.80s/it] 67%|██████▋ | 1080/1610 [4:37:49<2:10:34, 14.78s/it] {'loss': 0.0007, 'grad_norm': 1.260931977061791, 'learning_rate': 3.2919254658385094e-07, 'completion_length': 154.8214340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.33900441229343414, 'kl': 0.017578125, 'epoch': 3.35} 67%|██████▋ | 1080/1610 [4:37:49<2:10:34, 14.78s/it] 67%|██████▋ | 1081/1610 [4:38:04<2:12:26, 15.02s/it] {'loss': 0.0008, 'grad_norm': 1.3875619391495242, 'learning_rate': 3.2857142857142857e-07, 'completion_length': 148.5714340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.36733537912368774, 'kl': 0.01934814453125, 'epoch': 3.36} 67%|██████▋ | 1081/1610 [4:38:04<2:12:26, 15.02s/it] 67%|██████▋ | 1082/1610 [4:38:18<2:08:41, 14.62s/it] {'loss': 0.0006, 'grad_norm': 0.9563722931996353, 'learning_rate': 3.279503105590062e-07, 'completion_length': 136.54464721679688, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.17495086044073105, 'kl': 0.016021728515625, 'epoch': 3.36} 67%|██████▋ | 1082/1610 [4:38:18<2:08:41, 14.62s/it] 67%|██████▋ | 1083/1610 [4:38:32<2:08:07, 14.59s/it] {'loss': 0.0007, 'grad_norm': 1.038910279390512, 'learning_rate': 3.273291925465838e-07, 'completion_length': 160.6339340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.18397441506385803, 'kl': 0.0166015625, 'epoch': 3.36} 67%|██████▋ | 1083/1610 [4:38:32<2:08:07, 14.59s/it] 67%|██████▋ | 1084/1610 [4:38:48<2:09:26, 14.77s/it] {'loss': 0.0007, 'grad_norm': 1.2039292017958299, 'learning_rate': 3.2670807453416147e-07, 'completion_length': 153.69644165039062, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.182762049138546, 'kl': 0.018096923828125, 'epoch': 3.37} 67%|██████▋ | 1084/1610 [4:38:48<2:09:26, 14.77s/it] 67%|██████▋ | 1085/1610 [4:39:02<2:08:06, 14.64s/it] {'loss': 0.0007, 'grad_norm': 1.1209977462325054, 'learning_rate': 3.260869565217391e-07, 'completion_length': 147.7410774230957, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2948834225535393, 'kl': 0.01678466796875, 'epoch': 3.37} 67%|██████▋ | 1085/1610 [4:39:02<2:08:06, 14.64s/it] 67%|██████▋ | 1086/1610 [4:39:16<2:05:38, 14.39s/it] {'loss': 0.0006, 'grad_norm': 0.8459622736164474, 'learning_rate': 3.2546583850931673e-07, 'completion_length': 140.71429443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.16653180867433548, 'kl': 0.0155029296875, 'epoch': 3.37} 67%|██████▋ | 1086/1610 [4:39:16<2:05:38, 14.39s/it] 68%|██████▊ | 1087/1610 [4:39:32<2:11:04, 15.04s/it] {'loss': 0.0006, 'grad_norm': 1.1145186500724746, 'learning_rate': 3.2484472049689437e-07, 'completion_length': 166.82144165039062, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.3059394210577011, 'kl': 0.014495849609375, 'epoch': 3.38} 68%|██████▊ | 1087/1610 [4:39:32<2:11:04, 15.04s/it] 68%|██████▊ | 1088/1610 [4:39:48<2:13:05, 15.30s/it] {'loss': 0.0008, 'grad_norm': 10.596069507510228, 'learning_rate': 3.2422360248447205e-07, 'completion_length': 167.70536041259766, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.370031401515007, 'kl': 0.02081298828125, 'epoch': 3.38} 68%|██████▊ | 1088/1610 [4:39:48<2:13:05, 15.30s/it] 68%|██████▊ | 1089/1610 [4:40:02<2:09:17, 14.89s/it] {'loss': 0.0007, 'grad_norm': 0.9664973046267096, 'learning_rate': 3.236024844720497e-07, 'completion_length': 147.21429443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.23266182094812393, 'kl': 0.016510009765625, 'epoch': 3.38} 68%|██████▊ | 1089/1610 [4:40:02<2:09:17, 14.89s/it] 68%|██████▊ | 1090/1610 [4:40:18<2:10:41, 15.08s/it] {'loss': 0.001, 'grad_norm': 1.5164430115256353, 'learning_rate': 3.229813664596273e-07, 'completion_length': 165.3482208251953, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910714626312256, 'reward_std': 0.3045148700475693, 'kl': 0.02392578125, 'epoch': 3.39} 68%|██████▊ | 1090/1610 [4:40:18<2:10:41, 15.08s/it] 68%|██████▊ | 1091/1610 [4:40:34<2:12:36, 15.33s/it] {'loss': 0.0007, 'grad_norm': 1.0432813692192535, 'learning_rate': 3.2236024844720495e-07, 'completion_length': 179.43750762939453, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6428571939468384, 'reward_std': 0.29670368134975433, 'kl': 0.01824951171875, 'epoch': 3.39} 68%|██████▊ | 1091/1610 [4:40:34<2:12:36, 15.33s/it] 68%|██████▊ | 1092/1610 [4:40:48<2:10:04, 15.07s/it] {'loss': 0.0007, 'grad_norm': 1.6790434636658793, 'learning_rate': 3.217391304347826e-07, 'completion_length': 152.40179443359375, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.3026946038007736, 'kl': 0.0186767578125, 'epoch': 3.39} 68%|██████▊ | 1092/1610 [4:40:48<2:10:04, 15.07s/it] 68%|██████▊ | 1093/1610 [4:41:03<2:09:57, 15.08s/it] {'loss': 0.0007, 'grad_norm': 3.154763121996625, 'learning_rate': 3.2111801242236027e-07, 'completion_length': 159.87500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.33184562623500824, 'kl': 0.01776123046875, 'epoch': 3.39} 68%|██████▊ | 1093/1610 [4:41:03<2:09:57, 15.08s/it] 68%|██████▊ | 1094/1610 [4:41:18<2:08:15, 14.91s/it] {'loss': 0.0008, 'grad_norm': 1.1848915589909967, 'learning_rate': 3.204968944099379e-07, 'completion_length': 154.00000762939453, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.2735324054956436, 'kl': 0.02069091796875, 'epoch': 3.4} 68%|██████▊ | 1094/1610 [4:41:18<2:08:15, 14.91s/it] 68%|██████▊ | 1095/1610 [4:41:33<2:09:40, 15.11s/it] {'loss': 0.0008, 'grad_norm': 1.0442415989844858, 'learning_rate': 3.198757763975155e-07, 'completion_length': 171.3482208251953, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2663346827030182, 'kl': 0.0211181640625, 'epoch': 3.4} 68%|██████▊ | 1095/1610 [4:41:33<2:09:40, 15.11s/it] 68%|██████▊ | 1096/1610 [4:41:48<2:09:27, 15.11s/it] {'loss': 0.0006, 'grad_norm': 2.2861189403786515, 'learning_rate': 3.192546583850931e-07, 'completion_length': 164.40179443359375, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.2967092841863632, 'kl': 0.016021728515625, 'epoch': 3.4} 68%|██████▊ | 1096/1610 [4:41:48<2:09:27, 15.11s/it] 68%|██████▊ | 1097/1610 [4:42:04<2:10:48, 15.30s/it] {'loss': 0.0006, 'grad_norm': 1.3725570557438236, 'learning_rate': 3.186335403726708e-07, 'completion_length': 152.16964721679688, 'rewards/accuracy_reward': 0.321428582072258, 'rewards/format_reward': 1.0, 'reward': 1.321428656578064, 'reward_std': 0.21972985565662384, 'kl': 0.015167236328125, 'epoch': 3.41} 68%|██████▊ | 1097/1610 [4:42:04<2:10:48, 15.30s/it] 68%|██████▊ | 1098/1610 [4:42:19<2:09:53, 15.22s/it] {'loss': 0.0006, 'grad_norm': 1.3235799850438996, 'learning_rate': 3.1801242236024843e-07, 'completion_length': 139.86608123779297, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.22363825142383575, 'kl': 0.014739990234375, 'epoch': 3.41} 68%|██████▊ | 1098/1610 [4:42:19<2:09:53, 15.22s/it] 68%|██████▊ | 1099/1610 [4:42:34<2:07:32, 14.98s/it] {'loss': 0.0006, 'grad_norm': 1.027926350922806, 'learning_rate': 3.1739130434782606e-07, 'completion_length': 137.93750762939453, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.208614781498909, 'kl': 0.01495361328125, 'epoch': 3.41} 68%|██████▊ | 1099/1610 [4:42:34<2:07:32, 14.98s/it] 68%|██████▊ | 1100/1610 [4:42:49<2:07:10, 14.96s/it] {'loss': 0.0007, 'grad_norm': 1.7429277011168385, 'learning_rate': 3.167701863354037e-07, 'completion_length': 147.74108123779297, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285715222358704, 'reward_std': 0.3700370043516159, 'kl': 0.0174560546875, 'epoch': 3.42} 68%|██████▊ | 1100/1610 [4:42:49<2:07:10, 14.96s/it] 68%|██████▊ | 1101/1610 [4:43:51<4:08:41, 29.31s/it] {'loss': 0.0006, 'grad_norm': 0.8874970168175542, 'learning_rate': 3.1614906832298133e-07, 'completion_length': 175.14286041259766, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.20411308109760284, 'kl': 0.01611328125, 'epoch': 3.42} 68%|██████▊ | 1101/1610 [4:43:51<4:08:41, 29.31s/it] 68%|██████▊ | 1102/1610 [4:44:06<3:32:06, 25.05s/it] {'loss': 0.0007, 'grad_norm': 1.9661257819330455, 'learning_rate': 3.15527950310559e-07, 'completion_length': 157.41964721679688, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.28706663846969604, 'kl': 0.01654052734375, 'epoch': 3.42} 68%|██████▊ | 1102/1610 [4:44:06<3:32:06, 25.05s/it] 69%|██████▊ | 1103/1610 [4:44:20<3:03:50, 21.76s/it] {'loss': 0.0007, 'grad_norm': 1.3189063415423987, 'learning_rate': 3.1490683229813665e-07, 'completion_length': 139.71429443359375, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.22605740278959274, 'kl': 0.018310546875, 'epoch': 3.43} 69%|██████▊ | 1103/1610 [4:44:20<3:03:50, 21.76s/it] 69%|██████▊ | 1104/1610 [4:44:36<2:47:44, 19.89s/it] {'loss': 0.0007, 'grad_norm': 1.3413068187843544, 'learning_rate': 3.142857142857143e-07, 'completion_length': 185.63394165039062, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.23717807233333588, 'kl': 0.0177001953125, 'epoch': 3.43} 69%|██████▊ | 1104/1610 [4:44:36<2:47:44, 19.89s/it] 69%|██████▊ | 1105/1610 [4:44:51<2:35:35, 18.49s/it] {'loss': 0.0008, 'grad_norm': 13.778800095933153, 'learning_rate': 3.136645962732919e-07, 'completion_length': 180.93750762939453, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.3850492835044861, 'kl': 0.01947021484375, 'epoch': 3.43} 69%|██████▊ | 1105/1610 [4:44:51<2:35:35, 18.49s/it] 69%|██████▊ | 1106/1610 [4:45:05<2:23:23, 17.07s/it] {'loss': 0.0005, 'grad_norm': 2.2789402450027865, 'learning_rate': 3.130434782608696e-07, 'completion_length': 145.05358123779297, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.16141663491725922, 'kl': 0.01220703125, 'epoch': 3.43} 69%|██████▊ | 1106/1610 [4:45:05<2:23:23, 17.07s/it] 69%|██████▉ | 1107/1610 [4:45:20<2:17:11, 16.37s/it] {'loss': 0.0007, 'grad_norm': 1.7513974797246141, 'learning_rate': 3.1242236024844723e-07, 'completion_length': 160.02679443359375, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.3499073088169098, 'kl': 0.017608642578125, 'epoch': 3.44} 69%|██████▉ | 1107/1610 [4:45:20<2:17:11, 16.37s/it] 69%|██████▉ | 1108/1610 [4:45:36<2:16:03, 16.26s/it] {'loss': 0.0007, 'grad_norm': 1.1674986488842969, 'learning_rate': 3.118012422360248e-07, 'completion_length': 174.40179443359375, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.19837883114814758, 'kl': 0.01641845703125, 'epoch': 3.44} 69%|██████▉ | 1108/1610 [4:45:36<2:16:03, 16.26s/it] 69%|██████▉ | 1109/1610 [4:45:51<2:14:14, 16.08s/it] {'loss': 0.0007, 'grad_norm': 0.9918619056655192, 'learning_rate': 3.1118012422360244e-07, 'completion_length': 166.68750762939453, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.18788283318281174, 'kl': 0.017486572265625, 'epoch': 3.44} 69%|██████▉ | 1109/1610 [4:45:51<2:14:14, 16.08s/it] 69%|██████▉ | 1110/1610 [4:46:06<2:10:46, 15.69s/it] {'loss': 0.0005, 'grad_norm': 0.8646260498929609, 'learning_rate': 3.105590062111801e-07, 'completion_length': 164.3482208251953, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.24619604647159576, 'kl': 0.01312255859375, 'epoch': 3.45} 69%|██████▉ | 1110/1610 [4:46:06<2:10:46, 15.69s/it] 69%|██████▉ | 1111/1610 [4:46:21<2:07:05, 15.28s/it] {'loss': 0.0006, 'grad_norm': 1.672060467927227, 'learning_rate': 3.0993788819875776e-07, 'completion_length': 122.87500762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.32255640625953674, 'kl': 0.0159912109375, 'epoch': 3.45} 69%|██████▉ | 1111/1610 [4:46:21<2:07:05, 15.28s/it] 69%|██████▉ | 1112/1610 [4:46:35<2:04:34, 15.01s/it] {'loss': 0.0006, 'grad_norm': 1.8327654168700935, 'learning_rate': 3.093167701863354e-07, 'completion_length': 160.31250762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.3123260587453842, 'kl': 0.014892578125, 'epoch': 3.45} 69%|██████▉ | 1112/1610 [4:46:35<2:04:34, 15.01s/it] 69%|██████▉ | 1113/1610 [4:46:49<2:01:54, 14.72s/it] {'loss': 0.0005, 'grad_norm': 1.2101307895047877, 'learning_rate': 3.08695652173913e-07, 'completion_length': 145.6339340209961, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2501044422388077, 'kl': 0.01373291015625, 'epoch': 3.46} 69%|██████▉ | 1113/1610 [4:46:49<2:01:54, 14.72s/it] 69%|██████▉ | 1114/1610 [4:47:04<2:03:14, 14.91s/it] {'loss': 0.0006, 'grad_norm': 1.1118150610044835, 'learning_rate': 3.0807453416149066e-07, 'completion_length': 180.3571548461914, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.22215458750724792, 'kl': 0.015960693359375, 'epoch': 3.46} 69%|██████▉ | 1114/1610 [4:47:04<2:03:14, 14.91s/it] 69%|██████▉ | 1115/1610 [4:47:18<2:00:26, 14.60s/it] {'loss': 0.0006, 'grad_norm': 0.9881967537163734, 'learning_rate': 3.0745341614906834e-07, 'completion_length': 143.58929061889648, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875001192092896, 'reward_std': 0.20619001984596252, 'kl': 0.0155181884765625, 'epoch': 3.46} 69%|██████▉ | 1115/1610 [4:47:18<2:00:26, 14.60s/it] 69%|██████▉ | 1116/1610 [4:47:34<2:03:15, 14.97s/it] {'loss': 0.0009, 'grad_norm': 1.1114245416787274, 'learning_rate': 3.06832298136646e-07, 'completion_length': 172.7589340209961, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3078041896224022, 'kl': 0.02239990234375, 'epoch': 3.47} 69%|██████▉ | 1116/1610 [4:47:34<2:03:15, 14.97s/it] 69%|██████▉ | 1117/1610 [4:47:49<2:04:00, 15.09s/it] {'loss': 0.0006, 'grad_norm': 1.8448555087935639, 'learning_rate': 3.062111801242236e-07, 'completion_length': 147.86608123779297, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.2948834300041199, 'kl': 0.0140380859375, 'epoch': 3.47} 69%|██████▉ | 1117/1610 [4:47:49<2:04:00, 15.09s/it] 69%|██████▉ | 1118/1610 [4:48:06<2:06:28, 15.42s/it] {'loss': 0.0008, 'grad_norm': 1.5985359336983802, 'learning_rate': 3.0559006211180124e-07, 'completion_length': 167.40179443359375, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.29609018564224243, 'kl': 0.02056884765625, 'epoch': 3.47} 69%|██████▉ | 1118/1610 [4:48:06<2:06:28, 15.42s/it] 70%|██████▉ | 1119/1610 [4:48:20<2:04:59, 15.27s/it] {'loss': 0.0006, 'grad_norm': 1.4499289475985848, 'learning_rate': 3.049689440993788e-07, 'completion_length': 138.50893783569336, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.1956884115934372, 'kl': 0.0147705078125, 'epoch': 3.48} 70%|██████▉ | 1119/1610 [4:48:20<2:04:59, 15.27s/it] 70%|██████▉ | 1120/1610 [4:48:36<2:05:58, 15.43s/it] {'loss': 0.0008, 'grad_norm': 1.0501231852086375, 'learning_rate': 3.043478260869565e-07, 'completion_length': 160.17857360839844, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.2960957884788513, 'kl': 0.018798828125, 'epoch': 3.48} 70%|██████▉ | 1120/1610 [4:48:36<2:05:58, 15.43s/it] 70%|██████▉ | 1121/1610 [4:48:52<2:05:54, 15.45s/it] {'loss': 0.0006, 'grad_norm': 1.7859034015241961, 'learning_rate': 3.0372670807453414e-07, 'completion_length': 144.52679061889648, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4196429252624512, 'reward_std': 0.3667364865541458, 'kl': 0.016204833984375, 'epoch': 3.48} 70%|██████▉ | 1121/1610 [4:48:52<2:05:54, 15.45s/it] 70%|██████▉ | 1122/1610 [4:49:06<2:01:54, 14.99s/it] {'loss': 0.0006, 'grad_norm': 1.1050299581373992, 'learning_rate': 3.0310559006211177e-07, 'completion_length': 127.96428680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.21374450623989105, 'kl': 0.01531982421875, 'epoch': 3.48} 70%|██████▉ | 1122/1610 [4:49:06<2:01:54, 14.99s/it] 70%|██████▉ | 1123/1610 [4:49:21<2:02:28, 15.09s/it] {'loss': 0.0007, 'grad_norm': 1.3454535897284834, 'learning_rate': 3.024844720496894e-07, 'completion_length': 165.9464340209961, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2858598679304123, 'kl': 0.016265869140625, 'epoch': 3.49} 70%|██████▉ | 1123/1610 [4:49:21<2:02:28, 15.09s/it] 70%|██████▉ | 1124/1610 [4:49:35<1:58:23, 14.62s/it] {'loss': 0.0005, 'grad_norm': 1.3308624136332723, 'learning_rate': 3.018633540372671e-07, 'completion_length': 123.2589340209961, 'rewards/accuracy_reward': 0.785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.27805428206920624, 'kl': 0.01300048828125, 'epoch': 3.49} 70%|██████▉ | 1124/1610 [4:49:35<1:58:23, 14.62s/it] 70%|██████▉ | 1125/1610 [4:49:50<2:00:15, 14.88s/it] {'loss': 0.0007, 'grad_norm': 1.1142950156061155, 'learning_rate': 3.012422360248447e-07, 'completion_length': 177.21429443359375, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.2314438372850418, 'kl': 0.017822265625, 'epoch': 3.49} 70%|██████▉ | 1125/1610 [4:49:50<2:00:15, 14.88s/it] 70%|██████▉ | 1126/1610 [4:50:06<2:01:37, 15.08s/it] {'loss': 0.0007, 'grad_norm': 1.7924743491609405, 'learning_rate': 3.0062111801242235e-07, 'completion_length': 165.90178680419922, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.36795443296432495, 'kl': 0.01788330078125, 'epoch': 3.5} 70%|██████▉ | 1126/1610 [4:50:06<2:01:37, 15.08s/it] 70%|███████ | 1127/1610 [4:50:19<1:57:38, 14.61s/it] {'loss': 0.0005, 'grad_norm': 2.486555414143092, 'learning_rate': 3e-07, 'completion_length': 131.3482208251953, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2726622372865677, 'kl': 0.011871337890625, 'epoch': 3.5} 70%|███████ | 1127/1610 [4:50:19<1:57:38, 14.61s/it] 70%|███████ | 1128/1610 [4:50:35<2:00:51, 15.05s/it] {'loss': 0.0006, 'grad_norm': 1.1387161236921721, 'learning_rate': 2.993788819875776e-07, 'completion_length': 169.8482208251953, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.13346679508686066, 'kl': 0.015869140625, 'epoch': 3.5} 70%|███████ | 1128/1610 [4:50:35<2:00:51, 15.05s/it] 70%|███████ | 1129/1610 [4:50:51<2:03:10, 15.36s/it] {'loss': 0.0007, 'grad_norm': 1.6333051481677305, 'learning_rate': 2.987577639751553e-07, 'completion_length': 172.49108123779297, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.446428656578064, 'reward_std': 0.34725306928157806, 'kl': 0.01806640625, 'epoch': 3.51} 70%|███████ | 1129/1610 [4:50:51<2:03:10, 15.36s/it] 70%|███████ | 1130/1610 [4:51:07<2:02:52, 15.36s/it] {'loss': 0.0007, 'grad_norm': 1.2517790078213207, 'learning_rate': 2.9813664596273294e-07, 'completion_length': 159.37500762939453, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.36429715156555176, 'kl': 0.0185546875, 'epoch': 3.51} 70%|███████ | 1130/1610 [4:51:07<2:02:52, 15.36s/it] 70%|███████ | 1131/1610 [4:51:22<2:01:43, 15.25s/it] {'loss': 0.0006, 'grad_norm': 1.6202881199490304, 'learning_rate': 2.975155279503105e-07, 'completion_length': 137.32144165039062, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.24229325354099274, 'kl': 0.0162353515625, 'epoch': 3.51} 70%|███████ | 1131/1610 [4:51:22<2:01:43, 15.25s/it] 70%|███████ | 1132/1610 [4:51:37<2:01:12, 15.21s/it] {'loss': 0.0008, 'grad_norm': 2.582187940745588, 'learning_rate': 2.9689440993788815e-07, 'completion_length': 158.93750762939453, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928572535514832, 'reward_std': 0.32976868748664856, 'kl': 0.01885986328125, 'epoch': 3.52} 70%|███████ | 1132/1610 [4:51:37<2:01:12, 15.21s/it] 70%|███████ | 1133/1610 [4:51:54<2:05:30, 15.79s/it] {'loss': 0.0008, 'grad_norm': 4.12139909477086, 'learning_rate': 2.9627329192546583e-07, 'completion_length': 178.54464721679688, 'rewards/accuracy_reward': 0.5, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.33844442665576935, 'kl': 0.0194091796875, 'epoch': 3.52} 70%|███████ | 1133/1610 [4:51:54<2:05:30, 15.79s/it] 70%|███████ | 1134/1610 [4:52:08<2:01:36, 15.33s/it] {'loss': 0.0006, 'grad_norm': 2.7189630107606604, 'learning_rate': 2.9565217391304347e-07, 'completion_length': 146.1607208251953, 'rewards/accuracy_reward': 0.5625000447034836, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.25280046463012695, 'kl': 0.014984130859375, 'epoch': 3.52} 70%|███████ | 1134/1610 [4:52:08<2:01:36, 15.33s/it] 70%|███████ | 1135/1610 [4:52:23<2:00:06, 15.17s/it] {'loss': 0.0007, 'grad_norm': 0.9479681359831416, 'learning_rate': 2.950310559006211e-07, 'completion_length': 151.08036041259766, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5000000596046448, 'reward_std': 0.22875341773033142, 'kl': 0.0167236328125, 'epoch': 3.52} 70%|███████ | 1135/1610 [4:52:23<2:00:06, 15.17s/it] 71%|███████ | 1136/1610 [4:52:39<2:02:31, 15.51s/it] {'loss': 0.0008, 'grad_norm': 1.0202075485244868, 'learning_rate': 2.9440993788819873e-07, 'completion_length': 172.9553680419922, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4553571939468384, 'reward_std': 0.27493715286254883, 'kl': 0.0206298828125, 'epoch': 3.53} 71%|███████ | 1136/1610 [4:52:39<2:02:31, 15.51s/it] 71%|███████ | 1137/1610 [4:52:55<2:01:56, 15.47s/it] {'loss': 0.0006, 'grad_norm': 2.3681398048686644, 'learning_rate': 2.9378881987577636e-07, 'completion_length': 152.95536041259766, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.27804866433143616, 'kl': 0.014251708984375, 'epoch': 3.53} 71%|███████ | 1137/1610 [4:52:55<2:01:56, 15.47s/it] 71%|███████ | 1138/1610 [4:53:10<2:00:54, 15.37s/it] {'loss': 0.0006, 'grad_norm': 1.0534686449317339, 'learning_rate': 2.9316770186335405e-07, 'completion_length': 163.5357208251953, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875001192092896, 'reward_std': 0.19178561866283417, 'kl': 0.01629638671875, 'epoch': 3.53} 71%|███████ | 1138/1610 [4:53:10<2:00:54, 15.37s/it] 71%|███████ | 1139/1610 [4:53:26<2:03:21, 15.71s/it] {'loss': 0.0008, 'grad_norm': 0.9291419471729334, 'learning_rate': 2.925465838509317e-07, 'completion_length': 184.9821548461914, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.14639316499233246, 'kl': 0.02105712890625, 'epoch': 3.54} 71%|███████ | 1139/1610 [4:53:26<2:03:21, 15.71s/it] 71%|███████ | 1140/1610 [4:53:42<2:03:02, 15.71s/it] {'loss': 0.0006, 'grad_norm': 5.693618464918152, 'learning_rate': 2.919254658385093e-07, 'completion_length': 148.71429443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.19057884812355042, 'kl': 0.014556884765625, 'epoch': 3.54} 71%|███████ | 1140/1610 [4:53:42<2:03:02, 15.71s/it] 71%|███████ | 1141/1610 [4:53:58<2:04:30, 15.93s/it] {'loss': 0.0007, 'grad_norm': 3.482361334768924, 'learning_rate': 2.9130434782608695e-07, 'completion_length': 165.24108123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160714626312256, 'reward_std': 0.2701839953660965, 'kl': 0.01849365234375, 'epoch': 3.54} 71%|███████ | 1141/1610 [4:53:58<2:04:30, 15.93s/it] 71%|███████ | 1142/1610 [4:54:13<2:01:30, 15.58s/it] {'loss': 0.0005, 'grad_norm': 1.9156147796387129, 'learning_rate': 2.9068322981366463e-07, 'completion_length': 133.36607360839844, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.24680951237678528, 'kl': 0.012115478515625, 'epoch': 3.55} 71%|███████ | 1142/1610 [4:54:13<2:01:30, 15.58s/it] 71%|███████ | 1143/1610 [4:54:28<2:00:24, 15.47s/it] {'loss': 0.0006, 'grad_norm': 0.7575111436865277, 'learning_rate': 2.900621118012422e-07, 'completion_length': 168.10714721679688, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.21973545849323273, 'kl': 0.015655517578125, 'epoch': 3.55} 71%|███████ | 1143/1610 [4:54:28<2:00:24, 15.47s/it] 71%|███████ | 1144/1610 [4:54:42<1:55:24, 14.86s/it] {'loss': 0.0006, 'grad_norm': 3.088484499346124, 'learning_rate': 2.8944099378881985e-07, 'completion_length': 128.27679443359375, 'rewards/accuracy_reward': 0.4017857164144516, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.1731250137090683, 'kl': 0.015350341796875, 'epoch': 3.55} 71%|███████ | 1144/1610 [4:54:42<1:55:24, 14.86s/it] 71%|███████ | 1145/1610 [4:54:57<1:55:13, 14.87s/it] {'loss': 0.0006, 'grad_norm': 1.9302108351281044, 'learning_rate': 2.888198757763975e-07, 'completion_length': 177.2232208251953, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.24889205396175385, 'kl': 0.015899658203125, 'epoch': 3.56} 71%|███████ | 1145/1610 [4:54:57<1:55:13, 14.87s/it] 71%|███████ | 1146/1610 [4:55:12<1:56:10, 15.02s/it] {'loss': 0.0006, 'grad_norm': 1.2143629757271486, 'learning_rate': 2.881987577639751e-07, 'completion_length': 151.08036041259766, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.19751425087451935, 'kl': 0.0155029296875, 'epoch': 3.56} 71%|███████ | 1146/1610 [4:55:12<1:56:10, 15.02s/it] 71%|███████ | 1147/1610 [4:55:27<1:55:55, 15.02s/it] {'loss': 0.0005, 'grad_norm': 1.3194352459539527, 'learning_rate': 2.875776397515528e-07, 'completion_length': 145.56250762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.19178561866283417, 'kl': 0.011810302734375, 'epoch': 3.56} 71%|███████ | 1147/1610 [4:55:27<1:55:55, 15.02s/it] 71%|███████▏ | 1148/1610 [4:55:42<1:55:20, 14.98s/it] {'loss': 0.0007, 'grad_norm': 1.1164387778291065, 'learning_rate': 2.8695652173913043e-07, 'completion_length': 157.3482208251953, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.22336697578430176, 'kl': 0.01702880859375, 'epoch': 3.57} 71%|███████▏ | 1148/1610 [4:55:42<1:55:20, 14.98s/it] 71%|███████▏ | 1149/1610 [4:55:57<1:56:00, 15.10s/it] {'loss': 0.0007, 'grad_norm': 1.3914716310942277, 'learning_rate': 2.8633540372670806e-07, 'completion_length': 149.2857208251953, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2726622372865677, 'kl': 0.0166015625, 'epoch': 3.57} 71%|███████▏ | 1149/1610 [4:55:57<1:56:00, 15.10s/it] 71%|███████▏ | 1150/1610 [4:56:13<1:57:13, 15.29s/it] {'loss': 0.0007, 'grad_norm': 0.971959326203144, 'learning_rate': 2.857142857142857e-07, 'completion_length': 182.75000762939453, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.3839285969734192, 'reward_std': 0.24889205396175385, 'kl': 0.0181884765625, 'epoch': 3.57} 71%|███████▏ | 1150/1610 [4:56:13<1:57:13, 15.29s/it] 71%|███████▏ | 1151/1610 [4:56:29<1:58:04, 15.43s/it] {'loss': 0.0007, 'grad_norm': 0.8827364389377199, 'learning_rate': 2.850931677018634e-07, 'completion_length': 176.7053680419922, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.24498926103115082, 'kl': 0.0174560546875, 'epoch': 3.57} 71%|███████▏ | 1151/1610 [4:56:29<1:58:04, 15.43s/it] 72%|███████▏ | 1152/1610 [4:56:45<1:59:27, 15.65s/it] {'loss': 0.0008, 'grad_norm': 1.3126280875731435, 'learning_rate': 2.84472049689441e-07, 'completion_length': 155.58929443359375, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2702430933713913, 'kl': 0.020263671875, 'epoch': 3.58} 72%|███████▏ | 1152/1610 [4:56:45<1:59:27, 15.65s/it] 72%|███████▏ | 1153/1610 [4:56:58<1:53:36, 14.92s/it] {'loss': 0.0006, 'grad_norm': 1.5471809727404873, 'learning_rate': 2.8385093167701864e-07, 'completion_length': 127.16071701049805, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.26841723918914795, 'kl': 0.013916015625, 'epoch': 3.58} 72%|███████▏ | 1153/1610 [4:56:58<1:53:36, 14.92s/it] 72%|███████▏ | 1154/1610 [4:57:12<1:51:52, 14.72s/it] {'loss': 0.0007, 'grad_norm': 2.520036868818031, 'learning_rate': 2.832298136645963e-07, 'completion_length': 140.6339340209961, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.2960902154445648, 'kl': 0.01806640625, 'epoch': 3.58} 72%|███████▏ | 1154/1610 [4:57:12<1:51:52, 14.72s/it] 72%|███████▏ | 1155/1610 [4:57:29<1:55:14, 15.20s/it] {'loss': 0.0008, 'grad_norm': 1.1424252888722914, 'learning_rate': 2.8260869565217386e-07, 'completion_length': 186.36608123779297, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.32013723254203796, 'kl': 0.01983642578125, 'epoch': 3.59} 72%|███████▏ | 1155/1610 [4:57:29<1:55:14, 15.20s/it] 72%|███████▏ | 1156/1610 [4:57:44<1:55:37, 15.28s/it] {'loss': 0.0007, 'grad_norm': 2.908450478734031, 'learning_rate': 2.8198757763975154e-07, 'completion_length': 172.36608123779297, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 1.0, 'reward': 1.321428656578064, 'reward_std': 0.2735324054956436, 'kl': 0.01776123046875, 'epoch': 3.59} 72%|███████▏ | 1156/1610 [4:57:44<1:55:37, 15.28s/it] 72%|███████▏ | 1157/1610 [4:58:00<1:57:02, 15.50s/it] {'loss': 0.0007, 'grad_norm': 0.9669876586539681, 'learning_rate': 2.813664596273292e-07, 'completion_length': 177.80358123779297, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.24889206886291504, 'kl': 0.01666259765625, 'epoch': 3.59} 72%|███████▏ | 1157/1610 [4:58:00<1:57:02, 15.50s/it] 72%|███████▏ | 1158/1610 [4:58:16<1:58:08, 15.68s/it] {'loss': 0.0006, 'grad_norm': 1.7678436367724217, 'learning_rate': 2.807453416149068e-07, 'completion_length': 143.50894165039062, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.23326967656612396, 'kl': 0.014556884765625, 'epoch': 3.6} 72%|███████▏ | 1158/1610 [4:58:16<1:58:08, 15.68s/it] 72%|███████▏ | 1159/1610 [4:58:34<2:02:48, 16.34s/it] {'loss': 0.0008, 'grad_norm': 1.0614640643516213, 'learning_rate': 2.8012422360248444e-07, 'completion_length': 195.9732208251953, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.383928656578064, 'reward_std': 0.24167976528406143, 'kl': 0.02099609375, 'epoch': 3.6} 72%|███████▏ | 1159/1610 [4:58:34<2:02:48, 16.34s/it] 72%|███████▏ | 1160/1610 [4:58:50<2:00:17, 16.04s/it] {'loss': 0.0006, 'grad_norm': 1.3841570526478626, 'learning_rate': 2.7950310559006207e-07, 'completion_length': 142.37500762939453, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.283163845539093, 'kl': 0.0146484375, 'epoch': 3.6} 72%|███████▏ | 1160/1610 [4:58:50<2:00:17, 16.04s/it] 72%|███████▏ | 1161/1610 [4:59:04<1:56:54, 15.62s/it] {'loss': 0.0008, 'grad_norm': 1.188873123223441, 'learning_rate': 2.7888198757763976e-07, 'completion_length': 181.46429443359375, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6339285969734192, 'reward_std': 0.33185119926929474, 'kl': 0.01873779296875, 'epoch': 3.61} 72%|███████▏ | 1161/1610 [4:59:04<1:56:54, 15.62s/it] 72%|███████▏ | 1162/1610 [4:59:20<1:56:44, 15.63s/it] {'loss': 0.0009, 'grad_norm': 2.658604227342203, 'learning_rate': 2.782608695652174e-07, 'completion_length': 180.81250762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.31561537086963654, 'kl': 0.021728515625, 'epoch': 3.61} 72%|███████▏ | 1162/1610 [4:59:20<1:56:44, 15.63s/it] 72%|███████▏ | 1163/1610 [4:59:35<1:54:39, 15.39s/it] {'loss': 0.0008, 'grad_norm': 1.6417208858566474, 'learning_rate': 2.77639751552795e-07, 'completion_length': 158.2053680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2669336050748825, 'kl': 0.01904296875, 'epoch': 3.61} 72%|███████▏ | 1163/1610 [4:59:35<1:54:39, 15.39s/it] 72%|███████▏ | 1164/1610 [4:59:50<1:54:07, 15.35s/it] {'loss': 0.0008, 'grad_norm': 1.2104376908181782, 'learning_rate': 2.7701863354037266e-07, 'completion_length': 170.36607360839844, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.7053571939468384, 'reward_std': 0.35038691759109497, 'kl': 0.0205078125, 'epoch': 3.61} 72%|███████▏ | 1164/1610 [4:59:50<1:54:07, 15.35s/it] 72%|███████▏ | 1165/1610 [5:00:05<1:52:23, 15.16s/it] {'loss': 0.0006, 'grad_norm': 1.9688271251681149, 'learning_rate': 2.7639751552795034e-07, 'completion_length': 137.0178680419922, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.31684231758117676, 'kl': 0.01422119140625, 'epoch': 3.62} 72%|███████▏ | 1165/1610 [5:00:05<1:52:23, 15.16s/it] 72%|███████▏ | 1166/1610 [5:00:21<1:53:53, 15.39s/it] {'loss': 0.0006, 'grad_norm': 0.8108777034570104, 'learning_rate': 2.7577639751552797e-07, 'completion_length': 160.7589340209961, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.7321429252624512, 'reward_std': 0.22419263422489166, 'kl': 0.01507568359375, 'epoch': 3.62} 72%|███████▏ | 1166/1610 [5:00:21<1:53:53, 15.39s/it] 72%|███████▏ | 1167/1610 [5:00:36<1:54:01, 15.44s/it] {'loss': 0.0009, 'grad_norm': 1.2105544294871275, 'learning_rate': 2.7515527950310555e-07, 'completion_length': 149.46429443359375, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4910714626312256, 'reward_std': 0.3149629980325699, 'kl': 0.02191162109375, 'epoch': 3.62} 72%|███████▏ | 1167/1610 [5:00:36<1:54:01, 15.44s/it] 73%|███████▎ | 1168/1610 [5:00:51<1:53:26, 15.40s/it] {'loss': 0.0006, 'grad_norm': 1.3193251979551273, 'learning_rate': 2.745341614906832e-07, 'completion_length': 150.1696548461914, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.1995968148112297, 'kl': 0.014434814453125, 'epoch': 3.63} 73%|███████▎ | 1168/1610 [5:00:51<1:53:26, 15.40s/it] 73%|███████▎ | 1169/1610 [5:01:05<1:49:40, 14.92s/it] {'loss': 0.0007, 'grad_norm': 2.7612473029812112, 'learning_rate': 2.739130434782608e-07, 'completion_length': 138.76786041259766, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.22936689853668213, 'kl': 0.0166015625, 'epoch': 3.63} 73%|███████▎ | 1169/1610 [5:01:05<1:49:40, 14.92s/it] 73%|███████▎ | 1170/1610 [5:01:21<1:51:40, 15.23s/it] {'loss': 0.0007, 'grad_norm': 1.5013666594797033, 'learning_rate': 2.732919254658385e-07, 'completion_length': 143.27679443359375, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.30330249667167664, 'kl': 0.01824951171875, 'epoch': 3.63} 73%|███████▎ | 1170/1610 [5:01:21<1:51:40, 15.23s/it] 73%|███████▎ | 1171/1610 [5:01:37<1:52:57, 15.44s/it] {'loss': 0.0006, 'grad_norm': 1.661095073120663, 'learning_rate': 2.7267080745341614e-07, 'completion_length': 164.11607360839844, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.2501044273376465, 'kl': 0.014739990234375, 'epoch': 3.64} 73%|███████▎ | 1171/1610 [5:01:37<1:52:57, 15.44s/it] 73%|███████▎ | 1172/1610 [5:01:53<1:53:11, 15.51s/it] {'loss': 0.0007, 'grad_norm': 1.3499941856219326, 'learning_rate': 2.7204968944099377e-07, 'completion_length': 147.10714721679688, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.696428656578064, 'reward_std': 0.2726622223854065, 'kl': 0.01837158203125, 'epoch': 3.64} 73%|███████▎ | 1172/1610 [5:01:53<1:53:11, 15.51s/it] 73%|███████▎ | 1173/1610 [5:02:09<1:54:11, 15.68s/it] {'loss': 0.0007, 'grad_norm': 2.3959292660661853, 'learning_rate': 2.714285714285714e-07, 'completion_length': 176.23214721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6696429252624512, 'reward_std': 0.24883295595645905, 'kl': 0.018341064453125, 'epoch': 3.64} 73%|███████▎ | 1173/1610 [5:02:09<1:54:11, 15.68s/it] 73%|███████▎ | 1174/1610 [5:02:24<1:52:28, 15.48s/it] {'loss': 0.0006, 'grad_norm': 1.1891165613500938, 'learning_rate': 2.708074534161491e-07, 'completion_length': 153.02679443359375, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.2663346827030182, 'kl': 0.015869140625, 'epoch': 3.65} 73%|███████▎ | 1174/1610 [5:02:24<1:52:28, 15.48s/it] 73%|███████▎ | 1175/1610 [5:02:37<1:46:43, 14.72s/it] {'loss': 0.0006, 'grad_norm': 1.4741191246050762, 'learning_rate': 2.701863354037267e-07, 'completion_length': 128.06250762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803572535514832, 'reward_std': 0.27535825967788696, 'kl': 0.016204833984375, 'epoch': 3.65} 73%|███████▎ | 1175/1610 [5:02:37<1:46:43, 14.72s/it] 73%|███████▎ | 1176/1610 [5:02:52<1:46:37, 14.74s/it] {'loss': 0.0008, 'grad_norm': 1.229197160473285, 'learning_rate': 2.6956521739130435e-07, 'completion_length': 144.96429443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2792610377073288, 'kl': 0.01898193359375, 'epoch': 3.65} 73%|███████▎ | 1176/1610 [5:02:52<1:46:37, 14.74s/it] 73%|███████▎ | 1177/1610 [5:03:06<1:46:14, 14.72s/it] {'loss': 0.0008, 'grad_norm': 0.9283560530793747, 'learning_rate': 2.68944099378882e-07, 'completion_length': 159.0982208251953, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.22754104435443878, 'kl': 0.01904296875, 'epoch': 3.66} 73%|███████▎ | 1177/1610 [5:03:06<1:46:14, 14.72s/it] 73%|███████▎ | 1178/1610 [5:03:22<1:47:09, 14.88s/it] {'loss': 0.0005, 'grad_norm': 1.5049578853213903, 'learning_rate': 2.6832298136645956e-07, 'completion_length': 134.3303680419922, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.20802147686481476, 'kl': 0.01214599609375, 'epoch': 3.66} 73%|███████▎ | 1178/1610 [5:03:22<1:47:09, 14.88s/it] 73%|███████▎ | 1179/1610 [5:03:37<1:47:48, 15.01s/it] {'loss': 0.0006, 'grad_norm': 1.0753493660374984, 'learning_rate': 2.6770186335403725e-07, 'completion_length': 164.35714721679688, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.23656462132930756, 'kl': 0.015045166015625, 'epoch': 3.66} 73%|███████▎ | 1179/1610 [5:03:37<1:47:48, 15.01s/it] 73%|███████▎ | 1180/1610 [5:03:51<1:46:29, 14.86s/it] {'loss': 0.0006, 'grad_norm': 1.537625366158758, 'learning_rate': 2.670807453416149e-07, 'completion_length': 172.2053680419922, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.36491623520851135, 'kl': 0.01507568359375, 'epoch': 3.66} 73%|███████▎ | 1180/1610 [5:03:51<1:46:29, 14.86s/it] 73%|███████▎ | 1181/1610 [5:04:08<1:49:16, 15.28s/it] {'loss': 0.0008, 'grad_norm': 1.0752533408335139, 'learning_rate': 2.664596273291925e-07, 'completion_length': 174.12500762939453, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.29367104172706604, 'kl': 0.02056884765625, 'epoch': 3.67} 73%|███████▎ | 1181/1610 [5:04:08<1:49:16, 15.28s/it] 73%|███████▎ | 1182/1610 [5:04:22<1:46:05, 14.87s/it] {'loss': 0.0006, 'grad_norm': 3.9973691083858163, 'learning_rate': 2.6583850931677015e-07, 'completion_length': 126.98215103149414, 'rewards/accuracy_reward': 0.6875000596046448, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.13225442171096802, 'kl': 0.016143798828125, 'epoch': 3.67} 73%|███████▎ | 1182/1610 [5:04:22<1:46:05, 14.87s/it] 73%|███████▎ | 1183/1610 [5:04:37<1:47:58, 15.17s/it] {'loss': 0.0007, 'grad_norm': 2.2342467112444817, 'learning_rate': 2.6521739130434783e-07, 'completion_length': 150.46429443359375, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.273532398045063, 'kl': 0.01715087890625, 'epoch': 3.67} 73%|███████▎ | 1183/1610 [5:04:37<1:47:58, 15.17s/it] 74%|███████▎ | 1184/1610 [5:04:53<1:48:25, 15.27s/it] {'loss': 0.0009, 'grad_norm': 1.0249191830998476, 'learning_rate': 2.6459627329192547e-07, 'completion_length': 178.15179443359375, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.3252524137496948, 'kl': 0.0218505859375, 'epoch': 3.68} 74%|███████▎ | 1184/1610 [5:04:53<1:48:25, 15.27s/it] 74%|███████▎ | 1185/1610 [5:05:09<1:49:37, 15.48s/it] {'loss': 0.0009, 'grad_norm': 3.5436828376630594, 'learning_rate': 2.639751552795031e-07, 'completion_length': 203.1428680419922, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.28828461468219757, 'kl': 0.021484375, 'epoch': 3.68} 74%|███████▎ | 1185/1610 [5:05:09<1:49:37, 15.48s/it] 74%|███████▎ | 1186/1610 [5:05:24<1:48:34, 15.37s/it] {'loss': 0.0008, 'grad_norm': 1.2980768764184565, 'learning_rate': 2.6335403726708073e-07, 'completion_length': 175.73214721679688, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.38357122242450714, 'kl': 0.02001953125, 'epoch': 3.68} 74%|███████▎ | 1186/1610 [5:05:24<1:48:34, 15.37s/it] 74%|███████▎ | 1187/1610 [5:05:38<1:45:56, 15.03s/it] {'loss': 0.0005, 'grad_norm': 1.2522966161308793, 'learning_rate': 2.6273291925465836e-07, 'completion_length': 133.7589340209961, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875001192092896, 'reward_std': 0.2474084198474884, 'kl': 0.012420654296875, 'epoch': 3.69} 74%|███████▎ | 1187/1610 [5:05:38<1:45:56, 15.03s/it] 74%|███████▍ | 1188/1610 [5:05:52<1:43:57, 14.78s/it] {'loss': 0.0006, 'grad_norm': 2.218505586860447, 'learning_rate': 2.6211180124223605e-07, 'completion_length': 127.91965103149414, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.19057324528694153, 'kl': 0.015869140625, 'epoch': 3.69} 74%|███████▍ | 1188/1610 [5:05:52<1:43:57, 14.78s/it] 74%|███████▍ | 1189/1610 [5:06:08<1:44:32, 14.90s/it] {'loss': 0.0009, 'grad_norm': 2.1256011245801045, 'learning_rate': 2.614906832298137e-07, 'completion_length': 146.55358123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2609482705593109, 'kl': 0.02362060546875, 'epoch': 3.69} 74%|███████▍ | 1189/1610 [5:06:08<1:44:32, 14.90s/it] 74%|███████▍ | 1190/1610 [5:06:23<1:45:34, 15.08s/it] {'loss': 0.0006, 'grad_norm': 5.3431275485491785, 'learning_rate': 2.6086956521739126e-07, 'completion_length': 164.7589340209961, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 0.9732142984867096, 'reward': 1.4821429252624512, 'reward_std': 0.2314438372850418, 'kl': 0.014312744140625, 'epoch': 3.7} 74%|███████▍ | 1190/1610 [5:06:23<1:45:34, 15.08s/it] 74%|███████▍ | 1191/1610 [5:06:36<1:40:22, 14.37s/it] {'loss': 0.0005, 'grad_norm': 1.1870615038450696, 'learning_rate': 2.602484472049689e-07, 'completion_length': 117.68750381469727, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.2507179081439972, 'kl': 0.01336669921875, 'epoch': 3.7} 74%|███████▍ | 1191/1610 [5:06:36<1:40:22, 14.37s/it] 74%|███████▍ | 1192/1610 [5:06:51<1:42:12, 14.67s/it] {'loss': 0.0008, 'grad_norm': 1.497698877434726, 'learning_rate': 2.596273291925466e-07, 'completion_length': 164.6964340209961, 'rewards/accuracy_reward': 0.4196428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4107143878936768, 'reward_std': 0.3432939499616623, 'kl': 0.02044677734375, 'epoch': 3.7} 74%|███████▍ | 1192/1610 [5:06:51<1:42:12, 14.67s/it] 74%|███████▍ | 1193/1610 [5:07:05<1:40:52, 14.51s/it] {'loss': 0.0007, 'grad_norm': 1.8007703437717448, 'learning_rate': 2.590062111801242e-07, 'completion_length': 140.96429443359375, 'rewards/accuracy_reward': 0.7232142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.1800716370344162, 'kl': 0.017303466796875, 'epoch': 3.7} 74%|███████▍ | 1193/1610 [5:07:05<1:40:52, 14.51s/it] 74%|███████▍ | 1194/1610 [5:07:21<1:43:30, 14.93s/it] {'loss': 0.0009, 'grad_norm': 2.1659821963280406, 'learning_rate': 2.5838509316770184e-07, 'completion_length': 168.7232208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2831694483757019, 'kl': 0.021484375, 'epoch': 3.71} 74%|███████▍ | 1194/1610 [5:07:21<1:43:30, 14.93s/it] 74%|███████▍ | 1195/1610 [5:07:38<1:46:01, 15.33s/it] {'loss': 0.0007, 'grad_norm': 1.3742935445771591, 'learning_rate': 2.577639751552795e-07, 'completion_length': 164.9464340209961, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.288892462849617, 'kl': 0.0181884765625, 'epoch': 3.71} 74%|███████▍ | 1195/1610 [5:07:38<1:46:01, 15.33s/it] 74%|███████▍ | 1196/1610 [5:07:53<1:46:45, 15.47s/it] {'loss': 0.0007, 'grad_norm': 0.7806422514508067, 'learning_rate': 2.571428571428571e-07, 'completion_length': 174.15178680419922, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.21973545849323273, 'kl': 0.01861572265625, 'epoch': 3.71} 74%|███████▍ | 1196/1610 [5:07:53<1:46:45, 15.47s/it] 74%|███████▍ | 1197/1610 [5:08:09<1:47:29, 15.62s/it] {'loss': 0.0009, 'grad_norm': 2.1771803584418343, 'learning_rate': 2.565217391304348e-07, 'completion_length': 152.0625, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.607142984867096, 'reward_std': 0.3018188327550888, 'kl': 0.0223388671875, 'epoch': 3.72} 74%|███████▍ | 1197/1610 [5:08:09<1:47:29, 15.62s/it] 74%|███████▍ | 1198/1610 [5:08:24<1:45:53, 15.42s/it] {'loss': 0.0006, 'grad_norm': 1.4024619353549512, 'learning_rate': 2.5590062111801243e-07, 'completion_length': 157.99108123779297, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.22996579855680466, 'kl': 0.01568603515625, 'epoch': 3.72} 74%|███████▍ | 1198/1610 [5:08:24<1:45:53, 15.42s/it] 74%|███████▍ | 1199/1610 [5:08:41<1:49:10, 15.94s/it] {'loss': 0.0008, 'grad_norm': 1.1526187829886847, 'learning_rate': 2.5527950310559006e-07, 'completion_length': 181.81250762939453, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.446428656578064, 'reward_std': 0.32255636155605316, 'kl': 0.0196533203125, 'epoch': 3.72} 74%|███████▍ | 1199/1610 [5:08:41<1:49:10, 15.94s/it] 75%|███████▍ | 1200/1610 [5:08:55<1:43:46, 15.19s/it] {'loss': 0.0007, 'grad_norm': 2.023786009399374, 'learning_rate': 2.546583850931677e-07, 'completion_length': 131.91964721679688, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.2798599749803543, 'kl': 0.0184326171875, 'epoch': 3.73} 75%|███████▍ | 1200/1610 [5:08:55<1:43:46, 15.19s/it] 75%|███████▍ | 1201/1610 [5:10:12<3:50:23, 33.80s/it] {'loss': 0.0007, 'grad_norm': 0.926288630430629, 'learning_rate': 2.540372670807454e-07, 'completion_length': 110.3839340209961, 'rewards/accuracy_reward': 0.6250000447034836, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.17885925620794296, 'kl': 0.0179443359375, 'epoch': 3.73} 75%|███████▍ | 1201/1610 [5:10:12<3:50:23, 33.80s/it] 75%|███████▍ | 1202/1610 [5:10:24<3:05:43, 27.31s/it] {'loss': 0.0006, 'grad_norm': 1.1268788418763456, 'learning_rate': 2.5341614906832296e-07, 'completion_length': 158.6428680419922, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.24498926848173141, 'kl': 0.016082763671875, 'epoch': 3.73} 75%|███████▍ | 1202/1610 [5:10:24<3:05:43, 27.31s/it] 75%|███████▍ | 1203/1610 [5:10:37<2:35:43, 22.96s/it] {'loss': 0.0011, 'grad_norm': 1.6061435337090775, 'learning_rate': 2.527950310559006e-07, 'completion_length': 182.8303680419922, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.28195706009864807, 'kl': 0.0269775390625, 'epoch': 3.74} 75%|███████▍ | 1203/1610 [5:10:37<2:35:43, 22.96s/it] 75%|███████▍ | 1204/1610 [5:10:48<2:10:01, 19.22s/it] {'loss': 0.0008, 'grad_norm': 1.3868646641549287, 'learning_rate': 2.521739130434782e-07, 'completion_length': 131.31250762939453, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.2831694483757019, 'kl': 0.019256591796875, 'epoch': 3.74} 75%|███████▍ | 1204/1610 [5:10:48<2:10:01, 19.22s/it] 75%|███████▍ | 1205/1610 [5:10:59<1:53:20, 16.79s/it] {'loss': 0.0007, 'grad_norm': 1.4780051507022156, 'learning_rate': 2.5155279503105585e-07, 'completion_length': 143.3214340209961, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3150164633989334, 'kl': 0.01837158203125, 'epoch': 3.74} 75%|███████▍ | 1205/1610 [5:10:59<1:53:20, 16.79s/it] 75%|███████▍ | 1206/1610 [5:11:10<1:41:40, 15.10s/it] {'loss': 0.0009, 'grad_norm': 1.8904540414876112, 'learning_rate': 2.5093167701863354e-07, 'completion_length': 148.4732208251953, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.23327527940273285, 'kl': 0.021484375, 'epoch': 3.75} 75%|███████▍ | 1206/1610 [5:11:10<1:41:40, 15.10s/it] 75%|███████▍ | 1207/1610 [5:11:23<1:37:58, 14.59s/it] {'loss': 0.0007, 'grad_norm': 2.9003398316067175, 'learning_rate': 2.5031055900621117e-07, 'completion_length': 148.80357360839844, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.4018840342760086, 'kl': 0.017822265625, 'epoch': 3.75} 75%|███████▍ | 1207/1610 [5:11:23<1:37:58, 14.59s/it] 75%|███████▌ | 1208/1610 [5:11:39<1:40:00, 14.93s/it] {'loss': 0.0009, 'grad_norm': 1.014602687895598, 'learning_rate': 2.496894409937888e-07, 'completion_length': 185.90179443359375, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928571939468384, 'reward_std': 0.3108424097299576, 'kl': 0.021728515625, 'epoch': 3.75} 75%|███████▌ | 1208/1610 [5:11:39<1:40:00, 14.93s/it] 75%|███████▌ | 1209/1610 [5:11:54<1:40:47, 15.08s/it] {'loss': 0.0007, 'grad_norm': 1.4129381806909882, 'learning_rate': 2.4906832298136644e-07, 'completion_length': 156.6964340209961, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.2702430784702301, 'kl': 0.01739501953125, 'epoch': 3.75} 75%|███████▌ | 1209/1610 [5:11:54<1:40:47, 15.08s/it] 75%|███████▌ | 1210/1610 [5:12:10<1:40:51, 15.13s/it] {'loss': 0.0006, 'grad_norm': 1.2529969228769848, 'learning_rate': 2.4844720496894407e-07, 'completion_length': 155.1339340209961, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.27474477887153625, 'kl': 0.01568603515625, 'epoch': 3.76} 75%|███████▌ | 1210/1610 [5:12:10<1:40:51, 15.13s/it] 75%|███████▌ | 1211/1610 [5:12:25<1:40:32, 15.12s/it] {'loss': 0.0006, 'grad_norm': 1.1290700497717587, 'learning_rate': 2.4782608695652176e-07, 'completion_length': 160.8928680419922, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.2566976547241211, 'kl': 0.014923095703125, 'epoch': 3.76} 75%|███████▌ | 1211/1610 [5:12:25<1:40:32, 15.12s/it] 75%|███████▌ | 1212/1610 [5:12:40<1:39:41, 15.03s/it] {'loss': 0.0006, 'grad_norm': 1.3448422665640056, 'learning_rate': 2.472049689440994e-07, 'completion_length': 140.27679443359375, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.2579156309366226, 'kl': 0.015106201171875, 'epoch': 3.76} 75%|███████▌ | 1212/1610 [5:12:40<1:39:41, 15.03s/it] 75%|███████▌ | 1213/1610 [5:12:55<1:40:30, 15.19s/it] {'loss': 0.0007, 'grad_norm': 1.0218094337523471, 'learning_rate': 2.46583850931677e-07, 'completion_length': 157.4107208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.2726622223854065, 'kl': 0.01727294921875, 'epoch': 3.77} 75%|███████▌ | 1213/1610 [5:12:55<1:40:30, 15.19s/it] 75%|███████▌ | 1214/1610 [5:13:12<1:43:47, 15.73s/it] {'loss': 0.0008, 'grad_norm': 2.0497902689899425, 'learning_rate': 2.4596273291925465e-07, 'completion_length': 157.56250762939453, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.27414587140083313, 'kl': 0.02032470703125, 'epoch': 3.77} 75%|███████▌ | 1214/1610 [5:13:12<1:43:47, 15.73s/it] 75%|███████▌ | 1215/1610 [5:13:27<1:42:09, 15.52s/it] {'loss': 0.0008, 'grad_norm': 1.590264217957843, 'learning_rate': 2.453416149068323e-07, 'completion_length': 159.5446548461914, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.24229323863983154, 'kl': 0.01873779296875, 'epoch': 3.77} 75%|███████▌ | 1215/1610 [5:13:27<1:42:09, 15.52s/it] 76%|███████▌ | 1216/1610 [5:13:43<1:43:19, 15.73s/it] {'loss': 0.0007, 'grad_norm': 1.1848237017543974, 'learning_rate': 2.447204968944099e-07, 'completion_length': 163.90178680419922, 'rewards/accuracy_reward': 0.4196428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.30659182369709015, 'kl': 0.016265869140625, 'epoch': 3.78} 76%|███████▌ | 1216/1610 [5:13:43<1:43:19, 15.73s/it] 76%|███████▌ | 1217/1610 [5:13:58<1:41:41, 15.52s/it] {'loss': 0.0008, 'grad_norm': 5.211008813406126, 'learning_rate': 2.4409937888198755e-07, 'completion_length': 153.50000762939453, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.312326043844223, 'kl': 0.02117919921875, 'epoch': 3.78} 76%|███████▌ | 1217/1610 [5:13:58<1:41:41, 15.52s/it] 76%|███████▌ | 1218/1610 [5:14:13<1:39:28, 15.23s/it] {'loss': 0.0006, 'grad_norm': 1.265914496959096, 'learning_rate': 2.4347826086956524e-07, 'completion_length': 148.06250762939453, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2858598530292511, 'kl': 0.01568603515625, 'epoch': 3.78} 76%|███████▌ | 1218/1610 [5:14:13<1:39:28, 15.23s/it] 76%|███████▌ | 1219/1610 [5:14:28<1:39:26, 15.26s/it] {'loss': 0.0006, 'grad_norm': 0.7508363745177372, 'learning_rate': 2.4285714285714287e-07, 'completion_length': 136.87500762939453, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285714626312256, 'reward_std': 0.16672416776418686, 'kl': 0.013763427734375, 'epoch': 3.79} 76%|███████▌ | 1219/1610 [5:14:28<1:39:26, 15.26s/it] 76%|███████▌ | 1220/1610 [5:14:44<1:40:57, 15.53s/it] {'loss': 0.0009, 'grad_norm': 2.5136863162193865, 'learning_rate': 2.422360248447205e-07, 'completion_length': 159.7321548461914, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.3749372363090515, 'kl': 0.02294921875, 'epoch': 3.79} 76%|███████▌ | 1220/1610 [5:14:44<1:40:57, 15.53s/it] 76%|███████▌ | 1221/1610 [5:14:59<1:38:15, 15.15s/it] {'loss': 0.0006, 'grad_norm': 1.1422811414477922, 'learning_rate': 2.4161490683229813e-07, 'completion_length': 150.8839340209961, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.23925502598285675, 'kl': 0.014892578125, 'epoch': 3.79} 76%|███████▌ | 1221/1610 [5:14:59<1:38:15, 15.15s/it] 76%|███████▌ | 1222/1610 [5:15:12<1:35:18, 14.74s/it] {'loss': 0.0007, 'grad_norm': 1.270365723773827, 'learning_rate': 2.4099378881987577e-07, 'completion_length': 136.24107360839844, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2987918108701706, 'kl': 0.017822265625, 'epoch': 3.8} 76%|███████▌ | 1222/1610 [5:15:12<1:35:18, 14.74s/it] 76%|███████▌ | 1223/1610 [5:15:29<1:38:39, 15.30s/it] {'loss': 0.0007, 'grad_norm': 0.9729439607862425, 'learning_rate': 2.403726708074534e-07, 'completion_length': 173.40179443359375, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6785715222358704, 'reward_std': 0.18458790704607964, 'kl': 0.0181884765625, 'epoch': 3.8} 76%|███████▌ | 1223/1610 [5:15:29<1:38:39, 15.30s/it] 76%|███████▌ | 1224/1610 [5:15:45<1:38:57, 15.38s/it] {'loss': 0.0006, 'grad_norm': 1.9368547939381964, 'learning_rate': 2.3975155279503103e-07, 'completion_length': 156.15179443359375, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.28767113387584686, 'kl': 0.015960693359375, 'epoch': 3.8} 76%|███████▌ | 1224/1610 [5:15:45<1:38:57, 15.38s/it] 76%|███████▌ | 1225/1610 [5:16:00<1:38:24, 15.34s/it] {'loss': 0.0009, 'grad_norm': 1.4850799085147353, 'learning_rate': 2.391304347826087e-07, 'completion_length': 147.66964721679688, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.25791002810001373, 'kl': 0.02252197265625, 'epoch': 3.8} 76%|███████▌ | 1225/1610 [5:16:00<1:38:24, 15.34s/it] 76%|███████▌ | 1226/1610 [5:16:14<1:35:55, 14.99s/it] {'loss': 0.0007, 'grad_norm': 1.1734151583156365, 'learning_rate': 2.385093167701863e-07, 'completion_length': 140.12500762939453, 'rewards/accuracy_reward': 0.767857164144516, 'rewards/format_reward': 1.0, 'reward': 1.7678571939468384, 'reward_std': 0.21400120854377747, 'kl': 0.016845703125, 'epoch': 3.81} 76%|███████▌ | 1226/1610 [5:16:14<1:35:55, 14.99s/it] 76%|███████▌ | 1227/1610 [5:16:30<1:37:00, 15.20s/it] {'loss': 0.0007, 'grad_norm': 0.88561419136059, 'learning_rate': 2.3788819875776398e-07, 'completion_length': 150.5357208251953, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.7321429252624512, 'reward_std': 0.23981501162052155, 'kl': 0.01763916015625, 'epoch': 3.81} 76%|███████▌ | 1227/1610 [5:16:30<1:37:00, 15.20s/it] 76%|███████▋ | 1228/1610 [5:16:45<1:36:46, 15.20s/it] {'loss': 0.0007, 'grad_norm': 1.1192013341640614, 'learning_rate': 2.3726708074534161e-07, 'completion_length': 161.70536422729492, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.18397442996501923, 'kl': 0.0179443359375, 'epoch': 3.81} 76%|███████▋ | 1228/1610 [5:16:45<1:36:46, 15.20s/it] 76%|███████▋ | 1229/1610 [5:17:00<1:36:21, 15.17s/it] {'loss': 0.0008, 'grad_norm': 2.6665575384215083, 'learning_rate': 2.3664596273291925e-07, 'completion_length': 143.4910774230957, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.3237687647342682, 'kl': 0.01971435546875, 'epoch': 3.82} 76%|███████▋ | 1229/1610 [5:17:00<1:36:21, 15.17s/it] 76%|███████▋ | 1230/1610 [5:17:14<1:34:13, 14.88s/it] {'loss': 0.0007, 'grad_norm': 1.3879546780449306, 'learning_rate': 2.3602484472049688e-07, 'completion_length': 130.39286041259766, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.23326967656612396, 'kl': 0.01690673828125, 'epoch': 3.82} 76%|███████▋ | 1230/1610 [5:17:14<1:34:13, 14.88s/it] 76%|███████▋ | 1231/1610 [5:17:29<1:33:59, 14.88s/it] {'loss': 0.0007, 'grad_norm': 1.0943807955178513, 'learning_rate': 2.354037267080745e-07, 'completion_length': 150.4732208251953, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.28707222640514374, 'kl': 0.017303466796875, 'epoch': 3.82} 76%|███████▋ | 1231/1610 [5:17:29<1:33:59, 14.88s/it] 77%|███████▋ | 1232/1610 [5:17:45<1:35:06, 15.10s/it] {'loss': 0.0007, 'grad_norm': 3.1724505193179438, 'learning_rate': 2.3478260869565217e-07, 'completion_length': 159.0714340209961, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.29097501933574677, 'kl': 0.0172119140625, 'epoch': 3.83} 77%|███████▋ | 1232/1610 [5:17:45<1:35:06, 15.10s/it] 77%|███████▋ | 1233/1610 [5:18:00<1:35:48, 15.25s/it] {'loss': 0.0008, 'grad_norm': 1.1477966284919485, 'learning_rate': 2.341614906832298e-07, 'completion_length': 167.85714721679688, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.2987862303853035, 'kl': 0.02105712890625, 'epoch': 3.83} 77%|███████▋ | 1233/1610 [5:18:00<1:35:48, 15.25s/it] 77%|███████▋ | 1234/1610 [5:18:16<1:36:03, 15.33s/it] {'loss': 0.0008, 'grad_norm': 2.9819313344461764, 'learning_rate': 2.3354037267080746e-07, 'completion_length': 188.39286041259766, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.3435651957988739, 'kl': 0.0208740234375, 'epoch': 3.83} 77%|███████▋ | 1234/1610 [5:18:16<1:36:03, 15.33s/it] 77%|███████▋ | 1235/1610 [5:18:31<1:36:03, 15.37s/it] {'loss': 0.0006, 'grad_norm': 0.9292241674411141, 'learning_rate': 2.3291925465838507e-07, 'completion_length': 165.5, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2248450219631195, 'kl': 0.01617431640625, 'epoch': 3.84} 77%|███████▋ | 1235/1610 [5:18:31<1:36:03, 15.37s/it] 77%|███████▋ | 1236/1610 [5:18:46<1:35:04, 15.25s/it] {'loss': 0.0007, 'grad_norm': 1.232714495414221, 'learning_rate': 2.3229813664596273e-07, 'completion_length': 168.2678680419922, 'rewards/accuracy_reward': 0.508928582072258, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.36160115897655487, 'kl': 0.017913818359375, 'epoch': 3.84} 77%|███████▋ | 1236/1610 [5:18:46<1:35:04, 15.25s/it] 77%|███████▋ | 1237/1610 [5:19:01<1:34:10, 15.15s/it] {'loss': 0.0008, 'grad_norm': 1.4801701720837206, 'learning_rate': 2.3167701863354036e-07, 'completion_length': 155.1071548461914, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3336770832538605, 'kl': 0.01947021484375, 'epoch': 3.84} 77%|███████▋ | 1237/1610 [5:19:01<1:34:10, 15.15s/it] 77%|███████▋ | 1238/1610 [5:19:15<1:31:45, 14.80s/it] {'loss': 0.0006, 'grad_norm': 3.11558103548536, 'learning_rate': 2.31055900621118e-07, 'completion_length': 130.35715103149414, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.20349960029125214, 'kl': 0.016082763671875, 'epoch': 3.84} 77%|███████▋ | 1238/1610 [5:19:15<1:31:45, 14.80s/it] 77%|███████▋ | 1239/1610 [5:19:29<1:30:15, 14.60s/it] {'loss': 0.0007, 'grad_norm': 1.9937311053698523, 'learning_rate': 2.3043478260869565e-07, 'completion_length': 156.81250762939453, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.3318512290716171, 'kl': 0.0169677734375, 'epoch': 3.85} 77%|███████▋ | 1239/1610 [5:19:29<1:30:15, 14.60s/it] 77%|███████▋ | 1240/1610 [5:19:44<1:30:27, 14.67s/it] {'loss': 0.0006, 'grad_norm': 0.9337287356945322, 'learning_rate': 2.2981366459627326e-07, 'completion_length': 157.31250762939453, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.143968403339386, 'kl': 0.015411376953125, 'epoch': 3.85} 77%|███████▋ | 1240/1610 [5:19:44<1:30:27, 14.67s/it] 77%|███████▋ | 1241/1610 [5:19:58<1:27:53, 14.29s/it] {'loss': 0.0008, 'grad_norm': 2.4755378362382565, 'learning_rate': 2.2919254658385092e-07, 'completion_length': 137.33036041259766, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.723214328289032, 'reward_std': 0.19690079241991043, 'kl': 0.02117919921875, 'epoch': 3.85} 77%|███████▋ | 1241/1610 [5:19:58<1:27:53, 14.29s/it] 77%|███████▋ | 1242/1610 [5:20:13<1:29:39, 14.62s/it] {'loss': 0.0008, 'grad_norm': 2.009468205790764, 'learning_rate': 2.2857142857142855e-07, 'completion_length': 149.89286041259766, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2819514572620392, 'kl': 0.01922607421875, 'epoch': 3.86} 77%|███████▋ | 1242/1610 [5:20:13<1:29:39, 14.62s/it] 77%|███████▋ | 1243/1610 [5:20:29<1:31:34, 14.97s/it] {'loss': 0.0007, 'grad_norm': 2.0664243445631256, 'learning_rate': 2.279503105590062e-07, 'completion_length': 168.51786041259766, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857909202576, 'reward_std': 0.2726622223854065, 'kl': 0.017913818359375, 'epoch': 3.86} 77%|███████▋ | 1243/1610 [5:20:29<1:31:34, 14.97s/it] 77%|███████▋ | 1244/1610 [5:20:44<1:32:04, 15.09s/it] {'loss': 0.0008, 'grad_norm': 2.1811610174021254, 'learning_rate': 2.2732919254658384e-07, 'completion_length': 155.0982208251953, 'rewards/accuracy_reward': 0.3571428805589676, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.3123260587453842, 'kl': 0.0194091796875, 'epoch': 3.86} 77%|███████▋ | 1244/1610 [5:20:44<1:32:04, 15.09s/it] 77%|███████▋ | 1245/1610 [5:20:59<1:32:06, 15.14s/it] {'loss': 0.0006, 'grad_norm': 0.9294754397750592, 'learning_rate': 2.267080745341615e-07, 'completion_length': 169.75000762939453, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.11394162476062775, 'kl': 0.01495361328125, 'epoch': 3.87} 77%|███████▋ | 1245/1610 [5:20:59<1:32:06, 15.14s/it] 77%|███████▋ | 1246/1610 [5:21:14<1:30:58, 15.00s/it] {'loss': 0.0007, 'grad_norm': 3.396109481946907, 'learning_rate': 2.260869565217391e-07, 'completion_length': 135.3303680419922, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.22754104435443878, 'kl': 0.01788330078125, 'epoch': 3.87} 77%|███████▋ | 1246/1610 [5:21:14<1:30:58, 15.00s/it] 77%|███████▋ | 1247/1610 [5:21:29<1:30:06, 14.89s/it] {'loss': 0.0007, 'grad_norm': 3.017182386923689, 'learning_rate': 2.2546583850931674e-07, 'completion_length': 152.0714340209961, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.24229325354099274, 'kl': 0.01739501953125, 'epoch': 3.87} 77%|███████▋ | 1247/1610 [5:21:29<1:30:06, 14.89s/it] 78%|███████▊ | 1248/1610 [5:21:43<1:29:14, 14.79s/it] {'loss': 0.0006, 'grad_norm': 1.0574922981677348, 'learning_rate': 2.248447204968944e-07, 'completion_length': 173.5446548461914, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.1956884115934372, 'kl': 0.014984130859375, 'epoch': 3.88} 78%|███████▊ | 1248/1610 [5:21:43<1:29:14, 14.79s/it] 78%|███████▊ | 1249/1610 [5:21:59<1:31:00, 15.13s/it] {'loss': 0.0008, 'grad_norm': 6.168995246890554, 'learning_rate': 2.2422360248447203e-07, 'completion_length': 161.6339340209961, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.30659742653369904, 'kl': 0.02081298828125, 'epoch': 3.88} 78%|███████▊ | 1249/1610 [5:21:59<1:31:00, 15.13s/it] 78%|███████▊ | 1250/1610 [5:22:15<1:31:47, 15.30s/it] {'loss': 0.0008, 'grad_norm': 1.0903811147294036, 'learning_rate': 2.236024844720497e-07, 'completion_length': 180.0446548461914, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.27535825967788696, 'kl': 0.0208740234375, 'epoch': 3.88} 78%|███████▊ | 1250/1610 [5:22:15<1:31:47, 15.30s/it] 78%|███████▊ | 1251/1610 [5:22:31<1:32:17, 15.42s/it] {'loss': 0.0008, 'grad_norm': 1.0154304478220453, 'learning_rate': 2.2298136645962732e-07, 'completion_length': 179.00000762939453, 'rewards/accuracy_reward': 0.4375, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.2645144462585449, 'kl': 0.02008056640625, 'epoch': 3.89} 78%|███████▊ | 1251/1610 [5:22:31<1:32:17, 15.42s/it] 78%|███████▊ | 1252/1610 [5:22:45<1:29:39, 15.03s/it] {'loss': 0.0006, 'grad_norm': 1.3083061998087264, 'learning_rate': 2.2236024844720495e-07, 'completion_length': 143.13393783569336, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2513168156147003, 'kl': 0.0155029296875, 'epoch': 3.89} 78%|███████▊ | 1252/1610 [5:22:45<1:29:39, 15.03s/it] 78%|███████▊ | 1253/1610 [5:23:01<1:31:27, 15.37s/it] {'loss': 0.0007, 'grad_norm': 1.1653457067365618, 'learning_rate': 2.217391304347826e-07, 'completion_length': 178.91964721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2858598530292511, 'kl': 0.01708984375, 'epoch': 3.89} 78%|███████▊ | 1253/1610 [5:23:01<1:31:27, 15.37s/it] 78%|███████▊ | 1254/1610 [5:23:16<1:30:49, 15.31s/it] {'loss': 0.0008, 'grad_norm': 3.440121053824639, 'learning_rate': 2.2111801242236025e-07, 'completion_length': 160.46428680419922, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.2215556800365448, 'kl': 0.01971435546875, 'epoch': 3.89} 78%|███████▊ | 1254/1610 [5:23:16<1:30:49, 15.31s/it] 78%|███████▊ | 1255/1610 [5:23:30<1:28:45, 15.00s/it] {'loss': 0.0007, 'grad_norm': 1.507767798313301, 'learning_rate': 2.2049689440993788e-07, 'completion_length': 144.21429061889648, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.27083637565374374, 'kl': 0.016876220703125, 'epoch': 3.9} 78%|███████▊ | 1255/1610 [5:23:30<1:28:45, 15.00s/it] 78%|███████▊ | 1256/1610 [5:23:45<1:27:47, 14.88s/it] {'loss': 0.0004, 'grad_norm': 1.3046813651924403, 'learning_rate': 2.198757763975155e-07, 'completion_length': 127.2410774230957, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.22936688363552094, 'kl': 0.01043701171875, 'epoch': 3.9} 78%|███████▊ | 1256/1610 [5:23:45<1:27:47, 14.88s/it] 78%|███████▊ | 1257/1610 [5:24:01<1:30:40, 15.41s/it] {'loss': 0.0006, 'grad_norm': 1.2979048555767916, 'learning_rate': 2.1925465838509317e-07, 'completion_length': 153.3214340209961, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5357143878936768, 'reward_std': 0.21703942120075226, 'kl': 0.014617919921875, 'epoch': 3.9} 78%|███████▊ | 1257/1610 [5:24:01<1:30:40, 15.41s/it] 78%|███████▊ | 1258/1610 [5:24:17<1:30:21, 15.40s/it] {'loss': 0.0006, 'grad_norm': 1.0928412577959545, 'learning_rate': 2.1863354037267078e-07, 'completion_length': 158.73214721679688, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.1995968148112297, 'kl': 0.014373779296875, 'epoch': 3.91} 78%|███████▊ | 1258/1610 [5:24:17<1:30:21, 15.40s/it] 78%|███████▊ | 1259/1610 [5:24:32<1:29:44, 15.34s/it] {'loss': 0.0008, 'grad_norm': 1.6147639094118555, 'learning_rate': 2.1801242236024844e-07, 'completion_length': 163.42858123779297, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.3628336787223816, 'kl': 0.0211181640625, 'epoch': 3.91} 78%|███████▊ | 1259/1610 [5:24:32<1:29:44, 15.34s/it] 78%|███████▊ | 1260/1610 [5:24:47<1:28:10, 15.12s/it] {'loss': 0.0007, 'grad_norm': 1.0384734827101536, 'learning_rate': 2.1739130434782607e-07, 'completion_length': 162.12500762939453, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553572535514832, 'reward_std': 0.27804867923259735, 'kl': 0.01824951171875, 'epoch': 3.91} 78%|███████▊ | 1260/1610 [5:24:47<1:28:10, 15.12s/it] 78%|███████▊ | 1261/1610 [5:25:00<1:25:33, 14.71s/it] {'loss': 0.0005, 'grad_norm': 1.8233659165919602, 'learning_rate': 2.1677018633540373e-07, 'completion_length': 141.11607360839844, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.4022262394428253, 'kl': 0.01361083984375, 'epoch': 3.92} 78%|███████▊ | 1261/1610 [5:25:00<1:25:33, 14.71s/it] 78%|███████▊ | 1262/1610 [5:25:15<1:25:01, 14.66s/it] {'loss': 0.0008, 'grad_norm': 1.4476306197548043, 'learning_rate': 2.1614906832298136e-07, 'completion_length': 146.49108123779297, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.3084232658147812, 'kl': 0.01995849609375, 'epoch': 3.92} 78%|███████▊ | 1262/1610 [5:25:15<1:25:01, 14.66s/it] 78%|███████▊ | 1263/1610 [5:25:31<1:26:49, 15.01s/it] {'loss': 0.001, 'grad_norm': 1.6310482577330818, 'learning_rate': 2.1552795031055902e-07, 'completion_length': 162.55358123779297, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3150164783000946, 'kl': 0.02496337890625, 'epoch': 3.92} 78%|███████▊ | 1263/1610 [5:25:31<1:26:49, 15.01s/it] 79%|███████▊ | 1264/1610 [5:25:46<1:26:29, 15.00s/it] {'loss': 0.0009, 'grad_norm': 0.9786400416751999, 'learning_rate': 2.1490683229813662e-07, 'completion_length': 155.7678680419922, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.1509094163775444, 'kl': 0.02215576171875, 'epoch': 3.93} 79%|███████▊ | 1264/1610 [5:25:46<1:26:29, 15.00s/it] 79%|███████▊ | 1265/1610 [5:26:02<1:27:43, 15.26s/it] {'loss': 0.0009, 'grad_norm': 1.09021035408538, 'learning_rate': 2.1428571428571426e-07, 'completion_length': 167.86608123779297, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.25521962344646454, 'kl': 0.02154541015625, 'epoch': 3.93} 79%|███████▊ | 1265/1610 [5:26:02<1:27:43, 15.26s/it] 79%|███████▊ | 1266/1610 [5:26:18<1:28:58, 15.52s/it] {'loss': 0.001, 'grad_norm': 1.3729983430970045, 'learning_rate': 2.1366459627329192e-07, 'completion_length': 167.80358123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2753526568412781, 'kl': 0.02557373046875, 'epoch': 3.93} 79%|███████▊ | 1266/1610 [5:26:18<1:28:58, 15.52s/it] 79%|███████▊ | 1267/1610 [5:26:32<1:26:23, 15.11s/it] {'loss': 0.0008, 'grad_norm': 1.3147752308374905, 'learning_rate': 2.1304347826086955e-07, 'completion_length': 159.06250762939453, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.28133800625801086, 'kl': 0.020599365234375, 'epoch': 3.93} 79%|███████▊ | 1267/1610 [5:26:32<1:26:23, 15.11s/it] 79%|███████▉ | 1268/1610 [5:26:47<1:26:28, 15.17s/it] {'loss': 0.0009, 'grad_norm': 1.8403893946484506, 'learning_rate': 2.124223602484472e-07, 'completion_length': 165.0446548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.22996579110622406, 'kl': 0.0234375, 'epoch': 3.94} 79%|███████▉ | 1268/1610 [5:26:47<1:26:28, 15.17s/it] 79%|███████▉ | 1269/1610 [5:27:01<1:24:23, 14.85s/it] {'loss': 0.0007, 'grad_norm': 1.34315588167978, 'learning_rate': 2.1180124223602484e-07, 'completion_length': 147.1339340209961, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857313156128, 'reward_std': 0.325252428650856, 'kl': 0.017913818359375, 'epoch': 3.94} 79%|███████▉ | 1269/1610 [5:27:01<1:24:23, 14.85s/it] 79%|███████▉ | 1270/1610 [5:27:16<1:23:45, 14.78s/it] {'loss': 0.0009, 'grad_norm': 1.2760767940067532, 'learning_rate': 2.1118012422360247e-07, 'completion_length': 149.56250762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.23535223305225372, 'kl': 0.02130126953125, 'epoch': 3.94} 79%|███████▉ | 1270/1610 [5:27:16<1:23:45, 14.78s/it] 79%|███████▉ | 1271/1610 [5:27:32<1:25:49, 15.19s/it] {'loss': 0.0007, 'grad_norm': 1.733184147078869, 'learning_rate': 2.105590062111801e-07, 'completion_length': 184.24108123779297, 'rewards/accuracy_reward': 0.6875000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6875000596046448, 'reward_std': 0.24889206886291504, 'kl': 0.018646240234375, 'epoch': 3.95} 79%|███████▉ | 1271/1610 [5:27:32<1:25:49, 15.19s/it] 79%|███████▉ | 1272/1610 [5:27:47<1:25:49, 15.24s/it] {'loss': 0.0007, 'grad_norm': 1.462660029278602, 'learning_rate': 2.0993788819875776e-07, 'completion_length': 149.89286041259766, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.2868573069572449, 'kl': 0.01751708984375, 'epoch': 3.95} 79%|███████▉ | 1272/1610 [5:27:47<1:25:49, 15.24s/it] 79%|███████▉ | 1273/1610 [5:28:02<1:24:51, 15.11s/it] {'loss': 0.0008, 'grad_norm': 1.3471252812235204, 'learning_rate': 2.093167701863354e-07, 'completion_length': 163.4553680419922, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.244989275932312, 'kl': 0.018798828125, 'epoch': 3.95} 79%|███████▉ | 1273/1610 [5:28:02<1:24:51, 15.11s/it] 79%|███████▉ | 1274/1610 [5:28:18<1:26:00, 15.36s/it] {'loss': 0.0008, 'grad_norm': 0.992963940193121, 'learning_rate': 2.0869565217391303e-07, 'completion_length': 199.2232208251953, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.23326966166496277, 'kl': 0.02117919921875, 'epoch': 3.96} 79%|███████▉ | 1274/1610 [5:28:18<1:26:00, 15.36s/it] 79%|███████▉ | 1275/1610 [5:28:33<1:25:25, 15.30s/it] {'loss': 0.0008, 'grad_norm': 2.1074941634444424, 'learning_rate': 2.080745341614907e-07, 'completion_length': 164.40179443359375, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.27535825967788696, 'kl': 0.0191650390625, 'epoch': 3.96} 79%|███████▉ | 1275/1610 [5:28:33<1:25:25, 15.30s/it] 79%|███████▉ | 1276/1610 [5:28:48<1:24:21, 15.15s/it] {'loss': 0.0008, 'grad_norm': 3.2497356418741807, 'learning_rate': 2.074534161490683e-07, 'completion_length': 151.25000762939453, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.26572123169898987, 'kl': 0.019287109375, 'epoch': 3.96} 79%|███████▉ | 1276/1610 [5:28:48<1:24:21, 15.15s/it] 79%|███████▉ | 1277/1610 [5:29:02<1:21:41, 14.72s/it] {'loss': 0.0006, 'grad_norm': 2.0571558551807105, 'learning_rate': 2.0683229813664595e-07, 'completion_length': 141.68750762939453, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.23266181349754333, 'kl': 0.01507568359375, 'epoch': 3.97} 79%|███████▉ | 1277/1610 [5:29:02<1:21:41, 14.72s/it] 79%|███████▉ | 1278/1610 [5:29:16<1:20:53, 14.62s/it] {'loss': 0.0008, 'grad_norm': 1.0146896956388423, 'learning_rate': 2.0621118012422359e-07, 'completion_length': 156.70536041259766, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857313156128, 'reward_std': 0.34208156168460846, 'kl': 0.01922607421875, 'epoch': 3.97} 79%|███████▉ | 1278/1610 [5:29:16<1:20:53, 14.62s/it] 79%|███████▉ | 1279/1610 [5:29:30<1:19:50, 14.47s/it] {'loss': 0.0008, 'grad_norm': 1.1109209581515014, 'learning_rate': 2.0559006211180125e-07, 'completion_length': 148.9464340209961, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.30963005125522614, 'kl': 0.01934814453125, 'epoch': 3.97} 79%|███████▉ | 1279/1610 [5:29:30<1:19:50, 14.47s/it] 80%|███████▉ | 1280/1610 [5:29:45<1:19:38, 14.48s/it] {'loss': 0.0008, 'grad_norm': 3.8530861224281265, 'learning_rate': 2.0496894409937888e-07, 'completion_length': 155.74107360839844, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.2344820573925972, 'kl': 0.01922607421875, 'epoch': 3.98} 80%|███████▉ | 1280/1610 [5:29:45<1:19:38, 14.48s/it] 80%|███████▉ | 1281/1610 [5:30:00<1:20:16, 14.64s/it] {'loss': 0.0009, 'grad_norm': 2.0388854252975004, 'learning_rate': 2.0434782608695654e-07, 'completion_length': 147.37500762939453, 'rewards/accuracy_reward': 0.366071455180645, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.22363264858722687, 'kl': 0.0228271484375, 'epoch': 3.98} 80%|███████▉ | 1281/1610 [5:30:00<1:20:16, 14.64s/it] 80%|███████▉ | 1282/1610 [5:30:15<1:20:52, 14.79s/it] {'loss': 0.0007, 'grad_norm': 1.1871870119150805, 'learning_rate': 2.0372670807453414e-07, 'completion_length': 155.3571548461914, 'rewards/accuracy_reward': 0.5625000447034836, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.22605738043785095, 'kl': 0.017822265625, 'epoch': 3.98} 80%|███████▉ | 1282/1610 [5:30:15<1:20:52, 14.79s/it] 80%|███████▉ | 1283/1610 [5:30:30<1:21:14, 14.91s/it] {'loss': 0.0007, 'grad_norm': 1.1045328586541785, 'learning_rate': 2.0310559006211178e-07, 'completion_length': 161.39286041259766, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.1956884115934372, 'kl': 0.0172119140625, 'epoch': 3.98} 80%|███████▉ | 1283/1610 [5:30:30<1:21:14, 14.91s/it] 80%|███████▉ | 1284/1610 [5:30:45<1:20:58, 14.90s/it] {'loss': 0.0008, 'grad_norm': 2.7882394873456797, 'learning_rate': 2.0248447204968943e-07, 'completion_length': 160.1339340209961, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.3240400403738022, 'kl': 0.01934814453125, 'epoch': 3.99} 80%|███████▉ | 1284/1610 [5:30:45<1:20:58, 14.90s/it] 80%|███████▉ | 1285/1610 [5:31:01<1:22:24, 15.21s/it] {'loss': 0.0008, 'grad_norm': 1.3547581524596681, 'learning_rate': 2.0186335403726707e-07, 'completion_length': 143.4464340209961, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.3174468129873276, 'kl': 0.020263671875, 'epoch': 3.99} 80%|███████▉ | 1285/1610 [5:31:01<1:22:24, 15.21s/it] 80%|███████▉ | 1286/1610 [5:31:16<1:22:03, 15.20s/it] {'loss': 0.0007, 'grad_norm': 1.447480305726055, 'learning_rate': 2.0124223602484473e-07, 'completion_length': 168.87500762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.20801585912704468, 'kl': 0.01678466796875, 'epoch': 3.99} 80%|███████▉ | 1286/1610 [5:31:16<1:22:03, 15.20s/it] 80%|███████▉ | 1287/1610 [5:31:32<1:22:07, 15.26s/it] {'loss': 0.0007, 'grad_norm': 1.7706397188030591, 'learning_rate': 2.0062111801242236e-07, 'completion_length': 153.02679443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5803571939468384, 'reward_std': 0.3661229908466339, 'kl': 0.018310546875, 'epoch': 4.0} 80%|███████▉ | 1287/1610 [5:31:32<1:22:07, 15.26s/it] 80%|████████ | 1288/1610 [5:31:46<1:20:19, 14.97s/it] {'loss': 0.0007, 'grad_norm': 0.7033074567785159, 'learning_rate': 2e-07, 'completion_length': 132.98214721679688, 'rewards/accuracy_reward': 0.7500000596046448, 'rewards/format_reward': 1.0, 'reward': 1.7500001192092896, 'reward_std': 0.150909423828125, 'kl': 0.017364501953125, 'epoch': 4.0} 80%|████████ | 1288/1610 [5:31:46<1:20:19, 14.97s/it] 80%|████████ | 1289/1610 [5:32:01<1:20:09, 14.98s/it] {'loss': 0.0007, 'grad_norm': 1.0742902454880738, 'learning_rate': 1.9937888198757762e-07, 'completion_length': 161.85714721679688, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25670325756073, 'kl': 0.01788330078125, 'epoch': 4.0} 80%|████████ | 1289/1610 [5:32:01<1:20:09, 14.98s/it] 80%|████████ | 1290/1610 [5:32:16<1:19:37, 14.93s/it] {'loss': 0.0007, 'grad_norm': 1.1488831261549888, 'learning_rate': 1.9875776397515526e-07, 'completion_length': 137.89286041259766, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.28195707499980927, 'kl': 0.016815185546875, 'epoch': 4.01} 80%|████████ | 1290/1610 [5:32:16<1:19:37, 14.93s/it] 80%|████████ | 1291/1610 [5:32:31<1:19:23, 14.93s/it] {'loss': 0.0008, 'grad_norm': 1.2828580744839682, 'learning_rate': 1.9813664596273292e-07, 'completion_length': 160.49107360839844, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.25791002810001373, 'kl': 0.018768310546875, 'epoch': 4.01} 80%|████████ | 1291/1610 [5:32:31<1:19:23, 14.93s/it] 80%|████████ | 1292/1610 [5:32:45<1:18:49, 14.87s/it] {'loss': 0.0007, 'grad_norm': 1.5487386394040812, 'learning_rate': 1.9751552795031055e-07, 'completion_length': 150.83928680419922, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.32646478712558746, 'kl': 0.0172119140625, 'epoch': 4.01} 80%|████████ | 1292/1610 [5:32:45<1:18:49, 14.87s/it] 80%|████████ | 1293/1610 [5:33:01<1:19:47, 15.10s/it] {'loss': 0.0008, 'grad_norm': 1.304114414635018, 'learning_rate': 1.968944099378882e-07, 'completion_length': 170.61607360839844, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.28768008947372437, 'kl': 0.01947021484375, 'epoch': 4.02} 80%|████████ | 1293/1610 [5:33:01<1:19:47, 15.10s/it] 80%|████████ | 1294/1610 [5:33:17<1:20:21, 15.26s/it] {'loss': 0.0006, 'grad_norm': 1.666909043957914, 'learning_rate': 1.962732919254658e-07, 'completion_length': 134.04464721679688, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.3958396017551422, 'kl': 0.015960693359375, 'epoch': 4.02} 80%|████████ | 1294/1610 [5:33:17<1:20:21, 15.26s/it] 80%|████████ | 1295/1610 [5:33:31<1:19:08, 15.07s/it] {'loss': 0.0006, 'grad_norm': 1.77965143878238, 'learning_rate': 1.9565217391304347e-07, 'completion_length': 151.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.26181842386722565, 'kl': 0.0159912109375, 'epoch': 4.02} 80%|████████ | 1295/1610 [5:33:31<1:19:08, 15.07s/it] 80%|████████ | 1296/1610 [5:33:47<1:20:12, 15.33s/it] {'loss': 0.0009, 'grad_norm': 0.952149431880141, 'learning_rate': 1.950310559006211e-07, 'completion_length': 176.4464340209961, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.2248506247997284, 'kl': 0.023681640625, 'epoch': 4.02} 80%|████████ | 1296/1610 [5:33:47<1:20:12, 15.33s/it] 81%|████████ | 1297/1610 [5:34:03<1:20:38, 15.46s/it] {'loss': 0.0006, 'grad_norm': 1.1544295149312611, 'learning_rate': 1.9440993788819876e-07, 'completion_length': 179.0982208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2410808801651001, 'kl': 0.01519775390625, 'epoch': 4.03} 81%|████████ | 1297/1610 [5:34:03<1:20:38, 15.46s/it] 81%|████████ | 1298/1610 [5:34:17<1:17:29, 14.90s/it] {'loss': 0.0008, 'grad_norm': 2.5129560039649337, 'learning_rate': 1.937888198757764e-07, 'completion_length': 150.52679443359375, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.21582705527544022, 'kl': 0.019683837890625, 'epoch': 4.03} 81%|████████ | 1298/1610 [5:34:17<1:17:29, 14.90s/it] 81%|████████ | 1299/1610 [5:34:31<1:16:53, 14.83s/it] {'loss': 0.0006, 'grad_norm': 2.1135896285075737, 'learning_rate': 1.9316770186335403e-07, 'completion_length': 162.91964721679688, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.25280046463012695, 'kl': 0.014801025390625, 'epoch': 4.03} 81%|████████ | 1299/1610 [5:34:31<1:16:53, 14.83s/it] 81%|████████ | 1300/1610 [5:34:46<1:16:20, 14.78s/it] {'loss': 0.0007, 'grad_norm': 0.8334886398775776, 'learning_rate': 1.9254658385093166e-07, 'completion_length': 155.71429443359375, 'rewards/accuracy_reward': 0.6785714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1995968073606491, 'kl': 0.0177001953125, 'epoch': 4.04} 81%|████████ | 1300/1610 [5:34:46<1:16:20, 14.78s/it] 81%|████████ | 1301/1610 [5:36:06<2:56:53, 34.35s/it] {'loss': 0.0007, 'grad_norm': 1.256316511944648, 'learning_rate': 1.919254658385093e-07, 'completion_length': 128.04464721679688, 'rewards/accuracy_reward': 0.8392857313156128, 'rewards/format_reward': 1.0, 'reward': 1.8392857313156128, 'reward_std': 0.17824579775333405, 'kl': 0.017364501953125, 'epoch': 4.04} 81%|████████ | 1301/1610 [5:36:06<2:56:53, 34.35s/it] 81%|████████ | 1302/1610 [5:36:21<2:27:08, 28.66s/it] {'loss': 0.0008, 'grad_norm': 1.5073336818894132, 'learning_rate': 1.9130434782608695e-07, 'completion_length': 145.33036041259766, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5535714626312256, 'reward_std': 0.3245943933725357, 'kl': 0.0211181640625, 'epoch': 4.04} 81%|████████ | 1302/1610 [5:36:21<2:27:08, 28.66s/it] 81%|████████ | 1303/1610 [5:36:37<2:06:42, 24.76s/it] {'loss': 0.0008, 'grad_norm': 1.4755467902637214, 'learning_rate': 1.9068322981366459e-07, 'completion_length': 163.92857360839844, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.27206334471702576, 'kl': 0.01898193359375, 'epoch': 4.05} 81%|████████ | 1303/1610 [5:36:37<2:06:42, 24.76s/it] 81%|████████ | 1304/1610 [5:36:51<1:49:52, 21.54s/it] {'loss': 0.0006, 'grad_norm': 1.4074441521105543, 'learning_rate': 1.9006211180124224e-07, 'completion_length': 129.57143783569336, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.1671452671289444, 'kl': 0.015716552734375, 'epoch': 4.05} 81%|████████ | 1304/1610 [5:36:51<1:49:52, 21.54s/it] 81%|████████ | 1305/1610 [5:37:09<1:44:26, 20.54s/it] {'loss': 0.0009, 'grad_norm': 1.5517231736233101, 'learning_rate': 1.8944099378881988e-07, 'completion_length': 197.21429443359375, 'rewards/accuracy_reward': 0.3214285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3125000596046448, 'reward_std': 0.3661229908466339, 'kl': 0.02374267578125, 'epoch': 4.05} 81%|████████ | 1305/1610 [5:37:09<1:44:26, 20.54s/it] 81%|████████ | 1306/1610 [5:37:24<1:35:14, 18.80s/it] {'loss': 0.0007, 'grad_norm': 1.702887961052559, 'learning_rate': 1.888198757763975e-07, 'completion_length': 146.61608123779297, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.167145274579525, 'kl': 0.01873779296875, 'epoch': 4.06} 81%|████████ | 1306/1610 [5:37:24<1:35:14, 18.80s/it] 81%|████████ | 1307/1610 [5:37:39<1:28:56, 17.61s/it] {'loss': 0.001, 'grad_norm': 1.1083972091594856, 'learning_rate': 1.8819875776397514e-07, 'completion_length': 163.5982208251953, 'rewards/accuracy_reward': 0.4821428656578064, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2603347897529602, 'kl': 0.0250244140625, 'epoch': 4.06} 81%|████████ | 1307/1610 [5:37:39<1:28:56, 17.61s/it] 81%|████████ | 1308/1610 [5:37:52<1:21:15, 16.14s/it] {'loss': 0.0006, 'grad_norm': 0.988311695526743, 'learning_rate': 1.8757763975155277e-07, 'completion_length': 123.96429443359375, 'rewards/accuracy_reward': 0.7857142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7857143878936768, 'reward_std': 0.17313061654567719, 'kl': 0.014068603515625, 'epoch': 4.06} 81%|████████ | 1308/1610 [5:37:52<1:21:15, 16.14s/it] 81%|████████▏ | 1309/1610 [5:38:06<1:18:35, 15.67s/it] {'loss': 0.0006, 'grad_norm': 1.3093363705930678, 'learning_rate': 1.8695652173913043e-07, 'completion_length': 140.2857208251953, 'rewards/accuracy_reward': 0.6696428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.20349960774183273, 'kl': 0.013946533203125, 'epoch': 4.07} 81%|████████▏ | 1309/1610 [5:38:06<1:18:35, 15.67s/it] 81%|████████▏ | 1310/1610 [5:38:21<1:17:41, 15.54s/it] {'loss': 0.0007, 'grad_norm': 1.0927073188072915, 'learning_rate': 1.8633540372670807e-07, 'completion_length': 154.77679443359375, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.21973545849323273, 'kl': 0.01824951171875, 'epoch': 4.07} 81%|████████▏ | 1310/1610 [5:38:21<1:17:41, 15.54s/it] 81%|████████▏ | 1311/1610 [5:38:39<1:20:17, 16.11s/it] {'loss': 0.0008, 'grad_norm': 1.1304924199298458, 'learning_rate': 1.8571428571428572e-07, 'completion_length': 181.4196548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.571428656578064, 'reward_std': 0.30659741163253784, 'kl': 0.0201416015625, 'epoch': 4.07} 81%|████████▏ | 1311/1610 [5:38:39<1:20:17, 16.11s/it] 81%|████████▏ | 1312/1610 [5:38:52<1:16:03, 15.31s/it] {'loss': 0.0007, 'grad_norm': 1.1496289533438353, 'learning_rate': 1.8509316770186333e-07, 'completion_length': 146.39286041259766, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.2786475718021393, 'kl': 0.01763916015625, 'epoch': 4.07} 81%|████████▏ | 1312/1610 [5:38:52<1:16:03, 15.31s/it] 82%|████████▏ | 1313/1610 [5:39:09<1:17:41, 15.70s/it] {'loss': 0.0009, 'grad_norm': 0.9938700134214618, 'learning_rate': 1.84472049689441e-07, 'completion_length': 184.25000762939453, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.446428656578064, 'reward_std': 0.262010782957077, 'kl': 0.02313232421875, 'epoch': 4.08} 82%|████████▏ | 1313/1610 [5:39:09<1:17:41, 15.70s/it] 82%|████████▏ | 1314/1610 [5:39:24<1:16:02, 15.41s/it] {'loss': 0.001, 'grad_norm': 1.2015201081709117, 'learning_rate': 1.8385093167701862e-07, 'completion_length': 170.85714721679688, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.3219574838876724, 'kl': 0.02581787109375, 'epoch': 4.08} 82%|████████▏ | 1314/1610 [5:39:24<1:16:02, 15.41s/it] 82%|████████▏ | 1315/1610 [5:39:38<1:14:51, 15.23s/it] {'loss': 0.0008, 'grad_norm': 1.4537720285743878, 'learning_rate': 1.8322981366459628e-07, 'completion_length': 163.13394165039062, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.2540072351694107, 'kl': 0.01983642578125, 'epoch': 4.08} 82%|████████▏ | 1315/1610 [5:39:38<1:14:51, 15.23s/it] 82%|████████▏ | 1316/1610 [5:39:54<1:15:39, 15.44s/it] {'loss': 0.0008, 'grad_norm': 1.2515362806961756, 'learning_rate': 1.8260869565217391e-07, 'completion_length': 173.9553680419922, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.27535825967788696, 'kl': 0.02105712890625, 'epoch': 4.09} 82%|████████▏ | 1316/1610 [5:39:54<1:15:39, 15.44s/it] 82%|████████▏ | 1317/1610 [5:40:11<1:17:42, 15.91s/it] {'loss': 0.0007, 'grad_norm': 1.3126871025367974, 'learning_rate': 1.8198757763975152e-07, 'completion_length': 145.42857360839844, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3571429252624512, 'reward_std': 0.33056554198265076, 'kl': 0.0179443359375, 'epoch': 4.09} 82%|████████▏ | 1317/1610 [5:40:11<1:17:42, 15.91s/it] 82%|████████▏ | 1318/1610 [5:40:26<1:15:28, 15.51s/it] {'loss': 0.0007, 'grad_norm': 1.745866463000115, 'learning_rate': 1.8136645962732918e-07, 'completion_length': 154.8839340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.39918240904808044, 'kl': 0.0179443359375, 'epoch': 4.09} 82%|████████▏ | 1318/1610 [5:40:26<1:15:28, 15.51s/it] 82%|████████▏ | 1319/1610 [5:40:41<1:15:11, 15.50s/it] {'loss': 0.0006, 'grad_norm': 1.3335461358586753, 'learning_rate': 1.807453416149068e-07, 'completion_length': 166.07144165039062, 'rewards/accuracy_reward': 0.7321428656578064, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.2948834300041199, 'kl': 0.015899658203125, 'epoch': 4.1} 82%|████████▏ | 1319/1610 [5:40:41<1:15:11, 15.50s/it] 82%|████████▏ | 1320/1610 [5:40:55<1:12:36, 15.02s/it] {'loss': 0.0006, 'grad_norm': 1.1727985775787153, 'learning_rate': 1.8012422360248447e-07, 'completion_length': 143.05358123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.33575403690338135, 'kl': 0.0155029296875, 'epoch': 4.1} 82%|████████▏ | 1320/1610 [5:40:55<1:12:36, 15.02s/it] 82%|████████▏ | 1321/1610 [5:41:11<1:13:21, 15.23s/it] {'loss': 0.0007, 'grad_norm': 0.9761271055286971, 'learning_rate': 1.795031055900621e-07, 'completion_length': 162.8839340209961, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5178572535514832, 'reward_std': 0.27543583512306213, 'kl': 0.01824951171875, 'epoch': 4.1} 82%|████████▏ | 1321/1610 [5:41:11<1:13:21, 15.23s/it] 82%|████████▏ | 1322/1610 [5:41:27<1:13:35, 15.33s/it] {'loss': 0.0008, 'grad_norm': 1.1199707906079885, 'learning_rate': 1.7888198757763976e-07, 'completion_length': 170.74107360839844, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.29097504913806915, 'kl': 0.0201416015625, 'epoch': 4.11} 82%|████████▏ | 1322/1610 [5:41:27<1:13:35, 15.33s/it] 82%|████████▏ | 1323/1610 [5:41:40<1:11:07, 14.87s/it] {'loss': 0.0006, 'grad_norm': 2.0101733209171937, 'learning_rate': 1.7826086956521737e-07, 'completion_length': 130.75000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2852463722229004, 'kl': 0.015777587890625, 'epoch': 4.11} 82%|████████▏ | 1323/1610 [5:41:40<1:11:07, 14.87s/it] 82%|████████▏ | 1324/1610 [5:41:57<1:13:14, 15.36s/it] {'loss': 0.0009, 'grad_norm': 1.0159680439616983, 'learning_rate': 1.7763975155279503e-07, 'completion_length': 182.3214340209961, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4642857313156128, 'reward_std': 0.2501044422388077, 'kl': 0.02337646484375, 'epoch': 4.11} 82%|████████▏ | 1324/1610 [5:41:57<1:13:14, 15.36s/it] 82%|████████▏ | 1325/1610 [5:42:12<1:13:05, 15.39s/it] {'loss': 0.0008, 'grad_norm': 5.749435561972975, 'learning_rate': 1.7701863354037266e-07, 'completion_length': 149.23214721679688, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.3240400403738022, 'kl': 0.02020263671875, 'epoch': 4.11} 82%|████████▏ | 1325/1610 [5:42:12<1:13:05, 15.39s/it] 82%|████████▏ | 1326/1610 [5:42:28<1:13:10, 15.46s/it] {'loss': 0.0007, 'grad_norm': 2.3090069587296314, 'learning_rate': 1.763975155279503e-07, 'completion_length': 161.18750762939453, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.3201316297054291, 'kl': 0.0179443359375, 'epoch': 4.12} 82%|████████▏ | 1326/1610 [5:42:28<1:13:10, 15.46s/it] 82%|████████▏ | 1327/1610 [5:42:43<1:12:36, 15.39s/it] {'loss': 0.0008, 'grad_norm': 1.069752221596774, 'learning_rate': 1.7577639751552795e-07, 'completion_length': 158.0178680419922, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.2597213163971901, 'kl': 0.01995849609375, 'epoch': 4.12} 82%|████████▏ | 1327/1610 [5:42:43<1:12:36, 15.39s/it] 82%|████████▏ | 1328/1610 [5:42:58<1:11:56, 15.31s/it] {'loss': 0.0007, 'grad_norm': 0.9571208615689357, 'learning_rate': 1.7515527950310558e-07, 'completion_length': 168.05358123779297, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.23535223305225372, 'kl': 0.01806640625, 'epoch': 4.12} 82%|████████▏ | 1328/1610 [5:42:58<1:11:56, 15.31s/it] 83%|████████▎ | 1329/1610 [5:43:13<1:10:12, 14.99s/it] {'loss': 0.0006, 'grad_norm': 2.6770663037739255, 'learning_rate': 1.7453416149068322e-07, 'completion_length': 133.54464721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2735324054956436, 'kl': 0.015625, 'epoch': 4.13} 83%|████████▎ | 1329/1610 [5:43:13<1:10:12, 14.99s/it] 83%|████████▎ | 1330/1610 [5:43:30<1:13:08, 15.67s/it] {'loss': 0.001, 'grad_norm': 2.2156114312539077, 'learning_rate': 1.7391304347826085e-07, 'completion_length': 190.4107208251953, 'rewards/accuracy_reward': 0.6517857611179352, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6428571939468384, 'reward_std': 0.35496899485588074, 'kl': 0.0244140625, 'epoch': 4.13} 83%|████████▎ | 1330/1610 [5:43:30<1:13:08, 15.67s/it] 83%|████████▎ | 1331/1610 [5:43:47<1:14:39, 16.05s/it] {'loss': 0.0009, 'grad_norm': 0.9625096717828113, 'learning_rate': 1.732919254658385e-07, 'completion_length': 199.63394165039062, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 0.9821429252624512, 'reward': 1.5803571939468384, 'reward_std': 0.3036864697933197, 'kl': 0.02288818359375, 'epoch': 4.13} 83%|████████▎ | 1331/1610 [5:43:47<1:14:39, 16.05s/it] 83%|████████▎ | 1332/1610 [5:44:03<1:14:31, 16.08s/it] {'loss': 0.0007, 'grad_norm': 2.1498162405553005, 'learning_rate': 1.7267080745341614e-07, 'completion_length': 172.2678680419922, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.24167977273464203, 'kl': 0.0181884765625, 'epoch': 4.14} 83%|████████▎ | 1332/1610 [5:44:03<1:14:31, 16.08s/it] 83%|████████▎ | 1333/1610 [5:44:18<1:12:29, 15.70s/it] {'loss': 0.0008, 'grad_norm': 1.7136042896402448, 'learning_rate': 1.720496894409938e-07, 'completion_length': 147.35714721679688, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.2314494401216507, 'kl': 0.01953125, 'epoch': 4.14} 83%|████████▎ | 1333/1610 [5:44:18<1:12:29, 15.70s/it] 83%|████████▎ | 1334/1610 [5:44:33<1:11:38, 15.57s/it] {'loss': 0.0007, 'grad_norm': 1.191266979790396, 'learning_rate': 1.7142857142857143e-07, 'completion_length': 172.2589340209961, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.3129395395517349, 'kl': 0.01678466796875, 'epoch': 4.14} 83%|████████▎ | 1334/1610 [5:44:33<1:11:38, 15.57s/it] 83%|████████▎ | 1335/1610 [5:44:46<1:08:19, 14.91s/it] {'loss': 0.0008, 'grad_norm': 1.3467680806116522, 'learning_rate': 1.7080745341614904e-07, 'completion_length': 146.52678680419922, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.35770396888256073, 'kl': 0.019073486328125, 'epoch': 4.15} 83%|████████▎ | 1335/1610 [5:44:46<1:08:19, 14.91s/it] 83%|████████▎ | 1336/1610 [5:45:01<1:07:18, 14.74s/it] {'loss': 0.0008, 'grad_norm': 1.3115861109973939, 'learning_rate': 1.701863354037267e-07, 'completion_length': 142.5714340209961, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.23265622556209564, 'kl': 0.019287109375, 'epoch': 4.15} 83%|████████▎ | 1336/1610 [5:45:01<1:07:18, 14.74s/it] 83%|████████▎ | 1337/1610 [5:45:17<1:09:34, 15.29s/it] {'loss': 0.001, 'grad_norm': 1.1261429872523079, 'learning_rate': 1.6956521739130433e-07, 'completion_length': 196.81250762939453, 'rewards/accuracy_reward': 0.446428582072258, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.3240400403738022, 'kl': 0.02520751953125, 'epoch': 4.15} 83%|████████▎ | 1337/1610 [5:45:17<1:09:34, 15.29s/it] 83%|████████▎ | 1338/1610 [5:45:32<1:08:22, 15.08s/it] {'loss': 0.0008, 'grad_norm': 1.6873256873273947, 'learning_rate': 1.68944099378882e-07, 'completion_length': 164.6339340209961, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.28828462958335876, 'kl': 0.019195556640625, 'epoch': 4.16} 83%|████████▎ | 1338/1610 [5:45:32<1:08:22, 15.08s/it] 83%|████████▎ | 1339/1610 [5:45:47<1:08:29, 15.16s/it] {'loss': 0.0009, 'grad_norm': 0.9531317780950104, 'learning_rate': 1.6832298136645962e-07, 'completion_length': 173.5357208251953, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.22754664719104767, 'kl': 0.02337646484375, 'epoch': 4.16} 83%|████████▎ | 1339/1610 [5:45:47<1:08:29, 15.16s/it] 83%|████████▎ | 1340/1610 [5:46:02<1:07:11, 14.93s/it] {'loss': 0.0008, 'grad_norm': 1.219357088732346, 'learning_rate': 1.6770186335403728e-07, 'completion_length': 137.43750762939453, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.22875341773033142, 'kl': 0.02001953125, 'epoch': 4.16} 83%|████████▎ | 1340/1610 [5:46:02<1:07:11, 14.93s/it] 83%|████████▎ | 1341/1610 [5:46:18<1:08:29, 15.28s/it] {'loss': 0.0008, 'grad_norm': 1.1818727683188766, 'learning_rate': 1.6708074534161489e-07, 'completion_length': 168.48214721679688, 'rewards/accuracy_reward': 0.5446428805589676, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5357143878936768, 'reward_std': 0.29751475155353546, 'kl': 0.01995849609375, 'epoch': 4.16} 83%|████████▎ | 1341/1610 [5:46:18<1:08:29, 15.28s/it] 83%|████████▎ | 1342/1610 [5:46:33<1:08:20, 15.30s/it] {'loss': 0.001, 'grad_norm': 1.2928757613161757, 'learning_rate': 1.6645962732919252e-07, 'completion_length': 163.9732208251953, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2858598679304123, 'kl': 0.0252685546875, 'epoch': 4.17} 83%|████████▎ | 1342/1610 [5:46:33<1:08:20, 15.30s/it] 83%|████████▎ | 1343/1610 [5:46:47<1:06:26, 14.93s/it] {'loss': 0.0009, 'grad_norm': 0.9630767293088789, 'learning_rate': 1.6583850931677018e-07, 'completion_length': 165.06250762939453, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285714626312256, 'reward_std': 0.32585133612155914, 'kl': 0.02301025390625, 'epoch': 4.17} 83%|████████▎ | 1343/1610 [5:46:47<1:06:26, 14.93s/it] 83%|████████▎ | 1344/1610 [5:47:02<1:06:20, 14.96s/it] {'loss': 0.0006, 'grad_norm': 2.9983558123806944, 'learning_rate': 1.652173913043478e-07, 'completion_length': 142.95536041259766, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2987918108701706, 'kl': 0.015472412109375, 'epoch': 4.17} 83%|████████▎ | 1344/1610 [5:47:02<1:06:20, 14.96s/it] 84%|████████▎ | 1345/1610 [5:47:17<1:05:58, 14.94s/it] {'loss': 0.0006, 'grad_norm': 2.977656034942276, 'learning_rate': 1.6459627329192547e-07, 'completion_length': 172.04464721679688, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.1995968073606491, 'kl': 0.01593017578125, 'epoch': 4.18} 84%|████████▎ | 1345/1610 [5:47:17<1:05:58, 14.94s/it] 84%|████████▎ | 1346/1610 [5:47:32<1:05:35, 14.91s/it] {'loss': 0.0009, 'grad_norm': 1.1553764643847082, 'learning_rate': 1.639751552795031e-07, 'completion_length': 162.87500762939453, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.23898375034332275, 'kl': 0.0235595703125, 'epoch': 4.18} 84%|████████▎ | 1346/1610 [5:47:32<1:05:35, 14.91s/it] 84%|████████▎ | 1347/1610 [5:47:46<1:04:37, 14.74s/it] {'loss': 0.0005, 'grad_norm': 1.4728575439966773, 'learning_rate': 1.6335403726708073e-07, 'completion_length': 136.5357208251953, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.23898935317993164, 'kl': 0.013458251953125, 'epoch': 4.18} 84%|████████▎ | 1347/1610 [5:47:46<1:04:37, 14.74s/it] 84%|████████▎ | 1348/1610 [5:48:03<1:06:34, 15.24s/it] {'loss': 0.0008, 'grad_norm': 1.3897727200875096, 'learning_rate': 1.6273291925465837e-07, 'completion_length': 176.93750762939453, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.28768569231033325, 'kl': 0.021240234375, 'epoch': 4.19} 84%|████████▎ | 1348/1610 [5:48:03<1:06:34, 15.24s/it] 84%|████████▍ | 1349/1610 [5:48:17<1:05:26, 15.04s/it] {'loss': 0.0008, 'grad_norm': 1.066836455792855, 'learning_rate': 1.6211180124223603e-07, 'completion_length': 163.67858123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.2567032426595688, 'kl': 0.02008056640625, 'epoch': 4.19} 84%|████████▍ | 1349/1610 [5:48:17<1:05:26, 15.04s/it] 84%|████████▍ | 1350/1610 [5:48:32<1:05:07, 15.03s/it] {'loss': 0.0006, 'grad_norm': 1.0145021877746028, 'learning_rate': 1.6149068322981366e-07, 'completion_length': 139.7232208251953, 'rewards/accuracy_reward': 0.651785746216774, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.12956400215625763, 'kl': 0.015380859375, 'epoch': 4.19} 84%|████████▍ | 1350/1610 [5:48:32<1:05:07, 15.03s/it] 84%|████████▍ | 1351/1610 [5:48:47<1:04:43, 15.00s/it] {'loss': 0.0006, 'grad_norm': 1.0256652278529002, 'learning_rate': 1.608695652173913e-07, 'completion_length': 144.5089340209961, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.30390140414237976, 'kl': 0.015411376953125, 'epoch': 4.2} 84%|████████▍ | 1351/1610 [5:48:47<1:04:43, 15.00s/it] 84%|████████▍ | 1352/1610 [5:49:01<1:03:01, 14.66s/it] {'loss': 0.0007, 'grad_norm': 1.006438126420131, 'learning_rate': 1.6024844720496895e-07, 'completion_length': 131.69644165039062, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.17495647072792053, 'kl': 0.0169677734375, 'epoch': 4.2} 84%|████████▍ | 1352/1610 [5:49:01<1:03:01, 14.66s/it] 84%|████████▍ | 1353/1610 [5:49:16<1:03:53, 14.92s/it] {'loss': 0.0008, 'grad_norm': 2.031233063727292, 'learning_rate': 1.5962732919254656e-07, 'completion_length': 187.46429443359375, 'rewards/accuracy_reward': 0.6250000447034836, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.2954968959093094, 'kl': 0.0208740234375, 'epoch': 4.2} 84%|████████▍ | 1353/1610 [5:49:16<1:03:53, 14.92s/it] 84%|████████▍ | 1354/1610 [5:49:33<1:05:56, 15.46s/it] {'loss': 0.0008, 'grad_norm': 0.8561496848976479, 'learning_rate': 1.5900621118012422e-07, 'completion_length': 158.3928680419922, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.18539898097515106, 'kl': 0.02099609375, 'epoch': 4.2} 84%|████████▍ | 1354/1610 [5:49:33<1:05:56, 15.46s/it] 84%|████████▍ | 1355/1610 [5:49:48<1:05:07, 15.32s/it] {'loss': 0.0011, 'grad_norm': 1.228847637571133, 'learning_rate': 1.5838509316770185e-07, 'completion_length': 157.98214721679688, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.2792610377073288, 'kl': 0.0274658203125, 'epoch': 4.21} 84%|████████▍ | 1355/1610 [5:49:48<1:05:07, 15.32s/it] 84%|████████▍ | 1356/1610 [5:50:02<1:03:31, 15.01s/it] {'loss': 0.0008, 'grad_norm': 1.1124036675499467, 'learning_rate': 1.577639751552795e-07, 'completion_length': 167.15179443359375, 'rewards/accuracy_reward': 0.2946428805589676, 'rewards/format_reward': 1.0, 'reward': 1.2946429252624512, 'reward_std': 0.19838443398475647, 'kl': 0.01971435546875, 'epoch': 4.21} 84%|████████▍ | 1356/1610 [5:50:02<1:03:31, 15.01s/it] 84%|████████▍ | 1357/1610 [5:50:17<1:02:40, 14.86s/it] {'loss': 0.0007, 'grad_norm': 0.7760550201074942, 'learning_rate': 1.5714285714285714e-07, 'completion_length': 179.85714721679688, 'rewards/accuracy_reward': 0.3125000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3125000596046448, 'reward_std': 0.20020468533039093, 'kl': 0.017822265625, 'epoch': 4.21} 84%|████████▍ | 1357/1610 [5:50:17<1:02:40, 14.86s/it] 84%|████████▍ | 1358/1610 [5:50:31<1:01:12, 14.57s/it] {'loss': 0.0008, 'grad_norm': 1.9395783947751846, 'learning_rate': 1.565217391304348e-07, 'completion_length': 135.06250762939453, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.2993997037410736, 'kl': 0.02105712890625, 'epoch': 4.22} 84%|████████▍ | 1358/1610 [5:50:31<1:01:12, 14.57s/it] 84%|████████▍ | 1359/1610 [5:50:48<1:03:32, 15.19s/it] {'loss': 0.0009, 'grad_norm': 1.3034710355375745, 'learning_rate': 1.559006211180124e-07, 'completion_length': 177.30358123779297, 'rewards/accuracy_reward': 0.3660714477300644, 'rewards/format_reward': 1.0, 'reward': 1.3660714626312256, 'reward_std': 0.2119186520576477, 'kl': 0.0224609375, 'epoch': 4.22} 84%|████████▍ | 1359/1610 [5:50:48<1:03:32, 15.19s/it] 84%|████████▍ | 1360/1610 [5:51:01<1:01:29, 14.76s/it] {'loss': 0.0006, 'grad_norm': 1.1725386220559562, 'learning_rate': 1.5527950310559004e-07, 'completion_length': 125.11608123779297, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.19057324528694153, 'kl': 0.0159912109375, 'epoch': 4.22} 84%|████████▍ | 1360/1610 [5:51:01<1:01:29, 14.76s/it] 85%|████████▍ | 1361/1610 [5:51:14<59:04, 14.24s/it] {'loss': 0.0006, 'grad_norm': 1.7595028912243955, 'learning_rate': 1.546583850931677e-07, 'completion_length': 119.25000762939453, 'rewards/accuracy_reward': 0.7232142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.1866704449057579, 'kl': 0.01495361328125, 'epoch': 4.23} 85%|████████▍ | 1361/1610 [5:51:14<59:04, 14.24s/it] 85%|████████▍ | 1362/1610 [5:51:28<58:17, 14.10s/it] {'loss': 0.0009, 'grad_norm': 2.258331452741998, 'learning_rate': 1.5403726708074533e-07, 'completion_length': 146.37500762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.30061204731464386, 'kl': 0.02276611328125, 'epoch': 4.23} 85%|████████▍ | 1362/1610 [5:51:28<58:17, 14.10s/it] 85%|████████▍ | 1363/1610 [5:51:44<1:00:22, 14.67s/it] {'loss': 0.0009, 'grad_norm': 2.1618333224619812, 'learning_rate': 1.53416149068323e-07, 'completion_length': 161.0982208251953, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6517857909202576, 'reward_std': 0.30842325091362, 'kl': 0.0223388671875, 'epoch': 4.23} 85%|████████▍ | 1363/1610 [5:51:44<1:00:22, 14.67s/it] 85%|████████▍ | 1364/1610 [5:51:58<59:04, 14.41s/it] {'loss': 0.0008, 'grad_norm': 1.3123205089069199, 'learning_rate': 1.5279503105590062e-07, 'completion_length': 146.56250762939453, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.3724505752325058, 'kl': 0.0194091796875, 'epoch': 4.24} 85%|████████▍ | 1364/1610 [5:51:58<59:04, 14.41s/it] 85%|████████▍ | 1365/1610 [5:52:12<58:17, 14.28s/it] {'loss': 0.0007, 'grad_norm': 1.0360409641846613, 'learning_rate': 1.5217391304347825e-07, 'completion_length': 159.7678680419922, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2702430859208107, 'kl': 0.01739501953125, 'epoch': 4.24} 85%|████████▍ | 1365/1610 [5:52:12<58:17, 14.28s/it] 85%|████████▍ | 1366/1610 [5:52:27<58:48, 14.46s/it] {'loss': 0.0006, 'grad_norm': 1.8986053848056272, 'learning_rate': 1.5155279503105589e-07, 'completion_length': 142.0357208251953, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2501044273376465, 'kl': 0.01483154296875, 'epoch': 4.24} 85%|████████▍ | 1366/1610 [5:52:27<58:48, 14.46s/it] 85%|████████▍ | 1367/1610 [5:52:42<59:02, 14.58s/it] {'loss': 0.0007, 'grad_norm': 1.204798159920807, 'learning_rate': 1.5093167701863354e-07, 'completion_length': 167.2053680419922, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2702374756336212, 'kl': 0.0181884765625, 'epoch': 4.25} 85%|████████▍ | 1367/1610 [5:52:42<59:02, 14.58s/it] 85%|████████▍ | 1368/1610 [5:52:55<57:43, 14.31s/it] {'loss': 0.0007, 'grad_norm': 4.976886808005056, 'learning_rate': 1.5031055900621118e-07, 'completion_length': 142.6607208251953, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.24949097633361816, 'kl': 0.01641845703125, 'epoch': 4.25} 85%|████████▍ | 1368/1610 [5:52:55<57:43, 14.31s/it] 85%|████████▌ | 1369/1610 [5:53:11<59:30, 14.82s/it] {'loss': 0.001, 'grad_norm': 4.999481029892092, 'learning_rate': 1.496894409937888e-07, 'completion_length': 189.56250762939453, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2948834300041199, 'kl': 0.0240478515625, 'epoch': 4.25} 85%|████████▌ | 1369/1610 [5:53:11<59:30, 14.82s/it] 85%|████████▌ | 1370/1610 [5:53:27<1:00:08, 15.03s/it] {'loss': 0.0011, 'grad_norm': 1.776892575324987, 'learning_rate': 1.4906832298136647e-07, 'completion_length': 162.43750762939453, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.30659739673137665, 'kl': 0.0283203125, 'epoch': 4.25} 85%|████████▌ | 1370/1610 [5:53:27<1:00:08, 15.03s/it] 85%|████████▌ | 1371/1610 [5:53:43<1:01:00, 15.32s/it] {'loss': 0.0008, 'grad_norm': 1.9467465773637045, 'learning_rate': 1.4844720496894407e-07, 'completion_length': 173.64286041259766, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.3033080995082855, 'kl': 0.0191650390625, 'epoch': 4.26} 85%|████████▌ | 1371/1610 [5:53:43<1:01:00, 15.32s/it] 85%|████████▌ | 1372/1610 [5:53:59<1:01:13, 15.43s/it] {'loss': 0.0007, 'grad_norm': 1.876084301035996, 'learning_rate': 1.4782608695652173e-07, 'completion_length': 148.7946548461914, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.35440902411937714, 'kl': 0.01654052734375, 'epoch': 4.26} 85%|████████▌ | 1372/1610 [5:53:59<1:01:13, 15.43s/it] 85%|████████▌ | 1373/1610 [5:54:13<1:00:08, 15.23s/it] {'loss': 0.0007, 'grad_norm': 1.1763563388459968, 'learning_rate': 1.4720496894409937e-07, 'completion_length': 153.5357208251953, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2915885001420975, 'kl': 0.017486572265625, 'epoch': 4.26} 85%|████████▌ | 1373/1610 [5:54:13<1:00:08, 15.23s/it] 85%|████████▌ | 1374/1610 [5:54:29<1:00:08, 15.29s/it] {'loss': 0.0009, 'grad_norm': 1.2025738652437723, 'learning_rate': 1.4658385093167703e-07, 'completion_length': 145.37500762939453, 'rewards/accuracy_reward': 0.4732143133878708, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.4553572535514832, 'reward_std': 0.18355892598628998, 'kl': 0.0218505859375, 'epoch': 4.27} 85%|████████▌ | 1374/1610 [5:54:29<1:00:08, 15.29s/it] 85%|████████▌ | 1375/1610 [5:54:45<1:00:31, 15.45s/it] {'loss': 0.0007, 'grad_norm': 1.2758815902841423, 'learning_rate': 1.4596273291925466e-07, 'completion_length': 172.69644165039062, 'rewards/accuracy_reward': 0.3750000149011612, 'rewards/format_reward': 1.0, 'reward': 1.3750000596046448, 'reward_std': 0.21313104033470154, 'kl': 0.016815185546875, 'epoch': 4.27} 85%|████████▌ | 1375/1610 [5:54:45<1:00:31, 15.45s/it] 85%|████████▌ | 1376/1610 [5:54:58<57:59, 14.87s/it] {'loss': 0.0007, 'grad_norm': 1.6260460472554112, 'learning_rate': 1.4534161490683232e-07, 'completion_length': 136.28572463989258, 'rewards/accuracy_reward': 0.4196428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4196429252624512, 'reward_std': 0.2894969880580902, 'kl': 0.016845703125, 'epoch': 4.27} 85%|████████▌ | 1376/1610 [5:54:58<57:59, 14.87s/it] 86%|████████▌ | 1377/1610 [5:55:14<58:38, 15.10s/it] {'loss': 0.0009, 'grad_norm': 1.0835908792973397, 'learning_rate': 1.4472049689440992e-07, 'completion_length': 176.84822845458984, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.2831694334745407, 'kl': 0.0220947265625, 'epoch': 4.28} 86%|████████▌ | 1377/1610 [5:55:14<58:38, 15.10s/it] 86%|████████▌ | 1378/1610 [5:55:28<57:53, 14.97s/it] {'loss': 0.0006, 'grad_norm': 1.2618944794851423, 'learning_rate': 1.4409937888198756e-07, 'completion_length': 132.0714340209961, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.3492938429117203, 'kl': 0.015045166015625, 'epoch': 4.28} 86%|████████▌ | 1378/1610 [5:55:28<57:53, 14.97s/it] 86%|████████▌ | 1379/1610 [5:55:44<58:18, 15.15s/it] {'loss': 0.0007, 'grad_norm': 1.1718435494638144, 'learning_rate': 1.4347826086956521e-07, 'completion_length': 157.35714721679688, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.20801587402820587, 'kl': 0.01654052734375, 'epoch': 4.28} 86%|████████▌ | 1379/1610 [5:55:44<58:18, 15.15s/it] 86%|████████▌ | 1380/1610 [5:55:58<56:38, 14.78s/it] {'loss': 0.0011, 'grad_norm': 0.8757761611873451, 'learning_rate': 1.4285714285714285e-07, 'completion_length': 172.17858123779297, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.2248506173491478, 'kl': 0.02740478515625, 'epoch': 4.29} 86%|████████▌ | 1380/1610 [5:55:58<56:38, 14.78s/it] 86%|████████▌ | 1381/1610 [5:56:12<55:09, 14.45s/it] {'loss': 0.0007, 'grad_norm': 1.2617197433454106, 'learning_rate': 1.422360248447205e-07, 'completion_length': 139.51786041259766, 'rewards/accuracy_reward': 0.7142857611179352, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.20021028071641922, 'kl': 0.0185546875, 'epoch': 4.29} 86%|████████▌ | 1381/1610 [5:56:12<55:09, 14.45s/it] 86%|████████▌ | 1382/1610 [5:56:26<55:18, 14.56s/it] {'loss': 0.0007, 'grad_norm': 1.2428056671431638, 'learning_rate': 1.4161490683229814e-07, 'completion_length': 153.7232208251953, 'rewards/accuracy_reward': 0.5178571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.23535221815109253, 'kl': 0.016510009765625, 'epoch': 4.29} 86%|████████▌ | 1382/1610 [5:56:26<55:18, 14.56s/it] 86%|████████▌ | 1383/1610 [5:56:40<54:25, 14.38s/it] {'loss': 0.0008, 'grad_norm': 1.3026967726430585, 'learning_rate': 1.4099378881987577e-07, 'completion_length': 148.11607360839844, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428571939468384, 'reward_std': 0.3525831699371338, 'kl': 0.0194091796875, 'epoch': 4.3} 86%|████████▌ | 1383/1610 [5:56:40<54:25, 14.38s/it] 86%|████████▌ | 1384/1610 [5:56:54<53:48, 14.29s/it] {'loss': 0.001, 'grad_norm': 1.0221099841444272, 'learning_rate': 1.403726708074534e-07, 'completion_length': 159.2946548461914, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.30330808460712433, 'kl': 0.02490234375, 'epoch': 4.3} 86%|████████▌ | 1384/1610 [5:56:54<53:48, 14.29s/it] 86%|████████▌ | 1385/1610 [5:57:10<55:09, 14.71s/it] {'loss': 0.0009, 'grad_norm': 1.3962187430614235, 'learning_rate': 1.3975155279503104e-07, 'completion_length': 158.8214340209961, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.35111968219280243, 'kl': 0.02215576171875, 'epoch': 4.3} 86%|████████▌ | 1385/1610 [5:57:10<55:09, 14.71s/it] 86%|████████▌ | 1386/1610 [5:57:26<56:06, 15.03s/it] {'loss': 0.0007, 'grad_norm': 0.9085556274518807, 'learning_rate': 1.391304347826087e-07, 'completion_length': 153.10714721679688, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.14579425752162933, 'kl': 0.016937255859375, 'epoch': 4.3} 86%|████████▌ | 1386/1610 [5:57:26<56:06, 15.03s/it] 86%|████████▌ | 1387/1610 [5:57:41<56:20, 15.16s/it] {'loss': 0.0007, 'grad_norm': 1.8585555933708586, 'learning_rate': 1.3850931677018633e-07, 'completion_length': 154.58036041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.25791002810001373, 'kl': 0.0172119140625, 'epoch': 4.31} 86%|████████▌ | 1387/1610 [5:57:41<56:20, 15.16s/it] 86%|████████▌ | 1388/1610 [5:57:57<56:57, 15.39s/it] {'loss': 0.0009, 'grad_norm': 1.4897731730648998, 'learning_rate': 1.3788819875776399e-07, 'completion_length': 177.6964340209961, 'rewards/accuracy_reward': 0.4375000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4285714626312256, 'reward_std': 0.37662459909915924, 'kl': 0.02130126953125, 'epoch': 4.31} 86%|████████▌ | 1388/1610 [5:57:57<56:57, 15.39s/it] 86%|████████▋ | 1389/1610 [5:58:12<55:57, 15.19s/it] {'loss': 0.0009, 'grad_norm': 1.0360004069132978, 'learning_rate': 1.372670807453416e-07, 'completion_length': 172.67858123779297, 'rewards/accuracy_reward': 0.4553571492433548, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.3078097850084305, 'kl': 0.023681640625, 'epoch': 4.31} 86%|████████▋ | 1389/1610 [5:58:12<55:57, 15.19s/it] 86%|████████▋ | 1390/1610 [5:58:26<54:40, 14.91s/it] {'loss': 0.0009, 'grad_norm': 1.4400021899741067, 'learning_rate': 1.3664596273291925e-07, 'completion_length': 155.05358123779297, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2726622223854065, 'kl': 0.0223388671875, 'epoch': 4.32} 86%|████████▋ | 1390/1610 [5:58:26<54:40, 14.91s/it] 86%|████████▋ | 1391/1610 [5:58:41<53:56, 14.78s/it] {'loss': 0.0008, 'grad_norm': 1.7419205276559875, 'learning_rate': 1.3602484472049688e-07, 'completion_length': 128.93750381469727, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.3408748060464859, 'kl': 0.0194091796875, 'epoch': 4.32} 86%|████████▋ | 1391/1610 [5:58:41<53:56, 14.78s/it] 86%|████████▋ | 1392/1610 [5:58:55<53:35, 14.75s/it] {'loss': 0.0008, 'grad_norm': 7.63017102329505, 'learning_rate': 1.3540372670807454e-07, 'completion_length': 161.71428680419922, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.25791002810001373, 'kl': 0.019683837890625, 'epoch': 4.32} 86%|████████▋ | 1392/1610 [5:58:55<53:35, 14.75s/it] 87%|████████▋ | 1393/1610 [5:59:10<53:42, 14.85s/it] {'loss': 0.0009, 'grad_norm': 1.2432801313352084, 'learning_rate': 1.3478260869565218e-07, 'completion_length': 165.31250762939453, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3750001192092896, 'reward_std': 0.28168581426143646, 'kl': 0.0233154296875, 'epoch': 4.33} 87%|████████▋ | 1393/1610 [5:59:10<53:42, 14.85s/it] 87%|████████▋ | 1394/1610 [5:59:25<52:47, 14.66s/it] {'loss': 0.0006, 'grad_norm': 1.6746560302060616, 'learning_rate': 1.3416149068322978e-07, 'completion_length': 138.64286041259766, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.3733353018760681, 'kl': 0.0142822265625, 'epoch': 4.33} 87%|████████▋ | 1394/1610 [5:59:25<52:47, 14.66s/it] 87%|████████▋ | 1395/1610 [5:59:38<51:26, 14.35s/it] {'loss': 0.0007, 'grad_norm': 2.522810509620746, 'learning_rate': 1.3354037267080744e-07, 'completion_length': 140.9107208251953, 'rewards/accuracy_reward': 0.589285746216774, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.30721086263656616, 'kl': 0.017730712890625, 'epoch': 4.33} 87%|████████▋ | 1395/1610 [5:59:38<51:26, 14.35s/it] 87%|████████▋ | 1396/1610 [5:59:54<52:16, 14.66s/it] {'loss': 0.001, 'grad_norm': 1.2971364963181937, 'learning_rate': 1.3291925465838507e-07, 'completion_length': 178.95536041259766, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3501584231853485, 'kl': 0.0252685546875, 'epoch': 4.34} 87%|████████▋ | 1396/1610 [5:59:54<52:16, 14.66s/it] 87%|████████▋ | 1397/1610 [6:00:10<53:20, 15.03s/it] {'loss': 0.0011, 'grad_norm': 1.614235518985025, 'learning_rate': 1.3229813664596273e-07, 'completion_length': 155.6964340209961, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5178571939468384, 'reward_std': 0.234482042491436, 'kl': 0.02752685546875, 'epoch': 4.34} 87%|████████▋ | 1397/1610 [6:00:10<53:20, 15.03s/it] 87%|████████▋ | 1398/1610 [6:00:26<54:10, 15.33s/it] {'loss': 0.0009, 'grad_norm': 1.0023703122187828, 'learning_rate': 1.3167701863354037e-07, 'completion_length': 202.67858123779297, 'rewards/accuracy_reward': 0.3482142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3392857909202576, 'reward_std': 0.24988949298858643, 'kl': 0.022705078125, 'epoch': 4.34} 87%|████████▋ | 1398/1610 [6:00:26<54:10, 15.33s/it] 87%|████████▋ | 1399/1610 [6:00:40<53:09, 15.12s/it] {'loss': 0.0008, 'grad_norm': 1.852927138337809, 'learning_rate': 1.3105590062111802e-07, 'completion_length': 141.31250762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.33184564113616943, 'kl': 0.02069091796875, 'epoch': 4.34} 87%|████████▋ | 1399/1610 [6:00:40<53:09, 15.12s/it] 87%|████████▋ | 1400/1610 [6:00:54<52:00, 14.86s/it] {'loss': 0.0008, 'grad_norm': 10.417025469684674, 'learning_rate': 1.3043478260869563e-07, 'completion_length': 150.99108123779297, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.29999861121177673, 'kl': 0.01922607421875, 'epoch': 4.35} 87%|████████▋ | 1400/1610 [6:00:54<52:00, 14.86s/it] 87%|████████▋ | 1401/1610 [6:02:13<1:58:02, 33.89s/it] {'loss': 0.0009, 'grad_norm': 1.0440791737289314, 'learning_rate': 1.298136645962733e-07, 'completion_length': 173.55358123779297, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.32915520668029785, 'kl': 0.0234375, 'epoch': 4.35} 87%|████████▋ | 1401/1610 [6:02:13<1:58:02, 33.89s/it] 87%|████████▋ | 1402/1610 [6:02:24<1:34:14, 27.18s/it] {'loss': 0.0007, 'grad_norm': 2.092033352239161, 'learning_rate': 1.2919254658385092e-07, 'completion_length': 149.3214340209961, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.36552970111370087, 'kl': 0.01837158203125, 'epoch': 4.35} 87%|████████▋ | 1402/1610 [6:02:24<1:34:14, 27.18s/it] 87%|████████▋ | 1403/1610 [6:02:40<1:21:35, 23.65s/it] {'loss': 0.001, 'grad_norm': 1.1684424866022214, 'learning_rate': 1.2857142857142855e-07, 'completion_length': 187.44644165039062, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.3267304599285126, 'kl': 0.02435302734375, 'epoch': 4.36} 87%|████████▋ | 1403/1610 [6:02:40<1:21:35, 23.65s/it] 87%|████████▋ | 1404/1610 [6:02:53<1:10:38, 20.57s/it] {'loss': 0.0006, 'grad_norm': 1.1987852683631677, 'learning_rate': 1.2795031055900621e-07, 'completion_length': 120.87500762939453, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.2540128529071808, 'kl': 0.01568603515625, 'epoch': 4.36} 87%|████████▋ | 1404/1610 [6:02:53<1:10:38, 20.57s/it] 87%|████████▋ | 1405/1610 [6:03:08<1:04:56, 19.01s/it] {'loss': 0.0009, 'grad_norm': 1.8058317990459383, 'learning_rate': 1.2732919254658385e-07, 'completion_length': 162.26786041259766, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.3213440179824829, 'kl': 0.02349853515625, 'epoch': 4.36} 87%|████████▋ | 1405/1610 [6:03:08<1:04:56, 19.01s/it] 87%|████████▋ | 1406/1610 [6:03:24<1:01:05, 17.97s/it] {'loss': 0.0007, 'grad_norm': 1.2284539759258566, 'learning_rate': 1.2670807453416148e-07, 'completion_length': 159.86607360839844, 'rewards/accuracy_reward': 0.6964285969734192, 'rewards/format_reward': 1.0, 'reward': 1.6964285969734192, 'reward_std': 0.2702374905347824, 'kl': 0.018096923828125, 'epoch': 4.37} 87%|████████▋ | 1406/1610 [6:03:24<1:01:05, 17.97s/it] 87%|████████▋ | 1407/1610 [6:03:41<59:52, 17.70s/it] {'loss': 0.0009, 'grad_norm': 0.9540954991823867, 'learning_rate': 1.260869565217391e-07, 'completion_length': 174.01786041259766, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.633928656578064, 'reward_std': 0.2254050076007843, 'kl': 0.021484375, 'epoch': 4.37} 87%|████████▋ | 1407/1610 [6:03:41<59:52, 17.70s/it] 87%|████████▋ | 1408/1610 [6:03:58<58:40, 17.43s/it] {'loss': 0.0008, 'grad_norm': 1.1750912849047117, 'learning_rate': 1.2546583850931677e-07, 'completion_length': 164.2946548461914, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.24498365819454193, 'kl': 0.01983642578125, 'epoch': 4.37} 87%|████████▋ | 1408/1610 [6:03:58<58:40, 17.43s/it] 88%|████████▊ | 1409/1610 [6:04:12<54:45, 16.35s/it] {'loss': 0.0007, 'grad_norm': 1.2243056852382472, 'learning_rate': 1.248447204968944e-07, 'completion_length': 146.68750762939453, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.26450884342193604, 'kl': 0.016632080078125, 'epoch': 4.38} 88%|████████▊ | 1409/1610 [6:04:12<54:45, 16.35s/it] 88%|████████▊ | 1410/1610 [6:04:28<54:25, 16.33s/it] {'loss': 0.0006, 'grad_norm': 0.8516566226220437, 'learning_rate': 1.2422360248447204e-07, 'completion_length': 181.06250762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.5714285969734192, 'reward_std': 0.2377769872546196, 'kl': 0.015472412109375, 'epoch': 4.38} 88%|████████▊ | 1410/1610 [6:04:28<54:25, 16.33s/it] 88%|████████▊ | 1411/1610 [6:04:44<54:15, 16.36s/it] {'loss': 0.0007, 'grad_norm': 1.5636351772524384, 'learning_rate': 1.236024844720497e-07, 'completion_length': 157.3839340209961, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.26450882852077484, 'kl': 0.01702880859375, 'epoch': 4.38} 88%|████████▊ | 1411/1610 [6:04:44<54:15, 16.36s/it] 88%|████████▊ | 1412/1610 [6:04:58<51:18, 15.55s/it] {'loss': 0.0008, 'grad_norm': 1.3546176430605747, 'learning_rate': 1.2298136645962733e-07, 'completion_length': 142.85714721679688, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.24437017738819122, 'kl': 0.019287109375, 'epoch': 4.39} 88%|████████▊ | 1412/1610 [6:04:58<51:18, 15.55s/it] 88%|████████▊ | 1413/1610 [6:05:14<51:29, 15.68s/it] {'loss': 0.001, 'grad_norm': 1.0867362561049931, 'learning_rate': 1.2236024844720496e-07, 'completion_length': 185.0714340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.20801587402820587, 'kl': 0.02496337890625, 'epoch': 4.39} 88%|████████▊ | 1413/1610 [6:05:14<51:29, 15.68s/it] 88%|████████▊ | 1414/1610 [6:05:28<49:39, 15.20s/it] {'loss': 0.0009, 'grad_norm': 1.7145768389968759, 'learning_rate': 1.2173913043478262e-07, 'completion_length': 147.86608123779297, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.33486928045749664, 'kl': 0.02252197265625, 'epoch': 4.39} 88%|████████▊ | 1414/1610 [6:05:28<49:39, 15.20s/it] 88%|████████▊ | 1415/1610 [6:05:44<49:37, 15.27s/it] {'loss': 0.0008, 'grad_norm': 0.9802802398120825, 'learning_rate': 1.2111801242236025e-07, 'completion_length': 162.6428680419922, 'rewards/accuracy_reward': 0.625, 'rewards/format_reward': 1.0, 'reward': 1.6250001192092896, 'reward_std': 0.26841722428798676, 'kl': 0.0196533203125, 'epoch': 4.39} 88%|████████▊ | 1415/1610 [6:05:44<49:37, 15.27s/it] 88%|████████▊ | 1416/1610 [6:06:00<50:27, 15.61s/it] {'loss': 0.001, 'grad_norm': 1.0560601753825263, 'learning_rate': 1.2049689440993788e-07, 'completion_length': 178.93750762939453, 'rewards/accuracy_reward': 0.7500000298023224, 'rewards/format_reward': 1.0, 'reward': 1.7500000596046448, 'reward_std': 0.21703945100307465, 'kl': 0.024169921875, 'epoch': 4.4} 88%|████████▊ | 1416/1610 [6:06:00<50:27, 15.61s/it] 88%|████████▊ | 1417/1610 [6:06:16<50:30, 15.70s/it] {'loss': 0.0007, 'grad_norm': 2.166798415165841, 'learning_rate': 1.1987577639751552e-07, 'completion_length': 157.65179443359375, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.34538546204566956, 'kl': 0.01763916015625, 'epoch': 4.4} 88%|████████▊ | 1417/1610 [6:06:16<50:30, 15.70s/it] 88%|████████▊ | 1418/1610 [6:06:31<50:07, 15.66s/it] {'loss': 0.0007, 'grad_norm': 1.0035497323098348, 'learning_rate': 1.1925465838509315e-07, 'completion_length': 175.3214340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.20080358535051346, 'kl': 0.0172119140625, 'epoch': 4.4} 88%|████████▊ | 1418/1610 [6:06:31<50:07, 15.66s/it] 88%|████████▊ | 1419/1610 [6:06:47<49:19, 15.49s/it] {'loss': 0.0008, 'grad_norm': 1.2474043111539308, 'learning_rate': 1.1863354037267081e-07, 'completion_length': 164.29464721679688, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2606060355901718, 'kl': 0.01947021484375, 'epoch': 4.41} 88%|████████▊ | 1419/1610 [6:06:47<49:19, 15.49s/it] 88%|████████▊ | 1420/1610 [6:07:00<47:21, 14.95s/it] {'loss': 0.001, 'grad_norm': 1.8055596660104993, 'learning_rate': 1.1801242236024844e-07, 'completion_length': 139.6964340209961, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.336966410279274, 'kl': 0.02557373046875, 'epoch': 4.41} 88%|████████▊ | 1420/1610 [6:07:00<47:21, 14.95s/it] 88%|████████▊ | 1421/1610 [6:07:16<47:30, 15.08s/it] {'loss': 0.0008, 'grad_norm': 3.0984366484448453, 'learning_rate': 1.1739130434782609e-07, 'completion_length': 162.08928680419922, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.473214328289032, 'reward_std': 0.3634326159954071, 'kl': 0.02069091796875, 'epoch': 4.41} 88%|████████▊ | 1421/1610 [6:07:16<47:30, 15.08s/it] 88%|████████▊ | 1422/1610 [6:07:31<47:21, 15.11s/it] {'loss': 0.0008, 'grad_norm': 1.0953521561074786, 'learning_rate': 1.1677018633540373e-07, 'completion_length': 157.04464721679688, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.20020467042922974, 'kl': 0.0194091796875, 'epoch': 4.42} 88%|████████▊ | 1422/1610 [6:07:31<47:21, 15.11s/it] 88%|████████▊ | 1423/1610 [6:07:46<47:38, 15.29s/it] {'loss': 0.0009, 'grad_norm': 1.7688205712321214, 'learning_rate': 1.1614906832298136e-07, 'completion_length': 162.67858123779297, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.325252428650856, 'kl': 0.0218505859375, 'epoch': 4.42} 88%|████████▊ | 1423/1610 [6:07:46<47:38, 15.29s/it] 88%|████████▊ | 1424/1610 [6:08:02<47:57, 15.47s/it] {'loss': 0.0008, 'grad_norm': 1.3672782276415225, 'learning_rate': 1.15527950310559e-07, 'completion_length': 153.40179443359375, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.607142984867096, 'reward_std': 0.2500453591346741, 'kl': 0.018829345703125, 'epoch': 4.42} 88%|████████▊ | 1424/1610 [6:08:02<47:57, 15.47s/it] 89%|████████▊ | 1425/1610 [6:08:17<46:40, 15.14s/it] {'loss': 0.0008, 'grad_norm': 2.9421494086943905, 'learning_rate': 1.1490683229813663e-07, 'completion_length': 137.56250762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2948834300041199, 'kl': 0.01953125, 'epoch': 4.43} 89%|████████▊ | 1425/1610 [6:08:17<46:40, 15.14s/it] 89%|████████▊ | 1426/1610 [6:08:31<45:56, 14.98s/it] {'loss': 0.0008, 'grad_norm': 1.9712024014122325, 'learning_rate': 1.1428571428571427e-07, 'completion_length': 136.86608123779297, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5089285969734192, 'reward_std': 0.388957679271698, 'kl': 0.01971435546875, 'epoch': 4.43} 89%|████████▊ | 1426/1610 [6:08:31<45:56, 14.98s/it] 89%|████████▊ | 1427/1610 [6:08:47<45:58, 15.08s/it] {'loss': 0.0009, 'grad_norm': 1.1089609230924802, 'learning_rate': 1.1366459627329192e-07, 'completion_length': 142.9732208251953, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.29097503423690796, 'kl': 0.0216064453125, 'epoch': 4.43} 89%|████████▊ | 1427/1610 [6:08:47<45:58, 15.08s/it] 89%|████████▊ | 1428/1610 [6:09:01<44:55, 14.81s/it] {'loss': 0.0007, 'grad_norm': 1.6186973923197823, 'learning_rate': 1.1304347826086955e-07, 'completion_length': 137.9464340209961, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.3123260587453842, 'kl': 0.018310546875, 'epoch': 4.43} 89%|████████▊ | 1428/1610 [6:09:01<44:55, 14.81s/it] 89%|████████▉ | 1429/1610 [6:09:17<45:44, 15.16s/it] {'loss': 0.0009, 'grad_norm': 1.4779529623127439, 'learning_rate': 1.124223602484472e-07, 'completion_length': 167.61607360839844, 'rewards/accuracy_reward': 0.401785746216774, 'rewards/format_reward': 1.0, 'reward': 1.4017857909202576, 'reward_std': 0.27804867178201675, 'kl': 0.0230712890625, 'epoch': 4.44} 89%|████████▉ | 1429/1610 [6:09:17<45:44, 15.16s/it] 89%|████████▉ | 1430/1610 [6:09:33<46:23, 15.46s/it] {'loss': 0.0009, 'grad_norm': 1.4292792523909588, 'learning_rate': 1.1180124223602484e-07, 'completion_length': 180.19644165039062, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.2022872269153595, 'kl': 0.021514892578125, 'epoch': 4.44} 89%|████████▉ | 1430/1610 [6:09:33<46:23, 15.46s/it] 89%|████████▉ | 1431/1610 [6:09:48<45:32, 15.27s/it] {'loss': 0.001, 'grad_norm': 0.9721679566352986, 'learning_rate': 1.1118012422360248e-07, 'completion_length': 165.92858123779297, 'rewards/accuracy_reward': 0.5000000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5000001192092896, 'reward_std': 0.26181840896606445, 'kl': 0.024169921875, 'epoch': 4.44} 89%|████████▉ | 1431/1610 [6:09:48<45:32, 15.27s/it] 89%|████████▉ | 1432/1610 [6:10:02<44:42, 15.07s/it] {'loss': 0.0008, 'grad_norm': 1.1456794879660204, 'learning_rate': 1.1055900621118012e-07, 'completion_length': 154.6607208251953, 'rewards/accuracy_reward': 0.4285714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.25583308935165405, 'kl': 0.02008056640625, 'epoch': 4.45} 89%|████████▉ | 1432/1610 [6:10:02<44:42, 15.07s/it] 89%|████████▉ | 1433/1610 [6:10:19<45:32, 15.44s/it] {'loss': 0.001, 'grad_norm': 1.6839573614528767, 'learning_rate': 1.0993788819875776e-07, 'completion_length': 152.4732208251953, 'rewards/accuracy_reward': 0.5178571790456772, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.22276806831359863, 'kl': 0.0244140625, 'epoch': 4.45} 89%|████████▉ | 1433/1610 [6:10:19<45:32, 15.44s/it] 89%|████████▉ | 1434/1610 [6:10:35<46:22, 15.81s/it] {'loss': 0.0007, 'grad_norm': 0.9165934454947371, 'learning_rate': 1.0931677018633539e-07, 'completion_length': 168.22322845458984, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6160715222358704, 'reward_std': 0.19751424342393875, 'kl': 0.018157958984375, 'epoch': 4.45} 89%|████████▉ | 1434/1610 [6:10:35<46:22, 15.81s/it] 89%|████████▉ | 1435/1610 [6:10:50<45:19, 15.54s/it] {'loss': 0.0006, 'grad_norm': 1.5283597981340968, 'learning_rate': 1.0869565217391303e-07, 'completion_length': 142.35714721679688, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14970265328884125, 'kl': 0.01568603515625, 'epoch': 4.46} 89%|████████▉ | 1435/1610 [6:10:50<45:19, 15.54s/it] 89%|████████▉ | 1436/1610 [6:11:07<45:45, 15.78s/it] {'loss': 0.0008, 'grad_norm': 1.3673209648452038, 'learning_rate': 1.0807453416149068e-07, 'completion_length': 172.60714721679688, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2663346827030182, 'kl': 0.02081298828125, 'epoch': 4.46} 89%|████████▉ | 1436/1610 [6:11:07<45:45, 15.78s/it] 89%|████████▉ | 1437/1610 [6:11:21<44:30, 15.44s/it] {'loss': 0.0008, 'grad_norm': 1.1620586318764978, 'learning_rate': 1.0745341614906831e-07, 'completion_length': 153.3482208251953, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803572535514832, 'reward_std': 0.28707224130630493, 'kl': 0.0211181640625, 'epoch': 4.46} 89%|████████▉ | 1437/1610 [6:11:21<44:30, 15.44s/it] 89%|████████▉ | 1438/1610 [6:11:36<43:38, 15.23s/it] {'loss': 0.0007, 'grad_norm': 1.3535467727981187, 'learning_rate': 1.0683229813664596e-07, 'completion_length': 153.65178680419922, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2819514572620392, 'kl': 0.0177001953125, 'epoch': 4.47} 89%|████████▉ | 1438/1610 [6:11:36<43:38, 15.23s/it] 89%|████████▉ | 1439/1610 [6:11:51<43:09, 15.14s/it] {'loss': 0.0008, 'grad_norm': 1.9277512912658266, 'learning_rate': 1.062111801242236e-07, 'completion_length': 137.05358123779297, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.36073654890060425, 'kl': 0.01971435546875, 'epoch': 4.47} 89%|████████▉ | 1439/1610 [6:11:51<43:09, 15.14s/it] 89%|████████▉ | 1440/1610 [6:12:06<43:02, 15.19s/it] {'loss': 0.0009, 'grad_norm': 1.5115843938958513, 'learning_rate': 1.0559006211180124e-07, 'completion_length': 154.9464340209961, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.29488343745470047, 'kl': 0.023681640625, 'epoch': 4.47} 89%|████████▉ | 1440/1610 [6:12:06<43:02, 15.19s/it] 90%|████████▉ | 1441/1610 [6:12:23<44:21, 15.75s/it] {'loss': 0.001, 'grad_norm': 1.1337892722112648, 'learning_rate': 1.0496894409937888e-07, 'completion_length': 176.00894165039062, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.21455559134483337, 'kl': 0.0240478515625, 'epoch': 4.48} 90%|████████▉ | 1441/1610 [6:12:23<44:21, 15.75s/it] 90%|████████▉ | 1442/1610 [6:12:38<43:04, 15.38s/it] {'loss': 0.0007, 'grad_norm': 1.2080668078933454, 'learning_rate': 1.0434782608695651e-07, 'completion_length': 147.1607208251953, 'rewards/accuracy_reward': 0.4017857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4017857313156128, 'reward_std': 0.2792666554450989, 'kl': 0.018646240234375, 'epoch': 4.48} 90%|████████▉ | 1442/1610 [6:12:38<43:04, 15.38s/it] 90%|████████▉ | 1443/1610 [6:12:52<41:59, 15.09s/it] {'loss': 0.0007, 'grad_norm': 1.0864857755825421, 'learning_rate': 1.0372670807453415e-07, 'completion_length': 158.7321548461914, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.15360543876886368, 'kl': 0.0169677734375, 'epoch': 4.48} 90%|████████▉ | 1443/1610 [6:12:52<41:59, 15.09s/it] 90%|████████▉ | 1444/1610 [6:13:07<41:34, 15.03s/it] {'loss': 0.0008, 'grad_norm': 1.7650136935306162, 'learning_rate': 1.0310559006211179e-07, 'completion_length': 157.6428680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.33575402200222015, 'kl': 0.02008056640625, 'epoch': 4.48} 90%|████████▉ | 1444/1610 [6:13:07<41:34, 15.03s/it] 90%|████████▉ | 1445/1610 [6:13:22<41:10, 14.97s/it] {'loss': 0.0008, 'grad_norm': 1.103302159410433, 'learning_rate': 1.0248447204968944e-07, 'completion_length': 156.5625114440918, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5446429252624512, 'reward_std': 0.2158270627260208, 'kl': 0.019073486328125, 'epoch': 4.49} 90%|████████▉ | 1445/1610 [6:13:22<41:10, 14.97s/it] 90%|████████▉ | 1446/1610 [6:13:37<40:43, 14.90s/it] {'loss': 0.0009, 'grad_norm': 1.3242991587858768, 'learning_rate': 1.0186335403726707e-07, 'completion_length': 151.89286041259766, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.2891491949558258, 'kl': 0.02191162109375, 'epoch': 4.49} 90%|████████▉ | 1446/1610 [6:13:37<40:43, 14.90s/it] 90%|████████▉ | 1447/1610 [6:13:53<41:52, 15.42s/it] {'loss': 0.0009, 'grad_norm': 2.929066628470045, 'learning_rate': 1.0124223602484472e-07, 'completion_length': 154.4821548461914, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2765706330537796, 'kl': 0.02276611328125, 'epoch': 4.49} 90%|████████▉ | 1447/1610 [6:13:53<41:52, 15.42s/it] 90%|████████▉ | 1448/1610 [6:14:07<40:26, 14.98s/it] {'loss': 0.0009, 'grad_norm': 1.6296586216225073, 'learning_rate': 1.0062111801242236e-07, 'completion_length': 131.8214340209961, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.33575402200222015, 'kl': 0.02203369140625, 'epoch': 4.5} 90%|████████▉ | 1448/1610 [6:14:07<40:26, 14.98s/it] 90%|█████████ | 1449/1610 [6:14:22<40:07, 14.95s/it] {'loss': 0.0007, 'grad_norm': 1.3752859307988354, 'learning_rate': 1e-07, 'completion_length': 153.33036041259766, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.633928656578064, 'reward_std': 0.30531174689531326, 'kl': 0.0185546875, 'epoch': 4.5} 90%|█████████ | 1449/1610 [6:14:22<40:07, 14.95s/it] 90%|█████████ | 1450/1610 [6:14:38<40:43, 15.27s/it] {'loss': 0.0007, 'grad_norm': 0.7926667599565141, 'learning_rate': 9.937888198757763e-08, 'completion_length': 158.5714340209961, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.15360544621944427, 'kl': 0.018310546875, 'epoch': 4.5} 90%|█████████ | 1450/1610 [6:14:38<40:43, 15.27s/it] 90%|█████████ | 1451/1610 [6:14:54<40:46, 15.39s/it] {'loss': 0.0006, 'grad_norm': 3.406358781486329, 'learning_rate': 9.875776397515527e-08, 'completion_length': 167.25894165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.2410808652639389, 'kl': 0.01446533203125, 'epoch': 4.51} 90%|█████████ | 1451/1610 [6:14:54<40:46, 15.39s/it] 90%|█████████ | 1452/1610 [6:15:10<41:00, 15.58s/it] {'loss': 0.0007, 'grad_norm': 0.9531529289477821, 'learning_rate': 9.81366459627329e-08, 'completion_length': 174.62500762939453, 'rewards/accuracy_reward': 0.464285746216774, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4553572535514832, 'reward_std': 0.2363496720790863, 'kl': 0.01678466796875, 'epoch': 4.51} 90%|█████████ | 1452/1610 [6:15:10<41:00, 15.58s/it] 90%|█████████ | 1453/1610 [6:15:25<40:31, 15.49s/it] {'loss': 0.0008, 'grad_norm': 1.2338196799029144, 'learning_rate': 9.751552795031055e-08, 'completion_length': 164.3303680419922, 'rewards/accuracy_reward': 0.4553571790456772, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.3279428035020828, 'kl': 0.02105712890625, 'epoch': 4.51} 90%|█████████ | 1453/1610 [6:15:25<40:31, 15.49s/it] 90%|█████████ | 1454/1610 [6:15:41<40:35, 15.61s/it] {'loss': 0.0008, 'grad_norm': 1.6140657107865093, 'learning_rate': 9.68944099378882e-08, 'completion_length': 163.80358123779297, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4732143878936768, 'reward_std': 0.2566976398229599, 'kl': 0.02117919921875, 'epoch': 4.52} 90%|█████████ | 1454/1610 [6:15:41<40:35, 15.61s/it] 90%|█████████ | 1455/1610 [6:15:56<39:38, 15.34s/it] {'loss': 0.0007, 'grad_norm': 1.5221733480714474, 'learning_rate': 9.627329192546583e-08, 'completion_length': 162.89286041259766, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2314438372850418, 'kl': 0.016448974609375, 'epoch': 4.52} 90%|█████████ | 1455/1610 [6:15:56<39:38, 15.34s/it] 90%|█████████ | 1456/1610 [6:16:11<39:03, 15.22s/it] {'loss': 0.0008, 'grad_norm': 1.1315877924455213, 'learning_rate': 9.565217391304348e-08, 'completion_length': 152.83929061889648, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.26841163635253906, 'kl': 0.020782470703125, 'epoch': 4.52} 90%|█████████ | 1456/1610 [6:16:11<39:03, 15.22s/it] 90%|█████████ | 1457/1610 [6:16:25<38:05, 14.93s/it] {'loss': 0.0007, 'grad_norm': 1.8343709920173183, 'learning_rate': 9.503105590062112e-08, 'completion_length': 139.77679443359375, 'rewards/accuracy_reward': 0.5535714477300644, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2609482407569885, 'kl': 0.01849365234375, 'epoch': 4.52} 90%|█████████ | 1457/1610 [6:16:25<38:05, 14.93s/it] 91%|█████████ | 1458/1610 [6:16:39<37:15, 14.71s/it] {'loss': 0.0007, 'grad_norm': 1.46165393067486, 'learning_rate': 9.440993788819875e-08, 'completion_length': 130.91964721679688, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.24889205396175385, 'kl': 0.018280029296875, 'epoch': 4.53} 91%|█████████ | 1458/1610 [6:16:39<37:15, 14.71s/it] 91%|█████████ | 1459/1610 [6:16:54<37:15, 14.81s/it] {'loss': 0.0007, 'grad_norm': 1.4775430162487628, 'learning_rate': 9.378881987577639e-08, 'completion_length': 146.99107360839844, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.4464285969734192, 'reward_std': 0.22363825142383575, 'kl': 0.01873779296875, 'epoch': 4.53} 91%|█████████ | 1459/1610 [6:16:54<37:15, 14.81s/it] 91%|█████████ | 1460/1610 [6:17:08<36:02, 14.41s/it] {'loss': 0.0006, 'grad_norm': 0.9503617075128578, 'learning_rate': 9.316770186335403e-08, 'completion_length': 139.12500762939453, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928572535514832, 'reward_std': 0.22215460240840912, 'kl': 0.01611328125, 'epoch': 4.53} 91%|█████████ | 1460/1610 [6:17:08<36:02, 14.41s/it] 91%|█████████ | 1461/1610 [6:17:23<36:30, 14.70s/it] {'loss': 0.0009, 'grad_norm': 1.0723342977355217, 'learning_rate': 9.254658385093167e-08, 'completion_length': 171.68750762939453, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.2597358673810959, 'kl': 0.02313232421875, 'epoch': 4.54} 91%|█████████ | 1461/1610 [6:17:23<36:30, 14.70s/it] 91%|█████████ | 1462/1610 [6:17:39<37:22, 15.15s/it] {'loss': 0.0007, 'grad_norm': 1.0454022769151439, 'learning_rate': 9.192546583850931e-08, 'completion_length': 181.56250762939453, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.410714328289032, 'reward_std': 0.1956884115934372, 'kl': 0.0169677734375, 'epoch': 4.54} 91%|█████████ | 1462/1610 [6:17:39<37:22, 15.15s/it] 91%|█████████ | 1463/1610 [6:17:55<37:34, 15.34s/it] {'loss': 0.0008, 'grad_norm': 0.9478865528106292, 'learning_rate': 9.130434782608696e-08, 'completion_length': 147.36608123779297, 'rewards/accuracy_reward': 0.8392857611179352, 'rewards/format_reward': 1.0, 'reward': 1.8392857909202576, 'reward_std': 0.17885926365852356, 'kl': 0.01995849609375, 'epoch': 4.54} 91%|█████████ | 1463/1610 [6:17:55<37:34, 15.34s/it] 91%|█████████ | 1464/1610 [6:18:12<38:10, 15.69s/it] {'loss': 0.0011, 'grad_norm': 1.8603972028788978, 'learning_rate': 9.068322981366459e-08, 'completion_length': 159.7232208251953, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.2248506247997284, 'kl': 0.0262451171875, 'epoch': 4.55} 91%|█████████ | 1464/1610 [6:18:12<38:10, 15.69s/it] 91%|█████████ | 1465/1610 [6:18:29<39:02, 16.15s/it] {'loss': 0.0011, 'grad_norm': 1.5579571538411938, 'learning_rate': 9.006211180124224e-08, 'completion_length': 177.16072845458984, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.37784260511398315, 'kl': 0.02777099609375, 'epoch': 4.55} 91%|█████████ | 1465/1610 [6:18:29<39:02, 16.15s/it] 91%|█████████ | 1466/1610 [6:18:44<38:06, 15.88s/it] {'loss': 0.0006, 'grad_norm': 1.4425552299994626, 'learning_rate': 8.944099378881988e-08, 'completion_length': 169.7946548461914, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.30061204731464386, 'kl': 0.01593017578125, 'epoch': 4.55} 91%|█████████ | 1466/1610 [6:18:44<38:06, 15.88s/it] 91%|█████████ | 1467/1610 [6:18:58<36:29, 15.31s/it] {'loss': 0.0008, 'grad_norm': 5.532966345526184, 'learning_rate': 8.881987577639751e-08, 'completion_length': 148.45536422729492, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.128351628780365, 'kl': 0.020782470703125, 'epoch': 4.56} 91%|█████████ | 1467/1610 [6:18:58<36:29, 15.31s/it] 91%|█████████ | 1468/1610 [6:19:12<35:08, 14.85s/it] {'loss': 0.0006, 'grad_norm': 1.2652592362712713, 'learning_rate': 8.819875776397515e-08, 'completion_length': 129.48214721679688, 'rewards/accuracy_reward': 0.7946428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7946429252624512, 'reward_std': 0.19178561121225357, 'kl': 0.0157470703125, 'epoch': 4.56} 91%|█████████ | 1468/1610 [6:19:12<35:08, 14.85s/it] 91%|█████████ | 1469/1610 [6:19:27<35:27, 15.09s/it] {'loss': 0.0011, 'grad_norm': 1.1419918463101622, 'learning_rate': 8.757763975155279e-08, 'completion_length': 164.0357208251953, 'rewards/accuracy_reward': 0.7142857313156128, 'rewards/format_reward': 1.0, 'reward': 1.7142857909202576, 'reward_std': 0.22875342518091202, 'kl': 0.0274658203125, 'epoch': 4.56} 91%|█████████ | 1469/1610 [6:19:27<35:27, 15.09s/it] 91%|█████████▏| 1470/1610 [6:19:43<35:38, 15.27s/it] {'loss': 0.0007, 'grad_norm': 1.6079211525470367, 'learning_rate': 8.695652173913042e-08, 'completion_length': 160.11607360839844, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.3207506984472275, 'kl': 0.0184326171875, 'epoch': 4.57} 91%|█████████▏| 1470/1610 [6:19:43<35:38, 15.27s/it] 91%|█████████▏| 1471/1610 [6:19:58<35:17, 15.23s/it] {'loss': 0.0007, 'grad_norm': 0.6434365177894316, 'learning_rate': 8.633540372670807e-08, 'completion_length': 160.55358123779297, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.16531942784786224, 'kl': 0.017608642578125, 'epoch': 4.57} 91%|█████████▏| 1471/1610 [6:19:58<35:17, 15.23s/it] 91%|█████████▏| 1472/1610 [6:20:14<35:25, 15.41s/it] {'loss': 0.0008, 'grad_norm': 1.1809887084488127, 'learning_rate': 8.571428571428572e-08, 'completion_length': 162.1607208251953, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.182762049138546, 'kl': 0.02093505859375, 'epoch': 4.57} 91%|█████████▏| 1472/1610 [6:20:14<35:25, 15.41s/it] 91%|█████████▏| 1473/1610 [6:20:29<35:01, 15.34s/it] {'loss': 0.0009, 'grad_norm': 0.9315540237304959, 'learning_rate': 8.509316770186335e-08, 'completion_length': 161.70536041259766, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.20411307364702225, 'kl': 0.02325439453125, 'epoch': 4.57} 91%|█████████▏| 1473/1610 [6:20:29<35:01, 15.34s/it] 92%|█████████▏| 1474/1610 [6:20:44<34:26, 15.19s/it] {'loss': 0.0008, 'grad_norm': 1.5253035761894147, 'learning_rate': 8.4472049689441e-08, 'completion_length': 148.9107208251953, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.26875942945480347, 'kl': 0.0203857421875, 'epoch': 4.58} 92%|█████████▏| 1474/1610 [6:20:44<34:26, 15.19s/it] 92%|█████████▏| 1475/1610 [6:20:59<33:49, 15.04s/it] {'loss': 0.0011, 'grad_norm': 2.2562063959450493, 'learning_rate': 8.385093167701864e-08, 'completion_length': 153.7589340209961, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.339656800031662, 'kl': 0.0264892578125, 'epoch': 4.58} 92%|█████████▏| 1475/1610 [6:20:59<33:49, 15.04s/it] 92%|█████████▏| 1476/1610 [6:21:16<35:10, 15.75s/it] {'loss': 0.0008, 'grad_norm': 1.1168666898431847, 'learning_rate': 8.322981366459626e-08, 'completion_length': 165.5089340209961, 'rewards/accuracy_reward': 0.5625, 'rewards/format_reward': 1.0, 'reward': 1.5625001192092896, 'reward_std': 0.1827620565891266, 'kl': 0.01995849609375, 'epoch': 4.58} 92%|█████████▏| 1476/1610 [6:21:16<35:10, 15.75s/it] 92%|█████████▏| 1477/1610 [6:21:30<33:47, 15.24s/it] {'loss': 0.0006, 'grad_norm': 2.2023854921781747, 'learning_rate': 8.26086956521739e-08, 'completion_length': 141.39286422729492, 'rewards/accuracy_reward': 0.7410714626312256, 'rewards/format_reward': 1.0, 'reward': 1.7410715222358704, 'reward_std': 0.1866704523563385, 'kl': 0.015960693359375, 'epoch': 4.59} 92%|█████████▏| 1477/1610 [6:21:30<33:47, 15.24s/it] 92%|█████████▏| 1478/1610 [6:21:45<33:01, 15.01s/it] {'loss': 0.0009, 'grad_norm': 1.3453595027755028, 'learning_rate': 8.198757763975155e-08, 'completion_length': 142.5714340209961, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3474680036306381, 'kl': 0.0216064453125, 'epoch': 4.59} 92%|█████████▏| 1478/1610 [6:21:45<33:01, 15.01s/it] 92%|█████████▏| 1479/1610 [6:22:00<33:16, 15.24s/it] {'loss': 0.0009, 'grad_norm': 1.8734547297344568, 'learning_rate': 8.136645962732918e-08, 'completion_length': 149.51786422729492, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.2663346827030182, 'kl': 0.0225830078125, 'epoch': 4.59} 92%|█████████▏| 1479/1610 [6:22:00<33:16, 15.24s/it] 92%|█████████▏| 1480/1610 [6:22:18<34:14, 15.80s/it] {'loss': 0.0008, 'grad_norm': 0.6257671646666686, 'learning_rate': 8.074534161490683e-08, 'completion_length': 191.58929443359375, 'rewards/accuracy_reward': 0.5535714328289032, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.1476055383682251, 'kl': 0.01971435546875, 'epoch': 4.6} 92%|█████████▏| 1480/1610 [6:22:18<34:14, 15.80s/it] 92%|█████████▏| 1481/1610 [6:22:32<33:21, 15.52s/it] {'loss': 0.0007, 'grad_norm': 3.535813549095223, 'learning_rate': 8.012422360248448e-08, 'completion_length': 144.91964721679688, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.26814597845077515, 'kl': 0.018310546875, 'epoch': 4.6} 92%|█████████▏| 1481/1610 [6:22:32<33:21, 15.52s/it] 92%|█████████▏| 1482/1610 [6:22:46<31:43, 14.87s/it] {'loss': 0.0006, 'grad_norm': 1.1367502697196874, 'learning_rate': 7.950310559006211e-08, 'completion_length': 129.45536041259766, 'rewards/accuracy_reward': 0.7232142984867096, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.17495646327733994, 'kl': 0.014678955078125, 'epoch': 4.6} 92%|█████████▏| 1482/1610 [6:22:46<31:43, 14.87s/it] 92%|█████████▏| 1483/1610 [6:23:01<31:35, 14.92s/it] {'loss': 0.0008, 'grad_norm': 1.1985393059402487, 'learning_rate': 7.888198757763975e-08, 'completion_length': 146.79464721679688, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.3174412101507187, 'kl': 0.018951416015625, 'epoch': 4.61} 92%|█████████▏| 1483/1610 [6:23:01<31:35, 14.92s/it] 92%|█████████▏| 1484/1610 [6:23:15<30:54, 14.72s/it] {'loss': 0.0008, 'grad_norm': 0.8847370329774433, 'learning_rate': 7.82608695652174e-08, 'completion_length': 158.02678680419922, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.25791002810001373, 'kl': 0.02008056640625, 'epoch': 4.61} 92%|█████████▏| 1484/1610 [6:23:15<30:54, 14.72s/it] 92%|█████████▏| 1485/1610 [6:23:30<30:58, 14.87s/it] {'loss': 0.0008, 'grad_norm': 1.2375457807454866, 'learning_rate': 7.763975155279502e-08, 'completion_length': 159.27679443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857313156128, 'reward_std': 0.3525831699371338, 'kl': 0.01971435546875, 'epoch': 4.61} 92%|█████████▏| 1485/1610 [6:23:30<30:58, 14.87s/it] 92%|█████████▏| 1486/1610 [6:23:46<31:10, 15.09s/it] {'loss': 0.0006, 'grad_norm': 1.3388806681149257, 'learning_rate': 7.701863354037266e-08, 'completion_length': 139.9464340209961, 'rewards/accuracy_reward': 0.7053571939468384, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.696428656578064, 'reward_std': 0.37697242200374603, 'kl': 0.014923095703125, 'epoch': 4.61} 92%|█████████▏| 1486/1610 [6:23:46<31:10, 15.09s/it] 92%|█████████▏| 1487/1610 [6:24:00<30:03, 14.66s/it] {'loss': 0.0006, 'grad_norm': 1.580302838645341, 'learning_rate': 7.639751552795031e-08, 'completion_length': 128.84821701049805, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.354414626955986, 'kl': 0.01416015625, 'epoch': 4.62} 92%|█████████▏| 1487/1610 [6:24:00<30:03, 14.66s/it] 92%|█████████▏| 1488/1610 [6:24:15<30:08, 14.82s/it] {'loss': 0.0008, 'grad_norm': 1.0120445614820515, 'learning_rate': 7.577639751552794e-08, 'completion_length': 181.50000762939453, 'rewards/accuracy_reward': 0.4910714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.2086147665977478, 'kl': 0.0196533203125, 'epoch': 4.62} 92%|█████████▏| 1488/1610 [6:24:15<30:08, 14.82s/it] 92%|█████████▏| 1489/1610 [6:24:30<30:01, 14.89s/it] {'loss': 0.0007, 'grad_norm': 1.0068433535549288, 'learning_rate': 7.515527950310559e-08, 'completion_length': 146.8214340209961, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.18276765942573547, 'kl': 0.017425537109375, 'epoch': 4.62} 92%|█████████▏| 1489/1610 [6:24:30<30:01, 14.89s/it] 93%|█████████▎| 1490/1610 [6:24:46<30:48, 15.40s/it] {'loss': 0.0007, 'grad_norm': 1.306950111820371, 'learning_rate': 7.453416149068323e-08, 'completion_length': 164.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3525831997394562, 'kl': 0.01739501953125, 'epoch': 4.63} 93%|█████████▎| 1490/1610 [6:24:46<30:48, 15.40s/it] 93%|█████████▎| 1491/1610 [6:25:02<30:51, 15.56s/it] {'loss': 0.001, 'grad_norm': 1.0293253639544186, 'learning_rate': 7.391304347826087e-08, 'completion_length': 195.0178680419922, 'rewards/accuracy_reward': 0.6339285969734192, 'rewards/format_reward': 1.0, 'reward': 1.633928656578064, 'reward_std': 0.35137078166007996, 'kl': 0.02423095703125, 'epoch': 4.63} 93%|█████████▎| 1491/1610 [6:25:02<30:51, 15.56s/it] 93%|█████████▎| 1492/1610 [6:25:17<29:55, 15.22s/it] {'loss': 0.0008, 'grad_norm': 1.0564179584667972, 'learning_rate': 7.329192546583851e-08, 'completion_length': 145.95536041259766, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.23569442331790924, 'kl': 0.0194091796875, 'epoch': 4.63} 93%|█████████▎| 1492/1610 [6:25:17<29:55, 15.22s/it] 93%|█████████▎| 1493/1610 [6:25:30<28:26, 14.59s/it] {'loss': 0.0008, 'grad_norm': 4.767183376340243, 'learning_rate': 7.267080745341616e-08, 'completion_length': 122.0535774230957, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.3174412250518799, 'kl': 0.01971435546875, 'epoch': 4.64} 93%|█████████▎| 1493/1610 [6:25:30<28:26, 14.59s/it] 93%|█████████▎| 1494/1610 [6:25:46<29:04, 15.04s/it] {'loss': 0.0009, 'grad_norm': 2.731390200682142, 'learning_rate': 7.204968944099378e-08, 'completion_length': 161.8839340209961, 'rewards/accuracy_reward': 0.6160714477300644, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.167145274579525, 'kl': 0.0233154296875, 'epoch': 4.64} 93%|█████████▎| 1494/1610 [6:25:46<29:04, 15.04s/it] 93%|█████████▎| 1495/1610 [6:26:00<28:28, 14.86s/it] {'loss': 0.0007, 'grad_norm': 1.900413403950408, 'learning_rate': 7.142857142857142e-08, 'completion_length': 125.38393783569336, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.36491625010967255, 'kl': 0.016510009765625, 'epoch': 4.64} 93%|█████████▎| 1495/1610 [6:26:00<28:28, 14.86s/it] 93%|█████████▎| 1496/1610 [6:26:17<29:19, 15.44s/it] {'loss': 0.0008, 'grad_norm': 1.1962579204628956, 'learning_rate': 7.080745341614907e-08, 'completion_length': 183.89286041259766, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.607142984867096, 'reward_std': 0.2776331752538681, 'kl': 0.02008056640625, 'epoch': 4.65} 93%|█████████▎| 1496/1610 [6:26:17<29:19, 15.44s/it] 93%|█████████▎| 1497/1610 [6:26:32<28:40, 15.23s/it] {'loss': 0.0007, 'grad_norm': 1.736246722480892, 'learning_rate': 7.01863354037267e-08, 'completion_length': 131.91965103149414, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.28706663846969604, 'kl': 0.01715087890625, 'epoch': 4.65} 93%|█████████▎| 1497/1610 [6:26:32<28:40, 15.23s/it] 93%|█████████▎| 1498/1610 [6:26:46<27:35, 14.78s/it] {'loss': 0.0007, 'grad_norm': 7.029996818943682, 'learning_rate': 6.956521739130435e-08, 'completion_length': 133.73215103149414, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.3240456283092499, 'kl': 0.01641845703125, 'epoch': 4.65} 93%|█████████▎| 1498/1610 [6:26:46<27:35, 14.78s/it] 93%|█████████▎| 1499/1610 [6:27:01<27:27, 14.84s/it] {'loss': 0.0008, 'grad_norm': 1.8252173003140622, 'learning_rate': 6.894409937888199e-08, 'completion_length': 159.18750762939453, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.3252524137496948, 'kl': 0.019775390625, 'epoch': 4.66} 93%|█████████▎| 1499/1610 [6:27:01<27:27, 14.84s/it] 93%|█████████▎| 1500/1610 [6:27:17<27:59, 15.27s/it] {'loss': 0.001, 'grad_norm': 1.1039156452567338, 'learning_rate': 6.832298136645963e-08, 'completion_length': 160.0357208251953, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2540072351694107, 'kl': 0.0242919921875, 'epoch': 4.66} 93%|█████████▎| 1500/1610 [6:27:17<27:59, 15.27s/it] 93%|█████████▎| 1501/1610 [6:28:24<55:52, 30.76s/it] {'loss': 0.0008, 'grad_norm': 0.9686448819576985, 'learning_rate': 6.770186335403727e-08, 'completion_length': 170.70536041259766, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.6071429252624512, 'reward_std': 0.2410808801651001, 'kl': 0.0198974609375, 'epoch': 4.66} 93%|█████████▎| 1501/1610 [6:28:24<55:52, 30.76s/it] 93%|█████████▎| 1502/1610 [6:28:39<47:06, 26.17s/it] {'loss': 0.001, 'grad_norm': 1.348939297744216, 'learning_rate': 6.708074534161489e-08, 'completion_length': 164.69644165039062, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4910715222358704, 'reward_std': 0.23656460642814636, 'kl': 0.02606201171875, 'epoch': 4.66} 93%|█████████▎| 1502/1610 [6:28:39<47:06, 26.17s/it] 93%|█████████▎| 1503/1610 [6:28:54<40:28, 22.69s/it] {'loss': 0.0008, 'grad_norm': 0.9516097194995695, 'learning_rate': 6.645962732919254e-08, 'completion_length': 165.02679443359375, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.20740240067243576, 'kl': 0.02081298828125, 'epoch': 4.67} 93%|█████████▎| 1503/1610 [6:28:54<40:28, 22.69s/it] 93%|█████████▎| 1504/1610 [6:29:08<35:34, 20.14s/it] {'loss': 0.0006, 'grad_norm': 0.9021143652379415, 'learning_rate': 6.583850931677018e-08, 'completion_length': 149.2857208251953, 'rewards/accuracy_reward': 0.2767857313156128, 'rewards/format_reward': 1.0, 'reward': 1.2767857909202576, 'reward_std': 0.2182518094778061, 'kl': 0.016021728515625, 'epoch': 4.67} 93%|█████████▎| 1504/1610 [6:29:08<35:34, 20.14s/it] 93%|█████████▎| 1505/1610 [6:29:23<32:18, 18.46s/it] {'loss': 0.0007, 'grad_norm': 1.439587902759633, 'learning_rate': 6.521739130434782e-08, 'completion_length': 129.28571701049805, 'rewards/accuracy_reward': 0.8035714626312256, 'rewards/format_reward': 1.0, 'reward': 1.8035715222358704, 'reward_std': 0.17885925620794296, 'kl': 0.016845703125, 'epoch': 4.67} 93%|█████████▎| 1505/1610 [6:29:23<32:18, 18.46s/it] 94%|█████████▎| 1506/1610 [6:29:37<30:01, 17.32s/it] {'loss': 0.0008, 'grad_norm': 1.1220817435299975, 'learning_rate': 6.459627329192546e-08, 'completion_length': 145.9464340209961, 'rewards/accuracy_reward': 0.392857164144516, 'rewards/format_reward': 1.0, 'reward': 1.3928572535514832, 'reward_std': 0.2831694483757019, 'kl': 0.0203857421875, 'epoch': 4.68} 94%|█████████▎| 1506/1610 [6:29:37<30:01, 17.32s/it] 94%|█████████▎| 1507/1610 [6:29:51<27:50, 16.22s/it] {'loss': 0.0007, 'grad_norm': 1.1269806036388823, 'learning_rate': 6.397515527950311e-08, 'completion_length': 134.5446434020996, 'rewards/accuracy_reward': 0.7321428656578064, 'rewards/format_reward': 1.0, 'reward': 1.732142984867096, 'reward_std': 0.18788282573223114, 'kl': 0.01715087890625, 'epoch': 4.68} 94%|█████████▎| 1507/1610 [6:29:51<27:50, 16.22s/it] 94%|█████████▎| 1508/1610 [6:30:07<27:20, 16.09s/it] {'loss': 0.0009, 'grad_norm': 1.1346665885523948, 'learning_rate': 6.335403726708074e-08, 'completion_length': 178.99108123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.26572681218385696, 'kl': 0.02191162109375, 'epoch': 4.68} 94%|█████████▎| 1508/1610 [6:30:07<27:20, 16.09s/it] 94%|█████████▎| 1509/1610 [6:30:23<27:04, 16.08s/it] {'loss': 0.0007, 'grad_norm': 5.890096996384099, 'learning_rate': 6.273291925465838e-08, 'completion_length': 169.06250762939453, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.2597358673810959, 'kl': 0.017364501953125, 'epoch': 4.69} 94%|█████████▎| 1509/1610 [6:30:23<27:04, 16.08s/it] 94%|█████████▍| 1510/1610 [6:30:36<25:34, 15.34s/it] {'loss': 0.0007, 'grad_norm': 1.7881560623559163, 'learning_rate': 6.211180124223602e-08, 'completion_length': 136.9107208251953, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.349907323718071, 'kl': 0.017578125, 'epoch': 4.69} 94%|█████████▍| 1510/1610 [6:30:36<25:34, 15.34s/it] 94%|█████████▍| 1511/1610 [6:30:51<24:45, 15.01s/it] {'loss': 0.0009, 'grad_norm': 0.9750683542781635, 'learning_rate': 6.149068322981366e-08, 'completion_length': 146.63394165039062, 'rewards/accuracy_reward': 0.7321428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7321429252624512, 'reward_std': 0.23327529430389404, 'kl': 0.02142333984375, 'epoch': 4.69} 94%|█████████▍| 1511/1610 [6:30:51<24:45, 15.01s/it] 94%|█████████▍| 1512/1610 [6:31:06<24:48, 15.19s/it] {'loss': 0.0007, 'grad_norm': 1.1333992213581832, 'learning_rate': 6.086956521739131e-08, 'completion_length': 167.50000762939453, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6071429252624512, 'reward_std': 0.2669335827231407, 'kl': 0.01690673828125, 'epoch': 4.7} 94%|█████████▍| 1512/1610 [6:31:06<24:48, 15.19s/it] 94%|█████████▍| 1513/1610 [6:31:21<24:32, 15.18s/it] {'loss': 0.0006, 'grad_norm': 1.3331749883721788, 'learning_rate': 6.024844720496894e-08, 'completion_length': 136.73214721679688, 'rewards/accuracy_reward': 0.5625000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5625000596046448, 'reward_std': 0.19447603821754456, 'kl': 0.014801025390625, 'epoch': 4.7} 94%|█████████▍| 1513/1610 [6:31:21<24:32, 15.18s/it] 94%|█████████▍| 1514/1610 [6:31:37<24:28, 15.30s/it] {'loss': 0.0009, 'grad_norm': 1.5164585892272648, 'learning_rate': 5.962732919254657e-08, 'completion_length': 163.5357208251953, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.24498926103115082, 'kl': 0.0225830078125, 'epoch': 4.7} 94%|█████████▍| 1514/1610 [6:31:37<24:28, 15.30s/it] 94%|█████████▍| 1515/1610 [6:31:53<24:29, 15.47s/it] {'loss': 0.0012, 'grad_norm': 4.200015649670413, 'learning_rate': 5.900621118012422e-08, 'completion_length': 178.74108123779297, 'rewards/accuracy_reward': 0.4375000149011612, 'rewards/format_reward': 1.0, 'reward': 1.4375000596046448, 'reward_std': 0.29757943749427795, 'kl': 0.030029296875, 'epoch': 4.7} 94%|█████████▍| 1515/1610 [6:31:53<24:29, 15.47s/it] 94%|█████████▍| 1516/1610 [6:32:07<23:35, 15.06s/it] {'loss': 0.0006, 'grad_norm': 1.8723620010450577, 'learning_rate': 5.8385093167701866e-08, 'completion_length': 132.1607208251953, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.28047342598438263, 'kl': 0.01458740234375, 'epoch': 4.71} 94%|█████████▍| 1516/1610 [6:32:07<23:35, 15.06s/it] 94%|█████████▍| 1517/1610 [6:32:23<23:58, 15.46s/it] {'loss': 0.0006, 'grad_norm': 1.169027299316568, 'learning_rate': 5.77639751552795e-08, 'completion_length': 170.85714721679688, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.535714328289032, 'reward_std': 0.20801587402820587, 'kl': 0.01605224609375, 'epoch': 4.71} 94%|█████████▍| 1517/1610 [6:32:23<23:58, 15.46s/it] 94%|█████████▍| 1518/1610 [6:32:38<23:15, 15.17s/it] {'loss': 0.0006, 'grad_norm': 0.9758176543825294, 'learning_rate': 5.714285714285714e-08, 'completion_length': 142.4196548461914, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.2570309042930603, 'kl': 0.016204833984375, 'epoch': 4.71} 94%|█████████▍| 1518/1610 [6:32:38<23:15, 15.17s/it] 94%|█████████▍| 1519/1610 [6:32:54<23:25, 15.45s/it] {'loss': 0.001, 'grad_norm': 1.2125094431341443, 'learning_rate': 5.6521739130434777e-08, 'completion_length': 165.1428680419922, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.36052724719047546, 'kl': 0.02545166015625, 'epoch': 4.72} 94%|█████████▍| 1519/1610 [6:32:54<23:25, 15.45s/it] 94%|█████████▍| 1520/1610 [6:33:10<23:26, 15.63s/it] {'loss': 0.0008, 'grad_norm': 1.2705255300395024, 'learning_rate': 5.590062111801242e-08, 'completion_length': 187.25894165039062, 'rewards/accuracy_reward': 0.4910714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.3084176629781723, 'kl': 0.020263671875, 'epoch': 4.72} 94%|█████████▍| 1520/1610 [6:33:10<23:26, 15.63s/it] 94%|█████████▍| 1521/1610 [6:33:24<22:37, 15.25s/it] {'loss': 0.0007, 'grad_norm': 1.2979624824953955, 'learning_rate': 5.527950310559006e-08, 'completion_length': 150.33036422729492, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.31561538577079773, 'kl': 0.01678466796875, 'epoch': 4.72} 94%|█████████▍| 1521/1610 [6:33:24<22:37, 15.25s/it] 95%|█████████▍| 1522/1610 [6:33:39<22:12, 15.15s/it] {'loss': 0.0007, 'grad_norm': 1.2640032487918695, 'learning_rate': 5.4658385093167694e-08, 'completion_length': 138.11607360839844, 'rewards/accuracy_reward': 0.5892857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.244989275932312, 'kl': 0.017608642578125, 'epoch': 4.73} 95%|█████████▍| 1522/1610 [6:33:39<22:12, 15.15s/it] 95%|█████████▍| 1523/1610 [6:33:55<22:03, 15.21s/it] {'loss': 0.0007, 'grad_norm': 0.9925337334822708, 'learning_rate': 5.403726708074534e-08, 'completion_length': 157.11608123779297, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5892857909202576, 'reward_std': 0.3051137775182724, 'kl': 0.017822265625, 'epoch': 4.73} 95%|█████████▍| 1523/1610 [6:33:55<22:03, 15.21s/it] 95%|█████████▍| 1524/1610 [6:34:08<21:10, 14.77s/it] {'loss': 0.0007, 'grad_norm': 0.9630328010830084, 'learning_rate': 5.341614906832298e-08, 'completion_length': 136.3214340209961, 'rewards/accuracy_reward': 0.4107142984867096, 'rewards/format_reward': 1.0, 'reward': 1.4107143878936768, 'reward_std': 0.12175280973315239, 'kl': 0.016510009765625, 'epoch': 4.73} 95%|█████████▍| 1524/1610 [6:34:08<21:10, 14.77s/it] 95%|█████████▍| 1525/1610 [6:34:24<21:27, 15.15s/it] {'loss': 0.0007, 'grad_norm': 1.6963538994693848, 'learning_rate': 5.279503105590062e-08, 'completion_length': 172.5446548461914, 'rewards/accuracy_reward': 0.3571428656578064, 'rewards/format_reward': 1.0, 'reward': 1.3571429252624512, 'reward_std': 0.2960902079939842, 'kl': 0.018524169921875, 'epoch': 4.74} 95%|█████████▍| 1525/1610 [6:34:24<21:27, 15.15s/it] 95%|█████████▍| 1526/1610 [6:34:39<21:10, 15.13s/it] {'loss': 0.0009, 'grad_norm': 4.850792720559224, 'learning_rate': 5.217391304347826e-08, 'completion_length': 142.3482208251953, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.3414882570505142, 'kl': 0.02154541015625, 'epoch': 4.74} 95%|█████████▍| 1526/1610 [6:34:40<21:10, 15.13s/it] 95%|█████████▍| 1527/1610 [6:34:54<20:40, 14.95s/it] {'loss': 0.0007, 'grad_norm': 1.2525156439639977, 'learning_rate': 5.1552795031055897e-08, 'completion_length': 146.75000762939453, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.2630308046936989, 'kl': 0.01776123046875, 'epoch': 4.74} 95%|█████████▍| 1527/1610 [6:34:54<20:40, 14.95s/it] 95%|█████████▍| 1528/1610 [6:35:08<20:05, 14.70s/it] {'loss': 0.0009, 'grad_norm': 1.593883025877494, 'learning_rate': 5.0931677018633536e-08, 'completion_length': 136.83929061889648, 'rewards/accuracy_reward': 0.4285714328289032, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.30659741163253784, 'kl': 0.02264404296875, 'epoch': 4.75} 95%|█████████▍| 1528/1610 [6:35:08<20:05, 14.70s/it] 95%|█████████▍| 1529/1610 [6:35:24<20:24, 15.12s/it] {'loss': 0.0009, 'grad_norm': 3.8677362528209907, 'learning_rate': 5.031055900621118e-08, 'completion_length': 165.8839340209961, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.27804867923259735, 'kl': 0.02337646484375, 'epoch': 4.75} 95%|█████████▍| 1529/1610 [6:35:24<20:24, 15.12s/it] 95%|█████████▌| 1530/1610 [6:35:38<19:47, 14.84s/it] {'loss': 0.0009, 'grad_norm': 1.467487162127565, 'learning_rate': 4.9689440993788814e-08, 'completion_length': 156.92857360839844, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.25912240892648697, 'kl': 0.02325439453125, 'epoch': 4.75} 95%|█████████▌| 1530/1610 [6:35:38<19:47, 14.84s/it] 95%|█████████▌| 1531/1610 [6:35:53<19:18, 14.66s/it] {'loss': 0.0009, 'grad_norm': 1.1641625632909873, 'learning_rate': 4.906832298136645e-08, 'completion_length': 150.31250762939453, 'rewards/accuracy_reward': 0.6428571939468384, 'rewards/format_reward': 1.0, 'reward': 1.6428572535514832, 'reward_std': 0.2702374756336212, 'kl': 0.021728515625, 'epoch': 4.75} 95%|█████████▌| 1531/1610 [6:35:53<19:18, 14.66s/it] 95%|█████████▌| 1532/1610 [6:36:09<19:33, 15.04s/it] {'loss': 0.0009, 'grad_norm': 1.158965703085203, 'learning_rate': 4.84472049689441e-08, 'completion_length': 173.48214721679688, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.29670368134975433, 'kl': 0.02264404296875, 'epoch': 4.76} 95%|█████████▌| 1532/1610 [6:36:09<19:33, 15.04s/it] 95%|█████████▌| 1533/1610 [6:36:23<18:55, 14.75s/it] {'loss': 0.0007, 'grad_norm': 0.8874422761651579, 'learning_rate': 4.782608695652174e-08, 'completion_length': 154.05358123779297, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.1379830539226532, 'kl': 0.018035888671875, 'epoch': 4.76} 95%|█████████▌| 1533/1610 [6:36:23<18:55, 14.75s/it] 95%|█████████▌| 1534/1610 [6:36:38<18:53, 14.91s/it] {'loss': 0.0008, 'grad_norm': 1.70331757745286, 'learning_rate': 4.720496894409938e-08, 'completion_length': 153.33036041259766, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 1.0, 'reward': 1.5178571939468384, 'reward_std': 0.3104946166276932, 'kl': 0.01873779296875, 'epoch': 4.76} 95%|█████████▌| 1534/1610 [6:36:38<18:53, 14.91s/it] 95%|█████████▌| 1535/1610 [6:36:51<17:51, 14.29s/it] {'loss': 0.0006, 'grad_norm': 2.015357319216344, 'learning_rate': 4.6583850931677016e-08, 'completion_length': 117.23214721679688, 'rewards/accuracy_reward': 0.6160714328289032, 'rewards/format_reward': 1.0, 'reward': 1.6160715222358704, 'reward_std': 0.221555694937706, 'kl': 0.014190673828125, 'epoch': 4.77} 95%|█████████▌| 1535/1610 [6:36:51<17:51, 14.29s/it] 95%|█████████▌| 1536/1610 [6:37:07<18:16, 14.82s/it] {'loss': 0.0011, 'grad_norm': 0.9065879591386171, 'learning_rate': 4.5962732919254656e-08, 'completion_length': 194.4107208251953, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.1956884190440178, 'kl': 0.0264892578125, 'epoch': 4.77} 95%|█████████▌| 1536/1610 [6:37:07<18:16, 14.82s/it] 95%|█████████▌| 1537/1610 [6:37:23<18:20, 15.08s/it] {'loss': 0.0009, 'grad_norm': 1.9071846494064744, 'learning_rate': 4.5341614906832295e-08, 'completion_length': 160.21429443359375, 'rewards/accuracy_reward': 0.4821428805589676, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.2669336050748825, 'kl': 0.02349853515625, 'epoch': 4.77} 95%|█████████▌| 1537/1610 [6:37:23<18:20, 15.08s/it] 96%|█████████▌| 1538/1610 [6:37:36<17:39, 14.72s/it] {'loss': 0.0008, 'grad_norm': 1.2709017257704078, 'learning_rate': 4.472049689440994e-08, 'completion_length': 120.66072463989258, 'rewards/accuracy_reward': 0.6517857313156128, 'rewards/format_reward': 1.0, 'reward': 1.6517857909202576, 'reward_std': 0.20411308109760284, 'kl': 0.01947021484375, 'epoch': 4.78} 96%|█████████▌| 1538/1610 [6:37:36<17:39, 14.72s/it] 96%|█████████▌| 1539/1610 [6:37:52<17:42, 14.96s/it] {'loss': 0.0009, 'grad_norm': 1.5881738841853137, 'learning_rate': 4.409937888198757e-08, 'completion_length': 149.12500762939453, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.5000000596046448, 'reward_std': 0.36733539402484894, 'kl': 0.0224609375, 'epoch': 4.78} 96%|█████████▌| 1539/1610 [6:37:52<17:42, 14.96s/it] 96%|█████████▌| 1540/1610 [6:38:08<17:53, 15.34s/it] {'loss': 0.0008, 'grad_norm': 1.1902287851743019, 'learning_rate': 4.347826086956521e-08, 'completion_length': 160.79464721679688, 'rewards/accuracy_reward': 0.6696428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.23717807233333588, 'kl': 0.019805908203125, 'epoch': 4.78} 96%|█████████▌| 1540/1610 [6:38:08<17:53, 15.34s/it] 96%|█████████▌| 1541/1610 [6:38:22<16:58, 14.75s/it] {'loss': 0.0008, 'grad_norm': 1.6440717710125348, 'learning_rate': 4.285714285714286e-08, 'completion_length': 140.1339340209961, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.31561537086963654, 'kl': 0.02020263671875, 'epoch': 4.79} 96%|█████████▌| 1541/1610 [6:38:22<16:58, 14.75s/it] 96%|█████████▌| 1542/1610 [6:38:35<16:20, 14.42s/it] {'loss': 0.0006, 'grad_norm': 1.0625939840355432, 'learning_rate': 4.22360248447205e-08, 'completion_length': 141.9107208251953, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 1.0, 'reward': 1.4910714626312256, 'reward_std': 0.2540072351694107, 'kl': 0.015899658203125, 'epoch': 4.79} 96%|█████████▌| 1542/1610 [6:38:35<16:20, 14.42s/it] 96%|█████████▌| 1543/1610 [6:38:50<16:21, 14.65s/it] {'loss': 0.0008, 'grad_norm': 1.193201776955076, 'learning_rate': 4.161490683229813e-08, 'completion_length': 152.52679443359375, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.2954968959093094, 'kl': 0.01904296875, 'epoch': 4.79} 96%|█████████▌| 1543/1610 [6:38:50<16:21, 14.65s/it] 96%|█████████▌| 1544/1610 [6:39:06<16:30, 15.01s/it] {'loss': 0.0011, 'grad_norm': 1.4673237952664249, 'learning_rate': 4.0993788819875776e-08, 'completion_length': 158.8214340209961, 'rewards/accuracy_reward': 0.5625000149011612, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5535715222358704, 'reward_std': 0.23386859148740768, 'kl': 0.02777099609375, 'epoch': 4.8} 96%|█████████▌| 1544/1610 [6:39:06<16:30, 15.01s/it] 96%|█████████▌| 1545/1610 [6:39:22<16:25, 15.17s/it] {'loss': 0.001, 'grad_norm': 1.3831738328324887, 'learning_rate': 4.0372670807453415e-08, 'completion_length': 172.86608123779297, 'rewards/accuracy_reward': 0.642857164144516, 'rewards/format_reward': 0.9642857313156128, 'reward': 1.6071429252624512, 'reward_std': 0.36781711876392365, 'kl': 0.025390625, 'epoch': 4.8} 96%|█████████▌| 1545/1610 [6:39:22<16:25, 15.17s/it] 96%|█████████▌| 1546/1610 [6:39:38<16:25, 15.40s/it] {'loss': 0.0008, 'grad_norm': 2.40347085036652, 'learning_rate': 3.9751552795031054e-08, 'completion_length': 182.70536041259766, 'rewards/accuracy_reward': 0.383928582072258, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.2624262869358063, 'kl': 0.020416259765625, 'epoch': 4.8} 96%|█████████▌| 1546/1610 [6:39:38<16:25, 15.40s/it] 96%|█████████▌| 1547/1610 [6:39:51<15:37, 14.89s/it] {'loss': 0.0008, 'grad_norm': 1.8531071434416382, 'learning_rate': 3.91304347826087e-08, 'completion_length': 130.45536041259766, 'rewards/accuracy_reward': 0.4285714477300644, 'rewards/format_reward': 1.0, 'reward': 1.4285715222358704, 'reward_std': 0.36101342737674713, 'kl': 0.020050048828125, 'epoch': 4.8} 96%|█████████▌| 1547/1610 [6:39:51<15:37, 14.89s/it] 96%|█████████▌| 1548/1610 [6:40:06<15:24, 14.91s/it] {'loss': 0.0011, 'grad_norm': 1.2438954579367754, 'learning_rate': 3.850931677018633e-08, 'completion_length': 166.02679443359375, 'rewards/accuracy_reward': 0.3839285969734192, 'rewards/format_reward': 1.0, 'reward': 1.383928656578064, 'reward_std': 0.26182402670383453, 'kl': 0.026611328125, 'epoch': 4.81} 96%|█████████▌| 1548/1610 [6:40:06<15:24, 14.91s/it] 96%|█████████▌| 1549/1610 [6:40:22<15:26, 15.19s/it] {'loss': 0.001, 'grad_norm': 1.2461256596298291, 'learning_rate': 3.788819875776397e-08, 'completion_length': 164.9732208251953, 'rewards/accuracy_reward': 0.535714328289032, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5267857909202576, 'reward_std': 0.3462556302547455, 'kl': 0.0255126953125, 'epoch': 4.81} 96%|█████████▌| 1549/1610 [6:40:22<15:26, 15.19s/it] 96%|█████████▋| 1550/1610 [6:40:40<15:50, 15.84s/it] {'loss': 0.001, 'grad_norm': 1.6905146104572315, 'learning_rate': 3.726708074534162e-08, 'completion_length': 205.0982208251953, 'rewards/accuracy_reward': 0.3750000298023224, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.3660715222358704, 'reward_std': 0.3012199103832245, 'kl': 0.02569580078125, 'epoch': 4.81} 96%|█████████▋| 1550/1610 [6:40:40<15:50, 15.84s/it] 96%|█████████▋| 1551/1610 [6:40:55<15:22, 15.64s/it] {'loss': 0.0009, 'grad_norm': 0.9611536728834877, 'learning_rate': 3.6645962732919256e-08, 'completion_length': 169.6339340209961, 'rewards/accuracy_reward': 0.4464285969734192, 'rewards/format_reward': 1.0, 'reward': 1.446428656578064, 'reward_std': 0.244989275932312, 'kl': 0.02239990234375, 'epoch': 4.82} 96%|█████████▋| 1551/1610 [6:40:55<15:22, 15.64s/it] 96%|█████████▋| 1552/1610 [6:41:12<15:36, 16.14s/it] {'loss': 0.0008, 'grad_norm': 5.511936070707088, 'learning_rate': 3.602484472049689e-08, 'completion_length': 184.27679443359375, 'rewards/accuracy_reward': 0.3482143133878708, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.3303572535514832, 'reward_std': 0.3045148700475693, 'kl': 0.02093505859375, 'epoch': 4.82} 96%|█████████▋| 1552/1610 [6:41:12<15:36, 16.14s/it] 96%|█████████▋| 1553/1610 [6:41:26<14:44, 15.52s/it] {'loss': 0.0007, 'grad_norm': 1.081501422531074, 'learning_rate': 3.5403726708074535e-08, 'completion_length': 142.05357360839844, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.20021028816699982, 'kl': 0.01727294921875, 'epoch': 4.82} 96%|█████████▋| 1553/1610 [6:41:26<14:44, 15.52s/it] 97%|█████████▋| 1554/1610 [6:41:41<14:26, 15.47s/it] {'loss': 0.0008, 'grad_norm': 1.0228957725102774, 'learning_rate': 3.4782608695652174e-08, 'completion_length': 161.02679443359375, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.28707224130630493, 'kl': 0.02093505859375, 'epoch': 4.83} 97%|█████████▋| 1554/1610 [6:41:41<14:26, 15.47s/it] 97%|█████████▋| 1555/1610 [6:41:56<13:49, 15.08s/it] {'loss': 0.0005, 'grad_norm': 1.5819862627688968, 'learning_rate': 3.416149068322981e-08, 'completion_length': 136.08036041259766, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.20080358907580376, 'kl': 0.01275634765625, 'epoch': 4.83} 97%|█████████▋| 1555/1610 [6:41:56<13:49, 15.08s/it] 97%|█████████▋| 1556/1610 [6:42:11<13:30, 15.02s/it] {'loss': 0.0009, 'grad_norm': 2.7852175697072084, 'learning_rate': 3.3540372670807445e-08, 'completion_length': 173.65179443359375, 'rewards/accuracy_reward': 0.5803571939468384, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.2630307972431183, 'kl': 0.0224609375, 'epoch': 4.83} 97%|█████████▋| 1556/1610 [6:42:11<13:30, 15.02s/it] 97%|█████████▋| 1557/1610 [6:42:27<13:43, 15.54s/it] {'loss': 0.0009, 'grad_norm': 1.0593887720505633, 'learning_rate': 3.291925465838509e-08, 'completion_length': 184.2321548461914, 'rewards/accuracy_reward': 0.660714328289032, 'rewards/format_reward': 0.9821428656578064, 'reward': 1.6428571939468384, 'reward_std': 0.18270856887102127, 'kl': 0.022857666015625, 'epoch': 4.84} 97%|█████████▋| 1557/1610 [6:42:27<13:43, 15.54s/it] 97%|█████████▋| 1558/1610 [6:42:42<13:18, 15.36s/it] {'loss': 0.0007, 'grad_norm': 1.2615635814682156, 'learning_rate': 3.229813664596273e-08, 'completion_length': 137.86607360839844, 'rewards/accuracy_reward': 0.6964286267757416, 'rewards/format_reward': 1.0, 'reward': 1.696428656578064, 'reward_std': 0.23535223305225372, 'kl': 0.0169677734375, 'epoch': 4.84} 97%|█████████▋| 1558/1610 [6:42:42<13:18, 15.36s/it] 97%|█████████▋| 1559/1610 [6:42:57<12:50, 15.10s/it] {'loss': 0.0008, 'grad_norm': 1.743686492890986, 'learning_rate': 3.167701863354037e-08, 'completion_length': 173.45536041259766, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.3078097999095917, 'kl': 0.019561767578125, 'epoch': 4.84} 97%|█████████▋| 1559/1610 [6:42:57<12:50, 15.10s/it] 97%|█████████▋| 1560/1610 [6:43:11<12:22, 14.85s/it] {'loss': 0.0006, 'grad_norm': 1.190811562543057, 'learning_rate': 3.105590062111801e-08, 'completion_length': 136.0982208251953, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.2921874076128006, 'kl': 0.01434326171875, 'epoch': 4.84} 97%|█████████▋| 1560/1610 [6:43:11<12:22, 14.85s/it] 97%|█████████▋| 1561/1610 [6:43:27<12:30, 15.31s/it] {'loss': 0.0008, 'grad_norm': 2.2808597855092145, 'learning_rate': 3.0434782608695655e-08, 'completion_length': 142.5089340209961, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.3189248740673065, 'kl': 0.02056884765625, 'epoch': 4.85} 97%|█████████▋| 1561/1610 [6:43:27<12:30, 15.31s/it] 97%|█████████▋| 1562/1610 [6:43:42<12:04, 15.09s/it] {'loss': 0.0008, 'grad_norm': 1.78337178845142, 'learning_rate': 2.981366459627329e-08, 'completion_length': 138.0803680419922, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.4088250547647476, 'kl': 0.0203857421875, 'epoch': 4.85} 97%|█████████▋| 1562/1610 [6:43:42<12:04, 15.09s/it] 97%|█████████▋| 1563/1610 [6:43:57<11:46, 15.03s/it] {'loss': 0.0007, 'grad_norm': 1.837068337168433, 'learning_rate': 2.9192546583850933e-08, 'completion_length': 144.7589340209961, 'rewards/accuracy_reward': 0.6696428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6696429252624512, 'reward_std': 0.2928008586168289, 'kl': 0.01702880859375, 'epoch': 4.85} 97%|█████████▋| 1563/1610 [6:43:57<11:46, 15.03s/it] 97%|█████████▋| 1564/1610 [6:44:12<11:30, 15.00s/it] {'loss': 0.0011, 'grad_norm': 1.5793849998345246, 'learning_rate': 2.857142857142857e-08, 'completion_length': 152.94644165039062, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.38357122242450714, 'kl': 0.02752685546875, 'epoch': 4.86} 97%|█████████▋| 1564/1610 [6:44:12<11:30, 15.00s/it] 97%|█████████▋| 1565/1610 [6:44:28<11:28, 15.30s/it] {'loss': 0.0006, 'grad_norm': 0.9009712188903309, 'learning_rate': 2.795031055900621e-08, 'completion_length': 139.50000762939453, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.1956884115934372, 'kl': 0.01568603515625, 'epoch': 4.86} 97%|█████████▋| 1565/1610 [6:44:28<11:28, 15.30s/it] 97%|█████████▋| 1566/1610 [6:44:43<11:16, 15.38s/it] {'loss': 0.0008, 'grad_norm': 3.0000072898920376, 'learning_rate': 2.7329192546583847e-08, 'completion_length': 178.1696548461914, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803571939468384, 'reward_std': 0.35831740498542786, 'kl': 0.020233154296875, 'epoch': 4.86} 97%|█████████▋| 1566/1610 [6:44:43<11:16, 15.38s/it] 97%|█████████▋| 1567/1610 [6:44:59<11:07, 15.51s/it] {'loss': 0.0008, 'grad_norm': 1.2650041203348774, 'learning_rate': 2.670807453416149e-08, 'completion_length': 162.87500762939453, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.19959120452404022, 'kl': 0.02008056640625, 'epoch': 4.87} 97%|█████████▋| 1567/1610 [6:44:59<11:07, 15.51s/it] 97%|█████████▋| 1568/1610 [6:45:14<10:39, 15.22s/it] {'loss': 0.0008, 'grad_norm': 0.9032943764619329, 'learning_rate': 2.608695652173913e-08, 'completion_length': 158.39286041259766, 'rewards/accuracy_reward': 0.5982143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.1379830539226532, 'kl': 0.01953125, 'epoch': 4.87} 97%|█████████▋| 1568/1610 [6:45:14<10:39, 15.22s/it] 97%|█████████▋| 1569/1610 [6:45:28<10:18, 15.08s/it] {'loss': 0.0008, 'grad_norm': 0.9952811339918909, 'learning_rate': 2.5465838509316768e-08, 'completion_length': 152.18750762939453, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.21192426979541779, 'kl': 0.0201416015625, 'epoch': 4.87} 97%|█████████▋| 1569/1610 [6:45:28<10:18, 15.08s/it] 98%|█████████▊| 1570/1610 [6:45:42<09:48, 14.72s/it] {'loss': 0.0006, 'grad_norm': 1.141132284150871, 'learning_rate': 2.4844720496894407e-08, 'completion_length': 133.68750381469727, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.21703942120075226, 'kl': 0.015045166015625, 'epoch': 4.88} 98%|█████████▊| 1570/1610 [6:45:42<09:48, 14.72s/it] 98%|█████████▊| 1571/1610 [6:45:57<09:34, 14.72s/it] {'loss': 0.0006, 'grad_norm': 2.025490838268765, 'learning_rate': 2.422360248447205e-08, 'completion_length': 135.03572463989258, 'rewards/accuracy_reward': 0.4732142984867096, 'rewards/format_reward': 1.0, 'reward': 1.473214328289032, 'reward_std': 0.17104806751012802, 'kl': 0.014923095703125, 'epoch': 4.88} 98%|█████████▊| 1571/1610 [6:45:57<09:34, 14.72s/it] 98%|█████████▊| 1572/1610 [6:46:14<09:47, 15.46s/it] {'loss': 0.0009, 'grad_norm': 2.0875263078602875, 'learning_rate': 2.360248447204969e-08, 'completion_length': 156.5625, 'rewards/accuracy_reward': 0.4910714626312256, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.4821429252624512, 'reward_std': 0.2776331752538681, 'kl': 0.02313232421875, 'epoch': 4.88} 98%|█████████▊| 1572/1610 [6:46:14<09:47, 15.46s/it] 98%|█████████▊| 1573/1610 [6:46:29<09:24, 15.25s/it] {'loss': 0.0008, 'grad_norm': 1.1058587247150804, 'learning_rate': 2.2981366459627328e-08, 'completion_length': 130.86608123779297, 'rewards/accuracy_reward': 0.580357164144516, 'rewards/format_reward': 1.0, 'reward': 1.5803572535514832, 'reward_std': 0.1866704523563385, 'kl': 0.01934814453125, 'epoch': 4.89} 98%|█████████▊| 1573/1610 [6:46:29<09:24, 15.25s/it] 98%|█████████▊| 1574/1610 [6:46:43<09:00, 15.02s/it] {'loss': 0.0008, 'grad_norm': 2.1028726681066283, 'learning_rate': 2.236024844720497e-08, 'completion_length': 164.4553680419922, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3661285936832428, 'kl': 0.020263671875, 'epoch': 4.89} 98%|█████████▊| 1574/1610 [6:46:43<09:00, 15.02s/it] 98%|█████████▊| 1575/1610 [6:46:59<08:52, 15.22s/it] {'loss': 0.0008, 'grad_norm': 2.2275752102324002, 'learning_rate': 2.1739130434782606e-08, 'completion_length': 147.9732208251953, 'rewards/accuracy_reward': 0.598214328289032, 'rewards/format_reward': 1.0, 'reward': 1.598214328289032, 'reward_std': 0.27743521332740784, 'kl': 0.02117919921875, 'epoch': 4.89} 98%|█████████▊| 1575/1610 [6:46:59<08:52, 15.22s/it] 98%|█████████▊| 1576/1610 [6:47:14<08:32, 15.07s/it] {'loss': 0.001, 'grad_norm': 1.8957149948798548, 'learning_rate': 2.111801242236025e-08, 'completion_length': 146.57144165039062, 'rewards/accuracy_reward': 0.5000000298023224, 'rewards/format_reward': 1.0, 'reward': 1.5000000596046448, 'reward_std': 0.2831694483757019, 'kl': 0.0250244140625, 'epoch': 4.89} 98%|█████████▊| 1576/1610 [6:47:14<08:32, 15.07s/it] 98%|█████████▊| 1577/1610 [6:47:30<08:29, 15.45s/it] {'loss': 0.0006, 'grad_norm': 1.738959612939753, 'learning_rate': 2.0496894409937888e-08, 'completion_length': 174.21429443359375, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5625000596046448, 'reward_std': 0.3219574987888336, 'kl': 0.015655517578125, 'epoch': 4.9} 98%|█████████▊| 1577/1610 [6:47:30<08:29, 15.45s/it] 98%|█████████▊| 1578/1610 [6:47:46<08:13, 15.41s/it] {'loss': 0.0007, 'grad_norm': 0.9382239568117289, 'learning_rate': 1.9875776397515527e-08, 'completion_length': 164.51786041259766, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535715222358704, 'reward_std': 0.24558257311582565, 'kl': 0.016998291015625, 'epoch': 4.9} 98%|█████████▊| 1578/1610 [6:47:46<08:13, 15.41s/it] 98%|█████████▊| 1579/1610 [6:47:59<07:38, 14.81s/it] {'loss': 0.0006, 'grad_norm': 0.9339284611400813, 'learning_rate': 1.9254658385093166e-08, 'completion_length': 128.51786422729492, 'rewards/accuracy_reward': 0.723214328289032, 'rewards/format_reward': 1.0, 'reward': 1.7232143878936768, 'reward_std': 0.1827620565891266, 'kl': 0.015045166015625, 'epoch': 4.9} 98%|█████████▊| 1579/1610 [6:47:59<07:38, 14.81s/it] 98%|█████████▊| 1580/1610 [6:48:12<07:11, 14.39s/it] {'loss': 0.0005, 'grad_norm': 0.9764213658656415, 'learning_rate': 1.863354037267081e-08, 'completion_length': 143.23215103149414, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785715222358704, 'reward_std': 0.11272924765944481, 'kl': 0.01275634765625, 'epoch': 4.91} 98%|█████████▊| 1580/1610 [6:48:12<07:11, 14.39s/it] 98%|█████████▊| 1581/1610 [6:48:27<06:58, 14.45s/it] {'loss': 0.0008, 'grad_norm': 1.3334381131359196, 'learning_rate': 1.8012422360248444e-08, 'completion_length': 158.7232208251953, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.2474084123969078, 'kl': 0.01959228515625, 'epoch': 4.91} 98%|█████████▊| 1581/1610 [6:48:27<06:58, 14.45s/it] 98%|█████████▊| 1582/1610 [6:48:43<06:55, 14.84s/it] {'loss': 0.0006, 'grad_norm': 4.537219181308332, 'learning_rate': 1.7391304347826087e-08, 'completion_length': 187.81250762939453, 'rewards/accuracy_reward': 0.5267857611179352, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.3045148551464081, 'kl': 0.015869140625, 'epoch': 4.91} 98%|█████████▊| 1582/1610 [6:48:43<06:55, 14.84s/it] 98%|█████████▊| 1583/1610 [6:48:59<06:53, 15.30s/it] {'loss': 0.001, 'grad_norm': 1.8486001866408683, 'learning_rate': 1.6770186335403723e-08, 'completion_length': 154.3303680419922, 'rewards/accuracy_reward': 0.5446428656578064, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.5357143878936768, 'reward_std': 0.2135206162929535, 'kl': 0.02484130859375, 'epoch': 4.92} 98%|█████████▊| 1583/1610 [6:48:59<06:53, 15.30s/it] 98%|█████████▊| 1584/1610 [6:49:14<06:33, 15.14s/it] {'loss': 0.0008, 'grad_norm': 1.7414275040946658, 'learning_rate': 1.6149068322981365e-08, 'completion_length': 141.3214340209961, 'rewards/accuracy_reward': 0.6607142984867096, 'rewards/format_reward': 1.0, 'reward': 1.6607143878936768, 'reward_std': 0.17885926365852356, 'kl': 0.02044677734375, 'epoch': 4.92} 98%|█████████▊| 1584/1610 [6:49:14<06:33, 15.14s/it] 98%|█████████▊| 1585/1610 [6:49:28<06:11, 14.85s/it] {'loss': 0.0007, 'grad_norm': 3.104878197026203, 'learning_rate': 1.5527950310559004e-08, 'completion_length': 140.68750381469727, 'rewards/accuracy_reward': 0.6785714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6785714626312256, 'reward_std': 0.21703943610191345, 'kl': 0.018218994140625, 'epoch': 4.92} 98%|█████████▊| 1585/1610 [6:49:28<06:11, 14.85s/it] 99%|█████████▊| 1586/1610 [6:49:42<05:50, 14.61s/it] {'loss': 0.0007, 'grad_norm': 2.5653376412559004, 'learning_rate': 1.4906832298136644e-08, 'completion_length': 145.6964340209961, 'rewards/accuracy_reward': 0.5357142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.21192426979541779, 'kl': 0.01690673828125, 'epoch': 4.93} 99%|█████████▊| 1586/1610 [6:49:42<05:50, 14.61s/it] 99%|█████████▊| 1587/1610 [6:49:58<05:47, 15.13s/it] {'loss': 0.0008, 'grad_norm': 0.7141934885095615, 'learning_rate': 1.4285714285714284e-08, 'completion_length': 171.4107208251953, 'rewards/accuracy_reward': 0.6071428656578064, 'rewards/format_reward': 1.0, 'reward': 1.607142984867096, 'reward_std': 0.23386859148740768, 'kl': 0.0211181640625, 'epoch': 4.93} 99%|█████████▊| 1587/1610 [6:49:58<05:47, 15.13s/it] 99%|█████████▊| 1588/1610 [6:50:12<05:24, 14.76s/it] {'loss': 0.0006, 'grad_norm': 1.4161839688863618, 'learning_rate': 1.3664596273291924e-08, 'completion_length': 136.96429061889648, 'rewards/accuracy_reward': 0.5089285969734192, 'rewards/format_reward': 1.0, 'reward': 1.508928656578064, 'reward_std': 0.39649760723114014, 'kl': 0.015960693359375, 'epoch': 4.93} 99%|█████████▊| 1588/1610 [6:50:12<05:24, 14.76s/it] 99%|█████████▊| 1589/1610 [6:50:26<05:06, 14.59s/it] {'loss': 0.0007, 'grad_norm': 0.5151014577925807, 'learning_rate': 1.3043478260869564e-08, 'completion_length': 135.06250762939453, 'rewards/accuracy_reward': 0.7946428954601288, 'rewards/format_reward': 1.0, 'reward': 1.7946429252624512, 'reward_std': 0.07514797151088715, 'kl': 0.01715087890625, 'epoch': 4.93} 99%|█████████▊| 1589/1610 [6:50:26<05:06, 14.59s/it] 99%|█████████▉| 1590/1610 [6:50:43<05:03, 15.18s/it] {'loss': 0.001, 'grad_norm': 3.1531273294834294, 'learning_rate': 1.2422360248447204e-08, 'completion_length': 173.31250762939453, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.14579425007104874, 'kl': 0.0240478515625, 'epoch': 4.94} 99%|█████████▉| 1590/1610 [6:50:43<05:03, 15.18s/it] 99%|█████████▉| 1591/1610 [6:50:57<04:39, 14.73s/it] {'loss': 0.0008, 'grad_norm': 1.6340719363005334, 'learning_rate': 1.1801242236024844e-08, 'completion_length': 124.32143020629883, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.33306361734867096, 'kl': 0.01898193359375, 'epoch': 4.94} 99%|█████████▉| 1591/1610 [6:50:57<04:39, 14.73s/it] 99%|█████████▉| 1592/1610 [6:51:11<04:23, 14.66s/it] {'loss': 0.0006, 'grad_norm': 1.5197999367071588, 'learning_rate': 1.1180124223602485e-08, 'completion_length': 134.87500762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.3174412101507187, 'kl': 0.015411376953125, 'epoch': 4.94} 99%|█████████▉| 1592/1610 [6:51:11<04:23, 14.66s/it] 99%|█████████▉| 1593/1610 [6:51:27<04:13, 14.89s/it] {'loss': 0.0008, 'grad_norm': 2.9008090151615082, 'learning_rate': 1.0559006211180124e-08, 'completion_length': 157.11608123779297, 'rewards/accuracy_reward': 0.526785746216774, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.20411306619644165, 'kl': 0.01934814453125, 'epoch': 4.95} 99%|█████████▉| 1593/1610 [6:51:27<04:13, 14.89s/it] 99%|█████████▉| 1594/1610 [6:51:41<03:54, 14.67s/it] {'loss': 0.001, 'grad_norm': 1.1228832642725368, 'learning_rate': 9.937888198757763e-09, 'completion_length': 156.43750762939453, 'rewards/accuracy_reward': 0.5357143133878708, 'rewards/format_reward': 1.0, 'reward': 1.5357143878936768, 'reward_std': 0.29097503423690796, 'kl': 0.02545166015625, 'epoch': 4.95} 99%|█████████▉| 1594/1610 [6:51:41<03:54, 14.67s/it] 99%|█████████▉| 1595/1610 [6:51:55<03:37, 14.52s/it] {'loss': 0.0007, 'grad_norm': 1.1882203210512965, 'learning_rate': 9.316770186335404e-09, 'completion_length': 131.70536041259766, 'rewards/accuracy_reward': 0.6250000298023224, 'rewards/format_reward': 1.0, 'reward': 1.6250000596046448, 'reward_std': 0.3219631016254425, 'kl': 0.01690673828125, 'epoch': 4.95} 99%|█████████▉| 1595/1610 [6:51:55<03:37, 14.52s/it] 99%|█████████▉| 1596/1610 [6:52:11<03:29, 14.94s/it] {'loss': 0.0007, 'grad_norm': 1.3912300454952669, 'learning_rate': 8.695652173913043e-09, 'completion_length': 151.1339340209961, 'rewards/accuracy_reward': 0.6071428954601288, 'rewards/format_reward': 1.0, 'reward': 1.6071429252624512, 'reward_std': 0.267547070980072, 'kl': 0.01800537109375, 'epoch': 4.96} 99%|█████████▉| 1596/1610 [6:52:11<03:29, 14.94s/it] 99%|█████████▉| 1597/1610 [6:52:26<03:16, 15.11s/it] {'loss': 0.0008, 'grad_norm': 1.0914624458357343, 'learning_rate': 8.074534161490683e-09, 'completion_length': 146.2857208251953, 'rewards/accuracy_reward': 0.5892857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5892857909202576, 'reward_std': 0.2702374905347824, 'kl': 0.018951416015625, 'epoch': 4.96} 99%|█████████▉| 1597/1610 [6:52:26<03:16, 15.11s/it] 99%|█████████▉| 1598/1610 [6:52:41<02:58, 14.86s/it] {'loss': 0.0008, 'grad_norm': 2.524848050087344, 'learning_rate': 7.453416149068322e-09, 'completion_length': 122.87500381469727, 'rewards/accuracy_reward': 0.5982142984867096, 'rewards/format_reward': 1.0, 'reward': 1.5982143878936768, 'reward_std': 0.2377713918685913, 'kl': 0.019256591796875, 'epoch': 4.96} 99%|█████████▉| 1598/1610 [6:52:41<02:58, 14.86s/it] 99%|█████████▉| 1599/1610 [6:52:56<02:44, 14.95s/it] {'loss': 0.001, 'grad_norm': 1.5330167540641406, 'learning_rate': 6.832298136645962e-09, 'completion_length': 175.60714721679688, 'rewards/accuracy_reward': 0.5267857313156128, 'rewards/format_reward': 1.0, 'reward': 1.5267857909202576, 'reward_std': 0.33185121417045593, 'kl': 0.02386474609375, 'epoch': 4.97} 99%|█████████▉| 1599/1610 [6:52:56<02:44, 14.95s/it] 99%|█████████▉| 1600/1610 [6:53:11<02:30, 15.03s/it] {'loss': 0.0009, 'grad_norm': 1.2284371198950597, 'learning_rate': 6.211180124223602e-09, 'completion_length': 146.32144165039062, 'rewards/accuracy_reward': 0.6160714626312256, 'rewards/format_reward': 1.0, 'reward': 1.6160714626312256, 'reward_std': 0.28707224130630493, 'kl': 0.02142333984375, 'epoch': 4.97} 99%|█████████▉| 1600/1610 [6:53:11<02:30, 15.03s/it] 99%|█████████▉| 1601/1610 [6:54:26<04:56, 32.96s/it] {'loss': 0.001, 'grad_norm': 1.3900456068110234, 'learning_rate': 5.5900621118012426e-09, 'completion_length': 166.7232208251953, 'rewards/accuracy_reward': 0.4642857313156128, 'rewards/format_reward': 1.0, 'reward': 1.4642857909202576, 'reward_std': 0.37454767525196075, 'kl': 0.02435302734375, 'epoch': 4.97} 99%|█████████▉| 1601/1610 [6:54:26<04:56, 32.96s/it] 100%|█████████▉| 1602/1610 [6:54:37<03:32, 26.55s/it] {'loss': 0.0007, 'grad_norm': 1.0558287531613897, 'learning_rate': 4.968944099378882e-09, 'completion_length': 143.7589340209961, 'rewards/accuracy_reward': 0.705357164144516, 'rewards/format_reward': 1.0, 'reward': 1.7053571939468384, 'reward_std': 0.1866704523563385, 'kl': 0.0166015625, 'epoch': 4.98} 100%|█████████▉| 1602/1610 [6:54:37<03:32, 26.55s/it] 100%|█████████▉| 1603/1610 [6:54:51<02:39, 22.75s/it] {'loss': 0.0009, 'grad_norm': 1.2489477933637716, 'learning_rate': 4.347826086956522e-09, 'completion_length': 159.5, 'rewards/accuracy_reward': 0.517857164144516, 'rewards/format_reward': 0.9910714626312256, 'reward': 1.508928656578064, 'reward_std': 0.2954913079738617, 'kl': 0.022705078125, 'epoch': 4.98} 100%|█████████▉| 1603/1610 [6:54:51<02:39, 22.75s/it] 100%|█████████▉| 1604/1610 [6:55:05<01:59, 19.96s/it] {'loss': 0.0008, 'grad_norm': 0.8643220421689738, 'learning_rate': 3.726708074534161e-09, 'completion_length': 175.39286041259766, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.571428656578064, 'reward_std': 0.26181842386722565, 'kl': 0.02105712890625, 'epoch': 4.98} 100%|█████████▉| 1604/1610 [6:55:05<01:59, 19.96s/it] 100%|█████████▉| 1605/1610 [6:55:18<01:29, 17.87s/it] {'loss': 0.0007, 'grad_norm': 1.3747440454069138, 'learning_rate': 3.105590062111801e-09, 'completion_length': 187.68750762939453, 'rewards/accuracy_reward': 0.5178571492433548, 'rewards/format_reward': 1.0, 'reward': 1.5178572535514832, 'reward_std': 0.3201316148042679, 'kl': 0.018707275390625, 'epoch': 4.98} 100%|█████████▉| 1605/1610 [6:55:18<01:29, 17.87s/it] 100%|█████████▉| 1606/1610 [6:55:31<01:05, 16.42s/it] {'loss': 0.001, 'grad_norm': 2.242181804226964, 'learning_rate': 2.484472049689441e-09, 'completion_length': 158.61608123779297, 'rewards/accuracy_reward': 0.5446428954601288, 'rewards/format_reward': 1.0, 'reward': 1.5446429252624512, 'reward_std': 0.19569401443004608, 'kl': 0.0240478515625, 'epoch': 4.99} 100%|█████████▉| 1606/1610 [6:55:31<01:05, 16.42s/it] 100%|█████████▉| 1607/1610 [6:55:42<00:44, 14.96s/it] {'loss': 0.0008, 'grad_norm': 2.360052035654233, 'learning_rate': 1.8633540372670804e-09, 'completion_length': 139.33929443359375, 'rewards/accuracy_reward': 0.4821428954601288, 'rewards/format_reward': 1.0, 'reward': 1.4821429252624512, 'reward_std': 0.30061207711696625, 'kl': 0.0191650390625, 'epoch': 4.99} 100%|█████████▉| 1607/1610 [6:55:42<00:44, 14.96s/it] 100%|█████████▉| 1608/1610 [6:55:53<00:27, 13.81s/it] {'loss': 0.0007, 'grad_norm': 1.5686092323266818, 'learning_rate': 1.2422360248447204e-09, 'completion_length': 140.38393783569336, 'rewards/accuracy_reward': 0.455357164144516, 'rewards/format_reward': 1.0, 'reward': 1.4553571939468384, 'reward_std': 0.2540072500705719, 'kl': 0.0167236328125, 'epoch': 4.99} 100%|█████████▉| 1608/1610 [6:55:53<00:27, 13.81s/it] 100%|█████████▉| 1609/1610 [6:56:05<00:13, 13.05s/it] {'loss': 0.0006, 'grad_norm': 1.8487353478540784, 'learning_rate': 6.211180124223602e-10, 'completion_length': 147.3125, 'rewards/accuracy_reward': 0.5714285969734192, 'rewards/format_reward': 1.0, 'reward': 1.5714285969734192, 'reward_std': 0.3279484361410141, 'kl': 0.015289306640625, 'epoch': 5.0} 100%|█████████▉| 1609/1610 [6:56:05<00:13, 13.05s/it] 100%|██████████| 1610/1610 [6:56:17<00:00, 12.92s/it] {'loss': 0.0008, 'grad_norm': 1.1730439712684815, 'learning_rate': 0.0, 'completion_length': 149.50000762939453, 'rewards/accuracy_reward': 0.5535714626312256, 'rewards/format_reward': 1.0, 'reward': 1.5535714626312256, 'reward_std': 0.2735324203968048, 'kl': 0.019134521484375, 'epoch': 5.0} 100%|██████████| 1610/1610 [6:56:17<00:00, 12.92s/it] {'train_runtime': 25052.5504, 'train_samples_per_second': 0.9, 'train_steps_per_second': 0.064, 'train_loss': 0.0005337236383773179, 'epoch': 5.0} 100%|██████████| 1610/1610 [6:57:29<00:00, 12.92s/it] 100%|██████████| 1610/1610 [6:57:29<00:00, 15.56s/it] wandb: wandb: 🚀 View run R1-Resume-COT-VLLM-Correct-Qwen2-VL-2B-GRPO-GEOQA-4k5-2025-02-23-13-19-32 at: https://wandb.ai/tanhuajie264-peking-university/vison-open-r1/runs/hqzrcjxh wandb: Find logs at: wandb/run-20250223_132155-hqzrcjxh/logs